Skip to content

Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.

fix(assistant): handle dormant macOS selection capture safely (#1952)

Gabriel Steinea122650006f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 4bcf61fd9..8144e8c05 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Fixed
+
+- **Assistant selection capture handles dormant macOS accessibility trees.** When accessibility exposes only a browser or Electron window, selection capture now tries a synthetic copy instead of assuming nothing is selected. The caret probe also waits long enough for its native retries to finish. Automatic answer delivery still requires a verified writable field at capture and delivery time: an unknown target keeps the answer in the panel and, with Auto-Paste enabled, copies it for manual paste. Dormant inputs still require manual paste because an empty copy cannot distinguish them from an integrated terminal, a page without an input, or some live selections. (#1952)
+
## [1.10.2] - 2026-09-15
A hotfix for 1.10.1. Generate AI Summary with your own API key or an enterprise provider no longer times out on a real transcript, a recording with nothing substantive in it gets a one-line summary instead of a blank page and an error, and OpenRouter models can read screenshots again.
diff --git a/CLAUDE.md b/CLAUDE.md
index fee16f8a9..3701623df 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -673,7 +673,7 @@ Detects meetings via three independent sources, orchestrated by `MeetingDetectio
### 17. Voice Assistant Hotkey
-A dedicated global hotkey that starts a dictation whose transcript is sent straight to the voice assistant as a command — no wake word ("Hey [AgentName]") needed — and that always bypasses the cleanup model. Standalone commands never type at an unverified cursor: with auto-paste enabled and a writable caret verified at capture time (an opaque `caret` session in `selectionManager.js`, revalidated before pasting; terminals and fields with a live selection excluded — the native probes check the focused element's own selection state, closing the clipboard-capture blind spots), the completed answer pastes at that caret and the pill returns to idle. Otherwise the answer streams into a floating assistant panel attached to the dictation pill (there is no separate assistant window) and, with auto-paste enabled, is also copied to the clipboard for manual paste (the Copy button confirms for six seconds). The pill window is content-protected while the panel is open (the panel never appears in screen shares).
+A dedicated global hotkey that starts a dictation whose transcript is sent straight to the voice assistant as a command — no wake word ("Hey [AgentName]") needed — and that always bypasses the cleanup model. With auto-paste enabled, a caret verified as writable at capture time (an opaque `caret` session in `selectionManager.js`, revalidated before pasting) receives the completed answer and the pill returns to idle. A dormant macOS accessibility tree reports "unknown": synthetic copy can still recover selected text, but cannot establish a safe caret destination. Unknown targets therefore stay on the panel route, since they can also be integrated terminals, pages without inputs, or fields with selections hidden by clipboard capture. A verified caret whose accessibility becomes unknown before delivery falls back the same way. Terminal and secure-field checks still apply to verified targets. Otherwise the answer streams into a floating assistant panel attached to the dictation pill (there is no separate assistant window) and, with auto-paste enabled, is also copied to the clipboard for manual paste (the Copy button confirms for six seconds). The pill window is content-protected while the panel is open (the panel never appears in screen shares).
**Flow**:
diff --git a/resources/macos-text-monitor.swift b/resources/macos-text-monitor.swift
index 1048acdf3..bec23d687 100644
--- a/resources/macos-text-monitor.swift
+++ b/resources/macos-text-monitor.swift
@@ -1,3 +1,4 @@
+import Carbon
import Cocoa
import Foundation
import Darwin
@@ -31,6 +32,16 @@ func writeTextOutput(_ prefix: String, _ value: String) {
writeOutput("\(prefix):\(truncated)")
}
+func elementRole(_ element: AXUIElement) -> String? {
+ var roleValue: AnyObject?
+ guard AXUIElementCopyAttributeValue(
+ element,
+ kAXRoleAttribute as CFString,
+ &roleValue
+ ) == .success else { return nil }
+ return roleValue as? String
+}
+
func isEditableTextElement(_ element: AXUIElement) -> Bool {
// A live selection means an editable verdict would let generated text
// paste over the user's highlighted text (--editable-target mode probes
@@ -62,12 +73,7 @@ func isEditableTextElement(_ element: AXUIElement) -> Bool {
return false
}
- var roleValue: AnyObject?
- guard AXUIElementCopyAttributeValue(
- element,
- kAXRoleAttribute as CFString,
- &roleValue
- ) == .success, let role = roleValue as? String,
+ guard let role = elementRole(element),
["AXTextField", "AXTextArea", "AXComboBox"].contains(role)
else {
return false
@@ -201,16 +207,35 @@ for attempt in 1...maxRetries {
}
}
+// The editable probe answers in three states: UNKNOWN is the dormant-tree
+// signature (Chromium-family apps never resolve a focused element, or resolve
+// only the bare window), telling the caller accessibility cannot see the field
+// — distinct from a resolved non-text element, a confirmed NOT_EDITABLE.
+if editableTargetMode {
+ // Secure input is a session-global flag set while a password field holds
+ // keyboard focus, covering the fields a dormant tree hides from the
+ // AXSecureTextField subrole check.
+ if IsSecureEventInputEnabled() {
+ writeOutput("NOT_EDITABLE")
+ exit(0)
+ }
+ guard let element = focusedElement else {
+ writeOutput("UNKNOWN")
+ exit(0)
+ }
+ if isEditableTextElement(element) {
+ writeOutput("EDITABLE")
+ } else {
+ writeOutput(elementRole(element) == "AXWindow" ? "UNKNOWN" : "NOT_EDITABLE")
+ }
+ exit(0)
+}
+
guard let resolvedElement = focusedElement else {
writeOutput("NO_ELEMENT")
exit(1)
}
-if editableTargetMode {
- writeOutput(isEditableTextElement(resolvedElement) ? "EDITABLE" : "NOT_EDITABLE")
- exit(0)
-}
-
// Selection capture is a short-lived, read-only mode used before an AI edit.
// It deliberately uses the same AX client as the monitor so a packaged app's
// trusted native helper does not need to rely on System Events automation.
@@ -232,7 +257,11 @@ if selectionReadMode {
}
if selectionResult == .noValue || selectionResult == .attributeUnsupported {
- writeOutput("NONE:")
+ // A dormant Chromium tree reports the window itself as the focused
+ // element; its unsupported AXSelectedText is not evidence of an empty
+ // selection, so surface it as UNKNOWN and let the synthetic-copy path
+ // read the real selection state.
+ writeOutput(elementRole(resolvedElement) == "AXWindow" ? "UNKNOWN:" : "NONE:")
exit(0)
}
diff --git a/src/helpers/selectionManager.js b/src/helpers/selectionManager.js
index 0396efb26..a1d897c69 100644
--- a/src/helpers/selectionManager.js
+++ b/src/helpers/selectionManager.js
@@ -336,9 +336,10 @@ class SelectionManager {
};
}
// Chromium browsers (and any app whose accessibility tree stays dormant)
- // never resolve a focused element, so the read above cannot tell a selection
- // from an empty field. A synthetic copy still can — the same route Windows
- // and Linux take by default.
+ // never resolve a focused element — or resolve only the bare window (the
+ // native read's "unknown" state) — so the read above cannot tell a
+ // selection from an empty field. A synthetic copy still can — the same
+ // route Windows and Linux take by default.
return this._readMacSelectionViaClipboard(pid, expectedTarget, probeEditable);
}
@@ -498,8 +499,24 @@ class SelectionManager {
// to match — so resolve the executable name before a Wayland terminal's
// empty prompt can read as a writable caret.
if (target.kind === "atspi-pid" && (await this._isTerminalPid(target.id))) return capture;
- const editable = await this.textEditMonitor?.isFocusedEditable?.(target);
- return editable ? { status: "editable", target } : capture;
+ // macOS targets normally carry the copier's NSWorkspace app name, already
+ // matched above; only an unnamed target needs its executable resolved so a
+ // terminal cannot become a caret destination. Never resolve a
+ // named one — `ps` reports bundle paths like "Visual Studio Code" whose
+ // "st" substring would misread editors as terminals.
+ if (
+ target.kind === "mac-pid" &&
+ !this._targetSignature(target) &&
+ (await this._isTerminalPid(target.pid))
+ ) {
+ return capture;
+ }
+ const verdict = await this.textEditMonitor?.isFocusedEditable?.(target);
+ if (verdict === "editable") return { status: "editable", target };
+ // An empty copy cannot distinguish a caret from an integrated terminal or
+ // a page without an input; it can also hide a selection matching the clipboard.
+ // Unknown accessibility must keep the panel fallback, including on revalidation.
+ return capture;
}
_isTerminalTarget(...targets) {
@@ -630,9 +647,8 @@ class SelectionManager {
// Wayland) still holds pre-copy content; snapshot it so stale text can't
// be mistaken for the copied selection. Known limitation: a clipboard that
// already held exactly the selected text reads as "no selection". The
- // command then falls back to the Assistant panel — never to a caret paste,
- // because the editable probe reads the focused element's own selection
- // state and refuses a field with a live selection.
+ // editable probe must independently verify an empty writable field before
+ // caret delivery; a live selection or unknown accessibility keeps the panel.
const baseline = new Set([...beforeWrite, ...this.clipboardManager._readClipboardTextAll()]);
const copyResult = await sendCopy();
diff --git a/src/helpers/textEditMonitor.js b/src/helpers/textEditMonitor.js
index a36d443dc..d92c9e317 100644
--- a/src/helpers/textEditMonitor.js
+++ b/src/helpers/textEditMonitor.js
@@ -226,6 +226,13 @@ class TextEditMonitor extends EventEmitter {
resolve({ state: "none" });
return;
}
+ // The focused element resolved but is the bare window — the
+ // dormant-Chromium signature. The selection state is unreadable,
+ // not empty; the caller's synthetic-copy fallback can still read it.
+ if (output === "UNKNOWN:") {
+ resolve({ state: "unknown" });
+ return;
+ }
if (output.startsWith("SELECTED_B64:")) {
try {
resolve({
@@ -297,14 +304,18 @@ class TextEditMonitor extends EventEmitter {
);
}
- async isFocusedEditable(target, timeoutMs = 1000) {
+ // Resolves "editable" | "not_editable" | "unknown". "unknown" is macOS-only:
+ // the dormant-Chromium signature, where accessibility cannot inspect the
+ // focused field at all. The default timeout reuses SELECTED_TEXT_TIMEOUT_MS
+ // because a dormant tree's verdict only arrives after the same retry ladder.
+ async isFocusedEditable(target, timeoutMs = SELECTED_TEXT_TIMEOUT_MS) {
const resolved = this.resolveBinary();
- if (!resolved) return false;
+ if (!resolved) return "not_editable";
let args;
if (process.platform === "darwin") {
const pid = target?.kind === "mac-pid" ? target.pid : this.lastTargetPid;
- if (!pid) return false;
+ if (!pid) return "not_editable";
args = [...resolved.args, "--editable-target", String(pid)];
} else {
args = [...resolved.args, "--probe-editable"];
@@ -315,6 +326,8 @@ class TextEditMonitor extends EventEmitter {
// then emits its monitor output and keeps running — closing stdin and
// resolving on that first line keeps the stale case a fast "not editable"
// rather than a hang until the timeout.
+ const toVerdict = (line) =>
+ line === "EDITABLE" ? "editable" : line === "UNKNOWN" ? "unknown" : "not_editable";
return new Promise((resolve) => {
const child = spawn(resolved.command, args, { stdio: ["pipe", "pipe", "ignore"] });
let buffered = "";
@@ -328,16 +341,16 @@ class TextEditMonitor extends EventEmitter {
} catch {}
resolve(verdict);
};
- const timer = setTimeout(() => settle(false), timeoutMs);
+ const timer = setTimeout(() => settle("not_editable"), timeoutMs);
child.stdin.on("error", () => {});
child.stdin.end();
child.stdout.on("data", (data) => {
buffered += data.toString();
const newline = buffered.indexOf("\n");
- if (newline !== -1) settle(buffered.slice(0, newline).trim() === "EDITABLE");
+ if (newline !== -1) settle(toVerdict(buffered.slice(0, newline).trim()));
});
- child.on("error", () => settle(false));
- child.on("close", () => settle(buffered.trim() === "EDITABLE"));
+ child.on("error", () => settle("not_editable"));
+ child.on("close", () => settle(toVerdict(buffered.trim())));
});
}
diff --git a/test/helpers/selectionManager.test.js b/test/helpers/selectionManager.test.js
index a646842c6..4b0e1b7f9 100644
--- a/test/helpers/selectionManager.test.js
+++ b/test/helpers/selectionManager.test.js
@@ -142,7 +142,7 @@ test("a terminal-flagged target is refused as a caret destination without probin
textEditMonitor: {
isFocusedEditable: async (target) => {
probes.push(target);
- return true;
+ return "editable";
},
},
platform: "win32",
@@ -182,7 +182,7 @@ test("a Linux AT-SPI terminal pid never becomes a caret delivery target", async
textEditMonitor: {
isFocusedEditable: async (target) => {
probes.push(target);
- return true;
+ return "editable";
},
},
platform: "linux",
@@ -562,7 +562,7 @@ test("a terminal target reads as no selection", async () => {
};
const manager = new SelectionManager({
clipboardManager,
- textEditMonitor: { isFocusedEditable: async () => true },
+ textEditMonitor: { isFocusedEditable: async () => "editable" },
platform: "linux",
now: () => 1000,
});
@@ -648,6 +648,122 @@ test("a macOS line copy in a line-copy editor reads as no selection", async () =
assert.equal((await manager.captureSelectedText()).status, "none");
});
+test("an unverifiable macOS field with no selection stays on the panel route", async () => {
+ const { manager } = makeMacClipboardHarness({ copied: null });
+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";
+
+ const result = await manager.captureSelectedText({ probeEditable: true });
+ assert.equal(result.status, "none");
+ assert.equal(result.sessionId, undefined);
+});
+
+test("a confirmed non-editable macOS focus stays on the panel route", async () => {
+ const { manager } = makeMacClipboardHarness({ copied: null });
+ manager.textEditMonitor.isFocusedEditable = async () => "not_editable";
+
+ assert.equal((await manager.captureSelectedText({ probeEditable: true })).status, "none");
+});
+
+test("an unknown verdict in a terminal is still refused as a caret", async () => {
+ const { manager } = makeMacClipboardHarness({ copyOutput: "COPY_OK 42 Ghostty", copied: null });
+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";
+
+ assert.equal((await manager.captureSelectedText({ probeEditable: true })).status, "none");
+});
+
+// COPY_OK reports the app name best-effort; when it is missing, the executable
+// must be resolved before an unnamed terminal can be trusted as a caret.
+test("an unnamed macOS target resolves the executable before trusting a caret", async () => {
+ const { manager } = makeMacClipboardHarness({ copyOutput: "COPY_OK 42", copied: null });
+ manager.textEditMonitor.isFocusedEditable = async () => "editable";
+ manager._readExecutablePath = async () => "/Applications/Ghostty.app/Contents/MacOS/ghostty";
+
+ assert.equal((await manager.captureSelectedText({ probeEditable: true })).status, "none");
+});
+
+test("a caret verified after clipboard capture delivers the assistant response", async () => {
+ const { manager } = makeMacClipboardHarness({ copied: null });
+ const pastes = [];
+ manager.textEditMonitor.isFocusedEditable = async () => "editable";
+ manager.clipboardManager._pasteText = async (text, options) => {
+ pastes.push({ text, options });
+ return { restoreComplete: Promise.resolve() };
+ };
+
+ const capture = await manager.captureSelectedText({ probeEditable: true });
+ assert.deepEqual(await manager.pasteAtCapturedTarget(capture.sessionId, "Agent response"), {
+ success: true,
+ });
+ assert.equal(pastes[0].text, "Agent response");
+});
+
+for (const { name, appName, copied } of [
+ { name: "selection matches the clipboard", appName: "Dia", copied: "user clipboard" },
+ { name: "selection is a full code line", appName: "Code", copied: "const value = 1;\n" },
+ { name: "focus moves to an integrated terminal", appName: "Code", copied: null },
+ { name: "focus moves to a browser page", appName: "Chrome", copied: null },
+]) {
+ test(`a verified caret stops delivery when AX becomes unknown and ${name}`, async () => {
+ const { manager } = makeMacClipboardHarness({ copyOutput: `COPY_OK 42 ${appName}`, copied });
+ const pastes = [];
+ manager.textEditMonitor.getSelectedText = async () => ({ state: "none", editable: true });
+ manager._readExecutablePath = async () => appName;
+ manager.clipboardManager._pasteText = async (text) => {
+ pastes.push(text);
+ return { restoreComplete: Promise.resolve() };
+ };
+ const capture = await manager.captureSelectedText({ probeEditable: true });
+ assert.equal(capture.status, "editable");
+
+ // The app/PID is unchanged, but the focused field is no longer inspectable.
+ manager.textEditMonitor.getSelectedText = async () => ({ state: "unknown" });
+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";
+
+ assert.deepEqual(await manager.pasteAtCapturedTarget(capture.sessionId, "Agent response"), {
+ success: false,
+ code: "target_changed",
+ });
+ assert.deepEqual(pastes, []);
+ });
+}
+
+test("an assistant answer remains available when a dormant browser has no input", async () => {
+ const { createAssistantResponseDelivery, deliverAssistantResponse } =
+ await import("../../src/helpers/assistantResponseDelivery.ts");
+ const { manager } = makeMacClipboardHarness({ copyOutput: "COPY_OK 42 Chrome" });
+ const pastes = [];
+ const writes = [];
+ manager.textEditMonitor.getSelectedText = async () => ({ state: "unknown" });
+ manager.textEditMonitor.isFocusedEditable = async () => "unknown";
+ // Posting Cmd+V succeeds even when there is no field to receive it.
+ manager.clipboardManager._pasteText = async (text) => {
+ pastes.push(text);
+ return { restoreComplete: Promise.resolve() };
+ };
+
+ const capture = await manager.captureSelectedText({ probeEditable: true });
+ const delivery = createAssistantResponseDelivery({
+ autoPasteEnabled: true,
+ deliverySessionId: capture.sessionId,
+ restoreClipboard: true,
+ allowClipboardFallback: false,
+ });
+ const result = await deliverAssistantResponse(delivery, "Agent response", {
+ electronAPI: {
+ pasteAtCapturedTarget: (...args) => manager.pasteAtCapturedTarget(...args),
+ writeClipboard: async (text) => {
+ writes.push(text);
+ return { success: true };
+ },
+ },
+ clipboard: {},
+ });
+
+ assert.deepEqual(result, { pasted: false, copied: true });
+ assert.deepEqual(pastes, []);
+ assert.deepEqual(writes, ["Agent response"]);
+});
+
test("a failed macOS copy stays non-fatal so the command still runs", async () => {
const { manager } = makeMacClipboardHarness({ copied: null });
manager._runCopyHelper = async () => ({ success: false, stdout: "", stderr: "" });
diff --git a/test/helpers/textEditMonitorSelection.test.js b/test/helpers/textEditMonitorSelection.test.js
index d706d73c1..449a7cb36 100644
--- a/test/helpers/textEditMonitorSelection.test.js
+++ b/test/helpers/textEditMonitorSelection.test.js
@@ -99,17 +99,25 @@ test("getSelectedText identifies an empty writable field", darwinOnly, async ()
assert.deepEqual(await m.getSelectedText(42), { state: "none", editable: true });
});
-test("isFocusedEditable accepts only an explicit native editable verdict", darwinOnly, async () => {
- const editable = new TextEditMonitor();
- editable.resolveBinary = () => ({ command: "/bin/sh", args: ["-c", 'echo "EDITABLE"'] });
- const readOnly = new TextEditMonitor();
- readOnly.resolveBinary = () => ({
- command: "/bin/sh",
- args: ["-c", 'echo "NOT_EDITABLE"'],
- });
-
- assert.equal(await editable.isFocusedEditable({ kind: "mac-pid", pid: 42 }), true);
- assert.equal(await readOnly.isFocusedEditable({ kind: "mac-pid", pid: 42 }), false);
+test("isFocusedEditable maps the native verdicts, including the dormant-tree UNKNOWN", darwinOnly, async () => {
+ const withOutput = (line) => {
+ const m = new TextEditMonitor();
+ m.resolveBinary = () => ({ command: "/bin/sh", args: ["-c", `echo "${line}"`] });
+ return m;
+ };
+
+ assert.equal(
+ await withOutput("EDITABLE").isFocusedEditable({ kind: "mac-pid", pid: 42 }),
+ "editable"
+ );
+ assert.equal(
+ await withOutput("NOT_EDITABLE").isFocusedEditable({ kind: "mac-pid", pid: 42 }),
+ "not_editable"
+ );
+ assert.equal(
+ await withOutput("UNKNOWN").isFocusedEditable({ kind: "mac-pid", pid: 42 }),
+ "unknown"
+ );
});
test("isFocusedEditable answers from a stale binary's first monitor line", darwinOnly, async () => {
@@ -123,10 +131,20 @@ test("isFocusedEditable answers from a stale binary's first monitor line", darwi
});
const startedAt = Date.now();
- assert.equal(await m.isFocusedEditable({ kind: "mac-pid", pid: 42 }), false);
+ assert.equal(await m.isFocusedEditable({ kind: "mac-pid", pid: 42 }), "not_editable");
assert.ok(Date.now() - startedAt < 900, "must resolve on the first output line");
});
+// The bare window as the focused element is the dormant-Chromium signature —
+// the selection state is unreadable, not empty, so the caller must fall to the
+// synthetic-copy read instead of treating it as "no selection".
+test("getSelectedText surfaces the native UNKNOWN verdict", darwinOnly, async () => {
+ const m = new TextEditMonitor();
+ m.resolveBinary = () => ({ command: "/bin/sh", args: ["-c", 'echo "UNKNOWN:"'] });
+
+ assert.deepEqual(await m.getSelectedText(42), { state: "unknown" });
+});
+
test("a non--25212 native failure is not cached", darwinOnly, async () => {
const m = new TextEditMonitor();
m.resolveBinary = () => ({