Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

fix(voice): reduce dictation delivery stalls and retain unconfirmed pastes (#280)

Ruben Lütke763c81bfced1
diff --git a/docs/os-parity.md b/docs/os-parity.md
index 1b1e3867fc..34760d8d2d 100644
--- a/docs/os-parity.md
+++ b/docs/os-parity.md
@@ -601,7 +601,7 @@ experiences today.
|---|---|---|---|---|---|
| P-29 | Low | Subscription voice | The dedicated ChatGPT-subscription voice login is an interactive browser flow, so a headless Linux host — and a graphical Linux desktop that ships no terminal emulator able to host the login for its full lifetime — can never CONNECT the profile there (an existing login still reports ready and calls work through the browser voice bridge) | `jarvis/codex_app_server.py::_login_required_state`, `_linux_login_terminal_missing`, `start_codex_subscription_login`, `jarvis/codex_auth.py::_LINUX_LOGIN_TERMINALS` | Both cases report the same `lifecycle_unavailable` truth on every surface (card, activation, voice-mode, Test), each with its own actionable reason — "run Jarvis on a desktop" or "install one of these terminals" — and never an enabled Connect button that can only produce an error toast |
| P-24 | Medium | Dictation shortcut | The global dictation/call shortcut needs `pynput` on Linux/X11, and `pynput` hard-requires `evdev` — which is published **source-only** (verified on PyPI 2026-07-28: evdev 1.9.3 ships an sdist and no wheels) and compiles against the kernel headers. Putting it in `[full]` would break the one advertised install path on a stock `python:3.11-slim`, so it is the opt-in `[desktop-linux]` extra instead. Wayland is a separate, unfixable-by-install case: the compositor owns global shortcuts by design (the XDG `GlobalShortcuts` portal lets the *compositor* assign the keys, and no wlroots compositor implements it at all) | `pyproject.toml` (`desktop-linux`), `jarvis/platform/probes.py::has_hotkey`, `jarvis/trigger/backends/noop.py::explain_unavailable` | X11 without the extra: no global shortcut, and the log/UI now names the actual cause and the exact `pip install` that fixes it (it used to blame Wayland unconditionally). Wayland: no global shortcut at all — bind a compositor shortcut to `jarvis api dictation start`. On both, dictation still works from the Jarvis Bar, the Dictation view and the CLI, and voice still works via the wake word |
-| P-25 | Medium | Dictation insertion | Pasting the transcript into another application is blocked, silently, in three OS-specific situations: Windows UIPI when the foreground window is elevated and Jarvis is not (`SendInput` reports success and the input is discarded), macOS Secure Input while a password field is focused, and Wayland outright (no synthetic input). Detection exists for the first two; Wayland is refused up front. Two further silent failures are Windows-only in their FIX: a chord the target does not bind as "paste" (an xterm.js terminal in a Tauri/Electron app swallows Ctrl+V as `^V`), and a target that reads the clipboard late (an async WebView bridge on a busy machine) after the 120 ms restore timer had already put the previous clipboard back | `jarvis/dictation/insert.py::describe_target`, `_insert_windows_verified`, `jarvis/platform/clipboard_offer.py`, `jarvis/platform/input_isolation.py::windows_foreground_window_is_elevated`, `macos_secure_input_enabled` | All three blocks degrade to the SAME honest outcome instead of silence: the transcript is left on the clipboard, the result is reported as `clipboard_only`, and the bar plus the Dictation view say why and that Ctrl+V will paste it. **Windows** additionally offers the text with delayed rendering and watches who reads it: on a host without a clipboard watcher (no Remote Desktop client, clipboard history off — the default) a paste is proven by the target's read, silence cascades Ctrl+V → Ctrl+Shift+V → Shift+Insert → typing (line breaks as Shift+Enter), and the route is remembered per executable; on a host with a watcher the offer is blind, ONE chord goes out (never a guessed second paste) and the previous clipboard is restored after a 2 s grace only if the dictated text is still on it. **macOS / Linux X11**: plain chord + 120 ms timer restore, unchanged — no delayed-rendering equivalent exists there (NSPasteboard promises and X11 selections notify the owner too, but are a follow-up). macOS Secure Input detection is implemented but has not been verified on real hardware from this machine |
+| P-25 | Medium | Dictation insertion | Pasting the transcript into another application is blocked, silently, in three OS-specific situations: Windows UIPI when the foreground window is elevated and Jarvis is not (`SendInput` reports success and the input is discarded), macOS Secure Input while a password field is focused, and Wayland outright (no synthetic input). Detection exists for the first two; Wayland is refused up front. Two further silent failures are Windows-only in their FIX: a chord the target does not bind as "paste" (an xterm.js terminal in a Tauri/Electron app swallows Ctrl+V as `^V`), and a target that reads the clipboard late (an async WebView bridge on a busy machine) after the 120 ms restore timer had already put the previous clipboard back | `jarvis/dictation/insert.py::describe_target`, `_insert_windows_verified`, `jarvis/platform/clipboard_offer.py`, `jarvis/platform/input_isolation.py::windows_foreground_window_is_elevated`, `macos_secure_input_enabled` | All three blocks degrade to the SAME honest outcome instead of silence: the transcript is left on the clipboard, the result is reported as `clipboard_only`, and the bar plus the Dictation view say why and that Ctrl+V will paste it. **Windows** offers the text with delayed rendering and sends exactly one configured paste shortcut. Only a render requested by the captured target process confirms clipboard consumption; merely opening the clipboard or a watcher reading it is not confirmation. A missing acknowledgement never triggers another shortcut, character typing, or an executable-wide learned route. Unconfirmed delivery reports `paste_sent` and leaves the transcript on the clipboard for a late or manual paste, without a restore timer. Focus changes or held modifiers before key emission leave the text available as `clipboard_only`. Confirmed reads may restore the previous clipboard only while the offer still owns it. The explicit typing setting remains available. Regression tests cover delayed readers, watchers, focus changes, and offer shutdown; these checks do not establish compatibility with every third-party editor. **macOS / Linux X11**: plain chord + 120 ms timer restore, unchanged — no delayed-rendering equivalent exists there (NSPasteboard promises and X11 selections notify the owner too, but are a follow-up). macOS Secure Input detection is implemented but has not been verified on real hardware from this machine |
| P-04 | Medium | CU typing | Linux desktop Unicode text input needs the system `xdotool` binary (pip cannot install it); the pyautogui fallback used on Linux drops non-ASCII chars (umlauts, CJK, emoji) without it | `jarvis/cu/actuate/posix.py::type_text`, `jarvis/plugins/tool/type_text.py` | With `xdotool` (installer provisions it since 2026-07-15): fine. Without, the drop is now reported HONESTLY (2026-07-23): an all-non-ASCII text fails with an actionable "install xdotool" error, and a mixed text types its ASCII portion and warns that the rest was dropped — no more silent success |
| P-05 | Low | Wiki | Wiki search hard-fails (RuntimeError with actionable apt/pysqlite3 remediation) on distros whose system SQLite lacks FTS5 | `jarvis/memory/wiki/fts_index.py:279` | `python:3.11-slim` and macOS ship FTS5 — only exotic/old distros affected; message is honest. Decision 2026-07-16: kept as honest hard error — a pysqlite3 shim would rewire seven wiki modules for an exotic audience |
| P-07 | Low | Audio | No macOS/Linux host-API preference exists (the Windows-name-driven tables are intentionally inert off Windows — documented in-code since 2026-07-16), and headset-name heuristics are Windows-centric | `jarvis/audio/player.py`, `jarvis/audio/capture.py` | Device auto-pick falls back to OS default order — works, less clever than on Windows |
diff --git a/jarvis/dictation/insert.py b/jarvis/dictation/insert.py
index 1de27da96e..dd2e1e6cb2 100644
--- a/jarvis/dictation/insert.py
+++ b/jarvis/dictation/insert.py
@@ -32,6 +32,7 @@ dependency. Nothing is imported at module scope that a headless host lacks.
from __future__ import annotations
import logging
+import os
import sys
import threading
import time
@@ -39,6 +40,7 @@ from dataclasses import dataclass
from typing import Literal
log = logging.getLogger(__name__)
+_INSERT_LOCK = threading.Lock()
#: Paste chords by name. ``auto`` resolves per platform at call time.
#: Ctrl+V is wrong in many terminals (it is an interrupt or a literal there),
@@ -117,13 +119,8 @@ CUSTOM_CHORD_KEYS: dict[str, str] = {
"down": "down",
}
-#: ``paste_sent`` exists because Jarvis does not paste — it asks the foreground
-#: application to paste by sending a synthetic chord, and an application that
-#: does not bind that chord simply ignores it. There is no error and nothing to
-#: read back (see the module docstring), so for a chord we did not curate the
-#: only true statement is "the keystroke went out; what the app did with it is
-#: unknown". Reporting ``inserted`` there would be the exact lie this module
-#: exists to avoid.
+#: ``paste_sent`` means the chord was sent but its delivery was not observed.
+#: Keep the transcript available for manual recovery and never retry on silence.
InsertStatus = Literal["inserted", "paste_sent", "clipboard_only", "unavailable"]
@@ -145,7 +142,7 @@ class InsertResult:
``clipboard_only`` is a SUCCESS state, not a failure: the text is one
Ctrl+V away and the user was told so. ``paste_sent`` is the honest middle
- ground for a user-recorded chord (the keystroke went out, the outcome is
+ ground for unconfirmed delivery (the keystroke went out, the outcome is
unknown, so the transcript is left on the clipboard). Only ``unavailable``
means the text could not even be parked.
"""
@@ -236,9 +233,7 @@ def normalize_paste_chord(value: str) -> tuple[str, str]:
def paste_chord_is_curated(label: str) -> bool:
"""Is this the label of a chord we know actually means "paste"?
- The curated names (and ``auto``, which resolves to one of them) are chords
- that paste in real applications, so a sent-without-error chord may honestly
- be reported as ``inserted``. A recorded combo carries no such warrant.
+ This describes a known binding, not whether a particular field pasted.
"""
return str(label or "").strip().lower() in PASTE_CHORDS
@@ -416,13 +411,39 @@ def insert_text(
delay_ms: int = 120,
delay_after_ms: int = 120,
restore_clipboard: bool = True,
+) -> InsertResult:
+ """Deliver once; overlapping repeat requests must not replace an active offer."""
+ if not _INSERT_LOCK.acquire(blocking=False):
+ return InsertResult(
+ "unavailable",
+ "Another dictation is being pasted. This text remains in dictation history.",
+ False,
+ )
+ try:
+ return _insert_text(
+ text, method=method, paste_chord=paste_chord, delay_ms=delay_ms,
+ delay_after_ms=delay_after_ms, restore_clipboard=restore_clipboard,
+ )
+ finally:
+ _INSERT_LOCK.release()
+
+
+def _insert_text(
+ text: str,
+ *,
+ method: str,
+ paste_chord: str,
+ delay_ms: int,
+ delay_after_ms: int,
+ restore_clipboard: bool,
) -> InsertResult:
"""Insert ``text`` into the focused field. Never raises.
- The clipboard route (default) writes the text, sends the paste chord and
- puts the previous clipboard content back. The ``type`` route synthesises
- the characters instead — correct for the rare control that ignores paste,
- but slow and easily mangled by editor autocomplete, so it is opt-in.
+ The clipboard route writes text before sending one paste chord. Windows
+ restores the previous text only after the target requests the offer;
+ unconfirmed delivery keeps the transcript available for manual paste.
+ The ``type`` route synthesises characters instead; it remains opt-in
+ because character delivery is slow and can trigger editor autocomplete.
Either way the text is written to the clipboard FIRST, so every failure
path below degrades to "it is one Ctrl+V away" rather than to silence.
@@ -441,6 +462,7 @@ def insert_text(
)
from jarvis.platform import clipboard
+ target = _foreground_target()
# 1. Remember what was there. ``None`` means the clipboard is unreachable
# (not that it was empty) — restoring on that would CLEAR it, so the
@@ -508,6 +530,9 @@ def insert_text(
)
if method == "type":
+ blocked = _input_block_reason(target)
+ if blocked:
+ return InsertResult("clipboard_only" if parked else "unavailable", blocked, parked)
try:
actuator.type_text(text)
except Exception as exc: # noqa: BLE001
@@ -520,11 +545,15 @@ def insert_text(
),
clipboard_holds_text=parked,
)
- restored = _restore(clipboard, previous) if restore_clipboard else False
+ restored = (
+ _restore(clipboard, previous)
+ if restore_clipboard and _clipboard_still_holds(clipboard, text)
+ else False
+ )
return InsertResult(
status="inserted",
detail="",
- clipboard_holds_text=not restored,
+ clipboard_holds_text=parked and not restored,
method="type",
clipboard_restored=restored,
)
@@ -537,6 +566,7 @@ def insert_text(
delay_ms=delay_ms,
delay_after_ms=delay_after_ms,
previous=previous if restore_clipboard else None,
+ target=target,
)
if verified is not None:
return verified
@@ -546,6 +576,9 @@ def insert_text(
# Load-bearing: without it the target app can still be holding the
# PREVIOUS clipboard content when the chord arrives, and pastes that.
time.sleep(delay_ms / 1000.0)
+ blocked = _input_block_reason(target)
+ if blocked:
+ return InsertResult("clipboard_only", blocked, parked)
try:
actuator.key_combo(chord)
except Exception as exc: # noqa: BLE001
@@ -563,10 +596,9 @@ def insert_text(
# snatches the text away before the target app has read it.
time.sleep(delay_after_ms / 1000.0)
- if not paste_chord_is_curated(chord_name):
- # A recorded chord. It was sent without error, but "sent" is not
- # "pasted": an application that does not bind this combination simply
- # ignores it, and there is nothing to read back (module docstring).
+ if os.name == "nt" or not paste_chord_is_curated(chord_name):
+ # Without Windows rendering evidence, or with an arbitrary recorded
+ # chord, sending input does not establish that the field pasted it.
# So the clipboard is deliberately NOT restored — putting the previous
# content back here would delete the one copy the user can still reach
# if the chord landed nowhere.
@@ -582,7 +614,11 @@ def insert_text(
clipboard_restored=False,
)
- restored = _restore(clipboard, previous) if restore_clipboard else False
+ restored = (
+ _restore(clipboard, previous)
+ if restore_clipboard and _clipboard_still_holds(clipboard, text)
+ else False
+ )
return InsertResult(
status="inserted",
detail="",
@@ -592,125 +628,85 @@ def insert_text(
)
-#: Chords tried, in order, when the configured one is not answered by a
-#: clipboard read. Windows only — each of them means "paste" somewhere real:
-#: Ctrl+V nearly everywhere, Ctrl+Shift+V in terminals and "paste as plain
-#: text" fields, Shift+Insert in the classic controls and most terminals.
-WINDOWS_CHORD_CASCADE: tuple[str, ...] = ("ctrl_v", "ctrl_shift_v", "shift_insert")
-
-#: How long the target application gets to read the clipboard after a chord
-#: before that chord is declared "not a paste here". An xterm.js paste through
-#: a Tauri/Electron IPC bridge answers in well under 100 ms; the margin covers
-#: a busy machine.
-PASTE_READ_WAIT_S: float = 0.6
-
-#: On a blind host (see ``_insert_windows_verified``) the previous clipboard
-#: content comes back after this long, in the background, and only if the
-#: clipboard still holds the dictated text. Long enough for a WebView that
-#: pastes through an async IPC bridge on a busy machine; a copy the user
-#: makes in the meantime is never overwritten.
-RESTORE_GRACE_S: float = 2.0
-
-#: Processes that read EVERY clipboard change and therefore prove nothing —
-#: remote-desktop clipboard sync and the Windows clipboard-history service.
-#: Measured: ``msrdc.exe`` reads within 5 ms of every write (2026-08-24).
-_CLIPBOARD_WATCHER_EXES: frozenset[str] = frozenset(
- {"msrdc.exe", "mstsc.exe", "rdpclip.exe", "svchost.exe", "vmware-tray.exe", "vmtoolsd.exe"}
-)
-
-#: What worked last time for a given foreground executable, so the second
-#: dictation into the same app skips straight to the route that landed. A
-#: value is a curated chord name or ``"type"``. Process-local on purpose: an
-#: app update can change its bindings, and a stale file would outlive it.
-_LEARNED_ROUTES: dict[str, str] = {}
+# A missing render event never proves a failed paste. Slow applications and
+# clipboard watchers can both consume the same offer without another event.
+PASTE_READ_WAIT_S: float = 0.35
def _clipboard_offer_factory():
- """The delayed-rendering clipboard offer, or ``None`` off Windows.
-
- A function rather than an import so tests can swap the factory and so a
- host without the mechanism costs one attribute check, not an import.
- """
- if sys.platform != "win32":
+ """The delayed-rendering clipboard offer, or ``None`` off Windows."""
+ if os.name != "nt" or sys.platform != "win32":
return None
try:
from jarvis.platform.clipboard_offer import ClipboardOffer, available
- except Exception: # noqa: BLE001 — a missing helper is the plain path, not an error
+ except Exception: # noqa: BLE001 - preserve the plain paste fallback
log.debug("clipboard offer unavailable", exc_info=True)
return None
return ClipboardOffer if available() else None
-def _foreground_exe() -> str:
- """Executable name of the foreground window's owner, ``""`` when unknown."""
- if sys.platform != "win32":
- return ""
+def _foreground_target() -> tuple[int, int] | None:
+ """Snapshot the foreground HWND and PID without UIA or focus changes."""
+ if os.name != "nt" or sys.platform != "win32":
+ return None
try:
- import ctypes # noqa: PLC0415 — lazy (HN-7)
+ import ctypes # noqa: PLC0415
from ctypes import wintypes # noqa: PLC0415
- from jarvis.platform.clipboard_offer import _exe_name
-
user32 = ctypes.WinDLL("user32", use_last_error=True)
+ user32.GetForegroundWindow.argtypes = []
user32.GetForegroundWindow.restype = wintypes.HWND
+ user32.GetWindowThreadProcessId.argtypes = [
+ wintypes.HWND, ctypes.POINTER(wintypes.DWORD),
+ ]
+ user32.GetWindowThreadProcessId.restype = wintypes.DWORD
hwnd = user32.GetForegroundWindow()
if not hwnd:
- return ""
+ return None
pid = wintypes.DWORD()
user32.GetWindowThreadProcessId(hwnd, ctypes.byref(pid))
- return _exe_name(int(pid.value))
- except Exception: # noqa: BLE001 — unknown app, no learning, still a paste
- log.debug("could not read the foreground executable", exc_info=True)
- return ""
+ return (int(hwnd), int(pid.value)) if pid.value else None
+ except Exception: # noqa: BLE001 - no probe means no delivery evidence
+ log.debug("could not read the foreground target", exc_info=True)
+ return None
-def _type_with_soft_newlines(actuator: object, text: str) -> None:
- """Type *text*, sending each line break as Shift+Enter.
+def _input_block_reason(target: tuple[int, int] | None) -> str:
+ """Do not send a chord to a changed window or with physically held modifiers."""
+ if os.name != "nt" or sys.platform != "win32":
+ return ""
+ if target is not None and _foreground_target() != target:
+ return (
+ "The focused window changed. The text is on your clipboard; "
+ "paste it where you want it."
+ )
+ try:
+ import ctypes # noqa: PLC0415
+ from ctypes import wintypes # noqa: PLC0415
- A plain Enter SUBMITS in the places that refuse to paste — a coding agent's
- terminal prompt, a chat composer — so a dictated paragraph with a line
- break would go out half-written. Shift+Enter is the line break in those
- places and a harmless line break in an ordinary editor.
- """
- lines = text.replace("\r\n", "\n").replace("\r", "\n").split("\n")
- for index, line in enumerate(lines):
- if index:
- actuator.key_combo(["shift", "enter"]) # type: ignore[attr-defined]
- if line:
- actuator.type_text(line, delay_s=0.002) # type: ignore[attr-defined]
+ user32 = ctypes.WinDLL("user32", use_last_error=True)
+ user32.GetAsyncKeyState.argtypes = [ctypes.c_int]
+ user32.GetAsyncKeyState.restype = wintypes.SHORT
+ # SendInput does not reset held keys. Never manufacture key-up events
+ # for the user's keys: retain the transcript for a deliberate paste.
+ if any(user32.GetAsyncKeyState(vk) & 0x8000 for vk in (0x10, 0x11, 0x12, 0x5B, 0x5C)):
+ return (
+ "A modifier key is still held. Release it and paste the text "
+ "from your clipboard."
+ )
+ except Exception: # noqa: BLE001 - optional keyboard-state probe
+ log.debug("could not inspect modifier keys before dictation paste", exc_info=True)
+ return ""
def _clipboard_still_holds(clipboard_module: object, text: str) -> bool:
- """Is our text still what the clipboard shows? Unreadable counts as yes."""
+ """Restore only positively identified text; unreadable means do not write."""
try:
current = clipboard_module.read_text() # type: ignore[attr-defined]
- except Exception: # noqa: BLE001 — cannot check, do not clobber
- return True
- if current is None:
- return True
- return current.replace("\r\n", "\n") == text.replace("\r\n", "\n")
-
-
-def _restore_later(
- clipboard_module: object, previous: str | None, text: str, grace_s: float
-) -> None:
- """Put the previous clipboard back after *grace_s*, off the delivery path.
-
- Only if the clipboard still holds OUR text: if the user copied something
- else in the meantime, that copy is theirs and stays. This is the blind-host
- answer to a target that reads the clipboard late (an async WebView bridge
- on a busy machine): the old timer fired at 120 ms and handed such an app
- the previous content — the maintainer's 2026-08-24 report.
- """
- if previous is None:
- return
-
- def _run() -> None:
- time.sleep(grace_s)
- if _clipboard_still_holds(clipboard_module, text):
- _restore(clipboard_module, previous)
-
- threading.Thread(target=_run, name="jarvis-clipboard-restore", daemon=True).start()
+ except Exception: # noqa: BLE001 - unknown ownership must not clobber a copy
+ log.debug("clipboard ownership check failed; skipping restore", exc_info=True)
+ return False
+ return current is not None and current.replace("\r\n", "\n") == text.replace("\r\n", "\n")
def _insert_windows_verified(
@@ -722,165 +718,86 @@ def _insert_windows_verified(
delay_ms: int,
delay_after_ms: int,
previous: str | None,
+ target: tuple[int, int] | None = None,
) -> InsertResult | None:
- """Paste with evidence, and fall back to typing only when it is SAFE.
-
- Windows only; ``None`` on any other host or when the delayed-rendering
- offer cannot be made, and the caller uses the plain chord path.
-
- The text is offered with delayed rendering
- (:mod:`jarvis.platform.clipboard_offer`), which makes the first clipboard
- read visible with the reader's process name. Two hosts follow:
-
- * **Sighted** — nobody read during the settle window. A paste now shows up
- as a read by the foreground app; silence for :data:`PASTE_READ_WAIT_S`
- proves the chord was not a paste there, so the next chord of
- :data:`WINDOWS_CHORD_CASCADE` is tried and, when none is answered, the
- text is typed in — every focused field accepts keystrokes. The route
- that landed is remembered per executable for this process.
- * **Blind** — a watcher (Remote Desktop clipboard sync, clipboard history)
- read during the settle window, so the system has cached the text and a
- later paste leaves no render event. Then ONE chord goes out and nothing
- is inferred from silence: a second chord or typing on a guess would
- paste the prompt twice. What the blind host still gets is the restore
- fix — the previous clipboard comes back only after a read was seen or
- after a long grace, never on the old 120 ms timer.
+ """Send one paste request and retain the transcript unless delivery is observed.
+
+ Only a render requested by the captured foreground process is evidence.
+ A watcher reading, merely opening the clipboard, or no event at all says
+ nothing about whether the focused field inserted text. In particular, a
+ timeout must never emit another chord or type a duplicate transcript.
"""
offer_cls = _clipboard_offer_factory()
if offer_cls is None:
return None
-
- import os # noqa: PLC0415
-
- exe = _foreground_exe()
- configured, _keys = resolve_paste_chord(paste_chord)
- order: list[str] = []
- learned = _LEARNED_ROUTES.get(exe) if exe else None
- for name in ((learned,) if learned else ()) + (configured,) + WINDOWS_CHORD_CASCADE:
- if name and name != "type" and name not in order:
- order.append(name)
- if learned == "type":
- order = []
-
- own_pid = os.getpid()
- for chord_name in order:
- offer = offer_cls(text)
- blind = False
- try:
- if not offer.start():
- log.debug("clipboard offer could not take the clipboard; plain paste path")
- return None
- time.sleep(max(delay_ms, 20) / 1000.0)
- # Whoever read during the settle window is a watcher, not the
- # paste — and it has just consumed the one render there is.
- exclude = {own_pid}
- for early in offer.reads():
- exclude.add(early.pid)
- if early.observed == "render":
- blind = True
- sent_at = offer.elapsed()
- _name, keys = resolve_paste_chord(chord_name)
- try:
- actuator.key_combo(keys) # type: ignore[attr-defined]
- except Exception as exc: # noqa: BLE001
- log.warning("paste chord %s failed: %s", chord_name, exc)
- return InsertResult(
- status="clipboard_only",
- detail=(
- "The paste shortcut could not be sent. The text is on your "
- "clipboard — press Ctrl+V."
- ),
- clipboard_holds_text=True,
- )
- read = offer.wait_for_read(
- exclude_pids=exclude, after_s=sent_at, timeout_s=PASTE_READ_WAIT_S
+ chord_name, keys = resolve_paste_chord(paste_chord)
+ offer = offer_cls(text)
+ sent = False
+ read = None
+ restore_text = None
+ lost_ownership = False
+ try:
+ if not offer.start():
+ return None
+ if delay_ms > 0:
+ time.sleep(delay_ms / 1000.0)
+ blocked = _input_block_reason(target)
+ if blocked or offer.lost_ownership:
+ return InsertResult(
+ "clipboard_only" if not offer.lost_ownership else "unavailable",
+ blocked or (
+ "The clipboard changed before pasting. "
+ "Recover the text from dictation history."
+ ),
+ not offer.lost_ownership,
)
- while read is not None and read.exe in _CLIPBOARD_WATCHER_EXES:
- # A slow watcher answering after the chord: from here on the
- # host is blind for the same reason as above.
- exclude.add(read.pid)
- if read.observed == "render":
- blind = True
- read = offer.wait_for_read(
- exclude_pids=exclude, after_s=sent_at, timeout_s=PASTE_READ_WAIT_S
- )
- finally:
- # Renders the text for real: the clipboard keeps it either way.
- offer.stop()
-
- if read is not None:
- log.info(
- "dictation paste: %s read the clipboard %.0f ms after %s (%s)",
- read.exe or f"pid {read.pid}",
- (read.at - sent_at) * 1000.0,
- chord_name,
- read.observed,
+ early_render = any(item.observed == "render" for item in offer.reads())
+ sent_at = offer.elapsed()
+ # Mark before sending: an exception may follow partial input. Never
+ # fall back to another chord after input could have reached the app.
+ sent = True
+ actuator.key_combo(keys) # type: ignore[attr-defined]
+ if not early_render:
+ candidate = offer.wait_for_read(
+ exclude_pids={os.getpid()}, after_s=sent_at, timeout_s=PASTE_READ_WAIT_S,
)
- if exe:
- _LEARNED_ROUTES[exe] = chord_name
+ if (
+ candidate is not None
+ and candidate.observed == "render"
+ and target is not None
+ and candidate.pid == target[1]
+ ):
+ read = candidate
+ lost_ownership = offer.lost_ownership
+ if read is not None and not lost_ownership:
if delay_after_ms > 0:
time.sleep(delay_after_ms / 1000.0)
- restored = _restore(clipboard_module, previous)
- return InsertResult(
- status="inserted",
- detail="",
- clipboard_holds_text=not restored,
- method=f"clipboard+{chord_name}",
- clipboard_restored=restored,
- )
-
- if blind:
- log.info(
- "dictation paste: %s sent to %s; a clipboard watcher makes this host "
- "blind, so the paste is assumed and the clipboard is restored after "
- "%.1f s at the earliest",
- chord_name,
- exe or "the foreground app",
- RESTORE_GRACE_S,
- )
- _restore_later(clipboard_module, previous, text, RESTORE_GRACE_S)
- return InsertResult(
- status="inserted" if paste_chord_is_curated(chord_name) else "paste_sent",
- detail="",
- clipboard_holds_text=True,
- method=f"clipboard+{chord_name}",
- clipboard_restored=False,
- )
-
- log.info(
- "dictation paste: %s was not answered by a clipboard read in %s; trying the next route",
- chord_name,
- exe or "the foreground app",
- )
+ restore_text = previous
+ except Exception: # noqa: BLE001 - preserve the transcript on a failed probe/send
+ log.warning("dictation paste could not be confirmed", exc_info=True)
+ if not sent:
+ return None
+ finally:
+ offer.stop(restore_text=restore_text)
+ lost_ownership = offer.lost_ownership
- # Sighted host, and no chord was a paste in this application: type it in.
- # The clipboard still holds the text (the offers rendered it) and stays
- # that way — if the keystrokes land nowhere, that copy is the way back.
- try:
- _type_with_soft_newlines(actuator, text)
- except Exception as exc: # noqa: BLE001
- log.warning("synthetic typing failed: %s", exc)
+ if read is not None:
+ restored = offer.restored
+ log.info("dictation paste: target requested text after %s", chord_name)
return InsertResult(
- status="clipboard_only",
- detail=(
- "This app does not paste on any known shortcut and typing into it "
- "did not work. The text is on your clipboard."
- ),
- clipboard_holds_text=True,
+ "inserted", "", not restored and not lost_ownership,
+ f"clipboard+{chord_name}", restored,
)
- if exe:
- _LEARNED_ROUTES[exe] = "type"
- log.info(
- "dictation paste: no paste shortcut is answered in %s; typed %d chars",
- exe or "the foreground app",
- len(text),
- )
+
+ log.info("dictation paste: %s sent; delivery unconfirmed, retaining transcript", chord_name)
return InsertResult(
- status="inserted",
- detail="",
- clipboard_holds_text=True,
- method="type",
- clipboard_restored=False,
+ "paste_sent",
+ (
+ "The paste shortcut was sent. If no text appeared, paste it from "
+ "your clipboard or dictation history."
+ ),
+ not lost_ownership,
+ f"clipboard+{chord_name}",
)
diff --git a/jarvis/dictation/outcomes.py b/jarvis/dictation/outcomes.py
index b2e6386098..df42ecddb0 100644
--- a/jarvis/dictation/outcomes.py
+++ b/jarvis/dictation/outcomes.py
@@ -17,13 +17,13 @@ What each value means
``inserted``
The text was typed or pasted into the focused window. The happy path.
``paste_sent``
- A user-recorded paste shortcut was sent to the focused window and no error
+ A paste shortcut was sent to the focused window and no error
came back — but Jarvis does not paste, it asks the application in front to
paste, and an application that does not bind that shortcut simply ignores
it. There is nothing to read back, so this is as far as honesty reaches:
the keystroke went out, the result is unknown, and the text is left on the
- clipboard instead of being restored away. Only ever produced for a custom
- chord; the curated ones report ``inserted``.
+ clipboard instead of being restored away. This includes curated Windows
+ shortcuts when clipboard consumption cannot be attributed to the target.
``clipboard_only``
Insertion was not possible, so the text was left on the clipboard. The
user still has their words; they just have to paste them.
diff --git a/jarvis/platform/clipboard_offer.py b/jarvis/platform/clipboard_offer.py
index 04db8424af..3de8504fd5 100644
--- a/jarvis/platform/clipboard_offer.py
+++ b/jarvis/platform/clipboard_offer.py
@@ -1,51 +1,20 @@
-"""Offer text on the Windows clipboard and OBSERVE who reads it.
-
-Why this exists
----------------
-Jarvis does not paste; it asks the foreground application to paste by sending
-a synthetic chord. Two things about that are unknowable from the outside:
-whether the chord means "paste" in that application at all (an xterm.js
-terminal left to itself swallows Ctrl+V as ``^V`` and cancels the browser's
-paste), and WHEN the application reads the clipboard — a WebView that pastes
-through an async IPC bridge can read hundreds of milliseconds later on a busy
-machine, after a timer-based "restore the previous clipboard" has already put
-the old content back (the 2026-08-24 BridgeMind report).
-
-Windows offers one mechanism that turns both into a measurement: **delayed
-rendering**. A clipboard owner may publish a format with a ``NULL`` handle;
-the data is requested through ``WM_RENDERFORMAT`` only when some process
-actually calls ``GetClipboardData``. Because the reader must hold the
-clipboard open at that moment, ``GetOpenClipboardWindow`` names it. So
-"did the paste land, and when?" becomes "which process read the clipboard
-after the chord went out?" — a fact, not a guess.
-
-The one limit, measured: the system renders ONCE. After the first reader the
-text is cached and later readers are served silently. A host with a clipboard
-watcher — a Remote Desktop client (``msrdc.exe`` reads within 5 ms of every
-write) or the clipboard-history service — therefore consumes the render before
-the chord is even sent, and from then on the offer is *blind*: a paste can no
-longer be proven absent, only (by polling for who holds the clipboard open)
-occasionally proven present. Callers must treat the two states differently;
-:mod:`jarvis.dictation.insert` does.
-
-Design
-------
-* One hidden message-only window on a dedicated thread owns the clipboard for
- the duration of the offer and answers ``WM_RENDERFORMAT`` with the text.
-* Every render request is recorded with the reader's pid and executable name.
-* On :meth:`ClipboardOffer.stop` the text is rendered for real before the
- window goes away (``WM_RENDERALLFORMATS`` contract), so the clipboard still
- holds the text afterwards — the "one Ctrl+V away" guarantee of
- :mod:`jarvis.dictation.insert` survives.
-* Losing ownership (another app copied something) is recorded, never raised.
-
-Windows only, imported lazily by its one caller. On any other OS
-:func:`available` is ``False`` and nothing here is constructed.
+"""Observe real Windows clipboard rendering without treating silence as failure.
+
+A message-only window owns a delayed Unicode text offer. The first successful
+render records the reader PID; cached reads produce no further events. Clipboard
+watchers can consume that first render, so no event is never proof that a paste
+failed, and merely opening the clipboard is never insertion evidence.
+
+The dedicated thread waits on native messages. Stopping renders the transcript
+for later manual paste. A caller with delivery evidence may request restoration;
+ownership is checked under the same clipboard lock as the replacement, preserving
+any newer user copy. All native imports are lazy and this helper is inert off Windows.
"""
from __future__ import annotations
import logging
+import os
import sys
import threading
import time
@@ -72,50 +41,13 @@ class ClipboardRead:
#: Seconds since the offer went up.
at: float
#: ``render`` — the system asked us for the text (a real ``GetClipboardData``);
- #: ``open`` — the process merely held the clipboard open, seen by polling.
- #: Once the text has been rendered and cached, ``open`` is the only
- #: evidence left (see :meth:`ClipboardOffer.rendered`).
+ #: Historical consumers may supply ``open``; it is not delivery evidence.
observed: str = "render"
def available() -> bool:
"""Can a delayed-rendering offer be made on this host?"""
- return sys.platform == "win32"
-
-
-def _exe_name(pid: int) -> str:
- """Executable name for *pid*, ``""`` when it cannot be read. Never raises."""
- if pid <= 0:
- return ""
- try:
- import ctypes # noqa: PLC0415 — lazy (HN-7)
- from ctypes import wintypes # noqa: PLC0415
-
- kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
- kernel32.OpenProcess.argtypes = [wintypes.DWORD, wintypes.BOOL, wintypes.DWORD]
- kernel32.OpenProcess.restype = wintypes.HANDLE
- kernel32.QueryFullProcessImageNameW.argtypes = [
- wintypes.HANDLE,
- wintypes.DWORD,
- wintypes.LPWSTR,
- ctypes.POINTER(wintypes.DWORD),
- ]
- kernel32.QueryFullProcessImageNameW.restype = wintypes.BOOL
- kernel32.CloseHandle.argtypes = [wintypes.HANDLE]
- handle = kernel32.OpenProcess(0x1000, False, pid) # QUERY_LIMITED_INFORMATION
- if not handle:
- return ""
- try:
- buf = ctypes.create_unicode_buffer(1024)
- size = wintypes.DWORD(1024)
- if not kernel32.QueryFullProcessImageNameW(handle, 0, buf, ctypes.byref(size)):
- return ""
- return buf.value.replace("/", "\\").rsplit("\\", 1)[-1].lower()
- finally:
- kernel32.CloseHandle(handle)
- except Exception: # noqa: BLE001 — a nameless reader is still a reader
- log.debug("could not resolve exe name for pid %s", pid, exc_info=True)
- return ""
+ return os.name == "nt" and sys.platform == "win32"
class ClipboardOffer:
@@ -140,6 +72,8 @@ class ClipboardOffer:
self._read_event = threading.Event()
self._ready = threading.Event()
self._done = threading.Event()
+ self._stop_requested = threading.Event()
+ self._lifecycle_lock = threading.Lock()
self._thread: threading.Thread | None = None
self._hwnd: int = 0
self._started_at = 0.0
@@ -148,6 +82,8 @@ class ClipboardOffer:
#: True once the text was handed to a reader. From then on the system
#: caches it and answers later readers itself — no more render events.
self.rendered = False
+ self.restored = False
+ self._restore_text: str | None = None
self._wndproc_ref: object = None # keeps the ctypes callback alive
# -- public ---------------------------------------------------------
@@ -156,12 +92,20 @@ class ClipboardOffer:
"""Take the clipboard. ``True`` when the offer is up."""
if not available():
return False
- self._thread = threading.Thread(
- target=self._run, name="jarvis-clipboard-offer", daemon=True
- )
- self._thread.start()
- self._ready.wait(timeout_s)
- return self._ok
+ with self._lifecycle_lock:
+ if self._stop_requested.is_set():
+ return False
+ if self._thread is None:
+ self._thread = threading.Thread(
+ target=self._run, name="jarvis-clipboard-offer", daemon=True
+ )
+ self._thread.start()
+ if not self._ready.wait(timeout_s):
+ # A late worker must never take the clipboard after our caller
+ # already chose its fallback delivery route.
+ self.stop()
+ return False
+ return self._ok and not self._done.is_set()
def reads(self) -> list[ClipboardRead]:
with self._lock:
@@ -183,30 +127,55 @@ class ClipboardOffer:
return read
self._read_event.clear()
remaining = deadline - time.monotonic()
- if remaining <= 0 or self._done.is_set():
+ if remaining <= 0 or self._done.is_set() or self.lost_ownership:
return None
- self._read_event.wait(min(remaining, 0.05))
+ self._read_event.wait(remaining)
def elapsed(self) -> float:
"""Seconds since the offer went up."""
return time.monotonic() - self._started_at if self._started_at else 0.0
- def stop(self, *, timeout_s: float = 1.0) -> None:
- """Render the text for real and give the window up. Idempotent."""
- if self._thread is None:
+ def stop(self, *, timeout_s: float = 2.0, restore_text: str | None = None) -> None:
+ """Finish the offer, optionally restoring text only while we still own it."""
+ self._restore_text = restore_text
+ self._stop_requested.set()
+ thread = self._thread
+ if thread is None:
return
if self._hwnd:
try:
import ctypes # noqa: PLC0415
-
- ctypes.WinDLL("user32").PostMessageW(self._hwnd, _WM_APP_STOP, 0, 0)
+ from ctypes import wintypes # noqa: PLC0415
+
+ user32 = ctypes.WinDLL("user32", use_last_error=True)
+ user32.PostMessageW.argtypes = [
+ wintypes.HWND, wintypes.UINT, wintypes.WPARAM, wintypes.LPARAM,
+ ]
+ user32.PostMessageW.restype = wintypes.BOOL
+ if not user32.PostMessageW(self._hwnd, _WM_APP_STOP, 0, 0):
+ log.debug("clipboard offer stop message was refused")
except Exception: # noqa: BLE001 — the thread's own timeout covers it
log.debug("could not post stop to the clipboard offer window", exc_info=True)
- self._thread.join(timeout_s)
- self._thread = None
+ thread.join(timeout_s)
+ if not thread.is_alive():
+ self._thread = None
# -- thread body ----------------------------------------------------
+ def _restore_owned_text(self, user32, hwnd: int, render) -> None:
+ """Check ownership and replace text under one native clipboard lock."""
+ if self._restore_text is None or not user32.OpenClipboard(hwnd):
+ return
+ try:
+ if user32.GetClipboardOwner() == hwnd and user32.EmptyClipboard():
+ self.restored = render(self._restore_text)
+ if self.restored:
+ self._text = self._restore_text
+ else:
+ render()
+ finally:
+ user32.CloseClipboard()
+
def _run(self) -> None:
try:
self._pump()
@@ -219,6 +188,8 @@ class ClipboardOffer:
self._read_event.set()
def _pump(self) -> None:
+ if not available() or self._stop_requested.is_set():
+ return
import ctypes # noqa: PLC0415
from ctypes import wintypes # noqa: PLC0415
@@ -251,6 +222,17 @@ class ClipboardOffer:
]
user32.CreateWindowExW.restype = wintypes.HWND
user32.DestroyWindow.argtypes = [wintypes.HWND]
+ user32.IsWindow.argtypes = [wintypes.HWND]
+ user32.IsWindow.restype = wintypes.BOOL
+ user32.UnregisterClassW.argtypes = [wintypes.LPCWSTR, wintypes.HINSTANCE]
+ user32.UnregisterClassW.restype = wintypes.BOOL
+ user32.PeekMessageW.argtypes = [
+ ctypes.POINTER(wintypes.MSG), wintypes.HWND, wintypes.UINT, wintypes.UINT,
+ wintypes.UINT,
+ ]
+ user32.TranslateMessage.argtypes = [ctypes.POINTER(wintypes.MSG)]
+ user32.DispatchMessageW.argtypes = [ctypes.POINTER(wintypes.MSG)]
+ user32.DispatchMessageW.restype = ctypes.c_ssize_t
user32.OpenClipboard.argtypes = [wintypes.HWND]
user32.SetClipboardData.argtypes = [wintypes.UINT, wintypes.HANDLE]
user32.SetClipboardData.restype = wintypes.HANDLE
@@ -266,9 +248,9 @@ class ClipboardOffer:
kernel32.GetModuleHandleW.argtypes = [wintypes.LPCWSTR]
kernel32.GetModuleHandleW.restype = wintypes.HMODULE
- def render() -> bool:
+ def render(value: str | None = None) -> bool:
"""Hand the real text to whoever asked. Clipboard must be open."""
- buf = ctypes.create_unicode_buffer(self._text)
+ buf = ctypes.create_unicode_buffer(self._text if value is None else value)
size = ctypes.sizeof(buf)
handle = kernel32.GlobalAlloc(_GMEM_MOVEABLE, size)
if not handle:
@@ -298,12 +280,14 @@ class ClipboardOffer:
try:
if msg == _WM_RENDERFORMAT:
pid = reader_pid()
- read = ClipboardRead(pid=pid, exe=_exe_name(pid), at=self.elapsed())
- with self._lock:
- self._reads.append(read)
- self._read_event.set()
if int(wparam) == _CF_UNICODETEXT and render():
self.rendered = True
+ # Publish only successfully rendered data, and keep
+ # filesystem/process queries off the target's read.
+ read = ClipboardRead(pid=pid, exe="", at=self.elapsed())
+ with self._lock:
+ self._reads.append(read)
+ self._read_event.set()
return 0
if msg == _WM_RENDERALLFORMATS:
# We are about to stop owning the clipboard: leave the text
@@ -317,9 +301,10 @@ class ClipboardOffer:
return 0
if msg == _WM_DESTROYCLIPBOARD:
self.lost_ownership = True
+ self._read_event.set()
return 0
if msg == _WM_APP_STOP:
- user32.DestroyWindow(hwnd)
+ self._stop_requested.set()
return 0
if msg == _WM_DESTROY:
user32.PostQuitMessage(0)
@@ -363,6 +348,8 @@ class ClipboardOffer:
taken = False
for _attempt in range(10):
+ if self._stop_requested.is_set():
+ break
if user32.OpenClipboard(hwnd):
try:
user32.EmptyClipboard()
@@ -386,12 +373,8 @@ class ClipboardOffer:
self._ok = True
self._ready.set()
- # A message pump that also POLLS: once the text is rendered the system
- # serves later readers from its cache and sends no more render events,
- # so the only trace a paste leaves is the reader briefly holding the
- # clipboard open. Sampling that every millisecond is cheap and catches
- # a WebView's multi-format read comfortably; a missed sample only
- # means "no positive evidence", never a wrong claim.
+ # Clipboard opening is not a read or an insertion acknowledgment.
+ # Wait for native messages; never sample the clipboard every millisecond.
msg = wintypes.MSG()
user32.MsgWaitForMultipleObjectsEx.argtypes = [
wintypes.DWORD,
@@ -400,38 +383,28 @@ class ClipboardOffer:
wintypes.DWORD,
wintypes.DWORD,
]
- seen_open: set[int] = set()
running = True
try:
- while running:
+ while running and not self._stop_requested.is_set():
while user32.PeekMessageW(ctypes.byref(msg), None, 0, 0, 1):
if msg.message == 0x0012: # WM_QUIT
running = False
break
user32.TranslateMessage(ctypes.byref(msg))
user32.DispatchMessageW(ctypes.byref(msg))
- if not running:
+ if not running or self._stop_requested.is_set():
break
- holder = user32.GetOpenClipboardWindow()
- if holder and holder != hwnd:
- pid = wintypes.DWORD()
- user32.GetWindowThreadProcessId(holder, ctypes.byref(pid))
- if pid.value and pid.value not in seen_open:
- seen_open.add(int(pid.value))
- read = ClipboardRead(
- pid=int(pid.value),
- exe=_exe_name(int(pid.value)),
- at=self.elapsed(),
- observed="open",
- )
- with self._lock:
- self._reads.append(read)
- self._read_event.set()
# QS_ALLINPUT = 0x04FF, MWMO_INPUTAVAILABLE = 0x0004
- user32.MsgWaitForMultipleObjectsEx(0, None, 1, 0x04FF, 0x0004)
+ user32.MsgWaitForMultipleObjectsEx(0, None, 1000, 0x04FF, 0x0004)
finally:
- user32.UnregisterClassW(class_name, wc.hInstance)
- self._hwnd = 0
+ # A newer copy of identical text still belongs to its new owner.
+ try:
+ self._restore_owned_text(user32, hwnd, render)
+ finally:
+ if user32.IsWindow(hwnd):
+ user32.DestroyWindow(hwnd)
+ user32.UnregisterClassW(class_name, wc.hInstance)
+ self._hwnd = 0
__all__ = ["ClipboardOffer", "ClipboardRead", "available"]
diff --git a/jarvis/speech/pipeline.py b/jarvis/speech/pipeline.py
index 5ed402f2a9..9106092286 100644
--- a/jarvis/speech/pipeline.py
+++ b/jarvis/speech/pipeline.py
@@ -1398,11 +1398,15 @@ _DICTATION_TAIL_REREAD_BACK_S = 0.5
# How long the release waits for the incremental polish worker to finish the
# windows it already has — normally the last one, whose formatting started the
-# moment it was read. One configured polish budget plus a little: past that
-# the formatter is not answering and the whole-text pass (which has its own
-# ceiling and fails open to the raw text) takes over.
+# moment it was read. Past that the worker is retired and its completed prefix
+# is retained, leaving only unfinished text for the bounded final polish pass.
_DICTATION_PREFIX_POLISH_WAIT_S = 3.0
+# Dictation notifications update UI state; they must not inherit the bus's
+# five-second observer timeout before a finished transcript can be delivered.
+_DICTATION_EVENT_TIMEOUT_S = 1.0
+_DICTATION_POLISH_CANCEL_WAIT_S = 0.25
+
class _SessionInputBuffer:
"""Replayable bounded handoff for one continuously captured mic stream.
@@ -11878,6 +11882,8 @@ class SpeechPipeline:
stt_models: list[str] = []
detected_languages: list[str] = []
stt_latency_ms = 0.0
+ warmup_wait_ms = 0.0
+ stt_queue_wait_ms = 0.0
stt_calls = 0
final_window_count = 0
# Windows whose transcript stopped at a mid-recording pause and were
@@ -11951,6 +11957,7 @@ class SpeechPipeline:
prefix_polish_windows = 0
prefix_polish_deltas = 0
prefix_polish_failed = False
+ prefix_polish_stopped = False
prefix_polish_task: asyncio.Task[None] | None = None
def _append_unique(values: list[str], value: object) -> None:
@@ -12057,7 +12064,7 @@ class SpeechPipeline:
# ``stt_failures`` is appended to, never rebound, so it needs no
# ``nonlocal`` — the list object itself is the shared state.
nonlocal stt, stt_error, stt_error_detail, session_language
- nonlocal stt_calls, stt_latency_ms
+ nonlocal stt_calls, stt_latency_ms, warmup_wait_ms, stt_queue_wait_ms
ceiling = max(
float(getattr(self, "_stt_final_timeout_s", 8.0) or 8.0),
(len(pcm) / bytes_per_second)
@@ -12070,8 +12077,14 @@ class SpeechPipeline:
# Startup/live-switch warm-up and this call share one native task.
# Joining it avoids both an 11 s cold final decode and a concurrent
# model call that would return TranscribeBusy (AP-24).
+ queued_at = time.perf_counter()
async with stt_gate:
- stt = await self._join_dictation_warmup(stt)
+ stt_queue_wait_ms += (time.perf_counter() - queued_at) * 1000.0
+ warmup_started = time.perf_counter()
+ try:
+ stt = await self._join_dictation_warmup(stt)
+ finally:
+ warmup_wait_ms += (time.perf_counter() - warmup_started) * 1000.0
if probe:
inference_active.set()
call_started = time.perf_counter()
@@ -12424,7 +12437,7 @@ class SpeechPipeline:
continue
last_published = live
try:
- await self._publish_event(
+ await self._publish_dictation_event(
DictationTranscript(
source_layer="speech.dictation",
text=live,
@@ -12722,7 +12735,7 @@ class SpeechPipeline:
nonlocal prefix_polish_deltas, prefix_polish_failed
from jarvis.dictation.merge import merge_transcripts
- while not prefix_polish_failed:
+ while not prefix_polish_failed and not prefix_polish_stopped:
reading = final_reads.get(prefix_polish_windows)
if reading is None:
return
@@ -12740,6 +12753,8 @@ class SpeechPipeline:
delta = merged[len(prefix_polish_raw) :].strip()
if delta:
piece, _status = await _polish_delta(delta, prefix_polish_text)
+ if prefix_polish_stopped:
+ return
if piece:
prefix_polish_text = " ".join(
part for part in (prefix_polish_text, piece) if part
@@ -12751,7 +12766,7 @@ class SpeechPipeline:
def _kick_prefix_polish() -> None:
"""Start the formatting worker unless it is running or given up."""
nonlocal prefix_polish_task
- if prefix_polish_failed or not final_quality_pass:
+ if prefix_polish_failed or prefix_polish_stopped or not final_quality_pass:
return
if not bool(getattr(cfg, "polish", True)) or bool(getattr(cfg, "translate", False)):
return
@@ -12764,10 +12779,11 @@ class SpeechPipeline:
async def _settle_prefix_polish() -> None:
"""Let the worker finish the windows it already has — normally the
last one, started the moment it was read — within one polish budget.
- Past that the whole-text pass takes over rather than making the
- user wait on a formatter that is not answering.
+ Past that, retain the completed prefix and retire its worker. The
+ finish path formats only the remaining tail instead of repeating
+ every completed window while the old formatter is still running.
"""
- nonlocal prefix_polish_failed
+ nonlocal prefix_polish_failed, prefix_polish_stopped
_kick_prefix_polish()
task = prefix_polish_task
if task is None:
@@ -12777,10 +12793,26 @@ class SpeechPipeline:
asyncio.shield(task), timeout=_DICTATION_PREFIX_POLISH_WAIT_S
)
except TimeoutError:
- prefix_polish_failed = True
+ prefix_polish_stopped = True
+ task.cancel()
+ done, _pending = await asyncio.wait(
+ (task,), timeout=_DICTATION_POLISH_CANCEL_WAIT_S
+ )
+
+ def consume(done_task: asyncio.Task) -> None:
+ if done_task.cancelled():
+ return
+ if done_task.exception() is not None:
+ log.debug("Retired dictation formatter finished with an error")
+
+ if done:
+ consume(task)
+ else:
+ # No late result may mutate the captured prefix above.
+ task.add_done_callback(consume)
log.info(
"incremental dictation polish did not finish within %.1fs; "
- "formatting the whole text instead.",
+ "keeping its completed prefix and formatting the remaining tail.",
_DICTATION_PREFIX_POLISH_WAIT_S,
)
except asyncio.CancelledError:
@@ -13223,8 +13255,9 @@ class SpeechPipeline:
# sets an event and does not know when the stream actually stopped.
# A hangup skips it: nothing will be transcribed, and
# ``DictationCompleted`` follows immediately.
+ capture_closed_at = time.perf_counter()
if not hung_up:
- await self._publish_event(
+ await self._publish_dictation_event(
DictationTranscribing(source_layer="speech.dictation")
)
@@ -13340,6 +13373,10 @@ class SpeechPipeline:
f"final_windows:{final_window_count}",
f"final_windows_prefetched:{final_prefetched}",
f"release_wait_ms:{release_wait_ms}",
+ f"warmup_wait_ms:{round(warmup_wait_ms)}",
+ f"stt_queue_wait_ms:{round(stt_queue_wait_ms)}",
+ "post_recording_wait_ms:"
+ f"{round((time.perf_counter() - capture_closed_at) * 1000.0)}",
f"truncation_repairs:{truncation_repairs}",
f"tail_repairs:{tail_repairs}",
f"pause_trim_ms:{round(pause_trim_bytes * 1000 / bytes_per_second)}",
@@ -13564,6 +13601,26 @@ class SpeechPipeline:
return ""
return counts[0][0]
+ async def _publish_dictation_event(self, event: Any) -> bool:
+ """Deliver to healthy observers in order without waiting on a dead UI.
+
+ The bus fans out concurrently, so an unrelated stalled observer cannot
+ delay a healthy recipient. The bounded await still joins cancellation;
+ no publication task is left behind to reorder a later dictation.
+ """
+ try:
+ await asyncio.wait_for(
+ self._publish_event(event), timeout=_DICTATION_EVENT_TIMEOUT_S
+ )
+ except TimeoutError:
+ log.warning(
+ "Dictation event %s exceeded %.1fs; abandoning stalled observers.",
+ type(event).__name__,
+ _DICTATION_EVENT_TIMEOUT_S,
+ )
+ return False
+ return True
+
async def _finish_dictation(
self,
*,
@@ -13638,6 +13695,7 @@ class SpeechPipeline:
(``[dictation].keep_failed_audio``), which is what a later Restore
transcribes again.
"""
+ finish_started = time.perf_counter()
stt_error = normalize_stt_failure(stt_error)
cleaned = raw_text
removed_words = 0
@@ -13948,20 +14006,52 @@ class SpeechPipeline:
log.debug("dictation target resolution failed", exc_info=True)
resolved_target = "insert"
- # Publish the final transcript before inserting: the app's own fields,
- # the chat composer and the bar all listen for it, and they should
- # update even if insertion fails.
- try:
- await self._publish_event(
- DictationTranscript(
- source_layer="speech.dictation",
- text=cleaned,
- is_final=True,
- target=resolved_target,
+ formatting_ms = round((time.perf_counter() - finish_started) * 1000.0)
+ notification_ms = 0
+ insertion_ms = 0
+ final_notification_status = "complete"
+ insert_result = None
+
+ async def notify_final() -> None:
+ nonlocal notification_ms, final_notification_status
+ started = time.perf_counter()
+ try:
+ complete = await self._publish_dictation_event(
+ DictationTranscript(
+ source_layer="speech.dictation",
+ text=cleaned,
+ is_final=True,
+ target=resolved_target,
+ )
)
- )
+ if not complete:
+ final_notification_status = "observer_timeout"
+ except Exception:
+ final_notification_status = "failed"
+ log.warning("dictation final publish failed", exc_info=True)
+ finally:
+ notification_ms = round((time.perf_counter() - started) * 1000.0)
+
+ async def insert_external() -> None:
+ nonlocal insert_result, insertion_ms
+ started = time.perf_counter()
+ try:
+ insert_result = await asyncio.to_thread(self._insert_dictation, cleaned)
+ except Exception:
+ log.warning("dictation insertion failed", exc_info=True)
+ finally:
+ insertion_ms = round((time.perf_counter() - started) * 1000.0)
+
+ # Healthy UI clients receive the final text even if OS insertion fails.
+ # An external paste starts alongside notification: its destination must
+ # not drift while an unrelated UI observer is holding up the bus.
+ try:
+ if resolved_target == "insert" and cleaned.strip() and not hung_up:
+ await asyncio.gather(notify_final(), insert_external())
+ else:
+ await notify_final()
except Exception as exc: # noqa: BLE001
- log.debug("dictation final publish failed: %s", exc)
+ log.warning("dictation final delivery failed: %s", exc)
outcome_name = "chat"
detail = ""
@@ -13986,10 +14076,15 @@ class SpeechPipeline:
# an unexplained empty result after the user clearly spoke.
detail = rejected_detail
elif resolved_target == "insert":
- insert_result = await asyncio.to_thread(self._insert_dictation, cleaned)
- outcome_name = insert_result.status
- detail = insert_result.detail
- method = insert_result.method
+ if insert_result is None:
+ outcome_name = "failed"
+ detail = "The text could not be inserted."
+ if bool(getattr(cfg, "history_enabled", True)):
+ detail += " It is available in dictation history."
+ else:
+ outcome_name = insert_result.status
+ detail = insert_result.detail
+ method = insert_result.method
# A dictation that delivered SOME words and permanently lost others is
# not a success, whichever way the surviving fragment was delivered.
@@ -14065,12 +14160,21 @@ class SpeechPipeline:
dropped_audio_s,
)
+ delivery_audit = (
+ *stt_audit,
+ f"formatting_wait_ms:{formatting_ms}",
+ f"final_notification_wait_ms:{notification_ms}",
+ f"insertion_wait_ms:{insertion_ms}",
+ f"finish_wait_ms:{round((time.perf_counter() - finish_started) * 1000.0)}",
+ f"final_notification:{final_notification_status}",
+ )
+
# Mark the turn closed BEFORE the publish attempt: this flag answers
# "does the teardown still owe a terminal event", and a publish that
# raised is not a reason to fire a second, contradictory completion.
self._dictation_completion_published = True
try:
- await self._publish_event(
+ await self._publish_dictation_event(
DictationCompleted(
source_layer="speech.dictation",
text=cleaned,
@@ -14091,7 +14195,7 @@ class SpeechPipeline:
stt_latency_ms=max(0, int(stt_latency_ms)),
stt_calls=max(0, int(stt_calls)),
stt_errors=stt_errors,
- stt_audit=stt_audit,
+ stt_audit=delivery_audit,
audio_sample_rate_hz=max(0, int(audio_sample_rate_hz)),
audio_rms=max(0.0, float(audio_rms)),
audio_clipping_ratio=max(0.0, float(audio_clipping_ratio)),
@@ -14122,7 +14226,7 @@ class SpeechPipeline:
stt_latency_ms=max(0, int(stt_latency_ms)),
stt_calls=max(0, int(stt_calls)),
stt_errors=stt_errors,
- stt_audit=stt_audit,
+ stt_audit=delivery_audit,
audio_sample_rate_hz=max(0, int(audio_sample_rate_hz)),
audio_rms=max(0.0, float(audio_rms)),
audio_clipping_ratio=max(0.0, float(audio_clipping_ratio)),
diff --git a/tests/unit/core/test_dictation_paste_chord_custom.py b/tests/unit/core/test_dictation_paste_chord_custom.py
index 33d79a1a78..c6866e1669 100644
--- a/tests/unit/core/test_dictation_paste_chord_custom.py
+++ b/tests/unit/core/test_dictation_paste_chord_custom.py
@@ -2,10 +2,9 @@
Jarvis does not paste. It asks whatever application is in front to paste, by
sending a synthetic chord — and an application that does not bind that chord
-ignores it silently. That is fine for the curated chords (they mean "paste"
-somewhere real), and it is the whole problem for a chord the user recorded:
-there is no error and nothing to read back, so reporting ``inserted`` would be
-a lie in the one module whose entire docstring is about not lying.
+ignores it silently. A recorded chord without delivery evidence is therefore
+reported as ``paste_sent``. On Windows, this also applies to curated chords
+when the native clipboard observation mechanism is unavailable.
So the vocabulary widened in two directions at once and both are pinned here:
the config field accepts a combo, and the delivery report gains ``paste_sent``
@@ -14,6 +13,8 @@ for the case where the keystroke went out and the result is unknown.
from __future__ import annotations
+from types import SimpleNamespace
+
import pytest
import jarvis.platform
@@ -43,7 +44,14 @@ def _install_fake_clipboard(monkeypatch: pytest.MonkeyPatch, fake: object) -> No
would be ignored and the test would drive the REAL system clipboard of
whoever ran the suite. Patch the attribute the code reads.
"""
+ from jarvis.dictation import insert as insert_mod
+
monkeypatch.setattr(jarvis.platform, "clipboard", fake)
+ # The delayed-rendering offer bypasses the clipboard facade. Without this
+ # seam, a fake clipboard can coexist with a REAL native clipboard owner.
+ monkeypatch.setattr(insert_mod, "_clipboard_offer_factory", lambda: None)
+ monkeypatch.setattr(insert_mod, "_foreground_target", lambda: None)
+ monkeypatch.setattr(insert_mod, "_input_block_reason", lambda target: "")
# ----------------------------------------------------------------------
@@ -135,11 +143,19 @@ def test_paste_sent_is_part_of_the_outcome_vocabulary() -> None:
assert "paste_sent" in DICTATION_OUTCOMES
+ ("os_name", "platform"), [("nt", "win32"), ("posix", "darwin"), ("posix", "linux")],
+)
def test_a_custom_chord_reports_paste_sent_and_keeps_the_clipboard(
monkeypatch: pytest.MonkeyPatch,
+ os_name: str,
+ platform: str,
) -> None:
from jarvis.dictation import insert as insert_mod
+ monkeypatch.setattr(insert_mod, "os", SimpleNamespace(name=os_name))
+ monkeypatch.setattr(insert_mod, "sys", SimpleNamespace(platform=platform))
+
class _Clipboard:
def __init__(self) -> None:
self.content = "something the user had copied"
@@ -187,12 +203,20 @@ def test_a_custom_chord_reports_paste_sent_and_keeps_the_clipboard(
assert result.detail
-def test_a_curated_chord_still_reports_inserted_and_restores(
+ ("os_name", "platform"), [("nt", "win32"), ("posix", "darwin"), ("posix", "linux")],
+)
+def test_a_curated_chord_without_native_evidence_preserves_platform_contract(
monkeypatch: pytest.MonkeyPatch,
+ os_name: str,
+ platform: str,
) -> None:
- """The honesty change must not cost the normal path its clean report."""
+ """Windows retains unconfirmed text; existing POSIX delivery is unchanged."""
from jarvis.dictation import insert as insert_mod
+ monkeypatch.setattr(insert_mod, "os", SimpleNamespace(name=os_name))
+ monkeypatch.setattr(insert_mod, "sys", SimpleNamespace(platform=platform))
+
class _Clipboard:
def __init__(self) -> None:
self.content = "previous"
@@ -226,7 +250,14 @@ def test_a_curated_chord_still_reports_inserted_and_restores(
"dictated text", paste_chord="ctrl_v", delay_ms=0, delay_after_ms=0
)
- assert result.status == "inserted"
- assert result.clipboard_restored is True
- assert clipboard.writes == ["dictated text", "previous"]
- assert clipboard.content == "previous"
+ if platform == "win32":
+ assert result.status == "paste_sent"
+ assert result.clipboard_restored is False
+ assert result.clipboard_holds_text is True
+ assert clipboard.writes == ["dictated text"]
+ assert clipboard.content == "dictated text"
+ else:
+ assert result.status == "inserted"
+ assert result.clipboard_restored is True
+ assert clipboard.writes == ["dictated text", "previous"]
+ assert clipboard.content == "previous"
diff --git a/tests/unit/dictation/test_clipboard_offer_lifecycle.py b/tests/unit/dictation/test_clipboard_offer_lifecycle.py
new file mode 100644
index 0000000000..76a66b80e0
--- /dev/null
+++ b/tests/unit/dictation/test_clipboard_offer_lifecycle.py
@@ -0,0 +1,218 @@
+"""Clipboard offers must not outlive a timeout or leak their owning thread."""
+from __future__ import annotations
+
+import threading
+
+from jarvis.platform import clipboard_offer as mod
+
+
+class EventOffer(mod.ClipboardOffer):
+ def __init__(self, *, delayed=False):
+ super().__init__("test")
+ self.delayed = delayed
+ self.runs = 0
+ self.offered = False
+
+ def _pump(self):
+ self.runs += 1
+ if self.delayed:
+ self._stop_requested.wait(0.2)
+ if self._stop_requested.is_set():
+ return
+ self.offered = True
+ self._ok = True
+ self._ready.set()
+ self._stop_requested.wait(2)
+
+
+def test_start_and_stop_are_idempotent(monkeypatch):
+ monkeypatch.setattr(mod, "available", lambda: True)
+ offer = EventOffer()
+ assert offer.start()
+ thread = offer._thread
+ assert offer.start()
+ assert offer._thread is thread
+ assert offer.runs == 1
+ offer.stop()
+ offer.stop()
+ assert not thread.is_alive()
+ assert offer._thread is None
+ assert not offer.start()
+
+
+def test_start_timeout_cancels_before_delayed_worker_can_take_clipboard(monkeypatch):
+ monkeypatch.setattr(mod, "available", lambda: True)
+ offer = EventOffer(delayed=True)
+ assert not offer.start(timeout_s=0.001)
+ assert offer._stop_requested.is_set()
+ assert offer._done.wait(1)
+ assert not offer.offered
+ assert offer._thread is None
+
+
+def test_stop_timeout_keeps_handle_for_later_join(monkeypatch):
+ monkeypatch.setattr(mod, "available", lambda: True)
+ release = threading.Event()
+ offer = EventOffer()
+
+ def pump():
+ offer._ok = True
+ offer._ready.set()
+ release.wait(1)
+
+ offer._pump = pump
+ try:
+ assert offer.start()
+ thread = offer._thread
+ offer.stop(timeout_s=0)
+ assert offer._thread is thread
+ finally:
+ release.set()
+ offer.stop()
+ assert not thread.is_alive()
+ assert offer._thread is None
+
+
+def test_read_wait_wakes_when_offer_loses_ownership():
+ offer = EventOffer()
+ offer.lost_ownership = True
+ assert offer.wait_for_read(exclude_pids=set(), after_s=0, timeout_s=10) is None
+
+
+class OwnedClipboard:
+ def __init__(self, owner):
+ self.owner = owner
+ self.locked = False
+ self.emptied = False
+
+ def OpenClipboard(self, hwnd):
+ self.locked = True
+ return True
+
+ def GetClipboardOwner(self):
+ assert self.locked
+ return self.owner
+
+ def EmptyClipboard(self):
+ assert self.locked
+ self.emptied = True
+ return True
+
+ def CloseClipboard(self):
+ self.locked = False
+
+
+def test_restore_cannot_overwrite_a_new_owner_even_with_same_text():
+ clipboard = OwnedClipboard(owner=456)
+ offer = EventOffer()
+ offer._restore_text = "previous"
+ writes = []
+ offer._restore_owned_text(clipboard, 123, lambda text=None: writes.append(text))
+ assert not writes
+ assert not clipboard.emptied
+ assert not clipboard.locked
+ assert not offer.restored
+
+
+def test_restore_checks_and_writes_under_the_same_lock():
+ clipboard = OwnedClipboard(owner=123)
+ offer = EventOffer()
+ offer._restore_text = "previous"
+ writes = []
+
+ def render(text=None):
+ assert clipboard.locked
+ writes.append(text)
+ return True
+
+ offer._restore_owned_text(clipboard, 123, render)
+ assert writes == ["previous"]
+ assert offer.restored
+ assert offer._text == "previous"
+ assert not clipboard.locked
+
+
+def test_failed_restore_republishes_transcript_before_unlocking():
+ clipboard = OwnedClipboard(owner=123)
+ offer = EventOffer()
+ offer._restore_text = "previous"
+ writes = []
+
+ def render(text=None):
+ assert clipboard.locked
+ writes.append(text)
+ return text is None
+
+ offer._restore_owned_text(clipboard, 123, render)
+ assert writes == ["previous", None]
+ assert not offer.restored
+ assert not clipboard.locked
+
+
+class NativeFunction:
+ """ctypes-callable stand-in retaining the signatures assigned by the pump."""
+ def __init__(self, call=lambda *args: 0):
+ self.call = call
+ self.argtypes = None
+ self.restype = None
+
+ def __call__(self, *args):
+ return self.call(*args)
+
+
+class NativeLibrary:
+ def __init__(self, **calls):
+ self.calls = {name: NativeFunction(call) for name, call in calls.items()}
+
+ def __getattr__(self, name):
+ return self.calls.setdefault(name, NativeFunction())
+
+
+def test_stop_dispatched_during_message_drain_never_enters_one_second_wait(monkeypatch):
+ """Exercise the actual native pump with a stop arriving inside DispatchMessage."""
+ import ctypes
+
+ offer = mod.ClipboardOffer("test")
+ waits = []
+ callbacks = []
+ drained = False
+
+ def register_class(pointer):
+ callbacks.append(pointer._obj.lpfnWndProc)
+ return 1
+
+ def peek_message(pointer, *args):
+ nonlocal drained
+ if drained:
+ return 0
+ drained = True
+ pointer._obj.message = mod._WM_APP_STOP
+ return 1
+
+ def dispatch_message(pointer):
+ return callbacks[0](123, pointer._obj.message, 0, 0)
+
+ user32 = NativeLibrary(
+ RegisterClassW=register_class,
+ CreateWindowExW=lambda *args: 123,
+ OpenClipboard=lambda *args: 1,
+ PeekMessageW=peek_message,
+ DispatchMessageW=dispatch_message,
+ IsWindow=lambda *args: 1,
+ MsgWaitForMultipleObjectsEx=lambda *args: waits.append(args[2]),
+ )
+ kernel32 = NativeLibrary(GetModuleHandleW=lambda *args: 1)
+ monkeypatch.setattr(mod, "available", lambda: True)
+ monkeypatch.setattr(ctypes, "WINFUNCTYPE", ctypes.CFUNCTYPE, raising=False)
+ monkeypatch.setattr(ctypes, "set_last_error", lambda value: None, raising=False)
+ monkeypatch.setattr(ctypes, "get_last_error", lambda: 0, raising=False)
+ monkeypatch.setattr(
+ ctypes, "WinDLL", lambda name, **kwargs: user32 if name == "user32" else kernel32,
+ raising=False,
+ )
+
+ offer._pump()
+
+ assert offer._stop_requested.is_set()
+ assert waits == []
+ assert offer._hwnd == 0
diff --git a/tests/unit/dictation/test_delivery_latency.py b/tests/unit/dictation/test_delivery_latency.py
new file mode 100644
index 0000000000..6f53d7622a
--- /dev/null
+++ b/tests/unit/dictation/test_delivery_latency.py
@@ -0,0 +1,130 @@
+"""Finished text is independent of a stalled observer, with ordered recovery."""
+
+from __future__ import annotations
+
+import asyncio
+import threading
+from types import SimpleNamespace
+
+import pytest
+
+from jarvis.core.bus import EventBus
+from jarvis.core.config import DictationConfig
+from jarvis.core.events import DictationCompleted, DictationTranscript
+from jarvis.speech import pipeline as pipeline_mod
+from jarvis.speech.pipeline import SpeechPipeline
+
+
+def _pipeline(monkeypatch):
+ monkeypatch.setattr(pipeline_mod, "_DICTATION_EVENT_TIMEOUT_S", 0.03)
+ pipe = SpeechPipeline.__new__(SpeechPipeline)
+ pipe._bus = EventBus()
+ pipe._dictation_cfg = DictationConfig(polish=False)
+ recorded = []
+
+ async def record(**values):
+ recorded.append(values)
+
+ pipe._record_dictation = record
+ return pipe, recorded
+
+
+async def _finish(pipe, *, target="chat"):
+ return await pipe._finish_dictation(
+ raw_text="These words must arrive.", language="en", duration_s=2,
+ target=target, hung_up=False,
+ )
+
+
+async def test_healthy_chat_receives_final_and_completion_beside_stalled_observer(monkeypatch):
+ pipe, recorded = _pipeline(monkeypatch)
+ healthy = []
+ cancelled = []
+
+ async def receive(event):
+ healthy.append(event)
+
+ async def stalled(event):
+ try:
+ await asyncio.Event().wait()
+ finally:
+ cancelled.append(type(event))
+
+ pipe._bus.subscribe_all(receive)
+ pipe._bus.subscribe_all(stalled)
+ result = await asyncio.wait_for(_finish(pipe), timeout=1)
+
+ assert result == "These words must arrive."
+ assert [type(event) for event in healthy] == [DictationTranscript, DictationCompleted]
+ assert cancelled == [DictationTranscript, DictationCompleted]
+ assert recorded[0]["cleaned"] == result
+ assert "final_notification:observer_timeout" in recorded[0]["stt_audit"]
+ assert recorded[0]["stt_audit"] == healthy[-1].stt_audit
+
+
+async def test_external_insertion_does_not_wait_for_notification(monkeypatch):
+ pipe, recorded = _pipeline(monkeypatch)
+ inserted = threading.Event()
+ observer_started = asyncio.Event()
+ observer_release = asyncio.Event()
+ monkeypatch.setattr(pipeline_mod, "_DICTATION_EVENT_TIMEOUT_S", 1)
+
+ async def stalled(event):
+ if isinstance(event, DictationTranscript):
+ observer_started.set()
+ await observer_release.wait()
+
+ def insert(text):
+ inserted.set()
+ return SimpleNamespace(status="inserted", detail="", method="fake")
+
+ pipe._bus.subscribe_all(stalled)
+ pipe._insert_dictation = insert
+ task = asyncio.create_task(_finish(pipe, target="insert"))
+ try:
+ await asyncio.wait_for(observer_started.wait(), timeout=1)
+ assert await asyncio.to_thread(inserted.wait, 0.5)
+ assert not task.done()
+ finally:
+ observer_release.set()
+ await asyncio.wait_for(task, timeout=1)
+ assert recorded[0]["outcome_name"] == "inserted"
+
+
+async def test_an_insertion_exception_still_preserves_text_and_terminal_event(monkeypatch):
+ pipe, recorded = _pipeline(monkeypatch)
+ events = []
+
+ async def receive(event):
+ events.append(event)
+
+ def insert(text):
+ raise RuntimeError("fake unavailable desktop")
+
+ pipe._bus.subscribe_all(receive)
+ pipe._insert_dictation = insert
+ result = await _finish(pipe, target="insert")
+ assert result == "These words must arrive."
+ assert events[-1].outcome == "failed"
+ assert recorded[0]["cleaned"] == result
+ assert recorded[0]["outcome_name"] == "failed"
+
+
+async def test_cancellation_retires_notification_before_next_turn(monkeypatch):
+ pipe, _recorded = _pipeline(monkeypatch)
+ entered, retired = asyncio.Event(), asyncio.Event()
+
+ async def stalled(event):
+ entered.set()
+ try:
+ await asyncio.Event().wait()
+ finally:
+ retired.set()
+
+ pipe._bus.subscribe_all(stalled)
+ task = asyncio.create_task(_finish(pipe))
+ await entered.wait()
+ task.cancel()
+ with pytest.raises(asyncio.CancelledError):
+ await task
+ assert retired.is_set()
diff --git a/tests/unit/dictation/test_incremental_polish.py b/tests/unit/dictation/test_incremental_polish.py
index 0603812edb..14447bb544 100644
--- a/tests/unit/dictation/test_incremental_polish.py
+++ b/tests/unit/dictation/test_incremental_polish.py
@@ -285,3 +285,48 @@ async def test_translation_keeps_the_whole_text_pass(
assert calls[0].get("translate_to") == "en"
assert not calls[0].get("preceding_text")
assert _audit(events, "polish_mode") == "whole"
+
+
+async def test_stalled_prefix_worker_is_retired_without_reformatting_completed_text(
+ _no_preview: None, monkeypatch: pytest.MonkeyPatch
+) -> None:
+ import jarvis.dictation.polish as polish
+ import jarvis.speech.pipeline as pipeline_mod
+
+ monkeypatch.setattr(pipeline_mod, "_DICTATION_PREFIX_POLISH_WAIT_S", 0.03)
+ blocked, retired = asyncio.Event(), asyncio.Event()
+ calls: list[dict[str, Any]] = []
+
+ async def format_piece(raw: str, **kwargs: Any) -> PolishOutcome:
+ calls.append({"raw": raw, **kwargs})
+ if len(calls) == 2:
+ blocked.set()
+ try:
+ await asyncio.Event().wait()
+ finally:
+ retired.set()
+ elif len(calls) > 2:
+ assert retired.is_set(), "replacement overlapped the retired formatter"
+ return PolishOutcome(text=raw.upper(), status="applied")
+
+ monkeypatch.setattr(polish, "polish_transcript", format_piece)
+ mic = _FakeMic(_voiced(12.0))
+ pipe, events = _session_pipeline(_ScriptedSTT(), mic)
+ task = asyncio.create_task(pipe._dictation_session())
+ try:
+ await asyncio.wait_for(mic.delivered.wait(), timeout=3)
+ await asyncio.wait_for(blocked.wait(), timeout=3)
+ pipe._dictation_stop_event.set()
+ await asyncio.wait_for(task, timeout=3)
+ finally:
+ if not task.done():
+ task.cancel()
+ with contextlib.suppress(asyncio.CancelledError):
+ await task
+
+ assert retired.is_set()
+ assert len(calls) == 3
+ assert "alpha" not in calls[-1]["raw"]
+ assert calls[-1]["preceding_text"].startswith("ALPHA")
+ assert _final(events).text.count("ALPHA") == 1
+ assert _audit(events, "polish_mode") == "incremental"
diff --git a/tests/unit/dictation/test_insert.py b/tests/unit/dictation/test_insert.py
index cf82ad3421..710cdebe3b 100644
--- a/tests/unit/dictation/test_insert.py
+++ b/tests/unit/dictation/test_insert.py
@@ -8,6 +8,7 @@ the clipboard before any keystroke, and it stays there when the paste fails.**
from __future__ import annotations
import sys
+from types import SimpleNamespace
import pytest
@@ -75,6 +76,9 @@ def wired(monkeypatch: pytest.MonkeyPatch):
# These tests pin the PLAIN chord path (every OS). The Windows path that
# watches who reads the clipboard has its own file: test_insert_verified.py.
monkeypatch.setattr(insert_mod, "_clipboard_offer_factory", lambda: None)
+ # Preserve coverage of the existing POSIX plain route. Windows degrades
+ # honestly to paste_sent if its read-observation helper is unavailable.
+ monkeypatch.setattr(insert_mod, "os", SimpleNamespace(name="posix"))
return clipboard, actuator
diff --git a/tests/unit/dictation/test_insert_verified.py b/tests/unit/dictation/test_insert_verified.py
index 0b64f0840c..e22e7e550b 100644
--- a/tests/unit/dictation/test_insert_verified.py
+++ b/tests/unit/dictation/test_insert_verified.py
@@ -1,310 +1,244 @@
-"""The Windows paste path that watches WHO reads the clipboard.
-
-A synthetic paste chord is a request, not a paste: an xterm.js terminal in a
-Tauri/Electron app swallows Ctrl+V, and a WebView that pastes through an async
-bridge may read the clipboard long after a 120 ms "restore the old clipboard"
-timer has fired. The delayed-rendering offer turns both into observations. These
-tests drive ``_insert_windows_verified`` with a fake offer so each host state
-(sighted / blind) and each route (chord, cascade, typing) is pinned without a
-real clipboard. They run on every OS — the platform gate is the factory, and
-the factory is what is swapped.
-"""
-
+"""A paste timeout must never duplicate input or erase an unconfirmed transcript."""
from __future__ import annotations
from dataclasses import dataclass, field
+from types import SimpleNamespace
import pytest
from jarvis.dictation import insert as insert_mod
from jarvis.dictation.insert import insert_text
-from .test_insert import FakeActuator, FakeClipboard # noqa: TID252 — sibling fakes
+from .test_insert import FakeActuator, FakeClipboard # noqa: TID252
-SELF_PID = 4242
-WATCHER_PID = 77
-TARGET_PID = 9001
+TARGET = (123, 9001)
@dataclass
class FakeRead:
pid: int
- exe: str
- at: float
+ exe: str = ""
+ at: float = 0.1
observed: str = "render"
@dataclass
class FakeOffer:
- """Scripted clipboard offer: who reads, and when (offer-relative seconds)."""
-
text: str
script: list[FakeRead] = field(default_factory=list)
- started: bool = False
stopped: bool = False
- clock: float = 0.0
+ lost_ownership: bool = False
+ restored: bool = False
+ clipboard: FakeClipboard | None = None
+ waits: list[float] = field(default_factory=list)
- def start(self) -> bool:
- self.started = True
+ def start(self):
return True
- def reads(self) -> list[FakeRead]:
- return [r for r in self.script if r.at <= self.clock]
+ def reads(self):
+ return [r for r in self.script if r.at <= 0.05]
- def elapsed(self) -> float:
- return self.clock
+ def elapsed(self):
+ return 0.05
def wait_for_read(self, *, exclude_pids, after_s, timeout_s):
- self.clock = after_s + timeout_s
- for read in self.script:
- if after_s <= read.at <= self.clock and read.pid not in exclude_pids:
- return read
- return None
+ self.waits.append(timeout_s)
+ return next((r for r in self.script if after_s <= r.at <= after_s + timeout_s
+ and r.pid not in exclude_pids), None)
- def stop(self) -> None:
+ def stop(self, *, restore_text=None):
self.stopped = True
-
-
-class OfferFactory:
- """Hands out one scripted offer per chord attempt, in order."""
-
- def __init__(self, *scripts: list[FakeRead]) -> None:
- self.scripts = list(scripts)
- self.offers: list[FakeOffer] = []
-
- def __call__(self, text: str) -> FakeOffer:
- script = self.scripts.pop(0) if self.scripts else []
- offer = FakeOffer(text, script)
- self.offers.append(offer)
- return offer
+ if restore_text is not None and not self.lost_ownership:
+ self.restored = self.clipboard.write_text(restore_text)
@pytest.fixture()
-def verified(monkeypatch: pytest.MonkeyPatch):
- """insert_text with the verified path active and every OS call faked."""
- clipboard = FakeClipboard()
- actuator = FakeActuator()
-
+def verified(monkeypatch):
+ clipboard, actuator = FakeClipboard(), FakeActuator()
import jarvis.platform.clipboard as real_clipboard
-
monkeypatch.setattr(real_clipboard, "read_text", clipboard.read_text)
monkeypatch.setattr(real_clipboard, "write_text", clipboard.write_text)
monkeypatch.setattr(
- insert_mod, "describe_target", lambda: insert_mod.TargetReport(True, "", "")
+ insert_mod, "describe_target", lambda: insert_mod.TargetReport(True, "", ""),
)
monkeypatch.setattr("jarvis.cu.actuate.get_actuator", lambda: actuator)
- monkeypatch.setattr(insert_mod.time, "sleep", lambda _s: None)
- monkeypatch.setattr(insert_mod, "_foreground_exe", lambda: "target.exe")
- monkeypatch.setattr("os.getpid", lambda: SELF_PID)
- monkeypatch.setattr(insert_mod, "_LEARNED_ROUTES", {})
-
- # The settle window advances the fake clock so early reads become visible.
- def install(factory: OfferFactory) -> None:
- def make(text: str) -> FakeOffer:
- offer = factory(text)
- offer.clock = 0.05
- return offer
-
- monkeypatch.setattr(insert_mod, "_clipboard_offer_factory", lambda: make)
-
- restores: list[tuple[str | None, str, float]] = []
- monkeypatch.setattr(
- insert_mod,
- "_restore_later",
- lambda cb, previous, text, grace: restores.append((previous, text, grace)),
- )
- return clipboard, actuator, install, restores
-
-
-# --------------------------------------------------------------------------
-# Sighted host: nobody read during the settle window, so silence is proof.
-# --------------------------------------------------------------------------
-
+ monkeypatch.setattr(insert_mod.time, "sleep", lambda _: None)
+ monkeypatch.setattr(insert_mod, "_foreground_target", lambda: TARGET)
+ monkeypatch.setattr(insert_mod, "_input_block_reason", lambda _: "")
+ monkeypatch.setattr(insert_mod, "os", SimpleNamespace(name="nt", getpid=lambda: 4242))
+ offers = []
+
+ def install(*reads):
+ offer = FakeOffer("dictated text", list(reads))
+ offer.clipboard = clipboard
+ offers.append(offer)
+ monkeypatch.setattr(insert_mod, "_clipboard_offer_factory", lambda: lambda text: offer)
+ return offer
-def test_sighted_paste_is_proven_by_the_target_reading(verified) -> None:
- clipboard, actuator, install, restores = verified
- factory = OfferFactory([FakeRead(TARGET_PID, "app.exe", at=0.1)])
- install(factory)
+ return clipboard, actuator, install
- result = insert_text("dictated text", paste_chord="ctrl_v")
+def test_target_render_confirms_one_paste_and_restores(verified):
+ clipboard, actuator, install = verified
+ offer = install(FakeRead(TARGET[1]))
+ result = insert_text("dictated text")
assert result.status == "inserted"
- assert result.method == "clipboard+ctrl_v"
- assert actuator.combos == [["ctrl", "v"]]
- # Restored right away — the read was seen, no need to wait.
- assert result.clipboard_restored is True
+ assert result.clipboard_restored
assert clipboard.content == "previous contents"
- assert restores == []
- assert insert_mod._LEARNED_ROUTES == {"target.exe": "ctrl_v"}
-
-
-def test_sighted_silence_cascades_to_the_next_chord(verified) -> None:
- """Ctrl+V unanswered, Ctrl+Shift+V answered: one paste, the right one."""
- _clipboard, actuator, install, _restores = verified
- factory = OfferFactory([], [FakeRead(TARGET_PID, "app.exe", at=0.2)])
- install(factory)
-
- result = insert_text("dictated text", paste_chord="ctrl_v")
-
- assert result.status == "inserted"
- assert result.method == "clipboard+ctrl_shift_v"
- assert actuator.combos == [["ctrl", "v"], ["ctrl", "shift", "v"]]
- assert all(o.stopped for o in factory.offers)
- assert insert_mod._LEARNED_ROUTES == {"target.exe": "ctrl_shift_v"}
-
-
-def test_sighted_no_chord_answered_types_with_soft_newlines(verified) -> None:
- """An app that binds no paste chord still gets the words — typed."""
- clipboard, actuator, install, _restores = verified
- install(OfferFactory([], [], []))
-
- result = insert_text("first line\nsecond line", paste_chord="ctrl_v")
-
- assert result.status == "inserted"
- assert result.method == "type"
- assert actuator.combos == [
- ["ctrl", "v"],
- ["ctrl", "shift", "v"],
- ["shift", "insert"],
- ["shift", "enter"], # the line break, never a submitting Enter
- ]
- assert actuator.typed == ["first line", "second line"]
- # The text stays on the clipboard: if the keystrokes landed nowhere it is
- # the user's way back.
- assert result.clipboard_holds_text is True
- assert clipboard.content == "first line\nsecond line"
- assert insert_mod._LEARNED_ROUTES == {"target.exe": "type"}
-
-
-def test_learned_type_route_skips_the_chords(verified) -> None:
- _clipboard, actuator, install, _restores = verified
- insert_mod._LEARNED_ROUTES["target.exe"] = "type"
- install(OfferFactory())
-
- result = insert_text("again", paste_chord="ctrl_v")
-
- assert result.method == "type"
- assert actuator.combos == []
- assert actuator.typed == ["again"]
+ assert actuator.combos == [["ctrl", "v"]]
+ assert offer.stopped
+
+
+ [],
+ [FakeRead(TARGET[1], at=3.0)],
+ [FakeRead(77)],
+ [FakeRead(TARGET[1], observed="open")],
+])
+def test_missing_late_unrelated_or_open_evidence_never_retries_or_erases(verified, reads):
+ clipboard, actuator, install = verified
+ offer = install(*reads)
+ result = insert_text("dictated text")
+ assert result.status == "paste_sent"
+ assert result.clipboard_holds_text
+ assert not result.clipboard_restored
+ assert clipboard.writes == ["dictated text"]
+ assert actuator.combos == [["ctrl", "v"]]
+ assert actuator.typed == []
+ assert sum(offer.waits) <= insert_mod.PASTE_READ_WAIT_S
+ assert offer.stopped
-def test_learned_chord_goes_first(verified) -> None:
- _clipboard, actuator, install, _restores = verified
- insert_mod._LEARNED_ROUTES["target.exe"] = "shift_insert"
- install(OfferFactory([FakeRead(TARGET_PID, "app.exe", at=0.1)]))
+def test_early_watcher_returns_without_waiting_and_keeps_text(verified):
+ clipboard, actuator, install = verified
+ offer = install(FakeRead(77, at=0.004))
+ result = insert_text("dictated text")
+ assert result.status == "paste_sent"
+ assert offer.waits == []
+ assert clipboard.content == "dictated text"
+ assert actuator.combos == [["ctrl", "v"]]
- result = insert_text("again", paste_chord="ctrl_v")
+def test_failure_in_one_field_cannot_change_route_for_later_field(verified):
+ _, actuator, install = verified
+ install()
+ insert_text("first")
+ install(FakeRead(TARGET[1]))
+ result = insert_text("second", paste_chord="shift_insert")
+ assert actuator.combos == [["ctrl", "v"], ["shift", "insert"]]
+ assert actuator.typed == []
assert result.method == "clipboard+shift_insert"
- assert actuator.combos == [["shift", "insert"]]
-
-
-# --------------------------------------------------------------------------
-# Blind host: a watcher consumed the one render, silence proves nothing.
-# --------------------------------------------------------------------------
-
-
-def test_blind_host_sends_one_chord_and_restores_late(verified) -> None:
- """RDP clipboard sync read at arm time: no cascade, no typing, no 120 ms timer."""
- clipboard, actuator, install, restores = verified
- install(OfferFactory([FakeRead(WATCHER_PID, "msrdc.exe", at=0.004)]))
- result = insert_text("dictated text", paste_chord="ctrl_v")
- assert result.status == "inserted"
- assert result.method == "clipboard+ctrl_v"
- assert actuator.combos == [["ctrl", "v"]] # exactly one — never a double paste
- assert actuator.typed == []
- # Not restored on the spot; scheduled with the long grace instead.
- assert result.clipboard_restored is False
- assert result.clipboard_holds_text is True
- assert clipboard.content == "dictated text"
- assert restores == [("previous contents", "dictated text", insert_mod.RESTORE_GRACE_S)]
- assert insert_mod._LEARNED_ROUTES == {}
+def test_custom_chord_remains_exactly_the_configured_chord(verified):
+ _, actuator, install = verified
+ install()
+ result = insert_text("dictated text", paste_chord="ctrl+alt+insert")
+ assert result.status == "paste_sent"
+ assert actuator.combos == [["ctrl", "alt", "insert"]]
-def test_slow_watcher_after_the_chord_also_means_blind(verified) -> None:
- _clipboard, actuator, install, restores = verified
- install(OfferFactory([FakeRead(WATCHER_PID, "svchost.exe", at=0.2)]))
+def test_native_probe_failure_after_sending_cannot_send_again(verified):
+ clipboard, actuator, install = verified
+ offer = install()
- result = insert_text("dictated text", paste_chord="ctrl_v")
+ def fail(**kwargs):
+ raise OSError("probe unavailable")
- assert result.status == "inserted"
+ offer.wait_for_read = fail
+ result = insert_text("dictated text")
+ assert result.status == "paste_sent"
assert actuator.combos == [["ctrl", "v"]]
- assert len(restores) == 1
-
-
-def test_blind_host_still_proves_a_paste_by_polling(verified) -> None:
- """The watcher cached the text, but the target was SEEN holding the clipboard."""
- clipboard, actuator, install, restores = verified
- install(
- OfferFactory(
- [
- FakeRead(WATCHER_PID, "msrdc.exe", at=0.004),
- FakeRead(TARGET_PID, "webview.exe", at=0.3, observed="open"),
- ]
- )
- )
+ assert clipboard.content == "dictated text"
+ assert offer.stopped
- result = insert_text("dictated text", paste_chord="ctrl_v")
- assert result.status == "inserted"
+def test_failed_offer_uses_one_plain_chord_but_keeps_windows_clipboard(verified):
+ clipboard, actuator, install = verified
+ offer = install()
+ offer.start = lambda: False
+ result = insert_text("dictated text")
+ assert result.status == "paste_sent"
assert actuator.combos == [["ctrl", "v"]]
- assert result.clipboard_restored is True
- assert clipboard.content == "previous contents"
- assert restores == []
-
-
-def test_blind_custom_chord_stays_honest(verified) -> None:
- """A user-recorded chord with no evidence is ``paste_sent``, not ``inserted``."""
- _clipboard, _actuator, install, _restores = verified
- install(OfferFactory([FakeRead(WATCHER_PID, "msrdc.exe", at=0.004)]))
+ assert clipboard.content == "dictated text"
+ assert offer.stopped
- result = insert_text("dictated text", paste_chord="ctrl+alt+insert")
- assert result.status == "paste_sent"
- assert result.method == "clipboard+ctrl+alt+insert"
+def test_changed_focus_or_held_modifier_stops_before_input(verified, monkeypatch):
+ clipboard, actuator, install = verified
+ offer = install()
+ monkeypatch.setattr(insert_mod, "_input_block_reason", lambda _: "A modifier is held.")
+ result = insert_text("dictated text")
+ assert result.status == "clipboard_only"
+ assert actuator.combos == []
+ assert clipboard.content == "dictated text"
+ assert offer.stopped
-# --------------------------------------------------------------------------
-# Degradations
-# --------------------------------------------------------------------------
+def test_user_copy_during_confirmed_paste_is_not_overwritten(verified):
+ clipboard, _, install = verified
+ offer = install(FakeRead(TARGET[1]))
+ def stop(*, restore_text=None):
+ clipboard.content = "user copy"
+ offer.lost_ownership = True
+ offer.stop = stop
+ result = insert_text("dictated text")
+ assert not result.clipboard_restored
+ assert clipboard.content == "user copy"
-def test_offer_that_cannot_take_the_clipboard_uses_the_plain_path(verified) -> None:
- clipboard, actuator, install, _restores = verified
+def test_unreadable_clipboard_prevents_restore(monkeypatch):
+ clipboard = FakeClipboard(initial=None)
+ assert not insert_mod._clipboard_still_holds(clipboard, "dictated text")
- class Refusing(FakeOffer):
- def start(self) -> bool:
- return False
+ def fail():
+ raise OSError("clipboard busy")
- install(OfferFactory())
- monkeypatch_factory = lambda: Refusing # noqa: E731
- insert_mod._clipboard_offer_factory = monkeypatch_factory # type: ignore[assignment]
+ monkeypatch.setattr(clipboard, "read_text", fail)
+ assert not insert_mod._clipboard_still_holds(clipboard, "dictated text")
- result = insert_text("dictated text", paste_chord="ctrl_v")
- assert result.status == "inserted"
- assert actuator.combos == [["ctrl", "v"]]
+def test_overlapping_delivery_does_not_replace_active_clipboard(verified):
+ clipboard, actuator, install = verified
+ install()
+ insert_mod._INSERT_LOCK.acquire()
+ try:
+ result = insert_text("other text")
+ finally:
+ insert_mod._INSERT_LOCK.release()
+ assert result.status == "unavailable"
assert clipboard.content == "previous contents"
+ assert actuator.combos == []
-def test_restore_later_only_restores_our_own_text(monkeypatch: pytest.MonkeyPatch) -> None:
- """A copy the user makes during the grace window is never overwritten."""
- clipboard = FakeClipboard(initial="dictated text")
- monkeypatch.setattr(insert_mod.time, "sleep", lambda _s: None)
+def test_explicit_typing_is_still_available(verified):
+ _, actuator, install = verified
+ install()
+ result = insert_text("dictated text", method="type")
+ assert result.method == "type"
+ assert actuator.typed == ["dictated text"]
+ assert actuator.combos == []
+
- import threading
[email protected]("held_vk", [0x10, 0x11, 0x12, 0x5B, 0x5C])
+def test_native_modifier_snapshot_refuses_modified_paste(monkeypatch, held_vk):
+ import ctypes
- class Immediate(threading.Thread):
- def start(self) -> None: # run inline for the test
- self.run()
+ def key_state(vk):
+ return 0x8000 if vk == held_vk else 0
- monkeypatch.setattr(insert_mod.threading, "Thread", Immediate)
+ monkeypatch.setattr(insert_mod, "os", SimpleNamespace(name="nt"))
+ monkeypatch.setattr(insert_mod, "sys", SimpleNamespace(platform="win32"))
+ monkeypatch.setattr(insert_mod, "_foreground_target", lambda: TARGET)
+ monkeypatch.setattr(
+ ctypes, "WinDLL", lambda *a, **kw: SimpleNamespace(GetAsyncKeyState=key_state),
+ raising=False,
+ )
+ assert "modifier" in insert_mod._input_block_reason(TARGET)
- insert_mod._restore_later(clipboard, "previous", "dictated text", 0.0)
- assert clipboard.content == "previous"
- clipboard.content = "something the user copied"
- insert_mod._restore_later(clipboard, "previous", "dictated text", 0.0)
- assert clipboard.content == "something the user copied"
+def test_foreground_change_prevents_keyboard_probe_and_paste(monkeypatch):
+ monkeypatch.setattr(insert_mod, "os", SimpleNamespace(name="nt"))
+ monkeypatch.setattr(insert_mod, "sys", SimpleNamespace(platform="win32"))
+ monkeypatch.setattr(insert_mod, "_foreground_target", lambda: (124, 9002))
+ assert "window changed" in insert_mod._input_block_reason(TARGET)