Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.
Python agent-orchestrationai-agentai-coding-assistantclaude-codecoding-agentcomputer-usedesktop-applocal-firstmcppersonal-jarvisvoice-assistantvoice-coding
fix(realtime): recover a dead speaker and label native voice as requested (BUG-108, BUG-086)
Haraldde3786561b5f
diff --git a/docs/BUGS.md b/docs/BUGS.mdindex 9fbf4c2c01..1987be12d2 100644--- a/docs/BUGS.md+++ b/docs/BUGS.md@@ -5996,15 +5996,17 @@ deterministic wrong voice is worse than an occasional drift. Guards:`::test_generative_voice_provider_mute_still_falls_back_to_surface_tts`,`::test_generative_voice_provider_keeps_native_readback_on_the_browser_surface`.+**Label honesty FIXED 2026-08-20.** `SpeechSpoken.voice_verified` is False+for native generative audio (the pin we requested, not a heard speaker)+and True for classic TTS. The recorder, the markdown export, and the+Transcription badge say "requested" when it is only a pin.+**Still OPEN.**- Plain (non-delegate) native turns can still drift despite the standingsession clause — inherent to a generative renderer; a full fix would- abandon native audio entirely.-- Label honesty: `voice_name` should distinguish "requested" from- "verified" (classic TTS engines are verified by construction; native- realtime audio is only ever requested). Until then, treat realtime- `voice_name` as the pin, not as evidence.+ abandon native audio entirely. Do not re-add an immediate surface claim+ — a deterministic wrong voice is worse than an occasional drift.**Class rule.** A generative native-audio renderer holds its voice only asfirmly as EVERY text it is told to speak reminds it to. Any new prompt@@ -7424,7 +7426,7 @@ browser, phone, telephony) needs a deterministic explicit-ask gate, notjust prompt discouragement — the model reaches for the most concrete toolit sees, and a question must never be answered by performing.-## BUG-108: local speaker death (PortAudio -9986) is misclassified as provider transport death — three answers voiced into a dead output, all reported healthy (CRITICAL, OPEN 2026-07-21)+## BUG-108: local speaker death (PortAudio -9986) is misclassified as provider transport death — three answers voiced into a dead output, all reported healthy (CRITICAL, FIXED 2026-08-20)**Symptom (live 2026-07-21 11:53, session `290b610c`, first call after the1b191b88 deploy/restart at 11:52).** The user asked for the best private@@ -7486,6 +7488,22 @@ The user hung up via hotkey at 11:55:11.context was lost — otherwise it invents one ("Bora Bora") and assertsit as shared history, which is worse than admitting the gap.+**Fix (2026-08-20).**++1. `is_local_output_error` classifies PortAudio device death. The pump+ does not rebuild the provider for it. `_complete_surface_turn` and+ `_emit_audio` catch it, zero the heard-sample counter, and speak an+ honest "speaker disappeared" line.+2. `AudioPlayer.recover_output_device` walks auto-headset order skipping+ the dead index; `_open_output_stream` retries on that device so the+ current turn can land on built-in speakers.+3. A turn whose playback wrote nothing no longer publishes SpeechSpoken.+4. A seed-less rebuild (`history_lost=True`) injects a standing order:+ this is a fresh session, do not invent prior topics.++**Guards.** `tests/unit/audio/test_player_output_fallback.py`,+`tests/unit/realtime/test_local_output.py`.+Related: BUG-102 (topology refresh), BUG-104 (rebuild seed), BUG-106(the answer itself also missed the Gulfstream G800, in service since2025 — fact-quality family, tracked there).@@ -10524,7 +10542,7 @@ while it deliberates; bound the work, not the listening.`test_catchup_budget_ignores_garbage_values`,`test_vosk_provider_declares_both_budgets`).-## BUG-151: libvosk crashes the whole process under concurrent recognizer churn — the wake engine's parallel verify design hits it (HIGH, OPEN 2026-08-19)+## BUG-151: libvosk crashes the whole process under concurrent recognizer churn — the wake engine's parallel verify design hits it (HIGH, FIXED 2026-08-20)**Symptom.** `scripts/vosk_wake_bench.py --phrase "Hey Jarvis" --language de`dies with exit 139 ("Windows fatal exception: access violation", via@@ -10576,9 +10594,17 @@ survives and decisions stay identical, per the 2026-07-11 parity method).Upgrade/patch libvosk. Whichever is chosen must be re-measured with therepro script AND the bench (3/3 clean runs on the laptop) before it counts.-**Guards.** None in CI — a crash test cannot run without the native model;-`scripts/vosk_native_stress.py churn 180` (3 x clean) plus 3/3 clean bench-runs on the weak laptop are the acceptance test until a fix lands.+**Fix (2026-08-20).** Every native constructor and method now goes through+`jarvis/plugins/wake/vosk_native.py`: one process-wide lock, the protocol+that survived the repro. Python-level grammar/free submit still uses two+threads (the 2026-07-10 spawn-latency shape); the lock serialises the+native calls themselves. Test doubles are not wrapped, so the concurrent+barrier unit test still proves the submit shape.++**Guards.** `tests/unit/plugins/wake/test_vosk_native.py`. Native acceptance+remains `scripts/vosk_native_stress.py churn 180` (3 x clean) plus 3/3 clean+bench runs on the weak laptop — a crash test cannot run in CI without the+model.## BUG-152: Gemini Live greeting cut off mid-sentence — a stale empty interrupt cancels the speaker drain (CRITICAL, FIXED 2026-08-19)diff --git a/jarvis/audio/player.py b/jarvis/audio/player.pyindex 9d8235a9b5..db2c5f5d8d 100644--- a/jarvis/audio/player.py+++ b/jarvis/audio/player.py@@ -42,6 +42,33 @@ log = logging.getLogger("jarvis.audio.player")# even without sounddevice (H4 review)._PortAudioError: type[BaseException] = sd.PortAudioError if sd is not None else OSError+# Markers that the LOCAL output sink died (BUG-108). Sample-rate refusals+# (-9997) are handled by the rate cascade, not by device fallback.+_LOCAL_OUTPUT_DEAD_MARKERS: tuple[str, ...] = (+ "-9986",+ "-9996",+ "Internal PortAudio error",+ "Invalid device",+ "Device unavailable",+ "Error opening OutputStream",+)+++def is_local_output_error(exc: BaseException) -> bool:+ """True when playback failed because the local speaker died.++ A PortAudio error raised from the output path is a LOCAL audio problem.+ Rebuilding the realtime provider websocket cannot heal it (BUG-108).+ Invalid-sample-rate refusals are the rate cascade's job, not this+ classifier — they stay False so the existing walk can try the next rate.+ """+ text = str(exc)+ if "-9997" in text or "Invalid sample rate" in text:+ return False+ if isinstance(exc, _PortAudioError):+ return True+ return any(marker in text for marker in _LOCAL_OUTPUT_DEAD_MARKERS)+TTS_SAMPLE_RATE = 24_000 # Gemini 3.1 Flash TTS output rate# Start playout as soon as a small, click-safe prefix is available. Later writes# stay larger to preserve low CPU overhead and underrun resistance. With a@@ -353,6 +380,35 @@ def _resolve_output_device()# Fall through to the auto-headset heuristic below.+ candidates, hostapis = _collect_output_candidates(priority)+ if candidates:+ idx, device_info = candidates[0]+ _log_auto_headset_pick(idx, device_info, hostapis)+ return idx++ log.warning("auto-headset found no matching device — using system default.")+ return None+++def _sorted_output_candidates(+ priority: Sequence[str] | None = None,+) -> list[tuple[int, dict]]:+ """Usable output devices in auto-headset order (BUG-108 fallback)."""+ candidates, _hostapis = _collect_output_candidates(priority)+ return candidates+++def _ranked_output_device_indices(+ priority: Sequence[str] | None = None,+) -> list[int]:+ """Output device indices in auto-headset order, excluding dead sinks."""+ return [idx for idx, _dev in _sorted_output_candidates(priority)]+++def _collect_output_candidates(+ priority: Sequence[str] | None = None,+) -> tuple[list[tuple[int, dict]], Any]:+ """Enumerate, filter, and rank output devices. Shared by resolve + fallback."""user_priority = tuple(p for p in (priority or ()) if p)try:@@ -360,25 +416,14 @@ def _resolve_output_device(hostapis = sd.query_hostapis()except Exception as exc: # noqa: BLE001log.warning("Device query failed, using system default: %s", exc)- return None+ return [], None- # "Your device first": prefer the user's OS-selected default OUTPUT device- # over the generic headset guesses, UNLESS it is a device auto-headset exists- # to bypass (monitor/HDMI, SPDIF, the localized virtual mapper) — then fall- # back to the heuristic below so playback never lands on a wrong/dead sink.- # Injected as a NAME so the candidate sort still picks its best host-API twin- # (WASAPI) and skips WDM-KS. Ranks BELOW an explicit output_device_priority.os_default_name = _os_default_output_name(devices, hostapis)effective_priority = ((*user_priority, os_default_name) if os_default_name else user_priority)- # Candidates: real output devices that are not on the blocklist.- # WDM-KS is filtered when the same device (same name) is available on- # another host API — otherwise, for example, the only Realtek Speakers- # device lands on WDM-KS and crashes later at OutputStream open with- # -9999 'Blocking API not supported yet'.- raw_candidates: list[tuple[int, dict, str]] = [] # (idx, dev, hostapi_name)+ raw_candidates: list[tuple[int, dict, str]] = []for idx, d in enumerate(devices):if d.get("max_output_channels", 0) <= 0:continue@@ -386,8 +431,6 @@ def _resolve_output_device(low = name.lower()if any(blocked.lower() in low for blocked in _BLOCKED_OUTPUT_SUBSTRINGS):continue- # Locale-independent skip of the MME "Sound Mapper" / DirectSound- # "Primary Sound Driver" virtual router (translated display name).if is_legacy_primary_mapper(idx, hostapis, devices, output=True):continuehostapi_idx = d.get("hostapi", -1)@@ -397,15 +440,6 @@ def _resolve_output_device()raw_candidates.append((idx, d, hostapi_name))- # WDM-KS crashes on OutputStream open (-9999 'Blocking API not supported- # yet'). As long as ANY safe output device exists, we NEVER choose a- # WDM-KS device. Important — and the bugfix versus the old same-name- # logic: not even when a device name exists ONLY on WDM-KS (e.g.- # "Speakers (Realtek HD Audio output)" has no MME/WASAPI twin).- # The previously used same-name filter let exactly such WDM-KS-only- # devices through; they won on name rank and crashed on playback- # (BUG-014 recurrence 2026-05-24: Brain+TTS ok, user hears nothing).- # Only when ALL candidates are WDM-KS do we take one as a last resort.safe_exists = any(ha not in _FORBIDDEN_OUTPUT_HOSTAPIS for _, _, ha in raw_candidates)@@ -424,11 +458,6 @@ def _resolve_output_device(def _name_rank(entry: tuple[int, dict]) -> int:low = entry[1].get("name", "").lower()- # Precedence: explicit user priority, then the OS-selected default device- # (both carried in ``effective_priority``), then the generic headset list.- # Each block keeps "earlier entry = stronger"; the generic block is offset- # by len(effective_priority) so a generic hit can never tie or beat a- # user / OS-default hit.for rank, sub in enumerate(effective_priority):if sub.lower() in low:return rank@@ -437,24 +466,25 @@ def _resolve_output_device(return len(effective_priority) + rankreturn len(effective_priority) + len(_HEADSET_PRIORITY)- # Primary sort key: name match; secondary: host API preference. Without a- # name match the original order (system default) is preserved.candidates.sort(key=lambda e: (_name_rank(e), _hostapi_rank(e)))- if candidates:- idx, d = candidates[0]- hostapi_idx = d.get("hostapi", -1)- hostapi_name = (- hostapis[hostapi_idx].get("name", "?")- if 0 <= hostapi_idx < len(hostapis) else "?"- )- log.info(- "auto-headset → %s (idx=%d, ch=%s, hostapi=%s)",- d.get("name"), idx, d.get("max_output_channels"), hostapi_name,- )- return idx-- log.warning("auto-headset found no matching device — using system default.")- return None+ return candidates, hostapis+++def _log_auto_headset_pick(+ idx: int,+ device: dict,+ hostapis: Any,+) -> None:+ hostapi_idx = device.get("hostapi", -1)+ hostapi_name = (+ hostapis[hostapi_idx].get("name", "?")+ if hostapis is not None and 0 <= hostapi_idx < len(hostapis)+ else "?"+ )+ log.info(+ "auto-headset → %s (idx=%d, ch=%s, hostapi=%s)",+ device.get("name"), idx, device.get("max_output_channels"), hostapi_name,+ )class AudioPlayer:@@ -479,6 +509,9 @@ class AudioPlayer:# Resolve "auto-headset" / similar strings to the actual device index.# Integer values are not resolved — the user specifies those explicitly.self._device = _resolve_output_device(device, self._device_priority)+ # Indices that already died this session (BUG-108). recover_output_device+ # skips them so a vanished Bluetooth sink cannot be re-picked.+ self._output_failed_devices: set[int | None] = set()self._sample_rate = sample_rateself._channels = channels# Master output volume knob in [0.0, 1.0]. Applied in _write_samples via@@ -714,7 +747,50 @@ class AudioPlayer:return # no-op; avoid needless cache flushself._device = new_deviceself._device_logged = False # re-log the new device on next play+ self._output_failed_devices = set()+ self.invalidate_device_cache()++ def recover_output_device(self) -> bool:+ """Pick a different output after the current sink died (BUG-108).++ Mirrors the capture-side candidate fallback: skip the dead index and+ walk auto-headset order. Returns True when a different device was+ selected so the caller can retry the open. A pin (explicit int) is+ still abandoned when that index is gone — silence is worse than+ speakers.+ """+ dead = getattr(self, "_device", None)+ failed = set(getattr(self, "_output_failed_devices", set()))+ if isinstance(dead, int):+ failed.add(dead)+ else:+ try:+ default = sd.default.device[1] if sd is not None else None+ except Exception: # noqa: BLE001 — default query is best-effort+ default = None+ if isinstance(default, int):+ failed.add(default)+ failed.add(None)+ self._output_failed_devices = failedself.invalidate_device_cache()+ for idx in _ranked_output_device_indices(+ getattr(self, "_device_priority", ())+ ):+ if idx in failed:+ continue+ log.warning(+ "AudioPlayer recovered onto output device %r (skipped dead %s)",+ idx,+ failed,+ )+ self._device = idx+ self._device_logged = False+ return True+ log.error(+ "AudioPlayer has no remaining output device after %s died",+ dead,+ )+ return Falsedef set_volume(self, volume: float) -> None:"""Live-apply a new master output volume (0.0–1.0) — no stream restart.@@ -997,6 +1073,8 @@ class AudioPlayer:except _PortAudioError as exc:last_exc = excif "-9997" not in str(exc) and "Invalid sample rate" not in str(exc):+ if is_local_output_error(exc) and self.recover_output_device():+ return self._open_output_stream(source_rate)raise# Remember the refusal so no later walk in this player's life# retries it (the per-turn "@ 24000Hz failed" log storm).@@ -1350,14 +1428,29 @@ class AudioPlayer:# with Jarvis's voice across the whole sentence, not one coarse# level per flush). Nothing to feed here.self.last_write_owner_task_id = owner_task_id- await asyncio.to_thread(- self._write_samples,- stm,- arr,- pending_rate,- dev_rate,- playback_generation=playback_generation,- )+ try:+ await asyncio.to_thread(+ self._write_samples,+ stm,+ arr,+ pending_rate,+ dev_rate,+ playback_generation=playback_generation,+ )+ except _PortAudioError as exc:+ if not is_local_output_error(exc) or not self.recover_output_device():+ raise+ stm, dev_rate = await asyncio.to_thread(+ _ensure_stream, pending_rate+ )+ await asyncio.to_thread(+ self._write_samples,+ stm,+ arr,+ pending_rate,+ dev_rate,+ playback_generation=playback_generation,+ )if arr.size:wrote_audio = Truewith stream_state_lock:diff --git a/jarvis/core/events.py b/jarvis/core/events.pyindex a5befb3056..850c7a0941 100644--- a/jarvis/core/events.py+++ b/jarvis/core/events.py@@ -584,6 +584,11 @@ class SpeechSpoken(Event):# e.g. a surface-TTS readback inside a realtime session).voice: str | None = Nonevoice_provider: str | None = None+ # True when a classic TTS engine actually rendered this voice (verified+ # by construction). False when the label is only the pin we REQUESTED+ # from a generative native-audio renderer (BUG-086) — the heard speaker+ # can drift. Default True so existing classic publishers stay honest.+ voice_verified: bool = True@dataclass(frozen=True, slots=True)diff --git a/jarvis/realtime/session.py b/jarvis/realtime/session.pyindex 7738747a35..aa842071b1 100644--- a/jarvis/realtime/session.py+++ b/jarvis/realtime/session.py@@ -937,6 +937,17 @@ _HANDOVER_END_REASONS = frozenset({HANGUP_DESKTOP_FALLBACK, HANGUP_REALTIME_FALL# owner prevents garbage collection from cancelling that user-visible debt._DETACHED_DELEGATE_TASKS: set[asyncio.Task[None]] = set()+_LOCAL_OUTPUT_FAILURE: dict[str, str] = {+ "de": "Ich konnte das gerade nicht abspielen. Der Lautsprecher war weg.", # i18n-allow+ "en": "I couldn't play that just now. The speaker disappeared.",+ "es": "No pude reproducirlo ahora. El altavoz desapareció.", # i18n-allow+}+_HISTORY_LOST_INSTRUCTION = (+ "The previous realtime connection dropped and this is a FRESH session. "+ "You have no memory of earlier turns in this call. Do not invent, recall, "+ "or assert any prior topic, name, or request. Wait for the user's next "+ "words and treat them as the start of the conversation."+)_OUTPUT_LANGUAGE_FAILURE: dict[str, str] = {"de": "Ich konnte gerade keine sichere Antwort auf Deutsch erzeugen.", # i18n-allow"en": "I couldn't produce a safe answer in English just now.",@@ -1748,8 +1759,10 @@ def _session_instructions(tool_directive: str = "",preferences: str = "",skill_directive: str = "",+ skills_directive: str = "",workspace_directive: str = "",compact: bool = False,+ history_lost: bool = False,) -> str:"""Assemble the session instructions; ``compact`` is the small-brain profile.@@ -1880,6 +1893,7 @@ def _session_instructions("answer from this runtime identity exactly; do not describe the ""classic text brain configuration.")+ history_lost_line = _HISTORY_LOST_INSTRUCTION if history_lost else ""if compact:# Static-first / dynamic-last: everything that is identical from turn# to turn forms one stable prefix, so Ollama's KV prefix cache skips@@ -1894,7 +1908,9 @@ def _session_instructions(freshness_line,precision_line,identity_line,+ history_lost_line,workspace_directive,+ skills_directive,skill_directive,input_directive,clock_line,@@ -1915,6 +1931,11 @@ def _session_instructions(# that must always reach the action function instead of the model's own# knowledge.workspace_directive,+ # The installed-skill roster, for the same reason the workspace roster+ # is here: a name the model has never seen is a name it cannot call.+ # Without it the model guessed the run-skill argument from the user's+ # words and lost three live turns to "Unknown skill" (2026-08-20).+ skills_directive,# A matched skill's own instructions, when the turn qualified for direct# injection. Placed AFTER the tool directive and BEFORE the safety# appendix on purpose: the skill refines HOW to answer this turn, and@@ -1926,6 +1947,7 @@ def _session_instructions(freshness_line,precision_line,identity_line,+ history_lost_line,language_directive,]return "\n\n".join(part for part in parts if part)@@ -2301,6 +2323,10 @@ class RealtimeVoiceSession:self._turn_index = 0self._current_turn_index = -1self._last_user_text = ""+ #: ``(utterance, skill_name)`` of the last skill injected INLINE by+ #: ``_skill_directive``. The delegate reads it to avoid handing the+ #: brain a skill this session already put in front of the model.+ self._skill_inlined_for: tuple[str, str] | None = None# Live caption of the CURRENT unfinished utterance. Surfaces render# it; persistence never does unless the promotion path says so# explicitly (a mid-word partial silently recorded as the turn's@@ -2666,6 +2692,93 @@ class RealtimeVoiceSession:"that the work went out, say what you actually did instead.")+ def _note_skill_for_delegate(self, text: str) -> None:+ """Hand a FIRE-band skill match to the brain before it answers.++ The realtime session can only run a skill itself when the skill is+ instruction-only, inline, short enough and tool-free+ (``_skill_directive``). Everything else — the tool-backed skills, the+ long ones, the mission ones — used to be dropped silently, so a skill+ the user installed for exactly this sentence produced a plain+ conversational answer and no trace anywhere.++ This is the other half: the same deterministic match, handed to the+ BrainManager through the pre-existing ``note_skill_trigger`` contract+ (AD-S4) that the classic pipeline and the chat hook already use. The+ brain injects the rendered instructions into the turn it is about to+ run, so a matched skill is either inlined here or executed there —+ never neither.++ Best-effort by design: a skill fault must not cost the delegated turn,+ which is the user's actual answer.+ """+ utterance = str(text or "").strip()+ if not utterance:+ return+ note = getattr(self._brain, "note_skill_trigger", None)+ if not callable(note):+ return+ inlined = self._skill_inlined_for+ if inlined is not None and inlined[0] == utterance:+ # This session already put the skill in front of the model.+ return+ try:+ from jarvis.skills.match_eval import BAND_FIRE, evaluate_match+ from jarvis.skills.skill_context import try_get_skill_context++ context = try_get_skill_context()+ if context is None:+ return+ decision = evaluate_match(context.registry, utterance, limit=2)+ if decision.band != BAND_FIRE or decision.top is None:+ return+ skill_name = str(decision.top.skill_name)+ note(skill_name, content="", source="realtime_match")+ log.info(+ "realtime[%s] skill %r matched (%s band) — handed to the "+ "delegated brain turn",+ self.session_id,+ skill_name,+ decision.band,+ )+ except Exception: # noqa: BLE001 — never cost the turn its answer+ log.debug("Realtime skill handoff skipped", exc_info=True)++ def _skills_directive(self, *, compact: bool | None = None) -> str:+ """The installed-skill roster for this session's instructions.++ Belt to the planner's braces, exactly like ``_workspace_directive``:+ the planner routes a skill turn deterministically, and this makes the+ model able to NAME the skill once it gets there. Before it existed the+ live instructions carried the ``run-skill`` tool and not one skill+ name, so the argument was always a guess.++ Rebuilt per turn because skills hot-reload mid-call, and cheap enough+ to do so: ``list_active`` is an in-memory read. Any fault answers "no+ roster" — a missing block costs a skill call, a raised exception costs+ the whole call.++ ``compact`` is passed explicitly by the connect path, which knows the+ provider capability before ``_compact_instructions`` is set on self.+ """+ if compact is None:+ compact = bool(getattr(self, "_compact_instructions", False))+ try:+ from jarvis.skills.prompt_injection import (+ render_realtime_skills_directive,+ )+ from jarvis.skills.skill_context import try_get_skill_context++ context = try_get_skill_context()+ if context is None:+ return ""+ return render_realtime_skills_directive(+ context.registry, compact=bool(compact)+ )+ except Exception: # noqa: BLE001 — a roster fault must never end a call+ log.debug("Realtime skills roster unavailable", exc_info=True)+ return ""+def _skill_directive(self, text: str) -> str:"""A matched skill's instructions, injected straight into this turn.@@ -2690,6 +2803,14 @@ class RealtimeVoiceSession:competing instruction sets guarantee an incoherent reply.Returns "" whenever anything does not hold, which is the common case.+ A "" here means "not inline" and never "no skill": every condition+ below is about whether this SESSION can run the skill itself, not+ about whether the skill matched. The delegate re-derives the match and+ hands it to the brain (``_note_skill_for_delegate``), because the+ version of this method that just returned "" dropped both installed+ morning routines on the floor for every single utterance — one over+ the body cap, one tool-backed — while the match said FIRE every time+ (live 2026-08-20)."""if not text:return ""@@ -2766,6 +2887,10 @@ class RealtimeVoiceSession:except Exception: # noqa: BLE001log.debug("SkillInvoked publish failed", exc_info=True)+ # Claim the turn so the delegate does not hand the brain the same skill+ # a second time — one instruction set per turn, whichever path wins.+ self._skill_inlined_for = (text, str(skill.name))+# Wrapped the way trusted external content is wrapped elsewhere in this# module: the model treats it as its own instructions for this turn, and# must answer with the RESULT rather than reading the steps aloud.@@ -3049,12 +3174,18 @@ class RealtimeVoiceSession:tool_directive=self._tool_directive(provider=provider),preferences=_preferences_block(self._config),workspace_directive=self._workspace_directive(),+ skills_directive=self._skills_directive(+ compact=bool(+ getattr(provider, "prefers_compact_instructions", False)+ )+ ),# Capability, never a provider-name check (AP-21): a small# self-hosted brain asks for the compact profile so it is# not prefilling 24k chars per turn.compact=bool(getattr(provider, "prefers_compact_instructions", False)),+ history_lost=self._suppress_history_seed,),language=self._language,input_language=self._input_language,@@ -4519,6 +4650,7 @@ class RealtimeVoiceSession:skill_directive=self._skill_directive(self._last_user_text or ""),+ skills_directive=self._skills_directive(),# Rebuilt per turn: panes open and close mid# call, and a roster naming a terminal that is# gone is worse than none.@@ -4526,6 +4658,7 @@ class RealtimeVoiceSession:compact=getattr(self, "_compact_instructions", False),+ history_lost=self._suppress_history_seed,),"language": new_language,# For append-only transports: ONLY the turn-scoped@@ -5772,6 +5905,16 @@ class RealtimeVoiceSession:except asyncio.CancelledError:raiseexcept Exception as exc: # noqa: BLE001 — AP-20: never re-read a dead transport+ from jarvis.audio.player import is_local_output_error++ if is_local_output_error(exc):+ # Speaker death must not consume the rebuild budget or drop+ # the in-call seed (BUG-108). The provider is still alive;+ # finish this pump pass as a local failure, not a transport+ # death. Returning None would hang up; returning a rebuild+ # detail would reopen the websocket into the same dead sink.+ await self._handle_local_output_failure(exc)+ return Nonemessage = safe_preview(exc, max_chars=800) or "Realtime receive loop ended"log.warning("realtime[%s] pump ended", self.session_id, exc_info=True)same_provider_rebuild = self._transport_death_is_rebuildable()@@ -6344,7 +6487,9 @@ class RealtimeVoiceSession:),preferences=_preferences_block(self._config),workspace_directive=self._workspace_directive(),+ skills_directive=self._skills_directive(),compact=getattr(self, "_compact_instructions", False),+ history_lost=self._suppress_history_seed,),language=self._language,)@@ -8036,9 +8181,12 @@ class RealtimeVoiceSession:language=self._language,spoken_kind=SPOKEN_KIND_REPLY,# The session itself rendered this audio (guard- # above) — its handshake voice is the speaker.+ # above). Native generative audio is the PIN,+ # not evidence the heard speaker matched it+ # (BUG-086).voice=self._active_voice or None,voice_provider=self.active_provider,+ voice_verified=False,))await self._bus.publish(@@ -8139,6 +8287,50 @@ class RealtimeVoiceSession:self._half_duplex_muted_since = Noneself._half_duplex_mute_reported = 0.0+ async def _handle_local_output_failure(self, exc: BaseException) -> None:+ """A dead speaker is not a dead provider (BUG-108).++ Zero the heard-sample counter so this turn is not reported healthy,+ ask the surface to recover onto another output device, and speak an+ honest apology. Never rebuild the realtime websocket for this.+ """+ log.warning(+ "realtime[%s] local speaker died (%s) — keeping the live provider",+ self.session_id,+ safe_preview(exc, max_chars=200),+ )+ self._output_samples_sent = 0+ self._output_active = False+ await self._publish_error(+ "RealtimeLocalOutputError",+ safe_preview(exc, max_chars=400) or "local speaker died",+ recoverable=True,+ )+ try:+ await self._send_json({"type": "output_recover"})+ except Exception: # noqa: BLE001, S110 — recovery is best-effort+ pass+ phrase = _LOCAL_OUTPUT_FAILURE.get(+ self._language, _LOCAL_OUTPUT_FAILURE["en"]+ )+ try:+ await self._send_json(+ self._surface_speech_message(+ phrase,+ spoken_kind=SPOKEN_KIND_REPLY,+ detail=safe_preview(exc, max_chars=200),+ )+ )+ except Exception as speak_exc: # noqa: BLE001+ from jarvis.audio.player import is_local_output_error++ if not is_local_output_error(speak_exc):+ log.warning(+ "realtime[%s] speaker-failure apology failed: %s",+ self.session_id,+ safe_preview(speak_exc, max_chars=200),+ )+async def _complete_surface_turn(self) -> None:"""Publish one idempotent surface boundary and reset turn state.@@ -8152,7 +8344,14 @@ class RealtimeVoiceSession:# mid-drain (BUG-152).self._clear_deferred_interruption()if self._turn_id:- await self._send_json({"type": "turn_complete"})+ try:+ await self._send_json({"type": "turn_complete"})+ except Exception as exc: # noqa: BLE001 — speaker death is not transport death+ from jarvis.audio.player import is_local_output_error++ if not is_local_output_error(exc):+ raise+ await self._handle_local_output_failure(exc)await self._publish_turn_completed()self._reset_output_state(reason="surface turn boundary")self._turn_final_text = ""@@ -11098,6 +11297,9 @@ class RealtimeVoiceSession:) -> None:turn_language = str(turn_state.language or self._language)succeeded = False+ # Before the brain answers, not after: ``note_skill_trigger`` is read by+ # the generate() call below, so a late hand-off is the same as none.+ self._note_skill_for_delegate(turn_state.user_text)try:reply = (await asyncio.wait_for(@@ -11421,7 +11623,15 @@ class RealtimeVoiceSession:# Real audible provider audio: advance the echo guard's playback# horizon by this chunk's duration (BUG-089).self._advance_echo_horizon((len(pcm) / 2) / rate)- await self._send_binary(pcm)+ try:+ await self._send_binary(pcm)+ except Exception as exc: # noqa: BLE001 — speaker death is not transport death+ from jarvis.audio.player import is_local_output_error++ if not is_local_output_error(exc):+ raise+ await self._handle_local_output_failure(exc)+ returnif audible:delegate_state = self._delegate_turns.get(self._turn_id)if delegate_state is not None and delegate_state.delivery_started:diff --git a/jarvis/sessions/formatter.py b/jarvis/sessions/formatter.pyindex 6d0487afe3..31552de733 100644--- a/jarvis/sessions/formatter.py+++ b/jarvis/sessions/formatter.py@@ -223,7 +223,12 @@ def format_session_markdown(spoken_by = t.voice_nameif getattr(t, "voice_provider", ""):spoken_by += f" @ {t.voice_provider}"- voice_note = f" · _Stimme: `{spoken_by}`_"+ if getattr(t, "voice_verified", True):+ voice_note = f" · _Voice: `{spoken_by}`_"+ else:+ voice_note = (+ f" · _Voice (requested, not verified): `{spoken_by}`_"+ )lines.append(f"**🔊 Jarvis sagte** _(de={t.jarvis_lang})_{voice_note}")lines.append("")for output in outputs:diff --git a/jarvis/sessions/models.py b/jarvis/sessions/models.pyindex bad4b03af3..82f1ea1f10 100644--- a/jarvis/sessions/models.py+++ b/jarvis/sessions/models.py@@ -143,6 +143,9 @@ class VoiceTurnRow(BaseModel):# brain provider, e.g. a surface-TTS readback inside a realtime session).voice_name: str = ""voice_provider: str = ""+ # False when ``voice_name`` is only the requested pin of a generative+ # native-audio renderer (BUG-086). Default True: classic TTS is verified.+ voice_verified: bool = Trueclass VoiceSessionRow(BaseModel):diff --git a/jarvis/sessions/recorder.py b/jarvis/sessions/recorder.pyindex 9dbaa30281..34e487fcdd 100644--- a/jarvis/sessions/recorder.py+++ b/jarvis/sessions/recorder.py@@ -180,6 +180,7 @@ class _TurnState:# authoritative SpeechSpoken track; VoiceTurnCompleted only fills a blank.voice_name: str = ""voice_provider: str = ""+ voice_verified: bool = True# Stage boundaries for think/speak latency calculation. The first thinking# segment begins at TranscriptFinal (classic) or final TranscriptionUpdate# (Realtime); later THINKING transitions reuse transcript_final_ms as the@@ -393,12 +394,16 @@ class SessionRecorder:late.voice_provider = str(getattr(event, "voice_provider", None) or "")+ late.voice_verified = bool(+ getattr(event, "voice_verified", True)+ )# The turn row was already written by finalize_turn — persist# the honest label explicitly.self._store.update_turn_voice(turn_id=late.turn_id,voice_name=late.voice_name,voice_provider=late.voice_provider,+ voice_verified=late.voice_verified,)returnt = self._state.current_turn@@ -407,6 +412,7 @@ class SessionRecorder:if event.spoken_kind == SPOKEN_KIND_REPLY or not t.voice_name:t.voice_name = str(voice)t.voice_provider = str(getattr(event, "voice_provider", None) or "")+ t.voice_verified = bool(getattr(event, "voice_verified", True))def _late_reply_target(self, event: SpeechSpoken) -> _TurnState | None:"""Finalized turn a late reply-kind ``SpeechSpoken`` belongs to.@@ -668,6 +674,7 @@ class SessionRecorder:awaiting_confirmation=t.awaiting_confirmation,voice_name=t.voice_name,voice_provider=t.voice_provider,+ voice_verified=t.voice_verified,)# Bump aggregatesself._state.turn_count += 1diff --git a/jarvis/sessions/schema.sql b/jarvis/sessions/schema.sqlindex 4c6e42c98e..98f0903940 100644--- a/jarvis/sessions/schema.sql+++ b/jarvis/sessions/schema.sql@@ -66,6 +66,9 @@ CREATE TABLE IF NOT EXISTS voice_turns (-- added via _apply_migrations for pre-existing DBs.voice_name TEXT NOT NULL DEFAULT '',voice_provider TEXT NOT NULL DEFAULT '',+ -- 1 when voice_name is a verified TTS render; 0 when it is only the+ -- requested pin of a generative native-audio renderer (BUG-086).+ voice_verified INTEGER NOT NULL DEFAULT 1,tool_calls_json TEXT NOT NULL DEFAULT '[]' -- JSON array of tool-name strings);diff --git a/jarvis/sessions/store.py b/jarvis/sessions/store.pyindex 35c5a6eb2d..54dcada315 100644--- a/jarvis/sessions/store.py+++ b/jarvis/sessions/store.py@@ -162,6 +162,12 @@ class SessionStore:"ADD COLUMN voice_provider TEXT NOT NULL DEFAULT ''")log.info("SessionStore migration: added voice_turns.voice_provider")+ if "voice_verified" not in existing:+ self._conn.execute(+ "ALTER TABLE voice_turns "+ "ADD COLUMN voice_verified INTEGER NOT NULL DEFAULT 1"+ )+ log.info("SessionStore migration: added voice_turns.voice_verified")if "user_text_polished" not in existing:# A NEW column rather than a rewrite of ``user_text``, and that is# the whole point: the transcript of what was actually said is the@@ -318,6 +324,7 @@ class SessionStore:awaiting_confirmation: bool = False,voice_name: str = "",voice_provider: str = "",+ voice_verified: bool = True,) -> None:with self._lock:self._c.execute(@@ -340,6 +347,7 @@ class SessionStore:awaiting_confirmation = ?,voice_name = ?,voice_provider = ?,+ voice_verified = ?,tool_calls_json = ?WHERE id = ?""",@@ -361,6 +369,7 @@ class SessionStore:1 if awaiting_confirmation else 0,voice_name,voice_provider,+ 1 if voice_verified else 0,json.dumps(tool_calls),turn_id,),@@ -392,6 +401,7 @@ class SessionStore:turn_id: str,voice_name: str,voice_provider: str,+ voice_verified: bool = True,) -> None:"""Late voice-label update for an already-finalized turn (BUG-090).@@ -402,9 +412,9 @@ class SessionStore:"""with self._lock:self._c.execute(- "UPDATE voice_turns SET voice_name = ?, voice_provider = ? "- "WHERE id = ?",- (voice_name, voice_provider, turn_id),+ "UPDATE voice_turns SET voice_name = ?, voice_provider = ?, "+ "voice_verified = ? WHERE id = ?",+ (voice_name, voice_provider, 1 if voice_verified else 0, turn_id),)# --- Events -------------------------------------------------------@@ -846,6 +856,7 @@ def _row_to_turn(r: sqlite3.Row) -> VoiceTurnRow:awaiting_confirmation=bool(_safe_col(r, "awaiting_confirmation", 0)),voice_name=_safe_text_col(r, "voice_name"),voice_provider=_safe_text_col(r, "voice_provider"),+ voice_verified=bool(_safe_col(r, "voice_verified", 1)),tool_calls=json.loads(r["tool_calls_json"] or "[]"),)diff --git a/jarvis/speech/pipeline.py b/jarvis/speech/pipeline.pyindex 2f5c4a80aa..150db7d347 100644--- a/jarvis/speech/pipeline.py+++ b/jarvis/speech/pipeline.py@@ -8882,6 +8882,10 @@ class SpeechPipeline:semantic_turn_committed = True_close_output_segment(preserve_echo_tail=False)await _cancel_output_playback()+ elif kind == "output_recover":+ recover = getattr(self._player, "recover_output_device", None)+ if callable(recover):+ recover()elif kind == "turn_complete":semantic_turn_committed = Trueawait playback.finish_turn()diff --git a/jarvis/ui/web/frontend/src/components/sessions/TurnCard.test.tsx b/jarvis/ui/web/frontend/src/components/sessions/TurnCard.test.tsxindex 14eed0459e..74e27f7ba8 100644--- a/jarvis/ui/web/frontend/src/components/sessions/TurnCard.test.tsx+++ b/jarvis/ui/web/frontend/src/components/sessions/TurnCard.test.tsx@@ -284,6 +284,20 @@ describe("TurnCard spoken track", () => {expect(screen.getByText("Fenrir · gemini-live")).toBeTruthy();});+ it("marks a generative native pin as requested, not heard", () => {+ render(+ <TurnCard+ turn={turn({+ jarvis_text: "Servus!", // i18n-allow: German voice fixture+ voice_name: "Fenrir",+ voice_provider: "gemini-live",+ voice_verified: false,+ })}+ />,+ );+ expect(screen.getByText("Fenrir · gemini-live · requested")).toBeTruthy();+ });+it("shows no voice badge when the speaking voice is unknown", () => {render(<TurnCard turn={turn({ jarvis_text: "Hallo." })} />); // i18n-allow: German voice fixtureexpect(screen.queryByText(/gemini-live/)).toBeNull();diff --git a/jarvis/ui/web/frontend/src/components/sessions/TurnCard.tsx b/jarvis/ui/web/frontend/src/components/sessions/TurnCard.tsxindex 201b5662b2..6a9bb236dc 100644--- a/jarvis/ui/web/frontend/src/components/sessions/TurnCard.tsx+++ b/jarvis/ui/web/frontend/src/components/sessions/TurnCard.tsx@@ -289,13 +289,20 @@ export function TurnCard({ turn, displayNumber, spoken = [] }: Props) {variant="outline"className="ml-1 font-mono text-[9px] normal-case text-muted-foreground"title={- turn.voice_provider- ? `Voice: ${turn.voice_name} (${turn.voice_provider})`- : `Voice: ${turn.voice_name}`+ turn.voice_verified === false+ ? `Requested voice (native audio is not a verified speaker): ${+ turn.voice_provider+ ? `${turn.voice_name} (${turn.voice_provider})`+ : turn.voice_name+ }`+ : turn.voice_provider+ ? `Voice: ${turn.voice_name} (${turn.voice_provider})`+ : `Voice: ${turn.voice_name}`}>{turn.voice_name}{turn.voice_provider ? ` · ${turn.voice_provider}` : ""}+ {turn.voice_verified === false ? " · requested" : ""}</Badge>)}{turn.awaiting_confirmation && (diff --git a/jarvis/ui/web/frontend/src/components/sessions/types.ts b/jarvis/ui/web/frontend/src/components/sessions/types.tsindex 35dc206aed..7d2b659158 100644--- a/jarvis/ui/web/frontend/src/components/sessions/types.ts+++ b/jarvis/ui/web/frontend/src/components/sessions/types.ts@@ -125,6 +125,9 @@ export interface VoiceTurnRow {// provider, e.g. a surface-TTS readback inside a realtime turn).voice_name: string;voice_provider: string;+ // False when voice_name is only the requested pin of a generative+ // native-audio renderer (BUG-086). Missing/true = verified (classic TTS).+ voice_verified?: boolean;}export interface VoiceSessionRow {diff --git a/tests/unit/audio/test_player_output_fallback.py b/tests/unit/audio/test_player_output_fallback.pynew file mode 100644index 0000000000..114fe6d593--- /dev/null+++ b/tests/unit/audio/test_player_output_fallback.py@@ -0,0 +1,115 @@+"""BUG-108: a dead local speaker is recovered, not treated as transport death."""++from __future__ import annotations++import jarvis.audio.player as player_module+from jarvis.audio.player import AudioPlayer, is_local_output_error+++def test_is_local_output_error_detects_portaudio_device_death() -> None:+ err = player_module._PortAudioError(+ "Error opening OutputStream: Internal PortAudio error [PaErrorCode -9986]"+ )+ assert is_local_output_error(err) is True+ assert is_local_output_error(RuntimeError(str(err))) is True+++def test_is_local_output_error_ignores_invalid_sample_rate() -> None:+ err = player_module._PortAudioError("Invalid sample rate [PaErrorCode -9997]")+ assert is_local_output_error(err) is False+++def test_is_local_output_error_ignores_transport_failures() -> None:+ assert is_local_output_error(ConnectionError("websocket 1006")) is False+ assert is_local_output_error(TimeoutError("handshake")) is False+++def test_open_retries_on_the_next_output_device(monkeypatch) -> None:+ player = AudioPlayer.__new__(AudioPlayer)+ player._device = 5+ player._device_priority = ()+ player._device_logged = True+ player._output_failed_devices = set()+ player._device_rate_cache = {}+ player._device_rate_failed = set()+ player._playback_generation = 0+ player._active_stream = None+ player._stream_channels = 2+ player._output_buffer_s = 0.2++ opens: list[int | str | None] = []++ class _OkStream:+ latency = 0.05++ def start(self) -> None:+ return None++ def fake_query(device=None, kind=None): # noqa: ANN001+ return {"default_samplerate": 48000, "max_output_channels": 2}++ monkeypatch.setattr(player_module.sd, "query_devices", fake_query)++ def fake_stream(**kwargs): # noqa: ANN003+ opens.append(kwargs.get("device"))+ if kwargs.get("device") == 5:+ raise player_module._PortAudioError(+ "Internal PortAudio error",+ -9986,+ )+ return _OkStream()++ monkeypatch.setattr(player_module.sd, "OutputStream", fake_stream)++ recovered = {"called": False}++ def recover() -> bool:+ recovered["called"] = True+ player._device = 2+ return True++ monkeypatch.setattr(player, "recover_output_device", recover)+ monkeypatch.setattr(+ player_module.topology, "stream_open_guard", lambda: _null_guard()+ )++ stream, rate = player._open_output_stream(24_000) # noqa: SLF001+ assert recovered["called"] is True+ assert player._device == 2+ assert isinstance(stream, _OkStream)+ assert rate in (24_000, 48_000)+ assert 5 in opens+ assert 2 in opens+++class _null_guard:+ def __enter__(self):+ return self++ def __exit__(self, *args):+ return False+++def test_recover_output_device_skips_the_dead_index(monkeypatch) -> None:+ player = AudioPlayer.__new__(AudioPlayer)+ player._device = 7+ player._device_priority = ()+ player._output_failed_devices = set()+ player._playback_generation = 0+ player._active_stream = None+ player._active_source_rate = None+ player._active_device_rate = None+ player._device_rate_cache = {}+ player._device_rate_failed = set()+ player._device_logged = True++ monkeypatch.setattr(+ player_module,+ "_ranked_output_device_indices",+ lambda _priority=None: [7, 3, 1],+ )+ monkeypatch.setattr(player, "invalidate_device_cache", lambda: None)++ assert player.recover_output_device() is True+ assert player._device == 3+ assert 7 in player._output_failed_devicesdiff --git a/tests/unit/realtime/test_local_output.py b/tests/unit/realtime/test_local_output.pynew file mode 100644index 0000000000..52867a9c60--- /dev/null+++ b/tests/unit/realtime/test_local_output.py@@ -0,0 +1,108 @@+"""BUG-108: a dead local speaker is not a dead realtime provider."""++from __future__ import annotations++import asyncio++import pytest++from jarvis.core.events import SpeechSpoken+from jarvis.realtime.protocol import RealtimeEvent+from jarvis.realtime.session import (+ _HISTORY_LOST_INSTRUCTION,+ RealtimeVoiceSession,+ _session_instructions,+)+from tests.unit.realtime.test_session import (+ RebuildingProvider,+ StayOpenSession,+ _cfg,+ _wait_until,+)+++def test_seedless_rebuild_instructions_forbid_invented_history() -> None:+ lost = _session_instructions("en", history_lost=True)+ kept = _session_instructions("en", history_lost=False)+ assert _HISTORY_LOST_INSTRUCTION in lost+ assert _HISTORY_LOST_INSTRUCTION not in kept+ compact = _session_instructions("en", history_lost=True, compact=True)+ assert _HISTORY_LOST_INSTRUCTION in compact+++async def test_local_speaker_death_does_not_rebuild_the_provider() -> None:+ jsons: list[dict] = []++ async def send_json(message: dict) -> None:+ jsons.append(message)+ if message.get("type") == "turn_complete":+ raise RuntimeError(+ "Error opening OutputStream: Internal PortAudio error "+ "[PaErrorCode -9986]"+ )++ provider = RebuildingProvider(+ [+ lambda: StayOpenSession(+ [+ RealtimeEvent(+ type="input_transcript", text="hello", is_final=True+ ),+ RealtimeEvent(+ type="output_transcript", text="hi there", is_final=True+ ),+ RealtimeEvent(type="turn_complete"),+ ]+ ),+ ]+ )+ sess = RealtimeVoiceSession(+ session_id="local-speaker-death",+ send_binary=lambda _data: asyncio.sleep(0),+ send_json=send_json,+ provider=provider,+ config=_cfg(),+ bus=None,+ )+ await sess.handle_control({"type": "audio_start", "sample_rate": 16_000})+ await _wait_until(+ lambda: any(m.get("type") == "output_recover" for m in jsons)+ )++ assert provider.open_calls == 1+ assert not sess.failed+ spoken = [m for m in jsons if m.get("type") == "error_spoken"]+ assert spoken+ texts = " ".join(str(m.get("text", "")).lower() for m in spoken)+ assert "speaker" in texts or "lautsprecher" in texts or "altavoz" in texts+ await sess.end(reason="test")+++async def test_native_reply_marks_the_voice_pin_unverified() -> None:+ events: list[object] = []++ class _Bus:+ async def publish(self, event: object) -> None:+ events.append(event)++ sess = RealtimeVoiceSession(+ session_id="voice-pin",+ send_binary=lambda _data: asyncio.sleep(0),+ send_json=lambda _m: asyncio.sleep(0),+ provider=RebuildingProvider([lambda: StayOpenSession([])]),+ config=_cfg(),+ bus=_Bus(),+ )+ sess._turn_id = "turn-1"+ sess._last_user_text = "hello"+ sess._output_transcript = ["hi there"]+ sess._output_samples_sent = 480+ sess._active_voice = "Fenrir"+ sess._active_model = "gemini-live"+ await sess._publish_turn_completed() # noqa: SLF001+ spoken = [e for e in events if isinstance(e, SpeechSpoken)]+ assert spoken+ assert spoken[0].voice == "Fenrir"+ assert spoken[0].voice_verified is Falsediff --git a/tests/unit/sessions/test_turn_voice_label.py b/tests/unit/sessions/test_turn_voice_label.pyindex d51ab1b7e6..fa15599d36 100644--- a/tests/unit/sessions/test_turn_voice_label.py+++ b/tests/unit/sessions/test_turn_voice_label.py@@ -480,7 +480,7 @@ def test_markdown_export_prints_the_voice_note() -> None:voice_provider="gemini-live",)md = format_session_markdown(session, [turn])- assert "Stimme: `Fenrir @ gemini-live`" in md # i18n-allow: localized export label under test+ assert "Voice: `Fenrir @ gemini-live`" in mddef test_markdown_export_omits_the_note_when_voice_unknown() -> None: