Skip to content

Dodaios is an open-source agent OS for your desktop: it runs your coding agents, browser and apps by voice or chat, on your own machine. Formerly Personal Jarvis.

feat(voice): the Thinking pause decides when a turn is taken — in both engines, and the user may keep talking

Harald74eb9c852497
diff --git a/docs/product/everyday-use/voice-conversations.md b/docs/product/everyday-use/voice-conversations.md
index 6960eaaef5..e8145cdec2 100644
--- a/docs/product/everyday-use/voice-conversations.md
+++ b/docs/product/everyday-use/voice-conversations.md
@@ -52,10 +52,12 @@ contain several turns.
press the configured **Call (answer / start talking)** shortcut. Wake
detection opens the microphone; it does not answer or transcribe the
request itself.
-2. **Speak while the app shows Listening.** Pipeline waits for a pause long
- enough to mark the end of your request. Realtime lets the live voice model
- detect that boundary. The Pipeline **Thinking pause** setting therefore
- does not control Realtime.
+2. **Speak while the app shows Listening.** Jarvis waits for a pause long
+ enough to mark the end of your request — the **Thinking pause** setting
+ says how long, in Pipeline and Realtime alike. If you keep talking inside
+ that pause, the new words are added to the same request instead of being
+ sent as a second one; a clearly finished sentence is answered as soon as
+ the pause has passed.
3. **Jarvis prepares one understood request.** Pipeline converts the captured
audio into text first. Realtime receives transcript updates from the live
connection. Your Speech Dictionary corrects known names and terms in both
diff --git a/docs/product/personalize-and-connect/settings-and-appearance.md b/docs/product/personalize-and-connect/settings-and-appearance.md
index fa735de6e5..0fe2b7498c 100644
--- a/docs/product/personalize-and-connect/settings-and-appearance.md
+++ b/docs/product/personalize-and-connect/settings-and-appearance.md
@@ -50,7 +50,7 @@ that have their own guides.
| **Realtime voice (browser)** | Pipeline or Realtime voice mode | Saves immediately. An active desktop call reconnects when its mode changes; otherwise the next call uses the choice |
| **System Prompt** | The assistant's general style and behavior | On the next message; no restart |
| **Wake Word** | Activation, phrase, spoken language, detection engine, and self-test | Usually live when desktop voice is ready; otherwise on the next voice start |
-| **Thinking pause** | End-of-speech timing for Pipeline mode; in Realtime mode the voice model detects the end of your turn itself | Live with Pipeline running; otherwise on the next Pipeline start |
+| **Thinking pause** | How long you may pause before Jarvis takes your turn — one value for Pipeline and Realtime. If you keep talking inside the pause, the words are added to the same request instead of being sent twice | Live with Pipeline running; Realtime uses the value on the next turn (some voice models on the next call) |
| **Volume** | Local spoken-output volume | Live in running desktop voice; browser-owned Realtime uses browser or system volume |
| **Audio devices** | Local microphone and voice output | Live for devices known to the running app; a newly connected device may need the next app start |
| **Voice Keybinds** | **Call (answer / start talking)** and **Hangup** shortcuts | Live with the desktop voice listener running; otherwise on the next voice start |
@@ -174,7 +174,7 @@ keys are allowed but warn you that they also fire while editing text.
microphone, speaker, activation phrase, and spoken-language setting. Voice
keybinds provide another way to start the desktop voice path.
4. [Languages and Voices](languages-and-voices) controls recognition and reply
- language. Thinking pause affects Pipeline only. Local Volume and Audio
+ language. Thinking pause applies to Pipeline and Realtime. Local Volume and Audio
device controls cover desktop voice, while browser-owned Realtime uses the
browser's audio devices and volume.
5. [Permissions](permissions) decides whether macOS allows microphone input,
diff --git a/jarvis/plugins/realtime/gemini_live.py b/jarvis/plugins/realtime/gemini_live.py
index 08863c8c80..c46c78bece 100644
--- a/jarvis/plugins/realtime/gemini_live.py
+++ b/jarvis/plugins/realtime/gemini_live.py
@@ -909,17 +909,26 @@ class GeminiLiveProvider:
voice_name=voice
)
)
- # An unset window (None/0) sends no silence_duration_ms at all, so
- # Gemini keeps deciding turn ends on its own timing; only an explicit
- # override forces a fixed window (the 2026-07-21 directive).
- silence_ms = getattr(cfg, "silence_duration_ms", None)
- # End-of-speech SENSITIVITY is a different knob and the one that
- # matters here: Gemini's native default reads an ordinary mid-sentence
- # pause as the end of the turn. Live 2026-08-13 16:46/16:47 — one
- # spoken brief for a coding pane was committed twice while the
- # microphone still carried the user's voice, and the pane was handed a
- # quarter of the sentence. LOW reads a pause as a pause and costs a
- # finished short utterance nothing.
+ # This transport answers on its own activity boundary
+ # (``creates_responses_automatically``): Jarvis cannot hold a reply
+ # back once Gemini has closed the user's turn, so the ONE lever for
+ # "wait for a clear pause before you take the turn" is Gemini's own
+ # silence window. The user's Thinking pause (``turn_pause_ms``, the
+ # same Settings value the classic pipeline endpoints on) is folded in
+ # here; an explicit ``silence_duration_ms`` override still wins, and
+ # with neither set Gemini keeps deciding turn ends on its own timing.
+ # A user who resumes inside the window continues the SAME activity —
+ # the words append, nothing is submitted twice (2026-08-18).
+ silence_ms = getattr(cfg, "silence_duration_ms", None) or getattr(
+ cfg, "turn_pause_ms", None
+ )
+ # End-of-speech SENSITIVITY is a different knob and stays: Gemini's
+ # native default reads an ordinary mid-sentence pause as the end of
+ # the turn. Live 2026-08-13 16:46/16:47 — one spoken brief for a
+ # coding pane was committed twice while the microphone still carried
+ # the user's voice, and the pane was handed a quarter of the sentence.
+ # LOW reads a pause as a pause; the silence window above then says how
+ # long that pause may be.
aad_kwargs: dict[str, Any] = {}
if silence_ms:
aad_kwargs["silence_duration_ms"] = int(silence_ms)
diff --git a/jarvis/plugins/realtime/openai_realtime.py b/jarvis/plugins/realtime/openai_realtime.py
index d518748d4f..b0cd9eab44 100644
--- a/jarvis/plugins/realtime/openai_realtime.py
+++ b/jarvis/plugins/realtime/openai_realtime.py
@@ -261,17 +261,22 @@ def _session_payload(
"interrupt_response": False,
}
# An unset window (None/0) keeps OpenAI's native server-VAD default so the
- # realtime model decides the turn end itself; only an explicit override is
- # forwarded.
+ # server commits the input on its own timing; only an explicit override is
+ # forwarded. The user's Thinking pause (``cfg.turn_pause_ms``) is
+ # deliberately NOT applied here: this is a manual-response transport
+ # (``create_response`` is False), so the SESSION waits out that pause on
+ # its own microphone before it requests the response — the transcription
+ # latency then counts towards the pause instead of on top of it, and a
+ # final that lands while the user is still talking appends to the open
+ # turn. A server window on top would make every finished sentence wait
+ # twice (the lag the 2026-07-21 directive rejected).
silence_ms = getattr(cfg, "silence_duration_ms", None)
if turn_detection == "server_vad" and silence_ms:
turn_detection_config["silence_duration_ms"] = int(silence_ms)
# OpenAI's name for end-of-speech sensitivity is semantic-VAD "eagerness":
# low = let the user take their time before the turn is called over. Plain
# server_vad has no equivalent knob at all, so a session configured for it
- # honestly keeps the provider default rather than faking patience with a
- # fixed silence window (which would tax every short utterance — the very
- # trade the 2026-07-21 directive rejected).
+ # honestly keeps the provider default; the pause is the session's job.
sensitivity = str(getattr(cfg, "end_of_speech_sensitivity", "") or "").lower()
if turn_detection == "semantic_vad" and sensitivity in {"low", "high"}:
turn_detection_config["eagerness"] = sensitivity
diff --git a/jarvis/realtime/protocol.py b/jarvis/realtime/protocol.py
index 87320ca013..5cf9ed72a4 100644
--- a/jarvis/realtime/protocol.py
+++ b/jarvis/realtime/protocol.py
@@ -94,27 +94,47 @@ class RealtimeSessionConfig:
# GA Realtime schema rejects requesting text and audio simultaneously.
modalities: tuple[str, ...] = ("audio",)
turn_detection: str = "server_vad" # "server_vad" | "semantic_vad"
- # None (the default) = the provider's NATIVE turn detection decides when
- # the user's turn ends. The Settings "Thinking pause"
- # (SpeechConfig.vad_silence_ms) endpoints the classic pipeline only —
- # forcing it into realtime sessions made the realtime model wait the full
- # window after every utterance, which reads as "done speaking but still
- # listening" (maintainer directive 2026-07-21). An explicit int still
- # overrides the provider default for callers that need a fixed window.
+ # An EXPLICIT fixed silence window for the provider's native VAD. None
+ # (the default) sends nothing: the provider commits the user's input on
+ # its own timing. This is a raw override for callers that need a fixed
+ # window; the user-facing pause travels as ``turn_pause_ms`` below.
silence_duration_ms: int | None = None
# How eagerly the provider's native VAD may declare the user finished.
# "low" = read a pause as a pause; "high" = the provider's own eager
# default; None = send nothing and inherit whatever the provider ships.
- #
- # This is deliberately NOT ``silence_duration_ms``. A fixed silence window
- # taxes EVERY utterance with the same wait, which is why the 2026-07-21
- # directive removed it — that directive stands. End-of-speech sensitivity
- # changes only how a PAUSE is interpreted, so a finished short sentence
- # still commits immediately while a hesitation inside a long order no
- # longer ends the turn (live 2026-08-13 16:46/16:47: a spoken brief for a
- # coding pane was closed twice mid-sentence and delivered truncated).
- # Providers without the concept ignore it.
+ # End-of-speech sensitivity changes only how a PAUSE is interpreted, so a
+ # finished short sentence still commits immediately while a hesitation
+ # inside a long order no longer ends the turn (live 2026-08-13
+ # 16:46/16:47: a spoken brief for a coding pane was closed twice
+ # mid-sentence and delivered truncated). Providers without the concept
+ # ignore it.
end_of_speech_sensitivity: str | None = "low"
+ # How long the user may PAUSE before their turn is taken — the Settings
+ # "Thinking pause" (SpeechConfig.vad_silence_ms), one value for both voice
+ # engines since 2026-08-18. Which lever applies is a transport
+ # CAPABILITY, never a provider name (AP-21):
+ #
+ # * A transport that answers on its own boundary
+ # (``creates_responses_automatically``) has exactly one lever — its
+ # native end-of-turn detection — so it folds this window into that
+ # detection (Gemini Live: ``automatic_activity_detection
+ # .silence_duration_ms``). A user who resumes inside the window keeps
+ # the same activity: the words append, nothing is submitted twice.
+ # * A transport whose responses Jarvis requests itself must IGNORE this
+ # field: the session applies the same pause locally, on the last
+ # voiced microphone frame, before it calls ``request_response`` — so
+ # the transcription latency counts towards the pause instead of on top
+ # of it, and a final that arrives while the user is still talking
+ # appends to the open turn (one response for the whole request).
+ #
+ # History: a fixed window on every realtime transport was removed on
+ # 2026-07-21 because the extra wait after a clearly finished sentence read
+ # as "done speaking but still listening"; on 2026-08-18 the maintainer
+ # asked for the opposite — wait for a clear pause, and keep listening
+ # while the sentence continues — so the pause is back, as the SAME
+ # setting the classic pipeline uses, adjustable in Settings → Voice.
+ # None = the provider's own timing (tests, callers without a config).
+ turn_pause_ms: int | None = None
tools: tuple[dict[str, Any], ...] = ()
# Bounded transcript of the call so far, oldest first, as
# ``{"role": "user" | "assistant", "text": ...}`` mappings. A fresh
diff --git a/jarvis/realtime/session.py b/jarvis/realtime/session.py
index 4e10fb20da..d4474e484d 100644
--- a/jarvis/realtime/session.py
+++ b/jarvis/realtime/session.py
@@ -357,6 +357,36 @@ _MIC_HOLD_ABSOLUTE_CAP_S = 45.0
# was held by the microphone settles for this long before giving up on the
# remaining words. Paid ONLY on a turn the provider already cut short.
_UTTERANCE_TAIL_SETTLE_S = 2.5
+# The Thinking pause — how long the user may pause before their turn is
+# TAKEN. One value for both voice engines (SpeechConfig.vad_silence_ms, the
+# Settings → Voice slider); these mirror that field's own bounds so a stray
+# value can never wedge a call, and the default is the pipeline's 1.5 s.
+#
+# On a transport whose responses Jarvis requests itself, the pause is measured
+# HERE, on the microphone (``_turn_pause_settled``): a final input transcript
+# does not request the response until the last voiced frame is at least this
+# old. Whatever the provider committed, the user gets to keep talking — the
+# next final appends to the same turn (``_note_user_final``) and ONE response
+# answers the whole request. A transport that answers on its own boundary
+# receives the same value as its native silence window instead
+# (``RealtimeSessionConfig.turn_pause_ms``). Maintainer directive
+# 2026-08-18: "wait for a clear pause; when I keep talking, append."
+_TURN_PAUSE_DEFAULT_MS = 1_500
+_TURN_PAUSE_MIN_MS = 500
+_TURN_PAUSE_MAX_MS = 5_000
+# How often the pause waiter re-checks the microphone. Fine enough that a
+# settled pause is noticed within a frame or two, coarse enough to be free.
+_TURN_PAUSE_POLL_S = 0.05
+# A stale voiced stamp only proves silence if the microphone kept REPORTING
+# in between. Frames normally arrive every 20-100 ms; when the last processed
+# frame is older than this, the stream stalled — typically the event loop was
+# blocked (a first-turn import measured 0.7 s in tests) and the frames are
+# still queued — and the pause is not settled until the stream catches up.
+_MIC_FRAME_STALL_S = 0.25
+# ...unless it never does: a microphone that has sent nothing for this long is
+# closed or paused, not stalled, and holding a reply on it would mute the
+# assistant. Past this the voiced stamp is trusted as-is (bounded, AP-30).
+_MIC_STREAM_GONE_S = 1.5
# In-place transport rebuild (BUG-071). A provider server may drop the duplex
# WebSocket at any time mid-call (live incident 2026-07-17 10:44: Gemini Live
# closed with ``1006 abnormal closure`` right as a 69 s surface-TTS fallback
@@ -2193,6 +2223,19 @@ class RealtimeVoiceSession:
# The one local answer to "is the user talking right now" while the
# provider owns turn detection (see _USER_VOICE_PEAK).
self._last_voiced_input_monotonic = 0.0
+ # Monotonic stamp of the last microphone frame processed at all, voiced
+ # or silent: the proof that the voiced stamp above is CURRENT rather
+ # than stale behind a stalled frame stream (see _MIC_FRAME_STALL_S).
+ self._last_input_frame_monotonic = 0.0
+ # The Thinking-pause waiter of the CURRENT turn on a manual-response
+ # transport: a final transcript arrived, but the microphone had not
+ # yet been quiet for the configured pause, so the response request is
+ # deferred to this task (see ``_request_native_response_after_pause``).
+ # ``_turn_pause_held_input_ids`` collects the provider item ids of the
+ # finals folded into that one deferred request, so the eventual
+ # request marks every one of them as answered.
+ self._turn_pause_waiter: asyncio.Task[None] | None = None
+ self._turn_pause_held_input_ids: set[str] = set()
self._loop_lag = _LoopLagProbe()
# A write-only transport stall does not necessarily wake the provider
# receive iterator. Queue a rebuild request for the long-lived pump so
@@ -2819,10 +2862,14 @@ class RealtimeVoiceSession:
input_sample_rate=input_rate,
output_sample_rate=output_rate,
modalities=("audio",),
- # silence_duration_ms stays at its None default: the realtime
- # model's native turn detection decides when the user is done.
- # The Settings "Thinking pause" endpoints the classic pipeline
- # only (maintainer directive 2026-07-21).
+ # silence_duration_ms stays at its None default (no raw
+ # override). The user's Thinking pause travels as
+ # turn_pause_ms: a transport that answers on its own boundary
+ # folds it into its native turn detection; a manual-response
+ # transport ignores it because THIS session waits out the
+ # same pause on the microphone before requesting the response
+ # (``_turn_pause_settled``). One setting, both engines.
+ turn_pause_ms=self._turn_pause_ms(),
tools=self._declared_tools(),
# Empty at the first open of a call; after an in-place
# transport rebuild (or a mid-call cross-family fallback) it
@@ -3157,6 +3204,214 @@ class RealtimeVoiceSession:
return False
return (time.monotonic() - stamp) < _USER_SPEAKING_HOLD_S
+ # ------------------------------------------------------------------
+ # The Thinking pause: how long the user may pause before the turn is taken
+ # ------------------------------------------------------------------
+
+ def _turn_pause_ms(self) -> int:
+ """The user's Thinking pause in ms, read live from the config.
+
+ The Settings → Voice slider writes ``speech.vad_silence_ms`` into the
+ running config, so a manual-response transport picks a new value up on
+ the very next turn; an automatic-response transport bakes it into its
+ native turn detection at open and follows on the next call. Clamped to
+ the field's own bounds — a stray value can slow a call, never wedge it.
+ """
+ speech_cfg = getattr(self._config, "speech", None)
+ raw = getattr(speech_cfg, "vad_silence_ms", None)
+ try:
+ ms = int(raw) if raw is not None else _TURN_PAUSE_DEFAULT_MS
+ except (TypeError, ValueError):
+ ms = _TURN_PAUSE_DEFAULT_MS
+ return max(_TURN_PAUSE_MIN_MS, min(_TURN_PAUSE_MAX_MS, ms))
+
+ def _turn_pause_settled(self) -> bool:
+ """True once the microphone has been quiet for the whole Thinking pause.
+
+ The pause is measured from the last voiced input frame, so the time the
+ provider spent committing and transcribing the audio already counts
+ towards it — a finished sentence pays little or nothing extra. A
+ microphone that never carried voice (below the peak gate, or a text
+ surface) has no evidence to hold the turn on and reads as settled: the
+ debounce can only DELAY a request while the user audibly talks on, it
+ can never deafen a quiet talker. Speaker echo is excluded exactly as in
+ ``_user_is_speaking`` — output frames never stamp the microphone.
+ ``_user_speech_active`` is the provider's own "the user started again"
+ edge, held until its transcript arrives; while it stands the pause has
+ not settled either, whatever the peak gate saw.
+
+ An old voiced stamp is trusted only while the frame stream is current:
+ if the microphone stopped REPORTING (an event loop blocked by a
+ first-turn import leaves the frames queued, the stamp stale, and the
+ user mid-word) the pause is not settled until the stream catches up —
+ bounded by ``_MIC_STREAM_GONE_S``, past which a silent microphone is
+ a closed one and can hold nothing.
+ """
+ if self._user_speech_active:
+ return False
+ stamp = self._last_voiced_input_monotonic
+ if not stamp:
+ return True
+ now = time.monotonic()
+ if (now - stamp) < self._turn_pause_ms() / 1000.0:
+ return False
+ stream_age = now - self._last_input_frame_monotonic
+ return not (_MIC_FRAME_STALL_S < stream_age < _MIC_STREAM_GONE_S)
+
+ def _turn_held_for_pause(self) -> bool:
+ """A final arrived, and its response waits for the user's pause to settle."""
+ waiter = self._turn_pause_waiter
+ return waiter is not None and not waiter.done()
+
+ def _hold_native_response_for_pause(self, input_item_id: str) -> None:
+ """Defer this turn's response request until the Thinking pause settles.
+
+ Idempotent per turn: a later final that lands while the waiter runs
+ only adds its item id — the waiter re-reads the growing turn text on
+ every tick, so the ONE request it eventually makes answers the whole
+ request, and every folded item is marked answered with it.
+ """
+ if input_item_id:
+ self._turn_pause_held_input_ids.add(input_item_id)
+ if self._turn_held_for_pause():
+ return
+ turn_id = self._turn_id
+ log.info(
+ "realtime[%s] holding the response request — the user is still "
+ "talking %.2fs after the provider closed its input turn; waiting "
+ "for a %.1fs pause",
+ self.session_id,
+ max(0.0, time.monotonic() - self._last_voiced_input_monotonic)
+ if self._last_voiced_input_monotonic
+ else 0.0,
+ self._turn_pause_ms() / 1000.0,
+ )
+ self._turn_pause_waiter = asyncio.create_task(
+ self._request_native_response_after_pause(turn_id),
+ name=f"rt-turn-pause-{self.session_id}",
+ )
+
+ def _cancel_turn_pause_waiter(self) -> None:
+ waiter = self._turn_pause_waiter
+ self._turn_pause_waiter = None
+ self._turn_pause_held_input_ids = set()
+ if waiter is not None and not waiter.done():
+ waiter.cancel()
+
+ async def _request_native_response_after_pause(self, turn_id: str) -> None:
+ """Request the deferred response the moment the user's pause settles.
+
+ Every check is a reason NOT to request: another path answered the turn
+ (a delegate, a tool call, the retry paths), the turn moved on, the
+ session ended. Otherwise the request fires when ``_turn_pause_settled``
+ first holds. Two bounds keep a stuck floor from muting the assistant
+ (AP-30, never silent): a loud microphone that produces no new WORDS for
+ ``_MIC_HOLD_STALE_TRANSCRIPT_S`` is room noise, not a talking user, and
+ the absolute ceiling ends any wait. Both are logged as what they are.
+ """
+ started = time.monotonic()
+ stale_deadline = started + _MIC_HOLD_STALE_TRANSCRIPT_S
+ hard_deadline = started + _MIC_HOLD_ABSOLUTE_CAP_S
+ last_text = self._last_user_text
+ try:
+ while True:
+ await asyncio.sleep(_TURN_PAUSE_POLL_S)
+ if self._ended or self._failed.is_set():
+ return
+ if self._response_requested_for_turn or self._turn_id != turn_id:
+ # Answered by another path, or the turn is over — the
+ # next turn's own final requests its own response.
+ return
+ now = time.monotonic()
+ if self._last_user_text != last_text:
+ # Words are the one thing a stuck floor cannot produce:
+ # a growing transcript renews the microphone's authority.
+ last_text = self._last_user_text
+ stale_deadline = now + _MIC_HOLD_STALE_TRANSCRIPT_S
+ if self._turn_pause_settled():
+ log.info(
+ "realtime[%s] the user's pause settled after a %.2fs "
+ "hold — requesting one response for the whole turn "
+ "(%d words)",
+ self.session_id,
+ now - started,
+ len(str(self._last_user_text or "").split()),
+ )
+ await self._request_native_response()
+ return
+ if now >= stale_deadline:
+ log.warning(
+ "realtime[%s] the microphone stayed loud for %.1fs "
+ "without a single new word; treating the floor as "
+ "stuck and requesting the response",
+ self.session_id,
+ _MIC_HOLD_STALE_TRANSCRIPT_S,
+ )
+ await self._request_native_response()
+ return
+ if now >= hard_deadline:
+ log.warning(
+ "realtime[%s] the pause hold reached its %.0fs "
+ "ceiling; requesting the response for the %d words "
+ "the turn has",
+ self.session_id,
+ _MIC_HOLD_ABSOLUTE_CAP_S,
+ len(str(self._last_user_text or "").split()),
+ )
+ await self._request_native_response()
+ return
+ except asyncio.CancelledError:
+ raise
+ except Exception: # noqa: BLE001 — a failed hold must never mute the turn
+ log.exception(
+ "realtime[%s] the pause hold failed; requesting the response now",
+ self.session_id,
+ )
+ try:
+ await self._request_native_response()
+ except Exception: # noqa: BLE001 — logged above; nothing more to do
+ log.debug("realtime[%s] fallback request failed", self.session_id)
+ finally:
+ if self._turn_pause_waiter is asyncio.current_task():
+ self._turn_pause_waiter = None
+
+ async def _request_native_response(self) -> None:
+ """Ask a manual-response transport for THIS turn's answer, once.
+
+ The one place the request is made after a final input transcript —
+ directly when the pause is already settled, or from the pause waiter.
+ Marks the turn as requested and every folded input item as answered,
+ and lifts the post-barge-in output guard on transports that isolate
+ response generations (the same bookkeeping the inline path did).
+ """
+ if self._response_requested_for_turn:
+ return
+ try:
+ await self._session.request_response(required_tool=None)
+ except TypeError:
+ # Compatibility with third-party realtime adapters built against
+ # the older no-argument protocol.
+ await self._session.request_response()
+ if bool(getattr(self._session, "isolates_response_generations", False)):
+ self._drop_provider_output_until_new_response = False
+ self._response_requested_for_turn = True
+ held = self._turn_pause_held_input_ids
+ self._turn_pause_held_input_ids = set()
+ for item_id in held:
+ self._note_input_answered(item_id)
+
+ def _note_input_answered(self, input_item_id: str) -> None:
+ """Remember that a provider input item has had its response requested."""
+ if not input_item_id:
+ return
+ self._response_requested_input_ids.add(input_item_id)
+ if len(self._response_requested_input_ids) > _ANSWERED_INPUT_ID_MAX:
+ # Bounded: a long call must not accumulate one entry per
+ # utterance for its whole lifetime.
+ self._response_requested_input_ids = set(
+ tuple(self._response_requested_input_ids)[-_ANSWERED_INPUT_ID_MAX:]
+ )
+
def owes_the_user_a_reply(self) -> bool:
"""True while a turn is being worked on and nothing is audible yet.
@@ -3225,10 +3480,11 @@ class RealtimeVoiceSession:
return
if not pcm16:
return
+ self._last_input_frame_monotonic = time.monotonic()
if not self._output_active and _pcm16_peak(pcm16) >= _USER_VOICE_PEAK:
# Measured on the frame we are about to FORWARD, so the floor
# tracks exactly the audio the provider is judging.
- self._last_voiced_input_monotonic = time.monotonic()
+ self._last_voiced_input_monotonic = self._last_input_frame_monotonic
target_session = self._session
try:
await asyncio.wait_for(
@@ -4242,38 +4498,37 @@ class RealtimeVoiceSession:
event.is_final
and input_observed
and not self._response_requested_for_turn
+ and not self._delegate_required_for_turn
+ and not bool(
+ getattr(
+ self._session,
+ "creates_responses_automatically",
+ False,
+ )
+ )
+ and not self._turn_pause_settled()
+ ):
+ # The provider closed its input turn, but the
+ # microphone says the user is still talking (or has
+ # not been quiet for the Thinking pause yet). On a
+ # transport whose responses Jarvis requests itself
+ # the request simply WAITS: nothing is submitted, the
+ # turn stays open, and a later final appends to it —
+ # so ONE response answers the whole request instead
+ # of a first answer talking over the user's second
+ # half (maintainer directive 2026-08-18). The turn is
+ # NOT marked as requested here — that is what lets the
+ # next final of this turn re-enter this branch.
+ self._hold_native_response_for_pause(input_item_id)
+ elif (
+ event.is_final
+ and input_observed
+ and not self._response_requested_for_turn
):
if not self._delegate_required_for_turn:
- try:
- await self._session.request_response(
- required_tool=None
- )
- except TypeError:
- # Compatibility with third-party realtime adapters
- # built against the older no-argument protocol.
- await self._session.request_response()
- if bool(
- getattr(
- self._session,
- "isolates_response_generations",
- False,
- )
- ):
- self._drop_provider_output_until_new_response = False
+ await self._request_native_response()
self._response_requested_for_turn = True
- if input_item_id:
- self._response_requested_input_ids.add(input_item_id)
- if (
- len(self._response_requested_input_ids)
- > _ANSWERED_INPUT_ID_MAX
- ):
- # Bounded: a long call must not accumulate one
- # entry per utterance for its whole lifetime.
- self._response_requested_input_ids = set(
- tuple(self._response_requested_input_ids)[
- -_ANSWERED_INPUT_ID_MAX:
- ]
- )
+ self._note_input_answered(input_item_id)
elif event.type == "handoff_requested" and (
self._stale_generation_drop_active()
):
@@ -4655,6 +4910,27 @@ class RealtimeVoiceSession:
)
self._deferred_provider_speech_start = True
self._arm_interruption_settle()
+ elif (
+ event.type == "speech_started"
+ and self._turn_held_for_pause()
+ and not self._output_active
+ and self._gate.pending_audio_ms <= 0
+ ):
+ # The user resumed INSIDE the Thinking pause: no response
+ # was requested for this turn yet and nothing is playing,
+ # so there is no reply to cut and no reason to close the
+ # turn. Keep it open — the next final appends to it and
+ # the held request answers the whole request. Closing here
+ # (the barge-in path below) would split one spoken request
+ # into two recorded turns and cost the second half its
+ # first half in the answer.
+ self._user_speech_active = True
+ log.info(
+ "realtime[%s] the user resumed inside the Thinking "
+ "pause — keeping the turn open for the rest of the "
+ "sentence",
+ self.session_id,
+ )
elif event.type == "speech_started":
await self._begin_user_speech_turn()
await self._barge_in(interrupt_provider=True)
@@ -7479,6 +7755,9 @@ class RealtimeVoiceSession:
self._output_language_retry_task = None
if retry_task is not None and not retry_task.done():
retry_task.cancel()
+ # A response held for the Thinking pause belongs to the turn that is
+ # closing; the next turn's own final requests its own response.
+ self._cancel_turn_pause_waiter()
self._turn_id = ""
self._turn_trace_id = None
self._latency_tracker = None
@@ -10858,6 +11137,7 @@ class RealtimeVoiceSession:
self._cancel_turn_stall_watchdog()
self._cancel_interruption_settle()
self._cancel_tool_transcript_wait()
+ self._cancel_turn_pause_waiter()
if self._end_call_timer is not None and not self._end_call_timer.done():
self._end_call_timer.cancel()
self._end_call_timer = None
diff --git a/jarvis/speech/continuation_buffer.py b/jarvis/speech/continuation_buffer.py
index 20921cebdb..2756353644 100644
--- a/jarvis/speech/continuation_buffer.py
+++ b/jarvis/speech/continuation_buffer.py
@@ -44,6 +44,14 @@ logger = logging.getLogger(__name__)
_DEFAULT_TIMEOUT_S: Final[float] = 8.0
_DEFAULT_MAX_CHAIN: Final[int] = 3
+#: ``last_reason`` of a fragment held by :meth:`ContinuationBuffer.hold` — the
+#: text was COMPLETE, but the microphone reported the user speaking again
+#: before it could be dispatched (the pipeline's Thinking-pause hold,
+#: 2026-08-18). Distinct from every ``completion.REASON_*`` value so the
+#: pipeline can tell a mic hold from a syntactic one (no clarifying question
+#: for it, an immediate release on a VAD false start).
+REASON_MIC_RESUMED: Final[str] = "mic_resumed"
+
class ContinuationBuffer:
"""Holds an open-ended voice fragment until the continuation arrives."""
@@ -96,6 +104,41 @@ class ContinuationBuffer:
"""
return self._last_reason if self._fragments else ""
+ def hold(self, text: str, language: str = "") -> None:
+ """Hold a COMPLETE utterance because the user is already talking again.
+
+ The pipeline's Thinking-pause hold (2026-08-18): the final transcript
+ of one utterance arrived while the VAD already reported the next one
+ starting. Dispatching now would answer half a request and let the
+ instant ack talk over the user's second half; the next utterance's
+ ``process()`` joins the held text with it instead — the same append
+ the syntactic hold does, without any judgement about the words. Uses
+ the buffer's ordinary deadline so a hold whose continuation never
+ finalizes is still dropped before an unrelated later turn; the
+ pipeline's drain timer dispatches it long before that.
+
+ Replaces nothing: a fragment already held (a syntactic hold, or a
+ previous mic hold) is joined into one entry, so a chain of holds still
+ counts as ONE fragment against ``max_chain`` and never flushes early
+ just because the user paused twice.
+ """
+ cleaned = " ".join(str(text or "").split())
+ if not cleaned:
+ return
+ del language # accepted for symmetry with ``process``; unused
+ if self._fragments:
+ self._fragments = [" ".join([*self._fragments, cleaned])]
+ else:
+ self._fragments = [cleaned]
+ self._last_reason = REASON_MIC_RESUMED
+ self._deadline = time.monotonic() + self._timeout_s
+ logger.info(
+ "ContinuationBuffer: holding a complete utterance — the microphone "
+ "says the user is already speaking again (%d chars, deadline=+%.1fs)",
+ len(self._fragments[0]),
+ self._timeout_s,
+ )
+
def discard(self) -> None:
"""Drop the buffer unconditionally. Called by hangup / cancel paths."""
if self._fragments:
diff --git a/jarvis/speech/pipeline.py b/jarvis/speech/pipeline.py
index f516a1948c..41d01490c9 100644
--- a/jarvis/speech/pipeline.py
+++ b/jarvis/speech/pipeline.py
@@ -127,7 +127,7 @@ from jarvis.speech.completion import (
is_cancel,
is_incomplete,
)
-from jarvis.speech.continuation_buffer import ContinuationBuffer
+from jarvis.speech.continuation_buffer import REASON_MIC_RESUMED, ContinuationBuffer
from jarvis.speech.continuation_window import ContinuationWindow
from jarvis.speech.echo_guard import SelfEchoGuard
from jarvis.speech.hangup import (
@@ -1407,6 +1407,27 @@ _DRAIN_HOLDS_FLOOR: frozenset[TurnTakingState] = frozenset({
TurnTakingState.WAITING_FOR_FINAL_TRANSCRIPT,
})
+# The pipeline's Thinking-pause hold (maintainer directive 2026-08-18: "wait
+# for a clear pause; when I keep talking, append"). The VAD already waits the
+# configured pause before it ends an utterance — but its final TRANSCRIPT
+# arrives a recognizer round-trip later, and by then the user may audibly be
+# into the next sentence. Dispatching at that moment answers half a request
+# and lets the instant ack talk over the user's second half; the recombine
+# window then cancels and re-dispatches the joined text — a wasted brain call
+# and an audible stumble. So a complete utterance whose final lands while the
+# VAD reports speech again is HELD (``ContinuationBuffer.hold``) and joined
+# with the next utterance instead: one dispatch, no interruption. Bounded by
+# the drain timer, which defers only while the user holds the floor.
+#
+# How long a mic-held text waits for its continuation once the floor is free
+# again (the next utterance transcribed to nothing, or never came). Short:
+# the VAD's own pause has already been paid; this only bridges the gap
+# between "the floor is free" and "nothing more is coming".
+_MIC_HOLD_DRAIN_S = 1.0
+# A VAD false start (a cough, a chair) is a promise of speech that never came:
+# release the held text at once — a frame later than the false-start verdict.
+_MIC_HOLD_RELEASE_S = 0.15
+
# Hang-up patterns + the END_CALL sentinel live in jarvis/speech/hangup.py
# (shared, stdlib-only, also imported by jarvis/telephony/session.py). HANGUP_RE,
@@ -3524,6 +3545,11 @@ class SpeechPipeline:
def _on_vad_speech_start(self) -> None:
log.info("voice activity start")
self._schedule_turn_state(TurnTakingState.USER_SPEAKING)
+ # The microphone's own word on "the user is talking again" — read by
+ # the turn handler when a final transcript lands (Thinking-pause hold).
+ # Cleared at the next endpoint, so it always describes speech AFTER
+ # the utterance currently being transcribed.
+ self._vad_speech_after_endpoint = True
# A resumed utterance freezes the continuation grace so a slow follow-up
# still recombines with the just-finished turn (session 71f2d2de). The
# SAME freeze must reach the pre-dispatch ContinuationBuffer: a fragment
@@ -3547,6 +3573,7 @@ class SpeechPipeline:
def _on_vad_silence_cancel(self) -> None:
log.info("silence timer cancel")
self._schedule_turn_state(TurnTakingState.USER_SPEAKING)
+ self._vad_speech_after_endpoint = True
def _on_vad_endpoint(self, reason: str) -> None:
log.info("voice activity stop: reason=%s", reason)
@@ -3558,6 +3585,8 @@ class SpeechPipeline:
# same field is also exposed as a C-signal to a future completeness
# classifier ("max_utterance" = hard-chopped utterance).
self._last_endpoint_reason = reason
+ # This utterance is over; whatever speech follows is the NEXT one.
+ self._vad_speech_after_endpoint = False
self._cancel_stale_probe()
self._reset_probe_state()
if reason != "false_start":
@@ -3573,6 +3602,10 @@ class SpeechPipeline:
TurnTakingState.LISTENING,
only_from=TurnTakingState.USER_SPEAKING,
)
+ # ...and a text held for that promised speech is released now:
+ # nothing is coming, the user gets their answer at once instead
+ # of after the drain grace.
+ self._release_mic_hold_soon()
def _reset_probe_state(self) -> None:
self._probe_last_text = ""
@@ -12945,6 +12978,10 @@ class SpeechPipeline:
)
log.info("→ Transcribing (%.1f KB) via %s …", len(pcm) / 1024, utt_stt_name)
await self._set_turn_state(TurnTakingState.WAITING_FOR_FINAL_TRANSCRIPT)
+ # Only speech the VAD reports from HERE on counts as "the user resumed"
+ # for the Thinking-pause hold — a stale flag from a path that never
+ # endpointed (push-to-talk, a hotkey call) must not hold this turn.
+ self._vad_speech_after_endpoint = False
transcript = await self._transcribe_final(pcm)
if transcript is None:
# AD-OE6 zero-silent-drop: every retry of the final transcription
@@ -13122,6 +13159,18 @@ class SpeechPipeline:
)
text = coalesced
+ # Thinking-pause hold: the words are complete, but the MICROPHONE says
+ # the user is already talking again (the VAD reported speech after
+ # this utterance's endpoint, while the recognizer was still working).
+ # Dispatching now would answer half a request and let the instant ack
+ # talk over the second half; hold the text and let the next utterance
+ # join it — one dispatch for the whole request. Bounded: a drain timer
+ # dispatches the held text once the floor is free, and a VAD false
+ # start releases it at once (``_release_mic_hold_soon``).
+ if self._mic_says_user_resumed():
+ self._hold_for_resumed_speech(text, lang)
+ return True
+
# Privacy-Voice-Toggle (Wave-2 B7): matcht Privacy-Phrasen aus Config,
# pausiert/resumed den VisionContextProvider und spricht kurzen ACK
# BEVOR das Brain aufgerufen wird. Brain-Call wird uebersprungen.
@@ -13744,7 +13793,83 @@ class SpeechPipeline:
# --- Continuation drain timer (autonomous flush; AD-OE6 zero-silent-drop) - #
- def _arm_continuation_drain(self, lang: str) -> None:
+ def _mic_says_user_resumed(self) -> bool:
+ """The VAD reported speech AFTER the utterance now being finalized.
+
+ Two readings of the same microphone, either suffices: the explicit
+ flag ``_on_vad_speech_start`` / ``_on_vad_silence_cancel`` raise and
+ ``_on_vad_endpoint`` clears, and the turn state itself, which those
+ callbacks move to USER_SPEAKING (the transcription of THIS utterance
+ runs in WAITING_FOR_FINAL_TRANSCRIPT, so USER_SPEAKING at final time
+ can only mean a newer start).
+ """
+ if bool(getattr(self, "_vad_speech_after_endpoint", False)):
+ return True
+ return (
+ getattr(self, "_turn_state", TurnTakingState.IDLE)
+ is TurnTakingState.USER_SPEAKING
+ )
+
+ def _hold_for_resumed_speech(self, text: str, lang: str) -> None:
+ """Park a complete utterance until the sentence the user began finishes.
+
+ The next final joins it through the ordinary ``ContinuationBuffer``
+ path; the drain timer (deferred while the floor is held) dispatches it
+ if that final never carries words; a VAD false start releases it at
+ once. The turn state is left to the VAD — it already reads
+ USER_SPEAKING, which is the truth.
+ """
+ buf = getattr(self, "_continuation_buffer", None)
+ if buf is None:
+ return
+ try:
+ buf.hold(text, language=lang)
+ except Exception: # noqa: BLE001 — a hold that fails must not lose the turn
+ log.warning("Thinking-pause hold failed; dispatching as-is", exc_info=True)
+ return
+ # A recombine earlier this turn parked the prior dispatched text for a
+ # history drop that ``_arm_continuation`` would apply at dispatch. The
+ # held text WILL dispatch (joined, or drained) — outside this turn, so
+ # apply the drop now; leaving it would let the brain see the prior
+ # fragment twice, once as its own turn and once inside the join.
+ prior = getattr(self, "_continuation_pending_drop", None)
+ if prior:
+ self._continuation_pending_drop = None
+ brain = getattr(self, "_brain", None)
+ if brain is not None and hasattr(brain, "drop_last_turn"):
+ try:
+ brain.drop_last_turn(prior)
+ except Exception: # noqa: BLE001 — history hygiene never crashes the turn
+ log.debug("drop_last_turn failed (non-fatal)", exc_info=True)
+ log.info(
+ "⏸ Thinking-pause hold: the user is already speaking again — holding "
+ "%r for the rest of the sentence",
+ text[:80],
+ )
+ self._arm_continuation_drain(lang, delay_s=_MIC_HOLD_DRAIN_S)
+
+ def _release_mic_hold_soon(self) -> None:
+ """A promised continuation will not come (VAD false start): drain now."""
+ buf = getattr(self, "_continuation_buffer", None)
+ if buf is None or not buf.has_pending():
+ return
+ if getattr(buf, "last_reason", "") != REASON_MIC_RESUMED:
+ return
+ # The language the hold was armed with (always set by the hold); the
+ # single resolver decides otherwise, exactly as a fresh turn would.
+ lang = getattr(self, "_mic_hold_language", "")
+ if not lang:
+ try:
+ lang = self._output_language(None, "")
+ except Exception: # noqa: BLE001 — never let a release crash the callback
+ lang = ""
+ log.info(
+ "Thinking-pause hold: the speech that held the text was a false "
+ "start — releasing it now"
+ )
+ self._arm_continuation_drain(lang, delay_s=_MIC_HOLD_RELEASE_S)
+
+ def _arm_continuation_drain(self, lang: str, *, delay_s: float | None = None) -> None:
"""Arm an autonomous drain timer for a silently-held continuation fragment.
The ``ContinuationBuffer`` holds an open-ended fragment with NO timer of
@@ -13763,7 +13888,9 @@ class SpeechPipeline:
utterance arrives (``_handle_utterance``); deferred while the user holds
the floor so it never pre-empts a continuation in progress. Fail-open: a
missing event loop (sync teardown / tests) is a no-op. The drain delay
- matches the buffer's own discard deadline (``timeout_s``).
+ matches the buffer's own discard deadline (``timeout_s``) unless the
+ caller names a shorter one (the Thinking-pause hold, whose pause the
+ VAD already paid).
"""
self._cancel_continuation_drain()
buf = getattr(self, "_continuation_buffer", None)
@@ -13773,9 +13900,14 @@ class SpeechPipeline:
loop = asyncio.get_running_loop()
except RuntimeError:
return
+ # Remember the language of the held text so a later release from a
+ # sync VAD callback (no ``lang`` in hand) drains it in the same one.
+ self._mic_hold_language = lang
# buf is non-None here (checked above) and ``timeout_s`` is a guaranteed
# property — match the drain delay to the buffer's own discard deadline.
- delay_s = max(0.05, float(buf.timeout_s))
+ if delay_s is None:
+ delay_s = float(buf.timeout_s)
+ delay_s = max(0.05, float(delay_s))
self._continuation_drain_task = loop.create_task(
self._continuation_drain_fire(delay_s, lang),
name="continuation-drain",
@@ -13815,7 +13947,9 @@ class SpeechPipeline:
"(state=%s); re-arming so it can coalesce.",
self._turn_state.name,
)
- self._arm_continuation_drain(lang)
+ # Same delay as before: a short Thinking-pause release stays short
+ # across deferrals instead of growing into the full buffer grace.
+ self._arm_continuation_drain(lang, delay_s=delay_s)
return
fragment = buf.flush_pending()
if not fragment:
diff --git a/jarvis/ui/web/frontend/src/i18n/locales/de.json b/jarvis/ui/web/frontend/src/i18n/locales/de.json
index 4fd80fa00f..7a1c63b869 100644
--- a/jarvis/ui/web/frontend/src/i18n/locales/de.json
+++ b/jarvis/ui/web/frontend/src/i18n/locales/de.json
@@ -1590,7 +1590,7 @@
"settings_view": {
"silence_window": {
"title": "Denkpause",
- "description": "Wie lange {name} in Stille wartet, bevor er das Gesagte abschickt. Höher stellen, um mehr Zeit zum Innehalten und Zu-Ende-Denken zu haben. Gilt für den Pipeline-Sprachmodus — im Realtime-Modus erkennt das Sprachmodell das Ende deines Satzes selbst.",
+ "description": "Wie lange {name} in Stille wartet, bevor er deinen Satz übernimmt. Höher stellen, um mehr Zeit zum Innehalten und Zu-Ende-Denken zu haben; sprichst du innerhalb der Pause weiter, wird das Gesagte an dieselbe Anfrage angehängt. Gilt für beide Sprachmodi — Pipeline sofort, Realtime ab dem nächsten Satz (manche Sprachmodelle ab dem nächsten Anruf).",
"unit_seconds": "s",
"reset": "Auf Standard zurücksetzen (1,5 s)",
"saved_toast": "Denkpause auf {0} gesetzt.",
diff --git a/jarvis/ui/web/frontend/src/i18n/locales/en.json b/jarvis/ui/web/frontend/src/i18n/locales/en.json
index 1751c86485..4ec8667266 100644
--- a/jarvis/ui/web/frontend/src/i18n/locales/en.json
+++ b/jarvis/ui/web/frontend/src/i18n/locales/en.json
@@ -1590,7 +1590,7 @@
"settings_view": {
"silence_window": {
"title": "Thinking pause",
- "description": "How long {name} waits in silence before it sends what you said. Raise it to give yourself more time to pause and finish a thought. Applies to the Pipeline voice mode — in Realtime mode the voice model detects the end of your turn itself.",
+ "description": "How long {name} waits in silence before it takes your turn. Raise it to give yourself more time to pause and finish a thought; if you keep talking inside the pause, the words are added to the same request. Applies to both voice modes — Pipeline right away, Realtime on the next turn (some voice models on the next call).",
"unit_seconds": "s",
"reset": "Reset to default (1.5 s)",
"saved_toast": "Thinking pause set to {0}.",
diff --git a/jarvis/ui/web/frontend/src/i18n/locales/es.json b/jarvis/ui/web/frontend/src/i18n/locales/es.json
index 7fa08a474c..c241d1180b 100644
--- a/jarvis/ui/web/frontend/src/i18n/locales/es.json
+++ b/jarvis/ui/web/frontend/src/i18n/locales/es.json
@@ -1590,7 +1590,7 @@
"settings_view": {
"silence_window": {
"title": "Pausa para pensar",
- "description": "Cuánto espera {name} en silencio antes de enviar lo que dijiste. Súbelo para darte más tiempo para pausar y terminar una idea. Se aplica al modo de voz Pipeline — en el modo Realtime, el modelo de voz detecta por sí mismo el final de tu turno.",
+ "description": "Cuánto espera {name} en silencio antes de tomar tu turno. Súbelo para darte más tiempo para pausar y terminar una idea; si sigues hablando dentro de la pausa, lo dicho se añade a la misma petición. Se aplica a ambos modos de voz — Pipeline de inmediato, Realtime a partir del siguiente turno (algunos modelos de voz a partir de la siguiente llamada).",
"unit_seconds": "s",
"reset": "Restablecer al valor predeterminado (1,5 s)",
"saved_toast": "Pausa para pensar fijada en {0}.",
diff --git a/jarvis/ui/web/frontend/src/views/settings/SilenceWindowGroup.tsx b/jarvis/ui/web/frontend/src/views/settings/SilenceWindowGroup.tsx
index 8388d10f03..bdb9462b01 100644
--- a/jarvis/ui/web/frontend/src/views/settings/SilenceWindowGroup.tsx
+++ b/jarvis/ui/web/frontend/src/views/settings/SilenceWindowGroup.tsx
@@ -5,14 +5,16 @@ import { useEventStore } from "@/store/events";
import { useT } from "@/i18n";
/**
- * "Thinking pause" slider inside the Settings view. Tunes the voice endpoint
- * silence window (how long Jarvis waits in silence before submitting). Range
- * 0.5–5.0 s, step 0.1 s, default 1.5 s. The label tracks the slider live; the
- * PUT fires on release (pointer/key up) so a 0.1 s-step drag does not storm the
- * backend. The change persists to jarvis.toml and applies live to the classic
- * pipeline ONLY (a headless host falls back to "applies on next start").
- * Realtime sessions deliberately ignore it — the realtime model's native turn
- * detection decides the turn end (maintainer directive 2026-07-21).
+ * "Thinking pause" slider inside the Settings view. Tunes how long Jarvis
+ * waits in silence before it takes the user's turn — ONE value for both voice
+ * engines (maintainer directive 2026-08-18; the pipeline-only scope of
+ * 2026-07-21 is history). Range 0.5–5.0 s, step 0.1 s, default 1.5 s. The
+ * label tracks the slider live; the PUT fires on release (pointer/key up) so
+ * a 0.1 s-step drag does not storm the backend. The change persists to
+ * jarvis.toml and applies live to the classic pipeline; a realtime session
+ * reads it on its next turn (a self-answering voice model bakes it into its
+ * own turn detection at open, so there it lands on the next call). A headless
+ * host falls back to "applies on next start".
*/
export function SilenceWindowGroup() {
const t = useT();
diff --git a/tests/unit/realtime/test_gemini_live.py b/tests/unit/realtime/test_gemini_live.py
index 4ab82380d2..c3fffeeab6 100644
--- a/tests/unit/realtime/test_gemini_live.py
+++ b/tests/unit/realtime/test_gemini_live.py
@@ -611,16 +611,62 @@ async def test_every_selectable_model_uses_live_audio_and_transcriptions(
assert holder["client"].closed is True
+async def test_thinking_pause_becomes_geminis_silence_window(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The user's Thinking pause is the ONE lever on a self-answering transport.
+
+ Gemini answers on its own activity boundary — Jarvis cannot hold a reply
+ back once the turn is closed — so "wait for a clear pause before you take
+ the turn" (maintainer 2026-08-18) can only be Gemini's own silence window.
+ A user who resumes inside it continues the same activity: the words
+ append, nothing is submitted twice. LOW sensitivity stays alongside.
+ """
+ from google.genai import types
+
+ holder = _patch_genai_client(monkeypatch)
+ provider = GeminiLiveProvider(api_key="test-key")
+
+ session = await provider.open_session(
+ RealtimeSessionConfig(voice="Puck", turn_pause_ms=1_500)
+ )
+ _selected, config = holder["client"].aio.live.connect_calls[0]
+ detection = config.realtime_input_config.automatic_activity_detection
+ assert detection.silence_duration_ms == 1_500
+ assert detection.end_of_speech_sensitivity == types.EndSensitivity.END_SENSITIVITY_LOW
+ assert detection.disabled is False
+ await session.close()
+
+
+async def test_explicit_silence_window_outranks_the_thinking_pause(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ holder = _patch_genai_client(monkeypatch)
+ provider = GeminiLiveProvider(api_key="test-key")
+
+ session = await provider.open_session(
+ RealtimeSessionConfig(
+ voice="Puck", silence_duration_ms=2_700, turn_pause_ms=1_500
+ )
+ )
+ _selected, config = holder["client"].aio.live.connect_calls[0]
+ detection = config.realtime_input_config.automatic_activity_detection
+ assert detection.silence_duration_ms == 2_700
+ await session.close()
+
+
@pytest.mark.asyncio
async def test_default_config_asks_gemini_to_sit_through_a_pause(
monkeypatch: pytest.MonkeyPatch,
) -> None:
- """A default session is patient about pauses without a forced window.
+ """Without a configured pause the session is patient, not fixed-windowed.
Live 2026-08-13 16:46/16:47: Gemini's own default read a mid-sentence
pause as end-of-turn and a coding pane was briefed with a quarter of the
- sentence. LOW fixes that; silence_duration_ms must STAY unset, because a
- fixed window taxes every short utterance (directive 2026-07-21).
+ sentence. LOW fixes that; with no ``turn_pause_ms`` and no explicit
+ override, silence_duration_ms stays unset (the provider's own timing).
"""
from google.genai import types
diff --git a/tests/unit/realtime/test_openai_realtime.py b/tests/unit/realtime/test_openai_realtime.py
index d194a1b96f..c831e23af4 100644
--- a/tests/unit/realtime/test_openai_realtime.py
+++ b/tests/unit/realtime/test_openai_realtime.py
@@ -173,12 +173,36 @@ async def test_default_config_keeps_native_server_vad_window(
)
payload = holder["client"].realtime.last_conn.session_updates[0]
turn_detection = payload["audio"]["input"]["turn_detection"]
- # No forced window: OpenAI's native server-VAD default decides the turn
- # end (the Settings "Thinking pause" endpoints the pipeline only).
+ # No forced window: OpenAI's native server-VAD default commits the input;
+ # the user's Thinking pause is waited out by the session itself.
assert "silence_duration_ms" not in turn_detection
await session.close()
+async def test_thinking_pause_stays_off_the_server_vad(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """A manual-response transport must not double-wait the user's pause.
+
+ ``create_response`` is False here, so the SESSION applies the Thinking
+ pause on the microphone before it requests the response. Forwarding the
+ same value as a server silence window would make every finished sentence
+ wait twice (the lag the 2026-07-21 directive rejected).
+ """
+ holder = _patch_openai_client(monkeypatch)
+ provider = OpenAIRealtimeProvider(api_key="test-key")
+
+ session = await provider.open_session(
+ RealtimeSessionConfig(voice="echo", language="en", turn_pause_ms=2_700)
+ )
+ payload = holder["client"].realtime.last_conn.session_updates[0]
+ turn_detection = payload["audio"]["input"]["turn_detection"]
+ assert "silence_duration_ms" not in turn_detection
+ assert turn_detection["create_response"] is False
+ await session.close()
+
+
@pytest.mark.asyncio
async def test_text_update_creates_tool_free_audio_response(
monkeypatch: pytest.MonkeyPatch,
diff --git a/tests/unit/realtime/test_thinking_pause.py b/tests/unit/realtime/test_thinking_pause.py
index e4544ba454..3249dec565 100644
--- a/tests/unit/realtime/test_thinking_pause.py
+++ b/tests/unit/realtime/test_thinking_pause.py
@@ -1,65 +1,114 @@
-"""The Settings thinking pause must NOT leak into realtime sessions.
+"""The Thinking pause is ONE setting for both voice engines (2026-08-18).
-The "Thinking pause" (SpeechConfig.vad_silence_ms) endpoints the classic
-pipeline only. Realtime providers own their native turn detection — forcing
-the pipeline window into a realtime session made the model wait the full
-window after every utterance ("done speaking but still listening",
-maintainer directive 2026-07-21).
+``SpeechConfig.vad_silence_ms`` — the Settings → Voice slider — used to
+endpoint the classic pipeline only (directive 2026-07-21: a fixed window on
+every realtime transport read as "done speaking but still listening"). On
+2026-08-18 the maintainer asked for the opposite: wait for a clear pause
+before the turn is taken, and when the user keeps talking, append instead of
+submitting twice. So the same value now reaches every realtime session, and
+which lever applies is a transport CAPABILITY:
+
+* a transport that answers on its own boundary folds the pause into its
+ native turn detection (``RealtimeSessionConfig.turn_pause_ms``);
+* a transport whose responses Jarvis requests itself is held by the SESSION:
+ a final input transcript does not request the response until the
+ microphone has been quiet for the whole pause, and every final that lands
+ meanwhile appends to the open turn — one response for the whole request.
"""
from __future__ import annotations
+import array
import asyncio
+import time
from types import SimpleNamespace
+from typing import Any
import pytest
+from jarvis.realtime import session as session_module
+from jarvis.realtime.protocol import RealtimeEvent
from jarvis.realtime.session import RealtimeVoiceSession
+_FIRST = "What is the capital"
+_SECOND = "of Peru, and how many people live there"
+
+
+def _voiced_frame(peak: int = 6_000, samples: int = 320) -> bytes:
+ """One 20 ms 16 kHz frame loud enough to count as the user's voice."""
+ return array.array("h", [peak, -peak] * (samples // 2)).tobytes()
+
+
+def _silent_frame(samples: int = 320) -> bytes:
+ return array.array("h", [0] * samples).tobytes()
-class _Session:
- session_id = "thinking-pause"
+
+class _ManualWire:
+ """A manual-response transport (OpenAI-shaped) driven event by event."""
+
+ session_id = "manual-wire"
creates_responses_automatically = False
isolates_response_generations = True
- async def send_audio(self, _chunk):
- return None
+ def __init__(self) -> None:
+ self.events: asyncio.Queue[RealtimeEvent | None] = asyncio.Queue()
+ self.requests: list[float] = []
+ self.requested = asyncio.Event()
+ self.interrupts = 0
+
+ def emit(self, event: RealtimeEvent) -> None:
+ self.events.put_nowait(event)
async def receive(self):
- if False:
- yield None
+ while True:
+ event = await self.events.get()
+ if event is None:
+ return
+ yield event
- async def update_session(self, **_kwargs):
+ async def send_audio(self, _chunk: Any) -> None:
return None
- async def request_response(self, **_kwargs):
+ async def update_session(self, **_kwargs: Any) -> None:
return None
- async def send_text(self, _text):
- return None
+ async def request_response(self, **_kwargs: Any) -> None:
+ self.requests.append(time.monotonic())
+ self.requested.set()
- async def truncate(self, _audio_end_ms):
+ async def send_text(self, _text: str) -> None:
return None
- async def interrupt(self):
+ async def truncate(self, _audio_end_ms: int) -> None:
return None
- async def send_tool_result(self, _call_id, _name, _result):
- return None
+ async def interrupt(self, **_kwargs: Any) -> None:
+ self.interrupts += 1
- async def close(self):
+ async def send_tool_result(self, *_args: Any) -> None:
return None
+ async def close(self) -> None:
+ self.events.put_nowait(None)
+
+
+class _AutoWire(_ManualWire):
+ """A self-answering transport (Gemini-shaped): no request to hold."""
+
+ session_id = "auto-wire"
+ creates_responses_automatically = True
+
class _Provider:
supports_realtime = True
input_sample_rate = 16_000
output_sample_rate = 24_000
- def __init__(self, name: str, *, fail: bool = False) -> None:
+ def __init__(self, name: str, *, wire: _ManualWire | None = None, fail: bool = False) -> None:
self.name = name
self.fail = fail
self.opened_with = None
+ self.wire = wire or _ManualWire()
async def can_open_duplex_session(self):
return True
@@ -68,7 +117,7 @@ class _Provider:
self.opened_with = config
if self.fail:
raise RuntimeError("simulated provider outage")
- return _Session()
+ return self.wire
def _config(silence_ms: int):
@@ -81,22 +130,275 @@ def _config(silence_ms: int):
)
-async def test_pipeline_pause_never_reaches_primary_or_fallback():
- primary = _Provider("first-family", fail=True)
- fallback = _Provider("second-family")
- session = RealtimeVoiceSession(
- session_id="thinking-pause",
+def _build(provider: _Provider, *, pause_ms: int, session_id: str) -> RealtimeVoiceSession:
+ return RealtimeVoiceSession(
+ session_id=session_id,
send_binary=lambda _data: asyncio.sleep(0),
send_json=lambda _message: asyncio.sleep(0),
- providers=[primary, fallback],
- config=_config(2_700),
+ providers=[provider],
+ config=_config(pause_ms),
+ bus=None,
+ browser_sample_rate=16_000,
+ surface="desktop",
)
+
+def _final(text: str, item_id: str) -> RealtimeEvent:
+ return RealtimeEvent(type="input_transcript", text=text, is_final=True, item_id=item_id)
+
+
+def _fast_pause(monkeypatch: pytest.MonkeyPatch) -> None:
+ """Scale the pause and its safety caps down so the tests stay sub-second."""
+ monkeypatch.setattr(session_module, "_TURN_PAUSE_MIN_MS", 50)
+ monkeypatch.setattr(session_module, "_USER_SPEAKING_HOLD_S", 0.12)
+ monkeypatch.setattr(session_module, "_MIC_HOLD_STALE_TRANSCRIPT_S", 0.6)
+ monkeypatch.setattr(session_module, "_MIC_HOLD_ABSOLUTE_CAP_S", 10.0)
+
+
+async def test_pause_reaches_every_provider_family_as_turn_pause_ms():
+ """Primary and cross-family fallback both open with the user's pause.
+
+ ``silence_duration_ms`` stays None — that is the raw override, not the
+ setting. The pause travels as ``turn_pause_ms`` and each transport applies
+ the lever it has (native window, or the session-side hold).
+ """
+ primary = _Provider("first-family", fail=True)
+ fallback = _Provider("second-family")
+ session = _build(primary, pause_ms=2_700, session_id="thinking-pause")
+ session._providers = [primary, fallback]
+
await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ try:
+ assert primary.opened_with.turn_pause_ms == 2_700
+ assert fallback.opened_with.turn_pause_ms == 2_700
+ assert primary.opened_with.silence_duration_ms is None
+ assert fallback.opened_with.silence_duration_ms is None
+ finally:
+ await session.end(reason="test")
- # Even with a long pipeline Thinking pause configured, every realtime
- # provider family opens with the provider-native default (None).
- assert primary.opened_with.silence_duration_ms is None
- assert fallback.opened_with.silence_duration_ms is None
- await session.end(reason="test")
+
+def test_pause_is_clamped_to_the_settings_bounds():
+ session = _build(_Provider("p"), pause_ms=99_999, session_id="clamp-high")
+ assert session._turn_pause_ms() == session_module._TURN_PAUSE_MAX_MS
+ session = _build(_Provider("p"), pause_ms=1, session_id="clamp-low")
+ assert session._turn_pause_ms() == session_module._TURN_PAUSE_MIN_MS
+ session = _build(_Provider("p"), pause_ms=1_500, session_id="clamp-none")
+ session._config = SimpleNamespace(brain=SimpleNamespace(reply_language="en"))
+ assert session._turn_pause_ms() == session_module._TURN_PAUSE_DEFAULT_MS
+
+
+async def test_a_final_while_the_user_talks_on_waits_and_answers_once(
+ _fast_pause: None,
+) -> None:
+ """The provider committed mid-sentence; the response waits for the pause.
+
+ The user keeps the floor, the provider's second commit lands, and only
+ once the microphone has been quiet for the whole pause is ONE response
+ requested — for both halves, on the same turn.
+ """
+ provider = _Provider("manual")
+ wire = provider.wire
+ session = _build(provider, pause_ms=200, session_id="held-turn")
+
+ speaking = asyncio.Event()
+ speaking.set()
+ stop = asyncio.Event()
+
+ async def _microphone() -> None:
+ # A real microphone never stops reporting: voice while the user
+ # talks, silent frames afterwards.
+ while not stop.is_set():
+ frame = _voiced_frame() if speaking.is_set() else _silent_frame()
+ await session.handle_audio_frame(frame)
+ await asyncio.sleep(0.02)
+
+ await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ mic = asyncio.create_task(_microphone())
+ try:
+ await asyncio.sleep(0.05)
+ wire.emit(_final(_FIRST, "item-1"))
+ # Generous: the first final of a session pays its lazy imports.
+ await asyncio.wait_for(_until(session._turn_held_for_pause), timeout=3.0)
+ assert wire.requests == [], "requested while the user was still talking"
+ first_turn = session._turn_id
+ assert first_turn
+
+ # The server VAD's own "the user started again" edge inside the pause
+ # must NOT split the turn — nothing is playing, there is nothing to cut.
+ wire.emit(RealtimeEvent(type="speech_started"))
+ await asyncio.sleep(0.1)
+ assert session._turn_id == first_turn
+ assert wire.interrupts == 0
+
+ wire.emit(_final(_SECOND, "item-2"))
+ await asyncio.sleep(0.3)
+ assert wire.requests == [], "requested before the pause settled"
+ assert session._last_user_text == f"{_FIRST} {_SECOND}"
+ assert session._turn_held_for_pause() is True
+
+ speaking.clear()
+ quiet_at = time.monotonic()
+ await asyncio.wait_for(wire.requested.wait(), timeout=2.0)
+ waited = wire.requests[0] - quiet_at
+ # Requested only after the pause (0.2 s), never before it — and not
+ # long after it either (no stream-gone bound was needed: the mic kept
+ # reporting silence).
+ assert 0.15 <= waited < 1.0, f"answered {waited:.3f}s after the mic went quiet"
+ await asyncio.sleep(0.1)
+ assert len(wire.requests) == 1, "one response for the whole request"
+ assert session._response_requested_for_turn is True
+ assert {"item-1", "item-2"} <= session._response_requested_input_ids
+ assert session._turn_id == first_turn
+ assert session._turn_held_for_pause() is False
+ finally:
+ stop.set()
+ if not mic.done():
+ mic.cancel()
+ await session.end(reason="test")
+
+
+async def _until(predicate) -> None: # noqa: ANN001 - test helper
+ while not predicate(): # noqa: ASYNC110 - polling a plain predicate in a test
+ await asyncio.sleep(0.02)
+
+
+async def test_a_quiet_microphone_answers_at_once(_fast_pause: None) -> None:
+ """No voice on the mic = no evidence to hold on: the request is immediate.
+
+ This is the degradation for a quiet talker below the peak gate and for
+ text-only surfaces — the pause can delay a request while the user audibly
+ talks on, it can never deafen the assistant.
+ """
+ provider = _Provider("manual")
+ wire = provider.wire
+ session = _build(provider, pause_ms=5_000, session_id="quiet-mic")
+ await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ try:
+ emitted_at = time.monotonic()
+ wire.emit(_final(_FIRST, "item-1"))
+ await asyncio.wait_for(wire.requested.wait(), timeout=1.0)
+ assert wire.requests[0] - emitted_at < 0.5
+ assert session._turn_held_for_pause() is False
+ finally:
+ await session.end(reason="test")
+
+
+async def test_a_settled_pause_answers_at_once(_fast_pause: None) -> None:
+ """The user spoke, then was quiet longer than the pause: no extra wait.
+
+ The microphone keeps reporting SILENT frames after the words — that is
+ what makes the old voiced stamp trustworthy (a stream that merely stalled
+ proves nothing, see ``_MIC_FRAME_STALL_S``).
+ """
+ provider = _Provider("manual")
+ wire = provider.wire
+ session = _build(provider, pause_ms=100, session_id="settled")
+
+ stop = asyncio.Event()
+
+ async def _quiet_room() -> None:
+ while not stop.is_set():
+ await session.handle_audio_frame(_silent_frame())
+ await asyncio.sleep(0.02)
+
+ await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ room = asyncio.create_task(_quiet_room())
+ try:
+ await session.handle_audio_frame(_voiced_frame())
+ await asyncio.sleep(0.3)
+ emitted_at = time.monotonic()
+ wire.emit(_final(_FIRST, "item-1"))
+ await asyncio.wait_for(wire.requested.wait(), timeout=2.0)
+ # The first final of a session pays its lazy imports on the loop; the
+ # bound only has to prove there was no pause-shaped wait on top.
+ assert wire.requests[0] - emitted_at < 1.0
+ assert session._turn_held_for_pause() is False
+ finally:
+ stop.set()
+ if not room.done():
+ room.cancel()
+ await session.end(reason="test")
+
+
+async def test_a_stalled_frame_stream_does_not_pass_for_silence(
+ _fast_pause: None,
+) -> None:
+ """Loop hiccup: the stamp is old because nothing was PROCESSED, not heard.
+
+ Frames flowed, then stopped arriving for longer than a frame interval —
+ the user may be mid-word with the frames still queued. The pause holds
+ until the stream reports again; once it does with silence, it settles.
+ """
+ provider = _Provider("manual")
+ session = _build(provider, pause_ms=100, session_id="stalled-stream")
+ await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ try:
+ await session.handle_audio_frame(_voiced_frame())
+ await asyncio.sleep(0.4) # older than the pause AND the stall margin
+ assert session._turn_pause_settled() is False
+ # The stream catches up with silence: the old voiced stamp is real.
+ await session.handle_audio_frame(_silent_frame())
+ assert session._turn_pause_settled() is True
+ finally:
+ await session.end(reason="test")
+
+
+async def test_a_loud_but_wordless_floor_still_gets_its_answer(
+ _fast_pause: None,
+) -> None:
+ """A stuck floor costs a bounded delay, never the answer (AP-30)."""
+ provider = _Provider("manual")
+ wire = provider.wire
+ session = _build(provider, pause_ms=200, session_id="stuck-floor")
+
+ stop_speaking = asyncio.Event()
+
+ async def _hold_the_floor() -> None:
+ while not stop_speaking.is_set():
+ await session.handle_audio_frame(_voiced_frame())
+ await asyncio.sleep(0.02)
+
+ await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ floor = asyncio.create_task(_hold_the_floor())
+ try:
+ wire.emit(_final(_FIRST, "item-1"))
+ # The mic stays loud, no new words ever arrive: the stale window
+ # (0.6 s here) releases the request while the floor is still "held".
+ await asyncio.wait_for(wire.requested.wait(), timeout=3.0)
+ assert len(wire.requests) == 1
+ finally:
+ stop_speaking.set()
+ if not floor.done():
+ floor.cancel()
+ await session.end(reason="test")
+
+
+async def test_a_self_answering_transport_is_never_held(_fast_pause: None) -> None:
+ """Gemini-shaped wires answer on their own boundary; the hold is not theirs.
+
+ Their pause lives in ``turn_pause_ms`` → native silence window; the
+ session must not park a request it could never make.
+ """
+ provider = _Provider("auto", wire=_AutoWire())
+ wire = provider.wire
+ session = _build(provider, pause_ms=5_000, session_id="auto-answer")
+
+ await session.handle_control({"type": "audio_start", "sample_rate": 16_000})
+ try:
+ for _ in range(5):
+ await session.handle_audio_frame(_voiced_frame())
+ wire.emit(_final(_FIRST, "item-1"))
+ await asyncio.sleep(0.15)
+ assert session._turn_held_for_pause() is False
+ assert session._response_requested_for_turn is True
+ assert "item-1" in session._response_requested_input_ids
+ finally:
+ await session.end(reason="test")
diff --git a/tests/unit/speech/test_thinking_pause_hold.py b/tests/unit/speech/test_thinking_pause_hold.py
new file mode 100644
index 0000000000..6dafbcbb75
--- /dev/null
+++ b/tests/unit/speech/test_thinking_pause_hold.py
@@ -0,0 +1,233 @@
+"""The pipeline's Thinking-pause hold: the microphone outranks the transcript.
+
+Maintainer directive 2026-08-18: "wait for a clear pause before you take the
+turn; when I keep talking, append." The VAD already waits the configured pause
+before it ends an utterance — but its final TRANSCRIPT lands a recognizer
+round-trip later, and by then the user may audibly be into the next sentence.
+Dispatching at that moment answers half a request and lets the instant ack
+talk over the second half. So a COMPLETE utterance whose final arrives while
+the VAD reports speech again is held (``ContinuationBuffer.hold``) and joined
+with the next utterance: one dispatch, no interruption. Bounded by the drain
+timer (short, deferred only while the user holds the floor) and released at
+once on a VAD false start.
+
+Driven against a real ``ContinuationBuffer`` with the ``__new__`` stubbing
+pattern of ``test_continuation_drain``.
+"""
+from __future__ import annotations
+
+import asyncio
+from unittest.mock import MagicMock
+
+import pytest
+
+from jarvis.speech import pipeline as pipeline_module
+from jarvis.speech.continuation_buffer import REASON_MIC_RESUMED, ContinuationBuffer
+from jarvis.speech.pipeline import SpeechPipeline, TurnTakingState
+
+_FIRST = "Wie spät ist es" # i18n-allow: spoken test utterance
+_SECOND = "in Berlin gerade" # i18n-allow: spoken test utterance
+
+
+def _make_pipe(*, buffer_timeout_s: float = 8.0) -> SpeechPipeline:
+ pipe = SpeechPipeline.__new__(SpeechPipeline)
+ pipe._continuation_buffer = ContinuationBuffer(timeout_s=buffer_timeout_s)
+ pipe._continuation_drain_task = None
+ pipe._clarify_timer_task = None
+ pipe._turn_state = TurnTakingState.WAITING_FOR_FINAL_TRANSCRIPT
+ pipe._vad_speech_after_endpoint = False
+ pipe._continuation_pending_drop = None
+ pipe._brain = None
+
+ voice_cfg = MagicMock()
+ voice_cfg.clarify_incomplete_enabled = False
+ cfg = MagicMock()
+ cfg.voice = voice_cfg
+ pipe._config = cfg
+
+ pipe._dispatched: list[tuple[str, str]] = []
+ pipe._state_history: list[TurnTakingState] = []
+
+ async def _fake_dispatch(text: str, lang: str = "de") -> None:
+ pipe._dispatched.append((text, lang))
+
+ async def _fake_set_turn_state(
+ state: TurnTakingState, *, only_from: TurnTakingState | None = None
+ ) -> None:
+ if only_from is not None and pipe._turn_state is not only_from:
+ return
+ pipe._state_history.append(state)
+ pipe._turn_state = state
+
+ pipe._handle_flushed_pending_text = _fake_dispatch # type: ignore[method-assign]
+ pipe._set_turn_state = _fake_set_turn_state # type: ignore[method-assign]
+ return pipe
+
+
+# --------------------------------------------------------------------------- #
+# The microphone's word #
+# --------------------------------------------------------------------------- #
+
+
+def test_no_hold_without_microphone_evidence() -> None:
+ pipe = _make_pipe()
+ assert pipe._mic_says_user_resumed() is False
+
+
+def test_a_vad_start_after_the_endpoint_means_the_user_resumed() -> None:
+ pipe = _make_pipe()
+ pipe._on_vad_speech_start()
+ assert pipe._mic_says_user_resumed() is True
+
+
+def test_a_silence_cancel_after_the_endpoint_means_the_user_resumed() -> None:
+ pipe = _make_pipe()
+ pipe._on_vad_silence_cancel()
+ assert pipe._mic_says_user_resumed() is True
+
+
+def test_user_speaking_state_alone_is_enough() -> None:
+ """The turn state is moved to USER_SPEAKING by the same VAD callbacks."""
+ pipe = _make_pipe()
+ pipe._turn_state = TurnTakingState.USER_SPEAKING
+ assert pipe._mic_says_user_resumed() is True
+
+
+async def test_the_endpoint_clears_the_resumed_flag() -> None:
+ """Whatever the VAD heard BEFORE this utterance ended is not a resume."""
+ pipe = _make_pipe()
+ pipe._on_vad_speech_start()
+ pipe._on_vad_endpoint("silence")
+ await asyncio.sleep(0)
+ assert bool(pipe._vad_speech_after_endpoint) is False
+
+
+# --------------------------------------------------------------------------- #
+# The hold itself #
+# --------------------------------------------------------------------------- #
+
+
+async def test_hold_joins_with_the_next_utterance() -> None:
+ """The held first half and the second half dispatch as ONE text."""
+ pipe = _make_pipe()
+ pipe._on_vad_speech_start() # the user talks on while the STT worked
+ pipe._hold_for_resumed_speech(_FIRST, "de")
+ try:
+ buf = pipe._continuation_buffer
+ assert buf.has_pending()
+ assert buf.last_reason == REASON_MIC_RESUMED
+ assert pipe._continuation_drain_task is not None
+ # ...the next final arrives, as ``_handle_utterance_turn`` processes it:
+ pipe._cancel_continuation_drain()
+ joined = buf.process(_SECOND, language="de")
+ assert joined == f"{_FIRST} {_SECOND}"
+ assert not buf.has_pending()
+ assert pipe._dispatched == [], "the join dispatches inline, never twice"
+ finally:
+ pipe._cancel_continuation_drain()
+
+
+async def test_hold_releases_at_once_on_a_vad_false_start(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The speech that held the text was a cough: the answer comes right away."""
+ monkeypatch.setattr(pipeline_module, "_MIC_HOLD_DRAIN_S", 5.0) # would be slow
+ monkeypatch.setattr(pipeline_module, "_MIC_HOLD_RELEASE_S", 0.05)
+ pipe = _make_pipe()
+ pipe._on_vad_speech_start()
+ pipe._turn_state = TurnTakingState.USER_SPEAKING
+ pipe._hold_for_resumed_speech(_FIRST, "de")
+ try:
+ pipe._on_vad_endpoint("false_start")
+ await asyncio.sleep(0.3)
+ assert pipe._dispatched == [(_FIRST, "de")]
+ assert not pipe._continuation_buffer.has_pending()
+ finally:
+ pipe._cancel_continuation_drain()
+
+
+async def test_hold_drains_after_the_short_grace_once_the_floor_is_free(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """The next utterance carried no words: the held text is answered anyway."""
+ monkeypatch.setattr(pipeline_module, "_MIC_HOLD_DRAIN_S", 0.08)
+ pipe = _make_pipe()
+ pipe._on_vad_speech_start()
+ await asyncio.sleep(0) # the scheduled USER_SPEAKING lands...
+ pipe._turn_state = TurnTakingState.LISTENING # ...then the follow-up came and went
+ pipe._hold_for_resumed_speech(_FIRST, "de")
+ try:
+ await asyncio.sleep(0.3)
+ assert pipe._dispatched == [(_FIRST, "de")]
+ finally:
+ pipe._cancel_continuation_drain()
+
+
+async def test_hold_waits_while_the_user_speaks_and_stays_short_afterwards(
+ monkeypatch: pytest.MonkeyPatch,
+) -> None:
+ """Deferrals keep the SHORT release; the drain never pre-empts the user."""
+ monkeypatch.setattr(pipeline_module, "_MIC_HOLD_DRAIN_S", 0.06)
+ pipe = _make_pipe()
+ pipe._on_vad_speech_start()
+ pipe._turn_state = TurnTakingState.USER_SPEAKING
+ pipe._hold_for_resumed_speech(_FIRST, "de")
+ try:
+ await asyncio.sleep(0.3) # several grace windows: the user is talking
+ assert pipe._dispatched == []
+ assert pipe._continuation_buffer.has_pending()
+ pipe._turn_state = TurnTakingState.LISTENING
+ await asyncio.sleep(0.25) # well under the 8 s buffer grace
+ assert pipe._dispatched == [(_FIRST, "de")]
+ finally:
+ pipe._cancel_continuation_drain()
+
+
+async def test_hold_applies_a_parked_history_drop() -> None:
+ """A recombine's deferred history drop lands with the hold, not never."""
+ pipe = _make_pipe()
+ brain = MagicMock()
+ pipe._brain = brain
+ pipe._continuation_pending_drop = "the prior dispatched text"
+ pipe._on_vad_speech_start()
+ pipe._hold_for_resumed_speech(f"the prior dispatched text {_SECOND}", "en")
+ try:
+ brain.drop_last_turn.assert_called_once_with("the prior dispatched text")
+ assert pipe._continuation_pending_drop is None
+ finally:
+ pipe._cancel_continuation_drain()
+
+
+# --------------------------------------------------------------------------- #
+# ContinuationBuffer.hold #
+# --------------------------------------------------------------------------- #
+
+
+def test_buffer_hold_folds_into_an_existing_syntactic_fragment() -> None:
+ buf = ContinuationBuffer(timeout_s=8.0, max_chain=2)
+ assert buf.process("Nimm den Bus oder", language="de") is None # i18n-allow: test utterance
+ buf.hold("die Bahn nach Hause", language="de") # i18n-allow: test utterance
+ # ONE fragment, so a chain of holds never trips max_chain early.
+ assert buf.has_pending()
+ assert buf.last_reason == REASON_MIC_RESUMED
+ joined = buf.process("und dann zurück.", language="de") # i18n-allow: test utterance
+ assert joined == "Nimm den Bus oder die Bahn nach Hause und dann zurück." # i18n-allow
+
+
+def test_buffer_hold_ignores_empty_text() -> None:
+ buf = ContinuationBuffer()
+ buf.hold(" ")
+ assert not buf.has_pending()
+ assert buf.last_reason == ""
+
+
+def test_buffer_hold_normalizes_whitespace() -> None:
+ buf = ContinuationBuffer()
+ buf.hold(" what is\nthe time ")
+ assert buf.flush_pending() == "what is the time"