Skip to content

Open source Grok Bot alternative, Bring your own agent and subscription: message Claude Code, Codex, Gemini CLI, Cursor, Copilot and 40+ AI agents as bots on your own computer, from iPhone, Mac, Windows, Linux or terminal. Agent-agnostic and free. Also a Muse and Dots alternative. Built in Rust.

fix(apps): calls and dictation say when speech recognition fails

leepokaif851b9fe27b5
diff --git a/apps/android/WhatsNew.md b/apps/android/WhatsNew.md
index 0355d0df..0d6a0507 100644
--- a/apps/android/WhatsNew.md
+++ b/apps/android/WhatsNew.md
@@ -10,12 +10,14 @@ else a generic "Bug fixes and improvements" line. Delete old sections.
## 2.38.1
### zh-Hant
- 修正與電腦的連線中斷時,app 可能閃退的問題。
+- 語音通話中語音辨識偶爾出錯時,會送出已聽到的內容並繼續聆聽,不再直接結束通話。
- 介面新增巴西葡萄牙文:可在設定選擇「Português (Brasil)」,或跟隨系統的葡萄牙文語言。
- 新增俄文介面:在設定選擇「Русский」,或跟隨系統的俄文語言。
- 用 QR Code 配對時,如果電腦當下沒能加入你的帳號(例如手機連不上 Codync 雲端,或電腦的雲端關著),下次連上時會自動補上。
### en-US
- Fixed a crash when the connection to your computer dropped.
+- A one-off speech recognition error in a voice call now sends what was heard and keeps listening, instead of ending the call.
- Choose Português (Brasil) in Settings, or follow your Portuguese system language.
- Choose Русский in Settings, or follow your Russian system language.
- A computer you pair with a QR code that couldn't join your account at the time (the phone couldn't reach the Codync cloud, or the computer's cloud was off, say) joins it the next time it connects.
diff --git a/apps/android/core/src/main/kotlin/com/codync/android/core/Voice.kt b/apps/android/core/src/main/kotlin/com/codync/android/core/Voice.kt
index 73501c83..3cd3da34 100644
--- a/apps/android/core/src/main/kotlin/com/codync/android/core/Voice.kt
+++ b/apps/android/core/src/main/kotlin/com/codync/android/core/Voice.kt
@@ -79,7 +79,7 @@ interface VoiceAudio {
/** Serial audio ownership: no late recognizer/TTS callback can submit or restart a retired call. */
class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settings: VoiceSettings,
- private val send: (String) -> Unit) : VoiceEngine {
+ private val clock: () -> Long = { System.nanoTime() / 1_000_000 }, private val send: (String) -> Unit) : VoiceEngine {
private sealed interface Event {
data class Input(val value: VoiceInput) : Event
data class Submit(val generation: Long, val revision: Long) : Event
@@ -97,6 +97,8 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
override val state = mutableState.asStateFlow()
private var generation = 0L; private var revision = 0L; private var utterance = 0L
private var silence: Job? = null
+ /** When the current recognition started (null while not listening), and how many in a row failed right after. */
+ private var listenedAt: Long? = null; private var quickFailures = 0
private val replies = ArrayDeque<String>()
init { scope.launch {
try {
@@ -132,6 +134,8 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
override suspend fun awaitEnd() { job.join() }
private suspend fun stopListening() {
+ // A recognition that ran a while works, however it ended.
+ listenedAt?.let { if (clock() - it >= QUICK_FAILURE_MS) quickFailures = 0 }; listenedAt = null
generation++; revision++; silence?.cancel(); silence = null
audio.stopListening()
mutableState.update { it.copy(level = 0f) }
@@ -139,7 +143,7 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
private suspend fun listen() {
mutableState.update { it.copy(phase = VoicePhase.Listening, heard = "", level = 0f) }
if (state.value.muted) return
- generation++
+ generation++; listenedAt = clock()
audio.listen(generation) { post(Event.Input(it)) }
}
private suspend fun speakNext() {
@@ -160,6 +164,15 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
if (text.isNotEmpty()) send(text)
listen() // Bot work never stops listening to the next utterance.
}
+ /**
+ * The recognizer stopped with an error: what was heard goes and it listens again, as on the iPhone, unless it failed
+ * within [QUICK_FAILURE_MS] of listening [FAILURES_IN_A_ROW] times running (listening again won't help).
+ */
+ private suspend fun recognitionFailed(message: String) {
+ val startedAt = listenedAt ?: return
+ quickFailures = if (clock() - startedAt < QUICK_FAILURE_MS) quickFailures + 1 else 0
+ if (quickFailures >= FAILURES_IN_A_ROW) fail(message) else submit()
+ }
private suspend fun fail(message: String) {
stopListening(); utterance++; replies.clear(); audio.stopSpeaking()
mutableState.update { it.copy(phase = VoicePhase.Failed, error = message) }
@@ -170,6 +183,7 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
is Event.Input -> when (val input = event.value) {
is VoiceInput.Text -> if (input.generation == generation && state.value.phase == VoicePhase.Listening && !state.value.muted) {
require(input.value.length <= 16 * 1024)
+ if (input.value.isNotBlank()) quickFailures = 0
if (input.value != state.value.heard) {
mutableState.update { it.copy(heard = input.value) }; revision++; silence?.cancel()
val owner = generation; val version = revision
@@ -181,7 +195,7 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
}
is VoiceInput.Level -> if (input.generation == generation && state.value.phase == VoicePhase.Listening && !state.value.muted)
mutableState.update { it.copy(level = if (input.value.isFinite()) input.value.coerceIn(0f, 1f) else 0f) }
- is VoiceInput.Failed -> if (input.generation == generation) fail(input.message)
+ is VoiceInput.Failed -> if (input.generation == generation) recognitionFailed(input.message)
}
is Event.Submit -> if (event.generation == generation && event.revision == revision && state.value.phase == VoicePhase.Listening && !state.value.muted) submit()
is Event.Reply -> {
@@ -214,4 +228,5 @@ class VoiceSession(parent: CoroutineScope, private val audio: VoiceAudio, settin
}
}
}
+ private companion object { const val QUICK_FAILURE_MS = 2_000L; const val FAILURES_IN_A_ROW = 3 }
}
diff --git a/apps/android/core/src/test/kotlin/com/codync/android/core/VoiceTest.kt b/apps/android/core/src/test/kotlin/com/codync/android/core/VoiceTest.kt
index e143360f..2db56b94 100644
--- a/apps/android/core/src/test/kotlin/com/codync/android/core/VoiceTest.kt
+++ b/apps/android/core/src/test/kotlin/com/codync/android/core/VoiceTest.kt
@@ -32,6 +32,7 @@ class VoiceTest {
override suspend fun stopSpeaking() { speakerStops++ }
override suspend fun close() { closed++; if (failedClose) error("Fixture close failure") }
fun text(value: String, final: Boolean = false, owner: Long = generation) { input?.invoke(VoiceInput.Text(owner, value, final)) }
+ fun failed() { input?.invoke(VoiceInput.Failed(generation, "Couldn't hear you.")) }
}
@Test fun markdownBecomesSpeechAndChunksKeepSurrogatePairsIntact() {
val text = SpokenText.from("# Title\n\n**Hello**, [friend](https://example.com) and `code`.\n```sh\nsecret command\n```\n| A | B |\n|---|---|\n| one | two |")
@@ -43,7 +44,7 @@ class VoiceTest {
}
@Test fun unchangedPartialResultsDoNotDelaySubmissionAndLateFinalsCannotResend() = runTest {
val audio = Audio(); val sent = mutableListOf<String>()
- val session = VoiceSession(backgroundScope, audio, VoiceSettings(), sent::add)
+ val session = VoiceSession(backgroundScope, audio, VoiceSettings(), send = sent::add)
runCurrent(); val original = audio.generation
audio.text(" first utterance "); runCurrent(); advanceTimeBy(1_000)
audio.text(" first utterance "); runCurrent(); advanceTimeBy(500); runCurrent()
@@ -55,9 +56,28 @@ class VoiceTest {
assertEquals(VoicePhase.Listening, session.state.value.phase)
session.end(); runCurrent(); session.awaitEnd(); assertEquals(1, audio.closed)
}
+ @Test fun aRecognizerErrorSendsWhatWasHeardAndOnlyRepeatedQuickOnesEndTheCall() = runTest {
+ val audio = Audio(); val sent = mutableListOf<String>()
+ val session = VoiceSession(backgroundScope, audio, VoiceSettings(), { testScheduler.currentTime }, sent::add)
+ fun failed() { audio.failed(); runCurrent() }
+ runCurrent()
+ audio.text("turn on the lights"); runCurrent(); failed(); failed()
+ assertEquals(listOf("turn on the lights"), sent)
+ // A recognition the call stopped after a while worked: the count starts over.
+ advanceTimeBy(3_000); session.mute(true); runCurrent(); session.mute(false); runCurrent()
+ failed(); failed()
+ // So does one that failed only after a while.
+ advanceTimeBy(3_000); failed()
+ failed(); failed()
+ assertEquals(VoicePhase.Listening, session.state.value.phase)
+ failed()
+ assertEquals(VoicePhase.Failed, session.state.value.phase); assertEquals("Couldn't hear you.", session.state.value.error)
+ assertEquals(listOf("turn on the lights"), sent)
+ session.end(); runCurrent(); session.awaitEnd()
+ }
@Test fun muteAndAudioInterruptionDiscardPartialSpeechAndRespectMuteOnReturn() = runTest {
val audio = Audio(); val sent = mutableListOf<String>()
- val session = VoiceSession(backgroundScope, audio, VoiceSettings(), sent::add)
+ val session = VoiceSession(backgroundScope, audio, VoiceSettings(), send = sent::add)
runCurrent(); val original = audio.generation
audio.text("Never send this"); runCurrent(); session.mute(true); runCurrent()
audio.text("Never send this", true, original); advanceTimeBy(3_000); runCurrent()
diff --git a/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceEngine.swift b/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceEngine.swift
index c96fa647..5f118316 100644
--- a/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceEngine.swift
+++ b/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceEngine.swift
@@ -32,9 +32,7 @@ final class OnDeviceEngine: VoiceEngine {
}
private let send: (String) -> Void
- /// The language picked in the call settings, else the phone's.
- private let recognizer = UserDefaults.standard.string(forKey: OnDeviceEngine.languageKey)
- .flatMap { $0.isEmpty ? nil : SFSpeechRecognizer(locale: Locale(identifier: $0)) } ?? SFSpeechRecognizer()
+ private let recognizer = OnDeviceSpeech.recognizer()
/// Created when the microphone first opens: touching the audio system starts it.
@ObservationIgnored private lazy var engine = AVAudioEngine()
/// The microphone tap is on.
@@ -43,6 +41,9 @@ final class OnDeviceEngine: VoiceEngine {
private var task: SFSpeechRecognitionTask?
/// Bumped per utterance so late callbacks from a cancelled recognition are dropped.
private var generation = 0
+ /// When the current recognition started, and its failures in a row.
+ private var listenedAt = Date.distantPast
+ private var failures = RecognitionFailures()
private var silence: Task<Void, Never>?
private let synthesizer = AVSpeechSynthesizer()
private var speechDone: SpeechDone?
@@ -80,12 +81,10 @@ final class OnDeviceEngine: VoiceEngine {
guard await AVAudioApplication.requestRecordPermission() else {
return fail("Allow the microphone for Codync in Settings to talk to your bots.", .microphone)
}
- guard await Self.speechAuthorized() else {
+ guard await OnDeviceSpeech.authorized() else {
return fail("Allow Speech Recognition for Codync in Settings to talk to your bots.", .microphone)
}
- guard let recognizer, recognizer.isAvailable else {
- return fail("Speech recognition isn't available for this language right now.", .provider)
- }
+ guard let recognizer, recognizer.isAvailable else { return fail(OnDeviceSpeech.unavailable, .provider) }
guard !ended else { return }
do {
try CallAudioSession.activate()
@@ -168,10 +167,7 @@ final class OnDeviceEngine: VoiceEngine {
heard = ""
guard !muted, !ended, let recognizer else { return }
generation += 1
- let request = SFSpeechAudioBufferRecognitionRequest()
- request.shouldReportPartialResults = true
- request.addsPunctuation = true
- if recognizer.supportsOnDeviceRecognition { request.requiresOnDeviceRecognition = true }
+ let request = OnDeviceSpeech.request(for: recognizer)
self.request = request
let input = engine.inputNode
input.removeTap(onBus: 0)
@@ -187,10 +183,12 @@ final class OnDeviceEngine: VoiceEngine {
log.error("audio engine: \(error.localizedDescription, privacy: .public)")
return fail("Couldn't use the microphone: \(error.localizedDescription)", .microphone)
}
+ listenedAt = .now
task = recognizer.recognitionTask(with: request, resultHandler: Self.results(self, generation: generation))
}
private func stopListening() {
+ if task != nil { failures.stopped(after: Date.now.timeIntervalSince(listenedAt)) }
silence?.cancel()
silence = nil
generation += 1
@@ -209,6 +207,7 @@ final class OnDeviceEngine: VoiceEngine {
fileprivate func recognized(_ text: String, final: Bool, generation: Int) {
guard generation == self.generation, phase == .listening else { return }
heard = text
+ if !text.isEmpty { failures.reset() }
silence?.cancel()
if final { return submit() }
silence = Task {
@@ -226,6 +225,14 @@ final class OnDeviceEngine: VoiceEngine {
listen()
}
+ /// The recognizer stopped with an error: what was heard goes and it listens again, unless it keeps failing right
+ /// away (otherwise the call would show listening and never hear anything).
+ fileprivate func recognitionFailed(generation: Int) {
+ guard generation == self.generation, phase == .listening else { return }
+ if failures.failed(after: Date.now.timeIntervalSince(listenedAt)) { return fail(OnDeviceSpeech.unheard, .provider) }
+ submit()
+ }
+
private func finishedSpeaking() {
guard phase == .speaking else { return }
listen()
@@ -254,17 +261,14 @@ final class OnDeviceEngine: VoiceEngine {
}
private nonisolated static func results(_ session: OnDeviceEngine, generation: Int) -> @Sendable (SFSpeechRecognitionResult?, (any Error)?) -> Void {
- { result, _ in
- guard let result else { return }
- let text = result.bestTranscription.formattedString
- let final = result.isFinal
- Task { @MainActor in session.recognized(text, final: final, generation: generation) }
- }
- }
-
- private nonisolated static func speechAuthorized() async -> Bool {
- await withCheckedContinuation { done in
- SFSpeechRecognizer.requestAuthorization { done.resume(returning: $0 == .authorized) }
+ { result, error in
+ let text = result?.bestTranscription.formattedString
+ let final = result?.isFinal ?? false
+ let failed = error != nil
+ Task { @MainActor in
+ if let text { session.recognized(text, final: final, generation: generation) }
+ if failed { session.recognitionFailed(generation: generation) }
+ }
}
}
diff --git a/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceSpeech.swift b/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceSpeech.swift
new file mode 100644
index 00000000..1d8b0d3e
--- /dev/null
+++ b/apps/ios/Kit/Sources/CodyncUI/Call/OnDeviceSpeech.swift
@@ -0,0 +1,59 @@
+import Foundation
+import Speech
+
+/// On-device speech recognition as a call and dictation both use it: the recognizer, its request, the permission, and
+/// what a failure means (the same as on Android).
+enum OnDeviceSpeech {
+ static let unavailable = "Speech recognition isn't available for this language right now."
+ static let unheard = "Couldn't hear you. Check the microphone and Speech Recognition in Settings."
+
+ /// The language picked in the call settings, else the phone's.
+ static func recognizer() -> SFSpeechRecognizer? {
+ UserDefaults.standard.string(forKey: OnDeviceEngine.languageKey)
+ .flatMap { $0.isEmpty ? nil : SFSpeechRecognizer(locale: Locale(identifier: $0)) } ?? SFSpeechRecognizer()
+ }
+
+ /// Words as they're heard, punctuated, recognized on the phone when the recognizer can.
+ static func request(for recognizer: SFSpeechRecognizer) -> SFSpeechAudioBufferRecognitionRequest {
+ let request = SFSpeechAudioBufferRecognitionRequest()
+ request.shouldReportPartialResults = true
+ request.addsPunctuation = true
+ if recognizer.supportsOnDeviceRecognition { request.requiresOnDeviceRecognition = true }
+ return request
+ }
+
+ /// Whether the user lets the app use Speech Recognition (asked the first time).
+ static func authorized() async -> Bool {
+ await withCheckedContinuation { done in
+ SFSpeechRecognizer.requestAuthorization { done.resume(returning: $0 == .authorized) }
+ }
+ }
+
+ /// Only silence was heard ("No speech detected"): dictation just ends, without a message.
+ static func isSilence(_ error: any Error) -> Bool {
+ let error = error as NSError
+ return error.domain == "kAFAssistantErrorDomain" && error.code == 1110
+ }
+}
+
+/// A call's recognitions that fail right after listening starts: `limit` in a row means listening again won't help
+/// (no speech assets, Siri and Dictation off).
+struct RecognitionFailures {
+ private static let quick: TimeInterval = 2
+ private static let limit = 3
+ private var quickInARow = 0
+
+ /// A recognition failed `elapsed` seconds in; true once it's broken.
+ mutating func failed(after elapsed: TimeInterval) -> Bool {
+ quickInARow = elapsed < Self.quick ? quickInARow + 1 : 0
+ return quickInARow >= Self.limit
+ }
+
+ /// The call stopped a recognition `elapsed` seconds in: one that ran a while works.
+ mutating func stopped(after elapsed: TimeInterval) {
+ if elapsed >= Self.quick { quickInARow = 0 }
+ }
+
+ /// Words came through: the recognizer works.
+ mutating func reset() { quickInARow = 0 }
+}
diff --git a/apps/ios/Kit/Sources/CodyncUI/Thread/Dictation.swift b/apps/ios/Kit/Sources/CodyncUI/Thread/Dictation.swift
index 284da4d7..bce565a1 100644
--- a/apps/ios/Kit/Sources/CodyncUI/Thread/Dictation.swift
+++ b/apps/ios/Kit/Sources/CodyncUI/Thread/Dictation.swift
@@ -47,7 +47,8 @@ final class Dictation {
private(set) var startedAt = Date.now
/// The microphone's level while listening, 0…1.
private(set) var level: Float = 0
- /// Why the last dictation ended early (no network or microphone access), until the next.
+ /// Why the last dictation didn't start or ended early (a permission, the microphone, the network, the recognizer),
+ /// until the next.
var failure: String?
var needsKey = false
@ObservationIgnored private lazy var engine = AVAudioEngine()
@@ -148,17 +149,29 @@ final class Dictation {
}
private func recognize(generation: Int, edit: @escaping ((String) -> String) -> Void) async {
- guard await AVAudioApplication.requestRecordPermission(), await Self.authorized(), generation == self.generation else { return }
- let recognizer = UserDefaults.standard.string(forKey: OnDeviceEngine.languageKey)
- .flatMap { $0.isEmpty ? nil : SFSpeechRecognizer(locale: Locale(identifier: $0)) } ?? SFSpeechRecognizer()
- guard let recognizer, recognizer.isAvailable else { return }
- let request = SFSpeechAudioBufferRecognitionRequest()
- request.shouldReportPartialResults = true
- request.addsPunctuation = true
- if recognizer.supportsOnDeviceRecognition { request.requiresOnDeviceRecognition = true }
+ let micAllowed = await AVAudioApplication.requestRecordPermission()
+ guard generation == self.generation else { return }
+ guard micAllowed else {
+ failure = "Allow microphone access in Settings to dictate."
+ return
+ }
+ let speechAllowed = await OnDeviceSpeech.authorized()
+ guard generation == self.generation else { return }
+ guard speechAllowed else {
+ failure = "Allow Speech Recognition in Settings to dictate."
+ return
+ }
+ guard let recognizer = OnDeviceSpeech.recognizer(), recognizer.isAvailable else {
+ failure = OnDeviceSpeech.unavailable
+ return
+ }
+ let request = OnDeviceSpeech.request(for: recognizer)
guard Self.openMic(engine, bufferSize: 1024, tap: { _ in OnDeviceEngine.tap(request) { [weak self] level in
Task { @MainActor in if self?.listening == true { self?.level = level } }
- } }) else { return }
+ } }) else {
+ failure = "The microphone isn't available."
+ return
+ }
self.edit = edit
self.request = request
startedAt = .now
@@ -263,17 +276,15 @@ final class Dictation {
{ result, error in
let text = result?.bestTranscription.formattedString
let final = result?.isFinal ?? false
+ // Silence just ends it; any other failure says why instead of ending without a word.
+ let failure = error.flatMap { OnDeviceSpeech.isSilence($0) ? nil : OnDeviceSpeech.unheard }
let failed = error != nil
Task { @MainActor in
if let text { dictation.recognized(text, final: final, generation: generation) }
- if failed, generation == dictation.generation { dictation.stop() }
+ guard failed, generation == dictation.generation else { return }
+ dictation.failure = failure
+ dictation.stop()
}
}
}
-
- private nonisolated static func authorized() async -> Bool {
- await withCheckedContinuation { done in
- SFSpeechRecognizer.requestAuthorization { done.resume(returning: $0 == .authorized) }
- }
- }
}
diff --git a/apps/ios/Kit/Tests/CodyncUITests/OnDeviceSpeechTests.swift b/apps/ios/Kit/Tests/CodyncUITests/OnDeviceSpeechTests.swift
new file mode 100644
index 00000000..5545506e
--- /dev/null
+++ b/apps/ios/Kit/Tests/CodyncUITests/OnDeviceSpeechTests.swift
@@ -0,0 +1,21 @@
+import Foundation
+import Testing
+@testable import CodyncUI
+
+/// A call's recognizer is broken only once it fails right after listening three times running: words, a failure later
+/// on, or a recognition the call stopped after a while start the count over. Dictation ends quietly on silence alone.
+@Test func aRecognizerIsBrokenOnlyWhenItFailsRightAwayThreeTimesRunning() {
+ var failures = RecognitionFailures()
+ func failed(after elapsed: TimeInterval...) -> [Bool] { elapsed.map { failures.failed(after: $0) } }
+ #expect(failed(after: 0.5, 0.5) == [false, false])
+ failures.stopped(after: 1)
+ #expect(failed(after: 0.5) == [true])
+
+ failures.reset()
+ #expect(failed(after: 0.5, 0.5) == [false, false])
+ failures.stopped(after: 30)
+ #expect(failed(after: 0.5, 5, 0.5, 0.5) == [false, false, false, false])
+
+ #expect(OnDeviceSpeech.isSilence(NSError(domain: "kAFAssistantErrorDomain", code: 1110)))
+ #expect(!OnDeviceSpeech.isSilence(NSError(domain: "kLSRErrorDomain", code: 201)))
+}
diff --git a/apps/ios/WhatsNew.md b/apps/ios/WhatsNew.md
index d9070aed..77491f4a 100644
--- a/apps/ios/WhatsNew.md
+++ b/apps/ios/WhatsNew.md
@@ -9,12 +9,14 @@ a version without one gets a generic "Bug fixes and improvements" line. Delete o
## 2.38.1
### zh-Hant
- 修正語音通話中,bot 唸完回覆、要再聽你說話時 app 可能閃退的問題。
+- 語音辨識出問題時,語音通話和聽寫會告訴你,不再一直停在聆聽或默默結束。
- 介面新增巴西葡萄牙文:可在設定選擇「Português (Brasil)」,或跟隨系統的葡萄牙文語言。
- 新增俄文介面:在設定選擇「Русский」,或跟隨系統的俄文語言。
- 用 QR Code 配對時,如果電腦當下沒能加入你的帳號(例如手機連不上 Codync 雲端,或電腦的雲端關著),下次連上時會自動補上。
### en-US
- Fixed a crash in voice calls when the bot finished reading a reply and started listening again.
+- When speech recognition stops working, voice calls and dictation tell you, instead of listening forever or ending without a word.
- Choose Português (Brasil) in Settings, or follow your Portuguese system language.
- Choose Русский in Settings, or follow your Russian system language.
- A computer you pair with a QR code that couldn't join your account at the time (the phone couldn't reach the Codync cloud, or the computer's cloud was off, say) joins it the next time it connects.
diff --git a/docs/features/voice-call.md b/docs/features/voice-call.md
index b514f1c6..4a150579 100644
--- a/docs/features/voice-call.md
+++ b/docs/features/voice-call.md
@@ -86,8 +86,11 @@ on-device (its window doesn't reach the host).
the window that lost it (`stopped`) and ignores that window's stop.
- iPhone (`Thread/Dictation.swift`, pill in `Thread/RecordingPill.swift`, menu in
`Thread/DictationEngineMenu.swift`, dots shared with the call bar in `LevelDots.swift`):
- `SFSpeechRecognizer` on device; with no microphone route (some simulators) the mic does nothing.
- A provider's errors (no key, no connection) show in a dialog.
+ `SFSpeechRecognizer` on device, set up as the call sets it up (`Call/OnDeviceSpeech.swift`). Why it couldn't
+ start or stopped early shows in the *Couldn't dictate* dialog: a denied permission, no microphone route (some
+ simulators), no recognizer for the language, a recognizer error ("Couldn't hear you. Check the microphone and
+ Speech Recognition in Settings."), and a provider's errors (no key, no connection). A pause with nothing said ends
+ it without a dialog, as on Android.
- Android (`Dictation.kt`, pill and engine menu in `DictationControls.kt`, dots shared with the call
bar): Android's on-device recognizer (Android 12+) in the call's language. Where the phone has none,
or it lacks the language, the phone's own recognizer listens instead, and that one may send the
@@ -365,6 +368,12 @@ carries audio and talks to these methods:
resume." (as Android does). Route changes (AirPods) restart the iPhone's audio engine (`AVAudioEngineConfigurationChange`);
the desktop restarts the microphone (and moves OpenAI's track) on `devicechange`. The desktop's on-device engine
uses the speech helper's own microphone and isn't restarted.
+- On-device recognition that stops with an error mid-call sends what was heard and listens again. The iPhone
+ (`OnDeviceEngine`, counted by `RecognitionFailures`) and Android (`VoiceSession`) fail the call with the error
+ (iPhone: "Couldn't hear you. Check the microphone and Speech Recognition in Settings.", `provider`) after three in
+ a row that each failed within 2 s of listening (no speech assets, Siri and Dictation off); words heard, or a
+ recognition that ran longer, start the count over. The desktop's on-device engine fails when the speech helper gives
+ up again within 1 s of its last restart.
- After the voice's turn, playback that reports no progress for 4 s returns the call to listening, so a stuck player
(#214) can't hold the bot's replies back (`onStall`, counted as stuck `speaking`).
- Each engine of a call sends one `voice_call_quality` when it's replaced or the call ends: milliseconds until ready,
diff --git a/web/content/docs/voice.mdx b/web/content/docs/voice.mdx
index 2d3554fa..67c69a8c 100644
--- a/web/content/docs/voice.mdx
+++ b/web/content/docs/voice.mdx
@@ -58,6 +58,7 @@ Under **Engine**, pick one. The choice is kept on this device. Changed during a
- Each reply is read aloud in the language it's written in, skipping code blocks and formatting. Set **Reading speed** to Slower, Normal or Faster. The microphone pauses while a reply is read.
- When the bot asks for approval, the call tells you what it asks; answer it in the chat. If it runs into a problem, the call tells you that too.
- A reply that arrives while the call briefly loses its connection is read once it's back.
+- If speech recognition keeps failing, the call stops and asks you to check the microphone and speech recognition settings.
In the desktop app, on-device speech is available on macOS only. On Windows and Linux, calls use OpenAI or Gemini; without a key, the call asks you to add one in call settings.
@@ -110,7 +111,7 @@ While it listens, the microphone becomes a recording pill with a stop button, th
The small arrow beside the microphone (**Dictation engine**) chooses **On device**, **OpenAI** or **Gemini**. It's kept on this device, separate from the call engine.
-- **On device:** words appear as they're recognized, in the language set for calls. On the iPhone and Android, stop keeps what was recognized so far. In the desktop app, dictation keeps listening through pauses until you press stop.
+- **On device:** words appear as they're recognized, in the language set for calls. On the iPhone and Android, stop keeps what was recognized so far, and dictation tells you why if it can't start or stops on an error; a pause with nothing said just ends it. In the desktop app, dictation keeps listening through pauses until you press stop.
- **OpenAI or Gemini:** Codync first checks that the computer has the provider's key (if not, voice settings open so you can configure it before anything is recorded). The pill appears once the microphone is open, so nothing you say is lost. Nothing is typed while you record. When you press stop, the pill shows "Transcribing…" and the text lands in the box when it arrives. It uses the **Speech to text** model from Voice chat settings, and the recording counts toward that provider's minutes this month.
If dictation uses a provider that your calls don't, Voice chat settings show a separate section for it ("For dictation in the message box.") with its key status and speech-to-text model.