Skip to content

Open source Grok Bot alternative, Bring your own agent and subscription: message Claude Code, Codex, Gemini CLI, Cursor, Copilot and 40+ AI agents as bots on your own computer, from iPhone, Mac, Windows, Linux or terminal. Agent-agnostic and free. Also a Muse and Dots alternative. Built in Rust.

docs: plan host-owned voice calls

leepokai8143f93b4f43
diff --git a/docs/plans/voice-architecture.md b/docs/plans/voice-architecture.md
new file mode 100644
index 00000000..a4189363
--- /dev/null
+++ b/docs/plans/voice-architecture.md
@@ -0,0 +1,71 @@
+# Voice calls owned by the host
+
+Status: in progress (owner's go-ahead, 2026-10-25). Supersedes [voice call transcripts](voice-call-transcripts.md),
+which becomes stage 1 of this plan. Current behavior: [voice call](../features/voice-call.md).
+
+A realtime voice model (Gemini Live, OpenAI Realtime) stays in front of the bot: coding agents are text-only and
+slow, so the voice talks and the bot works. What changes is how it hands over. Today each client (iPhone, Android,
+desktop) has its own `CallOperator` with the instructions, tools and reply hand-off; the voice knows nothing of the
+user, the bot's memory or the chat; it may rephrase or translate what the user said; and nothing of the call is kept.
+
+```
+phone / desktop (audio only) host (all logic) the bot
+ mic ⇄ Gemini / OpenAI ←→ Call: context, tools, transcript ←→ does the work
+ (audio never crosses the host) progress and replies pushed back
+```
+
+## Stages
+
+Each stage ships on its own (after-change, merged into `dev`), all clients together.
+
+### 1. The call on the host
+
+- **`startCall`** `{botId, callId, provider?, model?, voice?, mode}` creates the call and returns everything a
+ client needs: the provider credential (realtime and speech modes), the voice's instructions and the tool
+ declarations. The host writes the instructions: the bot's profile (name, description, group members), what the
+ bot's memory holds for this user (the shared "yours" `MEMORY.md` and the bot's `MEMORY.md`, size-capped), the
+ last few chat messages in spoken form and the bot's status. On-device calls get a call too (no credential).
+- **`callTool`** `{callId, name, arguments}` runs the voice's tool calls on the host: `send_to_bot`, `bot_status`,
+ `recent_messages`, `answer_approval`, `stop_bot`, `search_memory`. The clients' `CallOperator`s go.
+- **`callLines`** `{callId, lines: [{from: user | voice, text, at}]}` appends the transcript as it is spoken (both
+ providers' transcription events; Chinese in the reader's script). Kept on the host, so a crash or a dropped
+ connection loses nothing.
+- **Original words:** `send_to_bot` carries what the user said since the last hand-off, word for word, beside the
+ voice's wording, so the bot sees both and memory quotes only the user.
+- **Hidden in the chat:** everything a call sends to the bot, and the bot's replies in those turns, carry the call's
+ id and stay out of the chat on every client; the bot gets them as usual.
+- **`endCall`** `{callId, seconds}` writes the call's line: a waveform icon, a title a one-shot of the bot's harness
+ writes from the transcript (`chat::oneshot`), and the duration. Tapping it opens the transcript (the user's lines,
+ the voice's lines and the bot's messages, in order). `logCall` stays for older apps.
+- **Memory:** dreaming reads the transcript; the user's spoken lines are the user's own words, the voice's lines and
+ the voice-written `send_to_bot` text are context only. A call's user lines count toward the dream trigger.
+
+### 2. The bot's work reaches the call
+
+- The host pushes what the voice should say to the client holding the call (a `call` event, which older clients
+ ignore): the bot's final replies, approvals waiting, failures, and short progress while it works ("running the
+ tests", "changed three files"), rate-limited. The client hands each to the provider as text. "What are you doing?"
+ is answered from real progress, and a long silence gets a short update.
+- Rules in the instructions: anything about code or the computer goes to the bot; the voice talks, reads results and
+ asks for what's missing, and never invents results.
+
+### 3. One engine state machine
+
+- Every client's engines share the phases: connecting → ready (the provider confirmed the session and the microphone
+ is delivering audio) → listening / speaking, with the connecting line until ready.
+- Drops resume (Gemini session resumption, a fresh OpenAI session with the transcript so far); route changes and
+ interruptions (a phone call, another app's audio) recover without hanging up.
+- Recorded provider event streams replay through each client's engine in tests, including the device cases found so
+ far (#214: the engine reconfigured at start, the player stuck).
+- Call quality in analytics, never content: time to ready, reconnects, a phase stuck too long.
+
+### 4. Platform
+
+- iPhone: CallKit for the call (lock screen, Dynamic Island, AirPods controls, interruptions handled by the system),
+ and the bot's progress in the call's Live Activity.
+- Android: the call's foreground notification shows the same progress.
+
+## Compatibility
+
+New methods and fields are additive. The phone apps need a host with `startCall`, so they raise `minHost` to the
+release that ships stage 1 (the desktop app bundles its host). `logCall` stays until `minApp` passes it.