Open source Grok Bot alternative, Bring your own agent and subscription: message Claude Code, Codex, Gemini CLI, Cursor, Copilot and 40+ AI agents as bots on your own computer, from iPhone, Mac, Windows, Linux or terminal. Agent-agnostic and free. Also a Muse and Dots alternative. Built in Rust.
Rust acpagent-client-protocolai-agentsai-assistantclaudeclaude-codecodexcoding-agentscursorgeminigemini-cligrok-bot-alternativeiosiphonelinuxmacosmcpopen-sourcerustself-hosted
docs: plan host-owned voice calls
leepokai8143f93b4f43
diff --git a/docs/plans/voice-architecture.md b/docs/plans/voice-architecture.mdnew file mode 100644index 00000000..a4189363--- /dev/null+++ b/docs/plans/voice-architecture.md@@ -0,0 +1,71 @@+# Voice calls owned by the host++Status: in progress (owner's go-ahead, 2026-10-25). Supersedes [voice call transcripts](voice-call-transcripts.md),+which becomes stage 1 of this plan. Current behavior: [voice call](../features/voice-call.md).++A realtime voice model (Gemini Live, OpenAI Realtime) stays in front of the bot: coding agents are text-only and+slow, so the voice talks and the bot works. What changes is how it hands over. Today each client (iPhone, Android,+desktop) has its own `CallOperator` with the instructions, tools and reply hand-off; the voice knows nothing of the+user, the bot's memory or the chat; it may rephrase or translate what the user said; and nothing of the call is kept.++```+phone / desktop (audio only) host (all logic) the bot+ mic ⇄ Gemini / OpenAI ←→ Call: context, tools, transcript ←→ does the work+ (audio never crosses the host) progress and replies pushed back+```++## Stages++Each stage ships on its own (after-change, merged into `dev`), all clients together.++### 1. The call on the host++- **`startCall`** `{botId, callId, provider?, model?, voice?, mode}` creates the call and returns everything a+ client needs: the provider credential (realtime and speech modes), the voice's instructions and the tool+ declarations. The host writes the instructions: the bot's profile (name, description, group members), what the+ bot's memory holds for this user (the shared "yours" `MEMORY.md` and the bot's `MEMORY.md`, size-capped), the+ last few chat messages in spoken form and the bot's status. On-device calls get a call too (no credential).+- **`callTool`** `{callId, name, arguments}` runs the voice's tool calls on the host: `send_to_bot`, `bot_status`,+ `recent_messages`, `answer_approval`, `stop_bot`, `search_memory`. The clients' `CallOperator`s go.+- **`callLines`** `{callId, lines: [{from: user | voice, text, at}]}` appends the transcript as it is spoken (both+ providers' transcription events; Chinese in the reader's script). Kept on the host, so a crash or a dropped+ connection loses nothing.+- **Original words:** `send_to_bot` carries what the user said since the last hand-off, word for word, beside the+ voice's wording, so the bot sees both and memory quotes only the user.+- **Hidden in the chat:** everything a call sends to the bot, and the bot's replies in those turns, carry the call's+ id and stay out of the chat on every client; the bot gets them as usual.+- **`endCall`** `{callId, seconds}` writes the call's line: a waveform icon, a title a one-shot of the bot's harness+ writes from the transcript (`chat::oneshot`), and the duration. Tapping it opens the transcript (the user's lines,+ the voice's lines and the bot's messages, in order). `logCall` stays for older apps.+- **Memory:** dreaming reads the transcript; the user's spoken lines are the user's own words, the voice's lines and+ the voice-written `send_to_bot` text are context only. A call's user lines count toward the dream trigger.++### 2. The bot's work reaches the call++- The host pushes what the voice should say to the client holding the call (a `call` event, which older clients+ ignore): the bot's final replies, approvals waiting, failures, and short progress while it works ("running the+ tests", "changed three files"), rate-limited. The client hands each to the provider as text. "What are you doing?"+ is answered from real progress, and a long silence gets a short update.+- Rules in the instructions: anything about code or the computer goes to the bot; the voice talks, reads results and+ asks for what's missing, and never invents results.++### 3. One engine state machine++- Every client's engines share the phases: connecting → ready (the provider confirmed the session and the microphone+ is delivering audio) → listening / speaking, with the connecting line until ready.+- Drops resume (Gemini session resumption, a fresh OpenAI session with the transcript so far); route changes and+ interruptions (a phone call, another app's audio) recover without hanging up.+- Recorded provider event streams replay through each client's engine in tests, including the device cases found so+ far (#214: the engine reconfigured at start, the player stuck).+- Call quality in analytics, never content: time to ready, reconnects, a phase stuck too long.++### 4. Platform++- iPhone: CallKit for the call (lock screen, Dynamic Island, AirPods controls, interruptions handled by the system),+ and the bot's progress in the call's Live Activity.+- Android: the call's foreground notification shows the same progress.++## Compatibility++New methods and fields are additive. The phone apps need a host with `startCall`, so they raise `minHost` to the+release that ships stage 1 (the desktop app bundles its host). `logCall` stays until `minApp` passes it.