Skip to content

Cursor plugin specification and official plugins

TypeScript

feat(grok-voice): add Grok Voice plugin (add-voice, add-dictation, add-read-aloud, debug-voice) (#337)

Eric Zakariasson2b8ae2ee306f
diff --git a/.cursor-plugin/marketplace.json b/.cursor-plugin/marketplace.json
index 7eb2de2f..d7e5a268 100644
--- a/.cursor-plugin/marketplace.json
+++ b/.cursor-plugin/marketplace.json
@@ -73,6 +73,11 @@
"source": "pstack",
"description": "if you want to go fast, go deep first. pstack helps you write less, but higher quality code. rigorous agent workflows you can parallelize with confidence."
},
+ {
+ "name": "grok-voice",
+ "source": "grok-voice",
+ "description": "Add Grok voice to an app: realtime speech-to-speech, speech-to-text dictation, text-to-speech read-aloud, and a log-driven fix loop for voice sessions."
+ },
{
"name": "advisor",
"source": "advisor",
diff --git a/README.md b/README.md
index 5408d433..758e4e97 100644
--- a/README.md
+++ b/README.md
@@ -7,19 +7,20 @@ Official Cursor plugins for popular developer tools, frameworks, and SaaS produc
| `name` | Plugin | Author | Category | `description` (from marketplace) |
|:-------|:-------|:-------|:---------|:-------------------------------------|
| `teaching` | [Teaching](teaching/) | Cursor | Utilities | Skill mapping, practice plans, and learning retrospectives. |
-| `continual-learning` | [Continual Learning](continual-learning/) | Cursor | Developer Tools | Incremental transcript-driven memory updates for AGENTS.md using high-signal bullet points only. |
-| `cursor-team-kit` | [Cursor Team Kit](cursor-team-kit/) | Cursor | Developer Tools | Internal team workflows for CI, code review, shipping, local automation, and verification. |
+| `continual-learning` | [Continual Learning](continual-learning/) | Eric Zakariasson | Developer Tools | Incremental transcript-driven memory updates for AGENTS.md using high-signal bullet points only. |
+| `cursor-team-kit` | [Cursor Team Kit](cursor-team-kit/) | Eric Zakariasson | Developer Tools | Internal team workflows for CI, code review, shipping, local automation, and verification. |
| `thermos` | [Thermos](thermos/) | Cursor | Developer Tools | Thermo-nuclear branch review: deep security/correctness audits, harsh code-quality rubrics, parallel subagents, thermos orchestration, and optional merge-ready PR flows. |
| `create-plugin` | [Create Plugin](create-plugin/) | Cursor | Developer Tools | Scaffold and validate new agent plugins. |
| `ralph-loop` | [Ralph Loop](ralph-loop/) | Cursor | Developer Tools | Iterative self-referential AI loops using the Ralph Wiggum technique. |
| `agent-compatibility` | [Agent Compatibility](agent-compatibility/) | Cursor | Developer Tools | CLI-backed repo compatibility scans plus agents that audit startup, validation, and docs against reality. |
-| `cli-for-agent` | [CLI for Agents](cli-for-agent/) | Cursor | Developer Tools | Patterns for designing CLIs that coding agents can run reliably: flags, help with examples, pipelines, errors, idempotency, dry-run. |
+| `cli-for-agent` | [CLI for Agents](cli-for-agent/) | Eric Zakariasson | Developer Tools | Patterns for designing CLIs that coding agents can run reliably: flags, help with examples, pipelines, errors, idempotency, dry-run. |
| `pr-review-canvas` | [PR Review Canvas](pr-review-canvas/) | Cursor | Developer Tools | Render PR diffs as review canvases grouped by importance. |
| `docs-canvas` | [Docs Canvas](docs-canvas/) | Cursor | Developer Tools | Render documentation as a navigable canvas. |
| `cursor-sdk` | [Cursor SDK](cursor-sdk/) | Cursor | Developer Tools | Build apps, scripts, and automations with the TypeScript SDK. |
| `orchestrate` | [Orchestrate](orchestrate/) | Cursor | Developer Tools | Fan large tasks out across parallel cloud agents with planners, workers, verifiers, and structured handoffs. |
| `pstack` | [pstack](pstack/) | Lauren Tan | Developer Tools | if you want to go fast, go deep first. pstack helps you write less, but higher quality code. rigorous agent workflows you can parallelize with confidence. |
| `advisor` | [Advisor](advisor/) | Cursor | Developer Tools | Consult a stronger model before major decisions, when stuck, and before declaring done. |
+| `grok-voice` | [Grok Voice](grok-voice/) | Eric Zakariasson | Developer Tools | Add Grok voice to an app: realtime speech-to-speech, speech-to-text dictation, text-to-speech read-aloud, and a log-driven fix loop for voice sessions. |
| `gmail` | [Gmail](third_party/gmail/) | Cursor | Productivity | Search, read, draft, and manage email. |
| `google-drive` | [Google Drive](third_party/google-drive/) | Cursor | Productivity | Search, read, create, and share files. |
| `google-calendar` | [Google Calendar](third_party/google-calendar/) | Cursor | Productivity | Search events and schedule meetings. |
diff --git a/cli-for-agent/.cursor-plugin/plugin.json b/cli-for-agent/.cursor-plugin/plugin.json
index c0d3a06a..a03b6e8d 100644
--- a/cli-for-agent/.cursor-plugin/plugin.json
+++ b/cli-for-agent/.cursor-plugin/plugin.json
@@ -4,8 +4,7 @@
"version": "1.0.0",
"description": "Patterns for designing CLIs that coding agents can run reliably: non-interactive flags, layered help with examples, pipelines, actionable errors, idempotency, and dry-run.",
"author": {
- "name": "Cursor",
- "email": "[email protected]"
+ "name": "Eric Zakariasson"
},
"homepage": "https://github.com/cursor/plugins/tree/main/cli-for-agent",
"repository": "https://github.com/cursor/plugins",
diff --git a/continual-learning/.cursor-plugin/plugin.json b/continual-learning/.cursor-plugin/plugin.json
index 8382ba9a..3255db70 100644
--- a/continual-learning/.cursor-plugin/plugin.json
+++ b/continual-learning/.cursor-plugin/plugin.json
@@ -4,8 +4,7 @@
"version": "1.0.0",
"description": "Incrementally learns durable user preferences and workspace facts from transcript changes and keeps AGENTS.md up to date with plain bullet points.",
"author": {
- "name": "Cursor",
- "email": "[email protected]"
+ "name": "Eric Zakariasson"
},
"homepage": "https://github.com/cursor/plugins",
"repository": "https://github.com/cursor/plugins",
diff --git a/cursor-team-kit/.cursor-plugin/plugin.json b/cursor-team-kit/.cursor-plugin/plugin.json
index b642937d..23a67962 100644
--- a/cursor-team-kit/.cursor-plugin/plugin.json
+++ b/cursor-team-kit/.cursor-plugin/plugin.json
@@ -4,8 +4,7 @@
"version": "1.2.0",
"description": "Internal engineering team workflows for CI, code review, shipping, control-cli, control-ui, verify-this, test reliability, code cleanup, and work summaries. Designed to work without requiring third-party service integrations.",
"author": {
- "name": "Cursor",
- "email": "[email protected]"
+ "name": "Eric Zakariasson"
},
"homepage": "https://github.com/cursor/plugins",
"repository": "https://github.com/cursor/plugins",
diff --git a/grok-voice/.cursor-plugin/plugin.json b/grok-voice/.cursor-plugin/plugin.json
new file mode 100644
index 00000000..fff49d04
--- /dev/null
+++ b/grok-voice/.cursor-plugin/plugin.json
@@ -0,0 +1,32 @@
+{
+ "name": "grok-voice",
+ "displayName": "Grok Voice",
+ "version": "0.1.0",
+ "description": "Add Grok voice to an app: realtime speech-to-speech, speech-to-text dictation, text-to-speech read-aloud, and a log-driven fix loop for voice sessions.",
+ "author": {
+ "name": "Eric Zakariasson"
+ },
+ "homepage": "https://github.com/cursor/plugins/tree/main/grok-voice",
+ "repository": "https://github.com/cursor/plugins",
+ "license": "MIT",
+ "category": "developer-tools",
+ "keywords": [
+ "grok",
+ "xai",
+ "voice",
+ "realtime",
+ "speech-to-speech",
+ "speech-to-text",
+ "text-to-speech",
+ "dictation",
+ "tts",
+ "stt"
+ ],
+ "tags": [
+ "voice",
+ "grok",
+ "ai",
+ "developer-tools"
+ ],
+ "skills": "./skills/"
+}
diff --git a/grok-voice/LICENSE b/grok-voice/LICENSE
new file mode 100644
index 00000000..ca2bba77
--- /dev/null
+++ b/grok-voice/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Cursor
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
diff --git a/grok-voice/README.md b/grok-voice/README.md
new file mode 100644
index 00000000..ae939b8a
--- /dev/null
+++ b/grok-voice/README.md
@@ -0,0 +1,16 @@
+# Grok Voice
+
+Cursor plugin with four skills for building voice features on Grok: realtime speech-to-speech, speech-to-text dictation, text-to-speech read-aloud, and a debug loop for voice sessions.
+
+## What it includes
+
+- `add-voice`: build speech-to-speech into an app, or replace an STT-LLM-TTS cascade / OpenAI Realtime with it. Wires the realtime session, safe auth, and the app mic; adds a waveform Voice Mode button to the composer.
+- `add-dictation`: speech-to-text. Batch for a mic button or recorded audio (files, uploads, URLs) with word timestamps, diarization, and subtitles; streaming through a relay for live text.
+- `add-read-aloud`: text-to-speech. Batch MP3 for a speaker button on replies; streaming PCM through a relay so audio starts before the reply finishes.
+- `debug-voice`: proposes a plan, installs a dev-only log pipeline (client logger → local NDJSON) in the app's own language and conventions, then runs the fix loop: match the user's report to log signatures, fix one thing, re-test.
+
+Icon convention across the skills: waveform = voice mode, microphone = dictation, speaker = read aloud.
+
+## When to use it
+
+Use when a user asks to add voice, dictation, or read-aloud to an app with Grok, runs `/add-voice`, `/add-dictation`, `/add-read-aloud`, or `/debug-voice`, or reports that voice mode misbehaves and needs logs to diagnose it.
diff --git a/grok-voice/skills/add-dictation/SKILL.md b/grok-voice/skills/add-dictation/SKILL.md
new file mode 100644
index 00000000..6d42d06f
--- /dev/null
+++ b/grok-voice/skills/add-dictation/SKILL.md
@@ -0,0 +1,164 @@
+---
+name: add-dictation
+description: >-
+ Use when the user runs /add-dictation or wants speech turned into text with
+ Grok speech-to-text: a mic button that dictates into the composer, live
+ captions, or transcribing recorded audio (files, uploads, URLs) with word
+ timestamps, diarization, subtitles, meeting notes. STT, transcribe,
+ transcription. For a voice agent that talks back use /add-voice.
+---
+
+# Add Dictation
+
+Add Grok Speech to Text to an existing app: a mic button that dictates into the composer, live captions, or transcripts of recorded audio. Run on `/add-dictation`, typed **Dictate**, or clear “transcribe” intent. Cursor has no mic; wire the **app**, not the IDE.
+
+## Docs
+
+- https://docs.x.ai/developers/model-capabilities/audio/speech-to-text
+- Pricing (cite docs only): https://docs.x.ai/developers/pricing
+
+## Pick the path
+
+| Need | Path |
+| --- | --- |
+| Tap, speak, tap, text appears. Uploaded files. URLs. | **Batch** `POST https://api.x.ai/v1/stt` (default) |
+| Text appears while speaking: captions, long dictation, push-to-talk | **Streaming** `wss://api.x.ai/v1/stt` through a backend relay |
+
+Batch is the default for a composer mic button: one request, no socket, the key never leaves the server. Go streaming only when the UX needs interim text.
+
+## Auth
+
+- Bearer `XAI_API_KEY`, server side only. The STT docs document no ephemeral-token flow, and browsers cannot set WebSocket headers, so browser streaming goes through your backend relay. Do not invent a token flow.
+- Never put the key in a client bundle. Do not paste keys in chat.
+
+## Steps
+
+1. **Map the app**
+ - Composer or input component, where the text should land (insert at cursor vs replace), server framework, package manager.
+ - The **microphone icon belongs to dictation**. If `/add-voice` is installed, its waveform primary button stays as is; add the mic as a secondary ghost button beside it.
+ - Existing mic capture? If `/add-voice` ran, its PCM capture can feed streaming STT; pass its rate as `sample_rate`. 16 kHz is the model’s native rate; other supported rates (8000, 16000, 22050, 24000, 44100, 48000) are resampled server side.
+
+2. **Batch path (default)**
+ - Client: `MediaRecorder` → `Blob` → `POST` to your own route. The endpoint auto-detects containers (WAV, MP3, OGG, Opus, FLAC, AAC, MP4, M4A, MKV, WebM), so send whatever `MediaRecorder` produces.
+ - Server: forward as `multipart/form-data`. Option fields first, **`file` last**; fields after `file` may be ignored. `file` or `url`, max 500 MB.
+
+```ts
+// server (any runtime with fetch + FormData)
+export async function transcribe(blob: Blob, filename: string) {
+ const form = new FormData();
+ form.append("format", "true"); // written-form numbers/currency; requires language
+ form.append("language", "en");
+ // form.append("keyterm", "Acme"); // repeat per term, ≤100 terms × 50 chars
+ form.append("file", blob, filename); // last
+ const res = await fetch("https://api.x.ai/v1/stt", {
+ method: "POST",
+ headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` },
+ body: form,
+ });
+ if (!res.ok) throw new Error(`STT ${res.status}`); // 400 bad input, 413 >500 MB, 429 back off, 502 url fetch failed, 503 retry
+ return (await res.json()) as {
+ text: string; language: string; duration: number;
+ words?: { text: string; start: number; end: number; speaker?: number }[];
+ channels?: { index: number; text: string; words: unknown[] }[];
+ };
+}
+```
+
+```ts
+// client
+const mime = MediaRecorder.isTypeSupported("audio/webm;codecs=opus") ? "audio/webm;codecs=opus" : "audio/mp4";
+const rec = new MediaRecorder(stream, { mimeType: mime });
+const parts: BlobPart[] = [];
+rec.ondataavailable = (e) => parts.push(e.data);
+rec.onstop = async () => {
+ const fd = new FormData();
+ fd.append("file", new Blob(parts, { type: mime }), "dictation");
+ const { text } = await (await fetch("/api/dictation", { method: "POST", body: fd })).json();
+ insertAtCursor(text);
+};
+rec.start(); // second tap: rec.stop()
+```
+
+3. **Streaming path**
+ - Relay: server holds the key, upgrades the browser socket, forwards binary frames and client control messages up, JSON events down. Build the query string server side.
+
+```ts
+import { WebSocketServer, WebSocket } from "ws";
+
+new WebSocketServer({ port: 8788 }).on("connection", (client) => {
+ const q = new URLSearchParams({ sample_rate: "16000", encoding: "pcm", interim_results: "true", language: "en" });
+ const up = new WebSocket(`wss://api.x.ai/v1/stt?${q}`, { headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` } });
+ up.on("message", (d) => client.send(d.toString())); // transcript.* and error events
+ client.on("message", (d, isBinary) => up.readyState === WebSocket.OPEN && up.send(d, { binary: isBinary })); // audio + finalize/audio.done
+ const end = () => { client.close(); up.close(); };
+ up.on("close", end); up.on("error", end); client.on("close", end);
+});
+```
+
+ - Browser capture: PCM16 little-endian, mono, 16 kHz, **100 ms frames = 3,200 bytes**, raw binary, no base64. Wait for `transcript.created` before sending. `MediaRecorder` output is a container, not raw frames; do not stream it.
+
+```ts
+const ws = new WebSocket(relayUrl); ws.binaryType = "arraybuffer";
+const ctx = new AudioContext({ sampleRate: 16000 }); // if ctx.sampleRate !== 16000, downsample in the worklet
+await ctx.audioWorklet.addModule("/pcm16-worklet.js"); // Float32 → Int16LE, posts one 3,200-byte frame per 100 ms
+const node = new AudioWorkletNode(ctx, "pcm16");
+ctx.createMediaStreamSource(stream).connect(node);
+let ready = false;
+node.port.onmessage = (e) => ready && ws.readyState === WebSocket.OPEN && ws.send(e.data);
+
+let committed = "", locked = "", live = "";
+ws.addEventListener("message", (e) => {
+ const ev = JSON.parse(e.data);
+ if (ev.type === "transcript.created") ready = true;
+ else if (ev.type === "transcript.partial") {
+ if (ev.speech_final) { committed += ev.text + " "; locked = ""; live = ""; } // complete stitched utterance
+ else if (ev.is_final) { locked += ev.text + " "; live = ""; } // chunk final: text will not change
+ else live = ev.text; // interim: may change
+ render(committed + locked + live);
+ } else if (ev.type === "transcript.done") ws.close(); // after audio.done
+ else if (ev.type === "error") showError(ev.message); // most errors close the socket
+});
+// stop: ws.send(JSON.stringify({ type: "audio.done" }))
+// push-to-talk release: ws.send(JSON.stringify({ type: "Finalize" })) then keep streaming (docs show both `finalize` and `Finalize`; the examples use `Finalize`)
+```
+
+4. **Options** (query params for streaming, form fields for batch)
+
+| Want | Set |
+| --- | --- |
+| Text while speaking | `interim_results=true` |
+| “one hundred dollars” → `$100` | streaming: `language=en`; batch: `format=true` + `language=en` |
+| Product names, jargon | `keyterm=` repeated |
+| Not cut off mid-sentence while dictating numbers | `smart_turn=0.7&smart_turn_timeout=3000` |
+| Faster or slower end of utterance | `endpointing=` ms, default 400 |
+| Who said what (meetings) | `diarize=true` → `words[].speaker` |
+| Agent and customer on separate channels | `multichannel=true&channels=2` (PCM only, not Opus) |
+| Keep “um”, “uh” | `filler_words=true` (removed by default) |
+| Low bandwidth or mobile | `encoding=opus`, exactly one raw Opus packet per frame, omit `sample_rate` |
+| Raw audio to batch | `audio_format=pcm|mulaw|alaw` + `sample_rate` |
+| Quiet or telephony audio | lower `vad_threshold` (streaming default 0.08, batch 0.5) |
+
+5. **Python twin (only if the server is Python)**
+
+```python
+import os, requests
+r = requests.post(
+ "https://api.x.ai/v1/stt",
+ headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
+ data=[("format", "true"), ("language", "en")],
+ files={"file": ("dictation.webm", blob, "audio/webm")}, # requests sends data fields before files
+)
+r.raise_for_status(); text = r.json()["text"]
+# streaming: websockets.connect(url, additional_headers={"Authorization": f"Bearer {key}"}); await ws.send(pcm_bytes)
+```
+
+6. **Smoke**
+ - Batch: `curl -X POST https://api.x.ai/v1/stt -H "Authorization: Bearer $XAI_API_KEY" -F language=en -F [email protected]` → 200 with `text`. Same call with `-F format=true` and no `language` → 400.
+ - Streaming: dictate two sentences with a pause between them. Expect interim text, then a final; no duplicated or vanished words at the utterance boundary (if words vanish, the stitched `speech_final` text did not include the chunk finals: append instead of replacing `locked`). `audio.done` → `transcript.done`, socket closes.
+ - Search the client bundle for `XAI_API_KEY`; it must not be there.
+ - Debug from logs with `/debug-voice`; swap its hook points to `transcript.*` events.
+
+## Out of scope
+
+- Speech that talks back (`/add-voice`), speaking text (`/add-read-aloud`)
+- Inventing an STT token flow, endpoints, or event names not in the docs
diff --git a/grok-voice/skills/add-read-aloud/SKILL.md b/grok-voice/skills/add-read-aloud/SKILL.md
new file mode 100644
index 00000000..1fdb96d9
--- /dev/null
+++ b/grok-voice/skills/add-read-aloud/SKILL.md
@@ -0,0 +1,183 @@
+---
+name: add-read-aloud
+description: >-
+ Use when the user runs /add-read-aloud or wants the app to speak text with
+ Grok text-to-speech: read-aloud button on assistant replies, auto-speak, TTS,
+ voice output, narration, IVR prompts, speech tags, voice_id. For a two-way
+ voice agent use /add-voice. For speech-to-text use /add-dictation.
+---
+
+# Add Read Aloud
+
+Add Grok Text to Speech to an existing app: a speaker button on assistant replies, auto-speak, or narration of any text. Run on `/add-read-aloud`, typed **Read aloud**, or clear “speak this” / “TTS” intent. Cursor has no speaker; wire the **app**, not the IDE.
+
+## Docs
+
+- https://docs.x.ai/developers/model-capabilities/audio/text-to-speech
+- API reference: https://docs.x.ai/developers/rest-api-reference/inference/voice
+- Custom voices: https://docs.x.ai/developers/model-capabilities/audio/custom-voices
+- Pricing (cite docs only): https://docs.x.ai/developers/pricing
+
+## Pick the path
+
+| Need | Path |
+| --- | --- |
+| Tap speaker, hear the finished reply. Narrate a page. Generate a file. | **Batch** `POST https://api.x.ai/v1/tts` (default) |
+| Audio starts while the LLM is still streaming; barge-in; texts over 15,000 chars | **Streaming** `wss://api.x.ai/v1/tts` through a backend relay |
+
+Batch is the default for a read-aloud button: one request, one MP3, cacheable, the key never leaves the server. Go streaming only when the UX needs audio before the text is complete. `POST /v1/tts` has no documented streaming flag; do not invent one.
+
+## Auth
+
+- Bearer `XAI_API_KEY`, server side only. The TTS docs document no ephemeral-token flow, and browsers cannot set WebSocket headers, so browser streaming goes through your backend relay.
+- Never put the key in a client bundle. Do not paste keys in chat.
+
+## Steps
+
+1. **Map the app**
+ - Where assistant messages render, where per-message actions live (copy, regenerate), how the reply stream ends, server framework, package manager.
+ - The **speaker icon belongs to read aloud**. Waveform is voice mode (`/add-voice`), microphone is dictation (`/add-dictation`). Put a ghost speaker button in the message action row; loading shows a spinner, playing shows a stop square. One utterance at a time: starting a new one stops the current one.
+ - Align the action row to the reply’s **text edge**, not the button’s box: an icon button centres its glyph, so if the assistant bubble has no padding pull the row left by that inset (e.g. `-ml-[5px]` for a 12–14 px icon in a 24 px button). Measure in the browser; `getBoundingClientRect` on the `<p>` and the `<svg>` should share a left edge.
+ - Render the button only once the reply has **finished streaming**; on a live message it would speak a partial reply.
+ - With many messages on screen, keep player state (active message id, `loading | playing`, last error) in one shared store (`useSyncExternalStore`, a signal, whatever the app uses) so every button reflects it and errors can surface in the app's existing status area. A per-button `let current` is not enough.
+ - Auto-speak: opt-in toggle, off by default, and only after a user gesture on the page (autoplay policy). Never auto-speak on load.
+ - If `/add-voice` is installed, its `AudioContext` and PCM player can play streaming TTS; do not add a second audio graph.
+
+2. **Prepare the text**
+ - Speak prose, not markup. Strip markdown: headings → text, `**bold**` → text, links → link text, inline code → the code, fenced blocks → `[pause] Code block omitted.`, tables → one sentence per row or omit. Keep punctuation; it drives pacing.
+ - Neutralise speech tags that arrive inside the reply (`[laugh]`, `<whisper>`…) so the model’s text cannot steer delivery. Strip only the **documented tag names** (list in step 5), not every bracket: `[1]` citations and `[note]` must survive.
+ - Batch limit is **15,000 characters per request**. Split longer text on paragraph, then sentence, then word boundaries and play the parts in order; fetch part N+1 while N plays or there is a silent gap at every boundary. Or use streaming.
+ - Cache by `hash(text + voice_id + language + speed)`; the same reply is often replayed.
+
+3. **Batch path (default)**
+ - Server: your route takes `{ text, voice_id?, language? }`, validates the shape of each (`voice_id` `^[a-z0-9-]{1,64}$`, `language` BCP-47 or `auto`), forwards JSON, streams the body back with the upstream `Content-Type` and `Cache-Control: no-store`. Map upstream 404 to “unknown voice” so the client gets a readable error. Default output is MP3 at 24 kHz / 128 kbps, playable everywhere in the browser.
+ - `language`: default to `"auto"` for a chat app, where replies follow the user’s language; pin `"en"` etc. only for fixed-language products.
+
+```ts
+// server (any runtime with fetch)
+export async function speak(text: string, voice_id = "eve", language = "auto") {
+ if (!text.trim() || text.length > 15_000) throw new Error("TTS text must be 1–15,000 chars");
+ const res = await fetch("https://api.x.ai/v1/tts", {
+ method: "POST",
+ headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}`, "Content-Type": "application/json" },
+ body: JSON.stringify({
+ text,
+ voice_id,
+ language, // required: `auto` or BCP-47 (`en`, `pt-BR`); omitting it → 422
+ // output_format: { codec: "mp3", sample_rate: 24000, bit_rate: 128000 }, // default
+ // speed: 1.0, // 0.7–1.5
+ // text_normalization: true, // "$5" → "five dollars"
+ // replace: { nginx: "/ˈɛndʒɪn ˈɛks/" },
+ }),
+ });
+ if (!res.ok) throw new Error(`TTS ${res.status}`); // 400 bad text/format, 401 key, 404 unknown voice_id, 422 missing required field (e.g. language), 429/500/503 back off and retry
+ return new Response(res.body, { headers: { "Content-Type": res.headers.get("content-type") ?? "audio/mpeg" } });
+}
+```
+
+```ts
+// client
+let current: HTMLAudioElement | null = null;
+async function readAloud(text: string, voiceId = "eve") {
+ current?.pause(); current = null; // one utterance at a time
+ const res = await fetch("/api/tts", { method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ text, voice_id: voiceId }) });
+ if (!res.ok) throw new Error("TTS request failed");
+ const url = URL.createObjectURL(await res.blob());
+ const audio = new Audio(url);
+ audio.addEventListener("ended", () => URL.revokeObjectURL(url)); // avoid blob leaks
+ current = audio;
+ await audio.play(); // call from the click handler’s promise chain
+}
+function stop() { current?.pause(); current = null; }
+```
+
+ - Safari: `audio.duration` is `Infinity` on blob URLs. If you need a progress bar, decode with `AudioContext.decodeAudioData(buf.slice(0))` or request `with_timestamps: true` and read `duration` from the JSON envelope (audio is then base64 in `audio`).
+ - Safari suspends an `AudioContext` created outside a gesture for good. Create it synchronously in the click handler, before any `await`.
+
+4. **Streaming path**
+ - Relay: server holds the key, upgrades the browser socket, builds the query string, forwards client JSON up and server JSON down. Request `codec=pcm` for the browser: raw PCM16 chunks can be scheduled as they arrive, while MP3 chunks cannot be decoded piecemeal without `MediaSource`.
+
+```ts
+import { WebSocketServer, WebSocket } from "ws";
+
+new WebSocketServer({ port: 8789 }).on("connection", (client) => {
+ const q = new URLSearchParams({ language: "en", voice: "eve", codec: "pcm", sample_rate: "24000" /* optimize_streaming_latency: "1" */ });
+ const up = new WebSocket(`wss://api.x.ai/v1/tts?${q}`, { headers: { Authorization: `Bearer ${process.env.XAI_API_KEY}` } });
+ up.on("message", (d) => client.send(d.toString())); // audio.delta, audio.done, audio.clear, session.updated, error
+ client.on("message", (d) => up.readyState === WebSocket.OPEN && up.send(d.toString())); // text.delta, text.done, text.clear, session.update
+ const end = () => { client.close(); up.close(); };
+ up.on("close", end); up.on("error", end); client.on("close", end);
+});
+```
+
+ - Client: one socket per chat session (it stays open across utterances; 50 concurrent sessions per team, permit TTL 600 s, so reconnect on close). Forward LLM tokens as `text.delta` (each ≤ 15,000 chars), send `text.done` when the reply finishes, `text.clear` on stop or when a new reply starts; drop queued audio on `audio.clear`.
+
+```ts
+const ws = new WebSocket(relayUrl);
+const ctx = new AudioContext({ sampleRate: 24000 }); // create in the click handler; resume if suspended
+let playhead = 0, sources: AudioBufferSourceNode[] = [];
+
+ws.addEventListener("message", (e) => {
+ const ev = JSON.parse(e.data);
+ if (ev.type === "audio.delta") {
+ const bytes = Uint8Array.from(atob(ev.delta), (c) => c.charCodeAt(0));
+ const pcm = new Int16Array(bytes.buffer, 0, bytes.byteLength >> 1);
+ const buf = ctx.createBuffer(1, pcm.length, 24000);
+ const ch = buf.getChannelData(0);
+ for (let i = 0; i < pcm.length; i++) ch[i] = pcm[i] / 32768;
+ const src = ctx.createBufferSource(); src.buffer = buf; src.connect(ctx.destination);
+ playhead = Math.max(playhead, ctx.currentTime + 0.15); // ~150 ms lead so chunks butt together
+ src.start(playhead); playhead += buf.duration; sources.push(src);
+ } else if (ev.type === "audio.done") { /* utterance finished; socket stays open */ }
+ else if (ev.type === "audio.clear") { sources.forEach((s) => s.stop()); sources = []; playhead = 0; }
+ else if (ev.type === "error") showError(ev.message);
+});
+// on each LLM token: ws.send(JSON.stringify({ type: "text.delta", delta: token }))
+// on reply finished: ws.send(JSON.stringify({ type: "text.done" }))
+// on stop / barge-in: ws.send(JSON.stringify({ type: "text.clear" })) → wait for audio.clear before the next text.delta
+```
+
+ - Words split across `text.delta` boundaries are fine; matching and synthesis run across deltas.
+
+5. **Options** (JSON fields for batch, query params for streaming)
+
+| Want | Set |
+| --- | --- |
+| Different voice | `voice_id` (batch) / `voice` (streaming). Built-ins from `GET /v1/tts/voices`: `eve` (default), `ara`, `rex`, `leo`, `luna`, `atlas`, `aurora`, `orion`, … 28 total, all multilingual, case-insensitive. Custom voice: 8-char id from the console or `GET /v1/custom-voices` |
+| Non-English or mixed | `language`: `en`, `ar-EG`, `ar-SA`, `ar-AE`, `bn`, `zh`, `fr`, `de`, `hi`, `id`, `it`, `ja`, `ko`, `pt-BR`, `pt-PT`, `ru`, `es-MX`, `es-ES`, `tr`, `vi`, or `auto` |
+| Faster or slower | `speed` 0.7–1.5 |
+| “$5”, “Dr.”, “3/4” spoken as words | `text_normalization: true` |
+| Brand names, acronyms, jargon | `replace: { "Acme Mobile": "Acme Mobull", "nginx": "/ˈɛndʒɪn ˈɛks/" }`; ≤200 entries, keys ≤100 chars (letters, digits, apostrophes, spaces), values ≤128; whole-word, case-insensitive, longest match wins. Streaming: `session.update { replace }` before the first `text.delta` |
+| Expressive delivery | Inline `[pause]`, `[long-pause]`, `[laugh]`, `[chuckle]`, `[giggle]`, `[cry]`, `[sigh]`, `[breath]`, `[inhale]`, `[exhale]`, `[tsk]`, `[tongue-click]`, `[lip-smack]`, `[hum-tune]`. Wrapping `<whisper>`, `<soft>`, `<loud>`, `<emphasis>`, `<build-intensity>`, `<decrease-intensity>`, `<slow>`, `<fast>`, `<higher-pitch>`, `<lower-pitch>`, `<singing>`, `<sing-song>` around whole phrases |
+| Captions, karaoke, lip-sync | `with_timestamps: true` → JSON `{ audio (base64), content_type, duration, audio_timestamps: { graph_chars[], graph_times[][start,end] } }`; step through `graph_chars` in order, never slice input by index |
+| First audio sooner (streaming) | `optimize_streaming_latency=1` (docs also list `2`; API reference lists `0`/`1`) |
+| Telephony / IVR | `output_format: { codec: "mulaw" \| "alaw", sample_rate: 8000 }`; not playable in browsers |
+| Editing, post-production | `codec: "wav"`, `sample_rate: 44100` or `48000` |
+| Smaller files | `codec: "mp3"`, `bit_rate: 64000` |
+
+6. **Python twin (only if the server is Python)**
+
+```python
+import os, requests
+r = requests.post(
+ "https://api.x.ai/v1/tts",
+ headers={"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"},
+ json={"text": text, "voice_id": "eve", "language": "en"},
+)
+r.raise_for_status(); audio_bytes = r.content # audio/mpeg
+# streaming: websockets.connect(url, additional_headers={"Authorization": f"Bearer {key}"}); send {"type":"text.delta",...}, {"type":"text.done"}; read audio.delta / audio.done
+```
+
+7. **Smoke**
+ - Batch: `curl -X POST https://api.x.ai/v1/tts -H "Authorization: Bearer $XAI_API_KEY" -H "Content-Type: application/json" -d '{"text":"Hello from read aloud.","voice_id":"eve","language":"en"}' --output /tmp/hello.mp3` → 200 `audio/mpeg` (MP3, 24 kHz, 128 kbps, mono), plays. Omit `language` → 422 (observed; the docs’ table only lists 400). `voice_id: "nope"` → 404.
+ - In the app: no speaker while a reply streams; it appears when the reply ends. Click → spinner → stop square → back to speaker when audio ends; click again mid-play → stops at once; click a second reply while the first plays → first stops, second plays. Safari: first play works from the click, `URL.revokeObjectURL` fires on `ended`.
+ - Text prep: a reply with a fenced block, a table, a `[1]` citation and a stray `[laugh]` → spoken as “Code block omitted”, one sentence per row, the citation intact, the tag gone. Unit-test this; it is pure.
+ - Streaming: send a two-sentence reply token by token; audio should start before `text.done`. Send `text.clear` mid-utterance → `audio.clear`, playback stops with nothing stale. Second utterance on the same socket → fresh `audio.delta`s, no bleed from the first.
+ - Search the client bundle for `XAI_API_KEY`; it must not be there. Against a running dev server, fetch `/`, collect the `/_next/static/chunks/*.js` (or equivalent) URLs it references, and grep each; do not rely on a production build you have not made.
+ - Debug from logs with `/debug-voice`; swap its hook points to `audio.delta` (byte counts), `audio.done`, `audio.clear`, `error`.
+
+## Out of scope
+
+- A voice agent that listens and answers (`/add-voice`), speech to text (`/add-dictation`)
+- Voice cloning beyond passing an existing custom `voice_id`
+- Inventing a TTS token flow, a `stream` flag on `POST /v1/tts`, or event names not in the docs
diff --git a/grok-voice/skills/add-voice/SKILL.md b/grok-voice/skills/add-voice/SKILL.md
new file mode 100644
index 00000000..aee71871
--- /dev/null
+++ b/grok-voice/skills/add-voice/SKILL.md
@@ -0,0 +1,123 @@
+---
+name: add-voice
+description: >-
+ Use when the user runs /add-voice, types Voice Mode, or asks to add Grok
+ realtime voice to an app, including replacing an STT-LLM-TTS cascade or
+ OpenAI Realtime. Wire speech-to-speech, safe auth, and app mic. Composer: waveform button,
+ mic icon reserved for dictation. For mic-to-text only use /add-dictation; to speak text replies use
+ /add-read-aloud. To add debug logging and fix from logs use /debug-voice.
+---
+
+# Add Voice
+
+Add Grok Speech to Speech to an existing app. Run on `/add-voice`, typed **Voice Mode**, or clear “add Grok voice” intent.
+
+## Goal
+
+Working duplex path: user-app mic in, audio out, `wss://api.x.ai/v1/realtime?model=grok-voice-latest`, safe auth. Cursor has no native mic; wire the **app** (or a sample client), not the IDE.
+
+## Protocol first
+
+Language-agnostic event loop. **TypeScript samples default.** Short Python twins only where the client API differs (e.g. `ws` vs `websockets`).
+
+## Docs
+
+- https://docs.x.ai/developers/model-capabilities/audio/speech-to-speech
+- https://docs.x.ai/developers/model-capabilities/audio/ephemeral-tokens
+- Pricing (cite docs only): https://docs.x.ai/developers/pricing (~$0.08/min STS + $0.004/text item; max session 120 min)
+
+## Steps
+
+1. **Map the app**
+ - Stack: none, OpenAI Realtime, STT→LLM→TTS cascade, TTS/STT only.
+ - Client: web / Node / iOS / Android / server.
+ - If a cascade or OpenAI Realtime exists: replace it with the single duplex loop below (URL, model, voice, event diffs); keep standalone `/v1/stt` or `/v1/tts` only if the product still needs one-shot listen or speak outside the agent.
+
+2. **Auth**
+ - Server: Bearer `XAI_API_KEY`.
+ - Browser/mobile: backend `POST https://api.x.ai/v1/realtime/client_secrets`, client uses ephemeral token (Bearer or browser `sec-websocket-protocol`: `xai-client-secret.<token>`).
+ - Never put a long-lived key in client bundles. Do not paste keys in chat.
+
+3. **Connect + session**
+ - URL: `wss://api.x.ai/v1/realtime?model=grok-voice-latest`
+ - On open: `session.update` with `voice` (default `eve`), `instructions`, `turn_detection: { type: "server_vad" }` (or `null` for push-to-talk), PCM 24 kHz unless the app already standardizes elsewhere.
+ - Set `audio.input.transcription.model: "grok-transcribe"` or no user transcript arrives (`conversation.item.input_audio_transcription.updated` is cumulative, not a delta).
+ - Tools if needed: `web_search`, `x_search`, `file_search`, `mcp`, custom `function`.
+
+4. **Audio I/O (app-side)**
+ - One `AudioContext` per session for capture and playback, created inside the user gesture (autoplay policy). Ask for 24 kHz; if the browser gives another rate, resample before sending.
+ - Mic → AudioWorklet in ~100 ms chunks → `input_audio_buffer.append` (or binary transport). Start WS and mic in parallel; buffer early audio, flush on open.
+ - Play `response.output_audio.delta` immediately; schedule with a ~150 ms lead so chunks butt together. On `input_audio_buffer.speech_started`, stop everything queued (barge-in).
+ - Transcript rows: create the user row on `input_audio_buffer.committed` (`item_id`), fill it on `…transcription.updated`; assistant text from `response.output_audio_transcript.delta` / `.done`, close the turn on `response.done`.
+ - On function tools: `function_call_output`, finish playback, then `response.create`.
+
+5. **Composer UI convention**
+ - One primary button, right side of the composer. Empty composer → **waveform** icon (stroked, e.g. Phosphor `WaveformIcon weight="bold"`; never the `fill` weight, which renders as a blob at 16 px), starts voice mode. Any text present → classic **send** arrow; in voice mode that text goes into the live session (`conversation.item.create` + `response.create`). Text reply streaming → stop square.
+ - While voice is live the same button shows an **animated waveform** (4 bars, ~3 px wide, 2 px gap, ~16 px tall, min scale 0.4 so they stay legible in a 28 px button) and ends the session on click. Phase drives the animation: listening slow, speaking fast, connecting/thinking slower and slightly dimmed (opacity ≥ 0.75). Honor `prefers-reduced-motion`. No X button, no pulsing ring.
+ - Status lives **in the composer, not around it**: the placeholder reads `Connecting…` / `Listening…` / `Thinking…` / `Speaking…`, plus an `sr-only` `role="status"`. No separate status row.
+ - The **microphone icon is reserved for dictation** (`/add-dictation`). Never use it for voice mode.
+
+6. **TS skeleton (default)**
+
+```ts
+const url = "wss://api.x.ai/v1/realtime?model=grok-voice-latest";
+// Node: pass Authorization header. Browser: use xai-client-secret.<token> protocol.
+const ws = new WebSocket(url /* , { headers: { Authorization: `Bearer ${token}` } } */);
+
+ws.addEventListener("open", () => {
+ ws.send(JSON.stringify({
+ type: "session.update",
+ session: {
+ voice: "eve",
+ instructions: "You are a helpful voice agent.",
+ turn_detection: { type: "server_vad" },
+ },
+ }));
+});
+
+ws.addEventListener("message", (ev) => {
+ const event = JSON.parse(String(ev.data));
+ if (event.type === "response.output_audio.delta") {
+ // decode base64 PCM and play
+ }
+});
+```
+
+7. **Python twin (only if the app is Python)**
+
+```python
+import json, os, websockets
+
+url = "wss://api.x.ai/v1/realtime?model=grok-voice-latest"
+headers = {"Authorization": f"Bearer {os.environ['XAI_API_KEY']}"}
+
+async with websockets.connect(url, additional_headers=headers) as ws:
+ await ws.send(json.dumps({
+ "type": "session.update",
+ "session": {
+ "voice": "eve",
+ "instructions": "You are a helpful voice agent.",
+ "turn_detection": {"type": "server_vad"},
+ },
+ }))
+ async for raw in ws:
+ event = json.loads(raw)
+ if event.get("type") == "response.output_audio.delta":
+ pass # decode and play
+```
+
+8. **Instrument (before the first human test)**
+ - Run `/debug-voice`: it proposes a plan, then installs a dev-only log sink (`POST /api/voice/log` → `.voice-logs/<sessionId>.ndjson`, gitignored), a client logger with audio reduced to byte counts, and the session id in the UI, in the app's own language.
+ - The same skill carries the fix loop and the symptom → log signature → fix table.
+
+9. **Smoke**
+ - Text turn via `conversation.item.create` + `response.create`; confirm audio or transcript events.
+ - Confirm no long-lived key in client (grep the built client bundle for the env name and `client_secrets`).
+ - Hand the app to the user with **headphones**. On speakers the mic hears the reply and the model answers itself; that is echo, not a bug in the loop.
+ - Iterate with `/debug-voice`.
+
+## Out of scope
+
+- Speech-to-text only (`/add-dictation`), speaking text (`/add-read-aloud`)
+- Image generation and text-only inference
+- Invented endpoints, events, or CLI flags
diff --git a/grok-voice/skills/debug-voice/SKILL.md b/grok-voice/skills/debug-voice/SKILL.md
new file mode 100644
index 00000000..c7ee7570
--- /dev/null
+++ b/grok-voice/skills/debug-voice/SKILL.md
@@ -0,0 +1,198 @@
+---
+name: debug-voice
+description: >-
+ Use when the user runs /debug-voice, says voice mode has flaws, asks to see or
+ capture what happened in a Grok realtime voice session, or a voice integration
+ has no debug logging yet. Proposes a plan, then installs a dev-only log
+ pipeline (client logger → local NDJSON) in the app's own language and
+ conventions, then runs the fix loop: match the user's report to log
+ signatures, fix one thing, re-test.
+---
+
+# Debug Voice
+
+Make a voice session readable after the fact, then fix from evidence. The agent cannot hear the app; the log is its ears, the user is its judge. No audio, no tokens, never in prod.
+
+Works for any stack. The pipeline is a small contract (below); implement it in whatever the app already uses.
+
+## Workflow
+
+1. **Map** the app (read only).
+2. **Plan**: write the change list, show it, **stop**. Nothing is edited until the user aligns.
+3. **Install** the agreed pieces in the app's language, framework, and conventions.
+4. **Verify** the sink, hand the app to the user.
+5. **Fix loop** from the log.
+
+## 1. Map
+
+Find, and note the paths:
+
+- Voice client: where realtime events are received and sent, mic capture, audio playback, token fetch.
+- Server: framework, how routes are declared, where shared server code lives, how env is read, what "production" means here.
+- Conventions: language(s), module system, formatter, where scripts or tasks live (`package.json`, `Makefile`, `pyproject`, `justfile`), `.gitignore`.
+- Client kind: browser, mobile, desktop, CLI. A non-browser client still POSTs the same JSON; a single-process app can skip HTTP and append to the file directly.
+- Where audio deltas are handled. They must be counted, never logged.
+
+## 2. Plan, then stop
+
+Fill this in with real paths and the app's language, post it, and wait for a yes or a trimmed list. Do not edit files before that.
+
+```markdown
+## Debug voice: plan
+
+Add
+- <path>: client logger (batch, redact, flush) in <language>
+- <path>: dev-only sink `POST /api/voice/log` → `.voice-logs/<sessionId>.ndjson`
+- <path> (optional): summary command `<cmd>`; otherwise read the NDJSON with jq
+
+Modify
+- <voice client file>: hook points start, token, mic, env, ws.*, client/server events,
+ audio.in (2 s windows), audio.out.first, audio.out, play.stop, stop
+- <token route>: append `server.token { ok, status, ms, upstream }` (never the token)
+- <UI file>: session id in the voice status line and in voice error messages
+- .gitignore: `/.voice-logs`
+- <scripts file>: a `voice:logs` task (only if the summary command is wanted)
+
+Logged: event names and non-audio fields, timings, byte counts, mic RMS.
+Never: tokens, API keys, raw audio, strings over 400 chars.
+Off in production unless `VOICE_LOG=1`.
+
+Reply "go", or strike lines you do not want.
+```
+
+## 3. Install: the contract
+
+Match the app. Same language as the surrounding code, same route style, same formatter. Write the pieces from the contract below; do not introduce a second language or toolchain for logging.
+
+**Session id**: 8 lowercase hex chars from a UUID. The sink accepts `^[a-z0-9]{4,64}$`; it becomes a file name.
+
+**Entry** (one JSON object per line):
+
+| Field | Client | Server |
+| --- | --- | --- |
+| `t` | ms since the logger started | absent; the reader aligns by `ts` |
+| `ts` | epoch ms | epoch ms |
+| `kind` | `start`, `server`, `client`, `error`, `audio.in`, … | `server.token`, … |
+| `src` | absent | `"server"` |
+| rest | the hook's fields, redacted | the hook's fields |
+
+**Redaction, applied client side before buffering**: on audio event types (`response.output_audio.delta`, `response.audio.delta`, `input_audio_buffer.append`) replace `delta` / `audio` with `bytes` = decoded base64 length; strings over 400 chars cut to 400 + `…[N chars]`; objects deeper than 4 → `"[depth]"`; arrays over 50 items truncated.
+
+**Client logger**: buffer entries; flush as `POST <sink> {"sessionId","entries":[…]}` every 1 s or at 200 entries; on stop flush with keepalive (or the platform's "survive navigation" equivalent); swallow every transport error, logging must never throw into the voice path. Also mirror entries to the console in dev.
+
+**Sink**: `POST /api/voice/log`, JSON body. `404` unless dev or `VOICE_LOG=1`. `400` if `sessionId` fails the regex or `entries` is not an array. Append at most 500 entries per request, drop any line over 16,000 chars, to `.voice-logs/<sessionId>.ndjson`, creating the directory. Reply `204`.
+
+Pseudocode for any server:
+
+```
+handle POST /api/voice/log:
+ if production and VOICE_LOG != "1": return 404
+ body = parse json or return 400
+ if not regex(body.sessionId) or not list(body.entries): return 400
+ mkdir .voice-logs; append join(json(e) for e in body.entries[:500] if len < 16000) to .voice-logs/{sessionId}.ndjson
+ return 204
+```
+
+Pseudocode for the client logger:
+
+```
+logger(sessionId, sink):
+ buffer = []; started = now()
+ log(kind, data): buffer.push({ ...redact(data), t: now() - started, ts: epoch_ms(), kind }); schedule flush (1 s timer, or immediately at 200 entries)
+ server(event, extra): log("server", { ...redact_event(event), ...extra }) # never per audio delta
+ client(event): log("client", redact_event(event)) # never per audio chunk
+ error(where, err, extra): log("error", { where, name, message, ...extra })
+ flush(final=false): POST sink {"sessionId","entries": buffer}; buffer = []; ignore all errors; keepalive when final
+ close(): flush(final=true)
+```
+
+## Hook points
+
+`kind` and fields; the shape is the same in every language.
+
+| When | `kind` and fields |
+| --- | --- |
+| Session start | `start { url, target_rate }` |
+| Token fetched / failed | `token.ok { ms }` / `error { where: "token", name, message, ms }` |
+| Mic granted / denied | `mic.ok { ms, label, settings }` / `error { where: "mic", … }` |
+| Audio graph ready | `env { ua, mic_rate, mic_state, play_rate, play_state, capture_frames, target_rate }` |
+| Socket | `ws.connecting`, `ws.open { ms }`, `ws.error`, `ws.close { code, reason, wasClean, by }` |
+| Every event sent, except audio chunks | `client { …redacted event }` |
+| Every event received, except audio deltas | `server { …redacted event, phase }` |
+| Phase change (dedupe) | `phase { phase }` |
+| Mic chunks, aggregated per 2 s | `audio.in { chunks, bytes, rms_max, rms_avg, pending, mic_state, phase }` |
+| Pre-open buffer sent on open | `audio.flush { chunks }` |
+| First audio delta of a response | `audio.out.first { response_id, bytes, since_response_created_ms, since_speech_stopped_ms, play_state }` |
+| `response.done` | `audio.out { response_id, status, deltas, bytes, audio_ms, wall_ms, max_gap_ms, queued_ms, underruns, drain_ms_max }` |
+| Barge-in stop | `play.stop { reason, dropped_ms }` |
+| User stop | `stop { by: "client", phase }`, then flush |
+| Token route (server) | `server.token { ok, status, ms, upstream }` |
+
+In the message handler: if the event is an audio delta, count bytes and gaps and play it; otherwise log it as `server` with the current phase, then run the existing handling. Keep `speechStoppedT` from `input_audio_buffer.speech_stopped` and `createdT` from `response.created`; report both distances on the first delta. In the player, count an underrun when the next scheduled time is already in the past mid-response, track the largest drain, reset on `response.created`.
+
+UI: show the id in the voice status line (`Listening · session ab12cd34`) and append `(voice session <id>)` to voice errors, so the user can name the run.
+
+## 4. Verify
+
+```bash
+curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:<port>/api/voice/log \
+ -H 'Content-Type: application/json' \
+ -d '{"sessionId":"smoke001","entries":[{"t":0,"ts":0,"kind":"start"}]}' # 204
+curl -s -o /dev/null -w "%{http_code}\n" -X POST localhost:<port>/api/voice/log \
+ -H 'Content-Type: application/json' -d '{"sessionId":"../x","entries":[]}' # 400
+cat .voice-logs/smoke001.ndjson && rm .voice-logs/smoke001.ndjson
+```
+
+Then hand off: the user tests in the real app and reports what they said, what they heard, when it went wrong, and the session id.
+
+## Reading a log
+
+Any of these; none needs the app's toolchain:
+
+```bash
+f=.voice-logs/<id>.ndjson
+jq -r 'select(.kind|IN("start","token.ok","mic.ok","env","ws.open","ws.close","server.token","stop","error")) | "\(.t // .ts)ms \(.kind) \(.type // "") \(.message // "")"' $f # milestones and errors
+jq -r 'select(.kind=="server") | .type' $f | sort | uniq -c | sort -rn # server event counts
+jq -c 'select(.kind|IN("audio.out.first","audio.out"))' $f # per-turn latency, gaps, underruns
+jq -c 'select(.kind=="audio.in")' $f # mic windows, rms
+```
+
+Or read the file; it is one object per line in time order. Server entries have only `ts`; align them to the client clock with the `start` entry's `ts`. If the user wants a summary command, write one in the app's language that prints: milestones, server event counts, one line per `audio.out` turn with its `audio.out.first` latencies, mic window totals, errors and closes, and the last 25 entries.
+
+## 5. Fix loop
+
+1. **Instrument** (steps 1–4) if the app has no `.voice-logs` pipeline yet.
+2. **User tests** in the real app and describes the run.
+3. **Read the log** around the failing `t`.
+4. **Match** symptom → signature → fix (table below). No matching signature: add logging first, re-test, then fix.
+5. **Fix one thing**, re-test, confirm the signature is gone in a fresh log.
+6. **Write it down**: append a confirmed row under "Confirmed from sessions". If the fix changes how voice should be built, update `/add-voice` too.
+
+## Symptom → log signature → fix (starter rows)
+
+| Symptom (user) | Signature (log) | Fix |
+| --- | --- | --- |
+| Silence, but transcript appears | `env.play_state` or `audio.out.first.play_state` = `suspended` | Create and resume the playback audio context inside the user gesture; one context per session, not per turn |
+| Assistant interrupts itself | `speech_started` with `phase=speaking`; `audio.in.rms_max` rises only during playback | Echo. Confirm with headphones (if it stops, it is echo). Keep echo cancellation on, lower speaker volume, or gate mic sends while `speaking` |
+| Choppy, stuttering | `audio.out.underruns` > 0, `drain_ms_max` high, `max_gap_ms` far above chunk length | Schedule a small lead (150–250 ms) before the first chunk plays; do not rebuild the audio context per turn |
+| Crackle, wrong pitch or speed | `session.update` rate ≠ buffer rate; odd `bytes` | One rate everywhere (`audio.input/output.format.rate`, player buffer); even-byte alignment |
+| Never connects, or closes at once | `ws.close` before `session.updated`; `server.token.ok=false` | Mint a token per click (300 s), protocol `xai-client-secret.<token>`, `model` in URL; read `server.token.status` and `upstream` |
+| Mic does nothing | `audio.in.rms_max` ≈ 0 in every window; `mic.ok.label` unexpected | Wrong device or OS permission; check `mic.ok.settings`, `label`, `mic_state` |
+| No user transcript | no `conversation.item.input_audio_transcription.updated` | Set `audio.input.transcription.model: "grok-transcribe"` in `session.update` |
+| Slow first word | `audio.out.first.since_speech_stopped_ms` high | Try `reasoning.effort: "none"`; shorter `instructions`; check `token.ok.ms` and `ws.open.ms` for connect cost |
+| User text appears after the reply | `...transcription.updated` `t` > `response.created` `t` | Create the user row on `input_audio_buffer.committed` (`item_id`), fill it on `updated` |
+| First words cut off | `ws.open.ms` large and `audio.flush.chunks` at the buffer cap | Start mic before the socket, buffer early audio, raise the pre-open cap |
+
+## Confirmed from sessions
+
+Append after a fix is verified in a fresh log. Format: `YYYY-MM-DD · symptom · signature · fix · file(s)`.
+
+## Rules
+
+- Plan first; no edits before the user aligns on the change list.
+- Never tokens, keys, or raw audio. Audio becomes byte counts.
+- Dev only. The sink returns 404 in production unless `VOICE_LOG=1`. `.voice-logs/` is gitignored.
+- Logging never throws into the voice path.
+- Aggregate audio; never log per chunk.
+- One change per re-test so the log tells you which fix worked.
+- Do not invent xAI event names; confirm in https://docs.x.ai/developers/model-capabilities/audio/speech-to-speech before adding a signature.