The open-source AI voice studio. Clone, dictate, create.
fix TorchScript error in frozen builds and update docs for TADA
James Pine273483ffcfab
diff --git a/README.md b/README.mdindex 614e4f21..1acc9a4f 100644--- a/README.md+++ b/README.md@@ -59,10 +59,10 @@## What is Voicebox?-Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.+Voicebox is a **local-first voice cloning studio** — a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.- **Complete privacy** — models and voice data stay on your machine-- **4 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo+- **5 TTS engines** — Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA- **23 languages** — from English to Arabic, Japanese, Hindi, Swahili, and more- **Post-processing effects** — pitch shift, reverb, delay, chorus, compression, and filters- **Expressive speech** — paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo@@ -93,7 +93,7 @@ Voicebox is a **local-first voice cloning studio** — a free and open-source al### Multi-Engine Voice Cloning-Four TTS engines with different strengths, switchable per-generation:+Five TTS engines with different strengths, switchable per-generation:| Engine | Languages | Strengths || --------------------------- | --------- | ---------------------------------------------------------------------------------------------------------------------------------------- |@@ -101,6 +101,7 @@ Four TTS engines with different strengths, switchable per-generation:| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU || **Chatterbox Multilingual** | 23 | Broadest language coverage — Arabic, Danish, Finnish, Greek, Hebrew, Hindi, Malay, Norwegian, Polish, Swahili, Swedish, Turkish and more || **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |+| **TADA** (1B / 3B) | 10 | HumeAI speech-language model — 700s+ coherent audio, text-acoustic dual alignment |### Emotions & Paralinguistic Tags@@ -230,7 +231,7 @@ Full API documentation available at `http://localhost:17493/docs`.| Frontend | React, TypeScript, Tailwind CSS || State | Zustand, React Query || Backend | FastAPI (Python) |-| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |+| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA || Effects | Pedalboard (Spotify) || Transcription | Whisper / Whisper Turbo (PyTorch or MLX) || Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |@@ -245,7 +246,7 @@ Full API documentation available at `http://localhost:17493/docs`.| ----------------------- | ---------------------------------------------- || **Real-time Streaming** | Stream audio as it generates, word by word || **Voice Design** | Create new voices from text descriptions |-| **More Models** | XTTS, Bark, and other open-source voice models |+| **More Models** | XTTS, Bark, and other open-source voice models || **Plugin Architecture** | Extend with custom models and effects || **Mobile Companion** | Control Voicebox from your phone |diff --git a/backend/utils/dac_shim.py b/backend/utils/dac_shim.pyindex ea1294fc..89968c18 100644--- a/backend/utils/dac_shim.py+++ b/backend/utils/dac_shim.py@@ -24,7 +24,9 @@ import torch.nn as nn# ── Snake activation (from dac/nn/layers.py) ────────────────────────+# NOTE: The original DAC code uses @torch.jit.script here for a 1.4x+# speedup. We omit it because TorchScript calls inspect.getsource()+# which fails inside a PyInstaller frozen binary (no .py source files).def snake(x: torch.Tensor, alpha: torch.Tensor) -> torch.Tensor:shape = x.shapex = x.reshape(shape[0], shape[1], -1)diff --git a/docs/content/docs/developer/tts-engines.mdx b/docs/content/docs/developer/tts-engines.mdxindex f95e7a71..90135a37 100644--- a/docs/content/docs/developer/tts-engines.mdx+++ b/docs/content/docs/developer/tts-engines.mdx@@ -490,7 +490,7 @@ Based on the current model landscape, these are candidates for future integratio| **Fish Speech** | 50+ | Medium | Word-level control via inline text | Ready || **Kokoro-82M** | English | 82M | CPU realtime, Apache 2.0 | Ready || **XTTS-v2** | 17+ | Medium | Zero-shot cloning | Ready |-| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Needs vetting |+| **HumeAI TADA** | EN (1B), Multi (3B) | Medium | 700s+ coherent audio, synced transcripts | Shipped || **MOSS-TTS** | Multilingual | Medium | Text-to-voice design, multi-speaker dialogue | Needs vetting || **Pocket TTS** | English | ~100M | CPU-first, >1× realtime | Needs vetting |diff --git a/docs/content/docs/index.mdx b/docs/content/docs/index.mdxindex cbe120c3..9a7da078 100644--- a/docs/content/docs/index.mdx+++ b/docs/content/docs/index.mdx@@ -3,12 +3,12 @@ title: "Voicebox Documentation"description: "Voicebox is a local-first voice cloning studio -- a free and open-source alternative to ElevenLabs."----Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.+Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.- **Complete privacy** -- models and voice data stay on your machine-- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo+- **5 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbodiff --git a/docs/content/docs/overview/introduction.mdx b/docs/content/docs/overview/introduction.mdxindex 134f1d71..d740a61e 100644--- a/docs/content/docs/overview/introduction.mdx+++ b/docs/content/docs/overview/introduction.mdx@@ -5,10 +5,10 @@ description: "Voicebox is a local-first voice cloning studio -- a free and open-## What is Voicebox?-Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 4 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.+Voicebox is a **local-first voice cloning studio** -- a free and open-source alternative to ElevenLabs. Clone voices from a few seconds of audio, generate speech in 23 languages across 5 TTS engines, apply post-processing effects, and compose multi-voice projects with a timeline editor.- **Complete privacy** -- models and voice data stay on your machine-- **4 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, and Chatterbox Turbo+- **5 TTS engines** -- Qwen3-TTS, LuxTTS, Chatterbox Multilingual, Chatterbox Turbo, and HumeAI TADA- **23 languages** -- from English to Arabic, Japanese, Hindi, Swahili, and more- **Post-processing effects** -- pitch shift, reverb, delay, chorus, compression, and filters- **Expressive speech** -- paralinguistic tags like `[laugh]`, `[sigh]`, `[gasp]` via Chatterbox Turbo@@ -20,7 +20,7 @@ Voicebox is a **local-first voice cloning studio** -- a free and open-source alt## TTS Engines-Four engines with different strengths, switchable per-generation:+Five engines with different strengths, switchable per-generation:| Engine | Languages | Strengths ||--------|-----------|-----------|@@ -28,6 +28,7 @@ Four engines with different strengths, switchable per-generation:| **LuxTTS** | English | Lightweight (~1GB VRAM), 48kHz output, 150x realtime on CPU || **Chatterbox Multilingual** | 23 | Broadest language coverage || **Chatterbox Turbo** | English | Fast 350M model with paralinguistic emotion/sound tags |+| **TADA** (1B / 3B) | 10 | HumeAI speech-language model -- 700s+ coherent audio |## GPU Support@@ -56,7 +57,7 @@ Four engines with different strengths, switchable per-generation:| Frontend | React, TypeScript, Tailwind CSS || State | Zustand, React Query || Backend | FastAPI (Python) |-| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo |+| TTS Engines | Qwen3-TTS, LuxTTS, Chatterbox, Chatterbox Turbo, TADA || Effects | Pedalboard (Spotify) || Transcription | Whisper / Whisper Turbo (PyTorch or MLX) || Inference | MLX (Apple Silicon) / PyTorch (CUDA/ROCm/XPU/CPU) |diff --git a/docs/notes/PROJECT_STATUS.md b/docs/notes/PROJECT_STATUS.mdindex 628dfa0a..71c06453 100644--- a/docs/notes/PROJECT_STATUS.md+++ b/docs/notes/PROJECT_STATUS.md@@ -36,6 +36,10 @@│ │ │ │ Qwen3-TTS│ │LuxTTS │ │Chatterbox │ │ │ ││ │ │ │(Py/MLX) │ │ │ │(MTL+Turbo)│ │ │ ││ │ │ └──────────┘ └───────┘ └───────────┘ │ │ │+│ │ │ ┌──────────┐ │ │ │+│ │ │ │ TADA │ │ │ │+│ │ │ │(1B / 3B) │ │ │ │+│ │ │ └──────────┘ │ │ ││ │ └─────────────────────────────────────────┘ │ ││ │ ┌───────────┐ ┌─────────┐ │ ││ │ │ STTBackend│ │ Profiles│ │ │@@ -59,6 +63,7 @@| LuxTTS | `backend/backends/luxtts_backend.py` | LuxTTS — fast, CPU-friendly || Chatterbox MTL | `backend/backends/chatterbox_backend.py` | Chatterbox Multilingual — 23 languages || Chatterbox Turbo | `backend/backends/chatterbox_turbo_backend.py` | Chatterbox Turbo — English, paralinguistic tags |+| TADA | `backend/backends/hume_backend.py` | HumeAI TADA — 1B English + 3B Multilingual || Platform detect | `backend/platform_detect.py` | Apple Silicon → MLX, else → PyTorch || API types | `backend/models.py` | Pydantic request/response models || HF progress | `backend/utils/hf_progress.py` | HFProgressTracker (tqdm patching for download progress) |@@ -78,7 +83,7 @@```POST /generate1. Look up voice profile from DB- 2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo)+ 2. Resolve engine from request (qwen | luxtts | chatterbox | chatterbox_turbo | tada)3. Get backend: get_tts_backend_for_engine(engine) # thread-safe singleton per engine4. Check model cache → if missing, trigger background download, return HTTP 2025. Load model (lazy): tts_backend.load_model(model_size)@@ -104,7 +109,8 @@ POST /generate- LuxTTS integration — fast, CPU-friendly English TTS (PR #254)- Chatterbox Multilingual TTS — 23 languages including Hebrew (PR #257)- Instruct parameter UI exists but is non-functional across all backends (see #224, Known Limitations)-- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo)+- HumeAI TADA integration — 1B English + 3B Multilingual speech-language model (PR #296)+- Single flat model dropdown (Qwen 1.7B, Qwen 0.6B, LuxTTS, Chatterbox, Chatterbox Turbo, TADA 1B, TADA 3B)- Centralized model config registry (`ModelConfig` dataclass) — no per-engine dispatch maps in `main.py`- Shared `EngineModelSelector` component — engine/model dropdown defined once, used in both generation forms@@ -136,6 +142,8 @@ POST /generate| LuxTTS | `luxtts` | English | ~300 MB | CPU-friendly, 48 kHz, fast | None || Chatterbox | `chatterbox-tts` | 23 (incl. Hebrew, Arabic, Hindi, etc.) | ~3.2 GB | Zero-shot cloning, multilingual | Partial — `exaggeration` float (0-1) for expressiveness || Chatterbox Turbo | `chatterbox-turbo` | English | ~1.5 GB | Paralinguistic tags ([laugh], [cough]), 350M params, low latency | Partial — inline tags only, no separate instruct param |+| TADA 1B | `tada-1b` | English | ~4 GB | HumeAI speech-language model, 700s+ coherent audio | None |+| TADA 3B Multilingual | `tada-3b-ml` | 10 (en, ar, zh, de, es, fr, it, ja, pl, pt) | ~8 GB | Multilingual, text-acoustic dual alignment | None |### Multi-Engine Architecture (Shipped)@@ -143,7 +151,7 @@ The singleton TTS backend blocker described in the previous version of this doc- **Thread-safe backend registry** (`_tts_backends` dict + `_tts_backends_lock`) with double-checked locking- **Per-engine backend instances** — each engine gets its own singleton, loaded lazily-- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo'`+- **Engine field on GenerationRequest** — frontend sends `engine: 'qwen' | 'luxtts' | 'chatterbox' | 'chatterbox_turbo' | 'tada'`- **Per-engine language filtering** — `ENGINE_LANGUAGES` map in frontend, backend regex accepts all languages- **Per-engine voice prompts** — `create_voice_prompt_for_profile()` dispatches to the correct backend- **Trim post-processing** — `trim_tts_output()` for Chatterbox engines (cuts trailing silence/hallucination)@@ -337,7 +345,7 @@ Notable requests:| **CosyVoice2-0.5B** | 3-10s zero-shot | Very fast | 24 kHz | Multilingual | Low | **Yes** — `inference_instruct2()`, works with cloning | Ready | Best instruct candidate || **Fish Speech** | 10-30s few-shot | Real-time | 24-44 kHz | 50+ | Medium | **Yes** — inline text descriptions, word-level control | Ready | Multi-engine arch in place || **MOSS-TTS Family** | Zero-shot | — | — | Multilingual | Medium | **Yes** — text prompts for style + timbre design | Needs vetting | Apache 2.0, multi-speaker dialogue |-| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | — | EN (1B), Multilingual (3B) | Medium | Partial — automatic prosody from text context | Needs vetting | MIT, 700s+ coherent, synced transcript output |+| **HumeAI TADA 1B/3B** | Zero-shot | 5× faster than LLM-TTS | 24 kHz | EN (1B), Multilingual (3B) | Medium | Partial — automatic prosody from text context | **Shipped** | PR #296, MIT, 700s+ coherent || **VoxCPM 1.5** | Zero-shot (seconds) | ~0.15 RTF streaming | — | Bilingual (EN/ZH) | Medium | Partial — automatic context-aware prosody | Needs vetting | Apache 2.0, tokenizer-free continuous diffusion || **Kokoro-82M** | 3s instant | CPU realtime | 24 kHz | English | Tiny (82M) | Partial — automatic style inference | Ready | Apache 2.0, multi-engine arch in place || **XTTS-v2** | 6s zero-shot | Mid-GPU | 24 kHz | 17+ | Medium | Partial — style transfer from ref audio only | Ready | Multi-engine arch in place |@@ -475,7 +483,7 @@ The generation form now uses a flat model dropdown with engine-based routing. Pe| `/history/{id}/export` | GET | Export generation ZIP || `/history/{id}/export-audio` | GET | Export audio only || `/transcribe` | POST | Transcribe audio (Whisper) |-| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, Whisper) |+| `/models/status` | GET | All model statuses (Qwen, LuxTTS, Chatterbox, Chatterbox Turbo, TADA, Whisper) || `/models/download` | POST | Trigger model download || `/models/download/cancel` | POST | Cancel/dismiss download || `/models/{name}` | DELETE | Delete downloaded model |