The open-source AI voice studio. Clone, dictate, create.
better docs
Jamie Pine9396c6c86d17
diff --git a/CHANGELOG.md b/CHANGELOG.mdnew file mode 100644index 00000000..8b3845a0--- /dev/null+++ b/CHANGELOG.md@@ -0,0 +1,68 @@+# Changelog++All notable changes to Voicebox will be documented in this file.++The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).++## [0.1.0] - 2026-01-25++### Added++#### Core Features+- **Voice Cloning** - Clone voices from audio samples using Qwen3-TTS (1.7B and 0.6B models)+- **Voice Profile Management** - Create, edit, and organize voice profiles with multiple samples+- **Speech Generation** - Generate high-quality speech from text using cloned voices+- **Generation History** - Track all generations with search and filtering capabilities+- **Audio Transcription** - Automatic transcription powered by Whisper+- **In-App Recording** - Record audio samples directly in the app with waveform visualization++#### Desktop App+- **Tauri Desktop App** - Native desktop application for macOS, Windows, and Linux+- **Local Server Mode** - Embedded Python server runs automatically+- **Remote Server Mode** - Connect to a remote Voicebox server on your network+- **Auto-Updates** - Automatic update notifications and installation++#### API+- **REST API** - Full REST API for voice synthesis and profile management+- **OpenAPI Documentation** - Interactive API docs at `/docs` endpoint+- **Type-Safe Client** - Auto-generated TypeScript client from OpenAPI schema++#### Technical+- **Voice Prompt Caching** - Fast regeneration with cached voice prompts+- **Multi-Sample Support** - Combine multiple audio samples for better voice quality+- **GPU/CPU/MPS Support** - Automatic device detection and optimization+- **Model Management** - Lazy loading and VRAM management+- **SQLite Database** - Local data persistence++### Technical Details++- Built with Tauri v2 (Rust + React)+- FastAPI backend with async Python+- TypeScript frontend with React Query and Zustand+- Qwen3-TTS for voice cloning+- Whisper for transcription++### Platform Support++- macOS (Apple Silicon and Intel)+- Windows+- Linux (AppImage)++---++## [Unreleased]++### Planned+- Real-time streaming synthesis+- Conversation mode with multiple speakers+- Voice effects (pitch shift, reverb, M3GAN-style)+- Timeline-based audio editor+- Additional voice models (XTTS, Bark)+- Voice design from text descriptions+- Project system for saving sessions+- Plugin architecture++---++[0.1.0]: https://github.com/jamiepine/voicebox/releases/tag/v0.1.0diff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdnew file mode 100644index 00000000..dbadfd95--- /dev/null+++ b/CONTRIBUTING.md@@ -0,0 +1,252 @@+# Contributing to Voicebox++Thank you for your interest in contributing to Voicebox! This document provides guidelines and instructions for contributing.++## Code of Conduct++- Be respectful and inclusive+- Welcome newcomers and help them learn+- Focus on constructive feedback+- Respect different viewpoints and experiences++## Getting Started++### Prerequisites++- [Bun](https://bun.sh) - Package manager+- [Rust](https://rustup.rs) - For Tauri desktop app+- [Python 3.11+](https://python.org) - For backend+- Git++### Development Setup++1. **Fork and clone the repository**+ ```bash+ git clone https://github.com/YOUR_USERNAME/voicebox.git+ cd voicebox+ ```++2. **Install dependencies**+ ```bash+ bun install+ cd backend && pip install -r requirements.txt && cd ..+ ```++3. **Set up the database**+ ```bash+ cd backend+ python -c "from database import init_db; init_db()"+ ```++4. **Start development**+ ```bash+ # Terminal 1: Backend server+ bun run dev:server++ # Terminal 2: Desktop app+ bun run dev+ ```++## Development Workflow++### 1. Create a Branch++```bash+git checkout -b feature/your-feature-name+# or+git checkout -b fix/your-bug-fix+```++### 2. Make Your Changes++- Write clean, readable code+- Follow existing code style+- Add comments for complex logic+- Update documentation as needed++### 3. Test Your Changes++- Test manually in the app+- Ensure backend API endpoints work+- Check for TypeScript/Python errors+- Verify UI components render correctly++### 4. Commit Your Changes++Write clear, descriptive commit messages:++```bash+git commit -m "Add feature: voice profile export"+git commit -m "Fix: audio playback stops after 30 seconds"+```++### 5. Push and Create Pull Request++```bash+git push origin feature/your-feature-name+```++Then create a pull request on GitHub with:+- Clear description of changes+- Screenshots (for UI changes)+- Reference to related issues++## Code Style++### TypeScript/React++- Use TypeScript strict mode+- Follow React best practices+- Use functional components with hooks+- Prefer named exports+- Format with Biome (runs automatically)++```typescript+// Good+export function ProfileCard({ profile }: { profile: Profile }) {+ return <div>{profile.name}</div>;+}++// Avoid+export const ProfileCard = (props) => { ... }+```++### Python++- Follow PEP 8 style guide+- Use type hints+- Use async/await for I/O operations+- Format with Black (if configured)++```python+# Good+async def create_profile(name: str, language: str) -> Profile:+ """Create a new voice profile."""+ ...++# Avoid+def create_profile(name, language):+ ...+```++### Rust++- Follow Rust conventions+- Use meaningful variable names+- Handle errors explicitly+- Format with `rustfmt`++## Project Structure++```+voicebox/+├── app/ # Shared React frontend+│ └── src/+│ ├── components/ # UI components+│ ├── lib/ # Utilities and API client+│ └── hooks/ # React hooks+├── backend/ # Python FastAPI server+│ ├── main.py # API routes+│ ├── tts.py # Voice synthesis+│ └── ...+├── tauri/ # Desktop app wrapper+│ └── src-tauri/ # Rust backend+└── scripts/ # Build scripts+```++## Areas for Contribution++### 🐛 Bug Fixes++- Check existing issues for bugs to fix+- Test your fix thoroughly+- Add tests if possible++### ✨ New Features++- Check the roadmap in README.md+- Discuss major features in an issue first+- Keep features focused and well-scoped++### 📚 Documentation++- Improve README clarity+- Add code comments+- Write API documentation+- Create tutorials or guides++### 🎨 UI/UX Improvements++- Improve accessibility+- Enhance visual design+- Optimize performance+- Add animations/transitions++### 🔧 Infrastructure++- Improve build process+- Add CI/CD improvements+- Optimize bundle size+- Add testing infrastructure++## API Development++When adding new API endpoints:++1. **Add route in `backend/main.py`**+2. **Create Pydantic models in `backend/models.py`**+3. **Implement business logic in appropriate module**+4. **Update OpenAPI schema** (automatic with FastAPI)+5. **Regenerate TypeScript client:**+ ```bash+ bun run generate:api+ ```+6. **Update `backend/README.md`** with endpoint documentation++## Testing++Currently, testing is primarily manual. When adding tests:++- **Backend**: Use pytest for Python tests+- **Frontend**: Use Vitest for React component tests+- **E2E**: Use Playwright for end-to-end tests (future)++## Pull Request Process++1. **Update documentation** if needed+2. **Ensure code follows style guidelines**+3. **Test your changes thoroughly**+4. **Update CHANGELOG.md** with your changes+5. **Request review** from maintainers++### PR Checklist++- [ ] Code follows style guidelines+- [ ] Documentation updated+- [ ] Changes tested+- [ ] No breaking changes (or documented)+- [ ] CHANGELOG.md updated++## Release Process++Releases are managed by maintainers:++1. Version bump in `tauri.conf.json` and `Cargo.toml`+2. Update CHANGELOG.md+3. Create git tag: `git tag v0.2.0`+4. Push tag: `git push --tags`+5. GitHub Actions builds and releases++## Questions?++- Open an issue for bugs or feature requests+- Check existing issues and discussions+- Review the codebase to understand patterns++## License++By contributing, you agree that your contributions will be licensed under the MIT License.++---++Thank you for contributing to Voicebox! 🎉diff --git a/CURRENT_STATE.md b/CURRENT_STATE.mddeleted file mode 100644index 975b1447..00000000--- a/CURRENT_STATE.md+++ /dev/null@@ -1,447 +0,0 @@-# voicebox - Current State Overview--**Last Updated:** January 25, 2026-**Status:** ✅ MVP Core Features Working - Voice generation from Tauri app successful!-------## 🎯 What We Have--### ✅ **Fully Implemented & Working**--#### **Backend (Python FastAPI)**-- **Voice Profile Management**- - Create, read, update, delete profiles- - Add multiple audio samples per profile- - Multi-reference voice combination (combines multiple samples)- - Profile storage in SQLite + file system (`data/profiles/`)--- **Voice Generation**- - Qwen3-TTS model integration (1.7B and 0.6B support)- - Automatic model downloading from HuggingFace Hub- - Voice prompt caching for instant re-generation- - Support for English and Chinese- - Seed-based reproducibility- - GPU/CPU/MPS device detection--- **Generation History**- - Full CRUD operations- - Search by text content- - Filter by profile- - Pagination support- - Statistics endpoint- - Audio file storage (`data/generations/`)--- **Audio Transcription**- - Whisper integration for speech-to-text- - Language detection/selection- - Used for reference text extraction from samples--- **Database**- - SQLite with SQLAlchemy ORM- - Tables: `profiles`, `profile_samples`, `generations`, `projects` (ready for future)- - Automatic schema initialization--- **API Endpoints**- - RESTful API with FastAPI- - OpenAPI schema generation- - CORS enabled- - Health check endpoint- - File serving for audio files--#### **Frontend (React + TypeScript + Tauri)**-- **Voice Profile UI**- - Profile list with cards- - Create/edit profile dialog- - Upload audio samples with transcription- - Sample management (view/delete)- - Profile detail view--- **Generation UI**- - Form with profile selection- - Text input (up to 5000 chars)- - Language selection (en/zh)- - Optional seed input- - Loading states and error handling--- **History UI**- - Table view with pagination- - Search functionality- - Play audio inline- - Download audio files- - Delete generations--- **Server Settings**- - Connection form (local/remote mode)- - Server status display- - Health check integration--- **State Management**- - React Query for server state- - Zustand for client state (server URL, connection status)- - Type-safe API client--- **UI Components**- - shadcn/ui component library- - Tailwind CSS styling- - Responsive design- - Toast notifications- - Form validation with Zod--#### **Tauri Desktop App**-- **Rust Backend**- - Sidecar management for Python server- - Start/stop server commands- - Remote mode support (0.0.0.0 binding)- - Process lifecycle management--- **Build System**- - Tauri v2 configuration- - Platform-specific builds- - Dev tools in debug mode-------## 🏗️ Architecture--### **Project Structure**-```-voicebox/-├── app/ # Shared React frontend-│ ├── src/-│ │ ├── components/ # React components-│ │ │ ├── VoiceProfiles/ ✅ Complete-│ │ │ ├── Generation/ ✅ Complete-│ │ │ ├── History/ ✅ Complete-│ │ │ ├── ServerSettings/ ✅ Complete-│ │ │ └── AudioStudio/ 📦 Placeholder (future)-│ │ ├── lib/-│ │ │ ├── api/ # Type-safe API client ✅-│ │ │ ├── hooks/ # React Query hooks ✅-│ │ │ └── utils/ # Utilities ✅-│ │ └── stores/ # Zustand stores ✅-│-├── backend/ # Python FastAPI server-│ ├── main.py # FastAPI app + routes ✅-│ ├── models.py # Pydantic models ✅-│ ├── database.py # SQLAlchemy ORM ✅-│ ├── profiles.py # Profile management ✅-│ ├── history.py # History management ✅-│ ├── tts.py # Qwen3-TTS integration ✅-│ ├── transcribe.py # Whisper integration ✅-│ ├── studio.py # Audio studio (future)-│ └── utils/-│ ├── audio.py # Audio processing ✅-│ ├── cache.py # Voice prompt caching ✅-│ └── validation.py # Validation helpers ✅-│-├── tauri/ # Tauri desktop wrapper-│ ├── src/ # React entry point ✅-│ └── src-tauri/ # Rust backend ✅-│ └── src/main.rs # Sidecar management ✅-│-├── data/ # User data directory-│ ├── profiles/ # Profile audio samples-│ ├── generations/ # Generated audio files-│ ├── cache/ # Cached voice prompts-│ └── voicebox.db # SQLite database-│-└── scripts/ # Build & generation scripts- ├── generate-api.sh # OpenAPI client generation- └── build-server.sh # Python binary build-```--### **Data Flow**--```-User Action (Tauri App)- ↓-React Component (Form Submit)- ↓-React Query Hook (useGeneration)- ↓-API Client (apiClient.generateSpeech)- ↓-HTTP Request → FastAPI Backend- ↓-Backend Route Handler (/generate)- ↓-Business Logic:- 1. Get profile from DB- 2. Create voice prompt (with caching)- 3. Generate audio with Qwen3-TTS- 4. Save audio file- 5. Create history entry- ↓-Response (GenerationResponse)- ↓-React Query Cache Update- ↓-UI Refresh (History table updates)-```--### **Key Technologies**--| Layer | Technology | Purpose |-|-------|-----------|---------|-| **Desktop Framework** | Tauri v2 | Native desktop app wrapper |-| **Frontend Framework** | React 18 | UI components |-| **Language** | TypeScript | Type safety |-| **Styling** | Tailwind CSS | Utility-first CSS |-| **UI Components** | shadcn/ui | Component library |-| **State Management** | React Query + Zustand | Server & client state |-| **Form Handling** | React Hook Form + Zod | Form validation |-| **Backend Framework** | FastAPI | Async REST API |-| **Database** | SQLite + SQLAlchemy | Data persistence |-| **ML Models** | Qwen3-TTS + Whisper | Voice cloning + transcription |-| **Audio Processing** | librosa + soundfile | Audio I/O and processing |-| **Package Manager** | Bun | Fast JS/TS package management |-| **Build Tool** | Vite | Frontend bundling |-------## 🔑 Key Features & Capabilities--### **1. Voice Profile System**-- **Multi-sample support**: Add multiple audio samples per profile-- **Automatic combination**: Multiple samples are combined for better quality-- **Voice prompt caching**: Re-use voice prompts for instant re-generation-- **Audio validation**: Ensures samples meet quality requirements--### **2. Generation Pipeline**-- **Lazy model loading**: Model loads on first use-- **Device detection**: Automatically uses GPU if available-- **Caching layer**: Voice prompts cached by audio hash + text-- **Error handling**: Graceful degradation and clear error messages--### **3. History & Search**-- **Full-text search**: Search generations by text content-- **Pagination**: Efficient loading of large histories-- **Audio playback**: Inline audio player-- **File management**: Download and delete operations--### **4. Server/Client Architecture**-- **Local mode**: Backend runs alongside Tauri app-- **Remote mode**: Connect to remote GPU machine-- **One-click server**: Start server from UI-- **Connection management**: Persistent server URL storage-------## 📊 Database Schema--### **Tables**--```sql--- Voice Profiles-profiles- - id (PK, UUID)- - name (unique)- - description- - language (en/zh)- - created_at- - updated_at---- Profile Samples-profile_samples- - id (PK, UUID)- - profile_id (FK → profiles.id)- - audio_path- - reference_text---- Generations-generations- - id (PK, UUID)- - profile_id (FK → profiles.id)- - text- - language- - audio_path- - duration (seconds)- - seed (optional)- - created_at---- Projects (ready for future)-projects- - id (PK, UUID)- - name- - data (JSON)- - created_at- - updated_at-```-------## 🎨 UI Components Status--| Component | Status | Features |-|-----------|--------|----------|-| **ProfileList** | ✅ Complete | List, create, empty state |-| **ProfileCard** | ✅ Complete | Display profile info |-| **ProfileForm** | ✅ Complete | Create/edit dialog |-| **ProfileDetail** | ✅ Complete | View samples, add samples |-| **SampleUpload** | ✅ Complete | File upload + transcription |-| **GenerationForm** | ✅ Complete | Full generation form |-| **HistoryTable** | ✅ Complete | Table, search, pagination, play/download |-| **ConnectionForm** | ✅ Complete | Server URL input |-| **ServerStatus** | ✅ Complete | Health check display |-| **AudioStudio** | 📦 Placeholder | Timeline editor (future) |-------## 🔌 API Endpoints--### **Profiles**-- `POST /profiles` - Create profile-- `GET /profiles` - List all profiles-- `GET /profiles/{id}` - Get profile-- `PUT /profiles/{id}` - Update profile-- `DELETE /profiles/{id}` - Delete profile-- `POST /profiles/{id}/samples` - Add sample-- `GET /profiles/{id}/samples` - List samples-- `DELETE /profiles/samples/{id}` - Delete sample--### **Generation**-- `POST /generate` - Generate speech--### **History**-- `GET /history` - List generations (with filters)-- `GET /history/{id}` - Get generation-- `DELETE /history/{id}` - Delete generation-- `GET /history/stats` - Get statistics--### **Transcription**-- `POST /transcribe` - Transcribe audio--### **Audio**-- `GET /audio/{id}` - Serve audio file--### **Health**-- `GET /health` - Health check with model status--### **Model Management**-- `POST /models/load` - Load TTS model-- `POST /models/unload` - Unload TTS model-------## 🚀 What's Next (Planned Features)--### **Phase 2: Advanced Features**-- [ ] Multi-reference voice combination UI-- [ ] Batch generation (multiple variations)-- [ ] Advanced audio normalization-- [ ] Export options (MP3, OGG, etc.)-- [ ] M3GAN voice effect--### **Phase 3: Audio Studio**-- [ ] Timeline-based audio editor-- [ ] Word-level timestamps-- [ ] Project system (save/load sessions)-- [ ] Audio effects and filters-- [ ] Multi-track editing--### **Phase 4: Voice Design**-- [ ] Text-to-voice (no reference needed)-- [ ] Preset voices with style control-- [ ] Conversation mode (multi-speaker)-- [ ] Custom audio effects library-------## 📝 Code Quality Standards--- ✅ **Type safety**: TypeScript strict mode, Pydantic models-- ✅ **Modular architecture**: No files over 500 lines-- ✅ **Error handling**: Comprehensive error messages-- ✅ **Caching**: Voice prompt caching for performance-- ✅ **Database**: SQLAlchemy ORM with proper relationships-- ✅ **API design**: RESTful with OpenAPI schema-- ✅ **UI/UX**: Responsive, accessible, loading states-------## 🧪 Testing Status--- ✅ **Manual testing**: Voice generation working end-to-end-- 📦 **Unit tests**: Not yet implemented-- 📦 **Integration tests**: Not yet implemented-- 📦 **E2E tests**: Not yet implemented-------## 📦 Dependencies--### **Backend**-- FastAPI - Web framework-- SQLAlchemy - ORM-- Pydantic - Validation-- Qwen3-TTS - Voice cloning model-- Whisper - Speech recognition-- librosa - Audio processing-- soundfile - Audio I/O-- PyTorch - ML framework--### **Frontend**-- React 18 - UI framework-- TypeScript - Type safety-- React Query - Server state-- Zustand - Client state-- React Hook Form - Forms-- Zod - Schema validation-- Tailwind CSS - Styling-- shadcn/ui - Components-- Lucide React - Icons--### **Desktop**-- Tauri v2 - Desktop framework-- Rust - System backend-------## 🎯 Current Capabilities Summary--✅ **Working End-to-End:**-1. Create voice profiles with audio samples-2. Generate speech from text using cloned voices-3. View and manage generation history-4. Play and download generated audio-5. Search and filter history-6. Connect to local or remote backend-7. Automatic model downloading-8. Voice prompt caching for speed--🎉 **You just successfully generated voice from the Tauri app!**-------## 🔍 Key Files Reference--### **Backend Core**-- `backend/main.py` - FastAPI app and routes-- `backend/tts.py` - Qwen3-TTS model wrapper-- `backend/profiles.py` - Profile business logic-- `backend/history.py` - History business logic-- `backend/database.py` - Database models--### **Frontend Core**-- `app/src/App.tsx` - Main app component-- `app/src/lib/api/client.ts` - API client-- `app/src/lib/hooks/` - React Query hooks-- `app/src/stores/` - Zustand stores--### **Tauri**-- `tauri/src-tauri/src/main.rs` - Rust backend-- `tauri/src/main.tsx` - React entry point-------## 💡 Development Workflow--1. **Start backend**: `bun run dev:server` (or via Tauri)-2. **Start frontend**: `bun run dev` (Tauri) or `bun run dev:web` (web)-3. **Generate API client**: `bun run generate:api` (after backend changes)-4. **Build server binary**: `bun run build:server` (for Tauri bundling)-------**Ready to build more features! 🚀**diff --git a/LICENSE b/LICENSEnew file mode 100644index 00000000..5eea3a01--- /dev/null+++ b/LICENSE@@ -0,0 +1,21 @@+MIT License++Copyright (c) 2026 Voicebox Contributors++Permission is hereby granted, free of charge, to any person obtaining a copy+of this software and associated documentation files (the "Software"), to deal+in the Software without restriction, including without limitation the rights+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell+copies of the Software, and to permit persons to whom the Software is+furnished to do so, subject to the following conditions:++The above copyright notice and this permission notice shall be included in all+copies or substantial portions of the Software.++THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE+SOFTWARE.diff --git a/README.md b/README.mdindex 86835a64..e482939a 100644--- a/README.md+++ b/README.md@@ -182,14 +182,9 @@ Voicebox aims to be the **one-stop shop for everything voice** — cloning, synt## Development-### Prerequisites+See [SETUP.md](SETUP.md) for detailed setup instructions.-- [Bun](https://bun.sh) (package manager)-- [Rust](https://rustup.rs) (for Tauri)-- [Python 3.11+](https://python.org) (for backend)-- CUDA-capable GPU recommended (CPU inference supported but slower)--### Setup+### Quick Start```bash# Clone the repo@@ -206,6 +201,8 @@ cd backend && pip install -r requirements.txt && cd ..bun run dev```+**Prerequisites:** [Bun](https://bun.sh), [Rust](https://rustup.rs), [Python 3.11+](https://python.org). CUDA-capable GPU recommended (CPU inference supported but slower).+### Project Structure```@@ -222,18 +219,22 @@ voicebox/## Contributing-Contributions welcome! Whether it's bug fixes, new features, or documentation improvements.+Contributions welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.1. Fork the repo2. Create a feature branch3. Make your changes4. Submit a PR+## Security++Found a security vulnerability? Please report it responsibly. See [SECURITY.md](SECURITY.md) for details.+---## License-MIT License — use it however you want.+MIT License — see [LICENSE](LICENSE) for details.---diff --git a/SECURITY.md b/SECURITY.mdnew file mode 100644index 00000000..bdf1e917--- /dev/null+++ b/SECURITY.md@@ -0,0 +1,92 @@+# Security Policy++## Supported Versions++We release patches for security vulnerabilities. Which versions are eligible for receiving such patches depends on the CVSS v3.0 Rating:++| Version | Supported |+| ------- | ------------------ |+| 0.1.x | :white_check_mark: |+| < 0.1 | :x: |++## Reporting a Vulnerability++If you discover a security vulnerability, please report it responsibly:++1. **Do not** open a public GitHub issue+2. Email security details to: [[email protected]](mailto:[email protected])+3. Include:+ - Description of the vulnerability+ - Steps to reproduce+ - Potential impact+ - Suggested fix (if any)++We will:+- Acknowledge receipt within 48 hours+- Provide a timeline for addressing the issue+- Keep you informed of progress+- Credit you in the security advisory (if desired)++## Security Best Practices++### For Users++- **Keep Voicebox updated** - Updates include security patches+- **Verify downloads** - Only download from official releases+- **Local processing** - Voice data stays on your machine+- **Network security** - Use HTTPS when connecting to remote servers++### For Developers++- **Dependencies** - Keep all dependencies up to date+- **Code review** - All PRs require review before merging+- **Secrets** - Never commit API keys or signing keys+- **Signing** - All releases are cryptographically signed++## Known Security Considerations++### Local Processing++Voicebox processes all audio locally by default. Your voice data never leaves your machine unless you explicitly enable remote server mode.++### Remote Server Mode++When connecting to a remote server:+- Ensure the server is on a trusted network+- Use HTTPS for remote connections+- Verify server identity before connecting++### Auto-Updates++- Updates are cryptographically signed+- Signature verification happens before installation+- Only HTTPS endpoints are allowed++### Python Server++The embedded Python server:+- Runs locally by default (localhost only)+- Can be configured for remote access+- Uses standard FastAPI security practices++## Disclosure Timeline++- **Day 0**: Vulnerability reported+- **Day 1-2**: Initial assessment and acknowledgment+- **Day 3-7**: Investigation and fix development+- **Day 8-14**: Testing and release preparation+- **Day 15+**: Public disclosure (if applicable)++Timeline may vary based on severity and complexity.++## Security Updates++Security updates will be:+- Released as patch versions (e.g., 0.1.1)+- Documented in CHANGELOG.md+- Announced via GitHub releases+- Automatically delivered via auto-updater++---++Thank you for helping keep Voicebox secure! 🔒diff --git a/SETUP.md b/SETUP.mdindex b99b6580..5bc265ab 100644--- a/SETUP.md+++ b/SETUP.md@@ -1,6 +1,6 @@-# voicebox Setup Guide+# Development Setup Guide-Quick start guide for setting up the voicebox development environment.+This guide will help you set up the Voicebox development environment.## Prerequisites@@ -174,20 +174,23 @@ voicebox/## Troubleshooting-### Backend won't start+See [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) for common issues and solutions.++### Quick Fixes++**Backend won't start:**- Check Python version: `python --version` (needs 3.11+)- Ensure virtual environment is activated- Install dependencies: `pip install -r requirements.txt`-### Tauri build fails+**Tauri build fails:**- Ensure Rust is installed: `rustc --version`- Install Tauri CLI: `bunx @tauri-apps/cli install`-- Check `tauri/src-tauri/Cargo.toml` for correct dependencies+- Clean build: `cd tauri/src-tauri && cargo clean`-### OpenAPI client generation fails+**OpenAPI client generation fails:**- Ensure backend is running on port 8000- Check `curl http://localhost:8000/openapi.json` returns valid JSON-- Install openapi-typescript-codegen: `bun add -d openapi-typescript-codegen`## Model Downloads@@ -199,9 +202,13 @@ First-time usage will be slower due to model downloads, but subsequent runs will## Next Steps-1. ✅ TTS model loading implemented in `backend/tts.py`-2. ✅ API routes implemented in `backend/main.py`-3. Build React components in `app/src/components/`-4. Connect frontend to backend via generated API client+- Read [CONTRIBUTING.md](CONTRIBUTING.md) for contribution guidelines+- Check [docs/TROUBLESHOOTING.md](docs/TROUBLESHOOTING.md) if you encounter issues+- Review [backend/README.md](backend/README.md) for API documentation+- See [README.md](README.md) for project overview++## Additional Resources-See [README.md](./README.md) for architecture details and [docs/](./docs/) for detailed documentation.+- [Auto-Updater Setup](docs/AUTOUPDATER_QUICKSTART.md) - Configure automatic updates+- [Security Policy](SECURITY.md) - Security reporting and best practices+- [Changelog](CHANGELOG.md) - Version history and changesdiff --git a/docs/ANALYSIS.md b/docs/ANALYSIS.mddeleted file mode 100644index da3cac90..00000000--- a/docs/ANALYSIS.md+++ /dev/null@@ -1,1215 +0,0 @@-# Qwen3-TTS Implementation Analysis--Comprehensive analysis of five existing Qwen3-TTS implementations to inform the architecture of voicebox.-------## Projects Analyzed--1. **voice** - Rust CLI with Python backend-2. **Voice-Clone-Studio** - Feature-rich Gradio app-3. **Qwen3-TTS_server** - FastAPI REST API wrapper-4. **mimic** - Web app with audio studio (conceptually the best)-5. **qwen3-tts-enhanced** - Production-quality single-file Gradio app-------## 1. voice (Rust CLI)--**Repository:** `/Users/jamespine/Projects/voice`--### Architecture--**Dual-mode design:**-- One-shot mode: Single generation via CLI args-- Server mode: Long-running HTTP server (port 3000)--**Language split:**-- Rust: CLI, HTTP server, IPC orchestration-- Python: TTS inference, model management--**IPC Pattern:**-```-Rust → spawn Python subprocess → JSON over stdin/stdout → Rust-```--### Python Backend (`tts.py`)--**Model Management:**-- Lazy loading via `get_tts_model()`-- Qwen3-TTS CustomVoice models (0.6B/1.7B)-- `torch_dtype=torch.bfloat16` for VRAM efficiency-- Flash Attention 2 with SDPA fallback--**Voice Profile System:**-- Profiles stored in `~/.config/voice/profiles/`-- Each profile: `audio.wav` + `reference.txt`-- Automatic voice prompt creation and caching-- Hash-based cache invalidation (MD5 of audio + text)--**Generation Pipeline:**-```python-1. Load model (lazy, cached)-2. Get voice prompt (cached if available)-3. Set seed for reproducibility-4. model.generate_voice_clone(text, language, voice_prompt)-5. Save to temp file-6. Return path as JSON-```--**Special Features:**-- M3GAN voice effect (pitch shift + formant preservation)-- Uses `librosa` + `soundfile` for audio processing-- Voice profile listing with metadata--### Rust Frontend--**HTTP Server (`main.rs`):**-- Axum web framework-- Single endpoint: `POST /generate`-- JSON request/response-- Spawns Python subprocess per request--**CLI (`cli.rs`):**-- Clap for argument parsing-- Commands: `generate`, `server`, `list-voices`-- Profile management integrated--**Process Management:**-```rust-let mut child = Command::new("python3")- .arg("tts.py")- .stdin(Stdio::piped())- .stdout(Stdio::piped())- .spawn()?;-```--### Strengths--1. **Clean separation of concerns** - Rust for I/O, Python for ML-2. **Dual-mode flexibility** - CLI and server in one binary-3. **Voice profile abstraction** - Easy to add/manage voices-4. **M3GAN effect** - Unique feature, well-implemented-5. **Good error handling** - Rust's `Result` type enforced--### Weaknesses--1. **Synchronous inference** - Blocks during generation-2. **No concurrent requests** - Server spawns subprocess per request-3. **Limited caching** - Only voice prompts, not models-4. **No generation history** - Fire and forget-5. **Basic HTTP API** - No auth, rate limiting, or WebSocket streaming--### Key Learnings--- IPC via JSON stdin/stdout is simple and works-- Voice profile pattern is user-friendly-- For Tauri, we can skip the IPC layer and use direct HTTP/IPC channels-- M3GAN effect is a differentiator worth preserving-------## 2. Voice-Clone-Studio--**Repository:** `/Users/jamespine/Projects/Voice-Clone-Studio`--### Architecture--**Monolithic Gradio App:**-- Single file: `voice_clone_studio.py` (2,815 lines)-- Gradio for web UI-- Global state for model management-- Tab-based organization (6 tabs)--### Model Support--**Dual Engine:**-1. **Qwen3-TTS** - Base, CustomVoice, VoiceDesign-2. **VibeVoice TTS** - 1.5B/Large, multi-speaker--**Model sizes:**-- Qwen Small: 0.6B (~1GB VRAM)-- Qwen Large: 1.7B (~3GB VRAM)-- VibeVoice Small: 1.5B (~3GB VRAM)-- VibeVoice Large: ~6GB VRAM-- VibeVoice ASR: 7B (~14GB VRAM)--### Voice Prompt Caching--**Smart caching system:**-```python-def get_or_create_voice_prompt(audio_path, reference_text):- cache_key = hashlib.md5(audio_bytes + text.encode()).hexdigest()-- # Check in-memory cache- if cache_key in _voice_prompt_cache:- return _voice_prompt_cache[cache_key]-- # Check disk cache- prompt_file = f"{audio_path}.{cache_key}.prompt"- if os.path.exists(prompt_file):- prompt = torch.load(prompt_file)- _voice_prompt_cache[cache_key] = prompt- return prompt-- # Create new- prompt = model.create_voice_clone_prompt(audio, text)- torch.save(prompt, prompt_file)- _voice_prompt_cache[cache_key] = prompt- return prompt-```--**Cache invalidation:**-- Hash changes if audio or text changes-- Orphaned `.prompt` files cleaned up on demand--### Features--**Tab 1: Voice Clone**-- Clone from samples with Qwen or VibeVoice-- Sample selection dropdown-- Language selection (en/zh)-- Seed control for reproducibility-- Model size selection--**Tab 2: Conversation**-- Multi-speaker dialogue generation-- Script format: `[1]: Text`, `[2]: Text`-- Automatic speaker assignment-- Pause duration control between speakers--**Tab 3: Voice Presets**-- 9 pre-built Qwen speakers-- Style control (narrative, conversational, etc.)-- No sample needed--**Tab 4: Voice Design**-- Generate voices from text descriptions-- "Young female, energetic, bright tone"-- Qwen VoiceDesign model (1.7B only)--**Tab 5: Prep Samples**-- Audio/video file upload-- Auto-transcription (Whisper or VibeVoice ASR)-- Audio editing (trim, normalize, mono conversion)-- Save as voice sample--**Tab 6: Output History**-- Browse generated files-- Metadata display (timestamp, seed, engine, text)-- Re-generate from metadata--### VRAM Management--**Lazy loading + mutual exclusion:**-```python-def get_tts_model():- global _tts_model, _whisper_model, _vibe_voice_model-- # Unload ASR to free VRAM- if _whisper_model:- del _whisper_model- _whisper_model = None- if _vibe_voice_model:- del _vibe_voice_model- _vibe_voice_model = None-- torch.cuda.empty_cache()-- if not _tts_model:- _tts_model = load_model(...)-- return _tts_model-```--### Audio Processing--**Pipeline:**-1. **Input normalization:**- - Peak normalization to [-1, 1]- - 0.85 scaling (15% headroom)- - Stereo → mono (average channels)--2. **Video extraction:**- - ffmpeg subprocess for audio extraction- - 24kHz mono output--3. **Transcription:**- - Whisper medium model- - VibeVoice ASR with speaker diarization- - Auto-cleans `[Speaker X]:` labels--4. **Generation:**- - Seed control via `torch.manual_seed()`- - bfloat16 inference- - Direct .wav output (soundfile)--### Strengths--1. **Voice prompt caching** - Brilliant UX (⚡ cached indicator)-2. **Dual engine support** - Qwen + VibeVoice flexibility-3. **Feature-rich** - Voice design, presets, conversations, long-form-4. **VRAM efficiency** - Smart loading/unloading-5. **Video support** - Extract audio from video files-6. **Metadata tracking** - Every output has reproducibility data-7. **Sample management** - Integrated prep workspace--### Weaknesses--1. **2,815-line single file** - Impossible to maintain-2. **Global state everywhere** - Testing nightmare-3. **Duplicated code** - 5 model loaders with identical fallback logic-4. **No separation of concerns** - UI + logic + audio all mixed-5. **No concurrency** - All operations block UI-6. **No error recovery** - Generic error messages-7. **Hardcoded parameters** - CFG scale, inference steps in function bodies-8. **No tests** - Zero test coverage-9. **No logging** - Only print statements--### Key Learnings--**Adopt:**-- Voice prompt caching with hash validation-- Lazy model loading with automatic unloading-- Metadata alongside outputs-- Sample management patterns-- Status indicators (cached vs not)--**Avoid:**-- Monolithic files over 2,000 lines-- Global mutable state-- Duplicated logic without abstraction-- Hardcoded parameters-------## 3. Qwen3-TTS_server--**Repository:** `/Users/jamespine/Projects/Qwen3-TTS_server`--### Architecture--**FastAPI REST API:**-- Single server process-- Three endpoints: `/generate`, `/clone`, `/health`-- Deployed on RunPod with Docker-- Port 8000 (configurable)--### Project Structure--```-Qwen3-TTS_server/-├── main.py # FastAPI app-├── models/-│ └── tts.py # Model management-├── api/-│ └── routes.py # Endpoint definitions-├── utils/-│ └── audio.py # Audio processing-├── config.py # Settings-├── Dockerfile # Multi-stage build-└── requirements.txt # Dependencies-```--### API Design--**POST /generate**-```json-{- "text": "Hello world",- "language": "en",- "speaker": "default",- "seed": 42-}--Response:-{- "audio_url": "https://...",- "duration": 2.5,- "sample_rate": 24000-}-```--**POST /clone**-```json-{- "text": "Hello world",- "language": "en",- "reference_audio": "base64_encoded_audio",- "reference_text": "This is my voice",- "seed": 42-}--Response:-{- "audio_url": "https://...",- "duration": 2.5,- "sample_rate": 24000-}-```--**GET /health**-```json-{- "status": "healthy",- "model_loaded": true,- "gpu_available": true,- "vram_used_mb": 1024-}-```--### Model Management--**Singleton pattern:**-```python-class ModelManager:- _instance = None- _model = None-- def __new__(cls):- if not cls._instance:- cls._instance = super().__new__(cls)- return cls._instance-- def get_model(self):- if not self._model:- self._model = self._load_model()- return self._model-```--**Lazy loading:**-- Model loaded on first request-- Kept in memory for subsequent requests-- No unloading (server dedicated to TTS)--### Deployment--**Docker multi-stage build:**-```dockerfile-FROM nvidia/cuda:12.1.0-runtime-ubuntu22.04 AS base-# Install dependencies--FROM base AS builder-# Install Python packages--FROM base AS runtime-COPY --from=builder /usr/local/lib/python3.10 /usr/local/lib/python3.10-# Runtime only-```--**RunPod integration:**-- Configured for GPU instances-- Automatic model download on startup-- Health checks for orchestration--### Strengths--1. **Clean API design** - RESTful, well-documented-2. **Proper separation** - Routes, models, utils in separate modules-3. **Singleton model manager** - Better than global variables-4. **Health endpoint** - Essential for deployment-5. **Docker deployment** - Production-ready containerization-6. **Base64 audio input** - No file upload needed for cloning--### Weaknesses--1. **No authentication** - Wide open API-2. **No rate limiting** - DoS vulnerable-3. **Sequential inference** - No request queuing-4. **No caching** - Voice prompts recreated every time-5. **No streaming** - Returns only after full generation-6. **No WebSocket** - Can't send progress updates-7. **Synchronous endpoints** - Blocks during generation-8. **No storage** - Audio URLs expire quickly-9. **Limited error handling** - Generic 500 errors--### Key Learnings--**Adopt:**-- FastAPI for REST API-- Modular structure (routes, models, utils)-- Health endpoint for monitoring-- Base64 audio input option-- Docker deployment pattern--**Improve:**-- Add async/await throughout-- Implement request queue-- Add WebSocket for streaming-- Cache voice prompts-- Authentication and rate limiting-------## 4. mimic--**Repository:** `/Users/jamespine/Projects/mimic`--### Architecture--**Three-tier web app:**-- Frontend: Vanilla JS (no framework)-- Backend: Python FastAPI-- Database: SQLite--**Best-structured backend** of all projects analyzed.--### Project Structure--```-mimic/-├── backend/-│ ├── main.py # FastAPI app-│ ├── models.py # Pydantic models-│ ├── tts.py # TTS inference-│ ├── transcribe.py # Whisper ASR-│ ├── profiles.py # Voice profile management-│ ├── history.py # Generation history-│ ├── studio.py # Audio studio features-│ └── database.py # SQLite ORM-├── frontend/-│ ├── index.html-│ ├── app.js # Main app (2,794 lines)-│ ├── studio.js # Audio studio (2,363 lines)-│ ├── profiles.js # Profile management-│ └── history.js # History UI-└── data/- ├── profiles/ # Voice profiles- ├── generations/ # Generated audio- └── mimic.db # SQLite database-```--### Backend Design--**Async/await throughout:**-```python[email protected]("/generate")-async def generate(request: GenerateRequest):- audio = await tts.generate_async(- text=request.text,- profile_id=request.profile_id,- language=request.language- )-- history_entry = await db.create_generation(- profile_id=request.profile_id,- text=request.text,- audio_path=audio.path- )-- return history_entry-```--**Modular separation:**-- `models.py` - Pydantic request/response models-- `tts.py` - TTS inference logic only-- `transcribe.py` - ASR logic only-- `profiles.py` - CRUD for voice profiles-- `history.py` - CRUD for generation history-- `studio.py` - Audio editing features-- `database.py` - SQLAlchemy ORM--### Features--**Voice Profiles:**-- Create from audio file + reference text-- Multiple samples per profile (combined)-- Metadata: name, description, tags, language-- Thumbnail generation from waveform--**Generation History:**-- SQLite database with full-text search-- Filters: profile, date range, language-- Regeneration from history-- Export to various formats--**Audio Studio:**-- Timeline-based editing-- Multiple audio tracks-- Word-level timestamps (Whisper alignment)-- Trim, fade, volume control-- Export with normalization--**Projects:**-- Save/load studio sessions-- Project metadata and versioning-- Export project as single file--### Database Schema--```sql-CREATE TABLE profiles (- id INTEGER PRIMARY KEY,- name TEXT UNIQUE,- description TEXT,- language TEXT,- created_at TIMESTAMP,- updated_at TIMESTAMP-);--CREATE TABLE profile_samples (- id INTEGER PRIMARY KEY,- profile_id INTEGER,- audio_path TEXT,- reference_text TEXT,- FOREIGN KEY (profile_id) REFERENCES profiles(id)-);--CREATE TABLE generations (- id INTEGER PRIMARY KEY,- profile_id INTEGER,- text TEXT,- language TEXT,- audio_path TEXT,- duration REAL,- seed INTEGER,- created_at TIMESTAMP,- FOREIGN KEY (profile_id) REFERENCES profiles(id)-);--CREATE TABLE projects (- id INTEGER PRIMARY KEY,- name TEXT,- data JSON,- created_at TIMESTAMP,- updated_at TIMESTAMP-);-```--### Frontend Design--**Major issue: Monolithic classes**--**app.js (2,794 lines):**-```javascript-class MimicApp {- constructor() {- this.profiles = [];- this.generations = [];- this.currentProfile = null;- this.currentGeneration = null;- // ... 50+ properties- }-- // 80+ methods, no organization- async loadProfiles() { ... }- async createProfile() { ... }- async deleteProfile() { ... }- async generate() { ... }- async loadHistory() { ... }- // ... hundreds more lines-}-```--**studio.js (2,363 lines):**-```javascript-class AudioStudio {- constructor() {- this.wavesurfer = null;- this.timeline = null;- this.tracks = [];- this.regions = [];- this.words = [];- // ... 40+ properties- }-- // 60+ methods, all mixed- initWavesurfer() { ... }- addTrack() { ... }- removeTrack() { ... }- playPause() { ... }- exportAudio() { ... }- // ... hundreds more lines-}-```--**Global state everywhere:**-```javascript-let app = null;-let studio = null;-let currentProfile = null;-let isGenerating = false;-```--**No module system:**-- All files loaded via `<script>` tags-- No bundler, no tree-shaking-- jQuery for DOM manipulation-- No type safety--### Audio Studio Implementation--**WaveSurfer.js integration:**-```javascript-this.wavesurfer = WaveSurfer.create({- container: '#waveform',- waveColor: '#4a9eff',- progressColor: '#1e3a8a',- cursorColor: '#ef4444',- height: 128,- normalize: true,- plugins: [- TimelinePlugin.create(),- RegionsPlugin.create()- ]-});-```--**Word-level timestamps:**-- Whisper alignment API-- Each word: `{ word, start, end }`-- Clickable timeline-- Visual highlighting during playback--**Track mixing:**-- Multiple audio files on timeline-- Independent volume control-- Fade in/out per track-- Export as single mixed audio--### Strengths--1. **Best backend structure** - Modular, async, clean separation-2. **Database-backed** - Proper persistence layer-3. **Audio studio** - Timeline editing is unique-4. **Word-level timestamps** - Great UX for editing-5. **Project system** - Save/load sessions-6. **Search and filters** - Full-text search in history-7. **Multi-sample profiles** - Combine multiple references-8. **Proper API design** - RESTful with Pydantic validation--### Weaknesses--1. **Frontend is amateur:**- - 2,794-line app.js- - 2,363-line studio.js- - Global state everywhere- - No module system- - No TypeScript- - jQuery instead of modern framework--2. **No real-time updates** - Polling instead of WebSocket-3. **No concurrency** - Single generation at a time-4. **No batch processing** - One at a time only-5. **Limited audio formats** - WAV only-6. **No export options** - Can't export profiles or projects easily--### Key Learnings--**Backend patterns to adopt:**-- Async/await throughout-- Modular file structure (tts.py, transcribe.py, profiles.py, history.py)-- SQLite for persistence-- Pydantic for validation-- Proper CRUD separation--**Frontend patterns to avoid:**-- Monolithic classes over 2,000 lines-- Global state-- jQuery-based architecture-- No type safety--**Features to adopt:**-- Audio studio with timeline-- Word-level timestamps-- Project system-- Multi-sample voice profiles-- Generation history with search-------## 5. qwen3-tts-enhanced--**Repository:** `/Users/jamespine/Projects/qwen3-tts-enhanced`--### Architecture--**Production-quality Gradio app:**-- Single file: `app.py` (1,892 lines)-- Much cleaner than Voice-Clone-Studio despite being monolithic-- Clear section comments and function organization-- Cross-platform support (Windows, Mac, Linux)--### Unique Features--**Multi-Reference Voice Cloning:**-```python-def combine_references(audio_paths, reference_texts):- """Combine multiple voice samples for better quality"""- combined_audio = []- combined_text = []-- for audio_path, text in zip(audio_paths, reference_texts):- audio, sr = sf.read(audio_path)- audio = normalize_audio(audio)- combined_audio.append(audio)- combined_text.append(text)-- # Mix with equal weighting and normalization- mixed = np.concatenate(combined_audio)- mixed = normalize_audio(mixed)-- combined_text_str = " ".join(combined_text)-- return mixed, combined_text_str-```--**Batch Variation Generation:**-- Generate N variations of same text-- Different seeds for each-- Compare and pick best output-- Parallel generation support--**Smart Audio Mixing:**-```python-def normalize_audio(audio, target_db=-20):- """Normalize to target loudness with peak limiting"""- # Convert to float32- audio = audio.astype(np.float32)-- # Calculate current RMS- rms = np.sqrt(np.mean(audio**2))-- # Calculate target RMS- target_rms = 10**(target_db / 20)-- # Apply gain- if rms > 0:- gain = target_rms / rms- audio = audio * gain-- # Peak limiting to prevent clipping- audio = np.clip(audio, -1.0, 1.0)-- return audio-```--### Code Organization--**Clear sections:**-```python-# ============================================-# CONFIGURATION AND CONSTANTS-# ============================================--# ============================================-# MODEL MANAGEMENT-# ============================================--# ============================================-# AUDIO PROCESSING-# ============================================--# ============================================-# GENERATION FUNCTIONS-# ============================================--# ============================================-# UI COMPONENTS-# ============================================--# ============================================-# APPLICATION LAUNCH-# ============================================-```--**Consistent patterns:**-- All functions have docstrings-- Type hints on critical functions-- Error handling with context-- Progress callbacks throughout--### Cross-Platform Support--**Graceful degradation:**-```python-def get_audio_device():- """Get audio playback device, cross-platform"""- try:- import sounddevice as sd- devices = sd.query_devices()- return sd.default.device- except ImportError:- logger.warning("sounddevice not available, using fallback")- return None- except Exception as e:- logger.warning(f"Audio device detection failed: {e}")- return None--def play_audio(audio_path):- """Play audio with platform-specific fallback"""- device = get_audio_device()-- if device is not None:- # Use sounddevice- audio, sr = sf.read(audio_path)- sd.play(audio, sr)- elif sys.platform == 'darwin':- # macOS fallback- subprocess.run(['afplay', audio_path])- elif sys.platform == 'win32':- # Windows fallback- import winsound- winsound.PlaySound(audio_path, winsound.SND_FILENAME)- else:- # Linux fallback- subprocess.run(['aplay', audio_path])-```--**Path handling:**-```python-def ensure_cross_platform_path(path):- """Convert paths to platform-specific format"""- return os.path.normpath(path)--def get_config_dir():- """Get config directory, respecting platform conventions"""- if sys.platform == 'win32':- return os.path.join(os.getenv('APPDATA'), 'qwen-tts')- elif sys.platform == 'darwin':- return os.path.expanduser('~/Library/Application Support/qwen-tts')- else:- return os.path.expanduser('~/.config/qwen-tts')-```--### Quality Improvements--**Reference audio validation:**-```python-def validate_reference_audio(audio_path):- """Check if reference audio is suitable for cloning"""- audio, sr = sf.read(audio_path)-- # Check duration (3-10 seconds ideal)- duration = len(audio) / sr- if duration < 2:- return False, "Audio too short (minimum 2 seconds)"- if duration > 30:- return False, "Audio too long (maximum 30 seconds)"-- # Check for silence- rms = np.sqrt(np.mean(audio**2))- if rms < 0.01:- return False, "Audio is too quiet or silent"-- # Check for clipping- if np.abs(audio).max() > 0.99:- return False, "Audio is clipping (reduce input gain)"-- # Check sample rate- if sr < 16000:- return False, f"Sample rate too low ({sr}Hz, minimum 16kHz)"-- return True, "Audio is suitable"-```--**Generation quality settings:**-```python-# Conservative defaults for quality-DEFAULT_CFG_SCALE = 4.5 # Higher = more faithful to prompt-DEFAULT_STEPS = 20 # More steps = better quality-DEFAULT_TEMPERATURE = 1.0--# Fast mode (lower quality)-FAST_CFG_SCALE = 3.0-FAST_STEPS = 10-```--### Backward Compatibility--**Config migration:**-```python-def migrate_config(config):- """Migrate old config to new format"""- version = config.get('version', 1)-- if version == 1:- # Add new fields from version 2- config['batch_size'] = 1- config['version'] = 2-- if version == 2:- # Add new fields from version 3- config['multi_reference'] = False- config['version'] = 3-- return config-```--**Legacy sample format:**-```python-def load_sample(sample_path):- """Load voice sample, supporting both old and new formats"""- # Try new format (JSON metadata)- json_path = sample_path.replace('.wav', '.json')- if os.path.exists(json_path):- with open(json_path) as f:- metadata = json.load(f)- return {- 'audio': sample_path,- 'text': metadata['reference_text'],- 'language': metadata.get('language', 'en')- }-- # Try old format (text file)- txt_path = sample_path.replace('.wav', '.txt')- if os.path.exists(txt_path):- with open(txt_path) as f:- text = f.read().strip()- return {- 'audio': sample_path,- 'text': text,- 'language': 'en' # Assume English- }-- # No metadata found- return None-```--### Strengths--1. **Multi-reference cloning** - Combine samples for better quality-2. **Batch variations** - Generate multiple outputs, pick best-3. **Smart audio normalization** - RMS targeting + peak limiting-4. **Cross-platform support** - Graceful degradation everywhere-5. **Backward compatibility** - Config migration, legacy formats-6. **Reference validation** - Check audio before generation-7. **Quality presets** - Fast vs. high-quality modes-8. **Clean code** - Well-organized despite being single file-9. **Good error messages** - Contextual, actionable-10. **Progress callbacks** - User sees what's happening--### Weaknesses--1. **Still monolithic** - 1,892 lines in one file-2. **No database** - File-based storage-3. **No history** - Can't search past generations-4. **No API** - Gradio UI only-5. **No concurrent generation** - One at a time-6. **No streaming** - Wait for full generation--### Key Learnings--**Production patterns to adopt:**-- Multi-reference combination for quality-- Audio validation before generation-- Cross-platform graceful degradation-- Config migration for backward compatibility-- RMS normalization with peak limiting-- Quality presets (fast vs. high-quality)-- Clear section organization in code-- Progress callbacks everywhere--**Clean code practices:**-- Docstrings on all functions-- Type hints where useful-- Error messages with context-- Consistent naming conventions-------## Comparative Analysis--### Model Management--| Project | Pattern | VRAM Strategy | Caching |-|---------|---------|--------------|---------|-| voice | Lazy load | Single model | Voice prompts only |-| Voice-Clone-Studio | Lazy + mutex | Unload on switch | Voice prompts + disk |-| Qwen3-TTS_server | Singleton | Keep in memory | None |-| mimic | Lazy async | Single model | None |-| qwen3-tts-enhanced | Lazy load | Single model | Voice prompts only |--**Winner:** Voice-Clone-Studio (most sophisticated caching)--### Architecture--| Project | Backend | Frontend | Separation | Modularity |-|---------|---------|----------|------------|------------|-| voice | Python subprocess | Rust CLI | Excellent | Good |-| Voice-Clone-Studio | Gradio monolith | Gradio | None | Poor |-| Qwen3-TTS_server | FastAPI | None (API only) | Excellent | Excellent |-| mimic | FastAPI | Vanilla JS | Good | Backend: Excellent, Frontend: Poor |-| qwen3-tts-enhanced | Gradio monolith | Gradio | None | Good (sections) |--**Winner:** mimic (backend), Qwen3-TTS_server (overall separation)--### Features--| Feature | voice | Voice-Clone-Studio | Qwen3-TTS_server | mimic | qwen3-tts-enhanced |-|---------|-------|-------------------|-----------------|-------|-------------------|-| Voice cloning | ✓ | ✓ | ✓ | ✓ | ✓ |-| Multi-reference | ✗ | ✗ | ✗ | ✓ | ✓ |-| Voice design | ✗ | ✓ | ✗ | ✗ | ✗ |-| Presets | ✗ | ✓ (9) | ✗ | ✗ | ✗ |-| Conversations | ✗ | ✓ | ✗ | ✗ | ✗ |-| History | ✗ | ✓ (file) | ✗ | ✓ (database) | ✗ |-| Audio studio | ✗ | ✗ | ✗ | ✓ | ✗ |-| Batch generation | ✗ | ✗ | ✗ | ✗ | ✓ |-| M3GAN effect | ✓ | ✗ | ✗ | ✗ | ✗ |--**Winner:** mimic (most comprehensive feature set)--### Code Quality--| Project | Lines | Organization | Type Safety | Tests | Documentation |-|---------|-------|--------------|-------------|-------|---------------|-| voice | ~500 | Good | Rust: Yes, Python: No | None | README only |-| Voice-Clone-Studio | 2,815 | Poor | No | None | Good README |-| Qwen3-TTS_server | ~800 | Excellent | Partial (Pydantic) | None | API docs |-| mimic | ~4,000 | Backend: Good, Frontend: Poor | Backend: Partial | None | Basic |-| qwen3-tts-enhanced | 1,892 | Good | Partial | None | Good |--**Winner:** Qwen3-TTS_server (best organized)--### Production Readiness--| Project | Deployment | Error Handling | Cross-Platform | Graceful Degradation | Monitoring |-|---------|-----------|----------------|----------------|---------------------|------------|-| voice | Binary | Good (Rust) | Yes | Limited | None |-| Voice-Clone-Studio | Python script | Basic | Yes | Good (flash attn) | None |-| Qwen3-TTS_server | Docker | Basic | Linux (container) | None | Health endpoint |-| mimic | Python script | Good | Yes | Limited | None |-| qwen3-tts-enhanced | Python script | Excellent | Excellent | Excellent | None |--**Winner:** qwen3-tts-enhanced (most robust), Qwen3-TTS_server (deployment)-------## Recommended Architecture for voicebox--### Backend (Python + FastAPI)--**Structure (from mimic):**-```-backend/-├── main.py # FastAPI app-├── models.py # Pydantic models-├── tts.py # TTS inference-├── transcribe.py # Whisper ASR-├── profiles.py # Voice profile management-├── history.py # Generation history-├── studio.py # Audio editing-├── effects.py # M3GAN, etc.-├── database.py # SQLite ORM-└── utils/- ├── audio.py # Audio processing- ├── cache.py # Voice prompt caching- └── validation.py # Input validation-```--**Patterns to adopt:**-- Async/await throughout (mimic)-- Voice prompt caching (Voice-Clone-Studio)-- Multi-reference combination (qwen3-tts-enhanced)-- Audio validation (qwen3-tts-enhanced)-- Cross-platform audio (qwen3-tts-enhanced)-- Health endpoint (Qwen3-TTS_server)-- Graceful degradation (qwen3-tts-enhanced)--### Frontend (Tauri + TypeScript + React)--**Structure:**-```-frontend/-├── src/-│ ├── components/-│ │ ├── VoiceProfiles/-│ │ ├── Generation/-│ │ ├── AudioStudio/-│ │ └── History/-│ ├── lib/-│ │ ├── api.ts # Backend API client-│ │ ├── audio.ts # Audio utilities-│ │ └── store.ts # State management-│ ├── types/-│ │ └── index.ts # TypeScript types-│ └── App.tsx-└── src-tauri/- ├── src/- │ └── main.rs # Tauri backend- └── tauri.conf.json-```--**Avoid monolithic components:**-- Keep components under 300 lines-- Proper state management (Zustand or Jotai)-- TypeScript everywhere-- Component-based architecture--### Features Priority--**Phase 1 (MVP):**-1. Voice profile management-2. Single-reference voice cloning-3. Generation history (database)-4. Basic audio playback--**Phase 2:**-1. Multi-reference combination-2. Batch variation generation-3. M3GAN effect-4. Audio normalization--**Phase 3:**-1. Audio studio with timeline-2. Word-level timestamps-3. Project system-4. Export options--**Phase 4:**-1. Voice design-2. Preset voices-3. Conversation mode-4. Advanced effects--### Technology Stack--**Backend:**-- FastAPI (async REST API)-- SQLAlchemy (database ORM)-- Pydantic (validation)-- Qwen3-TTS (model)-- Whisper (transcription)-- librosa + soundfile (audio)--**Frontend:**-- Tauri (desktop framework)-- React (UI framework)-- TypeScript (type safety)-- Tailwind CSS (styling)-- Zustand (state management)-- WaveSurfer.js (audio visualization)--**Database:**-- SQLite (local storage)-- Alembic (migrations)--### Key Differentiators--What will make voicebox better:--1. **Clean architecture** - Avoid monolithic files from all projects-2. **TypeScript frontend** - Type safety unlike mimic-3. **Desktop-first** - Native feel via Tauri-4. **Voice prompt caching** - Fast generations like Voice-Clone-Studio-5. **Multi-reference** - Quality like qwen3-tts-enhanced-6. **Audio studio** - Timeline editing like mimic-7. **M3GAN effect** - Unique feature from voice-8. **Production code** - No amateur patterns-9. **Proper state management** - No global variables-10. **Modular from day one** - Easy to extend-------## Conclusion--All five projects have valuable lessons:--- **voice**: Clean Rust/Python split, M3GAN effect-- **Voice-Clone-Studio**: Brilliant caching, feature-rich but poorly organized-- **Qwen3-TTS_server**: Best modular structure, API design-- **mimic**: Best backend architecture, great features, terrible frontend-- **qwen3-tts-enhanced**: Production patterns, quality focus, cross-platform--voicebox will cherry-pick the best patterns from each while avoiding their architectural mistakes.diff --git a/docs/AUTOUPDATER.md b/docs/AUTOUPDATER.mdindex 621d01eb..06de6002 100644--- a/docs/AUTOUPDATER.md+++ b/docs/AUTOUPDATER.md@@ -1,6 +1,6 @@-# Tauri v2 Autoupdater Setup+# Auto-Updater Documentation-The autoupdater has been configured for this project. Follow these steps to complete the setup.+Voicebox includes automatic updates powered by Tauri's updater plugin. This document explains how it works for both users and developers.## 1. Generate Signing Keys@@ -148,28 +148,15 @@ Add your private key to GitHub secrets:- Add `TAURI_SIGNING_PRIVATE_KEY` with the content of `~/.tauri/voicebox.key`- Add `TAURI_SIGNING_PRIVATE_KEY_PASSWORD` (empty string if no password)-## 5. Frontend Integration (Already Completed)+## Frontend Integration-The frontend integration is already complete in this project with the following components:+The frontend integration is complete with automatic update notifications and manual update checks:-### Automatic Update Notifications-- `app/src/components/UpdateNotification.tsx` - Shows a banner when updates are available-- Automatically checks for updates on app startup-- Displays download/install progress-- Only shows in Tauri desktop builds+- **Update Notification Banner** - Appears automatically when updates are available+- **Settings Panel** - Manual "Check for Updates" button in Settings tab+- **Update Hook** - React hook handles all update operations-### Manual Update Check-- `app/src/components/ServerSettings/UpdateStatus.tsx` - Settings panel for updates-- Allows manual update checks via "Check for Updates" button-- Shows current version and update status-- Located in the Settings tab (only visible in Tauri builds)--### Update Hook-- `app/src/hooks/useAutoUpdater.ts` - React hook for update functionality-- Handles update checking, downloading, and installation-- Includes Tauri context detection (won't run in web builds)--The components are already integrated into the main App layout.+See `docs/AUTOUPDATER_QUICKSTART.md` for a quick setup guide.## Security Notesdiff --git a/docs/BACKEND_COMPLETE.md b/docs/BACKEND_COMPLETE.mddeleted file mode 100644index 8ec00905..00000000--- a/docs/BACKEND_COMPLETE.md+++ /dev/null@@ -1,246 +0,0 @@-# 🎉 Backend Implementation Complete!--Your voicebox backend is fully implemented and ready for frontend integration.--## What Was Built--### 📦 Complete Backend (1,500 lines, 12 files)--I've implemented a **production-quality FastAPI backend** based on the best patterns from your reference projects:--#### Core Modules--1. **TTS Module** (`backend/tts.py`)- - Qwen3-TTS model loading and inference- - Voice prompt creation with caching- - Multi-reference combination- - Model size switching (1.7B/0.6B)- - Async generation--2. **Profiles Module** (`backend/profiles.py`)- - Full CRUD for voice profiles- - Multi-sample support per profile- - Audio validation- - Automatic sample combination--3. **History Module** (`backend/history.py`)- - Generation tracking with full metadata- - Search and filtering- - Pagination- - Statistics--4. **Transcription Module** (`backend/transcribe.py`)- - Whisper ASR integration- - Language hints- - Model management--5. **Database Module** (`backend/database.py`)- - SQLite with SQLAlchemy ORM- - Clean schema design- - Proper relationships--6. **Utils Module** (`backend/utils/`)- - Audio processing and validation- - Voice prompt caching (memory + disk)- - Input validation--7. **API Module** (`backend/main.py`)- - 20+ REST endpoints- - File upload/download- - Health checks- - Model management--## 🎯 What's Different from References--### Better Than ALL References--| Feature | Your Backend | Reference Projects |-|---------|-------------|-------------------|-| **Code Organization** | ✅ 12 modular files (~1,500 lines) | ❌ 1-2 monolithic files (2,815 lines) |-| **Type Safety** | ✅ 100% Pydantic + type hints | ❌ Little to no typing |-| **Async/Await** | ✅ Full async throughout | ⚠️ Partial or none |-| **Caching** | ✅ Voice prompts (memory + disk) | ⚠️ Partial or none |-| **Multi-Sample** | ✅ Advanced combination | ⚠️ Basic or none |-| **Database** | ✅ SQLite with search | ❌ File-based |-| **API Design** | ✅ 20+ RESTful endpoints | ⚠️ 3 endpoints or Gradio only |-| **Error Handling** | ✅ Detailed + contextual | ⚠️ Generic |--### Pattern Sources--- ✅ **Architecture** from mimic (best structured)-- ✅ **Caching** from Voice-Clone-Studio (brilliant implementation)-- ✅ **Audio processing** from qwen3-tts-enhanced (quality focus)-- ✅ **API design** from Qwen3-TTS_server (clean REST)-- ✅ **Best practices** from professional software engineering--### What We Avoided--- ❌ No 2,815-line monolithic files-- ❌ No global mutable state-- ❌ No synchronous blocking-- ❌ No code duplication-- ❌ No poor separation of concerns--## 📚 Documentation Created--1. **`backend/README.md`** - Complete API documentation-2. **`backend/IMPLEMENTATION_STATUS.md`** - Implementation status-3. **`backend/example_usage.py`** - Working example client-4. **`docs/BACKEND_IMPLEMENTATION.md`** - Implementation details-5. **`docs/COMPETITIVE_ANALYSIS.md`** - Comparison with references--## 🚀 Ready For--### ✅ Immediate Integration--The backend is ready for:-- Tauri desktop app integration-- Web app deployment-- OpenAPI client generation-- Production deployment--### 🔌 All Endpoints Working--```-Health:- GET /health--Profiles:- POST /profiles- GET /profiles- GET /profiles/{id}- PUT /profiles/{id}- DELETE /profiles/{id}- POST /profiles/{id}/samples- GET /profiles/{id}/samples- DELETE /profiles/samples/{id}--Generation:- POST /generate--History:- GET /history- GET /history/{id}- DELETE /history/{id}- GET /history/stats--Audio:- GET /audio/{id}--Transcription:- POST /transcribe--Models:- POST /models/load- POST /models/unload-```--## 🎬 Next Steps--### 1. Test the Backend--```bash-# Terminal 1: Start backend-cd backend-python -m venv venv-source venv/bin/activate-pip install -r requirements.txt-python main.py--# Terminal 2: Test it-curl http://localhost:8000/health-python backend/example_usage.py-```--### 2. Generate OpenAPI Client--```bash-# Start backend first, then:-curl http://localhost:8000/openapi.json > app/openapi.json--cd app-npx openapi-typescript-codegen \- --input openapi.json \- --output src/lib/api \- --client fetch-```--### 3. Build Frontend--Now you can build the Tauri frontend that:-- Creates voice profiles-- Uploads audio samples-- Generates speech-- Views history-- Downloads audio--## 📊 Performance--- **First generation:** 6-10 seconds (creates prompt)-- **Cached generation:** 1-2 seconds (uses cache)-- **Model loading:** 3-5 seconds (one-time)-- **Voice prompt cache:** Persists across restarts--## 🎯 Key Benefits--1. **Maintainable** - Clean, modular, documented-2. **Type-safe** - Catch errors at development time-3. **Fast** - Caching makes repeat generations instant-4. **Complete** - All core features implemented-5. **Professional** - Production-ready patterns throughout--## 🔮 Future Enhancements (Optional)--These are planned but not blocking frontend work:--**Phase 2:**-- WebSocket streaming for progress-- Batch generation endpoint-- Audio effects (M3GAN)-- Voice design--**Phase 3:**-- Audio studio timeline-- Word-level timestamps-- Project management--**Phase 4:**-- Authentication-- Rate limiting-- Docker deployment-- CI/CD--## 📖 Reference Projects Analyzed--Based on analysis of:-- **voice** - Rust CLI with Python backend-- **Voice-Clone-Studio** - Feature-rich Gradio app-- **Qwen3-TTS_server** - Clean FastAPI wrapper-- **mimic** - Web app with best backend structure-- **qwen3-tts-enhanced** - Production-quality Gradio--## ✨ Summary--**Your backend is:**-- ✅ Fully implemented (20+ endpoints)-- ✅ Production-ready (error handling, health checks)-- ✅ Well-documented (5 documentation files)-- ✅ Type-safe (100% Pydantic)-- ✅ Performant (voice prompt caching)-- ✅ Maintainable (clean architecture)--**Status:** READY FOR FRONTEND INTEGRATION--**No blockers.** You can start building the Tauri app immediately!-------## Questions?--See the documentation:-- `backend/README.md` - API reference-- `backend/example_usage.py` - Usage examples-- `docs/BACKEND_IMPLEMENTATION.md` - Implementation details-- `docs/COMPETITIVE_ANALYSIS.md` - vs. reference projects--Happy building! 🚀diff --git a/docs/BACKEND_IMPLEMENTATION.md b/docs/BACKEND_IMPLEMENTATION.mddeleted file mode 100644index 2a1a8366..00000000--- a/docs/BACKEND_IMPLEMENTATION.md+++ /dev/null@@ -1,368 +0,0 @@-# Backend Implementation Summary--Complete implementation of the voicebox backend based on analysis of reference projects.--## What's Been Built--### ✅ Core Modules (100% Complete)--#### 1. TTS Module (`tts.py`)-**Pattern Source:** mimic + Voice-Clone-Studio--**Features:**-- Lazy model loading with device detection (CPU/CUDA/MPS)-- Voice prompt creation with dual caching (memory + disk)-- Multi-reference combination for quality improvement-- Async generation with seed control-- Model size switching (1.7B/0.6B)-- Proper memory management and cleanup--**Key Improvements Over References:**-- Cleaner async/await patterns than mimic-- Better error handling than Voice-Clone-Studio-- Proper type hints throughout-- Modular design vs monolithic files--#### 2. Profiles Module (`profiles.py`)-**Pattern Source:** mimic + qwen3-tts-enhanced--**Features:**-- Full CRUD operations for voice profiles-- Multi-sample support per profile-- Audio validation before adding samples-- Automatic sample combination for generation-- File storage organization in `data/profiles/`-- Database persistence with timestamps--**Key Improvements:**-- Better separation of concerns than mimic-- Proper async implementation-- Validation integrated at module level-- Cleaner API than reference implementations--#### 3. History Module (`history.py`)-**Pattern Source:** mimic--**Features:**-- Generation history tracking with full metadata-- Search and filtering capabilities-- Pagination support-- Statistics endpoint-- Audio file cleanup on deletion-- Profile-based filtering--**Key Improvements:**-- Returns total count for pagination-- Statistics aggregation-- Better query patterns-- Proper cleanup of associated files--#### 4. Transcribe Module (`transcribe.py`)-**Pattern Source:** Voice-Clone-Studio + mimic--**Features:**-- Whisper model loading and transcription-- Language hint support-- Word-level timestamps (placeholder for full implementation)-- Model size selection-- VRAM management--**Differences:**-- Simplified vs Voice-Clone-Studio's complex setup-- Prepared for future timestamp integration-- Better device handling--#### 5. Database Module (`database.py`)-**Pattern Source:** mimic--**Features:**-- SQLite with SQLAlchemy ORM-- Proper foreign key relationships-- Automatic timestamp management-- UUID primary keys-- Clean session management--**Schema:**-- `profiles` - Voice profile metadata-- `profile_samples` - Multi-sample support-- `generations` - Complete generation history-- `projects` - Future audio studio projects--#### 6. Models Module (`models.py`)-**Pattern Source:** Qwen3-TTS_server + mimic--**Features:**-- Pydantic v2 models for validation-- Request/response models separated-- Proper field validation-- Type safety throughout-- `from_attributes` for ORM compatibility--#### 7. Utils Module--##### `audio.py`-**Pattern Source:** qwen3-tts-enhanced + Voice-Clone-Studio--- RMS normalization with peak limiting-- Audio loading with resampling-- Audio saving in consistent format-- Reference audio validation (duration, RMS, clipping)--##### `cache.py`-**Pattern Source:** Voice-Clone-Studio (their best pattern)--- MD5-based cache key generation-- Dual caching (memory + disk)-- Automatic cache invalidation-- Corrupted cache file handling-- Persistent across server restarts--##### `validation.py`-**Pattern Source:** Original design--- Text validation-- Language code validation-- File path validation-- Reusable validation patterns--### ✅ API Implementation (`main.py`)--**Pattern Source:** Qwen3-TTS_server + mimic--**Complete REST API:**-- 20+ endpoints covering all features-- Proper HTTP status codes-- File upload handling-- File serving for audio-- Health check with model status-- Model management endpoints-- Error handling with details-- CORS configuration--**Endpoints Organized:**-1. Health & Info (2)-2. Voice Profiles (8)-3. Generation (1)-4. History (4)-5. Audio Files (1)-6. Transcription (1)-7. Model Management (2)--## Architecture Comparison--### Reference Projects Analysis--| Aspect | voice | Voice-Clone-Studio | Qwen3-TTS_server | mimic | voicebox |-|--------|-------|-------------------|------------------|-------|----------|-| **Code Organization** | Good | Poor (2815 lines) | Excellent | Backend: Good | Excellent |-| **Type Safety** | Rust: Yes, Python: No | No | Partial | Partial | Full (Pydantic) |-| **Async/Await** | No (subprocess) | No | Limited | Full | Full |-| **Caching** | Voice prompts | Voice prompts + disk | None | None | Voice prompts + disk |-| **Multi-Sample** | No | No | No | Yes | Yes |-| **Database** | File-based | File-based | None | SQLite | SQLite |-| **API Design** | HTTP basic | Gradio only | REST clean | REST good | REST excellent |-| **Error Handling** | Good | Basic | Basic | Good | Excellent |-| **File Lines** | ~500 | 2815 | ~800 | ~4000 | ~1500 |--### What Makes voicebox Better--#### 1. **Clean Architecture**-- No monolithic files (largest file: ~300 lines in main.py)-- Proper module separation-- Each file has single responsibility-- Easy to test and maintain--#### 2. **Production-Ready Patterns**-- Full async/await (not bolted on)-- Proper error handling with context-- Type safety throughout-- Database transactions-- Resource cleanup--#### 3. **Best Patterns from Each Reference**-- Voice prompt caching → Voice-Clone-Studio-- Multi-sample profiles → qwen3-tts-enhanced + mimic-- Audio normalization → qwen3-tts-enhanced-- API structure → Qwen3-TTS_server-- Database design → mimic-- VRAM management → Voice-Clone-Studio--#### 4. **Avoiding Reference Mistakes**-- ❌ No 2000+ line files-- ❌ No global mutable state-- ❌ No code duplication-- ❌ No mixed concerns-- ❌ No poor error messages--## API Feature Matrix--| Feature | Implemented | Source Pattern |-|---------|-------------|----------------|-| Voice profile CRUD | ✅ | mimic |-| Multi-sample profiles | ✅ | qwen3-tts-enhanced + mimic |-| Voice prompt caching | ✅ | Voice-Clone-Studio |-| Generation with seed | ✅ | All |-| History tracking | ✅ | mimic |-| History search | ✅ | mimic |-| Transcription | ✅ | Voice-Clone-Studio |-| Audio validation | ✅ | qwen3-tts-enhanced |-| Model management | ✅ | Original |-| File serving | ✅ | mimic |-| Health checks | ✅ | Qwen3-TTS_server |-| Statistics | ✅ | Original |-| Batch generation | ⏳ | TODO |-| WebSocket streaming | ⏳ | TODO |-| Audio effects (M3GAN) | ⏳ | TODO |-| Voice design | ⏳ | TODO |-| Audio studio | ⏳ | TODO |-| Projects | ⏳ | TODO |--## File Structure--```-backend/-├── main.py # 300 lines - FastAPI app + all routes-├── models.py # 100 lines - Pydantic models-├── tts.py # 200 lines - TTS inference-├── transcribe.py # 150 lines - Whisper ASR-├── profiles.py # 250 lines - Profile management-├── history.py # 150 lines - History management-├── studio.py # 70 lines - Audio studio (skeleton)-├── database.py # 90 lines - SQLite ORM-├── requirements.txt # Dependencies-├── README.md # Complete API documentation-├── example_usage.py # Example client code-└── utils/- ├── __init__.py- ├── audio.py # 120 lines - Audio processing- ├── cache.py # 90 lines - Voice prompt caching- └── validation.py # 65 lines - Input validation--Total: ~1,500 lines (clean, maintainable, type-safe)-```--Compare to references:-- voice: ~500 lines (but limited features)-- Voice-Clone-Studio: 2,815 lines in ONE file-- Qwen3-TTS_server: ~800 lines (but no history/profiles)-- mimic backend: ~1,200 lines (our closest match, but less clean)--## Testing Strategy--### Manual Testing-1. Start server: `python -m backend.main`-2. Run example: `python backend/example_usage.py`-3. Test with curl/Postman--### Unit Testing (TODO)-```-tests/-├── test_tts.py-├── test_profiles.py-├── test_history.py-├── test_transcribe.py-├── test_audio.py-└── test_cache.py-```--## Performance Characteristics--### Voice Prompt Caching-- **First generation:** ~5-10 seconds- - Load model: 3-5s- - Create prompt: 2-3s- - Generate: 1-2s--- **Subsequent generations:** ~1-2 seconds- - Model loaded: 0s- - Prompt cached: 0s- - Generate: 1-2s--### Multi-Sample Profiles-- Combining 2-3 samples: +1-2 seconds on first use-- Cached after first use-- Better quality than single sample--### Model Sizes-- **1.7B:** Best quality, ~3GB VRAM, slower on CPU-- **0.6B:** Good quality, ~1GB VRAM, faster on CPU--## Next Steps--### Phase 1: Testing & Polish-1. Add unit tests-2. Add integration tests-3. Error handling edge cases-4. Documentation improvements--### Phase 2: Advanced Features-1. Batch generation endpoint-2. WebSocket for progress-3. Audio effects (M3GAN, pitch, etc.)-4. Voice design (text-to-voice)--### Phase 3: Audio Studio-1. Word-level timestamps (full implementation)-2. Timeline mixing-3. Trim/fade operations-4. Project save/load-5. Export options--### Phase 4: Production Features-1. Authentication & authorization-2. Rate limiting-3. Usage tracking-4. Model caching strategies-5. Distributed generation (multiple GPUs)--## Deployment--### Development-```bash-cd backend-python -m venv venv-source venv/bin/activate-pip install -r requirements.txt-python main.py-```--### Production-```bash-# Using uvicorn directly-uvicorn backend.main:app --host 0.0.0.0 --port 8000 --workers 4--# Or using gunicorn-gunicorn backend.main:app -w 4 -k uvicorn.workers.UvicornWorker-```--### Docker (TODO)-```dockerfile-FROM python:3.11-# ... setup-CMD ["uvicorn", "backend.main:app", "--host", "0.0.0.0"]-```--## Conclusion--The voicebox backend is **production-ready** for:-- ✅ Voice profile management-- ✅ Multi-sample voice cloning-- ✅ Generation history-- ✅ Transcription-- ✅ Basic audio processing--It successfully combines:-- **Best architecture** from mimic-- **Best caching** from Voice-Clone-Studio-- **Best audio processing** from qwen3-tts-enhanced-- **Best API design** from Qwen3-TTS_server-- **Best practices** from professional software engineering--While avoiding:-- ❌ Monolithic files-- ❌ Global state-- ❌ Poor separation of concerns-- ❌ Code duplication-- ❌ Weak typing--The result is a clean, maintainable, production-quality backend that's ready for the Tauri frontend integration.diff --git a/docs/BIOME_SETUP.md b/docs/BIOME_SETUP.mddeleted file mode 100644index e1bead8d..00000000--- a/docs/BIOME_SETUP.md+++ /dev/null@@ -1,268 +0,0 @@-# Biome Setup Complete--Biome v2.3.12 is now configured for voicebox.--## What Was Configured--### ✅ Installed-- `@biomejs/[email protected]` (exact version pinned)-- Removed ESLint and all related dependencies--### ✅ Configuration Files Created--**`biome.json`** - Main configuration:-- Formatter: 2-space indents, 100 char line width-- Linter: Recommended rules + React best practices-- JavaScript: Single quotes, double quotes for JSX-- Tailwind CSS: `@tailwind` directives allowed--**`.vscode/settings.json`** - IDE integration:-- Biome as default formatter-- Format on save enabled-- Auto-import organization-- Prettier and ESLint disabled--**`.vscode/extensions.json`** - Recommended extensions:-- Biome (biomejs.biome)-- Tailwind CSS IntelliSense-- Rust Analyzer-- Tauri Extension--**`.biomeignore`** - Ignored files:-- `node_modules`, `dist`, `target`-- Generated API client-- Config files-- Lock files--### ✅ Package Scripts--Run from root:-```bash-bun run lint # Check linting issues-bun run lint:fix # Fix linting issues-bun run format # Format all files-bun run format:check # Check formatting-bun run check # Check everything (lint + format)-bun run check:fix # Fix everything-bun run ci # Strict check for CI/CD-```--Run from `app/`:-```bash-bun run lint # Lint app/src-bun run lint:fix # Fix lint issues-bun run format # Format app/src-bun run check # Check app/src-```--## Current Status--✅ **26 files checked**-✅ **1 warning** (accessibility - safe to ignore for now)-✅ **0 errors**--The single warning is:-```-app/src/App.tsx:14:11 - Provide explicit type prop for button-```--This is a good accessibility practice but not blocking. Add `type="button"` when you build real components.--## Biome vs ESLint + Prettier--| Feature | Biome | ESLint + Prettier |-|---------|-------|------------------|-| Speed | ~15ms for 26 files | ~500ms+ |-| Single tool | ✅ | ❌ (2 tools) |-| TypeScript support | ✅ Native | ⚠️ Plugins needed |-| JSON/CSS formatting | ✅ | ⚠️ Limited |-| Auto-fix | ✅ | ⚠️ Partial |-| Import sorting | ✅ Built-in | ❌ Needs plugin |--## Configuration Highlights--### Linting Rules--**Enabled (errors):**-- `noUnusedImports` - Remove unused imports-- `noDoubleEquals` - Use `===` instead of `==`-- `useHookAtTopLevel` - React hooks at component top level-- `useExhaustiveDependencies` - Complete React hook deps--**Enabled (warnings):**-- `noUnusedVariables` - Warn on unused vars (not error)-- `noExplicitAny` - Discourage `any` type-- `useButtonType` - Accessibility for buttons--**Disabled:**-- `noNonNullAssertion` - Allow `!` in React (safe with `getElementById`)-- `useFilenamingConvention` - Allow flexible naming-- `noUnknownAtRules` - Allow Tailwind CSS directives--### Formatting Style--```typescript-// Single quotes for JS/TS-import { foo } from 'bar';--// Double quotes for JSX-<Component prop="value" />--// Always semicolons-const x = 5;--// Always arrow parens-const fn = (x) => x + 1;--// Trailing commas-const obj = {- a: 1,- b: 2,-};-```--## VS Code Integration--1. **Install extension:**- - Search "Biome" in VS Code extensions- - Install "Biome" by Biomejs--2. **Automatic:**- - Format on save ✅- - Auto-import organization ✅- - Inline errors/warnings ✅- - Quick fixes ✅--3. **Manual formatting:**- - macOS: `⇧⌥F`- - Windows/Linux: `Shift+Alt+F`--## CI/CD Integration--Add to GitHub Actions:--```yaml-- name: Setup Bun- uses: oven-sh/setup-bun@v2--- name: Install dependencies- run: bun install--- name: Check code quality- run: bun run ci-```--The `ci` command is strict and fails if any fixes are needed.--## Migration Notes--### Removed-- ❌ `eslint`-- ❌ `@typescript-eslint/eslint-plugin`-- ❌ `@typescript-eslint/parser`-- ❌ `eslint-plugin-react-hooks`-- ❌ `eslint-plugin-react-refresh`-- ❌ `.eslintrc.cjs`--### Why Biome?--From your CLAUDE.md:-> "You are a senior software engineer specializing in Rust and TypeScript. You pride yourself on clean production ready code."--Biome is:-- **Written in Rust** - Aligns with your stack (Tauri is Rust)-- **Fast** - 20-30x faster than ESLint-- **Simple** - One tool instead of two (ESLint + Prettier)-- **Production-ready** - Used by Meta, Vercel, and other large teams-- **Type-aware** - Understands TypeScript natively--## Common Commands--```bash-# Format everything-bun run format--# Fix all auto-fixable issues-bun run check:fix--# Check before commit (no changes)-bun run ci--# Format specific file-bunx biome format --write app/src/App.tsx--# Check specific directory-bunx biome check app/src/components-```--## Advanced Configuration--### Add custom rules--Edit `biome.json`:-```json-{- "linter": {- "rules": {- "complexity": {- "noExcessiveCognitiveComplexity": {- "level": "error",- "options": {- "maxAllowedComplexity": 15- }- }- }- }- }-}-```--### Per-file configuration--Use `overrides` in `biome.json`:-```json-{- "overrides": [- {- "includes": ["app/src/lib/api/**"],- "linter": {- "enabled": false- }- }- ]-}-```--## Troubleshooting--### Biome not formatting in VS Code--1. Open Command Palette (`Cmd+Shift+P`)-2. Search "Format Document With..."-3. Select "Biome"-4. Check if Biome extension is installed--### Conflicts with Prettier--Make sure Prettier is disabled in VS Code settings (already configured in `.vscode/settings.json`).--### Performance issues--Biome is extremely fast, but if you experience issues:-```bash-# Clear Biome cache-rm -rf .biome-cache--# Reinstall-bun remove @biomejs/biome-bun add -D -E @biomejs/biome-```--## Next Steps--1. ✅ Biome is ready to use-2. ✅ Run `bun run format` to format existing code-3. ✅ Install Biome VS Code extension-4. 🚀 Start building frontend components--All future code will be automatically formatted and linted on save!diff --git a/docs/COMPETITIVE_ANALYSIS.md b/docs/COMPETITIVE_ANALYSIS.mddeleted file mode 100644index 72d41662..00000000--- a/docs/COMPETITIVE_ANALYSIS.md+++ /dev/null@@ -1,377 +0,0 @@-# Competitive Analysis: voicebox vs Reference Implementations--Detailed comparison showing how voicebox improves upon each reference project.--## Executive Summary--voicebox combines the **best patterns** from 5 reference implementations while avoiding their **architectural mistakes**. The result is a production-quality system that's maintainable, type-safe, and feature-rich.-------## 1. vs. voice (Rust CLI)--### What voice Does Well-- ✅ Clean Rust/Python separation-- ✅ M3GAN voice effect-- ✅ Voice profile abstraction-- ✅ Good error handling in Rust--### What voicebox Does Better-| Aspect | voice | voicebox |-|--------|-------|----------|-| **Concurrency** | Spawns subprocess per request | Async with persistent model |-| **Caching** | Voice prompts only | Voice prompts + disk |-| **History** | None | Full database with search |-| **API** | Basic HTTP | Full REST with 20+ endpoints |-| **Multi-sample** | No | Yes |-| **Type safety** | Python: No | Full Pydantic |-| **Database** | File-based | SQLite with migrations |--### Architecture Comparison-```-voice:-Rust HTTP → spawn Python → JSON IPC → generate → return--voicebox:-FastAPI → async TTS → cached prompt → generate → save to DB-```--**Winner:** voicebox (persistent models, caching, database)-------## 2. vs. Voice-Clone-Studio (Gradio)--### What Voice-Clone-Studio Does Well-- ✅ Brilliant voice prompt caching (memory + disk)-- ✅ Dual engine support (Qwen + VibeVoice)-- ✅ Feature-rich (voice design, presets, conversations)-- ✅ VRAM efficiency (smart loading/unloading)-- ✅ Metadata tracking--### What voicebox Does Better-| Aspect | Voice-Clone-Studio | voicebox |-|--------|-------------------|----------|-| **Code organization** | 2,815 lines in ONE file | ~1,500 lines across 12 files |-| **State management** | Global mutable state | Proper dependency injection |-| **Type safety** | None | Full Pydantic + type hints |-| **Testing** | Impossible | Easy (modular) |-| **API** | Gradio only | REST + future WebSocket |-| **Separation of concerns** | All mixed | Clean modules |-| **Error handling** | Generic messages | Contextual errors |-| **Code duplication** | 5 identical model loaders | Single abstraction |--### Code Quality Comparison-```python-# Voice-Clone-Studio-def generate_voice_clone(...): # Line 450- global _tts_model, _whisper_model- if _whisper_model:- del _whisper_model- _whisper_model = None- # ... 200 more lines of mixed logic--# voicebox-async def generate(self, text: str, voice_prompt: dict, ...) -> Tuple[np.ndarray, int]:- """Generate audio from text using voice prompt."""- self.load_model()- # ... clean, focused logic-```--**Winner:** voicebox (maintainable architecture)-------## 3. vs. Qwen3-TTS_server (FastAPI)--### What Qwen3-TTS_server Does Well-- ✅ Clean API design-- ✅ Proper separation (routes, models, utils)-- ✅ Singleton model manager-- ✅ Health endpoint-- ✅ Docker deployment-- ✅ Base64 audio input--### What voicebox Does Better-| Aspect | Qwen3-TTS_server | voicebox |-|--------|-----------------|----------|-| **Authentication** | None | TODO (planned) |-| **Rate limiting** | None | TODO (planned) |-| **Concurrency** | Sequential | Async throughout |-| **Caching** | None | Voice prompts cached |-| **Streaming** | No | TODO (WebSocket planned) |-| **Storage** | Temporary | Persistent database |-| **History** | None | Full tracking + search |-| **Profiles** | None | Full CRUD + samples |-| **Error handling** | Basic | Detailed + contextual |-| **Features** | 3 endpoints | 20+ endpoints |--### Feature Matrix-| Feature | Qwen3-TTS_server | voicebox |-|---------|-----------------|----------|-| Generate | ✅ | ✅ |-| Clone | ✅ | ✅ |-| Health | ✅ | ✅ |-| Profiles | ❌ | ✅ |-| Multi-sample | ❌ | ✅ |-| History | ❌ | ✅ |-| Search | ❌ | ✅ |-| Transcription | ❌ | ✅ |-| File serving | ❌ | ✅ |-| Statistics | ❌ | ✅ |--**Winner:** voicebox (far more features)-------## 4. vs. mimic (Web App)--### What mimic Does Well-- ✅ **Best backend structure** of all references-- ✅ Async/await throughout-- ✅ Database-backed persistence-- ✅ Audio studio with timeline-- ✅ Word-level timestamps-- ✅ Project system-- ✅ Full-text search--### What voicebox Does Better-| Aspect | mimic | voicebox |-|--------|-------|----------|-| **Type safety** | Partial | Full Pydantic |-| **Caching** | None | Voice prompts |-| **Multi-sample** | Basic | Advanced (combination) |-| **Audio validation** | Limited | Comprehensive |-| **API docs** | Basic | Auto-generated OpenAPI |-| **Model management** | Manual | Lazy + auto-cleanup |-| **Error messages** | Generic | Detailed + actionable |-| **Code organization** | Good | Excellent |--### Backend Comparison-```-mimic backend:-~1,200 lines, async, modular, but:-- No caching-- Basic multi-sample-- No audio validation-- Manual model management--voicebox backend:-~1,500 lines, async, modular, plus:-- Voice prompt caching-- Advanced multi-sample with combination-- Comprehensive validation-- Automatic lazy loading-```--### Where mimic is Still Ahead-- ⚠️ **Audio studio** - Timeline editing, mixing-- ⚠️ **Word timestamps** - Full implementation-- ⚠️ **Projects** - Save/load sessions--**Planned for voicebox Phase 3**--**Winner:** voicebox (backend), but mimic has features we'll add later-------## 5. vs. qwen3-tts-enhanced (Gradio)--### What qwen3-tts-enhanced Does Well-- ✅ Multi-reference cloning-- ✅ Batch variations-- ✅ Smart audio normalization-- ✅ Cross-platform support-- ✅ Backward compatibility-- ✅ Audio validation-- ✅ Quality presets-- ✅ Clean code (despite being monolithic)-- ✅ Good error messages--### What voicebox Does Better-| Aspect | qwen3-tts-enhanced | voicebox |-|--------|-------------------|----------|-| **Architecture** | 1,892 lines in one file | 12 modular files |-| **Database** | File-based | SQLite |-| **History** | None | Full tracking |-| **API** | Gradio only | REST API |-| **Concurrency** | One at a time | Async support |-| **Profiles** | File-based | Database CRUD |--### What We Adopted-- ✅ Multi-reference combination-- ✅ Audio validation patterns-- ✅ RMS normalization-- ✅ Cross-platform audio handling-- ✅ Good error messages--**Winner:** voicebox (better architecture, adopted best features)-------## Composite Feature Matrix--| Feature | voice | Voice-Clone-Studio | Qwen3-TTS_server | mimic | qwen3-tts-enhanced | **voicebox** |-|---------|-------|-------------------|------------------|-------|-------------------|--------------|-| **Architecture** | | | | | | |-| Modular code | ⚠️ | ❌ | ✅ | ✅ | ⚠️ | ✅ |-| Type safety | ⚠️ | ❌ | ⚠️ | ⚠️ | ❌ | ✅ |-| Async/await | ❌ | ❌ | ⚠️ | ✅ | ❌ | ✅ |-| Database | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ |-| REST API | ⚠️ | ❌ | ✅ | ✅ | ❌ | ✅ |-| **Features** | | | | | | |-| Voice cloning | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |-| Multi-sample | ❌ | ❌ | ❌ | ✅ | ✅ | ✅ |-| Voice prompt cache | ⚠️ | ✅ | ❌ | ❌ | ⚠️ | ✅ |-| History tracking | ❌ | ⚠️ | ❌ | ✅ | ❌ | ✅ |-| Search | ❌ | ❌ | ❌ | ✅ | ❌ | ✅ |-| Transcription | ❌ | ✅ | ❌ | ✅ | ❌ | ✅ |-| Audio validation | ❌ | ❌ | ❌ | ⚠️ | ✅ | ✅ |-| Audio studio | ❌ | ❌ | ❌ | ✅ | ❌ | ⏳ |-| Voice design | ❌ | ✅ | ❌ | ❌ | ❌ | ⏳ |-| M3GAN effect | ✅ | ❌ | ❌ | ❌ | ❌ | ⏳ |-| **Quality** | | | | | | |-| Multi-reference | ❌ | ❌ | ❌ | ⚠️ | ✅ | ✅ |-| Normalization | ❌ | ⚠️ | ❌ | ⚠️ | ✅ | ✅ |-| Quality presets | ❌ | ❌ | ❌ | ❌ | ✅ | ⏳ |-| **Production** | | | | | | |-| Error handling | ✅ | ⚠️ | ⚠️ | ✅ | ✅ | ✅ |-| Health checks | ❌ | ❌ | ✅ | ❌ | ❌ | ✅ |-| Model management | ⚠️ | ✅ | ⚠️ | ⚠️ | ⚠️ | ✅ |-| Docker ready | ❌ | ❌ | ✅ | ❌ | ❌ | ⏳ |--Legend:-- ✅ Excellent/Complete-- ⚠️ Partial/Basic-- ❌ Missing/Poor-- ⏳ Planned-------## Code Quality Metrics--### Lines of Code-| Project | Total Lines | Largest File | Files |-|---------|------------|--------------|-------|-| voice | ~500 | main.rs (200) | 5 |-| Voice-Clone-Studio | 2,815 | voice_clone_studio.py (2,815) | 1 |-| Qwen3-TTS_server | ~800 | server.py (400) | 4 |-| mimic backend | ~1,200 | app.js (2,794) | 7 |-| qwen3-tts-enhanced | 1,892 | app.py (1,892) | 1 |-| **voicebox** | **~1,500** | **main.py (300)** | **12** |--### Maintainability Score-| Project | Organization | Type Safety | Modularity | Testing | Total |-|---------|-------------|-------------|------------|---------|-------|-| voice | 7/10 | 5/10 | 7/10 | 0/10 | **19/40** |-| Voice-Clone-Studio | 2/10 | 0/10 | 1/10 | 0/10 | **3/40** |-| Qwen3-TTS_server | 9/10 | 5/10 | 9/10 | 0/10 | **23/40** |-| mimic | 8/10 | 5/10 | 8/10 | 0/10 | **21/40** |-| qwen3-tts-enhanced | 6/10 | 3/10 | 3/10 | 0/10 | **12/40** |-| **voicebox** | **10/10** | **10/10** | **10/10** | **3/10** | **33/40** |-------## Performance Comparison--### Voice Prompt Generation-| Project | First Gen | Cached Gen | Cache Type |-|---------|-----------|------------|------------|-| voice | 8-12s | 8-12s | None |-| Voice-Clone-Studio | 6-10s | 1-2s | Memory + Disk |-| Qwen3-TTS_server | 8-12s | 8-12s | None |-| mimic | 8-12s | 8-12s | None |-| qwen3-tts-enhanced | 6-10s | 6-10s | Basic |-| **voicebox** | **6-10s** | **1-2s** | **Memory + Disk** |--### Multi-Sample Combination-| Project | Supports | Method | Quality |-|---------|----------|--------|---------|-| voice | ❌ | - | - |-| Voice-Clone-Studio | ❌ | - | - |-| Qwen3-TTS_server | ❌ | - | - |-| mimic | ✅ | Simple concat | Good |-| qwen3-tts-enhanced | ✅ | Normalized concat | Excellent |-| **voicebox** | ✅ | **Normalized concat** | **Excellent** |-------## What voicebox Achieves--### Combines Best of All References-1. **Architecture** from mimic + Qwen3-TTS_server-2. **Caching** from Voice-Clone-Studio-3. **Audio processing** from qwen3-tts-enhanced-4. **Effects** from voice (planned)-5. **Features** from all projects--### Avoids All Major Pitfalls-1. ❌ No monolithic files (Voice-Clone-Studio, qwen3-tts-enhanced)-2. ❌ No global state (Voice-Clone-Studio, voice)-3. ❌ No synchronous blocking (voice, Voice-Clone-Studio)-4. ❌ No missing features (Qwen3-TTS_server)-5. ❌ No poor separation (Voice-Clone-Studio)--### Production-Ready From Day One-- ✅ Type-safe with Pydantic-- ✅ Async/await throughout-- ✅ Proper error handling-- ✅ Database persistence-- ✅ Clean architecture-- ✅ Easy to test-- ✅ Auto-generated API docs-- ✅ Health monitoring-------## Future Roadmap--### Phase 1: Current State ✅-- [x] Core TTS with caching-- [x] Profile management-- [x] Multi-sample support-- [x] History tracking-- [x] Transcription-- [x] REST API--### Phase 2: Next Quarter-- [ ] WebSocket streaming-- [ ] Batch generation-- [ ] Audio effects (M3GAN)-- [ ] Voice design-- [ ] Unit tests (80% coverage)--### Phase 3: Following Quarter-- [ ] Audio studio (from mimic)-- [ ] Word-level timestamps-- [ ] Project management-- [ ] Export options--### Phase 4: Production-- [ ] Authentication-- [ ] Rate limiting-- [ ] Docker deployment-- [ ] CI/CD pipeline-- [ ] Monitoring & logging-------## Conclusion--voicebox backend is:--1. **Most maintainable** - Clean architecture, modular, type-safe-2. **Most feature-rich** - Combines features from all references-3. **Best performance** - Caching + async + proper pooling-4. **Production-ready** - Error handling, health checks, monitoring-5. **Future-proof** - Easy to extend, test, deploy--It's the **only implementation** that combines:-- ✅ Clean code (Qwen3-TTS_server)-- ✅ Advanced caching (Voice-Clone-Studio)-- ✅ Quality audio (qwen3-tts-enhanced)-- ✅ Full features (mimic)-- ✅ Type safety (none had this)-- ✅ Production patterns (our innovation)--**Result:** A professional-grade system ready for the Tauri frontend and real-world deployment.diff --git a/docs/FRONTEND_PLAN.md b/docs/FRONTEND_PLAN.mddeleted file mode 100644index 18a28be3..00000000--- a/docs/FRONTEND_PLAN.md+++ /dev/null@@ -1,1334 +0,0 @@-# Frontend Implementation Plan--Complete plan for building the voicebox frontend with modern React, TypeScript, shadcn/ui, and full type safety.-------## Technology Stack--### Core-- **React 18** - UI framework with concurrent features-- **TypeScript (strict mode)** - Full type safety-- **Vite** - Fast build tool and dev server-- **Bun** - Package manager--### UI & Styling-- **shadcn/ui** - Headless component primitives (new-york style)-- **Tailwind CSS v4** - Utility-first styling-- **Radix UI** - Accessible primitives (via shadcn/ui)-- **Lucide React** - Icon system-- **class-variance-authority (cva)** - Component variants-- **tailwind-merge** - Smart class merging--### State Management-- **React Query v5** - Server state (API calls, caching)-- **Zustand** - Client state (UI state, modals, selections)-- **React Hook Form** - Form state and validation-- **Zod** - Runtime schema validation--### Audio-- **WaveSurfer.js** - Audio waveform visualization-- **Web Audio API** - Audio recording/playback-- **MediaRecorder API** - Voice recording--### Type Safety-- **OpenAPI TypeScript Codegen** - Generate API client from FastAPI schema-- **Zod** - Runtime validation matching backend Pydantic models-- **TypeScript strict mode** - Compiler enforcement-------## Architecture Overview--```-app/src/-├── components/-│ ├── ui/ # shadcn/ui primitives (auto-generated)-│ │ ├── button.tsx-│ │ ├── dialog.tsx-│ │ ├── form.tsx-│ │ ├── input.tsx-│ │ ├── select.tsx-│ │ ├── slider.tsx-│ │ ├── table.tsx-│ │ ├── tabs.tsx-│ │ ├── card.tsx-│ │ ├── badge.tsx-│ │ ├── toast.tsx-│ │ └── ...-│ │-│ ├── VoiceProfiles/ # Voice profile management-│ │ ├── ProfileList.tsx-│ │ ├── ProfileCard.tsx-│ │ ├── ProfileForm.tsx-│ │ ├── SampleUpload.tsx-│ │ └── SampleList.tsx-│ │-│ ├── Generation/ # Voice generation-│ │ ├── GenerationForm.tsx-│ │ ├── GenerationPreview.tsx-│ │ └── GenerationSettings.tsx-│ │-│ ├── History/ # Generation history-│ │ ├── HistoryTable.tsx-│ │ ├── HistoryFilter.tsx-│ │ └── HistoryPlayer.tsx-│ │-│ ├── AudioStudio/ # Audio editing (Phase 3)-│ │ ├── Timeline.tsx-│ │ ├── Waveform.tsx-│ │ ├── Controls.tsx-│ │ └── TrackList.tsx-│ │-│ └── ServerSettings/ # Server connection-│ ├── ConnectionForm.tsx-│ ├── ServerStatus.tsx-│ └── LocalServerToggle.tsx-│-├── lib/-│ ├── api/ # Generated OpenAPI client-│ │ ├── core/-│ │ ├── models/-│ │ └── services/-│ │ ├── ProfilesService.ts-│ │ ├── GenerationService.ts-│ │ └── HistoryService.ts-│ │-│ ├── hooks/ # React Query hooks-│ │ ├── useProfiles.ts-│ │ ├── useGeneration.ts-│ │ ├── useHistory.ts-│ │ ├── useTranscription.ts-│ │ └── useServer.ts-│ │-│ ├── schemas/ # Zod schemas (match backend)-│ │ ├── profile.ts-│ │ ├── generation.ts-│ │ └── history.ts-│ │-│ └── utils/-│ ├── cn.ts # Class name utility (shadcn)-│ ├── audio.ts # Audio utilities-│ └── format.ts # Formatting helpers-│-├── stores/ # Zustand stores-│ ├── uiStore.ts # UI state (modals, sidebar)-│ ├── playerStore.ts # Audio player state-│ └── serverStore.ts # Server connection state-│-├── types/ # TypeScript types-│ ├── index.ts-│ ├── api.ts # Augment generated types-│ └── tauri.ts # Tauri-specific types-│-├── App.tsx # Main app component-├── main.tsx # Entry point-└── index.css # Global styles + Tailwind-```-------## Setup Steps--### 1. Install shadcn/ui and Dependencies--```bash-cd app--# Core dependencies (if not installed)-bun add @tanstack/react-query zustand react-hook-form zod @hookform/resolvers wavesurfer.js--# shadcn/ui setup-bunx --bun shadcn-ui@latest init--# Select:-# - Style: new-york-# - Base color: slate (or zinc for darker theme)-# - CSS variables: yes-```--This creates:-- `components.json` - shadcn/ui configuration-- `components/ui/` - UI primitives directory-- Installs: `class-variance-authority`, `clsx`, `tailwind-merge`, `lucide-react`, `tailwindcss-animate`--### 2. Update Vite Config--**app/vite.config.ts:**-```typescript-import path from 'node:path';-import react from '@vitejs/plugin-react';-import { defineConfig } from 'vite';--export default defineConfig({- plugins: [react()],- resolve: {- alias: {- '@': path.resolve(__dirname, './src'),- },- },-});-```--### 3. Update TypeScript Config--**app/tsconfig.json:**-```json-{- "compilerOptions": {- "target": "ES2020",- "lib": ["ES2020", "DOM", "DOM.Iterable"],- "module": "ESNext",- "skipLibCheck": true,-- "moduleResolution": "bundler",- "allowImportingTsExtensions": true,- "resolveJsonModule": true,- "isolatedModules": true,- "noEmit": true,- "jsx": "react-jsx",-- "strict": true,- "noUnusedLocals": true,- "noUnusedParameters": true,- "noFallthroughCasesInSwitch": true,-- "baseUrl": ".",- "paths": {- "@/*": ["./src/*"]- }- },- "include": ["src"],- "references": [{ "path": "./tsconfig.node.json" }]-}-```--### 4. Add Essential shadcn/ui Components--```bash-# Forms and inputs-bunx --bun shadcn-ui@latest add button-bunx --bun shadcn-ui@latest add input-bunx --bun shadcn-ui@latest add form-bunx --bun shadcn-ui@latest add label-bunx --bun shadcn-ui@latest add select-bunx --bun shadcn-ui@latest add textarea-bunx --bun shadcn-ui@latest add slider--# Layout and display-bunx --bun shadcn-ui@latest add card-bunx --bun shadcn-ui@latest add tabs-bunx --bun shadcn-ui@latest add separator-bunx --bun shadcn-ui@latest add badge-bunx --bun shadcn-ui@latest add avatar--# Feedback-bunx --bun shadcn-ui@latest add toast-bunx --bun shadcn-ui@latest add alert-bunx --bun shadcn-ui@latest add progress--# Overlays-bunx --bun shadcn-ui@latest add dialog-bunx --bun shadcn-ui@latest add dropdown-menu-bunx --bun shadcn-ui@latest add popover--# Data display-bunx --bun shadcn-ui@latest add table-bunx --bun shadcn-ui@latest add scroll-area-```--### 5. Setup Providers--**app/src/main.tsx:**-```typescript-import React from 'react';-import ReactDOM from 'react-dom/client';-import { QueryClient, QueryClientProvider } from '@tanstack/react-query';-import { ReactQueryDevtools } from '@tanstack/react-query-devtools';-import App from './App';-import './index.css';--const queryClient = new QueryClient({- defaultOptions: {- queries: {- staleTime: 1000 * 60 * 5, // 5 minutes- gcTime: 1000 * 60 * 10, // 10 minutes (formerly cacheTime)- retry: 1,- refetchOnWindowFocus: false,- },- },-});--ReactDOM.createRoot(document.getElementById('root')!).render(- <React.StrictMode>- <QueryClientProvider client={queryClient}>- <App />- <ReactQueryDevtools initialIsOpen={false} />- </QueryClientProvider>- </React.StrictMode>,-);-```--### 6. Setup Zustand Stores--**app/src/stores/uiStore.ts:**-```typescript-import { create } from 'zustand';--interface UIStore {- // Sidebar- sidebarOpen: boolean;- setSidebarOpen: (open: boolean) => void;-- // Modals- profileDialogOpen: boolean;- setProfileDialogOpen: (open: boolean) => void;-- generationDialogOpen: boolean;- setGenerationDialogOpen: (open: boolean) => void;-- // Theme- theme: 'light' | 'dark';- setTheme: (theme: 'light' | 'dark') => void;-}--export const useUIStore = create<UIStore>((set) => ({- sidebarOpen: true,- setSidebarOpen: (open) => set({ sidebarOpen: open }),-- profileDialogOpen: false,- setProfileDialogOpen: (open) => set({ profileDialogOpen: open }),-- generationDialogOpen: false,- setGenerationDialogOpen: (open) => set({ generationDialogOpen: open }),-- theme: 'light',- setTheme: (theme) => set({ theme }),-}));-```--**app/src/stores/serverStore.ts:**-```typescript-import { create } from 'zustand';-import { persist } from 'zustand/middleware';--interface ServerStore {- serverUrl: string;- setServerUrl: (url: string) => void;-- isConnected: boolean;- setIsConnected: (connected: boolean) => void;-- mode: 'local' | 'remote';- setMode: (mode: 'local' | 'remote') => void;-}--export const useServerStore = create<ServerStore>()(- persist(- (set) => ({- serverUrl: 'http://localhost:8000',- setServerUrl: (url) => set({ serverUrl: url }),-- isConnected: false,- setIsConnected: (connected) => set({ isConnected: connected }),-- mode: 'local',- setMode: (mode) => set({ mode }),- }),- {- name: 'voicebox-server',- },- ),-);-```-------## React Query Hooks--### useProfiles Hook--**app/src/lib/hooks/useProfiles.ts:**-```typescript-import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query';-import { ProfilesService } from '@/lib/api/services/ProfilesService';-import type { VoiceProfileCreate, VoiceProfileResponse } from '@/lib/api/models';--export function useProfiles() {- return useQuery({- queryKey: ['profiles'],- queryFn: () => ProfilesService.listProfiles(),- });-}--export function useProfile(profileId: string) {- return useQuery({- queryKey: ['profiles', profileId],- queryFn: () => ProfilesService.getProfile({ profileId }),- enabled: !!profileId,- });-}--export function useCreateProfile() {- const queryClient = useQueryClient();-- return useMutation({- mutationFn: (data: VoiceProfileCreate) =>- ProfilesService.createProfile({ data }),- onSuccess: () => {- queryClient.invalidateQueries({ queryKey: ['profiles'] });- },- });-}--export function useUpdateProfile() {- const queryClient = useQueryClient();-- return useMutation({- mutationFn: ({ profileId, data }: { profileId: string; data: VoiceProfileCreate }) =>- ProfilesService.updateProfile({ profileId, data }),- onSuccess: (_, variables) => {- queryClient.invalidateQueries({ queryKey: ['profiles'] });- queryClient.invalidateQueries({ queryKey: ['profiles', variables.profileId] });- },- });-}--export function useDeleteProfile() {- const queryClient = useQueryClient();-- return useMutation({- mutationFn: (profileId: string) =>- ProfilesService.deleteProfile({ profileId }),- onSuccess: () => {- queryClient.invalidateQueries({ queryKey: ['profiles'] });- },- });-}--export function useAddSample() {- const queryClient = useQueryClient();-- return useMutation({- mutationFn: ({- profileId,- file,- referenceText- }: {- profileId: string;- file: File;- referenceText: string;- }) =>- ProfilesService.addProfileSample({- profileId,- file,- referenceText,- }),- onSuccess: (_, variables) => {- queryClient.invalidateQueries({- queryKey: ['profiles', variables.profileId, 'samples']- });- },- });-}-```--### useGeneration Hook--**app/src/lib/hooks/useGeneration.ts:**-```typescript-import { useMutation, useQueryClient } from '@tanstack/react-query';-import { GenerationService } from '@/lib/api/services/GenerationService';-import type { GenerationRequest } from '@/lib/api/models';--export function useGeneration() {- const queryClient = useQueryClient();-- return useMutation({- mutationFn: (data: GenerationRequest) =>- GenerationService.generateSpeech({ data }),- onSuccess: () => {- // Invalidate history to show new generation- queryClient.invalidateQueries({ queryKey: ['history'] });- },- });-}-```--### useHistory Hook--**app/src/lib/hooks/useHistory.ts:**-```typescript-import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';-import { HistoryService } from '@/lib/api/services/HistoryService';-import type { HistoryQuery } from '@/lib/api/models';--export function useHistory(query?: HistoryQuery) {- return useQuery({- queryKey: ['history', query],- queryFn: () => HistoryService.listHistory(query),- });-}--export function useGeneration(generationId: string) {- return useQuery({- queryKey: ['history', generationId],- queryFn: () => HistoryService.getGeneration({ generationId }),- enabled: !!generationId,- });-}--export function useDeleteGeneration() {- const queryClient = useQueryClient();-- return useMutation({- mutationFn: (generationId: string) =>- HistoryService.deleteGeneration({ generationId }),- onSuccess: () => {- queryClient.invalidateQueries({ queryKey: ['history'] });- },- });-}-```-------## Component Implementation--### Phase 1: Voice Profiles (Week 1)--#### ProfileList Component--**app/src/components/VoiceProfiles/ProfileList.tsx:**-```typescript-import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';-import { Button } from '@/components/ui/button';-import { Badge } from '@/components/ui/badge';-import { Mic, Plus, Trash2 } from 'lucide-react';-import { useProfiles, useDeleteProfile } from '@/lib/hooks/useProfiles';-import { useUIStore } from '@/stores/uiStore';-import { ProfileForm } from './ProfileForm';--export function ProfileList() {- const { data: profiles, isLoading } = useProfiles();- const deleteProfile = useDeleteProfile();- const setDialogOpen = useUIStore((state) => state.setProfileDialogOpen);-- if (isLoading) {- return <div>Loading profiles...</div>;- }-- return (- <div className="space-y-4">- <div className="flex items-center justify-between">- <h2 className="text-2xl font-bold">Voice Profiles</h2>- <Button onClick={() => setDialogOpen(true)}>- <Plus className="mr-2 h-4 w-4" />- New Profile- </Button>- </div>-- <div className="grid gap-4 md:grid-cols-2 lg:grid-cols-3">- {profiles?.map((profile) => (- <Card key={profile.id}>- <CardHeader>- <CardTitle className="flex items-center justify-between">- <span className="flex items-center gap-2">- <Mic className="h-5 w-5" />- {profile.name}- </span>- <Button- variant="ghost"- size="icon"- onClick={() => deleteProfile.mutate(profile.id)}- >- <Trash2 className="h-4 w-4 text-destructive" />- </Button>- </CardTitle>- </CardHeader>- <CardContent>- <p className="text-sm text-muted-foreground">- {profile.description}- </p>- <div className="mt-2 flex gap-2">- <Badge variant="outline">{profile.language}</Badge>- <Badge variant="secondary">- {profile.sample_count} samples- </Badge>- </div>- </CardContent>- </Card>- ))}- </div>-- <ProfileForm />- </div>- );-}-```--#### ProfileForm Component--**app/src/components/VoiceProfiles/ProfileForm.tsx:**-```typescript-import { useForm } from 'react-hook-form';-import { zodResolver } from '@hookform/resolvers/zod';-import * as z from 'zod';-import {- Dialog,- DialogContent,- DialogDescription,- DialogHeader,- DialogTitle,-} from '@/components/ui/dialog';-import {- Form,- FormControl,- FormField,- FormItem,- FormLabel,- FormMessage,-} from '@/components/ui/form';-import { Input } from '@/components/ui/input';-import { Textarea } from '@/components/ui/textarea';-import {- Select,- SelectContent,- SelectItem,- SelectTrigger,- SelectValue,-} from '@/components/ui/select';-import { Button } from '@/components/ui/button';-import { useCreateProfile } from '@/lib/hooks/useProfiles';-import { useUIStore } from '@/stores/uiStore';--const profileSchema = z.object({- name: z.string().min(1, 'Name is required').max(100),- description: z.string().optional(),- language: z.enum(['en', 'zh']),- tags: z.string().optional(),-});--type ProfileFormValues = z.infer<typeof profileSchema>;--export function ProfileForm() {- const open = useUIStore((state) => state.profileDialogOpen);- const setOpen = useUIStore((state) => state.setProfileDialogOpen);- const createProfile = useCreateProfile();-- const form = useForm<ProfileFormValues>({- resolver: zodResolver(profileSchema),- defaultValues: {- name: '',- description: '',- language: 'en',- tags: '',- },- });-- async function onSubmit(data: ProfileFormValues) {- const tags = data.tags ? data.tags.split(',').map((t) => t.trim()) : [];-- await createProfile.mutateAsync({- ...data,- tags,- });-- form.reset();- setOpen(false);- }-- return (- <Dialog open={open} onOpenChange={setOpen}>- <DialogContent>- <DialogHeader>- <DialogTitle>Create Voice Profile</DialogTitle>- <DialogDescription>- Add a new voice profile with samples- </DialogDescription>- </DialogHeader>-- <Form {...form}>- <form onSubmit={form.handleSubmit(onSubmit)} className="space-y-4">- <FormField- control={form.control}- name="name"- render={({ field }) => (- <FormItem>- <FormLabel>Name</FormLabel>- <FormControl>- <Input placeholder="My Voice" {...field} />- </FormControl>- <FormMessage />- </FormItem>- )}- />-- <FormField- control={form.control}- name="description"- render={({ field }) => (- <FormItem>- <FormLabel>Description</FormLabel>- <FormControl>- <Textarea- placeholder="Describe this voice..."- {...field}- />- </FormControl>- <FormMessage />- </FormItem>- )}- />-- <FormField- control={form.control}- name="language"- render={({ field }) => (- <FormItem>- <FormLabel>Language</FormLabel>- <Select- onValueChange={field.onChange}- defaultValue={field.value}- >- <FormControl>- <SelectTrigger>- <SelectValue />- </SelectTrigger>- </FormControl>- <SelectContent>- <SelectItem value="en">English</SelectItem>- <SelectItem value="zh">Chinese</SelectItem>- </SelectContent>- </Select>- <FormMessage />- </FormItem>- )}- />-- <FormField- control={form.control}- name="tags"- render={({ field }) => (- <FormItem>- <FormLabel>Tags</FormLabel>- <FormControl>- <Input- placeholder="tag1, tag2, tag3"- {...field}- />- </FormControl>- <FormMessage />- </FormItem>- )}- />-- <div className="flex gap-2">- <Button- type="button"- variant="outline"- onClick={() => setOpen(false)}- >- Cancel- </Button>- <Button- type="submit"- disabled={createProfile.isPending}- >- {createProfile.isPending ? 'Creating...' : 'Create Profile'}- </Button>- </div>- </form>- </Form>- </DialogContent>- </Dialog>- );-}-```--### Phase 2: Generation (Week 2)--#### GenerationForm Component--**app/src/components/Generation/GenerationForm.tsx:**-```typescript-import { useForm } from 'react-hook-form';-import { zodResolver } from '@hookform/resolvers/zod';-import * as z from 'zod';-import {- Form,- FormControl,- FormDescription,- FormField,- FormItem,- FormLabel,- FormMessage,-} from '@/components/ui/form';-import { Input } from '@/components/ui/input';-import { Textarea } from '@/components/ui/textarea';-import {- Select,- SelectContent,- SelectItem,- SelectTrigger,- SelectValue,-} from '@/components/ui/select';-import { Button } from '@/components/ui/button';-import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card';-import { Loader2 } from 'lucide-react';-import { useGeneration } from '@/lib/hooks/useGeneration';-import { useProfiles } from '@/lib/hooks/useProfiles';-import { useToast } from '@/hooks/use-toast';--const generationSchema = z.object({- profileId: z.string().min(1, 'Please select a voice profile'),- text: z.string().min(1, 'Text is required').max(5000),- language: z.enum(['en', 'zh']),- seed: z.number().int().optional(),-});--type GenerationFormValues = z.infer<typeof generationSchema>;--export function GenerationForm() {- const { data: profiles } = useProfiles();- const generation = useGeneration();- const { toast } = useToast();-- const form = useForm<GenerationFormValues>({- resolver: zodResolver(generationSchema),- defaultValues: {- profileId: '',- text: '',- language: 'en',- seed: undefined,- },- });-- async function onSubmit(data: GenerationFormValues) {- try {- const result = await generation.mutateAsync({- profile_id: data.profileId,- text: data.text,- language: data.language,- seed: data.seed,- });-- toast({- title: 'Generation complete!',- description: `Audio generated (${result.duration.toFixed(2)}s)`,- });-- form.reset();- } catch (error) {- toast({- title: 'Generation failed',- description: error.message,- variant: 'destructive',- });- }- }-- return (- <Card>- <CardHeader>- <CardTitle>Generate Speech</CardTitle>- </CardHeader>- <CardContent>- <Form {...form}>- <form onSubmit={form.handleSubmit(onSubmit)} className="space-y-4">- <FormField- control={form.control}- name="profileId"- render={({ field }) => (- <FormItem>- <FormLabel>Voice Profile</FormLabel>- <Select- onValueChange={field.onChange}- defaultValue={field.value}- >- <FormControl>- <SelectTrigger>- <SelectValue placeholder="Select a voice" />- </SelectTrigger>- </FormControl>- <SelectContent>- {profiles?.map((profile) => (- <SelectItem key={profile.id} value={profile.id}>- {profile.name}- </SelectItem>- ))}- </SelectContent>- </Select>- <FormMessage />- </FormItem>- )}- />-- <FormField- control={form.control}- name="text"- render={({ field }) => (- <FormItem>- <FormLabel>Text to Speak</FormLabel>- <FormControl>- <Textarea- placeholder="Enter the text you want to generate..."- className="min-h-[200px]"- {...field}- />- </FormControl>- <FormDescription>- Max 5000 characters- </FormDescription>- <FormMessage />- </FormItem>- )}- />-- <div className="grid gap-4 md:grid-cols-2">- <FormField- control={form.control}- name="language"- render={({ field }) => (- <FormItem>- <FormLabel>Language</FormLabel>- <Select- onValueChange={field.onChange}- defaultValue={field.value}- >- <FormControl>- <SelectTrigger>- <SelectValue />- </SelectTrigger>- </FormControl>- <SelectContent>- <SelectItem value="en">English</SelectItem>- <SelectItem value="zh">Chinese</SelectItem>- </SelectContent>- </Select>- <FormMessage />- </FormItem>- )}- />-- <FormField- control={form.control}- name="seed"- render={({ field }) => (- <FormItem>- <FormLabel>Seed (optional)</FormLabel>- <FormControl>- <Input- type="number"- placeholder="Random"- {...field}- onChange={(e) =>- field.onChange(- e.target.value ? parseInt(e.target.value) : undefined- )- }- />- </FormControl>- <FormDescription>- For reproducible results- </FormDescription>- <FormMessage />- </FormItem>- )}- />- </div>-- <Button- type="submit"- className="w-full"- disabled={generation.isPending}- >- {generation.isPending ? (- <>- <Loader2 className="mr-2 h-4 w-4 animate-spin" />- Generating...- </>- ) : (- 'Generate Speech'- )}- </Button>- </form>- </Form>- </CardContent>- </Card>- );-}-```--### Phase 3: History (Week 2)--#### HistoryTable Component--**app/src/components/History/HistoryTable.tsx:**-```typescript-import { useState } from 'react';-import {- Table,- TableBody,- TableCell,- TableHead,- TableHeader,- TableRow,-} from '@/components/ui/table';-import { Button } from '@/components/ui/button';-import { Badge } from '@/components/ui/badge';-import { Play, Download, Trash2 } from 'lucide-react';-import { useHistory, useDeleteGeneration } from '@/lib/hooks/useHistory';-import { formatDistance } from 'date-fns';--export function HistoryTable() {- const [page, setPage] = useState(0);- const limit = 20;-- const { data: history, isLoading } = useHistory({- limit,- offset: page * limit,- });-- const deleteGeneration = useDeleteGeneration();-- if (isLoading) {- return <div>Loading history...</div>;- }-- return (- <div className="space-y-4">- <h2 className="text-2xl font-bold">Generation History</h2>-- <Table>- <TableHeader>- <TableRow>- <TableHead>Text</TableHead>- <TableHead>Profile</TableHead>- <TableHead>Language</TableHead>- <TableHead>Duration</TableHead>- <TableHead>Created</TableHead>- <TableHead className="text-right">Actions</TableHead>- </TableRow>- </TableHeader>- <TableBody>- {history?.map((gen) => (- <TableRow key={gen.id}>- <TableCell className="max-w-[300px] truncate">- {gen.text}- </TableCell>- <TableCell>{gen.profile_name}</TableCell>- <TableCell>- <Badge variant="outline">{gen.language}</Badge>- </TableCell>- <TableCell>{gen.duration.toFixed(2)}s</TableCell>- <TableCell>- {formatDistance(new Date(gen.created_at), new Date(), {- addSuffix: true,- })}- </TableCell>- <TableCell className="text-right">- <div className="flex justify-end gap-2">- <Button variant="ghost" size="icon">- <Play className="h-4 w-4" />- </Button>- <Button variant="ghost" size="icon">- <Download className="h-4 w-4" />- </Button>- <Button- variant="ghost"- size="icon"- onClick={() => deleteGeneration.mutate(gen.id)}- >- <Trash2 className="h-4 w-4 text-destructive" />- </Button>- </div>- </TableCell>- </TableRow>- ))}- </TableBody>- </Table>-- <div className="flex justify-between">- <Button- variant="outline"- onClick={() => setPage((p) => Math.max(0, p - 1))}- disabled={page === 0}- >- Previous- </Button>- <Button- variant="outline"- onClick={() => setPage((p) => p + 1)}- disabled={!history || history.length < limit}- >- Next- </Button>- </div>- </div>- );-}-```-------## Implementation Phases--### Phase 1: Foundation (Week 1)-**Goal:** Setup, OpenAPI client, basic UI--1. **Day 1-2: Setup**- - ✅ Install shadcn/ui and dependencies- - ✅ Generate OpenAPI client- - ✅ Setup providers (React Query, Toaster)- - ✅ Create Zustand stores- - ✅ Setup routing (if needed)--2. **Day 3-4: Voice Profiles**- - ProfileList component- - ProfileCard component- - ProfileForm (create/edit)- - Profile deletion- - Sample upload UI--3. **Day 5-7: Testing & Polish**- - Test all CRUD operations- - Error handling- - Loading states- - Empty states--### Phase 2: Generation & History (Week 2)-**Goal:** Core voice generation functionality--1. **Day 1-3: Generation**- - GenerationForm component- - Real-time generation- - Progress indication- - Audio preview player- - Download functionality--2. **Day 4-7: History**- - HistoryTable component- - Filtering and search- - Pagination- - Audio playback- - Export functionality--### Phase 3: Audio Studio (Week 3-4)-**Goal:** Timeline editing and advanced features--1. **Week 3: Basic Studio**- - Waveform visualization (WaveSurfer.js)- - Timeline component- - Playback controls- - Multiple tracks--2. **Week 4: Advanced Features**- - Word-level timestamps- - Editing (trim, split)- - Audio effects- - Project save/load--### Phase 4: Polish & Features (Week 5+)-**Goal:** Production polish--1. **Server Settings**- - Connection management- - Local server toggle- - Health monitoring--2. **UX Improvements**- - Keyboard shortcuts- - Drag and drop- - Batch operations- - Export options-------## Type Safety Best Practices--### 1. OpenAPI Generated Types--```typescript-// Use generated types from @/lib/api/models-import type {- VoiceProfileResponse,- GenerationRequest,- HistoryQuery,-} from '@/lib/api/models';--// Never use `any`-function handleProfile(profile: VoiceProfileResponse) {- // Fully typed-}-```--### 2. Zod Schemas Match Backend--```typescript-// app/src/lib/schemas/profile.ts-import { z } from 'zod';--// Match backend Pydantic model-export const profileCreateSchema = z.object({- name: z.string().min(1).max(100),- description: z.string().optional(),- language: z.enum(['en', 'zh']),- tags: z.array(z.string()).optional(),-});--export type ProfileCreate = z.infer<typeof profileCreateSchema>;-```--### 3. React Hook Form with Zod--```typescript-const form = useForm<ProfileCreate>({- resolver: zodResolver(profileCreateSchema),-});--// Fully typed, runtime validated-```--### 4. Type-Safe Event Handlers--```typescript-// Good-function handleSubmit(data: ProfileCreate) {- // data is typed-}--// Bad-function handleSubmit(data: any) {- // No type safety-}-```-------## Performance Optimizations--### 1. React Query Configuration--```typescript-const queryClient = new QueryClient({- defaultOptions: {- queries: {- staleTime: 1000 * 60 * 5, // Don't refetch for 5 minutes- gcTime: 1000 * 60 * 10, // Keep unused data for 10 minutes- retry: 1, // Only retry once- },- },-});-```--### 2. Component Code Splitting--```typescript-import { lazy } from 'react';--const AudioStudio = lazy(() => import('@/components/AudioStudio'));--// Lazy load heavy components-```--### 3. Virtualized Lists--For large history tables:-```bash-bunx --bun shadcn-ui@latest add table-```--Use with TanStack Virtual for performance.--### 4. Debounced Search--```typescript-import { useDebouncedValue } from '@/hooks/useDebouncedValue';--const [search, setSearch] = useState('');-const debouncedSearch = useDebouncedValue(search, 300);--// Use debouncedSearch in query-useHistory({ search: debouncedSearch });-```-------## Testing Strategy--### Unit Tests (Vitest)--```bash-bun add -D vitest @testing-library/react @testing-library/jest-dom-```--**app/src/components/VoiceProfiles/ProfileCard.test.tsx:**-```typescript-import { describe, it, expect } from 'vitest';-import { render, screen } from '@testing-library/react';-import { ProfileCard } from './ProfileCard';--describe('ProfileCard', () => {- it('renders profile name', () => {- render(<ProfileCard profile={{ name: 'Test Voice' }} />);- expect(screen.getByText('Test Voice')).toBeInTheDocument();- });-});-```--### Component Tests (Storybook - Optional)--```bash-bunx storybook@latest init-```-------## Accessibility--shadcn/ui components are built on Radix UI with full accessibility:--- ✅ Keyboard navigation-- ✅ Screen reader support-- ✅ ARIA attributes-- ✅ Focus management--**Additional improvements:**-1. Add `aria-label` to icon buttons-2. Use semantic HTML (`<main>`, `<nav>`, `<section>`)-3. Ensure color contrast meets WCAG AA-4. Add loading announcements for screen readers-------## Summary--**Tech Stack:**-- React 18 + TypeScript (strict)-- shadcn/ui + Radix UI-- React Query + Zustand-- React Hook Form + Zod-- Tailwind CSS v4-- WaveSurfer.js--**Type Safety:**-- ✅ OpenAPI generated client-- ✅ Zod runtime validation-- ✅ TypeScript strict mode-- ✅ No `any` types--**Implementation:**-- Phase 1: Voice Profiles (Week 1)-- Phase 2: Generation & History (Week 2)-- Phase 3: Audio Studio (Week 3-4)-- Phase 4: Polish (Week 5+)--**Next Steps:**-1. Run `bunx --bun shadcn-ui@latest init`-2. Generate OpenAPI client (`bun run generate:api`)-3. Add essential shadcn/ui components-4. Build ProfileList component-5. Test with backend--Ready to build a production-quality, type-safe voice cloning app! 🚀diff --git a/docs/ICON_WORKFLOW.md b/docs/ICON_WORKFLOW.mddeleted file mode 100644index d5b4f3e6..00000000--- a/docs/ICON_WORKFLOW.md+++ /dev/null@@ -1,150 +0,0 @@-# Icon Update Workflow for voicebox--## Prerequisites-- Xcode Command Line Tools installed (`xcode-select --install`)-- Your icon designed in Icon Composer and saved as `voicebox.icon` in the project root--## Directory Structure-```-voicebox/-├── voicebox.icon/ # macOS 26 Liquid Glass icon bundle-│ ├── icon.json-│ └── Assets/-│ └── Voicebox.png # Your source image-├── tauri/-│ ├── assets/-│ │ └── voicebox_exports/ # Your 1024x1024 exports-│ └── src-tauri/-│ ├── gen/ # Auto-generated at build time-│ │ ├── Assets.car # Liquid Glass assets-│ │ ├── voicebox.icns # Generated fallback icon-│ │ └── partial.plist-│ └── icons/ # Tauri fallback icons (all platforms)-│ ├── icon.icns-│ ├── icon.ico-│ ├── 32x32.png-│ ├── 128x128.png-│ └── ...-```--## Step 1: Update the Liquid Glass Icon--Edit `voicebox.icon/` in Icon Composer (or manually update `icon.json` and `Assets/`).--The build script (`build.rs`) automatically compiles this during `cargo build`.--## Step 2: Regenerate Fallback Icons--After updating the `.icon` bundle, regenerate the fallback icons:--```bash-cd tauri/src-tauri--# Trigger rebuild to generate new icns-cargo build--# Copy the generated icns to icons folder-cp gen/voicebox.icns icons/icon.icns--# Generate PNGs from the icns-sips -s format png -z 32 32 gen/voicebox.icns --out icons/32x32.png-sips -s format png -z 64 64 gen/voicebox.icns --out icons/64x64.png-sips -s format png -z 128 128 gen/voicebox.icns --out icons/128x128.png-sips -s format png -z 256 256 gen/voicebox.icns --out icons/[email protected]-sips -s format png -z 512 512 gen/voicebox.icns --out icons/icon.png--# Windows Square logos-for size in 30 44 71 89 107 142 150 284 310; do- sips -s format png -z $size $size gen/voicebox.icns --out "icons/Square${size}x${size}Logo.png"-done-```--## Step 3: Rebuild the App--```bash-cd tauri && bun run tauri build-```--## Step 4: Clear Icon Cache (if icons don't update)--```bash-sudo rm -rf /Library/Caches/com.apple.iconservices.store-sudo killall Finder-sudo killall Dock-```-------## How It Works--| File | Purpose |-|------|---------|-| `voicebox.icon/` | Source for macOS 26 Liquid Glass icon |-| `build.rs` | Compiles `.icon` → `Assets.car` + `voicebox.icns` at build time |-| `gen/Assets.car` | Liquid Glass assets (macOS 26+) |-| `gen/voicebox.icns` | Auto-generated fallback icon |-| `icons/icon.icns` | Tauri's fallback for older macOS |-| `icons/*.png` | Tauri's fallback for other platforms |-| `Info.plist` | Points to `voicebox` as icon name |-| `tauri.conf.json` | Bundles `gen/*` to Resources root |--## Key Config Files--### `build.rs` — Compiles the icon--```rust-xcrun actool --app-icon voicebox ... voicebox.icon-```--### `tauri.conf.json` — Bundles generated assets to Resources root--```json-"resources": {- "gen/Assets.car": "./",- "gen/voicebox.icns": "./",- "gen/partial.plist": "./"-}-```--### `Info.plist` — Tells macOS which icon to use--```xml-<key>CFBundleIconFile</key>-<string>voicebox</string>-<key>CFBundleIconName</key>-<string>voicebox</string>-```--## Quick Reference Script--Save this as `scripts/update-icons.sh`:--```bash-#!/bin/bash-set -e--cd "$(dirname "$0")/../tauri/src-tauri"--echo "Building to generate new icons..."-cargo build--echo "Copying icns to icons folder..."-cp gen/voicebox.icns icons/icon.icns--echo "Generating PNG icons..."-sips -s format png -z 32 32 gen/voicebox.icns --out icons/32x32.png-sips -s format png -z 64 64 gen/voicebox.icns --out icons/64x64.png-sips -s format png -z 128 128 gen/voicebox.icns --out icons/128x128.png-sips -s format png -z 256 256 gen/voicebox.icns --out icons/[email protected]-sips -s format png -z 512 512 gen/voicebox.icns --out icons/icon.png--echo "Generating Windows Square logos..."-for size in 30 44 71 89 107 142 150 284 310; do- sips -s format png -z $size $size gen/voicebox.icns --out "icons/Square${size}x${size}Logo.png"-done--echo "Done! Icons updated."-echo "Run 'bun run tauri build' to rebuild the app."-```--Make it executable: `chmod +x scripts/update-icons.sh`diff --git a/docs/IMPLEMENTATION_STATUS.md b/docs/IMPLEMENTATION_STATUS.mddeleted file mode 100644index 87b900ae..00000000--- a/docs/IMPLEMENTATION_STATUS.md+++ /dev/null@@ -1,287 +0,0 @@-# Backend Implementation Status--## ✅ COMPLETE - Ready for Frontend Integration--### What's Been Built--The voicebox backend is **fully implemented** and production-ready with the following features:--#### Core Modules (100% Complete)--1. **TTS Module** (`tts.py`) - 200 lines- - Lazy model loading with device detection- - Voice prompt creation and caching- - Multi-reference combination- - Async generation with seed control- - Model size switching (1.7B/0.6B)- - Memory management--2. **Profiles Module** (`profiles.py`) - 250 lines- - Full CRUD operations- - Multi-sample support- - Audio validation- - Automatic sample combination- - File storage management--3. **History Module** (`history.py`) - 150 lines- - Generation tracking- - Search and filtering- - Pagination- - Statistics- - File cleanup--4. **Transcribe Module** (`transcribe.py`) - 150 lines- - Whisper transcription- - Language hints- - Model size selection- - VRAM management--5. **Database Module** (`database.py`) - 90 lines- - SQLite with SQLAlchemy- - Clean schema design- - Foreign keys- - UUID primary keys--6. **Utils Module** - 300 lines total- - Audio processing (normalization, validation)- - Voice prompt caching (memory + disk)- - Input validation--7. **API Module** (`main.py`) - 300 lines- - 20+ REST endpoints- - File upload handling- - File serving- - Health checks- - Model management--### API Endpoints--#### Implemented ✅-- `GET /` - Root-- `GET /health` - Health check-- `POST /profiles` - Create profile-- `GET /profiles` - List profiles-- `GET /profiles/{id}` - Get profile-- `PUT /profiles/{id}` - Update profile-- `DELETE /profiles/{id}` - Delete profile-- `POST /profiles/{id}/samples` - Add sample-- `GET /profiles/{id}/samples` - List samples-- `DELETE /profiles/samples/{id}` - Delete sample-- `POST /generate` - Generate speech-- `GET /history` - List history-- `GET /history/{id}` - Get generation-- `DELETE /history/{id}` - Delete generation-- `GET /history/stats` - Statistics-- `GET /audio/{id}` - Download audio-- `POST /transcribe` - Transcribe audio-- `POST /models/load` - Load model-- `POST /models/unload` - Unload model--#### Total: 20 endpoints, all tested and working--### Testing--```bash-# 1. Start server-cd backend-python -m venv venv-source venv/bin/activate-pip install -r requirements.txt-python main.py--# 2. Run example-python example_usage.py--# 3. Test with curl-curl http://localhost:8000/health-```--### Documentation--- ✅ `README.md` - Complete API documentation-- ✅ `IMPLEMENTATION_STATUS.md` - This file-- ✅ `example_usage.py` - Example client code-- ✅ `../docs/BACKEND_IMPLEMENTATION.md` - Implementation details-- ✅ `../docs/COMPETITIVE_ANALYSIS.md` - Comparison with references--### Code Quality--- **Total lines:** ~1,500 (clean, maintainable)-- **Largest file:** 300 lines (main.py)-- **Type safety:** 100% (Pydantic + type hints)-- **Async/await:** 100%-- **Modularity:** Excellent (12 files)-- **Error handling:** Comprehensive-- **Documentation:** Complete--### Next Steps for Integration--1. **Frontend can now:**- - Create voice profiles- - Upload audio samples- - Generate speech- - View history- - Download audio files- - Transcribe audio--2. **Frontend needs to:**- - Call REST API endpoints- - Handle file uploads- - Display UI for profiles/history- - Play audio files--3. **Backend ready for:**- - Tauri integration- - Web deployment- - Docker containerization- - Production deployment--### Future Enhancements (Not Blocking)--#### Phase 2 (Next)-- WebSocket streaming-- Batch generation-- Audio effects-- Voice design-- Unit tests--#### Phase 3 (Later)-- Audio studio-- Word-level timestamps-- Projects-- Export options--#### Phase 4 (Production)-- Authentication-- Rate limiting-- Docker-- CI/CD--### Dependencies--All dependencies in `requirements.txt`:-```-fastapi>=0.109.0-uvicorn[standard]>=0.27.0-pydantic>=2.5.0-sqlalchemy>=2.0.0-torch>=2.1.0-transformers>=4.36.0-librosa>=0.10.0-soundfile>=0.12.0-python-multipart>=0.0.6-```--### Performance--- **First generation:** 6-10 seconds (creates prompt + generates)-- **Cached generation:** 1-2 seconds (uses cached prompt)-- **Model loading:** 3-5 seconds (one-time)-- **Transcription:** 2-5 seconds (depends on audio length)--### Architecture Benefits--1. **Modular** - Easy to extend-2. **Type-safe** - Catch errors early-3. **Async** - Non-blocking operations-4. **Cached** - Fast repeated generations-5. **Persistent** - Database-backed-6. **Clean** - Maintainable code-7. **Documented** - Complete API docs--### Comparison to References--**voicebox is:**-- ✅ More maintainable than Voice-Clone-Studio (no 2815-line files)-- ✅ More feature-rich than Qwen3-TTS_server (20 vs 3 endpoints)-- ✅ Better caching than mimic (voice prompts cached)-- ✅ Better typed than all references (100% Pydantic)-- ✅ Better organized than qwen3-tts-enhanced (12 files vs 1)--### Status: READY FOR FRONTEND ✅--The backend is **complete and production-ready** for:-- ✅ Tauri desktop app integration-- ✅ Web app deployment-- ✅ API client generation (OpenAPI)-- ✅ Real-world usage--**No blockers remaining.** Frontend can begin integration immediately.-------## Quick Start for Frontend Developers--### 1. Start Backend-```bash-cd backend-python main.py-```--### 2. Test Connection-```bash-curl http://localhost:8000/health-```--### 3. Create Profile-```bash-curl -X POST http://localhost:8000/profiles \- -H "Content-Type: application/json" \- -d '{"name": "Test Voice", "language": "en"}'-```--### 4. Generate OpenAPI Client-```bash-# OpenAPI spec available at:-http://localhost:8000/openapi.json--# Use with openapi-typescript-codegen-npx openapi-typescript-codegen \- --input http://localhost:8000/openapi.json \- --output ./src/lib/api \- --client fetch-```--### 5. Build Your UI-```typescript-import { ProfilesService, GenerateService } from '@/lib/api';--// Create profile-const profile = await ProfilesService.createProfile({- name: 'My Voice',- language: 'en',-});--// Generate speech-const generation = await GenerateService.generateSpeech({- profile_id: profile.id,- text: 'Hello world',- language: 'en',-});--// Download audio-const audioUrl = `/audio/${generation.id}`;-```-------## Summary--**Backend Status:** ✅ COMPLETE--**Lines of Code:** ~1,500 (clean, maintainable)--**Test Coverage:** Manual testing complete, unit tests TODO--**Documentation:** 100% complete--**Ready for:** Frontend integration, deployment, production--**Next Steps:** Build Tauri frontend, integrate API-------**Questions?** See:-- `README.md` for API documentation-- `example_usage.py` for usage examples-- `../docs/BACKEND_IMPLEMENTATION.md` for implementation detailsdiff --git a/docs/TAURI_PLAN.md b/docs/TAURI_PLAN.mddeleted file mode 100644index 5fe12b6f..00000000--- a/docs/TAURI_PLAN.md+++ /dev/null@@ -1,754 +0,0 @@-# Tauri App Plan--Plan for building voicebox as a Tauri 2.0 desktop app with shared frontend code for web deployment.-------## Project Structure--```-voicebox/-├── app/ # Shared React frontend (used by both web & desktop)-│ ├── src/-│ │ ├── components/-│ │ │ ├── VoiceProfiles/-│ │ │ ├── Generation/-│ │ │ ├── AudioStudio/-│ │ │ ├── History/-│ │ │ └── ServerSettings/-│ │ ├── lib/-│ │ │ ├── api/ # Generated OpenAPI client-│ │ │ ├── hooks/ # React Query hooks-│ │ │ └── utils/-│ │ ├── types/-│ │ ├── App.tsx-│ │ └── main.tsx-│ ├── package.json-│ ├── tsconfig.json-│ ├── vite.config.ts-│ └── tailwind.config.ts-│-├── tauri/ # Tauri desktop app-│ ├── src/ # Thin wrapper, imports from ../app-│ │ └── main.tsx # Entry point that renders App from ../app-│ ├── src-tauri/ # Rust backend-│ │ ├── src/-│ │ │ └── main.rs-│ │ ├── icons/-│ │ ├── binaries/ # Bundled Python server-│ │ │ ├── voicebox-server-x86_64-apple-darwin-│ │ │ ├── voicebox-server-aarch64-apple-darwin-│ │ │ ├── voicebox-server-x86_64-unknown-linux-gnu-│ │ │ └── voicebox-server-x86_64-pc-windows-msvc.exe-│ │ ├── capabilities/-│ │ │ └── default.json-│ │ ├── Cargo.toml-│ │ ├── Cargo.lock-│ │ ├── tauri.conf.json-│ │ └── build.rs-│ ├── package.json-│ └── vite.config.ts # Points to ../app-│-├── web/ # Web deployment-│ ├── src/-│ │ └── main.tsx # Entry point that renders App from ../app-│ ├── package.json-│ └── vite.config.ts # Points to ../app-│-├── backend/ # Python FastAPI server-│ ├── main.py-│ ├── models.py-│ ├── tts.py-│ ├── transcribe.py-│ ├── profiles.py-│ ├── history.py-│ ├── studio.py-│ ├── database.py-│ ├── utils/-│ ├── requirements.txt-│ └── build_binary.py # PyInstaller build script-│-├── scripts/-│ ├── build-server.sh # Build Python server for all platforms-│ └── generate-api.sh # Generate OpenAPI client-│-├── package.json # Root workspace config (Bun workspaces)-└── bun.lockb # Bun lockfile-```-------## Technology Stack--### Desktop App (Tauri 2.0)-- **Tauri 2.9.5+** - Latest stable version-- **Rust** - Tauri backend-- **React + TypeScript** - Frontend (shared with web)-- **Vite** - Build tool-- **Bun** - Fast package manager and JavaScript runtime--### Shared Frontend-- **React 18+** - UI framework-- **TypeScript** - Type safety-- **Vite** - Build tool and dev server-- **Bun** - Package manager (faster than npm/yarn/pnpm)-- **React Query** - Server state management-- **Zustand** - Client state management-- **Tailwind CSS** - Styling-- **WaveSurfer.js** - Audio visualization--### Backend Bundling-- **PyInstaller** - Bundle Python server as standalone binary-- **FastAPI** - Python web framework-- **Tauri Sidecar** - Execute bundled Python server-------## Bundling Python Server with Tauri--### 1. Build Python Server as Standalone Binary--**Using PyInstaller:**-```python-# backend/build_binary.py-import PyInstaller.__main__-import sys-import os--def build_server():- PyInstaller.__main__.run([- 'main.py',- '--onefile',- '--name', 'voicebox-server',- '--add-data', 'data:data', # Include data files- '--hidden-import', 'torch',- '--hidden-import', 'transformers',- '--hidden-import', 'fastapi',- '--collect-all', 'qwen-tts',- '--noconfirm',- ])--if __name__ == '__main__':- build_server()-```--**Build script for all platforms:**-```bash-#!/bin/bash-# scripts/build-server.sh--# Determine platform-PLATFORM=$(rustc --print host-tuple)--# Build Python binary-cd backend-python build_binary.py--# Rename with platform triple-cd dist-mv voicebox-server ../src-tauri/binaries/voicebox-server-${PLATFORM}--echo "Built voicebox-server-${PLATFORM}"-```--**Platform-specific binaries needed:**-- macOS Intel: `voicebox-server-x86_64-apple-darwin`-- macOS ARM: `voicebox-server-aarch64-apple-darwin`-- Linux: `voicebox-server-x86_64-unknown-linux-gnu`-- Windows: `voicebox-server-x86_64-pc-windows-msvc.exe`--### 2. Configure Tauri to Bundle Binary--**tauri/src-tauri/tauri.conf.json:**-```json-{- "bundle": {- "identifier": "sh.voicebox.app",- "externalBin": [- "binaries/voicebox-server"- ],- "resources": [- "binaries/*"- ]- },- "build": {- "beforeBuildCommand": "bun run build",- "devPath": "http://localhost:5173",- "distDir": "../dist"- }-}-```--**Capabilities (src-tauri/capabilities/default.json):**-```json-{- "identifier": "default",- "description": "Default permissions",- "permissions": [- "core:default",- "shell:allow-execute",- "shell:allow-spawn",- "fs:default"- ]-}-```--### 3. Launch Python Server from Tauri--**tauri/src-tauri/src/main.rs:**-```rust-use tauri::{command, Manager};-use tauri_plugin_shell::ShellExt;-use std::sync::Mutex;--struct ServerState {- child: Mutex<Option<tauri_plugin_shell::process::CommandChild>>,-}--#[command]-async fn start_server(app: tauri::AppHandle, state: tauri::State<'_, ServerState>) -> Result<String, String> {- let sidecar = app.shell()- .sidecar("voicebox-server")- .map_err(|e| format!("Failed to get sidecar: {}", e))?;-- let (mut rx, child) = sidecar- .spawn()- .map_err(|e| format!("Failed to spawn: {}", e))?;-- // Store child process- *state.child.lock().unwrap() = Some(child);-- // Wait for server to be ready (listen for startup log)- tokio::spawn(async move {- while let Some(event) = rx.recv().await {- if let tauri_plugin_shell::process::CommandEvent::Stdout(line) = event {- if String::from_utf8_lossy(&line).contains("Uvicorn running") {- break;- }- }- }- });-- Ok("Server started on http://localhost:8000".to_string())-}--#[command]-async fn stop_server(state: tauri::State<'_, ServerState>) -> Result<(), String> {- if let Some(child) = state.child.lock().unwrap().take() {- child.kill().map_err(|e| format!("Failed to kill: {}", e))?;- }- Ok(())-}--#[cfg_attr(mobile, tauri::mobile_entry_point)]-pub fn run() {- tauri::Builder::default()- .plugin(tauri_plugin_shell::init())- .manage(ServerState {- child: Mutex::new(None),- })- .invoke_handler(tauri::generate_handler![start_server, stop_server])- .run(tauri::generate_context!())- .expect("error while running tauri application");-}-```--### 4. Call from Frontend--**app/src/lib/hooks/useServer.ts:**-```typescript-import { invoke } from '@tauri-apps/api/core';-import { useState } from 'react';--export function useServer() {- const [serverUrl, setServerUrl] = useState<string>('http://localhost:8000');- const [isRunning, setIsRunning] = useState(false);-- const startServer = async () => {- try {- const url = await invoke<string>('start_server');- setServerUrl(url);- setIsRunning(true);- return url;- } catch (error) {- console.error('Failed to start server:', error);- throw error;- }- };-- const stopServer = async () => {- try {- await invoke('stop_server');- setIsRunning(false);- } catch (error) {- console.error('Failed to stop server:', error);- throw error;- }- };-- return { serverUrl, isRunning, startServer, stopServer };-}-```-------## Shared Frontend Approach--### App Package Structure--The `app/` directory contains all React code that's shared between desktop and web.--**app/vite.config.ts:**-```typescript-import { defineConfig } from 'vite';-import react from '@vitejs/plugin-react';-import path from 'path';--export default defineConfig({- plugins: [react()],- resolve: {- alias: {- '@': path.resolve(__dirname, './src'),- },- },- build: {- lib: {- entry: path.resolve(__dirname, 'src/main.tsx'),- formats: ['es'],- },- rollupOptions: {- external: ['react', 'react-dom'],- },- },-});-```--### Tauri Wrapper--**tauri/src/main.tsx:**-```typescript-import React from 'react';-import ReactDOM from 'react-dom/client';-import App from '../app/src/App';-import '../app/src/index.css';--ReactDOM.createRoot(document.getElementById('root')!).render(- <React.StrictMode>- <App />- </React.StrictMode>-);-```--**tauri/vite.config.ts:**-```typescript-import { defineConfig } from 'vite';-import react from '@vitejs/plugin-react';-import path from 'path';--export default defineConfig({- plugins: [react()],- resolve: {- alias: {- '@': path.resolve(__dirname, '../app/src'),- },- },- clearScreen: false,- server: {- port: 5173,- strictPort: true,- },- envPrefix: ['VITE_', 'TAURI_'],- build: {- target: 'es2021',- minify: !process.env.TAURI_DEBUG,- sourcemap: !!process.env.TAURI_DEBUG,- outDir: 'dist',- },-});-```--**tauri/package.json:**-```json-{- "name": "@voicebox/tauri",- "private": true,- "version": "0.1.0",- "type": "module",- "scripts": {- "dev": "vite",- "build": "tsc && vite build",- "tauri": "tauri"- },- "dependencies": {- "@tauri-apps/api": "^2.0.0",- "@tauri-apps/plugin-shell": "^2.0.0",- "react": "^18.3.0",- "react-dom": "^18.3.0"- },- "devDependencies": {- "@tauri-apps/cli": "^2.0.0",- "@vitejs/plugin-react": "^4.3.0",- "typescript": "^5.6.0",- "vite": "^5.4.0"- }-}-```--### Web Wrapper--**web/src/main.tsx:**-```typescript-import React from 'react';-import ReactDOM from 'react-dom/client';-import App from '../../app/src/App';-import '../../app/src/index.css';--ReactDOM.createRoot(document.getElementById('root')!).render(- <React.StrictMode>- <App />- </React.StrictMode>-);-```--**web/vite.config.ts:**-```typescript-import { defineConfig } from 'vite';-import react from '@vitejs/plugin-react';-import path from 'path';--export default defineConfig({- plugins: [react()],- resolve: {- alias: {- '@': path.resolve(__dirname, '../app/src'),- },- },- build: {- outDir: 'dist',- },-});-```-------## Development Workflow--### 1. Initial Setup--```bash-# Install Bun (if not installed)-curl -fsSL https://bun.sh/install | bash--# Create Tauri app with official CLI-cd voicebox-bunx create-tauri-app tauri--# Move app code to shared directory-mkdir app-# Move tauri/src/* to app/src/--# Create web directory-mkdir web-cd web-bunx create-vite . --template react-ts--# Setup Bun workspace in root package.json-cat > package.json << 'EOF'-{- "name": "voicebox",- "private": true,- "workspaces": ["app", "tauri", "web"]-}-EOF--# Install all dependencies-bun install-```--### 2. Development--**Terminal 1 - Backend (Python FastAPI):**-```bash-cd backend-python -m venv venv-source venv/bin/activate # or venv\Scripts\activate on Windows-pip install -r requirements.txt-uvicorn main:app --reload --port 8000-```--**Terminal 2 - Frontend (Tauri dev mode):**-```bash-cd tauri-bun run tauri dev-```--This will:-1. Start Vite dev server on port 5173-2. Launch Tauri window pointing to localhost:5173-3. Hot reload on code changes--**For web development:**-```bash-cd web-bun run dev-```--### 3. Building for Production--**Build Python server:**-```bash-./scripts/build-server.sh-```--**Build Tauri app:**-```bash-cd tauri-bun run tauri build-```--This will:-1. Build React frontend with Vite-2. Bundle Python server binary-3. Create platform-specific installers:- - macOS: `.app`, `.dmg`- - Windows: `.exe`, `.msi`- - Linux: `.deb`, `.AppImage`--**Build web app:**-```bash-cd web-bun run build-```-------## Platform-Specific Considerations--### macOS-- Need both Intel and ARM builds-- Sign and notarize for distribution outside App Store-- Request permissions for microphone access (audio recording)--**tauri.conf.json additions:**-```json-{- "bundle": {- "macOS": {- "minimumSystemVersion": "10.15",- "entitlements": "src-tauri/Entitlements.plist"- }- }-}-```--**Entitlements.plist:**-```xml-<?xml version="1.0" encoding="UTF-8"?>-<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">-<plist version="1.0">-<dict>- <key>com.apple.security.device.audio-input</key>- <true/>-</dict>-</plist>-```--### Windows-- Build with NSIS or WiX installer-- Consider code signing for SmartScreen--### Linux-- Provide multiple formats: .deb, .AppImage, .rpm-- Consider Flatpak for broader distribution-------## OpenAPI Client Generation--**scripts/generate-api.sh:**-```bash-#!/bin/bash--# Start backend if not running-if ! curl -s http://localhost:8000/openapi.json > /dev/null; then- echo "Starting backend..."- cd backend- uvicorn main:app --port 8000 &- BACKEND_PID=$!- sleep 5-fi--# Download OpenAPI schema-curl http://localhost:8000/openapi.json > app/openapi.json--# Generate TypeScript client-cd app-bunx openapi-typescript-codegen \- --input openapi.json \- --output src/lib/api \- --client fetch--echo "API client generated in app/src/lib/api"--# Kill backend if we started it-if [ ! -z "$BACKEND_PID" ]; then- kill $BACKEND_PID-fi-```--**Add to package.json:**-```json-{- "scripts": {- "generate:api": "./scripts/generate-api.sh"- }-}-```-------## Server Mode Architecture--### Local Mode (Default)-1. Tauri app starts-2. App invokes `start_server` command-3. Rust spawns bundled Python binary as sidecar-4. Frontend connects to `http://localhost:8000`-5. All features work locally--### Remote Mode (One-Click)-1. User clicks "Start Server" on GPU machine-2. Tauri invokes `start_server` with `--host 0.0.0.0` flag-3. Server displays connection URL (e.g., `http://192.168.1.100:8000`)-4. User enters URL in client app-5. Client connects to remote server-6. All API calls go to remote machine--**Rust command with args:**-```rust-#[command]-async fn start_server(- app: tauri::AppHandle,- state: tauri::State<'_, ServerState>,- remote: bool,-) -> Result<String, String> {- let mut sidecar = app.shell().sidecar("voicebox-server")- .map_err(|e| format!("Failed to get sidecar: {}", e))?;-- if remote {- sidecar = sidecar.args(["--host", "0.0.0.0"]);- }-- // ... rest of spawn logic-}-```-------## CI/CD for Multi-Platform Builds--**GitHub Actions workflow:**-```yaml-name: Build-on:- push:- tags:- - 'v*'--jobs:- build:- strategy:- matrix:- platform: [macos-latest, ubuntu-latest, windows-latest]- runs-on: ${{ matrix.platform }}- steps:- - uses: actions/checkout@v4-- - name: Setup Bun- uses: oven-sh/setup-bun@v2- with:- bun-version: latest-- - name: Setup Rust- uses: dtolnay/rust-toolchain@stable-- - name: Setup Python- uses: actions/setup-python@v5- with:- python-version: '3.11'-- - name: Install dependencies- run: |- cd tauri- bun install-- - name: Build Python server- run: |- cd backend- pip install -r requirements.txt- pip install pyinstaller- python build_binary.py-- - name: Build Tauri app- uses: tauri-apps/tauri-action@v0- env:- GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}- with:- projectPath: tauri- tagName: v__VERSION__- releaseName: 'voicebox v__VERSION__'-```-------## Key Decisions--### Why This Structure?--1. **Shared `app/` directory** - Single source of truth for UI code-2. **Thin wrappers** - `tauri/` and `web/` just configure build tools-3. **Sidecar pattern** - Bundle Python server without modifying Tauri core-4. **PyInstaller** - Creates standalone Python binary with all dependencies-5. **Platform-specific binaries** - Tauri automatically selects correct binary per platform--### Benefits--- ✅ No code duplication between web and desktop-- ✅ Python server bundled - users don't install Python-- ✅ Single command to build everything-- ✅ Type-safe API calls via OpenAPI generation-- ✅ Native performance with Tauri-- ✅ Web fallback for unsupported platforms-- ✅ Fast development with Bun (20-30x faster installs than npm)--### Tradeoffs--- ⚠️ Large bundle size (Python runtime + ML models + Tauri)-- ⚠️ Need to build Python binary for each platform-- ⚠️ First launch slow (model loading)-- ⚠️ Separate web build doesn't include server (requires separate backend deployment)-------## Next Steps--1. Set up monorepo structure-2. Initialize Tauri app with `bunx create-tauri-app`-3. Create shared `app/` directory-4. Configure Vite to share code-5. Build Python server with PyInstaller-6. Configure Tauri sidecar-7. Test on macOS, Windows, Linux-8. Set up CI/CD for multi-platform builds-------## Resources--- [Tauri 2.0 Documentation](https://v2.tauri.app/)-- [Tauri Sidecar Guide](https://v2.tauri.app/develop/sidecar/)-- [Bun Documentation](https://bun.sh/docs)-- [Bun Workspaces](https://bun.sh/docs/install/workspaces)-- [PyInstaller Documentation](https://pyinstaller.org/)-- [React Query Documentation](https://tanstack.com/query/latest)-- [OpenAPI TypeScript Codegen](https://github.com/ferdikoomen/openapi-typescript-codegen)diff --git a/docs/TROUBLESHOOTING.md b/docs/TROUBLESHOOTING.mdnew file mode 100644index 00000000..2b930db4--- /dev/null+++ b/docs/TROUBLESHOOTING.md@@ -0,0 +1,280 @@+# Troubleshooting Guide++Common issues and solutions for Voicebox.++## Installation Issues++### macOS: "Voicebox cannot be opened because it is from an unidentified developer"++**Solution:**+1. Right-click the `.dmg` file+2. Select "Open"+3. Click "Open" in the security dialog+4. Alternatively, go to System Settings → Privacy & Security → Allow Voicebox++### Windows: "Windows protected your PC"++**Solution:**+1. Click "More info"+2. Click "Run anyway"+3. Windows Defender may flag new software; this is normal for unsigned apps++### Linux: AppImage won't run++**Solution:**+```bash+chmod +x voicebox-*.AppImage+./voicebox-*.AppImage+```++## Runtime Issues++### Server won't start++**Symptoms:** App opens but shows "Server not connected"++**Solutions:**+1. **Check Python installation**+ ```bash+ python --version # Should be 3.11++ ```++2. **Check server binary exists**+ - Look in `tauri/src-tauri/binaries/` for your platform+ - Binary should match your system architecture++3. **Check permissions**+ ```bash+ # macOS/Linux+ chmod +x tauri/src-tauri/binaries/voicebox-server-*+ ```++4. **Check logs**+ - macOS: Open Console.app and search for "voicebox"+ - Linux: Check `~/.local/share/voicebox/` for logs+ - Windows: Check Event Viewer++### "Model download failed"++**Symptoms:** First generation fails with download error++**Solutions:**+1. **Check internet connection**+ - Models download from HuggingFace Hub (~2-4GB)+ - First download may take several minutes++2. **Check disk space**+ - Models are cached in `~/.cache/huggingface/`+ - Ensure at least 5GB free space++3. **Manual download** (if automatic fails)+ ```bash+ pip install huggingface_hub+ huggingface-cli download Qwen/Qwen3-TTS-12Hz-1.7B-Base+ ```++### "Out of memory" errors++**Symptoms:** Generation fails with CUDA/VRAM errors++**Solutions:**+1. **Use smaller model**+ - Switch to 0.6B model instead of 1.7B+ - Settings → Model Management → Load 0.6B++2. **Close other applications**+ - Free up GPU memory+ - Close browser tabs, other ML apps++3. **Use CPU mode**+ - Slower but works without GPU+ - Backend automatically falls back to CPU++### Audio playback issues++**Symptoms:** Generated audio won't play++**Solutions:**+1. **Check audio format**+ - Audio is saved as WAV files+ - Ensure your system supports WAV playback++2. **Try downloading audio**+ - Right-click → Download+ - Play in external player++3. **Check browser permissions** (web version)+ - Allow audio autoplay in browser settings++### Slow generation++**Symptoms:** Generation takes >30 seconds++**Solutions:**+1. **Use GPU** (if available)+ - Check Settings → Server Status+ - Should show "GPU available: true"++2. **Enable caching**+ - Voice prompts are cached automatically+ - Second generation with same voice should be faster++3. **Use smaller model**+ - 0.6B model is faster than 1.7B+ - Quality difference is minimal for most voices++4. **Check system resources**+ - Close other CPU/GPU intensive apps+ - Ensure adequate RAM (8GB+ recommended)++## API Issues++### "Connection refused" when using API++**Solutions:**+1. **Check server is running**+ ```bash+ curl http://localhost:8000/health+ ```++2. **Check remote mode**+ - If connecting remotely, ensure server is started with `--host 0.0.0.0`+ - Check firewall settings++3. **Check port availability**+ - Default port is 8000+ - Ensure no other service is using it++### CORS errors in browser++**Solutions:**+1. **Use desktop app** (recommended)+ - Desktop app doesn't have CORS restrictions++2. **Configure CORS** (for web deployment)+ - Update `backend/main.py` CORS settings+ - Add your domain to allowed origins++## Update Issues++### "Update check failed"++**Solutions:**+1. **Check internet connection**+ - Updates are fetched from GitHub releases++2. **Check GitHub access**+ - Ensure `github.com` is accessible+ - Check firewall/proxy settings++3. **Manual update**+ - Download latest release from GitHub+ - Install manually++### "Invalid signature" error++**Solutions:**+1. **Re-download installer**+ - Signature may be corrupted+ - Download fresh copy from GitHub++2. **Check release integrity**+ - Verify `.sig` file matches installer+ - Report issue if signature is invalid++## Data Issues++### Profiles disappeared++**Solutions:**+1. **Check data directory**+ - macOS: `~/Library/Application Support/voicebox/`+ - Windows: `%APPDATA%/voicebox/`+ - Linux: `~/.local/share/voicebox/`++2. **Check database**+ - Database: `data/voicebox.db`+ - Ensure file exists and is readable++3. **Restore from backup**+ - Profiles can be exported/imported+ - Check for backup files++### "Database locked" error++**Solutions:**+1. **Close other instances**+ - Ensure only one Voicebox instance is running++2. **Restart app**+ - Close and reopen Voicebox++3. **Check file permissions**+ - Ensure database file is writable+ - Check directory permissions++## Development Issues++### Build fails++**Solutions:**+1. **Check Rust installation**+ ```bash+ rustc --version+ rustup update+ ```++2. **Check Tauri dependencies**+ ```bash+ cd tauri+ bun install+ ```++3. **Clean build**+ ```bash+ cd tauri/src-tauri+ cargo clean+ cd ../..+ bun run build+ ```++### API client generation fails++**Solutions:**+1. **Start backend server**+ ```bash+ bun run dev:server+ ```++2. **Check OpenAPI endpoint**+ ```bash+ curl http://localhost:8000/openapi.json+ ```++3. **Regenerate client**+ ```bash+ bun run generate:api+ ```++## Still Having Issues?++1. **Check existing issues**+ - Search GitHub issues for similar problems+ - Check closed issues for solutions++2. **Create new issue**+ - Include:+ - OS and version+ - Voicebox version+ - Steps to reproduce+ - Error messages/logs+ - Screenshots (if applicable)++3. **Get help**+ - Check documentation in `docs/`+ - Review `backend/README.md` for API details+ - See `CONTRIBUTING.md` for development help++---++For more help, open an issue on [GitHub](https://github.com/jamiepine/voicebox/issues).