Skip to content

Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.

feat: updated docs

Gabriel Stein6da919f4fc10
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 161ec67f2..706cfa745 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [Unreleased]
+### Added
+- **Automatic Python Installation**: The app now detects and offers to install Python automatically
+ - macOS: Uses Homebrew if available, falls back to official installer
+ - Windows: Downloads and installs official Python with proper PATH configuration
+ - Linux: Uses system package manager (apt, yum, or pacman)
+- **Enhanced Developer Experience**:
+ - Added MIT LICENSE file
+ - Improved documentation for personal vs distribution builds
+ - Added FAQ section to README
+ - Added security information section
+ - Clearer prerequisites and setup instructions
+
### Added
- **Agent Naming System**: Personalize your AI assistant with a custom name for more natural interactions
- Name your agent during onboarding (step 6 of 8)
diff --git a/CLAUDE.md b/CLAUDE.md
new file mode 100644
index 000000000..faa1fdee0
--- /dev/null
+++ b/CLAUDE.md
@@ -0,0 +1,267 @@
+# OpenWispr Technical Reference for AI Assistants
+
+This document provides comprehensive technical details about the OpenWispr project architecture for AI assistants working on the codebase.
+
+## Project Overview
+
+OpenWispr is an Electron-based desktop dictation application that uses OpenAI Whisper for speech-to-text transcription. It supports both local (privacy-focused) and cloud (OpenAI API) processing modes.
+
+## Architecture Overview
+
+### Core Technologies
+- **Frontend**: React 19, TypeScript, Tailwind CSS v4, Vite
+- **Desktop Framework**: Electron 36 with context isolation
+- **Database**: better-sqlite3 for local transcription history
+- **UI Components**: shadcn/ui with Radix primitives
+- **Speech Processing**: OpenAI Whisper (local Python bridge + API)
+- **Audio Processing**: FFmpeg (bundled via ffmpeg-static)
+
+### Key Architectural Decisions
+
+1. **Dual Window Architecture**:
+ - Main Window: Minimal overlay for dictation (draggable, always on top)
+ - Control Panel: Full settings interface (normal window)
+ - Both use same React codebase with URL-based routing
+
+2. **Process Separation**:
+ - Main Process: Electron main, IPC handlers, database operations
+ - Renderer Process: React app with context isolation
+ - Preload Script: Secure bridge between processes
+
+3. **Audio Pipeline**:
+ - MediaRecorder API → Blob → ArrayBuffer → Base64 → IPC → File → FFmpeg → Whisper
+ - Automatic cleanup of temporary files after processing
+
+## File Structure and Responsibilities
+
+### Main Process Files
+
+- **main.js**: Application entry point, initializes all managers
+- **preload.js**: Exposes safe IPC methods to renderer via window.api
+
+### Helper Modules (src/helpers/)
+
+- **audioManager.js**: Handles audio device management
+- **clipboard.js**: Cross-platform clipboard operations with AppleScript fallback
+- **database.js**: SQLite operations for transcription history
+- **debugLogger.js**: Debug logging system with file output
+- **devServerManager.js**: Vite dev server integration
+- **dragManager.js**: Window dragging functionality
+- **environment.js**: Environment variable and OpenAI API management
+- **hotkeyManager.js**: Global hotkey registration and management
+- **ipcHandlers.js**: Centralized IPC handler registration
+- **menuManager.js**: Application menu management
+- **pythonInstaller.js**: Automatic Python installation for all platforms
+- **tray.js**: System tray icon and menu
+- **whisper.js**: Local Whisper integration and Python bridge
+- **windowConfig.js**: Centralized window configuration
+- **windowManager.js**: Window creation and lifecycle management
+
+### React Components (src/components/)
+
+- **App.jsx**: Main dictation interface with recording states
+- **ControlPanel.tsx**: Settings, history, model management UI
+- **OnboardingFlow.tsx**: 8-step first-time setup wizard
+- **SettingsPage.tsx**: Comprehensive settings interface
+- **WhisperModelPicker.tsx**: Model selection and download UI
+- **ui/**: Reusable UI components (buttons, cards, inputs, etc.)
+
+### React Hooks (src/hooks/)
+
+- **useAudioRecording.js**: MediaRecorder API wrapper with error handling
+- **useClipboard.ts**: Clipboard operations hook
+- **useDialogs.ts**: Electron dialog integration
+- **useHotkey.js**: Hotkey state management
+- **useLocalStorage.ts**: Type-safe localStorage wrapper
+- **usePermissions.ts**: System permission checks
+- **usePython.ts**: Python installation state
+- **useSettings.ts**: Application settings management
+- **useWhisper.ts**: Whisper model management
+
+### Services
+
+- **ReasoningService.js**: AI processing for agent-addressed commands
+ - Detects when user addresses their named agent
+ - Routes to appropriate AI provider (OpenAI/Anthropic)
+ - Removes agent name from final output
+
+### Python Bridge
+
+- **whisper_bridge.py**: Standalone Python script for local Whisper
+ - Accepts audio file path and model selection
+ - Returns JSON with transcription result
+ - Handles FFmpeg path resolution for bundled executable
+ - 30-second timeout protection
+
+## Key Implementation Details
+
+### 1. FFmpeg Integration
+
+FFmpeg is bundled with the app and doesn't require system installation:
+```javascript
+// FFmpeg is unpacked from ASAR to app.asar.unpacked/node_modules/ffmpeg-static/
+// Python bridge receives FFmpeg path via environment variables:
+// FFMPEG_PATH, FFMPEG_EXECUTABLE, FFMPEG_BINARY
+```
+
+### 2. Audio Recording Flow
+
+1. User presses hotkey → MediaRecorder starts
+2. Audio chunks collected in array
+3. User presses hotkey again → Recording stops
+4. Blob created from chunks → Converted to ArrayBuffer
+5. Sent via IPC as Base64 string (size limits)
+6. Main process writes to temporary file
+7. Whisper processes file → Result sent back
+8. Temporary file deleted
+
+### 3. Local Whisper Models
+
+Models stored in `~/.cache/whisper/`:
+- tiny: 39MB (fastest, lowest quality)
+- base: 74MB (recommended balance)
+- small: 244MB (better quality)
+- medium: 769MB (high quality)
+- large: 1.5GB (best quality)
+- turbo: 809MB (fast with good quality)
+
+### 4. Database Schema
+
+```sql
+CREATE TABLE transcriptions (
+ id INTEGER PRIMARY KEY AUTOINCREMENT,
+ timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
+ original_text TEXT NOT NULL,
+ processed_text TEXT,
+ is_processed BOOLEAN DEFAULT 0,
+ processing_method TEXT DEFAULT 'none',
+ agent_name TEXT,
+ error TEXT
+);
+```
+
+### 5. Settings Storage
+
+Settings stored in localStorage with these keys:
+- `whisperModel`: Selected Whisper model
+- `useLocalWhisper`: Boolean for local vs cloud
+- `openaiApiKey`: Encrypted API key
+- `anthropicApiKey`: Encrypted API key
+- `language`: Selected language code
+- `agentName`: User's custom agent name
+- `reasoningModel`: Selected AI model for processing
+- `hotkey`: Custom hotkey configuration
+- `hasCompletedOnboarding`: Onboarding completion flag
+
+### 6. Language Support
+
+58 languages supported (see src/utils/languages.ts):
+- Each language has a two-letter code and label
+- "auto" for automatic detection
+- Passed to Whisper via --language parameter
+
+### 7. Agent Naming System
+
+- User names their agent during onboarding (step 6/8)
+- Name stored in localStorage and database
+- ReasoningService detects "Hey [AgentName]" patterns
+- AI processes command and removes agent reference from output
+
+### 8. Debug Mode
+
+Enable with `--debug` flag or `OPENWISPR_DEBUG=true`:
+- Logs saved to platform-specific app data directory
+- Comprehensive logging of audio pipeline
+- FFmpeg path resolution details
+- Audio level analysis
+
+## Development Guidelines
+
+### Adding New Features
+
+1. **New IPC Channel**: Add to both ipcHandlers.js and preload.js
+2. **New Setting**: Update useSettings.ts and SettingsPage.tsx
+3. **New UI Component**: Follow shadcn/ui patterns in src/components/ui
+4. **New Manager**: Create in src/helpers/, initialize in main.js
+
+### Testing Checklist
+
+- [ ] Test both local and cloud processing modes
+- [ ] Verify hotkey works globally
+- [ ] Check clipboard pasting on all platforms
+- [ ] Test with different audio input devices
+- [ ] Verify Python auto-installation
+- [ ] Test all Whisper models
+- [ ] Check agent naming functionality
+
+### Common Issues and Solutions
+
+1. **No Audio Detected**:
+ - Check FFmpeg path resolution
+ - Verify microphone permissions
+ - Check audio levels in debug logs
+
+2. **Transcription Fails**:
+ - Ensure Python/Whisper installed
+ - Check temporary file creation
+ - Verify FFmpeg is executable
+
+3. **Clipboard Not Working**:
+ - macOS: Check accessibility permissions
+ - Use AppleScript fallback on macOS
+
+4. **Build Issues**:
+ - Use `npm run pack` for unsigned builds
+ - Signing requires Apple Developer account
+ - ASAR unpacking needed for FFmpeg/Python bridge
+
+### Platform-Specific Notes
+
+**macOS**:
+- Requires accessibility permissions for clipboard
+- Uses AppleScript for reliable pasting
+- Notarization needed for distribution
+
+**Windows**:
+- Python installer handles PATH automatically
+- No special permissions needed
+- NSIS installer for distribution
+
+**Linux**:
+- Multiple package manager support
+- Standard XDG directories
+- AppImage for distribution
+
+## Code Style and Conventions
+
+- Use TypeScript for new React components
+- Follow existing patterns in helpers/
+- Descriptive error messages for users
+- Comprehensive debug logging
+- Clean up resources (files, listeners)
+- Handle edge cases gracefully
+
+## Performance Considerations
+
+- Whisper model size vs speed tradeoff
+- Audio blob size limits for IPC (10MB)
+- Temporary file cleanup
+- Memory usage with large models
+- Process timeout protection (30s)
+
+## Security Considerations
+
+- API keys stored in system keychain when possible
+- Context isolation enabled
+- No remote code execution
+- Sanitized file paths
+- Limited IPC surface area
+
+## Future Enhancements to Consider
+
+- Streaming transcription support
+- Custom wake word detection
+- Multi-language UI
+- Cloud model selection
+- Batch transcription
+- Export formats beyond clipboard
\ No newline at end of file
diff --git a/LICENSE b/LICENSE
new file mode 100644
index 000000000..382baee3c
--- /dev/null
+++ b/LICENSE
@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2024 OpenWispr Team
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.
\ No newline at end of file
diff --git a/README.md b/README.md
index a5ba96f83..d7886a427 100644
--- a/README.md
+++ b/README.md
@@ -2,6 +2,10 @@
An open source desktop dictation application that converts speech to text using OpenAI Whisper. Features both local and cloud processing options for maximum flexibility and privacy.
+## License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. This means you can freely use, modify, and distribute this software for personal or commercial purposes.
+
## Features
- 🎤 **Global Hotkey**: Customizable hotkey to start/stop dictation from anywhere (default: backtick `)
@@ -17,38 +21,66 @@ An open source desktop dictation application that converts speech to text using
- ⚡ **Automatic Pasting**: Transcribed text automatically pastes at your cursor location
- 🖱️ **Draggable Interface**: Move the dictation panel anywhere on your screen
+## Prerequisites
+
+- **Node.js 18+** and npm (Download from [nodejs.org](https://nodejs.org/))
+- **macOS 10.15+**, **Windows 10+**, or **Linux**
+- **Python 3.7+** (Optional - the app can install it automatically for local Whisper processing)
+
## Quick Start
-1. **Install dependencies**:
+### For Personal Use (Recommended)
+
+1. **Clone the repository**:
+ ```bash
+ git clone https://github.com/HeroTools/open-wispr.git
+ cd open-wispr
+ ```
+
+2. **Install dependencies**:
```bash
npm install
```
-2. **Optional: Set up OpenAI API key** (only needed for cloud processing):
+3. **Optional: Set up OpenAI API key** (only needed for cloud processing):
- **Method A - .env file**:
+ **Method A - Environment file**:
```bash
cp env.example .env
# Edit .env and add your OpenAI API key
```
- **Method B - Control Panel**:
- ```bash
- # Run the app and configure the API key through the Control Panel
- ```
+ **Method B - In-app configuration**:
+ - Run the app and configure the API key through the Control Panel
-3. **Run in development**:
+4. **Run the application**:
```bash
- npm run dev
+ npm run dev # Development mode with hot reload
+ # OR
+ npm start # Production mode
```
-4. **Build for production**:
- ```bash
- # Build for your platform
- npm run build:mac # macOS
- npm run build:win # Windows
- npm run build:linux # Linux
- ```
+### Building for Personal Use (Optional)
+
+If you want to build a standalone app for personal use:
+
+```bash
+# Build without code signing (personal use only)
+npm run pack
+
+# The built app will be in the 'dist' folder
+```
+
+### Building for Distribution
+
+For maintainers who need to distribute signed builds:
+
+```bash
+# Requires code signing certificates and notarization setup
+npm run build:mac # macOS (requires Apple Developer account)
+npm run build:win # Windows (requires code signing cert)
+npm run build:linux # Linux
+```
### First Time Setup
@@ -236,14 +268,23 @@ DEBUG=false
### Local Whisper Setup
-For local processing, OpenWispr will automatically:
-1. Install Python Whisper package via pip
-2. Download your chosen model (tiny, base, small, medium, large)
-3. Handle all transcription locally
+For local processing, OpenWispr offers automated setup:
+
+1. **Automatic Python Installation** (if needed):
+ - The app will detect if Python is missing
+ - Offers to install Python 3.11 automatically
+ - macOS: Uses Homebrew if available, otherwise official installer
+ - Windows: Downloads and installs official Python
+ - Linux: Uses system package manager (apt, yum, or pacman)
+
+2. **Automatic Whisper Setup**:
+ - Installs OpenAI Whisper package via pip
+ - Downloads your chosen model on first use
+ - Handles all transcription locally
**Requirements**:
-- Python 3.7+ installed on your system
- Sufficient disk space for models (39MB - 1.5GB depending on model)
+- Admin/sudo access may be required for Python installation
### Customization
@@ -257,11 +298,30 @@ For local processing, OpenWispr will automatically:
## Contributing
+We welcome contributions! Please follow these steps:
+
1. Fork the repository
-2. Create a feature branch
-3. Make your changes
-4. Run tests and lint
-5. Submit a pull request
+2. Create a feature branch (`git checkout -b feature/amazing-feature`)
+3. Commit your changes (`git commit -m 'Add amazing feature'`)
+4. Push to the branch (`git push origin feature/amazing-feature`)
+5. Open a Pull Request
+
+### Development Guidelines
+
+- Run `npm run lint` before committing
+- Follow the existing code style
+- Update documentation as needed
+- Test on your target platform before submitting
+## Security
+
+OpenWispr is designed with privacy and security in mind:
+
+- **Local Processing Option**: Keep your voice data completely private
+- **No Analytics**: We don't collect any usage data or telemetry
+- **Open Source**: All code is available for review
+- **Secure Storage**: API keys are stored securely in your system's keychain/credential manager
+- **Minimal Permissions**: Only requests necessary permissions (microphone, accessibility)
+
## Troubleshooting
### Common Issues
@@ -296,3 +356,35 @@ For local processing, OpenWispr will automatically:
- **Cloud Processing**: Generally faster but requires internet connection
- **Model Selection**: tiny (fastest) → base (recommended) → small → medium → large (best quality)
- **Permissions**: Ensure all required permissions are granted for smooth operation
+
+## FAQ
+
+**Q: Is OpenWispr really free?**
+A: Yes! OpenWispr is open source and free to use. You only pay for OpenAI API usage if you choose cloud processing.
+
+**Q: Which processing method should I use?**
+A: Use local processing for privacy and offline use. Use cloud processing for speed and convenience.
+
+**Q: Can I use this commercially?**
+A: Yes! The MIT license allows commercial use.
+
+**Q: How do I change the hotkey?**
+A: Open the Control Panel (right-click tray icon) and go to Settings. You can set any key as your hotkey.
+
+**Q: Is my data secure?**
+A: With local processing, your audio never leaves your device. With cloud processing, audio is sent to OpenAI's servers (see their privacy policy).
+
+**Q: What languages are supported?**
+A: OpenWispr supports 58 languages including English, Spanish, French, German, Chinese, Japanese, and more. Set your preferred language in the .env file or use auto-detect.
+
+## Project Status
+
+OpenWispr is actively maintained and ready for production use. Current version: 1.0.1
+
+- ✅ Core functionality complete
+- ✅ Cross-platform support
+- ✅ Local and cloud processing
+- ✅ Automatic Python/Whisper installation
+- ✅ Agent naming system
+- ✅ Draggable interface
+- 🚧 Continuous improvements and bug fixes