Skip to content

Voice-to-text dictation app with local (Nvidia Parakeet/Whisper) and cloud models (BYOK). Privacy-first and available cross-platform.

Nvidia parakeet cross platform (#155)

Gabriel Steinbed49fedf903
diff --git a/CHANGELOG.md b/CHANGELOG.md
index b38215414..650b3019c 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
## [1.2.16] - 2026-01-26
### Added
+- **NVIDIA Parakeet Support**: Fast local transcription via sherpa-onnx runtime with INT8 quantized models
+ - `parakeet-tdt-0.6b-v3`: Multilingual (25 languages), ~680MB
- **Windows Push-to-Talk**: Native Windows key listener with low-level keyboard hook for true push-to-talk functionality
- Supports compound hotkeys like `Ctrl+Shift+F11` or `CommandOrControl+Space`
- Prebuilt binary automatically downloaded from GitHub releases
@@ -25,6 +27,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- **Dev Server Port Alignment**: Development server port configuration improved for consistency
### Fixed
+- **Whisper Model Path Resolution**: Fixed `large` and `turbo` model lookup failures by using registry-defined filenames (`ggml-large-v3.bin`, `ggml-large-v3-turbo.bin`) instead of hardcoded pattern
- **Windows Production Build**: Fixed Windows production build issues with proper binary bundling
- **Code Quality**: Various code quality improvements in download scripts and dev server management
diff --git a/CLAUDE.md b/CLAUDE.md
index e870d05eb..eed808a49 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -13,7 +13,7 @@ OpenWhispr is an Electron-based desktop dictation application that uses whisper.
- **Desktop Framework**: Electron 36 with context isolation
- **Database**: better-sqlite3 for local transcription history
- **UI Components**: shadcn/ui with Radix primitives
-- **Speech Processing**: whisper.cpp (bundled native binary) + OpenAI API
+- **Speech Processing**: whisper.cpp + NVIDIA Parakeet (via sherpa-onnx) + OpenAI API
- **Audio Processing**: FFmpeg (bundled via ffmpeg-static)
### Key Architectural Decisions
@@ -76,6 +76,8 @@ OpenWhispr is an Electron-based desktop dictation application that uses whisper.
- **menuManager.js**: Application menu management
- **tray.js**: System tray icon and menu
- **whisper.js**: Local whisper.cpp integration and model management
+- **parakeet.js**: NVIDIA Parakeet model management via sherpa-onnx
+- **parakeetServer.js**: sherpa-onnx CLI wrapper for transcription
- **windowConfig.js**: Centralized window configuration
- **windowManager.js**: Window creation and lifecycle management
@@ -118,12 +120,28 @@ OpenWhispr is an Electron-based desktop dictation application that uses whisper.
- GGML model downloads from HuggingFace
- Models stored in `~/.cache/openwhispr/whisper-models/`
+### NVIDIA Parakeet Integration (via sherpa-onnx)
+
+- **parakeet.js**: Model management for NVIDIA Parakeet ASR models
+ - Uses sherpa-onnx runtime for cross-platform ONNX inference
+ - Bundled binaries in `resources/bin/sherpa-onnx-{platform}-{arch}`
+ - INT8 quantized models for efficient CPU inference
+ - Models stored in `~/.cache/openwhispr/parakeet-models/`
+ - Server pre-warming on startup when `LOCAL_TRANSCRIPTION_PROVIDER=nvidia` is set
+ - Provider preference persisted to `.env` via `saveAllKeysToEnvFile()` on server start/stop
+
+- **Available Models**:
+ - `parakeet-tdt-0.6b-v3`: Multilingual (25 languages), ~680MB
+
+- **Download URLs**: Models from sherpa-onnx ASR models release on GitHub
+
### Build Scripts (scripts/)
- **download-whisper-cpp.js**: Downloads whisper.cpp binaries from GitHub releases
- **download-llama-server.js**: Downloads llama.cpp server for local LLM inference
- **download-nircmd.js**: Downloads nircmd.exe for Windows clipboard operations
- **download-windows-key-listener.js**: Downloads prebuilt Windows key listener binary
+- **download-sherpa-onnx.js**: Downloads sherpa-onnx binaries for Parakeet support
- **build-globe-listener.js**: Compiles macOS Globe key listener from Swift source
- **build-windows-key-listener.js**: Compiles Windows key listener (for local development)
- **run-electron.js**: Development script to launch Electron with proper environment
@@ -195,6 +213,10 @@ Settings stored in localStorage with these keys:
- `hasCompletedOnboarding`: Onboarding completion flag
- `customDictionary`: JSON array of words/phrases for improved transcription accuracy
+Environment variables persisted to `.env` (via `saveAllKeysToEnvFile()`):
+- `LOCAL_TRANSCRIPTION_PROVIDER`: Transcription engine (`nvidia` for Parakeet)
+- `PARAKEET_MODEL`: Selected Parakeet model name (e.g., `parakeet-tdt-0.6b-v3`)
+
### 6. Language Support
58 languages supported (see src/utils/languages.ts):
diff --git a/README.md b/README.md
index 4c5b324d1..47d754cbf 100644
--- a/README.md
+++ b/README.md
@@ -27,6 +27,7 @@ This project is licensed under the MIT License - see the [LICENSE](LICENSE) file
- πŸ“± **Control Panel**: Manage settings, view history, and configure API keys
- πŸ—„οΈ **Transcription History**: SQLite database stores all your transcriptions locally
- πŸ”§ **Model Management**: Download and manage local Whisper models (tiny, base, small, medium, large, turbo)
+- ⚑ **NVIDIA Parakeet**: Fast local transcription via sherpa-onnx (multilingual, 25 languages)
- 🧹 **Model Cleanup**: One-click removal of cached Whisper models with uninstall hooks to keep disks tidy
- 🌐 **Cross-Platform**: Works on macOS, Windows, and Linux
- ⚑ **Automatic Pasting**: Transcribed text automatically pastes at your cursor location
@@ -380,7 +381,7 @@ open-whispr/
- **Desktop**: Electron 36 with context isolation
- **UI Components**: shadcn/ui with Radix primitives
- **Database**: better-sqlite3 for local transcription storage
-- **Speech-to-Text**: OpenAI Whisper (powered by whisper.cpp for local, OpenAI API for cloud)
+- **Speech-to-Text**: OpenAI Whisper (whisper.cpp) + NVIDIA Parakeet (sherpa-onnx) for local, OpenAI API for cloud
- **Icons**: Lucide React for consistent iconography
## Development
@@ -490,6 +491,21 @@ For local processing, OpenWhispr uses OpenAI's Whisper model via whisper.cpp - a
**Upgrading from Python-based version**: If you previously used the Python-based Whisper, you'll need to re-download models in GGML format. You can safely delete the old Python environment (`~/.openwhispr/python/`) and PyTorch models (`~/.cache/whisper/`) to reclaim disk space.
+### Local Parakeet Setup (Alternative)
+
+OpenWhispr also supports NVIDIA Parakeet models via sherpa-onnx - a fast alternative to Whisper:
+
+1. **Bundled Binary**: sherpa-onnx is bundled with the app for all platforms
+2. **INT8 Quantized Models**: Efficient CPU inference
+3. **Models stored in**: `~/.cache/openwhispr/parakeet-models/`
+
+**Available Models**:
+- `parakeet-tdt-0.6b-v3`: Multilingual (25 languages), ~680MB
+
+**When to use Parakeet vs Whisper**:
+- **Parakeet**: Best for speed-critical use cases or lower-end hardware
+- **Whisper**: Best for quality-critical use cases or when you need specific model sizes
+
### Customization
- **Hotkey**: Change in the Control Panel (default: backtick `) - fully customizable
@@ -602,6 +618,8 @@ OpenWhispr is actively maintained and ready for production use. Current version:
- **[OpenAI Whisper](https://github.com/openai/whisper)** - The speech recognition model that powers both local and cloud transcription
- **[whisper.cpp](https://github.com/ggerganov/whisper.cpp)** - High-performance C++ implementation of Whisper for local processing
+- **[NVIDIA Parakeet](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v3)** - Fast ASR model for efficient local transcription
+- **[sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx)** - Cross-platform ONNX runtime for Parakeet model inference
- **[Electron](https://www.electronjs.org/)** - Cross-platform desktop application framework
- **[React](https://react.dev/)** - UI component library
- **[shadcn/ui](https://ui.shadcn.com/)** - Beautiful UI components built on Radix primitives
diff --git a/electron-builder.json b/electron-builder.json
index 01c79650c..ca51a9eac 100644
--- a/electron-builder.json
+++ b/electron-builder.json
@@ -88,7 +88,7 @@
{
"from": "resources/bin/",
"to": "bin/",
- "filter": ["whisper-cpp-*", "whisper-server-*", "llama-server-*", "windows-key-listener*", "*.dylib", "*.dll", "*.so*"]
+ "filter": ["whisper-cpp-*", "whisper-server-*", "llama-server-*", "sherpa-onnx-*", "windows-key-listener*", "*.dylib", "*.dll", "*.so*"]
}
],
"mac": {
diff --git a/main.js b/main.js
index 0aeebbab6..c5cc2885c 100644
--- a/main.js
+++ b/main.js
@@ -44,6 +44,7 @@ const WindowManager = require("./src/helpers/windowManager");
const DatabaseManager = require("./src/helpers/database");
const ClipboardManager = require("./src/helpers/clipboard");
const WhisperManager = require("./src/helpers/whisper");
+const ParakeetManager = require("./src/helpers/parakeet");
const TrayManager = require("./src/helpers/tray");
const IPCHandlers = require("./src/helpers/ipcHandlers");
const UpdateManager = require("./src/updater");
@@ -58,6 +59,7 @@ let hotkeyManager = null;
let databaseManager = null;
let clipboardManager = null;
let whisperManager = null;
+let parakeetManager = null;
let trayManager = null;
let updateManager = null;
let globeKeyManager = null;
@@ -103,6 +105,7 @@ function initializeManagers() {
databaseManager = new DatabaseManager();
clipboardManager = new ClipboardManager();
whisperManager = new WhisperManager();
+ parakeetManager = new ParakeetManager();
trayManager = new TrayManager();
updateManager = new UpdateManager();
globeKeyManager = new GlobeKeyManager();
@@ -144,6 +147,7 @@ function initializeManagers() {
databaseManager,
clipboardManager,
whisperManager,
+ parakeetManager,
windowManager,
updateManager,
windowsKeyManager,
@@ -180,6 +184,19 @@ async function startApp() {
debugLogger.debug("Whisper startup init error (non-fatal)", { error: err.message });
});
+ // Initialize Parakeet manager at startup (don't await to avoid blocking)
+ // Settings can be provided via environment variables for server pre-warming:
+ // - LOCAL_TRANSCRIPTION_PROVIDER=nvidia to enable parakeet
+ // - PARAKEET_MODEL=parakeet-tdt-0.6b-v3 (model name)
+ const parakeetSettings = {
+ localTranscriptionProvider: process.env.LOCAL_TRANSCRIPTION_PROVIDER || "",
+ parakeetModel: process.env.PARAKEET_MODEL || "parakeet-tdt-0.6b-v3",
+ };
+ parakeetManager.initializeAtStartup(parakeetSettings).catch((err) => {
+ // Parakeet not being available at startup is not critical
+ debugLogger.debug("Parakeet startup init error (non-fatal)", { error: err.message });
+ });
+
// Pre-warm llama-server if local reasoning is configured
// Settings can be provided via environment variables:
// - REASONING_PROVIDER=local to enable local reasoning
@@ -526,6 +543,10 @@ if (gotSingleInstanceLock) {
if (whisperManager) {
whisperManager.stopServer().catch(() => {});
}
+ // Stop parakeet WS server if running
+ if (parakeetManager) {
+ parakeetManager.stopServer().catch(() => {});
+ }
// Stop llama-server if running
const modelManager = require("./src/helpers/modelManagerBridge").default;
modelManager.stopServer().catch(() => {});
diff --git a/package-lock.json b/package-lock.json
index 5424126fd..c1c326663 100644
--- a/package-lock.json
+++ b/package-lock.json
@@ -32,7 +32,8 @@
"tailwind-merge": "^3.3.1",
"tar": "^7.4.3",
"tw-animate-css": "^1.3.4",
- "unzipper": "^0.12.3"
+ "unzipper": "^0.12.3",
+ "ws": "^8.19.0"
},
"devDependencies": {
"@electron/notarize": "^3.0.1",
@@ -88,7 +89,6 @@
"integrity": "sha512-H3mcG6ZDLTlYfaSNi0iOKkigqMFvkTKlGUYlD8GW7nNOYRrevuA46iTypPyv+06V3fEmvvazfntkBU34L0azAw==",
"dev": true,
"license": "MIT",
- "peer": true,
"dependencies": {
"@babel/code-frame": "^7.28.6",
"@babel/generator": "^7.28.6",
@@ -821,6 +821,7 @@
"dev": true,
"license": "BSD-2-Clause",
"optional": true,
+ "peer": true,
"dependencies": {
"cross-dirname": "^0.1.0",
"debug": "^4.3.4",
@@ -842,6 +843,7 @@
"dev": true,
"license": "MIT",
"optional": true,
+ "peer": true,
"dependencies": {
"graceful-fs": "^4.2.0",
"jsonfile": "^6.0.1",
@@ -858,6 +860,7 @@
"dev": true,
"license": "MIT",
"optional": true,
+ "peer": true,
"dependencies": {
"universalify": "^2.0.0"
},
@@ -872,6 +875,7 @@
"dev": true,
"license": "MIT",
"optional": true,
+ "peer": true,
"engines": {
"node": ">= 10.0.0"
}
@@ -3640,7 +3644,6 @@
"resolved": "https://registry.npmjs.org/@types/react/-/react-19.2.9.tgz",
"integrity": "sha512-Lpo8kgb/igvMIPeNV2rsYKTgaORYdO1XGVZ4Qz3akwOj0ySGYMPlQWa8BaLn0G63D1aSaAQ5ldR06wCpChQCjA==",
"license": "MIT",
- "peer": true,
"dependencies": {
"csstype": "^3.2.2"
}
@@ -3651,7 +3654,6 @@
"integrity": "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ==",
"devOptional": true,
"license": "MIT",
- "peer": true,
"peerDependencies": {
"@types/react": "^19.2.0"
}
@@ -3736,7 +3738,6 @@
"integrity": "sha512-nm3cvFN9SqZGXjmw5bZ6cGmvJSyJPn0wU9gHAZZHDnZl2wF9PhHv78Xf06E0MaNk4zLVHL8hb2/c32XvyJOLQg==",
"dev": true,
"license": "MIT",
- "peer": true,
"dependencies": {
"@typescript-eslint/scope-manager": "8.53.1",
"@typescript-eslint/types": "8.53.1",
@@ -4021,7 +4022,6 @@
"integrity": "sha512-NZyJarBfL7nWwIq+FDL6Zp/yHEhePMNnnJ0y3qfieCrmNvYct8uvtiV41UvlSe6apAfk0fY1FbWx+NwfmpvtTg==",
"dev": true,
"license": "MIT",
- "peer": true,
"bin": {
"acorn": "bin/acorn"
},
@@ -4055,7 +4055,6 @@
"integrity": "sha512-j3fVLgvTo527anyYyJOGTYJbG+vnnQYvE0m5mmkc1TK+nxAppkCLMIL0aZ4dblVCNoGShhm+kzE4ZUykBoMg4g==",
"devOptional": true,
"license": "MIT",
- "peer": true,
"dependencies": {
"fast-deep-equal": "^3.1.1",
"fast-json-stable-stringify": "^2.0.0",
@@ -4703,7 +4702,6 @@
}
],
"license": "MIT",
- "peer": true,
"dependencies": {
"baseline-browser-mapping": "^2.9.0",
"caniuse-lite": "^1.0.30001759",
@@ -5367,7 +5365,8 @@
"integrity": "sha512-+R08/oI0nl3vfPcqftZRpytksBXDzOUveBq/NBVx0sUp1axwzPQrKinNx5yd5sxPu8j1wIy8AfnVQ+5eFdha6Q==",
"dev": true,
"license": "MIT",
- "optional": true
+ "optional": true,
+ "peer": true
},
"node_modules/cross-env": {
"version": "10.1.0",
@@ -5682,7 +5681,6 @@
"integrity": "sha512-ce4Ogns4VMeisIuCSK0C62umG0lFy012jd8LMZ6w/veHUeX4fqfDrGe+HTWALAEwK6JwKP+dhPvizhArSOsFbg==",
"dev": true,
"license": "MIT",
- "peer": true,
"dependencies": {
"app-builder-lib": "26.4.0",
"builder-util": "26.3.4",
@@ -6109,6 +6107,7 @@
"dev": true,
"hasInstallScript": true,
"license": "MIT",
+ "peer": true,
"dependencies": {
"@electron/asar": "^3.2.1",
"debug": "^4.1.1",
@@ -6129,6 +6128,7 @@
"integrity": "sha512-YJDaCJZEnBmcbw13fvdAM9AwNOJwOzrE4pqMqBq5nFiEqXUqHwlK4B+3pUw6JNvfSPtX05xFHtYy/1ni01eGCw==",
"dev": true,
"license": "MIT",
+ "peer": true,
"dependencies": {
"graceful-fs": "^4.1.2",
"jsonfile": "^4.0.0",
@@ -6323,7 +6323,6 @@
"integrity": "sha512-LEyamqS7W5HB3ujJyvi0HQK/dtVINZvd5mAAp9eT5S/ujByGjiZLCzPcHVzuXbpJDJF/cxwHlfceVUDZ2lnSTw==",
"dev": true,
"license": "MIT",
- "peer": true,
"dependencies": {
"@eslint-community/eslint-utils": "^4.8.0",
"@eslint-community/regexpp": "^4.12.1",
@@ -9264,6 +9263,7 @@
"integrity": "sha512-FP+p8RB8OWpF3YZBCrP5gtADmtXApB5AMLn+vdyA+PyxCjrCs00mjyUozssO33cwDeT3wNGdLxJ5M//YqtHAJw==",
"dev": true,
"license": "MIT",
+ "peer": true,
"dependencies": {
"minimist": "^1.2.6"
},
@@ -9830,7 +9830,6 @@
"integrity": "sha512-5gTmgEY/sqK6gFXLIsQNH19lWb4ebPDLA4SdLP7dsWkIXHWlG66oPuVvXSGFPppYZz8ZDZq0dYYrbHfBCVUb1Q==",
"dev": true,
"license": "MIT",
- "peer": true,
"engines": {
"node": ">=12"
},
@@ -9873,7 +9872,6 @@
}
],
"license": "MIT",
- "peer": true,
"dependencies": {
"nanoid": "^3.3.11",
"picocolors": "^1.1.1",
@@ -9897,6 +9895,7 @@
"dev": true,
"license": "MIT",
"optional": true,
+ "peer": true,
"dependencies": {
"commander": "^9.4.0"
},
@@ -9914,6 +9913,7 @@
"dev": true,
"license": "MIT",
"optional": true,
+ "peer": true,
"engines": {
"node": "^12.20.0 || >=14"
}
@@ -10128,7 +10128,6 @@
"resolved": "https://registry.npmjs.org/react/-/react-19.2.3.tgz",
"integrity": "sha512-Ku/hhYbVjOQnXDZFv2+RibmLFGwFdeeKHFcOTlrt7xplBnya5OGn/hIRDsqDiSUcfORsDC7MPxwork8jBwsIWA==",
"license": "MIT",
- "peer": true,
"engines": {
"node": ">=0.10.0"
}
@@ -10138,7 +10137,6 @@
"resolved": "https://registry.npmjs.org/react-dom/-/react-dom-19.2.3.tgz",
"integrity": "sha512-yELu4WmLPw5Mr/lmeEpox5rw3RETacE++JgHqQzd2dg+YbJuat3jH4ingc+WPZhxaoFzdv9y33G+F7Nl5O0GBg==",
"license": "MIT",
- "peer": true,
"dependencies": {
"scheduler": "^0.27.0"
},
@@ -10449,6 +10447,7 @@
"deprecated": "Rimraf versions prior to v4 are no longer supported",
"dev": true,
"license": "ISC",
+ "peer": true,
"dependencies": {
"glob": "^7.1.3"
},
@@ -11181,6 +11180,7 @@
"integrity": "sha512-yYrrsWnrXMcdsnu/7YMYAofM1ktpL5By7vZhf15CrXijWWrEYZks5AXBudalfSWJLlnen/QUJUB5aoB0kqZUGA==",
"dev": true,
"license": "MIT",
+ "peer": true,
"dependencies": {
"mkdirp": "^0.5.1",
"rimraf": "~2.6.2"
@@ -12019,7 +12019,6 @@
"integrity": "sha512-+Oxm7q9hDoLMyJOYfUYBuHQo+dkAloi33apOPP56pzj+vsdJDzr+j1NISE5pyaAuKL4A3UD34qd0lx5+kfKp2g==",
"dev": true,
"license": "MIT",
- "peer": true,
"dependencies": {
"esbuild": "^0.25.0",
"fdir": "^6.4.4",
@@ -12178,6 +12177,27 @@
"integrity": "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ==",
"license": "ISC"
},
+ "node_modules/ws": {
+ "version": "8.19.0",
+ "resolved": "https://registry.npmjs.org/ws/-/ws-8.19.0.tgz",
+ "integrity": "sha512-blAT2mjOEIi0ZzruJfIhb3nps74PRWTCz1IjglWEEpQl5XS/UNama6u2/rjFkDDouqr4L67ry+1aGIALViWjDg==",
+ "license": "MIT",
+ "engines": {
+ "node": ">=10.0.0"
+ },
+ "peerDependencies": {
+ "bufferutil": "^4.0.1",
+ "utf-8-validate": ">=5.0.2"
+ },
+ "peerDependenciesMeta": {
+ "bufferutil": {
+ "optional": true
+ },
+ "utf-8-validate": {
+ "optional": true
+ }
+ }
+ },
"node_modules/xml2js": {
"version": "0.4.23",
"resolved": "https://registry.npmjs.org/xml2js/-/xml2js-0.4.23.tgz",
diff --git a/package.json b/package.json
index b4fb81784..1e95c59f2 100644
--- a/package.json
+++ b/package.json
@@ -15,25 +15,27 @@
"predev:main": "npm run compile:native",
"dev:main": "cross-env NODE_ENV=development node scripts/run-electron.js --dev",
"dev:renderer": "cd src && vite",
- "prebuild": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server",
+ "prebuild": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server && npm run download:sherpa-onnx",
"build": "cd src && vite build && cd .. && electron-builder",
"postinstall": "electron-builder install-app-deps",
"build:renderer": "cd src && vite build",
- "prebuild:mac": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server",
+ "prebuild:mac": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server && npm run download:sherpa-onnx",
"build:mac": "npm run build:renderer && electron-builder --mac",
- "prebuild:win": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server",
+ "prebuild:win": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server && npm run download:sherpa-onnx",
"build:win": "npm run build:renderer && electron-builder --win",
- "prebuild:linux": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server",
+ "prebuild:linux": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server && npm run download:sherpa-onnx",
"build:linux": "npm run build:renderer && electron-builder --linux",
"setup": "node setup.js",
- "prepack": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server",
+ "prepack": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server && npm run download:sherpa-onnx",
"pack": "npm run build:renderer && cross-env CSC_IDENTITY_AUTO_DISCOVERY=false electron-builder --dir",
- "predist": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server",
+ "predist": "npm run compile:native && npm run download:whisper-cpp && npm run download:llama-server && npm run download:sherpa-onnx",
"dist": "npm run build:renderer && electron-builder",
"download:whisper-cpp": "node scripts/download-whisper-cpp.js --current",
"download:whisper-cpp:all": "node scripts/download-whisper-cpp.js --all",
"download:llama-server": "node scripts/download-llama-server.js --current",
"download:llama-server:all": "node scripts/download-llama-server.js --all",
+ "download:sherpa-onnx": "node scripts/download-sherpa-onnx.js --current",
+ "download:sherpa-onnx:all": "node scripts/download-sherpa-onnx.js --all",
"format:js": "eslint . --fix && cd src && eslint . --fix && prettier --write \"**/*.{js,jsx,ts,tsx,json,css,md}\"",
"format": "npm run format:js",
"format:check": "eslint . && cd src && eslint . && prettier --check \"**/*.{js,jsx,ts,tsx,json,css,md}\"",
@@ -98,6 +100,7 @@
"tailwind-merge": "^3.3.1",
"tar": "^7.4.3",
"tw-animate-css": "^1.3.4",
- "unzipper": "^0.12.3"
+ "unzipper": "^0.12.3",
+ "ws": "^8.19.0"
}
}
diff --git a/preload.js b/preload.js
index 4ffbe8c99..46ca3decb 100644
--- a/preload.js
+++ b/preload.js
@@ -104,6 +104,29 @@ contextBridge.exposeInMainWorld("electronAPI", {
whisperServerStop: () => ipcRenderer.invoke("whisper-server-stop"),
whisperServerStatus: () => ipcRenderer.invoke("whisper-server-status"),
+ // Local Parakeet (NVIDIA) functions
+ transcribeLocalParakeet: (audioBlob, options) =>
+ ipcRenderer.invoke("transcribe-local-parakeet", audioBlob, options),
+ checkParakeetInstallation: () =>
+ ipcRenderer.invoke("check-parakeet-installation"),
+ downloadParakeetModel: (modelName) =>
+ ipcRenderer.invoke("download-parakeet-model", modelName),
+ onParakeetDownloadProgress: registerListener("parakeet-download-progress"),
+ checkParakeetModelStatus: (modelName) =>
+ ipcRenderer.invoke("check-parakeet-model-status", modelName),
+ listParakeetModels: () => ipcRenderer.invoke("list-parakeet-models"),
+ deleteParakeetModel: (modelName) =>
+ ipcRenderer.invoke("delete-parakeet-model", modelName),
+ deleteAllParakeetModels: () => ipcRenderer.invoke("delete-all-parakeet-models"),
+ cancelParakeetDownload: () => ipcRenderer.invoke("cancel-parakeet-download"),
+ getParakeetDiagnostics: () => ipcRenderer.invoke("get-parakeet-diagnostics"),
+
+ // Parakeet server functions (faster repeated transcriptions)
+ parakeetServerStart: (modelName) =>
+ ipcRenderer.invoke("parakeet-server-start", modelName),
+ parakeetServerStop: () => ipcRenderer.invoke("parakeet-server-stop"),
+ parakeetServerStatus: () => ipcRenderer.invoke("parakeet-server-status"),
+
// Window control functions
windowMinimize: () => ipcRenderer.invoke("window-minimize"),
windowMaximize: () => ipcRenderer.invoke("window-maximize"),
diff --git a/scripts/download-sherpa-onnx.js b/scripts/download-sherpa-onnx.js
new file mode 100644
index 000000000..063d448f9
--- /dev/null
+++ b/scripts/download-sherpa-onnx.js
@@ -0,0 +1,225 @@
+#!/usr/bin/env node
+const fs = require("fs");
+const path = require("path");
+const { execSync } = require("child_process");
+const {
+ downloadFile,
+ findBinaryInDir,
+ parseArgs,
+ setExecutable,
+ cleanupFiles,
+} = require("./lib/download-utils");
+
+const SHERPA_ONNX_VERSION = "1.12.23";
+const GITHUB_RELEASE_URL = `https://github.com/k2-fsa/sherpa-onnx/releases/download/v${SHERPA_ONNX_VERSION}`;
+
+// Binary configurations for each platform
+// Note: macOS uses universal2 builds that work on both arm64 and x64
+const BINARIES = {
+ "darwin-arm64": {
+ archiveName: `sherpa-onnx-v${SHERPA_ONNX_VERSION}-osx-universal2-shared.tar.bz2`,
+ binaryPath: "sherpa-onnx-offline-websocket-server",
+ outputName: "sherpa-onnx-ws-darwin-arm64",
+ libPattern: "*.dylib",
+ },
+ "darwin-x64": {
+ archiveName: `sherpa-onnx-v${SHERPA_ONNX_VERSION}-osx-universal2-shared.tar.bz2`,
+ binaryPath: "sherpa-onnx-offline-websocket-server",
+ outputName: "sherpa-onnx-ws-darwin-x64",
+ libPattern: "*.dylib",
+ },
+ "win32-x64": {
+ archiveName: `sherpa-onnx-v${SHERPA_ONNX_VERSION}-win-x64-shared.tar.bz2`,
+ binaryPath: "sherpa-onnx-offline-websocket-server.exe",
+ outputName: "sherpa-onnx-ws-win32-x64.exe",
+ libPattern: "*.dll",
+ },
+ "linux-x64": {
+ archiveName: `sherpa-onnx-v${SHERPA_ONNX_VERSION}-linux-x64-shared.tar.bz2`,
+ binaryPath: "sherpa-onnx-offline-websocket-server",
+ outputName: "sherpa-onnx-ws-linux-x64",
+ libPattern: "*.so*",
+ },
+};
+
+const BIN_DIR = path.join(__dirname, "..", "resources", "bin");
+
+function getDownloadUrl(archiveName) {
+ return `${GITHUB_RELEASE_URL}/${archiveName}`;
+}
+
+function extractTarBz2(archivePath, destDir) {
+ fs.mkdirSync(destDir, { recursive: true });
+ // tar is available on Windows 10+ and all Unix systems
+ execSync(`tar -xjf "${archivePath}" -C "${destDir}"`, { stdio: "inherit" });
+}
+
+function findLibrariesInDir(dir, pattern, maxDepth = 5, currentDepth = 0) {
+ if (currentDepth >= maxDepth) return [];
+
+ const results = [];
+ try {
+ const entries = fs.readdirSync(dir, { withFileTypes: true });
+
+ for (const entry of entries) {
+ const fullPath = path.join(dir, entry.name);
+
+ if (entry.isDirectory()) {
+ results.push(...findLibrariesInDir(fullPath, pattern, maxDepth, currentDepth + 1));
+ } else if (matchesPattern(entry.name, pattern)) {
+ results.push(fullPath);
+ }
+ }
+ } catch {
+ // Ignore permission errors
+ }
+
+ return results;
+}
+
+function matchesPattern(filename, pattern) {
+ if (pattern === "*.dylib") {
+ return filename.endsWith(".dylib");
+ } else if (pattern === "*.dll") {
+ return filename.endsWith(".dll");
+ } else if (pattern === "*.so*") {
+ return /\.so(\.\d+)*$/.test(filename) || filename.endsWith(".so");
+ }
+ return false;
+}
+
+async function downloadBinary(platformArch, config) {
+ if (!config) {
+ console.log(` ${platformArch}: Not supported`);
+ return false;
+ }
+
+ const outputPath = path.join(BIN_DIR, config.outputName);
+
+ if (fs.existsSync(outputPath)) {
+ console.log(` ${platformArch}: Already exists, skipping`);
+ return true;
+ }
+
+ const url = getDownloadUrl(config.archiveName);
+ console.log(` ${platformArch}: Downloading from ${url}`);
+
+ const archivePath = path.join(BIN_DIR, config.archiveName);
+
+ try {
+ await downloadFile(url, archivePath);
+
+ const extractDir = path.join(BIN_DIR, `temp-sherpa-${platformArch}`);
+ fs.mkdirSync(extractDir, { recursive: true });
+ extractTarBz2(archivePath, extractDir);
+
+ // Find the binary (may be in a subdirectory)
+ const binaryName = path.basename(config.binaryPath);
+ let binaryPath = findBinaryInDir(extractDir, binaryName);
+
+ if (binaryPath && fs.existsSync(binaryPath)) {
+ fs.copyFileSync(binaryPath, outputPath);
+ setExecutable(outputPath);
+ console.log(` ${platformArch}: Extracted to ${config.outputName}`);
+
+ // Copy shared libraries
+ if (config.libPattern) {
+ const libraries = findLibrariesInDir(extractDir, config.libPattern);
+
+ for (const libPath of libraries) {
+ const libName = path.basename(libPath);
+ const destPath = path.join(BIN_DIR, libName);
+
+ if (!fs.existsSync(destPath)) {
+ fs.copyFileSync(libPath, destPath);
+ setExecutable(destPath);
+ console.log(` ${platformArch}: Copied library ${libName}`);
+ }
+ }
+ }
+ } else {
+ console.error(` ${platformArch}: Binary '${binaryName}' not found in archive`);
+ return false;
+ }
+
+ // Cleanup
+ fs.rmSync(extractDir, { recursive: true, force: true });
+ if (fs.existsSync(archivePath)) fs.unlinkSync(archivePath);
+ return true;
+ } catch (error) {
+ console.error(` ${platformArch}: Failed - ${error.message}`);
+ if (fs.existsSync(archivePath)) fs.unlinkSync(archivePath);
+ return false;
+ }
+}
+
+async function main() {
+ console.log(`\nDownloading sherpa-onnx binaries (v${SHERPA_ONNX_VERSION})...\n`);
+
+ fs.mkdirSync(BIN_DIR, { recursive: true });
+
+ const args = parseArgs();
+
+ if (args.isCurrent) {
+ if (!BINARIES[args.platformArch]) {
+ console.error(`Unsupported platform/arch: ${args.platformArch}`);
+ process.exitCode = 1;
+ return;
+ }
+
+ console.log(`Downloading for target platform (${args.platformArch}):`);
+ const ok = await downloadBinary(args.platformArch, BINARIES[args.platformArch]);
+ if (!ok) {
+ console.error(`Failed to download binaries for ${args.platformArch}`);
+ process.exitCode = 1;
+ return;
+ }
+
+ // Remove old CLI-style binaries replaced by WS server binaries
+ const oldBinaryName =
+ args.platformArch.startsWith("win32")
+ ? `sherpa-onnx-${args.platformArch}.exe`
+ : `sherpa-onnx-${args.platformArch}`;
+ const oldBinaryPath = path.join(BIN_DIR, oldBinaryName);
+ if (fs.existsSync(oldBinaryPath)) {
+ console.log(` Removing old CLI binary: ${oldBinaryName}`);
+ fs.unlinkSync(oldBinaryPath);
+ }
+
+ if (args.shouldCleanup) {
+ cleanupFiles(BIN_DIR, "sherpa-onnx", `sherpa-onnx-ws-${args.platformArch}`);
+ }
+ } else {
+ console.log("Downloading binaries for all platforms:");
+ for (const platformArch of Object.keys(BINARIES)) {
+ await downloadBinary(platformArch, BINARIES[platformArch]);
+ }
+ }
+
+ console.log("\n---");
+
+ const files = fs.readdirSync(BIN_DIR).filter((f) => f.startsWith("sherpa-onnx"));
+ if (files.length > 0) {
+ console.log("Available sherpa-onnx binaries:\n");
+ files.forEach((f) => {
+ const stats = fs.statSync(path.join(BIN_DIR, f));
+ console.log(` - ${f} (${Math.round(stats.size / 1024 / 1024)}MB)`);
+ });
+ } else {
+ console.log("No binaries downloaded yet.");
+ console.log(`\nCheck: https://github.com/k2-fsa/sherpa-onnx/releases/tag/v${SHERPA_ONNX_VERSION}`);
+ }
+}
+
+// Export config for potential imports
+module.exports = {
+ SHERPA_ONNX_VERSION,
+ BINARIES,
+ BIN_DIR,
+ getDownloadUrl,
+};
+
+// Only run main() when executed directly
+if (require.main === module) {
+ main().catch(console.error);
+}
diff --git a/src/components/SettingsPage.tsx b/src/components/SettingsPage.tsx
index bcd37c71b..3a0112757 100644
--- a/src/components/SettingsPage.tsx
+++ b/src/components/SettingsPage.tsx
@@ -50,6 +50,8 @@ export default function SettingsPage({ activeSection = "general" }: SettingsPage
const {
useLocalWhisper,
whisperModel,
+ localTranscriptionProvider,
+ parakeetModel,
allowOpenAIFallback,
cloudTranscriptionProvider,
cloudTranscriptionModel,
@@ -72,6 +74,8 @@ export default function SettingsPage({ activeSection = "general" }: SettingsPage
setSelectedMicDeviceId,
setUseLocalWhisper,
setWhisperModel,
+ setLocalTranscriptionProvider,
+ setParakeetModel,
setAllowOpenAIFallback,
setCloudTranscriptionProvider,
setCloudTranscriptionModel,
@@ -755,8 +759,18 @@ export default function SettingsPage({ activeSection = "general" }: SettingsPage
onCloudProviderSelect={setCloudTranscriptionProvider}
selectedCloudModel={cloudTranscriptionModel}
onCloudModelSelect={setCloudTranscriptionModel}
- selectedLocalModel={whisperModel}
- onLocalModelSelect={setWhisperModel}
+ selectedLocalModel={
+ localTranscriptionProvider === "nvidia" ? parakeetModel : whisperModel
+ }
+ onLocalModelSelect={(modelId) => {
+ if (localTranscriptionProvider === "nvidia") {
+ setParakeetModel(modelId);
+ } else {
+ setWhisperModel(modelId);
+ }
+ }}
+ selectedLocalProvider={localTranscriptionProvider}
+ onLocalProviderSelect={setLocalTranscriptionProvider}
useLocalWhisper={useLocalWhisper}
onModeChange={(isLocal) => {
setUseLocalWhisper(isLocal);
diff --git a/src/components/TranscriptionModelPicker.tsx b/src/components/TranscriptionModelPicker.tsx
index 7a4b68e61..68d459ab5 100644
--- a/src/components/TranscriptionModelPicker.tsx
+++ b/src/components/TranscriptionModelPicker.tsx
@@ -14,18 +14,42 @@ import {
getTranscriptionProviders,
TranscriptionProviderData,
WHISPER_MODEL_INFO,
+ PARAKEET_MODEL_INFO,
} from "../models/ModelRegistry";
import { MODEL_PICKER_COLORS, type ColorScheme } from "../utils/modelPickerStyles";
import { getProviderIcon } from "../utils/providerIcons";
import { API_ENDPOINTS } from "../config/constants";
import { createExternalLinkHandler } from "../utils/externalLinks";
-interface WhisperModel {
+interface LocalModel {
model: string;
size_mb?: number;
downloaded?: boolean;
}
+interface LocalModelCardProps {
+ modelId: string;
+ name: string;
+ description: string;
+ size: string;
+ actualSizeMb?: number;
+ isSelected: boolean;
+ isDownloaded: boolean;
+ isDownloading: boolean;
+ isCancelling: boolean;
+ recommended?: boolean;
+ provider: string;
+ languageLabel?: string;
+ onSelect: () => void;
+ onDelete: () => void;
+ onDownload: () => void;
+ onCancel: () => void;
+ styles: ReturnType<(typeof MODEL_PICKER_COLORS)[keyof typeof MODEL_PICKER_COLORS]>;
+}
+
+// Backwards compatibility alias
+type WhisperModel = LocalModel;
+
interface TranscriptionModelPickerProps {
selectedCloudProvider: string;
onCloudProviderSelect: (providerId: string) => void;
@@ -59,7 +83,7 @@ const VALID_CLOUD_PROVIDER_IDS = CLOUD_PROVIDER_TABS.map((p) => p.id);
const LOCAL_PROVIDER_TABS = [
{ id: "whisper", name: "OpenAI Whisper" },
- { id: "nvidia", name: "Nvidia", disabled: true, badge: "Coming Soon" },
+ { id: "nvidia", name: "NVIDIA Parakeet" },
];
export default function TranscriptionModelPicker({
@@ -85,10 +109,14 @@ export default function TranscriptionModelPicker({
variant = "settings",
}: TranscriptionModelPickerProps) {
const [localModels, setLocalModels] = useState<WhisperModel[]>([]);
+ const [parakeetModels, setParakeetModels] = useState<WhisperModel[]>([]);
const [internalLocalProvider, setInternalLocalProvider] = useState(selectedLocalProvider);
const hasLoadedRef = useRef(false);
+ const hasLoadedParakeetRef = useRef(false);
const isLoadingRef = useRef(false);
+ const isLoadingParakeetRef = useRef(false);
const loadLocalModelsRef = useRef<(() => Promise<void>) | null>(null);
+ const loadParakeetModelsRef = useRef<(() => Promise<void>) | null>(null);
const ensureValidCloudSelectionRef = useRef<(() => void) | null>(null);
const selectedLocalModelRef = useRef(selectedLocalModel);
const onLocalModelSelectRef = useRef(onLocalModelSelect);
@@ -137,6 +165,23 @@ export default function TranscriptionModelPicker({
}
}, [validateAndSelectModel]);
+ const loadParakeetModels = useCallback(async () => {
+ if (isLoadingParakeetRef.current) return;
+ isLoadingParakeetRef.current = true;
+
+ try {
+ const result = await window.electronAPI?.listParakeetModels();
+ if (result?.success) {
+ setParakeetModels(result.models);
+ }
+ } catch (error) {
+ console.error("[TranscriptionModelPicker] Failed to load Parakeet models:", error);
+ setParakeetModels([]);
+ } finally {
+ isLoadingParakeetRef.current = false;
+ }
+ }, []);
+
const ensureValidCloudSelection = useCallback(() => {
const isValidProvider = VALID_CLOUD_PROVIDER_IDS.includes(selectedCloudProvider);
@@ -178,21 +223,28 @@ export default function TranscriptionModelPicker({
useEffect(() => {
loadLocalModelsRef.current = loadLocalModels;
}, [loadLocalModels]);
+ useEffect(() => {
+ loadParakeetModelsRef.current = loadParakeetModels;
+ }, [loadParakeetModels]);
useEffect(() => {
ensureValidCloudSelectionRef.current = ensureValidCloudSelection;
}, [ensureValidCloudSelection]);
useEffect(() => {
if (useLocalWhisper) {
- if (!hasLoadedRef.current) {
+ if (internalLocalProvider === "whisper" && !hasLoadedRef.current) {
hasLoadedRef.current = true;
loadLocalModelsRef.current?.();
+ } else if (internalLocalProvider === "nvidia" && !hasLoadedParakeetRef.current) {
+ hasLoadedParakeetRef.current = true;
+ loadParakeetModelsRef.current?.();
}
} else {
hasLoadedRef.current = false;
+ hasLoadedParakeetRef.current = false;
ensureValidCloudSelectionRef.current?.();
}
- }, [useLocalWhisper]);
+ }, [useLocalWhisper, internalLocalProvider]);
useEffect(() => {
const handleModelsCleared = () => loadLocalModels();
@@ -213,6 +265,19 @@ export default function TranscriptionModelPicker({
onDownloadComplete: loadLocalModels,
});
+ const {
+ downloadingModel: downloadingParakeetModel,
+ downloadProgress: parakeetDownloadProgress,
+ downloadModel: downloadParakeetModel,
+ deleteModel: deleteParakeetModel,
+ isDownloadingModel: isDownloadingParakeetModel,
+ cancelDownload: cancelParakeetDownload,
+ isCancelling: isCancellingParakeet,
+ } = useModelDownload({
+ modelType: "parakeet",
+ onDownloadComplete: loadParakeetModels,
+ });
+
const handleModeChange = useCallback(
(isLocal: boolean) => {
onModeChange(isLocal);
@@ -255,6 +320,25 @@ export default function TranscriptionModelPicker({
[onLocalProviderSelect]
);
+ // Wrapper to set both model and provider when selecting a local model
+ const handleWhisperModelSelect = useCallback(
+ (modelId: string) => {
+ onLocalProviderSelect?.("whisper");
+ setInternalLocalProvider("whisper");
+ onLocalModelSelect(modelId);
+ },
+ [onLocalModelSelect, onLocalProviderSelect]
+ );
+
+ const handleParakeetModelSelect = useCallback(
+ (modelId: string) => {
+ onLocalProviderSelect?.("nvidia");
+ setInternalLocalProvider("nvidia");
+ onLocalModelSelect(modelId);
+ },
+ [onLocalModelSelect, onLocalProviderSelect]
+ );
+
const handleBaseUrlBlur = useCallback(() => {
if (!setCloudTranscriptionBaseUrl || selectedCloudProvider !== "custom") return;
@@ -326,16 +410,134 @@ export default function TranscriptionModelPicker({
}, [currentCloudProvider, selectedCloudProvider]);
const progressDisplay = useMemo(() => {
- if (!downloadingModel || !useLocalWhisper) return null;
- const modelInfo = WHISPER_MODEL_INFO[downloadingModel];
- return (
- <DownloadProgressBar
- modelName={modelInfo?.name || downloadingModel}
- progress={downloadProgress}
- styles={styles}
- />
- );
- }, [downloadingModel, downloadProgress, useLocalWhisper, styles]);
+ if (!useLocalWhisper) return null;
+
+ if (downloadingModel && internalLocalProvider === "whisper") {
+ const modelInfo = WHISPER_MODEL_INFO[downloadingModel];
+ return (
+ <DownloadProgressBar
+ modelName={modelInfo?.name || downloadingModel}
+ progress={downloadProgress}
+ styles={styles}
+ />
+ );
+ }
+
+ if (downloadingParakeetModel && internalLocalProvider === "nvidia") {
+ const modelInfo = PARAKEET_MODEL_INFO[downloadingParakeetModel];
+ return (
+ <DownloadProgressBar
+ modelName={modelInfo?.name || downloadingParakeetModel}
+ progress={parakeetDownloadProgress}
+ styles={styles}
+ />
+ );
+ }
+
+ return null;
+ }, [
+ downloadingModel,
+ downloadProgress,
+ downloadingParakeetModel,
+ parakeetDownloadProgress,
+ useLocalWhisper,
+ internalLocalProvider,
+ styles,
+ ]);
+
+ // Shared component for rendering local model cards (Whisper and Parakeet)
+ const LocalModelCard = ({
+ modelId,
+ name,
+ description,
+ size,
+ actualSizeMb,
+ isSelected,
+ isDownloaded,
+ isDownloading,
+ isCancelling,
+ recommended,
+ provider,
+ languageLabel,
+ onSelect,
+ onDelete,
+ onDownload,
+ onCancel,
+ styles: cardStyles,
+ }: LocalModelCardProps) => (
+ <div
+ key={modelId}
+ className={`p-3 rounded-lg border-2 transition-all ${
+ isSelected ? cardStyles.modelCard.selected : cardStyles.modelCard.default
+ }`}
+ >
+ <div className="flex items-center justify-between">
+ <div className="flex-1">
+ <div className="flex items-center gap-2">
+ <ProviderIcon provider={provider} className="w-4 h-4" />
+ <span className="font-medium text-gray-900">{name}</span>
+ {isSelected && <span className={cardStyles.badges.selected}>βœ“ Selected</span>}
+ {recommended && <span className={cardStyles.badges.recommended}>Recommended</span>}
+ </div>
+ <div className="flex items-center gap-2 mt-1">
+ <span className="text-xs text-gray-600">{description}</span>
+ <span className="text-xs text-gray-500">
+ β€’ {actualSizeMb ? `${actualSizeMb}MB` : size}
+ </span>
+ {languageLabel && <span className="text-xs text-blue-600">{languageLabel}</span>}
+ {isDownloaded && (
+ <span className={cardStyles.badges.downloaded}>
+ <Check className="inline w-3 h-3 mr-1" />
+ Downloaded
+ </span>
+ )}
+ </div>
+ </div>
+
+ <div className="flex gap-2">
+ {isDownloaded ? (
+ <>
+ {!isSelected && (
+ <Button
+ onClick={onSelect}
+ size="sm"
+ variant="outline"
+ className={cardStyles.buttons.select}
+ >
+ Select
+ </Button>
+ )}
+ <Button
+ onClick={onDelete}
+ size="sm"
+ variant="outline"
+ className={cardStyles.buttons.delete}
+ >
+ <Trash2 size={14} />
+ <span className="ml-1">Delete</span>
+ </Button>
+ </>
+ ) : isDownloading ? (
+ <Button
+ onClick={onCancel}
+ disabled={isCancelling}
+ size="sm"
+ variant="outline"
+ className="text-red-600 border-red-300 hover:bg-red-50"
+ >
+ <X size={14} />
+ <span className="ml-1">{isCancelling ? "..." : "Cancel"}</span>
+ </Button>
+ ) : (
+ <Button onClick={onDownload} size="sm" className={cardStyles.buttons.download}>
+ <Download size={14} />
+ <span className="ml-1">Download</span>
+ </Button>
+ )}
+ </div>
+ </div>
+ </div>
+ );
const renderLocalModels = () => (
<div className="space-y-2">
@@ -346,93 +548,106 @@ export default function TranscriptionModelPicker({
description: "Model",
size: "Unknown",
};
- const isSelected = modelId === selectedLocalModel;
- const isDownloading = isDownloadingModel(modelId);
- const isDownloaded = model.downloaded;
return (
- <div
+ <LocalModelCard
key={modelId}
- className={`p-3 rounded-lg border-2 transition-all ${
- isSelected ? styles.modelCard.selected : styles.modelCard.default
- }`}
- >
- <div className="flex items-center justify-between">
- <div className="flex-1">
- <div className="flex items-center gap-2">
- <ProviderIcon provider="whisper" className="w-4 h-4" />
- <span className="font-medium text-gray-900">{info.name}</span>
- {isSelected && <span className={styles.badges.selected}>βœ“ Selected</span>}
- {info.recommended && (
- <span className={styles.badges.recommended}>Recommended</span>
- )}
- </div>
- <div className="flex items-center gap-2 mt-1">
- <span className="text-xs text-gray-600">{info.description}</span>
- <span className="text-xs text-gray-500">
- β€’ {model.size_mb ? `${model.size_mb}MB` : info.size}
- </span>
- {isDownloaded && (
- <span className={styles.badges.downloaded}>
- <Check className="inline w-3 h-3 mr-1" />
- Downloaded
- </span>
- )}
- </div>
- </div>
-
- <div className="flex gap-2">
- {isDownloaded ? (
- <>
- {!isSelected && (
- <Button
- onClick={() => onLocalModelSelect(modelId)}
- size="sm"
- variant="outline"
- className={styles.buttons.select}
- >
- Select
- </Button>
- )}
- <Button
- onClick={() => handleDelete(modelId)}
- size="sm"
- variant="outline"
- className={styles.buttons.delete}
- >
- <Trash2 size={14} />
- <span className="ml-1">Delete</span>
- </Button>
- </>
- ) : isDownloading ? (
- <Button
- onClick={cancelDownload}
- disabled={isCancelling}
- size="sm"
- variant="outline"
- className="text-red-600 border-red-300 hover:bg-red-50"
- >
- <X size={14} />
- <span className="ml-1">{isCancelling ? "..." : "Cancel"}</span>
- </Button>
- ) : (
- <Button
- onClick={() => downloadModel(modelId, onLocalModelSelect)}
- size="sm"
- className={styles.buttons.download}
- >
- <Download size={14} />
- <span className="ml-1">Download</span>
- </Button>
- )}
- </div>
- </div>
- </div>
+ modelId={modelId}
+ name={info.name}
+ description={info.description}
+ size={info.size}
+ actualSizeMb={model.size_mb}
+ isSelected={modelId === selectedLocalModel}
+ isDownloaded={model.downloaded ?? false}
+ isDownloading={isDownloadingModel(modelId)}
+ isCancelling={isCancelling}
+ recommended={info.recommended}
+ provider="whisper"
+ onSelect={() => handleWhisperModelSelect(modelId)}
+ onDelete={() => handleDelete(modelId)}
+ onDownload={() => downloadModel(modelId, handleWhisperModelSelect)}
+ onCancel={cancelDownload}
+ styles={styles}
+ />
);
})}
</div>
);
+ const handleParakeetDelete = useCallback(
+ (modelId: string) => {
+ showConfirmDialog({
+ title: "Delete Model",
+ description:
+ "Are you sure you want to delete this model? You'll need to re-download it if you want to use it again.",
+ onConfirm: async () => {
+ await deleteParakeetModel(modelId, async () => {
+ const result = await window.electronAPI?.listParakeetModels();
+ if (result?.success) {
+ setParakeetModels(result.models);
+ }
+ });
+ },
+ variant: "destructive",
+ });
+ },
+ [showConfirmDialog, deleteParakeetModel]
+ );
+
+ // Helper to get language label for Parakeet models
+ const getParakeetLanguageLabel = (language: string) => {
+ return language === "multilingual" ? "25 languages" : "English";
+ };
+
+ const renderParakeetModels = () => {
+ // When no models are loaded yet, show all available models from registry
+ const modelsToRender =
+ parakeetModels.length === 0
+ ? Object.entries(PARAKEET_MODEL_INFO).map(([modelId, info]) => ({
+ model: modelId,
+ downloaded: false,
+ size_mb: info.sizeMb,
+ }))
+ : parakeetModels;
+
+ return (
+ <div className="space-y-2">
+ {modelsToRender.map((model) => {
+ const modelId = model.model;
+ const info = PARAKEET_MODEL_INFO[modelId] || {
+ name: modelId,
+ description: "NVIDIA Parakeet Model",
+ size: "Unknown",
+ language: "en",
+ };
+
+ return (
+ <LocalModelCard
+ key={modelId}
+ modelId={modelId}
+ name={info.name}
+ description={info.description}
+ size={info.size}
+ actualSizeMb={model.size_mb}
+ isSelected={modelId === selectedLocalModel}
+ isDownloaded={model.downloaded ?? false}
+ isDownloading={isDownloadingParakeetModel(modelId)}
+ isCancelling={isCancellingParakeet}
+ recommended={info.recommended}
+ provider="nvidia"
+ languageLabel={getParakeetLanguageLabel(info.language)}
+ onSelect={() => handleParakeetModelSelect(modelId)}
+ onDelete={() => handleParakeetDelete(modelId)}
+ onDownload={() => downloadParakeetModel(modelId, handleParakeetModelSelect)}
+ onCancel={cancelParakeetDownload}
+ styles={styles}
+ />
+ );
+ })}
+ </div>
+ );
+ };
+
const renderLocalProviderTab = (
provider: (typeof LOCAL_PROVIDER_TABS)[0],
isSelected: boolean
@@ -652,9 +867,7 @@ export default function TranscriptionModelPicker({
<h5 className={`${styles.header} mb-3`}>Available Models</h5>
{internalLocalProvider === "whisper" && renderLocalModels()}
- {internalLocalProvider === "nvidia" && (
- <p className="text-sm text-gray-500">Nvidia GPU acceleration coming soon.</p>
- )}
+ {internalLocalProvider === "nvidia" && renderParakeetModels()}
</div>
</div>
)}
diff --git a/src/helpers/audioManager.js b/src/helpers/audioManager.js
index 5790857e9..4ad7c38b2 100644
--- a/src/helpers/audioManager.js
+++ b/src/helpers/audioManager.js
@@ -221,12 +221,22 @@ class AudioManager {
try {
const useLocalWhisper = localStorage.getItem("useLocalWhisper") === "true";
+ const localProvider = localStorage.getItem("localTranscriptionProvider") || "whisper";
const whisperModel = localStorage.getItem("whisperModel") || "base";
+ const parakeetModel = localStorage.getItem("parakeetModel") || "parakeet-tdt-0.6b-v3";
let result;
+ let activeModel;
if (useLocalWhisper) {
- result = await this.processWithLocalWhisper(audioBlob, whisperModel, metadata);
+ if (localProvider === "nvidia") {
+ activeModel = parakeetModel;
+ result = await this.processWithLocalParakeet(audioBlob, parakeetModel, metadata);
+ } else {
+ activeModel = whisperModel;
+ result = await this.processWithLocalWhisper(audioBlob, whisperModel, metadata);
+ }
} else {
+ activeModel = this.getTranscriptionModel();
result = await this.processWithOpenAIAPI(audioBlob, metadata);
}
@@ -235,8 +245,8 @@ class AudioManager {
const roundTripDurationMs = Math.round(performance.now() - pipelineStart);
const timingData = {
- mode: useLocalWhisper ? "local" : "cloud",
- model: useLocalWhisper ? whisperModel : this.getTranscriptionModel(),
+ mode: useLocalWhisper ? `local-${localProvider}` : "cloud",
+ model: activeModel,
audioDurationMs: metadata.durationSeconds
? Math.round(metadata.durationSeconds * 1000)
: null,
@@ -359,6 +369,80 @@ class AudioManager {
}
}
+ async processWithLocalParakeet(audioBlob, model = "parakeet-tdt-0.6b-v3", metadata = {}) {
+ const timings = {};
+
+ try {
+ const arrayBuffer = await audioBlob.arrayBuffer();
+ const language = localStorage.getItem("preferredLanguage");
+ const options = { model };
+ if (language && language !== "auto") {
+ options.language = language;
+ }
+
+ logger.debug(
+ "Parakeet transcription starting",
+ {
+ audioFormat: audioBlob.type,
+ audioSizeBytes: audioBlob.size,
+ model,
+ },
+ "performance"
+ );
+
+ const transcriptionStart = performance.now();
+ const result = await window.electronAPI.transcribeLocalParakeet(arrayBuffer, options);
+ timings.transcriptionProcessingDurationMs = Math.round(
+ performance.now() - transcriptionStart
+ );
+
+ logger.debug(
+ "Parakeet transcription complete",
+ {
+ transcriptionProcessingDurationMs: timings.transcriptionProcessingDurationMs,
+ success: result.success,
+ },
+ "performance"
+ );
+
+ if (result.success && result.text) {
+ const reasoningStart = performance.now();
+ const text = await this.processTranscription(result.text, "local-parakeet");
+ timings.reasoningProcessingDurationMs = Math.round(performance.now() - reasoningStart);
+
+ if (text !== null && text !== undefined) {
+ return { success: true, text: text || result.text, source: "local-parakeet", timings };
+ } else {
+ throw new Error("No text transcribed");
+ }
+ } else if (result.success === false && result.message === "No audio detected") {
+ throw new Error("No audio detected");
+ } else {
+ throw new Error(result.message || result.error || "Parakeet transcription failed");
+ }
+ } catch (error) {
+ if (error.message === "No audio detected") {
+ throw error;
+ }
+
+ const allowOpenAIFallback = localStorage.getItem("allowOpenAIFallback") === "true";
+ const isLocalMode = localStorage.getItem("useLocalWhisper") === "true";
+
+ if (allowOpenAIFallback && isLocalMode) {
+ try {
+ const fallbackResult = await this.processWithOpenAIAPI(audioBlob, metadata);
+ return { ...fallbackResult, source: "openai-fallback" };
+ } catch (fallbackError) {
+ throw new Error(
+ `Parakeet failed: ${error.message}. OpenAI fallback also failed: ${fallbackError.message}`
+ );
+ }
+ } else {
+ throw new Error(`Parakeet failed: ${error.message}`);
+ }
+ }
+ }
+
async getAPIKey() {
// Get the current transcription provider
const provider =
diff --git a/src/helpers/debugLogger.js b/src/helpers/debugLogger.js
index 17636f98e..5c50d9b6b 100644
--- a/src/helpers/debugLogger.js
+++ b/src/helpers/debugLogger.js
@@ -383,6 +383,11 @@ class DebugLogger {
this.debug(`Whisper Pipeline - ${stage}`, details, "whisper");
}
+ logSTTPipeline(stage, details) {
+ if (!this.isDebugEnabled()) return;
+ this.debug(`STT Pipeline - ${stage}`, details, "stt");
+ }
+
getLogPath() {
return this.logFile;
}
diff --git a/src/helpers/environment.js b/src/helpers/environment.js
index eea5e8c0a..d6492a155 100644
--- a/src/helpers/environment.js
+++ b/src/helpers/environment.js
@@ -132,6 +132,12 @@ OPENAI_API_KEY=${apiKey}
if (process.env.CUSTOM_REASONING_API_KEY) {
envContent += `CUSTOM_REASONING_API_KEY=${process.env.CUSTOM_REASONING_API_KEY}\n`;
}
+ if (process.env.LOCAL_TRANSCRIPTION_PROVIDER) {
+ envContent += `LOCAL_TRANSCRIPTION_PROVIDER=${process.env.LOCAL_TRANSCRIPTION_PROVIDER}\n`;
+ }
+ if (process.env.PARAKEET_MODEL) {
+ envContent += `PARAKEET_MODEL=${process.env.PARAKEET_MODEL}\n`;
+ }
fs.writeFileSync(envPath, envContent, "utf8");
diff --git a/src/helpers/ffmpegUtils.js b/src/helpers/ffmpegUtils.js
new file mode 100644
index 000000000..f72e34109
--- /dev/null
+++ b/src/helpers/ffmpegUtils.js
@@ -0,0 +1,259 @@
+const { spawn } = require("child_process");
+const fs = require("fs");
+const path = require("path");
+const debugLogger = require("./debugLogger");
+
+let cachedFFmpegPath = null;
+
+/**
+ * Get the path to FFmpeg binary, checking bundled and system locations
+ * @returns {string|null} Path to FFmpeg binary or null if not found
+ */
+function getFFmpegPath() {
+ if (cachedFFmpegPath) return cachedFFmpegPath;
+
+ try {
+ let ffmpegPath = require("ffmpeg-static");
+ ffmpegPath = path.normalize(ffmpegPath);
+
+ if (process.platform === "win32" && !ffmpegPath.endsWith(".exe")) {
+ ffmpegPath += ".exe";
+ }
+
+ // Try unpacked ASAR path first (production builds unpack ffmpeg-static)
+ const unpackedPath = ffmpegPath.includes("app.asar")
+ ? ffmpegPath.replace(/app\.asar([/\\])/, "app.asar.unpacked$1")
+ : null;
+
+ if (unpackedPath && fs.existsSync(unpackedPath)) {
+ if (process.platform !== "win32") {
+ try {
+ fs.accessSync(unpackedPath, fs.constants.X_OK);
+ } catch {
+ try {
+ fs.chmodSync(unpackedPath, 0o755);
+ } catch (chmodErr) {
+ debugLogger.warn("Failed to chmod FFmpeg", { error: chmodErr.message });
+ }
+ }
+ }
+ cachedFFmpegPath = unpackedPath;
+ return unpackedPath;
+ }
+
+ // Try original path (development or if not in ASAR)
+ if (fs.existsSync(ffmpegPath)) {
+ if (process.platform !== "win32") {
+ try {
+ fs.accessSync(ffmpegPath, fs.constants.X_OK);
+ } catch {
+ debugLogger.debug("FFmpeg exists but not executable", { ffmpegPath });
+ throw new Error("Not executable");
+ }
+ }
+ cachedFFmpegPath = ffmpegPath;
+ return ffmpegPath;
+ }
+ } catch (err) {
+ debugLogger.debug("Bundled FFmpeg not available", { error: err.message });
+ }
+
+ // Try system FFmpeg locations
+ const systemCandidates =
+ process.platform === "darwin"
+ ? ["/opt/homebrew/bin/ffmpeg", "/usr/local/bin/ffmpeg"]
+ : process.platform === "win32"
+ ? ["C:\\ffmpeg\\bin\\ffmpeg.exe"]
+ : ["/usr/bin/ffmpeg", "/usr/local/bin/ffmpeg"];
+
+ for (const candidate of systemCandidates) {
+ if (fs.existsSync(candidate)) {
+ cachedFFmpegPath = candidate;
+ return candidate;
+ }
+ }
+
+ // Search PATH environment
+ const pathEnv = process.env.PATH || "";
+ const pathSep = process.platform === "win32" ? ";" : ":";
+ const pathDirs = pathEnv.split(pathSep).map((entry) => entry.replace(/^"|"$/g, ""));
+ const pathBinary = process.platform === "win32" ? "ffmpeg.exe" : "ffmpeg";
+
+ for (const dir of pathDirs) {
+ if (!dir) continue;
+ const candidate = path.join(dir, pathBinary);
+ if (!fs.existsSync(candidate)) continue;
+ if (process.platform !== "win32") {
+ try {
+ fs.accessSync(candidate, fs.constants.X_OK);
+ } catch {
+ continue;
+ }
+ }
+ cachedFFmpegPath = candidate;
+ return candidate;
+ }
+
+ debugLogger.debug("FFmpeg not found");
+ return null;
+}
+
+/**
+ * Check if a buffer contains WAV audio format by examining the RIFF header
+ * @param {Buffer} buffer - Audio buffer to check
+ * @returns {boolean} True if buffer is WAV format
+ */
+function isWavFormat(buffer) {
+ if (!buffer || buffer.length < 12) return false;
+
+ // Check for RIFF header and WAVE format
+ return (
+ buffer[0] === 0x52 && // R
+ buffer[1] === 0x49 && // I
+ buffer[2] === 0x46 && // F
+ buffer[3] === 0x46 && // F
+ buffer[8] === 0x57 && // W
+ buffer[9] === 0x41 && // A
+ buffer[10] === 0x56 && // V
+ buffer[11] === 0x45 // E
+ );
+}
+
+/**
+ * Convert audio file to 16kHz mono WAV format using FFmpeg
+ * @param {string} inputPath - Path to input audio file
+ * @param {string} outputPath - Path to output WAV file
+ * @param {Object} options - Conversion options
+ * @param {number} options.sampleRate - Output sample rate (default: 16000)
+ * @param {number} options.channels - Output channels (default: 1 for mono)
+ * @returns {Promise<void>}
+ */
+function convertToWav(inputPath, outputPath, options = {}) {
+ const { sampleRate = 16000, channels = 1 } = options;
+
+ return new Promise((resolve, reject) => {
+ const ffmpegPath = getFFmpegPath();
+ if (!ffmpegPath) {
+ reject(new Error("FFmpeg not found - required for audio conversion"));
+ return;
+ }
+
+ const args = [
+ "-i",
+ inputPath,
+ "-ar",
+ String(sampleRate),
+ "-ac",
+ String(channels),
+ "-c:a",
+ "pcm_s16le",
+ "-y", // Overwrite output file
+ outputPath,
+ ];
+
+ debugLogger.debug("Converting audio with FFmpeg", {
+ input: inputPath,
+ output: outputPath,
+ sampleRate,
+ channels,
+ });
+
+ const proc = spawn(ffmpegPath, args, {
+ stdio: ["ignore", "pipe", "pipe"],
+ windowsHide: true,
+ });
+
+ let stderr = "";
+
+ proc.stderr.on("data", (data) => {
+ stderr += data.toString();
+ });
+
+ proc.on("error", (error) => {
+ reject(new Error(`FFmpeg process error: ${error.message}`));
+ });
+
+ proc.on("close", (code) => {
+ if (code !== 0) {
+ const stderrPreview = stderr.slice(-500).trim();
+ debugLogger.debug("FFmpeg conversion failed", { code, stderr: stderrPreview });
+ reject(
+ new Error(`FFmpeg exited with code ${code}${stderrPreview ? `: ${stderrPreview}` : ""}`)
+ );
+ return;
+ }
+
+ // Verify output file exists and has content
+ if (!fs.existsSync(outputPath)) {
+ reject(new Error("FFmpeg conversion produced no output file"));
+ return;
+ }
+
+ const stats = fs.statSync(outputPath);
+ if (stats.size === 0) {
+ reject(new Error("FFmpeg conversion produced empty output file"));
+ return;
+ }
+
+ debugLogger.debug("FFmpeg conversion complete", { outputSize: stats.size });
+ resolve();
+ });
+ });
+}
+
+function wavToFloat32Samples(wavBuffer) {
+ if (!isWavFormat(wavBuffer)) {
+ throw new Error("Buffer is not a valid WAV file");
+ }
+
+ // Parse WAV header to find data chunk
+ let offset = 12; // Skip RIFF header (4) + size (4) + WAVE (4)
+ let dataOffset = -1;
+ let dataSize = 0;
+ let bitsPerSample = 16;
+
+ while (offset < wavBuffer.length - 8) {
+ const chunkId = wavBuffer.toString("ascii", offset, offset + 4);
+ const chunkSize = wavBuffer.readUInt32LE(offset + 4);
+
+ if (chunkId === "fmt ") {
+ bitsPerSample = wavBuffer.readUInt16LE(offset + 22);
+ } else if (chunkId === "data") {
+ dataOffset = offset + 8;
+ dataSize = chunkSize;
+ break;
+ }
+
+ offset += 8 + chunkSize;
+ }
+
+ if (dataOffset < 0) {
+ throw new Error("WAV data chunk not found");
+ }
+
+ const bytesPerSample = bitsPerSample / 8;
+ const numSamples = Math.floor(dataSize / bytesPerSample);
+ const float32 = Buffer.alloc(numSamples * 4);
+
+ for (let i = 0; i < numSamples; i++) {
+ const sampleOffset = dataOffset + i * bytesPerSample;
+ const intVal =
+ bitsPerSample === 16 ? wavBuffer.readInt16LE(sampleOffset) : wavBuffer.readInt8(sampleOffset);
+ const maxVal = bitsPerSample === 16 ? 32768 : 128;
+ float32.writeFloatLE(intVal / maxVal, i * 4);
+ }
+
+ return float32;
+}
+
+function clearCache() {
+ cachedFFmpegPath = null;
+}
+
+module.exports = {
+ getFFmpegPath,
+ isWavFormat,
+ convertToWav,
+ wavToFloat32Samples,
+ clearCache,
+};
diff --git a/src/helpers/globeKeyManager.js b/src/helpers/globeKeyManager.js
index c890e7aed..08d721e0a 100644
--- a/src/helpers/globeKeyManager.js
+++ b/src/helpers/globeKeyManager.js
@@ -67,7 +67,7 @@ class GlobeKeyManager extends EventEmitter {
this.process.on("exit", (code, signal) => {
this.process = null;
- if (code !== 0) {
+ if (code !== 0 && signal !== "SIGINT" && signal !== "SIGTERM") {
const error = new Error(
`Globe key listener exited with code ${code ?? "null"} signal ${signal ?? "null"}`
);
diff --git a/src/helpers/hotkeyManager.js b/src/helpers/hotkeyManager.js
index bd610c243..712077e58 100644
--- a/src/helpers/hotkeyManager.js
+++ b/src/helpers/hotkeyManager.js
@@ -228,13 +228,7 @@ class HotkeyManager {
}
};
- if (mainWindow.webContents.isLoading()) {
- mainWindow.webContents.once("did-finish-load", () =>
- setTimeout(registerGnomeHotkey, HOTKEY_REGISTRATION_DELAY_MS)
- );
- } else {
- setTimeout(registerGnomeHotkey, HOTKEY_REGISTRATION_DELAY_MS);
- }
+ setTimeout(registerGnomeHotkey, HOTKEY_REGISTRATION_DELAY_MS);
this.isInitialized = true;
return;
}
@@ -244,11 +238,9 @@ class HotkeyManager {
globalShortcut.unregisterAll();
}
- mainWindow.webContents.once("did-finish-load", () => {
- setTimeout(() => {
- this.loadSavedHotkeyOrDefault(mainWindow, callback);
- }, HOTKEY_REGISTRATION_DELAY_MS);
- });
+ setTimeout(() => {
+ this.loadSavedHotkeyOrDefault(mainWindow, callback);
+ }, HOTKEY_REGISTRATION_DELAY_MS);
this.isInitialized = true;
}
@@ -258,7 +250,6 @@ class HotkeyManager {
const savedHotkey = await mainWindow.webContents.executeJavaScript(`
localStorage.getItem("dictationKey") || ""
`);
-
if (savedHotkey && savedHotkey.trim() !== "") {
const result = this.setupShortcuts(savedHotkey, callback);
if (result.success) {
@@ -294,7 +285,7 @@ class HotkeyManager {
const fallbackResult = this.setupShortcuts(fallback, callback);
if (fallbackResult.success) {
debugLogger.log(`[HotkeyManager] Fallback hotkey "${fallback}" registered successfully`);
- this.saveHotkeyToRenderer(fallback);
+ await this.saveHotkeyToRenderer(fallback);
this.notifyHotkeyFallback(defaultHotkey, fallback);
return;
}
@@ -308,17 +299,24 @@ class HotkeyManager {
}
}
- saveHotkeyToRenderer(hotkey) {
+ async saveHotkeyToRenderer(hotkey) {
+ // Escape the hotkey string to prevent injection issues
+ const escapedHotkey = hotkey.replace(/\\/g, "\\\\").replace(/"/g, '\\"');
+
if (this.mainWindow && !this.mainWindow.isDestroyed()) {
- this.mainWindow.webContents
- .executeJavaScript(
- `
- localStorage.setItem("dictationKey", "${hotkey}");
- `
- )
- .catch((err) => {
- debugLogger.error("[HotkeyManager] Failed to save hotkey to localStorage:", err.message);
- });
+ try {
+ await this.mainWindow.webContents.executeJavaScript(
+ `localStorage.setItem("dictationKey", "${escapedHotkey}"); true;`
+ );
+ debugLogger.log(`[HotkeyManager] Saved hotkey "${hotkey}" to localStorage`);
+ return true;
+ } catch (err) {
+ debugLogger.error("[HotkeyManager] Failed to save hotkey to localStorage:", err.message);
+ return false;
+ }
+ } else {
+ debugLogger.warn("[HotkeyManager] Main window not available for saving hotkey");
+ return false;
}
}
@@ -359,7 +357,12 @@ class HotkeyManager {
};
}
this.currentHotkey = hotkey;
- this.saveHotkeyToRenderer(hotkey);
+ const saved = await this.saveHotkeyToRenderer(hotkey);
+ if (!saved) {
+ debugLogger.warn(
+ "[HotkeyManager] GNOME hotkey registered but failed to persist to localStorage"
+ );
+ }
return {
success: true,
message: `Hotkey updated to: ${hotkey} (via GNOME native shortcut)`,
@@ -368,7 +371,12 @@ class HotkeyManager {
const result = this.setupShortcuts(hotkey, callback);
if (result.success) {
- this.saveHotkeyToRenderer(hotkey);
+ const saved = await this.saveHotkeyToRenderer(hotkey);
+ if (!saved) {
+ debugLogger.warn(
+ "[HotkeyManager] Hotkey registered but failed to persist to localStorage"
+ );
+ }
return { success: true, message: `Hotkey updated to: ${hotkey}` };
} else {
return {
diff --git a/src/helpers/ipcHandlers.js b/src/helpers/ipcHandlers.js
index ebc1a81a7..4cde850f9 100644
--- a/src/helpers/ipcHandlers.js
+++ b/src/helpers/ipcHandlers.js
@@ -11,6 +11,7 @@ class IPCHandlers {
this.databaseManager = managers.databaseManager;
this.clipboardManager = managers.clipboardManager;
this.whisperManager = managers.whisperManager;
+ this.parakeetManager = managers.parakeetManager;
this.windowManager = managers.windowManager;
this.updateManager = managers.updateManager;
this.windowsKeyManager = managers.windowsKeyManager;
@@ -307,6 +308,129 @@ class IPCHandlers {
return this.whisperManager.checkFFmpegAvailability();
});
+ // Parakeet (NVIDIA) handlers
+ ipcMain.handle("transcribe-local-parakeet", async (event, audioBlob, options = {}) => {
+ debugLogger.log("transcribe-local-parakeet called", {
+ audioBlobType: typeof audioBlob,
+ audioBlobSize: audioBlob?.byteLength || audioBlob?.length || 0,
+ options,
+ });
+
+ try {
+ const result = await this.parakeetManager.transcribeLocalParakeet(audioBlob, options);
+
+ debugLogger.log("Parakeet result", {
+ success: result.success,
+ hasText: !!result.text,
+ message: result.message,
+ error: result.error,
+ });
+
+ if (!result.success && result.message === "No audio detected") {
+ debugLogger.log("Sending no-audio-detected event to renderer");
+ event.sender.send("no-audio-detected");
+ }
+
+ return result;
+ } catch (error) {
+ debugLogger.error("Local Parakeet transcription error", error);
+ const errorMessage = error.message || "Unknown error";
+
+ if (errorMessage.includes("sherpa-onnx") && errorMessage.includes("not found")) {
+ return {
+ success: false,
+ error: "parakeet_not_found",
+ message: "Parakeet binary is missing. Please reinstall the app.",
+ };
+ }
+ if (errorMessage.includes("model") && errorMessage.includes("not downloaded")) {
+ return {
+ success: false,
+ error: "model_not_found",
+ message: errorMessage,
+ };
+ }
+
+ throw error;
+ }
+ });
+
+ ipcMain.handle("check-parakeet-installation", async () => {
+ return this.parakeetManager.checkInstallation();
+ });
+
+ ipcMain.handle("download-parakeet-model", async (event, modelName) => {
+ try {
+ const result = await this.parakeetManager.downloadParakeetModel(
+ modelName,
+ (progressData) => {
+ event.sender.send("parakeet-download-progress", progressData);
+ }
+ );
+
+ event.sender.send("parakeet-download-progress", {
+ type: "complete",
+ model: modelName,
+ result: result,
+ });
+
+ return result;
+ } catch (error) {
+ event.sender.send("parakeet-download-progress", {
+ type: "error",
+ model: modelName,
+ error: error.message,
+ });
+
+ throw error;
+ }
+ });
+
+ ipcMain.handle("check-parakeet-model-status", async (_event, modelName) => {
+ return this.parakeetManager.checkModelStatus(modelName);
+ });
+
+ ipcMain.handle("list-parakeet-models", async () => {
+ return this.parakeetManager.listParakeetModels();
+ });
+
+ ipcMain.handle("delete-parakeet-model", async (_event, modelName) => {
+ return this.parakeetManager.deleteParakeetModel(modelName);
+ });
+
+ ipcMain.handle("delete-all-parakeet-models", async () => {
+ return this.parakeetManager.deleteAllParakeetModels();
+ });
+
+ ipcMain.handle("cancel-parakeet-download", async () => {
+ return this.parakeetManager.cancelDownload();
+ });
+
+ ipcMain.handle("get-parakeet-diagnostics", async () => {
+ return this.parakeetManager.getDiagnostics();
+ });
+
+ // Parakeet server handlers (for faster repeated transcriptions)
+ ipcMain.handle("parakeet-server-start", async (event, modelName) => {
+ const result = await this.parakeetManager.startServer(modelName);
+ process.env.LOCAL_TRANSCRIPTION_PROVIDER = "nvidia";
+ process.env.PARAKEET_MODEL = modelName;
+ this.environmentManager.saveAllKeysToEnvFile();
+ return result;
+ });
+
+ ipcMain.handle("parakeet-server-stop", async () => {
+ const result = await this.parakeetManager.stopServer();
+ delete process.env.LOCAL_TRANSCRIPTION_PROVIDER;
+ delete process.env.PARAKEET_MODEL;
+ this.environmentManager.saveAllKeysToEnvFile();
+ return result;
+ });
+
+ ipcMain.handle("parakeet-server-status", async () => {
+ return this.parakeetManager.getServerStatus();
+ });
+
// Utility handlers
ipcMain.handle("cleanup-app", async (event) => {
try {
diff --git a/src/helpers/modelDirUtils.js b/src/helpers/modelDirUtils.js
new file mode 100644
index 000000000..b7136c9d2
--- /dev/null
+++ b/src/helpers/modelDirUtils.js
@@ -0,0 +1,10 @@
+const { app } = require("electron");
+const os = require("os");
+const path = require("path");
+
+function getModelsDirForService(service) {
+ const homeDir = app?.getPath?.("home") || os.homedir();
+ return path.join(homeDir, ".cache", "openwhispr", `${service}-models`);
+}
+
+module.exports = { getModelsDirForService };
diff --git a/src/helpers/parakeet.js b/src/helpers/parakeet.js
new file mode 100644
index 000000000..58b18917c
--- /dev/null
+++ b/src/helpers/parakeet.js
@@ -0,0 +1,675 @@
+const fs = require("fs");
+const fsPromises = require("fs").promises;
+const path = require("path");
+const https = require("https");
+const { spawn } = require("child_process");
+const debugLogger = require("./debugLogger");
+const ParakeetServerManager = require("./parakeetServer");
+const { getModelsDirForService } = require("./modelDirUtils");
+
+const modelRegistryData = require("../models/modelRegistryData.json");
+
+function getParakeetModelConfig(modelName) {
+ const modelInfo = modelRegistryData.parakeetModels[modelName];
+ if (!modelInfo) return null;
+ return {
+ url: modelInfo.downloadUrl,
+ size: modelInfo.sizeMb * 1_000_000,
+ language: modelInfo.language,
+ supportedLanguages: modelInfo.supportedLanguages || [],
+ extractDir: modelInfo.extractDir,
+ };
+}
+
+function getValidModelNames() {
+ return Object.keys(modelRegistryData.parakeetModels);
+}
+
+class ParakeetManager {
+ constructor() {
+ this.currentDownloadProcess = null;
+ this.isInitialized = false;
+ this.serverManager = new ParakeetServerManager();
+ }
+
+ getModelsDir() {
+ return getModelsDirForService("parakeet");
+ }
+
+ validateModelName(modelName) {
+ const validModels = getValidModelNames();
+ if (!validModels.includes(modelName)) {
+ throw new Error(
+ `Invalid Parakeet model: ${modelName}. Valid models: ${validModels.join(", ")}`
+ );
+ }
+ return true;
+ }
+
+ getModelPath(modelName) {
+ this.validateModelName(modelName);
+ return path.join(this.getModelsDir(), modelName);
+ }
+
+ async initializeAtStartup(settings = {}) {
+ const startTime = Date.now();
+
+ try {
+ this.isInitialized = true;
+
+ await this.logDependencyStatus();
+
+ const { localTranscriptionProvider, parakeetModel } = settings;
+
+ if (
+ localTranscriptionProvider === "nvidia" &&
+ parakeetModel &&
+ this.serverManager.isAvailable()
+ ) {
+ if (this.serverManager.isModelDownloaded(parakeetModel)) {
+ debugLogger.info("Pre-warming parakeet server", { model: parakeetModel });
+
+ try {
+ const serverStartTime = Date.now();
+ await this.serverManager.startServer(parakeetModel);
+ debugLogger.info("Parakeet server pre-warmed successfully", {
+ model: parakeetModel,
+ startupTimeMs: Date.now() - serverStartTime,
+ });
+ } catch (err) {
+ debugLogger.warn("Parakeet server pre-warm failed (will start on first use)", {
+ error: err.message,
+ model: parakeetModel,
+ });
+ }
+ } else {
+ debugLogger.debug("Skipping parakeet server pre-warm: model not downloaded", {
+ model: parakeetModel,
+ });
+ }
+ } else {
+ debugLogger.debug("Skipping parakeet server pre-warm", {
+ reason:
+ localTranscriptionProvider !== "nvidia"
+ ? "provider not nvidia"
+ : !parakeetModel
+ ? "no model selected"
+ : "server binary not available",
+ });
+ }
+ } catch (error) {
+ debugLogger.warn("Parakeet initialization error", { error: error.message });
+ this.isInitialized = true;
+ }
+
+ debugLogger.info("Parakeet initialization complete", {
+ totalTimeMs: Date.now() - startTime,
+ binaryAvailable: this.serverManager.isAvailable(),
+ });
+ }
+
+ async logDependencyStatus() {
+ const status = {
+ sherpaOnnx: {
+ available: this.serverManager.isAvailable(),
+ path: this.serverManager.getBinaryPath(),
+ },
+ models: [],
+ };
+
+ // Check downloaded models
+ for (const modelName of getValidModelNames()) {
+ const modelPath = this.getModelPath(modelName);
+ if (this.serverManager.isModelDownloaded(modelName)) {
+ try {
+ const encoderPath = path.join(modelPath, "encoder.int8.onnx");
+ const stats = fs.statSync(encoderPath);
+ status.models.push({
+ name: modelName,
+ size: `${Math.round(stats.size / (1024 * 1024))}MB`,
+ });
+ } catch {
+ // Skip if can't stat
+ }
+ }
+ }
+
+ debugLogger.info("Parakeet dependency check", status);
+
+ const binaryStatus = status.sherpaOnnx.available
+ ? `βœ“ ${status.sherpaOnnx.path}`
+ : "βœ— Not found";
+ const modelsStatus =
+ status.models.length > 0
+ ? status.models.map((m) => `${m.name}`).join(", ")
+ : "None downloaded";
+
+ debugLogger.info(`[Parakeet] sherpa-onnx: ${binaryStatus}`);
+ debugLogger.info(`[Parakeet] Models: ${modelsStatus}`);
+ }
+
+ async checkInstallation() {
+ const binaryPath = this.serverManager.getBinaryPath();
+ if (!binaryPath) {
+ return { installed: false, working: false };
+ }
+
+ return {
+ installed: true,
+ working: this.serverManager.isAvailable(),
+ path: binaryPath,
+ };
+ }
+
+ async startServer(modelName) {
+ this.validateModelName(modelName);
+ return this.serverManager.startServer(modelName);
+ }
+
+ async stopServer() {
+ await this.serverManager.stopServer();
+ }
+
+ getServerStatus() {
+ return this.serverManager.getServerStatus();
+ }
+
+ async transcribeLocalParakeet(audioBlob, options = {}) {
+ debugLogger.logSTTPipeline("transcribeLocalParakeet - start", {
+ options,
+ audioBlobType: audioBlob?.constructor?.name,
+ audioBlobSize: audioBlob?.byteLength || audioBlob?.size || 0,
+ serverAvailable: this.serverManager.isAvailable(),
+ });
+
+ if (!this.serverManager.isAvailable()) {
+ throw new Error(
+ "sherpa-onnx binary not found. Please ensure the app is installed correctly."
+ );
+ }
+
+ const model = options.model || "parakeet-tdt-0.6b-v3";
+
+ if (!this.serverManager.isModelDownloaded(model)) {
+ throw new Error(
+ `Parakeet model "${model}" not downloaded. Please download it from Settings.`
+ );
+ }
+
+ let audioBuffer;
+ if (Buffer.isBuffer(audioBlob)) {
+ audioBuffer = audioBlob;
+ } else if (ArrayBuffer.isView(audioBlob)) {
+ audioBuffer = Buffer.from(audioBlob.buffer, audioBlob.byteOffset, audioBlob.byteLength);
+ } else if (audioBlob instanceof ArrayBuffer) {
+ audioBuffer = Buffer.from(audioBlob);
+ } else if (typeof audioBlob === "string") {
+ audioBuffer = Buffer.from(audioBlob, "base64");
+ } else if (audioBlob && audioBlob.buffer && typeof audioBlob.byteLength === "number") {
+ audioBuffer = Buffer.from(audioBlob.buffer, audioBlob.byteOffset || 0, audioBlob.byteLength);
+ } else {
+ throw new Error(`Unsupported audio data type: ${typeof audioBlob}`);
+ }
+
+ if (!audioBuffer || audioBuffer.length === 0) {
+ throw new Error("Audio buffer is empty - no audio data received");
+ }
+
+ debugLogger.logSTTPipeline("transcribeLocalParakeet - processing", {
+ bufferSize: audioBuffer.length,
+ model,
+ });
+
+ const startTime = Date.now();
+ const language = options.language || "auto";
+ const result = await this.serverManager.transcribe(audioBuffer, { modelName: model, language });
+ const elapsed = Date.now() - startTime;
+
+ debugLogger.logSTTPipeline("transcribeLocalParakeet - completed", {
+ elapsed,
+ textLength: result.text?.length || 0,
+ });
+
+ return this.parseParakeetResult(result);
+ }
+
+ parseParakeetResult(output) {
+ debugLogger.debug("parseParakeetResult", {
+ hasOutput: !!output,
+ hasText: !!output?.text,
+ textLength: output?.text?.length || 0,
+ });
+
+ if (!output || !output.text) {
+ return { success: false, message: "No audio detected" };
+ }
+
+ const text = output.text.trim();
+
+ if (!text || text.length === 0) {
+ return { success: false, message: "No audio detected" };
+ }
+
+ return { success: true, text };
+ }
+
+ // Model management methods
+ async downloadParakeetModel(modelName, progressCallback = null) {
+ this.validateModelName(modelName);
+ const modelConfig = getParakeetModelConfig(modelName);
+
+ const modelPath = this.getModelPath(modelName);
+ const modelsDir = this.getModelsDir();
+
+ // Create models directory
+ await fsPromises.mkdir(modelsDir, { recursive: true });
+
+ // Check if already downloaded
+ if (this.serverManager.isModelDownloaded(modelName)) {
+ return {
+ model: modelName,
+ downloaded: true,
+ path: modelPath,
+ success: true,
+ };
+ }
+
+ const tempPath = path.join(modelsDir, `${modelName}.tar.bz2.tmp`);
+ const archivePath = path.join(modelsDir, `${modelName}.tar.bz2`);
+
+ let activeRequest = null;
+ let activeFile = null;
+ let isCancelled = false;
+
+ const cleanup = () => {
+ if (activeRequest) {
+ activeRequest.destroy();
+ activeRequest = null;
+ }
+ if (activeFile) {
+ activeFile.close();
+ activeFile = null;
+ }
+ try {
+ fs.unlinkSync(tempPath);
+ } catch {}
+ try {
+ fs.unlinkSync(archivePath);
+ } catch {}
+ };
+
+ this.currentDownloadProcess = {
+ abort: () => {
+ isCancelled = true;
+ cleanup();
+ },
+ };
+
+ return new Promise((resolve, reject) => {
+ const downloadWithRedirect = (url, redirectCount = 0) => {
+ if (isCancelled) {
+ reject(new Error("Download cancelled by user"));
+ return;
+ }
+
+ if (redirectCount > 5) {
+ cleanup();
+ reject(new Error("Too many redirects"));
+ return;
+ }
+
+ activeRequest = https.get(url, (response) => {
+ if (isCancelled) {
+ cleanup();
+ reject(new Error("Download cancelled by user"));
+ return;
+ }
+
+ if (response.statusCode === 302 || response.statusCode === 301) {
+ const redirectUrl = response.headers.location;
+ if (!redirectUrl) {
+ cleanup();
+ reject(new Error("Redirect without location header"));
+ return;
+ }
+ downloadWithRedirect(redirectUrl, redirectCount + 1);
+ return;
+ }
+
+ if (response.statusCode !== 200) {
+ cleanup();
+ reject(new Error(`Failed to download model: HTTP ${response.statusCode}`));
+ return;
+ }
+
+ const totalSize = parseInt(response.headers["content-length"], 10) || modelConfig.size;
+ let downloadedSize = 0;
+
+ activeFile = fs.createWriteStream(tempPath);
+
+ response.on("data", (chunk) => {
+ if (isCancelled) {
+ cleanup();
+ return;
+ }
+
+ downloadedSize += chunk.length;
+ const percentage = Math.round((downloadedSize / totalSize) * 100);
+
+ if (progressCallback) {
+ progressCallback({
+ type: "progress",
+ model: modelName,
+ downloaded_bytes: downloadedSize,
+ total_bytes: totalSize,
+ percentage,
+ });
+ }
+ });
+
+ response.pipe(activeFile);
+
+ activeFile.on("finish", async () => {
+ if (isCancelled) {
+ cleanup();
+ reject(new Error("Download cancelled by user"));
+ return;
+ }
+
+ activeFile.close();
+ activeFile = null;
+ this.currentDownloadProcess = null;
+
+ try {
+ await fsPromises.rename(tempPath, archivePath);
+
+ await this._extractModel(archivePath, modelName);
+
+ await fsPromises.unlink(archivePath);
+
+ if (progressCallback) {
+ progressCallback({
+ type: "complete",
+ model: modelName,
+ percentage: 100,
+ });
+ }
+
+ resolve({
+ model: modelName,
+ downloaded: true,
+ path: modelPath,
+ success: true,
+ });
+ } catch (extractError) {
+ cleanup();
+ reject(new Error(`Failed to extract model: ${extractError.message}`));
+ }
+ });
+
+ activeFile.on("error", (err) => {
+ cleanup();
+ reject(err);
+ });
+
+ response.on("error", (err) => {
+ cleanup();
+ reject(err);
+ });
+ });
+
+ activeRequest.on("error", (err) => {
+ cleanup();
+ reject(err);
+ });
+
+ activeRequest.setTimeout(600000, () => {
+ cleanup();
+ reject(new Error("Download request timed out"));
+ });
+ };
+
+ downloadWithRedirect(modelConfig.url);
+ });
+ }
+
+ async _extractModel(archivePath, modelName) {
+ const modelsDir = this.getModelsDir();
+ const modelConfig = getParakeetModelConfig(modelName);
+ const extractDir = path.join(modelsDir, `temp-extract-${modelName}`);
+
+ try {
+ // Create temp extract directory
+ await fsPromises.mkdir(extractDir, { recursive: true });
+
+ // Extract tar.bz2 asynchronously (tar is available on Windows 10+ and all Unix systems)
+ await this._runTarExtract(archivePath, extractDir);
+
+ // Find the extracted directory (usually has a specific name)
+ const extractedDir = path.join(extractDir, modelConfig.extractDir);
+ const targetDir = this.getModelPath(modelName);
+
+ // Move to final location
+ if (fs.existsSync(extractedDir)) {
+ // Remove target if exists
+ if (fs.existsSync(targetDir)) {
+ await fsPromises.rm(targetDir, { recursive: true, force: true });
+ }
+ await fsPromises.rename(extractedDir, targetDir);
+ } else {
+ // If extracted directory doesn't match expected name, try to find it
+ const entries = await fsPromises.readdir(extractDir);
+ let modelDir = null;
+
+ for (const entry of entries) {
+ const entryPath = path.join(extractDir, entry);
+ const stat = await fsPromises.stat(entryPath);
+ if (stat.isDirectory() && entry.includes("parakeet")) {
+ modelDir = entry;
+ break;
+ }
+ }
+
+ if (modelDir) {
+ if (fs.existsSync(targetDir)) {
+ await fsPromises.rm(targetDir, { recursive: true, force: true });
+ }
+ await fsPromises.rename(path.join(extractDir, modelDir), targetDir);
+ } else {
+ throw new Error("Could not find model directory in extracted archive");
+ }
+ }
+
+ // Cleanup temp directory
+ await fsPromises.rm(extractDir, { recursive: true, force: true });
+
+ debugLogger.info("Parakeet model extracted", { modelName, targetDir });
+ } catch (error) {
+ // Cleanup on error
+ try {
+ await fsPromises.rm(extractDir, { recursive: true, force: true });
+ } catch {}
+ throw error;
+ }
+ }
+
+ _runTarExtract(archivePath, extractDir) {
+ return new Promise((resolve, reject) => {
+ const tarProcess = spawn("tar", ["-xjf", archivePath, "-C", extractDir], {
+ stdio: ["ignore", "pipe", "pipe"],
+ });
+
+ let stderr = "";
+
+ tarProcess.stderr.on("data", (data) => {
+ stderr += data.toString();
+ });
+
+ tarProcess.on("close", (code) => {
+ if (code === 0) {
+ resolve();
+ } else {
+ reject(new Error(`tar extraction failed with code ${code}: ${stderr}`));
+ }
+ });
+
+ tarProcess.on("error", (err) => {
+ reject(new Error(`Failed to start tar process: ${err.message}`));
+ });
+ });
+ }
+
+ async cancelDownload() {
+ if (this.currentDownloadProcess) {
+ this.currentDownloadProcess.abort();
+ this.currentDownloadProcess = null;
+ return { success: true, message: "Download cancelled" };
+ }
+ return { success: false, error: "No active download to cancel" };
+ }
+
+ async checkModelStatus(modelName) {
+ const modelPath = this.getModelPath(modelName);
+
+ if (this.serverManager.isModelDownloaded(modelName)) {
+ try {
+ const encoderPath = path.join(modelPath, "encoder.int8.onnx");
+ const stats = fs.statSync(encoderPath);
+ return {
+ model: modelName,
+ downloaded: true,
+ path: modelPath,
+ size_bytes: stats.size,
+ size_mb: Math.round(stats.size / (1024 * 1024)),
+ success: true,
+ };
+ } catch {
+ return { model: modelName, downloaded: false, success: true };
+ }
+ }
+
+ return { model: modelName, downloaded: false, success: true };
+ }
+
+ async listParakeetModels() {
+ const models = getValidModelNames();
+ const modelInfo = [];
+
+ for (const model of models) {
+ const status = await this.checkModelStatus(model);
+ modelInfo.push(status);
+ }
+
+ return {
+ models: modelInfo,
+ cache_dir: this.getModelsDir(),
+ success: true,
+ };
+ }
+
+ async deleteParakeetModel(modelName) {
+ const modelPath = this.getModelPath(modelName);
+
+ if (fs.existsSync(modelPath)) {
+ try {
+ const encoderPath = path.join(modelPath, "encoder.int8.onnx");
+ let freedBytes = 0;
+
+ if (fs.existsSync(encoderPath)) {
+ const stats = fs.statSync(encoderPath);
+ freedBytes = stats.size;
+ }
+
+ fs.rmSync(modelPath, { recursive: true, force: true });
+
+ return {
+ model: modelName,
+ deleted: true,
+ freed_bytes: freedBytes,
+ freed_mb: Math.round(freedBytes / (1024 * 1024)),
+ success: true,
+ };
+ } catch (error) {
+ return { model: modelName, deleted: false, error: error.message, success: false };
+ }
+ }
+
+ return { model: modelName, deleted: false, error: "Model not found", success: false };
+ }
+
+ async deleteAllParakeetModels() {
+ const modelsDir = this.getModelsDir();
+ let totalFreed = 0;
+ let deletedCount = 0;
+
+ try {
+ if (!fs.existsSync(modelsDir)) {
+ return { success: true, deleted_count: 0, freed_bytes: 0, freed_mb: 0 };
+ }
+
+ const entries = fs.readdirSync(modelsDir, { withFileTypes: true });
+ for (const entry of entries) {
+ if (entry.isDirectory()) {
+ const dirPath = path.join(modelsDir, entry.name);
+ try {
+ // Estimate size from encoder file
+ const encoderPath = path.join(dirPath, "encoder.int8.onnx");
+ if (fs.existsSync(encoderPath)) {
+ const stats = fs.statSync(encoderPath);
+ totalFreed += stats.size;
+ }
+
+ fs.rmSync(dirPath, { recursive: true, force: true });
+ deletedCount++;
+ } catch {
+ // Continue with other models if one fails
+ }
+ }
+ }
+
+ return {
+ success: true,
+ deleted_count: deletedCount,
+ freed_bytes: totalFreed,
+ freed_mb: Math.round(totalFreed / (1024 * 1024)),
+ };
+ } catch (error) {
+ return { success: false, error: error.message };
+ }
+ }
+
+ async getDiagnostics() {
+ const diagnostics = {
+ platform: process.platform,
+ arch: process.arch,
+ resourcesPath: process.resourcesPath || null,
+ isPackaged: !!process.resourcesPath && !process.resourcesPath.includes("node_modules"),
+ sherpaOnnx: { available: false, path: null },
+ modelsDir: this.getModelsDir(),
+ models: [],
+ };
+
+ // Check sherpa-onnx
+ const binaryPath = this.serverManager.getBinaryPath();
+ if (binaryPath) {
+ diagnostics.sherpaOnnx = { available: true, path: binaryPath };
+ }
+
+ // Check downloaded models
+ try {
+ const modelsDir = this.getModelsDir();
+ if (fs.existsSync(modelsDir)) {
+ const entries = fs.readdirSync(modelsDir, { withFileTypes: true });
+ diagnostics.models = entries
+ .filter((e) => e.isDirectory() && this.serverManager.isModelDownloaded(e.name))
+ .map((e) => e.name);
+ }
+ } catch {
+ // Ignore errors reading models dir
+ }
+
+ return diagnostics;
+ }
+}
+
+module.exports = ParakeetManager;
diff --git a/src/helpers/parakeetServer.js b/src/helpers/parakeetServer.js
new file mode 100644
index 000000000..28a71c10f
--- /dev/null
+++ b/src/helpers/parakeetServer.js
@@ -0,0 +1,155 @@
+const fs = require("fs");
+const os = require("os");
+const path = require("path");
+const debugLogger = require("./debugLogger");
+const { getModelsDirForService } = require("./modelDirUtils");
+const { getFFmpegPath, isWavFormat, convertToWav, wavToFloat32Samples } = require("./ffmpegUtils");
+const ParakeetWsServer = require("./parakeetWsServer");
+
+class ParakeetServerManager {
+ constructor() {
+ this.wsServer = new ParakeetWsServer();
+ }
+
+ getBinaryPath() {
+ return this.wsServer.getWsBinaryPath();
+ }
+
+ isAvailable() {
+ return this.wsServer.isAvailable();
+ }
+
+ getModelsDir() {
+ return getModelsDirForService("parakeet");
+ }
+
+ isModelDownloaded(modelName) {
+ const modelDir = path.join(this.getModelsDir(), modelName);
+ const requiredFiles = [
+ "encoder.int8.onnx",
+ "decoder.int8.onnx",
+ "joiner.int8.onnx",
+ "tokens.txt",
+ ];
+
+ if (!fs.existsSync(modelDir)) return false;
+
+ for (const file of requiredFiles) {
+ if (!fs.existsSync(path.join(modelDir, file))) {
+ return false;
+ }
+ }
+
+ return true;
+ }
+
+ async _ensureWav(audioBuffer) {
+ const isWav = isWavFormat(audioBuffer);
+ if (isWav) return { wavBuffer: audioBuffer, filesToCleanup: [] };
+
+ const ffmpegPath = getFFmpegPath();
+ if (!ffmpegPath) {
+ throw new Error(
+ "FFmpeg not found - required for audio conversion. Please ensure FFmpeg is installed."
+ );
+ }
+
+ const tempDir = os.tmpdir();
+ const timestamp = Date.now();
+ const tempInputPath = path.join(tempDir, `parakeet-input-${timestamp}.webm`);
+ const tempWavPath = path.join(tempDir, `parakeet-${timestamp}.wav`);
+
+ fs.writeFileSync(tempInputPath, audioBuffer);
+
+ const inputStats = fs.statSync(tempInputPath);
+ debugLogger.debug("Converting audio to WAV", { inputSize: inputStats.size });
+
+ await convertToWav(tempInputPath, tempWavPath, { sampleRate: 16000, channels: 1 });
+
+ const outputStats = fs.statSync(tempWavPath);
+ debugLogger.debug("FFmpeg conversion complete", { outputSize: outputStats.size });
+
+ const wavBuffer = fs.readFileSync(tempWavPath);
+ return { wavBuffer, filesToCleanup: [tempInputPath, tempWavPath] };
+ }
+
+ async transcribe(audioBuffer, options = {}) {
+ const { modelName = "parakeet-tdt-0.6b-v3", language = "auto" } = options;
+
+ const modelDir = path.join(this.getModelsDir(), modelName);
+ if (!this.isModelDownloaded(modelName)) {
+ throw new Error(`Parakeet model "${modelName}" not downloaded`);
+ }
+
+ debugLogger.debug("Parakeet transcription request", {
+ modelName,
+ language,
+ audioSize: audioBuffer?.length || 0,
+ isWavFormat: isWavFormat(audioBuffer),
+ });
+
+ const { wavBuffer, filesToCleanup } = await this._ensureWav(audioBuffer);
+ try {
+ if (!this.wsServer.ready || this.wsServer.modelName !== modelName) {
+ await this.wsServer.start(modelName, modelDir);
+ }
+ const samples = wavToFloat32Samples(wavBuffer);
+ const result = await this.wsServer.transcribe(samples, 16000);
+ return { ...result, language };
+ } finally {
+ this._cleanupFiles(filesToCleanup);
+ }
+ }
+
+ _cleanupFiles(filePaths) {
+ for (const filePath of filePaths) {
+ try {
+ if (fs.existsSync(filePath)) {
+ fs.unlinkSync(filePath);
+ }
+ } catch (err) {
+ debugLogger.warn("Failed to cleanup temp audio file", {
+ path: filePath,
+ error: err.message,
+ });
+ }
+ }
+ }
+
+ async startServer(modelName) {
+ if (!this.wsServer.isAvailable()) {
+ return { success: false, reason: "parakeet WS server binary not found" };
+ }
+
+ const modelDir = path.join(this.getModelsDir(), modelName);
+ if (!this.isModelDownloaded(modelName)) {
+ return { success: false, reason: `Model "${modelName}" not downloaded` };
+ }
+
+ try {
+ await this.wsServer.start(modelName, modelDir);
+ return { success: true, port: this.wsServer.port };
+ } catch (error) {
+ debugLogger.error("Failed to start parakeet WS server", { error: error.message });
+ return { success: false, reason: error.message };
+ }
+ }
+
+ async stopServer() {
+ await this.wsServer.stop();
+ }
+
+ getServerStatus() {
+ return this.wsServer.getStatus();
+ }
+
+ getStatus() {
+ return {
+ available: this.isAvailable(),
+ binaryPath: this.getBinaryPath(),
+ modelsDir: this.getModelsDir(),
+ };
+ }
+}
+
+module.exports = ParakeetServerManager;
diff --git a/src/helpers/parakeetWsServer.js b/src/helpers/parakeetWsServer.js
new file mode 100644
index 000000000..fd64948f0
--- /dev/null
+++ b/src/helpers/parakeetWsServer.js
@@ -0,0 +1,300 @@
+const { spawn } = require("child_process");
+const fs = require("fs");
+const os = require("os");
+const path = require("path");
+const WebSocket = require("ws");
+const debugLogger = require("./debugLogger");
+const {
+ findAvailablePort,
+ resolveBinaryPath,
+ gracefulStopProcess,
+} = require("../utils/serverUtils");
+
+const PORT_RANGE_START = 6006;
+const PORT_RANGE_END = 6029;
+const STARTUP_TIMEOUT_MS = 60000;
+const HEALTH_CHECK_INTERVAL_MS = 5000;
+const TRANSCRIPTION_TIMEOUT_MS = 300000;
+
+class ParakeetWsServer {
+ constructor() {
+ this.process = null;
+ this.port = null;
+ this.ready = false;
+ this.modelName = null;
+ this.modelDir = null;
+ this.startupPromise = null;
+ this.healthCheckInterval = null;
+ this.transcribing = false;
+ this.cachedWsBinaryPath = null;
+ }
+
+ getWsBinaryPath() {
+ if (this.cachedWsBinaryPath) return this.cachedWsBinaryPath;
+
+ const platformArch = `${process.platform}-${process.arch}`;
+ const binaryName =
+ process.platform === "win32"
+ ? `sherpa-onnx-ws-${platformArch}.exe`
+ : `sherpa-onnx-ws-${platformArch}`;
+
+ const resolved = resolveBinaryPath(binaryName);
+ if (resolved) this.cachedWsBinaryPath = resolved;
+ return resolved;
+ }
+
+ isAvailable() {
+ return this.getWsBinaryPath() !== null;
+ }
+
+ async start(modelName, modelDir) {
+ if (this.startupPromise) return this.startupPromise;
+ if (this.ready && this.modelName === modelName) return;
+ if (this.process) await this.stop();
+
+ this.startupPromise = this._doStart(modelName, modelDir);
+ try {
+ await this.startupPromise;
+ } finally {
+ this.startupPromise = null;
+ }
+ }
+
+ async _doStart(modelName, modelDir) {
+ const wsBinary = this.getWsBinaryPath();
+ if (!wsBinary) throw new Error("sherpa-onnx WS server binary not found");
+ if (!fs.existsSync(modelDir)) throw new Error(`Model directory not found: ${modelDir}`);
+
+ this.port = await findAvailablePort(PORT_RANGE_START, PORT_RANGE_END);
+ this.modelName = modelName;
+ this.modelDir = modelDir;
+
+ const args = [
+ `--tokens=${path.join(modelDir, "tokens.txt")}`,
+ `--encoder=${path.join(modelDir, "encoder.int8.onnx")}`,
+ `--decoder=${path.join(modelDir, "decoder.int8.onnx")}`,
+ `--joiner=${path.join(modelDir, "joiner.int8.onnx")}`,
+ `--port=${this.port}`,
+ `--num-threads=${Math.max(1, Math.floor(os.cpus().length * 0.75))}`,
+ ];
+
+ debugLogger.debug("Starting parakeet WS server", { port: this.port, modelName, args });
+
+ this.process = spawn(wsBinary, args, {
+ stdio: ["ignore", "pipe", "pipe"],
+ windowsHide: true,
+ cwd: os.tmpdir(),
+ });
+
+ let stderrBuffer = "";
+ let exitCode = null;
+ let readyResolve = null;
+ const readyFromStderr = new Promise((resolve) => {
+ readyResolve = resolve;
+ });
+
+ this.process.stdout.on("data", (data) => {
+ debugLogger.debug("parakeet-ws stdout", { data: data.toString().trim() });
+ });
+
+ this.process.stderr.on("data", (data) => {
+ stderrBuffer += data.toString();
+ debugLogger.debug("parakeet-ws stderr", { data: data.toString().trim() });
+ if (data.toString().includes("Listening on:")) {
+ readyResolve(true);
+ }
+ });
+
+ this.process.on("error", (error) => {
+ debugLogger.error("parakeet-ws process error", { error: error.message });
+ this.ready = false;
+ readyResolve(false);
+ });
+
+ this.process.on("close", (code) => {
+ exitCode = code;
+ debugLogger.debug("parakeet-ws process exited", { code });
+ this.ready = false;
+ this.process = null;
+ this.stopHealthCheck();
+ readyResolve(false);
+ });
+
+ await this._waitForReady(readyFromStderr, () => ({ stderr: stderrBuffer, exitCode }));
+ this._startHealthCheck();
+
+ debugLogger.info("parakeet-ws server started successfully", {
+ port: this.port,
+ model: modelName,
+ });
+ }
+
+ async _waitForReady(readySignal, getProcessInfo) {
+ const startTime = Date.now();
+
+ const timeoutPromise = new Promise((_, reject) => {
+ setTimeout(
+ () => reject(new Error(`parakeet-ws failed to start within ${STARTUP_TIMEOUT_MS}ms`)),
+ STARTUP_TIMEOUT_MS
+ );
+ });
+
+ const ready = await Promise.race([readySignal, timeoutPromise]);
+
+ if (!ready) {
+ const info = getProcessInfo ? getProcessInfo() : {};
+ const stderr = info.stderr ? info.stderr.trim().slice(0, 200) : "";
+ const details = stderr || (info.exitCode !== null ? `exit code: ${info.exitCode}` : "");
+ throw new Error(`parakeet-ws process died during startup${details ? `: ${details}` : ""}`);
+ }
+
+ this.ready = true;
+ debugLogger.debug("parakeet-ws ready", { startupTimeMs: Date.now() - startTime });
+ }
+
+ _isProcessAlive() {
+ if (!this.process || this.process.killed) return false;
+ try {
+ process.kill(this.process.pid, 0);
+ return true;
+ } catch {
+ return false;
+ }
+ }
+
+ _startHealthCheck() {
+ this.stopHealthCheck();
+ this.healthCheckInterval = setInterval(() => {
+ if (!this.process) {
+ this.stopHealthCheck();
+ return;
+ }
+ if (this.transcribing) return;
+
+ if (!this._isProcessAlive()) {
+ debugLogger.warn("parakeet-ws health check failed: process not alive");
+ this.ready = false;
+ this.stopHealthCheck();
+ }
+ }, HEALTH_CHECK_INTERVAL_MS);
+ }
+
+ stopHealthCheck() {
+ if (this.healthCheckInterval) {
+ clearInterval(this.healthCheckInterval);
+ this.healthCheckInterval = null;
+ }
+ }
+
+ transcribe(samplesBuffer, sampleRate) {
+ if (!this.ready || !this.process) {
+ throw new Error("parakeet-ws server is not running");
+ }
+
+ this.transcribing = true;
+
+ return new Promise((resolve, reject) => {
+ const startTime = Date.now();
+ let result = "";
+
+ const done =
+ (fn) =>
+ (...args) => {
+ this.transcribing = false;
+ fn(...args);
+ };
+
+ const timeout = setTimeout(() => {
+ try {
+ ws.close();
+ } catch {}
+ done(reject)(new Error("parakeet-ws transcription timed out"));
+ }, TRANSCRIPTION_TIMEOUT_MS);
+
+ const ws = new WebSocket(`ws://127.0.0.1:${this.port}`);
+
+ ws.on("open", () => {
+ // sherpa-onnx offline WS binary protocol:
+ // [int32LE sample_rate][int32LE num_audio_bytes][float32 samples...]
+ const message = Buffer.alloc(8 + samplesBuffer.length);
+ message.writeInt32LE(sampleRate, 0);
+ message.writeInt32LE(samplesBuffer.length, 4);
+ samplesBuffer.copy(message, 8);
+
+ debugLogger.debug("parakeet-ws sending audio", {
+ samplesBytes: samplesBuffer.length,
+ sampleRate,
+ });
+
+ ws.send(message, (err) => {
+ if (err) {
+ debugLogger.error("parakeet-ws send error", { error: err.message });
+ }
+ });
+ });
+
+ ws.on("message", (data) => {
+ result += data.toString();
+ ws.send("Done");
+ });
+
+ ws.on("close", (code) => {
+ clearTimeout(timeout);
+ const elapsed = Date.now() - startTime;
+
+ debugLogger.debug("parakeet-ws transcription completed", {
+ elapsed,
+ code,
+ resultLength: result.length,
+ resultPreview: result.slice(0, 200),
+ });
+
+ try {
+ const parsed = JSON.parse(result);
+ done(resolve)({ text: (parsed.text || "").trim(), elapsed });
+ } catch {
+ done(resolve)({ text: result.trim(), elapsed });
+ }
+ });
+
+ ws.on("error", (error) => {
+ clearTimeout(timeout);
+ done(reject)(new Error(`parakeet-ws transcription failed: ${error.message}`));
+ });
+ });
+ }
+
+ async stop() {
+ this.stopHealthCheck();
+
+ if (!this.process) {
+ this.ready = false;
+ return;
+ }
+
+ debugLogger.debug("Stopping parakeet-ws server");
+
+ try {
+ await gracefulStopProcess(this.process);
+ } catch (error) {
+ debugLogger.error("Error stopping parakeet-ws server", { error: error.message });
+ }
+
+ this.process = null;
+ this.ready = false;
+ this.port = null;
+ this.modelName = null;
+ this.modelDir = null;
+ }
+
+ getStatus() {
+ return {
+ available: this.isAvailable(),
+ running: this.ready && this.process !== null,
+ port: this.port,
+ modelName: this.modelName,
+ };
+ }
+}
+
+module.exports = ParakeetWsServer;
diff --git a/src/helpers/whisper.js b/src/helpers/whisper.js
index ace841912..8dc76f17a 100644
--- a/src/helpers/whisper.js
+++ b/src/helpers/whisper.js
@@ -1,42 +1,28 @@
-const { app } = require("electron");
const fs = require("fs");
const fsPromises = require("fs").promises;
-const os = require("os");
const path = require("path");
const https = require("https");
const debugLogger = require("./debugLogger");
const WhisperServerManager = require("./whisperServer");
+const { getModelsDirForService } = require("./modelDirUtils");
+
+const modelRegistryData = require("../models/modelRegistryData.json");
-// Cache TTL for availability checks
const CACHE_TTL_MS = 30000;
-// GGML model definitions with HuggingFace URLs
-const WHISPER_MODELS = {
- tiny: {
- url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny.bin",
- size: 75_000_000,
- },
- base: {
- url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin",
- size: 142_000_000,
- },
- small: {
- url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin",
- size: 466_000_000,
- },
- medium: {
- url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-medium.bin",
- size: 1_500_000_000,
- },
- large: {
- url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3.bin",
- size: 3_000_000_000,
- },
- turbo: {
- url: "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin",
- size: 1_600_000_000,
- },
-};
+function getWhisperModelConfig(modelName) {
+ const modelInfo = modelRegistryData.whisperModels[modelName];
+ if (!modelInfo) return null;
+ return {
+ url: modelInfo.downloadUrl,
+ size: modelInfo.sizeMb * 1_000_000,
+ fileName: modelInfo.fileName,
+ };
+}
+
+function getValidModelNames() {
+ return Object.keys(modelRegistryData.whisperModels);
+}
class WhisperManager {
constructor() {
@@ -50,13 +36,12 @@ class WhisperManager {
}
getModelsDir() {
- const homeDir = app?.getPath?.("home") || os.homedir();
- return path.join(homeDir, ".cache", "openwhispr", "whisper-models");
+ return getModelsDirForService("whisper");
}
validateModelName(modelName) {
// Only allow known model names to prevent path traversal attacks
- const validModels = Object.keys(WHISPER_MODELS);
+ const validModels = getValidModelNames();
if (!validModels.includes(modelName)) {
throw new Error(`Invalid model name: ${modelName}. Valid models: ${validModels.join(", ")}`);
}
@@ -65,7 +50,8 @@ class WhisperManager {
getModelPath(modelName) {
this.validateModelName(modelName);
- return path.join(this.getModelsDir(), `ggml-${modelName}.bin`);
+ const config = getWhisperModelConfig(modelName);
+ return path.join(this.getModelsDir(), config.fileName);
}
async initializeAtStartup(settings = {}) {
@@ -157,7 +143,7 @@ class WhisperManager {
}
// Check downloaded models
- for (const modelName of Object.keys(WHISPER_MODELS)) {
+ for (const modelName of getValidModelNames()) {
const modelPath = this.getModelPath(modelName);
if (fs.existsSync(modelPath)) {
try {
@@ -277,14 +263,16 @@ class WhisperManager {
// Convert audioBlob to Buffer if needed
let audioBuffer;
- if (audioBlob instanceof ArrayBuffer) {
- audioBuffer = Buffer.from(audioBlob);
- } else if (audioBlob instanceof Uint8Array) {
+ if (Buffer.isBuffer(audioBlob)) {
+ audioBuffer = audioBlob;
+ } else if (ArrayBuffer.isView(audioBlob)) {
+ audioBuffer = Buffer.from(audioBlob.buffer, audioBlob.byteOffset, audioBlob.byteLength);
+ } else if (audioBlob instanceof ArrayBuffer) {
audioBuffer = Buffer.from(audioBlob);
} else if (typeof audioBlob === "string") {
audioBuffer = Buffer.from(audioBlob, "base64");
- } else if (audioBlob && audioBlob.buffer) {
- audioBuffer = Buffer.from(audioBlob.buffer);
+ } else if (audioBlob && audioBlob.buffer && typeof audioBlob.byteLength === "number") {
+ audioBuffer = Buffer.from(audioBlob.buffer, audioBlob.byteOffset || 0, audioBlob.byteLength);
} else {
throw new Error(`Unsupported audio data type: ${typeof audioBlob}`);
}
@@ -371,7 +359,7 @@ class WhisperManager {
// Model management methods
async downloadWhisperModel(modelName, progressCallback = null) {
this.validateModelName(modelName);
- const modelConfig = WHISPER_MODELS[modelName];
+ const modelConfig = getWhisperModelConfig(modelName);
const modelPath = this.getModelPath(modelName);
const modelsDir = this.getModelsDir();
@@ -394,7 +382,6 @@ class WhisperManager {
const tempPath = `${modelPath}.tmp`;
- // Track active download for cancellation
let activeRequest = null;
let activeFile = null;
let isCancelled = false;
@@ -411,7 +398,6 @@ class WhisperManager {
fs.unlink(tempPath, () => {});
};
- // Store cancellation function
this.currentDownloadProcess = {
abort: () => {
isCancelled = true;
@@ -541,7 +527,6 @@ class WhisperManager {
reject(err);
});
- // Request timeout (10 minutes for large models)
activeRequest.setTimeout(600000, () => {
cleanup();
reject(new Error("Download request timed out"));
@@ -580,7 +565,7 @@ class WhisperManager {
}
async listWhisperModels() {
- const models = Object.keys(WHISPER_MODELS);
+ const models = getValidModelNames();
const modelInfo = [];
for (const model of models) {
diff --git a/src/hooks/useModelDownload.ts b/src/hooks/useModelDownload.ts
index 0399ab21d..32e415521 100644
--- a/src/hooks/useModelDownload.ts
+++ b/src/hooks/useModelDownload.ts
@@ -14,7 +14,7 @@ export interface DownloadProgress {
eta?: number;
}
-export type ModelType = "whisper" | "llm";
+export type ModelType = "whisper" | "llm" | "parakeet";
interface UseModelDownloadOptions {
modelType: ModelType;
@@ -109,10 +109,15 @@ export function useModelDownload({
}, []);
useEffect(() => {
- const dispose =
- modelType === "whisper"
- ? window.electronAPI?.onWhisperDownloadProgress(handleWhisperProgress)
- : window.electronAPI?.onModelDownloadProgress(handleLLMProgress);
+ let dispose: (() => void) | undefined;
+
+ if (modelType === "whisper") {
+ dispose = window.electronAPI?.onWhisperDownloadProgress(handleWhisperProgress);
+ } else if (modelType === "parakeet") {
+ dispose = window.electronAPI?.onParakeetDownloadProgress(handleWhisperProgress);
+ } else {
+ dispose = window.electronAPI?.onModelDownloadProgress(handleLLMProgress);
+ }
return () => {
dispose?.();
@@ -147,6 +152,16 @@ export function useModelDownload({
} else {
success = result?.success ?? false;
}
+ } else if (modelType === "parakeet") {
+ const result = await window.electronAPI?.downloadParakeetModel(modelId);
+ if (!result?.success && !result?.error?.includes("interrupted by user")) {
+ showAlertDialog({
+ title: "Download Failed",
+ description: `Failed to download model: ${result?.error}`,
+ });
+ } else {
+ success = result?.success ?? false;
+ }
} else {
const result = (await window.electronAPI?.modelDownload?.(modelId)) as
| { success: boolean; error?: string }
@@ -200,6 +215,14 @@ export function useModelDownload({
description: `Model deleted successfully! Freed ${result.freed_mb}MB of disk space.`,
});
}
+ } else if (modelType === "parakeet") {
+ const result = await window.electronAPI?.deleteParakeetModel(modelId);
+ if (result?.success) {
+ toast({
+ title: "Model Deleted",
+ description: `Model deleted successfully! Freed ${result.freed_mb}MB of disk space.`,
+ });
+ }
} else {
await window.electronAPI?.modelDelete?.(modelId);
toast({
@@ -227,6 +250,8 @@ export function useModelDownload({
try {
if (modelType === "whisper") {
await window.electronAPI?.cancelWhisperDownload();
+ } else if (modelType === "parakeet") {
+ await window.electronAPI?.cancelParakeetDownload();
} else {
await window.electronAPI?.modelCancelDownload?.(downloadingModel);
}
diff --git a/src/hooks/useSettings.ts b/src/hooks/useSettings.ts
index f7c6edc7a..474b04bd7 100644
--- a/src/hooks/useSettings.ts
+++ b/src/hooks/useSettings.ts
@@ -8,6 +8,8 @@ import ReasoningService from "../services/ReasoningService";
export interface TranscriptionSettings {
useLocalWhisper: boolean;
whisperModel: string;
+ localTranscriptionProvider: string;
+ parakeetModel: string;
allowOpenAIFallback: boolean;
allowLocalFallback: boolean;
fallbackWhisperModel: string;
@@ -55,6 +57,20 @@ export function useSettings() {
deserialize: String,
});
+ const [localTranscriptionProvider, setLocalTranscriptionProvider] = useLocalStorage(
+ "localTranscriptionProvider",
+ "whisper",
+ {
+ serialize: String,
+ deserialize: String,
+ }
+ );
+
+ const [parakeetModel, setParakeetModel] = useLocalStorage("parakeetModel", "", {
+ serialize: String,
+ deserialize: String,
+ });
+
const [allowOpenAIFallback, setAllowOpenAIFallback] = useLocalStorage(
"allowOpenAIFallback",
false,
@@ -392,6 +408,9 @@ export function useSettings() {
(settings: Partial<TranscriptionSettings>) => {
if (settings.useLocalWhisper !== undefined) setUseLocalWhisper(settings.useLocalWhisper);
if (settings.whisperModel !== undefined) setWhisperModel(settings.whisperModel);
+ if (settings.localTranscriptionProvider !== undefined)
+ setLocalTranscriptionProvider(settings.localTranscriptionProvider);
+ if (settings.parakeetModel !== undefined) setParakeetModel(settings.parakeetModel);
if (settings.allowOpenAIFallback !== undefined)
setAllowOpenAIFallback(settings.allowOpenAIFallback);
if (settings.allowLocalFallback !== undefined)
@@ -411,6 +430,8 @@ export function useSettings() {
[
setUseLocalWhisper,
setWhisperModel,
+ setLocalTranscriptionProvider,
+ setParakeetModel,
setAllowOpenAIFallback,
setAllowLocalFallback,
setFallbackWhisperModel,
@@ -447,6 +468,8 @@ export function useSettings() {
return {
useLocalWhisper,
whisperModel,
+ localTranscriptionProvider,
+ parakeetModel,
allowOpenAIFallback,
allowLocalFallback,
fallbackWhisperModel,
@@ -466,6 +489,8 @@ export function useSettings() {
dictationKey,
setUseLocalWhisper,
setWhisperModel,
+ setLocalTranscriptionProvider,
+ setParakeetModel,
setAllowOpenAIFallback,
setAllowLocalFallback,
setFallbackWhisperModel,
diff --git a/src/models/ModelRegistry.ts b/src/models/ModelRegistry.ts
index e4a9afdd8..cf62d753d 100644
--- a/src/models/ModelRegistry.ts
+++ b/src/models/ModelRegistry.ts
@@ -61,12 +61,35 @@ export interface WhisperModelInfo {
description: string;
size: string;
sizeMb: number;
+ fileName: string;
+ downloadUrl: string;
recommended?: boolean;
}
+export interface WhisperModelConfig {
+ url: string;
+ size: number;
+ fileName: string;
+}
+
export type WhisperModelsMap = Record<string, WhisperModelInfo>;
+export interface ParakeetModelInfo {
+ name: string;
+ description: string;
+ size: string;
+ sizeMb: number;
+ language: string;
+ supportedLanguages: string[];
+ recommended?: boolean;
+ downloadUrl: string;
+ extractDir: string;
+}
+
+export type ParakeetModelsMap = Record<string, ParakeetModelInfo>;
+
interface ModelRegistryData {
+ parakeetModels: ParakeetModelsMap;
whisperModels: WhisperModelsMap;
transcriptionProviders: TranscriptionProviderData[];
cloudProviders: CloudProviderData[];
@@ -285,3 +308,27 @@ export function getCloudModel(modelId: string): CloudModelDefinition | undefined
}
return undefined;
}
+
+export function getParakeetModels(): ParakeetModelsMap {
+ return modelData.parakeetModels;
+}
+
+export function getParakeetModelInfo(modelId: string): ParakeetModelInfo | undefined {
+ return modelData.parakeetModels[modelId];
+}
+
+export const PARAKEET_MODEL_INFO = modelData.parakeetModels;
+
+export function getWhisperModelConfig(modelId: string): WhisperModelConfig | null {
+ const modelInfo = modelData.whisperModels[modelId];
+ if (!modelInfo) return null;
+ return {
+ url: modelInfo.downloadUrl,
+ size: modelInfo.sizeMb * 1_000_000,
+ fileName: modelInfo.fileName,
+ };
+}
+
+export function getValidWhisperModelNames(): string[] {
+ return Object.keys(modelData.whisperModels);
+}
diff --git a/src/models/modelRegistryData.json b/src/models/modelRegistryData.json
index c92a01401..4b74d0fa1 100644
--- a/src/models/modelRegistryData.json
+++ b/src/models/modelRegistryData.json
@@ -1,36 +1,92 @@
{
+ "parakeetModels": {
+ "parakeet-tdt-0.6b-v3": {
+ "name": "Parakeet TDT 0.6B",
+ "description": "Fast multilingual ASR with auto language detection (25 languages)",
+ "size": "680MB",
+ "sizeMb": 680,
+ "language": "multilingual",
+ "supportedLanguages": [
+ "en",
+ "de",
+ "es",
+ "fr",
+ "it",
+ "pt",
+ "nl",
+ "pl",
+ "ru",
+ "uk",
+ "zh",
+ "ja",
+ "ko",
+ "ar",
+ "hi",
+ "tr",
+ "vi",
+ "th",
+ "id",
+ "cs",
+ "ro",
+ "hu",
+ "el",
+ "he",
+ "sv"
+ ],
+ "recommended": true,
+ "downloadUrl": "https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8.tar.bz2",
+ "extractDir": "sherpa-onnx-nemo-parakeet-tdt-0.6b-v3-int8"
+ }
+ },
"whisperModels": {
"tiny": {
"name": "Tiny",
"description": "Fastest, lower quality",
- "size": "39MB",
- "sizeMb": 39
+ "size": "75MB",
+ "sizeMb": 75,
+ "fileName": "ggml-tiny.bin",
+ "downloadUrl": "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-tiny.bin"
},
"base": {
"name": "Base",
"description": "Good balance",
- "size": "74MB",
- "sizeMb": 74,
+ "size": "142MB",
+ "sizeMb": 142,
+ "fileName": "ggml-base.bin",
+ "downloadUrl": "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-base.bin",
"recommended": true
},
"small": {
"name": "Small",
"description": "Better quality, slower",
- "size": "244MB",
- "sizeMb": 244
+ "size": "466MB",
+ "sizeMb": 466,
+ "fileName": "ggml-small.bin",
+ "downloadUrl": "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin"
+ },
+ "medium": {
+ "name": "Medium",
+ "description": "High quality",
+ "size": "1.5GB",
+ "sizeMb": 1500,
+ "fileName": "ggml-medium.bin",
+ "downloadUrl": "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-medium.bin"
},
- "medium": { "name": "Medium", "description": "High quality", "size": "769MB", "sizeMb": 769 },
"large": {
"name": "Large",
"description": "Best quality, slowest",
- "size": "1.5GB",
- "sizeMb": 1550
+ "size": "3GB",
+ "sizeMb": 3000,
+ "fileName": "ggml-large-v3.bin",
+ "downloadUrl": "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3.bin"
},
"turbo": {
"name": "Turbo",
"description": "Fast with good quality",
- "size": "809MB",
- "sizeMb": 809
+ "size": "1.6GB",
+ "sizeMb": 1600,
+ "fileName": "ggml-large-v3-turbo.bin",
+ "downloadUrl": "https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-large-v3-turbo.bin"
}
},
"transcriptionProviders": [
diff --git a/src/types/electron.ts b/src/types/electron.ts
index 13bb19743..18d005a3c 100644
--- a/src/types/electron.ts
+++ b/src/types/electron.ts
@@ -92,6 +92,63 @@ export interface WhisperDownloadProgressData {
result?: any;
}
+export interface ParakeetCheckResult {
+ installed: boolean;
+ working: boolean;
+ path?: string;
+}
+
+export interface ParakeetModelResult {
+ success: boolean;
+ model: string;
+ downloaded: boolean;
+ path?: string;
+ size_bytes?: number;
+ size_mb?: number;
+ error?: string;
+}
+
+export interface ParakeetModelDeleteResult {
+ success: boolean;
+ model: string;
+ deleted: boolean;
+ freed_bytes?: number;
+ freed_mb?: number;
+ error?: string;
+}
+
+export interface ParakeetModelsListResult {
+ success: boolean;
+ models: Array<{ model: string; downloaded: boolean; size_mb?: number }>;
+ cache_dir: string;
+}
+
+export interface ParakeetDownloadProgressData {
+ type: string;
+ model: string;
+ percentage?: number;
+ downloaded_bytes?: number;
+ total_bytes?: number;
+ error?: string;
+}
+
+export interface ParakeetTranscriptionResult {
+ success: boolean;
+ text?: string;
+ message?: string;
+ error?: string;
+}
+
+export interface ParakeetDiagnosticsResult {
+ platform: string;
+ arch: string;
+ resourcesPath: string | null;
+ isPackaged: boolean;
+ sherpaOnnx: { available: boolean; path: string | null };
+ modelsDir: string;
+ models: string[];
+}
+
export interface PasteToolsResult {
platform: "darwin" | "win32" | "linux";
available: boolean;
@@ -170,6 +227,33 @@ declare global {
error?: string;
}>;
+ // Parakeet operations (NVIDIA via sherpa-onnx)
+ transcribeLocalParakeet: (
+ audioBlob: ArrayBuffer,
+ options?: { model?: string; language?: string }
+ ) => Promise<ParakeetTranscriptionResult>;
+ checkParakeetInstallation: () => Promise<ParakeetCheckResult>;
+ downloadParakeetModel: (modelName: string) => Promise<ParakeetModelResult>;
+ onParakeetDownloadProgress: (
+ callback: (event: any, data: ParakeetDownloadProgressData) => void
+ ) => (() => void) | void;
+ checkParakeetModelStatus: (modelName: string) => Promise<ParakeetModelResult>;
+ listParakeetModels: () => Promise<ParakeetModelsListResult>;
+ deleteParakeetModel: (modelName: string) => Promise<ParakeetModelDeleteResult>;
+ deleteAllParakeetModels: () => Promise<{
+ success: boolean;
+ deleted_count?: number;
+ freed_bytes?: number;
+ freed_mb?: number;
+ error?: string;
+ }>;
+ cancelParakeetDownload: () => Promise<{
+ success: boolean;
+ message?: string;
+ error?: string;
+ }>;
+ getParakeetDiagnostics: () => Promise<ParakeetDiagnosticsResult>;
+
// Local AI model management
modelGetAll: () => Promise<any[]>;
modelCheck: (modelId: string) => Promise<boolean>;
diff --git a/src/utils/serverUtils.js b/src/utils/serverUtils.js
new file mode 100644
index 000000000..b28550895
--- /dev/null
+++ b/src/utils/serverUtils.js
@@ -0,0 +1,77 @@
+const fs = require("fs");
+const net = require("net");
+const path = require("path");
+const { killProcess } = require("./process");
+
+const GRACEFUL_STOP_TIMEOUT_MS = 5000;
+
+function isPortAvailable(port) {
+ return new Promise((resolve) => {
+ const server = net.createServer();
+ server.once("error", () => resolve(false));
+ server.once("listening", () => {
+ server.close();
+ resolve(true);
+ });
+ server.listen(port, "127.0.0.1");
+ });
+}
+
+async function findAvailablePort(rangeStart, rangeEnd) {
+ for (let port = rangeStart; port <= rangeEnd; port++) {
+ if (await isPortAvailable(port)) return port;
+ }
+ throw new Error(`No available ports in range ${rangeStart}-${rangeEnd}`);
+}
+
+function resolveBinaryPath(binaryName) {
+ const candidates = [];
+
+ if (process.resourcesPath) {
+ candidates.push(path.join(process.resourcesPath, "bin", binaryName));
+ }
+
+ const projectBinDir = path.resolve(__dirname, "..", "..", "resources", "bin");
+ candidates.push(path.join(projectBinDir, binaryName));
+
+ for (const candidate of candidates) {
+ if (fs.existsSync(candidate)) {
+ try {
+ fs.statSync(candidate);
+ return candidate;
+ } catch {
+ // Can't access binary
+ }
+ }
+ }
+
+ return null;
+}
+
+async function gracefulStopProcess(proc) {
+ killProcess(proc, "SIGTERM");
+
+ await new Promise((resolve) => {
+ const timeout = setTimeout(() => {
+ if (proc) killProcess(proc, "SIGKILL");
+ resolve();
+ }, GRACEFUL_STOP_TIMEOUT_MS);
+
+ if (proc) {
+ proc.once("close", () => {
+ clearTimeout(timeout);
+ resolve();
+ });
+ } else {
+ clearTimeout(timeout);
+ resolve();
+ }
+ });
+}
+
+module.exports = {
+ findAvailablePort,
+ isPortAvailable,
+ resolveBinaryPath,
+ gracefulStopProcess,
+};