Speakr is a personal, self-hosted web application designed for transcribing audio recordings
Python
Fix ASR compatibility for whisper-asr-webservice (#149)
Murtaza Nasir8102dd7d8f8d
diff --git a/README.md b/README.mdindex 4922e18e..c08a172f 100644--- a/README.md+++ b/README.md@@ -128,8 +128,8 @@ docker compose up -d```**Note:** ASR option requires running an additional ASR service container alongside Speakr:-- **For voice profiles & speaker embeddings:** Use [WhisperX ASR Service](https://github.com/murtaza-nasir/whisperx-asr-service) (recommended)-- **For basic speaker diarization:** Use [OpenAI Whisper ASR Webservice](https://github.com/ahmetoner/whisper-asr-webservice)+- **For voice profiles & speaker embeddings:** Use [WhisperX ASR Service](https://github.com/murtaza-nasir/whisperx-asr-service) and set `ASR_RETURN_SPEAKER_EMBEDDINGS=true` in your `.env`+- **For basic speaker diarization:** Use [OpenAI Whisper ASR Webservice](https://github.com/ahmetoner/whisper-asr-webservice) (speaker embeddings not supported)> **⚠️ PyTorch 2.6 Users:** If you encounter a "Weights only load failed" error with WhisperX, add `TORCH_FORCE_NO_WEIGHTS_ONLY_LOAD=true` to your ASR container's environment in docker-compose.yml. See [troubleshooting](https://murtaza-nasir.github.io/speakr/troubleshooting#pytorch-26-weights-loading-error-whisperx-asr-service) for details.diff --git a/config/env.asr.example b/config/env.asr.exampleindex 989e0131..1d1845a7 100644--- a/config/env.asr.example+++ b/config/env.asr.example@@ -69,6 +69,7 @@ ASR_BASE_URL=http://whisper-asr:9000# ASR_DIARIZE=true # Auto-enabled with ASR (set to false to disable)# ASR_MIN_SPEAKERS=1 # Default minimum speakers# ASR_MAX_SPEAKERS=5 # Default maximum speakers+# ASR_RETURN_SPEAKER_EMBEDDINGS=false # Only enable for WhisperX ASR service# --- Application Settings ---# Set to "true" to allow user registration, "false" to disablediff --git a/config/env.whisperx.example b/config/env.whisperx.exampleindex 28837f55..f6e2eb46 100644--- a/config/env.whisperx.example+++ b/config/env.whisperx.example@@ -80,6 +80,9 @@ ASR_BASE_URL=http://whisperx-asr:9000# ASR_MIN_SPEAKERS=1 # Default minimum speakers# ASR_MAX_SPEAKERS=5 # Default maximum speakers+# Enable speaker embeddings for voice profile matching (WhisperX only)+ASR_RETURN_SPEAKER_EMBEDDINGS=true+# --- Application Settings ---# Set to "true" to allow user registration, "false" to disableALLOW_REGISTRATION=falsediff --git a/docs/admin-guide/whisperx-setup.md b/docs/admin-guide/whisperx-setup.mdindex bab507bc..3c373d32 100644--- a/docs/admin-guide/whisperx-setup.md+++ b/docs/admin-guide/whisperx-setup.md@@ -15,12 +15,14 @@ WhisperX is an advanced ASR (Automatic Speech Recognition) service that provides- ✅ Better speaker diarization accuracy (Pyannote.audio 4.0)- ✅ More precise word-level timestamps- ✅ Improved multi-speaker handling+- ✅ **Voice profile support** with 256-dimensional speaker embeddings+- ✅ Automatic speaker recognition across recordings- ✅ Active development and updates- ✅ Production-ready Docker deployment**vs. Standard Whisper ASR:**-- Standard: Simple, lightweight, good for single speakers-- WhisperX: Advanced diarization, better for meetings/conversations+- Standard: Simple, lightweight, good for single speakers, no voice profiles+- WhisperX: Advanced diarization, voice profiles, better for meetings/conversations## Prerequisites@@ -149,8 +151,13 @@ USE_ASR_ENDPOINT=true# Point to WhisperX serviceASR_BASE_URL=http://whisperx-asr-api:9000++# Enable voice profile features (speaker embeddings)+ASR_RETURN_SPEAKER_EMBEDDINGS=true```+> **Important:** The `ASR_RETURN_SPEAKER_EMBEDDINGS=true` setting is required to enable voice profile features. This setting is only supported by WhisperX and should not be enabled when using the basic OpenAI Whisper ASR Webservice.+Restart Speakr:```bash@@ -181,6 +188,7 @@ Update Speakr's `.env`:```bashUSE_ASR_ENDPOINT=trueASR_BASE_URL=http://GPU_MACHINE_IP:9000+ASR_RETURN_SPEAKER_EMBEDDINGS=true```Replace `GPU_MACHINE_IP` with actual IP address.@@ -392,11 +400,15 @@ Tested on RTX 3080 (10GB VRAM):| Transcription Quality | Excellent | Excellent || Word Timestamps | Good | Excellent || Speaker Diarization | Good | Excellent |+| Voice Profiles | ❌ Not supported | ✅ 256-dim embeddings |+| Speaker Recognition | ❌ Manual only | ✅ Automatic matching || Setup Complexity | Low | Medium || Resource Usage | Lower | Higher || Active Development | Moderate | High || Production Ready | Yes | Yes |+> **Note:** To enable voice profile features with WhisperX, you must set `ASR_RETURN_SPEAKER_EMBEDDINGS=true` in Speakr's `.env` file. This setting is disabled by default for compatibility with the basic ASR webservice.+## Best Practices1. **Start with small model** to verify setupdiff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.mdindex 5406055f..96f533be 100644--- a/docs/getting-started/installation.md+++ b/docs/getting-started/installation.md@@ -101,6 +101,7 @@ If you want speaker diarization to identify who's speaking in your recordings, y- Better timestamp alignment between speakers and words- **Required for:** Voice profiles, automatic speaker recognition, speaker embeddings- **Environment file:** `config/env.whisperx.example`+ - **Required setting:** `ASR_RETURN_SPEAKER_EMBEDDINGS=true` to enable voice profile features2. **OpenAI Whisper ASR Webservice (Basic Diarization)** - For basic speaker diarization without voice profiles- Repository: [ahmetoner/openai-whisper-asr-webservice](https://github.com/ahmetoner/openai-whisper-asr-webservice)@@ -109,6 +110,7 @@ If you want speaker diarization to identify who's speaking in your recordings, y- **Supports:** Basic speaker identification (Speaker 1, Speaker 2, etc.)- **Does not support:** Voice profiles, speaker embeddings, automatic speaker recognition- **Environment file:** `config/env.asr.example`+ - **Note:** Do not set `ASR_RETURN_SPEAKER_EMBEDDINGS=true` with this service as it will cause errors> **Important:** Before proceeding with this configuration, you'll need to set up one of the ASR service containers. See [Running ASR Service for Speaker Diarization](#running-asr-service-for-speaker-diarization) for complete instructions on deploying both containers together or separately.@@ -140,6 +142,16 @@ The ASR_BASE_URL depends on your deployment architecture:Speaker diarization is automatically enabled when using ASR endpoints. The system will identify different speakers in your recordings and label them as Speaker 1, Speaker 2, and so on. You can optionally override the default speaker detection settings by uncommenting and adjusting ASR_MIN_SPEAKERS and ASR_MAX_SPEAKERS in your environment file.+**Voice Profile Configuration:**++If you're using WhisperX ASR Service and want to enable voice profile features (automatic speaker recognition across recordings), add this to your `.env` file:++```bash+ASR_RETURN_SPEAKER_EMBEDDINGS=true+```++This setting is disabled by default because it's only supported by WhisperX. If you're using the basic OpenAI Whisper ASR Webservice, leave this setting disabled or omit it entirely to avoid errors.+### Step 4: Configure System SettingsOne of Speakr's conveniences is automatic admin account creation. Instead of going through a registration process, you define the admin credentials in your environment file, and Speakr creates the account automatically on first startup. This ensures you can log in immediately without any additional setup steps:diff --git a/src/config/app_config.py b/src/config/app_config.pyindex dcc75a37..a078c8ca 100644--- a/src/config/app_config.py+++ b/src/config/app_config.py@@ -31,10 +31,13 @@ if USE_ASR_ENDPOINT:ASR_DIARIZE = os.environ.get('ASR_DIARIZE', 'true').lower() == 'true'ASR_MIN_SPEAKERS = os.environ.get('ASR_MIN_SPEAKERS')ASR_MAX_SPEAKERS = os.environ.get('ASR_MAX_SPEAKERS')+ # Speaker embeddings are only supported by WhisperX ASR service, not the basic whisper-asr-webservice+ ASR_RETURN_SPEAKER_EMBEDDINGS = os.environ.get('ASR_RETURN_SPEAKER_EMBEDDINGS', 'false').lower() == 'true'else:ASR_DIARIZE = FalseASR_MIN_SPEAKERS = NoneASR_MAX_SPEAKERS = None+ ASR_RETURN_SPEAKER_EMBEDDINGS = FalseENABLE_CHUNKING = os.environ.get('ENABLE_CHUNKING', 'true').lower() == 'true'CHUNK_SIZE_MB = int(os.environ.get('CHUNK_SIZE_MB', '20'))diff --git a/src/tasks/processing.py b/src/tasks/processing.pyindex 7481350a..9601449f 100644--- a/src/tasks/processing.py+++ b/src/tasks/processing.py@@ -26,7 +26,7 @@ from src.services.embeddings import process_recording_chunksfrom src.services.llm import is_using_openai_api, call_llm_completion, format_api_error_message, TEXT_MODEL_NAME, client, http_client_no_proxyfrom src.utils import extract_json_object, safe_json_loadsfrom src.audio_chunking import AudioChunkingService, ChunkProcessingError, ChunkingNotSupportedError-from src.config.app_config import ASR_DIARIZE, ASR_BASE_URL, transcription_api_key, transcription_base_url, chunking_service, ENABLE_CHUNKING+from src.config.app_config import ASR_DIARIZE, ASR_BASE_URL, ASR_RETURN_SPEAKER_EMBEDDINGS, transcription_api_key, transcription_base_url, chunking_service, ENABLE_CHUNKINGfrom src.file_exporter import export_recording, ENABLE_AUTO_EXPORT# Configuration for internal sharing@@ -954,9 +954,14 @@ def transcribe_audio_asr(app_context, recording_id, filepath, original_filename,if language:params['language'] = languageif diarize:- params['enable_diarization'] = diarize- # Request speaker embeddings when diarization is enabled- params['return_speaker_embeddings'] = True+ # Send both parameter names for compatibility:+ # - 'diarize' is used by whisper-asr-webservice+ # - 'enable_diarization' is used by WhisperX+ params['diarize'] = True+ params['enable_diarization'] = True+ # Only request speaker embeddings if explicitly enabled (WhisperX only)+ if ASR_RETURN_SPEAKER_EMBEDDINGS:+ params['return_speaker_embeddings'] = Trueif min_speakers:params['min_speakers'] = min_speakersif max_speakers:@@ -975,9 +980,20 @@ def transcribe_audio_asr(app_context, recording_id, filepath, original_filename,response = client.post(url, params=params, files=files, timeout=timeout)current_app.logger.info(f"ASR request completed with status: {response.status_code}")response.raise_for_status()-- # Parse the JSON response from ASR (moved here so it's accessible)- asr_response_data = response.json()++ # Parse the JSON response from ASR+ # Try to parse as JSON first, fall back to content-type check for error handling+ response_text = response.text+ try:+ asr_response_data = response.json()+ except Exception as json_err:+ # If JSON parsing fails, check if it looks like HTML (error page)+ if response_text.strip().startswith('<'):+ current_app.logger.error(f"ASR returned HTML error page (status {response.status_code}): {response_text[:500]}")+ raise Exception(f"ASR service returned HTML error page (status {response.status_code})")+ else:+ current_app.logger.error(f"ASR returned non-JSON response (status {response.status_code}): {response_text[:500]}")+ raise Exception(f"ASR service returned invalid response: {json_err}")# Extract speaker embeddings if presentif 'speaker_embeddings' in asr_response_data: