Speakr is a personal, self-hosted web application designed for transcribing audio recordings
Python
Add migration guide and custom connectors documentation
Murtaza Nasireafd910a3b30
diff --git a/docs/admin-guide/migration-guide.md b/docs/admin-guide/migration-guide.mdnew file mode 100644index 00000000..087590ec--- /dev/null+++ b/docs/admin-guide/migration-guide.md@@ -0,0 +1,192 @@+# Migration Guide: Connector Architecture++This guide helps you migrate from the legacy transcription configuration to the new connector-based architecture introduced in Speakr v0.8.++## Overview++Speakr now uses a **connector-based architecture** for transcription services. This provides:++- **Simplified configuration** - Fewer environment variables needed+- **Auto-detection** - Speakr can attempt to automatically select the right connector+- **Better feature support** - Data-driven UI that adapts to connector capabilities+- **Extensibility** - Possibility to add custom connectors for new providers++## Backwards Compatibility++**Your existing configuration will continue to work.** The new architecture maintains full backwards compatibility with legacy environment variables. However, you may see deprecation warnings in the logs for certain settings.++## What's Changed++### Deprecated Environment Variables++| Deprecated Variable | Status | Migration |+|---------------------|--------|-----------|+| `USE_ASR_ENDPOINT=true` | Still works, logs warning | Just set `ASR_BASE_URL` instead |+| `WHISPER_MODEL` | Still works, logs warning | Use `TRANSCRIPTION_MODEL` instead |++### New Environment Variables++| Variable | Description |+|----------|-------------|+| `TRANSCRIPTION_CONNECTOR` | Explicit connector selection (optional, auto-detected) |+| `TRANSCRIPTION_MODEL` | Model name for OpenAI connectors |++### Auto-Detection Priority++Speakr automatically selects a connector based on your configuration:++1. **Explicit selection** - If `TRANSCRIPTION_CONNECTOR` is set, use that connector+2. **ASR mode** - If `ASR_BASE_URL` is set, use the ASR Endpoint connector+3. **OpenAI Transcribe** - If `TRANSCRIPTION_MODEL` contains `gpt-4o`, use OpenAI Transcribe connector+4. **Default** - Use OpenAI Whisper connector with `TRANSCRIPTION_MODEL` or `whisper-1`++## Migration Examples++### From Legacy ASR Configuration++**Before (Legacy):**+```bash+USE_ASR_ENDPOINT=true+ASR_BASE_URL=http://whisperx-asr:9000+ASR_DIARIZE=true+ASR_RETURN_SPEAKER_EMBEDDINGS=true+```++**After (New - Minimal):**+```bash+ASR_BASE_URL=http://whisperx-asr:9000+ASR_RETURN_SPEAKER_EMBEDDINGS=true+```++The `USE_ASR_ENDPOINT=true` is no longer needed—setting `ASR_BASE_URL` automatically enables ASR mode. Diarization is enabled by default for ASR endpoints.++### From Legacy Whisper Configuration++**Before (Legacy):**+```bash+TRANSCRIPTION_BASE_URL=https://api.openai.com/v1+TRANSCRIPTION_API_KEY=sk-xxx+WHISPER_MODEL=whisper-1+```++**After (New):**+```bash+TRANSCRIPTION_API_KEY=sk-xxx+TRANSCRIPTION_MODEL=whisper-1+```++The base URL defaults to OpenAI's API, and `TRANSCRIPTION_MODEL` replaces the deprecated `WHISPER_MODEL`.++### Upgrading to OpenAI Diarization++If you want speaker diarization without running a self-hosted ASR service:++**New Configuration:**+```bash+TRANSCRIPTION_API_KEY=sk-xxx+TRANSCRIPTION_MODEL=gpt-4o-transcribe-diarize+```++This uses OpenAI's built-in diarization. The connector is auto-detected from the model name.++## Chunking Behavior Changes++The new architecture makes chunking **connector-aware**:++| Connector | Chunking Behavior |+|-----------|-------------------|+| **ASR Endpoint** | Handled internally—your `CHUNK_*` settings are ignored |+| **OpenAI Transcribe** | Handled internally via `chunking_strategy=auto`—your settings are ignored |+| **OpenAI Whisper** | Uses your `CHUNK_LIMIT` and `CHUNK_OVERLAP_SECONDS` settings |++If you were manually configuring chunking for ASR endpoints, you can remove those settings as they no longer have any effect.++## UI Feature Changes++Some UI features are now **data-driven** rather than configuration-driven:++| Feature | Old Behavior | New Behavior |+|---------|--------------|--------------|+| Speaker identification button | Shown when `USE_ASR_ENDPOINT=true` | Shown when transcription has diarization data |+| Min/Max speakers in reprocess | Always shown for ASR | Only shown when connector supports it |+| Bubble view toggle | Based on config | Based on whether transcription has dialogue |++This means features automatically appear when available, regardless of which connector produced the transcription.++## Verifying Your Migration++After updating your configuration:++1. **Check the logs** - Look for deprecation warnings:+ ```bash+ docker compose logs app | grep -i deprecat+ ```++2. **Test transcription** - Upload a test file and verify it transcribes correctly++3. **Check system info** - Visit `/api/system/info` to see the active connector:+ ```json+ {+ "transcription": {+ "connector": "asr_endpoint",+ "supports_diarization": true,+ "supports_speaker_embeddings": true+ }+ }+ ```++## Recommended Configuration++### For Self-Hosted (Best Quality)++Using WhisperX ASR Service for superior transcription and diarization:++```bash+# Transcription+ASR_BASE_URL=http://whisperx-asr:9000+ASR_RETURN_SPEAKER_EMBEDDINGS=true++# Text generation+TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1+TEXT_MODEL_API_KEY=sk-or-v1-xxx+TEXT_MODEL_NAME=openai/gpt-4o-mini+```++### For Cloud-Based (No Self-Hosting)++Using OpenAI's transcription with diarization:++```bash+# Transcription+TRANSCRIPTION_API_KEY=sk-xxx+TRANSCRIPTION_MODEL=gpt-4o-transcribe-diarize++# Text generation+TEXT_MODEL_BASE_URL=https://openrouter.ai/api/v1+TEXT_MODEL_API_KEY=sk-or-v1-xxx+TEXT_MODEL_NAME=openai/gpt-4o-mini+```++## Troubleshooting++### "Connector not found" Error++Ensure you have the correct environment variables set. Check the auto-detection priority above.++### Features Missing After Migration++If UI features like speaker identification are missing:+- Verify the transcription actually contains diarization data+- Check that your connector supports the feature (e.g., voice profiles require ASR endpoint)++### Deprecation Warnings in Logs++These are informational only—your configuration still works. Update your `.env` file at your convenience to use the new variable names.++## Getting Help++If you encounter issues during migration:++1. Check the [troubleshooting guide](../troubleshooting.md)+2. Review the [installation guide](../getting-started/installation.md) for complete configuration examples+3. Open an issue on [GitHub](https://github.com/murtaza-nasir/speakr/issues)diff --git a/docs/advanced/custom-connectors.md b/docs/advanced/custom-connectors.mdnew file mode 100644index 00000000..2f371a42--- /dev/null+++ b/docs/advanced/custom-connectors.md@@ -0,0 +1,545 @@+# Creating Custom Transcription Connectors++This guide explains how to create custom transcription connectors for Speakr. Connectors allow you to integrate new transcription providers like Deepgram, AssemblyAI, Google Cloud Speech-to-Text, or your own custom services.++## Architecture Overview++Speakr uses a connector-based architecture where each transcription provider is implemented as a connector class that extends `BaseTranscriptionConnector`. The connector registry manages all available connectors and handles auto-detection from environment variables.++```+src/services/transcription/+├── base.py # Base classes and data types+├── registry.py # Connector registry and factory+├── exceptions.py # Custom exceptions+└── connectors/+ ├── __init__.py+ ├── openai_whisper.py # OpenAI Whisper connector+ ├── openai_transcribe.py # OpenAI GPT-4o transcribe connector+ └── asr_endpoint.py # Self-hosted ASR connector+```++## Base Classes++### TranscriptionCapability++Capabilities that connectors can declare support for:++```python+from enum import Enum, auto++class TranscriptionCapability(Enum):+ DIARIZATION = auto() # Speaker diarization+ CHUNKING = auto() # Automatic file chunking for large files+ TIMESTAMPS = auto() # Word/segment timestamps+ LANGUAGE_DETECTION = auto() # Auto language detection+ KNOWN_SPEAKERS = auto() # Support for known speaker references+ SPEAKER_EMBEDDINGS = auto() # Return speaker embeddings+ SPEAKER_COUNT_CONTROL = auto() # Support for min/max speaker count+ STREAMING = auto() # Real-time streaming transcription+```++### ConnectorSpecifications++Provider-specific constraints and requirements:++```python+from dataclasses import dataclass+from typing import Optional, FrozenSet++@dataclass+class ConnectorSpecifications:+ # Size constraints+ max_file_size_bytes: Optional[int] = None # None = unlimited++ # Duration constraints+ max_duration_seconds: Optional[int] = None+ min_duration_for_chunking: Optional[int] = None++ # Chunking behavior+ handles_chunking_internally: bool = False # Provider handles large files+ requires_chunking_param: bool = False # Must send chunking_strategy param+ recommended_chunk_seconds: int = 600 # 10 minutes default++ # Audio format support+ supported_codecs: Optional[FrozenSet[str]] = None # Override defaults+ unsupported_codecs: Optional[FrozenSet[str]] = None # Exclude from defaults+```++### Request/Response Types++```python+@dataclass+class TranscriptionRequest:+ audio_file: BinaryIO+ filename: str+ mime_type: Optional[str] = None+ language: Optional[str] = None++ # Diarization options+ diarize: bool = False+ min_speakers: Optional[int] = None+ max_speakers: Optional[int] = None++ # Advanced options+ prompt: Optional[str] = None+ temperature: Optional[float] = None+ extra_options: Dict[str, Any] = field(default_factory=dict)+++@dataclass+class TranscriptionSegment:+ text: str+ speaker: Optional[str] = None+ start_time: Optional[float] = None+ end_time: Optional[float] = None+ confidence: Optional[float] = None+ words: Optional[List[Dict[str, Any]]] = None+++@dataclass+class TranscriptionResponse:+ text: str+ segments: Optional[List[TranscriptionSegment]] = None+ language: Optional[str] = None+ duration: Optional[float] = None+ speakers: Optional[List[str]] = None+ speaker_embeddings: Optional[Dict[str, List[float]]] = None+ provider: str = ""+ model: str = ""+ raw_response: Optional[Dict[str, Any]] = None+```++## Creating a Custom Connector++### Step 1: Create the Connector File++Create a new file in `src/services/transcription/connectors/`:++```python+# src/services/transcription/connectors/deepgram.py++"""+Deepgram transcription connector.++Integrates with Deepgram's speech-to-text API.+"""++import logging+import httpx+from typing import Dict, Any, Set++from ..base import (+ BaseTranscriptionConnector,+ TranscriptionCapability,+ TranscriptionRequest,+ TranscriptionResponse,+ TranscriptionSegment,+ ConnectorSpecifications,+)+from ..exceptions import TranscriptionError, ConfigurationError, ProviderError++logger = logging.getLogger(__name__)+++class DeepgramConnector(BaseTranscriptionConnector):+ """Connector for Deepgram speech-to-text API."""++ # Declare what capabilities this connector supports+ CAPABILITIES: Set[TranscriptionCapability] = {+ TranscriptionCapability.DIARIZATION,+ TranscriptionCapability.TIMESTAMPS,+ TranscriptionCapability.LANGUAGE_DETECTION,+ TranscriptionCapability.SPEAKER_COUNT_CONTROL,+ }++ PROVIDER_NAME = "deepgram"++ # Define provider constraints+ SPECIFICATIONS = ConnectorSpecifications(+ max_file_size_bytes=2 * 1024 * 1024 * 1024, # 2GB+ handles_chunking_internally=True, # Deepgram handles large files+ )++ def __init__(self, config: Dict[str, Any]):+ """+ Initialize the Deepgram connector.++ Args:+ config: Configuration dict with keys:+ - api_key: Deepgram API key (required)+ - model: Model name (default: 'nova-2')+ - language: Default language (optional)+ """+ super().__init__(config)++ self.api_key = config['api_key']+ self.model = config.get('model', 'nova-2')+ self.default_language = config.get('language')+ self.base_url = 'https://api.deepgram.com/v1/listen'++ def _validate_config(self) -> None:+ """Validate required configuration."""+ if not self.config.get('api_key'):+ raise ConfigurationError("api_key is required for Deepgram connector")++ def transcribe(self, request: TranscriptionRequest) -> TranscriptionResponse:+ """+ Transcribe audio using Deepgram.++ Args:+ request: Standardized transcription request++ Returns:+ TranscriptionResponse with segments and speaker information+ """+ try:+ # Build query parameters+ params = {+ 'model': self.model,+ 'punctuate': 'true',+ 'utterances': 'true',+ }++ # Add diarization if requested+ if request.diarize:+ params['diarize'] = 'true'+ if request.min_speakers:+ params['diarize_version'] = '2' # Required for speaker count+ if request.max_speakers:+ params['diarize_version'] = '2'++ # Add language+ language = request.language or self.default_language+ if language:+ params['language'] = language+ else:+ params['detect_language'] = 'true'++ # Prepare headers+ headers = {+ 'Authorization': f'Token {self.api_key}',+ 'Content-Type': request.mime_type or 'audio/wav',+ }++ # Read audio content+ audio_content = request.audio_file.read()++ # Make request+ timeout = httpx.Timeout(300.0, connect=60.0)++ with httpx.Client() as client:+ response = client.post(+ self.base_url,+ params=params,+ headers=headers,+ content=audio_content,+ timeout=timeout+ )+ response.raise_for_status()+ data = response.json()++ return self._parse_response(data)++ except httpx.HTTPStatusError as e:+ raise ProviderError(+ f"Deepgram request failed with status {e.response.status_code}",+ provider=self.PROVIDER_NAME,+ status_code=e.response.status_code+ ) from e++ except Exception as e:+ raise TranscriptionError(f"Deepgram transcription failed: {e}") from e++ def _parse_response(self, data: Dict[str, Any]) -> TranscriptionResponse:+ """Parse Deepgram response into standardized format."""+ segments = []+ speakers = set()++ # Extract from utterances (for diarization)+ results = data.get('results', {})+ utterances = results.get('utterances', [])++ for utt in utterances:+ speaker = f"SPEAKER_{utt.get('speaker', 0):02d}"+ speakers.add(speaker)++ segments.append(TranscriptionSegment(+ text=utt.get('transcript', ''),+ speaker=speaker,+ start_time=utt.get('start'),+ end_time=utt.get('end'),+ confidence=utt.get('confidence')+ ))++ # Get full transcript+ channels = results.get('channels', [])+ full_text = ''+ if channels:+ alternatives = channels[0].get('alternatives', [])+ if alternatives:+ full_text = alternatives[0].get('transcript', '')++ # Get detected language+ detected_language = None+ if channels:+ detected_language = channels[0].get('detected_language')++ return TranscriptionResponse(+ text=full_text,+ segments=segments if segments else None,+ speakers=sorted(list(speakers)) if speakers else None,+ language=detected_language,+ provider=self.PROVIDER_NAME,+ model=self.model,+ raw_response=data+ )++ def health_check(self) -> bool:+ """Check if Deepgram API is reachable."""+ try:+ with httpx.Client(timeout=10.0) as client:+ response = client.get(+ 'https://api.deepgram.com/v1/projects',+ headers={'Authorization': f'Token {self.api_key}'}+ )+ return response.status_code < 500+ except Exception:+ return False++ @classmethod+ def get_config_schema(cls) -> Dict[str, Any]:+ """Return JSON schema for configuration."""+ return {+ "type": "object",+ "required": ["api_key"],+ "properties": {+ "api_key": {+ "type": "string",+ "description": "Deepgram API key"+ },+ "model": {+ "type": "string",+ "default": "nova-2",+ "description": "Deepgram model name"+ },+ "language": {+ "type": "string",+ "description": "Default language code (e.g., 'en')"+ }+ }+ }+```++### Step 2: Register the Connector++Add your connector to the registry in `src/services/transcription/registry.py`:++```python+def _register_builtin_connectors(self):+ """Register all built-in connectors."""+ from .connectors.openai_whisper import OpenAIWhisperConnector+ from .connectors.openai_transcribe import OpenAITranscribeConnector+ from .connectors.asr_endpoint import ASREndpointConnector+ from .connectors.deepgram import DeepgramConnector # Add this++ self.register('openai_whisper', OpenAIWhisperConnector)+ self.register('openai_transcribe', OpenAITranscribeConnector)+ self.register('asr_endpoint', ASREndpointConnector)+ self.register('deepgram', DeepgramConnector) # Add this+```++### Step 3: Add Environment Configuration++Add config building logic to `_build_config_from_env()` in the registry:++```python+def _build_config_from_env(self, connector_name: str) -> Dict[str, Any]:+ # ... existing code ...++ elif connector_name == 'deepgram':+ return {+ 'api_key': os.environ.get('DEEPGRAM_API_KEY', ''),+ 'model': os.environ.get('DEEPGRAM_MODEL', 'nova-2'),+ 'language': os.environ.get('DEEPGRAM_LANGUAGE'),+ }++ # ... rest of method ...+```++### Step 4: Add Auto-Detection (Optional)++If you want auto-detection based on environment variables, add logic to `initialize_from_env()`:++```python+def initialize_from_env(self) -> BaseTranscriptionConnector:+ # ... existing code ...++ # Add before the default case:+ elif os.environ.get('DEEPGRAM_API_KEY'):+ connector_name = 'deepgram'+ logger.info("Auto-detected Deepgram from DEEPGRAM_API_KEY")++ # ... rest of method ...+```++## Usage++### Manual Selection++Explicitly select the connector in the `.env` file:++```bash+TRANSCRIPTION_CONNECTOR=deepgram+DEEPGRAM_API_KEY=your-api-key+DEEPGRAM_MODEL=nova-2+```++### Auto-Detection++If you added auto-detection logic, just setting the API key is enough:++```bash+DEEPGRAM_API_KEY=your-api-key+```++## Best Practices++### Error Handling++Use the appropriate exception types:++```python+from ..exceptions import TranscriptionError, ConfigurationError, ProviderError++# For configuration issues (missing API key, invalid settings)+raise ConfigurationError("api_key is required")++# For provider-specific errors (API errors, rate limits)+raise ProviderError(+ "Rate limit exceeded",+ provider=self.PROVIDER_NAME,+ status_code=429+)++# For general transcription failures+raise TranscriptionError("Failed to process audio")+```++### Logging++Use structured logging for debugging:++```python+import logging+logger = logging.getLogger(__name__)++logger.info(f"Sending request to {self.base_url}")+logger.debug(f"Request params: {params}")+logger.error(f"Transcription failed: {error}")+```++### Capability Declaration++Only declare capabilities your connector actually supports:++```python+CAPABILITIES: Set[TranscriptionCapability] = {+ TranscriptionCapability.DIARIZATION, # If you support speaker labels+ TranscriptionCapability.TIMESTAMPS, # If you return timing info+ TranscriptionCapability.LANGUAGE_DETECTION,# If you can detect language+ TranscriptionCapability.SPEAKER_COUNT_CONTROL, # If you support min/max speakers+}+```++### Specifications++Accurately declare your provider's constraints:++```python+SPECIFICATIONS = ConnectorSpecifications(+ max_file_size_bytes=100 * 1024 * 1024, # Provider's file size limit+ max_duration_seconds=14400, # 4 hours max+ handles_chunking_internally=True, # Set True if provider handles large files+ unsupported_codecs=frozenset({'opus'}), # Codecs to convert before sending+)+```++### Config Schema++Provide a config schema for documentation and validation:++```python+@classmethod+def get_config_schema(cls) -> Dict[str, Any]:+ return {+ "type": "object",+ "required": ["api_key"],+ "properties": {+ "api_key": {+ "type": "string",+ "description": "Your API key"+ },+ # ... other properties+ }+ }+```++## Testing Your Connector++Create a test file to verify your connector works:++```python+# tests/test_deepgram_connector.py++import os+import io+import pytest+from src.services.transcription.connectors.deepgram import DeepgramConnector+from src.services.transcription.base import TranscriptionRequest++def connector():+ config = {+ 'api_key': os.environ.get('DEEPGRAM_API_KEY', 'test-key'),+ 'model': 'nova-2'+ }+ return DeepgramConnector(config)++def test_config_validation():+ with pytest.raises(ConfigurationError):+ DeepgramConnector({}) # Missing api_key++def test_capabilities(connector):+ assert connector.supports_diarization+ assert TranscriptionCapability.TIMESTAMPS in connector.CAPABILITIES++def test_transcribe(connector):+ # Load a test audio file+ with open('tests/fixtures/test_audio.wav', 'rb') as f:+ request = TranscriptionRequest(+ audio_file=f,+ filename='test.wav',+ mime_type='audio/wav',+ diarize=True+ )+ response = connector.transcribe(request)++ assert response.text+ assert response.provider == 'deepgram'+```++## Contributing++If you create a connector for a popular provider, consider contributing it back to the project:++1. Fork the repository+2. Add your connector following this guide+3. Add tests and documentation+4. Submit a pull request++Popular providers we'd love to see connectors for:+- Deepgram+- AssemblyAI+- Google Cloud Speech-to-Text+- Amazon Transcribe+- Azure Speech Servicesdiff --git a/docs/getting-started/installation.md b/docs/getting-started/installation.mdindex 5223367e..2e94acc3 100644--- a/docs/getting-started/installation.md+++ b/docs/getting-started/installation.md@@ -6,13 +6,13 @@ This comprehensive guide covers deploying Speakr for production use, including dBefore diving into installation, it's helpful to understand how Speakr works. The application integrates with external APIs for two main purposes: transcription services that convert your audio to text, and text generation services that power features like summaries, titles, and interactive chat. Speakr is designed to be flexible, supporting both cloud-based services like OpenAI and self-hosted solutions running on your own infrastructure.-Speakr uses a **connector-based architecture** for transcription services, providing a unified interface regardless of which provider you choose. Three connectors are available:+Speakr uses a **connector-based architecture** for transcription services, providing a unified interface regardless of which provider you choose. Three connectors are currently available:1. **ASR Endpoint** (Recommended for best quality) - For self-hosted solutions like WhisperX that offer GPU-accelerated transcription with superior accuracy, voice profiles, and speaker diarization2. **OpenAI Transcribe** - Uses `gpt-4o-transcribe-diarize` for cloud-based speaker diarization without requiring additional containers-3. **OpenAI Whisper** - Legacy whisper-1 model for basic transcription+3. **OpenAI Whisper** - Legacy whisper-1 model for basic transcription using any legacy OpenAI compatible whisper API-The connector is automatically selected based on your configuration. For text generation, Speakr uses the OpenAI Chat Completions API format with the `/chat/completions` endpoint, which is widely supported across different AI providers.+The connector can be specified or inferred based on your configuration. For text generation, Speakr uses the OpenAI Chat Completions API format with the `/chat/completions` endpoint, which is widely supported across different AI providers.## Prerequisites