Speakr is a personal, self-hosted web application designed for transcribing audio recordings
Python
Preserve original audio and recover Tencent COS transcription tasks
ZOLO67aef54ce0c8
diff --git a/config/env.transcription.example b/config/env.transcription.exampleindex a289dd2b..2045b05a 100644--- a/config/env.transcription.example+++ b/config/env.transcription.example@@ -9,6 +9,8 @@# TENCENT_ASR_TIMEOUT=1800# TENCENT_ASR_POLL_INTERVAL=5# TENCENT_ASR_HOTWORD_ID=your-hotword-table-id+# TENCENT_ASR_STAGING_PREFIX=tencent-asr+# TENCENT_ASR_STATE_DIR=/data/instance/tencent-tasks# Files over 5 MiB require Speakr S3 storage; see docs/advanced/tencent-asr.md.## Speakr supports multiple transcription providers through a connector-baseddiff --git a/docs/advanced/tencent-asr.md b/docs/advanced/tencent-asr.mdindex b6d42108..94a837f5 100644--- a/docs/advanced/tencent-asr.md+++ b/docs/advanced/tencent-asr.md@@ -36,7 +36,11 @@ in seconds (maximum timeout 10800).## Large files and storage-Files up to 5 MiB are sent directly. Larger files use the existing Speakr S3+The connector declares original-audio passthrough, so Speakr skips audio+preprocessing (including forced WebM conversion) and the MP3 error-retry path+for it. This applies to audio recordings; video extraction is a separate app+feature. The connector never compresses or splits audio. Files up to 5 MiB are sent+directly. Larger files use the existing Speakr S3backend: configure `S3_BUCKET_NAME`, `S3_REGION`, `S3_ENDPOINT_URL`,`S3_ACCESS_KEY_ID` and `S3_SECRET_ACCESS_KEY`. The endpoint and signed downloadURL must be reachable by Tencent over the public internet; a NAS-only address@@ -45,20 +49,73 @@ S3-compatible client, not a separate COS SDK; verify your chosen storageprovider's compatibility before relying on it. Local recording storage canremain selected (`FILE_STORAGE_BACKEND=local`).-Staging objects use the `tencent-asr/` prefix and 24-hour signed URLs. They are+Tencent COS example (use your own bucket, credentials and region):++```env+FILE_STORAGE_BACKEND=local+S3_BUCKET_NAME=your-bucket-1234567890+S3_REGION=ap-hongkong+S3_ENDPOINT_URL=https://cos.ap-hongkong.myqcloud.com+S3_ACCESS_KEY_ID=your-cos-secret-id+S3_SECRET_ACCESS_KEY=your-cos-secret-key+S3_USE_PATH_STYLE=false+S3_VERIFY_SSL=true+TENCENT_ASR_STAGING_PREFIX=recordings/speakr+```++These storage credentials need object upload, read and delete permissions.+They may differ from the ASR credentials. COS uses virtual-host addressing;+the backend disables optional AWS checksum trailers for COS compatibility.+Staging uploads explicitly set a private object ACL, even for an existing+image bucket. Check that no bucket policy independently grants public access.+There is no requirement to store audio with Alibaba or AWS.++Staging objects use the configurable `TENCENT_ASR_STAGING_PREFIX` (default+`tencent-asr/`) and 24-hour signed URLs. They aredeleted after a terminal task result. Configure a bucket lifecycle rule toexpire this prefix after at least two days: timed-out tasks, interruptedworkers, and uncertain submissions retain their objects so Tencent can finishdownloading them. These objects contain audio; storage charges may apply.-The API limits URL jobs to 1 GiB and five hours.+The API limits URL jobs to 1 GiB and five hours. The operator should configure+expiration and incomplete-multipart-upload cleanup on this prefix only;+Speakr does not change bucket lifecycle policies automatically.++Live validation used an unchanged 9.64 MB WebM recording (598.74 seconds) in+a private COS object: downloaded bytes matched SHA-256, anonymous access was+denied, Tencent returned 176 timestamped segments, and deletion was verified.+This is a functional test, not an accuracy or performance benchmark.## Failures and retries-Transient polling failures retry the same task. **Submission is never retried-by the connector.** Task IDs are logged and included in timeout errors. There-is no automatic resume across worker restarts: check the Tencent ASR console-before manually retrying a failed/timeout recording, since a new transcription-can create another billable task. A network timeout during submission can mean-Tencent accepted it even though no task ID reached Speakr.+Checkpoints are stored in `tencent-tasks/` under the Flask instance directory,+or in `TENCENT_ASR_STATE_DIR` when set. Keep that directory on persistent storage.+It contains task IDs, object locators and completed transcription results, but+not credentials or signed URLs. Treat it as private application data. Every+worker serving the same instance must share it; per-file OS locks prevent+concurrent submission. Distributed deployments need filesystem lock semantics+and are not validated by this connector.++An identical audio file, ASR account and recognition parameters reuse the saved+task. After a polling failure or worker restart, retrying the recording resumes+polling rather than submitting again. Completed results are reused; results are+saved before deleting staging audio. Failed cleanup retains its locator and is+retried when that recording is processed again, with bucket lifecycle cleanup+as a fallback. There is no new background polling thread: recovery occurs when+Speakr's job queue or the user invokes transcription again.++**Ambiguous submissions never retry automatically.** If the worker dies after+starting submission but before saving the returned task ID, the checkpoint+blocks further submission until an operator reconciles it with Tencent's+console. Tasks older than 23 hours also stop instead of querying a potentially+reused task ID (Tencent retains results for 24 hours). A known provider rejection+can be retried after fixing configuration or quota. Failed remote tasks require+operator review. Checkpoints are not automatically expired to avoid silently+charging for the same recording again.++To intentionally request fresh recognition with unchanged settings, stop work+on that recording, confirm the old remote task is finished and clean up any+staging locator, then back up/remove only its matching checkpoint JSON. This+explicit operation can create another billable task. A user-facing reset action+is not included yet.API reference: [CreateRecTask](https://cloud.tencent.com/document/api/1093/37823).diff --git a/src/services/storage/s3.py b/src/services/storage/s3.pyindex e5f0c3d0..a60a0383 100644--- a/src/services/storage/s3.py+++ b/src/services/storage/s3.py@@ -19,6 +19,11 @@ def _is_aliyun_oss_endpoint(endpoint_url: Optional[str]) -> bool:return hostname.endswith('.aliyuncs.com')+def _is_tencent_cos_endpoint(endpoint_url: Optional[str]) -> bool:+ hostname = urlparse(endpoint_url or '').hostname or ''+ return hostname.startswith('cos.') and hostname.endswith('.myqcloud.com')++def _botocore_supports_request_checksum() -> bool:"""True if the installed botocore accepts request_checksum_calculation.@@ -78,17 +83,19 @@ class S3StorageBackend:'signature_version': 's3v4','s3': {'addressing_style': (- 'virtual' if _is_aliyun_oss_endpoint(resolved_endpoint)+ 'virtual' if (_is_aliyun_oss_endpoint(resolved_endpoint) or _is_tencent_cos_endpoint(resolved_endpoint))else 'path' if self.use_path_style else 'auto'),},}- # Aliyun OSS rejects the aws-chunked transfer encoding triggered by- # request checksums, so opt out for OSS endpoints. AWS/MinIO keep the+ # OSS/COS need compatibility with the older S3 upload wire format;+ # disable optional checksum trailers. AWS/MinIO keep the# botocore default. Guarded by the version that introduced the option.- if (_is_aliyun_oss_endpoint(resolved_endpoint)+ if ((_is_aliyun_oss_endpoint(resolved_endpoint) or _is_tencent_cos_endpoint(resolved_endpoint))and _botocore_supports_request_checksum()):config_kwargs['request_checksum_calculation'] = 'when_required'+ if _is_tencent_cos_endpoint(resolved_endpoint):+ config_kwargs['response_checksum_validation'] = 'when_required'client_kwargs['config'] = Config(**config_kwargs)return boto3.client(**client_kwargs)@@ -126,9 +133,11 @@ class S3StorageBackend:stat = self.stat(StorageLocator(scheme='s3', raw=self.build_locator(key), bucket=self.bucket, key=key))return StoredObject(locator=self.build_locator(key), key=key, size=stat.size, content_type=stat.content_type, etag=stat.etag)- def upload_local_file(self, local_path: str, key: str, content_type: Optional[str] = None, metadata: Optional[dict] = None, delete_source: bool = False) -> StoredObject:+ def upload_local_file(self, local_path: str, key: str, content_type: Optional[str] = None, metadata: Optional[dict] = None, delete_source: bool = False, acl: Optional[str] = None) -> StoredObject:client = self._get_client()extra = {}+ if acl:+ extra['ACL'] = aclif content_type:extra['ContentType'] = content_typeif metadata:diff --git a/src/services/transcription/base.py b/src/services/transcription/base.pyindex 287eb5d4..da06ddec 100644--- a/src/services/transcription/base.py+++ b/src/services/transcription/base.py@@ -51,6 +51,8 @@ class ConnectorSpecifications:# Codecs this connector doesn't support (removed from defaults)# Merged with AUDIO_UNSUPPORTED_CODECS env varunsupported_codecs: Optional[FrozenSet[str]] = None+ # Provider accepts original audio; do not transcode or repair its container.+ preserve_original_audio: bool = False# Default specifications for connectors that don't define their owndiff --git a/src/services/transcription/connectors/tencent_asr.py b/src/services/transcription/connectors/tencent_asr.pyindex c08e99d6..57fe19be 100644--- a/src/services/transcription/connectors/tencent_asr.py+++ b/src/services/transcription/connectors/tencent_asr.py@@ -6,6 +6,7 @@ import hmacimport ioimport jsonimport logging+import osimport tempfileimport timefrom datetime import datetime, timezone@@ -17,6 +18,7 @@ import httpxfrom ..base import (BaseTranscriptionConnector, ConnectorSpecifications,TranscriptionCapability, TranscriptionResponse, TranscriptionSegment)from ..exceptions import ConfigurationError, ProviderError, TranscriptionError+from ..tencent_state import locked_checkpointlogger = logging.getLogger(__name__)HOST = 'asr.tencentcloudapi.com'@@ -31,7 +33,7 @@ class TencentASRConnector(BaseTranscriptionConnector):TranscriptionCapability.TIMESTAMPS, TranscriptionCapability.HOTWORDS}SPECIFICATIONS = ConnectorSpecifications(max_file_size_bytes=FILE_LIMIT, max_duration_seconds=18000,- handles_chunking_internally=True)+ handles_chunking_internally=True, preserve_original_audio=True)def _validate_config(self):self.secret_id = self.config.get('secret_id', '').strip()@@ -50,6 +52,10 @@ class TencentASRConnector(BaseTranscriptionConnector):raise ConfigurationError('Tencent requires 0 < poll_interval <= timeout <= 10800 seconds')self.hotword_id = self.config.get('hotword_id', '').strip()self.default_diarize = self.config.get('diarize', True)+ self.state_dir = self.config.get('state_dir')+ self.staging_prefix = (self.config.get('staging_prefix') or 'tencent-asr').strip('/')+ if not self.staging_prefix or '..' in self.staging_prefix.split('/'):+ raise ConfigurationError('Tencent staging prefix must be a nonempty object prefix without ..')def _headers(self, action, body, timestamp):date = datetime.fromtimestamp(timestamp, timezone.utc).strftime('%Y-%m-%d')@@ -92,7 +98,8 @@ class TencentASRConnector(BaseTranscriptionConnector):elif code.startswith(('AuthFailure', 'UnauthorizedOperation')):hint = ' Check credentials, CAM permissions and system clock.'raise ProviderError(f'Tencent {action}: {code}.{hint}', provider=self.PROVIDER_NAME,- status_code=429 if code.startswith('RequestLimitExceeded') else 400)+ status_code=(429 if code.startswith('RequestLimitExceeded')+ else 500 if code.startswith('InternalError') else 400))if not isinstance(data['Data'], dict):raise ValueError('Invalid task data')return data['Data']@@ -115,15 +122,19 @@ class TencentASRConnector(BaseTranscriptionConnector):params['HotwordId'] = self.hotword_idreturn params- def _stage(self, path, request):+ def _stage(self, path, request, state=None, save=None):from src.services.storage import get_storage_service, parse_locatorstorage = get_storage_service()if not storage.s3:raise ConfigurationError('Tencent audio over 5 MiB requires Speakr S3 storage with a publicly reachable endpoint')- key = 'tencent-asr/' + uuid4().hex + Path(request.filename).suffix+ key = self.staging_prefix + '/' + uuid4().hex + Path(request.filename).suffixlocator = parse_locator(storage.s3.build_locator(key))+ if state is not None:+ state['staging_locator'] = storage.s3.build_locator(key)+ state['staging_endpoint'] = storage.s3.endpoint_url+ save() # Remember the key even if the worker dies during upload.try:- storage.s3.upload_local_file(path, key, content_type=request.mime_type or 'application/octet-stream')+ storage.s3.upload_local_file(path, key, content_type=request.mime_type or 'application/octet-stream', acl='private')url = storage.s3.presign_get_url(locator, expires_seconds=86400)return storage.s3, locator, urlexcept Exception:@@ -134,8 +145,10 @@ class TencentASRConnector(BaseTranscriptionConnector):def _cleanup(storage, locator):try:storage.delete(locator, missing_ok=True)+ return Trueexcept Exception:logger.warning('Tencent ASR staging cleanup failed; configure a tencent-asr/ bucket lifecycle rule')+ return Falsedef _poll(self, client, task_id):deadline = time.monotonic() + self.poll_timeout@@ -154,63 +167,105 @@ class TencentASRConnector(BaseTranscriptionConnector):raisetime.sleep(min(self.poll_interval, max(0, deadline - time.monotonic())))raise TranscriptionError(f'Tencent task {task_id}: polling timed out. The remote task may still be running; '- 'a new transcription submits a new billable task.')+ 'retry the same recording to resume polling the saved task.')++ def _state_directory(self):+ if self.state_dir:+ return self.state_dir+ from flask import current_app+ return os.path.join(current_app.instance_path, 'tencent-tasks')++ def _cleanup_checkpoint(self, state, save):+ if not state.get('staging_locator'):+ return+ from src.services.storage import get_storage_service, parse_locator+ storage = get_storage_service().s3+ if not storage or storage.endpoint_url != state.get('staging_endpoint'):+ logger.warning('Tencent staging cleanup deferred: original storage endpoint is unavailable')+ return+ if self._cleanup(storage, parse_locator(state['staging_locator'])):+ state.pop('staging_locator', None)+ state.pop('staging_endpoint', None)+ save()def transcribe(self, request):params = self._parameters(request)- storage = locator = None- submitted = terminal = Falsetry:- try:- request.audio_file.seek(0)- except (AttributeError, OSError, io.UnsupportedOperation):- pass- data = request.audio_file.read(INLINE_LIMIT + 1)- if not data:+ request.audio_file.seek(0)+ except (AttributeError, OSError, io.UnsupportedOperation):+ pass+ # Spool without transcoding; hash original bytes to identify safe retries.+ with tempfile.TemporaryDirectory(prefix='speakr_tencent_') as directory:+ path = Path(directory) / 'audio'+ digest = hashlib.sha256()+ size = 0+ with path.open('wb') as output:+ while True:+ chunk = request.audio_file.read(1024 * 1024)+ if not chunk:+ break+ size += len(chunk)+ if size > FILE_LIMIT:+ raise TranscriptionError('Tencent ASR: audio exceeds 1 GiB URL limit')+ output.write(chunk)+ digest.update(chunk)+ if not size:raise TranscriptionError('Tencent ASR: empty audio file')- if len(data) <= INLINE_LIMIT:- params.update(SourceType=1, Data=base64.b64encode(data).decode(), DataLen=len(data))- else:- with tempfile.TemporaryDirectory(prefix='speakr_tencent_') as directory:- path = str(Path(directory) / 'audio')- size = len(data)- with open(path, 'wb') as output:- output.write(data)- while True:- chunk = request.audio_file.read(1024 * 1024)- if not chunk:- break- size += len(chunk)- if size > FILE_LIMIT:- raise TranscriptionError('Tencent ASR: audio exceeds 1 GiB URL limit')- output.write(chunk)- storage, locator, url = self._stage(path, request)- params.update(SourceType=0, Url=url)- with httpx.Client(timeout=httpx.Timeout(60, connect=15), follow_redirects=False) as client:- # Mark before POST: a transport failure can occur after acceptance.- submitted = True- try:- result = self._call(client, 'CreateRecTask', params)- except ProviderError as exc:- # A definite rejection cannot be downloading the staged object.- terminal = bool(exc.status_code and 400 <= exc.status_code < 500)- raise- task_id = result.get('TaskId')- if not isinstance(task_id, int) or isinstance(task_id, bool):- raise ProviderError('Tencent submit returned no task ID', provider=self.PROVIDER_NAME)- logger.info('Tencent ASR task submitted: %s', task_id)- result = self._poll(client, task_id)- terminal = True- if result['Status'] == 3:- raise TranscriptionError(f'Tencent task {task_id} failed; check the ASR console for details')- return self._parse_result(result, params['EngineModelType'], request.diarize)- except httpx.HTTPError:- raise TranscriptionError('Tencent ASR network error. Submission was not retried: check the ASR console '- 'before retrying to avoid duplicate charges.') from None- finally:- # On timeout or ambiguous POST failure, Tencent may still need the URL.- if storage and (terminal or not submitted):- self._cleanup(storage, locator)+ identity = json.dumps([self.secret_id, digest.hexdigest(), params], sort_keys=True)+ fingerprint = hashlib.sha256(identity.encode()).hexdigest()+ with locked_checkpoint(self._state_directory(), fingerprint) as (state, save):+ state.update(source_sha256=digest.hexdigest(), source_bytes=size)+ save()+ if state.get('status') in ('completed', 'failed'):+ self._cleanup_checkpoint(state, save)+ if state['status'] == 'failed':+ raise TranscriptionError('Saved Tencent task failed; inspect the checkpoint before explicitly starting another billable task')+ return self._parse_result(state['result'], params['EngineModelType'], request.diarize)+ if state.get('status') == 'submitting':+ raise TranscriptionError('Tencent submission outcome is uncertain. Check the ASR console and saved checkpoint; automatic resubmission is blocked to prevent duplicate charges')+ with httpx.Client(timeout=httpx.Timeout(60, connect=15), follow_redirects=False) as client:+ task_id = state.get('task_id')+ if task_id:+ if time.time() - state['submitted_at'] >= 23 * 3600:+ raise TranscriptionError('Saved Tencent task is nearing its 24-hour expiry; automatic resubmission is blocked. Check the ASR console')+ logger.info('Resuming Tencent ASR task: %s', task_id)+ else:+ # A previous interrupted upload has no ASR task yet.+ self._cleanup_checkpoint(state, save)+ if state.get('staging_locator'):+ raise TranscriptionError('Previous Tencent staging cleanup is pending; check object storage before retrying')+ state.update(status='preparing')+ save()+ if size <= INLINE_LIMIT:+ params.update(SourceType=1, Data=base64.b64encode(path.read_bytes()).decode(), DataLen=size)+ else:+ _, _, url = self._stage(str(path), request, state, save)+ params.update(SourceType=0, Url=url)+ state.update(status='submitting', submitted_at=time.time())+ save() # Crash after this point must not silently submit twice.+ try:+ result = self._call(client, 'CreateRecTask', params)+ except ProviderError as exc:+ if exc.status_code and 400 <= exc.status_code < 500:+ state['status'] = 'rejected'+ save()+ self._cleanup_checkpoint(state, save)+ raise+ except httpx.HTTPError:+ raise TranscriptionError('Tencent ASR network error. Submission was not retried; its outcome is uncertain. Check the ASR console') from None+ task_id = result.get('TaskId')+ if not isinstance(task_id, int) or isinstance(task_id, bool):+ raise ProviderError('Tencent submit returned no task ID; automatic resubmission is blocked', provider=self.PROVIDER_NAME)+ state.update(status='pending', task_id=task_id)+ save()+ logger.info('Tencent ASR task submitted: %s', task_id)+ result = self._poll(client, task_id)+ state.update(status='completed' if result['Status'] == 2 else 'failed', result=result)+ save() # Keep result before deleting the only cloud audio copy.+ self._cleanup_checkpoint(state, save)+ if result['Status'] == 3:+ raise TranscriptionError(f'Tencent task {task_id} failed; check the ASR console for details')+ return self._parse_result(result, params['EngineModelType'], request.diarize)def _parse_result(self, result, model, diarize):segments = []diff --git a/src/services/transcription/registry.py b/src/services/transcription/registry.pyindex 8c9a1397..e0b01193 100644--- a/src/services/transcription/registry.py+++ b/src/services/transcription/registry.py@@ -362,6 +362,8 @@ class ConnectorRegistry:'timeout': os.environ.get('TENCENT_ASR_TIMEOUT', '1800'),'poll_interval': os.environ.get('TENCENT_ASR_POLL_INTERVAL', '5'),'hotword_id': os.environ.get('TENCENT_ASR_HOTWORD_ID', ''),+ 'state_dir': os.environ.get('TENCENT_ASR_STATE_DIR') or None,+ 'staging_prefix': os.environ.get('TENCENT_ASR_STAGING_PREFIX', 'tencent-asr'),'diarize': os.environ.get('ASR_DIARIZE', 'true').lower() == 'true',}diff --git a/src/services/transcription/tencent_state.py b/src/services/transcription/tencent_state.pynew file mode 100644index 00000000..353650f3--- /dev/null+++ b/src/services/transcription/tencent_state.py@@ -0,0 +1,57 @@+"""Durable Tencent task checkpoints; one OS lock per audio/options fingerprint."""+import json+import os+from contextlib import contextmanager+from pathlib import Path+from uuid import uuid4++from .exceptions import TranscriptionError+++@contextmanager+def locked_checkpoint(directory, fingerprint):+ root = Path(directory)+ root.mkdir(parents=True, exist_ok=True, mode=0o700)+ path = root / (fingerprint + '.json')+ with (root / (fingerprint + '.lock')).open('a+b') as lock:+ lock.seek(0)+ if os.name == 'nt':+ import msvcrt+ if not lock.read(1):+ lock.write(b'0')+ lock.flush()+ lock.seek(0)+ try:+ msvcrt.locking(lock.fileno(), msvcrt.LK_NBLCK, 1)+ except OSError:+ raise TranscriptionError('This Tencent recording is already being processed; wait before retrying') from None+ else:+ import fcntl+ try:+ fcntl.flock(lock, fcntl.LOCK_EX | fcntl.LOCK_NB)+ except BlockingIOError:+ raise TranscriptionError('This Tencent recording is already being processed; wait before retrying') from None+ try:+ try:+ state = json.loads(path.read_text()) if path.exists() else {}+ except (ValueError, OSError):+ raise TranscriptionError('Tencent checkpoint cannot be read; refusing a potentially duplicate submission') from None++ def save():+ temporary = root / (fingerprint + '.' + uuid4().hex + '.tmp')+ try:+ fd = os.open(temporary, os.O_WRONLY | os.O_CREAT | os.O_EXCL, 0o600)+ with os.fdopen(fd, 'w', encoding='utf-8') as output:+ json.dump(state, output, ensure_ascii=False)+ output.flush()+ os.fsync(output.fileno())+ os.replace(temporary, path)+ finally:+ temporary.unlink(missing_ok=True)+ yield state, save+ finally:+ if os.name == 'nt':+ lock.seek(0)+ msvcrt.locking(lock.fileno(), msvcrt.LK_UNLCK, 1)+ else:+ fcntl.flock(lock, fcntl.LOCK_UN)diff --git a/src/tasks/processing.py b/src/tasks/processing.pyindex 3100913c..134f1c59 100644--- a/src/tasks/processing.py+++ b/src/tasks/processing.py@@ -2153,7 +2153,7 @@ def transcribe_with_connector(app_context, recording_id, filepath, original_file])# Only retry with MP3 conversion on first attempt for format errors- if attempt == 0 and is_format_error and not converted_filepath:+ if attempt == 0 and is_format_error and not converted_filepath and not getattr(connector_specs, 'preserve_original_audio', False):current_app.logger.warning(f"Transcription failed with possible format error: {e}")current_app.logger.info(f"Attempting MP3 conversion and retry...")diff --git a/src/utils/audio_conversion.py b/src/utils/audio_conversion.pyindex 8b92616b..77202fba 100644--- a/src/utils/audio_conversion.py+++ b/src/utils/audio_conversion.py@@ -151,6 +151,13 @@ def convert_if_needed(# Get original file sizeoriginal_size = os.path.getsize(filepath)+ if getattr(connector_specs, 'preserve_original_audio', False):+ import mimetypes+ return ConversionResult(+ output_path=filepath,+ mime_type=mimetypes.guess_type(original_filename)[0] or 'application/octet-stream',+ was_converted=False, was_compressed=False,+ original_size=original_size, final_size=original_size)# Probe if codec info not providedif codec_info is None:diff --git a/tests/test_tencent_asr.py b/tests/test_tencent_asr.pyindex 27cb61ab..f3f84317 100644--- a/tests/test_tencent_asr.py+++ b/tests/test_tencent_asr.py@@ -13,9 +13,9 @@ from src.services.transcription.exceptions import ConfigurationError, ProviderEr@pytest.fixture-def connector():+def connector(tmp_path):return mod.TencentASRConnector({'secret_id': 'test-id', 'secret_key': 'test-secret',- 'poll_interval': .001, 'timeout': .05})+ 'poll_interval': .001, 'timeout': .5, 'state_dir': str(tmp_path)})def request(data=b'audio', **kwargs):@@ -113,10 +113,11 @@ def test_large_file_uses_private_staging_and_cleans_terminal(connector, monkeypamonkeypatch.setattr(mod, 'INLINE_LIMIT', 5)storage = Mock()locator = object()- def stage(path, req):+ def stage(path, req, *args):assert mod.Path(path).read_bytes() == b'123456'return storage, locator, 'https://bucket/file?signed=PRIVATE'monkeypatch.setattr(connector, '_stage', stage)+ monkeypatch.setattr(connector, '_cleanup_checkpoint', lambda state, save: storage.delete(locator, missing_ok=True) if state.get('status') in ('completed','failed') else None)calls = transport(monkeypatch, [{'Data': {'TaskId': 42}}, {'Data': {'Status': status, 'Result': 'text'}}])if status == 2:assert connector.transcribe(request(b'123456')).text == 'text'@@ -199,6 +200,7 @@ def test_submission_rejection_removes_staging(connector, monkeypatch):monkeypatch.setattr(mod, 'INLINE_LIMIT', 5)storage = Mock()monkeypatch.setattr(connector, '_stage', lambda *args: (storage, 'locator', 'https://bucket/file'))+ monkeypatch.setattr(connector, '_cleanup_checkpoint', lambda state, save: storage.delete('locator', missing_ok=True) if state.get('status') == 'rejected' else None)transport(monkeypatch, [{'Error': {'Code': 'AuthFailure.SecretIdNotFound'}}])with pytest.raises(ProviderError):connector.transcribe(request(b'123456'))diff --git a/tests/test_tencent_recovery.py b/tests/test_tencent_recovery.pynew file mode 100644index 00000000..19cc6153--- /dev/null+++ b/tests/test_tencent_recovery.py@@ -0,0 +1,141 @@+"""Recovery and cross-process duplication tests for paid Tencent ASR tasks."""+import io+import json+import subprocess+import sys+from unittest.mock import Mock++import httpx+import pytest++from src.services.transcription.base import TranscriptionRequest+from src.services.transcription.connectors.tencent_asr import TencentASRConnector+from src.services.transcription.exceptions import TranscriptionError+from src.services.transcription.tencent_state import locked_checkpoint+++def connector(tmp_path):+ return TencentASRConnector({'secret_id':'id','secret_key':'secret','state_dir':str(tmp_path)})+++def audio():+ return TranscriptionRequest(audio_file=io.BytesIO(b'original bytes'),filename='meeting.webm',diarize=True)+++def test_new_worker_resumes_saved_task(tmp_path):+ first=connector(tmp_path)+ first._call=Mock(return_value={'TaskId':123})+ first._poll=Mock(side_effect=TranscriptionError('network interrupted'))+ with pytest.raises(TranscriptionError):+ first.transcribe(audio())+ second=connector(tmp_path)+ second._call=Mock(side_effect=AssertionError('must not resubmit'))+ second._poll=Mock(return_value={'Status':2,'Result':'recovered'})+ assert second.transcribe(audio()).text=='recovered'+ assert second._poll.call_args.args[1]==123+ third=connector(tmp_path)+ third._call=Mock(side_effect=AssertionError('must use saved result'))+ third._poll=Mock(side_effect=AssertionError('must use saved result'))+ assert third.transcribe(audio()).text=='recovered'+++def test_uncertain_post_blocks_new_worker(tmp_path):+ first=connector(tmp_path)+ first._call=Mock(side_effect=httpx.ReadTimeout('sensitive URL'))+ with pytest.raises(TranscriptionError,match='uncertain'):+ first.transcribe(audio())+ second=connector(tmp_path)+ second._call=Mock(side_effect=AssertionError('duplicate submission'))+ with pytest.raises(TranscriptionError,match='automatic resubmission is blocked'):+ second.transcribe(audio())+++def test_expired_task_never_reused_or_resubmitted(tmp_path):+ first=connector(tmp_path)+ first._call=Mock(return_value={'TaskId':123})+ first._poll=Mock(side_effect=TranscriptionError('interrupted'))+ with pytest.raises(TranscriptionError): first.transcribe(audio())+ path=next(tmp_path.glob('*.json'))+ state=json.loads(path.read_text())+ state['submitted_at']=0+ path.write_text(json.dumps(state))+ with pytest.raises(TranscriptionError,match='expiry'):+ connector(tmp_path).transcribe(audio())+++def test_corrupt_checkpoint_does_not_submit(tmp_path):+ path=tmp_path/'fingerprint.json'+ path.write_text('{')+ with pytest.raises(TranscriptionError,match='cannot be read'):+ with locked_checkpoint(tmp_path,'fingerprint'): pass+++def test_real_process_lock_and_crash_release(tmp_path):+ program='''import sys,time+from src.services.transcription.tencent_state import locked_checkpoint+with locked_checkpoint(sys.argv[1], 'same-audio') as (state, save):+ state.update(status='submitting')+ save()+ print('LOCKED',flush=True)+ time.sleep(120)+'''+ child=subprocess.Popen([sys.executable,'-c',program,str(tmp_path)],stdout=subprocess.PIPE,text=True)+ try:+ assert child.stdout.readline().strip()=='LOCKED'+ with pytest.raises(TranscriptionError,match='already being processed'):+ with locked_checkpoint(tmp_path,'same-audio'): pass+ finally:+ child.kill()+ child.wait(timeout=10)+ with locked_checkpoint(tmp_path,'same-audio') as (state,save):+ assert state['status']=='submitting'+++def test_failed_cleanup_preserves_locator_for_retry(tmp_path,monkeypatch):+ import src.services.storage as storage_module+ from types import SimpleNamespace+ storage=Mock(endpoint_url='https://cos.ap-hongkong.myqcloud.com')+ monkeypatch.setattr(storage_module,'get_storage_service',lambda:SimpleNamespace(s3=storage))+ c=connector(tmp_path)+ state={'status':'completed','staging_locator':'s3://test/recordings/audio.webm',+ 'staging_endpoint':storage.endpoint_url}+ save=Mock()+ storage.delete.side_effect=RuntimeError('network')+ c._cleanup_checkpoint(state,save)+ assert 'staging_locator' in state+ storage.delete.side_effect=None+ c._cleanup_checkpoint(state,save)+ assert 'staging_locator' not in state+ save.assert_called_once()+++def test_cos_client_configuration_and_private_upload(tmp_path):+ from unittest.mock import patch+ from src.services.storage.s3 import S3StorageBackend+ backend=S3StorageBackend(bucket='test-123',region='ap-hongkong',+ endpoint_url='https://cos.ap-hongkong.myqcloud.com')+ with patch('boto3.client') as factory:+ backend._create_client()+ config=factory.call_args.kwargs['config']+ assert config.s3['addressing_style']=='virtual'+ assert config.request_checksum_calculation=='when_required'+ assert config.response_checksum_validation=='when_required'+ path=tmp_path/'audio.webm'+ path.write_bytes(b'original audio unchanged')+ fake=Mock()+ with patch.object(backend,'_get_client',return_value=fake), patch.object(backend,'stat'):+ backend.upload_local_file(str(path),'recordings/audio.webm',acl='private')+ fake.upload_file.assert_called_once_with(str(path),'test-123','recordings/audio.webm',ExtraArgs={'ACL':'private'})+ assert path.read_bytes()==b'original audio unchanged'+++def test_webm_pipeline_preserves_original_even_when_compression_enabled(tmp_path,monkeypatch):+ import src.utils.audio_conversion as conversion+ p=tmp_path/'meeting.webm'+ p.write_bytes(b'original webm bytes')+ monkeypatch.setattr(conversion,'AUDIO_COMPRESS_UPLOADS',True)+ monkeypatch.setattr(conversion,'get_codec_info',Mock(side_effect=AssertionError('must not inspect or convert')))+ result=conversion.convert_if_needed(str(p),connector_specs=TencentASRConnector.SPECIFICATIONS)+ assert result.output_path==str(p)+ assert not result.was_compressed and not result.was_converted+ assert p.read_bytes()==b'original webm bytes'