Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
feat(transcribe): emulate Amazon Transcribe batch transcription jobs (#1593)
ppettitau315a13bae1cf
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 35ebec4d..46027612 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,9 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]+### Added+- **Amazon Transcribe (`transcribe`)** is a new service emulator covering batch transcription jobs: `StartTranscriptionJob`, `GetTranscriptionJob`, `ListTranscriptionJobs` and `DeleteTranscriptionJob`, verified against botocore `transcribe-2017-10-26`. A job walks `QUEUED` → `IN_PROGRESS` → `COMPLETED` on a background task paced by `TRANSCRIBE_JOB_RUN_SECONDS` (default 2, `0` completes immediately), reads its media object from MiniStack's S3 store through an `s3://`, path-style or virtual-hosted HTTPS URI, and writes a transcript document back to S3 in the real Transcribe result format: `results.transcripts`, word-level `results.items`, `results.audio_segments`, and `results.segments` / `results.speaker_labels` when `ShowAlternatives` and `ShowSpeakerLabels` are set. `IdentifyLanguage` fills in `LanguageCode` and `IdentifiedLanguageScore` on completion while `IdentifyMultipleLanguages` reports `LanguageCodes` and leaves `LanguageCode` unset, as AWS does. `ContentRedaction` decides whether the `redacted-` copy replaces the transcript or sits beside it, and `Subtitles` writes WebVTT and SubRip files reported as `SubtitleFileUris`, redacted and renamed to match when redaction is on. Output lands at `OutputBucketName` / `OutputKey`, or in an auto-created `ministack-transcribe-output` bucket when neither is given, so the reported URIs are always fetchable. Media that cannot be read fails the job asynchronously with a `FailureReason` rather than erroring at `Start`, matching AWS, and a job left mid-flight by a restart fails on restore instead of stranding pollers. Reaching a terminal state publishes a `Transcribe Job State Change` event on the default bus, so an S3 → Lambda → Transcribe → EventBridge pipeline runs end to end locally. There is **no speech recognition**: the transcript is deterministic canned text keyed on a digest of the media URI, the same approach Bedrock Runtime takes for model output, which keeps integration tests reproducible. Streaming transcription, the call-analytics and medical job families, custom vocabularies and the tagging operations are not implemented. State is account- and region-scoped and persisted. Contributed by @ppettitau.+## [1.5.5] — 2026-09-01### Addeddiff --git a/README.md b/README.mdindex c700df44..cb252482 100644--- a/README.md+++ b/README.md@@ -427,6 +427,7 @@ subnet = ec2.create_subnet(| **Bedrock Agent** | 72 operations — agents (versions, aliases, action groups, collaborators, agent knowledge bases, PrepareAgent), knowledge bases (data sources, ingestion jobs — `StartIngestionJob` reads the S3 data source and reports real statistics, `FAILED` with `failureReasons` when the source is unreadable — documents), flows (aliases, versions, ValidateFlowDefinition, PrepareFlow), prompts with versions, tagging | All 72 operations verified against botocore `bedrock-agent-2023-06-05` || **Bedrock Agent Runtime** | InvokeAgent, InvokeInlineAgent, GetAgentMemory, DeleteAgentMemory, Retrieve, RetrieveAndGenerate, RetrieveAndGenerateStream, Rerank, CreateSession, GetSession, UpdateSession, DeleteSession, EndSession, ListSessions, CreateInvocation, ListInvocations, PutInvocationStep, GetInvocationStep, ListInvocationSteps, InvokeFlow, StartFlowExecution, StopFlowExecution, GetFlowExecution, ListFlowExecutions, ListFlowExecutionEvents, GetExecutionFlowSnapshot, OptimizePrompt | All 31 operations verified against botocore `bedrock-agent-runtime-2023-07-26`; eventstream responses on streaming operations; `Retrieve` searches the documents Knowledge Base ingestion stored (lexical scoring — no embeddings) and returns content, `s3Location` and score; `InvokeAgent` answers a canned reply without orchestration and emits no trace events — it never fabricates an `ORCHESTRATE` trace || **Bedrock AgentCore** | Control plane (`bedrock-agentcore-control`): CreateAgentRuntime, GetAgentRuntime, ListAgentRuntimes, UpdateAgentRuntime, DeleteAgentRuntime, ListAgentRuntimeVersions, CreateAgentRuntimeEndpoint, GetAgentRuntimeEndpoint, ListAgentRuntimeEndpoints, UpdateAgentRuntimeEndpoint, DeleteAgentRuntimeEndpoint. Data plane (`bedrock-agentcore`): InvokeAgentRuntime | Agent-runtime lifecycle with `CREATING`/`UPDATING`→`READY` transitions, versioned ARNs (`agent/{uuid}:{version}`, `agentEndpoint/{uuid}`); request/response shapes verified against botocore `bedrock-agentcore-control` / `bedrock-agentcore`; account- and region-scoped state; `InvokeAgentRuntime` is a deterministic echo (no model inference), matching the requested emulation scope |+| **Transcribe** | StartTranscriptionJob, GetTranscriptionJob, ListTranscriptionJobs, DeleteTranscriptionJob | Batch transcription jobs only; streaming is a separate endpoint (`transcribestreaming`). All four operations verified against botocore `transcribe-2017-10-26`. Jobs walk `QUEUED` → `IN_PROGRESS` → `COMPLETED` over `TRANSCRIBE_JOB_RUN_SECONDS` (default 2, `0` completes immediately); the media object is read from MiniStack S3 via an `s3://`, path-style or virtual-hosted HTTPS URI, and an unreadable URI fails the job with a `FailureReason` rather than erroring at `Start`, as on AWS; the transcript is written back to S3 in the real result format (`results.transcripts`, word-level `results.items`, `results.audio_segments`, plus `results.segments` and `results.speaker_labels` when the matching `Settings` are set) at `OutputBucketName`/`OutputKey`, or to an auto-created `ministack-transcribe-output` bucket. `IdentifyLanguage` and `IdentifyMultipleLanguages` resolve a language on completion, `ContentRedaction` controls whether a `redacted-` copy replaces or accompanies the transcript, and `Subtitles` writes WebVTT and SubRip files, redacted alongside the transcript when redaction is on. Reaching a terminal state publishes a `Transcribe Job State Change` event, so EventBridge rules downstream of a job fire. There is no speech recognition, so the transcript is deterministic canned text keyed on a digest of the media URI || **MSK** | CreateCluster, ListClusters, DescribeCluster, DeleteCluster, GetBootstrapBrokers, ListNodes, CreateConfiguration, ListConfigurations, DescribeConfiguration, ListConfigurationRevisions, DescribeConfigurationRevision, BatchAssociateScramSecret, BatchDisassociateScramSecret, ListScramSecrets, TagResource, UntagResource, ListTagsForResource | Kafka control plane; `GetBootstrapBrokers` honors `MINISTACK_MSK_BOOTSTRAP` so clients route to a real broker you bring (Redpanda, Kafka, KRaft) — the Kafka wire protocol itself is not emulated || **WAF v2** | CreateWebACL, GetWebACL, UpdateWebACL, DeleteWebACL, ListWebACLs, AssociateWebACL, DisassociateWebACL, GetWebACLForResource, ListResourcesForWebACL, CreateIPSet, GetIPSet, UpdateIPSet, DeleteIPSet, ListIPSets, CreateRuleGroup, GetRuleGroup, UpdateRuleGroup, DeleteRuleGroup, ListRuleGroups, TagResource, UntagResource, ListTagsForResource, CheckCapacity, DescribeManagedRuleGroup | LockToken enforced on Update/Delete; resource associations tracked || **Step Functions** | CreateStateMachine, DeleteStateMachine, DescribeStateMachine, UpdateStateMachine, ListStateMachines, StartExecution, StartSyncExecution, StopExecution, DescribeExecution, DescribeStateMachineForExecution, ListExecutions, GetExecutionHistory, SendTaskSuccess, SendTaskFailure, SendTaskHeartbeat, CreateActivity, DeleteActivity, DescribeActivity, ListActivities, GetActivityTask, TestState, TagResource, UntagResource, ListTagsForResource | Full ASL interpreter; Retry/Catch; waitForTaskToken; Activities (worker pattern); Pass/Task/Choice/Wait/Succeed/Fail/Map/Parallel; TestState API with mock and inspectionLevel support; SFN_MOCK_CONFIG for AWS SFN Local compatible mock testing; intrinsic functions (States.StringToJson, States.JsonToString, States.JsonMerge, States.Format); nested startExecution.sync |@@ -850,6 +851,7 @@ end-to-end without any client config.| `MINISTACK_MSK_BOOTSTRAP_TLS` | _(unset)_ | TLS bootstrap string (`BootstrapBrokerStringTls`) || `MINISTACK_MSK_BOOTSTRAP_SASL_SCRAM` | _(unset)_ | SASL/SCRAM bootstrap string (`BootstrapBrokerStringSaslScram`) || `MINISTACK_MSK_BOOTSTRAP_SASL_IAM` | _(unset)_ | SASL/IAM bootstrap string (`BootstrapBrokerStringSaslIam`) |+| `TRANSCRIBE_JOB_RUN_SECONDS` | `2` | Wall-clock time a Transcribe job spends between `QUEUED` and `COMPLETED`. Set `0` to complete immediately, or raise it when a test needs to observe `IN_PROGRESS` |### API Gateway HTTP proxy execution modeldiff --git a/ministack/app.py b/ministack/app.pyindex 1f89ade4..136d0a27 100644--- a/ministack/app.py+++ b/ministack/app.py@@ -366,6 +366,7 @@ SERVICE_REGISTRY = {"states": {"module": "stepfunctions", "aliases": ("step-functions", "stepfunctions")},"sts": {"module": "sts"},"tagging": {"module": "tagging"},+ "transcribe": {"module": "transcribe"},"transfer": {"module": "transfer"},"waf": {"module": "waf_v1"},"waf-regional": {"module": "waf_v1"},@@ -430,6 +431,7 @@ _state_map = {"bedrock_agent_runtime": "bedrock_agent_runtime","bedrock_agentcore": "bedrock_agentcore","msk": "msk",+ "transcribe": "transcribe",}SERVICE_NAME_ALIASES = {diff --git a/ministack/core/iam_actions.py b/ministack/core/iam_actions.pyindex e1a5509a..2556d1c6 100644--- a/ministack/core/iam_actions.py+++ b/ministack/core/iam_actions.py@@ -97,6 +97,7 @@ SERVICE_TO_IAM_NAMESPACE: dict[str, str] = {"states": "states","sts": "sts","tagging": "tag",+ "transcribe": "transcribe","transfer": "transfer","waf": "waf","waf-regional": "waf-regional",diff --git a/ministack/core/router.py b/ministack/core/router.pyindex d22657f4..9aee28bb 100644--- a/ministack/core/router.py+++ b/ministack/core/router.py@@ -143,6 +143,10 @@ SERVICE_PATTERNS = {"target_prefixes": ["AmazonAthena"],"host_patterns": [r"athena\."],},+ "transcribe": {+ "target_prefixes": ["Transcribe."],+ "host_patterns": [r"transcribe\."],+ },"airflow": {"host_patterns": [r"airflow\."],"path_patterns": [diff --git a/ministack/services/transcribe.py b/ministack/services/transcribe.pynew file mode 100644index 00000000..c1006a9a--- /dev/null+++ b/ministack/services/transcribe.py@@ -0,0 +1,973 @@+"""+Amazon Transcribe emulator.++JSON 1.1 protocol with X-Amz-Target prefix ``Transcribe``+(``transcribe-2017-10-26``).++Implemented:+ StartTranscriptionJob, GetTranscriptionJob, ListTranscriptionJobs,+ DeleteTranscriptionJob.++Jobs run as a background task that walks QUEUED -> IN_PROGRESS -> COMPLETED+over ``TRANSCRIBE_JOB_RUN_SECONDS`` (0 completes immediately), reads the media+object out of MiniStack's S3 store, and writes a transcript document back to+S3 in the real Transcribe result format. Entering a terminal state publishes a+``Transcribe Job State Change`` event so EventBridge rules downstream of a+transcription job fire the way they do on AWS.++The transcript document carries the structures the real service produces:+``results.transcripts``, word-level ``results.items``, ``results.segments``+when ``Settings.ShowAlternatives`` is set, ``results.audio_segments``, and+``results.speaker_labels`` when ``Settings.ShowSpeakerLabels`` is set.++Language identification is resolved rather than ignored. ``IdentifyLanguage``+sets ``LanguageCode`` and ``IdentifiedLanguageScore``; ``IdentifyMultipleLanguages``+sets ``LanguageCodes``, which is the only place the resolved code appears, as+on AWS. Both pick deterministically from ``LanguageOptions``.++``ContentRedaction`` drives the output layout the way AWS does: ``redacted``+writes only ``redacted-<key>``, ``redacted_and_unredacted`` writes both.+``Subtitles`` writes real WebVTT and SubRip files alongside the transcript and+reports them as ``SubtitleFileUris``. Subtitles carry the transcript text, so a+redacted job's subtitle files are redacted and named to match.++There is no speech recognition here. The transcript text is deterministic+canned output derived from the media URI, following the same approach as+``bedrock_runtime._mock_reply``: identical input gives identical output, and+different input gives distinguishable output. Speaker labels and alternatives+are synthesised from that text, so they are well-formed and stable rather+than meaningful.++Deferred:+ Streaming transcription (a separate endpoint), call analytics and medical+ job families, custom vocabularies / vocabulary filters / language models,+ and TagResource / UntagResource / ListTagsForResource. ``ModelSettings``,+ ``JobExecutionSettings``, ``LanguageIdSettings``, ``ToxicityDetection`` and+ ``Tags`` round-trip on the job record without changing the output.+"""++import asyncio+import base64+import contextvars+import copy+import hashlib+import json+import logging+import os+import re+import time+from urllib.parse import urlparse++from ministack.core.persistence import load_state+from ministack.core.responses import (+ AccountRegionScopedDict,+ error_response_json,+ get_account_id,+ get_region,+ json_response,+ new_uuid,+ now_iso,+ request_scope,+)++logger = logging.getLogger("transcribe")++# How long a job spends between QUEUED and COMPLETED. Same knob shape as+# GLUE_CRAWLER_RUN_SECONDS: tests that assert on IN_PROGRESS need a non-zero+# value, tests that just want a result set it to 0.+_JOB_RUN_SECONDS = float(os.environ.get("TRANSCRIBE_JOB_RUN_SECONDS", "2"))++# Where transcripts land when the caller supplies no OutputBucketName. Real+# Transcribe uses a service-managed bucket and hands back a presigned URL; the+# local equivalent is a bucket MiniStack owns, so the URI is always fetchable.+_SERVICE_BUCKET = "ministack-transcribe-output"++_TERMINAL_STATUSES = ("COMPLETED", "FAILED")++_MEDIA_UNREADABLE = (+ "The S3 URI that you provided can't be accessed. Make sure that you have "+ "read permission and try your request again."+)++# AWS publishes the pattern as ^[0-9a-zA-Z._-]+$ with no bound; the 1-200+# length limit comes from the shape's min/max rather than the pattern itself.+_JOB_NAME_RE = re.compile(r"^[0-9a-zA-Z._-]{1,200}$")++_HOST_SANITY_RE = re.compile(r"^[A-Za-z0-9.\-\[\]:]{1,255}$")++# Virtual-hosted-style S3 host, e.g. "my-bucket.s3.ap-southeast-2.amazonaws.com".+# Path-style hosts ("s3-ap-southeast-2.amazonaws.com") deliberately do not+# match: they have no bucket component before the ".s3" label.+_S3_VIRTUAL_HOST_RE = re.compile(+ r"^(?P<bucket>[a-z0-9][a-z0-9.\-]*?)\.s3(?:[.-][a-z0-9-]+)*\.(?:amazonaws\.com|localhost)$"+)++# The pattern AWS reports when Media.MediaFileUri is rejected.+_MEDIA_URI_PATTERN = "(s3://|http(s*)://).+"++_DEFAULT_HOST = os.environ.get("MINISTACK_HOST", "localhost")+_DEFAULT_PORT = os.environ.get("GATEWAY_PORT", os.environ.get("EDGE_PORT", "4566"))++# The caller's Host header, so TranscriptFileUri points at an address the+# caller can actually reach. A Lambda container calling in over the compose+# network reaches MiniStack by service name, not by localhost. Same approach+# as sqs._request_host.+_request_host: contextvars.ContextVar[str] = contextvars.ContextVar(+ "_transcribe_request_host", default=""+)++_jobs = AccountRegionScopedDict() # job_name -> job record++# Public members of the TranscriptionJob shape, in the order botocore models+# them. Internal bookkeeping lives under keys prefixed with "_" and never+# reaches the wire.+_JOB_MEMBERS = (+ "TranscriptionJobName",+ "TranscriptionJobStatus",+ "LanguageCode",+ "MediaSampleRateHertz",+ "MediaFormat",+ "Media",+ "Transcript",+ "StartTime",+ "CreationTime",+ "CompletionTime",+ "FailureReason",+ "Settings",+ "ModelSettings",+ "JobExecutionSettings",+ "ContentRedaction",+ "IdentifyLanguage",+ "IdentifyMultipleLanguages",+ "LanguageOptions",+ "IdentifiedLanguageScore",+ "LanguageCodes",+ "Tags",+ "Subtitles",+ "LanguageIdSettings",+ "ToxicityDetection",+)++# Members echoed straight back from the request without changing behaviour.+_ECHOED_MEMBERS = (+ "ModelSettings",+ "JobExecutionSettings",+ "LanguageIdSettings",+ "ToxicityDetection",+ "Tags",+ "LanguageOptions",+ "ContentRedaction",+)++# Fallback when a language-identification job supplies no LanguageOptions to+# choose from. AWS would infer it from the audio; there is no audio here.+_DEFAULT_LANGUAGE = "en-US"++# Fixed score for an identified language. Real Transcribe reports its+# confidence; a mock has none, so it reports certainty rather than inventing a+# number that varies.+_IDENTIFIED_LANGUAGE_SCORE = 1.0++_SUMMARY_MEMBERS = (+ "TranscriptionJobName",+ "CreationTime",+ "StartTime",+ "CompletionTime",+ "LanguageCode",+ "TranscriptionJobStatus",+ "FailureReason",+ "OutputLocationType",+)+++# ---------------------------------------------------------------------------+# State+# ---------------------------------------------------------------------------+++def reset():+ _jobs.clear()+++def get_state():+ return copy.deepcopy({"jobs": _jobs})+++def restore_state(data):+ if not data:+ return+ _jobs.clear()+ jobs = data.get("jobs")+ # Not `or {}`: AccountRegionScopedDict.__bool__ is scope-relative, and+ # restore runs in the default account/region. A snapshot holding jobs only+ # in other scopes would read as falsy and every one of them would be lost.+ if jobs is not None:+ _jobs.update(jobs)+ _fail_orphaned_jobs()+++def load_persisted_state(data):+ restore_state(data)+++def _fail_orphaned_jobs():+ """A job that was mid-flight when the process stopped has no worker any+ more. Leaving it QUEUED or IN_PROGRESS strands every caller polling+ GetTranscriptionJob forever, so it is failed the way AWS fails a job it+ cannot finish."""+ for job in _jobs.all_values():+ if job.get("TranscriptionJobStatus") not in _TERMINAL_STATUSES:+ job["TranscriptionJobStatus"] = "FAILED"+ job["FailureReason"] = "Internal Failure. The job did not survive a MiniStack restart."+ job["CompletionTime"] = time.time()++try:+ _restored = load_state("transcribe")+ if _restored:+ restore_state(_restored)+except Exception:+ logger.exception("Failed to restore persisted Transcribe state; continuing with fresh store")+++# ---------------------------------------------------------------------------+# Helpers+# ---------------------------------------------------------------------------+++def _bad_request(message):+ return error_response_json("BadRequestException", message, 400)+++def _public_job(job):+ """Project a stored job onto the TranscriptionJob wire shape, dropping+ members AWS omits when they have no value."""+ return {k: job[k] for k in _JOB_MEMBERS if job.get(k) is not None}+++def _public_summary(job):+ return {k: job[k] for k in _SUMMARY_MEMBERS if job.get(k) is not None}+++def _external_netloc():+ host = _request_host.get()+ if host and _HOST_SANITY_RE.match(host):+ return host+ return f"{_DEFAULT_HOST}:{_DEFAULT_PORT}"+++def _parse_s3_uri(uri):+ """Return (bucket, key) for an S3 location, or (None, None).++ AWS models Media.MediaFileUri as ``(s3://|http(s*)://).+``, so an HTTPS S3+ URL is as valid as an ``s3://`` one and SDK callers do emit both. Both+ path-style (``s3.<region>.amazonaws.com/<bucket>/<key>``) and+ virtual-hosted style (``<bucket>.s3.<region>.amazonaws.com/<key>``) are+ accepted; an unrecognised host falls back to path-style, which is how a+ MiniStack gateway URL resolves."""+ if not isinstance(uri, str):+ return None, None++ parsed = urlparse(uri)+ scheme = (parsed.scheme or "").lower()++ if scheme == "s3":+ bucket = parsed.netloc+ key = parsed.path.lstrip("/")+ elif scheme in ("http", "https"):+ path = parsed.path.lstrip("/")+ match = _S3_VIRTUAL_HOST_RE.match(parsed.hostname or "")+ if match:+ bucket, key = match.group("bucket"), path+ else:+ bucket, _, key = path.partition("/")+ else:+ return None, None++ if not bucket or not key:+ return None, None+ return bucket, key+++_SEGMENT_WORDS = 6+_WORD_SECONDS = 0.5+_MAX_SPEAKERS = 10+++def _mock_transcript(media_uri, redacted=False):+ """Deterministic canned transcript. The digest of the media URI keeps+ output for different media distinguishable while staying stable across+ runs, which is the whole point of using fixtures instead of real ASR.++ Long enough to split into several segments, so speaker labels, alternatives+ and multi-language spans have something to distribute across rather than+ collapsing onto one segment.++ The redacted variant masks the digest, so it is the identifying part of+ the text that disappears, the way real redaction removes PII."""+ if redacted:+ subject = "[PII]"+ else:+ digest = hashlib.sha256(media_uri.encode("utf-8")).hexdigest()[:8]+ subject = f"media#{digest}"+ return (+ f"[ministack mock transcript] deterministic placeholder speech for {subject} "+ "generated locally by ministack with no speech recognition performed on the audio"+ )+++def _fmt_time(seconds):+ return f"{seconds:.1f}"+++def _word_items(text):+ """Word-level items in Transcribe's result format, with synthetic timings+ so consumers that walk `results.items` have something well-formed to walk.+ Any `language_code` tag is stamped on by the caller, which knows which+ segment each item belongs to."""+ items = []+ for index, word in enumerate(text.split()):+ start = index * _WORD_SECONDS+ item = {+ "type": "pronunciation",+ "start_time": _fmt_time(start),+ "end_time": _fmt_time(start + _WORD_SECONDS),+ "alternatives": [{"confidence": "1.0", "content": word}],+ }+ items.append(item)+ return items+++def _alternatives(transcript, items, max_alternatives):+ """The primary reading plus synthetic runners-up. Real Transcribe returns+ competing hypotheses when ShowAlternatives is set; there is no recogniser+ here to disagree with itself, so the extras are labelled as what they are+ rather than dressed up as genuine alternatives."""+ alternatives = [{"transcript": transcript, "items": items}]+ for rank in range(1, max(1, min(max_alternatives, 10))):+ alternatives.append({"transcript": f"{transcript} [alternative {rank}]", "items": items})+ return alternatives+++def _transcript_document(job_name, text, settings, language_codes):+ """Build the transcript JSON in the shape real Transcribe writes to S3.++ `segments` appears only under ShowAlternatives and `speaker_labels` only+ under ShowSpeakerLabels, matching the service: a consumer that never asked+ for diarisation should not find it there."""+ settings = settings or {}+ show_speakers = bool(settings.get("ShowSpeakerLabels"))+ show_alternatives = bool(settings.get("ShowAlternatives"))+ max_alternatives = settings.get("MaxAlternatives") or 2+ max_speakers = max(1, min(settings.get("MaxSpeakerLabels") or 2, _MAX_SPEAKERS))++ # Only tag items with a language when more than one was resolved; a+ # single-language job carries the code on the job record instead. The tag+ # is applied per segment below, so an item always agrees with the audio+ # segment that owns it.+ multi_language = len(language_codes) > 1+ items = _word_items(text)++ words = text.split()+ groups = [+ list(range(start, min(start + _SEGMENT_WORDS, len(items))))+ for start in range(0, len(items), _SEGMENT_WORDS)+ ]++ segments = []+ audio_segments = []+ speaker_segments = []++ for position, indices in enumerate(groups):+ segment_text = " ".join(words[indices[0] : indices[-1] + 1])+ segment_items = [items[i] for i in indices]+ start_time = segment_items[0]["start_time"]+ end_time = segment_items[-1]["end_time"]+ speaker_label = f"spk_{position % max_speakers}"+ segment_language = language_codes[position % len(language_codes)] if language_codes else None+ if multi_language:+ for item in segment_items:+ item["language_code"] = segment_language++ segments.append(+ {+ "start_time": start_time,+ "end_time": end_time,+ "alternatives": _alternatives(segment_text, segment_items, max_alternatives),+ }+ )++ audio_segment = {+ "id": position,+ "transcript": segment_text,+ "start_time": start_time,+ "end_time": end_time,+ "items": indices,+ }+ if language_codes:+ audio_segment["language_code"] = segment_language+ if show_speakers:+ audio_segment["speaker_label"] = speaker_label+ audio_segments.append(audio_segment)++ if show_speakers:+ speaker_segments.append(+ {+ "start_time": start_time,+ "end_time": end_time,+ "speaker_label": speaker_label,+ "items": [+ {+ "start_time": item["start_time"],+ "end_time": item["end_time"],+ "speaker_label": speaker_label,+ }+ for item in segment_items+ ],+ }+ )++ results = {+ "transcripts": [{"transcript": text}],+ "items": items,+ "audio_segments": audio_segments,+ }+ if show_alternatives:+ results["segments"] = segments+ if show_speakers:+ results["speaker_labels"] = {+ "speakers": min(len(groups), max_speakers),+ "segments": speaker_segments,+ }++ return {+ "jobName": job_name,+ "accountId": get_account_id(),+ "status": "COMPLETED",+ "results": results,+ }+++# ---------------------------------------------------------------------------+# Subtitles+# ---------------------------------------------------------------------------+++def _subtitle_cue_time(seconds, separator):+ hours, remainder = divmod(float(seconds), 3600)+ minutes, secs = divmod(remainder, 60)+ whole, fraction = divmod(secs, 1)+ return f"{int(hours):02d}:{int(minutes):02d}:{int(whole):02d}{separator}{int(fraction * 1000):03d}"+++def _render_subtitles(document, fmt, start_index):+ """Render the transcript's audio segments as SubRip or WebVTT."""+ srt = fmt == "srt"+ separator = "," if srt else "."+ blocks = []+ for offset, segment in enumerate(document["results"]["audio_segments"]):+ start = _subtitle_cue_time(segment["start_time"], separator)+ end = _subtitle_cue_time(segment["end_time"], separator)+ blocks.append(+ f"{offset + start_index}\n{start} --> {end}\n{segment['transcript']}\n"+ )+ body = "\n".join(blocks)+ return body if srt else f"WEBVTT\n\n{body}"+++def _resolve_languages(job):+ """The language codes a language-identification job would have detected.++ There is no audio to analyse, so the choice comes from LanguageOptions in+ request order, which keeps it deterministic and lets a test pin the+ outcome by listing the language it wants first."""+ if job.get("LanguageCode"):+ return [job["LanguageCode"]]+ # Duplicates would otherwise report the same language twice, each with the+ # full duration, so the spans sum to more audio than the job contains.+ options = []+ for code in job.get("LanguageOptions") or []:+ if isinstance(code, str) and code not in options:+ options.append(code)+ if job.get("IdentifyMultipleLanguages"):+ return options or [_DEFAULT_LANGUAGE]+ return [options[0]] if options else [_DEFAULT_LANGUAGE]+++def _apply_languages(job, codes, document):+ """Record the resolved language on the job the way AWS does.++ IdentifyLanguage fills LanguageCode and IdentifiedLanguageScore.+ IdentifyMultipleLanguages fills LanguageCodes and leaves LanguageCode+ unset, so a caller that wants the source language has to check both. This+ only happens once the job completes; AWS cannot know the language at+ submission time, so neither does this."""+ if job.get("LanguageCode"):+ return+ if job.get("IdentifyMultipleLanguages"):+ durations = {}+ for segment in document["results"]["audio_segments"]:+ code = segment.get("language_code")+ span = float(segment["end_time"]) - float(segment["start_time"])+ durations[code] = round(durations.get(code, 0.0) + span, 1)+ job["LanguageCodes"] = [+ {"LanguageCode": code, "DurationInSeconds": durations.get(code, 0.0)}+ for code in codes+ ]+ else:+ job["LanguageCode"] = codes[0]+ job["IdentifiedLanguageScore"] = _IDENTIFIED_LANGUAGE_SCORE+++def _redacted_key(key):+ """Redacted output sits beside the transcript with `redacted-` prefixed to+ the file name, not to the whole key, so it stays in the same prefix."""+ prefix, separator, name = key.rpartition("/")+ return f"{prefix}{separator}redacted-{name}"+++def _subtitle_key(key, fmt):+ base = key[: -len(".json")] if key.endswith(".json") else key+ return f"{base}.{fmt}"+++def _emit_state_change(job_name, status):+ """Publish `Transcribe Job State Change`, the event AWS emits when a job+ reaches a terminal state. Pipelines that chain off a completed+ transcription depend on it, so a failure to deliver is logged rather than+ swallowed silently, and never propagates into the job itself."""+ try:+ from ministack.services import eventbridge as _eb++ _eb._dispatch_event(+ {+ "EventId": new_uuid(),+ "Source": "aws.transcribe",+ "DetailType": "Transcribe Job State Change",+ "Detail": json.dumps(+ {+ "TranscriptionJobName": job_name,+ "TranscriptionJobStatus": status,+ }+ ),+ "EventBusName": "default",+ "Time": now_iso(),+ "Resources": [],+ "Account": get_account_id(),+ "Region": get_region(),+ }+ )+ except Exception:+ logger.exception("Transcribe: failed to publish state change for job %s", job_name)+++def _sort_key(job):+ """Creation time with the job name breaking ties, so the ordering is total+ and stable across calls. Callers reverse it to get newest first."""+ return (job.get("CreationTime") or 0.0, job.get("TranscriptionJobName") or "")+++def _encode_token(job):+ """Tokens carry the sort position of the last item returned, not an index.+ An offset would skip an item whenever a job was deleted between pages and+ repeat one whenever a job was created."""+ created, name = _sort_key(job)+ raw = json.dumps({"ct": created, "n": name}).encode("utf-8")+ return base64.urlsafe_b64encode(raw).decode("ascii")+++def _decode_token(token):+ try:+ raw = json.loads(base64.urlsafe_b64decode(token.encode("ascii")).decode("utf-8"))+ return (float(raw["ct"]), str(raw["n"]))+ except Exception:+ return None+++# ---------------------------------------------------------------------------+# Job execution+# ---------------------------------------------------------------------------+++def _live_job(job_name, run_id):+ """The job this worker was started for, or None if it has been deleted or+ replaced. DeleteTranscriptionJob does not cancel a running worker, so a+ job re-created under the same name would otherwise be driven by both its+ own worker and the orphaned one: completing early, writing the transcript+ twice and emitting the state change event twice."""+ job = _jobs.get(job_name)+ if job is None or job.get("_run_id") != run_id:+ return None+ return job+++async def _run_job(job_name, run_id, account_id, region, netloc):+ """Walk a job to a terminal state. Runs as a background task, so it pins+ the account and region it was started for rather than inheriting whatever+ request happens to be in flight."""+ with request_scope(account_id, region):+ try:+ await _run_job_inner(job_name, run_id, netloc)+ except Exception:+ logger.exception("Transcribe: job %s crashed", job_name)+ job = _live_job(job_name, run_id)+ if job is not None and job.get("TranscriptionJobStatus") not in _TERMINAL_STATUSES:+ job["TranscriptionJobStatus"] = "FAILED"+ job["FailureReason"] = "Internal Failure. Please try your request again."+ job["CompletionTime"] = time.time()+ _emit_state_change(job_name, "FAILED")+++async def _run_job_inner(job_name, run_id, netloc):+ half = _JOB_RUN_SECONDS / 2 if _JOB_RUN_SECONDS > 0 else 0++ if half:+ await asyncio.sleep(half)++ job = _live_job(job_name, run_id)+ if job is None:+ return++ job["TranscriptionJobStatus"] = "IN_PROGRESS"+ job["StartTime"] = time.time()++ if half:+ await asyncio.sleep(half)++ job = _live_job(job_name, run_id)+ if job is None:+ return++ def _fail(reason):+ if _live_job(job_name, run_id) is None:+ return+ job["TranscriptionJobStatus"] = "FAILED"+ job["FailureReason"] = reason+ job["CompletionTime"] = time.time()+ logger.info("Transcribe: job %s failed: %s", job_name, reason)+ _emit_state_change(job_name, "FAILED")++ from ministack.services import s3 as s3_svc++ media_uri = job["Media"]["MediaFileUri"]+ media_bucket, media_key = _parse_s3_uri(media_uri)+ if s3_svc._get_object_data(media_bucket, media_key) is None:+ _fail(_MEDIA_UNREADABLE)+ return++ text = _mock_transcript(media_uri)+ codes = _resolve_languages(job)+ document = _transcript_document(job_name, text, job.get("Settings"), codes)++ bucket = job["_output_bucket"]+ key = job["_output_key"]++ if job["OutputLocationType"] == "SERVICE_BUCKET" and s3_svc._ensure_bucket(bucket) is None:+ s3_svc._create_bucket(bucket, b"")++ def _write(target_key, payload, content_type):+ body = payload.encode("utf-8")+ resp = s3_svc._put_object(+ bucket,+ target_key,+ body,+ {"content-type": content_type, "content-length": str(len(body))},+ )+ if isinstance(resp, tuple) and resp[0] >= 300:+ return False+ return True++ # RedactionOutput decides the layout: "redacted" replaces the transcript+ # with the masked one, "redacted_and_unredacted" writes both. Transcript+ # then carries whichever URIs actually exist.+ redaction = job.get("ContentRedaction") or {}+ redaction_output = str(redaction.get("RedactionOutput") or "").lower()+ transcript_uris = {}+ writes = []+ subtitle_document = document+ subtitle_base_key = key++ if redaction:+ redacted_document = _transcript_document(+ job_name, _mock_transcript(media_uri, redacted=True), job.get("Settings"), codes+ )+ redacted_key = _redacted_key(key)+ writes.append((redacted_key, json.dumps(redacted_document), "application/json"))+ transcript_uris["RedactedTranscriptFileUri"] = f"http://{netloc}/{bucket}/{redacted_key}"+ # Subtitles carry the transcript text, so a redacted job's subtitle+ # files have to be redacted too, and named for what they contain.+ # Otherwise the one layout whose point is that the raw text never+ # reaches the bucket writes the raw text to the bucket.+ subtitle_document = redacted_document+ subtitle_base_key = redacted_key+ if redaction_output == "redacted_and_unredacted":+ writes.append((key, json.dumps(document), "application/json"))+ transcript_uris["TranscriptFileUri"] = f"http://{netloc}/{bucket}/{key}"+ else:+ writes.append((key, json.dumps(document), "application/json"))+ transcript_uris["TranscriptFileUri"] = f"http://{netloc}/{bucket}/{key}"++ subtitles = job.get("_subtitles_request") or {}+ subtitle_formats = [f for f in (subtitles.get("Formats") or []) if f in ("vtt", "srt")]+ subtitle_start = subtitles.get("OutputStartIndex")+ subtitle_start = 0 if subtitle_start is None else int(subtitle_start)+ subtitle_uris = []+ for fmt in subtitle_formats:+ subtitle_key = _subtitle_key(subtitle_base_key, fmt)+ writes.append(+ (subtitle_key, _render_subtitles(subtitle_document, fmt, subtitle_start), "text/plain")+ )+ subtitle_uris.append(f"http://{netloc}/{bucket}/{subtitle_key}")++ for target_key, payload, content_type in writes:+ if not _write(target_key, payload, content_type):+ _fail(+ f"The output bucket that you provided can't be written to: "+ f"s3://{bucket}/{target_key}. "+ "Make sure that the bucket exists and try your request again."+ )+ return++ _apply_languages(job, codes, document)+ if subtitle_formats:+ job["Subtitles"] = {+ "Formats": subtitle_formats,+ "SubtitleFileUris": subtitle_uris,+ "OutputStartIndex": subtitle_start,+ }+ job["Transcript"] = transcript_uris+ job["TranscriptionJobStatus"] = "COMPLETED"+ job["CompletionTime"] = time.time()+ logger.info(+ "Transcribe: job %s completed, %d object(s) written to s3://%s",+ job_name,+ len(writes),+ bucket,+ )+ _emit_state_change(job_name, "COMPLETED")+++# ---------------------------------------------------------------------------+# Operations+# ---------------------------------------------------------------------------+++def _start_transcription_job(data):+ name = data.get("TranscriptionJobName")+ if not isinstance(name, str) or not name.strip():+ return _bad_request(+ "1 validation error detected: Value null at 'transcriptionJobName' "+ "failed to satisfy constraint: Member must not be null"+ )+ if not _JOB_NAME_RE.match(name):+ return _bad_request(+ f"1 validation error detected: Value '{name}' at 'transcriptionJobName' failed to "+ "satisfy constraint: Member must satisfy regular expression pattern: ^[0-9a-zA-Z._-]+$"+ )++ media = data.get("Media")+ if not isinstance(media, dict) or not media.get("MediaFileUri"):+ return _bad_request(+ "1 validation error detected: Value null at 'media' failed to satisfy "+ "constraint: Member must not be null"+ )++ media_uri = media["MediaFileUri"]+ if _parse_s3_uri(media_uri) == (None, None):+ return _bad_request(+ f"1 validation error detected: Value '{media_uri}' at 'media.mediaFileUri' failed "+ "to satisfy constraint: Member must satisfy regular expression pattern: "+ f"{_MEDIA_URI_PATTERN}"+ )++ language_code = data.get("LanguageCode")+ identify = data.get("IdentifyLanguage") or data.get("IdentifyMultipleLanguages")+ if not language_code and not identify:+ return _bad_request(+ "You must include either LanguageCode or IdentifyLanguage or "+ "IdentifyMultipleLanguages in your request. Please try your request again."+ )++ if name in _jobs:+ return error_response_json(+ "ConflictException",+ "The requested job name already exists. Use a different job name.",+ 400,+ )++ output_bucket = data.get("OutputBucketName")+ if output_bucket:+ output_key = data.get("OutputKey") or f"{name}.json"+ # AWS treats a key ending in "/" as a prefix and appends the job name.+ if output_key.endswith("/"):+ output_key = f"{output_key}{name}.json"+ location_type = "CUSTOMER_BUCKET"+ else:+ output_bucket = _SERVICE_BUCKET+ # Region-qualified: S3 buckets are account-scoped but jobs are+ # region-scoped, so same-named jobs in two regions are two jobs and+ # must not resolve to one transcript object.+ output_key = f"{get_account_id()}/{get_region()}/{name}.json"+ location_type = "SERVICE_BUCKET"++ run_id = new_uuid()+ job = {+ "TranscriptionJobName": name,+ "TranscriptionJobStatus": "QUEUED",+ "LanguageCode": language_code,+ "MediaSampleRateHertz": data.get("MediaSampleRateHertz"),+ "MediaFormat": data.get("MediaFormat"),+ "Media": copy.deepcopy(media),+ "Transcript": None,+ "StartTime": None,+ "CreationTime": time.time(),+ "CompletionTime": None,+ "FailureReason": None,+ "Settings": copy.deepcopy(data.get("Settings")) if data.get("Settings") else None,+ # The identification flags are echoed as sent; the codes they resolve+ # to are filled in by the worker, because AWS cannot know the language+ # until the job has run.+ "IdentifyLanguage": data.get("IdentifyLanguage"),+ "IdentifyMultipleLanguages": data.get("IdentifyMultipleLanguages"),+ "IdentifiedLanguageScore": None,+ "LanguageCodes": None,+ "Subtitles": None,+ "OutputLocationType": location_type,+ "_run_id": run_id,+ "_output_bucket": output_bucket,+ "_output_key": output_key,+ }+ for member in _ECHOED_MEMBERS:+ value = data.get(member)+ job[member] = copy.deepcopy(value) if value is not None else None+ # Subtitles is requested with Formats/OutputStartIndex but reported as+ # Formats/SubtitleFileUris/OutputStartIndex, so it is held aside until the+ # worker has written the files and can report where they landed.+ job["_subtitles_request"] = copy.deepcopy(data.get("Subtitles")) or None+ _jobs[name] = job++ asyncio.create_task(+ _run_job(name, run_id, get_account_id(), get_region(), _external_netloc())+ )++ return json_response({"TranscriptionJob": _public_job(job)})+++def _get_transcription_job(data):+ name = data.get("TranscriptionJobName")+ if not isinstance(name, str) or not name.strip():+ return _bad_request(+ "1 validation error detected: Value null at 'transcriptionJobName' "+ "failed to satisfy constraint: Member must not be null"+ )++ job = _jobs.get(name)+ if job is None:+ return error_response_json(+ "NotFoundException",+ "The requested job couldn't be found. Check the job name and try your request again.",+ 400,+ )++ return json_response({"TranscriptionJob": _public_job(job)})+++def _list_transcription_jobs(data):+ status = data.get("Status")+ if status is not None and status not in ("QUEUED", "IN_PROGRESS", "FAILED", "COMPLETED"):+ return _bad_request(+ f"1 validation error detected: Value '{status}' at 'status' failed to satisfy "+ "constraint: Member must satisfy enum value set: "+ "[COMPLETED, IN_PROGRESS, QUEUED, FAILED]"+ )++ contains = data.get("JobNameContains")++ max_results = data.get("MaxResults")+ if max_results is None:+ max_results = 100+ else:+ try:+ max_results = int(max_results)+ except (TypeError, ValueError):+ return _bad_request("MaxResults must be an integer")+ if max_results < 1 or max_results > 100:+ return _bad_request(+ f"1 validation error detected: Value '{max_results}' at 'maxResults' failed to "+ "satisfy constraint: Member must have value less than or equal to 100"+ )++ cursor = None+ token = data.get("NextToken")+ if token:+ cursor = _decode_token(token)+ if cursor is None:+ return _bad_request("The NextToken that you provided is invalid.")++ jobs = list(_jobs.values())+ if status:+ jobs = [j for j in jobs if j.get("TranscriptionJobStatus") == status]+ if contains:+ # AWS documents this search as case insensitive.+ needle = str(contains).lower()+ jobs = [j for j in jobs if needle in j["TranscriptionJobName"].lower()]++ # Most recently created first, matching the console and the API.+ jobs.sort(key=_sort_key, reverse=True)+ if cursor is not None:+ jobs = [j for j in jobs if _sort_key(j) < cursor]++ page = jobs[:max_results]+ result = {"TranscriptionJobSummaries": [_public_summary(j) for j in page]}+ if status:+ result["Status"] = status+ if len(jobs) > max_results:+ result["NextToken"] = _encode_token(page[-1])+ return json_response(result)+++def _delete_transcription_job(data):+ name = data.get("TranscriptionJobName")+ if not isinstance(name, str) or not name.strip():+ return _bad_request(+ "1 validation error detected: Value null at 'transcriptionJobName' "+ "failed to satisfy constraint: Member must not be null"+ )++ # botocore models no NotFoundException on this operation: deleting a job+ # that isn't there succeeds.+ _jobs.pop(name, None)+ return json_response({})+++_DISPATCH = {+ "StartTranscriptionJob": _start_transcription_job,+ "GetTranscriptionJob": _get_transcription_job,+ "ListTranscriptionJobs": _list_transcription_jobs,+ "DeleteTranscriptionJob": _delete_transcription_job,+}+++async def handle_request(method, path, headers, body, query_params):+ _request_host.set(headers.get("host", "") or headers.get("Host", ""))++ target = headers.get("X-Amz-Target") or headers.get("x-amz-target") or ""+ action = target.split(".", 1)[1] if "." in target else target+ if not action:+ return error_response_json("InvalidAction", "missing X-Amz-Target", 400)++ body_text = body.decode("utf-8") if isinstance(body, bytes) else (body or "")+ try:+ payload = json.loads(body_text) if body_text else {}+ except json.JSONDecodeError:+ return error_response_json("SerializationException", "invalid JSON body", 400)++ fn = _DISPATCH.get(action)+ if fn is None:+ return error_response_json(+ "InvalidAction",+ f"Operation '{action}' not implemented",+ 400,+ )++ logger.debug("Transcribe: %s %s", action, body_text)+ return fn(payload)diff --git a/tests/conftest.py b/tests/conftest.pyindex 5675fe02..acaecf82 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -728,6 +728,11 @@ def s3tables():return make_client("s3tables")[email protected](scope="session")+def transcribe():+ return make_client("transcribe")++class FakeDockerContainer:"""Container double for tests that observe lifecycle without a daemon."""diff --git a/tests/test_persistence.py b/tests/test_persistence.pyindex 5dde53ba..728047aa 100644--- a/tests/test_persistence.py+++ b/tests/test_persistence.py@@ -645,6 +645,97 @@ def test_backup_round_trip():_round_trip("backup", "backup", populate, observe)+def test_transcribe_round_trip():+ import time as _time++ def populate(mod):+ mod._jobs["job-test"] = {+ "TranscriptionJobName": "job-test",+ "TranscriptionJobStatus": "COMPLETED",+ "LanguageCode": "en-US",+ "Media": {"MediaFileUri": "s3://media/call.mp3"},+ "Transcript": {+ "TranscriptFileUri": "http://localhost:4566/out/job-test.json"+ },+ "CreationTime": _time.time(),+ "StartTime": _time.time(),+ "CompletionTime": _time.time(),+ "OutputLocationType": "SERVICE_BUCKET",+ "_run_id": "run-1",+ "_output_bucket": "out",+ "_output_key": "job-test.json",+ }++ def observe(mod):+ job = mod._jobs.get("job-test")+ assert job is not None+ assert job["TranscriptionJobStatus"] == "COMPLETED"+ # A completed job's transcript pointer and media reference are what+ # callers read after a warm boot; losing either leaves GetTranscriptionJob+ # reporting COMPLETED with nothing to fetch.+ assert job["Transcript"]["TranscriptFileUri"].endswith("job-test.json")+ assert job["Media"]["MediaFileUri"] == "s3://media/call.mp3"+ assert job["_output_bucket"] == "out"++ _round_trip("transcribe", "transcribe", populate, observe)+++def test_transcribe_jobs_round_trip_outside_boot_region():+ """Transcribe jobs are region-scoped. `AccountRegionScopedDict.__bool__`+ is scope-relative, so a snapshot holding jobs only in regions other than+ the one restore runs in must not be treated as empty."""+ import time as _time++ from ministack.core.responses import request_scope++ def populate(mod):+ with request_scope("000000000000", "eu-west-1"):+ mod._jobs["eu-job"] = {+ "TranscriptionJobName": "eu-job",+ "TranscriptionJobStatus": "COMPLETED",+ "CreationTime": _time.time(),+ "CompletionTime": _time.time(),+ "Media": {"MediaFileUri": "s3://b/a.mp3"},+ "OutputLocationType": "SERVICE_BUCKET",+ "_output_bucket": "b",+ "_output_key": "k.json",+ }++ def observe(mod):+ with request_scope("000000000000", "eu-west-1"):+ job = mod._jobs.get("eu-job")+ assert job is not None, "a job outside the restoring region was dropped"+ assert job["TranscriptionJobStatus"] == "COMPLETED"++ _round_trip("transcribe", "transcribe", populate, observe)+++def test_transcribe_round_trip_fails_a_job_left_mid_flight():+ """A restart leaves no worker behind, so a job restored as QUEUED or+ IN_PROGRESS would strand every caller polling GetTranscriptionJob."""+ import time as _time++ def populate(mod):+ mod._jobs["stuck-job"] = {+ "TranscriptionJobName": "stuck-job",+ "TranscriptionJobStatus": "IN_PROGRESS",+ "CreationTime": _time.time(),+ "Media": {"MediaFileUri": "s3://b/a.mp3"},+ "OutputLocationType": "SERVICE_BUCKET",+ "_output_bucket": "b",+ "_output_key": "k.json",+ }++ def observe(mod):+ job = mod._jobs.get("stuck-job")+ assert job is not None+ assert job["TranscriptionJobStatus"] == "FAILED"+ assert job["CompletionTime"] is not None+ assert "restart" in job["FailureReason"]++ _round_trip("transcribe", "transcribe", populate, observe)++def test_cloudformation_round_trip():"""CloudFormation stack metadata (stacks, events, exports, change sets)survives a PERSIST_STATE stop/restore cycle — otherwise ListStacks /diff --git a/tests/test_transcribe.py b/tests/test_transcribe.pynew file mode 100644index 00000000..b93a2892--- /dev/null+++ b/tests/test_transcribe.py@@ -0,0 +1,979 @@+import json+import os+import time+import uuid++import boto3+import pytest+from botocore.exceptions import ClientError++_ENDPOINT = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")+++def _unique(prefix):+ return f"{prefix}-{uuid.uuid4().hex[:8]}"+++def _wait_for_status(transcribe, job_name, statuses=("COMPLETED", "FAILED"), timeout=30):+ deadline = time.time() + timeout+ job = None+ while time.time() < deadline:+ job = transcribe.get_transcription_job(TranscriptionJobName=job_name)["TranscriptionJob"]+ if job["TranscriptionJobStatus"] in statuses:+ return job+ time.sleep(0.2)+ raise AssertionError(+ f"job {job_name} never reached {statuses}; last status "+ f"{job['TranscriptionJobStatus'] if job else 'unknown'}"+ )+++def _split_transcript_uri(uri):+ """`http://host:port/bucket/key` -> (bucket, key)."""+ _, _, rest = uri.partition("://")+ _, _, path = rest.partition("/")+ bucket, _, key = path.partition("/")+ return bucket, key+++def media(s3):+ bucket = _unique("transcribe-media")+ key = "audio/call.mp3"+ s3.create_bucket(Bucket=bucket)+ s3.put_object(Bucket=bucket, Key=key, Body=b"not really audio")+ return bucket, key, f"s3://{bucket}/{key}"+++def test_transcribe_job_completes_and_writes_a_transcript(transcribe, s3, media):+ """The end-to-end contract the issue asks for: a job reaches COMPLETED on its+ own, and TranscriptFileUri resolves to a real object in the real Transcribe+ result shape, so a consumer can fetch and parse it without special-casing."""+ _, _, media_uri = media+ job_name = _unique("job")++ started = transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ MediaFormat="mp3",+ Media={"MediaFileUri": media_uri},+ )["TranscriptionJob"]++ assert started["TranscriptionJobName"] == job_name+ assert started["TranscriptionJobStatus"] == "QUEUED"+ assert started["LanguageCode"] == "en-US"+ assert started["Media"] == {"MediaFileUri": media_uri}+ # AWS omits Transcript until the job completes.+ assert "Transcript" not in started++ job = _wait_for_status(transcribe, job_name)+ assert job["TranscriptionJobStatus"] == "COMPLETED"+ assert job["CompletionTime"] >= job["StartTime"] >= job["CreationTime"]++ bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ document = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())++ assert document["jobName"] == job_name+ assert document["status"] == "COMPLETED"+ text = document["results"]["transcripts"][0]["transcript"]+ assert text+ assert len(document["results"]["items"]) == len(text.split())+ assert document["results"]["items"][0]["alternatives"][0]["content"] == text.split()[0]+++def test_transcribe_transcript_is_deterministic_per_media(transcribe, s3, media):+ """There is no ASR here, so the value of the mock rests entirely on being+ reproducible. The same media must transcribe identically on every run or+ integration tests asserting on transcript content become flaky."""+ _, _, media_uri = media+ texts = []++ for _ in range(2):+ job_name = _unique("job")+ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)+ bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ document = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())+ texts.append(document["results"]["transcripts"][0]["transcript"])++ assert texts[0] == texts[1]+++def test_transcribe_different_media_gives_different_transcripts(transcribe, s3, media):+ """The flip side of determinism: a constant string would satisfy the previous+ test while making it impossible to tell two jobs apart."""+ bucket, _, first_uri = media+ s3.put_object(Bucket=bucket, Key="audio/other.mp3", Body=b"also not audio")+ second_uri = f"s3://{bucket}/audio/other.mp3"++ texts = []+ for media_uri in (first_uri, second_uri):+ job_name = _unique("job")+ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)+ out_bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ document = json.loads(s3.get_object(Bucket=out_bucket, Key=key)["Body"].read())+ texts.append(document["results"]["transcripts"][0]["transcript"])++ assert texts[0] != texts[1]+++def test_transcribe_writes_to_the_caller_supplied_output_bucket(transcribe, s3, media):+ """OutputBucketName/OutputKey must be honoured verbatim. A CDK stack that+ points Transcribe at its own bucket and then reads a fixed key downstream+ breaks if the transcript lands anywhere else."""+ _, _, media_uri = media+ out_bucket = _unique("transcribe-out")+ s3.create_bucket(Bucket=out_bucket)+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ OutputBucketName=out_bucket,+ OutputKey="results/transcript.json",+ )++ job = _wait_for_status(transcribe, job_name)+ assert job["TranscriptionJobStatus"] == "COMPLETED"+ assert job["Transcript"]["TranscriptFileUri"].endswith(+ f"/{out_bucket}/results/transcript.json"+ )+ # The object is really there.+ s3.head_object(Bucket=out_bucket, Key="results/transcript.json")++ summary = next(+ s+ for s in transcribe.list_transcription_jobs(JobNameContains=job_name)[+ "TranscriptionJobSummaries"+ ]+ if s["TranscriptionJobName"] == job_name+ )+ assert summary["OutputLocationType"] == "CUSTOMER_BUCKET"+++def test_transcribe_output_key_ending_in_slash_is_a_prefix(transcribe, s3, media):+ """AWS treats a trailing slash as a prefix and appends <jobName>.json rather+ than writing an object whose key ends in a slash."""+ _, _, media_uri = media+ out_bucket = _unique("transcribe-out")+ s3.create_bucket(Bucket=out_bucket)+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ OutputBucketName=out_bucket,+ OutputKey="nested/prefix/",+ )++ job = _wait_for_status(transcribe, job_name)+ assert job["Transcript"]["TranscriptFileUri"].endswith(+ f"/{out_bucket}/nested/prefix/{job_name}.json"+ )+++def test_transcribe_defaults_to_a_service_managed_bucket(transcribe, media):+ """With no OutputBucketName, real Transcribe stores the transcript in a bucket+ it owns and hands back a presigned URL. MiniStack provisions its own bucket+ instead, so the URI is fetchable locally rather than being a dead link."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )++ job = _wait_for_status(transcribe, job_name)+ bucket, _ = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ assert bucket == "ministack-transcribe-output"++ summary = next(+ s+ for s in transcribe.list_transcription_jobs(JobNameContains=job_name)[+ "TranscriptionJobSummaries"+ ]+ if s["TranscriptionJobName"] == job_name+ )+ assert summary["OutputLocationType"] == "SERVICE_BUCKET"+++def test_transcribe_unreadable_media_fails_the_job(transcribe, s3):+ """AWS accepts the request and fails the job asynchronously with a+ FailureReason rather than rejecting Start, because the media is not read+ until the job runs. A poller must be able to observe FAILED."""+ bucket = _unique("transcribe-media")+ s3.create_bucket(Bucket=bucket)+ job_name = _unique("job")++ # Start succeeds; AWS only discovers the object is unreadable once the job runs.+ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": f"s3://{bucket}/missing.mp3"},+ )++ job = _wait_for_status(transcribe, job_name)+ assert job["TranscriptionJobStatus"] == "FAILED"+ assert "can't be accessed" in job["FailureReason"]+ assert "Transcript" not in job+++def test_transcribe_duplicate_job_name_conflicts(transcribe, media):+ _, _, media_uri = media+ job_name = _unique("job")+ kwargs = dict(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ transcribe.start_transcription_job(**kwargs)++ with pytest.raises(ClientError) as exc:+ transcribe.start_transcription_job(**kwargs)+ assert exc.value.response["Error"]["Code"] == "ConflictException"+++def test_transcribe_get_unknown_job_is_not_found(transcribe):+ """NotFoundException is modelled on GetTranscriptionJob but not on Delete, so+ the two operations deliberately differ on a missing job."""+ with pytest.raises(ClientError) as exc:+ transcribe.get_transcription_job(TranscriptionJobName=_unique("nope"))+ assert exc.value.response["Error"]["Code"] == "NotFoundException"+++def test_transcribe_start_requires_a_language_directive(transcribe, media):+ """LanguageCode is not a required member, but AWS still rejects a request that+ supplies neither it nor an IdentifyLanguage flag."""+ _, _, media_uri = media+ with pytest.raises(ClientError) as exc:+ transcribe.start_transcription_job(+ TranscriptionJobName=_unique("job"),+ Media={"MediaFileUri": media_uri},+ )+ assert exc.value.response["Error"]["Code"] == "BadRequestException"+++def test_transcribe_identify_language_without_options_falls_back_to_a_default(transcribe, media):+ """`LanguageOptions` is optional, so a caller can ask for identification and+ name no candidates at all. The job still has to resolve to something, or a+ consumer reading LanguageCode off the completed job gets nothing."""+ _, _, media_uri = media+ job_name = _unique("job")+ started = transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ IdentifyLanguage=True,+ Media={"MediaFileUri": media_uri},+ )["TranscriptionJob"]+ assert started["TranscriptionJobStatus"] == "QUEUED"+ assert "LanguageCode" not in started++ job = _wait_for_status(transcribe, job_name)+ assert job["LanguageCode"] == "en-US"+++def test_transcribe_start_rejects_an_unusable_media_uri(transcribe):+ """AWS models MediaFileUri as `(s3://|http(s*)://).+`, so the scheme check+ is not what makes a URI bad. An unsupported scheme and an HTTP URL with no+ key to address are both unusable and both rejected."""+ for uri in ("ftp://example.com/bucket/audio.mp3", "https://example.com/audio.mp3"):+ with pytest.raises(ClientError) as exc:+ transcribe.start_transcription_job(+ TranscriptionJobName=_unique("job"),+ LanguageCode="en-US",+ Media={"MediaFileUri": uri},+ )+ assert exc.value.response["Error"]["Code"] == "BadRequestException", uri++[email protected]("style", ["path", "path-dash", "virtual"])+def test_transcribe_accepts_https_s3_media_uris(transcribe, s3, media, style):+ """The AWS SDKs hand Transcribe an HTTPS S3 URL as readily as an `s3://`+ one, and AWS accepts both. Rejecting the HTTPS form fails the job at+ submission for any caller that builds its URI that way."""+ bucket, key, _ = media+ uri = {+ "path": f"https://s3.us-east-1.amazonaws.com/{bucket}/{key}",+ "path-dash": f"https://s3-us-east-1.amazonaws.com/{bucket}/{key}",+ "virtual": f"https://{bucket}.s3.us-east-1.amazonaws.com/{key}",+ }[style]++ job_name = _unique("job")+ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": uri},+ )+ job = _wait_for_status(transcribe, job_name)+ # The media must actually have been located and read, not merely accepted.+ assert job["TranscriptionJobStatus"] == "COMPLETED", job.get("FailureReason")+++def test_transcribe_identify_language_resolves_a_language_code(transcribe, media):+ """`IdentifyLanguage` leaves the caller with no language at submission+ time; AWS fills LanguageCode in once the job runs, alongside the+ confidence score. A consumer reading the completed job must find it."""+ _, _, media_uri = media+ job_name = _unique("job")++ started = transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ IdentifyLanguage=True,+ LanguageOptions=["fr-FR", "en-US"],+ Media={"MediaFileUri": media_uri},+ )["TranscriptionJob"]+ assert "LanguageCode" not in started++ job = _wait_for_status(transcribe, job_name)+ assert job["IdentifyLanguage"] is True+ assert job["LanguageCode"] == "fr-FR"+ assert job["IdentifiedLanguageScore"] == pytest.approx(1.0)+++def test_transcribe_identify_multiple_languages_reports_language_codes(transcribe, media):+ """A multi-language job reports `LanguageCodes` and deliberately leaves+ `LanguageCode` unset, exactly as AWS does, so a consumer resolving the+ source language has to fall back to the list."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ IdentifyMultipleLanguages=True,+ LanguageOptions=["en-AU", "fr-FR"],+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)++ assert "LanguageCode" not in job+ codes = job["LanguageCodes"]+ assert [entry["LanguageCode"] for entry in codes] == ["en-AU", "fr-FR"]+ # Every reported language must own some of the audio, otherwise the span+ # breakdown is decorative.+ assert all(entry["DurationInSeconds"] > 0 for entry in codes)+++def test_transcribe_transcript_carries_segments_and_speaker_labels(transcribe, s3, media):+ """`ShowSpeakerLabels` and `ShowAlternatives` change what the transcript+ document contains, not just what the job echoes. Consumers read+ `results.segments`, `results.audio_segments` and `results.speaker_labels`+ to render a transcript, so those have to be present and index-aligned."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ Settings={+ "ShowSpeakerLabels": True,+ "MaxSpeakerLabels": 10,+ "ShowAlternatives": True,+ "MaxAlternatives": 4,+ },+ )+ job = _wait_for_status(transcribe, job_name)+ bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ results = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())["results"]++ segments = results["segments"]+ audio_segments = results["audio_segments"]+ speaker_segments = results["speaker_labels"]["segments"]+ assert len(segments) == len(audio_segments) == len(speaker_segments) > 1++ for segment, audio, speaker in zip(segments, audio_segments, speaker_segments):+ assert segment["start_time"] == audio["start_time"] == speaker["start_time"]+ assert len(segment["alternatives"]) == 4+ assert audio["speaker_label"] == speaker["speaker_label"]+ assert audio["speaker_label"].startswith("spk_")++ # The joined text must still be the transcript, not a reshuffled version.+ assert " ".join(a["transcript"] for a in audio_segments) == (+ results["transcripts"][0]["transcript"]+ )+++def test_transcribe_omits_diarisation_when_not_requested(transcribe, s3, media):+ """A caller that never asked for speaker labels or alternatives should not+ find them in the output, or it cannot tell the settings had any effect."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)+ bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ results = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())["results"]++ assert "speaker_labels" not in results+ assert "segments" not in results+ assert results["audio_segments"]+ assert results["items"]+++def test_transcribe_redaction_writes_only_the_redacted_transcript(transcribe, s3, media):+ """`RedactionOutput=redacted` replaces the transcript rather than adding to+ it: AWS writes `redacted-<name>` and no unredacted copy, and reports the+ location as RedactedTranscriptFileUri."""+ bucket, _, media_uri = media+ job_name = _unique("job")+ output_key = "out/job.json"++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ OutputBucketName=bucket,+ OutputKey=output_key,+ ContentRedaction={+ "RedactionType": "PII",+ "RedactionOutput": "redacted",+ "PiiEntityTypes": ["ALL"],+ },+ )+ job = _wait_for_status(transcribe, job_name)++ assert job["ContentRedaction"]["RedactionOutput"] == "redacted"+ assert "TranscriptFileUri" not in job["Transcript"]+ _, redacted_key = _split_transcript_uri(job["Transcript"]["RedactedTranscriptFileUri"])+ # The prefix moves onto the file name, not the whole key.+ assert redacted_key == "out/redacted-job.json"++ document = json.loads(s3.get_object(Bucket=bucket, Key=redacted_key)["Body"].read())+ assert "[PII]" in document["results"]["transcripts"][0]["transcript"]++ with pytest.raises(ClientError):+ s3.get_object(Bucket=bucket, Key=output_key)+++def test_transcribe_redaction_can_keep_both_transcripts(transcribe, s3, media):+ """`redacted_and_unredacted` is the variant that keeps the original, and+ both URIs have to be reported for a caller to reach either."""+ bucket, _, media_uri = media+ job_name = _unique("job")+ output_key = "both/job.json"++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ OutputBucketName=bucket,+ OutputKey=output_key,+ ContentRedaction={+ "RedactionType": "PII",+ "RedactionOutput": "redacted_and_unredacted",+ "PiiEntityTypes": ["ALL"],+ },+ )+ job = _wait_for_status(transcribe, job_name)++ _, plain_key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ _, redacted_key = _split_transcript_uri(job["Transcript"]["RedactedTranscriptFileUri"])+ assert plain_key == output_key+ assert redacted_key == "both/redacted-job.json"++ plain = json.loads(s3.get_object(Bucket=bucket, Key=plain_key)["Body"].read())+ redacted = json.loads(s3.get_object(Bucket=bucket, Key=redacted_key)["Body"].read())+ plain_text = plain["results"]["transcripts"][0]["transcript"]+ redacted_text = redacted["results"]["transcripts"][0]["transcript"]+ assert "[PII]" not in plain_text+ assert "[PII]" in redacted_text+++def test_transcribe_writes_requested_subtitle_files(transcribe, s3, media):+ """Subtitles are reported as SubtitleFileUris, so the files have to exist+ and be well-formed for each requested format."""+ bucket, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ OutputBucketName=bucket,+ OutputKey="subs/job.json",+ Subtitles={"Formats": ["vtt", "srt"], "OutputStartIndex": 1},+ )+ job = _wait_for_status(transcribe, job_name)++ subtitles = job["Subtitles"]+ assert subtitles["Formats"] == ["vtt", "srt"]+ assert subtitles["OutputStartIndex"] == 1+ assert len(subtitles["SubtitleFileUris"]) == 2++ bodies = {}+ for uri in subtitles["SubtitleFileUris"]:+ _, sub_key = _split_transcript_uri(uri)+ bodies[sub_key] = s3.get_object(Bucket=bucket, Key=sub_key)["Body"].read().decode()++ assert bodies["subs/job.vtt"].startswith("WEBVTT")+ # WebVTT separates the cue timestamp with a dot, SubRip with a comma.+ assert " --> " in bodies["subs/job.vtt"]+ assert "00:00:00.000 --> " in bodies["subs/job.vtt"]+ assert bodies["subs/job.srt"].startswith("1\n00:00:00,000 --> ")+++def test_transcribe_redaction_redacts_the_subtitle_files_too(transcribe, s3, media):+ """Subtitles carry the transcript text. If they are rendered from the+ unredacted document, `RedactionOutput=redacted` writes the raw text to the+ bucket in a `.vtt` while claiming it never left, which defeats the whole+ point of that layout."""+ bucket, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ OutputBucketName=bucket,+ OutputKey="redsubs/job.json",+ ContentRedaction={+ "RedactionType": "PII",+ "RedactionOutput": "redacted",+ "PiiEntityTypes": ["ALL"],+ },+ Subtitles={"Formats": ["vtt"]},+ )+ job = _wait_for_status(transcribe, job_name)++ _, subtitle_key = _split_transcript_uri(job["Subtitles"]["SubtitleFileUris"][0])+ # The name has to match the content, otherwise `redsubs/job.vtt` sits next+ # to `redsubs/redacted-job.json` looking like the unredacted counterpart.+ assert subtitle_key == "redsubs/redacted-job.vtt"++ body = s3.get_object(Bucket=bucket, Key=subtitle_key)["Body"].read().decode()+ assert "[PII]" in body++ # The digest token is what redaction masks; finding it means the raw+ # transcript reached the bucket.+ _, redacted_key = _split_transcript_uri(job["Transcript"]["RedactedTranscriptFileUri"])+ document = json.loads(s3.get_object(Bucket=bucket, Key=redacted_key)["Body"].read())+ assert "media#" not in body, document["results"]["transcripts"][0]["transcript"]++ with pytest.raises(ClientError):+ s3.get_object(Bucket=bucket, Key="redsubs/job.vtt")+++def test_transcribe_multi_language_items_agree_with_their_segment(transcribe, s3, media):+ """Every word item belongs to exactly one audio segment, so its+ `language_code` has to be that segment's. Tagging all items with the first+ resolved language gives a document that contradicts itself, and a consumer+ grouping items by language gets the wrong grouping."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ IdentifyMultipleLanguages=True,+ LanguageOptions=["en-AU", "fr-FR"],+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)+ bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ results = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())["results"]++ items = results["items"]+ audio_segments = results["audio_segments"]+ # Both languages must actually appear, or the check is vacuous.+ assert len({s["language_code"] for s in audio_segments}) == 2++ for segment in audio_segments:+ for index in segment["items"]:+ assert items[index]["language_code"] == segment["language_code"], (+ f"item {index} disagrees with audio segment {segment['id']}"+ )+++def test_transcribe_single_language_items_carry_no_language_tag(transcribe, s3, media):+ """A single-language job carries the code on the job record, so repeating+ it on every item is noise the real service does not emit."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)+ bucket, key = _split_transcript_uri(job["Transcript"]["TranscriptFileUri"])+ results = json.loads(s3.get_object(Bucket=bucket, Key=key)["Body"].read())["results"]++ assert all("language_code" not in item for item in results["items"])+++def test_transcribe_identify_multiple_languages_reports_each_language_once(transcribe, media):+ """Repeated LanguageOptions must not become repeated LanguageCodes entries,+ each claiming the full audio. The reported spans would then sum to more+ audio than the job contains."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ IdentifyMultipleLanguages=True,+ LanguageOptions=["en-AU", "en-AU"],+ Media={"MediaFileUri": media_uri},+ )+ job = _wait_for_status(transcribe, job_name)++ codes = job["LanguageCodes"]+ assert [entry["LanguageCode"] for entry in codes] == ["en-AU"]+++ "language_params",+ [+ {"IdentifyMultipleLanguages": True, "LanguageOptions": ["en-AU", "fr-FR"]},+ {"IdentifyMultipleLanguages": True, "LanguageOptions": ["en-AU"]},+ {"IdentifyMultipleLanguages": True},+ ],+ ids=["two-options", "one-option", "no-options"],+)+def test_transcribe_identify_multiple_languages_always_reports_the_array(+ transcribe, media, language_params+):+ """A multi-language job reports `LanguageCodes` however many options it was+ given, including one or none. Collapsing to a scalar `LanguageCode` when+ only one language resolves is a fidelity gap: AWS populates the array+ whenever the flag is set."""+ _, _, media_uri = media+ job_name = _unique("job")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ Media={"MediaFileUri": media_uri},+ **language_params,+ )+ job = _wait_for_status(transcribe, job_name)++ assert job["IdentifyMultipleLanguages"] is True+ assert "LanguageCode" not in job+ assert job["LanguageCodes"]+++def test_transcribe_start_rejects_an_invalid_job_name(transcribe, media):+ _, _, media_uri = media+ with pytest.raises(ClientError) as exc:+ transcribe.start_transcription_job(+ TranscriptionJobName="not a valid name",+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ assert exc.value.response["Error"]["Code"] == "BadRequestException"+++def test_transcribe_list_filters_by_status_and_name(transcribe, media):+ _, _, media_uri = media+ # A hex marker can come out all digits, for which `.upper()` is the+ # identity and the case-insensitivity assertion below proves nothing.+ marker = f"zz{uuid.uuid4().hex[:8]}"+ job_name = f"filter-{marker}"++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ _wait_for_status(transcribe, job_name)++ listed = transcribe.list_transcription_jobs(JobNameContains=marker)+ assert [s["TranscriptionJobName"] for s in listed["TranscriptionJobSummaries"]] == [job_name]++ # The search is not case sensitive on AWS.+ assert transcribe.list_transcription_jobs(JobNameContains=marker.upper())[+ "TranscriptionJobSummaries"+ ]++ completed = transcribe.list_transcription_jobs(+ Status="COMPLETED", JobNameContains=marker+ )+ assert completed["Status"] == "COMPLETED"+ assert len(completed["TranscriptionJobSummaries"]) == 1++ assert (+ transcribe.list_transcription_jobs(Status="FAILED", JobNameContains=marker)[+ "TranscriptionJobSummaries"+ ]+ == []+ )+++def test_transcribe_list_paginates(transcribe, media):+ """MaxResults must cap the page and hand back a NextToken that resumes exactly+ where the previous page stopped, with no gaps or repeats."""+ _, _, media_uri = media+ marker = uuid.uuid4().hex[:8]+ names = {f"page-{marker}-{i}" for i in range(3)}+ for name in names:+ transcribe.start_transcription_job(+ TranscriptionJobName=name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )++ first = transcribe.list_transcription_jobs(JobNameContains=marker, MaxResults=2)+ assert len(first["TranscriptionJobSummaries"]) == 2+ assert first["NextToken"]++ second = transcribe.list_transcription_jobs(+ JobNameContains=marker, MaxResults=2, NextToken=first["NextToken"]+ )+ assert len(second["TranscriptionJobSummaries"]) == 1+ assert "NextToken" not in second++ seen = {+ s["TranscriptionJobName"]+ for s in first["TranscriptionJobSummaries"] + second["TranscriptionJobSummaries"]+ }+ assert seen == names+++def test_transcribe_delete_removes_the_job_and_is_idempotent(transcribe, media):+ """botocore models no NotFoundException on DeleteTranscriptionJob, so deleting+ a job that is not there succeeds rather than erroring."""+ _, _, media_uri = media+ job_name = _unique("job")+ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ _wait_for_status(transcribe, job_name)++ transcribe.delete_transcription_job(TranscriptionJobName=job_name)++ with pytest.raises(ClientError) as exc:+ transcribe.get_transcription_job(TranscriptionJobName=job_name)+ assert exc.value.response["Error"]["Code"] == "NotFoundException"++ # AWS models no NotFoundException on delete, so a second call succeeds.+ transcribe.delete_transcription_job(TranscriptionJobName=job_name)+++def test_transcribe_list_pagination_survives_a_deletion_between_pages(transcribe, media):+ _, _, media_uri = media+ marker = uuid.uuid4().hex[:8]+ names = [f"del-{marker}-{i}" for i in range(4)]+ for name in names:+ transcribe.start_transcription_job(+ TranscriptionJobName=name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )++ first = transcribe.list_transcription_jobs(JobNameContains=marker, MaxResults=2)+ returned = [s["TranscriptionJobName"] for s in first["TranscriptionJobSummaries"]]++ # Drop a job that page one already returned. An offset-based token would+ # shift the remaining jobs down a slot and skip one.+ transcribe.delete_transcription_job(TranscriptionJobName=returned[0])++ second = transcribe.list_transcription_jobs(+ JobNameContains=marker, MaxResults=2, NextToken=first["NextToken"]+ )+ returned += [s["TranscriptionJobName"] for s in second["TranscriptionJobSummaries"]]++ assert set(returned) == set(names)+ assert len(returned) == len(set(returned))+++def test_transcribe_list_rejects_a_malformed_next_token(transcribe):+ """A token the service did not issue must be refused rather than silently+ treated as the first page, which would loop a paginating client forever."""+ with pytest.raises(ClientError) as exc:+ transcribe.list_transcription_jobs(NextToken="not-a-real-token")+ assert exc.value.response["Error"]["Code"] == "BadRequestException"+++def test_transcribe_recreating_a_deleted_job_name_emits_one_event(+ transcribe, eb, sqs, media+):+ """A deleted job's worker is not cancelled. Without a per-run guard it+ would also drive the replacement job, completing it early and emitting a+ second state change event, so anything chained off the event runs twice."""+ _, _, media_uri = media+ suffix = uuid.uuid4().hex[:8]+ queue_url = sqs.create_queue(QueueName=f"transcribe-rerun-{suffix}")["QueueUrl"]+ queue_arn = sqs.get_queue_attributes(+ QueueUrl=queue_url, AttributeNames=["QueueArn"]+ )["Attributes"]["QueueArn"]++ job_name = f"rerun-{suffix}"+ rule_name = f"transcribe-rerun-rule-{suffix}"+ eb.put_rule(+ Name=rule_name,+ EventPattern=json.dumps(+ {+ "source": ["aws.transcribe"],+ "detail": {"TranscriptionJobName": [job_name]},+ }+ ),+ State="ENABLED",+ )+ eb.put_targets(Rule=rule_name, Targets=[{"Id": "1", "Arn": queue_arn}])++ try:+ start = dict(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ transcribe.start_transcription_job(**start)+ # Delete while still QUEUED, then immediately reuse the name.+ transcribe.delete_transcription_job(TranscriptionJobName=job_name)+ transcribe.start_transcription_job(**start)++ job = _wait_for_status(transcribe, job_name)+ assert job["TranscriptionJobStatus"] == "COMPLETED"++ # Drain well past the point the orphaned worker would have fired.+ deadline = time.time() + 12+ events = []+ while time.time() < deadline:+ batch = sqs.receive_message(+ QueueUrl=queue_url, MaxNumberOfMessages=10, WaitTimeSeconds=1+ ).get("Messages", [])+ events += batch+ for message in batch:+ sqs.delete_message(+ QueueUrl=queue_url, ReceiptHandle=message["ReceiptHandle"]+ )++ assert len(events) == 1, (+ f"expected exactly one state change event, got {len(events)}: "+ f"{[json.loads(e['Body'])['detail'] for e in events]}"+ )+ finally:+ eb.remove_targets(Rule=rule_name, Ids=["1"])+ eb.delete_rule(Name=rule_name)+ sqs.delete_queue(QueueUrl=queue_url)+++def test_transcribe_jobs_are_region_scoped(transcribe, s3, media):+ bucket, _, east_uri = media+ s3.put_object(Bucket=bucket, Key="audio/west.mp3", Body=b"a different sound")+ west_uri = f"s3://{bucket}/audio/west.mp3"++ west = boto3.client(+ "transcribe",+ endpoint_url=_ENDPOINT,+ aws_access_key_id="test",+ aws_secret_access_key="test",+ region_name="us-west-2",+ )+ job_name = _unique("regional")++ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": east_uri},+ )++ assert west.list_transcription_jobs(JobNameContains=job_name)[+ "TranscriptionJobSummaries"+ ] == []+ with pytest.raises(ClientError) as exc:+ west.get_transcription_job(TranscriptionJobName=job_name)+ assert exc.value.response["Error"]["Code"] == "NotFoundException"++ # The background worker pins the region it was started for, so the job+ # still finishes in us-east-1 rather than leaking into whichever request+ # happens to be in flight.+ east_job = _wait_for_status(transcribe, job_name)+ assert east_job["TranscriptionJobStatus"] == "COMPLETED"++ # A same-named job in another region is a different job, transcribing+ # different media. S3 buckets are account-scoped rather than region-scoped,+ # so the two transcripts must not collide on one object.+ west.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": west_uri},+ )+ west_job = _wait_for_status(west, job_name)+ assert west_job["TranscriptionJobStatus"] == "COMPLETED"++ east_uri_out = east_job["Transcript"]["TranscriptFileUri"]+ west_uri_out = west_job["Transcript"]["TranscriptFileUri"]+ assert east_uri_out != west_uri_out++ def _text(transcript_uri):+ out_bucket, key = _split_transcript_uri(transcript_uri)+ document = json.loads(s3.get_object(Bucket=out_bucket, Key=key)["Body"].read())+ return document["results"]["transcripts"][0]["transcript"]++ # Each region's transcript is the one for its own media.+ assert _text(east_uri_out) != _text(west_uri_out)+++def test_transcribe_job_state_change_reaches_an_eventbridge_target(+ transcribe, eb, sqs, media+):+ """The hop the reporter's pipeline depends on: S3 -> Lambda -> Transcribe ->+ EventBridge -> Translate/Bedrock. Without the terminal-state event the four+ operations work but nothing downstream of the job ever runs."""+ _, _, media_uri = media+ suffix = uuid.uuid4().hex[:8]+ queue_url = sqs.create_queue(QueueName=f"transcribe-events-{suffix}")["QueueUrl"]+ queue_arn = sqs.get_queue_attributes(+ QueueUrl=queue_url, AttributeNames=["QueueArn"]+ )["Attributes"]["QueueArn"]++ job_name = f"evented-{suffix}"+ rule_name = f"transcribe-rule-{suffix}"+ eb.put_rule(+ Name=rule_name,+ EventPattern=json.dumps(+ {+ "source": ["aws.transcribe"],+ "detail-type": ["Transcribe Job State Change"],+ "detail": {"TranscriptionJobName": [job_name]},+ }+ ),+ State="ENABLED",+ )+ eb.put_targets(Rule=rule_name, Targets=[{"Id": "1", "Arn": queue_arn}])++ try:+ transcribe.start_transcription_job(+ TranscriptionJobName=job_name,+ LanguageCode="en-US",+ Media={"MediaFileUri": media_uri},+ )+ _wait_for_status(transcribe, job_name)++ deadline = time.time() + 15+ messages = []+ while time.time() < deadline and not messages:+ messages = sqs.receive_message(+ QueueUrl=queue_url, MaxNumberOfMessages=10, WaitTimeSeconds=1+ ).get("Messages", [])++ assert messages, "no Transcribe Job State Change event was delivered"+ event = json.loads(messages[0]["Body"])+ assert event["source"] == "aws.transcribe"+ assert event["detail-type"] == "Transcribe Job State Change"+ assert event["detail"] == {+ "TranscriptionJobName": job_name,+ "TranscriptionJobStatus": "COMPLETED",+ }+ finally:+ eb.remove_targets(Rule=rule_name, Ids=["1"])+ eb.delete_rule(Name=rule_name)+ sqs.delete_queue(QueueUrl=queue_url)