Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
feat(translate): batch text translation jobs (#1667)
ppettitaua9770ed18ea7
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex fa2d58ba..72fe4dc6 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -18,6 +18,11 @@ Versioning follows [Semantic Versioning](https://semver.org/).### Fixed- **CloudFormation — `Capabilities` are enforced under `AUTH=true`** — `CreateStack`, `UpdateStack` and `CreateChangeSet` accepted a template with IAM resources or a `Transform` whatever the request acknowledged, so a deploy that CloudFormation refuses with `InsufficientCapabilitiesException` went through. With `AUTH=true` they now answer that error (HTTP 400, `Requires capabilities : [CAPABILITY_IAM]`, measured) and create nothing: an IAM resource needs `CAPABILITY_IAM` or `CAPABILITY_NAMED_IAM`, one with a custom name `CAPABILITY_NAMED_IAM`, a template with a `Transform` `CAPABILITY_AUTO_EXPAND` on `CreateStack` and `UpdateStack` (not on a change set, as the API documents). Without `AUTH` nothing changes; `GetTemplateSummary` keeps reporting the same set. Contributed by @iot-rocket.+## [Unreleased]++### Added+- **Amazon Translate (`translate`)** — new service emulator for batch text translation jobs: `StartTextTranslationJob`, `DescribeTextTranslationJob`, `ListTextTranslationJobs`, `StopTextTranslationJob`, verified against botocore `translate-2017-07-01`. Jobs walk `SUBMITTED` → `IN_PROGRESS` → `COMPLETED` (paced by `TRANSLATE_JOB_RUN_SECONDS`), read every document under `InputDataConfig.S3Uri` from MiniStack S3, and write output to the location the real service uses — `OutputDataConfig.S3Uri` is reported as `s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/` — as `<target-code>.<input name>`, with a `details/<code>.auxiliary-translation-details.json` summary per target language. `text/plain`, `text/html` and `application/x-xliff+xml` are rewritten in place (an XLIFF job gets a populated `<target>` in every `<trans-unit>` and `target-language` on every `<file>`) and the Office ZIP types are copied through; an unreadable document ends the job `COMPLETED_WITH_ERROR`, and an empty or unreachable input location fails it with a `Message`. `ClientToken` is idempotent, `StopTextTranslationJob` reaches a running worker, and terminal states publish a `Translate TextTranslationJob State Change` event on the default bus. A Node.js Lambda that leaves `@aws-sdk/client-translate` unbundled reaches the service through the local executor's stub. The translated text is deterministic canned output — there is no machine translation. The synchronous `TranslateText` / `TranslateDocument` pair, custom terminologies, parallel data, `ListLanguages` and tagging are not implemented. Contributed by @ppettitau.+## [1.5.9] — 2026-09-08### Addeddiff --git a/README.md b/README.mdindex bf705c93..4018a27c 100644--- a/README.md+++ b/README.md@@ -428,6 +428,7 @@ subnet = ec2.create_subnet(| **Bedrock Agent Runtime** | InvokeAgent, InvokeInlineAgent, GetAgentMemory, DeleteAgentMemory, Retrieve, RetrieveAndGenerate, RetrieveAndGenerateStream, Rerank, CreateSession, GetSession, UpdateSession, DeleteSession, EndSession, ListSessions, CreateInvocation, ListInvocations, PutInvocationStep, GetInvocationStep, ListInvocationSteps, InvokeFlow, StartFlowExecution, StopFlowExecution, GetFlowExecution, ListFlowExecutions, ListFlowExecutionEvents, GetExecutionFlowSnapshot, OptimizePrompt | All 31 operations verified against botocore `bedrock-agent-runtime-2023-07-26`; eventstream responses on streaming operations; `Retrieve` searches the documents Knowledge Base ingestion stored (lexical scoring — no embeddings) and returns content, `s3Location` and score; `InvokeAgent` answers a canned reply without orchestration and emits no trace events — it never fabricates an `ORCHESTRATE` trace || **Bedrock AgentCore** | Control plane (`bedrock-agentcore-control`): CreateAgentRuntime, GetAgentRuntime, ListAgentRuntimes, UpdateAgentRuntime, DeleteAgentRuntime, ListAgentRuntimeVersions, CreateAgentRuntimeEndpoint, GetAgentRuntimeEndpoint, ListAgentRuntimeEndpoints, UpdateAgentRuntimeEndpoint, DeleteAgentRuntimeEndpoint. Data plane (`bedrock-agentcore`): InvokeAgentRuntime | Agent-runtime lifecycle with `CREATING`/`UPDATING`→`READY` transitions, versioned ARNs (`agent/{uuid}:{version}`, `agentEndpoint/{uuid}`); request/response shapes verified against botocore `bedrock-agentcore-control` / `bedrock-agentcore`; account- and region-scoped state; `InvokeAgentRuntime` is a deterministic echo (no model inference), matching the requested emulation scope || **Transcribe** | StartTranscriptionJob, GetTranscriptionJob, ListTranscriptionJobs, DeleteTranscriptionJob | Batch transcription jobs only; streaming is a separate endpoint (`transcribestreaming`). All four operations verified against botocore `transcribe-2017-10-26`. Jobs walk `QUEUED` → `IN_PROGRESS` → `COMPLETED` over `TRANSCRIBE_JOB_RUN_SECONDS` (default 2, `0` completes immediately); the media object is read from MiniStack S3 via an `s3://`, path-style or virtual-hosted HTTPS URI, and an unreadable URI fails the job with a `FailureReason` rather than erroring at `Start`, as on AWS; the transcript is written back to S3 in the real result format (`results.transcripts`, word-level `results.items`, `results.audio_segments`, plus `results.segments` and `results.speaker_labels` when the matching `Settings` are set) at `OutputBucketName`/`OutputKey`, or to an auto-created `ministack-transcribe-output` bucket. `IdentifyLanguage` and `IdentifyMultipleLanguages` resolve a language on completion, `ContentRedaction` controls whether a `redacted-` copy replaces or accompanies the transcript, and `Subtitles` writes WebVTT and SubRip files, redacted alongside the transcript when redaction is on. Reaching a terminal state publishes a `Transcribe Job State Change` event, so EventBridge rules downstream of a job fire. There is no speech recognition, so the transcript is deterministic canned text keyed on a digest of the media URI |+| **Translate** | StartTextTranslationJob, DescribeTextTranslationJob, ListTextTranslationJobs, StopTextTranslationJob | Batch text translation jobs only; the synchronous `TranslateText` / `TranslateDocument` pair and the custom terminology / parallel data families are not implemented. All four operations verified against botocore `translate-2017-07-01`. Jobs walk `SUBMITTED` → `IN_PROGRESS` → `COMPLETED` over `TRANSLATE_JOB_RUN_SECONDS` (default 2, `0` completes immediately); every document under `InputDataConfig.S3Uri` is read from MiniStack S3 and written back to the rewritten output location AWS uses — `OutputDataConfig.S3Uri` is reported as `s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/` — as `<target-code>.<input name>`, with a `details/<code>.auxiliary-translation-details.json` summary per target language. `text/plain` is translated line by line, `text/html` keeps its markup and translates only text nodes, `application/x-xliff+xml` gets a populated `<target>` in every `<trans-unit>` plus `target-language` on every `<file>`, and the three Office ZIP types are copied through. A document that can't be read as its content type is counted in `JobDetails` and the job ends `COMPLETED_WITH_ERROR`; an empty or unreadable input location fails the job with a `Message` rather than erroring at `Start`. `ClientToken` is idempotent, `StopTextTranslationJob` reaches a running worker, and reaching a terminal state publishes a `Translate TextTranslationJob State Change` event, so EventBridge rules downstream of a job fire. There is no machine translation, so the output is deterministic canned text derived from the source || **MSK** | CreateCluster, ListClusters, DescribeCluster, DeleteCluster, GetBootstrapBrokers, ListNodes, CreateConfiguration, ListConfigurations, DescribeConfiguration, ListConfigurationRevisions, DescribeConfigurationRevision, BatchAssociateScramSecret, BatchDisassociateScramSecret, ListScramSecrets, TagResource, UntagResource, ListTagsForResource | Kafka control plane; `GetBootstrapBrokers` honors `MINISTACK_MSK_BOOTSTRAP` so clients route to a real broker you bring (Redpanda, Kafka, KRaft) — the Kafka wire protocol itself is not emulated || **WAF v2** | CreateWebACL, GetWebACL, UpdateWebACL, DeleteWebACL, ListWebACLs, AssociateWebACL, DisassociateWebACL, GetWebACLForResource, ListResourcesForWebACL, CreateIPSet, GetIPSet, UpdateIPSet, DeleteIPSet, ListIPSets, CreateRuleGroup, GetRuleGroup, UpdateRuleGroup, DeleteRuleGroup, ListRuleGroups, TagResource, UntagResource, ListTagsForResource, CheckCapacity, DescribeManagedRuleGroup | LockToken enforced on Update/Delete; resource associations tracked || **Step Functions** | CreateStateMachine, DeleteStateMachine, DescribeStateMachine, UpdateStateMachine, ListStateMachines, StartExecution, StartSyncExecution, StopExecution, DescribeExecution, DescribeStateMachineForExecution, ListExecutions, GetExecutionHistory, SendTaskSuccess, SendTaskFailure, SendTaskHeartbeat, CreateActivity, DeleteActivity, DescribeActivity, ListActivities, GetActivityTask, TestState, TagResource, UntagResource, ListTagsForResource | Full ASL interpreter; Retry/Catch; waitForTaskToken; Activities (worker pattern); Pass/Task/Choice/Wait/Succeed/Fail/Map/Parallel; TestState API with mock and inspectionLevel support; SFN_MOCK_CONFIG for AWS SFN Local compatible mock testing; intrinsic functions (States.StringToJson, States.JsonToString, States.JsonMerge, States.Format); nested startExecution.sync |@@ -859,6 +860,7 @@ end-to-end without any client config.| `MINISTACK_MSK_BOOTSTRAP_SASL_SCRAM` | _(unset)_ | SASL/SCRAM bootstrap string (`BootstrapBrokerStringSaslScram`) || `MINISTACK_MSK_BOOTSTRAP_SASL_IAM` | _(unset)_ | SASL/IAM bootstrap string (`BootstrapBrokerStringSaslIam`) || `TRANSCRIBE_JOB_RUN_SECONDS` | `2` | Wall-clock time a Transcribe job spends between `QUEUED` and `COMPLETED`. Set `0` to complete immediately, or raise it when a test needs to observe `IN_PROGRESS` |+| `TRANSLATE_JOB_RUN_SECONDS` | `2` | Wall-clock time a Translate batch job spends between `SUBMITTED` and `COMPLETED`. Set `0` to complete immediately, or raise it when a test needs to observe `IN_PROGRESS` or stop a running job. Also settable without a restart by posting `{"translate._JOB_RUN_SECONDS": 10}` to `/_ministack/config` |### API Gateway HTTP proxy execution modeldiff --git a/ministack/app.py b/ministack/app.pyindex 3f850a79..b526fc9e 100644--- a/ministack/app.py+++ b/ministack/app.py@@ -398,6 +398,7 @@ SERVICE_REGISTRY = {"sts": {"module": "sts"},"tagging": {"module": "tagging"},"transcribe": {"module": "transcribe"},+ "translate": {"module": "translate"},"transfer": {"module": "transfer"},"waf": {"module": "waf_v1"},"waf-regional": {"module": "waf_v1"},@@ -494,6 +495,7 @@ _state_map = {"bedrock_agentcore": "bedrock_agentcore","msk": "msk","transcribe": "transcribe",+ "translate": "translate",}SERVICE_NAME_ALIASES = {@@ -1222,6 +1224,7 @@ async def _handle_admin_config_request(path: str, method: str, body: bytes):"athena.ATHENA_DATA_DIR","stepfunctions._sfn_mock_config","stepfunctions._SFN_WAIT_SCALE",+ "translate._JOB_RUN_SECONDS","lambda_svc.LAMBDA_EXECUTOR","cloudtrail._recording_enabled","alb.TARGET_CONNECT_TIMEOUT",@@ -1243,14 +1246,14 @@ async def _handle_admin_config_request(path: str, method: str, body: bytes):mod_name, var_name = key.rsplit(".", 1)try:mod = __import__(f"ministack.services.{mod_name}", fromlist=[var_name])- if key == "stepfunctions._SFN_WAIT_SCALE":+ if key in ("stepfunctions._SFN_WAIT_SCALE", "translate._JOB_RUN_SECONDS"):try:float_value = float(value)except (ValueError, TypeError):- logger.warning("/_ministack/config: invalid SFN_WAIT_SCALE=%r", value)+ logger.warning("/_ministack/config: invalid %s=%r", var_name, value)continueif not math.isfinite(float_value) or float_value < 0:- logger.warning("/_ministack/config: invalid SFN_WAIT_SCALE=%r", value)+ logger.warning("/_ministack/config: invalid %s=%r", var_name, value)continuevalue = float_valueelif key == "cloudtrail._recording_enabled":diff --git a/ministack/core/iam_actions.py b/ministack/core/iam_actions.pyindex 030ad868..95303340 100644--- a/ministack/core/iam_actions.py+++ b/ministack/core/iam_actions.py@@ -103,6 +103,7 @@ SERVICE_TO_IAM_NAMESPACE: dict[str, str] = {"sts": "sts","tagging": "tag","transcribe": "transcribe",+ "translate": "translate","transfer": "transfer","waf": "waf","waf-regional": "waf-regional",diff --git a/ministack/core/lambda_runtime.py b/ministack/core/lambda_runtime.pyindex 015baa48..bc3e5025 100644--- a/ministack/core/lambda_runtime.py+++ b/ministack/core/lambda_runtime.py@@ -578,6 +578,7 @@ fs.write = function(fd, ...args) {"@aws-sdk/client-servicediscovery": "Route53AutoNaming_v20170314","@aws-sdk/client-resource-groups-tagging-api": "ResourceGroupsTaggingAPI_20170126","@aws-sdk/client-cloudtrail": "com.amazonaws.cloudtrail.v20131101.CloudTrail_20131101",+ "@aws-sdk/client-translate": "AWSShineFrontendService_20170701",};function _jsonRpcRequest(targetPrefix, opName, params) {diff --git a/ministack/core/router.py b/ministack/core/router.pyindex e51a8c09..9f8d006c 100644--- a/ministack/core/router.py+++ b/ministack/core/router.py@@ -151,6 +151,10 @@ SERVICE_PATTERNS = {"target_prefixes": ["Transcribe."],"host_patterns": [r"transcribe\."],},+ "translate": {+ "target_prefixes": ["AWSShineFrontendService_"],+ "host_patterns": [r"translate\."],+ },"airflow": {"host_patterns": [r"airflow\."],"path_patterns": [diff --git a/ministack/services/translate.py b/ministack/services/translate.pynew file mode 100644index 00000000..e895448b--- /dev/null+++ b/ministack/services/translate.py@@ -0,0 +1,1052 @@+# Copyright (c) 2026 MiniStack Contributors. SPDX-License-Identifier: MIT+# Copies or substantial portions, including AI-assisted ports or rewrites, must retain this notice (see LICENSE).+"""+Amazon Translate emulator.++JSON 1.1 protocol with X-Amz-Target prefix ``AWSShineFrontendService_20170701``+(``translate-2017-07-01``).++Implemented:+ StartTextTranslationJob, DescribeTextTranslationJob, ListTextTranslationJobs,+ StopTextTranslationJob.++Batch jobs run as a background task that walks SUBMITTED -> IN_PROGRESS ->+COMPLETED over ``TRANSLATE_JOB_RUN_SECONDS`` (0 completes immediately). The+worker reads every document under ``InputDataConfig.S3Uri`` out of MiniStack's+S3 store, writes one translated document per input per target language, and+publishes a ``Translate TextTranslationJob State Change`` event so EventBridge+rules downstream of a batch job fire the way they do on AWS.++Output lands where the real service puts it rather than where the caller asked:+``OutputDataConfig.S3Uri`` is rewritten to+``s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/`` and reported that+way from ``DescribeTextTranslationJob``, which is the location a consumer has+to read. Translated documents keep the input name with the target language code+prefixed onto it (``fr.transcript.xlf``), and each target language gets a+``details/<code>.auxiliary-translation-details.json`` summary alongside them, as+on AWS.++There is no machine translation here. The translated text is deterministic+canned output derived from the source text, following the same approach as+``bedrock_runtime._mock_reply`` and ``transcribe._mock_transcript``: identical+input gives identical output, and different input gives distinguishable output.+The document is rewritten in place rather than replaced, so the structure a+consumer parses is real:++ * ``text/plain`` is translated line by line, so line counts survive.+ * ``text/html`` keeps its markup and translates only the text nodes.+ * ``application/x-xliff+xml`` gets a populated ``<target>`` in every+ ``<trans-unit>`` and ``target-language`` on every ``<file>``, which is what+ an XLIFF round trip merges back onto its source segments.+ * The Office content types are ZIP containers, so they are copied through+ byte for byte and counted as translated rather than corrupted.++A document that cannot be decoded or parsed is counted in+``JobDetails.DocumentsWithErrorsCount`` and the job ends+``COMPLETED_WITH_ERROR``, as on AWS. An input location with no readable+documents fails the job with a ``Message`` rather than erroring at ``Start``.+``Message`` is filled on success as well as failure, which is what the service+reference shows a finished job reporting.++``ClientToken`` is honoured: replaying a start request with a token already+seen returns the original job instead of starting a second one.++Deferred:+ TranslateText and TranslateDocument (the synchronous operations),+ the custom terminology and parallel data resource families, ListLanguages+ and TagResource / UntagResource / ListTagsForResource. ``TerminologyNames``,+ ``ParallelDataNames``, ``Settings`` and ``OutputDataConfig.EncryptionKey``+ round-trip on the job record without changing the output.+"""++import asyncio+import base64+import copy+import json+import logging+import os+import re+import time+import xml.etree.ElementTree as ET++from defusedxml.common import DefusedXmlException+from defusedxml.ElementTree import fromstring as _xml_fromstring++from ministack.core.persistence import load_state+from ministack.core.responses import (+ AccountRegionScopedDict,+ error_response_json,+ get_account_id,+ get_region,+ json_response,+ new_uuid,+ now_iso,+ request_scope,+)++logger = logging.getLogger("translate")++# How long a job spends between SUBMITTED and COMPLETED. Same knob shape as+# TRANSLATE_JOB_RUN_SECONDS: tests that assert on IN_PROGRESS need a non-zero+# value, tests that just want a result set it to 0. Also settable at runtime+# through /_ministack/config, so a test can pin the pace without a restart.+_JOB_RUN_SECONDS = float(os.environ.get("TRANSLATE_JOB_RUN_SECONDS", "2"))++# Granularity at which a running job notices StopTextTranslationJob. Without+# slicing the wait, a stop request would sit unobserved until the next phase+# boundary, so a job with a long run time would stay STOP_REQUESTED for as+# long as the caller was prepared to wait.+_STOP_POLL_SECONDS = 0.1++_TERMINAL_STATUSES = ("COMPLETED", "COMPLETED_WITH_ERROR", "FAILED", "STOPPED")++_JOB_STATUSES = (+ "SUBMITTED",+ "IN_PROGRESS",+ "COMPLETED",+ "COMPLETED_WITH_ERROR",+ "FAILED",+ "STOP_REQUESTED",+ "STOPPED",+)++# AWS models JobName as ^([\p{L}\p{Z}\p{N}_.:/=+\-%@]*)$. Python's re has no+# Unicode property escapes, so \w and \s stand in for \p{L}\p{N}_ and \p{Z};+# the effect on a real job name is the same.+_JOB_NAME_RE = re.compile(r"^[\w\s.:/=+\-%@]{1,256}$", re.UNICODE)+_CLIENT_TOKEN_RE = re.compile(r"^[a-zA-Z0-9-]{1,64}$")+_ROLE_ARN_RE = re.compile(r"^arn:aws(-[^:]+)?:iam::[0-9]{12}:role/.+$")+_S3_URI_RE = re.compile(r"^s3://[a-z0-9][.\-a-z0-9]{1,61}[a-z0-9](/.*)?$")++_MAX_TARGET_LANGUAGES = 10++# The content types batch translation accepts. The three Office types are ZIP+# containers rather than text, which is why they are handled separately below.+_PLAIN_TEXT = "text/plain"+_HTML = "text/html"+_XLIFF = "application/x-xliff+xml"+_DOCX = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"+_PPTX = "application/vnd.openxmlformats-officedocument.presentationml.presentation"+_XLSX = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"++_OPAQUE_CONTENT_TYPES = (_DOCX, _PPTX, _XLSX)+_SUPPORTED_CONTENT_TYPES = (_PLAIN_TEXT, _HTML, _XLIFF) + _OPAQUE_CONTENT_TYPES++# What "auto" resolves to. Real Translate detects the source language per+# document with Comprehend; there is no detector here, so the resolved code is+# fixed and only ever surfaces in the auxiliary details file. The job record+# keeps reporting "auto", as on AWS.+_DEFAULT_SOURCE_LANGUAGE = "en"++_XLIFF_12_NS = "urn:oasis:names:tc:xliff:document:1.2"++_INPUT_UNREADABLE = (+ "The input location that you provided can't be accessed. Make sure that the "+ "S3 folder exists and that Amazon Translate has read permission, then try "+ "your request again."+)++_NO_INPUT_DOCUMENTS = (+ "No documents were found at the input location that you provided. Make sure "+ "that the S3 folder contains at least one document and try your request again."+)++_OUTPUT_UNWRITABLE = (+ "The output location that you provided can't be written to. Make sure that "+ "the S3 bucket exists and that Amazon Translate has write permission, then "+ "try your request again."+)++# Public members of the TextTranslationJobProperties shape, in the order+# botocore models them. Internal bookkeeping lives under keys prefixed with "_"+# and never reaches the wire.+_JOB_MEMBERS = (+ "JobId",+ "JobName",+ "JobStatus",+ "JobDetails",+ "SourceLanguageCode",+ "TargetLanguageCodes",+ "TerminologyNames",+ "ParallelDataNames",+ "Message",+ "SubmittedTime",+ "EndTime",+ "InputDataConfig",+ "OutputDataConfig",+ "DataAccessRoleArn",+ "Settings",+)++# Members echoed straight back from the request without changing behaviour.+_ECHOED_MEMBERS = ("TerminologyNames", "ParallelDataNames", "Settings")++_jobs = AccountRegionScopedDict() # job_id -> job record+_client_tokens = AccountRegionScopedDict() # ClientToken -> job_id+++# ---------------------------------------------------------------------------+# State+# ---------------------------------------------------------------------------+++def reset():+ _jobs.clear()+ _client_tokens.clear()+++def get_state():+ return copy.deepcopy({"jobs": _jobs, "client_tokens": _client_tokens})+++def restore_state(data):+ if not data:+ return+ _jobs.clear()+ _client_tokens.clear()+ # Not `or {}`: AccountRegionScopedDict.__bool__ is scope-relative, and+ # restore runs in the default account/region. A snapshot holding jobs only+ # in other scopes would read as falsy and every one of them would be lost.+ jobs = data.get("jobs")+ if jobs is not None:+ _jobs.update(jobs)+ tokens = data.get("client_tokens")+ if tokens is not None:+ _client_tokens.update(tokens)+ _fail_orphaned_jobs()+++def load_persisted_state(data):+ restore_state(data)+++def _fail_orphaned_jobs():+ """A job that was mid-flight when the process stopped has no worker any+ more. Leaving it SUBMITTED, IN_PROGRESS or STOP_REQUESTED strands every+ caller polling DescribeTextTranslationJob forever, so it is failed the way+ AWS fails a job it cannot finish."""+ for job in _jobs.all_values():+ if job.get("JobStatus") not in _TERMINAL_STATUSES:+ job["JobStatus"] = "FAILED"+ job["Message"] = "Internal Failure. The job did not survive a MiniStack restart."+ job["EndTime"] = time.time()+++try:+ _restored = load_state("translate")+ if _restored:+ restore_state(_restored)+except Exception:+ logger.exception("Failed to restore persisted Translate state; continuing with fresh store")+++# ---------------------------------------------------------------------------+# Helpers+# ---------------------------------------------------------------------------+++def _invalid_parameter(message):+ return error_response_json("InvalidParameterValueException", message, 400)+++def _invalid_request(message):+ return error_response_json("InvalidRequestException", message, 400)+++def _not_found(message):+ return error_response_json("ResourceNotFoundException", message, 400)+++def _public_job(job):+ """Project a stored job onto the TextTranslationJobProperties wire shape,+ dropping members AWS omits when they have no value."""+ return {k: job[k] for k in _JOB_MEMBERS if job.get(k) is not None}+++def _parse_s3_uri(uri):+ """Return (bucket, prefix) for an ``s3://`` folder URI, or (None, None).++ Translate models both S3Uri members as ``s3://...`` only — unlike+ Transcribe's Media, an HTTPS form is not accepted — and treats the path as+ a folder, so a URI without a trailing slash still addresses a prefix."""+ if not isinstance(uri, str) or not _S3_URI_RE.match(uri):+ return None, None+ rest = uri[len("s3://"):]+ bucket, _, prefix = rest.partition("/")+ if prefix and not prefix.endswith("/"):+ prefix += "/"+ return bucket, prefix+++def _output_folder_prefix(job_id, output_prefix):+ """The folder real Translate creates inside the caller's output location.++ Every batch job writes to ``<prefix><account-id>-TranslateText-<JobId>/``+ rather than to the prefix itself, so two jobs pointed at one location do+ not overwrite each other and a consumer can tell whose output it is+ reading."""+ return f"{output_prefix}{get_account_id()}-TranslateText-{job_id}/"+++def _sort_key(job):+ """Submission time with the job id breaking ties, so the ordering is total+ and stable across calls. Callers reverse it to get newest first."""+ return (job.get("SubmittedTime") or 0.0, job.get("JobId") or "")+++def _encode_token(job):+ """Tokens carry the sort position of the last item returned, not an index.+ An offset would repeat an item whenever a job was created between pages."""+ submitted, job_id = _sort_key(job)+ raw = json.dumps({"st": submitted, "id": job_id}).encode("utf-8")+ return base64.urlsafe_b64encode(raw).decode("ascii")+++def _decode_token(token):+ try:+ raw = json.loads(base64.urlsafe_b64decode(token.encode("ascii")).decode("utf-8"))+ return (float(raw["st"]), str(raw["id"]))+ except Exception:+ return None+++def _emit_state_change(job_id, status):+ """Publish `Translate TextTranslationJob State Change`, the event AWS emits+ when a batch job reaches a terminal state. Pipelines that chain off a+ finished translation depend on it, so a failure to deliver is logged rather+ than swallowed silently, and never propagates into the job itself."""+ try:+ from ministack.services import eventbridge as _eb++ _eb._dispatch_event(+ {+ "EventId": new_uuid(),+ "Source": "aws.translate",+ "DetailType": "Translate TextTranslationJob State Change",+ # Lowercase members: the Translate event detail is camelCase+ # with a leading lowercase letter, unlike the API shape.+ "Detail": json.dumps({"jobId": job_id, "jobStatus": status}),+ "EventBusName": "default",+ "Time": now_iso(),+ "Resources": [],+ "Account": get_account_id(),+ "Region": get_region(),+ }+ )+ except Exception:+ logger.exception("Translate: failed to publish state change for job %s", job_id)+++# ---------------------------------------------------------------------------+# Mock translation+# ---------------------------------------------------------------------------+++def _mock_translation(text, target_language_code):+ """Deterministic canned translation.++ Tagging the source with its target language keeps the output reproducible+ (an integration test can assert on it), distinguishable per target language+ (a two-language job writes two different documents), and obviously not a+ real translation. Whitespace-only text is left alone so a document's+ blank lines and indentation survive the round trip."""+ if not text or not text.strip():+ return text+ leading = text[: len(text) - len(text.lstrip())]+ trailing = text[len(text.rstrip()):]+ return f"{leading}[{target_language_code}] {text.strip()}{trailing}"+++def _translate_plain_text(text, target_language_code):+ """Line by line, so the line count and blank-line structure of the source+ survive. A consumer that zips source and translated lines together still+ lines up."""+ return "\n".join(_mock_translation(line, target_language_code) for line in text.split("\n"))+++# Splits an HTML document into markup and the text between it. The capture+# group keeps the tags in the result, so reassembly is a plain join. A `>` is+# legal inside a quoted attribute value and inside a comment, so neither can+# be allowed to terminate the match — ending a tag early would translate the+# rest of an attribute and leave the element's real text alone.+_HTML_SPLIT_RE = re.compile(r"(<!--.*?-->|<(?:\"[^\"]*\"|'[^']*'|[^>\"'])*>)", re.DOTALL)++# Text inside these elements is code, not prose. Real Translate leaves it be,+# and translating it would break the document.+_HTML_OPAQUE_ELEMENTS = ("script", "style")++_HTML_TAG_NAME_RE = re.compile(r"^<\s*(/?)\s*([a-zA-Z][a-zA-Z0-9]*)")+++def _translate_html(text, target_language_code):+ """Translate the text nodes and leave the markup untouched, so the result+ is still the document the caller uploaded."""+ parts = _HTML_SPLIT_RE.split(text)+ opaque_depth = 0+ out = []+ for part in parts:+ if part.startswith("<"):+ match = _HTML_TAG_NAME_RE.match(part)+ if match and match.group(2).lower() in _HTML_OPAQUE_ELEMENTS:+ if match.group(1):+ opaque_depth = max(0, opaque_depth - 1)+ elif not part.rstrip().endswith("/>"):+ opaque_depth += 1+ out.append(part)+ elif opaque_depth:+ out.append(part)+ else:+ out.append(_mock_translation(part, target_language_code))+ return "".join(out)+++def _xliff_qname(namespace_uri, tag):+ return f"{{{namespace_uri}}}{tag}" if namespace_uri else tag+++def _translate_xliff(text, target_language_code):+ """Fill in every ``<trans-unit>``'s ``<target>`` from its ``<source>`` and+ stamp ``target-language`` on every ``<file>``.++ This is the shape an XLIFF round trip needs: the caller wrote source+ segments, and merges the translated ``<target>`` text back onto them by+ ``trans-unit`` id. Inline markup inside a ``<source>`` (``<g>``, ``<x/>``)+ is flattened to its text, because there is no real translation to+ re-distribute across it.++ The document comes from the caller's S3 bucket, so it is parsed with+ ``defusedxml`` like every other untrusted XML parse in the emulator.+ ``ET`` is still used to build and serialise, which is safe."""+ root = _xml_fromstring(text)+ namespace_uri = root.tag[1:].split("}")[0] if root.tag.startswith("{") else ""+ if namespace_uri:+ # Otherwise ElementTree serialises the default namespace as `ns0:`,+ # and an XLIFF tool reading the output would not recognise the+ # elements it just round-tripped.+ ET.register_namespace("", namespace_uri)++ file_tag = _xliff_qname(namespace_uri, "file")+ unit_tag = _xliff_qname(namespace_uri, "trans-unit")+ source_tag = _xliff_qname(namespace_uri, "source")+ target_tag = _xliff_qname(namespace_uri, "target")++ for file_el in root.iter(file_tag):+ file_el.set("target-language", target_language_code)++ for unit in root.iter(unit_tag):+ source = unit.find(source_tag)+ if source is None:+ continue+ source_text = "".join(source.itertext())+ target = unit.find(target_tag)+ if target is None:+ target = ET.Element(target_tag)+ unit.insert(list(unit).index(source) + 1, target)+ else:+ for child in list(target):+ target.remove(child)+ target.text = _mock_translation(source_text, target_language_code)+ target.tail = source.tail++ return ET.tostring(root, encoding="unicode", xml_declaration=True)+++def _translate_document(body, content_type, target_language_code):+ """Return the translated bytes for one document, or None if the document+ cannot be read as its declared content type.++ None is a per-document error, not a job failure: AWS counts it in+ DocumentsWithErrorsCount and finishes the job COMPLETED_WITH_ERROR."""+ if content_type in _OPAQUE_CONTENT_TYPES:+ # A ZIP container. Rewriting one would mean rewriting the OOXML parts+ # inside it; copying it through keeps the document openable, which is+ # what a pipeline that just moves files around needs.+ return body++ try:+ text = body.decode("utf-8")+ except UnicodeDecodeError:+ return None++ try:+ if content_type == _XLIFF:+ return _translate_xliff(text, target_language_code).encode("utf-8")+ if content_type == _HTML:+ return _translate_html(text, target_language_code).encode("utf-8")+ return _translate_plain_text(text, target_language_code).encode("utf-8")+ except (ET.ParseError, DefusedXmlException):+ # DefusedXmlException is not a ParseError subclass, so a document with a+ # forbidden DTD or entity has to be named explicitly here or it would+ # escape to _run_job and fail the whole job instead of just this file.+ return None+++# ---------------------------------------------------------------------------+# Job execution+# ---------------------------------------------------------------------------+++def _live_job(job_id, run_id):+ """The job this worker was started for, or None if it has been replaced.+ Job ids are generated per start, so this only differs from a plain lookup+ after a restore, but it keeps a restored record from being driven by a+ worker that no longer owns it."""+ job = _jobs.get(job_id)+ if job is None or job.get("_run_id") != run_id:+ return None+ return job+++async def _wait(job_id, run_id, seconds):+ """Sleep in slices, returning False as soon as the job disappears or a stop+ is requested, so StopTextTranslationJob takes effect promptly instead of at+ the next phase boundary."""+ remaining = seconds+ while remaining > 0:+ await asyncio.sleep(min(_STOP_POLL_SECONDS, remaining))+ remaining -= _STOP_POLL_SECONDS+ job = _live_job(job_id, run_id)+ if job is None or job.get("JobStatus") == "STOP_REQUESTED":+ return False+ job = _live_job(job_id, run_id)+ return job is not None and job.get("JobStatus") != "STOP_REQUESTED"+++def _finalize_stop(job_id, run_id):+ """Complete the transition StopTextTranslationJob started. The stop handler+ only records the request; the worker owns the job's terminal state, so it+ is the one that sets STOPPED and emits the event."""+ job = _live_job(job_id, run_id)+ if job is None or job.get("JobStatus") != "STOP_REQUESTED":+ return+ job["JobStatus"] = "STOPPED"+ job["EndTime"] = time.time()+ logger.info("Translate: job %s stopped", job_id)+ _emit_state_change(job_id, "STOPPED")+++async def _run_job(job_id, run_id, account_id, region):+ """Walk a job to a terminal state. Runs as a background task, so it pins+ the account and region it was started for rather than inheriting whatever+ request happens to be in flight."""+ with request_scope(account_id, region):+ try:+ await _run_job_inner(job_id, run_id)+ except Exception:+ logger.exception("Translate: job %s crashed", job_id)+ job = _live_job(job_id, run_id)+ if job is not None and job.get("JobStatus") not in _TERMINAL_STATUSES:+ job["JobStatus"] = "FAILED"+ job["Message"] = "Internal Failure. Please try your request again."+ job["EndTime"] = time.time()+ _emit_state_change(job_id, "FAILED")+++async def _run_job_inner(job_id, run_id):+ half = _JOB_RUN_SECONDS / 2 if _JOB_RUN_SECONDS > 0 else 0.0++ if not await _wait(job_id, run_id, half):+ _finalize_stop(job_id, run_id)+ return++ job = _live_job(job_id, run_id)+ if job is None:+ return+ job["JobStatus"] = "IN_PROGRESS"++ if not await _wait(job_id, run_id, half):+ _finalize_stop(job_id, run_id)+ return++ job = _live_job(job_id, run_id)+ if job is None:+ return++ _translate_job(job_id, run_id, job)+++def _fail(job_id, run_id, job, message):+ if _live_job(job_id, run_id) is None:+ return+ job["JobStatus"] = "FAILED"+ job["Message"] = message+ job["EndTime"] = time.time()+ logger.info("Translate: job %s failed: %s", job_id, message)+ _emit_state_change(job_id, "FAILED")+++def _input_documents(bucket, prefix):+ """Every readable object under the input prefix, oldest key first.++ Folder markers (a zero-length object whose key ends in "/") are S3 console+ artefacts, not documents, so they are skipped rather than translated into+ an empty output file."""+ from ministack.services import s3 as s3_svc++ store = s3_svc._buckets.get(bucket)+ if store is None:+ return None++ documents = []+ for key in sorted(store.get("objects", {})):+ if not key.startswith(prefix) or key.endswith("/"):+ continue+ relative = key[len(prefix):]+ if not relative:+ continue+ body = s3_svc._get_object_data(bucket, key)+ if body is None:+ continue+ documents.append((relative, body))+ return documents+++def _translated_key(relative_key, target_language_code):+ """``folder/mySourceText.txt`` -> ``folder/fr.mySourceText.txt``. The+ language code goes on the file name, not the path, so a nested input layout+ is preserved in the output."""+ folder, separator, name = relative_key.rpartition("/")+ return f"{folder}{separator}{target_language_code}.{name}"+++def _auxiliary_details(job, target_language_code, input_uri, output_uri, entries, characters):+ """The ``details/<code>.auxiliary-translation-details.json`` summary real+ Translate writes beside the output. Counts are strings there, so they are+ strings here."""+ source_code = job["SourceLanguageCode"]+ if source_code == "auto":+ source_code = _DEFAULT_SOURCE_LANGUAGE+ return {+ "sourceLanguageCode": source_code,+ "targetLanguageCode": target_language_code,+ "charactersTranslated": str(characters),+ "documentCountWithCustomerError": str(+ sum(1 for entry in entries if entry["targetFile"] is None)+ ),+ "documentCountWithServerError": "0",+ "inputDataPrefix": input_uri,+ "outputDataPrefix": output_uri,+ "details": [+ {+ "sourceFile": entry["sourceFile"],+ "targetFile": entry["targetFile"],+ "auxiliaryData": {"appliedTerminologies": []},+ }+ for entry in entries+ if entry["targetFile"] is not None+ ],+ }+++def _translate_job(job_id, run_id, job):+ from ministack.services import s3 as s3_svc++ input_uri = job["InputDataConfig"]["S3Uri"]+ content_type = job["InputDataConfig"]["ContentType"]+ input_bucket, input_prefix = _parse_s3_uri(input_uri)++ documents = _input_documents(input_bucket, input_prefix)+ if documents is None:+ _fail(job_id, run_id, job, _INPUT_UNREADABLE)+ return+ if not documents:+ _fail(job_id, run_id, job, _NO_INPUT_DOCUMENTS)+ return++ output_bucket = job["_output_bucket"]+ output_prefix = job["_output_prefix"]+ if s3_svc._ensure_bucket(output_bucket) is None:+ _fail(job_id, run_id, job, _OUTPUT_UNWRITABLE)+ return++ def _write(key, body, write_content_type):+ response = s3_svc._put_object(+ output_bucket,+ key,+ body,+ {"content-type": write_content_type, "content-length": str(len(body))},+ )+ return not (isinstance(response, tuple) and response[0] >= 300)++ # A document that fails for one target language fails for all of them, so+ # the error count is per input document rather than per output file, which+ # is how JobDetails reports it.+ failed_documents = set()++ for target_language_code in job["TargetLanguageCodes"]:+ entries = []+ characters = 0+ for relative_key, body in documents:+ translated = _translate_document(body, content_type, target_language_code)+ if translated is None:+ failed_documents.add(relative_key)+ entries.append({"sourceFile": relative_key, "targetFile": None})+ logger.info(+ "Translate: job %s could not read %s as %s", job_id, relative_key, content_type+ )+ continue+ target_key = _translated_key(relative_key, target_language_code)+ if not _write(f"{output_prefix}{target_key}", translated, content_type):+ _fail(job_id, run_id, job, _OUTPUT_UNWRITABLE)+ return+ characters += len(body)+ entries.append({"sourceFile": relative_key, "targetFile": target_key})++ output_uri = f"s3://{output_bucket}/{output_prefix}"+ details = _auxiliary_details(+ job, target_language_code, input_uri, output_uri, entries, characters+ )+ details_key = (+ f"{output_prefix}details/{target_language_code}.auxiliary-translation-details.json"+ )+ if not _write(details_key, json.dumps(details).encode("utf-8"), "application/json"):+ _fail(job_id, run_id, job, _OUTPUT_UNWRITABLE)+ return++ translated_count = len(documents) - len(failed_documents)+ job["JobDetails"] = {+ "TranslatedDocumentsCount": translated_count,+ "DocumentsWithErrorsCount": len(failed_documents),+ "InputDocumentsCount": len(documents),+ }+ status = "COMPLETED_WITH_ERROR" if failed_documents else "COMPLETED"+ job["JobStatus"] = status+ # AWS fills Message on success too, not only on failure — its documented+ # completion text is what a caller sees on a finished job.+ if failed_documents:+ job["Message"] = (+ f"Your job has completed with errors. {len(failed_documents)} of "+ f"{len(documents)} documents could not be processed."+ )+ else:+ job["Message"] = "Your job has completed successfully."+ job["EndTime"] = time.time()+ logger.info(+ "Translate: job %s %s, %d of %d document(s) translated into %s under s3://%s/%s",+ job_id,+ status.lower(),+ translated_count,+ len(documents),+ ", ".join(job["TargetLanguageCodes"]),+ output_bucket,+ output_prefix,+ )+ _emit_state_change(job_id, status)+++# ---------------------------------------------------------------------------+# Operations+# ---------------------------------------------------------------------------+++def _validate_start(data):+ """Everything StartTextTranslationJob rejects before a job exists. Returns+ an error response, or None when the request is good."""+ name = data.get("JobName")+ if name is not None and not _JOB_NAME_RE.match(str(name)):+ return _invalid_parameter(+ f"1 validation error detected: Value '{name}' at 'jobName' failed to satisfy "+ "constraint: Member must satisfy regular expression pattern: "+ r"^([\p{L}\p{Z}\p{N}_.:/=+\-%@]*)$"+ )++ token = data.get("ClientToken")+ if not isinstance(token, str) or not _CLIENT_TOKEN_RE.match(token):+ return _invalid_parameter(+ f"1 validation error detected: Value '{token}' at 'clientToken' failed to satisfy "+ "constraint: Member must satisfy regular expression pattern: ^[a-zA-Z0-9-]+$"+ )++ role_arn = data.get("DataAccessRoleArn")+ if not isinstance(role_arn, str) or not _ROLE_ARN_RE.match(role_arn):+ return _invalid_parameter(+ f"1 validation error detected: Value '{role_arn}' at 'dataAccessRoleArn' failed to "+ "satisfy constraint: Member must satisfy regular expression pattern: "+ "arn:aws(-[^:]+)?:iam::[0-9]{12}:role/.+"+ )++ input_config = data.get("InputDataConfig")+ if not isinstance(input_config, dict):+ input_config = {}+ if _parse_s3_uri(input_config.get("S3Uri")) == (None, None):+ return _invalid_parameter(+ f"1 validation error detected: Value '{input_config.get('S3Uri')}' at "+ "'inputDataConfig.s3Uri' failed to satisfy constraint: Member must satisfy regular "+ r"expression pattern: s3://[a-z0-9][\.\-a-z0-9]{1,61}[a-z0-9](/.*)?"+ )++ content_type = input_config.get("ContentType")+ if content_type not in _SUPPORTED_CONTENT_TYPES:+ return _invalid_request(+ f"Unsupported content type: {content_type}. Amazon Translate supports the following "+ f"content types: {', '.join(_SUPPORTED_CONTENT_TYPES)}."+ )++ output_config = data.get("OutputDataConfig")+ if not isinstance(output_config, dict):+ output_config = {}+ if _parse_s3_uri(output_config.get("S3Uri")) == (None, None):+ return _invalid_parameter(+ f"1 validation error detected: Value '{output_config.get('S3Uri')}' at "+ "'outputDataConfig.s3Uri' failed to satisfy constraint: Member must satisfy regular "+ r"expression pattern: s3://[a-z0-9][\.\-a-z0-9]{1,61}[a-z0-9](/.*)?"+ )++ source = data.get("SourceLanguageCode")+ if not isinstance(source, str) or not 2 <= len(source) <= 5:+ return _invalid_parameter(+ f"1 validation error detected: Value '{source}' at 'sourceLanguageCode' failed to "+ "satisfy constraint: Member must have length between 2 and 5"+ )++ targets = data.get("TargetLanguageCodes")+ if not isinstance(targets, list) or not targets:+ return _invalid_parameter(+ "1 validation error detected: Value at 'targetLanguageCodes' failed to satisfy "+ "constraint: Member must have length greater than or equal to 1"+ )+ if len(targets) > _MAX_TARGET_LANGUAGES:+ return _invalid_parameter(+ f"1 validation error detected: Value at 'targetLanguageCodes' failed to satisfy "+ f"constraint: Member must have length less than or equal to {_MAX_TARGET_LANGUAGES}"+ )+ for target in targets:+ if not isinstance(target, str) or not 2 <= len(target) <= 5:+ return _invalid_parameter(+ f"1 validation error detected: Value '{target}' at "+ "'targetLanguageCodes' failed to satisfy constraint: Member must have length "+ "between 2 and 5"+ )+ if source != "auto" and target == source:+ return error_response_json(+ "UnsupportedLanguagePairException",+ f"Unsupported language pair: {source} to {target}.",+ 400,+ {"SourceLanguageCode": source, "TargetLanguageCode": target},+ )+ return None+++def _start_text_translation_job(data):+ error = _validate_start(data)+ if error is not None:+ return error++ # ClientToken is an idempotency token, so a retried start — an SDK retry, a+ # replayed EventBridge delivery — must resolve to the job the first call+ # created rather than a second copy of it writing to a second output folder.+ token = data["ClientToken"]+ existing_id = _client_tokens.get(token)+ if existing_id is not None:+ existing = _jobs.get(existing_id)+ if existing is not None:+ return json_response(+ {"JobId": existing["JobId"], "JobStatus": existing["JobStatus"]}+ )++ job_id = new_uuid().replace("-", "")+ output_bucket, output_prefix = _parse_s3_uri(data["OutputDataConfig"]["S3Uri"])+ job_output_prefix = _output_folder_prefix(job_id, output_prefix)++ run_id = new_uuid()+ job = {+ "JobId": job_id,+ "JobName": data.get("JobName"),+ "JobStatus": "SUBMITTED",+ "JobDetails": None,+ "SourceLanguageCode": data["SourceLanguageCode"],+ "TargetLanguageCodes": list(data["TargetLanguageCodes"]),+ "Message": None,+ "SubmittedTime": time.time(),+ "EndTime": None,+ "InputDataConfig": copy.deepcopy(data["InputDataConfig"]),+ # Reported as the rewritten location from the moment the job exists,+ # the way AWS does: the caller's S3Uri is where the folder goes, not+ # where the documents land, and a consumer reading the original prefix+ # would find nothing.+ "OutputDataConfig": dict(+ data["OutputDataConfig"], S3Uri=f"s3://{output_bucket}/{job_output_prefix}"+ ),+ "DataAccessRoleArn": data["DataAccessRoleArn"],+ "_run_id": run_id,+ "_client_token": token,+ "_output_bucket": output_bucket,+ "_output_prefix": job_output_prefix,+ }+ for member in _ECHOED_MEMBERS:+ value = data.get(member)+ job[member] = copy.deepcopy(value) if value is not None else None++ _jobs[job_id] = job+ _client_tokens[token] = job_id++ asyncio.create_task(_run_job(job_id, run_id, get_account_id(), get_region()))++ return json_response({"JobId": job_id, "JobStatus": job["JobStatus"]})+++def _describe_text_translation_job(data):+ job_id = data.get("JobId")+ if not isinstance(job_id, str) or not job_id.strip():+ return _invalid_parameter(+ "1 validation error detected: Value null at 'jobId' failed to satisfy "+ "constraint: Member must not be null"+ )++ job = _jobs.get(job_id)+ if job is None:+ return _not_found(+ "The batch translation job that you requested could not be found. Check the job ID "+ "and try your request again."+ )++ return json_response({"TextTranslationJobProperties": _public_job(job)})+++def _list_text_translation_jobs(data):+ job_filter = data.get("Filter")+ if not isinstance(job_filter, dict):+ job_filter = {}+ name = job_filter.get("JobName")+ status = job_filter.get("JobStatus")+ before = job_filter.get("SubmittedBeforeTime")+ after = job_filter.get("SubmittedAfterTime")++ if status is not None and status not in _JOB_STATUSES:+ return _invalid_parameter(+ f"1 validation error detected: Value '{status}' at 'filter.jobStatus' failed to "+ "satisfy constraint: Member must satisfy enum value set: "+ f"[{', '.join(_JOB_STATUSES)}]"+ )++ # Coerce the submission-time bounds up front: the wire form is a number, but+ # a caller can send anything, and filtering on an unparseable bound inside+ # the comprehension below would surface as a 500 only once a job exists.+ for field, value in (("submittedBeforeTime", before), ("submittedAfterTime", after)):+ if value is None:+ continue+ try:+ float(value)+ except (TypeError, ValueError):+ return _invalid_parameter(+ f"1 validation error detected: Value '{value}' at 'filter.{field}' failed to "+ "satisfy constraint: Member must be a timestamp"+ )++ # "You can only set one filter at a time." The two submission-time bounds+ # describe a single window, so they count as one filter between them.+ selected = sum(+ 1 for criterion in (name, status, before if before is not None else after)+ if criterion is not None+ )+ if selected > 1:+ return error_response_json(+ "InvalidFilterException",+ "You can specify only one filter at a time. Remove the extra filters and try your "+ "request again.",+ 400,+ )++ max_results = data.get("MaxResults")+ if max_results is None:+ max_results = 100+ else:+ try:+ max_results = int(max_results)+ except (TypeError, ValueError):+ return _invalid_parameter("MaxResults must be an integer")+ if max_results < 1 or max_results > 500:+ return _invalid_parameter(+ f"1 validation error detected: Value '{max_results}' at 'maxResults' failed to "+ "satisfy constraint: Member must have value less than or equal to 500"+ )++ cursor = None+ token = data.get("NextToken")+ if token:+ cursor = _decode_token(token)+ if cursor is None:+ return _invalid_parameter("The NextToken that you provided is invalid.")++ jobs = list(_jobs.values())+ if name:+ # AWS documents this only as "filters the list of jobs by name". A+ # prefix match is what the console's search box produces and what the+ # equivalent Comprehend filter does.+ needle = str(name).lower()+ jobs = [j for j in jobs if (j.get("JobName") or "").lower().startswith(needle)]+ if status:+ jobs = [j for j in jobs if j.get("JobStatus") == status]+ if before is not None:+ jobs = [j for j in jobs if (j.get("SubmittedTime") or 0.0) < float(before)]+ if after is not None:+ jobs = [j for j in jobs if (j.get("SubmittedTime") or 0.0) > float(after)]++ # Most recently submitted first, matching the console and the API.+ jobs.sort(key=_sort_key, reverse=True)+ if cursor is not None:+ jobs = [j for j in jobs if _sort_key(j) < cursor]++ page = jobs[:max_results]+ result = {"TextTranslationJobPropertiesList": [_public_job(j) for j in page]}+ if len(jobs) > max_results:+ result["NextToken"] = _encode_token(page[-1])+ return json_response(result)+++def _stop_text_translation_job(data):+ job_id = data.get("JobId")+ if not isinstance(job_id, str) or not job_id.strip():+ return _invalid_parameter(+ "1 validation error detected: Value null at 'jobId' failed to satisfy "+ "constraint: Member must not be null"+ )++ job = _jobs.get(job_id)+ if job is None:+ return _not_found(+ "The batch translation job that you requested could not be found. Check the job ID "+ "and try your request again."+ )++ # A job that already finished is reported as it stands: AWS documents that+ # a job which completes before it can be stopped stays COMPLETED.+ if job["JobStatus"] in _TERMINAL_STATUSES or job["JobStatus"] == "STOP_REQUESTED":+ return json_response({"JobId": job_id, "JobStatus": job["JobStatus"]})++ job["JobStatus"] = "STOP_REQUESTED"+ return json_response({"JobId": job_id, "JobStatus": job["JobStatus"]})+++_DISPATCH = {+ "StartTextTranslationJob": _start_text_translation_job,+ "DescribeTextTranslationJob": _describe_text_translation_job,+ "ListTextTranslationJobs": _list_text_translation_jobs,+ "StopTextTranslationJob": _stop_text_translation_job,+}+++async def handle_request(method, path, headers, body, query_params):+ target = headers.get("X-Amz-Target") or headers.get("x-amz-target") or ""+ action = target.split(".", 1)[1] if "." in target else target+ if not action:+ return error_response_json("InvalidAction", "missing X-Amz-Target", 400)++ body_text = body.decode("utf-8") if isinstance(body, bytes) else (body or "")+ try:+ payload = json.loads(body_text) if body_text else {}+ except json.JSONDecodeError:+ return error_response_json("SerializationException", "invalid JSON body", 400)+ if not isinstance(payload, dict):+ return error_response_json("SerializationException", "invalid JSON body", 400)++ fn = _DISPATCH.get(action)+ if fn is None:+ return error_response_json(+ "InvalidAction",+ f"Operation '{action}' not implemented",+ 400,+ )++ logger.debug("Translate: %s %s", action, body_text)+ return fn(payload)diff --git a/tests/conftest.py b/tests/conftest.pyindex acaecf82..f26c53a6 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -733,6 +733,11 @@ def transcribe():return make_client("transcribe")[email protected](scope="session")+def translate():+ return make_client("translate")++class FakeDockerContainer:"""Container double for tests that observe lifecycle without a daemon."""diff --git a/tests/test_lambda.py b/tests/test_lambda.pyindex 51408fc5..e99d6183 100644--- a/tests/test_lambda.py+++ b/tests/test_lambda.py@@ -7646,6 +7646,7 @@ const { OrganizationsClient, ListAccountsCommand } = require("@aws-sdk/client-orconst { CodeBuildClient, ListProjectsCommand } = require("@aws-sdk/client-codebuild");const { CloudTrailClient, LookupEventsCommand } = require("@aws-sdk/client-cloudtrail");const { ServiceDiscoveryClient, ListServicesCommand } = require("@aws-sdk/client-servicediscovery");+const { TranslateClient, StartTextTranslationJobCommand } = require("@aws-sdk/client-translate");exports.handler = async () => ({sqs: typeof SQSClient === "function" && typeof SendMessageCommand === "function",kms: typeof KMSClient === "function" && typeof EncryptCommand === "function",@@ -7660,6 +7661,7 @@ exports.handler = async () => ({cb: typeof CodeBuildClient === "function" && typeof ListProjectsCommand === "function",ct: typeof CloudTrailClient === "function" && typeof LookupEventsCommand === "function",sd: typeof ServiceDiscoveryClient === "function" && typeof ListServicesCommand === "function",+ tr: typeof TranslateClient === "function" && typeof StartTextTranslationJobCommand === "function",});"""result = _run_nodejs_worker(handler_js)@@ -7669,6 +7671,47 @@ exports.handler = async () => ({assert ok is True, f"Stub not resolved for service key: {svc!r}"+def test_nodejs_worker_translate_stub_sends_shine_target():+ """The Translate stub addresses the service by its real target prefix.++ Translate's awsJson1.1 target prefix is AWSShineFrontendService_20170701,+ which is what router.py matches on, so a handler that never bundles+ @aws-sdk/client-translate still reaches the service module.+ """+ handler_js = """\+const http = require("http");+const srv = http.createServer((req, res) => {+ const target = req.headers["x-amz-target"];+ let body = "";+ req.on("data", (c) => { body += c; });+ req.on("end", () => {+ res.writeHead(200, { "Content-Type": "application/x-amz-json-1.1" });+ res.end(JSON.stringify({ JobId: "abc", JobStatus: "SUBMITTED", _target: target, _body: body }));+ });+});+srv.listen(0, "127.0.0.1", () => {+ process.env.AWS_ENDPOINT_URL = "http://127.0.0.1:" + srv.address().port;+ const { TranslateClient, StartTextTranslationJobCommand } = require("@aws-sdk/client-translate");+ new TranslateClient({})+ .send(new StartTextTranslationJobCommand({ JobName: "job" }))+ .then((out) => { srv.close(); exports._result = out; })+ .catch((e) => { srv.close(); exports._result = { error: String(e) }; });+});+exports.handler = () => new Promise((res) => {+ const wait = () => exports._result ? res(exports._result) : setTimeout(wait, 10);+ wait();+});+"""+ result = _run_nodejs_worker(handler_js)+ assert result.get("status") == "ok", f"Invocation failed: {result}"+ r = result["result"]+ assert r.get("_target") == "AWSShineFrontendService_20170701.StartTextTranslationJob", (+ f"X-Amz-Target was {r.get('_target')!r}"+ )+ assert json.loads(r["_body"]) == {"JobName": "job"}+ assert r["JobId"] == "abc"++def test_nodejs_worker_https_localhost_downgraded_to_http():"""https.request to localhost is downgraded to HTTP so cfn-response.js works.diff --git a/tests/test_persistence.py b/tests/test_persistence.pyindex 30aa67f0..41a25a12 100644--- a/tests/test_persistence.py+++ b/tests/test_persistence.py@@ -756,6 +756,120 @@ def test_transcribe_round_trip_fails_a_job_left_mid_flight():_round_trip("transcribe", "transcribe", populate, observe)+def test_translate_round_trip():+ import time as _time++ def populate(mod):+ mod._jobs["1c1838f470806ab9c3e0057f14717bed"] = {+ "JobId": "1c1838f470806ab9c3e0057f14717bed",+ "JobName": "nightly-transcript-translation",+ "JobStatus": "COMPLETED",+ "JobDetails": {+ "TranslatedDocumentsCount": 2,+ "DocumentsWithErrorsCount": 0,+ "InputDocumentsCount": 2,+ },+ "SourceLanguageCode": "en",+ "TargetLanguageCodes": ["fr"],+ "SubmittedTime": _time.time(),+ "EndTime": _time.time(),+ "InputDataConfig": {+ "S3Uri": "s3://corpus/input/",+ "ContentType": "application/x-xliff+xml",+ },+ "OutputDataConfig": {+ "S3Uri": (+ "s3://corpus/output/000000000000-TranslateText-"+ "1c1838f470806ab9c3e0057f14717bed/"+ )+ },+ "DataAccessRoleArn": "arn:aws:iam::000000000000:role/TranslateBatchRole",+ "_run_id": "run-1",+ "_client_token": "token-1",+ "_output_bucket": "corpus",+ "_output_prefix": "output/000000000000-TranslateText-1c1838f470806ab9c3e0057f14717bed/",+ }+ mod._client_tokens["token-1"] = "1c1838f470806ab9c3e0057f14717bed"++ def observe(mod):+ job = mod._jobs.get("1c1838f470806ab9c3e0057f14717bed")+ assert job is not None+ assert job["JobStatus"] == "COMPLETED"+ # The rewritten output location is the only pointer a caller has to the+ # translated documents; losing it leaves Describe reporting COMPLETED+ # with nothing to fetch.+ assert job["OutputDataConfig"]["S3Uri"].endswith(+ "000000000000-TranslateText-1c1838f470806ab9c3e0057f14717bed/"+ )+ assert job["JobDetails"]["TranslatedDocumentsCount"] == 2+ # The idempotency map has to survive too, or a client retrying with the+ # same token after a restart starts a duplicate job.+ assert mod._client_tokens.get("token-1") == "1c1838f470806ab9c3e0057f14717bed"++ _round_trip("translate", "translate", populate, observe)+++def test_translate_jobs_round_trip_outside_boot_region():+ """Translate jobs are region-scoped. `AccountRegionScopedDict.__bool__` is+ scope-relative, so a snapshot holding jobs only in regions other than the+ one restore runs in must not be treated as empty."""+ import time as _time++ from ministack.core.responses import request_scope++ def populate(mod):+ with request_scope("000000000000", "eu-west-1"):+ mod._jobs["eu-job"] = {+ "JobId": "eu-job",+ "JobStatus": "COMPLETED",+ "SourceLanguageCode": "en",+ "TargetLanguageCodes": ["de"],+ "SubmittedTime": _time.time(),+ "EndTime": _time.time(),+ "InputDataConfig": {"S3Uri": "s3://b/in/", "ContentType": "text/plain"},+ "OutputDataConfig": {"S3Uri": "s3://b/out/"},+ "_output_bucket": "b",+ "_output_prefix": "out/",+ }++ def observe(mod):+ with request_scope("000000000000", "eu-west-1"):+ job = mod._jobs.get("eu-job")+ assert job is not None, "a job outside the restoring region was dropped"+ assert job["JobStatus"] == "COMPLETED"++ _round_trip("translate", "translate", populate, observe)+++def test_translate_round_trip_fails_a_job_left_mid_flight():+ """A restart leaves no worker behind, so a job restored as SUBMITTED,+ IN_PROGRESS or STOP_REQUESTED would strand every caller polling+ DescribeTextTranslationJob."""+ import time as _time++ def populate(mod):+ mod._jobs["stuck-job"] = {+ "JobId": "stuck-job",+ "JobStatus": "IN_PROGRESS",+ "SourceLanguageCode": "en",+ "TargetLanguageCodes": ["fr"],+ "SubmittedTime": _time.time(),+ "InputDataConfig": {"S3Uri": "s3://b/in/", "ContentType": "text/plain"},+ "OutputDataConfig": {"S3Uri": "s3://b/out/"},+ "_output_bucket": "b",+ "_output_prefix": "out/",+ }++ def observe(mod):+ job = mod._jobs.get("stuck-job")+ assert job is not None+ assert job["JobStatus"] == "FAILED"+ assert job["EndTime"] is not None+ assert "restart" in job["Message"]++ _round_trip("translate", "translate", populate, observe)++def test_cloudformation_round_trip():"""CloudFormation stack metadata (stacks, events, exports, change sets)survives a PERSIST_STATE stop/restore cycle — otherwise ListStacks /diff --git a/tests/test_translate.py b/tests/test_translate.pynew file mode 100644index 00000000..d3437b2a--- /dev/null+++ b/tests/test_translate.py@@ -0,0 +1,679 @@+import json+import os+import time+import uuid++import pytest+from botocore.exceptions import ClientError++_ENDPOINT = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")++_ROLE_ARN = "arn:aws:iam::000000000000:role/TranslateBatchRole"++# Mirrors the server's own default for TRANSLATE_JOB_RUN_SECONDS, so a test that+# pins the pace can hand the setting back afterwards.+_RUN_SECONDS = float(os.environ.get("TRANSLATE_JOB_RUN_SECONDS", "2"))++_XLIFF = """<?xml version="1.0" encoding="UTF-8"?>+<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">+ <file original="transcript" source-language="en" datatype="plaintext">+ <body>+ <trans-unit id="seg-1"><source>Good morning everyone</source></trans-unit>+ <trans-unit id="seg-2"><source>Thanks for joining the call</source></trans-unit>+ </body>+ </file>+</xliff>+"""+++def _unique(prefix):+ return f"{prefix}-{uuid.uuid4().hex[:8]}"+++def _wait_for_status(+ translate,+ job_id,+ statuses=("COMPLETED", "COMPLETED_WITH_ERROR", "FAILED", "STOPPED"),+ timeout=30,+):+ deadline = time.time() + timeout+ job = None+ while time.time() < deadline:+ job = translate.describe_text_translation_job(JobId=job_id)[+ "TextTranslationJobProperties"+ ]+ if job["JobStatus"] in statuses:+ return job+ time.sleep(0.2)+ raise AssertionError(+ f"job {job_id} never reached {statuses}; last status "+ f"{job['JobStatus'] if job else 'unknown'}"+ )+++def _split_s3_uri(uri):+ """`s3://bucket/prefix/` -> (bucket, prefix)."""+ rest = uri[len("s3://"):]+ bucket, _, prefix = rest.partition("/")+ return bucket, prefix+++def _start(translate, **overrides):+ params = {+ "JobName": _unique("job"),+ "InputDataConfig": {"S3Uri": "s3://placeholder/input/", "ContentType": "text/plain"},+ "OutputDataConfig": {"S3Uri": "s3://placeholder/output/"},+ "DataAccessRoleArn": _ROLE_ARN,+ "SourceLanguageCode": "en",+ "TargetLanguageCodes": ["fr"],+ }+ params.update(overrides)+ params.setdefault("ClientToken", uuid.uuid4().hex)+ return translate.start_text_translation_job(**params)+++def corpus(s3):+ """An input folder holding two plain-text documents, and an output folder+ in the same bucket."""+ bucket = _unique("translate-batch")+ s3.create_bucket(Bucket=bucket)+ s3.put_object(Bucket=bucket, Key="input/greeting.txt", Body=b"Good morning everyone")+ s3.put_object(Bucket=bucket, Key="input/closing.txt", Body=b"Thanks for joining")+ return bucket, f"s3://{bucket}/input/", f"s3://{bucket}/output/"+++def test_translate_job_completes_and_writes_translated_documents(translate, s3, corpus):+ """The end-to-end contract the issue asks for: a job reaches COMPLETED on+ its own and the translated documents are real objects under the location+ Describe reports, so a consumer can fetch them without special-casing."""+ bucket, input_uri, output_uri = corpus++ started = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )+ assert started["JobStatus"] == "SUBMITTED"+ job_id = started["JobId"]+ assert 1 <= len(job_id) <= 32++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "COMPLETED"+ assert job["EndTime"] >= job["SubmittedTime"]+ assert job["JobDetails"] == {+ "TranslatedDocumentsCount": 2,+ "DocumentsWithErrorsCount": 0,+ "InputDocumentsCount": 2,+ }+ # AWS reports Message on a successful job too, not only on a failed one.+ assert job["Message"] == "Your job has completed successfully."++ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ assert out_bucket == bucket+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.greeting.txt")["Body"].read()+ assert body.decode("utf-8") == "[fr] Good morning everyone"+++def test_translate_rewrites_the_output_location_the_way_aws_does(translate, corpus):+ """Output does not land at the prefix the caller supplied: AWS creates+ `<accountId>-TranslateText-<JobId>/` inside it and reports that back. A+ consumer reading the supplied prefix would find nothing, so Describe has to+ report the rewritten location from the start."""+ _, input_uri, output_uri = corpus++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]++ described = translate.describe_text_translation_job(JobId=job_id)[+ "TextTranslationJobProperties"+ ]+ reported = described["OutputDataConfig"]["S3Uri"]+ assert reported == f"{output_uri}000000000000-TranslateText-{job_id}/"+ # Reported before the job runs, not only once it finishes.+ assert described["JobStatus"] in ("SUBMITTED", "IN_PROGRESS", "COMPLETED")++ finished = _wait_for_status(translate, job_id)+ assert finished["OutputDataConfig"]["S3Uri"] == reported+++def test_translate_xliff_target_is_populated_from_source(translate, s3):+ """The pipeline in the issue writes XLIFF and merges the translated+ `<target>` back onto its timed transcript segments. Without a populated+ `<target>` per `<trans-unit>`, and `target-language` on the `<file>`, there+ is nothing to merge."""+ import xml.etree.ElementTree as ET++ bucket = _unique("translate-xliff")+ s3.create_bucket(Bucket=bucket)+ s3.put_object(Bucket=bucket, Key="in/transcript.xlf", Body=_XLIFF.encode("utf-8"))++ job_id = _start(+ translate,+ InputDataConfig={+ "S3Uri": f"s3://{bucket}/in/",+ "ContentType": "application/x-xliff+xml",+ },+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},+ TargetLanguageCodes=["es"],+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "COMPLETED"++ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}es.transcript.xlf")["Body"].read()++ ns = {"x": "urn:oasis:names:tc:xliff:document:1.2"}+ root = ET.fromstring(body)+ assert root.find("x:file", ns).get("target-language") == "es"+ targets = {+ unit.get("id"): unit.find("x:target", ns).text+ for unit in root.iter("{urn:oasis:names:tc:xliff:document:1.2}trans-unit")+ }+ assert targets == {+ "seg-1": "[es] Good morning everyone",+ "seg-2": "[es] Thanks for joining the call",+ }+ # The source segments must survive untouched, or the merge has nothing to+ # key on.+ sources = {+ unit.get("id"): unit.find("x:source", ns).text+ for unit in root.iter("{urn:oasis:names:tc:xliff:document:1.2}trans-unit")+ }+ assert sources["seg-1"] == "Good morning everyone"+++def test_translate_html_keeps_its_markup(translate, s3):+ """An HTML job must return HTML. Translating the tags along with the text+ would hand the caller back a broken document."""+ bucket = _unique("translate-html")+ s3.create_bucket(Bucket=bucket)+ s3.put_object(+ Bucket=bucket,+ Key="in/page.html",+ Body=b"<html><body><p>Hello</p><script>var x = 1;</script></body></html>",+ )++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": f"s3://{bucket}/in/", "ContentType": "text/html"},+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.page.html")["Body"].read()++ assert body.decode("utf-8") == (+ "<html><body><p>[fr] Hello</p><script>var x = 1;</script></body></html>"+ )+++def test_translate_writes_one_document_per_target_language(translate, s3, corpus):+ """`TargetLanguageCodes` takes up to ten languages and every input document+ is translated into each of them."""+ _, input_uri, output_uri = corpus++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ TargetLanguageCodes=["fr", "de"],+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ keys = {+ obj["Key"]+ for obj in s3.list_objects_v2(Bucket=out_bucket, Prefix=out_prefix).get("Contents", [])+ }+ for language in ("fr", "de"):+ for name in ("greeting.txt", "closing.txt"):+ assert f"{out_prefix}{language}.{name}" in keys++ french = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.greeting.txt")["Body"].read()+ german = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}de.greeting.txt")["Body"].read()+ assert french != german+++def test_translate_writes_the_auxiliary_details_file(translate, s3, corpus):+ """Every batch job writes `details/<code>.auxiliary-translation-details.json`+ beside its output. Tools that reconcile a job's input and output read it."""+ _, input_uri, output_uri = corpus++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ key = f"{out_prefix}details/fr.auxiliary-translation-details.json"+ details = json.loads(s3.get_object(Bucket=out_bucket, Key=key)["Body"].read())++ assert details["sourceLanguageCode"] == "en"+ assert details["targetLanguageCode"] == "fr"+ assert details["inputDataPrefix"] == input_uri+ assert details["outputDataPrefix"] == job["OutputDataConfig"]["S3Uri"]+ assert {entry["sourceFile"] for entry in details["details"]} == {+ "greeting.txt",+ "closing.txt",+ }+ assert {entry["targetFile"] for entry in details["details"]} == {+ "fr.greeting.txt",+ "fr.closing.txt",+ }+++def test_translate_is_deterministic_for_the_same_input(translate, s3, corpus):+ """There is no machine translation here, so the value of the mock rests+ entirely on being reproducible. The same document must translate identically+ on every run or integration tests asserting on output become flaky."""+ _, input_uri, output_uri = corpus+ outputs = []++ for _ in range(2):+ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]+ job = _wait_for_status(translate, job_id)+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ outputs.append(+ s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.greeting.txt")["Body"].read()+ )++ assert outputs[0] == outputs[1]+++def test_translate_nested_input_keys_keep_their_layout(translate, s3):+ """The language code is prefixed onto the file name, not onto the key, so a+ job whose input folder has subfolders writes its output into the matching+ subfolders instead of flattening them into one namespace."""+ bucket = _unique("translate-nested")+ s3.create_bucket(Bucket=bucket)+ s3.put_object(Bucket=bucket, Key="in/2026/01/notes.txt", Body=b"Nested document")+ s3.put_object(Bucket=bucket, Key="in/", Body=b"")++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": f"s3://{bucket}/in/", "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "COMPLETED"+ # The folder marker is not a document.+ assert job["JobDetails"]["InputDocumentsCount"] == 1++ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}2026/01/fr.notes.txt")["Body"].read()+ assert body.decode("utf-8") == "[fr] Nested document"+++def test_translate_unreadable_document_completes_with_error(translate, s3):+ """A document that can't be read as its declared content type is a+ per-document failure, not a job failure: AWS counts it and finishes the job+ COMPLETED_WITH_ERROR, so the documents that did translate are still there."""+ bucket = _unique("translate-mixed")+ s3.create_bucket(Bucket=bucket)+ s3.put_object(Bucket=bucket, Key="in/good.xlf", Body=_XLIFF.encode("utf-8"))+ s3.put_object(Bucket=bucket, Key="in/broken.xlf", Body=b"<xliff><unclosed>")++ job_id = _start(+ translate,+ InputDataConfig={+ "S3Uri": f"s3://{bucket}/in/",+ "ContentType": "application/x-xliff+xml",+ },+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "COMPLETED_WITH_ERROR"+ assert job["JobDetails"] == {+ "TranslatedDocumentsCount": 1,+ "DocumentsWithErrorsCount": 1,+ "InputDocumentsCount": 2,+ }+ assert "errors" in job["Message"]++ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ keys = {+ obj["Key"]+ for obj in s3.list_objects_v2(Bucket=out_bucket, Prefix=out_prefix).get("Contents", [])+ }+ assert f"{out_prefix}fr.good.xlf" in keys+ assert f"{out_prefix}fr.broken.xlf" not in keys+++def test_translate_empty_input_folder_fails_the_job_not_the_start_call(translate, s3):+ """AWS accepts the job and fails it asynchronously with a `Message`. A+ caller that expects `Start` to succeed and then polls would otherwise see an+ exception where the service gives it a failed job to inspect."""+ bucket = _unique("translate-empty")+ s3.create_bucket(Bucket=bucket)++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": f"s3://{bucket}/in/", "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "FAILED"+ assert "no documents" in job["Message"].lower()+++def test_translate_missing_output_bucket_fails_the_job(translate, s3, corpus):+ """The output bucket is only touched once the job runs, so a bucket that+ isn't there surfaces as a failed job with a reason rather than silently+ dropping every translated document."""+ _, input_uri, _ = corpus++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": f"s3://{_unique('translate-absent')}/out/"},+ )["JobId"]++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "FAILED"+ assert "output location" in job["Message"].lower()+++def test_translate_client_token_is_idempotent(translate, corpus):+ """`ClientToken` is an idempotency token. A retried start — an SDK retry, a+ replayed event — must resolve to the original job rather than starting a+ second one writing to a second output folder."""+ _, input_uri, output_uri = corpus+ token = uuid.uuid4().hex++ first = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ ClientToken=token,+ )+ second = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ ClientToken=token,+ )++ assert first["JobId"] == second["JobId"]+++def test_translate_stops_a_running_job(translate, s3, corpus):+ """StopTextTranslationJob has to actually reach the worker. A job that keeps+ running after a stop writes output the caller has already abandoned.++ The run time is pinned high for this test so the stop is guaranteed to land+ mid-flight; otherwise a fast server completes the job first and the whole+ STOP_REQUESTED -> STOPPED path goes unexercised while the test still passes.+ The finished-before-stop case is covered separately below."""+ from conftest import _ministack_config++ _, input_uri, output_uri = corpus++ _ministack_config({"translate._JOB_RUN_SECONDS": 10})+ try:+ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]++ stopped = translate.stop_text_translation_job(JobId=job_id)+ assert stopped["JobId"] == job_id+ assert stopped["JobStatus"] == "STOP_REQUESTED"++ job = _wait_for_status(translate, job_id)+ assert job["JobStatus"] == "STOPPED"+ assert job["EndTime"] >= job["SubmittedTime"]+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])+ listing = s3.list_objects_v2(Bucket=out_bucket, Prefix=out_prefix)+ assert listing.get("KeyCount", 0) == 0+ finally:+ _ministack_config({"translate._JOB_RUN_SECONDS": _RUN_SECONDS})+++def test_translate_stopping_a_finished_job_leaves_it_alone(translate, corpus):+ """AWS documents that a job which completes before it can be stopped stays+ COMPLETED. Reporting STOPPED would tell the caller its output is gone."""+ _, input_uri, output_uri = corpus++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]+ _wait_for_status(translate, job_id)++ stopped = translate.stop_text_translation_job(JobId=job_id)+ assert stopped["JobStatus"] == "COMPLETED"+++def test_translate_describe_unknown_job_is_not_found(translate):+ with pytest.raises(ClientError) as excinfo:+ translate.describe_text_translation_job(JobId="0" * 32)+ assert excinfo.value.response["Error"]["Code"] == "ResourceNotFoundException"+++def test_translate_stop_unknown_job_is_not_found(translate):+ with pytest.raises(ClientError) as excinfo:+ translate.stop_text_translation_job(JobId="0" * 32)+ assert excinfo.value.response["Error"]["Code"] == "ResourceNotFoundException"+++def test_translate_lists_jobs_newest_first_and_filters_by_name(translate, corpus):+ """The console and SDK pair `List` with `Describe`. Ordering has to be+ newest first and the name filter has to narrow, or a caller paging through+ a busy account never finds the job it just started."""+ _, input_uri, output_uri = corpus+ prefix = _unique("listcase")+ names = [f"{prefix}-a", f"{prefix}-b"]+ ids = []+ for name in names:+ ids.append(+ _start(+ translate,+ JobName=name,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]+ )++ listed = translate.list_text_translation_jobs(Filter={"JobName": prefix})[+ "TextTranslationJobPropertiesList"+ ]+ assert [job["JobId"] for job in listed] == list(reversed(ids))+ assert [job["JobName"] for job in listed] == list(reversed(names))+++def test_translate_list_pages(translate, corpus):+ """A NextToken has to resume where the page ended. An offset-based token+ repeats a job whenever another is submitted between pages."""+ _, input_uri, output_uri = corpus+ prefix = _unique("paging")+ for index in range(3):+ _start(+ translate,+ JobName=f"{prefix}-{index}",+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )++ seen = []+ token = None+ while True:+ kwargs = {"Filter": {"JobName": prefix}, "MaxResults": 2}+ if token:+ kwargs["NextToken"] = token+ page = translate.list_text_translation_jobs(**kwargs)+ seen.extend(job["JobName"] for job in page["TextTranslationJobPropertiesList"])+ token = page.get("NextToken")+ if not token:+ break++ assert seen == [f"{prefix}-{index}" for index in reversed(range(3))]+++def test_translate_list_rejects_more_than_one_filter(translate):+ """AWS allows one filter at a time. Accepting two and silently applying one+ would hand back a result set the caller did not ask for."""+ with pytest.raises(ClientError) as excinfo:+ translate.list_text_translation_jobs(+ Filter={"JobName": "anything", "JobStatus": "COMPLETED"}+ )+ assert excinfo.value.response["Error"]["Code"] == "InvalidFilterException"+++def test_translate_rejects_an_unsupported_content_type(translate, corpus):+ _, input_uri, output_uri = corpus+ with pytest.raises(ClientError) as excinfo:+ _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "application/pdf"},+ OutputDataConfig={"S3Uri": output_uri},+ )+ assert excinfo.value.response["Error"]["Code"] == "InvalidRequestException"+++def test_translate_rejects_a_same_language_pair(translate, corpus):+ """Translating a language into itself is not a translation, and AWS models a+ dedicated exception for it that callers branch on."""+ _, input_uri, output_uri = corpus+ with pytest.raises(ClientError) as excinfo:+ _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ SourceLanguageCode="en",+ TargetLanguageCodes=["en"],+ )+ assert excinfo.value.response["Error"]["Code"] == "UnsupportedLanguagePairException"+++def test_translate_rejects_a_malformed_role_arn(translate, corpus):+ _, input_uri, output_uri = corpus+ with pytest.raises(ClientError) as excinfo:+ _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ DataAccessRoleArn="arn:aws:iam::12345:role/TooShortAccount",+ )+ assert excinfo.value.response["Error"]["Code"] == "InvalidParameterValueException"+++def test_translate_rejects_a_non_s3_input_uri(translate, corpus):+ _, _, output_uri = corpus+ with pytest.raises(ClientError) as excinfo:+ _start(+ translate,+ InputDataConfig={+ "S3Uri": "https://example.com/input/",+ "ContentType": "text/plain",+ },+ OutputDataConfig={"S3Uri": output_uri},+ )+ assert excinfo.value.response["Error"]["Code"] == "InvalidParameterValueException"+++def test_translate_rejects_more_than_ten_target_languages(translate, corpus):+ _, input_uri, output_uri = corpus+ with pytest.raises(ClientError) as excinfo:+ _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ TargetLanguageCodes=[f"x{index}" for index in range(11)],+ )+ assert excinfo.value.response["Error"]["Code"] == "InvalidParameterValueException"+++def test_translate_echoes_terminology_and_settings(translate, corpus):+ """`TerminologyNames`, `ParallelDataNames` and `Settings` don't change the+ output here, but dropping them would make Describe disagree with what the+ caller submitted."""+ _, input_uri, output_uri = corpus++ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ TerminologyNames=["product-names"],+ ParallelDataNames=["prior-translations"],+ Settings={"Formality": "FORMAL", "Profanity": "MASK"},+ )["JobId"]++ job = translate.describe_text_translation_job(JobId=job_id)[+ "TextTranslationJobProperties"+ ]+ assert job["TerminologyNames"] == ["product-names"]+ assert job["ParallelDataNames"] == ["prior-translations"]+ assert job["Settings"] == {"Formality": "FORMAL", "Profanity": "MASK"}+ assert job["DataAccessRoleArn"] == _ROLE_ARN+ assert job["SourceLanguageCode"] == "en"+ assert job["TargetLanguageCodes"] == ["fr"]+++def test_translate_publishes_a_job_state_change_event(translate, s3, eb, sqs, corpus):+ """AWS emits `Translate TextTranslationJob State Change` when a batch job+ reaches a terminal state. The second Lambda in the issue's pipeline is+ triggered by that event; without it the chain stops at translation."""+ _, input_uri, output_uri = corpus++ queue_url = sqs.create_queue(QueueName=_unique("translate-events"))["QueueUrl"]+ queue_arn = sqs.get_queue_attributes(QueueUrl=queue_url, AttributeNames=["QueueArn"])[+ "Attributes"+ ]["QueueArn"]+ rule_name = _unique("translate-rule")+ eb.put_rule(+ Name=rule_name,+ EventPattern=json.dumps(+ {+ "source": ["aws.translate"],+ "detail-type": ["Translate TextTranslationJob State Change"],+ }+ ),+ )+ eb.put_targets(Rule=rule_name, Targets=[{"Id": "q", "Arn": queue_arn}])++ try:+ job_id = _start(+ translate,+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},+ OutputDataConfig={"S3Uri": output_uri},+ )["JobId"]+ _wait_for_status(translate, job_id)++ deadline = time.time() + 15+ detail = None+ while time.time() < deadline and detail is None:+ received = sqs.receive_message(+ QueueUrl=queue_url, MaxNumberOfMessages=10, WaitTimeSeconds=1+ )+ for message in received.get("Messages", []):+ envelope = json.loads(message["Body"])+ body = json.loads(envelope["detail"]) if isinstance(+ envelope.get("detail"), str+ ) else envelope.get("detail")+ if body and body.get("jobId") == job_id:+ detail = body+ break++ assert detail is not None, "no Translate state change event reached the rule target"+ assert detail["jobStatus"] == "COMPLETED"+ finally:+ eb.remove_targets(Rule=rule_name, Ids=["q"])+ eb.delete_rule(Name=rule_name)+ sqs.delete_queue(QueueUrl=queue_url)