Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

feat(translate): batch text translation jobs (#1667)

ppettitaua9770ed18ea7
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fa2d58ba..72fe4dc6 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -18,6 +18,11 @@ Versioning follows [Semantic Versioning](https://semver.org/).
### Fixed
- **CloudFormation — `Capabilities` are enforced under `AUTH=true`** — `CreateStack`, `UpdateStack` and `CreateChangeSet` accepted a template with IAM resources or a `Transform` whatever the request acknowledged, so a deploy that CloudFormation refuses with `InsufficientCapabilitiesException` went through. With `AUTH=true` they now answer that error (HTTP 400, `Requires capabilities : [CAPABILITY_IAM]`, measured) and create nothing: an IAM resource needs `CAPABILITY_IAM` or `CAPABILITY_NAMED_IAM`, one with a custom name `CAPABILITY_NAMED_IAM`, a template with a `Transform` `CAPABILITY_AUTO_EXPAND` on `CreateStack` and `UpdateStack` (not on a change set, as the API documents). Without `AUTH` nothing changes; `GetTemplateSummary` keeps reporting the same set. Contributed by @iot-rocket.
+## [Unreleased]
+
+### Added
+- **Amazon Translate (`translate`)** — new service emulator for batch text translation jobs: `StartTextTranslationJob`, `DescribeTextTranslationJob`, `ListTextTranslationJobs`, `StopTextTranslationJob`, verified against botocore `translate-2017-07-01`. Jobs walk `SUBMITTED` → `IN_PROGRESS` → `COMPLETED` (paced by `TRANSLATE_JOB_RUN_SECONDS`), read every document under `InputDataConfig.S3Uri` from MiniStack S3, and write output to the location the real service uses — `OutputDataConfig.S3Uri` is reported as `s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/` — as `<target-code>.<input name>`, with a `details/<code>.auxiliary-translation-details.json` summary per target language. `text/plain`, `text/html` and `application/x-xliff+xml` are rewritten in place (an XLIFF job gets a populated `<target>` in every `<trans-unit>` and `target-language` on every `<file>`) and the Office ZIP types are copied through; an unreadable document ends the job `COMPLETED_WITH_ERROR`, and an empty or unreachable input location fails it with a `Message`. `ClientToken` is idempotent, `StopTextTranslationJob` reaches a running worker, and terminal states publish a `Translate TextTranslationJob State Change` event on the default bus. A Node.js Lambda that leaves `@aws-sdk/client-translate` unbundled reaches the service through the local executor's stub. The translated text is deterministic canned output — there is no machine translation. The synchronous `TranslateText` / `TranslateDocument` pair, custom terminologies, parallel data, `ListLanguages` and tagging are not implemented. Contributed by @ppettitau.
+
## [1.5.9] — 2026-09-08
### Added
diff --git a/README.md b/README.md
index bf705c93..4018a27c 100644
--- a/README.md
+++ b/README.md
@@ -428,6 +428,7 @@ subnet = ec2.create_subnet(
| **Bedrock Agent Runtime** | InvokeAgent, InvokeInlineAgent, GetAgentMemory, DeleteAgentMemory, Retrieve, RetrieveAndGenerate, RetrieveAndGenerateStream, Rerank, CreateSession, GetSession, UpdateSession, DeleteSession, EndSession, ListSessions, CreateInvocation, ListInvocations, PutInvocationStep, GetInvocationStep, ListInvocationSteps, InvokeFlow, StartFlowExecution, StopFlowExecution, GetFlowExecution, ListFlowExecutions, ListFlowExecutionEvents, GetExecutionFlowSnapshot, OptimizePrompt | All 31 operations verified against botocore `bedrock-agent-runtime-2023-07-26`; eventstream responses on streaming operations; `Retrieve` searches the documents Knowledge Base ingestion stored (lexical scoring — no embeddings) and returns content, `s3Location` and score; `InvokeAgent` answers a canned reply without orchestration and emits no trace events — it never fabricates an `ORCHESTRATE` trace |
| **Bedrock AgentCore** | Control plane (`bedrock-agentcore-control`): CreateAgentRuntime, GetAgentRuntime, ListAgentRuntimes, UpdateAgentRuntime, DeleteAgentRuntime, ListAgentRuntimeVersions, CreateAgentRuntimeEndpoint, GetAgentRuntimeEndpoint, ListAgentRuntimeEndpoints, UpdateAgentRuntimeEndpoint, DeleteAgentRuntimeEndpoint. Data plane (`bedrock-agentcore`): InvokeAgentRuntime | Agent-runtime lifecycle with `CREATING`/`UPDATING`→`READY` transitions, versioned ARNs (`agent/{uuid}:{version}`, `agentEndpoint/{uuid}`); request/response shapes verified against botocore `bedrock-agentcore-control` / `bedrock-agentcore`; account- and region-scoped state; `InvokeAgentRuntime` is a deterministic echo (no model inference), matching the requested emulation scope |
| **Transcribe** | StartTranscriptionJob, GetTranscriptionJob, ListTranscriptionJobs, DeleteTranscriptionJob | Batch transcription jobs only; streaming is a separate endpoint (`transcribestreaming`). All four operations verified against botocore `transcribe-2017-10-26`. Jobs walk `QUEUED` → `IN_PROGRESS` → `COMPLETED` over `TRANSCRIBE_JOB_RUN_SECONDS` (default 2, `0` completes immediately); the media object is read from MiniStack S3 via an `s3://`, path-style or virtual-hosted HTTPS URI, and an unreadable URI fails the job with a `FailureReason` rather than erroring at `Start`, as on AWS; the transcript is written back to S3 in the real result format (`results.transcripts`, word-level `results.items`, `results.audio_segments`, plus `results.segments` and `results.speaker_labels` when the matching `Settings` are set) at `OutputBucketName`/`OutputKey`, or to an auto-created `ministack-transcribe-output` bucket. `IdentifyLanguage` and `IdentifyMultipleLanguages` resolve a language on completion, `ContentRedaction` controls whether a `redacted-` copy replaces or accompanies the transcript, and `Subtitles` writes WebVTT and SubRip files, redacted alongside the transcript when redaction is on. Reaching a terminal state publishes a `Transcribe Job State Change` event, so EventBridge rules downstream of a job fire. There is no speech recognition, so the transcript is deterministic canned text keyed on a digest of the media URI |
+| **Translate** | StartTextTranslationJob, DescribeTextTranslationJob, ListTextTranslationJobs, StopTextTranslationJob | Batch text translation jobs only; the synchronous `TranslateText` / `TranslateDocument` pair and the custom terminology / parallel data families are not implemented. All four operations verified against botocore `translate-2017-07-01`. Jobs walk `SUBMITTED` → `IN_PROGRESS` → `COMPLETED` over `TRANSLATE_JOB_RUN_SECONDS` (default 2, `0` completes immediately); every document under `InputDataConfig.S3Uri` is read from MiniStack S3 and written back to the rewritten output location AWS uses — `OutputDataConfig.S3Uri` is reported as `s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/` — as `<target-code>.<input name>`, with a `details/<code>.auxiliary-translation-details.json` summary per target language. `text/plain` is translated line by line, `text/html` keeps its markup and translates only text nodes, `application/x-xliff+xml` gets a populated `<target>` in every `<trans-unit>` plus `target-language` on every `<file>`, and the three Office ZIP types are copied through. A document that can't be read as its content type is counted in `JobDetails` and the job ends `COMPLETED_WITH_ERROR`; an empty or unreadable input location fails the job with a `Message` rather than erroring at `Start`. `ClientToken` is idempotent, `StopTextTranslationJob` reaches a running worker, and reaching a terminal state publishes a `Translate TextTranslationJob State Change` event, so EventBridge rules downstream of a job fire. There is no machine translation, so the output is deterministic canned text derived from the source |
| **MSK** | CreateCluster, ListClusters, DescribeCluster, DeleteCluster, GetBootstrapBrokers, ListNodes, CreateConfiguration, ListConfigurations, DescribeConfiguration, ListConfigurationRevisions, DescribeConfigurationRevision, BatchAssociateScramSecret, BatchDisassociateScramSecret, ListScramSecrets, TagResource, UntagResource, ListTagsForResource | Kafka control plane; `GetBootstrapBrokers` honors `MINISTACK_MSK_BOOTSTRAP` so clients route to a real broker you bring (Redpanda, Kafka, KRaft) — the Kafka wire protocol itself is not emulated |
| **WAF v2** | CreateWebACL, GetWebACL, UpdateWebACL, DeleteWebACL, ListWebACLs, AssociateWebACL, DisassociateWebACL, GetWebACLForResource, ListResourcesForWebACL, CreateIPSet, GetIPSet, UpdateIPSet, DeleteIPSet, ListIPSets, CreateRuleGroup, GetRuleGroup, UpdateRuleGroup, DeleteRuleGroup, ListRuleGroups, TagResource, UntagResource, ListTagsForResource, CheckCapacity, DescribeManagedRuleGroup | LockToken enforced on Update/Delete; resource associations tracked |
| **Step Functions** | CreateStateMachine, DeleteStateMachine, DescribeStateMachine, UpdateStateMachine, ListStateMachines, StartExecution, StartSyncExecution, StopExecution, DescribeExecution, DescribeStateMachineForExecution, ListExecutions, GetExecutionHistory, SendTaskSuccess, SendTaskFailure, SendTaskHeartbeat, CreateActivity, DeleteActivity, DescribeActivity, ListActivities, GetActivityTask, TestState, TagResource, UntagResource, ListTagsForResource | Full ASL interpreter; Retry/Catch; waitForTaskToken; Activities (worker pattern); Pass/Task/Choice/Wait/Succeed/Fail/Map/Parallel; TestState API with mock and inspectionLevel support; SFN_MOCK_CONFIG for AWS SFN Local compatible mock testing; intrinsic functions (States.StringToJson, States.JsonToString, States.JsonMerge, States.Format); nested startExecution.sync |
@@ -859,6 +860,7 @@ end-to-end without any client config.
| `MINISTACK_MSK_BOOTSTRAP_SASL_SCRAM` | _(unset)_ | SASL/SCRAM bootstrap string (`BootstrapBrokerStringSaslScram`) |
| `MINISTACK_MSK_BOOTSTRAP_SASL_IAM` | _(unset)_ | SASL/IAM bootstrap string (`BootstrapBrokerStringSaslIam`) |
| `TRANSCRIBE_JOB_RUN_SECONDS` | `2` | Wall-clock time a Transcribe job spends between `QUEUED` and `COMPLETED`. Set `0` to complete immediately, or raise it when a test needs to observe `IN_PROGRESS` |
+| `TRANSLATE_JOB_RUN_SECONDS` | `2` | Wall-clock time a Translate batch job spends between `SUBMITTED` and `COMPLETED`. Set `0` to complete immediately, or raise it when a test needs to observe `IN_PROGRESS` or stop a running job. Also settable without a restart by posting `{"translate._JOB_RUN_SECONDS": 10}` to `/_ministack/config` |
### API Gateway HTTP proxy execution model
diff --git a/ministack/app.py b/ministack/app.py
index 3f850a79..b526fc9e 100644
--- a/ministack/app.py
+++ b/ministack/app.py
@@ -398,6 +398,7 @@ SERVICE_REGISTRY = {
"sts": {"module": "sts"},
"tagging": {"module": "tagging"},
"transcribe": {"module": "transcribe"},
+ "translate": {"module": "translate"},
"transfer": {"module": "transfer"},
"waf": {"module": "waf_v1"},
"waf-regional": {"module": "waf_v1"},
@@ -494,6 +495,7 @@ _state_map = {
"bedrock_agentcore": "bedrock_agentcore",
"msk": "msk",
"transcribe": "transcribe",
+ "translate": "translate",
}
SERVICE_NAME_ALIASES = {
@@ -1222,6 +1224,7 @@ async def _handle_admin_config_request(path: str, method: str, body: bytes):
"athena.ATHENA_DATA_DIR",
"stepfunctions._sfn_mock_config",
"stepfunctions._SFN_WAIT_SCALE",
+ "translate._JOB_RUN_SECONDS",
"lambda_svc.LAMBDA_EXECUTOR",
"cloudtrail._recording_enabled",
"alb.TARGET_CONNECT_TIMEOUT",
@@ -1243,14 +1246,14 @@ async def _handle_admin_config_request(path: str, method: str, body: bytes):
mod_name, var_name = key.rsplit(".", 1)
try:
mod = __import__(f"ministack.services.{mod_name}", fromlist=[var_name])
- if key == "stepfunctions._SFN_WAIT_SCALE":
+ if key in ("stepfunctions._SFN_WAIT_SCALE", "translate._JOB_RUN_SECONDS"):
try:
float_value = float(value)
except (ValueError, TypeError):
- logger.warning("/_ministack/config: invalid SFN_WAIT_SCALE=%r", value)
+ logger.warning("/_ministack/config: invalid %s=%r", var_name, value)
continue
if not math.isfinite(float_value) or float_value < 0:
- logger.warning("/_ministack/config: invalid SFN_WAIT_SCALE=%r", value)
+ logger.warning("/_ministack/config: invalid %s=%r", var_name, value)
continue
value = float_value
elif key == "cloudtrail._recording_enabled":
diff --git a/ministack/core/iam_actions.py b/ministack/core/iam_actions.py
index 030ad868..95303340 100644
--- a/ministack/core/iam_actions.py
+++ b/ministack/core/iam_actions.py
@@ -103,6 +103,7 @@ SERVICE_TO_IAM_NAMESPACE: dict[str, str] = {
"sts": "sts",
"tagging": "tag",
"transcribe": "transcribe",
+ "translate": "translate",
"transfer": "transfer",
"waf": "waf",
"waf-regional": "waf-regional",
diff --git a/ministack/core/lambda_runtime.py b/ministack/core/lambda_runtime.py
index 015baa48..bc3e5025 100644
--- a/ministack/core/lambda_runtime.py
+++ b/ministack/core/lambda_runtime.py
@@ -578,6 +578,7 @@ fs.write = function(fd, ...args) {
"@aws-sdk/client-servicediscovery": "Route53AutoNaming_v20170314",
"@aws-sdk/client-resource-groups-tagging-api": "ResourceGroupsTaggingAPI_20170126",
"@aws-sdk/client-cloudtrail": "com.amazonaws.cloudtrail.v20131101.CloudTrail_20131101",
+ "@aws-sdk/client-translate": "AWSShineFrontendService_20170701",
};
function _jsonRpcRequest(targetPrefix, opName, params) {
diff --git a/ministack/core/router.py b/ministack/core/router.py
index e51a8c09..9f8d006c 100644
--- a/ministack/core/router.py
+++ b/ministack/core/router.py
@@ -151,6 +151,10 @@ SERVICE_PATTERNS = {
"target_prefixes": ["Transcribe."],
"host_patterns": [r"transcribe\."],
},
+ "translate": {
+ "target_prefixes": ["AWSShineFrontendService_"],
+ "host_patterns": [r"translate\."],
+ },
"airflow": {
"host_patterns": [r"airflow\."],
"path_patterns": [
diff --git a/ministack/services/translate.py b/ministack/services/translate.py
new file mode 100644
index 00000000..e895448b
--- /dev/null
+++ b/ministack/services/translate.py
@@ -0,0 +1,1052 @@
+# Copyright (c) 2026 MiniStack Contributors. SPDX-License-Identifier: MIT
+# Copies or substantial portions, including AI-assisted ports or rewrites, must retain this notice (see LICENSE).
+"""
+Amazon Translate emulator.
+
+JSON 1.1 protocol with X-Amz-Target prefix ``AWSShineFrontendService_20170701``
+(``translate-2017-07-01``).
+
+Implemented:
+ StartTextTranslationJob, DescribeTextTranslationJob, ListTextTranslationJobs,
+ StopTextTranslationJob.
+
+Batch jobs run as a background task that walks SUBMITTED -> IN_PROGRESS ->
+COMPLETED over ``TRANSLATE_JOB_RUN_SECONDS`` (0 completes immediately). The
+worker reads every document under ``InputDataConfig.S3Uri`` out of MiniStack's
+S3 store, writes one translated document per input per target language, and
+publishes a ``Translate TextTranslationJob State Change`` event so EventBridge
+rules downstream of a batch job fire the way they do on AWS.
+
+Output lands where the real service puts it rather than where the caller asked:
+``OutputDataConfig.S3Uri`` is rewritten to
+``s3://<bucket>/<prefix><account-id>-TranslateText-<JobId>/`` and reported that
+way from ``DescribeTextTranslationJob``, which is the location a consumer has
+to read. Translated documents keep the input name with the target language code
+prefixed onto it (``fr.transcript.xlf``), and each target language gets a
+``details/<code>.auxiliary-translation-details.json`` summary alongside them, as
+on AWS.
+
+There is no machine translation here. The translated text is deterministic
+canned output derived from the source text, following the same approach as
+``bedrock_runtime._mock_reply`` and ``transcribe._mock_transcript``: identical
+input gives identical output, and different input gives distinguishable output.
+The document is rewritten in place rather than replaced, so the structure a
+consumer parses is real:
+
+ * ``text/plain`` is translated line by line, so line counts survive.
+ * ``text/html`` keeps its markup and translates only the text nodes.
+ * ``application/x-xliff+xml`` gets a populated ``<target>`` in every
+ ``<trans-unit>`` and ``target-language`` on every ``<file>``, which is what
+ an XLIFF round trip merges back onto its source segments.
+ * The Office content types are ZIP containers, so they are copied through
+ byte for byte and counted as translated rather than corrupted.
+
+A document that cannot be decoded or parsed is counted in
+``JobDetails.DocumentsWithErrorsCount`` and the job ends
+``COMPLETED_WITH_ERROR``, as on AWS. An input location with no readable
+documents fails the job with a ``Message`` rather than erroring at ``Start``.
+``Message`` is filled on success as well as failure, which is what the service
+reference shows a finished job reporting.
+
+``ClientToken`` is honoured: replaying a start request with a token already
+seen returns the original job instead of starting a second one.
+
+Deferred:
+ TranslateText and TranslateDocument (the synchronous operations),
+ the custom terminology and parallel data resource families, ListLanguages
+ and TagResource / UntagResource / ListTagsForResource. ``TerminologyNames``,
+ ``ParallelDataNames``, ``Settings`` and ``OutputDataConfig.EncryptionKey``
+ round-trip on the job record without changing the output.
+"""
+
+import asyncio
+import base64
+import copy
+import json
+import logging
+import os
+import re
+import time
+import xml.etree.ElementTree as ET
+
+from defusedxml.common import DefusedXmlException
+from defusedxml.ElementTree import fromstring as _xml_fromstring
+
+from ministack.core.persistence import load_state
+from ministack.core.responses import (
+ AccountRegionScopedDict,
+ error_response_json,
+ get_account_id,
+ get_region,
+ json_response,
+ new_uuid,
+ now_iso,
+ request_scope,
+)
+
+logger = logging.getLogger("translate")
+
+# How long a job spends between SUBMITTED and COMPLETED. Same knob shape as
+# TRANSLATE_JOB_RUN_SECONDS: tests that assert on IN_PROGRESS need a non-zero
+# value, tests that just want a result set it to 0. Also settable at runtime
+# through /_ministack/config, so a test can pin the pace without a restart.
+_JOB_RUN_SECONDS = float(os.environ.get("TRANSLATE_JOB_RUN_SECONDS", "2"))
+
+# Granularity at which a running job notices StopTextTranslationJob. Without
+# slicing the wait, a stop request would sit unobserved until the next phase
+# boundary, so a job with a long run time would stay STOP_REQUESTED for as
+# long as the caller was prepared to wait.
+_STOP_POLL_SECONDS = 0.1
+
+_TERMINAL_STATUSES = ("COMPLETED", "COMPLETED_WITH_ERROR", "FAILED", "STOPPED")
+
+_JOB_STATUSES = (
+ "SUBMITTED",
+ "IN_PROGRESS",
+ "COMPLETED",
+ "COMPLETED_WITH_ERROR",
+ "FAILED",
+ "STOP_REQUESTED",
+ "STOPPED",
+)
+
+# AWS models JobName as ^([\p{L}\p{Z}\p{N}_.:/=+\-%@]*)$. Python's re has no
+# Unicode property escapes, so \w and \s stand in for \p{L}\p{N}_ and \p{Z};
+# the effect on a real job name is the same.
+_JOB_NAME_RE = re.compile(r"^[\w\s.:/=+\-%@]{1,256}$", re.UNICODE)
+_CLIENT_TOKEN_RE = re.compile(r"^[a-zA-Z0-9-]{1,64}$")
+_ROLE_ARN_RE = re.compile(r"^arn:aws(-[^:]+)?:iam::[0-9]{12}:role/.+$")
+_S3_URI_RE = re.compile(r"^s3://[a-z0-9][.\-a-z0-9]{1,61}[a-z0-9](/.*)?$")
+
+_MAX_TARGET_LANGUAGES = 10
+
+# The content types batch translation accepts. The three Office types are ZIP
+# containers rather than text, which is why they are handled separately below.
+_PLAIN_TEXT = "text/plain"
+_HTML = "text/html"
+_XLIFF = "application/x-xliff+xml"
+_DOCX = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
+_PPTX = "application/vnd.openxmlformats-officedocument.presentationml.presentation"
+_XLSX = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
+
+_OPAQUE_CONTENT_TYPES = (_DOCX, _PPTX, _XLSX)
+_SUPPORTED_CONTENT_TYPES = (_PLAIN_TEXT, _HTML, _XLIFF) + _OPAQUE_CONTENT_TYPES
+
+# What "auto" resolves to. Real Translate detects the source language per
+# document with Comprehend; there is no detector here, so the resolved code is
+# fixed and only ever surfaces in the auxiliary details file. The job record
+# keeps reporting "auto", as on AWS.
+_DEFAULT_SOURCE_LANGUAGE = "en"
+
+_XLIFF_12_NS = "urn:oasis:names:tc:xliff:document:1.2"
+
+_INPUT_UNREADABLE = (
+ "The input location that you provided can't be accessed. Make sure that the "
+ "S3 folder exists and that Amazon Translate has read permission, then try "
+ "your request again."
+)
+
+_NO_INPUT_DOCUMENTS = (
+ "No documents were found at the input location that you provided. Make sure "
+ "that the S3 folder contains at least one document and try your request again."
+)
+
+_OUTPUT_UNWRITABLE = (
+ "The output location that you provided can't be written to. Make sure that "
+ "the S3 bucket exists and that Amazon Translate has write permission, then "
+ "try your request again."
+)
+
+# Public members of the TextTranslationJobProperties shape, in the order
+# botocore models them. Internal bookkeeping lives under keys prefixed with "_"
+# and never reaches the wire.
+_JOB_MEMBERS = (
+ "JobId",
+ "JobName",
+ "JobStatus",
+ "JobDetails",
+ "SourceLanguageCode",
+ "TargetLanguageCodes",
+ "TerminologyNames",
+ "ParallelDataNames",
+ "Message",
+ "SubmittedTime",
+ "EndTime",
+ "InputDataConfig",
+ "OutputDataConfig",
+ "DataAccessRoleArn",
+ "Settings",
+)
+
+# Members echoed straight back from the request without changing behaviour.
+_ECHOED_MEMBERS = ("TerminologyNames", "ParallelDataNames", "Settings")
+
+_jobs = AccountRegionScopedDict() # job_id -> job record
+_client_tokens = AccountRegionScopedDict() # ClientToken -> job_id
+
+
+# ---------------------------------------------------------------------------
+# State
+# ---------------------------------------------------------------------------
+
+
+def reset():
+ _jobs.clear()
+ _client_tokens.clear()
+
+
+def get_state():
+ return copy.deepcopy({"jobs": _jobs, "client_tokens": _client_tokens})
+
+
+def restore_state(data):
+ if not data:
+ return
+ _jobs.clear()
+ _client_tokens.clear()
+ # Not `or {}`: AccountRegionScopedDict.__bool__ is scope-relative, and
+ # restore runs in the default account/region. A snapshot holding jobs only
+ # in other scopes would read as falsy and every one of them would be lost.
+ jobs = data.get("jobs")
+ if jobs is not None:
+ _jobs.update(jobs)
+ tokens = data.get("client_tokens")
+ if tokens is not None:
+ _client_tokens.update(tokens)
+ _fail_orphaned_jobs()
+
+
+def load_persisted_state(data):
+ restore_state(data)
+
+
+def _fail_orphaned_jobs():
+ """A job that was mid-flight when the process stopped has no worker any
+ more. Leaving it SUBMITTED, IN_PROGRESS or STOP_REQUESTED strands every
+ caller polling DescribeTextTranslationJob forever, so it is failed the way
+ AWS fails a job it cannot finish."""
+ for job in _jobs.all_values():
+ if job.get("JobStatus") not in _TERMINAL_STATUSES:
+ job["JobStatus"] = "FAILED"
+ job["Message"] = "Internal Failure. The job did not survive a MiniStack restart."
+ job["EndTime"] = time.time()
+
+
+try:
+ _restored = load_state("translate")
+ if _restored:
+ restore_state(_restored)
+except Exception:
+ logger.exception("Failed to restore persisted Translate state; continuing with fresh store")
+
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+
+def _invalid_parameter(message):
+ return error_response_json("InvalidParameterValueException", message, 400)
+
+
+def _invalid_request(message):
+ return error_response_json("InvalidRequestException", message, 400)
+
+
+def _not_found(message):
+ return error_response_json("ResourceNotFoundException", message, 400)
+
+
+def _public_job(job):
+ """Project a stored job onto the TextTranslationJobProperties wire shape,
+ dropping members AWS omits when they have no value."""
+ return {k: job[k] for k in _JOB_MEMBERS if job.get(k) is not None}
+
+
+def _parse_s3_uri(uri):
+ """Return (bucket, prefix) for an ``s3://`` folder URI, or (None, None).
+
+ Translate models both S3Uri members as ``s3://...`` only — unlike
+ Transcribe's Media, an HTTPS form is not accepted — and treats the path as
+ a folder, so a URI without a trailing slash still addresses a prefix."""
+ if not isinstance(uri, str) or not _S3_URI_RE.match(uri):
+ return None, None
+ rest = uri[len("s3://"):]
+ bucket, _, prefix = rest.partition("/")
+ if prefix and not prefix.endswith("/"):
+ prefix += "/"
+ return bucket, prefix
+
+
+def _output_folder_prefix(job_id, output_prefix):
+ """The folder real Translate creates inside the caller's output location.
+
+ Every batch job writes to ``<prefix><account-id>-TranslateText-<JobId>/``
+ rather than to the prefix itself, so two jobs pointed at one location do
+ not overwrite each other and a consumer can tell whose output it is
+ reading."""
+ return f"{output_prefix}{get_account_id()}-TranslateText-{job_id}/"
+
+
+def _sort_key(job):
+ """Submission time with the job id breaking ties, so the ordering is total
+ and stable across calls. Callers reverse it to get newest first."""
+ return (job.get("SubmittedTime") or 0.0, job.get("JobId") or "")
+
+
+def _encode_token(job):
+ """Tokens carry the sort position of the last item returned, not an index.
+ An offset would repeat an item whenever a job was created between pages."""
+ submitted, job_id = _sort_key(job)
+ raw = json.dumps({"st": submitted, "id": job_id}).encode("utf-8")
+ return base64.urlsafe_b64encode(raw).decode("ascii")
+
+
+def _decode_token(token):
+ try:
+ raw = json.loads(base64.urlsafe_b64decode(token.encode("ascii")).decode("utf-8"))
+ return (float(raw["st"]), str(raw["id"]))
+ except Exception:
+ return None
+
+
+def _emit_state_change(job_id, status):
+ """Publish `Translate TextTranslationJob State Change`, the event AWS emits
+ when a batch job reaches a terminal state. Pipelines that chain off a
+ finished translation depend on it, so a failure to deliver is logged rather
+ than swallowed silently, and never propagates into the job itself."""
+ try:
+ from ministack.services import eventbridge as _eb
+
+ _eb._dispatch_event(
+ {
+ "EventId": new_uuid(),
+ "Source": "aws.translate",
+ "DetailType": "Translate TextTranslationJob State Change",
+ # Lowercase members: the Translate event detail is camelCase
+ # with a leading lowercase letter, unlike the API shape.
+ "Detail": json.dumps({"jobId": job_id, "jobStatus": status}),
+ "EventBusName": "default",
+ "Time": now_iso(),
+ "Resources": [],
+ "Account": get_account_id(),
+ "Region": get_region(),
+ }
+ )
+ except Exception:
+ logger.exception("Translate: failed to publish state change for job %s", job_id)
+
+
+# ---------------------------------------------------------------------------
+# Mock translation
+# ---------------------------------------------------------------------------
+
+
+def _mock_translation(text, target_language_code):
+ """Deterministic canned translation.
+
+ Tagging the source with its target language keeps the output reproducible
+ (an integration test can assert on it), distinguishable per target language
+ (a two-language job writes two different documents), and obviously not a
+ real translation. Whitespace-only text is left alone so a document's
+ blank lines and indentation survive the round trip."""
+ if not text or not text.strip():
+ return text
+ leading = text[: len(text) - len(text.lstrip())]
+ trailing = text[len(text.rstrip()):]
+ return f"{leading}[{target_language_code}] {text.strip()}{trailing}"
+
+
+def _translate_plain_text(text, target_language_code):
+ """Line by line, so the line count and blank-line structure of the source
+ survive. A consumer that zips source and translated lines together still
+ lines up."""
+ return "\n".join(_mock_translation(line, target_language_code) for line in text.split("\n"))
+
+
+# Splits an HTML document into markup and the text between it. The capture
+# group keeps the tags in the result, so reassembly is a plain join. A `>` is
+# legal inside a quoted attribute value and inside a comment, so neither can
+# be allowed to terminate the match — ending a tag early would translate the
+# rest of an attribute and leave the element's real text alone.
+_HTML_SPLIT_RE = re.compile(r"(<!--.*?-->|<(?:\"[^\"]*\"|'[^']*'|[^>\"'])*>)", re.DOTALL)
+
+# Text inside these elements is code, not prose. Real Translate leaves it be,
+# and translating it would break the document.
+_HTML_OPAQUE_ELEMENTS = ("script", "style")
+
+_HTML_TAG_NAME_RE = re.compile(r"^<\s*(/?)\s*([a-zA-Z][a-zA-Z0-9]*)")
+
+
+def _translate_html(text, target_language_code):
+ """Translate the text nodes and leave the markup untouched, so the result
+ is still the document the caller uploaded."""
+ parts = _HTML_SPLIT_RE.split(text)
+ opaque_depth = 0
+ out = []
+ for part in parts:
+ if part.startswith("<"):
+ match = _HTML_TAG_NAME_RE.match(part)
+ if match and match.group(2).lower() in _HTML_OPAQUE_ELEMENTS:
+ if match.group(1):
+ opaque_depth = max(0, opaque_depth - 1)
+ elif not part.rstrip().endswith("/>"):
+ opaque_depth += 1
+ out.append(part)
+ elif opaque_depth:
+ out.append(part)
+ else:
+ out.append(_mock_translation(part, target_language_code))
+ return "".join(out)
+
+
+def _xliff_qname(namespace_uri, tag):
+ return f"{{{namespace_uri}}}{tag}" if namespace_uri else tag
+
+
+def _translate_xliff(text, target_language_code):
+ """Fill in every ``<trans-unit>``'s ``<target>`` from its ``<source>`` and
+ stamp ``target-language`` on every ``<file>``.
+
+ This is the shape an XLIFF round trip needs: the caller wrote source
+ segments, and merges the translated ``<target>`` text back onto them by
+ ``trans-unit`` id. Inline markup inside a ``<source>`` (``<g>``, ``<x/>``)
+ is flattened to its text, because there is no real translation to
+ re-distribute across it.
+
+ The document comes from the caller's S3 bucket, so it is parsed with
+ ``defusedxml`` like every other untrusted XML parse in the emulator.
+ ``ET`` is still used to build and serialise, which is safe."""
+ root = _xml_fromstring(text)
+ namespace_uri = root.tag[1:].split("}")[0] if root.tag.startswith("{") else ""
+ if namespace_uri:
+ # Otherwise ElementTree serialises the default namespace as `ns0:`,
+ # and an XLIFF tool reading the output would not recognise the
+ # elements it just round-tripped.
+ ET.register_namespace("", namespace_uri)
+
+ file_tag = _xliff_qname(namespace_uri, "file")
+ unit_tag = _xliff_qname(namespace_uri, "trans-unit")
+ source_tag = _xliff_qname(namespace_uri, "source")
+ target_tag = _xliff_qname(namespace_uri, "target")
+
+ for file_el in root.iter(file_tag):
+ file_el.set("target-language", target_language_code)
+
+ for unit in root.iter(unit_tag):
+ source = unit.find(source_tag)
+ if source is None:
+ continue
+ source_text = "".join(source.itertext())
+ target = unit.find(target_tag)
+ if target is None:
+ target = ET.Element(target_tag)
+ unit.insert(list(unit).index(source) + 1, target)
+ else:
+ for child in list(target):
+ target.remove(child)
+ target.text = _mock_translation(source_text, target_language_code)
+ target.tail = source.tail
+
+ return ET.tostring(root, encoding="unicode", xml_declaration=True)
+
+
+def _translate_document(body, content_type, target_language_code):
+ """Return the translated bytes for one document, or None if the document
+ cannot be read as its declared content type.
+
+ None is a per-document error, not a job failure: AWS counts it in
+ DocumentsWithErrorsCount and finishes the job COMPLETED_WITH_ERROR."""
+ if content_type in _OPAQUE_CONTENT_TYPES:
+ # A ZIP container. Rewriting one would mean rewriting the OOXML parts
+ # inside it; copying it through keeps the document openable, which is
+ # what a pipeline that just moves files around needs.
+ return body
+
+ try:
+ text = body.decode("utf-8")
+ except UnicodeDecodeError:
+ return None
+
+ try:
+ if content_type == _XLIFF:
+ return _translate_xliff(text, target_language_code).encode("utf-8")
+ if content_type == _HTML:
+ return _translate_html(text, target_language_code).encode("utf-8")
+ return _translate_plain_text(text, target_language_code).encode("utf-8")
+ except (ET.ParseError, DefusedXmlException):
+ # DefusedXmlException is not a ParseError subclass, so a document with a
+ # forbidden DTD or entity has to be named explicitly here or it would
+ # escape to _run_job and fail the whole job instead of just this file.
+ return None
+
+
+# ---------------------------------------------------------------------------
+# Job execution
+# ---------------------------------------------------------------------------
+
+
+def _live_job(job_id, run_id):
+ """The job this worker was started for, or None if it has been replaced.
+ Job ids are generated per start, so this only differs from a plain lookup
+ after a restore, but it keeps a restored record from being driven by a
+ worker that no longer owns it."""
+ job = _jobs.get(job_id)
+ if job is None or job.get("_run_id") != run_id:
+ return None
+ return job
+
+
+async def _wait(job_id, run_id, seconds):
+ """Sleep in slices, returning False as soon as the job disappears or a stop
+ is requested, so StopTextTranslationJob takes effect promptly instead of at
+ the next phase boundary."""
+ remaining = seconds
+ while remaining > 0:
+ await asyncio.sleep(min(_STOP_POLL_SECONDS, remaining))
+ remaining -= _STOP_POLL_SECONDS
+ job = _live_job(job_id, run_id)
+ if job is None or job.get("JobStatus") == "STOP_REQUESTED":
+ return False
+ job = _live_job(job_id, run_id)
+ return job is not None and job.get("JobStatus") != "STOP_REQUESTED"
+
+
+def _finalize_stop(job_id, run_id):
+ """Complete the transition StopTextTranslationJob started. The stop handler
+ only records the request; the worker owns the job's terminal state, so it
+ is the one that sets STOPPED and emits the event."""
+ job = _live_job(job_id, run_id)
+ if job is None or job.get("JobStatus") != "STOP_REQUESTED":
+ return
+ job["JobStatus"] = "STOPPED"
+ job["EndTime"] = time.time()
+ logger.info("Translate: job %s stopped", job_id)
+ _emit_state_change(job_id, "STOPPED")
+
+
+async def _run_job(job_id, run_id, account_id, region):
+ """Walk a job to a terminal state. Runs as a background task, so it pins
+ the account and region it was started for rather than inheriting whatever
+ request happens to be in flight."""
+ with request_scope(account_id, region):
+ try:
+ await _run_job_inner(job_id, run_id)
+ except Exception:
+ logger.exception("Translate: job %s crashed", job_id)
+ job = _live_job(job_id, run_id)
+ if job is not None and job.get("JobStatus") not in _TERMINAL_STATUSES:
+ job["JobStatus"] = "FAILED"
+ job["Message"] = "Internal Failure. Please try your request again."
+ job["EndTime"] = time.time()
+ _emit_state_change(job_id, "FAILED")
+
+
+async def _run_job_inner(job_id, run_id):
+ half = _JOB_RUN_SECONDS / 2 if _JOB_RUN_SECONDS > 0 else 0.0
+
+ if not await _wait(job_id, run_id, half):
+ _finalize_stop(job_id, run_id)
+ return
+
+ job = _live_job(job_id, run_id)
+ if job is None:
+ return
+ job["JobStatus"] = "IN_PROGRESS"
+
+ if not await _wait(job_id, run_id, half):
+ _finalize_stop(job_id, run_id)
+ return
+
+ job = _live_job(job_id, run_id)
+ if job is None:
+ return
+
+ _translate_job(job_id, run_id, job)
+
+
+def _fail(job_id, run_id, job, message):
+ if _live_job(job_id, run_id) is None:
+ return
+ job["JobStatus"] = "FAILED"
+ job["Message"] = message
+ job["EndTime"] = time.time()
+ logger.info("Translate: job %s failed: %s", job_id, message)
+ _emit_state_change(job_id, "FAILED")
+
+
+def _input_documents(bucket, prefix):
+ """Every readable object under the input prefix, oldest key first.
+
+ Folder markers (a zero-length object whose key ends in "/") are S3 console
+ artefacts, not documents, so they are skipped rather than translated into
+ an empty output file."""
+ from ministack.services import s3 as s3_svc
+
+ store = s3_svc._buckets.get(bucket)
+ if store is None:
+ return None
+
+ documents = []
+ for key in sorted(store.get("objects", {})):
+ if not key.startswith(prefix) or key.endswith("/"):
+ continue
+ relative = key[len(prefix):]
+ if not relative:
+ continue
+ body = s3_svc._get_object_data(bucket, key)
+ if body is None:
+ continue
+ documents.append((relative, body))
+ return documents
+
+
+def _translated_key(relative_key, target_language_code):
+ """``folder/mySourceText.txt`` -> ``folder/fr.mySourceText.txt``. The
+ language code goes on the file name, not the path, so a nested input layout
+ is preserved in the output."""
+ folder, separator, name = relative_key.rpartition("/")
+ return f"{folder}{separator}{target_language_code}.{name}"
+
+
+def _auxiliary_details(job, target_language_code, input_uri, output_uri, entries, characters):
+ """The ``details/<code>.auxiliary-translation-details.json`` summary real
+ Translate writes beside the output. Counts are strings there, so they are
+ strings here."""
+ source_code = job["SourceLanguageCode"]
+ if source_code == "auto":
+ source_code = _DEFAULT_SOURCE_LANGUAGE
+ return {
+ "sourceLanguageCode": source_code,
+ "targetLanguageCode": target_language_code,
+ "charactersTranslated": str(characters),
+ "documentCountWithCustomerError": str(
+ sum(1 for entry in entries if entry["targetFile"] is None)
+ ),
+ "documentCountWithServerError": "0",
+ "inputDataPrefix": input_uri,
+ "outputDataPrefix": output_uri,
+ "details": [
+ {
+ "sourceFile": entry["sourceFile"],
+ "targetFile": entry["targetFile"],
+ "auxiliaryData": {"appliedTerminologies": []},
+ }
+ for entry in entries
+ if entry["targetFile"] is not None
+ ],
+ }
+
+
+def _translate_job(job_id, run_id, job):
+ from ministack.services import s3 as s3_svc
+
+ input_uri = job["InputDataConfig"]["S3Uri"]
+ content_type = job["InputDataConfig"]["ContentType"]
+ input_bucket, input_prefix = _parse_s3_uri(input_uri)
+
+ documents = _input_documents(input_bucket, input_prefix)
+ if documents is None:
+ _fail(job_id, run_id, job, _INPUT_UNREADABLE)
+ return
+ if not documents:
+ _fail(job_id, run_id, job, _NO_INPUT_DOCUMENTS)
+ return
+
+ output_bucket = job["_output_bucket"]
+ output_prefix = job["_output_prefix"]
+ if s3_svc._ensure_bucket(output_bucket) is None:
+ _fail(job_id, run_id, job, _OUTPUT_UNWRITABLE)
+ return
+
+ def _write(key, body, write_content_type):
+ response = s3_svc._put_object(
+ output_bucket,
+ key,
+ body,
+ {"content-type": write_content_type, "content-length": str(len(body))},
+ )
+ return not (isinstance(response, tuple) and response[0] >= 300)
+
+ # A document that fails for one target language fails for all of them, so
+ # the error count is per input document rather than per output file, which
+ # is how JobDetails reports it.
+ failed_documents = set()
+
+ for target_language_code in job["TargetLanguageCodes"]:
+ entries = []
+ characters = 0
+ for relative_key, body in documents:
+ translated = _translate_document(body, content_type, target_language_code)
+ if translated is None:
+ failed_documents.add(relative_key)
+ entries.append({"sourceFile": relative_key, "targetFile": None})
+ logger.info(
+ "Translate: job %s could not read %s as %s", job_id, relative_key, content_type
+ )
+ continue
+ target_key = _translated_key(relative_key, target_language_code)
+ if not _write(f"{output_prefix}{target_key}", translated, content_type):
+ _fail(job_id, run_id, job, _OUTPUT_UNWRITABLE)
+ return
+ characters += len(body)
+ entries.append({"sourceFile": relative_key, "targetFile": target_key})
+
+ output_uri = f"s3://{output_bucket}/{output_prefix}"
+ details = _auxiliary_details(
+ job, target_language_code, input_uri, output_uri, entries, characters
+ )
+ details_key = (
+ f"{output_prefix}details/{target_language_code}.auxiliary-translation-details.json"
+ )
+ if not _write(details_key, json.dumps(details).encode("utf-8"), "application/json"):
+ _fail(job_id, run_id, job, _OUTPUT_UNWRITABLE)
+ return
+
+ translated_count = len(documents) - len(failed_documents)
+ job["JobDetails"] = {
+ "TranslatedDocumentsCount": translated_count,
+ "DocumentsWithErrorsCount": len(failed_documents),
+ "InputDocumentsCount": len(documents),
+ }
+ status = "COMPLETED_WITH_ERROR" if failed_documents else "COMPLETED"
+ job["JobStatus"] = status
+ # AWS fills Message on success too, not only on failure — its documented
+ # completion text is what a caller sees on a finished job.
+ if failed_documents:
+ job["Message"] = (
+ f"Your job has completed with errors. {len(failed_documents)} of "
+ f"{len(documents)} documents could not be processed."
+ )
+ else:
+ job["Message"] = "Your job has completed successfully."
+ job["EndTime"] = time.time()
+ logger.info(
+ "Translate: job %s %s, %d of %d document(s) translated into %s under s3://%s/%s",
+ job_id,
+ status.lower(),
+ translated_count,
+ len(documents),
+ ", ".join(job["TargetLanguageCodes"]),
+ output_bucket,
+ output_prefix,
+ )
+ _emit_state_change(job_id, status)
+
+
+# ---------------------------------------------------------------------------
+# Operations
+# ---------------------------------------------------------------------------
+
+
+def _validate_start(data):
+ """Everything StartTextTranslationJob rejects before a job exists. Returns
+ an error response, or None when the request is good."""
+ name = data.get("JobName")
+ if name is not None and not _JOB_NAME_RE.match(str(name)):
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{name}' at 'jobName' failed to satisfy "
+ "constraint: Member must satisfy regular expression pattern: "
+ r"^([\p{L}\p{Z}\p{N}_.:/=+\-%@]*)$"
+ )
+
+ token = data.get("ClientToken")
+ if not isinstance(token, str) or not _CLIENT_TOKEN_RE.match(token):
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{token}' at 'clientToken' failed to satisfy "
+ "constraint: Member must satisfy regular expression pattern: ^[a-zA-Z0-9-]+$"
+ )
+
+ role_arn = data.get("DataAccessRoleArn")
+ if not isinstance(role_arn, str) or not _ROLE_ARN_RE.match(role_arn):
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{role_arn}' at 'dataAccessRoleArn' failed to "
+ "satisfy constraint: Member must satisfy regular expression pattern: "
+ "arn:aws(-[^:]+)?:iam::[0-9]{12}:role/.+"
+ )
+
+ input_config = data.get("InputDataConfig")
+ if not isinstance(input_config, dict):
+ input_config = {}
+ if _parse_s3_uri(input_config.get("S3Uri")) == (None, None):
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{input_config.get('S3Uri')}' at "
+ "'inputDataConfig.s3Uri' failed to satisfy constraint: Member must satisfy regular "
+ r"expression pattern: s3://[a-z0-9][\.\-a-z0-9]{1,61}[a-z0-9](/.*)?"
+ )
+
+ content_type = input_config.get("ContentType")
+ if content_type not in _SUPPORTED_CONTENT_TYPES:
+ return _invalid_request(
+ f"Unsupported content type: {content_type}. Amazon Translate supports the following "
+ f"content types: {', '.join(_SUPPORTED_CONTENT_TYPES)}."
+ )
+
+ output_config = data.get("OutputDataConfig")
+ if not isinstance(output_config, dict):
+ output_config = {}
+ if _parse_s3_uri(output_config.get("S3Uri")) == (None, None):
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{output_config.get('S3Uri')}' at "
+ "'outputDataConfig.s3Uri' failed to satisfy constraint: Member must satisfy regular "
+ r"expression pattern: s3://[a-z0-9][\.\-a-z0-9]{1,61}[a-z0-9](/.*)?"
+ )
+
+ source = data.get("SourceLanguageCode")
+ if not isinstance(source, str) or not 2 <= len(source) <= 5:
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{source}' at 'sourceLanguageCode' failed to "
+ "satisfy constraint: Member must have length between 2 and 5"
+ )
+
+ targets = data.get("TargetLanguageCodes")
+ if not isinstance(targets, list) or not targets:
+ return _invalid_parameter(
+ "1 validation error detected: Value at 'targetLanguageCodes' failed to satisfy "
+ "constraint: Member must have length greater than or equal to 1"
+ )
+ if len(targets) > _MAX_TARGET_LANGUAGES:
+ return _invalid_parameter(
+ f"1 validation error detected: Value at 'targetLanguageCodes' failed to satisfy "
+ f"constraint: Member must have length less than or equal to {_MAX_TARGET_LANGUAGES}"
+ )
+ for target in targets:
+ if not isinstance(target, str) or not 2 <= len(target) <= 5:
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{target}' at "
+ "'targetLanguageCodes' failed to satisfy constraint: Member must have length "
+ "between 2 and 5"
+ )
+ if source != "auto" and target == source:
+ return error_response_json(
+ "UnsupportedLanguagePairException",
+ f"Unsupported language pair: {source} to {target}.",
+ 400,
+ {"SourceLanguageCode": source, "TargetLanguageCode": target},
+ )
+ return None
+
+
+def _start_text_translation_job(data):
+ error = _validate_start(data)
+ if error is not None:
+ return error
+
+ # ClientToken is an idempotency token, so a retried start — an SDK retry, a
+ # replayed EventBridge delivery — must resolve to the job the first call
+ # created rather than a second copy of it writing to a second output folder.
+ token = data["ClientToken"]
+ existing_id = _client_tokens.get(token)
+ if existing_id is not None:
+ existing = _jobs.get(existing_id)
+ if existing is not None:
+ return json_response(
+ {"JobId": existing["JobId"], "JobStatus": existing["JobStatus"]}
+ )
+
+ job_id = new_uuid().replace("-", "")
+ output_bucket, output_prefix = _parse_s3_uri(data["OutputDataConfig"]["S3Uri"])
+ job_output_prefix = _output_folder_prefix(job_id, output_prefix)
+
+ run_id = new_uuid()
+ job = {
+ "JobId": job_id,
+ "JobName": data.get("JobName"),
+ "JobStatus": "SUBMITTED",
+ "JobDetails": None,
+ "SourceLanguageCode": data["SourceLanguageCode"],
+ "TargetLanguageCodes": list(data["TargetLanguageCodes"]),
+ "Message": None,
+ "SubmittedTime": time.time(),
+ "EndTime": None,
+ "InputDataConfig": copy.deepcopy(data["InputDataConfig"]),
+ # Reported as the rewritten location from the moment the job exists,
+ # the way AWS does: the caller's S3Uri is where the folder goes, not
+ # where the documents land, and a consumer reading the original prefix
+ # would find nothing.
+ "OutputDataConfig": dict(
+ data["OutputDataConfig"], S3Uri=f"s3://{output_bucket}/{job_output_prefix}"
+ ),
+ "DataAccessRoleArn": data["DataAccessRoleArn"],
+ "_run_id": run_id,
+ "_client_token": token,
+ "_output_bucket": output_bucket,
+ "_output_prefix": job_output_prefix,
+ }
+ for member in _ECHOED_MEMBERS:
+ value = data.get(member)
+ job[member] = copy.deepcopy(value) if value is not None else None
+
+ _jobs[job_id] = job
+ _client_tokens[token] = job_id
+
+ asyncio.create_task(_run_job(job_id, run_id, get_account_id(), get_region()))
+
+ return json_response({"JobId": job_id, "JobStatus": job["JobStatus"]})
+
+
+def _describe_text_translation_job(data):
+ job_id = data.get("JobId")
+ if not isinstance(job_id, str) or not job_id.strip():
+ return _invalid_parameter(
+ "1 validation error detected: Value null at 'jobId' failed to satisfy "
+ "constraint: Member must not be null"
+ )
+
+ job = _jobs.get(job_id)
+ if job is None:
+ return _not_found(
+ "The batch translation job that you requested could not be found. Check the job ID "
+ "and try your request again."
+ )
+
+ return json_response({"TextTranslationJobProperties": _public_job(job)})
+
+
+def _list_text_translation_jobs(data):
+ job_filter = data.get("Filter")
+ if not isinstance(job_filter, dict):
+ job_filter = {}
+ name = job_filter.get("JobName")
+ status = job_filter.get("JobStatus")
+ before = job_filter.get("SubmittedBeforeTime")
+ after = job_filter.get("SubmittedAfterTime")
+
+ if status is not None and status not in _JOB_STATUSES:
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{status}' at 'filter.jobStatus' failed to "
+ "satisfy constraint: Member must satisfy enum value set: "
+ f"[{', '.join(_JOB_STATUSES)}]"
+ )
+
+ # Coerce the submission-time bounds up front: the wire form is a number, but
+ # a caller can send anything, and filtering on an unparseable bound inside
+ # the comprehension below would surface as a 500 only once a job exists.
+ for field, value in (("submittedBeforeTime", before), ("submittedAfterTime", after)):
+ if value is None:
+ continue
+ try:
+ float(value)
+ except (TypeError, ValueError):
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{value}' at 'filter.{field}' failed to "
+ "satisfy constraint: Member must be a timestamp"
+ )
+
+ # "You can only set one filter at a time." The two submission-time bounds
+ # describe a single window, so they count as one filter between them.
+ selected = sum(
+ 1 for criterion in (name, status, before if before is not None else after)
+ if criterion is not None
+ )
+ if selected > 1:
+ return error_response_json(
+ "InvalidFilterException",
+ "You can specify only one filter at a time. Remove the extra filters and try your "
+ "request again.",
+ 400,
+ )
+
+ max_results = data.get("MaxResults")
+ if max_results is None:
+ max_results = 100
+ else:
+ try:
+ max_results = int(max_results)
+ except (TypeError, ValueError):
+ return _invalid_parameter("MaxResults must be an integer")
+ if max_results < 1 or max_results > 500:
+ return _invalid_parameter(
+ f"1 validation error detected: Value '{max_results}' at 'maxResults' failed to "
+ "satisfy constraint: Member must have value less than or equal to 500"
+ )
+
+ cursor = None
+ token = data.get("NextToken")
+ if token:
+ cursor = _decode_token(token)
+ if cursor is None:
+ return _invalid_parameter("The NextToken that you provided is invalid.")
+
+ jobs = list(_jobs.values())
+ if name:
+ # AWS documents this only as "filters the list of jobs by name". A
+ # prefix match is what the console's search box produces and what the
+ # equivalent Comprehend filter does.
+ needle = str(name).lower()
+ jobs = [j for j in jobs if (j.get("JobName") or "").lower().startswith(needle)]
+ if status:
+ jobs = [j for j in jobs if j.get("JobStatus") == status]
+ if before is not None:
+ jobs = [j for j in jobs if (j.get("SubmittedTime") or 0.0) < float(before)]
+ if after is not None:
+ jobs = [j for j in jobs if (j.get("SubmittedTime") or 0.0) > float(after)]
+
+ # Most recently submitted first, matching the console and the API.
+ jobs.sort(key=_sort_key, reverse=True)
+ if cursor is not None:
+ jobs = [j for j in jobs if _sort_key(j) < cursor]
+
+ page = jobs[:max_results]
+ result = {"TextTranslationJobPropertiesList": [_public_job(j) for j in page]}
+ if len(jobs) > max_results:
+ result["NextToken"] = _encode_token(page[-1])
+ return json_response(result)
+
+
+def _stop_text_translation_job(data):
+ job_id = data.get("JobId")
+ if not isinstance(job_id, str) or not job_id.strip():
+ return _invalid_parameter(
+ "1 validation error detected: Value null at 'jobId' failed to satisfy "
+ "constraint: Member must not be null"
+ )
+
+ job = _jobs.get(job_id)
+ if job is None:
+ return _not_found(
+ "The batch translation job that you requested could not be found. Check the job ID "
+ "and try your request again."
+ )
+
+ # A job that already finished is reported as it stands: AWS documents that
+ # a job which completes before it can be stopped stays COMPLETED.
+ if job["JobStatus"] in _TERMINAL_STATUSES or job["JobStatus"] == "STOP_REQUESTED":
+ return json_response({"JobId": job_id, "JobStatus": job["JobStatus"]})
+
+ job["JobStatus"] = "STOP_REQUESTED"
+ return json_response({"JobId": job_id, "JobStatus": job["JobStatus"]})
+
+
+_DISPATCH = {
+ "StartTextTranslationJob": _start_text_translation_job,
+ "DescribeTextTranslationJob": _describe_text_translation_job,
+ "ListTextTranslationJobs": _list_text_translation_jobs,
+ "StopTextTranslationJob": _stop_text_translation_job,
+}
+
+
+async def handle_request(method, path, headers, body, query_params):
+ target = headers.get("X-Amz-Target") or headers.get("x-amz-target") or ""
+ action = target.split(".", 1)[1] if "." in target else target
+ if not action:
+ return error_response_json("InvalidAction", "missing X-Amz-Target", 400)
+
+ body_text = body.decode("utf-8") if isinstance(body, bytes) else (body or "")
+ try:
+ payload = json.loads(body_text) if body_text else {}
+ except json.JSONDecodeError:
+ return error_response_json("SerializationException", "invalid JSON body", 400)
+ if not isinstance(payload, dict):
+ return error_response_json("SerializationException", "invalid JSON body", 400)
+
+ fn = _DISPATCH.get(action)
+ if fn is None:
+ return error_response_json(
+ "InvalidAction",
+ f"Operation '{action}' not implemented",
+ 400,
+ )
+
+ logger.debug("Translate: %s %s", action, body_text)
+ return fn(payload)
diff --git a/tests/conftest.py b/tests/conftest.py
index acaecf82..f26c53a6 100644
--- a/tests/conftest.py
+++ b/tests/conftest.py
@@ -733,6 +733,11 @@ def transcribe():
return make_client("transcribe")
[email protected](scope="session")
+def translate():
+ return make_client("translate")
+
+
class FakeDockerContainer:
"""Container double for tests that observe lifecycle without a daemon."""
diff --git a/tests/test_lambda.py b/tests/test_lambda.py
index 51408fc5..e99d6183 100644
--- a/tests/test_lambda.py
+++ b/tests/test_lambda.py
@@ -7646,6 +7646,7 @@ const { OrganizationsClient, ListAccountsCommand } = require("@aws-sdk/client-or
const { CodeBuildClient, ListProjectsCommand } = require("@aws-sdk/client-codebuild");
const { CloudTrailClient, LookupEventsCommand } = require("@aws-sdk/client-cloudtrail");
const { ServiceDiscoveryClient, ListServicesCommand } = require("@aws-sdk/client-servicediscovery");
+const { TranslateClient, StartTextTranslationJobCommand } = require("@aws-sdk/client-translate");
exports.handler = async () => ({
sqs: typeof SQSClient === "function" && typeof SendMessageCommand === "function",
kms: typeof KMSClient === "function" && typeof EncryptCommand === "function",
@@ -7660,6 +7661,7 @@ exports.handler = async () => ({
cb: typeof CodeBuildClient === "function" && typeof ListProjectsCommand === "function",
ct: typeof CloudTrailClient === "function" && typeof LookupEventsCommand === "function",
sd: typeof ServiceDiscoveryClient === "function" && typeof ListServicesCommand === "function",
+ tr: typeof TranslateClient === "function" && typeof StartTextTranslationJobCommand === "function",
});
"""
result = _run_nodejs_worker(handler_js)
@@ -7669,6 +7671,47 @@ exports.handler = async () => ({
assert ok is True, f"Stub not resolved for service key: {svc!r}"
+def test_nodejs_worker_translate_stub_sends_shine_target():
+ """The Translate stub addresses the service by its real target prefix.
+
+ Translate's awsJson1.1 target prefix is AWSShineFrontendService_20170701,
+ which is what router.py matches on, so a handler that never bundles
+ @aws-sdk/client-translate still reaches the service module.
+ """
+ handler_js = """\
+const http = require("http");
+const srv = http.createServer((req, res) => {
+ const target = req.headers["x-amz-target"];
+ let body = "";
+ req.on("data", (c) => { body += c; });
+ req.on("end", () => {
+ res.writeHead(200, { "Content-Type": "application/x-amz-json-1.1" });
+ res.end(JSON.stringify({ JobId: "abc", JobStatus: "SUBMITTED", _target: target, _body: body }));
+ });
+});
+srv.listen(0, "127.0.0.1", () => {
+ process.env.AWS_ENDPOINT_URL = "http://127.0.0.1:" + srv.address().port;
+ const { TranslateClient, StartTextTranslationJobCommand } = require("@aws-sdk/client-translate");
+ new TranslateClient({})
+ .send(new StartTextTranslationJobCommand({ JobName: "job" }))
+ .then((out) => { srv.close(); exports._result = out; })
+ .catch((e) => { srv.close(); exports._result = { error: String(e) }; });
+});
+exports.handler = () => new Promise((res) => {
+ const wait = () => exports._result ? res(exports._result) : setTimeout(wait, 10);
+ wait();
+});
+"""
+ result = _run_nodejs_worker(handler_js)
+ assert result.get("status") == "ok", f"Invocation failed: {result}"
+ r = result["result"]
+ assert r.get("_target") == "AWSShineFrontendService_20170701.StartTextTranslationJob", (
+ f"X-Amz-Target was {r.get('_target')!r}"
+ )
+ assert json.loads(r["_body"]) == {"JobName": "job"}
+ assert r["JobId"] == "abc"
+
+
def test_nodejs_worker_https_localhost_downgraded_to_http():
"""https.request to localhost is downgraded to HTTP so cfn-response.js works.
diff --git a/tests/test_persistence.py b/tests/test_persistence.py
index 30aa67f0..41a25a12 100644
--- a/tests/test_persistence.py
+++ b/tests/test_persistence.py
@@ -756,6 +756,120 @@ def test_transcribe_round_trip_fails_a_job_left_mid_flight():
_round_trip("transcribe", "transcribe", populate, observe)
+def test_translate_round_trip():
+ import time as _time
+
+ def populate(mod):
+ mod._jobs["1c1838f470806ab9c3e0057f14717bed"] = {
+ "JobId": "1c1838f470806ab9c3e0057f14717bed",
+ "JobName": "nightly-transcript-translation",
+ "JobStatus": "COMPLETED",
+ "JobDetails": {
+ "TranslatedDocumentsCount": 2,
+ "DocumentsWithErrorsCount": 0,
+ "InputDocumentsCount": 2,
+ },
+ "SourceLanguageCode": "en",
+ "TargetLanguageCodes": ["fr"],
+ "SubmittedTime": _time.time(),
+ "EndTime": _time.time(),
+ "InputDataConfig": {
+ "S3Uri": "s3://corpus/input/",
+ "ContentType": "application/x-xliff+xml",
+ },
+ "OutputDataConfig": {
+ "S3Uri": (
+ "s3://corpus/output/000000000000-TranslateText-"
+ "1c1838f470806ab9c3e0057f14717bed/"
+ )
+ },
+ "DataAccessRoleArn": "arn:aws:iam::000000000000:role/TranslateBatchRole",
+ "_run_id": "run-1",
+ "_client_token": "token-1",
+ "_output_bucket": "corpus",
+ "_output_prefix": "output/000000000000-TranslateText-1c1838f470806ab9c3e0057f14717bed/",
+ }
+ mod._client_tokens["token-1"] = "1c1838f470806ab9c3e0057f14717bed"
+
+ def observe(mod):
+ job = mod._jobs.get("1c1838f470806ab9c3e0057f14717bed")
+ assert job is not None
+ assert job["JobStatus"] == "COMPLETED"
+ # The rewritten output location is the only pointer a caller has to the
+ # translated documents; losing it leaves Describe reporting COMPLETED
+ # with nothing to fetch.
+ assert job["OutputDataConfig"]["S3Uri"].endswith(
+ "000000000000-TranslateText-1c1838f470806ab9c3e0057f14717bed/"
+ )
+ assert job["JobDetails"]["TranslatedDocumentsCount"] == 2
+ # The idempotency map has to survive too, or a client retrying with the
+ # same token after a restart starts a duplicate job.
+ assert mod._client_tokens.get("token-1") == "1c1838f470806ab9c3e0057f14717bed"
+
+ _round_trip("translate", "translate", populate, observe)
+
+
+def test_translate_jobs_round_trip_outside_boot_region():
+ """Translate jobs are region-scoped. `AccountRegionScopedDict.__bool__` is
+ scope-relative, so a snapshot holding jobs only in regions other than the
+ one restore runs in must not be treated as empty."""
+ import time as _time
+
+ from ministack.core.responses import request_scope
+
+ def populate(mod):
+ with request_scope("000000000000", "eu-west-1"):
+ mod._jobs["eu-job"] = {
+ "JobId": "eu-job",
+ "JobStatus": "COMPLETED",
+ "SourceLanguageCode": "en",
+ "TargetLanguageCodes": ["de"],
+ "SubmittedTime": _time.time(),
+ "EndTime": _time.time(),
+ "InputDataConfig": {"S3Uri": "s3://b/in/", "ContentType": "text/plain"},
+ "OutputDataConfig": {"S3Uri": "s3://b/out/"},
+ "_output_bucket": "b",
+ "_output_prefix": "out/",
+ }
+
+ def observe(mod):
+ with request_scope("000000000000", "eu-west-1"):
+ job = mod._jobs.get("eu-job")
+ assert job is not None, "a job outside the restoring region was dropped"
+ assert job["JobStatus"] == "COMPLETED"
+
+ _round_trip("translate", "translate", populate, observe)
+
+
+def test_translate_round_trip_fails_a_job_left_mid_flight():
+ """A restart leaves no worker behind, so a job restored as SUBMITTED,
+ IN_PROGRESS or STOP_REQUESTED would strand every caller polling
+ DescribeTextTranslationJob."""
+ import time as _time
+
+ def populate(mod):
+ mod._jobs["stuck-job"] = {
+ "JobId": "stuck-job",
+ "JobStatus": "IN_PROGRESS",
+ "SourceLanguageCode": "en",
+ "TargetLanguageCodes": ["fr"],
+ "SubmittedTime": _time.time(),
+ "InputDataConfig": {"S3Uri": "s3://b/in/", "ContentType": "text/plain"},
+ "OutputDataConfig": {"S3Uri": "s3://b/out/"},
+ "_output_bucket": "b",
+ "_output_prefix": "out/",
+ }
+
+ def observe(mod):
+ job = mod._jobs.get("stuck-job")
+ assert job is not None
+ assert job["JobStatus"] == "FAILED"
+ assert job["EndTime"] is not None
+ assert "restart" in job["Message"]
+
+ _round_trip("translate", "translate", populate, observe)
+
+
def test_cloudformation_round_trip():
"""CloudFormation stack metadata (stacks, events, exports, change sets)
survives a PERSIST_STATE stop/restore cycle — otherwise ListStacks /
diff --git a/tests/test_translate.py b/tests/test_translate.py
new file mode 100644
index 00000000..d3437b2a
--- /dev/null
+++ b/tests/test_translate.py
@@ -0,0 +1,679 @@
+import json
+import os
+import time
+import uuid
+
+import pytest
+from botocore.exceptions import ClientError
+
+_ENDPOINT = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")
+
+_ROLE_ARN = "arn:aws:iam::000000000000:role/TranslateBatchRole"
+
+# Mirrors the server's own default for TRANSLATE_JOB_RUN_SECONDS, so a test that
+# pins the pace can hand the setting back afterwards.
+_RUN_SECONDS = float(os.environ.get("TRANSLATE_JOB_RUN_SECONDS", "2"))
+
+_XLIFF = """<?xml version="1.0" encoding="UTF-8"?>
+<xliff version="1.2" xmlns="urn:oasis:names:tc:xliff:document:1.2">
+ <file original="transcript" source-language="en" datatype="plaintext">
+ <body>
+ <trans-unit id="seg-1"><source>Good morning everyone</source></trans-unit>
+ <trans-unit id="seg-2"><source>Thanks for joining the call</source></trans-unit>
+ </body>
+ </file>
+</xliff>
+"""
+
+
+def _unique(prefix):
+ return f"{prefix}-{uuid.uuid4().hex[:8]}"
+
+
+def _wait_for_status(
+ translate,
+ job_id,
+ statuses=("COMPLETED", "COMPLETED_WITH_ERROR", "FAILED", "STOPPED"),
+ timeout=30,
+):
+ deadline = time.time() + timeout
+ job = None
+ while time.time() < deadline:
+ job = translate.describe_text_translation_job(JobId=job_id)[
+ "TextTranslationJobProperties"
+ ]
+ if job["JobStatus"] in statuses:
+ return job
+ time.sleep(0.2)
+ raise AssertionError(
+ f"job {job_id} never reached {statuses}; last status "
+ f"{job['JobStatus'] if job else 'unknown'}"
+ )
+
+
+def _split_s3_uri(uri):
+ """`s3://bucket/prefix/` -> (bucket, prefix)."""
+ rest = uri[len("s3://"):]
+ bucket, _, prefix = rest.partition("/")
+ return bucket, prefix
+
+
+def _start(translate, **overrides):
+ params = {
+ "JobName": _unique("job"),
+ "InputDataConfig": {"S3Uri": "s3://placeholder/input/", "ContentType": "text/plain"},
+ "OutputDataConfig": {"S3Uri": "s3://placeholder/output/"},
+ "DataAccessRoleArn": _ROLE_ARN,
+ "SourceLanguageCode": "en",
+ "TargetLanguageCodes": ["fr"],
+ }
+ params.update(overrides)
+ params.setdefault("ClientToken", uuid.uuid4().hex)
+ return translate.start_text_translation_job(**params)
+
+
+def corpus(s3):
+ """An input folder holding two plain-text documents, and an output folder
+ in the same bucket."""
+ bucket = _unique("translate-batch")
+ s3.create_bucket(Bucket=bucket)
+ s3.put_object(Bucket=bucket, Key="input/greeting.txt", Body=b"Good morning everyone")
+ s3.put_object(Bucket=bucket, Key="input/closing.txt", Body=b"Thanks for joining")
+ return bucket, f"s3://{bucket}/input/", f"s3://{bucket}/output/"
+
+
+def test_translate_job_completes_and_writes_translated_documents(translate, s3, corpus):
+ """The end-to-end contract the issue asks for: a job reaches COMPLETED on
+ its own and the translated documents are real objects under the location
+ Describe reports, so a consumer can fetch them without special-casing."""
+ bucket, input_uri, output_uri = corpus
+
+ started = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )
+ assert started["JobStatus"] == "SUBMITTED"
+ job_id = started["JobId"]
+ assert 1 <= len(job_id) <= 32
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "COMPLETED"
+ assert job["EndTime"] >= job["SubmittedTime"]
+ assert job["JobDetails"] == {
+ "TranslatedDocumentsCount": 2,
+ "DocumentsWithErrorsCount": 0,
+ "InputDocumentsCount": 2,
+ }
+ # AWS reports Message on a successful job too, not only on a failed one.
+ assert job["Message"] == "Your job has completed successfully."
+
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ assert out_bucket == bucket
+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.greeting.txt")["Body"].read()
+ assert body.decode("utf-8") == "[fr] Good morning everyone"
+
+
+def test_translate_rewrites_the_output_location_the_way_aws_does(translate, corpus):
+ """Output does not land at the prefix the caller supplied: AWS creates
+ `<accountId>-TranslateText-<JobId>/` inside it and reports that back. A
+ consumer reading the supplied prefix would find nothing, so Describe has to
+ report the rewritten location from the start."""
+ _, input_uri, output_uri = corpus
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+
+ described = translate.describe_text_translation_job(JobId=job_id)[
+ "TextTranslationJobProperties"
+ ]
+ reported = described["OutputDataConfig"]["S3Uri"]
+ assert reported == f"{output_uri}000000000000-TranslateText-{job_id}/"
+ # Reported before the job runs, not only once it finishes.
+ assert described["JobStatus"] in ("SUBMITTED", "IN_PROGRESS", "COMPLETED")
+
+ finished = _wait_for_status(translate, job_id)
+ assert finished["OutputDataConfig"]["S3Uri"] == reported
+
+
+def test_translate_xliff_target_is_populated_from_source(translate, s3):
+ """The pipeline in the issue writes XLIFF and merges the translated
+ `<target>` back onto its timed transcript segments. Without a populated
+ `<target>` per `<trans-unit>`, and `target-language` on the `<file>`, there
+ is nothing to merge."""
+ import xml.etree.ElementTree as ET
+
+ bucket = _unique("translate-xliff")
+ s3.create_bucket(Bucket=bucket)
+ s3.put_object(Bucket=bucket, Key="in/transcript.xlf", Body=_XLIFF.encode("utf-8"))
+
+ job_id = _start(
+ translate,
+ InputDataConfig={
+ "S3Uri": f"s3://{bucket}/in/",
+ "ContentType": "application/x-xliff+xml",
+ },
+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},
+ TargetLanguageCodes=["es"],
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "COMPLETED"
+
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}es.transcript.xlf")["Body"].read()
+
+ ns = {"x": "urn:oasis:names:tc:xliff:document:1.2"}
+ root = ET.fromstring(body)
+ assert root.find("x:file", ns).get("target-language") == "es"
+ targets = {
+ unit.get("id"): unit.find("x:target", ns).text
+ for unit in root.iter("{urn:oasis:names:tc:xliff:document:1.2}trans-unit")
+ }
+ assert targets == {
+ "seg-1": "[es] Good morning everyone",
+ "seg-2": "[es] Thanks for joining the call",
+ }
+ # The source segments must survive untouched, or the merge has nothing to
+ # key on.
+ sources = {
+ unit.get("id"): unit.find("x:source", ns).text
+ for unit in root.iter("{urn:oasis:names:tc:xliff:document:1.2}trans-unit")
+ }
+ assert sources["seg-1"] == "Good morning everyone"
+
+
+def test_translate_html_keeps_its_markup(translate, s3):
+ """An HTML job must return HTML. Translating the tags along with the text
+ would hand the caller back a broken document."""
+ bucket = _unique("translate-html")
+ s3.create_bucket(Bucket=bucket)
+ s3.put_object(
+ Bucket=bucket,
+ Key="in/page.html",
+ Body=b"<html><body><p>Hello</p><script>var x = 1;</script></body></html>",
+ )
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": f"s3://{bucket}/in/", "ContentType": "text/html"},
+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.page.html")["Body"].read()
+
+ assert body.decode("utf-8") == (
+ "<html><body><p>[fr] Hello</p><script>var x = 1;</script></body></html>"
+ )
+
+
+def test_translate_writes_one_document_per_target_language(translate, s3, corpus):
+ """`TargetLanguageCodes` takes up to ten languages and every input document
+ is translated into each of them."""
+ _, input_uri, output_uri = corpus
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ TargetLanguageCodes=["fr", "de"],
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ keys = {
+ obj["Key"]
+ for obj in s3.list_objects_v2(Bucket=out_bucket, Prefix=out_prefix).get("Contents", [])
+ }
+ for language in ("fr", "de"):
+ for name in ("greeting.txt", "closing.txt"):
+ assert f"{out_prefix}{language}.{name}" in keys
+
+ french = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.greeting.txt")["Body"].read()
+ german = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}de.greeting.txt")["Body"].read()
+ assert french != german
+
+
+def test_translate_writes_the_auxiliary_details_file(translate, s3, corpus):
+ """Every batch job writes `details/<code>.auxiliary-translation-details.json`
+ beside its output. Tools that reconcile a job's input and output read it."""
+ _, input_uri, output_uri = corpus
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ key = f"{out_prefix}details/fr.auxiliary-translation-details.json"
+ details = json.loads(s3.get_object(Bucket=out_bucket, Key=key)["Body"].read())
+
+ assert details["sourceLanguageCode"] == "en"
+ assert details["targetLanguageCode"] == "fr"
+ assert details["inputDataPrefix"] == input_uri
+ assert details["outputDataPrefix"] == job["OutputDataConfig"]["S3Uri"]
+ assert {entry["sourceFile"] for entry in details["details"]} == {
+ "greeting.txt",
+ "closing.txt",
+ }
+ assert {entry["targetFile"] for entry in details["details"]} == {
+ "fr.greeting.txt",
+ "fr.closing.txt",
+ }
+
+
+def test_translate_is_deterministic_for_the_same_input(translate, s3, corpus):
+ """There is no machine translation here, so the value of the mock rests
+ entirely on being reproducible. The same document must translate identically
+ on every run or integration tests asserting on output become flaky."""
+ _, input_uri, output_uri = corpus
+ outputs = []
+
+ for _ in range(2):
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+ job = _wait_for_status(translate, job_id)
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ outputs.append(
+ s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}fr.greeting.txt")["Body"].read()
+ )
+
+ assert outputs[0] == outputs[1]
+
+
+def test_translate_nested_input_keys_keep_their_layout(translate, s3):
+ """The language code is prefixed onto the file name, not onto the key, so a
+ job whose input folder has subfolders writes its output into the matching
+ subfolders instead of flattening them into one namespace."""
+ bucket = _unique("translate-nested")
+ s3.create_bucket(Bucket=bucket)
+ s3.put_object(Bucket=bucket, Key="in/2026/01/notes.txt", Body=b"Nested document")
+ s3.put_object(Bucket=bucket, Key="in/", Body=b"")
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": f"s3://{bucket}/in/", "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "COMPLETED"
+ # The folder marker is not a document.
+ assert job["JobDetails"]["InputDocumentsCount"] == 1
+
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ body = s3.get_object(Bucket=out_bucket, Key=f"{out_prefix}2026/01/fr.notes.txt")["Body"].read()
+ assert body.decode("utf-8") == "[fr] Nested document"
+
+
+def test_translate_unreadable_document_completes_with_error(translate, s3):
+ """A document that can't be read as its declared content type is a
+ per-document failure, not a job failure: AWS counts it and finishes the job
+ COMPLETED_WITH_ERROR, so the documents that did translate are still there."""
+ bucket = _unique("translate-mixed")
+ s3.create_bucket(Bucket=bucket)
+ s3.put_object(Bucket=bucket, Key="in/good.xlf", Body=_XLIFF.encode("utf-8"))
+ s3.put_object(Bucket=bucket, Key="in/broken.xlf", Body=b"<xliff><unclosed>")
+
+ job_id = _start(
+ translate,
+ InputDataConfig={
+ "S3Uri": f"s3://{bucket}/in/",
+ "ContentType": "application/x-xliff+xml",
+ },
+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "COMPLETED_WITH_ERROR"
+ assert job["JobDetails"] == {
+ "TranslatedDocumentsCount": 1,
+ "DocumentsWithErrorsCount": 1,
+ "InputDocumentsCount": 2,
+ }
+ assert "errors" in job["Message"]
+
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ keys = {
+ obj["Key"]
+ for obj in s3.list_objects_v2(Bucket=out_bucket, Prefix=out_prefix).get("Contents", [])
+ }
+ assert f"{out_prefix}fr.good.xlf" in keys
+ assert f"{out_prefix}fr.broken.xlf" not in keys
+
+
+def test_translate_empty_input_folder_fails_the_job_not_the_start_call(translate, s3):
+ """AWS accepts the job and fails it asynchronously with a `Message`. A
+ caller that expects `Start` to succeed and then polls would otherwise see an
+ exception where the service gives it a failed job to inspect."""
+ bucket = _unique("translate-empty")
+ s3.create_bucket(Bucket=bucket)
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": f"s3://{bucket}/in/", "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": f"s3://{bucket}/out/"},
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "FAILED"
+ assert "no documents" in job["Message"].lower()
+
+
+def test_translate_missing_output_bucket_fails_the_job(translate, s3, corpus):
+ """The output bucket is only touched once the job runs, so a bucket that
+ isn't there surfaces as a failed job with a reason rather than silently
+ dropping every translated document."""
+ _, input_uri, _ = corpus
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": f"s3://{_unique('translate-absent')}/out/"},
+ )["JobId"]
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "FAILED"
+ assert "output location" in job["Message"].lower()
+
+
+def test_translate_client_token_is_idempotent(translate, corpus):
+ """`ClientToken` is an idempotency token. A retried start — an SDK retry, a
+ replayed event — must resolve to the original job rather than starting a
+ second one writing to a second output folder."""
+ _, input_uri, output_uri = corpus
+ token = uuid.uuid4().hex
+
+ first = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ ClientToken=token,
+ )
+ second = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ ClientToken=token,
+ )
+
+ assert first["JobId"] == second["JobId"]
+
+
+def test_translate_stops_a_running_job(translate, s3, corpus):
+ """StopTextTranslationJob has to actually reach the worker. A job that keeps
+ running after a stop writes output the caller has already abandoned.
+
+ The run time is pinned high for this test so the stop is guaranteed to land
+ mid-flight; otherwise a fast server completes the job first and the whole
+ STOP_REQUESTED -> STOPPED path goes unexercised while the test still passes.
+ The finished-before-stop case is covered separately below."""
+ from conftest import _ministack_config
+
+ _, input_uri, output_uri = corpus
+
+ _ministack_config({"translate._JOB_RUN_SECONDS": 10})
+ try:
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+
+ stopped = translate.stop_text_translation_job(JobId=job_id)
+ assert stopped["JobId"] == job_id
+ assert stopped["JobStatus"] == "STOP_REQUESTED"
+
+ job = _wait_for_status(translate, job_id)
+ assert job["JobStatus"] == "STOPPED"
+ assert job["EndTime"] >= job["SubmittedTime"]
+ out_bucket, out_prefix = _split_s3_uri(job["OutputDataConfig"]["S3Uri"])
+ listing = s3.list_objects_v2(Bucket=out_bucket, Prefix=out_prefix)
+ assert listing.get("KeyCount", 0) == 0
+ finally:
+ _ministack_config({"translate._JOB_RUN_SECONDS": _RUN_SECONDS})
+
+
+def test_translate_stopping_a_finished_job_leaves_it_alone(translate, corpus):
+ """AWS documents that a job which completes before it can be stopped stays
+ COMPLETED. Reporting STOPPED would tell the caller its output is gone."""
+ _, input_uri, output_uri = corpus
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+ _wait_for_status(translate, job_id)
+
+ stopped = translate.stop_text_translation_job(JobId=job_id)
+ assert stopped["JobStatus"] == "COMPLETED"
+
+
+def test_translate_describe_unknown_job_is_not_found(translate):
+ with pytest.raises(ClientError) as excinfo:
+ translate.describe_text_translation_job(JobId="0" * 32)
+ assert excinfo.value.response["Error"]["Code"] == "ResourceNotFoundException"
+
+
+def test_translate_stop_unknown_job_is_not_found(translate):
+ with pytest.raises(ClientError) as excinfo:
+ translate.stop_text_translation_job(JobId="0" * 32)
+ assert excinfo.value.response["Error"]["Code"] == "ResourceNotFoundException"
+
+
+def test_translate_lists_jobs_newest_first_and_filters_by_name(translate, corpus):
+ """The console and SDK pair `List` with `Describe`. Ordering has to be
+ newest first and the name filter has to narrow, or a caller paging through
+ a busy account never finds the job it just started."""
+ _, input_uri, output_uri = corpus
+ prefix = _unique("listcase")
+ names = [f"{prefix}-a", f"{prefix}-b"]
+ ids = []
+ for name in names:
+ ids.append(
+ _start(
+ translate,
+ JobName=name,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+ )
+
+ listed = translate.list_text_translation_jobs(Filter={"JobName": prefix})[
+ "TextTranslationJobPropertiesList"
+ ]
+ assert [job["JobId"] for job in listed] == list(reversed(ids))
+ assert [job["JobName"] for job in listed] == list(reversed(names))
+
+
+def test_translate_list_pages(translate, corpus):
+ """A NextToken has to resume where the page ended. An offset-based token
+ repeats a job whenever another is submitted between pages."""
+ _, input_uri, output_uri = corpus
+ prefix = _unique("paging")
+ for index in range(3):
+ _start(
+ translate,
+ JobName=f"{prefix}-{index}",
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )
+
+ seen = []
+ token = None
+ while True:
+ kwargs = {"Filter": {"JobName": prefix}, "MaxResults": 2}
+ if token:
+ kwargs["NextToken"] = token
+ page = translate.list_text_translation_jobs(**kwargs)
+ seen.extend(job["JobName"] for job in page["TextTranslationJobPropertiesList"])
+ token = page.get("NextToken")
+ if not token:
+ break
+
+ assert seen == [f"{prefix}-{index}" for index in reversed(range(3))]
+
+
+def test_translate_list_rejects_more_than_one_filter(translate):
+ """AWS allows one filter at a time. Accepting two and silently applying one
+ would hand back a result set the caller did not ask for."""
+ with pytest.raises(ClientError) as excinfo:
+ translate.list_text_translation_jobs(
+ Filter={"JobName": "anything", "JobStatus": "COMPLETED"}
+ )
+ assert excinfo.value.response["Error"]["Code"] == "InvalidFilterException"
+
+
+def test_translate_rejects_an_unsupported_content_type(translate, corpus):
+ _, input_uri, output_uri = corpus
+ with pytest.raises(ClientError) as excinfo:
+ _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "application/pdf"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )
+ assert excinfo.value.response["Error"]["Code"] == "InvalidRequestException"
+
+
+def test_translate_rejects_a_same_language_pair(translate, corpus):
+ """Translating a language into itself is not a translation, and AWS models a
+ dedicated exception for it that callers branch on."""
+ _, input_uri, output_uri = corpus
+ with pytest.raises(ClientError) as excinfo:
+ _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ SourceLanguageCode="en",
+ TargetLanguageCodes=["en"],
+ )
+ assert excinfo.value.response["Error"]["Code"] == "UnsupportedLanguagePairException"
+
+
+def test_translate_rejects_a_malformed_role_arn(translate, corpus):
+ _, input_uri, output_uri = corpus
+ with pytest.raises(ClientError) as excinfo:
+ _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ DataAccessRoleArn="arn:aws:iam::12345:role/TooShortAccount",
+ )
+ assert excinfo.value.response["Error"]["Code"] == "InvalidParameterValueException"
+
+
+def test_translate_rejects_a_non_s3_input_uri(translate, corpus):
+ _, _, output_uri = corpus
+ with pytest.raises(ClientError) as excinfo:
+ _start(
+ translate,
+ InputDataConfig={
+ "S3Uri": "https://example.com/input/",
+ "ContentType": "text/plain",
+ },
+ OutputDataConfig={"S3Uri": output_uri},
+ )
+ assert excinfo.value.response["Error"]["Code"] == "InvalidParameterValueException"
+
+
+def test_translate_rejects_more_than_ten_target_languages(translate, corpus):
+ _, input_uri, output_uri = corpus
+ with pytest.raises(ClientError) as excinfo:
+ _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ TargetLanguageCodes=[f"x{index}" for index in range(11)],
+ )
+ assert excinfo.value.response["Error"]["Code"] == "InvalidParameterValueException"
+
+
+def test_translate_echoes_terminology_and_settings(translate, corpus):
+ """`TerminologyNames`, `ParallelDataNames` and `Settings` don't change the
+ output here, but dropping them would make Describe disagree with what the
+ caller submitted."""
+ _, input_uri, output_uri = corpus
+
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ TerminologyNames=["product-names"],
+ ParallelDataNames=["prior-translations"],
+ Settings={"Formality": "FORMAL", "Profanity": "MASK"},
+ )["JobId"]
+
+ job = translate.describe_text_translation_job(JobId=job_id)[
+ "TextTranslationJobProperties"
+ ]
+ assert job["TerminologyNames"] == ["product-names"]
+ assert job["ParallelDataNames"] == ["prior-translations"]
+ assert job["Settings"] == {"Formality": "FORMAL", "Profanity": "MASK"}
+ assert job["DataAccessRoleArn"] == _ROLE_ARN
+ assert job["SourceLanguageCode"] == "en"
+ assert job["TargetLanguageCodes"] == ["fr"]
+
+
+def test_translate_publishes_a_job_state_change_event(translate, s3, eb, sqs, corpus):
+ """AWS emits `Translate TextTranslationJob State Change` when a batch job
+ reaches a terminal state. The second Lambda in the issue's pipeline is
+ triggered by that event; without it the chain stops at translation."""
+ _, input_uri, output_uri = corpus
+
+ queue_url = sqs.create_queue(QueueName=_unique("translate-events"))["QueueUrl"]
+ queue_arn = sqs.get_queue_attributes(QueueUrl=queue_url, AttributeNames=["QueueArn"])[
+ "Attributes"
+ ]["QueueArn"]
+ rule_name = _unique("translate-rule")
+ eb.put_rule(
+ Name=rule_name,
+ EventPattern=json.dumps(
+ {
+ "source": ["aws.translate"],
+ "detail-type": ["Translate TextTranslationJob State Change"],
+ }
+ ),
+ )
+ eb.put_targets(Rule=rule_name, Targets=[{"Id": "q", "Arn": queue_arn}])
+
+ try:
+ job_id = _start(
+ translate,
+ InputDataConfig={"S3Uri": input_uri, "ContentType": "text/plain"},
+ OutputDataConfig={"S3Uri": output_uri},
+ )["JobId"]
+ _wait_for_status(translate, job_id)
+
+ deadline = time.time() + 15
+ detail = None
+ while time.time() < deadline and detail is None:
+ received = sqs.receive_message(
+ QueueUrl=queue_url, MaxNumberOfMessages=10, WaitTimeSeconds=1
+ )
+ for message in received.get("Messages", []):
+ envelope = json.loads(message["Body"])
+ body = json.loads(envelope["detail"]) if isinstance(
+ envelope.get("detail"), str
+ ) else envelope.get("detail")
+ if body and body.get("jobId") == job_id:
+ detail = body
+ break
+
+ assert detail is not None, "no Translate state change event reached the rule target"
+ assert detail["jobStatus"] == "COMPLETED"
+ finally:
+ eb.remove_targets(Rule=rule_name, Ids=["q"])
+ eb.delete_rule(Name=rule_name)
+ sqs.delete_queue(QueueUrl=queue_url)