Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
feat(s3): provision AWS::S3::MultiRegionAccessPoint and serve its alias (#1623)
mm-salesquezed7253ba1deca
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 125b2e5b..baaa393d 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]### Added+- **S3 — Multi-Region Access Points deploy and serve** — `AWS::S3::MultiRegionAccessPoint` rolled the stack back with `Unsupported resource type` and the MRAP hostname resolved nowhere, so a CDK app fronting regional buckets with one could not deploy. The type now provisions (minting the 13-character alias and remembering its member buckets) and `<alias>.mrap.accesspoint.s3-global.amazonaws.com` resolves onto the existing virtual-hosted S3 path, so path-style rewriting, IAM enforcement and the handler apply unchanged. The member carrying the request region is served, else the first. State is account-scoped, cleared by `/_ministack/reset` and persisted. SigV4A, which S3 requires for an MRAP, is not verified — consistent with the documented no-SigV4 stance: such a request has its presign parameters dropped and is served rather than refused for a signature that could never have matched, and because a SigV4A credential scope carries no region it falls back to the first member. The s3control control plane (`CreateMultiRegionAccessPoint` and the async request token, Get/List/Delete, policies) is not included. Contributed by @mm-salesqueze.- **CloudFormation — the Cognito resources update in place** — a stack update that changed an `AWS::Cognito::UserPool`, `UserPoolClient`, `IdentityPool` or `UserPoolGroup` fell through to the type's create handler, and for the three with a generated id that meant a second resource under a new id: `Ref` flipped to the new one, the old one was orphaned together with its data (the users in a pool, the role mapping of an identity pool) and nothing in the stack events said so. The four now update the existing record through the service's own path (`UpdateUserPool`, `UpdateUserPoolClient`, `UpdateIdentityPool`, `AddCustomAttributes` for a grown `Schema`; a group's `Description`, `Precedence` and `RoleArn` directly, since the service has no `UpdateGroup`), keeping the physical id and everything stored behind it, and a property the new template drops reverts to the default `CreateUserPool` / `CreateUserPoolClient` fills in, as the resource reference says Cognito does. The pool and the app client are also created through those two calls now, so the record carries every property the update can change (`DeletionProtection`, `EmailConfiguration`, `AccountRecoverySetting`, token validity, `PreventUserExistenceErrors`, …) instead of the handful the create handler copied, and a replaced client carries its whole template over. The properties the resource references mark *Replacement* replace: a `UserPoolClient` whose `UserPoolId` or `GenerateSecret` changed, and a renamed or moved group, are created before the old one is removed and `Ref` follows the new id, while a move to another pool under an unchanged `GroupName` gets CloudFormation's own refusal for a custom-named resource. `AliasAttributes`, `UsernameAttributes` and `UsernameConfiguration` are marked *No interruption* on the reference, but the first two are fixed at `CreateUserPool` and the reference itself calls the third immutable once set, so a change fails the update with a message naming the property rather than altering the record. A group declared without `Precedence` gets none, the null the reference documents as the default, instead of 0; `Fn::GetAtt` gains `UserPoolId` on the pool, `ClientId` on the client and `Name` on the identity pool. Contributed by @iot-rocket.- **CloudFormation — `EnabledMfas` applies on an `AWS::Cognito::UserPool` update** — the property is *No interruption* on the resource reference and creating a pool with `SOFTWARE_TOKEN_MFA` in it already switched the software-token block on, but an update that added or removed the entry left `GetUserPoolMfaConfig` where it was, so a CDK `mfaSecondFactor` change never reached the emulated pool. The update now goes through `SetUserPoolMfaConfig`: the entry arriving switches the block on, its removal or the property being dropped switches it off, and a pool that never listed it is left without the block. Contributed by @iot-rocket.- **CloudFormation — `AWS::SecretsManager::Secret` updates in place** — a stack update that changed a secret fell through to the create handler, which minted a second secret under a new ARN: `Ref` and `GetAtt Arn` flipped to it, the original was orphaned together with its versions, and nothing in the stack events said so. The secret now updates through `UpdateSecret` under the same ARN: a changed `SecretString` or `GenerateSecretString` publishes a new `AWSCURRENT` version and the previous one stays behind as `AWSPREVIOUS`; `Description` and `KmsKeyId` are sent only when the template declares or drops them, so a value set outside the stack on a property the template never declared is left alone; `ReplicaRegions` is applied on create and update through `ReplicateSecretToRegions`, and deleting the stack purges the replicas with the secret. `Name` requires replacement and follows the convention already in place: the renamed secret is created before the old one is deleted. Known limits: `Ref` still returns the secret name where AWS returns the ARN (set by the create handler, unchanged here; `GetAtt Arn` carries the ARN, which the reference documents as `Id`), `Type` is not stored by the Secrets Manager service and is ignored, and a region dropped from `ReplicaRegions` keeps its replica because the service has no `RemoveRegionsFromReplication`. Contributed by @iot-rocket.diff --git a/ministack/app.py b/ministack/app.pyindex 791996e8..69ed9db5 100644--- a/ministack/app.py+++ b/ministack/app.py@@ -1662,8 +1662,50 @@ async def _handle_alb_request(host: str, path: str, method: str, headers: dict,return 500, {"Content-Type": "application/json"}, json.dumps({"message": str(e)}).encode()+_MRAP_HOST_RE = re.compile(+ r"^([a-z0-9]+)\.mrap\.accesspoint\.s3-global\.amazonaws\.com$", re.IGNORECASE+)+++def _resolve_mrap_host(host: str):+ """The member bucket for a Multi-Region Access Point host, or None.++ An MRAP is addressed as `<alias>.mrap.accesspoint.s3-global.amazonaws.com`,+ which is not a bucket name and so never matches the virtual-hosted rules.+ Resolving it to a member bucket lets the whole existing S3 vhost path —+ rewrite to path-style, IAM enforcement, the handler — apply unchanged.+ """+ match = _MRAP_HOST_RE.match(host.split(":")[0].strip())+ if not match:+ return None+ try:+ return _get_module("s3").resolve_mrap_bucket(match.group(1).lower())+ except Exception:+ return None++async def _handle_s3_vhost_request(host: str, path: str, method: str, headers: dict, body: bytes, query_params: dict):"""Handle virtual-hosted S3 requests before generic routing."""+ mrap_bucket = _resolve_mrap_host(host)+ if mrap_bucket:+ # SigV4A (`AWS4-ECDSA-P256-SHA256`) is what S3 requires for an MRAP and+ # what MiniStack does not implement: verifying it means ECDSA P-256 key+ # derivation, and the emulator does not verify header-signed SigV4+ # requests either. So the presign parameters are dropped and the request+ # is served unverified rather than rejected for a signature that could+ # never have matched. A plain SigV4 presign is left alone and still+ # verifies.+ algorithm = (query_params.get("X-Amz-Algorithm") or [""])[0]+ if "ECDSA" in str(algorithm).upper():+ query_params = {+ k: v for k, v in query_params.items()+ if not k.lower().startswith("x-amz-")+ }+ mrap_path = "/" + mrap_bucket + (path if path != "/" else "/")+ return await _get_module("s3").handle_request(+ method, mrap_path, headers, body, query_params, signed_path=path+ )+bucket = _extract_s3_vhost_bucket(host)if not bucket or _S3_VHOST_EXCLUDE_RE.search(host) or bucket in _NON_S3_VHOST_NAMES:return Nonediff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.pyindex 62fcfcef..24bdadb0 100644--- a/ministack/services/cloudformation/provisioners.py+++ b/ministack/services/cloudformation/provisioners.py@@ -491,6 +491,54 @@ def _s3_apply_notification(name, notif):)+# ---------------------------------------------------------------------------+# S3 Multi-Region Access Point+# ---------------------------------------------------------------------------++def _s3_mrap_create(logical_id, props, stack_name):+ """Provision an AWS::S3::MultiRegionAccessPoint.++ On AWS this is asynchronous — `CreateMultiRegionAccessPoint` returns a+ request token you poll — but CloudFormation hides that behind the resource,+ and every attribute a template can read (`Alias`, `CreatedAt`) is known the+ moment the access point exists. So it is created synchronously here.++ `Regions` is a list of `{Bucket}`; the member names are kept so the data+ plane can resolve the alias to one of them.+ """+ name = props.get("Name") or _physical_name(stack_name, logical_id, lowercase=True, max_len=50)+ buckets = [+ r.get("Bucket") for r in (props.get("Regions") or [])+ if isinstance(r, dict) and r.get("Bucket")+ ]++ for existing in _s3._mraps.values():+ if existing.get("Name") == name:+ return existing["Alias"], {"Alias": existing["Alias"],+ "CreatedAt": existing["CreatedAt"]}++ alias = _s3.new_mrap_alias()+ created_at = now_iso()+ _s3._mraps[alias] = {+ "Name": name,+ "Alias": alias,+ "Regions": buckets,+ "CreatedAt": created_at,+ "PublicAccessBlockConfiguration": props.get("PublicAccessBlockConfiguration") or {},+ }+ logger.info("Created MultiRegionAccessPoint %s alias=%s over %d bucket(s)",+ name, alias, len(buckets))+ # The physical id is the NAME (what DeleteMultiRegionAccessPoint takes);+ # `Alias` is the attribute a distribution origin is built from.+ return name, {"Alias": alias, "CreatedAt": created_at}+++def _s3_mrap_delete(physical_id, props):+ for alias, record in list(_s3._mraps.items()):+ if record.get("Name") == physical_id:+ _s3._mraps.pop(alias, None)++def _s3_create(logical_id, props, stack_name):name = props.get("BucketName") or _physical_name(stack_name, logical_id, lowercase=True, max_len=63)_s3._buckets.setdefault(name, {@@ -7677,6 +7725,7 @@ _RESOURCE_HANDLERS = {"delete": _opensearch_domain_delete,},"AWS::S3::Bucket": {"create": _s3_create, "update": _s3_update, "delete": _s3_delete},+ "AWS::S3::MultiRegionAccessPoint": {"create": _s3_mrap_create, "delete": _s3_mrap_delete},"AWS::S3::BucketPolicy": {"create": _s3_bucket_policy_create,"update": _s3_bucket_policy_update,diff --git a/ministack/services/s3.py b/ministack/services/s3.pyindex 0ef0a44d..ed2b64dc 100644--- a/ministack/services/s3.py+++ b/ministack/services/s3.py@@ -77,6 +77,36 @@ _bucket_versioning = AccountScopedDict()_bucket_encryption = AccountScopedDict()_bucket_lifecycle = AccountScopedDict()_bucket_cors = AccountScopedDict()+# Multi-Region Access Points. Keyed by ALIAS rather than name, because the alias+# is what the data plane is addressed by: `<alias>.mrap.accesspoint.s3-global.amazonaws.com`.+_mraps = AccountScopedDict() # Alias -> {Name, Alias, Regions: [bucket, ...], CreatedAt}+++def new_mrap_alias() -> str:+ """A 13-character alias, the shape S3 mints for a Multi-Region Access Point."""+ return new_uuid().replace("-", "")[:13].lower()+++def resolve_mrap_bucket(alias: str):+ """The bucket an MRAP alias serves, or None.++ A real MRAP routes to whichever member bucket is nearest the caller. Nearest+ has no meaning in a single-process emulator, so the member whose name carries+ the request region wins and the first member is the fallback — deterministic,+ and it makes a single-region MRAP (the common case in a local stack) resolve+ to the only bucket it has.+ """+ record = _mraps.get(alias)+ if not record:+ return None+ buckets = record.get("Regions") or []+ if not buckets:+ return None+ region = get_region()+ for bucket in buckets:+ if region and region in bucket:+ return bucket+ return buckets[0]_bucket_acl = AccountScopedDict()_bucket_websites = AccountScopedDict()_bucket_logging_config = AccountScopedDict()@@ -123,6 +153,10 @@ _PERSISTED_BUCKET_DICTS = {"bucket_request_payment_config": _bucket_request_payment_config,"bucket_object_lock": _bucket_object_lock,"bucket_replication": _bucket_replication,+ # An access point outliving a restart matters more than most: its alias is+ # baked into client configuration, so member buckets coming back without the+ # alias fronting them fails where a missing bucket would not.+ "mraps": _mraps,}@@ -6528,6 +6562,7 @@ def reset():_object_retention,_object_legal_hold,_object_versions,+ _mraps,):d.clear()diff --git a/tests/test_cfn.py b/tests/test_cfn.pyindex 283009f7..3edd9507 100644--- a/tests/test_cfn.py+++ b/tests/test_cfn.py@@ -6270,6 +6270,62 @@ def test_cfn_cloudfront_distribution_supports_invalidations(cfn, cloudfront):assert stack["StackStatus"] == "DELETE_COMPLETE"++def _mrap_template(name, buckets):+ return json.dumps({+ "AWSTemplateFormatVersion": "2010-09-09",+ "Resources": {+ **{f"B{i}": {"Type": "AWS::S3::Bucket", "Properties": {"BucketName": b}}+ for i, b in enumerate(buckets)},+ "Mrap": {"Type": "AWS::S3::MultiRegionAccessPoint",+ "DependsOn": [f"B{i}" for i in range(len(buckets))],+ "Properties": {"Name": name,+ "Regions": [{"Bucket": b} for b in buckets]}},+ },+ "Outputs": {"Alias": {"Value": {"Fn::GetAtt": ["Mrap", "Alias"]}}},+ })+++def _mrap_get(alias, key, region=None):+ """GET through the MRAP hostname. The host is sent explicitly rather than+ resolved: <alias>.mrap.accesspoint.s3-global.amazonaws.com is a real public+ suffix, so letting DNS see it would leave the test dependent on egress."""+ import urllib.request+ endpoint = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")+ req = urllib.request.Request(f"{endpoint}/{key}")+ req.add_header("Host", f"{alias}.mrap.accesspoint.s3-global.amazonaws.com")+ if region:+ req.add_header("Authorization",+ "AWS4-HMAC-SHA256 "+ f"Credential=test/20260101/{region}/s3/aws4_request, "+ "SignedHeaders=host, Signature=unsigned")+ with urllib.request.urlopen(req, timeout=10) as resp:+ return resp.status, resp.read().decode()+++def test_cfn_s3_multi_region_access_point(cfn, s3):+ """AWS::S3::MultiRegionAccessPoint used to roll the stack back, so a CDK app+ fronting regional buckets with one could not deploy at all. It now provisions+ and hands back the 13-character alias — the only attribute a template reads,+ and the name the data plane is addressed by."""+ buckets = ["mrap-cfn-us-east-1-app", "mrap-cfn-eu-west-1-app"]+ cfn.create_stack(StackName="cfn-s3-mrap",+ TemplateBody=_mrap_template("cfn-mrap-app", buckets))+ stack = _wait_stack(cfn, "cfn-s3-mrap")+ assert stack["StackStatus"] == "CREATE_COMPLETE"++ alias = {o["OutputKey"]: o["OutputValue"] for o in stack["Outputs"]}["Alias"]+ assert len(alias) == 13 and alias.islower()++ cfn.delete_stack(StackName="cfn-s3-mrap")+ _wait_stack(cfn, "cfn-s3-mrap")+ # The alias stops resolving with the stack.+ import urllib.error+ with pytest.raises(urllib.error.HTTPError) as ei:+ _mrap_get(alias, "anything")+ assert ei.value.code in (403, 404)++def test_cfn_auto_named_s3_bucket_stable_across_updates(cfn, s3):"""Regression: auto-named S3 buckets (no explicit BucketName) must keepthe same physical resource ID across stack updates. Before the fix,diff --git a/tests/test_persistence.py b/tests/test_persistence.pyindex 728047aa..7d6dd024 100644--- a/tests/test_persistence.py+++ b/tests/test_persistence.py@@ -645,6 +645,26 @@ def test_backup_round_trip():_round_trip("backup", "backup", populate, observe)+def test_s3_mrap_round_trip():+ """A multi-region access point outlives a restart. Its alias is baked into+ client configuration, so member buckets coming back without the alias+ fronting them fails where a missing bucket would not."""+ def populate(mod):+ mod._mraps["ea0672af90364"] = {+ "Name": "app-release",+ "Alias": "ea0672af90364",+ "Regions": ["app-us-east-1-release", "app-eu-west-1-release"],+ "CreatedAt": "2026-01-01T00:00:00Z",+ }++ def observe(mod):+ mrap = mod._mraps["ea0672af90364"]+ assert mrap["Name"] == "app-release"+ assert mrap["Regions"] == ["app-us-east-1-release", "app-eu-west-1-release"]++ _round_trip("s3", "s3", populate, observe)++def test_transcribe_round_trip():import time as _timediff --git a/tests/test_s3.py b/tests/test_s3.pyindex c3da628f..15ccf25a 100644--- a/tests/test_s3.py+++ b/tests/test_s3.py@@ -6146,3 +6146,110 @@ def test_unicode_s3_metadata(s3):head = s3.head_object(Bucket="unicode-meta", Key="file.bin")assert unquote(head["Metadata"]["filename"]) == "résumé.pdf"assert unquote(head["Metadata"]["author"]) == "Ñoño"+++def _wait_stack(cfn, name, timeout=30):+ """Poll until the stack reaches a terminal status; a deleted stack is+ addressable only by its id, so describe-by-name failing means gone."""+ deadline = time.time() + timeout+ status = "UNKNOWN"+ while time.time() < deadline:+ try:+ stack = cfn.describe_stacks(StackName=name)["Stacks"][0]+ except ClientError as exc:+ if "does not exist" in str(exc):+ return {"StackStatus": "DELETE_COMPLETE", "StackName": name}+ raise+ status = stack["StackStatus"]+ if status.endswith("_COMPLETE") or status.endswith("_FAILED"):+ return stack+ time.sleep(0.2)+ raise AssertionError(f"stack {name} stuck in {status}")+++def _mrap_template(name, buckets):+ return json.dumps({+ "AWSTemplateFormatVersion": "2010-09-09",+ "Resources": {+ **{f"B{i}": {"Type": "AWS::S3::Bucket", "Properties": {"BucketName": b}}+ for i, b in enumerate(buckets)},+ "Mrap": {"Type": "AWS::S3::MultiRegionAccessPoint",+ "DependsOn": [f"B{i}" for i in range(len(buckets))],+ "Properties": {"Name": name,+ "Regions": [{"Bucket": b} for b in buckets]}},+ },+ "Outputs": {"Alias": {"Value": {"Fn::GetAtt": ["Mrap", "Alias"]}}},+ })+++def _mrap_get(alias, key, region=None):+ """GET through the MRAP hostname. The host is sent explicitly rather than+ resolved: <alias>.mrap.accesspoint.s3-global.amazonaws.com is a real public+ suffix, so letting DNS see it would leave the test dependent on egress."""+ import urllib.request+ endpoint = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")+ req = urllib.request.Request(f"{endpoint}/{key}")+ req.add_header("Host", f"{alias}.mrap.accesspoint.s3-global.amazonaws.com")+ if region:+ req.add_header("Authorization",+ "AWS4-HMAC-SHA256 "+ f"Credential=test/20260101/{region}/s3/aws4_request, "+ "SignedHeaders=host, Signature=unsigned")+ with urllib.request.urlopen(req, timeout=10) as resp:+ return resp.status, resp.read().decode()+++def test_s3_mrap_alias_serves_the_region_member(cfn, s3):+ """The MRAP hostname resolved nowhere — it is not a bucket name and matched+ no virtual-host rule. It now lands on the existing vhost path, and the member+ carrying the request region is served rather than whichever is listed first."""+ buckets = ["mrap-serve-us-east-1-app", "mrap-serve-eu-west-1-app"]+ cfn.create_stack(StackName="cfn-s3-mrap-serve",+ TemplateBody=_mrap_template("serve-mrap-app", buckets))+ stack = _wait_stack(cfn, "cfn-s3-mrap-serve")+ assert stack["StackStatus"] == "CREATE_COMPLETE"+ alias = {o["OutputKey"]: o["OutputValue"] for o in stack["Outputs"]}["Alias"]++ s3.put_object(Bucket=buckets[0], Key="who.txt", Body=b"US BUCKET")+ s3.put_object(Bucket=buckets[1], Key="who.txt", Body=b"EU BUCKET")++ # us-east-1 is listed first, so serving the eu-west-1 member proves the+ # member is chosen by region rather than taken off the front of the list.+ assert _mrap_get(alias, "who.txt", region="eu-west-1") == (200, "EU BUCKET")+ assert _mrap_get(alias, "who.txt", region="us-east-1") == (200, "US BUCKET")++ # A request that carries no region at all (SigV4A's credential scope has+ # none) falls back to the first member rather than failing.+ assert _mrap_get(alias, "who.txt") == (200, "US BUCKET")++ cfn.delete_stack(StackName="cfn-s3-mrap-serve")+ _wait_stack(cfn, "cfn-s3-mrap-serve")+++def _reset():+ import urllib.request+ endpoint = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")+ urllib.request.urlopen(+ urllib.request.Request(f"{endpoint}/_ministack/reset", data=b"", method="POST"),+ timeout=10)+++def test_s3_mrap_cleared_by_reset(cfn, s3):+ """Every module-level store must be cleared by reset() — /_ministack/reset is+ what test isolation depends on, so an access point surviving it would leak+ an alias into the next test."""+ buckets = ["mrap-reset-us-east-1-app"]+ cfn.create_stack(StackName="cfn-s3-mrap-reset",+ TemplateBody=_mrap_template("reset-mrap-app", buckets))+ stack = _wait_stack(cfn, "cfn-s3-mrap-reset")+ alias = {o["OutputKey"]: o["OutputValue"] for o in stack["Outputs"]}["Alias"]+ s3.put_object(Bucket=buckets[0], Key="who.txt", Body=b"US BUCKET")+ assert _mrap_get(alias, "who.txt") == (200, "US BUCKET")++ _reset()++ import urllib.error+ with pytest.raises(urllib.error.HTTPError) as ei:+ _mrap_get(alias, "who.txt")+ assert ei.value.code in (403, 404)