Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
fix(cfn): SSM fix + iam charset (#1444)
Nahuel Nucera59819962a31f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex b557c077..b2b9d4c3 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -22,7 +22,11 @@ Versioning follows [Semantic Versioning](https://semver.org/).- **S3 — versioning edge cases: the null version, delete-marker signals, and versioned copies** — five behaviors around the edges of object versioning now match AWS. A PUT on a Suspended bucket minted a fresh UUID version id like an Enabled one; it is now stored as the `null` version — replacing any previous null version, answered without `x-amz-version-id` — and a suspended DELETE mints its marker under the literal `null` id the same way. The object written before versioning was enabled used to vanish from the index at the first versioned overwrite; it now stays addressable as `VersionId=null` and becomes current again when the newer version is deleted. Delete markers announce themselves: `DeleteObjects` without a per-key VersionId on a versioned bucket now creates a marker (for keys that never existed too) and reports `DeleteMarker`/`DeleteMarkerVersionId`, and a GET/HEAD 404 for a key hidden by a marker carries `x-amz-delete-marker: true` with the marker's id. `UploadPartCopy` ignored the copy source's `?versionId=` and copied the current bytes; it now copies that exact version and echoes `x-amz-copy-source-version-id`. A versioned `CopyObject` dropped the source version's user metadata (the index kept it; the synthesized source object just didn't carry it). And both copy operations decoded the copy-source header before splitting at `?`, so a key legitimately named `?versionId` — sent percent-encoded — read as an empty key with a version qualifier; the raw header is split first now. The five version-minting sites (PutObject, POST, CopyObject, CompleteMultipartUpload, and the delete-marker path) shared five hand-copied bookkeeping blocks; they now go through one helper, which is what makes the null-version rules uniform. Contributed by @gaul.- **S3 — `CompleteMultipartUpload` honors `If-Match` / `If-None-Match`** — conditional writes landed on PutObject, but CompleteMultipartUpload accepted the same headers and ignored them, so a create-once (`If-None-Match: *`) or compare-and-swap (`If-Match: <etag>`) upload that took the multipart path silently overwrote whatever was there. The complete now evaluates the same preconditions PutObject does, against the current object at complete time — 412 on a violated condition, 404 `NoSuchKey` when If-Match addresses a key with no current object. That last rule also tightens PutObject: the `*` form previously answered the RFC 7232 412 for a missing key, but AWS documents a single If-Match error row — `NoSuchKey` — and ceph/s3-tests pins 404 for both forms, so both operations now answer it. The idempotent complete-retry replay still bypasses the check, since the original request passed its conditions when it committed. Contributed by @gaul.- **S3 — canned ACLs are stored, and object ACLs bind to versions** — SDKs send `put-bucket-acl --acl public-read` as an `x-amz-acl` header with an empty body, and `PutBucketAcl` read only the body, so the canned grant vanished and the bucket read back owner-only — a caller that gates anonymous access on the reported grants (S3Proxy's `aws-s3` backend among them) then treated every bucket as private. `CreateBucket`'s `x-amz-acl` was dropped the same way. Both now validate the canned value (`InvalidArgument` otherwise) and store the grants it implies, `log-delivery-write` included, exactly as the object side has since #1322; an XML body must be well-formed (`MalformedACLError`) and a request with neither header nor body answers `MissingSecurityHeader`, as on AWS. Object ACLs are now per-version like tags (#1058): `?versionId=` reads or writes that version's ACL, a fresh version starts from the default policy instead of inheriting its predecessor's, the canned value on `PutObject` sticks to the version that PUT created, and version and bucket deletes drop the stored entries. The browser-POST form names its field `acl` rather than `x-amz-acl`, so it never matched the header list and was discarded; it now applies to the uploaded object. Contributed by @gaul.-- **S3 — `CRC64NVME` checksums are computed instead of refused** — CRC-64/NVME is the algorithm current AWS SDKs and the CLI checksum uploads with by default, and MiniStack answered it with `InvalidRequest` ("requires optional native dependencies"), so a stock `aws s3 cp` or `aws s3api put-object` failed before it began unless the caller knew to set `AWS_REQUEST_CHECKSUM_CALCULATION=when_required`. The algorithm is plain arithmetic — the reflected form of polynomial `0xAD93D23594C93659` with all-ones init and xorout — so it is now computed from a byte-at-a-time table in the stdlib, adding no dependency: `PutObject` and `CopyObject` validate a client-supplied `x-amz-checksum-crc64nvme` (`BadDigest` on mismatch) and `GetObject` / `HeadObject` return it under `x-amz-checksum-mode: ENABLED`, typed `FULL_OBJECT`. The tests pin the implementation to the algorithm's published check value (`b"123456789"` → `0xAE8B14860A799888`) and to a bit-at-a-time reference rather than to itself. `CRC32C` still requires the native `google-crc32c` and is still refused.+- **S3 — `CRC64NVME` checksums are computed instead of refused** — CRC-64/NVME is the algorithm current AWS SDKs and the CLI checksum uploads with by default, and MiniStack answered it with `InvalidRequest` ("requires optional native dependencies"), so a stock `aws s3 cp` or `aws s3api put-object` failed before it began unless the caller knew to set `AWS_REQUEST_CHECKSUM_CALCULATION=when_required`. The algorithm is plain arithmetic — the reflected form of polynomial `0xAD93D23594C93659` with all-ones init and xorout — so it is now computed from a byte-at-a-time table in the stdlib, adding no dependency: `PutObject` and `CopyObject` validate a client-supplied `x-amz-checksum-crc64nvme` (`BadDigest` on mismatch) and `GetObject` / `HeadObject` return it under `x-amz-checksum-mode: ENABLED`, typed `FULL_OBJECT`. The tests pin the implementation to the algorithm's published check value (`b"123456789"` → `0xAE8B14860A799888`) and to a bit-at-a-time reference rather than to itself. `CRC32C` still requires the native `google-crc32c` and is still refused. Contributed by @gaul.+- **CloudFormation — auto-generated physical names keep their uniqueness suffix when truncated** — `_physical_name` built the full `{stackName}-{logicalId}-{SUFFIX}` string and only then sliced it to `max_len` from the end, so when a deeply-nested stack's own generated name already exceeded a resource's name cap (e.g. Lambda's 64-char `FunctionName` limit), every resource in that stack truncated to an identical string and collapsed onto one physical resource — every Lambda in the nested stack became a single function, so only one ever ran. The suffix (a deterministic hash of stack name and logical id) that guarantees uniqueness is now preserved: the `{stackName}-{logicalId}-` prefix is truncated and the suffix always kept, matching how AWS and the CDK generate physical names. Contributed by @ryan-bennett.+- **IAM — role `Description` charset is validated** — `CreateRole`, `UpdateRole` and `UpdateRoleDescription` now reject a description outside `[\u0009\u000A\u000D\u0020-\u007E\u00A1-\u00FF]` or longer than 1000 characters with a `400 ValidationError`, matching IAM. Reported by @iot-rocket.+- **CloudFormation — `AWS::SSM::Parameter` goes through the SSM API** — instead of writing the store directly, so a create over an existing name fails (`ParameterAlreadyExists`), updates increment `Version` and record history, a `Name` change replaces the parameter, a `SecureString` `Type` is rejected (as CloudFormation does), and `Fn::GetAtt` exposes `Arn` / `Type` / `Value`. Reported by @iot-rocket.+- **CloudFormation — `AWS::SSM::Parameter::Value<...>` is re-resolved on `UpdateStack`** — the SSM parameter name is kept and re-resolved on every operation, so an update with `UsePreviousValue=true` picks up a value changed in Parameter Store since the last deploy. Reported by @iot-rocket.## [1.4.19] — 2026-08-16diff --git a/ministack/services/cloudformation/engine.py b/ministack/services/cloudformation/engine.pyindex faaffe44..cc4af767 100644--- a/ministack/services/cloudformation/engine.py+++ b/ministack/services/cloudformation/engine.py@@ -170,8 +170,21 @@ def _resolve_parameters(template: dict, provided_params: list[dict],if entry is not None and entry.get("UsePreviousValue"):prev = previous_params.get(name)if prev is not None:- value = prev["Value"] if isinstance(prev, dict) else prev- already_resolved = True+ if (ptype.startswith(_SSM_PARAMETER_VALUE_PREFIX)+ and isinstance(prev, dict) and "SsmName" in prev):+ # An `AWS::SSM::Parameter::Value<...>` type is re-resolved on+ # every stack operation, as real CloudFormation does: the+ # prior deployment stored the SSM parameter NAME (`SsmName`)+ # for exactly this, so `UsePreviousValue=true` (what `cdk+ # deploy` sends for every non-overridden parameter) picks up+ # a value that changed in Parameter Store since the last+ # deploy. Leaving `already_resolved` False re-runs the SSM+ # lookup below. Stacks persisted before `SsmName` was tracked+ # fall through to the last resolved value (the old behavior).+ value = prev["SsmName"]+ else:+ value = prev["Value"] if isinstance(prev, dict) else prev+ already_resolved = Trueelif "Default" in defn:value = defn["Default"]else:@@ -186,15 +199,18 @@ def _resolve_parameters(template: dict, provided_params: list[dict],value = str(value) if value is not None else ""+ ssm_name = Noneif ptype.startswith(_SSM_PARAMETER_VALUE_PREFIX) and not already_resolved:# `value` up to here is the SSM parameter *name* (the template- # parameter's Default, or a caller-supplied override) — resolve- # it against the SSM store the same way real CloudFormation- # does before Ref ever sees it. Local import to avoid a- # cloudformation/ssm circular import at module load time (see- # ecs.py's identical pattern for the same reason).+ # parameter's Default, a caller-supplied override, or the previous+ # deployment's stored `SsmName`) — resolve it against the SSM store+ # the same way real CloudFormation does before Ref ever sees it, and+ # remember the name so the next update can re-resolve it. Local+ # import to avoid a cloudformation/ssm circular import at module load+ # time (see ecs.py's identical pattern for the same reason).from ministack.services import ssmparam_name = value+ ssm_name = valueresolved_value = ssm.resolve_parameter_value(param_name)if resolved_value is None:raise ValueError(@@ -222,7 +238,12 @@ def _resolve_parameters(template: dict, provided_params: list[dict],pass# AWS-specific types treated as String -- no extra validation- resolved[name] = {"Value": value, "NoEcho": no_echo}+ out = {"Value": value, "NoEcho": no_echo}+ if ssm_name is not None:+ # Persisted so a later UpdateStack with UsePreviousValue re-resolves+ # the SSM name instead of reusing this now-stale resolved value.+ out["SsmName"] = ssm_name+ resolved[name] = outreturn resolveddiff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.pyindex 670ff3f5..b92e7a49 100644--- a/ministack/services/cloudformation/provisioners.py+++ b/ministack/services/cloudformation/provisioners.py@@ -1046,31 +1046,90 @@ def _iam_ip_delete(physical_id, props):# --- SSM Parameter ----def _ssm_create(logical_id, props, stack_name):- name = props.get("Name") or f"/{stack_name}/{logical_id}"+# CloudFormation supports only these two parameter types; `SecureString` in+# particular is not supported (aws-resource-ssm-parameter reference).+_SSM_CFN_PARAMETER_TYPES = ("String", "StringList")+++def _ssm_check_type(props):ptype = props.get("Type", "String")- value = props.get("Value", "")- description = props.get("Description", "")- param_arn = _ssm._param_arn(name)+ if ptype not in _SSM_CFN_PARAMETER_TYPES:+ raise ValueError(+ f"AWS::SSM::Parameter Type '{ptype}' is not supported by "+ "CloudFormation (allowed values: String, StringList)")+- _ssm._parameters[name] = {+def _ssm_put_data(name, props):+ """PutParameter payload for an ``AWS::SSM::Parameter`` resource.++ CloudFormation ``Tags`` is a map; the SSM API wants a ``[{Key, Value}]`` list.+ """+ data = {"Name": name,- "Type": ptype,- "Value": value,- "Version": 1,- "LastModifiedDate": _ssm._now_epoch(),- "ARN": param_arn,- "DataType": "text",- "Description": description,+ "Type": props.get("Type", "String"),+ "Value": props.get("Value", ""),+ "Description": props.get("Description", ""),"Tier": props.get("Tier", "Standard"),"AllowedPattern": props.get("AllowedPattern", ""),- "Tags": [],+ "DataType": props.get("DataType", "text"),}- return name, {"Type": ptype, "Value": value}+ if props.get("Policies"):+ data["Policies"] = props["Policies"]+ tags = props.get("Tags")+ if isinstance(tags, dict):+ data["Tags"] = [{"Key": k, "Value": v} for k, v in tags.items()]+ return data+++def _ssm_attrs(name, data):+ # Ref returns the parameter name; Fn::GetAtt exposes Arn / Type / Value.+ return {"Arn": _ssm._param_arn(name), "Type": data["Type"], "Value": data["Value"]}+++def _ssm_create(logical_id, props, stack_name):+ # Go through PutParameter rather than writing the SSM store directly, so the+ # two doors into the same store behave alike: a create over an existing+ # parameter fails as real CloudFormation does (`ParameterAlreadyExists`), and+ # Version/history stay consistent with the API path.+ _ssm_check_type(props)+ name = props.get("Name") or f"/{stack_name}/{logical_id}"+ data = _ssm_put_data(name, props)+ status, _headers, body = _ssm._put_parameter(data)+ if status >= 400:+ raise ValueError(f"AWS::SSM::Parameter create failed: {body!r}")+ return name, _ssm_attrs(name, data)+++def _ssm_update(physical_id, old_props, new_props, stack_name):+ _ssm_check_type(new_props)+ new_name = new_props.get("Name")+ if new_name and new_name != physical_id:+ # Name is Update requires: Replacement — create the new parameter and+ # drop the old one, returning the new physical id.+ data = _ssm_put_data(new_name, new_props)+ status, _headers, body = _ssm._put_parameter(data)+ if status >= 400:+ raise ValueError(f"AWS::SSM::Parameter replace failed: {body!r}")+ # Drop the old parameter through the SSM path so its history and tags go+ # with it (a bare store pop orphaned both).+ _ssm._delete_parameter({"Name": physical_id})+ return new_name, _ssm_attrs(new_name, data)+ # Every other property is No interruption: overwrite in place through+ # PutParameter, so Version increments and history grows (a bare store write+ # pinned Version at 1 forever).+ data = _ssm_put_data(physical_id, new_props)+ data["Overwrite"] = True+ status, _headers, body = _ssm._put_parameter(data)+ if status >= 400:+ raise ValueError(f"AWS::SSM::Parameter update failed: {body!r}")+ return physical_id, _ssm_attrs(physical_id, data)def _ssm_delete(physical_id, props):- _ssm._parameters.pop(physical_id, None)+ # Delete through the SSM path (not a bare store pop) so history and tags are+ # cleaned too; a missing parameter is a no-op, so repeated/post-reset deletes+ # converge.+ _ssm._delete_parameter({"Name": physical_id})# --- AppConfig Application ---@@ -2382,16 +2441,15 @@ def _apigw_api_key_create(logical_id, props, stack_name):def _apigw_api_key_update(physical_id, old_props, new_props, stack_name):- # Value is immutable in CloudFormation; changing it replaces the key.- # Everything else updates the existing record in place.- if old_props.get("Value") != new_props.get("Value"):+ # Value and Name are both Replacement in CloudFormation; changing either+ # replaces the key. Everything else updates the existing record in place.+ if (old_props.get("Value") != new_props.get("Value")+ or old_props.get("Name") != new_props.get("Name")):_apigw_v1._delete_api_key(physical_id)return _apigw_api_key_create(physical_id, new_props, stack_name)key = _apigw_v1._api_keys.get(physical_id)if key is None:return _apigw_api_key_create(physical_id, new_props, stack_name)- if new_props.get("Name"):- key["name"] = new_props["Name"]key["description"] = new_props.get("Description", "")key["enabled"] = new_props.get("Enabled", True)key["customerId"] = new_props.get("CustomerId", key.get("customerId", ""))@@ -2454,8 +2512,9 @@ def _apigw_usage_plan_delete(physical_id, props):def _apigw_usage_plan_key_create(logical_id, props, stack_name):"""Provision an ``AWS::ApiGateway::UsagePlanKey`` (associate a key with a plan).- Ref returns the API key id; the delete handler reads the usage plan id back- from the resource's own properties, so the physical id need not encode it.+ Ref returns ``{keyId}:{usagePlanId}`` — the physical id AWS assigns this+ resource — and the delete handler reads both ids back from the resource's+ own properties."""plan_id = props.get("UsagePlanId", "")key_id = props.get("KeyId", "")@@ -2463,7 +2522,7 @@ def _apigw_usage_plan_key_create(logical_id, props, stack_name):status, _headers, body = _apigw_v1._create_usage_plan_key(plan_id, data)if status >= 400:raise ValueError(f"AWS::ApiGateway::UsagePlanKey create failed: {body!r}")- return key_id, {}+ return f"{key_id}:{plan_id}", {}def _apigw_usage_plan_key_delete(physical_id, props):@@ -5902,7 +5961,7 @@ _RESOURCE_HANDLERS = {"AWS::IAM::Role": {"create": _iam_role_create, "delete": _iam_role_delete},"AWS::IAM::Policy": {"create": _iam_policy_create, "delete": _iam_policy_delete},"AWS::IAM::InstanceProfile": {"create": _iam_ip_create, "delete": _iam_ip_delete},- "AWS::SSM::Parameter": {"create": _ssm_create, "delete": _ssm_delete},+ "AWS::SSM::Parameter": {"create": _ssm_create, "update": _ssm_update, "delete": _ssm_delete},"AWS::AppConfig::Application": {"create": _appconfig_application_create,"delete": _appconfig_application_delete,diff --git a/ministack/services/iam.py b/ministack/services/iam.pyindex fe4e239e..491646ee 100644--- a/ministack/services/iam.py+++ b/ministack/services/iam.py@@ -36,6 +36,7 @@ import copyimport jsonimport loggingimport os+import reimport timefrom urllib.parse import parse_qsfrom urllib.parse import quote as _url_quote@@ -457,8 +458,41 @@ def _delete_user(p):# -------------------- Role management --------------------+# IAM restricts a role's Description to tab / CR / LF plus printable ASCII and+# the Latin-1 supplement (botocore `roleDescriptionType`), max 1000 chars. A+# character outside that set — an em dash, say — is a 400 ValidationError on+# real AWS; without this check MiniStack stored it verbatim, so an invalid+# template deployed fine locally and only failed once it reached a real account.+_ROLE_DESCRIPTION_RE = re.compile("[\u0009\u000A\u000D\u0020-\u007E\u00A1-\u00FF]*")++++def _validate_role_description(description):+ """None when the description is absent or valid; a 400 ValidationError otherwise."""+ if description is None:+ return None+ if len(description) > 1000:+ return _error(+ 400, "ValidationError",+ "1 validation error detected: Value at 'description' failed to satisfy "+ "constraint: Member must have length less than or equal to 1000",+ ns="iam")+ if _ROLE_DESCRIPTION_RE.fullmatch(description) is None:+ return _error(+ 400, "ValidationError",+ "1 validation error detected: Value at 'description' failed to satisfy "+ "constraint: Member must satisfy regular expression pattern: "+ "[\\u0009\\u000A\\u000D\\u0020-\\u007E\\u00A1-\\u00FF]*",+ ns="iam")+ return None++def _create_role(p):name = _p(p, "RoleName")+ description = _p(p, "Description")+ desc_err = _validate_role_description(description)+ if desc_err:+ return desc_errif name in _roles:return _error(409, "EntityAlreadyExists",f"Role with name {name} already exists.", ns="iam")@@ -470,7 +504,7 @@ def _create_role(p):"CreateDate": _now(),"Path": path,"AssumeRolePolicyDocument": _p(p, "AssumeRolePolicyDocument"),- "Description": _p(p, "Description"),+ "Description": description,"MaxSessionDuration": int(_p(p, "MaxSessionDuration") or 3600),"PermissionsBoundary": _p(p, "PermissionsBoundary"),"AttachedPolicies": [],@@ -531,7 +565,11 @@ def _update_role(p):if not role:return _error(404, "NoSuchEntity", f"Role {name} not found.", ns="iam")if "Description" in p:- role["Description"] = _p(p, "Description")+ description = _p(p, "Description")+ desc_err = _validate_role_description(description)+ if desc_err:+ return desc_err+ role["Description"] = descriptionif "MaxSessionDuration" in p:role["MaxSessionDuration"] = int(_p(p, "MaxSessionDuration", "3600"))return _xml(200, "UpdateRoleResponse", "<UpdateRoleResult></UpdateRoleResult>", ns="iam")@@ -552,7 +590,11 @@ def _update_role_description(p):if not role:return _error(404, "NoSuchEntity",f"Role {name} not found.", ns="iam")- role["Description"] = _p(p, "Description")+ description = _p(p, "Description")+ desc_err = _validate_role_description(description)+ if desc_err:+ return desc_err+ role["Description"] = descriptionreturn _xml(200, "UpdateRoleDescriptionResponse",f"<UpdateRoleDescriptionResult><Role>{_role_xml(name)}</Role></UpdateRoleDescriptionResult>",ns="iam")diff --git a/tests/test_cfn.py b/tests/test_cfn.pyindex 72fbfa94..095625e5 100644--- a/tests/test_cfn.py+++ b/tests/test_cfn.py@@ -6748,8 +6748,8 @@ def test_cfn_apigateway_usage_plan_and_key_lifecycle(cfn, apigw_v1):plan_id = outputs["PlanRef"]assert outputs["PlanGetAtt"] == plan_id- # UsagePlanKey Ref is the associated API key id.- assert outputs["PlanKeyRef"] == outputs["KeyRef"]+ # UsagePlanKey Ref is "{keyId}:{usagePlanId}" (the AWS-documented physical id).+ assert outputs["PlanKeyRef"] == f"{outputs['KeyRef']}:{outputs['PlanRef']}"plan = apigw_v1.get_usage_plan(usagePlanId=plan_id)assert plan["name"] == f"plan-{suffix}"