Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(cfn): apply AWS::IoT::Policy updates instead of rolling the stack… (#1406)

maximooseminedc652dfcf101
diff --git a/CHANGELOG.md b/CHANGELOG.md
index f7324e55..1d96d2d0 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).
### Added
- **AWS IoT Jobs — control plane and device data plane** — `CreateJob` fell through to `Unsupported IoT path` and the `iot-jobs-data` endpoint did not exist, so device OTA loops could not be rehearsed locally. The `iot` service now serves the nine job operations and a new `iot-jobs-data` service the four device ones (incl. the `$next` sentinel), sharing one store and the AWS execution state machine. A `documentSource` job gets a placeholder, not the S3 object. Contributed by @iot-rocket.
### Fixed
+- **CloudFormation — `AWS::IoT::Policy` updates apply instead of rolling the stack back** — the type had a create and a delete but no update handler, so a stack update fell through to create, hit `ResourceAlreadyExistsException` on the policy it had provisioned itself, and rolled back with the document unchanged: an IoT policy could be deployed once and never edited again. A changed `PolicyDocument` is now a no-interruption update stored as a new policy version and made the default, so `Ref` keeps naming the same policy and `GetPolicy` returns the new document; a changed `PolicyName` is a replacement, creating the policy under the new name and removing the old one. Because IoT caps a policy at five versions, the update prunes the oldest non-default versions before storing a new document, as the CloudFormation handler does on AWS.
- **EC2 — instance public IP and DNS reach the SDKs, and generated addresses are addresses** — `DescribeInstances` and `RunInstances` emitted the public address under `publicIpAddress` / `publicDnsName`, which are not the tags the EC2 wire schema defines (`ipAddress` / `dnsName`, per botocore's `ec2-2016-11-15` model), so every SDK dropped both fields silently and `PublicIpAddress` came back absent on instances that had one. They now ride the real tags. Fixing that exposed a second one: `_random_ip` appended two octets whatever it was given, so a one-octet prefix produced `52.55.218` — `AllocateAddress` has been handing back that shape as `PublicIp` all along, and no address parser accepts it. The generator now completes any prefix to four octets. Contributed by @iot-rocket.
- **S3 — `CRC64NVME` checksums are computed instead of refused** — CRC-64/NVME is the algorithm current AWS SDKs and the CLI checksum uploads with by default, and MiniStack answered it with `InvalidRequest` ("requires optional native dependencies"), so a stock `aws s3 cp` or `aws s3api put-object` failed before it began unless the caller knew to set `AWS_REQUEST_CHECKSUM_CALCULATION=when_required`. The algorithm is plain arithmetic — the reflected form of polynomial `0xAD93D23594C93659` with all-ones init and xorout — so it is now computed from a byte-at-a-time table in the stdlib, adding no dependency: `PutObject` and `CopyObject` validate a client-supplied `x-amz-checksum-crc64nvme` (`BadDigest` on mismatch) and `GetObject` / `HeadObject` return it under `x-amz-checksum-mode: ENABLED`, typed `FULL_OBJECT`. The tests pin the implementation to the algorithm's published check value (`b"123456789"` → `0xAE8B14860A799888`) and to a bit-at-a-time reference rather than to itself. `CRC32C` still requires the native `google-crc32c` and is still refused.
diff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.py
index ca36aa99..7068d9db 100644
--- a/ministack/services/cloudformation/provisioners.py
+++ b/ministack/services/cloudformation/provisioners.py
@@ -5614,17 +5614,69 @@ def _iot_thing_type_delete(physical_id, props):
_iot._thing_types.pop(physical_id, None)
+def _iot_policy_document(props):
+ doc = props.get("PolicyDocument")
+ return json.dumps(doc) if isinstance(doc, (dict, list)) else doc
+
+
+_IOT_POLICY_VERSION_LIMIT = 5
+
+
+def _iot_policy_prune_versions(name):
+ """Make room for one more version, the way the CloudFormation handler does.
+
+ IoT caps a policy at five versions and ``CreatePolicyVersion`` answers
+ ``VersionsLimitExceeded`` once that is reached, so CloudFormation deletes
+ the oldest non-default versions before storing a new document. MiniStack
+ does not enforce the cap today; pruning here keeps the version list the
+ same shape it has on AWS either way.
+ """
+ resp = _iot._list_policy_versions(name)
+ if resp[0] >= 400:
+ return
+ versions = json.loads(resp[2])["policyVersions"]
+ surplus = len(versions) - _IOT_POLICY_VERSION_LIMIT + 1
+ if surplus <= 0:
+ return
+ prunable = sorted(
+ (v["versionId"] for v in versions if not v["isDefaultVersion"]), key=int
+ )
+ for version_id in prunable[:surplus]:
+ _iot._delete_policy_version(name, version_id)
+
+
def _iot_policy_create(logical_id, props, stack_name):
name = props.get("PolicyName") or _physical_name(stack_name, logical_id)
- doc = props.get("PolicyDocument")
- if isinstance(doc, (dict, list)):
- doc = json.dumps(doc)
- resp = _iot._create_policy(name, {"policyDocument": doc})
+ resp = _iot._create_policy(name, {"policyDocument": _iot_policy_document(props)})
if resp[0] >= 400:
raise ValueError(f"AWS::IoT::Policy create failed: {resp[2]!r}")
return name, {"Arn": _iot._policy_arn(name), "Id": name}
+def _iot_policy_update(physical_id, old_props, new_props, stack_name, logical_id=None):
+ """Apply a policy change in place, the way CloudFormation does.
+
+ A new ``PolicyDocument`` is a no-interruption update: IoT stores it as a new
+ version and makes it the default, so ``Ref`` keeps naming the same policy.
+ Renaming is a replacement — the new policy is created and the old one
+ removed, since a policy no template still declares is not left behind.
+ """
+ name = new_props.get("PolicyName") or _physical_name(
+ stack_name, logical_id or physical_id
+ )
+ if name != physical_id:
+ created = _iot_policy_create(logical_id or physical_id, new_props, stack_name)
+ _iot_policy_delete(physical_id, old_props)
+ return created
+ _iot_policy_prune_versions(name)
+ resp = _iot._create_policy_version(
+ name, {"policyDocument": _iot_policy_document(new_props)}, {"setAsDefault": "true"}
+ )
+ if resp[0] >= 400:
+ raise ValueError(f"AWS::IoT::Policy update failed: {resp[2]!r}")
+ return name, {"Arn": _iot._policy_arn(name), "Id": name}
+
+
def _iot_policy_delete(physical_id, props):
_iot._policies.pop(physical_id, None)
@@ -5862,7 +5914,12 @@ _RESOURCE_HANDLERS = {
"AWS::RDS::DBInstance": {"create": _rds_db_instance_create, "delete": _rds_db_instance_delete},
"AWS::IoT::TopicRule": {"create": _iot_topic_rule_create, "delete": _iot_topic_rule_delete},
"AWS::IoT::ThingType": {"create": _iot_thing_type_create, "delete": _iot_thing_type_delete},
- "AWS::IoT::Policy": {"create": _iot_policy_create, "delete": _iot_policy_delete},
+ "AWS::IoT::Policy": {
+ "create": _iot_policy_create,
+ "update": _iot_policy_update,
+ "update_with_logical_id": True,
+ "delete": _iot_policy_delete,
+ },
"AWS::Cognito::IdentityPoolRoleAttachment": {
"create": _cognito_identity_pool_role_attachment_create,
"delete": _cognito_identity_pool_role_attachment_delete,
diff --git a/tests/test_cfn.py b/tests/test_cfn.py
index 9cb9f0bc..2bc7bb62 100644
--- a/tests/test_cfn.py
+++ b/tests/test_cfn.py
@@ -552,6 +552,104 @@ def test_cfn_iot_and_cognito_role_attachment(cfn, iot_client, cognito_identity):
_wait_stack(cfn, "cfn-iot-cog")
+def test_cfn_iot_policy_document_update_applies_in_place(cfn, iot_client):
+ """A changed PolicyDocument updates the policy instead of rolling the stack
+ back: IoT stores a new default version and Ref keeps the same name."""
+ def template(actions):
+ return json.dumps({
+ "Resources": {"Pol": {"Type": "AWS::IoT::Policy", "Properties": {
+ "PolicyName": "cfn-pol-upd",
+ "PolicyDocument": {"Version": "2012-10-17", "Statement": [
+ {"Effect": "Allow", "Action": actions, "Resource": "*"}]}}}},
+ "Outputs": {"Name": {"Value": {"Ref": "Pol"}}},
+ })
+
+ cfn.create_stack(StackName="cfn-iot-pol-upd", TemplateBody=template(["iot:Connect"]))
+ assert _wait_stack(cfn, "cfn-iot-pol-upd")["StackStatus"] == "CREATE_COMPLETE"
+
+ cfn.update_stack(
+ StackName="cfn-iot-pol-upd",
+ TemplateBody=template(["iot:Connect", "iot:Publish"]),
+ )
+ stack = _wait_stack(cfn, "cfn-iot-pol-upd")
+ assert stack["StackStatus"] == "UPDATE_COMPLETE"
+ assert next(
+ o["OutputValue"] for o in stack["Outputs"] if o["OutputKey"] == "Name"
+ ) == "cfn-pol-upd"
+
+ policy = iot_client.get_policy(policyName="cfn-pol-upd")
+ assert json.loads(policy["policyDocument"])["Statement"][0]["Action"] == [
+ "iot:Connect", "iot:Publish",
+ ]
+ assert policy["defaultVersionId"] == "2"
+ versions = iot_client.list_policy_versions(policyName="cfn-pol-upd")["policyVersions"]
+ assert {v["versionId"] for v in versions} == {"1", "2"}
+
+ cfn.delete_stack(StackName="cfn-iot-pol-upd")
+ _wait_stack(cfn, "cfn-iot-pol-upd")
+
+
+def test_cfn_iot_policy_updates_stay_under_the_version_cap(cfn, iot_client):
+ """IoT keeps at most five versions of a policy, so repeated updates prune the
+ oldest non-default version rather than growing the list without bound."""
+ def template(count):
+ return json.dumps({
+ "Resources": {"Pol": {"Type": "AWS::IoT::Policy", "Properties": {
+ "PolicyName": "cfn-pol-cap",
+ "PolicyDocument": {"Version": "2012-10-17", "Statement": [
+ {"Effect": "Allow", "Action": "iot:Connect",
+ "Resource": [f"arn:aws:iot:*:*:client/c{i}" for i in range(count)]}]}}}},
+ })
+
+ cfn.create_stack(StackName="cfn-iot-pol-cap", TemplateBody=template(1))
+ assert _wait_stack(cfn, "cfn-iot-pol-cap")["StackStatus"] == "CREATE_COMPLETE"
+
+ for count in range(2, 9):
+ cfn.update_stack(StackName="cfn-iot-pol-cap", TemplateBody=template(count))
+ assert _wait_stack(cfn, "cfn-iot-pol-cap")["StackStatus"] == "UPDATE_COMPLETE"
+
+ versions = iot_client.list_policy_versions(policyName="cfn-pol-cap")["policyVersions"]
+ assert len(versions) == 5
+ assert {v["versionId"] for v in versions} == {"4", "5", "6", "7", "8"}
+
+ policy = iot_client.get_policy(policyName="cfn-pol-cap")
+ assert policy["defaultVersionId"] == "8"
+ assert len(json.loads(policy["policyDocument"])["Statement"][0]["Resource"]) == 8
+
+ cfn.delete_stack(StackName="cfn-iot-pol-cap")
+ _wait_stack(cfn, "cfn-iot-pol-cap")
+
+
+def test_cfn_iot_policy_rename_replaces_the_policy(cfn, iot_client):
+ """Renaming is a replacement — the new policy exists under the new name and
+ the old one does not survive the update."""
+ def template(name):
+ return json.dumps({
+ "Resources": {"Pol": {"Type": "AWS::IoT::Policy", "Properties": {
+ "PolicyName": name,
+ "PolicyDocument": {"Version": "2012-10-17", "Statement": [
+ {"Effect": "Allow", "Action": "iot:Connect", "Resource": "*"}]}}}},
+ "Outputs": {"Name": {"Value": {"Ref": "Pol"}}},
+ })
+
+ cfn.create_stack(StackName="cfn-iot-pol-ren", TemplateBody=template("cfn-pol-before"))
+ assert _wait_stack(cfn, "cfn-iot-pol-ren")["StackStatus"] == "CREATE_COMPLETE"
+
+ cfn.update_stack(StackName="cfn-iot-pol-ren", TemplateBody=template("cfn-pol-after"))
+ stack = _wait_stack(cfn, "cfn-iot-pol-ren")
+ assert stack["StackStatus"] == "UPDATE_COMPLETE"
+ assert next(
+ o["OutputValue"] for o in stack["Outputs"] if o["OutputKey"] == "Name"
+ ) == "cfn-pol-after"
+
+ assert iot_client.get_policy(policyName="cfn-pol-after")["policyName"] == "cfn-pol-after"
+ with pytest.raises(iot_client.exceptions.ResourceNotFoundException):
+ iot_client.get_policy(policyName="cfn-pol-before")
+
+ cfn.delete_stack(StackName="cfn-iot-pol-ren")
+ _wait_stack(cfn, "cfn-iot-pol-ren")
+
+
def test_cfn_lambda_layer_version_permission(cfn, lam):
"""AWS::Lambda::LayerVersionPermission attaches a statement to the real
layer version's policy, readable via GetLayerVersionPolicy. (#1345, item 5)"""