Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(iot): stack tags on IoT resources and provisioning template checks (#2191)

iot-rocketda1ba805b857
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 56152223..3b4c9e89 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,8 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Fixed
+- **IoT / CloudFormation — stack tags and provisioning templates** — stack-level tags never reached IoT resources; they now reach thing types, thing groups, policies, topic rules, CA certificates and provisioning templates without the `aws:cloudformation:` tags (job templates get none, as on AWS), and an update that only changes tags no longer adds a policy version. An `AWS::IoT::ProvisioningTemplate` `Enabled` of `"false"` from a parameter or the template enabled the template; it now disables it. `CreateProvisioningTemplate` and the CloudFormation type accepted any certificate resource; they now refuse one without a `Status` of `Active` or `Inactive` (any case) or whose `CertificateId` is not a reference to `AWS::IoT::Certificate::Id`, a second certificate resource and a resource without `Properties`, with AWS's messages. Contributed by @iot-rocket.
### Added
- **WAF v2 — regex pattern sets** — `CreateRegexPatternSet`, `GetRegexPatternSet`, `UpdateRegexPatternSet`, `DeleteRegexPatternSet` and `ListRegexPatternSets` now manage regional and CloudFront-scoped regex pattern sets, with their ARN (`regexpatternset`), tags and `LockToken` checks on update and delete (`WAFOptimisticLockException` for a stale token), as AWS does. Web ACL and rule group rules are stored as sent, so a `RegexPatternSetReferenceStatement` pointing at a pattern set ARN round-trips; the statement is not evaluated, as MiniStack does not inspect requests.
### Fixed
diff --git a/README.md b/README.md
index 4fdc2596..b07d87a7 100644
--- a/README.md
+++ b/README.md
@@ -467,7 +467,7 @@ subnet = ec2.create_subnet(
| **Intrinsic Functions** | Ref, Fn::GetAtt, Fn::Join, Fn::Sub (both forms), Fn::Select, Fn::Split, Fn::If, Fn::Equals, Fn::And, Fn::Or, Fn::Not, Fn::Base64, Fn::FindInMap, Fn::ImportValue, Fn::GetAZs, Fn::Cidr |
| **Pseudo-Parameters** | AWS::StackName, AWS::StackId, AWS::Region, AWS::AccountId, AWS::URLSuffix, AWS::Partition, AWS::NoValue |
| **Parameters** | Default values, AllowedValues validation, NoEcho masking, String/Number/CommaDelimitedList types |
-| **Stack Tags** | The `Tags` of CreateStack, UpdateStack and change sets reach every resource whose type carries a tag property, together with `aws:cloudformation:stack-name`, `stack-id` and `logical-id`; a key the template sets wins |
+| **Stack Tags** | The `Tags` of CreateStack, UpdateStack and change sets reach every resource whose type carries a tag property, together with `aws:cloudformation:stack-name`, `stack-id` and `logical-id` (IoT types get the stack's tags without these three, IoT job templates get none); a key the template sets wins |
| **Conditions** | Fn::Equals, Fn::And, Fn::Or, Fn::Not — conditional resource creation |
| **Rules** | The `Rules` section runs after the parameters resolve and before any resource is touched (CreateStack, UpdateStack, CreateChangeSet): `RuleCondition`, `Assertions` with `AssertDescription`, Fn::And/Or/Not/Equals/If/Contains/EachMemberEquals/EachMemberIn and Ref; Fn::RefAll, Fn::ValueOf and Fn::ValueOfAll read the EC2 store for VPC, subnet and security-group ids |
| **Transforms** | `AWS::Serverless-2016-10-31` (SAM, in the `full` image), `AWS::LanguageExtensions` (`Fn::ForEach` over a literal list, a `CommaDelimitedList` parameter, `Fn::FindInMap` or `Fn::Split`, nested and inside `Properties`; `${Id}` / `&{Id}` in the fragment keys, `${Id}` in `Fn::Sub` and a `Ref` to the identifier; `Fn::Length`, `Fn::ToJsonString`) and the embedded `AWS::Include` transform (`Fn::Transform` with an `s3://` `Location`, JSON or YAML snippets) |
diff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.py
index 61fbaa6f..a6123768 100644
--- a/ministack/services/cloudformation/provisioners.py
+++ b/ministack/services/cloudformation/provisioners.py
@@ -1140,6 +1140,12 @@ _STACK_TAG_PROPERTY: dict[str, tuple[str, str]] = {
"AWS::Events::EventBus": ("Tags", "list"),
"AWS::IAM::Role": ("Tags", "list"),
"AWS::IAM::User": ("Tags", "list"),
+ "AWS::IoT::CACertificate": ("Tags", "list"),
+ "AWS::IoT::Policy": ("Tags", "list"),
+ "AWS::IoT::ProvisioningTemplate": ("Tags", "list"),
+ "AWS::IoT::ThingGroup": ("Tags", "list"),
+ "AWS::IoT::ThingType": ("Tags", "list"),
+ "AWS::IoT::TopicRule": ("Tags", "list"),
"AWS::KMS::Key": ("Tags", "list"),
"AWS::Kinesis::Stream": ("Tags", "list"),
"AWS::Lambda::Function": ("Tags", "list"),
@@ -1182,9 +1188,11 @@ def _with_stack_tags(resource_type: str, props: dict, stack_tags: list,
return props
prop, shape = spec
extra = _tag_map(stack_tags)
- extra["aws:cloudformation:stack-name"] = stack_name
- extra["aws:cloudformation:stack-id"] = stack_id
- extra["aws:cloudformation:logical-id"] = logical_id
+ # IoT refuses aws: tag keys and lists none of these on its resources.
+ if not resource_type.startswith("AWS::IoT::"):
+ extra["aws:cloudformation:stack-name"] = stack_name
+ extra["aws:cloudformation:stack-id"] = stack_id
+ extra["aws:cloudformation:logical-id"] = logical_id
own = props.get(prop)
if shape == "map":
merged = {**extra, **(own if isinstance(own, dict) else _tag_map(own))}
@@ -13616,12 +13624,13 @@ def _iot_policy_update(physical_id, old_props, new_props, stack_name, logical_id
created = _iot_policy_create(logical_id or physical_id, new_props, stack_name)
_delete_predecessor(_iot_policy_delete, physical_id, old_props)
return created
- _iot_policy_prune_versions(name)
- resp = _iot._create_policy_version(
- name, {"policyDocument": _iot_policy_document(new_props)}, {"setAsDefault": "true"}
- )
- if resp[0] >= 400:
- raise ValueError(f"AWS::IoT::Policy update failed: {resp[2]!r}")
+ if _iot_policy_document(new_props) != _iot_policy_document(old_props):
+ _iot_policy_prune_versions(name)
+ resp = _iot._create_policy_version(
+ name, {"policyDocument": _iot_policy_document(new_props)}, {"setAsDefault": "true"}
+ )
+ if resp[0] >= 400:
+ raise ValueError(f"AWS::IoT::Policy update failed: {resp[2]!r}")
_iot_update_tags(_iot._policy_arn(name), old_props, new_props)
return name, {"Arn": _iot._policy_arn(name), "Id": name}
@@ -13661,7 +13670,7 @@ def _iot_provisioning_template_payload(name, props):
if "Description" in props:
payload["description"] = props["Description"]
if "Enabled" in props:
- payload["enabled"] = props["Enabled"]
+ payload["enabled"] = _cfn_bool(props["Enabled"])
if "TemplateType" in props:
payload["type"] = props["TemplateType"]
if props.get("PreProvisioningHook"):
diff --git a/ministack/services/iot.py b/ministack/services/iot.py
index d97822e8..45715e25 100644
--- a/ministack/services/iot.py
+++ b/ministack/services/iot.py
@@ -1926,9 +1926,25 @@ def _validate_provisioning_template_name(name: str | None) -> tuple | None:
return None
+def _provisioning_template_resource_error(resource, certificates: int) -> str | None:
+ """Why the service refuses one resource of a template body, or None."""
+ props = resource.get("Properties")
+ if props is None:
+ return "Properties of a resource cannot be null."
+ if resource.get("Type") != "AWS::IoT::Certificate":
+ return None
+ if certificates:
+ return "Only one certificate resource can be defined"
+ if not isinstance(props, dict) or props.get("CertificateId") != {"Ref": "AWS::IoT::Certificate::Id"}:
+ return "CertificateId must contain a reference to AWS::IoT::Certificate::Id"
+ status = props.get("Status")
+ if not isinstance(status, str) or status.lower() not in ("active", "inactive"):
+ return 'A certificate resource must specify a Status of either "Active" or "Inactive"'
+ return None
+
+
def _validate_provisioning_template_body(body) -> tuple | None:
- """The live service refuses a body without an AWS::IoT::Certificate
- resource; its message is quoted verbatim below."""
+ """Refuse a body the live service refuses, with its messages verbatim."""
doc = body
if not isinstance(doc, dict):
try:
@@ -1943,10 +1959,17 @@ def _validate_provisioning_template_body(body) -> tuple | None:
)
resources = doc.get("Resources")
values = resources.values() if isinstance(resources, dict) else ()
- if not any(
- isinstance(r, dict) and r.get("Type") == "AWS::IoT::Certificate"
- for r in values
- ):
+ certificates = 0
+ for resource in values:
+ if not isinstance(resource, dict):
+ continue
+ reason = _provisioning_template_resource_error(resource, certificates)
+ if reason:
+ return error_response_json(
+ "InvalidRequestException", f"The template body is invalid: {reason}", 400
+ )
+ certificates += resource.get("Type") == "AWS::IoT::Certificate"
+ if not certificates:
return error_response_json(
"InvalidRequestException",
"The template body is invalid: Template must have certificate resource.",
diff --git a/tests/test_cfn.py b/tests/test_cfn.py
index 4d80e47d..ea4bdeb7 100644
--- a/tests/test_cfn.py
+++ b/tests/test_cfn.py
@@ -1413,7 +1413,8 @@ def _cfn_provisioning_template_body(marker="one"):
"Resources": {
"certificate": {"Type": "AWS::IoT::Certificate",
"Properties": {"CertificateId":
- {"Ref": "AWS::IoT::Certificate::Id"}}},
+ {"Ref": "AWS::IoT::Certificate::Id"},
+ "Status": "ACTIVE"}},
"thing": {"Type": "AWS::IoT::Thing",
"Properties": {"ThingName": {"Ref": "SerialNumber"},
"AttributePayload": {"Attributes":
@@ -1475,6 +1476,60 @@ def test_cfn_iot_provisioning_template(cfn, iot_client):
iot_client.describe_provisioning_template(templateName="cfn-provtemplate")
+def test_cfn_iot_provisioning_template_enabled_from_a_string(cfn, iot_client):
+ """Enabled as the string 'false' disables the template; a parameter update to 'true' enables it in place."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ stack_name, by_ref, literal = f"cfn-pt-enabled-{uid}", f"cfn-pt-ref-{uid}", f"cfn-pt-str-{uid}"
+
+ def template(name, enabled):
+ return {"Type": "AWS::IoT::ProvisioningTemplate", "Properties": {
+ "TemplateName": name, "Enabled": enabled,
+ "ProvisioningRoleArn": "arn:aws:iam::000000000000:role/provisioning",
+ "TemplateBody": _cfn_provisioning_template_body()}}
+
+ body = json.dumps({
+ "Parameters": {"EnabledParam": {"Type": "String", "Default": "false"}},
+ "Resources": {"ByRef": template(by_ref, {"Ref": "EnabledParam"}),
+ "Literal": template(literal, "false")},
+ })
+
+ def enabled(name):
+ return iot_client.describe_provisioning_template(templateName=name)["enabled"]
+
+ cfn.create_stack(StackName=stack_name, TemplateBody=body)
+ try:
+ stack = _wait_stack(cfn, stack_name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ assert (enabled(by_ref), enabled(literal)) == (False, False)
+ cfn.update_stack(StackName=stack_name, TemplateBody=body, Parameters=[
+ {"ParameterKey": "EnabledParam", "ParameterValue": "true"}])
+ stack = _wait_stack(cfn, stack_name)
+ assert stack["StackStatus"] == "UPDATE_COMPLETE", stack.get("StackStatusReason")
+ assert (enabled(by_ref), enabled(literal)) == (True, False)
+ finally:
+ _delete_cfn_test_stack(cfn, stack_name)
+
+
+def test_cfn_iot_provisioning_template_certificate_without_status_fails_the_stack(cfn, iot_client):
+ """A certificate resource without Status fails the create and rolls the stack back with no template left."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ stack_name, name = f"cfn-pt-nostatus-{uid}", f"cfn-pt-nostatus-{uid}"
+ body = json.loads(_cfn_provisioning_template_body())
+ del body["Resources"]["certificate"]["Properties"]["Status"]
+ cfn.create_stack(StackName=stack_name, TemplateBody=json.dumps({"Resources": {"PT": {
+ "Type": "AWS::IoT::ProvisioningTemplate", "Properties": {
+ "TemplateName": name, "TemplateBody": json.dumps(body),
+ "ProvisioningRoleArn": "arn:aws:iam::000000000000:role/provisioning"}}}}))
+ try:
+ assert _wait_stack(cfn, stack_name)["StackStatus"] == "ROLLBACK_COMPLETE"
+ assert ("The template body is invalid: A certificate resource must specify a Status"
+ in _stack_event_reasons(cfn, stack_name))
+ with pytest.raises(iot_client.exceptions.ResourceNotFoundException):
+ iot_client.describe_provisioning_template(templateName=name)
+ finally:
+ _delete_cfn_test_stack(cfn, stack_name)
+
+
def test_cfn_iot_provisioning_template_rename_replaces(cfn, iot_client):
"""A TemplateName change replaces the template the way AWS::IoT::Policy's
rename does: new physical id, old template gone."""
@@ -27718,6 +27773,65 @@ def test_cfn_iot_tags_follow_the_template(cfn, iot_client):
assert listed() == [{}] * 3
+def test_cfn_iot_stack_tags_reach_the_tagged_types(cfn, iot_client):
+ """Stack tags reach every tagged IoT type but JobTemplate, yield to a same-key template tag, and update in place."""
+ pytest.importorskip("cryptography")
+ from ministack.core.x509_utils import generate_ca
+
+ uid = _uuid_mod.uuid4().hex[:8]
+ stack_name = f"cfn-iot-stacktags-{uid}"
+ ca_pem, _key = generate_ca(common_name=f"cfn-stacktags-{uid}")
+ own = [{"Key": "own", "Value": "1"}]
+ body = json.dumps({"Resources": {
+ "Type": {"Type": "AWS::IoT::ThingType", "Properties": {"ThingTypeName": f"cfn-st-type-{uid}", "Tags": own}},
+ "Group": {"Type": "AWS::IoT::ThingGroup", "Properties": {
+ "ThingGroupName": f"cfn-st-group-{uid}", "Tags": own + [{"Key": "shared", "Value": "own"}]}},
+ "Policy": {"Type": "AWS::IoT::Policy", "Properties": {
+ "PolicyName": f"cfn-st-policy-{uid}", "Tags": own, "PolicyDocument": {
+ "Version": "2012-10-17",
+ "Statement": [{"Effect": "Deny", "Action": "iot:Publish", "Resource": "*"}]}}},
+ "Rule": {"Type": "AWS::IoT::TopicRule", "Properties": {
+ "RuleName": f"cfn_st_rule_{uid}", "Tags": own,
+ "TopicRulePayload": {"Sql": "SELECT * FROM 't'", "Actions": [], "RuleDisabled": True}}},
+ "Ca": {"Type": "AWS::IoT::CACertificate", "Properties": {
+ "CACertificatePem": ca_pem, "CertificateMode": "SNI_ONLY", "Status": "INACTIVE", "Tags": own}},
+ "Pt": {"Type": "AWS::IoT::ProvisioningTemplate", "Properties": {
+ "TemplateName": f"cfn-st-pt-{uid}", "TemplateBody": _cfn_provisioning_template_body(),
+ "ProvisioningRoleArn": "arn:aws:iam::000000000000:role/provisioning"}},
+ "Jt": {"Type": "AWS::IoT::JobTemplate", "Properties": {
+ "JobTemplateId": f"cfn-st-jt-{uid}", "Description": "d", "Document": "{}", "Tags": own}},
+ }, "Outputs": {"Ca": {"Value": {"Ref": "Ca"}}}})
+
+ def tags_by_resource():
+ return [{t["Key"]: t["Value"] for t in iot_client.list_tags_for_resource(resourceArn=arn)["tags"]}
+ for arn in arns]
+
+ cfn.create_stack(StackName=stack_name, TemplateBody=body,
+ Tags=[{"Key": "stacktag", "Value": "s"}, {"Key": "shared", "Value": "stack"}])
+ try:
+ stack = _wait_stack(cfn, stack_name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ prefix = "arn:aws:iot:us-east-1:000000000000:"
+ arns = [f"{prefix}thingtype/cfn-st-type-{uid}", f"{prefix}thinggroup/cfn-st-group-{uid}",
+ f"{prefix}policy/cfn-st-policy-{uid}", f"{prefix}rule/cfn_st_rule_{uid}",
+ f"{prefix}cacert/{_output(stack, 'Ca')}", f"{prefix}provisioningtemplate/cfn-st-pt-{uid}",
+ f"{prefix}jobtemplate/cfn-st-jt-{uid}"]
+ stacked = {"own": "1", "shared": "stack", "stacktag": "s"}
+ assert tags_by_resource() == [stacked, {**stacked, "shared": "own"}, stacked, stacked, stacked,
+ {"shared": "stack", "stacktag": "s"}, {"own": "1"}]
+
+ cfn.update_stack(StackName=stack_name, TemplateBody=body,
+ Tags=[{"Key": "stacktag", "Value": "t"}, {"Key": "added", "Value": "1"}])
+ stack = _wait_stack(cfn, stack_name)
+ assert stack["StackStatus"] == "UPDATE_COMPLETE", stack.get("StackStatusReason")
+ stacked = {"own": "1", "stacktag": "t", "added": "1"}
+ assert tags_by_resource() == [stacked, {**stacked, "shared": "own"}, stacked, stacked, stacked,
+ {"stacktag": "t", "added": "1"}, {"own": "1"}]
+ assert len(iot_client.list_policy_versions(policyName=f"cfn-st-policy-{uid}")["policyVersions"]) == 1
+ finally:
+ _delete_cfn_test_stack(cfn, stack_name)
+
+
def test_cfn_kms_alias_target_change_in_place_and_rename_replaces(cfn, kms_client):
"""A TargetKeyId change re-points the alias under its name (Ref keeps its
value), an AliasName change replaces the alias — the new name exists, the
diff --git a/tests/test_iot.py b/tests/test_iot.py
index df98b9ea..c2c4ec30 100644
--- a/tests/test_iot.py
+++ b/tests/test_iot.py
@@ -1903,7 +1903,8 @@ def _provisioning_template_body() -> str:
"Resources": {
"certificate": {
"Type": "AWS::IoT::Certificate",
- "Properties": {"CertificateId": {"Ref": "AWS::IoT::Certificate::Id"}},
+ "Properties": {"CertificateId": {"Ref": "AWS::IoT::Certificate::Id"},
+ "Status": "ACTIVE"},
},
"thing": {
"Type": "AWS::IoT::Thing",
@@ -2009,6 +2010,56 @@ def test_iot_provisioning_template_body_must_carry_certificate_resource(iot_clie
iot_client.describe_provisioning_template(templateName=name)
+def test_iot_provisioning_template_body_checks_the_certificate_resource(iot_client):
+ """The template body's certificate resource needs a CertificateId reference and an Active or Inactive Status."""
+ role = "arn:aws:iam::000000000000:role/provisioning"
+ cert_id = {"CertificateId": {"Ref": "AWS::IoT::Certificate::Id"}}
+ status_msg = 'A certificate resource must specify a Status of either "Active" or "Inactive"'
+ id_msg = "CertificateId must contain a reference to AWS::IoT::Certificate::Id"
+ thing = {"Type": "AWS::IoT::Thing", "Properties": {"ThingName": "t"}}
+
+ def cert(**props):
+ return {"Type": "AWS::IoT::Certificate", "Properties": props}
+
+ cases = [
+ ("FLEET_PROVISIONING", {"c": cert(**cert_id)}, status_msg),
+ ("JITP", {"c": cert(**cert_id)}, status_msg),
+ ("FLEET_PROVISIONING", {"c": cert(**cert_id, Status="PENDING_ACTIVATION")}, status_msg),
+ ("FLEET_PROVISIONING", {"c": cert(**cert_id, Status="")}, status_msg),
+ ("FLEET_PROVISIONING", {"c": cert(**cert_id, Status=None)}, status_msg),
+ ("FLEET_PROVISIONING", {"c": cert(**cert_id, Status={"Ref": "S"})}, status_msg),
+ ("JITP", {"c": cert()}, id_msg),
+ ("FLEET_PROVISIONING", {"c": cert(CertificateId="abc", Status="ACTIVE")}, id_msg),
+ ("FLEET_PROVISIONING", {"c": cert(CertificateSigningRequest={"Ref": "Csr"}, Status="ACTIVE")}, id_msg),
+ ("FLEET_PROVISIONING", {"c": {"Type": "AWS::IoT::Certificate"}},
+ "Properties of a resource cannot be null."),
+ ("FLEET_PROVISIONING", {"c": cert(**cert_id, Status="ACTIVE"),
+ "t": {"Type": "AWS::IoT::Thing", "Properties": None}},
+ "Properties of a resource cannot be null."),
+ ("FLEET_PROVISIONING", {"a": cert(**cert_id, Status="ACTIVE"), "b": cert(**cert_id)},
+ "Only one certificate resource can be defined"),
+ ("FLEET_PROVISIONING", {"a": cert(**cert_id), "b": cert(**cert_id, Status="ACTIVE")}, status_msg),
+ ]
+ for kind, resources, message in cases:
+ name = _unique("provtemplate")
+ with pytest.raises(ClientError) as ei:
+ iot_client.create_provisioning_template(
+ templateName=name, templateBody=json.dumps({"Resources": resources}),
+ provisioningRoleArn=role, type=kind)
+ assert ei.value.response["Error"]["Code"] == "InvalidRequestException"
+ assert ei.value.response["Error"]["Message"] == f"The template body is invalid: {message}", resources
+ with pytest.raises(ClientError):
+ iot_client.describe_provisioning_template(templateName=name)
+
+ for kind, status in (("FLEET_PROVISIONING", "active"), ("JITP", "Inactive"),
+ ("FLEET_PROVISIONING", "AcTiVe")):
+ name = _unique("provtemplate")
+ iot_client.create_provisioning_template(
+ templateName=name, provisioningRoleArn=role, type=kind,
+ templateBody=json.dumps({"Resources": {"c": cert(**cert_id, Status=status), "t": thing}}))
+ iot_client.delete_provisioning_template(templateName=name)
+
+
def test_iot_provisioning_template_name_validation(iot_client):
role = "arn:aws:iam::000000000000:role/provisioning"
for bad in ("x" * 37, "no:colons"):