Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
feat(cloudformation): Rules section, template quotas, AWS::Include (#1661)
iot-rocketd64d603655ac
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 90fe0aef..d2684d4b 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,12 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]+### Added+- **CloudFormation — the `Rules` section is evaluated** — a template's rules were ignored; they now run after the parameters resolve and before any resource is touched, on `CreateStack`, `UpdateStack` and `CreateChangeSet`: a rule whose `RuleCondition` is false is skipped, a false `Assert` refuses the operation with its `AssertDescription` as a `ValidationError`, and the CDK's `CheckBootstrapVersion` rule therefore refuses an outdated bootstrap version instead of passing silently. `Fn::And`, `Fn::Or`, `Fn::Not`, `Fn::Equals`, `Fn::If`, `Fn::Contains`, `Fn::EachMemberEquals`, `Fn::EachMemberIn` and `Ref` are evaluated; `Fn::RefAll`, `Fn::ValueOf` and `Fn::ValueOfAll` are served from the EC2 store for the VPC, subnet and security-group id types, and a rule over any other type is skipped with a warning. A function outside the rule set is refused as a template format error. Contributed by @iot-rocket.+- **CloudFormation — the `AWS::Include` transform** — an embedded `Fn::Transform` naming `AWS::Include` was carried into the stack as a literal key; it is now replaced by the contents of the S3 object its `Location` points to (JSON or plain YAML, a key-value object) before the template is validated: next to a sibling resource inside `Resources`, as a whole `Properties` map, inside one next to other properties, in `Mappings`, or in `Outputs`. A location that is not an `s3://` URI, a snippet with a YAML shorthand tag, a missing object, a snippet that is not an object and a nested include are refused before a stack exists, with the sentence a real account answers where one is known. `UsePreviousTemplate` and a change set without a body reuse the stored processed template, so a snippet edited in S3 since the deploy is not picked up, as on AWS. Contributed by @iot-rocket.++### Fixed+- **CloudFormation — the template and stack quotas are enforced** — a template of any size, with any number of resources, parameters, outputs or mappings, and a stack name of any shape were accepted, so a template that a real account refuses deployed. `CreateStack`, `UpdateStack`, `CreateChangeSet`, `ValidateTemplate` and `GetTemplateSummary` now refuse a `TemplateBody` over 51,200 bytes and a template behind `TemplateURL` over 1,000,000 bytes; the engine refuses more than 500 resources, 200 parameters, 200 outputs, 200 mappings or 200 attributes in a mapping, a name over 255 characters, a description over 1,024 bytes and a parameter default or value over 4,096 bytes; `CreateStack` and a `CREATE` change set refuse a stack name that is not alphanumeric-and-hyphens starting with a letter or longer than 128 characters, joined with an oversized body into the one message the API's parameter validation gives. All of it before a stack record exists, with the messages of the API's parameter validation and the template format errors. `ValidateTemplate` also takes `TemplateURL` now. Contributed by @iot-rocket.### Fixed- **CloudFormation — `Capabilities` are enforced under `AUTH=true`** — `CreateStack`, `UpdateStack` and `CreateChangeSet` accepted a template with IAM resources or a `Transform` whatever the request acknowledged, so a deploy that CloudFormation refuses with `InsufficientCapabilitiesException` went through. With `AUTH=true` they now answer that error (HTTP 400, `Requires capabilities : [CAPABILITY_IAM]`, measured) and create nothing: an IAM resource needs `CAPABILITY_IAM` or `CAPABILITY_NAMED_IAM`, one with a custom name `CAPABILITY_NAMED_IAM`, a template with a `Transform` `CAPABILITY_AUTO_EXPAND` on `CreateStack` and `UpdateStack` (not on a change set, as the API documents). Without `AUTH` nothing changes; `GetTemplateSummary` keeps reporting the same set. Contributed by @iot-rocket.diff --git a/README.md b/README.mdindex b5484780..bf705c93 100644--- a/README.md+++ b/README.md@@ -460,8 +460,11 @@ subnet = ec2.create_subnet(| **Parameters** | Default values, AllowedValues validation, NoEcho masking, String/Number/CommaDelimitedList types || **Stack Tags** | The `Tags` of CreateStack, UpdateStack and change sets reach every resource whose type carries a tag property, together with `aws:cloudformation:stack-name`, `stack-id` and `logical-id`; a key the template sets wins || **Conditions** | Fn::Equals, Fn::And, Fn::Or, Fn::Not — conditional resource creation |+| **Rules** | The `Rules` section runs after the parameters resolve and before any resource is touched (CreateStack, UpdateStack, CreateChangeSet): `RuleCondition`, `Assertions` with `AssertDescription`, Fn::And/Or/Not/Equals/If/Contains/EachMemberEquals/EachMemberIn and Ref; Fn::RefAll, Fn::ValueOf and Fn::ValueOfAll read the EC2 store for VPC, subnet and security-group ids |+| **Transforms** | `AWS::Serverless-2016-10-31` (SAM, in the `full` image) and the embedded `AWS::Include` transform (`Fn::Transform` with an `s3://` `Location`, JSON or YAML snippets) || **Rollback** | Configurable via `DisableRollback` — on failure, previously created resources are cleaned up in reverse dependency order || **Async Status** | Stacks deploy asynchronously (`CREATE_IN_PROGRESS` → `CREATE_COMPLETE`) — poll with DescribeStacks |+| **Quotas** | The template and stack quotas of the CloudFormation quotas page are enforced before a stack exists: 51,200 bytes of `TemplateBody`, 1,000,000 bytes behind `TemplateURL`, 500 resources, 200 parameters / outputs / mappings, 200 attributes per mapping, 255-character names, a 1,024-byte description, 4,096-byte parameter values, and a stack name of up to 128 alphanumeric characters and hyphens starting with a letter |**Supported Resource Types:**diff --git a/ministack/services/cloudformation/changesets.py b/ministack/services/cloudformation/changesets.pyindex 3cf81a03..66eae044 100644--- a/ministack/services/cloudformation/changesets.py+++ b/ministack/services/cloudformation/changesets.py@@ -17,7 +17,16 @@ from .engine import (_resolve_refs,validate_template_support,)-from .helpers import _error, _esc, _extract_members, _p, _page, _resolve_template, _xml+from .helpers import (+ _error,+ _esc,+ _extract_members,+ _p,+ _page,+ _request_problems,+ _resolve_template,+ _xml,+)from .stacks import (_add_event,_create_stack_task_in_region,@@ -99,9 +108,15 @@ def _create_change_set(params):return _error("AlreadyExistsException",f"ChangeSet [{cs_name}] already exists")+ if cs_type == "CREATE":+ # The request-level constraints of a new stack, joined as the API does.+ if request_error := _request_problems(params, stack_name):+ return request_error+template_body, resolve_err = _resolve_template(params)if resolve_err:return resolve_err+ template_given = bool(template_body)provided_params = _extract_members(params, "Parameters")tags = _extract_members(params, "Tags")@@ -198,9 +213,13 @@ def _create_change_set(params):except ValueError as exc:return _rejected(str(exc))+ if cs_type == "UPDATE" and not template_given and stack.get("_template"):+ # As UpdateStack with UsePreviousTemplate: the stored processed+ # template, so an AWS::Include snippet changed in S3 is not picked up.+ template = copy.deepcopy(stack["_template"])try:validate_template_support(- template, _evaluate_conditions(template, param_values))+ template, _evaluate_conditions(template, param_values), params=param_values)except ValueError as exc:return _rejected(str(exc))diff --git a/ministack/services/cloudformation/engine.py b/ministack/services/cloudformation/engine.pyindex 6e8dd834..81ae9d0b 100644--- a/ministack/services/cloudformation/engine.py+++ b/ministack/services/cloudformation/engine.py@@ -20,6 +20,8 @@ import yamlfrom ministack.core.responses import get_account_id, get_region+logger = logging.getLogger("cloudformation")+# Sentinel for AWS::NoValue_NO_VALUE = object()@@ -233,7 +235,8 @@ def _resolve_dynamic_references(value, previous: dict | None = None,return walk(value), resolved-def validate_template_support(template: dict, conditions: dict) -> None:+def validate_template_support(template: dict, conditions: dict,+ params: dict | None = None) -> None:"""Reject up front what provisioning could only fail on halfway through.Real CloudFormation validates resource types before it touches anything:@@ -257,9 +260,16 @@ def validate_template_support(template: dict, conditions: dict) -> None:and they pass. CreateStack, UpdateStack and CreateChangeSet apply the SAMtransform before calling this, so their expanded templates are validatedas usual.++ ``params`` (the resolved ``{name: {Value, NoEcho}}`` map) turns on the+ ``Rules`` section: CreateStack, UpdateStack and CreateChangeSet pass it,+ ValidateTemplate (no parameter values) does not."""from .provisioners import _RESOURCE_HANDLERS+ if isinstance(template, _IncludeFailed):+ raise ValueError(template.error)+ _validate_template_limits(template)unrecognized: set[str] = set()if not template.get("Transform"):for res in (template.get("Resources") or {}).values():@@ -294,6 +304,94 @@ def validate_template_support(template: dict, conditions: dict) -> None:+ ", ".join(unsupported)+ " (supported: ssm, ssm-secure, secretsmanager)")+ if params is not None:+ _evaluate_rules(template, params, conditions)+++# ===========================================================================+# Template and stack quotas+# ===========================================================================++# The CloudFormation quotas page (cloudformation-limits.html): section sizes,+# name lengths, the template description and the parameter value.+_SECTION_QUOTAS = (+ ("Resources", "resources", "Resource", 500),+ ("Parameters", "parameters", "Parameter", 200),+ ("Outputs", "outputs", "Output", 200),+ ("Mappings", "mappings", "Mapping", 200),+)+_MAPPING_ATTRIBUTES_MAX = 200+_NAME_MAX_CHARS = 255+_DESCRIPTION_MAX_BYTES = 1024+_PARAMETER_VALUE_MAX_BYTES = 4096+++def _quota_error(noun: str, count: int, maximum: int) -> ValueError:+ # The sentence a real account answers for too many resources ("Template+ # format error: Number of resources, 536, is greater than maximum allowed,+ # 500"); the other sections reuse it, unmeasured.+ return ValueError(+ f"Template format error: Number of {noun}, {count}, is greater than "+ f"maximum allowed, {maximum}")+++def _validate_template_limits(template: dict) -> None:+ """The template quotas that need nothing but the template: 500 resources,+ 200 parameters, outputs and mappings, 200 attributes per mapping, names+ of 255 characters, a description of 1,024 bytes and a parameter default+ of 4,096 bytes. Raises ``ValueError`` with the message the caller wraps+ as a ``ValidationError``; the wordings other than the resource count and+ the parameter default are unmeasured."""+ for section, noun, singular, maximum in _SECTION_QUOTAS:+ members = template.get(section) or {}+ if not isinstance(members, dict):+ continue+ if len(members) > maximum:+ raise _quota_error(noun, len(members), maximum)+ for name in members:+ if len(str(name)) > _NAME_MAX_CHARS:+ raise ValueError(+ f"Template format error: {singular} name {str(name)[:32]}... may "+ f"not exceed {_NAME_MAX_CHARS} characters")+ for name, mapping in (template.get("Mappings") or {}).items():+ if not isinstance(mapping, dict):+ continue+ if len(mapping) > _MAPPING_ATTRIBUTES_MAX:+ raise _quota_error(f"attributes in mapping {name}", len(mapping),+ _MAPPING_ATTRIBUTES_MAX)+ for attribute in mapping:+ if len(str(attribute)) > _NAME_MAX_CHARS:+ raise ValueError(+ f"Template format error: Mapping attribute name {str(attribute)[:32]}"+ f"... of mapping {name} may not exceed {_NAME_MAX_CHARS} characters")+ description = template.get("Description")+ if isinstance(description, str) and len(description.encode("utf-8")) > _DESCRIPTION_MAX_BYTES:+ raise ValueError(+ f"Template format error: Template description may not exceed "+ f"{_DESCRIPTION_MAX_BYTES} bytes in size")+ for name, defn in (template.get("Parameters") or {}).items():+ default = defn.get("Default") if isinstance(defn, dict) else None+ if default is not None and len(str(default).encode("utf-8")) > _PARAMETER_VALUE_MAX_BYTES:+ # Quoted from a report of the error ("Template format error:+ # Parameter 'EnvironmentVariables' default value '[****]' length is+ # greater than 4096.").+ raise ValueError(+ f"Template format error: Parameter '{name}' default value '{default}' "+ f"length is greater than {_PARAMETER_VALUE_MAX_BYTES}.")+++def _check_provided_parameter_values(provided_params: list[dict]) -> None:+ """The request-level constraint on a parameter value (4,096 bytes): the+ message follows the API's parameter validation, with the 1-based member+ index of the request; unmeasured."""+ for index, entry in enumerate(provided_params, 1):+ value = entry.get("Value", "")+ if value is not None and len(str(value).encode("utf-8")) > _PARAMETER_VALUE_MAX_BYTES:+ raise ValueError(+ f"1 validation error detected: Value '{value}' at "+ f"'parameters.{index}.member.parameterValue' failed to satisfy "+ "constraint: Member must have length less than or equal to "+ f"{_PARAMETER_VALUE_MAX_BYTES}")# ===========================================================================@@ -379,6 +477,7 @@ def _resolve_parameters(template: dict, provided_params: list[dict],Returns dict of param_name -> {Value, NoEcho}."""+ _check_provided_parameter_values(provided_params)param_defs = template.get("Parameters", {})provided_map = {p["Key"]: p for p in provided_params if "Key" in p}previous_params = previous_params or {}@@ -536,6 +635,313 @@ def _evaluate_conditions(template: dict, params: dict) -> dict:return evaluated+# ===========================================================================+# Rules section+# ===========================================================================++# The functions the Rules section accepts (rules-section-structure.html lists+# them; Ref may be nested in all but Fn::ValueOf and Fn::ValueOfAll).+_RULE_FUNCTIONS = frozenset({+ "Fn::And", "Fn::Or", "Fn::Not", "Fn::Equals", "Fn::If", "Fn::Contains",+ "Fn::EachMemberEquals", "Fn::EachMemberIn", "Fn::RefAll", "Fn::ValueOf",+ "Fn::ValueOfAll", "Ref",+})++# Fn::RefAll / Fn::ValueOf / Fn::ValueOfAll read the account: the three+# parameter types whose attributes the reference documents are served from the+# in-process EC2 store, keyed by the attribute names the Rules reference lists.+_RULE_ACCOUNT_TYPES = {+ "AWS::EC2::VPC::Id": ("_vpcs", {"DefaultNetworkAcl": "DefaultNetworkAclId",+ "DefaultSecurityGroup": "DefaultSecurityGroupId"}),+ "AWS::EC2::Subnet::Id": ("_subnets", {"AvailabilityZone": "AvailabilityZone",+ "VpcId": "VpcId"}),+ "AWS::EC2::SecurityGroup::Id": ("_security_groups", {}),+}+++class _RuleUnsupported(Exception):+ """A rule needs an account lookup this emulator does not serve; the rule is+ skipped with a warning instead of failing the stack operation."""+++# The argument list each rule function takes (intrinsic-function-reference-rules):+# ``Fn::Not`` one condition, ``Fn::If`` a condition and two values, the rest+# two elements; ``Fn::And`` / ``Fn::Or`` any list, ``Ref`` / ``Fn::RefAll`` a name.+_RULE_ARITY = {+ "Fn::Equals": 2, "Fn::Contains": 2, "Fn::EachMemberEquals": 2,+ "Fn::EachMemberIn": 2, "Fn::ValueOf": 2, "Fn::ValueOfAll": 2,+ "Fn::Not": 1, "Fn::If": 3,+}+++def _check_rule_expression(node, rule: str) -> None:+ """Arity and argument types of every rule function under ``rule``: a+ malformed argument is a template format error (unmeasured wording), not+ an evaluation crash. ``Fn::ValueOf`` and ``Fn::ValueOfAll`` take two+ strings: the reference says no other function can be used within them."""+ if isinstance(node, list):+ for value in node:+ _check_rule_expression(value, rule)+ return+ if not isinstance(node, dict):+ return+ for fn, args in node.items():+ if fn in ("Ref", "Fn::RefAll"):+ if not isinstance(args, str):+ raise ValueError(+ f"Template format error: {fn} in rule {rule} must name a "+ + ("parameter" if fn == "Ref" else "parameter type"))+ continue+ if fn in ("Fn::And", "Fn::Or"):+ if not isinstance(args, list) or not args:+ raise ValueError(+ f"Template format error: {fn} in rule {rule} must be a list of "+ "conditions")+ elif fn in _RULE_ARITY:+ if not isinstance(args, list) or len(args) != _RULE_ARITY[fn]:+ raise ValueError(+ f"Template format error: {fn} in rule {rule} must be a list of "+ f"{_RULE_ARITY[fn]} elements")+ if fn in ("Fn::ValueOf", "Fn::ValueOfAll") and not all(+ isinstance(a, str) for a in args):+ raise ValueError(+ f"Template format error: {fn} in rule {rule} takes two strings; "+ "no function can be used within it")+ _check_rule_expression(args, rule)+++def _rule_functions_used(node, found: set) -> None:+ if isinstance(node, dict):+ for key, value in node.items():+ if key.startswith("Fn::") or key == "Ref":+ found.add(key)+ _rule_functions_used(value, found)+ elif isinstance(node, list):+ for value in node:+ _rule_functions_used(value, found)+++def _check_rules_section(template: dict) -> None:+ """The shape checks that need nothing but the template: every rule carries+ an ``Assertions`` list, every assertion an ``Assert``, only the rule+ functions appear, and every function has the arguments it takes. The+ function message is measured (a real account answered ``Template format+ error: Following functions are not supported in the Rules block of the+ template: [Fn::If]``, cloudformation-coverage-roadmap issue 921, 2021;+ the bracket holds the offending names sorted and comma-joined, the join+ unmeasured); the shape messages are unmeasured."""+ rules = template.get("Rules")+ if rules is None:+ return+ if not isinstance(rules, dict):+ raise ValueError("Template format error: Rules must be a map of rule name to rule")+ for name, rule in rules.items():+ if not isinstance(rule, dict) or not isinstance(rule.get("Assertions"), list):+ raise ValueError(+ f"Template format error: Rule {name} must contain an Assertions list")+ for assertion in rule["Assertions"]:+ if not isinstance(assertion, dict) or "Assert" not in assertion:+ raise ValueError(+ f"Template format error: Every assertion of rule {name} must "+ "contain an Assert")+ used: set[str] = set()+ _rule_functions_used(rules, used)+ unsupported = sorted(used - _RULE_FUNCTIONS)+ if unsupported:+ raise ValueError(+ "Template format error: Following functions are not supported in the "+ "Rules block of the template: [" + ", ".join(unsupported) + "]")+ for name, rule in rules.items():+ _check_rule_expression(rule.get("RuleCondition"), name)+ for assertion in rule["Assertions"]:+ _check_rule_expression(assertion["Assert"], name)+++def _rule_inner_type(ptype: str) -> tuple[str, bool]:+ """The type a parameter's declared type wraps and whether it is a list:+ the SSM ``Value<...>`` wrapper is stripped first, then ``List<...>``+ (a ``CommaDelimitedList`` is a list of strings)."""+ if ptype.startswith(_SSM_PARAMETER_VALUE_PREFIX) and ptype.endswith(">"):+ ptype = ptype[len(_SSM_PARAMETER_VALUE_PREFIX):-1]+ if ptype == "CommaDelimitedList":+ return "String", True+ if ptype.startswith("List<") and ptype.endswith(">"):+ return ptype[len("List<"):-1], True+ return ptype, False+++def _rule_param_value(name: str, defn: dict, params: dict):+ """A parameter as ``Ref`` sees it inside Rules: list-typed parameters are+ lists of their trimmed members, everything else the string value."""+ value = params[name]["Value"]+ _, is_list = _rule_inner_type(str(defn.get("Type", "String")))+ if is_list:+ return [m.strip() for m in str(value).split(",")] if value != "" else []+ return str(value)+++def _rule_account_records(parameter_type: str):+ """The EC2 records behind an AWS-specific parameter type, or+ ``_RuleUnsupported`` for a type this emulator does not list."""+ spec = _RULE_ACCOUNT_TYPES.get(parameter_type)+ if spec is None:+ raise _RuleUnsupported(+ f"Fn::RefAll / Fn::ValueOfAll over {parameter_type} is not supported")+ from ministack.services import ec2+ store = getattr(ec2, spec[0])+ return {rid: store[rid] for rid in list(store)}, spec[1]+++def _rule_attribute(parameter_type: str, resource_id: str, attribute: str) -> str:+ """One attribute of one account resource, as ``Fn::ValueOf`` returns it;+ a resource the store does not hold yields an empty string."""+ from ministack.services import ec2+ records, attributes = _rule_account_records(parameter_type)+ record = records.get(resource_id) or {}+ if attribute.startswith("Tags."):+ key = attribute[len("Tags."):]+ for tag in ec2._tags.get(resource_id, []) or []:+ if tag.get("Key") == key:+ return str(tag.get("Value", ""))+ return ""+ field = attributes.get(attribute)+ if field is None:+ raise _RuleUnsupported(+ f"attribute {attribute} of {parameter_type} is not supported")+ return str(record.get(field, ""))+++def _evaluate_rules(template: dict, params: dict, conditions: dict) -> None:+ """Evaluate the ``Rules`` section against the resolved parameters.++ The Rules reference: a rule's ``Assertions`` are checked when its+ ``RuleCondition`` is absent or evaluates to true; every ``Assert`` must+ evaluate to true or the stack is neither created nor updated, and the+ ``AssertDescription`` is the message. ``Fn::RefAll``, ``Fn::ValueOf`` and+ ``Fn::ValueOfAll`` are served for ``AWS::EC2::VPC::Id``,+ ``AWS::EC2::Subnet::Id`` and ``AWS::EC2::SecurityGroup::Id`` from the+ in-process EC2 store; a rule over any other type is skipped with a+ warning. Raises ``ValueError`` with the message the caller wraps as a+ ``ValidationError`` (the failure text is unmeasured).+ """+ rules = template.get("Rules")+ if not rules:+ return+ param_defs = template.get("Parameters") or {}+ pseudo = {+ "AWS::Region": get_region(),+ "AWS::AccountId": get_account_id(),+ "AWS::URLSuffix": "amazonaws.com",+ "AWS::Partition": "aws",+ }++ def as_list(value, fn):+ if not isinstance(value, list):+ raise ValueError(+ f"Template error: {fn} expects a list as its first argument, got "+ f"'{value}'")+ return [str(v) for v in value]++ def as_bool(value, fn):+ if not isinstance(value, bool):+ raise ValueError(+ f"Template error: every argument of {fn} must evaluate to true or "+ f"false, got '{value}'")+ return value++ def ev(node):+ if not isinstance(node, dict) or len(node) != 1:+ return [ev(v) for v in node] if isinstance(node, list) else node+ fn, args = next(iter(node.items()))+ if fn == "Ref":+ if args in params:+ return _rule_param_value(args, param_defs.get(args, {}), params)+ if args in pseudo:+ return pseudo[args]+ raise ValueError(+ f"Template format error: Unresolved resource dependencies [{args}] "+ "in the Rules block of the template")+ if fn == "Fn::Equals":+ left, right = ev(args[0]), ev(args[1])+ if isinstance(left, list) or isinstance(right, list):+ return left == right+ return str(left) == str(right)+ if fn == "Fn::And":+ return all(as_bool(ev(a), fn) for a in args)+ if fn == "Fn::Or":+ return any(as_bool(ev(a), fn) for a in args)+ if fn == "Fn::Not":+ return not as_bool(ev(args[0]), fn)+ if fn == "Fn::If":+ chosen = ev(args[0])+ if isinstance(args[0], str):+ if args[0] not in conditions:+ raise ValueError(+ f"Template error: Fn::If refers to condition {args[0]} "+ "which is not defined in the Conditions block")+ chosen = conditions[args[0]]+ return ev(args[1]) if as_bool(chosen, fn) else ev(args[2])+ if fn == "Fn::Contains":+ return str(ev(args[1])) in as_list(ev(args[0]), fn)+ if fn == "Fn::EachMemberEquals":+ wanted = str(ev(args[1]))+ return all(m == wanted for m in as_list(ev(args[0]), fn))+ if fn == "Fn::EachMemberIn":+ allowed = as_list(ev(args[1]), fn)+ return all(m in allowed for m in as_list(ev(args[0]), fn))+ if fn == "Fn::RefAll":+ records, _ = _rule_account_records(str(args))+ return list(records)+ if fn == "Fn::ValueOfAll":+ ptype, attribute = str(args[0]), str(args[1])+ records, _ = _rule_account_records(ptype)+ return [_rule_attribute(ptype, rid, attribute) for rid in records]+ if fn == "Fn::ValueOf":+ pname, attribute = str(args[0]), str(args[1])+ if pname not in params:+ raise ValueError(+ f"Template format error: Unresolved resource dependencies "+ f"[{pname}] in the Rules block of the template")+ base, _ = _rule_inner_type(str(param_defs.get(pname, {}).get("Type", "String")))+ value = _rule_param_value(pname, param_defs.get(pname, {}), params)+ if isinstance(value, list):+ return [_rule_attribute(base, v, attribute) for v in value]+ return _rule_attribute(base, value, attribute)+ raise ValueError(+ "Template format error: Following functions are not supported in the "+ f"Rules block of the template: [{fn}]")++ for name, rule in rules.items():+ try:+ condition = rule.get("RuleCondition")+ if condition is not None:+ chosen = ev(condition)+ if not isinstance(chosen, bool):+ raise ValueError(+ f"Template error: the RuleCondition of rule {name} must "+ f"evaluate to true or false, got '{chosen}'")+ if not chosen:+ continue+ for index, assertion in enumerate(rule["Assertions"], 1):+ result = ev(assertion["Assert"])+ if not isinstance(result, bool):+ raise ValueError(+ f"Template error: assertion {index} of rule {name} must "+ f"evaluate to true or false, got '{result}'")+ if not result:+ description = assertion.get("AssertDescription") \+ or f"assertion {index} evaluated to false"+ raise ValueError(f"Template error: rule {name} failed: {description}")+ except _RuleUnsupported as exc:+ logger.warning("Rules: skipping rule %s: %s", name, exc)+ except (TypeError, IndexError, KeyError) as exc:+ # The shape check above refuses what it knows; anything that still+ # does not evaluate is a template error, never an internal one.+ raise ValueError(+ f"Template format error: rule {name} could not be evaluated: "+ f"{exc}") from exc++# ===========================================================================# Intrinsic Function Resolver# ===========================================================================@@ -602,6 +1008,7 @@ def _validate_template_statics(template: dict) -> None:for section in ("Resources", "Outputs", "Conditions"):walk(template.get(section) or {})+ _check_rules_section(template)def _unknown_attribute_message(res: dict, logical_id: str, attr: str) -> str:@@ -999,6 +1406,146 @@ def _topological_sort(resources: dict, conditions: dict) -> list:return result+# ===========================================================================+# AWS::Include transform+# ===========================================================================++_INCLUDE_TRANSFORM = "AWS::Include"+_INCLUDE_ERROR = "Transform AWS::Include failed with: "+# The reference: the transform may be used anywhere except the Parameters+# section and the template version; the top-level Transform form is not+# expanded here.+_INCLUDE_SKIPPED_SECTIONS = ("Parameters", "AWSTemplateFormatVersion", "Transform")+++class _IncludeError(ValueError):+ """An ``AWS::Include`` that could not be expanded; the message is what the+ caller answers, unwrapped."""+++class _IncludeFailed(dict):+ """The template as sent, carrying the message of an include that failed.++ The handlers apply the transforms in one step and wrap everything that+ step raises as ``Template format error: ...``; a real account answers+ the include sentence bare (``Transform AWS::Include failed with: ...``),+ so the failure travels with the template to ``validate_template_support``,+ which raises it as is. UpdateStack with ``UsePreviousTemplate`` swaps in+ the stored processed template before that point, so a snippet that went+ missing since the deploy does not fail the update."""++ def __init__(self, template: dict, error: str):+ super().__init__(template)+ self.error = error+++def _is_include(node) -> bool:+ """True for a map that carries an ``Fn::Transform`` naming ``AWS::Include``."""+ spec = node.get("Fn::Transform") if isinstance(node, dict) else None+ return isinstance(spec, dict) and spec.get("Name") == _INCLUDE_TRANSFORM+++def _include_location(node: dict):+ """The ``Location`` parameter of an include node, or None."""+ parameters = node["Fn::Transform"].get("Parameters")+ return parameters.get("Location") if isinstance(parameters, dict) else None+++def _parse_snippet(text: str):+ """A snippet is JSON or plain YAML: the reference says shorthand+ notations are not supported in snippets, and a real account answers the+ sentence below for a ``!Ref`` inside one (re:Post, "Include Parameters+ in an AWS Include file"), so the plain SafeLoader is used, not the+ template loader with its tag constructors."""+ text = text.strip()+ try:+ return json.loads(text) if text.startswith("{") \+ else yaml.load(text, Loader=yaml.SafeLoader)+ except Exception as exc:+ raise _IncludeError(+ _INCLUDE_ERROR + "The specified S3 object's content should be valid "+ "Yaml/JSON") from exc+++def _fetch_include_snippet(location) -> dict:+ """The key-value object behind an ``s3://bucket/key`` location. The+ location sentence and the content sentence are the ones a real account+ answers; the missing-object, key-value and nesting sentences are+ unmeasured."""+ if not isinstance(location, str) or not location.startswith("s3://"):+ raise _IncludeError(_INCLUDE_ERROR + "The location parameter is not a valid S3 uri.")+ bucket, _, key = location[len("s3://"):].partition("/")+ if not bucket or not key:+ raise _IncludeError(_INCLUDE_ERROR + "The location parameter is not a valid S3 uri.")+ from ministack.services import s3 as _s3+ data = _s3._get_object_data(bucket, key)+ if data is None:+ raise _IncludeError(_INCLUDE_ERROR + f"The S3 object {location} does not exist.")+ snippet = _parse_snippet(data.decode("utf-8"))+ if not isinstance(snippet, dict):+ raise _IncludeError(_INCLUDE_ERROR + f"The snippet at {location} must be a "+ "key-value object.")+ nested: list = []+ _find_includes(snippet, nested)+ if nested:+ raise _IncludeError(_INCLUDE_ERROR + f"The snippet at {location} uses AWS::Include, "+ "which cannot be nested.")+ return snippet+++def _find_includes(node, found: list) -> None:+ if isinstance(node, dict):+ if _is_include(node):+ found.append(node)+ for value in node.values():+ _find_includes(value, found)+ elif isinstance(node, list):+ for value in node:+ _find_includes(value, found)+++def _apply_include_transform(template: dict) -> dict:+ """Replace every embedded ``Fn::Transform`` that names ``AWS::Include``+ by the contents of the S3 object it points to, the way the transform+ reference describes: the snippet is inserted at the location of the+ transform. A node that holds nothing but the transform becomes the+ snippet; a node with other keys (``Resources`` next to a sibling+ resource, a ``Properties`` map) takes the snippet's keys, the snippet+ winning a duplicate key (unmeasured). The ``Parameters`` section and the+ template version are not walked. An include that cannot be expanded+ returns the template as ``_IncludeFailed``, which the validation step+ turns into the ``ValidationError``."""+ found: list = []+ for section, value in template.items():+ if section not in _INCLUDE_SKIPPED_SECTIONS:+ _find_includes(value, found)+ if not found:+ return template++ def walk(node):+ if isinstance(node, list):+ return [walk(v) for v in node]+ if not isinstance(node, dict):+ return node+ if _is_include(node):+ snippet = _fetch_include_snippet(_include_location(node))+ rest = {k: walk(v) for k, v in node.items() if k != "Fn::Transform"}+ if not rest:+ return snippet+ rest.update(snippet)+ return rest+ return {k: walk(v) for k, v in node.items()}++ expanded = dict(template)+ try:+ for section, value in template.items():+ if section not in _INCLUDE_SKIPPED_SECTIONS:+ expanded[section] = walk(value)+ except _IncludeError as exc:+ return _IncludeFailed(template, str(exc))+ return expanded++# ===========================================================================# SAM Transform# ===========================================================================@@ -1011,6 +1558,12 @@ logging.getLogger("samtranslator.feature_toggle.feature_toggle").setLevel(loggin_NO_IAM_POLICY_LOADER = SimpleNamespace(load=lambda: {})def _apply_sam_transform_if_applicable(template: dict) -> dict:+ """The transforms a stack operation applies before validation: every+ embedded ``AWS::Include`` first, then the SAM transform when the template+ declares it."""+ template = _apply_include_transform(template)+ if isinstance(template, _IncludeFailed):+ return templatedeclared = template.get("Transform")transforms = declared if isinstance(declared, list) else [declared]if _SAM_TRANSFORM not in transforms:diff --git a/ministack/services/cloudformation/handlers.py b/ministack/services/cloudformation/handlers.pyindex d38c7b26..cee04756 100644--- a/ministack/services/cloudformation/handlers.py+++ b/ministack/services/cloudformation/handlers.py@@ -36,6 +36,7 @@ from .helpers import (_extract_string_members,_p,_page,+ _request_problems,_resolve_template,_xml,)@@ -205,6 +206,9 @@ def _create_stack(params):stack_name = _p(params, "StackName")if not stack_name:return _error("ValidationError", "StackName is required")+ # The request-level constraints, joined into one message as the API does.+ if request_error := _request_problems(params, stack_name):+ return request_errortemplate_body, resolve_err = _resolve_template(params)if resolve_err:@@ -248,7 +252,7 @@ def _create_stack(params):conditions = _evaluate_conditions(template, param_values)try:- validate_template_support(template, conditions)+ validate_template_support(template, conditions, params=param_values)except ValueError as exc:return _error("ValidationError", str(exc))@@ -883,9 +887,15 @@ def _update_stack(params):except ValueError as exc:return _error("ValidationError", str(exc))+ if use_previous_template and stack.get("_template"):+ # The stored template is the processed one: an AWS::Include snippet+ # edited or removed in S3 since the deploy is not picked up (the+ # transform reference: "your stack doesn't automatically pick up+ # those changes").+ template = copy.deepcopy(stack["_template"])try:validate_template_support(- template, _evaluate_conditions(template, param_values))+ template, _evaluate_conditions(template, param_values), params=param_values)except ValueError as exc:return _error("ValidationError", str(exc))@@ -954,7 +964,9 @@ def _update_stack(params):# --- ValidateTemplate ---def _validate_template(params):- template_body = _p(params, "TemplateBody")+ template_body, resolve_err = _resolve_template(params)+ if resolve_err:+ return resolve_errif not template_body:return _error("ValidationError", "TemplateBody is required")diff --git a/ministack/services/cloudformation/helpers.py b/ministack/services/cloudformation/helpers.pyindex 1f96773e..6254a644 100644--- a/ministack/services/cloudformation/helpers.py+++ b/ministack/services/cloudformation/helpers.py@@ -5,6 +5,7 @@ CloudFormation helpers — XML response formatting and parameter extraction util"""import logging+import refrom html import escape as _escfrom urllib.parse import urlparse@@ -14,6 +15,16 @@ logger = logging.getLogger("cloudformation")CFN_NS = "http://cloudformation.amazonaws.com/doc/2010-05-08/"+# The CloudFormation quotas page: the template body of a request and of an+# S3 object behind TemplateURL. CreateStack documents the stack name as+# "only alphanumeric characters (case sensitive) and hyphens", starting with+# a letter and no longer than 128 characters.+TEMPLATE_BODY_MAX_BYTES = 51200+TEMPLATE_URL_MAX_BYTES = 1000000+STACK_NAME_MAX_CHARS = 128+STACK_NAME_PATTERN = "[a-zA-Z][-a-zA-Z0-9]*"+STACK_NAME_RE = re.compile(STACK_NAME_PATTERN)+def _p(params, key, default=""):"""Extract a single value from parsed query-string params."""@@ -88,10 +99,74 @@ def _validate_stack_tags(tags):+def validation_error_message(problems: list[str]) -> str:+ """The API's parameter validation joins every request-level violation into+ one message (quoted: ``2 validation errors detected: Value '' at+ 'stackName' failed to satisfy constraint: ...; Value '' at 'stackName'+ failed to satisfy constraint: ...``)."""+ plural = "s" if len(problems) > 1 else ""+ return f"{len(problems)} validation error{plural} detected: " + "; ".join(problems)+++def stack_name_problems(stack_name: str) -> list[str]:+ """The constraints CreateStack documents for ``StackName``: alphanumeric+ characters and hyphens, a letter first, at most 128 characters. One+ sentence per violated constraint, in the API's parameter validation+ wording (quoted for the pattern: ``Value 'x_y' at 'stackName' failed to+ satisfy constraint: Member must satisfy regular expression pattern:+ [a-zA-Z][-a-zA-Z0-9]*``; the length sentence follows the shape of other+ API parameters, unmeasured)."""+ problems = []+ if len(stack_name) > STACK_NAME_MAX_CHARS:+ problems.append(+ f"Value '{stack_name}' at 'stackName' failed to satisfy constraint: Member "+ f"must have length less than or equal to {STACK_NAME_MAX_CHARS}")+ if not STACK_NAME_RE.fullmatch(stack_name):+ problems.append(+ f"Value '{stack_name}' at 'stackName' failed to satisfy constraint: Member "+ f"must satisfy regular expression pattern: {STACK_NAME_PATTERN}")+ return problems+++def _template_body_problem(body: str) -> str | None:+ """The request-level constraint on ``TemplateBody`` (the quota page:+ "Template body size in a request", 51,200 bytes), as one sentence of the+ API's parameter validation, which echoes the value (quoted: ``Value+ '<the template>' at 'templateBody' failed to satisfy constraint: Member+ must have length less than or equal to 51200``); None when it holds.+ CreateStack joins it with the stack name problems into one message."""+ if body and len(body.encode("utf-8")) > TEMPLATE_BODY_MAX_BYTES:+ return (f"Value '{body}' at 'templateBody' failed to satisfy constraint: Member "+ f"must have length less than or equal to {TEMPLATE_BODY_MAX_BYTES}")+ return None+++def _request_problems(params, stack_name: str = ""):+ """The request-level constraints of a call that carries a template: the+ stack name where the call names one, and the size of an inline+ ``TemplateBody``. The API reports every violation of a request in one+ message, so both are collected before it answers. Returns the error+ response, or None when the request passes."""+ problems = stack_name_problems(stack_name) if stack_name else []+ body_problem = _template_body_problem(_p(params, "TemplateBody"))+ if body_problem:+ problems.append(body_problem)+ if problems:+ return _error("ValidationError", validation_error_message(problems))+ return None++def _resolve_template(params):"""Resolve TemplateBody or TemplateURL to a template string.If TemplateURL is provided, fetch the template from S3.- Returns (template_body, error_tuple) — error_tuple is None on success."""+ Returns (template_body, error_tuple) — error_tuple is None on success.++ The request-level constraints are checked here, so every action that reads+ a template applies them. CreateStack and a CREATE change set check them+ earlier, together with the stack name, and return before this point."""+ request_error = _request_problems(params)+ if request_error:+ return None, request_errorreturn _resolve_document(params, "TemplateBody", "TemplateURL", "Template")@@ -121,6 +196,13 @@ def _resolve_document(params, body_key, url_key, label):obj_data = _s3._get_object_data(bucket_name, key)if obj_data is None:return None, _error("ValidationError", f"{label} not found at {url}")+ if body_key == "TemplateBody" and len(obj_data) > TEMPLATE_URL_MAX_BYTES:+ # "Template body size in an Amazon S3 object", 1 MB on the quota+ # page; the service enforces 1,000,000 bytes and answers with+ # this sentence (quoted from reports of the error).+ return None, _error(+ "ValidationError",+ f"Template may not exceed {TEMPLATE_URL_MAX_BYTES} bytes in size.")return obj_data.decode("utf-8"), Noneexcept Exception as e:logger.warning("Failed to fetch %s %s: %s", url_key, url, e)diff --git a/tests/test_cfn.py b/tests/test_cfn.pyindex c3745ca8..a4263de3 100644--- a/tests/test_cfn.py+++ b/tests/test_cfn.py@@ -18015,3 +18015,728 @@ def test_cfn_update_stack_without_changes_is_refused(cfn, sqs):finally:_delete_cfn_test_stack(cfn, stack_name)+++# ---------------------------------------------------------------------------+# Rules section+# ---------------------------------------------------------------------------++def _rules_template(rules, params):+ """A template with the given Rules and Parameters and one resource that+ provisions instantly."""+ return json.dumps({+ "Parameters": params,+ "Rules": rules,+ "Resources": {"Handle": {"Type": "AWS::CloudFormation::WaitConditionHandle"}},+ })+++def _refused_by_rules(cfn, stack_name, body, parameters, expected):+ with pytest.raises(ClientError) as exc:+ cfn.create_stack(StackName=stack_name, TemplateBody=body, Parameters=parameters)+ assert exc.value.response["Error"]["Code"] == "ValidationError"+ assert exc.value.response["Error"]["Message"] == expected+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=stack_name)+++def test_cfn_rules_cdk_check_bootstrap_version(cfn, ssm):+ """The rule every CDK-synthesized template carries: `BootstrapVersion` is an+ `AWS::SSM::Parameter::Value<String>` and the assertion refuses the values+ 1 to 5. The rule passes when the SSM parameter holds "30" and refuses the+ stack with the AssertDescription when it holds "5", on CreateStack,+ UpdateStack and CreateChangeSet (both types), and no stack record is left+ behind. The failure text is unmeasured."""+ uid = _uuid_mod.uuid4().hex[:8]+ ssm_name = f"/cdk-bootstrap/{uid}/version"+ description = ("CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' "+ "with a recent version of the CDK CLI.")+ template = json.dumps({+ "Parameters": {"BootstrapVersion": {+ "Type": "AWS::SSM::Parameter::Value<String>", "Default": ssm_name,+ "Description": "Version of the CDK Bootstrap resources in this environment, "+ "automatically retrieved from SSM Parameter Store."}},+ "Rules": {"CheckBootstrapVersion": {"Assertions": [{+ "Assert": {"Fn::Not": [{"Fn::Contains": [+ ["1", "2", "3", "4", "5"], {"Ref": "BootstrapVersion"}]}]},+ "AssertDescription": description}]}},+ "Resources": {"Handle": {"Type": "AWS::CloudFormation::WaitConditionHandle"}},+ })+ expected = f"Template error: rule CheckBootstrapVersion failed: {description}"+ stack_name = f"cfn-rules-cdk-{uid}"++ ssm.put_parameter(Name=ssm_name, Value="5", Type="String")+ try:+ _refused_by_rules(cfn, stack_name, template, [], expected)+ with pytest.raises(ClientError) as exc:+ cfn.create_change_set(StackName=stack_name, ChangeSetName="cs",+ ChangeSetType="CREATE", TemplateBody=template)+ assert exc.value.response["Error"]["Message"] == expected+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=stack_name)++ ssm.put_parameter(Name=ssm_name, Value="30", Type="String", Overwrite=True)+ cfn.create_stack(StackName=stack_name, TemplateBody=template)+ stack = _wait_stack(cfn, stack_name)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")++ ssm.put_parameter(Name=ssm_name, Value="5", Type="String", Overwrite=True)+ with pytest.raises(ClientError) as exc:+ cfn.update_stack(StackName=stack_name, TemplateBody=template,+ Tags=[{"Key": "stage", "Value": "two"}])+ assert exc.value.response["Error"]["Code"] == "ValidationError"+ assert exc.value.response["Error"]["Message"] == expected+ # UsePreviousTemplate re-resolves the SSM parameter and re-runs the rule.+ with pytest.raises(ClientError) as exc:+ cfn.update_stack(StackName=stack_name, UsePreviousTemplate=True,+ Parameters=[{"ParameterKey": "BootstrapVersion",+ "UsePreviousValue": True}],+ Tags=[{"Key": "stage", "Value": "two"}])+ assert exc.value.response["Error"]["Message"] == expected+ with pytest.raises(ClientError) as exc:+ cfn.create_change_set(StackName=stack_name, ChangeSetName="cs",+ TemplateBody=template, Tags=[{"Key": "stage", "Value": "two"}])+ assert exc.value.response["Error"]["Message"] == expected+ assert cfn.describe_stacks(StackName=stack_name)["Stacks"][0]["StackStatus"] \+ == "CREATE_COMPLETE"+ assert cfn.list_change_sets(StackName=stack_name)["Summaries"] == []++ ssm.put_parameter(Name=ssm_name, Value="31", Type="String", Overwrite=True)+ cfn.update_stack(StackName=stack_name, TemplateBody=template,+ Tags=[{"Key": "stage", "Value": "two"}])+ stack = _wait_stack(cfn, stack_name)+ assert stack["StackStatus"] == "UPDATE_COMPLETE", stack.get("StackStatusReason")+ finally:+ _delete_cfn_test_stack(cfn, stack_name)+ try:+ ssm.delete_parameter(Name=ssm_name)+ except ClientError:+ pass+++def test_cfn_rules_condition_and_list_functions(cfn):+ """A rule whose RuleCondition is false is skipped; Fn::Contains,+ Fn::EachMemberEquals and Fn::EachMemberIn see a CommaDelimitedList+ parameter as a list of trimmed members; Fn::And, Fn::Or, Fn::Equals and+ Fn::If nest; the pseudo parameters resolve."""+ uid = _uuid_mod.uuid4().hex[:8]+ params = {+ "Env": {"Type": "String", "Default": "test"},+ "Size": {"Type": "String", "Default": "t3.medium"},+ "Subnets": {"Type": "CommaDelimitedList", "Default": "a, b"},+ "Zones": {"Type": "CommaDelimitedList", "Default": "us-east-1a,us-east-1b"},+ }+ rules = {+ "ProdOnly": { # RuleCondition false: the failing assertion is never checked+ "RuleCondition": {"Fn::Equals": [{"Ref": "Env"}, "prod"]},+ "Assertions": [{"Assert": {"Fn::Equals": [{"Ref": "Size"}, "t3.large"]},+ "AssertDescription": "prod needs t3.large"}]},+ "TestSize": {+ "RuleCondition": {"Fn::Equals": [{"Ref": "Env"}, "test"]},+ "Assertions": [+ {"Assert": {"Fn::Contains": [["t3.medium", "t3.small"], {"Ref": "Size"}]},+ "AssertDescription": "test needs t3.medium or t3.small"},+ {"Assert": {"Fn::EachMemberIn": [{"Ref": "Subnets"}, ["a", "b", "c"]]}},+ {"Assert": {"Fn::EachMemberEquals": [{"Ref": "Subnets"}, "a"]},+ "AssertDescription": "all subnets must be a"}]},+ "Nested": {"Assertions": [{"Assert": {"Fn::And": [+ {"Fn::Or": [{"Fn::Equals": [{"Ref": "AWS::Region"}, "us-east-1"]},+ {"Fn::Equals": [{"Ref": "AWS::AccountId"}, "000000000000"]}]},+ {"Fn::Not": [{"Fn::Equals": [{"Ref": "AWS::Partition"}, "aws-cn"]}]},+ {"Fn::Contains": [{"Ref": "Zones"},+ {"Fn::If": [{"Fn::Equals": [{"Ref": "Env"}, "test"]},+ "us-east-1b", "us-east-1z"]}]},+ ]}}]},+ }+ body = _rules_template(rules, params)+ name = f"cfn-rules-fn-{uid}"++ _refused_by_rules(cfn, name, body, [{"ParameterKey": "Subnets", "ParameterValue": "a,b,z"}],+ "Template error: rule TestSize failed: assertion 2 evaluated to false")+ _refused_by_rules(cfn, name, body, [{"ParameterKey": "Subnets", "ParameterValue": "a,b"},+ {"ParameterKey": "Size", "ParameterValue": "t3.large"}],+ "Template error: rule TestSize failed: test needs t3.medium or t3.small")+ _refused_by_rules(cfn, name, body, [{"ParameterKey": "Env", "ParameterValue": "prod"}],+ "Template error: rule ProdOnly failed: prod needs t3.large")++ cfn.create_stack(StackName=name, TemplateBody=body,+ Parameters=[{"ParameterKey": "Subnets", "ParameterValue": "a, a"}])+ try:+ stack = _wait_stack(cfn, name)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")+ finally:+ _delete_cfn_test_stack(cfn, name)+++def test_cfn_rules_shape_and_function_errors(cfn):+ """An assertion that is not a boolean, a RuleCondition that is not a+ boolean, a Ref to something that is not a parameter, a function outside+ the rule set (measured for ``[Fn::If]`` in cloudformation-coverage-roadmap+ issue 921; the bracket lists the offending names), a rule without+ Assertions and a malformed argument list are refused before a stack+ exists with a ValidationError, never an internal error; the function+ and shape checks also run on ValidateTemplate."""+ uid = _uuid_mod.uuid4().hex[:8]+ params = {"P": {"Type": "String", "Default": "x"}}+ name = f"cfn-rules-shape-{uid}"++ _refused_by_rules(+ cfn, name, _rules_template({"R": {"Assertions": [{"Assert": {"Ref": "P"}}]}}, params),+ [], "Template error: assertion 1 of rule R must evaluate to true or false, got 'x'")+ _refused_by_rules(+ cfn, name, _rules_template(+ {"R": {"Assertions": [{"Assert": {"Fn::Equals": [{"Ref": "Handle"}, "x"]}}]}},+ params),+ [], "Template format error: Unresolved resource dependencies [Handle] in the Rules "+ "block of the template")+ unsupported = _rules_template(+ {"R": {"Assertions": [{"Assert": {"Fn::Equals": [{"Fn::Sub": "${P}"}, "x"]}}]}}, params)+ expected = ("Template format error: Following functions are not supported in the Rules "+ "block of the template: [Fn::Sub]")+ _refused_by_rules(cfn, name, unsupported, [], expected)+ with pytest.raises(ClientError) as exc:+ cfn.validate_template(TemplateBody=unsupported)+ assert exc.value.response["Error"]["Message"] == expected+ _refused_by_rules(cfn, name, _rules_template({"R": {"RuleCondition": True}}, params), [],+ "Template format error: Rule R must contain an Assertions list")+ _refused_by_rules(+ cfn, name, _rules_template({"R": {"RuleCondition": {"Ref": "P"},+ "Assertions": [{"Assert": True}]}}, params),+ [], "Template error: the RuleCondition of rule R must evaluate to true or false, got 'x'")+ for assertion, expected in (+ ({"Fn::Equals": "x"},+ "Template format error: Fn::Equals in rule R must be a list of 2 elements"),+ ({"Fn::Equals": [{"Ref": "P"}]},+ "Template format error: Fn::Equals in rule R must be a list of 2 elements"),+ ({"Ref": ["P"]}, "Template format error: Ref in rule R must name a parameter"),+ ({"Fn::ValueOf": "P"},+ "Template format error: Fn::ValueOf in rule R must be a list of 2 elements"),+ ({"Fn::ValueOf": [{"Ref": "P"}, "VpcId"]},+ "Template format error: Fn::ValueOf in rule R takes two strings; no function "+ "can be used within it"),+ ({"Fn::And": None},+ "Template format error: Fn::And in rule R must be a list of conditions"),+ ({"Fn::Not": [{"Fn::Equals": [{"Ref": "P"}, "x"]}, True]},+ "Template format error: Fn::Not in rule R must be a list of 1 elements"),+ ):+ body = _rules_template({"R": {"Assertions": [{"Assert": assertion}]}}, params)+ _refused_by_rules(cfn, name, body, [], expected)+ with pytest.raises(ClientError) as exc:+ cfn.validate_template(TemplateBody=body)+ assert exc.value.response["Error"]["Message"] == expected+ # A template whose rules pass validates; ValidateTemplate has no values to+ # evaluate against, so a rule that would fail on CreateStack passes there.+ failing = _rules_template(+ {"R": {"Assertions": [{"Assert": {"Fn::Equals": [{"Ref": "P"}, "y"]},+ "AssertDescription": "P must be y"}]}}, params)+ cfn.validate_template(TemplateBody=failing)+ _refused_by_rules(cfn, name, failing, [], "Template error: rule R failed: P must be y")+++def test_cfn_rules_account_lookups(cfn, ec2, ssm):+ """Fn::RefAll, Fn::ValueOf and Fn::ValueOfAll are served from the EC2 store+ for the VPC, subnet and security-group parameter types, with every+ attribute the reference lists, also behind an SSM ``Value<List<...>>``+ parameter type; a rule over any other type is skipped (logged), not+ failed."""+ uid = _uuid_mod.uuid4().hex[:8]+ vpc_id = ec2.create_vpc(CidrBlock="10.77.0.0/16", TagSpecifications=[{+ "ResourceType": "vpc", "Tags": [{"Key": "Department", "Value": "IT"}]}])["Vpc"]["VpcId"]+ subnet_ids, other_vpc, other_subnet = [], None, None+ try:+ subnet_ids = [+ ec2.create_subnet(VpcId=vpc_id, CidrBlock=f"10.77.{i}.0/24",+ AvailabilityZone="us-east-1a")["Subnet"]["SubnetId"]+ for i in (1, 2)]+ other_vpc = ec2.create_vpc(CidrBlock="10.78.0.0/16")["Vpc"]["VpcId"]+ other_subnet = ec2.create_subnet(+ VpcId=other_vpc, CidrBlock="10.78.1.0/24")["Subnet"]["SubnetId"]+ vpc = {+ "DefaultSecurityGroupId": ec2.describe_security_groups(Filters=[+ {"Name": "vpc-id", "Values": [vpc_id]},+ {"Name": "group-name", "Values": ["default"]}])["SecurityGroups"][0]["GroupId"],+ "DefaultNetworkAclId": ec2.describe_network_acls(Filters=[+ {"Name": "vpc-id", "Values": [vpc_id]}])["NetworkAcls"][0]["NetworkAclId"],+ }+ ssm_name = f"/cfn-rules/{uid}/subnets"+ ssm.put_parameter(Name=ssm_name, Value=",".join(subnet_ids), Type="StringList")+ _rules_account_lookups(cfn, uid, vpc_id, subnet_ids, other_vpc, other_subnet,+ vpc, ssm_name)+ finally:+ for sid in subnet_ids + ([other_subnet] if other_subnet else []):+ ec2.delete_subnet(SubnetId=sid)+ ec2.delete_vpc(VpcId=vpc_id)+ if other_vpc:+ ec2.delete_vpc(VpcId=other_vpc)+ try:+ ssm.delete_parameter(Name=f"/cfn-rules/{uid}/subnets")+ except ClientError:+ pass+++def _rules_account_lookups(cfn, uid, vpc_id, subnet_ids, other_vpc, other_subnet, vpc, ssm_name):+ params = {+ "VpcId": {"Type": "AWS::EC2::VPC::Id"},+ "Subnets": {"Type": "List<AWS::EC2::Subnet::Id>"},+ "SsmSubnets": {"Type": "AWS::SSM::Parameter::Value<List<AWS::EC2::Subnet::Id>>",+ "Default": ssm_name},+ "KeyName": {"Type": "AWS::EC2::KeyPair::KeyName", "Default": "any"},+ }+ rules = {+ "VpcExists": {"Assertions": [{+ "Assert": {"Fn::Contains": [{"Fn::RefAll": "AWS::EC2::VPC::Id"}, {"Ref": "VpcId"}]},+ "AssertDescription": "The VPC must exist"}]},+ "Department": {"Assertions": [{+ "Assert": {"Fn::Equals": [{"Fn::ValueOf": ["VpcId", "Tags.Department"]}, "IT"]},+ "AssertDescription": "The VPC must belong to IT"}]},+ "SubnetsInVpc": {"Assertions": [{+ "Assert": {"Fn::EachMemberEquals": [{"Fn::ValueOf": ["Subnets", "VpcId"]},+ {"Ref": "VpcId"}]},+ "AssertDescription": "All subnets must be in the VPC"}]},+ "VpcAttributes": {"Assertions": [+ {"Assert": {"Fn::Equals": [{"Fn::ValueOf": ["VpcId", "DefaultSecurityGroup"]},+ vpc["DefaultSecurityGroupId"]]}},+ {"Assert": {"Fn::Equals": [{"Fn::ValueOf": ["VpcId", "DefaultNetworkAcl"]},+ vpc["DefaultNetworkAclId"]]}},+ {"Assert": {"Fn::EachMemberEquals": [{"Fn::ValueOf": ["Subnets", "AvailabilityZone"]},+ "us-east-1a"]},+ "AssertDescription": "All subnets must be in us-east-1a"}]},+ "SubnetsExist": {"Assertions": [{+ "Assert": {"Fn::EachMemberIn": [{"Ref": "Subnets"},+ {"Fn::RefAll": "AWS::EC2::Subnet::Id"}]}}]},+ "SsmSubnetsInVpc": {"Assertions": [{+ "Assert": {"Fn::EachMemberEquals": [{"Fn::ValueOf": ["SsmSubnets", "VpcId"]},+ {"Ref": "VpcId"}]},+ "AssertDescription": "All SSM subnets must be in the VPC"}]},+ "SomeVpcIsIT": {"Assertions": [{+ "Assert": {"Fn::Contains": [{"Fn::ValueOfAll": ["AWS::EC2::VPC::Id", "Tags.Department"]},+ "IT"]}}]},+ "Skipped": {"Assertions": [{ # key pairs are not listed: skipped, never failed+ "Assert": {"Fn::Contains": [{"Fn::RefAll": "AWS::EC2::KeyPair::KeyName"},+ {"Ref": "KeyName"}]}}]},+ }+ body = _rules_template(rules, params)+ name = f"cfn-rules-account-{uid}"++ def parameters(vpc, subnets):+ return [{"ParameterKey": "VpcId", "ParameterValue": vpc},+ {"ParameterKey": "Subnets", "ParameterValue": ",".join(subnets)}]++ _refused_by_rules(cfn, name, body, parameters("vpc-0000000000000dead", subnet_ids),+ "Template error: rule VpcExists failed: The VPC must exist")+ _refused_by_rules(cfn, name, body, parameters(vpc_id, subnet_ids + [other_subnet]),+ "Template error: rule SubnetsInVpc failed: All subnets must be in the VPC")+ _refused_by_rules(cfn, name, body, parameters(other_vpc, [other_subnet]),+ "Template error: rule Department failed: The VPC must belong to IT")++ cfn.create_stack(StackName=name, TemplateBody=body, Parameters=parameters(vpc_id, subnet_ids))+ try:+ stack = _wait_stack(cfn, name)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")+ finally:+ _delete_cfn_test_stack(cfn, name)+++# ---------------------------------------------------------------------------+# Template and stack quotas+# ---------------------------------------------------------------------------++_HANDLE = {"Type": "AWS::CloudFormation::WaitConditionHandle"}+++def _refused_quota(cfn, stack_name, body, expected, parameters=None):+ with pytest.raises(ClientError) as exc:+ cfn.create_stack(StackName=stack_name, TemplateBody=body, Parameters=parameters or [])+ assert exc.value.response["Error"]["Code"] == "ValidationError"+ message = exc.value.response["Error"]["Message"]+ if callable(expected):+ assert expected(message), message+ else:+ assert message == expected+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=stack_name)+++def test_cfn_template_body_size_quotas(cfn, s3):+ """A TemplateBody of 51,200 bytes is accepted and one of 51,201 refused by+ the request-level constraint (quoted: ``at 'templateBody' failed to+ satisfy constraint: Member must have length less than or equal to+ 51200``), on CreateStack, UpdateStack, ValidateTemplate and+ GetTemplateSummary; a template behind TemplateURL may not exceed+ 1,000,000 bytes (quoted: ``Template may not exceed 1000000 bytes in+ size.``). ValidateTemplate reads its template through the same path, so it+ takes a TemplateURL now."""+ uid = _uuid_mod.uuid4().hex[:8]+ name = f"cfn-body-size-{uid}"++ def body_of(size):+ skeleton = json.dumps({"Metadata": {"Pad": ""}, "Resources": {"Handle": _HANDLE}})+ return json.dumps({"Metadata": {"Pad": "x" * (size - len(skeleton))},+ "Resources": {"Handle": _HANDLE}})++ assert len(body_of(51200).encode()) == 51200+ too_big = body_of(51201)++ def refused(message):+ return (message.startswith("1 validation error detected: Value '")+ and message.endswith("' at 'templateBody' failed to satisfy constraint: "+ "Member must have length less than or equal to 51200"))++ _refused_quota(cfn, name, too_big, refused)+ with pytest.raises(ClientError) as exc:+ cfn.validate_template(TemplateBody=too_big)+ assert refused(exc.value.response["Error"]["Message"])+ with pytest.raises(ClientError) as exc:+ cfn.get_template_summary(TemplateBody=too_big)+ assert refused(exc.value.response["Error"]["Message"])++ cfn.create_stack(StackName=name, TemplateBody=body_of(51200))+ try:+ stack = _wait_stack(cfn, name)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")+ with pytest.raises(ClientError) as exc:+ cfn.update_stack(StackName=name, TemplateBody=too_big)+ assert refused(exc.value.response["Error"]["Message"])+ finally:+ _delete_cfn_test_stack(cfn, name)++ bucket = f"cfn-body-size-{uid}"+ s3.create_bucket(Bucket=bucket)+ try:+ endpoint = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566")+ s3.put_object(Bucket=bucket, Key="big.json", Body=body_of(1000001).encode())+ s3.put_object(Bucket=bucket, Key="ok.json", Body=body_of(1000000).encode())+ s3.put_object(Bucket=bucket, Key="validate.json", Body=json.dumps({+ "Description": "a template behind a URL",+ "Parameters": {"P": {"Type": "String", "Default": "d"}},+ "Resources": {"Handle": _HANDLE}}).encode())+ # ValidateTemplate took TemplateBody only; it reads the same path now.+ validated = cfn.validate_template(TemplateURL=f"{endpoint}/{bucket}/validate.json")+ assert validated["Description"] == "a template behind a URL"+ assert [p["ParameterKey"] for p in validated["Parameters"]] == ["P"]+ with pytest.raises(ClientError) as exc:+ cfn.create_stack(StackName=name, TemplateURL=f"{endpoint}/{bucket}/big.json")+ assert exc.value.response["Error"]["Code"] == "ValidationError"+ assert exc.value.response["Error"]["Message"] == \+ "Template may not exceed 1000000 bytes in size."+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=name)+ cfn.create_stack(StackName=name, TemplateURL=f"{endpoint}/{bucket}/ok.json")+ try:+ stack = _wait_stack(cfn, name)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")+ finally:+ _delete_cfn_test_stack(cfn, name)+ finally:+ for key in ("big.json", "ok.json", "validate.json"):+ s3.delete_object(Bucket=bucket, Key=key)+ s3.delete_bucket(Bucket=bucket)+++def test_cfn_template_section_quotas(cfn):+ """501 resources (quoted: ``Template format error: Number of resources,+ 536, is greater than maximum allowed, 500``), 201 parameters, outputs or+ mappings, 201 attributes in a mapping, a 256-character name of any+ section, a 256-character mapping attribute name, a 1,025-byte+ description, a parameter default over 4,096 bytes (quoted shape) and a+ provided value over 4,096 bytes are refused before a stack exists; the+ maximum of each passes the check in process."""+ from ministack.services.cloudformation.engine import validate_template_support++ uid = _uuid_mod.uuid4().hex[:8]+ name = f"cfn-quotas-{uid}"++ def template(**sections):+ return {"Resources": {"Handle": _HANDLE}, **sections}++ resources = {f"H{i}": _HANDLE for i in range(501)}+ _refused_quota(cfn, name, json.dumps({"Resources": resources}),+ "Template format error: Number of resources, 501, is greater than "+ "maximum allowed, 500")+ params = {f"P{i}": {"Type": "String", "Default": "x"} for i in range(201)}+ _refused_quota(cfn, name, json.dumps(template(Parameters=params)),+ "Template format error: Number of parameters, 201, is greater than "+ "maximum allowed, 200")+ outputs = {f"O{i}": {"Value": "x"} for i in range(201)}+ _refused_quota(cfn, name, json.dumps(template(Outputs=outputs)),+ "Template format error: Number of outputs, 201, is greater than "+ "maximum allowed, 200")+ mappings = {f"M{i}": {"a": {"b": "c"}} for i in range(201)}+ _refused_quota(cfn, name, json.dumps(template(Mappings=mappings)),+ "Template format error: Number of mappings, 201, is greater than "+ "maximum allowed, 200")+ wide = {"M": {f"a{i}": {"b": "c"} for i in range(201)}}+ _refused_quota(cfn, name, json.dumps(template(Mappings=wide)),+ "Template format error: Number of attributes in mapping M, 201, is "+ "greater than maximum allowed, 200")+ long_name = "R" * 256+ _refused_quota(cfn, name, json.dumps({"Resources": {long_name: _HANDLE}}),+ f"Template format error: Resource name {'R' * 32}... may not exceed "+ "255 characters")+ _refused_quota(cfn, name, json.dumps(template(Description="d" * 1025)),+ "Template format error: Template description may not exceed 1024 "+ "bytes in size")+ big_default = "v" * 4097+ _refused_quota(cfn, name, json.dumps(template(+ Parameters={"P": {"Type": "String", "Default": big_default}})),+ f"Template format error: Parameter 'P' default value '{big_default}' length is "+ "greater than 4096.")+ _refused_quota(cfn, name, json.dumps(template(Parameters={"P": {"Type": "String"}})),+ f"1 validation error detected: Value '{big_default}' at "+ "'parameters.1.member.parameterValue' failed to satisfy constraint: "+ "Member must have length less than or equal to 4096",+ parameters=[{"ParameterKey": "P", "ParameterValue": big_default}])++ # The name-length branch of the other sections and the attribute name of+ # a mapping, in process: they refuse before a stack record exists too, but+ # the message is what this checks.+ for singular, section in (+ ("Parameter", {"Parameters": {"N" * 256: {"Type": "String"}}}),+ ("Output", {"Outputs": {"N" * 256: {"Value": "x"}}}),+ ("Mapping", {"Mappings": {"N" * 256: {"a": {"b": "c"}}}}),+ ):+ with pytest.raises(ValueError) as err:+ validate_template_support(template(**section), {})+ assert str(err.value) == (+ f"Template format error: {singular} name {'N' * 32}... may not exceed "+ "255 characters")+ with pytest.raises(ValueError) as err:+ validate_template_support(template(Mappings={"M": {"a" * 256: {"b": "c"}}}), {})+ assert str(err.value) == (+ f"Template format error: Mapping attribute name {'a' * 32}... of mapping M "+ "may not exceed 255 characters")++ # The maximum of each quota passes (checked in process: 500 handles would+ # deploy, but the check is what this test is about).+ validate_template_support({+ "Description": "d" * 1024,+ "Parameters": {f"P{i}": {"Type": "String", "Default": "v" * 4096} for i in range(200)},+ "Mappings": {f"M{i}": {f"a{j}": {"b": "c"} for j in range(200)} for i in range(200)},+ "Resources": {"R" * 252 + f"{i:03d}": _HANDLE for i in range(500)},+ "Outputs": {f"O{i}": {"Value": "x"} for i in range(200)},+ }, {})+++def test_cfn_stack_name_quotas(cfn):+ """CreateStack and a CREATE change set refuse a stack name that is not+ alphanumeric-and-hyphens starting with a letter (quoted: ``at+ 'stackName' failed to satisfy constraint: Member must satisfy regular+ expression pattern``) or longer than 128 characters, before a stack+ exists; a 128-character name is accepted."""+ uid = _uuid_mod.uuid4().hex[:8]+ body = json.dumps({"Resources": {"Handle": _HANDLE}})+ pattern = ("failed to satisfy constraint: Member must satisfy regular expression "+ "pattern: [a-zA-Z][-a-zA-Z0-9]*")+ for bad in (f"cfn_name_{uid}", f"1cfn-name-{uid}", f"cfn.name.{uid}"):+ _refused_quota(cfn, bad, body,+ f"1 validation error detected: Value '{bad}' at 'stackName' {pattern}")+ with pytest.raises(ClientError) as exc:+ cfn.create_change_set(StackName=f"cfn_cs_{uid}", ChangeSetName="cs",+ ChangeSetType="CREATE", TemplateBody=body)+ assert exc.value.response["Error"]["Message"] == \+ f"1 validation error detected: Value 'cfn_cs_{uid}' at 'stackName' {pattern}"+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=f"cfn_cs_{uid}")++ long = f"cfn-long-{uid}-" + "x" * (129 - len(f"cfn-long-{uid}-"))+ assert len(long) == 129+ _refused_quota(cfn, long, body,+ f"1 validation error detected: Value '{long}' at 'stackName' failed to "+ "satisfy constraint: Member must have length less than or equal to 128")+ both = f"cfn_long_{uid}_" + "x" * (129 - len(f"cfn_long_{uid}_"))+ _refused_quota(cfn, both, body,+ f"2 validation errors detected: Value '{both}' at 'stackName' failed to "+ "satisfy constraint: Member must have length less than or equal to 128; "+ f"Value '{both}' at 'stackName' {pattern}")++ # Every request-level violation is reported in one message: a bad name+ # and an oversized body together, on CreateStack and on a CREATE change+ # set alike (the name problem first, order unmeasured).+ big = json.dumps({"Metadata": {"Pad": "x" * 51200}, "Resources": {"Handle": _HANDLE}})+ joined = (f"2 validation errors detected: Value 'cfn_both_{uid}' at 'stackName' {pattern}; "+ f"Value '{big}' at 'templateBody' failed to satisfy constraint: Member must have "+ "length less than or equal to 51200")+ _refused_quota(cfn, f"cfn_both_{uid}", big, joined)+ with pytest.raises(ClientError) as exc:+ cfn.create_change_set(StackName=f"cfn_both_{uid}", ChangeSetName="cs",+ ChangeSetType="CREATE", TemplateBody=big)+ assert exc.value.response["Error"]["Message"] == joined+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=f"cfn_both_{uid}")++ ok = long[:128]+ cfn.create_stack(StackName=ok, TemplateBody=body)+ try:+ stack = _wait_stack(cfn, ok)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")+ finally:+ _delete_cfn_test_stack(cfn, ok)+++# ---------------------------------------------------------------------------+# AWS::Include transform+# ---------------------------------------------------------------------------++def test_cfn_include_transform(cfn, s3, ssm):+ """An embedded ``Fn::Transform`` naming ``AWS::Include`` is replaced by the+ S3 object it points to (JSON or plain YAML, a key-value object): next to+ a sibling resource inside Resources, as a whole Properties map, inside a+ Properties map next to other properties, inside Outputs and inside+ Mappings (the reference names the mappings section as an example of where+ the transform may sit). A location+ that is not an ``s3://`` URI and a snippet with a YAML shorthand tag are+ refused with the sentences a real account answers, bare (no ``Template+ format error:`` prefix); a missing object, a snippet that is not an+ object and a nested include are refused too, all before a stack exists.+ On UpdateStack and CreateChangeSet the include is expanded the same way+ and the change set lists the included resource; with UsePreviousTemplate+ the stored processed template is used, so a snippet edited in S3 since+ the deploy is not picked up."""+ uid = _uuid_mod.uuid4().hex[:8]+ bucket = f"cfn-include-{uid}"+ name = f"cfn-include-{uid}"+ s3.create_bucket(Bucket=bucket)+ try:+ _include_transform(cfn, s3, ssm, uid, bucket, name)+ finally:+ _delete_cfn_test_stack(cfn, name)+ for key in ("resources.yaml", "properties.json", "value.json", "list.json",+ "nested.json", "shorthand.yaml", "outputs.json", "second.json",+ "mappings.json"):+ s3.delete_object(Bucket=bucket, Key=key)+ s3.delete_bucket(Bucket=bucket)+++def _include_transform(cfn, s3, ssm, uid, bucket, name):+ s3.put_object(Bucket=bucket, Key="resources.yaml", Body=(+ "Included:\n"+ " Type: AWS::SSM::Parameter\n"+ " Properties:\n"+ f" Name: /cfn-include/{uid}/from-yaml\n"+ " Type: String\n"+ " Value: {Ref: Handle}\n").encode())+ s3.put_object(Bucket=bucket, Key="shorthand.yaml", Body=(+ "Included:\n"+ " Type: AWS::SSM::Parameter\n"+ " Properties:\n"+ f" Name: /cfn-include/{uid}/shorthand\n"+ " Type: String\n"+ " Value: !Ref Handle\n").encode())+ s3.put_object(Bucket=bucket, Key="properties.json", Body=json.dumps({+ "Name": f"/cfn-include/{uid}/whole", "Type": "String", "Value": "whole"}).encode())+ s3.put_object(Bucket=bucket, Key="value.json", Body=json.dumps({+ "Value": "merged", "Type": "String"}).encode())+ s3.put_object(Bucket=bucket, Key="outputs.json", Body=json.dumps({+ "IncludedOutput": {"Value": {"Ref": "Handle"}}}).encode())+ s3.put_object(Bucket=bucket, Key="second.json", Body=json.dumps({+ "Second": {"Type": "AWS::SSM::Parameter", "Properties": {+ "Name": f"/cfn-include/{uid}/second", "Type": "String", "Value": "second"}}}).encode())+ s3.put_object(Bucket=bucket, Key="mappings.json", Body=json.dumps({+ "Sizes": {"small": {"Value": "from-map"}}}).encode())+ s3.put_object(Bucket=bucket, Key="list.json", Body=b"[1, 2]")+ s3.put_object(Bucket=bucket, Key="nested.json", Body=json.dumps({+ "Fn::Transform": {"Name": "AWS::Include",+ "Parameters": {"Location": f"s3://{bucket}/value.json"}}}).encode())++ def include(key):+ return {"Fn::Transform": {"Name": "AWS::Include",+ "Parameters": {"Location": f"s3://{bucket}/{key}"}}}++ template = {+ "Resources": {+ "Handle": _HANDLE,+ "Whole": {"Type": "AWS::SSM::Parameter", "Properties": include("properties.json")},+ "Merged": {"Type": "AWS::SSM::Parameter", "Properties": {+ "Name": f"/cfn-include/{uid}/merged", **include("value.json")}},+ "FromMap": {"Type": "AWS::SSM::Parameter", "Properties": {+ "Name": f"/cfn-include/{uid}/from-map", "Type": "String",+ "Value": {"Fn::FindInMap": ["Sizes", "small", "Value"]}}},+ **include("resources.yaml"),+ },+ "Mappings": include("mappings.json"),+ "Outputs": include("outputs.json"),+ }++ def refused(body, expected):+ with pytest.raises(ClientError) as exc:+ cfn.create_stack(StackName=name, TemplateBody=json.dumps(body))+ assert exc.value.response["Error"]["Code"] == "ValidationError"+ assert exc.value.response["Error"]["Message"] == expected, \+ exc.value.response["Error"]["Message"]+ with pytest.raises(ClientError):+ cfn.describe_stacks(StackName=name)++ def with_include(key_or_location):+ location = key_or_location if "://" in key_or_location \+ else f"s3://{bucket}/{key_or_location}"+ return {"Resources": {"Handle": _HANDLE, "Fn::Transform": {+ "Name": "AWS::Include", "Parameters": {"Location": location}}}}++ refused(with_include(f"https://{bucket}.s3.amazonaws.com/resources.yaml"),+ "Transform AWS::Include failed with: The location parameter is not a valid S3 uri.")+ refused(with_include("shorthand.yaml"),+ "Transform AWS::Include failed with: The specified S3 object's content should be "+ "valid Yaml/JSON")+ refused(with_include("missing.yaml"),+ f"Transform AWS::Include failed with: The S3 object s3://{bucket}/missing.yaml "+ "does not exist.")+ refused(with_include("list.json"),+ f"Transform AWS::Include failed with: The snippet at s3://{bucket}/list.json must "+ "be a key-value object.")+ refused(with_include("nested.json"),+ f"Transform AWS::Include failed with: The snippet at s3://{bucket}/nested.json uses "+ "AWS::Include, which cannot be nested.")++ cfn.create_stack(StackName=name, TemplateBody=json.dumps(template))+ stack = _wait_stack(cfn, name)+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")+ logical_ids = sorted(r["LogicalResourceId"] for r in+ cfn.describe_stack_resources(StackName=name)["StackResources"])+ assert logical_ids == ["FromMap", "Handle", "Included", "Merged", "Whole"]+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/whole")["Parameter"]["Value"] == "whole"+ # The Mappings section is expanded too, so Fn::FindInMap reads the snippet.+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/from-map")["Parameter"]["Value"] \+ == "from-map"+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/merged")["Parameter"]["Value"] \+ == "merged"+ handle_url = ssm.get_parameter(Name=f"/cfn-include/{uid}/from-yaml")["Parameter"]["Value"]+ assert handle_url.startswith("http")+ assert _output(stack, "IncludedOutput") == handle_url+ # The stored template is the one the caller sent: the include is not+ # baked into it.+ stored = cfn.get_template(StackName=name)["TemplateBody"]+ assert "Fn::Transform" in stored["Resources"] and "Included" not in stored["Resources"]++ # A snippet edited in S3 after the deploy is not picked up by an update+ # that reuses the template: nothing changed from the stack's point of view.+ s3.put_object(Bucket=bucket, Key="value.json", Body=json.dumps({+ "Value": "edited", "Type": "String"}).encode())+ with pytest.raises(ClientError) as exc:+ cfn.update_stack(StackName=name, UsePreviousTemplate=True)+ assert exc.value.response["Error"]["Message"] == "No updates are to be performed."+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/merged")["Parameter"]["Value"] \+ == "merged"++ # A change set and an update with a new body expand the include again:+ # the change set lists the included resource, the update provisions it.+ second = json.loads(json.dumps(template))+ second["Resources"].update(include("second.json"))+ cfn.create_change_set(StackName=name, ChangeSetName="second",+ TemplateBody=json.dumps(second))+ for _ in range(40):+ described = cfn.describe_change_set(StackName=name, ChangeSetName="second")+ if described["Status"] in ("CREATE_COMPLETE", "FAILED"):+ break+ time.sleep(0.25)+ assert described["Status"] == "CREATE_COMPLETE", described.get("StatusReason")+ added = sorted(c["ResourceChange"]["LogicalResourceId"] for c in described["Changes"]+ if c["ResourceChange"]["Action"] == "Add")+ assert added == ["Second"]+ cfn.delete_change_set(StackName=name, ChangeSetName="second")+ cfn.update_stack(StackName=name, TemplateBody=json.dumps(second))+ stack = _wait_stack(cfn, name)+ assert stack["StackStatus"] == "UPDATE_COMPLETE", stack.get("StackStatusReason")+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/second")["Parameter"]["Value"] \+ == "second"+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/merged")["Parameter"]["Value"] \+ == "edited"