Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

feat(cloudformation): Rules section, template quotas, AWS::Include (#1661)

iot-rocketd64d603655ac
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 90fe0aef..d2684d4b 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,12 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Added
+- **CloudFormation — the `Rules` section is evaluated** — a template's rules were ignored; they now run after the parameters resolve and before any resource is touched, on `CreateStack`, `UpdateStack` and `CreateChangeSet`: a rule whose `RuleCondition` is false is skipped, a false `Assert` refuses the operation with its `AssertDescription` as a `ValidationError`, and the CDK's `CheckBootstrapVersion` rule therefore refuses an outdated bootstrap version instead of passing silently. `Fn::And`, `Fn::Or`, `Fn::Not`, `Fn::Equals`, `Fn::If`, `Fn::Contains`, `Fn::EachMemberEquals`, `Fn::EachMemberIn` and `Ref` are evaluated; `Fn::RefAll`, `Fn::ValueOf` and `Fn::ValueOfAll` are served from the EC2 store for the VPC, subnet and security-group id types, and a rule over any other type is skipped with a warning. A function outside the rule set is refused as a template format error. Contributed by @iot-rocket.
+- **CloudFormation — the `AWS::Include` transform** — an embedded `Fn::Transform` naming `AWS::Include` was carried into the stack as a literal key; it is now replaced by the contents of the S3 object its `Location` points to (JSON or plain YAML, a key-value object) before the template is validated: next to a sibling resource inside `Resources`, as a whole `Properties` map, inside one next to other properties, in `Mappings`, or in `Outputs`. A location that is not an `s3://` URI, a snippet with a YAML shorthand tag, a missing object, a snippet that is not an object and a nested include are refused before a stack exists, with the sentence a real account answers where one is known. `UsePreviousTemplate` and a change set without a body reuse the stored processed template, so a snippet edited in S3 since the deploy is not picked up, as on AWS. Contributed by @iot-rocket.
+
+### Fixed
+- **CloudFormation — the template and stack quotas are enforced** — a template of any size, with any number of resources, parameters, outputs or mappings, and a stack name of any shape were accepted, so a template that a real account refuses deployed. `CreateStack`, `UpdateStack`, `CreateChangeSet`, `ValidateTemplate` and `GetTemplateSummary` now refuse a `TemplateBody` over 51,200 bytes and a template behind `TemplateURL` over 1,000,000 bytes; the engine refuses more than 500 resources, 200 parameters, 200 outputs, 200 mappings or 200 attributes in a mapping, a name over 255 characters, a description over 1,024 bytes and a parameter default or value over 4,096 bytes; `CreateStack` and a `CREATE` change set refuse a stack name that is not alphanumeric-and-hyphens starting with a letter or longer than 128 characters, joined with an oversized body into the one message the API's parameter validation gives. All of it before a stack record exists, with the messages of the API's parameter validation and the template format errors. `ValidateTemplate` also takes `TemplateURL` now. Contributed by @iot-rocket.
### Fixed
- **CloudFormation — `Capabilities` are enforced under `AUTH=true`** — `CreateStack`, `UpdateStack` and `CreateChangeSet` accepted a template with IAM resources or a `Transform` whatever the request acknowledged, so a deploy that CloudFormation refuses with `InsufficientCapabilitiesException` went through. With `AUTH=true` they now answer that error (HTTP 400, `Requires capabilities : [CAPABILITY_IAM]`, measured) and create nothing: an IAM resource needs `CAPABILITY_IAM` or `CAPABILITY_NAMED_IAM`, one with a custom name `CAPABILITY_NAMED_IAM`, a template with a `Transform` `CAPABILITY_AUTO_EXPAND` on `CreateStack` and `UpdateStack` (not on a change set, as the API documents). Without `AUTH` nothing changes; `GetTemplateSummary` keeps reporting the same set. Contributed by @iot-rocket.
diff --git a/README.md b/README.md
index b5484780..bf705c93 100644
--- a/README.md
+++ b/README.md
@@ -460,8 +460,11 @@ subnet = ec2.create_subnet(
| **Parameters** | Default values, AllowedValues validation, NoEcho masking, String/Number/CommaDelimitedList types |
| **Stack Tags** | The `Tags` of CreateStack, UpdateStack and change sets reach every resource whose type carries a tag property, together with `aws:cloudformation:stack-name`, `stack-id` and `logical-id`; a key the template sets wins |
| **Conditions** | Fn::Equals, Fn::And, Fn::Or, Fn::Not — conditional resource creation |
+| **Rules** | The `Rules` section runs after the parameters resolve and before any resource is touched (CreateStack, UpdateStack, CreateChangeSet): `RuleCondition`, `Assertions` with `AssertDescription`, Fn::And/Or/Not/Equals/If/Contains/EachMemberEquals/EachMemberIn and Ref; Fn::RefAll, Fn::ValueOf and Fn::ValueOfAll read the EC2 store for VPC, subnet and security-group ids |
+| **Transforms** | `AWS::Serverless-2016-10-31` (SAM, in the `full` image) and the embedded `AWS::Include` transform (`Fn::Transform` with an `s3://` `Location`, JSON or YAML snippets) |
| **Rollback** | Configurable via `DisableRollback` — on failure, previously created resources are cleaned up in reverse dependency order |
| **Async Status** | Stacks deploy asynchronously (`CREATE_IN_PROGRESS` → `CREATE_COMPLETE`) — poll with DescribeStacks |
+| **Quotas** | The template and stack quotas of the CloudFormation quotas page are enforced before a stack exists: 51,200 bytes of `TemplateBody`, 1,000,000 bytes behind `TemplateURL`, 500 resources, 200 parameters / outputs / mappings, 200 attributes per mapping, 255-character names, a 1,024-byte description, 4,096-byte parameter values, and a stack name of up to 128 alphanumeric characters and hyphens starting with a letter |
**Supported Resource Types:**
diff --git a/ministack/services/cloudformation/changesets.py b/ministack/services/cloudformation/changesets.py
index 3cf81a03..66eae044 100644
--- a/ministack/services/cloudformation/changesets.py
+++ b/ministack/services/cloudformation/changesets.py
@@ -17,7 +17,16 @@ from .engine import (
_resolve_refs,
validate_template_support,
)
-from .helpers import _error, _esc, _extract_members, _p, _page, _resolve_template, _xml
+from .helpers import (
+ _error,
+ _esc,
+ _extract_members,
+ _p,
+ _page,
+ _request_problems,
+ _resolve_template,
+ _xml,
+)
from .stacks import (
_add_event,
_create_stack_task_in_region,
@@ -99,9 +108,15 @@ def _create_change_set(params):
return _error("AlreadyExistsException",
f"ChangeSet [{cs_name}] already exists")
+ if cs_type == "CREATE":
+ # The request-level constraints of a new stack, joined as the API does.
+ if request_error := _request_problems(params, stack_name):
+ return request_error
+
template_body, resolve_err = _resolve_template(params)
if resolve_err:
return resolve_err
+ template_given = bool(template_body)
provided_params = _extract_members(params, "Parameters")
tags = _extract_members(params, "Tags")
@@ -198,9 +213,13 @@ def _create_change_set(params):
except ValueError as exc:
return _rejected(str(exc))
+ if cs_type == "UPDATE" and not template_given and stack.get("_template"):
+ # As UpdateStack with UsePreviousTemplate: the stored processed
+ # template, so an AWS::Include snippet changed in S3 is not picked up.
+ template = copy.deepcopy(stack["_template"])
try:
validate_template_support(
- template, _evaluate_conditions(template, param_values))
+ template, _evaluate_conditions(template, param_values), params=param_values)
except ValueError as exc:
return _rejected(str(exc))
diff --git a/ministack/services/cloudformation/engine.py b/ministack/services/cloudformation/engine.py
index 6e8dd834..81ae9d0b 100644
--- a/ministack/services/cloudformation/engine.py
+++ b/ministack/services/cloudformation/engine.py
@@ -20,6 +20,8 @@ import yaml
from ministack.core.responses import get_account_id, get_region
+logger = logging.getLogger("cloudformation")
+
# Sentinel for AWS::NoValue
_NO_VALUE = object()
@@ -233,7 +235,8 @@ def _resolve_dynamic_references(value, previous: dict | None = None,
return walk(value), resolved
-def validate_template_support(template: dict, conditions: dict) -> None:
+def validate_template_support(template: dict, conditions: dict,
+ params: dict | None = None) -> None:
"""Reject up front what provisioning could only fail on halfway through.
Real CloudFormation validates resource types before it touches anything:
@@ -257,9 +260,16 @@ def validate_template_support(template: dict, conditions: dict) -> None:
and they pass. CreateStack, UpdateStack and CreateChangeSet apply the SAM
transform before calling this, so their expanded templates are validated
as usual.
+
+ ``params`` (the resolved ``{name: {Value, NoEcho}}`` map) turns on the
+ ``Rules`` section: CreateStack, UpdateStack and CreateChangeSet pass it,
+ ValidateTemplate (no parameter values) does not.
"""
from .provisioners import _RESOURCE_HANDLERS
+ if isinstance(template, _IncludeFailed):
+ raise ValueError(template.error)
+ _validate_template_limits(template)
unrecognized: set[str] = set()
if not template.get("Transform"):
for res in (template.get("Resources") or {}).values():
@@ -294,6 +304,94 @@ def validate_template_support(template: dict, conditions: dict) -> None:
+ ", ".join(unsupported)
+ " (supported: ssm, ssm-secure, secretsmanager)"
)
+ if params is not None:
+ _evaluate_rules(template, params, conditions)
+
+
+# ===========================================================================
+# Template and stack quotas
+# ===========================================================================
+
+# The CloudFormation quotas page (cloudformation-limits.html): section sizes,
+# name lengths, the template description and the parameter value.
+_SECTION_QUOTAS = (
+ ("Resources", "resources", "Resource", 500),
+ ("Parameters", "parameters", "Parameter", 200),
+ ("Outputs", "outputs", "Output", 200),
+ ("Mappings", "mappings", "Mapping", 200),
+)
+_MAPPING_ATTRIBUTES_MAX = 200
+_NAME_MAX_CHARS = 255
+_DESCRIPTION_MAX_BYTES = 1024
+_PARAMETER_VALUE_MAX_BYTES = 4096
+
+
+def _quota_error(noun: str, count: int, maximum: int) -> ValueError:
+ # The sentence a real account answers for too many resources ("Template
+ # format error: Number of resources, 536, is greater than maximum allowed,
+ # 500"); the other sections reuse it, unmeasured.
+ return ValueError(
+ f"Template format error: Number of {noun}, {count}, is greater than "
+ f"maximum allowed, {maximum}")
+
+
+def _validate_template_limits(template: dict) -> None:
+ """The template quotas that need nothing but the template: 500 resources,
+ 200 parameters, outputs and mappings, 200 attributes per mapping, names
+ of 255 characters, a description of 1,024 bytes and a parameter default
+ of 4,096 bytes. Raises ``ValueError`` with the message the caller wraps
+ as a ``ValidationError``; the wordings other than the resource count and
+ the parameter default are unmeasured."""
+ for section, noun, singular, maximum in _SECTION_QUOTAS:
+ members = template.get(section) or {}
+ if not isinstance(members, dict):
+ continue
+ if len(members) > maximum:
+ raise _quota_error(noun, len(members), maximum)
+ for name in members:
+ if len(str(name)) > _NAME_MAX_CHARS:
+ raise ValueError(
+ f"Template format error: {singular} name {str(name)[:32]}... may "
+ f"not exceed {_NAME_MAX_CHARS} characters")
+ for name, mapping in (template.get("Mappings") or {}).items():
+ if not isinstance(mapping, dict):
+ continue
+ if len(mapping) > _MAPPING_ATTRIBUTES_MAX:
+ raise _quota_error(f"attributes in mapping {name}", len(mapping),
+ _MAPPING_ATTRIBUTES_MAX)
+ for attribute in mapping:
+ if len(str(attribute)) > _NAME_MAX_CHARS:
+ raise ValueError(
+ f"Template format error: Mapping attribute name {str(attribute)[:32]}"
+ f"... of mapping {name} may not exceed {_NAME_MAX_CHARS} characters")
+ description = template.get("Description")
+ if isinstance(description, str) and len(description.encode("utf-8")) > _DESCRIPTION_MAX_BYTES:
+ raise ValueError(
+ f"Template format error: Template description may not exceed "
+ f"{_DESCRIPTION_MAX_BYTES} bytes in size")
+ for name, defn in (template.get("Parameters") or {}).items():
+ default = defn.get("Default") if isinstance(defn, dict) else None
+ if default is not None and len(str(default).encode("utf-8")) > _PARAMETER_VALUE_MAX_BYTES:
+ # Quoted from a report of the error ("Template format error:
+ # Parameter 'EnvironmentVariables' default value '[****]' length is
+ # greater than 4096.").
+ raise ValueError(
+ f"Template format error: Parameter '{name}' default value '{default}' "
+ f"length is greater than {_PARAMETER_VALUE_MAX_BYTES}.")
+
+
+def _check_provided_parameter_values(provided_params: list[dict]) -> None:
+ """The request-level constraint on a parameter value (4,096 bytes): the
+ message follows the API's parameter validation, with the 1-based member
+ index of the request; unmeasured."""
+ for index, entry in enumerate(provided_params, 1):
+ value = entry.get("Value", "")
+ if value is not None and len(str(value).encode("utf-8")) > _PARAMETER_VALUE_MAX_BYTES:
+ raise ValueError(
+ f"1 validation error detected: Value '{value}' at "
+ f"'parameters.{index}.member.parameterValue' failed to satisfy "
+ "constraint: Member must have length less than or equal to "
+ f"{_PARAMETER_VALUE_MAX_BYTES}")
# ===========================================================================
@@ -379,6 +477,7 @@ def _resolve_parameters(template: dict, provided_params: list[dict],
Returns dict of param_name -> {Value, NoEcho}.
"""
+ _check_provided_parameter_values(provided_params)
param_defs = template.get("Parameters", {})
provided_map = {p["Key"]: p for p in provided_params if "Key" in p}
previous_params = previous_params or {}
@@ -536,6 +635,313 @@ def _evaluate_conditions(template: dict, params: dict) -> dict:
return evaluated
+# ===========================================================================
+# Rules section
+# ===========================================================================
+
+# The functions the Rules section accepts (rules-section-structure.html lists
+# them; Ref may be nested in all but Fn::ValueOf and Fn::ValueOfAll).
+_RULE_FUNCTIONS = frozenset({
+ "Fn::And", "Fn::Or", "Fn::Not", "Fn::Equals", "Fn::If", "Fn::Contains",
+ "Fn::EachMemberEquals", "Fn::EachMemberIn", "Fn::RefAll", "Fn::ValueOf",
+ "Fn::ValueOfAll", "Ref",
+})
+
+# Fn::RefAll / Fn::ValueOf / Fn::ValueOfAll read the account: the three
+# parameter types whose attributes the reference documents are served from the
+# in-process EC2 store, keyed by the attribute names the Rules reference lists.
+_RULE_ACCOUNT_TYPES = {
+ "AWS::EC2::VPC::Id": ("_vpcs", {"DefaultNetworkAcl": "DefaultNetworkAclId",
+ "DefaultSecurityGroup": "DefaultSecurityGroupId"}),
+ "AWS::EC2::Subnet::Id": ("_subnets", {"AvailabilityZone": "AvailabilityZone",
+ "VpcId": "VpcId"}),
+ "AWS::EC2::SecurityGroup::Id": ("_security_groups", {}),
+}
+
+
+class _RuleUnsupported(Exception):
+ """A rule needs an account lookup this emulator does not serve; the rule is
+ skipped with a warning instead of failing the stack operation."""
+
+
+# The argument list each rule function takes (intrinsic-function-reference-rules):
+# ``Fn::Not`` one condition, ``Fn::If`` a condition and two values, the rest
+# two elements; ``Fn::And`` / ``Fn::Or`` any list, ``Ref`` / ``Fn::RefAll`` a name.
+_RULE_ARITY = {
+ "Fn::Equals": 2, "Fn::Contains": 2, "Fn::EachMemberEquals": 2,
+ "Fn::EachMemberIn": 2, "Fn::ValueOf": 2, "Fn::ValueOfAll": 2,
+ "Fn::Not": 1, "Fn::If": 3,
+}
+
+
+def _check_rule_expression(node, rule: str) -> None:
+ """Arity and argument types of every rule function under ``rule``: a
+ malformed argument is a template format error (unmeasured wording), not
+ an evaluation crash. ``Fn::ValueOf`` and ``Fn::ValueOfAll`` take two
+ strings: the reference says no other function can be used within them."""
+ if isinstance(node, list):
+ for value in node:
+ _check_rule_expression(value, rule)
+ return
+ if not isinstance(node, dict):
+ return
+ for fn, args in node.items():
+ if fn in ("Ref", "Fn::RefAll"):
+ if not isinstance(args, str):
+ raise ValueError(
+ f"Template format error: {fn} in rule {rule} must name a "
+ + ("parameter" if fn == "Ref" else "parameter type"))
+ continue
+ if fn in ("Fn::And", "Fn::Or"):
+ if not isinstance(args, list) or not args:
+ raise ValueError(
+ f"Template format error: {fn} in rule {rule} must be a list of "
+ "conditions")
+ elif fn in _RULE_ARITY:
+ if not isinstance(args, list) or len(args) != _RULE_ARITY[fn]:
+ raise ValueError(
+ f"Template format error: {fn} in rule {rule} must be a list of "
+ f"{_RULE_ARITY[fn]} elements")
+ if fn in ("Fn::ValueOf", "Fn::ValueOfAll") and not all(
+ isinstance(a, str) for a in args):
+ raise ValueError(
+ f"Template format error: {fn} in rule {rule} takes two strings; "
+ "no function can be used within it")
+ _check_rule_expression(args, rule)
+
+
+def _rule_functions_used(node, found: set) -> None:
+ if isinstance(node, dict):
+ for key, value in node.items():
+ if key.startswith("Fn::") or key == "Ref":
+ found.add(key)
+ _rule_functions_used(value, found)
+ elif isinstance(node, list):
+ for value in node:
+ _rule_functions_used(value, found)
+
+
+def _check_rules_section(template: dict) -> None:
+ """The shape checks that need nothing but the template: every rule carries
+ an ``Assertions`` list, every assertion an ``Assert``, only the rule
+ functions appear, and every function has the arguments it takes. The
+ function message is measured (a real account answered ``Template format
+ error: Following functions are not supported in the Rules block of the
+ template: [Fn::If]``, cloudformation-coverage-roadmap issue 921, 2021;
+ the bracket holds the offending names sorted and comma-joined, the join
+ unmeasured); the shape messages are unmeasured."""
+ rules = template.get("Rules")
+ if rules is None:
+ return
+ if not isinstance(rules, dict):
+ raise ValueError("Template format error: Rules must be a map of rule name to rule")
+ for name, rule in rules.items():
+ if not isinstance(rule, dict) or not isinstance(rule.get("Assertions"), list):
+ raise ValueError(
+ f"Template format error: Rule {name} must contain an Assertions list")
+ for assertion in rule["Assertions"]:
+ if not isinstance(assertion, dict) or "Assert" not in assertion:
+ raise ValueError(
+ f"Template format error: Every assertion of rule {name} must "
+ "contain an Assert")
+ used: set[str] = set()
+ _rule_functions_used(rules, used)
+ unsupported = sorted(used - _RULE_FUNCTIONS)
+ if unsupported:
+ raise ValueError(
+ "Template format error: Following functions are not supported in the "
+ "Rules block of the template: [" + ", ".join(unsupported) + "]")
+ for name, rule in rules.items():
+ _check_rule_expression(rule.get("RuleCondition"), name)
+ for assertion in rule["Assertions"]:
+ _check_rule_expression(assertion["Assert"], name)
+
+
+def _rule_inner_type(ptype: str) -> tuple[str, bool]:
+ """The type a parameter's declared type wraps and whether it is a list:
+ the SSM ``Value<...>`` wrapper is stripped first, then ``List<...>``
+ (a ``CommaDelimitedList`` is a list of strings)."""
+ if ptype.startswith(_SSM_PARAMETER_VALUE_PREFIX) and ptype.endswith(">"):
+ ptype = ptype[len(_SSM_PARAMETER_VALUE_PREFIX):-1]
+ if ptype == "CommaDelimitedList":
+ return "String", True
+ if ptype.startswith("List<") and ptype.endswith(">"):
+ return ptype[len("List<"):-1], True
+ return ptype, False
+
+
+def _rule_param_value(name: str, defn: dict, params: dict):
+ """A parameter as ``Ref`` sees it inside Rules: list-typed parameters are
+ lists of their trimmed members, everything else the string value."""
+ value = params[name]["Value"]
+ _, is_list = _rule_inner_type(str(defn.get("Type", "String")))
+ if is_list:
+ return [m.strip() for m in str(value).split(",")] if value != "" else []
+ return str(value)
+
+
+def _rule_account_records(parameter_type: str):
+ """The EC2 records behind an AWS-specific parameter type, or
+ ``_RuleUnsupported`` for a type this emulator does not list."""
+ spec = _RULE_ACCOUNT_TYPES.get(parameter_type)
+ if spec is None:
+ raise _RuleUnsupported(
+ f"Fn::RefAll / Fn::ValueOfAll over {parameter_type} is not supported")
+ from ministack.services import ec2
+ store = getattr(ec2, spec[0])
+ return {rid: store[rid] for rid in list(store)}, spec[1]
+
+
+def _rule_attribute(parameter_type: str, resource_id: str, attribute: str) -> str:
+ """One attribute of one account resource, as ``Fn::ValueOf`` returns it;
+ a resource the store does not hold yields an empty string."""
+ from ministack.services import ec2
+ records, attributes = _rule_account_records(parameter_type)
+ record = records.get(resource_id) or {}
+ if attribute.startswith("Tags."):
+ key = attribute[len("Tags."):]
+ for tag in ec2._tags.get(resource_id, []) or []:
+ if tag.get("Key") == key:
+ return str(tag.get("Value", ""))
+ return ""
+ field = attributes.get(attribute)
+ if field is None:
+ raise _RuleUnsupported(
+ f"attribute {attribute} of {parameter_type} is not supported")
+ return str(record.get(field, ""))
+
+
+def _evaluate_rules(template: dict, params: dict, conditions: dict) -> None:
+ """Evaluate the ``Rules`` section against the resolved parameters.
+
+ The Rules reference: a rule's ``Assertions`` are checked when its
+ ``RuleCondition`` is absent or evaluates to true; every ``Assert`` must
+ evaluate to true or the stack is neither created nor updated, and the
+ ``AssertDescription`` is the message. ``Fn::RefAll``, ``Fn::ValueOf`` and
+ ``Fn::ValueOfAll`` are served for ``AWS::EC2::VPC::Id``,
+ ``AWS::EC2::Subnet::Id`` and ``AWS::EC2::SecurityGroup::Id`` from the
+ in-process EC2 store; a rule over any other type is skipped with a
+ warning. Raises ``ValueError`` with the message the caller wraps as a
+ ``ValidationError`` (the failure text is unmeasured).
+ """
+ rules = template.get("Rules")
+ if not rules:
+ return
+ param_defs = template.get("Parameters") or {}
+ pseudo = {
+ "AWS::Region": get_region(),
+ "AWS::AccountId": get_account_id(),
+ "AWS::URLSuffix": "amazonaws.com",
+ "AWS::Partition": "aws",
+ }
+
+ def as_list(value, fn):
+ if not isinstance(value, list):
+ raise ValueError(
+ f"Template error: {fn} expects a list as its first argument, got "
+ f"'{value}'")
+ return [str(v) for v in value]
+
+ def as_bool(value, fn):
+ if not isinstance(value, bool):
+ raise ValueError(
+ f"Template error: every argument of {fn} must evaluate to true or "
+ f"false, got '{value}'")
+ return value
+
+ def ev(node):
+ if not isinstance(node, dict) or len(node) != 1:
+ return [ev(v) for v in node] if isinstance(node, list) else node
+ fn, args = next(iter(node.items()))
+ if fn == "Ref":
+ if args in params:
+ return _rule_param_value(args, param_defs.get(args, {}), params)
+ if args in pseudo:
+ return pseudo[args]
+ raise ValueError(
+ f"Template format error: Unresolved resource dependencies [{args}] "
+ "in the Rules block of the template")
+ if fn == "Fn::Equals":
+ left, right = ev(args[0]), ev(args[1])
+ if isinstance(left, list) or isinstance(right, list):
+ return left == right
+ return str(left) == str(right)
+ if fn == "Fn::And":
+ return all(as_bool(ev(a), fn) for a in args)
+ if fn == "Fn::Or":
+ return any(as_bool(ev(a), fn) for a in args)
+ if fn == "Fn::Not":
+ return not as_bool(ev(args[0]), fn)
+ if fn == "Fn::If":
+ chosen = ev(args[0])
+ if isinstance(args[0], str):
+ if args[0] not in conditions:
+ raise ValueError(
+ f"Template error: Fn::If refers to condition {args[0]} "
+ "which is not defined in the Conditions block")
+ chosen = conditions[args[0]]
+ return ev(args[1]) if as_bool(chosen, fn) else ev(args[2])
+ if fn == "Fn::Contains":
+ return str(ev(args[1])) in as_list(ev(args[0]), fn)
+ if fn == "Fn::EachMemberEquals":
+ wanted = str(ev(args[1]))
+ return all(m == wanted for m in as_list(ev(args[0]), fn))
+ if fn == "Fn::EachMemberIn":
+ allowed = as_list(ev(args[1]), fn)
+ return all(m in allowed for m in as_list(ev(args[0]), fn))
+ if fn == "Fn::RefAll":
+ records, _ = _rule_account_records(str(args))
+ return list(records)
+ if fn == "Fn::ValueOfAll":
+ ptype, attribute = str(args[0]), str(args[1])
+ records, _ = _rule_account_records(ptype)
+ return [_rule_attribute(ptype, rid, attribute) for rid in records]
+ if fn == "Fn::ValueOf":
+ pname, attribute = str(args[0]), str(args[1])
+ if pname not in params:
+ raise ValueError(
+ f"Template format error: Unresolved resource dependencies "
+ f"[{pname}] in the Rules block of the template")
+ base, _ = _rule_inner_type(str(param_defs.get(pname, {}).get("Type", "String")))
+ value = _rule_param_value(pname, param_defs.get(pname, {}), params)
+ if isinstance(value, list):
+ return [_rule_attribute(base, v, attribute) for v in value]
+ return _rule_attribute(base, value, attribute)
+ raise ValueError(
+ "Template format error: Following functions are not supported in the "
+ f"Rules block of the template: [{fn}]")
+
+ for name, rule in rules.items():
+ try:
+ condition = rule.get("RuleCondition")
+ if condition is not None:
+ chosen = ev(condition)
+ if not isinstance(chosen, bool):
+ raise ValueError(
+ f"Template error: the RuleCondition of rule {name} must "
+ f"evaluate to true or false, got '{chosen}'")
+ if not chosen:
+ continue
+ for index, assertion in enumerate(rule["Assertions"], 1):
+ result = ev(assertion["Assert"])
+ if not isinstance(result, bool):
+ raise ValueError(
+ f"Template error: assertion {index} of rule {name} must "
+ f"evaluate to true or false, got '{result}'")
+ if not result:
+ description = assertion.get("AssertDescription") \
+ or f"assertion {index} evaluated to false"
+ raise ValueError(f"Template error: rule {name} failed: {description}")
+ except _RuleUnsupported as exc:
+ logger.warning("Rules: skipping rule %s: %s", name, exc)
+ except (TypeError, IndexError, KeyError) as exc:
+ # The shape check above refuses what it knows; anything that still
+ # does not evaluate is a template error, never an internal one.
+ raise ValueError(
+ f"Template format error: rule {name} could not be evaluated: "
+ f"{exc}") from exc
+
+
# ===========================================================================
# Intrinsic Function Resolver
# ===========================================================================
@@ -602,6 +1008,7 @@ def _validate_template_statics(template: dict) -> None:
for section in ("Resources", "Outputs", "Conditions"):
walk(template.get(section) or {})
+ _check_rules_section(template)
def _unknown_attribute_message(res: dict, logical_id: str, attr: str) -> str:
@@ -999,6 +1406,146 @@ def _topological_sort(resources: dict, conditions: dict) -> list:
return result
+# ===========================================================================
+# AWS::Include transform
+# ===========================================================================
+
+_INCLUDE_TRANSFORM = "AWS::Include"
+_INCLUDE_ERROR = "Transform AWS::Include failed with: "
+# The reference: the transform may be used anywhere except the Parameters
+# section and the template version; the top-level Transform form is not
+# expanded here.
+_INCLUDE_SKIPPED_SECTIONS = ("Parameters", "AWSTemplateFormatVersion", "Transform")
+
+
+class _IncludeError(ValueError):
+ """An ``AWS::Include`` that could not be expanded; the message is what the
+ caller answers, unwrapped."""
+
+
+class _IncludeFailed(dict):
+ """The template as sent, carrying the message of an include that failed.
+
+ The handlers apply the transforms in one step and wrap everything that
+ step raises as ``Template format error: ...``; a real account answers
+ the include sentence bare (``Transform AWS::Include failed with: ...``),
+ so the failure travels with the template to ``validate_template_support``,
+ which raises it as is. UpdateStack with ``UsePreviousTemplate`` swaps in
+ the stored processed template before that point, so a snippet that went
+ missing since the deploy does not fail the update."""
+
+ def __init__(self, template: dict, error: str):
+ super().__init__(template)
+ self.error = error
+
+
+def _is_include(node) -> bool:
+ """True for a map that carries an ``Fn::Transform`` naming ``AWS::Include``."""
+ spec = node.get("Fn::Transform") if isinstance(node, dict) else None
+ return isinstance(spec, dict) and spec.get("Name") == _INCLUDE_TRANSFORM
+
+
+def _include_location(node: dict):
+ """The ``Location`` parameter of an include node, or None."""
+ parameters = node["Fn::Transform"].get("Parameters")
+ return parameters.get("Location") if isinstance(parameters, dict) else None
+
+
+def _parse_snippet(text: str):
+ """A snippet is JSON or plain YAML: the reference says shorthand
+ notations are not supported in snippets, and a real account answers the
+ sentence below for a ``!Ref`` inside one (re:Post, "Include Parameters
+ in an AWS Include file"), so the plain SafeLoader is used, not the
+ template loader with its tag constructors."""
+ text = text.strip()
+ try:
+ return json.loads(text) if text.startswith("{") \
+ else yaml.load(text, Loader=yaml.SafeLoader)
+ except Exception as exc:
+ raise _IncludeError(
+ _INCLUDE_ERROR + "The specified S3 object's content should be valid "
+ "Yaml/JSON") from exc
+
+
+def _fetch_include_snippet(location) -> dict:
+ """The key-value object behind an ``s3://bucket/key`` location. The
+ location sentence and the content sentence are the ones a real account
+ answers; the missing-object, key-value and nesting sentences are
+ unmeasured."""
+ if not isinstance(location, str) or not location.startswith("s3://"):
+ raise _IncludeError(_INCLUDE_ERROR + "The location parameter is not a valid S3 uri.")
+ bucket, _, key = location[len("s3://"):].partition("/")
+ if not bucket or not key:
+ raise _IncludeError(_INCLUDE_ERROR + "The location parameter is not a valid S3 uri.")
+ from ministack.services import s3 as _s3
+ data = _s3._get_object_data(bucket, key)
+ if data is None:
+ raise _IncludeError(_INCLUDE_ERROR + f"The S3 object {location} does not exist.")
+ snippet = _parse_snippet(data.decode("utf-8"))
+ if not isinstance(snippet, dict):
+ raise _IncludeError(_INCLUDE_ERROR + f"The snippet at {location} must be a "
+ "key-value object.")
+ nested: list = []
+ _find_includes(snippet, nested)
+ if nested:
+ raise _IncludeError(_INCLUDE_ERROR + f"The snippet at {location} uses AWS::Include, "
+ "which cannot be nested.")
+ return snippet
+
+
+def _find_includes(node, found: list) -> None:
+ if isinstance(node, dict):
+ if _is_include(node):
+ found.append(node)
+ for value in node.values():
+ _find_includes(value, found)
+ elif isinstance(node, list):
+ for value in node:
+ _find_includes(value, found)
+
+
+def _apply_include_transform(template: dict) -> dict:
+ """Replace every embedded ``Fn::Transform`` that names ``AWS::Include``
+ by the contents of the S3 object it points to, the way the transform
+ reference describes: the snippet is inserted at the location of the
+ transform. A node that holds nothing but the transform becomes the
+ snippet; a node with other keys (``Resources`` next to a sibling
+ resource, a ``Properties`` map) takes the snippet's keys, the snippet
+ winning a duplicate key (unmeasured). The ``Parameters`` section and the
+ template version are not walked. An include that cannot be expanded
+ returns the template as ``_IncludeFailed``, which the validation step
+ turns into the ``ValidationError``."""
+ found: list = []
+ for section, value in template.items():
+ if section not in _INCLUDE_SKIPPED_SECTIONS:
+ _find_includes(value, found)
+ if not found:
+ return template
+
+ def walk(node):
+ if isinstance(node, list):
+ return [walk(v) for v in node]
+ if not isinstance(node, dict):
+ return node
+ if _is_include(node):
+ snippet = _fetch_include_snippet(_include_location(node))
+ rest = {k: walk(v) for k, v in node.items() if k != "Fn::Transform"}
+ if not rest:
+ return snippet
+ rest.update(snippet)
+ return rest
+ return {k: walk(v) for k, v in node.items()}
+
+ expanded = dict(template)
+ try:
+ for section, value in template.items():
+ if section not in _INCLUDE_SKIPPED_SECTIONS:
+ expanded[section] = walk(value)
+ except _IncludeError as exc:
+ return _IncludeFailed(template, str(exc))
+ return expanded
+
+
# ===========================================================================
# SAM Transform
# ===========================================================================
@@ -1011,6 +1558,12 @@ logging.getLogger("samtranslator.feature_toggle.feature_toggle").setLevel(loggin
_NO_IAM_POLICY_LOADER = SimpleNamespace(load=lambda: {})
def _apply_sam_transform_if_applicable(template: dict) -> dict:
+ """The transforms a stack operation applies before validation: every
+ embedded ``AWS::Include`` first, then the SAM transform when the template
+ declares it."""
+ template = _apply_include_transform(template)
+ if isinstance(template, _IncludeFailed):
+ return template
declared = template.get("Transform")
transforms = declared if isinstance(declared, list) else [declared]
if _SAM_TRANSFORM not in transforms:
diff --git a/ministack/services/cloudformation/handlers.py b/ministack/services/cloudformation/handlers.py
index d38c7b26..cee04756 100644
--- a/ministack/services/cloudformation/handlers.py
+++ b/ministack/services/cloudformation/handlers.py
@@ -36,6 +36,7 @@ from .helpers import (
_extract_string_members,
_p,
_page,
+ _request_problems,
_resolve_template,
_xml,
)
@@ -205,6 +206,9 @@ def _create_stack(params):
stack_name = _p(params, "StackName")
if not stack_name:
return _error("ValidationError", "StackName is required")
+ # The request-level constraints, joined into one message as the API does.
+ if request_error := _request_problems(params, stack_name):
+ return request_error
template_body, resolve_err = _resolve_template(params)
if resolve_err:
@@ -248,7 +252,7 @@ def _create_stack(params):
conditions = _evaluate_conditions(template, param_values)
try:
- validate_template_support(template, conditions)
+ validate_template_support(template, conditions, params=param_values)
except ValueError as exc:
return _error("ValidationError", str(exc))
@@ -883,9 +887,15 @@ def _update_stack(params):
except ValueError as exc:
return _error("ValidationError", str(exc))
+ if use_previous_template and stack.get("_template"):
+ # The stored template is the processed one: an AWS::Include snippet
+ # edited or removed in S3 since the deploy is not picked up (the
+ # transform reference: "your stack doesn't automatically pick up
+ # those changes").
+ template = copy.deepcopy(stack["_template"])
try:
validate_template_support(
- template, _evaluate_conditions(template, param_values))
+ template, _evaluate_conditions(template, param_values), params=param_values)
except ValueError as exc:
return _error("ValidationError", str(exc))
@@ -954,7 +964,9 @@ def _update_stack(params):
# --- ValidateTemplate ---
def _validate_template(params):
- template_body = _p(params, "TemplateBody")
+ template_body, resolve_err = _resolve_template(params)
+ if resolve_err:
+ return resolve_err
if not template_body:
return _error("ValidationError", "TemplateBody is required")
diff --git a/ministack/services/cloudformation/helpers.py b/ministack/services/cloudformation/helpers.py
index 1f96773e..6254a644 100644
--- a/ministack/services/cloudformation/helpers.py
+++ b/ministack/services/cloudformation/helpers.py
@@ -5,6 +5,7 @@ CloudFormation helpers — XML response formatting and parameter extraction util
"""
import logging
+import re
from html import escape as _esc
from urllib.parse import urlparse
@@ -14,6 +15,16 @@ logger = logging.getLogger("cloudformation")
CFN_NS = "http://cloudformation.amazonaws.com/doc/2010-05-08/"
+# The CloudFormation quotas page: the template body of a request and of an
+# S3 object behind TemplateURL. CreateStack documents the stack name as
+# "only alphanumeric characters (case sensitive) and hyphens", starting with
+# a letter and no longer than 128 characters.
+TEMPLATE_BODY_MAX_BYTES = 51200
+TEMPLATE_URL_MAX_BYTES = 1000000
+STACK_NAME_MAX_CHARS = 128
+STACK_NAME_PATTERN = "[a-zA-Z][-a-zA-Z0-9]*"
+STACK_NAME_RE = re.compile(STACK_NAME_PATTERN)
+
def _p(params, key, default=""):
"""Extract a single value from parsed query-string params."""
@@ -88,10 +99,74 @@ def _validate_stack_tags(tags):
+def validation_error_message(problems: list[str]) -> str:
+ """The API's parameter validation joins every request-level violation into
+ one message (quoted: ``2 validation errors detected: Value '' at
+ 'stackName' failed to satisfy constraint: ...; Value '' at 'stackName'
+ failed to satisfy constraint: ...``)."""
+ plural = "s" if len(problems) > 1 else ""
+ return f"{len(problems)} validation error{plural} detected: " + "; ".join(problems)
+
+
+def stack_name_problems(stack_name: str) -> list[str]:
+ """The constraints CreateStack documents for ``StackName``: alphanumeric
+ characters and hyphens, a letter first, at most 128 characters. One
+ sentence per violated constraint, in the API's parameter validation
+ wording (quoted for the pattern: ``Value 'x_y' at 'stackName' failed to
+ satisfy constraint: Member must satisfy regular expression pattern:
+ [a-zA-Z][-a-zA-Z0-9]*``; the length sentence follows the shape of other
+ API parameters, unmeasured)."""
+ problems = []
+ if len(stack_name) > STACK_NAME_MAX_CHARS:
+ problems.append(
+ f"Value '{stack_name}' at 'stackName' failed to satisfy constraint: Member "
+ f"must have length less than or equal to {STACK_NAME_MAX_CHARS}")
+ if not STACK_NAME_RE.fullmatch(stack_name):
+ problems.append(
+ f"Value '{stack_name}' at 'stackName' failed to satisfy constraint: Member "
+ f"must satisfy regular expression pattern: {STACK_NAME_PATTERN}")
+ return problems
+
+
+def _template_body_problem(body: str) -> str | None:
+ """The request-level constraint on ``TemplateBody`` (the quota page:
+ "Template body size in a request", 51,200 bytes), as one sentence of the
+ API's parameter validation, which echoes the value (quoted: ``Value
+ '<the template>' at 'templateBody' failed to satisfy constraint: Member
+ must have length less than or equal to 51200``); None when it holds.
+ CreateStack joins it with the stack name problems into one message."""
+ if body and len(body.encode("utf-8")) > TEMPLATE_BODY_MAX_BYTES:
+ return (f"Value '{body}' at 'templateBody' failed to satisfy constraint: Member "
+ f"must have length less than or equal to {TEMPLATE_BODY_MAX_BYTES}")
+ return None
+
+
+def _request_problems(params, stack_name: str = ""):
+ """The request-level constraints of a call that carries a template: the
+ stack name where the call names one, and the size of an inline
+ ``TemplateBody``. The API reports every violation of a request in one
+ message, so both are collected before it answers. Returns the error
+ response, or None when the request passes."""
+ problems = stack_name_problems(stack_name) if stack_name else []
+ body_problem = _template_body_problem(_p(params, "TemplateBody"))
+ if body_problem:
+ problems.append(body_problem)
+ if problems:
+ return _error("ValidationError", validation_error_message(problems))
+ return None
+
+
def _resolve_template(params):
"""Resolve TemplateBody or TemplateURL to a template string.
If TemplateURL is provided, fetch the template from S3.
- Returns (template_body, error_tuple) — error_tuple is None on success."""
+ Returns (template_body, error_tuple) — error_tuple is None on success.
+
+ The request-level constraints are checked here, so every action that reads
+ a template applies them. CreateStack and a CREATE change set check them
+ earlier, together with the stack name, and return before this point."""
+ request_error = _request_problems(params)
+ if request_error:
+ return None, request_error
return _resolve_document(params, "TemplateBody", "TemplateURL", "Template")
@@ -121,6 +196,13 @@ def _resolve_document(params, body_key, url_key, label):
obj_data = _s3._get_object_data(bucket_name, key)
if obj_data is None:
return None, _error("ValidationError", f"{label} not found at {url}")
+ if body_key == "TemplateBody" and len(obj_data) > TEMPLATE_URL_MAX_BYTES:
+ # "Template body size in an Amazon S3 object", 1 MB on the quota
+ # page; the service enforces 1,000,000 bytes and answers with
+ # this sentence (quoted from reports of the error).
+ return None, _error(
+ "ValidationError",
+ f"Template may not exceed {TEMPLATE_URL_MAX_BYTES} bytes in size.")
return obj_data.decode("utf-8"), None
except Exception as e:
logger.warning("Failed to fetch %s %s: %s", url_key, url, e)
diff --git a/tests/test_cfn.py b/tests/test_cfn.py
index c3745ca8..a4263de3 100644
--- a/tests/test_cfn.py
+++ b/tests/test_cfn.py
@@ -18015,3 +18015,728 @@ def test_cfn_update_stack_without_changes_is_refused(cfn, sqs):
finally:
_delete_cfn_test_stack(cfn, stack_name)
+
+
+# ---------------------------------------------------------------------------
+# Rules section
+# ---------------------------------------------------------------------------
+
+def _rules_template(rules, params):
+ """A template with the given Rules and Parameters and one resource that
+ provisions instantly."""
+ return json.dumps({
+ "Parameters": params,
+ "Rules": rules,
+ "Resources": {"Handle": {"Type": "AWS::CloudFormation::WaitConditionHandle"}},
+ })
+
+
+def _refused_by_rules(cfn, stack_name, body, parameters, expected):
+ with pytest.raises(ClientError) as exc:
+ cfn.create_stack(StackName=stack_name, TemplateBody=body, Parameters=parameters)
+ assert exc.value.response["Error"]["Code"] == "ValidationError"
+ assert exc.value.response["Error"]["Message"] == expected
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=stack_name)
+
+
+def test_cfn_rules_cdk_check_bootstrap_version(cfn, ssm):
+ """The rule every CDK-synthesized template carries: `BootstrapVersion` is an
+ `AWS::SSM::Parameter::Value<String>` and the assertion refuses the values
+ 1 to 5. The rule passes when the SSM parameter holds "30" and refuses the
+ stack with the AssertDescription when it holds "5", on CreateStack,
+ UpdateStack and CreateChangeSet (both types), and no stack record is left
+ behind. The failure text is unmeasured."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ ssm_name = f"/cdk-bootstrap/{uid}/version"
+ description = ("CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' "
+ "with a recent version of the CDK CLI.")
+ template = json.dumps({
+ "Parameters": {"BootstrapVersion": {
+ "Type": "AWS::SSM::Parameter::Value<String>", "Default": ssm_name,
+ "Description": "Version of the CDK Bootstrap resources in this environment, "
+ "automatically retrieved from SSM Parameter Store."}},
+ "Rules": {"CheckBootstrapVersion": {"Assertions": [{
+ "Assert": {"Fn::Not": [{"Fn::Contains": [
+ ["1", "2", "3", "4", "5"], {"Ref": "BootstrapVersion"}]}]},
+ "AssertDescription": description}]}},
+ "Resources": {"Handle": {"Type": "AWS::CloudFormation::WaitConditionHandle"}},
+ })
+ expected = f"Template error: rule CheckBootstrapVersion failed: {description}"
+ stack_name = f"cfn-rules-cdk-{uid}"
+
+ ssm.put_parameter(Name=ssm_name, Value="5", Type="String")
+ try:
+ _refused_by_rules(cfn, stack_name, template, [], expected)
+ with pytest.raises(ClientError) as exc:
+ cfn.create_change_set(StackName=stack_name, ChangeSetName="cs",
+ ChangeSetType="CREATE", TemplateBody=template)
+ assert exc.value.response["Error"]["Message"] == expected
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=stack_name)
+
+ ssm.put_parameter(Name=ssm_name, Value="30", Type="String", Overwrite=True)
+ cfn.create_stack(StackName=stack_name, TemplateBody=template)
+ stack = _wait_stack(cfn, stack_name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+
+ ssm.put_parameter(Name=ssm_name, Value="5", Type="String", Overwrite=True)
+ with pytest.raises(ClientError) as exc:
+ cfn.update_stack(StackName=stack_name, TemplateBody=template,
+ Tags=[{"Key": "stage", "Value": "two"}])
+ assert exc.value.response["Error"]["Code"] == "ValidationError"
+ assert exc.value.response["Error"]["Message"] == expected
+ # UsePreviousTemplate re-resolves the SSM parameter and re-runs the rule.
+ with pytest.raises(ClientError) as exc:
+ cfn.update_stack(StackName=stack_name, UsePreviousTemplate=True,
+ Parameters=[{"ParameterKey": "BootstrapVersion",
+ "UsePreviousValue": True}],
+ Tags=[{"Key": "stage", "Value": "two"}])
+ assert exc.value.response["Error"]["Message"] == expected
+ with pytest.raises(ClientError) as exc:
+ cfn.create_change_set(StackName=stack_name, ChangeSetName="cs",
+ TemplateBody=template, Tags=[{"Key": "stage", "Value": "two"}])
+ assert exc.value.response["Error"]["Message"] == expected
+ assert cfn.describe_stacks(StackName=stack_name)["Stacks"][0]["StackStatus"] \
+ == "CREATE_COMPLETE"
+ assert cfn.list_change_sets(StackName=stack_name)["Summaries"] == []
+
+ ssm.put_parameter(Name=ssm_name, Value="31", Type="String", Overwrite=True)
+ cfn.update_stack(StackName=stack_name, TemplateBody=template,
+ Tags=[{"Key": "stage", "Value": "two"}])
+ stack = _wait_stack(cfn, stack_name)
+ assert stack["StackStatus"] == "UPDATE_COMPLETE", stack.get("StackStatusReason")
+ finally:
+ _delete_cfn_test_stack(cfn, stack_name)
+ try:
+ ssm.delete_parameter(Name=ssm_name)
+ except ClientError:
+ pass
+
+
+def test_cfn_rules_condition_and_list_functions(cfn):
+ """A rule whose RuleCondition is false is skipped; Fn::Contains,
+ Fn::EachMemberEquals and Fn::EachMemberIn see a CommaDelimitedList
+ parameter as a list of trimmed members; Fn::And, Fn::Or, Fn::Equals and
+ Fn::If nest; the pseudo parameters resolve."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ params = {
+ "Env": {"Type": "String", "Default": "test"},
+ "Size": {"Type": "String", "Default": "t3.medium"},
+ "Subnets": {"Type": "CommaDelimitedList", "Default": "a, b"},
+ "Zones": {"Type": "CommaDelimitedList", "Default": "us-east-1a,us-east-1b"},
+ }
+ rules = {
+ "ProdOnly": { # RuleCondition false: the failing assertion is never checked
+ "RuleCondition": {"Fn::Equals": [{"Ref": "Env"}, "prod"]},
+ "Assertions": [{"Assert": {"Fn::Equals": [{"Ref": "Size"}, "t3.large"]},
+ "AssertDescription": "prod needs t3.large"}]},
+ "TestSize": {
+ "RuleCondition": {"Fn::Equals": [{"Ref": "Env"}, "test"]},
+ "Assertions": [
+ {"Assert": {"Fn::Contains": [["t3.medium", "t3.small"], {"Ref": "Size"}]},
+ "AssertDescription": "test needs t3.medium or t3.small"},
+ {"Assert": {"Fn::EachMemberIn": [{"Ref": "Subnets"}, ["a", "b", "c"]]}},
+ {"Assert": {"Fn::EachMemberEquals": [{"Ref": "Subnets"}, "a"]},
+ "AssertDescription": "all subnets must be a"}]},
+ "Nested": {"Assertions": [{"Assert": {"Fn::And": [
+ {"Fn::Or": [{"Fn::Equals": [{"Ref": "AWS::Region"}, "us-east-1"]},
+ {"Fn::Equals": [{"Ref": "AWS::AccountId"}, "000000000000"]}]},
+ {"Fn::Not": [{"Fn::Equals": [{"Ref": "AWS::Partition"}, "aws-cn"]}]},
+ {"Fn::Contains": [{"Ref": "Zones"},
+ {"Fn::If": [{"Fn::Equals": [{"Ref": "Env"}, "test"]},
+ "us-east-1b", "us-east-1z"]}]},
+ ]}}]},
+ }
+ body = _rules_template(rules, params)
+ name = f"cfn-rules-fn-{uid}"
+
+ _refused_by_rules(cfn, name, body, [{"ParameterKey": "Subnets", "ParameterValue": "a,b,z"}],
+ "Template error: rule TestSize failed: assertion 2 evaluated to false")
+ _refused_by_rules(cfn, name, body, [{"ParameterKey": "Subnets", "ParameterValue": "a,b"},
+ {"ParameterKey": "Size", "ParameterValue": "t3.large"}],
+ "Template error: rule TestSize failed: test needs t3.medium or t3.small")
+ _refused_by_rules(cfn, name, body, [{"ParameterKey": "Env", "ParameterValue": "prod"}],
+ "Template error: rule ProdOnly failed: prod needs t3.large")
+
+ cfn.create_stack(StackName=name, TemplateBody=body,
+ Parameters=[{"ParameterKey": "Subnets", "ParameterValue": "a, a"}])
+ try:
+ stack = _wait_stack(cfn, name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ finally:
+ _delete_cfn_test_stack(cfn, name)
+
+
+def test_cfn_rules_shape_and_function_errors(cfn):
+ """An assertion that is not a boolean, a RuleCondition that is not a
+ boolean, a Ref to something that is not a parameter, a function outside
+ the rule set (measured for ``[Fn::If]`` in cloudformation-coverage-roadmap
+ issue 921; the bracket lists the offending names), a rule without
+ Assertions and a malformed argument list are refused before a stack
+ exists with a ValidationError, never an internal error; the function
+ and shape checks also run on ValidateTemplate."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ params = {"P": {"Type": "String", "Default": "x"}}
+ name = f"cfn-rules-shape-{uid}"
+
+ _refused_by_rules(
+ cfn, name, _rules_template({"R": {"Assertions": [{"Assert": {"Ref": "P"}}]}}, params),
+ [], "Template error: assertion 1 of rule R must evaluate to true or false, got 'x'")
+ _refused_by_rules(
+ cfn, name, _rules_template(
+ {"R": {"Assertions": [{"Assert": {"Fn::Equals": [{"Ref": "Handle"}, "x"]}}]}},
+ params),
+ [], "Template format error: Unresolved resource dependencies [Handle] in the Rules "
+ "block of the template")
+ unsupported = _rules_template(
+ {"R": {"Assertions": [{"Assert": {"Fn::Equals": [{"Fn::Sub": "${P}"}, "x"]}}]}}, params)
+ expected = ("Template format error: Following functions are not supported in the Rules "
+ "block of the template: [Fn::Sub]")
+ _refused_by_rules(cfn, name, unsupported, [], expected)
+ with pytest.raises(ClientError) as exc:
+ cfn.validate_template(TemplateBody=unsupported)
+ assert exc.value.response["Error"]["Message"] == expected
+ _refused_by_rules(cfn, name, _rules_template({"R": {"RuleCondition": True}}, params), [],
+ "Template format error: Rule R must contain an Assertions list")
+ _refused_by_rules(
+ cfn, name, _rules_template({"R": {"RuleCondition": {"Ref": "P"},
+ "Assertions": [{"Assert": True}]}}, params),
+ [], "Template error: the RuleCondition of rule R must evaluate to true or false, got 'x'")
+ for assertion, expected in (
+ ({"Fn::Equals": "x"},
+ "Template format error: Fn::Equals in rule R must be a list of 2 elements"),
+ ({"Fn::Equals": [{"Ref": "P"}]},
+ "Template format error: Fn::Equals in rule R must be a list of 2 elements"),
+ ({"Ref": ["P"]}, "Template format error: Ref in rule R must name a parameter"),
+ ({"Fn::ValueOf": "P"},
+ "Template format error: Fn::ValueOf in rule R must be a list of 2 elements"),
+ ({"Fn::ValueOf": [{"Ref": "P"}, "VpcId"]},
+ "Template format error: Fn::ValueOf in rule R takes two strings; no function "
+ "can be used within it"),
+ ({"Fn::And": None},
+ "Template format error: Fn::And in rule R must be a list of conditions"),
+ ({"Fn::Not": [{"Fn::Equals": [{"Ref": "P"}, "x"]}, True]},
+ "Template format error: Fn::Not in rule R must be a list of 1 elements"),
+ ):
+ body = _rules_template({"R": {"Assertions": [{"Assert": assertion}]}}, params)
+ _refused_by_rules(cfn, name, body, [], expected)
+ with pytest.raises(ClientError) as exc:
+ cfn.validate_template(TemplateBody=body)
+ assert exc.value.response["Error"]["Message"] == expected
+ # A template whose rules pass validates; ValidateTemplate has no values to
+ # evaluate against, so a rule that would fail on CreateStack passes there.
+ failing = _rules_template(
+ {"R": {"Assertions": [{"Assert": {"Fn::Equals": [{"Ref": "P"}, "y"]},
+ "AssertDescription": "P must be y"}]}}, params)
+ cfn.validate_template(TemplateBody=failing)
+ _refused_by_rules(cfn, name, failing, [], "Template error: rule R failed: P must be y")
+
+
+def test_cfn_rules_account_lookups(cfn, ec2, ssm):
+ """Fn::RefAll, Fn::ValueOf and Fn::ValueOfAll are served from the EC2 store
+ for the VPC, subnet and security-group parameter types, with every
+ attribute the reference lists, also behind an SSM ``Value<List<...>>``
+ parameter type; a rule over any other type is skipped (logged), not
+ failed."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ vpc_id = ec2.create_vpc(CidrBlock="10.77.0.0/16", TagSpecifications=[{
+ "ResourceType": "vpc", "Tags": [{"Key": "Department", "Value": "IT"}]}])["Vpc"]["VpcId"]
+ subnet_ids, other_vpc, other_subnet = [], None, None
+ try:
+ subnet_ids = [
+ ec2.create_subnet(VpcId=vpc_id, CidrBlock=f"10.77.{i}.0/24",
+ AvailabilityZone="us-east-1a")["Subnet"]["SubnetId"]
+ for i in (1, 2)]
+ other_vpc = ec2.create_vpc(CidrBlock="10.78.0.0/16")["Vpc"]["VpcId"]
+ other_subnet = ec2.create_subnet(
+ VpcId=other_vpc, CidrBlock="10.78.1.0/24")["Subnet"]["SubnetId"]
+ vpc = {
+ "DefaultSecurityGroupId": ec2.describe_security_groups(Filters=[
+ {"Name": "vpc-id", "Values": [vpc_id]},
+ {"Name": "group-name", "Values": ["default"]}])["SecurityGroups"][0]["GroupId"],
+ "DefaultNetworkAclId": ec2.describe_network_acls(Filters=[
+ {"Name": "vpc-id", "Values": [vpc_id]}])["NetworkAcls"][0]["NetworkAclId"],
+ }
+ ssm_name = f"/cfn-rules/{uid}/subnets"
+ ssm.put_parameter(Name=ssm_name, Value=",".join(subnet_ids), Type="StringList")
+ _rules_account_lookups(cfn, uid, vpc_id, subnet_ids, other_vpc, other_subnet,
+ vpc, ssm_name)
+ finally:
+ for sid in subnet_ids + ([other_subnet] if other_subnet else []):
+ ec2.delete_subnet(SubnetId=sid)
+ ec2.delete_vpc(VpcId=vpc_id)
+ if other_vpc:
+ ec2.delete_vpc(VpcId=other_vpc)
+ try:
+ ssm.delete_parameter(Name=f"/cfn-rules/{uid}/subnets")
+ except ClientError:
+ pass
+
+
+def _rules_account_lookups(cfn, uid, vpc_id, subnet_ids, other_vpc, other_subnet, vpc, ssm_name):
+ params = {
+ "VpcId": {"Type": "AWS::EC2::VPC::Id"},
+ "Subnets": {"Type": "List<AWS::EC2::Subnet::Id>"},
+ "SsmSubnets": {"Type": "AWS::SSM::Parameter::Value<List<AWS::EC2::Subnet::Id>>",
+ "Default": ssm_name},
+ "KeyName": {"Type": "AWS::EC2::KeyPair::KeyName", "Default": "any"},
+ }
+ rules = {
+ "VpcExists": {"Assertions": [{
+ "Assert": {"Fn::Contains": [{"Fn::RefAll": "AWS::EC2::VPC::Id"}, {"Ref": "VpcId"}]},
+ "AssertDescription": "The VPC must exist"}]},
+ "Department": {"Assertions": [{
+ "Assert": {"Fn::Equals": [{"Fn::ValueOf": ["VpcId", "Tags.Department"]}, "IT"]},
+ "AssertDescription": "The VPC must belong to IT"}]},
+ "SubnetsInVpc": {"Assertions": [{
+ "Assert": {"Fn::EachMemberEquals": [{"Fn::ValueOf": ["Subnets", "VpcId"]},
+ {"Ref": "VpcId"}]},
+ "AssertDescription": "All subnets must be in the VPC"}]},
+ "VpcAttributes": {"Assertions": [
+ {"Assert": {"Fn::Equals": [{"Fn::ValueOf": ["VpcId", "DefaultSecurityGroup"]},
+ vpc["DefaultSecurityGroupId"]]}},
+ {"Assert": {"Fn::Equals": [{"Fn::ValueOf": ["VpcId", "DefaultNetworkAcl"]},
+ vpc["DefaultNetworkAclId"]]}},
+ {"Assert": {"Fn::EachMemberEquals": [{"Fn::ValueOf": ["Subnets", "AvailabilityZone"]},
+ "us-east-1a"]},
+ "AssertDescription": "All subnets must be in us-east-1a"}]},
+ "SubnetsExist": {"Assertions": [{
+ "Assert": {"Fn::EachMemberIn": [{"Ref": "Subnets"},
+ {"Fn::RefAll": "AWS::EC2::Subnet::Id"}]}}]},
+ "SsmSubnetsInVpc": {"Assertions": [{
+ "Assert": {"Fn::EachMemberEquals": [{"Fn::ValueOf": ["SsmSubnets", "VpcId"]},
+ {"Ref": "VpcId"}]},
+ "AssertDescription": "All SSM subnets must be in the VPC"}]},
+ "SomeVpcIsIT": {"Assertions": [{
+ "Assert": {"Fn::Contains": [{"Fn::ValueOfAll": ["AWS::EC2::VPC::Id", "Tags.Department"]},
+ "IT"]}}]},
+ "Skipped": {"Assertions": [{ # key pairs are not listed: skipped, never failed
+ "Assert": {"Fn::Contains": [{"Fn::RefAll": "AWS::EC2::KeyPair::KeyName"},
+ {"Ref": "KeyName"}]}}]},
+ }
+ body = _rules_template(rules, params)
+ name = f"cfn-rules-account-{uid}"
+
+ def parameters(vpc, subnets):
+ return [{"ParameterKey": "VpcId", "ParameterValue": vpc},
+ {"ParameterKey": "Subnets", "ParameterValue": ",".join(subnets)}]
+
+ _refused_by_rules(cfn, name, body, parameters("vpc-0000000000000dead", subnet_ids),
+ "Template error: rule VpcExists failed: The VPC must exist")
+ _refused_by_rules(cfn, name, body, parameters(vpc_id, subnet_ids + [other_subnet]),
+ "Template error: rule SubnetsInVpc failed: All subnets must be in the VPC")
+ _refused_by_rules(cfn, name, body, parameters(other_vpc, [other_subnet]),
+ "Template error: rule Department failed: The VPC must belong to IT")
+
+ cfn.create_stack(StackName=name, TemplateBody=body, Parameters=parameters(vpc_id, subnet_ids))
+ try:
+ stack = _wait_stack(cfn, name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ finally:
+ _delete_cfn_test_stack(cfn, name)
+
+
+# ---------------------------------------------------------------------------
+# Template and stack quotas
+# ---------------------------------------------------------------------------
+
+_HANDLE = {"Type": "AWS::CloudFormation::WaitConditionHandle"}
+
+
+def _refused_quota(cfn, stack_name, body, expected, parameters=None):
+ with pytest.raises(ClientError) as exc:
+ cfn.create_stack(StackName=stack_name, TemplateBody=body, Parameters=parameters or [])
+ assert exc.value.response["Error"]["Code"] == "ValidationError"
+ message = exc.value.response["Error"]["Message"]
+ if callable(expected):
+ assert expected(message), message
+ else:
+ assert message == expected
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=stack_name)
+
+
+def test_cfn_template_body_size_quotas(cfn, s3):
+ """A TemplateBody of 51,200 bytes is accepted and one of 51,201 refused by
+ the request-level constraint (quoted: ``at 'templateBody' failed to
+ satisfy constraint: Member must have length less than or equal to
+ 51200``), on CreateStack, UpdateStack, ValidateTemplate and
+ GetTemplateSummary; a template behind TemplateURL may not exceed
+ 1,000,000 bytes (quoted: ``Template may not exceed 1000000 bytes in
+ size.``). ValidateTemplate reads its template through the same path, so it
+ takes a TemplateURL now."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ name = f"cfn-body-size-{uid}"
+
+ def body_of(size):
+ skeleton = json.dumps({"Metadata": {"Pad": ""}, "Resources": {"Handle": _HANDLE}})
+ return json.dumps({"Metadata": {"Pad": "x" * (size - len(skeleton))},
+ "Resources": {"Handle": _HANDLE}})
+
+ assert len(body_of(51200).encode()) == 51200
+ too_big = body_of(51201)
+
+ def refused(message):
+ return (message.startswith("1 validation error detected: Value '")
+ and message.endswith("' at 'templateBody' failed to satisfy constraint: "
+ "Member must have length less than or equal to 51200"))
+
+ _refused_quota(cfn, name, too_big, refused)
+ with pytest.raises(ClientError) as exc:
+ cfn.validate_template(TemplateBody=too_big)
+ assert refused(exc.value.response["Error"]["Message"])
+ with pytest.raises(ClientError) as exc:
+ cfn.get_template_summary(TemplateBody=too_big)
+ assert refused(exc.value.response["Error"]["Message"])
+
+ cfn.create_stack(StackName=name, TemplateBody=body_of(51200))
+ try:
+ stack = _wait_stack(cfn, name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ with pytest.raises(ClientError) as exc:
+ cfn.update_stack(StackName=name, TemplateBody=too_big)
+ assert refused(exc.value.response["Error"]["Message"])
+ finally:
+ _delete_cfn_test_stack(cfn, name)
+
+ bucket = f"cfn-body-size-{uid}"
+ s3.create_bucket(Bucket=bucket)
+ try:
+ endpoint = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566")
+ s3.put_object(Bucket=bucket, Key="big.json", Body=body_of(1000001).encode())
+ s3.put_object(Bucket=bucket, Key="ok.json", Body=body_of(1000000).encode())
+ s3.put_object(Bucket=bucket, Key="validate.json", Body=json.dumps({
+ "Description": "a template behind a URL",
+ "Parameters": {"P": {"Type": "String", "Default": "d"}},
+ "Resources": {"Handle": _HANDLE}}).encode())
+ # ValidateTemplate took TemplateBody only; it reads the same path now.
+ validated = cfn.validate_template(TemplateURL=f"{endpoint}/{bucket}/validate.json")
+ assert validated["Description"] == "a template behind a URL"
+ assert [p["ParameterKey"] for p in validated["Parameters"]] == ["P"]
+ with pytest.raises(ClientError) as exc:
+ cfn.create_stack(StackName=name, TemplateURL=f"{endpoint}/{bucket}/big.json")
+ assert exc.value.response["Error"]["Code"] == "ValidationError"
+ assert exc.value.response["Error"]["Message"] == \
+ "Template may not exceed 1000000 bytes in size."
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=name)
+ cfn.create_stack(StackName=name, TemplateURL=f"{endpoint}/{bucket}/ok.json")
+ try:
+ stack = _wait_stack(cfn, name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ finally:
+ _delete_cfn_test_stack(cfn, name)
+ finally:
+ for key in ("big.json", "ok.json", "validate.json"):
+ s3.delete_object(Bucket=bucket, Key=key)
+ s3.delete_bucket(Bucket=bucket)
+
+
+def test_cfn_template_section_quotas(cfn):
+ """501 resources (quoted: ``Template format error: Number of resources,
+ 536, is greater than maximum allowed, 500``), 201 parameters, outputs or
+ mappings, 201 attributes in a mapping, a 256-character name of any
+ section, a 256-character mapping attribute name, a 1,025-byte
+ description, a parameter default over 4,096 bytes (quoted shape) and a
+ provided value over 4,096 bytes are refused before a stack exists; the
+ maximum of each passes the check in process."""
+ from ministack.services.cloudformation.engine import validate_template_support
+
+ uid = _uuid_mod.uuid4().hex[:8]
+ name = f"cfn-quotas-{uid}"
+
+ def template(**sections):
+ return {"Resources": {"Handle": _HANDLE}, **sections}
+
+ resources = {f"H{i}": _HANDLE for i in range(501)}
+ _refused_quota(cfn, name, json.dumps({"Resources": resources}),
+ "Template format error: Number of resources, 501, is greater than "
+ "maximum allowed, 500")
+ params = {f"P{i}": {"Type": "String", "Default": "x"} for i in range(201)}
+ _refused_quota(cfn, name, json.dumps(template(Parameters=params)),
+ "Template format error: Number of parameters, 201, is greater than "
+ "maximum allowed, 200")
+ outputs = {f"O{i}": {"Value": "x"} for i in range(201)}
+ _refused_quota(cfn, name, json.dumps(template(Outputs=outputs)),
+ "Template format error: Number of outputs, 201, is greater than "
+ "maximum allowed, 200")
+ mappings = {f"M{i}": {"a": {"b": "c"}} for i in range(201)}
+ _refused_quota(cfn, name, json.dumps(template(Mappings=mappings)),
+ "Template format error: Number of mappings, 201, is greater than "
+ "maximum allowed, 200")
+ wide = {"M": {f"a{i}": {"b": "c"} for i in range(201)}}
+ _refused_quota(cfn, name, json.dumps(template(Mappings=wide)),
+ "Template format error: Number of attributes in mapping M, 201, is "
+ "greater than maximum allowed, 200")
+ long_name = "R" * 256
+ _refused_quota(cfn, name, json.dumps({"Resources": {long_name: _HANDLE}}),
+ f"Template format error: Resource name {'R' * 32}... may not exceed "
+ "255 characters")
+ _refused_quota(cfn, name, json.dumps(template(Description="d" * 1025)),
+ "Template format error: Template description may not exceed 1024 "
+ "bytes in size")
+ big_default = "v" * 4097
+ _refused_quota(cfn, name, json.dumps(template(
+ Parameters={"P": {"Type": "String", "Default": big_default}})),
+ f"Template format error: Parameter 'P' default value '{big_default}' length is "
+ "greater than 4096.")
+ _refused_quota(cfn, name, json.dumps(template(Parameters={"P": {"Type": "String"}})),
+ f"1 validation error detected: Value '{big_default}' at "
+ "'parameters.1.member.parameterValue' failed to satisfy constraint: "
+ "Member must have length less than or equal to 4096",
+ parameters=[{"ParameterKey": "P", "ParameterValue": big_default}])
+
+ # The name-length branch of the other sections and the attribute name of
+ # a mapping, in process: they refuse before a stack record exists too, but
+ # the message is what this checks.
+ for singular, section in (
+ ("Parameter", {"Parameters": {"N" * 256: {"Type": "String"}}}),
+ ("Output", {"Outputs": {"N" * 256: {"Value": "x"}}}),
+ ("Mapping", {"Mappings": {"N" * 256: {"a": {"b": "c"}}}}),
+ ):
+ with pytest.raises(ValueError) as err:
+ validate_template_support(template(**section), {})
+ assert str(err.value) == (
+ f"Template format error: {singular} name {'N' * 32}... may not exceed "
+ "255 characters")
+ with pytest.raises(ValueError) as err:
+ validate_template_support(template(Mappings={"M": {"a" * 256: {"b": "c"}}}), {})
+ assert str(err.value) == (
+ f"Template format error: Mapping attribute name {'a' * 32}... of mapping M "
+ "may not exceed 255 characters")
+
+ # The maximum of each quota passes (checked in process: 500 handles would
+ # deploy, but the check is what this test is about).
+ validate_template_support({
+ "Description": "d" * 1024,
+ "Parameters": {f"P{i}": {"Type": "String", "Default": "v" * 4096} for i in range(200)},
+ "Mappings": {f"M{i}": {f"a{j}": {"b": "c"} for j in range(200)} for i in range(200)},
+ "Resources": {"R" * 252 + f"{i:03d}": _HANDLE for i in range(500)},
+ "Outputs": {f"O{i}": {"Value": "x"} for i in range(200)},
+ }, {})
+
+
+def test_cfn_stack_name_quotas(cfn):
+ """CreateStack and a CREATE change set refuse a stack name that is not
+ alphanumeric-and-hyphens starting with a letter (quoted: ``at
+ 'stackName' failed to satisfy constraint: Member must satisfy regular
+ expression pattern``) or longer than 128 characters, before a stack
+ exists; a 128-character name is accepted."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ body = json.dumps({"Resources": {"Handle": _HANDLE}})
+ pattern = ("failed to satisfy constraint: Member must satisfy regular expression "
+ "pattern: [a-zA-Z][-a-zA-Z0-9]*")
+ for bad in (f"cfn_name_{uid}", f"1cfn-name-{uid}", f"cfn.name.{uid}"):
+ _refused_quota(cfn, bad, body,
+ f"1 validation error detected: Value '{bad}' at 'stackName' {pattern}")
+ with pytest.raises(ClientError) as exc:
+ cfn.create_change_set(StackName=f"cfn_cs_{uid}", ChangeSetName="cs",
+ ChangeSetType="CREATE", TemplateBody=body)
+ assert exc.value.response["Error"]["Message"] == \
+ f"1 validation error detected: Value 'cfn_cs_{uid}' at 'stackName' {pattern}"
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=f"cfn_cs_{uid}")
+
+ long = f"cfn-long-{uid}-" + "x" * (129 - len(f"cfn-long-{uid}-"))
+ assert len(long) == 129
+ _refused_quota(cfn, long, body,
+ f"1 validation error detected: Value '{long}' at 'stackName' failed to "
+ "satisfy constraint: Member must have length less than or equal to 128")
+ both = f"cfn_long_{uid}_" + "x" * (129 - len(f"cfn_long_{uid}_"))
+ _refused_quota(cfn, both, body,
+ f"2 validation errors detected: Value '{both}' at 'stackName' failed to "
+ "satisfy constraint: Member must have length less than or equal to 128; "
+ f"Value '{both}' at 'stackName' {pattern}")
+
+ # Every request-level violation is reported in one message: a bad name
+ # and an oversized body together, on CreateStack and on a CREATE change
+ # set alike (the name problem first, order unmeasured).
+ big = json.dumps({"Metadata": {"Pad": "x" * 51200}, "Resources": {"Handle": _HANDLE}})
+ joined = (f"2 validation errors detected: Value 'cfn_both_{uid}' at 'stackName' {pattern}; "
+ f"Value '{big}' at 'templateBody' failed to satisfy constraint: Member must have "
+ "length less than or equal to 51200")
+ _refused_quota(cfn, f"cfn_both_{uid}", big, joined)
+ with pytest.raises(ClientError) as exc:
+ cfn.create_change_set(StackName=f"cfn_both_{uid}", ChangeSetName="cs",
+ ChangeSetType="CREATE", TemplateBody=big)
+ assert exc.value.response["Error"]["Message"] == joined
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=f"cfn_both_{uid}")
+
+ ok = long[:128]
+ cfn.create_stack(StackName=ok, TemplateBody=body)
+ try:
+ stack = _wait_stack(cfn, ok)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ finally:
+ _delete_cfn_test_stack(cfn, ok)
+
+
+# ---------------------------------------------------------------------------
+# AWS::Include transform
+# ---------------------------------------------------------------------------
+
+def test_cfn_include_transform(cfn, s3, ssm):
+ """An embedded ``Fn::Transform`` naming ``AWS::Include`` is replaced by the
+ S3 object it points to (JSON or plain YAML, a key-value object): next to
+ a sibling resource inside Resources, as a whole Properties map, inside a
+ Properties map next to other properties, inside Outputs and inside
+ Mappings (the reference names the mappings section as an example of where
+ the transform may sit). A location
+ that is not an ``s3://`` URI and a snippet with a YAML shorthand tag are
+ refused with the sentences a real account answers, bare (no ``Template
+ format error:`` prefix); a missing object, a snippet that is not an
+ object and a nested include are refused too, all before a stack exists.
+ On UpdateStack and CreateChangeSet the include is expanded the same way
+ and the change set lists the included resource; with UsePreviousTemplate
+ the stored processed template is used, so a snippet edited in S3 since
+ the deploy is not picked up."""
+ uid = _uuid_mod.uuid4().hex[:8]
+ bucket = f"cfn-include-{uid}"
+ name = f"cfn-include-{uid}"
+ s3.create_bucket(Bucket=bucket)
+ try:
+ _include_transform(cfn, s3, ssm, uid, bucket, name)
+ finally:
+ _delete_cfn_test_stack(cfn, name)
+ for key in ("resources.yaml", "properties.json", "value.json", "list.json",
+ "nested.json", "shorthand.yaml", "outputs.json", "second.json",
+ "mappings.json"):
+ s3.delete_object(Bucket=bucket, Key=key)
+ s3.delete_bucket(Bucket=bucket)
+
+
+def _include_transform(cfn, s3, ssm, uid, bucket, name):
+ s3.put_object(Bucket=bucket, Key="resources.yaml", Body=(
+ "Included:\n"
+ " Type: AWS::SSM::Parameter\n"
+ " Properties:\n"
+ f" Name: /cfn-include/{uid}/from-yaml\n"
+ " Type: String\n"
+ " Value: {Ref: Handle}\n").encode())
+ s3.put_object(Bucket=bucket, Key="shorthand.yaml", Body=(
+ "Included:\n"
+ " Type: AWS::SSM::Parameter\n"
+ " Properties:\n"
+ f" Name: /cfn-include/{uid}/shorthand\n"
+ " Type: String\n"
+ " Value: !Ref Handle\n").encode())
+ s3.put_object(Bucket=bucket, Key="properties.json", Body=json.dumps({
+ "Name": f"/cfn-include/{uid}/whole", "Type": "String", "Value": "whole"}).encode())
+ s3.put_object(Bucket=bucket, Key="value.json", Body=json.dumps({
+ "Value": "merged", "Type": "String"}).encode())
+ s3.put_object(Bucket=bucket, Key="outputs.json", Body=json.dumps({
+ "IncludedOutput": {"Value": {"Ref": "Handle"}}}).encode())
+ s3.put_object(Bucket=bucket, Key="second.json", Body=json.dumps({
+ "Second": {"Type": "AWS::SSM::Parameter", "Properties": {
+ "Name": f"/cfn-include/{uid}/second", "Type": "String", "Value": "second"}}}).encode())
+ s3.put_object(Bucket=bucket, Key="mappings.json", Body=json.dumps({
+ "Sizes": {"small": {"Value": "from-map"}}}).encode())
+ s3.put_object(Bucket=bucket, Key="list.json", Body=b"[1, 2]")
+ s3.put_object(Bucket=bucket, Key="nested.json", Body=json.dumps({
+ "Fn::Transform": {"Name": "AWS::Include",
+ "Parameters": {"Location": f"s3://{bucket}/value.json"}}}).encode())
+
+ def include(key):
+ return {"Fn::Transform": {"Name": "AWS::Include",
+ "Parameters": {"Location": f"s3://{bucket}/{key}"}}}
+
+ template = {
+ "Resources": {
+ "Handle": _HANDLE,
+ "Whole": {"Type": "AWS::SSM::Parameter", "Properties": include("properties.json")},
+ "Merged": {"Type": "AWS::SSM::Parameter", "Properties": {
+ "Name": f"/cfn-include/{uid}/merged", **include("value.json")}},
+ "FromMap": {"Type": "AWS::SSM::Parameter", "Properties": {
+ "Name": f"/cfn-include/{uid}/from-map", "Type": "String",
+ "Value": {"Fn::FindInMap": ["Sizes", "small", "Value"]}}},
+ **include("resources.yaml"),
+ },
+ "Mappings": include("mappings.json"),
+ "Outputs": include("outputs.json"),
+ }
+
+ def refused(body, expected):
+ with pytest.raises(ClientError) as exc:
+ cfn.create_stack(StackName=name, TemplateBody=json.dumps(body))
+ assert exc.value.response["Error"]["Code"] == "ValidationError"
+ assert exc.value.response["Error"]["Message"] == expected, \
+ exc.value.response["Error"]["Message"]
+ with pytest.raises(ClientError):
+ cfn.describe_stacks(StackName=name)
+
+ def with_include(key_or_location):
+ location = key_or_location if "://" in key_or_location \
+ else f"s3://{bucket}/{key_or_location}"
+ return {"Resources": {"Handle": _HANDLE, "Fn::Transform": {
+ "Name": "AWS::Include", "Parameters": {"Location": location}}}}
+
+ refused(with_include(f"https://{bucket}.s3.amazonaws.com/resources.yaml"),
+ "Transform AWS::Include failed with: The location parameter is not a valid S3 uri.")
+ refused(with_include("shorthand.yaml"),
+ "Transform AWS::Include failed with: The specified S3 object's content should be "
+ "valid Yaml/JSON")
+ refused(with_include("missing.yaml"),
+ f"Transform AWS::Include failed with: The S3 object s3://{bucket}/missing.yaml "
+ "does not exist.")
+ refused(with_include("list.json"),
+ f"Transform AWS::Include failed with: The snippet at s3://{bucket}/list.json must "
+ "be a key-value object.")
+ refused(with_include("nested.json"),
+ f"Transform AWS::Include failed with: The snippet at s3://{bucket}/nested.json uses "
+ "AWS::Include, which cannot be nested.")
+
+ cfn.create_stack(StackName=name, TemplateBody=json.dumps(template))
+ stack = _wait_stack(cfn, name)
+ assert stack["StackStatus"] == "CREATE_COMPLETE", stack.get("StackStatusReason")
+ logical_ids = sorted(r["LogicalResourceId"] for r in
+ cfn.describe_stack_resources(StackName=name)["StackResources"])
+ assert logical_ids == ["FromMap", "Handle", "Included", "Merged", "Whole"]
+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/whole")["Parameter"]["Value"] == "whole"
+ # The Mappings section is expanded too, so Fn::FindInMap reads the snippet.
+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/from-map")["Parameter"]["Value"] \
+ == "from-map"
+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/merged")["Parameter"]["Value"] \
+ == "merged"
+ handle_url = ssm.get_parameter(Name=f"/cfn-include/{uid}/from-yaml")["Parameter"]["Value"]
+ assert handle_url.startswith("http")
+ assert _output(stack, "IncludedOutput") == handle_url
+ # The stored template is the one the caller sent: the include is not
+ # baked into it.
+ stored = cfn.get_template(StackName=name)["TemplateBody"]
+ assert "Fn::Transform" in stored["Resources"] and "Included" not in stored["Resources"]
+
+ # A snippet edited in S3 after the deploy is not picked up by an update
+ # that reuses the template: nothing changed from the stack's point of view.
+ s3.put_object(Bucket=bucket, Key="value.json", Body=json.dumps({
+ "Value": "edited", "Type": "String"}).encode())
+ with pytest.raises(ClientError) as exc:
+ cfn.update_stack(StackName=name, UsePreviousTemplate=True)
+ assert exc.value.response["Error"]["Message"] == "No updates are to be performed."
+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/merged")["Parameter"]["Value"] \
+ == "merged"
+
+ # A change set and an update with a new body expand the include again:
+ # the change set lists the included resource, the update provisions it.
+ second = json.loads(json.dumps(template))
+ second["Resources"].update(include("second.json"))
+ cfn.create_change_set(StackName=name, ChangeSetName="second",
+ TemplateBody=json.dumps(second))
+ for _ in range(40):
+ described = cfn.describe_change_set(StackName=name, ChangeSetName="second")
+ if described["Status"] in ("CREATE_COMPLETE", "FAILED"):
+ break
+ time.sleep(0.25)
+ assert described["Status"] == "CREATE_COMPLETE", described.get("StatusReason")
+ added = sorted(c["ResourceChange"]["LogicalResourceId"] for c in described["Changes"]
+ if c["ResourceChange"]["Action"] == "Add")
+ assert added == ["Second"]
+ cfn.delete_change_set(StackName=name, ChangeSetName="second")
+ cfn.update_stack(StackName=name, TemplateBody=json.dumps(second))
+ stack = _wait_stack(cfn, name)
+ assert stack["StackStatus"] == "UPDATE_COMPLETE", stack.get("StackStatusReason")
+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/second")["Parameter"]["Value"] \
+ == "second"
+ assert ssm.get_parameter(Name=f"/cfn-include/{uid}/merged")["Parameter"]["Value"] \
+ == "edited"