Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

fix(cloudformation): merge Fn::Transform AWS::Include before provisioning, and honour GetTemplate TemplateStage (#2886)

Facundo Paredes77b4881fe40e
diff --git a/docs/services/cloudformation.md b/docs/services/cloudformation.md
index 11c1500dbe..f80af88160 100644
--- a/docs/services/cloudformation.md
+++ b/docs/services/cloudformation.md
@@ -223,6 +223,57 @@ before provisioning:
The `AWS::SSM::Parameter` **resource** type exposes `Value`, `Type`, and `Name` attributes through
`Ref` / `Fn::GetAtt` so downstream resources can consume a parameter the same stack creates.
+## AWS::Include (`Fn::Transform`)
+
+`Fn::Transform` with `Name: AWS::Include` splices the YAML/JSON snippet at `Parameters.Location`
+into its enclosing mapping before any other intrinsic resolves, on both `CreateStack`/`UpdateStack`
+and the `CreateChangeSet` preview. `Location` must be an `s3://bucket/key` URI: the template
+CloudFormation itself receives always carries one, because `aws cloudformation package` rewrites
+every local path before a stack ever sees it. A `Location` that is not `s3://` fails the stack with
+a `ValidationError` naming it, rather than silently dropping the include. A `Location` that is a
+well-formed `s3://` URI but cannot be read (the bucket or key does not exist) fails the stack with
+the underlying S3 error naming it, for example `NoSuchKey` at HTTP 404, instead of a
+`ValidationError`, since that error is what S3 itself already reports and floci has no more specific
+answer to give.
+
+A snippet may not itself use `AWS::Include`; nesting is rejected rather than expanded or looped.
+`Location` must be a plain string; a `Ref` or another intrinsic in its place is rejected, naming
+the rejected node, rather than resolved.
+
+Two known deviations from AWS:
+
+- The snippet is parsed with floci's CloudFormation-aware YAML parser, so it accepts CloudFormation
+ YAML short tags (`!Ref`, `!Sub`, ...) where AWS's own `AWS::Include` documentation says a snippet
+ does not.
+- AWS's own `Fn::Transform` documentation shows a `Location` written as an intrinsic function (its
+ example uses `Ref`) and describes it as accepted; floci does not resolve one and rejects the
+ template instead. This deviation comes from reading AWS's documentation, not from a request
+ measured against a real account.
+
+`CAPABILITY_AUTO_EXPAND` is not one of them. floci requires no capability for a template that
+declares `AWS::Include`, and neither does AWS on the change-set path: a `CreateChangeSet` carrying
+an embedded `Fn::Transform`/`AWS::Include` was accepted against a real account with
+`CAPABILITY_IAM` alone. The `CreateStack` path was not measured.
+
+`GetTemplateSummary` does not expand the include: it reports `AWS::Include` in the template's
+`Transform`/`DeclaredTransforms` and neither fetches nor validates the snippet, matching AWS.
+`GetTemplate` accepts `TemplateStage` (`Original`, the default, or `Processed`) and validates it
+against that enum, rejecting anything else with a `ValidationError` naming the value, matching AWS.
+`Original` returns the template exactly as submitted, `Fn::Transform` node and all. `Processed`
+returns the merged and SAM-expanded tree, the same one a `CreateChangeSet` preview diffs against.
+
+floci expands neither `AWS::LanguageExtensions`, nor a third-party macro, nor the top-level
+`Transform: {Name: AWS::Include, Parameters: {Location: ...}}` form. A template carrying only one of
+those three keeps `Processed` equal to `Original`, byte for byte.
+
+Two triggers break that equality, because `executeTemplate` re-serializes the persisted body when
+either one fires: an embedded `Fn::Transform` that merges, and a SAM transform. A SAM transform goes
+further, because `SamTransformProcessor` removes the whole `Transform` section unconditionally, so a
+macro co-declared beside SAM is absent from `Processed` as well.
+
+Real AWS's answer for these shapes is unmeasured, co-declared with SAM or not. `StagesAvailable`
+always lists both stages, matching AWS.
+
## Conditions
Template `Conditions` are evaluated before provisioning. A resource whose `Condition` evaluates to
diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessor.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessor.java
new file mode 100644
index 0000000000..e3ad432c5a
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessor.java
@@ -0,0 +1,180 @@
+package io.github.hectorvent.floci.services.cloudformation;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.services.s3.S3Service;
+
+import java.nio.charset.StandardCharsets;
+import java.util.Iterator;
+import java.util.Map;
+
+/**
+ * Expands {@code Fn::Transform} nodes whose {@code Name} is {@code AWS::Include}: splices the
+ * YAML/JSON snippet the node's {@code Location} points at into the enclosing mapping, in place of
+ * the {@code Fn::Transform} key, before any other intrinsic resolves, mirroring
+ * {@link SamTransformProcessor}, which runs on the same parsed template ahead of provisioning.
+ *
+ * <p>{@code Location} accepts only an {@code s3://} URI. The template CloudFormation itself
+ * receives always carries one: {@code aws cloudformation package} rewrites every local path before
+ * a stack ever sees it, so a local path never reaches this processor in practice, and floci does
+ * not read one on the caller's behalf (the CloudFormation package performs no filesystem access,
+ * and a containerised floci sees none of the caller's paths regardless).
+ */
+class AwsIncludeProcessor {
+
+ static final String AWS_INCLUDE = "AWS::Include";
+ private static final String FN_TRANSFORM = "Fn::Transform";
+ private static final String S3_SCHEME = "s3://";
+
+ private final ObjectMapper objectMapper;
+ private final S3Service s3Service;
+
+ AwsIncludeProcessor(ObjectMapper objectMapper, S3Service s3Service) {
+ this.objectMapper = objectMapper;
+ this.s3Service = s3Service;
+ }
+
+ /**
+ * Returns a copy of {@code template} with every {@code Fn::Transform}/{@code AWS::Include}
+ * node replaced by the keys of the snippet it names, merged into the node's enclosing mapping
+ * alongside any keys already written there.
+ *
+ * <p>Reference identity is part of this method's contract, not an incidental optimization: a
+ * template carrying no {@code AWS::Include} is returned as the exact same {@code JsonNode}
+ * instance passed in, unmodified, on every {@code executeTemplate} and every
+ * {@code DescribeChangeSet} - the overwhelming majority of templates. Callers rely on that:
+ * they compare the reference this method returns against the one they passed in to tell a real
+ * merge apart from a no-op one, and a copy here even for a no-op merge would make every caller
+ * treat an untouched template as changed.
+ */
+ JsonNode mergeIncludes(JsonNode template) {
+ if (!containsAwsInclude(template)) {
+ return template;
+ }
+ JsonNode merged = template.deepCopy();
+ merge(merged);
+ return merged;
+ }
+
+ private void merge(JsonNode node) {
+ if (node.isObject()) {
+ mergeObject((ObjectNode) node);
+ } else if (node.isArray()) {
+ node.forEach(this::merge);
+ }
+ }
+
+ private void mergeObject(ObjectNode obj) {
+ JsonNode transform = obj.get(FN_TRANSFORM);
+ if (transform != null && AWS_INCLUDE.equals(transform.path("Name").asText(""))) {
+ spliceInclude(obj, transform);
+ }
+ // Walk every remaining field, including the snippet's own keys just merged in above:
+ // spliceInclude already rejects a snippet carrying its own AWS::Include, so revisiting
+ // them here cannot re-trigger a merge - it only reaches unrelated Fn::Transform nodes
+ // elsewhere in the template.
+ Iterator<Map.Entry<String, JsonNode>> fields = obj.fields();
+ while (fields.hasNext()) {
+ merge(fields.next().getValue());
+ }
+ }
+
+ private void spliceInclude(ObjectNode obj, JsonNode transform) {
+ String location = describeLocation(transform);
+ S3Location s3Location = parseS3Location(location);
+ JsonNode snippet = fetchSnippet(s3Location, location);
+
+ if (!snippet.isObject()) {
+ throw new AwsException("ValidationError",
+ "AWS::Include snippet at " + location + " must be a YAML/JSON mapping", 400);
+ }
+ if (containsAwsInclude(snippet)) {
+ throw new AwsException("ValidationError",
+ "AWS::Include snippet at " + location + " may not itself use AWS::Include", 400);
+ }
+
+ obj.remove(FN_TRANSFORM);
+ obj.setAll((ObjectNode) snippet);
+ }
+
+ /**
+ * Renders {@code transform.Parameters.Location} for an error message. A well-formed
+ * {@code Location} is a plain string, but a node such as {@code {Ref: InputValue}}, the form
+ * AWS's own {@code Fn::Transform} documentation uses in its {@code AWS::Include} example, is a
+ * mapping. {@code Ref} resolution is not implemented here, so that mapping is serialized
+ * instead, and the rejection message names it. A {@code Parameters} that is not an object
+ * (missing, explicitly {@code null}, a scalar, or a list) names the whole {@code Fn::Transform}
+ * node instead, so the message never presents an unrelated value as the rejected
+ * {@code Location}. A {@code Location} missing from an object {@code Parameters} names the
+ * enclosing {@code Parameters} mapping. A {@code Location} present but blank, including
+ * whitespace only, is rendered as its quoted string, not elided, so the message never leaves an
+ * empty tail.
+ */
+ private String describeLocation(JsonNode transform) {
+ JsonNode parameters = transform.path("Parameters");
+ if (!parameters.isObject()) {
+ return transform.toString();
+ }
+ JsonNode locationNode = parameters.path("Location");
+ if (locationNode.isTextual() && !locationNode.asText().isBlank()) {
+ return locationNode.asText();
+ }
+ JsonNode described = locationNode.isMissingNode() || locationNode.isNull() ? parameters : locationNode;
+ return described.toString();
+ }
+
+ /**
+ * Returns whether {@code node} contains an unexpanded {@code Fn::Transform} whose {@code Name}
+ * is {@code AWS::Include}, at any depth. A foreign macro under a different {@code Name} does not
+ * count: AWS documents that a snippet may not itself use {@code AWS::Include}, not that it may
+ * carry no other transform. Package-private: also used by {@code CloudFormationService} to
+ * report {@code AWS::Include} in {@code GetTemplateSummary}'s {@code DeclaredTransforms} and to
+ * skip {@code validateConditionDependencies} for a template the merge has not run against yet.
+ */
+ boolean containsAwsInclude(JsonNode node) {
+ if (node.isObject()) {
+ JsonNode transform = node.get(FN_TRANSFORM);
+ if (transform != null && AWS_INCLUDE.equals(transform.path("Name").asText(""))) {
+ return true;
+ }
+ }
+ if (node.isContainerNode()) {
+ for (JsonNode child : node) {
+ if (containsAwsInclude(child)) {
+ return true;
+ }
+ }
+ }
+ return false;
+ }
+
+ private JsonNode fetchSnippet(S3Location location, String rawLocation) {
+ try {
+ byte[] data = s3Service.getObject(location.bucket(), location.key()).getData();
+ return new CloudFormationYamlParser(objectMapper).parse(new String(data, StandardCharsets.UTF_8));
+ } catch (AwsException e) {
+ throw new AwsException(e.getErrorCode(),
+ "Unable to read AWS::Include snippet at " + rawLocation + ": " + e.getMessage(),
+ e.getHttpStatus());
+ } catch (Exception e) {
+ throw new AwsException("ValidationError",
+ "Unable to read AWS::Include snippet at " + rawLocation + ": " + e.getMessage(), 400);
+ }
+ }
+
+ private record S3Location(String bucket, String key) {}
+
+ private S3Location parseS3Location(String location) {
+ if (location.startsWith(S3_SCHEME)) {
+ String withoutScheme = location.substring(S3_SCHEME.length());
+ int slash = withoutScheme.indexOf('/');
+ if (slash > 0 && slash < withoutScheme.length() - 1) {
+ return new S3Location(withoutScheme.substring(0, slash), withoutScheme.substring(slash + 1));
+ }
+ }
+ throw new AwsException("ValidationError",
+ "Fn::Transform AWS::Include Location must be an s3://bucket/key URI, got " + location, 400);
+ }
+}
diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java
index 776457efab..bf9f7a9f2e 100644
--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java
+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java
@@ -443,17 +443,24 @@ public class CloudFormationQueryHandler {
private Response getTemplate(MultivaluedMap<String, String> params, String region) {
String stackName = params.getFirst("StackName");
+ String templateStage = params.getFirst("TemplateStage");
try {
- String template = cfnService.getTemplate(stackName, region);
- String xml = new XmlBuilder()
+ String template = cfnService.getTemplate(stackName, templateStage, region);
+ var xml = new XmlBuilder()
.start("GetTemplateResponse", CF_NS)
.start("GetTemplateResult")
- .elem("TemplateBody", template)
- .end("GetTemplateResult")
- .raw(AwsQueryResponse.responseMetadata())
- .end("GetTemplateResponse")
- .build();
- return Response.ok(xml).type("text/xml").build();
+ .elem("TemplateBody", template);
+
+ xml.start("StagesAvailable");
+ for (String stage : cfnService.templateStagesAvailable()) {
+ xml.elem("member", stage);
+ }
+ xml.end("StagesAvailable");
+
+ xml.end("GetTemplateResult")
+ .raw(AwsQueryResponse.responseMetadata())
+ .end("GetTemplateResponse");
+ return Response.ok(xml.build()).type("text/xml").build();
} catch (AwsException e) {
return xmlError(e.getErrorCode(), e.getMessage(), e.getHttpStatus());
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java
index e2ee26e954..c0c0d842d5 100644
--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java
@@ -68,6 +68,7 @@ public class CloudFormationService implements ResourceProvider {
private final EmulatorConfig config;
private final RegionResolver regionResolver;
private final SamTransformProcessor samTransformProcessor;
+ private final AwsIncludeProcessor awsIncludeProcessor;
private final Clock clock;
// Persisted state so stacks survive a restart (criteria #10, #11). The in-memory maps above are
@@ -90,6 +91,7 @@ public class CloudFormationService implements ResourceProvider {
this.config = config;
this.regionResolver = regionResolver;
this.samTransformProcessor = new SamTransformProcessor(objectMapper);
+ this.awsIncludeProcessor = new AwsIncludeProcessor(objectMapper, s3Service);
this.clock = clock;
this.storageAccount = config.defaultAccountId();
this.stackBackend = storageFactory.create(
@@ -386,6 +388,9 @@ public class CloudFormationService implements ResourceProvider {
Stack stack = getStackOrThrow(cs.getStackName(), region);
try {
JsonNode newTemplate = parseTemplate(cs.getTemplateBody());
+ // Merge Fn::Transform/AWS::Include snippets before SAM expansion, matching AWS order:
+ // an included fragment may itself carry SAM resources.
+ newTemplate = awsIncludeProcessor.mergeIncludes(newTemplate);
if (samTransformProcessor.hasSamTransform(newTemplate)) {
// The deployed stack's template is always the SAM-expanded form (see
// executeTemplate); comparing the change set's raw SAM source against it would
@@ -674,9 +679,50 @@ public class CloudFormationService implements ResourceProvider {
// ── GetTemplate ───────────────────────────────────────────────────────────
- public String getTemplate(String stackName, String region) {
+ private static final String STAGE_PROCESSED = "Processed";
+ private static final String STAGE_ORIGINAL = "Original";
+
+ // AWS's own enum order for the ValidationError message (measured against a real account).
+ private static final List<String> TEMPLATE_STAGE_ENUM = List.of(STAGE_PROCESSED, STAGE_ORIGINAL);
+ // AWS's own StagesAvailable order (Original first), which every GetTemplate call reports.
+ private static final List<String> TEMPLATE_STAGES_AVAILABLE = List.of(STAGE_ORIGINAL, STAGE_PROCESSED);
+
+ public String getTemplate(String stackName, String templateStage, String region) {
+ // AWS validates TemplateStage before it looks the stack up (measured against a real
+ // account: an invalid stage is rejected the same way whether or not the stack exists), so
+ // this must run before getStackOrThrow.
+ String stage = validateTemplateStage(templateStage);
Stack stack = getStackOrThrow(stackName, region);
- return stack.getTemplateBody() != null ? stack.getTemplateBody() : "{}";
+ // Processed is the SAM/AWS::Include-expanded form templateBody holds after executeTemplate.
+ // Original (also the default, matching real AWS) is the template exactly as the caller
+ // submitted it. originalTemplateBody is only absent for stacks persisted by a floci version
+ // predating this field, hence the fallback to templateBody; getTemplateSummary reads the
+ // same field for the same reason.
+ String body = STAGE_PROCESSED.equals(stage) ? stack.getTemplateBody() : stack.getOriginalTemplateBody();
+ if (body == null) {
+ body = stack.getTemplateBody();
+ }
+ return body != null ? body : "{}";
+ }
+
+ private String validateTemplateStage(String templateStage) {
+ // null means the caller omitted TemplateStage; "" means the caller sent it present and
+ // empty. AWS rejects the latter (measured against a real account) and only defaults the
+ // former, so this must not treat blank the same as absent.
+ if (templateStage == null) {
+ return STAGE_ORIGINAL;
+ }
+ if (!TEMPLATE_STAGE_ENUM.contains(templateStage)) {
+ throw new AwsException("ValidationError",
+ "1 validation error detected: Value '" + templateStage + "' at 'templateStage' failed to "
+ + "satisfy constraint: Member must satisfy enum value set: ["
+ + String.join(", ", TEMPLATE_STAGE_ENUM) + "]", 400);
+ }
+ return templateStage;
+ }
+
+ public List<String> templateStagesAvailable() {
+ return TEMPLATE_STAGES_AVAILABLE;
}
// ── GetTemplateSummary ────────────────────────────────────────────────────
@@ -780,6 +826,12 @@ public class CloudFormationService implements ResourceProvider {
}
});
}
+ // AWS reports AWS::Include in DeclaredTransforms for the embedded Fn::Transform form too,
+ // even with no top-level Transform section (measured against us-east-1).
+ if (!declaredTransforms.contains(AwsIncludeProcessor.AWS_INCLUDE)
+ && awsIncludeProcessor.containsAwsInclude(template)) {
+ declaredTransforms.add(AwsIncludeProcessor.AWS_INCLUDE);
+ }
List<String> iamResourceTypes = resourceTypes.stream()
.filter(t -> t.startsWith("AWS::IAM::"))
@@ -931,14 +983,28 @@ public class CloudFormationService implements ResourceProvider {
JsonNode template = parseTemplate(templateBody);
stack.setOriginalTemplateBody(templateBody);
+ // Merge Fn::Transform/AWS::Include snippets before SAM expansion, matching AWS order:
+ // an included fragment may itself carry SAM resources. mergeIncludes returns the same
+ // reference, unchanged, when the template carries no AWS::Include, which is what lets
+ // the check below tell a real merge apart from a no-op one.
+ JsonNode beforeInclude = template;
+ template = awsIncludeProcessor.mergeIncludes(template);
+ boolean includeMerged = template != beforeInclude;
+
// Apply SAM transform if the template declares AWS::Serverless-2016-10-31
- if (samTransformProcessor.hasSamTransform(template)) {
+ boolean hasSamTransform = samTransformProcessor.hasSamTransform(template);
+ if (hasSamTransform) {
LOG.infov("Applying SAM transform for stack {0}", stack.getStackName());
template = samTransformProcessor.expandSamTemplate(template);
- // Store the expanded template so GetTemplate returns the transformed version
- templateBody = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(template);
}
+ // Persist the merged/expanded tree, not the raw submitted body, whenever either
+ // processor actually changed it, so the change-set baseline diffs against it instead of
+ // against a stale Fn::Transform node. A template neither processor touched keeps its
+ // submitted body byte for byte.
+ if (includeMerged || hasSamTransform) {
+ templateBody = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(template);
+ }
stack.setTemplateBody(templateBody);
// Merge default parameter values from the template with caller-supplied params
@@ -1376,6 +1442,12 @@ public class CloudFormationService implements ResourceProvider {
stack.getResolvedParameters().putAll(previousState.resolvedParameters());
if (rollbackFailures.isEmpty()) {
stack.setTemplateBody(previousState.templateBody());
+ // GetTemplate reads originalTemplateBody for TemplateStage=Original and templateBody
+ // for TemplateStage=Processed: without restoring originalTemplateBody here too, a
+ // rolled-back update would keep serving the failed attempt's submitted body under
+ // Original even though every other piece of state (resources, parameters, outputs)
+ // was restored.
+ stack.setOriginalTemplateBody(previousState.originalTemplateBody());
}
try {
restoreOutputAndExportState(stack, region, previousState);
@@ -1519,6 +1591,7 @@ public class CloudFormationService implements ResourceProvider {
private StackUpdateSnapshot snapshotForUpdate(Stack stack) {
return new StackUpdateSnapshot(
stack.getTemplateBody(),
+ stack.getOriginalTemplateBody(),
new LinkedHashMap<>(stack.getParameters()),
new LinkedHashMap<>(stack.getResolvedParameters()),
new LinkedHashMap<>(stack.getOutputs()),
@@ -1551,6 +1624,7 @@ public class CloudFormationService implements ResourceProvider {
private record StackUpdateSnapshot(
String templateBody,
+ String originalTemplateBody,
Map<String, String> parameters,
Map<String, String> resolvedParameters,
Map<String, String> outputs,
@@ -1808,6 +1882,10 @@ public class CloudFormationService implements ResourceProvider {
* template synchronously ("Template format error: Unresolved resource dependencies [...]")
* rather than silently skipping the dependent, so mirror that instead of dropping the resource.
* Malformed or SAM templates are left for the execution path, which surfaces their own errors.
+ * A template carrying an unexpanded {@code Fn::Transform}/{@code AWS::Include} is left for the
+ * same reason: a {@code Conditions} section spliced in from a snippet is invisible here, since
+ * the merge has not run yet, and treating it as absent would fail a template whose dependency
+ * graph the execution path resolves correctly.
*/
private void validateConditionDependencies(String templateBody, Map<String, String> params,
String region, String accountId) {
@@ -1819,7 +1897,7 @@ public class CloudFormationService implements ResourceProvider {
e.getMessage());
return;
}
- if (samTransformProcessor.hasSamTransform(template)) {
+ if (samTransformProcessor.hasSamTransform(template) || awsIncludeProcessor.containsAwsInclude(template)) {
return;
}
JsonNode resources = template.path("Resources");
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeIntegrationTest.java
new file mode 100644
index 0000000000..201dcc0749
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeIntegrationTest.java
@@ -0,0 +1,688 @@
+package io.github.hectorvent.floci.services.cloudformation;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.github.hectorvent.floci.services.s3.S3Service;
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.quarkus.test.junit.QuarkusTest;
+import jakarta.inject.Inject;
+import org.jboss.logging.Logger;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+
+import static io.restassured.RestAssured.given;
+import static org.hamcrest.MatcherAssert.assertThat;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.not;
+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;
+
+/**
+ * The deployed outcome of {@code Fn::Transform}/{@code AWS::Include}: a {@code
+ * DefinitionSubstitutions} mapping written as an include reaches the state machine's actual
+ * definition, closing the symptom in "Request": 23 of 67 state machines in one repository
+ * deploying with a literal {@code ${expandItems}}. The state machine is declared as
+ * {@code AWS::StepFunctions::StateMachine}, not the SAM {@code AWS::Serverless::StateMachine} form
+ * ({@code fix/sam-serverless-statemachine}'s subject); this test does not depend on that transform.
+ */
+@QuarkusTest
+class AwsIncludeIntegrationTest {
+
+ private static final Logger LOG = Logger.getLogger(AwsIncludeIntegrationTest.class);
+ private static final String SFN_CONTENT_TYPE = "application/x-amz-json-1.0";
+
+ @Inject
+ S3Service s3Service;
+
+ private final ObjectMapper mapper = new ObjectMapper();
+
+ private final List<String> stacksToDelete = new ArrayList<>();
+ private final Map<String, List<String>> objectsToDelete = new LinkedHashMap<>();
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @AfterEach
+ void cleanUp() {
+ for (String stackName : stacksToDelete) {
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DeleteStack")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/");
+ }
+ stacksToDelete.clear();
+
+ objectsToDelete.forEach((bucket, keys) -> {
+ try {
+ for (String key : keys) {
+ s3Service.deleteObject(bucket, key);
+ }
+ s3Service.deleteBucket(bucket);
+ } catch (RuntimeException e) {
+ // A bucket that a test (or a resource it provisioned) left with an object this
+ // test never tracked throws BucketNotEmpty here. Left unguarded, that throw used
+ // to abort the forEach before objectsToDelete.clear() ran, so the next @AfterEach
+ // retried the same stale bucket/key pairs on top of its own, cascading one failed
+ // delete into every test that ran afterward. Catching per-bucket keeps the loop
+ // going for the buckets that follow and lets clear() below always run.
+ LOG.warnv(e, "AwsIncludeIntegrationTest cleanup could not remove bucket {0}", bucket);
+ }
+ });
+ objectsToDelete.clear();
+ }
+
+ private String putSnippet(String bucket, String key, String content) {
+ s3Service.createBucket(bucket, "us-east-1");
+ s3Service.putObject(bucket, key, content.getBytes(StandardCharsets.UTF_8), "text/yaml", Map.of());
+ objectsToDelete.computeIfAbsent(bucket, b -> new ArrayList<>()).add(key);
+ return "s3://" + bucket + "/" + key;
+ }
+
+ @Test
+ void includedSubstitutionsReachTheStateMachineDefinition() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "aws-include-it-" + suffix;
+ String key = "definition-substitutions.yaml";
+ String stackName = "aws-include-stack-" + suffix;
+ String stateMachineName = "aws-include-sm-" + suffix;
+ String stateMachineArn = "arn:aws:states:us-east-1:000000000000:stateMachine:" + stateMachineName;
+ stacksToDelete.add(stackName);
+
+ String location = putSnippet(bucket, key, "expandItems: substituted-value\n");
+
+ String template = stateMachineTemplate(stateMachineName, location);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<StackId>"));
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStacks")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"));
+
+ String definition = given()
+ .header("X-Amz-Target", "AWSStepFunctions.DescribeStateMachine")
+ .contentType(SFN_CONTENT_TYPE)
+ .body("{\"stateMachineArn\":\"" + stateMachineArn + "\"}")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body("definition", containsString("substituted-value"))
+ .body("definition", not(containsString("${expandItems}")))
+ .extract().path("definition");
+
+ assertDoesNotThrow(() -> mapper.readTree(definition),
+ "the deployed definition does not parse as JSON: " + definition);
+ }
+
+ @Test
+ void stackWithUnreadableSnippetFailsCreation() {
+ // No bucket is created for this location: AWS::Include fails the same way any other
+ // unreadable AWS resource reference does, before any resource in the stack is attempted:
+ // the same class of pre-provisioning failure CloudFormationSsmParameterIntegrationTest pins
+ // for a missing SSM parameter, and CREATE_FAILED (not a rollback) is what that class of
+ // failure produces today: CloudFormationService's create-path exception handler sets
+ // CREATE_FAILED directly, and only a resource that reached the provisioning loop drives the
+ // ROLLBACK_IN_PROGRESS/ROLLBACK_COMPLETE transition.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stackName = "aws-include-missing-stack-" + suffix;
+ String stateMachineName = "aws-include-missing-sm-" + suffix;
+ String location = "s3://aws-include-it-missing-bucket-" + suffix + "/definition-substitutions.yaml";
+ stacksToDelete.add(stackName);
+
+ String template = stateMachineTemplate(stateMachineName, location);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStacks")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<StackStatus>CREATE_FAILED</StackStatus>"))
+ .body(containsString(location))
+ .body(not(containsString("CREATE_COMPLETE")));
+
+ String resourcesXml = given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStackResources")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .extract().asString();
+
+ // The merge fails before the resource loop starts, so the state machine was never attempted:
+ // it never even reaches CREATE_FAILED as a tracked resource, unlike a resource that fails
+ // during provisioning itself.
+ assertThat(resourcesXml, not(containsString("AWS::StepFunctions::StateMachine")));
+ }
+
+ @Test
+ void getTemplateSummaryDoesNotExpandTheInclude() {
+ // Same unreachable location as the unhappy path above, but through GetTemplateSummary:
+ // AWS answers this with the transform declared and the snippet never fetched (measured
+ // against us-east-1). Reusing the merge seam for GetTemplateSummary would instead fail the
+ // summary on an unreachable location, which is the receipt this test exists to catch on a
+ // future refactor.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stateMachineName = "aws-include-summary-sm-" + suffix;
+ String location = "s3://aws-include-it-missing-bucket-" + suffix + "/definition-substitutions.yaml";
+
+ String template = stateMachineTemplate(stateMachineName, location);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplateSummary")
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<member>AWS::StepFunctions::StateMachine</member>"))
+ // Anchored to the enclosing element, not just to a bare <member>: TemplateSummary
+ // also renders ResourceTypes and Capabilities as <member> lists, so an unanchored
+ // containsString cannot tell DeclaredTransforms apart from either of them and would
+ // stay green even if AWS::Include regressed into the wrong list.
+ // Measured against us-east-1: AWS reports AWS::Include in DeclaredTransforms for the
+ // embedded Fn::Transform form too, with no top-level Transform section and no fetch of
+ // the (here unreachable) snippet.
+ .body(containsString("<DeclaredTransforms><member>AWS::Include</member></DeclaredTransforms>"));
+ }
+
+ @Test
+ void getTemplateReturnsTheSubmittedYamlWhileTheChangeSetBaselineStaysMerged() {
+ // Superseded assertions, and why they were wrong: this test used to assert that GetTemplate
+ // returned the MERGED body ("merged-value" present, "Fn::Transform" absent). That pinned a
+ // real fix (a live Fn::Transform node was leaking into the deployed template because the
+ // re-serialize sat inside the SAM-only branch) but overshot it: real CloudFormation's
+ // GetTemplate defaults to TemplateStage=Original and returns the template exactly as the
+ // caller submitted it, not the SAM/Include-expanded form (GetTemplateSummary already reads
+ // stack.getOriginalTemplateBody() for the same reason). So GetTemplate must return the
+ // caller's raw YAML, Fn::Transform node and all, while CreateChangeSet's baseline diff
+ // keeps comparing against the merged body (stack.getTemplateBody()) so a resubmit of the
+ // identical template still reports zero changes instead of a spurious Modify.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "aws-include-it-persist-" + suffix;
+ String key = "definition-substitutions.yaml";
+ String stackName = "aws-include-persist-stack-" + suffix;
+ String parameterName = "/aws-include-it/persist-" + suffix;
+ stacksToDelete.add(stackName);
+
+ String location = putSnippet(bucket, key, "mergedFlag: merged-value\n");
+
+ String template = """
+ Resources:
+ IncludeParam:
+ Type: AWS::SSM::Parameter
+ Metadata:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ Properties:
+ Name: %s
+ Type: String
+ Value: unmerged-value
+ """.formatted(location, parameterName);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStacks")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"));
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ // The submitted YAML, verbatim: the Fn::Transform node survives (Original stage, not
+ // merged) and the YAML key syntax ("Value: unmerged-value", no quotes around either
+ // side) rules out a reserialized JSON pretty-print of the merged tree. "mergedFlag" is
+ // the snippet's own key, so its presence would mean the merge ran despite Original.
+ .body(containsString("Fn::Transform"))
+ .body(containsString("Value: unmerged-value"))
+ .body(not(containsString("mergedFlag")));
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateChangeSet")
+ .formParam("StackName", stackName)
+ .formParam("ChangeSetName", "identical-resubmit-cs")
+ .formParam("ChangeSetType", "UPDATE")
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeChangeSet")
+ .formParam("StackName", stackName)
+ .formParam("ChangeSetName", "identical-resubmit-cs")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ // The baseline diff still uses the merged body (stack.getTemplateBody(), untouched by
+ // this fix): resubmitting the identical template must not report a spurious Modify.
+ .body(not(containsString("<Action>Modify</Action>")));
+ }
+
+ @Test
+ void getTemplateStageSelectsBetweenTheMergedBodyAndTheSubmittedBody() {
+ // One stack carrying both a SAM resource and an embedded AWS::Include, so TemplateStage
+ // exercises both processors at once: Processed is the merged/SAM-expanded body
+ // (stack.getTemplateBody()) with AWS::Serverless::Function expanded away and the include
+ // merged in, Original is the caller's raw YAML (stack.getOriginalTemplateBody()), and no
+ // TemplateStage at all still defaults to Original, byte for byte.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "aws-include-it-stage-" + suffix;
+ String key = "stage-definition.yaml";
+ String stackName = "aws-include-stage-stack-" + suffix;
+ String functionName = "aws-include-stage-func-" + suffix;
+ String parameterName = "/aws-include-it/stage-" + suffix;
+ stacksToDelete.add(stackName);
+
+ String location = putSnippet(bucket, key, "mergedFlag: merged-value\n");
+
+ String template = """
+ AWSTemplateFormatVersion: '2010-09-09'
+ Transform: AWS::Serverless-2016-10-31
+ Resources:
+ StageFunction:
+ Type: AWS::Serverless::Function
+ Properties:
+ FunctionName: %s
+ Handler: index.handler
+ Runtime: nodejs22.x
+ InlineCode: |
+ exports.handler = async () => ({ statusCode: 200, body: 'ok' });
+ IncludeParam:
+ Type: AWS::SSM::Parameter
+ Metadata:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ Properties:
+ Name: %s
+ Type: String
+ Value: unmerged-value
+ """.formatted(functionName, location, parameterName);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStacks")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"));
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .formParam("TemplateStage", "Processed")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("AWS::Lambda::Function"))
+ .body(containsString("mergedFlag"))
+ .body(not(containsString("AWS::Serverless::Function")))
+ .body(not(containsString("Fn::Transform")));
+
+ String originalBody = given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .formParam("TemplateStage", "Original")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("AWS::Serverless::Function"))
+ .body(containsString("Fn::Transform"))
+ .body(not(containsString("mergedFlag")))
+ .extract().path("GetTemplateResponse.GetTemplateResult.TemplateBody");
+
+ String defaultStageBody = given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .extract().path("GetTemplateResponse.GetTemplateResult.TemplateBody");
+
+ assertThat(defaultStageBody, equalTo(originalBody));
+ }
+
+ @Test
+ void getTemplateWithInvalidStageRejectsWithTheAwsEnumValidationError() {
+ // Verbatim against real AWS (measured on us-east-1): an invalid TemplateStage value is
+ // rejected with ValidationError naming the enum, on any existing stack.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stackName = "aws-include-stage-bogus-stack-" + suffix;
+ String parameterName = "/aws-include-it/stage-bogus-" + suffix;
+ stacksToDelete.add(stackName);
+
+ String template = """
+ Resources:
+ BogusStageParam:
+ Type: AWS::SSM::Parameter
+ Properties:
+ Name: %s
+ Type: String
+ Value: plain-value
+ """.formatted(parameterName);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .formParam("TemplateStage", "Bogus")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(400)
+ .body(containsString("<Code>ValidationError</Code>"))
+ .body(containsString("Value &apos;Bogus&apos; at &apos;templateStage&apos; failed to satisfy "
+ + "constraint: Member must satisfy enum value set: [Processed, Original]"));
+ }
+
+ @Test
+ void getTemplateWithInvalidStageValidatesBeforeCheckingWhetherTheStackExists() {
+ // Verbatim against real AWS (measured on us-east-1): `--template-stage Bogus` against a
+ // stack name that was never created still answers with the enum ValidationError, not
+ // "Stack with id ... does not exist", proving AWS validates the request shape before it
+ // looks the stack up. This stack name is never created here, on purpose: a test that
+ // creates the stack first (like the one above) cannot tell this ordering apart from
+ // floci validating after a successful lookup.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stackName = "aws-include-stage-bogus-no-stack-" + suffix;
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .formParam("TemplateStage", "Bogus")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(400)
+ .body(containsString("<Code>ValidationError</Code>"))
+ .body(containsString("Value &apos;Bogus&apos; at &apos;templateStage&apos; failed to satisfy "
+ + "constraint: Member must satisfy enum value set: [Processed, Original]"))
+ .body(not(containsString("does not exist")));
+ }
+
+ @Test
+ void getTemplateWithEmptyStringStageRejectsWithTheAwsEnumValidationError() {
+ // Verbatim against real AWS (measured on us-east-1): `--template-stage ""` is rejected the
+ // same as any other value outside the enum. params.getFirst("TemplateStage") returns "" for
+ // this request, distinct from the omitted case (null), which defaults to Original.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stackName = "aws-include-stage-empty-stack-" + suffix;
+ String parameterName = "/aws-include-it/stage-empty-" + suffix;
+ stacksToDelete.add(stackName);
+
+ String template = """
+ Resources:
+ EmptyStageParam:
+ Type: AWS::SSM::Parameter
+ Properties:
+ Name: %s
+ Type: String
+ Value: plain-value
+ """.formatted(parameterName);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .formParam("TemplateStage", "")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(400)
+ .body(containsString("<Code>ValidationError</Code>"))
+ .body(containsString("Value &apos;&apos; at &apos;templateStage&apos; failed to satisfy "
+ + "constraint: Member must satisfy enum value set: [Processed, Original]"));
+ }
+
+ @Test
+ void getTemplateOnAPlainStackReturnsIdenticalBodiesAcrossStagesAndAdvertisesBothInStagesAvailable() {
+ // No transform at all: real AWS still accepts TemplateStage on a plain stack and returns
+ // the identical body for both stages (measured md5-identical on us-east-1), and
+ // StagesAvailable lists both stages on every GetTemplate call, including one with no
+ // stage requested, which floci never returned before this change.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stackName = "aws-include-stage-plain-stack-" + suffix;
+ String parameterName = "/aws-include-it/stage-plain-" + suffix;
+ stacksToDelete.add(stackName);
+
+ String template = """
+ Resources:
+ PlainStageParam:
+ Type: AWS::SSM::Parameter
+ Properties:
+ Name: %s
+ Type: String
+ Value: plain-value
+ """.formatted(parameterName);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ String processedBody = given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .formParam("TemplateStage", "Processed")
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .extract().path("GetTemplateResponse.GetTemplateResult.TemplateBody");
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "GetTemplate")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body("GetTemplateResponse.GetTemplateResult.TemplateBody", equalTo(processedBody))
+ .body(containsString(
+ "<StagesAvailable><member>Original</member><member>Processed</member></StagesAvailable>"));
+ }
+
+ @Test
+ void conditionsFromTheIncludedSnippetReachProvisioningInsteadOfBeingRejected() {
+ // validateConditionDependencies used to preflight the RAW, unmerged body: a Conditions
+ // section spliced in from a snippet was invisible there, so a resource conditioned on it
+ // defaulted to excluded and a dependent resource failed CreateStack synchronously with a
+ // spurious "Unresolved resource dependencies" error, even though the merge that runs later
+ // in executeTemplate would have resolved the condition to true.
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "aws-include-it-cond-" + suffix;
+ String key = "conditions.yaml";
+ String stackName = "aws-include-cond-stack-" + suffix;
+ String conditionedParamName = "/aws-include-it/conditioned-" + suffix;
+ String dependentParamName = "/aws-include-it/dependent-" + suffix;
+ stacksToDelete.add(stackName);
+
+ String location = putSnippet(bucket, key, "Conditions:\n FeatureEnabled: true\n");
+
+ String template = """
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ Resources:
+ ConditionedParam:
+ Type: AWS::SSM::Parameter
+ Condition: FeatureEnabled
+ Properties:
+ Name: %s
+ Type: String
+ Value: conditioned-value
+ DependentParam:
+ Type: AWS::SSM::Parameter
+ DependsOn: ConditionedParam
+ Properties:
+ Name: %s
+ Type: String
+ Value: dependent-value
+ """.formatted(location, conditionedParamName, dependentParamName);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "CreateStack")
+ .formParam("StackName", stackName)
+ .formParam("TemplateBody", template)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200);
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStacks")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"))
+ .body(not(containsString("Unresolved resource dependencies")));
+
+ given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStackResources")
+ .formParam("StackName", stackName)
+ .when()
+ .post("/")
+ .then()
+ .statusCode(200)
+ .body(containsString("<LogicalResourceId>ConditionedParam</LogicalResourceId>"))
+ .body(containsString("<LogicalResourceId>DependentParam</LogicalResourceId>"));
+ }
+
+ private static String stateMachineTemplate(String stateMachineName, String location) {
+ return """
+ Resources:
+ QueryPool:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ StateMachineName: %s
+ RoleArn: arn:aws:iam::000000000000:role/aws-include-it-role
+ DefinitionString: |-
+ {"StartAt":"Done","States":{"Done":{"Type":"Pass","Result":"${expandItems}","End":true}}}
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ """.formatted(stateMachineName, location);
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessorTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessorTest.java
new file mode 100644
index 0000000000..b36ac3ec03
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessorTest.java
@@ -0,0 +1,460 @@
+package io.github.hectorvent.floci.services.cloudformation;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.services.s3.S3Service;
+import io.github.hectorvent.floci.services.s3.model.S3Object;
+import org.junit.jupiter.api.Test;
+
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.List;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotSame;
+import static org.junit.jupiter.api.Assertions.assertSame;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * {@code Fn::Transform} with {@code Name: AWS::Include} splices the S3 snippet a
+ * {@code DefinitionSubstitutions} mapping points at into the template before any other intrinsic
+ * resolves, so the fragment's keys land in the enclosing mapping and {@code Fn::Transform} itself
+ * does not survive into it. {@code Location} accepts only an {@code s3://} URI: the template
+ * CloudFormation itself receives always carries one, because {@code aws cloudformation package}
+ * rewrites every local path before a stack ever sees it.
+ */
+class AwsIncludeProcessorTest {
+
+ private static final String BUCKET = "cfn-snippets";
+ private static final String KEY = "substitutions.yaml";
+ private static final String LOCATION = "s3://" + BUCKET + "/" + KEY;
+ private static final Path FRAGMENT_PATH =
+ Path.of("src/test/resources/cloudformation/aws-include/definition-substitutions.yaml");
+
+ // The exact value SnakeYAML 2.6 parses out of the fragment's folded (>-) block scalar: three
+ // line breaks preserved, because its continuation lines are more indented than the first, and
+ // AWS::Include's merge does not fold them (it splices the parsed node, unmodified).
+ private static final String EXPAND_ITEMS =
+ "{% (\n"
+ + " $c := function($v) { $exists($v.S) ? $v.S : $exists($v.N) ? $number($v.N) : $v };\n"
+ + " $map($states.result.Items, function($i) { $merge($map($keys($i), function($k) { {$k: $c($lookup($i, $k))} })) })\n"
+ + ") %}";
+
+ private final ObjectMapper mapper = new ObjectMapper();
+ private final S3Service s3Service = mock(S3Service.class);
+ private final AwsIncludeProcessor processor = new AwsIncludeProcessor(mapper, s3Service);
+
+ private void stubSnippet(String bucket, String key, String content) {
+ S3Object object = new S3Object();
+ object.setData(content.getBytes(StandardCharsets.UTF_8));
+ when(s3Service.getObject(bucket, key)).thenReturn(object);
+ }
+
+ private void stubMeasuredFragment() throws Exception {
+ stubSnippet(BUCKET, KEY, Files.readString(FRAGMENT_PATH));
+ }
+
+ private static String blockForm() {
+ return """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ """.formatted(LOCATION);
+ }
+
+ private static String flowForm() {
+ return """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform: { Name: AWS::Include, Parameters: { Location: %s } }
+ """.formatted(LOCATION);
+ }
+
+ private JsonNode mergedSubstitutions(String templateYaml) throws Exception {
+ JsonNode template = new CloudFormationYamlParser(mapper).parse(templateYaml);
+ JsonNode merged = processor.mergeIncludes(template);
+ return merged.at("/Resources/OrdersStateMachine/Properties/DefinitionSubstitutions");
+ }
+
+ @Test
+ void blockFormIncludeMergesSnippetKeysIntoSubstitutions() throws Exception {
+ stubMeasuredFragment();
+
+ JsonNode substitutions = mergedSubstitutions(blockForm());
+
+ assertEquals(EXPAND_ITEMS, substitutions.path("expandItems").asText(),
+ "AWS::Include did not merge the snippet's scalar verbatim; substitutions are " + substitutions);
+ assertEquals("orders", substitutions.get("orderTableName").asText());
+ assertFalse(substitutions.has("Fn::Transform"),
+ "the Fn::Transform node survived the merge");
+ }
+
+ @Test
+ void flowFormIncludeMergesSnippetKeysIntoSubstitutions() throws Exception {
+ stubMeasuredFragment();
+
+ JsonNode substitutions = mergedSubstitutions(flowForm());
+
+ assertEquals(EXPAND_ITEMS, substitutions.path("expandItems").asText(),
+ "AWS::Include did not merge the snippet's scalar verbatim; substitutions are " + substitutions);
+ assertEquals("orders", substitutions.get("orderTableName").asText());
+ assertFalse(substitutions.has("Fn::Transform"),
+ "the Fn::Transform node survived the merge");
+ }
+
+ @Test
+ void mergeIncludesReturnsTheSameReferenceWhenTemplateCarriesNoInclude() throws Exception {
+ JsonNode template = new CloudFormationYamlParser(mapper).parse("""
+ Resources:
+ Bucket:
+ Type: AWS::S3::Bucket
+ """);
+
+ JsonNode merged = processor.mergeIncludes(template);
+
+ // CloudFormationService.executeTemplate tells a real merge apart from a no-op one by
+ // comparing this return value's reference against the one it passed in; a copy here, even
+ // an unmodified one, would make every executeTemplate call reserialize the submitted body.
+ assertSame(template, merged,
+ "a template with no AWS::Include must come back as the exact same JsonNode instance");
+ }
+
+ @Test
+ void mergeIncludesReturnsADifferentReferenceWhenTemplateCarriesAnInclude() throws Exception {
+ stubMeasuredFragment();
+ JsonNode template = new CloudFormationYamlParser(mapper).parse(blockForm());
+
+ JsonNode merged = processor.mergeIncludes(template);
+
+ assertNotSame(template, merged,
+ "a template whose Fn::Transform/AWS::Include was actually merged must not come back "
+ + "as the caller's own unmodified reference");
+ }
+
+ @Test
+ void includeKeepsSiblingKeysOfTheTransformAndAppendsSnippetKeysAfterThem() throws Exception {
+ stubMeasuredFragment();
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ alreadyWrittenKey: alreadyWrittenValue
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ """.formatted(LOCATION);
+
+ JsonNode substitutions = mergedSubstitutions(template);
+
+ assertEquals("alreadyWrittenValue", substitutions.get("alreadyWrittenKey").asText(),
+ "a key already written beside Fn::Transform did not survive the merge");
+ assertEquals(EXPAND_ITEMS, substitutions.path("expandItems").asText());
+ assertEquals("orders", substitutions.get("orderTableName").asText());
+
+ // Matches the shape read back from a deployed stack's processed template:
+ // the sibling keys already written beside Fn::Transform keep their original order, and the
+ // snippet's own keys are appended after them, not interleaved or reordered.
+ List<String> fieldOrder = new ArrayList<>();
+ substitutions.fieldNames().forEachRemaining(fieldOrder::add);
+ assertEquals(List.of("alreadyWrittenKey", "expandItems", "orderTableName"), fieldOrder,
+ "sibling keys must precede the snippet's keys, both in their original order: " + fieldOrder);
+ }
+
+ @Test
+ void includeWithNonS3LocationIsRejected() {
+ String blockFormLocalPath = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: ./src/shared/substitutions.yaml
+ """;
+ AwsException relative = assertThrows(AwsException.class,
+ () -> mergedSubstitutions(blockFormLocalPath));
+ assertTrue(relative.getMessage().contains("./src/shared/substitutions.yaml"),
+ "message does not name the rejected location: " + relative.getMessage());
+
+ String blockFormBareName = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: swagger.yaml
+ """;
+ AwsException bare = assertThrows(AwsException.class,
+ () -> mergedSubstitutions(blockFormBareName));
+ assertTrue(bare.getMessage().contains("swagger.yaml"),
+ "message does not name the rejected location: " + bare.getMessage());
+ }
+
+ @Test
+ void locationAsAMappingNamesTheNodeInTheRejectionMessage() {
+ // Location: {Ref: InputValue} is the form AWS's own Fn::Transform documentation uses in its
+ // only AWS::Include example. Ref resolution is not implemented; the node must still be named.
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: { Ref: InputValue }
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertTrue(e.getMessage().contains("\"Ref\":\"InputValue\""),
+ "message does not name the rejected Location node: " + e.getMessage());
+ }
+
+ @Test
+ void includeWithUnreadableSnippetFailsTheStack() {
+ when(s3Service.getObject(BUCKET, KEY))
+ .thenThrow(new AwsException("NoSuchKey", "The specified key does not exist.", 404));
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(blockForm()));
+ assertTrue(e.getMessage().contains(LOCATION),
+ "message does not name the unreadable location: " + e.getMessage());
+ // A well-formed s3:// Location that cannot be read surfaces S3's own error, not a generic
+ // ValidationError: S3 already reports the precise cause and floci has nothing more specific
+ // to say.
+ assertEquals("NoSuchKey", e.getErrorCode());
+ assertEquals(404, e.getHttpStatus());
+ }
+
+ @Test
+ void nestedIncludeInsideASnippetIsRejected() {
+ String innerLocation = "s3://" + BUCKET + "/inner-fragment";
+ stubSnippet(BUCKET, KEY, """
+ expandItems:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ """.formatted(innerLocation));
+
+ assertThrows(AwsException.class, () -> mergedSubstitutions(blockForm()));
+ }
+
+ @Test
+ void snippetContainingAForeignTransformIsNotRejected() throws Exception {
+ // AWS's own considerations bullet is narrow: "You can't use AWS::Include to reference a
+ // template snippet that also uses AWS::Include." A snippet carrying an unrelated macro under
+ // Fn::Transform must pass through untouched, not fail the whole merge.
+ stubSnippet(BUCKET, KEY, """
+ someResource:
+ Fn::Transform:
+ Name: SomeOtherMacro
+ Parameters:
+ Foo: bar
+ orderTableName: orders
+ """);
+
+ JsonNode substitutions = mergedSubstitutions(blockForm());
+
+ assertEquals("SomeOtherMacro", substitutions.at("/someResource/Fn::Transform/Name").asText(),
+ "the foreign transform node did not survive the merge untouched");
+ assertEquals("bar", substitutions.at("/someResource/Fn::Transform/Parameters/Foo").asText());
+ assertEquals("orders", substitutions.get("orderTableName").asText());
+ }
+
+ @Test
+ void missingLocationNamesTheParametersObjectInsteadOfLeavingAnEmptyTail() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters: {}
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertTrue(e.getMessage().endsWith("got {}"),
+ "an absent Location must name the empty Parameters mapping, not leave an empty tail: "
+ + e.getMessage());
+ }
+
+ @Test
+ void parametersAbsentEntirelyNamesTheTransformNodeInsteadOfLeavingAnEmptyTail() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "
+ + "got {\"Name\":\"AWS::Include\"}", e.getMessage());
+ }
+
+ @Test
+ void twoResourcesEachWithTheirOwnFnTransformBothMerge() throws Exception {
+ // The measured corpus carries 41 Fn::Transform nodes in one template, one per resource's
+ // own Resources.<id>.Properties.DefinitionSubstitutions - never a single node shared across
+ // resources. A regression that merges the first one found and stops before walking the
+ // rest of the template would pass every single-resource test above and still ship broken.
+ stubMeasuredFragment();
+ String secondBucket = "second-fragment-bucket";
+ String secondKey = "second-fragment";
+ String secondLocation = "s3://" + secondBucket + "/" + secondKey;
+ stubSnippet(secondBucket, secondKey, "secondOrderTableName: second-orders\n");
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ InventoryStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: %s
+ """.formatted(LOCATION, secondLocation);
+
+ JsonNode merged = processor.mergeIncludes(new CloudFormationYamlParser(mapper).parse(template));
+ JsonNode first = merged.at("/Resources/OrdersStateMachine/Properties/DefinitionSubstitutions");
+ JsonNode second = merged.at("/Resources/InventoryStateMachine/Properties/DefinitionSubstitutions");
+
+ assertEquals("orders", first.path("orderTableName").asText(),
+ "the first resource's Fn::Transform did not merge; substitutions are " + first);
+ assertFalse(first.has("Fn::Transform"));
+ assertEquals("second-orders", second.path("secondOrderTableName").asText(),
+ "the second resource's Fn::Transform did not merge; a regression that splices the "
+ + "first include and stops would leave this unset: substitutions are " + second);
+ assertFalse(second.has("Fn::Transform"));
+ }
+
+ @Test
+ void emptyStringLocationRendersItsQuotedEmptyStringInsteadOfLeavingAnEmptyTail() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: ""
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "
+ + "got \"\"", e.getMessage());
+ }
+
+ @Test
+ void whitespaceOnlyLocationRendersItsQuotedStringInsteadOfLeavingAnEmptyTail() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ Location: " "
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "
+ + "got \" \"", e.getMessage());
+ }
+
+ @Test
+ void explicitNullParametersNamesTheTransformNodeInsteadOfLeavingAnEmptyTail() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters:
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "
+ + "got {\"Name\":\"AWS::Include\",\"Parameters\":null}", e.getMessage());
+ }
+
+ @Test
+ void stringParametersNamesTheTransformNodeInsteadOfLeakingItselfAsTheLocation() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters: "s3://b/k"
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "
+ + "got {\"Name\":\"AWS::Include\",\"Parameters\":\"s3://b/k\"}", e.getMessage());
+ }
+
+ @Test
+ void listParametersNamesTheTransformNodeInsteadOfLeakingItselfAsTheLocation() {
+ String template = """
+ Resources:
+ OrdersStateMachine:
+ Type: AWS::StepFunctions::StateMachine
+ Properties:
+ DefinitionSubstitutions:
+ Fn::Transform:
+ Name: AWS::Include
+ Parameters: [1, 2]
+ """;
+
+ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));
+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "
+ + "got {\"Name\":\"AWS::Include\",\"Parameters\":[1,2]}", e.getMessage());
+ }
+}
diff --git a/src/test/resources/cloudformation/aws-include/definition-substitutions.yaml b/src/test/resources/cloudformation/aws-include/definition-substitutions.yaml
new file mode 100644
index 0000000000..f5e720d198
--- /dev/null
+++ b/src/test/resources/cloudformation/aws-include/definition-substitutions.yaml
@@ -0,0 +1,12 @@
+# Fragment referenced by Fn::Transform / AWS::Include in AwsIncludeProcessorTest, mirroring the
+# shape of a measured DefinitionSubstitutions block. The folded block scalar below keeps its line
+# breaks under SnakeYAML, because its continuation lines are more indented than the first, and
+# AWS::Include's merge splices the parsed node unmodified: it does not fold the value. Folding, if
+# any, happens later, when the value is substituted into a definition and that definition is
+# reparsed as YAML - a separate concern from the merge this fragment exercises.
+expandItems: >-
+ {% (
+ $c := function($v) { $exists($v.S) ? $v.S : $exists($v.N) ? $number($v.N) : $v };
+ $map($states.result.Items, function($i) { $merge($map($keys($i), function($k) { {$k: $c($lookup($i, $k))} })) })
+ ) %}
+orderTableName: orders