Light, fluffy, and always free - The AWS Local Emulator alternative
fix(cloudformation): merge Fn::Transform AWS::Include before provisioning, and honour GetTemplate TemplateStage (#2886)
Facundo Paredes77b4881fe40e
diff --git a/docs/services/cloudformation.md b/docs/services/cloudformation.mdindex 11c1500dbe..f80af88160 100644--- a/docs/services/cloudformation.md+++ b/docs/services/cloudformation.md@@ -223,6 +223,57 @@ before provisioning:The `AWS::SSM::Parameter` **resource** type exposes `Value`, `Type`, and `Name` attributes through`Ref` / `Fn::GetAtt` so downstream resources can consume a parameter the same stack creates.+## AWS::Include (`Fn::Transform`)++`Fn::Transform` with `Name: AWS::Include` splices the YAML/JSON snippet at `Parameters.Location`+into its enclosing mapping before any other intrinsic resolves, on both `CreateStack`/`UpdateStack`+and the `CreateChangeSet` preview. `Location` must be an `s3://bucket/key` URI: the template+CloudFormation itself receives always carries one, because `aws cloudformation package` rewrites+every local path before a stack ever sees it. A `Location` that is not `s3://` fails the stack with+a `ValidationError` naming it, rather than silently dropping the include. A `Location` that is a+well-formed `s3://` URI but cannot be read (the bucket or key does not exist) fails the stack with+the underlying S3 error naming it, for example `NoSuchKey` at HTTP 404, instead of a+`ValidationError`, since that error is what S3 itself already reports and floci has no more specific+answer to give.++A snippet may not itself use `AWS::Include`; nesting is rejected rather than expanded or looped.+`Location` must be a plain string; a `Ref` or another intrinsic in its place is rejected, naming+the rejected node, rather than resolved.++Two known deviations from AWS:++- The snippet is parsed with floci's CloudFormation-aware YAML parser, so it accepts CloudFormation+ YAML short tags (`!Ref`, `!Sub`, ...) where AWS's own `AWS::Include` documentation says a snippet+ does not.+- AWS's own `Fn::Transform` documentation shows a `Location` written as an intrinsic function (its+ example uses `Ref`) and describes it as accepted; floci does not resolve one and rejects the+ template instead. This deviation comes from reading AWS's documentation, not from a request+ measured against a real account.++`CAPABILITY_AUTO_EXPAND` is not one of them. floci requires no capability for a template that+declares `AWS::Include`, and neither does AWS on the change-set path: a `CreateChangeSet` carrying+an embedded `Fn::Transform`/`AWS::Include` was accepted against a real account with+`CAPABILITY_IAM` alone. The `CreateStack` path was not measured.++`GetTemplateSummary` does not expand the include: it reports `AWS::Include` in the template's+`Transform`/`DeclaredTransforms` and neither fetches nor validates the snippet, matching AWS.+`GetTemplate` accepts `TemplateStage` (`Original`, the default, or `Processed`) and validates it+against that enum, rejecting anything else with a `ValidationError` naming the value, matching AWS.+`Original` returns the template exactly as submitted, `Fn::Transform` node and all. `Processed`+returns the merged and SAM-expanded tree, the same one a `CreateChangeSet` preview diffs against.++floci expands neither `AWS::LanguageExtensions`, nor a third-party macro, nor the top-level+`Transform: {Name: AWS::Include, Parameters: {Location: ...}}` form. A template carrying only one of+those three keeps `Processed` equal to `Original`, byte for byte.++Two triggers break that equality, because `executeTemplate` re-serializes the persisted body when+either one fires: an embedded `Fn::Transform` that merges, and a SAM transform. A SAM transform goes+further, because `SamTransformProcessor` removes the whole `Transform` section unconditionally, so a+macro co-declared beside SAM is absent from `Processed` as well.++Real AWS's answer for these shapes is unmeasured, co-declared with SAM or not. `StagesAvailable`+always lists both stages, matching AWS.+## ConditionsTemplate `Conditions` are evaluated before provisioning. A resource whose `Condition` evaluates todiff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessor.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessor.javanew file mode 100644index 0000000000..e3ad432c5a--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessor.java@@ -0,0 +1,180 @@+package io.github.hectorvent.floci.services.cloudformation;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.services.s3.S3Service;++import java.nio.charset.StandardCharsets;+import java.util.Iterator;+import java.util.Map;++/**+ * Expands {@code Fn::Transform} nodes whose {@code Name} is {@code AWS::Include}: splices the+ * YAML/JSON snippet the node's {@code Location} points at into the enclosing mapping, in place of+ * the {@code Fn::Transform} key, before any other intrinsic resolves, mirroring+ * {@link SamTransformProcessor}, which runs on the same parsed template ahead of provisioning.+ *+ * <p>{@code Location} accepts only an {@code s3://} URI. The template CloudFormation itself+ * receives always carries one: {@code aws cloudformation package} rewrites every local path before+ * a stack ever sees it, so a local path never reaches this processor in practice, and floci does+ * not read one on the caller's behalf (the CloudFormation package performs no filesystem access,+ * and a containerised floci sees none of the caller's paths regardless).+ */+class AwsIncludeProcessor {++ static final String AWS_INCLUDE = "AWS::Include";+ private static final String FN_TRANSFORM = "Fn::Transform";+ private static final String S3_SCHEME = "s3://";++ private final ObjectMapper objectMapper;+ private final S3Service s3Service;++ AwsIncludeProcessor(ObjectMapper objectMapper, S3Service s3Service) {+ this.objectMapper = objectMapper;+ this.s3Service = s3Service;+ }++ /**+ * Returns a copy of {@code template} with every {@code Fn::Transform}/{@code AWS::Include}+ * node replaced by the keys of the snippet it names, merged into the node's enclosing mapping+ * alongside any keys already written there.+ *+ * <p>Reference identity is part of this method's contract, not an incidental optimization: a+ * template carrying no {@code AWS::Include} is returned as the exact same {@code JsonNode}+ * instance passed in, unmodified, on every {@code executeTemplate} and every+ * {@code DescribeChangeSet} - the overwhelming majority of templates. Callers rely on that:+ * they compare the reference this method returns against the one they passed in to tell a real+ * merge apart from a no-op one, and a copy here even for a no-op merge would make every caller+ * treat an untouched template as changed.+ */+ JsonNode mergeIncludes(JsonNode template) {+ if (!containsAwsInclude(template)) {+ return template;+ }+ JsonNode merged = template.deepCopy();+ merge(merged);+ return merged;+ }++ private void merge(JsonNode node) {+ if (node.isObject()) {+ mergeObject((ObjectNode) node);+ } else if (node.isArray()) {+ node.forEach(this::merge);+ }+ }++ private void mergeObject(ObjectNode obj) {+ JsonNode transform = obj.get(FN_TRANSFORM);+ if (transform != null && AWS_INCLUDE.equals(transform.path("Name").asText(""))) {+ spliceInclude(obj, transform);+ }+ // Walk every remaining field, including the snippet's own keys just merged in above:+ // spliceInclude already rejects a snippet carrying its own AWS::Include, so revisiting+ // them here cannot re-trigger a merge - it only reaches unrelated Fn::Transform nodes+ // elsewhere in the template.+ Iterator<Map.Entry<String, JsonNode>> fields = obj.fields();+ while (fields.hasNext()) {+ merge(fields.next().getValue());+ }+ }++ private void spliceInclude(ObjectNode obj, JsonNode transform) {+ String location = describeLocation(transform);+ S3Location s3Location = parseS3Location(location);+ JsonNode snippet = fetchSnippet(s3Location, location);++ if (!snippet.isObject()) {+ throw new AwsException("ValidationError",+ "AWS::Include snippet at " + location + " must be a YAML/JSON mapping", 400);+ }+ if (containsAwsInclude(snippet)) {+ throw new AwsException("ValidationError",+ "AWS::Include snippet at " + location + " may not itself use AWS::Include", 400);+ }++ obj.remove(FN_TRANSFORM);+ obj.setAll((ObjectNode) snippet);+ }++ /**+ * Renders {@code transform.Parameters.Location} for an error message. A well-formed+ * {@code Location} is a plain string, but a node such as {@code {Ref: InputValue}}, the form+ * AWS's own {@code Fn::Transform} documentation uses in its {@code AWS::Include} example, is a+ * mapping. {@code Ref} resolution is not implemented here, so that mapping is serialized+ * instead, and the rejection message names it. A {@code Parameters} that is not an object+ * (missing, explicitly {@code null}, a scalar, or a list) names the whole {@code Fn::Transform}+ * node instead, so the message never presents an unrelated value as the rejected+ * {@code Location}. A {@code Location} missing from an object {@code Parameters} names the+ * enclosing {@code Parameters} mapping. A {@code Location} present but blank, including+ * whitespace only, is rendered as its quoted string, not elided, so the message never leaves an+ * empty tail.+ */+ private String describeLocation(JsonNode transform) {+ JsonNode parameters = transform.path("Parameters");+ if (!parameters.isObject()) {+ return transform.toString();+ }+ JsonNode locationNode = parameters.path("Location");+ if (locationNode.isTextual() && !locationNode.asText().isBlank()) {+ return locationNode.asText();+ }+ JsonNode described = locationNode.isMissingNode() || locationNode.isNull() ? parameters : locationNode;+ return described.toString();+ }++ /**+ * Returns whether {@code node} contains an unexpanded {@code Fn::Transform} whose {@code Name}+ * is {@code AWS::Include}, at any depth. A foreign macro under a different {@code Name} does not+ * count: AWS documents that a snippet may not itself use {@code AWS::Include}, not that it may+ * carry no other transform. Package-private: also used by {@code CloudFormationService} to+ * report {@code AWS::Include} in {@code GetTemplateSummary}'s {@code DeclaredTransforms} and to+ * skip {@code validateConditionDependencies} for a template the merge has not run against yet.+ */+ boolean containsAwsInclude(JsonNode node) {+ if (node.isObject()) {+ JsonNode transform = node.get(FN_TRANSFORM);+ if (transform != null && AWS_INCLUDE.equals(transform.path("Name").asText(""))) {+ return true;+ }+ }+ if (node.isContainerNode()) {+ for (JsonNode child : node) {+ if (containsAwsInclude(child)) {+ return true;+ }+ }+ }+ return false;+ }++ private JsonNode fetchSnippet(S3Location location, String rawLocation) {+ try {+ byte[] data = s3Service.getObject(location.bucket(), location.key()).getData();+ return new CloudFormationYamlParser(objectMapper).parse(new String(data, StandardCharsets.UTF_8));+ } catch (AwsException e) {+ throw new AwsException(e.getErrorCode(),+ "Unable to read AWS::Include snippet at " + rawLocation + ": " + e.getMessage(),+ e.getHttpStatus());+ } catch (Exception e) {+ throw new AwsException("ValidationError",+ "Unable to read AWS::Include snippet at " + rawLocation + ": " + e.getMessage(), 400);+ }+ }++ private record S3Location(String bucket, String key) {}++ private S3Location parseS3Location(String location) {+ if (location.startsWith(S3_SCHEME)) {+ String withoutScheme = location.substring(S3_SCHEME.length());+ int slash = withoutScheme.indexOf('/');+ if (slash > 0 && slash < withoutScheme.length() - 1) {+ return new S3Location(withoutScheme.substring(0, slash), withoutScheme.substring(slash + 1));+ }+ }+ throw new AwsException("ValidationError",+ "Fn::Transform AWS::Include Location must be an s3://bucket/key URI, got " + location, 400);+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.javaindex 776457efab..bf9f7a9f2e 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java@@ -443,17 +443,24 @@ public class CloudFormationQueryHandler {private Response getTemplate(MultivaluedMap<String, String> params, String region) {String stackName = params.getFirst("StackName");+ String templateStage = params.getFirst("TemplateStage");try {- String template = cfnService.getTemplate(stackName, region);- String xml = new XmlBuilder()+ String template = cfnService.getTemplate(stackName, templateStage, region);+ var xml = new XmlBuilder().start("GetTemplateResponse", CF_NS).start("GetTemplateResult")- .elem("TemplateBody", template)- .end("GetTemplateResult")- .raw(AwsQueryResponse.responseMetadata())- .end("GetTemplateResponse")- .build();- return Response.ok(xml).type("text/xml").build();+ .elem("TemplateBody", template);++ xml.start("StagesAvailable");+ for (String stage : cfnService.templateStagesAvailable()) {+ xml.elem("member", stage);+ }+ xml.end("StagesAvailable");++ xml.end("GetTemplateResult")+ .raw(AwsQueryResponse.responseMetadata())+ .end("GetTemplateResponse");+ return Response.ok(xml.build()).type("text/xml").build();} catch (AwsException e) {return xmlError(e.getErrorCode(), e.getMessage(), e.getHttpStatus());}diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.javaindex e2ee26e954..c0c0d842d5 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java@@ -68,6 +68,7 @@ public class CloudFormationService implements ResourceProvider {private final EmulatorConfig config;private final RegionResolver regionResolver;private final SamTransformProcessor samTransformProcessor;+ private final AwsIncludeProcessor awsIncludeProcessor;private final Clock clock;// Persisted state so stacks survive a restart (criteria #10, #11). The in-memory maps above are@@ -90,6 +91,7 @@ public class CloudFormationService implements ResourceProvider {this.config = config;this.regionResolver = regionResolver;this.samTransformProcessor = new SamTransformProcessor(objectMapper);+ this.awsIncludeProcessor = new AwsIncludeProcessor(objectMapper, s3Service);this.clock = clock;this.storageAccount = config.defaultAccountId();this.stackBackend = storageFactory.create(@@ -386,6 +388,9 @@ public class CloudFormationService implements ResourceProvider {Stack stack = getStackOrThrow(cs.getStackName(), region);try {JsonNode newTemplate = parseTemplate(cs.getTemplateBody());+ // Merge Fn::Transform/AWS::Include snippets before SAM expansion, matching AWS order:+ // an included fragment may itself carry SAM resources.+ newTemplate = awsIncludeProcessor.mergeIncludes(newTemplate);if (samTransformProcessor.hasSamTransform(newTemplate)) {// The deployed stack's template is always the SAM-expanded form (see// executeTemplate); comparing the change set's raw SAM source against it would@@ -674,9 +679,50 @@ public class CloudFormationService implements ResourceProvider {// ── GetTemplate ───────────────────────────────────────────────────────────- public String getTemplate(String stackName, String region) {+ private static final String STAGE_PROCESSED = "Processed";+ private static final String STAGE_ORIGINAL = "Original";++ // AWS's own enum order for the ValidationError message (measured against a real account).+ private static final List<String> TEMPLATE_STAGE_ENUM = List.of(STAGE_PROCESSED, STAGE_ORIGINAL);+ // AWS's own StagesAvailable order (Original first), which every GetTemplate call reports.+ private static final List<String> TEMPLATE_STAGES_AVAILABLE = List.of(STAGE_ORIGINAL, STAGE_PROCESSED);++ public String getTemplate(String stackName, String templateStage, String region) {+ // AWS validates TemplateStage before it looks the stack up (measured against a real+ // account: an invalid stage is rejected the same way whether or not the stack exists), so+ // this must run before getStackOrThrow.+ String stage = validateTemplateStage(templateStage);Stack stack = getStackOrThrow(stackName, region);- return stack.getTemplateBody() != null ? stack.getTemplateBody() : "{}";+ // Processed is the SAM/AWS::Include-expanded form templateBody holds after executeTemplate.+ // Original (also the default, matching real AWS) is the template exactly as the caller+ // submitted it. originalTemplateBody is only absent for stacks persisted by a floci version+ // predating this field, hence the fallback to templateBody; getTemplateSummary reads the+ // same field for the same reason.+ String body = STAGE_PROCESSED.equals(stage) ? stack.getTemplateBody() : stack.getOriginalTemplateBody();+ if (body == null) {+ body = stack.getTemplateBody();+ }+ return body != null ? body : "{}";+ }++ private String validateTemplateStage(String templateStage) {+ // null means the caller omitted TemplateStage; "" means the caller sent it present and+ // empty. AWS rejects the latter (measured against a real account) and only defaults the+ // former, so this must not treat blank the same as absent.+ if (templateStage == null) {+ return STAGE_ORIGINAL;+ }+ if (!TEMPLATE_STAGE_ENUM.contains(templateStage)) {+ throw new AwsException("ValidationError",+ "1 validation error detected: Value '" + templateStage + "' at 'templateStage' failed to "+ + "satisfy constraint: Member must satisfy enum value set: ["+ + String.join(", ", TEMPLATE_STAGE_ENUM) + "]", 400);+ }+ return templateStage;+ }++ public List<String> templateStagesAvailable() {+ return TEMPLATE_STAGES_AVAILABLE;}// ── GetTemplateSummary ────────────────────────────────────────────────────@@ -780,6 +826,12 @@ public class CloudFormationService implements ResourceProvider {}});}+ // AWS reports AWS::Include in DeclaredTransforms for the embedded Fn::Transform form too,+ // even with no top-level Transform section (measured against us-east-1).+ if (!declaredTransforms.contains(AwsIncludeProcessor.AWS_INCLUDE)+ && awsIncludeProcessor.containsAwsInclude(template)) {+ declaredTransforms.add(AwsIncludeProcessor.AWS_INCLUDE);+ }List<String> iamResourceTypes = resourceTypes.stream().filter(t -> t.startsWith("AWS::IAM::"))@@ -931,14 +983,28 @@ public class CloudFormationService implements ResourceProvider {JsonNode template = parseTemplate(templateBody);stack.setOriginalTemplateBody(templateBody);+ // Merge Fn::Transform/AWS::Include snippets before SAM expansion, matching AWS order:+ // an included fragment may itself carry SAM resources. mergeIncludes returns the same+ // reference, unchanged, when the template carries no AWS::Include, which is what lets+ // the check below tell a real merge apart from a no-op one.+ JsonNode beforeInclude = template;+ template = awsIncludeProcessor.mergeIncludes(template);+ boolean includeMerged = template != beforeInclude;+// Apply SAM transform if the template declares AWS::Serverless-2016-10-31- if (samTransformProcessor.hasSamTransform(template)) {+ boolean hasSamTransform = samTransformProcessor.hasSamTransform(template);+ if (hasSamTransform) {LOG.infov("Applying SAM transform for stack {0}", stack.getStackName());template = samTransformProcessor.expandSamTemplate(template);- // Store the expanded template so GetTemplate returns the transformed version- templateBody = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(template);}+ // Persist the merged/expanded tree, not the raw submitted body, whenever either+ // processor actually changed it, so the change-set baseline diffs against it instead of+ // against a stale Fn::Transform node. A template neither processor touched keeps its+ // submitted body byte for byte.+ if (includeMerged || hasSamTransform) {+ templateBody = objectMapper.writerWithDefaultPrettyPrinter().writeValueAsString(template);+ }stack.setTemplateBody(templateBody);// Merge default parameter values from the template with caller-supplied params@@ -1376,6 +1442,12 @@ public class CloudFormationService implements ResourceProvider {stack.getResolvedParameters().putAll(previousState.resolvedParameters());if (rollbackFailures.isEmpty()) {stack.setTemplateBody(previousState.templateBody());+ // GetTemplate reads originalTemplateBody for TemplateStage=Original and templateBody+ // for TemplateStage=Processed: without restoring originalTemplateBody here too, a+ // rolled-back update would keep serving the failed attempt's submitted body under+ // Original even though every other piece of state (resources, parameters, outputs)+ // was restored.+ stack.setOriginalTemplateBody(previousState.originalTemplateBody());}try {restoreOutputAndExportState(stack, region, previousState);@@ -1519,6 +1591,7 @@ public class CloudFormationService implements ResourceProvider {private StackUpdateSnapshot snapshotForUpdate(Stack stack) {return new StackUpdateSnapshot(stack.getTemplateBody(),+ stack.getOriginalTemplateBody(),new LinkedHashMap<>(stack.getParameters()),new LinkedHashMap<>(stack.getResolvedParameters()),new LinkedHashMap<>(stack.getOutputs()),@@ -1551,6 +1624,7 @@ public class CloudFormationService implements ResourceProvider {private record StackUpdateSnapshot(String templateBody,+ String originalTemplateBody,Map<String, String> parameters,Map<String, String> resolvedParameters,Map<String, String> outputs,@@ -1808,6 +1882,10 @@ public class CloudFormationService implements ResourceProvider {* template synchronously ("Template format error: Unresolved resource dependencies [...]")* rather than silently skipping the dependent, so mirror that instead of dropping the resource.* Malformed or SAM templates are left for the execution path, which surfaces their own errors.+ * A template carrying an unexpanded {@code Fn::Transform}/{@code AWS::Include} is left for the+ * same reason: a {@code Conditions} section spliced in from a snippet is invisible here, since+ * the merge has not run yet, and treating it as absent would fail a template whose dependency+ * graph the execution path resolves correctly.*/private void validateConditionDependencies(String templateBody, Map<String, String> params,String region, String accountId) {@@ -1819,7 +1897,7 @@ public class CloudFormationService implements ResourceProvider {e.getMessage());return;}- if (samTransformProcessor.hasSamTransform(template)) {+ if (samTransformProcessor.hasSamTransform(template) || awsIncludeProcessor.containsAwsInclude(template)) {return;}JsonNode resources = template.path("Resources");diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeIntegrationTest.javanew file mode 100644index 0000000000..201dcc0749--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeIntegrationTest.java@@ -0,0 +1,688 @@+package io.github.hectorvent.floci.services.cloudformation;++import com.fasterxml.jackson.databind.ObjectMapper;+import io.github.hectorvent.floci.services.s3.S3Service;+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import jakarta.inject.Inject;+import org.jboss.logging.Logger;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;++import java.nio.charset.StandardCharsets;+import java.util.ArrayList;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;++import static io.restassured.RestAssured.given;+import static org.hamcrest.MatcherAssert.assertThat;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.equalTo;+import static org.hamcrest.Matchers.not;+import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;++/**+ * The deployed outcome of {@code Fn::Transform}/{@code AWS::Include}: a {@code+ * DefinitionSubstitutions} mapping written as an include reaches the state machine's actual+ * definition, closing the symptom in "Request": 23 of 67 state machines in one repository+ * deploying with a literal {@code ${expandItems}}. The state machine is declared as+ * {@code AWS::StepFunctions::StateMachine}, not the SAM {@code AWS::Serverless::StateMachine} form+ * ({@code fix/sam-serverless-statemachine}'s subject); this test does not depend on that transform.+ */+@QuarkusTest+class AwsIncludeIntegrationTest {++ private static final Logger LOG = Logger.getLogger(AwsIncludeIntegrationTest.class);+ private static final String SFN_CONTENT_TYPE = "application/x-amz-json-1.0";++ @Inject+ S3Service s3Service;++ private final ObjectMapper mapper = new ObjectMapper();++ private final List<String> stacksToDelete = new ArrayList<>();+ private final Map<String, List<String>> objectsToDelete = new LinkedHashMap<>();++ @BeforeAll+ static void configureRestAssured() {+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @AfterEach+ void cleanUp() {+ for (String stackName : stacksToDelete) {+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DeleteStack")+ .formParam("StackName", stackName)+ .when()+ .post("/");+ }+ stacksToDelete.clear();++ objectsToDelete.forEach((bucket, keys) -> {+ try {+ for (String key : keys) {+ s3Service.deleteObject(bucket, key);+ }+ s3Service.deleteBucket(bucket);+ } catch (RuntimeException e) {+ // A bucket that a test (or a resource it provisioned) left with an object this+ // test never tracked throws BucketNotEmpty here. Left unguarded, that throw used+ // to abort the forEach before objectsToDelete.clear() ran, so the next @AfterEach+ // retried the same stale bucket/key pairs on top of its own, cascading one failed+ // delete into every test that ran afterward. Catching per-bucket keeps the loop+ // going for the buckets that follow and lets clear() below always run.+ LOG.warnv(e, "AwsIncludeIntegrationTest cleanup could not remove bucket {0}", bucket);+ }+ });+ objectsToDelete.clear();+ }++ private String putSnippet(String bucket, String key, String content) {+ s3Service.createBucket(bucket, "us-east-1");+ s3Service.putObject(bucket, key, content.getBytes(StandardCharsets.UTF_8), "text/yaml", Map.of());+ objectsToDelete.computeIfAbsent(bucket, b -> new ArrayList<>()).add(key);+ return "s3://" + bucket + "/" + key;+ }++ @Test+ void includedSubstitutionsReachTheStateMachineDefinition() throws Exception {+ String suffix = Long.toString(System.nanoTime(), 36);+ String bucket = "aws-include-it-" + suffix;+ String key = "definition-substitutions.yaml";+ String stackName = "aws-include-stack-" + suffix;+ String stateMachineName = "aws-include-sm-" + suffix;+ String stateMachineArn = "arn:aws:states:us-east-1:000000000000:stateMachine:" + stateMachineName;+ stacksToDelete.add(stackName);++ String location = putSnippet(bucket, key, "expandItems: substituted-value\n");++ String template = stateMachineTemplate(stateMachineName, location);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"));++ String definition = given()+ .header("X-Amz-Target", "AWSStepFunctions.DescribeStateMachine")+ .contentType(SFN_CONTENT_TYPE)+ .body("{\"stateMachineArn\":\"" + stateMachineArn + "\"}")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body("definition", containsString("substituted-value"))+ .body("definition", not(containsString("${expandItems}")))+ .extract().path("definition");++ assertDoesNotThrow(() -> mapper.readTree(definition),+ "the deployed definition does not parse as JSON: " + definition);+ }++ @Test+ void stackWithUnreadableSnippetFailsCreation() {+ // No bucket is created for this location: AWS::Include fails the same way any other+ // unreadable AWS resource reference does, before any resource in the stack is attempted:+ // the same class of pre-provisioning failure CloudFormationSsmParameterIntegrationTest pins+ // for a missing SSM parameter, and CREATE_FAILED (not a rollback) is what that class of+ // failure produces today: CloudFormationService's create-path exception handler sets+ // CREATE_FAILED directly, and only a resource that reached the provisioning loop drives the+ // ROLLBACK_IN_PROGRESS/ROLLBACK_COMPLETE transition.+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "aws-include-missing-stack-" + suffix;+ String stateMachineName = "aws-include-missing-sm-" + suffix;+ String location = "s3://aws-include-it-missing-bucket-" + suffix + "/definition-substitutions.yaml";+ stacksToDelete.add(stackName);++ String template = stateMachineTemplate(stateMachineName, location);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_FAILED</StackStatus>"))+ .body(containsString(location))+ .body(not(containsString("CREATE_COMPLETE")));++ String resourcesXml = given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStackResources")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .extract().asString();++ // The merge fails before the resource loop starts, so the state machine was never attempted:+ // it never even reaches CREATE_FAILED as a tracked resource, unlike a resource that fails+ // during provisioning itself.+ assertThat(resourcesXml, not(containsString("AWS::StepFunctions::StateMachine")));+ }++ @Test+ void getTemplateSummaryDoesNotExpandTheInclude() {+ // Same unreachable location as the unhappy path above, but through GetTemplateSummary:+ // AWS answers this with the transform declared and the snippet never fetched (measured+ // against us-east-1). Reusing the merge seam for GetTemplateSummary would instead fail the+ // summary on an unreachable location, which is the receipt this test exists to catch on a+ // future refactor.+ String suffix = Long.toString(System.nanoTime(), 36);+ String stateMachineName = "aws-include-summary-sm-" + suffix;+ String location = "s3://aws-include-it-missing-bucket-" + suffix + "/definition-substitutions.yaml";++ String template = stateMachineTemplate(stateMachineName, location);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplateSummary")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<member>AWS::StepFunctions::StateMachine</member>"))+ // Anchored to the enclosing element, not just to a bare <member>: TemplateSummary+ // also renders ResourceTypes and Capabilities as <member> lists, so an unanchored+ // containsString cannot tell DeclaredTransforms apart from either of them and would+ // stay green even if AWS::Include regressed into the wrong list.+ // Measured against us-east-1: AWS reports AWS::Include in DeclaredTransforms for the+ // embedded Fn::Transform form too, with no top-level Transform section and no fetch of+ // the (here unreachable) snippet.+ .body(containsString("<DeclaredTransforms><member>AWS::Include</member></DeclaredTransforms>"));+ }++ @Test+ void getTemplateReturnsTheSubmittedYamlWhileTheChangeSetBaselineStaysMerged() {+ // Superseded assertions, and why they were wrong: this test used to assert that GetTemplate+ // returned the MERGED body ("merged-value" present, "Fn::Transform" absent). That pinned a+ // real fix (a live Fn::Transform node was leaking into the deployed template because the+ // re-serialize sat inside the SAM-only branch) but overshot it: real CloudFormation's+ // GetTemplate defaults to TemplateStage=Original and returns the template exactly as the+ // caller submitted it, not the SAM/Include-expanded form (GetTemplateSummary already reads+ // stack.getOriginalTemplateBody() for the same reason). So GetTemplate must return the+ // caller's raw YAML, Fn::Transform node and all, while CreateChangeSet's baseline diff+ // keeps comparing against the merged body (stack.getTemplateBody()) so a resubmit of the+ // identical template still reports zero changes instead of a spurious Modify.+ String suffix = Long.toString(System.nanoTime(), 36);+ String bucket = "aws-include-it-persist-" + suffix;+ String key = "definition-substitutions.yaml";+ String stackName = "aws-include-persist-stack-" + suffix;+ String parameterName = "/aws-include-it/persist-" + suffix;+ stacksToDelete.add(stackName);++ String location = putSnippet(bucket, key, "mergedFlag: merged-value\n");++ String template = """+ Resources:+ IncludeParam:+ Type: AWS::SSM::Parameter+ Metadata:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ Properties:+ Name: %s+ Type: String+ Value: unmerged-value+ """.formatted(location, parameterName);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ // The submitted YAML, verbatim: the Fn::Transform node survives (Original stage, not+ // merged) and the YAML key syntax ("Value: unmerged-value", no quotes around either+ // side) rules out a reserialized JSON pretty-print of the merged tree. "mergedFlag" is+ // the snippet's own key, so its presence would mean the merge ran despite Original.+ .body(containsString("Fn::Transform"))+ .body(containsString("Value: unmerged-value"))+ .body(not(containsString("mergedFlag")));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "identical-resubmit-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "identical-resubmit-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ // The baseline diff still uses the merged body (stack.getTemplateBody(), untouched by+ // this fix): resubmitting the identical template must not report a spurious Modify.+ .body(not(containsString("<Action>Modify</Action>")));+ }++ @Test+ void getTemplateStageSelectsBetweenTheMergedBodyAndTheSubmittedBody() {+ // One stack carrying both a SAM resource and an embedded AWS::Include, so TemplateStage+ // exercises both processors at once: Processed is the merged/SAM-expanded body+ // (stack.getTemplateBody()) with AWS::Serverless::Function expanded away and the include+ // merged in, Original is the caller's raw YAML (stack.getOriginalTemplateBody()), and no+ // TemplateStage at all still defaults to Original, byte for byte.+ String suffix = Long.toString(System.nanoTime(), 36);+ String bucket = "aws-include-it-stage-" + suffix;+ String key = "stage-definition.yaml";+ String stackName = "aws-include-stage-stack-" + suffix;+ String functionName = "aws-include-stage-func-" + suffix;+ String parameterName = "/aws-include-it/stage-" + suffix;+ stacksToDelete.add(stackName);++ String location = putSnippet(bucket, key, "mergedFlag: merged-value\n");++ String template = """+ AWSTemplateFormatVersion: '2010-09-09'+ Transform: AWS::Serverless-2016-10-31+ Resources:+ StageFunction:+ Type: AWS::Serverless::Function+ Properties:+ FunctionName: %s+ Handler: index.handler+ Runtime: nodejs22.x+ InlineCode: |+ exports.handler = async () => ({ statusCode: 200, body: 'ok' });+ IncludeParam:+ Type: AWS::SSM::Parameter+ Metadata:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ Properties:+ Name: %s+ Type: String+ Value: unmerged-value+ """.formatted(functionName, location, parameterName);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .formParam("TemplateStage", "Processed")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("AWS::Lambda::Function"))+ .body(containsString("mergedFlag"))+ .body(not(containsString("AWS::Serverless::Function")))+ .body(not(containsString("Fn::Transform")));++ String originalBody = given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .formParam("TemplateStage", "Original")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("AWS::Serverless::Function"))+ .body(containsString("Fn::Transform"))+ .body(not(containsString("mergedFlag")))+ .extract().path("GetTemplateResponse.GetTemplateResult.TemplateBody");++ String defaultStageBody = given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .extract().path("GetTemplateResponse.GetTemplateResult.TemplateBody");++ assertThat(defaultStageBody, equalTo(originalBody));+ }++ @Test+ void getTemplateWithInvalidStageRejectsWithTheAwsEnumValidationError() {+ // Verbatim against real AWS (measured on us-east-1): an invalid TemplateStage value is+ // rejected with ValidationError naming the enum, on any existing stack.+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "aws-include-stage-bogus-stack-" + suffix;+ String parameterName = "/aws-include-it/stage-bogus-" + suffix;+ stacksToDelete.add(stackName);++ String template = """+ Resources:+ BogusStageParam:+ Type: AWS::SSM::Parameter+ Properties:+ Name: %s+ Type: String+ Value: plain-value+ """.formatted(parameterName);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .formParam("TemplateStage", "Bogus")+ .when()+ .post("/")+ .then()+ .statusCode(400)+ .body(containsString("<Code>ValidationError</Code>"))+ .body(containsString("Value 'Bogus' at 'templateStage' failed to satisfy "+ + "constraint: Member must satisfy enum value set: [Processed, Original]"));+ }++ @Test+ void getTemplateWithInvalidStageValidatesBeforeCheckingWhetherTheStackExists() {+ // Verbatim against real AWS (measured on us-east-1): `--template-stage Bogus` against a+ // stack name that was never created still answers with the enum ValidationError, not+ // "Stack with id ... does not exist", proving AWS validates the request shape before it+ // looks the stack up. This stack name is never created here, on purpose: a test that+ // creates the stack first (like the one above) cannot tell this ordering apart from+ // floci validating after a successful lookup.+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "aws-include-stage-bogus-no-stack-" + suffix;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .formParam("TemplateStage", "Bogus")+ .when()+ .post("/")+ .then()+ .statusCode(400)+ .body(containsString("<Code>ValidationError</Code>"))+ .body(containsString("Value 'Bogus' at 'templateStage' failed to satisfy "+ + "constraint: Member must satisfy enum value set: [Processed, Original]"))+ .body(not(containsString("does not exist")));+ }++ @Test+ void getTemplateWithEmptyStringStageRejectsWithTheAwsEnumValidationError() {+ // Verbatim against real AWS (measured on us-east-1): `--template-stage ""` is rejected the+ // same as any other value outside the enum. params.getFirst("TemplateStage") returns "" for+ // this request, distinct from the omitted case (null), which defaults to Original.+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "aws-include-stage-empty-stack-" + suffix;+ String parameterName = "/aws-include-it/stage-empty-" + suffix;+ stacksToDelete.add(stackName);++ String template = """+ Resources:+ EmptyStageParam:+ Type: AWS::SSM::Parameter+ Properties:+ Name: %s+ Type: String+ Value: plain-value+ """.formatted(parameterName);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .formParam("TemplateStage", "")+ .when()+ .post("/")+ .then()+ .statusCode(400)+ .body(containsString("<Code>ValidationError</Code>"))+ .body(containsString("Value '' at 'templateStage' failed to satisfy "+ + "constraint: Member must satisfy enum value set: [Processed, Original]"));+ }++ @Test+ void getTemplateOnAPlainStackReturnsIdenticalBodiesAcrossStagesAndAdvertisesBothInStagesAvailable() {+ // No transform at all: real AWS still accepts TemplateStage on a plain stack and returns+ // the identical body for both stages (measured md5-identical on us-east-1), and+ // StagesAvailable lists both stages on every GetTemplate call, including one with no+ // stage requested, which floci never returned before this change.+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "aws-include-stage-plain-stack-" + suffix;+ String parameterName = "/aws-include-it/stage-plain-" + suffix;+ stacksToDelete.add(stackName);++ String template = """+ Resources:+ PlainStageParam:+ Type: AWS::SSM::Parameter+ Properties:+ Name: %s+ Type: String+ Value: plain-value+ """.formatted(parameterName);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ String processedBody = given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .formParam("TemplateStage", "Processed")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .extract().path("GetTemplateResponse.GetTemplateResult.TemplateBody");++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetTemplate")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body("GetTemplateResponse.GetTemplateResult.TemplateBody", equalTo(processedBody))+ .body(containsString(+ "<StagesAvailable><member>Original</member><member>Processed</member></StagesAvailable>"));+ }++ @Test+ void conditionsFromTheIncludedSnippetReachProvisioningInsteadOfBeingRejected() {+ // validateConditionDependencies used to preflight the RAW, unmerged body: a Conditions+ // section spliced in from a snippet was invisible there, so a resource conditioned on it+ // defaulted to excluded and a dependent resource failed CreateStack synchronously with a+ // spurious "Unresolved resource dependencies" error, even though the merge that runs later+ // in executeTemplate would have resolved the condition to true.+ String suffix = Long.toString(System.nanoTime(), 36);+ String bucket = "aws-include-it-cond-" + suffix;+ String key = "conditions.yaml";+ String stackName = "aws-include-cond-stack-" + suffix;+ String conditionedParamName = "/aws-include-it/conditioned-" + suffix;+ String dependentParamName = "/aws-include-it/dependent-" + suffix;+ stacksToDelete.add(stackName);++ String location = putSnippet(bucket, key, "Conditions:\n FeatureEnabled: true\n");++ String template = """+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ Resources:+ ConditionedParam:+ Type: AWS::SSM::Parameter+ Condition: FeatureEnabled+ Properties:+ Name: %s+ Type: String+ Value: conditioned-value+ DependentParam:+ Type: AWS::SSM::Parameter+ DependsOn: ConditionedParam+ Properties:+ Name: %s+ Type: String+ Value: dependent-value+ """.formatted(location, conditionedParamName, dependentParamName);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"))+ .body(not(containsString("Unresolved resource dependencies")));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStackResources")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>ConditionedParam</LogicalResourceId>"))+ .body(containsString("<LogicalResourceId>DependentParam</LogicalResourceId>"));+ }++ private static String stateMachineTemplate(String stateMachineName, String location) {+ return """+ Resources:+ QueryPool:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ StateMachineName: %s+ RoleArn: arn:aws:iam::000000000000:role/aws-include-it-role+ DefinitionString: |-+ {"StartAt":"Done","States":{"Done":{"Type":"Pass","Result":"${expandItems}","End":true}}}+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ """.formatted(stateMachineName, location);+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessorTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessorTest.javanew file mode 100644index 0000000000..b36ac3ec03--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/AwsIncludeProcessorTest.java@@ -0,0 +1,460 @@+package io.github.hectorvent.floci.services.cloudformation;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.services.s3.S3Service;+import io.github.hectorvent.floci.services.s3.model.S3Object;+import org.junit.jupiter.api.Test;++import java.nio.charset.StandardCharsets;+import java.nio.file.Files;+import java.nio.file.Path;+import java.util.ArrayList;+import java.util.List;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertNotSame;+import static org.junit.jupiter.api.Assertions.assertSame;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.when;++/**+ * {@code Fn::Transform} with {@code Name: AWS::Include} splices the S3 snippet a+ * {@code DefinitionSubstitutions} mapping points at into the template before any other intrinsic+ * resolves, so the fragment's keys land in the enclosing mapping and {@code Fn::Transform} itself+ * does not survive into it. {@code Location} accepts only an {@code s3://} URI: the template+ * CloudFormation itself receives always carries one, because {@code aws cloudformation package}+ * rewrites every local path before a stack ever sees it.+ */+class AwsIncludeProcessorTest {++ private static final String BUCKET = "cfn-snippets";+ private static final String KEY = "substitutions.yaml";+ private static final String LOCATION = "s3://" + BUCKET + "/" + KEY;+ private static final Path FRAGMENT_PATH =+ Path.of("src/test/resources/cloudformation/aws-include/definition-substitutions.yaml");++ // The exact value SnakeYAML 2.6 parses out of the fragment's folded (>-) block scalar: three+ // line breaks preserved, because its continuation lines are more indented than the first, and+ // AWS::Include's merge does not fold them (it splices the parsed node, unmodified).+ private static final String EXPAND_ITEMS =+ "{% (\n"+ + " $c := function($v) { $exists($v.S) ? $v.S : $exists($v.N) ? $number($v.N) : $v };\n"+ + " $map($states.result.Items, function($i) { $merge($map($keys($i), function($k) { {$k: $c($lookup($i, $k))} })) })\n"+ + ") %}";++ private final ObjectMapper mapper = new ObjectMapper();+ private final S3Service s3Service = mock(S3Service.class);+ private final AwsIncludeProcessor processor = new AwsIncludeProcessor(mapper, s3Service);++ private void stubSnippet(String bucket, String key, String content) {+ S3Object object = new S3Object();+ object.setData(content.getBytes(StandardCharsets.UTF_8));+ when(s3Service.getObject(bucket, key)).thenReturn(object);+ }++ private void stubMeasuredFragment() throws Exception {+ stubSnippet(BUCKET, KEY, Files.readString(FRAGMENT_PATH));+ }++ private static String blockForm() {+ return """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ """.formatted(LOCATION);+ }++ private static String flowForm() {+ return """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform: { Name: AWS::Include, Parameters: { Location: %s } }+ """.formatted(LOCATION);+ }++ private JsonNode mergedSubstitutions(String templateYaml) throws Exception {+ JsonNode template = new CloudFormationYamlParser(mapper).parse(templateYaml);+ JsonNode merged = processor.mergeIncludes(template);+ return merged.at("/Resources/OrdersStateMachine/Properties/DefinitionSubstitutions");+ }++ @Test+ void blockFormIncludeMergesSnippetKeysIntoSubstitutions() throws Exception {+ stubMeasuredFragment();++ JsonNode substitutions = mergedSubstitutions(blockForm());++ assertEquals(EXPAND_ITEMS, substitutions.path("expandItems").asText(),+ "AWS::Include did not merge the snippet's scalar verbatim; substitutions are " + substitutions);+ assertEquals("orders", substitutions.get("orderTableName").asText());+ assertFalse(substitutions.has("Fn::Transform"),+ "the Fn::Transform node survived the merge");+ }++ @Test+ void flowFormIncludeMergesSnippetKeysIntoSubstitutions() throws Exception {+ stubMeasuredFragment();++ JsonNode substitutions = mergedSubstitutions(flowForm());++ assertEquals(EXPAND_ITEMS, substitutions.path("expandItems").asText(),+ "AWS::Include did not merge the snippet's scalar verbatim; substitutions are " + substitutions);+ assertEquals("orders", substitutions.get("orderTableName").asText());+ assertFalse(substitutions.has("Fn::Transform"),+ "the Fn::Transform node survived the merge");+ }++ @Test+ void mergeIncludesReturnsTheSameReferenceWhenTemplateCarriesNoInclude() throws Exception {+ JsonNode template = new CloudFormationYamlParser(mapper).parse("""+ Resources:+ Bucket:+ Type: AWS::S3::Bucket+ """);++ JsonNode merged = processor.mergeIncludes(template);++ // CloudFormationService.executeTemplate tells a real merge apart from a no-op one by+ // comparing this return value's reference against the one it passed in; a copy here, even+ // an unmodified one, would make every executeTemplate call reserialize the submitted body.+ assertSame(template, merged,+ "a template with no AWS::Include must come back as the exact same JsonNode instance");+ }++ @Test+ void mergeIncludesReturnsADifferentReferenceWhenTemplateCarriesAnInclude() throws Exception {+ stubMeasuredFragment();+ JsonNode template = new CloudFormationYamlParser(mapper).parse(blockForm());++ JsonNode merged = processor.mergeIncludes(template);++ assertNotSame(template, merged,+ "a template whose Fn::Transform/AWS::Include was actually merged must not come back "+ + "as the caller's own unmodified reference");+ }++ @Test+ void includeKeepsSiblingKeysOfTheTransformAndAppendsSnippetKeysAfterThem() throws Exception {+ stubMeasuredFragment();+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ alreadyWrittenKey: alreadyWrittenValue+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ """.formatted(LOCATION);++ JsonNode substitutions = mergedSubstitutions(template);++ assertEquals("alreadyWrittenValue", substitutions.get("alreadyWrittenKey").asText(),+ "a key already written beside Fn::Transform did not survive the merge");+ assertEquals(EXPAND_ITEMS, substitutions.path("expandItems").asText());+ assertEquals("orders", substitutions.get("orderTableName").asText());++ // Matches the shape read back from a deployed stack's processed template:+ // the sibling keys already written beside Fn::Transform keep their original order, and the+ // snippet's own keys are appended after them, not interleaved or reordered.+ List<String> fieldOrder = new ArrayList<>();+ substitutions.fieldNames().forEachRemaining(fieldOrder::add);+ assertEquals(List.of("alreadyWrittenKey", "expandItems", "orderTableName"), fieldOrder,+ "sibling keys must precede the snippet's keys, both in their original order: " + fieldOrder);+ }++ @Test+ void includeWithNonS3LocationIsRejected() {+ String blockFormLocalPath = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: ./src/shared/substitutions.yaml+ """;+ AwsException relative = assertThrows(AwsException.class,+ () -> mergedSubstitutions(blockFormLocalPath));+ assertTrue(relative.getMessage().contains("./src/shared/substitutions.yaml"),+ "message does not name the rejected location: " + relative.getMessage());++ String blockFormBareName = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: swagger.yaml+ """;+ AwsException bare = assertThrows(AwsException.class,+ () -> mergedSubstitutions(blockFormBareName));+ assertTrue(bare.getMessage().contains("swagger.yaml"),+ "message does not name the rejected location: " + bare.getMessage());+ }++ @Test+ void locationAsAMappingNamesTheNodeInTheRejectionMessage() {+ // Location: {Ref: InputValue} is the form AWS's own Fn::Transform documentation uses in its+ // only AWS::Include example. Ref resolution is not implemented; the node must still be named.+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: { Ref: InputValue }+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertTrue(e.getMessage().contains("\"Ref\":\"InputValue\""),+ "message does not name the rejected Location node: " + e.getMessage());+ }++ @Test+ void includeWithUnreadableSnippetFailsTheStack() {+ when(s3Service.getObject(BUCKET, KEY))+ .thenThrow(new AwsException("NoSuchKey", "The specified key does not exist.", 404));++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(blockForm()));+ assertTrue(e.getMessage().contains(LOCATION),+ "message does not name the unreadable location: " + e.getMessage());+ // A well-formed s3:// Location that cannot be read surfaces S3's own error, not a generic+ // ValidationError: S3 already reports the precise cause and floci has nothing more specific+ // to say.+ assertEquals("NoSuchKey", e.getErrorCode());+ assertEquals(404, e.getHttpStatus());+ }++ @Test+ void nestedIncludeInsideASnippetIsRejected() {+ String innerLocation = "s3://" + BUCKET + "/inner-fragment";+ stubSnippet(BUCKET, KEY, """+ expandItems:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ """.formatted(innerLocation));++ assertThrows(AwsException.class, () -> mergedSubstitutions(blockForm()));+ }++ @Test+ void snippetContainingAForeignTransformIsNotRejected() throws Exception {+ // AWS's own considerations bullet is narrow: "You can't use AWS::Include to reference a+ // template snippet that also uses AWS::Include." A snippet carrying an unrelated macro under+ // Fn::Transform must pass through untouched, not fail the whole merge.+ stubSnippet(BUCKET, KEY, """+ someResource:+ Fn::Transform:+ Name: SomeOtherMacro+ Parameters:+ Foo: bar+ orderTableName: orders+ """);++ JsonNode substitutions = mergedSubstitutions(blockForm());++ assertEquals("SomeOtherMacro", substitutions.at("/someResource/Fn::Transform/Name").asText(),+ "the foreign transform node did not survive the merge untouched");+ assertEquals("bar", substitutions.at("/someResource/Fn::Transform/Parameters/Foo").asText());+ assertEquals("orders", substitutions.get("orderTableName").asText());+ }++ @Test+ void missingLocationNamesTheParametersObjectInsteadOfLeavingAnEmptyTail() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters: {}+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertTrue(e.getMessage().endsWith("got {}"),+ "an absent Location must name the empty Parameters mapping, not leave an empty tail: "+ + e.getMessage());+ }++ @Test+ void parametersAbsentEntirelyNamesTheTransformNodeInsteadOfLeavingAnEmptyTail() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "+ + "got {\"Name\":\"AWS::Include\"}", e.getMessage());+ }++ @Test+ void twoResourcesEachWithTheirOwnFnTransformBothMerge() throws Exception {+ // The measured corpus carries 41 Fn::Transform nodes in one template, one per resource's+ // own Resources.<id>.Properties.DefinitionSubstitutions - never a single node shared across+ // resources. A regression that merges the first one found and stops before walking the+ // rest of the template would pass every single-resource test above and still ship broken.+ stubMeasuredFragment();+ String secondBucket = "second-fragment-bucket";+ String secondKey = "second-fragment";+ String secondLocation = "s3://" + secondBucket + "/" + secondKey;+ stubSnippet(secondBucket, secondKey, "secondOrderTableName: second-orders\n");+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ InventoryStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: %s+ """.formatted(LOCATION, secondLocation);++ JsonNode merged = processor.mergeIncludes(new CloudFormationYamlParser(mapper).parse(template));+ JsonNode first = merged.at("/Resources/OrdersStateMachine/Properties/DefinitionSubstitutions");+ JsonNode second = merged.at("/Resources/InventoryStateMachine/Properties/DefinitionSubstitutions");++ assertEquals("orders", first.path("orderTableName").asText(),+ "the first resource's Fn::Transform did not merge; substitutions are " + first);+ assertFalse(first.has("Fn::Transform"));+ assertEquals("second-orders", second.path("secondOrderTableName").asText(),+ "the second resource's Fn::Transform did not merge; a regression that splices the "+ + "first include and stops would leave this unset: substitutions are " + second);+ assertFalse(second.has("Fn::Transform"));+ }++ @Test+ void emptyStringLocationRendersItsQuotedEmptyStringInsteadOfLeavingAnEmptyTail() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: ""+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "+ + "got \"\"", e.getMessage());+ }++ @Test+ void whitespaceOnlyLocationRendersItsQuotedStringInsteadOfLeavingAnEmptyTail() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ Location: " "+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "+ + "got \" \"", e.getMessage());+ }++ @Test+ void explicitNullParametersNamesTheTransformNodeInsteadOfLeavingAnEmptyTail() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters:+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "+ + "got {\"Name\":\"AWS::Include\",\"Parameters\":null}", e.getMessage());+ }++ @Test+ void stringParametersNamesTheTransformNodeInsteadOfLeakingItselfAsTheLocation() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters: "s3://b/k"+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "+ + "got {\"Name\":\"AWS::Include\",\"Parameters\":\"s3://b/k\"}", e.getMessage());+ }++ @Test+ void listParametersNamesTheTransformNodeInsteadOfLeakingItselfAsTheLocation() {+ String template = """+ Resources:+ OrdersStateMachine:+ Type: AWS::StepFunctions::StateMachine+ Properties:+ DefinitionSubstitutions:+ Fn::Transform:+ Name: AWS::Include+ Parameters: [1, 2]+ """;++ AwsException e = assertThrows(AwsException.class, () -> mergedSubstitutions(template));+ assertEquals("Fn::Transform AWS::Include Location must be an s3://bucket/key URI, "+ + "got {\"Name\":\"AWS::Include\",\"Parameters\":[1,2]}", e.getMessage());+ }+}diff --git a/src/test/resources/cloudformation/aws-include/definition-substitutions.yaml b/src/test/resources/cloudformation/aws-include/definition-substitutions.yamlnew file mode 100644index 0000000000..f5e720d198--- /dev/null+++ b/src/test/resources/cloudformation/aws-include/definition-substitutions.yaml@@ -0,0 +1,12 @@+# Fragment referenced by Fn::Transform / AWS::Include in AwsIncludeProcessorTest, mirroring the+# shape of a measured DefinitionSubstitutions block. The folded block scalar below keeps its line+# breaks under SnakeYAML, because its continuation lines are more indented than the first, and+# AWS::Include's merge splices the parsed node unmodified: it does not fold the value. Folding, if+# any, happens later, when the value is substituted into a definition and that definition is+# reparsed as YAML - a separate concern from the merge this fragment exercises.+expandItems: >-+ {% (+ $c := function($v) { $exists($v.S) ? $v.S : $exists($v.N) ? $number($v.N) : $v };+ $map($states.result.Items, function($i) { $merge($map($keys($i), function($k) { {$k: $c($lookup($i, $k))} })) })+ ) %}+orderTableName: orders