Light, fluffy, and always free - The AWS Local Emulator alternative
fix(cloudformation): copy a SAM function's VersionDescription to its version (#6770)
Jackson Pricea6cf29ea8659
diff --git a/compatibility-tests/sdk-test-awscli/test/cloudformation.bats b/compatibility-tests/sdk-test-awscli/test/cloudformation.batsindex 4e0ad28012..10f0eae52c 100644--- a/compatibility-tests/sdk-test-awscli/test/cloudformation.bats+++ b/compatibility-tests/sdk-test-awscli/test/cloudformation.bats@@ -368,3 +368,47 @@ EOFassert_success[ "$(json_get "$output" '[.Versions[].Version] | join(",")')" = '$LATEST,1,2' ]}++sam_version_description_template() {+ cat << EOF+Transform: AWS::Serverless-2016-10-31+Resources:+ Fn:+ Type: AWS::Serverless::Function+ Properties:+ FunctionName: $1+ Runtime: nodejs20.x+ Handler: index.handler+ AutoPublishAlias: live+ VersionDescription: release $2+ InlineCode: "exports.handler = async () => ({ release: '$2' });"+EOF+}++@test "CloudFormation: SAM VersionDescription describes each published version" {+ local fn+ fn=$(unique_name bats-sam-version-description)+ sam_version_description_template "$fn" first > "$TEMPLATE_FILE"+ run aws_cmd cloudformation deploy \+ --stack-name "$STACK_NAME" \+ --template-file "$TEMPLATE_FILE" \+ --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND+ assert_success++ run aws_cmd lambda get-function-configuration --function-name "$fn" --qualifier live+ assert_success+ [ "$(json_get "$output" '.Version')" = "1" ]+ [ "$(json_get "$output" '.Description')" = "release first" ]++ # The next version takes the new description, and the first keeps its own.+ sam_version_description_template "$fn" second > "$TEMPLATE_FILE"+ run aws_cmd cloudformation deploy \+ --stack-name "$STACK_NAME" \+ --template-file "$TEMPLATE_FILE" \+ --capabilities CAPABILITY_IAM CAPABILITY_AUTO_EXPAND+ assert_success++ run aws_cmd lambda list-versions-by-function --function-name "$fn"+ assert_success+ [ "$(json_get "$output" '[.Versions[] | select(.Version != "$LATEST") | .Version + "=" + .Description] | join(",")')" = '1=release first,2=release second' ]+}diff --git a/docs/services/cloudformation.md b/docs/services/cloudformation.mdindex 10ea328e45..a2e9d5f31d 100644--- a/docs/services/cloudformation.md+++ b/docs/services/cloudformation.md@@ -458,15 +458,16 @@ references are resolved first, one that is another intrinsic function is joinedparameter holding an empty value still appends it: a description then ends in a space, and afunction without one gets an empty `Description`. `Globals` can set`AutoPublishAliasAllProperties` and `VersionDeletionPolicy`, but as in SAM not-`AutoPublishCodeSha256`, which fails the transform there. floci does not honour a resource's-`IgnoreGlobals`, so no function can opt out of a `VersionDeletionPolicy` in `Globals` with it: one-without `AutoPublishAlias` fails the transform with+`AutoPublishCodeSha256` or `VersionDescription`, which fail the transform there. floci does not+honour a resource's `IgnoreGlobals`, so no function can opt out of a `VersionDeletionPolicy` in+`Globals` with it: one without `AutoPublishAlias` fails the transform with`'VersionDeletionPolicy' requires 'AutoPublishAlias'.`, as it does in SAM when it does not ignorethat global.A new id publishes a version, and moves the alias to it, only when the function's code or-configuration changed since its last version. Otherwise the new version resource takes the last-version and the alias stays where it is, as when an update changes only one of these:+configuration changed since its last version, or, in floci, also when its `VersionDescription`+differs from that version's description (see below). Otherwise the new version resource takes the+last version and the alias stays where it is, as when an update changes only one of these:- A change to `SnapStart`: floci does not give the function `SnapStart`.- Under `AutoPublishAliasAllProperties`, a change to `Tags` or `ReservedConcurrentExecutions`@@ -487,6 +488,19 @@ attributes, unless a transform such as `AWS::LanguageExtensions` resolves them fimplements no such transform: it reads any `DeletionPolicy` that is not a plain string as `Delete`(see [Deletion Policies](#deletion-policies)).+As in SAM, a function's `VersionDescription` becomes its version's `Description`, copied as+written, so an intrinsic function in it is resolved when the version is published. It is not part+of the hash, so a new `VersionDescription` alone keeps the version's id, and CloudFormation can+change a version's `Description` only by replacing the version. AWS fails that update: in 2021,+[SAM issue #2786](https://github.com/aws/serverless-application-model/issues/2786) showed a change+set that modified the version in place, then `Internal Failure` and a rollback. floci completes it+and keeps the version, with the description it was published with, although its change set+reports the version replaced when the description is changed or removed. The next new id then+publishes, with the new description, even if the function has not changed since. On AWS that+update is expected to fail like the `Tags` change above, since+[PublishVersion](https://docs.aws.amazon.com/lambda/latest/api/API_PublishVersion.html) publishes+nothing for a function whose code and configuration have not changed; this is not measured.+Changes made to `$LATEST` outside the stack, including Lambda's reactive S3 sync, do not reach thealias until an update publishes a new version. Event sources and API integrations still invoke thefunction rather than the alias, so they run `$LATEST`.diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.javaindex 8989b223ce..939bd58da5 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessor.java@@ -97,11 +97,14 @@ class SamTransformProcessor {// (including the early return below when Resources is absent or not an object).JsonNode globals = expanded.path("Globals");expanded.remove("Globals");- // SAM refuses any Globals property it does not list for a section. AutoPublishCodeSha256 is- // one of them: one value there would name every function's version.- if (globals.path("Function").has("AutoPublishCodeSha256")) {- throw new AwsException("ValidationError", "'Globals' section is invalid. 'AutoPublishCodeSha256' "- + "is not a supported property of 'Function'.", 400);+ // SAM refuses any Globals property it does not list for a section. AutoPublishCodeSha256 and+ // VersionDescription are two of them: one value there would name, or describe, every+ // function's version.+ for (String property : List.of("AutoPublishCodeSha256", "VersionDescription")) {+ if (globals.path("Function").has(property)) {+ throw new AwsException("ValidationError", "'Globals' section is invalid. '" + property+ + "' is not a supported property of 'Function'.", 400);+ }}JsonNode resources = expanded.path("Resources");@@ -1020,6 +1023,7 @@ class SamTransformProcessor {versionDef.put("Type", "AWS::Lambda::Version");ObjectNode versionProps = objectMapper.createObjectNode();versionProps.set("FunctionName", ref(functionLogicalId));+ copyRenamed(properties, "VersionDescription", versionProps, "Description");versionDef.set("Properties", versionProps);versionDef.setAll(passthrough.deepCopy());// The version a new hash replaces leaves the template on that update; like SAM, keep itdiff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformIntegrationTest.javaindex b73661bd5f..c343c4beba 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformIntegrationTest.java@@ -20,6 +20,7 @@ import java.util.zip.ZipOutputStream;import static io.restassured.RestAssured.given;import static org.hamcrest.MatcherAssert.assertThat;import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.emptyOrNullString;import static org.hamcrest.Matchers.equalTo;import static org.hamcrest.Matchers.hasItem;import static org.hamcrest.Matchers.hasItems;@@ -527,6 +528,146 @@ class SamTransformIntegrationTest {.body("__type", equalTo("ProvisionedConcurrencyConfigNotFoundException"));}+ /**+ * VersionDescription describes the version the alias publishes, resolved as CloudFormation+ * resolves it, and not the function; each version keeps the description it was published with.+ */+ @Test+ void samVersionDescriptionDescribesEachPublishedVersion() {+ String stackName = "sam-version-description-stack";+ String function = "sam-version-description-func";+ stacksToDelete.add(stackName);+ String template = """+ Transform: AWS::Serverless-2016-10-31+ Parameters:+ Release:+ Type: String+ Resources:+ Function:+ Type: AWS::Serverless::Function+ Properties:+ FunctionName: sam-version-description-func+ Handler: index.handler+ Runtime: nodejs20.x+ AutoPublishAlias: live+ VersionDescription:+ Fn::Sub: release ${Release}+ InlineCode: "exports.handler = async () => ({ release: '%s' });"+ """;+ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack").formParam("StackName", stackName)+ .formParam("TemplateBody", template.formatted("first"))+ .formParam("Parameters.member.1.ParameterKey", "Release")+ .formParam("Parameters.member.1.ParameterValue", "1")+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when().post("/").then().statusCode(200);+ waitForStackStatus(stackName, "CREATE_COMPLETE");+ given().queryParam("Qualifier", "live")+ .when().get("/2015-03-31/functions/" + function + "/configuration")+ .then().statusCode(200).body("Version", equalTo("1")).body("Description", equalTo("release 1"));++ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "UpdateStack").formParam("StackName", stackName)+ .formParam("TemplateBody", template.formatted("second"))+ .formParam("Parameters.member.1.ParameterKey", "Release")+ .formParam("Parameters.member.1.ParameterValue", "2")+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when().post("/").then().statusCode(200);+ waitForStackStatus(stackName, "UPDATE_COMPLETE");++ given().queryParam("Qualifier", "live")+ .when().get("/2015-03-31/functions/" + function + "/configuration")+ .then().statusCode(200).body("Version", equalTo("2")).body("Description", equalTo("release 2"));+ given().when().get("/2015-03-31/functions/" + function + "/versions")+ .then().statusCode(200)+ .body("Versions.Version", equalTo(List.of("$LATEST", "1", "2")))+ .body("Versions.find { it.Version == '1' }.Description", equalTo("release 1"))+ .body("Versions.find { it.Version == '$LATEST' }.Description", emptyOrNullString());+ }++ /**+ * Under AutoPublishAliasAllProperties a Tags change names a new id but publishes nothing, since+ * the function has not changed. A new VersionDescription alone keeps the id: the change set+ * reports the version replaced, but floci keeps it with its first description, where AWS fails+ * the update. The next new id, from another Tags change, publishes with the new description,+ * which no longer matches the last version's.+ */+ @Test+ void samVersionDescriptionAloneKeepsThePublishedVersion() {+ String stackName = "sam-version-description-only-stack";+ String function = "sam-version-description-only-func";+ stacksToDelete.add(stackName);+ String template = """+ Transform: AWS::Serverless-2016-10-31+ Resources:+ Function:+ Type: AWS::Serverless::Function+ Properties:+ FunctionName: sam-version-description-only-func+ Handler: index.handler+ Runtime: nodejs20.x+ AutoPublishAlias: live+ AutoPublishAliasAllProperties: true+ VersionDescription: %s+ Tags:+ build: %s+ InlineCode: "exports.handler = async () => ({});"+ """;+ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack").formParam("StackName", stackName)+ .formParam("TemplateBody", template.formatted("release 1", "a"))+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when().post("/").then().statusCode(200);+ waitForStackStatus(stackName, "CREATE_COMPLETE");++ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "UpdateStack").formParam("StackName", stackName)+ .formParam("TemplateBody", template.formatted("release 1", "b"))+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when().post("/").then().statusCode(200);+ waitForStackStatus(stackName, "UPDATE_COMPLETE");+ given().when().get("/2015-03-31/functions/" + function + "/aliases/live")+ .then().statusCode(200).body("FunctionVersion", equalTo("1"));++ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet").formParam("StackName", stackName)+ .formParam("ChangeSetName", "described").formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template.formatted("release 2", "b"))+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when().post("/").then().statusCode(200);+ String changeSet = given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet").formParam("StackName", stackName)+ .formParam("ChangeSetName", "described")+ .when().post("/").then().statusCode(200).extract().asString();+ List<String> versionChanges = XmlParser.extractGroups(changeSet, "ResourceChange").stream()+ .filter(change -> change.get("LogicalResourceId").startsWith("FunctionVersion"))+ .map(change -> change.get("Action") + " " + change.get("Replacement"))+ .toList();+ assertThat(versionChanges, equalTo(List.of("Modify True")));+ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "ExecuteChangeSet").formParam("StackName", stackName)+ .formParam("ChangeSetName", "described")+ .when().post("/").then().statusCode(200);+ waitForStackStatus(stackName, "UPDATE_COMPLETE");+ given().when().get("/2015-03-31/functions/" + function + "/versions")+ .then().statusCode(200)+ .body("Versions.Version", equalTo(List.of("$LATEST", "1")))+ .body("Versions.find { it.Version == '1' }.Description", equalTo("release 1"));++ given().contentType("application/x-www-form-urlencoded")+ .formParam("Action", "UpdateStack").formParam("StackName", stackName)+ .formParam("TemplateBody", template.formatted("release 2", "c"))+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when().post("/").then().statusCode(200);+ waitForStackStatus(stackName, "UPDATE_COMPLETE");+ given().when().get("/2015-03-31/functions/" + function + "/aliases/live")+ .then().statusCode(200).body("FunctionVersion", equalTo("2"));+ given().when().get("/2015-03-31/functions/" + function + "/versions")+ .then().statusCode(200)+ .body("Versions.find { it.Version == '1' }.Description", equalTo("release 1"))+ .body("Versions.find { it.Version == '2' }.Description", equalTo("release 2"));+ }+@Testvoid samFunction_withoutAutoPublishAlias_createsNoAlias() {String stackName = "sam-no-alias-stack";diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessorTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessorTest.javaindex 2998523eb1..a113584bbb 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessorTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/SamTransformProcessorTest.java@@ -578,6 +578,136 @@ class SamTransformProcessorTest {+ "'Function'.", refused.getMessage());}+ /**+ * SAM gives the version the function's VersionDescription as its Description, as written, so+ * CloudFormation resolves an intrinsic function in it. The function does not take it, and+ * without AutoPublishAlias there is no version to take it.+ */+ @Test+ void expandSamTemplate_versionDescriptionDescribesTheVersion() throws Exception {+ String template = """+ {+ "Transform": "AWS::Serverless-2016-10-31",+ "Parameters": {"Release": {"Type": "String", "Default": "1"}},+ "Resources": {+ "MyFunc": {+ "Type": "AWS::Serverless::Function",+ "Properties": {+ "Handler": "index.handler", "Runtime": "nodejs20.x",%s+ "InlineCode": "exports.handler = async () => ({});", "VersionDescription": %s+ }+ }+ }+ }+ """;+ String alias = " \"AutoPublishAlias\": \"live\",";++ JsonNode resources = processor.expandSamTemplate(objectMapper.readTree(+ template.formatted(alias, "\"first release\""))).path("Resources");+ assertEquals("first release",+ resources.path("MyFuncVersion102360dfc3").path("Properties").path("Description").asText());+ assertFalse(resources.path("MyFunc").path("Properties").has("Description"), "the function takes no description");+ assertEquals("{\"Fn::Sub\":\"release ${Release}\"}", versionProperties(template.formatted(alias,+ "{\"Fn::Sub\": \"release ${Release}\"}")).path("Description").toString(),+ "an intrinsic function is copied unresolved, as SAM copies it");+ assertEquals("\"\"", versionProperties(template.formatted(alias, "\"\"")).path("Description").toString(),+ "SAM keeps an empty value");+ assertFalse(versionProperties(template.formatted(alias, "null")).has("Description"),+ "SAM drops a null value");++ JsonNode unpublished = processor.expandSamTemplate(objectMapper.readTree(+ template.formatted("", "\"first release\""))).path("Resources");+ assertTrue(unpublished.has("MyFuncRole"));+ assertEquals(2, unpublished.size(), "only the function and its role: " + unpublished);+ assertFalse(unpublished.path("MyFunc").path("Properties").has("Description"));+ }++ /** The version resource SAM's translator gives this function, id and Description included. */+ @Test+ void expandSamTemplate_versionDescriptionGivesSamsVersionResource() throws Exception {+ JsonNode resources = processor.expandSamTemplate(objectMapper.readTree("""+ {+ "Transform": "AWS::Serverless-2016-10-31",+ "Resources": {+ "MinimalFunction": {+ "Type": "AWS::Serverless::Function",+ "Properties": {+ "CodeUri": "s3://sam-demo-bucket/hello.zip", "Handler": "hello.handler",+ "Runtime": "python2.7", "AutoPublishAlias": "live", "VersionDescription": "sam-testing"+ }+ }+ }+ }+ """)).path("Resources");++ assertEquals(objectMapper.readTree("""+ {"Type": "AWS::Lambda::Version", "DeletionPolicy": "Retain",+ "Properties": {"Description": "sam-testing", "FunctionName": {"Ref": "MinimalFunction"}}}+ """), resources.path("MinimalFunctionVersion640128d35d"), resources.toString());+ }++ /**+ * SAM leaves VersionDescription out of the version's id, by default, under+ * AutoPublishAliasAllProperties and beside a given AutoPublishCodeSha256, so a new one alone+ * names no new version. The version takes it without the AutoPublishCodeSha256.+ */+ @Test+ void expandSamTemplate_versionDescriptionIsNotPartOfTheVersionId() throws Exception {+ String template = """+ {+ "Transform": "AWS::Serverless-2016-10-31",+ "Resources": {+ "MyFunc": {+ "Type": "AWS::Serverless::Function",+ "Properties": {+ "Handler": "index.handler", "Runtime": "nodejs20.x", "AutoPublishAlias": "live",+ "InlineCode": "exports.handler = async () => ({});"%s%s+ }+ }+ }+ }+ """;+ String described = ", \"VersionDescription\": \"first release\"";+ String codeSha256 = ", \"AutoPublishCodeSha256\": \"abc123\"";++ for (String versionProperty : List.of("", ", \"AutoPublishAliasAllProperties\": true", codeSha256)) {+ assertEquals(publishedVersionId(template.formatted(versionProperty, "")),+ publishedVersionId(template.formatted(versionProperty, described)), versionProperty);+ }+ assertEquals("MyFuncVersion102360dfc3", publishedVersionId(template.formatted("", described)));+ assertEquals("first release", processor.expandSamTemplate(objectMapper.readTree(+ template.formatted(codeSha256, described))).path("Resources").path("MyFuncVersionabc123")+ .path("Properties").path("Description").asText());+ }++ /** SAM's Globals do not take VersionDescription either, which would describe every function's version. */+ @Test+ void expandSamTemplate_globalsRefuseVersionDescription() throws Exception {+ JsonNode template = objectMapper.readTree("""+ {+ "Transform": "AWS::Serverless-2016-10-31",+ "Globals": {"Function": {"AutoPublishAlias": "live", "VersionDescription": "every release"}},+ "Resources": {+ "MyFunc": {+ "Type": "AWS::Serverless::Function",+ "Properties": {"Handler": "index.handler", "Runtime": "nodejs20.x",+ "InlineCode": "exports.handler = async () => ({});"}+ }+ }+ }+ """);++ AwsException refused = assertThrows(AwsException.class, () -> processor.expandSamTemplate(template));+ assertEquals("ValidationError", refused.getErrorCode());+ assertEquals("'Globals' section is invalid. 'VersionDescription' is not a supported property of "+ + "'Function'.", refused.getMessage());+ }++ private JsonNode versionProperties(String template) throws Exception {+ return processor.expandSamTemplate(objectMapper.readTree(template)).path("Resources")+ .path("MyFuncVersion102360dfc3").path("Properties");+ }+/*** SAM retains each version it publishes unless the function sets a {@code DeletionPolicy}, which* the version and alias both take, or a {@code VersionDeletionPolicy}; a function on a capacity