Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

fix(cloudformation): check S3 ownership rules against the resource schema (#5834)

Jackson Price9099cec5e815
diff --git a/docs/services/cloudformation.md b/docs/services/cloudformation.md
index 116c3693db..f907372ed2 100644
--- a/docs/services/cloudformation.md
+++ b/docs/services/cloudformation.md
@@ -505,6 +505,41 @@ Dynamic-reference expansion is currently scoped to these RDS credential properti
- On `UpdateStack` (no interruption, as in AWS), a cluster's list is checked first, then the roles the template gained are added, and the roles the previous template declared and the new one drops are removed. A role associated outside the stack is kept while the template leaves it out; declaring it fails the update with `DBClusterRoleAlreadyExists`, as in AWS. If any step of an update is refused, the roles it already added are removed and the ones it removed are put back, so the database matches what the stack still records and a retry starts from there. A rolled-back cluster replacement restores the record of which roles the stack declared.
- An Aurora cluster member (`AWS::RDS::DBInstance` with `DBClusterIdentifier`) cannot carry `AssociatedRoles`: the schema documents the property as not applicable there, since the cluster manages its roles. The resource fails with a `ValidationError` pointing at the cluster's `AssociatedRoles`.
+## S3 ownership controls
+
+`AWS::S3::Bucket` applies `OwnershipControls` through the same S3 call as
+`PutBucketOwnershipControls`, on create and in place on update.
+
+- A malformed rule fails the resource before the bucket is created or changed, and the stack
+ rolls back. The resource's status reason gives the first fault found: `OwnershipControls` is
+ checked first, then each rule in turn, then whether a rule repeats another, then S3's own check.
+- A fault the resource schema defines is reported as
+ `Model validation failed (#/OwnershipControls...: <reason>)`: a value of the wrong type
+ (`OwnershipControls` or a rule that is not an object, `Rules` that is not a list, an
+ `ObjectOwnership` that is a list or an object), a key other than `Rules` on `OwnershipControls`
+ or other than `ObjectOwnership` on a rule (`extraneous key [<key>] is not permitted`), or a
+ rule listed twice (`array items are not unique`), comparing the rules after intrinsic
+ functions such as `Ref` are resolved.
+- A rule without `ObjectOwnership`, which S3 requires, fails among that rule's own checks in the
+ same form, with `required key [ObjectOwnership] not found`. The schema accepts such a rule, so
+ CloudFormation's reason for a template with one can differ.
+- What S3 refuses then fails with the message of the `MalformedXML` error that
+ `PutBucketOwnershipControls` returns, `The XML you provided was not well-formed or did not
+ validate against our published schema.`: no rule, more than one rule, or a mode other than
+ `BucketOwnerEnforced`, `BucketOwnerPreferred` or `ObjectWriter`. AWS's published resource
+ schema refuses two of these itself, a missing `Rules` (`required key [Rules] not found`) and a
+ mode outside the three (its `enum`), so CloudFormation's reason for them may differ.
+- An update that removes `OwnershipControls` from a template that declared it deletes the
+ controls, as a removed `NotificationConfiguration` is cleared; real AWS's answer for that
+ removal is unmeasured. Controls set outside the stack are left alone while neither the previous
+ nor the new template declares the property.
+- A failed update, including one where a later resource fails, puts back the controls the bucket
+ had before the update, with the rest of its configuration (see
+ [Failed update rollbacks](#failed-update-rollbacks)).
+- Floci's S3 gives a new bucket no ownership controls, so a bucket that never declares the
+ property answers `GetBucketOwnershipControls` with `OwnershipControlsNotFoundError`, where AWS
+ reports its default, `BucketOwnerEnforced`.
+
## Account-Aware Provisioning
Resources provisioned by `CreateStack` / `UpdateStack` land in the **caller's account** namespace
diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisioner.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisioner.java
index f04fc5b616..3d85d68075 100644
--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisioner.java
+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisioner.java
@@ -19,6 +19,8 @@ import jakarta.enterprise.context.ApplicationScoped;
import org.jboss.logging.Logger;
import java.util.ArrayList;
+import java.util.HashSet;
+import java.util.Iterator;
import java.util.List;
import java.util.Map;
import java.util.Set;
@@ -246,6 +248,14 @@ public class S3CfnProvisioner implements CfnResourceProvisioner {
s3Service.putBucketLifecycle(bucketName, lifecycleXml(lifecycle.get("Rules"), ctx), null);
}
+ /**
+ * The document the template declares, or null when it declares none. A key the registry schema
+ * does not define, a value of the wrong type, a rule without {@code ObjectOwnership} or a rule
+ * listed twice fails in the class's {@code Model validation failed} form. The repeat is checked
+ * once every rule passes on its own, so a malformed rule fails for its own fault even when
+ * repeated. The document then goes through S3's own check, which answers {@code MalformedXML}
+ * for no rule, more than one, or an unknown mode.
+ */
private String ownershipControlsXml(JsonNode props, ProvisionContext ctx) {
JsonNode declaredNode = declared(props, "OwnershipControls");
if (declaredNode == null) {
@@ -256,17 +266,18 @@ public class S3CfnProvisioner implements CfnResourceProvisioner {
return null;
}
String path = "#/OwnershipControls";
- requireObject(controls, path);
+ requireOnly(requireObject(controls, path), path, "Rules");
JsonNode rules = declaredArray(controls, "Rules", path);
XmlBuilder xml = new XmlBuilder().start("OwnershipControls", AwsNamespaces.S3);
for (int index = 0; rules != null && index < rules.size(); index++) {
String rulePath = path + "/Rules/" + index;
- JsonNode rule = requireObject(rules.get(index), rulePath);
+ JsonNode rule = requireOnly(requireObject(rules.get(index), rulePath), rulePath, "ObjectOwnership");
String ownership = required(rule, "ObjectOwnership", rulePath);
xml.start("Rule");
xml.elem("ObjectOwnership", ownership);
xml.end("Rule");
}
+ requireDistinct(rules, path + "/Rules");
String document = xml.end("OwnershipControls").build();
S3Service.validateOwnershipControls(document);
return document;
@@ -474,6 +485,36 @@ public class S3CfnProvisioner implements CfnResourceProvisioner {
return value;
}
+ /**
+ * The object itself when it holds no key besides {@code keys}. Any other key fails, as the
+ * schema's {@code additionalProperties: false} does, rather than being dropped unread.
+ */
+ private static JsonNode requireOnly(JsonNode object, String pointer, String... keys) {
+ Set<String> allowed = Set.of(keys);
+ for (Iterator<String> names = object.fieldNames(); names.hasNext(); ) {
+ String name = names.next();
+ if (!allowed.contains(name)) {
+ throw new AwsException("InvalidRequest", "Model validation failed (" + pointer
+ + ": extraneous key [" + name + "] is not permitted)", 400);
+ }
+ }
+ return object;
+ }
+
+ /**
+ * Fails when an item of the list repeats another, as the schema's {@code uniqueItems} does. A
+ * list that is not declared, null here, has no repeat.
+ */
+ private static void requireDistinct(JsonNode items, String pointer) {
+ Set<JsonNode> seen = new HashSet<>();
+ for (int index = 0; items != null && index < items.size(); index++) {
+ if (!seen.add(items.get(index))) {
+ throw new AwsException("InvalidRequest",
+ "Model validation failed (" + pointer + ": array items are not unique)", 400);
+ }
+ }
+ }
+
private static AwsException typeMismatch(String pointer, String expected, JsonNode value) {
return new AwsException("InvalidRequest", "Model validation failed (" + pointer
+ ": expected type: " + expected + ", found: " + schemaTypeName(value) + ")", 400);
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationS3BucketPropertiesIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationS3BucketPropertiesIntegrationTest.java
index caee36dac7..2de78f7b4b 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationS3BucketPropertiesIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationS3BucketPropertiesIntegrationTest.java
@@ -155,6 +155,55 @@ class CloudFormationS3BucketPropertiesIntegrationTest {
+ marker + "\" }]" + controls + " } } } }";
}
+ /** A key the registry schema does not define fails the create, and the stack leaves no bucket. */
+ @Test
+ void anOwnershipRuleWithAKeyTheSchemaDoesNotDefineFailsTheCreate() throws InterruptedException {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "cfn-ownership-extra-" + suffix;
+ String stackId = stack("CreateStack", "cfn-ownership-extra-stack-" + suffix, """
+ { "Resources": { "Bucket": { "Type": "AWS::S3::Bucket", "Properties": {
+ "BucketName": "%s",
+ "OwnershipControls": { "Rules": [
+ { "ObjectOwnership": "BucketOwnerEnforced", "Owner": "me" } ] } } } } }
+ """.formatted(bucket));
+ await(stackId, "ROLLBACK_COMPLETE");
+
+ String events = stackEvents(stackId);
+ assertTrue(events.contains(
+ "Model validation failed (#/OwnershipControls/Rules/0: extraneous key [Owner] is not permitted)"),
+ "the create failed on the rule: " + events);
+ given().when().head("/" + bucket).then().statusCode(404);
+ }
+
+ /**
+ * The template is resolved before the schema is checked, so two rules that resolve to the same
+ * mode are a repeated rule. The update rolls back and the bucket keeps its tag and mode.
+ */
+ @Test
+ void ownershipRulesThatResolveToTheSameModeFailTheUpdate() throws InterruptedException {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "cfn-ownership-repeated-" + suffix;
+ String stackName = "cfn-ownership-repeated-stack-" + suffix;
+ String stackId = stack("CreateStack", stackName, ownershipTemplate(bucket, "BucketOwnerEnforced", "before"));
+ await(stackId, "CREATE_COMPLETE");
+
+ stack("UpdateStack", stackName, """
+ { "Parameters": { "Mode": { "Type": "String", "Default": "ObjectWriter" } },
+ "Resources": { "Bucket": { "Type": "AWS::S3::Bucket", "Properties": {
+ "BucketName": "%s", "Tags": [{ "Key": "marker", "Value": "after" }],
+ "OwnershipControls": { "Rules": [
+ { "ObjectOwnership": { "Ref": "Mode" } }, { "ObjectOwnership": "ObjectWriter" } ] } } } } }
+ """.formatted(bucket));
+ await(stackId, "UPDATE_ROLLBACK_COMPLETE");
+
+ String events = stackEvents(stackId);
+ assertTrue(events.contains("Model validation failed (#/OwnershipControls/Rules: array items are not unique)"),
+ "the update failed on the repeated rule: " + events);
+ bucketGet(bucket, "ownershipControls")
+ .body(containsString("<ObjectOwnership>BucketOwnerEnforced</ObjectOwnership>"));
+ bucketGet(bucket, "tagging").body(containsString("<Value>before</Value>"));
+ }
+
@Test
void theBucketCarriesWhatItsTemplateDeclares() throws InterruptedException {
String suffix = Long.toString(System.nanoTime(), 36);
@@ -280,6 +329,14 @@ class CloudFormationS3BucketPropertiesIntegrationTest {
return given().when().get("/" + bucket + "?" + subresource).then().statusCode(200);
}
+ private static String stackEvents(String stackId) {
+ return given()
+ .contentType("application/x-www-form-urlencoded")
+ .formParam("Action", "DescribeStackEvents")
+ .formParam("StackName", stackId)
+ .when().post("/").then().statusCode(200).extract().asString();
+ }
+
private static String stack(String action, String stackName, String template) {
String xml = given()
.contentType("application/x-www-form-urlencoded")
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisionerTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisionerTest.java
index c361b56f5e..fbc33cf7d7 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisionerTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/S3CfnProvisionerTest.java
@@ -115,6 +115,76 @@ class S3CfnProvisionerTest {
verify(s3, never()).putBucketOwnershipControls(anyString(), anyString());
}
+ /**
+ * The registry schema allows only {@code Rules} on the controls and only {@code ObjectOwnership}
+ * on a rule, no rule listed twice, and an object, a list or a string at each level. An unknown
+ * key is reported ahead of a wrong type beside it, and the schema is checked before S3's own
+ * rules, so a repeated rule is refused for the repetition rather than for the rule count.
+ */
+ @ParameterizedTest
+ @CsvSource(delimiter = '|', textBlock = """
+ {"Rules": [{"ObjectOwnership": "ObjectWriter"}], "Rule": []} | #/OwnershipControls: extraneous key [Rule] is not permitted
+ {"Rule": [{"ObjectOwnership": "ObjectWriter"}]} | #/OwnershipControls: extraneous key [Rule] is not permitted
+ {"Rules": "ObjectWriter", "Extra": 1} | #/OwnershipControls: extraneous key [Extra] is not permitted
+ {"Rules": [{"ObjectOwnership": "ObjectWriter"}], "ObjectOwnership": "ObjectWriter"} | #/OwnershipControls: extraneous key [ObjectOwnership] is not permitted
+ {"Rules": [{"ObjectOwnership": "ObjectWriter", "Owner": "me"}]} | #/OwnershipControls/Rules/0: extraneous key [Owner] is not permitted
+ {"Rules": [{"Objectownership": "ObjectWriter"}]} | #/OwnershipControls/Rules/0: extraneous key [Objectownership] is not permitted
+ {"Rules": [{"Rules": []}]} | #/OwnershipControls/Rules/0: extraneous key [Rules] is not permitted
+ {"Rules": [{"ObjectOwnership": "ObjectWriter"}, {"ObjectOwnership": "ObjectWriter"}]} | #/OwnershipControls/Rules: array items are not unique
+ {"Rules": [{"ObjectOwnership": "ObjectWriter"}, {"ObjectOwnership": "BucketOwnerPreferred"}, {"ObjectOwnership": "ObjectWriter"}]} | #/OwnershipControls/Rules: array items are not unique
+ "BucketOwnerEnforced" | #/OwnershipControls: expected type: JSONObject, found: String
+ [{"Rules": [{"ObjectOwnership": "ObjectWriter"}]}] | #/OwnershipControls: expected type: JSONObject, found: JSONArray
+ {"Rules": {"ObjectOwnership": "ObjectWriter"}} | #/OwnershipControls/Rules: expected type: JSONArray, found: JSONObject
+ {"Rules": ["ObjectWriter"]} | #/OwnershipControls/Rules/0: expected type: JSONObject, found: String
+ {"Rules": [{"ObjectOwnership": {"Mode": "ObjectWriter"}}]} | #/OwnershipControls/Rules/0/ObjectOwnership: expected type: String, found: JSONObject
+ """)
+ void ownershipControlsTheSchemaRefusesFailBeforeCreatingTheBucket(String controls, String reason) {
+ assertOwnershipControlsFailModelValidation(controls, reason);
+ }
+
+ /**
+ * Every rule is checked on its own before any repeat, so a malformed rule fails for its own
+ * fault, as a single one does, whether it is the repeated rule or comes after a repeat.
+ */
+ @ParameterizedTest
+ @CsvSource(delimiter = '|', textBlock = """
+ {"Rules": [{}, {}]} | #/OwnershipControls/Rules/0: required key [ObjectOwnership] not found
+ {"Rules": ["ObjectWriter", "ObjectWriter"]} | #/OwnershipControls/Rules/0: expected type: JSONObject, found: String
+ {"Rules": [{"ObjectOwnership": "ObjectWriter"}, {"ObjectOwnership": "ObjectWriter"}, {}]} | #/OwnershipControls/Rules/2: required key [ObjectOwnership] not found
+ """)
+ void aRepeatedRuleIsCheckedOnItsOwnFirst(String controls, String reason) {
+ assertOwnershipControlsFailModelValidation(controls, reason);
+ }
+
+ private void assertOwnershipControlsFailModelValidation(String controls, String reason) {
+ AwsException e = assertThrows(AwsException.class, () -> provisionBucket("""
+ {"BucketName": "my-bucket", "OwnershipControls": %s}
+ """.formatted(controls)));
+
+ assertEquals("InvalidRequest", e.getErrorCode());
+ assertEquals("Model validation failed (" + reason + ")", e.getMessage());
+ verify(s3, never()).createBucket(anyString(), anyString());
+ verify(s3, never()).putBucketOwnershipControls(anyString(), anyString());
+ }
+
+ @Test
+ void anOwnershipRuleTheSchemaRefusesFailsAnUpdateBeforeTheBucketChanges() {
+ AwsException e = assertThrows(AwsException.class, () -> updateBucket("""
+ {"BucketName": "b", "OwnershipControls": {"Rules": [{"ObjectOwnership": "ObjectWriter"}]}}
+ """, """
+ {"BucketName": "b", "Tags": [{"Key": "k", "Value": "v"}],
+ "OwnershipControls": {"Rules": [{"ObjectOwnership": "BucketOwnerEnforced", "Owner": "me"}]}}
+ """));
+
+ assertEquals("InvalidRequest", e.getErrorCode());
+ assertEquals("Model validation failed (#/OwnershipControls/Rules/0: extraneous key [Owner] is not permitted)",
+ e.getMessage());
+ verify(s3, never()).deleteBucketCors(anyString());
+ verify(s3, never()).putBucketTagging(anyString(), any());
+ verify(s3, never()).putBucketOwnershipControls(anyString(), anyString());
+ verify(s3, never()).deleteBucketOwnershipControls(anyString());
+ }
+
@Test
void refIsTheBucketNameAndGetAttExposesTheDocumentedAttributes() {
StackResource r = provisionBucket("""