Light, fluffy, and always free - The AWS Local Emulator alternative
feat(iam): read a Query list's constraints from the model (#6625)
Priamo Germosen0ec28aee68e9
diff --git a/docs/services/iam.md b/docs/services/iam.mdindex a2f05eeeb5..fe06cfd7f6 100644--- a/docs/services/iam.md+++ b/docs/services/iam.md@@ -171,6 +171,27 @@ hand-written check the rest of the service still uses stops at the first and repalone. A boolean member publishes no constraint to hold it to, so a `SetAsDefault` that is neither`true` nor `false` reaches the operation, which reads anything but `true` as false.+A list is read the same way, from the parameters the protocol numbers under its name:+`Audience.member.1` or the bare `Audience.1`, with a member of a list of structures carrying its+fields under that prefix as `Tags.member.1.Key` does. Each is named in a message the way this+service already names one, `filter.1.member` and `tags.1.member.key`, under the index the request+spelled rather than the position the member sits at, and the count is held to the list's own+bounds with the value left out of the message. A member outside its enum is reported under its own+index for the same reason, which is a reading: two other services here answer one against the+whole list instead, and nothing recorded says which form AWS answers on this protocol. A list of+no members is how an absent list arrives, so a required one reads as null rather than as a list+breaking a minimum of 1. A structure member arrives as `TaskPolicyArn.Arn` and is named+`taskPolicyArn.arn`. A map is not read yet, nor is anything nested deeper than a list of+structures. The request form is published, `AcquireRole`'s own sample numbering a map as+`ReplacementValues.entry.1.key` and `.entry.1.value`, but the message this service answers for one+names the whole map rather than the entry that broke a constraint, and nothing here says which AWS+names, so reading a map would move a merged answer on a guess. `GetAccountAuthorizationDetails` is+the first operation read through the walk, and nothing of it was read before: a `Filter` member+outside the model's enum, a `MaxItems` outside 1 to 1000 and a `Marker` breaking its length or its+pattern were all answered with a 200. A number too large for a 64-bit integer is a number all the+same, so a `MaxItems` of `9223372036854775808` breaks the maximum rather than reading as no number+at all.+### Permission Boundaries| Action | Description |diff --git a/src/main/java/io/github/hectorvent/floci/core/common/AwsQueryLists.java b/src/main/java/io/github/hectorvent/floci/core/common/AwsQueryLists.javaindex 287240144f..04105569fc 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/AwsQueryLists.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/AwsQueryLists.java@@ -4,6 +4,7 @@ import jakarta.ws.rs.core.MultivaluedMap;import java.util.Comparator;import java.util.List;+import java.util.regex.Matcher;import java.util.regex.Pattern;/**@@ -38,6 +39,38 @@ public final class AwsQueryLists {.toList();}+ /**+ * One member of a Query list: the prefix its own parameters are named under, and the index the+ * request numbered it with, read back as it was spelled rather than as the number it reads as.+ */+ public record IndexedMember(String prefix, String index) {+ }++ /**+ * The members of {@code list}, in the order their indexes read, each with the prefix its own+ * parameters are named under. Unlike {@link #values}, a member of a list of structures counts:+ * its fields arrive under the prefix rather than at it, as {@code Tags.member.1.Key} does, and+ * a member present with an empty value is still a member.+ *+ * <p>The index is kept as the request spelled it, {@code 007} and all, and a segment that is no+ * number is not one: with a null {@code member}, {@code SubnetIds.SubnetIdentifier.4} is a list+ * naming its own member rather than a member numbered {@code SubnetIdentifier}, and this reads+ * the generic and bare forms only, as {@link #memberKeys} does.+ */+ public static List<IndexedMember> members(MultivaluedMap<String, String> params, String list,+ String member) {+ String names = member == null ? "" : "|\\." + Pattern.quote(member);+ Pattern keys = Pattern.compile("(" + Pattern.quote(list) + "(?:\\.member" + names+ + ")?\\.(\\d+))(?:\\..*)?");+ return params.keySet().stream()+ .map(keys::matcher)+ .filter(Matcher::matches)+ .map(matcher -> new IndexedMember(matcher.group(1), matcher.group(2)))+ .distinct()+ .sorted(Comparator.comparingInt(one -> memberIndex(one.prefix())))+ .toList();+ }+/** The number a member's key ends in, or {@link Integer#MAX_VALUE} for a key that ends in none. */public static int memberIndex(String key) {int lastDot = key.lastIndexOf('.');diff --git a/src/main/java/io/github/hectorvent/floci/core/common/QueryConstraintCheck.java b/src/main/java/io/github/hectorvent/floci/core/common/QueryConstraintCheck.javaindex d9837dac17..1ed3855737 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/QueryConstraintCheck.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/QueryConstraintCheck.java@@ -4,6 +4,7 @@ import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;import jakarta.ws.rs.core.MultivaluedMap;+import java.math.BigInteger;import java.util.ArrayList;import java.util.HashMap;import java.util.LinkedHashSet;@@ -35,10 +36,19 @@ import java.util.regex.Pattern;* member present and empty is a value: the protocol carries it, so it breaks its minimum length* rather than reading as absent.*- * <p>Scalars only for now. An operation whose input shape carries a list or a map is refused when- * the file loads rather than passed over quietly, since reading one means reading- * {@code Name.member.N} and {@code Name.entry.N.key} and naming each index as the request spelled- * it.+ * <p>A list arrives numbered under its own name, {@code Audience.member.1} or the bare+ * {@code Audience.1}, and a member of a list of structures carries its fields under that prefix,+ * {@code Tags.member.1.Key}. Each is named in a message the way this package already names it:+ * {@code audience.1.member} and {@code tags.1.member.key}, with the index the request spelled+ * rather than the position the member sits at. A list of no members is how an absent list arrives+ * on this protocol, so a required one reads as null rather than as a list breaking its minimum, and+ * the count is held to the list's own bounds with the value left out of the message. A structure+ * member arrives as {@code TaskPolicyArn.Arn} and is named {@code taskPolicyArn.arn}.+ *+ * <p>A map is refused when the file loads rather than passed over quietly: this package answers one+ * today in a form of its own, {@code Map keys must satisfy constraint: [...]} against the whole map,+ * and which of the two forms AWS answers for a Query map is a question no source here settles. So is+ * anything nested deeper than a list of structures or a structure of scalars.*/public final class QueryConstraintCheck {@@ -47,6 +57,9 @@ public final class QueryConstraintCheck {/** The shape types a Query parameter can carry on its own, without an index or an entry. */private static final Set<String> SCALAR_TYPES = Set.of("string", "integer", "long", "boolean");+ /** A number as the protocol writes one, of any width: the model's bounds are what hold it. */+ private static final Pattern INTEGER = Pattern.compile("[+-]?\\d+");+private final JsonNode operations;private final JsonNode shapes;private final Map<String, Pattern> patterns = new HashMap<>();@@ -60,7 +73,7 @@ public final class QueryConstraintCheck {}}for (Map.Entry<String, JsonNode> operation : operations.properties()) {- requireScalarMembers(operation.getKey(), operation.getValue().asText());+ requireReadableMembers(operation.getKey(), operation.getValue().asText());}}@@ -74,16 +87,50 @@ public final class QueryConstraintCheck {return new QueryConstraintCheck(model);}- private void requireScalarMembers(String operation, String inputShape) {+ private void requireReadableMembers(String operation, String inputShape) {for (Map.Entry<String, JsonNode> member : shapes.path(inputShape).path("members").properties()) {- String type = shapes.path(member.getValue().asText()).path("type").asText();+ requireReadableMember(operation, member.getKey(), member.getValue().asText());+ }+ }++ private void requireReadableMember(String operation, String member, String shapeName) {+ String type = shapes.path(shapeName).path("type").asText();+ if (SCALAR_TYPES.contains(type)) {+ return;+ }+ switch (type) {+ case "list" -> requireReadableElement(operation, member, shapes.path(shapeName).path("member").asText());+ case "structure" -> requireScalarFields(operation, member, shapeName);+ default -> throw refusal(operation, member, type);+ }+ }++ private void requireReadableElement(String operation, String member, String elementShape) {+ String type = shapes.path(elementShape).path("type").asText();+ if (SCALAR_TYPES.contains(type)) {+ return;+ }+ if ("structure".equals(type)) {+ requireScalarFields(operation, member, elementShape);+ return;+ }+ throw refusal(operation, member, "list of " + type);+ }++ private void requireScalarFields(String operation, String member, String structure) {+ for (Map.Entry<String, JsonNode> field : shapes.path(structure).path("members").properties()) {+ String type = shapes.path(field.getValue().asText()).path("type").asText();if (!SCALAR_TYPES.contains(type)) {- throw new IllegalStateException(operation + "'s " + member.getKey() + " is a " + type- + ", which this check does not read yet");+ throw refusal(operation, member + "." + field.getKey(), type);}}}+ private static IllegalStateException refusal(String operation, String member, String type) {+ return new IllegalStateException(operation + "'s " + member + " is a " + type+ + ", which this check does not read yet");+ }+/*** Throws the Query protocol's {@code ValidationError} when {@code params} breaks a constraint* the model declares for {@code operation}. An operation the vendored file does not carry@@ -102,24 +149,90 @@ public final class QueryConstraintCheck {if (!shape.hasNonNull("members")) {return List.of();}+ return structureViolations(shape, "", "", params);+ }++ /**+ * The violations the members of {@code structure} carry, in the order the model lists them.+ * {@code prefix} is what the request names them under and {@code path} what a message calls+ * them: both are empty at the top level, and name the member above inside a structure or inside+ * a list's member.+ */+ private List<String> structureViolations(JsonNode structure, String prefix, String path,+ MultivaluedMap<String, String> params) {Set<String> required = new LinkedHashSet<>();- shape.path("required").forEach(name -> required.add(name.asText()));+ structure.path("required").forEach(name -> required.add(name.asText()));List<String> violations = new ArrayList<>();- for (Map.Entry<String, JsonNode> member : shape.path("members").properties()) {+ for (Map.Entry<String, JsonNode> member : structure.path("members").properties()) {String name = member.getKey();- String value = params == null ? null : params.getFirst(name);- String path = memberPath(name);- if (value == null) {- if (required.contains(name)) {- violations.add(SmithyValidationErrors.notNull(path));+ String shapeName = member.getValue().asText();+ String memberPath = path + memberPath(name);+ String param = prefix + name;+ boolean mandatory = required.contains(name);+ JsonNode shape = shapes.path(shapeName);+ switch (shape.path("type").asText()) {+ case "list" -> violations.addAll(listViolations(shape, param, memberPath, mandatory, params));+ case "structure" -> violations.addAll(nestedViolations(shape, param, memberPath, mandatory, params));+ default -> {+ String value = params == null ? null : params.getFirst(param);+ if (value == null) {+ if (mandatory) {+ violations.add(SmithyValidationErrors.notNull(memberPath));+ }+ } else {+ violations.addAll(violations(shapeName, value, memberPath));+ }}- continue;}- violations.addAll(violations(member.getValue().asText(), value, path));}return violations;}+ /**+ * A list's members, held to the list's own bounds and then each to its element's. A list of no+ * members is how an absent list arrives, which is what makes a minimum of 1 unreachable: only a+ * list asking for two or more members can report one.+ */+ private List<String> listViolations(JsonNode shape, String param, String path, boolean required,+ MultivaluedMap<String, String> params) {+ List<AwsQueryLists.IndexedMember> members = params == null+ ? List.of()+ : AwsQueryLists.members(params, param, null);+ if (members.isEmpty()) {+ return required ? List.of(SmithyValidationErrors.notNull(path)) : List.of();+ }+ List<String> violations = new ArrayList<>();+ violations.addAll(single(SmithyValidationErrors.sizeOutOfRange(path, members.size(),+ shape.path("min").asInt(0), shape.path("max").asInt(Integer.MAX_VALUE))));+ String elementShape = shape.path("member").asText();+ boolean structure = "structure".equals(shapes.path(elementShape).path("type").asText());+ for (AwsQueryLists.IndexedMember member : members) {+ String memberPath = path + "." + member.index() + ".member";+ if (structure) {+ violations.addAll(structureViolations(shapes.path(elementShape),+ member.prefix() + ".", memberPath + ".", params));+ } else {+ String value = params.getFirst(member.prefix());+ violations.addAll(violations(elementShape, value == null ? "" : value, memberPath));+ }+ }+ return violations;+ }++ /**+ * A structure member, whose fields arrive under its own name. A request naming none of them is+ * a request without the member at all, so a required one reads as null.+ */+ private List<String> nestedViolations(JsonNode shape, String param, String path, boolean required,+ MultivaluedMap<String, String> params) {+ boolean present = params != null+ && params.keySet().stream().anyMatch(key -> key.startsWith(param + "."));+ if (!present) {+ return required ? List.of(SmithyValidationErrors.notNull(path)) : List.of();+ }+ return structureViolations(shape, param + ".", path + ".", params);+ }+/** The member's name as a validation message spells it, which is the request's with a lower* first letter. */private static String memberPath(String name) {@@ -158,17 +271,28 @@ public final class QueryConstraintCheck {* A value that is no number is left to the operation: this package answers one with* {@code InvalidParameterValue} rather than with a constraint violation, and which member* reports it that way is the operation's own reading.+ *+ * <p>A number too large for a {@code long} is a number all the same, and the bound it breaks is+ * one that fits in one, so the range is compared without a width of its own. The value is+ * written as the number reads rather than as the request spelled it, a leading zero or plus+ * gone, which is what a parsed value already answered.*/private static String numberViolation(JsonNode shape, String value, String path) {- long number;- try {- number = Long.parseLong(value.trim());- } catch (NumberFormatException expected) {+ String text = value.trim();+ if (!INTEGER.matcher(text).matches()) {// Not a number at all, so no range of the model's is what it breaks.return null;}- return SmithyValidationErrors.outOfRange(path, number,- shape.path("min").asLong(Long.MIN_VALUE), shape.path("max").asLong(Long.MAX_VALUE));+ BigInteger number = new BigInteger(text);+ long min = shape.path("min").asLong(Long.MIN_VALUE);+ long max = shape.path("max").asLong(Long.MAX_VALUE);+ if (number.compareTo(BigInteger.valueOf(min)) < 0) {+ return SmithyValidationErrors.valueTooSmall(path, number.toString(), min);+ }+ if (number.compareTo(BigInteger.valueOf(max)) > 0) {+ return SmithyValidationErrors.valueTooLarge(path, number.toString(), max);+ }+ return null;}private static String enumViolation(JsonNode shape, String value, String path) {diff --git a/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java b/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.javaindex c9fbdb9a65..de34c23514 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java@@ -75,16 +75,29 @@ public final class SmithyValidationErrors {/** The violation for a number outside {@code min} to {@code max}, or {@code null} for one inside. */public static String outOfRange(String path, long value, long min, long max) {if (value < min) {- return "Value '" + value + "' at '" + path + "' failed to satisfy constraint: "- + "Member must have value greater than or equal to " + min;+ return valueTooSmall(path, Long.toString(value), min);}if (value > max) {- return "Value '" + value + "' at '" + path + "' failed to satisfy constraint: "- + "Member must have value less than or equal to " + max;+ return valueTooLarge(path, Long.toString(value), max);}return null;}+ /**+ * The violation for a number below {@code min}, with the value already written: a number too+ * large for a {@code long} is still a number, and the bound it breaks is one that fits in one.+ */+ public static String valueTooSmall(String path, String value, long min) {+ return "Value '" + value + "' at '" + path + "' failed to satisfy constraint: "+ + "Member must have value greater than or equal to " + min;+ }++ /** The violation for a number above {@code max}, with the value already written. */+ public static String valueTooLarge(String path, String value, long max) {+ return "Value '" + value + "' at '" + path + "' failed to satisfy constraint: "+ + "Member must have value less than or equal to " + max;+ }+/** The violation for a string outside its length bounds, or {@code null} for one inside. */public static String lengthOutOfRange(String path, String value, int min, int max) {int length = value.codePointCount(0, value.length());@@ -98,6 +111,24 @@ public final class SmithyValidationErrors {return null;}+ /**+ * The violation for a list holding a member count outside its bounds, or {@code null} for one+ * inside. The offending value is left out, which is the form this package already answers for a+ * tag list and for an audience: quoting it would echo every member of the list back in the+ * message.+ */+ public static String sizeOutOfRange(String path, int size, int min, int max) {+ if (size < min) {+ return "Value at '" + path + "' failed to satisfy constraint: "+ + "Member must have length greater than or equal to " + min;+ }+ if (size > max) {+ return "Value at '" + path + "' failed to satisfy constraint: "+ + "Member must have length less than or equal to " + max;+ }+ return null;+ }+/** Also used by services that reject whitespace-only input as an empty member. */public static String lengthTooShort(String path, String value, int min) {return "Value '" + value + "' at '" + path + "' failed to satisfy constraint: "diff --git a/src/main/java/io/github/hectorvent/floci/services/iam/IamRequestShapes.java b/src/main/java/io/github/hectorvent/floci/services/iam/IamRequestShapes.javaindex 5a80c17ff5..e2b0a2693f 100644--- a/src/main/java/io/github/hectorvent/floci/services/iam/IamRequestShapes.java+++ b/src/main/java/io/github/hectorvent/floci/services/iam/IamRequestShapes.java@@ -11,11 +11,13 @@ import jakarta.ws.rs.core.MultivaluedMap;* ValidationError} whether or not the thing it names exists, the ordering #6402 settled for this* package.*- * <p>The vendored file carries one wave of operations rather than all 180, and grows a wave at a- * time: the managed-policy family first, where no operation checks a declared constraint by hand,- * so the model adds the refusals AWS answers without rewording an answer already given.- * {@code AttachRolePolicy} and {@code DetachRolePolicy} are the exception and the parity case,- * {@code IamService.validateRoleName} already answering {@code roleName} in the same form.+ * <p>The vendored file carries a wave of operations rather than all 180, and grows a wave at a+ * time, each one chosen where no operation checks a declared constraint by hand, so the model adds+ * the refusals AWS answers without rewording an answer already given. The managed-policy family+ * came first, with {@code AttachRolePolicy} and {@code DetachRolePolicy} as the exception and the+ * parity case, {@code IamService.validateRoleName} already answering {@code roleName} in the same+ * form. {@code GetAccountAuthorizationDetails} is the first read through the list walk, its+ * {@code Filter}, {@code MaxItems} and {@code Marker} having been read by nothing at all.*/final class IamRequestShapes {diff --git a/src/main/resources/aws/iam-request-shapes.json b/src/main/resources/aws/iam-request-shapes.jsonindex 9bccce9f5f..0885222f05 100644--- a/src/main/resources/aws/iam-request-shapes.json+++ b/src/main/resources/aws/iam-request-shapes.json@@ -15,7 +15,8 @@"CreatePolicyVersion": "CreatePolicyVersionRequest","GetPolicyVersion": "GetPolicyVersionRequest","DeletePolicyVersion": "DeletePolicyVersionRequest",- "SetDefaultPolicyVersion": "SetDefaultPolicyVersionRequest"+ "SetDefaultPolicyVersion": "SetDefaultPolicyVersionRequest",+ "GetAccountAuthorizationDetails": "GetAccountAuthorizationDetailsRequest"},"shapes": {"AttachGroupPolicyRequest": {@@ -116,6 +117,24 @@"PolicyArn": "arnType"}},+ "EntityType": {+ "type": "string",+ "enum": [+ "User",+ "Role",+ "Group",+ "LocalManagedPolicy",+ "AWSManagedPolicy"+ ]+ },+ "GetAccountAuthorizationDetailsRequest": {+ "type": "structure",+ "members": {+ "Filter": "entityListType",+ "MaxItems": "maxItemsType",+ "Marker": "markerType"+ }+ },"GetPolicyRequest": {"type": "structure","required": [@@ -155,12 +174,27 @@"booleanType": {"type": "boolean"},+ "entityListType": {+ "type": "list",+ "member": "EntityType"+ },"groupNameType": {"type": "string","min": 1,"max": 128,"pattern": "[\\w+=,.@-]+"},+ "markerType": {+ "type": "string",+ "min": 1,+ "max": 320,+ "pattern": "[\\u0020-\\u00FF]+"+ },+ "maxItemsType": {+ "type": "integer",+ "min": 1,+ "max": 1000+ },"policyDocumentType": {"type": "string","min": 1,diff --git a/src/test/java/io/github/hectorvent/floci/core/common/AwsQueryListsTest.java b/src/test/java/io/github/hectorvent/floci/core/common/AwsQueryListsTest.javaindex 4396f9837d..1b3d5e1c49 100644--- a/src/test/java/io/github/hectorvent/floci/core/common/AwsQueryListsTest.java+++ b/src/test/java/io/github/hectorvent/floci/core/common/AwsQueryListsTest.java@@ -50,6 +50,38 @@ class AwsQueryListsTest {assertEquals(List.of(), AwsQueryLists.values(params, "TagKeys", "TagKey"));}+ @Test+ void membersCarryThePrefixTheirFieldsArriveUnderAndTheIndexAsSpelled() {+ MultivaluedMap<String, String> params = new MultivaluedHashMap<>();+ params.putSingle("Tags.member.7.Key", "team");+ params.putSingle("Tags.member.7.Value", "platform");+ params.putSingle("Tags.member.2.Key", "owner");+ params.putSingle("Audience.1", "");+ params.putSingle("Audience.member.3", "second");+ params.putSingle("OtherTags.member.1.Key", "another list");++ assertEquals(List.of(new AwsQueryLists.IndexedMember("Tags.member.2", "2"),+ new AwsQueryLists.IndexedMember("Tags.member.7", "7")),+ AwsQueryLists.members(params, "Tags", null),+ "one member per index, in the order the indexes read");+ assertEquals(List.of(new AwsQueryLists.IndexedMember("Audience.1", "1"),+ new AwsQueryLists.IndexedMember("Audience.member.3", "3")),+ AwsQueryLists.members(params, "Audience", null),+ "the bare numbered form is a member too, empty value and all");+ assertEquals(List.of(), AwsQueryLists.members(params, "Filter", null));+ }++ @Test+ void aMemberIsReadUnderTheNameItsShapeGivesItToo() {+ MultivaluedMap<String, String> params = new MultivaluedHashMap<>();+ params.putSingle("SubnetIds.SubnetIdentifier.4", "fourth");++ assertEquals(List.of(new AwsQueryLists.IndexedMember("SubnetIds.SubnetIdentifier.4", "4")),+ AwsQueryLists.members(params, "SubnetIds", "SubnetIdentifier"));+ assertEquals(List.of(), AwsQueryLists.members(params, "SubnetIds", null),+ "without the name, the generic and bare forms only");+ }+@Testvoid aMemberIndexIsTheNumberTheKeyEndsIn() {assertEquals(12, AwsQueryLists.memberIndex("Filters.Filter.3.Values.Value.12"));diff --git a/src/test/java/io/github/hectorvent/floci/core/common/QueryConstraintCheckTest.java b/src/test/java/io/github/hectorvent/floci/core/common/QueryConstraintCheckTest.javaindex 3ede31bd64..8960c37fb7 100644--- a/src/test/java/io/github/hectorvent/floci/core/common/QueryConstraintCheckTest.java+++ b/src/test/java/io/github/hectorvent/floci/core/common/QueryConstraintCheckTest.java@@ -33,10 +33,38 @@ class QueryConstraintCheckTest {}""";+ private static final String LIST_MODEL = """+ {+ "operations": {"Vend": "VendRequest"},+ "shapes": {+ "VendRequest": {"type": "structure", "required": ["Audience", "Target"], "members": {+ "Audience": "audienceList", "Names": "nameList", "Tags": "tagList",+ "Target": "targetType"}},+ "audienceList": {"type": "list", "member": "audienceType", "min": 1, "max": 2},+ "audienceType": {"type": "string", "min": 1, "max": 6},+ "nameList": {"type": "list", "member": "nameType", "min": 2},+ "tagList": {"type": "list", "member": "tagType", "max": 2},+ "tagType": {"type": "structure", "required": ["Key", "Value"], "members": {+ "Key": "nameType", "Value": "valueType"}},+ "targetType": {"type": "structure", "required": ["Arn"], "members": {+ "Arn": "arnType", "Note": "valueType"}},+ "nameType": {"type": "string", "min": 1, "max": 8, "pattern": "[\\\\w+=,.@-]+"},+ "valueType": {"type": "string", "max": 4},+ "arnType": {"type": "string", "min": 20, "max": 2048}+ }+ }+ """;++ private static final String ARN = "arn:aws:iam::000000000000:role/r";+private static QueryConstraintCheck check() throws Exception {return QueryConstraintCheck.of(MAPPER.readTree(MODEL));}+ private static QueryConstraintCheck lists() throws Exception {+ return QueryConstraintCheck.of(MAPPER.readTree(LIST_MODEL));+ }+private static MultivaluedMap<String, String> params(String... pairs) {MultivaluedMap<String, String> params = new MultivaluedHashMap<>();for (int pair = 0; pair < pairs.length; pair += 2) {@@ -114,6 +142,32 @@ class QueryConstraintCheckTest {assertEquals(List.of(), check().violations("Put", words));}+ /**+ * A number too large for a {@code long} is a number all the same, and the bound it breaks is+ * one that fits in one. Reading it as no number at all would have let it past the maximum.+ */+ @Test+ void aNumberTooLargeForALongIsStillHeldToItsRange() throws Exception {+ assertEquals(List.of("Value '9223372036854775808' at 'age' failed to satisfy constraint: "+ + "Member must have value less than or equal to 30"),+ check().violations("Put", params("Name", "ok", "Arn", "a".repeat(20),+ "Age", "9223372036854775808")));++ assertEquals(List.of("Value '-9223372036854775809' at 'age' failed to satisfy constraint: "+ + "Member must have value greater than or equal to 1"),+ check().violations("Put", params("Name", "ok", "Arn", "a".repeat(20),+ "Age", "-9223372036854775809")));++ assertEquals(List.of("Value '31' at 'age' failed to satisfy constraint: "+ + "Member must have value less than or equal to 30"),+ check().violations("Put", params("Name", "ok", "Arn", "a".repeat(20), "Age", "+31")),+ "a value is written as the number reads, the plus gone, as a parsed one already was");++ assertEquals(List.of(), check().violations("Put",+ params("Name", "ok", "Arn", "a".repeat(20), "Age", "+2")),+ "and a signed value inside the range is still inside it");+ }+/** A boolean declares nothing a request can break, so nothing is checked for one. */@Testvoid aBooleanIsNotChecked() throws Exception {@@ -142,46 +196,163 @@ class QueryConstraintCheckTest {}/**- * A list or a map is read from numbered parameters rather than from one, so an operation- * carrying either is refused when the file loads rather than passed over without its- * constraints.+ * A map is read from numbered entries whose key is a value of its own, and this package answers+ * one today against the whole map rather than against an entry, so an operation carrying one is+ * refused when the file loads rather than passed over without its constraints. So is anything+ * nested deeper than a list of structures or a structure of scalars.*/@Test- void anOperationCarryingAListIsRefusedWhenTheFileLoads() throws Exception {- JsonNode model = MAPPER.readTree("""+ void anOperationCarryingAMapOrDeeperNestingIsRefusedWhenTheFileLoads() throws Exception {+ JsonNode map = MAPPER.readTree("""+ {+ "operations": {"Put": "PutRequest"},+ "shapes": {+ "PutRequest": {"type": "structure", "members": {"Properties": "propertyMap"}},+ "propertyMap": {"type": "map", "key": "nameType", "value": "nameType"},+ "nameType": {"type": "string", "min": 1}+ }+ }+ """);++ IllegalStateException refused =+ assertThrows(IllegalStateException.class, () -> QueryConstraintCheck.of(map));+ assertTrue(refused.getMessage().contains("Put's Properties is a map"), refused.getMessage());++ JsonNode listOfLists = MAPPER.readTree("""{- "operations": {"Tag": "TagRequest"},+ "operations": {"Put": "PutRequest"},"shapes": {- "TagRequest": {"type": "structure", "members": {"Tags": "tagList"}},- "tagList": {"type": "list", "member": "nameType", "max": 50},+ "PutRequest": {"type": "structure", "members": {"Rows": "rowList"}},+ "rowList": {"type": "list", "member": "nameList"},+ "nameList": {"type": "list", "member": "nameType"},"nameType": {"type": "string", "min": 1}}}""");- IllegalStateException thrown =- assertThrows(IllegalStateException.class, () -> QueryConstraintCheck.of(model));+ IllegalStateException nested =+ assertThrows(IllegalStateException.class, () -> QueryConstraintCheck.of(listOfLists));+ assertTrue(nested.getMessage().contains("Put's Rows is a list of list"), nested.getMessage());- assertTrue(thrown.getMessage().contains("Tag's Tags is a list"), thrown.getMessage());+ JsonNode structureInStructure = MAPPER.readTree("""+ {+ "operations": {"Put": "PutRequest"},+ "shapes": {+ "PutRequest": {"type": "structure", "members": {"Target": "targetType"}},+ "targetType": {"type": "structure", "members": {"Inner": "innerType"}},+ "innerType": {"type": "structure", "members": {"Name": "nameType"}},+ "nameType": {"type": "string", "min": 1}+ }+ }+ """);++ IllegalStateException deeper = assertThrows(IllegalStateException.class,+ () -> QueryConstraintCheck.of(structureInStructure));+ assertTrue(deeper.getMessage().contains("Put's Target.Inner is a structure"),+ deeper.getMessage());+ }++ /**+ * A list's members are held to the element's own constraints, each named under the index the+ * request spelled rather than the position it sits at, which is what #6478 settled for a member+ * of a Query list. A member present and empty is still a member.+ */+ @Test+ void aListsMembersAreNamedUnderTheIndexTheRequestSpelled() throws Exception {+ List<String> violations = lists().violations("Vend", params(+ "Audience.member.1", "", "Audience.member.7", "overlong", "Target.Arn", ARN));++ assertEquals(List.of(+ "Value '' at 'audience.1.member' failed to satisfy constraint: "+ + "Member must have length greater than or equal to 1",+ "Value 'overlong' at 'audience.7.member' failed to satisfy constraint: "+ + "Member must have length less than or equal to 6"),+ violations);++ assertEquals(List.of("Value '' at 'audience.1.member' failed to satisfy constraint: "+ + "Member must have length greater than or equal to 1"),+ lists().violations("Vend", params("Audience.1", "", "Target.Arn", ARN)),+ "the bare numbered form is a member too");+ }++ /**+ * The count is held to the list's own bounds, with the value left out of the message: quoting it+ * would echo every member of the list back. A list of no members is how an absent list arrives+ * on this protocol, so a required one reads as null and a minimum of 1 is unreachable.+ */+ @Test+ void aListIsHeldToItsOwnBoundsAndOneOfNoMembersReadsAsAbsent() throws Exception {+ assertEquals(List.of("Value at 'audience' failed to satisfy constraint: "+ + "Member must have length less than or equal to 2"),+ lists().violations("Vend", params("Audience.member.1", "a", "Audience.member.2", "b",+ "Audience.member.3", "c", "Target.Arn", ARN)));++ assertEquals(List.of("Value null at 'audience' failed to satisfy constraint: "+ + "Member must not be null"),+ lists().violations("Vend", params("Target.Arn", ARN)),+ "a required list of no members is the member reading as null");++ assertEquals(List.of("Value at 'names' failed to satisfy constraint: "+ + "Member must have length greater than or equal to 2"),+ lists().violations("Vend", params("Audience.member.1", "a", "Target.Arn", ARN,+ "Names.member.1", "one")),+ "a minimum above one is reachable, where a minimum of one is not");+ }++ /** A member of a list of structures carries its fields under its own prefix. */+ @Test+ void aListOfStructuresNamesEachFieldUnderItsMember() throws Exception {+ List<String> violations = lists().violations("Vend", params(+ "Audience.member.1", "a", "Target.Arn", ARN,+ "Tags.member.1.Key", "", "Tags.member.1.Value", "toolong",+ "Tags.member.2.Key", "ok"));++ assertEquals(List.of(+ "Value '' at 'tags.1.member.key' failed to satisfy constraint: "+ + "Member must have length greater than or equal to 1",+ "Value '' at 'tags.1.member.key' failed to satisfy constraint: "+ + "Member must satisfy regular expression pattern: [\\w+=,.@-]+",+ "Value 'toolong' at 'tags.1.member.value' failed to satisfy constraint: "+ + "Member must have length less than or equal to 4",+ "Value null at 'tags.2.member.value' failed to satisfy constraint: "+ + "Member must not be null"),+ violations);+ }++ /**+ * A structure member arrives under its own name, and a request naming none of its fields is a+ * request without the member at all.+ */+ @Test+ void aStructureMemberNamesItsFieldsUnderItAndReadsAsAbsentWithNone() throws Exception {+ assertEquals(List.of(+ "Value null at 'target.arn' failed to satisfy constraint: Member must not be null",+ "Value 'toolong' at 'target.note' failed to satisfy constraint: "+ + "Member must have length less than or equal to 4"),+ lists().violations("Vend", params("Audience.member.1", "a", "Target.Note", "toolong")));++ assertEquals(List.of("Value null at 'target' failed to satisfy constraint: "+ + "Member must not be null"),+ lists().violations("Vend", params("Audience.member.1", "a")));}/*** The vendored IAM file itself, read the way the handler reads it. Loading it is the check on- * it: every operation's members have to be scalars and every pattern has to be one Java's- * engine compiles, and a wave that breaks either would otherwise surface as a failure on the- * first IAM request rather than here. Naming the twelve operations keeps the wave's own- * contract, so an operation renamed or dropped in the model is a failure with a name on it.+ * it: every operation's members have to be ones this walk reads and every pattern has to be one+ * Java's engine compiles, and a wave that breaks either would otherwise surface as a failure on+ * the first IAM request rather than here. Naming the operations keeps each wave's own contract,+ * so one renamed or dropped in the model is a failure with a name on it.*/@Test- void theVendoredIamFileLoadsAndCarriesTheOperationsThisWaveClaims() {+ void theVendoredIamFileLoadsAndCarriesTheOperationsTheseWavesClaim() {QueryConstraintCheck iam =QueryConstraintCheck.load("aws/iam-request-shapes.json", "IAM request shapes");- List<String> wave = List.of("AttachGroupPolicy", "AttachRolePolicy", "AttachUserPolicy",- "DetachGroupPolicy", "DetachRolePolicy", "DetachUserPolicy", "GetPolicy",- "DeletePolicy", "CreatePolicyVersion", "GetPolicyVersion", "DeletePolicyVersion",- "SetDefaultPolicyVersion");- for (String operation : wave) {+ List<String> managedPolicyFamily = List.of("AttachGroupPolicy", "AttachRolePolicy",+ "AttachUserPolicy", "DetachGroupPolicy", "DetachRolePolicy", "DetachUserPolicy",+ "GetPolicy", "DeletePolicy", "CreatePolicyVersion", "GetPolicyVersion",+ "DeletePolicyVersion", "SetDefaultPolicyVersion");+ for (String operation : managedPolicyFamily) {assertTrue(iam.violations(operation, params()).stream().allMatch(violation -> violation.contains("must not be null")),operation + " reported something other than its absent required members");@@ -192,5 +363,13 @@ class QueryConstraintCheckTest {assertEquals(List.of("Value 'v1' at 'policyArn' failed to satisfy constraint: Member must"+ " have length greater than or equal to 20"),iam.violations("GetPolicy", params("PolicyArn", "v1")));++ assertEquals(List.of(), iam.violations("GetAccountAuthorizationDetails", params()),+ "every member of it is optional, so a request naming none of them breaks nothing");+ assertEquals(List.of("Value 'Users' at 'filter.1.member' failed to satisfy constraint:"+ + " Member must satisfy enum value set:"+ + " [User, Role, Group, LocalManagedPolicy, AWSManagedPolicy]"),+ iam.violations("GetAccountAuthorizationDetails", params("Filter.member.1", "Users")),+ "the one list this wave opts in resolves to its member's enum");}}diff --git a/src/test/java/io/github/hectorvent/floci/services/iam/IamRequestConstraintsIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/iam/IamRequestConstraintsIntegrationTest.javaindex fd69ad82bd..9775d27125 100644--- a/src/test/java/io/github/hectorvent/floci/services/iam/IamRequestConstraintsIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/iam/IamRequestConstraintsIntegrationTest.java@@ -10,11 +10,12 @@ import static org.hamcrest.Matchers.equalTo;import static org.hamcrest.Matchers.not;/**- * The declared constraints the managed-policy family is now read against, from the model rather- * than from a hand-written check. What they change is the status: a value that breaks a declared- * constraint used to reach the store and answer {@code NoSuchEntity}, where AWS's validation layer- * runs ahead of the operation and answers {@code ValidationError}, which is the ordering #6402- * settled for this package.+ * The declared constraints the managed-policy family and {@code GetAccountAuthorizationDetails} are+ * now read against, from the model rather than from a hand-written check. What they change is the+ * status: a value that breaks a declared constraint used to reach the store and answer+ * {@code NoSuchEntity}, or on the second of those to be answered with a 200, where AWS's validation+ * layer runs ahead of the operation and answers {@code ValidationError}, which is the ordering+ * #6402 settled for this package.** <p>{@code AttachRolePolicy}'s {@code RoleName} is the parity case rather than a change:* {@code IamService.validateRoleName} already answered it in this form, from the same bounds and@@ -230,4 +231,79 @@ class IamRequestConstraintsIntegrationTest {.when().post("/").then().statusCode(404).body(CODE, equalTo("NoSuchEntity"));}++ /**+ * {@code GetAccountAuthorizationDetails} is the first operation read through the list walk, and+ * none of its three members was read at all before: a {@code Filter} outside the model's enum,+ * a {@code MaxItems} outside its range and a {@code Marker} breaking its length all reached the+ * operation and were answered with a 200.+ */+ @Test+ void aFilterMemberOutsideItsEnumIsRefusedWhereItUsedToBeIgnored() {+ iam("GetAccountAuthorizationDetails")+ .formParam("Filter.member.1", "Users")+ .when().post("/").then().statusCode(400)+ .body(CODE, equalTo("ValidationError"))+ .body(MESSAGE, equalTo("1 validation error detected: Value 'Users' at"+ + " 'filter.1.member' failed to satisfy constraint: Member must satisfy enum"+ + " value set: [User, Role, Group, LocalManagedPolicy, AWSManagedPolicy]"));++ iam("GetAccountAuthorizationDetails")+ .formParam("Filter.member.1", "User")+ .when().post("/").then().statusCode(200)+ .body(containsString("GetAccountAuthorizationDetailsResult"));+ }++ /**+ * Each member of a list is named under the index the request spelled rather than the position+ * it sits at, and the two are reported together in one message.+ */+ @Test+ void twoBrokenListMembersAreReportedTogetherUnderTheIndexesTheRequestSpelled() {+ iam("GetAccountAuthorizationDetails")+ .formParam("Filter.member.2", "Users")+ .formParam("Filter.member.9", "Policies")+ .when().post("/").then().statusCode(400)+ .body(CODE, equalTo("ValidationError"))+ .body(MESSAGE, containsString("2 validation errors detected: Value 'Users' at"+ + " 'filter.2.member'"))+ .body(MESSAGE, containsString("Value 'Policies' at 'filter.9.member'"));+ }++ /** {@code maxItemsType} is 1 to 1000, which this operation reads nothing of today. */+ @Test+ void aMaxItemsOutsideItsRangeIsRefused() {+ iam("GetAccountAuthorizationDetails")+ .formParam("MaxItems", "1001")+ .when().post("/").then().statusCode(400)+ .body(CODE, equalTo("ValidationError"))+ .body(MESSAGE, equalTo("1 validation error detected: Value '1001' at 'maxItems' failed"+ + " to satisfy constraint: Member must have value less than or equal to 1000"));++ iam("GetAccountAuthorizationDetails")+ .formParam("MaxItems", "1000")+ .when().post("/").then().statusCode(200);++ // A number too large for a long is a number all the same, and 1000 is still the bound.+ iam("GetAccountAuthorizationDetails")+ .formParam("MaxItems", "9223372036854775808")+ .when().post("/").then().statusCode(400)+ .body(CODE, equalTo("ValidationError"))+ .body(MESSAGE, equalTo("1 validation error detected: Value '9223372036854775808' at"+ + " 'maxItems' failed to satisfy constraint: Member must have value less than"+ + " or equal to 1000"));+ }++ /** A marker sent empty breaks its minimum and its pattern, and both are reported. */+ @Test+ void anEmptyMarkerBreaksItsLengthAndItsPattern() {+ iam("GetAccountAuthorizationDetails")+ .formParam("Marker", "")+ .when().post("/").then().statusCode(400)+ .body(CODE, equalTo("ValidationError"))+ .body(MESSAGE, containsString("2 validation errors detected: Value '' at 'marker'"+ + " failed to satisfy constraint: Member must have length greater than or"+ + " equal to 1"))+ .body(MESSAGE, containsString("Member must satisfy regular expression pattern:"));+ }}diff --git a/tools/aws/regen_iam_shapes.py b/tools/aws/regen_iam_shapes.pyindex 3908057f50..56f0a93258 100644--- a/tools/aws/regen_iam_shapes.py+++ b/tools/aws/regen_iam_shapes.py@@ -24,15 +24,23 @@ OUTPUT = REPO_ROOT / "src/main/resources/aws/iam-request-shapes.json"MODEL = Path("iam/2010-05-08")GENERATOR = "tools/aws/regen_iam_shapes.py"-# The managed-policy family: every member a scalar, and no operation here checks a declared-# constraint by hand today, so the vendored shapes add the refusals AWS answers without rewording-# anything already answered. `AttachRolePolicy` and `DetachRolePolicy` are the exception and the-# parity case: `IamService.validateRoleName` already answers `roleName` in Smithy's form.+# One wave per group, each chosen so that no operation in it checks a declared constraint by hand,+# which is what keeps the vendored shapes adding the refusals AWS answers rather than rewording an+# answer already given.+#+# The managed-policy family, every member a scalar. `AttachRolePolicy` and `DetachRolePolicy` are+# the exception and the parity case: `IamService.validateRoleName` already answers `roleName` in+# Smithy's form.+#+# `GetAccountAuthorizationDetails` is the first operation read through the list walk. Its three+# members are declared and none of them is read at all today: `Filter` is a list of an enum,+# `MaxItems` a range and `Marker` a length and a pattern.OPERATIONS = ("AttachGroupPolicy", "AttachRolePolicy", "AttachUserPolicy","DetachGroupPolicy", "DetachRolePolicy", "DetachUserPolicy","GetPolicy", "DeletePolicy","CreatePolicyVersion", "GetPolicyVersion", "DeletePolicyVersion", "SetDefaultPolicyVersion",+ "GetAccountAuthorizationDetails",)