Light, fluffy, and always free - The AWS Local Emulator alternative
fix(partitions): source the China CloudFront suffix and GovCloud API Gateway zones (#5618)
Hector Ventura3afb5b430e72
diff --git a/docs/configuration/partitions.md b/docs/configuration/partitions.mdindex b84d54d878..0657efd32e 100644--- a/docs/configuration/partitions.md+++ b/docs/configuration/partitions.md@@ -114,12 +114,16 @@ back to `aws` for a label no partition publishes.ids are the same literal in every partition, as the Sign-In model spells them.- **Hosted zones and console URLs**: load balancer and S3 website hosted zone ids come from aper-region table (`aws/region-facts.json`, generated from the Terraform provider and the CDK,- which transcribe the AWS General Reference); Network load balancers have their own zone,- distinct from the Application/Classic one. The CloudFront hosted zone is published for `aws`- and `aws-cn`; the SAML sign-on URL for five partitions. The console link in a CodePipeline+ which transcribe the AWS General Reference, and from the China and GovCloud user guides); Network+ load balancers have their own zone, distinct from the Application/Classic one. The CloudFront+ hosted zone is published for `aws` and `aws-cn`; the SAML sign-on URL for five partitions. The console link in a CodePipelineapproval notification uses the console host of the pipeline's partition (`console.amazonaws.cn`in China, `console.amazonaws-us-gov.com` in GovCloud), as the China and GovCloud user guides- state it; the path after the host is the commercial console's. The ISO and EUSC regions have no load+ state it; the path after the host is the commercial console's. A generated CloudFront+ distribution domain name ends in `cloudfront.cn` in China, as the China guide names the default+ CloudFront domain, unless `floci.services.cloudfront.domain-suffix` is set. A regional API Gateway+ custom domain reports the regional hosted zone the GovCloud guide states for `us-gov-west-1` and+ `us-gov-east-1`. The ISO and EUSC regions have no loadbalancer hosted zones, and only the two ISO-F regions have an S3 website zone; where a table hasno row the field is omitted rather than guessed. An `EDGE` API Gateway REST API or custom domain isrefused outside the commercial partition (`BadRequestException`): GovCloud, EUSC and ISO have no@@ -244,13 +248,13 @@ partition.These values have no published source Floci can cite, so it does not guess them; they keepthe commercial value or Floci's own base host until sourced:-- the China CloudFront distribution domain suffix (only the API host is published);- the STS web-identity audience outside the commercial partition;- the EKS Pod Identity token audience (`pods.eks.amazonaws.com`) outside the commercial partition;- the Lambda function-URL host in the ISO and EUSC partitions (function URLs are refused in GovCloudand China, where the partition guides list them as not available);- whether AWS managed policy documents differ in content in China or GovCloud;-- the API Gateway regional hosted zone per region;+- the API Gateway regional hosted zone per region outside GovCloud: only the two GovCloud zones+ are sourced, and every other region reports one placeholder zone id;- the availability-zone id prefix of the ISO and EUSC regions: zone ids there keep the whole region(`us-iso-east-1-az1`), where the other partitions get AWS's short prefix (`use1-az1`, `usgw1-az1`,`cnn1-az1`);diff --git a/docs/services/cloudfront.md b/docs/services/cloudfront.mdindex f58b563669..7daaec200a 100644--- a/docs/services/cloudfront.md+++ b/docs/services/cloudfront.md@@ -266,7 +266,7 @@ PUT, PATCH and DELETE).| Property | Env var | Default | Description ||---|---|---|---|| `floci.services.cloudfront.enabled` | `FLOCI_SERVICES_CLOUDFRONT_ENABLED` | `true` | Enable or disable the service |-| `floci.services.cloudfront.domain-suffix` | `FLOCI_SERVICES_CLOUDFRONT_DOMAIN_SUFFIX` | `cloudfront.net` | Domain suffix for generated distribution domain names |+| `floci.services.cloudfront.domain-suffix` | `FLOCI_SERVICES_CLOUDFRONT_DOMAIN_SUFFIX` | unset | Domain suffix for generated distribution domain names. Unset, it follows the request's partition: `cloudfront.net`, or `cloudfront.cn` in China. A configured suffix applies in every partition || `floci.services.cloudfront.allowed-private-origin-hosts` | `FLOCI_SERVICES_CLOUDFRONT_ALLOWED_PRIVATE_ORIGIN_HOSTS` | `[]` | Exact custom-origin hosts permitted to resolve to private/non-routable addresses (comma-separated in the environment variable) |## CLI Examplesdiff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.javaindex fd4a8eec52..f7d5ffa737 100644--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java@@ -2383,8 +2383,11 @@ public interface EmulatorConfig {@WithDefault("true")boolean enabled();- @WithDefault("cloudfront.net") // partition-literal: configurable; the China CDN suffix has no published source- String domainSuffix();+ /**+ * Suffix of generated distribution domain names, in every partition when set. Unset, the+ * suffix follows the request's partition ({@code CloudFrontEdgeDomain#domainSuffix}).+ */+ Optional<String> domainSuffix();/*** Exact custom-origin hostnames allowed to resolve to private or otherwise non-routablediff --git a/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java b/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.javaindex 5cde2fd447..a8c8d362b6 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java@@ -13,9 +13,10 @@ import java.util.Set;/*** The per-region and per-partition constants AWS publishes only as prose tables, vendored in* {@code aws/region-facts.json} by {@code tools/aws/regen_region_facts.py} from the Terraform- * provider, the CDK and the China and GovCloud user guides: load balancer and S3 website hosted- * zone ids, the CloudFront hosted zone per partition, the SAML sign-on URL and the management- * console host per partition and the VPC endpoint service-name prefixes. Every accessor is {@link Optional}: the ISO and EUSC partitions have no published+ * provider, the CDK, the botocore models and the China and GovCloud user guides: load balancer,+ * S3 website and GovCloud API Gateway hosted zone ids, the CloudFront hosted zone and distribution+ * domain suffix per partition, the SAML sign-on URL and the management console host per partition+ * and the VPC endpoint service-name prefixes. Every accessor is {@link Optional}: the ISO and EUSC partitions have no published* hosted zones, and a service that needs one there has nothing faithful to answer.*/public final class AwsRegionFacts {@@ -48,6 +49,22 @@ public final class AwsRegionFacts {return regionText(region, "s3WebsiteHostedZoneId");}+ /**+ * The hosted zone of an API Gateway regional endpoint, which a Route 53 alias to a regional+ * custom domain targets. Published in the corpus for the two GovCloud regions only.+ */+ public static Optional<String> apiGatewayRegionalHostedZoneId(String region) {+ return regionText(region, "apiGatewayRegionalHostedZoneId");+ }++ /**+ * The suffix of a distribution's generated domain name: {@code cloudfront.net}, and+ * {@code cloudfront.cn} in {@code aws-cn}. Absent for the partitions without CloudFront.+ */+ public static Optional<String> cloudFrontDomainSuffix(String partition) {+ return partitionText(partition, "cloudfrontDomainSuffix");+ }+/** Published for {@code aws} and {@code aws-cn} only, where CloudFront exists. */public static Optional<String> cloudFrontHostedZoneId(String partition) {return partitionText(partition, "cloudfrontHostedZoneId");diff --git a/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java b/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.javaindex 6c3c7f1709..4c40ce6ebc 100644--- a/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java+++ b/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java@@ -4,19 +4,41 @@ import java.util.UUID;/*** The CloudFront distribution AWS puts in front of a custom domain, an edge-optimized API Gateway- * domain or an AppSync custom domain: a distribution domain name on the configured CloudFront- * suffix, and the CloudFront hosted zone of the region's partition, which a Route 53 alias record- * targets. The zone is null in a partition without CloudFront.+ * domain or an AppSync custom domain: a distribution domain name on the CloudFront suffix of the+ * partition (or the configured one), and the partition's CloudFront hosted zone, which a Route 53+ * alias record targets. The zone is null in a partition without CloudFront.** @param domainName the distribution domain name, {@code d<13 hex>.<suffix>}* @param hostedZoneId the partition's CloudFront hosted zone, or null where there is none*/public record CloudFrontEdgeDomain(String domainName, String hostedZoneId) {- public static CloudFrontEdgeDomain create(String region, String domainSuffix) {+ /**+ * @param partition the partition the domain is created in, as the caller's+ * {@link RegionResolver#partitionForRegion} resolves it, so a region no+ * partition publishes falls to the deployment's partition, not the+ * commercial one+ * @param configuredSuffix {@code floci.services.cloudfront.domain-suffix}, or null when unset+ */+ public static CloudFrontEdgeDomain create(String partition, String configuredSuffix) {String distributionId = "d" + UUID.randomUUID().toString().replace("-", "").substring(0, 13);- String hostedZoneId = AwsRegionFacts.cloudFrontHostedZoneId(- AwsPartitions.forRegionOrCommercial(region).id()).orElse(null);- return new CloudFrontEdgeDomain(distributionId + "." + domainSuffix, hostedZoneId);+ String hostedZoneId = AwsRegionFacts.cloudFrontHostedZoneId(partition).orElse(null);+ return new CloudFrontEdgeDomain(distributionId + "." + domainSuffix(configuredSuffix, partition), hostedZoneId);+ }++ /**+ * The suffix of a generated distribution domain name. An explicitly configured suffix wins in+ * every partition; otherwise it is the partition's own ({@code cloudfront.cn} in China), and+ * the commercial one in a partition that has no CloudFront and so no suffix to publish.+ *+ * @param configuredSuffix {@code floci.services.cloudfront.domain-suffix}, or null when unset+ */+ public static String domainSuffix(String configuredSuffix, String partition) {+ if (configuredSuffix != null && !configuredSuffix.isBlank()) {+ return configuredSuffix;+ }+ return AwsRegionFacts.cloudFrontDomainSuffix(partition)+ .or(() -> AwsRegionFacts.cloudFrontDomainSuffix(AwsRegions.DEFAULT_PARTITION))+ .orElseThrow();}}diff --git a/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java b/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.javaindex 2349d4240b..fd13c8d2e1 100644--- a/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java+++ b/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java@@ -11,6 +11,7 @@ import io.github.hectorvent.floci.core.common.AwsArnUtils;import io.github.hectorvent.floci.core.common.AwsException;import io.github.hectorvent.floci.core.common.AwsPartition;import io.github.hectorvent.floci.core.common.AwsPartitions;+import io.github.hectorvent.floci.core.common.AwsRegionFacts;import io.github.hectorvent.floci.core.common.AwsRegions;import io.github.hectorvent.floci.core.common.CloudFrontEdgeDomain;import io.github.hectorvent.floci.core.common.RegionResolver;@@ -86,6 +87,12 @@ public class ApiGatewayService implements ResourceProvider {/** The only partition where API Gateway offers edge-optimized APIs and custom domain names. */private static final Set<String> EDGE_OPTIMIZED_PARTITIONS = Set.of("aws");+ /**+ * The regional hosted zone a custom domain reports in a region with no published one. Only+ * the two GovCloud regions have a sourced zone ({@link AwsRegionFacts#apiGatewayRegionalHostedZoneId}).+ */+ private static final String UNSOURCED_REGIONAL_HOSTED_ZONE_ID = "Z2FDTNDATAQYL2"; // partition-literal: placeholder kept where no per-region source exists+private static final ObjectMapper JSON = new ObjectMapper();/** Documented default page size for GetUsage. */@@ -2115,7 +2122,8 @@ public class ApiGatewayService implements ResourceProvider {domain.setRegionalCertificateName((String) request.get("regionalCertificateName"));domain.setRegionalCertificateArn((String) request.get("regionalCertificateArn"));domain.setRegionalDomainName(domainName + ".regional.local");- domain.setRegionalHostedZoneId("Z2FDTNDATAQYL2"); // partition-literal: no published per-region source for the regional zone+ domain.setRegionalHostedZoneId(AwsRegionFacts.apiGatewayRegionalHostedZoneId(region)+ .orElse(UNSOURCED_REGIONAL_HOSTED_ZONE_ID));applyEndpointType(domain, endpointType, region);domain.setSecurityPolicy((String) request.getOrDefault("securityPolicy", "TLS_1_2"));// Nothing is provisioned behind the domain, so it is usable as soon as it exists.@@ -2168,7 +2176,8 @@ public class ApiGatewayService implements ResourceProvider {domain.setDistributionDomainName(null);domain.setDistributionHostedZoneId(null);} else if (domain.getDistributionDomainName() == null) {- CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(region, config.services().cloudfront().domainSuffix());+ CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(regionResolver.partitionForRegion(region),+ config.services().cloudfront().domainSuffix().orElse(null));domain.setDistributionDomainName(edge.domainName());domain.setDistributionHostedZoneId(edge.hostedZoneId());}diff --git a/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java b/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.javaindex 2cbbb38e5d..1267166b41 100644--- a/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java+++ b/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java@@ -60,7 +60,7 @@ public class AppSyncService {private final ObjectMapper objectMapper;private final Clock clock;private final String baseUrl;- private final String cloudFrontDomainSuffix;+ private final String configuredCloudFrontDomainSuffix;private final LambdaAuthorizerCache lambdaAuthorizerCache;@Inject@@ -90,7 +90,7 @@ public class AppSyncService {this.objectMapper = objectMapper;this.clock = clock;this.baseUrl = trimTrailingSlash(config.effectiveBaseUrl());- this.cloudFrontDomainSuffix = config.services().cloudfront().domainSuffix();+ this.configuredCloudFrontDomainSuffix = config.services().cloudfront().domainSuffix().orElse(null);this.lambdaAuthorizerCache = lambdaAuthorizerCache;}@@ -761,7 +761,8 @@ public class AppSyncService {* Gateway domain gets.*/private void applyCloudFrontFront(DomainName dn, String region) {- CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(region, cloudFrontDomainSuffix);+ CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(regionResolver.partitionForRegion(region),+ configuredCloudFrontDomainSuffix);dn.setAppsyncDomainName(edge.domainName());dn.setHostedZoneId(edge.hostedZoneId());}diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java b/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.javaindex 1c3e790999..d0a8b83ce1 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java@@ -3,6 +3,7 @@ package io.github.hectorvent.floci.services.cloudfront;import com.fasterxml.jackson.core.type.TypeReference;import io.github.hectorvent.floci.config.EmulatorConfig;import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.core.common.CloudFrontEdgeDomain;import io.github.hectorvent.floci.core.common.RegionResolver;import io.github.hectorvent.floci.core.common.dns.EmbeddedDnsServer;import io.github.hectorvent.floci.core.storage.StorageBackend;@@ -126,7 +127,7 @@ public class CloudFrontService implements ServerCertificateReferenceProvider {private final String accountId;private final RegionResolver regionResolver;private final IamService iamService;- private final String domainSuffix;+ private final String configuredDomainSuffix;/*** Host suffixes under which every distribution is served as {@code <id><suffix>}, whatever* {@code floci.services.cloudfront.domain-suffix} makes the assigned domain name. They carry the@@ -182,7 +183,7 @@ public class CloudFrontService implements ServerCertificateReferenceProvider {new TypeReference<Map<String, MonitoringSubscription>>() {});this.accountId = config.defaultAccountId();this.regionResolver = regionResolver;- this.domainSuffix = config.services().cloudfront().domainSuffix();+ this.configuredDomainSuffix = config.services().cloudfront().domainSuffix().orElse(null);this.localDeliverySuffixes = localDeliverySuffixes(config);}@@ -459,10 +460,12 @@ public class CloudFrontService implements ServerCertificateReferenceProvider {/*** The domain name a new distribution is served under. AWS assigns a lower-case host, and a* browser lower-cases the authority it sends, so an upper-case id in the host would never match- * the resource a signed URL was signed for.+ * the resource a signed URL was signed for. The suffix is the request partition's unless one+ * is configured.*/private String domainNameFor(String id) {- return id.toLowerCase(Locale.ROOT) + "." + domainSuffix;+ return id.toLowerCase(Locale.ROOT) + "."+ + CloudFrontEdgeDomain.domainSuffix(configuredDomainSuffix, regionResolver.getPartition());}private static List<String> localDeliverySuffixes(EmulatorConfig config) {diff --git a/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java b/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.javaindex c338a01cef..4ca8e5f0ee 100644--- a/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java+++ b/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java@@ -13,6 +13,7 @@ import io.github.hectorvent.floci.core.common.AwsPartition;import io.github.hectorvent.floci.core.common.AwsPartitions;import io.github.hectorvent.floci.core.common.AwsRegions;import io.github.hectorvent.floci.core.common.Base64Url;+import io.github.hectorvent.floci.core.common.CloudFrontEdgeDomain;import io.github.hectorvent.floci.core.common.Digests;import io.github.hectorvent.floci.core.common.PaginatedResult;import io.github.hectorvent.floci.core.common.Pagination;@@ -112,10 +113,10 @@ public class CognitoService implements ResourceProvider {// JVM-local stripes bound lock memory without retaining one lock for every user key.private static final int USER_LOCK_STRIPES = 512;/**- * The commercial CloudFront suffix for the test constructors; CDI wires- * {@code floci.services.cloudfront.domain-suffix}.+ * No configured CloudFront suffix, for the test constructors: the suffix then follows the+ * request's partition. CDI wires {@code floci.services.cloudfront.domain-suffix}.*/- private static final String DEFAULT_CLOUDFRONT_DOMAIN_SUFFIX = "cloudfront.net"; // partition-literal: test-shaped constructor default+ private static final String UNCONFIGURED_CLOUDFRONT_DOMAIN_SUFFIX = null;private static final Logger LOG = Logger.getLogger(CognitoService.class);private static final ObjectMapper MAPPER = new ObjectMapper();@@ -219,7 +220,7 @@ public class CognitoService implements ResourceProvider {storageFactory.create("cognito", "cognito-revoked-tokens.json",new TypeReference<Map<String, RevokedTokenInfo>>() {}),trimTrailingSlash(emulatorConfig.effectiveBaseUrl()),- emulatorConfig.services().cloudfront().domainSuffix(),+ emulatorConfig.services().cloudfront().domainSuffix().orElse(null),regionResolver,lambdaService,acmService,@@ -242,7 +243,7 @@ public class CognitoService implements ResourceProvider {LambdaService lambdaService,AcmService acmService) {this(poolStore, clientStore, resourceServerStore, userStore, groupStore, revokedTokenStore, baseUrl,- DEFAULT_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver, lambdaService, acmService);+ UNCONFIGURED_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver, lambdaService, acmService);}CognitoService(StorageBackend<String, UserPool> poolStore,@@ -259,7 +260,7 @@ public class CognitoService implements ResourceProvider {CognitoMessageDispatcher messageDispatcher,TlsCertificateManager certificateManager) {this(poolStore, clientStore, resourceServerStore, domainStore, identityProviderStore, userStore,- groupStore, revokedTokenStore, baseUrl, DEFAULT_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver,+ groupStore, revokedTokenStore, baseUrl, UNCONFIGURED_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver,lambdaService, acmService, verificationCodeService, messageDispatcher, certificateManager);}@@ -1934,8 +1935,10 @@ public class CognitoService implements ResourceProvider {return AwsArnUtils.Arn.global(partition, "cloudfront", userPoolDomain.getAwsAccountId(), "distribution/" + id).toString();}+ /** A distribution name on the configured CloudFront suffix, or the request partition's when none is. */private String generateCloudFrontDomain() {- return UUID.randomUUID().toString().replace("-", "").substring(0, 14) + "." + cloudFrontDomainSuffix;+ return UUID.randomUUID().toString().replace("-", "").substring(0, 14) + "."+ + CloudFrontEdgeDomain.domainSuffix(cloudFrontDomainSuffix, regionResolver.getPartition());}private String generateDomainVersion() {diff --git a/src/main/resources/aws/region-facts.json b/src/main/resources/aws/region-facts.jsonindex 7903b1a653..011b24fc25 100644--- a/src/main/resources/aws/region-facts.json+++ b/src/main/resources/aws/region-facts.json@@ -3,7 +3,8 @@"generator": "tools/aws/regen_region_facts.py","terraform": "terraform-provider-aws internal/service/elb/hosted_zone_id_data_source.go, internal/service/elbv2/hosted_zone_id_data_source.go, internal/service/s3/hosted_zones.go, internal/conns/awsclient.go","cdk": "aws-cdk packages/aws-cdk-lib/region-info/build-tools/fact-tables.ts, packages/aws-cdk-lib/aws-route53-targets/lib/cloudfront-target.ts, packages/aws-cdk-lib/aws-ec2/lib/vpc-endpoint.ts",- "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) cn/console.md, gov/configure-account.md, gov/govcloud-organizations.md",+ "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) cn/cloudfront.md, cn/console.md, gov/configure-account.md, gov/govcloud-abp.md, gov/govcloud-organizations.md",+ "botocore": "botocore cloudfront/2020-05-31/service-2.json (Distribution$DomainName, DistributionSummary$DomainName)","s3WebsiteConflicts": "terraform wins over the CDK where they disagree (it cites the S3 website-endpoint table)"},"partitions": {@@ -11,13 +12,15 @@"vpcEndpointServiceNamePrefix": "com.amazonaws.vpce","cloudfrontHostedZoneId": "Z2FDTNDATAQYW2","samlSignOnUrl": "https://signin.aws.amazon.com/saml",- "consoleHost": "console.aws.amazon.com"+ "consoleHost": "console.aws.amazon.com",+ "cloudfrontDomainSuffix": "cloudfront.net"},"aws-cn": {"vpcEndpointServiceNamePrefix": "cn.com.amazonaws.vpce","cloudfrontHostedZoneId": "Z3RFFRIM2A3IF5","samlSignOnUrl": "https://signin.amazonaws.cn/saml",- "consoleHost": "console.amazonaws.cn"+ "consoleHost": "console.amazonaws.cn",+ "cloudfrontDomainSuffix": "cloudfront.cn"},"aws-eusc": {"vpcEndpointServiceNamePrefix": "eu.amazonaws.vpce"@@ -386,13 +389,15 @@"albHostedZoneId": "Z166TLBEWOO7G0","classicElbHostedZoneId": "Z166TLBEWOO7G0","nlbHostedZoneId": "Z1ZSMQQ6Q24QQ8",- "s3WebsiteHostedZoneId": "Z2NIFVYYW2VKV1"+ "s3WebsiteHostedZoneId": "Z2NIFVYYW2VKV1",+ "apiGatewayRegionalHostedZoneId": "Z3SE9ATJYCRCZJ"},"us-gov-west-1": {"albHostedZoneId": "Z33AYJ8TM3BH4J","classicElbHostedZoneId": "Z33AYJ8TM3BH4J","nlbHostedZoneId": "ZMG1MZ2THAWF1",- "s3WebsiteHostedZoneId": "Z31GFT0UA1I2HV"+ "s3WebsiteHostedZoneId": "Z31GFT0UA1I2HV",+ "apiGatewayRegionalHostedZoneId": "Z1K6XKP9SAGWDV"},"us-iso-east-1": {"vpcEndpointExceptionPrefix": "gov.ic.c2s",diff --git a/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java b/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.javaindex 068e3eca12..82f61ed493 100644--- a/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java+++ b/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java@@ -61,6 +61,26 @@ class AwsRegionFactsTest {assertTrue(AwsRegionFacts.consoleHost(null).isEmpty());}+ /** The botocore model's example name for the commercial partition, the China guide for {@code aws-cn}. */+ @Test+ void cloudFrontDomainSuffixIsPerPartitionWhereCloudFrontExists() {+ assertEquals(Optional.of("cloudfront.net"), AwsRegionFacts.cloudFrontDomainSuffix("aws"));+ assertEquals(Optional.of("cloudfront.cn"), AwsRegionFacts.cloudFrontDomainSuffix("aws-cn"));+ assertTrue(AwsRegionFacts.cloudFrontDomainSuffix("aws-us-gov").isEmpty());+ assertTrue(AwsRegionFacts.cloudFrontDomainSuffix("aws-eusc").isEmpty());+ assertTrue(AwsRegionFacts.cloudFrontDomainSuffix(null).isEmpty());+ }++ /** The GovCloud guide states the two regional endpoint zones; no source covers another region. */+ @Test+ void apiGatewayRegionalZoneIsPublishedForGovCloudOnly() {+ assertEquals(Optional.of("Z1K6XKP9SAGWDV"), AwsRegionFacts.apiGatewayRegionalHostedZoneId("us-gov-west-1"));+ assertEquals(Optional.of("Z3SE9ATJYCRCZJ"), AwsRegionFacts.apiGatewayRegionalHostedZoneId("us-gov-east-1"));+ assertTrue(AwsRegionFacts.apiGatewayRegionalHostedZoneId("us-east-1").isEmpty());+ assertTrue(AwsRegionFacts.apiGatewayRegionalHostedZoneId("cn-north-1").isEmpty());+ assertTrue(AwsRegionFacts.apiGatewayRegionalHostedZoneId(null).isEmpty());+ }+@ParameterizedTest@CsvSource({"us-east-1, s3, com.amazonaws.us-east-1.s3",diff --git a/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java b/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.javaindex fe278c4f2b..98cbf29b52 100644--- a/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java+++ b/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java@@ -12,17 +12,35 @@ class CloudFrontEdgeDomainTest {@Testvoid namesADistributionOnTheConfiguredSuffix() {- CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create("us-east-1", "cloudfront.net");+ CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create("aws", "cloudfront.net");assertTrue(edge.domainName().matches("d[0-9a-f]{13}\\.cloudfront\\.net"), edge.domainName());- assertNotEquals(edge.domainName(), CloudFrontEdgeDomain.create("us-east-1", "cloudfront.net").domainName());+ assertNotEquals(edge.domainName(), CloudFrontEdgeDomain.create("aws", "cloudfront.net").domainName());+ }++ @Test+ void anUnconfiguredSuffixIsThePartitionsOwn() {+ assertTrue(CloudFrontEdgeDomain.create("aws", null).domainName().endsWith(".cloudfront.net"));+ assertTrue(CloudFrontEdgeDomain.create("aws-cn", null).domainName().endsWith(".cloudfront.cn"));+ assertEquals("cloudfront.net", CloudFrontEdgeDomain.domainSuffix(null, "aws"));+ assertEquals("cloudfront.cn", CloudFrontEdgeDomain.domainSuffix(null, "aws-cn"));+ assertEquals("cloudfront.cn", CloudFrontEdgeDomain.domainSuffix(" ", "aws-cn"), "blank is unset");+ assertEquals("cloudfront.net", CloudFrontEdgeDomain.domainSuffix(null, "aws-us-gov"),+ "a partition without CloudFront keeps the commercial suffix");+ assertEquals("cloudfront.net", CloudFrontEdgeDomain.domainSuffix(null, null));+ }++ @Test+ void aConfiguredSuffixWinsInEveryPartition() {+ assertEquals("cloudfront.local", CloudFrontEdgeDomain.domainSuffix("cloudfront.local", "aws-cn"));+ assertTrue(CloudFrontEdgeDomain.create("aws-cn", "cloudfront.net").domainName().endsWith(".cloudfront.net"));}@Testvoid hostedZoneIsThePartitionsCloudFrontZone() {- assertEquals("Z2FDTNDATAQYW2", CloudFrontEdgeDomain.create("eu-west-1", "cloudfront.net").hostedZoneId());- assertEquals("Z3RFFRIM2A3IF5", CloudFrontEdgeDomain.create("cn-north-1", "cloudfront.net").hostedZoneId());- assertNull(CloudFrontEdgeDomain.create("us-gov-west-1", "cloudfront.net").hostedZoneId(),+ assertEquals("Z2FDTNDATAQYW2", CloudFrontEdgeDomain.create("aws", "cloudfront.net").hostedZoneId());+ assertEquals("Z3RFFRIM2A3IF5", CloudFrontEdgeDomain.create("aws-cn", "cloudfront.net").hostedZoneId());+ assertNull(CloudFrontEdgeDomain.create("aws-us-gov", "cloudfront.net").hostedZoneId(),"GovCloud has no CloudFront");}}diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.javaindex c12c0f797f..be31cc3503 100644--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java@@ -11,6 +11,7 @@ import org.junit.jupiter.api.Test;import java.util.List;import java.util.Map;+import java.util.Optional;import java.util.concurrent.atomic.AtomicBoolean;import static org.junit.jupiter.api.Assertions.assertEquals;@@ -46,7 +47,7 @@ class ApiGatewayDomainTlsServiceTest {when(storageFactory.create(anyString(), anyString(), any())).thenAnswer(invocation -> AccountAwareStorageBackend.inMemory("000000000000"));certificateManager = mock(TlsCertificateManager.class);EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());service = new ApiGatewayService(storageFactory, config, certificateManager,new RegionResolver(REGION, "000000000000"));}diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.javaindex 0f15ed6c3a..068eee72ea 100644--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java@@ -13,6 +13,7 @@ import org.junit.jupiter.api.Test;import java.util.HashMap;import java.util.List;import java.util.Map;+import java.util.Optional;import static org.junit.jupiter.api.Assertions.assertEquals;import static org.junit.jupiter.api.Assertions.assertThrows;@@ -41,7 +42,7 @@ class ApiGatewayEdgeDomainPartitionTest {stores.computeIfAbsent(invocation.getArgument(1, String.class),ignored -> AccountAwareStorageBackend.inMemory("000000000000")));EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class),new RegionResolver(GOV_REGION, "000000000000"));}diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.javaindex bb292d2506..eb153be7ab 100644--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java@@ -16,6 +16,7 @@ import org.junit.jupiter.params.provider.ValueSource;import java.util.HashMap;import java.util.List;import java.util.Map;+import java.util.Optional;import static org.hamcrest.MatcherAssert.assertThat;import static org.hamcrest.Matchers.containsString;@@ -43,7 +44,7 @@ class ApiGatewayEdgeRestApiPartitionTest {stores.computeIfAbsent(invocation.getArgument(1, String.class),ignored -> AccountAwareStorageBackend.inMemory("000000000000")));EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class),new RegionResolver("us-east-1", "000000000000"));}diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayRegionalHostedZoneTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayRegionalHostedZoneTest.javanew file mode 100644index 0000000000..7144f5894c--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayRegionalHostedZoneTest.java@@ -0,0 +1,57 @@+package io.github.hectorvent.floci.services.apigateway;++import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.config.TlsCertificateManager;+import io.github.hectorvent.floci.core.common.RegionResolver;+import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;+import io.github.hectorvent.floci.core.storage.StorageFactory;+import io.github.hectorvent.floci.services.apigateway.model.CustomDomain;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.params.ParameterizedTest;+import org.junit.jupiter.params.provider.CsvSource;++import java.util.List;+import java.util.Map;+import java.util.Optional;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.mockito.ArgumentMatchers.any;+import static org.mockito.ArgumentMatchers.anyString;+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.when;++/**+ * The Route 53 hosted zone a regional custom domain reports. The GovCloud user guide states it for+ * its two regions; every other region has no source and keeps the value it always reported.+ */+class ApiGatewayRegionalHostedZoneTest {++ private ApiGatewayService service;++ @BeforeEach+ void setUp() {+ StorageFactory storageFactory = mock(StorageFactory.class);+ when(storageFactory.create(anyString(), anyString(), any()))+ .thenAnswer(invocation -> AccountAwareStorageBackend.inMemory("000000000000"));+ EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());+ service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class),+ new RegionResolver("us-east-1", "000000000000"));+ }++ @ParameterizedTest+ @CsvSource({+ "us-gov-west-1, Z1K6XKP9SAGWDV",+ "us-gov-east-1, Z3SE9ATJYCRCZJ",+ "us-east-1, Z2FDTNDATAQYL2",+ "cn-north-1, Z2FDTNDATAQYL2",+ "eusc-de-east-1, Z2FDTNDATAQYL2"})+ void regionalDomainReportsTheRegionsPublishedZone(String region, String hostedZoneId) {+ CustomDomain domain = service.createDomainName(region, Map.of(+ "domainName", "api." + region + ".example.com",+ "endpointConfiguration", Map.of("types", List.of("REGIONAL"))));++ assertEquals(hostedZoneId, domain.getRegionalHostedZoneId());+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.javaindex d036c70dd1..85e3fc43e0 100644--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java@@ -15,6 +15,7 @@ import org.junit.jupiter.api.Test;import java.util.HashMap;import java.util.List;import java.util.Map;+import java.util.Optional;import java.util.Set;import static org.junit.jupiter.api.Assertions.assertEquals;@@ -55,7 +56,7 @@ class ApiGatewayV2ApiMappingServiceTest {when(storageFactory.create(anyString(), anyString(), any())).thenAnswer(invocation -> AccountAwareStorageBackend.inMemory("000000000000"));EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());RegionResolver regionResolver = new RegionResolver(REGION, "000000000000");v2Service = new ApiGatewayV2Service(storageFactory, config, regionResolver);service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class), regionResolver,diff --git a/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.javaindex 6ca9eefc94..f207e09b2f 100644--- a/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java@@ -30,6 +30,7 @@ import java.time.ZoneOffset;import java.util.HashMap;import java.util.List;import java.util.Map;+import java.util.Optional;import static org.junit.jupiter.api.Assertions.assertEquals;import static org.junit.jupiter.api.Assertions.assertThrows;@@ -50,6 +51,8 @@ class AppSyncServiceTest {private AppSyncService service;private AccountAwareStorageBackend<ApiKey> apiKeyStoreOverride;private String region = "us-east-1";+ /** The deployment partition, or null to derive it from {@link #region}. */+ private String partition;@BeforeEachvoid setUp() {@@ -59,7 +62,8 @@ class AppSyncServiceTest {/*** AppSync fronts a custom domain with CloudFront, so a DNS alias to {@code appsyncDomainName}* lives in the partition's CloudFront hosted zone, as the CDK's AppSyncTarget builds it. A- * random zone id made every such alias point nowhere.+ * random zone id made every such alias point nowhere. With no suffix configured the+ * distribution name takes the partition's own.*/@Testvoid customDomainIsFrontedByTheCloudFrontZoneOfItsPartition() {@@ -72,6 +76,24 @@ class AppSyncServiceTest {DomainName china = newService(Clock.fixed(NOW, ZoneOffset.UTC)).createDomainName(Map.of("domainName", "api.example.cn"));assertEquals("Z3RFFRIM2A3IF5", china.getHostedZoneId());+ assertTrue(china.getAppsyncDomainName().matches("d[0-9a-f]{13}\\.cloudfront\\.cn"),+ china.getAppsyncDomainName());+ }++ /**+ * A region no partition publishes, accepted under {@code allow-unknown-regions}, belongs to the+ * deployment's partition, so a China deployment fronts the domain with China's CloudFront+ * suffix and zone rather than the commercial ones.+ */+ @Test+ void customDomainInAnUnpublishedRegionTakesTheDeploymentsPartition() {+ region = "xx-test-1";+ partition = "aws-cn";+ DomainName domain = newService(Clock.fixed(NOW, ZoneOffset.UTC))+ .createDomainName(Map.of("domainName", "api.example.cn"));++ assertEquals("Z3RFFRIM2A3IF5", domain.getHostedZoneId());+ assertTrue(domain.getAppsyncDomainName().endsWith(".cloudfront.cn"), domain.getAppsyncDomainName());}@Test@@ -406,11 +428,12 @@ class AppSyncServiceTest {lambdaAuthorizerCache = new LambdaAuthorizerCache();EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);when(config.effectiveBaseUrl()).thenReturn(baseUrl);- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());return new AppSyncService(storageFactory,config,- new RegionResolver(region, "000000000000"),+ partition == null ? new RegionResolver(region, "000000000000")+ : new RegionResolver(region, "000000000000", partition),schemaRegistry,mock(SchemaCreationWorker.class),requestContext,diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.javaindex 3eb688692a..8c591ca616 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java@@ -127,6 +127,7 @@ class CloudFrontServiceTest {return serviceWithDomainSuffix(domainSuffix, "us-east-1");}+ /** A null {@code domainSuffix} is {@code floci.services.cloudfront.domain-suffix} left unset. */private CloudFrontService serviceWithDomainSuffix(String domainSuffix, String region) {StorageFactory storageFactory = Mockito.mock(StorageFactory.class);when(storageFactory.create(Mockito.anyString(), Mockito.anyString(), Mockito.any()))@@ -143,7 +144,7 @@ class CloudFrontServiceTest {when(dnsConfig.extraSuffixes()).thenReturn(Optional.empty());when(config.services()).thenReturn(servicesConfig);when(servicesConfig.cloudfront()).thenReturn(cloudFrontConfig);- when(cloudFrontConfig.domainSuffix()).thenReturn(domainSuffix);+ when(cloudFrontConfig.domainSuffix()).thenReturn(Optional.ofNullable(domainSuffix));return new CloudFrontService(storageFactory, config, new RegionResolver(region, ACCOUNT));}@@ -323,7 +324,7 @@ class CloudFrontServiceTest {@Testvoid createDistributionUsesDefaultDomainSuffix() {- CloudFrontService service = serviceWithDomainSuffix("cloudfront.net");+ CloudFrontService service = serviceWithDomainSuffix(null);Distribution dist = service.createDistribution(new Distribution(), Map.of());@@ -364,6 +365,33 @@ class CloudFrontServiceTest {"Expected configured suffix, got: " + dist.getDomainName());}+ /**+ * China's distributions are named under {@code cloudfront.cn}; GovCloud has no CloudFront and+ * so no suffix of its own, and keeps the commercial one.+ */+ @Test+ void anUnconfiguredDomainSuffixFollowsTheRequestsPartition() {+ CloudFrontService china = serviceWithDomainSuffix(null, "cn-north-1");+ CloudFrontService govCloud = serviceWithDomainSuffix(null, "us-gov-west-1");++ Distribution dist = china.createDistribution(new Distribution(), Map.of());+ StreamingDistribution streaming = china.createStreamingDistribution(new StreamingDistribution());++ assertEquals(dist.getId().toLowerCase(Locale.ROOT) + ".cloudfront.cn", dist.getDomainName());+ assertEquals(streaming.getId().toLowerCase(Locale.ROOT) + ".cloudfront.cn", streaming.getDomainName());+ assertTrue(govCloud.createDistribution(new Distribution(), Map.of()).getDomainName().endsWith(".cloudfront.net"));+ }++ @Test+ void aConfiguredDomainSuffixWinsOverThePartitions() {+ CloudFrontService china = serviceWithDomainSuffix("cloudfront.local", "cn-north-1");++ Distribution dist = china.createDistribution(new Distribution(), Map.of());++ assertTrue(dist.getDomainName().endsWith(".cloudfront.local"),+ "Expected configured suffix, got: " + dist.getDomainName());+ }+@Testvoid createStreamingDistributionHonorsConfiguredDomainSuffix() {CloudFrontService service = serviceWithDomainSuffix("cloudfront.local");diff --git a/tools/aws/regen_region_facts.py b/tools/aws/regen_region_facts.pyindex 5381dd5d69..fb654d16f0 100644--- a/tools/aws/regen_region_facts.py+++ b/tools/aws/regen_region_facts.py@@ -1,5 +1,5 @@#!/usr/bin/env python3-"""Regenerate src/main/resources/aws/region-facts.json from the AWS CDK, Terraform and user-guide sources.+"""Regenerate src/main/resources/aws/region-facts.json from the AWS CDK, Terraform, botocore and user-guide sources.AWS publishes a handful of per-region and per-partition constants only as prose tables (theGeneral Reference), never as SDK data. The two open-source projects that transcribe them, and the@@ -31,6 +31,19 @@ China and GovCloud user guides for what neither transcribes, are the sources herEither rendering parses: the page's text (what the `aws-refs` skill's docs driver extracts fromthe `.html`), or the Markdown AWS serves at the same URL with `.md` in place of `.html`, whichwrites the China host as a link.+- The suffix of a CloudFront distribution's domain name: `cloudfront.net` from the example the+ botocore CloudFront model gives for `Distribution$DomainName` and+ `DistributionSummary$DomainName`, and `cloudfront.cn` for `aws-cn` from the China guide's+ CloudFront page (`CLOUDFRONT_SUFFIX_SENTENCES`),+ https://docs.amazonaws.cn/en_us/aws/latest/userguide/cloudfront.html -> cn/cloudfront.md, whose+ Markdown rendering escapes the asterisk (`\\*.cloudfront.cn`). The other partitions have no+ CloudFront.+- The Route 53 hosted zone of an API Gateway regional endpoint, for the two GovCloud regions only:+ the GovCloud guide's API Gateway page (`APIGATEWAY_REGIONAL_ZONE_SENTENCE`),+ https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/govcloud-abp.html -> gov/govcloud-abp.md,+ with the region+ resolved through the region descriptions in `partitions.json`. No source in `local/aws` gives+ the zone for any other region.Nothing is hand-typed. When the checkouts are absent (CI), `--check` only verifies the vendoredfile parses and keeps its shape; a regeneration needs `local/aws`.@@ -53,6 +66,7 @@ PARTITIONS = REPO_ROOT / "src/main/resources/aws/partitions.json"LOCAL_TERRAFORM = REPO_ROOT / "local/aws/terraform-provider-aws"LOCAL_CDK = REPO_ROOT / "local/aws/aws-cdk"LOCAL_GUIDES = REPO_ROOT / "local/aws/aws-docs/partition-guides"+LOCAL_BOTOCORE_DATA = REPO_ROOT / "local/aws/botocore/botocore/data"TF_ELB = "internal/service/elb/hosted_zone_id_data_source.go"TF_ELBV2 = "internal/service/elbv2/hosted_zone_id_data_source.go"@@ -61,6 +75,7 @@ TF_CLIENT = "internal/conns/awsclient.go"CDK_FACT_TABLES = "packages/aws-cdk-lib/region-info/build-tools/fact-tables.ts"CDK_CLOUDFRONT_TARGET = "packages/aws-cdk-lib/aws-route53-targets/lib/cloudfront-target.ts"CDK_VPC_ENDPOINT = "packages/aws-cdk-lib/aws-ec2/lib/vpc-endpoint.ts"+BOTOCORE_CLOUDFRONT = "cloudfront/2020-05-31/service-2.json"GO_ROW_RE = re.compile(r'^\s*endpoints\.([A-Za-z0-9]+)RegionID:\s*"([A-Z0-9]+)",')TS_ROW_RE = re.compile(r"^\s*'([a-z0-9-]+)':\s*'([A-Z0-9]+)',")@@ -86,6 +101,24 @@ CONSOLE_HOST_SENTENCES = {"aws-us-gov": ("gov/configure-account.md", re.compile(r"Sign in to the AWS GovCloud \(US\) console and open the IAM console at https://([a-z0-9.-]+)/iam")),}+# The commercial suffix is the one in the example distribution name of these two model members.+BOTOCORE_CLOUDFRONT_DOMAIN_SHAPES = ("Distribution", "DistributionSummary")+BOTOCORE_CLOUDFRONT_EXAMPLE_RE = re.compile(r"<code>d[0-9a-z]+\.(cloudfront\.[a-z]+)</code>")+CLOUDFRONT_SUFFIX_SENTENCES = {+ "aws-cn": ("cn/cloudfront.md", re.compile(+ r"You can.t use the default CloudFront domain, \\?\*\.(cloudfront\.[a-z]+), to serve content\.")),+}+# One sentence per region, each naming the region by the description partitions.json carries. The+# page writes "Route 53" with a no-break space.+APIGATEWAY_REGIONAL_ZONE_PAGE = "gov/govcloud-abp.md"+# The page states a zone for every region of this partition, so a run that finds fewer is refused.+APIGATEWAY_REGIONAL_ZONE_PARTITION = "aws-us-gov"+APIGATEWAY_REGIONAL_ZONE_SENTENCE = re.compile(+ r"The Amazon Route\s53 Hosted Zone ID for the regional endpoint in the (AWS GovCloud \([A-Za-z-]+\)) Region"+ r" is (Z[0-9A-Z]+)\.")+GUIDE_PAGES = sorted({page for page, _ in CONSOLE_HOST_SENTENCES.values()}+ | {page for page, _ in CLOUDFRONT_SUFFIX_SENTENCES.values()}+ | {APIGATEWAY_REGIONAL_ZONE_PAGE})GO_PARTITION_IDS = {"AwsCn": "aws-cn", "AwsUsGov": "aws-us-gov", "AwsIso": "aws-iso", "AwsIsoB": "aws-iso-b","AwsIsoE": "aws-iso-e", "AwsIsoF": "aws-iso-f", "Aws": "aws"}@@ -176,28 +209,79 @@ def parse_vpc_endpoint_exceptions(text: str) -> tuple[dict[str, list[str]], dictreturn services, prefixes, cn_suffix-def parse_console_host(text: str, sentence: re.Pattern[str], page: str) -> str:- """The one host `sentence` states in a guide page; a page that lost the sentence, or states two- different hosts with it, is an error rather than a guess."""- hosts = sorted({match.group(1).rstrip(".") for match in sentence.finditer(text)})- if len(hosts) != 1:- raise ValueError(f"{page}: expected one console host from /{sentence.pattern}/, found {hosts}")- return hosts[0]+def parse_guide_sentence(text: str, sentence: re.Pattern[str], page: str) -> str:+ """The one value `sentence` states in a guide page; a page that lost the sentence, or states two+ different values with it, is an error rather than a guess."""+ values = sorted({match.group(1).rstrip(".") for match in sentence.finditer(text)})+ if len(values) != 1:+ raise ValueError(f"{page}: expected one value from /{sentence.pattern}/, found {values}")+ return values[0]+++def parse_guide_sentences(guides: Path, sentences: dict[str, tuple[str, re.Pattern[str]]]) -> dict[str, str]:+ return {partition: parse_guide_sentence((guides / page).read_text(encoding="utf-8"), sentence, page)+ for partition, (page, sentence) in sentences.items()}def parse_console_hosts(guides: Path) -> dict[str, str]:- return {partition: parse_console_host((guides / page).read_text(encoding="utf-8"), sentence, page)- for partition, (page, sentence) in CONSOLE_HOST_SENTENCES.items()}+ return parse_guide_sentences(guides, CONSOLE_HOST_SENTENCES)+++def parse_botocore_cloudfront_suffix(text: str) -> str:+ """The suffix of the example distribution name in the model's `DomainName` documentation."""+ shapes = json.loads(text)["shapes"]+ suffixes = set()+ for shape in BOTOCORE_CLOUDFRONT_DOMAIN_SHAPES:+ documentation = shapes[shape]["members"]["DomainName"].get("documentation", "")+ found = set(BOTOCORE_CLOUDFRONT_EXAMPLE_RE.findall(documentation))+ if len(found) != 1:+ raise ValueError(f"cloudfront model: {shape}$DomainName gives no single example distribution name: {found}")+ suffixes |= found+ if len(suffixes) != 1:+ raise ValueError(f"cloudfront model: the DomainName examples disagree on the suffix: {sorted(suffixes)}")+ return suffixes.pop()+++def parse_cloudfront_suffixes(botocore_data: Path, guides: Path) -> dict[str, str]:+ suffixes = {"aws": parse_botocore_cloudfront_suffix((botocore_data / BOTOCORE_CLOUDFRONT).read_text(encoding="utf-8"))}+ suffixes.update(parse_guide_sentences(guides, CLOUDFRONT_SUFFIX_SENTENCES))+ return suffixes+++def parse_apigateway_regional_zones(text: str, partitions_doc: dict) -> dict[str, str]:+ """region id -> hosted zone for every GovCloud region, from the GovCloud API Gateway page."""+ by_description = {r["description"]: r["id"] for p in partitions_doc["partitions"] for r in p["regions"]+ if r.get("description")}+ zones: dict[str, str] = {}+ for description, zone in APIGATEWAY_REGIONAL_ZONE_SENTENCE.findall(text):+ region = by_description.get(description)+ if region is None:+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: {description!r} is not a region description "+ "in partitions.json")+ if zones.setdefault(region, zone) != zone:+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: two regional hosted zones for {region}")+ if not zones:+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: no regional endpoint hosted zone sentence found")+ missing = sorted(apigateway_zone_regions(partitions_doc) - zones.keys())+ if missing:+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: no regional endpoint hosted zone for {', '.join(missing)}")+ return zones+++def apigateway_zone_regions(partitions_doc: dict) -> set[str]:+ """The regions the GovCloud API Gateway page must state a regional hosted zone for."""+ return {region["id"] for partition in partitions_doc["partitions"]+ if partition["id"] == APIGATEWAY_REGIONAL_ZONE_PARTITION for region in partition["regions"]}def guides_present(guides: Path) -> bool:- return all((guides / page).exists() for page, _ in CONSOLE_HOST_SENTENCES.values())+ return all((guides / page).exists() for page in GUIDE_PAGES)# --------------------------------------------------------------------------- ## Generation# --------------------------------------------------------------------------- #-def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dict:+def build(terraform: Path, cdk: Path, guides: Path, botocore_data: Path, partitions_doc: dict) -> dict:elb = parse_go_map((terraform / TF_ELB).read_text(encoding="utf-8"), "hostedZoneIDPerRegionMap")elbv2_text = (terraform / TF_ELBV2).read_text(encoding="utf-8")alb = parse_go_map(elbv2_text, "hostedZoneIDPerRegionALBMap")@@ -216,6 +300,9 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dicif elb != alb:raise ValueError("the Classic and ALB hosted-zone maps differ; the generator assumed they are one table")console_hosts = parse_console_hosts(guides)+ cloudfront_suffixes = parse_cloudfront_suffixes(botocore_data, guides)+ apigateway_zones = parse_apigateway_regional_zones(+ (guides / APIGATEWAY_REGIONAL_ZONE_PAGE).read_text(encoding="utf-8"), partitions_doc)partition_ids = [p["id"] for p in partitions_doc["partitions"]]region_partition = {r["id"]: p for p in partitions_doc["partitions"] for r in p["regions"]}@@ -223,7 +310,7 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dics3_website = dict(s3_cdk)s3_website.update(s3_tf) # Terraform cites the website-endpoint table; it wins on conflicts- region_ids = sorted(set(alb) | set(nlb) | set(s3_website) | set(vpce_services))+ region_ids = sorted(set(alb) | set(nlb) | set(s3_website) | set(vpce_services) | set(apigateway_zones))regions = {}for region in region_ids:if region not in region_partition:@@ -241,6 +328,8 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dicentry["vpcEndpointPrefixServices"] = vpce_services[region]if region in vpce_cn_suffix:entry["vpcEndpointCnSuffixServices"] = vpce_cn_suffix[region]+ if region in apigateway_zones:+ entry["apiGatewayRegionalHostedZoneId"] = apigateway_zones[region]regions[region] = entrypartitions = {}@@ -252,6 +341,8 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dicentry["samlSignOnUrl"] = saml[partition_id]if partition_id in console_hosts:entry["consoleHost"] = console_hosts[partition_id]+ if partition_id in cloudfront_suffixes:+ entry["cloudfrontDomainSuffix"] = cloudfront_suffixes[partition_id]partitions[partition_id] = entryreturn {@@ -259,8 +350,8 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dic"generator": "tools/aws/regen_region_facts.py","terraform": f"terraform-provider-aws {TF_ELB}, {TF_ELBV2}, {TF_S3}, {TF_CLIENT}","cdk": f"aws-cdk {CDK_FACT_TABLES}, {CDK_CLOUDFRONT_TARGET}, {CDK_VPC_ENDPOINT}",- "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) "- + ", ".join(sorted(page for page, _ in CONSOLE_HOST_SENTENCES.values())),+ "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) " + ", ".join(GUIDE_PAGES),+ "botocore": f"botocore {BOTOCORE_CLOUDFRONT} (Distribution$DomainName, DistributionSummary$DomainName)","s3WebsiteConflicts": "terraform wins over the CDK where they disagree (it cites the S3 website-endpoint table)",},"partitions": partitions,@@ -285,6 +376,7 @@ SAML_URL_RE = re.compile(r"https://[a-z0-9.-]+/saml")VPCE_PREFIX_RE = re.compile(r"[a-z0-9-]+(\.[a-z0-9-]+)*\.vpce")REVERSED_DNS_RE = re.compile(r"[a-z0-9-]+(\.[a-z0-9-]+)+")CONSOLE_HOST_RE = re.compile(r"console(\.[a-z0-9-]+){2,}")+CLOUDFRONT_SUFFIX_RE = re.compile(r"cloudfront\.[a-z]+")def self_check(path: Path, partitions_path: Path = PARTITIONS) -> list[str]:@@ -322,6 +414,11 @@ def self_check(path: Path, partitions_path: Path = PARTITIONS) -> list[str]:problems.append(f"{path}: {partition['id']}.vpcEndpointServiceNamePrefix should be {expected!r}")if partition["id"] in CONSOLE_HOST_SENTENCES and "consoleHost" not in entry:problems.append(f"{path}: {partition['id']}.consoleHost is missing")+ if partition["id"] in {"aws"} | CLOUDFRONT_SUFFIX_SENTENCES.keys() and "cloudfrontDomainSuffix" not in entry:+ problems.append(f"{path}: {partition['id']}.cloudfrontDomainSuffix is missing")+ for region in sorted(apigateway_zone_regions({"partitions": partitions_doc})):+ if "apiGatewayRegionalHostedZoneId" not in document["regions"].get(region, {}):+ problems.append(f"{path}: {region}.apiGatewayRegionalHostedZoneId is missing")known = {region["id"] for partition in partitions_doc for region in partition["regions"]}for region in document["regions"]:if region not in known:@@ -330,7 +427,8 @@ def self_check(path: Path, partitions_path: Path = PARTITIONS) -> list[str]:for partition, entry in document["partitions"].items():for key, value in entry.items():pattern = {"cloudfrontHostedZoneId": HOSTED_ZONE_ID_RE, "samlSignOnUrl": SAML_URL_RE,- "vpcEndpointServiceNamePrefix": VPCE_PREFIX_RE, "consoleHost": CONSOLE_HOST_RE}.get(key)+ "vpcEndpointServiceNamePrefix": VPCE_PREFIX_RE, "consoleHost": CONSOLE_HOST_RE,+ "cloudfrontDomainSuffix": CLOUDFRONT_SUFFIX_RE}.get(key)if pattern is None:problems.append(f"{path}: {partition}.{key} is not a known field")elif not isinstance(value, str) or not pattern.fullmatch(value):@@ -357,13 +455,14 @@ def main(argv: list[str] | None = None) -> int:parser.add_argument("--terraform", type=Path, default=LOCAL_TERRAFORM, help="terraform-provider-aws checkout")parser.add_argument("--cdk", type=Path, default=LOCAL_CDK, help="aws-cdk checkout")parser.add_argument("--guides", type=Path, default=LOCAL_GUIDES, help="aws-docs/partition-guides snapshots")+ parser.add_argument("--botocore-data", type=Path, default=LOCAL_BOTOCORE_DATA, help="botocore data directory")parser.add_argument("--partitions", type=Path, default=PARTITIONS, help=argparse.SUPPRESS)parser.add_argument("--output", type=Path, default=OUTPUT, help=argparse.SUPPRESS)args = parser.parse_args(argv)rel = args.output.relative_to(REPO_ROOT) if args.output.is_relative_to(REPO_ROOT) else args.outputsources_present = ((args.terraform / TF_ELB).exists() and (args.cdk / CDK_FACT_TABLES).exists()- and guides_present(args.guides))+ and guides_present(args.guides) and (args.botocore_data / BOTOCORE_CLOUDFRONT).exists())if not sources_present:if args.check:problems = self_check(args.output, args.partitions)@@ -373,10 +472,11 @@ def main(argv: list[str] | None = None) -> int:return 1print(f"{rel}: sources not checked out, shape verified only")return 0- raise SystemExit("error: regenerating needs the terraform-provider-aws and aws-cdk checkouts and the "- "aws-docs/partition-guides snapshots under local/aws")+ raise SystemExit("error: regenerating needs the terraform-provider-aws, aws-cdk and botocore checkouts "+ "and the aws-docs/partition-guides snapshots under local/aws")- document = build(args.terraform, args.cdk, args.guides, json.loads(args.partitions.read_text(encoding="utf-8")))+ document = build(args.terraform, args.cdk, args.guides, args.botocore_data,+ json.loads(args.partitions.read_text(encoding="utf-8")))rendered = render(document)if args.check:current = args.output.read_text(encoding="utf-8") if args.output.exists() else ""diff --git a/tools/aws/test_regen_region_facts.py b/tools/aws/test_regen_region_facts.pyindex 2893b4d38a..a718cb2140 100644--- a/tools/aws/test_regen_region_facts.py+++ b/tools/aws/test_regen_region_facts.py@@ -120,26 +120,47 @@ GUIDE_CHINA_MARKDOWN = """The Amazon Web Services Management Console is available in both Chinese and English and can be accessed at [https://console.amazonaws.cn](https://console.amazonaws.cn). The Amazon Web Services Management Console is dedicated to the China Regions."""GUIDES = {"aws": GUIDE_COMMERCIAL, "aws-cn": GUIDE_CHINA, "aws-us-gov": GUIDE_GOVCLOUD}+GUIDE_CHINA_CLOUDFRONT = """+You can\u2019t use the default CloudFront domain, *.cloudfront.cn, to serve content. You must add an alternate domain name.+"""+GUIDE_GOVCLOUD_APIGATEWAY = (+ "Amazon API Gateway edge-optimized API and edge-optimized custom domain name are not available.\n\n"+ "The Amazon Route\u00a053 Hosted Zone ID for the regional endpoint in the AWS GovCloud (US-West) Region is"+ " Z1K6XKP9SAGWDV. The Amazon Route\u00a053 Hosted Zone ID for the regional endpoint in the AWS GovCloud (US-East)"+ " Region is Z3SE9ATJYCRCZJ.\n")+BOTOCORE_CLOUDFRONT = json.dumps({"shapes": {+ "Distribution": {"members": {"DomainName": {"shape": "string", "documentation":+ "<p>The distribution's CloudFront domain name. For example: <code>d111111abcdef8.cloudfront.net</code>.</p>"}}},+ "DistributionSummary": {"members": {"DomainName": {"shape": "string", "documentation":+ "<p>The domain name that corresponds to the distribution, for example,"+ " <code>d111111abcdef8.cloudfront.net</code>.</p>"}}},+}})PARTITIONS = {"partitions": [{"id": "aws", "dnsSuffix": "amazonaws.com", "regions": [{"id": "us-east-1"}, {"id": "af-south-1"}]},{"id": "aws-cn", "dnsSuffix": "amazonaws.com.cn", "regions": [{"id": "cn-north-1"}, {"id": "cn-northwest-1"}]},- {"id": "aws-us-gov", "dnsSuffix": "amazonaws.com", "regions": [{"id": "us-gov-east-1"}]},+ {"id": "aws-us-gov", "dnsSuffix": "amazonaws.com", "regions": [+ {"id": "us-gov-east-1", "description": "AWS GovCloud (US-East)"},+ {"id": "us-gov-west-1", "description": "AWS GovCloud (US-West)"}]},{"id": "aws-iso-f", "dnsSuffix": "csp.hci.ic.gov", "regions": [{"id": "us-isof-south-1"}]},]}-def write_sources(tmp_path: Path) -> tuple[Path, Path, Path]:+def write_sources(tmp_path: Path) -> tuple[Path, Path, Path, Path]:tf = tmp_path / "terraform"cdk = tmp_path / "cdk"guides = tmp_path / "guides"+ botocore = tmp_path / "botocore"for root, rel, body in [(tf, r.TF_ELB, GO_ELB), (tf, r.TF_ELBV2, GO_ELBV2), (tf, r.TF_S3, GO_S3), (tf, r.TF_CLIENT, GO_CLIENT),(cdk, r.CDK_FACT_TABLES, TS_FACTS), (cdk, r.CDK_CLOUDFRONT_TARGET, TS_CLOUDFRONT), (cdk, r.CDK_VPC_ENDPOINT, TS_VPCE),+ (guides, r.CLOUDFRONT_SUFFIX_SENTENCES["aws-cn"][0], GUIDE_CHINA_CLOUDFRONT),+ (guides, r.APIGATEWAY_REGIONAL_ZONE_PAGE, GUIDE_GOVCLOUD_APIGATEWAY),+ (botocore, r.BOTOCORE_CLOUDFRONT, BOTOCORE_CLOUDFRONT),] + [(guides, r.CONSOLE_HOST_SENTENCES[partition][0], body) for partition, body in GUIDES.items()]:path = root / relpath.parent.mkdir(parents=True, exist_ok=True)- path.write_text(body)- return tf, cdk, guides+ path.write_text(body, encoding="utf-8")+ return tf, cdk, guides, botocore@pytest.mark.parametrize("camel,region", [@@ -179,35 +200,73 @@ def test_vpc_endpoint_exceptions_are_per_region_and_service():def test_console_hosts_come_from_the_guide_sentences(tmp_path):- _, _, guides = write_sources(tmp_path)+ _, _, guides, _ = write_sources(tmp_path)assert r.parse_console_hosts(guides) == {"aws": "console.aws.amazon.com", "aws-cn": "console.amazonaws.cn","aws-us-gov": "console.amazonaws-us-gov.com"}assert r.guides_present(guides)assert not r.guides_present(tmp_path / "nowhere")+def test_the_china_cloudfront_suffix_parses_from_the_markdown_escaped_form():+ page, sentence = r.CLOUDFRONT_SUFFIX_SENTENCES["aws-cn"]+ markdown = GUIDE_CHINA_CLOUDFRONT.replace("*.cloudfront.cn", "\\*.cloudfront.cn")+ assert "\\*.cloudfront.cn" in markdown+ assert r.parse_guide_sentence(markdown, sentence, page) == "cloudfront.cn"++def test_the_china_console_host_parses_from_the_markdown_link_form():page, sentence = r.CONSOLE_HOST_SENTENCES["aws-cn"]- assert r.parse_console_host(GUIDE_CHINA_MARKDOWN, sentence, page) == "console.amazonaws.cn"+ assert r.parse_guide_sentence(GUIDE_CHINA_MARKDOWN, sentence, page) == "console.amazonaws.cn"def test_a_guide_page_without_its_sentence_or_with_two_hosts_is_an_error():page, sentence = r.CONSOLE_HOST_SENTENCES["aws-us-gov"]with pytest.raises(ValueError, match="configure-account"):- r.parse_console_host("Sign in to the AWS GovCloud (US) console.", sentence, page)+ r.parse_guide_sentence("Sign in to the AWS GovCloud (US) console.", sentence, page)with pytest.raises(ValueError, match="console.example.com"):- r.parse_console_host(GUIDE_GOVCLOUD + GUIDE_GOVCLOUD.replace("console.amazonaws-us-gov.com", "console.example.com"),+ r.parse_guide_sentence(GUIDE_GOVCLOUD + GUIDE_GOVCLOUD.replace("console.amazonaws-us-gov.com", "console.example.com"),sentence, page)# the China guide's other spelling sits in a different sentence and is not picked uppage, sentence = r.CONSOLE_HOST_SENTENCES["aws-cn"]with pytest.raises(ValueError, match="console.md"):- r.parse_console_host("a graphical user interface that you access online at https://console.www.amazonaws.cn .",+ r.parse_guide_sentence("a graphical user interface that you access online at https://console.www.amazonaws.cn .",sentence, page)+def test_cloudfront_suffixes_come_from_the_model_example_and_the_china_guide(tmp_path):+ _, _, guides, botocore = write_sources(tmp_path)+ assert r.parse_cloudfront_suffixes(botocore, guides) == {"aws": "cloudfront.net", "aws-cn": "cloudfront.cn"}+ with pytest.raises(ValueError, match="disagree"):+ r.parse_botocore_cloudfront_suffix(BOTOCORE_CLOUDFRONT.replace("cloudfront.net", "cloudfront.example", 1))+ with pytest.raises(ValueError, match="DistributionSummary"):+ r.parse_botocore_cloudfront_suffix(BOTOCORE_CLOUDFRONT.replace(+ "for example, <code>d111111abcdef8.cloudfront.net</code>", "a CloudFront name"))+ page, sentence = r.CLOUDFRONT_SUFFIX_SENTENCES["aws-cn"]+ with pytest.raises(ValueError, match="cloudfront.md"):+ r.parse_guide_sentence("CloudFront is available in the China Regions.", sentence, page)+++def test_api_gateway_regional_zones_resolve_the_region_by_its_description():+ assert r.parse_apigateway_regional_zones(GUIDE_GOVCLOUD_APIGATEWAY, PARTITIONS) == {+ "us-gov-west-1": "Z1K6XKP9SAGWDV", "us-gov-east-1": "Z3SE9ATJYCRCZJ"}+ with pytest.raises(ValueError, match="no regional endpoint hosted zone"):+ r.parse_apigateway_regional_zones("Portals are not available.", PARTITIONS)+ with pytest.raises(ValueError, match="not a region description"):+ r.parse_apigateway_regional_zones(GUIDE_GOVCLOUD_APIGATEWAY.replace("(US-West)", "(US-North)"), PARTITIONS)+ with pytest.raises(ValueError, match="two regional hosted zones for us-gov-east-1"):+ r.parse_apigateway_regional_zones(GUIDE_GOVCLOUD_APIGATEWAY.replace("(US-West)", "(US-East)"), PARTITIONS)+ # a page that loses one region's sentence is refused rather than dropping that zone+ west_only = GUIDE_GOVCLOUD_APIGATEWAY.replace(+ " The Amazon Route\u00a053 Hosted Zone ID for the regional endpoint in the AWS GovCloud (US-East)"+ " Region is Z3SE9ATJYCRCZJ.", "")+ assert "(US-West)" in west_only and "(US-East)" not in west_only+ with pytest.raises(ValueError, match="no regional endpoint hosted zone for us-gov-east-1"):+ r.parse_apigateway_regional_zones(west_only, PARTITIONS)++def test_build_merges_sources_with_terraform_winning_the_s3_conflicts(tmp_path):- tf, cdk, guides = write_sources(tmp_path)- document = r.build(tf, cdk, guides, PARTITIONS)+ tf, cdk, guides, botocore = write_sources(tmp_path)+ document = r.build(tf, cdk, guides, botocore, PARTITIONS)regions = document["regions"]assert regions["us-east-1"] == {"albHostedZoneId": "Z35SXDOTRQ7X7K", "classicElbHostedZoneId": "Z35SXDOTRQ7X7K","nlbHostedZoneId": "Z26RNL4JYFTOTI", "s3WebsiteHostedZoneId": "Z3AQBSTGFYJSTF"}@@ -221,24 +280,30 @@ def test_build_merges_sources_with_terraform_winning_the_s3_conflicts(tmp_path):assert partitions["aws-cn"] == {"vpcEndpointServiceNamePrefix": "cn.com.amazonaws.vpce","cloudfrontHostedZoneId": "Z3RFFRIM2A3IF5","samlSignOnUrl": "https://signin.amazonaws.cn/saml",- "consoleHost": "console.amazonaws.cn"}- assert partitions["aws-us-gov"]["consoleHost"] == "console.amazonaws-us-gov.com"+ "consoleHost": "console.amazonaws.cn",+ "cloudfrontDomainSuffix": "cloudfront.cn"}+ assert partitions["aws"]["cloudfrontDomainSuffix"] == "cloudfront.net"+ assert partitions["aws-us-gov"] == {"vpcEndpointServiceNamePrefix": "com.amazonaws.vpce",+ "consoleHost": "console.amazonaws-us-gov.com"}+ assert regions["us-gov-west-1"] == {"apiGatewayRegionalHostedZoneId": "Z1K6XKP9SAGWDV"}+ assert regions["us-gov-east-1"]["apiGatewayRegionalHostedZoneId"] == "Z3SE9ATJYCRCZJ"+ assert "apiGatewayRegionalHostedZoneId" not in regions["us-east-1"]assert partitions["aws-iso-f"] == {"vpcEndpointServiceNamePrefix": "gov.ic.hci.csp.vpce"}def test_build_refuses_a_region_the_partition_data_does_not_publish(tmp_path):- tf, cdk, guides = write_sources(tmp_path)+ tf, cdk, guides, botocore = write_sources(tmp_path)with pytest.raises(ValueError, match="af-south-1"):- r.build(tf, cdk, guides, {"partitions": [p for p in PARTITIONS["partitions"] if p["id"] != "aws"]+ r.build(tf, cdk, guides, botocore, {"partitions": [p for p in PARTITIONS["partitions"] if p["id"] != "aws"]+ [{"id": "aws", "dnsSuffix": "amazonaws.com", "regions": [{"id": "us-east-1"}]}]})def test_cli_write_check_and_self_check(tmp_path, capsys):- tf, cdk, guides = write_sources(tmp_path)+ tf, cdk, guides, botocore = write_sources(tmp_path)partitions = tmp_path / "partitions.json"partitions.write_text(json.dumps(PARTITIONS))output = tmp_path / "out" / "region-facts.json"- common = ["--terraform", str(tf), "--cdk", str(cdk), "--guides", str(guides),+ common = ["--terraform", str(tf), "--cdk", str(cdk), "--guides", str(guides), "--botocore-data", str(botocore),"--partitions", str(partitions), "--output", str(output)]assert r.main(["--check"] + common) == 1assert r.main(common) == 0@@ -248,10 +313,12 @@ def test_cli_write_check_and_self_check(tmp_path, capsys):assert r.main(["--check", "--terraform", str(tmp_path / "nowhere"), "--cdk", str(tmp_path / "nowhere"),"--partitions", str(partitions), "--output", str(output)]) == 0assert "shape verified only" in capsys.readouterr().out- # nor with the checkouts but without the guide snapshots- assert r.main(["--check", "--terraform", str(tf), "--cdk", str(cdk), "--guides", str(tmp_path / "nowhere"),- "--partitions", str(partitions), "--output", str(output)]) == 0- assert "shape verified only" in capsys.readouterr().out+ # nor with the checkouts but without the guide snapshots, or without the botocore model+ for absent in ("--guides", "--botocore-data"):+ partial = [str(tmp_path / "nowhere") if previous == absent else argument+ for previous, argument in zip([""] + common, common)]+ assert r.main(["--check"] + partial) == 0, absent+ assert "shape verified only" in capsys.readouterr().outoutput.write_text('{"partitions": {}, "regions": {"us-east-1": {"albHostedZoneId": "nope"}}}')assert r.main(["--check", "--terraform", str(tmp_path / "nowhere"), "--cdk", str(tmp_path / "nowhere"),"--partitions", str(partitions), "--output", str(output)]) == 1@@ -259,17 +326,18 @@ def test_cli_write_check_and_self_check(tmp_path, capsys):def test_repo_file_matches_the_local_checkouts_when_present():if (not (r.LOCAL_TERRAFORM / r.TF_ELB).exists() or not (r.LOCAL_CDK / r.CDK_FACT_TABLES).exists()- or not r.guides_present(r.LOCAL_GUIDES)):+ or not r.guides_present(r.LOCAL_GUIDES)+ or not (r.LOCAL_BOTOCORE_DATA / r.BOTOCORE_CLOUDFRONT).exists()):pytest.skip("local/aws checkouts not present")- document = r.build(r.LOCAL_TERRAFORM, r.LOCAL_CDK, r.LOCAL_GUIDES, json.loads(r.PARTITIONS.read_text(encoding="utf-8")))+ document = r.build(r.LOCAL_TERRAFORM, r.LOCAL_CDK, r.LOCAL_GUIDES, r.LOCAL_BOTOCORE_DATA, json.loads(r.PARTITIONS.read_text(encoding="utf-8")))assert r.strip_source(r.render(document)) == r.strip_source(r.OUTPUT.read_text(encoding="utf-8"))def test_self_check_holds_the_values_to_what_partitions_json_publishes(tmp_path):partitions = tmp_path / "partitions.json"partitions.write_text(json.dumps(PARTITIONS))- tf, cdk, guides = write_sources(tmp_path)- good = r.build(tf, cdk, guides, PARTITIONS)+ tf, cdk, guides, botocore = write_sources(tmp_path)+ good = r.build(tf, cdk, guides, botocore, PARTITIONS)output = tmp_path / "region-facts.json"def problems(mutate):@@ -298,6 +366,16 @@ def test_self_check_holds_the_values_to_what_partitions_json_publishes(tmp_path)"missing China console host": lambda d: d["partitions"]["aws-cn"].pop("consoleHost"),"missing GovCloud console host": lambda d: d["partitions"]["aws-us-gov"].pop("consoleHost"),"missing commercial console host": lambda d: d["partitions"]["aws"].pop("consoleHost"),+ "missing commercial CloudFront suffix": lambda d: d["partitions"]["aws"].pop("cloudfrontDomainSuffix"),+ "missing China CloudFront suffix": lambda d: d["partitions"]["aws-cn"].pop("cloudfrontDomainSuffix"),+ "missing us-gov-west-1 API Gateway zone": lambda d: d["regions"]["us-gov-west-1"].pop(+ "apiGatewayRegionalHostedZoneId"),+ "missing us-gov-east-1 API Gateway zone": lambda d: d["regions"]["us-gov-east-1"].pop(+ "apiGatewayRegionalHostedZoneId"),+ "CloudFront suffix that is a whole name": lambda d: d["partitions"][partition].update(+ cloudfrontDomainSuffix="d111111abcdef8.cloudfront.net"),+ "API Gateway zone that is not a zone id": lambda d: d["regions"][region].update(+ apiGatewayRegionalHostedZoneId="regional"),"missing partition": lambda d: d["partitions"].pop(partition),"partition entry not an object": lambda d: d["partitions"].update({partition: "invalid"}),"region entry not an object": lambda d: d["regions"].update({region: ["invalid"]}),