Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

fix(partitions): source the China CloudFront suffix and GovCloud API Gateway zones (#5618)

Hector Ventura3afb5b430e72
diff --git a/docs/configuration/partitions.md b/docs/configuration/partitions.md
index b84d54d878..0657efd32e 100644
--- a/docs/configuration/partitions.md
+++ b/docs/configuration/partitions.md
@@ -114,12 +114,16 @@ back to `aws` for a label no partition publishes.
ids are the same literal in every partition, as the Sign-In model spells them.
- **Hosted zones and console URLs**: load balancer and S3 website hosted zone ids come from a
per-region table (`aws/region-facts.json`, generated from the Terraform provider and the CDK,
- which transcribe the AWS General Reference); Network load balancers have their own zone,
- distinct from the Application/Classic one. The CloudFront hosted zone is published for `aws`
- and `aws-cn`; the SAML sign-on URL for five partitions. The console link in a CodePipeline
+ which transcribe the AWS General Reference, and from the China and GovCloud user guides); Network
+ load balancers have their own zone, distinct from the Application/Classic one. The CloudFront
+ hosted zone is published for `aws` and `aws-cn`; the SAML sign-on URL for five partitions. The console link in a CodePipeline
approval notification uses the console host of the pipeline's partition (`console.amazonaws.cn`
in China, `console.amazonaws-us-gov.com` in GovCloud), as the China and GovCloud user guides
- state it; the path after the host is the commercial console's. The ISO and EUSC regions have no load
+ state it; the path after the host is the commercial console's. A generated CloudFront
+ distribution domain name ends in `cloudfront.cn` in China, as the China guide names the default
+ CloudFront domain, unless `floci.services.cloudfront.domain-suffix` is set. A regional API Gateway
+ custom domain reports the regional hosted zone the GovCloud guide states for `us-gov-west-1` and
+ `us-gov-east-1`. The ISO and EUSC regions have no load
balancer hosted zones, and only the two ISO-F regions have an S3 website zone; where a table has
no row the field is omitted rather than guessed. An `EDGE` API Gateway REST API or custom domain is
refused outside the commercial partition (`BadRequestException`): GovCloud, EUSC and ISO have no
@@ -244,13 +248,13 @@ partition.
These values have no published source Floci can cite, so it does not guess them; they keep
the commercial value or Floci's own base host until sourced:
-- the China CloudFront distribution domain suffix (only the API host is published);
- the STS web-identity audience outside the commercial partition;
- the EKS Pod Identity token audience (`pods.eks.amazonaws.com`) outside the commercial partition;
- the Lambda function-URL host in the ISO and EUSC partitions (function URLs are refused in GovCloud
and China, where the partition guides list them as not available);
- whether AWS managed policy documents differ in content in China or GovCloud;
-- the API Gateway regional hosted zone per region;
+- the API Gateway regional hosted zone per region outside GovCloud: only the two GovCloud zones
+ are sourced, and every other region reports one placeholder zone id;
- the availability-zone id prefix of the ISO and EUSC regions: zone ids there keep the whole region
(`us-iso-east-1-az1`), where the other partitions get AWS's short prefix (`use1-az1`, `usgw1-az1`,
`cnn1-az1`);
diff --git a/docs/services/cloudfront.md b/docs/services/cloudfront.md
index f58b563669..7daaec200a 100644
--- a/docs/services/cloudfront.md
+++ b/docs/services/cloudfront.md
@@ -266,7 +266,7 @@ PUT, PATCH and DELETE).
| Property | Env var | Default | Description |
|---|---|---|---|
| `floci.services.cloudfront.enabled` | `FLOCI_SERVICES_CLOUDFRONT_ENABLED` | `true` | Enable or disable the service |
-| `floci.services.cloudfront.domain-suffix` | `FLOCI_SERVICES_CLOUDFRONT_DOMAIN_SUFFIX` | `cloudfront.net` | Domain suffix for generated distribution domain names |
+| `floci.services.cloudfront.domain-suffix` | `FLOCI_SERVICES_CLOUDFRONT_DOMAIN_SUFFIX` | unset | Domain suffix for generated distribution domain names. Unset, it follows the request's partition: `cloudfront.net`, or `cloudfront.cn` in China. A configured suffix applies in every partition |
| `floci.services.cloudfront.allowed-private-origin-hosts` | `FLOCI_SERVICES_CLOUDFRONT_ALLOWED_PRIVATE_ORIGIN_HOSTS` | `[]` | Exact custom-origin hosts permitted to resolve to private/non-routable addresses (comma-separated in the environment variable) |
## CLI Examples
diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
index fd4a8eec52..f7d5ffa737 100644
--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
@@ -2383,8 +2383,11 @@ public interface EmulatorConfig {
@WithDefault("true")
boolean enabled();
- @WithDefault("cloudfront.net") // partition-literal: configurable; the China CDN suffix has no published source
- String domainSuffix();
+ /**
+ * Suffix of generated distribution domain names, in every partition when set. Unset, the
+ * suffix follows the request's partition ({@code CloudFrontEdgeDomain#domainSuffix}).
+ */
+ Optional<String> domainSuffix();
/**
* Exact custom-origin hostnames allowed to resolve to private or otherwise non-routable
diff --git a/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java b/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java
index 5cde2fd447..a8c8d362b6 100644
--- a/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java
+++ b/src/main/java/io/github/hectorvent/floci/core/common/AwsRegionFacts.java
@@ -13,9 +13,10 @@ import java.util.Set;
/**
* The per-region and per-partition constants AWS publishes only as prose tables, vendored in
* {@code aws/region-facts.json} by {@code tools/aws/regen_region_facts.py} from the Terraform
- * provider, the CDK and the China and GovCloud user guides: load balancer and S3 website hosted
- * zone ids, the CloudFront hosted zone per partition, the SAML sign-on URL and the management
- * console host per partition and the VPC endpoint service-name prefixes. Every accessor is {@link Optional}: the ISO and EUSC partitions have no published
+ * provider, the CDK, the botocore models and the China and GovCloud user guides: load balancer,
+ * S3 website and GovCloud API Gateway hosted zone ids, the CloudFront hosted zone and distribution
+ * domain suffix per partition, the SAML sign-on URL and the management console host per partition
+ * and the VPC endpoint service-name prefixes. Every accessor is {@link Optional}: the ISO and EUSC partitions have no published
* hosted zones, and a service that needs one there has nothing faithful to answer.
*/
public final class AwsRegionFacts {
@@ -48,6 +49,22 @@ public final class AwsRegionFacts {
return regionText(region, "s3WebsiteHostedZoneId");
}
+ /**
+ * The hosted zone of an API Gateway regional endpoint, which a Route 53 alias to a regional
+ * custom domain targets. Published in the corpus for the two GovCloud regions only.
+ */
+ public static Optional<String> apiGatewayRegionalHostedZoneId(String region) {
+ return regionText(region, "apiGatewayRegionalHostedZoneId");
+ }
+
+ /**
+ * The suffix of a distribution's generated domain name: {@code cloudfront.net}, and
+ * {@code cloudfront.cn} in {@code aws-cn}. Absent for the partitions without CloudFront.
+ */
+ public static Optional<String> cloudFrontDomainSuffix(String partition) {
+ return partitionText(partition, "cloudfrontDomainSuffix");
+ }
+
/** Published for {@code aws} and {@code aws-cn} only, where CloudFront exists. */
public static Optional<String> cloudFrontHostedZoneId(String partition) {
return partitionText(partition, "cloudfrontHostedZoneId");
diff --git a/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java b/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java
index 6c3c7f1709..4c40ce6ebc 100644
--- a/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java
+++ b/src/main/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomain.java
@@ -4,19 +4,41 @@ import java.util.UUID;
/**
* The CloudFront distribution AWS puts in front of a custom domain, an edge-optimized API Gateway
- * domain or an AppSync custom domain: a distribution domain name on the configured CloudFront
- * suffix, and the CloudFront hosted zone of the region's partition, which a Route 53 alias record
- * targets. The zone is null in a partition without CloudFront.
+ * domain or an AppSync custom domain: a distribution domain name on the CloudFront suffix of the
+ * partition (or the configured one), and the partition's CloudFront hosted zone, which a Route 53
+ * alias record targets. The zone is null in a partition without CloudFront.
*
* @param domainName the distribution domain name, {@code d<13 hex>.<suffix>}
* @param hostedZoneId the partition's CloudFront hosted zone, or null where there is none
*/
public record CloudFrontEdgeDomain(String domainName, String hostedZoneId) {
- public static CloudFrontEdgeDomain create(String region, String domainSuffix) {
+ /**
+ * @param partition the partition the domain is created in, as the caller's
+ * {@link RegionResolver#partitionForRegion} resolves it, so a region no
+ * partition publishes falls to the deployment's partition, not the
+ * commercial one
+ * @param configuredSuffix {@code floci.services.cloudfront.domain-suffix}, or null when unset
+ */
+ public static CloudFrontEdgeDomain create(String partition, String configuredSuffix) {
String distributionId = "d" + UUID.randomUUID().toString().replace("-", "").substring(0, 13);
- String hostedZoneId = AwsRegionFacts.cloudFrontHostedZoneId(
- AwsPartitions.forRegionOrCommercial(region).id()).orElse(null);
- return new CloudFrontEdgeDomain(distributionId + "." + domainSuffix, hostedZoneId);
+ String hostedZoneId = AwsRegionFacts.cloudFrontHostedZoneId(partition).orElse(null);
+ return new CloudFrontEdgeDomain(distributionId + "." + domainSuffix(configuredSuffix, partition), hostedZoneId);
+ }
+
+ /**
+ * The suffix of a generated distribution domain name. An explicitly configured suffix wins in
+ * every partition; otherwise it is the partition's own ({@code cloudfront.cn} in China), and
+ * the commercial one in a partition that has no CloudFront and so no suffix to publish.
+ *
+ * @param configuredSuffix {@code floci.services.cloudfront.domain-suffix}, or null when unset
+ */
+ public static String domainSuffix(String configuredSuffix, String partition) {
+ if (configuredSuffix != null && !configuredSuffix.isBlank()) {
+ return configuredSuffix;
+ }
+ return AwsRegionFacts.cloudFrontDomainSuffix(partition)
+ .or(() -> AwsRegionFacts.cloudFrontDomainSuffix(AwsRegions.DEFAULT_PARTITION))
+ .orElseThrow();
}
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java b/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java
index 2349d4240b..fd13c8d2e1 100644
--- a/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayService.java
@@ -11,6 +11,7 @@ import io.github.hectorvent.floci.core.common.AwsArnUtils;
import io.github.hectorvent.floci.core.common.AwsException;
import io.github.hectorvent.floci.core.common.AwsPartition;
import io.github.hectorvent.floci.core.common.AwsPartitions;
+import io.github.hectorvent.floci.core.common.AwsRegionFacts;
import io.github.hectorvent.floci.core.common.AwsRegions;
import io.github.hectorvent.floci.core.common.CloudFrontEdgeDomain;
import io.github.hectorvent.floci.core.common.RegionResolver;
@@ -86,6 +87,12 @@ public class ApiGatewayService implements ResourceProvider {
/** The only partition where API Gateway offers edge-optimized APIs and custom domain names. */
private static final Set<String> EDGE_OPTIMIZED_PARTITIONS = Set.of("aws");
+ /**
+ * The regional hosted zone a custom domain reports in a region with no published one. Only
+ * the two GovCloud regions have a sourced zone ({@link AwsRegionFacts#apiGatewayRegionalHostedZoneId}).
+ */
+ private static final String UNSOURCED_REGIONAL_HOSTED_ZONE_ID = "Z2FDTNDATAQYL2"; // partition-literal: placeholder kept where no per-region source exists
+
private static final ObjectMapper JSON = new ObjectMapper();
/** Documented default page size for GetUsage. */
@@ -2115,7 +2122,8 @@ public class ApiGatewayService implements ResourceProvider {
domain.setRegionalCertificateName((String) request.get("regionalCertificateName"));
domain.setRegionalCertificateArn((String) request.get("regionalCertificateArn"));
domain.setRegionalDomainName(domainName + ".regional.local");
- domain.setRegionalHostedZoneId("Z2FDTNDATAQYL2"); // partition-literal: no published per-region source for the regional zone
+ domain.setRegionalHostedZoneId(AwsRegionFacts.apiGatewayRegionalHostedZoneId(region)
+ .orElse(UNSOURCED_REGIONAL_HOSTED_ZONE_ID));
applyEndpointType(domain, endpointType, region);
domain.setSecurityPolicy((String) request.getOrDefault("securityPolicy", "TLS_1_2"));
// Nothing is provisioned behind the domain, so it is usable as soon as it exists.
@@ -2168,7 +2176,8 @@ public class ApiGatewayService implements ResourceProvider {
domain.setDistributionDomainName(null);
domain.setDistributionHostedZoneId(null);
} else if (domain.getDistributionDomainName() == null) {
- CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(region, config.services().cloudfront().domainSuffix());
+ CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(regionResolver.partitionForRegion(region),
+ config.services().cloudfront().domainSuffix().orElse(null));
domain.setDistributionDomainName(edge.domainName());
domain.setDistributionHostedZoneId(edge.hostedZoneId());
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java b/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java
index 2cbbb38e5d..1267166b41 100644
--- a/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/appsync/AppSyncService.java
@@ -60,7 +60,7 @@ public class AppSyncService {
private final ObjectMapper objectMapper;
private final Clock clock;
private final String baseUrl;
- private final String cloudFrontDomainSuffix;
+ private final String configuredCloudFrontDomainSuffix;
private final LambdaAuthorizerCache lambdaAuthorizerCache;
@Inject
@@ -90,7 +90,7 @@ public class AppSyncService {
this.objectMapper = objectMapper;
this.clock = clock;
this.baseUrl = trimTrailingSlash(config.effectiveBaseUrl());
- this.cloudFrontDomainSuffix = config.services().cloudfront().domainSuffix();
+ this.configuredCloudFrontDomainSuffix = config.services().cloudfront().domainSuffix().orElse(null);
this.lambdaAuthorizerCache = lambdaAuthorizerCache;
}
@@ -761,7 +761,8 @@ public class AppSyncService {
* Gateway domain gets.
*/
private void applyCloudFrontFront(DomainName dn, String region) {
- CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(region, cloudFrontDomainSuffix);
+ CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create(regionResolver.partitionForRegion(region),
+ configuredCloudFrontDomainSuffix);
dn.setAppsyncDomainName(edge.domainName());
dn.setHostedZoneId(edge.hostedZoneId());
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java b/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java
index 1c3e790999..d0a8b83ce1 100644
--- a/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontService.java
@@ -3,6 +3,7 @@ package io.github.hectorvent.floci.services.cloudfront;
import com.fasterxml.jackson.core.type.TypeReference;
import io.github.hectorvent.floci.config.EmulatorConfig;
import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.core.common.CloudFrontEdgeDomain;
import io.github.hectorvent.floci.core.common.RegionResolver;
import io.github.hectorvent.floci.core.common.dns.EmbeddedDnsServer;
import io.github.hectorvent.floci.core.storage.StorageBackend;
@@ -126,7 +127,7 @@ public class CloudFrontService implements ServerCertificateReferenceProvider {
private final String accountId;
private final RegionResolver regionResolver;
private final IamService iamService;
- private final String domainSuffix;
+ private final String configuredDomainSuffix;
/**
* Host suffixes under which every distribution is served as {@code <id><suffix>}, whatever
* {@code floci.services.cloudfront.domain-suffix} makes the assigned domain name. They carry the
@@ -182,7 +183,7 @@ public class CloudFrontService implements ServerCertificateReferenceProvider {
new TypeReference<Map<String, MonitoringSubscription>>() {});
this.accountId = config.defaultAccountId();
this.regionResolver = regionResolver;
- this.domainSuffix = config.services().cloudfront().domainSuffix();
+ this.configuredDomainSuffix = config.services().cloudfront().domainSuffix().orElse(null);
this.localDeliverySuffixes = localDeliverySuffixes(config);
}
@@ -459,10 +460,12 @@ public class CloudFrontService implements ServerCertificateReferenceProvider {
/**
* The domain name a new distribution is served under. AWS assigns a lower-case host, and a
* browser lower-cases the authority it sends, so an upper-case id in the host would never match
- * the resource a signed URL was signed for.
+ * the resource a signed URL was signed for. The suffix is the request partition's unless one
+ * is configured.
*/
private String domainNameFor(String id) {
- return id.toLowerCase(Locale.ROOT) + "." + domainSuffix;
+ return id.toLowerCase(Locale.ROOT) + "."
+ + CloudFrontEdgeDomain.domainSuffix(configuredDomainSuffix, regionResolver.getPartition());
}
private static List<String> localDeliverySuffixes(EmulatorConfig config) {
diff --git a/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java b/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java
index c338a01cef..4ca8e5f0ee 100644
--- a/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/cognito/CognitoService.java
@@ -13,6 +13,7 @@ import io.github.hectorvent.floci.core.common.AwsPartition;
import io.github.hectorvent.floci.core.common.AwsPartitions;
import io.github.hectorvent.floci.core.common.AwsRegions;
import io.github.hectorvent.floci.core.common.Base64Url;
+import io.github.hectorvent.floci.core.common.CloudFrontEdgeDomain;
import io.github.hectorvent.floci.core.common.Digests;
import io.github.hectorvent.floci.core.common.PaginatedResult;
import io.github.hectorvent.floci.core.common.Pagination;
@@ -112,10 +113,10 @@ public class CognitoService implements ResourceProvider {
// JVM-local stripes bound lock memory without retaining one lock for every user key.
private static final int USER_LOCK_STRIPES = 512;
/**
- * The commercial CloudFront suffix for the test constructors; CDI wires
- * {@code floci.services.cloudfront.domain-suffix}.
+ * No configured CloudFront suffix, for the test constructors: the suffix then follows the
+ * request's partition. CDI wires {@code floci.services.cloudfront.domain-suffix}.
*/
- private static final String DEFAULT_CLOUDFRONT_DOMAIN_SUFFIX = "cloudfront.net"; // partition-literal: test-shaped constructor default
+ private static final String UNCONFIGURED_CLOUDFRONT_DOMAIN_SUFFIX = null;
private static final Logger LOG = Logger.getLogger(CognitoService.class);
private static final ObjectMapper MAPPER = new ObjectMapper();
@@ -219,7 +220,7 @@ public class CognitoService implements ResourceProvider {
storageFactory.create("cognito", "cognito-revoked-tokens.json",
new TypeReference<Map<String, RevokedTokenInfo>>() {}),
trimTrailingSlash(emulatorConfig.effectiveBaseUrl()),
- emulatorConfig.services().cloudfront().domainSuffix(),
+ emulatorConfig.services().cloudfront().domainSuffix().orElse(null),
regionResolver,
lambdaService,
acmService,
@@ -242,7 +243,7 @@ public class CognitoService implements ResourceProvider {
LambdaService lambdaService,
AcmService acmService) {
this(poolStore, clientStore, resourceServerStore, userStore, groupStore, revokedTokenStore, baseUrl,
- DEFAULT_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver, lambdaService, acmService);
+ UNCONFIGURED_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver, lambdaService, acmService);
}
CognitoService(StorageBackend<String, UserPool> poolStore,
@@ -259,7 +260,7 @@ public class CognitoService implements ResourceProvider {
CognitoMessageDispatcher messageDispatcher,
TlsCertificateManager certificateManager) {
this(poolStore, clientStore, resourceServerStore, domainStore, identityProviderStore, userStore,
- groupStore, revokedTokenStore, baseUrl, DEFAULT_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver,
+ groupStore, revokedTokenStore, baseUrl, UNCONFIGURED_CLOUDFRONT_DOMAIN_SUFFIX, regionResolver,
lambdaService, acmService, verificationCodeService, messageDispatcher, certificateManager);
}
@@ -1934,8 +1935,10 @@ public class CognitoService implements ResourceProvider {
return AwsArnUtils.Arn.global(partition, "cloudfront", userPoolDomain.getAwsAccountId(), "distribution/" + id).toString();
}
+ /** A distribution name on the configured CloudFront suffix, or the request partition's when none is. */
private String generateCloudFrontDomain() {
- return UUID.randomUUID().toString().replace("-", "").substring(0, 14) + "." + cloudFrontDomainSuffix;
+ return UUID.randomUUID().toString().replace("-", "").substring(0, 14) + "."
+ + CloudFrontEdgeDomain.domainSuffix(cloudFrontDomainSuffix, regionResolver.getPartition());
}
private String generateDomainVersion() {
diff --git a/src/main/resources/aws/region-facts.json b/src/main/resources/aws/region-facts.json
index 7903b1a653..011b24fc25 100644
--- a/src/main/resources/aws/region-facts.json
+++ b/src/main/resources/aws/region-facts.json
@@ -3,7 +3,8 @@
"generator": "tools/aws/regen_region_facts.py",
"terraform": "terraform-provider-aws internal/service/elb/hosted_zone_id_data_source.go, internal/service/elbv2/hosted_zone_id_data_source.go, internal/service/s3/hosted_zones.go, internal/conns/awsclient.go",
"cdk": "aws-cdk packages/aws-cdk-lib/region-info/build-tools/fact-tables.ts, packages/aws-cdk-lib/aws-route53-targets/lib/cloudfront-target.ts, packages/aws-cdk-lib/aws-ec2/lib/vpc-endpoint.ts",
- "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) cn/console.md, gov/configure-account.md, gov/govcloud-organizations.md",
+ "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) cn/cloudfront.md, cn/console.md, gov/configure-account.md, gov/govcloud-abp.md, gov/govcloud-organizations.md",
+ "botocore": "botocore cloudfront/2020-05-31/service-2.json (Distribution$DomainName, DistributionSummary$DomainName)",
"s3WebsiteConflicts": "terraform wins over the CDK where they disagree (it cites the S3 website-endpoint table)"
},
"partitions": {
@@ -11,13 +12,15 @@
"vpcEndpointServiceNamePrefix": "com.amazonaws.vpce",
"cloudfrontHostedZoneId": "Z2FDTNDATAQYW2",
"samlSignOnUrl": "https://signin.aws.amazon.com/saml",
- "consoleHost": "console.aws.amazon.com"
+ "consoleHost": "console.aws.amazon.com",
+ "cloudfrontDomainSuffix": "cloudfront.net"
},
"aws-cn": {
"vpcEndpointServiceNamePrefix": "cn.com.amazonaws.vpce",
"cloudfrontHostedZoneId": "Z3RFFRIM2A3IF5",
"samlSignOnUrl": "https://signin.amazonaws.cn/saml",
- "consoleHost": "console.amazonaws.cn"
+ "consoleHost": "console.amazonaws.cn",
+ "cloudfrontDomainSuffix": "cloudfront.cn"
},
"aws-eusc": {
"vpcEndpointServiceNamePrefix": "eu.amazonaws.vpce"
@@ -386,13 +389,15 @@
"albHostedZoneId": "Z166TLBEWOO7G0",
"classicElbHostedZoneId": "Z166TLBEWOO7G0",
"nlbHostedZoneId": "Z1ZSMQQ6Q24QQ8",
- "s3WebsiteHostedZoneId": "Z2NIFVYYW2VKV1"
+ "s3WebsiteHostedZoneId": "Z2NIFVYYW2VKV1",
+ "apiGatewayRegionalHostedZoneId": "Z3SE9ATJYCRCZJ"
},
"us-gov-west-1": {
"albHostedZoneId": "Z33AYJ8TM3BH4J",
"classicElbHostedZoneId": "Z33AYJ8TM3BH4J",
"nlbHostedZoneId": "ZMG1MZ2THAWF1",
- "s3WebsiteHostedZoneId": "Z31GFT0UA1I2HV"
+ "s3WebsiteHostedZoneId": "Z31GFT0UA1I2HV",
+ "apiGatewayRegionalHostedZoneId": "Z1K6XKP9SAGWDV"
},
"us-iso-east-1": {
"vpcEndpointExceptionPrefix": "gov.ic.c2s",
diff --git a/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java b/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java
index 068e3eca12..82f61ed493 100644
--- a/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java
+++ b/src/test/java/io/github/hectorvent/floci/core/common/AwsRegionFactsTest.java
@@ -61,6 +61,26 @@ class AwsRegionFactsTest {
assertTrue(AwsRegionFacts.consoleHost(null).isEmpty());
}
+ /** The botocore model's example name for the commercial partition, the China guide for {@code aws-cn}. */
+ @Test
+ void cloudFrontDomainSuffixIsPerPartitionWhereCloudFrontExists() {
+ assertEquals(Optional.of("cloudfront.net"), AwsRegionFacts.cloudFrontDomainSuffix("aws"));
+ assertEquals(Optional.of("cloudfront.cn"), AwsRegionFacts.cloudFrontDomainSuffix("aws-cn"));
+ assertTrue(AwsRegionFacts.cloudFrontDomainSuffix("aws-us-gov").isEmpty());
+ assertTrue(AwsRegionFacts.cloudFrontDomainSuffix("aws-eusc").isEmpty());
+ assertTrue(AwsRegionFacts.cloudFrontDomainSuffix(null).isEmpty());
+ }
+
+ /** The GovCloud guide states the two regional endpoint zones; no source covers another region. */
+ @Test
+ void apiGatewayRegionalZoneIsPublishedForGovCloudOnly() {
+ assertEquals(Optional.of("Z1K6XKP9SAGWDV"), AwsRegionFacts.apiGatewayRegionalHostedZoneId("us-gov-west-1"));
+ assertEquals(Optional.of("Z3SE9ATJYCRCZJ"), AwsRegionFacts.apiGatewayRegionalHostedZoneId("us-gov-east-1"));
+ assertTrue(AwsRegionFacts.apiGatewayRegionalHostedZoneId("us-east-1").isEmpty());
+ assertTrue(AwsRegionFacts.apiGatewayRegionalHostedZoneId("cn-north-1").isEmpty());
+ assertTrue(AwsRegionFacts.apiGatewayRegionalHostedZoneId(null).isEmpty());
+ }
+
@ParameterizedTest
@CsvSource({
"us-east-1, s3, com.amazonaws.us-east-1.s3",
diff --git a/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java b/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java
index fe278c4f2b..98cbf29b52 100644
--- a/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java
+++ b/src/test/java/io/github/hectorvent/floci/core/common/CloudFrontEdgeDomainTest.java
@@ -12,17 +12,35 @@ class CloudFrontEdgeDomainTest {
@Test
void namesADistributionOnTheConfiguredSuffix() {
- CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create("us-east-1", "cloudfront.net");
+ CloudFrontEdgeDomain edge = CloudFrontEdgeDomain.create("aws", "cloudfront.net");
assertTrue(edge.domainName().matches("d[0-9a-f]{13}\\.cloudfront\\.net"), edge.domainName());
- assertNotEquals(edge.domainName(), CloudFrontEdgeDomain.create("us-east-1", "cloudfront.net").domainName());
+ assertNotEquals(edge.domainName(), CloudFrontEdgeDomain.create("aws", "cloudfront.net").domainName());
+ }
+
+ @Test
+ void anUnconfiguredSuffixIsThePartitionsOwn() {
+ assertTrue(CloudFrontEdgeDomain.create("aws", null).domainName().endsWith(".cloudfront.net"));
+ assertTrue(CloudFrontEdgeDomain.create("aws-cn", null).domainName().endsWith(".cloudfront.cn"));
+ assertEquals("cloudfront.net", CloudFrontEdgeDomain.domainSuffix(null, "aws"));
+ assertEquals("cloudfront.cn", CloudFrontEdgeDomain.domainSuffix(null, "aws-cn"));
+ assertEquals("cloudfront.cn", CloudFrontEdgeDomain.domainSuffix(" ", "aws-cn"), "blank is unset");
+ assertEquals("cloudfront.net", CloudFrontEdgeDomain.domainSuffix(null, "aws-us-gov"),
+ "a partition without CloudFront keeps the commercial suffix");
+ assertEquals("cloudfront.net", CloudFrontEdgeDomain.domainSuffix(null, null));
+ }
+
+ @Test
+ void aConfiguredSuffixWinsInEveryPartition() {
+ assertEquals("cloudfront.local", CloudFrontEdgeDomain.domainSuffix("cloudfront.local", "aws-cn"));
+ assertTrue(CloudFrontEdgeDomain.create("aws-cn", "cloudfront.net").domainName().endsWith(".cloudfront.net"));
}
@Test
void hostedZoneIsThePartitionsCloudFrontZone() {
- assertEquals("Z2FDTNDATAQYW2", CloudFrontEdgeDomain.create("eu-west-1", "cloudfront.net").hostedZoneId());
- assertEquals("Z3RFFRIM2A3IF5", CloudFrontEdgeDomain.create("cn-north-1", "cloudfront.net").hostedZoneId());
- assertNull(CloudFrontEdgeDomain.create("us-gov-west-1", "cloudfront.net").hostedZoneId(),
+ assertEquals("Z2FDTNDATAQYW2", CloudFrontEdgeDomain.create("aws", "cloudfront.net").hostedZoneId());
+ assertEquals("Z3RFFRIM2A3IF5", CloudFrontEdgeDomain.create("aws-cn", "cloudfront.net").hostedZoneId());
+ assertNull(CloudFrontEdgeDomain.create("aws-us-gov", "cloudfront.net").hostedZoneId(),
"GovCloud has no CloudFront");
}
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java
index c12c0f797f..be31cc3503 100644
--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayDomainTlsServiceTest.java
@@ -11,6 +11,7 @@ import org.junit.jupiter.api.Test;
import java.util.List;
import java.util.Map;
+import java.util.Optional;
import java.util.concurrent.atomic.AtomicBoolean;
import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -46,7 +47,7 @@ class ApiGatewayDomainTlsServiceTest {
when(storageFactory.create(anyString(), anyString(), any())).thenAnswer(invocation -> AccountAwareStorageBackend.inMemory("000000000000"));
certificateManager = mock(TlsCertificateManager.class);
EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");
+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());
service = new ApiGatewayService(storageFactory, config, certificateManager,
new RegionResolver(REGION, "000000000000"));
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java
index 0f15ed6c3a..068eee72ea 100644
--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeDomainPartitionTest.java
@@ -13,6 +13,7 @@ import org.junit.jupiter.api.Test;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.Optional;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
@@ -41,7 +42,7 @@ class ApiGatewayEdgeDomainPartitionTest {
stores.computeIfAbsent(invocation.getArgument(1, String.class),
ignored -> AccountAwareStorageBackend.inMemory("000000000000")));
EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");
+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());
service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class),
new RegionResolver(GOV_REGION, "000000000000"));
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java
index bb292d2506..eb153be7ab 100644
--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayEdgeRestApiPartitionTest.java
@@ -16,6 +16,7 @@ import org.junit.jupiter.params.provider.ValueSource;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.Optional;
import static org.hamcrest.MatcherAssert.assertThat;
import static org.hamcrest.Matchers.containsString;
@@ -43,7 +44,7 @@ class ApiGatewayEdgeRestApiPartitionTest {
stores.computeIfAbsent(invocation.getArgument(1, String.class),
ignored -> AccountAwareStorageBackend.inMemory("000000000000")));
EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");
+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());
service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class),
new RegionResolver("us-east-1", "000000000000"));
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayRegionalHostedZoneTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayRegionalHostedZoneTest.java
new file mode 100644
index 0000000000..7144f5894c
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayRegionalHostedZoneTest.java
@@ -0,0 +1,57 @@
+package io.github.hectorvent.floci.services.apigateway;
+
+import io.github.hectorvent.floci.config.EmulatorConfig;
+import io.github.hectorvent.floci.config.TlsCertificateManager;
+import io.github.hectorvent.floci.core.common.RegionResolver;
+import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;
+import io.github.hectorvent.floci.core.storage.StorageFactory;
+import io.github.hectorvent.floci.services.apigateway.model.CustomDomain;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyString;
+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * The Route 53 hosted zone a regional custom domain reports. The GovCloud user guide states it for
+ * its two regions; every other region has no source and keeps the value it always reported.
+ */
+class ApiGatewayRegionalHostedZoneTest {
+
+ private ApiGatewayService service;
+
+ @BeforeEach
+ void setUp() {
+ StorageFactory storageFactory = mock(StorageFactory.class);
+ when(storageFactory.create(anyString(), anyString(), any()))
+ .thenAnswer(invocation -> AccountAwareStorageBackend.inMemory("000000000000"));
+ EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());
+ service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class),
+ new RegionResolver("us-east-1", "000000000000"));
+ }
+
+ @ParameterizedTest
+ @CsvSource({
+ "us-gov-west-1, Z1K6XKP9SAGWDV",
+ "us-gov-east-1, Z3SE9ATJYCRCZJ",
+ "us-east-1, Z2FDTNDATAQYL2",
+ "cn-north-1, Z2FDTNDATAQYL2",
+ "eusc-de-east-1, Z2FDTNDATAQYL2"})
+ void regionalDomainReportsTheRegionsPublishedZone(String region, String hostedZoneId) {
+ CustomDomain domain = service.createDomainName(region, Map.of(
+ "domainName", "api." + region + ".example.com",
+ "endpointConfiguration", Map.of("types", List.of("REGIONAL"))));
+
+ assertEquals(hostedZoneId, domain.getRegionalHostedZoneId());
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java
index d036c70dd1..85e3fc43e0 100644
--- a/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/apigateway/ApiGatewayV2ApiMappingServiceTest.java
@@ -15,6 +15,7 @@ import org.junit.jupiter.api.Test;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.Optional;
import java.util.Set;
import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -55,7 +56,7 @@ class ApiGatewayV2ApiMappingServiceTest {
when(storageFactory.create(anyString(), anyString(), any()))
.thenAnswer(invocation -> AccountAwareStorageBackend.inMemory("000000000000"));
EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");
+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());
RegionResolver regionResolver = new RegionResolver(REGION, "000000000000");
v2Service = new ApiGatewayV2Service(storageFactory, config, regionResolver);
service = new ApiGatewayService(storageFactory, config, mock(TlsCertificateManager.class), regionResolver,
diff --git a/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java
index 6ca9eefc94..f207e09b2f 100644
--- a/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/appsync/AppSyncServiceTest.java
@@ -30,6 +30,7 @@ import java.time.ZoneOffset;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
+import java.util.Optional;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
@@ -50,6 +51,8 @@ class AppSyncServiceTest {
private AppSyncService service;
private AccountAwareStorageBackend<ApiKey> apiKeyStoreOverride;
private String region = "us-east-1";
+ /** The deployment partition, or null to derive it from {@link #region}. */
+ private String partition;
@BeforeEach
void setUp() {
@@ -59,7 +62,8 @@ class AppSyncServiceTest {
/**
* AppSync fronts a custom domain with CloudFront, so a DNS alias to {@code appsyncDomainName}
* lives in the partition's CloudFront hosted zone, as the CDK's AppSyncTarget builds it. A
- * random zone id made every such alias point nowhere.
+ * random zone id made every such alias point nowhere. With no suffix configured the
+ * distribution name takes the partition's own.
*/
@Test
void customDomainIsFrontedByTheCloudFrontZoneOfItsPartition() {
@@ -72,6 +76,24 @@ class AppSyncServiceTest {
DomainName china = newService(Clock.fixed(NOW, ZoneOffset.UTC))
.createDomainName(Map.of("domainName", "api.example.cn"));
assertEquals("Z3RFFRIM2A3IF5", china.getHostedZoneId());
+ assertTrue(china.getAppsyncDomainName().matches("d[0-9a-f]{13}\\.cloudfront\\.cn"),
+ china.getAppsyncDomainName());
+ }
+
+ /**
+ * A region no partition publishes, accepted under {@code allow-unknown-regions}, belongs to the
+ * deployment's partition, so a China deployment fronts the domain with China's CloudFront
+ * suffix and zone rather than the commercial ones.
+ */
+ @Test
+ void customDomainInAnUnpublishedRegionTakesTheDeploymentsPartition() {
+ region = "xx-test-1";
+ partition = "aws-cn";
+ DomainName domain = newService(Clock.fixed(NOW, ZoneOffset.UTC))
+ .createDomainName(Map.of("domainName", "api.example.cn"));
+
+ assertEquals("Z3RFFRIM2A3IF5", domain.getHostedZoneId());
+ assertTrue(domain.getAppsyncDomainName().endsWith(".cloudfront.cn"), domain.getAppsyncDomainName());
}
@Test
@@ -406,11 +428,12 @@ class AppSyncServiceTest {
lambdaAuthorizerCache = new LambdaAuthorizerCache();
EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
when(config.effectiveBaseUrl()).thenReturn(baseUrl);
- when(config.services().cloudfront().domainSuffix()).thenReturn("cloudfront.net");
+ when(config.services().cloudfront().domainSuffix()).thenReturn(Optional.empty());
return new AppSyncService(
storageFactory,
config,
- new RegionResolver(region, "000000000000"),
+ partition == null ? new RegionResolver(region, "000000000000")
+ : new RegionResolver(region, "000000000000", partition),
schemaRegistry,
mock(SchemaCreationWorker.class),
requestContext,
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java
index 3eb688692a..8c591ca616 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudfront/CloudFrontServiceTest.java
@@ -127,6 +127,7 @@ class CloudFrontServiceTest {
return serviceWithDomainSuffix(domainSuffix, "us-east-1");
}
+ /** A null {@code domainSuffix} is {@code floci.services.cloudfront.domain-suffix} left unset. */
private CloudFrontService serviceWithDomainSuffix(String domainSuffix, String region) {
StorageFactory storageFactory = Mockito.mock(StorageFactory.class);
when(storageFactory.create(Mockito.anyString(), Mockito.anyString(), Mockito.any()))
@@ -143,7 +144,7 @@ class CloudFrontServiceTest {
when(dnsConfig.extraSuffixes()).thenReturn(Optional.empty());
when(config.services()).thenReturn(servicesConfig);
when(servicesConfig.cloudfront()).thenReturn(cloudFrontConfig);
- when(cloudFrontConfig.domainSuffix()).thenReturn(domainSuffix);
+ when(cloudFrontConfig.domainSuffix()).thenReturn(Optional.ofNullable(domainSuffix));
return new CloudFrontService(storageFactory, config, new RegionResolver(region, ACCOUNT));
}
@@ -323,7 +324,7 @@ class CloudFrontServiceTest {
@Test
void createDistributionUsesDefaultDomainSuffix() {
- CloudFrontService service = serviceWithDomainSuffix("cloudfront.net");
+ CloudFrontService service = serviceWithDomainSuffix(null);
Distribution dist = service.createDistribution(new Distribution(), Map.of());
@@ -364,6 +365,33 @@ class CloudFrontServiceTest {
"Expected configured suffix, got: " + dist.getDomainName());
}
+ /**
+ * China's distributions are named under {@code cloudfront.cn}; GovCloud has no CloudFront and
+ * so no suffix of its own, and keeps the commercial one.
+ */
+ @Test
+ void anUnconfiguredDomainSuffixFollowsTheRequestsPartition() {
+ CloudFrontService china = serviceWithDomainSuffix(null, "cn-north-1");
+ CloudFrontService govCloud = serviceWithDomainSuffix(null, "us-gov-west-1");
+
+ Distribution dist = china.createDistribution(new Distribution(), Map.of());
+ StreamingDistribution streaming = china.createStreamingDistribution(new StreamingDistribution());
+
+ assertEquals(dist.getId().toLowerCase(Locale.ROOT) + ".cloudfront.cn", dist.getDomainName());
+ assertEquals(streaming.getId().toLowerCase(Locale.ROOT) + ".cloudfront.cn", streaming.getDomainName());
+ assertTrue(govCloud.createDistribution(new Distribution(), Map.of()).getDomainName().endsWith(".cloudfront.net"));
+ }
+
+ @Test
+ void aConfiguredDomainSuffixWinsOverThePartitions() {
+ CloudFrontService china = serviceWithDomainSuffix("cloudfront.local", "cn-north-1");
+
+ Distribution dist = china.createDistribution(new Distribution(), Map.of());
+
+ assertTrue(dist.getDomainName().endsWith(".cloudfront.local"),
+ "Expected configured suffix, got: " + dist.getDomainName());
+ }
+
@Test
void createStreamingDistributionHonorsConfiguredDomainSuffix() {
CloudFrontService service = serviceWithDomainSuffix("cloudfront.local");
diff --git a/tools/aws/regen_region_facts.py b/tools/aws/regen_region_facts.py
index 5381dd5d69..fb654d16f0 100644
--- a/tools/aws/regen_region_facts.py
+++ b/tools/aws/regen_region_facts.py
@@ -1,5 +1,5 @@
#!/usr/bin/env python3
-"""Regenerate src/main/resources/aws/region-facts.json from the AWS CDK, Terraform and user-guide sources.
+"""Regenerate src/main/resources/aws/region-facts.json from the AWS CDK, Terraform, botocore and user-guide sources.
AWS publishes a handful of per-region and per-partition constants only as prose tables (the
General Reference), never as SDK data. The two open-source projects that transcribe them, and the
@@ -31,6 +31,19 @@ China and GovCloud user guides for what neither transcribes, are the sources her
Either rendering parses: the page's text (what the `aws-refs` skill's docs driver extracts from
the `.html`), or the Markdown AWS serves at the same URL with `.md` in place of `.html`, which
writes the China host as a link.
+- The suffix of a CloudFront distribution's domain name: `cloudfront.net` from the example the
+ botocore CloudFront model gives for `Distribution$DomainName` and
+ `DistributionSummary$DomainName`, and `cloudfront.cn` for `aws-cn` from the China guide's
+ CloudFront page (`CLOUDFRONT_SUFFIX_SENTENCES`),
+ https://docs.amazonaws.cn/en_us/aws/latest/userguide/cloudfront.html -> cn/cloudfront.md, whose
+ Markdown rendering escapes the asterisk (`\\*.cloudfront.cn`). The other partitions have no
+ CloudFront.
+- The Route 53 hosted zone of an API Gateway regional endpoint, for the two GovCloud regions only:
+ the GovCloud guide's API Gateway page (`APIGATEWAY_REGIONAL_ZONE_SENTENCE`),
+ https://docs.aws.amazon.com/govcloud-us/latest/UserGuide/govcloud-abp.html -> gov/govcloud-abp.md,
+ with the region
+ resolved through the region descriptions in `partitions.json`. No source in `local/aws` gives
+ the zone for any other region.
Nothing is hand-typed. When the checkouts are absent (CI), `--check` only verifies the vendored
file parses and keeps its shape; a regeneration needs `local/aws`.
@@ -53,6 +66,7 @@ PARTITIONS = REPO_ROOT / "src/main/resources/aws/partitions.json"
LOCAL_TERRAFORM = REPO_ROOT / "local/aws/terraform-provider-aws"
LOCAL_CDK = REPO_ROOT / "local/aws/aws-cdk"
LOCAL_GUIDES = REPO_ROOT / "local/aws/aws-docs/partition-guides"
+LOCAL_BOTOCORE_DATA = REPO_ROOT / "local/aws/botocore/botocore/data"
TF_ELB = "internal/service/elb/hosted_zone_id_data_source.go"
TF_ELBV2 = "internal/service/elbv2/hosted_zone_id_data_source.go"
@@ -61,6 +75,7 @@ TF_CLIENT = "internal/conns/awsclient.go"
CDK_FACT_TABLES = "packages/aws-cdk-lib/region-info/build-tools/fact-tables.ts"
CDK_CLOUDFRONT_TARGET = "packages/aws-cdk-lib/aws-route53-targets/lib/cloudfront-target.ts"
CDK_VPC_ENDPOINT = "packages/aws-cdk-lib/aws-ec2/lib/vpc-endpoint.ts"
+BOTOCORE_CLOUDFRONT = "cloudfront/2020-05-31/service-2.json"
GO_ROW_RE = re.compile(r'^\s*endpoints\.([A-Za-z0-9]+)RegionID:\s*"([A-Z0-9]+)",')
TS_ROW_RE = re.compile(r"^\s*'([a-z0-9-]+)':\s*'([A-Z0-9]+)',")
@@ -86,6 +101,24 @@ CONSOLE_HOST_SENTENCES = {
"aws-us-gov": ("gov/configure-account.md", re.compile(
r"Sign in to the AWS GovCloud \(US\) console and open the IAM console at https://([a-z0-9.-]+)/iam")),
}
+# The commercial suffix is the one in the example distribution name of these two model members.
+BOTOCORE_CLOUDFRONT_DOMAIN_SHAPES = ("Distribution", "DistributionSummary")
+BOTOCORE_CLOUDFRONT_EXAMPLE_RE = re.compile(r"<code>d[0-9a-z]+\.(cloudfront\.[a-z]+)</code>")
+CLOUDFRONT_SUFFIX_SENTENCES = {
+ "aws-cn": ("cn/cloudfront.md", re.compile(
+ r"You can.t use the default CloudFront domain, \\?\*\.(cloudfront\.[a-z]+), to serve content\.")),
+}
+# One sentence per region, each naming the region by the description partitions.json carries. The
+# page writes "Route 53" with a no-break space.
+APIGATEWAY_REGIONAL_ZONE_PAGE = "gov/govcloud-abp.md"
+# The page states a zone for every region of this partition, so a run that finds fewer is refused.
+APIGATEWAY_REGIONAL_ZONE_PARTITION = "aws-us-gov"
+APIGATEWAY_REGIONAL_ZONE_SENTENCE = re.compile(
+ r"The Amazon Route\s53 Hosted Zone ID for the regional endpoint in the (AWS GovCloud \([A-Za-z-]+\)) Region"
+ r" is (Z[0-9A-Z]+)\.")
+GUIDE_PAGES = sorted({page for page, _ in CONSOLE_HOST_SENTENCES.values()}
+ | {page for page, _ in CLOUDFRONT_SUFFIX_SENTENCES.values()}
+ | {APIGATEWAY_REGIONAL_ZONE_PAGE})
GO_PARTITION_IDS = {"AwsCn": "aws-cn", "AwsUsGov": "aws-us-gov", "AwsIso": "aws-iso", "AwsIsoB": "aws-iso-b",
"AwsIsoE": "aws-iso-e", "AwsIsoF": "aws-iso-f", "Aws": "aws"}
@@ -176,28 +209,79 @@ def parse_vpc_endpoint_exceptions(text: str) -> tuple[dict[str, list[str]], dict
return services, prefixes, cn_suffix
-def parse_console_host(text: str, sentence: re.Pattern[str], page: str) -> str:
- """The one host `sentence` states in a guide page; a page that lost the sentence, or states two
- different hosts with it, is an error rather than a guess."""
- hosts = sorted({match.group(1).rstrip(".") for match in sentence.finditer(text)})
- if len(hosts) != 1:
- raise ValueError(f"{page}: expected one console host from /{sentence.pattern}/, found {hosts}")
- return hosts[0]
+def parse_guide_sentence(text: str, sentence: re.Pattern[str], page: str) -> str:
+ """The one value `sentence` states in a guide page; a page that lost the sentence, or states two
+ different values with it, is an error rather than a guess."""
+ values = sorted({match.group(1).rstrip(".") for match in sentence.finditer(text)})
+ if len(values) != 1:
+ raise ValueError(f"{page}: expected one value from /{sentence.pattern}/, found {values}")
+ return values[0]
+
+
+def parse_guide_sentences(guides: Path, sentences: dict[str, tuple[str, re.Pattern[str]]]) -> dict[str, str]:
+ return {partition: parse_guide_sentence((guides / page).read_text(encoding="utf-8"), sentence, page)
+ for partition, (page, sentence) in sentences.items()}
def parse_console_hosts(guides: Path) -> dict[str, str]:
- return {partition: parse_console_host((guides / page).read_text(encoding="utf-8"), sentence, page)
- for partition, (page, sentence) in CONSOLE_HOST_SENTENCES.items()}
+ return parse_guide_sentences(guides, CONSOLE_HOST_SENTENCES)
+
+
+def parse_botocore_cloudfront_suffix(text: str) -> str:
+ """The suffix of the example distribution name in the model's `DomainName` documentation."""
+ shapes = json.loads(text)["shapes"]
+ suffixes = set()
+ for shape in BOTOCORE_CLOUDFRONT_DOMAIN_SHAPES:
+ documentation = shapes[shape]["members"]["DomainName"].get("documentation", "")
+ found = set(BOTOCORE_CLOUDFRONT_EXAMPLE_RE.findall(documentation))
+ if len(found) != 1:
+ raise ValueError(f"cloudfront model: {shape}$DomainName gives no single example distribution name: {found}")
+ suffixes |= found
+ if len(suffixes) != 1:
+ raise ValueError(f"cloudfront model: the DomainName examples disagree on the suffix: {sorted(suffixes)}")
+ return suffixes.pop()
+
+
+def parse_cloudfront_suffixes(botocore_data: Path, guides: Path) -> dict[str, str]:
+ suffixes = {"aws": parse_botocore_cloudfront_suffix((botocore_data / BOTOCORE_CLOUDFRONT).read_text(encoding="utf-8"))}
+ suffixes.update(parse_guide_sentences(guides, CLOUDFRONT_SUFFIX_SENTENCES))
+ return suffixes
+
+
+def parse_apigateway_regional_zones(text: str, partitions_doc: dict) -> dict[str, str]:
+ """region id -> hosted zone for every GovCloud region, from the GovCloud API Gateway page."""
+ by_description = {r["description"]: r["id"] for p in partitions_doc["partitions"] for r in p["regions"]
+ if r.get("description")}
+ zones: dict[str, str] = {}
+ for description, zone in APIGATEWAY_REGIONAL_ZONE_SENTENCE.findall(text):
+ region = by_description.get(description)
+ if region is None:
+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: {description!r} is not a region description "
+ "in partitions.json")
+ if zones.setdefault(region, zone) != zone:
+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: two regional hosted zones for {region}")
+ if not zones:
+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: no regional endpoint hosted zone sentence found")
+ missing = sorted(apigateway_zone_regions(partitions_doc) - zones.keys())
+ if missing:
+ raise ValueError(f"{APIGATEWAY_REGIONAL_ZONE_PAGE}: no regional endpoint hosted zone for {', '.join(missing)}")
+ return zones
+
+
+def apigateway_zone_regions(partitions_doc: dict) -> set[str]:
+ """The regions the GovCloud API Gateway page must state a regional hosted zone for."""
+ return {region["id"] for partition in partitions_doc["partitions"]
+ if partition["id"] == APIGATEWAY_REGIONAL_ZONE_PARTITION for region in partition["regions"]}
def guides_present(guides: Path) -> bool:
- return all((guides / page).exists() for page, _ in CONSOLE_HOST_SENTENCES.values())
+ return all((guides / page).exists() for page in GUIDE_PAGES)
# --------------------------------------------------------------------------- #
# Generation
# --------------------------------------------------------------------------- #
-def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dict:
+def build(terraform: Path, cdk: Path, guides: Path, botocore_data: Path, partitions_doc: dict) -> dict:
elb = parse_go_map((terraform / TF_ELB).read_text(encoding="utf-8"), "hostedZoneIDPerRegionMap")
elbv2_text = (terraform / TF_ELBV2).read_text(encoding="utf-8")
alb = parse_go_map(elbv2_text, "hostedZoneIDPerRegionALBMap")
@@ -216,6 +300,9 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dic
if elb != alb:
raise ValueError("the Classic and ALB hosted-zone maps differ; the generator assumed they are one table")
console_hosts = parse_console_hosts(guides)
+ cloudfront_suffixes = parse_cloudfront_suffixes(botocore_data, guides)
+ apigateway_zones = parse_apigateway_regional_zones(
+ (guides / APIGATEWAY_REGIONAL_ZONE_PAGE).read_text(encoding="utf-8"), partitions_doc)
partition_ids = [p["id"] for p in partitions_doc["partitions"]]
region_partition = {r["id"]: p for p in partitions_doc["partitions"] for r in p["regions"]}
@@ -223,7 +310,7 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dic
s3_website = dict(s3_cdk)
s3_website.update(s3_tf) # Terraform cites the website-endpoint table; it wins on conflicts
- region_ids = sorted(set(alb) | set(nlb) | set(s3_website) | set(vpce_services))
+ region_ids = sorted(set(alb) | set(nlb) | set(s3_website) | set(vpce_services) | set(apigateway_zones))
regions = {}
for region in region_ids:
if region not in region_partition:
@@ -241,6 +328,8 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dic
entry["vpcEndpointPrefixServices"] = vpce_services[region]
if region in vpce_cn_suffix:
entry["vpcEndpointCnSuffixServices"] = vpce_cn_suffix[region]
+ if region in apigateway_zones:
+ entry["apiGatewayRegionalHostedZoneId"] = apigateway_zones[region]
regions[region] = entry
partitions = {}
@@ -252,6 +341,8 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dic
entry["samlSignOnUrl"] = saml[partition_id]
if partition_id in console_hosts:
entry["consoleHost"] = console_hosts[partition_id]
+ if partition_id in cloudfront_suffixes:
+ entry["cloudfrontDomainSuffix"] = cloudfront_suffixes[partition_id]
partitions[partition_id] = entry
return {
@@ -259,8 +350,8 @@ def build(terraform: Path, cdk: Path, guides: Path, partitions_doc: dict) -> dic
"generator": "tools/aws/regen_region_facts.py",
"terraform": f"terraform-provider-aws {TF_ELB}, {TF_ELBV2}, {TF_S3}, {TF_CLIENT}",
"cdk": f"aws-cdk {CDK_FACT_TABLES}, {CDK_CLOUDFRONT_TARGET}, {CDK_VPC_ENDPOINT}",
- "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) "
- + ", ".join(sorted(page for page, _ in CONSOLE_HOST_SENTENCES.values())),
+ "docs": "aws-docs/partition-guides (snapshots of the China and GovCloud user guides) " + ", ".join(GUIDE_PAGES),
+ "botocore": f"botocore {BOTOCORE_CLOUDFRONT} (Distribution$DomainName, DistributionSummary$DomainName)",
"s3WebsiteConflicts": "terraform wins over the CDK where they disagree (it cites the S3 website-endpoint table)",
},
"partitions": partitions,
@@ -285,6 +376,7 @@ SAML_URL_RE = re.compile(r"https://[a-z0-9.-]+/saml")
VPCE_PREFIX_RE = re.compile(r"[a-z0-9-]+(\.[a-z0-9-]+)*\.vpce")
REVERSED_DNS_RE = re.compile(r"[a-z0-9-]+(\.[a-z0-9-]+)+")
CONSOLE_HOST_RE = re.compile(r"console(\.[a-z0-9-]+){2,}")
+CLOUDFRONT_SUFFIX_RE = re.compile(r"cloudfront\.[a-z]+")
def self_check(path: Path, partitions_path: Path = PARTITIONS) -> list[str]:
@@ -322,6 +414,11 @@ def self_check(path: Path, partitions_path: Path = PARTITIONS) -> list[str]:
problems.append(f"{path}: {partition['id']}.vpcEndpointServiceNamePrefix should be {expected!r}")
if partition["id"] in CONSOLE_HOST_SENTENCES and "consoleHost" not in entry:
problems.append(f"{path}: {partition['id']}.consoleHost is missing")
+ if partition["id"] in {"aws"} | CLOUDFRONT_SUFFIX_SENTENCES.keys() and "cloudfrontDomainSuffix" not in entry:
+ problems.append(f"{path}: {partition['id']}.cloudfrontDomainSuffix is missing")
+ for region in sorted(apigateway_zone_regions({"partitions": partitions_doc})):
+ if "apiGatewayRegionalHostedZoneId" not in document["regions"].get(region, {}):
+ problems.append(f"{path}: {region}.apiGatewayRegionalHostedZoneId is missing")
known = {region["id"] for partition in partitions_doc for region in partition["regions"]}
for region in document["regions"]:
if region not in known:
@@ -330,7 +427,8 @@ def self_check(path: Path, partitions_path: Path = PARTITIONS) -> list[str]:
for partition, entry in document["partitions"].items():
for key, value in entry.items():
pattern = {"cloudfrontHostedZoneId": HOSTED_ZONE_ID_RE, "samlSignOnUrl": SAML_URL_RE,
- "vpcEndpointServiceNamePrefix": VPCE_PREFIX_RE, "consoleHost": CONSOLE_HOST_RE}.get(key)
+ "vpcEndpointServiceNamePrefix": VPCE_PREFIX_RE, "consoleHost": CONSOLE_HOST_RE,
+ "cloudfrontDomainSuffix": CLOUDFRONT_SUFFIX_RE}.get(key)
if pattern is None:
problems.append(f"{path}: {partition}.{key} is not a known field")
elif not isinstance(value, str) or not pattern.fullmatch(value):
@@ -357,13 +455,14 @@ def main(argv: list[str] | None = None) -> int:
parser.add_argument("--terraform", type=Path, default=LOCAL_TERRAFORM, help="terraform-provider-aws checkout")
parser.add_argument("--cdk", type=Path, default=LOCAL_CDK, help="aws-cdk checkout")
parser.add_argument("--guides", type=Path, default=LOCAL_GUIDES, help="aws-docs/partition-guides snapshots")
+ parser.add_argument("--botocore-data", type=Path, default=LOCAL_BOTOCORE_DATA, help="botocore data directory")
parser.add_argument("--partitions", type=Path, default=PARTITIONS, help=argparse.SUPPRESS)
parser.add_argument("--output", type=Path, default=OUTPUT, help=argparse.SUPPRESS)
args = parser.parse_args(argv)
rel = args.output.relative_to(REPO_ROOT) if args.output.is_relative_to(REPO_ROOT) else args.output
sources_present = ((args.terraform / TF_ELB).exists() and (args.cdk / CDK_FACT_TABLES).exists()
- and guides_present(args.guides))
+ and guides_present(args.guides) and (args.botocore_data / BOTOCORE_CLOUDFRONT).exists())
if not sources_present:
if args.check:
problems = self_check(args.output, args.partitions)
@@ -373,10 +472,11 @@ def main(argv: list[str] | None = None) -> int:
return 1
print(f"{rel}: sources not checked out, shape verified only")
return 0
- raise SystemExit("error: regenerating needs the terraform-provider-aws and aws-cdk checkouts and the "
- "aws-docs/partition-guides snapshots under local/aws")
+ raise SystemExit("error: regenerating needs the terraform-provider-aws, aws-cdk and botocore checkouts "
+ "and the aws-docs/partition-guides snapshots under local/aws")
- document = build(args.terraform, args.cdk, args.guides, json.loads(args.partitions.read_text(encoding="utf-8")))
+ document = build(args.terraform, args.cdk, args.guides, args.botocore_data,
+ json.loads(args.partitions.read_text(encoding="utf-8")))
rendered = render(document)
if args.check:
current = args.output.read_text(encoding="utf-8") if args.output.exists() else ""
diff --git a/tools/aws/test_regen_region_facts.py b/tools/aws/test_regen_region_facts.py
index 2893b4d38a..a718cb2140 100644
--- a/tools/aws/test_regen_region_facts.py
+++ b/tools/aws/test_regen_region_facts.py
@@ -120,26 +120,47 @@ GUIDE_CHINA_MARKDOWN = """
The Amazon Web Services Management Console is available in both Chinese and English and can be accessed at [https://console.amazonaws.cn](https://console.amazonaws.cn). The Amazon Web Services Management Console is dedicated to the China Regions.
"""
GUIDES = {"aws": GUIDE_COMMERCIAL, "aws-cn": GUIDE_CHINA, "aws-us-gov": GUIDE_GOVCLOUD}
+GUIDE_CHINA_CLOUDFRONT = """
+You can\u2019t use the default CloudFront domain, *.cloudfront.cn, to serve content. You must add an alternate domain name.
+"""
+GUIDE_GOVCLOUD_APIGATEWAY = (
+ "Amazon API Gateway edge-optimized API and edge-optimized custom domain name are not available.\n\n"
+ "The Amazon Route\u00a053 Hosted Zone ID for the regional endpoint in the AWS GovCloud (US-West) Region is"
+ " Z1K6XKP9SAGWDV. The Amazon Route\u00a053 Hosted Zone ID for the regional endpoint in the AWS GovCloud (US-East)"
+ " Region is Z3SE9ATJYCRCZJ.\n")
+BOTOCORE_CLOUDFRONT = json.dumps({"shapes": {
+ "Distribution": {"members": {"DomainName": {"shape": "string", "documentation":
+ "<p>The distribution's CloudFront domain name. For example: <code>d111111abcdef8.cloudfront.net</code>.</p>"}}},
+ "DistributionSummary": {"members": {"DomainName": {"shape": "string", "documentation":
+ "<p>The domain name that corresponds to the distribution, for example,"
+ " <code>d111111abcdef8.cloudfront.net</code>.</p>"}}},
+}})
PARTITIONS = {"partitions": [
{"id": "aws", "dnsSuffix": "amazonaws.com", "regions": [{"id": "us-east-1"}, {"id": "af-south-1"}]},
{"id": "aws-cn", "dnsSuffix": "amazonaws.com.cn", "regions": [{"id": "cn-north-1"}, {"id": "cn-northwest-1"}]},
- {"id": "aws-us-gov", "dnsSuffix": "amazonaws.com", "regions": [{"id": "us-gov-east-1"}]},
+ {"id": "aws-us-gov", "dnsSuffix": "amazonaws.com", "regions": [
+ {"id": "us-gov-east-1", "description": "AWS GovCloud (US-East)"},
+ {"id": "us-gov-west-1", "description": "AWS GovCloud (US-West)"}]},
{"id": "aws-iso-f", "dnsSuffix": "csp.hci.ic.gov", "regions": [{"id": "us-isof-south-1"}]},
]}
-def write_sources(tmp_path: Path) -> tuple[Path, Path, Path]:
+def write_sources(tmp_path: Path) -> tuple[Path, Path, Path, Path]:
tf = tmp_path / "terraform"
cdk = tmp_path / "cdk"
guides = tmp_path / "guides"
+ botocore = tmp_path / "botocore"
for root, rel, body in [
(tf, r.TF_ELB, GO_ELB), (tf, r.TF_ELBV2, GO_ELBV2), (tf, r.TF_S3, GO_S3), (tf, r.TF_CLIENT, GO_CLIENT),
(cdk, r.CDK_FACT_TABLES, TS_FACTS), (cdk, r.CDK_CLOUDFRONT_TARGET, TS_CLOUDFRONT), (cdk, r.CDK_VPC_ENDPOINT, TS_VPCE),
+ (guides, r.CLOUDFRONT_SUFFIX_SENTENCES["aws-cn"][0], GUIDE_CHINA_CLOUDFRONT),
+ (guides, r.APIGATEWAY_REGIONAL_ZONE_PAGE, GUIDE_GOVCLOUD_APIGATEWAY),
+ (botocore, r.BOTOCORE_CLOUDFRONT, BOTOCORE_CLOUDFRONT),
] + [(guides, r.CONSOLE_HOST_SENTENCES[partition][0], body) for partition, body in GUIDES.items()]:
path = root / rel
path.parent.mkdir(parents=True, exist_ok=True)
- path.write_text(body)
- return tf, cdk, guides
+ path.write_text(body, encoding="utf-8")
+ return tf, cdk, guides, botocore
@pytest.mark.parametrize("camel,region", [
@@ -179,35 +200,73 @@ def test_vpc_endpoint_exceptions_are_per_region_and_service():
def test_console_hosts_come_from_the_guide_sentences(tmp_path):
- _, _, guides = write_sources(tmp_path)
+ _, _, guides, _ = write_sources(tmp_path)
assert r.parse_console_hosts(guides) == {"aws": "console.aws.amazon.com", "aws-cn": "console.amazonaws.cn",
"aws-us-gov": "console.amazonaws-us-gov.com"}
assert r.guides_present(guides)
assert not r.guides_present(tmp_path / "nowhere")
+def test_the_china_cloudfront_suffix_parses_from_the_markdown_escaped_form():
+ page, sentence = r.CLOUDFRONT_SUFFIX_SENTENCES["aws-cn"]
+ markdown = GUIDE_CHINA_CLOUDFRONT.replace("*.cloudfront.cn", "\\*.cloudfront.cn")
+ assert "\\*.cloudfront.cn" in markdown
+ assert r.parse_guide_sentence(markdown, sentence, page) == "cloudfront.cn"
+
+
def test_the_china_console_host_parses_from_the_markdown_link_form():
page, sentence = r.CONSOLE_HOST_SENTENCES["aws-cn"]
- assert r.parse_console_host(GUIDE_CHINA_MARKDOWN, sentence, page) == "console.amazonaws.cn"
+ assert r.parse_guide_sentence(GUIDE_CHINA_MARKDOWN, sentence, page) == "console.amazonaws.cn"
def test_a_guide_page_without_its_sentence_or_with_two_hosts_is_an_error():
page, sentence = r.CONSOLE_HOST_SENTENCES["aws-us-gov"]
with pytest.raises(ValueError, match="configure-account"):
- r.parse_console_host("Sign in to the AWS GovCloud (US) console.", sentence, page)
+ r.parse_guide_sentence("Sign in to the AWS GovCloud (US) console.", sentence, page)
with pytest.raises(ValueError, match="console.example.com"):
- r.parse_console_host(GUIDE_GOVCLOUD + GUIDE_GOVCLOUD.replace("console.amazonaws-us-gov.com", "console.example.com"),
+ r.parse_guide_sentence(GUIDE_GOVCLOUD + GUIDE_GOVCLOUD.replace("console.amazonaws-us-gov.com", "console.example.com"),
sentence, page)
# the China guide's other spelling sits in a different sentence and is not picked up
page, sentence = r.CONSOLE_HOST_SENTENCES["aws-cn"]
with pytest.raises(ValueError, match="console.md"):
- r.parse_console_host("a graphical user interface that you access online at https://console.www.amazonaws.cn .",
+ r.parse_guide_sentence("a graphical user interface that you access online at https://console.www.amazonaws.cn .",
sentence, page)
+def test_cloudfront_suffixes_come_from_the_model_example_and_the_china_guide(tmp_path):
+ _, _, guides, botocore = write_sources(tmp_path)
+ assert r.parse_cloudfront_suffixes(botocore, guides) == {"aws": "cloudfront.net", "aws-cn": "cloudfront.cn"}
+ with pytest.raises(ValueError, match="disagree"):
+ r.parse_botocore_cloudfront_suffix(BOTOCORE_CLOUDFRONT.replace("cloudfront.net", "cloudfront.example", 1))
+ with pytest.raises(ValueError, match="DistributionSummary"):
+ r.parse_botocore_cloudfront_suffix(BOTOCORE_CLOUDFRONT.replace(
+ "for example, <code>d111111abcdef8.cloudfront.net</code>", "a CloudFront name"))
+ page, sentence = r.CLOUDFRONT_SUFFIX_SENTENCES["aws-cn"]
+ with pytest.raises(ValueError, match="cloudfront.md"):
+ r.parse_guide_sentence("CloudFront is available in the China Regions.", sentence, page)
+
+
+def test_api_gateway_regional_zones_resolve_the_region_by_its_description():
+ assert r.parse_apigateway_regional_zones(GUIDE_GOVCLOUD_APIGATEWAY, PARTITIONS) == {
+ "us-gov-west-1": "Z1K6XKP9SAGWDV", "us-gov-east-1": "Z3SE9ATJYCRCZJ"}
+ with pytest.raises(ValueError, match="no regional endpoint hosted zone"):
+ r.parse_apigateway_regional_zones("Portals are not available.", PARTITIONS)
+ with pytest.raises(ValueError, match="not a region description"):
+ r.parse_apigateway_regional_zones(GUIDE_GOVCLOUD_APIGATEWAY.replace("(US-West)", "(US-North)"), PARTITIONS)
+ with pytest.raises(ValueError, match="two regional hosted zones for us-gov-east-1"):
+ r.parse_apigateway_regional_zones(GUIDE_GOVCLOUD_APIGATEWAY.replace("(US-West)", "(US-East)"), PARTITIONS)
+ # a page that loses one region's sentence is refused rather than dropping that zone
+ west_only = GUIDE_GOVCLOUD_APIGATEWAY.replace(
+ " The Amazon Route\u00a053 Hosted Zone ID for the regional endpoint in the AWS GovCloud (US-East)"
+ " Region is Z3SE9ATJYCRCZJ.", "")
+ assert "(US-West)" in west_only and "(US-East)" not in west_only
+ with pytest.raises(ValueError, match="no regional endpoint hosted zone for us-gov-east-1"):
+ r.parse_apigateway_regional_zones(west_only, PARTITIONS)
+
+
def test_build_merges_sources_with_terraform_winning_the_s3_conflicts(tmp_path):
- tf, cdk, guides = write_sources(tmp_path)
- document = r.build(tf, cdk, guides, PARTITIONS)
+ tf, cdk, guides, botocore = write_sources(tmp_path)
+ document = r.build(tf, cdk, guides, botocore, PARTITIONS)
regions = document["regions"]
assert regions["us-east-1"] == {"albHostedZoneId": "Z35SXDOTRQ7X7K", "classicElbHostedZoneId": "Z35SXDOTRQ7X7K",
"nlbHostedZoneId": "Z26RNL4JYFTOTI", "s3WebsiteHostedZoneId": "Z3AQBSTGFYJSTF"}
@@ -221,24 +280,30 @@ def test_build_merges_sources_with_terraform_winning_the_s3_conflicts(tmp_path):
assert partitions["aws-cn"] == {"vpcEndpointServiceNamePrefix": "cn.com.amazonaws.vpce",
"cloudfrontHostedZoneId": "Z3RFFRIM2A3IF5",
"samlSignOnUrl": "https://signin.amazonaws.cn/saml",
- "consoleHost": "console.amazonaws.cn"}
- assert partitions["aws-us-gov"]["consoleHost"] == "console.amazonaws-us-gov.com"
+ "consoleHost": "console.amazonaws.cn",
+ "cloudfrontDomainSuffix": "cloudfront.cn"}
+ assert partitions["aws"]["cloudfrontDomainSuffix"] == "cloudfront.net"
+ assert partitions["aws-us-gov"] == {"vpcEndpointServiceNamePrefix": "com.amazonaws.vpce",
+ "consoleHost": "console.amazonaws-us-gov.com"}
+ assert regions["us-gov-west-1"] == {"apiGatewayRegionalHostedZoneId": "Z1K6XKP9SAGWDV"}
+ assert regions["us-gov-east-1"]["apiGatewayRegionalHostedZoneId"] == "Z3SE9ATJYCRCZJ"
+ assert "apiGatewayRegionalHostedZoneId" not in regions["us-east-1"]
assert partitions["aws-iso-f"] == {"vpcEndpointServiceNamePrefix": "gov.ic.hci.csp.vpce"}
def test_build_refuses_a_region_the_partition_data_does_not_publish(tmp_path):
- tf, cdk, guides = write_sources(tmp_path)
+ tf, cdk, guides, botocore = write_sources(tmp_path)
with pytest.raises(ValueError, match="af-south-1"):
- r.build(tf, cdk, guides, {"partitions": [p for p in PARTITIONS["partitions"] if p["id"] != "aws"]
+ r.build(tf, cdk, guides, botocore, {"partitions": [p for p in PARTITIONS["partitions"] if p["id"] != "aws"]
+ [{"id": "aws", "dnsSuffix": "amazonaws.com", "regions": [{"id": "us-east-1"}]}]})
def test_cli_write_check_and_self_check(tmp_path, capsys):
- tf, cdk, guides = write_sources(tmp_path)
+ tf, cdk, guides, botocore = write_sources(tmp_path)
partitions = tmp_path / "partitions.json"
partitions.write_text(json.dumps(PARTITIONS))
output = tmp_path / "out" / "region-facts.json"
- common = ["--terraform", str(tf), "--cdk", str(cdk), "--guides", str(guides),
+ common = ["--terraform", str(tf), "--cdk", str(cdk), "--guides", str(guides), "--botocore-data", str(botocore),
"--partitions", str(partitions), "--output", str(output)]
assert r.main(["--check"] + common) == 1
assert r.main(common) == 0
@@ -248,10 +313,12 @@ def test_cli_write_check_and_self_check(tmp_path, capsys):
assert r.main(["--check", "--terraform", str(tmp_path / "nowhere"), "--cdk", str(tmp_path / "nowhere"),
"--partitions", str(partitions), "--output", str(output)]) == 0
assert "shape verified only" in capsys.readouterr().out
- # nor with the checkouts but without the guide snapshots
- assert r.main(["--check", "--terraform", str(tf), "--cdk", str(cdk), "--guides", str(tmp_path / "nowhere"),
- "--partitions", str(partitions), "--output", str(output)]) == 0
- assert "shape verified only" in capsys.readouterr().out
+ # nor with the checkouts but without the guide snapshots, or without the botocore model
+ for absent in ("--guides", "--botocore-data"):
+ partial = [str(tmp_path / "nowhere") if previous == absent else argument
+ for previous, argument in zip([""] + common, common)]
+ assert r.main(["--check"] + partial) == 0, absent
+ assert "shape verified only" in capsys.readouterr().out
output.write_text('{"partitions": {}, "regions": {"us-east-1": {"albHostedZoneId": "nope"}}}')
assert r.main(["--check", "--terraform", str(tmp_path / "nowhere"), "--cdk", str(tmp_path / "nowhere"),
"--partitions", str(partitions), "--output", str(output)]) == 1
@@ -259,17 +326,18 @@ def test_cli_write_check_and_self_check(tmp_path, capsys):
def test_repo_file_matches_the_local_checkouts_when_present():
if (not (r.LOCAL_TERRAFORM / r.TF_ELB).exists() or not (r.LOCAL_CDK / r.CDK_FACT_TABLES).exists()
- or not r.guides_present(r.LOCAL_GUIDES)):
+ or not r.guides_present(r.LOCAL_GUIDES)
+ or not (r.LOCAL_BOTOCORE_DATA / r.BOTOCORE_CLOUDFRONT).exists()):
pytest.skip("local/aws checkouts not present")
- document = r.build(r.LOCAL_TERRAFORM, r.LOCAL_CDK, r.LOCAL_GUIDES, json.loads(r.PARTITIONS.read_text(encoding="utf-8")))
+ document = r.build(r.LOCAL_TERRAFORM, r.LOCAL_CDK, r.LOCAL_GUIDES, r.LOCAL_BOTOCORE_DATA, json.loads(r.PARTITIONS.read_text(encoding="utf-8")))
assert r.strip_source(r.render(document)) == r.strip_source(r.OUTPUT.read_text(encoding="utf-8"))
def test_self_check_holds_the_values_to_what_partitions_json_publishes(tmp_path):
partitions = tmp_path / "partitions.json"
partitions.write_text(json.dumps(PARTITIONS))
- tf, cdk, guides = write_sources(tmp_path)
- good = r.build(tf, cdk, guides, PARTITIONS)
+ tf, cdk, guides, botocore = write_sources(tmp_path)
+ good = r.build(tf, cdk, guides, botocore, PARTITIONS)
output = tmp_path / "region-facts.json"
def problems(mutate):
@@ -298,6 +366,16 @@ def test_self_check_holds_the_values_to_what_partitions_json_publishes(tmp_path)
"missing China console host": lambda d: d["partitions"]["aws-cn"].pop("consoleHost"),
"missing GovCloud console host": lambda d: d["partitions"]["aws-us-gov"].pop("consoleHost"),
"missing commercial console host": lambda d: d["partitions"]["aws"].pop("consoleHost"),
+ "missing commercial CloudFront suffix": lambda d: d["partitions"]["aws"].pop("cloudfrontDomainSuffix"),
+ "missing China CloudFront suffix": lambda d: d["partitions"]["aws-cn"].pop("cloudfrontDomainSuffix"),
+ "missing us-gov-west-1 API Gateway zone": lambda d: d["regions"]["us-gov-west-1"].pop(
+ "apiGatewayRegionalHostedZoneId"),
+ "missing us-gov-east-1 API Gateway zone": lambda d: d["regions"]["us-gov-east-1"].pop(
+ "apiGatewayRegionalHostedZoneId"),
+ "CloudFront suffix that is a whole name": lambda d: d["partitions"][partition].update(
+ cloudfrontDomainSuffix="d111111abcdef8.cloudfront.net"),
+ "API Gateway zone that is not a zone id": lambda d: d["regions"][region].update(
+ apiGatewayRegionalHostedZoneId="regional"),
"missing partition": lambda d: d["partitions"].pop(partition),
"partition entry not an object": lambda d: d["partitions"].update({partition: "invalid"}),
"region entry not an object": lambda d: d["regions"].update({region: ["invalid"]}),