Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(lambdaweb): serve the MultiRegion global domain in the default region (#6460)

TATSUNO “Taz” Yasuhiroacb9fd2cd695
diff --git a/docs/services/lambda-web-functions.md b/docs/services/lambda-web-functions.md
index d61a43d7ff..7e40656d26 100644
--- a/docs/services/lambda-web-functions.md
+++ b/docs/services/lambda-web-functions.md
@@ -67,17 +67,17 @@ curl -H "Host: <domainName>" http://localhost:4566/
The `Host` must match the domain exactly, as on AWS: another case or another port is `404` with `lambda-web-error-type: Platform.InvalidEndpoint`.
-A CloudFront distribution of this Floci can use the endpoint as a custom origin, named by its `domainName` without the port as on AWS. Under an `always-amz-auth` origin access control, CloudFront signs the requests to an `IamAuth` endpoint, and the function's resource policy has to allow `cloudfront.amazonaws.com` when IAM enforcement is on. See [CloudFront](cloudfront.md).
+A CloudFront distribution of this Floci can use the endpoint as a custom origin, named by its `domainName` without the port as on AWS. Under an `always-amz-auth` origin access control, CloudFront signs the requests to an `IamAuth` endpoint, and the function's resource policy has to allow `cloudfront.amazonaws.com` when IAM enforcement is on. Its `aws:RequestedRegion` is the region that serves the request, as on AWS. See [CloudFront](cloudfront.md).
An execution environment is a `public.ecr.aws/lambda/nodejs:24` container that a request starts when every environment of the endpoint is full. The code is the revision's S3 object unpacked into `/var/task`. Node starts `/var/runtime/index.js` with the flags AWS uses, as on AWS, and that runtime imports `index.js` or the file `AWS_LAMBDA_NODEJS_ENTRYPOINT` names. So `require.main` is not the app's module, as on AWS. A `--max-old-space-size` or `--max-semi-space-size` in `NODE_OPTIONS` takes the place of both default heap flags, as on AWS. Other flags there do not replace a default flag, so `--max-http-header-size` in `NODE_OPTIONS` has no effect, and a flag Node does not allow in `NODE_OPTIONS` fails the start with `502 Runtime.ExitError`, as on AWS. The app sees `AWS_LAMBDA_HTTP_ENDPOINT=0.0.0.0:3000`, `AWS_LAMBDA_INITIALIZATION_TYPE=native-http`, `AWS_EXECUTION_ENV`, `AWS_LAMBDA_FUNCTION_MEMORY_SIZE=2048`, `AWS_LAMBDA_LOG_LEVEL`, `AWS_LAMBDA_METADATA_API`, `AWS_LAMBDA_METADATA_TOKEN`, `AWS_LAMBDA_RUNTIME_API`, `AWS_ACCOUNT_ID`, `AWS_REGION`, credentials and `AWS_ENDPOINT_URL` for Floci, `NODE_PATH`, and its own variables. The server has 9 seconds to listen. An environment that sees no request for `floci.services.lambda.container-idle-timeout-seconds` is stopped with `SIGTERM` and one second, as on scale to zero. Node runs under Docker's init, so a server without a `SIGTERM` handler stops at once, as on AWS. A process that exits between requests is replaced by a fresh environment on the next request.
Each endpoint scales on its own, as on AWS. An environment takes up to the revision's `maxConcurrencyPerEnvironment` requests at once, the least busy one first, so requests rotate over the environments. A request that finds every environment full starts another one, or waits for one that is starting and has room. When the endpoint already runs `maxEnvironments`, an idle environment of another revision makes room, and without one the request is refused with `429` and `Platform.EndpointMaxEnvThrottle`. A `rateLimit` admits a burst of up to `rateLimit` requests that refills at `rateLimit` per second, and refuses the rest with `429` and `Platform.EndpointRequestThrottle`. AWS adjusts the concurrency of each environment under `maxConcurrencyPerEnvironment` from the measured load, so a burst there can spread over more environments than in Floci, and the account quotas are not applied.
-Each regional domain of a `PerRegion` endpoint is served in its own region, as on AWS. Its environments see that region in `AWS_REGION` and `AWS_DEFAULT_REGION` and get that region's zone from the metadata endpoint, and `maxEnvironments` and `rateLimit` hold in each region on its own. Floci serves the global domain of a `MultiRegion` endpoint in the home region, where AWS serves it from a region near the caller.
+Each regional domain of a `PerRegion` endpoint is served in its own region, as on AWS. Its environments see that region in `AWS_REGION` and `AWS_DEFAULT_REGION` and get that region's zone from the metadata endpoint, and `maxEnvironments` and `rateLimit` hold in each region on its own. AWS serves the global domain of a `MultiRegion` endpoint from the region nearest the caller. Floci takes `floci.default-region` for where the caller is, so it serves the global domain in that region when the endpoint has it, and in the home region otherwise. The serving region then works as for a regional domain.
Requests are forwarded over HTTP/1.1 with the headers AWS adds and drops: `lambda-web-request-id`, `lambda-web-endpoint-name`, `x-forwarded-proto`, `x-forwarded-port` and `Connection: close` are set, the client address is appended as an `X-Forwarded-For` line, and `Connection`, `Keep-Alive`, `Upgrade`, `Expect`, `Trailer`, the `Proxy-*` headers, `X-Real-IP`, the method override headers and any `lambda-web-*` header the client sent are dropped. A body without a `Content-Length` reaches the app chunked. The path reaches the app without dot segments, as on AWS, and repeated slashes stay. Responses carry `lambda-web-request-id` and `lambda-web-logstream`, lose their connection headers and trailers, and stream as the app writes them. Requests with two revisions in `revisionWeights` are split by weight.
-An `IamAuth` endpoint takes a request signed with SigV4 for the service `lambda` in the region of the domain, in the `Authorization` header or in a presigned query. The request must carry `X-Amz-Content-SHA256`. Lambda takes its value as the payload hash and does not read the body, so `UNSIGNED-PAYLOAD` works. The signing time must be within five minutes. Floci verifies keys whose secret it holds: `test` with the secret `test`, IAM user keys and STS credentials with their session token. Any other key is refused. When `floci.services.iam.enforcement-enabled` is on, the signer also needs `lambda:InvokeWebFunctionEndpoint` on the endpoint ARN. Within one account the identity policy or the function's resource policy can allow it, and across accounts both must. `lambda:Resource/WebFunctionAuthType` and the function's tags as `aws:ResourceTag/<key>` are in the context, and `aws:RequestedRegion` is the region of the domain, as on AWS. The `test` key is not evaluated, as elsewhere in Floci. A `MultiRegion` endpoint also takes SigV4 for the region that serves the request, which in Floci is always the home region. Its SigV4a requests are refused, because Floci does not verify SigV4a yet. On every endpoint, an `Authorization` that is an AWS signature is removed before the request reaches the app, together with the `X-Amz-Security-Token` header. Other `Authorization` values pass. A presigned query reaches the app unchanged, as on AWS.
+An `IamAuth` endpoint takes a request signed with SigV4 for the service `lambda` in the region of the domain, in the `Authorization` header or in a presigned query. The request must carry `X-Amz-Content-SHA256`. Lambda takes its value as the payload hash and does not read the body, so `UNSIGNED-PAYLOAD` works. The signing time must be within five minutes. Floci verifies keys whose secret it holds: `test` with the secret `test`, IAM user keys and STS credentials with their session token. Any other key is refused. When `floci.services.iam.enforcement-enabled` is on, the signer also needs `lambda:InvokeWebFunctionEndpoint` on the endpoint ARN. Within one account the identity policy or the function's resource policy can allow it, and across accounts both must. `lambda:Resource/WebFunctionAuthType` and the function's tags as `aws:ResourceTag/<key>` are in the context, and `aws:RequestedRegion` is the region that serves the request, as on AWS. The `test` key is not evaluated, as elsewhere in Floci. The global domain of a `MultiRegion` endpoint takes SigV4 for the region that serves the request, as on AWS. Its SigV4a requests are refused, because Floci does not verify SigV4a yet. On every endpoint, an `Authorization` that is an AWS signature is removed before the request reaches the app, together with the `X-Amz-Security-Token` header. Other `Authorization` values pass. A presigned query reaches the app unchanged, as on AWS.
The revision's `timeoutSeconds` bounds the whole exchange, as on AWS: a response that has not started by then is the 504 page, and one that is still streaming is cut. The request body is taken in full before the answer, so a slow upload gets its 504 once the body is in.
diff --git a/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuth.java b/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuth.java
index 59f84286f7..a63ad8142e 100644
--- a/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuth.java
+++ b/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuth.java
@@ -137,6 +137,8 @@ public class LambdaWebIamAuth {
Map<String, List<String>> context = resourceContext(target);
context.putAll(IamConditionContextResolver.serviceCallContext(principal, endpointArn, target.accountId(),
distributionArn, AwsArnUtils.parse(distributionArn).accountId()));
+ // The endpoint ARN names the home region, not the region that serves.
+ context.put("aws:RequestedRegion", List.of(target.servingRegion()));
String policy = target.function().policy;
boolean allowed = evaluator.evaluateServicePrincipal(isBlank(policy) ? null : List.of(policy), principal,
ACTION, endpointArn, context) == Decision.ALLOW;
diff --git a/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebService.java b/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebService.java
index 83d3cece2a..14059e827c 100644
--- a/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebService.java
@@ -230,11 +230,17 @@ public class LambdaWebService implements Resettable {
return Optional.ofNullable(originHosts.get(host));
}
- /** Floci serves the MultiRegion global domain in the home region. */
+ /**
+ * AWS serves the MultiRegion global domain in the region nearest the caller. Floci takes the default region for
+ * that when the endpoint has it, and the home region otherwise.
+ */
private void indexDomains(WebFunction fn, WebFunctionEndpoint endpoint) {
AwsArnUtils.Arn arn = AwsArnUtils.parse(fn.functionArn);
+ String defaultRegion = regionResolver.getDefaultRegion();
+ String servingRegion = "MultiRegion".equals(endpoint.endpointType) && endpoint.regions.contains(defaultRegion)
+ ? defaultRegion : arn.region();
index(endpoint.domainName, new EndpointLocator(arn.accountId(), arn.region(), fn.functionName,
- endpoint.endpointName, arn.region()));
+ endpoint.endpointName, servingRegion));
endpoint.regionalDomainNames.forEach((region, domain) -> index(domain, new EndpointLocator(arn.accountId(),
arn.region(), fn.functionName, endpoint.endpointName, region)));
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebDataPlaneIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebDataPlaneIntegrationTest.java
index fe7827dac7..dca520e42d 100644
--- a/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebDataPlaneIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebDataPlaneIntegrationTest.java
@@ -17,6 +17,7 @@ import java.time.Duration;
import java.util.List;
import java.util.Map;
+import static io.github.hectorvent.floci.testing.EnforcementFixtures.auth;
import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.matchesPattern;
@@ -331,6 +332,34 @@ class LambdaWebDataPlaneIntegrationTest {
given().when().delete(BASE + "/web-functions/" + functionName).then().statusCode(204);
}
+ @Test
+ void theGlobalDomainOfAFunctionInAnotherRegionCountsAPageInTheDefaultRegion() {
+ String functionName = "dp-global-" + Long.toString(System.nanoTime(), 36);
+ createFunction("us-west-2", functionName, "ApplicationManaged", LambdaWebZips.indexJs(""), ROLE);
+ String west = auth("test", "us-west-2", "lambda");
+ String endpoints = BASE + "/web-functions/" + functionName + "/endpoints";
+ String revisionId = given().header("Authorization", west).when().get(endpoints + "/dev").then().statusCode(200)
+ .extract().path("revisionWeights[0].revisionId");
+ String global = given().header("Authorization", west).contentType("application/json")
+ .body("""
+ {"endpointName": "global", "endpointType": "MultiRegion", "regions": ["us-east-1"],
+ "authType": "IamAuth", "revisionWeights": [{"revisionId": "%s", "weight": 100}]}"""
+ .formatted(revisionId))
+ .when().put(endpoints)
+ .then().statusCode(202)
+ .extract().path("domainName");
+ given().header("Host", global).when().get("/").then().statusCode(403);
+
+ Map<String, String> endpoint = Map.of("FunctionName", functionName, "EndpointName", "global");
+ metrics.awaitPublished(Duration.ofSeconds(5));
+ assertEquals(1, LambdaWebMetricStatistics.of("us-east-1", LambdaWebMetrics.NAMESPACE,
+ "HTTPResponseStatusClass4XX", endpoint, "Sum"));
+ assertEquals(1, LambdaWebMetricStatistics.of("us-west-2", LambdaWebMetrics.NAMESPACE,
+ "HTTPResponseStatusClass4XX", endpoint, "Sum"));
+ given().header("Authorization", west).when().delete(BASE + "/web-functions/" + functionName)
+ .then().statusCode(204);
+ }
+
@Test
void otherHostsStillReachTheControlPlane() {
given().when().get(BASE + "/web-account-settings").then().statusCode(200);
@@ -349,10 +378,14 @@ class LambdaWebDataPlaneIntegrationTest {
}
private String createFunction(String functionName, String authType, byte[] code, String role) {
+ return createFunction("us-east-1", functionName, authType, code, role);
+ }
+
+ private String createFunction(String region, String functionName, String authType, byte[] code, String role) {
LambdaWebRoles.trustLambda(iamService, role);
s3Service.createBucket(BUCKET, "us-east-1");
s3Service.putObject(BUCKET, "app.zip", code, "application/zip", Map.of());
- return given().contentType("application/json")
+ return given().header("Authorization", auth("test", region, "lambda")).contentType("application/json")
.body("""
{"functionName": "%s",
"revisionConfig": {
diff --git a/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuthTest.java b/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuthTest.java
index b83dfd9f51..da12b2a363 100644
--- a/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuthTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebIamAuthTest.java
@@ -280,10 +280,27 @@ class LambdaWebIamAuthTest {
assertFalse(allowsCloudFront(iamAuth.cloudFrontSignature(DISTRIBUTION_ARN)));
}
+ @Test
+ void underEnforcementCloudFrontRequestsTheRegionThatServes() {
+ when(config.services().iam().enforcementEnabled()).thenReturn(true);
+ Map<String, String> signature = iamAuth.cloudFrontSignature(DISTRIBUTION_ARN);
+ String allow = cloudFrontStatement("Allow", "aws:SourceArn", DISTRIBUTION_ARN);
+ function.policy = cloudFrontPolicy(allow, cloudFrontStatement("Deny", "aws:RequestedRegion", "us-west-2"));
+ assertTrue(allowsCloudFront(signature));
+ assertFalse(allowsCloudFront(signature, WEST_HOST, "us-west-2"));
+ function.policy = cloudFrontPolicy(allow, cloudFrontStatement("Deny", "aws:RequestedRegion", "us-east-1"));
+ assertFalse(allowsCloudFront(signature));
+ assertTrue(allowsCloudFront(signature, WEST_HOST, "us-west-2"));
+ }
+
private boolean allowsCloudFront(Map<String, String> signature) {
- MultiMap headers = MultiMap.caseInsensitiveMultiMap().add("Host", HOST).addAll(signature);
+ return allowsCloudFront(signature, HOST, "us-east-1");
+ }
+
+ private boolean allowsCloudFront(Map<String, String> signature, String host, String servingRegion) {
+ MultiMap headers = MultiMap.caseInsensitiveMultiMap().add("Host", host).addAll(signature);
return iamAuth.allows(new Inbound("POST", "/", null, headers, MultiMap.caseInsensitiveMultiMap()),
- new WebFunctionTarget(ACCOUNT, "us-east-1", function, endpoint, "us-east-1"));
+ new WebFunctionTarget(ACCOUNT, "us-east-1", function, endpoint, servingRegion));
}
private static String cloudFrontPolicy(String... statements) {
diff --git a/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebServiceTest.java
index faeaaa8ed8..9c9a49f3f2 100644
--- a/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/lambdaweb/LambdaWebServiceTest.java
@@ -651,6 +651,28 @@ class LambdaWebServiceTest {
assertTrue(service.endpointDomain(host).isEmpty(), "a deleted endpoint's domain is gone");
}
+ @Test
+ void theGlobalDomainIsServedInTheDefaultRegionWhenTheEndpointHasIt() {
+ LambdaWebService service = service();
+ service.createFunction("us-west-2", createRequest("app"));
+ String revisionId = service.getEndpoint("us-west-2", "app", "dev").revisionWeights.get(0).revisionId;
+ String near = service.createEndpoint("us-west-2", "app", multiRegion("near", revisionId, REGION)).domainName;
+ String far = service.createEndpoint("us-west-2", "app", multiRegion("far", revisionId, "eu-west-1")).domainName;
+
+ LambdaWebService.WebFunctionTarget target = service.locate(near).orElseThrow();
+ assertEquals("us-west-2", target.region());
+ assertEquals(REGION, target.servingRegion());
+ assertEquals("us-west-2", service.locate(far).orElseThrow().servingRegion());
+ }
+
+ private ObjectNode multiRegion(String name, String revisionId, String region) {
+ ObjectNode request = mapper.createObjectNode().put("endpointName", name).put("endpointType", "MultiRegion")
+ .put("authType", "ApplicationManaged");
+ request.putArray("regions").add(region);
+ request.putArray("revisionWeights").addObject().put("revisionId", revisionId).put("weight", 100);
+ return request;
+ }
+
@Test
void eachRevisionKeepsTheCodeTheObjectHadWhenItWasCreated() throws IOException {
S3Service s3 = mock(S3Service.class);