Light, fluffy, and always free - The AWS Local Emulator alternative
fix(lambda): make the code-volume populate concurrency configurable (#2830)
Brian Taylor968ff78e2529
diff --git a/docs/configuration/environment-variables.md b/docs/configuration/environment-variables.mdindex da64eddbe8..bc179c84da 100644--- a/docs/configuration/environment-variables.md+++ b/docs/configuration/environment-variables.md@@ -197,6 +197,7 @@ See [Initialization Hooks](./initialization-hooks.md) for lifecycle phases and s| `FLOCI_SERVICES_LAMBDA_CONTAINER_IDLE_TIMEOUT_SECONDS` | `300` | Seconds of inactivity before an idle Lambda container is removed || `FLOCI_SERVICES_LAMBDA_REGION_CONCURRENCY_LIMIT` | `1000` | Maximum concurrent Lambda invocations across all functions in a region || `FLOCI_SERVICES_LAMBDA_UNRESERVED_CONCURRENCY_MIN` | `100` | Minimum unreserved concurrency pool |+| `FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY` | `max(2, cpus/2)` | Maximum concurrent first-time code-volume populates (functions whose unpacked code is at least 32 MB). The default is derived from the CPU count the JVM sees, so a CPU-constrained Floci container collapses it to 2 and concurrent cold starts of distinct functions serialise into pairs; set this to decouple the cap from the CPU allocation || `FLOCI_SERVICES_LAMBDA_HOT_RELOAD_ENABLED` | `false` | Watch Lambda code directories for changes and reload without redeployment || `FLOCI_SERVICES_LAMBDA_HOT_RELOAD_ALLOWED_PATHS` | _(none)_ | Comma-separated host paths that hot-reload is allowed to watch || `FLOCI_SERVICES_LAMBDA_DOCKER_NETWORK` | _(none)_ | Docker network for Lambda containers (overrides `FLOCI_SERVICES_DOCKER_NETWORK`) |diff --git a/docs/services/lambda.md b/docs/services/lambda.mdindex 949d82480c..d402f1cc8d 100644--- a/docs/services/lambda.md+++ b/docs/services/lambda.md@@ -207,6 +207,7 @@ These AWS Lambda operations have no handler in Floci. Calls will return `404` or| `FLOCI_SERVICES_LAMBDA_EXTRA_HOSTS` | *(unset)* | Comma-separated `hostname:ip` entries added to each Lambda container's `/etc/hosts`; `ip` may be `host-gateway`, mirroring `docker run --add-host` || `FLOCI_SERVICES_LAMBDA_DOCKER_HOST_OVERRIDE` | *(unset)* | Explicit host/IP that spawned Lambda containers use to reach Floci's Runtime API, bypassing auto-detection || `FLOCI_SERVICES_LAMBDA_CONTAINER_NAME_PREFIX` | `floci` | Base name prefix for spawned Lambda containers and code volumes (e.g. `acme` → `acme-<function>-<id>` containers, `acme-code-<function>-<hash>` volumes). Must be a valid Docker name segment (`[A-Za-z0-9][A-Za-z0-9_.-]*`); invalid values are ignored with a warning |+| `FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY` | `max(2, cpus/2)` | Maximum concurrent first-time code-volume populates. See the note below || `FLOCI_SERVICES_LAMBDA_EXECUTOR` | `docker` | Execution backend: `docker` (containers) or `kubernetes` (pods) || `FLOCI_SERVICES_LAMBDA_KUBERNETES_NAMESPACE` | `default` | Namespace Lambda pods are created in || `FLOCI_SERVICES_LAMBDA_KUBERNETES_LABELS` | *(unset)* | Extra pod labels as comma-separated `key=value` entries |@@ -226,6 +227,25 @@ These AWS Lambda operations have no handler in Floci. Calls will return `404` ordocker volume prune --filter label=floci=true```+!!! note "Concurrent cold starts of large functions"+ A function whose unpacked code is at least 32 MB has that code streamed once into a+ read-only Docker volume, so later cold starts mount it instead of copying. Those+ first-time *populates* are capped — a burst of them overwhelms the Docker daemon — and+ the default cap is `max(2, cpus/2)`, derived from the CPU count the JVM sees.++ Because the JVM honours the container's cgroup CPU quota, running Floci with a small CPU+ allocation collapses the cap to 2, and a burst of cold starts across *distinct* large+ functions completes in waves of two rather than in parallel. Six such functions invoked+ at once take roughly three times the wall-clock of one. Raise the cap to decouple it from+ the CPU allocation:++ ```bash+ FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY=8+ ```++ Only first-time populates are gated. Warm containers, already-populated volumes, and+ functions under 32 MB are never serialised by this.+### Runtime API host overrideWhen a Lambda container starts, it calls back into Floci's Runtime API to fetchdiff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.javaindex 3a56d9cb1e..40fdf62332 100644--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java@@ -1840,6 +1840,21 @@ public interface EmulatorConfig {*/Optional<String> containerNamePrefix();+ /**+ * Maximum concurrent first-time code-volume populates. Populating streams a large+ * function's unpacked code into a helper container, so a burst of them can overwhelm+ * the Docker daemon; this caps how many run at once.+ *+ * <p>Unset derives {@code max(2, availableProcessors() / 2)}. That derivation reads the+ * JVM's view of the cgroup CPU quota, so a CPU-constrained Floci container collapses the+ * cap to 2 and concurrent cold starts of distinct functions serialize into pairs. Set+ * this to decouple the cap from the CPU allocation. Values below 1 are ignored with a+ * warning rather than deadlocking every populate.+ *+ * Env var: FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY+ */+ Optional<Integer> codeVolumePopulateConcurrency();+/*** Extra /etc/hosts entries added to every Lambda container, as "hostname:ip" pairs.* The ip may be the literal "host-gateway" to map to the Docker host, mirroringdiff --git a/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java b/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.javaindex 75f9ba72a3..9176617746 100644--- a/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java+++ b/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java@@ -146,6 +146,7 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {this.layerService = layerService;this.awsEnv = awsEnv;this.executionRoleCredentials = executionRoleCredentials;+ this.populateSemaphore = new java.util.concurrent.Semaphore(resolvePopulateConcurrency(config));}@PostConstruct@@ -603,12 +604,40 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {// overwhelmed the Docker daemon, so copies hung/failed and left half-built "Created" containers.// This gates ONLY the heavy populate — not every launch — so ordinary cold starts (volume mounts// for already-populated large code, or the small-code direct copy) are never serialized.- private static final java.util.concurrent.Semaphore POPULATE_SEMAPHORE =- new java.util.concurrent.Semaphore(Math.max(2, Runtime.getRuntime().availableProcessors() / 2));+ //+ // Per-instance rather than static so the permit count can come from configuration; Floci runs a+ // single @ApplicationScoped launcher, so this is still one gate for the whole emulator.+ private final java.util.concurrent.Semaphore populateSemaphore;++ /**+ * Permits for {@link #populateSemaphore}: the configured value when set and positive,+ * otherwise {@code max(2, availableProcessors() / 2)}.+ *+ * <p>The derived default reads the JVM's view of the cgroup CPU quota, so a CPU-constrained+ * Floci container collapses it to 2 and concurrent cold starts of distinct functions+ * serialize into pairs. The populate itself is IO-bound (streaming a tar into a helper+ * container), so CPU count is a weak proxy for how many the daemon can take — hence the+ * override. The default is unchanged, since the daemon overload the cap exists to prevent is+ * real and its safe ceiling is host-specific.+ */+ static int resolvePopulateConcurrency(EmulatorConfig config) {+ int derived = Math.max(2, Runtime.getRuntime().availableProcessors() / 2);+ Optional<Integer> configured = config.services().lambda().codeVolumePopulateConcurrency();+ if (configured.isEmpty()) {+ return derived;+ }+ int permits = configured.get();+ if (permits < 1) {+ LOG.warnv("Ignoring floci.services.lambda.code-volume-populate-concurrency {0}: must be "+ + "at least 1; using {1}", permits, derived);+ return derived;+ }+ return permits;+ }- private static void acquirePopulatePermit(String functionName) {+ private void acquirePopulatePermit(String functionName) {try {- POPULATE_SEMAPHORE.acquire();+ populateSemaphore.acquire();} catch (InterruptedException ie) {Thread.currentThread().interrupt();throw new RuntimeException("Interrupted while waiting to populate code volume for " + functionName, ie);@@ -871,7 +900,7 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {LOG.warnv("Could not remove code-volume helper {0}: {1}", helperId, e.getMessage());}}- POPULATE_SEMAPHORE.release();+ populateSemaphore.release();}}@@ -1097,7 +1126,7 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {Path sourceDir, String remotePath, String functionName,DirectoryTarWriter tarWriter, boolean failOnTarFailure) {// No per-copy gating here: the heavy /var/task populate for large code already holds a- // POPULATE_SEMAPHORE permit; small-code direct copies and layer copies are light enough+ // populateSemaphore permit; small-code direct copies and layer copies are light enough// to run unthrottled.try (java.io.PipedOutputStream pos = new java.io.PipedOutputStream();java.io.PipedInputStream pis = new java.io.PipedInputStream(pos, TAR_PIPE_BUFFER_BYTES)) {diff --git a/src/main/resources/application.yml b/src/main/resources/application.ymlindex 96cc8ace1b..4ee259e915 100644--- a/src/main/resources/application.yml+++ b/src/main/resources/application.yml@@ -265,6 +265,7 @@ floci:# aws-config-path: # Host path to bind-mount (read-only) at /opt/aws-config for real credential discovery# extra-hosts: # Extra /etc/hosts entries for Lambda containers, "hostname:ip" ("host-gateway" allowed as ip)# container-name-prefix: floci # FLOCI_SERVICES_LAMBDA_CONTAINER_NAME_PREFIX — base name prefix for spawned containers and code volumes ([A-Za-z0-9][A-Za-z0-9_.-]*; invalid values fall back to "floci")+ # code-volume-populate-concurrency: # FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY — max concurrent first-time code-volume populates (default max(2, cpus/2), which a CPU-constrained container collapses to 2)executor: docker # docker | kubernetes (run Lambda environments as pods)# kubernetes:# namespace: default # Namespace Lambda pods are created indiff --git a/src/test/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncherPopulateConcurrencyTest.java b/src/test/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncherPopulateConcurrencyTest.javanew file mode 100644index 0000000000..72c0524ce5--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncherPopulateConcurrencyTest.java@@ -0,0 +1,62 @@+package io.github.hectorvent.floci.services.lambda.launcher;++import io.github.hectorvent.floci.config.EmulatorConfig;+import org.junit.jupiter.api.Test;++import java.util.Optional;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.when;++/**+ * Tests for the configurable code-volume populate concurrency+ * ({@code floci.services.lambda.code-volume-populate-concurrency}). Plain {@code mock()} rather+ * than the Mockito extension, for the same strict-stubbing reason as+ * {@link ContainerLauncherNamePrefixTest}.+ */+class ContainerLauncherPopulateConcurrencyTest {++ private static int derivedDefault() {+ return Math.max(2, Runtime.getRuntime().availableProcessors() / 2);+ }++ @Test+ void derivesFromCpuCountWhenUnset() {+ assertEquals(derivedDefault(), ContainerLauncher.resolvePopulateConcurrency(config(null)));+ }++ @Test+ void usesConfiguredValueWhenSet() {+ assertEquals(1, ContainerLauncher.resolvePopulateConcurrency(config(1)));+ assertEquals(16, ContainerLauncher.resolvePopulateConcurrency(config(16)));+ }++ @Test+ void configuredValueDecouplesTheCapFromTheCpuCount() {+ // The point of the option: a CPU-constrained container derives 2, which serializes+ // concurrent cold starts of distinct large functions into pairs. An explicit value must+ // win over the derivation whether it is above or below it.+ int derived = derivedDefault();+ assertEquals(derived + 4, ContainerLauncher.resolvePopulateConcurrency(config(derived + 4)));+ assertEquals(1, ContainerLauncher.resolvePopulateConcurrency(config(1)));+ }++ @Test+ void ignoresNonPositiveValues() {+ // Zero permits would deadlock every populate, and a negative Semaphore count would make+ // acquire() block until as many releases arrived. Fall back instead of hanging.+ assertEquals(derivedDefault(), ContainerLauncher.resolvePopulateConcurrency(config(0)));+ assertEquals(derivedDefault(), ContainerLauncher.resolvePopulateConcurrency(config(-1)));+ }++ private static EmulatorConfig config(Integer permits) {+ EmulatorConfig config = mock(EmulatorConfig.class);+ EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);+ EmulatorConfig.LambdaServiceConfig lambda = mock(EmulatorConfig.LambdaServiceConfig.class);+ when(config.services()).thenReturn(services);+ when(services.lambda()).thenReturn(lambda);+ when(lambda.codeVolumePopulateConcurrency()).thenReturn(Optional.ofNullable(permits));+ return config;+ }+}