Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

fix(lambda): make the code-volume populate concurrency configurable (#2830)

Brian Taylor968ff78e2529
diff --git a/docs/configuration/environment-variables.md b/docs/configuration/environment-variables.md
index da64eddbe8..bc179c84da 100644
--- a/docs/configuration/environment-variables.md
+++ b/docs/configuration/environment-variables.md
@@ -197,6 +197,7 @@ See [Initialization Hooks](./initialization-hooks.md) for lifecycle phases and s
| `FLOCI_SERVICES_LAMBDA_CONTAINER_IDLE_TIMEOUT_SECONDS` | `300` | Seconds of inactivity before an idle Lambda container is removed |
| `FLOCI_SERVICES_LAMBDA_REGION_CONCURRENCY_LIMIT` | `1000` | Maximum concurrent Lambda invocations across all functions in a region |
| `FLOCI_SERVICES_LAMBDA_UNRESERVED_CONCURRENCY_MIN` | `100` | Minimum unreserved concurrency pool |
+| `FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY` | `max(2, cpus/2)` | Maximum concurrent first-time code-volume populates (functions whose unpacked code is at least 32 MB). The default is derived from the CPU count the JVM sees, so a CPU-constrained Floci container collapses it to 2 and concurrent cold starts of distinct functions serialise into pairs; set this to decouple the cap from the CPU allocation |
| `FLOCI_SERVICES_LAMBDA_HOT_RELOAD_ENABLED` | `false` | Watch Lambda code directories for changes and reload without redeployment |
| `FLOCI_SERVICES_LAMBDA_HOT_RELOAD_ALLOWED_PATHS` | _(none)_ | Comma-separated host paths that hot-reload is allowed to watch |
| `FLOCI_SERVICES_LAMBDA_DOCKER_NETWORK` | _(none)_ | Docker network for Lambda containers (overrides `FLOCI_SERVICES_DOCKER_NETWORK`) |
diff --git a/docs/services/lambda.md b/docs/services/lambda.md
index 949d82480c..d402f1cc8d 100644
--- a/docs/services/lambda.md
+++ b/docs/services/lambda.md
@@ -207,6 +207,7 @@ These AWS Lambda operations have no handler in Floci. Calls will return `404` or
| `FLOCI_SERVICES_LAMBDA_EXTRA_HOSTS` | *(unset)* | Comma-separated `hostname:ip` entries added to each Lambda container's `/etc/hosts`; `ip` may be `host-gateway`, mirroring `docker run --add-host` |
| `FLOCI_SERVICES_LAMBDA_DOCKER_HOST_OVERRIDE` | *(unset)* | Explicit host/IP that spawned Lambda containers use to reach Floci's Runtime API, bypassing auto-detection |
| `FLOCI_SERVICES_LAMBDA_CONTAINER_NAME_PREFIX` | `floci` | Base name prefix for spawned Lambda containers and code volumes (e.g. `acme` → `acme-<function>-<id>` containers, `acme-code-<function>-<hash>` volumes). Must be a valid Docker name segment (`[A-Za-z0-9][A-Za-z0-9_.-]*`); invalid values are ignored with a warning |
+| `FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY` | `max(2, cpus/2)` | Maximum concurrent first-time code-volume populates. See the note below |
| `FLOCI_SERVICES_LAMBDA_EXECUTOR` | `docker` | Execution backend: `docker` (containers) or `kubernetes` (pods) |
| `FLOCI_SERVICES_LAMBDA_KUBERNETES_NAMESPACE` | `default` | Namespace Lambda pods are created in |
| `FLOCI_SERVICES_LAMBDA_KUBERNETES_LABELS` | *(unset)* | Extra pod labels as comma-separated `key=value` entries |
@@ -226,6 +227,25 @@ These AWS Lambda operations have no handler in Floci. Calls will return `404` or
docker volume prune --filter label=floci=true
```
+!!! note "Concurrent cold starts of large functions"
+ A function whose unpacked code is at least 32 MB has that code streamed once into a
+ read-only Docker volume, so later cold starts mount it instead of copying. Those
+ first-time *populates* are capped — a burst of them overwhelms the Docker daemon — and
+ the default cap is `max(2, cpus/2)`, derived from the CPU count the JVM sees.
+
+ Because the JVM honours the container's cgroup CPU quota, running Floci with a small CPU
+ allocation collapses the cap to 2, and a burst of cold starts across *distinct* large
+ functions completes in waves of two rather than in parallel. Six such functions invoked
+ at once take roughly three times the wall-clock of one. Raise the cap to decouple it from
+ the CPU allocation:
+
+ ```bash
+ FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY=8
+ ```
+
+ Only first-time populates are gated. Warm containers, already-populated volumes, and
+ functions under 32 MB are never serialised by this.
+
### Runtime API host override
When a Lambda container starts, it calls back into Floci's Runtime API to fetch
diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
index 3a56d9cb1e..40fdf62332 100644
--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
@@ -1840,6 +1840,21 @@ public interface EmulatorConfig {
*/
Optional<String> containerNamePrefix();
+ /**
+ * Maximum concurrent first-time code-volume populates. Populating streams a large
+ * function's unpacked code into a helper container, so a burst of them can overwhelm
+ * the Docker daemon; this caps how many run at once.
+ *
+ * <p>Unset derives {@code max(2, availableProcessors() / 2)}. That derivation reads the
+ * JVM's view of the cgroup CPU quota, so a CPU-constrained Floci container collapses the
+ * cap to 2 and concurrent cold starts of distinct functions serialize into pairs. Set
+ * this to decouple the cap from the CPU allocation. Values below 1 are ignored with a
+ * warning rather than deadlocking every populate.
+ *
+ * Env var: FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY
+ */
+ Optional<Integer> codeVolumePopulateConcurrency();
+
/**
* Extra /etc/hosts entries added to every Lambda container, as "hostname:ip" pairs.
* The ip may be the literal "host-gateway" to map to the Docker host, mirroring
diff --git a/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java b/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java
index 75f9ba72a3..9176617746 100644
--- a/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java
+++ b/src/main/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncher.java
@@ -146,6 +146,7 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {
this.layerService = layerService;
this.awsEnv = awsEnv;
this.executionRoleCredentials = executionRoleCredentials;
+ this.populateSemaphore = new java.util.concurrent.Semaphore(resolvePopulateConcurrency(config));
}
@PostConstruct
@@ -603,12 +604,40 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {
// overwhelmed the Docker daemon, so copies hung/failed and left half-built "Created" containers.
// This gates ONLY the heavy populate — not every launch — so ordinary cold starts (volume mounts
// for already-populated large code, or the small-code direct copy) are never serialized.
- private static final java.util.concurrent.Semaphore POPULATE_SEMAPHORE =
- new java.util.concurrent.Semaphore(Math.max(2, Runtime.getRuntime().availableProcessors() / 2));
+ //
+ // Per-instance rather than static so the permit count can come from configuration; Floci runs a
+ // single @ApplicationScoped launcher, so this is still one gate for the whole emulator.
+ private final java.util.concurrent.Semaphore populateSemaphore;
+
+ /**
+ * Permits for {@link #populateSemaphore}: the configured value when set and positive,
+ * otherwise {@code max(2, availableProcessors() / 2)}.
+ *
+ * <p>The derived default reads the JVM's view of the cgroup CPU quota, so a CPU-constrained
+ * Floci container collapses it to 2 and concurrent cold starts of distinct functions
+ * serialize into pairs. The populate itself is IO-bound (streaming a tar into a helper
+ * container), so CPU count is a weak proxy for how many the daemon can take — hence the
+ * override. The default is unchanged, since the daemon overload the cap exists to prevent is
+ * real and its safe ceiling is host-specific.
+ */
+ static int resolvePopulateConcurrency(EmulatorConfig config) {
+ int derived = Math.max(2, Runtime.getRuntime().availableProcessors() / 2);
+ Optional<Integer> configured = config.services().lambda().codeVolumePopulateConcurrency();
+ if (configured.isEmpty()) {
+ return derived;
+ }
+ int permits = configured.get();
+ if (permits < 1) {
+ LOG.warnv("Ignoring floci.services.lambda.code-volume-populate-concurrency {0}: must be "
+ + "at least 1; using {1}", permits, derived);
+ return derived;
+ }
+ return permits;
+ }
- private static void acquirePopulatePermit(String functionName) {
+ private void acquirePopulatePermit(String functionName) {
try {
- POPULATE_SEMAPHORE.acquire();
+ populateSemaphore.acquire();
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted while waiting to populate code volume for " + functionName, ie);
@@ -871,7 +900,7 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {
LOG.warnv("Could not remove code-volume helper {0}: {1}", helperId, e.getMessage());
}
}
- POPULATE_SEMAPHORE.release();
+ populateSemaphore.release();
}
}
@@ -1097,7 +1126,7 @@ public class ContainerLauncher implements LambdaRuntimeLauncher {
Path sourceDir, String remotePath, String functionName,
DirectoryTarWriter tarWriter, boolean failOnTarFailure) {
// No per-copy gating here: the heavy /var/task populate for large code already holds a
- // POPULATE_SEMAPHORE permit; small-code direct copies and layer copies are light enough
+ // populateSemaphore permit; small-code direct copies and layer copies are light enough
// to run unthrottled.
try (java.io.PipedOutputStream pos = new java.io.PipedOutputStream();
java.io.PipedInputStream pis = new java.io.PipedInputStream(pos, TAR_PIPE_BUFFER_BYTES)) {
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
index 96cc8ace1b..4ee259e915 100644
--- a/src/main/resources/application.yml
+++ b/src/main/resources/application.yml
@@ -265,6 +265,7 @@ floci:
# aws-config-path: # Host path to bind-mount (read-only) at /opt/aws-config for real credential discovery
# extra-hosts: # Extra /etc/hosts entries for Lambda containers, "hostname:ip" ("host-gateway" allowed as ip)
# container-name-prefix: floci # FLOCI_SERVICES_LAMBDA_CONTAINER_NAME_PREFIX — base name prefix for spawned containers and code volumes ([A-Za-z0-9][A-Za-z0-9_.-]*; invalid values fall back to "floci")
+ # code-volume-populate-concurrency: # FLOCI_SERVICES_LAMBDA_CODE_VOLUME_POPULATE_CONCURRENCY — max concurrent first-time code-volume populates (default max(2, cpus/2), which a CPU-constrained container collapses to 2)
executor: docker # docker | kubernetes (run Lambda environments as pods)
# kubernetes:
# namespace: default # Namespace Lambda pods are created in
diff --git a/src/test/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncherPopulateConcurrencyTest.java b/src/test/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncherPopulateConcurrencyTest.java
new file mode 100644
index 0000000000..72c0524ce5
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/lambda/launcher/ContainerLauncherPopulateConcurrencyTest.java
@@ -0,0 +1,62 @@
+package io.github.hectorvent.floci.services.lambda.launcher;
+
+import io.github.hectorvent.floci.config.EmulatorConfig;
+import org.junit.jupiter.api.Test;
+
+import java.util.Optional;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+/**
+ * Tests for the configurable code-volume populate concurrency
+ * ({@code floci.services.lambda.code-volume-populate-concurrency}). Plain {@code mock()} rather
+ * than the Mockito extension, for the same strict-stubbing reason as
+ * {@link ContainerLauncherNamePrefixTest}.
+ */
+class ContainerLauncherPopulateConcurrencyTest {
+
+ private static int derivedDefault() {
+ return Math.max(2, Runtime.getRuntime().availableProcessors() / 2);
+ }
+
+ @Test
+ void derivesFromCpuCountWhenUnset() {
+ assertEquals(derivedDefault(), ContainerLauncher.resolvePopulateConcurrency(config(null)));
+ }
+
+ @Test
+ void usesConfiguredValueWhenSet() {
+ assertEquals(1, ContainerLauncher.resolvePopulateConcurrency(config(1)));
+ assertEquals(16, ContainerLauncher.resolvePopulateConcurrency(config(16)));
+ }
+
+ @Test
+ void configuredValueDecouplesTheCapFromTheCpuCount() {
+ // The point of the option: a CPU-constrained container derives 2, which serializes
+ // concurrent cold starts of distinct large functions into pairs. An explicit value must
+ // win over the derivation whether it is above or below it.
+ int derived = derivedDefault();
+ assertEquals(derived + 4, ContainerLauncher.resolvePopulateConcurrency(config(derived + 4)));
+ assertEquals(1, ContainerLauncher.resolvePopulateConcurrency(config(1)));
+ }
+
+ @Test
+ void ignoresNonPositiveValues() {
+ // Zero permits would deadlock every populate, and a negative Semaphore count would make
+ // acquire() block until as many releases arrived. Fall back instead of hanging.
+ assertEquals(derivedDefault(), ContainerLauncher.resolvePopulateConcurrency(config(0)));
+ assertEquals(derivedDefault(), ContainerLauncher.resolvePopulateConcurrency(config(-1)));
+ }
+
+ private static EmulatorConfig config(Integer permits) {
+ EmulatorConfig config = mock(EmulatorConfig.class);
+ EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);
+ EmulatorConfig.LambdaServiceConfig lambda = mock(EmulatorConfig.LambdaServiceConfig.class);
+ when(config.services()).thenReturn(services);
+ when(services.lambda()).thenReturn(lambda);
+ when(lambda.codeVolumePopulateConcurrency()).thenReturn(Optional.ofNullable(permits));
+ return config;
+ }
+}