Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(opensearch): allow configuring a data volume for domain containers (#5581)

LiamCato8145ffd00ee3
diff --git a/docs/configuration/environment-variables.md b/docs/configuration/environment-variables.md
index 261156a121..3ddf6ecbff 100644
--- a/docs/configuration/environment-variables.md
+++ b/docs/configuration/environment-variables.md
@@ -446,6 +446,7 @@ These services spawn Docker containers. They require access to the Docker socket
| `FLOCI_SERVICES_OPENSEARCH_ENABLED` | `true` | Enable the OpenSearch service |
| `FLOCI_SERVICES_OPENSEARCH_MOCK` | `false` | When `true`, domains are created instantly without a real container (API only) |
| `FLOCI_SERVICES_OPENSEARCH_DEFAULT_IMAGE` | *(unset)* | Optional fixed Docker image for every OpenSearch domain; when unset, images resolve per requested `EngineVersion` |
+| `FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME` | *(unset)* | Data volume for domain containers: an absolute path bind-mounts `<path>/<account-id>/<region>/<domain-name>`; any other value creates a named volume `<name>-<account-id>-<region>-<domain-name>`. With `FLOCI_DOCKER_RESOURCE_NAMESPACE`, the layouts are `<path>/<namespace>/<account-id>/<region>/<domain-name>` and `<name>-<namespace>-<account-id>-<region>-<domain-name>`. Linux daemon paths starting with `/` retain Linux syntax on Windows; native Windows absolute paths retain native handling. A temporary root container prepares bind-directory ownership and owner write access using the selected image, leaving existing files untouched; preparation failure aborts startup. Storage is never removed on domain delete. Named volumes are created without Floci labels when absent, so Floci-scoped prunes and label sweeps leave them alone. When unset, Floci manages each domain's storage |
| `FLOCI_SERVICES_OPENSEARCH_PROXY_BASE_PORT` | `9400` | First port in the OpenSearch proxy range |
| `FLOCI_SERVICES_OPENSEARCH_PROXY_MAX_PORT` | `9499` | Last port in the OpenSearch proxy range |
| `FLOCI_SERVICES_OPENSEARCH_KEEP_RUNNING_ON_SHUTDOWN` | `false` | Keep OpenSearch containers running when Floci stops |
diff --git a/docs/configuration/storage.md b/docs/configuration/storage.md
index cfadb20d2b..ca3170b1dc 100644
--- a/docs/configuration/storage.md
+++ b/docs/configuration/storage.md
@@ -194,6 +194,41 @@ By default volumes survive resource deletion, matching real AWS behavior.
FLOCI_STORAGE_PRUNE_VOLUMES_ON_DELETE=true
```
+### OpenSearch data volume override
+
+`FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME` replaces the derived per-domain volume for
+OpenSearch domains. An absolute path bind-mounts `<path>/<account-id>/<region>/<domain-name>`;
+any other value is a Docker volume name and each domain gets its own
+`<name>-<account-id>-<region>-<domain-name>` volume. Account and Region come from the
+domain's stored ARN, so identical domain names in different accounts or Regions use
+separate storage. When `FLOCI_DOCKER_RESOURCE_NAMESPACE` is set, the layouts become
+`<path>/<namespace>/<account-id>/<region>/<domain-name>` and
+`<name>-<namespace>-<account-id>-<region>-<domain-name>`, separating namespaced deployments
+sharing one Docker daemon as well. Storage named through this variable is
+never removed on domain delete, regardless of storage mode and prune policy: the operator
+named it, so its lifecycle is theirs. A volume Floci creates when absent carries no Floci
+label, so neither a floci-scoped `docker volume prune --filter label=floci=true` nor a
+sweep of Floci-labelled child resources, a testcontainers module reaping a run's labelled
+children, removes it.
+
+Bind paths refer to the Docker daemon's host. Linux paths starting with `/` retain Linux
+path syntax when Floci runs on Windows; native Windows absolute paths retain their native
+handling. Before mounting the directory, a temporary root container using the selected
+OpenSearch image copies ownership from the image's data directory and grants the owner
+write access. Only the directory itself is adjusted; existing files retain their ownership
+and permissions. Preparation failure aborts domain startup.
+
+```bash
+# For domain my-search in account 000000000000 and Region us-east-1:
+FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME=opensearch-data
+# Volume: opensearch-data-000000000000-us-east-1-my-search
+FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME=/srv/floci/opensearch
+# Directory: /srv/floci/opensearch/000000000000/us-east-1/my-search
+# With FLOCI_DOCKER_RESOURCE_NAMESPACE=team-a:
+# Volume: opensearch-data-team-a-000000000000-us-east-1-my-search
+# Directory: /srv/floci/opensearch/team-a/000000000000/us-east-1/my-search
+```
+
### Host-path mode (advanced)
Set `FLOCI_STORAGE_HOST_PERSISTENT_PATH` to an **absolute host path** to use bind mounts instead
diff --git a/docs/services/opensearch.md b/docs/services/opensearch.md
index 497a98d378..c136f5e09f 100644
--- a/docs/services/opensearch.md
+++ b/docs/services/opensearch.md
@@ -101,6 +101,7 @@ The `2015-01-01` API used by `aws es` and Terraform's `aws_elasticsearch_domain`
| `FLOCI_SERVICES_OPENSEARCH_ENABLED` | `true` | Enable/disable the service |
| `FLOCI_SERVICES_OPENSEARCH_MOCK` | `false` | `true` = metadata only (no Docker) |
| `FLOCI_SERVICES_OPENSEARCH_DEFAULT_IMAGE` | *(unset)* | Optional fixed image used for every domain regardless of `EngineVersion`. Useful for private registry mirrors. When unset, images resolve per-version from the built-in version map. |
+| `FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME` | *(unset)* | Data volume for the spawned domain containers. An absolute path bind-mounts `<path>/<account-id>/<region>/<domain-name>` (nested under a `<namespace>` subdirectory when `FLOCI_DOCKER_RESOURCE_NAMESPACE` is set); any other value is a Docker volume name and each domain gets its own `<name>-<account-id>-<region>-<domain-name>` volume of it (`<name>-<namespace>-<account-id>-<region>-<domain-name>` when a resource namespace is set), so no two domains or namespaced deployments sharing a Docker daemon share a data directory. Storage named here is never removed on domain delete. When unset, each domain gets a Floci-managed volume. |
| `FLOCI_SERVICES_OPENSEARCH_PROXY_BASE_PORT` | `9400` | Port range start for real mode |
| `FLOCI_SERVICES_OPENSEARCH_PROXY_MAX_PORT` | `9499` | Port range end for real mode |
| `FLOCI_SERVICES_OPENSEARCH_KEEP_RUNNING_ON_SHUTDOWN` | `false` | Leave containers running after Floci stops |
@@ -132,7 +133,8 @@ services:
- **Version validation:** `CreateDomain`, `UpdateDomainConfig`, and `UpgradeDomain` reject unknown engine versions with `ValidationException`. `UpgradeDomain` also rejects targets that aren't reachable from the current version per AWS's documented upgrade matrix.
- **Cluster defaults:** `m5.large.search`, 1 instance, EBS enabled with 10 GiB `gp2` volume.
- **Instance type families:** `t3.*` / `m5/m6g/m7g.*` / `r5/r6g/r7g.*` / `c5/c6g/c7g.*` are EBS-backed (3584 GiB max). `i3.*` are local-NVMe instance-store. `or1.*` are S3-backed and surface a much larger volume ceiling on `DescribeInstanceTypeLimits` (8–36 TiB depending on size). Floci still boots one Docker container per domain regardless of family — the metadata fidelity matters for SDK clients that introspect, not for runtime data placement.
-- **Container storage:** each domain gets a named Docker volume (`floci-opensearch-{volumeId}`) created automatically. In memory mode the volume is removed on domain delete; in persistent modes it is retained unless `FLOCI_STORAGE_PRUNE_VOLUMES_ON_DELETE=true`.
+- **Container identity:** containers are named `floci-aws-opensearch-<account-id>-<region>-<domain-name>`, or `floci-aws-<namespace>-opensearch-<account-id>-<region>-<domain-name>` when `FLOCI_DOCKER_RESOURCE_NAMESPACE` is set. Account and Region come from the domain's stored ARN and are also recorded in its container labels. Restarts remove older containers by their persisted container ID and preserve the stored volume name. Legacy names that omit account and Region are never removed solely by name, because another domain may own them.
+- **Container storage:** each domain gets a named Docker volume (`floci-aws-opensearch-{volumeId}`) created automatically. In memory mode the volume is removed on domain delete; in persistent modes it is retained unless `FLOCI_STORAGE_PRUNE_VOLUMES_ON_DELETE=true`. Set `FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME` to choose the storage yourself: an absolute path bind-mounts `<path>/<account-id>/<region>/<domain-name>` (a directory on the Docker daemon's host), and any other value is a Docker volume name the domain gets its own `<name>-<account-id>-<region>-<domain-name>` volume of. When `FLOCI_DOCKER_RESOURCE_NAMESPACE` is set, the namespace is inserted into both forms, so two namespaced deployments sharing a Docker daemon never converge on one data directory through the same configured base. Linux daemon paths starting with `/` retain Linux path syntax when Floci runs on Windows; native Windows absolute paths retain their native handling. Before a configured bind directory is mounted, a temporary root container using the selected OpenSearch image copies ownership from the image's data directory and grants the owner write access. Only the directory itself is adjusted; existing files retain their ownership and permissions. Preparation failures abort domain startup. Storage named that way is never removed on domain delete, and a volume created when absent carries no Floci label, so neither a floci-scoped volume prune nor a testcontainers run's labelled-children sweep collects it: the operator named it, so its lifecycle is theirs.
## Examples
diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
index f7d5ffa737..a24a3d8e91 100644
--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
@@ -2116,6 +2116,29 @@ public interface EmulatorConfig {
@WithDefault("${floci.storage.persistent-path}/opensearch")
String dataPath();
+ /**
+ * Optional data volume for the domain containers Floci spawns, replacing the
+ * per-domain volume Floci derives by default. An absolute path is bind-mounted as
+ * {@code <path>/<account-id>/<region>/<domain-name>}, nested under a {@code <namespace>} subdirectory
+ * when a resource namespace is configured; any other value is a Docker volume
+ * name and each domain gets its own {@code <name>-<namespace>-<account-id>-<region>-<domain-name>}
+ * volume, without the namespace segment when none is configured. No two domains
+ * and no two namespaced deployments sharing a Docker daemon ever share a data
+ * directory. Storage named here is never removed on domain delete,
+ * regardless of storage mode and prune policy: the operator named it, so its
+ * lifecycle is theirs. A volume Floci creates when absent carries no Floci
+ * label, so neither a floci-scoped volume prune nor a sweep of Floci-labelled
+ * child resources, a testcontainers module reaping a run's labelled children,
+ * removes it. When Floci itself runs in a container, an absolute path
+ * resolves on the Docker daemon's host. Paths starting with / retain Linux path syntax
+ * even when Floci runs on Windows. A temporary root container using the selected
+ * image creates and prepares bind-directory ownership and owner write access there;
+ * existing files are untouched, and preparation failure aborts domain startup.
+ *
+ * Env var: FLOCI_SERVICES_OPENSEARCH_DATA_VOLUME
+ */
+ Optional<String> dataVolume();
+
@WithDefault("false")
boolean keepRunningOnShutdown();
}
diff --git a/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManager.java b/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManager.java
index 2f23262dda..935fae23f8 100644
--- a/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManager.java
+++ b/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManager.java
@@ -20,6 +20,7 @@ import com.github.dockerjava.api.model.HostConfig;
import com.github.dockerjava.api.model.Mount;
import com.github.dockerjava.api.model.MountType;
import com.github.dockerjava.api.model.Ports;
+import com.github.dockerjava.api.model.Volume;
import com.github.dockerjava.core.command.WaitContainerResultCallback;
import io.github.hectorvent.floci.config.ContainerCaBundle;
import io.github.hectorvent.floci.config.EmulatorConfig;
@@ -580,18 +581,33 @@ public class ContainerLifecycleManager {
* and do nothing, the volume analogue of start treating an HTTP 304 as success.
*/
public void ensureVolume(String volumeName) {
- createVolumeIfAbsent(volumeName);
+ createVolumeIfAbsent(volumeName, ContainerStorageHelper.defaultLabels(config));
+ }
+
+ /**
+ * {@link #ensureVolume} for a volume an operator named, created when absent without any
+ * Floci label, the umbrella and extra ones alike. The operator named the volume, so its
+ * lifecycle is theirs: nothing that sweeps Floci-labelled volumes may collect it, neither
+ * a floci-scoped {@code docker volume prune} nor a testcontainers module reaping a run's
+ * labelled child resources through labels injected into {@code floci.docker.extra-labels}.
+ */
+ public void ensureUnmanagedVolume(String volumeName) {
+ if (volumeExists(volumeName)) {
+ return;
+ }
+ dockerClient.createVolumeCmd()
+ .withName(volumeName)
+ // An explicit empty map, never mergedLabels: the operator-named volume
+ // carries no Floci label, so no label-based sweep collects it.
+ .withLabels(Map.of())
+ .exec();
+ LOG.debugv("Created unmanaged volume {0}", volumeName);
}
/** Creates a volume with resource ownership labels for recoverable cleanup. */
public void ensureVolume(String volumeName, Map<String, String> labels) {
createVolumeIfAbsent(volumeName, labels);
}
-
- private boolean createVolumeIfAbsent(String volumeName) {
- return createVolumeIfAbsent(volumeName, Map.of());
- }
-
/** Creates the named volume when it is missing; returns whether it was created. */
private boolean createVolumeIfAbsent(String volumeName, Map<String, String> labels) {
if (volumeExists(volumeName)) {
@@ -642,7 +658,7 @@ public class ContainerLifecycleManager {
Optional<String> rootPermissions, String initImage) {
// A volume created now is root:root 0755 again, so a memo left by a volume of the same name
// that was removed since (e.g. DeleteApplication, docker volume rm) is stale.
- if (createVolumeIfAbsent(volumeName)) {
+ if (createVolumeIfAbsent(volumeName, ContainerStorageHelper.defaultLabels(config))) {
initializedSharedVolumes.remove(volumeName);
}
if (rootPermissions.isEmpty() && ownerUid.isEmpty() && ownerGid.isEmpty()) {
@@ -697,11 +713,37 @@ public class ContainerLifecycleManager {
HostConfig hostConfig = HostConfig.newHostConfig().withMounts(List.of(
new Mount().withType(MountType.VOLUME).withSource(volumeName).withTarget(mount)));
- CreateContainerResponse created = dockerClient.createContainerCmd(image)
+ CreateContainerCmd command = dockerClient.createContainerCmd(image)
.withHostConfig(hostConfig)
- .withCmd("sh", "-c", script.toString())
- .withLabels(ContainerStorageHelper.defaultLabels(config))
- .exec();
+ .withCmd("sh", "-c", script.toString());
+ runStorageInitHelper(command, "shared volume " + volumeName);
+ LOG.infov("Initialised shared volume {0} root (cmd: {1})", volumeName, script);
+ }
+
+ /**
+ * Makes a bind directory writable by the owner of the image's data directory. The
+ * helper runs on the Docker daemon's host, using the workload image so custom image
+ * ownership is respected. Only the mount root is changed; existing files are untouched.
+ * Unlike shared-volume initialisation, failure propagates and every launch checks the root.
+ */
+ public void ensureBindDirectory(String hostPath, String image, String imageDataPath) {
+ imageCacheService.ensureImageExists(image);
+ String mount = "/floci-bind-directory";
+ String script = "ownership=$(stat -c '%u:%g' \"$1\") && "
+ + "chown \"$ownership\" \"$2\" && chmod u+rwx \"$2\"";
+ HostConfig hostConfig = HostConfig.newHostConfig()
+ .withBinds(new Bind(hostPath, new Volume(mount)));
+ CreateContainerCmd command = dockerClient.createContainerCmd(image)
+ .withHostConfig(hostConfig)
+ .withUser("0")
+ .withEntrypoint("sh")
+ .withCmd("-c", script, "floci-bind-init", imageDataPath, mount);
+ runStorageInitHelper(command, "bind directory " + hostPath);
+ LOG.infov("Initialised bind directory {0} using ownership from {1}:{2}", hostPath, image, imageDataPath);
+ }
+
+ private void runStorageInitHelper(CreateContainerCmd command, String description) {
+ CreateContainerResponse created = command.withLabels(ContainerStorageHelper.defaultLabels(config)).exec();
String helperId = created.getId();
try {
startContainer(helperId);
@@ -709,17 +751,14 @@ public class ContainerLifecycleManager {
.exec(new WaitContainerResultCallback())
.awaitStatusCode(60, TimeUnit.SECONDS);
if (status == null || status != 0) {
- // Throw so the caller leaves the volume unmemoised and retries on the next launch,
- // rather than leaving it root:root 0755 with no further attempt.
- throw new IllegalStateException("shared-volume init for " + volumeName
- + " exited with status " + status + " (cmd: " + script + ")");
+ throw new IllegalStateException("Storage init for " + description + " exited with status " + status);
}
- LOG.infov("Initialised shared volume {0} root (cmd: {1})", volumeName, script);
} finally {
try {
dockerClient.removeContainerCmd(helperId).withForce(true).exec();
- } catch (Exception ignore) {
- // best-effort cleanup of the one-off helper
+ } catch (Exception e) {
+ LOG.warnv("Failed to remove storage init helper {0} for {1}: {2}",
+ helperId, description, e.getMessage());
}
}
}
diff --git a/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerStorageHelper.java b/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerStorageHelper.java
index 115a5f00c8..72bd41e1bd 100644
--- a/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerStorageHelper.java
+++ b/src/main/java/io/github/hectorvent/floci/core/common/docker/ContainerStorageHelper.java
@@ -273,7 +273,11 @@ public final class ContainerStorageHelper {
return base.resolve(namespace).resolve(service).resolve(resourceId);
}
- private static String resourceNamespace(EmulatorConfig config) {
+ /**
+ * The configured resource namespace, sanitized for use inside Docker names and paths;
+ * empty when no namespace is configured or {@code config} carries no Docker config.
+ */
+ public static String resourceNamespace(EmulatorConfig config) {
if (config == null || config.docker() == null || config.docker().resourceNamespace() == null) {
return "";
}
@@ -338,6 +342,22 @@ public final class ContainerStorageHelper {
builder.withNamedVolume(volumeName, internalMount);
}
+ /**
+ * {@link #applyNamedVolume} for a volume an operator named through their own
+ * configuration: created when absent without any Floci label, so nothing that sweeps
+ * Floci-labelled volumes, a floci-scoped {@code docker volume prune} above all, and a
+ * testcontainers module reaping a run's labelled child resources above all, collects it.
+ * The operator named the volume, so its lifecycle is theirs.
+ */
+ public static void applyOperatorNamedVolume(
+ ContainerBuilder.Builder builder,
+ ContainerLifecycleManager lifecycleManager,
+ String volumeName,
+ String internalMount) {
+ lifecycleManager.ensureUnmanagedVolume(volumeName);
+ builder.withNamedVolume(volumeName, internalMount);
+ }
+
/**
* Removes the named volume on resource delete, honouring the configured prune policy.
*
diff --git a/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManager.java b/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManager.java
index 869f2339a1..3ab25b053e 100644
--- a/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManager.java
+++ b/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManager.java
@@ -1,7 +1,7 @@
package io.github.hectorvent.floci.services.opensearch;
import io.github.hectorvent.floci.config.EmulatorConfig;
-import io.github.hectorvent.floci.core.common.RegionResolver;
+import io.github.hectorvent.floci.core.common.AwsArnUtils;
import io.github.hectorvent.floci.core.common.docker.ContainerBuilder;
import io.github.hectorvent.floci.core.common.docker.ContainerDetector;
import io.github.hectorvent.floci.core.common.docker.ContainerLifecycleManager;
@@ -11,13 +11,16 @@ import io.github.hectorvent.floci.core.common.docker.ContainerSpec;
import io.github.hectorvent.floci.core.common.docker.ContainerStorageHelper;
import io.github.hectorvent.floci.core.common.docker.PortAllocator;
import io.github.hectorvent.floci.services.opensearch.model.Domain;
+import io.github.hectorvent.floci.services.opensearch.model.VolumeOwnership;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import org.jboss.logging.Logger;
+import java.io.File;
import java.net.HttpURLConnection;
import java.net.URI;
import java.nio.file.Path;
+import java.util.Optional;
/**
* Manages the Docker lifecycle of OpenSearch containers for real-mode domains.
@@ -28,6 +31,7 @@ public class OpenSearchDomainManager {
private static final Logger LOG = Logger.getLogger(OpenSearchDomainManager.class);
private static final int OPENSEARCH_PORT = 9200;
+ private static final String CONTAINER_DATA_PATH = "/usr/share/opensearch/data";
private final ContainerBuilder containerBuilder;
private final ContainerLifecycleManager lifecycleManager;
@@ -35,21 +39,18 @@ public class OpenSearchDomainManager {
private final PortAllocator portAllocator;
private final EmulatorConfig config;
private volatile boolean dockerUnavailableLogged;
- private final RegionResolver regionResolver;
@Inject
public OpenSearchDomainManager(ContainerBuilder containerBuilder,
ContainerLifecycleManager lifecycleManager,
ContainerDetector containerDetector,
PortAllocator portAllocator,
- EmulatorConfig config,
- RegionResolver regionResolver) {
+ EmulatorConfig config) {
this.containerBuilder = containerBuilder;
this.lifecycleManager = lifecycleManager;
this.containerDetector = containerDetector;
this.portAllocator = portAllocator;
this.config = config;
- this.regionResolver = regionResolver;
}
/**
@@ -98,11 +99,17 @@ public class OpenSearchDomainManager {
public void startDomain(Domain domain) {
String image = resolveImage(domain.getEngineVersion());
- String containerName = containerName(domain);
+ AwsArnUtils.Arn arn = AwsArnUtils.parse(domain.getArn());
+ String containerName = ContainerStorageHelper.resourceName(config, "opensearch", null,
+ arn.accountId() + "-" + arn.region() + "-" + domain.getDomainName());
LOG.infov("Starting OpenSearch container for domain: {0} (version={1}, image={2})",
domain.getDomainName(), domain.getEngineVersion(), image);
+ // Older container names omit account and Region, so only the persisted ID proves ownership.
+ if (domain.getContainerId() != null && !domain.getContainerId().isBlank()) {
+ lifecycleManager.removeIfExists(domain.getContainerId());
+ }
ContainerStorageHelper.removeStaleContainer(config, lifecycleManager, containerName);
// A restart of an existing domain has just removed the old container, so its
@@ -122,8 +129,7 @@ public class OpenSearchDomainManager {
// DeleteDomain, and every shutdown for each domain, waited out the whole stop timeout.
.withInit()
.withLabels(ContainerStorageHelper.resourceIdentityLabels(
- "opensearch", domain.getDomainName(), regionResolver.getAccountId(),
- regionResolver.getDefaultRegion()));
+ "opensearch", domain.getDomainName(), arn.accountId(), arn.region()));
// Container-name DNS only resolves on a user-defined Docker network, and
// nothing guarantees the spawned OpenSearch container and floci itself land
@@ -144,27 +150,32 @@ public class OpenSearchDomainManager {
config.services().opensearch().proxyMaxPort());
specBuilder.withPortBinding(OPENSEARCH_PORT, hostPort);
- applyEngineEnv(specBuilder, domain.getEngineVersion());
+ ContainerInfo info;
+ try {
+ applyEngineEnv(specBuilder, domain.getEngineVersion());
- if (ContainerStorageHelper.isNamedVolumeMode(config)) {
- ContainerStorageHelper.applyNamedVolume(specBuilder, lifecycleManager,
- resolveVolumeName(domain), "/usr/share/opensearch/data");
- } else {
- // Legacy host-path mode: host-persistent-path is an absolute path
- Path dataPath = ContainerStorageHelper.hostResourcePath(config, "opensearch", domain.getDomainName());
- if (!containerDetector.isRunningInContainer()) {
- ContainerStorageHelper.ensureHostDir(dataPath.toString());
+ Optional<String> configuredDataVolume = config.services().opensearch().dataVolume()
+ .filter(dataVolume -> !dataVolume.isBlank());
+ if (configuredDataVolume.isPresent()) {
+ applyConfiguredDataVolume(specBuilder, domain, configuredDataVolume.get(), image);
+ } else if (ContainerStorageHelper.isNamedVolumeMode(config)) {
+ domain.setVolumeOwnership(VolumeOwnership.FLOCI_MANAGED);
+ ContainerStorageHelper.applyNamedVolume(specBuilder, lifecycleManager,
+ resolveVolumeName(domain), CONTAINER_DATA_PATH);
+ } else {
+ // Legacy host-path mode: host-persistent-path is an absolute path
+ Path dataPath = ContainerStorageHelper.hostResourcePath(config, "opensearch", domain.getDomainName());
+ if (!containerDetector.isRunningInContainer()) {
+ ContainerStorageHelper.ensureHostDir(dataPath.toString());
+ }
+ String dataPathStr = dataPath.toAbsolutePath().normalize().toString();
+ String persistentPathStr = Path.of(config.storage().persistentPath()).toAbsolutePath().normalize().toString();
+ String hostDataPath = dataPathStr.replace(persistentPathStr, config.storage().hostPersistentPath());
+ specBuilder.withBind(hostDataPath, CONTAINER_DATA_PATH);
}
- String dataPathStr = dataPath.toAbsolutePath().normalize().toString();
- String persistentPathStr = Path.of(config.storage().persistentPath()).toAbsolutePath().normalize().toString();
- String hostDataPath = dataPathStr.replace(persistentPathStr, config.storage().hostPersistentPath());
- specBuilder.withBind(hostDataPath, "/usr/share/opensearch/data");
- }
- ContainerSpec spec = specBuilder.build();
+ ContainerSpec spec = specBuilder.build();
- ContainerInfo info;
- try {
info = lifecycleManager.createAndStart(spec);
} catch (RuntimeException e) {
portAllocator.release(hostPort);
@@ -220,8 +231,31 @@ public class OpenSearchDomainManager {
LOG.infov("Stopped OpenSearch container for domain {0}", domain.getDomainName());
}
+ /**
+ * Removes the domain's Docker volume on delete, honouring the configured prune policy.
+ * The decision rides on the {@link VolumeOwnership} marker persisted when the storage
+ * was selected, not on names recomputed from the configuration held at delete time: a
+ * recomputed name stops matching the persisted one as soon as the resource namespace
+ * changes, and a name composed from {@code floci.services.opensearch.data-volume} can
+ * be spelled exactly like one Floci derives. Records predating the marker also predate
+ * configured data volumes, so their volumes are always Floci-managed.
+ */
public void removeDomainStorage(Domain domain) {
- ContainerStorageHelper.removeNamedVolume(config, lifecycleManager, resolveVolumeName(domain));
+ VolumeOwnership ownership = domain.getVolumeOwnership();
+ if (ownership == VolumeOwnership.OPERATOR_BIND) {
+ LOG.infov("Retained the bind-mounted data directory of domain {0}. It was named by "
+ + "floci.services.opensearch.data-volume, which Floci never removes.",
+ domain.getDomainName());
+ return;
+ }
+ String volumeName = resolveVolumeName(domain);
+ if (ownership == VolumeOwnership.OPERATOR_NAMED) {
+ LOG.infov("Retained Docker volume {0}. It was named by "
+ + "floci.services.opensearch.data-volume, which Floci never removes. "
+ + "Remove manually: docker volume rm {0}", volumeName);
+ return;
+ }
+ ContainerStorageHelper.removeNamedVolume(config, lifecycleManager, volumeName);
}
/**
@@ -243,8 +277,55 @@ public class OpenSearchDomainManager {
config.services().opensearch().defaultImage(), engineVersion);
}
- private String containerName(Domain domain) {
- return ContainerStorageHelper.resourceName(config, "opensearch", null, domain.getDomainName());
+ /**
+ * Mounts the operator-configured {@code floci.services.opensearch.data-volume}. An
+ * absolute path is bind-mounted as {@code <path>/<account-id>/<region>/<domain-name>}, nested under a
+ * {@code <namespace>} subdirectory when a resource namespace is configured; any
+ * other value is a Docker volume name and the domain gets its own
+ * {@code <name>-<namespace>-<account-id>-<region>-<domain-name>} volume of it, without the namespace
+ * segment when none is configured. Per-domain, per-deployment naming is deliberate:
+ * two OpenSearch nodes sharing one data directory corrupt each other's cluster
+ * state, and two namespaced deployments on one Docker daemon must not converge on
+ * one volume through the same configured base. The domain record is stamped with
+ * the volume name and the operator ownership marker so
+ * {@link #removeDomainStorage} can tell operator-named storage, which it never
+ * deletes, from the kind Floci derives. The bind branch clears any managed name
+ * stamped at creation: a stale one would read as a volume the domain never mounted.
+ * A volume created when absent carries no Floci label, so no sweep of
+ * Floci-labelled volumes, a floci-scoped prune or a testcontainers run's
+ * labelled-children reap, collects operator storage.
+ */
+ private void applyConfiguredDataVolume(ContainerBuilder.Builder specBuilder, Domain domain,
+ String dataVolume, String image) {
+ AwsArnUtils.Arn arn = AwsArnUtils.parse(domain.getArn());
+ if (dataVolume.startsWith("/") || Path.of(dataVolume).isAbsolute()) {
+ domain.setVolumeOwnership(VolumeOwnership.OPERATOR_BIND);
+ domain.setDockerVolumeName(null);
+ String namespace = ContainerStorageHelper.resourceNamespace(config);
+ String hostPath;
+ if (dataVolume.startsWith("/") && File.separatorChar != '/') {
+ // A Linux daemon path must not be resolved against the Windows client's drive.
+ String base = dataVolume.endsWith("/") ? dataVolume : dataVolume + "/";
+ String prefix = namespace.isBlank() ? base : base + namespace + "/";
+ hostPath = prefix + arn.accountId() + "/" + arn.region() + "/" + domain.getDomainName();
+ } else {
+ Path domainDataPath = namespace.isBlank()
+ ? Path.of(dataVolume, arn.accountId(), arn.region(), domain.getDomainName())
+ : Path.of(dataVolume, namespace, arn.accountId(), arn.region(), domain.getDomainName());
+ if (!containerDetector.isRunningInContainer()) {
+ ContainerStorageHelper.ensureHostDir(domainDataPath.toString());
+ }
+ hostPath = domainDataPath.toAbsolutePath().normalize().toString();
+ }
+ lifecycleManager.ensureBindDirectory(hostPath, image, CONTAINER_DATA_PATH);
+ specBuilder.withBind(hostPath, CONTAINER_DATA_PATH);
+ return;
+ }
+ String volumeName = ContainerStorageHelper.prefixedDockerName(
+ config, dataVolume, arn.accountId() + "-" + arn.region() + "-" + domain.getDomainName());
+ domain.setDockerVolumeName(volumeName);
+ domain.setVolumeOwnership(VolumeOwnership.OPERATOR_NAMED);
+ ContainerStorageHelper.applyOperatorNamedVolume(specBuilder, lifecycleManager, volumeName, CONTAINER_DATA_PATH);
}
/**
diff --git a/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchService.java b/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchService.java
index 580cbc4efe..4e1cd6aaa1 100644
--- a/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/opensearch/OpenSearchService.java
@@ -20,6 +20,7 @@ import io.github.hectorvent.floci.services.opensearch.model.DomainEndpointOption
import io.github.hectorvent.floci.services.opensearch.model.EbsOptions;
import io.github.hectorvent.floci.services.opensearch.model.EncryptionAtRestOptions;
import io.github.hectorvent.floci.services.opensearch.model.NodeToNodeEncryptionOptions;
+import io.github.hectorvent.floci.services.opensearch.model.VolumeOwnership;
import io.github.hectorvent.floci.services.opensearch.model.VpcOptions;
import jakarta.annotation.PostConstruct;
import jakarta.annotation.PreDestroy;
@@ -136,6 +137,10 @@ public class OpenSearchService implements ResourceProvider {
// recomputed later. Only records predating this field fall back to the legacy name.
domain.setDockerVolumeName(ContainerStorageHelper.resourceName(
config, "opensearch", domain.getVolumeId(), domain.getDomainName()));
+ // The ownership marker travels with the name, so deletion removes what was persisted
+ // instead of recomputing ownership against whatever configuration holds at delete time.
+ // A configured data volume overwrites both when startDomain mounts operator storage.
+ domain.setVolumeOwnership(VolumeOwnership.FLOCI_MANAGED);
if (clusterConfig != null) {
domain.setClusterConfig(clusterConfig);
diff --git a/src/main/java/io/github/hectorvent/floci/services/opensearch/model/Domain.java b/src/main/java/io/github/hectorvent/floci/services/opensearch/model/Domain.java
index 0579c2f10d..f4fbde745b 100644
--- a/src/main/java/io/github/hectorvent/floci/services/opensearch/model/Domain.java
+++ b/src/main/java/io/github/hectorvent/floci/services/opensearch/model/Domain.java
@@ -82,6 +82,13 @@ public class Domain {
@JsonProperty("DockerVolumeName")
private String dockerVolumeName;
+ /**
+ * Who owns the storage the domain's container mounts; null on records written before
+ * this field existed, whose volumes are always Floci-managed.
+ */
+ @JsonProperty("VolumeOwnership")
+ private VolumeOwnership volumeOwnership;
+
@JsonProperty("CreatedAt")
@JsonFormat(shape = JsonFormat.Shape.NUMBER)
private Instant createdAt;
@@ -248,6 +255,14 @@ public class Domain {
this.dockerVolumeName = dockerVolumeName;
}
+ public VolumeOwnership getVolumeOwnership() {
+ return volumeOwnership;
+ }
+
+ public void setVolumeOwnership(VolumeOwnership volumeOwnership) {
+ this.volumeOwnership = volumeOwnership;
+ }
+
public Instant getCreatedAt() {
return createdAt;
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/opensearch/model/VolumeOwnership.java b/src/main/java/io/github/hectorvent/floci/services/opensearch/model/VolumeOwnership.java
new file mode 100644
index 0000000000..a34ceb4510
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/opensearch/model/VolumeOwnership.java
@@ -0,0 +1,27 @@
+package io.github.hectorvent.floci.services.opensearch.model;
+
+import io.quarkus.runtime.annotations.RegisterForReflection;
+
+/**
+ * Who owns the storage a domain's container mounts, stamped on the domain record when
+ * the storage is selected. Deletion acts on this marker rather than recomputing
+ * ownership from whatever configuration holds at delete time: a recomputed name no
+ * longer matches the persisted one as soon as the resource namespace changes, and a
+ * name an operator picked through {@code floci.services.opensearch.data-volume} can
+ * be spelled exactly like one Floci derives.
+ *
+ * <p>Null on records written before this field existed, which predate configured data volumes
+ * and therefore always have Floci-managed storage.</p>
+ */
+@RegisterForReflection
+public enum VolumeOwnership {
+
+ /** Floci derived the per-domain volume name; deletion removes it per storage policy. */
+ FLOCI_MANAGED,
+
+ /** The operator named the volume through {@code floci.services.opensearch.data-volume}; never removed. */
+ OPERATOR_NAMED,
+
+ /** The operator bind-mounted a host path through {@code floci.services.opensearch.data-volume}; never removed. */
+ OPERATOR_BIND
+}
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
index 7c652b975a..f3494caa54 100644
--- a/src/main/resources/application.yml
+++ b/src/main/resources/application.yml
@@ -595,6 +595,19 @@ floci:
# single image (e.g. private registry mirror). When unset, the image is
# picked from OpenSearchVersions.imageFor(EngineVersion).
# default-image: "registry.example.com/opensearch:2.19.5"
+ # data-volume: optional data volume for the spawned domain containers; an absolute
+ # path is bind-mounted as <path>/<account-id>/<region>/<domain-name> (nested under a <namespace>
+ # subdirectory when docker.resource-namespace is set), any other value is a Docker
+ # volume name and each domain gets its own <name>-<namespace>-<account-id>-<region>-<domain-name> volume
+ # of it (no namespace segment when unset). Storage named here is never removed on
+ # domain delete, and a volume created when absent carries no Floci label, so no
+ # sweep of Floci-labelled volumes collects it. When unset, each domain gets a
+ # Floci-managed volume.
+ # Linux daemon paths starting with / keep their path syntax on Windows.
+ # Bind-directory ownership is prepared by a temporary root container using the
+ # selected image. Existing files are untouched; init failure aborts domain startup.
+ # data-volume: "opensearch-data"
+ # data-volume: "/srv/floci/opensearch"
proxy-base-port: 9400
proxy-max-port: 9499
keep-running-on-shutdown: false
diff --git a/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerBindDirectoryDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerBindDirectoryDockerIntegrationTest.java
new file mode 100644
index 0000000000..7d076caa20
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerBindDirectoryDockerIntegrationTest.java
@@ -0,0 +1,87 @@
+package io.github.hectorvent.floci.core.common.docker;
+
+import com.github.dockerjava.api.DockerClient;
+import com.github.dockerjava.api.command.CreateContainerResponse;
+import com.github.dockerjava.api.model.Bind;
+import com.github.dockerjava.api.model.HostConfig;
+import com.github.dockerjava.api.model.Volume;
+import com.github.dockerjava.core.command.WaitContainerResultCallback;
+import io.github.hectorvent.floci.services.lambda.launcher.ImageCacheService;
+import io.github.hectorvent.floci.services.opensearch.OpenSearchVersions;
+import io.github.hectorvent.floci.testing.DockerAvailability;
+import io.github.hectorvent.floci.testing.TestImages;
+import io.quarkus.test.junit.QuarkusTest;
+import jakarta.inject.Inject;
+import org.junit.jupiter.api.Assumptions;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledOnOs;
+import org.junit.jupiter.api.condition.OS;
+import org.junit.jupiter.api.io.TempDir;
+
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.concurrent.TimeUnit;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+@QuarkusTest
+@EnabledOnOs(OS.LINUX)
+class ContainerBindDirectoryDockerIntegrationTest {
+
+ @Inject
+ DockerClient dockerClient;
+
+ @Inject
+ ContainerLifecycleManager lifecycleManager;
+
+ @Inject
+ ImageCacheService imageCacheService;
+
+ @Test
+ void nonRootImageCanWriteFreshBindWithoutChangingExistingFiles(@TempDir Path root) throws Exception {
+ Assumptions.assumeTrue(DockerAvailability.isAvailable(), "Docker daemon must be available");
+ Path directory = Files.createDirectory(root.resolve("domain"));
+ Path existing = Files.writeString(directory.resolve("existing"), "operator data");
+ int originalUid = (Integer) Files.getAttribute(directory, "unix:uid");
+ int originalGid = (Integer) Files.getAttribute(directory, "unix:gid");
+ String hostPath = directory.toString();
+ imageCacheService.ensureImageExists(TestImages.BUSYBOX);
+ try {
+ assertEquals(0, runBoundCommand(TestImages.BUSYBOX, hostPath, "0",
+ "chown 0:0 /test-data && chmod 0755 /test-data && "
+ + "chown 2222:2222 /test-data/existing && chmod 0600 /test-data/existing"));
+ String image = OpenSearchVersions.imageFor(OpenSearchVersions.DEFAULT_VERSION);
+
+ lifecycleManager.ensureBindDirectory(hostPath, image, "/usr/share/opensearch/data");
+
+ assertEquals(1000, Files.getAttribute(directory, "unix:uid"));
+ assertEquals(2222, Files.getAttribute(existing, "unix:uid"));
+ assertEquals(0600, (Integer) Files.getAttribute(existing, "unix:mode") & 0777);
+ assertEquals(0, runBoundCommand(image, hostPath, "1000:1000", "touch /test-data/created"));
+ assertTrue(Files.exists(directory.resolve("created")));
+ } finally {
+ assertEquals(0, runBoundCommand(TestImages.BUSYBOX, hostPath, "0",
+ "rm -f /test-data/created && chown " + originalUid + ":" + originalGid + " /test-data"));
+ }
+ }
+
+ private Integer runBoundCommand(String image, String hostPath, String user, String script) {
+ HostConfig hostConfig = HostConfig.newHostConfig()
+ .withBinds(new Bind(hostPath, new Volume("/test-data")));
+ CreateContainerResponse created = dockerClient.createContainerCmd(image)
+ .withHostConfig(hostConfig)
+ .withUser(user)
+ .withEntrypoint("sh")
+ .withCmd("-c", script)
+ .exec();
+ String id = created.getId();
+ try {
+ lifecycleManager.startContainer(id);
+ return dockerClient.waitContainerCmd(id).exec(new WaitContainerResultCallback())
+ .awaitStatusCode(60, TimeUnit.SECONDS);
+ } finally {
+ dockerClient.removeContainerCmd(id).withForce(true).exec();
+ }
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManagerVolumeTest.java b/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManagerVolumeTest.java
index e2ee42810e..bc64df6787 100644
--- a/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManagerVolumeTest.java
+++ b/src/test/java/io/github/hectorvent/floci/core/common/docker/ContainerLifecycleManagerVolumeTest.java
@@ -16,12 +16,14 @@ import com.github.dockerjava.api.command.StopContainerCmd;
import com.github.dockerjava.api.command.WaitContainerCmd;
import com.github.dockerjava.api.exception.DockerException;
import com.github.dockerjava.api.exception.NotFoundException;
+import com.github.dockerjava.api.model.HostConfig;
import com.github.dockerjava.core.command.WaitContainerResultCallback;
import io.github.hectorvent.floci.config.EmulatorConfig;
import io.github.hectorvent.floci.services.lambda.launcher.ImageCacheService;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
+import org.mockito.ArgumentCaptor;
import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
@@ -33,6 +35,7 @@ import java.util.Map;
import java.util.Optional;
import java.util.OptionalInt;
import java.util.Set;
+import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.function.IntConsumer;
import java.util.function.IntFunction;
@@ -567,6 +570,58 @@ class ContainerLifecycleManagerVolumeTest {
assertEquals(1, creates.get(), "a missing volume must be created");
}
+ /** A Floci-derived volume carries the labels a floci-scoped prune and sweep filter on. */
+ @Test
+ void ensureVolume_labelsCreatedVolumeAsFlociManaged() {
+ inspectVolumeBehaving(attempt -> {
+ throw new NotFoundException("No such volume: vol");
+ });
+ CreateVolumeCmd cmd = mock(CreateVolumeCmd.class, RETURNS_SELF);
+ when(dockerClient.createVolumeCmd()).thenReturn(cmd);
+ when(cmd.exec()).thenReturn(mock(CreateVolumeResponse.class));
+ ArgumentCaptor<Map<String, String>> labels = ArgumentCaptor.forClass(Map.class);
+
+ manager.ensureVolume("vol");
+
+ verify(cmd).withLabels(labels.capture());
+ assertEquals(Map.of("floci", "true", "floci_emulator", "floci-aws"), labels.getValue());
+ }
+
+ /**
+ * A volume an operator named is created without any Floci label: no sweep of
+ * Floci-labelled volumes, a floci-scoped prune or a testcontainers run's
+ * labelled-children reap, may collect operator storage.
+ */
+ @Test
+ void ensureUnmanagedVolume_createsMissingVolumeWithoutLabels() {
+ AtomicInteger inspects = inspectVolumeBehaving(attempt -> {
+ throw new NotFoundException("No such volume: vol");
+ });
+ CreateVolumeCmd cmd = mock(CreateVolumeCmd.class, RETURNS_SELF);
+ when(dockerClient.createVolumeCmd()).thenReturn(cmd);
+ when(cmd.exec()).thenReturn(mock(CreateVolumeResponse.class));
+ ArgumentCaptor<Map<String, String>> labels = ArgumentCaptor.forClass(Map.class);
+
+ manager.ensureUnmanagedVolume("vol");
+
+ assertEquals(1, inspects.get());
+ verify(cmd).withLabels(labels.capture());
+ assertTrue(labels.getValue().isEmpty(),
+ "an operator-named volume carries no label a Floci sweep filters on");
+ }
+
+ /** An existing operator-named volume is left exactly as the operator created it. */
+ @Test
+ void ensureUnmanagedVolume_doesNotCreateWhenVolumeAlreadyExists() {
+ AtomicInteger inspects = inspectVolumeBehaving(attempt ->
+ mock(InspectVolumeResponse.class));
+
+ manager.ensureUnmanagedVolume("vol");
+
+ assertEquals(1, inspects.get());
+ verify(dockerClient, never()).createVolumeCmd();
+ }
+
@Test
void ensureVolume_doesNotCreateWhenVolumeAlreadyExists() {
// This existence guard is what keeps ensureVolume idempotent now that transient-I/O
@@ -657,4 +712,61 @@ class ContainerLifecycleManagerVolumeTest {
verify(spyManager).startContainer("helper-id");
verify(dockerClient, never()).startContainerCmd("helper-id");
}
+
+ @Test
+ void ensureBindDirectoryRunsRootHelperUsingImageOwnershipAndCleansUp() {
+ CreateContainerCmd command = bindInitHelper(0);
+ ArgumentCaptor<HostConfig> hostConfig = ArgumentCaptor.forClass(HostConfig.class);
+
+ manager.ensureBindDirectory("/host/domain", "custom-opensearch", "/usr/share/opensearch/data");
+
+ verify(imageCacheService).ensureImageExists("custom-opensearch");
+ verify(command).withHostConfig(hostConfig.capture());
+ assertEquals("/host/domain", hostConfig.getValue().getBinds()[0].getPath());
+ assertEquals("/floci-bind-directory", hostConfig.getValue().getBinds()[0].getVolume().getPath());
+ verify(command).withUser("0");
+ verify(command).withEntrypoint("sh");
+ verify(command).withCmd("-c", "ownership=$(stat -c '%u:%g' \"$1\") && "
+ + "chown \"$ownership\" \"$2\" && chmod u+rwx \"$2\"",
+ "floci-bind-init", "/usr/share/opensearch/data", "/floci-bind-directory");
+ verify(dockerClient).startContainerCmd("helper-id");
+ verify(dockerClient).removeContainerCmd("helper-id");
+ }
+
+ @Test
+ void ensureBindDirectoryPropagatesFailedInitAndStillRemovesHelper() {
+ bindInitHelper(1);
+
+ IllegalStateException exception = assertThrows(IllegalStateException.class, () ->
+ manager.ensureBindDirectory("/host/domain", "custom-opensearch", "/usr/share/opensearch/data"));
+
+ assertTrue(exception.getMessage().contains("exited with status 1"));
+ verify(dockerClient).removeContainerCmd("helper-id");
+ }
+
+ @Test
+ void ensureBindDirectoryRechecksDirectoryOnEveryLaunch() {
+ bindInitHelper(0);
+
+ manager.ensureBindDirectory("/host/domain", "custom-opensearch", "/usr/share/opensearch/data");
+ manager.ensureBindDirectory("/host/domain", "custom-opensearch", "/usr/share/opensearch/data");
+
+ verify(dockerClient, times(2)).createContainerCmd("custom-opensearch");
+ }
+
+ private CreateContainerCmd bindInitHelper(int status) {
+ CreateContainerCmd command = mock(CreateContainerCmd.class, RETURNS_SELF);
+ when(dockerClient.createContainerCmd("custom-opensearch")).thenReturn(command);
+ CreateContainerResponse response = mock(CreateContainerResponse.class);
+ when(response.getId()).thenReturn("helper-id");
+ when(command.exec()).thenReturn(response);
+ when(dockerClient.startContainerCmd("helper-id")).thenReturn(mock(StartContainerCmd.class));
+ WaitContainerCmd wait = mock(WaitContainerCmd.class);
+ when(dockerClient.waitContainerCmd("helper-id")).thenReturn(wait);
+ WaitContainerResultCallback callback = mock(WaitContainerResultCallback.class);
+ when(wait.exec(any(WaitContainerResultCallback.class))).thenReturn(callback);
+ when(callback.awaitStatusCode(60, TimeUnit.SECONDS)).thenReturn(status);
+ when(dockerClient.removeContainerCmd("helper-id")).thenReturn(mock(RemoveContainerCmd.class, RETURNS_SELF));
+ return command;
+ }
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManagerTest.java b/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManagerTest.java
index f3471ee46b..8043fb268e 100644
--- a/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManagerTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchDomainManagerTest.java
@@ -1,42 +1,167 @@
package io.github.hectorvent.floci.services.opensearch;
import io.github.hectorvent.floci.config.EmulatorConfig;
-import io.github.hectorvent.floci.core.common.RegionResolver;
+import io.github.hectorvent.floci.core.common.AwsArnUtils;
import io.github.hectorvent.floci.core.common.docker.ContainerBuilder;
import io.github.hectorvent.floci.core.common.docker.ContainerDetector;
import io.github.hectorvent.floci.core.common.docker.ContainerLifecycleManager;
import io.github.hectorvent.floci.core.common.docker.ContainerSpec;
+import io.github.hectorvent.floci.core.common.docker.ContainerStorageHelper;
import io.github.hectorvent.floci.core.common.docker.PortAllocator;
import io.github.hectorvent.floci.services.opensearch.model.Domain;
+import io.github.hectorvent.floci.services.opensearch.model.VolumeOwnership;
+import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+import org.junit.jupiter.params.provider.ValueSource;
import org.mockito.ArgumentCaptor;
+import org.mockito.InOrder;
+import org.mockito.MockMakers;
+import org.mockito.MockedStatic;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
import java.util.Map;
import java.util.Optional;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyString;
+import static org.mockito.Mockito.CALLS_REAL_METHODS;
import static org.mockito.Mockito.RETURNS_SELF;
-import static org.mockito.Mockito.lenient;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.inOrder;
import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.times;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;
+import static org.mockito.Mockito.withSettings;
class OpenSearchDomainManagerTest {
+ private EmulatorConfig config;
+ private EmulatorConfig.DockerConfig docker;
+ private EmulatorConfig.StorageConfig storage;
+ private EmulatorConfig.OpenSearchServiceConfig opensearch;
+ private ContainerLifecycleManager lifecycleManager;
+ private ContainerBuilder.Builder builder;
+ private ContainerDetector containerDetector;
+ private PortAllocator portAllocator;
+ private OpenSearchDomainManager manager;
+
+ @BeforeEach
+ void setUp() {
+ config = mock(EmulatorConfig.class);
+ docker = mock(EmulatorConfig.DockerConfig.class);
+ storage = mock(EmulatorConfig.StorageConfig.class);
+ EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);
+ opensearch = mock(EmulatorConfig.OpenSearchServiceConfig.class);
+ when(config.docker()).thenReturn(docker);
+ when(config.storage()).thenReturn(storage);
+ when(config.services()).thenReturn(services);
+ when(services.opensearch()).thenReturn(opensearch);
+ when(services.dockerNetwork()).thenReturn(Optional.empty());
+ when(docker.resourceNamespace()).thenReturn(Optional.empty());
+ when(storage.hostPersistentPath()).thenReturn("floci-data");
+ when(storage.persistentPath()).thenReturn("/data");
+ when(storage.mode()).thenReturn("memory");
+ when(opensearch.proxyBasePort()).thenReturn(9400);
+ when(opensearch.proxyMaxPort()).thenReturn(9499);
+ when(opensearch.defaultImage()).thenReturn(Optional.of("opensearchproject/opensearch:2.11.0"));
+ when(opensearch.dataVolume()).thenReturn(Optional.empty());
+
+ portAllocator = mock(PortAllocator.class);
+ when(portAllocator.allocate(9400, 9499)).thenReturn(9400);
+ lifecycleManager = mock(ContainerLifecycleManager.class);
+ when(lifecycleManager.createAndStart(any())).thenReturn(
+ new ContainerLifecycleManager.ContainerInfo("container-id", Map.of(
+ 9200, new ContainerLifecycleManager.EndpointInfo("172.17.0.5", 9200))));
+ ContainerBuilder containerBuilder = mock(ContainerBuilder.class);
+ builder = mock(ContainerBuilder.Builder.class, RETURNS_SELF);
+ when(containerBuilder.newContainer(anyString())).thenReturn(builder);
+ when(builder.build()).thenReturn(new ContainerSpec("test-image"));
+ containerDetector = mock(ContainerDetector.class);
+ manager = new OpenSearchDomainManager(containerBuilder, lifecycleManager,
+ containerDetector, portAllocator, config);
+ }
+
+ @Test
+ void startDomainRunsTheContainerUnderAnInitProcess() {
+ Domain domain = domain("000000000000", "us-east-1");
+
+ manager.startDomain(domain);
+
+ verify(builder).withInit();
+ }
+
@Test
void startDomainLabelsContainerWithResourceIdentity() {
- Started started = startDomain("my-domain", 9400, false);
+ Domain domain = domain("111111111111", "eu-west-1");
- verify(started.builder()).withLabels(Map.of(
+ manager.startDomain(domain);
+
+ assertEquals(VolumeOwnership.FLOCI_MANAGED, domain.getVolumeOwnership());
+ verify(builder).withLabels(Map.of(
"io.floci", "aws",
"io.floci.service", "opensearch",
"io.floci.resource-id", "my-domain",
- "io.floci.account", "000000000000",
- "io.floci.region", "us-east-1"));
+ "io.floci.account", "111111111111",
+ "io.floci.region", "eu-west-1"));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", "team-a"})
+ void startDomainSeparatesContainerNamesAndCleanupAcrossAccountsAndRegions(String namespace) {
+ when(docker.resourceNamespace()).thenReturn(Optional.of(namespace));
+ List<Domain> domains = List.of(domain("000000000000", "us-east-1"),
+ domain("111111111111", "us-east-1"), domain("000000000000", "eu-west-1"));
+
+ for (Domain domain : domains) {
+ manager.startDomain(domain);
+ }
+
+ String prefix = namespace.isEmpty() ? "floci-aws-opensearch-" : "floci-aws-team-a-opensearch-";
+ List<String> expectedNames = List.of(prefix + "000000000000-us-east-1-my-domain",
+ prefix + "111111111111-us-east-1-my-domain", prefix + "000000000000-eu-west-1-my-domain");
+ ArgumentCaptor<String> names = ArgumentCaptor.forClass(String.class);
+ verify(builder, times(3)).withName(names.capture());
+ assertEquals(expectedNames, names.getAllValues());
+ ArgumentCaptor<String> cleanup = ArgumentCaptor.forClass(String.class);
+ verify(lifecycleManager, times(6)).removeIfExists(cleanup.capture());
+ String legacyPrefix = namespace.isEmpty() ? "floci-opensearch-" : "floci-team-a-opensearch-";
+ assertEquals(List.of(expectedNames.get(0), legacyPrefix + "000000000000-us-east-1-my-domain",
+ expectedNames.get(1), legacyPrefix + "111111111111-us-east-1-my-domain",
+ expectedNames.get(2), legacyPrefix + "000000000000-eu-west-1-my-domain"), cleanup.getAllValues());
+ }
+
+ @Test
+ void startDomainMigratesLegacyContainerByPersistedIdAndKeepsItsVolume() {
+ Domain domain = domain("111111111111", "eu-west-1");
+ domain.setContainerId("legacy-container-id");
+ domain.setHostPort(9401);
+ domain.setDockerVolumeName("floci-opensearch-abc123");
+ when(docker.resourceNamespace()).thenReturn(Optional.of("new-ns"));
+
+ manager.startDomain(domain);
+
+ InOrder order = inOrder(lifecycleManager, portAllocator);
+ order.verify(lifecycleManager).removeIfExists("legacy-container-id");
+ order.verify(portAllocator).release(9401);
+ order.verify(portAllocator).allocate(9400, 9499);
+ verify(builder).withName("floci-aws-new-ns-opensearch-111111111111-eu-west-1-my-domain");
+ verify(builder).withNamedVolume("floci-opensearch-abc123", "/usr/share/opensearch/data");
+ verify(lifecycleManager, never()).removeIfExists("floci-aws-new-ns-opensearch-my-domain");
+ verify(lifecycleManager, never()).removeIfExists("floci-new-ns-opensearch-my-domain");
+ assertEquals("floci-opensearch-abc123", domain.getDockerVolumeName());
+ assertEquals("container-id", domain.getContainerId());
}
/**
@@ -48,23 +173,19 @@ class OpenSearchDomainManagerTest {
*/
@Test
void startDomainPublishesAHostPortEvenWhenFlociRunsInAContainer() {
- Started started = startDomain("host-reach", 9401, true);
+ when(portAllocator.allocate(9400, 9499)).thenReturn(9401);
+ when(containerDetector.isRunningInContainer()).thenReturn(true);
- verify(started.portAllocator()).allocate(9400, 9499);
- verify(started.builder()).withPortBinding(9200, 9401);
- assertEquals("http://172.17.0.5:9200", started.domain().getEndpoint());
- assertEquals(Integer.valueOf(9401), started.domain().getHostPort());
- }
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setDomainName("host-reach");
+ domain.setEngineVersion("OpenSearch_2.11");
- /**
- * The image's entrypoint keeps the server as a child of a bash PID 1, which ignores SIGTERM.
- * Without an init process each DeleteDomain waits out the whole stop timeout.
- */
- @Test
- void startDomainRunsTheContainerUnderAnInitProcess() {
- Started started = startDomain("init", 9400, false);
+ manager.startDomain(domain);
- verify(started.builder()).withInit();
+ verify(portAllocator).allocate(9400, 9499);
+ verify(builder).withPortBinding(9200, 9401);
+ assertEquals("http://172.17.0.5:9200", domain.getEndpoint());
+ assertEquals(Integer.valueOf(9401), domain.getHostPort());
}
/**
@@ -74,21 +195,9 @@ class OpenSearchDomainManagerTest {
*/
@Test
void stopDomainReleasesTheHostPort() {
- EmulatorConfig config = mock(EmulatorConfig.class);
- EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);
- EmulatorConfig.OpenSearchServiceConfig opensearch = mock(EmulatorConfig.OpenSearchServiceConfig.class);
- when(config.services()).thenReturn(services);
- when(services.opensearch()).thenReturn(opensearch);
when(opensearch.keepRunningOnShutdown()).thenReturn(false);
- PortAllocator portAllocator = mock(PortAllocator.class);
- ContainerLifecycleManager lifecycleManager = mock(ContainerLifecycleManager.class);
-
- OpenSearchDomainManager manager = new OpenSearchDomainManager(mock(ContainerBuilder.class),
- lifecycleManager, mock(ContainerDetector.class), portAllocator, config,
- mock(RegionResolver.class));
-
- Domain domain = new Domain();
+ Domain domain = domain("000000000000", "us-east-1");
domain.setDomainName("teardown");
domain.setContainerId("container-id");
domain.setHostPort(9401);
@@ -107,21 +216,9 @@ class OpenSearchDomainManagerTest {
*/
@Test
void stopDomainRemovesTheContainerEvenWhenShutdownRetentionIsEnabled() {
- EmulatorConfig config = mock(EmulatorConfig.class);
- EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);
- EmulatorConfig.OpenSearchServiceConfig opensearch = mock(EmulatorConfig.OpenSearchServiceConfig.class);
- when(config.services()).thenReturn(services);
- when(services.opensearch()).thenReturn(opensearch);
when(opensearch.keepRunningOnShutdown()).thenReturn(true);
- PortAllocator portAllocator = mock(PortAllocator.class);
- ContainerLifecycleManager lifecycleManager = mock(ContainerLifecycleManager.class);
-
- OpenSearchDomainManager manager = new OpenSearchDomainManager(mock(ContainerBuilder.class),
- lifecycleManager, mock(ContainerDetector.class), portAllocator, config,
- mock(RegionResolver.class));
-
- Domain domain = new Domain();
+ Domain domain = domain("000000000000", "us-east-1");
domain.setDomainName("deleted-domain");
domain.setContainerId("container-id");
domain.setHostPort(9401);
@@ -132,57 +229,360 @@ class OpenSearchDomainManagerTest {
verify(portAllocator).release(9401);
}
- private record Started(ContainerBuilder.Builder builder, PortAllocator portAllocator, Domain domain) {
+ /**
+ * A configured Docker volume name gives every domain its own
+ * {@code <name>-<account-id>-<region>-<domain-name>} volume, so no two domains share a data directory;
+ * the volume is created, when absent, without Floci labels so no sweep of
+ * Floci-labelled volumes collects operator storage; and the domain record is
+ * stamped with that name and the operator ownership marker so delete can tell
+ * operator storage from Floci-derived storage.
+ */
+ @Test
+ void startDomainMountsConfiguredNamedDataVolume() {
+ when(opensearch.dataVolume()).thenReturn(Optional.of("opensearch-data"));
+
+ Domain domain = domain("000000000000", "us-east-1");
+
+ manager.startDomain(domain);
+
+ verify(lifecycleManager).ensureUnmanagedVolume("opensearch-data-000000000000-us-east-1-my-domain");
+ verify(builder).withNamedVolume("opensearch-data-000000000000-us-east-1-my-domain", "/usr/share/opensearch/data");
+ assertEquals("opensearch-data-000000000000-us-east-1-my-domain", domain.getDockerVolumeName());
+ assertEquals(VolumeOwnership.OPERATOR_NAMED, domain.getVolumeOwnership());
}
- /** Starts an OpenSearch 2.11 domain against a mocked builder and lifecycle, with no Docker. */
- private static Started startDomain(String domainName, int hostPort, boolean flociInContainer) {
- EmulatorConfig config = mock(EmulatorConfig.class);
- EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);
- EmulatorConfig.OpenSearchServiceConfig opensearch = mock(EmulatorConfig.OpenSearchServiceConfig.class);
- when(config.services()).thenReturn(services);
- when(services.opensearch()).thenReturn(opensearch);
- when(services.dockerNetwork()).thenReturn(Optional.empty());
- when(opensearch.proxyBasePort()).thenReturn(9400);
- when(opensearch.proxyMaxPort()).thenReturn(9499);
- when(opensearch.defaultImage()).thenReturn(Optional.of("opensearchproject/opensearch:2.11.0"));
- EmulatorConfig.DockerConfig docker = mock(EmulatorConfig.DockerConfig.class);
- when(config.docker()).thenReturn(docker);
- lenient().when(docker.logMaxSize()).thenReturn("10m");
- lenient().when(docker.logMaxFile()).thenReturn("3");
- EmulatorConfig.StorageConfig storage = mock(EmulatorConfig.StorageConfig.class);
- when(config.storage()).thenReturn(storage);
- when(storage.hostPersistentPath()).thenReturn("floci-data");
- when(storage.persistentPath()).thenReturn("/data");
+ /**
+ * A namespaced deployment composes the configured volume name with its namespace,
+ * so two namespaced deployments sharing a Docker daemon and the same configured
+ * base never converge on one volume for the same domain name.
+ */
+ @Test
+ void startDomainMountsConfiguredNamedDataVolumeUnderTheResourceNamespace() {
+ when(opensearch.dataVolume()).thenReturn(Optional.of("opensearch-data"));
+ when(docker.resourceNamespace()).thenReturn(Optional.of("team-a"));
- PortAllocator portAllocator = mock(PortAllocator.class);
- when(portAllocator.allocate(9400, 9499)).thenReturn(hostPort);
+ Domain domain = domain("000000000000", "us-east-1");
- ContainerLifecycleManager lifecycleManager = mock(ContainerLifecycleManager.class);
- when(lifecycleManager.createAndStart(any())).thenReturn(
- new ContainerLifecycleManager.ContainerInfo("container-id", Map.of(
- 9200, new ContainerLifecycleManager.EndpointInfo("172.17.0.5", 9200))));
+ manager.startDomain(domain);
- ContainerBuilder containerBuilder = mock(ContainerBuilder.class);
- ContainerBuilder.Builder builder = mock(ContainerBuilder.Builder.class, RETURNS_SELF);
- when(containerBuilder.newContainer(anyString())).thenReturn(builder);
- when(builder.build()).thenReturn(new ContainerSpec("test-image"));
+ verify(lifecycleManager).ensureUnmanagedVolume("opensearch-data-team-a-000000000000-us-east-1-my-domain");
+ verify(builder).withNamedVolume("opensearch-data-team-a-000000000000-us-east-1-my-domain",
+ "/usr/share/opensearch/data");
+ assertEquals("opensearch-data-team-a-000000000000-us-east-1-my-domain", domain.getDockerVolumeName());
+ assertEquals(VolumeOwnership.OPERATOR_NAMED, domain.getVolumeOwnership());
+ }
+
+ /**
+ * A configured absolute path bind-mounts {@code <path>/<account-id>/<region>/<domain-name>}: per-domain
+ * subdirectory, no named volume on the domain record, and the bind ownership
+ * marker in place of a volume name.
+ */
+ @Test
+ void startDomainBindsConfiguredHostDataPathPerDomain(@TempDir Path dataRoot) {
+ when(opensearch.dataVolume()).thenReturn(Optional.of(dataRoot.toString()));
- RegionResolver regionResolver = mock(RegionResolver.class);
- when(regionResolver.getAccountId()).thenReturn("000000000000");
- when(regionResolver.getDefaultRegion()).thenReturn("us-east-1");
+ Domain domain = domain("000000000000", "us-east-1");
- ContainerDetector containerDetector = mock(ContainerDetector.class);
- lenient().when(containerDetector.isRunningInContainer()).thenReturn(flociInContainer);
+ manager.startDomain(domain);
- OpenSearchDomainManager manager = new OpenSearchDomainManager(containerBuilder, lifecycleManager,
- containerDetector, portAllocator, config, regionResolver);
+ String expectedBind = dataRoot.resolve("000000000000").resolve("us-east-1").resolve("my-domain")
+ .toAbsolutePath().normalize().toString();
+ InOrder order = inOrder(lifecycleManager);
+ order.verify(lifecycleManager).ensureBindDirectory(expectedBind,
+ "opensearchproject/opensearch:2.11.0", "/usr/share/opensearch/data");
+ order.verify(lifecycleManager).createAndStart(any());
+ verify(builder).withBind(expectedBind, "/usr/share/opensearch/data");
+ verify(builder, never()).withNamedVolume(anyString(), anyString());
+ assertNull(domain.getDockerVolumeName());
+ assertEquals(VolumeOwnership.OPERATOR_BIND, domain.getVolumeOwnership());
+ }
- Domain domain = new Domain();
- domain.setDomainName(domainName);
- domain.setEngineVersion("OpenSearch_2.11");
+ /**
+ * A namespaced deployment nests the configured bind path under a namespace
+ * subdirectory, so two namespaced deployments sharing a Docker daemon host and
+ * the same configured base never converge on one data directory for the same
+ * domain name.
+ */
+ @Test
+ void startDomainBindsConfiguredHostDataPathUnderTheResourceNamespace(@TempDir Path dataRoot) {
+ when(opensearch.dataVolume()).thenReturn(Optional.of(dataRoot.toString()));
+ when(docker.resourceNamespace()).thenReturn(Optional.of("team-a"));
+
+ Domain domain = domain("000000000000", "us-east-1");
manager.startDomain(domain);
- return new Started(builder, portAllocator, domain);
+
+ String expectedBind = dataRoot.resolve("team-a").resolve("000000000000").resolve("us-east-1").resolve("my-domain")
+ .toAbsolutePath().normalize().toString();
+ InOrder order = inOrder(lifecycleManager);
+ order.verify(lifecycleManager).ensureBindDirectory(expectedBind,
+ "opensearchproject/opensearch:2.11.0", "/usr/share/opensearch/data");
+ order.verify(lifecycleManager).createAndStart(any());
+ verify(builder).withBind(expectedBind, "/usr/share/opensearch/data");
+ verify(builder, never()).withNamedVolume(anyString(), anyString());
+ assertNull(domain.getDockerVolumeName());
+ assertEquals(VolumeOwnership.OPERATOR_BIND, domain.getVolumeOwnership());
+ }
+
+ @ParameterizedTest
+ @CsvSource({
+ "/srv/floci/opensearch, '', /srv/floci/opensearch/",
+ "/srv/floci/opensearch, team-a, /srv/floci/opensearch/team-a/",
+ "/srv/floci/opensearch/, '', /srv/floci/opensearch/",
+ "/srv/floci/opensearch/, team-a, /srv/floci/opensearch/team-a/",
+ "/, '', /",
+ "/, team-a, /team-a/"
+ })
+ void startDomainPreservesLinuxDaemonBindPaths(String dataVolume, String namespace, String expectedPrefix) {
+ when(opensearch.dataVolume()).thenReturn(Optional.of(dataVolume));
+ when(docker.resourceNamespace()).thenReturn(Optional.of(namespace));
+ Domain domain = domain("000000000000", "us-east-1");
+ String expectedBind = expectedPrefix + "000000000000/us-east-1/my-domain";
+ try (MockedStatic<ContainerStorageHelper> storageHelper =
+ mockStatic(ContainerStorageHelper.class,
+ withSettings().mockMaker(MockMakers.INLINE).defaultAnswer(CALLS_REAL_METHODS))) {
+ storageHelper.when(() -> ContainerStorageHelper.ensureHostDir(anyString())).thenAnswer(invocation -> null);
+
+ manager.startDomain(domain);
+
+ if (Path.of("/").isAbsolute()) {
+ storageHelper.verify(() -> ContainerStorageHelper.ensureHostDir(expectedBind));
+ } else {
+ storageHelper.verify(() -> ContainerStorageHelper.ensureHostDir(anyString()), never());
+ }
+ }
+
+ verify(lifecycleManager).ensureBindDirectory(expectedBind,
+ "opensearchproject/opensearch:2.11.0", "/usr/share/opensearch/data");
+ verify(builder).withBind(expectedBind, "/usr/share/opensearch/data");
+ verify(builder, never()).withNamedVolume(anyString(), anyString());
+ verify(lifecycleManager, never()).ensureUnmanagedVolume(anyString());
+ assertNull(domain.getDockerVolumeName());
+ assertEquals(VolumeOwnership.OPERATOR_BIND, domain.getVolumeOwnership());
+ }
+
+ /** An operator-named volume is never deleted with the domain, whatever the storage policy. */
+ @Test
+ void removeDomainStorageRetainsOperatorNamedVolume() {
+ when(storage.pruneVolumesOnDelete()).thenReturn(true);
+
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setVolumeId("abc123");
+ domain.setDockerVolumeName("opensearch-data-000000000000-us-east-1-my-domain");
+ domain.setVolumeOwnership(VolumeOwnership.OPERATOR_NAMED);
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager, never()).removeVolume(anyString());
+ }
+
+ /** A volume Floci derived itself, current prefix, is still removed as before. */
+ @Test
+ void removeDomainStorageRemovesFlociDerivedVolume() {
+ when(storage.pruneVolumesOnDelete()).thenReturn(false);
+
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setVolumeId("abc123");
+ domain.setDockerVolumeName(
+ ContainerStorageHelper.resourceName(config, "opensearch", "abc123", "my-domain"));
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager).removeVolume("floci-aws-opensearch-abc123");
+ }
+
+ /** A pre-migration record's legacy-named volume is as deletable as a current-prefix one. */
+ @Test
+ void removeDomainStorageRemovesLegacyDerivedVolume() {
+ when(storage.pruneVolumesOnDelete()).thenReturn(false);
+
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setVolumeId("abc123");
+ domain.setDockerVolumeName(
+ ContainerStorageHelper.legacyResourceName(config, "opensearch", "abc123", "my-domain"));
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager).removeVolume("floci-opensearch-abc123");
+ }
+
+ /**
+ * A bind-backed domain mounts no named volume, so delete removes no Docker volume at
+ * all, whatever the storage policy: a stale managed name on the record must not send
+ * it after a volume the domain never mounted.
+ */
+ @Test
+ void removeDomainStorageRemovesNoDockerVolumeForBindBackedDomain() {
+ when(storage.pruneVolumesOnDelete()).thenReturn(true);
+
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setVolumeOwnership(VolumeOwnership.OPERATOR_BIND);
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager, never()).removeVolume(anyString());
+ }
+
+ @Test
+ void removeDomainStorageRemovesLegacyRecordRegardlessOfConfiguredDataVolume() {
+ when(storage.pruneVolumesOnDelete()).thenReturn(true);
+ when(opensearch.dataVolume()).thenReturn(Optional.of("floci-opensearch"));
+
+ Domain domain = domain("000000000000", "us-east-1");
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager).removeVolume("floci-opensearch-my-domain");
+ }
+
+ /**
+ * The marker, not the spelling, decides: an operator-named volume is retained even
+ * when its name is spelled exactly like the legacy name Floci derives for the same
+ * domain.
+ */
+ @Test
+ void removeDomainStorageRetainsOperatorNamedVolumeEvenWhenItMatchesTheLegacyDerivedName() {
+ when(storage.pruneVolumesOnDelete()).thenReturn(true);
+ when(opensearch.dataVolume()).thenReturn(Optional.of("floci-opensearch"));
+
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setDockerVolumeName("floci-opensearch-my-domain");
+ domain.setVolumeOwnership(VolumeOwnership.OPERATOR_NAMED);
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager, never()).removeVolume(anyString());
+ }
+
+ /**
+ * A managed volume is removed by its persisted name, not by names recomputed from
+ * the configuration held at delete time: a resource-namespace change between
+ * creation and deletion must not strand the volume under the prune policy.
+ */
+ @Test
+ void removeDomainStorageRemovesManagedVolumeByPersistedNameWhenTheNamespaceChanged() {
+ when(docker.resourceNamespace()).thenReturn(Optional.of("new-ns"));
+ when(storage.pruneVolumesOnDelete()).thenReturn(false);
+
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setVolumeId("abc123");
+ domain.setDockerVolumeName("floci-aws-opensearch-abc123");
+ domain.setVolumeOwnership(VolumeOwnership.FLOCI_MANAGED);
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager).removeVolume("floci-aws-opensearch-abc123");
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", "team-a"})
+ void configuredNamedStorageSeparatesAccountsAndRegionsAndReusesDomainIdentity(String namespace) {
+ when(docker.resourceNamespace()).thenReturn(Optional.of(namespace));
+ when(opensearch.dataVolume()).thenReturn(Optional.of("opensearch-data"));
+ List<Domain> domains = List.of(domain("000000000000", "us-east-1"),
+ domain("111111111111", "us-east-1"), domain("000000000000", "eu-west-1"),
+ domain("000000000000", "us-east-1"));
+
+ for (Domain domain : domains) {
+ manager.startDomain(domain);
+ }
+
+ String prefix = namespace.isEmpty() ? "opensearch-data-" : "opensearch-data-team-a-";
+ List<String> expectedNames = List.of(prefix + "000000000000-us-east-1-my-domain",
+ prefix + "111111111111-us-east-1-my-domain", prefix + "000000000000-eu-west-1-my-domain",
+ prefix + "000000000000-us-east-1-my-domain");
+ assertEquals(expectedNames, domains.stream().map(Domain::getDockerVolumeName).toList());
+ ArgumentCaptor<String> mounts = ArgumentCaptor.forClass(String.class);
+ verify(builder, times(4)).withNamedVolume(mounts.capture(), anyString());
+ assertEquals(expectedNames, mounts.getAllValues());
+ verify(lifecycleManager, times(2)).ensureUnmanagedVolume(expectedNames.getFirst());
+ verify(lifecycleManager).ensureUnmanagedVolume(expectedNames.get(1));
+ verify(lifecycleManager).ensureUnmanagedVolume(expectedNames.get(2));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", "team-a"})
+ void configuredBindStorageSeparatesAccountsAndRegionsAndReusesDomainIdentity(
+ String namespace, @TempDir Path dataRoot) {
+ when(docker.resourceNamespace()).thenReturn(Optional.of(namespace));
+ when(opensearch.dataVolume()).thenReturn(Optional.of(dataRoot.toString()));
+ List<Domain> domains = List.of(domain("000000000000", "us-east-1"),
+ domain("111111111111", "us-east-1"), domain("000000000000", "eu-west-1"),
+ domain("000000000000", "us-east-1"));
+
+ for (Domain domain : domains) {
+ manager.startDomain(domain);
+ }
+
+ Path root = dataRoot.resolve(namespace);
+ List<Path> expectedPaths = List.of(root.resolve("000000000000/us-east-1/my-domain"),
+ root.resolve("111111111111/us-east-1/my-domain"), root.resolve("000000000000/eu-west-1/my-domain"),
+ root.resolve("000000000000/us-east-1/my-domain"));
+ ArgumentCaptor<String> mounts = ArgumentCaptor.forClass(String.class);
+ verify(builder, times(4)).withBind(mounts.capture(), anyString());
+ assertEquals(expectedPaths.stream().map(path -> path.toAbsolutePath().normalize().toString()).toList(),
+ mounts.getAllValues());
+ for (Path path : expectedPaths) {
+ assertTrue(Files.isDirectory(path));
+ }
+ verify(builder, never()).withNamedVolume(anyString(), anyString());
+ }
+
+ @Test
+ void removeDomainStorageRemovesUnmarkedVolumeByPersistedNameWhenTheNamespaceChanged() {
+ when(docker.resourceNamespace()).thenReturn(Optional.of("new-ns"));
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setDockerVolumeName("floci-opensearch-old-ns-abc123");
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager).removeVolume("floci-opensearch-old-ns-abc123");
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"persistent", "hybrid", "wal"})
+ void removeDomainStorageRetainsUnmarkedVolumeWhenPruningIsDisabled(String mode) {
+ when(storage.mode()).thenReturn(mode);
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setDockerVolumeName("floci-opensearch-abc123");
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager, never()).removeVolume(anyString());
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"persistent", "hybrid", "wal"})
+ void removeDomainStorageRemovesUnmarkedVolumeWhenPruningIsEnabled(String mode) {
+ when(storage.mode()).thenReturn(mode);
+ when(storage.pruneVolumesOnDelete()).thenReturn(true);
+ Domain domain = domain("000000000000", "us-east-1");
+ domain.setDockerVolumeName("floci-opensearch-abc123");
+
+ manager.removeDomainStorage(domain);
+
+ verify(lifecycleManager).removeVolume("floci-opensearch-abc123");
+ }
+
+ @Test
+ void startDomainReleasesPortAndDoesNotStartWhenBindPreparationFails(@TempDir Path dataRoot) {
+ when(opensearch.dataVolume()).thenReturn(Optional.of(dataRoot.toString()));
+ doThrow(new IllegalStateException("init failed")).when(lifecycleManager)
+ .ensureBindDirectory(anyString(), anyString(), anyString());
+ Domain domain = domain("000000000000", "us-east-1");
+
+ assertThrows(IllegalStateException.class, () -> manager.startDomain(domain));
+
+ verify(portAllocator).release(9400);
+ verify(lifecycleManager, never()).createAndStart(any());
+ assertNull(domain.getHostPort());
+ }
+
+ private Domain domain(String accountId, String region) {
+ Domain domain = new Domain();
+ domain.setDomainName("my-domain");
+ domain.setEngineVersion("OpenSearch_2.11");
+ domain.setArn(AwsArnUtils.Arn.of("es", region, accountId, "domain/my-domain").toString());
+ return domain;
}
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchServiceTest.java
index 4c5bdd6238..f4d40c0f98 100644
--- a/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/opensearch/OpenSearchServiceTest.java
@@ -4,6 +4,7 @@ import io.github.hectorvent.floci.config.EmulatorConfig;
import io.github.hectorvent.floci.core.common.RegionResolver;
import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;
import io.github.hectorvent.floci.services.opensearch.model.Domain;
+import io.github.hectorvent.floci.services.opensearch.model.VolumeOwnership;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.mockito.Mockito;
@@ -55,6 +56,23 @@ class OpenSearchServiceTest {
verify(domainManager).tryStartDomain(any());
}
+ /**
+ * The volume name and the ownership marker are persisted at creation, so deletion
+ * acts on what was stamped instead of recomputing ownership from whatever
+ * configuration holds at delete time.
+ */
+ @Test
+ void createDomainStampsManagedVolumeNameAndOwnership() {
+ when(domainManager.tryStartDomain(any())).thenReturn(false);
+
+ Domain domain = service.createDomain("stamped-domain", "OpenSearch_2.11",
+ null, null, null, "us-east-1");
+
+ assertEquals(VolumeOwnership.FLOCI_MANAGED, domain.getVolumeOwnership());
+ assertTrue(domain.getDockerVolumeName().startsWith("floci-aws-opensearch-"),
+ "the stamped name carries the current managed prefix");
+ }
+
@Test
void createDomainWithoutDockerDaemonStillSucceedsAndReportsProcessingFalse() {
// tryStartDomain() returns false when no Docker daemon is reachable. The domain record is