Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(sagemaker): support GPU-backed training jobs (#4015)

Austin We14221fe621b
diff --git a/docs/services/sagemaker.md b/docs/services/sagemaker.md
index ddba8dd45a..f276e80fe5 100644
--- a/docs/services/sagemaker.md
+++ b/docs/services/sagemaker.md
@@ -44,10 +44,48 @@ Floci emulates SageMaker control-plane APIs and runs real Docker containers for
`CreateTrainingJob` starts `AlgorithmSpecification.TrainingImage` with command `train` unless `ContainerEntrypoint`/`ContainerArguments` are supplied. Floci writes SageMaker config files under `/opt/ml/input/config`, downloads channel data from S3 into `/opt/ml/input/data/<channel>`, waits for container exit, and uploads `/opt/ml/model` as `model.tar.gz` under `OutputDataConfig.S3OutputPath/<TrainingJobName>/output/`.
+## GPU-backed training
+
+On AWS, `ResourceConfig.InstanceType` *is* the hardware: `ml.g5.xlarge` provisions a machine that physically has one A10G. Floci has one host with whatever cards it has, so a GPU instance type is served by substituting a local device. That substitution is off by default and the SDK request is unchanged either way.
+
+How many accelerators each `ml.*` type carries is a property of AWS, so it ships with Floci in `sagemaker/instance-type-catalog.yaml`. Configuration covers only the local decision of which of this machine's devices may be handed out:
+
+```yaml
+floci:
+ services:
+ sagemaker:
+ gpu:
+ enabled: true
+ mode: cdi # cdi | device-ids | count
+ devices:
+ - nvidia.com/gpu=GPU-0e1f2a3b
+```
+
+| Setting | Meaning |
+| --- | --- |
+| `enabled` | Off by default. While off, every training container is CPU-only, exactly as before. |
+| `mode` | How the request reaches the daemon. `cdi` names [Container Device Interface](https://github.com/cncf-tags/container-device-interface) devices, `device-ids` uses daemon device ids, and `count` asks for a number and lets the daemon choose. |
+| `devices` | The devices Floci may use, for `cdi` and `device-ids`. Unset allows none, so a shared machine does not hand over every GPU by default. Ignored by `count`. |
+
+`mode` defaults to `cdi` because daemons differ: Podman resolves only CDI and will accept a count request while attaching no device ([containers/podman#22645](https://github.com/containers/podman/issues/22645)). Choosing CDI means a misconfiguration fails the container start rather than quietly training on CPU. Use `count` or `device-ids` against Docker.
+
+Floci does not reserve configured devices across concurrent training jobs. In `cdi` and `device-ids` modes, each job takes the first devices it needs from the allowlist, so two simultaneous one-GPU jobs can target the same local card and contend for memory.
+
+A training job fails with an explanatory `FailureReason`, rather than running on CPU, when:
+
+- the instance type belongs to a GPU family (`ml.g*`, `ml.p*`, `ml.inf*`, `ml.trn*`) that the catalog does not list, so its accelerator count is unknown
+- fewer devices are allowed than the instance type needs
+- `cdi` or `device-ids` is selected with no `devices` configured
+- `InstanceCount` is above one, since Floci runs a job as a single container
+
+Only the accelerator is substituted. An instance type also implies vCPU and memory, which Floci does not emulate for any service: EC2 instance types are metadata for `DescribeInstanceTypes` and do not size containers, and RDS records `DBInstanceClass` without acting on it. Resource limits are applied only where an API supplies an explicit number, as Lambda's `MemorySize` does.
+
## Endpoint hosting
`CreateEndpoint` starts the model image as a long-lived Docker container with command `serve`, port `8080`, `/ping` health checks, and `/invocations` runtime proxying. `ModelDataUrl` artifacts are downloaded from S3 and placed in `/opt/ml/model`.
+Endpoint containers are CPU-only for now; the `gpu` settings above apply to training jobs.
+
## Examples
```python
diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
index fde2bc5881..12fe844f1f 100644
--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
@@ -1138,6 +1138,57 @@ public interface EmulatorConfig {
boolean enabled();
Optional<String> dockerNetwork();
+
+ GpuConfig gpu();
+
+ /**
+ * Whether, and how, this host lends its accelerators to training containers.
+ *
+ * <p>How many GPUs an instance type has is an AWS fact and lives in the shipped
+ * catalog; this covers only the local decision of which devices Floci may use.
+ */
+ interface GpuConfig {
+ /**
+ * Off by default, so an existing deployment keeps launching CPU-only
+ * containers exactly as before.
+ */
+ @WithDefault("false")
+ boolean enabled();
+
+ /**
+ * How the device request is expressed on the wire, because daemons disagree.
+ *
+ * <p>Defaults to {@code cdi}: Podman resolves only that form, and accepts the
+ * {@code count} form while attaching no device
+ * (containers/podman#22645). Preferring CDI means a misconfiguration fails
+ * the container start instead of silently training on CPU. Use {@code count}
+ * or {@code device-ids} against Docker.
+ */
+ @WithDefault("cdi")
+ GpuRequestMode mode();
+
+ /**
+ * The devices Floci may hand out: CDI names such as
+ * {@code nvidia.com/gpu=GPU-<uuid>} for {@code cdi} mode, or daemon device ids
+ * for {@code device-ids} mode.
+ *
+ * <p>Unset allows no device, so a training job in those modes fails rather than
+ * starting. Defaulting to "every device the daemon exposes" would be the wrong
+ * behaviour on a machine sharing GPUs with other workloads.
+ *
+ * <p>Ignored in {@code count} mode, where the daemon does the choosing.
+ *
+ * <p>Optional rather than a defaulted list because SmallRye rejects an empty
+ * string as a collection default: unset simply means no device is allowed.
+ */
+ Optional<List<String>> devices();
+ }
+
+ enum GpuRequestMode {
+ CDI,
+ DEVICE_IDS,
+ COUNT
+ }
}
interface CodeDeployServiceConfig {
diff --git a/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuResolver.java b/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuResolver.java
new file mode 100644
index 0000000000..80449ac964
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuResolver.java
@@ -0,0 +1,130 @@
+package io.github.hectorvent.floci.services.sagemaker;
+
+import io.github.hectorvent.floci.config.EmulatorConfig;
+import io.github.hectorvent.floci.config.EmulatorConfig.SageMakerServiceConfig.GpuRequestMode;
+import io.github.hectorvent.floci.core.common.docker.ContainerBuilder;
+import jakarta.enterprise.context.ApplicationScoped;
+import jakarta.inject.Inject;
+
+import java.util.List;
+import java.util.Optional;
+
+/**
+ * Turns a SageMaker {@code InstanceType} into a device request on a container, or into a
+ * clear refusal.
+ *
+ * <p>Currently used by training jobs. It is kept separate from the training runner so the
+ * same rules can be reused if GPU-backed endpoint hosting is added later.
+ *
+ * <p>The design rule throughout: never answer a GPU request with a CPU container. A job
+ * that asks for {@code ml.g5.xlarge}, trains on CPU and reports {@code Completed} leaves
+ * behind an artifact that looks legitimate, which is worse than a failure. Every path that
+ * cannot honour the request raises {@link UnsupportedResourceException} instead.
+ *
+ * <p>Only the accelerator is mapped. An instance type also implies vCPU and memory, and
+ * Floci does not emulate those: the EC2 catalog is metadata for DescribeInstanceTypes and
+ * never sizes a container, and RDS keeps DBInstanceClass without acting on it. Limits are
+ * applied only where an AWS API hands over an explicit number, as Lambda's MemorySize and
+ * ECS task memory do.
+ */
+@ApplicationScoped
+public class SageMakerGpuResolver {
+
+ private final EmulatorConfig config;
+ private final SageMakerInstanceTypeCatalog catalog;
+
+ @Inject
+ public SageMakerGpuResolver(EmulatorConfig config, SageMakerInstanceTypeCatalog catalog) {
+ this.config = config;
+ this.catalog = catalog;
+ }
+
+ /**
+ * Raised when the request names hardware this host cannot stand in for. Callers surface
+ * the message as the AWS failure reason, so it has to say what is wrong and what would
+ * fix it.
+ */
+ public static class UnsupportedResourceException extends RuntimeException {
+ public UnsupportedResourceException(String message) {
+ super(message);
+ }
+ }
+
+ /**
+ * Applies a device request to {@code builder} when the instance type calls for one.
+ *
+ * @param instanceType the requested {@code ml.*} type, as given in the AWS request
+ * @param instanceCount instances requested; anything above one is refused, since a
+ * single container is not distributed training
+ * @throws UnsupportedResourceException when the request cannot be honoured
+ */
+ public void applyTo(ContainerBuilder.Builder builder, String instanceType, int instanceCount) {
+ EmulatorConfig.SageMakerServiceConfig.GpuConfig gpu = config.services().sagemaker().gpu();
+ if (!gpu.enabled()) {
+ // Feature off: behave exactly as before, including for GPU instance types.
+ // Turning it on is what opts a deployment into the stricter rules below.
+ return;
+ }
+
+ Optional<Integer> catalogued = catalog.gpuCount(instanceType);
+ if (catalogued.isEmpty()) {
+ if (SageMakerInstanceTypeCatalog.isAcceleratorFamily(instanceType)) {
+ throw new UnsupportedResourceException(
+ "Instance type " + instanceType + " is a GPU family but is not in Floci's "
+ + "SageMaker instance type catalog, so the number of accelerators it "
+ + "should receive is unknown. Add it to the catalog or choose a "
+ + "listed type.");
+ }
+ // An unlisted non-accelerator type is a CPU job, which needs nothing from us.
+ return;
+ }
+
+ int requested = catalogued.get();
+ if (requested == 0) {
+ return;
+ }
+ if (instanceCount > 1) {
+ throw new UnsupportedResourceException(
+ "Floci runs a training job as a single container, so InstanceCount " + instanceCount
+ + " with GPU instance type " + instanceType + " cannot be honoured. Use "
+ + "InstanceCount 1.");
+ }
+
+ List<String> devices = gpu.devices().orElse(List.of());
+ GpuRequestMode mode = gpu.mode();
+ if (mode != GpuRequestMode.COUNT && !devices.isEmpty() && devices.size() < requested) {
+ throw new UnsupportedResourceException(
+ "Instance type " + instanceType + " needs " + requested + " GPUs but only "
+ + devices.size() + " device(s) are allowed by "
+ + "floci.services.sagemaker.gpu.devices.");
+ }
+
+ switch (mode) {
+ case CDI -> {
+ requireDevicesConfigured(devices, "cdi", instanceType);
+ builder.withCdiDevices(devices.subList(0, requested));
+ }
+ case DEVICE_IDS -> {
+ requireDevicesConfigured(devices, "device-ids", instanceType);
+ builder.withGpuDeviceIds(devices.subList(0, requested));
+ }
+ // The daemon picks the devices, so no allowlist applies.
+ case COUNT -> builder.withGpuCount(requested);
+ }
+ }
+
+ /**
+ * Both explicit-device modes need to be told which devices exist. Defaulting to "all
+ * of them" would quietly hand Floci every GPU on a machine that is usually sharing
+ * them with something else.
+ */
+ private static void requireDevicesConfigured(List<String> devices, String mode, String instanceType) {
+ if (devices.isEmpty()) {
+ throw new UnsupportedResourceException(
+ "GPU support is enabled in " + mode + " mode but "
+ + "floci.services.sagemaker.gpu.devices is empty, so there is no device to "
+ + "give " + instanceType + ". List the devices Floci may use, or switch to "
+ + "count mode to let the daemon choose.");
+ }
+ }
+}
diff --git a/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerInstanceTypeCatalog.java b/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerInstanceTypeCatalog.java
new file mode 100644
index 0000000000..268f818540
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerInstanceTypeCatalog.java
@@ -0,0 +1,137 @@
+package io.github.hectorvent.floci.services.sagemaker;
+
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.dataformat.yaml.YAMLFactory;
+import io.quarkus.runtime.annotations.RegisterForReflection;
+import jakarta.enterprise.context.ApplicationScoped;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+
+/**
+ * How many accelerators each SageMaker {@code ml.*} instance type carries.
+ *
+ * <p>This is shipped data rather than configuration for the same reason
+ * {@code Ec2InstanceTypeCatalog} is: the GPU count of an {@code ml.g5.xlarge} is a fact
+ * about AWS, identical for everyone, and not something an operator should have to
+ * restate. What the local machine is willing to hand over is the separate, genuinely
+ * local decision, and lives under {@code floci.services.sagemaker.gpu}.
+ *
+ * <p>Deliberately not merged into the EC2 catalog: {@code ml.*} is its own namespace and
+ * these types must never appear in {@code DescribeInstanceTypes}.
+ */
+@ApplicationScoped
+public class SageMakerInstanceTypeCatalog {
+
+ private static final String CATALOG_RESOURCE_NAME = "sagemaker/instance-type-catalog.yaml";
+ private static final ObjectMapper YAML_MAPPER = new ObjectMapper(new YAMLFactory());
+
+ /**
+ * Families whose members carry accelerators. Used only to decide whether an unknown
+ * type should be refused rather than run on CPU: a caller naming an unlisted
+ * {@code ml.g*} clearly wanted a GPU, and silently giving them CPU would produce a
+ * job that reports Completed with a legitimate-looking artifact.
+ *
+ * <p>Never used to infer a count. Counts vary within a family, which is why the
+ * catalog is a table.
+ */
+ private static final Set<String> ACCELERATOR_FAMILY_PREFIXES =
+ Set.of("ml.g", "ml.p", "ml.inf", "ml.trn");
+
+ private volatile Loaded loaded;
+
+ public SageMakerInstanceTypeCatalog() {
+ // Load lazily, matching Ec2InstanceTypeCatalog: a deployment that never touches
+ // SageMaker should not pay for resource loading at bean construction.
+ }
+
+ SageMakerInstanceTypeCatalog(Catalog catalog) {
+ this.loaded = new Loaded(catalog);
+ }
+
+ /**
+ * The accelerator count for a type, or empty when the catalog does not describe it.
+ * An empty result is not "no GPU": see {@link #isAcceleratorFamily(String)}.
+ */
+ public Optional<Integer> gpuCount(String instanceType) {
+ if (instanceType == null || instanceType.isBlank()) {
+ return Optional.empty();
+ }
+ CatalogInstanceType found = loaded().byName.get(instanceType.trim());
+ return found == null ? Optional.empty() : Optional.of(found.gpuCount);
+ }
+
+ /**
+ * Whether the type belongs to a family that carries accelerators on AWS, regardless of
+ * whether this catalog lists the specific size.
+ *
+ * <p>Case sensitive, like {@link #gpuCount(String)} and like EC2's {@code find} and
+ * {@code familyOf}: AWS instance types are exact tokens, so folding case here would make
+ * this disagree with the catalog lookup and answer "GPU family" for a spelling the lookup
+ * then reports as missing from the catalog.
+ */
+ public static boolean isAcceleratorFamily(String instanceType) {
+ if (instanceType == null || instanceType.isBlank()) {
+ return false;
+ }
+ String normalized = instanceType.trim();
+ return ACCELERATOR_FAMILY_PREFIXES.stream().anyMatch(normalized::startsWith);
+ }
+
+ private Loaded loaded() {
+ Loaded current = loaded;
+ if (current == null) {
+ synchronized (this) {
+ current = loaded;
+ if (current == null) {
+ current = new Loaded(readCatalog());
+ loaded = current;
+ }
+ }
+ }
+ return current;
+ }
+
+ private static Catalog readCatalog() {
+ try (InputStream in = Thread.currentThread().getContextClassLoader()
+ .getResourceAsStream(CATALOG_RESOURCE_NAME)) {
+ if (in == null) {
+ throw new IllegalStateException("Missing catalog resource " + CATALOG_RESOURCE_NAME);
+ }
+ return YAML_MAPPER.readValue(in, Catalog.class);
+ } catch (IOException e) {
+ throw new IllegalStateException("Could not read " + CATALOG_RESOURCE_NAME, e);
+ }
+ }
+
+ private static final class Loaded {
+ private final Map<String, CatalogInstanceType> byName;
+
+ private Loaded(Catalog catalog) {
+ Map<String, CatalogInstanceType> index = new LinkedHashMap<>();
+ for (CatalogInstanceType type : catalog.instanceTypes) {
+ index.put(type.instanceType, type);
+ }
+ this.byName = Map.copyOf(index);
+ }
+ }
+
+ @RegisterForReflection
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ public static final class Catalog {
+ public List<CatalogInstanceType> instanceTypes = List.of();
+ }
+
+ @RegisterForReflection
+ @JsonIgnoreProperties(ignoreUnknown = true)
+ public static final class CatalogInstanceType {
+ public String instanceType;
+ public int gpuCount;
+ }
+}
diff --git a/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunner.java b/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunner.java
index 76c979dc3f..b7b696fee7 100644
--- a/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunner.java
+++ b/src/main/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunner.java
@@ -54,6 +54,7 @@ public class SageMakerTrainingRunner implements ContainerTeardown, Resettable {
private final ContainerDetector containerDetector;
private final S3Service s3Service;
private final ObjectMapper mapper;
+ private final SageMakerGpuResolver gpuResolver;
// Replaced by afterReset() after a state reset, whose container teardown shuts this pool down.
private volatile ExecutorService executor = Executors.newCachedThreadPool();
private final ConcurrentHashMap<String, String> containers = new ConcurrentHashMap<>();
@@ -66,7 +67,8 @@ public class SageMakerTrainingRunner implements ContainerTeardown, Resettable {
@Inject
public SageMakerTrainingRunner(ContainerBuilder containerBuilder, ContainerLifecycleManager lifecycleManager,
ContainerLogStreamer logStreamer, EmulatorConfig config,
- ContainerDetector containerDetector, S3Service s3Service, ObjectMapper mapper) {
+ ContainerDetector containerDetector, S3Service s3Service, ObjectMapper mapper,
+ SageMakerGpuResolver gpuResolver) {
this.containerBuilder = containerBuilder;
this.lifecycleManager = lifecycleManager;
this.logStreamer = logStreamer;
@@ -74,6 +76,7 @@ public class SageMakerTrainingRunner implements ContainerTeardown, Resettable {
this.containerDetector = containerDetector;
this.s3Service = s3Service;
this.mapper = mapper;
+ this.gpuResolver = gpuResolver;
}
public void runAsync(TrainingJobResource job, SageMakerService service) {
@@ -95,6 +98,10 @@ public class SageMakerTrainingRunner implements ContainerTeardown, Resettable {
.withEmbeddedDns()
.withLogRotation();
SageMakerEndpointManager.applyEntrypoint(builder, job.algorithmSpecification, "train");
+ // Throws when the requested instance type asks for hardware this host cannot
+ // stand in for, so the job fails with a reason rather than quietly training
+ // on CPU and producing an artifact that looks legitimate.
+ gpuResolver.applyTo(builder, instanceType(job), instanceCount(job));
ContainerSpec spec = builder.build();
containerId = lifecycleManager.create(spec);
containers.put(job.trainingJobName, containerId);
@@ -324,6 +331,16 @@ public class SageMakerTrainingRunner implements ContainerTeardown, Resettable {
}
}
+ private static String instanceType(TrainingJobResource job) {
+ return SageMakerEndpointManager.string(job.resourceConfig.get("InstanceType"));
+ }
+
+ /** Absent or unparseable means one, matching how a single-instance job is described. */
+ private static int instanceCount(TrainingJobResource job) {
+ Object value = job.resourceConfig.get("InstanceCount");
+ return value instanceof Number n ? n.intValue() : 1;
+ }
+
private Duration timeout(TrainingJobResource job) {
Object value = job.stoppingCondition.get("MaxRuntimeInSeconds");
if (value instanceof Number n) {
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
index ca95bf4c36..673803bd4c 100644
--- a/src/main/resources/application.yml
+++ b/src/main/resources/application.yml
@@ -41,12 +41,13 @@ quarkus:
add-all-charsets: true
resources:
# The Pricing service reads its bundled snapshot, EC2 reads its image
- # catalog (ec2/image-catalog.yaml), AppSync's GraphQL parser reads its
- # message bundles, and API Gateway's Velocity engine reads its default
- # properties at runtime via ClassLoader#getResourceAsStream. Quarkus's
- # static resource analysis cannot detect these dynamically-built paths,
- # so they are registered explicitly to ensure GraalVM includes every
- # file in the native image.
+ # catalog (ec2/image-catalog.yaml), SageMaker reads its instance type
+ # catalog (sagemaker/instance-type-catalog.yaml), AppSync's GraphQL
+ # parser reads its message bundles, and API Gateway's Velocity engine
+ # reads its default properties at runtime via
+ # ClassLoader#getResourceAsStream. Quarkus's static resource analysis
+ # cannot detect these dynamically-built paths, so they are registered
+ # explicitly to ensure GraalVM includes every file in the native image.
# The IAM managed-policy catalog (iam/*) is not listed on purpose.
# AwsManagedPolicies parses it while the image is built and the parsed
# catalog lives in the image heap, so the 3.5 MB of source files would
@@ -55,6 +56,7 @@ quarkus:
- pricing-snapshots/**
- ui/*.html
- ec2/*.yaml
+ - sagemaker/*.yaml
- i18n/*.properties
- org/apache/velocity/runtime/defaults/*.properties
# DriverManager finds the JDBC drivers through ServiceLoader, and the MySQL
@@ -623,6 +625,18 @@ floci:
sagemaker:
enabled: true
# docker-network: floci-network
+ gpu:
+ # Off by default: an existing deployment keeps launching CPU-only
+ # containers. How many GPUs an ml.* type has is shipped in
+ # sagemaker/instance-type-catalog.yaml; this is only about which of
+ # this machine's devices Floci may hand out.
+ enabled: false
+ # cdi | device-ids | count. Podman resolves only cdi, and accepts
+ # count while attaching no device, so cdi fails loudly instead.
+ mode: cdi
+ # Required for cdi and device-ids modes; unset allows no device.
+ # devices:
+ # - nvidia.com/gpu=GPU-<uuid>
batch:
enabled: true
runner-mode: immediate
diff --git a/src/main/resources/sagemaker/instance-type-catalog.yaml b/src/main/resources/sagemaker/instance-type-catalog.yaml
new file mode 100644
index 0000000000..44559eb12b
--- /dev/null
+++ b/src/main/resources/sagemaker/instance-type-catalog.yaml
@@ -0,0 +1,83 @@
+# Accelerator facts for SageMaker ml.* instance types.
+#
+# Kept separate from ec2/instance-type-catalog.yaml because ml.* is its own
+# namespace: these types are not EC2 instance types and must never surface in
+# DescribeInstanceTypes.
+#
+# gpuCount is a property of the AWS instance type, not of the machine Floci
+# runs on, which is why it is shipped data rather than configuration. What the
+# local host is willing to hand over is a separate operator decision under
+# floci.services.sagemaker.gpu.
+#
+# Like the EC2 catalog, this ships the sizes worth supporting rather than the
+# full AWS inventory. A GPU-family type that is missing here is refused rather
+# than guessed at, so adding a family means adding its rows.
+#
+# The counts below happen to be 1, 4 and 8 because that is what the g4dn, g5,
+# p3 and p4d families offer. That is not a rule to extrapolate from: other AWS
+# accelerated families carry exactly two (g3.8xlarge, g5g.16xlarge,
+# g7.12xlarge), and a size is not a multiplier either, since g4dn.16xlarge has
+# one GPU while the smaller g4dn.12xlarge has four. Hence a table.
+instanceTypes:
+ # CPU families. Listed so the resolver can tell "this type has no
+ # accelerator" apart from "this type is unknown to Floci".
+ - instanceType: ml.m5.large
+ gpuCount: 0
+ - instanceType: ml.m5.xlarge
+ gpuCount: 0
+ - instanceType: ml.m5.2xlarge
+ gpuCount: 0
+ - instanceType: ml.c5.large
+ gpuCount: 0
+ - instanceType: ml.c5.xlarge
+ gpuCount: 0
+ - instanceType: ml.c5.2xlarge
+ gpuCount: 0
+ - instanceType: ml.t3.medium
+ gpuCount: 0
+ - instanceType: ml.t3.large
+ gpuCount: 0
+
+ # G4dn: 1x NVIDIA T4, except the 12xlarge (4) and metal (8).
+ - instanceType: ml.g4dn.xlarge
+ gpuCount: 1
+ - instanceType: ml.g4dn.2xlarge
+ gpuCount: 1
+ - instanceType: ml.g4dn.4xlarge
+ gpuCount: 1
+ - instanceType: ml.g4dn.8xlarge
+ gpuCount: 1
+ - instanceType: ml.g4dn.12xlarge
+ gpuCount: 4
+ - instanceType: ml.g4dn.16xlarge
+ gpuCount: 1
+
+ # G5: 1x NVIDIA A10G, except the 12xlarge (4) and 48xlarge (8).
+ - instanceType: ml.g5.xlarge
+ gpuCount: 1
+ - instanceType: ml.g5.2xlarge
+ gpuCount: 1
+ - instanceType: ml.g5.4xlarge
+ gpuCount: 1
+ - instanceType: ml.g5.8xlarge
+ gpuCount: 1
+ - instanceType: ml.g5.12xlarge
+ gpuCount: 4
+ - instanceType: ml.g5.16xlarge
+ gpuCount: 1
+ - instanceType: ml.g5.24xlarge
+ gpuCount: 4
+ - instanceType: ml.g5.48xlarge
+ gpuCount: 8
+
+ # P3: NVIDIA V100.
+ - instanceType: ml.p3.2xlarge
+ gpuCount: 1
+ - instanceType: ml.p3.8xlarge
+ gpuCount: 4
+ - instanceType: ml.p3.16xlarge
+ gpuCount: 8
+
+ # P4d: 8x NVIDIA A100.
+ - instanceType: ml.p4d.24xlarge
+ gpuCount: 8
diff --git a/src/test/java/io/github/hectorvent/floci/config/NativeImageResourceIncludesTest.java b/src/test/java/io/github/hectorvent/floci/config/NativeImageResourceIncludesTest.java
new file mode 100644
index 0000000000..8be96e96a6
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/config/NativeImageResourceIncludesTest.java
@@ -0,0 +1,143 @@
+package io.github.hectorvent.floci.config;
+
+import org.junit.jupiter.api.Test;
+
+import java.io.IOException;
+import java.io.UncheckedIOException;
+import java.nio.file.FileSystems;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.nio.file.PathMatcher;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+import java.util.stream.Stream;
+
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Guards the GraalVM native-image build against a classpath resource that is read at runtime but
+ * never packaged.
+ *
+ * <p>Quarkus's static analysis finds an inline literal such as
+ * {@code getResourceAsStream("/redshift/bootstrap-catalog.sql")}, so those need no registration.
+ * It cannot follow a constant, so {@code getResourceAsStream(CATALOG_RESOURCE_NAME)} leaves the
+ * file out of the image and the first read fails at runtime with a missing-resource error. The
+ * remedy is a matching glob under {@code quarkus.native.resources.includes} in
+ * {@code application.yml}.
+ *
+ * <p>Like {@link NativeImageRuntimeInitializationTest}, this failure only surfaces in the slow
+ * native build and never in the JVM CI jobs, so it moves the check into ordinary CI. Regression
+ * context: the SageMaker instance type catalog shipped reading
+ * {@code sagemaker/instance-type-catalog.yaml} through a constant without the matching include,
+ * which would have failed every GPU training job in the native image while passing every JVM test.
+ */
+class NativeImageResourceIncludesTest {
+
+ private static final Path MAIN_SOURCE_ROOT = Path.of("src", "main", "java");
+ private static final Path APPLICATION_YML = Path.of("src", "main", "resources", "application.yml");
+ private static final Path MAIN_RESOURCE_ROOT = Path.of("src", "main", "resources");
+
+ private static final String JAVA_SUFFIX = ".java";
+
+ /** A {@code static final String NAME = "some/path.ext";} declaration holding a resource path. */
+ private static final Pattern RESOURCE_CONSTANT = Pattern.compile(
+ "static\\s+final\\s+String\\s+(\\w+)\\s*=\\s*\"(/?[\\w.\\-]+(?:/[\\w.\\-]+)+\\.[A-Za-z0-9]+)\"");
+
+ /**
+ * Read while the image is built, not at runtime. {@code AwsManagedPolicies} parses the catalog
+ * during the build and keeps the parsed form in the image heap, so including the 3.5 MB of
+ * source files would only be a second copy. {@code application.yml} says the same.
+ */
+ private static final Set<String> BUILD_TIME_ONLY = Set.of(
+ "iam/managed-policies.yaml",
+ "iam/managed-policy-documents.json",
+ "iam/managed-policy-versions.json");
+
+ @Test
+ void everyResourceReadThroughAConstantIsIncludedInTheNativeImage() throws IOException {
+ List<PathMatcher> includes = parseIncludes();
+ Map<String, String> offenders = new LinkedHashMap<>();
+
+ try (Stream<Path> sources = Files.walk(MAIN_SOURCE_ROOT)) {
+ sources.filter(p -> p.toString().endsWith(JAVA_SUFFIX)).forEach(source -> {
+ for (String resource : resourcesReadThroughAConstant(source)) {
+ if (BUILD_TIME_ONLY.contains(resource)) {
+ continue;
+ }
+ // Only a file this repository actually ships can be registered. A constant
+ // naming something from a dependency is that dependency's problem.
+ if (!Files.exists(MAIN_RESOURCE_ROOT.resolve(resource))) {
+ continue;
+ }
+ if (includes.stream().noneMatch(m -> m.matches(Path.of(resource)))) {
+ offenders.put(resource, source.toString());
+ }
+ }
+ });
+ }
+
+ assertTrue(offenders.isEmpty(),
+ "These resources are read at runtime through a constant, which Quarkus's static analysis "
+ + "cannot follow, but no glob in " + APPLICATION_YML + " under "
+ + "quarkus.native.resources.includes matches them. The native image will not contain "
+ + "them and the first read will fail at runtime. Add an include for each:\n "
+ + offenders.entrySet().stream()
+ .map(e -> e.getKey() + " (read by " + e.getValue() + ")")
+ .reduce((a, b) -> a + "\n " + b).orElse(""));
+ }
+
+ /** Resource paths declared as a constant in {@code javaFile} and passed to getResourceAsStream there. */
+ private List<String> resourcesReadThroughAConstant(Path javaFile) {
+ String source;
+ try {
+ source = Files.readString(javaFile);
+ } catch (IOException e) {
+ throw new UncheckedIOException("Unable to read source file " + javaFile, e);
+ }
+ List<String> found = new ArrayList<>();
+ if (!source.contains("getResourceAsStream")) {
+ return found;
+ }
+ Matcher constants = RESOURCE_CONSTANT.matcher(source);
+ while (constants.find()) {
+ String name = constants.group(1);
+ String value = constants.group(2);
+ if (source.contains("getResourceAsStream(" + name + ")")
+ || source.contains("getResourceAsStream(\n")
+ && source.contains(name + ")")) {
+ found.add(value.startsWith("/") ? value.substring(1) : value);
+ }
+ }
+ return found;
+ }
+
+ /** The {@code quarkus.native.resources.includes} globs, as matchers over classpath-relative paths. */
+ private List<PathMatcher> parseIncludes() throws IOException {
+ List<String> lines = Files.readAllLines(APPLICATION_YML);
+ List<PathMatcher> matchers = new ArrayList<>();
+ boolean inIncludes = false;
+ for (String raw : lines) {
+ String line = raw.strip();
+ if (line.startsWith("includes:")) {
+ inIncludes = true;
+ continue;
+ }
+ if (inIncludes) {
+ if (!line.startsWith("- ")) {
+ if (!line.isEmpty() && !line.startsWith("#")) {
+ break;
+ }
+ continue;
+ }
+ matchers.add(FileSystems.getDefault().getPathMatcher("glob:" + line.substring(2).strip()));
+ }
+ }
+ assertTrue(!matchers.isEmpty(), "Found no quarkus.native.resources.includes entries in " + APPLICATION_YML);
+ return matchers;
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuDockerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuDockerIntegrationTest.java
new file mode 100644
index 0000000000..95ebfbedff
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuDockerIntegrationTest.java
@@ -0,0 +1,181 @@
+package io.github.hectorvent.floci.services.sagemaker;
+
+import com.github.dockerjava.api.DockerClient;
+import io.github.hectorvent.floci.services.s3.S3Service;
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.quarkus.test.junit.QuarkusTest;
+import io.quarkus.test.junit.QuarkusTestProfile;
+import io.quarkus.test.junit.TestProfile;
+import io.restassured.response.Response;
+import jakarta.inject.Inject;
+import org.jboss.logging.Logger;
+import org.junit.jupiter.api.Assumptions;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.time.Duration;
+import java.util.HashMap;
+import java.util.Map;
+
+import static io.restassured.RestAssured.given;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Proves an {@code ml.*} GPU instance type reaches a real accelerator, end to end through
+ * {@code CreateTrainingJob} rather than through the container layer directly.
+ *
+ * <p>Opt-in for the same reason as {@link
+ * io.github.hectorvent.floci.core.common.docker.ContainerGpuDockerIntegrationTest}: it needs
+ * hardware no CI runner has. Run it with {@code -Dgpu.integration.enabled=true} on a host with a
+ * GPU, naming the device the same way the feature's own configuration does:
+ *
+ * <pre>
+ * ./mvnw test -Dtest=SageMakerGpuDockerIntegrationTest \
+ * -Dgpu.integration.enabled=true \
+ * -Dgpu.integration.cdi-device=nvidia.com/gpu=GPU-&lt;uuid&gt;
+ * </pre>
+ *
+ * <p>Without the flag it skips, leaving {@link SageMakerGpuResolverTest} and {@link
+ * SageMakerGpuIntegrationTest} as the coverage that runs by default. Those pin every path that
+ * refuses a request; this pins the one that honours it.
+ *
+ * <p>The assertion is the container's exit code, not a log line. The image runs {@code nvidia-smi},
+ * which the runtime injects alongside the device, so it is absent when no device was attached: a
+ * job that quietly fell back to CPU exits nonzero and reports Failed. Naming a device also makes
+ * the check exact, since the daemon must hand over that card rather than any card.
+ *
+ * <p>With no device named the test asks for every GPU using the count form, which is the shape to
+ * run against Docker. Podman accepts that form and attaches nothing
+ * (containers/podman#22645), so that combination fails here by design, which is the same reason
+ * {@code mode} defaults to {@code cdi}.
+ */
+@QuarkusTest
+@TestProfile(SageMakerGpuDockerIntegrationTest.GpuHardwareProfile.class)
+class SageMakerGpuDockerIntegrationTest {
+
+ private static final Logger LOG = Logger.getLogger(SageMakerGpuDockerIntegrationTest.class);
+
+ private static final String ENABLED_PROPERTY = "gpu.integration.enabled";
+ private static final String CDI_DEVICE_PROPERTY = "gpu.integration.cdi-device";
+ private static final String IMAGE_PROPERTY = "gpu.integration.image";
+ private static final String DEFAULT_IMAGE = "docker.io/nvidia/cuda:12.8.1-base-ubuntu24.04";
+
+ /**
+ * Mirrors the deployment-side configuration the feature documents. A named device selects
+ * {@code cdi} mode and the allowlist; with none, {@code count} lets the daemon choose.
+ */
+ public static class GpuHardwareProfile implements QuarkusTestProfile {
+ @Override
+ public Map<String, String> getConfigOverrides() {
+ Map<String, String> overrides = new HashMap<>();
+ overrides.put("floci.services.sagemaker.gpu.enabled", "true");
+ String device = System.getProperty(CDI_DEVICE_PROPERTY);
+ if (device != null && !device.isBlank()) {
+ overrides.put("floci.services.sagemaker.gpu.mode", "cdi");
+ overrides.put("floci.services.sagemaker.gpu.devices", device);
+ } else {
+ overrides.put("floci.services.sagemaker.gpu.mode", "count");
+ }
+ return overrides;
+ }
+ }
+
+ private static final String AUTH =
+ "AWS4-HMAC-SHA256 Credential=test/20260205/us-east-1/sagemaker/aws4_request";
+
+ @Inject
+ DockerClient dockerClient;
+
+ @Inject
+ S3Service s3Service;
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @BeforeEach
+ void requireGpuEnabledDaemon() {
+ Assumptions.assumeTrue(Boolean.getBoolean(ENABLED_PROPERTY),
+ "GPU hardware tests are opt-in: run with -D" + ENABLED_PROPERTY + "=true on a host with a GPU");
+ Assumptions.assumeTrue(isDockerAvailable(),
+ "A container daemon must be available for GPU integration tests");
+ }
+
+ @Test
+ void gpuInstanceTypeTrainsOnTheRequestedDevice() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String bucket = "sm-gpu-" + suffix;
+ s3Service.createBucket(bucket, "us-east-1");
+ String job = "sm-gpu-" + suffix;
+ String image = System.getProperty(IMAGE_PROPERTY, DEFAULT_IMAGE);
+
+ // nvidia-smi is injected with the device, so its absence is the clearest signal that
+ // nothing was attached. Writing the UUID into /opt/ml/model makes Floci upload the
+ // evidence rather than leaving it in a log.
+ //
+ // Redirect rather than pipe: a pipeline exits with the status of its last command, so
+ // `nvidia-smi -L | tee` reports success even when nvidia-smi could not reach the driver.
+ // That would be this test passing on a CPU container, which is the exact failure it exists
+ // to catch.
+ post("SageMaker.CreateTrainingJob", """
+ {
+ "TrainingJobName":"%s",
+ "AlgorithmSpecification":{
+ "TrainingImage":"%s",
+ "ContainerEntrypoint":["/bin/sh","-c"],
+ "ContainerArguments":["mkdir -p /opt/ml/model && nvidia-smi -L > /opt/ml/model/devices.txt && cat /opt/ml/model/devices.txt"]
+ },
+ "OutputDataConfig":{"S3OutputPath":"s3://%s/output"},
+ "ResourceConfig":{"InstanceType":"ml.g5.xlarge","InstanceCount":1,"VolumeSizeInGB":1},
+ "StoppingCondition":{"MaxRuntimeInSeconds":300}
+ }
+ """.formatted(job, image, bucket)).then().statusCode(200);
+
+ Response describe = awaitTerminal(job);
+ String status = describe.path("TrainingJobStatus");
+ String reason = describe.path("FailureReason");
+ assertEquals("Completed", status,
+ "ml.g5.xlarge did not reach a GPU: " + (reason == null ? "no FailureReason" : reason));
+
+ String artifact = describe.path("ModelArtifacts.S3ModelArtifacts");
+ assertTrue(artifact != null && artifact.endsWith("model.tar.gz"),
+ "no model artifact was uploaded: " + artifact);
+ LOG.infov("GPU training job completed, artifact at {0}", artifact);
+ }
+
+ private Response awaitTerminal(String job) throws Exception {
+ long deadline = System.currentTimeMillis() + Duration.ofSeconds(300).toMillis();
+ while (System.currentTimeMillis() < deadline) {
+ Response describe = post("SageMaker.DescribeTrainingJob",
+ "{\"TrainingJobName\":\"%s\"}".formatted(job));
+ describe.then().statusCode(200);
+ String status = describe.path("TrainingJobStatus");
+ if ("Failed".equals(status) || "Completed".equals(status) || "Stopped".equals(status)) {
+ return describe;
+ }
+ Thread.sleep(1000);
+ }
+ throw new AssertionError("Timed out waiting for training job " + job);
+ }
+
+ private Response post(String target, String body) {
+ return given().header("Authorization", AUTH)
+ .header("X-Amz-Target", target)
+ .contentType("application/x-amz-json-1.1")
+ .body(body)
+ .when().post("/");
+ }
+
+ private boolean isDockerAvailable() {
+ try {
+ dockerClient.pingCmd().exec();
+ return true;
+ } catch (Exception e) {
+ LOG.warn("No container daemon available for the SageMaker GPU integration test", e);
+ return false;
+ }
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuIntegrationTest.java
new file mode 100644
index 0000000000..4ac0719ba7
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuIntegrationTest.java
@@ -0,0 +1,140 @@
+package io.github.hectorvent.floci.services.sagemaker;
+
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.quarkus.test.junit.QuarkusTest;
+import io.quarkus.test.junit.TestProfile;
+import io.quarkus.test.junit.QuarkusTestProfile;
+import io.restassured.response.Response;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import java.time.Duration;
+import java.util.Map;
+
+import static io.restassured.RestAssured.given;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Covers the GPU refusal paths, which need no GPU and no container daemon.
+ *
+ * <p>{@link SageMakerGpuResolver} raises before the runner asks the daemon for anything, and
+ * {@code removeIfExists} on the line above it swallows a missing daemon, so every case here
+ * reaches its verdict on a plain CI runner. The hardware path is the opposite and lives in
+ * {@link SageMakerGpuDockerIntegrationTest}, which skips without a GPU.
+ *
+ * <p>What is being pinned is the contract that a request Floci cannot honour fails with a
+ * reason naming the fix, rather than falling back to CPU. A job that reports {@code Completed}
+ * after training on the wrong hardware leaves an artifact that looks legitimate.
+ *
+ * <p>The profile turns the feature on because {@code application.yml} ships it off. Devices are
+ * deliberately left unset, which is what makes the third case reachable.
+ */
+@QuarkusTest
+@TestProfile(SageMakerGpuIntegrationTest.GpuEnabledProfile.class)
+class SageMakerGpuIntegrationTest {
+
+ public static class GpuEnabledProfile implements QuarkusTestProfile {
+ @Override
+ public Map<String, String> getConfigOverrides() {
+ return Map.of("floci.services.sagemaker.gpu.enabled", "true");
+ }
+ }
+
+ private static final String AUTH =
+ "AWS4-HMAC-SHA256 Credential=test/20260205/us-east-1/sagemaker/aws4_request";
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @Test
+ void gpuFamilyMissingFromTheCatalogFailsNamingTheCatalog() throws Exception {
+ String job = submit("ml.p5.48xlarge", 1);
+ String reason = awaitFailureReason(job);
+
+ assertTrue(reason.contains("ml.p5.48xlarge"), reason);
+ assertTrue(reason.contains("catalog"), reason);
+ }
+
+ @Test
+ void instanceCountAboveOneFailsRatherThanTrainingOnOneContainer() throws Exception {
+ String job = submit("ml.g5.xlarge", 2);
+ String reason = awaitFailureReason(job);
+
+ assertTrue(reason.contains("InstanceCount 2"), reason);
+ assertTrue(reason.contains("ml.g5.xlarge"), reason);
+ }
+
+ @Test
+ void cdiModeWithNoConfiguredDevicesFailsRatherThanTakingEveryGpu() throws Exception {
+ String job = submit("ml.g5.xlarge", 1);
+ String reason = awaitFailureReason(job);
+
+ assertTrue(reason.contains("floci.services.sagemaker.gpu.devices"), reason);
+ }
+
+ /**
+ * Enabling the feature must not change what a CPU instance type does. The job's outcome
+ * depends on whether a daemon is present, so only the reason is asserted: whatever happens,
+ * it must not be the resolver refusing the request.
+ */
+ @Test
+ void cpuInstanceTypeIsNotRefusedWhenGpuSupportIsEnabled() throws Exception {
+ String job = submit("ml.m5.large", 1);
+ Response describe = awaitTerminal(job);
+ String reason = describe.path("FailureReason") == null ? "" : describe.path("FailureReason");
+
+ assertFalse(reason.contains("floci.services.sagemaker.gpu"), reason);
+ assertFalse(reason.contains("catalog"), reason);
+ }
+
+ private String submit(String instanceType, int instanceCount) {
+ String job = "sm-gpu-" + Long.toString(System.nanoTime(), 36);
+ post("SageMaker.CreateTrainingJob", """
+ {
+ "TrainingJobName":"%s",
+ "AlgorithmSpecification":{"TrainingImage":"busybox:stable","TrainingInputMode":"File"},
+ "OutputDataConfig":{"S3OutputPath":"s3://sm-gpu-out/output"},
+ "ResourceConfig":{"InstanceType":"%s","InstanceCount":%d,"VolumeSizeInGB":1},
+ "StoppingCondition":{"MaxRuntimeInSeconds":60}
+ }
+ """.formatted(job, instanceType, instanceCount))
+ .then().statusCode(200);
+ return job;
+ }
+
+ private String awaitFailureReason(String job) throws Exception {
+ Response describe = awaitTerminal(job);
+ assertEquals("Failed", describe.path("TrainingJobStatus"),
+ "expected the request to be refused, not honoured");
+ String reason = describe.path("FailureReason");
+ assertTrue(reason != null && !reason.isBlank(), "Failed with no FailureReason");
+ return reason;
+ }
+
+ private Response awaitTerminal(String job) throws Exception {
+ long deadline = System.currentTimeMillis() + Duration.ofSeconds(60).toMillis();
+ while (System.currentTimeMillis() < deadline) {
+ Response describe = post("SageMaker.DescribeTrainingJob",
+ "{\"TrainingJobName\":\"%s\"}".formatted(job));
+ describe.then().statusCode(200);
+ String status = describe.path("TrainingJobStatus");
+ if ("Failed".equals(status) || "Completed".equals(status) || "Stopped".equals(status)) {
+ return describe;
+ }
+ Thread.sleep(200);
+ }
+ throw new AssertionError("Timed out waiting for training job " + job + " to reach a terminal state");
+ }
+
+ private Response post(String target, String body) {
+ return given().header("Authorization", AUTH)
+ .header("X-Amz-Target", target)
+ .contentType("application/x-amz-json-1.1")
+ .body(body)
+ .when().post("/");
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuResolverTest.java b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuResolverTest.java
new file mode 100644
index 0000000000..1016944521
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerGpuResolverTest.java
@@ -0,0 +1,187 @@
+package io.github.hectorvent.floci.services.sagemaker;
+
+import io.github.hectorvent.floci.config.EmulatorConfig;
+import io.github.hectorvent.floci.config.EmulatorConfig.SageMakerServiceConfig.GpuRequestMode;
+import io.github.hectorvent.floci.core.common.docker.ContainerBuilder;
+import io.github.hectorvent.floci.core.common.dns.EmbeddedDnsServer;
+import io.github.hectorvent.floci.core.common.docker.ContainerSpec;
+import io.github.hectorvent.floci.core.common.docker.CurrentContainerNetworkResolver;
+import io.github.hectorvent.floci.core.common.docker.DockerHostResolver;
+import com.github.dockerjava.api.model.DeviceRequest;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+import java.util.Optional;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.when;
+
+class SageMakerGpuResolverTest {
+
+ private static final String UUID_A = "nvidia.com/gpu=GPU-aaaaaaaa";
+ private static final String UUID_B = "nvidia.com/gpu=GPU-bbbbbbbb";
+ private static final String UUID_C = "nvidia.com/gpu=GPU-cccccccc";
+ private static final String UUID_D = "nvidia.com/gpu=GPU-dddddddd";
+
+ @Test
+ void disabledByDefaultLeavesEvenGpuInstanceTypesAlone() {
+ Fixture fixture = new Fixture(false, GpuRequestMode.CDI, List.of(UUID_A));
+
+ ContainerSpec spec = fixture.resolve("ml.g5.xlarge", 1);
+
+ // The guard on every existing deployment: opting out means nothing changes,
+ // including for instance types that would otherwise ask for hardware.
+ assertEquals(List.of(), spec.deviceRequests());
+ }
+
+ @Test
+ void cpuInstanceTypeGetsNoDevice() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of(UUID_A));
+
+ assertEquals(List.of(), fixture.resolve("ml.m5.large", 1).deviceRequests());
+ }
+
+ @Test
+ void unknownNonAcceleratorTypeIsTreatedAsCpu() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of(UUID_A));
+
+ // Not in the catalog and not a GPU family: a CPU job Floci simply does not
+ // have metadata for, which is no reason to fail it.
+ assertEquals(List.of(), fixture.resolve("ml.c7i.12xlarge", 1).deviceRequests());
+ }
+
+ @Test
+ void gpuInstanceTypeRequestsItsCataloguedDeviceCount() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of(UUID_A, UUID_B));
+
+ ContainerSpec spec = fixture.resolve("ml.g5.xlarge", 1);
+
+ DeviceRequest request = spec.deviceRequests().get(0);
+ assertEquals("cdi", request.getDriver());
+ assertEquals(List.of(UUID_A), request.getDeviceIds());
+ }
+
+ @Test
+ void multiGpuTypeTakesAsManyDevicesAsTheCatalogSays() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI,
+ List.of(UUID_A, UUID_B, UUID_C, UUID_D));
+
+ // ml.g5.12xlarge is four devices on AWS, so a host offering four gets all four
+ // in one container. Counts come from the catalog rather than from matching
+ // ml.g*, because they vary within a family.
+ ContainerSpec spec = fixture.resolve("ml.g5.12xlarge", 1);
+
+ assertEquals(List.of(UUID_A, UUID_B, UUID_C, UUID_D),
+ spec.deviceRequests().get(0).getDeviceIds());
+ }
+
+ @Test
+ void devicesAreTakenInOrderSoASmallerTypeDoesNotClaimEverything() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI,
+ List.of(UUID_A, UUID_B, UUID_C, UUID_D));
+
+ // A one-GPU type on a four-GPU host takes one device, not the whole allowlist.
+ assertEquals(List.of(UUID_A), fixture.resolve("ml.g5.xlarge", 1)
+ .deviceRequests().get(0).getDeviceIds());
+ }
+
+ @Test
+ void multiGpuTypeIsRefusedWhenTheHostAllowsFewerDevices() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of(UUID_A, UUID_B));
+
+ SageMakerGpuResolver.UnsupportedResourceException thrown = assertThrows(
+ SageMakerGpuResolver.UnsupportedResourceException.class,
+ () -> fixture.resolve("ml.g5.12xlarge", 1));
+ assertTrue(thrown.getMessage().contains("needs 4 GPUs"), thrown.getMessage());
+ }
+
+ @Test
+ void deviceIdsModeUsesTheDaemonIdForm() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.DEVICE_IDS, List.of("0", "1"));
+
+ DeviceRequest request = fixture.resolve("ml.g5.xlarge", 1).deviceRequests().get(0);
+
+ assertEquals(List.of("0"), request.getDeviceIds());
+ assertEquals(List.of(List.of("gpu")), request.getCapabilities());
+ }
+
+ @Test
+ void countModeLetsTheDaemonChooseAndIgnoresTheAllowlist() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.COUNT, List.of());
+
+ DeviceRequest request = fixture.resolve("ml.g5.xlarge", 1).deviceRequests().get(0);
+
+ assertEquals(1, request.getCount());
+ }
+
+ @Test
+ void acceleratorTypeMissingFromTheCatalogIsRefused() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of(UUID_A));
+
+ // The case this whole class exists to prevent: the caller asked for a GPU, so
+ // running on CPU and reporting Completed would hand back an artifact that looks
+ // legitimate.
+ SageMakerGpuResolver.UnsupportedResourceException thrown = assertThrows(
+ SageMakerGpuResolver.UnsupportedResourceException.class,
+ () -> fixture.resolve("ml.p5.48xlarge", 1));
+ assertTrue(thrown.getMessage().contains("not in Floci's"), thrown.getMessage());
+ }
+
+ @Test
+ void multipleInstancesAreRefusedRatherThanPretended() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of(UUID_A, UUID_B));
+
+ SageMakerGpuResolver.UnsupportedResourceException thrown = assertThrows(
+ SageMakerGpuResolver.UnsupportedResourceException.class,
+ () -> fixture.resolve("ml.g5.xlarge", 2));
+ assertTrue(thrown.getMessage().contains("InstanceCount 2"), thrown.getMessage());
+ }
+
+ @Test
+ void explicitModeWithNoDevicesConfiguredIsRefused() {
+ Fixture fixture = new Fixture(true, GpuRequestMode.CDI, List.of());
+
+ SageMakerGpuResolver.UnsupportedResourceException thrown = assertThrows(
+ SageMakerGpuResolver.UnsupportedResourceException.class,
+ () -> fixture.resolve("ml.g5.xlarge", 1));
+ assertTrue(thrown.getMessage().contains("devices is empty"), thrown.getMessage());
+ }
+
+ /** Wires a real ContainerBuilder so the assertions see the request actually built. */
+ private static final class Fixture {
+ private final SageMakerGpuResolver resolver;
+ private final ContainerBuilder containerBuilder;
+
+ private Fixture(boolean enabled, GpuRequestMode mode, List<String> devices) {
+ EmulatorConfig config = mock(EmulatorConfig.class);
+ EmulatorConfig.ServicesConfig services = mock(EmulatorConfig.ServicesConfig.class);
+ EmulatorConfig.SageMakerServiceConfig sagemaker =
+ mock(EmulatorConfig.SageMakerServiceConfig.class);
+ EmulatorConfig.SageMakerServiceConfig.GpuConfig gpu =
+ mock(EmulatorConfig.SageMakerServiceConfig.GpuConfig.class);
+ EmulatorConfig.DockerConfig docker = mock(EmulatorConfig.DockerConfig.class);
+
+ when(config.services()).thenReturn(services);
+ when(services.sagemaker()).thenReturn(sagemaker);
+ when(sagemaker.gpu()).thenReturn(gpu);
+ when(gpu.enabled()).thenReturn(enabled);
+ when(gpu.mode()).thenReturn(mode);
+ when(gpu.devices()).thenReturn(Optional.of(devices));
+ when(config.docker()).thenReturn(docker);
+ when(docker.imageRegistryBase()).thenReturn(Optional.empty());
+
+ this.containerBuilder = new ContainerBuilder(config, mock(DockerHostResolver.class),
+ mock(EmbeddedDnsServer.class), mock(CurrentContainerNetworkResolver.class));
+ this.resolver = new SageMakerGpuResolver(config, new SageMakerInstanceTypeCatalog());
+ }
+
+ private ContainerSpec resolve(String instanceType, int instanceCount) {
+ ContainerBuilder.Builder builder = containerBuilder.newContainer("alpine");
+ resolver.applyTo(builder, instanceType, instanceCount);
+ return builder.build();
+ }
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunnerTest.java b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunnerTest.java
index 16da79984d..70136b4777 100644
--- a/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunnerTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/sagemaker/SageMakerTrainingRunnerTest.java
@@ -74,6 +74,7 @@ class SageMakerTrainingRunnerTest {
private static SageMakerTrainingRunner runner() {
return new SageMakerTrainingRunner(mock(ContainerBuilder.class), mock(ContainerLifecycleManager.class),
mock(ContainerLogStreamer.class), mock(EmulatorConfig.class, RETURNS_DEEP_STUBS),
- mock(ContainerDetector.class), mock(S3Service.class), new ObjectMapper());
+ mock(ContainerDetector.class), mock(S3Service.class), new ObjectMapper(),
+ mock(SageMakerGpuResolver.class));
}
}
diff --git a/src/test/resources/application.yml b/src/test/resources/application.yml
index df1e01a654..53bbd06470 100644
--- a/src/test/resources/application.yml
+++ b/src/test/resources/application.yml
@@ -332,6 +332,18 @@ floci:
sagemaker:
enabled: true
# docker-network: floci-network
+ gpu:
+ # Off by default: an existing deployment keeps launching CPU-only
+ # containers. How many GPUs an ml.* type has is shipped in
+ # sagemaker/instance-type-catalog.yaml; this is only about which of
+ # this machine's devices Floci may hand out.
+ enabled: false
+ # cdi | device-ids | count. Podman resolves only cdi, and accepts
+ # count while attaching no device, so cdi fails loudly instead.
+ mode: cdi
+ # Required for cdi and device-ids modes; unset allows no device.
+ # devices:
+ # - nvidia.com/gpu=GPU-<uuid>
batch:
enabled: true
runner-mode: immediate