Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

fix(ec2): require IMDSv2 tokens when HttpTokens is required (#5966)

Jackson Price7504d71e7674
diff --git a/docs/services/ec2.md b/docs/services/ec2.md
index 6608d8897b..d32a7244d4 100644
--- a/docs/services/ec2.md
+++ b/docs/services/ec2.md
@@ -86,6 +86,8 @@ If `KeyName` is specified at launch, Floci looks up the stored key pair's public
Key pairs created with `CreateKeyPair` return real RSA private key material, and the matching public key is injected at launch, so either a created or an imported key pair works for SSH.
+When the image lacks `sshd` or `scp`, Floci first installs OpenSSH with the image's package manager, from a shell that clears `LD_LIBRARY_PATH`, `PYTHONPATH` and `PYTHONHOME` as the IMDS proxy's helper shells do.
+
## Security Group Port Publishing
With `FLOCI_NETWORK_SECURITY_GROUP_ENFORCEMENT_ENABLED=true`, Floci prepares a separate Linux network namespace and an nftables default-deny policy before starting each Docker-backed instance. It checks new managed connections against both the sender's egress rules and the receiver's ingress rules, using emulated ENI addresses and security-group membership before translating to Docker addresses. Established replies use connection tracking. This filters packets but does not emulate VPC routing, NACLs, NAT gateways, or peering. Rootless Docker and Windows containers are unsupported.
@@ -118,6 +120,16 @@ CloudFormation `AWS::EC2::VPCEndpoint` resolves `IpAddressType` on create and up
Floci runs an IMDS-compatible HTTP server on port `9169` of the host. Each launched container receives the environment variable `AWS_EC2_METADATA_SERVICE_ENDPOINT` pointing to this server.
+Inside each launched container, the link-local address `169.254.169.254` is a proxy to this
+server. The helper shells that install and start the proxy clear `LD_LIBRARY_PATH`,
+`PYTHONPATH` and `PYTHONHOME` before running system tools, so an application's library search
+path and Python installation do not reach the package manager, `socat` and `curl`. The
+container's configured environment and the application's processes keep these variables. The
+proxy is ready once an IMDSv2 token request with a 60-second TTL succeeds and that token reads the
+instance ID. The check bypasses any HTTP proxy the image configures, a failed or empty token
+response does not fall back to a tokenless request, and the check passes on an instance that
+requires tokens.
+
`iam/info` returns the ARN and ID that IAM holds for the attached profile, and 404 when that
profile is missing, has no role or does not match the attached ARN. An EKS cluster node, whose
profile Floci synthesizes, reports the synthesized ARN and a stable ID instead while IAM holds no
@@ -135,7 +147,7 @@ curl -s -H "x-aws-ec2-metadata-token: $TOKEN" \
http://169.254.169.254/latest/meta-data/instance-id
```
-As on AWS, the token TTL must be an integer from 1 to 21600 seconds, or the `PUT` returns `400`. A metadata request that presents an unknown or expired token returns `401`, which tells the SDK to fetch a new token. A token is valid only on the instance that requested it: presenting it from another instance returns `401`. A request without a token header still uses IMDSv1; Floci does not enforce `HttpTokens=required`.
+As on AWS, the token TTL must be an integer from 1 to 21600 seconds, or the `PUT` returns `400`. A metadata request that presents an unknown or expired token returns `401`, which tells the SDK to fetch a new token. A token is valid only on the instance that requested it: presenting it from another instance returns `401`. A request without a token header uses IMDSv1 unless the instance's `HttpTokens` is `required`; then, as on AWS, a `GET` or `HEAD` returns `401` and no metadata, for every path. `HttpTokens` comes from `RunInstances` (directly or through its launch template), from an EC2 Fleet's launch template, from Auto Scaling (a launch configuration's `MetadataOptions`, or a launch template given directly or through a mixed instances policy), or later from `ModifyInstanceMetadataOptions`. EKS cluster nodes always accept IMDSv1: `ModifyInstanceMetadataOptions` refuses them with `OperationNotPermitted`, as their stop, start, reboot and terminate actions are refused.
User data is returned as the original base64-decoded bytes, without decompressing gzip content or converting binary data to text. Older records without the original encoding, or records with malformed saved encoding, use their saved UTF-8 text. The response uses `application/octet-stream`, matching the [AWS user-data retrieval contract](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instancedata-data-retrieval.html). Missing or empty data returns `404`; whitespace is preserved. Script execution retains its separate bounded gzip decoding.
@@ -185,7 +197,7 @@ When an EKS cluster is running, its synthesized node instance is registered and
- **Visibility**: Exposed via `DescribeInstances` and `DescribeInstanceStatus` with its synthesized instance ID (`i-...`), type (`m5.large`), running state, VPC, subnet, launch time, and cluster tags (`Name`, `kubernetes.io/cluster/<cluster-name>=owned`, `eks:cluster-name=<cluster-name>`). It can be queried by ID, filtered, or listed with other instances.
- **Operations**: Supports volume attachments (`AttachVolume`, `DetachVolume`), instance attribute inspection and modification, and resource tagging (`CreateTags`, `DeleteTags`, `DescribeTags`).
-- **Lifecycle protection**: Lifecycle actions that would mutate or delete the node through EC2 (`TerminateInstances`, `StopInstances`, `StartInstances`, `RebootInstances`) are rejected with `OperationNotPermitted` (HTTP 400) as a Floci emulation limitation because the instance directly represents the underlying k3s cluster container. In real AWS, cluster node instances in an Auto Scaling group can be terminated or stopped and are subsequently replaced by the ASG. In Floci, the node instance lifecycle is managed solely through the EKS cluster lifecycle.
+- **Lifecycle protection**: Lifecycle actions that would mutate or delete the node through EC2 (`TerminateInstances`, `StopInstances`, `StartInstances`, `RebootInstances`), and `ModifyInstanceMetadataOptions`, which the node's IMDS would not see, are rejected with `OperationNotPermitted` (HTTP 400) as a Floci emulation limitation because the instance directly represents the underlying k3s cluster container. In real AWS, cluster node instances in an Auto Scaling group can be terminated or stopped and are subsequently replaced by the ASG. In Floci, the node instance lifecycle is managed solely through the EKS cluster lifecycle.
## Default Resources
@@ -219,7 +231,7 @@ Floci seeds the following resources on first use in each region so Terraform, th
| DescribeInstanceCreditSpecifications | Returns the CPU credit option of burstable performance instances. A named instance id reports the option the instance acquired at launch, either the explicit `CreditSpecification.CpuCredits` or the family default, `standard` for t2 and `unlimited` for t3, t3a and t4g. An id that is not a burstable performance instance reports `standard`, and only an unknown id is an error. Naming no id returns the instances on the unlimited option, including one that kept `unlimited` after a resize onto a non-burstable type. `Filter.N` supports the modeled `instance-id` and narrows whichever set the request selected. `MaxResults` accepts 5 through 1000 and cannot be combined with instance ids. `DryRun=true` returns `DryRunOperation`. |
| DescribeInstanceAttribute | Returns a supported attribute for an instance. |
| ModifyInstanceAttribute | Updates supported mutable attributes for an instance. `UserData.Value` (base64) needs the instance stopped, as on AWS, and answers `IncorrectInstanceState` otherwise. |
-| ModifyInstanceMetadataOptions | Updates an instance's IMDS options, changing only the fields the request names. |
+| ModifyInstanceMetadataOptions | Updates an instance's IMDS options, changing only the fields the request names. An EKS cluster node is refused with `OperationNotPermitted`. |
### VPCs
diff --git a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManager.java b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManager.java
index 31e2a1f6e0..e1ec374f61 100644
--- a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManager.java
+++ b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManager.java
@@ -1600,6 +1600,9 @@ public class Ec2ContainerManager {
*/
static String[] sshdInstallProbeCommand() {
return new String[]{"sh", "-c",
+ // dnf and yum are Python programs: an image's PYTHONHOME or LD_LIBRARY_PATH made them
+ // fail to start, so sshd was never installed.
+ Ec2MetadataProxy.CLEAR_APPLICATION_PATHS + ";" +
"if ! command -v sshd >/dev/null 2>&1 || ! command -v scp >/dev/null 2>&1; then" +
" if command -v dnf >/dev/null 2>&1; then dnf install -y openssh-server openssh-clients >/dev/null 2>&1;" +
// yum is probed after dnf, and the order is load-bearing: Amazon Linux
diff --git a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxy.java b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxy.java
index 46d6839a44..6ce0af25b5 100644
--- a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxy.java
+++ b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxy.java
@@ -11,11 +11,21 @@ import java.util.Optional;
*/
public final class Ec2MetadataProxy {
+ /**
+ * Run first in the shells that install and start these proxies and in the sshd install probe.
+ * {@code docker exec} hands the shell the image's environment, so an application's library
+ * search path or Python installation would otherwise be what the package manager, {@code socat}
+ * and {@code curl} load. Only the helper shell changes; the container's configured environment
+ * and its processes keep these variables.
+ */
+ static final String CLEAR_APPLICATION_PATHS = "unset LD_LIBRARY_PATH PYTHONPATH PYTHONHOME";
+
private Ec2MetadataProxy() {}
public static String[] installCommand() {
return new String[]{"sh", "-c", String.join("\n",
"set -eu",
+ CLEAR_APPLICATION_PATHS,
"if command -v ip >/dev/null 2>&1 && command -v socat >/dev/null 2>&1 && command -v curl >/dev/null 2>&1; then exit 0; fi",
"if command -v apt-get >/dev/null 2>&1; then",
" apt-get update -qq >/dev/null",
@@ -43,12 +53,36 @@ public final class Ec2MetadataProxy {
public static String[] startCommand(String flociHost, int imdsPort) {
return startCommand("imds", "169.254.169.254", 80, flociHost, imdsPort,
- "curl -fsS --max-time 1 http://169.254.169.254/latest/meta-data/instance-id >/dev/null && exit 0");
+ imdsProbeCommand("http://169.254.169.254"));
+ }
+
+ /**
+ * Ready once an IMDSv2 token reads the instance ID, which also holds on an instance whose
+ * HttpTokens is required. A refused or empty token ends the attempt rather than falling back to
+ * a tokenless read, which an empty token header would otherwise become. Both requests bypass
+ * any HTTP proxy the image configures: the check is of the link-local path itself.
+ */
+ static String imdsProbeCommand(String endpoint) {
+ return "token=$(curl -fsS --noproxy '*' --max-time 1 -X PUT "
+ + "-H 'X-aws-ec2-metadata-token-ttl-seconds: 60' " + endpoint + "/latest/api/token)"
+ + " && [ -n \"$token\" ]"
+ + " && curl -fsS --noproxy '*' --max-time 1 -H \"X-aws-ec2-metadata-token: $token\" "
+ + endpoint + "/latest/meta-data/instance-id >/dev/null && exit 0";
}
public static String[] podIdentityStartCommand(String flociHost, int flociPort) {
return startCommand("pod-identity", "169.254.170.23", 80, flociHost, flociPort,
- "[ \"$(curl -s -o /dev/null -w '%{http_code}' --max-time 1 http://169.254.170.23/v1/credentials)\" != \"000\" ] && exit 0");
+ podIdentityProbeCommand("http://169.254.170.23"));
+ }
+
+ /**
+ * Ready once the relay answers at all: without a pod's token Floci answers 400, which still
+ * proves socat relays. Any HTTP proxy the image configures is bypassed, since a proxy's own
+ * answer would pass the same check.
+ */
+ static String podIdentityProbeCommand(String endpoint) {
+ return "[ \"$(curl -s --noproxy '*' -o /dev/null -w '%{http_code}' --max-time 1 " + endpoint
+ + "/v1/credentials)\" != \"000\" ] && exit 0";
}
public static String[] startCommand(String name, String bindIp, int bindPort,
@@ -57,6 +91,7 @@ public final class Ec2MetadataProxy {
String logFile = "/tmp/floci-" + name + "-proxy.log";
return new String[]{"sh", "-c", String.join("\n",
"set -eu",
+ CLEAR_APPLICATION_PATHS,
"ip addr show dev lo | grep -q '" + bindIp + "/32' || ip addr add " + bindIp + "/32 dev lo",
"if [ -f " + pidFile + " ] && kill -0 \"$(cat " + pidFile + ")\" 2>/dev/null; then",
" exit 0",
diff --git a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataServer.java b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataServer.java
index 72271ac8c9..455898738f 100644
--- a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataServer.java
+++ b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataServer.java
@@ -10,6 +10,7 @@ import io.github.hectorvent.floci.services.iam.model.InstanceProfile;
import io.github.hectorvent.floci.services.iam.model.SessionCredential;
import io.vertx.core.Vertx;
import io.vertx.core.buffer.Buffer;
+import io.vertx.core.http.HttpMethod;
import io.vertx.core.http.HttpServer;
import io.vertx.core.json.JsonObject;
import io.vertx.ext.web.Router;
@@ -136,6 +137,10 @@ public class Ec2MetadataServer {
Router router = Router.router(vertx);
router.route().handler(BodyHandler.create());
+ // Ahead of every route, so a path IMDS does not serve also answers 401 when the instance
+ // requires tokens, as on AWS. A method-free route, so methods IMDS does not serve keep the
+ // router's own answers rather than all becoming 405.
+ router.route().handler(this::refuseTokenlessReadsWhenTokensAreRequired);
// IMDSv2 token endpoint
router.put("/latest/api/token").handler(this::handleToken);
@@ -496,6 +501,24 @@ public class Ec2MetadataServer {
return containerIpToInstance.get(remoteIp);
}
+ /**
+ * An instance whose HttpTokens is required refuses IMDSv1: as AWS does, a GET or HEAD without a
+ * token gets 401 before any data is looked up, whatever the path. (No route serves HEAD, so one
+ * with a token gets the router's answer: 405 on a path a GET route serves, 404 elsewhere.)
+ */
+ private void refuseTokenlessReadsWhenTokensAreRequired(RoutingContext ctx) {
+ HttpMethod method = ctx.request().method();
+ String token = ctx.request().getHeader("x-aws-ec2-metadata-token");
+ if ((HttpMethod.GET.equals(method) || HttpMethod.HEAD.equals(method)) && (token == null || token.isBlank())) {
+ Instance inst = resolveInstanceByIp(ctx);
+ if (inst != null && "required".equals(inst.effectiveMetadataOptions().getHttpTokens())) {
+ ctx.response().setStatusCode(401).end();
+ return;
+ }
+ }
+ ctx.next();
+ }
+
private Instance resolveInstance(RoutingContext ctx) {
String remoteIp = ctx.request().remoteAddress().host();
Instance inst = containerIpToInstance.get(remoteIp);
@@ -503,7 +526,8 @@ public class Ec2MetadataServer {
// IMDSv2: a presented token must be valid; an invalid or expired one gets 401 so the
// caller fetches a new token. A token is not valid on another instance, so a caller whose
// IP maps to a different instance also gets 401; an unregistered IP defers to the token.
- // Requests without a token fall back to IMDSv1.
+ // Requests without a token fall back to IMDSv1; one whose instance requires tokens has
+ // already been refused before routing.
String token = ctx.request().getHeader("x-aws-ec2-metadata-token");
if (token != null && !token.isBlank()) {
SessionToken session = tokens.get(token);
diff --git a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2Service.java b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2Service.java
index ad074f7a0b..be2255d797 100644
--- a/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2Service.java
+++ b/src/main/java/io/github/hectorvent/floci/services/ec2/Ec2Service.java
@@ -4971,12 +4971,21 @@ public class Ec2Service implements ContainerTeardown, ResourceProvider, Resettab
/**
* ModifyInstanceMetadataOptions changes only the fields the caller names, as on AWS, so a
* call that sets HttpTokens alone leaves HttpEndpoint and the hop limit as they were.
+ *
+ * <p>A cluster node is refused, as its lifecycle actions are: IMDS serves the instance its
+ * cluster registered, and EC2 holds only a copy of it, so a change made here would be reported
+ * by DescribeInstances but never enforced.
*/
public Instance modifyInstanceMetadataOptions(String region, String instanceId,
LaunchTemplateData.MetadataOptions request) {
ensureDefaultResources(region);
validateMetadataOptions(request);
Instance inst = getRequiredInstance(region, instanceId);
+ if (isExternalInstance(region, instanceId)) {
+ throw new AwsException("OperationNotPermitted",
+ "The instance '" + instanceId + "' is a cluster node and its metadata options cannot be"
+ + " modified through EC2", 400);
+ }
inst.setMetadataOptions(LaunchTemplateData.MetadataOptions.merge(inst.effectiveMetadataOptions(), request));
instances.put(key(region, instanceId), inst);
return inst;
@@ -5149,9 +5158,12 @@ public class Ec2Service implements ContainerTeardown, ResourceProvider, Resettab
return result;
}
+ /**
+ * Whether a cluster's provider serves this instance. It stays a cluster node after an action has
+ * stored EC2's copy of it, because IMDS and the cluster keep serving the provider's record.
+ */
private boolean isExternalInstance(String region, String instanceId) {
- return instances.get(key(region, instanceId)).isEmpty()
- && findExternalInstance(callerAccountId(), region, instanceId).isPresent();
+ return findExternalInstance(callerAccountId(), region, instanceId).isPresent();
}
private Instance getRequiredInstance(String region, String instanceId) {
diff --git a/src/main/java/io/github/hectorvent/floci/services/ec2/model/Instance.java b/src/main/java/io/github/hectorvent/floci/services/ec2/model/Instance.java
index 25609ebcb2..23444ccfd8 100644
--- a/src/main/java/io/github/hectorvent/floci/services/ec2/model/Instance.java
+++ b/src/main/java/io/github/hectorvent/floci/services/ec2/model/Instance.java
@@ -63,8 +63,9 @@ public class Instance {
private boolean disableApiTermination = false;
// Null for an instance persisted before metadata options were stored; reads fall back to
- // AWS's launch defaults through effectiveMetadataOptions().
- private LaunchTemplateData.MetadataOptions metadataOptions;
+ // AWS's launch defaults through effectiveMetadataOptions(). Volatile because IMDS reads it on
+ // its event loop while ModifyInstanceMetadataOptions replaces it from a request thread.
+ private volatile LaunchTemplateData.MetadataOptions metadataOptions;
// The credit option this instance acquired, at launch from the request or from its burstable
// family's default, or at a resize onto a burstable type. Stored rather than derived from the
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManagerTest.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManagerTest.java
index 8efc292721..8068fcaf65 100644
--- a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManagerTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ContainerManagerTest.java
@@ -38,6 +38,7 @@ import io.github.hectorvent.floci.services.ec2.model.InstanceNetworkInterface;
import io.github.hectorvent.floci.services.ec2.net.VpcNetworkManager;
import io.github.hectorvent.floci.services.ec2.portforward.Ec2PortForwardManager;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.CsvSource;
import org.mockito.ArgumentCaptor;
@@ -45,6 +46,8 @@ import org.mockito.ArgumentCaptor;
import java.io.Closeable;
import java.io.InputStream;
import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
import java.time.Duration;
import java.util.ArrayList;
import java.util.Arrays;
@@ -961,6 +964,26 @@ class Ec2ContainerManagerTest {
"dnf must be probed before yum");
}
+ @Test
+ void sshdInstallProbeClearsApplicationPathsBeforeThePackageManagerRuns(@TempDir Path directory)
+ throws Exception {
+ // dnf and yum are Python programs, so an image's PYTHONHOME or LD_LIBRARY_PATH breaks the
+ // install. The fake dnf "installs" sshd and scp only when it runs with those cleared.
+ Path bin = Files.createDirectory(directory.resolve("bin"));
+ Path record = directory.resolve("tools");
+ HelperShellFixture.writeTool(bin, "dnf", HelperShellFixture.checkedToolEnvironment()
+ + "for tool in sshd scp; do\n"
+ + " printf '#!/bin/sh\\nexit 0\\n' > \"$TEST_BIN/$tool\"\n"
+ + " /bin/chmod +x \"$TEST_BIN/$tool\"\n"
+ + "done\n");
+
+ assertEquals(0, HelperShellFixture.run(Ec2ContainerManager.sshdInstallProbeCommand(), directory,
+ HelperShellFixture.applicationEnvironment(Map.of("PATH", bin.toString(),
+ "TEST_RECORD", record.toString(), "TEST_BIN", bin.toString()))),
+ HelperShellFixture.output(directory));
+ assertEquals("dnf\n", Files.readString(record));
+ }
+
@Test
void metadataProxyInstallCommandHandlesYumForAmazonLinux2() {
// The IMDS proxy dependency install had the same gap, and unlike the sshd probe it
@@ -1574,7 +1597,8 @@ class Ec2ContainerManagerTest {
}
private static final String[] SSHD_INSTALL_PROBE_CMD = {"sh", "-c",
- "if ! command -v sshd >/dev/null 2>&1 || ! command -v scp >/dev/null 2>&1; then"
+ "unset LD_LIBRARY_PATH PYTHONPATH PYTHONHOME;"
+ + "if ! command -v sshd >/dev/null 2>&1 || ! command -v scp >/dev/null 2>&1; then"
+ " if command -v dnf >/dev/null 2>&1; then dnf install -y openssh-server openssh-clients >/dev/null 2>&1;"
+ " elif command -v yum >/dev/null 2>&1; then yum install -y openssh-server openssh-clients >/dev/null 2>&1;"
+ " elif command -v apt-get >/dev/null 2>&1; then DEBIAN_FRONTEND=noninteractive apt-get install -y openssh-server openssh-client >/dev/null 2>&1;"
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2LaunchTemplateMetadataOptionsIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2LaunchTemplateMetadataOptionsIntegrationTest.java
index 7536094b8b..6b8124f880 100644
--- a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2LaunchTemplateMetadataOptionsIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2LaunchTemplateMetadataOptionsIntegrationTest.java
@@ -1,5 +1,6 @@
package io.github.hectorvent.floci.services.ec2;
+import io.github.hectorvent.floci.config.EmulatorConfig;
import io.github.hectorvent.floci.core.common.RequestScopes;
import io.github.hectorvent.floci.services.autoscaling.AutoScalingReconciler;
import io.github.hectorvent.floci.services.autoscaling.AutoScalingService;
@@ -15,8 +16,14 @@ import java.util.UUID;
import static io.restassured.RestAssured.given;
import static org.hamcrest.Matchers.equalTo;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;
+import static org.mockito.Mockito.mock;
-/** A launch template's MetadataOptions reach instances launched by EC2 Fleet and Auto Scaling. */
+/**
+ * A launch template's MetadataOptions reach instances launched by EC2 Fleet and Auto Scaling, and
+ * IMDS enforces them there.
+ */
@QuarkusTest
class Ec2LaunchTemplateMetadataOptionsIntegrationTest {
private static final String OWNER = "212121212121";
@@ -28,9 +35,12 @@ class Ec2LaunchTemplateMetadataOptionsIntegrationTest {
@Inject
AutoScalingService groups;
+ @Inject
+ Ec2Service ec2;
+
@ParameterizedTest
@ValueSource(strings = {"fleet", "asg"})
- void templateMetadataOptionsReachTheLaunchedInstance(String source) {
+ void templateMetadataOptionsReachTheLaunchedInstance(String source) throws Exception {
String name = "template-metadata-" + UUID.randomUUID().toString().substring(0, 8);
String template = request("ec2", "CreateLaunchTemplate").formParam("LaunchTemplateName", name)
.formParam("LaunchTemplateData.ImageId", "ami-amazonlinux2023")
@@ -64,6 +74,13 @@ class Ec2LaunchTemplateMetadataOptionsIntegrationTest {
request("ec2", "DescribeInstances").formParam("InstanceId.1", instance).post("/").then().statusCode(200)
.body(INSTANCE + "metadataOptions.httpTokens", equalTo("required"))
.body(INSTANCE + "metadataOptions.httpPutResponseHopLimit", equalTo("2"));
+ // IMDS then refuses the launched instance a tokenless read, as AWS does.
+ try (Ec2MetadataTestFixture imds = new Ec2MetadataTestFixture()) {
+ imds.start(null, mock(EmulatorConfig.class, RETURNS_DEEP_STUBS),
+ ec2.findInstanceForAccount(OWNER, "us-east-1", instance).orElseThrow());
+ assertEquals(401, imds.get("/latest/meta-data/instance-id", null).statusCode());
+ assertEquals(200, imds.get("/latest/meta-data/instance-id", imds.putToken("60").body()).statusCode());
+ }
} finally {
// Cleanup ignores the status, so a test that failed before launching keeps its own failure.
if ("asg".equals(source)) {
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxyTest.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxyTest.java
index 577d0d936b..26a1470a54 100644
--- a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxyTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataProxyTest.java
@@ -1,12 +1,28 @@
package io.github.hectorvent.floci.services.ec2;
import com.github.dockerjava.api.model.ContainerNetwork;
+import com.sun.net.httpserver.HttpServer;
+import io.github.hectorvent.floci.services.ec2.model.Instance;
+import io.github.hectorvent.floci.services.ec2.model.LaunchTemplateData;
import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.CsvSource;
+import java.net.InetSocketAddress;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.List;
import java.util.Map;
import java.util.Optional;
+import java.util.Set;
+import java.util.UUID;
+import java.util.concurrent.CopyOnWriteArrayList;
+import java.util.concurrent.atomic.AtomicInteger;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertTrue;
class Ec2MetadataProxyTest {
@@ -40,8 +56,10 @@ class Ec2MetadataProxyTest {
assertTrue(script.contains("if [ -f /tmp/floci-imds-proxy.pid ] && kill -0 \"$(cat /tmp/floci-imds-proxy.pid)\" 2>/dev/null; then\n exit 0\nfi"));
// Targets configured Floci host and IMDS port
assertTrue(script.contains("nohup socat TCP-LISTEN:80,bind=169.254.169.254,fork,reuseaddr TCP:10.0.0.1:9169 >/tmp/floci-imds-proxy.log 2>&1 &"));
- // Verifies via link-local curl
- assertTrue(script.contains("curl -fsS --max-time 1 http://169.254.169.254/latest/meta-data/instance-id >/dev/null && exit 0"));
+ // Verifies through an IMDSv2 token even when tokenless access is refused.
+ assertTrue(script.contains("-X PUT -H 'X-aws-ec2-metadata-token-ttl-seconds: 60'"));
+ assertTrue(script.contains("http://169.254.169.254/latest/api/token"));
+ assertTrue(script.contains("-H \"X-aws-ec2-metadata-token: $token\" http://169.254.169.254/latest/meta-data/instance-id"));
}
@Test
@@ -85,4 +103,239 @@ class Ec2MetadataProxyTest {
assertTrue(Ec2MetadataProxy.preferredMetadataSourceIp(null).isEmpty());
assertTrue(Ec2MetadataProxy.preferredMetadataSourceIp(Map.of()).isEmpty());
}
+
+ @Test
+ void installCommandClearsApplicationPathsBeforeThePackageManagerRuns(@TempDir Path directory)
+ throws Exception {
+ Path bin = Files.createDirectory(directory.resolve("bin"));
+ Path record = directory.resolve("tools");
+ Path arguments = directory.resolve("arguments");
+ HelperShellFixture.writeTool(bin, "dnf", HelperShellFixture.checkedToolEnvironment()
+ + "printf '%s\\n' \"$*\" > \"$TEST_ARGUMENTS\"\n");
+
+ assertEquals(0, HelperShellFixture.run(Ec2MetadataProxy.installCommand(), directory,
+ HelperShellFixture.applicationEnvironment(Map.of("PATH", bin.toString(),
+ "TEST_RECORD", record.toString(), "TEST_ARGUMENTS", arguments.toString()))),
+ HelperShellFixture.output(directory));
+
+ assertEquals("dnf\n", Files.readString(record));
+ assertEquals("install -y --allowerasing iproute socat curl ca-certificates\n",
+ Files.readString(arguments));
+ }
+
+ @Test
+ void startCommandClearsApplicationPathsBeforeAddressProxyAndProbeTools(@TempDir Path directory)
+ throws Exception {
+ Path bin = Files.createDirectory(directory.resolve("bin"));
+ Path record = directory.resolve("tools");
+ Path proxyReady = directory.resolve("proxy-ready");
+ HelperShellFixture.writeTool(bin, "ip", HelperShellFixture.checkedToolEnvironment()
+ + "printf '127.0.0.1/32\\n'\n");
+ HelperShellFixture.writeTool(bin, "socat", HelperShellFixture.checkedToolEnvironment()
+ + ": > \"$TEST_PROXY_READY\"\n");
+ HelperShellFixture.writeTool(bin, "curl", HelperShellFixture.checkedToolEnvironment()
+ + "for attempt in 1 2 3 4 5 6 7 8 9 10; do\n"
+ + " [ -f \"$TEST_PROXY_READY\" ] && exit 0\n"
+ + " /bin/sleep 0.1\n"
+ + "done\nexit 1\n");
+ String name = "unit-" + UUID.randomUUID();
+ try {
+ String[] command = Ec2MetadataProxy.startCommand(
+ name, "127.0.0.1", 80, "127.0.0.1", 9169, "curl fixture && exit 0");
+ assertEquals(0, HelperShellFixture.run(command, directory,
+ HelperShellFixture.applicationEnvironment(Map.of("PATH", bin + ":" + System.getenv("PATH"),
+ "TEST_RECORD", record.toString(), "TEST_PROXY_READY", proxyReady.toString()))),
+ HelperShellFixture.output(directory));
+ assertEquals(Set.of("ip", "socat", "curl"), Set.copyOf(Files.readAllLines(record)));
+ } finally {
+ deleteProxyFiles(name);
+ }
+ }
+
+ @Test
+ void readinessProbeReadsTheInstanceIdWithATokenWhenTheInstanceRequiresOne(@TempDir Path directory)
+ throws Exception {
+ try (Ec2MetadataTestFixture fixture = new Ec2MetadataTestFixture()) {
+ fixture.start();
+ requireTokens(fixture.instance());
+
+ assertEquals(0, HelperShellFixture.run(probe(fixture.endpoint()), directory, Map.of()),
+ HelperShellFixture.output(directory));
+ }
+ }
+
+ /** An image's proxy settings are for its application; the probe checks the link-local path itself. */
+ @Test
+ void readinessProbeIgnoresTheImagesHttpProxy(@TempDir Path directory) throws Exception {
+ List<String> proxied = new CopyOnWriteArrayList<>();
+ HttpServer proxy = failingProxy(proxied);
+ try (Ec2MetadataTestFixture fixture = new Ec2MetadataTestFixture()) {
+ fixture.start();
+ requireTokens(fixture.instance());
+
+ int result = HelperShellFixture.run(probe(fixture.endpoint()), directory, proxyEnvironment(proxy));
+
+ assertEquals(0, result, HelperShellFixture.output(directory));
+ assertEquals(List.of(), proxied);
+ } finally {
+ proxy.stop(0);
+ }
+ }
+
+ /**
+ * The pod identity relay answers 400 without a pod token, which still proves socat relays. A
+ * proxy that answered in its place would look the same to the status check, so the probe
+ * must not go through one.
+ */
+ @Test
+ void podIdentityProbeIgnoresTheImagesHttpProxy(@TempDir Path directory) throws Exception {
+ List<String> proxied = new CopyOnWriteArrayList<>();
+ AtomicInteger relayed = new AtomicInteger();
+ HttpServer proxy = null;
+ HttpServer relay = null;
+ try {
+ proxy = failingProxy(proxied);
+ relay = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
+ relay.createContext("/v1/credentials", exchange -> {
+ relayed.incrementAndGet();
+ exchange.sendResponseHeaders(400, -1);
+ exchange.close();
+ });
+ relay.start();
+ String[] probe = {"sh", "-c",
+ Ec2MetadataProxy.podIdentityProbeCommand("http://127.0.0.1:" + relay.getAddress().getPort())};
+
+ int result = HelperShellFixture.run(probe, directory, proxyEnvironment(proxy));
+
+ assertEquals(0, result, HelperShellFixture.output(directory));
+ assertEquals(1, relayed.get(), HelperShellFixture.output(directory));
+ assertEquals(List.of(), proxied);
+ } finally {
+ if (relay != null) {
+ relay.stop(0);
+ }
+ if (proxy != null) {
+ proxy.stop(0);
+ }
+ }
+ }
+
+ /**
+ * Floci's IMDS issues a token for every valid request, so the refused and empty token answers
+ * come from a stand-in that records what the probe sent.
+ */
+ @ParameterizedTest
+ @CsvSource({"403, rejected", "200, ''"})
+ void readinessProbeNeverFallsBackToATokenlessRead(int tokenStatus, String tokenBody, @TempDir Path directory)
+ throws Exception {
+ List<String> requests = new CopyOnWriteArrayList<>();
+ HttpServer server = metadataStandIn(requests, attempt -> tokenStatus, tokenBody);
+ try {
+ int result = HelperShellFixture.run(probe("http://127.0.0.1:" + server.getAddress().getPort()),
+ directory, Map.of());
+
+ assertNotEquals(0, result, HelperShellFixture.output(directory));
+ assertEquals(List.of("PUT token 60"), requests, HelperShellFixture.output(directory));
+ } finally {
+ server.stop(0);
+ }
+ }
+
+ /**
+ * The proxy's first probes run before socat listens, so a refused token has to end only that
+ * attempt: the generated start script, under {@code set -eu}, retries until a token is issued.
+ */
+ @Test
+ void startScriptRetriesTheTokenProbeUntilATokenIsIssued(@TempDir Path directory) throws Exception {
+ Path bin = Files.createDirectory(directory.resolve("bin"));
+ HelperShellFixture.writeTool(bin, "ip", "printf '127.0.0.1/32\\n'\n");
+ HelperShellFixture.writeTool(bin, "socat", "exit 0\n");
+ List<String> requests = new CopyOnWriteArrayList<>();
+ HttpServer server = metadataStandIn(requests, attempt -> attempt == 1 ? 403 : 200, "fixture-token");
+ String name = "unit-" + UUID.randomUUID();
+ try {
+ String[] command = Ec2MetadataProxy.startCommand(name, "127.0.0.1", 80, "127.0.0.1", 9169,
+ Ec2MetadataProxy.imdsProbeCommand("http://127.0.0.1:" + server.getAddress().getPort()));
+
+ int result = HelperShellFixture.run(command, directory,
+ Map.of("PATH", bin + ":" + System.getenv("PATH")));
+
+ assertEquals(0, result, HelperShellFixture.output(directory));
+ assertEquals(List.of("PUT token 60", "PUT token 60", "GET metadata fixture-token"), requests,
+ HelperShellFixture.output(directory));
+ } finally {
+ server.stop(0);
+ deleteProxyFiles(name);
+ }
+ }
+
+ /**
+ * Answers token requests with the status {@code tokenStatus} picks for each attempt, serves the
+ * instance ID to the issued token and, as an instance allowing IMDSv1 would, to a tokenless
+ * read, and records what it was sent.
+ */
+ private static HttpServer metadataStandIn(List<String> requests, TokenStatus tokenStatus, String tokenBody)
+ throws Exception {
+ AtomicInteger tokenRequests = new AtomicInteger();
+ HttpServer server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
+ server.createContext("/latest/api/token", exchange -> {
+ requests.add(exchange.getRequestMethod() + " token "
+ + exchange.getRequestHeaders().getFirst(Ec2MetadataTestFixture.TTL_HEADER));
+ byte[] body = tokenBody.getBytes(StandardCharsets.UTF_8);
+ exchange.sendResponseHeaders(tokenStatus.forAttempt(tokenRequests.incrementAndGet()),
+ body.length == 0 ? -1 : body.length);
+ exchange.getResponseBody().write(body);
+ exchange.close();
+ });
+ server.createContext("/latest/meta-data/instance-id", exchange -> {
+ String token = exchange.getRequestHeaders().getFirst(Ec2MetadataTestFixture.TOKEN_HEADER);
+ requests.add(exchange.getRequestMethod() + " metadata " + token);
+ byte[] body = "i-fixture".getBytes(StandardCharsets.UTF_8);
+ boolean accepted = token == null || !tokenBody.isEmpty() && tokenBody.equals(token);
+ exchange.sendResponseHeaders(accepted ? 200 : 401, body.length);
+ exchange.getResponseBody().write(body);
+ exchange.close();
+ });
+ server.start();
+ return server;
+ }
+
+ @FunctionalInterface
+ private interface TokenStatus {
+ int forAttempt(int attempt);
+ }
+
+ /** An HTTP proxy that records every request it is sent and answers each with 502. */
+ private static HttpServer failingProxy(List<String> proxied) throws Exception {
+ HttpServer proxy = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
+ proxy.createContext("/", exchange -> {
+ proxied.add(exchange.getRequestMethod() + " " + exchange.getRequestURI());
+ exchange.sendResponseHeaders(502, -1);
+ exchange.close();
+ });
+ proxy.start();
+ return proxy;
+ }
+
+ /** The proxy settings an image could carry, with nothing exempted. */
+ private static Map<String, String> proxyEnvironment(HttpServer proxy) {
+ String proxyUrl = "http://127.0.0.1:" + proxy.getAddress().getPort();
+ return Map.of("http_proxy", proxyUrl, "HTTP_PROXY", proxyUrl, "all_proxy", proxyUrl,
+ "ALL_PROXY", proxyUrl, "no_proxy", "", "NO_PROXY", "");
+ }
+
+ private static String[] probe(String endpoint) {
+ return new String[]{"sh", "-c", Ec2MetadataProxy.imdsProbeCommand(endpoint)};
+ }
+
+ private static void requireTokens(Instance instance) {
+ LaunchTemplateData.MetadataOptions options = LaunchTemplateData.MetadataOptions.launchDefaults();
+ options.setHttpTokens("required");
+ instance.setMetadataOptions(options);
+ }
+
+ private static void deleteProxyFiles(String name) throws Exception {
+ Files.deleteIfExists(Path.of("/tmp/floci-" + name + "-proxy.pid"));
+ Files.deleteIfExists(Path.of("/tmp/floci-" + name + "-proxy.log"));
+ }
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTestFixture.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTestFixture.java
index 677f44640c..92ad397073 100644
--- a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTestFixture.java
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTestFixture.java
@@ -84,6 +84,11 @@ final class Ec2MetadataTestFixture implements AutoCloseable {
return instance;
}
+ /** The server's base URL, for a client other than this fixture's own requests. */
+ String endpoint() {
+ return endpoint;
+ }
+
static Instance instance(String instanceId) {
Instance instance = new Instance();
instance.setInstanceId(instanceId);
@@ -100,6 +105,16 @@ final class Ec2MetadataTestFixture implements AutoCloseable {
return get(path, token, HttpResponse.BodyHandlers.ofString());
}
+ /** A request with any method and no body, with the token header when {@code token} is not null. */
+ HttpResponse<String> send(String method, String path, String token) throws Exception {
+ HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(endpoint + path))
+ .timeout(Duration.ofSeconds(5)).method(method, HttpRequest.BodyPublishers.noBody());
+ if (token != null) {
+ request.header(TOKEN_HEADER, token);
+ }
+ return client.send(request.build(), HttpResponse.BodyHandlers.ofString());
+ }
+
<T> HttpResponse<T> get(String path, String token, HttpResponse.BodyHandler<T> bodyHandler) throws Exception {
HttpRequest.Builder request = HttpRequest.newBuilder(URI.create(endpoint + path))
.timeout(Duration.ofSeconds(5));
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenRequirementIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenRequirementIntegrationTest.java
new file mode 100644
index 0000000000..f0b89c6a98
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenRequirementIntegrationTest.java
@@ -0,0 +1,76 @@
+package io.github.hectorvent.floci.services.ec2;
+
+import io.github.hectorvent.floci.config.EmulatorConfig;
+import io.github.hectorvent.floci.services.ec2.model.Instance;
+import io.github.hectorvent.floci.testing.EnforcementFixtures;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.specification.RequestSpecification;
+import jakarta.inject.Inject;
+import org.junit.jupiter.api.Test;
+
+import java.net.http.HttpResponse;
+
+import static io.restassured.RestAssured.given;
+import static org.hamcrest.Matchers.equalTo;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;
+import static org.mockito.Mockito.mock;
+
+/**
+ * The HttpTokens an instance is launched with, and any later ModifyInstanceMetadataOptions, decide
+ * whether IMDS still answers a request without a token: on AWS a required token turns IMDSv1 reads
+ * into 401. The stored instance is registered with a metadata server of its own, as a launched
+ * container would be.
+ */
+@QuarkusTest
+class Ec2MetadataTokenRequirementIntegrationTest {
+ private static final String ACCOUNT = "357924681357";
+ private static final String REGION = "us-east-1";
+ private static final String INSTANCE_ID = "/latest/meta-data/instance-id";
+
+ @Inject
+ Ec2Service ec2;
+
+ @Test
+ void launchAndModifiedTokenRequirementsApplyToTheRunningInstance() throws Exception {
+ String instanceId = request("RunInstances")
+ .formParam("ImageId", "ami-0abcdef1234567890").formParam("InstanceType", "t3.micro")
+ .formParam("MinCount", "1").formParam("MaxCount", "1")
+ .formParam("MetadataOptions.HttpTokens", "required")
+ .post("/").then().statusCode(200)
+ .extract().path("RunInstancesResponse.instancesSet.item.instanceId");
+ try (Ec2MetadataTestFixture fixture = new Ec2MetadataTestFixture()) {
+ Instance stored = ec2.findInstanceForAccount(ACCOUNT, REGION, instanceId).orElseThrow();
+ fixture.start(null, mock(EmulatorConfig.class, RETURNS_DEEP_STUBS), stored);
+
+ assertEquals(401, fixture.get(INSTANCE_ID, null).statusCode());
+ HttpResponse<String> withToken = fixture.get(INSTANCE_ID, fixture.putToken("60").body());
+ assertEquals(200, withToken.statusCode());
+ assertEquals(instanceId, withToken.body());
+
+ modifyHttpTokens(instanceId, "optional");
+ HttpResponse<String> withoutToken = fixture.get(INSTANCE_ID, null);
+ assertEquals(200, withoutToken.statusCode());
+ assertEquals(instanceId, withoutToken.body());
+
+ modifyHttpTokens(instanceId, "required");
+ assertEquals(401, fixture.get(INSTANCE_ID, null).statusCode());
+ } finally {
+ request("TerminateInstances").formParam("InstanceId.1", instanceId)
+ .post("/").then().statusCode(200);
+ }
+ }
+
+ private static void modifyHttpTokens(String instanceId, String httpTokens) {
+ request("ModifyInstanceMetadataOptions").formParam("InstanceId", instanceId)
+ .formParam("HttpTokens", httpTokens)
+ .post("/").then().statusCode(200)
+ .body("ModifyInstanceMetadataOptionsResponse.instanceMetadataOptions.httpTokens",
+ equalTo(httpTokens));
+ }
+
+ private static RequestSpecification request(String action) {
+ return given().header("Authorization", EnforcementFixtures.auth(ACCOUNT, REGION, "ec2"))
+ .contentType("application/x-www-form-urlencoded").formParam("Action", action);
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenTest.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenTest.java
index 0f0500cc38..f5e1ccb041 100644
--- a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2MetadataTokenTest.java
@@ -1,6 +1,7 @@
package io.github.hectorvent.floci.services.ec2;
import io.github.hectorvent.floci.services.ec2.model.Instance;
+import io.github.hectorvent.floci.services.ec2.model.LaunchTemplateData;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
@@ -15,7 +16,8 @@ import static org.junit.jupiter.api.Assertions.assertEquals;
/**
* IMDSv2 session tokens, as documented in the EC2 User Guide: the PUT must carry a TTL between
* 1 and 21,600 seconds (otherwise 400), and a GET that presents an invalid or expired token gets
- * 401 so the caller fetches a new token. A token is not valid on other instances.
+ * 401 so the caller fetches a new token. A token is not valid on other instances. When the
+ * instance's HttpTokens is required, a GET without a valid token also gets 401.
*/
class Ec2MetadataTokenTest {
@@ -100,6 +102,117 @@ class Ec2MetadataTokenTest {
assertEquals(200, response.statusCode());
}
+ @ParameterizedTest
+ @ValueSource(strings = {
+ "/latest/meta-data/instance-id",
+ "/latest/dynamic/instance-identity/document",
+ "/latest/user-data",
+ "/latest/meta-data/iam/info",
+ "/latest/meta-data/iam/security-credentials/",
+ "/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance",
+ "/latest/meta-data/tags/instance/"
+ })
+ void requiredTokenRefusesDataBeforeResolvingItsContents(String path) throws Exception {
+ setTokenRequirement("required");
+ instance.setUserData("private-user-data");
+
+ HttpResponse<String> response = getWithoutToken(path);
+
+ assertEquals(401, response.statusCode());
+ assertEquals("", response.body());
+ }
+
+ /** AWS refuses the request before looking at the path, so a listing IMDS does not serve is 401 too. */
+ @ParameterizedTest
+ @ValueSource(strings = {"/latest/meta-data/", "/latest/meta-data/no-such-item", "/latest/"})
+ void requiredTokenRefusesPathsImdsDoesNotServe(String path) throws Exception {
+ setTokenRequirement("required");
+
+ assertEquals(401, getWithoutToken(path).statusCode());
+ }
+
+ @Test
+ void requiredTokenRefusesATokenlessHeadRequest() throws Exception {
+ setTokenRequirement("required");
+
+ assertEquals(401, fixture.send("HEAD", "/latest/meta-data/instance-id", null).statusCode());
+ }
+
+ /** No route serves HEAD, so with a token the router answers as it would for any unserved method. */
+ @Test
+ void aHeadRequestWithATokenGetsTheRoutersAnswer() throws Exception {
+ setTokenRequirement("required");
+ String token = fixture.putToken("60").body();
+
+ assertEquals(405, fixture.send("HEAD", "/latest/meta-data/instance-id", token).statusCode());
+ assertEquals(404, fixture.send("HEAD", "/latest/meta-data/no-such-item", token).statusCode());
+ }
+
+ /** The token check sits ahead of the routes without claiming the methods IMDS does not serve. */
+ @Test
+ void aMethodImdsDoesNotServeIsStillNotFound() throws Exception {
+ setTokenRequirement("required");
+ String token = fixture.putToken("60").body();
+
+ assertEquals(404, fixture.send("POST", "/latest/meta-data/", token).statusCode());
+ assertEquals(404, fixture.send("POST", "/latest/meta-data/", null).statusCode());
+ }
+
+ @Test
+ void aPathImdsDoesNotServeIsStillNotFoundWithAToken() throws Exception {
+ setTokenRequirement("required");
+ String token = fixture.putToken("60").body();
+
+ assertEquals(404, fixture.get("/latest/meta-data/no-such-item", token).statusCode());
+ setTokenRequirement("optional");
+ assertEquals(404, getWithoutToken("/latest/meta-data/no-such-item").statusCode());
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"", " "})
+ void blankTokenDoesNotBypassRequiredTokens(String token) throws Exception {
+ setTokenRequirement("required");
+
+ assertEquals(401, getInstanceId(token).statusCode());
+ }
+
+ @Test
+ void requiredTokenPreservesIssuanceValidationAndExpiry() throws Exception {
+ setTokenRequirement("required");
+ HttpResponse<String> issued = fixture.putToken("60");
+ assertEquals(200, issued.statusCode());
+
+ HttpResponse<String> accepted = getInstanceId(issued.body());
+ assertEquals(200, accepted.statusCode());
+ assertEquals(instance.getInstanceId(), accepted.body());
+ assertEquals(401, getInstanceId("invalid-token").statusCode());
+
+ fixture.clock().advance(Duration.ofSeconds(60));
+ assertEquals(401, getInstanceId(issued.body()).statusCode());
+ }
+
+ @Test
+ void tokenRequirementTracksTheRegisteredInstancesCurrentOptions() throws Exception {
+ setTokenRequirement("optional");
+ assertEquals(200, getWithoutToken("/latest/meta-data/instance-id").statusCode());
+
+ setTokenRequirement("required");
+ assertEquals(401, getWithoutToken("/latest/meta-data/instance-id").statusCode());
+
+ setTokenRequirement("optional");
+ assertEquals(200, getWithoutToken("/latest/meta-data/instance-id").statusCode());
+ }
+
+ private void setTokenRequirement(String value) {
+ LaunchTemplateData.MetadataOptions options = LaunchTemplateData.MetadataOptions.launchDefaults();
+ options.setHttpTokens(value);
+ instance.setMetadataOptions(options);
+ }
+
+ private HttpResponse<String> getWithoutToken(String path) throws Exception {
+ return fixture.get(path, null);
+ }
+
private HttpResponse<String> getInstanceId(String token) throws Exception {
return fixture.get("/latest/meta-data/instance-id", token, HttpResponse.BodyHandlers.ofString());
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ServiceTest.java
index f3332c884f..07f9958df8 100644
--- a/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/Ec2ServiceTest.java
@@ -7256,6 +7256,86 @@ class Ec2ServiceTest {
assertEquals(400, rebootEx.getHttpStatus());
}
+ @Test
+ void clusterNodeMetadataOptionsCannotBeModifiedThroughEc2() {
+ Ec2Service service = new Ec2Service(mockConfig(true), mock(Ec2ContainerManager.class),
+ mock(Ec2PortForwardManager.class), mock(AmiImageResolver.class), mock(Ec2ImageCatalog.class),
+ new Ec2InstanceTypeCatalog(), new InMemoryStorageFactory());
+ Instance nodeInstance = new Instance();
+ nodeInstance.setInstanceId("i-node1234567890abc");
+ nodeInstance.setRegion("us-east-1");
+ nodeInstance.setPlacement(new Placement("us-east-1a"));
+ nodeInstance.setState(InstanceState.running());
+ nodeInstance.setInstanceType("m5.large");
+ service.setClusterNodeInstanceProvider(new ClusterNodeInstanceProvider() {
+ @Override
+ public Optional<Instance> findInstance(String accountId, String region, String instanceId) {
+ return "i-node1234567890abc".equals(instanceId) && ("us-east-1".equals(region) || region == null)
+ ? Optional.of(nodeInstance) : Optional.empty();
+ }
+
+ @Override
+ public List<Instance> listInstances(String accountId, String region) {
+ return "us-east-1".equals(region) || region == null ? List.of(nodeInstance) : List.of();
+ }
+ });
+ LaunchTemplateData.MetadataOptions required = new LaunchTemplateData.MetadataOptions();
+ required.setHttpTokens("required");
+
+ // IMDS serves the node from the instance its cluster registered, so a change made to the
+ // copy EC2 hands out would be reported by DescribeInstances and never enforced.
+ AwsException modifyEx = assertThrows(AwsException.class, () ->
+ service.modifyInstanceMetadataOptions("us-east-1", "i-node1234567890abc", required));
+ assertEquals("OperationNotPermitted", modifyEx.getErrorCode());
+ assertEquals(400, modifyEx.getHttpStatus());
+
+ Instance described = service.describeInstances("us-east-1", List.of("i-node1234567890abc"), Map.of())
+ .getFirst().getInstances().getFirst();
+ assertEquals("optional", described.effectiveMetadataOptions().getHttpTokens());
+ assertNull(nodeInstance.getMetadataOptions());
+ AwsException terminateEx = assertThrows(AwsException.class, () ->
+ service.terminateInstances("us-east-1", List.of("i-node1234567890abc")));
+ assertEquals("OperationNotPermitted", terminateEx.getErrorCode());
+ }
+
+ @Test
+ void clusterNodeStaysRefusedAfterAnotherActionStoredACopyOfIt() {
+ Ec2Service service = new Ec2Service(mockConfig(true), mock(Ec2ContainerManager.class),
+ mock(Ec2PortForwardManager.class), mock(AmiImageResolver.class), mock(Ec2ImageCatalog.class),
+ new Ec2InstanceTypeCatalog(), new InMemoryStorageFactory());
+ Instance nodeInstance = new Instance();
+ nodeInstance.setInstanceId("i-node1234567890abc");
+ nodeInstance.setRegion("us-east-1");
+ nodeInstance.setPlacement(new Placement("us-east-1a"));
+ nodeInstance.setState(InstanceState.running());
+ nodeInstance.setInstanceType("m5.large");
+ service.setClusterNodeInstanceProvider(new ClusterNodeInstanceProvider() {
+ @Override
+ public Optional<Instance> findInstance(String accountId, String region, String instanceId) {
+ return "i-node1234567890abc".equals(instanceId) && ("us-east-1".equals(region) || region == null)
+ ? Optional.of(nodeInstance) : Optional.empty();
+ }
+
+ @Override
+ public List<Instance> listInstances(String accountId, String region) {
+ return "us-east-1".equals(region) || region == null ? List.of(nodeInstance) : List.of();
+ }
+ });
+ // ModifyInstanceAttribute writes EC2's copy of the node into the instance store; the node is
+ // still the cluster's, so the actions refused for it stay refused.
+ service.modifyInstanceAttribute("us-east-1", "i-node1234567890abc", "sourceDestCheck", "false");
+ LaunchTemplateData.MetadataOptions required = new LaunchTemplateData.MetadataOptions();
+ required.setHttpTokens("required");
+
+ assertEquals("OperationNotPermitted", assertThrows(AwsException.class, () ->
+ service.modifyInstanceMetadataOptions("us-east-1", "i-node1234567890abc", required))
+ .getErrorCode());
+ assertEquals("OperationNotPermitted", assertThrows(AwsException.class, () ->
+ service.terminateInstances("us-east-1", List.of("i-node1234567890abc"))).getErrorCode());
+ assertEquals("OperationNotPermitted", assertThrows(AwsException.class, () ->
+ service.stopInstances("us-east-1", List.of("i-node1234567890abc"))).getErrorCode());
+ }
+
@Test
void ordinaryEc2InstancesUnaffectedByClusterNodeInstanceProvider() {
Ec2Service service = new Ec2Service(mockConfig(true), mock(Ec2ContainerManager.class),
diff --git a/src/test/java/io/github/hectorvent/floci/services/ec2/HelperShellFixture.java b/src/test/java/io/github/hectorvent/floci/services/ec2/HelperShellFixture.java
new file mode 100644
index 0000000000..9fbbf0b89b
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/ec2/HelperShellFixture.java
@@ -0,0 +1,74 @@
+package io.github.hectorvent.floci.services.ec2;
+
+import java.io.IOException;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.concurrent.TimeUnit;
+
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * Runs the helper shells Floci execs inside a guest (the IMDS proxy's, the sshd install probe) on
+ * this host with {@code /bin/sh}, against fake system tools on PATH, with the environment an
+ * application image hands a {@code docker exec}.
+ */
+final class HelperShellFixture {
+
+ /** What an application image can carry into a {@code docker exec}, plus one setting the shells must keep. */
+ static final Map<String, String> APPLICATION_ENVIRONMENT = Map.of(
+ "LD_LIBRARY_PATH", "/fixture/application/lib",
+ "PYTHONPATH", "/fixture/application/modules",
+ "PYTHONHOME", "/fixture/application/python",
+ "KEEP_SETTING", "fixture-value");
+
+ private HelperShellFixture() {}
+
+ /**
+ * The start of a fake tool that fails unless the application's library and Python paths are
+ * unset while its other settings survive, and then records the tool's name in TEST_RECORD.
+ */
+ static String checkedToolEnvironment() {
+ return "[ -z \"${LD_LIBRARY_PATH+x}\" ] || exit 91\n"
+ + "[ -z \"${PYTHONPATH+x}\" ] || exit 92\n"
+ + "[ -z \"${PYTHONHOME+x}\" ] || exit 93\n"
+ + "[ \"$KEEP_SETTING\" = fixture-value ] || exit 94\n"
+ + "printf '%s\\n' \"${0##*/}\" >> \"$TEST_RECORD\"\n";
+ }
+
+ /** {@link #APPLICATION_ENVIRONMENT} plus the test's own settings. */
+ static Map<String, String> applicationEnvironment(Map<String, String> settings) {
+ Map<String, String> environment = new HashMap<>(APPLICATION_ENVIRONMENT);
+ environment.putAll(settings);
+ return environment;
+ }
+
+ static void writeTool(Path bin, String name, String script) throws IOException {
+ Path tool = bin.resolve(name);
+ Files.writeString(tool, "#!/bin/sh\nset -eu\n" + script);
+ assertTrue(tool.toFile().setExecutable(true));
+ }
+
+ /** Runs a generated helper shell with {@code /bin/sh}, its output going to {@link #output}. */
+ static int run(String[] command, Path directory, Map<String, String> environment) throws Exception {
+ List<String> arguments = new ArrayList<>(List.of(command));
+ arguments.set(0, "/bin/sh");
+ ProcessBuilder builder = new ProcessBuilder(arguments);
+ builder.environment().putAll(environment);
+ builder.redirectErrorStream(true).redirectOutput(directory.resolve("shell-output").toFile());
+ Process process = builder.start();
+ try {
+ assertTrue(process.waitFor(5, TimeUnit.SECONDS), "Generated helper shell exceeded its bound");
+ return process.exitValue();
+ } finally {
+ process.destroyForcibly();
+ }
+ }
+
+ static String output(Path directory) throws IOException {
+ return Files.readString(directory.resolve("shell-output"));
+ }
+}