Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(stepfunctions): add the elasticmapreduce optimized integrations (#6261)

Priamo Germosen94c4572fed77
diff --git a/docs/services/emr.md b/docs/services/emr.md
index 37cfab145f..2b697d7ee0 100644
--- a/docs/services/emr.md
+++ b/docs/services/emr.md
@@ -62,12 +62,27 @@ steps added later complete, reporting `ALL_STEPS_COMPLETED`. A cluster with no s
`WAITING` after its steps complete. Automatic step-completion termination also applies when
termination protection is enabled.
`AddJobFlowSteps` rejects a terminated cluster without adding steps or changing its
-termination status.
+termination status, and answers an unknown cluster with `ValidationException` "Specified job flow ID
+does not exist.", as AWS does.
After automatic or manual termination, instance groups and fleets report `TERMINATED` with zero
running or provisioned capacity; their requested and target capacities remain recorded.
`ListInstances` retains each group's instances with their original synthetic IDs and `TERMINATED`
status for 30 days after termination. Capacity modifications and new groups or fleets are rejected
-on terminated clusters. `InstanceStates` filters the modeled `RUNNING` and `TERMINATED` instances.
+on terminated clusters. As on AWS, `ModifyInstanceGroups` answers `ValidationException` "An instance
+group may only be modified when the cluster is running or waiting.", and `ModifyInstanceFleet` answers
+`InvalidRequestException` "A job flow that is shutting down, terminated, or finished may not be
+modified." with `ErrorCode` `INVALID_JOB_SHUT_DOWN`. `InstanceStates` filters the modeled `RUNNING` and
+`TERMINATED` instances.
+
+`TerminateJobFlows`, the four `Set*` cluster setters and `CancelSteps` refuse a request without a
+member their model requires (`JobFlowIds` and the setting, or `StepIds`) with `ValidationException`
+"1 validation error detected: Value null at 'jobFlowIds' failed to satisfy constraint: Member must
+not be null", AWS's framework wording, as `RunJobFlow` does for `Name`. They used to read a missing
+list as empty and a missing setting as its default, and answer success. An empty `JobFlowIds` is
+still accepted.
+
+A state machine drives clusters and steps through the `elasticmapreduce:` integrations, `.sync`
+included; see [Step Functions: Amazon EMR tasks](step-functions.md#amazon-emr-tasks).
## Release catalogue
diff --git a/docs/services/step-functions.md b/docs/services/step-functions.md
index 51859550cb..490adae37c 100644
--- a/docs/services/step-functions.md
+++ b/docs/services/step-functions.md
@@ -80,12 +80,13 @@ that does not match the selected task account fails with
`HeartbeatSeconds` as a literal number.
`resourceType` is the integrated service (`states`, `ecs`, `aws-sdk:sqs`) and `resource` the
API with its suffix (`startExecution.sync:2`, `runTask.sync`), as AWS names them. A `.sync`
- Task (`states:startExecution.sync` and `.sync:2`, `ecs:runTask.sync`) also records
+ Task (`states:startExecution.sync` and `.sync:2`, `ecs:runTask.sync`, the `elasticmapreduce:`
+ ones) also records
`TaskSubmitted` after `TaskStarted`, once the call that started its job has returned and before
it waits on the job. `taskSubmittedEventDetails` carries `resourceType`, `resource`,
`outputDetails`, and that call's response as `output`, in PascalCase: `{"ExecutionArn",
"StartDate"}` with `StartDate` in epoch milliseconds for a nested execution, `{"Failures",
- "Tasks"}` for an ECS task. AWS folds the SDK's `SdkHttpMetadata` and `SdkResponseMetadata`
+ "Tasks"}` for an ECS task, `{"ClusterId"}`, `{"StepId"}` or `{}` for an EMR one. AWS folds the SDK's `SdkHttpMetadata` and `SdkResponseMetadata`
into that response as well; Floci records the response fields alone.
- A direct Lambda function ARN emits `LambdaFunctionScheduled`, `LambdaFunctionStarted`,
and then `LambdaFunctionSucceeded` or `LambdaFunctionFailed`. `LambdaFunctionScheduled`
@@ -269,7 +270,7 @@ One deviation. Every pause is capped at `floci.services.stepfunctions.max-wait-s
ASL carries two `TimeoutSeconds` fields and Floci enforces both, in the two terminal shapes
AWS uses. The state machine's own field bounds every state; a `Task`'s own field bounds one that
waits: for a task token (an activity, or a `.waitForTaskToken` integration) or for a job to end
-(`ecs:runTask.sync`, `states:startExecution.sync` and `.sync:2`). A Lambda or other SDK task that
+(`ecs:runTask.sync`, the `elasticmapreduce:` `.sync` ones, `states:startExecution.sync` and `.sync:2`). A Lambda or other SDK task that
returns directly is not bound by it.
The state machine's own `TimeoutSeconds` is the whole execution's budget. It is checked before
@@ -302,6 +303,8 @@ both carry the cause `The Task state in AWS Step Functions execution [<arn>] whi
this resource was aborted`. A `StopExecution` that lands while the state waits ends the wait and
stops the job with the same cause, and so does the execution's budget, and so does a failure in
another branch of the `Parallel`, or in another iteration of the `Map`, that the `Task` is in.
+An `addStep.sync` step is cancelled with `CancelSteps` instead, and the cluster of a
+`createCluster.sync` is left running (see [Amazon EMR tasks](#amazon-emr-tasks)).
One deviation. AWS starts the `TimeoutSeconds` clock when a worker picks the task up, the instant
it emits `ActivityStarted`. Floci emits `ActivityStarted` at schedule time, so both clocks start
@@ -840,6 +843,59 @@ Successful responses preserve their headers and status. JSON response text is pa
its content type, matching the live AWS integration; other text remains a string. Non-2xx responses
raise catchable `ApiGateway.<status>` failures.
+## Amazon EMR tasks
+
+`arn:...:states:::elasticmapreduce:` serves the optimized EMR integrations the Step Functions guide
+lists, request-response for all seven and `.sync` for `createCluster`, `terminateCluster` and
+`addStep`. Each one calls Floci's EMR in the state machine's Region, under the guide's field names:
+
+| Resource | EMR calls | Result |
+| --- | --- | --- |
+| `createCluster` | `RunJobFlow` | `{ClusterId}` |
+| `createCluster.sync` | `RunJobFlow`, then `DescribeCluster` until `WAITING` | `Cluster`, then `ClusterId` |
+| `setClusterTerminationProtection` | `SetTerminationProtection`, `ClusterId` as `JobFlowIds` | `{}` |
+| `terminateCluster` | `TerminateJobFlows`, `ClusterId` as `JobFlowIds` | `{}` |
+| `terminateCluster.sync` | the same, then `DescribeCluster` until `TERMINATED` or `TERMINATED_WITH_ERRORS` | `Cluster` |
+| `addStep` | `AddJobFlowSteps` with the one `Step`, and `ExecutionRoleArn` when given | `{StepId}` |
+| `addStep.sync` | the same, then `DescribeStep` until `COMPLETED` | `Step`, then `StepId` |
+| `cancelStep` | `CancelSteps` with the one `StepId` | `{CancelStepsInfo}` |
+| `modifyInstanceFleetByName` | `ListInstanceFleets` for the fleet named `InstanceFleetName`, then `ModifyInstanceFleet` | `{}` |
+| `modifyInstanceGroupByName` | `ListInstanceGroups` for the group named `InstanceGroupName`, then `ModifyInstanceGroups` | `{}` |
+
+A `.sync` result is the describe call's response with the starting call's result after it. That is
+the `createCluster.sync` output recorded on AWS: `Cluster`, the SDK's `SdkHttpMetadata` and
+`SdkResponseMetadata`, then `ClusterId`. Floci leaves the SDK metadata out, as it does everywhere.
+State machines read an `addStep.sync` result's `Step` (`$.result.Step.Status.State`); its `StepId`,
+and `terminateCluster.sync`'s whole result, follow the same rule without a recording of their own.
+The timestamps inside are EMR's epoch seconds, which no recording confirms either.
+
+A `.sync` Task records `TaskSubmitted` with the starting call's result, then polls every 100 ms under
+the same two clocks as the other `.sync` integrations (see [Timeouts](#timeouts)). A step that ends
+`FAILED`, `CANCELLED` or `INTERRUPTED`, and a cluster that terminates before it reaches `WAITING`,
+fail the Task with `States.TaskFailed`, the describe response as the cause: a JSON string, which a
+`Catch` reads with `States.StringToJson` (for example its `Step.Status.State`). When
+the wait ends early, `addStep.sync` cancels its step with `CancelSteps`, as Step Functions "will make
+a best-effort attempt to cancel the task". A cluster is never terminated: "Step Functions does not
+terminate an Amazon EMR cluster automatically if execution is stopped." EMR in Floci reaches each
+state at once, so every wait ends at its first poll.
+
+An error from EMR fails the Task with the SDK exception class under the `EMR.` prefix:
+`EMR.InvalidRequestException` and `EMR.InternalServerException` for the two errors EMR models,
+`EMR.AmazonElasticMapReduceException` for any other, such as the `ValidationException`
+"Specified job flow ID does not exist." that `AddJobFlowSteps` answers for an unknown cluster. The cause is EMR's message; AWS
+also appends `(Service: AmazonElasticMapReduce; Status Code: 400; Error Code: ...; Request ID: ...)`.
+
+The integrations refuse what the guide rules out with `States.TaskFailed`, since no source gives
+AWS's error; the `createCluster` messages are the guide's own rules, the others Floci's words: a
+`createCluster` whose `Instances.KeepJobFlowAliveWhenNoSteps` is not `true` or that carries `Steps`,
+an `addStep` without a `Step`, an `InstanceGroupId` or `InstanceFleetId` in a ByName request, and a
+name that matches no group or fleet, or more than one. They also refuse a request without the
+integration's own fields, before EMR sees it: a `ClusterId` (every API but `createCluster`),
+`cancelStep`'s `StepId`, and a ByName request's name. A member EMR itself requires, such as
+`TerminationProtected`, is left to EMR, which answers its validation error.
+`.waitForTaskToken`, which AWS does not offer for EMR, and `.sync` on any other API fail as an
+unsupported resource.
+
## Configuration
| Variable | Default | Description |
diff --git a/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java b/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java
index d665f81ae1..7e1a2913ff 100644
--- a/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java
+++ b/src/main/java/io/github/hectorvent/floci/core/common/SmithyValidationErrors.java
@@ -26,6 +26,11 @@ public final class SmithyValidationErrors {
return count + String.join("; ", violations);
}
+ /** The violation for a required member the request leaves out. */
+ public static String notNull(String path) {
+ return "Value null at '" + path + "' failed to satisfy constraint: Member must not be null";
+ }
+
/** The 400 error carrying {@link #message} under the protocol's validation error code. */
public static AwsException exception(String errorCode, List<String> violations) {
return new AwsException(errorCode, message(violations), 400);
diff --git a/src/main/java/io/github/hectorvent/floci/services/emr/EmrHandler.java b/src/main/java/io/github/hectorvent/floci/services/emr/EmrHandler.java
index 55d90f28b2..3d847aefec 100644
--- a/src/main/java/io/github/hectorvent/floci/services/emr/EmrHandler.java
+++ b/src/main/java/io/github/hectorvent/floci/services/emr/EmrHandler.java
@@ -10,7 +10,9 @@ import io.github.hectorvent.floci.core.common.AwsException;
import io.github.hectorvent.floci.core.common.AwsJson11Controller;
import io.github.hectorvent.floci.core.common.AwsJsonTimestamp;
import io.github.hectorvent.floci.core.common.AwsRegions;
+import io.github.hectorvent.floci.core.common.JsonErrorResponseUtils;
import io.github.hectorvent.floci.core.common.PaginatedResult;
+import io.github.hectorvent.floci.core.common.SmithyValidationErrors;
import io.github.hectorvent.floci.services.emr.model.EmrBlockPublicAccess;
import io.github.hectorvent.floci.services.emr.model.EmrCluster;
import io.github.hectorvent.floci.services.emr.model.EmrInstanceFleet;
@@ -98,6 +100,10 @@ public class EmrHandler {
.build();
};
} catch (AwsException e) {
+ if (e.getExtendedData() != null) {
+ // Members an error adds to its message, such as InvalidRequestException's ErrorCode.
+ return JsonErrorResponseUtils.createErrorResponse(e);
+ }
return Response.status(e.getHttpStatus())
.entity(new AwsErrorResponse(e.jsonType(), e.getMessage()))
.build();
@@ -180,29 +186,34 @@ public class EmrHandler {
}
private Response handleTerminateJobFlows(JsonNode request) {
+ requireMembers(request, "JobFlowIds");
service.terminateJobFlows(stringList(request.path("JobFlowIds")));
return Response.ok(objectMapper.createObjectNode()).build();
}
private Response handleSetTerminationProtection(JsonNode request) {
+ requireMembers(request, "JobFlowIds", "TerminationProtected");
service.setTerminationProtection(stringList(request.path("JobFlowIds")),
request.path("TerminationProtected").asBoolean(false));
return Response.ok(objectMapper.createObjectNode()).build();
}
private Response handleSetVisibleToAllUsers(JsonNode request) {
+ requireMembers(request, "JobFlowIds", "VisibleToAllUsers");
service.setVisibleToAllUsers(stringList(request.path("JobFlowIds")),
request.path("VisibleToAllUsers").asBoolean(true));
return Response.ok(objectMapper.createObjectNode()).build();
}
private Response handleSetKeepAlive(JsonNode request) {
+ requireMembers(request, "JobFlowIds", "KeepJobFlowAliveWhenNoSteps");
service.setKeepJobFlowAliveWhenNoSteps(stringList(request.path("JobFlowIds")),
request.path("KeepJobFlowAliveWhenNoSteps").asBoolean(false));
return Response.ok(objectMapper.createObjectNode()).build();
}
private Response handleSetUnhealthyNodeReplacement(JsonNode request) {
+ requireMembers(request, "JobFlowIds", "UnhealthyNodeReplacement");
service.setUnhealthyNodeReplacement(stringList(request.path("JobFlowIds")),
request.path("UnhealthyNodeReplacement").asBoolean(false));
return Response.ok(objectMapper.createObjectNode()).build();
@@ -244,6 +255,7 @@ public class EmrHandler {
}
private Response handleCancelSteps(JsonNode request) {
+ requireMembers(request, "StepIds");
List<EmrService.CancelStepInfo> infos = service.cancelSteps(
text(request, "ClusterId"), stringList(request.path("StepIds")));
ObjectNode response = objectMapper.createObjectNode();
@@ -860,6 +872,25 @@ public class EmrHandler {
return values;
}
+ /**
+ * Refuses a request that leaves out members its model requires, which would otherwise read as an
+ * empty list or a default flag and succeed on nothing. The wording is AWS's framework validation, as
+ * for RunJobFlow's Name; an explicitly empty JobFlowIds is still a valid list.
+ */
+ private static void requireMembers(JsonNode request, String... members) {
+ List<String> violations = new ArrayList<>();
+ for (String member : members) {
+ JsonNode value = request.path(member);
+ if (value.isMissingNode() || value.isNull()) {
+ violations.add(SmithyValidationErrors.notNull(
+ Character.toLowerCase(member.charAt(0)) + member.substring(1)));
+ }
+ }
+ if (!violations.isEmpty()) {
+ throw SmithyValidationErrors.validationException(violations);
+ }
+ }
+
private String text(JsonNode request, String field) {
JsonNode node = request.path(field);
return node.isMissingNode() || node.isNull() ? null : node.asText(null);
diff --git a/src/main/java/io/github/hectorvent/floci/services/emr/EmrService.java b/src/main/java/io/github/hectorvent/floci/services/emr/EmrService.java
index 228654a558..7c0b2eb1f8 100644
--- a/src/main/java/io/github/hectorvent/floci/services/emr/EmrService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/emr/EmrService.java
@@ -45,6 +45,9 @@ public class EmrService {
private static final String UPPER_ALNUM = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
private static final SecureRandom RANDOM = new SecureRandom();
+ // AWS's answer to a change on a terminated cluster, for AddJobFlowSteps and ModifyInstanceFleet.
+ private static final String JOB_FLOW_SHUT_DOWN =
+ "A job flow that is shutting down, terminated, or finished may not be modified.";
private final StorageBackend<String, EmrCluster> clusterStore;
private final StorageBackend<String, SecurityConfiguration> secConfigStore;
@@ -81,9 +84,7 @@ public class EmrService {
// Only a missing Name fails AWS's framework validation: the member is REQUIRED but its
// constraint is len=[0..256], so an explicit empty string is accepted by real EMR.
if (cluster.getName() == null) {
- throw SmithyValidationErrors.validationException(
- "Value null at 'name' failed to satisfy constraint: "
- + "Member must not be null");
+ throw SmithyValidationErrors.validationException(SmithyValidationErrors.notNull("name"));
}
String id = "j-" + randomId(13);
cluster.setId(id);
@@ -212,10 +213,12 @@ public class EmrService {
// ──────────────────────────── Steps ────────────────────────────
public synchronized List<String> addJobFlowSteps(String clusterId, List<EmrStep> steps) {
- EmrCluster cluster = requireCluster(clusterId);
+ // AWS's own wording for an unknown job flow here, not DescribeCluster's: a botocore ClientError
+ // (raster-foundry/raster-foundry#352) and a CloudTrail record on AWS re:Post both show it.
+ EmrCluster cluster = clusterId == null ? requireCluster(null) : findCluster(clusterId).orElseThrow(
+ () -> new AwsException("ValidationException", "Specified job flow ID does not exist.", 400));
if ("TERMINATED".equals(cluster.getState())) {
- throw new AwsException("ValidationException",
- "A job flow that is shutting down, terminated, or finished may not be modified.", 400);
+ throw new AwsException("ValidationException", JOB_FLOW_SHUT_DOWN, 400);
}
List<String> ids = new ArrayList<>();
for (EmrStep step : steps) {
@@ -315,7 +318,12 @@ public class EmrService {
List<EmrInstanceGroup> groups = new ArrayList<>();
for (InstanceGroupModification mod : modifications) {
EmrCluster found = clusterId != null ? requireCluster(clusterId) : clusterOwning(region, mod.instanceGroupId());
- rejectTerminatedCluster(found);
+ // AWS's answer here, recorded by Terraform users
+ // (hashicorp/terraform-provider-aws#1355 and #9400).
+ if ("TERMINATED".equals(found.getState())) {
+ throw new AwsException("ValidationException",
+ "An instance group may only be modified when the cluster is running or waiting.", 400);
+ }
// One copy per cluster, so two groups of the same cluster are changed and stored together.
EmrCluster cluster = touched.computeIfAbsent(found.getId(), id -> found);
EmrInstanceGroup group = requireInstanceGroup(cluster, mod.instanceGroupId());
@@ -368,7 +376,12 @@ public class EmrService {
if (instanceFleetId == null) {
throw invalid("InstanceFleetId is required.");
}
- rejectTerminatedCluster(cluster);
+ // AWS's answer for a cluster terminated a while before, ErrorCode member included
+ // (hashicorp/terraform-provider-aws#31375, 2023).
+ if ("TERMINATED".equals(cluster.getState())) {
+ throw new AwsException("InvalidRequestException", JOB_FLOW_SHUT_DOWN, 400,
+ Map.of("ErrorCode", "INVALID_JOB_SHUT_DOWN"));
+ }
EmrInstanceFleet fleet = cluster.getInstanceFleets().stream()
.filter(f -> instanceFleetId.equals(f.getId()))
.findFirst()
diff --git a/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java b/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java
index c4d262b511..5012487395 100644
--- a/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java
+++ b/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java
@@ -183,8 +183,8 @@ public class AslExecutor {
private static final int INLINE_MAP_MAX_CONCURRENCY = 40;
private static final int DISTRIBUTED_MAP_MAX_CONCURRENCY = 10_000;
- // How often a .sync integration (ecs:runTask, states:startExecution) re-reads the job it waits
- // on. The wait itself is bounded by the Task's TimeoutSeconds and the execution's budget.
+ // How often a .sync integration (ecs:runTask, elasticmapreduce, states:startExecution) re-reads the job
+ // it waits on. The wait itself is bounded by the Task's TimeoutSeconds and the execution's budget.
private static final long SYNC_POLL_INTERVAL_MS = 100;
// AWS caps the string input of States.Base64Encode/Base64Decode/Hash at 10,000 characters
@@ -267,6 +267,7 @@ public class AslExecutor {
private final Instance<StepFunctionsService> sfnService;
private final WebClient webClient;
private final ApiGatewayTaskInvoker apiGatewayTaskInvoker;
+ private final EmrTaskInvoker emrTaskInvoker;
private final EmulatorConfig config;
private final NestedTaskCredentials nestedTaskCredentials;
private final CustomResourceLiveness customResourceLiveness;
@@ -297,14 +298,15 @@ public class AslExecutor {
Instance<StepFunctionsService> sfnService, EmulatorConfig config, Vertx vertx,
CustomResourceLiveness customResourceLiveness, RedshiftDataJsonHandler redshiftDataHandler,
ServiceRegistry serviceRegistry, NestedTaskCredentials nestedTaskCredentials,
- LambdaTaskAuthorizer lambdaTaskAuthorizer, ApiGatewayTaskInvoker apiGatewayTaskInvoker) {
+ LambdaTaskAuthorizer lambdaTaskAuthorizer, ApiGatewayTaskInvoker apiGatewayTaskInvoker,
+ EmrTaskInvoker emrTaskInvoker) {
this(lambdaExecutor, targetResolver, dynamoDb, dynamoDbJsonHandler,
sqsJsonHandler, snsJsonHandler, cloudFormationHandler,
ec2Service, s3Service, ecsService, ecsJsonHandler,
eventBridgeHandler, schedulerService, schedulerController, rdsDataService,
objectMapper, jsonataEvaluator, sfnService, config, vertx, customResourceLiveness,
Clock.systemUTC(), TimeUnit.NANOSECONDS::sleep, null, redshiftDataHandler, serviceRegistry,
- nestedTaskCredentials, lambdaTaskAuthorizer, null, null, apiGatewayTaskInvoker);
+ nestedTaskCredentials, lambdaTaskAuthorizer, null, null, apiGatewayTaskInvoker, emrTaskInvoker);
}
AslExecutor(LambdaExecutorService lambdaExecutor, LambdaTargetResolver targetResolver,
@@ -322,8 +324,9 @@ public class AslExecutor {
RedshiftDataJsonHandler redshiftDataHandler, ServiceRegistry serviceRegistry,
NestedTaskCredentials nestedTaskCredentials, LambdaTaskAuthorizer lambdaTaskAuthorizer,
Duration expressMaxExecutionTimeOverride, LongSupplier nanoTimeOverride,
- ApiGatewayTaskInvoker apiGatewayTaskInvoker) {
+ ApiGatewayTaskInvoker apiGatewayTaskInvoker, EmrTaskInvoker emrTaskInvoker) {
this.apiGatewayTaskInvoker = apiGatewayTaskInvoker;
+ this.emrTaskInvoker = emrTaskInvoker;
this.nestedTaskCredentials = nestedTaskCredentials;
this.serviceRegistry = serviceRegistry;
this.lambdaTaskAuthorizer = lambdaTaskAuthorizer;
@@ -386,7 +389,7 @@ public class AslExecutor {
ec2Service, s3Service, ecsService, ecsJsonHandler,
eventBridgeHandler, schedulerService, schedulerController, rdsDataService,
objectMapper, jsonataEvaluator, sfnService, config, vertx, customResourceLiveness,
- Clock.systemUTC(), TimeUnit.NANOSECONDS::sleep, null, null, null, null, null, null, null, null);
+ Clock.systemUTC(), TimeUnit.NANOSECONDS::sleep, null, null, null, null, null, null, null, null, null);
}
// Kept for tests that do not exercise the Redshift Data API integration.
@@ -407,7 +410,7 @@ public class AslExecutor {
ec2Service, s3Service, ecsService, ecsJsonHandler,
eventBridgeHandler, schedulerService, schedulerController, rdsDataService,
objectMapper, jsonataEvaluator, sfnService, config, vertx, customResourceLiveness,
- clock, sleeper, maxWaitSecondsOverride, null, null, null, null, null, null, null);
+ clock, sleeper, maxWaitSecondsOverride, null, null, null, null, null, null, null, null);
}
AslExecutor(LambdaExecutorService lambdaExecutor, LambdaFunctionStore functionStore,
@@ -1444,6 +1447,14 @@ public class AslExecutor {
jobSubmitted);
}
+ // EMR optimized integrations: request-response and .sync. AWS supports no .waitForTaskToken
+ // variant, so one carrying a token falls through to Unsupported resource.
+ if (integration.isOptimizedService("elasticmapreduce") && taskToken == null) {
+ String region = extractRegionFromArn(sm.getStateMachineArn());
+ return invokeEmr(integration, input, region, executionArn, executionDeadlineNanos, taskDeadlineNanos,
+ jobSubmitted);
+ }
+
// AWS SDK service integrations: Step Functions
if (integration.isSdkService("sfn")) {
String region = extractRegionFromArn(sm.getStateMachineArn());
@@ -2193,6 +2204,43 @@ public class AslExecutor {
}
}
+ /**
+ * The {@code elasticmapreduce:} integrations. A {@code .sync} Task records its call's response as
+ * submitted and polls the cluster or step as {@code ecs:runTask.sync} polls its tasks, bounded by
+ * the same two clocks; whatever ends the wait first abandons the job.
+ */
+ private JsonNode invokeEmr(StatesIntegration integration, JsonNode input, String region, String executionArn,
+ long executionDeadlineNanos, long taskDeadlineNanos,
+ Consumer<JsonNode> jobSubmitted) throws Exception {
+ EmrTaskInvoker.Job job = emrTaskInvoker.start(integration, input, region);
+ if (integration.suffix().isEmpty()) {
+ return job.response();
+ }
+ try {
+ jobSubmitted.accept(job.response());
+ } catch (RuntimeException e) {
+ emrTaskInvoker.abandon(job, region);
+ throw e;
+ }
+ while (true) {
+ try {
+ sleepOrTimeOutTask(TimeUnit.MILLISECONDS.toNanos(SYNC_POLL_INTERVAL_MS),
+ executionDeadlineNanos, taskDeadlineNanos);
+ } catch (TaskTimedOutException | ExecutionTimedOutException | InterruptedException e) {
+ emrTaskInvoker.abandon(job, region);
+ throw e;
+ }
+ if (abortedByCaller(executionArn)) {
+ emrTaskInvoker.abandon(job, region);
+ throw new ExecutionAbortedException();
+ }
+ Optional<JsonNode> result = emrTaskInvoker.poll(job, region);
+ if (result.isPresent()) {
+ return result.get();
+ }
+ }
+ }
+
/**
* The reason AWS writes on a job it stops because the {@code .sync} Task waiting on it ended
* first, whichever way it ended: verbatim, as the ECS task's {@code stoppedReason} and the child
diff --git a/src/main/java/io/github/hectorvent/floci/services/stepfunctions/EmrTaskInvoker.java b/src/main/java/io/github/hectorvent/floci/services/stepfunctions/EmrTaskInvoker.java
new file mode 100644
index 0000000000..0449bf624d
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/stepfunctions/EmrTaskInvoker.java
@@ -0,0 +1,258 @@
+package io.github.hectorvent.floci.services.stepfunctions;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.core.common.RequestScopes;
+import io.github.hectorvent.floci.services.emr.EmrHandler;
+import jakarta.enterprise.context.ApplicationScoped;
+import jakarta.inject.Inject;
+import jakarta.ws.rs.core.Response;
+import org.jboss.logging.Logger;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Optional;
+import java.util.Set;
+
+/**
+ * The optimized {@code arn:<partition>:states:::elasticmapreduce:*} integrations. Each one is an EMR
+ * API under renamed fields, as the Step Functions guide's "Optimized Amazon EMR APIs" table lists
+ * them, run through {@link EmrHandler} in the state machine's Region. A {@code .sync} Task calls
+ * {@link #start}, then {@link #poll} until its cluster or step is done; {@link AslExecutor} owns the
+ * wait between polls.
+ */
+@ApplicationScoped
+public class EmrTaskInvoker {
+
+ private static final Logger LOG = Logger.getLogger(EmrTaskInvoker.class);
+
+ // A failure is named after the SDK exception class: the errors EMR models keep their own, and any
+ // other code is the base class. AWS re:Post, "AWS Step Function does not recognize job flow ID":
+ // a ValidationException fails addStep.sync as EMR.AmazonElasticMapReduceException.
+ private static final Set<String> MODELED_ERRORS = Set.of("InvalidRequestException", "InternalServerException");
+ private static final String BASE_ERROR = "AmazonElasticMapReduceException";
+ private static final Set<String> APIS = Set.of("createCluster", "setClusterTerminationProtection",
+ "terminateCluster", "addStep", "cancelStep", "modifyInstanceFleetByName", "modifyInstanceGroupByName");
+ private static final Set<String> SYNC_APIS = Set.of("createCluster", "terminateCluster", "addStep");
+ private static final Set<String> TERMINATED_STATES = Set.of("TERMINATED", "TERMINATED_WITH_ERRORS");
+ private static final Set<String> FAILED_STEP_STATES = Set.of("FAILED", "CANCELLED", "INTERRUPTED");
+
+ private final EmrHandler emr;
+ private final ObjectMapper objectMapper;
+
+ @Inject
+ public EmrTaskInvoker(EmrHandler emr, ObjectMapper objectMapper) {
+ this.emr = emr;
+ this.objectMapper = objectMapper;
+ }
+
+ /**
+ * A started call: the integration's response, and the cluster, or step, a {@code .sync} Task
+ * waits on.
+ */
+ record Job(String api, String clusterId, String stepId, ObjectNode response) {}
+
+ Job start(StatesIntegration integration, JsonNode input, String region) {
+ String api = integration.api();
+ if (!APIS.contains(api)
+ || !integration.suffix().isEmpty() && !(".sync".equals(integration.suffix()) && SYNC_APIS.contains(api))) {
+ throw unsupported(integration);
+ }
+ JsonNode request = input != null && input.isObject() ? input : objectMapper.createObjectNode();
+ // The integration's own fields, named as the Task wrote them: renamed into JobFlowIds or StepIds,
+ // a missing one would reach EMR as a "null" id.
+ String clusterId = "createCluster".equals(api) ? null : required(request, "ClusterId");
+ return switch (api) {
+ case "createCluster" -> createCluster(request, region);
+ case "setClusterTerminationProtection" -> new Job(api, clusterId, null,
+ call("SetTerminationProtection", withJobFlowIds(request), region));
+ case "terminateCluster" -> new Job(api, clusterId, null,
+ call("TerminateJobFlows", withJobFlowIds(request), region));
+ case "addStep" -> addStep(request, clusterId, region);
+ case "cancelStep" -> cancelStep(request, clusterId, region);
+ case "modifyInstanceFleetByName" -> modifyInstanceFleetByName(request, clusterId, region);
+ case "modifyInstanceGroupByName" -> modifyInstanceGroupByName(request, clusterId, region);
+ default -> throw unsupported(integration);
+ };
+ }
+
+ /**
+ * The Task's result once the job is done, empty while it runs. The result is the describe call's
+ * response with the starting call's merged in: a {@code createCluster.sync} output recorded on
+ * AWS is {@code Cluster}, the SDK metadata, then {@code ClusterId}.
+ */
+ Optional<JsonNode> poll(Job job, String region) {
+ if ("addStep".equals(job.api())) {
+ ObjectNode request = clusterRequest(job.clusterId());
+ request.put("StepId", job.stepId());
+ JsonNode described = call("DescribeStep", request, region);
+ String state = described.path("Step").path("Status").path("State").asText();
+ if (FAILED_STEP_STATES.contains(state)) {
+ throw failed(described);
+ }
+ return "COMPLETED".equals(state) ? Optional.of(merged(described, job.response())) : Optional.empty();
+ }
+ JsonNode described = call("DescribeCluster", clusterRequest(job.clusterId()), region);
+ String state = described.path("Cluster").path("Status").path("State").asText();
+ if ("terminateCluster".equals(job.api())) {
+ return TERMINATED_STATES.contains(state)
+ ? Optional.of(merged(described, job.response())) : Optional.empty();
+ }
+ if (TERMINATED_STATES.contains(state)) {
+ throw failed(described);
+ }
+ return "WAITING".equals(state) ? Optional.of(merged(described, job.response())) : Optional.empty();
+ }
+
+ /**
+ * Best effort, for a {@code .sync} Task that ends before its job does: Step Functions "will make a
+ * best-effort attempt to cancel the task", and addStep's IAM policy grants it CancelSteps. A cluster
+ * is left running, since "Step Functions does not terminate an Amazon EMR cluster automatically if
+ * execution is stopped."
+ */
+ void abandon(Job job, String region) {
+ if (!"addStep".equals(job.api())) {
+ return;
+ }
+ ObjectNode request = clusterRequest(job.clusterId());
+ request.putArray("StepIds").add(job.stepId());
+ try {
+ call("CancelSteps", request, region);
+ } catch (RuntimeException e) {
+ LOG.warnv("elasticmapreduce:addStep.sync ended before its step {0} did, and the step could not be "
+ + "cancelled: {1}", job.stepId(), e.getMessage());
+ }
+ }
+
+ private Job createCluster(JsonNode request, String region) {
+ // The two differences the guide lists from RunJobFlow. No source gives the error AWS answers either with.
+ if (!request.path("Instances").path("KeepJobFlowAliveWhenNoSteps").booleanValue()) {
+ throw refused("The field Instances.KeepJobFlowAliveWhenNoSteps is mandatory, "
+ + "and must have the Boolean value TRUE.");
+ }
+ if (request.has("Steps")) {
+ throw refused("The field Steps is not allowed.");
+ }
+ String clusterId = call("RunJobFlow", request, region).path("JobFlowId").asText();
+ return new Job("createCluster", clusterId, null, objectMapper.createObjectNode().put("ClusterId", clusterId));
+ }
+
+ private Job addStep(JsonNode request, String clusterId, String region) {
+ if (!request.path("Step").isObject()) {
+ throw refused("The field Step is required.");
+ }
+ ObjectNode add = request.deepCopy();
+ add.remove(List.of("ClusterId", "Step"));
+ add.put("JobFlowId", clusterId);
+ add.putArray("Steps").add(request.get("Step"));
+ String stepId = call("AddJobFlowSteps", add, region).path("StepIds").path(0).asText();
+ return new Job("addStep", clusterId, stepId, objectMapper.createObjectNode().put("StepId", stepId));
+ }
+
+ private Job cancelStep(JsonNode request, String clusterId, String region) {
+ String stepId = required(request, "StepId");
+ ObjectNode cancel = request.deepCopy();
+ cancel.remove("StepId");
+ cancel.putArray("StepIds").add(stepId);
+ JsonNode info = call("CancelSteps", cancel, region).path("CancelStepsInfoList").path(0);
+ ObjectNode response = objectMapper.createObjectNode();
+ response.set("CancelStepsInfo", info.isMissingNode() ? objectMapper.createObjectNode() : info);
+ return new Job("cancelStep", clusterId, null, response);
+ }
+
+ private Job modifyInstanceFleetByName(JsonNode request, String clusterId, String region) {
+ JsonNode fleet = request.path("InstanceFleet");
+ if (fleet.has("InstanceFleetId")) {
+ throw refused("The field InstanceFleet.InstanceFleetId is not allowed.");
+ }
+ String name = required(request, "InstanceFleetName");
+ JsonNode fleets = call("ListInstanceFleets", clusterRequest(clusterId), region).path("InstanceFleets");
+ ObjectNode modification = fleet.isObject() ? fleet.deepCopy() : objectMapper.createObjectNode();
+ modification.put("InstanceFleetId", idOfNamed(fleets, name, "instance fleet"));
+ ObjectNode modify = clusterRequest(clusterId);
+ modify.set("InstanceFleet", modification);
+ return new Job("modifyInstanceFleetByName", clusterId, null, call("ModifyInstanceFleet", modify, region));
+ }
+
+ private Job modifyInstanceGroupByName(JsonNode request, String clusterId, String region) {
+ JsonNode group = request.path("InstanceGroup");
+ if (group.has("InstanceGroupId")) {
+ throw refused("The field InstanceGroup.InstanceGroupId is not allowed.");
+ }
+ String name = required(request, "InstanceGroupName");
+ JsonNode groups = call("ListInstanceGroups", clusterRequest(clusterId), region).path("InstanceGroups");
+ ObjectNode modification = group.isObject() ? group.deepCopy() : objectMapper.createObjectNode();
+ modification.put("InstanceGroupId", idOfNamed(groups, name, "instance group"));
+ ObjectNode modify = clusterRequest(clusterId);
+ modify.putArray("InstanceGroups").add(modification);
+ return new Job("modifyInstanceGroupByName", clusterId, null, call("ModifyInstanceGroups", modify, region));
+ }
+
+ /** The guide asks for unique names whenever a ByName API is used; the lookup reads ListInstance*. */
+ private static String idOfNamed(JsonNode members, String name, String kind) {
+ List<String> ids = new ArrayList<>();
+ for (JsonNode member : members) {
+ if (name.equals(member.path("Name").asText(null))) {
+ ids.add(member.path("Id").asText());
+ }
+ }
+ if (ids.size() == 1) {
+ return ids.get(0);
+ }
+ throw refused(ids.isEmpty()
+ ? "The cluster has no " + kind + " named '" + name + "'."
+ : "The cluster has " + ids.size() + " " + kind + "s named '" + name + "'.");
+ }
+
+ /** The request with {@code ClusterId} renamed to EMR's {@code JobFlowIds}. */
+ private ObjectNode withJobFlowIds(JsonNode request) {
+ ObjectNode renamed = request.deepCopy();
+ renamed.putArray("JobFlowIds").add(renamed.remove("ClusterId"));
+ return renamed;
+ }
+
+ private ObjectNode clusterRequest(String clusterId) {
+ return objectMapper.createObjectNode().put("ClusterId", clusterId);
+ }
+
+ private ObjectNode merged(JsonNode described, ObjectNode response) {
+ ObjectNode result = described.deepCopy();
+ result.setAll(response);
+ return result;
+ }
+
+ /** Runs in the state machine's Region, which EMR reads to find its clusters. */
+ private ObjectNode call(String action, JsonNode request, String region) {
+ Response response = RequestScopes.callAs(null, region, () -> emr.handle(action, request, region));
+ if (response.getStatus() >= 400) {
+ JsonNode error = objectMapper.valueToTree(response.getEntity());
+ String type = error.path("__type").asText();
+ String name = MODELED_ERRORS.contains(type) ? type : BASE_ERROR;
+ throw new AslExecutor.FailStateException("EMR." + name, error.path("message").asText());
+ }
+ return response.getEntity() instanceof ObjectNode body ? body : objectMapper.createObjectNode();
+ }
+
+ /** A job that ended without succeeding: the cause is its describe response, as a JSON string. */
+ private static AslExecutor.FailStateException failed(JsonNode described) {
+ return new AslExecutor.FailStateException("States.TaskFailed", described.toString());
+ }
+
+ /** A field the integration's request names a cluster, step or member by, as a non-blank string. */
+ private static String required(JsonNode request, String field) {
+ JsonNode value = request.path(field);
+ if (!value.isTextual() || value.asText().isBlank()) {
+ throw refused("The field " + field + " is required.");
+ }
+ return value.asText();
+ }
+
+ private static AslExecutor.FailStateException refused(String cause) {
+ return new AslExecutor.FailStateException("States.TaskFailed", cause);
+ }
+
+ private static AslExecutor.FailStateException unsupported(StatesIntegration integration) {
+ return new AslExecutor.FailStateException("States.TaskFailed", "Unsupported resource: " + integration);
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/emr/EmrClusterRegionIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/emr/EmrClusterRegionIntegrationTest.java
index 6fbf8895d8..4ebeff695a 100644
--- a/src/test/java/io/github/hectorvent/floci/services/emr/EmrClusterRegionIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/emr/EmrClusterRegionIntegrationTest.java
@@ -42,7 +42,8 @@ class EmrClusterRegionIntegrationTest {
emr(OTHER, "AddJobFlowSteps", "{\"JobFlowId\":\"" + clusterId + "\",\"Steps\":[{\"Name\":\"s\","
+ "\"HadoopJarStep\":{\"Jar\":\"command-runner.jar\",\"Args\":[\"echo\"]}}]}")
.then().statusCode(400)
- .body("__type", equalTo("InvalidRequestException"));
+ .body("__type", equalTo("ValidationException"))
+ .body("message", equalTo("Specified job flow ID does not exist."));
emr(OTHER, "ListClusters", "{}")
.then().statusCode(200)
.body("Clusters.Id", not(hasItem(clusterId)));
diff --git a/src/test/java/io/github/hectorvent/floci/services/emr/EmrIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/emr/EmrIntegrationTest.java
index 7a0a017529..f8380c9cd5 100644
--- a/src/test/java/io/github/hectorvent/floci/services/emr/EmrIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/emr/EmrIntegrationTest.java
@@ -366,13 +366,13 @@ class EmrIntegrationTest {
@Test
@Order(17)
- void addStepsToUnknownClusterIsInvalidRequest() {
+ void addStepsToUnknownClusterIsAValidationException() {
emr("AddJobFlowSteps",
"{\"JobFlowId\":\"j-DOESNOTEXIST0\",\"Steps\":[{\"Name\":\"s\","
+ "\"HadoopJarStep\":{\"Jar\":\"command-runner.jar\"}}]}")
.then().statusCode(400)
- .body("__type", equalTo("InvalidRequestException"))
- .body("message", equalTo("Cluster id 'j-DOESNOTEXIST0' is not valid."));
+ .body("__type", equalTo("ValidationException"))
+ .body("message", equalTo("Specified job flow ID does not exist."));
}
@Test
@@ -404,6 +404,40 @@ class EmrIntegrationTest {
.body("Cluster.Status.State", equalTo("WAITING"));
}
+ @Test
+ void settersAndCancelStepsRefuseARequestWithoutTheMembersTheyRequire() {
+ String id = emr("RunJobFlow", "{\"Name\":\"required\",\"Instances\":{\"KeepJobFlowAliveWhenNoSteps\":true}}")
+ .then().statusCode(200).extract().path("JobFlowId");
+ String ids = "\"JobFlowIds\":[\"" + id + "\"]";
+
+ assertMissing("TerminateJobFlows", "{}", "jobFlowIds");
+ assertMissing("SetTerminationProtection", "{\"TerminationProtected\":true}", "jobFlowIds");
+ assertMissing("SetTerminationProtection", "{" + ids + "}", "terminationProtected");
+ assertMissing("SetVisibleToAllUsers", "{" + ids + "}", "visibleToAllUsers");
+ assertMissing("SetKeepJobFlowAliveWhenNoSteps", "{" + ids + "}", "keepJobFlowAliveWhenNoSteps");
+ assertMissing("SetUnhealthyNodeReplacement", "{" + ids + "}", "unhealthyNodeReplacement");
+ assertMissing("CancelSteps", "{\"ClusterId\":\"" + id + "\"}", "stepIds");
+ emr("SetVisibleToAllUsers", "{}").then().statusCode(400)
+ .body("message", equalTo("2 validation errors detected: "
+ + "Value null at 'jobFlowIds' failed to satisfy constraint: Member must not be null; "
+ + "Value null at 'visibleToAllUsers' failed to satisfy constraint: Member must not be null"));
+ emr("TerminateJobFlows", "{\"JobFlowIds\":[]}").then().statusCode(200);
+
+ emr("DescribeCluster", "{\"ClusterId\":\"" + id + "\"}").then().statusCode(200)
+ .body("Cluster.Status.State", equalTo("WAITING"))
+ .body("Cluster.TerminationProtected", equalTo(false))
+ .body("Cluster.VisibleToAllUsers", equalTo(true))
+ .body("Cluster.AutoTerminate", equalTo(false));
+ emr("TerminateJobFlows", "{" + ids + "}").then().statusCode(200);
+ }
+
+ private static void assertMissing(String action, String body, String member) {
+ emr(action, body).then().statusCode(400)
+ .body("__type", equalTo("ValidationException"))
+ .body("message", equalTo("1 validation error detected: Value null at '" + member
+ + "' failed to satisfy constraint: Member must not be null"));
+ }
+
// Catches: a cluster stored before the DNS fix keeps reporting its old us-east-1 master name.
@Test
void storedClusterReportsTheRegionalMasterDnsName() {
diff --git a/src/test/java/io/github/hectorvent/floci/services/emr/EmrTerminatedCapacityIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/emr/EmrTerminatedCapacityIntegrationTest.java
index 7a40269026..a1083c228a 100644
--- a/src/test/java/io/github/hectorvent/floci/services/emr/EmrTerminatedCapacityIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/emr/EmrTerminatedCapacityIntegrationTest.java
@@ -31,6 +31,10 @@ class EmrTerminatedCapacityIntegrationTest {
+ "\"TargetSpotCapacity\":3,\"InstanceTypeConfigs\":[{\"InstanceType\":\"m5.xlarge\"}]}]";
private static final String STEP = "{\"Name\":\"complete\","
+ "\"HadoopJarStep\":{\"Jar\":\"command-runner.jar\"}}";
+ private static final String GROUP_REFUSAL =
+ "An instance group may only be modified when the cluster is running or waiting.";
+ private static final String FLEET_REFUSAL =
+ "A job flow that is shutting down, terminated, or finished may not be modified.";
@BeforeAll
static void configure() {
@@ -133,7 +137,9 @@ class EmrTerminatedCapacityIntegrationTest {
emr("ModifyInstanceGroups", "{\"InstanceGroups\":["
+ "{\"InstanceGroupId\":\"" + activeGroup + "\",\"InstanceCount\":4},"
+ "{\"InstanceGroupId\":\"" + terminatedGroup + "\",\"InstanceCount\":4}]}")
- .then().statusCode(400).body("__type", equalTo("InvalidRequestException"));
+ .then().statusCode(400)
+ .body("__type", equalTo("ValidationException"))
+ .body("message", equalTo(GROUP_REFUSAL));
emr("ListInstanceGroups", clusterRequest(activeId)).then().statusCode(200)
.body("InstanceGroups.find { it.Id == '" + activeGroup + "' }.RunningInstanceCount", equalTo(2));
assertTerminatedGroup(terminatedId);
@@ -172,7 +178,9 @@ class EmrTerminatedCapacityIntegrationTest {
String clusterId = includeClusterId ? "\"ClusterId\":\"" + id + "\"," : "";
emr("ModifyInstanceGroups", "{" + clusterId + "\"InstanceGroups\":[{\"InstanceGroupId\":\""
+ coreGroupId(id) + "\",\"InstanceCount\":4}]}")
- .then().statusCode(400).body("__type", equalTo("InvalidRequestException"));
+ .then().statusCode(400)
+ .body("__type", equalTo("ValidationException"))
+ .body("message", equalTo(GROUP_REFUSAL));
assertTerminatedGroup(id);
}
@@ -181,7 +189,10 @@ class EmrTerminatedCapacityIntegrationTest {
.extract().jsonPath().getString("InstanceFleets.find { it.InstanceFleetType == 'CORE' }.Id");
emr("ModifyInstanceFleet", "{\"ClusterId\":\"" + id + "\",\"InstanceFleet\":{"
+ "\"InstanceFleetId\":\"" + fleetId + "\",\"TargetOnDemandCapacity\":4}}")
- .then().statusCode(400).body("__type", equalTo("InvalidRequestException"));
+ .then().statusCode(400)
+ .body("__type", equalTo("InvalidRequestException"))
+ .body("message", equalTo(FLEET_REFUSAL))
+ .body("ErrorCode", equalTo("INVALID_JOB_SHUT_DOWN"));
assertTerminatedFleet(id);
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEmrSyncTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEmrSyncTest.java
new file mode 100644
index 0000000000..b4846bdfeb
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEmrSyncTest.java
@@ -0,0 +1,208 @@
+package io.github.hectorvent.floci.services.stepfunctions;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.github.hectorvent.floci.services.emr.EmrHandler;
+import io.github.hectorvent.floci.services.emr.EmrService;
+import io.github.hectorvent.floci.services.emr.model.EmrCluster;
+import io.github.hectorvent.floci.services.emr.model.EmrStep;
+import io.github.hectorvent.floci.services.stepfunctions.model.Execution;
+import io.github.hectorvent.floci.services.stepfunctions.model.HistoryEvent;
+import io.github.hectorvent.floci.services.stepfunctions.model.StateMachine;
+import jakarta.enterprise.inject.Instance;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicLong;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.anyString;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+/**
+ * The {@code .sync} waits of the {@code elasticmapreduce:} integrations, on a cluster or step held in
+ * a state Floci's instant lifecycle never shows. Time is a test clock the waits move forward.
+ */
+class AslExecutorEmrSyncTest {
+
+ private static final String REGION = "eu-west-3";
+ private static final String ACCOUNT = "000000000000";
+ private static final String CLUSTER_ID = "j-SYNCTEST00001";
+ private static final String STEP_ID = "s-SYNCTEST00001";
+ private static final String ADD_STEP = """
+ {"ClusterId": "j-SYNCTEST00001",
+ "Step": {"Name": "job", "HadoopJarStep": {"Jar": "command-runner.jar", "Args": ["true"]}}}
+ """;
+ private static final String CREATE_CLUSTER = """
+ {"Name": "sync", "Instances": {"KeepJobFlowAliveWhenNoSteps": true}}
+ """;
+
+ private final ObjectMapper objectMapper = new ObjectMapper();
+ private final AtomicLong testNanos = new AtomicLong();
+ private EmrService emrService;
+ private StepFunctionsService sfnService;
+ private AslExecutor executor;
+ private List<HistoryEvent> history;
+
+ @BeforeEach
+ @SuppressWarnings("unchecked")
+ void setUp() {
+ emrService = mock(EmrService.class);
+ when(emrService.addJobFlowSteps(eq(CLUSTER_ID), any())).thenReturn(List.of(STEP_ID));
+ when(emrService.runJobFlow(any(), anyString())).thenAnswer(invocation -> {
+ EmrCluster cluster = invocation.getArgument(0);
+ cluster.setId(CLUSTER_ID);
+ return cluster;
+ });
+ sfnService = mock(StepFunctionsService.class);
+ when(sfnService.describeExecution(any())).thenAnswer(invocation -> parent(invocation.getArgument(0), "RUNNING"));
+ Instance<StepFunctionsService> instance = mock(Instance.class);
+ when(instance.get()).thenReturn(sfnService);
+ executor = AslExecutorFixture.builder()
+ .objectMapper(objectMapper)
+ .sfnService(instance)
+ .customResourceLiveness(null)
+ .emrTaskInvoker(new EmrTaskInvoker(new EmrHandler(emrService, objectMapper), objectMapper))
+ .nanoTime(testNanos::get)
+ .sleeper(testNanos::addAndGet)
+ .build();
+ }
+
+ @Test
+ void addStepSyncCancelsItsStepWhenTheTaskTimesOut() {
+ when(emrService.describeStep(CLUSTER_ID, STEP_ID)).thenReturn(step("RUNNING"));
+
+ Execution execution = run("""
+ {"StartAt": "Add", "States": {"Add": {"Type": "Task",
+ "Resource": "arn:aws:states:::elasticmapreduce:addStep.sync", "TimeoutSeconds": 1, "End": true}}}
+ """, ADD_STEP);
+
+ assertEquals("FAILED", execution.getStatus());
+ assertEquals("States.Timeout", execution.getError());
+ verify(emrService).cancelSteps(CLUSTER_ID, List.of(STEP_ID));
+ }
+
+ @Test
+ void addStepSyncFailsWithTheStepWhenTheStepFails() throws Exception {
+ EmrStep failed = step("FAILED");
+ failed.setStateChangeReason("Step failed");
+ when(emrService.describeStep(CLUSTER_ID, STEP_ID)).thenReturn(failed);
+
+ Execution execution = run(task("addStep.sync"), ADD_STEP);
+
+ assertEquals("FAILED", execution.getStatus());
+ assertEquals("States.TaskFailed", execution.getError());
+ JsonNode cause = objectMapper.readTree(execution.getCause());
+ assertEquals(STEP_ID, cause.path("Step").path("Id").asText());
+ assertEquals("FAILED", cause.path("Step").path("Status").path("State").asText());
+ verify(emrService, never()).cancelSteps(any(), any());
+ }
+
+ @Test
+ void addStepSyncWaitsUntilTheStepCompletes() throws Exception {
+ AtomicInteger polls = new AtomicInteger();
+ when(emrService.describeStep(CLUSTER_ID, STEP_ID))
+ .thenAnswer(invocation -> step(polls.incrementAndGet() < 3 ? "RUNNING" : "COMPLETED"));
+
+ Execution execution = run(task("addStep.sync"), ADD_STEP);
+
+ assertEquals("SUCCEEDED", execution.getStatus(), execution.getCause());
+ assertEquals(3, polls.get());
+ JsonNode output = objectMapper.readTree(execution.getOutput());
+ assertEquals(List.of("Step", "StepId"), fieldNames(output));
+ assertEquals("COMPLETED", output.path("Step").path("Status").path("State").asText());
+ assertEquals(STEP_ID, output.path("StepId").asText());
+ }
+
+ @Test
+ void createClusterSyncFailsWhenTheClusterTerminatesInsteadOfWaiting() throws Exception {
+ when(emrService.describeCluster(CLUSTER_ID)).thenReturn(cluster("TERMINATED_WITH_ERRORS"));
+
+ Execution execution = run(task("createCluster.sync"), CREATE_CLUSTER);
+
+ assertEquals("FAILED", execution.getStatus());
+ assertEquals("States.TaskFailed", execution.getError());
+ JsonNode cause = objectMapper.readTree(execution.getCause());
+ assertEquals("TERMINATED_WITH_ERRORS", cause.path("Cluster").path("Status").path("State").asText());
+ }
+
+ @Test
+ void stopExecutionLeavesTheClusterOfACreateClusterSyncRunning() {
+ when(emrService.describeCluster(CLUSTER_ID)).thenReturn(cluster("STARTING"));
+ AtomicInteger parentReads = new AtomicInteger();
+ when(sfnService.describeExecution(any())).thenAnswer(invocation ->
+ parent(invocation.getArgument(0), parentReads.incrementAndGet() < 3 ? "RUNNING" : "ABORTED"));
+
+ Execution execution = run(task("createCluster.sync"), CREATE_CLUSTER);
+
+ assertNull(execution.getError());
+ verify(emrService, never()).terminateJobFlows(any());
+ assertEquals(List.of(), history.stream().map(HistoryEvent::getType)
+ .filter(type -> type.equals("TaskFailed") || type.equals("TaskSucceeded")).toList());
+ }
+
+ private static String task(String api) {
+ return """
+ {"StartAt": "Emr", "States": {"Emr": {"Type": "Task",
+ "Resource": "arn:aws:states:::elasticmapreduce:%s", "End": true}}}
+ """.formatted(api);
+ }
+
+ private static EmrStep step(String state) {
+ EmrStep step = new EmrStep();
+ step.setId(STEP_ID);
+ step.setName("job");
+ step.setActionOnFailure("CONTINUE");
+ step.setState(state);
+ return step;
+ }
+
+ private static EmrCluster cluster(String state) {
+ EmrCluster cluster = new EmrCluster();
+ cluster.setId(CLUSTER_ID);
+ cluster.setName("sync");
+ cluster.setState(state);
+ return cluster;
+ }
+
+ private static Execution parent(String executionArn, String status) {
+ Execution parent = new Execution();
+ parent.setExecutionArn(executionArn);
+ parent.setStatus(status);
+ return parent;
+ }
+
+ private static List<String> fieldNames(JsonNode node) {
+ List<String> names = new ArrayList<>();
+ node.fieldNames().forEachRemaining(names::add);
+ return names;
+ }
+
+ private Execution run(String definition, String input) {
+ StateMachine stateMachine = new StateMachine();
+ stateMachine.setName("emr-sync-test");
+ stateMachine.setStateMachineArn("arn:aws:states:%s:%s:stateMachine:emr-sync-test".formatted(REGION, ACCOUNT));
+ stateMachine.setRoleArn("arn:aws:iam::%s:role/test-role".formatted(ACCOUNT));
+ stateMachine.setDefinition(definition);
+
+ Execution execution = new Execution();
+ execution.setName("emr-sync-execution");
+ execution.setExecutionArn("arn:aws:states:%s:%s:execution:emr-sync-test:emr-sync-execution"
+ .formatted(REGION, ACCOUNT));
+ execution.setStateMachineArn(stateMachine.getStateMachineArn());
+ execution.setInput(input);
+
+ history = new ArrayList<>();
+ executor.executeSync(stateMachine, execution, history, (updated, events) -> { });
+ return execution;
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorFixture.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorFixture.java
index 2fbf220295..87e7673f7f 100644
--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorFixture.java
+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorFixture.java
@@ -37,7 +37,7 @@ import static org.mockito.Mockito.mock;
* <p>Every other collaborator defaults to what the convenience constructors pass: a Mockito mock
* for each integration, the system clock, a real sleeper, {@code System.nanoTime()} for deadlines,
* no Wait cap override, AWS's 5-minute Express limit, and no RDS Data, Redshift Data, service
- * registry or Lambda task authorizer. A Lambda target resolver is built over the function store
+ * registry, Lambda task authorizer or EMR task invoker. A Lambda target resolver is built over the function store
* unless a test supplies its own.
*/
final class AslExecutorFixture {
@@ -82,6 +82,7 @@ final class AslExecutorFixture {
private LambdaTaskAuthorizer lambdaTaskAuthorizer;
private Duration expressMaxExecutionTime;
private LongSupplier nanoTime;
+ private EmrTaskInvoker emrTaskInvoker;
private Builder() {
}
@@ -236,6 +237,11 @@ final class AslExecutorFixture {
return this;
}
+ Builder emrTaskInvoker(EmrTaskInvoker emrTaskInvoker) {
+ this.emrTaskInvoker = emrTaskInvoker;
+ return this;
+ }
+
AslExecutor build() {
return new AslExecutor(lambdaExecutor,
targetResolver != null ? targetResolver : new LambdaTargetResolver(functionStore, null),
@@ -245,7 +251,7 @@ final class AslExecutorFixture {
jsonataEvaluator != null ? jsonataEvaluator : new JsonataEvaluator(objectMapper),
sfnService, config, vertx, customResourceLiveness, clock, sleeper, maxWaitSeconds,
redshiftDataHandler, serviceRegistry, null, lambdaTaskAuthorizer, expressMaxExecutionTime,
- nanoTime, null);
+ nanoTime, null, emrTaskInvoker);
}
}
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/StepFunctionsEmrIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/StepFunctionsEmrIntegrationTest.java
new file mode 100644
index 0000000000..8a89762dbb
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/StepFunctionsEmrIntegrationTest.java
@@ -0,0 +1,272 @@
+package io.github.hectorvent.floci.services.stepfunctions;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.github.hectorvent.floci.testing.StepFunctionsExecutions;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.response.Response;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.UUID;
+
+import static io.github.hectorvent.floci.testing.EmrRequests.emr;
+import static io.restassured.RestAssured.given;
+import static org.hamcrest.Matchers.equalTo;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/**
+ * The {@code elasticmapreduce:} optimized integrations against Floci's EMR, from a state machine in a
+ * Region other than the default, so every call has to reach the clusters of the state machine's own.
+ */
+@QuarkusTest
+class StepFunctionsEmrIntegrationTest {
+
+ private static final String REGION = "eu-west-3";
+ private static final String ROLE_ARN = "arn:aws:iam::000000000000:role/test-role";
+ private static final String RESOURCE = "arn:aws:states:::elasticmapreduce:";
+ private static final String GROUPS_CLUSTER = """
+ {"Name": "sfn-emr", "ReleaseLabel": "emr-7.5.0",
+ "Instances": {"KeepJobFlowAliveWhenNoSteps": true, "InstanceGroups": [
+ {"Name": "Primary", "InstanceRole": "MASTER", "InstanceType": "m5.xlarge", "InstanceCount": 1},
+ {"Name": "Core", "InstanceRole": "CORE", "InstanceType": "m5.xlarge", "InstanceCount": 2}]}}
+ """;
+ private static final String FLEETS_CLUSTER = """
+ {"Name": "sfn-emr-fleets", "ReleaseLabel": "emr-7.5.0",
+ "Instances": {"KeepJobFlowAliveWhenNoSteps": true, "InstanceFleets": [
+ {"Name": "Primary", "InstanceFleetType": "MASTER", "TargetOnDemandCapacity": 1,
+ "InstanceTypeConfigs": [{"InstanceType": "m5.xlarge"}]},
+ {"Name": "Core", "InstanceFleetType": "CORE", "TargetOnDemandCapacity": 2,
+ "InstanceTypeConfigs": [{"InstanceType": "m5.xlarge"}]}]}}
+ """;
+ private static final String STEP = """
+ {"Name": "job", "ActionOnFailure": "CONTINUE",
+ "HadoopJarStep": {"Jar": "command-runner.jar", "Args": ["spark-submit", "s3://bucket/job.py"]}}
+ """;
+ private static final ObjectMapper MAPPER = new ObjectMapper();
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @Test
+ void aClusterIsCreatedGivenAStepResizedAndTerminated() throws Exception {
+ String definition = """
+ {"StartAt": "Create", "States": {
+ "Create": {"Type": "Task", "Resource": "%1$screateCluster.sync", "Parameters": %2$s,
+ "ResultPath": "$.cluster", "Next": "Run"},
+ "Run": {"Type": "Task", "Resource": "%1$saddStep.sync",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "Step": %3$s},
+ "ResultPath": "$.step", "Next": "Queue"},
+ "Queue": {"Type": "Task", "Resource": "%1$saddStep",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "Step": %3$s},
+ "ResultPath": "$.queued", "Next": "Cancel"},
+ "Cancel": {"Type": "Task", "Resource": "%1$scancelStep",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "StepId.$": "$.queued.StepId"},
+ "ResultPath": "$.cancel", "Next": "Resize"},
+ "Resize": {"Type": "Task", "Resource": "%1$smodifyInstanceGroupByName",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "InstanceGroupName": "Core",
+ "InstanceGroup": {"InstanceCount": 4}},
+ "ResultPath": "$.resize", "Next": "Terminate"},
+ "Terminate": {"Type": "Task", "Resource": "%1$sterminateCluster.sync",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId"},
+ "ResultPath": "$.terminated", "End": true}}}
+ """.formatted(RESOURCE, GROUPS_CLUSTER, STEP);
+
+ Response execution = run(definition);
+
+ assertEquals("SUCCEEDED", execution.jsonPath().getString("status"), execution.asString());
+ JsonNode output = MAPPER.readTree(execution.jsonPath().getString("output"));
+ JsonNode cluster = output.path("cluster");
+ String clusterId = cluster.path("ClusterId").asText();
+ assertEquals(List.of("Cluster", "ClusterId"), fieldNames(cluster));
+ assertEquals(clusterId, cluster.path("Cluster").path("Id").asText());
+ assertEquals("WAITING", cluster.path("Cluster").path("Status").path("State").asText());
+ JsonNode step = output.path("step");
+ assertEquals(List.of("Step", "StepId"), fieldNames(step));
+ assertEquals(step.path("StepId").asText(), step.path("Step").path("Id").asText());
+ assertEquals("COMPLETED", step.path("Step").path("Status").path("State").asText());
+ String queuedId = output.path("queued").path("StepId").asText();
+ assertEquals(List.of("StepId"), fieldNames(output.path("queued")));
+ assertTrue(queuedId.startsWith("s-"), queuedId);
+ JsonNode cancelled = output.path("cancel").path("CancelStepsInfo");
+ assertEquals(queuedId, cancelled.path("StepId").asText());
+ assertEquals("FAILED", cancelled.path("Status").asText());
+ assertEquals(MAPPER.createObjectNode(), output.path("resize"));
+ assertEquals("TERMINATED", output.path("terminated").path("Cluster").path("Status").path("State").asText());
+
+ emr(REGION, "ListInstanceGroups", "{\"ClusterId\":\"" + clusterId + "\"}").then().statusCode(200)
+ .body("InstanceGroups.find { it.Name == 'Core' }.RequestedInstanceCount", equalTo(4));
+ emr("DescribeCluster", "{\"ClusterId\":\"" + clusterId + "\"}").then().statusCode(400);
+
+ JsonNode submitted = history(execution).stream()
+ .filter(event -> "TaskSubmitted".equals(event.path("type").asText()))
+ .findFirst().orElseThrow().path("taskSubmittedEventDetails");
+ assertEquals("elasticmapreduce", submitted.path("resourceType").asText());
+ assertEquals("createCluster.sync", submitted.path("resource").asText());
+ assertEquals(clusterId, MAPPER.readTree(submitted.path("output").asText()).path("ClusterId").asText());
+ }
+
+ @Test
+ void anInstanceFleetIsResizedByName() throws Exception {
+ String definition = """
+ {"StartAt": "Create", "States": {
+ "Create": {"Type": "Task", "Resource": "%1$screateCluster", "Parameters": %2$s,
+ "ResultPath": "$.cluster", "Next": "Resize"},
+ "Resize": {"Type": "Task", "Resource": "%1$smodifyInstanceFleetByName",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "InstanceFleetName": "Core",
+ "InstanceFleet": {"TargetOnDemandCapacity": 5}},
+ "ResultPath": "$.resize", "End": true}}}
+ """.formatted(RESOURCE, FLEETS_CLUSTER);
+
+ Response execution = run(definition);
+
+ assertEquals("SUCCEEDED", execution.jsonPath().getString("status"), execution.asString());
+ JsonNode output = MAPPER.readTree(execution.jsonPath().getString("output"));
+ assertEquals(List.of("ClusterId"), fieldNames(output.path("cluster")));
+ emr(REGION, "ListInstanceFleets", "{\"ClusterId\":\"" + output.path("cluster").path("ClusterId").asText() + "\"}")
+ .then().statusCode(200)
+ .body("InstanceFleets.find { it.Name == 'Core' }.TargetOnDemandCapacity", equalTo(5));
+ }
+
+ @Test
+ void createClusterRefusesWhatTheGuideRulesOut() {
+ assertFails(task("createCluster", "{\"Name\": \"transient\", \"Instances\": {}}"), "States.TaskFailed",
+ "The field Instances.KeepJobFlowAliveWhenNoSteps is mandatory, and must have the Boolean value TRUE.");
+ assertFails(task("createCluster", """
+ {"Name": "with-steps", "Instances": {"KeepJobFlowAliveWhenNoSteps": true}, "Steps": [%s]}
+ """.formatted(STEP)), "States.TaskFailed", "The field Steps is not allowed.");
+ }
+
+ @Test
+ void aTaskWithoutTheFieldsItsApiActsOnFails() {
+ assertFails(task("terminateCluster", "{}"), "States.TaskFailed", "The field ClusterId is required.");
+ assertFails(task("addStep.sync", "{\"ClusterId\": \" \", \"Step\": " + STEP + "}"), "States.TaskFailed",
+ "The field ClusterId is required.");
+ assertFails(task("cancelStep", "{\"ClusterId\": \"j-ANY\"}"), "States.TaskFailed",
+ "The field StepId is required.");
+ assertFails(task("setClusterTerminationProtection", "{\"ClusterId\": \"j-ANY\"}"),
+ "EMR.AmazonElasticMapReduceException", "1 validation error detected: Value null at "
+ + "'terminationProtected' failed to satisfy constraint: Member must not be null");
+ assertFails(task("modifyInstanceGroupByName", "{\"ClusterId\": \"j-ANY\", \"InstanceGroup\": {}}"),
+ "States.TaskFailed", "The field InstanceGroupName is required.");
+ assertFails(task("modifyInstanceFleetByName", "{\"ClusterId\": \"j-ANY\", \"InstanceFleet\": {}}"),
+ "States.TaskFailed", "The field InstanceFleetName is required.");
+ }
+
+ @Test
+ void emrErrorsAreNamedAfterTheSdkExceptionClass() {
+ assertFails(task("addStep.sync", "{\"ClusterId\": \"j-NOSUCHCLUSTER\", \"Step\": " + STEP + "}"),
+ "EMR.AmazonElasticMapReduceException", "Specified job flow ID does not exist.");
+ assertFails(task("cancelStep", "{\"ClusterId\": \"j-NOSUCHCLUSTER\", \"StepId\": \"s-NOSUCHSTEP\"}"),
+ "EMR.InvalidRequestException", "Cluster id 'j-NOSUCHCLUSTER' is not valid.");
+ String protectedCluster = """
+ {"StartAt": "Create", "States": {
+ "Create": {"Type": "Task", "Resource": "%1$screateCluster", "Parameters": %2$s,
+ "ResultPath": "$.cluster", "Next": "Protect"},
+ "Protect": {"Type": "Task", "Resource": "%1$ssetClusterTerminationProtection",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "TerminationProtected": true},
+ "ResultPath": null, "Next": "Terminate"},
+ "Terminate": {"Type": "Task", "Resource": "%1$sterminateCluster",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId"}, "End": true}}}
+ """.formatted(RESOURCE, GROUPS_CLUSTER);
+ assertFails(protectedCluster, "EMR.AmazonElasticMapReduceException",
+ "Could not shut down one or more job flows since they are termination protected.");
+ }
+
+ @Test
+ void aResizeAfterTheClusterTerminatedFailsAsOnAws() {
+ assertFails(resizeAfterTermination(GROUPS_CLUSTER, "modifyInstanceGroupByName",
+ "\"InstanceGroupName\": \"Core\", \"InstanceGroup\": {\"InstanceCount\": 4}"),
+ "EMR.AmazonElasticMapReduceException",
+ "An instance group may only be modified when the cluster is running or waiting.");
+ assertFails(resizeAfterTermination(FLEETS_CLUSTER, "modifyInstanceFleetByName",
+ "\"InstanceFleetName\": \"Core\", \"InstanceFleet\": {\"TargetOnDemandCapacity\": 5}"),
+ "EMR.InvalidRequestException",
+ "A job flow that is shutting down, terminated, or finished may not be modified.");
+ }
+
+ @Test
+ void aByNameLookupNeedsOneMatchAndAnUnsupportedPatternFails() {
+ String unknownGroup = """
+ {"StartAt": "Create", "States": {
+ "Create": {"Type": "Task", "Resource": "%1$screateCluster", "Parameters": %2$s,
+ "ResultPath": "$.cluster", "Next": "Resize"},
+ "Resize": {"Type": "Task", "Resource": "%1$smodifyInstanceGroupByName",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", "InstanceGroupName": "Task",
+ "InstanceGroup": {"InstanceCount": 1}}, "End": true}}}
+ """.formatted(RESOURCE, GROUPS_CLUSTER);
+ assertFails(unknownGroup, "States.TaskFailed", "The cluster has no instance group named 'Task'.");
+ assertFails(task("cancelStep.sync", "{\"ClusterId\": \"j-ANY\", \"StepId\": \"s-ANY\"}"),
+ "States.TaskFailed", "Unsupported resource: " + RESOURCE + "cancelStep.sync");
+ }
+
+ private static String resizeAfterTermination(String cluster, String api, String resize) {
+ return """
+ {"StartAt": "Create", "States": {
+ "Create": {"Type": "Task", "Resource": "%1$screateCluster", "Parameters": %2$s,
+ "ResultPath": "$.cluster", "Next": "Terminate"},
+ "Terminate": {"Type": "Task", "Resource": "%1$sterminateCluster.sync",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId"},
+ "ResultPath": null, "Next": "Resize"},
+ "Resize": {"Type": "Task", "Resource": "%1$s%3$s",
+ "Parameters": {"ClusterId.$": "$.cluster.ClusterId", %4$s}, "End": true}}}
+ """.formatted(RESOURCE, cluster, api, resize);
+ }
+
+ private static String task(String api, String parameters) {
+ return """
+ {"StartAt": "Emr", "States": {"Emr": {"Type": "Task", "Resource": "%s%s",
+ "Parameters": %s, "End": true}}}
+ """.formatted(RESOURCE, api, parameters);
+ }
+
+ private void assertFails(String definition, String error, String cause) {
+ Response execution = run(definition);
+ assertEquals("FAILED", execution.jsonPath().getString("status"), execution.asString());
+ assertEquals(error, execution.jsonPath().getString("error"));
+ assertEquals(cause, execution.jsonPath().getString("cause"));
+ }
+
+ private Response run(String definition) {
+ String stateMachineArn = sfn("CreateStateMachine", MAPPER.createObjectNode()
+ .put("name", "sfn-emr-" + UUID.randomUUID())
+ .put("definition", definition)
+ .put("roleArn", ROLE_ARN).toString())
+ .then().statusCode(200).extract().path("stateMachineArn");
+ String executionArn = sfn("StartExecution", MAPPER.createObjectNode()
+ .put("stateMachineArn", stateMachineArn).put("input", "{}").toString())
+ .then().statusCode(200).extract().path("executionArn");
+ return StepFunctionsExecutions.waitForTerminalState(executionArn);
+ }
+
+ private List<JsonNode> history(Response execution) throws Exception {
+ String body = sfn("GetExecutionHistory", MAPPER.createObjectNode()
+ .put("executionArn", execution.jsonPath().getString("executionArn")).toString())
+ .then().statusCode(200).extract().asString();
+ List<JsonNode> events = new ArrayList<>();
+ MAPPER.readTree(body).path("events").forEach(events::add);
+ return events;
+ }
+
+ private static Response sfn(String action, String body) {
+ return given()
+ .header("Authorization", "AWS4-HMAC-SHA256 Credential=AKID/20260215/" + REGION
+ + "/states/aws4_request, SignedHeaders=host, Signature=abc")
+ .header("X-Amz-Target", "AWSStepFunctions." + action)
+ .contentType("application/x-amz-json-1.0")
+ .body(body)
+ .when().post("/");
+ }
+
+ private static List<String> fieldNames(JsonNode node) {
+ List<String> names = new ArrayList<>();
+ node.fieldNames().forEachRemaining(names::add);
+ return names;
+ }
+}