Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(dms): add endpoint, replication instance and replication task lifecycles (#5001)

Allen Sanborne7ca2ece22c8
diff --git a/docs/services/dms.md b/docs/services/dms.md
index d7a07233ad..d406c2cb63 100644
--- a/docs/services/dms.md
+++ b/docs/services/dms.md
@@ -3,10 +3,12 @@
**Protocol:** AWS JSON 1.1
**Signing name:** `dms`
-Floci emulates the replication subnet group lifecycle, which is what
-`aws_dms_replication_subnet_group` needs to plan and apply. Subnets are resolved against
+Floci emulates the replication subnet group, endpoint, replication instance and
+replication task lifecycles, which is what `aws_dms_replication_subnet_group`,
+`aws_dms_endpoint`, `aws_dms_s3_endpoint`, `aws_dms_replication_instance` and
+`aws_dms_replication_task` need to plan, apply and destroy. Subnets are resolved against
the emulated EC2 service, so the VPC and Availability Zones a group reports are the ones
-those subnets actually have.
+those subnets actually have. No data is migrated: the resources are control-plane state.
## Supported Actions
@@ -19,6 +21,20 @@ those subnets actually have.
| `ListTagsForResource` | Lists the tags on one or more DMS resource ARNs. |
| `AddTagsToResource` | Merges tags into the resource, overwriting by key. |
| `RemoveTagsFromResource` | Removes the named tag keys from the resource. |
+| `CreateEndpoint` | Creates a source or target endpoint; secret members are accepted and never returned. |
+| `DescribeEndpoints` | Lists endpoints, filtered by `endpoint-arn`, `endpoint-type`, `endpoint-id` or `engine-name`. |
+| `ModifyEndpoint` | Updates an endpoint; settings structures merge unless `ExactSettings` is true. |
+| `DeleteEndpoint` | Deletes an endpoint that no replication task uses. |
+| `CreateReplicationInstance` | Creates a replication instance, `available` immediately. |
+| `DescribeReplicationInstances` | Lists replication instances, filtered by `replication-instance-arn`, `replication-instance-id`, `replication-instance-class` or `engine-version`. |
+| `ModifyReplicationInstance` | Updates a replication instance, applied immediately. |
+| `DeleteReplicationInstance` | Deletes a replication instance that no replication task uses. |
+| `CreateReplicationTask` | Creates a replication task between two existing endpoints on an existing instance, `ready` immediately. |
+| `DescribeReplicationTasks` | Lists replication tasks, filtered by `replication-task-arn`, `replication-task-id`, `migration-type`, `endpoint-arn` or `replication-instance-arn`. |
+| `ModifyReplicationTask` | Updates a replication task that is not running. |
+| `DeleteReplicationTask` | Deletes a replication task that is not running. |
+| `StartReplicationTask` | Moves a task to `running`. |
+| `StopReplicationTask` | Moves a running task to `stopped`. |
<!-- floci:actions:end -->
## Behaviour
@@ -53,13 +69,54 @@ those subnets actually have.
not a serialization one.
- Groups are scoped per account and Region and persist through `StorageFactory`.
+### Endpoints, replication instances and replication tasks
+
+- Each resource gets an ARN of the form `arn:aws:dms:<region>:<account>:<type>:<id>`, where
+ `<type>` is `endpoint`, `rep` or `task` and `<id>` is 26 characters of the base32
+ alphabet, or the request's `ResourceIdentifier` when one is given. Update and delete
+ calls address the resource by that ARN.
+- Identifiers must begin with a letter, contain only ASCII letters, digits and hyphens,
+ and not end with a hyphen or contain two consecutive hyphens (63 characters for an
+ instance, 255 otherwise). A replication instance identifier is stored lowercase, as AWS
+ documents; endpoint and task identifiers keep their case. A duplicate identifier returns
+ `ResourceAlreadyExistsFault`.
+- A describe whose `Filters` match nothing returns `ResourceNotFoundFault`, which the
+ Terraform provider reads as "gone". Without filters, no resources is an empty list.
+ Filter values compare case-insensitively. Describes paginate on `MaxRecords` and
+ `Marker` as subnet groups do.
+- **Endpoints** are `active` from creation. `EndpointType` is accepted lowercase and returned
+ uppercase (`SOURCE`, `TARGET`), as AWS does. `SslMode` defaults to `none`. Engine settings
+ structures (`MySQLSettings`, `S3Settings` and the rest) are stored as sent and returned
+ as stored, except that every member the DMS model types as a secret (`Password`,
+ `SaslPassword`, `AuthPassword` and so on) is dropped, because DMS never returns them.
+ `ModifyEndpoint` merges a settings structure into the stored one member by member, or
+ replaces it when `ExactSettings` is true.
+- **Replication instances** are `available` from creation and stay so through a modify. A
+ named `ReplicationSubnetGroupIdentifier` must exist (otherwise `ResourceNotFoundFault`)
+ and is returned as the full `ReplicationSubnetGroup` structure; without one the instance
+ reports the `default` group. Defaults: `AllocatedStorage` 50, `EngineVersion` `3.5.4`,
+ `PubliclyAccessible` true, `MultiAZ` false, `AutoMinorVersionUpgrade` true,
+ `NetworkType` `IPV4`, `AvailabilityZone` `<region>a`. `ModifyReplicationInstance` applies
+ every change at once whatever `ApplyImmediately` says.
+- **Replication tasks** need existing source and target endpoints and an existing instance
+ (otherwise `ResourceNotFoundFault`). `TableMappings` and `ReplicationTaskSettings` must be
+ JSON objects and are returned exactly as sent. A task is `ready` from creation;
+ `StartReplicationTask` makes it `running` and `StopReplicationTask` makes it `stopped`.
+ Stopping a task that is not running returns `InvalidResourceStateFault` with "is
+ currently not running", the message the Terraform provider treats as already stopped.
+ Modifying or deleting a running task returns `InvalidResourceStateFault`.
+- Deleting an endpoint or instance that a task references, or a subnet group that an
+ instance uses, returns `InvalidResourceStateFault`. A delete returns the resource with
+ status `deleting`, and it is gone from the next describe.
+
### Tagging
-`Tags` on `CreateReplicationSubnetGroup` are stored with the group, and the tagging trio
-works against the group's ARN, which is
-`arn:aws:dms:<region>:<account>:subgrp:<identifier>`. DMS does not return that ARN from
+`Tags` on every create are stored with the resource, and the tagging trio works against
+the resource's ARN. For a subnet group that is
+`arn:aws:dms:<region>:<account>:subgrp:<identifier>`; DMS does not return that ARN from
`DescribeReplicationSubnetGroups`, so Terraform builds it client side and Floci parses it
-back the same way.
+back the same way. Endpoints, instances and tasks are tagged through the ARN their create
+returned.
- `AddTagsToResource` merges by key, so re-tagging an existing key overwrites its value.
- `RemoveTagsFromResource` removes the named keys and ignores keys that are not present.
@@ -71,15 +128,24 @@ back the same way.
- An ARN that is unparseable, names a different DMS resource type, names another account
or Region, or names a group that does not exist returns `ResourceNotFoundFault`. Tagging
is not a cross-account or cross-Region operation on AWS and is not one here either.
-- Deleting a group deletes its tags with it.
+- Deleting a resource deletes its tags with it.
## Limitations
-- **Only the subnet group lifecycle is implemented.** Replication instances, endpoints,
- and replication tasks return `UnknownOperationException`.
-- **Tagging covers replication subnet groups only.** The tagging trio is implemented, but
- a `ResourceArn` naming a replication instance, endpoint, or task returns
- `ResourceNotFoundFault` because Floci holds no such resource.
+- **No migration runs.** A started task stays `running` until stopped; a full-load task
+ never finishes on its own, and `ReplicationTaskStats`, `StopReason` and
+ `LastFailureMessage` are not reported.
+- **Values AWS fills in are not invented.** An endpoint does not report
+ `EngineDisplayName`, `ExternalId` or the settings structure AWS derives from top-level
+ connection fields, and a task without `ReplicationTaskSettings` reports none rather than
+ the AWS defaults. Instances report no private or public IP addresses.
+- **`CdcStartTime` is ignored.** `CreateReplicationTask`, `ModifyReplicationTask` and
+ `StartReplicationTask` accept it, but it is not stored or returned. `CdcStartPosition` and
+ `CdcStopPosition` are stored by create and modify, and ignored by `StartReplicationTask`.
+- **Not implemented:** `TestConnection`, `DescribeConnections`, `MoveReplicationTask`,
+ `DescribeEndpointTypes`, `DescribeOrderableReplicationInstances`, replication configs
+ (`aws_dms_replication_config`, serverless DMS), event subscriptions, certificates and the
+ `WithoutSettings` flag of `DescribeReplicationTasks`.
- **No `ModifyReplicationSubnetGroup`.** A subnet change has to be a delete and recreate.
See the [AWS DMS API Reference](https://docs.aws.amazon.com/dms/latest/APIReference/Welcome.html).
diff --git a/docs/services/index.md b/docs/services/index.md
index 668f03b02f..67bb963fa1 100644
--- a/docs/services/index.md
+++ b/docs/services/index.md
@@ -73,7 +73,7 @@ Operation counts are exact. For dispatch-table services (Query and JSON 1.1) eac
| [Lake Formation](lakeformation.md) | `POST /<Action>` | REST JSON | 16 |
| [Neptune](neptune.md) | `POST /` with `Action=` param + Gremlin TCP proxy | Query + WebSocket | 14 |
| [DocumentDB](docdb.md) | `POST /` with `Action=` param + MongoDB container | Query + MongoDB wire | 13 |
-| [DMS](dms.md) | `POST /` + `X-Amz-Target: AmazonDMSv20160101.*` | JSON 1.1 | 6 |
+| [DMS](dms.md) | `POST /` + `X-Amz-Target: AmazonDMSv20160101.*` | JSON 1.1 | 20 |
| [Redshift](redshift.md) | `POST /` with `Action=` param + PostgreSQL container | Query + PostgreSQL wire (+ CFN) | 35 |
| [Redshift Data API](redshift-data.md) | `POST /` + `X-Amz-Target: RedshiftData.*` | JSON 1.1 | 11 |
| [Redshift Serverless](redshift-serverless.md) | `POST /` + `X-Amz-Target: RedshiftServerless.*` | JSON 1.1 | 14 |
diff --git a/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java b/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java
index 652b02591c..47e78925db 100644
--- a/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java
+++ b/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java
@@ -5,12 +5,15 @@ import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
import io.github.hectorvent.floci.core.common.PaginatedResult;
+import io.github.hectorvent.floci.services.dms.model.DmsResource;
import io.github.hectorvent.floci.services.dms.model.ReplicationSubnetGroup;
import io.github.hectorvent.floci.services.dms.model.ResourceTag;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import jakarta.ws.rs.core.Response;
+import java.util.function.Function;
+
@ApplicationScoped
public class DmsJsonHandler {
@@ -62,10 +65,66 @@ public class DmsJsonHandler {
service.removeTagsFromResource(request, region);
yield Response.ok(objectMapper.createObjectNode()).build();
}
+ case "CreateEndpoint" -> single("Endpoint", service.createEndpoint(request, region).getAttributes());
+ case "DescribeEndpoints" -> page("Endpoints", service.describeEndpoints(request, region),
+ DmsResource::getAttributes);
+ case "ModifyEndpoint" -> single("Endpoint", service.modifyEndpoint(request, region).getAttributes());
+ case "DeleteEndpoint" -> single("Endpoint", service.deleteEndpoint(request, region).getAttributes());
+ case "CreateReplicationInstance" -> single("ReplicationInstance",
+ instance(service.createReplicationInstance(request, region), region));
+ case "DescribeReplicationInstances" -> page("ReplicationInstances",
+ service.describeReplicationInstances(request, region), item -> instance(item, region));
+ case "ModifyReplicationInstance" -> single("ReplicationInstance",
+ instance(service.modifyReplicationInstance(request, region), region));
+ case "DeleteReplicationInstance" -> single("ReplicationInstance",
+ instance(service.deleteReplicationInstance(request, region), region));
+ case "CreateReplicationTask" -> single("ReplicationTask",
+ service.createReplicationTask(request, region).getAttributes());
+ case "DescribeReplicationTasks" -> page("ReplicationTasks",
+ service.describeReplicationTasks(request, region), DmsResource::getAttributes);
+ case "ModifyReplicationTask" -> single("ReplicationTask",
+ service.modifyReplicationTask(request, region).getAttributes());
+ case "DeleteReplicationTask" -> single("ReplicationTask",
+ service.deleteReplicationTask(request, region).getAttributes());
+ case "StartReplicationTask" -> single("ReplicationTask",
+ service.startReplicationTask(request, region).getAttributes());
+ case "StopReplicationTask" -> single("ReplicationTask",
+ service.stopReplicationTask(request, region).getAttributes());
default -> null;
};
}
+ private Response single(String member, ObjectNode item) {
+ ObjectNode response = objectMapper.createObjectNode();
+ response.set(member, item);
+ return Response.ok(response).build();
+ }
+
+ private Response page(String member, PaginatedResult<DmsResource> page,
+ Function<DmsResource, ObjectNode> render) {
+ ObjectNode response = objectMapper.createObjectNode();
+ ArrayNode items = response.putArray(member);
+ page.items().forEach(item -> items.add(render.apply(item)));
+ if (page.nextToken() != null) {
+ response.put("Marker", page.nextToken());
+ }
+ return Response.ok(response).build();
+ }
+
+ /**
+ * Expands the stored subnet group identifier into the full {@code ReplicationSubnetGroup}
+ * structure DMS returns, read at describe time so it reflects the group as it is now.
+ */
+ private ObjectNode instance(DmsResource resource, String region) {
+ ObjectNode node = resource.getAttributes().deepCopy();
+ String groupId = node.path("ReplicationSubnetGroup").path("ReplicationSubnetGroupIdentifier").asText(null);
+ if (groupId != null) {
+ service.findReplicationSubnetGroup(region, groupId)
+ .ifPresent(group -> node.set("ReplicationSubnetGroup", subnetGroup(group)));
+ }
+ return node;
+ }
+
private ObjectNode tagNode(ResourceTag tag) {
ObjectNode node = objectMapper.createObjectNode();
node.put("Key", tag.key());
diff --git a/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java b/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java
index 60b982f772..2c61cde28c 100644
--- a/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java
@@ -1,7 +1,12 @@
package io.github.hectorvent.floci.services.dms;
+import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.core.type.TypeReference;
import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.JsonNodeFactory;
+import com.fasterxml.jackson.databind.node.ObjectNode;
import io.github.hectorvent.floci.core.common.AwsArnUtils;
import io.github.hectorvent.floci.core.common.AwsException;
import io.github.hectorvent.floci.core.common.PaginatedResult;
@@ -10,6 +15,7 @@ import io.github.hectorvent.floci.core.common.RegionResolver;
import io.github.hectorvent.floci.core.common.Resettable;
import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;
import io.github.hectorvent.floci.core.storage.StorageFactory;
+import io.github.hectorvent.floci.services.dms.model.DmsResource;
import io.github.hectorvent.floci.services.dms.model.ReplicationSubnetGroup;
import io.github.hectorvent.floci.services.dms.model.ResourceTag;
import io.github.hectorvent.floci.services.ec2.Ec2Service;
@@ -17,12 +23,15 @@ import io.github.hectorvent.floci.services.ec2.model.Subnet;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
+import java.security.SecureRandom;
+import java.time.Instant;
import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.Locale;
import java.util.Map;
import java.util.Objects;
+import java.util.Optional;
import java.util.Set;
import java.util.function.Consumer;
import java.util.regex.Pattern;
@@ -39,7 +48,50 @@ public class DmsService implements Resettable {
private static final int MAXIMUM_MAX_RECORDS = 100;
private static final Set<String> RESERVED_TAG_PREFIXES = Set.of("aws:", "dms:");
+ /** Endpoint, replication instance and task identifiers share one shape in the DMS API reference. */
+ private static final Pattern RESOURCE_IDENTIFIER = Pattern.compile("[A-Za-z][0-9A-Za-z-]*");
+ private static final int RESOURCE_IDENTIFIER_ARN_MAX = 31;
+ private static final String RESOURCE_ID_ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";
+ private static final SecureRandom RANDOM = new SecureRandom();
+ private static final ObjectMapper JSON = new ObjectMapper();
+ private static final String TASK_RUNNING = "running";
+
+ /** Members of {@code Endpoint} that are strings on both CreateEndpoint and ModifyEndpoint. */
+ private static final List<String> ENDPOINT_STRING_MEMBERS = List.of("EngineName", "Username",
+ "ServerName", "DatabaseName", "ExtraConnectionAttributes", "CertificateArn",
+ "ServiceAccessRoleArn", "ExternalTableDefinition");
+ /** The engine-specific settings structures CreateEndpoint and ModifyEndpoint accept. */
+ private static final List<String> ENDPOINT_SETTINGS = List.of("DynamoDbSettings", "S3Settings",
+ "DmsTransferSettings", "MongoDbSettings", "KinesisSettings", "KafkaSettings",
+ "ElasticsearchSettings", "NeptuneSettings", "RedshiftSettings", "PostgreSQLSettings",
+ "MySQLSettings", "OracleSettings", "SybaseSettings", "MicrosoftSQLServerSettings",
+ "IBMDb2Settings", "DocDbSettings", "RedisSettings", "GcpMySQLSettings", "TimestreamSettings");
+ /**
+ * Every member the DMS model types as {@code SecretString}. DMS accepts them but never returns
+ * them, so they are dropped before a settings structure is stored.
+ */
+ private static final List<String> SECRET_MEMBERS = List.of("AsmPassword", "AuthPassword",
+ "CertificatePem", "Password", "S3ObjectUrl", "SaslPassword", "SecurityDbEncryption",
+ "SelectionRules", "SslClientKeyPassword");
+ private static final Set<String> SSL_MODES = Set.of("none", "require", "verify-ca", "verify-full");
+ private static final Set<String> MIGRATION_TYPES = Set.of("full-load", "cdc", "full-load-and-cdc");
+ private static final Set<String> START_TYPES =
+ Set.of("start-replication", "resume-processing", "reload-target");
+
+ /** One of the three ARN-addressed resource families and the names DMS uses for it. */
+ private record Kind(String label, AccountAwareStorageBackend<DmsResource> store, String arnType,
+ String idMember, String arnMember, int maxIdLength, boolean lowercaseId,
+ Map<String, List<String>> filters) {
+ }
+
+ /** A stored resource together with the storage key it lives under. */
+ private record Found(Kind kind, String key, DmsResource resource) {
+ }
+
private final AccountAwareStorageBackend<ReplicationSubnetGroup> subnetGroups;
+ private final Kind endpoints;
+ private final Kind instances;
+ private final Kind tasks;
private final Ec2Service ec2Service;
private final RegionResolver regionResolver;
@@ -47,10 +99,251 @@ public class DmsService implements Resettable {
public DmsService(StorageFactory storageFactory, Ec2Service ec2Service, RegionResolver regionResolver) {
this.subnetGroups = storageFactory.create("dms", "dms-replication-subnet-groups.json",
new TypeReference<Map<String, ReplicationSubnetGroup>>() {});
+ TypeReference<Map<String, DmsResource>> type = new TypeReference<>() {};
+ this.endpoints = new Kind("Endpoint", storageFactory.create("dms", "dms-endpoints.json", type),
+ "endpoint", "EndpointIdentifier", "EndpointArn", 255, false,
+ Map.of("endpoint-arn", List.of("EndpointArn"),
+ "endpoint-type", List.of("EndpointType"),
+ "endpoint-id", List.of("EndpointIdentifier"),
+ "engine-name", List.of("EngineName")));
+ this.instances = new Kind("Replication instance",
+ storageFactory.create("dms", "dms-replication-instances.json", type),
+ "rep", "ReplicationInstanceIdentifier", "ReplicationInstanceArn", 63, true,
+ Map.of("replication-instance-arn", List.of("ReplicationInstanceArn"),
+ "replication-instance-id", List.of("ReplicationInstanceIdentifier"),
+ "replication-instance-class", List.of("ReplicationInstanceClass"),
+ "engine-version", List.of("EngineVersion")));
+ this.tasks = new Kind("Replication task",
+ storageFactory.create("dms", "dms-replication-tasks.json", type),
+ "task", "ReplicationTaskIdentifier", "ReplicationTaskArn", 255, false,
+ Map.of("replication-task-arn", List.of("ReplicationTaskArn"),
+ "replication-task-id", List.of("ReplicationTaskIdentifier"),
+ "migration-type", List.of("MigrationType"),
+ "endpoint-arn", List.of("SourceEndpointArn", "TargetEndpointArn"),
+ "replication-instance-arn", List.of("ReplicationInstanceArn")));
this.ec2Service = ec2Service;
this.regionResolver = regionResolver;
}
+ // ---------------------------------------------------------------- endpoints
+
+ public synchronized DmsResource createEndpoint(JsonNode request, String region) {
+ String identifier = requireResourceIdentifier(endpoints, request);
+ String endpointType = requireEndpointType(request);
+ if (text(request, "EngineName") == null || text(request, "EngineName").isBlank()) {
+ throw invalidParameter("The parameter EngineName must be provided and must not be blank.");
+ }
+ ObjectNode attributes = JsonNodeFactory.instance.objectNode();
+ attributes.put("EndpointType", endpointType);
+ attributes.put("Status", "active");
+ attributes.put("SslMode", "none");
+ putText(attributes, request, "KmsKeyId");
+ applyEndpointMembers(attributes, request, false);
+ return create(endpoints, region, identifier, attributes, request);
+ }
+
+ public PaginatedResult<DmsResource> describeEndpoints(JsonNode request, String region) {
+ return describe(endpoints, request, region);
+ }
+
+ /**
+ * Settings structures merge member by member into what is stored, as DMS documents for
+ * {@code ExactSettings} false (the default); with {@code ExactSettings} true a structure in the
+ * request replaces the stored one.
+ */
+ public synchronized DmsResource modifyEndpoint(JsonNode request, String region) {
+ Found found = requireByArn(endpoints, request, endpoints.arnMember(), region);
+ ObjectNode attributes = found.resource().getAttributes().deepCopy();
+ rename(endpoints, region, found, attributes, request);
+ if (text(request, "EndpointType") != null) {
+ attributes.put("EndpointType", requireEndpointType(request));
+ }
+ applyEndpointMembers(attributes, request, bool(request, "ExactSettings", false));
+ return save(found, attributes);
+ }
+
+ public synchronized DmsResource deleteEndpoint(JsonNode request, String region) {
+ Found found = requireByArn(endpoints, request, endpoints.arnMember(), region);
+ String arn = arnOf(endpoints, found.resource());
+ boolean inUse = regionScan(tasks, region).stream()
+ .anyMatch(task -> arn.equals(member(task, "SourceEndpointArn"))
+ || arn.equals(member(task, "TargetEndpointArn")));
+ if (inUse) {
+ throw invalidState("Endpoint " + arn + " is in use by a replication task and cannot be deleted.");
+ }
+ DmsResource removed = remove(found);
+ removed.getAttributes().put("Status", "deleting");
+ return removed;
+ }
+
+ // ---------------------------------------------------------------- replication instances
+
+ /**
+ * The instance is {@code available} as soon as it is created: there is no host to provision, and
+ * the Terraform waiter accepts {@code available} on its first poll.
+ */
+ public synchronized DmsResource createReplicationInstance(JsonNode request, String region) {
+ String identifier = requireResourceIdentifier(instances, request);
+ if (text(request, "ReplicationInstanceClass") == null) {
+ throw invalidParameter("The parameter ReplicationInstanceClass must be provided.");
+ }
+ ObjectNode attributes = JsonNodeFactory.instance.objectNode();
+ attributes.put("ReplicationInstanceStatus", "available");
+ attributes.put("AllocatedStorage", 50);
+ attributes.put("InstanceCreateTime", now());
+ attributes.putArray("VpcSecurityGroups");
+ attributes.put("AvailabilityZone", region + "a");
+ attributes.put("PreferredMaintenanceWindow", "sun:06:00-sun:06:30");
+ attributes.put("MultiAZ", false);
+ attributes.put("EngineVersion", "3.5.4");
+ attributes.put("AutoMinorVersionUpgrade", true);
+ attributes.put("PubliclyAccessible", bool(request, "PubliclyAccessible", true));
+ attributes.put("NetworkType", "IPV4");
+ attributes.putArray("ReplicationInstancePrivateIpAddresses");
+ attributes.putArray("ReplicationInstancePublicIpAddresses");
+ putText(attributes, request, "AvailabilityZone");
+ putText(attributes, request, "KmsKeyId");
+ putText(attributes, request, "DnsNameServers");
+
+ String subnetGroup = text(request, "ReplicationSubnetGroupIdentifier");
+ if (subnetGroup == null) {
+ // DMS falls back to the account's "default" group, which is why that name is reserved.
+ subnetGroup = "default";
+ } else {
+ subnetGroup = subnetGroup.toLowerCase(Locale.ROOT);
+ if (subnetGroups.get(storageKey(region, subnetGroup)).isEmpty()) {
+ throw notFound(subnetGroup);
+ }
+ }
+ attributes.putObject("ReplicationSubnetGroup").put("ReplicationSubnetGroupIdentifier", subnetGroup);
+ applyInstanceMembers(attributes, request);
+ return create(instances, region, identifier, attributes, request);
+ }
+
+ public PaginatedResult<DmsResource> describeReplicationInstances(JsonNode request, String region) {
+ return describe(instances, request, region);
+ }
+
+ /**
+ * Applied at once whatever {@code ApplyImmediately} says: Floci has no maintenance window to
+ * defer to, and a change parked in {@code PendingModifiedValues} would read back as drift.
+ */
+ public synchronized DmsResource modifyReplicationInstance(JsonNode request, String region) {
+ Found found = requireByArn(instances, request, instances.arnMember(), region);
+ ObjectNode attributes = found.resource().getAttributes().deepCopy();
+ rename(instances, region, found, attributes, request);
+ applyInstanceMembers(attributes, request);
+ return save(found, attributes);
+ }
+
+ public synchronized DmsResource deleteReplicationInstance(JsonNode request, String region) {
+ Found found = requireByArn(instances, request, instances.arnMember(), region);
+ String arn = arnOf(instances, found.resource());
+ if (regionScan(tasks, region).stream().anyMatch(task -> arn.equals(member(task, "ReplicationInstanceArn")))) {
+ throw invalidState("Replication instance " + arn + " has replication tasks and cannot be deleted.");
+ }
+ DmsResource removed = remove(found);
+ removed.getAttributes().put("ReplicationInstanceStatus", "deleting");
+ return removed;
+ }
+
+ /**
+ * "default" cannot be created, so it is never stored: it resolves to the region's default-VPC
+ * subnets at read time, which is the group an instance created without one lands in.
+ */
+ public Optional<ReplicationSubnetGroup> findReplicationSubnetGroup(String region, String identifier) {
+ if (!"default".equals(identifier)) {
+ return subnetGroups.get(storageKey(region, identifier));
+ }
+ List<String> defaultSubnets = ec2Service.describeSubnets(region, List.of(), Map.of()).stream()
+ .filter(Subnet::isDefaultForAz)
+ .map(Subnet::getSubnetId)
+ .toList();
+ if (defaultSubnets.isEmpty()) {
+ return Optional.empty();
+ }
+ try {
+ return Optional.of(buildSubnetGroup(identifier, "default", defaultSubnets, region));
+ } catch (AwsException ignored) {
+ // Safe to swallow: buildSubnetGroup only rejects the default subnets when they cannot
+ // form a valid group (a default VPC trimmed below two AZs, or a subnet with no AZ).
+ // There is then no usable default group, so this answers exactly as when the region
+ // has no default subnets and the caller falls back to the bare identifier.
+ return Optional.empty();
+ }
+ }
+
+ // ---------------------------------------------------------------- replication tasks
+
+ /** A new task is {@code ready}, the state the Terraform create waiter targets. */
+ public synchronized DmsResource createReplicationTask(JsonNode request, String region) {
+ String identifier = requireResourceIdentifier(tasks, request);
+ ObjectNode attributes = JsonNodeFactory.instance.objectNode();
+ attributes.put("SourceEndpointArn", requireEndpointOfType(request, "SourceEndpointArn", "SOURCE", region));
+ attributes.put("TargetEndpointArn", requireEndpointOfType(request, "TargetEndpointArn", "TARGET", region));
+ attributes.put("ReplicationInstanceArn",
+ arnOf(instances, requireByArn(instances, request, "ReplicationInstanceArn", region).resource()));
+ if (text(request, "MigrationType") == null) {
+ throw invalidParameter("The parameter MigrationType must be provided.");
+ }
+ if (text(request, "TableMappings") == null) {
+ throw invalidParameter("The parameter TableMappings must be provided.");
+ }
+ attributes.put("Status", "ready");
+ attributes.put("ReplicationTaskCreationDate", now());
+ applyTaskMembers(attributes, request);
+ return create(tasks, region, identifier, attributes, request);
+ }
+
+ public PaginatedResult<DmsResource> describeReplicationTasks(JsonNode request, String region) {
+ return describe(tasks, request, region);
+ }
+
+ public synchronized DmsResource modifyReplicationTask(JsonNode request, String region) {
+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);
+ requireNotRunning(found, "modified");
+ ObjectNode attributes = found.resource().getAttributes().deepCopy();
+ rename(tasks, region, found, attributes, request);
+ applyTaskMembers(attributes, request);
+ return save(found, attributes);
+ }
+
+ public synchronized DmsResource deleteReplicationTask(JsonNode request, String region) {
+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);
+ requireNotRunning(found, "deleted");
+ DmsResource removed = remove(found);
+ removed.getAttributes().put("Status", "deleting");
+ return removed;
+ }
+
+ /**
+ * The task goes straight to {@code running} and stays there until stopped: no data moves, so a
+ * full-load task never finishes on its own the way it would against real endpoints.
+ */
+ public synchronized DmsResource startReplicationTask(JsonNode request, String region) {
+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);
+ String startType = text(request, "StartReplicationTaskType");
+ if (startType == null || !START_TYPES.contains(startType)) {
+ throw invalidParameter("StartReplicationTaskType must be one of " + START_TYPES + ".");
+ }
+ requireNotRunning(found, "started");
+ ObjectNode attributes = found.resource().getAttributes().deepCopy();
+ attributes.put("Status", TASK_RUNNING);
+ attributes.put("ReplicationTaskStartDate", now());
+ return save(found, attributes);
+ }
+
+ public synchronized DmsResource stopReplicationTask(JsonNode request, String region) {
+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);
+ if (!TASK_RUNNING.equals(member(found.resource(), "Status"))) {
+ throw invalidState("Replication task " + arnOf(tasks, found.resource())
+ + " is currently not running.");
+ }
+ ObjectNode attributes = found.resource().getAttributes().deepCopy();
+ attributes.put("Status", "stopped");
+ return save(found, attributes);
+ }
+
public synchronized ReplicationSubnetGroup createReplicationSubnetGroup(JsonNode request, String region) {
String identifier = requireIdentifier(request);
String description = requireDescription(request);
@@ -96,6 +389,13 @@ public class DmsService implements Resettable {
if (subnetGroups.get(key).isEmpty()) {
throw notFound(identifier);
}
+ boolean inUse = regionScan(instances, region).stream()
+ .anyMatch(instance -> identifier.equals(instance.getAttributes()
+ .path("ReplicationSubnetGroup").path("ReplicationSubnetGroupIdentifier").asText(null)));
+ if (inUse) {
+ throw invalidState("Replication subnet group " + identifier
+ + " is in use by a replication instance and cannot be deleted.");
+ }
subnetGroups.delete(key);
}
@@ -104,13 +404,13 @@ public class DmsService implements Resettable {
if (!arnList.isEmpty()) {
List<ResourceTag> tags = new ArrayList<>();
for (String arn : arnList) {
- resolveByArn(arn, region).getTags()
+ tagsByArn(arn, region)
.forEach((key, value) -> tags.add(new ResourceTag(arn, key, value)));
}
return tags;
}
String resourceArn = requireResourceArn(request);
- return resolveByArn(resourceArn, region).getTags().entrySet().stream()
+ return tagsByArn(resourceArn, region).entrySet().stream()
.map(entry -> new ResourceTag(null, entry.getKey(), entry.getValue()))
.toList();
}
@@ -138,6 +438,324 @@ public class DmsService implements Resettable {
@Override
public void clear() {
subnetGroups.clear();
+ endpoints.store().clear();
+ instances.store().clear();
+ tasks.store().clear();
+ }
+
+ // ---------------------------------------------------------------- shared resource plumbing
+
+ private DmsResource create(Kind kind, String region, String identifier, ObjectNode attributes,
+ JsonNode request) {
+ if (findByIdentifier(kind, region, identifier).isPresent()) {
+ throw new AwsException("ResourceAlreadyExistsFault",
+ kind.label() + " " + identifier + " already exists.", 400);
+ }
+ String resourceId = text(request, "ResourceIdentifier");
+ if (resourceId == null) {
+ resourceId = randomResourceId();
+ } else if (!isValidIdentifier(resourceId, RESOURCE_IDENTIFIER_ARN_MAX)) {
+ throw invalidParameter("ResourceIdentifier must be 1-" + RESOURCE_IDENTIFIER_ARN_MAX
+ + " ASCII letters, digits and hyphens, begin with a letter, and not end with a"
+ + " hyphen or contain two consecutive hyphens.");
+ }
+ String key = storageKey(region, resourceId);
+ if (kind.store().get(key).isPresent()) {
+ throw new AwsException("ResourceAlreadyExistsFault",
+ "ResourceIdentifier " + resourceId + " is already in use.", 400);
+ }
+ attributes.put(kind.idMember(), identifier);
+ attributes.put(kind.arnMember(), regionResolver.buildArn("dms", region, kind.arnType() + ":" + resourceId));
+ DmsResource resource = new DmsResource();
+ resource.setAttributes(attributes);
+ resource.setTags(readTags(request.get("Tags")));
+ kind.store().put(key, resource);
+ return resource;
+ }
+
+ /**
+ * Filters AND across names and OR across a filter's values. Values compare case-insensitively,
+ * so {@code endpoint-type=source} matches the {@code SOURCE} DMS returns. A filter that matches
+ * nothing is {@code ResourceNotFoundFault}, which is what the Terraform provider's finders
+ * treat as gone; an unfiltered describe of nothing is an empty list.
+ */
+ private PaginatedResult<DmsResource> describe(Kind kind, JsonNode request, String region) {
+ Integer maxRecords = maxRecords(request);
+ String marker = text(request, "Marker");
+ Map<String, List<String>> filters = filters(request, kind.filters().keySet());
+ List<DmsResource> matching = regionScan(kind, region).stream()
+ .filter(resource -> filters.entrySet().stream().allMatch(filter ->
+ kind.filters().get(filter.getKey()).stream()
+ .map(name -> member(resource, name))
+ .anyMatch(value -> value != null && filter.getValue().stream()
+ .anyMatch(value::equalsIgnoreCase))))
+ .toList();
+ if (!filters.isEmpty() && matching.isEmpty()) {
+ throw new AwsException("ResourceNotFoundFault",
+ "No " + kind.label() + " found matching provided filters.", 400);
+ }
+ return Pagination.paginate(matching, resource -> member(resource, kind.idMember()),
+ maxRecords, marker, DEFAULT_MAX_RECORDS, MAXIMUM_MAX_RECORDS, "InvalidParameterValueException");
+ }
+
+ private void rename(Kind kind, String region, Found found, ObjectNode attributes, JsonNode request) {
+ if (text(request, kind.idMember()) == null) {
+ return;
+ }
+ String identifier = requireResourceIdentifier(kind, request);
+ if (!identifier.equals(member(found.resource(), kind.idMember()))
+ && findByIdentifier(kind, region, identifier).isPresent()) {
+ throw new AwsException("ResourceAlreadyExistsFault",
+ kind.label() + " " + identifier + " already exists.", 400);
+ }
+ attributes.put(kind.idMember(), identifier);
+ }
+
+ private DmsResource save(Found found, ObjectNode attributes) {
+ found.resource().setAttributes(attributes);
+ found.kind().store().put(found.key(), found.resource());
+ return found.resource();
+ }
+
+ private static DmsResource remove(Found found) {
+ found.kind().store().delete(found.key());
+ DmsResource removed = new DmsResource();
+ removed.setAttributes(found.resource().getAttributes().deepCopy());
+ return removed;
+ }
+
+ private Optional<DmsResource> findByIdentifier(Kind kind, String region, String identifier) {
+ return regionScan(kind, region).stream()
+ .filter(resource -> identifier.equals(member(resource, kind.idMember())))
+ .findFirst();
+ }
+
+ private List<DmsResource> regionScan(Kind kind, String region) {
+ return kind.store().scan(key -> key.startsWith(region + "::"));
+ }
+
+ /** Resolves the ARN in {@code member} of the request to a stored resource of {@code kind}. */
+ private Found requireByArn(Kind kind, JsonNode request, String member, String region) {
+ String arn = text(request, member);
+ if (arn == null || arn.isBlank()) {
+ throw invalidParameter("The parameter " + member + " must be provided and must not be blank.");
+ }
+ String[] parsed = parseArn(arn, region);
+ if (!kind.arnType().equals(parsed[0])) {
+ throw notFoundForArn(arn);
+ }
+ String key = storageKey(region, parsed[1]);
+ return kind.store().get(key).map(resource -> new Found(kind, key, resource))
+ .orElseThrow(() -> notFoundForArn(arn));
+ }
+
+ private Kind kindForArnType(String arnType, String resourceArn) {
+ for (Kind kind : List.of(endpoints, instances, tasks)) {
+ if (kind.arnType().equals(arnType)) {
+ return kind;
+ }
+ }
+ throw notFoundForArn(resourceArn);
+ }
+
+ private String requireResourceIdentifier(Kind kind, JsonNode request) {
+ String value = text(request, kind.idMember());
+ if (value == null || value.isBlank()) {
+ throw invalidParameter("The parameter " + kind.idMember() + " must be provided and must not be blank.");
+ }
+ if (!isValidIdentifier(value, kind.maxIdLength())) {
+ throw invalidParameter(kind.idMember() + " must be 1-" + kind.maxIdLength()
+ + " ASCII letters, digits and hyphens, begin with a letter, and not end with a"
+ + " hyphen or contain two consecutive hyphens.");
+ }
+ return kind.lowercaseId() ? value.toLowerCase(Locale.ROOT) : value;
+ }
+
+ private static boolean isValidIdentifier(String value, int maxLength) {
+ return value.length() <= maxLength && RESOURCE_IDENTIFIER.matcher(value).matches()
+ && !value.contains("--") && !value.endsWith("-");
+ }
+
+ private String requireEndpointOfType(JsonNode request, String member, String type, String region) {
+ DmsResource endpoint = requireByArn(endpoints, request, member, region).resource();
+ String arn = arnOf(endpoints, endpoint);
+ if (!type.equals(member(endpoint, "EndpointType"))) {
+ throw invalidParameter(member + " " + arn + " is not a " + type.toLowerCase(Locale.ROOT) + " endpoint.");
+ }
+ return arn;
+ }
+
+ /** DMS accepts the lowercase enum value and returns it uppercase. */
+ private static String requireEndpointType(JsonNode request) {
+ String value = text(request, "EndpointType");
+ if (value == null || !Set.of("source", "target").contains(value.toLowerCase(Locale.ROOT))) {
+ throw invalidParameter("The parameter EndpointType must be source or target.");
+ }
+ return value.toUpperCase(Locale.ROOT);
+ }
+
+ private static void applyEndpointMembers(ObjectNode attributes, JsonNode request, boolean exactSettings) {
+ ENDPOINT_STRING_MEMBERS.forEach(name -> putText(attributes, request, name));
+ String sslMode = text(request, "SslMode");
+ if (sslMode != null) {
+ if (!SSL_MODES.contains(sslMode)) {
+ throw invalidParameter("SslMode must be one of " + SSL_MODES + ".");
+ }
+ attributes.put("SslMode", sslMode);
+ }
+ Integer port = integer(request, "Port");
+ if (port != null) {
+ attributes.put("Port", port);
+ }
+ for (String name : ENDPOINT_SETTINGS) {
+ JsonNode settings = request == null ? null : request.get(name);
+ if (settings == null || settings.isNull()) {
+ continue;
+ }
+ if (!settings.isObject()) {
+ throw serialization(name + " must be a structure.");
+ }
+ ObjectNode stored = ((ObjectNode) settings).deepCopy();
+ stored.remove(SECRET_MEMBERS);
+ if (!exactSettings && attributes.get(name) instanceof ObjectNode existing) {
+ existing.setAll(stored);
+ } else {
+ attributes.set(name, stored);
+ }
+ }
+ }
+
+ private static void applyInstanceMembers(ObjectNode attributes, JsonNode request) {
+ putText(attributes, request, "ReplicationInstanceClass");
+ putText(attributes, request, "PreferredMaintenanceWindow");
+ putText(attributes, request, "EngineVersion");
+ putText(attributes, request, "NetworkType");
+ String instanceClass = attributes.path("ReplicationInstanceClass").asText();
+ if (!instanceClass.startsWith("dms.")) {
+ throw invalidParameter("Invalid ReplicationInstanceClass " + instanceClass + ".");
+ }
+ Integer storage = integer(request, "AllocatedStorage");
+ if (storage != null) {
+ if (storage < 5 || storage > 6144) {
+ throw invalidParameter("AllocatedStorage must be between 5 and 6144.");
+ }
+ attributes.put("AllocatedStorage", storage);
+ }
+ for (String name : List.of("MultiAZ", "AutoMinorVersionUpgrade")) {
+ JsonNode value = request == null ? null : request.get(name);
+ if (value != null && !value.isNull()) {
+ attributes.put(name, bool(request, name, false));
+ }
+ }
+ JsonNode groups = request == null ? null : request.get("VpcSecurityGroupIds");
+ if (groups != null && !groups.isNull()) {
+ ArrayNode memberships = attributes.putArray("VpcSecurityGroups");
+ stringList(groups, "VpcSecurityGroupIds").forEach(id ->
+ memberships.addObject().put("VpcSecurityGroupId", id).put("Status", "active"));
+ }
+ JsonNode kerberos = request == null ? null : request.get("KerberosAuthenticationSettings");
+ if (kerberos != null && !kerberos.isNull()) {
+ if (!kerberos.isObject()) {
+ throw serialization("KerberosAuthenticationSettings must be a structure.");
+ }
+ // Only the members DMS models; anything else a caller sends is neither stored nor echoed.
+ ObjectNode stored = attributes.putObject("KerberosAuthenticationSettings");
+ for (String member : List.of("KeyCacheSecretId", "KeyCacheSecretIamArn", "Krb5FileContents")) {
+ putText(stored, kerberos, member);
+ }
+ }
+ }
+
+ private static void applyTaskMembers(ObjectNode attributes, JsonNode request) {
+ String migrationType = text(request, "MigrationType");
+ if (migrationType != null) {
+ if (!MIGRATION_TYPES.contains(migrationType)) {
+ throw invalidParameter("MigrationType must be one of " + MIGRATION_TYPES + ".");
+ }
+ attributes.put("MigrationType", migrationType);
+ }
+ for (String name : List.of("TableMappings", "ReplicationTaskSettings")) {
+ String value = text(request, name);
+ if (value != null) {
+ requireJsonObject(name, value);
+ attributes.put(name, value);
+ }
+ }
+ putText(attributes, request, "CdcStartPosition");
+ putText(attributes, request, "CdcStopPosition");
+ putText(attributes, request, "TaskData");
+ }
+
+ private static void requireJsonObject(String member, String value) {
+ try {
+ if (JSON.readTree(value) instanceof ObjectNode) {
+ return;
+ }
+ } catch (JsonProcessingException e) {
+ // reported below
+ }
+ throw invalidParameter(member + " must be a JSON object.");
+ }
+
+ private void requireNotRunning(Found found, String verb) {
+ if (TASK_RUNNING.equals(member(found.resource(), "Status"))) {
+ throw invalidState("Replication task " + arnOf(tasks, found.resource())
+ + " is running and cannot be " + verb + ". Stop it first.");
+ }
+ }
+
+ private static void putText(ObjectNode attributes, JsonNode request, String name) {
+ String value = text(request, name);
+ if (value != null) {
+ attributes.put(name, value);
+ }
+ }
+
+ private static String member(DmsResource resource, String name) {
+ return resource.getAttributes().path(name).asText(null);
+ }
+
+ private static String arnOf(Kind kind, DmsResource resource) {
+ return member(resource, kind.arnMember());
+ }
+
+ private static Integer integer(JsonNode request, String field) {
+ JsonNode node = request == null ? null : request.get(field);
+ if (node == null || node.isNull()) {
+ return null;
+ }
+ if (!node.isIntegralNumber() || !node.canConvertToInt()) {
+ throw serialization(field + " must be an integer.");
+ }
+ return node.intValue();
+ }
+
+ private static boolean bool(JsonNode request, String field, boolean defaultValue) {
+ JsonNode node = request == null ? null : request.get(field);
+ if (node == null || node.isNull()) {
+ return defaultValue;
+ }
+ if (!node.isBoolean()) {
+ throw serialization(field + " must be a boolean.");
+ }
+ return node.booleanValue();
+ }
+
+ private static double now() {
+ return Instant.now().toEpochMilli() / 1000.0;
+ }
+
+ /** DMS-generated ARN suffixes are 26 characters of the base32 alphabet. */
+ private static String randomResourceId() {
+ StringBuilder id = new StringBuilder(26);
+ for (int i = 0; i < 26; i++) {
+ id.append(RESOURCE_ID_ALPHABET.charAt(RANDOM.nextInt(RESOURCE_ID_ALPHABET.length())));
+ }
+ return id.toString();
+ }
+
+ private static AwsException invalidState(String message) {
+ return new AwsException("InvalidResourceStateFault", message, 400);
}
private ReplicationSubnetGroup buildSubnetGroup(String identifier, String description,
@@ -271,36 +889,41 @@ public class DmsService implements Resettable {
}
private static List<String> identifierFilters(JsonNode request) {
+ return filters(request, Set.of(SUBNET_GROUP_ID_FILTER)).getOrDefault(SUBNET_GROUP_ID_FILTER, List.of())
+ .stream().map(value -> value.toLowerCase(Locale.ROOT)).toList();
+ }
+
+ /** Parses {@code Filters} into name to values, in request order, rejecting a name not in {@code validNames}. */
+ private static Map<String, List<String>> filters(JsonNode request, Set<String> validNames) {
+ Map<String, List<String>> parsed = new LinkedHashMap<>();
JsonNode filters = request == null ? null : request.get("Filters");
if (filters == null || filters.isNull()) {
- return List.of();
+ return parsed;
}
if (!filters.isArray()) {
throw serialization("Filters must be a list of Name and Values pairs.");
}
- List<String> identifiers = new ArrayList<>();
for (JsonNode filter : filters) {
if (!filter.isObject()) {
throw serialization("Filters must be a list of Name and Values pairs.");
}
String name = text(filter, "Name");
- if (!SUBNET_GROUP_ID_FILTER.equals(name)) {
+ if (!validNames.contains(name)) {
throw invalidParameter("Invalid filter: " + name + ".");
}
JsonNode values = filter.get("Values");
if (values == null || values.isNull()) {
- throw invalidParameter("The filter " + SUBNET_GROUP_ID_FILTER + " must have values.");
+ throw invalidParameter("The filter " + name + " must have values.");
}
if (!values.isArray()) {
throw serialization("Filter Values must be a list of strings.");
}
if (values.isEmpty()) {
- throw invalidParameter("The filter " + SUBNET_GROUP_ID_FILTER + " must have values.");
+ throw invalidParameter("The filter " + name + " must have values.");
}
- stringList(values, "Filter Values")
- .forEach(value -> identifiers.add(value.toLowerCase(Locale.ROOT)));
+ parsed.computeIfAbsent(name, key -> new ArrayList<>()).addAll(stringList(values, "Filter Values"));
}
- return identifiers;
+ return parsed;
}
private static AwsException notFound(String identifier) {
@@ -353,31 +976,49 @@ public class DmsService implements Resettable {
private void updateTags(String resourceArn, String region,
Consumer<Map<String, String>> mutation) {
- String key = subnetGroupKeyForArn(resourceArn, region);
- ReplicationSubnetGroup group = subnetGroups.get(key).orElseThrow(() -> notFoundForArn(resourceArn));
- Map<String, String> tags = new LinkedHashMap<>(group.getTags());
+ String[] parsed = parseArn(resourceArn, region);
+ if (SUBNET_GROUP_ARN_RESOURCE_TYPE.equals(parsed[0])) {
+ String key = storageKey(region, parsed[1].toLowerCase(Locale.ROOT));
+ ReplicationSubnetGroup group = subnetGroups.get(key).orElseThrow(() -> notFoundForArn(resourceArn));
+ Map<String, String> tags = new LinkedHashMap<>(group.getTags());
+ mutation.accept(tags);
+ group.setTags(tags);
+ subnetGroups.put(key, group);
+ return;
+ }
+ Kind kind = kindForArnType(parsed[0], resourceArn);
+ String key = storageKey(region, parsed[1]);
+ DmsResource resource = kind.store().get(key).orElseThrow(() -> notFoundForArn(resourceArn));
+ Map<String, String> tags = new LinkedHashMap<>(resource.getTags());
mutation.accept(tags);
- group.setTags(tags);
- subnetGroups.put(key, group);
+ resource.setTags(tags);
+ kind.store().put(key, resource);
}
- private ReplicationSubnetGroup resolveByArn(String resourceArn, String region) {
- return subnetGroups.get(subnetGroupKeyForArn(resourceArn, region))
- .orElseThrow(() -> notFoundForArn(resourceArn));
+ private Map<String, String> tagsByArn(String resourceArn, String region) {
+ String[] parsed = parseArn(resourceArn, region);
+ if (SUBNET_GROUP_ARN_RESOURCE_TYPE.equals(parsed[0])) {
+ return subnetGroups.get(storageKey(region, parsed[1].toLowerCase(Locale.ROOT)))
+ .orElseThrow(() -> notFoundForArn(resourceArn)).getTags();
+ }
+ return kindForArnType(parsed[0], resourceArn).store().get(storageKey(region, parsed[1]))
+ .orElseThrow(() -> notFoundForArn(resourceArn)).getTags();
}
/**
- * DescribeReplicationSubnetGroups does not return an ARN, so Terraform builds
- * {@code arn:aws:dms:<region>:<account>:subgrp:<id>} itself and tags against that. Anything
- * that is not such an ARN names no DMS resource Floci holds, which is a ResourceNotFoundFault
- * rather than a parameter error.
+ * Splits a DMS ARN into its resource type and resource id. DescribeReplicationSubnetGroups does
+ * not return an ARN, so Terraform builds {@code arn:aws:dms:<region>:<account>:subgrp:<id>}
+ * itself and tags against that; endpoints ({@code endpoint}), replication instances
+ * ({@code rep}) and tasks ({@code task}) carry the ARN DMS minted. Anything that is not such an
+ * ARN names no DMS resource Floci holds, which is a ResourceNotFoundFault rather than a
+ * parameter error.
*
* <p>An ARN naming another account or another Region is treated the same way. Tagging is not a
* cross-account or cross-Region operation on AWS, and here it cannot be one either: storage is
* scoped to the caller's account and keyed by the caller's Region, so honouring a foreign ARN
- * would silently reach the caller's own group of that name instead of the one named.
+ * would silently reach the caller's own resource of that name instead of the one named.
*/
- private String subnetGroupKeyForArn(String resourceArn, String callerRegion) {
+ private String[] parseArn(String resourceArn, String callerRegion) {
AwsArnUtils.Arn arn;
try {
arn = AwsArnUtils.parse(resourceArn);
@@ -385,8 +1026,7 @@ public class DmsService implements Resettable {
throw notFoundForArn(resourceArn);
}
String[] resource = arn.resource().split(":", 2);
- if (!"dms".equals(arn.service()) || resource.length != 2
- || !SUBNET_GROUP_ARN_RESOURCE_TYPE.equals(resource[0]) || resource[1].isBlank()) {
+ if (!"dms".equals(arn.service()) || resource.length != 2 || resource[1].isBlank()) {
throw notFoundForArn(resourceArn);
}
if (!arn.accountId().isBlank() && !arn.accountId().equals(regionResolver.getAccountId())) {
@@ -395,7 +1035,7 @@ public class DmsService implements Resettable {
if (arn.region() != null && !arn.region().isBlank() && !arn.region().equals(callerRegion)) {
throw notFoundForArn(resourceArn);
}
- return storageKey(callerRegion, resource[1].toLowerCase(Locale.ROOT));
+ return resource;
}
private static List<String> arnList(JsonNode request) {
diff --git a/src/main/java/io/github/hectorvent/floci/services/dms/model/DmsResource.java b/src/main/java/io/github/hectorvent/floci/services/dms/model/DmsResource.java
new file mode 100644
index 0000000000..21a17fc127
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/dms/model/DmsResource.java
@@ -0,0 +1,43 @@
+package io.github.hectorvent.floci.services.dms.model;
+
+import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
+import com.fasterxml.jackson.databind.node.JsonNodeFactory;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.quarkus.runtime.annotations.RegisterForReflection;
+
+import java.util.Collections;
+import java.util.LinkedHashMap;
+import java.util.Map;
+
+/**
+ * An endpoint, replication instance or replication task. {@code attributes} holds the resource
+ * in the wire shape DMS returns it in ({@code Endpoint}, {@code ReplicationInstance},
+ * {@code ReplicationTask}), so a describe echoes exactly what was stored. Secret members such as
+ * {@code Password} are never stored, because DMS never returns them.
+ */
+@RegisterForReflection
+@JsonIgnoreProperties(ignoreUnknown = true)
+public class DmsResource {
+
+ private ObjectNode attributes = JsonNodeFactory.instance.objectNode();
+ private Map<String, String> tags = new LinkedHashMap<>();
+
+ public DmsResource() {
+ }
+
+ public ObjectNode getAttributes() {
+ return attributes;
+ }
+
+ public void setAttributes(ObjectNode attributes) {
+ this.attributes = attributes != null ? attributes : JsonNodeFactory.instance.objectNode();
+ }
+
+ public Map<String, String> getTags() {
+ return Collections.unmodifiableMap(new LinkedHashMap<>(tags));
+ }
+
+ public void setTags(Map<String, String> tags) {
+ this.tags = tags != null ? new LinkedHashMap<>(tags) : new LinkedHashMap<>();
+ }
+}
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
index f89a74dc81..1ab7b0c624 100644
--- a/src/main/resources/application.yml
+++ b/src/main/resources/application.yml
@@ -112,6 +112,7 @@ quarkus:
- --initialize-at-run-time=io.github.hectorvent.floci.services.elasticbeanstalk.ElasticBeanstalkService
- --initialize-at-run-time=io.github.hectorvent.floci.services.elb.ElbClassicService
- --initialize-at-run-time=io.github.hectorvent.floci.services.datasync.DataSyncService
+ - --initialize-at-run-time=io.github.hectorvent.floci.services.dms.DmsService
- --initialize-at-run-time=io.github.hectorvent.floci.services.lambda.launcher.LambdaExecutionRoleCredentials
- --initialize-at-run-time=io.github.hectorvent.floci.services.ecs.container.EcsTaskRoleCredentials
- --initialize-at-run-time=io.github.hectorvent.floci.services.ssoportal.SsoPortalService
diff --git a/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java
index d6234dfa59..66d2ff75de 100644
--- a/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java
@@ -278,7 +278,7 @@ class DmsIntegrationTest {
@Test
void unsupportedDmsActionReportsUnknownOperation() {
- dms("CreateReplicationInstance")
+ dms("CreateReplicationConfig")
.body("{}")
.when()
.post("/")
diff --git a/src/test/java/io/github/hectorvent/floci/services/dms/DmsLifecycleIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/dms/DmsLifecycleIntegrationTest.java
new file mode 100644
index 0000000000..a668328cbe
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/dms/DmsLifecycleIntegrationTest.java
@@ -0,0 +1,563 @@
+package io.github.hectorvent.floci.services.dms;
+
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.quarkus.test.junit.QuarkusTest;
+import io.restassured.response.ValidatableResponse;
+import io.restassured.specification.RequestSpecification;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import static io.restassured.RestAssured.given;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.hasItems;
+import static org.hamcrest.Matchers.hasKey;
+import static org.hamcrest.Matchers.hasSize;
+import static org.hamcrest.Matchers.matchesPattern;
+import static org.hamcrest.Matchers.not;
+import static org.hamcrest.Matchers.notNullValue;
+import static org.hamcrest.Matchers.nullValue;
+
+/**
+ * Endpoint, replication instance and replication task lifecycles, driven the way the Terraform
+ * provider drives them: every mutation is read back through a separate describe call rather than
+ * trusted from the mutating call's own response.
+ */
+@QuarkusTest
+class DmsLifecycleIntegrationTest {
+
+ private static final String CONTENT_TYPE = "application/x-amz-json-1.1";
+ private static final String TARGET_PREFIX = "AmazonDMSv20160101.";
+ private static final String ACCOUNT_ID = "723679240095";
+ private static final String AUTH_HEADER =
+ "AWS4-HMAC-SHA256 Credential=" + ACCOUNT_ID + "/20260101/us-east-1/dms/aws4_request";
+ private static final String SUBNET_A = "subnet-default-us-east-1-a";
+ private static final String SUBNET_B = "subnet-default-us-east-1-b";
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @Test
+ void endpointLifecycleIsReadableThroughDescribe() {
+ String arn = createEndpoint("lc-ep-source", "source",
+ ",\"Username\":\"admin\",\"Password\":\"hunter2\",\"ServerName\":\"db.example\","
+ + "\"Port\":3306,\"DatabaseName\":\"app\","
+ + "\"Tags\":[{\"Key\":\"env\",\"Value\":\"test\"}]");
+
+ describeEndpoint("lc-ep-source")
+ .body("Endpoints", hasSize(1))
+ .body("Endpoints[0].EndpointArn", equalTo(arn))
+ .body("Endpoints[0].EndpointType", equalTo("SOURCE"))
+ .body("Endpoints[0].EngineName", equalTo("mysql"))
+ .body("Endpoints[0].Status", equalTo("active"))
+ .body("Endpoints[0].Username", equalTo("admin"))
+ .body("Endpoints[0].ServerName", equalTo("db.example"))
+ .body("Endpoints[0].Port", equalTo(3306))
+ .body("Endpoints[0].DatabaseName", equalTo("app"))
+ .body("Endpoints[0].SslMode", equalTo("none"))
+ .body("Endpoints[0].Password", nullValue());
+
+ dms("DescribeEndpoints")
+ .body("{\"Filters\":[{\"Name\":\"endpoint-arn\",\"Values\":[\"" + arn + "\"]},"
+ + "{\"Name\":\"endpoint-type\",\"Values\":[\"source\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("Endpoints[0].EndpointIdentifier", equalTo("lc-ep-source"));
+
+ dms("ModifyEndpoint")
+ .body("{\"EndpointArn\":\"" + arn + "\",\"Port\":3307,\"SslMode\":\"require\"}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeEndpoint("lc-ep-source")
+ .body("Endpoints[0].Port", equalTo(3307))
+ .body("Endpoints[0].SslMode", equalTo("require"))
+ .body("Endpoints[0].ServerName", equalTo("db.example"));
+
+ dms("ListTagsForResource")
+ .body("{\"ResourceArn\":\"" + arn + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("TagList", hasSize(1))
+ .body("TagList[0].Key", equalTo("env"));
+
+ dms("DeleteEndpoint")
+ .body("{\"EndpointArn\":\"" + arn + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("Endpoint.Status", equalTo("deleting"));
+
+ dms("DescribeEndpoints")
+ .body("{\"Filters\":[{\"Name\":\"endpoint-id\",\"Values\":[\"lc-ep-source\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+ }
+
+ @Test
+ void endpointSettingsMergeOnModifyAndNeverReturnSecrets() {
+ String arn = createEndpoint("lc-ep-s3", "target",
+ ",\"S3Settings\":{\"BucketName\":\"b1\",\"ServiceAccessRoleArn\":\"arn:aws:iam::"
+ + ACCOUNT_ID + ":role/dms\",\"CsvDelimiter\":\",\"},"
+ + "\"MySQLSettings\":{\"ServerName\":\"x\",\"Password\":\"secret\"}");
+
+ dms("ModifyEndpoint")
+ .body("{\"EndpointArn\":\"" + arn + "\",\"S3Settings\":{\"BucketFolder\":\"cdc\"}}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeEndpoint("lc-ep-s3")
+ .body("Endpoints[0].EndpointType", equalTo("TARGET"))
+ .body("Endpoints[0].S3Settings.BucketName", equalTo("b1"))
+ .body("Endpoints[0].S3Settings.BucketFolder", equalTo("cdc"))
+ .body("Endpoints[0].S3Settings.CsvDelimiter", equalTo(","))
+ .body("Endpoints[0].MySQLSettings.ServerName", equalTo("x"))
+ .body("Endpoints[0].MySQLSettings.Password", nullValue());
+
+ dms("ModifyEndpoint")
+ .body("{\"EndpointArn\":\"" + arn + "\",\"ExactSettings\":true,"
+ + "\"S3Settings\":{\"BucketName\":\"b2\"}}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeEndpoint("lc-ep-s3")
+ .body("Endpoints[0].S3Settings.BucketName", equalTo("b2"))
+ .body("Endpoints[0].S3Settings.BucketFolder", nullValue());
+
+ delete("DeleteEndpoint", "EndpointArn", arn);
+ }
+
+ @Test
+ void duplicateEndpointIdentifierIsRejected() {
+ String arn = createEndpoint("lc-ep-dup", "source", "");
+ dms("CreateEndpoint")
+ .body("{\"EndpointIdentifier\":\"lc-ep-dup\",\"EndpointType\":\"source\","
+ + "\"EngineName\":\"mysql\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceAlreadyExistsFault"));
+ delete("DeleteEndpoint", "EndpointArn", arn);
+ }
+
+ @Test
+ void replicationInstanceLifecycleIsReadableThroughDescribe() {
+ dms("CreateReplicationSubnetGroup")
+ .body("{\"ReplicationSubnetGroupIdentifier\":\"lc-rep-subnets\","
+ + "\"ReplicationSubnetGroupDescription\":\"lifecycle\","
+ + "\"SubnetIds\":[\"" + SUBNET_A + "\",\"" + SUBNET_B + "\"]}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ String arn = dms("CreateReplicationInstance")
+ .body("{\"ReplicationInstanceIdentifier\":\"LC-Rep\","
+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\",\"AllocatedStorage\":20,"
+ + "\"ReplicationSubnetGroupIdentifier\":\"lc-rep-subnets\","
+ + "\"PubliclyAccessible\":false,\"MultiAZ\":false,"
+ + "\"AutoMinorVersionUpgrade\":true,\"VpcSecurityGroupIds\":[\"sg-1\"],"
+ + "\"Tags\":[{\"Key\":\"team\",\"Value\":\"data\"}]}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("ReplicationInstance.ReplicationInstanceArn",
+ matchesPattern("arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":rep:[A-Z2-7]{26}"))
+ .extract().path("ReplicationInstance.ReplicationInstanceArn");
+
+ describeInstance("lc-rep")
+ .body("ReplicationInstances", hasSize(1))
+ .body("ReplicationInstances[0].ReplicationInstanceArn", equalTo(arn))
+ .body("ReplicationInstances[0].ReplicationInstanceIdentifier", equalTo("lc-rep"))
+ .body("ReplicationInstances[0].ReplicationInstanceStatus", equalTo("available"))
+ .body("ReplicationInstances[0].ReplicationInstanceClass", equalTo("dms.t3.micro"))
+ .body("ReplicationInstances[0].AllocatedStorage", equalTo(20))
+ .body("ReplicationInstances[0].PubliclyAccessible", equalTo(false))
+ .body("ReplicationInstances[0].VpcSecurityGroups[0].VpcSecurityGroupId", equalTo("sg-1"))
+ .body("ReplicationInstances[0].ReplicationSubnetGroup.ReplicationSubnetGroupIdentifier",
+ equalTo("lc-rep-subnets"))
+ .body("ReplicationInstances[0].ReplicationSubnetGroup.VpcId", equalTo("vpc-default-us-east-1"))
+ .body("ReplicationInstances[0].EngineVersion", notNullValue());
+
+ dms("DeleteReplicationSubnetGroup")
+ .body("{\"ReplicationSubnetGroupIdentifier\":\"lc-rep-subnets\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidResourceStateFault"));
+
+ dms("ModifyReplicationInstance")
+ .body("{\"ReplicationInstanceArn\":\"" + arn + "\",\"ApplyImmediately\":true,"
+ + "\"ReplicationInstanceClass\":\"dms.c5.large\",\"AllocatedStorage\":100}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeInstance("lc-rep")
+ .body("ReplicationInstances[0].ReplicationInstanceClass", equalTo("dms.c5.large"))
+ .body("ReplicationInstances[0].AllocatedStorage", equalTo(100))
+ .body("ReplicationInstances[0].ReplicationInstanceStatus", equalTo("available"));
+
+ dms("ListTagsForResource")
+ .body("{\"ResourceArn\":\"" + arn + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("TagList[0].Key", equalTo("team"));
+
+ dms("DeleteReplicationInstance")
+ .body("{\"ReplicationInstanceArn\":\"" + arn + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("ReplicationInstance.ReplicationInstanceStatus", equalTo("deleting"));
+
+ dms("DescribeReplicationInstances")
+ .body("{\"Filters\":[{\"Name\":\"replication-instance-id\",\"Values\":[\"lc-rep\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+
+ delete("DeleteReplicationSubnetGroup", "ReplicationSubnetGroupIdentifier", "lc-rep-subnets");
+ }
+
+ @Test
+ void replicationInstanceWithoutSubnetGroupUsesDefault() {
+ String arn = createInstance("lc-rep-default");
+ describeInstance("lc-rep-default")
+ .body("ReplicationInstances[0].ReplicationSubnetGroup.ReplicationSubnetGroupIdentifier",
+ equalTo("default"))
+ .body("ReplicationInstances[0].ReplicationSubnetGroup.VpcId", equalTo("vpc-default-us-east-1"))
+ .body("ReplicationInstances[0].ReplicationSubnetGroup.SubnetGroupStatus", equalTo("Complete"))
+ .body("ReplicationInstances[0].ReplicationSubnetGroup.Subnets.SubnetIdentifier",
+ hasItems(SUBNET_A, SUBNET_B));
+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", arn);
+ }
+
+ @Test
+ void replicationInstanceStoresOnlyModeledKerberosMembers() {
+ dms("CreateReplicationInstance")
+ .body("{\"ReplicationInstanceIdentifier\":\"lc-rep-kerberos\","
+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\","
+ + "\"KerberosAuthenticationSettings\":{\"KeyCacheSecretId\":\"krb-cache\","
+ + "\"Krb5FileContents\":\"[libdefaults]\",\"Password\":\"hunter2\"}}")
+ .when().post("/")
+ .then().statusCode(200);
+ describeInstance("lc-rep-kerberos")
+ .body("ReplicationInstances[0].KerberosAuthenticationSettings.KeyCacheSecretId",
+ equalTo("krb-cache"))
+ .body("ReplicationInstances[0].KerberosAuthenticationSettings.Krb5FileContents",
+ equalTo("[libdefaults]"))
+ .body("ReplicationInstances[0].KerberosAuthenticationSettings", not(hasKey("Password")));
+ String arn = describeInstance("lc-rep-kerberos").extract()
+ .path("ReplicationInstances[0].ReplicationInstanceArn");
+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", arn);
+ }
+
+ @Test
+ void replicationInstanceWithUnknownSubnetGroupIsNotFound() {
+ dms("CreateReplicationInstance")
+ .body("{\"ReplicationInstanceIdentifier\":\"lc-rep-nogroup\","
+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\","
+ + "\"ReplicationSubnetGroupIdentifier\":\"lc-never-created\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+ }
+
+ @Test
+ void replicationTaskLifecycleAcrossStartAndStop() {
+ String source = createEndpoint("lc-task-src", "source", "");
+ String target = createEndpoint("lc-task-tgt", "target", "");
+ String instance = createInstance("lc-task-rep");
+
+ String task = dms("CreateReplicationTask")
+ .body("{\"ReplicationTaskIdentifier\":\"lc-task\",\"SourceEndpointArn\":\"" + source + "\","
+ + "\"TargetEndpointArn\":\"" + target + "\",\"ReplicationInstanceArn\":\"" + instance + "\","
+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("%"))
+ + ",\"Tags\":[{\"Key\":\"k\",\"Value\":\"v\"}]}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .extract().path("ReplicationTask.ReplicationTaskArn");
+
+ describeTask("lc-task")
+ .body("ReplicationTasks", hasSize(1))
+ .body("ReplicationTasks[0].ReplicationTaskArn", equalTo(task))
+ .body("ReplicationTasks[0].Status", equalTo("ready"))
+ .body("ReplicationTasks[0].SourceEndpointArn", equalTo(source))
+ .body("ReplicationTasks[0].TargetEndpointArn", equalTo(target))
+ .body("ReplicationTasks[0].ReplicationInstanceArn", equalTo(instance))
+ .body("ReplicationTasks[0].MigrationType", equalTo("full-load"))
+ .body("ReplicationTasks[0].TableMappings", equalTo(mappings("%")));
+
+ dms("DescribeReplicationTasks")
+ .body("{\"Filters\":[{\"Name\":\"endpoint-arn\",\"Values\":[\"" + target + "\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("ReplicationTasks[0].ReplicationTaskIdentifier", equalTo("lc-task"));
+
+ dms("ModifyReplicationTask")
+ .body("{\"ReplicationTaskArn\":\"" + task + "\",\"MigrationType\":\"full-load-and-cdc\","
+ + "\"TableMappings\":" + jsonString(mappings("app")) + "}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeTask("lc-task")
+ .body("ReplicationTasks[0].MigrationType", equalTo("full-load-and-cdc"))
+ .body("ReplicationTasks[0].TableMappings", equalTo(mappings("app")))
+ .body("ReplicationTasks[0].Status", equalTo("ready"));
+
+ dms("StartReplicationTask")
+ .body("{\"ReplicationTaskArn\":\"" + task + "\",\"StartReplicationTaskType\":\"start-replication\"}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeTask("lc-task").body("ReplicationTasks[0].Status", equalTo("running"));
+
+ dms("DeleteReplicationTask")
+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidResourceStateFault"));
+
+ dms("StopReplicationTask")
+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")
+ .when().post("/")
+ .then().statusCode(200);
+
+ describeTask("lc-task").body("ReplicationTasks[0].Status", equalTo("stopped"));
+
+ // The Terraform provider treats exactly this message as "already stopped".
+ dms("StopReplicationTask")
+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidResourceStateFault"))
+ .body("message", containsString("is currently not running"));
+
+ dms("DeleteEndpoint")
+ .body("{\"EndpointArn\":\"" + source + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidResourceStateFault"));
+
+ dms("DeleteReplicationInstance")
+ .body("{\"ReplicationInstanceArn\":\"" + instance + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidResourceStateFault"));
+
+ dms("DeleteReplicationTask")
+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("ReplicationTask.Status", equalTo("deleting"));
+
+ dms("DescribeReplicationTasks")
+ .body("{\"Filters\":[{\"Name\":\"replication-task-id\",\"Values\":[\"lc-task\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+
+ delete("DeleteEndpoint", "EndpointArn", source);
+ delete("DeleteEndpoint", "EndpointArn", target);
+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);
+ }
+
+ @Test
+ void tagsAddAndRemoveOnEndpointInstanceAndTaskArns() {
+ String source = createEndpoint("lc-tag-src", "source", "");
+ String target = createEndpoint("lc-tag-tgt", "target", "");
+ String instance = createInstance("lc-tag-rep");
+ String task = dms("CreateReplicationTask")
+ .body("{\"ReplicationTaskIdentifier\":\"lc-tag-task\",\"SourceEndpointArn\":\"" + source + "\","
+ + "\"TargetEndpointArn\":\"" + target + "\",\"ReplicationInstanceArn\":\"" + instance + "\","
+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("%")) + "}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .extract().path("ReplicationTask.ReplicationTaskArn");
+
+ for (String arn : new String[] {source, instance, task}) {
+ dms("AddTagsToResource")
+ .body("{\"ResourceArn\":\"" + arn + "\",\"Tags\":[{\"Key\":\"keep\",\"Value\":\"" + arn + "\"},"
+ + "{\"Key\":\"drop\",\"Value\":\"x\"}]}")
+ .when().post("/")
+ .then().statusCode(200);
+ dms("RemoveTagsFromResource")
+ .body("{\"ResourceArn\":\"" + arn + "\",\"TagKeys\":[\"drop\"]}")
+ .when().post("/")
+ .then().statusCode(200);
+ }
+
+ // Each ARN reads back only its own tags: a mutation routed to the wrong resource kind shows here.
+ for (String arn : new String[] {source, instance, task}) {
+ dms("ListTagsForResource")
+ .body("{\"ResourceArn\":\"" + arn + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("TagList", hasSize(1))
+ .body("TagList[0].Key", equalTo("keep"))
+ .body("TagList[0].Value", equalTo(arn));
+ }
+ dms("ListTagsForResource")
+ .body("{\"ResourceArn\":\"" + target + "\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("TagList", hasSize(0));
+
+ describeEndpoint("lc-tag-src").body("Endpoints[0].EndpointArn", equalTo(source));
+ describeInstance("lc-tag-rep").body("ReplicationInstances[0].ReplicationInstanceArn", equalTo(instance));
+ describeTask("lc-tag-task").body("ReplicationTasks[0].ReplicationTaskArn", equalTo(task));
+
+ delete("DeleteReplicationTask", "ReplicationTaskArn", task);
+ delete("DeleteEndpoint", "EndpointArn", source);
+ delete("DeleteEndpoint", "EndpointArn", target);
+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);
+ }
+
+ @Test
+ void replicationTaskNamingUnknownEndpointIsNotFound() {
+ dms("CreateReplicationTask")
+ .body("{\"ReplicationTaskIdentifier\":\"lc-task-orphan\","
+ + "\"SourceEndpointArn\":\"arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":endpoint:NOPE\","
+ + "\"TargetEndpointArn\":\"arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":endpoint:NOPE2\","
+ + "\"ReplicationInstanceArn\":\"arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":rep:NOPE\","
+ + "\"MigrationType\":\"full-load\",\"TableMappings\":\"{}\"}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+ }
+
+ @Test
+ void replicationTaskWithReversedEndpointsIsRejected() {
+ String source = createEndpoint("lc-rev-src", "source", "");
+ String target = createEndpoint("lc-rev-tgt", "target", "");
+ String instance = createInstance("lc-rev-rep");
+ dms("CreateReplicationTask")
+ .body("{\"ReplicationTaskIdentifier\":\"lc-task-reversed\","
+ + "\"SourceEndpointArn\":\"" + target + "\","
+ + "\"TargetEndpointArn\":\"" + source + "\","
+ + "\"ReplicationInstanceArn\":\"" + instance + "\","
+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("app")) + "}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidParameterValueException"));
+ dms("DescribeReplicationTasks")
+ .body("{\"Filters\":[{\"Name\":\"replication-task-id\",\"Values\":[\"lc-task-reversed\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);
+ delete("DeleteEndpoint", "EndpointArn", source);
+ delete("DeleteEndpoint", "EndpointArn", target);
+ }
+
+ @Test
+ void replicationTaskWithSourceEndpointAsTargetIsRejected() {
+ String source = createEndpoint("lc-src-as-tgt", "source", "");
+ String instance = createInstance("lc-src-as-tgt-rep");
+ dms("CreateReplicationTask")
+ .body("{\"ReplicationTaskIdentifier\":\"lc-task-src-as-tgt\","
+ + "\"SourceEndpointArn\":\"" + source + "\","
+ + "\"TargetEndpointArn\":\"" + source + "\","
+ + "\"ReplicationInstanceArn\":\"" + instance + "\","
+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("app")) + "}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("InvalidParameterValueException"))
+ .body("message", containsString("TargetEndpointArn"));
+ dms("DescribeReplicationTasks")
+ .body("{\"Filters\":[{\"Name\":\"replication-task-id\",\"Values\":[\"lc-task-src-as-tgt\"]}]}")
+ .when().post("/")
+ .then()
+ .statusCode(400)
+ .body("__type", equalTo("ResourceNotFoundFault"));
+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);
+ delete("DeleteEndpoint", "EndpointArn", source);
+ }
+
+ private static String mappings(String schema) {
+ return "{\"rules\":[{\"rule-type\":\"selection\",\"rule-id\":\"1\",\"rule-name\":\"1\","
+ + "\"object-locator\":{\"schema-name\":\"" + schema + "\",\"table-name\":\"%\"},"
+ + "\"rule-action\":\"include\"}]}";
+ }
+
+ private static String jsonString(String value) {
+ return "\"" + value.replace("\\", "\\\\").replace("\"", "\\\"") + "\"";
+ }
+
+ private static String createEndpoint(String identifier, String type, String extraMembers) {
+ return dms("CreateEndpoint")
+ .body("{\"EndpointIdentifier\":\"" + identifier + "\",\"EndpointType\":\"" + type + "\","
+ + "\"EngineName\":\"" + ("target".equals(type) ? "s3" : "mysql") + "\"" + extraMembers + "}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .body("Endpoint.Password", nullValue())
+ .extract().path("Endpoint.EndpointArn");
+ }
+
+ private static String createInstance(String identifier) {
+ return dms("CreateReplicationInstance")
+ .body("{\"ReplicationInstanceIdentifier\":\"" + identifier + "\","
+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\"}")
+ .when().post("/")
+ .then()
+ .statusCode(200)
+ .extract().path("ReplicationInstance.ReplicationInstanceArn");
+ }
+
+ private static ValidatableResponse describeEndpoint(String identifier) {
+ return describe("DescribeEndpoints", "endpoint-id", identifier);
+ }
+
+ private static ValidatableResponse describeInstance(String identifier) {
+ return describe("DescribeReplicationInstances", "replication-instance-id", identifier);
+ }
+
+ private static ValidatableResponse describeTask(String identifier) {
+ return describe("DescribeReplicationTasks", "replication-task-id", identifier);
+ }
+
+ private static ValidatableResponse describe(String action, String filter,
+ String value) {
+ return dms(action)
+ .body("{\"Filters\":[{\"Name\":\"" + filter + "\",\"Values\":[\"" + value + "\"]}]}")
+ .when().post("/")
+ .then().statusCode(200);
+ }
+
+ private static void delete(String action, String member, String value) {
+ dms(action)
+ .body("{\"" + member + "\":\"" + value + "\"}")
+ .when().post("/")
+ .then().statusCode(200);
+ }
+
+ private static RequestSpecification dms(String action) {
+ return given()
+ .contentType(CONTENT_TYPE)
+ .header("X-Amz-Target", TARGET_PREFIX + action)
+ .header("Authorization", AUTH_HEADER);
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java
index 3da3b7659e..a8dd1ccd96 100644
--- a/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java
@@ -30,6 +30,7 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyList;
import static org.mockito.ArgumentMatchers.anyMap;
+import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.when;
@@ -48,6 +49,8 @@ class DmsServiceTest {
StorageFactory storageFactory = mock(StorageFactory.class);
AccountAwareStorageBackend<ReplicationSubnetGroup> store =
AccountAwareStorageBackend.inMemory(ACCOUNT_ID);
+ when(storageFactory.create(eq("dms"), anyString(), any(TypeReference.class)))
+ .thenAnswer(invocation -> AccountAwareStorageBackend.inMemory(ACCOUNT_ID));
when(storageFactory.create(eq("dms"), eq("dms-replication-subnet-groups.json"), any(TypeReference.class)))
.thenReturn((AccountAwareStorageBackend) store);