Light, fluffy, and always free - The AWS Local Emulator alternative
feat(dms): add endpoint, replication instance and replication task lifecycles (#5001)
Allen Sanborne7ca2ece22c8
diff --git a/docs/services/dms.md b/docs/services/dms.mdindex d7a07233ad..d406c2cb63 100644--- a/docs/services/dms.md+++ b/docs/services/dms.md@@ -3,10 +3,12 @@**Protocol:** AWS JSON 1.1**Signing name:** `dms`-Floci emulates the replication subnet group lifecycle, which is what-`aws_dms_replication_subnet_group` needs to plan and apply. Subnets are resolved against+Floci emulates the replication subnet group, endpoint, replication instance and+replication task lifecycles, which is what `aws_dms_replication_subnet_group`,+`aws_dms_endpoint`, `aws_dms_s3_endpoint`, `aws_dms_replication_instance` and+`aws_dms_replication_task` need to plan, apply and destroy. Subnets are resolved againstthe emulated EC2 service, so the VPC and Availability Zones a group reports are the ones-those subnets actually have.+those subnets actually have. No data is migrated: the resources are control-plane state.## Supported Actions@@ -19,6 +21,20 @@ those subnets actually have.| `ListTagsForResource` | Lists the tags on one or more DMS resource ARNs. || `AddTagsToResource` | Merges tags into the resource, overwriting by key. || `RemoveTagsFromResource` | Removes the named tag keys from the resource. |+| `CreateEndpoint` | Creates a source or target endpoint; secret members are accepted and never returned. |+| `DescribeEndpoints` | Lists endpoints, filtered by `endpoint-arn`, `endpoint-type`, `endpoint-id` or `engine-name`. |+| `ModifyEndpoint` | Updates an endpoint; settings structures merge unless `ExactSettings` is true. |+| `DeleteEndpoint` | Deletes an endpoint that no replication task uses. |+| `CreateReplicationInstance` | Creates a replication instance, `available` immediately. |+| `DescribeReplicationInstances` | Lists replication instances, filtered by `replication-instance-arn`, `replication-instance-id`, `replication-instance-class` or `engine-version`. |+| `ModifyReplicationInstance` | Updates a replication instance, applied immediately. |+| `DeleteReplicationInstance` | Deletes a replication instance that no replication task uses. |+| `CreateReplicationTask` | Creates a replication task between two existing endpoints on an existing instance, `ready` immediately. |+| `DescribeReplicationTasks` | Lists replication tasks, filtered by `replication-task-arn`, `replication-task-id`, `migration-type`, `endpoint-arn` or `replication-instance-arn`. |+| `ModifyReplicationTask` | Updates a replication task that is not running. |+| `DeleteReplicationTask` | Deletes a replication task that is not running. |+| `StartReplicationTask` | Moves a task to `running`. |+| `StopReplicationTask` | Moves a running task to `stopped`. |<!-- floci:actions:end -->## Behaviour@@ -53,13 +69,54 @@ those subnets actually have.not a serialization one.- Groups are scoped per account and Region and persist through `StorageFactory`.+### Endpoints, replication instances and replication tasks++- Each resource gets an ARN of the form `arn:aws:dms:<region>:<account>:<type>:<id>`, where+ `<type>` is `endpoint`, `rep` or `task` and `<id>` is 26 characters of the base32+ alphabet, or the request's `ResourceIdentifier` when one is given. Update and delete+ calls address the resource by that ARN.+- Identifiers must begin with a letter, contain only ASCII letters, digits and hyphens,+ and not end with a hyphen or contain two consecutive hyphens (63 characters for an+ instance, 255 otherwise). A replication instance identifier is stored lowercase, as AWS+ documents; endpoint and task identifiers keep their case. A duplicate identifier returns+ `ResourceAlreadyExistsFault`.+- A describe whose `Filters` match nothing returns `ResourceNotFoundFault`, which the+ Terraform provider reads as "gone". Without filters, no resources is an empty list.+ Filter values compare case-insensitively. Describes paginate on `MaxRecords` and+ `Marker` as subnet groups do.+- **Endpoints** are `active` from creation. `EndpointType` is accepted lowercase and returned+ uppercase (`SOURCE`, `TARGET`), as AWS does. `SslMode` defaults to `none`. Engine settings+ structures (`MySQLSettings`, `S3Settings` and the rest) are stored as sent and returned+ as stored, except that every member the DMS model types as a secret (`Password`,+ `SaslPassword`, `AuthPassword` and so on) is dropped, because DMS never returns them.+ `ModifyEndpoint` merges a settings structure into the stored one member by member, or+ replaces it when `ExactSettings` is true.+- **Replication instances** are `available` from creation and stay so through a modify. A+ named `ReplicationSubnetGroupIdentifier` must exist (otherwise `ResourceNotFoundFault`)+ and is returned as the full `ReplicationSubnetGroup` structure; without one the instance+ reports the `default` group. Defaults: `AllocatedStorage` 50, `EngineVersion` `3.5.4`,+ `PubliclyAccessible` true, `MultiAZ` false, `AutoMinorVersionUpgrade` true,+ `NetworkType` `IPV4`, `AvailabilityZone` `<region>a`. `ModifyReplicationInstance` applies+ every change at once whatever `ApplyImmediately` says.+- **Replication tasks** need existing source and target endpoints and an existing instance+ (otherwise `ResourceNotFoundFault`). `TableMappings` and `ReplicationTaskSettings` must be+ JSON objects and are returned exactly as sent. A task is `ready` from creation;+ `StartReplicationTask` makes it `running` and `StopReplicationTask` makes it `stopped`.+ Stopping a task that is not running returns `InvalidResourceStateFault` with "is+ currently not running", the message the Terraform provider treats as already stopped.+ Modifying or deleting a running task returns `InvalidResourceStateFault`.+- Deleting an endpoint or instance that a task references, or a subnet group that an+ instance uses, returns `InvalidResourceStateFault`. A delete returns the resource with+ status `deleting`, and it is gone from the next describe.+### Tagging-`Tags` on `CreateReplicationSubnetGroup` are stored with the group, and the tagging trio-works against the group's ARN, which is-`arn:aws:dms:<region>:<account>:subgrp:<identifier>`. DMS does not return that ARN from+`Tags` on every create are stored with the resource, and the tagging trio works against+the resource's ARN. For a subnet group that is+`arn:aws:dms:<region>:<account>:subgrp:<identifier>`; DMS does not return that ARN from`DescribeReplicationSubnetGroups`, so Terraform builds it client side and Floci parses it-back the same way.+back the same way. Endpoints, instances and tasks are tagged through the ARN their create+returned.- `AddTagsToResource` merges by key, so re-tagging an existing key overwrites its value.- `RemoveTagsFromResource` removes the named keys and ignores keys that are not present.@@ -71,15 +128,24 @@ back the same way.- An ARN that is unparseable, names a different DMS resource type, names another accountor Region, or names a group that does not exist returns `ResourceNotFoundFault`. Taggingis not a cross-account or cross-Region operation on AWS and is not one here either.-- Deleting a group deletes its tags with it.+- Deleting a resource deletes its tags with it.## Limitations-- **Only the subnet group lifecycle is implemented.** Replication instances, endpoints,- and replication tasks return `UnknownOperationException`.-- **Tagging covers replication subnet groups only.** The tagging trio is implemented, but- a `ResourceArn` naming a replication instance, endpoint, or task returns- `ResourceNotFoundFault` because Floci holds no such resource.+- **No migration runs.** A started task stays `running` until stopped; a full-load task+ never finishes on its own, and `ReplicationTaskStats`, `StopReason` and+ `LastFailureMessage` are not reported.+- **Values AWS fills in are not invented.** An endpoint does not report+ `EngineDisplayName`, `ExternalId` or the settings structure AWS derives from top-level+ connection fields, and a task without `ReplicationTaskSettings` reports none rather than+ the AWS defaults. Instances report no private or public IP addresses.+- **`CdcStartTime` is ignored.** `CreateReplicationTask`, `ModifyReplicationTask` and+ `StartReplicationTask` accept it, but it is not stored or returned. `CdcStartPosition` and+ `CdcStopPosition` are stored by create and modify, and ignored by `StartReplicationTask`.+- **Not implemented:** `TestConnection`, `DescribeConnections`, `MoveReplicationTask`,+ `DescribeEndpointTypes`, `DescribeOrderableReplicationInstances`, replication configs+ (`aws_dms_replication_config`, serverless DMS), event subscriptions, certificates and the+ `WithoutSettings` flag of `DescribeReplicationTasks`.- **No `ModifyReplicationSubnetGroup`.** A subnet change has to be a delete and recreate.See the [AWS DMS API Reference](https://docs.aws.amazon.com/dms/latest/APIReference/Welcome.html).diff --git a/docs/services/index.md b/docs/services/index.mdindex 668f03b02f..67bb963fa1 100644--- a/docs/services/index.md+++ b/docs/services/index.md@@ -73,7 +73,7 @@ Operation counts are exact. For dispatch-table services (Query and JSON 1.1) eac| [Lake Formation](lakeformation.md) | `POST /<Action>` | REST JSON | 16 || [Neptune](neptune.md) | `POST /` with `Action=` param + Gremlin TCP proxy | Query + WebSocket | 14 || [DocumentDB](docdb.md) | `POST /` with `Action=` param + MongoDB container | Query + MongoDB wire | 13 |-| [DMS](dms.md) | `POST /` + `X-Amz-Target: AmazonDMSv20160101.*` | JSON 1.1 | 6 |+| [DMS](dms.md) | `POST /` + `X-Amz-Target: AmazonDMSv20160101.*` | JSON 1.1 | 20 || [Redshift](redshift.md) | `POST /` with `Action=` param + PostgreSQL container | Query + PostgreSQL wire (+ CFN) | 35 || [Redshift Data API](redshift-data.md) | `POST /` + `X-Amz-Target: RedshiftData.*` | JSON 1.1 | 11 || [Redshift Serverless](redshift-serverless.md) | `POST /` + `X-Amz-Target: RedshiftServerless.*` | JSON 1.1 | 14 |diff --git a/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java b/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.javaindex 652b02591c..47e78925db 100644--- a/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java+++ b/src/main/java/io/github/hectorvent/floci/services/dms/DmsJsonHandler.java@@ -5,12 +5,15 @@ import com.fasterxml.jackson.databind.ObjectMapper;import com.fasterxml.jackson.databind.node.ArrayNode;import com.fasterxml.jackson.databind.node.ObjectNode;import io.github.hectorvent.floci.core.common.PaginatedResult;+import io.github.hectorvent.floci.services.dms.model.DmsResource;import io.github.hectorvent.floci.services.dms.model.ReplicationSubnetGroup;import io.github.hectorvent.floci.services.dms.model.ResourceTag;import jakarta.enterprise.context.ApplicationScoped;import jakarta.inject.Inject;import jakarta.ws.rs.core.Response;+import java.util.function.Function;+@ApplicationScopedpublic class DmsJsonHandler {@@ -62,10 +65,66 @@ public class DmsJsonHandler {service.removeTagsFromResource(request, region);yield Response.ok(objectMapper.createObjectNode()).build();}+ case "CreateEndpoint" -> single("Endpoint", service.createEndpoint(request, region).getAttributes());+ case "DescribeEndpoints" -> page("Endpoints", service.describeEndpoints(request, region),+ DmsResource::getAttributes);+ case "ModifyEndpoint" -> single("Endpoint", service.modifyEndpoint(request, region).getAttributes());+ case "DeleteEndpoint" -> single("Endpoint", service.deleteEndpoint(request, region).getAttributes());+ case "CreateReplicationInstance" -> single("ReplicationInstance",+ instance(service.createReplicationInstance(request, region), region));+ case "DescribeReplicationInstances" -> page("ReplicationInstances",+ service.describeReplicationInstances(request, region), item -> instance(item, region));+ case "ModifyReplicationInstance" -> single("ReplicationInstance",+ instance(service.modifyReplicationInstance(request, region), region));+ case "DeleteReplicationInstance" -> single("ReplicationInstance",+ instance(service.deleteReplicationInstance(request, region), region));+ case "CreateReplicationTask" -> single("ReplicationTask",+ service.createReplicationTask(request, region).getAttributes());+ case "DescribeReplicationTasks" -> page("ReplicationTasks",+ service.describeReplicationTasks(request, region), DmsResource::getAttributes);+ case "ModifyReplicationTask" -> single("ReplicationTask",+ service.modifyReplicationTask(request, region).getAttributes());+ case "DeleteReplicationTask" -> single("ReplicationTask",+ service.deleteReplicationTask(request, region).getAttributes());+ case "StartReplicationTask" -> single("ReplicationTask",+ service.startReplicationTask(request, region).getAttributes());+ case "StopReplicationTask" -> single("ReplicationTask",+ service.stopReplicationTask(request, region).getAttributes());default -> null;};}+ private Response single(String member, ObjectNode item) {+ ObjectNode response = objectMapper.createObjectNode();+ response.set(member, item);+ return Response.ok(response).build();+ }++ private Response page(String member, PaginatedResult<DmsResource> page,+ Function<DmsResource, ObjectNode> render) {+ ObjectNode response = objectMapper.createObjectNode();+ ArrayNode items = response.putArray(member);+ page.items().forEach(item -> items.add(render.apply(item)));+ if (page.nextToken() != null) {+ response.put("Marker", page.nextToken());+ }+ return Response.ok(response).build();+ }++ /**+ * Expands the stored subnet group identifier into the full {@code ReplicationSubnetGroup}+ * structure DMS returns, read at describe time so it reflects the group as it is now.+ */+ private ObjectNode instance(DmsResource resource, String region) {+ ObjectNode node = resource.getAttributes().deepCopy();+ String groupId = node.path("ReplicationSubnetGroup").path("ReplicationSubnetGroupIdentifier").asText(null);+ if (groupId != null) {+ service.findReplicationSubnetGroup(region, groupId)+ .ifPresent(group -> node.set("ReplicationSubnetGroup", subnetGroup(group)));+ }+ return node;+ }+private ObjectNode tagNode(ResourceTag tag) {ObjectNode node = objectMapper.createObjectNode();node.put("Key", tag.key());diff --git a/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java b/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.javaindex 60b982f772..2c61cde28c 100644--- a/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java+++ b/src/main/java/io/github/hectorvent/floci/services/dms/DmsService.java@@ -1,7 +1,12 @@package io.github.hectorvent.floci.services.dms;+import com.fasterxml.jackson.core.JsonProcessingException;import com.fasterxml.jackson.core.type.TypeReference;import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ArrayNode;+import com.fasterxml.jackson.databind.node.JsonNodeFactory;+import com.fasterxml.jackson.databind.node.ObjectNode;import io.github.hectorvent.floci.core.common.AwsArnUtils;import io.github.hectorvent.floci.core.common.AwsException;import io.github.hectorvent.floci.core.common.PaginatedResult;@@ -10,6 +15,7 @@ import io.github.hectorvent.floci.core.common.RegionResolver;import io.github.hectorvent.floci.core.common.Resettable;import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;import io.github.hectorvent.floci.core.storage.StorageFactory;+import io.github.hectorvent.floci.services.dms.model.DmsResource;import io.github.hectorvent.floci.services.dms.model.ReplicationSubnetGroup;import io.github.hectorvent.floci.services.dms.model.ResourceTag;import io.github.hectorvent.floci.services.ec2.Ec2Service;@@ -17,12 +23,15 @@ import io.github.hectorvent.floci.services.ec2.model.Subnet;import jakarta.enterprise.context.ApplicationScoped;import jakarta.inject.Inject;+import java.security.SecureRandom;+import java.time.Instant;import java.util.ArrayList;import java.util.LinkedHashMap;import java.util.List;import java.util.Locale;import java.util.Map;import java.util.Objects;+import java.util.Optional;import java.util.Set;import java.util.function.Consumer;import java.util.regex.Pattern;@@ -39,7 +48,50 @@ public class DmsService implements Resettable {private static final int MAXIMUM_MAX_RECORDS = 100;private static final Set<String> RESERVED_TAG_PREFIXES = Set.of("aws:", "dms:");+ /** Endpoint, replication instance and task identifiers share one shape in the DMS API reference. */+ private static final Pattern RESOURCE_IDENTIFIER = Pattern.compile("[A-Za-z][0-9A-Za-z-]*");+ private static final int RESOURCE_IDENTIFIER_ARN_MAX = 31;+ private static final String RESOURCE_ID_ALPHABET = "ABCDEFGHIJKLMNOPQRSTUVWXYZ234567";+ private static final SecureRandom RANDOM = new SecureRandom();+ private static final ObjectMapper JSON = new ObjectMapper();+ private static final String TASK_RUNNING = "running";++ /** Members of {@code Endpoint} that are strings on both CreateEndpoint and ModifyEndpoint. */+ private static final List<String> ENDPOINT_STRING_MEMBERS = List.of("EngineName", "Username",+ "ServerName", "DatabaseName", "ExtraConnectionAttributes", "CertificateArn",+ "ServiceAccessRoleArn", "ExternalTableDefinition");+ /** The engine-specific settings structures CreateEndpoint and ModifyEndpoint accept. */+ private static final List<String> ENDPOINT_SETTINGS = List.of("DynamoDbSettings", "S3Settings",+ "DmsTransferSettings", "MongoDbSettings", "KinesisSettings", "KafkaSettings",+ "ElasticsearchSettings", "NeptuneSettings", "RedshiftSettings", "PostgreSQLSettings",+ "MySQLSettings", "OracleSettings", "SybaseSettings", "MicrosoftSQLServerSettings",+ "IBMDb2Settings", "DocDbSettings", "RedisSettings", "GcpMySQLSettings", "TimestreamSettings");+ /**+ * Every member the DMS model types as {@code SecretString}. DMS accepts them but never returns+ * them, so they are dropped before a settings structure is stored.+ */+ private static final List<String> SECRET_MEMBERS = List.of("AsmPassword", "AuthPassword",+ "CertificatePem", "Password", "S3ObjectUrl", "SaslPassword", "SecurityDbEncryption",+ "SelectionRules", "SslClientKeyPassword");+ private static final Set<String> SSL_MODES = Set.of("none", "require", "verify-ca", "verify-full");+ private static final Set<String> MIGRATION_TYPES = Set.of("full-load", "cdc", "full-load-and-cdc");+ private static final Set<String> START_TYPES =+ Set.of("start-replication", "resume-processing", "reload-target");++ /** One of the three ARN-addressed resource families and the names DMS uses for it. */+ private record Kind(String label, AccountAwareStorageBackend<DmsResource> store, String arnType,+ String idMember, String arnMember, int maxIdLength, boolean lowercaseId,+ Map<String, List<String>> filters) {+ }++ /** A stored resource together with the storage key it lives under. */+ private record Found(Kind kind, String key, DmsResource resource) {+ }+private final AccountAwareStorageBackend<ReplicationSubnetGroup> subnetGroups;+ private final Kind endpoints;+ private final Kind instances;+ private final Kind tasks;private final Ec2Service ec2Service;private final RegionResolver regionResolver;@@ -47,10 +99,251 @@ public class DmsService implements Resettable {public DmsService(StorageFactory storageFactory, Ec2Service ec2Service, RegionResolver regionResolver) {this.subnetGroups = storageFactory.create("dms", "dms-replication-subnet-groups.json",new TypeReference<Map<String, ReplicationSubnetGroup>>() {});+ TypeReference<Map<String, DmsResource>> type = new TypeReference<>() {};+ this.endpoints = new Kind("Endpoint", storageFactory.create("dms", "dms-endpoints.json", type),+ "endpoint", "EndpointIdentifier", "EndpointArn", 255, false,+ Map.of("endpoint-arn", List.of("EndpointArn"),+ "endpoint-type", List.of("EndpointType"),+ "endpoint-id", List.of("EndpointIdentifier"),+ "engine-name", List.of("EngineName")));+ this.instances = new Kind("Replication instance",+ storageFactory.create("dms", "dms-replication-instances.json", type),+ "rep", "ReplicationInstanceIdentifier", "ReplicationInstanceArn", 63, true,+ Map.of("replication-instance-arn", List.of("ReplicationInstanceArn"),+ "replication-instance-id", List.of("ReplicationInstanceIdentifier"),+ "replication-instance-class", List.of("ReplicationInstanceClass"),+ "engine-version", List.of("EngineVersion")));+ this.tasks = new Kind("Replication task",+ storageFactory.create("dms", "dms-replication-tasks.json", type),+ "task", "ReplicationTaskIdentifier", "ReplicationTaskArn", 255, false,+ Map.of("replication-task-arn", List.of("ReplicationTaskArn"),+ "replication-task-id", List.of("ReplicationTaskIdentifier"),+ "migration-type", List.of("MigrationType"),+ "endpoint-arn", List.of("SourceEndpointArn", "TargetEndpointArn"),+ "replication-instance-arn", List.of("ReplicationInstanceArn")));this.ec2Service = ec2Service;this.regionResolver = regionResolver;}+ // ---------------------------------------------------------------- endpoints++ public synchronized DmsResource createEndpoint(JsonNode request, String region) {+ String identifier = requireResourceIdentifier(endpoints, request);+ String endpointType = requireEndpointType(request);+ if (text(request, "EngineName") == null || text(request, "EngineName").isBlank()) {+ throw invalidParameter("The parameter EngineName must be provided and must not be blank.");+ }+ ObjectNode attributes = JsonNodeFactory.instance.objectNode();+ attributes.put("EndpointType", endpointType);+ attributes.put("Status", "active");+ attributes.put("SslMode", "none");+ putText(attributes, request, "KmsKeyId");+ applyEndpointMembers(attributes, request, false);+ return create(endpoints, region, identifier, attributes, request);+ }++ public PaginatedResult<DmsResource> describeEndpoints(JsonNode request, String region) {+ return describe(endpoints, request, region);+ }++ /**+ * Settings structures merge member by member into what is stored, as DMS documents for+ * {@code ExactSettings} false (the default); with {@code ExactSettings} true a structure in the+ * request replaces the stored one.+ */+ public synchronized DmsResource modifyEndpoint(JsonNode request, String region) {+ Found found = requireByArn(endpoints, request, endpoints.arnMember(), region);+ ObjectNode attributes = found.resource().getAttributes().deepCopy();+ rename(endpoints, region, found, attributes, request);+ if (text(request, "EndpointType") != null) {+ attributes.put("EndpointType", requireEndpointType(request));+ }+ applyEndpointMembers(attributes, request, bool(request, "ExactSettings", false));+ return save(found, attributes);+ }++ public synchronized DmsResource deleteEndpoint(JsonNode request, String region) {+ Found found = requireByArn(endpoints, request, endpoints.arnMember(), region);+ String arn = arnOf(endpoints, found.resource());+ boolean inUse = regionScan(tasks, region).stream()+ .anyMatch(task -> arn.equals(member(task, "SourceEndpointArn"))+ || arn.equals(member(task, "TargetEndpointArn")));+ if (inUse) {+ throw invalidState("Endpoint " + arn + " is in use by a replication task and cannot be deleted.");+ }+ DmsResource removed = remove(found);+ removed.getAttributes().put("Status", "deleting");+ return removed;+ }++ // ---------------------------------------------------------------- replication instances++ /**+ * The instance is {@code available} as soon as it is created: there is no host to provision, and+ * the Terraform waiter accepts {@code available} on its first poll.+ */+ public synchronized DmsResource createReplicationInstance(JsonNode request, String region) {+ String identifier = requireResourceIdentifier(instances, request);+ if (text(request, "ReplicationInstanceClass") == null) {+ throw invalidParameter("The parameter ReplicationInstanceClass must be provided.");+ }+ ObjectNode attributes = JsonNodeFactory.instance.objectNode();+ attributes.put("ReplicationInstanceStatus", "available");+ attributes.put("AllocatedStorage", 50);+ attributes.put("InstanceCreateTime", now());+ attributes.putArray("VpcSecurityGroups");+ attributes.put("AvailabilityZone", region + "a");+ attributes.put("PreferredMaintenanceWindow", "sun:06:00-sun:06:30");+ attributes.put("MultiAZ", false);+ attributes.put("EngineVersion", "3.5.4");+ attributes.put("AutoMinorVersionUpgrade", true);+ attributes.put("PubliclyAccessible", bool(request, "PubliclyAccessible", true));+ attributes.put("NetworkType", "IPV4");+ attributes.putArray("ReplicationInstancePrivateIpAddresses");+ attributes.putArray("ReplicationInstancePublicIpAddresses");+ putText(attributes, request, "AvailabilityZone");+ putText(attributes, request, "KmsKeyId");+ putText(attributes, request, "DnsNameServers");++ String subnetGroup = text(request, "ReplicationSubnetGroupIdentifier");+ if (subnetGroup == null) {+ // DMS falls back to the account's "default" group, which is why that name is reserved.+ subnetGroup = "default";+ } else {+ subnetGroup = subnetGroup.toLowerCase(Locale.ROOT);+ if (subnetGroups.get(storageKey(region, subnetGroup)).isEmpty()) {+ throw notFound(subnetGroup);+ }+ }+ attributes.putObject("ReplicationSubnetGroup").put("ReplicationSubnetGroupIdentifier", subnetGroup);+ applyInstanceMembers(attributes, request);+ return create(instances, region, identifier, attributes, request);+ }++ public PaginatedResult<DmsResource> describeReplicationInstances(JsonNode request, String region) {+ return describe(instances, request, region);+ }++ /**+ * Applied at once whatever {@code ApplyImmediately} says: Floci has no maintenance window to+ * defer to, and a change parked in {@code PendingModifiedValues} would read back as drift.+ */+ public synchronized DmsResource modifyReplicationInstance(JsonNode request, String region) {+ Found found = requireByArn(instances, request, instances.arnMember(), region);+ ObjectNode attributes = found.resource().getAttributes().deepCopy();+ rename(instances, region, found, attributes, request);+ applyInstanceMembers(attributes, request);+ return save(found, attributes);+ }++ public synchronized DmsResource deleteReplicationInstance(JsonNode request, String region) {+ Found found = requireByArn(instances, request, instances.arnMember(), region);+ String arn = arnOf(instances, found.resource());+ if (regionScan(tasks, region).stream().anyMatch(task -> arn.equals(member(task, "ReplicationInstanceArn")))) {+ throw invalidState("Replication instance " + arn + " has replication tasks and cannot be deleted.");+ }+ DmsResource removed = remove(found);+ removed.getAttributes().put("ReplicationInstanceStatus", "deleting");+ return removed;+ }++ /**+ * "default" cannot be created, so it is never stored: it resolves to the region's default-VPC+ * subnets at read time, which is the group an instance created without one lands in.+ */+ public Optional<ReplicationSubnetGroup> findReplicationSubnetGroup(String region, String identifier) {+ if (!"default".equals(identifier)) {+ return subnetGroups.get(storageKey(region, identifier));+ }+ List<String> defaultSubnets = ec2Service.describeSubnets(region, List.of(), Map.of()).stream()+ .filter(Subnet::isDefaultForAz)+ .map(Subnet::getSubnetId)+ .toList();+ if (defaultSubnets.isEmpty()) {+ return Optional.empty();+ }+ try {+ return Optional.of(buildSubnetGroup(identifier, "default", defaultSubnets, region));+ } catch (AwsException ignored) {+ // Safe to swallow: buildSubnetGroup only rejects the default subnets when they cannot+ // form a valid group (a default VPC trimmed below two AZs, or a subnet with no AZ).+ // There is then no usable default group, so this answers exactly as when the region+ // has no default subnets and the caller falls back to the bare identifier.+ return Optional.empty();+ }+ }++ // ---------------------------------------------------------------- replication tasks++ /** A new task is {@code ready}, the state the Terraform create waiter targets. */+ public synchronized DmsResource createReplicationTask(JsonNode request, String region) {+ String identifier = requireResourceIdentifier(tasks, request);+ ObjectNode attributes = JsonNodeFactory.instance.objectNode();+ attributes.put("SourceEndpointArn", requireEndpointOfType(request, "SourceEndpointArn", "SOURCE", region));+ attributes.put("TargetEndpointArn", requireEndpointOfType(request, "TargetEndpointArn", "TARGET", region));+ attributes.put("ReplicationInstanceArn",+ arnOf(instances, requireByArn(instances, request, "ReplicationInstanceArn", region).resource()));+ if (text(request, "MigrationType") == null) {+ throw invalidParameter("The parameter MigrationType must be provided.");+ }+ if (text(request, "TableMappings") == null) {+ throw invalidParameter("The parameter TableMappings must be provided.");+ }+ attributes.put("Status", "ready");+ attributes.put("ReplicationTaskCreationDate", now());+ applyTaskMembers(attributes, request);+ return create(tasks, region, identifier, attributes, request);+ }++ public PaginatedResult<DmsResource> describeReplicationTasks(JsonNode request, String region) {+ return describe(tasks, request, region);+ }++ public synchronized DmsResource modifyReplicationTask(JsonNode request, String region) {+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);+ requireNotRunning(found, "modified");+ ObjectNode attributes = found.resource().getAttributes().deepCopy();+ rename(tasks, region, found, attributes, request);+ applyTaskMembers(attributes, request);+ return save(found, attributes);+ }++ public synchronized DmsResource deleteReplicationTask(JsonNode request, String region) {+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);+ requireNotRunning(found, "deleted");+ DmsResource removed = remove(found);+ removed.getAttributes().put("Status", "deleting");+ return removed;+ }++ /**+ * The task goes straight to {@code running} and stays there until stopped: no data moves, so a+ * full-load task never finishes on its own the way it would against real endpoints.+ */+ public synchronized DmsResource startReplicationTask(JsonNode request, String region) {+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);+ String startType = text(request, "StartReplicationTaskType");+ if (startType == null || !START_TYPES.contains(startType)) {+ throw invalidParameter("StartReplicationTaskType must be one of " + START_TYPES + ".");+ }+ requireNotRunning(found, "started");+ ObjectNode attributes = found.resource().getAttributes().deepCopy();+ attributes.put("Status", TASK_RUNNING);+ attributes.put("ReplicationTaskStartDate", now());+ return save(found, attributes);+ }++ public synchronized DmsResource stopReplicationTask(JsonNode request, String region) {+ Found found = requireByArn(tasks, request, tasks.arnMember(), region);+ if (!TASK_RUNNING.equals(member(found.resource(), "Status"))) {+ throw invalidState("Replication task " + arnOf(tasks, found.resource())+ + " is currently not running.");+ }+ ObjectNode attributes = found.resource().getAttributes().deepCopy();+ attributes.put("Status", "stopped");+ return save(found, attributes);+ }+public synchronized ReplicationSubnetGroup createReplicationSubnetGroup(JsonNode request, String region) {String identifier = requireIdentifier(request);String description = requireDescription(request);@@ -96,6 +389,13 @@ public class DmsService implements Resettable {if (subnetGroups.get(key).isEmpty()) {throw notFound(identifier);}+ boolean inUse = regionScan(instances, region).stream()+ .anyMatch(instance -> identifier.equals(instance.getAttributes()+ .path("ReplicationSubnetGroup").path("ReplicationSubnetGroupIdentifier").asText(null)));+ if (inUse) {+ throw invalidState("Replication subnet group " + identifier+ + " is in use by a replication instance and cannot be deleted.");+ }subnetGroups.delete(key);}@@ -104,13 +404,13 @@ public class DmsService implements Resettable {if (!arnList.isEmpty()) {List<ResourceTag> tags = new ArrayList<>();for (String arn : arnList) {- resolveByArn(arn, region).getTags()+ tagsByArn(arn, region).forEach((key, value) -> tags.add(new ResourceTag(arn, key, value)));}return tags;}String resourceArn = requireResourceArn(request);- return resolveByArn(resourceArn, region).getTags().entrySet().stream()+ return tagsByArn(resourceArn, region).entrySet().stream().map(entry -> new ResourceTag(null, entry.getKey(), entry.getValue())).toList();}@@ -138,6 +438,324 @@ public class DmsService implements Resettable {@Overridepublic void clear() {subnetGroups.clear();+ endpoints.store().clear();+ instances.store().clear();+ tasks.store().clear();+ }++ // ---------------------------------------------------------------- shared resource plumbing++ private DmsResource create(Kind kind, String region, String identifier, ObjectNode attributes,+ JsonNode request) {+ if (findByIdentifier(kind, region, identifier).isPresent()) {+ throw new AwsException("ResourceAlreadyExistsFault",+ kind.label() + " " + identifier + " already exists.", 400);+ }+ String resourceId = text(request, "ResourceIdentifier");+ if (resourceId == null) {+ resourceId = randomResourceId();+ } else if (!isValidIdentifier(resourceId, RESOURCE_IDENTIFIER_ARN_MAX)) {+ throw invalidParameter("ResourceIdentifier must be 1-" + RESOURCE_IDENTIFIER_ARN_MAX+ + " ASCII letters, digits and hyphens, begin with a letter, and not end with a"+ + " hyphen or contain two consecutive hyphens.");+ }+ String key = storageKey(region, resourceId);+ if (kind.store().get(key).isPresent()) {+ throw new AwsException("ResourceAlreadyExistsFault",+ "ResourceIdentifier " + resourceId + " is already in use.", 400);+ }+ attributes.put(kind.idMember(), identifier);+ attributes.put(kind.arnMember(), regionResolver.buildArn("dms", region, kind.arnType() + ":" + resourceId));+ DmsResource resource = new DmsResource();+ resource.setAttributes(attributes);+ resource.setTags(readTags(request.get("Tags")));+ kind.store().put(key, resource);+ return resource;+ }++ /**+ * Filters AND across names and OR across a filter's values. Values compare case-insensitively,+ * so {@code endpoint-type=source} matches the {@code SOURCE} DMS returns. A filter that matches+ * nothing is {@code ResourceNotFoundFault}, which is what the Terraform provider's finders+ * treat as gone; an unfiltered describe of nothing is an empty list.+ */+ private PaginatedResult<DmsResource> describe(Kind kind, JsonNode request, String region) {+ Integer maxRecords = maxRecords(request);+ String marker = text(request, "Marker");+ Map<String, List<String>> filters = filters(request, kind.filters().keySet());+ List<DmsResource> matching = regionScan(kind, region).stream()+ .filter(resource -> filters.entrySet().stream().allMatch(filter ->+ kind.filters().get(filter.getKey()).stream()+ .map(name -> member(resource, name))+ .anyMatch(value -> value != null && filter.getValue().stream()+ .anyMatch(value::equalsIgnoreCase))))+ .toList();+ if (!filters.isEmpty() && matching.isEmpty()) {+ throw new AwsException("ResourceNotFoundFault",+ "No " + kind.label() + " found matching provided filters.", 400);+ }+ return Pagination.paginate(matching, resource -> member(resource, kind.idMember()),+ maxRecords, marker, DEFAULT_MAX_RECORDS, MAXIMUM_MAX_RECORDS, "InvalidParameterValueException");+ }++ private void rename(Kind kind, String region, Found found, ObjectNode attributes, JsonNode request) {+ if (text(request, kind.idMember()) == null) {+ return;+ }+ String identifier = requireResourceIdentifier(kind, request);+ if (!identifier.equals(member(found.resource(), kind.idMember()))+ && findByIdentifier(kind, region, identifier).isPresent()) {+ throw new AwsException("ResourceAlreadyExistsFault",+ kind.label() + " " + identifier + " already exists.", 400);+ }+ attributes.put(kind.idMember(), identifier);+ }++ private DmsResource save(Found found, ObjectNode attributes) {+ found.resource().setAttributes(attributes);+ found.kind().store().put(found.key(), found.resource());+ return found.resource();+ }++ private static DmsResource remove(Found found) {+ found.kind().store().delete(found.key());+ DmsResource removed = new DmsResource();+ removed.setAttributes(found.resource().getAttributes().deepCopy());+ return removed;+ }++ private Optional<DmsResource> findByIdentifier(Kind kind, String region, String identifier) {+ return regionScan(kind, region).stream()+ .filter(resource -> identifier.equals(member(resource, kind.idMember())))+ .findFirst();+ }++ private List<DmsResource> regionScan(Kind kind, String region) {+ return kind.store().scan(key -> key.startsWith(region + "::"));+ }++ /** Resolves the ARN in {@code member} of the request to a stored resource of {@code kind}. */+ private Found requireByArn(Kind kind, JsonNode request, String member, String region) {+ String arn = text(request, member);+ if (arn == null || arn.isBlank()) {+ throw invalidParameter("The parameter " + member + " must be provided and must not be blank.");+ }+ String[] parsed = parseArn(arn, region);+ if (!kind.arnType().equals(parsed[0])) {+ throw notFoundForArn(arn);+ }+ String key = storageKey(region, parsed[1]);+ return kind.store().get(key).map(resource -> new Found(kind, key, resource))+ .orElseThrow(() -> notFoundForArn(arn));+ }++ private Kind kindForArnType(String arnType, String resourceArn) {+ for (Kind kind : List.of(endpoints, instances, tasks)) {+ if (kind.arnType().equals(arnType)) {+ return kind;+ }+ }+ throw notFoundForArn(resourceArn);+ }++ private String requireResourceIdentifier(Kind kind, JsonNode request) {+ String value = text(request, kind.idMember());+ if (value == null || value.isBlank()) {+ throw invalidParameter("The parameter " + kind.idMember() + " must be provided and must not be blank.");+ }+ if (!isValidIdentifier(value, kind.maxIdLength())) {+ throw invalidParameter(kind.idMember() + " must be 1-" + kind.maxIdLength()+ + " ASCII letters, digits and hyphens, begin with a letter, and not end with a"+ + " hyphen or contain two consecutive hyphens.");+ }+ return kind.lowercaseId() ? value.toLowerCase(Locale.ROOT) : value;+ }++ private static boolean isValidIdentifier(String value, int maxLength) {+ return value.length() <= maxLength && RESOURCE_IDENTIFIER.matcher(value).matches()+ && !value.contains("--") && !value.endsWith("-");+ }++ private String requireEndpointOfType(JsonNode request, String member, String type, String region) {+ DmsResource endpoint = requireByArn(endpoints, request, member, region).resource();+ String arn = arnOf(endpoints, endpoint);+ if (!type.equals(member(endpoint, "EndpointType"))) {+ throw invalidParameter(member + " " + arn + " is not a " + type.toLowerCase(Locale.ROOT) + " endpoint.");+ }+ return arn;+ }++ /** DMS accepts the lowercase enum value and returns it uppercase. */+ private static String requireEndpointType(JsonNode request) {+ String value = text(request, "EndpointType");+ if (value == null || !Set.of("source", "target").contains(value.toLowerCase(Locale.ROOT))) {+ throw invalidParameter("The parameter EndpointType must be source or target.");+ }+ return value.toUpperCase(Locale.ROOT);+ }++ private static void applyEndpointMembers(ObjectNode attributes, JsonNode request, boolean exactSettings) {+ ENDPOINT_STRING_MEMBERS.forEach(name -> putText(attributes, request, name));+ String sslMode = text(request, "SslMode");+ if (sslMode != null) {+ if (!SSL_MODES.contains(sslMode)) {+ throw invalidParameter("SslMode must be one of " + SSL_MODES + ".");+ }+ attributes.put("SslMode", sslMode);+ }+ Integer port = integer(request, "Port");+ if (port != null) {+ attributes.put("Port", port);+ }+ for (String name : ENDPOINT_SETTINGS) {+ JsonNode settings = request == null ? null : request.get(name);+ if (settings == null || settings.isNull()) {+ continue;+ }+ if (!settings.isObject()) {+ throw serialization(name + " must be a structure.");+ }+ ObjectNode stored = ((ObjectNode) settings).deepCopy();+ stored.remove(SECRET_MEMBERS);+ if (!exactSettings && attributes.get(name) instanceof ObjectNode existing) {+ existing.setAll(stored);+ } else {+ attributes.set(name, stored);+ }+ }+ }++ private static void applyInstanceMembers(ObjectNode attributes, JsonNode request) {+ putText(attributes, request, "ReplicationInstanceClass");+ putText(attributes, request, "PreferredMaintenanceWindow");+ putText(attributes, request, "EngineVersion");+ putText(attributes, request, "NetworkType");+ String instanceClass = attributes.path("ReplicationInstanceClass").asText();+ if (!instanceClass.startsWith("dms.")) {+ throw invalidParameter("Invalid ReplicationInstanceClass " + instanceClass + ".");+ }+ Integer storage = integer(request, "AllocatedStorage");+ if (storage != null) {+ if (storage < 5 || storage > 6144) {+ throw invalidParameter("AllocatedStorage must be between 5 and 6144.");+ }+ attributes.put("AllocatedStorage", storage);+ }+ for (String name : List.of("MultiAZ", "AutoMinorVersionUpgrade")) {+ JsonNode value = request == null ? null : request.get(name);+ if (value != null && !value.isNull()) {+ attributes.put(name, bool(request, name, false));+ }+ }+ JsonNode groups = request == null ? null : request.get("VpcSecurityGroupIds");+ if (groups != null && !groups.isNull()) {+ ArrayNode memberships = attributes.putArray("VpcSecurityGroups");+ stringList(groups, "VpcSecurityGroupIds").forEach(id ->+ memberships.addObject().put("VpcSecurityGroupId", id).put("Status", "active"));+ }+ JsonNode kerberos = request == null ? null : request.get("KerberosAuthenticationSettings");+ if (kerberos != null && !kerberos.isNull()) {+ if (!kerberos.isObject()) {+ throw serialization("KerberosAuthenticationSettings must be a structure.");+ }+ // Only the members DMS models; anything else a caller sends is neither stored nor echoed.+ ObjectNode stored = attributes.putObject("KerberosAuthenticationSettings");+ for (String member : List.of("KeyCacheSecretId", "KeyCacheSecretIamArn", "Krb5FileContents")) {+ putText(stored, kerberos, member);+ }+ }+ }++ private static void applyTaskMembers(ObjectNode attributes, JsonNode request) {+ String migrationType = text(request, "MigrationType");+ if (migrationType != null) {+ if (!MIGRATION_TYPES.contains(migrationType)) {+ throw invalidParameter("MigrationType must be one of " + MIGRATION_TYPES + ".");+ }+ attributes.put("MigrationType", migrationType);+ }+ for (String name : List.of("TableMappings", "ReplicationTaskSettings")) {+ String value = text(request, name);+ if (value != null) {+ requireJsonObject(name, value);+ attributes.put(name, value);+ }+ }+ putText(attributes, request, "CdcStartPosition");+ putText(attributes, request, "CdcStopPosition");+ putText(attributes, request, "TaskData");+ }++ private static void requireJsonObject(String member, String value) {+ try {+ if (JSON.readTree(value) instanceof ObjectNode) {+ return;+ }+ } catch (JsonProcessingException e) {+ // reported below+ }+ throw invalidParameter(member + " must be a JSON object.");+ }++ private void requireNotRunning(Found found, String verb) {+ if (TASK_RUNNING.equals(member(found.resource(), "Status"))) {+ throw invalidState("Replication task " + arnOf(tasks, found.resource())+ + " is running and cannot be " + verb + ". Stop it first.");+ }+ }++ private static void putText(ObjectNode attributes, JsonNode request, String name) {+ String value = text(request, name);+ if (value != null) {+ attributes.put(name, value);+ }+ }++ private static String member(DmsResource resource, String name) {+ return resource.getAttributes().path(name).asText(null);+ }++ private static String arnOf(Kind kind, DmsResource resource) {+ return member(resource, kind.arnMember());+ }++ private static Integer integer(JsonNode request, String field) {+ JsonNode node = request == null ? null : request.get(field);+ if (node == null || node.isNull()) {+ return null;+ }+ if (!node.isIntegralNumber() || !node.canConvertToInt()) {+ throw serialization(field + " must be an integer.");+ }+ return node.intValue();+ }++ private static boolean bool(JsonNode request, String field, boolean defaultValue) {+ JsonNode node = request == null ? null : request.get(field);+ if (node == null || node.isNull()) {+ return defaultValue;+ }+ if (!node.isBoolean()) {+ throw serialization(field + " must be a boolean.");+ }+ return node.booleanValue();+ }++ private static double now() {+ return Instant.now().toEpochMilli() / 1000.0;+ }++ /** DMS-generated ARN suffixes are 26 characters of the base32 alphabet. */+ private static String randomResourceId() {+ StringBuilder id = new StringBuilder(26);+ for (int i = 0; i < 26; i++) {+ id.append(RESOURCE_ID_ALPHABET.charAt(RANDOM.nextInt(RESOURCE_ID_ALPHABET.length())));+ }+ return id.toString();+ }++ private static AwsException invalidState(String message) {+ return new AwsException("InvalidResourceStateFault", message, 400);}private ReplicationSubnetGroup buildSubnetGroup(String identifier, String description,@@ -271,36 +889,41 @@ public class DmsService implements Resettable {}private static List<String> identifierFilters(JsonNode request) {+ return filters(request, Set.of(SUBNET_GROUP_ID_FILTER)).getOrDefault(SUBNET_GROUP_ID_FILTER, List.of())+ .stream().map(value -> value.toLowerCase(Locale.ROOT)).toList();+ }++ /** Parses {@code Filters} into name to values, in request order, rejecting a name not in {@code validNames}. */+ private static Map<String, List<String>> filters(JsonNode request, Set<String> validNames) {+ Map<String, List<String>> parsed = new LinkedHashMap<>();JsonNode filters = request == null ? null : request.get("Filters");if (filters == null || filters.isNull()) {- return List.of();+ return parsed;}if (!filters.isArray()) {throw serialization("Filters must be a list of Name and Values pairs.");}- List<String> identifiers = new ArrayList<>();for (JsonNode filter : filters) {if (!filter.isObject()) {throw serialization("Filters must be a list of Name and Values pairs.");}String name = text(filter, "Name");- if (!SUBNET_GROUP_ID_FILTER.equals(name)) {+ if (!validNames.contains(name)) {throw invalidParameter("Invalid filter: " + name + ".");}JsonNode values = filter.get("Values");if (values == null || values.isNull()) {- throw invalidParameter("The filter " + SUBNET_GROUP_ID_FILTER + " must have values.");+ throw invalidParameter("The filter " + name + " must have values.");}if (!values.isArray()) {throw serialization("Filter Values must be a list of strings.");}if (values.isEmpty()) {- throw invalidParameter("The filter " + SUBNET_GROUP_ID_FILTER + " must have values.");+ throw invalidParameter("The filter " + name + " must have values.");}- stringList(values, "Filter Values")- .forEach(value -> identifiers.add(value.toLowerCase(Locale.ROOT)));+ parsed.computeIfAbsent(name, key -> new ArrayList<>()).addAll(stringList(values, "Filter Values"));}- return identifiers;+ return parsed;}private static AwsException notFound(String identifier) {@@ -353,31 +976,49 @@ public class DmsService implements Resettable {private void updateTags(String resourceArn, String region,Consumer<Map<String, String>> mutation) {- String key = subnetGroupKeyForArn(resourceArn, region);- ReplicationSubnetGroup group = subnetGroups.get(key).orElseThrow(() -> notFoundForArn(resourceArn));- Map<String, String> tags = new LinkedHashMap<>(group.getTags());+ String[] parsed = parseArn(resourceArn, region);+ if (SUBNET_GROUP_ARN_RESOURCE_TYPE.equals(parsed[0])) {+ String key = storageKey(region, parsed[1].toLowerCase(Locale.ROOT));+ ReplicationSubnetGroup group = subnetGroups.get(key).orElseThrow(() -> notFoundForArn(resourceArn));+ Map<String, String> tags = new LinkedHashMap<>(group.getTags());+ mutation.accept(tags);+ group.setTags(tags);+ subnetGroups.put(key, group);+ return;+ }+ Kind kind = kindForArnType(parsed[0], resourceArn);+ String key = storageKey(region, parsed[1]);+ DmsResource resource = kind.store().get(key).orElseThrow(() -> notFoundForArn(resourceArn));+ Map<String, String> tags = new LinkedHashMap<>(resource.getTags());mutation.accept(tags);- group.setTags(tags);- subnetGroups.put(key, group);+ resource.setTags(tags);+ kind.store().put(key, resource);}- private ReplicationSubnetGroup resolveByArn(String resourceArn, String region) {- return subnetGroups.get(subnetGroupKeyForArn(resourceArn, region))- .orElseThrow(() -> notFoundForArn(resourceArn));+ private Map<String, String> tagsByArn(String resourceArn, String region) {+ String[] parsed = parseArn(resourceArn, region);+ if (SUBNET_GROUP_ARN_RESOURCE_TYPE.equals(parsed[0])) {+ return subnetGroups.get(storageKey(region, parsed[1].toLowerCase(Locale.ROOT)))+ .orElseThrow(() -> notFoundForArn(resourceArn)).getTags();+ }+ return kindForArnType(parsed[0], resourceArn).store().get(storageKey(region, parsed[1]))+ .orElseThrow(() -> notFoundForArn(resourceArn)).getTags();}/**- * DescribeReplicationSubnetGroups does not return an ARN, so Terraform builds- * {@code arn:aws:dms:<region>:<account>:subgrp:<id>} itself and tags against that. Anything- * that is not such an ARN names no DMS resource Floci holds, which is a ResourceNotFoundFault- * rather than a parameter error.+ * Splits a DMS ARN into its resource type and resource id. DescribeReplicationSubnetGroups does+ * not return an ARN, so Terraform builds {@code arn:aws:dms:<region>:<account>:subgrp:<id>}+ * itself and tags against that; endpoints ({@code endpoint}), replication instances+ * ({@code rep}) and tasks ({@code task}) carry the ARN DMS minted. Anything that is not such an+ * ARN names no DMS resource Floci holds, which is a ResourceNotFoundFault rather than a+ * parameter error.** <p>An ARN naming another account or another Region is treated the same way. Tagging is not a* cross-account or cross-Region operation on AWS, and here it cannot be one either: storage is* scoped to the caller's account and keyed by the caller's Region, so honouring a foreign ARN- * would silently reach the caller's own group of that name instead of the one named.+ * would silently reach the caller's own resource of that name instead of the one named.*/- private String subnetGroupKeyForArn(String resourceArn, String callerRegion) {+ private String[] parseArn(String resourceArn, String callerRegion) {AwsArnUtils.Arn arn;try {arn = AwsArnUtils.parse(resourceArn);@@ -385,8 +1026,7 @@ public class DmsService implements Resettable {throw notFoundForArn(resourceArn);}String[] resource = arn.resource().split(":", 2);- if (!"dms".equals(arn.service()) || resource.length != 2- || !SUBNET_GROUP_ARN_RESOURCE_TYPE.equals(resource[0]) || resource[1].isBlank()) {+ if (!"dms".equals(arn.service()) || resource.length != 2 || resource[1].isBlank()) {throw notFoundForArn(resourceArn);}if (!arn.accountId().isBlank() && !arn.accountId().equals(regionResolver.getAccountId())) {@@ -395,7 +1035,7 @@ public class DmsService implements Resettable {if (arn.region() != null && !arn.region().isBlank() && !arn.region().equals(callerRegion)) {throw notFoundForArn(resourceArn);}- return storageKey(callerRegion, resource[1].toLowerCase(Locale.ROOT));+ return resource;}private static List<String> arnList(JsonNode request) {diff --git a/src/main/java/io/github/hectorvent/floci/services/dms/model/DmsResource.java b/src/main/java/io/github/hectorvent/floci/services/dms/model/DmsResource.javanew file mode 100644index 0000000000..21a17fc127--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/dms/model/DmsResource.java@@ -0,0 +1,43 @@+package io.github.hectorvent.floci.services.dms.model;++import com.fasterxml.jackson.annotation.JsonIgnoreProperties;+import com.fasterxml.jackson.databind.node.JsonNodeFactory;+import com.fasterxml.jackson.databind.node.ObjectNode;+import io.quarkus.runtime.annotations.RegisterForReflection;++import java.util.Collections;+import java.util.LinkedHashMap;+import java.util.Map;++/**+ * An endpoint, replication instance or replication task. {@code attributes} holds the resource+ * in the wire shape DMS returns it in ({@code Endpoint}, {@code ReplicationInstance},+ * {@code ReplicationTask}), so a describe echoes exactly what was stored. Secret members such as+ * {@code Password} are never stored, because DMS never returns them.+ */+@RegisterForReflection+@JsonIgnoreProperties(ignoreUnknown = true)+public class DmsResource {++ private ObjectNode attributes = JsonNodeFactory.instance.objectNode();+ private Map<String, String> tags = new LinkedHashMap<>();++ public DmsResource() {+ }++ public ObjectNode getAttributes() {+ return attributes;+ }++ public void setAttributes(ObjectNode attributes) {+ this.attributes = attributes != null ? attributes : JsonNodeFactory.instance.objectNode();+ }++ public Map<String, String> getTags() {+ return Collections.unmodifiableMap(new LinkedHashMap<>(tags));+ }++ public void setTags(Map<String, String> tags) {+ this.tags = tags != null ? new LinkedHashMap<>(tags) : new LinkedHashMap<>();+ }+}diff --git a/src/main/resources/application.yml b/src/main/resources/application.ymlindex f89a74dc81..1ab7b0c624 100644--- a/src/main/resources/application.yml+++ b/src/main/resources/application.yml@@ -112,6 +112,7 @@ quarkus:- --initialize-at-run-time=io.github.hectorvent.floci.services.elasticbeanstalk.ElasticBeanstalkService- --initialize-at-run-time=io.github.hectorvent.floci.services.elb.ElbClassicService- --initialize-at-run-time=io.github.hectorvent.floci.services.datasync.DataSyncService+ - --initialize-at-run-time=io.github.hectorvent.floci.services.dms.DmsService- --initialize-at-run-time=io.github.hectorvent.floci.services.lambda.launcher.LambdaExecutionRoleCredentials- --initialize-at-run-time=io.github.hectorvent.floci.services.ecs.container.EcsTaskRoleCredentials- --initialize-at-run-time=io.github.hectorvent.floci.services.ssoportal.SsoPortalServicediff --git a/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.javaindex d6234dfa59..66d2ff75de 100644--- a/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/dms/DmsIntegrationTest.java@@ -278,7 +278,7 @@ class DmsIntegrationTest {@Testvoid unsupportedDmsActionReportsUnknownOperation() {- dms("CreateReplicationInstance")+ dms("CreateReplicationConfig").body("{}").when().post("/")diff --git a/src/test/java/io/github/hectorvent/floci/services/dms/DmsLifecycleIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/dms/DmsLifecycleIntegrationTest.javanew file mode 100644index 0000000000..a668328cbe--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/dms/DmsLifecycleIntegrationTest.java@@ -0,0 +1,563 @@+package io.github.hectorvent.floci.services.dms;++import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import io.restassured.response.ValidatableResponse;+import io.restassured.specification.RequestSpecification;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.equalTo;+import static org.hamcrest.Matchers.hasItems;+import static org.hamcrest.Matchers.hasKey;+import static org.hamcrest.Matchers.hasSize;+import static org.hamcrest.Matchers.matchesPattern;+import static org.hamcrest.Matchers.not;+import static org.hamcrest.Matchers.notNullValue;+import static org.hamcrest.Matchers.nullValue;++/**+ * Endpoint, replication instance and replication task lifecycles, driven the way the Terraform+ * provider drives them: every mutation is read back through a separate describe call rather than+ * trusted from the mutating call's own response.+ */+@QuarkusTest+class DmsLifecycleIntegrationTest {++ private static final String CONTENT_TYPE = "application/x-amz-json-1.1";+ private static final String TARGET_PREFIX = "AmazonDMSv20160101.";+ private static final String ACCOUNT_ID = "723679240095";+ private static final String AUTH_HEADER =+ "AWS4-HMAC-SHA256 Credential=" + ACCOUNT_ID + "/20260101/us-east-1/dms/aws4_request";+ private static final String SUBNET_A = "subnet-default-us-east-1-a";+ private static final String SUBNET_B = "subnet-default-us-east-1-b";++ @BeforeAll+ static void configureRestAssured() {+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @Test+ void endpointLifecycleIsReadableThroughDescribe() {+ String arn = createEndpoint("lc-ep-source", "source",+ ",\"Username\":\"admin\",\"Password\":\"hunter2\",\"ServerName\":\"db.example\","+ + "\"Port\":3306,\"DatabaseName\":\"app\","+ + "\"Tags\":[{\"Key\":\"env\",\"Value\":\"test\"}]");++ describeEndpoint("lc-ep-source")+ .body("Endpoints", hasSize(1))+ .body("Endpoints[0].EndpointArn", equalTo(arn))+ .body("Endpoints[0].EndpointType", equalTo("SOURCE"))+ .body("Endpoints[0].EngineName", equalTo("mysql"))+ .body("Endpoints[0].Status", equalTo("active"))+ .body("Endpoints[0].Username", equalTo("admin"))+ .body("Endpoints[0].ServerName", equalTo("db.example"))+ .body("Endpoints[0].Port", equalTo(3306))+ .body("Endpoints[0].DatabaseName", equalTo("app"))+ .body("Endpoints[0].SslMode", equalTo("none"))+ .body("Endpoints[0].Password", nullValue());++ dms("DescribeEndpoints")+ .body("{\"Filters\":[{\"Name\":\"endpoint-arn\",\"Values\":[\"" + arn + "\"]},"+ + "{\"Name\":\"endpoint-type\",\"Values\":[\"source\"]}]}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("Endpoints[0].EndpointIdentifier", equalTo("lc-ep-source"));++ dms("ModifyEndpoint")+ .body("{\"EndpointArn\":\"" + arn + "\",\"Port\":3307,\"SslMode\":\"require\"}")+ .when().post("/")+ .then().statusCode(200);++ describeEndpoint("lc-ep-source")+ .body("Endpoints[0].Port", equalTo(3307))+ .body("Endpoints[0].SslMode", equalTo("require"))+ .body("Endpoints[0].ServerName", equalTo("db.example"));++ dms("ListTagsForResource")+ .body("{\"ResourceArn\":\"" + arn + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("TagList", hasSize(1))+ .body("TagList[0].Key", equalTo("env"));++ dms("DeleteEndpoint")+ .body("{\"EndpointArn\":\"" + arn + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("Endpoint.Status", equalTo("deleting"));++ dms("DescribeEndpoints")+ .body("{\"Filters\":[{\"Name\":\"endpoint-id\",\"Values\":[\"lc-ep-source\"]}]}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));+ }++ @Test+ void endpointSettingsMergeOnModifyAndNeverReturnSecrets() {+ String arn = createEndpoint("lc-ep-s3", "target",+ ",\"S3Settings\":{\"BucketName\":\"b1\",\"ServiceAccessRoleArn\":\"arn:aws:iam::"+ + ACCOUNT_ID + ":role/dms\",\"CsvDelimiter\":\",\"},"+ + "\"MySQLSettings\":{\"ServerName\":\"x\",\"Password\":\"secret\"}");++ dms("ModifyEndpoint")+ .body("{\"EndpointArn\":\"" + arn + "\",\"S3Settings\":{\"BucketFolder\":\"cdc\"}}")+ .when().post("/")+ .then().statusCode(200);++ describeEndpoint("lc-ep-s3")+ .body("Endpoints[0].EndpointType", equalTo("TARGET"))+ .body("Endpoints[0].S3Settings.BucketName", equalTo("b1"))+ .body("Endpoints[0].S3Settings.BucketFolder", equalTo("cdc"))+ .body("Endpoints[0].S3Settings.CsvDelimiter", equalTo(","))+ .body("Endpoints[0].MySQLSettings.ServerName", equalTo("x"))+ .body("Endpoints[0].MySQLSettings.Password", nullValue());++ dms("ModifyEndpoint")+ .body("{\"EndpointArn\":\"" + arn + "\",\"ExactSettings\":true,"+ + "\"S3Settings\":{\"BucketName\":\"b2\"}}")+ .when().post("/")+ .then().statusCode(200);++ describeEndpoint("lc-ep-s3")+ .body("Endpoints[0].S3Settings.BucketName", equalTo("b2"))+ .body("Endpoints[0].S3Settings.BucketFolder", nullValue());++ delete("DeleteEndpoint", "EndpointArn", arn);+ }++ @Test+ void duplicateEndpointIdentifierIsRejected() {+ String arn = createEndpoint("lc-ep-dup", "source", "");+ dms("CreateEndpoint")+ .body("{\"EndpointIdentifier\":\"lc-ep-dup\",\"EndpointType\":\"source\","+ + "\"EngineName\":\"mysql\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceAlreadyExistsFault"));+ delete("DeleteEndpoint", "EndpointArn", arn);+ }++ @Test+ void replicationInstanceLifecycleIsReadableThroughDescribe() {+ dms("CreateReplicationSubnetGroup")+ .body("{\"ReplicationSubnetGroupIdentifier\":\"lc-rep-subnets\","+ + "\"ReplicationSubnetGroupDescription\":\"lifecycle\","+ + "\"SubnetIds\":[\"" + SUBNET_A + "\",\"" + SUBNET_B + "\"]}")+ .when().post("/")+ .then().statusCode(200);++ String arn = dms("CreateReplicationInstance")+ .body("{\"ReplicationInstanceIdentifier\":\"LC-Rep\","+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\",\"AllocatedStorage\":20,"+ + "\"ReplicationSubnetGroupIdentifier\":\"lc-rep-subnets\","+ + "\"PubliclyAccessible\":false,\"MultiAZ\":false,"+ + "\"AutoMinorVersionUpgrade\":true,\"VpcSecurityGroupIds\":[\"sg-1\"],"+ + "\"Tags\":[{\"Key\":\"team\",\"Value\":\"data\"}]}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("ReplicationInstance.ReplicationInstanceArn",+ matchesPattern("arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":rep:[A-Z2-7]{26}"))+ .extract().path("ReplicationInstance.ReplicationInstanceArn");++ describeInstance("lc-rep")+ .body("ReplicationInstances", hasSize(1))+ .body("ReplicationInstances[0].ReplicationInstanceArn", equalTo(arn))+ .body("ReplicationInstances[0].ReplicationInstanceIdentifier", equalTo("lc-rep"))+ .body("ReplicationInstances[0].ReplicationInstanceStatus", equalTo("available"))+ .body("ReplicationInstances[0].ReplicationInstanceClass", equalTo("dms.t3.micro"))+ .body("ReplicationInstances[0].AllocatedStorage", equalTo(20))+ .body("ReplicationInstances[0].PubliclyAccessible", equalTo(false))+ .body("ReplicationInstances[0].VpcSecurityGroups[0].VpcSecurityGroupId", equalTo("sg-1"))+ .body("ReplicationInstances[0].ReplicationSubnetGroup.ReplicationSubnetGroupIdentifier",+ equalTo("lc-rep-subnets"))+ .body("ReplicationInstances[0].ReplicationSubnetGroup.VpcId", equalTo("vpc-default-us-east-1"))+ .body("ReplicationInstances[0].EngineVersion", notNullValue());++ dms("DeleteReplicationSubnetGroup")+ .body("{\"ReplicationSubnetGroupIdentifier\":\"lc-rep-subnets\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidResourceStateFault"));++ dms("ModifyReplicationInstance")+ .body("{\"ReplicationInstanceArn\":\"" + arn + "\",\"ApplyImmediately\":true,"+ + "\"ReplicationInstanceClass\":\"dms.c5.large\",\"AllocatedStorage\":100}")+ .when().post("/")+ .then().statusCode(200);++ describeInstance("lc-rep")+ .body("ReplicationInstances[0].ReplicationInstanceClass", equalTo("dms.c5.large"))+ .body("ReplicationInstances[0].AllocatedStorage", equalTo(100))+ .body("ReplicationInstances[0].ReplicationInstanceStatus", equalTo("available"));++ dms("ListTagsForResource")+ .body("{\"ResourceArn\":\"" + arn + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("TagList[0].Key", equalTo("team"));++ dms("DeleteReplicationInstance")+ .body("{\"ReplicationInstanceArn\":\"" + arn + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("ReplicationInstance.ReplicationInstanceStatus", equalTo("deleting"));++ dms("DescribeReplicationInstances")+ .body("{\"Filters\":[{\"Name\":\"replication-instance-id\",\"Values\":[\"lc-rep\"]}]}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));++ delete("DeleteReplicationSubnetGroup", "ReplicationSubnetGroupIdentifier", "lc-rep-subnets");+ }++ @Test+ void replicationInstanceWithoutSubnetGroupUsesDefault() {+ String arn = createInstance("lc-rep-default");+ describeInstance("lc-rep-default")+ .body("ReplicationInstances[0].ReplicationSubnetGroup.ReplicationSubnetGroupIdentifier",+ equalTo("default"))+ .body("ReplicationInstances[0].ReplicationSubnetGroup.VpcId", equalTo("vpc-default-us-east-1"))+ .body("ReplicationInstances[0].ReplicationSubnetGroup.SubnetGroupStatus", equalTo("Complete"))+ .body("ReplicationInstances[0].ReplicationSubnetGroup.Subnets.SubnetIdentifier",+ hasItems(SUBNET_A, SUBNET_B));+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", arn);+ }++ @Test+ void replicationInstanceStoresOnlyModeledKerberosMembers() {+ dms("CreateReplicationInstance")+ .body("{\"ReplicationInstanceIdentifier\":\"lc-rep-kerberos\","+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\","+ + "\"KerberosAuthenticationSettings\":{\"KeyCacheSecretId\":\"krb-cache\","+ + "\"Krb5FileContents\":\"[libdefaults]\",\"Password\":\"hunter2\"}}")+ .when().post("/")+ .then().statusCode(200);+ describeInstance("lc-rep-kerberos")+ .body("ReplicationInstances[0].KerberosAuthenticationSettings.KeyCacheSecretId",+ equalTo("krb-cache"))+ .body("ReplicationInstances[0].KerberosAuthenticationSettings.Krb5FileContents",+ equalTo("[libdefaults]"))+ .body("ReplicationInstances[0].KerberosAuthenticationSettings", not(hasKey("Password")));+ String arn = describeInstance("lc-rep-kerberos").extract()+ .path("ReplicationInstances[0].ReplicationInstanceArn");+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", arn);+ }++ @Test+ void replicationInstanceWithUnknownSubnetGroupIsNotFound() {+ dms("CreateReplicationInstance")+ .body("{\"ReplicationInstanceIdentifier\":\"lc-rep-nogroup\","+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\","+ + "\"ReplicationSubnetGroupIdentifier\":\"lc-never-created\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));+ }++ @Test+ void replicationTaskLifecycleAcrossStartAndStop() {+ String source = createEndpoint("lc-task-src", "source", "");+ String target = createEndpoint("lc-task-tgt", "target", "");+ String instance = createInstance("lc-task-rep");++ String task = dms("CreateReplicationTask")+ .body("{\"ReplicationTaskIdentifier\":\"lc-task\",\"SourceEndpointArn\":\"" + source + "\","+ + "\"TargetEndpointArn\":\"" + target + "\",\"ReplicationInstanceArn\":\"" + instance + "\","+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("%"))+ + ",\"Tags\":[{\"Key\":\"k\",\"Value\":\"v\"}]}")+ .when().post("/")+ .then()+ .statusCode(200)+ .extract().path("ReplicationTask.ReplicationTaskArn");++ describeTask("lc-task")+ .body("ReplicationTasks", hasSize(1))+ .body("ReplicationTasks[0].ReplicationTaskArn", equalTo(task))+ .body("ReplicationTasks[0].Status", equalTo("ready"))+ .body("ReplicationTasks[0].SourceEndpointArn", equalTo(source))+ .body("ReplicationTasks[0].TargetEndpointArn", equalTo(target))+ .body("ReplicationTasks[0].ReplicationInstanceArn", equalTo(instance))+ .body("ReplicationTasks[0].MigrationType", equalTo("full-load"))+ .body("ReplicationTasks[0].TableMappings", equalTo(mappings("%")));++ dms("DescribeReplicationTasks")+ .body("{\"Filters\":[{\"Name\":\"endpoint-arn\",\"Values\":[\"" + target + "\"]}]}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("ReplicationTasks[0].ReplicationTaskIdentifier", equalTo("lc-task"));++ dms("ModifyReplicationTask")+ .body("{\"ReplicationTaskArn\":\"" + task + "\",\"MigrationType\":\"full-load-and-cdc\","+ + "\"TableMappings\":" + jsonString(mappings("app")) + "}")+ .when().post("/")+ .then().statusCode(200);++ describeTask("lc-task")+ .body("ReplicationTasks[0].MigrationType", equalTo("full-load-and-cdc"))+ .body("ReplicationTasks[0].TableMappings", equalTo(mappings("app")))+ .body("ReplicationTasks[0].Status", equalTo("ready"));++ dms("StartReplicationTask")+ .body("{\"ReplicationTaskArn\":\"" + task + "\",\"StartReplicationTaskType\":\"start-replication\"}")+ .when().post("/")+ .then().statusCode(200);++ describeTask("lc-task").body("ReplicationTasks[0].Status", equalTo("running"));++ dms("DeleteReplicationTask")+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidResourceStateFault"));++ dms("StopReplicationTask")+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")+ .when().post("/")+ .then().statusCode(200);++ describeTask("lc-task").body("ReplicationTasks[0].Status", equalTo("stopped"));++ // The Terraform provider treats exactly this message as "already stopped".+ dms("StopReplicationTask")+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidResourceStateFault"))+ .body("message", containsString("is currently not running"));++ dms("DeleteEndpoint")+ .body("{\"EndpointArn\":\"" + source + "\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidResourceStateFault"));++ dms("DeleteReplicationInstance")+ .body("{\"ReplicationInstanceArn\":\"" + instance + "\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidResourceStateFault"));++ dms("DeleteReplicationTask")+ .body("{\"ReplicationTaskArn\":\"" + task + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("ReplicationTask.Status", equalTo("deleting"));++ dms("DescribeReplicationTasks")+ .body("{\"Filters\":[{\"Name\":\"replication-task-id\",\"Values\":[\"lc-task\"]}]}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));++ delete("DeleteEndpoint", "EndpointArn", source);+ delete("DeleteEndpoint", "EndpointArn", target);+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);+ }++ @Test+ void tagsAddAndRemoveOnEndpointInstanceAndTaskArns() {+ String source = createEndpoint("lc-tag-src", "source", "");+ String target = createEndpoint("lc-tag-tgt", "target", "");+ String instance = createInstance("lc-tag-rep");+ String task = dms("CreateReplicationTask")+ .body("{\"ReplicationTaskIdentifier\":\"lc-tag-task\",\"SourceEndpointArn\":\"" + source + "\","+ + "\"TargetEndpointArn\":\"" + target + "\",\"ReplicationInstanceArn\":\"" + instance + "\","+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("%")) + "}")+ .when().post("/")+ .then()+ .statusCode(200)+ .extract().path("ReplicationTask.ReplicationTaskArn");++ for (String arn : new String[] {source, instance, task}) {+ dms("AddTagsToResource")+ .body("{\"ResourceArn\":\"" + arn + "\",\"Tags\":[{\"Key\":\"keep\",\"Value\":\"" + arn + "\"},"+ + "{\"Key\":\"drop\",\"Value\":\"x\"}]}")+ .when().post("/")+ .then().statusCode(200);+ dms("RemoveTagsFromResource")+ .body("{\"ResourceArn\":\"" + arn + "\",\"TagKeys\":[\"drop\"]}")+ .when().post("/")+ .then().statusCode(200);+ }++ // Each ARN reads back only its own tags: a mutation routed to the wrong resource kind shows here.+ for (String arn : new String[] {source, instance, task}) {+ dms("ListTagsForResource")+ .body("{\"ResourceArn\":\"" + arn + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("TagList", hasSize(1))+ .body("TagList[0].Key", equalTo("keep"))+ .body("TagList[0].Value", equalTo(arn));+ }+ dms("ListTagsForResource")+ .body("{\"ResourceArn\":\"" + target + "\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("TagList", hasSize(0));++ describeEndpoint("lc-tag-src").body("Endpoints[0].EndpointArn", equalTo(source));+ describeInstance("lc-tag-rep").body("ReplicationInstances[0].ReplicationInstanceArn", equalTo(instance));+ describeTask("lc-tag-task").body("ReplicationTasks[0].ReplicationTaskArn", equalTo(task));++ delete("DeleteReplicationTask", "ReplicationTaskArn", task);+ delete("DeleteEndpoint", "EndpointArn", source);+ delete("DeleteEndpoint", "EndpointArn", target);+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);+ }++ @Test+ void replicationTaskNamingUnknownEndpointIsNotFound() {+ dms("CreateReplicationTask")+ .body("{\"ReplicationTaskIdentifier\":\"lc-task-orphan\","+ + "\"SourceEndpointArn\":\"arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":endpoint:NOPE\","+ + "\"TargetEndpointArn\":\"arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":endpoint:NOPE2\","+ + "\"ReplicationInstanceArn\":\"arn:aws:dms:us-east-1:" + ACCOUNT_ID + ":rep:NOPE\","+ + "\"MigrationType\":\"full-load\",\"TableMappings\":\"{}\"}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));+ }++ @Test+ void replicationTaskWithReversedEndpointsIsRejected() {+ String source = createEndpoint("lc-rev-src", "source", "");+ String target = createEndpoint("lc-rev-tgt", "target", "");+ String instance = createInstance("lc-rev-rep");+ dms("CreateReplicationTask")+ .body("{\"ReplicationTaskIdentifier\":\"lc-task-reversed\","+ + "\"SourceEndpointArn\":\"" + target + "\","+ + "\"TargetEndpointArn\":\"" + source + "\","+ + "\"ReplicationInstanceArn\":\"" + instance + "\","+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("app")) + "}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidParameterValueException"));+ dms("DescribeReplicationTasks")+ .body("{\"Filters\":[{\"Name\":\"replication-task-id\",\"Values\":[\"lc-task-reversed\"]}]}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);+ delete("DeleteEndpoint", "EndpointArn", source);+ delete("DeleteEndpoint", "EndpointArn", target);+ }++ @Test+ void replicationTaskWithSourceEndpointAsTargetIsRejected() {+ String source = createEndpoint("lc-src-as-tgt", "source", "");+ String instance = createInstance("lc-src-as-tgt-rep");+ dms("CreateReplicationTask")+ .body("{\"ReplicationTaskIdentifier\":\"lc-task-src-as-tgt\","+ + "\"SourceEndpointArn\":\"" + source + "\","+ + "\"TargetEndpointArn\":\"" + source + "\","+ + "\"ReplicationInstanceArn\":\"" + instance + "\","+ + "\"MigrationType\":\"full-load\",\"TableMappings\":" + jsonString(mappings("app")) + "}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("InvalidParameterValueException"))+ .body("message", containsString("TargetEndpointArn"));+ dms("DescribeReplicationTasks")+ .body("{\"Filters\":[{\"Name\":\"replication-task-id\",\"Values\":[\"lc-task-src-as-tgt\"]}]}")+ .when().post("/")+ .then()+ .statusCode(400)+ .body("__type", equalTo("ResourceNotFoundFault"));+ delete("DeleteReplicationInstance", "ReplicationInstanceArn", instance);+ delete("DeleteEndpoint", "EndpointArn", source);+ }++ private static String mappings(String schema) {+ return "{\"rules\":[{\"rule-type\":\"selection\",\"rule-id\":\"1\",\"rule-name\":\"1\","+ + "\"object-locator\":{\"schema-name\":\"" + schema + "\",\"table-name\":\"%\"},"+ + "\"rule-action\":\"include\"}]}";+ }++ private static String jsonString(String value) {+ return "\"" + value.replace("\\", "\\\\").replace("\"", "\\\"") + "\"";+ }++ private static String createEndpoint(String identifier, String type, String extraMembers) {+ return dms("CreateEndpoint")+ .body("{\"EndpointIdentifier\":\"" + identifier + "\",\"EndpointType\":\"" + type + "\","+ + "\"EngineName\":\"" + ("target".equals(type) ? "s3" : "mysql") + "\"" + extraMembers + "}")+ .when().post("/")+ .then()+ .statusCode(200)+ .body("Endpoint.Password", nullValue())+ .extract().path("Endpoint.EndpointArn");+ }++ private static String createInstance(String identifier) {+ return dms("CreateReplicationInstance")+ .body("{\"ReplicationInstanceIdentifier\":\"" + identifier + "\","+ + "\"ReplicationInstanceClass\":\"dms.t3.micro\"}")+ .when().post("/")+ .then()+ .statusCode(200)+ .extract().path("ReplicationInstance.ReplicationInstanceArn");+ }++ private static ValidatableResponse describeEndpoint(String identifier) {+ return describe("DescribeEndpoints", "endpoint-id", identifier);+ }++ private static ValidatableResponse describeInstance(String identifier) {+ return describe("DescribeReplicationInstances", "replication-instance-id", identifier);+ }++ private static ValidatableResponse describeTask(String identifier) {+ return describe("DescribeReplicationTasks", "replication-task-id", identifier);+ }++ private static ValidatableResponse describe(String action, String filter,+ String value) {+ return dms(action)+ .body("{\"Filters\":[{\"Name\":\"" + filter + "\",\"Values\":[\"" + value + "\"]}]}")+ .when().post("/")+ .then().statusCode(200);+ }++ private static void delete(String action, String member, String value) {+ dms(action)+ .body("{\"" + member + "\":\"" + value + "\"}")+ .when().post("/")+ .then().statusCode(200);+ }++ private static RequestSpecification dms(String action) {+ return given()+ .contentType(CONTENT_TYPE)+ .header("X-Amz-Target", TARGET_PREFIX + action)+ .header("Authorization", AUTH_HEADER);+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.javaindex 3da3b7659e..a8dd1ccd96 100644--- a/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/dms/DmsServiceTest.java@@ -30,6 +30,7 @@ import static org.junit.jupiter.api.Assertions.assertTrue;import static org.mockito.ArgumentMatchers.any;import static org.mockito.ArgumentMatchers.anyList;import static org.mockito.ArgumentMatchers.anyMap;+import static org.mockito.ArgumentMatchers.anyString;import static org.mockito.ArgumentMatchers.eq;import static org.mockito.Mockito.mock;import static org.mockito.Mockito.when;@@ -48,6 +49,8 @@ class DmsServiceTest {StorageFactory storageFactory = mock(StorageFactory.class);AccountAwareStorageBackend<ReplicationSubnetGroup> store =AccountAwareStorageBackend.inMemory(ACCOUNT_ID);+ when(storageFactory.create(eq("dms"), anyString(), any(TypeReference.class)))+ .thenAnswer(invocation -> AccountAwareStorageBackend.inMemory(ACCOUNT_ID));when(storageFactory.create(eq("dms"), eq("dms-replication-subnet-groups.json"), any(TypeReference.class))).thenReturn((AccountAwareStorageBackend) store);