Light, fluffy, and always free - The AWS Local Emulator alternative
feat(dynamodb): optional DynamoDB Local backend (#4504)
Mario Toffiac326dce97738
diff --git a/docs/configuration/advanced/application-yml.md b/docs/configuration/advanced/application-yml.mdindex b33006a258..662c2deadc 100644--- a/docs/configuration/advanced/application-yml.md+++ b/docs/configuration/advanced/application-yml.md@@ -130,6 +130,10 @@ floci:dynamodb:enabled: true+ backend: native # native | local (forward calls to DynamoDB Local)+ # local-endpoint: http://dynamodb-local:8000 # DynamoDB Local base URL, required when backend is local+ local-connect-timeout-seconds: 2 # Seconds to wait for a connection to DynamoDB Local+ local-request-timeout-seconds: 10 # Seconds to wait for DynamoDB Local to answer a forwarded requestsns:enabled: truediff --git a/docs/configuration/environment-variables.md b/docs/configuration/environment-variables.mdindex 28c3edc8d1..c1625bd590 100644--- a/docs/configuration/environment-variables.md+++ b/docs/configuration/environment-variables.md@@ -202,6 +202,10 @@ See [Initialization Hooks](./initialization-hooks.md) for lifecycle phases and s| Variable | Default | Description ||---|---|---|| `FLOCI_SERVICES_DYNAMODB_ENABLED` | `true` | Enable the DynamoDB service |+| `FLOCI_SERVICES_DYNAMODB_BACKEND` | `native` | Engine behind the DynamoDB API: `native` or `local` (DynamoDB Local), case-insensitive. Any other value fails startup. See [DynamoDB Local backend](../services/dynamodb.md#dynamodb-local-backend) |+| `FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT` | _(none)_ | Base URL of DynamoDB Local, for example `http://dynamodb-local:8000`. Required when the backend is `local`. Must be `http` or `https` with a host. AWS endpoints (hosts under `amazonaws.com`, `amazonaws.com.cn` or `api.aws`) are rejected |+| `FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS` | `2` | Seconds to wait for a connection to DynamoDB Local |+| `FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS` | `10` | Seconds to wait for DynamoDB Local to answer a forwarded request |### Lambdadiff --git a/docs/services/dynamodb.md b/docs/services/dynamodb.mdindex cbf44c7670..6eb99b1660 100644--- a/docs/services/dynamodb.md+++ b/docs/services/dynamodb.md@@ -70,6 +70,88 @@ epoch time in the past. Each deletion writes a `REMOVE` record to the table's stforwarded to an active Kinesis streaming destination. Between sweeps, reads already leave expireditems out.+## DynamoDB Local backend++By default Floci stores tables in its own engine, the `native` backend. With+`FLOCI_SERVICES_DYNAMODB_BACKEND=local`, Floci forwards DynamoDB calls to an+[Amazon DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)+instance that you run.++Floci keeps serving the DynamoDB and DynamoDB Streams APIs on port 4566, in JSON and CBOR, with+the same authentication, IAM enforcement and CRC32 checksums as the native backend. Each call is+forwarded to DynamoDB Local over SigV4-signed HTTP/1.1. Floci never retries a forwarded request.+Floci does not start DynamoDB Local. Run it yourself, for example as a container:++```yaml+services:+ dynamodb-local:+ image: amazon/dynamodb-local:3.3.1+ command: ["-jar", "DynamoDBLocal.jar", "-inMemory"]+ floci:+ image: floci/floci:latest+ ports:+ - "4566:4566"+ environment:+ FLOCI_SERVICES_DYNAMODB_BACKEND: local+ FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT: http://dynamodb-local:8000+ depends_on:+ - dynamodb-local+```++Do not start DynamoDB Local with `-sharedDb`. It merges every account and region into one database.++Floci is tested with DynamoDB Local 3.3.1 and does not check the version it connects to. It relies+on DynamoDB Local's fixed `ddblocal` ARNs and on its separate database per access key and region.+A version that changes either one breaks ARN translation or account and region isolation.++At startup Floci waits up to 30 seconds for DynamoDB Local to answer, then fails. Any+non-success answer to that probe fails startup at once. `GET /_floci/info` reports the active+backend in `dynamodb_backend` (`native` or `local`).++### Accounts, regions and ARNs++Each account and region is a separate DynamoDB Local namespace. Floci signs forwarded calls with+the access key `floci<account-id>` and the caller's region, and DynamoDB Local keeps one database+per access key and region. A call from account `000000000000` in `eu-west-1` is signed with access+key `floci000000000000` and region `eu-west-1`.++Table, index and stream ARNs in replies are public ARNs for the caller's account and region, such+as `arn:aws:dynamodb:eu-west-1:000000000000:table/Users`. Floci rewrites the `ddblocal` ARNs that+DynamoDB Local returns. Stream records carry the caller's region. An ARN for another account or+region is rejected.++### What Floci adds++- **Tags.** DynamoDB Local has no tagging, so Floci handles `TagResource`, `UntagResource`,+ `ListTagsOfResource` and `Tags` on `CreateTable` itself. It keeps the tags in its DynamoDB+ storage, in the file `dynamodb-local-tags.json`. The tags follow Floci's storage mode, so with+ `memory` storage they are lost when Floci restarts while the tables stay in DynamoDB Local.+- **Stream consumers.** Lambda event source mappings, EventBridge Pipes and other Floci stream+ consumers read the DynamoDB Local streams through the Streams API.++### Limits++- **No emulator reset.** `POST /_floci/state/reset` is refused with HTTP 409. Reset the DynamoDB+ Local instance instead.+- **No replicas.** Replicas and global table replica updates (`UpdateTable` with `ReplicaUpdates`)+ are refused with a `ValidationException`. A CloudFormation global table with no extra replica+ regions deploys as a single table.+- **No `TableId`.** DynamoDB Local returns none, so `DescribeTable` has no `TableId` and+ CloudFormation `Fn::GetAtt` on `TableId` is empty.+- **Short names in batch replies.** Batch replies keyed by table always use the short table name,+ including when the request named the table by ARN.+- **No tag validation.** Floci does not check tag counts or tag keys.+- **Resource Explorer searches regions Floci knows.** DynamoDB Local cannot list its namespaces,+ so Resource Explorer lists every table in each region where Floci's storage holds at least one+ table record, including tables created there directly on DynamoDB Local. A region with no such+ record is not searched: one whose tables were all created directly on DynamoDB Local, or whose+ records were lost with `memory` storage.+- **No data migration.** Data does not move between `native` and `local`. After a switch, you see+ what that engine holds.+- **DynamoDB Local sets the feature set.** Operations and features follow DynamoDB Local; see the+ [DynamoDB Local documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)+ for what it supports. The other sections of this page describe the native backend.+## Configuration| Variable | Default | Description |@@ -79,6 +161,10 @@ items out.| `FLOCI_STORAGE_SERVICES_DYNAMODB_FLUSH_INTERVAL_MS` | `5000` | Flush interval for `hybrid`/`wal` storage modes (milliseconds) || `FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_ALLOCATION_SECONDS` | `4` | Seconds a vector index added by `UpdateTable` spends in resource allocation || `FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_BACKFILL_SECONDS` | `10` | Seconds that index then spends backfilling before it goes `ACTIVE` |+| `FLOCI_SERVICES_DYNAMODB_BACKEND` | `native` | Engine behind the DynamoDB API: `native` or `local`, case-insensitive. Any other value fails startup. See [DynamoDB Local backend](#dynamodb-local-backend) |+| `FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT` | *(none)* | Base URL of DynamoDB Local, for example `http://dynamodb-local:8000`. Required when the backend is `local`. Must be `http` or `https` with a host. AWS endpoints (hosts under `amazonaws.com`, `amazonaws.com.cn` or `api.aws`) are rejected |+| `FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS` | `2` | Seconds to wait for a connection to DynamoDB Local |+| `FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS` | `10` | Seconds to wait for DynamoDB Local to answer a forwarded request |### Storage and Performancediff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.javaindex 18ae70b2af..2a33312296 100644--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java@@ -1342,6 +1342,21 @@ public interface EmulatorConfig {*/@WithDefault("10")int vectorIndexBackfillSeconds();++ /** The engine behind the DynamoDB API: {@code native} or {@code local}. Env: FLOCI_SERVICES_DYNAMODB_BACKEND. */+ @WithDefault("native")+ String backend();++ /** The DynamoDB Local base URL, required when backend is local. Env: FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT. */+ Optional<String> localEndpoint();++ /** Env: FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS. */+ @WithDefault("2")+ int localConnectTimeoutSeconds();++ /** Env: FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS. */+ @WithDefault("10")+ int localRequestTimeoutSeconds();}interface SnsServiceConfig {diff --git a/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java b/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.javaindex 8f90b27a1d..3f6d21d210 100644--- a/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java+++ b/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java@@ -7,6 +7,7 @@ import io.github.hectorvent.floci.core.common.ContainerTeardown;import io.github.hectorvent.floci.core.common.ContainerTeardowns;import io.github.hectorvent.floci.core.common.ServiceRegistry;import io.github.hectorvent.floci.lifecycle.inithook.InitializationHook;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbBackendSelector;import jakarta.inject.Inject;import jakarta.ws.rs.GET;import jakarta.ws.rs.Path;@@ -44,6 +45,7 @@ public class EmulatorInfoController {private final Instance<ContainerTeardown> containerTeardowns;private final FlociCertificateAuthority certificateAuthority;private final EmulatorConfig config;+ private final DynamoDbBackendSelector dynamoDbBackendSelector;@Injectpublic EmulatorInfoController(ServiceRegistry serviceRegistry,@@ -52,7 +54,8 @@ public class EmulatorInfoController {Instance<Resettable> resettables,Instance<ContainerTeardown> containerTeardowns,FlociCertificateAuthority certificateAuthority,- EmulatorConfig config) {+ EmulatorConfig config,+ DynamoDbBackendSelector dynamoDbBackendSelector) {this.serviceRegistry = serviceRegistry;this.initLifecycleState = initLifecycleState;this.storageFactory = storageFactory;@@ -60,6 +63,7 @@ public class EmulatorInfoController {this.containerTeardowns = containerTeardowns;this.certificateAuthority = certificateAuthority;this.config = config;+ this.dynamoDbBackendSelector = dynamoDbBackendSelector;this.version = resolveVersion();}@@ -98,7 +102,8 @@ public class EmulatorInfoController {@GET@Path("/info")public Response info() {- return Response.ok(Map.of("version", version, "edition", "community", "original_edition", "floci-always-free")).build();+ return Response.ok(Map.of("version", version, "edition", "community", "original_edition", "floci-always-free",+ "dynamodb_backend", dynamoDbBackendSelector.selected())).build();}@GETdiff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.javaindex f3f7f7aa53..fddba393f8 100644--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java@@ -3,10 +3,8 @@ package io.github.hectorvent.floci.services.dynamodb.backend;import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.node.JsonNodeFactory;import com.fasterxml.jackson.databind.node.ObjectNode;-import io.github.hectorvent.floci.core.common.AwsException;import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Call;-import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;import java.util.ArrayList;import java.util.List;@@ -78,22 +76,13 @@ public final class DynamoDbApiStreamReader implements DynamoDbStreamReader {}private JsonNode call(Stream stream, String action, ObjectNode body) {- Reply reply;try {- reply = operations.execute(new Call(stream.scope(), Api.DYNAMODB_STREAMS, action, body));+ return operations.execute(new Call(stream.scope(), Api.DYNAMODB_STREAMS, action, body)).successBody();} catch (RuntimeException e) {throw e;} catch (Exception e) {throw new IllegalStateException(e);}- JsonNode replyBody = reply.body() == null ? JSON.missingNode() : reply.body();- if (reply.status() >= 300) {- String type = text(replyBody, "__type");- String code = type == null ? "InternalServerError" : type.substring(type.lastIndexOf('#') + 1);- String message = text(replyBody, "message");- throw new AwsException(code, message != null ? message : text(replyBody, "Message"), reply.status());- }- return replyBody;}private static String text(JsonNode node, String field) {diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackend.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackend.javanew file mode 100644index 0000000000..6a8acdf453--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackend.java@@ -0,0 +1,5 @@+package io.github.hectorvent.floci.services.dynamodb.backend;++/** One selectable DynamoDB engine: every seam interface from a single object. */+public interface DynamoDbBackend extends DynamoDbOperations, DynamoDbItemAccess, DynamoDbTableAccess,+ DynamoDbBackendLifecycle {}diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.javaindex 76a5eb04ab..551679deb4 100644--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java@@ -1,49 +1,71 @@package io.github.hectorvent.floci.services.dynamodb.backend;+import io.github.hectorvent.floci.config.EmulatorConfig;import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.CheckpointLifetime;import jakarta.enterprise.context.ApplicationScoped;import jakarta.enterprise.inject.Produces;import jakarta.inject.Inject;+import java.util.Locale;+/**- * The one immutable DynamoDB backend selection. Native is the only engine today; a configurable- * choice arrives with the alternative engine. {@link NativeDynamoDbBackend} is typed to its concrete- * class and {@link DynamoDbApiStreamReader} is not a bean, so these producers are the only- * unqualified beans for the five seam interfaces.+ * The one immutable DynamoDB backend selection, read from {@code floci.services.dynamodb.backend}:+ * {@code native} (the default) runs the in-process engine, {@code local} forwards to an external+ * DynamoDB Local. Both engines are {@code @ApplicationScoped} client proxies, so the unselected one+ * is never constructed. Each engine is typed to its concrete class and {@link DynamoDbApiStreamReader}+ * is not a bean, so these producers are the only unqualified beans for the five seam interfaces.*/@ApplicationScopedpublic class DynamoDbBackendSelector {- private final NativeDynamoDbBackend nativeBackend;+ private final String selected;+ private final DynamoDbBackend backend;@Inject- public DynamoDbBackendSelector(NativeDynamoDbBackend nativeBackend) {- this.nativeBackend = nativeBackend;+ public DynamoDbBackendSelector(EmulatorConfig config, NativeDynamoDbBackend nativeBackend,+ DynamoDbLocalBackend localBackend) {+ String configured = config.services().dynamodb().backend();+ selected = configured.trim().toLowerCase(Locale.ROOT);+ backend = switch (selected) {+ case "native" -> nativeBackend;+ case "local" -> localBackend;+ default -> throw new IllegalStateException(+ "floci.services.dynamodb.backend must be 'native' or 'local', got '" + configured + "'");+ };+ }++ /** The selected engine name, {@code native} or {@code local}. */+ public String selected() {+ return selected;}@ProducesDynamoDbOperations operations() {- return nativeBackend;+ return backend;}@ProducesDynamoDbItemAccess itemAccess() {- return nativeBackend;+ return backend;}@ProducesDynamoDbTableAccess tableAccess() {- return nativeBackend;+ return backend;}@ProducesDynamoDbBackendLifecycle lifecycle() {- return nativeBackend;+ return backend;}- /** Native stream history lives only in memory, so committed progress ends with the process. */+ /**+ * Native stream history lives only in memory, so committed progress ends with the process;+ * DynamoDB Local keeps stream history outside it, so progress lasts as long as the stream.+ */@ProducesDynamoDbStreamReader streamReader() {- return new DynamoDbApiStreamReader(nativeBackend, CheckpointLifetime.PROCESS);+ return new DynamoDbApiStreamReader(backend,+ "local".equals(selected) ? CheckpointLifetime.STREAM : CheckpointLifetime.PROCESS);}}diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackend.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackend.javanew file mode 100644index 0000000000..768d3d840c--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackend.java@@ -0,0 +1,627 @@+package io.github.hectorvent.floci.services.dynamodb.backend;++import com.fasterxml.jackson.core.type.TypeReference;+import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.PropertyNamingStrategies;+import com.fasterxml.jackson.databind.node.ArrayNode;+import com.fasterxml.jackson.databind.node.MissingNode;+import com.fasterxml.jackson.databind.node.NullNode;+import com.fasterxml.jackson.databind.node.ObjectNode;+import com.fasterxml.jackson.databind.node.TextNode;+import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.core.common.AwsArnUtils;+import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.core.common.AwsPartitions;+import io.github.hectorvent.floci.core.resource.ExplorerResource;+import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;+import io.github.hectorvent.floci.core.storage.StorageFactory;+import io.github.hectorvent.floci.services.dynamodb.DynamoDbTableNames;+import io.github.hectorvent.floci.services.dynamodb.model.AttributeDefinition;+import io.github.hectorvent.floci.services.dynamodb.model.GlobalSecondaryIndex;+import io.github.hectorvent.floci.services.dynamodb.model.KeySchemaElement;+import io.github.hectorvent.floci.services.dynamodb.model.LocalSecondaryIndex;+import io.github.hectorvent.floci.services.dynamodb.model.TableDefinition;+import jakarta.enterprise.context.ApplicationScoped;+import jakarta.enterprise.inject.Typed;+import jakarta.inject.Inject;+import org.jboss.logging.Logger;++import java.io.IOException;+import java.net.URI;+import java.net.URISyntaxException;+import java.time.Duration;+import java.time.Instant;+import java.util.ArrayList;+import java.util.LinkedHashMap;+import java.util.List;+import java.util.Map;+import java.util.Optional;+import java.util.regex.Pattern;++/**+ * The DynamoDB engine backed by an external, official DynamoDB Local. Wire calls are forwarded+ * through {@link DynamoDbLocalClient} under the caller's account and region; table and stream ARNs+ * are translated between the public shape and Local's fixed {@code ddblocal} shape. Local keeps no+ * tags, so Floci holds them in its own store keyed by account, region and table name.+ */+@ApplicationScoped+@Typed(DynamoDbLocalBackend.class)+public class DynamoDbLocalBackend implements DynamoDbBackend {++ private static final Logger LOG = Logger.getLogger(DynamoDbLocalBackend.class);++ private static final String LOCAL_ARN_PREFIX = "arn:aws:dynamodb:ddblocal:000000000000:"; // partition-literal: DynamoDB Local's fixed ARN prefix+ private static final String LOCAL_REGION = "ddblocal";+ private static final Duration READINESS_BUDGET = Duration.ofSeconds(30);+ private static final Scope PROBE = new Scope("000000000000", "us-east-1"); // partition-literal: readiness probe namespace, Local accepts any region+ private static final String REPLICAS_UNSUPPORTED = "Replicas are not supported by the DynamoDB Local backend";+ private static final Pattern PARTITION = Pattern.compile(AwsArnUtils.PARTITION_REGEX);+ private static final List<String> PUBLIC_ARN_FIELDS = List.of("TableArn", "IndexArn", "LatestStreamArn",+ "StreamArn", "LastEvaluatedStreamArn", "message", "Message");++ private final DynamoDbLocalClient client;+ private final AccountAwareStorageBackend<Map<String, String>> tags;+ private final ObjectMapper objectMapper;+ private final ObjectMapper aws;+ private final Duration readiness;++ @Inject+ public DynamoDbLocalBackend(EmulatorConfig config, StorageFactory storageFactory, ObjectMapper objectMapper) {+ this(new DynamoDbLocalClient(validatedEndpoint(config.services().dynamodb().localEndpoint()),+ Duration.ofSeconds(config.services().dynamodb().localConnectTimeoutSeconds()),+ Duration.ofSeconds(config.services().dynamodb().localRequestTimeoutSeconds()), objectMapper),+ storageFactory.create("dynamodb", "dynamodb-local-tags.json",+ new TypeReference<Map<String, Map<String, String>>>() {}),+ objectMapper, READINESS_BUDGET);+ }++ DynamoDbLocalBackend(DynamoDbLocalClient client, AccountAwareStorageBackend<Map<String, String>> tags,+ ObjectMapper objectMapper, Duration readiness) {+ this.client = client;+ this.tags = tags;+ this.objectMapper = objectMapper;+ this.aws = objectMapper.copy().setPropertyNamingStrategy(PropertyNamingStrategies.UPPER_CAMEL_CASE);+ this.readiness = readiness;+ }++ static URI validatedEndpoint(Optional<String> value) {+ String raw = value.map(String::trim).filter(v -> !v.isEmpty()).orElseThrow(() -> new IllegalStateException(+ "floci.services.dynamodb.local-endpoint is required when floci.services.dynamodb.backend is local"));+ URI uri;+ try {+ uri = new URI(raw);+ } catch (URISyntaxException e) {+ throw invalidEndpoint(raw, e);+ }+ String scheme = uri.getScheme();+ if (scheme == null || !(scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))+ || uri.getHost() == null || uri.getRawUserInfo() != null) {+ throw invalidEndpoint(raw, null);+ }+ String host = uri.getHost().replaceFirst("\\.$", "");+ if (AwsPartitions.isDnsSuffix(host) || AwsPartitions.stripKnownDnsSuffix(host).isPresent()) {+ throw new IllegalStateException("floci.services.dynamodb.local-endpoint '" + raw+ + "' must point at DynamoDB Local, not an AWS endpoint");+ }+ return uri;+ }++ private static IllegalStateException invalidEndpoint(String raw, Exception cause) {+ return new IllegalStateException("floci.services.dynamodb.local-endpoint '" + raw+ + "' is not a valid http or https URL", cause);+ }++ @Override+ public Reply execute(Call call) {+ Scope scope = call.scope();+ String action = call.action();+ JsonNode body = call.body() == null ? objectMapper.createObjectNode() : call.body().deepCopy();+ if (!"CreateTable".equals(action)) {+ toLocal(scope, body);+ }+ Reply served = call.api() != Api.DYNAMODB ? null : switch (action) {+ case "TagResource" -> {+ tagResource(scope, text(body, "ResourceArn"), tagsFrom(body.path("Tags")));+ yield new Reply(200, objectMapper.createObjectNode(), Map.of());+ }+ case "UntagResource" -> {+ List<String> keys = new ArrayList<>();+ body.path("TagKeys").forEach(key -> keys.add(key.asText()));+ untagResource(scope, text(body, "ResourceArn"), keys);+ yield new Reply(200, objectMapper.createObjectNode(), Map.of());+ }+ case "ListTagsOfResource" -> {+ String resourceArn = text(body, "ResourceArn");+ Map<String, String> found;+ try {+ found = listTagsOfResource(scope, resourceArn);+ } catch (AwsException e) {+ if (!"ResourceNotFoundException".equals(e.getErrorCode())) {+ throw e;+ }+ throw new AwsException("AccessDeniedException",+ "User is not authorized to perform: dynamodb:ListTagsOfResource on resource: "+ + resourceArn, 400);+ }+ ObjectNode reply = objectMapper.createObjectNode();+ ArrayNode tagList = reply.putArray("Tags");+ found.forEach((key, value) -> tagList.addObject().put("Key", key).put("Value", value));+ yield new Reply(200, reply, Map.of());+ }+ case "UpdateTable" -> {+ if (body.has("ReplicaUpdates")) {+ throw new AwsException("ValidationException", REPLICAS_UNSUPPORTED, 400);+ }+ yield null;+ }+ default -> null;+ };+ if (served != null) {+ return served;+ }+ Reply reply = send(scope, call.api(), action, body);+ if (reply.status() >= 200 && reply.status() < 300 && call.api() == Api.DYNAMODB) {+ JsonNode replyBody = reply.body() == null ? MissingNode.getInstance() : reply.body();+ String tableName = text(replyBody.path("TableDescription"), "TableName");+ if (tableName != null && "CreateTable".equals(action)) {+ tags.putForAccount(scope.accountId(), key(scope.region(), tableName), tagsFrom(body.path("Tags")));+ } else if (tableName != null && "DeleteTable".equals(action)) {+ tags.deleteForAccount(scope.accountId(), key(scope.region(), tableName));+ }+ }+ toPublic(scope, reply.body());+ return reply;+ }++ private Reply send(Scope scope, Api api, String action, JsonNode body) {+ try {+ return client.send(scope, api, action, body);+ } catch (IOException e) {+ throw notAnswered(e);+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ throw notAnswered(e);+ }+ }++ private AwsException notAnswered(Exception e) {+ return new AwsException("InternalServerError",+ "DynamoDB Local at " + client.endpoint() + " did not answer: " + e, 500);+ }++ static void toLocal(Scope scope, JsonNode node) {+ if (node instanceof ObjectNode object) {+ if (object.get("RequestItems") instanceof ObjectNode requestItems) {+ ObjectNode renamed = object.objectNode();+ for (Map.Entry<String, JsonNode> entry : requestItems.properties()) {+ String name = tableName(scope, entry.getKey());+ if (renamed.replace(name, entry.getValue()) != null) {+ throw new AwsException("ValidationException",+ "Table " + name + " is named more than once in RequestItems", 400);+ }+ }+ object.set("RequestItems", renamed);+ }+ for (Map.Entry<String, JsonNode> field : object.properties()) {+ JsonNode value = field.getValue();+ if (value.isTextual() && "TableName".equals(field.getKey())) {+ field.setValue(TextNode.valueOf(tableName(scope, value.asText())));+ } else if (value.isTextual()+ && ("StreamArn".equals(field.getKey()) || "ExclusiveStartStreamArn".equals(field.getKey()))) {+ field.setValue(TextNode.valueOf(localStreamArn(scope, value.asText())));+ } else if (value.isContainerNode()) {+ toLocal(scope, value);+ }+ }+ } else if (node instanceof ArrayNode array) {+ array.forEach(element -> toLocal(scope, element));+ }+ }++ static String tableName(Scope scope, String value) {+ if (!value.startsWith("arn:")) {+ return value;+ }+ DynamoDbTableNames.resolveWithRegion(value, scope.region());+ return ownTableName(scope, value);+ }++ // A well-formed table ARN of another account, region or partition names no table in this namespace.+ private static String ownTableName(Scope scope, String arn) {+ String name = DynamoDbTableNames.resolve(arn);+ if (!PARTITION.matcher(AwsArnUtils.parse(arn).partition()).matches()) {+ throw new AwsException("ValidationException", "Invalid table ARN: " + arn, 400);+ }+ if (!arn.startsWith(publicPrefix(scope))) {+ throw new AwsException("ResourceNotFoundException",+ "Requested resource not found: Table: " + arn + " not found", 400);+ }+ return name;+ }++ static String localStreamArn(Scope scope, String value) {+ if (!AwsArnUtils.isArn(value)) {+ return value;+ }+ AwsArnUtils.Arn arn = AwsArnUtils.parse(value);+ if (!"dynamodb".equals(arn.service()) || !arn.resource().contains("/stream/")) {+ return value;+ }+ if (!value.startsWith(publicPrefix(scope))) {+ throw new AwsException("ResourceNotFoundException",+ "Requested resource not found: Stream: " + value + " not found", 400);+ }+ return LOCAL_ARN_PREFIX + arn.resource();+ }++ private static String publicPrefix(Scope scope) {+ return AwsArnUtils.Arn.of("dynamodb", scope.region(), scope.accountId(), "").toString();+ }++ static void toPublic(Scope scope, JsonNode node) {+ toPublic(node, publicPrefix(scope), scope.region());+ }++ private static void toPublic(JsonNode node, String prefix, String region) {+ if (node instanceof ObjectNode object) {+ for (Map.Entry<String, JsonNode> field : object.properties()) {+ JsonNode value = field.getValue();+ if (value.isTextual() && PUBLIC_ARN_FIELDS.contains(field.getKey())) {+ field.setValue(TextNode.valueOf(value.asText().replace(LOCAL_ARN_PREFIX, prefix)));+ } else if (value.isTextual() && "awsRegion".equals(field.getKey())+ && LOCAL_REGION.equals(value.asText())) {+ field.setValue(TextNode.valueOf(region));+ } else if (value.isContainerNode()) {+ toPublic(value, prefix, region);+ }+ }+ } else if (node instanceof ArrayNode array) {+ array.forEach(element -> toPublic(element, prefix, region));+ }+ }++ private JsonNode call(Scope scope, String action, ObjectNode body) {+ return execute(new Call(scope, Api.DYNAMODB, action, body)).successBody();+ }++ private ObjectNode body(String tableName, Object... nameValuePairs) {+ ObjectNode body = aws.createObjectNode().put("TableName", tableName);+ for (int i = 0; i < nameValuePairs.length; i += 2) {+ Object value = nameValuePairs[i + 1];+ if (value == null || value instanceof NullNode || value instanceof MissingNode) {+ continue;+ }+ String name = (String) nameValuePairs[i];+ body.set(name, value instanceof JsonNode node ? node : aws.valueToTree(value));+ }+ return body;+ }++ @Override+ public void putItem(Scope scope, String tableName, JsonNode item, String conditionExpression,+ JsonNode expressionAttributeNames, JsonNode expressionAttributeValues) {+ call(scope, "PutItem", body(tableName, "Item", item, "ConditionExpression", conditionExpression,+ "ExpressionAttributeNames", expressionAttributeNames,+ "ExpressionAttributeValues", expressionAttributeValues));+ }++ @Override+ public JsonNode getItem(Scope scope, String tableName, JsonNode key) {+ return present(call(scope, "GetItem", body(tableName, "Key", key)).get("Item"));+ }++ @Override+ public void deleteItem(Scope scope, String tableName, JsonNode key, String conditionExpression,+ JsonNode expressionAttributeNames, JsonNode expressionAttributeValues) {+ call(scope, "DeleteItem", body(tableName, "Key", key, "ConditionExpression", conditionExpression,+ "ExpressionAttributeNames", expressionAttributeNames,+ "ExpressionAttributeValues", expressionAttributeValues));+ }++ @Override+ public JsonNode updateItem(Scope scope, String tableName, JsonNode key, JsonNode attributeUpdates,+ String updateExpression, JsonNode expressionAttributeNames,+ JsonNode expressionAttributeValues, String returnValues, String conditionExpression) {+ JsonNode reply = call(scope, "UpdateItem", body(tableName, "Key", key, "AttributeUpdates", attributeUpdates,+ "UpdateExpression", updateExpression, "ExpressionAttributeNames", expressionAttributeNames,+ "ExpressionAttributeValues", expressionAttributeValues, "ReturnValues", returnValues,+ "ConditionExpression", conditionExpression));+ if ("ALL_NEW".equals(returnValues) || "ALL_OLD".equals(returnValues)) {+ return present(reply.get("Attributes"));+ }+ return null;+ }++ @Override+ public ScanPage scan(Scope scope, String tableName, String filterExpression, JsonNode expressionAttributeNames,+ JsonNode expressionAttributeValues, JsonNode scanFilter, Integer limit,+ JsonNode exclusiveStartKey) {+ JsonNode reply = call(scope, "Scan", body(tableName, "FilterExpression", filterExpression,+ "ExpressionAttributeNames", expressionAttributeNames,+ "ExpressionAttributeValues", expressionAttributeValues, "ScanFilter", scanFilter, "Limit", limit,+ "ExclusiveStartKey", exclusiveStartKey));+ List<JsonNode> items = new ArrayList<>();+ reply.path("Items").forEach(items::add);+ return new ScanPage(items, reply.path("ScannedCount").asInt(), present(reply.get("LastEvaluatedKey")));+ }++ private static JsonNode present(JsonNode node) {+ return node == null || node.isNull() ? null : node;+ }++ @Override+ public TableDefinition createTable(Scope scope, String tableName, List<KeySchemaElement> keySchema,+ List<AttributeDefinition> attributeDefinitions, Long readCapacity,+ Long writeCapacity, List<GlobalSecondaryIndex> globalSecondaryIndexes,+ List<LocalSecondaryIndex> localSecondaryIndexes) {+ ObjectNode body = body(tableName, "KeySchema", keySchema, "AttributeDefinitions", attributeDefinitions);+ ObjectNode throughput = null;+ if (readCapacity != null && writeCapacity != null) {+ throughput = aws.createObjectNode()+ .put("ReadCapacityUnits", readCapacity)+ .put("WriteCapacityUnits", writeCapacity);+ body.put("BillingMode", "PROVISIONED").set("ProvisionedThroughput", throughput);+ } else {+ body.put("BillingMode", "PAY_PER_REQUEST");+ }+ if (globalSecondaryIndexes != null && !globalSecondaryIndexes.isEmpty()) {+ ArrayNode indexes = body.putArray("GlobalSecondaryIndexes");+ for (GlobalSecondaryIndex gsi : globalSecondaryIndexes) {+ ObjectNode index = index(indexes, gsi.getIndexName(), gsi.getKeySchema(), gsi.getProjectionType(),+ gsi.getNonKeyAttributes());+ if (throughput != null) {+ index.set("ProvisionedThroughput", throughput.deepCopy());+ }+ }+ }+ if (localSecondaryIndexes != null && !localSecondaryIndexes.isEmpty()) {+ ArrayNode indexes = body.putArray("LocalSecondaryIndexes");+ for (LocalSecondaryIndex lsi : localSecondaryIndexes) {+ index(indexes, lsi.getIndexName(), lsi.getKeySchema(), lsi.getProjectionType(),+ lsi.getNonKeyAttributes());+ }+ }+ return table(call(scope, "CreateTable", body).path("TableDescription"));+ }++ private ObjectNode index(ArrayNode indexes, String indexName, List<KeySchemaElement> keySchema,+ String projectionType, List<String> nonKeyAttributes) {+ ObjectNode index = indexes.addObject().put("IndexName", indexName);+ index.set("KeySchema", aws.valueToTree(keySchema));+ ObjectNode projection = index.putObject("Projection")+ .put("ProjectionType", projectionType == null ? "ALL" : projectionType);+ if (nonKeyAttributes != null && !nonKeyAttributes.isEmpty()) {+ projection.set("NonKeyAttributes", aws.valueToTree(nonKeyAttributes));+ }+ return index;+ }++ @Override+ public TableDefinition describeTable(Scope scope, String tableName) {+ return table(call(scope, "DescribeTable", body(tableName)).path("Table"));+ }++ @Override+ public Optional<TableDefinition> findTable(Scope scope, String tableName) {+ try {+ return Optional.of(describeTable(scope, tableName));+ } catch (AwsException e) {+ if ("ResourceNotFoundException".equals(e.getErrorCode())) {+ return Optional.empty();+ }+ throw e;+ }+ }++ // The seam's direct delete skips deletion protection, like native; the wire DeleteTable still enforces it.+ // ponytail: a DeleteTable that fails after the UpdateTable leaves protection off; the stack delete that asked+ // for removal reports DELETE_FAILED and a retry completes it.+ @Override+ public void deleteTable(Scope scope, String tableName) {+ if (describeTable(scope, tableName).isDeletionProtectionEnabled()) {+ call(scope, "UpdateTable", body(tableName, "DeletionProtectionEnabled", false));+ }+ call(scope, "DeleteTable", body(tableName));+ }++ @Override+ public TableDefinition enableStream(Scope scope, String tableName, String viewType) {+ String type = viewType == null ? "NEW_AND_OLD_IMAGES" : viewType;+ TableDefinition current = describeTable(scope, tableName);+ if (current.isStreamEnabled()) {+ if (viewType == null || type.equals(current.getStreamViewType())) {+ return current;+ }+ // ponytail: Local cannot retarget a live stream, so a new view type starts a new stream ARN+ updateStream(scope, tableName, false, null);+ }+ return updateStream(scope, tableName, true, type);+ }++ @Override+ public TableDefinition disableStream(Scope scope, String tableName) {+ TableDefinition current = describeTable(scope, tableName);+ if (!current.isStreamEnabled()) {+ return current;+ }+ return updateStream(scope, tableName, false, null);+ }++ private TableDefinition updateStream(Scope scope, String tableName, boolean enabled, String viewType) {+ ObjectNode body = body(tableName);+ ObjectNode spec = body.putObject("StreamSpecification").put("StreamEnabled", enabled);+ if (viewType != null) {+ spec.put("StreamViewType", viewType);+ }+ return table(call(scope, "UpdateTable", body).path("TableDescription"));+ }++ private TableDefinition table(JsonNode description) {+ TableDefinition table = aws.convertValue(description, TableDefinition.class);+ JsonNode spec = description.path("StreamSpecification");+ table.setStreamEnabled(spec.path("StreamEnabled").asBoolean(false));+ table.setStreamViewType(text(spec, "StreamViewType"));+ table.setStreamArn(table.isStreamEnabled() ? text(description, "LatestStreamArn") : null);+ table.setTableId(description.path("TableId").asText());+ return table;+ }++ @Override+ public Map<String, String> listTagsOfResource(Scope scope, String resourceArn) {+ return new LinkedHashMap<>(tags.getForAccount(scope.accountId(), existingTableKey(scope, resourceArn))+ .orElse(Map.of()));+ }++ // ponytail: no tag limits or tag-key validation; Floci records what callers send.+ @Override+ public void tagResource(Scope scope, String resourceArn, Map<String, String> tagsToAdd) {+ String key = existingTableKey(scope, resourceArn);+ synchronized (tags) {+ Map<String, String> current = new LinkedHashMap<>(tags.getForAccount(scope.accountId(), key).orElse(Map.of()));+ current.putAll(tagsToAdd);+ tags.putForAccount(scope.accountId(), key, current);+ }+ }++ @Override+ public void untagResource(Scope scope, String resourceArn, List<String> tagKeys) {+ String key = existingTableKey(scope, resourceArn);+ synchronized (tags) {+ Map<String, String> current = new LinkedHashMap<>(tags.getForAccount(scope.accountId(), key).orElse(Map.of()));+ tagKeys.forEach(current::remove);+ tags.putForAccount(scope.accountId(), key, current);+ }+ }++ private String existingTableKey(Scope scope, String resourceArn) {+ if (resourceArn == null || !resourceArn.startsWith("arn:")) {+ throw new AwsException("ValidationException", "Invalid TableArn", 400);+ }+ String name = ownTableName(scope, resourceArn);+ describeTable(scope, name);+ return key(scope.region(), name);+ }++ @Override+ public TableDefinition applyReplicaUpdates(Scope scope, String tableName, List<String> addRegions,+ List<String> removeRegions) {+ throw new AwsException("ValidationException", REPLICAS_UNSUPPORTED, 400);+ }++ // ponytail: a single-region global table has no replicas, so it is the table itself.+ @Override+ public TableDefinition ensureGlobalTable(Scope scope, String tableName) {+ return describeTable(scope, tableName);+ }++ // ponytail: Local cannot list its namespaces, so regions come from the tag store: a region Floci+ // has no table record for (made directly on Local, or lost with memory storage) is not listed.+ @Override+ public List<ExplorerResource> resources(String accountId) {+ List<String> regions = tags.keysForAccount(accountId).stream()+ .map(k -> k.substring(0, k.indexOf('/')))+ .distinct()+ .sorted()+ .toList();+ List<ExplorerResource> resources = new ArrayList<>();+ for (String region : regions) {+ Scope scope = new Scope(accountId, region);+ String start = null;+ do {+ ObjectNode request = objectMapper.createObjectNode();+ if (start != null) {+ request.put("ExclusiveStartTableName", start);+ }+ JsonNode page = call(scope, "ListTables", request);+ for (JsonNode tableName : page.path("TableNames")) {+ String name = tableName.asText();+ resources.add(new ExplorerResource(+ AwsArnUtils.Arn.of("dynamodb", region, accountId, "table/" + name).toString(),+ "dynamodb:table", "dynamodb", region, accountId, Instant.now(),+ tags.getForAccount(accountId, key(region, name)).orElse(Map.of())));+ }+ start = text(page, "LastEvaluatedTableName");+ } while (start != null);+ }+ return resources;+ }++ @Override+ public void start() {+ long deadline = System.nanoTime() + readiness.toNanos();+ while (true) {+ if (ready(deadline)) {+ LOG.infov("DynamoDB backend: DynamoDB Local at {0}", client.endpoint());+ return;+ }+ try {+ Thread.sleep(500);+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ throw new IllegalStateException("Interrupted while waiting for DynamoDB Local at "+ + client.endpoint(), e);+ }+ }+ }++ private boolean ready(long deadline) {+ Reply reply;+ try {+ reply = client.send(PROBE, Api.DYNAMODB, "ListTables", objectMapper.createObjectNode().put("Limit", 1),+ Duration.ofNanos(Math.max(1, deadline - System.nanoTime())));+ } catch (IOException e) {+ if (System.nanoTime() - deadline >= 0) {+ throw new IllegalStateException("DynamoDB Local at " + client.endpoint() + " is not reachable after "+ + readiness.toSeconds() + "s", e);+ }+ return false;+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ throw new IllegalStateException("Interrupted while waiting for DynamoDB Local at " + client.endpoint(), e);+ }+ if (reply.status() >= 200 && reply.status() < 300) {+ return true;+ }+ throw new IllegalStateException("DynamoDB Local at " + client.endpoint() + " answered ListTables with status "+ + reply.status() + ": " + reply.body());+ }++ @Override+ public void checkReset() {+ throw new AwsException("UnsupportedOperation", "The DynamoDB Local backend keeps its data outside Floci; "+ + "reset the external DynamoDB Local instance instead", 409);+ }++ @Override+ public void beforeReset() {+ }++ @Override+ public void reset() {+ }++ @Override+ public void afterReset() {+ }++ @Override+ public void stop() {+ client.close();+ }++ private static String key(String region, String tableName) {+ return region + "/" + tableName;+ }++ private static Map<String, String> tagsFrom(JsonNode tagList) {+ Map<String, String> result = new LinkedHashMap<>();+ tagList.forEach(tag -> result.put(tag.path("Key").asText(), tag.path("Value").asText()));+ return result;+ }++ private static String text(JsonNode node, String field) {+ JsonNode value = node == null ? null : node.get(field);+ return value == null || value.isNull() ? null : value.asText();+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClient.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClient.javanew file mode 100644index 0000000000..bdfdeb7b14--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClient.java@@ -0,0 +1,107 @@+package io.github.hectorvent.floci.services.dynamodb.backend;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import io.github.hectorvent.floci.core.common.auth.SigV4RequestValidator;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;++import java.io.IOException;+import java.net.URI;+import java.net.http.HttpClient;+import java.net.http.HttpRequest;+import java.net.http.HttpResponse;+import java.time.Duration;+import java.time.ZoneOffset;+import java.time.ZonedDateTime;+import java.time.format.DateTimeFormatter;+import java.util.Map;++/**+ * Forwards DynamoDB and DynamoDB Streams wire calls to an external DynamoDB Local over SigV4-signed+ * HTTP. The caller's account becomes the access key and its region the signing region, so Local+ * keeps each account and region apart. Requests are never retried.+ */+class DynamoDbLocalClient implements AutoCloseable {++ private static final String CONTENT_TYPE = "application/x-amz-json-1.0";+ private static final String SIGNED_HEADERS = "content-type;host;x-amz-date;x-amz-target";+ private static final String SERVICE = "dynamodb";+ private static final String SECRET_KEY = "floci";+ private static final DateTimeFormatter AMZ_DATE = DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmss'Z'");++ private final URI endpoint;+ private final Duration requestTimeout;+ private final ObjectMapper mapper;+ private final HttpClient http;+ private final String host;+ private final String path;++ DynamoDbLocalClient(URI endpoint, Duration connectTimeout, Duration requestTimeout, ObjectMapper mapper) {+ this.endpoint = endpoint;+ this.requestTimeout = requestTimeout;+ this.mapper = mapper;+ this.http = HttpClient.newBuilder()+ .version(HttpClient.Version.HTTP_1_1)+ .connectTimeout(connectTimeout)+ .build();+ this.host = endpoint.getHost() + (endpoint.getPort() != -1 ? ":" + endpoint.getPort() : "");+ String rawPath = endpoint.getRawPath();+ this.path = rawPath == null || rawPath.isEmpty() ? "/" : rawPath;+ }++ Reply send(Scope scope, Api api, String action, JsonNode body) throws IOException, InterruptedException {+ return send(scope, api, action, body, requestTimeout);+ }++ Reply send(Scope scope, Api api, String action, JsonNode body, Duration budget)+ throws IOException, InterruptedException {+ byte[] payload = mapper.writeValueAsBytes(body == null ? mapper.createObjectNode() : body);+ String target = (api == Api.DYNAMODB ? "DynamoDB_20120810." : "DynamoDBStreams_20120810.") + action;+ String amzDate = AMZ_DATE.format(ZonedDateTime.now(ZoneOffset.UTC));+ HttpRequest request = HttpRequest.newBuilder(endpoint.resolve(path))+ .timeout(budget.compareTo(requestTimeout) < 0 ? budget : requestTimeout)+ .header("Content-Type", CONTENT_TYPE)+ .header("X-Amz-Date", amzDate)+ .header("X-Amz-Target", target)+ .header("Authorization", authorization(scope, target, amzDate, payload))+ .POST(HttpRequest.BodyPublishers.ofByteArray(payload))+ .build();+ HttpResponse<byte[]> response = http.send(request, HttpResponse.BodyHandlers.ofByteArray());+ byte[] responseBody = response.body();+ JsonNode replyBody = responseBody == null || responseBody.length == 0 ? null : mapper.readTree(responseBody);+ return new Reply(response.statusCode(), replyBody, Map.of());+ }++ URI endpoint() {+ return endpoint;+ }++ @Override+ public void close() {+ http.shutdownNow();+ }++ private String authorization(Scope scope, String target, String amzDate, byte[] payload) {+ String date = amzDate.substring(0, 8);+ String credentialScope = date + "/" + scope.region() + "/" + SERVICE + "/aws4_request";+ try {+ String canonical = "POST\n" + path + "\n\n"+ + "content-type:" + CONTENT_TYPE + "\n"+ + "host:" + host + "\n"+ + "x-amz-date:" + amzDate + "\n"+ + "x-amz-target:" + target + "\n\n"+ + SIGNED_HEADERS + "\n"+ + SigV4RequestValidator.sha256Hex(payload);+ String toSign = "AWS4-HMAC-SHA256\n" + amzDate + "\n" + credentialScope + "\n"+ + SigV4RequestValidator.sha256Hex(canonical);+ byte[] signingKey = SigV4RequestValidator.deriveSigningKey(SECRET_KEY, date, scope.region(), SERVICE);+ String signature = SigV4RequestValidator.hexEncode(SigV4RequestValidator.hmacSha256(signingKey, toSign));+ return "AWS4-HMAC-SHA256 Credential=floci" + scope.accountId() + "/" + credentialScope+ + ", SignedHeaders=" + SIGNED_HEADERS + ", Signature=" + signature;+ } catch (Exception e) {+ throw new IllegalStateException("Failed to sign DynamoDB Local request", e);+ }+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.javaindex 566d2a0902..fe2a496a7f 100644--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java@@ -1,6 +1,8 @@package io.github.hectorvent.floci.services.dynamodb.backend;import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.node.MissingNode;+import io.github.hectorvent.floci.core.common.AwsException;import java.util.List;import java.util.Map;@@ -43,6 +45,23 @@ public interface DynamoDbOperations {public Reply {headers = headers == null ? Map.of() : Map.copyOf(headers);}++ /** The body of a 2xx reply; a non-2xx reply is thrown as the AwsException its AWS error body names. */+ public JsonNode successBody() {+ JsonNode replyBody = body == null ? MissingNode.getInstance() : body;+ if (status >= 300) {+ String type = text(replyBody, "__type");+ String code = type == null ? "InternalServerError" : type.substring(type.lastIndexOf('#') + 1);+ String message = text(replyBody, "message");+ throw new AwsException(code, message != null ? message : text(replyBody, "Message"), status);+ }+ return replyBody;+ }++ private static String text(JsonNode node, String field) {+ JsonNode value = node.get(field);+ return value == null || value.isNull() ? null : value.asText();+ }}Reply execute(Call call) throws Exception;diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.javaindex 71cd5998d5..6d374daaff 100644--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java@@ -30,8 +30,7 @@ import java.util.Optional;*/@ApplicationScoped@Typed(NativeDynamoDbBackend.class)-public class NativeDynamoDbBackend implements DynamoDbOperations, DynamoDbItemAccess, DynamoDbTableAccess,- DynamoDbBackendLifecycle {+public class NativeDynamoDbBackend implements DynamoDbBackend {private final NativeDynamoDbJsonHandler jsonHandler;private final NativeDynamoDbStreamsJsonHandler streamsHandler;diff --git a/src/main/resources/application.yml b/src/main/resources/application.ymlindex 7f9a55abbd..60a78fb133 100644--- a/src/main/resources/application.yml+++ b/src/main/resources/application.yml@@ -322,6 +322,10 @@ floci:enabled: truevector-index-allocation-seconds: 4 # FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_ALLOCATION_SECONDS. Resource allocation phase of a vector index added by UpdateTable.vector-index-backfill-seconds: 10 # FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_BACKFILL_SECONDS. Backfilling phase that follows it.+ backend: native # FLOCI_SERVICES_DYNAMODB_BACKEND: native or local+ # local-endpoint: http://dynamodb-local:8000 # FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT+ local-connect-timeout-seconds: 2 # FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS+ local-request-timeout-seconds: 10 # FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDSsns:enabled: truelambda:diff --git a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.javaindex 53ecfa5af5..08d35da364 100644--- a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java@@ -92,7 +92,8 @@ class EmulatorInfoControllerIntegrationTest {.statusCode(200).contentType("application/json").body("edition", equalTo("community"))- .body("version", notNullValue());+ .body("version", notNullValue())+ .body("dynamodb_backend", equalTo("native"));}@ParameterizedTestdiff --git a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.javaindex 5fdcba6e69..5e583ed2c3 100644--- a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java+++ b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java@@ -6,6 +6,7 @@ import io.github.hectorvent.floci.core.common.ContainerTeardown;import io.github.hectorvent.floci.core.common.Resettable;import io.github.hectorvent.floci.core.common.ServiceRegistry;import io.github.hectorvent.floci.core.storage.StorageFactory;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbBackendSelector;import jakarta.enterprise.inject.Instance;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.DisplayName;@@ -16,6 +17,7 @@ import org.mockito.Mock;import org.mockito.junit.jupiter.MockitoExtension;import java.util.List;+import java.util.Map;import static org.junit.jupiter.api.Assertions.assertEquals;import static org.junit.jupiter.api.Assertions.assertSame;@@ -40,6 +42,7 @@ class EmulatorInfoControllerTest {@Mock private Instance<ContainerTeardown> containerTeardowns;@Mock private FlociCertificateAuthority certificateAuthority;@Mock private EmulatorConfig config;+ @Mock private DynamoDbBackendSelector dynamoDbBackendSelector;@Mock private ContainerTeardown sageMakerTeardown;@Mock private ContainerTeardown batchTeardown;@Mock private Resettable resettable;@@ -49,7 +52,17 @@ class EmulatorInfoControllerTest {@BeforeEachvoid setUp() {controller = new EmulatorInfoController(serviceRegistry, initLifecycleState, storageFactory,- resettables, containerTeardowns, certificateAuthority, config);+ resettables, containerTeardowns, certificateAuthority, config, dynamoDbBackendSelector);+ }++ @Test+ @DisplayName("Should report the selected DynamoDB backend in info")+ void info_reportsTheSelectedDynamoDbBackend() {+ when(dynamoDbBackendSelector.selected()).thenReturn("local");++ Map<?, ?> info = (Map<?, ?>) controller.info().getEntity();++ assertEquals("local", info.get("dynamodb_backend"));}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.javaindex f7698f91c6..8ac2afd71c 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java@@ -115,6 +115,6 @@ class CloudWatchLogsMetricFilterResetTest {when(resettables.iterator()).thenAnswer(call -> List.of(resettable).iterator());Instance<ContainerTeardown> teardowns = mock(Instance.class);when(teardowns.iterator()).thenAnswer(call -> List.<ContainerTeardown>of().iterator());- return new EmulatorInfoController(null, null, f.factory, resettables, teardowns, null, null);+ return new EmulatorInfoController(null, null, f.factory, resettables, teardowns, null, null, null);}}diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.javaindex c4dfdfbe03..bf02eefe6f 100644--- a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java@@ -3,6 +3,7 @@ package io.github.hectorvent.floci.services.dynamodb.backend;import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.config.EmulatorConfig;import io.github.hectorvent.floci.core.common.AwsException;import io.github.hectorvent.floci.core.resource.ResourceProvider;import io.github.hectorvent.floci.services.dynamodb.DynamoDbFacade;@@ -18,8 +19,10 @@ import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReaderimport io.github.hectorvent.floci.services.dynamodb.model.AttributeDefinition;import io.github.hectorvent.floci.services.dynamodb.model.KeySchemaElement;import io.github.hectorvent.floci.services.dynamodb.model.TableDefinition;+import io.quarkus.arc.Arc;import io.quarkus.arc.ClientProxy;import io.quarkus.test.junit.QuarkusTest;+import jakarta.enterprise.context.ApplicationScoped;import jakarta.enterprise.inject.Instance;import jakarta.inject.Inject;import org.junit.jupiter.api.Test;@@ -28,11 +31,16 @@ import java.util.List;import java.util.UUID;import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;import static org.junit.jupiter.api.Assertions.assertInstanceOf;import static org.junit.jupiter.api.Assertions.assertNotEquals;import static org.junit.jupiter.api.Assertions.assertSame;import static org.junit.jupiter.api.Assertions.assertThrows;import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.verifyNoInteractions;+import static org.mockito.Mockito.when;/** The one DynamoDB backend CDI selects, and the one Resource Explorer provider in front of it. */@QuarkusTest@@ -64,16 +72,81 @@ class DynamoDbBackendSelectionTest {@InjectObjectMapper mapper;+ @Inject+ DynamoDbBackendSelector selector;+@Testvoid everyCapabilityIsTheOneNativeBackend() {Object backend = ClientProxy.unwrap(operations);+ assertEquals("native", selector.selected());assertInstanceOf(NativeDynamoDbBackend.class, backend);assertSame(backend, ClientProxy.unwrap(items));assertSame(backend, ClientProxy.unwrap(tables));assertSame(backend, ClientProxy.unwrap(lifecycle));}+ @Test+ void theUnselectedLocalBackendIsNeverConstructed() {+ assertFalse(Arc.container().getActiveContext(ApplicationScoped.class).getState().getContextualInstances()+ .keySet().stream().anyMatch(bean -> bean.getBeanClass() == DynamoDbLocalBackend.class));+ }++ @Test+ void nativeServesEverySeamWithProcessCheckpoints() {+ NativeDynamoDbBackend nativeBackend = mock(NativeDynamoDbBackend.class);+ DynamoDbLocalBackend localBackend = mock(DynamoDbLocalBackend.class);+ DynamoDbBackendSelector nativeSelector = new DynamoDbBackendSelector(config("native"), nativeBackend, localBackend);++ assertEquals("native", nativeSelector.selected());+ assertServesEverySeam(nativeBackend, nativeSelector);+ assertEquals(CheckpointLifetime.PROCESS, nativeSelector.streamReader().checkpointLifetime());+ verifyNoInteractions(localBackend);+ }++ @Test+ void localServesEverySeamWithStreamCheckpoints() {+ NativeDynamoDbBackend nativeBackend = mock(NativeDynamoDbBackend.class);+ DynamoDbLocalBackend localBackend = mock(DynamoDbLocalBackend.class);+ DynamoDbBackendSelector localSelector = new DynamoDbBackendSelector(config("local"), nativeBackend, localBackend);++ assertEquals("local", localSelector.selected());+ assertServesEverySeam(localBackend, localSelector);+ assertEquals(CheckpointLifetime.STREAM, localSelector.streamReader().checkpointLifetime());+ verifyNoInteractions(nativeBackend);+ }++ @Test+ void theBackendNameIsTrimmedAndCaseInsensitive() {+ DynamoDbLocalBackend localBackend = mock(DynamoDbLocalBackend.class);+ DynamoDbBackendSelector localSelector =+ new DynamoDbBackendSelector(config(" LOCAL "), mock(NativeDynamoDbBackend.class), localBackend);++ assertEquals("local", localSelector.selected());+ assertSame(localBackend, localSelector.operations());+ }++ @Test+ void anUnknownBackendFailsSelection() {+ IllegalStateException failure = assertThrows(IllegalStateException.class, () -> new DynamoDbBackendSelector(+ config("sqlite"), mock(NativeDynamoDbBackend.class), mock(DynamoDbLocalBackend.class)));++ assertEquals("floci.services.dynamodb.backend must be 'native' or 'local', got 'sqlite'", failure.getMessage());+ }++ private static EmulatorConfig config(String backend) {+ EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);+ when(config.services().dynamodb().backend()).thenReturn(backend);+ return config;+ }++ private static void assertServesEverySeam(DynamoDbBackend expected, DynamoDbBackendSelector selector) {+ assertSame(expected, selector.operations());+ assertSame(expected, selector.itemAccess());+ assertSame(expected, selector.tableAccess());+ assertSame(expected, selector.lifecycle());+ }+@Testvoid startupSchedulesTheTtlSweep() {assertTrue(Thread.getAllStackTraces().keySet().stream()diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendIntegrationTest.javanew file mode 100644index 0000000000..c85c2ac86a--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendIntegrationTest.java@@ -0,0 +1,302 @@+package io.github.hectorvent.floci.services.dynamodb.backend;++import com.fasterxml.jackson.databind.JsonNode;+import io.github.hectorvent.floci.services.dynamodb.DynamoDbFacade;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.CheckpointLifetime;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Cursor;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Position;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.RecordsPage;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Shard;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Stream;+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import io.quarkus.test.junit.QuarkusTestProfile;+import io.quarkus.test.junit.TestProfile;+import io.restassured.response.ValidatableResponse;+import jakarta.inject.Inject;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;+import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;++import java.util.ArrayList;+import java.util.List;+import java.util.Map;+import java.util.UUID;+import java.util.concurrent.ThreadLocalRandom;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.contains;+import static org.hamcrest.Matchers.containsInAnyOrder;+import static org.hamcrest.Matchers.endsWith;+import static org.hamcrest.Matchers.equalTo;+import static org.hamcrest.Matchers.hasSize;+import static org.hamcrest.Matchers.nullValue;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;++/** The DynamoDB API served by an external DynamoDB Local, reached at FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT. */+@QuarkusTest+@TestProfile(DynamoDbLocalBackendIntegrationTest.LocalProfile.class)+@EnabledIfEnvironmentVariable(named = DynamoDbLocalBackendIntegrationTest.ENDPOINT_VARIABLE, matches = ".+")+class DynamoDbLocalBackendIntegrationTest {++ static final String ENDPOINT_VARIABLE = "FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT";++ public static final class LocalProfile implements QuarkusTestProfile {+ @Override+ public Map<String, String> getConfigOverrides() {+ String endpoint = System.getenv(ENDPOINT_VARIABLE);+ return Map.of("floci.services.dynamodb.backend", "local",+ "floci.services.dynamodb.local-endpoint", endpoint == null ? "" : endpoint);+ }+ }++ private record Caller(String account, String region) {+ String authorization() {+ return "AWS4-HMAC-SHA256 Credential=" + account + "/20260215/" + region+ + "/dynamodb/aws4_request, SignedHeaders=host, Signature=abc";+ }+ }++ private record CreatedTable(Caller caller, String name) {}++ private static final String CONTENT_TYPE = "application/x-amz-json-1.0";+ private static final String ACCOUNT_A = randomAccount();+ private static final String ACCOUNT_B = otherAccountThan(ACCOUNT_A);+ private static final Caller A_EAST = new Caller(ACCOUNT_A, "us-east-1");+ private static final Caller A_WEST = new Caller(ACCOUNT_A, "eu-west-1");+ private static final Caller B_EAST = new Caller(ACCOUNT_B, "us-east-1");+ private static final String STREAMED = ", \"StreamSpecification\": {\"StreamEnabled\": true, "+ + "\"StreamViewType\": \"NEW_AND_OLD_IMAGES\"}";++ @Inject+ DynamoDbStreamReader streamReader;++ @Inject+ DynamoDbFacade dynamoDbFacade;++ private final List<CreatedTable> created = new ArrayList<>();++ @BeforeAll+ static void configureRestAssured() {+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @AfterEach+ void deleteCreatedTables() {+ for (CreatedTable table : created) {+ dynamoDb(table.caller(), "DeleteTable", "{\"TableName\": \"" + table.name() + "\"}").statusCode(200);+ }+ created.clear();+ }++ @Test+ void theSameTableNameIsIsolatedAcrossAccountsAndRegions() {+ String table = tableName("isolation");+ for (Caller caller : List.of(A_EAST, B_EAST, A_WEST)) {+ createTable(caller, table, "").statusCode(200);+ }+ putItem(A_EAST, table, "only-in-a-east");++ scanCount(A_EAST, table).body("Count", equalTo(1));+ scanCount(B_EAST, table).body("Count", equalTo(0));+ scanCount(A_WEST, table).body("Count", equalTo(0));+ dynamoDb(B_EAST, "GetItem", "{\"TableName\": \"" + table + "\", \"Key\": {\"pk\": {\"S\": \"only-in-a-east\"}}}")+ .statusCode(200)+ .body("Item", nullValue());+ }++ @Test+ void tableArnsArePublicAndAcceptedAsTableNames() {+ String table = tableName("arn");+ String arn = tableArn(A_WEST, table);++ createTable(A_WEST, table, "").statusCode(200).body("TableDescription.TableArn", equalTo(arn));++ dynamoDb(A_WEST, "DescribeTable", "{\"TableName\": \"" + arn + "\"}")+ .statusCode(200)+ .body("Table.TableName", equalTo(table))+ .body("Table.TableArn", equalTo(arn));+ }++ @Test+ void tagsRoundTripAlongsideCreateTableTags() {+ String table = tableName("tags");+ String arn = tableArn(B_EAST, table);+ createTable(B_EAST, table, ", \"Tags\": [{\"Key\": \"env\", \"Value\": \"test\"}]").statusCode(200);++ dynamoDb(B_EAST, "TagResource",+ "{\"ResourceArn\": \"" + arn + "\", \"Tags\": [{\"Key\": \"team\", \"Value\": \"data\"}]}")+ .statusCode(200);+ dynamoDb(B_EAST, "ListTagsOfResource", "{\"ResourceArn\": \"" + arn + "\"}")+ .statusCode(200)+ .body("Tags.Key", containsInAnyOrder("env", "team"));++ dynamoDb(B_EAST, "UntagResource", "{\"ResourceArn\": \"" + arn + "\", \"TagKeys\": [\"env\"]}")+ .statusCode(200);+ dynamoDb(B_EAST, "ListTagsOfResource", "{\"ResourceArn\": \"" + arn + "\"}")+ .statusCode(200)+ .body("Tags.Key", contains("team"))+ .body("Tags.Value", contains("data"));+ }++ @Test+ void listingTagsOfAMissingTableIsAccessDenied() {+ dynamoDb(B_EAST, "ListTagsOfResource", "{\"ResourceArn\": \"" + tableArn(B_EAST, tableName("absent")) + "\"}")+ .statusCode(400)+ .body("__type", endsWith("AccessDeniedException"));+ }++ @Test+ void streamRecordsCarryPublicArnsAndTheCallerRegion() {+ String table = tableName("stream");+ String streamArn = streamedTableWithOneItem(A_WEST, table);+ assertTrue(streamArn.startsWith(tableArn(A_WEST, table) + "/stream/"), streamArn);++ streams(A_WEST, "ListStreams", "{\"TableName\": \"" + table + "\"}")+ .statusCode(200)+ .body("Streams.StreamArn", contains(streamArn));+ String shardId = streams(A_WEST, "DescribeStream", "{\"StreamArn\": \"" + streamArn + "\"}")+ .statusCode(200)+ .body("StreamDescription.StreamArn", equalTo(streamArn))+ .extract().path("StreamDescription.Shards[0].ShardId");+ String iterator = streams(A_WEST, "GetShardIterator", "{\"StreamArn\": \"" + streamArn+ + "\", \"ShardId\": \"" + shardId + "\", \"ShardIteratorType\": \"TRIM_HORIZON\"}")+ .statusCode(200)+ .extract().path("ShardIterator");++ streams(A_WEST, "GetRecords", "{\"ShardIterator\": \"" + iterator + "\"}")+ .statusCode(200)+ .body("Records", hasSize(1))+ .body("Records[0].eventName", equalTo("INSERT"))+ .body("Records[0].awsRegion", equalTo("eu-west-1"))+ .body("Records[0].dynamodb.Keys.pk.S", equalTo("streamed"));+ }++ @Test+ void aStreamOfAnotherAccountIsNotFound() {+ String table = tableName("foreign-stream");+ String streamArn = streamedTableWithOneItem(A_EAST, table);++ streams(B_EAST, "DescribeStream", "{\"StreamArn\": \"" + streamArn + "\"}")+ .statusCode(400)+ .body("__type", endsWith("ResourceNotFoundException"));+ }++ @Test+ void theStreamReaderReadsTheSameRecordsWithStreamCheckpoints() {+ String table = tableName("reader");+ String streamArn = streamedTableWithOneItem(A_WEST, table);++ Stream stream = Stream.of(streamArn);+ List<Shard> shards = streamReader.shards(stream);+ Cursor cursor = streamReader.getShardIterator(stream, shards.get(0).shardId(), Position.TRIM_HORIZON, null);+ RecordsPage page = streamReader.getRecords(cursor, 10);++ assertEquals(1, page.records().size());+ JsonNode record = page.records().get(0).awsRecord();+ assertEquals("INSERT", record.path("eventName").asText());+ assertEquals("eu-west-1", record.path("awsRegion").asText());+ assertEquals("streamed", record.path("dynamodb").path("Keys").path("pk").path("S").asText());+ assertEquals(CheckpointLifetime.STREAM, streamReader.checkpointLifetime());+ }++ @Test+ void theDirectDeleteRemovesAProtectedTable() {+ String table = tableName("protected");+ createTable(A_WEST, table, ", \"DeletionProtectionEnabled\": true").statusCode(200);+ dynamoDb(A_WEST, "DeleteTable", "{\"TableName\": \"" + table + "\"}").statusCode(400);++ dynamoDbFacade.tables().deleteTable(new Scope(A_WEST.account(), A_WEST.region()), table);+ created.clear();++ dynamoDb(A_WEST, "DescribeTable", "{\"TableName\": \"" + table + "\"}")+ .statusCode(400)+ .body("__type", endsWith("ResourceNotFoundException"));+ }++ @Test+ void resetIsRefused() {+ given().when().post("/_floci/state/reset").then().statusCode(409);+ }++ @Test+ void infoReportsTheLocalBackend() {+ given()+ .when().get("/_floci/info")+ .then()+ .statusCode(200)+ .body("dynamodb_backend", equalTo("local"));+ }++ private String streamedTableWithOneItem(Caller caller, String table) {+ String streamArn = createTable(caller, table, STREAMED)+ .statusCode(200)+ .extract().path("TableDescription.LatestStreamArn");+ putItem(caller, table, "streamed");+ return streamArn;+ }++ private ValidatableResponse createTable(Caller caller, String table, String extraFields) {+ ValidatableResponse response = dynamoDb(caller, "CreateTable", """+ {+ "TableName": "%s",+ "KeySchema": [{"AttributeName": "pk", "KeyType": "HASH"}],+ "AttributeDefinitions": [{"AttributeName": "pk", "AttributeType": "S"}],+ "BillingMode": "PAY_PER_REQUEST"%s+ }+ """.formatted(table, extraFields));+ created.add(new CreatedTable(caller, table));+ return response;+ }++ private static void putItem(Caller caller, String table, String pk) {+ dynamoDb(caller, "PutItem", "{\"TableName\": \"" + table + "\", \"Item\": {\"pk\": {\"S\": \"" + pk + "\"}}}")+ .statusCode(200);+ }++ private static ValidatableResponse scanCount(Caller caller, String table) {+ return dynamoDb(caller, "Scan", "{\"TableName\": \"" + table + "\"}").statusCode(200);+ }++ private static ValidatableResponse dynamoDb(Caller caller, String action, String body) {+ return send(caller, "DynamoDB_20120810." + action, body);+ }++ private static ValidatableResponse streams(Caller caller, String action, String body) {+ return send(caller, "DynamoDBStreams_20120810." + action, body);+ }++ private static ValidatableResponse send(Caller caller, String target, String body) {+ return given()+ .header("X-Amz-Target", target)+ .header("Authorization", caller.authorization())+ .contentType(CONTENT_TYPE)+ .body(body)+ .when()+ .post("/")+ .then();+ }++ private static String tableArn(Caller caller, String table) {+ return "arn:aws:dynamodb:" + caller.region() + ":" + caller.account() + ":table/" + table;+ }++ private static String tableName(String purpose) {+ return "local-it-" + purpose + "-" + UUID.randomUUID();+ }++ private static String randomAccount() {+ return Long.toString(ThreadLocalRandom.current().nextLong(100_000_000_000L, 1_000_000_000_000L));+ }++ private static String otherAccountThan(String account) {+ String other = randomAccount();+ while (other.equals(account)) {+ other = randomAccount();+ }+ return other;+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendTest.javanew file mode 100644index 0000000000..9c2f2dc57d--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendTest.java@@ -0,0 +1,783 @@+package io.github.hectorvent.floci.services.dynamodb.backend;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.core.resource.ExplorerResource;+import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbItemAccess.ScanPage;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Call;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;+import io.github.hectorvent.floci.services.dynamodb.model.AttributeDefinition;+import io.github.hectorvent.floci.services.dynamodb.model.GlobalSecondaryIndex;+import io.github.hectorvent.floci.services.dynamodb.model.KeySchemaElement;+import io.github.hectorvent.floci.services.dynamodb.model.LocalSecondaryIndex;+import io.github.hectorvent.floci.services.dynamodb.model.TableDefinition;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;+import org.mockito.ArgumentCaptor;+import org.mockito.InOrder;++import java.io.IOException;+import java.net.ConnectException;+import java.net.URI;+import java.time.Duration;+import java.time.Instant;+import java.util.ArrayList;+import java.util.List;+import java.util.Map;+import java.util.Optional;+import java.util.Set;+import java.util.stream.Collectors;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertNull;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.ArgumentMatchers.any;+import static org.mockito.ArgumentMatchers.eq;+import static org.mockito.Mockito.atLeastOnce;+import static org.mockito.Mockito.inOrder;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.never;+import static org.mockito.Mockito.times;+import static org.mockito.Mockito.verify;+import static org.mockito.Mockito.when;++/** The DynamoDB Local engine: identity translation, Floci-held tags, typed operations and lifecycle. */+class DynamoDbLocalBackendTest {++ private static final String ACCOUNT = "123456789012";+ private static final String REGION = "eu-west-1";+ private static final Scope SCOPE = new Scope(ACCOUNT, REGION);+ private static final String LOCAL = "arn:aws:dynamodb:ddblocal:000000000000:";+ private static final String PUBLIC = "arn:aws:dynamodb:eu-west-1:123456789012:";+ private static final String ORDERS_ARN = PUBLIC + "table/orders";+ private static final String STREAM_RESOURCE = "table/orders/stream/2026-09-27T00:03:07.048";+ private static final String REPLICAS_UNSUPPORTED = "Replicas are not supported by the DynamoDB Local backend";++ private final ObjectMapper mapper = new ObjectMapper().findAndRegisterModules();+ private final DynamoDbLocalClient client = mock(DynamoDbLocalClient.class);+ private final AccountAwareStorageBackend<Map<String, String>> tags =+ AccountAwareStorageBackend.inMemory("000000000000");+ private DynamoDbLocalBackend backend;++ @BeforeEach+ void setUp() {+ when(client.endpoint()).thenReturn(URI.create("http://local:8000"));+ backend = new DynamoDbLocalBackend(client, tags, mapper, Duration.ofSeconds(5));+ }++ private JsonNode json(String text) throws IOException {+ return mapper.readTree(text);+ }++ private JsonNode normalized(JsonNode node) throws IOException {+ return mapper.readTree(mapper.writeValueAsBytes(node));+ }++ private void answer(Api api, String action, int status, String body) throws Exception {+ when(client.send(any(), eq(api), eq(action), any()))+ .thenAnswer(invocation -> new Reply(status, json(body), Map.of()));+ }++ private void answer(String action, int status, String body) throws Exception {+ answer(Api.DYNAMODB, action, status, body);+ }++ private List<JsonNode> sentBodies(Api api, String action) throws Exception {+ ArgumentCaptor<JsonNode> body = ArgumentCaptor.forClass(JsonNode.class);+ verify(client, atLeastOnce()).send(any(), eq(api), eq(action), body.capture());+ List<JsonNode> bodies = new ArrayList<>();+ for (JsonNode value : body.getAllValues()) {+ bodies.add(normalized(value));+ }+ return bodies;+ }++ private JsonNode sent(Api api, String action) throws Exception {+ List<JsonNode> bodies = sentBodies(api, action);+ return bodies.get(bodies.size() - 1);+ }++ private JsonNode sent(String action) throws Exception {+ return sent(Api.DYNAMODB, action);+ }++ private void neverSent(String action) throws Exception {+ verify(client, never()).send(any(), any(), eq(action), any());+ }++ private Reply execute(Api api, String action, String body) throws Exception {+ return backend.execute(new Call(SCOPE, api, action, json(body)));+ }++ private Reply execute(String action, String body) throws Exception {+ return execute(Api.DYNAMODB, action, body);+ }++ private void answerOrdersTable() throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL+ + "table/orders\",\"TableStatus\":\"ACTIVE\"}}");+ }++ @Test+ void tableArnTableNameBecomesShortName() throws Exception {+ answer("GetItem", 200, "{}");++ execute("GetItem", "{\"TableName\":\"" + ORDERS_ARN + "\",\"Key\":{\"pk\":{\"S\":\"a\"}}}");++ assertEquals(json("{\"TableName\":\"orders\",\"Key\":{\"pk\":{\"S\":\"a\"}}}"), sent("GetItem"));+ }++ @Test+ void requestItemsArnKeysBecomeShortNames() throws Exception {+ answer("BatchWriteItem", 200, "{\"UnprocessedItems\":{}}");++ execute("BatchWriteItem", "{\"RequestItems\":{\"" + ORDERS_ARN + "\":[{\"PutRequest\":{\"Item\":"+ + "{\"pk\":{\"S\":\"a\"}}}}],\"users\":[{\"DeleteRequest\":{\"Key\":{\"pk\":{\"S\":\"b\"}}}}]}}");++ JsonNode requestItems = sent("BatchWriteItem").path("RequestItems");+ List<String> names = new ArrayList<>();+ requestItems.fieldNames().forEachRemaining(names::add);+ assertEquals(List.of("orders", "users"), names);+ assertEquals(json("[{\"PutRequest\":{\"Item\":{\"pk\":{\"S\":\"a\"}}}}]"), requestItems.path("orders"));+ }++ @Test+ void transactItemsNestedTableNamesTranslated() throws Exception {+ answer("TransactWriteItems", 200, "{}");++ execute("TransactWriteItems", "{\"TransactItems\":["+ + "{\"Put\":{\"TableName\":\"" + ORDERS_ARN + "\",\"Item\":{\"pk\":{\"S\":\"a\"}}}},"+ + "{\"ConditionCheck\":{\"TableName\":\"" + PUBLIC + "table/users\",\"Key\":{\"pk\":{\"S\":\"b\"}},"+ + "\"ConditionExpression\":\"attribute_exists(pk)\"}},"+ + "{\"Delete\":{\"TableName\":\"audit\",\"Key\":{\"pk\":{\"S\":\"c\"}}}}]}");++ JsonNode items = sent("TransactWriteItems").path("TransactItems");+ assertEquals("orders", items.path(0).path("Put").path("TableName").asText());+ assertEquals("users", items.path(1).path("ConditionCheck").path("TableName").asText());+ assertEquals("audit", items.path(2).path("Delete").path("TableName").asText());+ }++ @Test+ void foreignAccountOrPartitionArnNotFound() throws Exception {+ for (String foreign : List.of("arn:aws:dynamodb:eu-west-1:999999999999:table/orders",+ "arn:aws-cn:dynamodb:eu-west-1:123456789012:table/orders")) {+ AwsException error = assertThrows(AwsException.class,+ () -> execute("GetItem", "{\"TableName\":\"" + foreign + "\",\"Key\":{}}"));++ assertEquals("ResourceNotFoundException", error.getErrorCode());+ assertEquals("Requested resource not found: Table: " + foreign + " not found", error.getMessage());+ }+ neverSent("GetItem");+ }++ @Test+ void requestItemsNamingOneTableTwiceRejected() throws Exception {+ AwsException error = assertThrows(AwsException.class, () -> execute("BatchWriteItem",+ "{\"RequestItems\":{\"" + ORDERS_ARN + "\":[],\"orders\":[]}}"));++ assertEquals("ValidationException", error.getErrorCode());+ neverSent("BatchWriteItem");+ }++ @Test+ void foreignRegionArnRejected() throws Exception {+ AwsException error = assertThrows(AwsException.class, () -> execute("GetItem",+ "{\"TableName\":\"arn:aws:dynamodb:us-east-1:123456789012:table/orders\",\"Key\":{}}"));++ assertEquals("ValidationException", error.getErrorCode());+ assertEquals("Region 'us-east-1' in ARN does not match request region 'eu-west-1'", error.getMessage());+ neverSent("GetItem");+ }++ @Test+ void createTableArnNameLeftForLocal() throws Exception {+ answer("CreateTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ValidationException\","+ + "\"message\":\"Invalid table name\"}");++ Reply reply = execute("CreateTable", "{\"TableName\":\"" + ORDERS_ARN + "\"}");++ assertEquals(ORDERS_ARN, sent("CreateTable").path("TableName").asText());+ assertEquals(400, reply.status());+ }++ @Test+ void streamArnTranslatedToLocalPrefix() throws Exception {+ answer(Api.DYNAMODB_STREAMS, "DescribeStream", 200, "{\"StreamDescription\":{}}");++ execute(Api.DYNAMODB_STREAMS, "DescribeStream", "{\"StreamArn\":\"" + PUBLIC + STREAM_RESOURCE + "\"}");++ assertEquals(LOCAL + STREAM_RESOURCE, sent(Api.DYNAMODB_STREAMS, "DescribeStream").path("StreamArn").asText());+ }++ @Test+ void listStreamsPagesWithPublicStreamArns() throws Exception {+ answer(Api.DYNAMODB_STREAMS, "ListStreams", 200,+ "{\"LastEvaluatedStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}");++ Reply reply = execute(Api.DYNAMODB_STREAMS, "ListStreams",+ "{\"ExclusiveStartStreamArn\":\"" + PUBLIC + STREAM_RESOURCE + "\"}");++ assertEquals(LOCAL + STREAM_RESOURCE,+ sent(Api.DYNAMODB_STREAMS, "ListStreams").path("ExclusiveStartStreamArn").asText());+ assertEquals(PUBLIC + STREAM_RESOURCE, reply.body().path("LastEvaluatedStreamArn").asText());+ }++ @Test+ void foreignStreamArnNotFound() throws Exception {+ for (String foreign : List.of("arn:aws:dynamodb:eu-west-1:999999999999:" + STREAM_RESOURCE,+ "arn:aws:dynamodb:us-east-1:123456789012:" + STREAM_RESOURCE,+ "arn:aws-cn:dynamodb:eu-west-1:123456789012:" + STREAM_RESOURCE)) {+ AwsException error = assertThrows(AwsException.class,+ () -> execute(Api.DYNAMODB_STREAMS, "DescribeStream", "{\"StreamArn\":\"" + foreign + "\"}"));++ assertEquals("ResourceNotFoundException", error.getErrorCode());+ assertEquals("Requested resource not found: Stream: " + foreign + " not found", error.getMessage());+ assertEquals(400, error.getHttpStatus());+ }+ neverSent("DescribeStream");+ }++ @Test+ void publicArnsInReply() throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL + "table/orders\","+ + "\"GlobalSecondaryIndexes\":[{\"IndexName\":\"g\","+ + "\"IndexArn\":\"" + LOCAL + "table/orders/index/g\"}],"+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");++ JsonNode table = execute("DescribeTable", "{\"TableName\":\"orders\"}").body().path("Table");++ assertEquals(ORDERS_ARN, table.path("TableArn").asText());+ assertEquals(ORDERS_ARN + "/index/g", table.path("GlobalSecondaryIndexes").path(0).path("IndexArn").asText());+ assertEquals(PUBLIC + STREAM_RESOURCE, table.path("LatestStreamArn").asText());+ }++ @Test+ void errorMessageArnRewritten() throws Exception {+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","+ + "\"message\":\"Requested resource not found: Table: " + LOCAL + "table/orders not found\"}");++ Reply reply = execute("DescribeTable", "{\"TableName\":\"orders\"}");++ assertEquals("Requested resource not found: Table: " + ORDERS_ARN + " not found",+ reply.body().path("message").asText());+ }++ @Test+ void recordAwsRegionRewritten() throws Exception {+ answer(Api.DYNAMODB_STREAMS, "GetRecords", 200,+ "{\"Records\":[{\"eventName\":\"INSERT\",\"awsRegion\":\"ddblocal\",\"dynamodb\":{}}]}");++ Reply reply = execute(Api.DYNAMODB_STREAMS, "GetRecords", "{\"ShardIterator\":\"it\"}");++ assertEquals(REGION, reply.body().path("Records").path(0).path("awsRegion").asText());+ }++ @Test+ void arnLookingItemAttributeUntouched() throws Exception {+ String item = "{\"TableArn\":{\"S\":\"" + LOCAL + "table/x\"},"+ + "\"StreamArn\":{\"S\":\"arn:aws:dynamodb:us-east-1:999999999999:" + STREAM_RESOURCE + "\"},"+ + "\"TableName\":{\"S\":\"" + PUBLIC + "table/other\"},"+ + "\"awsRegion\":{\"S\":\"ddblocal\"}}";+ answer("PutItem", 200, "{}");+ answer("GetItem", 200, "{\"Item\":" + item + "}");++ execute("PutItem", "{\"TableName\":\"orders\",\"Item\":" + item + "}");+ Reply reply = execute("GetItem", "{\"TableName\":\"orders\",\"Key\":{\"pk\":{\"S\":\"a\"}}}");++ assertEquals(json(item), sent("PutItem").path("Item"));+ assertEquals(json(item), reply.body().path("Item"));+ }++ @Test+ void callerBodyNotMutated() throws Exception {+ answer("GetItem", 200, "{}");+ JsonNode body = json("{\"TableName\":\"" + ORDERS_ARN + "\",\"Key\":{\"pk\":{\"S\":\"a\"}}}");+ JsonNode original = body.deepCopy();++ backend.execute(new Call(SCOPE, Api.DYNAMODB, "GetItem", body));++ assertEquals(original, body);+ }++ @Test+ void sendsUnderCallerScope() throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{}}");+ Scope caller = new Scope("210987654321", "ap-south-1");++ backend.execute(new Call(caller, Api.DYNAMODB, "DescribeTable", json("{\"TableName\":\"orders\"}")));++ ArgumentCaptor<Scope> scope = ArgumentCaptor.forClass(Scope.class);+ verify(client).send(scope.capture(), eq(Api.DYNAMODB), eq("DescribeTable"), any());+ assertEquals(caller, scope.getValue());+ }++ @Test+ void ioFailureIsInternalServerError() throws Exception {+ when(client.send(any(), any(), any(), any())).thenThrow(new IOException("boom"));++ AwsException error = assertThrows(AwsException.class,+ () -> execute("DescribeTable", "{\"TableName\":\"orders\"}"));++ assertEquals("InternalServerError", error.getErrorCode());+ assertEquals(500, error.getHttpStatus());+ assertEquals("DynamoDB Local at http://local:8000 did not answer: java.io.IOException: boom",+ error.getMessage());+ }++ @Test+ void untaggedTableListsNoTags() throws Exception {+ answerOrdersTable();++ assertEquals(Map.of(), backend.listTagsOfResource(SCOPE, ORDERS_ARN));+ }++ @Test+ void tagAbsentTableFails() throws Exception {+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","+ + "\"message\":\"Requested resource not found\"}");++ AwsException error = assertThrows(AwsException.class,+ () -> backend.tagResource(SCOPE, ORDERS_ARN, Map.of("env", "dev")));++ assertEquals("ResourceNotFoundException", error.getErrorCode());+ assertTrue(tags.keysForAccount(ACCOUNT).isEmpty());+ }++ @Test+ void wireListTagsOfAMissingTableIsAccessDenied() throws Exception {+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","+ + "\"message\":\"Requested resource not found\"}");++ AwsException error = assertThrows(AwsException.class,+ () -> execute("ListTagsOfResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\"}"));++ assertEquals("AccessDeniedException", error.getErrorCode());+ assertEquals("User is not authorized to perform: dynamodb:ListTagsOfResource on resource: " + ORDERS_ARN,+ error.getMessage());+ assertEquals(400, error.getHttpStatus());+ }++ @Test+ void tagArnOfAnUnknownPartitionIsValidation() throws Exception {+ String arn = "arn:notaws:dynamodb:eu-west-1:123456789012:table/orders";++ AwsException error = assertThrows(AwsException.class,+ () -> execute("TagResource", "{\"ResourceArn\":\"" + arn + "\",\"Tags\":[]}"));++ assertEquals("ValidationException", error.getErrorCode());+ neverSent("DescribeTable");+ }++ @Test+ void tagArnOfAnotherRegionOrPartitionIsNotFound() throws Exception {+ for (String foreign : List.of("arn:aws:dynamodb:us-east-1:123456789012:table/orders",+ "arn:aws-cn:dynamodb:cn-north-1:123456789012:table/orders")) {+ AwsException error = assertThrows(AwsException.class,+ () -> execute("UntagResource", "{\"ResourceArn\":\"" + foreign + "\",\"TagKeys\":[]}"));++ assertEquals("ResourceNotFoundException", error.getErrorCode());+ }+ neverSent("DescribeTable");+ }++ @Test+ void createTableRecordsRequestTags() throws Exception {+ answer("CreateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL+ + "table/orders\"}}");+ answerOrdersTable();++ execute("CreateTable", "{\"TableName\":\"orders\",\"Tags\":[{\"Key\":\"env\",\"Value\":\"dev\"}]}");++ assertEquals(Map.of("env", "dev"), backend.listTagsOfResource(SCOPE, ORDERS_ARN));+ }++ @Test+ void deleteTableDropsTags() throws Exception {+ tags.putForAccount(ACCOUNT, REGION + "/orders", Map.of("env", "dev"));+ answer("DeleteTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\",\"TableStatus\":\"DELETING\"}}");++ execute("DeleteTable", "{\"TableName\":\"" + ORDERS_ARN + "\"}");++ assertTrue(tags.getForAccount(ACCOUNT, REGION + "/orders").isEmpty());+ }++ @Test+ void directDeleteLiftsDeletionProtectionFirst() throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"DeletionProtectionEnabled\":true}}");+ answer("UpdateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");+ answer("DeleteTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");++ backend.deleteTable(SCOPE, "orders");++ InOrder order = inOrder(client);+ order.verify(client).send(any(), eq(Api.DYNAMODB), eq("UpdateTable"),+ eq(json("{\"TableName\":\"orders\",\"DeletionProtectionEnabled\":false}")));+ order.verify(client).send(any(), eq(Api.DYNAMODB), eq("DeleteTable"), eq(json("{\"TableName\":\"orders\"}")));+ }++ @Test+ void directDeleteOfAnUnprotectedTableSendsOnlyDeleteTable() throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"DeletionProtectionEnabled\":false}}");+ answer("DeleteTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");++ backend.deleteTable(SCOPE, "orders");++ neverSent("UpdateTable");+ assertEquals(json("{\"TableName\":\"orders\"}"), sent("DeleteTable"));+ }++ @Test+ void wireTagRoundTrip() throws Exception {+ answerOrdersTable();++ Reply tagged = execute("TagResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\",\"Tags\":["+ + "{\"Key\":\"env\",\"Value\":\"dev\"},{\"Key\":\"team\",\"Value\":\"a\"}]}");+ JsonNode listed = execute("ListTagsOfResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\"}").body();+ Reply untagged = execute("UntagResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\",\"TagKeys\":[\"env\"]}");+ JsonNode remaining = execute("ListTagsOfResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\"}").body();++ assertEquals(200, tagged.status());+ assertEquals(json("{}"), tagged.body());+ assertEquals(json("{\"Tags\":[{\"Key\":\"env\",\"Value\":\"dev\"},"+ + "{\"Key\":\"team\",\"Value\":\"a\"}]}"), listed);+ assertEquals(200, untagged.status());+ assertEquals(json("{\"Tags\":[{\"Key\":\"team\",\"Value\":\"a\"}]}"), remaining);+ neverSent("TagResource");+ neverSent("UntagResource");+ neverSent("ListTagsOfResource");+ }++ @Test+ void replicaUpdatesRejected() throws Exception {+ AwsException wire = assertThrows(AwsException.class, () -> execute("UpdateTable",+ "{\"TableName\":\"orders\",\"ReplicaUpdates\":[{\"Create\":{\"RegionName\":\"us-east-1\"}}]}"));+ AwsException typed = assertThrows(AwsException.class,+ () -> backend.applyReplicaUpdates(SCOPE, "orders", List.of("us-east-1"), List.of()));++ for (AwsException error : List.of(wire, typed)) {+ assertEquals("ValidationException", error.getErrorCode());+ assertEquals(REPLICAS_UNSUPPORTED, error.getMessage());+ assertEquals(400, error.getHttpStatus());+ }+ neverSent("UpdateTable");+ }++ @Test+ void ensureGlobalTableDescribes() throws Exception {+ answerOrdersTable();++ TableDefinition table = backend.ensureGlobalTable(SCOPE, "orders");++ assertEquals("orders", table.getTableName());+ assertEquals(ORDERS_ARN, table.getTableArn());+ assertEquals("", table.getTableId());+ }++ @Test+ void createTableTypedProvisionedJson() throws Exception {+ answer("CreateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL+ + "table/orders\"}}");++ TableDefinition table = backend.createTable(SCOPE, "orders",+ List.of(new KeySchemaElement("pk", "HASH"), new KeySchemaElement("sk", "RANGE")),+ List.of(new AttributeDefinition("pk", "S"), new AttributeDefinition("sk", "S"),+ new AttributeDefinition("g", "S"), new AttributeDefinition("l", "N")),+ 3L, 4L,+ List.of(new GlobalSecondaryIndex("gsi1", List.of(new KeySchemaElement("g", "HASH")), null,+ "INCLUDE", List.of("a"))),+ List.of(new LocalSecondaryIndex("lsi1", List.of(new KeySchemaElement("pk", "HASH"),+ new KeySchemaElement("l", "RANGE")), null, "KEYS_ONLY")));++ assertEquals(json("""+ {"TableName":"orders",+ "KeySchema":[{"AttributeName":"pk","KeyType":"HASH"},{"AttributeName":"sk","KeyType":"RANGE"}],+ "AttributeDefinitions":[{"AttributeName":"pk","AttributeType":"S"},+ {"AttributeName":"sk","AttributeType":"S"},{"AttributeName":"g","AttributeType":"S"},+ {"AttributeName":"l","AttributeType":"N"}],+ "BillingMode":"PROVISIONED",+ "ProvisionedThroughput":{"ReadCapacityUnits":3,"WriteCapacityUnits":4},+ "GlobalSecondaryIndexes":[{"IndexName":"gsi1","KeySchema":[{"AttributeName":"g","KeyType":"HASH"}],+ "Projection":{"ProjectionType":"INCLUDE","NonKeyAttributes":["a"]},+ "ProvisionedThroughput":{"ReadCapacityUnits":3,"WriteCapacityUnits":4}}],+ "LocalSecondaryIndexes":[{"IndexName":"lsi1","KeySchema":[{"AttributeName":"pk","KeyType":"HASH"},+ {"AttributeName":"l","KeyType":"RANGE"}],"Projection":{"ProjectionType":"KEYS_ONLY"}}]}+ """), sent("CreateTable"));+ assertEquals(ORDERS_ARN, table.getTableArn());+ assertEquals(List.of(REGION + "/orders"), List.copyOf(tags.keysForAccount(ACCOUNT)));+ }++ @Test+ void createTableTypedOnDemandJson() throws Exception {+ answer("CreateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");+ GlobalSecondaryIndex gsi = new GlobalSecondaryIndex();+ gsi.setIndexName("gsi1");+ gsi.setKeySchema(List.of(new KeySchemaElement("g", "HASH")));++ backend.createTable(SCOPE, "orders", List.of(new KeySchemaElement("pk", "HASH")),+ List.of(new AttributeDefinition("pk", "S"), new AttributeDefinition("g", "S")),+ null, null, List.of(gsi), List.of());++ assertEquals(json("""+ {"TableName":"orders",+ "KeySchema":[{"AttributeName":"pk","KeyType":"HASH"}],+ "AttributeDefinitions":[{"AttributeName":"pk","AttributeType":"S"},+ {"AttributeName":"g","AttributeType":"S"}],+ "BillingMode":"PAY_PER_REQUEST",+ "GlobalSecondaryIndexes":[{"IndexName":"gsi1","KeySchema":[{"AttributeName":"g","KeyType":"HASH"}],+ "Projection":{"ProjectionType":"ALL"}}]}+ """), sent("CreateTable"));+ }++ @Test+ void describeTableMapsStreamFields() throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL + "table/orders\","+ + "\"CreationDateTime\":1.790463342979E9,"+ + "\"StreamSpecification\":{\"StreamEnabled\":true,\"StreamViewType\":\"KEYS_ONLY\"},"+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");++ TableDefinition table = backend.describeTable(SCOPE, "orders");++ assertTrue(table.isStreamEnabled());+ assertEquals("KEYS_ONLY", table.getStreamViewType());+ assertEquals(PUBLIC + STREAM_RESOURCE, table.getStreamArn());+ assertEquals(Instant.ofEpochMilli(1790463342979L), table.getCreationDateTime());+ }++ @Test+ void findTableAbsentIsEmpty() throws Exception {+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","+ + "\"message\":\"Requested resource not found\"}");++ assertEquals(Optional.empty(), backend.findTable(SCOPE, "orders"));+ }++ private void answerStreamingOrdersTable(String viewType) throws Exception {+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\","+ + "\"StreamSpecification\":{\"StreamEnabled\":true,\"StreamViewType\":\"" + viewType + "\"},"+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");+ }++ @Test+ void enableStreamIdempotent() throws Exception {+ answerStreamingOrdersTable("NEW_AND_OLD_IMAGES");++ TableDefinition unspecified = backend.enableStream(SCOPE, "orders", null);+ TableDefinition same = backend.enableStream(SCOPE, "orders", "NEW_AND_OLD_IMAGES");++ assertEquals(PUBLIC + STREAM_RESOURCE, unspecified.getStreamArn());+ assertEquals("NEW_AND_OLD_IMAGES", same.getStreamViewType());+ neverSent("UpdateTable");+ }++ @Test+ void enableStreamWithNewViewTypeRestartsStream() throws Exception {+ answerStreamingOrdersTable("NEW_AND_OLD_IMAGES");+ answer("UpdateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\","+ + "\"StreamSpecification\":{\"StreamEnabled\":true,\"StreamViewType\":\"KEYS_ONLY\"},"+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");++ TableDefinition table = backend.enableStream(SCOPE, "orders", "KEYS_ONLY");++ assertEquals(List.of(+ json("{\"TableName\":\"orders\",\"StreamSpecification\":{\"StreamEnabled\":false}}"),+ json("{\"TableName\":\"orders\",\"StreamSpecification\":{\"StreamEnabled\":true,"+ + "\"StreamViewType\":\"KEYS_ONLY\"}}")),+ sentBodies(Api.DYNAMODB, "UpdateTable"));+ assertEquals("KEYS_ONLY", table.getStreamViewType());+ }++ @Test+ void disableStreamIdempotent() throws Exception {+ answerOrdersTable();++ TableDefinition table = backend.disableStream(SCOPE, "orders");++ assertFalse(table.isStreamEnabled());+ assertNull(table.getStreamArn());+ neverSent("UpdateTable");+ }++ @Test+ void updateItemReturnsOnlyImages() throws Exception {+ answer("UpdateItem", 200, "{\"Attributes\":{\"pk\":{\"S\":\"a\"},\"n\":{\"N\":\"2\"}}}");+ JsonNode key = json("{\"pk\":{\"S\":\"a\"}}");++ JsonNode updatedOnly = backend.updateItem(SCOPE, "orders", key, null, "SET n = :n", null,+ json("{\":n\":{\"N\":\"2\"}}"), "UPDATED_NEW", null);+ JsonNode allNew = backend.updateItem(SCOPE, "orders", key, null, "SET n = :n", null,+ json("{\":n\":{\"N\":\"2\"}}"), "ALL_NEW", null);++ assertNull(updatedOnly);+ assertEquals(json("{\"pk\":{\"S\":\"a\"},\"n\":{\"N\":\"2\"}}"), allNew);+ assertEquals(json("{\"TableName\":\"orders\",\"Key\":{\"pk\":{\"S\":\"a\"}},"+ + "\"UpdateExpression\":\"SET n = :n\","+ + "\"ExpressionAttributeValues\":{\":n\":{\"N\":\"2\"}},\"ReturnValues\":\"ALL_NEW\"}"),+ sent("UpdateItem"));+ }++ @Test+ void scanPage() throws Exception {+ answer("Scan", 200, "{\"Items\":[{\"pk\":{\"S\":\"a\"}},{\"pk\":{\"S\":\"b\"}}],\"Count\":2,"+ + "\"ScannedCount\":5,\"LastEvaluatedKey\":{\"pk\":{\"S\":\"b\"}}}");++ ScanPage page = backend.scan(SCOPE, "orders", null, null, null, null, 2, json("{\"pk\":{\"S\":\"0\"}}"));++ assertEquals(List.of(json("{\"pk\":{\"S\":\"a\"}}"), json("{\"pk\":{\"S\":\"b\"}}")), page.items());+ assertEquals(5, page.scannedCount());+ assertEquals(json("{\"pk\":{\"S\":\"b\"}}"), page.lastEvaluatedKey());+ assertEquals(json("{\"TableName\":\"orders\",\"Limit\":2,\"ExclusiveStartKey\":{\"pk\":{\"S\":\"0\"}}}"),+ sent("Scan"));+ }++ @Test+ void resourcesListsTablesOfKnownRegions() throws Exception {+ tags.putForAccount(ACCOUNT, REGION + "/orders", Map.of("env", "dev"));+ tags.putForAccount(ACCOUNT, "us-east-1/users", Map.of());+ when(client.send(any(), eq(Api.DYNAMODB), eq("ListTables"), any())).thenAnswer(invocation -> {+ Scope scope = invocation.getArgument(0);+ JsonNode body = invocation.getArgument(3);+ String page;+ if ("us-east-1".equals(scope.region())) {+ page = "{\"TableNames\":[\"users\"]}";+ } else if (body.has("ExclusiveStartTableName")) {+ page = "{\"TableNames\":[\"direct\"]}";+ } else {+ page = "{\"TableNames\":[\"orders\"],\"LastEvaluatedTableName\":\"orders\"}";+ }+ return new Reply(200, json(page), Map.of());+ });++ List<ExplorerResource> resources = backend.resources(ACCOUNT);++ Map<String, ExplorerResource> byArn = resources.stream()+ .collect(Collectors.toMap(ExplorerResource::arn, resource -> resource));+ assertEquals(Set.of(ORDERS_ARN, PUBLIC + "table/direct", "arn:aws:dynamodb:us-east-1:123456789012:table/users"),+ byArn.keySet());+ ExplorerResource orders = byArn.get(ORDERS_ARN);+ assertEquals("dynamodb:table", orders.resourceType());+ assertEquals(REGION, orders.region());+ assertEquals(ACCOUNT, orders.owningAccountId());+ assertEquals(Map.of("env", "dev"), orders.tags());+ assertEquals(Map.of(), byArn.get(PUBLIC + "table/direct").tags());+ assertTrue(sentBodies(Api.DYNAMODB, "ListTables").contains(json("{\"ExclusiveStartTableName\":\"orders\"}")));+ }++ @Test+ void resetRefusedWith409() {+ AwsException error = assertThrows(AwsException.class, backend::checkReset);++ assertEquals("UnsupportedOperation", error.getErrorCode());+ assertEquals(409, error.getHttpStatus());+ }++ @Test+ void startRetriesConnectionFailuresThenSucceeds() throws Exception {+ Scope probe = new Scope("000000000000", "us-east-1");+ when(client.send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), any(), any()))+ .thenThrow(new ConnectException("refused"))+ .thenReturn(new Reply(200, json("{\"TableNames\":[]}"), Map.of()));++ backend.start();++ verify(client, times(2)).send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), eq(json("{\"Limit\":1}")), any());+ }++ @Test+ void startRetriesAConnectionClosedWithoutAnswer() throws Exception {+ Scope probe = new Scope("000000000000", "us-east-1");+ when(client.send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), any(), any()))+ .thenThrow(new IOException("HTTP/1.1 header parser received no bytes"))+ .thenReturn(new Reply(200, json("{\"TableNames\":[]}"), Map.of()));++ backend.start();++ verify(client, times(2)).send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), any(), any());+ }++ @Test+ void startCapsEachProbeAtTheRemainingBudget() throws Exception {+ ArgumentCaptor<Duration> budget = ArgumentCaptor.forClass(Duration.class);+ when(client.send(any(), any(), any(), any(), budget.capture()))+ .thenReturn(new Reply(200, json("{\"TableNames\":[]}"), Map.of()));++ backend.start();++ assertTrue(budget.getValue().isPositive());+ assertTrue(budget.getValue().compareTo(Duration.ofSeconds(5)) <= 0);+ }++ @Test+ void startFailsWhenBudgetExpires() throws Exception {+ backend = new DynamoDbLocalBackend(client, tags, mapper, Duration.ZERO);+ when(client.send(any(), any(), any(), any(), any())).thenThrow(new ConnectException("refused"));++ IllegalStateException error = assertThrows(IllegalStateException.class, backend::start);++ assertEquals("DynamoDB Local at http://local:8000 is not reachable after 0s", error.getMessage());+ }++ @Test+ void startFailsOnNon2xx() throws Exception {+ when(client.send(any(), any(), any(), any(), any())).thenReturn(new Reply(500, json("{\"message\":\"x\"}"), Map.of()));++ IllegalStateException error = assertThrows(IllegalStateException.class, backend::start);++ assertEquals("DynamoDB Local at http://local:8000 answered ListTables with status 500: {\"message\":\"x\"}",+ error.getMessage());+ }++ @Test+ void validatedEndpointMissing() {+ for (Optional<String> missing : List.of(Optional.<String>empty(), Optional.of(" "))) {+ IllegalStateException error = assertThrows(IllegalStateException.class,+ () -> DynamoDbLocalBackend.validatedEndpoint(missing));++ assertEquals("floci.services.dynamodb.local-endpoint is required when floci.services.dynamodb.backend "+ + "is local", error.getMessage());+ }+ }++ @Test+ void validatedEndpointFtpScheme() {+ IllegalStateException error = assertThrows(IllegalStateException.class,+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of("ftp://localhost:8000")));++ assertTrue(error.getMessage().contains("ftp://localhost:8000"), error.getMessage());+ }++ @Test+ void validatedEndpointAwsHost() {+ IllegalStateException error = assertThrows(IllegalStateException.class,+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of("https://dynamodb.us-east-1.amazonaws.com")));++ assertTrue(error.getMessage().endsWith("must point at DynamoDB Local, not an AWS endpoint"),+ error.getMessage());+ }++ @Test+ void validatedEndpointAwsHostInAnyPartitionOrWithTrailingDot() {+ for (String endpoint : List.of("https://dynamodb.us-east-1.amazonaws.com.",+ "https://dynamodb.cn-north-1.amazonaws.com.cn", "https://dynamodb.eu-isoe-west-1.cloud.adc-e.uk")) {+ assertThrows(IllegalStateException.class,+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of(endpoint)), endpoint);+ }+ }++ @Test+ void validatedEndpointUserInfoRejected() {+ assertThrows(IllegalStateException.class,+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of("http://user:secret@localhost:8000")));+ }++ @Test+ void validatedEndpointValid() {+ assertEquals(URI.create("http://localhost:8000"),+ DynamoDbLocalBackend.validatedEndpoint(Optional.of("http://localhost:8000")));+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClientTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClientTest.javanew file mode 100644index 0000000000..41a9f63c5e--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClientTest.java@@ -0,0 +1,247 @@+package io.github.hectorvent.floci.services.dynamodb.backend;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ObjectNode;+import com.sun.net.httpserver.Headers;+import com.sun.net.httpserver.HttpExchange;+import com.sun.net.httpserver.HttpServer;+import io.github.hectorvent.floci.core.common.auth.SigV4RequestValidator;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.Test;++import java.io.IOException;+import java.io.OutputStream;+import java.net.ConnectException;+import java.net.InetAddress;+import java.net.InetSocketAddress;+import java.net.ServerSocket;+import java.net.URI;+import java.net.http.HttpTimeoutException;+import java.nio.charset.StandardCharsets;+import java.time.Duration;+import java.util.concurrent.atomic.AtomicInteger;+import java.util.concurrent.atomic.AtomicReference;+import java.util.regex.Matcher;+import java.util.regex.Pattern;++import static org.junit.jupiter.api.Assertions.assertArrayEquals;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertNotNull;+import static org.junit.jupiter.api.Assertions.assertNull;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;++/** The signed HTTP client that forwards DynamoDB wire calls to an external DynamoDB Local. */+class DynamoDbLocalClientTest {++ private static final Scope SCOPE = new Scope("123456789012", "eu-west-1");+ private static final Pattern AUTHORIZATION = Pattern.compile(+ "AWS4-HMAC-SHA256 Credential=([^/]+)/(\\d{8})/([^/]+)/dynamodb/aws4_request, "+ + "SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=([0-9a-f]{64})");++ private final ObjectMapper mapper = new ObjectMapper();+ private final AtomicReference<Received> received = new AtomicReference<>();+ private final AtomicInteger hits = new AtomicInteger();+ private HttpServer server;+ private DynamoDbLocalClient client;++ private record Received(String path, Headers headers, byte[] body) {+ }++ @FunctionalInterface+ private interface Responder {+ void respond(HttpExchange exchange) throws Exception;+ }++ @AfterEach+ void tearDown() {+ if (client != null) {+ client.close();+ }+ if (server != null) {+ server.stop(0);+ }+ }++ @Test+ void signsAndForwardsExactBody() throws Exception {+ start(exchange -> reply(exchange, 200, "{}"));+ ObjectNode body = (ObjectNode) mapper.readTree("""+ {"TableName":"orders","Item":{+ "pk":{"S":"a"},+ "blob":{"B":"AAEC"},+ "blobs":{"BS":["AAEC","AwQF"]},+ "numbers":{"NS":["1","2.5"]},+ "big":{"N":"12345678901234567890123456789012345678"},+ "nested":{"M":{"inner":{"B":"BgcI"}}}}}+ """);++ Reply result = client.send(SCOPE, Api.DYNAMODB, "PutItem", body);++ assertEquals(200, result.status());+ Received request = received.get();+ assertArrayEquals(mapper.writeValueAsBytes(body), request.body());+ assertEquals("DynamoDB_20120810.PutItem", request.headers().getFirst("X-Amz-Target"));+ assertEquals("application/x-amz-json-1.0", request.headers().getFirst("Content-Type"));+ String amzDate = request.headers().getFirst("X-Amz-Date");+ assertNotNull(amzDate);+ String date = amzDate.substring(0, 8);+ String authorization = request.headers().getFirst("Authorization");+ assertTrue(authorization.contains(+ "Credential=floci123456789012/" + date + "/eu-west-1/dynamodb/aws4_request"),+ authorization);+ Matcher matcher = AUTHORIZATION.matcher(authorization);+ assertTrue(matcher.matches(), authorization);+ assertEquals(expectedSignature(request, date), matcher.group(4));+ }++ @Test+ void streamsTargetPrefix() throws Exception {+ start(exchange -> reply(exchange, 200, "{}"));++ client.send(SCOPE, Api.DYNAMODB_STREAMS, "GetRecords", mapper.createObjectNode());++ assertEquals("DynamoDBStreams_20120810.GetRecords", received.get().headers().getFirst("X-Amz-Target"));+ }++ @Test+ void errorBodyPreserved() throws Exception {+ String error = "{\"__type\":\"com.amazonaws.dynamodb.v20120810#TransactionCanceledException\","+ + "\"message\":\"x\",\"CancellationReasons\":[{\"Code\":\"ConditionalCheckFailed\"}]}";+ start(exchange -> reply(exchange, 400, error));++ Reply result = client.send(SCOPE, Api.DYNAMODB, "TransactWriteItems", mapper.createObjectNode());++ assertEquals(400, result.status());+ assertEquals(mapper.readTree(error), result.body());+ }++ @Test+ void unknownOperationPassesThrough() throws Exception {+ start(exchange -> reply(exchange, 200, "{}"));++ client.send(SCOPE, Api.DYNAMODB, "SomeFutureAction", mapper.createObjectNode());++ assertEquals("DynamoDB_20120810.SomeFutureAction", received.get().headers().getFirst("X-Amz-Target"));+ }++ @Test+ void nullBodySendsEmptyObject() throws Exception {+ start(exchange -> reply(exchange, 200, "{}"));++ client.send(SCOPE, Api.DYNAMODB, "ListTables", null);++ assertEquals("{}", new String(received.get().body(), StandardCharsets.UTF_8));+ }++ @Test+ void emptyResponseBodyIsNull() throws Exception {+ start(exchange -> reply(exchange, 200, ""));++ Reply result = client.send(SCOPE, Api.DYNAMODB, "ListTables", mapper.createObjectNode());++ assertEquals(200, result.status());+ assertNull(result.body());+ }++ @Test+ void connectionRefused() throws Exception {+ int port;+ try (ServerSocket socket = new ServerSocket(0, 1, InetAddress.getByName("127.0.0.1"))) {+ port = socket.getLocalPort();+ }+ client = new DynamoDbLocalClient(URI.create("http://127.0.0.1:" + port),+ Duration.ofSeconds(2), Duration.ofSeconds(2), mapper);++ assertThrows(ConnectException.class,+ () -> client.send(SCOPE, Api.DYNAMODB, "ListTables", mapper.createObjectNode()));+ }++ @Test+ void timedOutRequestIsSentOnce() throws Exception {+ start(exchange -> {+ hits.incrementAndGet();+ Thread.sleep(2000);+ reply(exchange, 200, "{}");+ }, Duration.ofMillis(200));++ assertThrows(HttpTimeoutException.class,+ () -> client.send(SCOPE, Api.DYNAMODB, "ListTables", mapper.createObjectNode()));+ Thread.sleep(500);+ assertEquals(1, hits.get());+ }++ @Test+ void aShorterBudgetCutsTheRequestTimeout() throws Exception {+ start(exchange -> {+ Thread.sleep(2000);+ reply(exchange, 200, "{}");+ }, Duration.ofSeconds(5));++ assertThrows(HttpTimeoutException.class, () -> client.send(SCOPE, Api.DYNAMODB, "ListTables",+ mapper.createObjectNode(), Duration.ofMillis(200)));+ }++ @Test+ void aLongerBudgetKeepsTheRequestTimeout() throws Exception {+ start(exchange -> {+ Thread.sleep(2000);+ reply(exchange, 200, "{}");+ }, Duration.ofMillis(200));++ assertThrows(HttpTimeoutException.class, () -> client.send(SCOPE, Api.DYNAMODB, "ListTables",+ mapper.createObjectNode(), Duration.ofSeconds(5)));+ }++ private void start(Responder responder) throws IOException {+ start(responder, Duration.ofSeconds(5));+ }++ private void start(Responder responder, Duration requestTimeout) throws IOException {+ server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);+ server.createContext("/", exchange -> {+ try (exchange) {+ received.set(new Received(exchange.getRequestURI().getRawPath(),+ exchange.getRequestHeaders(), exchange.getRequestBody().readAllBytes()));+ responder.respond(exchange);+ } catch (Exception e) {+ throw new IOException(e);+ }+ });+ server.start();+ URI endpoint = URI.create("http://127.0.0.1:" + server.getAddress().getPort());+ client = new DynamoDbLocalClient(endpoint, Duration.ofSeconds(2), requestTimeout, mapper);+ }++ private static void reply(HttpExchange exchange, int status, String body) throws IOException {+ byte[] bytes = body.getBytes(StandardCharsets.UTF_8);+ exchange.getResponseHeaders().set("Content-Type", "application/x-amz-json-1.0");+ exchange.sendResponseHeaders(status, bytes.length == 0 ? -1 : bytes.length);+ if (bytes.length > 0) {+ try (OutputStream out = exchange.getResponseBody()) {+ out.write(bytes);+ }+ }+ }++ private String expectedSignature(Received request, String date) throws Exception {+ Headers headers = request.headers();+ String amzDate = headers.getFirst("X-Amz-Date");+ String canonical = "POST\n" + request.path() + "\n\n"+ + "content-type:" + headers.getFirst("Content-Type") + "\n"+ + "host:" + headers.getFirst("Host") + "\n"+ + "x-amz-date:" + amzDate + "\n"+ + "x-amz-target:" + headers.getFirst("X-Amz-Target") + "\n\n"+ + "content-type;host;x-amz-date;x-amz-target\n"+ + SigV4RequestValidator.sha256Hex(request.body());+ String credentialScope = date + "/eu-west-1/dynamodb/aws4_request";+ String toSign = "AWS4-HMAC-SHA256\n" + amzDate + "\n" + credentialScope + "\n"+ + SigV4RequestValidator.sha256Hex(canonical);+ byte[] key = SigV4RequestValidator.deriveSigningKey("floci", date, "eu-west-1", "dynamodb");+ return SigV4RequestValidator.hexEncode(SigV4RequestValidator.hmacSha256(key, toSign));+ }+}diff --git a/src/test/resources/application.yml b/src/test/resources/application.ymlindex 01faad0b02..cf67c9d5d6 100644--- a/src/test/resources/application.yml+++ b/src/test/resources/application.yml@@ -141,6 +141,7 @@ floci:enabled: truevector-index-allocation-seconds: 0vector-index-backfill-seconds: 0+ backend: nativesns:enabled: truelambda: