Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(dynamodb): optional DynamoDB Local backend (#4504)

Mario Toffiac326dce97738
diff --git a/docs/configuration/advanced/application-yml.md b/docs/configuration/advanced/application-yml.md
index b33006a258..662c2deadc 100644
--- a/docs/configuration/advanced/application-yml.md
+++ b/docs/configuration/advanced/application-yml.md
@@ -130,6 +130,10 @@ floci:
dynamodb:
enabled: true
+ backend: native # native | local (forward calls to DynamoDB Local)
+ # local-endpoint: http://dynamodb-local:8000 # DynamoDB Local base URL, required when backend is local
+ local-connect-timeout-seconds: 2 # Seconds to wait for a connection to DynamoDB Local
+ local-request-timeout-seconds: 10 # Seconds to wait for DynamoDB Local to answer a forwarded request
sns:
enabled: true
diff --git a/docs/configuration/environment-variables.md b/docs/configuration/environment-variables.md
index 28c3edc8d1..c1625bd590 100644
--- a/docs/configuration/environment-variables.md
+++ b/docs/configuration/environment-variables.md
@@ -202,6 +202,10 @@ See [Initialization Hooks](./initialization-hooks.md) for lifecycle phases and s
| Variable | Default | Description |
|---|---|---|
| `FLOCI_SERVICES_DYNAMODB_ENABLED` | `true` | Enable the DynamoDB service |
+| `FLOCI_SERVICES_DYNAMODB_BACKEND` | `native` | Engine behind the DynamoDB API: `native` or `local` (DynamoDB Local), case-insensitive. Any other value fails startup. See [DynamoDB Local backend](../services/dynamodb.md#dynamodb-local-backend) |
+| `FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT` | _(none)_ | Base URL of DynamoDB Local, for example `http://dynamodb-local:8000`. Required when the backend is `local`. Must be `http` or `https` with a host. AWS endpoints (hosts under `amazonaws.com`, `amazonaws.com.cn` or `api.aws`) are rejected |
+| `FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS` | `2` | Seconds to wait for a connection to DynamoDB Local |
+| `FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS` | `10` | Seconds to wait for DynamoDB Local to answer a forwarded request |
### Lambda
diff --git a/docs/services/dynamodb.md b/docs/services/dynamodb.md
index cbf44c7670..6eb99b1660 100644
--- a/docs/services/dynamodb.md
+++ b/docs/services/dynamodb.md
@@ -70,6 +70,88 @@ epoch time in the past. Each deletion writes a `REMOVE` record to the table's st
forwarded to an active Kinesis streaming destination. Between sweeps, reads already leave expired
items out.
+## DynamoDB Local backend
+
+By default Floci stores tables in its own engine, the `native` backend. With
+`FLOCI_SERVICES_DYNAMODB_BACKEND=local`, Floci forwards DynamoDB calls to an
+[Amazon DynamoDB Local](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)
+instance that you run.
+
+Floci keeps serving the DynamoDB and DynamoDB Streams APIs on port 4566, in JSON and CBOR, with
+the same authentication, IAM enforcement and CRC32 checksums as the native backend. Each call is
+forwarded to DynamoDB Local over SigV4-signed HTTP/1.1. Floci never retries a forwarded request.
+Floci does not start DynamoDB Local. Run it yourself, for example as a container:
+
+```yaml
+services:
+ dynamodb-local:
+ image: amazon/dynamodb-local:3.3.1
+ command: ["-jar", "DynamoDBLocal.jar", "-inMemory"]
+ floci:
+ image: floci/floci:latest
+ ports:
+ - "4566:4566"
+ environment:
+ FLOCI_SERVICES_DYNAMODB_BACKEND: local
+ FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT: http://dynamodb-local:8000
+ depends_on:
+ - dynamodb-local
+```
+
+Do not start DynamoDB Local with `-sharedDb`. It merges every account and region into one database.
+
+Floci is tested with DynamoDB Local 3.3.1 and does not check the version it connects to. It relies
+on DynamoDB Local's fixed `ddblocal` ARNs and on its separate database per access key and region.
+A version that changes either one breaks ARN translation or account and region isolation.
+
+At startup Floci waits up to 30 seconds for DynamoDB Local to answer, then fails. Any
+non-success answer to that probe fails startup at once. `GET /_floci/info` reports the active
+backend in `dynamodb_backend` (`native` or `local`).
+
+### Accounts, regions and ARNs
+
+Each account and region is a separate DynamoDB Local namespace. Floci signs forwarded calls with
+the access key `floci<account-id>` and the caller's region, and DynamoDB Local keeps one database
+per access key and region. A call from account `000000000000` in `eu-west-1` is signed with access
+key `floci000000000000` and region `eu-west-1`.
+
+Table, index and stream ARNs in replies are public ARNs for the caller's account and region, such
+as `arn:aws:dynamodb:eu-west-1:000000000000:table/Users`. Floci rewrites the `ddblocal` ARNs that
+DynamoDB Local returns. Stream records carry the caller's region. An ARN for another account or
+region is rejected.
+
+### What Floci adds
+
+- **Tags.** DynamoDB Local has no tagging, so Floci handles `TagResource`, `UntagResource`,
+ `ListTagsOfResource` and `Tags` on `CreateTable` itself. It keeps the tags in its DynamoDB
+ storage, in the file `dynamodb-local-tags.json`. The tags follow Floci's storage mode, so with
+ `memory` storage they are lost when Floci restarts while the tables stay in DynamoDB Local.
+- **Stream consumers.** Lambda event source mappings, EventBridge Pipes and other Floci stream
+ consumers read the DynamoDB Local streams through the Streams API.
+
+### Limits
+
+- **No emulator reset.** `POST /_floci/state/reset` is refused with HTTP 409. Reset the DynamoDB
+ Local instance instead.
+- **No replicas.** Replicas and global table replica updates (`UpdateTable` with `ReplicaUpdates`)
+ are refused with a `ValidationException`. A CloudFormation global table with no extra replica
+ regions deploys as a single table.
+- **No `TableId`.** DynamoDB Local returns none, so `DescribeTable` has no `TableId` and
+ CloudFormation `Fn::GetAtt` on `TableId` is empty.
+- **Short names in batch replies.** Batch replies keyed by table always use the short table name,
+ including when the request named the table by ARN.
+- **No tag validation.** Floci does not check tag counts or tag keys.
+- **Resource Explorer searches regions Floci knows.** DynamoDB Local cannot list its namespaces,
+ so Resource Explorer lists every table in each region where Floci's storage holds at least one
+ table record, including tables created there directly on DynamoDB Local. A region with no such
+ record is not searched: one whose tables were all created directly on DynamoDB Local, or whose
+ records were lost with `memory` storage.
+- **No data migration.** Data does not move between `native` and `local`. After a switch, you see
+ what that engine holds.
+- **DynamoDB Local sets the feature set.** Operations and features follow DynamoDB Local; see the
+ [DynamoDB Local documentation](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/DynamoDBLocal.html)
+ for what it supports. The other sections of this page describe the native backend.
+
## Configuration
| Variable | Default | Description |
@@ -79,6 +161,10 @@ items out.
| `FLOCI_STORAGE_SERVICES_DYNAMODB_FLUSH_INTERVAL_MS` | `5000` | Flush interval for `hybrid`/`wal` storage modes (milliseconds) |
| `FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_ALLOCATION_SECONDS` | `4` | Seconds a vector index added by `UpdateTable` spends in resource allocation |
| `FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_BACKFILL_SECONDS` | `10` | Seconds that index then spends backfilling before it goes `ACTIVE` |
+| `FLOCI_SERVICES_DYNAMODB_BACKEND` | `native` | Engine behind the DynamoDB API: `native` or `local`, case-insensitive. Any other value fails startup. See [DynamoDB Local backend](#dynamodb-local-backend) |
+| `FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT` | *(none)* | Base URL of DynamoDB Local, for example `http://dynamodb-local:8000`. Required when the backend is `local`. Must be `http` or `https` with a host. AWS endpoints (hosts under `amazonaws.com`, `amazonaws.com.cn` or `api.aws`) are rejected |
+| `FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS` | `2` | Seconds to wait for a connection to DynamoDB Local |
+| `FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS` | `10` | Seconds to wait for DynamoDB Local to answer a forwarded request |
### Storage and Performance
diff --git a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
index 18ae70b2af..2a33312296 100644
--- a/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
+++ b/src/main/java/io/github/hectorvent/floci/config/EmulatorConfig.java
@@ -1342,6 +1342,21 @@ public interface EmulatorConfig {
*/
@WithDefault("10")
int vectorIndexBackfillSeconds();
+
+ /** The engine behind the DynamoDB API: {@code native} or {@code local}. Env: FLOCI_SERVICES_DYNAMODB_BACKEND. */
+ @WithDefault("native")
+ String backend();
+
+ /** The DynamoDB Local base URL, required when backend is local. Env: FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT. */
+ Optional<String> localEndpoint();
+
+ /** Env: FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS. */
+ @WithDefault("2")
+ int localConnectTimeoutSeconds();
+
+ /** Env: FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS. */
+ @WithDefault("10")
+ int localRequestTimeoutSeconds();
}
interface SnsServiceConfig {
diff --git a/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java b/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java
index 8f90b27a1d..3f6d21d210 100644
--- a/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java
+++ b/src/main/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoController.java
@@ -7,6 +7,7 @@ import io.github.hectorvent.floci.core.common.ContainerTeardown;
import io.github.hectorvent.floci.core.common.ContainerTeardowns;
import io.github.hectorvent.floci.core.common.ServiceRegistry;
import io.github.hectorvent.floci.lifecycle.inithook.InitializationHook;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbBackendSelector;
import jakarta.inject.Inject;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
@@ -44,6 +45,7 @@ public class EmulatorInfoController {
private final Instance<ContainerTeardown> containerTeardowns;
private final FlociCertificateAuthority certificateAuthority;
private final EmulatorConfig config;
+ private final DynamoDbBackendSelector dynamoDbBackendSelector;
@Inject
public EmulatorInfoController(ServiceRegistry serviceRegistry,
@@ -52,7 +54,8 @@ public class EmulatorInfoController {
Instance<Resettable> resettables,
Instance<ContainerTeardown> containerTeardowns,
FlociCertificateAuthority certificateAuthority,
- EmulatorConfig config) {
+ EmulatorConfig config,
+ DynamoDbBackendSelector dynamoDbBackendSelector) {
this.serviceRegistry = serviceRegistry;
this.initLifecycleState = initLifecycleState;
this.storageFactory = storageFactory;
@@ -60,6 +63,7 @@ public class EmulatorInfoController {
this.containerTeardowns = containerTeardowns;
this.certificateAuthority = certificateAuthority;
this.config = config;
+ this.dynamoDbBackendSelector = dynamoDbBackendSelector;
this.version = resolveVersion();
}
@@ -98,7 +102,8 @@ public class EmulatorInfoController {
@GET
@Path("/info")
public Response info() {
- return Response.ok(Map.of("version", version, "edition", "community", "original_edition", "floci-always-free")).build();
+ return Response.ok(Map.of("version", version, "edition", "community", "original_edition", "floci-always-free",
+ "dynamodb_backend", dynamoDbBackendSelector.selected())).build();
}
@GET
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java
index f3f7f7aa53..fddba393f8 100644
--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbApiStreamReader.java
@@ -3,10 +3,8 @@ package io.github.hectorvent.floci.services.dynamodb.backend;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.node.JsonNodeFactory;
import com.fasterxml.jackson.databind.node.ObjectNode;
-import io.github.hectorvent.floci.core.common.AwsException;
import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;
import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Call;
-import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;
import java.util.ArrayList;
import java.util.List;
@@ -78,22 +76,13 @@ public final class DynamoDbApiStreamReader implements DynamoDbStreamReader {
}
private JsonNode call(Stream stream, String action, ObjectNode body) {
- Reply reply;
try {
- reply = operations.execute(new Call(stream.scope(), Api.DYNAMODB_STREAMS, action, body));
+ return operations.execute(new Call(stream.scope(), Api.DYNAMODB_STREAMS, action, body)).successBody();
} catch (RuntimeException e) {
throw e;
} catch (Exception e) {
throw new IllegalStateException(e);
}
- JsonNode replyBody = reply.body() == null ? JSON.missingNode() : reply.body();
- if (reply.status() >= 300) {
- String type = text(replyBody, "__type");
- String code = type == null ? "InternalServerError" : type.substring(type.lastIndexOf('#') + 1);
- String message = text(replyBody, "message");
- throw new AwsException(code, message != null ? message : text(replyBody, "Message"), reply.status());
- }
- return replyBody;
}
private static String text(JsonNode node, String field) {
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackend.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackend.java
new file mode 100644
index 0000000000..6a8acdf453
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackend.java
@@ -0,0 +1,5 @@
+package io.github.hectorvent.floci.services.dynamodb.backend;
+
+/** One selectable DynamoDB engine: every seam interface from a single object. */
+public interface DynamoDbBackend extends DynamoDbOperations, DynamoDbItemAccess, DynamoDbTableAccess,
+ DynamoDbBackendLifecycle {}
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java
index 76a5eb04ab..551679deb4 100644
--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelector.java
@@ -1,49 +1,71 @@
package io.github.hectorvent.floci.services.dynamodb.backend;
+import io.github.hectorvent.floci.config.EmulatorConfig;
import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.CheckpointLifetime;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.inject.Produces;
import jakarta.inject.Inject;
+import java.util.Locale;
+
/**
- * The one immutable DynamoDB backend selection. Native is the only engine today; a configurable
- * choice arrives with the alternative engine. {@link NativeDynamoDbBackend} is typed to its concrete
- * class and {@link DynamoDbApiStreamReader} is not a bean, so these producers are the only
- * unqualified beans for the five seam interfaces.
+ * The one immutable DynamoDB backend selection, read from {@code floci.services.dynamodb.backend}:
+ * {@code native} (the default) runs the in-process engine, {@code local} forwards to an external
+ * DynamoDB Local. Both engines are {@code @ApplicationScoped} client proxies, so the unselected one
+ * is never constructed. Each engine is typed to its concrete class and {@link DynamoDbApiStreamReader}
+ * is not a bean, so these producers are the only unqualified beans for the five seam interfaces.
*/
@ApplicationScoped
public class DynamoDbBackendSelector {
- private final NativeDynamoDbBackend nativeBackend;
+ private final String selected;
+ private final DynamoDbBackend backend;
@Inject
- public DynamoDbBackendSelector(NativeDynamoDbBackend nativeBackend) {
- this.nativeBackend = nativeBackend;
+ public DynamoDbBackendSelector(EmulatorConfig config, NativeDynamoDbBackend nativeBackend,
+ DynamoDbLocalBackend localBackend) {
+ String configured = config.services().dynamodb().backend();
+ selected = configured.trim().toLowerCase(Locale.ROOT);
+ backend = switch (selected) {
+ case "native" -> nativeBackend;
+ case "local" -> localBackend;
+ default -> throw new IllegalStateException(
+ "floci.services.dynamodb.backend must be 'native' or 'local', got '" + configured + "'");
+ };
+ }
+
+ /** The selected engine name, {@code native} or {@code local}. */
+ public String selected() {
+ return selected;
}
@Produces
DynamoDbOperations operations() {
- return nativeBackend;
+ return backend;
}
@Produces
DynamoDbItemAccess itemAccess() {
- return nativeBackend;
+ return backend;
}
@Produces
DynamoDbTableAccess tableAccess() {
- return nativeBackend;
+ return backend;
}
@Produces
DynamoDbBackendLifecycle lifecycle() {
- return nativeBackend;
+ return backend;
}
- /** Native stream history lives only in memory, so committed progress ends with the process. */
+ /**
+ * Native stream history lives only in memory, so committed progress ends with the process;
+ * DynamoDB Local keeps stream history outside it, so progress lasts as long as the stream.
+ */
@Produces
DynamoDbStreamReader streamReader() {
- return new DynamoDbApiStreamReader(nativeBackend, CheckpointLifetime.PROCESS);
+ return new DynamoDbApiStreamReader(backend,
+ "local".equals(selected) ? CheckpointLifetime.STREAM : CheckpointLifetime.PROCESS);
}
}
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackend.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackend.java
new file mode 100644
index 0000000000..768d3d840c
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackend.java
@@ -0,0 +1,627 @@
+package io.github.hectorvent.floci.services.dynamodb.backend;
+
+import com.fasterxml.jackson.core.type.TypeReference;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.PropertyNamingStrategies;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.MissingNode;
+import com.fasterxml.jackson.databind.node.NullNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.fasterxml.jackson.databind.node.TextNode;
+import io.github.hectorvent.floci.config.EmulatorConfig;
+import io.github.hectorvent.floci.core.common.AwsArnUtils;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.core.common.AwsPartitions;
+import io.github.hectorvent.floci.core.resource.ExplorerResource;
+import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;
+import io.github.hectorvent.floci.core.storage.StorageFactory;
+import io.github.hectorvent.floci.services.dynamodb.DynamoDbTableNames;
+import io.github.hectorvent.floci.services.dynamodb.model.AttributeDefinition;
+import io.github.hectorvent.floci.services.dynamodb.model.GlobalSecondaryIndex;
+import io.github.hectorvent.floci.services.dynamodb.model.KeySchemaElement;
+import io.github.hectorvent.floci.services.dynamodb.model.LocalSecondaryIndex;
+import io.github.hectorvent.floci.services.dynamodb.model.TableDefinition;
+import jakarta.enterprise.context.ApplicationScoped;
+import jakarta.enterprise.inject.Typed;
+import jakarta.inject.Inject;
+import org.jboss.logging.Logger;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.URISyntaxException;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.LinkedHashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.regex.Pattern;
+
+/**
+ * The DynamoDB engine backed by an external, official DynamoDB Local. Wire calls are forwarded
+ * through {@link DynamoDbLocalClient} under the caller's account and region; table and stream ARNs
+ * are translated between the public shape and Local's fixed {@code ddblocal} shape. Local keeps no
+ * tags, so Floci holds them in its own store keyed by account, region and table name.
+ */
+@ApplicationScoped
+@Typed(DynamoDbLocalBackend.class)
+public class DynamoDbLocalBackend implements DynamoDbBackend {
+
+ private static final Logger LOG = Logger.getLogger(DynamoDbLocalBackend.class);
+
+ private static final String LOCAL_ARN_PREFIX = "arn:aws:dynamodb:ddblocal:000000000000:"; // partition-literal: DynamoDB Local's fixed ARN prefix
+ private static final String LOCAL_REGION = "ddblocal";
+ private static final Duration READINESS_BUDGET = Duration.ofSeconds(30);
+ private static final Scope PROBE = new Scope("000000000000", "us-east-1"); // partition-literal: readiness probe namespace, Local accepts any region
+ private static final String REPLICAS_UNSUPPORTED = "Replicas are not supported by the DynamoDB Local backend";
+ private static final Pattern PARTITION = Pattern.compile(AwsArnUtils.PARTITION_REGEX);
+ private static final List<String> PUBLIC_ARN_FIELDS = List.of("TableArn", "IndexArn", "LatestStreamArn",
+ "StreamArn", "LastEvaluatedStreamArn", "message", "Message");
+
+ private final DynamoDbLocalClient client;
+ private final AccountAwareStorageBackend<Map<String, String>> tags;
+ private final ObjectMapper objectMapper;
+ private final ObjectMapper aws;
+ private final Duration readiness;
+
+ @Inject
+ public DynamoDbLocalBackend(EmulatorConfig config, StorageFactory storageFactory, ObjectMapper objectMapper) {
+ this(new DynamoDbLocalClient(validatedEndpoint(config.services().dynamodb().localEndpoint()),
+ Duration.ofSeconds(config.services().dynamodb().localConnectTimeoutSeconds()),
+ Duration.ofSeconds(config.services().dynamodb().localRequestTimeoutSeconds()), objectMapper),
+ storageFactory.create("dynamodb", "dynamodb-local-tags.json",
+ new TypeReference<Map<String, Map<String, String>>>() {}),
+ objectMapper, READINESS_BUDGET);
+ }
+
+ DynamoDbLocalBackend(DynamoDbLocalClient client, AccountAwareStorageBackend<Map<String, String>> tags,
+ ObjectMapper objectMapper, Duration readiness) {
+ this.client = client;
+ this.tags = tags;
+ this.objectMapper = objectMapper;
+ this.aws = objectMapper.copy().setPropertyNamingStrategy(PropertyNamingStrategies.UPPER_CAMEL_CASE);
+ this.readiness = readiness;
+ }
+
+ static URI validatedEndpoint(Optional<String> value) {
+ String raw = value.map(String::trim).filter(v -> !v.isEmpty()).orElseThrow(() -> new IllegalStateException(
+ "floci.services.dynamodb.local-endpoint is required when floci.services.dynamodb.backend is local"));
+ URI uri;
+ try {
+ uri = new URI(raw);
+ } catch (URISyntaxException e) {
+ throw invalidEndpoint(raw, e);
+ }
+ String scheme = uri.getScheme();
+ if (scheme == null || !(scheme.equalsIgnoreCase("http") || scheme.equalsIgnoreCase("https"))
+ || uri.getHost() == null || uri.getRawUserInfo() != null) {
+ throw invalidEndpoint(raw, null);
+ }
+ String host = uri.getHost().replaceFirst("\\.$", "");
+ if (AwsPartitions.isDnsSuffix(host) || AwsPartitions.stripKnownDnsSuffix(host).isPresent()) {
+ throw new IllegalStateException("floci.services.dynamodb.local-endpoint '" + raw
+ + "' must point at DynamoDB Local, not an AWS endpoint");
+ }
+ return uri;
+ }
+
+ private static IllegalStateException invalidEndpoint(String raw, Exception cause) {
+ return new IllegalStateException("floci.services.dynamodb.local-endpoint '" + raw
+ + "' is not a valid http or https URL", cause);
+ }
+
+ @Override
+ public Reply execute(Call call) {
+ Scope scope = call.scope();
+ String action = call.action();
+ JsonNode body = call.body() == null ? objectMapper.createObjectNode() : call.body().deepCopy();
+ if (!"CreateTable".equals(action)) {
+ toLocal(scope, body);
+ }
+ Reply served = call.api() != Api.DYNAMODB ? null : switch (action) {
+ case "TagResource" -> {
+ tagResource(scope, text(body, "ResourceArn"), tagsFrom(body.path("Tags")));
+ yield new Reply(200, objectMapper.createObjectNode(), Map.of());
+ }
+ case "UntagResource" -> {
+ List<String> keys = new ArrayList<>();
+ body.path("TagKeys").forEach(key -> keys.add(key.asText()));
+ untagResource(scope, text(body, "ResourceArn"), keys);
+ yield new Reply(200, objectMapper.createObjectNode(), Map.of());
+ }
+ case "ListTagsOfResource" -> {
+ String resourceArn = text(body, "ResourceArn");
+ Map<String, String> found;
+ try {
+ found = listTagsOfResource(scope, resourceArn);
+ } catch (AwsException e) {
+ if (!"ResourceNotFoundException".equals(e.getErrorCode())) {
+ throw e;
+ }
+ throw new AwsException("AccessDeniedException",
+ "User is not authorized to perform: dynamodb:ListTagsOfResource on resource: "
+ + resourceArn, 400);
+ }
+ ObjectNode reply = objectMapper.createObjectNode();
+ ArrayNode tagList = reply.putArray("Tags");
+ found.forEach((key, value) -> tagList.addObject().put("Key", key).put("Value", value));
+ yield new Reply(200, reply, Map.of());
+ }
+ case "UpdateTable" -> {
+ if (body.has("ReplicaUpdates")) {
+ throw new AwsException("ValidationException", REPLICAS_UNSUPPORTED, 400);
+ }
+ yield null;
+ }
+ default -> null;
+ };
+ if (served != null) {
+ return served;
+ }
+ Reply reply = send(scope, call.api(), action, body);
+ if (reply.status() >= 200 && reply.status() < 300 && call.api() == Api.DYNAMODB) {
+ JsonNode replyBody = reply.body() == null ? MissingNode.getInstance() : reply.body();
+ String tableName = text(replyBody.path("TableDescription"), "TableName");
+ if (tableName != null && "CreateTable".equals(action)) {
+ tags.putForAccount(scope.accountId(), key(scope.region(), tableName), tagsFrom(body.path("Tags")));
+ } else if (tableName != null && "DeleteTable".equals(action)) {
+ tags.deleteForAccount(scope.accountId(), key(scope.region(), tableName));
+ }
+ }
+ toPublic(scope, reply.body());
+ return reply;
+ }
+
+ private Reply send(Scope scope, Api api, String action, JsonNode body) {
+ try {
+ return client.send(scope, api, action, body);
+ } catch (IOException e) {
+ throw notAnswered(e);
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw notAnswered(e);
+ }
+ }
+
+ private AwsException notAnswered(Exception e) {
+ return new AwsException("InternalServerError",
+ "DynamoDB Local at " + client.endpoint() + " did not answer: " + e, 500);
+ }
+
+ static void toLocal(Scope scope, JsonNode node) {
+ if (node instanceof ObjectNode object) {
+ if (object.get("RequestItems") instanceof ObjectNode requestItems) {
+ ObjectNode renamed = object.objectNode();
+ for (Map.Entry<String, JsonNode> entry : requestItems.properties()) {
+ String name = tableName(scope, entry.getKey());
+ if (renamed.replace(name, entry.getValue()) != null) {
+ throw new AwsException("ValidationException",
+ "Table " + name + " is named more than once in RequestItems", 400);
+ }
+ }
+ object.set("RequestItems", renamed);
+ }
+ for (Map.Entry<String, JsonNode> field : object.properties()) {
+ JsonNode value = field.getValue();
+ if (value.isTextual() && "TableName".equals(field.getKey())) {
+ field.setValue(TextNode.valueOf(tableName(scope, value.asText())));
+ } else if (value.isTextual()
+ && ("StreamArn".equals(field.getKey()) || "ExclusiveStartStreamArn".equals(field.getKey()))) {
+ field.setValue(TextNode.valueOf(localStreamArn(scope, value.asText())));
+ } else if (value.isContainerNode()) {
+ toLocal(scope, value);
+ }
+ }
+ } else if (node instanceof ArrayNode array) {
+ array.forEach(element -> toLocal(scope, element));
+ }
+ }
+
+ static String tableName(Scope scope, String value) {
+ if (!value.startsWith("arn:")) {
+ return value;
+ }
+ DynamoDbTableNames.resolveWithRegion(value, scope.region());
+ return ownTableName(scope, value);
+ }
+
+ // A well-formed table ARN of another account, region or partition names no table in this namespace.
+ private static String ownTableName(Scope scope, String arn) {
+ String name = DynamoDbTableNames.resolve(arn);
+ if (!PARTITION.matcher(AwsArnUtils.parse(arn).partition()).matches()) {
+ throw new AwsException("ValidationException", "Invalid table ARN: " + arn, 400);
+ }
+ if (!arn.startsWith(publicPrefix(scope))) {
+ throw new AwsException("ResourceNotFoundException",
+ "Requested resource not found: Table: " + arn + " not found", 400);
+ }
+ return name;
+ }
+
+ static String localStreamArn(Scope scope, String value) {
+ if (!AwsArnUtils.isArn(value)) {
+ return value;
+ }
+ AwsArnUtils.Arn arn = AwsArnUtils.parse(value);
+ if (!"dynamodb".equals(arn.service()) || !arn.resource().contains("/stream/")) {
+ return value;
+ }
+ if (!value.startsWith(publicPrefix(scope))) {
+ throw new AwsException("ResourceNotFoundException",
+ "Requested resource not found: Stream: " + value + " not found", 400);
+ }
+ return LOCAL_ARN_PREFIX + arn.resource();
+ }
+
+ private static String publicPrefix(Scope scope) {
+ return AwsArnUtils.Arn.of("dynamodb", scope.region(), scope.accountId(), "").toString();
+ }
+
+ static void toPublic(Scope scope, JsonNode node) {
+ toPublic(node, publicPrefix(scope), scope.region());
+ }
+
+ private static void toPublic(JsonNode node, String prefix, String region) {
+ if (node instanceof ObjectNode object) {
+ for (Map.Entry<String, JsonNode> field : object.properties()) {
+ JsonNode value = field.getValue();
+ if (value.isTextual() && PUBLIC_ARN_FIELDS.contains(field.getKey())) {
+ field.setValue(TextNode.valueOf(value.asText().replace(LOCAL_ARN_PREFIX, prefix)));
+ } else if (value.isTextual() && "awsRegion".equals(field.getKey())
+ && LOCAL_REGION.equals(value.asText())) {
+ field.setValue(TextNode.valueOf(region));
+ } else if (value.isContainerNode()) {
+ toPublic(value, prefix, region);
+ }
+ }
+ } else if (node instanceof ArrayNode array) {
+ array.forEach(element -> toPublic(element, prefix, region));
+ }
+ }
+
+ private JsonNode call(Scope scope, String action, ObjectNode body) {
+ return execute(new Call(scope, Api.DYNAMODB, action, body)).successBody();
+ }
+
+ private ObjectNode body(String tableName, Object... nameValuePairs) {
+ ObjectNode body = aws.createObjectNode().put("TableName", tableName);
+ for (int i = 0; i < nameValuePairs.length; i += 2) {
+ Object value = nameValuePairs[i + 1];
+ if (value == null || value instanceof NullNode || value instanceof MissingNode) {
+ continue;
+ }
+ String name = (String) nameValuePairs[i];
+ body.set(name, value instanceof JsonNode node ? node : aws.valueToTree(value));
+ }
+ return body;
+ }
+
+ @Override
+ public void putItem(Scope scope, String tableName, JsonNode item, String conditionExpression,
+ JsonNode expressionAttributeNames, JsonNode expressionAttributeValues) {
+ call(scope, "PutItem", body(tableName, "Item", item, "ConditionExpression", conditionExpression,
+ "ExpressionAttributeNames", expressionAttributeNames,
+ "ExpressionAttributeValues", expressionAttributeValues));
+ }
+
+ @Override
+ public JsonNode getItem(Scope scope, String tableName, JsonNode key) {
+ return present(call(scope, "GetItem", body(tableName, "Key", key)).get("Item"));
+ }
+
+ @Override
+ public void deleteItem(Scope scope, String tableName, JsonNode key, String conditionExpression,
+ JsonNode expressionAttributeNames, JsonNode expressionAttributeValues) {
+ call(scope, "DeleteItem", body(tableName, "Key", key, "ConditionExpression", conditionExpression,
+ "ExpressionAttributeNames", expressionAttributeNames,
+ "ExpressionAttributeValues", expressionAttributeValues));
+ }
+
+ @Override
+ public JsonNode updateItem(Scope scope, String tableName, JsonNode key, JsonNode attributeUpdates,
+ String updateExpression, JsonNode expressionAttributeNames,
+ JsonNode expressionAttributeValues, String returnValues, String conditionExpression) {
+ JsonNode reply = call(scope, "UpdateItem", body(tableName, "Key", key, "AttributeUpdates", attributeUpdates,
+ "UpdateExpression", updateExpression, "ExpressionAttributeNames", expressionAttributeNames,
+ "ExpressionAttributeValues", expressionAttributeValues, "ReturnValues", returnValues,
+ "ConditionExpression", conditionExpression));
+ if ("ALL_NEW".equals(returnValues) || "ALL_OLD".equals(returnValues)) {
+ return present(reply.get("Attributes"));
+ }
+ return null;
+ }
+
+ @Override
+ public ScanPage scan(Scope scope, String tableName, String filterExpression, JsonNode expressionAttributeNames,
+ JsonNode expressionAttributeValues, JsonNode scanFilter, Integer limit,
+ JsonNode exclusiveStartKey) {
+ JsonNode reply = call(scope, "Scan", body(tableName, "FilterExpression", filterExpression,
+ "ExpressionAttributeNames", expressionAttributeNames,
+ "ExpressionAttributeValues", expressionAttributeValues, "ScanFilter", scanFilter, "Limit", limit,
+ "ExclusiveStartKey", exclusiveStartKey));
+ List<JsonNode> items = new ArrayList<>();
+ reply.path("Items").forEach(items::add);
+ return new ScanPage(items, reply.path("ScannedCount").asInt(), present(reply.get("LastEvaluatedKey")));
+ }
+
+ private static JsonNode present(JsonNode node) {
+ return node == null || node.isNull() ? null : node;
+ }
+
+ @Override
+ public TableDefinition createTable(Scope scope, String tableName, List<KeySchemaElement> keySchema,
+ List<AttributeDefinition> attributeDefinitions, Long readCapacity,
+ Long writeCapacity, List<GlobalSecondaryIndex> globalSecondaryIndexes,
+ List<LocalSecondaryIndex> localSecondaryIndexes) {
+ ObjectNode body = body(tableName, "KeySchema", keySchema, "AttributeDefinitions", attributeDefinitions);
+ ObjectNode throughput = null;
+ if (readCapacity != null && writeCapacity != null) {
+ throughput = aws.createObjectNode()
+ .put("ReadCapacityUnits", readCapacity)
+ .put("WriteCapacityUnits", writeCapacity);
+ body.put("BillingMode", "PROVISIONED").set("ProvisionedThroughput", throughput);
+ } else {
+ body.put("BillingMode", "PAY_PER_REQUEST");
+ }
+ if (globalSecondaryIndexes != null && !globalSecondaryIndexes.isEmpty()) {
+ ArrayNode indexes = body.putArray("GlobalSecondaryIndexes");
+ for (GlobalSecondaryIndex gsi : globalSecondaryIndexes) {
+ ObjectNode index = index(indexes, gsi.getIndexName(), gsi.getKeySchema(), gsi.getProjectionType(),
+ gsi.getNonKeyAttributes());
+ if (throughput != null) {
+ index.set("ProvisionedThroughput", throughput.deepCopy());
+ }
+ }
+ }
+ if (localSecondaryIndexes != null && !localSecondaryIndexes.isEmpty()) {
+ ArrayNode indexes = body.putArray("LocalSecondaryIndexes");
+ for (LocalSecondaryIndex lsi : localSecondaryIndexes) {
+ index(indexes, lsi.getIndexName(), lsi.getKeySchema(), lsi.getProjectionType(),
+ lsi.getNonKeyAttributes());
+ }
+ }
+ return table(call(scope, "CreateTable", body).path("TableDescription"));
+ }
+
+ private ObjectNode index(ArrayNode indexes, String indexName, List<KeySchemaElement> keySchema,
+ String projectionType, List<String> nonKeyAttributes) {
+ ObjectNode index = indexes.addObject().put("IndexName", indexName);
+ index.set("KeySchema", aws.valueToTree(keySchema));
+ ObjectNode projection = index.putObject("Projection")
+ .put("ProjectionType", projectionType == null ? "ALL" : projectionType);
+ if (nonKeyAttributes != null && !nonKeyAttributes.isEmpty()) {
+ projection.set("NonKeyAttributes", aws.valueToTree(nonKeyAttributes));
+ }
+ return index;
+ }
+
+ @Override
+ public TableDefinition describeTable(Scope scope, String tableName) {
+ return table(call(scope, "DescribeTable", body(tableName)).path("Table"));
+ }
+
+ @Override
+ public Optional<TableDefinition> findTable(Scope scope, String tableName) {
+ try {
+ return Optional.of(describeTable(scope, tableName));
+ } catch (AwsException e) {
+ if ("ResourceNotFoundException".equals(e.getErrorCode())) {
+ return Optional.empty();
+ }
+ throw e;
+ }
+ }
+
+ // The seam's direct delete skips deletion protection, like native; the wire DeleteTable still enforces it.
+ // ponytail: a DeleteTable that fails after the UpdateTable leaves protection off; the stack delete that asked
+ // for removal reports DELETE_FAILED and a retry completes it.
+ @Override
+ public void deleteTable(Scope scope, String tableName) {
+ if (describeTable(scope, tableName).isDeletionProtectionEnabled()) {
+ call(scope, "UpdateTable", body(tableName, "DeletionProtectionEnabled", false));
+ }
+ call(scope, "DeleteTable", body(tableName));
+ }
+
+ @Override
+ public TableDefinition enableStream(Scope scope, String tableName, String viewType) {
+ String type = viewType == null ? "NEW_AND_OLD_IMAGES" : viewType;
+ TableDefinition current = describeTable(scope, tableName);
+ if (current.isStreamEnabled()) {
+ if (viewType == null || type.equals(current.getStreamViewType())) {
+ return current;
+ }
+ // ponytail: Local cannot retarget a live stream, so a new view type starts a new stream ARN
+ updateStream(scope, tableName, false, null);
+ }
+ return updateStream(scope, tableName, true, type);
+ }
+
+ @Override
+ public TableDefinition disableStream(Scope scope, String tableName) {
+ TableDefinition current = describeTable(scope, tableName);
+ if (!current.isStreamEnabled()) {
+ return current;
+ }
+ return updateStream(scope, tableName, false, null);
+ }
+
+ private TableDefinition updateStream(Scope scope, String tableName, boolean enabled, String viewType) {
+ ObjectNode body = body(tableName);
+ ObjectNode spec = body.putObject("StreamSpecification").put("StreamEnabled", enabled);
+ if (viewType != null) {
+ spec.put("StreamViewType", viewType);
+ }
+ return table(call(scope, "UpdateTable", body).path("TableDescription"));
+ }
+
+ private TableDefinition table(JsonNode description) {
+ TableDefinition table = aws.convertValue(description, TableDefinition.class);
+ JsonNode spec = description.path("StreamSpecification");
+ table.setStreamEnabled(spec.path("StreamEnabled").asBoolean(false));
+ table.setStreamViewType(text(spec, "StreamViewType"));
+ table.setStreamArn(table.isStreamEnabled() ? text(description, "LatestStreamArn") : null);
+ table.setTableId(description.path("TableId").asText());
+ return table;
+ }
+
+ @Override
+ public Map<String, String> listTagsOfResource(Scope scope, String resourceArn) {
+ return new LinkedHashMap<>(tags.getForAccount(scope.accountId(), existingTableKey(scope, resourceArn))
+ .orElse(Map.of()));
+ }
+
+ // ponytail: no tag limits or tag-key validation; Floci records what callers send.
+ @Override
+ public void tagResource(Scope scope, String resourceArn, Map<String, String> tagsToAdd) {
+ String key = existingTableKey(scope, resourceArn);
+ synchronized (tags) {
+ Map<String, String> current = new LinkedHashMap<>(tags.getForAccount(scope.accountId(), key).orElse(Map.of()));
+ current.putAll(tagsToAdd);
+ tags.putForAccount(scope.accountId(), key, current);
+ }
+ }
+
+ @Override
+ public void untagResource(Scope scope, String resourceArn, List<String> tagKeys) {
+ String key = existingTableKey(scope, resourceArn);
+ synchronized (tags) {
+ Map<String, String> current = new LinkedHashMap<>(tags.getForAccount(scope.accountId(), key).orElse(Map.of()));
+ tagKeys.forEach(current::remove);
+ tags.putForAccount(scope.accountId(), key, current);
+ }
+ }
+
+ private String existingTableKey(Scope scope, String resourceArn) {
+ if (resourceArn == null || !resourceArn.startsWith("arn:")) {
+ throw new AwsException("ValidationException", "Invalid TableArn", 400);
+ }
+ String name = ownTableName(scope, resourceArn);
+ describeTable(scope, name);
+ return key(scope.region(), name);
+ }
+
+ @Override
+ public TableDefinition applyReplicaUpdates(Scope scope, String tableName, List<String> addRegions,
+ List<String> removeRegions) {
+ throw new AwsException("ValidationException", REPLICAS_UNSUPPORTED, 400);
+ }
+
+ // ponytail: a single-region global table has no replicas, so it is the table itself.
+ @Override
+ public TableDefinition ensureGlobalTable(Scope scope, String tableName) {
+ return describeTable(scope, tableName);
+ }
+
+ // ponytail: Local cannot list its namespaces, so regions come from the tag store: a region Floci
+ // has no table record for (made directly on Local, or lost with memory storage) is not listed.
+ @Override
+ public List<ExplorerResource> resources(String accountId) {
+ List<String> regions = tags.keysForAccount(accountId).stream()
+ .map(k -> k.substring(0, k.indexOf('/')))
+ .distinct()
+ .sorted()
+ .toList();
+ List<ExplorerResource> resources = new ArrayList<>();
+ for (String region : regions) {
+ Scope scope = new Scope(accountId, region);
+ String start = null;
+ do {
+ ObjectNode request = objectMapper.createObjectNode();
+ if (start != null) {
+ request.put("ExclusiveStartTableName", start);
+ }
+ JsonNode page = call(scope, "ListTables", request);
+ for (JsonNode tableName : page.path("TableNames")) {
+ String name = tableName.asText();
+ resources.add(new ExplorerResource(
+ AwsArnUtils.Arn.of("dynamodb", region, accountId, "table/" + name).toString(),
+ "dynamodb:table", "dynamodb", region, accountId, Instant.now(),
+ tags.getForAccount(accountId, key(region, name)).orElse(Map.of())));
+ }
+ start = text(page, "LastEvaluatedTableName");
+ } while (start != null);
+ }
+ return resources;
+ }
+
+ @Override
+ public void start() {
+ long deadline = System.nanoTime() + readiness.toNanos();
+ while (true) {
+ if (ready(deadline)) {
+ LOG.infov("DynamoDB backend: DynamoDB Local at {0}", client.endpoint());
+ return;
+ }
+ try {
+ Thread.sleep(500);
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new IllegalStateException("Interrupted while waiting for DynamoDB Local at "
+ + client.endpoint(), e);
+ }
+ }
+ }
+
+ private boolean ready(long deadline) {
+ Reply reply;
+ try {
+ reply = client.send(PROBE, Api.DYNAMODB, "ListTables", objectMapper.createObjectNode().put("Limit", 1),
+ Duration.ofNanos(Math.max(1, deadline - System.nanoTime())));
+ } catch (IOException e) {
+ if (System.nanoTime() - deadline >= 0) {
+ throw new IllegalStateException("DynamoDB Local at " + client.endpoint() + " is not reachable after "
+ + readiness.toSeconds() + "s", e);
+ }
+ return false;
+ } catch (InterruptedException e) {
+ Thread.currentThread().interrupt();
+ throw new IllegalStateException("Interrupted while waiting for DynamoDB Local at " + client.endpoint(), e);
+ }
+ if (reply.status() >= 200 && reply.status() < 300) {
+ return true;
+ }
+ throw new IllegalStateException("DynamoDB Local at " + client.endpoint() + " answered ListTables with status "
+ + reply.status() + ": " + reply.body());
+ }
+
+ @Override
+ public void checkReset() {
+ throw new AwsException("UnsupportedOperation", "The DynamoDB Local backend keeps its data outside Floci; "
+ + "reset the external DynamoDB Local instance instead", 409);
+ }
+
+ @Override
+ public void beforeReset() {
+ }
+
+ @Override
+ public void reset() {
+ }
+
+ @Override
+ public void afterReset() {
+ }
+
+ @Override
+ public void stop() {
+ client.close();
+ }
+
+ private static String key(String region, String tableName) {
+ return region + "/" + tableName;
+ }
+
+ private static Map<String, String> tagsFrom(JsonNode tagList) {
+ Map<String, String> result = new LinkedHashMap<>();
+ tagList.forEach(tag -> result.put(tag.path("Key").asText(), tag.path("Value").asText()));
+ return result;
+ }
+
+ private static String text(JsonNode node, String field) {
+ JsonNode value = node == null ? null : node.get(field);
+ return value == null || value.isNull() ? null : value.asText();
+ }
+}
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClient.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClient.java
new file mode 100644
index 0000000000..bdfdeb7b14
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClient.java
@@ -0,0 +1,107 @@
+package io.github.hectorvent.floci.services.dynamodb.backend;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.github.hectorvent.floci.core.common.auth.SigV4RequestValidator;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;
+
+import java.io.IOException;
+import java.net.URI;
+import java.net.http.HttpClient;
+import java.net.http.HttpRequest;
+import java.net.http.HttpResponse;
+import java.time.Duration;
+import java.time.ZoneOffset;
+import java.time.ZonedDateTime;
+import java.time.format.DateTimeFormatter;
+import java.util.Map;
+
+/**
+ * Forwards DynamoDB and DynamoDB Streams wire calls to an external DynamoDB Local over SigV4-signed
+ * HTTP. The caller's account becomes the access key and its region the signing region, so Local
+ * keeps each account and region apart. Requests are never retried.
+ */
+class DynamoDbLocalClient implements AutoCloseable {
+
+ private static final String CONTENT_TYPE = "application/x-amz-json-1.0";
+ private static final String SIGNED_HEADERS = "content-type;host;x-amz-date;x-amz-target";
+ private static final String SERVICE = "dynamodb";
+ private static final String SECRET_KEY = "floci";
+ private static final DateTimeFormatter AMZ_DATE = DateTimeFormatter.ofPattern("yyyyMMdd'T'HHmmss'Z'");
+
+ private final URI endpoint;
+ private final Duration requestTimeout;
+ private final ObjectMapper mapper;
+ private final HttpClient http;
+ private final String host;
+ private final String path;
+
+ DynamoDbLocalClient(URI endpoint, Duration connectTimeout, Duration requestTimeout, ObjectMapper mapper) {
+ this.endpoint = endpoint;
+ this.requestTimeout = requestTimeout;
+ this.mapper = mapper;
+ this.http = HttpClient.newBuilder()
+ .version(HttpClient.Version.HTTP_1_1)
+ .connectTimeout(connectTimeout)
+ .build();
+ this.host = endpoint.getHost() + (endpoint.getPort() != -1 ? ":" + endpoint.getPort() : "");
+ String rawPath = endpoint.getRawPath();
+ this.path = rawPath == null || rawPath.isEmpty() ? "/" : rawPath;
+ }
+
+ Reply send(Scope scope, Api api, String action, JsonNode body) throws IOException, InterruptedException {
+ return send(scope, api, action, body, requestTimeout);
+ }
+
+ Reply send(Scope scope, Api api, String action, JsonNode body, Duration budget)
+ throws IOException, InterruptedException {
+ byte[] payload = mapper.writeValueAsBytes(body == null ? mapper.createObjectNode() : body);
+ String target = (api == Api.DYNAMODB ? "DynamoDB_20120810." : "DynamoDBStreams_20120810.") + action;
+ String amzDate = AMZ_DATE.format(ZonedDateTime.now(ZoneOffset.UTC));
+ HttpRequest request = HttpRequest.newBuilder(endpoint.resolve(path))
+ .timeout(budget.compareTo(requestTimeout) < 0 ? budget : requestTimeout)
+ .header("Content-Type", CONTENT_TYPE)
+ .header("X-Amz-Date", amzDate)
+ .header("X-Amz-Target", target)
+ .header("Authorization", authorization(scope, target, amzDate, payload))
+ .POST(HttpRequest.BodyPublishers.ofByteArray(payload))
+ .build();
+ HttpResponse<byte[]> response = http.send(request, HttpResponse.BodyHandlers.ofByteArray());
+ byte[] responseBody = response.body();
+ JsonNode replyBody = responseBody == null || responseBody.length == 0 ? null : mapper.readTree(responseBody);
+ return new Reply(response.statusCode(), replyBody, Map.of());
+ }
+
+ URI endpoint() {
+ return endpoint;
+ }
+
+ @Override
+ public void close() {
+ http.shutdownNow();
+ }
+
+ private String authorization(Scope scope, String target, String amzDate, byte[] payload) {
+ String date = amzDate.substring(0, 8);
+ String credentialScope = date + "/" + scope.region() + "/" + SERVICE + "/aws4_request";
+ try {
+ String canonical = "POST\n" + path + "\n\n"
+ + "content-type:" + CONTENT_TYPE + "\n"
+ + "host:" + host + "\n"
+ + "x-amz-date:" + amzDate + "\n"
+ + "x-amz-target:" + target + "\n\n"
+ + SIGNED_HEADERS + "\n"
+ + SigV4RequestValidator.sha256Hex(payload);
+ String toSign = "AWS4-HMAC-SHA256\n" + amzDate + "\n" + credentialScope + "\n"
+ + SigV4RequestValidator.sha256Hex(canonical);
+ byte[] signingKey = SigV4RequestValidator.deriveSigningKey(SECRET_KEY, date, scope.region(), SERVICE);
+ String signature = SigV4RequestValidator.hexEncode(SigV4RequestValidator.hmacSha256(signingKey, toSign));
+ return "AWS4-HMAC-SHA256 Credential=floci" + scope.accountId() + "/" + credentialScope
+ + ", SignedHeaders=" + SIGNED_HEADERS + ", Signature=" + signature;
+ } catch (Exception e) {
+ throw new IllegalStateException("Failed to sign DynamoDB Local request", e);
+ }
+ }
+}
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java
index 566d2a0902..fe2a496a7f 100644
--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbOperations.java
@@ -1,6 +1,8 @@
package io.github.hectorvent.floci.services.dynamodb.backend;
import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.node.MissingNode;
+import io.github.hectorvent.floci.core.common.AwsException;
import java.util.List;
import java.util.Map;
@@ -43,6 +45,23 @@ public interface DynamoDbOperations {
public Reply {
headers = headers == null ? Map.of() : Map.copyOf(headers);
}
+
+ /** The body of a 2xx reply; a non-2xx reply is thrown as the AwsException its AWS error body names. */
+ public JsonNode successBody() {
+ JsonNode replyBody = body == null ? MissingNode.getInstance() : body;
+ if (status >= 300) {
+ String type = text(replyBody, "__type");
+ String code = type == null ? "InternalServerError" : type.substring(type.lastIndexOf('#') + 1);
+ String message = text(replyBody, "message");
+ throw new AwsException(code, message != null ? message : text(replyBody, "Message"), status);
+ }
+ return replyBody;
+ }
+
+ private static String text(JsonNode node, String field) {
+ JsonNode value = node.get(field);
+ return value == null || value.isNull() ? null : value.asText();
+ }
}
Reply execute(Call call) throws Exception;
diff --git a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java
index 71cd5998d5..6d374daaff 100644
--- a/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java
+++ b/src/main/java/io/github/hectorvent/floci/services/dynamodb/backend/NativeDynamoDbBackend.java
@@ -30,8 +30,7 @@ import java.util.Optional;
*/
@ApplicationScoped
@Typed(NativeDynamoDbBackend.class)
-public class NativeDynamoDbBackend implements DynamoDbOperations, DynamoDbItemAccess, DynamoDbTableAccess,
- DynamoDbBackendLifecycle {
+public class NativeDynamoDbBackend implements DynamoDbBackend {
private final NativeDynamoDbJsonHandler jsonHandler;
private final NativeDynamoDbStreamsJsonHandler streamsHandler;
diff --git a/src/main/resources/application.yml b/src/main/resources/application.yml
index 7f9a55abbd..60a78fb133 100644
--- a/src/main/resources/application.yml
+++ b/src/main/resources/application.yml
@@ -322,6 +322,10 @@ floci:
enabled: true
vector-index-allocation-seconds: 4 # FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_ALLOCATION_SECONDS. Resource allocation phase of a vector index added by UpdateTable.
vector-index-backfill-seconds: 10 # FLOCI_SERVICES_DYNAMODB_VECTOR_INDEX_BACKFILL_SECONDS. Backfilling phase that follows it.
+ backend: native # FLOCI_SERVICES_DYNAMODB_BACKEND: native or local
+ # local-endpoint: http://dynamodb-local:8000 # FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT
+ local-connect-timeout-seconds: 2 # FLOCI_SERVICES_DYNAMODB_LOCAL_CONNECT_TIMEOUT_SECONDS
+ local-request-timeout-seconds: 10 # FLOCI_SERVICES_DYNAMODB_LOCAL_REQUEST_TIMEOUT_SECONDS
sns:
enabled: true
lambda:
diff --git a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java
index 53ecfa5af5..08d35da364 100644
--- a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java
+++ b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerIntegrationTest.java
@@ -92,7 +92,8 @@ class EmulatorInfoControllerIntegrationTest {
.statusCode(200)
.contentType("application/json")
.body("edition", equalTo("community"))
- .body("version", notNullValue());
+ .body("version", notNullValue())
+ .body("dynamodb_backend", equalTo("native"));
}
@ParameterizedTest
diff --git a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java
index 5fdcba6e69..5e583ed2c3 100644
--- a/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java
+++ b/src/test/java/io/github/hectorvent/floci/lifecycle/EmulatorInfoControllerTest.java
@@ -6,6 +6,7 @@ import io.github.hectorvent.floci.core.common.ContainerTeardown;
import io.github.hectorvent.floci.core.common.Resettable;
import io.github.hectorvent.floci.core.common.ServiceRegistry;
import io.github.hectorvent.floci.core.storage.StorageFactory;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbBackendSelector;
import jakarta.enterprise.inject.Instance;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.DisplayName;
@@ -16,6 +17,7 @@ import org.mockito.Mock;
import org.mockito.junit.jupiter.MockitoExtension;
import java.util.List;
+import java.util.Map;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertSame;
@@ -40,6 +42,7 @@ class EmulatorInfoControllerTest {
@Mock private Instance<ContainerTeardown> containerTeardowns;
@Mock private FlociCertificateAuthority certificateAuthority;
@Mock private EmulatorConfig config;
+ @Mock private DynamoDbBackendSelector dynamoDbBackendSelector;
@Mock private ContainerTeardown sageMakerTeardown;
@Mock private ContainerTeardown batchTeardown;
@Mock private Resettable resettable;
@@ -49,7 +52,17 @@ class EmulatorInfoControllerTest {
@BeforeEach
void setUp() {
controller = new EmulatorInfoController(serviceRegistry, initLifecycleState, storageFactory,
- resettables, containerTeardowns, certificateAuthority, config);
+ resettables, containerTeardowns, certificateAuthority, config, dynamoDbBackendSelector);
+ }
+
+ @Test
+ @DisplayName("Should report the selected DynamoDB backend in info")
+ void info_reportsTheSelectedDynamoDbBackend() {
+ when(dynamoDbBackendSelector.selected()).thenReturn("local");
+
+ Map<?, ?> info = (Map<?, ?>) controller.info().getEntity();
+
+ assertEquals("local", info.get("dynamodb_backend"));
}
@Test
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java
index f7698f91c6..8ac2afd71c 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudwatch/logs/CloudWatchLogsMetricFilterResetTest.java
@@ -115,6 +115,6 @@ class CloudWatchLogsMetricFilterResetTest {
when(resettables.iterator()).thenAnswer(call -> List.of(resettable).iterator());
Instance<ContainerTeardown> teardowns = mock(Instance.class);
when(teardowns.iterator()).thenAnswer(call -> List.<ContainerTeardown>of().iterator());
- return new EmulatorInfoController(null, null, f.factory, resettables, teardowns, null, null);
+ return new EmulatorInfoController(null, null, f.factory, resettables, teardowns, null, null, null);
}
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java
index c4dfdfbe03..bf02eefe6f 100644
--- a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbBackendSelectionTest.java
@@ -3,6 +3,7 @@ package io.github.hectorvent.floci.services.dynamodb.backend;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.config.EmulatorConfig;
import io.github.hectorvent.floci.core.common.AwsException;
import io.github.hectorvent.floci.core.resource.ResourceProvider;
import io.github.hectorvent.floci.services.dynamodb.DynamoDbFacade;
@@ -18,8 +19,10 @@ import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader
import io.github.hectorvent.floci.services.dynamodb.model.AttributeDefinition;
import io.github.hectorvent.floci.services.dynamodb.model.KeySchemaElement;
import io.github.hectorvent.floci.services.dynamodb.model.TableDefinition;
+import io.quarkus.arc.Arc;
import io.quarkus.arc.ClientProxy;
import io.quarkus.test.junit.QuarkusTest;
+import jakarta.enterprise.context.ApplicationScoped;
import jakarta.enterprise.inject.Instance;
import jakarta.inject.Inject;
import org.junit.jupiter.api.Test;
@@ -28,11 +31,16 @@ import java.util.List;
import java.util.UUID;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertInstanceOf;
import static org.junit.jupiter.api.Assertions.assertNotEquals;
import static org.junit.jupiter.api.Assertions.assertSame;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.RETURNS_DEEP_STUBS;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verifyNoInteractions;
+import static org.mockito.Mockito.when;
/** The one DynamoDB backend CDI selects, and the one Resource Explorer provider in front of it. */
@QuarkusTest
@@ -64,16 +72,81 @@ class DynamoDbBackendSelectionTest {
@Inject
ObjectMapper mapper;
+ @Inject
+ DynamoDbBackendSelector selector;
+
@Test
void everyCapabilityIsTheOneNativeBackend() {
Object backend = ClientProxy.unwrap(operations);
+ assertEquals("native", selector.selected());
assertInstanceOf(NativeDynamoDbBackend.class, backend);
assertSame(backend, ClientProxy.unwrap(items));
assertSame(backend, ClientProxy.unwrap(tables));
assertSame(backend, ClientProxy.unwrap(lifecycle));
}
+ @Test
+ void theUnselectedLocalBackendIsNeverConstructed() {
+ assertFalse(Arc.container().getActiveContext(ApplicationScoped.class).getState().getContextualInstances()
+ .keySet().stream().anyMatch(bean -> bean.getBeanClass() == DynamoDbLocalBackend.class));
+ }
+
+ @Test
+ void nativeServesEverySeamWithProcessCheckpoints() {
+ NativeDynamoDbBackend nativeBackend = mock(NativeDynamoDbBackend.class);
+ DynamoDbLocalBackend localBackend = mock(DynamoDbLocalBackend.class);
+ DynamoDbBackendSelector nativeSelector = new DynamoDbBackendSelector(config("native"), nativeBackend, localBackend);
+
+ assertEquals("native", nativeSelector.selected());
+ assertServesEverySeam(nativeBackend, nativeSelector);
+ assertEquals(CheckpointLifetime.PROCESS, nativeSelector.streamReader().checkpointLifetime());
+ verifyNoInteractions(localBackend);
+ }
+
+ @Test
+ void localServesEverySeamWithStreamCheckpoints() {
+ NativeDynamoDbBackend nativeBackend = mock(NativeDynamoDbBackend.class);
+ DynamoDbLocalBackend localBackend = mock(DynamoDbLocalBackend.class);
+ DynamoDbBackendSelector localSelector = new DynamoDbBackendSelector(config("local"), nativeBackend, localBackend);
+
+ assertEquals("local", localSelector.selected());
+ assertServesEverySeam(localBackend, localSelector);
+ assertEquals(CheckpointLifetime.STREAM, localSelector.streamReader().checkpointLifetime());
+ verifyNoInteractions(nativeBackend);
+ }
+
+ @Test
+ void theBackendNameIsTrimmedAndCaseInsensitive() {
+ DynamoDbLocalBackend localBackend = mock(DynamoDbLocalBackend.class);
+ DynamoDbBackendSelector localSelector =
+ new DynamoDbBackendSelector(config(" LOCAL "), mock(NativeDynamoDbBackend.class), localBackend);
+
+ assertEquals("local", localSelector.selected());
+ assertSame(localBackend, localSelector.operations());
+ }
+
+ @Test
+ void anUnknownBackendFailsSelection() {
+ IllegalStateException failure = assertThrows(IllegalStateException.class, () -> new DynamoDbBackendSelector(
+ config("sqlite"), mock(NativeDynamoDbBackend.class), mock(DynamoDbLocalBackend.class)));
+
+ assertEquals("floci.services.dynamodb.backend must be 'native' or 'local', got 'sqlite'", failure.getMessage());
+ }
+
+ private static EmulatorConfig config(String backend) {
+ EmulatorConfig config = mock(EmulatorConfig.class, RETURNS_DEEP_STUBS);
+ when(config.services().dynamodb().backend()).thenReturn(backend);
+ return config;
+ }
+
+ private static void assertServesEverySeam(DynamoDbBackend expected, DynamoDbBackendSelector selector) {
+ assertSame(expected, selector.operations());
+ assertSame(expected, selector.itemAccess());
+ assertSame(expected, selector.tableAccess());
+ assertSame(expected, selector.lifecycle());
+ }
+
@Test
void startupSchedulesTheTtlSweep() {
assertTrue(Thread.getAllStackTraces().keySet().stream()
diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendIntegrationTest.java
new file mode 100644
index 0000000000..c85c2ac86a
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendIntegrationTest.java
@@ -0,0 +1,302 @@
+package io.github.hectorvent.floci.services.dynamodb.backend;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import io.github.hectorvent.floci.services.dynamodb.DynamoDbFacade;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.CheckpointLifetime;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Cursor;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Position;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.RecordsPage;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Shard;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbStreamReader.Stream;
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.quarkus.test.junit.QuarkusTest;
+import io.quarkus.test.junit.QuarkusTestProfile;
+import io.quarkus.test.junit.TestProfile;
+import io.restassured.response.ValidatableResponse;
+import jakarta.inject.Inject;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.condition.EnabledIfEnvironmentVariable;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.UUID;
+import java.util.concurrent.ThreadLocalRandom;
+
+import static io.restassured.RestAssured.given;
+import static org.hamcrest.Matchers.contains;
+import static org.hamcrest.Matchers.containsInAnyOrder;
+import static org.hamcrest.Matchers.endsWith;
+import static org.hamcrest.Matchers.equalTo;
+import static org.hamcrest.Matchers.hasSize;
+import static org.hamcrest.Matchers.nullValue;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/** The DynamoDB API served by an external DynamoDB Local, reached at FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT. */
+@QuarkusTest
+@TestProfile(DynamoDbLocalBackendIntegrationTest.LocalProfile.class)
+@EnabledIfEnvironmentVariable(named = DynamoDbLocalBackendIntegrationTest.ENDPOINT_VARIABLE, matches = ".+")
+class DynamoDbLocalBackendIntegrationTest {
+
+ static final String ENDPOINT_VARIABLE = "FLOCI_SERVICES_DYNAMODB_LOCAL_ENDPOINT";
+
+ public static final class LocalProfile implements QuarkusTestProfile {
+ @Override
+ public Map<String, String> getConfigOverrides() {
+ String endpoint = System.getenv(ENDPOINT_VARIABLE);
+ return Map.of("floci.services.dynamodb.backend", "local",
+ "floci.services.dynamodb.local-endpoint", endpoint == null ? "" : endpoint);
+ }
+ }
+
+ private record Caller(String account, String region) {
+ String authorization() {
+ return "AWS4-HMAC-SHA256 Credential=" + account + "/20260215/" + region
+ + "/dynamodb/aws4_request, SignedHeaders=host, Signature=abc";
+ }
+ }
+
+ private record CreatedTable(Caller caller, String name) {}
+
+ private static final String CONTENT_TYPE = "application/x-amz-json-1.0";
+ private static final String ACCOUNT_A = randomAccount();
+ private static final String ACCOUNT_B = otherAccountThan(ACCOUNT_A);
+ private static final Caller A_EAST = new Caller(ACCOUNT_A, "us-east-1");
+ private static final Caller A_WEST = new Caller(ACCOUNT_A, "eu-west-1");
+ private static final Caller B_EAST = new Caller(ACCOUNT_B, "us-east-1");
+ private static final String STREAMED = ", \"StreamSpecification\": {\"StreamEnabled\": true, "
+ + "\"StreamViewType\": \"NEW_AND_OLD_IMAGES\"}";
+
+ @Inject
+ DynamoDbStreamReader streamReader;
+
+ @Inject
+ DynamoDbFacade dynamoDbFacade;
+
+ private final List<CreatedTable> created = new ArrayList<>();
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @AfterEach
+ void deleteCreatedTables() {
+ for (CreatedTable table : created) {
+ dynamoDb(table.caller(), "DeleteTable", "{\"TableName\": \"" + table.name() + "\"}").statusCode(200);
+ }
+ created.clear();
+ }
+
+ @Test
+ void theSameTableNameIsIsolatedAcrossAccountsAndRegions() {
+ String table = tableName("isolation");
+ for (Caller caller : List.of(A_EAST, B_EAST, A_WEST)) {
+ createTable(caller, table, "").statusCode(200);
+ }
+ putItem(A_EAST, table, "only-in-a-east");
+
+ scanCount(A_EAST, table).body("Count", equalTo(1));
+ scanCount(B_EAST, table).body("Count", equalTo(0));
+ scanCount(A_WEST, table).body("Count", equalTo(0));
+ dynamoDb(B_EAST, "GetItem", "{\"TableName\": \"" + table + "\", \"Key\": {\"pk\": {\"S\": \"only-in-a-east\"}}}")
+ .statusCode(200)
+ .body("Item", nullValue());
+ }
+
+ @Test
+ void tableArnsArePublicAndAcceptedAsTableNames() {
+ String table = tableName("arn");
+ String arn = tableArn(A_WEST, table);
+
+ createTable(A_WEST, table, "").statusCode(200).body("TableDescription.TableArn", equalTo(arn));
+
+ dynamoDb(A_WEST, "DescribeTable", "{\"TableName\": \"" + arn + "\"}")
+ .statusCode(200)
+ .body("Table.TableName", equalTo(table))
+ .body("Table.TableArn", equalTo(arn));
+ }
+
+ @Test
+ void tagsRoundTripAlongsideCreateTableTags() {
+ String table = tableName("tags");
+ String arn = tableArn(B_EAST, table);
+ createTable(B_EAST, table, ", \"Tags\": [{\"Key\": \"env\", \"Value\": \"test\"}]").statusCode(200);
+
+ dynamoDb(B_EAST, "TagResource",
+ "{\"ResourceArn\": \"" + arn + "\", \"Tags\": [{\"Key\": \"team\", \"Value\": \"data\"}]}")
+ .statusCode(200);
+ dynamoDb(B_EAST, "ListTagsOfResource", "{\"ResourceArn\": \"" + arn + "\"}")
+ .statusCode(200)
+ .body("Tags.Key", containsInAnyOrder("env", "team"));
+
+ dynamoDb(B_EAST, "UntagResource", "{\"ResourceArn\": \"" + arn + "\", \"TagKeys\": [\"env\"]}")
+ .statusCode(200);
+ dynamoDb(B_EAST, "ListTagsOfResource", "{\"ResourceArn\": \"" + arn + "\"}")
+ .statusCode(200)
+ .body("Tags.Key", contains("team"))
+ .body("Tags.Value", contains("data"));
+ }
+
+ @Test
+ void listingTagsOfAMissingTableIsAccessDenied() {
+ dynamoDb(B_EAST, "ListTagsOfResource", "{\"ResourceArn\": \"" + tableArn(B_EAST, tableName("absent")) + "\"}")
+ .statusCode(400)
+ .body("__type", endsWith("AccessDeniedException"));
+ }
+
+ @Test
+ void streamRecordsCarryPublicArnsAndTheCallerRegion() {
+ String table = tableName("stream");
+ String streamArn = streamedTableWithOneItem(A_WEST, table);
+ assertTrue(streamArn.startsWith(tableArn(A_WEST, table) + "/stream/"), streamArn);
+
+ streams(A_WEST, "ListStreams", "{\"TableName\": \"" + table + "\"}")
+ .statusCode(200)
+ .body("Streams.StreamArn", contains(streamArn));
+ String shardId = streams(A_WEST, "DescribeStream", "{\"StreamArn\": \"" + streamArn + "\"}")
+ .statusCode(200)
+ .body("StreamDescription.StreamArn", equalTo(streamArn))
+ .extract().path("StreamDescription.Shards[0].ShardId");
+ String iterator = streams(A_WEST, "GetShardIterator", "{\"StreamArn\": \"" + streamArn
+ + "\", \"ShardId\": \"" + shardId + "\", \"ShardIteratorType\": \"TRIM_HORIZON\"}")
+ .statusCode(200)
+ .extract().path("ShardIterator");
+
+ streams(A_WEST, "GetRecords", "{\"ShardIterator\": \"" + iterator + "\"}")
+ .statusCode(200)
+ .body("Records", hasSize(1))
+ .body("Records[0].eventName", equalTo("INSERT"))
+ .body("Records[0].awsRegion", equalTo("eu-west-1"))
+ .body("Records[0].dynamodb.Keys.pk.S", equalTo("streamed"));
+ }
+
+ @Test
+ void aStreamOfAnotherAccountIsNotFound() {
+ String table = tableName("foreign-stream");
+ String streamArn = streamedTableWithOneItem(A_EAST, table);
+
+ streams(B_EAST, "DescribeStream", "{\"StreamArn\": \"" + streamArn + "\"}")
+ .statusCode(400)
+ .body("__type", endsWith("ResourceNotFoundException"));
+ }
+
+ @Test
+ void theStreamReaderReadsTheSameRecordsWithStreamCheckpoints() {
+ String table = tableName("reader");
+ String streamArn = streamedTableWithOneItem(A_WEST, table);
+
+ Stream stream = Stream.of(streamArn);
+ List<Shard> shards = streamReader.shards(stream);
+ Cursor cursor = streamReader.getShardIterator(stream, shards.get(0).shardId(), Position.TRIM_HORIZON, null);
+ RecordsPage page = streamReader.getRecords(cursor, 10);
+
+ assertEquals(1, page.records().size());
+ JsonNode record = page.records().get(0).awsRecord();
+ assertEquals("INSERT", record.path("eventName").asText());
+ assertEquals("eu-west-1", record.path("awsRegion").asText());
+ assertEquals("streamed", record.path("dynamodb").path("Keys").path("pk").path("S").asText());
+ assertEquals(CheckpointLifetime.STREAM, streamReader.checkpointLifetime());
+ }
+
+ @Test
+ void theDirectDeleteRemovesAProtectedTable() {
+ String table = tableName("protected");
+ createTable(A_WEST, table, ", \"DeletionProtectionEnabled\": true").statusCode(200);
+ dynamoDb(A_WEST, "DeleteTable", "{\"TableName\": \"" + table + "\"}").statusCode(400);
+
+ dynamoDbFacade.tables().deleteTable(new Scope(A_WEST.account(), A_WEST.region()), table);
+ created.clear();
+
+ dynamoDb(A_WEST, "DescribeTable", "{\"TableName\": \"" + table + "\"}")
+ .statusCode(400)
+ .body("__type", endsWith("ResourceNotFoundException"));
+ }
+
+ @Test
+ void resetIsRefused() {
+ given().when().post("/_floci/state/reset").then().statusCode(409);
+ }
+
+ @Test
+ void infoReportsTheLocalBackend() {
+ given()
+ .when().get("/_floci/info")
+ .then()
+ .statusCode(200)
+ .body("dynamodb_backend", equalTo("local"));
+ }
+
+ private String streamedTableWithOneItem(Caller caller, String table) {
+ String streamArn = createTable(caller, table, STREAMED)
+ .statusCode(200)
+ .extract().path("TableDescription.LatestStreamArn");
+ putItem(caller, table, "streamed");
+ return streamArn;
+ }
+
+ private ValidatableResponse createTable(Caller caller, String table, String extraFields) {
+ ValidatableResponse response = dynamoDb(caller, "CreateTable", """
+ {
+ "TableName": "%s",
+ "KeySchema": [{"AttributeName": "pk", "KeyType": "HASH"}],
+ "AttributeDefinitions": [{"AttributeName": "pk", "AttributeType": "S"}],
+ "BillingMode": "PAY_PER_REQUEST"%s
+ }
+ """.formatted(table, extraFields));
+ created.add(new CreatedTable(caller, table));
+ return response;
+ }
+
+ private static void putItem(Caller caller, String table, String pk) {
+ dynamoDb(caller, "PutItem", "{\"TableName\": \"" + table + "\", \"Item\": {\"pk\": {\"S\": \"" + pk + "\"}}}")
+ .statusCode(200);
+ }
+
+ private static ValidatableResponse scanCount(Caller caller, String table) {
+ return dynamoDb(caller, "Scan", "{\"TableName\": \"" + table + "\"}").statusCode(200);
+ }
+
+ private static ValidatableResponse dynamoDb(Caller caller, String action, String body) {
+ return send(caller, "DynamoDB_20120810." + action, body);
+ }
+
+ private static ValidatableResponse streams(Caller caller, String action, String body) {
+ return send(caller, "DynamoDBStreams_20120810." + action, body);
+ }
+
+ private static ValidatableResponse send(Caller caller, String target, String body) {
+ return given()
+ .header("X-Amz-Target", target)
+ .header("Authorization", caller.authorization())
+ .contentType(CONTENT_TYPE)
+ .body(body)
+ .when()
+ .post("/")
+ .then();
+ }
+
+ private static String tableArn(Caller caller, String table) {
+ return "arn:aws:dynamodb:" + caller.region() + ":" + caller.account() + ":table/" + table;
+ }
+
+ private static String tableName(String purpose) {
+ return "local-it-" + purpose + "-" + UUID.randomUUID();
+ }
+
+ private static String randomAccount() {
+ return Long.toString(ThreadLocalRandom.current().nextLong(100_000_000_000L, 1_000_000_000_000L));
+ }
+
+ private static String otherAccountThan(String account) {
+ String other = randomAccount();
+ while (other.equals(account)) {
+ other = randomAccount();
+ }
+ return other;
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendTest.java
new file mode 100644
index 0000000000..9c2f2dc57d
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalBackendTest.java
@@ -0,0 +1,783 @@
+package io.github.hectorvent.floci.services.dynamodb.backend;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.core.resource.ExplorerResource;
+import io.github.hectorvent.floci.core.storage.AccountAwareStorageBackend;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbItemAccess.ScanPage;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Call;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;
+import io.github.hectorvent.floci.services.dynamodb.model.AttributeDefinition;
+import io.github.hectorvent.floci.services.dynamodb.model.GlobalSecondaryIndex;
+import io.github.hectorvent.floci.services.dynamodb.model.KeySchemaElement;
+import io.github.hectorvent.floci.services.dynamodb.model.LocalSecondaryIndex;
+import io.github.hectorvent.floci.services.dynamodb.model.TableDefinition;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.mockito.ArgumentCaptor;
+import org.mockito.InOrder;
+
+import java.io.IOException;
+import java.net.ConnectException;
+import java.net.URI;
+import java.time.Duration;
+import java.time.Instant;
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+import java.util.Optional;
+import java.util.Set;
+import java.util.stream.Collectors;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.atLeastOnce;
+import static org.mockito.Mockito.inOrder;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
+
+/** The DynamoDB Local engine: identity translation, Floci-held tags, typed operations and lifecycle. */
+class DynamoDbLocalBackendTest {
+
+ private static final String ACCOUNT = "123456789012";
+ private static final String REGION = "eu-west-1";
+ private static final Scope SCOPE = new Scope(ACCOUNT, REGION);
+ private static final String LOCAL = "arn:aws:dynamodb:ddblocal:000000000000:";
+ private static final String PUBLIC = "arn:aws:dynamodb:eu-west-1:123456789012:";
+ private static final String ORDERS_ARN = PUBLIC + "table/orders";
+ private static final String STREAM_RESOURCE = "table/orders/stream/2026-09-27T00:03:07.048";
+ private static final String REPLICAS_UNSUPPORTED = "Replicas are not supported by the DynamoDB Local backend";
+
+ private final ObjectMapper mapper = new ObjectMapper().findAndRegisterModules();
+ private final DynamoDbLocalClient client = mock(DynamoDbLocalClient.class);
+ private final AccountAwareStorageBackend<Map<String, String>> tags =
+ AccountAwareStorageBackend.inMemory("000000000000");
+ private DynamoDbLocalBackend backend;
+
+ @BeforeEach
+ void setUp() {
+ when(client.endpoint()).thenReturn(URI.create("http://local:8000"));
+ backend = new DynamoDbLocalBackend(client, tags, mapper, Duration.ofSeconds(5));
+ }
+
+ private JsonNode json(String text) throws IOException {
+ return mapper.readTree(text);
+ }
+
+ private JsonNode normalized(JsonNode node) throws IOException {
+ return mapper.readTree(mapper.writeValueAsBytes(node));
+ }
+
+ private void answer(Api api, String action, int status, String body) throws Exception {
+ when(client.send(any(), eq(api), eq(action), any()))
+ .thenAnswer(invocation -> new Reply(status, json(body), Map.of()));
+ }
+
+ private void answer(String action, int status, String body) throws Exception {
+ answer(Api.DYNAMODB, action, status, body);
+ }
+
+ private List<JsonNode> sentBodies(Api api, String action) throws Exception {
+ ArgumentCaptor<JsonNode> body = ArgumentCaptor.forClass(JsonNode.class);
+ verify(client, atLeastOnce()).send(any(), eq(api), eq(action), body.capture());
+ List<JsonNode> bodies = new ArrayList<>();
+ for (JsonNode value : body.getAllValues()) {
+ bodies.add(normalized(value));
+ }
+ return bodies;
+ }
+
+ private JsonNode sent(Api api, String action) throws Exception {
+ List<JsonNode> bodies = sentBodies(api, action);
+ return bodies.get(bodies.size() - 1);
+ }
+
+ private JsonNode sent(String action) throws Exception {
+ return sent(Api.DYNAMODB, action);
+ }
+
+ private void neverSent(String action) throws Exception {
+ verify(client, never()).send(any(), any(), eq(action), any());
+ }
+
+ private Reply execute(Api api, String action, String body) throws Exception {
+ return backend.execute(new Call(SCOPE, api, action, json(body)));
+ }
+
+ private Reply execute(String action, String body) throws Exception {
+ return execute(Api.DYNAMODB, action, body);
+ }
+
+ private void answerOrdersTable() throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL
+ + "table/orders\",\"TableStatus\":\"ACTIVE\"}}");
+ }
+
+ @Test
+ void tableArnTableNameBecomesShortName() throws Exception {
+ answer("GetItem", 200, "{}");
+
+ execute("GetItem", "{\"TableName\":\"" + ORDERS_ARN + "\",\"Key\":{\"pk\":{\"S\":\"a\"}}}");
+
+ assertEquals(json("{\"TableName\":\"orders\",\"Key\":{\"pk\":{\"S\":\"a\"}}}"), sent("GetItem"));
+ }
+
+ @Test
+ void requestItemsArnKeysBecomeShortNames() throws Exception {
+ answer("BatchWriteItem", 200, "{\"UnprocessedItems\":{}}");
+
+ execute("BatchWriteItem", "{\"RequestItems\":{\"" + ORDERS_ARN + "\":[{\"PutRequest\":{\"Item\":"
+ + "{\"pk\":{\"S\":\"a\"}}}}],\"users\":[{\"DeleteRequest\":{\"Key\":{\"pk\":{\"S\":\"b\"}}}}]}}");
+
+ JsonNode requestItems = sent("BatchWriteItem").path("RequestItems");
+ List<String> names = new ArrayList<>();
+ requestItems.fieldNames().forEachRemaining(names::add);
+ assertEquals(List.of("orders", "users"), names);
+ assertEquals(json("[{\"PutRequest\":{\"Item\":{\"pk\":{\"S\":\"a\"}}}}]"), requestItems.path("orders"));
+ }
+
+ @Test
+ void transactItemsNestedTableNamesTranslated() throws Exception {
+ answer("TransactWriteItems", 200, "{}");
+
+ execute("TransactWriteItems", "{\"TransactItems\":["
+ + "{\"Put\":{\"TableName\":\"" + ORDERS_ARN + "\",\"Item\":{\"pk\":{\"S\":\"a\"}}}},"
+ + "{\"ConditionCheck\":{\"TableName\":\"" + PUBLIC + "table/users\",\"Key\":{\"pk\":{\"S\":\"b\"}},"
+ + "\"ConditionExpression\":\"attribute_exists(pk)\"}},"
+ + "{\"Delete\":{\"TableName\":\"audit\",\"Key\":{\"pk\":{\"S\":\"c\"}}}}]}");
+
+ JsonNode items = sent("TransactWriteItems").path("TransactItems");
+ assertEquals("orders", items.path(0).path("Put").path("TableName").asText());
+ assertEquals("users", items.path(1).path("ConditionCheck").path("TableName").asText());
+ assertEquals("audit", items.path(2).path("Delete").path("TableName").asText());
+ }
+
+ @Test
+ void foreignAccountOrPartitionArnNotFound() throws Exception {
+ for (String foreign : List.of("arn:aws:dynamodb:eu-west-1:999999999999:table/orders",
+ "arn:aws-cn:dynamodb:eu-west-1:123456789012:table/orders")) {
+ AwsException error = assertThrows(AwsException.class,
+ () -> execute("GetItem", "{\"TableName\":\"" + foreign + "\",\"Key\":{}}"));
+
+ assertEquals("ResourceNotFoundException", error.getErrorCode());
+ assertEquals("Requested resource not found: Table: " + foreign + " not found", error.getMessage());
+ }
+ neverSent("GetItem");
+ }
+
+ @Test
+ void requestItemsNamingOneTableTwiceRejected() throws Exception {
+ AwsException error = assertThrows(AwsException.class, () -> execute("BatchWriteItem",
+ "{\"RequestItems\":{\"" + ORDERS_ARN + "\":[],\"orders\":[]}}"));
+
+ assertEquals("ValidationException", error.getErrorCode());
+ neverSent("BatchWriteItem");
+ }
+
+ @Test
+ void foreignRegionArnRejected() throws Exception {
+ AwsException error = assertThrows(AwsException.class, () -> execute("GetItem",
+ "{\"TableName\":\"arn:aws:dynamodb:us-east-1:123456789012:table/orders\",\"Key\":{}}"));
+
+ assertEquals("ValidationException", error.getErrorCode());
+ assertEquals("Region 'us-east-1' in ARN does not match request region 'eu-west-1'", error.getMessage());
+ neverSent("GetItem");
+ }
+
+ @Test
+ void createTableArnNameLeftForLocal() throws Exception {
+ answer("CreateTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ValidationException\","
+ + "\"message\":\"Invalid table name\"}");
+
+ Reply reply = execute("CreateTable", "{\"TableName\":\"" + ORDERS_ARN + "\"}");
+
+ assertEquals(ORDERS_ARN, sent("CreateTable").path("TableName").asText());
+ assertEquals(400, reply.status());
+ }
+
+ @Test
+ void streamArnTranslatedToLocalPrefix() throws Exception {
+ answer(Api.DYNAMODB_STREAMS, "DescribeStream", 200, "{\"StreamDescription\":{}}");
+
+ execute(Api.DYNAMODB_STREAMS, "DescribeStream", "{\"StreamArn\":\"" + PUBLIC + STREAM_RESOURCE + "\"}");
+
+ assertEquals(LOCAL + STREAM_RESOURCE, sent(Api.DYNAMODB_STREAMS, "DescribeStream").path("StreamArn").asText());
+ }
+
+ @Test
+ void listStreamsPagesWithPublicStreamArns() throws Exception {
+ answer(Api.DYNAMODB_STREAMS, "ListStreams", 200,
+ "{\"LastEvaluatedStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}");
+
+ Reply reply = execute(Api.DYNAMODB_STREAMS, "ListStreams",
+ "{\"ExclusiveStartStreamArn\":\"" + PUBLIC + STREAM_RESOURCE + "\"}");
+
+ assertEquals(LOCAL + STREAM_RESOURCE,
+ sent(Api.DYNAMODB_STREAMS, "ListStreams").path("ExclusiveStartStreamArn").asText());
+ assertEquals(PUBLIC + STREAM_RESOURCE, reply.body().path("LastEvaluatedStreamArn").asText());
+ }
+
+ @Test
+ void foreignStreamArnNotFound() throws Exception {
+ for (String foreign : List.of("arn:aws:dynamodb:eu-west-1:999999999999:" + STREAM_RESOURCE,
+ "arn:aws:dynamodb:us-east-1:123456789012:" + STREAM_RESOURCE,
+ "arn:aws-cn:dynamodb:eu-west-1:123456789012:" + STREAM_RESOURCE)) {
+ AwsException error = assertThrows(AwsException.class,
+ () -> execute(Api.DYNAMODB_STREAMS, "DescribeStream", "{\"StreamArn\":\"" + foreign + "\"}"));
+
+ assertEquals("ResourceNotFoundException", error.getErrorCode());
+ assertEquals("Requested resource not found: Stream: " + foreign + " not found", error.getMessage());
+ assertEquals(400, error.getHttpStatus());
+ }
+ neverSent("DescribeStream");
+ }
+
+ @Test
+ void publicArnsInReply() throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL + "table/orders\","
+ + "\"GlobalSecondaryIndexes\":[{\"IndexName\":\"g\","
+ + "\"IndexArn\":\"" + LOCAL + "table/orders/index/g\"}],"
+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");
+
+ JsonNode table = execute("DescribeTable", "{\"TableName\":\"orders\"}").body().path("Table");
+
+ assertEquals(ORDERS_ARN, table.path("TableArn").asText());
+ assertEquals(ORDERS_ARN + "/index/g", table.path("GlobalSecondaryIndexes").path(0).path("IndexArn").asText());
+ assertEquals(PUBLIC + STREAM_RESOURCE, table.path("LatestStreamArn").asText());
+ }
+
+ @Test
+ void errorMessageArnRewritten() throws Exception {
+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","
+ + "\"message\":\"Requested resource not found: Table: " + LOCAL + "table/orders not found\"}");
+
+ Reply reply = execute("DescribeTable", "{\"TableName\":\"orders\"}");
+
+ assertEquals("Requested resource not found: Table: " + ORDERS_ARN + " not found",
+ reply.body().path("message").asText());
+ }
+
+ @Test
+ void recordAwsRegionRewritten() throws Exception {
+ answer(Api.DYNAMODB_STREAMS, "GetRecords", 200,
+ "{\"Records\":[{\"eventName\":\"INSERT\",\"awsRegion\":\"ddblocal\",\"dynamodb\":{}}]}");
+
+ Reply reply = execute(Api.DYNAMODB_STREAMS, "GetRecords", "{\"ShardIterator\":\"it\"}");
+
+ assertEquals(REGION, reply.body().path("Records").path(0).path("awsRegion").asText());
+ }
+
+ @Test
+ void arnLookingItemAttributeUntouched() throws Exception {
+ String item = "{\"TableArn\":{\"S\":\"" + LOCAL + "table/x\"},"
+ + "\"StreamArn\":{\"S\":\"arn:aws:dynamodb:us-east-1:999999999999:" + STREAM_RESOURCE + "\"},"
+ + "\"TableName\":{\"S\":\"" + PUBLIC + "table/other\"},"
+ + "\"awsRegion\":{\"S\":\"ddblocal\"}}";
+ answer("PutItem", 200, "{}");
+ answer("GetItem", 200, "{\"Item\":" + item + "}");
+
+ execute("PutItem", "{\"TableName\":\"orders\",\"Item\":" + item + "}");
+ Reply reply = execute("GetItem", "{\"TableName\":\"orders\",\"Key\":{\"pk\":{\"S\":\"a\"}}}");
+
+ assertEquals(json(item), sent("PutItem").path("Item"));
+ assertEquals(json(item), reply.body().path("Item"));
+ }
+
+ @Test
+ void callerBodyNotMutated() throws Exception {
+ answer("GetItem", 200, "{}");
+ JsonNode body = json("{\"TableName\":\"" + ORDERS_ARN + "\",\"Key\":{\"pk\":{\"S\":\"a\"}}}");
+ JsonNode original = body.deepCopy();
+
+ backend.execute(new Call(SCOPE, Api.DYNAMODB, "GetItem", body));
+
+ assertEquals(original, body);
+ }
+
+ @Test
+ void sendsUnderCallerScope() throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{}}");
+ Scope caller = new Scope("210987654321", "ap-south-1");
+
+ backend.execute(new Call(caller, Api.DYNAMODB, "DescribeTable", json("{\"TableName\":\"orders\"}")));
+
+ ArgumentCaptor<Scope> scope = ArgumentCaptor.forClass(Scope.class);
+ verify(client).send(scope.capture(), eq(Api.DYNAMODB), eq("DescribeTable"), any());
+ assertEquals(caller, scope.getValue());
+ }
+
+ @Test
+ void ioFailureIsInternalServerError() throws Exception {
+ when(client.send(any(), any(), any(), any())).thenThrow(new IOException("boom"));
+
+ AwsException error = assertThrows(AwsException.class,
+ () -> execute("DescribeTable", "{\"TableName\":\"orders\"}"));
+
+ assertEquals("InternalServerError", error.getErrorCode());
+ assertEquals(500, error.getHttpStatus());
+ assertEquals("DynamoDB Local at http://local:8000 did not answer: java.io.IOException: boom",
+ error.getMessage());
+ }
+
+ @Test
+ void untaggedTableListsNoTags() throws Exception {
+ answerOrdersTable();
+
+ assertEquals(Map.of(), backend.listTagsOfResource(SCOPE, ORDERS_ARN));
+ }
+
+ @Test
+ void tagAbsentTableFails() throws Exception {
+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","
+ + "\"message\":\"Requested resource not found\"}");
+
+ AwsException error = assertThrows(AwsException.class,
+ () -> backend.tagResource(SCOPE, ORDERS_ARN, Map.of("env", "dev")));
+
+ assertEquals("ResourceNotFoundException", error.getErrorCode());
+ assertTrue(tags.keysForAccount(ACCOUNT).isEmpty());
+ }
+
+ @Test
+ void wireListTagsOfAMissingTableIsAccessDenied() throws Exception {
+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","
+ + "\"message\":\"Requested resource not found\"}");
+
+ AwsException error = assertThrows(AwsException.class,
+ () -> execute("ListTagsOfResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\"}"));
+
+ assertEquals("AccessDeniedException", error.getErrorCode());
+ assertEquals("User is not authorized to perform: dynamodb:ListTagsOfResource on resource: " + ORDERS_ARN,
+ error.getMessage());
+ assertEquals(400, error.getHttpStatus());
+ }
+
+ @Test
+ void tagArnOfAnUnknownPartitionIsValidation() throws Exception {
+ String arn = "arn:notaws:dynamodb:eu-west-1:123456789012:table/orders";
+
+ AwsException error = assertThrows(AwsException.class,
+ () -> execute("TagResource", "{\"ResourceArn\":\"" + arn + "\",\"Tags\":[]}"));
+
+ assertEquals("ValidationException", error.getErrorCode());
+ neverSent("DescribeTable");
+ }
+
+ @Test
+ void tagArnOfAnotherRegionOrPartitionIsNotFound() throws Exception {
+ for (String foreign : List.of("arn:aws:dynamodb:us-east-1:123456789012:table/orders",
+ "arn:aws-cn:dynamodb:cn-north-1:123456789012:table/orders")) {
+ AwsException error = assertThrows(AwsException.class,
+ () -> execute("UntagResource", "{\"ResourceArn\":\"" + foreign + "\",\"TagKeys\":[]}"));
+
+ assertEquals("ResourceNotFoundException", error.getErrorCode());
+ }
+ neverSent("DescribeTable");
+ }
+
+ @Test
+ void createTableRecordsRequestTags() throws Exception {
+ answer("CreateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL
+ + "table/orders\"}}");
+ answerOrdersTable();
+
+ execute("CreateTable", "{\"TableName\":\"orders\",\"Tags\":[{\"Key\":\"env\",\"Value\":\"dev\"}]}");
+
+ assertEquals(Map.of("env", "dev"), backend.listTagsOfResource(SCOPE, ORDERS_ARN));
+ }
+
+ @Test
+ void deleteTableDropsTags() throws Exception {
+ tags.putForAccount(ACCOUNT, REGION + "/orders", Map.of("env", "dev"));
+ answer("DeleteTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\",\"TableStatus\":\"DELETING\"}}");
+
+ execute("DeleteTable", "{\"TableName\":\"" + ORDERS_ARN + "\"}");
+
+ assertTrue(tags.getForAccount(ACCOUNT, REGION + "/orders").isEmpty());
+ }
+
+ @Test
+ void directDeleteLiftsDeletionProtectionFirst() throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"DeletionProtectionEnabled\":true}}");
+ answer("UpdateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");
+ answer("DeleteTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");
+
+ backend.deleteTable(SCOPE, "orders");
+
+ InOrder order = inOrder(client);
+ order.verify(client).send(any(), eq(Api.DYNAMODB), eq("UpdateTable"),
+ eq(json("{\"TableName\":\"orders\",\"DeletionProtectionEnabled\":false}")));
+ order.verify(client).send(any(), eq(Api.DYNAMODB), eq("DeleteTable"), eq(json("{\"TableName\":\"orders\"}")));
+ }
+
+ @Test
+ void directDeleteOfAnUnprotectedTableSendsOnlyDeleteTable() throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"DeletionProtectionEnabled\":false}}");
+ answer("DeleteTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");
+
+ backend.deleteTable(SCOPE, "orders");
+
+ neverSent("UpdateTable");
+ assertEquals(json("{\"TableName\":\"orders\"}"), sent("DeleteTable"));
+ }
+
+ @Test
+ void wireTagRoundTrip() throws Exception {
+ answerOrdersTable();
+
+ Reply tagged = execute("TagResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\",\"Tags\":["
+ + "{\"Key\":\"env\",\"Value\":\"dev\"},{\"Key\":\"team\",\"Value\":\"a\"}]}");
+ JsonNode listed = execute("ListTagsOfResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\"}").body();
+ Reply untagged = execute("UntagResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\",\"TagKeys\":[\"env\"]}");
+ JsonNode remaining = execute("ListTagsOfResource", "{\"ResourceArn\":\"" + ORDERS_ARN + "\"}").body();
+
+ assertEquals(200, tagged.status());
+ assertEquals(json("{}"), tagged.body());
+ assertEquals(json("{\"Tags\":[{\"Key\":\"env\",\"Value\":\"dev\"},"
+ + "{\"Key\":\"team\",\"Value\":\"a\"}]}"), listed);
+ assertEquals(200, untagged.status());
+ assertEquals(json("{\"Tags\":[{\"Key\":\"team\",\"Value\":\"a\"}]}"), remaining);
+ neverSent("TagResource");
+ neverSent("UntagResource");
+ neverSent("ListTagsOfResource");
+ }
+
+ @Test
+ void replicaUpdatesRejected() throws Exception {
+ AwsException wire = assertThrows(AwsException.class, () -> execute("UpdateTable",
+ "{\"TableName\":\"orders\",\"ReplicaUpdates\":[{\"Create\":{\"RegionName\":\"us-east-1\"}}]}"));
+ AwsException typed = assertThrows(AwsException.class,
+ () -> backend.applyReplicaUpdates(SCOPE, "orders", List.of("us-east-1"), List.of()));
+
+ for (AwsException error : List.of(wire, typed)) {
+ assertEquals("ValidationException", error.getErrorCode());
+ assertEquals(REPLICAS_UNSUPPORTED, error.getMessage());
+ assertEquals(400, error.getHttpStatus());
+ }
+ neverSent("UpdateTable");
+ }
+
+ @Test
+ void ensureGlobalTableDescribes() throws Exception {
+ answerOrdersTable();
+
+ TableDefinition table = backend.ensureGlobalTable(SCOPE, "orders");
+
+ assertEquals("orders", table.getTableName());
+ assertEquals(ORDERS_ARN, table.getTableArn());
+ assertEquals("", table.getTableId());
+ }
+
+ @Test
+ void createTableTypedProvisionedJson() throws Exception {
+ answer("CreateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL
+ + "table/orders\"}}");
+
+ TableDefinition table = backend.createTable(SCOPE, "orders",
+ List.of(new KeySchemaElement("pk", "HASH"), new KeySchemaElement("sk", "RANGE")),
+ List.of(new AttributeDefinition("pk", "S"), new AttributeDefinition("sk", "S"),
+ new AttributeDefinition("g", "S"), new AttributeDefinition("l", "N")),
+ 3L, 4L,
+ List.of(new GlobalSecondaryIndex("gsi1", List.of(new KeySchemaElement("g", "HASH")), null,
+ "INCLUDE", List.of("a"))),
+ List.of(new LocalSecondaryIndex("lsi1", List.of(new KeySchemaElement("pk", "HASH"),
+ new KeySchemaElement("l", "RANGE")), null, "KEYS_ONLY")));
+
+ assertEquals(json("""
+ {"TableName":"orders",
+ "KeySchema":[{"AttributeName":"pk","KeyType":"HASH"},{"AttributeName":"sk","KeyType":"RANGE"}],
+ "AttributeDefinitions":[{"AttributeName":"pk","AttributeType":"S"},
+ {"AttributeName":"sk","AttributeType":"S"},{"AttributeName":"g","AttributeType":"S"},
+ {"AttributeName":"l","AttributeType":"N"}],
+ "BillingMode":"PROVISIONED",
+ "ProvisionedThroughput":{"ReadCapacityUnits":3,"WriteCapacityUnits":4},
+ "GlobalSecondaryIndexes":[{"IndexName":"gsi1","KeySchema":[{"AttributeName":"g","KeyType":"HASH"}],
+ "Projection":{"ProjectionType":"INCLUDE","NonKeyAttributes":["a"]},
+ "ProvisionedThroughput":{"ReadCapacityUnits":3,"WriteCapacityUnits":4}}],
+ "LocalSecondaryIndexes":[{"IndexName":"lsi1","KeySchema":[{"AttributeName":"pk","KeyType":"HASH"},
+ {"AttributeName":"l","KeyType":"RANGE"}],"Projection":{"ProjectionType":"KEYS_ONLY"}}]}
+ """), sent("CreateTable"));
+ assertEquals(ORDERS_ARN, table.getTableArn());
+ assertEquals(List.of(REGION + "/orders"), List.copyOf(tags.keysForAccount(ACCOUNT)));
+ }
+
+ @Test
+ void createTableTypedOnDemandJson() throws Exception {
+ answer("CreateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\"}}");
+ GlobalSecondaryIndex gsi = new GlobalSecondaryIndex();
+ gsi.setIndexName("gsi1");
+ gsi.setKeySchema(List.of(new KeySchemaElement("g", "HASH")));
+
+ backend.createTable(SCOPE, "orders", List.of(new KeySchemaElement("pk", "HASH")),
+ List.of(new AttributeDefinition("pk", "S"), new AttributeDefinition("g", "S")),
+ null, null, List.of(gsi), List.of());
+
+ assertEquals(json("""
+ {"TableName":"orders",
+ "KeySchema":[{"AttributeName":"pk","KeyType":"HASH"}],
+ "AttributeDefinitions":[{"AttributeName":"pk","AttributeType":"S"},
+ {"AttributeName":"g","AttributeType":"S"}],
+ "BillingMode":"PAY_PER_REQUEST",
+ "GlobalSecondaryIndexes":[{"IndexName":"gsi1","KeySchema":[{"AttributeName":"g","KeyType":"HASH"}],
+ "Projection":{"ProjectionType":"ALL"}}]}
+ """), sent("CreateTable"));
+ }
+
+ @Test
+ void describeTableMapsStreamFields() throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\",\"TableArn\":\"" + LOCAL + "table/orders\","
+ + "\"CreationDateTime\":1.790463342979E9,"
+ + "\"StreamSpecification\":{\"StreamEnabled\":true,\"StreamViewType\":\"KEYS_ONLY\"},"
+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");
+
+ TableDefinition table = backend.describeTable(SCOPE, "orders");
+
+ assertTrue(table.isStreamEnabled());
+ assertEquals("KEYS_ONLY", table.getStreamViewType());
+ assertEquals(PUBLIC + STREAM_RESOURCE, table.getStreamArn());
+ assertEquals(Instant.ofEpochMilli(1790463342979L), table.getCreationDateTime());
+ }
+
+ @Test
+ void findTableAbsentIsEmpty() throws Exception {
+ answer("DescribeTable", 400, "{\"__type\":\"com.amazonaws.dynamodb.v20120810#ResourceNotFoundException\","
+ + "\"message\":\"Requested resource not found\"}");
+
+ assertEquals(Optional.empty(), backend.findTable(SCOPE, "orders"));
+ }
+
+ private void answerStreamingOrdersTable(String viewType) throws Exception {
+ answer("DescribeTable", 200, "{\"Table\":{\"TableName\":\"orders\","
+ + "\"StreamSpecification\":{\"StreamEnabled\":true,\"StreamViewType\":\"" + viewType + "\"},"
+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");
+ }
+
+ @Test
+ void enableStreamIdempotent() throws Exception {
+ answerStreamingOrdersTable("NEW_AND_OLD_IMAGES");
+
+ TableDefinition unspecified = backend.enableStream(SCOPE, "orders", null);
+ TableDefinition same = backend.enableStream(SCOPE, "orders", "NEW_AND_OLD_IMAGES");
+
+ assertEquals(PUBLIC + STREAM_RESOURCE, unspecified.getStreamArn());
+ assertEquals("NEW_AND_OLD_IMAGES", same.getStreamViewType());
+ neverSent("UpdateTable");
+ }
+
+ @Test
+ void enableStreamWithNewViewTypeRestartsStream() throws Exception {
+ answerStreamingOrdersTable("NEW_AND_OLD_IMAGES");
+ answer("UpdateTable", 200, "{\"TableDescription\":{\"TableName\":\"orders\","
+ + "\"StreamSpecification\":{\"StreamEnabled\":true,\"StreamViewType\":\"KEYS_ONLY\"},"
+ + "\"LatestStreamArn\":\"" + LOCAL + STREAM_RESOURCE + "\"}}");
+
+ TableDefinition table = backend.enableStream(SCOPE, "orders", "KEYS_ONLY");
+
+ assertEquals(List.of(
+ json("{\"TableName\":\"orders\",\"StreamSpecification\":{\"StreamEnabled\":false}}"),
+ json("{\"TableName\":\"orders\",\"StreamSpecification\":{\"StreamEnabled\":true,"
+ + "\"StreamViewType\":\"KEYS_ONLY\"}}")),
+ sentBodies(Api.DYNAMODB, "UpdateTable"));
+ assertEquals("KEYS_ONLY", table.getStreamViewType());
+ }
+
+ @Test
+ void disableStreamIdempotent() throws Exception {
+ answerOrdersTable();
+
+ TableDefinition table = backend.disableStream(SCOPE, "orders");
+
+ assertFalse(table.isStreamEnabled());
+ assertNull(table.getStreamArn());
+ neverSent("UpdateTable");
+ }
+
+ @Test
+ void updateItemReturnsOnlyImages() throws Exception {
+ answer("UpdateItem", 200, "{\"Attributes\":{\"pk\":{\"S\":\"a\"},\"n\":{\"N\":\"2\"}}}");
+ JsonNode key = json("{\"pk\":{\"S\":\"a\"}}");
+
+ JsonNode updatedOnly = backend.updateItem(SCOPE, "orders", key, null, "SET n = :n", null,
+ json("{\":n\":{\"N\":\"2\"}}"), "UPDATED_NEW", null);
+ JsonNode allNew = backend.updateItem(SCOPE, "orders", key, null, "SET n = :n", null,
+ json("{\":n\":{\"N\":\"2\"}}"), "ALL_NEW", null);
+
+ assertNull(updatedOnly);
+ assertEquals(json("{\"pk\":{\"S\":\"a\"},\"n\":{\"N\":\"2\"}}"), allNew);
+ assertEquals(json("{\"TableName\":\"orders\",\"Key\":{\"pk\":{\"S\":\"a\"}},"
+ + "\"UpdateExpression\":\"SET n = :n\","
+ + "\"ExpressionAttributeValues\":{\":n\":{\"N\":\"2\"}},\"ReturnValues\":\"ALL_NEW\"}"),
+ sent("UpdateItem"));
+ }
+
+ @Test
+ void scanPage() throws Exception {
+ answer("Scan", 200, "{\"Items\":[{\"pk\":{\"S\":\"a\"}},{\"pk\":{\"S\":\"b\"}}],\"Count\":2,"
+ + "\"ScannedCount\":5,\"LastEvaluatedKey\":{\"pk\":{\"S\":\"b\"}}}");
+
+ ScanPage page = backend.scan(SCOPE, "orders", null, null, null, null, 2, json("{\"pk\":{\"S\":\"0\"}}"));
+
+ assertEquals(List.of(json("{\"pk\":{\"S\":\"a\"}}"), json("{\"pk\":{\"S\":\"b\"}}")), page.items());
+ assertEquals(5, page.scannedCount());
+ assertEquals(json("{\"pk\":{\"S\":\"b\"}}"), page.lastEvaluatedKey());
+ assertEquals(json("{\"TableName\":\"orders\",\"Limit\":2,\"ExclusiveStartKey\":{\"pk\":{\"S\":\"0\"}}}"),
+ sent("Scan"));
+ }
+
+ @Test
+ void resourcesListsTablesOfKnownRegions() throws Exception {
+ tags.putForAccount(ACCOUNT, REGION + "/orders", Map.of("env", "dev"));
+ tags.putForAccount(ACCOUNT, "us-east-1/users", Map.of());
+ when(client.send(any(), eq(Api.DYNAMODB), eq("ListTables"), any())).thenAnswer(invocation -> {
+ Scope scope = invocation.getArgument(0);
+ JsonNode body = invocation.getArgument(3);
+ String page;
+ if ("us-east-1".equals(scope.region())) {
+ page = "{\"TableNames\":[\"users\"]}";
+ } else if (body.has("ExclusiveStartTableName")) {
+ page = "{\"TableNames\":[\"direct\"]}";
+ } else {
+ page = "{\"TableNames\":[\"orders\"],\"LastEvaluatedTableName\":\"orders\"}";
+ }
+ return new Reply(200, json(page), Map.of());
+ });
+
+ List<ExplorerResource> resources = backend.resources(ACCOUNT);
+
+ Map<String, ExplorerResource> byArn = resources.stream()
+ .collect(Collectors.toMap(ExplorerResource::arn, resource -> resource));
+ assertEquals(Set.of(ORDERS_ARN, PUBLIC + "table/direct", "arn:aws:dynamodb:us-east-1:123456789012:table/users"),
+ byArn.keySet());
+ ExplorerResource orders = byArn.get(ORDERS_ARN);
+ assertEquals("dynamodb:table", orders.resourceType());
+ assertEquals(REGION, orders.region());
+ assertEquals(ACCOUNT, orders.owningAccountId());
+ assertEquals(Map.of("env", "dev"), orders.tags());
+ assertEquals(Map.of(), byArn.get(PUBLIC + "table/direct").tags());
+ assertTrue(sentBodies(Api.DYNAMODB, "ListTables").contains(json("{\"ExclusiveStartTableName\":\"orders\"}")));
+ }
+
+ @Test
+ void resetRefusedWith409() {
+ AwsException error = assertThrows(AwsException.class, backend::checkReset);
+
+ assertEquals("UnsupportedOperation", error.getErrorCode());
+ assertEquals(409, error.getHttpStatus());
+ }
+
+ @Test
+ void startRetriesConnectionFailuresThenSucceeds() throws Exception {
+ Scope probe = new Scope("000000000000", "us-east-1");
+ when(client.send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), any(), any()))
+ .thenThrow(new ConnectException("refused"))
+ .thenReturn(new Reply(200, json("{\"TableNames\":[]}"), Map.of()));
+
+ backend.start();
+
+ verify(client, times(2)).send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), eq(json("{\"Limit\":1}")), any());
+ }
+
+ @Test
+ void startRetriesAConnectionClosedWithoutAnswer() throws Exception {
+ Scope probe = new Scope("000000000000", "us-east-1");
+ when(client.send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), any(), any()))
+ .thenThrow(new IOException("HTTP/1.1 header parser received no bytes"))
+ .thenReturn(new Reply(200, json("{\"TableNames\":[]}"), Map.of()));
+
+ backend.start();
+
+ verify(client, times(2)).send(eq(probe), eq(Api.DYNAMODB), eq("ListTables"), any(), any());
+ }
+
+ @Test
+ void startCapsEachProbeAtTheRemainingBudget() throws Exception {
+ ArgumentCaptor<Duration> budget = ArgumentCaptor.forClass(Duration.class);
+ when(client.send(any(), any(), any(), any(), budget.capture()))
+ .thenReturn(new Reply(200, json("{\"TableNames\":[]}"), Map.of()));
+
+ backend.start();
+
+ assertTrue(budget.getValue().isPositive());
+ assertTrue(budget.getValue().compareTo(Duration.ofSeconds(5)) <= 0);
+ }
+
+ @Test
+ void startFailsWhenBudgetExpires() throws Exception {
+ backend = new DynamoDbLocalBackend(client, tags, mapper, Duration.ZERO);
+ when(client.send(any(), any(), any(), any(), any())).thenThrow(new ConnectException("refused"));
+
+ IllegalStateException error = assertThrows(IllegalStateException.class, backend::start);
+
+ assertEquals("DynamoDB Local at http://local:8000 is not reachable after 0s", error.getMessage());
+ }
+
+ @Test
+ void startFailsOnNon2xx() throws Exception {
+ when(client.send(any(), any(), any(), any(), any())).thenReturn(new Reply(500, json("{\"message\":\"x\"}"), Map.of()));
+
+ IllegalStateException error = assertThrows(IllegalStateException.class, backend::start);
+
+ assertEquals("DynamoDB Local at http://local:8000 answered ListTables with status 500: {\"message\":\"x\"}",
+ error.getMessage());
+ }
+
+ @Test
+ void validatedEndpointMissing() {
+ for (Optional<String> missing : List.of(Optional.<String>empty(), Optional.of(" "))) {
+ IllegalStateException error = assertThrows(IllegalStateException.class,
+ () -> DynamoDbLocalBackend.validatedEndpoint(missing));
+
+ assertEquals("floci.services.dynamodb.local-endpoint is required when floci.services.dynamodb.backend "
+ + "is local", error.getMessage());
+ }
+ }
+
+ @Test
+ void validatedEndpointFtpScheme() {
+ IllegalStateException error = assertThrows(IllegalStateException.class,
+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of("ftp://localhost:8000")));
+
+ assertTrue(error.getMessage().contains("ftp://localhost:8000"), error.getMessage());
+ }
+
+ @Test
+ void validatedEndpointAwsHost() {
+ IllegalStateException error = assertThrows(IllegalStateException.class,
+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of("https://dynamodb.us-east-1.amazonaws.com")));
+
+ assertTrue(error.getMessage().endsWith("must point at DynamoDB Local, not an AWS endpoint"),
+ error.getMessage());
+ }
+
+ @Test
+ void validatedEndpointAwsHostInAnyPartitionOrWithTrailingDot() {
+ for (String endpoint : List.of("https://dynamodb.us-east-1.amazonaws.com.",
+ "https://dynamodb.cn-north-1.amazonaws.com.cn", "https://dynamodb.eu-isoe-west-1.cloud.adc-e.uk")) {
+ assertThrows(IllegalStateException.class,
+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of(endpoint)), endpoint);
+ }
+ }
+
+ @Test
+ void validatedEndpointUserInfoRejected() {
+ assertThrows(IllegalStateException.class,
+ () -> DynamoDbLocalBackend.validatedEndpoint(Optional.of("http://user:secret@localhost:8000")));
+ }
+
+ @Test
+ void validatedEndpointValid() {
+ assertEquals(URI.create("http://localhost:8000"),
+ DynamoDbLocalBackend.validatedEndpoint(Optional.of("http://localhost:8000")));
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClientTest.java b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClientTest.java
new file mode 100644
index 0000000000..41a9f63c5e
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/dynamodb/backend/DynamoDbLocalClientTest.java
@@ -0,0 +1,247 @@
+package io.github.hectorvent.floci.services.dynamodb.backend;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.sun.net.httpserver.Headers;
+import com.sun.net.httpserver.HttpExchange;
+import com.sun.net.httpserver.HttpServer;
+import io.github.hectorvent.floci.core.common.auth.SigV4RequestValidator;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Api;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Reply;
+import io.github.hectorvent.floci.services.dynamodb.backend.DynamoDbOperations.Scope;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.IOException;
+import java.io.OutputStream;
+import java.net.ConnectException;
+import java.net.InetAddress;
+import java.net.InetSocketAddress;
+import java.net.ServerSocket;
+import java.net.URI;
+import java.net.http.HttpTimeoutException;
+import java.nio.charset.StandardCharsets;
+import java.time.Duration;
+import java.util.concurrent.atomic.AtomicInteger;
+import java.util.concurrent.atomic.AtomicReference;
+import java.util.regex.Matcher;
+import java.util.regex.Pattern;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+/** The signed HTTP client that forwards DynamoDB wire calls to an external DynamoDB Local. */
+class DynamoDbLocalClientTest {
+
+ private static final Scope SCOPE = new Scope("123456789012", "eu-west-1");
+ private static final Pattern AUTHORIZATION = Pattern.compile(
+ "AWS4-HMAC-SHA256 Credential=([^/]+)/(\\d{8})/([^/]+)/dynamodb/aws4_request, "
+ + "SignedHeaders=content-type;host;x-amz-date;x-amz-target, Signature=([0-9a-f]{64})");
+
+ private final ObjectMapper mapper = new ObjectMapper();
+ private final AtomicReference<Received> received = new AtomicReference<>();
+ private final AtomicInteger hits = new AtomicInteger();
+ private HttpServer server;
+ private DynamoDbLocalClient client;
+
+ private record Received(String path, Headers headers, byte[] body) {
+ }
+
+ @FunctionalInterface
+ private interface Responder {
+ void respond(HttpExchange exchange) throws Exception;
+ }
+
+ @AfterEach
+ void tearDown() {
+ if (client != null) {
+ client.close();
+ }
+ if (server != null) {
+ server.stop(0);
+ }
+ }
+
+ @Test
+ void signsAndForwardsExactBody() throws Exception {
+ start(exchange -> reply(exchange, 200, "{}"));
+ ObjectNode body = (ObjectNode) mapper.readTree("""
+ {"TableName":"orders","Item":{
+ "pk":{"S":"a"},
+ "blob":{"B":"AAEC"},
+ "blobs":{"BS":["AAEC","AwQF"]},
+ "numbers":{"NS":["1","2.5"]},
+ "big":{"N":"12345678901234567890123456789012345678"},
+ "nested":{"M":{"inner":{"B":"BgcI"}}}}}
+ """);
+
+ Reply result = client.send(SCOPE, Api.DYNAMODB, "PutItem", body);
+
+ assertEquals(200, result.status());
+ Received request = received.get();
+ assertArrayEquals(mapper.writeValueAsBytes(body), request.body());
+ assertEquals("DynamoDB_20120810.PutItem", request.headers().getFirst("X-Amz-Target"));
+ assertEquals("application/x-amz-json-1.0", request.headers().getFirst("Content-Type"));
+ String amzDate = request.headers().getFirst("X-Amz-Date");
+ assertNotNull(amzDate);
+ String date = amzDate.substring(0, 8);
+ String authorization = request.headers().getFirst("Authorization");
+ assertTrue(authorization.contains(
+ "Credential=floci123456789012/" + date + "/eu-west-1/dynamodb/aws4_request"),
+ authorization);
+ Matcher matcher = AUTHORIZATION.matcher(authorization);
+ assertTrue(matcher.matches(), authorization);
+ assertEquals(expectedSignature(request, date), matcher.group(4));
+ }
+
+ @Test
+ void streamsTargetPrefix() throws Exception {
+ start(exchange -> reply(exchange, 200, "{}"));
+
+ client.send(SCOPE, Api.DYNAMODB_STREAMS, "GetRecords", mapper.createObjectNode());
+
+ assertEquals("DynamoDBStreams_20120810.GetRecords", received.get().headers().getFirst("X-Amz-Target"));
+ }
+
+ @Test
+ void errorBodyPreserved() throws Exception {
+ String error = "{\"__type\":\"com.amazonaws.dynamodb.v20120810#TransactionCanceledException\","
+ + "\"message\":\"x\",\"CancellationReasons\":[{\"Code\":\"ConditionalCheckFailed\"}]}";
+ start(exchange -> reply(exchange, 400, error));
+
+ Reply result = client.send(SCOPE, Api.DYNAMODB, "TransactWriteItems", mapper.createObjectNode());
+
+ assertEquals(400, result.status());
+ assertEquals(mapper.readTree(error), result.body());
+ }
+
+ @Test
+ void unknownOperationPassesThrough() throws Exception {
+ start(exchange -> reply(exchange, 200, "{}"));
+
+ client.send(SCOPE, Api.DYNAMODB, "SomeFutureAction", mapper.createObjectNode());
+
+ assertEquals("DynamoDB_20120810.SomeFutureAction", received.get().headers().getFirst("X-Amz-Target"));
+ }
+
+ @Test
+ void nullBodySendsEmptyObject() throws Exception {
+ start(exchange -> reply(exchange, 200, "{}"));
+
+ client.send(SCOPE, Api.DYNAMODB, "ListTables", null);
+
+ assertEquals("{}", new String(received.get().body(), StandardCharsets.UTF_8));
+ }
+
+ @Test
+ void emptyResponseBodyIsNull() throws Exception {
+ start(exchange -> reply(exchange, 200, ""));
+
+ Reply result = client.send(SCOPE, Api.DYNAMODB, "ListTables", mapper.createObjectNode());
+
+ assertEquals(200, result.status());
+ assertNull(result.body());
+ }
+
+ @Test
+ void connectionRefused() throws Exception {
+ int port;
+ try (ServerSocket socket = new ServerSocket(0, 1, InetAddress.getByName("127.0.0.1"))) {
+ port = socket.getLocalPort();
+ }
+ client = new DynamoDbLocalClient(URI.create("http://127.0.0.1:" + port),
+ Duration.ofSeconds(2), Duration.ofSeconds(2), mapper);
+
+ assertThrows(ConnectException.class,
+ () -> client.send(SCOPE, Api.DYNAMODB, "ListTables", mapper.createObjectNode()));
+ }
+
+ @Test
+ void timedOutRequestIsSentOnce() throws Exception {
+ start(exchange -> {
+ hits.incrementAndGet();
+ Thread.sleep(2000);
+ reply(exchange, 200, "{}");
+ }, Duration.ofMillis(200));
+
+ assertThrows(HttpTimeoutException.class,
+ () -> client.send(SCOPE, Api.DYNAMODB, "ListTables", mapper.createObjectNode()));
+ Thread.sleep(500);
+ assertEquals(1, hits.get());
+ }
+
+ @Test
+ void aShorterBudgetCutsTheRequestTimeout() throws Exception {
+ start(exchange -> {
+ Thread.sleep(2000);
+ reply(exchange, 200, "{}");
+ }, Duration.ofSeconds(5));
+
+ assertThrows(HttpTimeoutException.class, () -> client.send(SCOPE, Api.DYNAMODB, "ListTables",
+ mapper.createObjectNode(), Duration.ofMillis(200)));
+ }
+
+ @Test
+ void aLongerBudgetKeepsTheRequestTimeout() throws Exception {
+ start(exchange -> {
+ Thread.sleep(2000);
+ reply(exchange, 200, "{}");
+ }, Duration.ofMillis(200));
+
+ assertThrows(HttpTimeoutException.class, () -> client.send(SCOPE, Api.DYNAMODB, "ListTables",
+ mapper.createObjectNode(), Duration.ofSeconds(5)));
+ }
+
+ private void start(Responder responder) throws IOException {
+ start(responder, Duration.ofSeconds(5));
+ }
+
+ private void start(Responder responder, Duration requestTimeout) throws IOException {
+ server = HttpServer.create(new InetSocketAddress("127.0.0.1", 0), 0);
+ server.createContext("/", exchange -> {
+ try (exchange) {
+ received.set(new Received(exchange.getRequestURI().getRawPath(),
+ exchange.getRequestHeaders(), exchange.getRequestBody().readAllBytes()));
+ responder.respond(exchange);
+ } catch (Exception e) {
+ throw new IOException(e);
+ }
+ });
+ server.start();
+ URI endpoint = URI.create("http://127.0.0.1:" + server.getAddress().getPort());
+ client = new DynamoDbLocalClient(endpoint, Duration.ofSeconds(2), requestTimeout, mapper);
+ }
+
+ private static void reply(HttpExchange exchange, int status, String body) throws IOException {
+ byte[] bytes = body.getBytes(StandardCharsets.UTF_8);
+ exchange.getResponseHeaders().set("Content-Type", "application/x-amz-json-1.0");
+ exchange.sendResponseHeaders(status, bytes.length == 0 ? -1 : bytes.length);
+ if (bytes.length > 0) {
+ try (OutputStream out = exchange.getResponseBody()) {
+ out.write(bytes);
+ }
+ }
+ }
+
+ private String expectedSignature(Received request, String date) throws Exception {
+ Headers headers = request.headers();
+ String amzDate = headers.getFirst("X-Amz-Date");
+ String canonical = "POST\n" + request.path() + "\n\n"
+ + "content-type:" + headers.getFirst("Content-Type") + "\n"
+ + "host:" + headers.getFirst("Host") + "\n"
+ + "x-amz-date:" + amzDate + "\n"
+ + "x-amz-target:" + headers.getFirst("X-Amz-Target") + "\n\n"
+ + "content-type;host;x-amz-date;x-amz-target\n"
+ + SigV4RequestValidator.sha256Hex(request.body());
+ String credentialScope = date + "/eu-west-1/dynamodb/aws4_request";
+ String toSign = "AWS4-HMAC-SHA256\n" + amzDate + "\n" + credentialScope + "\n"
+ + SigV4RequestValidator.sha256Hex(canonical);
+ byte[] key = SigV4RequestValidator.deriveSigningKey("floci", date, "eu-west-1", "dynamodb");
+ return SigV4RequestValidator.hexEncode(SigV4RequestValidator.hmacSha256(key, toSign));
+ }
+}
diff --git a/src/test/resources/application.yml b/src/test/resources/application.yml
index 01faad0b02..cf67c9d5d6 100644
--- a/src/test/resources/application.yml
+++ b/src/test/resources/application.yml
@@ -141,6 +141,7 @@ floci:
enabled: true
vector-index-allocation-seconds: 0
vector-index-backfill-seconds: 0
+ backend: native
sns:
enabled: true
lambda: