Light, fluffy, and always free - The AWS Local Emulator alternative
fix(rds-data): return typed array result fields (#5665)
Aniket Shuklabb06acdeb11b
diff --git a/docs/services/rds-data.md b/docs/services/rds-data.mdindex 2fb3993fb4..2a3b0be343 100644--- a/docs/services/rds-data.md+++ b/docs/services/rds-data.md@@ -37,8 +37,8 @@ The deprecated `ExecuteSql` operation is recognized at `POST /ExecuteSql` and re- Requests sharing a `transactionId` do not queue behind an active request: `ExecuteStatement`, `BatchExecuteStatement`, `CommitTransaction`, and `RollbackTransaction` immediately return HTTP 400 `DatabaseErrorException` with `Transaction is still running a query`. Retry after the active request finishes. Other transactions remain independent, and expiry cleanup skips active requests. The idle timeout starts when the statement or batch finishes.- MySQL, MariaDB, and PostgreSQL resources are supported. Aurora PostgreSQL resources resolve to the same PostgreSQL execution path.- SQL is sent directly to the local database engine through JDBC. `SqlParameter` binding is supported for all engines: named `:placeholder` markers are rewritten to positional JDBC bind parameters and executed through a `PreparedStatement`. Colons inside string literals, quoted/backtick identifiers, comments, PostgreSQL `::` casts, and PostgreSQL dollar-quoted strings are left untouched. A placeholder used more than once binds its value at each position. Supported value variants are `stringValue`, `booleanValue`, `longValue`, `doubleValue`, `blobValue`, and `isNull`; `typeHint` values `DECIMAL`, `TIMESTAMP`, `DATE`, `TIME`, `UUID`, and `JSON` are honored. `arrayValue` parameters are not supported yet and return `BadRequestException`, as do malformed `parameters` payloads (not a JSON array, or an entry missing `name`).-- Result records include Data API field variants such as `stringValue`, `longValue`, `blobValue`, `booleanValue`, `doubleValue`, and `isNull`. `records` is returned only for statements that produce a result set — empty when the query matched no rows — and is omitted entirely from the response to a statement that reports an update count, as AWS does. An `INSERT`, `UPDATE`, `DELETE`, or DDL statement therefore answers with `numberOfRecordsUpdated` and `generatedFields` and no `records` field at all.-- `includeResultMetadata` returns the full AWS `ColumnMetadata` shape per column: `label`, `name`, `type`, `typeName`, `tableName`, `schemaName`, `nullable`, `precision`, `scale`, `isSigned`, `isCaseSensitive`, `isCurrency`, `isAutoIncrement`, and `arrayBaseColumnType`. `label` is the result-set label (the alias when the query aliases a column) and `name` is the underlying column name, falling back to the label for computed columns — clients that hydrate rows by `label` need it present. `type` is a JDBC type code and `typeName` is the engine's own type name (for example `4` and `int4` for a PostgreSQL integer), both taken from the driver so they match the engine actually serving the query. `arrayBaseColumnType` is always `0` because array columns are not mapped to `arrayValue` fields yet.+- Result records include Data API field variants such as `stringValue`, `longValue`, `blobValue`, `booleanValue`, `doubleValue`, `arrayValue`, and `isNull`. One-dimensional SQL arrays use `longValues`, `stringValues`, `booleanValues`, or `doubleValues` according to their JDBC element type, including typed empty arrays. Decimal arrays use strings to preserve precision. Multidimensional arrays return `UnsupportedResultException`, as documented for `ExecuteStatement`. Arrays with null elements or binary elements also return `UnsupportedResultException`: the supported `ArrayValue` members cannot represent those elements without losing information. PostgreSQL bit-string arrays are rejected with the same error before JDBC decodes them as booleans; PostgreSQL boolean arrays remain supported. A null array column returns `isNull`. `records` is returned only for statements that produce a result set, empty when the query matched no rows, and is omitted entirely from the response to a statement that reports an update count, as AWS does. An `INSERT`, `UPDATE`, `DELETE`, or DDL statement therefore answers with `numberOfRecordsUpdated` and `generatedFields` and no `records` field at all.+- `includeResultMetadata` returns the full AWS `ColumnMetadata` shape per column: `label`, `name`, `type`, `typeName`, `tableName`, `schemaName`, `nullable`, `precision`, `scale`, `isSigned`, `isCaseSensitive`, `isCurrency`, `isAutoIncrement`, and `arrayBaseColumnType`. `label` is the result-set label (the alias when the query aliases a column) and `name` is the underlying column name, falling back to the label for computed columns. Clients that hydrate rows by `label` need it present. `type` is a JDBC type code and `typeName` is the engine's own type name (for example `4` and `int4` for a PostgreSQL integer), both taken from the driver so they match the engine actually serving the query. `arrayBaseColumnType` is the JDBC element type of a non-null array value, including an empty array. It remains `0` for non-array columns, queries matching no rows, or array columns containing only null values, because JDBC result metadata alone does not expose the element type.- `ExecuteStatement` returns `generatedFields` for statements that report an update count on MySQL and MariaDB, carrying the auto-increment keys the engine generated. As on Aurora PostgreSQL, PostgreSQL resources always return an empty list — use a `RETURNING` clause to read generated values. `generatedFields` is omitted for statements that return a result set.- `BatchExecuteStatement` runs the statement once per entry in `parameterSets` through a JDBC batch and returns one `updateResults` entry per set, each carrying the `generatedFields` that set produced under the same engine rules as `ExecuteStatement` — a set inserting several rows reports one field per generated key. On MySQL and MariaDB the sets are executed one at a time so each entry owns its own keys, because `getGeneratedKeys()` after a JDBC batch reports every key the batch generated with nothing tying a key back to the set that produced it. It accepts `transactionId`, so a batch can take part in a Data API transaction; without one the batch commits automatically. An absent or empty `parameterSets` runs nothing and returns an empty `updateResults`: AWS runs the statement once per parameter set provided and points a caller who wants a single parameterless execution at one empty set (`[[]]`, which runs once here) or at `ExecuteStatement`. A statement that returns rows rather than an update count — a `SELECT`, or a CTE ending in one — is rejected with `BadRequestException` before any set runs, because a batch reports one `UpdateResult` per set and has nowhere to carry a result set, and AWS documents the operation as taking a DML statement. The check reads the driver's description of the prepared statement, so nothing executes first, and a DML statement that also reports rows through a PostgreSQL `RETURNING` clause is left alone: the batch still performs its writes, and `generatedFields` is empty on PostgreSQL anyway.- A result set in which any single row is larger than 64 KB is rejected with `UnsupportedResultException` and the message `Packet for query is too large`, as the Data API troubleshooting guide describes. A row is measured by its values as the driver returns them (the raw length of a binary value, the UTF-8 length of anything else), so JSON escaping and base64 encoding in the response do not count toward it.diff --git a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.javaindex 58d28826c1..7b175c8ac8 100644--- a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java+++ b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java@@ -18,9 +18,9 @@ import java.sql.SQLException;** <p>{@code type} is a {@link java.sql.Types} code and {@code typeName} is the* engine's own type name, exactly as AWS reports them (for example {@code 4}- * and {@code int4} for a PostgreSQL integer). {@code arrayBaseColumnType} is- * always {@code 0} because array columns are not mapped to {@code arrayValue}- * fields yet.+ * and {@code int4} for a PostgreSQL integer). {@code arrayBaseColumnType} starts+ * at {@code 0}: the result mapper fills it from non-null JDBC array values,+ * because {@code ResultSetMetaData} does not expose the array element type.*/final class RdsDataColumnMetadata {diff --git a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.javaindex 0bce95628d..f6f05bade2 100644--- a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java+++ b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java@@ -1,9 +1,12 @@package io.github.hectorvent.floci.services.rdsdata;import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ArrayNode;import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.core.common.AwsException;import java.math.BigDecimal;+import java.sql.Array;import java.sql.Blob;import java.sql.Date;import java.sql.SQLException;@@ -67,11 +70,49 @@ final class RdsDataFieldMapper {case Types.DATE -> field.put("stringValue", formatDate(value));case Types.TIME -> field.put("stringValue", formatTime(value));case Types.TIMESTAMP, Types.TIMESTAMP_WITH_TIMEZONE -> field.put("stringValue", formatTimestamp(value));+ case Types.ARRAY -> field.set("arrayValue", mapArray(objectMapper, (Array) value));default -> field.put("stringValue", value.toString());}return field;}+ private static ObjectNode mapArray(ObjectMapper objectMapper, Array array) throws SQLException {+ try {+ int baseType = array.getBaseType();+ // PostgreSQL reports both bool and bit strings as JDBC BIT; decoding bit strings as booleans loses bits.+ if (baseType == Types.BIT && "bit".equalsIgnoreCase(array.getBaseTypeName())) {+ throw new AwsException("UnsupportedResultException", "Bit-string arrays are not supported.", 400);+ }+ String fieldName = switch (baseType) {+ case Types.BIGINT, Types.INTEGER, Types.SMALLINT, Types.TINYINT -> "longValue";+ case Types.BOOLEAN, Types.BIT -> "booleanValue";+ case Types.FLOAT, Types.REAL, Types.DOUBLE -> "doubleValue";+ case Types.ARRAY -> throw new AwsException("UnsupportedResultException",+ "Multidimensional arrays are not supported.", 400);+ case Types.BINARY, Types.VARBINARY, Types.LONGVARBINARY, Types.BLOB ->+ throw new AwsException("UnsupportedResultException", "Binary arrays are not supported.", 400);+ default -> "stringValue";+ };+ Object values = array.getArray();+ ObjectNode arrayValue = objectMapper.createObjectNode();+ ArrayNode elements = arrayValue.putArray(fieldName + "s");+ // Qualify the reflection helper to distinguish it from JDBC's java.sql.Array.+ for (int index = 0; index < java.lang.reflect.Array.getLength(values); index++) {+ Object element = java.lang.reflect.Array.get(values, index);+ if (element == null) {+ throw new AwsException("UnsupportedResultException", "Null array elements are not supported.", 400);+ }+ if (element instanceof Array || element.getClass().isArray()) {+ throw new AwsException("UnsupportedResultException", "Multidimensional arrays are not supported.", 400);+ }+ elements.add(toField(objectMapper, element, baseType).get(fieldName));+ }+ return arrayValue;+ } finally {+ array.free();+ }+ }+private static String formatDate(Object value) {if (value instanceof Date date) {return date.toLocalDate().toString();diff --git a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.javaindex 69527b289d..6773a1e4dd 100644--- a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java+++ b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java@@ -19,6 +19,7 @@ import jakarta.inject.Inject;import org.jboss.logging.Logger;import java.nio.charset.StandardCharsets;+import java.sql.Array;import java.sql.Blob;import java.sql.Connection;import java.sql.PreparedStatement;@@ -502,10 +503,12 @@ public class RdsDataService implements Resettable {try (ResultSet rs = statement.getResultSet()) {ResultSetMetaData meta = rs.getMetaData();rejectUnsupportedResultTypes(meta, engine);- if (includeMetadata) {- response.set("columnMetadata", RdsDataColumnMetadata.toColumnMetadata(objectMapper, meta));+ ArrayNode columnMetadata = includeMetadata+ ? RdsDataColumnMetadata.toColumnMetadata(objectMapper, meta) : null;+ if (columnMetadata != null) {+ response.set("columnMetadata", columnMetadata);}- response.set("records", records(rs, meta));+ response.set("records", records(rs, meta, columnMetadata));}response.put("numberOfRecordsUpdated", 0L);} else {@@ -561,7 +564,7 @@ public class RdsDataService implements Resettable {}}- private ArrayNode records(ResultSet rs, ResultSetMetaData meta) throws SQLException {+ private ArrayNode records(ResultSet rs, ResultSetMetaData meta, ArrayNode columnMetadata) throws SQLException {ArrayNode records = objectMapper.createArrayNode();int columnCount = meta.getColumnCount();// Serialized size of the records array: its two brackets plus one comma between rows.@@ -571,6 +574,9 @@ public class RdsDataService implements Resettable {long rawRowBytes = 0;for (int i = 1; i <= columnCount; i++) {Object value = rs.getObject(i);+ if (columnMetadata != null && value instanceof Array array) {+ ((ObjectNode) columnMetadata.get(i - 1)).put("arrayBaseColumnType", array.getBaseType());+ }rawRowBytes += valueBytes(value);row.add(RdsDataFieldMapper.toField(objectMapper, value, meta.getColumnType(i)));}diff --git a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.javaindex 981fcfc505..b23e8b7cbf 100644--- a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java@@ -2,9 +2,11 @@ package io.github.hectorvent.floci.services.rdsdata;import com.fasterxml.jackson.databind.ObjectMapper;import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.core.common.AwsException;import org.junit.jupiter.api.Test;import java.math.BigDecimal;+import java.sql.Array;import java.sql.Date;import java.sql.Time;import java.sql.Timestamp;@@ -15,7 +17,11 @@ import java.time.LocalTime;import static org.junit.jupiter.api.Assertions.assertArrayEquals;import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertThrows;import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.verify;+import static org.mockito.Mockito.when;class RdsDataFieldMapperTest {@@ -75,4 +81,88 @@ class RdsDataFieldMapperTest {assertTrue(bitField.get("booleanValue").asBoolean());}++ @Test+ void mapsPrimitiveAndBoxedArraysToTypedDataApiValues() throws Exception {+ assertArrayField(Types.INTEGER, new Integer[] {1, 2, 3}, "{\"longValues\":[1,2,3]}");+ assertArrayField(Types.BIGINT, new long[] {Long.MIN_VALUE, Long.MAX_VALUE},+ "{\"longValues\":[-9223372036854775808,9223372036854775807]}");+ assertArrayField(Types.VARCHAR, new String[] {"a", "quotes\" and commas,"},+ "{\"stringValues\":[\"a\",\"quotes\\\" and commas,\"]}");+ assertArrayField(Types.BOOLEAN, new boolean[] {true, false}, "{\"booleanValues\":[true,false]}");+ assertArrayField(Types.DOUBLE, new double[] {1.5, -2.25}, "{\"doubleValues\":[1.5,-2.25]}");+ }++ @Test+ void preservesEmptyArrayTypesAndDecimalPrecision() throws Exception {+ assertArrayField(Types.INTEGER, new Integer[0], "{\"longValues\":[]}");+ assertArrayField(Types.VARCHAR, new String[0], "{\"stringValues\":[]}");+ assertArrayField(Types.BOOLEAN, new boolean[0], "{\"booleanValues\":[]}");+ assertArrayField(Types.DOUBLE, new double[0], "{\"doubleValues\":[]}");+ assertArrayField(Types.NUMERIC, new BigDecimal[] {new BigDecimal("12345678901234567890.123456789")},+ "{\"stringValues\":[\"12345678901234567890.123456789\"]}");+ assertEquals(objectMapper.readTree("{\"isNull\":true}"),+ RdsDataFieldMapper.toField(objectMapper, null, Types.ARRAY));+ }++ @Test+ void rejectsMultidimensionalArraysWithoutFlatteningThem() throws Exception {+ Array array = sqlArray(Types.INTEGER, new Integer[][] {{1, 2}, {3, 4}});+ AwsException error = assertThrows(AwsException.class,+ () -> RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY));++ assertEquals("UnsupportedResultException", error.getErrorCode());+ assertEquals(400, error.getHttpStatus());+ verify(array).free();++ Array nested = sqlArray(Types.ARRAY, new Object[0]);+ assertThrows(AwsException.class, () -> RdsDataFieldMapper.toField(objectMapper, nested, Types.ARRAY));+ }++ @Test+ void rejectsArrayElementsThatTheDataApiUnionCannotRepresent() throws Exception {+ Array nullable = sqlArray(Types.INTEGER, new Integer[] {1, null});+ Array binary = sqlArray(Types.BINARY, new byte[][] {{1, 2}});++ for (Array array : new Array[] {nullable, binary}) {+ AwsException error = assertThrows(AwsException.class,+ () -> RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY));+ assertEquals("UnsupportedResultException", error.getErrorCode());+ assertEquals(400, error.getHttpStatus());+ verify(array).free();+ }+ }++ @Test+ void rejectsPostgresBitStringsWithoutConvertingTheirBitsToBooleans() throws Exception {+ Array array = sqlArray(Types.BIT, new String[] {"10", "01"});+ when(array.getBaseTypeName()).thenReturn("bit");++ AwsException error = assertThrows(AwsException.class,+ () -> RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY));+ assertEquals("UnsupportedResultException", error.getErrorCode());+ assertEquals(400, error.getHttpStatus());+ verify(array).free();++ Array bool = sqlArray(Types.BIT, new Boolean[] {true, false});+ when(bool.getBaseTypeName()).thenReturn("bool");+ assertEquals(objectMapper.readTree("{\"arrayValue\":{\"booleanValues\":[true,false]}}"),+ RdsDataFieldMapper.toField(objectMapper, bool, Types.ARRAY));+ }++ private void assertArrayField(int baseType, Object values, String expected) throws Exception {+ Array array = sqlArray(baseType, values);+ ObjectNode field = RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY);++ assertEquals(objectMapper.readTree("{\"arrayValue\":" + expected + "}"),+ objectMapper.readTree(field.toString()));+ verify(array).free();+ }++ private static Array sqlArray(int baseType, Object values) throws Exception {+ Array array = mock(Array.class);+ when(array.getBaseType()).thenReturn(baseType);+ when(array.getArray()).thenReturn(values);+ return array;+ }}diff --git a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.javaindex 3a53d3f124..e7f124d81f 100644--- a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java@@ -53,6 +53,37 @@ class RdsDataServiceTest {private final ObjectMapper objectMapper = new ObjectMapper();+ @Test+ void returnsTypedArraysThroughTheJdbcExecutionPath() throws Exception {+ TestHarness harness = new TestHarness(DatabaseEngine.POSTGRES);+ ObjectNode request = harness.request("""+ select array[1,2,3], array['a','b'], array[true,false],+ array[cast(1.5 as double),cast(2.25 as double)],+ cast(null as integer array)+ """);+ request.put("includeResultMetadata", true);+ ObjectNode response = harness.service.executeStatement(request, REGION);++ assertEquals(objectMapper.readTree("""+ [[{"arrayValue":{"longValues":[1,2,3]}},+ {"arrayValue":{"stringValues":["a","b"]}},+ {"arrayValue":{"booleanValues":[true,false]}},+ {"arrayValue":{"doubleValues":[1.5,2.25]}}, {"isNull":true}]]+ """), objectMapper.readTree(response.get("records").toString()));+ assertEquals(Types.INTEGER, response.path("columnMetadata").get(0).path("arrayBaseColumnType").asInt());+ }++ @Test+ void rejectsMultidimensionalArrayResultsThroughTheJdbcExecutionPath() throws Exception {+ TestHarness harness = new TestHarness(DatabaseEngine.POSTGRES);+ AwsException error = assertThrows(AwsException.class, () -> harness.service.executeStatement(+ harness.request("select array[array[1,2],array[3,4]]"), REGION));++ assertEquals("UnsupportedResultException", error.getErrorCode());+ assertEquals(400, error.getHttpStatus());+ assertEquals(0, harness.gate.held.get());+ }+@Testvoid rejectsMultipleStatementsBeforeEitherWriteRuns() throws Exception {TestHarness harness = new TestHarness(DatabaseEngine.POSTGRES);