Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

fix(rds-data): return typed array result fields (#5665)

Aniket Shuklabb06acdeb11b
diff --git a/docs/services/rds-data.md b/docs/services/rds-data.md
index 2fb3993fb4..2a3b0be343 100644
--- a/docs/services/rds-data.md
+++ b/docs/services/rds-data.md
@@ -37,8 +37,8 @@ The deprecated `ExecuteSql` operation is recognized at `POST /ExecuteSql` and re
- Requests sharing a `transactionId` do not queue behind an active request: `ExecuteStatement`, `BatchExecuteStatement`, `CommitTransaction`, and `RollbackTransaction` immediately return HTTP 400 `DatabaseErrorException` with `Transaction is still running a query`. Retry after the active request finishes. Other transactions remain independent, and expiry cleanup skips active requests. The idle timeout starts when the statement or batch finishes.
- MySQL, MariaDB, and PostgreSQL resources are supported. Aurora PostgreSQL resources resolve to the same PostgreSQL execution path.
- SQL is sent directly to the local database engine through JDBC. `SqlParameter` binding is supported for all engines: named `:placeholder` markers are rewritten to positional JDBC bind parameters and executed through a `PreparedStatement`. Colons inside string literals, quoted/backtick identifiers, comments, PostgreSQL `::` casts, and PostgreSQL dollar-quoted strings are left untouched. A placeholder used more than once binds its value at each position. Supported value variants are `stringValue`, `booleanValue`, `longValue`, `doubleValue`, `blobValue`, and `isNull`; `typeHint` values `DECIMAL`, `TIMESTAMP`, `DATE`, `TIME`, `UUID`, and `JSON` are honored. `arrayValue` parameters are not supported yet and return `BadRequestException`, as do malformed `parameters` payloads (not a JSON array, or an entry missing `name`).
-- Result records include Data API field variants such as `stringValue`, `longValue`, `blobValue`, `booleanValue`, `doubleValue`, and `isNull`. `records` is returned only for statements that produce a result set — empty when the query matched no rows — and is omitted entirely from the response to a statement that reports an update count, as AWS does. An `INSERT`, `UPDATE`, `DELETE`, or DDL statement therefore answers with `numberOfRecordsUpdated` and `generatedFields` and no `records` field at all.
-- `includeResultMetadata` returns the full AWS `ColumnMetadata` shape per column: `label`, `name`, `type`, `typeName`, `tableName`, `schemaName`, `nullable`, `precision`, `scale`, `isSigned`, `isCaseSensitive`, `isCurrency`, `isAutoIncrement`, and `arrayBaseColumnType`. `label` is the result-set label (the alias when the query aliases a column) and `name` is the underlying column name, falling back to the label for computed columns — clients that hydrate rows by `label` need it present. `type` is a JDBC type code and `typeName` is the engine's own type name (for example `4` and `int4` for a PostgreSQL integer), both taken from the driver so they match the engine actually serving the query. `arrayBaseColumnType` is always `0` because array columns are not mapped to `arrayValue` fields yet.
+- Result records include Data API field variants such as `stringValue`, `longValue`, `blobValue`, `booleanValue`, `doubleValue`, `arrayValue`, and `isNull`. One-dimensional SQL arrays use `longValues`, `stringValues`, `booleanValues`, or `doubleValues` according to their JDBC element type, including typed empty arrays. Decimal arrays use strings to preserve precision. Multidimensional arrays return `UnsupportedResultException`, as documented for `ExecuteStatement`. Arrays with null elements or binary elements also return `UnsupportedResultException`: the supported `ArrayValue` members cannot represent those elements without losing information. PostgreSQL bit-string arrays are rejected with the same error before JDBC decodes them as booleans; PostgreSQL boolean arrays remain supported. A null array column returns `isNull`. `records` is returned only for statements that produce a result set, empty when the query matched no rows, and is omitted entirely from the response to a statement that reports an update count, as AWS does. An `INSERT`, `UPDATE`, `DELETE`, or DDL statement therefore answers with `numberOfRecordsUpdated` and `generatedFields` and no `records` field at all.
+- `includeResultMetadata` returns the full AWS `ColumnMetadata` shape per column: `label`, `name`, `type`, `typeName`, `tableName`, `schemaName`, `nullable`, `precision`, `scale`, `isSigned`, `isCaseSensitive`, `isCurrency`, `isAutoIncrement`, and `arrayBaseColumnType`. `label` is the result-set label (the alias when the query aliases a column) and `name` is the underlying column name, falling back to the label for computed columns. Clients that hydrate rows by `label` need it present. `type` is a JDBC type code and `typeName` is the engine's own type name (for example `4` and `int4` for a PostgreSQL integer), both taken from the driver so they match the engine actually serving the query. `arrayBaseColumnType` is the JDBC element type of a non-null array value, including an empty array. It remains `0` for non-array columns, queries matching no rows, or array columns containing only null values, because JDBC result metadata alone does not expose the element type.
- `ExecuteStatement` returns `generatedFields` for statements that report an update count on MySQL and MariaDB, carrying the auto-increment keys the engine generated. As on Aurora PostgreSQL, PostgreSQL resources always return an empty list — use a `RETURNING` clause to read generated values. `generatedFields` is omitted for statements that return a result set.
- `BatchExecuteStatement` runs the statement once per entry in `parameterSets` through a JDBC batch and returns one `updateResults` entry per set, each carrying the `generatedFields` that set produced under the same engine rules as `ExecuteStatement` — a set inserting several rows reports one field per generated key. On MySQL and MariaDB the sets are executed one at a time so each entry owns its own keys, because `getGeneratedKeys()` after a JDBC batch reports every key the batch generated with nothing tying a key back to the set that produced it. It accepts `transactionId`, so a batch can take part in a Data API transaction; without one the batch commits automatically. An absent or empty `parameterSets` runs nothing and returns an empty `updateResults`: AWS runs the statement once per parameter set provided and points a caller who wants a single parameterless execution at one empty set (`[[]]`, which runs once here) or at `ExecuteStatement`. A statement that returns rows rather than an update count — a `SELECT`, or a CTE ending in one — is rejected with `BadRequestException` before any set runs, because a batch reports one `UpdateResult` per set and has nowhere to carry a result set, and AWS documents the operation as taking a DML statement. The check reads the driver's description of the prepared statement, so nothing executes first, and a DML statement that also reports rows through a PostgreSQL `RETURNING` clause is left alone: the batch still performs its writes, and `generatedFields` is empty on PostgreSQL anyway.
- A result set in which any single row is larger than 64 KB is rejected with `UnsupportedResultException` and the message `Packet for query is too large`, as the Data API troubleshooting guide describes. A row is measured by its values as the driver returns them (the raw length of a binary value, the UTF-8 length of anything else), so JSON escaping and base64 encoding in the response do not count toward it.
diff --git a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java
index 58d28826c1..7b175c8ac8 100644
--- a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java
+++ b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataColumnMetadata.java
@@ -18,9 +18,9 @@ import java.sql.SQLException;
*
* <p>{@code type} is a {@link java.sql.Types} code and {@code typeName} is the
* engine's own type name, exactly as AWS reports them (for example {@code 4}
- * and {@code int4} for a PostgreSQL integer). {@code arrayBaseColumnType} is
- * always {@code 0} because array columns are not mapped to {@code arrayValue}
- * fields yet.
+ * and {@code int4} for a PostgreSQL integer). {@code arrayBaseColumnType} starts
+ * at {@code 0}: the result mapper fills it from non-null JDBC array values,
+ * because {@code ResultSetMetaData} does not expose the array element type.
*/
final class RdsDataColumnMetadata {
diff --git a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java
index 0bce95628d..f6f05bade2 100644
--- a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java
+++ b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapper.java
@@ -1,9 +1,12 @@
package io.github.hectorvent.floci.services.rdsdata;
import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.core.common.AwsException;
import java.math.BigDecimal;
+import java.sql.Array;
import java.sql.Blob;
import java.sql.Date;
import java.sql.SQLException;
@@ -67,11 +70,49 @@ final class RdsDataFieldMapper {
case Types.DATE -> field.put("stringValue", formatDate(value));
case Types.TIME -> field.put("stringValue", formatTime(value));
case Types.TIMESTAMP, Types.TIMESTAMP_WITH_TIMEZONE -> field.put("stringValue", formatTimestamp(value));
+ case Types.ARRAY -> field.set("arrayValue", mapArray(objectMapper, (Array) value));
default -> field.put("stringValue", value.toString());
}
return field;
}
+ private static ObjectNode mapArray(ObjectMapper objectMapper, Array array) throws SQLException {
+ try {
+ int baseType = array.getBaseType();
+ // PostgreSQL reports both bool and bit strings as JDBC BIT; decoding bit strings as booleans loses bits.
+ if (baseType == Types.BIT && "bit".equalsIgnoreCase(array.getBaseTypeName())) {
+ throw new AwsException("UnsupportedResultException", "Bit-string arrays are not supported.", 400);
+ }
+ String fieldName = switch (baseType) {
+ case Types.BIGINT, Types.INTEGER, Types.SMALLINT, Types.TINYINT -> "longValue";
+ case Types.BOOLEAN, Types.BIT -> "booleanValue";
+ case Types.FLOAT, Types.REAL, Types.DOUBLE -> "doubleValue";
+ case Types.ARRAY -> throw new AwsException("UnsupportedResultException",
+ "Multidimensional arrays are not supported.", 400);
+ case Types.BINARY, Types.VARBINARY, Types.LONGVARBINARY, Types.BLOB ->
+ throw new AwsException("UnsupportedResultException", "Binary arrays are not supported.", 400);
+ default -> "stringValue";
+ };
+ Object values = array.getArray();
+ ObjectNode arrayValue = objectMapper.createObjectNode();
+ ArrayNode elements = arrayValue.putArray(fieldName + "s");
+ // Qualify the reflection helper to distinguish it from JDBC's java.sql.Array.
+ for (int index = 0; index < java.lang.reflect.Array.getLength(values); index++) {
+ Object element = java.lang.reflect.Array.get(values, index);
+ if (element == null) {
+ throw new AwsException("UnsupportedResultException", "Null array elements are not supported.", 400);
+ }
+ if (element instanceof Array || element.getClass().isArray()) {
+ throw new AwsException("UnsupportedResultException", "Multidimensional arrays are not supported.", 400);
+ }
+ elements.add(toField(objectMapper, element, baseType).get(fieldName));
+ }
+ return arrayValue;
+ } finally {
+ array.free();
+ }
+ }
+
private static String formatDate(Object value) {
if (value instanceof Date date) {
return date.toLocalDate().toString();
diff --git a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java
index 69527b289d..6773a1e4dd 100644
--- a/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java
+++ b/src/main/java/io/github/hectorvent/floci/services/rdsdata/RdsDataService.java
@@ -19,6 +19,7 @@ import jakarta.inject.Inject;
import org.jboss.logging.Logger;
import java.nio.charset.StandardCharsets;
+import java.sql.Array;
import java.sql.Blob;
import java.sql.Connection;
import java.sql.PreparedStatement;
@@ -502,10 +503,12 @@ public class RdsDataService implements Resettable {
try (ResultSet rs = statement.getResultSet()) {
ResultSetMetaData meta = rs.getMetaData();
rejectUnsupportedResultTypes(meta, engine);
- if (includeMetadata) {
- response.set("columnMetadata", RdsDataColumnMetadata.toColumnMetadata(objectMapper, meta));
+ ArrayNode columnMetadata = includeMetadata
+ ? RdsDataColumnMetadata.toColumnMetadata(objectMapper, meta) : null;
+ if (columnMetadata != null) {
+ response.set("columnMetadata", columnMetadata);
}
- response.set("records", records(rs, meta));
+ response.set("records", records(rs, meta, columnMetadata));
}
response.put("numberOfRecordsUpdated", 0L);
} else {
@@ -561,7 +564,7 @@ public class RdsDataService implements Resettable {
}
}
- private ArrayNode records(ResultSet rs, ResultSetMetaData meta) throws SQLException {
+ private ArrayNode records(ResultSet rs, ResultSetMetaData meta, ArrayNode columnMetadata) throws SQLException {
ArrayNode records = objectMapper.createArrayNode();
int columnCount = meta.getColumnCount();
// Serialized size of the records array: its two brackets plus one comma between rows.
@@ -571,6 +574,9 @@ public class RdsDataService implements Resettable {
long rawRowBytes = 0;
for (int i = 1; i <= columnCount; i++) {
Object value = rs.getObject(i);
+ if (columnMetadata != null && value instanceof Array array) {
+ ((ObjectNode) columnMetadata.get(i - 1)).put("arrayBaseColumnType", array.getBaseType());
+ }
rawRowBytes += valueBytes(value);
row.add(RdsDataFieldMapper.toField(objectMapper, value, meta.getColumnType(i)));
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java
index 981fcfc505..b23e8b7cbf 100644
--- a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataFieldMapperTest.java
@@ -2,9 +2,11 @@ package io.github.hectorvent.floci.services.rdsdata;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.core.common.AwsException;
import org.junit.jupiter.api.Test;
import java.math.BigDecimal;
+import java.sql.Array;
import java.sql.Date;
import java.sql.Time;
import java.sql.Timestamp;
@@ -15,7 +17,11 @@ import java.time.LocalTime;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.when;
class RdsDataFieldMapperTest {
@@ -75,4 +81,88 @@ class RdsDataFieldMapperTest {
assertTrue(bitField.get("booleanValue").asBoolean());
}
+
+ @Test
+ void mapsPrimitiveAndBoxedArraysToTypedDataApiValues() throws Exception {
+ assertArrayField(Types.INTEGER, new Integer[] {1, 2, 3}, "{\"longValues\":[1,2,3]}");
+ assertArrayField(Types.BIGINT, new long[] {Long.MIN_VALUE, Long.MAX_VALUE},
+ "{\"longValues\":[-9223372036854775808,9223372036854775807]}");
+ assertArrayField(Types.VARCHAR, new String[] {"a", "quotes\" and commas,"},
+ "{\"stringValues\":[\"a\",\"quotes\\\" and commas,\"]}");
+ assertArrayField(Types.BOOLEAN, new boolean[] {true, false}, "{\"booleanValues\":[true,false]}");
+ assertArrayField(Types.DOUBLE, new double[] {1.5, -2.25}, "{\"doubleValues\":[1.5,-2.25]}");
+ }
+
+ @Test
+ void preservesEmptyArrayTypesAndDecimalPrecision() throws Exception {
+ assertArrayField(Types.INTEGER, new Integer[0], "{\"longValues\":[]}");
+ assertArrayField(Types.VARCHAR, new String[0], "{\"stringValues\":[]}");
+ assertArrayField(Types.BOOLEAN, new boolean[0], "{\"booleanValues\":[]}");
+ assertArrayField(Types.DOUBLE, new double[0], "{\"doubleValues\":[]}");
+ assertArrayField(Types.NUMERIC, new BigDecimal[] {new BigDecimal("12345678901234567890.123456789")},
+ "{\"stringValues\":[\"12345678901234567890.123456789\"]}");
+ assertEquals(objectMapper.readTree("{\"isNull\":true}"),
+ RdsDataFieldMapper.toField(objectMapper, null, Types.ARRAY));
+ }
+
+ @Test
+ void rejectsMultidimensionalArraysWithoutFlatteningThem() throws Exception {
+ Array array = sqlArray(Types.INTEGER, new Integer[][] {{1, 2}, {3, 4}});
+ AwsException error = assertThrows(AwsException.class,
+ () -> RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY));
+
+ assertEquals("UnsupportedResultException", error.getErrorCode());
+ assertEquals(400, error.getHttpStatus());
+ verify(array).free();
+
+ Array nested = sqlArray(Types.ARRAY, new Object[0]);
+ assertThrows(AwsException.class, () -> RdsDataFieldMapper.toField(objectMapper, nested, Types.ARRAY));
+ }
+
+ @Test
+ void rejectsArrayElementsThatTheDataApiUnionCannotRepresent() throws Exception {
+ Array nullable = sqlArray(Types.INTEGER, new Integer[] {1, null});
+ Array binary = sqlArray(Types.BINARY, new byte[][] {{1, 2}});
+
+ for (Array array : new Array[] {nullable, binary}) {
+ AwsException error = assertThrows(AwsException.class,
+ () -> RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY));
+ assertEquals("UnsupportedResultException", error.getErrorCode());
+ assertEquals(400, error.getHttpStatus());
+ verify(array).free();
+ }
+ }
+
+ @Test
+ void rejectsPostgresBitStringsWithoutConvertingTheirBitsToBooleans() throws Exception {
+ Array array = sqlArray(Types.BIT, new String[] {"10", "01"});
+ when(array.getBaseTypeName()).thenReturn("bit");
+
+ AwsException error = assertThrows(AwsException.class,
+ () -> RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY));
+ assertEquals("UnsupportedResultException", error.getErrorCode());
+ assertEquals(400, error.getHttpStatus());
+ verify(array).free();
+
+ Array bool = sqlArray(Types.BIT, new Boolean[] {true, false});
+ when(bool.getBaseTypeName()).thenReturn("bool");
+ assertEquals(objectMapper.readTree("{\"arrayValue\":{\"booleanValues\":[true,false]}}"),
+ RdsDataFieldMapper.toField(objectMapper, bool, Types.ARRAY));
+ }
+
+ private void assertArrayField(int baseType, Object values, String expected) throws Exception {
+ Array array = sqlArray(baseType, values);
+ ObjectNode field = RdsDataFieldMapper.toField(objectMapper, array, Types.ARRAY);
+
+ assertEquals(objectMapper.readTree("{\"arrayValue\":" + expected + "}"),
+ objectMapper.readTree(field.toString()));
+ verify(array).free();
+ }
+
+ private static Array sqlArray(int baseType, Object values) throws Exception {
+ Array array = mock(Array.class);
+ when(array.getBaseType()).thenReturn(baseType);
+ when(array.getArray()).thenReturn(values);
+ return array;
+ }
}
diff --git a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java
index 3a53d3f124..e7f124d81f 100644
--- a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java
@@ -53,6 +53,37 @@ class RdsDataServiceTest {
private final ObjectMapper objectMapper = new ObjectMapper();
+ @Test
+ void returnsTypedArraysThroughTheJdbcExecutionPath() throws Exception {
+ TestHarness harness = new TestHarness(DatabaseEngine.POSTGRES);
+ ObjectNode request = harness.request("""
+ select array[1,2,3], array['a','b'], array[true,false],
+ array[cast(1.5 as double),cast(2.25 as double)],
+ cast(null as integer array)
+ """);
+ request.put("includeResultMetadata", true);
+ ObjectNode response = harness.service.executeStatement(request, REGION);
+
+ assertEquals(objectMapper.readTree("""
+ [[{"arrayValue":{"longValues":[1,2,3]}},
+ {"arrayValue":{"stringValues":["a","b"]}},
+ {"arrayValue":{"booleanValues":[true,false]}},
+ {"arrayValue":{"doubleValues":[1.5,2.25]}}, {"isNull":true}]]
+ """), objectMapper.readTree(response.get("records").toString()));
+ assertEquals(Types.INTEGER, response.path("columnMetadata").get(0).path("arrayBaseColumnType").asInt());
+ }
+
+ @Test
+ void rejectsMultidimensionalArrayResultsThroughTheJdbcExecutionPath() throws Exception {
+ TestHarness harness = new TestHarness(DatabaseEngine.POSTGRES);
+ AwsException error = assertThrows(AwsException.class, () -> harness.service.executeStatement(
+ harness.request("select array[array[1,2],array[3,4]]"), REGION));
+
+ assertEquals("UnsupportedResultException", error.getErrorCode());
+ assertEquals(400, error.getHttpStatus());
+ assertEquals(0, harness.gate.held.get());
+ }
+
@Test
void rejectsMultipleStatementsBeforeEitherWriteRuns() throws Exception {
TestHarness harness = new TestHarness(DatabaseEngine.POSTGRES);