Light, fluffy, and always free - The AWS Local Emulator alternative
feat(redshift): wire the Simple Query DDL interceptor into the proxy (#3029)
Vũ Thế Huy4daba7ed6a3b
diff --git a/docs/services/redshift.md b/docs/services/redshift.mdindex 5e1a295b6e..01db4613f2 100644--- a/docs/services/redshift.md+++ b/docs/services/redshift.md@@ -135,9 +135,27 @@ cluster = redshift.create_cluster(print(cluster["Cluster"]["Endpoint"])```+## SQL Interceptor++Floci's Redshift auth proxy inspects frontend queries on the PostgreSQL wire protocol (Simple Query `'Q'` protocol) and rewrites common Redshift-specific table DDL so it runs on the plain PostgreSQL backend.++### DDL compatibility++- Redshift-only table DDL keywords are stripped before the statement is forwarded: `DISTSTYLE ALL|EVEN|KEY|AUTO`, `DISTKEY (<col>)` and column-level `DISTKEY`, `[COMPOUND|INTERLEAVED] SORTKEY (<cols>)` and column-level `SORTKEY`, and `ENCODE <codec>` for the real Redshift column encodings (`raw`, `az64`, `bytedict`, `delta`, `delta32k`, `lzo`, `mostly8`, `mostly16`, `mostly32`, `runlength`, `text255`, `text32k`, `zstd`) or `auto`.+- The rewrite only runs when the statement's first keyword is `CREATE TABLE` or `ALTER TABLE`. A `SELECT`, `INSERT`, function body, or string literal that merely contains one of these keywords is forwarded byte-for-byte. Single-quoted and dollar-quoted string literals are masked before the rewrite, so a keyword inside a quoted value (including in a later statement of a multi-statement query) is preserved.+- Columns legitimately named `distkey`, `sortkey`, or `encode` survive.++### Limitations++- Emulation runs on the **Simple Query protocol** (`'Q'`) only. Extended Query protocol statements (`Parse`/`Bind`/`Execute`) pass through untouched, including anything a JDBC `PreparedStatement` sends, and, with the pgjdbc default `preferQueryMode=extended`, plain `Statement` calls too. Connect with `preferQueryMode=simple` to exercise the interceptor from JDBC.+- The rewrite is textual (regex-based). It masks single-quoted string literals first, so `DEFAULT` / `CHECK` string values are safe, but it is **not** comment-aware and does not recognize escape strings (`E'...'`): an apostrophe inside a `--` or `/* */` comment can make the rewrite skip a Redshift clause. That fails safe: the statement then reaches PostgreSQL, which returns its own syntax error, but avoid apostrophes-in-comments in `CREATE TABLE` / `ALTER TABLE`.+- A `rewrite` failure or any statement the interceptor does not recognize is forwarded unmodified (fail-open); PostgreSQL then rejects the Redshift-only syntax itself.+- Simple Query ('Q') messages larger than 16 MiB bypass the interceptor and stream through verbatim without heap buffering; non-query traffic also streams through with no size limit.+- `COPY ... FROM 's3://...'` and `UNLOAD (...) TO 's3://...'` are **not** yet emulated (planned).+## Out of Scope-- Real Redshift SQL semantics — the data plane is stock PostgreSQL, so Redshift-only SQL (distribution/sort keys, `COPY`/`UNLOAD` from S3, `SUPER`/`SPECTRUM`) is not emulated.+- Real Redshift SQL semantics: the data plane is stock PostgreSQL. Redshift-only table DDL keywords (DISTSTYLE / DISTKEY / SORTKEY / ENCODE) are stripped so CREATE TABLE / ALTER TABLE executes (see [SQL Interceptor](#sql-interceptor)), but the distribution/sort behavior they request is not; COPY/UNLOAD from S3 and SUPER/SPECTRUM are not emulated.- Multi-node clusters — `NodeType` and `NumberOfNodes` are stored as metadata; every cluster is a single PostgreSQL container.- Parameter groups apply no real engine settings; values are stored and echoed back only.- Subnet groups, VPC routing, and security groups are metadata only.diff --git a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoder.java b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoder.javanew file mode 100644index 0000000000..b6bc43f358--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoder.java@@ -0,0 +1,192 @@+package io.github.hectorvent.floci.services.redshift.proxy;++import java.io.EOFException;+import java.io.IOException;+import java.io.InputStream;+import java.io.OutputStream;+import java.nio.charset.StandardCharsets;+import java.util.Objects;++/**+ * Frames the PostgreSQL wire-protocol byte stream sent by the frontend (client).+ *+ * <p>Every frontend message is {@code type(1 byte) · length(int32, includes itself) · body(length-4)}.+ * This decoder turns that stream into {@link FrontendMessage} records for+ * {@link RedshiftInterceptingBridge}, which forwards each message opaque except a Simple Query+ * ({@code 'Q'}): that one is rewritten via {@link RedshiftSqlInterceptor} and re-encoded with+ * {@link #encodeQuery(String)}.+ *+ * <p>The DDL path never reads the backend socket directly, so this decoder does no shared-heap+ * accounting: {@link #MAX_MESSAGE_BYTES} is the only guard, refusing an oversized declared length+ * before any body is read (the length field is attacker-controlled, up to ~2 GiB).+ */+public class PostgresWireDecoder {++ /** Largest single frontend message accepted; 16 MiB is far above any real SQL statement. */+ static final int MAX_MESSAGE_BYTES = 16 * 1024 * 1024;++ private final InputStream in;+ private boolean betweenMessages = true;++ public PostgresWireDecoder(InputStream in) {+ this.in = Objects.requireNonNull(in, "in must not be null");+ }++ /**+ * {@code true} when the decoder is at a clean boundary between messages: before the first+ * byte of a message and after a full message has been returned; {@code false} once a type+ * byte has been consumed. Lets the bridge tell "client idle between queries" from "client+ * stalled mid-message" when a read times out.+ */+ public boolean isBetweenMessages() {+ return betweenMessages;+ }++ /**+ * Read the next frontend message.+ *+ * @return the message, or {@code null} on a clean end-of-stream between messages.+ * @throws EOFException if EOF is hit inside the length field or body.+ * @throws IOException on an I/O error, a length below 4, or a length above {@link #MAX_MESSAGE_BYTES}.+ */+ public FrontendMessage nextMessage() throws IOException {+ return nextMessage(null);+ }++ /**+ * Read the next frontend message.+ *+ * <p>When {@code passthroughOut} is non-null and the incoming message is not a Simple Query+ * ({@code 'Q'}), its frame is streamed directly to {@code passthroughOut} without heap buffering+ * or size caps, preserving transparent passthrough for large messages such as {@code CopyData}.+ *+ * @param passthroughOut destination for opaque non-query frames, or {@code null} to buffer+ * @return the message, or {@code null} on a clean end-of-stream between messages.+ * @throws EOFException if EOF is hit inside the length field or body.+ * @throws IOException on an I/O error, a length below 4, or a length above {@link #MAX_MESSAGE_BYTES}.+ */+ public FrontendMessage nextMessage(OutputStream passthroughOut) throws IOException {+ betweenMessages = true;++ int typeByte = in.read();+ if (typeByte == -1) {+ return null; // clean EOF between messages+ }+ betweenMessages = false;+ char type = (char) typeByte;++ byte[] lengthBytes = readFully(4);+ int length = ((lengthBytes[0] & 0xFF) << 24)+ | ((lengthBytes[1] & 0xFF) << 16)+ | ((lengthBytes[2] & 0xFF) << 8)+ | (lengthBytes[3] & 0xFF);++ if (length < 4) {+ throw new IOException("Invalid message length: " + length);+ }++ if (passthroughOut != null && (type != 'Q' || length > MAX_MESSAGE_BYTES)) {+ passthroughOut.write(typeByte);+ passthroughOut.write(lengthBytes);+ int remaining = length - 4;+ if (remaining > 0) {+ byte[] buf = new byte[Math.min(remaining, 8192)];+ while (remaining > 0) {+ int toRead = Math.min(remaining, buf.length);+ int read = in.read(buf, 0, toRead);+ if (read == -1) {+ throw new EOFException("Unexpected EOF (expected " + (length - 4)+ + " bytes, got " + (length - 4 - remaining) + ")");+ }+ passthroughOut.write(buf, 0, read);+ remaining -= read;+ }+ }+ passthroughOut.flush();+ betweenMessages = true;+ return new FrontendMessage(type, null);+ }++ if (length > MAX_MESSAGE_BYTES) {+ throw new IOException("Refusing PostgreSQL message of " + length+ + " bytes (limit " + MAX_MESSAGE_BYTES + ")");+ }++ byte[] body = (length == 4) ? new byte[0] : readFully(length - 4);+ betweenMessages = true;+ return new FrontendMessage(type, body);+ }++ /** Read exactly {@code n} bytes or throw {@link EOFException}. */+ private byte[] readFully(int n) throws IOException {+ byte[] buf = new byte[n];+ int off = 0;+ while (off < n) {+ int read = in.read(buf, off, n - off);+ if (read == -1) {+ throw new EOFException("Unexpected EOF (expected " + n + " bytes, got " + off + ")");+ }+ off += read;+ }+ return buf;+ }++ /** Encode an SQL string as a Simple Query ({@code 'Q'}) packet with a NUL-terminated body. */+ public static byte[] encodeQuery(String sql) {+ if (sql == null) {+ sql = "";+ }+ byte[] sqlBytes = sql.getBytes(StandardCharsets.UTF_8);+ int bodyLength = sqlBytes.length + 1;+ int totalLength = 4 + bodyLength;++ byte[] packet = new byte[1 + 4 + bodyLength];+ packet[0] = 'Q';+ packet[1] = (byte) ((totalLength >> 24) & 0xFF);+ packet[2] = (byte) ((totalLength >> 16) & 0xFF);+ packet[3] = (byte) ((totalLength >> 8) & 0xFF);+ packet[4] = (byte) (totalLength & 0xFF);+ System.arraycopy(sqlBytes, 0, packet, 5, sqlBytes.length);+ packet[packet.length - 1] = 0x00;+ return packet;+ }++ /** A single PostgreSQL wire message from the client. */+ public record FrontendMessage(char type, byte[] body) {++ public boolean isQuery() {+ return type == 'Q' && body != null;+ }++ /** SQL text of a {@code 'Q'} (trailing NUL stripped), or {@code null} if this is not a {@code 'Q'}. */+ public String getSql() {+ if (!isQuery() || body == null || body.length == 0) {+ return null;+ }+ int len = body.length;+ if (body[len - 1] == 0) {+ len--;+ }+ return new String(body, 0, len, StandardCharsets.UTF_8);+ }++ /** {@code type · length · body}: a byte-exact round-trip of the original frame. */+ public byte[] toPacketBytes() {+ if (body == null) {+ throw new IllegalStateException("Message body was streamed to passthrough output and not retained");+ }+ int bodyLen = body.length;+ int lengthField = 4 + bodyLen;+ byte[] packet = new byte[1 + 4 + bodyLen];+ packet[0] = (byte) type;+ packet[1] = (byte) ((lengthField >> 24) & 0xFF);+ packet[2] = (byte) ((lengthField >> 16) & 0xFF);+ packet[3] = (byte) ((lengthField >> 8) & 0xFF);+ packet[4] = (byte) (lengthField & 0xFF);+ if (bodyLen > 0) {+ System.arraycopy(body, 0, packet, 5, bodyLen);+ }+ return packet;+ }+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.javaindex 1b8573857c..ba84abb089 100644--- a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java+++ b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java@@ -142,7 +142,9 @@ public class RedshiftAuthProxy {client, backend, masterUsername, masterPassword, dbName,false, sigV4, tlsCertificates, passwordValidator::validate);if (activeClient != null) {- PostgresProtocolHandler.bridge(activeClient, backend);+ // Redshift-only DDL (DISTKEY/SORTKEY/ENCODE/...) is rewritten for the plain+ // PostgreSQL backend on the way through; every other message is relayed verbatim.+ new RedshiftInterceptingBridge(activeClient, backend).run();}} catch (Exception e) {LOG.debugv("Redshift connection error for cluster {0}: {1}", clusterKey, e.getMessage());diff --git a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridge.java b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridge.javanew file mode 100644index 0000000000..c554f873b8--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridge.java@@ -0,0 +1,117 @@+package io.github.hectorvent.floci.services.redshift.proxy;++import org.jboss.logging.Logger;++import java.io.IOException;+import java.io.InputStream;+import java.io.OutputStream;+import java.net.Socket;+import java.util.concurrent.atomic.AtomicBoolean;++/**+ * Replaces the transparent {@code PostgresProtocolHandler.bridge} for Redshift connections so+ * Simple Query ({@code 'Q'}) DDL can be rewritten for a plain PostgreSQL backend before it is+ * forwarded.+ *+ * <p><b>Model.</b> Backend→client is a verbatim byte pump on a virtual thread, exactly as a+ * plain proxy: extended-protocol pipelines, {@code COPY … FROM STDIN}, asynchronous+ * {@code NotificationResponse} and backend EOF all flow through untouched. Client→backend is+ * a framed loop: every frontend message is forwarded opaque <em>except</em> a {@code 'Q'}, whose+ * SQL is run through {@link RedshiftSqlInterceptor#rewrite} and re-encoded only if it changed.+ *+ * <p><b>Fail-open.</b> A {@code rewrite} failure, or any statement the interceptor leaves+ * untouched, forwards the original bytes; PostgreSQL then returns its own error. The bridge never+ * closes the connection because of a rewrite failure.+ */+public class RedshiftInterceptingBridge {++ private static final Logger LOG = Logger.getLogger(RedshiftInterceptingBridge.class);++ private final Socket client;+ private final Socket backend;+ private final AtomicBoolean closed = new AtomicBoolean();++ public RedshiftInterceptingBridge(Socket client, Socket backend) {+ this.client = client;+ this.backend = backend;+ }++ public void run() {+ try {+ InputStream clientIn = client.getInputStream();+ OutputStream backendOut = backend.getOutputStream();++ Thread.ofVirtual().name("redshift-pump-backend-to-client").start(this::pumpBackendToClient);++ PostgresWireDecoder decoder = new PostgresWireDecoder(clientIn);+ while (true) {+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage(backendOut);+ if (msg == null) {+ break; // client EOF+ }++ if (!msg.isQuery()) {+ if (msg.type() == 'X') { // Terminate+ break;+ }+ continue;+ }++ String sql = msg.getSql();+ byte[] toBackend = msg.toPacketBytes();+ try {+ String rewritten = RedshiftSqlInterceptor.rewrite(sql);+ if (rewritten != sql) { // identity: rewrite returns the same instance when nothing matched+ toBackend = PostgresWireDecoder.encodeQuery(rewritten);+ }+ } catch (RuntimeException e) {+ LOG.warnv("RedshiftSqlInterceptor failed, forwarding original query: {0}", e.getMessage());+ toBackend = msg.toPacketBytes();+ }+ backendOut.write(toBackend);+ backendOut.flush();+ // The response returns over the untouched pump.+ }+ } catch (IOException e) {+ LOG.debugv(e, "RedshiftInterceptingBridge client loop ended");+ } catch (Exception e) {+ LOG.warnv(e, "Unexpected error in RedshiftInterceptingBridge");+ } finally {+ closeBoth();+ }+ }++ private void pumpBackendToClient() {+ try {+ InputStream backendIn = backend.getInputStream();+ OutputStream clientOut = client.getOutputStream();+ byte[] buf = new byte[8192];+ int n;+ while ((n = backendIn.read(buf)) != -1) {+ clientOut.write(buf, 0, n);+ clientOut.flush();+ }+ } catch (IOException e) {+ LOG.debugv(e, "backend->client pump ended");+ } finally {+ closeBoth();+ }+ }++ private void closeBoth() {+ if (closed.compareAndSet(false, true)) {+ closeQuietly(client, "client");+ closeQuietly(backend, "backend");+ }+ }++ private void closeQuietly(Socket s, String which) {+ try {+ if (s != null && !s.isClosed()) {+ s.close();+ }+ } catch (IOException e) {+ LOG.debugv(e, "error closing {0} socket", which);+ }+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.javaindex 2e0763cbde..7d8716beda 100644--- a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java@@ -17,8 +17,10 @@ import java.sql.SQLException;import java.sql.Statement;import java.sql.Types;import java.time.Duration;+import java.util.Properties;import java.util.UUID;import java.util.concurrent.atomic.AtomicBoolean;+import org.h2.Driver;import static org.junit.jupiter.api.Assertions.assertArrayEquals;import static org.junit.jupiter.api.Assertions.assertEquals;@@ -33,6 +35,10 @@ import static org.mockito.Mockito.when;class RdsDataServiceTest {+ static {+ Driver.load();+ }+private static final String RESOURCE_ARN = "arn:aws:rds:us-east-1:000000000000:cluster:test";private static final String FALLBACK_RESOURCE_ARN = "arn:aws:rds:us-west-2:111111111111:cluster:test";private static final String OTHER_RESOURCE_ARN = "arn:aws:rds:us-east-1:000000000000:cluster:other";@@ -864,14 +870,29 @@ class RdsDataServiceTest {String username,String password,String database) throws SQLException {- return DriverManager.getConnection(jdbcUrl, "sa", "");+ return getConnection();}};service = new RdsDataService(resolver, secrets, objectMapper, connectionFactory, transactionTtl);}+ private Connection getConnection() throws SQLException {+ try {+ return DriverManager.getConnection(jdbcUrl, "sa", "");+ } catch (SQLException e) {+ Properties props = new Properties();+ props.setProperty("user", "sa");+ props.setProperty("password", "");+ Connection conn = new Driver().connect(jdbcUrl, props);+ if (conn != null) {+ return conn;+ }+ throw e;+ }+ }+private void createTables() throws SQLException {- try (Connection connection = DriverManager.getConnection(jdbcUrl, "sa", "");+ try (Connection connection = getConnection();Statement statement = connection.createStatement()) {statement.execute("""create table data_api_items(@@ -897,7 +918,7 @@ class RdsDataServiceTest {* harnesses standing in for PostgreSQL.*/private void createEventsTable() throws SQLException {- try (Connection connection = DriverManager.getConnection(jdbcUrl, "sa", "");+ try (Connection connection = getConnection();Statement statement = connection.createStatement()) {statement.execute("""create table data_api_events(diff --git a/src/test/java/io/github/hectorvent/floci/services/redshift/RedshiftInterceptorIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/redshift/RedshiftInterceptorIntegrationTest.javanew file mode 100644index 0000000000..b2a3b854e3--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/redshift/RedshiftInterceptorIntegrationTest.java@@ -0,0 +1,109 @@+package io.github.hectorvent.floci.services.redshift;++import io.github.hectorvent.floci.services.redshift.model.Cluster;+import io.quarkus.test.junit.QuarkusTest;+import jakarta.inject.Inject;+import org.awaitility.Awaitility;+import org.awaitility.core.ConditionTimeoutException;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.Assumptions;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;++import java.sql.Connection;+import java.sql.DriverManager;+import java.sql.ResultSet;+import java.sql.SQLException;+import java.sql.Statement;+import java.time.Duration;+import java.util.Objects;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;++@QuarkusTest+class RedshiftInterceptorIntegrationTest {++ @Inject+ RedshiftService service;++ private String clusterId;++ @BeforeAll+ static void requireDocker() {+ Assumptions.assumeTrue(isDockerAvailable(), "Docker daemon must be available for Redshift interceptor integration tests");+ }++ private static boolean isDockerAvailable() {+ try {+ Process process = new ProcessBuilder("docker", "version", "--format", "{{.Server.Version}}")+ .redirectErrorStream(true)+ .start();+ int exit = process.waitFor();+ return exit == 0;+ } catch (Exception e) {+ return false;+ }+ }++ @AfterEach+ void cleanUp() {+ if (clusterId != null) {+ service.deleteCluster(clusterId);+ }+ }++ private static String jdbcUrl(Cluster c) {+ // Use 127.0.0.1 explicitly instead of c.getEndpoint().getAddress() to avoid UnknownHostException+ // in CI environments where floci.emulator.hostname is set to host.docker.internal.+ // preferQueryMode=simple forces pgjdbc to use the simple query protocol ('Q' messages)+ // rather than extended query protocol ('P'/'B'/'E'/'S' messages).+ return "jdbc:postgresql://127.0.0.1:" + c.getEndpoint().getPort() + "/dev?preferQueryMode=simple";+ }++ private static Connection waitForConnection(Cluster cluster, String username, String password) throws SQLException {+ try {+ return Awaitility.await()+ .atMost(Duration.ofSeconds(30))+ .pollInterval(Duration.ofMillis(500))+ .ignoreExceptions()+ .until(() -> DriverManager.getConnection(jdbcUrl(cluster), username, password), Objects::nonNull);+ } catch (ConditionTimeoutException e) {+ return DriverManager.getConnection(jdbcUrl(cluster), username, password); // throw original+ }+ }++ @Test+ void rewritesCreateTableDdlOverSimpleQueryProtocol() throws SQLException {+ clusterId = "it-interceptor-create";+ Cluster cluster = service.createCluster(clusterId, "dc2.large", "admin", "Secret123");++ try (Connection conn = waitForConnection(cluster, "admin", "Secret123");+ Statement st = conn.createStatement()) {+ st.execute("CREATE TABLE sales (id int ENCODE az64, d date) DISTSTYLE KEY DISTKEY (id) COMPOUND SORTKEY (d);");+ st.execute("INSERT INTO sales VALUES (1, '2026-01-01');");+ try (ResultSet rs = st.executeQuery("SELECT count(*) FROM sales")) {+ assertTrue(rs.next());+ assertEquals(1, rs.getInt(1));+ }+ }+ }++ @Test+ void rewritesAlterTableDdlOverSimpleQueryProtocol() throws SQLException {+ clusterId = "it-interceptor-alter";+ Cluster cluster = service.createCluster(clusterId, "dc2.large", "admin", "Secret123");++ try (Connection conn = waitForConnection(cluster, "admin", "Secret123");+ Statement st = conn.createStatement()) {+ st.execute("CREATE TABLE t (id int)");+ st.execute("ALTER TABLE t ADD COLUMN note varchar(20) ENCODE lzo;");+ st.execute("INSERT INTO t VALUES (1, 'test-note');");+ try (ResultSet rs = st.executeQuery("SELECT id, note FROM t WHERE id = 1")) {+ assertTrue(rs.next());+ assertEquals(1, rs.getInt(1));+ assertEquals("test-note", rs.getString(2));+ }+ }+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoderTest.java b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoderTest.javanew file mode 100644index 0000000000..3ebd828056--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoderTest.java@@ -0,0 +1,311 @@+package io.github.hectorvent.floci.services.redshift.proxy;++import org.junit.jupiter.api.Test;++import java.io.ByteArrayInputStream;+import java.io.ByteArrayOutputStream;+import java.io.EOFException;+import java.io.IOException;+import java.io.InputStream;+import java.io.OutputStream;+import java.nio.charset.StandardCharsets;++import static org.junit.jupiter.api.Assertions.assertArrayEquals;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertNotNull;+import static org.junit.jupiter.api.Assertions.assertNull;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;++class PostgresWireDecoderTest {++ @Test+ void decodesASimpleQueryAndThenReportsEof() throws IOException {+ byte[] packet = PostgresWireDecoder.encodeQuery("SELECT 1");+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(packet));++ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage();+ assertNotNull(msg);+ assertEquals('Q', msg.type());+ assertTrue(msg.isQuery());+ assertEquals("SELECT 1", msg.getSql());+ assertArrayEquals(packet, msg.toPacketBytes());++ assertNull(decoder.nextMessage());+ }++ @Test+ void decodesNonQueryMessagesWithoutInterpretingThem() throws IOException {+ byte[] terminate = new byte[]{'X', 0, 0, 0, 4};+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(terminate));++ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage();+ assertNotNull(msg);+ assertEquals('X', msg.type());+ assertFalse(msg.isQuery());+ assertNull(msg.getSql());+ assertEquals(0, msg.body().length);+ assertArrayEquals(terminate, msg.toPacketBytes());+ assertNull(decoder.nextMessage());++ byte[] parsePayload = "stmt1\0SELECT $1\0\0\0".getBytes(StandardCharsets.UTF_8);+ int length = 4 + parsePayload.length;+ byte[] parsePacket = new byte[1 + length];+ parsePacket[0] = 'P';+ parsePacket[1] = (byte) ((length >> 24) & 0xFF);+ parsePacket[2] = (byte) ((length >> 16) & 0xFF);+ parsePacket[3] = (byte) ((length >> 8) & 0xFF);+ parsePacket[4] = (byte) (length & 0xFF);+ System.arraycopy(parsePayload, 0, parsePacket, 5, parsePayload.length);++ PostgresWireDecoder.FrontendMessage parse =+ new PostgresWireDecoder(new ByteArrayInputStream(parsePacket)).nextMessage();+ assertNotNull(parse);+ assertEquals('P', parse.type());+ assertFalse(parse.isQuery());+ assertArrayEquals(parsePayload, parse.body());+ assertArrayEquals(parsePacket, parse.toPacketBytes());+ }++ @Test+ void decodesSeveralMessagesFromOneStream() throws IOException {+ ByteArrayOutputStream out = new ByteArrayOutputStream();+ out.write(PostgresWireDecoder.encodeQuery("SELECT 1"));+ out.write(PostgresWireDecoder.encodeQuery("SELECT 2"));+ out.write(new byte[]{'X', 0, 0, 0, 4});++ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(out.toByteArray()));+ assertEquals("SELECT 1", decoder.nextMessage().getSql());+ assertEquals("SELECT 2", decoder.nextMessage().getSql());+ assertEquals('X', decoder.nextMessage().type());+ assertNull(decoder.nextMessage());+ }++ @Test+ void reassemblesAMessageDeliveredTwoBytesAtATime() throws IOException {+ byte[] packet = PostgresWireDecoder.encodeQuery("SELECT * FROM users WHERE active = true");+ InputStream drip = new InputStream() {+ private int i = 0;+ @Override public int read() {+ if (i >= packet.length) {+ return -1;+ }+ return packet[i++] & 0xFF;+ }+ @Override public int read(byte[] b, int off, int len) {+ if (i >= packet.length) {+ return -1;+ }+ int n = Math.min(Math.min(len, 2), packet.length - i);+ System.arraycopy(packet, i, b, off, n);+ i += n;+ return n;+ }+ };+ PostgresWireDecoder.FrontendMessage msg = new PostgresWireDecoder(drip).nextMessage();+ assertEquals("SELECT * FROM users WHERE active = true", msg.getSql());+ assertArrayEquals(packet, msg.toPacketBytes());+ }++ @Test+ void returnsNullOnAnEmptyStream() throws IOException {+ assertNull(new PostgresWireDecoder(new ByteArrayInputStream(new byte[0])).nextMessage());+ }++ @Test+ void throwsEofWhenTheLengthFieldIsTruncated() {+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(new byte[]{'Q', 0, 0}));+ assertThrows(EOFException.class, decoder::nextMessage);+ }++ @Test+ void throwsEofWhenTheBodyIsTruncated() {+ PostgresWireDecoder decoder =+ new PostgresWireDecoder(new ByteArrayInputStream(new byte[]{'Q', 0, 0, 0, 10, 'S', 'E'}));+ assertThrows(EOFException.class, decoder::nextMessage);+ }++ @Test+ void rejectsALengthBelowFour() {+ PostgresWireDecoder decoder =+ new PostgresWireDecoder(new ByteArrayInputStream(new byte[]{'Q', 0, 0, 0, 2}));+ assertThrows(IOException.class, decoder::nextMessage);+ }++ @Test+ void refusesAnOversizedDeclaredLengthBeforeReadingTheBody() {+ byte[] hostile = new byte[]{'Q', 0x7F, (byte) 0xFF, (byte) 0xFF, (byte) 0xFF};+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(hostile));+ IOException ex = assertThrows(IOException.class, decoder::nextMessage);+ assertTrue(ex.getMessage().contains("Refusing"), ex.getMessage());+ }++ @Test+ void encodeQueryProducesAWellFormedQPacket() {+ byte[] encoded = PostgresWireDecoder.encodeQuery("SHOW search_path");+ assertEquals('Q', (char) encoded[0]);+ int length = ((encoded[1] & 0xFF) << 24) | ((encoded[2] & 0xFF) << 16)+ | ((encoded[3] & 0xFF) << 8) | (encoded[4] & 0xFF);+ assertEquals(21, length); // 4 + 16 chars + 1 NUL+ assertEquals(0x00, encoded[encoded.length - 1]);+ }++ @Test+ void isBetweenMessagesIsTrueAtBoundariesAndFalseMidMessage() throws IOException {+ byte[] two = new ByteArrayOutputStream() {{+ try {+ write(PostgresWireDecoder.encodeQuery("SELECT 1"));+ write(new byte[]{'Q', 0, 0, 0, 50}); // header promises 46 body bytes that never arrive+ } catch (IOException e) {+ throw new RuntimeException(e);+ }+ }}.toByteArray();++ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(two));+ assertTrue(decoder.isBetweenMessages());+ assertNotNull(decoder.nextMessage());+ assertTrue(decoder.isBetweenMessages());+ assertThrows(EOFException.class, decoder::nextMessage);+ assertFalse(decoder.isBetweenMessages()); // type byte was consumed before the stream ran dry+ }++ @Test+ void streamsOpaqueMessageLargerThanQueryLimitDirectlyToPassthroughOut() throws IOException {+ int payloadSize = 17 * 1024 * 1024; // 17 MiB, exceeds MAX_MESSAGE_BYTES+ int totalLength = 4 + payloadSize;+ byte[] header = new byte[]{+ 'd', // CopyData+ (byte) ((totalLength >> 24) & 0xFF),+ (byte) ((totalLength >> 16) & 0xFF),+ (byte) ((totalLength >> 8) & 0xFF),+ (byte) (totalLength & 0xFF)+ };++ InputStream in = new InputStream() {+ private int headerPos = 0;+ private int bodyRemaining = payloadSize;++ @Override+ public int read() {+ if (headerPos < header.length) {+ return header[headerPos++] & 0xFF;+ }+ if (bodyRemaining > 0) {+ bodyRemaining--;+ return 'A';+ }+ return -1;+ }++ @Override+ public int read(byte[] b, int off, int len) {+ if (headerPos < header.length) {+ int toCopy = Math.min(len, header.length - headerPos);+ System.arraycopy(header, headerPos, b, off, toCopy);+ headerPos += toCopy;+ return toCopy;+ }+ if (bodyRemaining > 0) {+ int toProvide = Math.min(len, bodyRemaining);+ for (int i = 0; i < toProvide; i++) {+ b[off + i] = 'A';+ }+ bodyRemaining -= toProvide;+ return toProvide;+ }+ return -1;+ }+ };++ long[] bytesWritten = new long[1];+ OutputStream out = new OutputStream() {+ @Override+ public void write(int b) {+ bytesWritten[0]++;+ }++ @Override+ public void write(byte[] b, int off, int len) {+ bytesWritten[0] += len;+ }+ };++ PostgresWireDecoder decoder = new PostgresWireDecoder(in);+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage(out);+ assertNotNull(msg);+ assertEquals('d', msg.type());+ assertNull(msg.body());+ assertEquals(1 + (long) totalLength, bytesWritten[0]);+ }++ @Test+ void streamsSimpleQueryLargerThanLimitDirectlyToPassthroughOut() throws IOException {+ int payloadSize = 17 * 1024 * 1024; // 17 MiB, exceeds MAX_MESSAGE_BYTES+ int totalLength = 4 + payloadSize;+ byte[] header = new byte[]{+ 'Q',+ (byte) ((totalLength >> 24) & 0xFF),+ (byte) ((totalLength >> 16) & 0xFF),+ (byte) ((totalLength >> 8) & 0xFF),+ (byte) (totalLength & 0xFF)+ };++ InputStream in = new InputStream() {+ private int headerPos = 0;+ private int bodyRemaining = payloadSize;++ @Override+ public int read() {+ if (headerPos < header.length) {+ return header[headerPos++] & 0xFF;+ }+ if (bodyRemaining > 0) {+ bodyRemaining--;+ return 'A';+ }+ return -1;+ }++ @Override+ public int read(byte[] b, int off, int len) {+ if (headerPos < header.length) {+ int toCopy = Math.min(len, header.length - headerPos);+ System.arraycopy(header, headerPos, b, off, toCopy);+ headerPos += toCopy;+ return toCopy;+ }+ if (bodyRemaining > 0) {+ int toProvide = Math.min(len, bodyRemaining);+ for (int i = 0; i < toProvide; i++) {+ b[off + i] = 'A';+ }+ bodyRemaining -= toProvide;+ return toProvide;+ }+ return -1;+ }+ };++ long[] bytesWritten = new long[1];+ OutputStream out = new OutputStream() {+ @Override+ public void write(int b) {+ bytesWritten[0]++;+ }++ @Override+ public void write(byte[] b, int off, int len) {+ bytesWritten[0] += len;+ }+ };++ PostgresWireDecoder decoder = new PostgresWireDecoder(in);+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage(out);+ assertNotNull(msg);+ assertEquals('Q', msg.type());+ assertFalse(msg.isQuery());+ assertNull(msg.body());+ assertEquals(1 + (long) totalLength, bytesWritten[0]);+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridgeTest.java b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridgeTest.javanew file mode 100644index 0000000000..b11b9fbe29--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridgeTest.java@@ -0,0 +1,141 @@+package io.github.hectorvent.floci.services.redshift.proxy;++import org.jboss.logging.Logger;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.Test;++import java.io.IOException;+import java.net.ServerSocket;+import java.net.Socket;+import java.nio.charset.StandardCharsets;++import static org.junit.jupiter.api.Assertions.assertArrayEquals;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertNotNull;+import static org.junit.jupiter.api.Assertions.assertTrue;++class RedshiftInterceptingBridgeTest {++ private static final Logger LOG = Logger.getLogger(RedshiftInterceptingBridgeTest.class);++ private ServerSocket clientListener;+ private ServerSocket backendListener;+ private Socket testClientEnd; // test writes client bytes here, reads pump output here+ private Socket bridgeClientEnd; // the bridge's "client" socket+ private Socket bridgeBackendEnd; // the bridge's "backend" socket+ private Socket testBackendEnd; // test reads what the bridge forwarded, writes backend replies+ private Thread bridgeThread;++ private void startBridge() throws IOException {+ clientListener = new ServerSocket(0);+ testClientEnd = new Socket("localhost", clientListener.getLocalPort());+ bridgeClientEnd = clientListener.accept();++ backendListener = new ServerSocket(0);+ bridgeBackendEnd = new Socket("localhost", backendListener.getLocalPort());+ testBackendEnd = backendListener.accept();++ RedshiftInterceptingBridge bridge = new RedshiftInterceptingBridge(bridgeClientEnd, bridgeBackendEnd);+ bridgeThread = Thread.ofVirtual().name("bridge-under-test").start(bridge::run);+ }++ @AfterEach+ void tearDown() {+ for (Socket s : new Socket[]{testClientEnd, bridgeClientEnd, bridgeBackendEnd, testBackendEnd}) {+ if (s != null && !s.isClosed()) {+ try {+ s.close();+ } catch (IOException e) {+ LOG.debugv(e, "Error closing socket during test cleanup");+ }+ }+ }+ for (ServerSocket ss : new ServerSocket[]{clientListener, backendListener}) {+ if (ss != null && !ss.isClosed()) {+ try {+ ss.close();+ } catch (IOException e) {+ LOG.debugv(e, "Error closing server socket during test cleanup");+ }+ }+ }+ }++ private PostgresWireDecoder.FrontendMessage nextForwarded() throws IOException {+ return new PostgresWireDecoder(testBackendEnd.getInputStream()).nextMessage();+ }++ @Test+ void rewritesRedshiftCreateTableBeforeForwarding() throws IOException {+ startBridge();+ String ddl = "CREATE TABLE sales (id int ENCODE az64, d date) "+ + "DISTSTYLE KEY DISTKEY (id) COMPOUND SORTKEY (d);";+ testClientEnd.getOutputStream().write(PostgresWireDecoder.encodeQuery(ddl));+ testClientEnd.getOutputStream().flush();++ PostgresWireDecoder.FrontendMessage forwarded = nextForwarded();+ assertNotNull(forwarded);+ assertEquals('Q', forwarded.type());+ String sent = forwarded.getSql().toUpperCase();+ assertFalse(sent.contains("DISTKEY"), sent);+ assertFalse(sent.contains("SORTKEY"), sent);+ assertFalse(sent.contains("DISTSTYLE"), sent);+ assertFalse(sent.contains("ENCODE"), sent);+ assertTrue(forwarded.getSql().contains("CREATE TABLE sales"), forwarded.getSql());+ }++ @Test+ void forwardsANonDdlQueryByteForByte() throws IOException {+ startBridge();+ byte[] packet = PostgresWireDecoder.encodeQuery("SELECT 'DISTKEY' AS not_a_keyword");+ testClientEnd.getOutputStream().write(packet);+ testClientEnd.getOutputStream().flush();++ assertArrayEquals(packet, nextForwarded().toPacketBytes());+ }++ @Test+ void forwardsAnExtendedProtocolMessageOpaque() throws IOException {+ startBridge();+ byte[] parsePayload = "s1\0SELECT $1\0\0\0".getBytes(StandardCharsets.UTF_8);+ int length = 4 + parsePayload.length;+ byte[] parsePacket = new byte[1 + length];+ parsePacket[0] = 'P';+ parsePacket[1] = (byte) ((length >> 24) & 0xFF);+ parsePacket[2] = (byte) ((length >> 16) & 0xFF);+ parsePacket[3] = (byte) ((length >> 8) & 0xFF);+ parsePacket[4] = (byte) (length & 0xFF);+ System.arraycopy(parsePayload, 0, parsePacket, 5, parsePayload.length);++ testClientEnd.getOutputStream().write(parsePacket);+ testClientEnd.getOutputStream().flush();++ assertArrayEquals(parsePacket, nextForwarded().toPacketBytes());+ }++ @Test+ void pumpsBackendBytesToTheClientUnchanged() throws IOException {+ startBridge();+ byte[] readyForQuery = new byte[]{'Z', 0, 0, 0, 5, 'I'};+ testBackendEnd.getOutputStream().write(readyForQuery);+ testBackendEnd.getOutputStream().flush();++ byte[] got = testClientEnd.getInputStream().readNBytes(readyForQuery.length);+ assertArrayEquals(readyForQuery, got);+ }++ @Test+ void terminateMessageIsForwardedAndEndsTheSession() throws Exception {+ startBridge();+ testClientEnd.getOutputStream().write(new byte[]{'X', 0, 0, 0, 4});+ testClientEnd.getOutputStream().flush();++ PostgresWireDecoder.FrontendMessage forwarded = nextForwarded();+ assertEquals('X', forwarded.type());++ bridgeThread.join(5_000);+ assertFalse(bridgeThread.isAlive(), "bridge did not stop after Terminate");+ assertEquals(-1, testClientEnd.getInputStream().read(), "bridge left the client socket open");+ }+}