Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(redshift): wire the Simple Query DDL interceptor into the proxy (#3029)

Vũ Thế Huy4daba7ed6a3b
diff --git a/docs/services/redshift.md b/docs/services/redshift.md
index 5e1a295b6e..01db4613f2 100644
--- a/docs/services/redshift.md
+++ b/docs/services/redshift.md
@@ -135,9 +135,27 @@ cluster = redshift.create_cluster(
print(cluster["Cluster"]["Endpoint"])
```
+## SQL Interceptor
+
+Floci's Redshift auth proxy inspects frontend queries on the PostgreSQL wire protocol (Simple Query `'Q'` protocol) and rewrites common Redshift-specific table DDL so it runs on the plain PostgreSQL backend.
+
+### DDL compatibility
+
+- Redshift-only table DDL keywords are stripped before the statement is forwarded: `DISTSTYLE ALL|EVEN|KEY|AUTO`, `DISTKEY (<col>)` and column-level `DISTKEY`, `[COMPOUND|INTERLEAVED] SORTKEY (<cols>)` and column-level `SORTKEY`, and `ENCODE <codec>` for the real Redshift column encodings (`raw`, `az64`, `bytedict`, `delta`, `delta32k`, `lzo`, `mostly8`, `mostly16`, `mostly32`, `runlength`, `text255`, `text32k`, `zstd`) or `auto`.
+- The rewrite only runs when the statement's first keyword is `CREATE TABLE` or `ALTER TABLE`. A `SELECT`, `INSERT`, function body, or string literal that merely contains one of these keywords is forwarded byte-for-byte. Single-quoted and dollar-quoted string literals are masked before the rewrite, so a keyword inside a quoted value (including in a later statement of a multi-statement query) is preserved.
+- Columns legitimately named `distkey`, `sortkey`, or `encode` survive.
+
+### Limitations
+
+- Emulation runs on the **Simple Query protocol** (`'Q'`) only. Extended Query protocol statements (`Parse`/`Bind`/`Execute`) pass through untouched, including anything a JDBC `PreparedStatement` sends, and, with the pgjdbc default `preferQueryMode=extended`, plain `Statement` calls too. Connect with `preferQueryMode=simple` to exercise the interceptor from JDBC.
+- The rewrite is textual (regex-based). It masks single-quoted string literals first, so `DEFAULT` / `CHECK` string values are safe, but it is **not** comment-aware and does not recognize escape strings (`E'...'`): an apostrophe inside a `--` or `/* */` comment can make the rewrite skip a Redshift clause. That fails safe: the statement then reaches PostgreSQL, which returns its own syntax error, but avoid apostrophes-in-comments in `CREATE TABLE` / `ALTER TABLE`.
+- A `rewrite` failure or any statement the interceptor does not recognize is forwarded unmodified (fail-open); PostgreSQL then rejects the Redshift-only syntax itself.
+- Simple Query ('Q') messages larger than 16 MiB bypass the interceptor and stream through verbatim without heap buffering; non-query traffic also streams through with no size limit.
+- `COPY ... FROM 's3://...'` and `UNLOAD (...) TO 's3://...'` are **not** yet emulated (planned).
+
## Out of Scope
-- Real Redshift SQL semantics — the data plane is stock PostgreSQL, so Redshift-only SQL (distribution/sort keys, `COPY`/`UNLOAD` from S3, `SUPER`/`SPECTRUM`) is not emulated.
+- Real Redshift SQL semantics: the data plane is stock PostgreSQL. Redshift-only table DDL keywords (DISTSTYLE / DISTKEY / SORTKEY / ENCODE) are stripped so CREATE TABLE / ALTER TABLE executes (see [SQL Interceptor](#sql-interceptor)), but the distribution/sort behavior they request is not; COPY/UNLOAD from S3 and SUPER/SPECTRUM are not emulated.
- Multi-node clusters — `NodeType` and `NumberOfNodes` are stored as metadata; every cluster is a single PostgreSQL container.
- Parameter groups apply no real engine settings; values are stored and echoed back only.
- Subnet groups, VPC routing, and security groups are metadata only.
diff --git a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoder.java b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoder.java
new file mode 100644
index 0000000000..b6bc43f358
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoder.java
@@ -0,0 +1,192 @@
+package io.github.hectorvent.floci.services.redshift.proxy;
+
+import java.io.EOFException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+import java.util.Objects;
+
+/**
+ * Frames the PostgreSQL wire-protocol byte stream sent by the frontend (client).
+ *
+ * <p>Every frontend message is {@code type(1 byte) · length(int32, includes itself) · body(length-4)}.
+ * This decoder turns that stream into {@link FrontendMessage} records for
+ * {@link RedshiftInterceptingBridge}, which forwards each message opaque except a Simple Query
+ * ({@code 'Q'}): that one is rewritten via {@link RedshiftSqlInterceptor} and re-encoded with
+ * {@link #encodeQuery(String)}.
+ *
+ * <p>The DDL path never reads the backend socket directly, so this decoder does no shared-heap
+ * accounting: {@link #MAX_MESSAGE_BYTES} is the only guard, refusing an oversized declared length
+ * before any body is read (the length field is attacker-controlled, up to ~2 GiB).
+ */
+public class PostgresWireDecoder {
+
+ /** Largest single frontend message accepted; 16 MiB is far above any real SQL statement. */
+ static final int MAX_MESSAGE_BYTES = 16 * 1024 * 1024;
+
+ private final InputStream in;
+ private boolean betweenMessages = true;
+
+ public PostgresWireDecoder(InputStream in) {
+ this.in = Objects.requireNonNull(in, "in must not be null");
+ }
+
+ /**
+ * {@code true} when the decoder is at a clean boundary between messages: before the first
+ * byte of a message and after a full message has been returned; {@code false} once a type
+ * byte has been consumed. Lets the bridge tell "client idle between queries" from "client
+ * stalled mid-message" when a read times out.
+ */
+ public boolean isBetweenMessages() {
+ return betweenMessages;
+ }
+
+ /**
+ * Read the next frontend message.
+ *
+ * @return the message, or {@code null} on a clean end-of-stream between messages.
+ * @throws EOFException if EOF is hit inside the length field or body.
+ * @throws IOException on an I/O error, a length below 4, or a length above {@link #MAX_MESSAGE_BYTES}.
+ */
+ public FrontendMessage nextMessage() throws IOException {
+ return nextMessage(null);
+ }
+
+ /**
+ * Read the next frontend message.
+ *
+ * <p>When {@code passthroughOut} is non-null and the incoming message is not a Simple Query
+ * ({@code 'Q'}), its frame is streamed directly to {@code passthroughOut} without heap buffering
+ * or size caps, preserving transparent passthrough for large messages such as {@code CopyData}.
+ *
+ * @param passthroughOut destination for opaque non-query frames, or {@code null} to buffer
+ * @return the message, or {@code null} on a clean end-of-stream between messages.
+ * @throws EOFException if EOF is hit inside the length field or body.
+ * @throws IOException on an I/O error, a length below 4, or a length above {@link #MAX_MESSAGE_BYTES}.
+ */
+ public FrontendMessage nextMessage(OutputStream passthroughOut) throws IOException {
+ betweenMessages = true;
+
+ int typeByte = in.read();
+ if (typeByte == -1) {
+ return null; // clean EOF between messages
+ }
+ betweenMessages = false;
+ char type = (char) typeByte;
+
+ byte[] lengthBytes = readFully(4);
+ int length = ((lengthBytes[0] & 0xFF) << 24)
+ | ((lengthBytes[1] & 0xFF) << 16)
+ | ((lengthBytes[2] & 0xFF) << 8)
+ | (lengthBytes[3] & 0xFF);
+
+ if (length < 4) {
+ throw new IOException("Invalid message length: " + length);
+ }
+
+ if (passthroughOut != null && (type != 'Q' || length > MAX_MESSAGE_BYTES)) {
+ passthroughOut.write(typeByte);
+ passthroughOut.write(lengthBytes);
+ int remaining = length - 4;
+ if (remaining > 0) {
+ byte[] buf = new byte[Math.min(remaining, 8192)];
+ while (remaining > 0) {
+ int toRead = Math.min(remaining, buf.length);
+ int read = in.read(buf, 0, toRead);
+ if (read == -1) {
+ throw new EOFException("Unexpected EOF (expected " + (length - 4)
+ + " bytes, got " + (length - 4 - remaining) + ")");
+ }
+ passthroughOut.write(buf, 0, read);
+ remaining -= read;
+ }
+ }
+ passthroughOut.flush();
+ betweenMessages = true;
+ return new FrontendMessage(type, null);
+ }
+
+ if (length > MAX_MESSAGE_BYTES) {
+ throw new IOException("Refusing PostgreSQL message of " + length
+ + " bytes (limit " + MAX_MESSAGE_BYTES + ")");
+ }
+
+ byte[] body = (length == 4) ? new byte[0] : readFully(length - 4);
+ betweenMessages = true;
+ return new FrontendMessage(type, body);
+ }
+
+ /** Read exactly {@code n} bytes or throw {@link EOFException}. */
+ private byte[] readFully(int n) throws IOException {
+ byte[] buf = new byte[n];
+ int off = 0;
+ while (off < n) {
+ int read = in.read(buf, off, n - off);
+ if (read == -1) {
+ throw new EOFException("Unexpected EOF (expected " + n + " bytes, got " + off + ")");
+ }
+ off += read;
+ }
+ return buf;
+ }
+
+ /** Encode an SQL string as a Simple Query ({@code 'Q'}) packet with a NUL-terminated body. */
+ public static byte[] encodeQuery(String sql) {
+ if (sql == null) {
+ sql = "";
+ }
+ byte[] sqlBytes = sql.getBytes(StandardCharsets.UTF_8);
+ int bodyLength = sqlBytes.length + 1;
+ int totalLength = 4 + bodyLength;
+
+ byte[] packet = new byte[1 + 4 + bodyLength];
+ packet[0] = 'Q';
+ packet[1] = (byte) ((totalLength >> 24) & 0xFF);
+ packet[2] = (byte) ((totalLength >> 16) & 0xFF);
+ packet[3] = (byte) ((totalLength >> 8) & 0xFF);
+ packet[4] = (byte) (totalLength & 0xFF);
+ System.arraycopy(sqlBytes, 0, packet, 5, sqlBytes.length);
+ packet[packet.length - 1] = 0x00;
+ return packet;
+ }
+
+ /** A single PostgreSQL wire message from the client. */
+ public record FrontendMessage(char type, byte[] body) {
+
+ public boolean isQuery() {
+ return type == 'Q' && body != null;
+ }
+
+ /** SQL text of a {@code 'Q'} (trailing NUL stripped), or {@code null} if this is not a {@code 'Q'}. */
+ public String getSql() {
+ if (!isQuery() || body == null || body.length == 0) {
+ return null;
+ }
+ int len = body.length;
+ if (body[len - 1] == 0) {
+ len--;
+ }
+ return new String(body, 0, len, StandardCharsets.UTF_8);
+ }
+
+ /** {@code type · length · body}: a byte-exact round-trip of the original frame. */
+ public byte[] toPacketBytes() {
+ if (body == null) {
+ throw new IllegalStateException("Message body was streamed to passthrough output and not retained");
+ }
+ int bodyLen = body.length;
+ int lengthField = 4 + bodyLen;
+ byte[] packet = new byte[1 + 4 + bodyLen];
+ packet[0] = (byte) type;
+ packet[1] = (byte) ((lengthField >> 24) & 0xFF);
+ packet[2] = (byte) ((lengthField >> 16) & 0xFF);
+ packet[3] = (byte) ((lengthField >> 8) & 0xFF);
+ packet[4] = (byte) (lengthField & 0xFF);
+ if (bodyLen > 0) {
+ System.arraycopy(body, 0, packet, 5, bodyLen);
+ }
+ return packet;
+ }
+ }
+}
diff --git a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java
index 1b8573857c..ba84abb089 100644
--- a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java
+++ b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftAuthProxy.java
@@ -142,7 +142,9 @@ public class RedshiftAuthProxy {
client, backend, masterUsername, masterPassword, dbName,
false, sigV4, tlsCertificates, passwordValidator::validate);
if (activeClient != null) {
- PostgresProtocolHandler.bridge(activeClient, backend);
+ // Redshift-only DDL (DISTKEY/SORTKEY/ENCODE/...) is rewritten for the plain
+ // PostgreSQL backend on the way through; every other message is relayed verbatim.
+ new RedshiftInterceptingBridge(activeClient, backend).run();
}
} catch (Exception e) {
LOG.debugv("Redshift connection error for cluster {0}: {1}", clusterKey, e.getMessage());
diff --git a/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridge.java b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridge.java
new file mode 100644
index 0000000000..c554f873b8
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridge.java
@@ -0,0 +1,117 @@
+package io.github.hectorvent.floci.services.redshift.proxy;
+
+import org.jboss.logging.Logger;
+
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.net.Socket;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+/**
+ * Replaces the transparent {@code PostgresProtocolHandler.bridge} for Redshift connections so
+ * Simple Query ({@code 'Q'}) DDL can be rewritten for a plain PostgreSQL backend before it is
+ * forwarded.
+ *
+ * <p><b>Model.</b> Backend&rarr;client is a verbatim byte pump on a virtual thread, exactly as a
+ * plain proxy: extended-protocol pipelines, {@code COPY … FROM STDIN}, asynchronous
+ * {@code NotificationResponse} and backend EOF all flow through untouched. Client&rarr;backend is
+ * a framed loop: every frontend message is forwarded opaque <em>except</em> a {@code 'Q'}, whose
+ * SQL is run through {@link RedshiftSqlInterceptor#rewrite} and re-encoded only if it changed.
+ *
+ * <p><b>Fail-open.</b> A {@code rewrite} failure, or any statement the interceptor leaves
+ * untouched, forwards the original bytes; PostgreSQL then returns its own error. The bridge never
+ * closes the connection because of a rewrite failure.
+ */
+public class RedshiftInterceptingBridge {
+
+ private static final Logger LOG = Logger.getLogger(RedshiftInterceptingBridge.class);
+
+ private final Socket client;
+ private final Socket backend;
+ private final AtomicBoolean closed = new AtomicBoolean();
+
+ public RedshiftInterceptingBridge(Socket client, Socket backend) {
+ this.client = client;
+ this.backend = backend;
+ }
+
+ public void run() {
+ try {
+ InputStream clientIn = client.getInputStream();
+ OutputStream backendOut = backend.getOutputStream();
+
+ Thread.ofVirtual().name("redshift-pump-backend-to-client").start(this::pumpBackendToClient);
+
+ PostgresWireDecoder decoder = new PostgresWireDecoder(clientIn);
+ while (true) {
+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage(backendOut);
+ if (msg == null) {
+ break; // client EOF
+ }
+
+ if (!msg.isQuery()) {
+ if (msg.type() == 'X') { // Terminate
+ break;
+ }
+ continue;
+ }
+
+ String sql = msg.getSql();
+ byte[] toBackend = msg.toPacketBytes();
+ try {
+ String rewritten = RedshiftSqlInterceptor.rewrite(sql);
+ if (rewritten != sql) { // identity: rewrite returns the same instance when nothing matched
+ toBackend = PostgresWireDecoder.encodeQuery(rewritten);
+ }
+ } catch (RuntimeException e) {
+ LOG.warnv("RedshiftSqlInterceptor failed, forwarding original query: {0}", e.getMessage());
+ toBackend = msg.toPacketBytes();
+ }
+ backendOut.write(toBackend);
+ backendOut.flush();
+ // The response returns over the untouched pump.
+ }
+ } catch (IOException e) {
+ LOG.debugv(e, "RedshiftInterceptingBridge client loop ended");
+ } catch (Exception e) {
+ LOG.warnv(e, "Unexpected error in RedshiftInterceptingBridge");
+ } finally {
+ closeBoth();
+ }
+ }
+
+ private void pumpBackendToClient() {
+ try {
+ InputStream backendIn = backend.getInputStream();
+ OutputStream clientOut = client.getOutputStream();
+ byte[] buf = new byte[8192];
+ int n;
+ while ((n = backendIn.read(buf)) != -1) {
+ clientOut.write(buf, 0, n);
+ clientOut.flush();
+ }
+ } catch (IOException e) {
+ LOG.debugv(e, "backend->client pump ended");
+ } finally {
+ closeBoth();
+ }
+ }
+
+ private void closeBoth() {
+ if (closed.compareAndSet(false, true)) {
+ closeQuietly(client, "client");
+ closeQuietly(backend, "backend");
+ }
+ }
+
+ private void closeQuietly(Socket s, String which) {
+ try {
+ if (s != null && !s.isClosed()) {
+ s.close();
+ }
+ } catch (IOException e) {
+ LOG.debugv(e, "error closing {0} socket", which);
+ }
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java
index 2e0763cbde..7d8716beda 100644
--- a/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/rdsdata/RdsDataServiceTest.java
@@ -17,8 +17,10 @@ import java.sql.SQLException;
import java.sql.Statement;
import java.sql.Types;
import java.time.Duration;
+import java.util.Properties;
import java.util.UUID;
import java.util.concurrent.atomic.AtomicBoolean;
+import org.h2.Driver;
import static org.junit.jupiter.api.Assertions.assertArrayEquals;
import static org.junit.jupiter.api.Assertions.assertEquals;
@@ -33,6 +35,10 @@ import static org.mockito.Mockito.when;
class RdsDataServiceTest {
+ static {
+ Driver.load();
+ }
+
private static final String RESOURCE_ARN = "arn:aws:rds:us-east-1:000000000000:cluster:test";
private static final String FALLBACK_RESOURCE_ARN = "arn:aws:rds:us-west-2:111111111111:cluster:test";
private static final String OTHER_RESOURCE_ARN = "arn:aws:rds:us-east-1:000000000000:cluster:other";
@@ -864,14 +870,29 @@ class RdsDataServiceTest {
String username,
String password,
String database) throws SQLException {
- return DriverManager.getConnection(jdbcUrl, "sa", "");
+ return getConnection();
}
};
service = new RdsDataService(resolver, secrets, objectMapper, connectionFactory, transactionTtl);
}
+ private Connection getConnection() throws SQLException {
+ try {
+ return DriverManager.getConnection(jdbcUrl, "sa", "");
+ } catch (SQLException e) {
+ Properties props = new Properties();
+ props.setProperty("user", "sa");
+ props.setProperty("password", "");
+ Connection conn = new Driver().connect(jdbcUrl, props);
+ if (conn != null) {
+ return conn;
+ }
+ throw e;
+ }
+ }
+
private void createTables() throws SQLException {
- try (Connection connection = DriverManager.getConnection(jdbcUrl, "sa", "");
+ try (Connection connection = getConnection();
Statement statement = connection.createStatement()) {
statement.execute("""
create table data_api_items(
@@ -897,7 +918,7 @@ class RdsDataServiceTest {
* harnesses standing in for PostgreSQL.
*/
private void createEventsTable() throws SQLException {
- try (Connection connection = DriverManager.getConnection(jdbcUrl, "sa", "");
+ try (Connection connection = getConnection();
Statement statement = connection.createStatement()) {
statement.execute("""
create table data_api_events(
diff --git a/src/test/java/io/github/hectorvent/floci/services/redshift/RedshiftInterceptorIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/redshift/RedshiftInterceptorIntegrationTest.java
new file mode 100644
index 0000000000..b2a3b854e3
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/redshift/RedshiftInterceptorIntegrationTest.java
@@ -0,0 +1,109 @@
+package io.github.hectorvent.floci.services.redshift;
+
+import io.github.hectorvent.floci.services.redshift.model.Cluster;
+import io.quarkus.test.junit.QuarkusTest;
+import jakarta.inject.Inject;
+import org.awaitility.Awaitility;
+import org.awaitility.core.ConditionTimeoutException;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assumptions;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+
+import java.sql.Connection;
+import java.sql.DriverManager;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.time.Duration;
+import java.util.Objects;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+@QuarkusTest
+class RedshiftInterceptorIntegrationTest {
+
+ @Inject
+ RedshiftService service;
+
+ private String clusterId;
+
+ @BeforeAll
+ static void requireDocker() {
+ Assumptions.assumeTrue(isDockerAvailable(), "Docker daemon must be available for Redshift interceptor integration tests");
+ }
+
+ private static boolean isDockerAvailable() {
+ try {
+ Process process = new ProcessBuilder("docker", "version", "--format", "{{.Server.Version}}")
+ .redirectErrorStream(true)
+ .start();
+ int exit = process.waitFor();
+ return exit == 0;
+ } catch (Exception e) {
+ return false;
+ }
+ }
+
+ @AfterEach
+ void cleanUp() {
+ if (clusterId != null) {
+ service.deleteCluster(clusterId);
+ }
+ }
+
+ private static String jdbcUrl(Cluster c) {
+ // Use 127.0.0.1 explicitly instead of c.getEndpoint().getAddress() to avoid UnknownHostException
+ // in CI environments where floci.emulator.hostname is set to host.docker.internal.
+ // preferQueryMode=simple forces pgjdbc to use the simple query protocol ('Q' messages)
+ // rather than extended query protocol ('P'/'B'/'E'/'S' messages).
+ return "jdbc:postgresql://127.0.0.1:" + c.getEndpoint().getPort() + "/dev?preferQueryMode=simple";
+ }
+
+ private static Connection waitForConnection(Cluster cluster, String username, String password) throws SQLException {
+ try {
+ return Awaitility.await()
+ .atMost(Duration.ofSeconds(30))
+ .pollInterval(Duration.ofMillis(500))
+ .ignoreExceptions()
+ .until(() -> DriverManager.getConnection(jdbcUrl(cluster), username, password), Objects::nonNull);
+ } catch (ConditionTimeoutException e) {
+ return DriverManager.getConnection(jdbcUrl(cluster), username, password); // throw original
+ }
+ }
+
+ @Test
+ void rewritesCreateTableDdlOverSimpleQueryProtocol() throws SQLException {
+ clusterId = "it-interceptor-create";
+ Cluster cluster = service.createCluster(clusterId, "dc2.large", "admin", "Secret123");
+
+ try (Connection conn = waitForConnection(cluster, "admin", "Secret123");
+ Statement st = conn.createStatement()) {
+ st.execute("CREATE TABLE sales (id int ENCODE az64, d date) DISTSTYLE KEY DISTKEY (id) COMPOUND SORTKEY (d);");
+ st.execute("INSERT INTO sales VALUES (1, '2026-01-01');");
+ try (ResultSet rs = st.executeQuery("SELECT count(*) FROM sales")) {
+ assertTrue(rs.next());
+ assertEquals(1, rs.getInt(1));
+ }
+ }
+ }
+
+ @Test
+ void rewritesAlterTableDdlOverSimpleQueryProtocol() throws SQLException {
+ clusterId = "it-interceptor-alter";
+ Cluster cluster = service.createCluster(clusterId, "dc2.large", "admin", "Secret123");
+
+ try (Connection conn = waitForConnection(cluster, "admin", "Secret123");
+ Statement st = conn.createStatement()) {
+ st.execute("CREATE TABLE t (id int)");
+ st.execute("ALTER TABLE t ADD COLUMN note varchar(20) ENCODE lzo;");
+ st.execute("INSERT INTO t VALUES (1, 'test-note');");
+ try (ResultSet rs = st.executeQuery("SELECT id, note FROM t WHERE id = 1")) {
+ assertTrue(rs.next());
+ assertEquals(1, rs.getInt(1));
+ assertEquals("test-note", rs.getString(2));
+ }
+ }
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoderTest.java b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoderTest.java
new file mode 100644
index 0000000000..3ebd828056
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/PostgresWireDecoderTest.java
@@ -0,0 +1,311 @@
+package io.github.hectorvent.floci.services.redshift.proxy;
+
+import org.junit.jupiter.api.Test;
+
+import java.io.ByteArrayInputStream;
+import java.io.ByteArrayOutputStream;
+import java.io.EOFException;
+import java.io.IOException;
+import java.io.InputStream;
+import java.io.OutputStream;
+import java.nio.charset.StandardCharsets;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+class PostgresWireDecoderTest {
+
+ @Test
+ void decodesASimpleQueryAndThenReportsEof() throws IOException {
+ byte[] packet = PostgresWireDecoder.encodeQuery("SELECT 1");
+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(packet));
+
+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage();
+ assertNotNull(msg);
+ assertEquals('Q', msg.type());
+ assertTrue(msg.isQuery());
+ assertEquals("SELECT 1", msg.getSql());
+ assertArrayEquals(packet, msg.toPacketBytes());
+
+ assertNull(decoder.nextMessage());
+ }
+
+ @Test
+ void decodesNonQueryMessagesWithoutInterpretingThem() throws IOException {
+ byte[] terminate = new byte[]{'X', 0, 0, 0, 4};
+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(terminate));
+
+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage();
+ assertNotNull(msg);
+ assertEquals('X', msg.type());
+ assertFalse(msg.isQuery());
+ assertNull(msg.getSql());
+ assertEquals(0, msg.body().length);
+ assertArrayEquals(terminate, msg.toPacketBytes());
+ assertNull(decoder.nextMessage());
+
+ byte[] parsePayload = "stmt1\0SELECT $1\0\0\0".getBytes(StandardCharsets.UTF_8);
+ int length = 4 + parsePayload.length;
+ byte[] parsePacket = new byte[1 + length];
+ parsePacket[0] = 'P';
+ parsePacket[1] = (byte) ((length >> 24) & 0xFF);
+ parsePacket[2] = (byte) ((length >> 16) & 0xFF);
+ parsePacket[3] = (byte) ((length >> 8) & 0xFF);
+ parsePacket[4] = (byte) (length & 0xFF);
+ System.arraycopy(parsePayload, 0, parsePacket, 5, parsePayload.length);
+
+ PostgresWireDecoder.FrontendMessage parse =
+ new PostgresWireDecoder(new ByteArrayInputStream(parsePacket)).nextMessage();
+ assertNotNull(parse);
+ assertEquals('P', parse.type());
+ assertFalse(parse.isQuery());
+ assertArrayEquals(parsePayload, parse.body());
+ assertArrayEquals(parsePacket, parse.toPacketBytes());
+ }
+
+ @Test
+ void decodesSeveralMessagesFromOneStream() throws IOException {
+ ByteArrayOutputStream out = new ByteArrayOutputStream();
+ out.write(PostgresWireDecoder.encodeQuery("SELECT 1"));
+ out.write(PostgresWireDecoder.encodeQuery("SELECT 2"));
+ out.write(new byte[]{'X', 0, 0, 0, 4});
+
+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(out.toByteArray()));
+ assertEquals("SELECT 1", decoder.nextMessage().getSql());
+ assertEquals("SELECT 2", decoder.nextMessage().getSql());
+ assertEquals('X', decoder.nextMessage().type());
+ assertNull(decoder.nextMessage());
+ }
+
+ @Test
+ void reassemblesAMessageDeliveredTwoBytesAtATime() throws IOException {
+ byte[] packet = PostgresWireDecoder.encodeQuery("SELECT * FROM users WHERE active = true");
+ InputStream drip = new InputStream() {
+ private int i = 0;
+ @Override public int read() {
+ if (i >= packet.length) {
+ return -1;
+ }
+ return packet[i++] & 0xFF;
+ }
+ @Override public int read(byte[] b, int off, int len) {
+ if (i >= packet.length) {
+ return -1;
+ }
+ int n = Math.min(Math.min(len, 2), packet.length - i);
+ System.arraycopy(packet, i, b, off, n);
+ i += n;
+ return n;
+ }
+ };
+ PostgresWireDecoder.FrontendMessage msg = new PostgresWireDecoder(drip).nextMessage();
+ assertEquals("SELECT * FROM users WHERE active = true", msg.getSql());
+ assertArrayEquals(packet, msg.toPacketBytes());
+ }
+
+ @Test
+ void returnsNullOnAnEmptyStream() throws IOException {
+ assertNull(new PostgresWireDecoder(new ByteArrayInputStream(new byte[0])).nextMessage());
+ }
+
+ @Test
+ void throwsEofWhenTheLengthFieldIsTruncated() {
+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(new byte[]{'Q', 0, 0}));
+ assertThrows(EOFException.class, decoder::nextMessage);
+ }
+
+ @Test
+ void throwsEofWhenTheBodyIsTruncated() {
+ PostgresWireDecoder decoder =
+ new PostgresWireDecoder(new ByteArrayInputStream(new byte[]{'Q', 0, 0, 0, 10, 'S', 'E'}));
+ assertThrows(EOFException.class, decoder::nextMessage);
+ }
+
+ @Test
+ void rejectsALengthBelowFour() {
+ PostgresWireDecoder decoder =
+ new PostgresWireDecoder(new ByteArrayInputStream(new byte[]{'Q', 0, 0, 0, 2}));
+ assertThrows(IOException.class, decoder::nextMessage);
+ }
+
+ @Test
+ void refusesAnOversizedDeclaredLengthBeforeReadingTheBody() {
+ byte[] hostile = new byte[]{'Q', 0x7F, (byte) 0xFF, (byte) 0xFF, (byte) 0xFF};
+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(hostile));
+ IOException ex = assertThrows(IOException.class, decoder::nextMessage);
+ assertTrue(ex.getMessage().contains("Refusing"), ex.getMessage());
+ }
+
+ @Test
+ void encodeQueryProducesAWellFormedQPacket() {
+ byte[] encoded = PostgresWireDecoder.encodeQuery("SHOW search_path");
+ assertEquals('Q', (char) encoded[0]);
+ int length = ((encoded[1] & 0xFF) << 24) | ((encoded[2] & 0xFF) << 16)
+ | ((encoded[3] & 0xFF) << 8) | (encoded[4] & 0xFF);
+ assertEquals(21, length); // 4 + 16 chars + 1 NUL
+ assertEquals(0x00, encoded[encoded.length - 1]);
+ }
+
+ @Test
+ void isBetweenMessagesIsTrueAtBoundariesAndFalseMidMessage() throws IOException {
+ byte[] two = new ByteArrayOutputStream() {{
+ try {
+ write(PostgresWireDecoder.encodeQuery("SELECT 1"));
+ write(new byte[]{'Q', 0, 0, 0, 50}); // header promises 46 body bytes that never arrive
+ } catch (IOException e) {
+ throw new RuntimeException(e);
+ }
+ }}.toByteArray();
+
+ PostgresWireDecoder decoder = new PostgresWireDecoder(new ByteArrayInputStream(two));
+ assertTrue(decoder.isBetweenMessages());
+ assertNotNull(decoder.nextMessage());
+ assertTrue(decoder.isBetweenMessages());
+ assertThrows(EOFException.class, decoder::nextMessage);
+ assertFalse(decoder.isBetweenMessages()); // type byte was consumed before the stream ran dry
+ }
+
+ @Test
+ void streamsOpaqueMessageLargerThanQueryLimitDirectlyToPassthroughOut() throws IOException {
+ int payloadSize = 17 * 1024 * 1024; // 17 MiB, exceeds MAX_MESSAGE_BYTES
+ int totalLength = 4 + payloadSize;
+ byte[] header = new byte[]{
+ 'd', // CopyData
+ (byte) ((totalLength >> 24) & 0xFF),
+ (byte) ((totalLength >> 16) & 0xFF),
+ (byte) ((totalLength >> 8) & 0xFF),
+ (byte) (totalLength & 0xFF)
+ };
+
+ InputStream in = new InputStream() {
+ private int headerPos = 0;
+ private int bodyRemaining = payloadSize;
+
+ @Override
+ public int read() {
+ if (headerPos < header.length) {
+ return header[headerPos++] & 0xFF;
+ }
+ if (bodyRemaining > 0) {
+ bodyRemaining--;
+ return 'A';
+ }
+ return -1;
+ }
+
+ @Override
+ public int read(byte[] b, int off, int len) {
+ if (headerPos < header.length) {
+ int toCopy = Math.min(len, header.length - headerPos);
+ System.arraycopy(header, headerPos, b, off, toCopy);
+ headerPos += toCopy;
+ return toCopy;
+ }
+ if (bodyRemaining > 0) {
+ int toProvide = Math.min(len, bodyRemaining);
+ for (int i = 0; i < toProvide; i++) {
+ b[off + i] = 'A';
+ }
+ bodyRemaining -= toProvide;
+ return toProvide;
+ }
+ return -1;
+ }
+ };
+
+ long[] bytesWritten = new long[1];
+ OutputStream out = new OutputStream() {
+ @Override
+ public void write(int b) {
+ bytesWritten[0]++;
+ }
+
+ @Override
+ public void write(byte[] b, int off, int len) {
+ bytesWritten[0] += len;
+ }
+ };
+
+ PostgresWireDecoder decoder = new PostgresWireDecoder(in);
+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage(out);
+ assertNotNull(msg);
+ assertEquals('d', msg.type());
+ assertNull(msg.body());
+ assertEquals(1 + (long) totalLength, bytesWritten[0]);
+ }
+
+ @Test
+ void streamsSimpleQueryLargerThanLimitDirectlyToPassthroughOut() throws IOException {
+ int payloadSize = 17 * 1024 * 1024; // 17 MiB, exceeds MAX_MESSAGE_BYTES
+ int totalLength = 4 + payloadSize;
+ byte[] header = new byte[]{
+ 'Q',
+ (byte) ((totalLength >> 24) & 0xFF),
+ (byte) ((totalLength >> 16) & 0xFF),
+ (byte) ((totalLength >> 8) & 0xFF),
+ (byte) (totalLength & 0xFF)
+ };
+
+ InputStream in = new InputStream() {
+ private int headerPos = 0;
+ private int bodyRemaining = payloadSize;
+
+ @Override
+ public int read() {
+ if (headerPos < header.length) {
+ return header[headerPos++] & 0xFF;
+ }
+ if (bodyRemaining > 0) {
+ bodyRemaining--;
+ return 'A';
+ }
+ return -1;
+ }
+
+ @Override
+ public int read(byte[] b, int off, int len) {
+ if (headerPos < header.length) {
+ int toCopy = Math.min(len, header.length - headerPos);
+ System.arraycopy(header, headerPos, b, off, toCopy);
+ headerPos += toCopy;
+ return toCopy;
+ }
+ if (bodyRemaining > 0) {
+ int toProvide = Math.min(len, bodyRemaining);
+ for (int i = 0; i < toProvide; i++) {
+ b[off + i] = 'A';
+ }
+ bodyRemaining -= toProvide;
+ return toProvide;
+ }
+ return -1;
+ }
+ };
+
+ long[] bytesWritten = new long[1];
+ OutputStream out = new OutputStream() {
+ @Override
+ public void write(int b) {
+ bytesWritten[0]++;
+ }
+
+ @Override
+ public void write(byte[] b, int off, int len) {
+ bytesWritten[0] += len;
+ }
+ };
+
+ PostgresWireDecoder decoder = new PostgresWireDecoder(in);
+ PostgresWireDecoder.FrontendMessage msg = decoder.nextMessage(out);
+ assertNotNull(msg);
+ assertEquals('Q', msg.type());
+ assertFalse(msg.isQuery());
+ assertNull(msg.body());
+ assertEquals(1 + (long) totalLength, bytesWritten[0]);
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridgeTest.java b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridgeTest.java
new file mode 100644
index 0000000000..b11b9fbe29
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/redshift/proxy/RedshiftInterceptingBridgeTest.java
@@ -0,0 +1,141 @@
+package io.github.hectorvent.floci.services.redshift.proxy;
+
+import org.jboss.logging.Logger;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Test;
+
+import java.io.IOException;
+import java.net.ServerSocket;
+import java.net.Socket;
+import java.nio.charset.StandardCharsets;
+
+import static org.junit.jupiter.api.Assertions.assertArrayEquals;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotNull;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+
+class RedshiftInterceptingBridgeTest {
+
+ private static final Logger LOG = Logger.getLogger(RedshiftInterceptingBridgeTest.class);
+
+ private ServerSocket clientListener;
+ private ServerSocket backendListener;
+ private Socket testClientEnd; // test writes client bytes here, reads pump output here
+ private Socket bridgeClientEnd; // the bridge's "client" socket
+ private Socket bridgeBackendEnd; // the bridge's "backend" socket
+ private Socket testBackendEnd; // test reads what the bridge forwarded, writes backend replies
+ private Thread bridgeThread;
+
+ private void startBridge() throws IOException {
+ clientListener = new ServerSocket(0);
+ testClientEnd = new Socket("localhost", clientListener.getLocalPort());
+ bridgeClientEnd = clientListener.accept();
+
+ backendListener = new ServerSocket(0);
+ bridgeBackendEnd = new Socket("localhost", backendListener.getLocalPort());
+ testBackendEnd = backendListener.accept();
+
+ RedshiftInterceptingBridge bridge = new RedshiftInterceptingBridge(bridgeClientEnd, bridgeBackendEnd);
+ bridgeThread = Thread.ofVirtual().name("bridge-under-test").start(bridge::run);
+ }
+
+ @AfterEach
+ void tearDown() {
+ for (Socket s : new Socket[]{testClientEnd, bridgeClientEnd, bridgeBackendEnd, testBackendEnd}) {
+ if (s != null && !s.isClosed()) {
+ try {
+ s.close();
+ } catch (IOException e) {
+ LOG.debugv(e, "Error closing socket during test cleanup");
+ }
+ }
+ }
+ for (ServerSocket ss : new ServerSocket[]{clientListener, backendListener}) {
+ if (ss != null && !ss.isClosed()) {
+ try {
+ ss.close();
+ } catch (IOException e) {
+ LOG.debugv(e, "Error closing server socket during test cleanup");
+ }
+ }
+ }
+ }
+
+ private PostgresWireDecoder.FrontendMessage nextForwarded() throws IOException {
+ return new PostgresWireDecoder(testBackendEnd.getInputStream()).nextMessage();
+ }
+
+ @Test
+ void rewritesRedshiftCreateTableBeforeForwarding() throws IOException {
+ startBridge();
+ String ddl = "CREATE TABLE sales (id int ENCODE az64, d date) "
+ + "DISTSTYLE KEY DISTKEY (id) COMPOUND SORTKEY (d);";
+ testClientEnd.getOutputStream().write(PostgresWireDecoder.encodeQuery(ddl));
+ testClientEnd.getOutputStream().flush();
+
+ PostgresWireDecoder.FrontendMessage forwarded = nextForwarded();
+ assertNotNull(forwarded);
+ assertEquals('Q', forwarded.type());
+ String sent = forwarded.getSql().toUpperCase();
+ assertFalse(sent.contains("DISTKEY"), sent);
+ assertFalse(sent.contains("SORTKEY"), sent);
+ assertFalse(sent.contains("DISTSTYLE"), sent);
+ assertFalse(sent.contains("ENCODE"), sent);
+ assertTrue(forwarded.getSql().contains("CREATE TABLE sales"), forwarded.getSql());
+ }
+
+ @Test
+ void forwardsANonDdlQueryByteForByte() throws IOException {
+ startBridge();
+ byte[] packet = PostgresWireDecoder.encodeQuery("SELECT 'DISTKEY' AS not_a_keyword");
+ testClientEnd.getOutputStream().write(packet);
+ testClientEnd.getOutputStream().flush();
+
+ assertArrayEquals(packet, nextForwarded().toPacketBytes());
+ }
+
+ @Test
+ void forwardsAnExtendedProtocolMessageOpaque() throws IOException {
+ startBridge();
+ byte[] parsePayload = "s1\0SELECT $1\0\0\0".getBytes(StandardCharsets.UTF_8);
+ int length = 4 + parsePayload.length;
+ byte[] parsePacket = new byte[1 + length];
+ parsePacket[0] = 'P';
+ parsePacket[1] = (byte) ((length >> 24) & 0xFF);
+ parsePacket[2] = (byte) ((length >> 16) & 0xFF);
+ parsePacket[3] = (byte) ((length >> 8) & 0xFF);
+ parsePacket[4] = (byte) (length & 0xFF);
+ System.arraycopy(parsePayload, 0, parsePacket, 5, parsePayload.length);
+
+ testClientEnd.getOutputStream().write(parsePacket);
+ testClientEnd.getOutputStream().flush();
+
+ assertArrayEquals(parsePacket, nextForwarded().toPacketBytes());
+ }
+
+ @Test
+ void pumpsBackendBytesToTheClientUnchanged() throws IOException {
+ startBridge();
+ byte[] readyForQuery = new byte[]{'Z', 0, 0, 0, 5, 'I'};
+ testBackendEnd.getOutputStream().write(readyForQuery);
+ testBackendEnd.getOutputStream().flush();
+
+ byte[] got = testClientEnd.getInputStream().readNBytes(readyForQuery.length);
+ assertArrayEquals(readyForQuery, got);
+ }
+
+ @Test
+ void terminateMessageIsForwardedAndEndsTheSession() throws Exception {
+ startBridge();
+ testClientEnd.getOutputStream().write(new byte[]{'X', 0, 0, 0, 4});
+ testClientEnd.getOutputStream().flush();
+
+ PostgresWireDecoder.FrontendMessage forwarded = nextForwarded();
+ assertEquals('X', forwarded.type());
+
+ bridgeThread.join(5_000);
+ assertFalse(bridgeThread.isAlive(), "bridge did not stop after Terminate");
+ assertEquals(-1, testClientEnd.getInputStream().read(), "bridge left the client socket open");
+ }
+}