Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
feat(dsql): AWS IAM GRANT and REVOKE with sys.iam_pg_role_mappings (#2439)
Toby Nelson805cbec168d4
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex e361a3d1..41738cef 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,9 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]+### Added+- **Aurora DSQL — `AWS IAM GRANT` and `AWS IAM REVOKE`** — with `DSQL_STRICT=1` both were a syntax error, so a migration that maps a Postgres role to an IAM role stopped there. Sent on their own over the simple query protocol (psql, psycopg2), a GRANT maps one or more ARNs to a role and a REVOKE unmaps them; the mappings read back through `sys.iam_pg_role_mappings` with DSQL's six columns and types, a rollback undoes a GRANT, and a GRANT after other DDL in the same transaction answers DSQL's `0A000`. Re-granting raises DSQL's NOTICE and revoking a missing mapping its WARNING; an unknown role answers `42704`, a value that is not an ARN `0P000`, and granting the signed-in role to itself `42501`. Contributed by @toby6578.+### Fixed- **Athena — the result file is saved at the reported `OutputLocation`** — a query's `<id>.csv` and `<id>.csv.metadata` are now saved directly under the result location, at the path that `GetQueryExecution` reports, as AWS does. Before, they were saved one folder too deep, at `<location>/<id>.csv/<id>.csv`, so a client that read the reported path got `NoSuchKey`.- **Athena — an enforced workgroup overrides the client-side result location** — when a workgroup has `EnforceWorkGroupConfiguration: true`, `StartQueryExecution` now writes results to the workgroup's `OutputLocation`, even when the request gives its own `ResultConfiguration.OutputLocation`, as AWS does. Before, the request's location was used.diff --git a/README.md b/README.mdindex 68810808..2603f1e8 100644--- a/README.md+++ b/README.md@@ -721,7 +721,7 @@ only `DeleteDBCluster` removes the container and its storage.Each `CreateCluster` returns an `ACTIVE` cluster with an endpoint. Set `DSQL_STRICT=1` to additionally spin up a real `postgres:16-alpine` container per cluster (capped by a fixed port window off `DSQL_BASE_PORT`, default 30 ports) behind an in-process Postgres wire-protocol proxy.-**The proxy fails like real DSQL**: SQL that DSQL rejects fails here too (`serial` columns, `CREATE EXTENSION`, `TRUNCATE`, temp tables, `ALTER COLUMN TYPE`, `SET NOT NULL`, materialized views, partitioned tables, ...), index creation is always asynchronous — plain `CREATE INDEX` is refused (`unsupported mode. please use CREATE INDEX ASYNC.`), as are `CONCURRENTLY` and `USING` — while expression index keys and partial indexes (`WHERE` predicate) are supported as long as the functions used are immutable (violations fail with `42P17` at submit, same as the backend; `INCLUDE` columns can't be expressions, the predicate can't hold a subquery), `CREATE INDEX ASYNC` and `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` return a `job_id`, `sys.jobs` / `sys.wait_for_job` are emulated (with the job id written literally or bound as `$1`; `CALL sys.wait_for_job` answers DSQL's `succeeded` column) (data that does not satisfy the constraint fails the job, not the submit — `sys.jobs` reports it `failed` with the violation as `details`), CHECK and FOREIGN KEY constraints via ALTER TABLE require `NOT VALID` and every other `ADD CONSTRAINT` form is refused, `ADD COLUMN` takes a name, a type and an optional `STORAGE` mode and nothing else, foreign keys are supported in full (all five referential actions, `MATCH FULL`/`MATCH SIMPLE`, deferrable — `DEFERRABLE` on any other kind of constraint is refused), `ALTER TABLE ... DROP COLUMN` works (several columns in one statement included) except on a primary key column, identity columns are bigint-only (CACHE 1 or ≥ 65536), index limits are enforced (8 key columns/index — checked before anything but `CONCURRENTLY`/`USING`, even the mode — 24/table), `CREATE`/`ALTER`/`DROP STATISTICS` work with DSQL's 5-per-table limit and statistics targets capped at 100 (`ALTER TABLE ... SET STATISTICS` is refused), `dsql.enable_batched_nestloop` defaults to `on` and takes only Boolean values, the rest of the `dsql.` prefix is reserved (`42602`), `work_mem`, `statement_timeout` and `default_statistics_target` can't be set, transactions allow only one DDL statement with no DDL+DML mixing, `SELECT ... FOR UPDATE` and `FOR KEY SHARE` are the only locking clauses supported (`FOR SHARE` and `FOR NO KEY UPDATE` are refused, at any nesting depth; the predicate itself is unrestricted), a transaction that outlives another session's DDL fails once with `40001` (OC001) and succeeds on retry, and the transaction limits (3,000 rows / 10 MiB / 5 min) are enforced.+**The proxy fails like real DSQL**: SQL that DSQL rejects fails here too (`serial` columns, `CREATE EXTENSION`, `TRUNCATE`, temp tables, `ALTER COLUMN TYPE`, `SET NOT NULL`, materialized views, partitioned tables, ...), index creation is always asynchronous — plain `CREATE INDEX` is refused (`unsupported mode. please use CREATE INDEX ASYNC.`), as are `CONCURRENTLY` and `USING` — while expression index keys and partial indexes (`WHERE` predicate) are supported as long as the functions used are immutable (violations fail with `42P17` at submit, same as the backend; `INCLUDE` columns can't be expressions, the predicate can't hold a subquery), `CREATE INDEX ASYNC` and `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` return a `job_id`, `sys.jobs` / `sys.wait_for_job` are emulated (with the job id written literally or bound as `$1`; `CALL sys.wait_for_job` answers DSQL's `succeeded` column) (data that does not satisfy the constraint fails the job, not the submit — `sys.jobs` reports it `failed` with the violation as `details`), `AWS IAM GRANT <role> TO '<arn>'[, ...]` and `AWS IAM REVOKE <role> FROM '<arn>'[, ...]` map Postgres roles to IAM ARNs, read back through `sys.iam_pg_role_mappings` and undone by a rollback like any other DDL (the mapping is recorded, not enforced: any user/password still signs in), CHECK and FOREIGN KEY constraints via ALTER TABLE require `NOT VALID` and every other `ADD CONSTRAINT` form is refused, `ADD COLUMN` takes a name, a type and an optional `STORAGE` mode and nothing else, foreign keys are supported in full (all five referential actions, `MATCH FULL`/`MATCH SIMPLE`, deferrable — `DEFERRABLE` on any other kind of constraint is refused), `ALTER TABLE ... DROP COLUMN` works (several columns in one statement included) except on a primary key column, identity columns are bigint-only (CACHE 1 or ≥ 65536), index limits are enforced (8 key columns/index — checked before anything but `CONCURRENTLY`/`USING`, even the mode — 24/table), `CREATE`/`ALTER`/`DROP STATISTICS` work with DSQL's 5-per-table limit and statistics targets capped at 100 (`ALTER TABLE ... SET STATISTICS` is refused), `dsql.enable_batched_nestloop` defaults to `on` and takes only Boolean values, the rest of the `dsql.` prefix is reserved (`42602`), `work_mem`, `statement_timeout` and `default_statistics_target` can't be set, transactions allow only one DDL statement with no DDL+DML mixing, `SELECT ... FOR UPDATE` and `FOR KEY SHARE` are the only locking clauses supported (`FOR SHARE` and `FOR NO KEY UPDATE` are refused, at any nesting depth; the predicate itself is unrestricted), a transaction that outlives another session's DDL fails once with `40001` (OC001) and succeeds on retry, and the transaction limits (3,000 rows / 10 MiB / 5 min) are enforced.CDC streams are metadata only: `CreateStream` validates its Kinesis target definition, enforces the 5-streams-per-cluster quota (`ServiceQuotaExceededException`) and reports `ACTIVE` right away, but no change record is ever delivered — a target whose Kinesis stream is absent (or, with `AUTH=true`, whose role cannot be assumed) lands `FAILED` with the service's `statusReason` code instead. Deleting a cluster deletes its streams.@@ -729,7 +729,7 @@ CDC streams are metadata only: `CreateStream` validates its Kinesis target defin**Both wire protocols are validated**: the simple protocol (psql, psycopg2) and the extended `Parse`/`Bind`/`Execute` protocol used by pgjdbc, pgx, asyncpg, psycopg3 and every ORM built on prepared statements. Transaction state is read from the backend's `ReadyForQuery` status byte rather than inferred from statement text, so `BEGIN`/`COMMIT` sent over either protocol keeps the transaction rules accurate. A statement the proxy rejects inside a transaction aborts that block exactly as a real error would — later statements fail with `25P02` and `COMMIT` reports `ROLLBACK`.-Limitations: SQL analysis is heuristic regex, not a real parser (statement splitting, comment stripping and parenthesis matching are lexer-aware, but clause-level rules are pattern matches); the proxy's immutability check for index expressions and predicates is a denylist of well-known volatile functions — the backend catalog enforces the rest; transaction row counting is static (VALUES tuples only — `INSERT ... SELECT`/UPDATE/DELETE row counts aren't tracked); OC001 emulation is optimistic (catalog version bumps on forwarded DDL, not on successful commit); foreign keys are enforced by the backend, so a referencing write racing a change to the referenced key waits on a row lock instead of failing the loser with `40001` (OC000) the way DSQL's optimistic resolution does; the 1 KiB PK/index key-size limit is not enforced; and in a pipelined extended-protocol batch (several statements before one `Sync`) the rules that need to inspect backend state — the 24-index and 5-statistics limits and the primary-key `DROP COLUMN` check — are skipped; `EXPLAIN` output is Postgres-shaped, not DSQL's plan nodes (`Full Scan (btree-table)`, `Nested Loop (Batched Join)`, ...); `dsql.*` settings are Postgres placeholders, so they're missing from `pg_settings`; DSQL refuses more settings than the three listed above (only those were measured), while `CREATE INDEX ASYNC` is rejected as it is in multi-statement `Q` batches. Error messages match live Aurora DSQL where verified, otherwise Postgres-style approximations (`0A000`, `25006`).+Limitations: SQL analysis is heuristic regex, not a real parser (statement splitting, comment stripping and parenthesis matching are lexer-aware, but clause-level rules are pattern matches); the proxy's immutability check for index expressions and predicates is a denylist of well-known volatile functions — the backend catalog enforces the rest; transaction row counting is static (VALUES tuples only — `INSERT ... SELECT`/UPDATE/DELETE row counts aren't tracked); OC001 emulation is optimistic (catalog version bumps on forwarded DDL, not on successful commit); foreign keys are enforced by the backend, so a referencing write racing a change to the referenced key waits on a row lock instead of failing the loser with `40001` (OC000) the way DSQL's optimistic resolution does; the 1 KiB PK/index key-size limit is not enforced; dropping a role that still has an IAM mapping is not refused, and `AWS IAM GRANT`/`REVOKE` are supported only as a statement of their own over the simple query protocol (psql, psycopg2), refused in a multi-statement `Q` batch and over the extended protocol; and in a pipelined extended-protocol batch (several statements before one `Sync`) the rules that need to inspect backend state — the 24-index and 5-statistics limits and the primary-key `DROP COLUMN` check — are skipped; `EXPLAIN` output is Postgres-shaped, not DSQL's plan nodes (`Full Scan (btree-table)`, `Nested Loop (Batched Join)`, ...); `dsql.*` settings are Postgres placeholders, so they're missing from `pg_settings`; DSQL refuses more settings than the three listed above (only those were measured), while `CREATE INDEX ASYNC` is rejected as it is in multi-statement `Q` batches. Error messages match live Aurora DSQL where verified, otherwise Postgres-style approximations (`0A000`, `25006`).---diff --git a/ministack/core/pgproxy.py b/ministack/core/pgproxy.pyindex 66ea64ef..0f0482db 100644--- a/ministack/core/pgproxy.py+++ b/ministack/core/pgproxy.py@@ -38,6 +38,10 @@ Documented limitations:SELECT`` / ``UPDATE`` / ``DELETE`` affected-row counts are not tracked.- OC001 emulation is optimistic: the catalog version bumps on forwarded DDL,not on successful commit.+- ``AWS IAM GRANT`` / ``REVOKE`` are supported only as a statement of their+ own over the simple query protocol (psql, psycopg2): they are refused in a+ multi-statement ``Q`` batch and over the extended protocol. Dropping a role that still has an IAM mapping is+ not refused, whereas DSQL refuses it (2BP01)."""import asyncio@@ -98,6 +102,16 @@ def _error_response(sqlstate, message, detail=None):return _frame(b"E", payload)+def _notice_response(severity, sqlstate, message):+ payload = (+ b"S" + severity.encode() + b"\0V" + severity.encode() + b"\0"+ + b"C" + sqlstate.encode() + b"\0"+ + b"M" + message.encode() + b"\0"+ + b"\0"+ )+ return _frame(b"N", payload)++def _ready(status=b"I"):return _frame(b"Z", status)@@ -604,7 +618,7 @@ def classify_statement(sql):elif kw == "EXPLAIN":inner = _explain_inner(s)return classify_statement(inner) if inner is not None else "other"- if kw in _DDL_KEYWORDS:+ if kw in _DDL_KEYWORDS or (kw == "AWS" and _AWS_IAM_RE.match(s)):return "ddl"if kw in _DML_KEYWORDS:return "dml"@@ -1531,6 +1545,10 @@ def validate(sql, txn_state=None):class _Conn:def __init__(self, cluster_id):self.cluster_id = cluster_id+ self.user = "admin" # the role the client signed in as+ self.database = "postgres"+ self.backend_host = None+ self.backend_port = Noneself.txn = TxnState()self.capture = None # asyncio.Queue while swallowing backend framesself.catalog_version = None # last catalog version this conn has seen@@ -1579,9 +1597,9 @@ async def _connect_backend_once(host, port, database):reader, writer = await asyncio.open_connection(host, port)params = b"user\0postgres\0database\0" + database.encode() + b"\0\0"payload = struct.pack("!I", _PROTOCOL_3) + params- writer.write(struct.pack("!I", len(payload) + 4) + payload)- await writer.drain()try:+ writer.write(struct.pack("!I", len(payload) + 4) + payload)+ await writer.drain()while True:type_byte, payload = await _read_frame(reader)if type_byte == b"E":@@ -1592,7 +1610,7 @@ async def _connect_backend_once(host, port, database):raise RuntimeError(f"backend requested unsupported auth method {code}")elif type_byte == b"Z":return reader, writer- except Exception:+ except BaseException:writer.close()raise@@ -1859,6 +1877,16 @@ async def _check_statistics_limit(conn, b_writer, sql):_ASYNC_BATCH_ERR = DsqlError("0A000", "asynchronous DDL is not supported in multi-statement queries")+# Not DSQL messages: IAM trust is supported only as a statement of its own over+# the simple query protocol.+_IAM_BATCH_ERR = DsqlError(+ "0A000",+ "AWS IAM GRANT and REVOKE must be sent as a statement of their own, not in a batch",+)+_IAM_EXTENDED_ERR = DsqlError(+ "0A000",+ "AWS IAM GRANT and REVOKE are supported over the simple query protocol only",+)def _sys_jobs_rows(conn, cols, job_filter):@@ -1908,19 +1936,294 @@ def _deferred(cols, param, resolve):}-async def _plan_statement(conn, s, b_writer, allow_probe=True):+# --- AWS IAM GRANT / REVOKE and sys.iam_pg_role_mappings -------------------+#+# DSQL keeps IAM trust in its own catalog. Here it lives in a table on the+# backend, read through a real sys.iam_pg_role_mappings view, so a ROLLBACK+# undoes a GRANT as it does on DSQL. The catalog is created the first time a+# database uses IAM trust, on a connection of its own. An ARN's iam_oid comes+# from the backend's oid counter, so it sits among the role oids as it does on+# DSQL. Behaviour, messages and the view's column types measured on Aurora+# DSQL, eu-west-1 2026-10-26.++_AWS_IAM_RE = re.compile(r"\s*AWS\s+IAM\s+(GRANT|REVOKE)\b", re.I)+_IAM_ARN_RE = re.compile(r"arn:[^:]+:iam::")+_IAM_TRUST_TABLE = "sys._ministack_iam_trust"+_IAM_CATALOG_SQL = (+ "SET lock_timeout = '5s'; "+ "DO $$ BEGIN CREATE ROLE admin; "+ "EXCEPTION WHEN duplicate_object OR unique_violation THEN NULL; END $$; "+ "CREATE SCHEMA IF NOT EXISTS sys; "+ f"CREATE TABLE IF NOT EXISTS {_IAM_TRUST_TABLE} ("+ "iam_oid oid NOT NULL, arn text COLLATE \"C\" NOT NULL, "+ "pg_role_oid oid NOT NULL, grantor name NOT NULL, "+ "PRIMARY KEY (pg_role_oid, arn)); "+ "CREATE OR REPLACE VIEW sys.iam_pg_role_mappings AS "+ "SELECT t.iam_oid, t.arn, t.pg_role_oid, r.rolname AS pg_role_name, "+ "g.oid AS grantor_pg_role_oid, t.grantor AS grantor_pg_role_name "+ f"FROM {_IAM_TRUST_TABLE} t JOIN pg_roles r ON r.oid = t.pg_role_oid "+ "LEFT JOIN pg_roles g ON g.rolname = t.grantor"+)+_IAM_VIEW_RE = re.compile(r"\biam_pg_role_mappings\b", re.I)+_iam_catalog_ready = set() # (cluster_id, database) pairs with the trust catalog+_iam_catalog_locks = {} # (cluster_id, database) -> asyncio.Lock+++class IamTrust:+ """A parsed ``AWS IAM GRANT <role> TO '<arn>'[, ...]`` (or REVOKE ... FROM)."""++ def __init__(self, action, role, arns):+ self.action = action+ self.role = role+ self.arns = arns++ @property+ def tag(self):+ return f"AWS IAM {self.action}"+++_IAM_TOKEN_RE = re.compile(+ r"\s*(?:(?P<str>'(?:[^']|'')*')|(?P<qid>\"(?:[^\"]|\"\")*\")"+ r"|(?P<word>[A-Za-z_][A-Za-z0-9_$]*)|(?P<punct>.))",+ re.S,+)+++def _iam_tokens(text):+ tokens, i, n = [], 0, len(text)+ while i < n:+ if text[i].isspace():+ i += 1+ continue+ if text.startswith(("--", "/*"), i):+ i = _skip_noise(text, i)+ continue+ m = _IAM_TOKEN_RE.match(text, i)+ tokens.append((m.lastgroup, m.group(m.lastgroup)))+ i = m.end()+ return tokens+++def _iam_syntax_error(token):+ if token is None:+ return DsqlError("42601", "syntax error at end of input")+ return DsqlError("42601", f'syntax error at or near "{token[1]}"')+++def match_aws_iam(sql):+ """IamTrust for an ``AWS IAM GRANT``/``REVOKE``, a DsqlError, or None."""+ m = _AWS_IAM_RE.match(sql)+ if not m:+ return None+ action = m.group(1).upper()+ tokens = _iam_tokens(sql[m.end():].rstrip().rstrip(";"))+ pos = 0++ def take():+ nonlocal pos+ token = tokens[pos] if pos < len(tokens) else None+ pos += 1+ return token++ role_token = take()+ if role_token is None or role_token[0] not in ("word", "qid"):+ return _iam_syntax_error(role_token)+ role, _ = fold_identifier(role_token[1])++ keyword = take()+ if keyword is None or keyword[0] != "word" or keyword[1].upper() != (+ "TO" if action == "GRANT" else "FROM"+ ):+ return _iam_syntax_error(keyword)++ arns = []+ while True:+ literal = take()+ if literal is None or literal[0] != "str":+ return _iam_syntax_error(literal)+ arn = literal[1][1:-1].replace("''", "'")+ if arn not in arns:+ arns.append(arn)+ separator = take()+ if separator is None:+ break+ if separator != ("punct", ","):+ return _iam_syntax_error(separator)++ for arn in arns:+ if not _IAM_ARN_RE.match(arn):+ return DsqlError("0P000", "IAM role must look like an IAM ARN")+ return IamTrust(action, role, arns)+++def _sql_literal(value):+ """A literal that reads the same whatever ``standard_conforming_strings`` is."""+ return "E'" + value.replace("\\", "\\\\").replace("'", "''") + "'"+++def _iam_trust_sql(trust, grantor):+ role = _sql_literal('"' + trust.role.replace('"', '""') + '"')+ arns = "ARRAY[" + ", ".join(_sql_literal(a) for a in trust.arns) + "]::text[]"+ head = (+ f"WITH r AS (SELECT {role}::regrole::oid AS oid), "+ f"a AS (SELECT arn, ord FROM unnest({arns}) WITH ORDINALITY AS u(arn, ord)), "+ )+ if trust.action == "GRANT":+ return head + (+ "existing AS (SELECT a.arn, a.ord, t.grantor FROM a "+ f"JOIN {_IAM_TRUST_TABLE} t ON t.arn = a.arn JOIN r ON t.pg_role_oid = r.oid), "+ f"ins AS (INSERT INTO {_IAM_TRUST_TABLE} (iam_oid, arn, pg_role_oid, grantor) "+ "SELECT COALESCE((SELECT x.iam_oid FROM "+ f"{_IAM_TRUST_TABLE} x WHERE x.arn = a.arn LIMIT 1), "+ "pg_nextoid('pg_authid', 'oid', 'pg_authid_oid_index')), "+ f"a.arn, r.oid, {_sql_literal(grantor)} FROM a CROSS JOIN r "+ "WHERE NOT EXISTS (SELECT 1 FROM existing e WHERE e.arn = a.arn) "+ "ON CONFLICT DO NOTHING) "+ "SELECT arn, grantor FROM existing ORDER BY ord"+ )+ return head + (+ f"del AS (DELETE FROM {_IAM_TRUST_TABLE} t USING r "+ "WHERE t.pg_role_oid = r.oid AND t.arn IN (SELECT arn FROM a) RETURNING t.arn) "+ "SELECT a.arn, NULL FROM a "+ "WHERE NOT EXISTS (SELECT 1 FROM del WHERE del.arn = a.arn) ORDER BY a.ord"+ )+++def _iam_notices(trust, rows):+ if trust.action == "GRANT":+ return b"".join(+ _notice_response(+ "NOTICE", "00000",+ f'IAM role "{arn}" has already been granted access to role '+ f'"{trust.role}" by role "{grantor}"',+ )+ for arn, grantor in rows+ )+ return b"".join(+ _notice_response(+ "WARNING", "01000",+ f'IAM role "{arn}" has not been granted access to role "{trust.role}"',+ )+ for arn, _ in rows+ )+++def _text_rows(frames):+ rows = []+ for type_byte, payload in frames:+ if type_byte != b"D":+ continue+ (nfields,) = struct.unpack("!H", payload[:2])+ i, row = 2, []+ for _ in range(nfields):+ (vlen,) = struct.unpack("!i", payload[i:i + 4])+ i += 4+ row.append(None if vlen < 0 else payload[i:i + vlen].decode("utf-8", "replace"))+ i += max(vlen, 0)+ rows.append(tuple(row))+ return rows+++async def _ensure_iam_catalog(conn):+ """Create the trust table and view in this connection's database, once.++ Runs on a backend connection of its own, so the DDL never lands inside the+ client's transaction or pipeline.+ """+ key = (conn.cluster_id, conn.database)+ if key in _iam_catalog_ready:+ return+ async with _iam_catalog_locks.setdefault(key, asyncio.Lock()):+ if key in _iam_catalog_ready:+ return+ try:+ error = await asyncio.wait_for(_create_iam_catalog(conn), timeout=30)+ except (OSError, RuntimeError, asyncio.IncompleteReadError, asyncio.TimeoutError) as e:+ error = repr(e)+ if error:+ # The statement still runs; the backend then reports the missing+ # relation, and the next statement tries again.+ logger.warning(+ "pgproxy: could not create the IAM trust catalog (cluster %s, database %s): %s",+ conn.cluster_id, conn.database, error,+ )+ return+ _iam_catalog_ready.add(key)+++async def _create_iam_catalog(conn):+ """Run the catalog DDL on a fresh backend connection; the backend's error, if any."""+ reader, writer = await _connect_backend_once(+ conn.backend_host, conn.backend_port, conn.database+ )+ try:+ writer.write(_frame(b"Q", _IAM_CATALOG_SQL.encode() + b"\0"))+ await writer.drain()+ error = None+ while True:+ type_byte, payload = await _read_frame(reader)+ if type_byte == b"E" and error is None:+ error = "%s %s" % _error_payload_fields(payload)+ if type_byte == b"Z":+ return error+ finally:+ writer.close()+++async def _run_iam_trust(conn, b_writer, trust):+ """Apply an IamTrust on the backend; return (frames for the client, backend_error)."""+ frames = await _run_backend_capture(conn, b_writer, _iam_trust_sql(trust, conn.user))+ backend_err = next((p for t, p in frames if t == b"E"), None)+ if backend_err is not None:+ return None, backend_err+ conn.catalog_version = _bump_catalog(conn.cluster_id)+ return _iam_notices(trust, _text_rows(frames)) + _command_complete(trust.tag), None+++async def _plan_iam(conn, s, extended):+ """("iam", IamTrust) or ("error", DsqlError) for an AWS IAM statement, else None."""+ trust = match_aws_iam(s)+ if trust is None:+ if _IAM_VIEW_RE.search(s):+ await _ensure_iam_catalog(conn)+ return None+ if extended:+ return "error", _IAM_EXTENDED_ERR+ if isinstance(trust, DsqlError):+ return "error", trust+ await _ensure_iam_catalog(conn)+ if conn.txn.in_txn and conn.txn.ddl_seen:+ return "error", DsqlError(+ "0A000", "multiple ddl statements not supported in a transaction"+ )+ result = validate(s, conn.txn)+ if isinstance(result, DsqlError):+ return "error", result+ if trust.action == "GRANT" and trust.role == conn.user:+ return "error", DsqlError(+ "42501",+ f'permission denied to grant IAM trust with role "{trust.role}"',+ f'Only roles with the ADMIN option on role "{trust.role}" may grant '+ "IAM trust to this role.",+ )+ return "iam", trust+++async def _plan_statement(conn, s, b_writer, allow_probe=True, extended=False):"""Decide what to do with one statement, for either wire protocol.Returns ``(kind, payload)`` where kind is one of:"error" — payload is a DsqlError to report"rows" — payload describes a synthetic result set (sys.jobs etc.)"rewrite" — payload is a Rewrite to run on the backend as a job+ "iam" — payload is an IamTrust to apply (simple protocol only)"forward" — payload is None; relay the statement untouched``allow_probe`` is False when the backend is mid extended-protocolsequence, where injecting a probe query would break the protocol. Thetext-only rules still apply; only the rules that need to look at backend- state are skipped.+ state are skipped. ``extended`` is True for the extended protocol, which+ refuses ``AWS IAM`` statements."""matched = match_sys_jobs(s)if matched is not False:@@ -1949,6 +2252,10 @@ async def _plan_statement(conn, s, b_writer, allow_probe=True):return "error", resultreturn "rows", result+ iam = await _plan_iam(conn, s, extended)+ if iam is not None:+ return iam+result = validate(s, conn.txn)if isinstance(result, DsqlError):return "error", result@@ -2090,6 +2397,15 @@ async def _handle_query(conn, sql, b_writer, c_writer):c_writer.write(out + _ready(_status(conn)))await c_writer.drain()return+ if kind == "iam":+ txn.apply(s)+ out, backend_err = await _run_iam_trust(conn, b_writer, result)+ if backend_err is not None:+ c_writer.write(_frame(b"E", backend_err) + _ready(_status(conn)))+ else:+ c_writer.write(out + _ready(_status(conn)))+ await c_writer.drain()+ returnif kind == "rewrite":txn.apply(s)# Run the rewritten statement on the backend, swallow its@@ -2137,6 +2453,12 @@ async def _handle_query(conn, sql, b_writer, c_writer):for s in stmts:if classify_statement(s) == "tcl_begin":explicit = True+ if _AWS_IAM_RE.match(strip_leading_comments(s)):+ _reject(conn, c_writer, _IAM_BATCH_ERR)+ await c_writer.drain()+ return+ if _IAM_VIEW_RE.search(s):+ await _ensure_iam_catalog(conn)result = validate(s, sim)if isinstance(result, (DsqlError, Rewrite)):if isinstance(result, Rewrite):@@ -2244,7 +2566,9 @@ async def _ext_parse(conn, payload, b_writer, c_writer):await _ext_error(conn, c_writer, _OC001_ERR)return- kind, result = await _plan_statement(conn, s, b_writer, allow_probe=allow_probe)+ kind, result = await _plan_statement(+ conn, s, b_writer, allow_probe=allow_probe, extended=True+ )if kind == "error":await _ext_error(conn, c_writer, result)return@@ -2434,6 +2758,9 @@ async def _handle_client(cluster_id, backend_host, backend_port, c_reader, c_wriawait c_writer.drain()database = params.get("database") or "postgres"+ conn.user = params.get("user") or conn.user+ conn.database = database+ conn.backend_host, conn.backend_port = backend_host, backend_portb_reader, b_writer = await _connect_backend(backend_host, backend_port, database)@@ -2494,6 +2821,10 @@ async def stop_proxy(cluster_id):server.close()await server.wait_closed()_jobs.pop(cluster_id, None)+ for key in [k for k in _iam_catalog_ready if k[0] == cluster_id]:+ _iam_catalog_ready.discard(key)+ for key in [k for k in _iam_catalog_locks if k[0] == cluster_id]:+ _iam_catalog_locks.pop(key, None)async def stop_all_proxies():diff --git a/tests/test_dsql.py b/tests/test_dsql.pyindex 36d9bffa..c519df7f 100644--- a/tests/test_dsql.py+++ b/tests/test_dsql.py@@ -7,6 +7,7 @@ import asyncioimport loggingimport osimport re+import secretsimport socketimport structimport threading@@ -710,6 +711,8 @@ class TestClassifyStatement:("SHOW server_version", "other"),("GRANT SELECT ON t TO r", "other"),("REVOKE SELECT ON t FROM r", "other"),+ ("AWS IAM GRANT r TO 'arn:aws:iam::123456789012:role/x'", "ddl"),+ ("aws iam revoke r FROM 'arn:aws:iam::123456789012:role/x'", "ddl"),# Leading comments must not hide the statement keyword.("-- migration 004\nCREATE TABLE t (a int)", "ddl"),("/* flyway */ INSERT INTO t VALUES (1)", "dml"),@@ -735,6 +738,63 @@ class TestClassifyStatement:assert pgproxy.classify_statement(sql) == expected+class TestAwsIamParsing:+ """``AWS IAM GRANT`` / ``REVOKE`` grammar and the errors DSQL raises for it+ before touching a role (measured on Aurora DSQL, eu-west-1 2026-10-26)."""++ def test_grant_with_several_arns(self):+ trust = pgproxy.match_aws_iam(+ "AWS IAM GRANT Probe_Role TO 'arn:aws:iam::123456789012:role/a', "+ "'arn:aws:iam::123456789012:user/b'"+ )+ assert (trust.action, trust.role, trust.arns) == (+ "GRANT", "probe_role",+ ["arn:aws:iam::123456789012:role/a", "arn:aws:iam::123456789012:user/b"],+ )+ assert trust.tag == "AWS IAM GRANT"++ def test_revoke_keeps_a_quoted_role_verbatim(self):+ trust = pgproxy.match_aws_iam(+ "aws iam revoke \"Mixed\" from 'arn:aws:iam::123456789012:role/a';"+ )+ assert (trust.action, trust.role, trust.tag) == ("REVOKE", "Mixed", "AWS IAM REVOKE")++ def test_other_statements_are_not_matched(self):+ assert pgproxy.match_aws_iam("GRANT r TO s") is None+ assert pgproxy.match_aws_iam("SELECT 'AWS IAM GRANT'") is None++ def test_an_arn_must_be_a_string_literal(self):+ err = pgproxy.match_aws_iam(+ 'AWS IAM GRANT probe_role TO "arn:aws:iam::123456789012:role/Quoted"'+ )+ assert err.sqlstate == "42601"+ assert err.message == (+ 'syntax error at or near ""arn:aws:iam::123456789012:role/Quoted""'+ )++ def test_a_value_that_is_not_an_arn_is_refused(self):+ err = pgproxy.match_aws_iam("AWS IAM GRANT probe_role TO 'not-an-arn'")+ assert (err.sqlstate, err.message) == ("0P000", "IAM role must look like an IAM ARN")++ def test_comments_inside_the_statement_are_ignored(self):+ trust = pgproxy.match_aws_iam(+ "AWS IAM GRANT r /* trust */ TO -- the API\n"+ "'arn:aws:iam::123456789012:role/a' -- done"+ )+ assert trust.arns == ["arn:aws:iam::123456789012:role/a"]++ def test_grant_needs_to_and_revoke_needs_from(self):+ assert pgproxy.match_aws_iam(+ "AWS IAM GRANT r FROM 'arn:aws:iam::123456789012:role/a'"+ ).message == 'syntax error at or near "FROM"'+ assert pgproxy.match_aws_iam(+ "AWS IAM REVOKE r TO 'arn:aws:iam::123456789012:role/a'"+ ).message == 'syntax error at or near "TO"'+ assert pgproxy.match_aws_iam("AWS IAM GRANT r TO").message == (+ "syntax error at end of input"+ )++class TestSqlLexing:"""Statement splitting and comment stripping must not be fooled byliterals — a ';' or ')' inside a string used to truncate the text a@@ -2722,6 +2782,324 @@ class TestTransactionAbortSemantics:assert c.simple("SELECT count(*) FROM ab_x").rows == [("0",)]+_PROBE_ROLE_ARN = "arn:aws:iam::123456789012:role/ministack-probe"+_PROBE_USER_ARN = "arn:aws:iam::123456789012:user/ministack-probe"+_IN_TXN_ARN = "arn:aws:iam::123456789012:role/InTxn"+++@requires_docker[email protected]_plane+class TestAwsIamTrust:+ """IAM trust over the wire. Messages, SQLSTATEs, the transaction rules and+ the shape of sys.iam_pg_role_mappings were measured on Aurora DSQL,+ eu-west-1 2026-10-26."""++ @pytest.fixture+ def role(self, dsql_proxy):+ name = f"iam_{secrets.token_hex(4)}"+ conn = _pg_connect(dsql_proxy)+ conn.cursor().execute(f"CREATE ROLE {name} WITH LOGIN")+ yield name+ cur = conn.cursor()+ cur.execute(+ f"AWS IAM REVOKE {name} FROM '{_PROBE_ROLE_ARN}', '{_PROBE_USER_ARN}', '{_IN_TXN_ARN}'"+ )+ cur.execute(f"DROP ROLE {name}")+ conn.close()++ @staticmethod+ def _mappings(cur, role):+ cur.execute(+ "SELECT arn, pg_role_name, grantor_pg_role_name "+ f"FROM sys.iam_pg_role_mappings WHERE pg_role_name = '{role}' ORDER BY arn"+ )+ return cur.fetchall()++ def test_the_view_has_dsqls_columns_and_types(self, dsql_proxy):+ conn = _pg_connect(dsql_proxy)+ try:+ cur = conn.cursor()+ cur.execute(+ "SELECT attname, format_type(atttypid, atttypmod), "+ "(SELECT collname FROM pg_collation WHERE oid = attcollation) "+ "FROM pg_attribute WHERE attrelid = 'sys.iam_pg_role_mappings'::regclass "+ "AND attnum > 0 ORDER BY attnum"+ )+ assert cur.fetchall() == [+ ("iam_oid", "oid", None),+ ("arn", "text", "C"),+ ("pg_role_oid", "oid", None),+ ("pg_role_name", "name", "C"),+ ("grantor_pg_role_oid", "oid", None),+ ("grantor_pg_role_name", "name", "C"),+ ]+ finally:+ conn.close()++ def test_grant_maps_the_arn_and_reports_the_grantor(self, dsql_proxy, role):+ conn = _pg_connect(dsql_proxy)+ try:+ cur = conn.cursor()+ cur.execute(f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}'")+ assert cur.statusmessage == "AWS IAM GRANT"+ assert self._mappings(cur, role) == [(_PROBE_ROLE_ARN, role, "admin")]+ cur.execute(+ "SELECT m.pg_role_oid = r.oid, m.grantor_pg_role_oid = g.oid "+ "FROM sys.iam_pg_role_mappings m JOIN pg_roles r ON r.rolname = m.pg_role_name "+ f"JOIN pg_roles g ON g.rolname = 'admin' WHERE m.pg_role_name = '{role}'"+ )+ assert cur.fetchall() == [(True, True)]+ finally:+ conn.close()++ def test_one_arn_keeps_one_iam_oid_across_roles(self, dsql_proxy, role):+ conn = _pg_connect(dsql_proxy)+ try:+ cur = conn.cursor()+ cur.execute(f"CREATE ROLE {role}_b")+ cur.execute(f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}'")+ cur.execute(f"AWS IAM GRANT {role}_b TO '{_PROBE_ROLE_ARN}'")+ cur.execute(+ "SELECT count(DISTINCT iam_oid) FROM sys.iam_pg_role_mappings "+ f"WHERE pg_role_name IN ('{role}', '{role}_b')"+ )+ assert cur.fetchone() == (1,)+ cur.execute(+ "SELECT count(*) FROM pg_roles r JOIN sys.iam_pg_role_mappings m "+ f"ON r.oid = m.iam_oid WHERE m.pg_role_name = '{role}'"+ )+ assert cur.fetchone() == (0,)+ cur.execute(f"AWS IAM REVOKE {role}_b FROM '{_PROBE_ROLE_ARN}'")+ cur.execute(f"DROP ROLE {role}_b")+ finally:+ conn.close()++ def test_regranting_raises_a_notice_and_grants_the_rest(self, dsql_proxy, role):+ conn = _pg_connect(dsql_proxy)+ try:+ cur = conn.cursor()+ cur.execute(f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}'")+ del conn.notices[:]+ cur.execute(f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}', '{_PROBE_USER_ARN}'")+ assert cur.statusmessage == "AWS IAM GRANT"+ assert conn.notices == [+ f'NOTICE: IAM role "{_PROBE_ROLE_ARN}" has already been granted access '+ f'to role "{role}" by role "admin"\n'+ ]+ assert self._mappings(cur, role) == [+ (_PROBE_ROLE_ARN, role, "admin"),+ (_PROBE_USER_ARN, role, "admin"),+ ]+ finally:+ conn.close()++ def test_an_unknown_role_is_refused(self, dsql_proxy):+ psycopg2 = pytest.importorskip("psycopg2")+ conn = _pg_connect(dsql_proxy)+ try:+ for sql in (+ f"AWS IAM GRANT missing_role TO '{_PROBE_ROLE_ARN}'",+ f"AWS IAM REVOKE missing_role FROM '{_PROBE_ROLE_ARN}'",+ ):+ with pytest.raises(psycopg2.Error) as exc:+ conn.cursor().execute(sql)+ assert exc.value.pgcode == "42704"+ assert 'role "missing_role" does not exist' in str(exc.value)+ finally:+ conn.close()++ def test_the_signed_in_role_cannot_grant_itself(self, dsql_proxy):+ psycopg2 = pytest.importorskip("psycopg2")+ conn = _pg_connect(dsql_proxy)+ try:+ with pytest.raises(psycopg2.Error) as exc:+ conn.cursor().execute(f"AWS IAM GRANT admin TO '{_PROBE_ROLE_ARN}'")+ assert exc.value.pgcode == "42501"+ assert exc.value.pgerror.startswith(+ 'ERROR: permission denied to grant IAM trust with role "admin"'+ )+ assert exc.value.diag.message_detail == (+ 'Only roles with the ADMIN option on role "admin" may grant IAM trust to this role.'+ )+ finally:+ conn.close()++ def test_rollback_undoes_a_grant(self, dsql_proxy, role):+ conn = _pg_connect(dsql_proxy, autocommit=False)+ try:+ cur = conn.cursor()+ cur.execute(f"AWS IAM GRANT {role} TO '{_IN_TXN_ARN}'")+ assert self._mappings(cur, role) == [(_IN_TXN_ARN, role, "admin")]+ conn.rollback()+ assert self._mappings(cur, role) == []+ finally:+ conn.close()++ def test_a_grant_after_other_ddl_in_a_transaction_is_refused(self, dsql_proxy, role):+ psycopg2 = pytest.importorskip("psycopg2")+ conn = _pg_connect(dsql_proxy, autocommit=False)+ try:+ cur = conn.cursor()+ cur.execute("CREATE TABLE iam_after_ddl (id int PRIMARY KEY)")+ with pytest.raises(psycopg2.Error) as exc:+ cur.execute(f"AWS IAM GRANT {role} TO '{_IN_TXN_ARN}'")+ assert exc.value.pgcode == "0A000"+ assert "multiple ddl statements not supported in a transaction" in str(exc.value)+ finally:+ conn.rollback()+ conn.close()++ def test_a_grant_after_a_select_in_a_transaction_commits(self, dsql_proxy, role):+ conn = _pg_connect(dsql_proxy, autocommit=False)+ try:+ cur = conn.cursor()+ cur.execute("SELECT 1")+ cur.execute(f"AWS IAM GRANT {role} TO '{_IN_TXN_ARN}'")+ conn.commit()+ assert self._mappings(cur, role) == [(_IN_TXN_ARN, role, "admin")]+ finally:+ conn.close()++ def test_revoke_removes_the_mapping_and_warns_when_there_is_none(self, dsql_proxy, role):+ conn = _pg_connect(dsql_proxy)+ try:+ cur = conn.cursor()+ cur.execute(f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}', '{_PROBE_USER_ARN}'")+ cur.execute(f"AWS IAM REVOKE {role} FROM '{_PROBE_ROLE_ARN}'")+ assert cur.statusmessage == "AWS IAM REVOKE"+ assert self._mappings(cur, role) == [(_PROBE_USER_ARN, role, "admin")]+ del conn.notices[:]+ cur.execute(f"AWS IAM REVOKE {role} FROM '{_PROBE_ROLE_ARN}'")+ assert cur.statusmessage == "AWS IAM REVOKE"+ assert conn.notices == [+ f'WARNING: IAM role "{_PROBE_ROLE_ARN}" has not been granted access '+ f'to role "{role}"\n'+ ]+ finally:+ conn.close()++ @pytest.mark.parametrize("failure", ["unreachable", "refused by the backend"])+ def test_a_catalog_that_cannot_be_created_leaves_the_connection_usable(+ self, dsql_proxy, pg_backend, role, monkeypatch, failure+ ):+ psycopg2 = pytest.importorskip("psycopg2")+ name = f"iam_db_{secrets.token_hex(4)}"+ admin = psycopg2.connect(+ host=pg_backend[0], port=pg_backend[1], user="postgres", dbname="postgres"+ )+ admin.autocommit = True+ admin.cursor().execute(f"CREATE DATABASE {name}")+ attempts = []++ async def refused(conn):+ attempts.append(conn.database)+ if failure == "unreachable":+ raise OSError("connection refused")+ return "55P03 canceling statement due to lock timeout"++ monkeypatch.setattr(pgproxy, "_create_iam_catalog", refused)+ try:+ conn = psycopg2.connect(+ host="127.0.0.1", port=dsql_proxy, user="admin", password="x", dbname=name+ )+ conn.autocommit = True+ try:+ for _ in range(2):+ with pytest.raises(psycopg2.Error) as exc:+ conn.cursor().execute(f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}'")+ assert exc.value.pgcode == "42P01"+ cur = conn.cursor()+ cur.execute("SELECT 1")+ assert cur.fetchone() == (1,)+ assert attempts == [name, name]+ finally:+ conn.close()+ finally:+ admin.cursor().execute(f"DROP DATABASE {name} WITH (FORCE)")+ admin.close()++ def test_the_extended_protocol_is_refused(self, dsql_proxy, role):+ with _WireClient(dsql_proxy) as c:+ for sql in (+ f"AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}'",+ f"AWS IAM GRANT admin TO '{_PROBE_ROLE_ARN}'",+ ):+ result = c.extended(sql)+ assert result.sqlstate == "0A000"+ assert "simple query protocol" in result.message+ assert c.extended(+ "SELECT count(*) FROM sys.iam_pg_role_mappings WHERE pg_role_name = $1", [role]+ ).rows == [("0",)]++ def test_quoting_holds_without_standard_conforming_strings(self, dsql_proxy, role):+ psycopg2 = pytest.importorskip("psycopg2")+ conn = _pg_connect(dsql_proxy)+ try:+ cur = conn.cursor()+ cur.execute("SET standard_conforming_strings = off")+ with pytest.raises(psycopg2.Error) as exc:+ cur.execute(+ "AWS IAM GRANT \"x\\' AS oid) SELECT 1, 2 WHERE false; "+ "CREATE TABLE iam_injected (id serial); --\" "+ "TO 'arn:aws:iam::123456789012:role/x'"+ )+ assert exc.value.pgcode == "42704"+ cur.execute("SET standard_conforming_strings = on")+ cur.execute(+ "SELECT count(*) FROM pg_tables WHERE tablename = 'iam_injected'"+ )+ assert cur.fetchone() == (0,)+ cur.execute(f"AWS IAM GRANT {role} TO 'arn:aws:iam::123456789012:role/a\\b'")+ assert [m[0] for m in self._mappings(cur, role)] == [+ "arn:aws:iam::123456789012:role/a\\b"+ ]+ cur.execute(f"AWS IAM REVOKE {role} FROM 'arn:aws:iam::123456789012:role/a\\b'")+ finally:+ conn.close()++ def test_a_multi_statement_batch_is_refused(self, dsql_proxy, role):+ with _WireClient(dsql_proxy) as c:+ result = c.simple(f"SELECT 1; AWS IAM GRANT {role} TO '{_PROBE_ROLE_ARN}'")+ assert result.sqlstate == "0A000"+ assert "not in a batch" in result.message+ assert c.simple(+ f"SELECT count(*) FROM sys.iam_pg_role_mappings WHERE pg_role_name = '{role}'"+ ).rows == [("0",)]++ def test_the_catalog_appears_only_in_a_database_that_uses_it(+ self, dsql_proxy, pg_backend+ ):+ psycopg2 = pytest.importorskip("psycopg2")+ name = f"iam_db_{secrets.token_hex(4)}"+ admin = psycopg2.connect(+ host=pg_backend[0], port=pg_backend[1], user="postgres", dbname="postgres"+ )+ admin.autocommit = True+ admin.cursor().execute(f"CREATE DATABASE {name}")+ try:+ main = _pg_connect(dsql_proxy)+ main.cursor().execute("SELECT count(*) FROM sys.iam_pg_role_mappings")+ main.close()+ with _WireClient(dsql_proxy, database=name) as c:+ result = c.extended(f"AWS IAM GRANT admin TO '{_PROBE_ROLE_ARN}'")+ assert result.sqlstate == "0A000"+ conn = psycopg2.connect(+ host="127.0.0.1", port=dsql_proxy, user="admin", password="x", dbname=name+ )+ conn.autocommit = True+ try:+ cur = conn.cursor()+ cur.execute("SELECT count(*) FROM pg_namespace WHERE nspname = 'sys'")+ assert cur.fetchone() == (0,)+ cur.execute("SELECT count(*) FROM sys.iam_pg_role_mappings")+ assert cur.fetchone() == (0,)+ finally:+ conn.close()+ finally:+ admin.cursor().execute(f"DROP DATABASE {name} WITH (FORCE)")+ admin.close()++@requires_docker@pytest.mark.data_planeclass TestLiveProxy: