Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

test(auth): validate rds (#1856)

Nahuel Nucera176584fa41a0
diff --git a/CHANGELOG.md b/CHANGELOG.md
index a9d6bf02..3872f1e5 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,18 +7,21 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
-### Fixed
-
-- **CloudFormation — `AWS::SQS::Queue` applies every queue property** — `RedrivePolicy`, `RedriveAllowPolicy`, `KmsMasterKeyId`, `KmsDataKeyReusePeriodSeconds`, `SqsManagedSseEnabled`, `DeduplicationScope` and `FifoThroughputLimit` were dropped on create and update, so a dead-letter queue declared in a template never received messages, and a value SQS refuses now fails the resource instead of being stored. Contributed by @iot-rocket.
-- **Lambda — Docker executor honors timeouts above 300 seconds** — pass the configured `Timeout` to AWS RIE through `AWS_LAMBDA_FUNCTION_TIMEOUT`, preventing its default 300-second limit from ending longer invocations early. Timeout updates recycle warm containers so the RIE deadline follows the new configuration (#1844).
-- **KMS — `GenerateDataKey` returns `KeyMaterialId`** — requests without `Recipient` now include the identifier of the wrapping key material. The identifier remains stable across requests and persistence reloads, and changes when the wrapping material changes. Fixes #1853.
-- **CloudFormation — a nested stack's update deletes the resources its template drops** — a resource removed from the child template, or created by a failed child update that was rolled back, stayed in its service and in the nested stack's resource list. Contributed by @iot-rocket.
### Added
-- **KMS — grants** — `CreateGrant`, `RevokeGrant` and `RetireGrant`, and `ListGrants` returns the grants they create instead of always an empty list. `CreateGrant` returns a `GrantId` and a `GrantToken`, follows the key state (`DisabledException` for a disabled key, `KMSInvalidStateException` pending deletion, replica deletion or import), rejects operations the key cannot perform (`Sign` / `Verify` / `GenerateMac` / `VerifyMac` on a symmetric encryption key, data key operations on an asymmetric key, anything but `GenerateMac` / `VerifyMac` / `DescribeKey` / `CreateGrant` / `RetireGrant` on an HMAC key) with `ValidationException`, and with a `Name` a retry with identical parameters returns the original `GrantId` with a new token. `ListGrants` filters by `GrantId` and `GranteePrincipal` and pages with `Limit` (1 to 100, default 50) and `Marker`. `RetireGrant` takes a `GrantToken`, or a `KeyId` and `GrantId`. All three honour `DryRun`, and grants persist with the key. Grants are recorded but not evaluated when authorizing other calls, as with key policies. Contributed by @DaviReisVieira.
+- **RDS — IAM database authentication for MySQL and Aurora MySQL** — users created `IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS'` log in with an SDK-generated token over `mysql_clear_password`, as on AWS. The instance or cluster must have `IAMDatabaseAuthenticationEnabled`; with `AUTH=true` the token and the `rds-db:connect` policy are verified too. `ModifyDBInstance` accepts `EnableIAMDatabaseAuthentication`. Contributed by @Areson.
+- **KMS — grants** — `CreateGrant`, `RevokeGrant` and `RetireGrant`; `ListGrants` now returns the grants they create, filtered by `GrantId` / `GranteePrincipal` and paged with `Limit` / `Marker`. `CreateGrant` follows the key state, rejects operations the key type cannot perform, and is idempotent for a named grant. Grants persist with the key and are not evaluated for authorization. Contributed by @DaviReisVieira.
+
### Fixed
+- **KMS — `KeyMaterialId`** — `GenerateDataKey`, `GenerateDataKeyWithoutPlaintext`, `GenerateDataKeyPair`, `GenerateDataKeyPairWithoutPlaintext`, `Decrypt`, `ImportKeyMaterial` and `DeleteImportedKeyMaterial` return the identifier of the key material they used, and `DescribeKey` reports it as `CurrentKeyMaterialId` for symmetric keys. The identifier stays the same until the key material changes. Contributed by @zlberto.
- **Lambda — VPC configuration includes `VpcId`** — `CreateFunction`, `GetFunction`, `GetFunctionConfiguration`, and `UpdateFunctionConfiguration` now report the VPC of the configured subnets. Previously, VPC-attached functions returned only subnet and security group IDs. Contributed by @jayjanssen.
+- **Lambda — Docker executor honors timeouts above 300 seconds** — pass the configured `Timeout` to AWS RIE through `AWS_LAMBDA_FUNCTION_TIMEOUT`, preventing its default 300-second limit from ending longer invocations early. Timeout updates recycle warm containers so the RIE deadline follows the new configuration. Contributed by @gakuto-cw21.
+- **Lambda — a Docker invocation that reaches its timeout fails** — the emulator's plain-text `Task timed out` reply was returned as a successful payload, so Step Functions recorded `TaskSucceeded` and skipped `Catch`. Contributed by @drakeo338.
+- **CloudFormation — `AWS::SQS::Queue` applies every queue property** — `RedrivePolicy`, `RedriveAllowPolicy`, `KmsMasterKeyId`, `KmsDataKeyReusePeriodSeconds`, `SqsManagedSseEnabled`, `DeduplicationScope` and `FifoThroughputLimit` were dropped on create and update, so a dead-letter queue declared in a template never received messages, and a value SQS refuses now fails the resource instead of being stored. A queue from a template also defaults to a 1 MiB `MaximumMessageSize` and SSE-SQS encryption, as on AWS. Contributed by @iot-rocket.
+- **CloudFormation — a nested stack's update deletes the resources its template drops** — a resource removed from the child template, or created by a failed child update that was rolled back, stayed in its service and in the nested stack's resource list. A dropped resource with a `Retain` or `Snapshot` `DeletionPolicy` is kept or snapshotted first, as on AWS. Contributed by @iot-rocket.
+- **RDS — a persisted instance stays reachable after a restart** — when its saved host port was taken, the instance moved to a new port but `Endpoint.Port` kept the old one.
+- **RDS — pending boolean modifications read as `true`** — `PendingModifiedValues` wrote Python `True`/`False`, which the SDKs parse as `false`.
## [1.5.18] — 2026-09-28
diff --git a/contrib/mysql-iam-plugin/README.md b/contrib/mysql-iam-plugin/README.md
index 2f2dd929..e874fdc0 100644
--- a/contrib/mysql-iam-plugin/README.md
+++ b/contrib/mysql-iam-plugin/README.md
@@ -3,90 +3,19 @@
This directory contains MiniStack's server-side compatibility plugin for
Aurora MySQL users declared with `AWSAuthenticationPlugin`.
-The initial L0 implementation deliberately rejects every login, matching
-MySQL's `mysql_no_login` behavior. It supports provider workflows that create,
-alter, grant, revoke, inspect, and drop IAM-authenticated users without
-enabling direct IAM logins before the isolated proxy topology is provisioned.
-
-## Stage 7: inactive Python gatekeeper
-
-`ministack/core/mysqlproxy.py` implements the narrow MySQL connection adapter.
-It relays ordinary password authentication to MySQL and delegates IAM admission
-to a caller-supplied Python authorizer. No C++ HTTP callback or libcurl dependency
-is needed. The opt-in connection fixture uses the existing `rds_iam._decision`
-implementation with a resource capability and SDK-signed tokens; it does not
-duplicate SigV4 or IAM policy evaluation.
-
-`aws_auth_plugin.cc` is the single plugin implementation. Its minimal accepting
-path is enabled by `-DMINISTACK_IAM_PROXY_AUTH=1` only in the isolated live test
-fixture. This compile-time switch defaults to zero; it is not a runtime setting
-or a second AUTH flag. **Normal builds do not enable acceptance.**
-`Dockerfile.full` continues to build the default reject-all mode from that source.
-The accepting variant must never replace that artifact while MySQL is directly
-reachable. A caller inside the trusted backend namespace can bypass the proxy;
-the tests demonstrate both that bypass and denial from outside the namespace.
-
-MySQL selects the private `ministack_iam_gate_v1` client method for IAM accounts.
-The adapter rejects client claims of that method and translates the backend's
-auth switch to `mysql_clear_password`. Account creation and changes between IAM
-and password methods therefore require no cached user list. MySQL still owns
-account locks, SQL identity, grants, and password verification.
-
-The SSLRequest and subsequent login must declare identical capabilities and
-character sets. Usernames are restricted to 1–32 printable, non-space ASCII
-characters without single quotes, with handshake collation IDs 8, 33, 45, 46 or
-255. Other forms fail closed instead of risking a different identity after
-MySQL's charset conversion, quote removal or truncation. Broader username and
-encoding support is deferred; these restrictions also apply to password users.
-
-With `AUTH=true`, IAM admission requires frontend TLS before requesting a token
-and applies the existing token/policy decision. False or unset remains permissive
-for IAM credentials and transport. Valid resource capability, current binding,
-and resource IAM enablement remain required by the authorizer in either mode.
-Password users retain MySQL's password checks and account-specific SSL rules.
-
-### Validation and limits
-
-Offline handshake tests run in the normal Python test lane without image builds:
-
-```sh
-uv run --extra dev pytest tests/test_mysqlproxy.py tests/test_rds_iam.py tests/test_rds_iam_plugin.py -q
-```
-
-The opt-in live tests compile both modes of the original plugin and connect through the Python
-adapter to MySQL 8.0 and 8.4. They cover signed-token acceptance/rejection,
-password users, account changes, grants, TLS rules, method spoofing, disabled
-LOCAL INFILE, and backend isolation. They are not part of normal PR CI and do
-not rebuild plugin images there. On the existing ARM64 validation environment:
-
-```sh
-docker build -f Dockerfile.full --target plugin-build-80 -t ministack-spike-compiler80:local .
-docker build -f Dockerfile.full --target plugin-build -t ministack-iam-stage7-build:local .
-uv run --extra dev pytest contrib/mysql-iam-plugin/tests/test_connections.py -q
-```
-
-Requires Docker, OpenSSL, a native `mysql` client on PATH, and the repo's dev
-dependencies. The Python helper image is `ghcr.io/ministackorg/ministack:full`;
-local source is mounted read-only. Fixtures own and remove their containers,
-volumes and network; Compose is unnecessary. The native client covers cold-cache
-plaintext RSA authentication because the installed PyMySQL 1.2.3 cold RSA path
-returns no packet to its caller. PyMySQL covers TLS and warmed password logins.
-
-This is not yet a provisioned integration. The adapter only forwards QUERY,
-QUIT, INIT_DB and PING; reauthentication, prepared statements and compression
-are unsupported. Frames are capped at 1 MiB, socket operations at five seconds,
-and idle sessions at ten seconds. These bounds are not a whole-handshake
-deadline or a concurrency limit. Backend TLS is encrypted but its certificate
-is not verified. Host-specific account selection is unproven: fixtures use `%`
-accounts and MySQL sees the proxy's loopback address. Broader client compatibility
-and fragmentation/multi-statement behavior still need validation.
-
-Item 8 of #1744 must provide isolated backend networking, endpoint/resource and
-capability provisioning, lifecycle handling, TLS certificates, and tests through
-normal MiniStack provisioning before activating the accepting shim. It should
-expose a supported internal decision API rather than the fixture's private seam.
-The authorizer fixture creates local state; it is not production provisioning
-or validation against AWS. Token expiry affects new logins, not existing sessions.
+Like AWS's plugin, it asks the client for `mysql_clear_password` and reads the
+RDS IAM token as the password, so the `mysql` client needs
+`--enable-cleartext-plugin`. The plugin POSTs the user and token to MiniStack's
+broker (`/_ministack/rds/iam-auth`) using the address and per-resource
+capability MiniStack writes to `/etc/ministack/rds-iam.conf` when the container
+is ready. It accepts only an explicit allow; a missing config, a timeout or a
+malformed answer denies, and MySQL returns its own `1045 Access denied`.
+
+The broker always requires `IAMDatabaseAuthenticationEnabled` on the instance or
+cluster. With `AUTH=true` it also verifies the token and evaluates
+`rds-db:connect`. Password users, TLS (`REQUIRE SSL`), account locks and grants
+stay MySQL's own. The broker call is plain HTTP, so IAM logins fail closed when
+MiniStack serves HTTPS only (`USE_SSL=1`).
## Bundled compatibility artifact
diff --git a/contrib/mysql-iam-plugin/aws_auth_plugin.cc b/contrib/mysql-iam-plugin/aws_auth_plugin.cc
index 7c6bbb23..66934ea7 100644
--- a/contrib/mysql-iam-plugin/aws_auth_plugin.cc
+++ b/contrib/mysql-iam-plugin/aws_auth_plugin.cc
@@ -1,17 +1,21 @@
/*
- * MiniStack's Aurora MySQL AWSAuthenticationPlugin compatibility adapter.
+ * MiniStack's Aurora MySQL AWSAuthenticationPlugin.
*
- * Normal builds reject logins until isolated proxy provisioning is available.
- * Only isolated connection tests currently enable the proxy-approved path.
- * That path does not verify credentials: clients must not reach it directly.
+ * Like AWS's plugin it asks the client for mysql_clear_password and reads the
+ * RDS IAM token as the password. MiniStack decides: the plugin forwards the
+ * user and token to its broker and accepts only an explicit allow. A missing
+ * config, a timeout or any malformed answer denies.
*/
+#include <netdb.h>
#include <stddef.h>
+#include <stdio.h>
#include <string.h>
+#include <sys/socket.h>
+#include <sys/time.h>
+#include <unistd.h>
-#ifndef MINISTACK_IAM_PROXY_AUTH
-#define MINISTACK_IAM_PROXY_AUTH 0
-#endif
+#include <string>
// MYSQL_ABI_CHECK omits MySQL's internal compiler headers. Dynamic plugin
// declarations still need the public-symbol visibility wrapper they provide.
@@ -21,24 +25,86 @@
#include <mysql/plugin_auth.h>
-static int authenticate(MYSQL_PLUGIN_VIO *vio,
- MYSQL_SERVER_AUTH_INFO *info) {
-#if MINISTACK_IAM_PROXY_AUTH
- if (!vio || !vio->read_packet || !info) return CR_ERROR;
+// Written by MiniStack when the container is ready: "<host> <port> <capability>".
+static const char CONFIG_PATH[] = "/etc/ministack/rds-iam.conf";
+
+static bool json_string(std::string &out, const std::string &value) {
+ out += '"';
+ for (unsigned char c : value) {
+ if (c < 0x20 || c == 0x7f) return false;
+ if (c == '"' || c == '\\') out += '\\';
+ out += static_cast<char>(c);
+ }
+ out += '"';
+ return true;
+}
+
+static bool broker_allows(const std::string &user, const std::string &token) {
+ char host[256], port[6], capability[65];
+ FILE *config = fopen(CONFIG_PATH, "r");
+ if (!config) return false;
+ int fields = fscanf(config, "%255s %5s %64s", host, port, capability);
+ fclose(config);
+ if (fields != 3 || strlen(capability) != 64) return false;
+
+ std::string body = "{\"username\":";
+ if (!json_string(body, user)) return false;
+ body += ",\"token\":";
+ if (!json_string(body, token)) return false;
+ body += '}';
+ std::string request = "POST /_ministack/rds/iam-auth HTTP/1.1\r\nHost: ";
+ request += host;
+ request += "\r\nContent-Type: application/json\r\nX-Ministack-RDS-Capability: ";
+ request += capability;
+ request += "\r\nContent-Length: " + std::to_string(body.size());
+ request += "\r\nConnection: close\r\n\r\n" + body;
+
+ struct addrinfo hints = {}, *addresses = nullptr;
+ hints.ai_socktype = SOCK_STREAM;
+ if (getaddrinfo(host, port, &hints, &addresses) != 0) return false;
+ int fd = -1;
+ struct timeval timeout = {3, 0};
+ for (struct addrinfo *a = addresses; a && fd < 0; a = a->ai_next) {
+ fd = socket(a->ai_family, a->ai_socktype, a->ai_protocol);
+ if (fd < 0) continue;
+ setsockopt(fd, SOL_SOCKET, SO_SNDTIMEO, &timeout, sizeof timeout);
+ setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &timeout, sizeof timeout);
+ if (connect(fd, a->ai_addr, a->ai_addrlen) != 0) {
+ close(fd);
+ fd = -1;
+ }
+ }
+ freeaddrinfo(addresses);
+ if (fd < 0) return false;
+
+ bool sent = true;
+ for (size_t offset = 0; sent && offset < request.size();) {
+ ssize_t n = send(fd, request.data() + offset, request.size() - offset, MSG_NOSIGNAL);
+ sent = n > 0;
+ if (sent) offset += n;
+ }
+ std::string response;
+ char buffer[1024];
+ ssize_t n;
+ while (sent && response.size() < 8192 && (n = recv(fd, buffer, sizeof buffer, 0)) > 0)
+ response.append(buffer, n);
+ close(fd);
+ return sent && response.compare(0, 13, "HTTP/1.1 200 ") == 0 &&
+ response.find("{\"allowed\":true}") != std::string::npos;
+}
+
+static int authenticate(MYSQL_PLUGIN_VIO *vio, MYSQL_SERVER_AUTH_INFO *info) {
unsigned char *packet = nullptr;
- info->password_used = PASSWORD_USED_YES;
int size = vio->read_packet(vio, &packet);
- if (size < 1 || !packet || packet[size - 1] != 0 || !info->user_name ||
- info->user_name_length == 0 ||
- strlen(info->authenticated_as) != info->user_name_length ||
- memcmp(info->authenticated_as, info->user_name, info->user_name_length))
+ if (size < 0 || !packet) return CR_ERROR;
+ info->password_used = PASSWORD_USED_YES;
+ std::string token(reinterpret_cast<char *>(packet), size);
+ if (!token.empty() && token.back() == '\0') token.pop_back();
+ std::string user(info->user_name, info->user_name_length);
+ // Anonymous accounts would authorize a name other than the account's.
+ if (token.empty() || user.empty() || user != info->authenticated_as)
return CR_ERROR;
- return CR_OK; // Python proxy owns IAM admission; isolation is mandatory.
-#else
- (void)vio;
- (void)info;
- return CR_ERROR;
-#endif
+ return broker_allows(user, token) ? CR_OK : CR_ERROR;
}
static int generate_authentication_string(char *outbuf,
@@ -70,11 +136,7 @@ static int set_salt(const char *password, unsigned int password_len,
static struct st_mysql_auth aws_auth_handler = {
MYSQL_AUTHENTICATION_INTERFACE_VERSION,
-#if MINISTACK_IAM_PROXY_AUTH
- "ministack_iam_gate_v1",
-#else
- NULL,
-#endif
+ "mysql_clear_password",
authenticate,
generate_authentication_string,
validate_authentication_string,
diff --git a/contrib/mysql-iam-plugin/tests/gatekeeper_server.py b/contrib/mysql-iam-plugin/tests/gatekeeper_server.py
deleted file mode 100644
index f4923ae4..00000000
--- a/contrib/mysql-iam-plugin/tests/gatekeeper_server.py
+++ /dev/null
@@ -1,16 +0,0 @@
-"""Fixture launcher; deliberately not part of normal RDS provisioning."""
-import os
-import ssl
-
-from proof_authorization import bootstrap
-
-from ministack.core.mysqlproxy import Handler, Server
-
-if __name__ == "__main__":
- with Server(("0.0.0.0", int(os.environ["SPIKE_PORT"])), Handler) as server:
- server.authorize = bootstrap()
- server.strict = os.environ.get("AUTH", "").lower() == "true"
- server.tls = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
- server.tls.minimum_version = ssl.TLSVersion.TLSv1_2
- server.tls.load_cert_chain("/tls/cert.pem", "/tls/key.pem")
- server.serve_forever()
diff --git a/contrib/mysql-iam-plugin/tests/proof_authorization.py b/contrib/mysql-iam-plugin/tests/proof_authorization.py
deleted file mode 100644
index 085c0390..00000000
--- a/contrib/mysql-iam-plugin/tests/proof_authorization.py
+++ /dev/null
@@ -1,39 +0,0 @@
-"""Fixture-only bootstrap of real MiniStack authorizer state, not provisioning."""
-import os
-
-from ministack.core import rds_iam
-from ministack.services import iam, rds
-
-ACCOUNT = "123456789012"
-REGION = "us-east-1"
-HOST = "database.spike.us-east-1.rds.amazonaws.com"
-KEY = "AKIASPIKELOCALTEST"
-
-
-def bootstrap():
- iam._users.set_scoped(ACCOUNT, None, "spike", {
- "UserName": "spike", "Arn": f"arn:aws:iam::{ACCOUNT}:user/spike",
- })
- iam._access_keys.set_scoped(ACCOUNT, None, KEY, {
- "UserName": "spike", "Status": "Active", "SecretAccessKey": os.environ["SPIKE_SECRET"],
- })
- allowed = ["iam_user", "iam_locked", "dynamic_user"]
- iam._user_inline_policies.set_scoped(ACCOUNT, None, "spike", {"connect": {
- "Statement": [{"Effect": "Allow", "Action": "rds-db:connect", "Resource": [
- f"arn:aws:rds-db:{REGION}:{ACCOUNT}:dbuser:db-SPIKE/{u}" for u in allowed
- ]}],
- }})
- rds._instances.set_scoped(ACCOUNT, REGION, "spike", {
- "Engine": "mysql", "DbiResourceId": "db-SPIKE",
- "IAMDatabaseAuthenticationEnabled": True,
- "Endpoint": {"Address": HOST, "Port": 3306},
- })
- cap = rds_iam.issue_capability(account_id=ACCOUNT, region=REGION,
- resource_kind="instance", resource_identifier="spike")
-
- def authorize(user, token, strict):
- # Reuse the existing capability/current-resource/AUTH decision. A
- # production refactor should expose this private seam as a public API.
- return rds_iam._decision(cap.encode(), {"username": user, "token": token}, strict)
-
- return authorize
diff --git a/contrib/mysql-iam-plugin/tests/test_connections.py b/contrib/mysql-iam-plugin/tests/test_connections.py
deleted file mode 100644
index 03127bf8..00000000
--- a/contrib/mysql-iam-plugin/tests/test_connections.py
+++ /dev/null
@@ -1,349 +0,0 @@
-"""Opt-in live tests: accepting shim + Python gate + real password users."""
-import datetime as dt
-import os
-import secrets
-import socket
-import ssl
-import subprocess
-import time
-from pathlib import Path
-from unittest.mock import patch
-
-import boto3
-import pymysql
-import pytest
-from proof_authorization import HOST, KEY, REGION
-
-SOURCE = Path(__file__).resolve().parent
-LABEL = "ministack.task=accepting-gate-spike"
-PYTHON_IMAGE = "ghcr.io/ministackorg/ministack:full"
-
-
-def signed(secret, *, user="iam_user", host=HOST, age=0):
- client = boto3.client("rds", region_name=REGION, aws_access_key_id=KEY, aws_secret_access_key=secret)
- now = dt.datetime.now(dt.timezone.utc) - dt.timedelta(seconds=age)
- with patch("botocore.auth.get_current_datetime", return_value=now):
- return client.generate_db_auth_token(DBHostname=host, Port=3306, DBUsername=user)
-
-
-def run(*args, **kwargs):
- return subprocess.run(args, text=True, capture_output=True, check=True, **kwargs).stdout.strip()
-
-
[email protected](scope="module", params=["8.0", "8.4"])
-def setup(request, tmp_path_factory):
- suffix = secrets.token_hex(4)
- db = "accepting-spike-" + suffix
- network = db + "-net"
- containers = []
- root, password, secret = [secrets.token_hex(16) for _ in range(3)]
- token = signed(secret)
- temp = tmp_path_factory.mktemp("accepting-shim")
- compiler = "ministack-spike-compiler80:local" if request.param == "8.0" else "ministack-iam-stage7-build:local"
- linked = run("docker", "run", "--rm", "--label", LABEL, "--entrypoint", "sh",
- "-v", f"{SOURCE.parent}:/src:ro", "-v", f"{temp}:/out", compiler, "-c",
- 'set -e; header=$(rpm -ql mysql-community-debugsource | grep "/include/mysql/plugin_auth.h$" | head -1); '
- 'g++ -Wall -Wextra -Werror -shared -fPIC -DMYSQL_ABI_CHECK -DMYSQL_DYNAMIC_PLUGIN '
- '-DMINISTACK_IAM_PROXY_AUTH=1 '
- '-I"$(dirname "$(dirname "$header")")" /src/aws_auth_plugin.cc -o /out/accepting_shim.so; '
- 'g++ -Wall -Wextra -Werror -shared -fPIC -DMYSQL_ABI_CHECK -DMYSQL_DYNAMIC_PLUGIN '
- '-I"$(dirname "$(dirname "$header")")" /src/aws_auth_plugin.cc -o /out/rejecting_shim.so; '
- 'ldd /out/accepting_shim.so; ldd /out/rejecting_shim.so')
- assert "libcurl" not in linked
- run("openssl", "req", "-x509", "-newkey", "rsa:2048", "-nodes", "-days", "1",
- "-subj", "/CN=localhost", "-addext", "subjectAltName=IP:127.0.0.1",
- "-keyout", str(temp / "key.pem"), "-out", str(temp / "cert.pem"))
- context = ssl.create_default_context(cafile=str(temp / "cert.pem"))
-
- def admin(sql):
- return run("docker", "exec", "-i", "-e", "MYSQL_PWD=" + root, db,
- "mysql", "-uroot", "-N", "-B", input=sql)
-
- run("docker", "network", "create", "--label", LABEL, network)
- try:
- run("docker", "run", "-d", "--name", db, "--label", LABEL, "--network", network,
- "-e", "MYSQL_ROOT_PASSWORD=" + root,
- "-p", "127.0.0.1::13306", "-p", "127.0.0.1::13307",
- # Deliberately publish the backend port too: loopback binding must
- # defeat even this accidental forwarding rule.
- "-p", "127.0.0.1::3306", "mysql:" + request.param, "--bind-address=127.0.0.1")
- containers.append(db)
- deadline = time.monotonic() + 120
- while True:
- try:
- admin("SELECT 1")
- # Entry-point temporary server also accepts socket connections;
- # require the final TCP server before installing the plugin.
- run("docker", "exec", "-e", "MYSQL_PWD=" + root, db, "mysql", "-h127.0.0.1", "-uroot", "-e", "SELECT 1")
- break
- except subprocess.CalledProcessError:
- if time.monotonic() > deadline:
- raise
- time.sleep(1)
- plugin_dir = admin("SELECT @@plugin_dir")
- run("docker", "cp", str(temp / "accepting_shim.so"), f"{db}:{plugin_dir}accepting_shim.so")
- run("docker", "cp", str(temp / "rejecting_shim.so"), f"{db}:{plugin_dir}rejecting_shim.so")
- admin("INSTALL PLUGIN AWSAuthenticationPlugin SONAME 'accepting_shim.so';"
- "CREATE DATABASE spike; CREATE TABLE spike.visible(id INT);"
- "INSERT INTO spike.visible VALUES (7); CREATE TABLE spike.hidden(id INT);")
- for user in ("iam_user", "iam_other", "iam_locked"):
- admin(f"CREATE USER '{user}'@'%' IDENTIFIED WITH AWSAuthenticationPlugin;"
- f"GRANT SELECT ON spike.visible TO '{user}'@'%';")
- for user in ("password_user", "password_locked", "password_tls"):
- admin(f"CREATE USER '{user}'@'%' IDENTIFIED WITH caching_sha2_password BY '{password}';"
- f"GRANT SELECT ON spike.visible TO '{user}'@'%';")
- admin("ALTER USER 'iam_locked'@'%' ACCOUNT LOCK;"
- "ALTER USER 'password_locked'@'%' ACCOUNT LOCK;"
- "ALTER USER 'password_tls'@'%' REQUIRE SSL;")
- ports = {}
- for strict, port in ((True, 13306), (False, 13307)):
- proxy = db + ("-strict" if strict else "-permissive")
- run("docker", "run", "-d", "--name", proxy, "--label", LABEL,
- "--network", "container:" + db, "--entrypoint", "python",
- "-v", f"{SOURCE}:/src:ro", "-v", f"{temp}:/tls:ro",
- "-v", f"{SOURCE.parents[2]}:/repo:ro", "-e", "PYTHONPATH=/repo",
- "-e", "SPIKE_PORT=" + str(port), "-e", "AUTH=" + str(strict).lower(),
- "-e", "SPIKE_SECRET=" + secret,
- PYTHON_IMAGE, "/src/gatekeeper_server.py")
- containers.append(proxy)
- ports[strict] = int(run("docker", "port", db, f"{port}/tcp").split(":")[-1])
- deadline = time.monotonic() + 20
- while True:
- try:
- with socket.create_connection(("127.0.0.1", ports[strict]), timeout=1) as sock:
- if not sock.recv(1):
- raise OSError("proxy not ready")
- break
- except OSError:
- if time.monotonic() > deadline:
- raise
- time.sleep(0.2)
- yield dict(db=db, network=network, ports=ports, tls=context, password=password,
- token=token, secret=secret, admin=admin)
- finally:
- for name in reversed(containers):
- run("docker", "rm", "-fv", name)
- run("docker", "network", "rm", network)
-
-
-def connect(setup, *, user="iam_user", password=None, strict=True, tls=True, local_infile=False):
- return pymysql.connect(host="127.0.0.1", port=setup["ports"][strict], user=user,
- password=setup["token"] if password is None else password,
- ssl=setup["tls"] if tls else None, ssl_disabled=not tls,
- local_infile=local_infile, autocommit=True,
- read_timeout=5, write_timeout=5, connect_timeout=5)
-
-
[email protected]("user", ["iam_user", "password_user"])
-def test_identity_grants_and_ddl(setup, user):
- password = setup["token"] if user == "iam_user" else setup["password"]
- with connect(setup, user=user, password=password) as conn, conn.cursor() as cur:
- cur.execute("SELECT CURRENT_USER()")
- assert cur.fetchone() == (user + "@%",)
- cur.execute("SELECT id FROM spike.visible")
- assert cur.fetchone() == (7,)
- for sql in ("SELECT * FROM spike.hidden", "INSERT INTO spike.visible VALUES(8)"):
- with pytest.raises(pymysql.MySQLError) as caught:
- cur.execute(sql)
- assert caught.value.args[0] == 1142
- cur.execute("SHOW CREATE USER CURRENT_USER()")
- expected = "AWSAuthenticationPlugin" if user == "iam_user" else "caching_sha2_password"
- assert expected in cur.fetchone()[0]
-
-
[email protected]("options", [
- {"password": "wrong"}, {"tls": False}, {"user": "iam_other"}, {"user": "unknown"},
- {"user": "iam_locked"},
-])
-def test_iam_rejections(setup, options):
- if options.get("user") == "iam_locked":
- options = {**options, "password": signed(setup["secret"], user="iam_locked")}
- with pytest.raises(pymysql.MySQLError):
- connect(setup, **options)
-
-
[email protected]("strict", [True, False])
[email protected]("tls", [True, False])
-def test_password_login_unchanged(setup, strict, tls):
- with connect(setup, user="password_user", password=setup["password"], strict=strict, tls=tls) as conn:
- conn.ping(reconnect=False)
- for user, password in (("password_user", "wrong"), ("password_locked", setup["password"])):
- with pytest.raises(pymysql.MySQLError):
- connect(setup, user=user, password=password, strict=strict, tls=tls)
-
-
-def test_password_account_tls_rule_preserved(setup):
- with connect(setup, user="password_tls", password=setup["password"]) as conn:
- conn.ping(reconnect=False)
- with pytest.raises(pymysql.MySQLError):
- connect(setup, user="password_tls", password=setup["password"], strict=False, tls=False)
-
-
-def test_iam_permissive_plaintext(setup):
- with connect(setup, password="anything", strict=False, tls=False) as conn:
- conn.ping(reconnect=False)
-
-
[email protected]("user", ["iam_user", "password_user"])
-def test_change_user_blocked(setup, user):
- password = setup["token"] if user == "iam_user" else setup["password"]
- with connect(setup, user=user, password=password) as conn:
- conn._execute_command(17, b"iam_other\0")
- with pytest.raises(pymysql.MySQLError):
- conn._read_packet()
-
-
-def test_backend_isolation_and_trusted_bypass(setup):
- port = int(run("docker", "port", setup["db"], "3306/tcp").split(":")[-1])
- with pytest.raises(pymysql.MySQLError):
- pymysql.connect(host="127.0.0.1", port=port, user="iam_user", password="anything",
- connect_timeout=2, read_timeout=2)
- # An ordinary container on the same bridge cannot reach MySQL's loopback.
- result = subprocess.run(["docker", "run", "--rm", "--label", LABEL,
- "--network", setup["network"], "--entrypoint", "python", PYTHON_IMAGE,
- "-c", "import socket; socket.create_connection(('" + setup["db"] + "',3306),2)"],
- capture_output=True, text=True)
- assert result.returncode != 0 and "ConnectionRefusedError" in result.stderr
- # Deliberate negative control: inside the trusted namespace the shim DOES
- # accept arbitrary credentials. Isolation, not the shim, is the security gate.
- output = run("docker", "run", "--rm", "--label", LABEL, "--network", "container:" + setup["db"],
- "-v", f"{SOURCE}:/src:ro", "-v", f"{SOURCE.parents[2]}:/repo:ro",
- "-e", "PYTHONPATH=/repo", "--entrypoint", "python", PYTHON_IMAGE, "/src/trusted_bypass.py")
- assert output == "backend accepted unverified login"
-
-
[email protected]("case", ["expired", "bad-signature", "wrong-host", "policy-denied"])
-def test_real_iam_denials(setup, case):
- args = {"secret": setup["secret"]}
- user = "iam_user"
- if case == "expired":
- args["age"] = 1800
- elif case == "bad-signature":
- args["secret"] = "incorrect-secret"
- elif case == "wrong-host":
- args["host"] = "other.spike.us-east-1.rds.amazonaws.com"
- else:
- user = args["user"] = "iam_other"
- with pytest.raises(pymysql.MySQLError):
- connect(setup, user=user, password=signed(**args))
-
-
-def test_account_changes_need_no_cached_classification(setup):
- admin = setup["admin"]
- admin("CREATE USER 'dynamic_user'@'%' IDENTIFIED WITH AWSAuthenticationPlugin;")
- try:
- token = signed(setup["secret"], user="dynamic_user")
- with connect(setup, user="dynamic_user", password=token) as conn:
- conn.ping(reconnect=False)
- admin("ALTER USER 'dynamic_user'@'%' IDENTIFIED WITH caching_sha2_password BY 'spike-password';")
- with pytest.raises(pymysql.MySQLError):
- connect(setup, user="dynamic_user", password=token)
- with connect(setup, user="dynamic_user", password="spike-password") as conn:
- conn.ping(reconnect=False)
- admin("ALTER USER 'dynamic_user'@'%' IDENTIFIED WITH AWSAuthenticationPlugin;")
- with pytest.raises(pymysql.MySQLError):
- connect(setup, user="dynamic_user", password="spike-password")
- with connect(setup, user="dynamic_user", password=token) as conn:
- conn.ping(reconnect=False)
- finally:
- admin("DROP USER 'dynamic_user'@'%';")
-
-
-def test_client_cannot_claim_private_auth_method(setup):
- from trusted_bypass import handshake
-
- from ministack.core.mysqlproxy import read, send
-
- # Permissive mode still must not permit the client to choose classification.
- with socket.create_connection(("127.0.0.1", setup["ports"][False]), timeout=3) as sock:
- read(sock)
- send(sock, 1, handshake(b"ministack_iam_gate_v1"))
- _, result = read(sock)
- assert result[:1] == b"\xff"
-
-
-def test_cold_password_rsa_with_mysql_client(setup):
- # PyMySQL 1.2.3's RSA full-auth path lacks a return packet; exercise this
- # path with the native client instead, without warming the auth cache.
- password = secrets.token_hex(16)
- setup["admin"]("CREATE USER 'cold_user'@'%' IDENTIFIED WITH caching_sha2_password BY '" + password + "';")
- try:
- result = subprocess.run(["mysql", "--no-defaults", "--local-infile=0", "--host=127.0.0.1",
- "--port=" + str(setup["ports"][True]), "--user=cold_user", "--ssl-mode=DISABLED",
- "--get-server-public-key", "-N", "-e", "SELECT CURRENT_USER()"],
- text=True, capture_output=True, env={**os.environ, "MYSQL_PWD": password})
- assert result.returncode == 0, (result.stderr, run("docker", "logs", setup["db"] + "-strict"))
- output = result.stdout.strip()
- assert output == "cold_user@%"
- finally:
- setup["admin"]("DROP USER 'cold_user'@'%';")
-
-
-def test_local_infile_stays_disabled(setup):
- admin = setup["admin"]
- admin("CREATE USER 'file_user'@'%' IDENTIFIED BY 'spike-password';"
- "GRANT INSERT ON spike.visible TO 'file_user'@'%'; SET GLOBAL local_infile=ON;")
- try:
- with connect(setup, user="file_user", password="spike-password", local_infile=True) as conn:
- with conn.cursor() as cur, pytest.raises(pymysql.MySQLError) as caught:
- cur.execute("LOAD DATA LOCAL INFILE '/nonexistent-spike-file' INTO TABLE spike.visible")
- assert caught.value.args[0] == 3948 # LOCAL disabled by negotiated capability
- finally:
- admin("DROP USER 'file_user'@'%'; SET GLOBAL local_infile=OFF;")
-
-
[email protected]("strict", [True, False])
[email protected]("change", ["database", "auth-length", "charset"])
-def test_tls_handshake_mismatch_rejected(setup, strict, change):
- from ministack.core.mysqlproxy import read, send
-
- flags = (1 << 9) | (1 << 15) | (1 << 19) | (1 << 11)
- header = flags.to_bytes(4, "little") + b"\0" * 4 + b"\x21" + b"\0" * 23
- altered = bytearray(header)
- if change == "charset":
- altered[8] = 8
- else:
- altered[:4] = (flags | (1 << (3 if change == "database" else 21))).to_bytes(4, "little")
- # With the database bit mismatch, MySQL sees the private method while an
- # unchecked proxy would parse it as a database and accept the public method.
- payload = bytes(altered) + b"iam_user\0\x01\0ministack_iam_gate_v1\0mysql_native_password\0"
- with socket.create_connection(("127.0.0.1", setup["ports"][strict]), timeout=3) as raw:
- read(raw)
- send(raw, 1, header)
- with setup["tls"].wrap_socket(raw, server_hostname="127.0.0.1") as conn:
- send(conn, 2, payload)
- _, packet = read(conn)
- assert packet[:1] == b"\xff" # Never OK or a cleartext token request.
-
-
[email protected]("strict", [True, False])
[email protected]("name,charset", [("é".encode(), 8), (b"'iam_user'", 33)])
-def test_ambiguous_sql_identity_rejected(setup, strict, name, charset):
- from ministack.core.mysqlproxy import read, send
-
- setup["admin"]("SET NAMES utf8mb4; CREATE USER IF NOT EXISTS 'é'@'%' IDENTIFIED WITH AWSAuthenticationPlugin;")
- try:
- flags = (1 << 9) | (1 << 15) | (1 << 19) | (1 << 11)
- header = flags.to_bytes(4, "little") + b"\0" * 4 + bytes([charset]) + b"\0" * 23
- with socket.create_connection(("127.0.0.1", setup["ports"][strict]), timeout=3) as raw:
- read(raw)
- send(raw, 1, header)
- with setup["tls"].wrap_socket(raw, server_hostname="127.0.0.1") as conn:
- send(conn, 2, header + name + b"\0\0mysql_native_password\0")
- _, packet = read(conn)
- assert packet[:1] == b"\xff"
- finally:
- setup["admin"]("SET NAMES utf8mb4; DROP USER 'é'@'%';")
-
-
-def test_bundled_shim_stays_fail_closed(setup):
- admin = setup["admin"]
- admin("UNINSTALL PLUGIN AWSAuthenticationPlugin;"
- "INSTALL PLUGIN AWSAuthenticationPlugin SONAME 'rejecting_shim.so';")
- try:
- for strict in (True, False):
- with pytest.raises(pymysql.MySQLError):
- connect(setup, strict=strict)
- finally:
- admin("UNINSTALL PLUGIN AWSAuthenticationPlugin;"
- "INSTALL PLUGIN AWSAuthenticationPlugin SONAME 'accepting_shim.so';")
diff --git a/contrib/mysql-iam-plugin/tests/trusted_bypass.py b/contrib/mysql-iam-plugin/tests/trusted_bypass.py
deleted file mode 100644
index aed54253..00000000
--- a/contrib/mysql-iam-plugin/tests/trusted_bypass.py
+++ /dev/null
@@ -1,21 +0,0 @@
-"""Negative control: only run inside the isolated MySQL network namespace."""
-import socket
-import struct
-
-from ministack.core.mysqlproxy import read, send
-
-
-def handshake(plugin=b"mysql_native_password"):
- flags = (1 << 9) | (1 << 15) | (1 << 19)
- return struct.pack("<IIB23x", flags, 1024 * 1024, 45) + b"iam_user\0\0" + plugin + b"\0"
-
-
-if __name__ == "__main__":
- with socket.create_connection(("127.0.0.1", 3306), timeout=3) as sock:
- read(sock)
- send(sock, 1, handshake())
- seq, packet = read(sock)
- assert packet.startswith(b"\xfeministack_iam_gate_v1\0")
- send(sock, seq + 1, b"anything\0")
- assert read(sock)[1][:1] == b"\0"
- print("backend accepted unverified login")
diff --git a/ministack/core/mysqlproxy.py b/ministack/core/mysqlproxy.py
deleted file mode 100644
index 3dc1881c..00000000
--- a/ministack/core/mysqlproxy.py
+++ /dev/null
@@ -1,215 +0,0 @@
-"""Inactive MySQL IAM adapter: relay password auth and gate IAM auth switches.
-
-Runs in MySQL's network namespace. MySQL listens on loopback only.
-No SQL/account rewriting. The caller supplies resource-bound authorization.
-Not provisioned by the runtime: backend isolation is mandatory before activation.
-"""
-import contextlib
-import select
-import socket
-import socketserver
-import ssl
-
-
-def exact(sock, n):
- data = b""
- while len(data) < n:
- chunk = sock.recv(n - len(data))
- if not chunk:
- raise EOFError
- data += chunk
- return data
-
-
-def read(sock):
- header = exact(sock, 4)
- n = int.from_bytes(header[:3], "little")
- if n > 1024 * 1024:
- raise ValueError("adapter packet limit")
- return header[3], exact(sock, n)
-
-
-def send(sock, seq, body):
- sock.sendall(len(body).to_bytes(3, "little") + bytes([seq % 256]) + body)
-
-
-def deny(sock, seq):
- send(sock, seq, b"\xff\x15\x04#28000IAM gate denied")
-
-
-def limited_greeting(packet):
- # Do not advertise wire modes that this adapter cannot relay safely.
- packet = bytearray(packet)
- lower = packet.index(0, 1) + 14
- upper = lower + 5
- flags = int.from_bytes(packet[lower:lower + 2], "little")
- flags |= int.from_bytes(packet[upper:upper + 2], "little") << 16
- flags &= ~((1 << 5) | (1 << 26) | (1 << 7))
- packet[lower:lower + 2] = (flags & 65535).to_bytes(2, "little")
- packet[upper:upper + 2] = (flags >> 16).to_bytes(2, "little")
- return bytes(packet)
-
-
-def validate_header(reply):
- """Accept only the explicitly supported protocol and character sets."""
- if len(reply) < 32:
- raise ValueError("short handshake")
- flags = int.from_bytes(reply[:4], "little")
- if not flags & (1 << 19) or not flags & (1 << 9):
- raise ValueError("plugin authentication and protocol 4.1 required")
- if flags & ((1 << 5) | (1 << 26) | (1 << 7)):
- raise ValueError("compression and local infile unsupported")
- if not flags & (1 << 15):
- raise ValueError("secure connection capability required")
- # latin1, utf8mb3, and the tested utf8mb4 collations share ASCII identity.
- if reply[8] not in (8, 33, 45, 46, 255):
- raise ValueError("unsupported handshake charset")
-
-
-def client_identity(reply):
- """Parse enough HandshakeResponse41 to reject private-plugin spoofing."""
- validate_header(reply)
- flags = int.from_bytes(reply[:4], "little")
- end = reply.index(0, 32)
- name = reply[32:end]
- # MySQL normalizes quoted names and converts the declared charset. Until
- # those forms are supported, restrict identity to unquoted printable ASCII
- # and MySQL's 32-character limit, preventing conversion or truncation drift.
- if not 1 <= len(name) <= 32 or any(c < 33 or c > 126 or c == 39 for c in name):
- raise ValueError("invalid username")
- username = name.decode("ascii")
- pos = end + 1
- if flags & (1 << 21):
- size = reply[pos]
- pos += 1
- if size >= 251:
- width = {252: 2, 253: 3, 254: 8}.get(size)
- if width is None or pos + width > len(reply):
- raise ValueError("invalid auth length")
- size = int.from_bytes(reply[pos:pos + width], "little")
- pos += width
- pos += size
- elif flags & (1 << 15):
- pos += 1 + reply[pos]
- else:
- raise ValueError("secure connection capability required")
- if flags & (1 << 3):
- pos = reply.index(0, pos) + 1
- plugin = reply[pos:reply.index(0, pos)]
- # In particular, never forward the shim's private name from a client:
- # it could suppress the backend auth switch on which the gate depends.
- if plugin not in (b"mysql_native_password", b"caching_sha2_password"):
- raise ValueError("unsupported client auth method")
- return username
-
-
-class Handler(socketserver.BaseRequestHandler):
- def handle(self):
- front = self.request
- back = None
- seq = 0
- phase = "handshake"
- try:
- front.settimeout(5)
- back = socket.create_connection(("127.0.0.1", 3306), timeout=5)
- seq, greeting = read(back)
- send(front, seq, limited_greeting(greeting))
- seq, reply = read(front)
- secure = False
- if len(reply) == 32 and int.from_bytes(reply[:4], "little") & 2048:
- tls_header = reply
- flags = int.from_bytes(reply[:4], "little") & ~(1 << 7)
- reply = flags.to_bytes(4, "little") + reply[4:]
- validate_header(reply)
- send(back, seq, reply)
- # Encryption only on the namespace-local backend leg; server
- # identity verification is not implemented by this adapter.
- back = ssl._create_unverified_context().wrap_socket(back)
- front = self.server.tls.wrap_socket(front, server_side=True)
- secure = True
- seq, reply = read(front)
- # MySQL retains the SSLRequest capabilities/charset. Do not
- # parse a different packet layout or identity on this side.
- if len(reply) < 32 or reply[:4] != tls_header[:4] or reply[8] != tls_header[8]:
- raise ValueError("TLS handshake header mismatch")
- # libmysqlclient can set LOCAL_FILES even when --local-infile=0.
- # We never advertised it and must not enable it on the backend.
- if len(reply) >= 4:
- flags = int.from_bytes(reply[:4], "little") & ~(1 << 7)
- reply = flags.to_bytes(4, "little") + reply[4:]
- username = client_identity(reply)
- phase = "authentication"
- send(back, seq, reply)
- # Only the trusted backend can select the private IAM method.
- # No client assertion or cached list classifies the SQL account.
- while True:
- seq, packet = read(back)
- if packet[:1] == b"\xff":
- send(front, seq, packet)
- return
- if packet[:1] == b"\x00":
- send(front, seq, packet)
- break
- is_iam = packet.startswith(b"\xfeministack_iam_gate_v1\0")
- if is_iam and self.server.strict and not secure:
- deny(front, seq)
- return
- if is_iam:
- packet = b"\xfemysql_clear_password\0"
- elif packet[:1] == b"\xfe" and packet[1:].split(b"\0", 1)[0] not in (
- b"mysql_native_password", b"caching_sha2_password",
- ):
- raise ValueError("unsupported backend auth method")
- send(front, seq, packet)
- if packet == b"\x01\x03": # caching_sha2 fast-auth success; OK follows
- continue
- seq, response = read(front)
- if is_iam:
- if (len(response) > 65537 or not response.endswith(b"\0") or b"\0" in response[:-1]
- or not self.server.authorize(username, response[:-1].decode(), self.server.strict)):
- deny(front, seq + 1)
- return
- # Do not expose the frontend token to the accepting shim.
- response = b"proxy-approved\0"
- send(back, seq, response)
- phase = "commands"
- while True:
- ready = [s for s in (front, back) if isinstance(s, ssl.SSLSocket) and s.pending()]
- if not ready:
- ready, _, _ = select.select([front, back], [], [], 10)
- if not ready:
- return
- for source in ready:
- if source is front:
- seq, payload = read(front)
- # Deliberate limited command set; reject CHANGE_USER,
- # prepared statements, and LOCAL INFILE exchanges.
- if seq != 0 or not payload or payload[0] not in (1, 2, 3, 14):
- deny(front, 1)
- return
- send(back, seq, payload)
- if payload[0] == 1:
- return
- else:
- data = back.recv(65536)
- if not data:
- return
- front.sendall(data)
- except Exception as exc:
- # Fail closed without logging usernames, tokens, or exception data.
- known = {"short handshake", "plugin authentication and protocol 4.1 required",
- "compression and local infile unsupported", "invalid username", "invalid auth length",
- "secure connection capability required", "unsupported client auth method"}
- reason = str(exc) if type(exc) is ValueError and str(exc) in known else type(exc).__name__
- print("MySQL adapter rejection", phase, reason, flush=True)
- with contextlib.suppress(OSError):
- deny(front, seq + 1)
- finally:
- front.close()
- if back:
- back.close()
-
-
-class Server(socketserver.ThreadingTCPServer):
- allow_reuse_address = True
- daemon_threads = True
diff --git a/ministack/core/rds_iam.py b/ministack/core/rds_iam.py
index e29e56b6..e552f5c1 100644
--- a/ministack/core/rds_iam.py
+++ b/ministack/core/rds_iam.py
@@ -1,15 +1,11 @@
# Copyright (c) 2026 MiniStack Contributors. SPDX-License-Identifier: MIT
# Copies or substantial portions, including AI-assisted ports or rewrites, must retain this notice (see LICENSE).
-"""Strict verification of SDK-generated RDS IAM database tokens.
+"""RDS IAM database authentication for the MySQL AWSAuthenticationPlugin.
-This module has no runtime callers and does not consult AUTH. Later integration
-must explicitly gate authentication/authorization at the call site and define
-the AUTH=false behavior; adding this primitive changes no existing request path.
-Resource enablement, rds-db:connect policies, and TLS are separate checks.
-
-The broker half serves POST /_ministack/rds/iam-auth for the MySQL plugin.
-Capabilities are process-local, never persisted, never issued over HTTP; AUTH
-gates IAM enforcement only, not capability checks or resource enablement.
+The plugin POSTs each IAM login to /_ministack/rds/iam-auth with its
+container's capability. Resource enablement is always required; AUTH=true
+also verifies the token and evaluates rds-db:connect. Capabilities are
+process-local, never persisted, never issued over HTTP.
"""
import asyncio
@@ -397,15 +393,24 @@ def _decision(capability, payload, auth_enabled):
return False
if not auth_enabled:
return True
- result = authorize_rds_iam_token(
- payload["token"], account_id=binding.account_id, region=binding.region,
- resource_kind=binding.resource_kind, resource_identifier=binding.resource_identifier,
- reader_endpoint=binding.reader_endpoint, db_user=payload["username"],
- )
- # RDS lifecycle workers do not hold the broker lock. Authorization
- # looks up state independently, so discard its result if the resource
- # or its owner was replaced or disabled while that lookup ran.
- return isinstance(result, AuthorizedRdsConnection) and _binding_current(binding)
+ targets = [(binding.resource_kind, binding.resource_identifier, binding.reader_endpoint)]
+ if binding.resource_kind == "cluster":
+ # One container serves the writer, reader and member endpoints.
+ targets.append(("cluster", binding.resource_identifier, not binding.reader_endpoint))
+ targets += [("instance", m.get("DBInstanceIdentifier"), False)
+ for m in binding.owner.get("DBClusterMembers", [])]
+ for kind, identifier, reader in targets:
+ result = authorize_rds_iam_token(
+ payload["token"], account_id=binding.account_id, region=binding.region,
+ resource_kind=kind, resource_identifier=identifier,
+ reader_endpoint=reader, db_user=payload["username"],
+ )
+ # RDS lifecycle workers do not hold the broker lock. Authorization
+ # looks up state independently, so discard its result if the resource
+ # or its owner was replaced or disabled while that lookup ran.
+ if isinstance(result, AuthorizedRdsConnection):
+ return _binding_current(binding)
+ return False
def _unique_object(pairs):
diff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.py
index 258341ac..f29087c0 100644
--- a/ministack/services/cloudformation/provisioners.py
+++ b/ministack/services/cloudformation/provisioners.py
@@ -1352,7 +1352,7 @@ def _s3_delete(physical_id, props):
_SQS_QUEUE_DEFAULTS = {
"VisibilityTimeout": "30",
- "MaximumMessageSize": "262144",
+ "MaximumMessageSize": "1048576",
"MessageRetentionPeriod": "345600",
"DelaySeconds": "0",
"ReceiveMessageWaitTimeSeconds": "0",
@@ -1375,6 +1375,8 @@ def _sqs_queue_fields(props, is_fifo):
fields[key] = str(value).lower()
elif value is not None:
fields[key] = str(value)
+ if "KmsMasterKeyId" not in fields and "SqsManagedSseEnabled" not in fields:
+ fields["SqsManagedSseEnabled"] = "true" # SSE-SQS unless the template sets encryption
if "RedrivePolicy" in fields:
_sqs._validate_redrive_policy(fields["RedrivePolicy"])
_sqs._validate_numeric_attrs(fields)
@@ -4033,10 +4035,35 @@ def _cfn_nested_stack_deploy(logical_id, props, parent_stack_name, *,
f"{status_prefix}_COMPLETE", physical_id=physical_id)
if is_update:
- for stale_id in set(prev_resources) - set(ordered):
+ snapshot = previous_stack_snapshot or {}
+ old_template = snapshot.get("_template", {}) or {}
+ old_defs = old_template.get("Resources", {}) or {}
+ old_conditions = snapshot.get("_conditions", conditions)
+ stale_ids = set(prev_resources) - set(ordered)
+ # Dependents first, as the top-level update removes them.
+ try:
+ removal_order = [lid for lid in _topological_sort(old_defs, old_conditions)
+ if lid in stale_ids]
+ except ValueError:
+ removal_order = []
+ removal_order += [lid for lid in stale_ids if lid not in removal_order]
+ for stale_id in reversed(removal_order):
old = prev_resources[stale_id]
provisioned.pop(stale_id, None)
+ policy = _resource_policy(
+ old_defs.get(stale_id), "DeletionPolicy", provisioned,
+ snapshot.get("_resolved_params", {}), old_conditions,
+ old_template.get("Mappings", {}), child_name, child_stack_id,
+ )
+ if policy in _RETAINING_POLICIES:
+ _add_event(child_stack_id, child_name, stale_id, old.get("ResourceType", ""),
+ "DELETE_SKIPPED", physical_id=old.get("PhysicalResourceId", ""))
+ continue
try:
+ if policy == "Snapshot":
+ _snapshot_resource(old.get("ResourceType", ""),
+ old.get("PhysicalResourceId", ""),
+ old.get("Properties", {}))
_delete_resource(old.get("ResourceType", ""),
old.get("PhysicalResourceId", ""),
old.get("Properties", {}),
diff --git a/ministack/services/kms.py b/ministack/services/kms.py
index d337193e..10610f99 100644
--- a/ministack/services/kms.py
+++ b/ministack/services/kms.py
@@ -250,6 +250,15 @@ def _arn(key_id):
return f"arn:aws:kms:{get_region()}:{get_account_id()}:key/{key_id}"
+def _key_material_id(rec):
+ """Stable for a key's symmetric material; changes when the material does."""
+ return hmac.new(
+ rec["_symmetric_key"],
+ b"ministack:kms:key-material-id:" + rec["KeyId"].encode("utf-8"),
+ hashlib.sha256,
+ ).hexdigest()
+
+
def _key_metadata(rec):
metadata = {
"KeyId": rec["KeyId"],
@@ -273,6 +282,8 @@ def _key_metadata(rec):
# that is, when its KeyState is PendingDeletion."
if "DeletionDate" in rec:
metadata["DeletionDate"] = rec["DeletionDate"]
+ if "_symmetric_key" in rec and rec["Origin"] in ("AWS_KMS", "EXTERNAL"):
+ metadata["CurrentKeyMaterialId"] = _key_material_id(rec)
# Both are reported only for a key with imported material.
if "ExpirationModel" in rec:
metadata["ExpirationModel"] = rec["ExpirationModel"]
@@ -1174,13 +1185,16 @@ def _decrypt(data):
400,
)
- return json_response({
+ response = {
"KeyId": rec["Arn"],
"Plaintext": base64.b64encode(plaintext).decode(),
"EncryptionAlgorithm": data.get(
"EncryptionAlgorithm", "SYMMETRIC_DEFAULT"
),
- })
+ }
+ if "_symmetric_key" in rec and "Recipient" not in data:
+ response["KeyMaterialId"] = _key_material_id(rec)
+ return json_response(response)
def _generate_data_key_common(data, action="GenerateDataKey"):
@@ -1238,13 +1252,7 @@ def _generate_data_key(data):
"CiphertextBlob": base64.b64encode(result).decode(),
}
if "Recipient" not in data:
- # Derive an emulator identifier from the wrapping material and key ID.
- # Both survive persistence and are shared by multi-Region replicas.
- response["KeyMaterialId"] = hmac.new(
- rec["_symmetric_key"],
- b"ministack:kms:key-material-id:" + rec["KeyId"].encode("utf-8"),
- hashlib.sha256,
- ).hexdigest()
+ response["KeyMaterialId"] = _key_material_id(rec)
return json_response(response)
@@ -1342,6 +1350,7 @@ def _generate_data_key_pair_common(data, action):
"KeyPairSpec": spec,
"PrivateKeyCiphertextBlob": base64.b64encode(ciphertext).decode(),
"PublicKey": base64.b64encode(public_der).decode(),
+ "KeyMaterialId": _key_material_id(rec),
}
return payload, private_der, None
@@ -1381,6 +1390,7 @@ def _generate_data_key_without_plaintext(data):
return json_response({
"KeyId": rec["Arn"],
"CiphertextBlob": base64.b64encode(result).decode(),
+ "KeyMaterialId": _key_material_id(rec),
})
@@ -1651,7 +1661,7 @@ def _revoke_grant(data):
grant_id = data.get("GrantId", "")
grant = _find_grant(rec, grant_id)
if not grant:
- return error_response_json("NotFoundException", f"Grant {grant_id} not found", 400)
+ return error_response_json("NotFoundException", f"Grant does not exist: {grant_id}", 400)
if data.get("DryRun"):
return _dry_run_error("RevokeGrant")
del _grants[grant_id]
@@ -1679,7 +1689,7 @@ def _retire_grant(data):
return error_response_json("NotFoundException", f"Key {key_id} not found", 400)
grant = _find_grant(rec, grant_id)
if not grant:
- return error_response_json("NotFoundException", f"Grant {grant_id} not found", 400)
+ return error_response_json("NotFoundException", f"Grant does not exist: {grant_id}", 400)
if data.get("DryRun"):
return _dry_run_error("RetireGrant")
del _grants[grant["GrantId"]]
@@ -2122,7 +2132,10 @@ def _import_key_material(data):
"_import_token_expires"):
rec.pop(field, None)
logger.info("ImportKeyMaterial: %s", rec["KeyId"])
- return json_response({"KeyId": rec["Arn"]})
+ response = {"KeyId": rec["Arn"]}
+ if "_symmetric_key" in rec:
+ response["KeyMaterialId"] = _key_material_id(rec)
+ return json_response(response)
def _delete_imported_key_material(data):
@@ -2137,13 +2150,16 @@ def _delete_imported_key_material(data):
if rec["KeyState"] == "PendingDeletion":
return error_response_json(
"KMSInvalidStateException", f"{rec['Arn']} is pending deletion.", 400)
+ response = {"KeyId": rec["Arn"]}
+ if "_symmetric_key" in rec:
+ response["KeyMaterialId"] = _key_material_id(rec)
for field in ("_symmetric_key", "_hmac_key", "_private_key", "_public_key_der"):
rec.pop(field, None)
rec.pop("ValidTo", None)
rec["KeyState"] = "PendingImport"
rec["Enabled"] = False
logger.info("DeleteImportedKeyMaterial: %s", rec["KeyId"])
- return json_response({"KeyId": rec["Arn"]})
+ return json_response(response)
def _tag_resource(data):
diff --git a/ministack/services/lambda_svc.py b/ministack/services/lambda_svc.py
index 6da9e6d3..5cb8ca84 100644
--- a/ministack/services/lambda_svc.py
+++ b/ministack/services/lambda_svc.py
@@ -4015,9 +4015,10 @@ def _invoke_rie(container, event: dict, timeout: int) -> dict:
if function_error is not None:
result["error"] = True
result["function_error"] = function_error
- if isinstance(parsed, str):
+ if isinstance(parsed, str) and _RIE_TIMEOUT_TEXT_RE.fullmatch(parsed):
# A bare timeout string: same shape as _rie_terminal_result.
result["body"] = {"errorMessage": parsed, "errorType": "Runtime.ExitError"}
+ result["timeout"] = True
return result
except (urllib.error.URLError, ConnectionRefusedError, OSError) as exc:
if not _rie_failure_is_retryable(exc):
diff --git a/ministack/services/rds.py b/ministack/services/rds.py
index 06c947de..f378919a 100644
--- a/ministack/services/rds.py
+++ b/ministack/services/rds.py
@@ -71,6 +71,7 @@ from ministack.core.responses import (
)
from ministack.services import secretsmanager
from ministack.services.rds_iam_plugin import (
+ configure_iam_auth_broker,
ensure_iam_auth_plugin,
iam_auth_plugin_enabled,
)
@@ -1132,6 +1133,10 @@ def _run_rds_container(docker_client, engine, container_kwargs, tls_names=(), tl
keep working, as they do on AWS without rds.force_ssl.
"""
if engine not in ("postgres", "aurora-postgresql"):
+ if _is_mysql_engine(engine):
+ # The IAM auth plugin calls back to a host-run MiniStack.
+ container_kwargs.setdefault("extra_hosts", {}).setdefault(
+ "host.docker.internal", "host-gateway")
return docker_client.containers.run(**container_kwargs)
# Injecting the certificate needs create -> put_archive -> start, so a
# client that cannot do that keeps the plain launch and serves plaintext.
@@ -2372,16 +2377,6 @@ def _start_rds_container_for_instance(db_id, instance):
# Legacy instances persisted before `_HostPort` was stored fall back to a
# fresh free port from `_next_port()`.
host_port = instance.get("_HostPort") or _next_port()
- # If the stored host port was claimed by something else between
- # restarts (another ministack, another db instance, a user app),
- # docker bind would fail with "port is already allocated". Fall
- # back to a fresh free port and persist it so subsequent restarts
- # converge on a stable mapping again.
- if not _is_host_port_free(host_port):
- logger.info("RDS: persisted host port %d for %s is in use; "
- "allocating fresh free port", host_port, db_id)
- host_port = _next_port()
- instance["_HostPort"] = host_port
image, env_vars, container_port, data_path = _docker_image_for_engine(
engine, engine_version, master_user, master_pass, db_name,
@@ -2424,6 +2419,18 @@ def _start_rds_container_for_instance(db_id, instance):
except Exception:
pass # No existing container with that name — fine
+ # Checked after our own stale container is gone. If the port is still
+ # taken, move and republish it so the endpoint stays reachable.
+ if not _is_host_port_free(host_port):
+ logger.info("RDS: persisted host port %d for %s is in use; "
+ "allocating fresh free port", host_port, db_id)
+ if endpoint.get("Port") == host_port:
+ endpoint["Port"] = _next_port()
+ host_port = endpoint["Port"]
+ else:
+ host_port = _next_port()
+ instance["_HostPort"] = host_port
+
ms_network = _get_ministack_network(docker_client)
container_kwargs = dict(
image=image, detach=True,
@@ -2841,9 +2848,33 @@ def _ensure_mysql_compatibility(
engine_series,
resource_id,
)
+ if plugin_ready:
+ configure_iam_auth_broker(
+ container,
+ "cluster" if engine.startswith("aurora") else "instance",
+ resource_id,
+ _iam_broker_host(),
+ os.environ.get("GATEWAY_PORT") or os.environ.get("EDGE_PORT") or "4566",
+ )
return procedures_ready, plugin_ready
+def _iam_broker_host():
+ """The address a MySQL container reaches this server's IAM broker on."""
+ docker_client = _get_docker()
+ ms_network = _get_ministack_network(docker_client) if docker_client else None
+ if ms_network:
+ try:
+ me = docker_client.containers.get(os.environ.get("HOSTNAME", ""))
+ address = me.attrs["NetworkSettings"]["Networks"][ms_network]["IPAddress"]
+ if address:
+ return address
+ except Exception:
+ pass
+ # Mapped to host-gateway on every MySQL container (_run_rds_container).
+ return "host.docker.internal"
+
+
def _mysql_replication_connection(cluster, *, timeout=None):
args = (cluster, _MYSQL_CONTROL_USER, _MYSQL_CONTROL_PASSWORD)
return (
@@ -5340,6 +5371,10 @@ def _modify_db_instance(p):
"MonitoringInterval": "MonitoringInterval",
"MonitoringRoleArn": "MonitoringRoleArn",
"CopyTagsToSnapshot": "CopyTagsToSnapshot",
+ # Aurora members take it from the DB cluster.
+ "EnableIAMDatabaseAuthentication": (
+ None if instance.get("DBClusterIdentifier") else "IAMDatabaseAuthenticationEnabled"
+ ),
}
pending = {}
@@ -5352,8 +5387,8 @@ def _modify_db_instance(p):
if param_key in ("AllocatedStorage", "BackupRetentionPeriod",
"MonitoringInterval", "Iops", "MaxAllocatedStorage"):
val = int(val)
- elif param_key in ("MultiAZ", "PubliclyAccessible",
- "DeletionProtection", "CopyTagsToSnapshot"):
+ elif param_key in ("MultiAZ", "PubliclyAccessible", "DeletionProtection",
+ "CopyTagsToSnapshot", "EnableIAMDatabaseAuthentication"):
val = val == "true"
if apply_immediately:
@@ -9140,6 +9175,7 @@ def _instance_xml(i):
pending_xml = ""
for pk, pv in i.get("PendingModifiedValues", {}).items():
+ pv = str(pv).lower() if isinstance(pv, bool) else pv
pending_xml += f"<{pk}>{pv}</{pk}>"
iops_xml = ""
diff --git a/ministack/services/rds_iam_plugin.py b/ministack/services/rds_iam_plugin.py
index 8598a569..f5e271a6 100644
--- a/ministack/services/rds_iam_plugin.py
+++ b/ministack/services/rds_iam_plugin.py
@@ -4,6 +4,7 @@
import io
import logging
+import os
import tarfile
from pathlib import Path
@@ -12,6 +13,8 @@ logger = logging.getLogger("rds")
PLUGIN_NAME = "AWSAuthenticationPlugin"
PLUGIN_FILE = "aws_auth_plugin.so"
DEFAULT_ARTIFACT_ROOT = "/opt/ministack/mysql-plugins"
+# Read by aws_auth_plugin.cc on every IAM login.
+CONFIG_PATH = "/etc/ministack/rds-iam.conf"
def iam_auth_plugin_enabled(engine_series=None):
@@ -154,6 +157,38 @@ def ensure_iam_auth_plugin(
pass
+def configure_iam_auth_broker(container, resource_kind, resource_identifier, broker_host, broker_port):
+ """Point the loaded plugin at MiniStack's broker with a fresh capability.
+
+ Re-issuing rotates the capability, so every ready path calls this again.
+ """
+ from ministack.core import rds_iam
+ from ministack.core.responses import get_account_id, get_region
+
+ try:
+ capability = rds_iam.issue_capability(
+ account_id=get_account_id(), region=get_region(),
+ resource_kind=resource_kind, resource_identifier=resource_identifier,
+ )
+ config = f"{broker_host} {broker_port} {capability}\n".encode()
+ stream = io.BytesIO()
+ with tarfile.open(fileobj=stream, mode="w") as archive:
+ directory = tarfile.TarInfo(os.path.dirname(CONFIG_PATH).lstrip("/"))
+ directory.type = tarfile.DIRTYPE
+ directory.mode = 0o755
+ archive.addfile(directory)
+ member = tarfile.TarInfo(CONFIG_PATH.lstrip("/"))
+ member.mode = 0o644
+ member.size = len(config)
+ archive.addfile(member, io.BytesIO(config))
+ if not container.put_archive("/", stream.getvalue()):
+ raise RuntimeError("Docker rejected the plugin config")
+ return True
+ except Exception as e:
+ logger.warning("RDS: failed to configure %s for %s: %s", PLUGIN_NAME, resource_identifier, e)
+ return False
+
+
def get_state() -> dict:
return {}
diff --git a/tests/test_cfn.py b/tests/test_cfn.py
index 3de8739b..abfbf5fb 100644
--- a/tests/test_cfn.py
+++ b/tests/test_cfn.py
@@ -17915,11 +17915,12 @@ def _sqs_stack_attrs(sqs, cfn, stack_name):
({"KmsMasterKeyId": "alias/aws/sqs", "KmsDataKeyReusePeriodSeconds": 600},
{"KmsMasterKeyId": "alias/aws/sqs", "KmsDataKeyReusePeriodSeconds": "600"}),
({"SqsManagedSseEnabled": False}, {"SqsManagedSseEnabled": "false"}),
+ ({}, {"SqsManagedSseEnabled": "true", "MaximumMessageSize": "1048576"}),
({"FifoQueue": True, "ContentBasedDeduplication": True,
"DeduplicationScope": "messageGroup", "FifoThroughputLimit": "perMessageGroupId"},
{"FifoQueue": "true", "ContentBasedDeduplication": "true",
"DeduplicationScope": "messageGroup", "FifoThroughputLimit": "perMessageGroupId"}),
-], ids=["redrive", "kms", "sse", "fifo"])
+], ids=["redrive", "kms", "sse", "defaults", "fifo"])
def test_cfn_sqs_queue_properties_reach_the_queue(cfn, sqs, props, expected):
"""Each queue property shows up in GetQueueAttributes, Json ones as compact JSON."""
stack_name = f"cfn-sqs-attrs-{_uuid_mod.uuid4().hex[:8]}"
@@ -22747,6 +22748,45 @@ def test_cfn_nested_stack_update_deletes_what_the_child_no_longer_has(
s3.delete_bucket(Bucket=bucket)
+def test_cfn_nested_stack_update_keeps_a_retained_resource_it_drops(cfn, s3, ssm):
+ suffix = _uuid_mod.uuid4().hex[:8]
+ bucket = f"cfn-nested-retain-{suffix}"
+ name = f"/cfn-nested-retain-{suffix}/Kept"
+ endpoint = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")
+ keep = {"Type": "AWS::SSM::Parameter", "Properties": {
+ "Name": f"/cfn-nested-retain-{suffix}/Keep", "Type": "String", "Value": "v"}}
+ kept = {"Type": "AWS::SSM::Parameter", "DeletionPolicy": "Retain",
+ "Properties": {"Name": name, "Type": "String", "Value": "v"}}
+
+ def parent(key):
+ return json.dumps({
+ "Resources": {"Nested": {
+ "Type": "AWS::CloudFormation::Stack",
+ "Properties": {"TemplateURL": f"{endpoint}/{bucket}/{key}"}}},
+ "Outputs": {"NestedId": {"Value": {"Ref": "Nested"}}},
+ })
+
+ stack_name = f"cfn-nested-retain-{suffix}"
+ s3.create_bucket(Bucket=bucket)
+ try:
+ s3.put_object(Bucket=bucket, Key="v1.json",
+ Body=json.dumps({"Resources": {"Keep": keep, "Kept": kept}}).encode())
+ s3.put_object(Bucket=bucket, Key="v2.json", Body=json.dumps({"Resources": {"Keep": keep}}).encode())
+ cfn.create_stack(StackName=stack_name, TemplateBody=parent("v1.json"))
+ nested_id = _output(_wait_stack(cfn, stack_name), "NestedId")
+ cfn.update_stack(StackName=stack_name, TemplateBody=parent("v2.json"))
+ assert _wait_stack(cfn, stack_name)["StackStatus"] == "UPDATE_COMPLETE"
+ listed = cfn.describe_stack_resources(StackName=nested_id)["StackResources"]
+ assert [r["LogicalResourceId"] for r in listed] == ["Keep"]
+ ssm.get_parameter(Name=name)
+ finally:
+ _delete_cfn_test_stack(cfn, stack_name)
+ ssm.delete_parameter(Name=name)
+ for key in ("v1.json", "v2.json"):
+ s3.delete_object(Bucket=bucket, Key=key)
+ s3.delete_bucket(Bucket=bucket)
+
+
def test_cfn_apigw_api_key_rename_replaces_it(cfn, apigw_v1):
"""Name is Replacement on AWS::ApiGateway::ApiKey: the renamed key is
created before the old one is removed, so Ref moves to a new key id and
diff --git a/tests/test_kms.py b/tests/test_kms.py
index dd79454b..86e74ff8 100644
--- a/tests/test_kms.py
+++ b/tests/test_kms.py
@@ -336,6 +336,32 @@ def test_kms_generate_data_key_material_id(kms_client, key_spec):
PendingWindowInDays=7,
)
+def test_kms_key_material_id_is_the_same_across_operations(kms_client):
+ key = kms_client.create_key(KeySpec="SYMMETRIC_DEFAULT", KeyUsage="ENCRYPT_DECRYPT")["KeyMetadata"]
+ try:
+ material_id = key["CurrentKeyMaterialId"]
+ assert re.fullmatch(r"[a-f0-9]{64}", material_id)
+ assert kms_client.describe_key(KeyId=key["KeyId"])["KeyMetadata"]["CurrentKeyMaterialId"] == material_id
+ generated = kms_client.generate_data_key(KeyId=key["KeyId"], KeySpec="AES_256")
+ assert generated["KeyMaterialId"] == material_id
+ without = kms_client.generate_data_key_without_plaintext(KeyId=key["KeyId"], KeySpec="AES_256")
+ assert without["KeyMaterialId"] == material_id
+ pair = kms_client.generate_data_key_pair(KeyId=key["KeyId"], KeyPairSpec="ECC_NIST_P256")
+ assert pair["KeyMaterialId"] == material_id
+ decrypted = kms_client.decrypt(CiphertextBlob=generated["CiphertextBlob"])
+ assert decrypted["KeyMaterialId"] == material_id
+ finally:
+ kms_client.schedule_key_deletion(KeyId=key["KeyId"], PendingWindowInDays=7)
+
+
+def test_kms_key_material_id_absent_for_asymmetric_keys(kms_client):
+ key = kms_client.create_key(KeySpec="RSA_2048", KeyUsage="SIGN_VERIFY")["KeyMetadata"]
+ try:
+ assert "CurrentKeyMaterialId" not in key
+ finally:
+ kms_client.schedule_key_deletion(KeyId=key["KeyId"], PendingWindowInDays=7)
+
+
def test_kms_generate_data_key_material_id_survives_state_roundtrip():
from ministack.services import kms as _kms
diff --git a/tests/test_lambda.py b/tests/test_lambda.py
index ab7cfba7..196850be 100644
--- a/tests/test_lambda.py
+++ b/tests/test_lambda.py
@@ -4875,7 +4875,7 @@ def test_lambda_cross_account_layer_under_auth_names_the_calling_user(monkeypatc
from ministack.services import iam as iam_svc
monkeypatch.setattr(app_mod, "AUTH", True)
- key, user = "AKIALAYERCONSUMER001", "layer-consumer" # sadscan:disable np.aws.1 - synthetic fixture key
+ key, user = "AKIALAYERCONSUMER001", "layer-consumer"
user_arn = f"arn:aws:iam::{_CALLER_ACCOUNT}:user/{user}"
seeded = [
(iam_svc._users, user, {"UserName": user, "Arn": user_arn, "UserId": "AIDALAYER", "AttachedPolicies": []}),
@@ -14119,6 +14119,7 @@ def test_invoke_rie_reports_a_bare_string_timeout_as_a_function_error():
"errorMessage": "Task timed out after 300.00 seconds",
"errorType": "Runtime.ExitError",
}
+ assert result["timeout"] is True
def test_classify_function_error_bare_timeout_string_is_unhandled():
diff --git a/tests/test_mysqlproxy.py b/tests/test_mysqlproxy.py
deleted file mode 100644
index dbce8a66..00000000
--- a/tests/test_mysqlproxy.py
+++ /dev/null
@@ -1,64 +0,0 @@
-"""Offline fail-closed checks for the adapter's explicitly limited handshake."""
-import socket
-
-import pytest
-
-from ministack.core.mysqlproxy import client_identity, read
-
-
-def handshake(plugin=b"mysql_native_password"):
- flags = (1 << 9) | (1 << 15) | (1 << 19)
- return (flags.to_bytes(4, "little") + b"\0" * 4 + b"\x21" + b"\0" * 23
- + b"iam_user\0\0" + plugin + b"\0")
-
-
[email protected]("plugin", [b"mysql_native_password", b"caching_sha2_password"])
-def test_supported_client_identity(plugin):
- assert client_identity(handshake(plugin)) == "iam_user"
-
-
[email protected]("payload", [
- b"", b"\0" * 32, handshake()[:-1], handshake(b"ministack_iam_gate_v1"),
- handshake(b"mysql_clear_password"), handshake(b"unknown"),
-])
-def test_malformed_or_spoofed_identity(payload):
- with pytest.raises((ValueError, IndexError)):
- client_identity(payload)
-
-
[email protected]("flag", [1 << 5, 1 << 26, 1 << 7])
-def test_unsupported_capabilities(flag):
- payload = handshake()
- payload = (int.from_bytes(payload[:4], "little") | flag).to_bytes(4, "little") + payload[4:]
- with pytest.raises(ValueError):
- client_identity(payload)
-
-
-def test_oversized_frame_rejected_before_reading_body():
- left, right = socket.socketpair()
- with left, right:
- left.sendall((1024 * 1024 + 1).to_bytes(3, "little") + b"\0")
- with pytest.raises(ValueError):
- read(right)
-
-
[email protected]("name", [b"'iam_user'", "é".encode(), b"x" * 33, b"a b", b"a\n", b""])
-def test_ambiguous_username_rejected(name):
- payload = handshake().replace(b"iam_user", name)
- with pytest.raises(ValueError):
- client_identity(payload)
-
-
[email protected]("charset", [0, 1, 28, 35, 54])
-def test_unsupported_charset_rejected(charset):
- payload = bytearray(handshake())
- payload[8] = charset
- with pytest.raises(ValueError):
- client_identity(payload)
-
-
[email protected]("charset", [8, 33, 45, 46, 255])
-def test_supported_charset_ascii_identity(charset):
- payload = bytearray(handshake())
- payload[8] = charset
- assert client_identity(payload) == "iam_user"
diff --git a/tests/test_rds.py b/tests/test_rds.py
index a8a206ac..5b4fb2d1 100644
--- a/tests/test_rds.py
+++ b/tests/test_rds.py
@@ -336,6 +336,31 @@ def test_rds_modify_instance_v2(rds):
assert inst["DBInstanceClass"] == "db.t3.small"
assert inst["AllocatedStorage"] == 50
+
+def test_rds_modify_instance_iam_database_authentication(rds):
+ rds.create_db_instance(
+ DBInstanceIdentifier="rds-mod-iam",
+ DBInstanceClass="db.t3.micro",
+ Engine="postgres",
+ MasterUsername="admin",
+ MasterUserPassword="pass",
+ AllocatedStorage=20,
+ )
+ inst = rds.modify_db_instance(
+ DBInstanceIdentifier="rds-mod-iam",
+ EnableIAMDatabaseAuthentication=True,
+ )["DBInstance"]
+ assert inst["IAMDatabaseAuthenticationEnabled"] is False
+ assert inst["PendingModifiedValues"] == {"IAMDatabaseAuthenticationEnabled": True}
+ rds.modify_db_instance(
+ DBInstanceIdentifier="rds-mod-iam",
+ EnableIAMDatabaseAuthentication=True,
+ ApplyImmediately=True,
+ )
+ inst = rds.describe_db_instances(DBInstanceIdentifier="rds-mod-iam")["DBInstances"][0]
+ assert inst["IAMDatabaseAuthenticationEnabled"] is True
+
+
def test_rds_create_instance_honors_preferred_maintenance_window(rds):
# Regression: CreateDBInstance previously hardcoded
# PreferredMaintenanceWindow to "sun:05:00-sun:06:00", silently
@@ -8155,9 +8180,9 @@ def test_aurora_user_and_grant_are_visible_through_reader(rds):
"8",
],
)
-def test_aurora_mysql_iam_plugin_ddl_and_reject_all(rds, engine_version):
+def test_aurora_mysql_iam_plugin_ddl_and_login(rds, engine_version):
with _live_cluster(rds, engine_version=engine_version) as (
- _cid, _wid, _rid, writer, _reader, _cluster,
+ cluster_id, _wid, _rid, writer, _reader, _cluster,
):
user = f"iam_{uuid.uuid4().hex[:8]}"
with _aurora_connect(writer["Endpoint"]) as conn:
@@ -8178,7 +8203,7 @@ def test_aurora_mysql_iam_plugin_ddl_and_reject_all(rds, engine_version):
)
cursor.execute(
"SELECT plugin, authentication_string FROM mysql.user "
- "WHERE User = %s AND Host = '%'",
+ "WHERE User = %s AND Host = '%%'",
(user,),
)
assert cursor.fetchone() == ("AWSAuthenticationPlugin", "RDS")
@@ -8188,14 +8213,32 @@ def test_aurora_mysql_iam_plugin_ddl_and_reject_all(rds, engine_version):
f"ALTER USER `{user}`@'%' WITH MAX_USER_CONNECTIONS 9"
)
+ import ssl
+
import pymysql
+ host, port = _host_dialable(writer["Endpoint"])
+ tls = ssl.create_default_context()
+ tls.check_hostname = False
+ tls.verify_mode = ssl.CERT_NONE
+ token = rds.generate_db_auth_token(
+ DBHostname=writer["Endpoint"]["Address"],
+ Port=writer["Endpoint"]["Port"],
+ DBUsername=user,
+ )
+
+ def iam_login():
+ return pymysql.connect(host=host, port=port, user=user, password=token,
+ ssl=tls, connect_timeout=5)
+
+ # IAM database authentication is off on the cluster.
with pytest.raises(pymysql.err.OperationalError):
- _aurora_connect(
- writer["Endpoint"],
- user=user,
- password="not-a-token",
- )
+ iam_login()
+ rds.modify_db_cluster(DBClusterIdentifier=cluster_id,
+ EnableIAMDatabaseAuthentication=True, ApplyImmediately=True)
+ with iam_login() as conn, conn.cursor() as cursor:
+ cursor.execute("SELECT CURRENT_USER()")
+ assert cursor.fetchone() == (f"{user}@%",)
with _aurora_connect(writer["Endpoint"]) as conn:
with conn.cursor() as cursor:
@@ -8229,7 +8272,7 @@ def test_aurora_mysql_rds_compatibility_procedures(rds):
with _aurora_connect(writer["Endpoint"]) as admin:
with admin.cursor() as cursor:
cursor.execute(
- f"CREATE USER `{user}`@'%' IDENTIFIED BY %s",
+ f"CREATE USER `{user}`@'%%' IDENTIFIED BY %s",
(user_password,),
)
for procedure_name in procedure_names:
@@ -8355,10 +8398,29 @@ def test_aurora_mysql_iam_plugin_survives_compute_replacement(rds):
)
cursor.execute(
"SELECT plugin FROM mysql.user "
- "WHERE User = %s AND Host = '%'",
+ "WHERE User = %s AND Host = '%%'",
(user,),
)
assert cursor.fetchone() == ("AWSAuthenticationPlugin",)
+
+ # The replacement container gets a fresh broker capability.
+ import pymysql
+
+ rds.modify_db_cluster(DBClusterIdentifier=cluster_id,
+ EnableIAMDatabaseAuthentication=True, ApplyImmediately=True)
+ host, port = _host_dialable(restarted_writer["Endpoint"])
+ token = rds.generate_db_auth_token(
+ DBHostname=restarted_writer["Endpoint"]["Address"],
+ Port=restarted_writer["Endpoint"]["Port"],
+ DBUsername=user,
+ )
+ with pymysql.connect(host=host, port=port, user=user, password=token,
+ connect_timeout=5) as conn, conn.cursor() as cursor:
+ cursor.execute("SELECT CURRENT_USER()")
+ assert cursor.fetchone() == (f"{user}@%",)
+
+ with _aurora_connect(restarted_writer["Endpoint"]) as conn:
+ with conn.cursor() as cursor:
cursor.execute(f"DROP USER `{user}`@'%'")
@@ -12613,7 +12675,7 @@ def test_aurora_mysql_global_replication_replays_and_streams_rows():
with conn.cursor() as cursor:
cursor.execute(
"SELECT plugin FROM mysql.user "
- "WHERE User = %s AND Host = '%'",
+ "WHERE User = %s AND Host = '%%'",
(iam_user,),
)
assert cursor.fetchone() == ("AWSAuthenticationPlugin",)
@@ -12756,6 +12818,93 @@ def test_aurora_mysql_global_replication_replays_and_streams_rows():
_delete_global_cluster(east, global_id)
+ not os.environ.get("DOCKER_NETWORK"),
+ reason="DOCKER_NETWORK not set -- live Aurora global replication",
+)
+def test_aurora_mysql_global_secondary_iam_login():
+ import pymysql
+
+ east = _regional_rds("us-east-1")
+ west = _regional_rds("us-west-2")
+ suffix = uuid.uuid4().hex[:10]
+ global_id = f"global-iam-{suffix}"
+ primary_id = f"global-iam-primary-{suffix}"
+ primary_instance_id = f"{primary_id}-writer"
+ secondary_id = f"global-iam-secondary-{suffix}"
+ secondary_instance_id = f"{secondary_id}-reader"
+ engine_version = "8.0.mysql_aurora.3.10.3"
+ user = f"iam_{suffix}"
+ primary_arn = secondary_arn = None
+ try:
+ primary_arn = east.create_db_cluster(
+ DBClusterIdentifier=primary_id, Engine="aurora-mysql", EngineVersion=engine_version,
+ MasterUsername="admin", MasterUserPassword=PASSWORD, DatabaseName=DATABASE,
+ EnableIAMDatabaseAuthentication=True,
+ )["DBCluster"]["DBClusterArn"]
+ east.create_db_instance(
+ DBInstanceIdentifier=primary_instance_id, DBClusterIdentifier=primary_id,
+ DBInstanceClass="db.r6g.large", Engine="aurora-mysql",
+ )
+ primary_instance = _wait_for_instance(east, primary_instance_id)
+ east.create_global_cluster(GlobalClusterIdentifier=global_id, SourceDBClusterIdentifier=primary_arn)
+ secondary_arn = west.create_db_cluster(
+ DBClusterIdentifier=secondary_id, Engine="aurora-mysql", EngineVersion=engine_version,
+ GlobalClusterIdentifier=global_id, MasterUsername="admin", MasterUserPassword=PASSWORD,
+ DatabaseName=DATABASE, EnableIAMDatabaseAuthentication=True,
+ )["DBCluster"]["DBClusterArn"]
+ west.create_db_instance(
+ DBInstanceIdentifier=secondary_instance_id, DBClusterIdentifier=secondary_id,
+ DBInstanceClass="db.r6g.large", Engine="aurora-mysql",
+ )
+ secondary_instance = _wait_for_instance(west, secondary_instance_id)
+ with _aurora_connect(primary_instance["Endpoint"]) as conn, conn.cursor() as cursor:
+ cursor.execute(
+ "SELECT COUNT(*) FROM INFORMATION_SCHEMA.PLUGINS "
+ "WHERE PLUGIN_NAME = 'AWSAuthenticationPlugin'"
+ )
+ if cursor.fetchone()[0] == 0:
+ pytest.skip("matching AWSAuthenticationPlugin artifact is absent")
+ # Created on the primary; replication carries it to the secondary.
+ cursor.execute(f"CREATE USER `{user}`@'%' IDENTIFIED WITH AWSAuthenticationPlugin AS 'RDS'")
+
+ endpoint = secondary_instance["Endpoint"]
+ host, port = _host_dialable(endpoint)
+ token = west.generate_db_auth_token(
+ DBHostname=endpoint["Address"], Port=endpoint["Port"], DBUsername=user,
+ )
+
+ def iam_login():
+ return pymysql.connect(host=host, port=port, user=user, password=token, connect_timeout=5)
+
+ deadline = time.monotonic() + 60
+ while True:
+ try:
+ with iam_login() as conn, conn.cursor() as cursor:
+ cursor.execute("SELECT CURRENT_USER()")
+ assert cursor.fetchone() == (f"{user}@%",)
+ break
+ except pymysql.MySQLError:
+ if time.monotonic() > deadline:
+ raise
+ time.sleep(1)
+ west.modify_db_cluster(DBClusterIdentifier=secondary_id,
+ EnableIAMDatabaseAuthentication=False, ApplyImmediately=True)
+ with pytest.raises(pymysql.err.OperationalError):
+ iam_login()
+ finally:
+ if secondary_arn:
+ _remove_global_member(east, global_id, secondary_arn)
+ if primary_arn:
+ _remove_global_member(east, global_id, primary_arn)
+ _delete_instance(west, secondary_instance_id)
+ _delete_instance(east, primary_instance_id)
+ _delete_cluster(west, secondary_id)
+ _delete_cluster(east, primary_id)
+ _delete_global_cluster(east, global_id)
+
+
# ---------------------------------------------------------------------------
# ManageMasterUserPassword — RDS-managed master user secrets
# ---------------------------------------------------------------------------
diff --git a/tests/test_rds_iam.py b/tests/test_rds_iam.py
index c6d12902..7f0cc70a 100644
--- a/tests/test_rds_iam.py
+++ b/tests/test_rds_iam.py
@@ -693,6 +693,26 @@ def test_aurora_parent_flag_and_replacement(ep_state, monkeypatch):
assert request(cap)[0] == 403
[email protected]("host,allowed", [
+ ("writer.example.com", True), ("reader.example.com", True), (AUTHZ_HOST, True), ("other.example.com", False),
+])
+def test_cluster_capability_accepts_every_endpoint_of_its_container(ep_state, monkeypatch, host, allowed):
+ monkeypatch.setattr(app_module, "AUTH", True)
+ ep_state.update(Engine="aurora-mysql", DBClusterIdentifier="cluster")
+ rds._clusters.set_scoped(ACCOUNT, REGION, "cluster", {
+ "Engine": "aurora-mysql", "DbClusterResourceId": "cluster-TEST", "IAMDatabaseAuthenticationEnabled": True,
+ "Endpoint": "writer.example.com", "ReaderEndpoint": "reader.example.com", "Port": 3306,
+ "DBClusterMembers": [{"DBInstanceIdentifier": "database"}],
+ })
+ iam._user_inline_policies.set_scoped(ACCOUNT, None, "alice", {"connect": _authz_policy(
+ resource=f"arn:aws:rds-db:{REGION}:{ACCOUNT}:dbuser:cluster-TEST/{AUTHZ_USER}")})
+ cap = rds_iam.issue_capability(account_id=ACCOUNT, region=REGION, resource_kind="cluster",
+ resource_identifier="cluster")
+ assert request(cap, payload={"username": AUTHZ_USER, "token": _authz_token(host=host)}) == (
+ 200 if allowed else 403, {"allowed": allowed},
+ )
+
+
@pytest.mark.parametrize("auth", [False, True])
def test_policy_denial_only_enforced_when_auth_enabled(ep_state, monkeypatch, auth):
monkeypatch.setattr(app_module, "AUTH", auth)
diff --git a/tests/test_rds_iam_plugin.py b/tests/test_rds_iam_plugin.py
index ecf97086..a9ce7889 100644
--- a/tests/test_rds_iam_plugin.py
+++ b/tests/test_rds_iam_plugin.py
@@ -329,3 +329,34 @@ def test_mariadb_compatibility_skips_plugin_and_roles(monkeypatch):
engine="mariadb",
) == (True, False)
assert calls == [("procedures", "db-1", "mariadb", None)]
+
+
+def test_rds_iam_plugin_broker_config_carries_a_live_capability(monkeypatch):
+ from ministack.core import rds_iam
+
+ issued = {}
+
+ def issue(**kwargs):
+ issued.update(kwargs)
+ return "c" * 64
+
+ monkeypatch.setattr(rds_iam, "issue_capability", issue)
+ container = FakeContainer()
+ assert plugin.configure_iam_auth_broker(container, "cluster", "aurora-1", "172.18.0.2", "4566")
+ assert issued["resource_kind"] == "cluster" and issued["resource_identifier"] == "aurora-1"
+ path, data = container.archives[0]
+ assert path == "/"
+ with tarfile.open(fileobj=io.BytesIO(data)) as archive:
+ member = archive.getmember(plugin.CONFIG_PATH.lstrip("/"))
+ assert member.mode == 0o644
+ assert archive.extractfile(member).read() == b"172.18.0.2 4566 " + b"c" * 64 + b"\n"
+
+
+def test_rds_iam_plugin_broker_config_failure_is_swallowed(monkeypatch, caplog):
+ from ministack.core import rds_iam
+
+ monkeypatch.setattr(rds_iam, "issue_capability", lambda **kwargs: "c" * 64)
+ with caplog.at_level(logging.WARNING, logger="rds"):
+ assert not plugin.configure_iam_auth_broker(
+ FakeContainer(put_result=False), "instance", "db-1", "host.docker.internal", "4566")
+ assert "failed to configure AWSAuthenticationPlugin for db-1" in caplog.text