Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(prerelease): reshape dsql to match ministack behaviour plus cleanup (#1351)

Nahuel Nuceradfd972a07b0b
diff --git a/CHANGELOG.md b/CHANGELOG.md
index cace2eee..1b3c090e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -12,6 +12,9 @@ Versioning follows [Semantic Versioning](https://semver.org/).
- **CloudFormation — resources unchanged by a stack update are left alone** — every previously-provisioned resource was reprocessed on every stack update regardless of whether its properties changed, and any resource type with no dedicated update handler fell back to calling create again — a fallback whose own name-generation regenerated a fresh random suffix on every call, and whose caller passed the wrong id into it, on top of that. An auto-named resource's "idempotent" create (the type framework's own documented expectation) silently produced a brand new, empty resource under a new identity on every single update, orphaning the real one — and anything referencing it via `Ref`/`Fn::GetAtt` picked up that new, wrong identity the moment it was reprocessed later in the same update, since real CloudFormation does propagate a changed reference. Found via a DynamoDB table silently duplicating on an unrelated update to a real multi-stack deploy. A genuinely unchanged resource is no longer touched at all, matching real CloudFormation; auto-generated names are now a deterministic hash of (stack name, logical id) rather than random, so the remaining case — a resource's properties *did* change but its type still has no update handler — reproduces the same name it started with. Also fixes the same symptom for `AWS::Events::EventBus` (previously failed a stack update outright with "already exists" once created) and any other resource type with no update handler and no name-collision guard of its own. Contributed by @ryan-bennett.
- **CloudFormation — `AWS::ApiGatewayV2::Authorizer` survives a property update** — the type had no update handler, so changing e.g. `JwtConfiguration.Audience` (trusting an additional app client) fell back to create — whose `authorizerId` is a random id on every call, with no stable name to derive from — minting a second, orphaned authorizer while every `Route`'s own (unchanged) `AuthorizerId` kept pointing at the original, now-stale one. A real update handler now mutates the existing authorizer record in place, matching API Gateway's own `UpdateAuthorizer`. Contributed by @ryan-bennett.
- **CloudFormation — `AWS::SSM::Parameter::Value<...>` template parameters now resolve against SSM Parameter Store** — a template parameter typed `AWS::SSM::Parameter::Value<String>` (or `<List<String>>` / `<AWS::EC2::Image::Id>`) takes an SSM parameter *name* as its Default/provided value; real CloudFormation resolves that name against Parameter Store before `Ref` ever sees it — the mechanism behind, e.g., CDK's `StringParameter.valueForStringParameter`. MiniStack treated these like any other AWS-specific parameter type and passed the literal name straight through, so `Ref` returned the parameter's name instead of its stored value, and any resource property built from it (or a nested stack composing two independently-deployed stacks through a shared parameter) silently received the wrong string. Resolution now looks the name up in the SSM store at parameter-resolution time; a name that doesn't exist fails `CreateStack`/`UpdateStack` synchronously with a `ValidationError`, matching AWS. Contributed by @ryan-bennett.
+- **S3 — versioned objects retain their custom metadata** — in a versioning-enabled bucket the per-version record stored only the wire basics (etag/size/storage class), dropping the object's `x-amz-meta-*` user metadata, preserved headers, and content-encoding; on top of that a `GetObject?versionId=` hand-built a five-header response that bypassed the only `x-amz-meta-*` emitter. A `PutObject` with `Metadata` followed by a `GetObject`/`HeadObject` of that version therefore came back with the metadata missing. Every version now stores its own metadata (across put, POST-form, copy, and multipart-complete), and versioned reads route through the shared header builder, so each version returns the metadata it was written with — matching S3, where metadata is per-version. Reported by @Kaphaalor.
+- **API Gateway (REST) — custom Lambda authorizers are invoked and enforced on the request path** — a method with `authorizationType: CUSTOM` stored its authorizer but never called it: requests reached the integration unauthenticated and `requestContext.authorizer` was never populated. `TOKEN` and `REQUEST` authorizers are now invoked on the data path with the AWS event shape (`type`/`authorizationToken`/`methodArn`, or the REQUEST parameter shape); a missing identity source returns `401` without invoking, the returned IAM policy is evaluated against the method ARN (explicit `Deny` and no-matching-`Allow` both `403`, an authorizer raising `Unauthorized` `401`), `authorizerResultTtlInSeconds` caches the decision, and on allow the returned `context` (values stringified) plus `principalId` are injected into `requestContext.authorizer`. `AWS_IAM` methods reject a request with no `Authorization` header (`403`); the SigV4 signature itself is not verified. Reported by @iot-rocket.
+- **Route 53 — a change is born `PENDING` and flips to `INSYNC`** — every `ChangeResourceRecordSets` / `CreateHostedZone` change was created `INSYNC`, so `GetChange` never returned `PENDING` and any waiter or poll loop (e.g. the `ResourceRecordSetsChanged` waiter) completed on its first read without exercising its retry path. Changes are now submitted `PENDING` and flip to `INSYNC` on the first `GetChange` read thereafter, reproducing the observable `PENDING`→`INSYNC` transition without a background timer. Reported by @jayjanssen.
## [1.4.15] — 2026-08-10
diff --git a/Dockerfile b/Dockerfile
index a4ae4172..f4b20618 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -62,7 +62,7 @@ ENV MINISTACK_VERSION=${MINISTACK_VERSION} \
RDS_PERSIST=0 \
DSQL_BASE_PORT=25432 \
DSQL_PERSIST=0 \
- DSQL_CONTAINERS=0 \
+ DSQL_STRICT=0 \
ELASTICACHE_BASE_PORT=16379 \
LAMBDA_EXECUTOR=local \
USE_SSL=0 \
diff --git a/Dockerfile.full b/Dockerfile.full
index ae9a3182..c9d21074 100644
--- a/Dockerfile.full
+++ b/Dockerfile.full
@@ -79,7 +79,7 @@ ENV MINISTACK_VERSION=${MINISTACK_VERSION} \
RDS_PERSIST=0 \
DSQL_BASE_PORT=25432 \
DSQL_PERSIST=0 \
- DSQL_CONTAINERS=0 \
+ DSQL_STRICT=0 \
ELASTICACHE_BASE_PORT=16379 \
LAMBDA_EXECUTOR=local \
USE_SSL=0 \
diff --git a/README.md b/README.md
index 79ed5bed..8f5d1d93 100644
--- a/README.md
+++ b/README.md
@@ -642,9 +642,9 @@ only `DeleteDBCluster` removes the container and its storage.
## Aurora DSQL
-Each `CreateCluster` returns an `ACTIVE` cluster with an endpoint. Set `DSQL_CONTAINERS=1` to additionally spin up a real `postgres:16-alpine` container per cluster (capped by `DSQL_MAX_CONTAINERS`, default 30) behind an in-process Postgres wire-protocol proxy.
+Each `CreateCluster` returns an `ACTIVE` cluster with an endpoint. Set `DSQL_STRICT=1` to additionally spin up a real `postgres:16-alpine` container per cluster (capped by a fixed port window off `DSQL_BASE_PORT`, default 30 ports) behind an in-process Postgres wire-protocol proxy.
-**The proxy fails like real DSQL**: SQL that DSQL rejects fails here too (`serial` columns, foreign keys, `CREATE EXTENSION`, `TRUNCATE`, temp tables, `ALTER COLUMN TYPE`, `SET NOT NULL`, materialized views, partitioned tables, non-btree/partial/expression indexes, ...), plain `CREATE INDEX` on a non-empty table is refused (`use CREATE INDEX ASYNC instead`), `CREATE INDEX ASYNC` and `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` return a `job_id`, `sys.jobs` / `sys.wait_for_job` are emulated, CHECK constraints via ALTER TABLE require `NOT VALID`, `ALTER TABLE ... DROP COLUMN` works (several columns in one statement included) except on a primary key column, identity columns are bigint-only (CACHE 1 or ≥ 65536), index limits are enforced (8 columns/index, 24/table), transactions allow only one DDL statement with no DDL+DML mixing, `SELECT ... FOR UPDATE` requires a single table with equality predicates on the key, a transaction that outlives another session's DDL fails once with `40001` (OC001) and succeeds on retry, and the transaction limits (3,000 rows / 10 MiB / 5 min) are enforced. `DSQL_STRICT=0` disables validation.
+**The proxy fails like real DSQL**: SQL that DSQL rejects fails here too (`serial` columns, foreign keys, `CREATE EXTENSION`, `TRUNCATE`, temp tables, `ALTER COLUMN TYPE`, `SET NOT NULL`, materialized views, partitioned tables, non-btree/partial/expression indexes, ...), plain `CREATE INDEX` on a non-empty table is refused (`use CREATE INDEX ASYNC instead`), `CREATE INDEX ASYNC` and `ALTER TABLE ASYNC ... VALIDATE CONSTRAINT` return a `job_id`, `sys.jobs` / `sys.wait_for_job` are emulated, CHECK constraints via ALTER TABLE require `NOT VALID`, `ALTER TABLE ... DROP COLUMN` works (several columns in one statement included) except on a primary key column, identity columns are bigint-only (CACHE 1 or ≥ 65536), index limits are enforced (8 columns/index, 24/table), transactions allow only one DDL statement with no DDL+DML mixing, `SELECT ... FOR UPDATE` requires a single table with equality predicates on the key, a transaction that outlives another session's DDL fails once with `40001` (OC001) and succeeds on retry, and the transaction limits (3,000 rows / 10 MiB / 5 min) are enforced.
`GetCluster` returns an `endpoint` of the form `localhost:<port>` (ports allocated from `DSQL_BASE_PORT`, default `25432`). Any user/password is accepted — IAM auth tokens from `generate_db_connect_auth_token` work but aren't validated. Connect with e.g. `psql "postgresql://admin:anything@localhost:25432/postgres"` (no TLS — use `sslmode=disable`/`prefer`; with psycopg2 use `autocommit=True`).
@@ -802,11 +802,9 @@ end-to-end without any client config.
| `RDS_PERSIST` | `0` | Set `1` to use Docker named volumes for RDS containers instead of tmpfs. Storage grows dynamically with no fixed cap |
| `MINISTACK_RDS_PUBLIC_ENDPOINT` | `0` | Set `1` when MiniStack itself runs in a Docker container but RDS clients connect from outside that network (remote MiniStack host, host-side clients, CI runners). `DescribeDBInstances` then returns `{MINISTACK_HOST, host_port}` — the externally-reachable host-published port — instead of the container-internal address. Set `MINISTACK_HOST` to the host clients will use |
| `DSQL_BASE_PORT` | `25432` | Starting host port for Aurora DSQL cluster proxies (one port per cluster) |
-| `DSQL_CONTAINERS` | `0` | Set `1` to run a real Postgres container per DSQL cluster (requires Docker; capped by `DSQL_MAX_CONTAINERS` — further clusters are metadata-only stubs). Default `0` is control-plane-only |
-| `DSQL_MAX_CONTAINERS` | `30` | Maximum number of real DSQL backend containers |
+| `DSQL_STRICT` | `0` | Set `1` to run a real Postgres container per DSQL cluster (requires Docker; capped by a fixed port window off `DSQL_BASE_PORT`, default 30 — further clusters are metadata-only stubs). Default `0` is control-plane-only |
| `DSQL_PG_IMAGE` | `postgres:16-alpine` | Image used for per-cluster DSQL backend containers |
| `DSQL_PERSIST` | `0` | Set `1` to use Docker named volumes for DSQL backend containers instead of tmpfs |
-| `DSQL_STRICT` | `1` | Set `0` to make the DSQL wire proxy a transparent passthrough (no SQL-subset validation) |
| `DOCKER_NETWORK` | _(unset)_ | Docker network for all container-backed services (RDS, DSQL, EKS, ElastiCache, Lambda). Set to your Docker Compose network name so containers can reach each other. Replaces `LAMBDA_DOCKER_NETWORK` |
| `ELASTICACHE_BASE_PORT` | `16379` | Starting host port for ElastiCache containers |
| `ELASTICACHE_CLUSTER_MODE_REAL` | `0` | Set `1` (requires `DOCKER_NETWORK`) to provision real Redis Cluster replication groups: `NumNodeGroups × (1+ReplicasPerNodeGroup)` cluster-enabled nodes wired with `redis-cli --cluster create`, serving real `CLUSTER SLOTS` / `MOVED` redirects for cluster-aware clients |
diff --git a/docker-compose.yml b/docker-compose.yml
index 45fa005d..a1879a8b 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -14,7 +14,7 @@ services:
- REDIS_PORT=6379
- RDS_BASE_PORT=15432
# - DSQL_BASE_PORT=25432 # first host port for DSQL cluster proxies
- # - DSQL_CONTAINERS=1 # run a real Postgres container per DSQL cluster (default: control-plane-only)
+ # - DSQL_STRICT=1 # run a real Postgres container per DSQL cluster (default: control-plane-only)
# - DOCKER_NETWORK=ministack_default # Docker network for RDS/DSQL/EKS/ElastiCache/Lambda containers
- ELASTICACHE_BASE_PORT=16379
volumes:
diff --git a/ministack/core/pgproxy.py b/ministack/core/pgproxy.py
index c4526f30..b4bbbf26 100644
--- a/ministack/core/pgproxy.py
+++ b/ministack/core/pgproxy.py
@@ -5,9 +5,6 @@ One proxy per DSQL cluster: clients connect to the proxy on the cluster's
localhost port; the proxy forwards frames to a real Postgres backend after
enforcing DSQL's PostgreSQL-compatibility subset.
-Validator behavior is gated by ``DSQL_STRICT`` (default "1"); "0" turns the
-proxy into a transparent passthrough.
-
Both wire protocols are validated: the simple ``Q`` protocol (psql, psycopg2)
and the extended ``Parse``/``Bind``/``Execute`` protocol (pgjdbc, pgx, asyncpg,
psycopg3, and every ORM that uses prepared statements).
@@ -33,7 +30,6 @@ Documented limitations:
import asyncio
import logging
-import os
import re
import secrets
import string
@@ -43,7 +39,6 @@ from datetime import datetime, timezone
logger = logging.getLogger("pgproxy")
-STRICT = os.environ.get("DSQL_STRICT", "1").strip().lower() not in ("0", "false", "no", "off")
# ---------------------------------------------------------------------------
# Wire-protocol constants and frame helpers
@@ -995,8 +990,6 @@ def validate(sql, txn_state=None):
Read-only with respect to ``txn_state`` — the caller applies state
changes (``TxnState.apply``) once the statement is actually forwarded.
"""
- if not STRICT:
- return None
s = strip_leading_comments(sql).strip()
if not s:
return None
@@ -1535,7 +1528,7 @@ async def _handle_query(conn, sql, b_writer, c_writer):
txn = conn.txn
stmts = split_statements(sql)
- if len(stmts) == 1 and STRICT:
+ if len(stmts) == 1:
s = strip_leading_comments(stmts[0])
# A statement rejected earlier in this transaction poisoned the block.
@@ -1595,9 +1588,9 @@ async def _handle_query(conn, sql, b_writer, c_writer):
await b_writer.drain()
return
- # Multi-statement batch (implicit transaction) or non-strict mode:
- # validate every statement up front, then forward the batch verbatim.
- if STRICT and stmts:
+ # Multi-statement batch (implicit transaction): validate every statement
+ # up front, then forward the batch verbatim.
+ if stmts:
# An aborted block only accepts an explicit rollback; forwarding a
# batch that opened with COMMIT would commit work the client was told
# had failed.
@@ -1704,7 +1697,7 @@ async def _ext_parse(conn, payload, b_writer, c_writer):
return
s = strip_leading_comments(sql).strip()
entry = {"sql": s, "synth": None}
- if not STRICT or not s:
+ if not s:
conn.ext_stmts[name] = entry
await _ext_forward(conn, b_writer, b"P", payload)
return
diff --git a/ministack/services/apigateway_v1.py b/ministack/services/apigateway_v1.py
index 81b13bbf..2d72b631 100644
--- a/ministack/services/apigateway_v1.py
+++ b/ministack/services/apigateway_v1.py
@@ -125,6 +125,10 @@ _resources = AccountRegionScopedDict() # rest_api_id -> {resource_id -
_stages_v1 = AccountRegionScopedDict() # rest_api_id -> {stage_name -> Stage}
_deployments_v1 = AccountRegionScopedDict() # rest_api_id -> {deployment_id -> Deployment}
_authorizers_v1 = AccountRegionScopedDict() # rest_api_id -> {authorizer_id -> Authorizer}
+# Data-plane cache of evaluated Lambda-authorizer results, keyed by
+# (rest_api_id, authorizer_id, cache_key). Value: (expiry_epoch, decision, context).
+# Ephemeral (not persisted); honors authorizerResultTtlInSeconds.
+_authorizer_cache = {}
_models = AccountRegionScopedDict() # rest_api_id -> {model_id -> Model}
_api_keys = AccountRegionScopedDict() # key_id -> ApiKey
_usage_plans = AccountRegionScopedDict() # plan_id -> UsagePlan
@@ -851,6 +855,7 @@ def reset():
_stages_v1.clear()
_deployments_v1.clear()
_authorizers_v1.clear()
+ _authorizer_cache.clear()
_models.clear()
_api_keys.clear()
_usage_plans.clear()
@@ -1192,6 +1197,283 @@ async def handle_execute(api_id, stage_name, method, path, headers, body, query_
_request_region.reset(region_token)
+# ---- Data plane: Lambda authorizers (custom authorizers) ----
+
+
+def _header_ci(headers, name):
+ """Case-insensitive single header lookup returning a str (or '')."""
+ target = name.lower()
+ for k, v in (headers or {}).items():
+ if k.lower() == target:
+ return v if isinstance(v, str) else (v[-1] if v else "")
+ return ""
+
+
+def _gw_error(status, message):
+ """A REST API gateway error response. Uses the lowercase `message` key that
+ API Gateway's default UNAUTHORIZED / MISSING_AUTHENTICATION_TOKEN / 5XX
+ gateway responses carry."""
+ return (
+ status,
+ {"Content-Type": "application/json"},
+ json.dumps({"message": message}).encode(),
+ )
+
+
+def _deny_error(explicit):
+ """The 403 an authorizer denial produces. The execute-api authorization
+ layer answers with a capitalised `Message` key (distinct from the lowercase
+ gateway-response bodies) — explicit Deny vs. no-matching-Allow differ only in
+ the trailing clause."""
+ msg = (
+ "User is not authorized to access this resource with an explicit deny"
+ if explicit
+ else "User is not authorized to access this resource"
+ )
+ return (
+ 403,
+ {"Content-Type": "application/json"},
+ json.dumps({"Message": msg}).encode(),
+ )
+
+
+def _build_method_arn(region, account_id, api_id, stage_name, method, request_path):
+ return (
+ f"arn:aws:execute-api:{region}:{account_id}:"
+ f"{api_id}/{stage_name}/{method}/{request_path.lstrip('/')}"
+ )
+
+
+def _arn_matches(pattern, arn):
+ """Match an IAM policy Resource against a method ARN, honoring `*`/`?` globs."""
+ regex = "^" + re.escape(pattern).replace(r"\*", ".*").replace(r"\?", ".") + "$"
+ return re.match(regex, arn) is not None
+
+
+def _evaluate_policy(policy_doc, method_arn):
+ """Evaluate an authorizer IAM policy against the method ARN.
+
+ Returns "Deny" (explicit deny matched — always wins), "Allow" (an Allow
+ matched and no Deny did), or "NoMatch" (implicit deny — no statement matched).
+ """
+ if not isinstance(policy_doc, dict):
+ return "NoMatch"
+ statements = policy_doc.get("Statement") or []
+ if isinstance(statements, dict):
+ statements = [statements]
+ allow = False
+ for st in statements:
+ if not isinstance(st, dict):
+ continue
+ resources = st.get("Resource")
+ if isinstance(resources, str):
+ resources = [resources]
+ if not any(_arn_matches(r, method_arn) for r in (resources or [])):
+ continue
+ if st.get("Effect") == "Deny":
+ return "Deny"
+ if st.get("Effect") == "Allow":
+ allow = True
+ return "Allow" if allow else "NoMatch"
+
+
+def _stringify_context(context):
+ """Authorizer context reaches the backend with every value stringified;
+ booleans become JSON-style `true`/`false`."""
+ out = {}
+ for k, v in (context or {}).items():
+ if isinstance(v, bool):
+ out[k] = "true" if v else "false"
+ elif v is None:
+ continue
+ elif isinstance(v, (str, int, float)):
+ out[k] = str(v)
+ # nested dict/list values are not supported by API Gateway — dropped.
+ return out
+
+
+async def _invoke_authorizer_lambda(authorizer, event, account_id, region):
+ """Invoke the authorizer's Lambda and return its raw execution result dict."""
+ lambda_ref = _extract_lambda_ref_from_integration_uri(authorizer.get("authorizerUri", ""))
+ from ministack.services import lambda_svc
+
+ func_data, func_config, func_name = lambda_svc._get_func_record_for_ref_in_scope(
+ lambda_ref, account_id=account_id, region=region,
+ )
+ if func_data is None or func_config is None:
+ return None
+ exec_record = lambda_svc._execution_record_for_config(func_data, func_config)
+ return await asyncio.to_thread(
+ lambda_svc._execute_function_with_config_scope, exec_record, event
+ )
+
+
+def _request_identity_sources(identity_source, headers, query_params, stage):
+ """Resolve a REQUEST authorizer's identitySource list to (all_present, values).
+
+ identitySource is a comma-separated list of `method.request.*` / stage-variable
+ mappings. Returns whether every source is present+non-empty and the ordered
+ values (used as the cache-key parts)."""
+ present = True
+ values = []
+ for raw in (identity_source or "").split(","):
+ src = raw.strip()
+ if not src:
+ continue
+ val = ""
+ if src.startswith("method.request.header."):
+ val = _header_ci(headers, src[len("method.request.header."):])
+ elif src.startswith("method.request.querystring."):
+ qn = src[len("method.request.querystring."):]
+ qv = (query_params or {}).get(qn)
+ val = (qv[0] if isinstance(qv, list) else qv) or ""
+ elif src.startswith("stageVariables."):
+ val = ((stage.get("variables") or {}).get(src[len("stageVariables."):])) or ""
+ elif src.startswith("context."):
+ val = "" # context identity sources not modeled; treated as absent
+ values.append(val)
+ if not val:
+ present = False
+ return present, values
+
+
+async def _authorize_request_v1(
+ api_id, stage_name, method_obj, method, request_path, resource,
+ headers, body, query_params, path_params, stage,
+ owner_account_id, owner_region,
+):
+ """Enforce a method's authorization before the integration runs.
+
+ Returns ``(error_response, authorizer_context)``: when ``error_response`` is
+ not None the caller must short-circuit with it; otherwise ``authorizer_context``
+ (possibly None) is injected into the integration's ``requestContext.authorizer``.
+ """
+ auth_type = (method_obj.get("authorizationType") or "NONE").upper()
+ if auth_type in ("", "NONE"):
+ return None, None
+ if auth_type == "AWS_IAM":
+ # We do not verify SigV4 signatures; match AWS only to the extent that a
+ # request with no Authorization header is rejected as unauthenticated.
+ if not _header_ci(headers, "authorization"):
+ return _gw_error(403, "Missing Authentication Token"), None
+ return None, None
+ if auth_type != "CUSTOM":
+ # COGNITO_USER_POOLS and any future type: not enforced here (pass through).
+ return None, None
+
+ authorizer_id = method_obj.get("authorizerId")
+ authorizer = _authorizers_v1.get(api_id, {}).get(authorizer_id) if authorizer_id else None
+ if not authorizer:
+ return _gw_error(500, "Internal server error"), None
+
+ method_arn = _build_method_arn(
+ owner_region, owner_account_id, api_id, stage_name, method, request_path
+ )
+ atype = (authorizer.get("type") or "TOKEN").upper()
+ ttl = int(authorizer.get("authorizerResultTtlInSeconds") or 0)
+
+ if atype == "TOKEN":
+ identity_source = authorizer.get("identitySource") or "method.request.header.Authorization"
+ header_name = (
+ identity_source[len("method.request.header."):]
+ if identity_source.startswith("method.request.header.")
+ else "Authorization"
+ )
+ token = _header_ci(headers, header_name)
+ if not token:
+ return _gw_error(401, "Unauthorized"), None
+ val_expr = authorizer.get("identityValidationExpression")
+ if val_expr and not re.fullmatch(val_expr, token):
+ return _gw_error(401, "Unauthorized"), None
+ cache_key = token
+ event = {"type": "TOKEN", "authorizationToken": token, "methodArn": method_arn}
+ else: # REQUEST
+ identity_source = authorizer.get("identitySource") or ""
+ all_present, id_values = _request_identity_sources(
+ identity_source, headers, query_params, stage
+ )
+ # With caching on and identity sources declared, a missing source is a
+ # 401 without invoking the authorizer (matches AWS).
+ if ttl > 0 and identity_source and not all_present:
+ return _gw_error(401, "Unauthorized"), None
+ cache_key = "|".join(id_values)
+ single_headers = {k: (v if isinstance(v, str) else v[-1]) for k, v in headers.items()}
+ multi_headers = {k: ([v] if isinstance(v, str) else list(v)) for k, v in headers.items()}
+ qs_params = {k: v[0] for k, v in query_params.items()} if query_params else None
+ mv_qs_params = {k: list(v) for k, v in query_params.items()} if query_params else None
+ event = {
+ "type": "REQUEST",
+ "methodArn": method_arn,
+ "resource": resource["path"],
+ "path": request_path,
+ "httpMethod": method,
+ "headers": single_headers,
+ "multiValueHeaders": multi_headers,
+ "queryStringParameters": qs_params,
+ "multiValueQueryStringParameters": mv_qs_params,
+ "pathParameters": path_params or None,
+ "stageVariables": stage.get("variables") or None,
+ "requestContext": {
+ "resourceId": resource["id"],
+ "stage": stage_name,
+ "resourcePath": resource["path"],
+ "httpMethod": method,
+ "apiId": api_id,
+ "accountId": owner_account_id,
+ },
+ }
+
+ # TTL cache: a hit replays the prior decision without invoking the Lambda.
+ ckey = (api_id, authorizer_id, cache_key)
+ if ttl > 0:
+ hit = _authorizer_cache.get(ckey)
+ if hit and hit[0] > time.time():
+ _, decision, cached_ctx = hit
+ if decision == "Allow":
+ return None, cached_ctx
+ return _deny_error(decision == "Deny"), None
+
+ result = await _invoke_authorizer_lambda(
+ authorizer, event, owner_account_id, owner_region
+ )
+ if result is None:
+ # Authorizer Lambda unresolved / not found → connection failure.
+ return _gw_error(500, "Internal server error"), None
+ if result.get("error"):
+ err_body = result.get("body") or {}
+ msg = err_body.get("errorMessage", "") if isinstance(err_body, dict) else str(err_body)
+ # A function that raises exactly "Unauthorized" maps to 401; any other
+ # uncaught error is an authorizer failure → 500.
+ if isinstance(msg, str) and msg.strip().lower() == "unauthorized":
+ return _gw_error(401, "Unauthorized"), None
+ return _gw_error(500, "Internal server error"), None
+
+ policy = result.get("body")
+ if isinstance(policy, (str, bytes)):
+ if isinstance(policy, bytes):
+ policy = policy.decode("utf-8", errors="replace")
+ try:
+ policy = json.loads(policy)
+ except json.JSONDecodeError:
+ policy = None
+ if not isinstance(policy, dict):
+ return _gw_error(500, "Internal server error"), None
+
+ decision = _evaluate_policy(policy.get("policyDocument"), method_arn)
+ if decision != "Allow":
+ if ttl > 0:
+ _authorizer_cache[ckey] = (time.time() + ttl, decision, None)
+ return _deny_error(decision == "Deny"), None
+
+ auth_ctx = _stringify_context(policy.get("context"))
+ principal_id = policy.get("principalId")
+ if principal_id is not None:
+ auth_ctx["principalId"] = str(principal_id)
+ if ttl > 0:
+ _authorizer_cache[ckey] = (time.time() + ttl, "Allow", auth_ctx)
+ return None, auth_ctx
+
+
async def _handle_execute_in_scope(
api_id, stage_name, method, path, headers, body, query_params,
owner_account_id, owner_region,
@@ -1221,6 +1503,16 @@ async def _handle_execute_in_scope(
if not integration:
return 500, {"Content-Type": "application/json"}, json.dumps({"message": "No integration configured"}).encode()
+ # Authorize before dispatching to the integration (custom Lambda authorizers,
+ # plus an Authorization-header presence check for AWS_IAM methods).
+ auth_error, authorizer_context = await _authorize_request_v1(
+ api_id, stage_name, method_obj, method, path, resource,
+ headers, body, query_params, path_params, stage,
+ owner_account_id, owner_region,
+ )
+ if auth_error is not None:
+ return auth_error
+
int_type = integration.get("type", "")
if int_type in ("AWS_PROXY", "AWS"):
@@ -1230,6 +1522,7 @@ async def _handle_execute_in_scope(
owner_account_id=owner_account_id,
owner_region=owner_region,
binary_media_types=api.get("binaryMediaTypes") or [],
+ authorizer_context=authorizer_context,
)
elif int_type in ("HTTP_PROXY", "HTTP"):
return await _invoke_http_proxy_v1(
@@ -1276,6 +1569,7 @@ async def _invoke_lambda_proxy_v1(
owner_account_id=None,
owner_region=None,
binary_media_types=None,
+ authorizer_context=None,
):
"""Invoke Lambda with API Gateway v1 payload format 1.0."""
uri = integration.get("uri", "")
@@ -1342,6 +1636,11 @@ async def _invoke_lambda_proxy_v1(
"isBase64Encoded": req_is_base64,
}
+ # A custom authorizer's returned context (values stringified) plus its
+ # principalId reach the integration under requestContext.authorizer.
+ if authorizer_context is not None:
+ event["requestContext"]["authorizer"] = authorizer_context
+
lambda_response, err = await _call_lambda(
lambda_ref,
event,
diff --git a/ministack/services/cloudformation/engine.py b/ministack/services/cloudformation/engine.py
index 49f1decc..faaffe44 100644
--- a/ministack/services/cloudformation/engine.py
+++ b/ministack/services/cloudformation/engine.py
@@ -130,18 +130,14 @@ _AWS_SPECIFIC_TYPES = {
"AWS::Route53::HostedZone::Id",
}
-# Unlike the other _AWS_SPECIFIC_TYPES entries (an AZ name, an AMI id, ...),
-# whose given value already *is* the value CloudFormation hands to Ref, an
-# ``AWS::SSM::Parameter::Value<...>`` parameter's given value is an SSM
-# parameter *name* — real CloudFormation resolves it against SSM Parameter
-# Store before Ref ever sees it. Kept as its own set so
-# ``AWS::SSM::Parameter::Type`` (a template-side type constraint, not an
-# SSM lookup at all) isn't accidentally swept in here too.
-_SSM_PARAMETER_VALUE_TYPES = {
- "AWS::SSM::Parameter::Value<String>",
- "AWS::SSM::Parameter::Value<List<String>>",
- "AWS::SSM::Parameter::Value<AWS::EC2::Image::Id>",
-}
+# Any ``AWS::SSM::Parameter::Value<...>`` type resolves its given value (an SSM
+# parameter *name*) against SSM Parameter Store before Ref ever sees it — this
+# is true for every inner type (String, List<String>, CommaDelimitedList, and
+# the AWS-specific and List<AWS-specific> forms), so match on the prefix rather
+# than an enumerated subset. The ``Value<`` in the prefix deliberately excludes
+# ``AWS::SSM::Parameter::Name`` (Ref returns the name) and
+# ``AWS::SSM::Parameter::Type`` (a template-side type constraint, not a lookup).
+_SSM_PARAMETER_VALUE_PREFIX = "AWS::SSM::Parameter::Value<"
def _resolve_parameters(template: dict, provided_params: list[dict],
@@ -190,7 +186,7 @@ def _resolve_parameters(template: dict, provided_params: list[dict],
value = str(value) if value is not None else ""
- if ptype in _SSM_PARAMETER_VALUE_TYPES and not already_resolved:
+ if ptype.startswith(_SSM_PARAMETER_VALUE_PREFIX) and not already_resolved:
# `value` up to here is the SSM parameter *name* (the template
# parameter's Default, or a caller-supplied override) — resolve
# it against the SSM store the same way real CloudFormation
diff --git a/ministack/services/dsql.py b/ministack/services/dsql.py
index 70af6ca8..d89e8735 100644
--- a/ministack/services/dsql.py
+++ b/ministack/services/dsql.py
@@ -4,9 +4,9 @@ Amazon Aurora DSQL Emulator (control plane + data plane wiring).
REST-JSON API (no X-Amz-Target; routed on method + path). Covers cluster
lifecycle, tags, and cluster policies.
-Data plane: with ``DSQL_CONTAINERS=1`` and a Docker daemon available, each
-cluster gets a real Postgres container (RDS idiom, capped by
-``DSQL_MAX_CONTAINERS``, default 30) fronted by an in-process wire-protocol
+Data plane: with ``DSQL_STRICT=1`` and a Docker daemon available, each
+cluster gets a real Postgres container (RDS idiom, capped by a fixed port
+window off ``DSQL_BASE_PORT``, default 30 ports) fronted by an in-process wire-protocol
proxy (`ministack.core.pgproxy`) that enforces DSQL's SQL subset. Otherwise
the cluster goes ACTIVE metadata-only — nothing listens on its port.
"""
@@ -33,11 +33,15 @@ from ministack.core.responses import (
logger = logging.getLogger("dsql")
BASE_PORT = int(os.environ.get("DSQL_BASE_PORT", "25432"))
-MAX_CONTAINERS = int(os.environ.get("DSQL_MAX_CONTAINERS", "30")) # cap on real Postgres backend containers (one per cluster)
+# Real Postgres backend containers are capped by a fixed port window off
+# DSQL_BASE_PORT (one port per cluster), not a separate knob: a cluster whose
+# allocated port falls outside [BASE_PORT, BASE_PORT + _BACKEND_PORT_WINDOW) is
+# served metadata-only.
+_BACKEND_PORT_WINDOW = 30
PG_IMAGE = os.environ.get("DSQL_PG_IMAGE", "postgres:16-alpine")
DSQL_PERSIST = os.environ.get("DSQL_PERSIST", "0").lower() in ("1", "true", "yes")
# Backend containers are opt-in: default is control-plane-only (metadata stubs).
-DSQL_CONTAINERS = os.environ.get("DSQL_CONTAINERS", "0").lower() in ("1", "true", "yes")
+DSQL_STRICT = os.environ.get("DSQL_STRICT", "0").lower() in ("1", "true", "yes")
_clusters = AccountScopedDict() # identifier -> cluster dict
_client_tokens = AccountScopedDict() # clientToken -> identifier (create idempotency)
@@ -188,9 +192,9 @@ def _docker_available():
def _backends_enabled():
"""Whether real Postgres backend containers should run for clusters.
- Opt-in via ``DSQL_CONTAINERS=1``; default is control-plane-only stubs.
+ Opt-in via ``DSQL_STRICT=1``; default is control-plane-only stubs.
"""
- return DSQL_CONTAINERS and _docker_available()
+ return DSQL_STRICT and _docker_available()
def _container_name(identifier):
@@ -411,14 +415,17 @@ def _create_cluster(data):
port = _next_port()
has_backend = _backends_enabled()
if has_backend:
+ # Cap concurrent backends to the port window off DSQL_BASE_PORT (one
+ # port per cluster). Counting live backends (not the raw port counter)
+ # so a deleted cluster frees capacity.
backend_count = sum(
1 for c in _clusters._data.values() if c.get("_has_backend")
)
- if backend_count >= MAX_CONTAINERS:
+ if backend_count >= _BACKEND_PORT_WINDOW:
logger.warning(
- "DSQL: backend container cap reached (%d) — cluster %s is "
- "metadata-only. Raise DSQL_MAX_CONTAINERS to allow more.",
- MAX_CONTAINERS,
+ "DSQL: backend port window off DSQL_BASE_PORT (%d ports) full "
+ "— cluster %s is metadata-only. Widen the range to allow more.",
+ _BACKEND_PORT_WINDOW,
identifier,
)
has_backend = False
@@ -429,7 +436,7 @@ def _create_cluster(data):
# With a backend the Postgres container + wire proxy spin up in the
# background; without one the cluster is metadata-only ACTIVE.
"status": "CREATING" if has_backend else "ACTIVE",
- "creationTime": time.time(),
+ "creationTime": int(time.time()),
# AWS creates clusters with deletion protection ON by default.
"deletionProtectionEnabled": bool(data.get("deletionProtectionEnabled", True)),
"encryptionDetails": _encryption_details(data.get("kmsEncryptionKey"), account_id, region),
@@ -844,7 +851,7 @@ def restore_state(data):
# `_respawn_backend` once the loop is known, or via the lifespan hook
# `start_restored_proxies`. The cap is deliberately not re-enforced here:
# clusters that had a backend get it back (best effort), so a restart can
- # transiently exceed MAX_CONTAINERS.
+ # transiently exceed the backend port window.
for cluster in _clusters._data.values():
if not isinstance(cluster, dict) or "identifier" not in cluster:
continue
diff --git a/ministack/services/route53.py b/ministack/services/route53.py
index eb295862..bb582310 100644
--- a/ministack/services/route53.py
+++ b/ministack/services/route53.py
@@ -438,7 +438,7 @@ def _create_hosted_zone(body: bytes, query_params: dict):
if caller_ref in _caller_refs:
existing_id = _caller_refs[caller_ref]
zone = _zones[existing_id]
- change = {"id": _change_id(), "status": "INSYNC", "submitted_at": _now_iso(), "comment": ""}
+ change = {"id": _change_id(), "status": "PENDING", "submitted_at": _now_iso(), "comment": ""}
def build(root):
_build_hosted_zone_el(root, zone)
_build_change_info_el(root, change)
@@ -458,7 +458,7 @@ def _create_hosted_zone(body: bytes, query_params: dict):
_caller_refs[caller_ref] = zone_id
change_id = _change_id()
- change = {"id": change_id, "status": "INSYNC", "submitted_at": _now_iso(), "comment": ""}
+ change = {"id": change_id, "status": "PENDING", "submitted_at": _now_iso(), "comment": ""}
_changes[change_id] = change
def build(root):
@@ -498,7 +498,7 @@ def _delete_hosted_zone(zone_id: str):
del _records[zone_id]
_caller_refs.pop(zone.get("caller_reference", ""), None)
change_id = _change_id()
- change = {"id": change_id, "status": "INSYNC", "submitted_at": _now_iso(), "comment": ""}
+ change = {"id": change_id, "status": "PENDING", "submitted_at": _now_iso(), "comment": ""}
_changes[change_id] = change
def build(root):
@@ -659,7 +659,7 @@ def _change_resource_record_sets(zone_id: str, body: bytes):
_records[zone_id] = current
change_id = _change_id()
- change = {"id": change_id, "status": "INSYNC", "submitted_at": _now_iso(), "comment": comment}
+ change = {"id": change_id, "status": "PENDING", "submitted_at": _now_iso(), "comment": comment}
_changes[change_id] = change
def build(root):
@@ -734,11 +734,19 @@ def _list_resource_record_sets(zone_id: str, query_params: dict):
def _get_change(change_id: str):
with _lock:
change = _changes.get(change_id)
- if not change:
- return _error_response("NoSuchChange", f"A change with the ID {change_id} does not exist.", 404)
+ if not change:
+ return _error_response("NoSuchChange", f"A change with the ID {change_id} does not exist.", 404)
+ # Route 53 returns PENDING right after a submission and flips to INSYNC
+ # once the change has propagated (typically within 60s). Emulate that
+ # transition lazily: the first GetChange observes PENDING, and every
+ # read thereafter observes INSYNC. No background timer, no opt-out knob
+ # — the observable PENDING→INSYNC transition is what waiters poll on.
+ observed = dict(change)
+ if change["status"] == "PENDING":
+ change["status"] = "INSYNC"
def build(root):
- _build_change_info_el(root, change)
+ _build_change_info_el(root, observed)
return _xml_response("GetChangeResponse", build)
diff --git a/ministack/services/s3.py b/ministack/services/s3.py
index 18da7669..d4166e07 100644
--- a/ministack/services/s3.py
+++ b/ministack/services/s3.py
@@ -2500,6 +2500,9 @@ def _put_object(bucket_name: str, key: str, body: bytes, headers: dict):
"is_latest": True,
"data": body,
"content_type": obj.get("content_type") or "application/octet-stream",
+ "content_encoding": obj.get("content_encoding"),
+ "metadata": obj.get("metadata", {}),
+ "preserved_headers": obj.get("preserved_headers", {}),
"storage_class": obj.get("storage_class") or "STANDARD",
"checksums": obj.get("checksums") or {},
})
@@ -2701,6 +2704,10 @@ def _post_object(bucket_name: str, body: bytes, headers: dict):
"size": obj["size"],
"is_latest": True,
"data": file_value,
+ "content_type": obj.get("content_type") or "application/octet-stream",
+ "content_encoding": obj.get("content_encoding"),
+ "metadata": obj.get("metadata", {}),
+ "preserved_headers": obj.get("preserved_headers", {}),
"storage_class": obj.get("storage_class") or "STANDARD",
})
for v in _object_versions[vkey][:-1]:
@@ -2806,24 +2813,19 @@ def _get_object(bucket_name: str, key: str, headers: dict, query_params: dict =
versions = _object_versions.get(vkey, [])
for v in versions:
if v["version_id"] == version_id:
- resp_headers = {
- "Content-Type": v.get("content_type") or "application/octet-stream",
- "ETag": v["etag"],
- "Content-Length": str(v["size"]),
- "Last-Modified": iso_to_rfc7231(v["last_modified"]),
- "x-amz-version-id": v["version_id"],
- }
- # Versioned reads honor `x-amz-checksum-mode: ENABLED` the same
- # way current-version reads do — the stored per-version
- # checksums are looked up and surfaced as `x-amz-checksum-*`.
- if (headers.get("x-amz-checksum-mode") or "").upper() == "ENABLED":
- stored = v.get("checksums") or {}
- for alg, val in stored.items():
- resp_headers[f"x-amz-checksum-{alg.lower()}"] = val
- if stored:
- resp_headers["x-amz-checksum-type"] = "FULL_OBJECT"
+ # Route through the shared header builder (same path as versioned
+ # HeadObject) so a version's user metadata (x-amz-meta-*),
+ # preserved headers and content-encoding are emitted — not just
+ # the five wire basics. Versioned reads honor
+ # `x-amz-checksum-mode: ENABLED` exactly as current-version reads do.
+ include_checksums = (headers.get("x-amz-checksum-mode") or "").upper() == "ENABLED"
+ vobj = _object_record_from_version(v)
+ resp_headers = _object_response_headers(
+ vobj, bucket_name, key, include_checksums=include_checksums)
resp_headers.update(_object_tagging_count_header(bucket_name, key, version_id))
- body = v["data"] if v["data"] is not None else _read_body(bucket_name, key, bucket["objects"].get(key, {}))
+ body = v.get("data")
+ if body is None:
+ body = _read_body(bucket_name, key, bucket["objects"].get(key, {}))
return 200, resp_headers, body
return _error("NoSuchVersion", "The specified version does not exist.", 404, f"/{bucket_name}/{key}")
@@ -3442,7 +3444,12 @@ def _copy_object(bucket_name: str, dest_key: str, headers: dict):
"size": dest_obj["size"],
"is_latest": True,
"data": src_body,
+ "content_type": dest_obj.get("content_type") or "application/octet-stream",
+ "content_encoding": dest_obj.get("content_encoding"),
+ "metadata": dest_obj.get("metadata", {}),
+ "preserved_headers": dest_obj.get("preserved_headers", {}),
"storage_class": dest_obj.get("storage_class") or "STANDARD",
+ "checksums": dest_obj.get("checksums") or {},
})
for v in _object_versions[vkey][:-1]:
v["is_latest"] = False
@@ -4623,6 +4630,11 @@ def _complete_multipart_upload(
"size": obj["size"],
"is_latest": True,
"data": combined,
+ "content_type": obj.get("content_type") or "application/octet-stream",
+ "content_encoding": obj.get("content_encoding"),
+ "metadata": obj.get("metadata", {}),
+ "preserved_headers": obj.get("preserved_headers", {}),
+ "storage_class": obj.get("storage_class") or "STANDARD",
})
for v in _object_versions[vkey][:-1]:
v["is_latest"] = False
@@ -5034,6 +5046,10 @@ def _load_persisted_bucket(account_id, bucket_name, bucket_path):
"size": size,
"is_latest": True,
"data": None,
+ "content_type": meta.get("content_type", "application/octet-stream"),
+ "content_encoding": meta.get("content_encoding"),
+ "metadata": meta.get("metadata", {}),
+ "preserved_headers": meta.get("preserved_headers", {}),
"storage_class": meta.get("storage_class", "STANDARD"),
"checksums": meta.get("checksums", {}),
})
diff --git a/tests/test_apigatewayv1.py b/tests/test_apigatewayv1.py
index e8937fd5..e5100242 100644
--- a/tests/test_apigatewayv1.py
+++ b/tests/test_apigatewayv1.py
@@ -1022,6 +1022,194 @@ def test_apigwv1_execute_lambda_proxy(apigw_v1, lam):
lam.delete_function(FunctionName=fname)
+def test_apigwv1_execute_token_authorizer(apigw_v1, lam):
+ """A CUSTOM TOKEN authorizer is invoked on the data path and enforced. (#1345)
+
+ - No identity-source header -> 401 without invoking the authorizer.
+ - Deny policy -> 403.
+ - Authorizer raises Unauthorized -> 401.
+ - Allow policy -> 200, with the returned context (values
+ stringified) plus principalId injected into requestContext.authorizer.
+ """
+ import urllib.request as _urlreq
+ import urllib.error as _urlerr
+ import uuid as _uuid
+
+ suffix = _uuid.uuid4().hex[:8]
+ auth_fn = f"intg-v1-authfn-{suffix}"
+ int_fn = f"intg-v1-intfn-{suffix}"
+
+ auth_code = (
+ b"def handler(event, context):\n"
+ b" tok = event.get('authorizationToken')\n"
+ b" arn = event['methodArn']\n"
+ b" if tok == 'deny-me':\n"
+ b" return _p('alice', 'Deny', arn, {})\n"
+ b" if tok == 'unauth':\n"
+ b" raise Exception('Unauthorized')\n"
+ b" return _p('alice', 'Allow', arn, {'user': 'alice', 'admin': True, 'count': 3})\n"
+ b"def _p(pid, effect, arn, ctx):\n"
+ b" return {'principalId': pid, 'context': ctx, 'policyDocument': {\n"
+ b" 'Version': '2012-10-17', 'Statement': [\n"
+ b" {'Action': 'execute-api:Invoke', 'Effect': effect, 'Resource': arn}]}}\n"
+ )
+ int_code = (
+ b"import json\n"
+ b"def handler(event, context):\n"
+ b" auth = event.get('requestContext', {}).get('authorizer')\n"
+ b" return {'statusCode': 200, 'body': json.dumps(auth)}\n"
+ )
+
+ def _mkfn(name, code):
+ buf = io.BytesIO()
+ with zipfile.ZipFile(buf, "w") as zf:
+ zf.writestr("index.py", code)
+ lam.create_function(
+ FunctionName=name, Runtime="python3.12",
+ Role="arn:aws:iam::000000000000:role/test-role",
+ Handler="index.handler", Code={"ZipFile": buf.getvalue()},
+ )
+
+ _mkfn(auth_fn, auth_code)
+ _mkfn(int_fn, int_code)
+
+ api_id = apigw_v1.create_rest_api(name=f"v1-authz-{suffix}")["id"]
+ try:
+ root = next(r for r in apigw_v1.get_resources(restApiId=api_id)["items"] if r["path"] == "/")
+ resource_id = apigw_v1.create_resource(
+ restApiId=api_id, parentId=root["id"], pathPart="private",
+ )["id"]
+
+ auth_uri = (
+ f"arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/"
+ f"arn:aws:lambda:us-east-1:000000000000:function:{auth_fn}/invocations"
+ )
+ authorizer_id = apigw_v1.create_authorizer(
+ restApiId=api_id, name="tok-auth", type="TOKEN",
+ authorizerUri=auth_uri,
+ identitySource="method.request.header.Authorization",
+ authorizerResultTtlInSeconds=0,
+ )["id"]
+
+ apigw_v1.put_method(
+ restApiId=api_id, resourceId=resource_id, httpMethod="GET",
+ authorizationType="CUSTOM", authorizerId=authorizer_id,
+ )
+ apigw_v1.put_integration(
+ restApiId=api_id, resourceId=resource_id, httpMethod="GET",
+ type="AWS_PROXY", integrationHttpMethod="POST",
+ uri=(
+ f"arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/"
+ f"arn:aws:lambda:us-east-1:000000000000:function:{int_fn}/invocations"
+ ),
+ )
+ dep_id = apigw_v1.create_deployment(restApiId=api_id)["id"]
+ apigw_v1.create_stage(restApiId=api_id, stageName="dev", deploymentId=dep_id)
+
+ host = f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}"
+ url = f"http://{host}/dev/private"
+
+ def _call(token):
+ req = _urlreq.Request(url, method="GET")
+ req.add_header("Host", host)
+ if token is not None:
+ req.add_header("Authorization", token)
+ try:
+ r = _urlreq.urlopen(req)
+ return r.status, r.read()
+ except _urlerr.HTTPError as e:
+ return e.code, e.read()
+
+ # No Authorization header -> 401, authorizer not invoked.
+ status, _ = _call(None)
+ assert status == 401
+
+ # Deny policy -> 403.
+ status, _ = _call("deny-me")
+ assert status == 403
+
+ # Authorizer raises "Unauthorized" -> 401.
+ status, _ = _call("unauth")
+ assert status == 401
+
+ # Allow policy -> 200; context stringified + principalId injected.
+ status, body = _call("allow-me")
+ assert status == 200
+ auth = json.loads(body)
+ assert auth is not None
+ assert auth["principalId"] == "alice"
+ assert auth["user"] == "alice"
+ assert auth["admin"] == "true"
+ assert auth["count"] == "3"
+ finally:
+ apigw_v1.delete_rest_api(restApiId=api_id)
+ lam.delete_function(FunctionName=auth_fn)
+ lam.delete_function(FunctionName=int_fn)
+
+
+def test_apigwv1_execute_aws_iam_requires_auth_header(apigw_v1, lam):
+ """AWS_IAM methods reject requests without an Authorization header (403). (#1345)
+
+ We do not verify the SigV4 signature; only the presence check is enforced.
+ """
+ import urllib.request as _urlreq
+ import urllib.error as _urlerr
+ import uuid as _uuid
+
+ fname = f"intg-v1-iam-{_uuid.uuid4().hex[:8]}"
+ code = b"def handler(event, context):\n return {'statusCode': 200, 'body': 'ok'}\n"
+ buf = io.BytesIO()
+ with zipfile.ZipFile(buf, "w") as zf:
+ zf.writestr("index.py", code)
+ lam.create_function(
+ FunctionName=fname, Runtime="python3.12",
+ Role="arn:aws:iam::000000000000:role/test-role",
+ Handler="index.handler", Code={"ZipFile": buf.getvalue()},
+ )
+
+ api_id = apigw_v1.create_rest_api(name=f"v1-iam-{fname}")["id"]
+ try:
+ root = next(r for r in apigw_v1.get_resources(restApiId=api_id)["items"] if r["path"] == "/")
+ resource_id = apigw_v1.create_resource(
+ restApiId=api_id, parentId=root["id"], pathPart="secure",
+ )["id"]
+ apigw_v1.put_method(
+ restApiId=api_id, resourceId=resource_id, httpMethod="GET",
+ authorizationType="AWS_IAM",
+ )
+ apigw_v1.put_integration(
+ restApiId=api_id, resourceId=resource_id, httpMethod="GET",
+ type="AWS_PROXY", integrationHttpMethod="POST",
+ uri=(
+ f"arn:aws:apigateway:us-east-1:lambda:path/2015-03-31/functions/"
+ f"arn:aws:lambda:us-east-1:000000000000:function:{fname}/invocations"
+ ),
+ )
+ dep_id = apigw_v1.create_deployment(restApiId=api_id)["id"]
+ apigw_v1.create_stage(restApiId=api_id, stageName="dev", deploymentId=dep_id)
+
+ host = f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}"
+ url = f"http://{host}/dev/secure"
+
+ req = _urlreq.Request(url, method="GET")
+ req.add_header("Host", host)
+ try:
+ r = _urlreq.urlopen(req)
+ status = r.status
+ except _urlerr.HTTPError as e:
+ status = e.code
+ assert status == 403
+
+ req = _urlreq.Request(url, method="GET")
+ req.add_header("Host", host)
+ req.add_header("Authorization", "AWS4-HMAC-SHA256 Credential=...")
+ r = _urlreq.urlopen(req)
+ assert r.status == 200
+ finally:
+ apigw_v1.delete_rest_api(restApiId=api_id)
+ lam.delete_function(FunctionName=fname)
+
+
def test_apigwv1_execute_lambda_proxy_multi_value_headers(apigw_v1, lam):
"""Payload format 1.0 `multiValueHeaders` yields one header line per value.
diff --git a/tests/test_dsql.py b/tests/test_dsql.py
index 2b589b92..a9df1d47 100644
--- a/tests/test_dsql.py
+++ b/tests/test_dsql.py
@@ -971,15 +971,15 @@ class TestReset:
class TestContainerCap:
def test_cap_exceeded_logs_and_degrades_to_stub(self, monkeypatch, caplog):
- """When the backend container cap is reached, creating another cluster
- logs a warning naming DSQL_MAX_CONTAINERS and goes metadata-only."""
+ """When the backend port window is exhausted, creating another cluster
+ logs a warning naming DSQL_BASE_PORT and goes metadata-only."""
import json
import logging
from ministack.services import dsql as dsql_mod
monkeypatch.setattr(dsql_mod, "_backends_enabled", lambda: True)
- monkeypatch.setattr(dsql_mod, "MAX_CONTAINERS", 0)
+ monkeypatch.setattr(dsql_mod, "_BACKEND_PORT_WINDOW", 0)
with caplog.at_level(logging.WARNING, logger="dsql"):
status, _, body = dsql_mod._create_cluster({})
@@ -991,7 +991,7 @@ class TestContainerCap:
assert cluster["status"] == "ACTIVE" # stub, not CREATING
assert dsql_mod._clusters[identifier]["_has_backend"] is False
assert any(
- "DSQL_MAX_CONTAINERS" in r.message and "cap reached" in r.message
+ "DSQL_BASE_PORT" in r.message and "metadata-only" in r.message
for r in caplog.records
if r.name == "dsql" and r.levelno == logging.WARNING
)
@@ -999,13 +999,13 @@ class TestContainerCap:
dsql_mod._clusters.clear()
def test_containers_disabled_means_stub_even_with_docker(self, monkeypatch):
- """Default posture: Docker available but DSQL_CONTAINERS unset/off —
+ """Default posture: Docker available but DSQL_STRICT unset/off —
clusters are metadata-only stubs and no backend is started."""
import json
from ministack.services import dsql as dsql_mod
- monkeypatch.setattr(dsql_mod, "DSQL_CONTAINERS", False)
+ monkeypatch.setattr(dsql_mod, "DSQL_STRICT", False)
monkeypatch.setattr(dsql_mod, "_docker_available", lambda: True)
assert dsql_mod._backends_enabled() is False
@@ -1023,9 +1023,9 @@ class TestContainerCap:
from ministack.services import dsql as dsql_mod
monkeypatch.setattr(dsql_mod, "_docker_available", lambda: True)
- monkeypatch.setattr(dsql_mod, "DSQL_CONTAINERS", False)
+ monkeypatch.setattr(dsql_mod, "DSQL_STRICT", False)
assert dsql_mod._backends_enabled() is False
- monkeypatch.setattr(dsql_mod, "DSQL_CONTAINERS", True)
+ monkeypatch.setattr(dsql_mod, "DSQL_STRICT", True)
assert dsql_mod._backends_enabled() is True
monkeypatch.setattr(dsql_mod, "_docker_available", lambda: False)
assert dsql_mod._backends_enabled() is False
@@ -1276,7 +1276,7 @@ def _pg_connect(port, autocommit=True):
@requires_docker
class TestContainersE2E:
- """End-to-end for DSQL_CONTAINERS=1: _create_cluster spins up a real
+ """End-to-end for DSQL_STRICT=1: _create_cluster spins up a real
Postgres container behind the wire proxy, reachable over SQL. Runs
in-process (flag monkeypatched on) wherever a Docker daemon exists."""
@@ -1288,7 +1288,7 @@ class TestContainersE2E:
from ministack.services import dsql as dsql_mod
- monkeypatch.setattr(dsql_mod, "DSQL_CONTAINERS", True)
+ monkeypatch.setattr(dsql_mod, "DSQL_STRICT", True)
# _start_backend stashes the test's ephemeral loop in this global;
# put the original back afterwards (all consumers guard is_running(),
# but don't leak a closed loop into other tests).
@@ -1311,7 +1311,7 @@ class TestContainersE2E:
def _select_one():
conn = psycopg2.connect(
host="127.0.0.1",
- port=cluster["port"],
+ port=int(cluster["endpoint"].split(":")[1]),
user="admin",
password="anything",
dbname="postgres",
diff --git a/tests/test_route53.py b/tests/test_route53.py
index 257bd791..9ab5c188 100644
--- a/tests/test_route53.py
+++ b/tests/test_route53.py
@@ -77,7 +77,8 @@ def test_route53_change_resource_record_sets_create(r53):
]
},
)
- assert change_resp["ChangeInfo"]["Status"] == "INSYNC"
+ # ChangeResourceRecordSets returns PENDING on submission (real Route 53).
+ assert change_resp["ChangeInfo"]["Status"] == "PENDING"
def test_route53_list_resource_record_sets(r53):
resp = r53.create_hosted_zone(Name="listrrs.com", CallerReference="ref-lrrs-1")
@@ -473,9 +474,13 @@ def test_route53_get_change(r53):
]
},
)
+ # Route 53 is born PENDING and flips to INSYNC once propagated. We emulate
+ # that lazily: the first GetChange sees PENDING, subsequent reads INSYNC —
+ # so a poll loop / SDK waiter actually exercises its retry path.
+ assert change_resp["ChangeInfo"]["Status"] == "PENDING"
change_id = change_resp["ChangeInfo"]["Id"].split("/")[-1]
- get_change = r53.get_change(Id=change_id)
- assert get_change["ChangeInfo"]["Status"] == "INSYNC"
+ assert r53.get_change(Id=change_id)["ChangeInfo"]["Status"] == "PENDING"
+ assert r53.get_change(Id=change_id)["ChangeInfo"]["Status"] == "INSYNC"
def test_route53_create_health_check(r53):
resp = r53.create_health_check(
diff --git a/tests/test_s3.py b/tests/test_s3.py
index ed3c99eb..07da10a4 100644
--- a/tests/test_s3.py
+++ b/tests/test_s3.py
@@ -596,6 +596,41 @@ def test_s3_object_metadata(s3):
assert resp["Metadata"]["custom-key"] == "custom-value"
assert resp["Metadata"]["another"] == "data"
+
+def test_s3_versioned_object_metadata(s3):
+ """User metadata must round-trip on a versioned GetObject(VersionId). (#1342)
+
+ Each version keeps its own metadata; addressing a version by id returns
+ that version's metadata, and the current-version read returns the latest.
+ """
+ bkt = "intg-s3-meta-versioned"
+ s3.create_bucket(Bucket=bkt)
+ s3.put_bucket_versioning(
+ Bucket=bkt, VersioningConfiguration={"Status": "Enabled"}
+ )
+
+ v1 = s3.put_object(
+ Bucket=bkt, Key="k", Body=b"one",
+ Metadata={"gen": "one"}, ContentEncoding="gzip",
+ )["VersionId"]
+ v2 = s3.put_object(
+ Bucket=bkt, Key="k", Body=b"two", Metadata={"gen": "two"},
+ )["VersionId"]
+ assert v1 and v2 and v1 != v2
+
+ g1 = s3.get_object(Bucket=bkt, Key="k", VersionId=v1)
+ assert g1["Metadata"]["gen"] == "one"
+ assert g1["ContentEncoding"] == "gzip"
+ assert g1["Body"].read() == b"one"
+
+ g2 = s3.get_object(Bucket=bkt, Key="k", VersionId=v2)
+ assert g2["Metadata"]["gen"] == "two"
+
+ # Current-version read (no VersionId) targets the latest.
+ cur = s3.get_object(Bucket=bkt, Key="k")
+ assert cur["Metadata"]["gen"] == "two"
+
+
def test_s3_bucket_tagging(s3):
bkt = "intg-s3-bkttags"
s3.create_bucket(Bucket=bkt)