Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

feat(apigatewayv2): JWT auth, HTTP proxy parameter mappings, and non-blocking JWKS/proxy I/O (#518)

marcin-nowak-scl4118a69809de
diff --git a/CHANGELOG.md b/CHANGELOG.md
index fda112ca..b031f99e 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -10,6 +10,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
### Added
+- **API Gateway (HTTP API and REST): JWT authorizers, HTTP proxy integrations, and non-blocking I/O** — HTTP API (`apigateway`) and REST API (`apigateway_v1`): enforce JWT `issuer` + `audience` validation against a JWKS URL, request/response **parameter mapping** for `HTTP` / `HTTP_PROXY` integrations, and `ministack.core.nonblocking_http` so upstream proxy and JWKS work does not block the event loop. Operator tuning: `MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS` (default `30`), `MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS` (default `5`).
- **DynamoDB Streams read API** — new `ministack/services/dynamodb_streams.py` exposes `ListStreams`, `DescribeStream`, `GetShardIterator`, and `GetRecords` via `boto3.client("dynamodbstreams")` and the `streams.dynamodb.*` host. Reads the records already captured by the main DynamoDB service (emitted from `PutItem`, `UpdateItem`, `DeleteItem`, `TransactWriteItems`, and `BatchWriteItem`) so the public Streams API and the internal Lambda ESM path share one source of truth. Supports all four iterator types (`TRIM_HORIZON`, `LATEST`, `AT_SEQUENCE_NUMBER`, `AFTER_SEQUENCE_NUMBER`) and all four stream view types (`NEW_AND_OLD_IMAGES`, `NEW_IMAGE`, `OLD_IMAGE`, `KEYS_ONLY`). Single synthetic shard per stream; opaque base64 iterator tokens. Unblocks `DynamoDbOutboxWorker`-style consumers.
- **DynamoDB → Kinesis streaming destination** — `EnableKinesisStreamingDestination`, `DisableKinesisStreamingDestination`, `DescribeKinesisStreamingDestination`, and `UpdateKinesisStreamingDestination` on `boto3.client("dynamodb")`. Item mutations from `PutItem` / `UpdateItem` / `DeleteItem` / `TransactWriteItems` / `BatchWriteItem` fan out to every ACTIVE destination as JSON-encoded records (via `kinesis.put_record_internal`) for Kinesis / Lambda ESM / Firehose-style consumers. DISABLED destinations remain on `Describe` for the ~24h AWS window; `DeleteTable` drops destinations. Coverage: `tests/test_dynamodb_kinesis_destination.py`.
- **Native HTTPS via `USE_SSL=1`** — the gateway listener now speaks TLS when `USE_SSL=1` (also accepts `true` / `yes`). Drop-in compatible with LocalStack's `USE_SSL=true`, so a `compose.yml` switching from LocalStack to MiniStack doesn't need TLS-specific changes. By default, MiniStack auto-generates a self-signed RSA cert (CN: `ministack-local`, SAN: `localhost`, `ministack`, `127.0.0.1`, `::1`) cached under `${TMPDIR}/ministack-tls/` so the cert survives restarts. To pin a specific cert (e.g. an `mkcert`-issued one for browser trust), set `MINISTACK_SSL_CERT` and `MINISTACK_SSL_KEY` to PEM paths. Auto-generation uses the `cryptography` package (already shipped in both the regular and `:full` images). Unblocks AWS SDKs that hardcode `https://` against Cognito Hosted UI endpoints (e.g. Amplify v6) without needing a separate TLS-terminating proxy. Closes #526.
diff --git a/README.md b/README.md
index c96b63e4..df11506b 100644
--- a/README.md
+++ b/README.md
@@ -369,7 +369,7 @@ subnet = ec2.create_subnet(
| **Backup** | CreateBackupVault, DescribeBackupVault, DeleteBackupVault, ListBackupVaults, CreateBackupPlan, GetBackupPlan, UpdateBackupPlan, DeleteBackupPlan, ListBackupPlans, ListBackupPlanVersions, CreateBackupSelection, GetBackupSelection, DeleteBackupSelection, ListBackupSelections, StartBackupJob, StopBackupJob, DescribeBackupJob, ListBackupJobs, TagResource, UntagResource, ListTags | In-memory; jobs complete immediately; vaults and plans participate in Resource Groups Tagging API |
| **WAF v2** | CreateWebACL, GetWebACL, UpdateWebACL, DeleteWebACL, ListWebACLs, AssociateWebACL, DisassociateWebACL, GetWebACLForResource, ListResourcesForWebACL, CreateIPSet, GetIPSet, UpdateIPSet, DeleteIPSet, ListIPSets, CreateRuleGroup, GetRuleGroup, UpdateRuleGroup, DeleteRuleGroup, ListRuleGroups, TagResource, UntagResource, ListTagsForResource, CheckCapacity, DescribeManagedRuleGroup | LockToken enforced on Update/Delete; resource associations tracked |
| **Step Functions** | CreateStateMachine, DeleteStateMachine, DescribeStateMachine, UpdateStateMachine, ListStateMachines, StartExecution, StartSyncExecution, StopExecution, DescribeExecution, DescribeStateMachineForExecution, ListExecutions, GetExecutionHistory, SendTaskSuccess, SendTaskFailure, SendTaskHeartbeat, CreateActivity, DeleteActivity, DescribeActivity, ListActivities, GetActivityTask, TestState, TagResource, UntagResource, ListTagsForResource | Full ASL interpreter; Retry/Catch; waitForTaskToken; Activities (worker pattern); Pass/Task/Choice/Wait/Succeed/Fail/Map/Parallel; TestState API with mock and inspectionLevel support; SFN_MOCK_CONFIG for AWS SFN Local compatible mock testing; intrinsic functions (States.StringToJson, States.JsonToString, States.JsonMerge, States.Format); nested startExecution.sync |
-| **API Gateway v2** | CreateApi, GetApi, GetApis, UpdateApi, DeleteApi, CreateRoute, GetRoute, GetRoutes, UpdateRoute, DeleteRoute, CreateIntegration, GetIntegration, GetIntegrations, UpdateIntegration, DeleteIntegration, CreateRouteResponse, GetRouteResponse, GetRouteResponses, UpdateRouteResponse, DeleteRouteResponse, CreateIntegrationResponse, GetIntegrationResponse, GetIntegrationResponses, UpdateIntegrationResponse, DeleteIntegrationResponse, CreateStage, GetStage, GetStages, UpdateStage, DeleteStage, CreateDeployment, GetDeployment, GetDeployments, DeleteDeployment, CreateAuthorizer, GetAuthorizer, GetAuthorizers, UpdateAuthorizer, DeleteAuthorizer, TagResource, UntagResource, GetTags, PostToConnection, GetConnection, DeleteConnection | **HTTP API** and **WebSocket API** (`protocolType=WEBSOCKET`); Lambda proxy (`AWS_PROXY`), HTTP proxy (`HTTP_PROXY`), and MOCK integrations; HTTP data plane via `{apiId}.execute-api.localhost` Host header or path-based `/_aws/execute-api/{apiId}/{stage}/{path}` (no DNS/Host override needed — works from browsers on macOS and strict clients); `$default` stage served from the URL root (no stage segment in the path); per-API `corsConfiguration` applied to preflights + dispatched responses; qualified-alias integration URIs (`arn:...:function:<name>:<alias>`) resolve to the alias's target version; WebSocket data plane on the same two URL forms, with `$connect` / `$disconnect` / `$default` / custom-action routing, `$request.body.*` RouteSelectionExpression, `@connections` management API (PostToConnection / GetConnection / DeleteConnection), per-connection outbox for server-side push; `{param}` / `{proxy+}` matching; JWT/Lambda authorizer CRUD; pin `apiId` across runs with the `ms-custom-id` tag |
+| **API Gateway v2** | CreateApi, GetApi, GetApis, UpdateApi, DeleteApi, CreateRoute, GetRoute, GetRoutes, UpdateRoute, DeleteRoute, CreateIntegration, GetIntegration, GetIntegrations, UpdateIntegration, DeleteIntegration, CreateRouteResponse, GetRouteResponse, GetRouteResponses, UpdateRouteResponse, DeleteRouteResponse, CreateIntegrationResponse, GetIntegrationResponse, GetIntegrationResponses, UpdateIntegrationResponse, DeleteIntegrationResponse, CreateStage, GetStage, GetStages, UpdateStage, DeleteStage, CreateDeployment, GetDeployment, GetDeployments, DeleteDeployment, CreateAuthorizer, GetAuthorizer, GetAuthorizers, UpdateAuthorizer, DeleteAuthorizer, TagResource, UntagResource, GetTags, PostToConnection, GetConnection, DeleteConnection | **HTTP API** and **WebSocket API** (`protocolType=WEBSOCKET`); Lambda proxy (`AWS_PROXY`), HTTP proxy (`HTTP_PROXY`), and MOCK integrations; HTTP data plane via `{apiId}.execute-api.localhost` Host header or path-based `/_aws/execute-api/{apiId}/{stage}/{path}` (no DNS/Host override needed — works from browsers on macOS and strict clients); `$default` stage served from the URL root (no stage segment in the path); per-API `corsConfiguration` applied to preflights + dispatched responses; request parameter mapping for HTTP_PROXY (`append/overwrite/remove` for headers/querystring plus `overwrite:path`) with context variables including `$context.authorizer.jwt.claims`; JWT data-plane authorization for HTTP routes (issuer/audience/time/scope checks) and claim propagation to integrations; qualified-alias integration URIs (`arn:...:function:<name>:<alias>`) resolve to the alias's target version; WebSocket data plane on the same two URL forms, with `$connect` / `$disconnect` / `$default` / custom-action routing, `$request.body.*` RouteSelectionExpression, `@connections` management API (PostToConnection / GetConnection / DeleteConnection), per-connection outbox for server-side push; `{param}` / `{proxy+}` matching; JWT/Lambda authorizer CRUD; pin `apiId` across runs with the `ms-custom-id` tag |
| **API Gateway v1** | CreateRestApi, GetRestApi, GetRestApis, UpdateRestApi, DeleteRestApi, CreateResource, GetResource, GetResources, UpdateResource, DeleteResource, PutMethod, GetMethod, DeleteMethod, UpdateMethod, PutMethodResponse, GetMethodResponse, DeleteMethodResponse, PutIntegration, GetIntegration, DeleteIntegration, UpdateIntegration, PutIntegrationResponse, GetIntegrationResponse, DeleteIntegrationResponse, CreateDeployment, GetDeployment, GetDeployments, UpdateDeployment, DeleteDeployment, CreateStage, GetStage, GetStages, UpdateStage, DeleteStage, CreateAuthorizer, GetAuthorizer, GetAuthorizers, UpdateAuthorizer, DeleteAuthorizer, CreateModel, GetModel, GetModels, DeleteModel, CreateApiKey, GetApiKey, GetApiKeys, UpdateApiKey, DeleteApiKey, CreateUsagePlan, GetUsagePlan, GetUsagePlans, UpdateUsagePlan, DeleteUsagePlan, CreateUsagePlanKey, GetUsagePlanKeys, DeleteUsagePlanKey, CreateDomainName, GetDomainName, GetDomainNames, DeleteDomainName, CreateBasePathMapping, GetBasePathMapping, GetBasePathMappings, DeleteBasePathMapping, TagResource, UntagResource, GetTags | REST API (v1) protocol; Lambda proxy format 1.0 (AWS_PROXY), HTTP proxy (HTTP_PROXY), MOCK integration; data plane via `{apiId}.execute-api.localhost` Host header, path-based `/_aws/execute-api/{apiId}/{stage}/{path}`, or legacy `/restapis/{apiId}/{stage}/_user_request_/{path}`; qualified-alias integration URIs (`arn:...:function:<name>:<alias>`) resolve to the alias's target version; resource tree with `{param}` and `{proxy+}` path matching; JSON Patch for all PATCH operations; state persistence; pin `id` across runs with the `ms-custom-id` tag |
| **ELBv2 / ALB** | CreateLoadBalancer, DescribeLoadBalancers, DeleteLoadBalancer, DescribeLoadBalancerAttributes, ModifyLoadBalancerAttributes, CreateTargetGroup, DescribeTargetGroups, ModifyTargetGroup, DeleteTargetGroup, DescribeTargetGroupAttributes, ModifyTargetGroupAttributes, CreateListener, DescribeListeners, ModifyListener, DeleteListener, CreateRule, DescribeRules, ModifyRule, DeleteRule, SetRulePriorities, RegisterTargets, DeregisterTargets, DescribeTargetHealth, AddTags, RemoveTags, DescribeTags | Control plane + data plane; ALB→Lambda live traffic routing; `path-pattern`, `host-header`, `http-method`, `query-string`, `http-header` rule conditions; `forward`, `redirect`, `fixed-response` actions; data plane via `{lb-name}.alb.localhost` Host header or `/_alb/{lb-name}/` path prefix |
| **KMS** | CreateKey, ListKeys, DescribeKey, GetPublicKey, Sign, Verify, Encrypt, Decrypt, GenerateDataKey, GenerateDataKeyWithoutPlaintext, CreateAlias, DeleteAlias, ListAliases, UpdateAlias, EnableKeyRotation, DisableKeyRotation, GetKeyRotationStatus, GetKeyPolicy, PutKeyPolicy, ListKeyPolicies, EnableKey, DisableKey, ScheduleKeyDeletion, CancelKeyDeletion, TagResource, UntagResource, ListResourceTags | 27 actions; RSA (2048/4096), ECC (SECG_P256K1, NIST P-256/384/521), and symmetric keys; PKCS1v15, PSS, and ECDSA signing; envelope encryption; alias resolution; key rotation; key policies; tags; enable/disable/schedule deletion; full Terraform `aws_kms_key` compatible; `cryptography` package included in Docker image |
@@ -658,6 +658,12 @@ ecs.stop_task(cluster="dev", task=task_arn)
| `SFN_MOCK_CONFIG` | _(unset)_ | Path to JSON file for Step Functions mock testing; compatible with AWS SFN Local format. Also accepts `LOCALSTACK_SFN_MOCK_CONFIG` |
| `ATHENA_ENGINE` | `auto` | SQL engine for Athena: `auto`, `duckdb`, `mock` |
| `SMTP_HOST` | _(unset)_ | SMTP server for SES email relay (e.g. `mailhog:1025`). When set, SES SendEmail/SendRawEmail actually deliver mail. When unset, emails are stored in-memory only |
+| `MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS` | `30` | API Gateway v1/v2 HTTP `HTTP_PROXY` / `HTTP` integration: upstream request timeout (seconds) |
+| `MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS` | `5` | API Gateway v2 JWT authorizer: JWKS document fetch timeout (seconds) |
+
+### API Gateway HTTP proxy execution model
+
+API Gateway v1/v2 HTTP proxy forwarding uses non-blocking event-loop semantics by offloading the upstream socket call to a worker thread. This preserves AWS-compatible response behavior while preventing long-running proxy calls from stalling unrelated requests (for example, parallel DynamoDB operations). The same non-blocking path is used for JWT **JWKS** fetches. Tune wall-clock limits with `MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS` and `MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS` at the bottom of the [Configuration](#configuration) table above.
### Startup Scripts
diff --git a/ministack/core/nonblocking_http.py b/ministack/core/nonblocking_http.py
new file mode 100644
index 00000000..a1193827
--- /dev/null
+++ b/ministack/core/nonblocking_http.py
@@ -0,0 +1,23 @@
+import asyncio
+import os
+import urllib.request
+
+
+def timeout_from_env(env_name: str, default_seconds: float) -> float:
+ raw = os.environ.get(env_name)
+ if raw is None:
+ return float(default_seconds)
+ try:
+ parsed = float(raw)
+ except ValueError:
+ return float(default_seconds)
+ return parsed if parsed > 0 else float(default_seconds)
+
+
+def _urlopen_sync(request_or_url, timeout_seconds: float):
+ with urllib.request.urlopen(request_or_url, timeout=timeout_seconds) as resp:
+ return resp.status, dict(resp.headers.items()), resp.read()
+
+
+async def urlopen(request_or_url, timeout_seconds: float):
+ return await asyncio.to_thread(_urlopen_sync, request_or_url, timeout_seconds)
diff --git a/ministack/services/apigateway.py b/ministack/services/apigateway.py
index 86f029b4..3121b4e0 100644
--- a/ministack/services/apigateway.py
+++ b/ministack/services/apigateway.py
@@ -41,15 +41,18 @@ Data plane:
"""
import asyncio
+import base64
import json
import logging
import os
import re
import time
import urllib.error
+import urllib.parse
import urllib.request
-from ministack.core.responses import AccountScopedDict, error_response_json, get_account_id, get_region, new_uuid
+from ministack.core.nonblocking_http import timeout_from_env, urlopen as nb_urlopen
+from ministack.core.responses import AccountScopedDict, get_account_id, error_response_json, new_uuid, get_region
_HOST = os.environ.get("MINISTACK_HOST", "localhost")
_PORT = os.environ.get("GATEWAY_PORT", "4566")
@@ -57,6 +60,8 @@ _PORT = os.environ.get("GATEWAY_PORT", "4566")
logger = logging.getLogger("apigateway")
REGION = os.environ.get("MINISTACK_REGION", "us-east-1")
+_PROXY_TIMEOUT_SECONDS = timeout_from_env("MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS", 30.0)
+_JWKS_TIMEOUT_SECONDS = timeout_from_env("MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS", 5.0)
# ---- Module-level state ----
_apis = AccountScopedDict() # api_id -> api object
@@ -68,6 +73,7 @@ _authorizers = AccountScopedDict() # api_id -> {authorizer_id -> authorizer ob
_api_tags = AccountScopedDict() # resource_arn -> {key -> value}
_route_responses = AccountScopedDict() # api_id -> {route_id -> {rr_id -> route_response}}
_integration_responses = AccountScopedDict() # api_id -> {integration_id -> {ir_id -> int_response}}
+_jwks_cache: dict = {}
# WebSocket connection registry — connections are not per-account-scoped at the store level
# because the @connections management API may arrive on any host/account; instead we store
@@ -76,12 +82,48 @@ _integration_responses = AccountScopedDict() # api_id -> {integration_id -> {i
# close_event (asyncio.Event), lastActiveAt, identity} }
_ws_connections: dict = {}
+_RESERVED_HEADER_EXACT = {
+ "authorization",
+ "connection",
+ "content-encoding",
+ "content-length",
+ "content-location",
+ "forwarded",
+ "keep-alive",
+ "origin",
+ "proxy-authenticate",
+ "proxy-authorization",
+ "te",
+ "trailers",
+ "transfer-encoding",
+ "upgrade",
+ "x-forwarded-for",
+ "x-forwarded-host",
+ "x-forwarded-proto",
+ "via",
+}
+_RESERVED_HEADER_PREFIXES = ("access-control-", "apigw-", "x-amz-", "x-amzn-")
+
# ---- Response helpers ----
+def _prune_none_for_apigw_json(obj):
+ """Drop None members so the wire never emits JSON `null` (real AWS typically omits optional nulls;
+ boto3 then maps the same shape for clients as AWS). Recurses into dict values."""
+ if isinstance(obj, dict):
+ return {k: _prune_none_for_apigw_json(v) for k, v in obj.items() if v is not None}
+ if isinstance(obj, list):
+ return [_prune_none_for_apigw_json(x) for x in obj]
+ return obj
+
+
def _apigw_response(data: dict, status: int = 200) -> tuple:
"""API Gateway v2 uses application/json (not application/x-amz-json-1.0)."""
- return status, {"Content-Type": "application/json"}, json.dumps(data, ensure_ascii=False).encode("utf-8")
+ return (
+ status,
+ {"Content-Type": "application/json"},
+ json.dumps(_prune_none_for_apigw_json(data), ensure_ascii=False).encode("utf-8"),
+ )
def _apigw_error(code: str, message: str, status: int) -> tuple:
@@ -374,6 +416,314 @@ def _cors_preflight_response(cors_cfg: dict, origin: str) -> tuple:
return 204, base, b""
+def _b64url_decode(segment: str) -> bytes:
+ padded = segment + "=" * ((4 - len(segment) % 4) % 4)
+ return base64.urlsafe_b64decode(padded.encode("utf-8"))
+
+
+def _jwt_unauthorized(message: str = "Unauthorized") -> tuple:
+ return 401, {"Content-Type": "application/json"}, json.dumps({"message": message}).encode("utf-8")
+
+
+def _jwt_forbidden(message: str = "Forbidden") -> tuple:
+ return 403, {"Content-Type": "application/json"}, json.dumps({"message": message}).encode("utf-8")
+
+
+def _get_stage_variables(api_id: str, stage: str) -> dict:
+ stage_obj = _stages.get(api_id, {}).get(stage) or {}
+ return stage_obj.get("stageVariables") or {}
+
+
+def _extract_token_from_identity_source(identity_source, headers: dict, query_params: dict) -> str | None:
+ sources = identity_source if isinstance(identity_source, list) else [identity_source]
+ headers_lc = {k.lower(): v for k, v in (headers or {}).items()}
+ for src in sources:
+ if not isinstance(src, str):
+ continue
+ if src.startswith("$request.header."):
+ name = src[len("$request.header.") :].lower()
+ value = headers_lc.get(name)
+ if not value:
+ continue
+ value = value.strip()
+ if value.lower().startswith("bearer "):
+ return value.split(" ", 1)[1].strip()
+ return value
+ if src.startswith("$request.querystring."):
+ name = src[len("$request.querystring.") :]
+ vals = query_params.get(name)
+ if not vals:
+ continue
+ value = vals[-1] if isinstance(vals, list) else vals
+ if isinstance(value, str) and value.lower().startswith("bearer "):
+ return value.split(" ", 1)[1].strip()
+ return str(value)
+ return None
+
+
+def _resolve_jwks_url(authorizer: dict) -> str | None:
+ jwt_cfg = authorizer.get("jwtConfiguration") or {}
+ issuer = jwt_cfg.get("issuer") or jwt_cfg.get("Issuer")
+ if not issuer:
+ return None
+ issuer = str(issuer).rstrip("/")
+ if issuer.startswith("https://cognito-idp.") and ".amazonaws.com/" in issuer:
+ pool_id = issuer.rsplit("/", 1)[-1]
+ return f"http://{_HOST}:{_PORT}/{pool_id}/.well-known/jwks.json"
+ return f"{issuer}/.well-known/jwks.json"
+
+
+async def _fetch_jwks(url: str) -> dict:
+ cached = _jwks_cache.get(url)
+ now = time.time()
+ if cached and cached.get("expiresAt", 0) > now:
+ return cached["jwks"]
+ _, _, body = await nb_urlopen(url, _JWKS_TIMEOUT_SECONDS)
+ payload = json.loads(body or b"{}")
+ _jwks_cache[url] = {
+ "jwks": payload,
+ "expiresAt": now + 7200,
+ }
+ return payload
+
+
+def _verify_rs256_signature(token: str, jwk: dict) -> bool:
+ try:
+ from cryptography.hazmat.primitives import hashes
+ from cryptography.hazmat.primitives.asymmetric import padding, rsa
+ except Exception:
+ return False
+
+ parts = token.split(".")
+ if len(parts) != 3:
+ return False
+ try:
+ n = int.from_bytes(_b64url_decode(jwk.get("n", "")), "big")
+ e = int.from_bytes(_b64url_decode(jwk.get("e", "")), "big")
+ pub = rsa.RSAPublicNumbers(e, n).public_key()
+ signed = f"{parts[0]}.{parts[1]}".encode("utf-8")
+ signature = _b64url_decode(parts[2])
+ pub.verify(signature, signed, padding.PKCS1v15(), hashes.SHA256())
+ return True
+ except Exception:
+ return False
+
+
+def _get_claim(claims: dict, path: str):
+ """Resolve a dot-separated claim path. Matches the AWS HTTP API parameter
+ mapping rule that only `.` and `_` are supported in context variable names
+ (no bracket / quote / colon)."""
+ if not path:
+ return None
+ cur = claims
+ for part in path.split("."):
+ if isinstance(cur, dict):
+ cur = cur.get(part)
+ else:
+ return None
+ return cur
+
+
+async def _validate_jwt_authorizer(route: dict, authorizer: dict, headers: dict, query_params: dict) -> tuple[dict | None, list | None, tuple | None]:
+ token = _extract_token_from_identity_source(authorizer.get("identitySource", []), headers, query_params)
+ if not token:
+ return None, None, _jwt_unauthorized()
+
+ parts = token.split(".")
+ if len(parts) != 3:
+ return None, None, _jwt_unauthorized()
+
+ try:
+ header = json.loads(_b64url_decode(parts[0]))
+ claims = json.loads(_b64url_decode(parts[1]))
+ except Exception:
+ return None, None, _jwt_unauthorized()
+
+ kid = header.get("kid")
+ jwks_url = _resolve_jwks_url(authorizer)
+ if not kid or not jwks_url:
+ return None, None, _jwt_unauthorized()
+
+ try:
+ keys = ((await _fetch_jwks(jwks_url)).get("keys") or [])
+ except Exception:
+ return None, None, _jwt_unauthorized()
+ jwk = next((k for k in keys if k.get("kid") == kid), None)
+ if not jwk or not _verify_rs256_signature(token, jwk):
+ return None, None, _jwt_unauthorized()
+
+ now = int(time.time())
+ jwt_cfg = authorizer.get("jwtConfiguration") or {}
+ issuer = jwt_cfg.get("issuer") or jwt_cfg.get("Issuer")
+ if issuer and claims.get("iss") != issuer:
+ return None, None, _jwt_unauthorized()
+ if "exp" in claims and int(claims["exp"]) <= now:
+ return None, None, _jwt_unauthorized()
+ if "nbf" in claims and int(claims["nbf"]) > now:
+ return None, None, _jwt_unauthorized()
+ if "iat" in claims and int(claims["iat"]) > now:
+ return None, None, _jwt_unauthorized()
+
+ aud_cfg = jwt_cfg.get("audience") or jwt_cfg.get("Audience") or []
+ aud_cfg = [str(a) for a in aud_cfg]
+ if aud_cfg:
+ token_aud = claims.get("aud")
+ if token_aud is None:
+ token_aud = claims.get("client_id")
+ token_aud_values = [str(token_aud)] if token_aud is not None else []
+ elif isinstance(token_aud, list):
+ token_aud_values = [str(a) for a in token_aud]
+ else:
+ token_aud_values = [str(token_aud)]
+ if not any(a in aud_cfg for a in token_aud_values):
+ return None, None, _jwt_unauthorized()
+
+ route_scopes = route.get("authorizationScopes") or []
+ token_scopes = []
+ raw_scope = claims.get("scope")
+ raw_scp = claims.get("scp")
+ if isinstance(raw_scope, str):
+ token_scopes.extend([s for s in raw_scope.split(" ") if s])
+ if isinstance(raw_scp, list):
+ token_scopes.extend([str(s) for s in raw_scp])
+ elif isinstance(raw_scp, str):
+ token_scopes.extend([s for s in raw_scp.split(" ") if s])
+ token_scopes = sorted(set(token_scopes))
+ if route_scopes and not any(scope in token_scopes for scope in route_scopes):
+ return None, None, _jwt_forbidden()
+
+ return claims, token_scopes, None
+
+
+def _is_reserved_header(name: str) -> bool:
+ lc = (name or "").lower()
+ if lc in _RESERVED_HEADER_EXACT:
+ return True
+ return any(lc.startswith(prefix) for prefix in _RESERVED_HEADER_PREFIXES)
+
+
+def _resolve_mapping_atom(expr: str, *, request_headers: dict, request_query: dict, path_params: dict, context_vars: dict, stage_vars: dict):
+ if expr.startswith("$request.header."):
+ name = expr[len("$request.header.") :].lower()
+ return request_headers.get(name)
+ if expr.startswith("$request.querystring."):
+ name = expr[len("$request.querystring.") :]
+ vals = request_query.get(name)
+ if vals is None:
+ return None
+ return ",".join(vals) if isinstance(vals, list) else str(vals)
+ if expr.startswith("$request.path."):
+ name = expr[len("$request.path.") :]
+ return path_params.get(name)
+ if expr == "$request.path":
+ return context_vars.get("request.path")
+ if expr.startswith("$stageVariables."):
+ key = expr[len("$stageVariables.") :]
+ return stage_vars.get(key)
+ if expr.startswith("$context."):
+ path = expr[len("$context.") :]
+ # AWS HTTP API parameter mapping supports only `.` and `_` in context
+ # variable names — bracket notation is NOT supported (see
+ # https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-parameter-mapping.html).
+ # Use only dot form for JWT claim lookup, e.g.
+ # $context.authorizer.jwt.claims.sub
+ # Claim names with characters outside `.` and `_` must be read by the
+ # backend, just like real API Gateway HTTP APIs.
+ if path.startswith("authorizer.jwt.claims."):
+ claim_path = path[len("authorizer.jwt.claims.") :]
+ return _get_claim(context_vars.get("authorizer.jwt.claims") or {}, claim_path)
+ if path.startswith("authorizer.claims."):
+ claim_path = path[len("authorizer.claims.") :]
+ return _get_claim(context_vars.get("authorizer.jwt.claims") or {}, claim_path)
+ return context_vars.get(path)
+ return expr
+
+
+def _resolve_mapping_value(value, *, request_headers: dict, request_query: dict, path_params: dict, context_vars: dict, stage_vars: dict):
+ if value is None:
+ return None
+ value = str(value)
+ if "${" in value:
+ def _sub(match):
+ atom = match.group(1)
+ resolved = _resolve_mapping_atom(
+ f"${atom}",
+ request_headers=request_headers,
+ request_query=request_query,
+ path_params=path_params,
+ context_vars=context_vars,
+ stage_vars=stage_vars,
+ )
+ return "" if resolved is None else str(resolved)
+
+ return re.sub(r"\$\{([^}]+)\}", _sub, value)
+ if value.startswith("$"):
+ resolved = _resolve_mapping_atom(
+ value,
+ request_headers=request_headers,
+ request_query=request_query,
+ path_params=path_params,
+ context_vars=context_vars,
+ stage_vars=stage_vars,
+ )
+ return None if resolved is None else str(resolved)
+ return value
+
+
+def _apply_request_parameter_mappings(
+ integration: dict,
+ *,
+ request_headers: dict,
+ request_query: dict,
+ request_path: str,
+ path_params: dict,
+ context_vars: dict,
+ stage_vars: dict,
+) -> tuple[dict, dict, str]:
+ out_headers = dict(request_headers)
+ out_query = {k: list(v) if isinstance(v, list) else [str(v)] for k, v in (request_query or {}).items()}
+ out_path = request_path
+ mappings = integration.get("requestParameters") or {}
+
+ for key, value in mappings.items():
+ if ":" not in key:
+ continue
+ op, location = key.split(":", 1)
+ resolved = _resolve_mapping_value(
+ value,
+ request_headers=request_headers,
+ request_query=request_query,
+ path_params=path_params,
+ context_vars=context_vars,
+ stage_vars=stage_vars,
+ )
+ if location.startswith("header."):
+ header_name = location[len("header.") :]
+ header_lc = header_name.lower()
+ if _is_reserved_header(header_name):
+ continue
+ if op == "remove":
+ out_headers.pop(header_lc, None)
+ elif resolved is not None:
+ if op == "append" and header_lc in out_headers:
+ out_headers[header_lc] = f"{out_headers[header_lc]},{resolved}"
+ else:
+ out_headers[header_lc] = resolved
+ elif location.startswith("querystring."):
+ query_name = location[len("querystring.") :]
+ if op == "remove":
+ out_query.pop(query_name, None)
+ elif resolved is not None:
+ if op == "append":
+ out_query.setdefault(query_name, []).append(resolved)
+ else:
+ out_query[query_name] = [resolved]
+ elif location == "path" and op == "overwrite" and resolved is not None:
+ out_path = resolved if resolved.startswith("/") else "/" + resolved
+
+ return out_headers, out_query, out_path
+
+
async def handle_execute(api_id, stage, path, method, headers, body, query_params):
"""Execute an API request through a deployed API (data plane)."""
api = _apis.get(api_id)
@@ -395,6 +745,30 @@ async def handle_execute(api_id, stage, path, method, headers, body, query_param
if not route:
return 404, {"Content-Type": "application/json"}, json.dumps({"message": "No route found"}).encode()
+ request_headers = {k.lower(): v for k, v in (headers or {}).items()}
+ route_key = route.get("routeKey", "$default")
+ route_path = None
+ rk_parts = route_key.split(" ", 1)
+ if len(rk_parts) == 2:
+ route_path = rk_parts[1]
+ path_params = _extract_path_params(route_path, path) if route_path else {}
+
+ auth_type = (route.get("authorizationType") or "NONE").upper()
+ authorizer_claims = None
+ authorizer_scopes = []
+ if auth_type == "JWT":
+ authorizer_id = route.get("authorizerId")
+ if not authorizer_id:
+ return _jwt_unauthorized()
+ authorizer = _authorizers.get(api_id, {}).get(authorizer_id)
+ if not authorizer:
+ return _jwt_unauthorized()
+ claims, scopes, auth_error = await _validate_jwt_authorizer(route, authorizer, request_headers, query_params or {})
+ if auth_error:
+ return auth_error
+ authorizer_claims = claims or {}
+ authorizer_scopes = scopes or []
+
raw_target = route.get("target", "").replace("integrations/", "")
# Target is "{integrationId}" — the current Ref / CFN physical ID.
# Legacy stacks created against the broken #480 provisioner (fixed in
@@ -405,16 +779,50 @@ async def handle_execute(api_id, stage, path, method, headers, body, query_param
return 500, {"Content-Type": "application/json"}, json.dumps({"message": "No integration configured"}).encode()
integration_type = integration.get("integrationType", "")
+ stage_vars = _get_stage_variables(api_id, stage)
+ context_vars = {
+ "requestId": new_uuid(),
+ "httpMethod": method,
+ "path": f"/{stage}{path}",
+ "routeKey": route_key,
+ "stage": stage,
+ "authorizer.jwt.claims": authorizer_claims or {},
+ "authorizer.jwt.scopes": authorizer_scopes,
+ }
if integration_type == "AWS_PROXY":
- route_key = route.get("routeKey", "$default")
- path_params = None
- rk_parts = route_key.split(" ", 1)
- if len(rk_parts) == 2:
- path_params = _extract_path_params(rk_parts[1], path) or None
- response = await _invoke_lambda_proxy(integration, api_id, stage, path, method, headers, body, query_params, route_key, path_params)
+ response = await _invoke_lambda_proxy(
+ integration,
+ api_id,
+ stage,
+ path,
+ method,
+ request_headers,
+ body,
+ query_params,
+ route_key,
+ (path_params or None),
+ authorizer_claims=authorizer_claims,
+ authorizer_scopes=authorizer_scopes,
+ )
elif integration_type == "HTTP_PROXY":
- response = await _invoke_http_proxy(integration, path, method, headers, body, query_params)
+ mapped_headers, mapped_query, mapped_path = _apply_request_parameter_mappings(
+ integration,
+ request_headers=request_headers,
+ request_query=query_params or {},
+ request_path=path,
+ path_params=path_params or {},
+ context_vars=context_vars,
+ stage_vars=stage_vars,
+ )
+ response = await _invoke_http_proxy(
+ integration,
+ mapped_path,
+ method,
+ mapped_headers,
+ body,
+ mapped_query,
+ )
else:
return 500, {"Content-Type": "application/json"}, json.dumps({"message": f"Unsupported integration type: {integration_type}"}).encode()
@@ -481,7 +889,21 @@ def _path_matches(route_path: str, request_path: str) -> bool:
return _extract_path_params(route_path, request_path) is not None
-async def _invoke_lambda_proxy(integration, api_id, stage, path, method, headers, body, query_params, route_key="$default", path_params=None):
+async def _invoke_lambda_proxy(
+ integration,
+ api_id,
+ stage,
+ path,
+ method,
+ headers,
+ body,
+ query_params,
+ route_key="$default",
+ path_params=None,
+ *,
+ authorizer_claims=None,
+ authorizer_scopes=None,
+):
"""Invoke a Lambda function using the API Gateway v2 proxy event format."""
from ministack.services import lambda_svc
@@ -531,6 +953,13 @@ async def _invoke_lambda_proxy(integration, api_id, stage, path, method, headers
"body": body.decode("utf-8", errors="replace") if body else None,
"isBase64Encoded": False,
}
+ if authorizer_claims is not None:
+ event["requestContext"]["authorizer"] = {
+ "jwt": {
+ "claims": authorizer_claims or {},
+ "scopes": authorizer_scopes or [],
+ }
+ }
# Route through the central _execute_function dispatcher so CloudWatch
# Logs emission and Docker log output work for API Gateway invocations.
@@ -557,16 +986,26 @@ async def _invoke_http_proxy(integration, path, method, headers, body, query_par
"""Forward a request to an HTTP backend."""
uri = integration.get("integrationUri", "")
url = uri.rstrip("/") + path
+ if query_params:
+ pairs = []
+ for key, vals in query_params.items():
+ if isinstance(vals, list):
+ for val in vals:
+ pairs.append((key, val))
+ else:
+ pairs.append((key, vals))
+ qs = urllib.parse.urlencode(pairs, doseq=True)
+ if qs:
+ url = f"{url}?{qs}"
req = urllib.request.Request(url, data=body or None, method=method)
for k, v in headers.items():
if k.lower() not in ("host", "content-length"):
req.add_header(k, v)
try:
- with urllib.request.urlopen(req, timeout=30) as resp:
- resp_body = resp.read()
- resp_headers = {"Content-Type": resp.headers.get("Content-Type", "application/json")}
- return resp.status, resp_headers, resp_body
+ status, resp_headers_raw, resp_body = await nb_urlopen(req, _PROXY_TIMEOUT_SECONDS)
+ resp_headers = {"Content-Type": resp_headers_raw.get("Content-Type", "application/json")}
+ return status, resp_headers, resp_body
except urllib.error.HTTPError as e:
return e.code, {"Content-Type": "application/json"}, e.read()
except Exception as ex:
@@ -684,9 +1123,15 @@ def _create_route(api_id, data):
"authorizationType": data.get("authorizationType", "NONE"),
"apiKeyRequired": data.get("apiKeyRequired", False),
"operationName": data.get("operationName", ""),
- "requestModels": data.get("requestModels", {}),
- "requestParameters": data.get("requestParameters", {}),
}
+ if data.get("authorizerId"):
+ route["authorizerId"] = data["authorizerId"]
+ if data.get("authorizationScopes") is not None:
+ route["authorizationScopes"] = list(data["authorizationScopes"] or [])
+ if data.get("requestModels"):
+ route["requestModels"] = data["requestModels"]
+ if data.get("requestParameters"):
+ route["requestParameters"] = data["requestParameters"]
_routes.setdefault(api_id, {})[route_id] = route
return _apigw_response(route, 201)
@@ -706,9 +1151,38 @@ def _update_route(api_id, route_id, data):
route = _routes.get(api_id, {}).get(route_id)
if not route:
return _apigw_error("NotFoundException", f"Route {route_id} not found", 404)
- for k in ("routeKey", "target", "authorizationType", "apiKeyRequired", "operationName"):
- if k in data:
- route[k] = data[k]
+ for k in (
+ "routeKey",
+ "target",
+ "authorizationType",
+ "authorizerId",
+ "authorizationScopes",
+ "apiKeyRequired",
+ "operationName",
+ ):
+ if k not in data:
+ continue
+ v = data[k]
+ if k == "authorizerId":
+ if v:
+ route["authorizerId"] = v
+ else:
+ route.pop("authorizerId", None)
+ elif k == "authorizationScopes":
+ if v is not None:
+ route["authorizationScopes"] = list(v)
+ else:
+ route.pop("authorizationScopes", None)
+ else:
+ route[k] = v
+ for rk in ("requestModels", "requestParameters"):
+ if rk not in data:
+ continue
+ v = data.get(rk)
+ if v:
+ route[rk] = v
+ else:
+ route.pop(rk, None)
return _apigw_response(route)
@@ -732,14 +1206,18 @@ def _create_integration(api_id, data):
"timeoutInMillis": data.get("timeoutInMillis", 30000),
"connectionType": data.get("connectionType", "INTERNET"),
"description": data.get("description", ""),
- "requestParameters": data.get("requestParameters", {}),
- "requestTemplates": data.get("requestTemplates", {}),
- "responseParameters": data.get("responseParameters", {}),
- # #439: contentHandlingStrategy (CONVERT_TO_TEXT | CONVERT_TO_BINARY)
- # is accepted on Create/Update and echoed back by Get; Terraform's
- # aws_apigatewayv2_integration otherwise plans to re-add it on every apply.
- "contentHandlingStrategy": data.get("contentHandlingStrategy"),
}
+ if data.get("requestParameters"):
+ integration["requestParameters"] = data["requestParameters"]
+ if data.get("requestTemplates"):
+ integration["requestTemplates"] = data["requestTemplates"]
+ if data.get("responseParameters"):
+ integration["responseParameters"] = data["responseParameters"]
+ # #439: contentHandlingStrategy (CONVERT_TO_TEXT | CONVERT_TO_BINARY)
+ # is accepted on Create/Update and echoed back by Get; Terraform's
+ # aws_apigatewayv2_integration otherwise plans to re-add it on every apply.
+ if data.get("contentHandlingStrategy") is not None:
+ integration["contentHandlingStrategy"] = data["contentHandlingStrategy"]
_integrations.setdefault(api_id, {})[int_id] = integration
return _apigw_response(integration, 201)
@@ -763,8 +1241,21 @@ def _update_integration(api_id, int_id, data):
"payloadFormatVersion", "timeoutInMillis", "connectionType",
"description", "requestParameters", "requestTemplates", "responseParameters",
"contentHandlingStrategy"):
- if k in data:
- integration[k] = data[k]
+ if k not in data:
+ continue
+ v = data[k]
+ if k in ("requestParameters", "requestTemplates", "responseParameters"):
+ if v:
+ integration[k] = v
+ else:
+ integration.pop(k, None)
+ elif k == "contentHandlingStrategy":
+ if v is not None:
+ integration[k] = v
+ else:
+ integration.pop("contentHandlingStrategy", None)
+ else:
+ integration[k] = v
return _apigw_response(integration)
diff --git a/ministack/services/apigateway_v1.py b/ministack/services/apigateway_v1.py
index f0303892..cf96f8f5 100644
--- a/ministack/services/apigateway_v1.py
+++ b/ministack/services/apigateway_v1.py
@@ -77,7 +77,8 @@ import time
import urllib.error
import urllib.request
-from ministack.core.responses import AccountScopedDict, get_account_id, get_region, new_uuid
+from ministack.core.nonblocking_http import timeout_from_env, urlopen as nb_urlopen
+from ministack.core.responses import AccountScopedDict, get_account_id, new_uuid, get_region
def _now_unix():
@@ -87,6 +88,7 @@ def _now_unix():
return int(time.time())
logger = logging.getLogger("apigateway_v1")
+_PROXY_TIMEOUT_SECONDS = timeout_from_env("MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS", 30.0)
REGION = os.environ.get("MINISTACK_REGION", "us-east-1")
@@ -918,10 +920,9 @@ async def _invoke_http_proxy_v1(integration, path, method, headers, body, query_
if k.lower() not in ("host", "content-length"):
req.add_header(k, v)
try:
- with urllib.request.urlopen(req, timeout=30) as resp:
- resp_body = resp.read()
- resp_headers = {"Content-Type": resp.headers.get("Content-Type", "application/json")}
- return resp.status, resp_headers, resp_body
+ status, resp_headers_raw, resp_body = await nb_urlopen(req, _PROXY_TIMEOUT_SECONDS)
+ resp_headers = {"Content-Type": resp_headers_raw.get("Content-Type", "application/json")}
+ return status, resp_headers, resp_body
except urllib.error.HTTPError as e:
return e.code, {"Content-Type": "application/json"}, e.read()
except Exception as ex:
diff --git a/ministack/services/cognito.py b/ministack/services/cognito.py
index 6b81c504..49c42b6c 100644
--- a/ministack/services/cognito.py
+++ b/ministack/services/cognito.py
@@ -87,11 +87,39 @@ logger = logging.getLogger("cognito")
_RSA_PRIVATE_KEY = None
_JWKS_KEY: dict = {}
+# Real Cognito keeps its signing key stable across restarts and exposes the
+# matching public key on the JWKS endpoint. Without persistence, every Python
+# process (server, tests, helper scripts) would generate a fresh RSA key and
+# tokens minted in one process would fail to verify in another. Persist the
+# key to STATE_DIR so all processes share the same key.
+_STATE_DIR = os.environ.get("STATE_DIR", "/tmp/ministack-state")
+_RSA_KEY_PATH = os.path.join(_STATE_DIR, "cognito-rsa-key.pem")
+
try:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import rsa
- _rsa_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
+ _rsa_key = None
+ if os.path.exists(_RSA_KEY_PATH):
+ try:
+ with open(_RSA_KEY_PATH, "rb") as _f:
+ _rsa_key = serialization.load_pem_private_key(_f.read(), password=None)
+ except Exception:
+ _rsa_key = None
+ if _rsa_key is None:
+ _rsa_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
+ try:
+ os.makedirs(_STATE_DIR, exist_ok=True)
+ _pem = _rsa_key.private_bytes(
+ encoding=serialization.Encoding.PEM,
+ format=serialization.PrivateFormat.PKCS8,
+ encryption_algorithm=serialization.NoEncryption(),
+ )
+ with open(_RSA_KEY_PATH, "wb") as _f:
+ _f.write(_pem)
+ except Exception:
+ # Persistence is best-effort — if it fails, fall back to in-memory key.
+ pass
_RSA_PRIVATE_KEY = _rsa_key
_pub = _rsa_key.public_key()
diff --git a/tests/test_apigatewayv1.py b/tests/test_apigatewayv1.py
index 941eb6de..dffb9f97 100644
--- a/tests/test_apigatewayv1.py
+++ b/tests/test_apigatewayv1.py
@@ -679,6 +679,61 @@ def test_apigwv1_execute_missing_resource_404(apigw_v1):
apigw_v1.delete_rest_api(restApiId=api_id)
+
+def test_apigwv1_http_proxy_does_not_block_parallel_ddb(monkeypatch):
+ import asyncio
+ from ministack.core import nonblocking_http
+ from ministack.services import apigateway_v1 as apigw_v1_mod
+ from ministack.services import dynamodb as ddb_mod
+
+ def _slow_urlopen(_request_or_url, _timeout_seconds):
+ time.sleep(0.4)
+ return 200, {"Content-Type": "application/json"}, b"{}"
+
+ monkeypatch.setattr(nonblocking_http, "_urlopen_sync", _slow_urlopen)
+
+ async def _run():
+ slow_call = asyncio.create_task(
+ apigw_v1_mod._invoke_http_proxy_v1(
+ {"uri": "http://example.test"},
+ "/slow",
+ "GET",
+ {},
+ None,
+ {},
+ )
+ )
+ await asyncio.sleep(0.05)
+ started = time.perf_counter()
+ status, _, _ = await ddb_mod.handle_request(
+ "POST",
+ "/",
+ {"x-amz-target": "DynamoDB_20120810.ListTables"},
+ b"{}",
+ {},
+ )
+ elapsed = time.perf_counter() - started
+ await slow_call
+ return status, elapsed
+
+ status, elapsed = asyncio.run(_run())
+ assert status == 200
+ assert elapsed < 0.2, f"Parallel DDB request was delayed for {elapsed:.2f}s"
+
+
+def test_apigwv1_http_proxy_timeout_is_configurable(monkeypatch):
+ import importlib
+ from ministack.services import apigateway_v1 as apigateway_v1_module
+
+ monkeypatch.setenv("MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS", "55")
+ reloaded = importlib.reload(apigateway_v1_module)
+ try:
+ assert reloaded._PROXY_TIMEOUT_SECONDS == 55.0
+ finally:
+ monkeypatch.delenv("MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS", raising=False)
+ importlib.reload(reloaded)
+
+
def test_apigwv1_no_conflict_with_v2(apigw_v1, apigw, lam):
"""v1 and v2 APIs can coexist; execute-api routes them independently."""
import urllib.request as _urlreq
diff --git a/tests/test_apigatewayv2.py b/tests/test_apigatewayv2.py
index fe79d67c..fec8e9e1 100644
--- a/tests/test_apigatewayv2.py
+++ b/tests/test_apigatewayv2.py
@@ -1,8 +1,16 @@
"""API Gateway v2 tests — HTTP API + WebSocket + common lifecycle."""
+import base64
import io
import json
import os
+import pytest
+import threading
+import time
+import uuid as _uuid_mod
+import zipfile
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+from botocore.exceptions import ClientError
import time
import uuid as _uuid_mod
import zipfile
@@ -29,6 +37,51 @@ def _make_zip_js(code: str, filename: str = "index.js") -> bytes:
_LAMBDA_ROLE = "arn:aws:iam::000000000000:role/lambda-role"
+
+def _start_echo_server():
+ captured = {"headers": {}, "path": "", "query": {}}
+
+ class _Handler(BaseHTTPRequestHandler):
+ def do_GET(self):
+ from urllib.parse import parse_qs, urlparse
+
+ parsed = urlparse(self.path)
+ captured["path"] = parsed.path
+ captured["query"] = parse_qs(parsed.query, keep_blank_values=True)
+ captured["headers"] = {k.lower(): v for k, v in self.headers.items()}
+ body = json.dumps(captured).encode("utf-8")
+ self.send_response(200)
+ self.send_header("Content-Type", "application/json")
+ self.send_header("Content-Length", str(len(body)))
+ self.end_headers()
+ self.wfile.write(body)
+
+ def log_message(self, _format, *_args):
+ return
+
+ server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
+ thread = threading.Thread(target=server.serve_forever, daemon=True)
+ thread.start()
+ return server, thread, captured
+
+
+def _make_signed_token(claims: dict) -> str:
+ from ministack.services import cognito as _cognito
+
+ if _cognito._RSA_PRIVATE_KEY is None:
+ pytest.skip("cryptography-backed Cognito signing key unavailable")
+
+ from cryptography.hazmat.primitives import hashes
+ from cryptography.hazmat.primitives.asymmetric import padding
+
+ header = {"alg": "RS256", "kid": "ministack-key-1"}
+ h = base64.urlsafe_b64encode(json.dumps(header).encode()).rstrip(b"=").decode()
+ p = base64.urlsafe_b64encode(json.dumps(claims).encode()).rstrip(b"=").decode()
+ signed = f"{h}.{p}".encode()
+ sig = _cognito._RSA_PRIVATE_KEY.sign(signed, padding.PKCS1v15(), hashes.SHA256())
+ s = base64.urlsafe_b64encode(sig).rstrip(b"=").decode()
+ return f"{h}.{p}.{s}"
+
def test_apigw_create_api(apigw):
resp = apigw.create_api(Name="test-api", ProtocolType="HTTP")
assert "ApiId" in resp
@@ -735,6 +788,232 @@ def test_apigw_authorizer_crud(apigw):
apigw.delete_api(ApiId=api_id)
+
+def test_apigw_route_persists_authorizer_fields(apigw):
+ api_id = apigw.create_api(Name=f"route-authz-{_uuid_mod.uuid4().hex[:8]}", ProtocolType="HTTP")["ApiId"]
+ auth_id = apigw.create_authorizer(
+ ApiId=api_id,
+ AuthorizerType="JWT",
+ Name="route-authz-jwt",
+ IdentitySource=["$request.header.Authorization"],
+ JwtConfiguration={"Audience": ["ms-client"], "Issuer": "https://cognito-idp.us-east-1.amazonaws.com/test-pool"},
+ )["AuthorizerId"]
+ route_id = apigw.create_route(
+ ApiId=api_id,
+ RouteKey="GET /secure",
+ AuthorizationType="JWT",
+ AuthorizerId=auth_id,
+ AuthorizationScopes=["scope:messages"],
+ )["RouteId"]
+ route = apigw.get_route(ApiId=api_id, RouteId=route_id)
+ assert route["AuthorizationType"] == "JWT"
+ # AWS omits optional fields when unset; here they are set, so they must be echoed.
+ # Use .get() so the assertion failure mode is "wrong value" rather than KeyError.
+ assert route.get("AuthorizerId") == auth_id
+ assert route.get("AuthorizationScopes") == ["scope:messages"]
+ apigw.delete_api(ApiId=api_id)
+
+
+def test_apigw_route_without_authorizer_optional_fields_open_route(apigw):
+ """An open route (no authorizer) must not leak `AuthorizerId` / `AuthorizationScopes`
+ on the wire. Real AWS omits unset optional fields; boto3 then omits them from the
+ deserialized response. This guards against regressions where defaults like `""` or
+ `[]` are stored on the route and surface as falsy-but-present keys."""
+ api_id = apigw.create_api(Name=f"route-open-{_uuid_mod.uuid4().hex[:8]}", ProtocolType="HTTP")["ApiId"]
+ route_id = apigw.create_route(ApiId=api_id, RouteKey="GET /open")["RouteId"]
+ route = apigw.get_route(ApiId=api_id, RouteId=route_id)
+ assert route.get("AuthorizerId") in (None, "")
+ assert route.get("AuthorizationScopes") in (None, [])
+ apigw.delete_api(ApiId=api_id)
+
+
+def test_apigw_integration_without_request_parameters_optional_map(apigw):
+ """An integration without RequestParameters must not echo `RequestParameters: {}`
+ or `null`. Real AWS omits the key when unset; the conditional-storage logic in
+ `_create_integration` should leave the key absent from the stored dict."""
+ api_id = apigw.create_api(Name=f"int-noparams-{_uuid_mod.uuid4().hex[:8]}", ProtocolType="HTTP")["ApiId"]
+ int_id = apigw.create_integration(
+ ApiId=api_id,
+ IntegrationType="AWS_PROXY",
+ IntegrationUri="arn:aws:lambda:us-east-1:000000000000:function:noop",
+ PayloadFormatVersion="2.0",
+ )["IntegrationId"]
+ integ = apigw.get_integration(ApiId=api_id, IntegrationId=int_id)
+ assert integ.get("RequestParameters") in (None, {})
+ apigw.delete_api(ApiId=api_id)
+
+
+def test_apigw_jwt_authorizer_enforced_in_data_plane(apigw):
+ import urllib.error as _urlerr
+ import urllib.request as _urlreq
+ from ministack.services import cognito as _cognito
+
+ api_id = apigw.create_api(Name=f"jwt-enforce-{_uuid_mod.uuid4().hex[:8]}", ProtocolType="HTTP")["ApiId"]
+ server, _thread, _captured = _start_echo_server()
+ try:
+ integ_id = apigw.create_integration(
+ ApiId=api_id,
+ IntegrationType="HTTP_PROXY",
+ IntegrationUri=f"http://127.0.0.1:{server.server_port}",
+ IntegrationMethod="GET",
+ )["IntegrationId"]
+ auth_id = apigw.create_authorizer(
+ ApiId=api_id,
+ AuthorizerType="JWT",
+ Name="jwt-auth",
+ IdentitySource=["$request.header.Authorization"],
+ JwtConfiguration={"Audience": ["ms-client"], "Issuer": "https://cognito-idp.us-east-1.amazonaws.com/test-pool"},
+ )["AuthorizerId"]
+ apigw.create_route(
+ ApiId=api_id,
+ RouteKey="GET /secure",
+ Target=f"integrations/{integ_id}",
+ AuthorizationType="JWT",
+ AuthorizerId=auth_id,
+ )
+ apigw.create_stage(ApiId=api_id, StageName="$default")
+
+ url = f"http://{api_id}.execute-api.localhost:{_EXECUTE_PORT}/$default/secure"
+ req_missing = _urlreq.Request(url, method="GET")
+ req_missing.add_header("Host", f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}")
+ with pytest.raises(_urlerr.HTTPError) as exc:
+ _urlreq.urlopen(req_missing)
+ assert exc.value.code == 401
+
+ token = _cognito._fake_token("user-1", "test-pool", "ms-client", token_type="access")
+ req_ok = _urlreq.Request(url, method="GET")
+ req_ok.add_header("Host", f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}")
+ req_ok.add_header("Authorization", f"Bearer {token}")
+ resp_ok = _urlreq.urlopen(req_ok)
+ assert resp_ok.status == 200
+ finally:
+ server.shutdown()
+ server.server_close()
+ apigw.delete_api(ApiId=api_id)
+
+
+def test_apigw_request_mapping_claims_to_headers(apigw):
+ import urllib.request as _urlreq
+
+ api_id = apigw.create_api(Name=f"jwt-map-{_uuid_mod.uuid4().hex[:8]}", ProtocolType="HTTP")["ApiId"]
+ server, _thread, _captured = _start_echo_server()
+ try:
+ integ_id = apigw.create_integration(
+ ApiId=api_id,
+ IntegrationType="HTTP_PROXY",
+ IntegrationUri=f"http://127.0.0.1:{server.server_port}",
+ IntegrationMethod="GET",
+ RequestParameters={
+ "overwrite:header.X-User-Id": "$context.authorizer.jwt.claims.sub",
+ "overwrite:header.X-Account-Id": "$context.authorizer.jwt.claims.account_id",
+ "overwrite:path": "/backend/messages",
+ "append:querystring.tag": "mapped",
+ },
+ )["IntegrationId"]
+ auth_id = apigw.create_authorizer(
+ ApiId=api_id,
+ AuthorizerType="JWT",
+ Name="jwt-map-auth",
+ IdentitySource=["$request.header.Authorization"],
+ JwtConfiguration={"Audience": ["ms-client"], "Issuer": "https://cognito-idp.us-east-1.amazonaws.com/test-pool"},
+ )["AuthorizerId"]
+ apigw.create_route(
+ ApiId=api_id,
+ RouteKey="GET /secure",
+ Target=f"integrations/{integ_id}",
+ AuthorizationType="JWT",
+ AuthorizerId=auth_id,
+ )
+ apigw.create_stage(ApiId=api_id, StageName="$default")
+
+ now = int(time.time())
+ token = _make_signed_token(
+ {
+ "sub": "auth0|abc123",
+ "iss": "https://cognito-idp.us-east-1.amazonaws.com/test-pool",
+ "aud": "ms-client",
+ "iat": now,
+ "nbf": now - 1,
+ "exp": now + 3600,
+ "scope": "scope:messages",
+ "account_id": "acc-777",
+ }
+ )
+
+ url = f"http://{api_id}.execute-api.localhost:{_EXECUTE_PORT}/$default/secure?tag=orig"
+ req = _urlreq.Request(url, method="GET")
+ req.add_header("Host", f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}")
+ req.add_header("Authorization", f"Bearer {token}")
+ resp = _urlreq.urlopen(req)
+ assert resp.status == 200
+ body = json.loads(resp.read())
+ assert body["path"] == "/backend/messages"
+ assert body["headers"]["x-user-id"] == "auth0|abc123"
+ assert body["headers"]["x-account-id"] == "acc-777"
+ assert body["query"]["tag"] == ["orig", "mapped"]
+ finally:
+ server.shutdown()
+ server.server_close()
+ apigw.delete_api(ApiId=api_id)
+
+
+def test_apigw_http_proxy_does_not_block_parallel_ddb(monkeypatch):
+ import asyncio
+ from ministack.core import nonblocking_http
+ from ministack.services import apigateway as apigw_mod
+ from ministack.services import dynamodb as ddb_mod
+
+ def _slow_urlopen(_request_or_url, _timeout_seconds):
+ time.sleep(0.4)
+ return 200, {"Content-Type": "application/json"}, b"{}"
+
+ monkeypatch.setattr(nonblocking_http, "_urlopen_sync", _slow_urlopen)
+
+ async def _run():
+ slow_call = asyncio.create_task(
+ apigw_mod._invoke_http_proxy(
+ {"integrationUri": "http://example.test"},
+ "/slow",
+ "GET",
+ {},
+ None,
+ {},
+ )
+ )
+ await asyncio.sleep(0.05)
+ started = time.perf_counter()
+ status, _, _ = await ddb_mod.handle_request(
+ "POST",
+ "/",
+ {"x-amz-target": "DynamoDB_20120810.ListTables"},
+ b"{}",
+ {},
+ )
+ elapsed = time.perf_counter() - started
+ await slow_call
+ return status, elapsed
+
+ status, elapsed = asyncio.run(_run())
+ assert status == 200
+ assert elapsed < 0.2, f"Parallel DDB request was delayed for {elapsed:.2f}s"
+
+
+def test_apigw_http_proxy_timeout_is_configurable(monkeypatch):
+ import importlib
+ from ministack.services import apigateway as apigateway_module
+
+ monkeypatch.setenv("MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS", "41")
+ monkeypatch.setenv("MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS", "9")
+ reloaded = importlib.reload(apigateway_module)
+ try:
+ assert reloaded._PROXY_TIMEOUT_SECONDS == 41.0
+ assert reloaded._JWKS_TIMEOUT_SECONDS == 9.0
+ finally:
+ monkeypatch.delenv("MINISTACK_APIGW_PROXY_TIMEOUT_SECONDS", raising=False)
+ monkeypatch.delenv("MINISTACK_APIGW_JWKS_TIMEOUT_SECONDS", raising=False)
+ importlib.reload(reloaded)
+
+
def test_apigw_routekey_in_lambda_event(apigw, lam):
"""routeKey in Lambda event should reflect the matched route, not hardcoded $default."""
import urllib.request as _urlreq