Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(alb, cloudfront): forward requests as sent, return only their own headers (#2417)

Jeff Heller7dcf4a22dd20
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 3eda48fc..ccd96918 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,23 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Fixed
+- **ALB — instance and IP targets receive the path as the client sent it** — the data plane forwarded the decoded path, so `%20` became a space and the request was refused with 502 (`URL can't contain control characters`), and `%2F` became `/`. The path now reaches the target with its percent-encoding, as AWS sends it; rules still match the decoded path. Contributed by @skialpine.
+- **ALB — `authenticate-oidc` on a 443 listener names its callback without the client's port** — the `redirect_uri` kept a port the client sent in `Host`. On a 443 listener it is now `https://<host>/oauth2/idpresponse`, as AWS sends it. Contributed by @skialpine.
+- **ALB — a redirect action's `Location` fills its placeholders as AWS does** — `#{protocol}` was always `http`, `#{port}` was dropped and `#{query}` was not applied, so the request's query string was lost. `#{protocol}` and `#{port}` are now the listener's, the port is always written (`http://host:80/path`), and the default `#{query}` keeps the request's query string; a configured `Query`, which was not stored, now applies and is described. Contributed by @skialpine.
+- **ALB — a Lambda target's event has the load balancer's shape** — it carried `queryStringParameters` with the first value of a repeated key, URL-decoded, plus `multiValueQueryStringParameters` and `multiValueHeaders`, and `body: null` without a body. With multi-value headers off (the default) it now carries `queryStringParameters` as the load balancer sends them, the last value of a repeated key and not decoded, no multi-value keys, and `body: ""`. Contributed by @skialpine.
+- **HTTP/1.1 status lines carry their reason phrase** — final responses were written as `HTTP/1.1 200 `, so a CloudFront Function saw an empty `statusDescription`. They now read `HTTP/1.1 200 OK`, `404 Not Found`. Contributed by @skialpine.
+- **Responses carry one `Date`** — a proxied response carried the target's `Date` and a second one from Hypercorn. The app now adds `Date` to a response that has none, including a WebSocket handshake, and Hypercorn adds none; an ALB writes its own `Date` in place of an instance or IP target's, as AWS does, and CloudFront keeps the origin's. Hypercorn's own error responses, such as a malformed request's 400 and a WebSocket handshake refused with `websocket.close`, carry one `Date` too. Contributed by @skialpine.
+- **CloudFront — `X-Amz-Cf-Id` and `Via` have CloudFront's shape** — the data plane sent 56 characters of standard base64 without padding as `X-Amz-Cf-Id` and a 13-character token in `Via`. It now sends URL-safe base64 of 40 bytes with its `==` padding, and `Via: 1.1 <32 hex>.cloudfront.net (CloudFront)`. Contributed by @skialpine.
+- **ALB — the listener's CORS response headers are applied** — `routing.http.response.access_control_*.header_value` set with `ModifyListenerAttributes` had no effect. Each non-empty value is now added to the listener's responses, replacing the target's header of that name. Contributed by @skialpine.
+- **ALB and CloudFront responses carry only their own headers** — the gateway added `x-amzn-requestid` and `x-amz-request-id` to both and `Access-Control-Allow-Origin: *` to ALB responses. They now carry what the target or origin sent, plus the load balancer's or CloudFront's own: `Server: awselb/2.0` on a response the load balancer makes itself (a Lambda target's, fixed-response, redirect, authentication, its own errors), and `Server: CloudFront` when the origin sends none. Contributed by @skialpine.
+- **Responses carry no Hypercorn `Server` header** — every response carried `Server: hypercorn-h11`, which no AWS endpoint sends. Hypercorn no longer adds it; services that send their own `Server` are unchanged. Contributed by @skialpine.
+- **ALB — `Accept-Encoding` reaches HTTP targets** — the data plane dropped the client's `Accept-Encoding`, so a target never compressed its response. It is now forwarded, and the target's encoded response is passed through unchanged. Contributed by @skialpine.
+- **API Gateway — an HTTP API's `rawQueryString` is the query string as the client sent it** — a bare key gained an `=` (`?flag` reached the function as `flag=`). Both the Lambda proxy event and the REQUEST authorizer event now carry the request's own query string. Contributed by @skialpine.
+- **ALB — an OPTIONS request reaches the target** — the gateway answered every preflight to a load balancer address with its own wildcard CORS response. The request is now forwarded to the target like any other method. Contributed by @skialpine.
+- **ALB — the query string reaches HTTP targets unchanged** — it was rebuilt from the first value of each key, so `?page=1&page=2` arrived as `?page=1`. The request's own query string is now forwarded, including from a CloudFront distribution whose origin is the load balancer. Contributed by @skialpine.
+- **ALB — instance and IP targets receive the client's `Host`** — they got their own address as `Host`. They now get the client's, as an Application Load Balancer sends it: on a listener on port 80 or 443 without a port, on any other listener with the listener's port in place of any port the client sent; `routing.http.preserve_host_header.enabled` sends it unmodified. Lambda targets keep the `Host` exactly as sent, and authentication uses the client's `Host`. Contributed by @skialpine.
+
## [1.5.32] — 2026-10-23
### Added
diff --git a/ministack/app.py b/ministack/app.py
index fb4d95fd..e5a1ffbb 100644
--- a/ministack/app.py
+++ b/ministack/app.py
@@ -24,6 +24,7 @@ import sys
import tempfile
import time
import uuid
+from email.utils import formatdate
from urllib.parse import parse_qs, quote, unquote, urlencode
_MINISTACK_HOST = os.environ.get("MINISTACK_HOST", "localhost")
@@ -245,6 +246,40 @@ from ministack.core.sigv4 import decode_header_value
# See ministack/core/hypercorn_compat.py for the rationale (issue #389).
_install_hypercorn_compat()
+# AWS endpoints never answer with Hypercorn's Server header; a service that sends one sets its own.
+# Set on the class so every start path (ministack, ministack -d, python -m hypercorn) gets it.
+from hypercorn.config import Config as _HypercornConfig
+
+_HypercornConfig.include_server_header = False
+# Hypercorn appends its own Date even when the response has one; the app adds one when it is missing.
+_HypercornConfig.include_date_header = False
+
+import h11 as _h11
+from hypercorn.protocol.events import EndBody as _EndBody
+from hypercorn.protocol.events import Response as _StreamResponse
+from hypercorn.protocol.h11 import H11Protocol as _H11Protocol
+from hypercorn.protocol.ws_stream import WSStream as _WSStream
+
+
+async def _send_error_response(self, status_code: int) -> None:
+ """Hypercorn's own error response (a malformed request's 400), with the Date its config no longer adds."""
+ await self._send_h11_event(_h11.Response(status_code=status_code, headers=[
+ (b"content-length", b"0"), (b"connection", b"close"),
+ (b"date", formatdate(usegmt=True).encode()), *self.config.response_headers("h11")]))
+ await self._send_h11_event(_h11.EndOfMessage())
+
+
+async def _send_ws_error_response(self, status_code: int) -> None:
+ """Hypercorn's own WebSocket handshake refusal (websocket.close before accept), with its Date."""
+ await self.send(_StreamResponse(stream_id=self.stream_id, status_code=status_code, headers=[
+ (b"content-length", b"0"), (b"connection", b"close"), (b"date", formatdate(usegmt=True).encode())]))
+ await self.send(_EndBody(stream_id=self.stream_id))
+ await self.config.log.access(self.scope, {"status": status_code, "headers": []}, time.time() - self.start_time)
+
+
+_H11Protocol._send_error_response = _send_error_response
+_WSStream._send_error_response = _send_ws_error_response
+
# ---------------------------------------------------------------------------
# Lazy service loader — modules are imported on first request, not at startup.
# This saves ~20 MB of idle RAM and speeds up boot.
@@ -1287,14 +1322,15 @@ async def _handle_pre_body_request(method: str, path: str, headers: dict, query_
"""Handle fast-path routes that do not require request body parsing."""
# OPTIONS on an execute-api host / path MUST flow through apigateway.handle_execute
# so the API's own corsConfiguration is applied (#406). A Function URL owns its
- # CORS config the same way, as does a registered custom domain. Skip the generic
- # wildcard preflight in all three cases.
+ # CORS config the same way, as does a registered custom domain. An OPTIONS
+ # request to a load balancer goes on to its target. Skip the generic wildcard preflight for each.
host = headers.get("host", "")
owns_cors = (
_parse_execute_api_url(host, path) is not None
or _parse_lambda_url(host, path) is not None
or _resolve_custom_domain_request(host, path) is not None
or _parse_cloudfront_dataplane_host(host) is not None
+ or _is_potential_alb_request(host, path)
)
for response in (
None if owns_cors else _handle_options_request(method, path, headers, query_params, request_id),
@@ -1991,7 +2027,7 @@ async def _handle_cloudfront_dataplane_request(
async def _handle_execute_api_request(
host: str, path: str, method: str, headers: dict, body: bytes, query_params: dict,
- raw_path: str | None = None,
+ raw_path: str | None = None, raw_query_string: str | None = None,
):
"""Handle API Gateway execute-api data plane requests (Host-based,
path-based, and registered custom domains)."""
@@ -2061,7 +2097,9 @@ async def _handle_execute_api_request(
http_api_path = apigw_v2.decode_http_api_path(raw_path)
if http_api_path.startswith(prefix):
execute_path = http_api_path[len(prefix):]
- status, resp_headers, resp_body = await apigw_v2.handle_execute(api_id, stage, execute_path, method, headers, body, query_params)
+ status, resp_headers, resp_body = await apigw_v2.handle_execute(
+ api_id, stage, execute_path, method, headers, body, query_params, raw_query_string=raw_query_string,
+ )
# An HTTP API names its request id apigw-requestid (16 base64 characters); x-amzn-requestid is a REST API's.
resp_headers.setdefault("apigw-requestid", base64.b64encode(os.urandom(11)).decode())
return status, resp_headers, resp_body
@@ -2162,7 +2200,10 @@ async def _handle_lambda_url_request(host: str, path: str, method: str, headers:
return 500, {"Content-Type": "application/json"}, json.dumps({"message": str(e)}).encode()
-async def _handle_alb_request(host: str, path: str, method: str, headers: dict, body: bytes, query_params: dict):
+async def _handle_alb_request(
+ host: str, path: str, method: str, headers: dict, body: bytes, query_params: dict, raw_query_string: str | None = None,
+ raw_path: str | None = None,
+):
"""Handle ALB data-plane requests for host-based and /_alb-prefixed addressing."""
if not _is_potential_alb_request(host, path):
return None
@@ -2170,12 +2211,17 @@ async def _handle_alb_request(host: str, path: str, method: str, headers: dict,
alb_module = _get_module("alb")
load_balancer = alb_module.find_lb_for_host(host)
dispatch_path = path
+ # Instance and IP targets get the path as the client sent it, percent-encoding included.
+ raw_dispatch_path = raw_path
if load_balancer is None and path.startswith(_ALB_PATH_PREFIX):
path_parts = path[len(_ALB_PATH_PREFIX) :].split("/", 1)
load_balancer = alb_module._find_lb_by_name(path_parts[0])
if load_balancer:
dispatch_path = "/" + path_parts[1] if len(path_parts) > 1 else "/"
+ if raw_path is not None and raw_path.startswith(_ALB_PATH_PREFIX):
+ raw_parts = raw_path[len(_ALB_PATH_PREFIX) :].split("/", 1)
+ raw_dispatch_path = "/" + raw_parts[1] if len(raw_parts) > 1 else "/"
if load_balancer is None:
return None
@@ -2189,7 +2235,8 @@ async def _handle_alb_request(host: str, path: str, method: str, headers: dict,
try:
return await alb_module.dispatch_request(
- load_balancer, method, dispatch_path, headers, body, query_params, alb_port
+ load_balancer, method, dispatch_path, headers, body, query_params, alb_port,
+ raw_query_string=raw_query_string, raw_path=raw_dispatch_path,
)
except Exception as e:
logger.exception("Error in ALB data-plane dispatch: %s", e)
@@ -2633,7 +2680,8 @@ async def _handle_special_data_plane_request(
host, path, raw_path if raw_path is not None else path, raw_query_string, method,
headers, body, query_params, client_ip,
):
- return _with_data_plane_headers(response, request_id, wildcard_cors=False)
+ # A distribution adds its own headers (Via, X-Amz-Cf-Id, X-Amz-Cf-Pop, X-Cache), not an API's request ids.
+ return response
# Before S3 virtual hosting, which would read the collection id as a bucket.
if _AOSS_COLLECTION_HOST_RE.match(host) and (
response := await _get_module("opensearchserverless").handle_dataplane(
@@ -2642,7 +2690,7 @@ async def _handle_special_data_plane_request(
):
return _with_data_plane_headers(response, request_id)
if response := await _handle_execute_api_request(
- host, path, method, headers, body, query_params, raw_path=raw_path
+ host, path, method, headers, body, query_params, raw_path=raw_path, raw_query_string=raw_query_string
):
return _with_data_plane_headers(response, request_id, wildcard_cors=False)
if response := await _handle_lambda_url_request(host, path, method, headers, body, query_params):
@@ -2651,8 +2699,9 @@ async def _handle_special_data_plane_request(
if AUTH and (target := _s3_cors_target(host, path, path_style=False)):
response[1].update(_get_module("s3").cors_response_headers(target[0], method, headers))
return _with_data_plane_headers(response, request_id, include_s3_id=True, wildcard_cors=not AUTH)
- if response := await _handle_alb_request(host, path, method, headers, body, query_params):
- return _with_data_plane_headers(response, request_id)
+ if response := await _handle_alb_request(host, path, method, headers, body, query_params, raw_query_string, raw_path):
+ # A load balancer passes its target's headers through; CORS and request ids are the application's.
+ return response
return None
@@ -3162,8 +3211,25 @@ async def _dispatch_service_request(
# ---------------------------------------------------------------------------
+_RESPONSE_START_MESSAGES = ("http.response.start", "websocket.accept", "websocket.http.response.start")
+
+
+def _with_date_header(send):
+ """AWS sends one Date per response: a proxied target's, or the endpoint's own when there is none."""
+ async def send_with_date(message):
+ if message["type"] in _RESPONSE_START_MESSAGES and not any(
+ name.lower() == b"date" for name, _ in message.get("headers", [])
+ ):
+ message = {**message, "headers": [*message.get("headers", []), (b"date", formatdate(usegmt=True).encode("ascii"))]}
+ await send(message)
+
+ return send_with_date
+
+
async def app(scope, receive, send):
"""ASGI application entry point."""
+ if scope["type"] in ("http", "websocket"):
+ send = _with_date_header(send)
if scope["type"] == "lifespan":
await _handle_lifespan(scope, receive, send)
return
diff --git a/ministack/core/hypercorn_compat.py b/ministack/core/hypercorn_compat.py
index 4310f05b..27c3698a 100644
--- a/ministack/core/hypercorn_compat.py
+++ b/ministack/core/hypercorn_compat.py
@@ -9,37 +9,35 @@ when the client is waiting on ``Expect: 100-continue`` (e.g. S3
``upload_file``). Injecting the standard reason phrase makes the wire
output ``HTTP/1.1 100 Continue\\r\\n``, which every SDK version accepts.
+Final responses get their standard phrase the same way: AWS endpoints answer
+``HTTP/1.1 200 OK``, and a CloudFront Function reads it as ``statusDescription``.
+
Issue: https://github.com/ministackorg/ministack/issues/389
Remove this module if h11 ever ships a default reason upstream.
"""
-import h11
-
-# RFC 9110 § 15.2 informational response reason phrases.
-_DEFAULT_REASONS = {
- 100: b"Continue",
- 101: b"Switching Protocols",
- 102: b"Processing",
- 103: b"Early Hints",
-}
+from http import HTTPStatus
-_original_post_init = h11.InformationalResponse.__post_init__
+import h11
-def _patched_post_init(self) -> None:
- _original_post_init(self)
- if not self.reason:
- default = _DEFAULT_REASONS.get(self.status_code)
- if default is not None:
+def _with_default_reason(original_post_init):
+ def _patched_post_init(self) -> None:
+ original_post_init(self)
+ if not self.reason:
+ try:
+ default = HTTPStatus(self.status_code).phrase.encode()
+ except ValueError:
+ return
# Frozen dataclass — bypass normal attribute protection.
object.__setattr__(self, "reason", default)
-
-_patched_post_init._ministack_patched = True # type: ignore[attr-defined]
+ _patched_post_init._ministack_patched = True # type: ignore[attr-defined]
+ return _patched_post_init
def install() -> None:
"""Install the reason-phrase patch. Idempotent."""
- if getattr(h11.InformationalResponse.__post_init__, "_ministack_patched", False):
- return
- h11.InformationalResponse.__post_init__ = _patched_post_init
+ for event in (h11.InformationalResponse, h11.Response):
+ if not getattr(event.__post_init__, "_ministack_patched", False):
+ event.__post_init__ = _with_default_reason(event.__post_init__)
diff --git a/ministack/services/alb.py b/ministack/services/alb.py
index 37663c6f..44d9c9ea 100644
--- a/ministack/services/alb.py
+++ b/ministack/services/alb.py
@@ -32,6 +32,7 @@ import socket
import string
import time
import warnings
+from email.utils import formatdate
from urllib.parse import parse_qs, urlencode
from xml.sax.saxutils import escape as _esc
@@ -270,6 +271,7 @@ def _parse_actions(params, prefix="DefaultActions"):
"Port": _p(params, f"{prefix}.member.{i}.RedirectConfig.Port", "#{port}"),
"Host": _p(params, f"{prefix}.member.{i}.RedirectConfig.Host", "#{host}"),
"Path": _p(params, f"{prefix}.member.{i}.RedirectConfig.Path", "/#{path}"),
+ "Query": _p(params, f"{prefix}.member.{i}.RedirectConfig.Query", "#{query}"),
"StatusCode": rc_code,
}
oidc_issuer = _p(params, f"{prefix}.member.{i}.AuthenticateOidcConfig.Issuer")
@@ -498,6 +500,7 @@ def _action_xml(a):
f"<Port>{rc.get('Port','#{port}')}</Port>"
f"<Host>{rc.get('Host','#{host}')}</Host>"
f"<Path>{rc.get('Path','/#{path}')}</Path>"
+ f"<Query>{_esc(rc.get('Query', '#{query}'))}</Query>"
f"<StatusCode>{rc.get('StatusCode','HTTP_301')}</StatusCode>"
f"</RedirectConfig>"
)
@@ -1947,6 +1950,9 @@ async def _authenticate_oidc(action, method, path, headers, body, query_params,
return (401, {"Content-Type": "text/plain"}, b"Unauthorized")
host = headers.get("host") or f"localhost:{listener_port}"
+ name, sep, client_port = host.rpartition(":")
+ if headers.get("x-forwarded-port") == "443" and sep and client_port.isdigit():
+ host = name # On a 443 listener AWS names the callback without the client's port.
proto = headers.get("x-forwarded-proto") or "http"
redirect_uri = f"{proto}://{host}{OIDC_CALLBACK_PATH}"
state = new_uuid()
@@ -1986,7 +1992,7 @@ def _oidc_first(value):
# ---------------------------------------------------------------------------
async def _execute_actions(actions, method, path, headers, body, query_params, listener_port=80,
- lb_arn=""):
+ lb_arn="", target_host=None, raw_query_string=None, raw_path=None):
"""Run a rule's actions in Order until one produces a response.
Authentication actions are gates: they either short-circuit with a redirect
@@ -2013,14 +2019,18 @@ async def _execute_actions(actions, method, path, headers, body, query_params, l
# request to a target that assumes the load balancer checked.
return (501, {"Content-Type": "application/json"},
json.dumps({"message": "authenticate-cognito is not implemented"}).encode())
- return await _execute_action(action, method, path, forwarded, body, query_params)
+ tg = _tgs.get(action.get("TargetGroupArn", "")) if atype == "forward" else None
+ if target_host and tg and tg.get("TargetType", "instance") in ("instance", "ip"):
+ # Authentication used the client's Host; an instance or IP target gets the load balancer's.
+ forwarded["host"] = target_host
+ return await _execute_action(action, method, path, forwarded, body, query_params, raw_query_string, raw_path)
# Every action was a gate that passed and none of them was terminal.
return (502, {"Content-Type": "application/json"},
json.dumps({"message": "Rule has no terminal action"}).encode())
-async def _execute_action(action, method, path, headers, body, query_params):
+async def _execute_action(action, method, path, headers, body, query_params, raw_query_string=None, raw_path=None):
atype = action.get("Type", "").lower()
if atype == "fixed-response":
@@ -2033,26 +2043,29 @@ async def _execute_action(action, method, path, headers, body, query_params):
if atype == "redirect":
rc = action.get("RedirectConfig", {})
code = int(rc.get("StatusCode", "HTTP_301").replace("HTTP_", ""))
+ # The #{...} placeholders take the listener's protocol and port (as X-Forwarded-Proto/Port carry
+ # them) and the request's query; the port is always written, as AWS does (`http://host:80/...`).
src_host = headers.get("host", "localhost").split(":")[0]
- proto = rc.get("Protocol", "#{protocol}").replace("#{protocol}", "http")
+ if raw_query_string is None:
+ raw_query_string = urlencode([(k, v) for k, vs in (query_params or {}).items()
+ for v in (vs if isinstance(vs, list) else [vs])])
+ proto = rc.get("Protocol", "#{protocol}").replace("#{protocol}", headers.get("x-forwarded-proto", "http")).lower()
rhost = rc.get("Host", "#{host}").replace("#{host}", src_host)
- rport = rc.get("Port", "#{port}").replace("#{port}", "")
+ rport = rc.get("Port", "#{port}").replace("#{port}", headers.get("x-forwarded-port", "80"))
rpath = rc.get("Path", "/#{path}").replace("#{path}", path.lstrip("/"))
- location = f"{proto}://{rhost}"
- if rport and rport not in ("80", "443", ""):
- location += f":{rport}"
- location += rpath
+ rquery = rc.get("Query", "#{query}").replace("#{query}", raw_query_string)
+ location = f"{proto}://{rhost}:{rport}{rpath}" + (f"?{rquery}" if rquery else "")
return code, {"Location": location, "Content-Type": "text/plain"}, b""
if atype == "forward":
tg_arn = action.get("TargetGroupArn", "")
- return await _forward_to_tg(tg_arn, method, path, headers, body, query_params)
+ return await _forward_to_tg(tg_arn, method, path, headers, body, query_params, raw_query_string, raw_path)
return (502, {"Content-Type": "application/json"},
json.dumps({"message": f"Unsupported action type: {atype}"}).encode())
-async def _forward_to_tg(tg_arn, method, path, headers, body, query_params):
+async def _forward_to_tg(tg_arn, method, path, headers, body, query_params, raw_query_string=None, raw_path=None):
tg = _tgs.get(tg_arn)
if not tg:
return (502, {"Content-Type": "application/json"},
@@ -2068,18 +2081,18 @@ async def _forward_to_tg(tg_arn, method, path, headers, body, query_params):
if target_type == "lambda":
func_id = registered[0]["Id"]
return await _invoke_lambda_target(func_id, tg_arn, method, path,
- headers, body, query_params)
+ headers, body, query_params, raw_query_string)
if target_type in ("instance", "ip"):
target = random.choice(registered)
- return await _proxy_http_target(target, tg, method, path,
- headers, body, query_params)
+ return await _proxy_http_target(target, tg, method, raw_path or path,
+ headers, body, query_params, raw_query_string)
return (502, {"Content-Type": "application/json"},
json.dumps({"message": f"Target type '{target_type}' not supported."}).encode())
-async def _invoke_lambda_target(function_ref, tg_arn, method, path, headers, body, query_params):
+async def _invoke_lambda_target(function_ref, tg_arn, method, path, headers, body, query_params, raw_query_string=None):
try:
from ministack.services import lambda_svc
except ImportError:
@@ -2102,7 +2115,7 @@ async def _invoke_lambda_target(function_ref, tg_arn, method, path, headers, bod
return (502, {"Content-Type": "application/json"},
json.dumps({"message": f"Lambda function '{func_name}' not found"}).encode())
- body_str = None
+ body_str = ""
is_b64 = False
if body:
try:
@@ -2111,20 +2124,24 @@ async def _invoke_lambda_target(function_ref, tg_arn, method, path, headers, bod
body_str = base64.b64encode(body).decode("ascii")
is_b64 = True
- qs_single = {k: (v[0] if isinstance(v, list) else v) for k, v in query_params.items()}
- qs_multi = {k: (v if isinstance(v, list) else [v]) for k, v in query_params.items()}
-
event = {
"requestContext": {"elb": {"targetGroupArn": tg_arn}},
"httpMethod": method.upper(),
"path": path,
- "queryStringParameters": qs_single,
- "multiValueQueryStringParameters": qs_multi,
"headers": {k.lower(): v for k, v in headers.items()},
- "multiValueHeaders": {k.lower(): [v] for k, v in headers.items()},
"body": body_str,
"isBase64Encoded": is_b64,
}
+ if any(a.get("Key") == "lambda.multi_value_headers.enabled" and a.get("Value") == "true"
+ for a in _tg_attrs.get(tg_arn, [])):
+ event["queryStringParameters"] = {k: (v[0] if isinstance(v, list) else v) for k, v in query_params.items()}
+ event["multiValueQueryStringParameters"] = {k: (v if isinstance(v, list) else [v]) for k, v in query_params.items()}
+ event["multiValueHeaders"] = {k.lower(): [v] for k, v in headers.items()}
+ elif raw_query_string is not None:
+ # As the load balancer sends them: the last value of a repeated key, not URL-decoded.
+ event["queryStringParameters"] = dict(p.partition("=")[::2] for p in raw_query_string.split("&") if p)
+ else:
+ event["queryStringParameters"] = {k: (v[-1] if isinstance(v, list) else v) for k, v in query_params.items()}
exec_record = lambda_svc._execution_record_for_config(func, config)
@@ -2186,7 +2203,7 @@ async def _await_http_disconnect(receive) -> None:
return
-async def _proxy_http_target(target, tg, method, path, headers, body, query_params):
+async def _proxy_http_target(target, tg, method, path, headers, body, query_params, raw_query_string=None):
"""Forward a data-plane request to an instance/ip target over HTTP.
The target Id is used as the host to connect to — an IP address for
@@ -2202,11 +2219,13 @@ async def _proxy_http_target(target, tg, method, path, headers, body, query_para
host = str(target.get("Id", ""))
port = int(target.get("Port") or tg.get("Port") or 80)
- qs = {k: (v[0] if isinstance(v, list) else v) for k, v in (query_params or {}).items()}
- target_path = path + (f"?{urlencode(qs)}" if qs else "")
+ # The request target reaches the target as received, like the CloudFront data plane forwards its query string.
+ if raw_query_string is None:
+ raw_query_string = urlencode({k: (v if isinstance(v, list) else [v])
+ for k, v in (query_params or {}).items()}, doseq=True)
+ target_path = path + (f"?{raw_query_string}" if raw_query_string else "")
- hop_by_hop = {"host", "content-length", "connection", "transfer-encoding",
- "accept-encoding"}
+ hop_by_hop = {"content-length", "connection", "transfer-encoding"}
fwd = {k.lower(): v for k, v in (headers or {}).items() if k.lower() not in hop_by_hop}
data = body if isinstance(body, (bytes, bytearray)) else (body.encode() if body else None)
@@ -2250,6 +2269,8 @@ async def _proxy_http_target(target, tg, method, path, headers, body, query_para
out_headers.setdefault(name.lower(), []).append(value)
out_headers = {name: values[0] if len(values) == 1 else values
for name, values in out_headers.items()}
+ # The load balancer writes its own Date in place of the target's.
+ out_headers["date"] = formatdate(usegmt=True)
status = resp.status
async def _stream(send, receive):
@@ -2306,6 +2327,16 @@ def _lb_attr(lb_arn, key, default):
return next((a["Value"] for a in _lb_attrs.get(lb_arn, []) if a.get("Key") == key), default)
+def _target_host_header(host, listener_port):
+ """The Host an ALB sends an instance or IP target: on listener port 80 or 443 the client's port is
+ dropped; on any other listener port it becomes the listener's, whatever port the client sent.
+ Observed on AWS (us-east-1, 2026-10-23); Lambda targets get the Host unmodified."""
+ name, sep, port = host.rpartition(":")
+ if not (sep and port.isdigit()):
+ name = host
+ return name if int(listener_port) in (80, 443) else f"{name}:{listener_port}"
+
+
def _target_headers(lb_arn, listener, headers, client_ip):
"""The request fields a target receives: X-Forwarded-* and X-Amzn-Trace-Id set by the load balancer.
@@ -2334,15 +2365,54 @@ def _target_headers(lb_arn, listener, headers, client_ip):
return out
-async def dispatch_request(lb, method, path, headers, body, query_params, port=80, client_ip=None):
+# Listener attributes that add a response header; an empty value leaves the response alone.
+# https://docs.aws.amazon.com/elasticloadbalancing/latest/application/header-modification.html#insert-header
+_LISTENER_RESPONSE_HEADERS = {
+ f"routing.http.response.{attr}.header_value": header
+ for attr, header in (
+ ("access_control_allow_origin", "Access-Control-Allow-Origin"),
+ ("access_control_allow_methods", "Access-Control-Allow-Methods"),
+ ("access_control_allow_headers", "Access-Control-Allow-Headers"),
+ ("access_control_allow_credentials", "Access-Control-Allow-Credentials"),
+ ("access_control_expose_headers", "Access-Control-Expose-Headers"),
+ ("access_control_max_age", "Access-Control-Max-Age"),
+ )
+}
+
+
+def _with_listener_response_headers(listener_arn, response):
+ """Add the listener's configured response headers, replacing the target's header of that name."""
+ status, headers, body = response
+ added = {_LISTENER_RESPONSE_HEADERS[a["Key"]]: a["Value"]
+ for a in _listener_attrs.get(listener_arn, [])
+ if a["Key"] in _LISTENER_RESPONSE_HEADERS and a["Value"]}
+ if not added:
+ return response
+ replaced = {name.lower() for name in added}
+ return status, {**{k: v for k, v in headers.items() if k.lower() not in replaced}, **added}, body
+
+
+def _with_alb_server(response):
+ """A response the load balancer makes itself (Lambda target, fixed-response, redirect, authentication, its own
+ errors) carries its Server header; an instance or IP target's response keeps the target's own, or none."""
+ from ministack.core.responses import StreamingResponse
+
+ status, headers, body = response
+ if isinstance(body, StreamingResponse) or any(k.lower() == "server" for k in headers):
+ return response
+ return status, {**headers, "Server": "awselb/2.0"}, body
+
+
+async def dispatch_request(lb, method, path, headers, body, query_params, port=80, client_ip=None,
+ raw_query_string=None, raw_path=None):
lb_arn = lb["LoadBalancerArn"]
candidates = [l for l in _listeners.values() if l["LoadBalancerArn"] == lb_arn]
matching = [l for l in candidates if l.get("Port", 80) == port] or candidates
if not matching:
- return (503, {"Content-Type": "application/json"},
- json.dumps({"message": f"No listeners configured for '{lb['LoadBalancerName']}'"}).encode())
+ return _with_alb_server((503, {"Content-Type": "application/json"},
+ json.dumps({"message": f"No listeners configured for '{lb['LoadBalancerName']}'"}).encode()))
listener = matching[0]
l_arn = listener["ListenerArn"]
@@ -2364,12 +2434,17 @@ async def dispatch_request(lb, method, path, headers, body, query_params, port=8
if actions:
target_headers = _target_headers(lb_arn, listener, headers,
client_ip or get_request_client_ip())
- return await _execute_actions(actions, method, path,
- target_headers, body, query_params, port,
- lb_arn=lb.get("LoadBalancerArn", ""))
-
- return (502, {"Content-Type": "application/json"},
- json.dumps({"message": "No matching ALB rule found"}).encode())
+ target_host = target_headers.get("host")
+ if target_host and _lb_attr(lb_arn, "routing.http.preserve_host_header.enabled", "false") != "true":
+ target_host = _target_host_header(target_host, listener.get("Port", 80))
+ response = await _execute_actions(actions, method, path,
+ target_headers, body, query_params, port,
+ lb_arn=lb.get("LoadBalancerArn", ""), target_host=target_host,
+ raw_query_string=raw_query_string, raw_path=raw_path)
+ return _with_listener_response_headers(l_arn, _with_alb_server(response))
+
+ return _with_alb_server((502, {"Content-Type": "application/json"},
+ json.dumps({"message": "No matching ALB rule found"}).encode()))
def reset():
diff --git a/ministack/services/apigateway.py b/ministack/services/apigateway.py
index cd8d9e98..238d4578 100644
--- a/ministack/services/apigateway.py
+++ b/ministack/services/apigateway.py
@@ -1001,10 +1001,23 @@ async def _invoke_request_authorizer_lambda(authorizer, event, account_id, regio
return payload if isinstance(payload, dict) else None
+def _raw_query_string(query_params, raw_query_string=None):
+ """rawQueryString: the query string as the client sent it. Without one (an internal caller),
+ re-encode the URL-decoded parameters: a raw space breaks lambda_http (http::Uri rejects it ->
+ 502), and a decoded '='/'&' inside a value would corrupt the query structure. safe="" encodes
+ everything except RFC-3986 unreserved (#1035)."""
+ if raw_query_string is not None:
+ return raw_query_string
+ return "&".join(
+ f"{urllib.parse.quote(k, safe='')}={urllib.parse.quote(val, safe='')}"
+ for k, vals in (query_params or {}).items() for val in vals
+ )
+
+
async def _authorize_request_v2(
api_id, stage, route, method, path, resource_path,
headers, query_params, path_params, stage_vars,
- owner_account_id, owner_region,
+ owner_account_id, owner_region, raw_query_string=None,
):
"""Enforce a REQUEST-type Lambda authorizer before an HTTP API route's
integration runs.
@@ -1079,10 +1092,7 @@ async def _authorize_request_v2(
}
else:
qs = {k: ",".join(v) for k, v in query_params.items()} if query_params else None
- raw_qs = "&".join(
- f"{urllib.parse.quote(k, safe='')}={urllib.parse.quote(val, safe='')}"
- for k, vals in (query_params or {}).items() for val in vals
- )
+ raw_qs = _raw_query_string(query_params, raw_query_string)
event = {
"version": "2.0",
"type": "REQUEST",
@@ -1316,7 +1326,7 @@ def _apply_request_parameter_mappings(
return out_headers, out_query, out_path
-async def handle_execute(api_id, stage, path, method, headers, body, query_params):
+async def handle_execute(api_id, stage, path, method, headers, body, query_params, raw_query_string=None):
"""Execute an API request through a deployed API (data plane)."""
scope = find_api_scope(api_id)
if scope is None:
@@ -1330,7 +1340,7 @@ async def handle_execute(api_id, stage, path, method, headers, body, query_param
try:
return await _handle_execute_in_scope(
api_id, stage, path, method, headers, body, query_params,
- owner_account_id, owner_region,
+ owner_account_id, owner_region, raw_query_string,
)
finally:
_request_account_id.reset(account_token)
@@ -1344,7 +1354,7 @@ def _http_api_not_found():
async def _handle_execute_in_scope(
api_id, stage, path, method, headers, body, query_params,
- owner_account_id, owner_region,
+ owner_account_id, owner_region, raw_query_string=None,
):
api = _apis.get(api_id)
if not api:
@@ -1409,7 +1419,7 @@ async def _handle_execute_in_scope(
auth_error, authorizer_lambda_ctx = await _authorize_request_v2(
api_id, stage, route, method, path, (route_path or path),
request_headers, query_params or {}, path_params, stage_vars,
- owner_account_id, owner_region,
+ owner_account_id, owner_region, raw_query_string,
)
if auth_error is not None:
return auth_error
@@ -1453,6 +1463,7 @@ async def _handle_execute_in_scope(
authorizer_iam=authorizer_iam,
owner_account_id=owner_account_id,
owner_region=owner_region,
+ raw_query_string=raw_query_string,
)
elif integration_type == "HTTP_PROXY":
mapped_headers, mapped_query, mapped_path = _apply_request_parameter_mappings(
@@ -1630,6 +1641,7 @@ async def _invoke_lambda_proxy(
authorizer_iam=None,
owner_account_id=None,
owner_region=None,
+ raw_query_string=None,
):
"""Invoke a Lambda function using the API Gateway v2 proxy event format."""
from ministack.services import lambda_svc
@@ -1652,14 +1664,7 @@ async def _invoke_lambda_proxy(
# Build API Gateway v2 proxy event (payload format 2.0)
# AWS API Gateway v2 joins multi-value query params with commas
qs = {k: ",".join(v) for k, v in query_params.items()} if query_params else None
- # rawQueryString must stay percent-encoded like real AWS. query_params here are
- # already URL-decoded, so re-encode each key/value: a raw space breaks lambda_http
- # (http::Uri rejects it -> 502), and a decoded '='/'&' inside a value would corrupt
- # the query structure. safe="" encodes everything except RFC-3986 unreserved (#1035).
- raw_qs = "&".join(
- f"{urllib.parse.quote(k, safe='')}={urllib.parse.quote(val, safe='')}"
- for k, vals in query_params.items() for val in vals
- )
+ raw_qs = _raw_query_string(query_params, raw_query_string)
# Binary request bodies are base64-encoded with isBase64Encoded=true; only
# the text content types AWS recognizes are passed through as UTF-8 strings.
if body:
diff --git a/ministack/services/cloudfront.py b/ministack/services/cloudfront.py
index 48b13584..3a2e3a7c 100644
--- a/ministack/services/cloudfront.py
+++ b/ministack/services/cloudfront.py
@@ -5854,11 +5854,11 @@ def _apply_response_headers_policy(headers: dict, policy_cfg: dict, viewer_origi
def _cf_request_id() -> str:
- return base64.b64encode(os.urandom(42)).decode().rstrip("=")
+ return base64.urlsafe_b64encode(os.urandom(40)).decode()
def _cf_via_hash() -> str:
- return "".join(random.choices(string.ascii_lowercase + string.digits, k=13))
+ return os.urandom(16).hex()
def _add_cloudfront_headers(headers: dict, x_cache: str, request_id: str | None = None) -> dict:
@@ -5867,6 +5867,8 @@ def _add_cloudfront_headers(headers: dict, x_cache: str, request_id: str | None
result["x-cache"] = x_cache
result["x-amz-cf-pop"] = "LOC50-C1"
result["x-amz-cf-id"] = request_id or _cf_request_id()
+ if not any(k.lower() == "server" for k in result):
+ result["server"] = "CloudFront"
return result
@@ -6074,7 +6076,7 @@ async def _fetch_from_alb_origin(lb: dict, origin: dict, method: str, uri: str,
status, response_headers, data = await asyncio.wait_for(
alb.dispatch_request(lb, method, uri, alb_headers, body or b"",
parse_qs(query_string, keep_blank_values=True), port,
- client_ip="127.0.0.1"),
+ client_ip="127.0.0.1", raw_query_string=query_string),
read_timeout,
)
except asyncio.TimeoutError as e:
diff --git a/tests/test_alb.py b/tests/test_alb.py
index 75dcc494..ab9fe90f 100644
--- a/tests/test_alb.py
+++ b/tests/test_alb.py
@@ -564,18 +564,20 @@ def test_elbv2_dataplane_event_shape(elbv2, lam):
)
lb_arn, tg_arn, l_arn, fn_arn = _alb_setup(elbv2, lam, "dp-alb-evt", "dp-alb-evt-fn", fn_code)
try:
- url = f"{_endpoint}/_alb/dp-alb-evt/check?foo=bar"
+ url = f"{_endpoint}/_alb/dp-alb-evt/check?foo=bar&a=1&a=3&q=x%20y&flag"
resp = _req.urlopen(_req.Request(url, method="GET"))
body = json.loads(resp.read())
assert "requestContext" in body
assert "elb" in body["requestContext"]
assert body["httpMethod"] == "GET"
assert body["path"] == "/check"
- assert body["queryStringParameters"].get("foo") == "bar"
+ # Single-value by default: the last value of a repeated key, not URL-decoded, and no multi-value keys.
+ assert body["queryStringParameters"] == {"foo": "bar", "a": "3", "q": "x%20y", "flag": ""}
+ assert "multiValueQueryStringParameters" not in body and "multiValueHeaders" not in body
+ assert body["body"] == ""
assert body["headers"]["x-forwarded-for"] == "127.0.0.1"
assert (body["headers"]["x-forwarded-proto"], body["headers"]["x-forwarded-port"]) == ("http", "80")
assert re.fullmatch(r"Root=1-[0-9a-f]{8}-[0-9a-f]{24}", body["headers"]["x-amzn-trace-id"])
- assert body["multiValueHeaders"]["x-forwarded-for"] == ["127.0.0.1"]
assert body["isBase64Encoded"] is False
finally:
_alb_teardown(elbv2, lam, lb_arn, tg_arn, l_arn, "dp-alb-evt-fn")
@@ -613,6 +615,7 @@ def test_elbv2_dataplane_fixed_response(elbv2, lam):
resp = _req.urlopen(_req.Request(url, method="GET"))
assert resp.status == 200
assert resp.read() == b"maintenance"
+ assert resp.headers["Server"] == "awselb/2.0"
finally:
elbv2.delete_listener(ListenerArn=l_arn)
elbv2.delete_target_group(TargetGroupArn=tg_arn)
@@ -666,6 +669,35 @@ def test_elbv2_dataplane_redirect(elbv2):
elbv2.delete_target_group(TargetGroupArn=tg_arn)
elbv2.delete_load_balancer(LoadBalancerArn=lb_arn)
+def test_elbv2_dataplane_redirect_location_keeps_the_listener_port_and_query(elbv2):
+ import http.client as _http
+ from urllib.parse import urlparse as _urlparse
+
+ lb_arn = elbv2.create_load_balancer(Name="dp-alb-redir-q")["LoadBalancers"][0]["LoadBalancerArn"]
+ l_arn = elbv2.create_listener(
+ LoadBalancerArn=lb_arn, Protocol="HTTP", Port=80,
+ DefaultActions=[{"Type": "redirect", "RedirectConfig": {"Path": "/elsewhere", "StatusCode": "HTTP_302"}}],
+ )["Listeners"][0]["ListenerArn"]
+ try:
+ parsed = _urlparse(_endpoint)
+ conn = _http.HTTPConnection(parsed.hostname, parsed.port or 4566)
+ conn.request("GET", "/_alb/dp-alb-redir-q/x?y=1", headers={"Host": "app.example.test:9000"})
+ resp = conn.getresponse()
+ assert (resp.status, resp.getheader("Location")) == (302, "http://app.example.test:80/elsewhere?y=1")
+ conn.close()
+ elbv2.modify_listener(ListenerArn=l_arn, DefaultActions=[{"Type": "redirect", "RedirectConfig": {
+ "Path": "/elsewhere", "Query": "z=1&#{query}", "StatusCode": "HTTP_302"}}])
+ listener = elbv2.describe_listeners(ListenerArns=[l_arn])["Listeners"][0]
+ assert listener["DefaultActions"][0]["RedirectConfig"]["Query"] == "z=1&#{query}"
+ conn = _http.HTTPConnection(parsed.hostname, parsed.port or 4566)
+ conn.request("GET", "/_alb/dp-alb-redir-q/x?y=1", headers={"Host": "app.example.test:9000"})
+ assert conn.getresponse().getheader("Location") == "http://app.example.test:80/elsewhere?z=1&y=1"
+ conn.close()
+ finally:
+ elbv2.delete_listener(ListenerArn=l_arn)
+ elbv2.delete_load_balancer(LoadBalancerArn=lb_arn)
+
+
def test_elbv2_dataplane_path_pattern_rule(elbv2, lam):
"""Path-pattern rule routes /api/* to one Lambda; default routes to another."""
import urllib.request as _req
@@ -873,6 +905,90 @@ def test_elbv2_dataplane_forward_ip_target(elbv2):
_alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+def _start_echo_server():
+ """Answer every GET with the request target and headers the target received, as JSON."""
+ import threading
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+ class _Handler(BaseHTTPRequestHandler):
+ def do_GET(self):
+ headers = {}
+ for name, value in self.headers.items():
+ headers.setdefault(name.lower(), []).append(value)
+ body = json.dumps({"path": self.path, "headers": headers}).encode()
+ self.send_response(200)
+ self.send_header("Content-Length", str(len(body)))
+ self.end_headers()
+ self.wfile.write(body)
+
+ def log_message(self, _format, *_args):
+ return
+
+ server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
+ threading.Thread(target=server.serve_forever, daemon=True).start()
+ return server
+
+
+def _stop_server(server):
+ server.shutdown()
+ server.server_close()
+
+
+def test_elbv2_dataplane_forwards_the_client_host_header(elbv2):
+ """The target sees the client's Host, not its own address. On a default listener port (80)
+ AWS removes a port the client sent."""
+ server = _start_echo_server()
+ lb_name = "dp-alb-host"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ try:
+ for sent, received in (("app.example.test", "app.example.test"), ("app.example.test:8443", "app.example.test")):
+ req = _req.Request(f"{_endpoint}/_alb/{lb_name}/", headers={"Host": sent})
+ assert json.loads(_req.urlopen(req).read())["headers"]["host"] == [received]
+ elbv2.modify_load_balancer_attributes(LoadBalancerArn=lb_arn, Attributes=[
+ {"Key": "routing.http.preserve_host_header.enabled", "Value": "true"}])
+ req = _req.Request(f"{_endpoint}/_alb/{lb_name}/", headers={"Host": "app.example.test:8443"})
+ assert json.loads(_req.urlopen(req).read())["headers"]["host"] == ["app.example.test:8443"]
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ _stop_server(server)
+
+
+def test_elbv2_dataplane_lambda_target_gets_the_host_as_sent(elbv2, lam):
+ fn_code = ("import json\n"
+ "def handler(event, context):\n"
+ " return {'statusCode': 200, 'body': json.dumps(event['headers'])}\n")
+ lb_arn, tg_arn, l_arn, _fn_arn = _alb_setup(elbv2, lam, "dp-alb-lhost", "dp-alb-lhost-fn", fn_code)
+ try:
+ req = _req.Request(f"{_endpoint}/_alb/dp-alb-lhost/", headers={"Host": "app.example.test:9000"})
+ assert json.loads(_req.urlopen(req).read())["host"] == "app.example.test:9000"
+ finally:
+ _alb_teardown(elbv2, lam, lb_arn, tg_arn, l_arn, "dp-alb-lhost-fn")
+
+
+def test_oidc_redirect_uri_on_a_443_listener_drops_the_clients_port(elbv2, lam):
+ name = "oidc-dp-443"
+ lb_arn, tg_arn, l_arn = _alb_with_oidc(elbv2, lam, name)
+ elbv2.modify_listener(ListenerArn=l_arn, Port=443, Protocol="HTTPS",
+ Certificates=[{"CertificateArn": "arn:aws:acm:us-east-1:000000000000:certificate/x"}])
+ try:
+ status, headers, _ = _oidc_get("/private/thing", name)
+ assert status == 302
+ assert parse_qs(urlparse(headers["Location"]).query)["redirect_uri"] == [
+ f"https://{name}.alb.localhost/oauth2/idpresponse"]
+ finally:
+ _alb_teardown(elbv2, lam, lb_arn, tg_arn, l_arn, f"{name}-fn")
+
+
+def test_elbv2_target_host_follows_the_listener_port():
+ from ministack.services import alb
+
+ assert alb._target_host_header("app.example.test:8443", 443) == "app.example.test"
+ assert alb._target_host_header("app.example.test", 8080) == "app.example.test:8080"
+ assert alb._target_host_header("app.example.test:9000", 8080) == "app.example.test:8080"
+ assert alb._target_host_header("[::1]:8080", 80) == "[::1]"
+ assert alb._target_host_header("[::1]", 8080) == "[::1]:8080"
+
+
def _start_chunked_server(chunks, delay):
"""Serve `chunks` with `delay` seconds between them and no Content-Length."""
import threading
@@ -1270,6 +1386,42 @@ def test_elbv2_dataplane_streams_target_response(elbv2):
thread.join(timeout=5)
+def test_elbv2_dataplane_response_carries_the_load_balancers_date_once(elbv2):
+ """A load balancer writes its own Date in place of the target's; the response carries one."""
+ import threading
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+ class _StaleDate(BaseHTTPRequestHandler):
+ def date_time_string(self, timestamp=None):
+ return "Mon, 01 Jan 2001 00:00:00 GMT"
+
+ def do_GET(self):
+ self.send_response(200)
+ self.send_header("Content-Length", "2")
+ self.end_headers()
+ self.wfile.write(b"ok")
+
+ def log_message(self, *args):
+ pass
+
+ server = ThreadingHTTPServer(("127.0.0.1", 0), _StaleDate)
+ thread = threading.Thread(target=server.serve_forever, daemon=True)
+ thread.start()
+ lb_name = "dp-alb-date"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ try:
+ resp = _req.urlopen(_req.Request(f"{_endpoint}/_alb/{lb_name}/", method="GET"))
+ resp.read()
+ dates = resp.headers.get_all("Date")
+ assert len(dates) == 1 and "2001" not in dates[0]
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ server.shutdown()
+ server.server_close()
+ thread.join(timeout=5)
+
+
@pytest.mark.serial
def test_elbv2_dataplane_non_streaming_body_is_intact(elbv2):
"""A plain Content-Length response survives the read loop unchanged.
@@ -1688,7 +1840,7 @@ def test_unauthenticated_request_redirects_to_the_identity_provider(elbv2, lam):
assert q["client_id"] == ["client-abc"]
assert q["response_type"] == ["code"]
assert q["scope"] == ["openid email"]
- assert q["redirect_uri"][0].endswith("/oauth2/idpresponse")
+ assert q["redirect_uri"] == [f"http://{name}.alb.localhost:{_EXECUTE_PORT}/oauth2/idpresponse"]
assert q["state"], "the provider must be given a state to return"
finally:
_alb_teardown(elbv2, lam, lb_arn, tg_arn, l_arn, f"{name}-fn")
@@ -2853,3 +3005,161 @@ def test_elbv2_target_headers_follow_xff_mode_and_trace_id_rules():
assert "x-forwarded-for" not in alb._target_headers(lb_arn, listener, {"x-forwarded-for": "127.0.0.4"}, "127.0.0.1")
finally:
alb._lb_attrs.pop(lb_arn, None)
+
+
+def test_elbv2_dataplane_forwards_the_path_as_sent(elbv2):
+ """The target receives the path with its percent-encoding, as the client sent it."""
+ server = _start_echo_server()
+ lb_name = "dp-alb-rawpath"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ try:
+ resp = _req.urlopen(f"{_endpoint}/_alb/{lb_name}/reason/Custom%20Thing/a%2Fb")
+ assert json.loads(resp.read())["path"] == "/reason/Custom%20Thing/a%2Fb"
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ _stop_server(server)
+
+
+def test_elbv2_dataplane_forwards_repeated_query_keys(elbv2):
+ """The target receives the query string as the client sent it."""
+ server = _start_echo_server()
+ lb_name = "dp-alb-query"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ try:
+ resp = _req.urlopen(f"{_endpoint}/_alb/{lb_name}/search?a=1&b=2&a=3&q=x%20y&flag")
+ assert json.loads(resp.read())["path"] == "/search?a=1&b=2&a=3&q=x%20y&flag"
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ _stop_server(server)
+
+
+def test_elbv2_dataplane_forwards_options_to_the_target(elbv2):
+ """An OPTIONS request reaches the target, and the client sees the target's status and headers;
+ the gateway answers no preflight of its own for a load balancer."""
+ import threading
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+ class _Handler(BaseHTTPRequestHandler):
+ def do_OPTIONS(self):
+ self.send_response(204)
+ self.send_header("Allow", "GET, OPTIONS")
+ self.send_header("Content-Length", "0")
+ self.end_headers()
+
+ def log_message(self, _format, *_args):
+ return
+
+ server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
+ threading.Thread(target=server.serve_forever, daemon=True).start()
+ lb_name = "dp-alb-options"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ try:
+ req = _req.Request(f"{_endpoint}/_alb/{lb_name}/", method="OPTIONS",
+ headers={"Origin": "https://app.example.test", "Access-Control-Request-Method": "GET"})
+ resp = _req.urlopen(req)
+ assert resp.status == 204
+ assert resp.headers.get("Allow") == "GET, OPTIONS"
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ _stop_server(server)
+
+
+def test_elbv2_dataplane_passes_accept_encoding_and_a_gzip_response_through(elbv2):
+ """The target receives the client's Accept-Encoding, and its gzip response reaches the client
+ still encoded."""
+ import gzip
+ import http.client
+ import threading
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+ class _Handler(BaseHTTPRequestHandler):
+ def do_GET(self):
+ gzipped = "gzip" in (self.headers.get("Accept-Encoding") or "")
+ body = gzip.compress(b"compressed hello") if gzipped else b"plain hello"
+ self.send_response(200)
+ if gzipped:
+ self.send_header("Content-Encoding", "gzip")
+ self.send_header("Content-Length", str(len(body)))
+ self.end_headers()
+ self.wfile.write(body)
+
+ def log_message(self, _format, *_args):
+ return
+
+ server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
+ threading.Thread(target=server.serve_forever, daemon=True).start()
+ lb_name = "dp-alb-gzip"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ parsed = urlparse(_endpoint)
+ try:
+ conn = http.client.HTTPConnection(parsed.hostname, parsed.port)
+ conn.request("GET", f"/_alb/{lb_name}/", headers={"Accept-Encoding": "gzip"})
+ resp = conn.getresponse()
+ assert resp.getheader("Content-Encoding") == "gzip"
+ assert gzip.decompress(resp.read()) == b"compressed hello"
+ conn.close()
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ _stop_server(server)
+
+
+def test_elbv2_dataplane_adds_no_cors_or_request_id_headers(elbv2):
+ """An ALB passes the target's response headers through; CORS and request ids are the application's."""
+ server, thread = _start_fixed_body_server(b"ok")
+ lb_name = "dp-alb-plain"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ try:
+ resp = _req.urlopen(f"{_endpoint}/_alb/{lb_name}/")
+ assert resp.read() == b"ok"
+ names = {name.lower() for name, _ in resp.getheaders()}
+ assert not names & {"access-control-allow-origin", "x-amzn-requestid", "x-amz-request-id"}
+ assert resp.headers["Server"].startswith("BaseHTTP") # the target's own, not the load balancer's
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ server.shutdown()
+ server.server_close()
+ thread.join(timeout=5)
+
+
+def test_elbv2_dataplane_applies_the_listeners_cors_response_headers(elbv2):
+ """A non-empty routing.http.response.access_control_*.header_value is added to the listener's
+ responses (replacing the target's own), and an empty one reverts to the default."""
+ import threading
+ from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+ class _Handler(BaseHTTPRequestHandler):
+ def do_GET(self):
+ self.send_response(200)
+ self.send_header("Access-Control-Allow-Origin", "https://target.example.test")
+ self.send_header("Content-Length", "0")
+ self.end_headers()
+
+ def log_message(self, _format, *_args):
+ return
+
+ server = ThreadingHTTPServer(("127.0.0.1", 0), _Handler)
+ threading.Thread(target=server.serve_forever, daemon=True).start()
+ lb_name = "dp-alb-cors"
+ lb_arn, tg_arn, l_arn = _alb_http_target_setup(elbv2, lb_name, "127.0.0.1", server.server_address[1])
+ prefix = "routing.http.response."
+
+ def fetch():
+ return _req.urlopen(f"{_endpoint}/_alb/{lb_name}/").headers
+
+ try:
+ assert fetch().get_all("Access-Control-Allow-Origin") == ["https://target.example.test"]
+ assert fetch().get("Access-Control-Allow-Methods") is None
+ elbv2.modify_listener_attributes(ListenerArn=l_arn, Attributes=[
+ {"Key": prefix + "access_control_allow_origin.header_value", "Value": "https://app.example.test"},
+ {"Key": prefix + "access_control_allow_methods.header_value", "Value": "GET,POST"},
+ ])
+ headers = fetch()
+ assert headers.get_all("Access-Control-Allow-Origin") == ["https://app.example.test"]
+ assert headers.get("Access-Control-Allow-Methods") == "GET,POST"
+ elbv2.modify_listener_attributes(ListenerArn=l_arn, Attributes=[
+ {"Key": prefix + "access_control_allow_origin.header_value", "Value": ""},
+ ])
+ assert fetch().get_all("Access-Control-Allow-Origin") == ["https://target.example.test"]
+ finally:
+ _alb_http_target_teardown(elbv2, lb_arn, tg_arn, l_arn)
+ _stop_server(server)
diff --git a/tests/test_apigatewayv2.py b/tests/test_apigatewayv2.py
index 984d7142..64c69b6f 100644
--- a/tests/test_apigatewayv2.py
+++ b/tests/test_apigatewayv2.py
@@ -1387,8 +1387,9 @@ def test_http_api_response_request_id_header(apigw, lam):
def test_apigw_raw_query_string_percent_encoded(apigw, lam):
- """rawQueryString must stay percent-encoded like AWS: a space in a value comes
- back as %20, not a literal space (which breaks lambda_http / http::Uri) — #1035."""
+ """rawQueryString is the query string as the client sent it: a space in a value stays
+ %20, not a literal space (which breaks lambda_http / http::Uri) — #1035 — and a bare key
+ gains no "="."""
import urllib.request as _urlreq
import uuid as _uuid
@@ -1418,6 +1419,9 @@ def test_apigw_raw_query_string_percent_encoded(apigw, lam):
body = json.loads(resp.read())
assert "openid%20profile" in body["rawQs"]
assert "openid profile" not in body["rawQs"] # no literal space
+ req = _urlreq.Request(url.split("?")[0] + "?flag&a=1", method="GET")
+ req.add_header("Host", f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}")
+ assert json.loads(_urlreq.urlopen(req).read())["rawQs"] == "flag&a=1"
finally:
apigw.delete_api(ApiId=api_id)
lam.delete_function(FunctionName=fname)
@@ -1807,7 +1811,8 @@ def test_apigw_request_authorizer_simple_response_enforced_in_data_plane(apigw,
b"def handler(event, context):\n"
b" tok = event.get('headers', {}).get('authorization', '')\n"
b" if tok == 'allow-me':\n"
- b" return {'isAuthorized': True, 'context': {'user': 'alice', 'admin': True}}\n"
+ b" return {'isAuthorized': True, 'context': {'user': 'alice', 'admin': True,\n"
+ b" 'rawQueryString': event.get('rawQueryString')}}\n"
b" return {'isAuthorized': False}\n"
)
int_code = (
@@ -1855,8 +1860,8 @@ def test_apigw_request_authorizer_simple_response_enforced_in_data_plane(apigw,
host = f"{api_id}.execute-api.localhost:{_EXECUTE_PORT}"
url = f"http://{host}/$default/private"
- def _call(token):
- req = _urlreq.Request(url, method="GET")
+ def _call(token, query=""):
+ req = _urlreq.Request(url + query, method="GET")
req.add_header("Host", host)
if token is not None:
req.add_header("Authorization", token)
@@ -1872,10 +1877,10 @@ def test_apigw_request_authorizer_simple_response_enforced_in_data_plane(apigw,
# Valid token -> isAuthorized: true -> 200, context reaches the
# integration under requestContext.authorizer.lambda.
- status, body = _call("allow-me")
+ status, body = _call("allow-me", "?flag&a=1")
assert status == 200
auth = json.loads(body)
- assert auth == {"lambda": {"user": "alice", "admin": True}}
+ assert auth == {"lambda": {"user": "alice", "admin": True, "rawQueryString": "flag&a=1"}}
finally:
apigw.delete_api(ApiId=api_id)
lam.delete_function(FunctionName=auth_fn)
diff --git a/tests/test_cloudfront.py b/tests/test_cloudfront.py
index e24f3f73..2f6e4e83 100644
--- a/tests/test_cloudfront.py
+++ b/tests/test_cloudfront.py
@@ -5813,6 +5813,7 @@ class _RawHeaderCaptureServer:
self.sock.bind(("127.0.0.1", 0))
self.sock.listen(5)
self.port = self.sock.getsockname()[1]
+ self.last_request_line = ""
self.last_header_lines: list = []
self._stop = False
self.thread = threading.Thread(target=self._serve, daemon=True)
@@ -5837,7 +5838,8 @@ class _RawHeaderCaptureServer:
data += chunk
head, _, _rest = data.partition(b"\r\n\r\n")
lines = head.decode("latin-1").split("\r\n")
- self.last_header_lines = lines[1:] # drop the request line
+ self.last_request_line = lines[0]
+ self.last_header_lines = lines[1:]
body = b"ok"
conn.sendall(
b"HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nContent-Length: "
@@ -6106,7 +6108,8 @@ def test_untrusted_origin_certificate_answers_502(cloudfront, untrusted_https_or
# ---------------------------------------------------------------------------
-def _alb_origin_dist(cloudfront, lb_dns: str, port: int, custom_headers: list, read_timeout: int = 30) -> str:
+def _alb_origin_dist(cloudfront, lb_dns: str, port: int, custom_headers: list, read_timeout: int = 30,
+ origin_request_policy_id: str | None = None) -> str:
suffix = _uuid_mod.uuid4().hex[:10]
origin = {
"Id": "alb-origin",
@@ -6132,6 +6135,8 @@ def _alb_origin_dist(cloudfront, lb_dns: str, port: int, custom_headers: list, r
"CachePolicyId": _CACHING_DISABLED,
},
}
+ if origin_request_policy_id:
+ dist_config["DefaultCacheBehavior"]["OriginRequestPolicyId"] = origin_request_policy_id
return cloudfront.create_distribution(DistributionConfig=dist_config)["Distribution"]["DomainName"]
@@ -6179,6 +6184,7 @@ def test_registered_alb_origin_without_a_listener_on_the_origin_port_answers_502
status, headers, _body = _get(dist_host, "/anything")
assert status == 502
assert headers.get("X-Cache") == "Error from cloudfront"
+ assert {k.lower(): v for k, v in headers.items()}["server"] == "CloudFront"
def test_unregistered_alb_hostname_is_still_refused(cloudfront):
@@ -6213,6 +6219,27 @@ def test_registered_alb_origin_relays_an_ip_target_response(cloudfront, elbv2, a
elbv2.delete_target_group(TargetGroupArn=tg_arn)
+def test_registered_alb_origin_forwards_the_query_string_as_the_viewer_sent_it(
+ cloudfront, elbv2, alb_origin, raw_header_server,
+):
+ tg_arn = elbv2.create_target_group(
+ Name=f"cf-alb-{_uuid_mod.uuid4().hex[:8]}", Protocol="HTTP", Port=raw_header_server.port,
+ VpcId="vpc-00000001", TargetType="ip",
+ )["TargetGroups"][0]["TargetGroupArn"]
+ try:
+ elbv2.register_targets(TargetGroupArn=tg_arn, Targets=[{"Id": "127.0.0.1", "Port": raw_header_server.port}])
+ elbv2.modify_listener(ListenerArn=alb_origin["listener_arn"],
+ DefaultActions=[{"Type": "forward", "TargetGroupArn": tg_arn}])
+ dist_host = _alb_origin_dist(cloudfront, alb_origin["dns"], 80, [],
+ origin_request_policy_id=_MANAGED_ALL_VIEWER_EXCEPT_HOST_ID)
+
+ status, _headers, _body = _get(dist_host, "/search?a=1&b=2&a=3&flag")
+ assert status == 200
+ assert raw_header_server.last_request_line == "GET /search?a=1&b=2&a=3&flag HTTP/1.1"
+ finally:
+ elbv2.delete_target_group(TargetGroupArn=tg_arn)
+
+
def test_registered_alb_origin_timeout_answers_504_and_closes_the_target_connection(cloudfront, elbv2, alb_origin):
# A target that reads the request and never answers.
listener = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
@@ -6351,3 +6378,23 @@ def test_function_url_origin_over_http_is_unreachable(cloudfront, function_url_o
def test_unregistered_function_url_hostname_is_still_refused(cloudfront):
host = f"{_uuid_mod.uuid4()}.lambda-url.us-east-1.on.aws"
assert _get(_function_url_dist(cloudfront, host), "/hello")[0] == 502
+
+
+def test_distribution_adds_no_request_id_headers_of_its_own(cf_raw_origin_stack):
+ """CloudFront adds Via, X-Amz-Cf-Id, X-Amz-Cf-Pop and X-Cache; an origin that sends no request id gets none."""
+ status, headers, _body = _get(cf_raw_origin_stack, "/hello")
+ assert status == 200
+ names = {name.lower() for name in headers}
+ assert "x-amz-cf-id" in names
+ assert not names & {"x-amzn-requestid", "x-amz-request-id"}
+ # The origin's own Server passes through.
+ assert {k.lower(): v for k, v in headers.items()}["server"].startswith("BaseHTTP")
+
+
+def test_request_id_has_cloudfronts_shape(cf_raw_origin_stack):
+ """X-Amz-Cf-Id is 40 random bytes in URL-safe base64 with its padding, as CloudFront sends it."""
+ status, headers, _body = _get(cf_raw_origin_stack, "/hello")
+ assert status == 200
+ lowered = {name.lower(): value for name, value in headers.items()}
+ assert re.fullmatch(r"[A-Za-z0-9_-]{54}==", lowered["x-amz-cf-id"]), lowered["x-amz-cf-id"]
+ assert re.fullmatch(r"1\.1 [0-9a-f]{32}\.cloudfront\.net \(CloudFront\)", lowered["via"]), lowered["via"]
diff --git a/tests/test_ministack.py b/tests/test_ministack.py
index 2f8408a4..3048e612 100644
--- a/tests/test_ministack.py
+++ b/tests/test_ministack.py
@@ -99,6 +99,64 @@ def test_localstack_unknown_paths_return_json_404():
)
+def test_responses_do_not_advertise_the_emulator_server():
+ """No AWS endpoint answers with Hypercorn's Server value."""
+ import urllib.request
+
+ resp = urllib.request.urlopen(f"{ENDPOINT}/_ministack/health")
+ assert not (resp.headers.get("Server") or "").startswith("hypercorn")
+
+
+def test_http_and_websocket_responses_carry_one_date_header():
+ """Regression guard: with Hypercorn's own Date off, an http response and a WebSocket handshake
+ response each still carry exactly one."""
+ import asyncio
+ import urllib.request
+ from urllib.parse import urlparse
+
+ websockets = pytest.importorskip("websockets")
+ parsed = urlparse(ENDPOINT)
+
+ async def handshake():
+ url = f"ws://prefix-ats.iot.us-east-1.{parsed.hostname}:{parsed.port}/mqtt"
+ async with websockets.connect(url, subprotocols=["mqtt"]) as ws:
+ return ws.response.headers.get_all("Date")
+
+ assert len(urllib.request.urlopen(f"{ENDPOINT}/_ministack/health").headers.get_all("Date")) == 1
+ assert len(asyncio.run(handshake())) == 1
+
+
+def test_a_malformed_request_is_answered_with_one_date():
+ import socket
+ from urllib.parse import urlparse
+
+ parsed = urlparse(ENDPOINT)
+ with socket.create_connection((parsed.hostname, parsed.port), timeout=5) as sock:
+ sock.sendall(b"GET / HTTP/1.1\r\nHost: localhost\r\nContent-Length: abc\r\n\r\n")
+ head = sock.recv(65536).split(b"\r\n\r\n")[0].lower().split(b"\r\n")
+ assert head[0].startswith(b"http/1.1 400")
+ assert sum(line.startswith(b"date:") for line in head) == 1
+
+
+def test_a_refused_websocket_handshake_carries_one_date():
+ import asyncio
+ from urllib.parse import urlparse
+
+ websockets = pytest.importorskip("websockets")
+ parsed = urlparse(ENDPOINT)
+
+ async def refused():
+ url = f"ws://{parsed.hostname}:{parsed.port}/_aws/execute-api/nosuchapi/stage"
+ with pytest.raises(websockets.exceptions.InvalidStatus) as exc:
+ async with websockets.connect(url):
+ pass
+ return exc.value.response
+
+ response = asyncio.run(refused())
+ assert response.status_code == 403
+ assert len(response.headers.get_all("Date")) == 1
+
+
def test_localstack_health_still_returns_200():
"""/_localstack/health must still return 200 after the interceptor is wired in."""
import urllib.request
@@ -549,6 +607,19 @@ def test_unit_h11_informational_has_reason_phrase():
f"expected canonical reason phrase, got: {out!r}"
+def test_unit_h11_response_has_reason_phrase():
+ """A final response carries its standard reason phrase too: AWS endpoints answer 'HTTP/1.1 404 Not Found'."""
+ import h11
+
+ import ministack.app # noqa: F401
+
+ conn = h11.Connection(our_role=h11.SERVER)
+ conn.receive_data(b"GET / HTTP/1.1\r\nHost: x\r\n\r\n")
+ conn.next_event()
+ out = conn.send(h11.Response(status_code=404, headers=[]))
+ assert out.startswith(b"HTTP/1.1 404 Not Found\r\n"), f"expected canonical reason phrase, got: {out!r}"
+
+
def test_wire_expect_100_continue_returns_canonical_status_line():
"""End-to-end: a raw PUT with Expect: 100-continue against ministack must
receive a 100 Continue with the reason phrase intact (issue #389)."""