Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(auth): treat unsigned requests as anonymous under AUTH=true (#2401)

iot-rocket9342a8a9a884
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 6fc43741..48fbf30f 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -17,6 +17,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).
- **Auth — SigV4a signatures are verified under `AUTH=true`** — requests signed with `AWS4-ECDSA-P256-SHA256`, which SDKs use for Multi-Region Access Points, were refused for every IAM user. Header and presigned S3 signatures are now checked with the P-256 key SigV4a derives from the secret, with their date or expiry, and an `X-Amz-Region-Set` that leaves out the serving region is refused (`RegionSetMismatch` on S3, the unknown-key error elsewhere). Contributed by @iot-rocket.
### Fixed
+- **Auth — requests without credentials are anonymous in every service** — under `AUTH=true` a request with no `Authorization` header and no query signature ran as the account root everywhere but S3, so unsigned STS, SQS, DynamoDB or Lambda calls were served. It is now refused in the service's own error, as on AWS: `MissingAuthenticationToken` 403 from Query and REST-XML services, `MissingAuthenticationTokenException` from JSON services, `AccessDenied` 403 from SQS, `UnrecognizedClientException` from Secrets Manager. Query credentials without `X-Amz-Algorithm` count as none. Operations the SDKs send unsigned (Cognito sign-in, `GetId`, `AssumeRoleWithWebIdentity`), SNS `ConfirmSubscription` and `Unsubscribe`, API Gateway and Function URL invokes and S3 keep their behaviour. Contributed by @iot-rocket.
- **AppSync Events — `AUTH=true` checks the Event API's authorization** — under `AUTH=true` an HTTP publish, a WebSocket connection or a subscription was accepted with any API key or none unless `APPSYNC_EVENTS_ENFORCE_AUTH` was set. It now needs what the API or the channel namespace accepts, as on AWS: a key registered on the API for `API_KEY`, a SigV4 signature for `AWS_IAM`, a bearer token for Cognito and OIDC. A refused publish answers 401 `UnauthorizedException`, with AWS's "You are not authorized to make this call." for a key the API does not know. `APPSYNC_EVENTS_ENFORCE_AUTH` works as before with `AUTH` unset. Contributed by @iot-rocket.
- **Cognito — pre token generation for `client_credentials`** — a `client_credentials` grant invoked a `V2_0` pre token generation trigger; it now invokes the trigger only when `LambdaVersion` is `V3_0`, as AWS documents and does. Contributed by @iot-rocket.
- **Cognito — `client_credentials` needs the flow** — the `/oauth2/token` endpoint gave a `client_credentials` token to an app client without `client_credentials` in `AllowedOAuthFlows`; such a client now gets `invalid_grant`, or `invalid_scope` first when a requested scope is unknown or none is left, as on AWS. Contributed by @iot-rocket.
diff --git a/ministack/core/iam_actions.py b/ministack/core/iam_actions.py
index 54bc73b4..1d396c13 100644
--- a/ministack/core/iam_actions.py
+++ b/ministack/core/iam_actions.py
@@ -2048,6 +2048,19 @@ def _tag_operations(service: str) -> frozenset[str]:
return _TAG_OPERATIONS[service]
+_UNSIGNED_OPERATIONS: dict[str, frozenset[str]] = {}
+
+
+def unsigned_operations(service: str) -> frozenset[str]:
+ """The operations of *service* its SDK model sends without credentials."""
+ if service not in _UNSIGNED_OPERATIONS:
+ _UNSIGNED_OPERATIONS[service] = frozenset(
+ op_name for name in _BOTOCORE_SERVICE_MAP.get(service) or [service]
+ for op_name, op in _load_botocore_model(name).get("operations", {}).items()
+ if op.get("authtype") == "none" or op.get("auth") == ["smithy.api#noAuth"])
+ return _UNSIGNED_OPERATIONS[service]
+
+
def request_tag_context(service: str, iam_action: str, body: bytes, params: dict) -> dict:
"""aws:RequestTag/<key> for every tag a request sets and aws:TagKeys for every key it sets or removes."""
if iam_action.split(":", 1)[-1] not in _tag_operations(service):
@@ -2124,6 +2137,10 @@ _AUTH_FAILURES: dict[str, tuple[str, int, str, tuple[str, int, str]]] = {
),
"IncompleteSignature": _INCOMPLETE,
"MissingAuthorizationParameter": _INCOMPLETE,
+ "MissingAuthenticationToken": (
+ "MissingAuthenticationToken", 403, "MissingAuthenticationTokenException",
+ ("AccessDenied", 403, "Access Denied"),
+ ),
"UnsupportedSignature": (
"IncompleteSignature", 400, "IncompleteSignatureException",
("InvalidRequest", 400, "The authorization mechanism you have provided is not supported. "
@@ -2137,10 +2154,37 @@ _EC2_AUTH_FAILURE = ("AuthFailure", 401, "AWS was not able to validate the provi
_EC2_FAILURES = {"ExpiredTokenException": ("RequestExpired", 400, "Request has expired.")}
+_MISSING_TOKEN_FAILURES = {
+ "secretsmanager": ("UnrecognizedClientException", 400, "The security token included in the request is invalid."),
+ "ec2": ("MissingParameter", 400, "The request must contain the parameter AWSAccessKeyId"),
+}
+
+
+def anonymous_denial_message(service: str, iam_action: str, resource_arn: str) -> str:
+ """The message of a refused request without credentials; SQS names the anonymous caller and the queue."""
+ if service == "sqs" and resource_arn.startswith("arn:"):
+ action = iam_action.lower()
+ return (f"User: anonymous is not authorized to perform: {action} on resource: {resource_arn} "
+ f"because no resource-based policy allows the {action} action")
+ return "Request is missing Authentication Token"
+
+
def _auth_failure(service: str, code: str, message: str, headers: dict) -> tuple[str, int, str]:
"""The code, status and message a service answers for a failed authentication."""
if service == "lambda" and code == "MissingAuthorizationParameter":
return "MissingAuthenticationTokenException", 403, "Missing Authentication Token"
+ if code == "MissingAuthenticationToken":
+ if service == "sqs":
+ from ministack.core.iam_evaluator import _request_origin
+
+ if not message.startswith("User: anonymous"):
+ scheme = "https" if _request_origin.get()[1] else "http"
+ message = f"Access to the resource {scheme}://{headers.get('host', '')}/ is denied."
+ return "AccessDenied", 403, message
+ if service in _MISSING_TOKEN_FAILURES:
+ return _MISSING_TOKEN_FAILURES[service]
+ if _SERVICE_PROTOCOL.get(service) is None and service != "dynamodb":
+ message = "Missing Authentication Token"
query_code, query_status, json_code, s3_failure = _AUTH_FAILURES[code]
protocol = _SERVICE_PROTOCOL.get(service)
if service == "s3":
@@ -2179,6 +2223,9 @@ def identity_denial_message(action: str, principal_arn: str, resource: str, expl
return f"User: {principal_arn} is not authorized to perform: {action} on resource: {resource} {reason}"
+_IOT_DATA_FAILURES = {"MissingAuthenticationToken": "Missing authentication"}
+
+
def access_denied_response(service: str, action: str, principal_arn: str,
request_id: str, *, error_code: str = "",
message: str = "", headers: dict | None = None,
@@ -2204,7 +2251,8 @@ def access_denied_response(service: str, action: str, principal_arn: str,
# for a failed authentication, no message for a policy denial.
return (
403, {"Content-Type": "application/json", "x-amzn-errortype": "ForbiddenException:"},
- json.dumps({"message": "Forbidden" if error_code else None, "traceId": str(uuid.uuid4())}).encode(),
+ json.dumps({"message": _IOT_DATA_FAILURES.get(error_code, "Forbidden") if error_code else None,
+ "traceId": str(uuid.uuid4())}).encode(),
)
if not message and explicit_deny and resource_policy_arn:
message = _resource_policy_denial(service, action, principal_arn, resource_policy_arn)
diff --git a/ministack/core/iam_evaluator.py b/ministack/core/iam_evaluator.py
index fa7b4473..8e28064c 100644
--- a/ministack/core/iam_evaluator.py
+++ b/ministack/core/iam_evaluator.py
@@ -1261,6 +1261,8 @@ _signed_request: contextvars.ContextVar[SignedRequest | None] = contextvars.Cont
"ministack_signed_request", default=None)
# The access key whose signature this request already passed, once checked.
_verified_key: contextvars.ContextVar[str] = contextvars.ContextVar("ministack_verified_key", default="")
+# Whether the request carries no credentials: no Authorization header and no query signature.
+_anonymous: contextvars.ContextVar[bool] = contextvars.ContextVar("ministack_anonymous", default=False)
def caller_verified(access_key_id: str, account_id: str) -> bool:
@@ -1286,6 +1288,9 @@ def pin_signed_request(
presigned = not authorization.startswith("AWS4-HMAC-SHA256") and (
_query_signed(query_params) or _sigv4a_s3_presign(query_params))
access_key_id = extract_access_key_id({"authorization": authorization}, query_params if presigned else None)
+ # AWS reads query credentials without X-Amz-Algorithm as none at all.
+ _anonymous.set(not authorization and not any(
+ _query_value(query_params or {}, name) for name in ("X-Amz-Algorithm", "AWSAccessKeyId")))
# The key the policy check will use; it must be the one the signature is checked for.
claimed = extract_access_key_id({"authorization": authorization}, query_params)
header_signed = authorization.startswith("AWS4-HMAC-SHA256") or _sigv4a_header(authorization, payload_hash)
@@ -1544,6 +1549,17 @@ def check_credentials(access_key_id: str) -> AuthError | None:
# Top-level enforcement entry point
# ---------------------------------------------------------------------------
+# Signed in the SDK models, yet served without credentials: the links in SNS subscription messages.
+_UNSIGNED_LINK_ACTIONS = frozenset({"sns:ConfirmSubscription", "sns:Unsubscribe"})
+
+
+def _served_unsigned(service: str, iam_action: str) -> bool:
+ """Whether AWS serves this action without credentials: an operation its SDK model sends unsigned, or a link."""
+ from ministack.core.iam_actions import unsigned_operations
+
+ return iam_action in _UNSIGNED_LINK_ACTIONS or iam_action.partition(":")[2] in unsigned_operations(service)
+
+
# Actions that AWS always allows regardless of IAM policies.
# https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_evaluation-logic.html
_ALWAYS_ALLOWED_ACTIONS = frozenset({
@@ -1623,6 +1639,10 @@ def enforce(access_key_id: str, iam_action: str, service: str,
account_id = get_account_id()
# S3, API Gateway methods and function URLs can be called anonymously: an unsigned request is not checked.
anonymous_ok = service == "s3" or iam_action in ("execute-api:Invoke", "lambda:InvokeFunctionUrl")
+ if _anonymous.get() and not anonymous_ok and not _served_unsigned(service, iam_action):
+ from ministack.core.iam_actions import anonymous_denial_message
+
+ return AuthError("MissingAuthenticationToken", anonymous_denial_message(service, iam_action, resource_arn))
unsigned = anonymous_ok and not access_key_id and _signed_request.get() is None
checked = None if unsigned else authenticate(access_key_id or "", account_id)
if isinstance(checked, AuthError):
diff --git a/tests/test_iam_auth.py b/tests/test_iam_auth.py
index d2459ee2..567fd42d 100644
--- a/tests/test_iam_auth.py
+++ b/tests/test_iam_auth.py
@@ -6335,6 +6335,139 @@ def test_unverifiable_credentials_are_refused_contract(auth_enabled, iam_user):
assert status >= 400 if auth_enabled else status == 200, (authorization, query)
+# A call per protocol family, and the code and status AWS answers it without credentials.
+_ANONYMOUS_CALLS = {
+ "sts": (lambda c: c.get_caller_identity(), "MissingAuthenticationToken", 403),
+ "sns": (lambda c: c.list_topics(), "MissingAuthenticationToken", 403),
+ "sqs": (lambda c: c.list_queues(), "AccessDenied", 403),
+ "dynamodb": (lambda c: c.list_tables(), "MissingAuthenticationTokenException", 400),
+ "kms": (lambda c: c.list_keys(), "MissingAuthenticationTokenException", 400),
+ "lambda": (lambda c: c.list_functions(), "MissingAuthenticationTokenException", 403),
+ "secretsmanager": (lambda c: c.list_secrets(), "UnrecognizedClientException", 400),
+}
+
+
+def _anonymous_client(service):
+ from botocore import UNSIGNED
+ from conftest import make_client
+
+ return make_client(service, {"signature_version": UNSIGNED})
+
+
+def _refusal(call):
+ """The error code and status a call answers, or None when it succeeds."""
+ try:
+ call()
+ except ClientError as e:
+ return e.response["Error"]["Code"], e.response["ResponseMetadata"]["HTTPStatusCode"]
+ return None
+
+
[email protected]("service", sorted(_ANONYMOUS_CALLS))
+def test_request_without_credentials_is_anonymous_contract(auth_enabled, iam_user, service):
+ """A request without credentials is refused in the service's own error; signed, the same call is served."""
+ call, code, status = _ANONYMOUS_CALLS[service]
+ signed = iam_user(policy={"Version": "2012-10-17",
+ "Statement": [{"Effect": "Allow", "Action": "*", "Resource": "*"}]}).client(service)
+ assert _refusal(lambda: call(signed)) is None
+ assert _refusal(lambda: call(_anonymous_client(service))) == ((code, status) if auth_enabled else None)
+
+
+def test_resource_policy_for_everyone_does_not_admit_anonymous_contract(auth_enabled):
+ """A queue or topic policy that allows "*" still needs a signed request."""
+ import uuid
+
+ from conftest import make_client
+
+ name = f"anonymous-{uuid.uuid4().hex[:8]}"
+ sqs, sns = make_client("sqs"), make_client("sns")
+ queue = sqs.create_queue(QueueName=name)["QueueUrl"]
+ queue_arn = sqs.get_queue_attributes(QueueUrl=queue, AttributeNames=["QueueArn"])["Attributes"]["QueueArn"]
+ topic = sns.create_topic(Name=name)["TopicArn"]
+ try:
+ sqs.set_queue_attributes(QueueUrl=queue, Attributes={"Policy": json.dumps({
+ "Version": "2012-10-17", "Statement": [
+ {"Effect": "Allow", "Principal": "*", "Action": "sqs:SendMessage", "Resource": queue_arn}]})})
+ sns.set_topic_attributes(TopicArn=topic, AttributeName="Policy", AttributeValue=json.dumps({
+ "Version": "2012-10-17", "Statement": [
+ {"Effect": "Allow", "Principal": "*", "Action": "SNS:Publish", "Resource": topic}]}))
+ sent = _refusal(lambda: _anonymous_client("sqs").send_message(QueueUrl=queue, MessageBody="m"))
+ published = _refusal(lambda: _anonymous_client("sns").publish(TopicArn=topic, Message="m"))
+ if auth_enabled:
+ assert (sent, published) == (("AccessDenied", 403), ("MissingAuthenticationToken", 403))
+ assert "Messages" not in sqs.receive_message(QueueUrl=queue)
+ else:
+ assert (sent, published) == (None, None)
+ finally:
+ sqs.delete_queue(QueueUrl=queue)
+ sns.delete_topic(TopicArn=topic)
+
+
+def test_query_credentials_without_an_algorithm_are_anonymous_contract(auth_enabled, iam_user):
+ """X-Amz-Credential and X-Amz-Signature without X-Amz-Algorithm count as no credentials, not as that key's user."""
+ import urllib.error
+ import urllib.request
+ from urllib.parse import parse_qsl, urlsplit
+
+ from botocore.auth import SigV4QueryAuth
+ from botocore.awsrequest import AWSRequest
+ from botocore.credentials import Credentials
+ from conftest import ENDPOINT, REGION, second_access_key
+
+ user = iam_user(policy={"Version": "2012-10-17",
+ "Statement": [{"Effect": "Allow", "Action": "*", "Resource": "*"}]})
+ with second_access_key(user) as (key, secret):
+ request = AWSRequest(method="GET", url=f"{ENDPOINT}/?Action=GetCallerIdentity&Version=2011-06-15")
+ SigV4QueryAuth(Credentials(key, secret), "sts", REGION, expires=300).add_auth(request)
+ query = [(k, v) for k, v in parse_qsl(urlsplit(request.url).query) if k != "X-Amz-Algorithm"]
+ try:
+ with urllib.request.urlopen(f"{ENDPOINT}/?{urlencode(query)}", timeout=10) as resp:
+ answer = resp.status, resp.read().decode()
+ except urllib.error.HTTPError as exc:
+ answer = exc.code, exc.read().decode()
+ if auth_enabled:
+ assert answer[0] == 403 and "<Code>MissingAuthenticationToken</Code>" in answer[1]
+ else:
+ assert answer[0] == 200
+
+
+def test_operations_aws_serves_without_credentials_contract(auth_enabled):
+ """Operations the SDKs send unsigned, the SNS subscription links and the health endpoint are still served."""
+ import urllib.request
+ import uuid
+
+ from conftest import ENDPOINT, make_client
+
+ with urllib.request.urlopen(f"{ENDPOINT}/_ministack/health", timeout=10) as resp:
+ assert resp.status == 200
+ idp, identity = _anonymous_client("cognito-idp"), _anonymous_client("cognito-identity")
+ assert _refusal(lambda: idp.initiate_auth(
+ ClientId="noclient", AuthFlow="USER_PASSWORD_AUTH", AuthParameters={"USERNAME": "u", "PASSWORD": "p"},
+ )) == ("ResourceNotFoundException", 400)
+ pool = make_client("cognito-identity").create_identity_pool(
+ IdentityPoolName=f"anonymous{uuid.uuid4().hex[:8]}", AllowUnauthenticatedIdentities=True)["IdentityPoolId"]
+ try:
+ assert identity.get_id(IdentityPoolId=pool)["IdentityId"]
+ finally:
+ make_client("cognito-identity").delete_identity_pool(IdentityPoolId=pool)
+ assumed = _refusal(lambda: _anonymous_client("sts").assume_role_with_web_identity(
+ RoleArn="arn:aws:iam::000000000000:role/none", RoleSessionName="session", WebIdentityToken="not-a-token"))
+ assert assumed is None or assumed[0] != "MissingAuthenticationToken"
+ sns = make_client("sns")
+ topic = sns.create_topic(Name=f"anonymous-{uuid.uuid4().hex[:8]}")["TopicArn"]
+ try:
+ assert _refusal(lambda: _anonymous_client("sns").confirm_subscription(TopicArn=topic, Token="0" * 64))[0] in (
+ "InvalidParameter", "InvalidParameterException")
+ finally:
+ sns.delete_topic(TopicArn=topic)
+ assert _refusal(lambda: idp.list_user_pools(MaxResults=1)) == (
+ ("MissingAuthenticationTokenException", 400) if auth_enabled else None)
+
+
@pytest.mark.auth_contract
def test_s3_sigv2_signatures_are_verified_contract(auth_enabled, iam_user):
"""SigV2 header and query signatures are checked for an IAM user, and still work when they are right."""