Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(eks): support AWS CLI kubeconfig authentication (#1717)

Johannes Grumböck27221e8e57db
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c69791f6..e044edc1 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,11 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Added
+- **EKS — managed Access Policy authorization in k3s** — `AmazonEKSClusterAdminPolicy`, `AmazonEKSAdminPolicy`, `AmazonEKSEditPolicy`, `AmazonEKSViewPolicy`, and `AmazonEKSAdminViewPolicy` now grant their published Kubernetes permissions under `AUTH=true`. Cluster and namespace scopes, including namespace wildcard patterns, are reconciled to managed k3s RBAC bindings after policy changes and when matching namespaces appear; disassociating a policy or deleting its Access Entry revokes the bindings.
+
+### Fixed
+- **EKS — AWS CLI kubeconfig authentication** — k3s accepts `aws eks get-token` credentials through a MiniStack TokenReview webhook, so `aws eks update-kubeconfig` works without copying a container's admin kubeconfig. `AUTH=false` permits local bearer tokens; `AUTH=true` validates IAM credentials and requires creator or Access Entry grants, including account-scoped STS sessions. Fixes #1140.
### Fixed
- **IAM — a resource-scoped policy matches, instead of denying** — `extract_resource_arn` returned `*` for three whole families of request, and a resource-scoped policy can never match `*`, so under `AUTH=true` the only grant that worked was one written as `Resource: "*"`. First, IoT mapped six path prefixes to ARNs but not `/jobs` or `/provisioning-templates`, and neither data plane had a branch at all: a client signs those with the `iotdata` and `iotjobsdata` scopes, so they arrive as their own service keys and fell through. That rules out what the CDK emits: its `AwsCustomResource` for an IoT job grants `iot:CreateJob` on exactly `job/<jobId>` plus the target `thinggroup/<name>`, the scoping AWS documents, and a CloudFormation stack carrying one failed to deploy here with `is not authorized to perform: iot:CreateJob` while deploying on AWS. A publish and a retained message now resolve to their topic, which carries every level (`topic/sensors/rack-1/temperature`), and a shadow or job execution to its thing. `Publish` also resolves an action again: its URI label is non-greedy, so the SDK percent-encodes the separators and the ASGI server hands them back decoded, and a route compiled as one segment then matched no topic below the first level. Second, the query-protocol services (EC2, CloudFormation, CloudWatch, Auto Scaling, ElastiCache) carry their parameters in a form-encoded body when the SDK POSTs, which botocore does, so nothing reached the query string and every branch reading it resolved nothing: `ec2:AuthorizeSecurityGroupIngress` scoped to one security group was denied for that group. The router already parsed that body to find `Action` and now keeps the rest of it, merged underneath the query string, which still wins. Third, an `execute-api` invoke was authorized against `*` as well, so a policy naming one API and stage, which is what `grantExecute` and every service-to-service grant emit, never matched; it now resolves to `arn:aws:execute-api:<region>:<account>:<api-id>/<stage>/<METHOD>/<path>`, built by the same helper the Lambda authorizer's method ARN uses, and authorized once the stage is known so a v2 API on `$default` is not charged for a stage segment it does not have. Contributed by @iot-rocket.
- **CloudFormation — `AWS::IAM::Policy` is provisioned inline, not as a managed policy** — the document was stored as an account-global customer-managed policy and attached to the Roles, Users and Groups the resource named, where CloudFormation embeds it on each of them the way `PutRolePolicy`, `PutUserPolicy` and `PutGroupPolicy` do. The difference is that an inline policy is scoped to its entity, so the same `PolicyName` can exist on many entities at once, and CDK relies on it: it derives `PolicyName` from the construct path, so every stack granting something to a role reached by the same path emits the same name. Twenty of one deployment's twenty-seven policies collapsed onto two records, and each deploy took the grants away from the stacks before it; under `AUTH=true` those roles were then denied the actions their own templates granted. The document now goes onto each entity through the existing inline handlers, so `ListRolePolicies`, `GetRolePolicy` and the `RolePolicyList` of `GetAccountAuthorizationDetails` report it and the IAM evaluator finds it. The physical id is generated rather than being the `PolicyName`, which is what AWS returns: measured against the account, a stack with an explicit `PolicyName` of `ms0049-refprobe-explicit-name` answers `Ref` with `ms0049--Pol-AW24HieEB0iX`, and `list-policies --scope Local` over an estate of forty deployed stacks counts zero. `AWS::IAM::ManagedPolicy` is unchanged. Contributed by @iot-rocket.
diff --git a/README.md b/README.md
index b74205e9..021cf98c 100644
--- a/README.md
+++ b/README.md
@@ -808,7 +808,7 @@ end-to-end without any client config.
| Variable | Default | Description |
|----------|---------|-------------|
| `GATEWAY_PORT` | `4566` | Port to listen on. Also accepts `EDGE_PORT` (LocalStack compatibility alias) |
-| `MINISTACK_HOST` | `localhost` | Hostname used in response URLs (SQS queues, SNS subscriptions, API Gateway endpoints, Lambda layers) |
+| `MINISTACK_HOST` | `localhost` | Hostname used in response URLs and EKS cluster endpoints. Set a hostname reachable by your clients, not a wildcard bind address |
| `MINISTACK_ACCOUNT_ID` | `000000000000` | Default AWS account ID. Overridden per-request when `AWS_ACCESS_KEY_ID` is a 12-digit number (see [Multi-Tenancy](#multi-tenancy)) |
| `MINISTACK_REGION` | `us-east-1` | AWS region reported in ARNs and service responses across all services |
| `LOG_LEVEL` | `INFO` | `DEBUG`, `INFO`, `WARNING`, `ERROR` |
@@ -1067,19 +1067,26 @@ describe can find nothing where AWS would still answer.
MiniStack's EKS spawns a real [k3s](https://k3s.io) cluster (75 MB image) when you create a cluster. `kubectl`, Helm, and any Kubernetes tooling work out of the box.
```bash
+# Local credentials for the AWS CLI. With AUTH=false any dummy key/secret
+# works; kubectl's aws eks get-token plugin inherits these variables.
+unset AWS_PROFILE AWS_SESSION_TOKEN
+export AWS_ACCESS_KEY_ID=test
+export AWS_SECRET_ACCESS_KEY=test
+export AWS_DEFAULT_REGION=eu-central-1
+
# Create an EKS cluster — k3s starts automatically
aws --endpoint-url=http://localhost:4566 eks create-cluster \
--name my-cluster --role-arn arn:aws:iam::000000000000:role/eks \
--resources-vpc-config subnetIds=subnet-1
-# Get the k3s kubeconfig (container name follows
-# ministack-eks-{region}-{name}; this example uses eu-central-1)
-docker exec ministack-eks-eu-central-1-my-cluster cat /etc/rancher/k3s/k3s.yaml \
- | sed "s/127.0.0.1:6443/localhost:$(docker port ministack-eks-eu-central-1-my-cluster 6443/tcp | cut -d: -f2)/" \
- > /tmp/ministack-kubeconfig.yaml
+# Wait for k3s to finish booting so DescribeCluster includes its CA bundle.
+aws --endpoint-url=http://localhost:4566 eks wait cluster-active --name my-cluster
-# Use kubectl against real Kubernetes
-export KUBECONFIG=/tmp/ministack-kubeconfig.yaml
+# Generate a normal AWS EKS kubeconfig. k3s sends the aws eks get-token
+# credential to MiniStack's authentication webhook.
+aws --endpoint-url=http://localhost:4566 eks update-kubeconfig --name my-cluster
+
+# Use kubectl against real Kubernetes (the default ~/.kube/config is updated)
kubectl get nodes # Real k3s node, Ready status
kubectl create deployment nginx --image=nginx:alpine
kubectl get pods # Real pod running
@@ -1092,6 +1099,48 @@ helm install my-redis bitnami/redis --set auth.enabled=false
aws --endpoint-url=http://localhost:4566 eks delete-cluster --name my-cluster
```
+To enforce IAM, start MiniStack with `AUTH=true` (for example,
+`AUTH=true python -m ministack`, or add `AUTH=true` to the container environment).
+The root key/secret default to `test`/`test`; custom values must match the
+`AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` of the MiniStack server.
+Under `AUTH=true`, create the IAM role before running the cluster example above:
+
+```bash
+aws --endpoint-url=http://localhost:4566 iam create-role \
+ --role-name eks \
+ --assume-role-policy-document \
+ '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"eks.amazonaws.com"},"Action":"sts:AssumeRole"}]}'
+```
+
+In strict mode, IAM policies govern EKS API calls such as `DescribeCluster`
+(needed by `update-kubeconfig`). Kubernetes access requires a separate grant:
+the principal that creates the cluster gets bootstrap administrator access unless
+`--access-config bootstrapClusterCreatorAdminPermissions=false` is supplied.
+The cluster's service role does not inherit that access. Other principals need
+an EKS Access Entry with `kubernetesGroups` bound through Kubernetes RBAC, or
+one of these EKS managed access policies:
+
+| Policy | Kubernetes permissions |
+|---|---|
+| `AmazonEKSClusterAdminPolicy` | Full cluster administration |
+| `AmazonEKSAdminPolicy` | Broad administration, including namespace Roles and RoleBindings |
+| `AmazonEKSEditPolicy` | Modify workloads and Secrets, but not RBAC administration |
+| `AmazonEKSViewPolicy` | Read workloads, excluding Secrets |
+| `AmazonEKSAdminViewPolicy` | Read all resources, including Secrets |
+
+Policies can use cluster or namespace scope. Namespace entries accept wildcard
+patterns such as `dev-*`; MiniStack reconciles their RBAC bindings when a
+matching namespace is created, and after association updates or deletions.
+EKS normally evaluates these policies in its control-plane authorizer. k3s has
+only Kubernetes RBAC, so MiniStack materializes equivalent, managed RBAC roles
+and bindings internally. User-supplied `kubernetesGroups` remain additive.
+Existing clusters without a recorded creator also need an explicit Access Entry.
+
+With `AUTH=false`, neither IAM policies nor Access Entries restrict Kubernetes
+access. The CLI still needs credentials to generate its exec token, but MiniStack
+does not check their secret, registration, or permissions. Restart MiniStack to
+change `AUTH`; k3s may retain previous authentication results for up to five minutes.
+
> **Note:** EKS requires Docker socket access (`-v /var/run/docker.sock:/var/run/docker.sock`) to spawn k3s containers. The k3s image is pulled on first `CreateCluster` call.
> **Security trade-off:** the k3s container is launched with `--privileged`. k3s server mode needs to remount `/sys/fs/cgroup`, which no granular Linux capability set permits — running unprivileged fails with `failed to evacuate root cgroup`. This grants the k3s container significant access on the Docker host. The trade-off is acceptable for local development against an emulator but should be considered before running MiniStack EKS on shared infrastructure. Omitting the Docker socket mount cleanly disables k3s and falls back to a static EKS mock.
diff --git a/ministack/app.py b/ministack/app.py
index 59052094..5705366b 100644
--- a/ministack/app.py
+++ b/ministack/app.py
@@ -2171,7 +2171,14 @@ def _unknown_query_error(body: bytes, request_id: str):
async def _dispatch_service_request(
method: str, path: str, headers: dict, body: bytes, query_params: dict, request_id: str
):
- """Dispatch a request through the generic service router."""
+ """Dispatch AWS service requests and Kubernetes TokenReviews."""
+ if method == "POST" and path.startswith("/eks-auth/"):
+ # TokenReview carries its own credentials; EKS authenticates the token.
+ return await _get_module("eks").handle_request(
+ method, path, headers, body, query_params
+ )
+
+ # Generic AWS service routing and IAM enforcement
routing_params = _routing_params(method, path, headers, body, query_params)
service = detect_service(method, path, headers, routing_params)
diff --git a/ministack/services/eks.py b/ministack/services/eks.py
index c98a26de..3cc63a38 100644
--- a/ministack/services/eks.py
+++ b/ministack/services/eks.py
@@ -12,11 +12,15 @@ Supports:
Nodegroups: CreateNodegroup, DescribeNodegroup, ListNodegroups, DeleteNodegroup
IdP configs: AssociateIdentityProviderConfig, DescribeIdentityProviderConfig,
DisassociateIdentityProviderConfig, ListIdentityProviderConfigs
+ Authentication: AWS IAM exec tokens through a k3s TokenReview webhook
Tags: TagResource, UntagResource, ListTagsForResource
"""
import base64
import copy
+import datetime as dt
+import fnmatch
+import hashlib
import importlib
import json
import logging
@@ -29,15 +33,30 @@ import urllib.parse
from ministack.core import container_reaper
from ministack.core.arn import ArnParseError, parse_arn
from ministack.core.concurrency import run_reentrant
+from ministack.core.iam_evaluator import (
+ AmbiguousAccessKeyError,
+ CredentialResolutionError,
+ find_iam_access_key_account,
+ resolve_caller_identity,
+ resolve_credential,
+)
from ministack.core.persistence import load_state
from ministack.core.responses import (
AccountRegionScopedDict,
AccountScopedDict,
+ _account_from_sts_session,
apply_image_prefix,
get_account_id,
get_region,
new_uuid,
)
+from ministack.core.router import extract_access_key_id
+from ministack.core.sigv4 import (
+ build_canonical_request,
+ build_string_to_sign,
+ calculate_signature,
+ signatures_match,
+)
logger = logging.getLogger("eks")
@@ -75,6 +94,23 @@ _port_counter_lock = threading.Lock()
_port_counter = [EKS_BASE_PORT]
_oidc_keypair_lock = threading.Lock()
_oidc_keypair = None # (private_key, jwk_dict, kid)
+_rbac_reconcilers_lock = threading.Lock()
+_rbac_reconcilers = {} # (account, region, cluster) -> (stop, wake)
+_rbac_reconcile_locks = {} # (account, region, cluster) -> Lock
+_EKS_RBAC_RECONCILE_INTERVAL = float(
+ os.environ.get("EKS_RBAC_RECONCILE_INTERVAL", "2")
+)
+
+
+def _cluster_endpoint(port):
+ """The endpoint DescribeCluster advertises for the kube-apiserver — the
+ host-published port ``https://{MINISTACK_HOST}:{port}``. The k3s container
+ publishes 6443 to this host port (``ports={"6443/tcp": port}``), so it is
+ reachable from the host (``aws eks update-kubeconfig`` + kubectl) and from
+ containers that can route to ``MINISTACK_HOST``. The same value is used on
+ every path (create, restart, restore).
+ """
+ return f"https://{_MINISTACK_HOST}:{port}"
def _ministack_issuer_base():
@@ -134,6 +170,7 @@ def _get_oidc_keypair():
def reset():
+ _stop_all_rbac_reconcilers()
_clusters.clear()
_nodegroups.clear()
_addons.clear()
@@ -240,7 +277,7 @@ def restore_state(data):
c["_docker_id"] = None
port = c.get("_port")
if port:
- c["endpoint"] = f"https://{_MINISTACK_HOST}:{port}"
+ c["endpoint"] = _cluster_endpoint(port)
@@ -328,17 +365,6 @@ def _get_ministack_network(client):
return None
-def _cluster_endpoint(port):
- """The endpoint DescribeCluster advertises for the kube-apiserver — the
- host-published port ``https://{MINISTACK_HOST}:{port}``. The k3s container
- publishes 6443 to this host port (``ports={"6443/tcp": port}``), so it is
- reachable from the host (``aws eks update-kubeconfig`` + kubectl) and from
- containers that can route to ``MINISTACK_HOST``. The same value is used on
- every path (create, restart, restore).
- """
- return f"https://{_MINISTACK_HOST}:{port}"
-
-
def _collect_oidc_state(cluster_name: str):
"""Return (apiserver_args, cfg_refs) for OIDC configs on a cluster.
@@ -406,6 +432,8 @@ def _k3s_run_kwargs(
"--disable=traefik,metrics-server,servicelb",
"--tls-san=0.0.0.0",
"--https-listen-port=6443",
+ "--kube-apiserver-arg=authentication-token-webhook-config-file=/etc/rancher/k3s/eks-auth-webhook.yaml",
+ "--kube-apiserver-arg=authentication-token-webhook-cache-ttl=5m",
]
if oidc_args:
command.extend(oidc_args)
@@ -444,6 +472,55 @@ def _k3s_run_kwargs(
return run_kwargs
+def _k3s_gateway_host(client, ms_network):
+ """Return the address a k3s container can use to call MiniStack."""
+ if ms_network and client is not None:
+ try:
+ self_container = client.containers.get(os.environ.get("HOSTNAME", ""))
+ nets = self_container.attrs["NetworkSettings"]["Networks"]
+ host = (nets.get(ms_network) or {}).get("IPAddress")
+ if host:
+ return host
+ except Exception:
+ pass
+ return "host.docker.internal"
+
+
+def _k3s_auth_webhook_config(client, ms_network, cluster_name, account_id=None, region=None):
+ """Build the kubeconfig consumed by the k3s token webhook authenticator.
+
+ The webhook endpoint is deliberately addressed through MiniStack's
+ container-network address rather than the host-published EKS port. This
+ works for both a shared Docker network and a host-run gateway.
+ """
+ from ministack.core import tls as _tls
+
+ host = _k3s_gateway_host(client, ms_network)
+ port = os.environ.get("GATEWAY_PORT") or os.environ.get("EDGE_PORT") or "4566"
+ scheme = "https" if _tls.use_ssl_enabled() else "http"
+ account_id = account_id or get_account_id()
+ region = region or get_region()
+ server = f"{scheme}://{host}:{port}/eks-auth/{account_id}/{region}/{cluster_name}"
+ return (
+ "apiVersion: v1\n"
+ "kind: Config\n"
+ "clusters:\n"
+ "- name: ministack-eks-auth\n"
+ " cluster:\n"
+ f" server: {server}\n"
+ " insecure-skip-tls-verify: true\n"
+ "users:\n"
+ "- name: ministack-eks-auth\n"
+ " user: {}\n"
+ "contexts:\n"
+ "- name: ministack-eks-auth\n"
+ " context:\n"
+ " cluster: ministack-eks-auth\n"
+ " user: ministack-eks-auth\n"
+ "current-context: ministack-eks-auth\n"
+ ).encode()
+
+
def _ecr_registry_hosts(cluster: dict) -> list[str]:
"""The local ECR registry hostnames this cluster's nodes should resolve
to MiniStack — derived from the cluster ARN (not contextvars) so the
@@ -493,13 +570,16 @@ def _k3s_registries_yaml(client, ms_network: str | None, ecr_hosts: list[str]) -
return ("\n".join(lines) + "\n").encode()
-def _start_k3s_container(client, run_kwargs: dict, registries_yaml: bytes | None):
+def _start_k3s_container(
+ client, run_kwargs: dict, registries_yaml: bytes | None,
+ auth_webhook_config: bytes | None = None,
+):
"""Start the k3s container, injecting registries.yaml before boot.
k3s reads /etc/rancher/k3s/registries.yaml once at startup, so the file
must exist before the entrypoint runs: create the container stopped,
upload the file with put_archive, then start it."""
- if not registries_yaml:
+ if not registries_yaml and not auth_webhook_config:
return client.containers.run(**run_kwargs)
import io
import tarfile
@@ -515,13 +595,19 @@ def _start_k3s_container(client, run_kwargs: dict, registries_yaml: bytes | None
try:
buf = io.BytesIO()
with tarfile.open(fileobj=buf, mode="w") as tar:
- info = tarfile.TarInfo("etc/rancher/k3s/registries.yaml")
- info.size = len(registries_yaml)
- info.mode = 0o644
- tar.addfile(info, io.BytesIO(registries_yaml))
+ for filename, content in (
+ ("etc/rancher/k3s/registries.yaml", registries_yaml),
+ ("etc/rancher/k3s/eks-auth-webhook.yaml", auth_webhook_config),
+ ):
+ if not content:
+ continue
+ info = tarfile.TarInfo(filename)
+ info.size = len(content)
+ info.mode = 0o644
+ tar.addfile(info, io.BytesIO(content))
container.put_archive("/", buf.getvalue())
except Exception as e:
- logger.warning("EKS: could not inject ECR registries.yaml: %s", e)
+ logger.warning("EKS: could not inject k3s startup configuration: %s", e)
container.start()
return container
@@ -561,7 +647,7 @@ def _extract_ca_cert(container, timeout=30):
# Clusters
# ---------------------------------------------------------------------------
-def _create_cluster(body):
+def _create_cluster(body, creator_arn=None):
name = body.get("name", "")
if not name:
return _error(400, "InvalidParameterException", "Cluster name is required.")
@@ -586,7 +672,7 @@ def _create_cluster(body):
# Build cluster record immediately (status CREATING) and return fast.
# k3s startup happens in background thread to avoid blocking the event loop.
- endpoint = f"https://{_MINISTACK_HOST}:{port}"
+ endpoint = _cluster_endpoint(port)
cluster = {
"name": name,
"arn": arn,
@@ -617,6 +703,7 @@ def _create_cluster(body):
"tags": body.get("tags", {}),
"encryptionConfig": body.get("encryptionConfig", []),
"accessConfig": body.get("accessConfig", {}),
+ "_creator_arn": creator_arn,
"_docker_id": None,
"_port": port,
}
@@ -625,7 +712,7 @@ def _create_cluster(body):
if cluster["tags"]:
_tags[arn] = dict(cluster["tags"])
- region = get_region()
+ account_id, region = get_account_id(), get_region()
oidc_args, _idp_cfg_refs = _collect_oidc_state(name)
node_labels = _collect_node_labels(cluster)
@@ -648,18 +735,24 @@ def _create_cluster(body):
)
registries_yaml = _k3s_registries_yaml(client, ms_network, _ecr_registry_hosts(cluster))
- container = _start_k3s_container(client, run_kwargs, registries_yaml)
+ auth_webhook_config = _k3s_auth_webhook_config(
+ client, ms_network, name, account_id, region
+ )
+ container = _start_k3s_container(
+ client, run_kwargs, registries_yaml, auth_webhook_config
+ )
cluster["_docker_id"] = container.id
cluster["endpoint"] = _cluster_endpoint(port)
cluster["certificateAuthority"]["data"] = _extract_ca_cert(container)
cluster["status"] = "ACTIVE"
+ _schedule_access_policy_reconcile(name, account_id, region)
except Exception as e:
logger.warning("EKS: failed to start k3s for %s — falling back to mock: %s", name, e)
cluster["status"] = "ACTIVE"
cluster["certificateAuthority"]["data"] = base64.b64encode(b"MOCK-CA-CERTIFICATE").decode()
# No container came up — advertise the host-published endpoint.
- cluster["endpoint"] = f"https://{_MINISTACK_HOST}:{port}"
+ cluster["endpoint"] = _cluster_endpoint(port)
threading.Thread(target=_bg_start, daemon=True, name=f"eks-{name}").start()
return _json_resp(200, {"cluster": _sanitize(cluster)})
@@ -683,6 +776,8 @@ def _delete_cluster(name):
if not cluster:
return _error(404, "ResourceNotFoundException", f"No cluster found for name: {name}.")
+ _stop_access_policy_reconciler(name, get_account_id(), get_region())
+
# Stop k3s container
container_id = cluster.get("_docker_id")
if container_id:
@@ -984,6 +1079,7 @@ def _delete_access_entry(cluster_name, principal_arn):
_access_policies.pop(ak, None)
_tags.pop(entry.get("accessEntryArn", ""), None)
_access_entries.pop(key, None)
+ _schedule_access_policy_reconcile(cluster_name)
return _json_resp(200, {})
@@ -1002,9 +1098,13 @@ def _update_access_entry(cluster_name, principal_arn, body):
def _associate_access_policy(cluster_name, principal_arn, body):
- if _ae_key(cluster_name, principal_arn) not in _access_entries:
+ entry = _access_entries.get(_ae_key(cluster_name, principal_arn))
+ if not entry:
return _error(404, "ResourceNotFoundException",
f"No access entry for principal {principal_arn}.")
+ if entry["type"] != "STANDARD":
+ return _error(400, "InvalidRequestException",
+ "Access policies can only be associated with STANDARD access entries.")
policy_arn = body.get("policyArn", "")
if not policy_arn:
return _error(400, "InvalidParameterException",
@@ -1014,20 +1114,31 @@ def _associate_access_policy(cluster_name, principal_arn, body):
if scope_type not in ("cluster", "namespace"):
return _error(400, "InvalidParameterException",
"accessScope.type must be 'cluster' or 'namespace'.")
- if scope_type == "namespace" and not access_scope.get("namespaces"):
- return _error(400, "InvalidParameterException",
- "namespaces is required when accessScope.type is 'namespace'.")
+ namespaces = access_scope.get("namespaces", [])
+ if scope_type == "namespace":
+ if (
+ not isinstance(namespaces, list)
+ or not namespaces
+ or not all(isinstance(namespace, str) and namespace for namespace in namespaces)
+ ):
+ return _error(400, "InvalidParameterException",
+ "namespaces must be a nonempty list when accessScope.type is 'namespace'.")
+ else:
+ namespaces = []
now = _now()
+ key = _ap_key(cluster_name, principal_arn, policy_arn)
+ existing = _access_policies.get(key) or {}
associated = {
"policyArn": policy_arn,
"accessScope": {
"type": scope_type,
- "namespaces": access_scope.get("namespaces", []),
+ "namespaces": namespaces,
},
- "associatedAt": now,
+ "associatedAt": existing.get("associatedAt", now),
"modifiedAt": now,
}
- _access_policies[_ap_key(cluster_name, principal_arn, policy_arn)] = associated
+ _access_policies[key] = associated
+ _schedule_access_policy_reconcile(cluster_name)
return _json_resp(200, {
"clusterName": cluster_name,
"principalArn": principal_arn,
@@ -1044,6 +1155,7 @@ def _disassociate_access_policy(cluster_name, principal_arn, policy_arn):
return _error(404, "ResourceNotFoundException",
f"Policy {policy_arn} is not associated with {principal_arn}.")
_access_policies.pop(key, None)
+ _schedule_access_policy_reconcile(cluster_name)
return _json_resp(200, {})
@@ -1061,6 +1173,358 @@ def _list_associated_access_policies(cluster_name, principal_arn, query):
})
+# ---------------------------------------------------------------------------
+# Access policy RBAC materialization
+# ---------------------------------------------------------------------------
+
+_ACCESS_POLICY_ARN_PREFIX = "cluster-access-policy/"
+_ACCESS_POLICY_BINDING_LABEL = "ministack.org/eks-access-policy"
+_ACCESS_POLICY_MANAGED_LABEL = "ministack.org/managed"
+
+# These are the published Kubernetes rules for the five general-purpose EKS
+# access policies. EKS normally evaluates them in its proprietary authorizer;
+# k3s has only Kubernetes RBAC, so MiniStack materializes equivalent roles.
+# Keep this table explicit rather than referring to the built-in user-facing
+# roles: their rules are only similar to, rather than identical to, EKS policy.
+_ACCESS_POLICY_RULES = {
+ "AmazonEKSClusterAdminPolicy": [
+ {"apiGroups": ["*"], "resources": ["*"], "verbs": ["*"]},
+ {"nonResourceURLs": ["*"], "verbs": ["*"]},
+ ],
+ "AmazonEKSAdminPolicy": [
+ {"apiGroups": ["apps"], "resources": ["daemonsets", "deployments", "deployments/rollback", "deployments/scale", "replicasets", "replicasets/scale", "statefulsets", "statefulsets/scale"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["apps"], "resources": ["controllerrevisions", "daemonsets", "daemonsets/status", "deployments", "deployments/scale", "deployments/status", "replicasets", "replicasets/scale", "replicasets/status", "statefulsets", "statefulsets/scale", "statefulsets/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["authorization.k8s.io"], "resources": ["localsubjectaccessreviews"], "verbs": ["create"]},
+ {"apiGroups": ["autoscaling"], "resources": ["horizontalpodautoscalers"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["autoscaling"], "resources": ["horizontalpodautoscalers", "horizontalpodautoscalers/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["batch"], "resources": ["cronjobs", "jobs"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["batch"], "resources": ["cronjobs", "cronjobs/status", "jobs", "jobs/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["discovery.k8s.io"], "resources": ["endpointslices"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["extensions"], "resources": ["daemonsets", "deployments", "deployments/rollback", "deployments/scale", "ingresses", "networkpolicies", "replicasets", "replicasets/scale", "replicationcontrollers/scale"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["extensions"], "resources": ["daemonsets", "daemonsets/status", "deployments", "deployments/scale", "deployments/status", "ingresses", "ingresses/status", "networkpolicies", "replicasets", "replicasets/scale", "replicasets/status", "replicationcontrollers/scale"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["networking.k8s.io"], "resources": ["ingresses", "ingresses/status", "networkpolicies"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["networking.k8s.io"], "resources": ["ingresses", "networkpolicies"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["policy"], "resources": ["poddisruptionbudgets"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["policy"], "resources": ["poddisruptionbudgets", "poddisruptionbudgets/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["rbac.authorization.k8s.io"], "resources": ["rolebindings", "roles"], "verbs": ["create", "delete", "deletecollection", "get", "list", "patch", "update", "watch"]},
+ {"apiGroups": [""], "resources": ["configmaps", "endpoints", "persistentvolumeclaims", "persistentvolumeclaims/status", "pods", "replicationcontrollers", "replicationcontrollers/scale", "serviceaccounts", "services", "services/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["pods/attach", "pods/exec", "pods/portforward", "pods/proxy", "secrets", "services/proxy"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["configmaps", "events", "persistentvolumeclaims", "replicationcontrollers", "replicationcontrollers/scale", "secrets", "serviceaccounts", "services", "services/proxy"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": [""], "resources": ["pods", "pods/attach", "pods/exec", "pods/portforward", "pods/proxy"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": [""], "resources": ["serviceaccounts"], "verbs": ["impersonate"]},
+ {"apiGroups": [""], "resources": ["bindings", "events", "limitranges", "namespaces/status", "pods/log", "pods/status", "replicationcontrollers/status", "resourcequotas", "resourcequotas/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["namespaces"], "verbs": ["get", "list", "watch"]},
+ ],
+ "AmazonEKSEditPolicy": [
+ {"apiGroups": ["apps"], "resources": ["daemonsets", "deployments", "deployments/rollback", "deployments/scale", "replicasets", "replicasets/scale", "statefulsets", "statefulsets/scale"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["apps"], "resources": ["controllerrevisions", "daemonsets", "daemonsets/status", "deployments", "deployments/scale", "deployments/status", "replicasets", "replicasets/scale", "replicasets/status", "statefulsets", "statefulsets/scale", "statefulsets/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["autoscaling"], "resources": ["horizontalpodautoscalers", "horizontalpodautoscalers/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["autoscaling"], "resources": ["horizontalpodautoscalers"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["batch"], "resources": ["cronjobs", "jobs"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["batch"], "resources": ["cronjobs", "cronjobs/status", "jobs", "jobs/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["discovery.k8s.io"], "resources": ["endpointslices"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["extensions"], "resources": ["daemonsets", "deployments", "deployments/rollback", "deployments/scale", "ingresses", "networkpolicies", "replicasets", "replicasets/scale", "replicationcontrollers/scale"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["extensions"], "resources": ["daemonsets", "daemonsets/status", "deployments", "deployments/scale", "deployments/status", "ingresses", "ingresses/status", "networkpolicies", "replicasets", "replicasets/scale", "replicasets/status", "replicationcontrollers/scale"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["networking.k8s.io"], "resources": ["ingresses", "networkpolicies"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["networking.k8s.io"], "resources": ["ingresses", "ingresses/status", "networkpolicies"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["policy"], "resources": ["poddisruptionbudgets"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": ["policy"], "resources": ["poddisruptionbudgets", "poddisruptionbudgets/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["namespaces"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["pods/attach", "pods/exec", "pods/portforward", "pods/proxy", "secrets", "services/proxy"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["serviceaccounts"], "verbs": ["impersonate"]},
+ {"apiGroups": [""], "resources": ["pods", "pods/attach", "pods/exec", "pods/portforward", "pods/proxy"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": [""], "resources": ["configmaps", "events", "persistentvolumeclaims", "replicationcontrollers", "replicationcontrollers/scale", "secrets", "serviceaccounts", "services", "services/proxy"], "verbs": ["create", "delete", "deletecollection", "patch", "update"]},
+ {"apiGroups": [""], "resources": ["configmaps", "endpoints", "persistentvolumeclaims", "persistentvolumeclaims/status", "pods", "replicationcontrollers", "replicationcontrollers/scale", "serviceaccounts", "services", "services/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["bindings", "events", "limitranges", "namespaces/status", "pods/log", "pods/status", "replicationcontrollers/status", "resourcequotas", "resourcequotas/status"], "verbs": ["get", "list", "watch"]},
+ ],
+ "AmazonEKSViewPolicy": [
+ {"apiGroups": ["apps"], "resources": ["controllerrevisions", "daemonsets", "daemonsets/status", "deployments", "deployments/scale", "deployments/status", "replicasets", "replicasets/scale", "replicasets/status", "statefulsets", "statefulsets/scale", "statefulsets/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["autoscaling"], "resources": ["horizontalpodautoscalers", "horizontalpodautoscalers/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["batch"], "resources": ["cronjobs", "cronjobs/status", "jobs", "jobs/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["discovery.k8s.io"], "resources": ["endpointslices"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["extensions"], "resources": ["daemonsets", "daemonsets/status", "deployments", "deployments/scale", "deployments/status", "ingresses", "ingresses/status", "networkpolicies", "replicasets", "replicasets/scale", "replicasets/status", "replicationcontrollers/scale"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["networking.k8s.io"], "resources": ["ingresses", "ingresses/status", "networkpolicies"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": ["policy"], "resources": ["poddisruptionbudgets", "poddisruptionbudgets/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["configmaps", "endpoints", "persistentvolumeclaims", "persistentvolumeclaims/status", "pods", "replicationcontrollers", "replicationcontrollers/scale", "serviceaccounts", "services", "services/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["bindings", "events", "limitranges", "namespaces/status", "pods/log", "pods/status", "replicationcontrollers/status", "resourcequotas", "resourcequotas/status"], "verbs": ["get", "list", "watch"]},
+ {"apiGroups": [""], "resources": ["namespaces"], "verbs": ["get", "list", "watch"]},
+ ],
+ "AmazonEKSAdminViewPolicy": [
+ {"apiGroups": ["*"], "resources": ["*"], "verbs": ["get", "list", "watch"]},
+ ],
+}
+
+
+def _access_policy_name(policy_arn):
+ """Return the supported EKS policy name for an AWS partition ARN."""
+ if not isinstance(policy_arn, str):
+ return None
+ match = re.fullmatch(
+ rf"arn:[^:]+:eks::aws:{_ACCESS_POLICY_ARN_PREFIX}([^/]+)",
+ policy_arn,
+ )
+ if not match or match.group(1) not in _ACCESS_POLICY_RULES:
+ return None
+ return match.group(1)
+
+
+def _access_policy_role_name(policy_arn):
+ name = _access_policy_name(policy_arn)
+ return f"ministack-eks-{name.lower()}" if name else None
+
+
+def _access_policy_binding_id(cluster_name, principal_arn, policy_arn):
+ value = "\x00".join((cluster_name, principal_arn, policy_arn)).encode()
+ return hashlib.sha256(value).hexdigest()[:24]
+
+
+def _access_policy_group(cluster_name, principal_arn, policy_arn):
+ return "ministack:eks:access:" + _access_policy_binding_id(
+ cluster_name, principal_arn, policy_arn
+ )
+
+
+def _access_policy_binding_name(cluster_name, principal_arn, policy_arn):
+ return "ministack-eks-" + _access_policy_binding_id(
+ cluster_name, principal_arn, policy_arn
+ )
+
+
+def _access_policy_associations(cluster_name, account_id, region):
+ """Yield supported policy associations for one cluster and tenant."""
+ prefix = f"{cluster_name}\x00"
+ for (account, policy_region, key), association in _access_policies.all_items():
+ if account != account_id or policy_region != region or not str(key).startswith(prefix):
+ continue
+ _name, principal_arn, policy_arn = str(key).split("\x00", 2)
+ if _access_policy_name(policy_arn):
+ yield principal_arn, policy_arn, association
+
+
+def _access_policy_rbac_objects(cluster_name, account_id, region, namespaces):
+ """Build MiniStack-managed RBAC objects for the cluster's associations."""
+ objects = [
+ {
+ "apiVersion": "rbac.authorization.k8s.io/v1",
+ "kind": "ClusterRole",
+ "metadata": {
+ "name": _access_policy_role_name(
+ f"arn:aws:eks::aws:{_ACCESS_POLICY_ARN_PREFIX}{name}"
+ ),
+ "labels": {_ACCESS_POLICY_MANAGED_LABEL: "true"},
+ },
+ "rules": rules,
+ }
+ for name, rules in _ACCESS_POLICY_RULES.items()
+ ]
+ for principal_arn, policy_arn, association in _access_policy_associations(
+ cluster_name, account_id, region
+ ):
+ scope = association.get("accessScope", {})
+ role_name = _access_policy_role_name(policy_arn)
+ binding_name = _access_policy_binding_name(
+ cluster_name, principal_arn, policy_arn
+ )
+ subject = {
+ "kind": "Group",
+ "apiGroup": "rbac.authorization.k8s.io",
+ "name": _access_policy_group(cluster_name, principal_arn, policy_arn),
+ }
+ metadata = {
+ "name": binding_name,
+ "labels": {_ACCESS_POLICY_BINDING_LABEL: "true"},
+ }
+ role_ref = {
+ "apiGroup": "rbac.authorization.k8s.io",
+ "kind": "ClusterRole",
+ "name": role_name,
+ }
+ if scope.get("type") == "cluster":
+ objects.append({
+ "apiVersion": "rbac.authorization.k8s.io/v1",
+ "kind": "ClusterRoleBinding",
+ "metadata": metadata,
+ "subjects": [subject],
+ "roleRef": role_ref,
+ })
+ continue
+ patterns = scope.get("namespaces") or []
+ for namespace in sorted({n for n in namespaces if any(
+ fnmatch.fnmatchcase(n, pattern) for pattern in patterns
+ )}):
+ objects.append({
+ "apiVersion": "rbac.authorization.k8s.io/v1",
+ "kind": "RoleBinding",
+ "metadata": {**metadata, "namespace": namespace},
+ "subjects": [subject],
+ "roleRef": role_ref,
+ })
+ return objects
+
+
+def _k3s_exec(container, command):
+ result = container.exec_run(command)
+ if hasattr(result, "exit_code"):
+ exit_code, output = result.exit_code, result.output
+ elif isinstance(result, tuple):
+ exit_code = result[0]
+ output = result[1] if len(result) > 1 else b""
+ else:
+ exit_code, output = 0, b""
+ return exit_code, output.decode(errors="replace") if isinstance(output, bytes) else output
+
+
+def _apply_access_policy_rbac(container, objects):
+ """Apply a JSON Kubernetes List without relying on a container temp file."""
+ content = json.dumps({"apiVersion": "v1", "kind": "List", "items": objects}).encode()
+ payload = base64.b64encode(content).decode("ascii")
+ # docker-py's exec_run does not expose a simple stdin path with an exit
+ # status. The payload is base64, so it is safe to pass as one shell token;
+ # decoding it in the container avoids Docker archive writes to k3s's tmpfs.
+ command = ["sh", "-c", f"printf '%s' '{payload}' | base64 -d | kubectl apply -f -"]
+ code, output = _k3s_exec(container, command)
+ if code:
+ raise RuntimeError(output)
+
+
+def _managed_access_policy_bindings(container, kind):
+ command = ["kubectl", "get", kind.lower(), "-l", f"{_ACCESS_POLICY_BINDING_LABEL}=true", "-o", "json"]
+ if kind == "RoleBinding":
+ command.append("--all-namespaces")
+ code, output = _k3s_exec(container, command)
+ if code:
+ raise RuntimeError(output)
+ return json.loads(output).get("items", [])
+
+
+def _binding_key(obj):
+ return (
+ obj.get("kind"), obj.get("metadata", {}).get("namespace", ""),
+ obj.get("metadata", {}).get("name"),
+ )
+
+
+def _reconcile_access_policy_rbac(cluster_name, account_id, region):
+ """Reconcile managed bindings after access-policy or namespace changes."""
+ key = (account_id, region, cluster_name)
+ with _rbac_reconcilers_lock:
+ lock = _rbac_reconcile_locks.setdefault(key, threading.Lock())
+ with lock:
+ cluster = _clusters.get_scoped(account_id, region, cluster_name)
+ if not cluster or not cluster.get("_docker_id"):
+ return False
+ client = _get_docker()
+ if not client:
+ return False
+ try:
+ container = client.containers.get(cluster["_docker_id"])
+ code, output = _k3s_exec(container, ["kubectl", "get", "namespaces", "-o", "json"])
+ if code:
+ raise RuntimeError(output)
+ namespaces = [item["metadata"]["name"] for item in json.loads(output).get("items", [])]
+ objects = _access_policy_rbac_objects(cluster_name, account_id, region, namespaces)
+ desired = {
+ _binding_key(obj) for obj in objects
+ if obj["kind"] in ("ClusterRoleBinding", "RoleBinding")
+ }
+ _apply_access_policy_rbac(container, objects)
+ for kind in ("ClusterRoleBinding", "RoleBinding"):
+ for existing in _managed_access_policy_bindings(container, kind):
+ binding_key = _binding_key(existing)
+ if binding_key in desired:
+ continue
+ command = ["kubectl", "delete", kind.lower(), binding_key[2]]
+ if binding_key[1]:
+ command.extend(["--namespace", binding_key[1]])
+ code, output = _k3s_exec(container, command)
+ if code:
+ raise RuntimeError(output)
+ return True
+ except Exception as e:
+ logger.debug("EKS: RBAC reconciliation for %s failed: %s", cluster_name, e)
+ return False
+
+
+def _has_namespace_access_policy(cluster_name, account_id, region):
+ return any(
+ association.get("accessScope", {}).get("type") == "namespace"
+ for _principal, _policy, association in _access_policy_associations(
+ cluster_name, account_id, region
+ )
+ )
+
+
+def _schedule_access_policy_reconcile(cluster_name, account_id=None, region=None):
+ account_id = account_id or get_account_id()
+ region = region or get_region()
+ key = (account_id, region, cluster_name)
+ with _rbac_reconcilers_lock:
+ running = _rbac_reconcilers.get(key)
+ if running:
+ running[1].set()
+ return
+ stop = threading.Event()
+ wake = threading.Event()
+ state = (stop, wake)
+ _rbac_reconcilers[key] = state
+
+ def reconcile_loop():
+ try:
+ while not stop.is_set():
+ wake.clear()
+ reconciled = _reconcile_access_policy_rbac(
+ cluster_name, account_id, region
+ )
+ if not reconciled:
+ cluster = _clusters.get_scoped(account_id, region, cluster_name)
+ if not cluster or (
+ cluster.get("status") == "ACTIVE"
+ and not cluster.get("_docker_id")
+ ):
+ return
+ wake.wait(_EKS_RBAC_RECONCILE_INTERVAL)
+ continue
+ watches_namespaces = _has_namespace_access_policy(
+ cluster_name, account_id, region
+ )
+ with _rbac_reconcilers_lock:
+ if _rbac_reconcilers.get(key) is not state:
+ return
+ if wake.is_set():
+ continue
+ if not watches_namespaces:
+ _rbac_reconcilers.pop(key, None)
+ return
+ wake.wait(_EKS_RBAC_RECONCILE_INTERVAL)
+ finally:
+ with _rbac_reconcilers_lock:
+ if _rbac_reconcilers.get(key) is state:
+ _rbac_reconcilers.pop(key, None)
+
+ threading.Thread(
+ target=reconcile_loop, daemon=True,
+ name=f"eks-rbac-{cluster_name}",
+ ).start()
+
+
+def _stop_access_policy_reconciler(cluster_name, account_id, region):
+ with _rbac_reconcilers_lock:
+ state = _rbac_reconcilers.pop((account_id, region, cluster_name), None)
+ if state:
+ stop, wake = state
+ stop.set()
+ wake.set()
+
+
+def _stop_all_rbac_reconcilers():
+ with _rbac_reconcilers_lock:
+ states = list(_rbac_reconcilers.values())
+ _rbac_reconcilers.clear()
+ for stop, wake in states:
+ stop.set()
+ wake.set()
+
+
# ---------------------------------------------------------------------------
# Encryption config (AssociateEncryptionConfig)
# ---------------------------------------------------------------------------
@@ -1147,17 +1611,26 @@ def _restart_k3s(cluster_name, oidc_args=None, idp_cfg_refs=None):
)
registries_yaml = _k3s_registries_yaml(client, ms_network, _ecr_registry_hosts(cluster))
- container = _start_k3s_container(client, run_kwargs, registries_yaml)
+ cluster_spec = parse_arn(cluster.get("arn", ""))
+ auth_webhook_config = _k3s_auth_webhook_config(
+ client, ms_network, cluster_name, cluster_spec.account_id, cluster_spec.region
+ )
+ container = _start_k3s_container(
+ client, run_kwargs, registries_yaml, auth_webhook_config
+ )
cluster["_docker_id"] = container.id
cluster["endpoint"] = _cluster_endpoint(cluster["_port"])
cluster["certificateAuthority"]["data"] = _extract_ca_cert(container)
_mark_idp_active()
+ _schedule_access_policy_reconcile(
+ cluster_name, cluster_spec.account_id, cluster_spec.region
+ )
except Exception as e:
logger.warning("EKS: failed to restart k3s for %s — falling back to mock: %s", cluster_name, e)
cluster["certificateAuthority"]["data"] = base64.b64encode(b"MOCK-CA-CERTIFICATE").decode()
# No container came up — advertise the host-published endpoint.
- cluster["endpoint"] = f"https://{_MINISTACK_HOST}:{cluster['_port']}"
+ cluster["endpoint"] = _cluster_endpoint(cluster["_port"])
_mark_idp_active()
threading.Thread(target=_bg_restart, daemon=True, name=f"eks-restart-{cluster_name}").start()
@@ -1457,6 +1930,280 @@ def _list_tags(arn):
return _json_resp(200, {"tags": _tags.get(arn, {})})
+# ---------------------------------------------------------------------------
+# AWS IAM authenticator compatible TokenReview webhook
+# ---------------------------------------------------------------------------
+
+_EKS_TOKEN_PREFIX = "k8s-aws-v1."
+_EKS_TOKEN_VALIDITY_SECONDS = 15 * 60
+
+
+def _token_review_response(review, *, authenticated=False, username="", uid="", groups=None,
+ audiences=None):
+ status = {"authenticated": authenticated}
+ if authenticated:
+ status["user"] = {
+ "username": username,
+ "uid": uid,
+ "groups": groups or [],
+ }
+ if audiences:
+ status["audiences"] = audiences
+ return _json_resp(200, {
+ "apiVersion": review.get("apiVersion", "authentication.k8s.io/v1"),
+ "kind": "TokenReview",
+ "status": status,
+ })
+
+
+def _decode_aws_iam_token(token):
+ if not isinstance(token, str) or len(token) > 16384 or not token.startswith(_EKS_TOKEN_PREFIX):
+ return None
+ encoded = token[len(_EKS_TOKEN_PREFIX):]
+ try:
+ encoded += "=" * (-len(encoded) % 4)
+ raw = base64.urlsafe_b64decode(encoded.encode("ascii"))
+ url = raw.decode("utf-8")
+ parsed = urllib.parse.urlsplit(url)
+ except (ValueError, UnicodeDecodeError, UnicodeEncodeError):
+ return None
+ if parsed.scheme != "https" or not parsed.netloc or parsed.path not in ("", "/"):
+ return None
+ query = urllib.parse.parse_qs(parsed.query, keep_blank_values=True)
+ if any(len(values) != 1 for values in query.values()):
+ return None
+ return parsed, query
+
+
+def _qp(query, name):
+ values = query.get(name) or query.get(name.lower())
+ if isinstance(values, (list, tuple)):
+ return values[0] if values else ""
+ return values or ""
+
+
+def _verify_eks_token(token, cluster_name):
+ """Verify an AWS CLI EKS exec token without calling the real STS service.
+
+ ``aws eks get-token`` signs a GET to regional STS with the cluster name in
+ the ``x-k8s-aws-id`` signed header. The URL's advertised 60-second
+ presign lifetime is intentionally not used here: kubectl caches the
+ resulting ExecCredential for roughly 14 minutes and the upstream
+ authenticator accepts a 15-minute token window.
+ """
+ decoded = _decode_aws_iam_token(token)
+ if not decoded:
+ return None
+ parsed, query = decoded
+ if (
+ _qp(query, "X-Amz-Algorithm") != "AWS4-HMAC-SHA256"
+ or _qp(query, "Action") != "GetCallerIdentity"
+ or _qp(query, "Version") != "2011-06-15"
+ ):
+ return None
+ credential_scope = _qp(query, "X-Amz-Credential").split("/")
+ amz_date = _qp(query, "X-Amz-Date")
+ signed_headers = _qp(query, "X-Amz-SignedHeaders").lower()
+ signature = _qp(query, "X-Amz-Signature")
+ if (
+ len(credential_scope) != 5
+ or not credential_scope[0]
+ or not amz_date
+ or not re.fullmatch(r"[0-9a-f]{64}", signature)
+ or credential_scope[3] != "sts"
+ or credential_scope[4] != "aws4_request"
+ or "host" not in signed_headers.split(";")
+ or "x-k8s-aws-id" not in signed_headers.split(";")
+ ):
+ return None
+
+ host = parsed.netloc
+ hostname = parsed.hostname or ""
+ if not re.fullmatch(r"sts(?:[.-][a-z0-9-]+)?\.amazonaws\.com", hostname):
+ return None
+ if hostname != f"sts.{credential_scope[2]}.amazonaws.com":
+ return None
+ try:
+ if not 0 <= int(_qp(query, "X-Amz-Expires")) <= _EKS_TOKEN_VALIDITY_SECONDS:
+ return None
+ signed_at = dt.datetime.strptime(amz_date, "%Y%m%dT%H%M%SZ").replace(tzinfo=dt.timezone.utc)
+ except ValueError:
+ return None
+ age = (dt.datetime.now(dt.timezone.utc) - signed_at).total_seconds()
+ if age < -300 or age > _EKS_TOKEN_VALIDITY_SECONDS or credential_scope[1] != amz_date[:8]:
+ return None
+
+ # The cluster ID is not present in the presigned query string. It is the
+ # value of a signed header on the original STS request, so inject the
+ # cluster being authenticated before rebuilding the canonical request.
+ signed_request_headers = {
+ "host": host,
+ "x-k8s-aws-id": cluster_name,
+ }
+ canonical_request = build_canonical_request(
+ "GET",
+ parsed.path or "/",
+ signed_request_headers,
+ query,
+ signed_headers,
+ # botocore's STS presigner signs an empty GET payload. Accepting only
+ # this hash also keeps the verifier compatible with the authenticator.
+ payload_hash="e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
+ )
+ string_to_sign = build_string_to_sign(
+ amz_date,
+ credential_scope[1],
+ credential_scope[2],
+ credential_scope[3],
+ canonical_request,
+ )
+ access_key_id = credential_scope[0]
+ try:
+ account_id = find_iam_access_key_account(access_key_id)
+ except AmbiguousAccessKeyError:
+ return None
+ # TokenReview calls have no AWS Authorization header. Resolve every kind
+ # of key independently of the gateway request's account context.
+ account_id = account_id or _account_from_sts_session(access_key_id)
+ if re.fullmatch(r"\d{12}", access_key_id):
+ account_id = access_key_id
+ credential = resolve_credential(
+ access_key_id,
+ account_id or os.environ.get("MINISTACK_ACCOUNT_ID", "000000000000"),
+ _qp(query, "X-Amz-Security-Token") or None,
+ )
+ if isinstance(credential, CredentialResolutionError):
+ return None
+ expected = calculate_signature(
+ credential.secret_access_key,
+ credential_scope[1],
+ credential_scope[2],
+ credential_scope[3],
+ string_to_sign,
+ )
+ if not signatures_match(expected, signature):
+ return None
+ return credential
+
+
+def _cluster_for_auth(cluster_name, account_id=None, region=None):
+ """Find a cluster across scoped stores without leaking tenant state."""
+ matches = [
+ (account, cluster_region, cluster)
+ for (account, cluster_region, name), cluster in _clusters.all_items()
+ if name == cluster_name
+ and (account_id is None or account == account_id)
+ and (region is None or cluster_region == region)
+ ]
+ return matches[0] if len(matches) == 1 else None
+
+
+def _role_arn_from_assumed_role(arn):
+ match = re.fullmatch(r"arn:([^:]+):sts::([^:]+):assumed-role/([^/]+)/[^/]+", arn or "")
+ if not match:
+ return ""
+ partition, account, role = match.groups()
+ # STS session ARNs omit the IAM role's path. Recover it from IAM so that
+ # access entries for role/team/developer match assumed-role/developer/...
+ from ministack.services import iam
+ record = iam._roles.get_scoped(account, None, role)
+ if record:
+ return record.get("Arn", "")
+ return f"arn:{partition}:iam::{account}:role/{role}"
+
+
+def _access_entry_for_principal(cluster_name, account_id, region, principal_arn):
+ candidates = [principal_arn]
+ role_arn = _role_arn_from_assumed_role(principal_arn)
+ if role_arn:
+ candidates.append(role_arn)
+ for candidate in candidates:
+ entry = _access_entries.get_scoped(
+ account_id, region, _ae_key(cluster_name, candidate)
+ )
+ if entry:
+ return entry
+ return None
+
+
+def _authenticate_token_review(cluster_name, account_id, region, review):
+ from ministack.app import AUTH
+
+ spec = review.get("spec")
+ if not isinstance(spec, dict) or not isinstance(spec.get("token"), str) or not spec["token"]:
+ return _token_review_response(review)
+
+ cluster_info = _cluster_for_auth(cluster_name, account_id, region)
+ if not cluster_info:
+ return _token_review_response(review)
+ account_id, region, cluster = cluster_info
+
+ # Permissive mode accepts local bearer credentials without requiring an
+ # IAM identity, a matching secret, or an access entry, just like the AWS
+ # control plane skips IAM enforcement when AUTH=false.
+ if not AUTH:
+ return _token_review_response(
+ review, authenticated=True, username="ministack-local", uid="ministack-local",
+ groups=["system:authenticated", "system:masters"],
+ )
+
+ credential = _verify_eks_token(spec["token"], cluster_name)
+ if not credential:
+ return _token_review_response(review)
+
+ # IAM authorizes AWS EKS API operations at the gateway. Kubernetes access
+ # additionally requires the bootstrap creator grant or an access entry;
+ # an IAM Allow for eks:* alone does not confer Kubernetes permissions.
+ entry = _access_entry_for_principal(
+ cluster_name, account_id, region, credential.principal_arn
+ )
+ username = credential.principal_arn
+ groups = ["system:authenticated"]
+ if entry:
+ username = entry.get("username") or username
+ groups.extend(entry.get("kubernetesGroups") or [])
+ # EKS evaluates managed access policies in a separate authorizer. k3s
+ # has only RBAC, so each policy becomes an internal RBAC group bound
+ # to a MiniStack-managed ClusterRole or RoleBinding.
+ groups.extend(
+ _access_policy_group(cluster_name, entry["principalArn"], policy_arn)
+ for principal_arn, policy_arn, _association in _access_policy_associations(
+ cluster_name, account_id, region
+ )
+ if principal_arn == entry["principalArn"]
+ )
+ elif (
+ credential.account_id == account_id
+ and cluster.get("accessConfig", {}).get("bootstrapClusterCreatorAdminPermissions", True)
+ and cluster.get("_creator_arn") == (
+ _role_arn_from_assumed_role(credential.principal_arn) or credential.principal_arn
+ )
+ ):
+ groups.append("system:masters")
+ else:
+ return _token_review_response(review)
+
+ # Deduplicate while preserving the caller's configured order.
+ groups = list(dict.fromkeys(groups))
+ return _token_review_response(
+ review,
+ authenticated=True,
+ username=username,
+ uid=credential.principal_id or credential.principal_arn,
+ groups=groups,
+ )
+
+
+def _handle_auth_webhook(cluster_name, account_id, region, body_bytes):
+ try:
+ review = json.loads(body_bytes) if body_bytes else {}
+ except (TypeError, json.JSONDecodeError):
+ review = {}
+ if not isinstance(review, dict):
+ review = {}
+ return _authenticate_token_review(cluster_name, account_id, region, review)
+
+
# ---------------------------------------------------------------------------
# Sanitize (remove internal fields)
# ---------------------------------------------------------------------------
@@ -1470,6 +2217,20 @@ def _sanitize(cluster):
# ---------------------------------------------------------------------------
def _handle_request_sync(method, path, headers, body_bytes, query_params):
+ # This private route is called by the k3s apiserver's authentication
+ # webhook, not by an AWS EKS client. Keep it outside the EKS JSON API
+ # namespace so the normal AWS action router never sees TokenReview data.
+ auth_match = re.fullmatch(
+ r"/eks-auth/(\d{12})/([A-Za-z0-9-]+)/([A-Za-z0-9_.-]+)", path
+ )
+ if auth_match and method == "POST":
+ return _handle_auth_webhook(
+ urllib.parse.unquote(auth_match.group(3)),
+ auth_match.group(1),
+ auth_match.group(2),
+ body_bytes,
+ )
+
try:
body = json.loads(body_bytes) if body_bytes else {}
except json.JSONDecodeError:
@@ -1479,7 +2240,9 @@ def _handle_request_sync(method, path, headers, body_bytes, query_params):
# POST /clusters
if path == "/clusters" and method == "POST":
- return _create_cluster(body)
+ identity = resolve_caller_identity(extract_access_key_id(headers, query_params))
+ creator_arn = identity["userArn"] if identity else None
+ return _create_cluster(body, _role_arn_from_assumed_role(creator_arn) or creator_arn)
# GET /clusters
if path == "/clusters" and method == "GET":
diff --git a/tests/test_eks.py b/tests/test_eks.py
index 812a656a..c32dafa4 100644
--- a/tests/test_eks.py
+++ b/tests/test_eks.py
@@ -4,16 +4,29 @@ Tests cluster CRUD, nodegroup CRUD, tags, and CloudFormation provisioning.
k3s Docker container tests require Docker socket access.
"""
import asyncio
+import base64
+import datetime as dt
import json
import os
+import threading
import time
import uuid
+from types import SimpleNamespace
+from unittest.mock import patch
from urllib.parse import quote
import boto3
import pytest
+from botocore.auth import SigV4Auth, SigV4QueryAuth
+from botocore.awsrequest import AWSRequest
+from botocore.credentials import Credentials
from botocore.exceptions import ClientError
+import ministack.app as app
+from ministack.core.responses import AccountRegionScopedDict, AccountScopedDict, request_scope
+from ministack.services import eks as eks_service
+from ministack.services import iam, sts
+
ENDPOINT = os.environ.get("MINISTACK_ENDPOINT", "http://localhost:4566").rstrip("/")
REGION = "us-east-1"
@@ -76,6 +89,19 @@ def _eks_direct_create_cluster(eks_service, name):
return body["cluster"]["arn"]
+def _eks_exec_token(cluster_name, access_key="test", secret_key="test"):
+ request = AWSRequest(
+ method="GET",
+ url="https://sts.us-east-1.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15",
+ headers={"x-k8s-aws-id": cluster_name},
+ )
+ SigV4QueryAuth(
+ Credentials(access_key, secret_key), "sts", "us-east-1", expires=60
+ ).add_auth(request)
+ encoded = base64.urlsafe_b64encode(request.url.encode()).decode().rstrip("=")
+ return "k8s-aws-v1." + encoded
+
+
# ---------------------------------------------------------------------------
# Cluster CRUD
# ---------------------------------------------------------------------------
@@ -1359,3 +1385,390 @@ def test_eks_restore_state_normalizes_endpoint_to_localhost():
assert restored["status"] == "ACTIVE" # endpoint stays non-empty for ACTIVE
finally:
eks_mod.reset()
+
+
+# ---------------------------------------------------------------------------
+# IAM exec-token authentication and AUTH mode enforcement (no Docker required)
+# ---------------------------------------------------------------------------
+
+_AUTH_ACCOUNT = "000000000000"
+_AUTH_OTHER_ACCOUNT = "111111111111"
+_AUTH_REGION = "eu-central-1"
+_AUTH_CLUSTER = "auth-cluster"
+_AUTH_USER_KEY = "AKIAEKSREVIEW"
+_AUTH_USER_ARN = f"arn:aws:iam::{_AUTH_ACCOUNT}:user/developer"
+_AUTH_ROLE_ARN = f"arn:aws:iam::{_AUTH_ACCOUNT}:role/control-plane"
+_AUTH_POLICY_PREFIX = "arn:aws:eks::aws:cluster-access-policy/"
+_AUTH_ADMIN_POLICY = _AUTH_POLICY_PREFIX + "AmazonEKSClusterAdminPolicy"
+
+
+def eks_auth_env(monkeypatch):
+ monkeypatch.setattr(app, "AUTH", True)
+ monkeypatch.setenv("AWS_ACCESS_KEY_ID", "test")
+ monkeypatch.setenv("AWS_SECRET_ACCESS_KEY", "test")
+ monkeypatch.setenv("MINISTACK_ACCOUNT_ID", _AUTH_ACCOUNT)
+ monkeypatch.delenv("AWS_SESSION_TOKEN", raising=False)
+ for name in ("_clusters", "_access_entries", "_access_policies", "_idp_configs", "_tags"):
+ monkeypatch.setattr(eks_service, name, AccountRegionScopedDict())
+ for name in ("_users", "_access_keys", "_roles", "_user_inline_policies"):
+ monkeypatch.setattr(iam, name, AccountScopedDict())
+ monkeypatch.setattr(sts, "_sessions", {})
+ monkeypatch.setattr(eks_service, "_get_docker", lambda: None)
+ with request_scope(_AUTH_ACCOUNT, _AUTH_REGION):
+ iam._users["developer"] = {"Arn": _AUTH_USER_ARN, "UserId": "developer-id"}
+ iam._access_keys[_AUTH_USER_KEY] = {
+ "UserName": "developer", "SecretAccessKey": "test", "Status": "Active",
+ }
+ iam._roles["control-plane"] = {"Arn": _AUTH_ROLE_ARN}
+ yield
+
+
+def _auth_token(key="test", secret="test", session=None, name=_AUTH_CLUSTER, signed_at=None):
+ request = AWSRequest(
+ method="GET",
+ url=f"https://sts.{_AUTH_REGION}.amazonaws.com/?Action=GetCallerIdentity&Version=2011-06-15",
+ headers={"x-k8s-aws-id": name},
+ )
+ signer = SigV4QueryAuth(Credentials(key, secret, session), "sts", _AUTH_REGION, expires=60)
+ if signed_at is None:
+ signer.add_auth(request)
+ else:
+ with patch("botocore.auth.get_current_datetime", return_value=signed_at):
+ signer.add_auth(request)
+ return "k8s-aws-v1." + base64.urlsafe_b64encode(request.url.encode()).decode().rstrip("=")
+
+
+def _auth_cluster(account=_AUTH_ACCOUNT, creator=None, bootstrap=True):
+ eks_service._clusters.set_scoped(account, _AUTH_REGION, _AUTH_CLUSTER, {
+ "arn": f"arn:aws:eks:{_AUTH_REGION}:{account}:cluster/{_AUTH_CLUSTER}",
+ "roleArn": f"arn:aws:iam::{account}:role/control-plane",
+ "_creator_arn": creator,
+ "accessConfig": {"bootstrapClusterCreatorAdminPermissions": bootstrap},
+ })
+
+
+def _auth_review(bearer, account=_AUTH_ACCOUNT, name=_AUTH_CLUSTER):
+ # Real k3s requests carry no AWS credentials; deliberately use the default
+ # request account even when authenticating to a different tenant's cluster.
+ with request_scope(_AUTH_ACCOUNT, "us-east-1"):
+ status, _, body = asyncio.run(app._dispatch_service_request(
+ "POST", f"/eks-auth/{account}/{_AUTH_REGION}/{name}", {},
+ json.dumps({"apiVersion": "authentication.k8s.io/v1", "spec": {"token": bearer}}).encode(),
+ {}, "review-test",
+ ))
+ assert status == 200
+ return json.loads(body)["status"]
+
+
+def _auth_api(method, path, body=None, key="test"):
+ payload = json.dumps(body or {}).encode()
+ request = AWSRequest(method=method, url=f"http://localhost:4566{path}", data=payload)
+ SigV4Auth(Credentials(key, "test"), "eks", _AUTH_REGION).add_auth(request)
+ status, _, result = asyncio.run(app._dispatch_service_request(
+ method, path, {k.lower(): v for k, v in request.headers.items()}, payload, {}, "api-test",
+ ))
+ return status, json.loads(result) if result else {}
+
+
+def _auth_session(account=_AUTH_ACCOUNT, role="developer", key="ASIAEKSREVIEW"):
+ sts._sessions[key] = {
+ "Arn": f"arn:aws:sts::{account}:assumed-role/{role}/session",
+ "AccountId": account, "UserId": "role-id:session", "SecretAccessKey": "test",
+ "SessionToken": "local-session", "Expiration": time.time() + 3600,
+ }
+ return key
+
+
+def test_eks_token_review_authenticates_root_exec_token(eks_mod, monkeypatch):
+ import ministack.app as app
+ monkeypatch.setattr(app, "AUTH", True)
+ name = f"auth-{_uid()}"
+ eks_mod._clusters[name] = {
+ "arn": f"arn:aws:eks:{REGION}:000000000000:cluster/{name}",
+ "roleArn": "arn:aws:iam::000000000000:role/eks-role",
+ "_creator_arn": "arn:aws:iam::000000000000:root",
+ }
+ token = _eks_exec_token(name)
+ status, _headers, response = _eks_direct(
+ eks_mod,
+ "POST",
+ f"/eks-auth/000000000000/{REGION}/{name}",
+ {
+ "apiVersion": "authentication.k8s.io/v1",
+ "kind": "TokenReview",
+ "spec": {"token": token},
+ },
+ )
+ assert status == 200
+ assert response["status"]["authenticated"] is True
+ assert response["status"]["user"]["username"] == "arn:aws:iam::000000000000:root"
+ assert "system:masters" in response["status"]["user"]["groups"]
+
+
+def test_eks_token_review_rejects_wrong_cluster_and_tampering(eks_mod, monkeypatch):
+ import ministack.app as app
+ monkeypatch.setattr(app, "AUTH", True)
+ name = f"auth-{_uid()}"
+ eks_mod._clusters[name] = {
+ "arn": f"arn:aws:eks:{REGION}:000000000000:cluster/{name}",
+ "roleArn": "arn:aws:iam::000000000000:role/eks-role",
+ }
+ token = _eks_exec_token(name)
+ wrong_cluster = _eks_exec_token("another-cluster")
+ for candidate in (wrong_cluster, token[:-1] + ("A" if token[-1] != "A" else "B")):
+ status, _headers, response = _eks_direct(
+ eks_mod,
+ "POST",
+ f"/eks-auth/000000000000/{REGION}/{name}",
+ {"spec": {"token": candidate}},
+ )
+ assert status == 200
+ assert response["status"]["authenticated"] is False
+
+
[email protected]("auth", [False, True])
[email protected]("credential", ["unknown-key", "wrong-secret", "unmapped-user", "opaque-token"])
+def test_eks_auth_modes_enforce_credentials_and_access_grants(eks_auth_env, monkeypatch, auth, credential):
+ monkeypatch.setattr(app, "AUTH", auth)
+ _auth_cluster()
+ bearer = {
+ "unknown-key": lambda: _auth_token("unknown"),
+ "wrong-secret": lambda: _auth_token(secret="wrong"),
+ "unmapped-user": lambda: _auth_token(_AUTH_USER_KEY),
+ "opaque-token": lambda: "local-bearer-token",
+ }[credential]()
+ result = _auth_review(bearer)
+ assert result["authenticated"] is (not auth)
+ if not auth:
+ assert "system:masters" in result["user"]["groups"]
+
+
[email protected]("auth", [False, True])
+def test_eks_auth_empty_tokens_and_missing_clusters_are_rejected(eks_auth_env, monkeypatch, auth):
+ monkeypatch.setattr(app, "AUTH", auth)
+ _auth_cluster()
+ assert _auth_review("")["authenticated"] is False
+ assert _auth_review(_auth_token(), name="missing")["authenticated"] is False
+
+
[email protected]("body", [b"null", b"[]", b"{", b'{"spec": []}'])
+def test_eks_auth_malformed_reviews_fail_closed(eks_auth_env, body):
+ assert json.loads(eks_service._handle_auth_webhook(_AUTH_CLUSTER, _AUTH_ACCOUNT, _AUTH_REGION, body)[2])["status"] == {
+ "authenticated": False,
+ }
+
+
[email protected]("auth", [False, True])
+def test_eks_auth_iam_enforcement_is_separate_from_kubernetes_access(eks_auth_env, monkeypatch, auth):
+ monkeypatch.setattr(app, "AUTH", auth)
+ _auth_cluster()
+ assert _auth_api("GET", f"/clusters/{_AUTH_CLUSTER}", key=_AUTH_USER_KEY)[0] == (403 if auth else 200)
+ iam._user_inline_policies["developer"] = {
+ "eks": {"Statement": [{"Effect": "Allow", "Action": "eks:*", "Resource": "*"}]},
+ }
+ assert _auth_api("GET", f"/clusters/{_AUTH_CLUSTER}", key=_AUTH_USER_KEY)[0] == 200
+ assert _auth_review(_auth_token(_AUTH_USER_KEY))["authenticated"] is (not auth)
+ eks_service._create_access_entry(_AUTH_CLUSTER, {"principalArn": _AUTH_USER_ARN, "kubernetesGroups": ["developers"]})
+ assert _auth_review(_auth_token(_AUTH_USER_KEY))["authenticated"] is True
+ if auth:
+ assert _auth_review(_auth_token(_AUTH_USER_KEY))["user"]["groups"] == ["system:authenticated", "developers"]
+
+
[email protected]("key,creator", [("test", f"arn:aws:iam::{_AUTH_ACCOUNT}:root"), (_AUTH_USER_KEY, _AUTH_USER_ARN)])
[email protected]("bootstrap", [False, True])
+def test_eks_auth_create_records_actual_caller_and_honors_bootstrap(eks_auth_env, key, creator, bootstrap):
+ iam._user_inline_policies["developer"] = {
+ "eks": {"Statement": [{"Effect": "Allow", "Action": "eks:*", "Resource": "*"}]},
+ }
+ status, body = _auth_api("POST", "/clusters", {
+ "name": _AUTH_CLUSTER, "roleArn": _AUTH_ROLE_ARN,
+ "accessConfig": {"bootstrapClusterCreatorAdminPermissions": bootstrap},
+ }, key=key)
+ assert status == 200
+ assert "_creator_arn" not in body["cluster"]
+ assert eks_service._clusters[_AUTH_CLUSTER]["_creator_arn"] == creator
+ assert _auth_review(_auth_token(key))["authenticated"] is bootstrap
+ # The service role never inherits the caller's bootstrap privilege.
+ assert _auth_review(_auth_token(_auth_session(role="control-plane"), session="local-session"))["authenticated"] is False
+
+
+def test_eks_auth_root_does_not_get_other_accounts_bootstrap_access(eks_auth_env):
+ _auth_cluster(_AUTH_OTHER_ACCOUNT, creator=f"arn:aws:iam::{_AUTH_OTHER_ACCOUNT}:root")
+ assert _auth_review(_auth_token(), _AUTH_OTHER_ACCOUNT)["authenticated"] is False
+ assert _auth_review(_auth_token(_AUTH_OTHER_ACCOUNT), _AUTH_OTHER_ACCOUNT)["user"]["groups"] == [
+ "system:authenticated", "system:masters",
+ ]
+
+
+def test_eks_auth_nondefault_sts_session_and_role_path_access_entry(eks_auth_env):
+ _auth_cluster(_AUTH_OTHER_ACCOUNT)
+ key = _auth_session(_AUTH_OTHER_ACCOUNT)
+ role_arn = f"arn:aws:iam::{_AUTH_OTHER_ACCOUNT}:role/team/developer"
+ iam._roles.set_scoped(_AUTH_OTHER_ACCOUNT, None, "developer", {"Arn": role_arn})
+ with request_scope(_AUTH_OTHER_ACCOUNT, _AUTH_REGION):
+ eks_service._create_access_entry(_AUTH_CLUSTER, {"principalArn": role_arn, "kubernetesGroups": ["developers"]})
+ assert _auth_review(_auth_token(key, session="local-session"), _AUTH_OTHER_ACCOUNT)["user"]["groups"] == [
+ "system:authenticated", "developers",
+ ]
+ assert _auth_review(_auth_token(key, session="wrong"), _AUTH_OTHER_ACCOUNT)["authenticated"] is False
+ sts._sessions[key]["Expiration"] = time.time() - 1
+ assert _auth_review(_auth_token(key, session="local-session"), _AUTH_OTHER_ACCOUNT)["authenticated"] is False
+
+
[email protected]("scope,policy,supported", [
+ ({"type": "cluster"}, _AUTH_ADMIN_POLICY, True),
+ ({"type": "namespace", "namespaces": ["dev"]}, _AUTH_ADMIN_POLICY, True),
+ ({"type": "cluster"}, _AUTH_POLICY_PREFIX + "AmazonEKSViewPolicy", True),
+ ({"type": "cluster"}, "fake-AmazonEKSClusterAdminPolicy", False),
+])
+def test_eks_auth_access_policies_map_to_internal_rbac_groups(eks_auth_env, monkeypatch, scope, policy, supported):
+ monkeypatch.setattr(eks_service, "_schedule_access_policy_reconcile", lambda *args: None)
+ _auth_cluster()
+ eks_service._create_access_entry(_AUTH_CLUSTER, {"principalArn": _AUTH_USER_ARN})
+ eks_service._associate_access_policy(_AUTH_CLUSTER, _AUTH_USER_ARN, {"policyArn": policy, "accessScope": scope})
+ result = _auth_review(_auth_token(_AUTH_USER_KEY))
+ assert result["authenticated"] is True
+ group = eks_service._access_policy_group(_AUTH_CLUSTER, _AUTH_USER_ARN, policy)
+ assert (group in result["user"]["groups"]) is supported
+ assert "system:masters" not in result["user"]["groups"]
+
+
+def _policy_allows(policy_name, verb, api_group, resource):
+ for rule in eks_service._ACCESS_POLICY_RULES[policy_name]:
+ groups = rule.get("apiGroups", [])
+ resources = rule.get("resources", [])
+ verbs = rule["verbs"]
+ if (
+ (api_group in groups or "*" in groups)
+ and (resource in resources or "*" in resources)
+ and (verb in verbs or "*" in verbs)
+ ):
+ return True
+ return False
+
+
[email protected]("policy,allowed,denied", [
+ ("AmazonEKSClusterAdminPolicy", ("delete", "rbac.authorization.k8s.io", "clusterroles"), ()),
+ ("AmazonEKSAdminPolicy", ("create", "rbac.authorization.k8s.io", "rolebindings"), ("delete", "", "nodes")),
+ ("AmazonEKSEditPolicy", ("create", "", "secrets"), ("create", "rbac.authorization.k8s.io", "roles")),
+ ("AmazonEKSViewPolicy", ("get", "", "pods"), ("get", "", "secrets")),
+ ("AmazonEKSAdminViewPolicy", ("get", "", "secrets"), ("create", "", "secrets")),
+])
+def test_eks_access_policy_rules_allow_and_deny_documented_operations(policy, allowed, denied):
+ assert _policy_allows(policy, *allowed)
+ if denied:
+ assert not _policy_allows(policy, *denied)
+
+
+def test_eks_access_policy_rbac_reconciles_scopes_namespaces_and_revocation(eks_auth_env, monkeypatch):
+ """The materialized k3s RBAC follows association updates and new matches."""
+ monkeypatch.setattr(eks_service, "_schedule_access_policy_reconcile", lambda *args: None)
+ _auth_cluster()
+ eks_service._clusters.get_scoped(_AUTH_ACCOUNT, _AUTH_REGION, _AUTH_CLUSTER)["_docker_id"] = "k3s"
+ policy = _AUTH_POLICY_PREFIX + "AmazonEKSViewPolicy"
+ with request_scope(_AUTH_ACCOUNT, _AUTH_REGION):
+ eks_service._create_access_entry(_AUTH_CLUSTER, {"principalArn": _AUTH_USER_ARN})
+ eks_service._associate_access_policy(_AUTH_CLUSTER, _AUTH_USER_ARN, {
+ "policyArn": policy,
+ "accessScope": {"type": "namespace", "namespaces": ["dev-*"]},
+ })
+
+ namespaces = ["default", "dev-api", "prod"]
+ captured = []
+ managed = {"ClusterRoleBinding": [], "RoleBinding": []}
+ deletes = []
+ container = SimpleNamespace()
+ client = SimpleNamespace(containers=SimpleNamespace(get=lambda _id: container))
+
+ def kubectl(_container, command):
+ if command[1:3] == ["get", "namespaces"]:
+ return 0, json.dumps({"items": [{"metadata": {"name": name}} for name in namespaces]})
+ if "delete" in command:
+ deletes.append(command)
+ return 0, ""
+
+ monkeypatch.setattr(eks_service, "_get_docker", lambda: client)
+ monkeypatch.setattr(eks_service, "_k3s_exec", kubectl)
+ monkeypatch.setattr(
+ eks_service, "_apply_access_policy_rbac",
+ lambda _container, objects: captured.append(objects),
+ )
+ monkeypatch.setattr(
+ eks_service, "_managed_access_policy_bindings",
+ lambda _container, kind: managed[kind],
+ )
+
+ assert eks_service._reconcile_access_policy_rbac(_AUTH_CLUSTER, _AUTH_ACCOUNT, _AUTH_REGION)
+ bindings = [obj for obj in captured[-1] if obj["kind"] == "RoleBinding"]
+ assert [binding["metadata"]["namespace"] for binding in bindings] == ["dev-api"]
+ assert bindings[0]["roleRef"]["name"] == "ministack-eks-amazoneksviewpolicy"
+ managed["RoleBinding"] = bindings
+
+ namespaces.append("dev-worker")
+ assert eks_service._reconcile_access_policy_rbac(_AUTH_CLUSTER, _AUTH_ACCOUNT, _AUTH_REGION)
+ bindings = [obj for obj in captured[-1] if obj["kind"] == "RoleBinding"]
+ assert {binding["metadata"]["namespace"] for binding in bindings} == {"dev-api", "dev-worker"}
+
+ with request_scope(_AUTH_ACCOUNT, _AUTH_REGION):
+ eks_service._associate_access_policy(_AUTH_CLUSTER, _AUTH_USER_ARN, {
+ "policyArn": policy, "accessScope": {"type": "cluster"},
+ })
+ managed["RoleBinding"] = bindings
+ assert eks_service._reconcile_access_policy_rbac(_AUTH_CLUSTER, _AUTH_ACCOUNT, _AUTH_REGION)
+ cluster_bindings = [obj for obj in captured[-1] if obj["kind"] == "ClusterRoleBinding"]
+ assert len(cluster_bindings) == 1
+ assert {command[2] for command in deletes} == {"rolebinding"}
+
+ managed["ClusterRoleBinding"] = cluster_bindings
+ with request_scope(_AUTH_ACCOUNT, _AUTH_REGION):
+ eks_service._disassociate_access_policy(_AUTH_CLUSTER, _AUTH_USER_ARN, policy)
+ assert eks_service._reconcile_access_policy_rbac(_AUTH_CLUSTER, _AUTH_ACCOUNT, _AUTH_REGION)
+ assert not [obj for obj in captured[-1] if obj["kind"].endswith("Binding")]
+ assert {command[2] for command in deletes} == {"rolebinding", "clusterrolebinding"}
+
+
+def test_eks_access_policy_changes_schedule_rbac_reconciliation(eks_auth_env, monkeypatch):
+ scheduled = []
+ policy = _AUTH_POLICY_PREFIX + "AmazonEKSEditPolicy"
+ monkeypatch.setattr(
+ eks_service, "_schedule_access_policy_reconcile",
+ lambda cluster_name, *args: scheduled.append(cluster_name),
+ )
+ _auth_cluster()
+ eks_service._create_access_entry(_AUTH_CLUSTER, {"principalArn": _AUTH_USER_ARN})
+ eks_service._associate_access_policy(_AUTH_CLUSTER, _AUTH_USER_ARN, {
+ "policyArn": policy, "accessScope": {"type": "cluster"},
+ })
+ eks_service._disassociate_access_policy(_AUTH_CLUSTER, _AUTH_USER_ARN, policy)
+ eks_service._delete_access_entry(_AUTH_CLUSTER, _AUTH_USER_ARN)
+ assert scheduled == [_AUTH_CLUSTER, _AUTH_CLUSTER, _AUTH_CLUSTER]
+
+
[email protected]("minutes,accepted", [(2, True), (14, True), (16, False), (-6, False)])
+def test_eks_auth_exec_token_lifetime_matches_kubectl_cache(eks_auth_env, minutes, accepted):
+ _auth_cluster(creator=f"arn:aws:iam::{_AUTH_ACCOUNT}:root")
+ signed_at = dt.datetime.now(dt.timezone.utc) - dt.timedelta(minutes=minutes)
+ assert _auth_review(_auth_token(signed_at=signed_at))["authenticated"] is accepted
+
+
+def test_eks_auth_background_webhook_preserves_account_and_region(eks_auth_env, monkeypatch):
+ finished = threading.Event()
+ captured = {}
+
+ def start(client, kwargs, registries, webhook):
+ captured["webhook"] = webhook.decode()
+ finished.set()
+ return SimpleNamespace(id="fake-container")
+
+ monkeypatch.setattr(eks_service, "_get_docker", lambda: object())
+ monkeypatch.setattr(eks_service, "_get_ministack_network", lambda client: None)
+ monkeypatch.setattr(eks_service, "_k3s_registries_yaml", lambda *args: None)
+ monkeypatch.setattr(eks_service, "_start_k3s_container", start)
+ monkeypatch.setattr(eks_service, "_extract_ca_cert", lambda *args: "fake-ca")
+ with request_scope(_AUTH_OTHER_ACCOUNT, _AUTH_REGION):
+ iam._roles["control-plane"] = {"Arn": f"arn:aws:iam::{_AUTH_OTHER_ACCOUNT}:role/control-plane"}
+ status, _, _ = eks_service._create_cluster({"name": _AUTH_CLUSTER, "roleArn": iam._roles["control-plane"]["Arn"]})
+ assert status == 200
+ assert finished.wait(3)
+ assert f"/eks-auth/{_AUTH_OTHER_ACCOUNT}/{_AUTH_REGION}/{_AUTH_CLUSTER}" in captured["webhook"]