Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
add KMS (#357)
Nahuel Nucerab6a2cd5251df
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex aa930fd0..e6300f43 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -7,6 +7,54 @@ Versioning follows [Semantic Versioning](https://semver.org/).---+## [1.2.20] — 2026-04-17++### Added+- **EKS service with k3s backend** — CreateCluster, DescribeCluster, ListClusters, DeleteCluster, CreateNodegroup, DescribeNodegroup, ListNodegroups, DeleteNodegroup, TagResource, UntagResource, ListTagsForResource. `CreateCluster` spawns a real k3s Docker container (75 MB) providing a full Kubernetes API server. `kubectl`, Helm, and any K8s tooling work out of the box. Cascading delete removes nodegroups and k3s container. CloudFormation `AWS::EKS::Cluster` and `AWS::EKS::Nodegroup` provisioners included.+- **Lambda layer S3 support** — `PublishLayerVersion` now accepts `S3Bucket`/`S3Key` in Content, matching real AWS behavior. Contributed by @Baptiste-Garcin (#356)+- **Lambda Docker executor rewritten with AWS RIE** — `LAMBDA_EXECUTOR=docker` now uses official AWS Lambda Runtime Interface Emulator images (`public.ecr.aws/lambda/*`) for all runtimes (Python, Node.js, provided). Events are POSTed to the RIE HTTP endpoint on port 8080, matching exact AWS Lambda execution semantics. Containers are kept warm between invocations and reused when the same function+code is invoked again. Cleaned up on `reset()` and shutdown. Added `nodejs22.x`, `nodejs24.x`, `python3.14` runtimes. Contributed by @fzonneveld (#302)+- **Lambda Windows compatibility** — replaced `select.select()` stderr polling with cross-platform background thread + queue. Fixes Lambda warm worker execution on Windows. Contributed by @davidtme (#350)+- **Lambda ESM poller on CFN create and state restore** — event source mappings created via CloudFormation or restored from persisted state now correctly start the background poller. Contributed by @davidtme (#350)++### Fixed++#### AWS Compliance (21 fixes from full-codebase audit)+- **KMS `Verify` error handling** — invalid signatures now raise `KMSInvalidSignatureException` (HTTP 400) instead of returning `SignatureValid: false` with HTTP 200, matching real AWS behavior.+- **KMS `Decrypt`/`GenerateDataKey`/`Sign`/`Verify`/`Encrypt` response `KeyId`** — all KMS crypto operations now return the full key ARN in the `KeyId` field instead of the bare UUID, matching real AWS.+- **KMS `PendingDeletion` state check** — `Encrypt`, `Decrypt`, `Sign`, `Verify`, and `GenerateDataKey` now return `KMSInvalidStateException` when called on a key scheduled for deletion or disabled. Previously these operations silently succeeded.+- **EC2 `TerminateInstances`/`StopInstances`/`StartInstances` unknown instance IDs** — now return `InvalidInstanceID.NotFound` error instead of silently succeeding with an empty response.+- **EC2 VPC `cidrBlockAssociationSet` missing** — `CreateVpc` and `DescribeVpcs` responses now include `<cidrBlockAssociationSet>` with the primary CIDR association. Fixes Terraform AWS provider v6 crash (`index out of range [0]`). Reported by @mspiller (#331)+- **SQS FIFO `DeduplicationScope: messageGroup`** — content-based deduplication now correctly scopes per message group when `DeduplicationScope` is `messageGroup`. Previously, two messages with the same body but different `MessageGroupId` values were incorrectly deduplicated. Contributed by @CSandyHub (#359)+- **SNS `ListSubscriptions` XML escaping** — endpoint URLs containing `&` or other XML special characters are now properly escaped, preventing malformed XML responses.+- **DynamoDB `DescribeTable` `LatestStreamArn` stability** — stream ARN and label are now set once when `StreamSpecification` is enabled instead of regenerated on every `DescribeTable` call. Fixes CDK drift detection and ESM setup failures.+- **SSM `GetParametersByPath` root path** — `GetParametersByPath` with `Path=/` and `Recursive=false` now correctly returns only top-level parameters instead of all parameters in the store.+- **ElastiCache `AutomaticFailover`/`MultiAZ` values** — `CreateReplicationGroup` and `ModifyReplicationGroup` now return `enabled`/`disabled` enum values instead of raw `true`/`false` strings, matching the AWS API contract.+- **Transfer Family pagination off-by-one** — `ListServers` and `ListUsers` no longer re-serve the token item when paginating, fixing duplicate entries across pages.+- **ECS `PutAccountSettingDefault` inconsistency** — now stores a plain string value like `PutAccountSetting`, fixing `ListAccountSettings` response shape when both endpoints were used.+- **IAM user inline policy persistence** — restructured `_user_inline_policies` from tuple keys `(user, policy)` to nested dict `{user: {policy: doc}}`. Tuple keys silently broke JSON serialization, causing all user inline policies to be lost on restart with `PERSIST_STATE=1`.+- **Route53 `reset()` multi-tenancy** — `reset()` now calls `.clear()` on existing `AccountScopedDict` instances instead of replacing them with plain `dict` objects, preserving multi-tenant isolation after reset.+- **STS `AssumeRoleWithWebIdentity` provider** — `Provider` field now uses the caller-supplied `ProviderId` instead of hardcoded `accounts.google.com`.+- **EKS state persistence** — `get_state()` now saves `port_counter` and strips Docker container IDs. `restore_state()` restores port counter and marks clusters as `FAILED` (k3s containers don't survive restart).++#### Architecture & Safety+- **Persistence `eval()` replaced with `ast.literal_eval`** — deserialization of `AccountScopedDict` keys no longer uses `eval()`, closing a code injection vector via crafted state files.+- **RDS `_wait_for_port` no longer blocks event loop** — container port wait now runs in a background thread. Previously a `CreateDBInstance` with Docker could block the entire ASGI server for up to 60 seconds.+- **RDS `get_state()` multi-account persistence** — `get_state()` now serializes instances as a full `AccountScopedDict`, capturing all accounts instead of only the default account at shutdown time.+- **RDS `_port_counter` thread safety** — port allocation now uses a `threading.Lock`, preventing potential duplicate ports under concurrent requests.+- **Lambda ESM poller account context** — background SQS/Kinesis/DynamoDB Streams pollers now iterate `_esms._data` directly and set the correct account context per ESM. Previously, event source mappings created under non-default accounts were silently never polled.++### Also Fixed+- **EC2 SecurityGroup duplicate detection ignoring Description** — `AuthorizeSecurityGroupIngress` duplicate check and `RevokeSecurityGroupIngress` now compare rules without the `Description` field, matching AWS behavior.+- **CloudWatch DeleteDashboards error** — deleting a nonexistent dashboard returned 500 InternalError instead of 404 DashboardNotFoundError.+- **Athena ListNamedQueries empty** — `ListNamedQueries` without a `WorkGroup` filter now returns all queries instead of only "primary" workgroup.+- **ElastiCache CreateCacheSubnetGroup missing Subnets** — response XML now includes `<Subnets>` element.+- **Cognito OAuth2 lazy loading** — OAuth2 endpoints now use lazy module loading, fixing crash when Cognito module wasn't pre-imported.+- **Cognito OAuth2 persistence** — `_authorization_codes` and `_refresh_tokens` now included in state persistence.+- **Lambda warm worker stuck after init failure** — broken workers are now invalidated so the next invocation gets a fresh process. Reported by @Baptiste-Garcin+- **Docker image missing `boto3`** — Lambda functions importing `boto3` now work out of the box. Real AWS Lambda runtimes pre-install `boto3`; the Docker image only had `botocore` (via `awscli`). Reported by @xPTM1219 (#362)++---+## [1.2.19] — 2026-04-16### Addeddiff --git a/Dockerfile b/Dockerfileindex 50b5c6a0..ce40e58d 100644--- a/Dockerfile+++ b/Dockerfile@@ -8,6 +8,7 @@ RUN pip install --no-cache-dir --no-compile \"pyyaml>=6.0" \"cryptography>=41.0" \"pymysql>=1.1" \+ "boto3>=1.34" \"awscli"# Strip awscli help examples (~25 MB) and Python cache files (~15 MB).diff --git a/README.md b/README.mdindex a15869f1..d763d1e7 100644--- a/README.md+++ b/README.md@@ -483,6 +483,7 @@ Unsupported resource types fail with `CREATE_FAILED` (or `ROLLBACK_COMPLETE` if| **AppConfig** | CreateApplication, GetApplication, ListApplications, UpdateApplication, DeleteApplication, CreateEnvironment, GetEnvironment, ListEnvironments, UpdateEnvironment, DeleteEnvironment, CreateConfigurationProfile, GetConfigurationProfile, ListConfigurationProfiles, UpdateConfigurationProfile, DeleteConfigurationProfile, CreateHostedConfigurationVersion, GetHostedConfigurationVersion, ListHostedConfigurationVersions, DeleteHostedConfigurationVersion, CreateDeploymentStrategy, GetDeploymentStrategy, ListDeploymentStrategies, UpdateDeploymentStrategy, DeleteDeploymentStrategy, StartDeployment, GetDeployment, ListDeployments, StopDeployment, TagResource, UntagResource, ListTagsForResource, StartConfigurationSession, GetLatestConfiguration | 33 operations; control plane + data plane; hosted configuration versions; deployments complete immediately; session-based configuration retrieval with token rotation || **Transfer Family** | CreateServer, DescribeServer, DeleteServer, ListServers, CreateUser, DescribeUser, DeleteUser, ListUsers, ImportSshPublicKey, DeleteSshPublicKey | 10 operations; SFTP server and user management; SSH key rotation; LOGICAL home directory mappings to S3; in-memory state || **EventBridge Scheduler** | CreateSchedule, GetSchedule, UpdateSchedule, DeleteSchedule, ListSchedules, CreateScheduleGroup, GetScheduleGroup, DeleteScheduleGroup, ListScheduleGroups, TagResource, UntagResource, ListTagsForResource | 12 actions; schedule groups with cascading deletes; `rate()`, `cron()`, `at()` expressions; group/prefix/state filters on list; default group auto-created; CFN `AWS::Scheduler::Schedule` and `AWS::Scheduler::ScheduleGroup` supported |+| **EKS** | CreateCluster, DescribeCluster, ListClusters, DeleteCluster, CreateNodegroup, DescribeNodegroup, ListNodegroups, DeleteNodegroup, TagResource, UntagResource, ListTagsForResource | 11 operations; `CreateCluster` spawns a real **k3s** container (75 MB) with a full Kubernetes API server; `kubectl`, Helm, and any K8s tooling work out of the box; cascading delete removes nodegroups and k3s container; CFN `AWS::EKS::Cluster` and `AWS::EKS::Nodegroup` supported |---@@ -690,7 +691,7 @@ Install DuckDB for full Athena SQL compatibility: `pip install ministack[full]`.When `PERSIST_STATE=1`, MiniStack saves service state to `STATE_DIR` on shutdown and reloads it on startup. Writes are atomic (write-to-tmp then rename) to prevent corruption on crash.-Services currently supporting persistence: **All services** — API Gateway v1/v2, ALB, ACM, AppConfig, AppSync, Athena, Cloud Map, CloudFront, CloudWatch, CloudWatch Logs, CodeBuild, Cognito, DynamoDB, EC2, ECR, ECS, EFS, ElastiCache, EMR, EventBridge, EventBridge Scheduler, Firehose, Glue, IAM/STS, Kinesis, KMS, Lambda, RDS, Route 53, S3, Secrets Manager, SES, SES v2, SNS, SQS, SSM, Step Functions, Transfer Family, WAF v2+Services currently supporting persistence: **All services** — API Gateway v1/v2, ALB, ACM, AppConfig, AppSync, Athena, Cloud Map, CloudFront, CloudWatch, CloudWatch Logs, CodeBuild, Cognito, DynamoDB, EC2, ECR, ECS, EFS, EKS, ElastiCache, EMR, EventBridge, EventBridge Scheduler, Firehose, Glue, IAM/STS, Kinesis, KMS, Lambda, RDS, Route 53, S3, Secrets Manager, SES, SES v2, SNS, SQS, SSM, Step Functions, Transfer Family, WAF v2```bashdocker run -p 4566:4566 \@@ -742,7 +743,7 @@ services:networks:infra-network:healthcheck:- test: "python -c "import urllib.request; urllib.request.urlopen('http://localhost:4566/_ministack/health')" || exit 1"+ test: "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:4566/_ministack/health')\" || exit 1"interval: 10stimeout: 2sstart_period: 5s@@ -769,7 +770,38 @@ networks:infra-network:```-Option prvivileged set to "true" is needed if /var/run/docker.sock is root owned.+Option privileged set to "true" is needed if /var/run/docker.sock is root owned.++### EKS with Real Kubernetes (k3s)++MiniStack's EKS spawns a real [k3s](https://k3s.io) cluster (75 MB image) when you create a cluster. `kubectl`, Helm, and any Kubernetes tooling work out of the box.++```bash+# Create an EKS cluster — k3s starts automatically+aws --endpoint-url=http://localhost:4566 eks create-cluster \+ --name my-cluster --role-arn arn:aws:iam::000000000000:role/eks \+ --resources-vpc-config subnetIds=subnet-1++# Get the k3s kubeconfig (container name follows ministack-eks-{name} pattern)+docker exec ministack-eks-my-cluster cat /etc/rancher/k3s/k3s.yaml \+ | sed "s/127.0.0.1:6443/localhost:$(docker port ministack-eks-my-cluster 6443/tcp | cut -d: -f2)/" \+ > /tmp/ministack-kubeconfig.yaml++# Use kubectl against real Kubernetes+export KUBECONFIG=/tmp/ministack-kubeconfig.yaml+kubectl get nodes # Real k3s node, Ready status+kubectl create deployment nginx --image=nginx:alpine+kubectl get pods # Real pod running++# Helm works too+helm repo add bitnami https://charts.bitnami.com/bitnami+helm install my-redis bitnami/redis --set auth.enabled=false++# Clean up — k3s container is removed automatically+aws --endpoint-url=http://localhost:4566 eks delete-cluster --name my-cluster+```++> **Note:** EKS requires Docker socket access (`-v /var/run/docker.sock:/var/run/docker.sock`) to spawn k3s containers. The k3s image is pulled on first `CreateCluster` call.### Lambda Warm Startsdiff --git a/ministack/app.py b/ministack/app.pyindex dd10c59b..8295496d 100644--- a/ministack/app.py+++ b/ministack/app.py@@ -152,6 +152,7 @@ SERVICE_HANDLERS = {"appconfig": _lazy_handler("appconfig"),"appconfigdata": _lazy_handler("appconfig"),"scheduler": _lazy_handler("scheduler"),+ "eks": _lazy_handler("eks"),}SERVICE_NAME_ALIASES = {@@ -207,7 +208,7 @@ BANNER = r"""SSM, EventBridge, Kinesis, CloudWatch, SES, SES v2, ACM, WAF v2, Step Functions,ECS, RDS, ElastiCache, Glue, Athena, API Gateway, Firehose, Route53,Cognito, EC2, EMR, EBS, EFS, ALB/ELBv2, CloudFormation, KMS, ECR, CloudFront,- AppSync, Cloud Map, S3 Files, RDS Data API, CodeBuild, AppConfig, Transfer+ AppSync, Cloud Map, S3 Files, RDS Data API, CodeBuild, AppConfig, Transfer, EKS"""@@ -674,6 +675,7 @@ async def _handle_lifespan(scope, receive, send):"servicediscovery": "servicediscovery", "s3files": "s3files","appconfig": "appconfig", "transfer": "transfer","scheduler": "scheduler", "autoscaling": "autoscaling",+ "eks": "eks",}save_dict = {}for key, mod_name in _state_map.items():@@ -693,7 +695,7 @@ def _stop_docker_containers():client = docker.from_env()except Exception:return- for label in ("ministack=rds", "ministack=ecs", "ministack=elasticache"):+ for label in ("ministack=rds", "ministack=ecs", "ministack=elasticache", "ministack=eks", "ministack=lambda"):try:for c in client.containers.list(filters={"label": label}):try:@@ -827,7 +829,7 @@ def _reset_all_state():"apigateway", "apigateway_v1", "firehose", "route53", "cognito", "ec2","emr", "alb", "acm", "ses_v2", "waf", "efs", "cloudformation", "kms","cloudfront", "codebuild", "ecr", "appsync", "servicediscovery",- "rds_data", "s3files", "appconfig", "transfer", "scheduler", "autoscaling", "iam",+ "rds_data", "s3files", "appconfig", "transfer", "scheduler", "autoscaling", "eks", "iam",]for mod_name in _ALL_SERVICE_MODULES:if mod_name in _loaded_modules:diff --git a/ministack/core/lambda_runtime.py b/ministack/core/lambda_runtime.pyindex b90db3fe..efdc165c 100644--- a/ministack/core/lambda_runtime.py+++ b/ministack/core/lambda_runtime.py@@ -12,6 +12,7 @@ import osimport shutilimport subprocessimport sys+import queueimport tempfileimport threadingimport time@@ -464,7 +465,7 @@ class Worker:self._stderr_thread = threading.Thread(target=self._read_stderr, daemon=True, name=f"stderr-{self.func_name}")- self._stderr_thread.start()+ self._stderr_thread.start()init = {"code_dir": code_dir,diff --git a/ministack/core/persistence.py b/ministack/core/persistence.pyindex 8f046a22..b75fcfb5 100644--- a/ministack/core/persistence.py+++ b/ministack/core/persistence.py@@ -4,6 +4,7 @@ When PERSIST_STATE=1, service state is saved to STATE_DIR on shutdownand reloaded on startup."""+import astimport jsonimport loggingimport os@@ -37,8 +38,8 @@ def _json_object_hook(obj):account_id, key_repr = k.split("\x00", 1)# Restore the original key (was serialized with repr())try:- original_key = eval(key_repr) # noqa: S307- except Exception:+ original_key = ast.literal_eval(key_repr)+ except (ValueError, SyntaxError):original_key = key_reprasd._data[(account_id, original_key)] = vreturn asddiff --git a/ministack/core/router.py b/ministack/core/router.pyindex fe512ac5..4db6e59f 100644--- a/ministack/core/router.py+++ b/ministack/core/router.py@@ -210,6 +210,10 @@ SERVICE_PATTERNS = {"path_prefixes": ["/schedules", "/schedule-groups"],"credential_scope": "scheduler",},+ "eks": {+ "host_patterns": [r"eks\."],+ "credential_scope": "eks",+ },}@@ -271,6 +275,7 @@ def detect_service(method: str, path: str, headers: dict, query_params: dict) ->"appconfig": "appconfig","appconfigdata": "appconfigdata","scheduler": "scheduler",+ "eks": "eks",}if svc_name in scope_map:return scope_map[svc_name]diff --git a/ministack/services/athena.py b/ministack/services/athena.pyindex b4f78623..0d80a89e 100644--- a/ministack/services/athena.py+++ b/ministack/services/athena.py@@ -643,10 +643,11 @@ def _get_named_query(data):def _list_named_queries(data):- workgroup = data.get("WorkGroup", "primary")- ids = [- qid for qid, nq in _named_queries.items() if nq.get("WorkGroup") == workgroup- ]+ workgroup = data.get("WorkGroup")+ if workgroup:+ ids = [qid for qid, nq in _named_queries.items() if nq.get("WorkGroup") == workgroup]+ else:+ ids = list(_named_queries.keys())return json_response({"NamedQueryIds": ids})diff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.pyindex a039d25c..82f67076 100644--- a/ministack/services/cloudformation/provisioners.py+++ b/ministack/services/cloudformation/provisioners.py@@ -705,6 +705,63 @@ def _scheduler_group_delete(physical_id, props):_sched._tags.pop(group.get("Arn", ""), None)+# --- EKS Cluster ---++def _eks_cluster_create(logical_id, props, stack_name):+ import ministack.services.eks as _eks+ name = props.get("Name") or _physical_name(stack_name, logical_id, max_len=100)+ body = {+ "name": name,+ "version": props.get("Version", "1.30"),+ "roleArn": props.get("RoleArn", f"arn:aws:iam::{get_account_id()}:role/eks-role"),+ "resourcesVpcConfig": props.get("ResourcesVpcConfig", {}),+ "tags": {t["Key"]: t["Value"] for t in props.get("Tags", [])},+ }+ _eks._create_cluster(body)+ arn = _eks._cluster_arn(name)+ cluster = _eks._clusters.get(name, {})+ return name, {+ "Arn": arn,+ "Endpoint": cluster.get("endpoint", ""),+ "CertificateAuthorityData": cluster.get("certificateAuthority", {}).get("data", ""),+ "ClusterSecurityGroupId": cluster.get("resourcesVpcConfig", {}).get("clusterSecurityGroupId", ""),+ "OpenIdConnectIssuerUrl": cluster.get("identity", {}).get("oidc", {}).get("issuer", ""),+ }+++def _eks_cluster_delete(physical_id, props):+ import ministack.services.eks as _eks+ _eks._delete_cluster(physical_id)+++def _eks_nodegroup_create(logical_id, props, stack_name):+ import ministack.services.eks as _eks+ cluster_name = props.get("ClusterName", "")+ ng_name = props.get("NodegroupName") or _physical_name(stack_name, logical_id, max_len=63)+ body = {+ "nodegroupName": ng_name,+ "scalingConfig": props.get("ScalingConfig", {"minSize": 1, "maxSize": 2, "desiredSize": 1}),+ "instanceTypes": props.get("InstanceTypes", ["t3.medium"]),+ "subnets": props.get("Subnets", []),+ "nodeRole": props.get("NodeRole", f"arn:aws:iam::{get_account_id()}:role/eks-node-role"),+ "amiType": props.get("AmiType", "AL2_x86_64"),+ "diskSize": props.get("DiskSize", 20),+ "labels": props.get("Labels", {}),+ "tags": {t["Key"]: t["Value"] for t in props.get("Tags", [])},+ }+ _eks._create_nodegroup(cluster_name, body)+ key = f"{cluster_name}/{ng_name}"+ ng = _eks._nodegroups.get(key, {})+ arn = ng.get("nodegroupArn", "")+ return ng_name, {"Arn": arn}+++def _eks_nodegroup_delete(physical_id, props):+ import ministack.services.eks as _eks+ cluster_name = props.get("ClusterName", "")+ _eks._delete_nodegroup(cluster_name, physical_id)++# --- Kinesis Stream ---def _kinesis_stream_create(logical_id, props, stack_name):@@ -2751,6 +2808,9 @@ _RESOURCE_HANDLERS = {# EventBridge Scheduler"AWS::Scheduler::Schedule": {"create": _scheduler_schedule_create, "delete": _scheduler_schedule_delete},"AWS::Scheduler::ScheduleGroup": {"create": _scheduler_group_create, "delete": _scheduler_group_delete},+ # EKS+ "AWS::EKS::Cluster": {"create": _eks_cluster_create, "delete": _eks_cluster_delete},+ "AWS::EKS::Nodegroup": {"create": _eks_nodegroup_create, "delete": _eks_nodegroup_delete},# CDK metadata — safe to ignore"AWS::CDK::Metadata": {"create": lambda lid, props, sn: (f"CDKMetadata-{lid}", {}), "delete": lambda pid, props: None},# AutoScalingdiff --git a/ministack/services/cloudwatch.py b/ministack/services/cloudwatch.pyindex a80daaf2..1026d535 100644--- a/ministack/services/cloudwatch.py+++ b/ministack/services/cloudwatch.py@@ -1295,7 +1295,7 @@ def _delete_dashboards(params, cbor_data, is_cbor, is_json=False):if missing:return _error("DashboardNotFoundError",f"Dashboard {', '.join(missing)} does not exist",- use_json=is_json, use_cbor=is_cbor)+ 404, use_json=is_json, use_cbor=is_cbor)for n in names:_dashboards.pop(n, None)diff --git a/ministack/services/dynamodb.py b/ministack/services/dynamodb.pyindex 2193f0a4..f4013164 100644--- a/ministack/services/dynamodb.py+++ b/ministack/services/dynamodb.py@@ -271,6 +271,10 @@ def _create_table(data):"StreamSpecification": data.get("StreamSpecification"),"SSEDescription": data.get("SSESpecification"),}+ if data.get("StreamSpecification"):+ stream_label = now_iso()+ _tables[name]["LatestStreamLabel"] = stream_label+ _tables[name]["LatestStreamArn"] = f"{_tables[name]['TableArn']}/stream/{stream_label}"if data.get("Tags"):_tags[_tables[name]["TableArn"]] = data["Tags"]return json_response({"TableDescription": _table_description(name)})@@ -376,8 +380,8 @@ def _table_description(name):desc["LocalSecondaryIndexes"] = t["LocalSecondaryIndexes"]if t.get("StreamSpecification"):desc["StreamSpecification"] = t["StreamSpecification"]- desc["LatestStreamLabel"] = now_iso()- desc["LatestStreamArn"] = f"{t['TableArn']}/stream/{now_iso()}"+ desc["LatestStreamLabel"] = t.get("LatestStreamLabel", "")+ desc["LatestStreamArn"] = t.get("LatestStreamArn", "")if t.get("SSEDescription"):desc["SSEDescription"] = t["SSEDescription"]desc["WarmThroughput"] = t.get("WarmThroughput", {diff --git a/ministack/services/ec2.py b/ministack/services/ec2.pyindex d363cb29..870a0bd4 100644--- a/ministack/services/ec2.py+++ b/ministack/services/ec2.py@@ -431,6 +431,9 @@ def _describe_instance_status(p):def _terminate_instances(p):ids = _parse_member_list(p, "InstanceId")+ for iid in ids:+ if iid not in _instances:+ return _error("InvalidInstanceID.NotFound", f"The instance ID '{iid}' does not exist", 400)items = ""for iid in ids:inst = _instances.get(iid)@@ -448,6 +451,9 @@ def _terminate_instances(p):def _stop_instances(p):ids = _parse_member_list(p, "InstanceId")+ for iid in ids:+ if iid not in _instances:+ return _error("InvalidInstanceID.NotFound", f"The instance ID '{iid}' does not exist", 400)items = ""for iid in ids:inst = _instances.get(iid)@@ -464,6 +470,9 @@ def _stop_instances(p):def _start_instances(p):ids = _parse_member_list(p, "InstanceId")+ for iid in ids:+ if iid not in _instances:+ return _error("InvalidInstanceID.NotFound", f"The instance ID '{iid}' does not exist", 400)items = ""for iid in ids:inst = _instances.get(iid)@@ -627,6 +636,19 @@ def _sg_rule_xml(sg_id, rule, idx, is_egress=False):return items+def _strip_descriptions(rule):+ """Return a copy of rule with Description stripped from all range entries for comparison."""+ r = dict(rule)+ for key in ("IpRanges", "Ipv6Ranges"):+ r[key] = [{k: v for k, v in entry.items() if k != "Description"} for entry in r.get(key, [])]+ return r+++def _rules_match(a, b):+ """Compare two SG rules ignoring Description fields (matches AWS behavior)."""+ return _strip_descriptions(a) == _strip_descriptions(b)++def _authorize_sg_ingress(p):sg_id = _p(p, "GroupId")sg = _security_groups.get(sg_id)@@ -635,7 +657,7 @@ def _authorize_sg_ingress(p):rules = _parse_ip_permissions(p, "IpPermissions")rule_items = ""for r in rules:- if r in sg["IpPermissions"]:+ if any(_rules_match(r, existing) for existing in sg["IpPermissions"]):return _error("InvalidPermission.Duplicate", "The specified rule already exists", 400)sg["IpPermissions"].append(r)idx = len(sg["IpPermissions"]) - 1@@ -651,10 +673,7 @@ def _revoke_sg_ingress(p):return _error("InvalidGroup.NotFound", f"Security group {sg_id} not found", 400)rules = _parse_ip_permissions(p, "IpPermissions")for r in rules:- try:- sg["IpPermissions"].remove(r)- except ValueError:- pass+ sg["IpPermissions"] = [e for e in sg["IpPermissions"] if not _rules_match(r, e)]return _xml(200, "RevokeSecurityGroupIngressResponse", "<return>true</return>")@@ -666,7 +685,7 @@ def _authorize_sg_egress(p):rules = _parse_ip_permissions(p, "IpPermissions")rule_items = ""for r in rules:- if r not in sg["IpPermissionsEgress"]:+ if not any(_rules_match(r, existing) for existing in sg["IpPermissionsEgress"]):sg["IpPermissionsEgress"].append(r)idx = len(sg["IpPermissionsEgress"]) - 1rule_items += _sg_rule_xml(sg_id, r, idx, is_egress=True)@@ -681,10 +700,7 @@ def _revoke_sg_egress(p):return _error("InvalidGroup.NotFound", f"Security group {sg_id} not found", 400)rules = _parse_ip_permissions(p, "IpPermissions")for r in rules:- try:- sg["IpPermissionsEgress"].remove(r)- except ValueError:- pass+ sg["IpPermissionsEgress"] = [e for e in sg["IpPermissionsEgress"] if not _rules_match(r, e)]return _xml(200, "RevokeSecurityGroupEgressResponse", "<return>true</return>")@@ -2015,10 +2031,19 @@ def _perm_xml(r):def _vpc_fields_xml(vpc, tag="item"):+ cidr = vpc['CidrBlock']+ assoc_id = vpc.get('_cidr_assoc_id', f"vpc-cidr-assoc-{vpc['VpcId'][4:]}")return f"""<{tag}><vpcId>{vpc['VpcId']}</vpcId><state>{vpc['State']}</state>- <cidrBlock>{vpc['CidrBlock']}</cidrBlock>+ <cidrBlock>{cidr}</cidrBlock>+ <cidrBlockAssociationSet>+ <item>+ <cidrBlock>{cidr}</cidrBlock>+ <associationId>{assoc_id}</associationId>+ <cidrBlockState><state>associated</state></cidrBlockState>+ </item>+ </cidrBlockAssociationSet><dhcpOptionsId>{vpc['DhcpOptionsId']}</dhcpOptionsId><instanceTenancy>{vpc['InstanceTenancy']}</instanceTenancy><isDefault>{'true' if vpc['IsDefault'] else 'false'}</isDefault>diff --git a/ministack/services/ecs.py b/ministack/services/ecs.pyindex a3735b8d..410a0b7c 100644--- a/ministack/services/ecs.py+++ b/ministack/services/ecs.py@@ -1427,8 +1427,13 @@ def _list_services_by_namespace(data):def _put_account_setting_default(data):name = data.get("name", "")value = data.get("value", "")- _account_settings[name] = {"name": name, "value": value, "principalArn": f"arn:aws:iam::{os.environ.get('MINISTACK_ACCOUNT_ID', '000000000000')}:root"}- return json_response({"setting": _account_settings[name]})+ _account_settings[name] = value+ return json_response({"setting": {+ "name": name,+ "value": value,+ "principalArn": f"arn:aws:iam::{get_account_id()}:root",+ "type": "account",+ }})def _delete_account_setting(data):diff --git a/ministack/services/eks.py b/ministack/services/eks.pynew file mode 100644index 00000000..efdb8332--- /dev/null+++ b/ministack/services/eks.py@@ -0,0 +1,532 @@+"""+EKS Service Emulator.+REST/JSON protocol — /clusters/* and /clusters/*/node-groups/* paths.++CreateCluster spawns a k3s Docker container providing a real Kubernetes+API server. DeleteCluster stops and removes it.++Supports:+ Clusters: CreateCluster, DescribeCluster, ListClusters, DeleteCluster+ Nodegroups: CreateNodegroup, DescribeNodegroup, ListNodegroups, DeleteNodegroup+ Tags: TagResource, UntagResource, ListTagsForResource+"""++import base64+import copy+import importlib+import json+import logging+import os+import re+import threading+import time++from ministack.core.responses import AccountScopedDict, get_account_id, json_response, error_response_json, new_uuid++logger = logging.getLogger("eks")++REGION = os.environ.get("MINISTACK_REGION", "us-east-1")+EKS_K3S_IMAGE = os.environ.get("EKS_K3S_IMAGE", "rancher/k3s:latest")+EKS_BASE_PORT = int(os.environ.get("EKS_BASE_PORT", "16443"))++try:+ docker_lib = importlib.import_module("docker")+ _docker_available = True+except ImportError:+ docker_lib = None+ _docker_available = False++# ---------------------------------------------------------------------------+# State+# ---------------------------------------------------------------------------++_clusters = AccountScopedDict() # name -> cluster record+_nodegroups = AccountScopedDict() # "cluster/nodegroup" -> nodegroup record+_tags = AccountScopedDict() # arn -> {key: value}+_port_counter_lock = threading.Lock()+_port_counter = [EKS_BASE_PORT]+++def reset():+ _clusters.clear()+ _nodegroups.clear()+ _tags.clear()+ _port_counter[0] = EKS_BASE_PORT+ _stop_all_k3s()+++def get_state():+ clusters = copy.deepcopy(_clusters)+ # Strip Docker container IDs (not restorable across restarts)+ if isinstance(clusters, AccountScopedDict):+ for key in list(clusters._data):+ clusters._data[key].pop("_docker_id", None)+ else:+ for c in clusters.values():+ c.pop("_docker_id", None)+ return {+ "clusters": clusters,+ "nodegroups": copy.deepcopy(_nodegroups),+ "tags": copy.deepcopy(_tags),+ "port_counter": _port_counter[0],+ }+++def restore_state(data):+ _clusters.update(data.get("clusters", {}))+ _nodegroups.update(data.get("nodegroups", {}))+ _tags.update(data.get("tags", {}))+ if "port_counter" in data:+ _port_counter[0] = data["port_counter"]+ # Restored clusters have no running k3s container — mark status accordingly+ if isinstance(_clusters, AccountScopedDict):+ for key in list(_clusters._data):+ c = _clusters._data[key]+ if c.get("status") == "ACTIVE":+ c["status"] = "FAILED"+ c["_docker_id"] = None+ else:+ for c in _clusters.values():+ if c.get("status") == "ACTIVE":+ c["status"] = "FAILED"+ c["_docker_id"] = None+++# ---------------------------------------------------------------------------+# Helpers+# ---------------------------------------------------------------------------++def _next_port():+ with _port_counter_lock:+ port = _port_counter[0]+ _port_counter[0] += 1+ return port+++def _cluster_arn(name):+ return f"arn:aws:eks:{REGION}:{get_account_id()}:cluster/{name}"+++def _nodegroup_arn(cluster_name, ng_name):+ return f"arn:aws:eks:{REGION}:{get_account_id()}:nodegroup/{cluster_name}/{ng_name}/{new_uuid()[:8]}"+++def _now():+ return time.time()+++def _json_resp(status, body):+ return status, {"Content-Type": "application/json"}, json.dumps(body).encode()+++def _error(status, code, message):+ return _json_resp(status, {"__type": code, "message": message})+++def _get_docker():+ if not _docker_available:+ return None+ try:+ return docker_lib.from_env()+ except Exception:+ return None+++def _get_ministack_network(client):+ """Detect the Docker network MiniStack is running on."""+ try:+ hostname = os.environ.get("HOSTNAME", "")+ if not hostname:+ return None+ self_container = client.containers.get(hostname)+ nets = list(self_container.attrs["NetworkSettings"]["Networks"].keys())+ return nets[0] if nets else None+ except Exception:+ return None+++def _wait_for_port(host, port, timeout=30):+ import socket+ deadline = time.time() + timeout+ while time.time() < deadline:+ try:+ with socket.create_connection((host, port), timeout=2):+ return True+ except (OSError, ConnectionRefusedError):+ time.sleep(0.5)+ return False+++def _stop_all_k3s():+ """Stop all k3s containers managed by MiniStack."""+ client = _get_docker()+ if not client:+ return+ try:+ for c in client.containers.list(filters={"label": "ministack=eks"}):+ try:+ c.stop(timeout=5)+ c.remove(v=True, force=True)+ except Exception:+ pass+ except Exception:+ pass+++def _extract_ca_cert(container, timeout=30):+ """Extract the CA certificate from a running k3s container."""+ deadline = time.time() + timeout+ while time.time() < deadline:+ try:+ _, output = container.exec_run("cat /var/lib/rancher/k3s/server/tls/server-ca.crt")+ cert = output.decode("utf-8", errors="replace").strip()+ if cert.startswith("-----BEGIN CERTIFICATE-----"):+ return base64.b64encode(cert.encode()).decode()+ except Exception:+ pass+ time.sleep(1)+ return ""+++# ---------------------------------------------------------------------------+# Clusters+# ---------------------------------------------------------------------------++def _create_cluster(body):+ name = body.get("name", "")+ if not name:+ return _error(400, "InvalidParameterException", "Cluster name is required.")+ if name in _clusters:+ return _error(409, "ResourceInUseException", f"Cluster already exists with name: {name}")++ arn = _cluster_arn(name)+ now = _now()+ version = body.get("version", "1.30")+ role_arn = body.get("roleArn", f"arn:aws:iam::{get_account_id()}:role/eks-role")+ vpc_config = body.get("resourcesVpcConfig", {})++ # Spawn k3s container+ endpoint = ""+ ca_data = ""+ container_id = None+ port = _next_port()++ client = _get_docker()+ if client:+ try:+ ms_network = _get_ministack_network(client)+ run_kwargs = dict(+ image=EKS_K3S_IMAGE,+ command=["server",+ "--disable=traefik,metrics-server,servicelb",+ "--tls-san=0.0.0.0",+ f"--https-listen-port=6443"],+ detach=True,+ privileged=True,+ ports={"6443/tcp": port},+ name=f"ministack-eks-{name}",+ labels={"ministack": "eks", "cluster_name": name},+ environment={+ "K3S_KUBECONFIG_MODE": "644",+ },+ tmpfs={"/run": "", "/var/run": ""},+ )+ if ms_network:+ run_kwargs["network"] = ms_network++ container = client.containers.run(**run_kwargs)+ container_id = container.id++ # Wait for API server+ if ms_network:+ container.reload()+ networks = container.attrs.get("NetworkSettings", {}).get("Networks", {})+ container_ip = networks.get(ms_network, {}).get("IPAddress", "")+ if container_ip and _wait_for_port(container_ip, 6443):+ endpoint = f"https://{container_ip}:6443"+ logger.info("EKS: k3s for %s ready at %s (network %s)", name, endpoint, ms_network)+ if not endpoint:+ if _wait_for_port("127.0.0.1", port):+ endpoint = f"https://localhost:{port}"+ logger.info("EKS: k3s for %s ready at %s", name, endpoint)+ else:+ logger.warning("EKS: k3s for %s did not become ready on port %d", name, port)+ endpoint = f"https://localhost:{port}"++ ca_data = _extract_ca_cert(container)++ except Exception as e:+ logger.warning("EKS: failed to start k3s for %s: %s", name, e)+ endpoint = f"https://localhost:{port}"+ else:+ endpoint = f"https://localhost:{port}"+ logger.info("EKS: Docker unavailable — cluster %s created without k3s backend", name)++ cluster = {+ "name": name,+ "arn": arn,+ "createdAt": now,+ "version": version,+ "endpoint": endpoint,+ "roleArn": role_arn,+ "resourcesVpcConfig": {+ "subnetIds": vpc_config.get("subnetIds", []),+ "securityGroupIds": vpc_config.get("securityGroupIds", []),+ "clusterSecurityGroupId": f"sg-{new_uuid()[:17].replace('-', '')}",+ "vpcId": vpc_config.get("vpcId", "vpc-00000000"),+ "endpointPublicAccess": vpc_config.get("endpointPublicAccess", True),+ "endpointPrivateAccess": vpc_config.get("endpointPrivateAccess", False),+ "publicAccessCidrs": vpc_config.get("publicAccessCidrs", ["0.0.0.0/0"]),+ },+ "kubernetesNetworkConfig": {+ "serviceIpv4Cidr": body.get("kubernetesNetworkConfig", {}).get("serviceIpv4Cidr", "10.100.0.0/16"),+ "ipFamily": "ipv4",+ },+ "logging": body.get("logging", {"clusterLogging": []}),+ "identity": {+ "oidc": {"issuer": f"https://oidc.eks.{REGION}.amazonaws.com/id/{new_uuid()[:32].replace('-', '').upper()}"}+ },+ "status": "ACTIVE",+ "certificateAuthority": {"data": ca_data},+ "platformVersion": f"eks.{int(time.time()) % 100}",+ "tags": body.get("tags", {}),+ "encryptionConfig": body.get("encryptionConfig", []),+ "accessConfig": body.get("accessConfig", {}),+ "_docker_id": container_id,+ "_port": port,+ }++ _clusters[name] = cluster+ if cluster["tags"]:+ _tags[arn] = dict(cluster["tags"])++ return _json_resp(200, {"cluster": _sanitize(cluster)})+++def _describe_cluster(name):+ cluster = _clusters.get(name)+ if not cluster:+ return _error(404, "ResourceNotFoundException", f"No cluster found for name: {name}.")+ return _json_resp(200, {"cluster": _sanitize(cluster)})+++def _list_clusters(query):+ max_results = int(query.get("maxResults", 100))+ names = list(_clusters.keys())[:max_results]+ return _json_resp(200, {"clusters": names})+++def _delete_cluster(name):+ cluster = _clusters.get(name)+ if not cluster:+ return _error(404, "ResourceNotFoundException", f"No cluster found for name: {name}.")++ # Stop k3s container+ container_id = cluster.get("_docker_id")+ if container_id:+ client = _get_docker()+ if client:+ try:+ c = client.containers.get(container_id)+ c.stop(timeout=5)+ c.remove(v=True, force=True)+ logger.info("EKS: stopped k3s container for %s", name)+ except Exception as e:+ logger.warning("EKS: failed to stop k3s for %s: %s", name, e)++ # Delete all nodegroups in this cluster+ ng_keys = [k for k in _nodegroups if k.startswith(f"{name}/")]+ for k in ng_keys:+ ng = _nodegroups.pop(k, None)+ if ng:+ _tags.pop(ng.get("nodegroupArn", ""), None)++ arn = cluster["arn"]+ cluster["status"] = "DELETING"+ result = _sanitize(cluster)+ _clusters.pop(name, None)+ _tags.pop(arn, None)++ return _json_resp(200, {"cluster": result})+++# ---------------------------------------------------------------------------+# Nodegroups+# ---------------------------------------------------------------------------++def _create_nodegroup(cluster_name, body):+ if cluster_name not in _clusters:+ return _error(404, "ResourceNotFoundException", f"No cluster found for name: {cluster_name}.")++ ng_name = body.get("nodegroupName", "")+ if not ng_name:+ return _error(400, "InvalidParameterException", "Nodegroup name is required.")++ key = f"{cluster_name}/{ng_name}"+ if key in _nodegroups:+ return _error(409, "ResourceInUseException", f"Nodegroup already exists with name: {ng_name}")++ arn = _nodegroup_arn(cluster_name, ng_name)+ now = _now()+ scaling = body.get("scalingConfig", {"minSize": 1, "maxSize": 2, "desiredSize": 1})++ nodegroup = {+ "nodegroupName": ng_name,+ "nodegroupArn": arn,+ "clusterName": cluster_name,+ "version": body.get("version", _clusters[cluster_name].get("version", "1.30")),+ "releaseVersion": body.get("releaseVersion", ""),+ "createdAt": now,+ "modifiedAt": now,+ "status": "ACTIVE",+ "capacityType": body.get("capacityType", "ON_DEMAND"),+ "scalingConfig": scaling,+ "instanceTypes": body.get("instanceTypes", ["t3.medium"]),+ "subnets": body.get("subnets", []),+ "amiType": body.get("amiType", "AL2_x86_64"),+ "nodeRole": body.get("nodeRole", f"arn:aws:iam::{get_account_id()}:role/eks-node-role"),+ "labels": body.get("labels", {}),+ "taints": body.get("taints", []),+ "diskSize": body.get("diskSize", 20),+ "health": {"issues": []},+ "resources": {+ "autoScalingGroups": [{"name": f"eks-{ng_name}-{new_uuid()[:8]}"}],+ "remoteAccessSecurityGroup": f"sg-{new_uuid()[:17].replace('-', '')}",+ },+ "tags": body.get("tags", {}),+ }++ _nodegroups[key] = nodegroup+ if nodegroup["tags"]:+ _tags[arn] = dict(nodegroup["tags"])++ return _json_resp(200, {"nodegroup": nodegroup})+++def _describe_nodegroup(cluster_name, ng_name):+ key = f"{cluster_name}/{ng_name}"+ ng = _nodegroups.get(key)+ if not ng:+ return _error(404, "ResourceNotFoundException",+ f"No node group found for name: {ng_name}.")+ return _json_resp(200, {"nodegroup": ng})+++def _list_nodegroups(cluster_name, query):+ if cluster_name not in _clusters:+ return _error(404, "ResourceNotFoundException", f"No cluster found for name: {cluster_name}.")+ max_results = int(query.get("maxResults", 100))+ names = [ng["nodegroupName"] for k, ng in _nodegroups.items()+ if k.startswith(f"{cluster_name}/")][:max_results]+ return _json_resp(200, {"nodegroups": names})+++def _delete_nodegroup(cluster_name, ng_name):+ key = f"{cluster_name}/{ng_name}"+ ng = _nodegroups.get(key)+ if not ng:+ return _error(404, "ResourceNotFoundException",+ f"No node group found for name: {ng_name}.")+ ng["status"] = "DELETING"+ result = dict(ng)+ _nodegroups.pop(key, None)+ _tags.pop(ng.get("nodegroupArn", ""), None)+ return _json_resp(200, {"nodegroup": result})+++# ---------------------------------------------------------------------------+# Tags+# ---------------------------------------------------------------------------++def _tag_resource(arn, body):+ tags = body.get("tags", {})+ existing = _tags.get(arn, {})+ existing.update(tags)+ _tags[arn] = existing+ return _json_resp(200, {})+++def _untag_resource(arn, query):+ keys = query.get("tagKeys", [])+ if isinstance(keys, str):+ keys = [keys]+ existing = _tags.get(arn, {})+ for k in keys:+ existing.pop(k, None)+ if existing:+ _tags[arn] = existing+ else:+ _tags.pop(arn, None)+ return _json_resp(200, {})+++def _list_tags(arn):+ return _json_resp(200, {"tags": _tags.get(arn, {})})+++# ---------------------------------------------------------------------------+# Sanitize (remove internal fields)+# ---------------------------------------------------------------------------++def _sanitize(cluster):+ return {k: v for k, v in cluster.items() if not k.startswith("_")}+++# ---------------------------------------------------------------------------+# Request Router+# ---------------------------------------------------------------------------++async def handle_request(method, path, headers, body_bytes, query_params):+ try:+ body = json.loads(body_bytes) if body_bytes else {}+ except json.JSONDecodeError:+ body = {}++ query = {k: (v[0] if isinstance(v, list) else v) for k, v in query_params.items()}++ # POST /clusters+ if path == "/clusters" and method == "POST":+ return _create_cluster(body)++ # GET /clusters+ if path == "/clusters" and method == "GET":+ return _list_clusters(query)++ # /clusters/{name}+ m = re.fullmatch(r"/clusters/([A-Za-z0-9_-]+)", path)+ if m:+ name = m.group(1)+ if method == "GET":+ return _describe_cluster(name)+ if method == "DELETE":+ return _delete_cluster(name)++ # POST /clusters/{name}/node-groups+ m = re.fullmatch(r"/clusters/([A-Za-z0-9_-]+)/node-groups", path)+ if m:+ cluster_name = m.group(1)+ if method == "POST":+ return _create_nodegroup(cluster_name, body)+ if method == "GET":+ return _list_nodegroups(cluster_name, query)++ # /clusters/{name}/node-groups/{ngName}+ m = re.fullmatch(r"/clusters/([A-Za-z0-9_-]+)/node-groups/([A-Za-z0-9_-]+)", path)+ if m:+ cluster_name, ng_name = m.group(1), m.group(2)+ if method == "GET":+ return _describe_nodegroup(cluster_name, ng_name)+ if method == "DELETE":+ return _delete_nodegroup(cluster_name, ng_name)++ # Tags: /tags/{arn+}+ if path.startswith("/tags/"):+ arn = path[6:]+ if method == "GET":+ return _list_tags(arn)+ if method == "POST":+ return _tag_resource(arn, body)+ if method == "DELETE":+ return _untag_resource(arn, query)++ return _error(400, "InvalidRequestException", f"No route for {method} {path}")diff --git a/ministack/services/elasticache.py b/ministack/services/elasticache.pyindex 33e6758a..415bf585 100644--- a/ministack/services/elasticache.py+++ b/ministack/services/elasticache.py@@ -424,8 +424,8 @@ def _create_replication_group(p):"AuthTokenEnabled": _p(p, "AuthToken") != "","TransitEncryptionEnabled": _p(p, "TransitEncryptionEnabled", "false").lower() == "true","AtRestEncryptionEnabled": _p(p, "AtRestEncryptionEnabled", "false").lower() == "true",- "AutomaticFailover": _p(p, "AutomaticFailoverEnabled", "false"),- "MultiAZ": _p(p, "MultiAZEnabled", "false"),+ "AutomaticFailover": "enabled" if _p(p, "AutomaticFailoverEnabled", "false").lower() == "true" else "disabled",+ "MultiAZ": "enabled" if _p(p, "MultiAZEnabled", "false").lower() == "true" else "disabled","ConfigurationEndpoint": {"Address": endpoint_host, "Port": endpoint_port} if num_node_groups > 1 else None,"ARN": arn,"_num_node_groups": num_node_groups,@@ -481,9 +481,9 @@ def _modify_replication_group(p):if _p(p, "SnapshotWindow"):rg["SnapshotWindow"] = _p(p, "SnapshotWindow")if _p(p, "AutomaticFailoverEnabled"):- rg["AutomaticFailover"] = _p(p, "AutomaticFailoverEnabled")+ rg["AutomaticFailover"] = "enabled" if _p(p, "AutomaticFailoverEnabled").lower() == "true" else "disabled"if _p(p, "MultiAZEnabled"):- rg["MultiAZ"] = _p(p, "MultiAZEnabled")+ rg["MultiAZ"] = "enabled" if _p(p, "MultiAZEnabled").lower() == "true" else "disabled"if _p(p, "EngineVersion"):rg["EngineVersion"] = _p(p, "EngineVersion")if _p(p, "CacheParameterGroupName"):@@ -570,10 +570,16 @@ def _create_subnet_group(p):"Subnets": subnets,"ARN": arn,}+ subnets_xml = "".join(+ f"<Subnet><SubnetIdentifier>{s['SubnetIdentifier']}</SubnetIdentifier>"+ f"<SubnetAvailabilityZone><Name>{s['SubnetAvailabilityZone']['Name']}</Name></SubnetAvailabilityZone>"+ f"</Subnet>" for s in subnets+ )return _xml(200, "CreateCacheSubnetGroupResponse",f"<CreateCacheSubnetGroupResult><CacheSubnetGroup>"f"<CacheSubnetGroupName>{name}</CacheSubnetGroupName>"f"<CacheSubnetGroupDescription>{desc}</CacheSubnetGroupDescription>"+ f"<Subnets>{subnets_xml}</Subnets>"f"<ARN>{arn}</ARN>"f"</CacheSubnetGroup></CreateCacheSubnetGroupResult>")diff --git a/ministack/services/iam.py b/ministack/services/iam.pyindex 449abcd5..1d96e0af 100644--- a/ministack/services/iam.py+++ b/ministack/services/iam.py@@ -1085,7 +1085,11 @@ def _put_user_policy(p):if user_name not in _users:return _error(404, "NoSuchEntity",f"The user with name {user_name} cannot be found.", ns="iam")- _user_inline_policies[(user_name, policy_name)] = policy_doc+ user_policies = _user_inline_policies.get(user_name)+ if user_policies is None:+ user_policies = {}+ _user_inline_policies[user_name] = user_policies+ user_policies[policy_name] = policy_docreturn _xml(200, "PutUserPolicyResponse", "", ns="iam")@@ -1095,7 +1099,7 @@ def _get_user_policy(p):if user_name not in _users:return _error(404, "NoSuchEntity",f"The user with name {user_name} cannot be found.", ns="iam")- doc = _user_inline_policies.get((user_name, policy_name))+ doc = (_user_inline_policies.get(user_name) or {}).get(policy_name)if doc is None:return _error(404, "NoSuchEntity",f"The user policy with name {policy_name} cannot be found.", ns="iam")@@ -1121,10 +1125,11 @@ def _delete_user_policy(p):if user_name not in _users:return _error(404, "NoSuchEntity",f"The user with name {user_name} cannot be found.", ns="iam")- if (user_name, policy_name) not in _user_inline_policies:+ user_policies = _user_inline_policies.get(user_name) or {}+ if policy_name not in user_policies:return _error(404, "NoSuchEntity",f"The user policy with name {policy_name} cannot be found.", ns="iam")- del _user_inline_policies[(user_name, policy_name)]+ del user_policies[policy_name]return _xml(200, "DeleteUserPolicyResponse", "", ns="iam")@@ -1135,8 +1140,7 @@ def _list_user_policies(p):f"The user with name {user_name} cannot be found.", ns="iam")members = "".join(f"<member>{pname}</member>"- for (uname, pname) in _user_inline_policies- if uname == user_name+ for pname in (_user_inline_policies.get(user_name) or {}))return _xml(200, "ListUserPoliciesResponse",f"<ListUserPoliciesResult><PolicyNames>{members}</PolicyNames>"diff --git a/ministack/services/kms.py b/ministack/services/kms.pyindex fd363347..82e2c01b 100644--- a/ministack/services/kms.py+++ b/ministack/services/kms.py@@ -147,6 +147,23 @@ def _resolve_key(key_id_or_arn):return None+def _check_key_state(rec):+ """Return an error response if the key is in an unusable state, else None."""+ if rec["KeyState"] == "PendingDeletion":+ return error_response_json(+ "KMSInvalidStateException",+ f"{rec['Arn']} is pending deletion.",+ 400,+ )+ if rec["KeyState"] == "Disabled":+ return error_response_json(+ "DisabledException",+ f"{rec['Arn']} is disabled.",+ 400,+ )+ return None++def _require_crypto(operation):if not HAS_CRYPTO:return error_response_json(@@ -291,7 +308,7 @@ def _get_public_key(data):400,)return json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"KeyUsage": rec["KeyUsage"],"KeySpec": rec["KeySpec"],"PublicKey": base64.b64encode(rec["_public_key_der"]).decode(),@@ -309,6 +326,9 @@ def _sign(data):rec = _resolve_key(key_id)if not rec:return error_response_json("NotFoundException", f"Key {key_id} not found", 400)+ err = _check_key_state(rec)+ if err:+ return errif "_private_key" not in rec:return error_response_json("UnsupportedOperationException",@@ -352,7 +372,7 @@ def _sign(data):logger.debug("Signed %d bytes with key %s (%s)", len(message), key_id, algorithm)return json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"Signature": base64.b64encode(signature).decode(),"SigningAlgorithm": algorithm,})@@ -367,6 +387,9 @@ def _verify(data):rec = _resolve_key(key_id)if not rec:return error_response_json("NotFoundException", f"Key {key_id} not found", 400)+ err = _check_key_state(rec)+ if err:+ return errif "_private_key" not in rec:return error_response_json("UnsupportedOperationException",@@ -406,11 +429,15 @@ def _verify(data):public_key.verify(signature, message, pad, hash_algo)valid = Trueexcept InvalidSignature:- valid = False+ return error_response_json(+ "KMSInvalidSignatureException",+ "Signature verification failed",+ 400,+ )return json_response({- "KeyId": rec["KeyId"],- "SignatureValid": valid,+ "KeyId": rec["Arn"],+ "SignatureValid": True,"SigningAlgorithm": algorithm,})@@ -466,6 +493,9 @@ def _encrypt(data):rec = _resolve_key(key_id)if not rec:return error_response_json("NotFoundException", f"Key {key_id} not found", 400)+ err = _check_key_state(rec)+ if err:+ return errplaintext_b64 = data.get("Plaintext", "")plaintext = base64.b64decode(plaintext_b64) if isinstance(plaintext_b64, str) else plaintext_b64@@ -511,7 +541,7 @@ def _encrypt(data):)return json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"CiphertextBlob": base64.b64encode(ciphertext).decode(),"EncryptionAlgorithm": data.get("EncryptionAlgorithm", "SYMMETRIC_DEFAULT"@@ -542,6 +572,9 @@ def _decrypt(data):"Unable to find the key for decryption",400,)+ err = _check_key_state(rec)+ if err:+ return errif "_symmetric_key" in rec:stored_ctx_hash = ciphertext[36:68]@@ -591,7 +624,7 @@ def _decrypt(data):)return json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"Plaintext": base64.b64encode(plaintext).decode(),"EncryptionAlgorithm": data.get("EncryptionAlgorithm", "SYMMETRIC_DEFAULT"@@ -607,6 +640,9 @@ def _generate_data_key_common(data):return None, None, error_response_json("NotFoundException", f"Key {key_id} not found", 400)+ err = _check_key_state(rec)+ if err:+ return None, None, errif "_symmetric_key" not in rec:return None, None, error_response_json("UnsupportedOperationException",@@ -643,7 +679,7 @@ def _generate_data_key(data):# result is an error response tuplereturn resultreturn json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"Plaintext": base64.b64encode(data_key).decode(),"CiphertextBlob": base64.b64encode(result).decode(),})@@ -654,7 +690,7 @@ def _generate_data_key_without_plaintext(data):if rec is None:return resultreturn json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"CiphertextBlob": base64.b64encode(result).decode(),})@@ -817,7 +853,7 @@ def _schedule_key_deletion(data):rec["Enabled"] = Falserec["DeletionDate"] = int(time.time() + (days * 86400))return json_response({- "KeyId": rec["KeyId"],+ "KeyId": rec["Arn"],"KeyState": "PendingDeletion","DeletionDate": rec["DeletionDate"],})@@ -829,7 +865,7 @@ def _cancel_key_deletion(data):return error_response_json("NotFoundException", f"Key {data.get('KeyId', '')} not found", 400)rec["KeyState"] = "Disabled"rec.pop("DeletionDate", None)- return json_response({"KeyId": rec["KeyId"]})+ return json_response({"KeyId": rec["Arn"]})# ---- Tags ----diff --git a/ministack/services/lambda_svc.py b/ministack/services/lambda_svc.pyindex ba93cdb8..b724e7ef 100644--- a/ministack/services/lambda_svc.py+++ b/ministack/services/lambda_svc.py@@ -29,35 +29,23 @@ import base64import copyimport hashlibimport importlib-import ioimport jsonimport loggingimport os-import pathlib-import randomimport reimport subprocessimport tempfileimport threadingimport time-import typingimport zipfilefrom datetime import datetime, timezone+from typing import Anyfrom urllib.parse import unquote-from ministack.core.persistence import load_state-from ministack.core.responses import AccountScopedDict, get_account_id, error_response_json, json_response, new_uuid+from ministack.core.persistence import load_state, PERSIST_STATE+from ministack.core.responses import AccountScopedDict, get_account_id, _request_account_id, error_response_json, json_response, new_uuidfrom ministack.core.lambda_runtime import get_or_create_worker, invalidate_worker-if typing.TYPE_CHECKING:- from mypy_boto3_lambda.type_defs import FunctionConfigurationResponseTypeDef, FunctionConfigurationTypeDef- from mypy_boto3_lambda.literals import RuntimeType- from docker.models.containers import Container-- class LambdaConfigType(FunctionConfigurationResponseTypeDef):- ImageUri: typing.Optional[str]--logger = logging.getLogger("lambda")REGION = os.environ.get("MINISTACK_REGION", os.environ.get("AWS_DEFAULT_REGION", "us-east-1"))@@ -65,11 +53,8 @@ LAMBDA_EXECUTOR = os.environ.get("LAMBDA_EXECUTOR", "local").lower()LAMBDA_DOCKER_VOLUME_MOUNT = os.environ.get("LAMBDA_REMOTE_DOCKER_VOLUME_MOUNT", "")LAMBDA_DOCKER_NETWORK = os.environ.get("LAMBDA_DOCKER_NETWORK", "")-if typing.TYPE_CHECKING:- from docker import DockerClient-try:- docker_lib: "DockerClient" = importlib.import_module("docker")+ docker_lib: Any = importlib.import_module("docker")_docker_available = Trueexcept ImportError:docker_lib = None@@ -82,32 +67,12 @@ _function_urls = AccountScopedDict() # function_name -> FunctionUrlConfig dict_poller_started = False_poller_lock = threading.Lock()-_containers_lock = threading.Lock()---class ContainerScopedDict(AccountScopedDict):- def clear(self):- """Stop running ccontainers when clear"""- with _containers_lock:- for containers in self._data.values():- for container in containers:- try:- container.stop()- container.remove()- except Exception:- pass- super().clear()---_containers = ContainerScopedDict()# ── Persistence ────────────────────────────────────────────-def get_state():"""Return JSON-serializable state. code_zip bytes are base64-encoded."""from ministack.core.responses import AccountScopedDict-funcs = AccountScopedDict()# Iterate _data directly to capture ALL accounts, not just current request contextfor scoped_key, func in _functions._data.items():@@ -129,7 +94,6 @@ def get_state():def restore_state(data):if data:from ministack.core.responses import AccountScopedDict-funcs = data.get("functions", {})if isinstance(funcs, AccountScopedDict):for scoped_key, func in funcs._data.items():@@ -163,12 +127,163 @@ if _restored:# Wrapper script executed inside the subprocess.# All configuration is passed through env vars; event data arrives on stdin.# ----------------------------------------------------------------------------import ministack.core-_WRAPPER_SCRIPT = (pathlib.Path(ministack.core.__file__).parent / "lambda_wrapper.py").read_text()+_WRAPPER_SCRIPT = """\+import sys, os, json++sys.path.insert(0, os.environ["_LAMBDA_CODE_DIR"])++_REAL_STDOUT = sys.__stdout__+# Match AWS Lambda semantics: logs go to CloudWatch (stderr here),+# while the Invoke response payload must be clean JSON on stdout.+sys.stdout = sys.stderr++for _ld in filter(None, os.environ.get("_LAMBDA_LAYERS_DIRS", "").split(os.pathsep)):+ _py = os.path.join(_ld, "python")+ if os.path.isdir(_py):+ sys.path.insert(0, _py)+ sys.path.insert(0, _ld)++_mod_path = os.environ["_LAMBDA_HANDLER_MODULE"]+_fn_name = os.environ["_LAMBDA_HANDLER_FUNC"]++event = json.loads(sys.stdin.read())++class LambdaContext:+ function_name = os.environ.get("AWS_LAMBDA_FUNCTION_NAME", "")+ memory_limit_in_mb = int(os.environ.get("AWS_LAMBDA_FUNCTION_MEMORY_SIZE", "128"))+ invoked_function_arn = os.environ.get("_LAMBDA_FUNCTION_ARN", "")+ aws_request_id = os.environ.get("AWS_LAMBDA_LOG_STREAM_NAME", "")+ log_group_name = "/aws/lambda/" + function_name+ log_stream_name = aws_request_id++ @staticmethod+ def get_remaining_time_in_millis():+ return int(float(os.environ.get("_LAMBDA_TIMEOUT", "3")) * 1000)++_mod = __import__(_mod_path)+for _part in _mod_path.split(".")[1:]:+ _mod = getattr(_mod, _part)+_result = getattr(_mod, _fn_name)(event, LambdaContext())+if _result is not None:+ _REAL_STDOUT.write(json.dumps(_result))+ _REAL_STDOUT.flush()+"""++# Docker variant: paths fixed to /var/task (code) and /opt (layers).+_DOCKER_WRAPPER_SCRIPT = """\+import sys, os, json++sys.path.insert(0, "/var/task")++_REAL_STDOUT = sys.__stdout__+# Match AWS Lambda semantics: logs go to CloudWatch (stderr here),+# while the Invoke response payload must be clean JSON on stdout.+sys.stdout = sys.stderr++for _ld in filter(None, os.environ.get("_LAMBDA_LAYERS_DIRS", "").split(":")):+ _py = os.path.join(_ld, "python")+ if os.path.isdir(_py):+ sys.path.insert(0, _py)+ sys.path.insert(0, _ld)++_mod_path = os.environ["_LAMBDA_HANDLER_MODULE"]+_fn_name = os.environ["_LAMBDA_HANDLER_FUNC"]++event = json.loads(sys.stdin.read())++class LambdaContext:+ function_name = os.environ.get("AWS_LAMBDA_FUNCTION_NAME", "")+ memory_limit_in_mb = int(os.environ.get("AWS_LAMBDA_FUNCTION_MEMORY_SIZE", "128"))+ invoked_function_arn = os.environ.get("_LAMBDA_FUNCTION_ARN", "")+ aws_request_id = os.environ.get("AWS_LAMBDA_LOG_STREAM_NAME", "")+ log_group_name = "/aws/lambda/" + function_name+ log_stream_name = aws_request_id++ @staticmethod+ def get_remaining_time_in_millis():+ return int(float(os.environ.get("_LAMBDA_TIMEOUT", "3")) * 1000)++_mod = __import__(_mod_path)+for _part in _mod_path.split(".")[1:]:+ _mod = getattr(_mod, _part)+_result = getattr(_mod, _fn_name)(event, LambdaContext())+if _result is not None:+ _REAL_STDOUT.write(json.dumps(_result))+ _REAL_STDOUT.flush()+"""+# Node.js wrapper — written to the code dir and executed with `node`.# Reads event from stdin, calls handler, writes JSON result to stdout.-_NODE_WRAPPER_SCRIPT = (pathlib.Path(ministack.core.__file__).parent / "lambda_wrapper_node.js").read_text()+_NODE_WRAPPER_SCRIPT = """\+const fs = require('fs');+const path = require('path');+const { pathToFileURL } = require('url');++const codeDir = process.env._LAMBDA_CODE_DIR || '/var/task';+const modPath = process.env._LAMBDA_HANDLER_MODULE;+const fnName = process.env._LAMBDA_HANDLER_FUNC;++// Prepend layer dirs to NODE_PATH+const layerDirs = (process.env._LAMBDA_LAYERS_DIRS || '').split(path.delimiter).filter(Boolean);+const nodePaths = layerDirs.map(d => path.join(d, 'nodejs', 'node_modules'))+ .concat(layerDirs)+ .concat([path.join(codeDir, 'node_modules'), codeDir]);+module.paths.unshift(...nodePaths);++const context = {+ functionName: process.env.AWS_LAMBDA_FUNCTION_NAME || '',+ memoryLimitInMB: process.env.AWS_LAMBDA_FUNCTION_MEMORY_SIZE || '128',+ invokedFunctionArn: process.env._LAMBDA_FUNCTION_ARN || '',+ awsRequestId: process.env.AWS_LAMBDA_LOG_STREAM_NAME || '',+ logGroupName: '/aws/lambda/' + (process.env.AWS_LAMBDA_FUNCTION_NAME || ''),+ logStreamName: process.env.AWS_LAMBDA_LOG_STREAM_NAME || '',+ getRemainingTimeInMillis: () => parseFloat(process.env._LAMBDA_TIMEOUT || '3') * 1000,+};++let input = '';+process.stdin.on('data', d => input += d);+process.stdin.on('end', async () => {+ const event = JSON.parse(input);+ const fullPath = path.resolve(codeDir, modPath);+ let mod;+ let resolvedPath;+ try {+ resolvedPath = require.resolve(fullPath);+ mod = require(resolvedPath);+ } catch (reqErr) {+ if (reqErr.code === 'ERR_REQUIRE_ESM' && resolvedPath) {+ mod = await import(pathToFileURL(resolvedPath).href);+ } else if (reqErr.code === 'MODULE_NOT_FOUND') {+ const mjsPath = fullPath + '.mjs';+ const missingHandlerEntry =+ (reqErr.message && reqErr.message.includes("'" + fullPath + "'")) ||+ (resolvedPath && reqErr.message && reqErr.message.includes("'" + resolvedPath + "'"));+ if (missingHandlerEntry && fs.existsSync(mjsPath)) {+ mod = await import(pathToFileURL(mjsPath).href);+ } else {+ throw reqErr;+ }+ } else {+ throw reqErr;+ }+ }+ const handler = mod[fnName] || (mod.default && mod.default[fnName]) || mod.default;+ if (typeof handler !== 'function') {+ process.stderr.write(+ "Handler '" + fnName + "' in module '" + modPath + "' is undefined or not a function"+ );+ process.exit(1);+ }+ Promise.resolve(handler(event, context)).then(result => {+ if (result !== undefined) process.stdout.write(JSON.stringify(result));+ }).catch(err => {+ process.stderr.write(String(err.stack || err));+ process.exit(1);+ });+});+"""+# ---------------------------------------------------------------------------# Helpers@@ -239,7 +354,6 @@ def _fetch_code_from_s3(bucket: str, key: str) -> bytes | None:"""Fetch Lambda code zip from the in-memory S3 service."""try:from ministack.services import s3 as s3_svc-obj = s3_svc._get_object_data(bucket, key)if obj is not None:return obj@@ -248,7 +362,7 @@ def _fetch_code_from_s3(bucket: str, key: str) -> bytes | None:return None-def _build_config(name: str, data: dict, code_zip: bytes | None = None) -> "LambdaConfigType":+def _build_config(name: str, data: dict, code_zip: bytes | None = None) -> dict:code_size = len(code_zip) if code_zip else 0code_sha = base64.b64encode(hashlib.sha256(code_zip).digest()).decode() if code_zip else ""is_image = data.get("PackageType", "Zip") == "Image"@@ -260,58 +374,33 @@ def _build_config(name: str, data: dict, code_zip: bytes | None = None) -> "Lambelif isinstance(layer, dict):layers_cfg.append(layer)- env = data.get("Environment", {"Variables": {}})+ env = data.get("Environment")+ if env is not None and "Variables" not in env:+ env["Variables"] = {}- config: "LambdaConfigType" = {- "Architectures": data.get("Architectures", ["x86_64"]),- "CapacityProviderConfig": {- "LambdaManagedInstancesCapacityProviderConfig": {"CapacityProviderArn": "lambda-default"}- },- "ConfigSha256": data.get("LambdaConfigSha256", ""),- "CodeSha256": code_sha,- "CodeSize": code_size,- "DeadLetterConfig": data.get("DeadLetterConfig", {}),- "Description": data.get("Description", ""),- "DurableConfig": data.get("DurableConfig", {}),- "Environment": env,- "EphemeralStorage": data.get("EphemeralStorage", {"Size": 512}),- "FileSystemConfigs": data.get("FileSystemConfigs", []),- "FunctionArn": _func_arn(name),+ config = {"FunctionName": name,+ "FunctionArn": _func_arn(name),+ "Runtime": data.get("Runtime", "" if is_image else "python3.12"),+ "Role": data.get("Role", f"arn:aws:iam::{get_account_id()}:role/lambda-role"),"Handler": data.get("Handler", "" if is_image else "index.handler"),- "ImageConfigResponse": {},- "ImageUri": data.get("ImageUri", ""),- "KMSKeyArn": data.get("KMSKeyArn", ""),+ "CodeSize": code_size,+ "CodeSha256": code_sha,+ "Description": data.get("Description", ""),+ "Timeout": data.get("Timeout", 3),+ "MemorySize": data.get("MemorySize", 128),"LastModified": _now_iso(),+ "Version": "$LATEST",+ "State": "Active",+ "StateReason": "",+ "StateReasonCode": "","LastUpdateStatus": "Successful","LastUpdateStatusReason": "",- "LastUpdateStatusReasonCode": data.get("LastUpdateStatusReasonCode", ""),- "Layers": layers_cfg,- "LoggingConfig": data.get(- "LoggingConfig",- {- "LogFormat": "Text",- "LogGroup": f"/aws/lambda/{name}",- },- ),- "MasterArn": data.get("MasterArn", ""),- "MemorySize": data.get("MemorySize", 128),+ "LastUpdateStatusReasonCode": "","PackageType": data.get("PackageType", "Zip"),- "Role": data.get("Role", f"arn:aws:iam::{get_account_id()}:role/lambda-role"),- "Runtime": data.get("Runtime", "" if is_image else "python3.9"),- "SigningJobArn": data.get("SigningJobArn", ""),- "SigningProfileVersionArn": data.get("SigningProfileVersionArn", ""),- "State": "Active",- "StateReason": "",- "StateReasonCode": data.get("StateReasonCode", "Idle"),- "ResponseMetadata": data.get("ResponseMetadata", {}),- "RevisionId": new_uuid(),- "RuntimeVersionConfig": data.get("RuntimeVersionConfig", {}),- "SnapStart": {"ApplyOn": "None", "OptimizationStatus": "Off"},- "TenancyConfig": data.get("TenancyConfig", {}),- "Timeout": data.get("Timeout", 3),+ "Architectures": data.get("Architectures", ["x86_64"]),+ "Layers": layers_cfg,"TracingConfig": data.get("TracingConfig", {"Mode": "PassThrough"}),- "Version": "$LATEST","VpcConfig": data.get("VpcConfig",{@@ -320,10 +409,31 @@ def _build_config(name: str, data: dict, code_zip: bytes | None = None) -> "Lamb"VpcId": "",},),+ "KMSKeyArn": data.get("KMSKeyArn", ""),+ "RevisionId": new_uuid(),+ "EphemeralStorage": data.get("EphemeralStorage", {"Size": 512}),+ "SnapStart": {"ApplyOn": "None", "OptimizationStatus": "Off"},+ "LoggingConfig": data.get(+ "LoggingConfig",+ {+ "LogFormat": "Text",+ "LogGroup": f"/aws/lambda/{name}",+ },+ ),+ "RuntimeVersionConfig": {+ "RuntimeVersionArn": "",+ },}+ if env is not None:+ config["Environment"] = env+ dlc = data.get("DeadLetterConfig")+ if dlc and dlc.get("TargetArn"):+ config["DeadLetterConfig"] = dlcreturn config++def _qp_first(query_params: dict, key: str, default: str = "") -> str:"""Return the first value for *key* from raw query_params (list or str)."""val = query_params.get(key, default)@@ -872,7 +982,7 @@ async def _invoke(name: str, event: dict, headers: dict, path_qualifier: str | N# RequestResponse — execute in worker thread so nested SDK calls# from the Lambda process can still reach this ASGI server.- result = await asyncio.to_thread(_execute_function, exec_record, event, async_mode=False)+ result = await asyncio.to_thread(_execute_function, exec_record, event)resp_headers: dict = {"Content-Type": "application/json",@@ -901,8 +1011,7 @@ async def _invoke(name: str, event: dict, headers: dict, path_qualifier: str | N# Runtime → Docker image mapping# -----------------------------------------------------------------------------_RUNTIME_IMAGE_MAP: "dict[RuntimeType, str]" = {+_RUNTIME_IMAGE_MAP: dict[str, str] = {"python3.8": "public.ecr.aws/lambda/python:3.8","python3.9": "public.ecr.aws/lambda/python:3.9","python3.10": "public.ecr.aws/lambda/python:3.10",@@ -918,300 +1027,310 @@ _RUNTIME_IMAGE_MAP: "dict[RuntimeType, str]" = {"nodejs24.x": "public.ecr.aws/lambda/nodejs:24","provided.al2023": "public.ecr.aws/lambda/provided:al2023","provided.al2": "public.ecr.aws/lambda/provided:al2",- "provided": "public.ecr.aws/lambda/provided:al2023",+ "provided": "public.ecr.aws/lambda/provided:latest",}-class LambdaFunc(typing.TypedDict):- config: "LambdaConfigType"- code_zip: bytes | None+def _docker_image_for_runtime(runtime: str) -> str | None:+ if runtime in _RUNTIME_IMAGE_MAP:+ return _RUNTIME_IMAGE_MAP[runtime]+ if runtime.startswith("python"):+ ver = runtime.replace("python", "")+ return f"public.ecr.aws/lambda/python:{ver}"+ if runtime.startswith("nodejs"):+ ver = runtime.replace("nodejs", "").rstrip(".x")+ return f"public.ecr.aws/lambda/nodejs:{ver}"+ if runtime.startswith("provided"):+ return "public.ecr.aws/lambda/provided:al2023"+ return None# ----------------------------------------------------------------------------# Function execution – Docker mode+# Function execution – Docker mode (RIE with warm containers)# ---------------------------------------------------------------------------+# Warm container cache: {cache_key: {"container": Container, "last_used": float}}+_warm_containers: dict[str, dict] = {}+_warm_containers_lock = threading.Lock()+_WARM_CONTAINER_TTL = 300 # 5 minutes, matching AWS warm container behaviour-def _get_function_code(config: "LambdaConfigType") -> str:- "Get a unique path for lambda with package type zip"- func_name = config["FunctionName"]- sha256 = config["CodeSha256"]- return f"/var/task/{func_name}/{sha256}-code"+def _warm_container_key(func_name: str, code_sha: str) -> str:+ acct = get_account_id()+ return f"{acct}:{func_name}:{code_sha}"-def _get_image_key(config: "LambdaConfigType") -> str:- "Get a unique path for a lambda with package type image"- if (image_uri := config.get("ImageUri")) is None:- raise ValueError("ImageUri is required for Image package type")- version = config["Version"]- return f"/var/task/{image_uri}/{version}"++def _get_warm_container(key: str):+ """Return a running warm container for the key, or None."""+ with _warm_containers_lock:+ entry = _warm_containers.get(key)+ if not entry:+ return None+ container = entry["container"]+ try:+ container.reload()+ if container.status == "running":+ entry["last_used"] = time.time()+ return container+ except Exception:+ pass+ # Container dead — remove from cache+ _warm_containers.pop(key, None)+ return None-def _execute_function_docker_rie(func: LambdaFunc, event: dict, async_mode: bool = True) -> dict:- """- Execute docker using AWS Runtime Interface Emulator (RIE). This allows us to- run the Lambda function in an environment very close to the actual Lambda- runtime, using the same base images and emulation layer as AWS.- If there is a running one, which is not doing something, we will- reuse it.- """- if not _docker_available:- return {- "body": {- "errorMessage": "Docker is required for Image-based Lambda functions",- "errorType": "Runtime.DockerUnavailable",- },- "error": True,- }- config = func["config"]- func_name = config["FunctionName"]- version = config["Version"]- # Need to modify key for package type image.- container_key = _get_function_code(config) if config["PackageType"] == "Zip" else _get_image_key(config)- with _containers_lock:- _containers.setdefault(container_key, [])- # Gert a local copy- containers = _containers[container_key].copy()-- # Shuffle order to divide the load- random.shuffle(containers)-- for container in containers:- container.reload()- if container.status == "running":+def _cache_warm_container(key: str, container):+ """Store a running container in the warm cache."""+ with _warm_containers_lock:+ # Evict old entry if exists+ old = _warm_containers.pop(key, None)+ if old:try:- ret = _invoke_function_in_running_container(container, event, async_mode=async_mode, tries=1)- logger.info("Successfully reused container for function %s (version: %s)", func_name, version)- return ret- except Exception as e:- logger.info(- "Could not reuse container (most likely busy) for function %s (version: %s): %s",- func_name,- version,- e,- )- container, ret = _invoke_function_in_container(func, event, async_mode=async_mode)- if container:- with _containers_lock:- _containers[container_key].append(container)- logger.info("Started container for function %s (version: %s)", func_name, version)- return ret---def _get_aws_endpoint_url() -> str | None:- return _normalize_endpoint_url(- os.environ.get(- "AWS_ENDPOINT_URL",- _normalize_endpoint_url(- os.environ.get("AWS_ENDPOINT_URL", _normalize_endpoint_url(os.environ.get("LOCALSTACK_HOSTNAME", "")))- ),- )- )+ old["container"].stop(timeout=2)+ old["container"].remove(force=True)+ except Exception:+ pass+ _warm_containers[key] = {"container": container, "last_used": time.time()}-def _invoke_function_in_container(- func: LambdaFunc, event: dict, async_mode: bool = True, tries=5-) -> "tuple[Container | None, dict]":- config = func["config"]- func_name = config["FunctionName"]- runtime = config["Runtime"]- sha256 = config["CodeSha256"]- code_zip = func["code_zip"]-- running_ministack_in_container = os.path.exists("/run/.containerenv") or os.path.exists("/.dockerenv")-- mounts = []- env_vars = config["Environment"].get("Variables") or {}- match config["PackageType"]:- case "Zip":- image = _RUNTIME_IMAGE_MAP[runtime]- if code_zip is None:- return None, {- "body": {- "errorMessage": "No code found for Zip package type",- "errorType": "Runtime.NoCode",- },- "error": True,- }- function_code = _get_function_code(config)- # We need volume mount for the code.- if not LAMBDA_DOCKER_VOLUME_MOUNT:- return None, {- "body": {- "errorMessage": "LAMBDA_DOCKER_VOLUME_MOUNT must be set to use lambda Docker execution",- "errorType": "Runtime.MissingConfiguration",- },- "error": True,- }+def _cleanup_warm_containers():+ """Remove all warm containers. Called on reset() and shutdown."""+ with _warm_containers_lock:+ for key, entry in list(_warm_containers.items()):+ try:+ entry["container"].stop(timeout=2)+ except Exception:+ pass+ try:+ entry["container"].remove(force=True)+ except Exception:+ pass+ _warm_containers.clear()- if running_ministack_in_container and not LAMBDA_DOCKER_NETWORK:- raise RuntimeError(- "LAMBDA_DOCKER_NETWORK must be set when running inside a container to use lambda Docker execution"- )- if not os.path.exists(function_code):- os.makedirs(function_code)- with zipfile.ZipFile(io.BytesIO(code_zip)) as zf:- zf.extractall(function_code)- # Need the mount to be read-write for RIE to work, but we can set the- # container side to read-only- mounts.append(- dict(- type="volume",- source=LAMBDA_DOCKER_VOLUME_MOUNT,- target="/var/task",- volume_subpath=function_code[1:],- read_only=True,- )- )- function_code_abs = os.path.abspath(function_code)- env_vars.update(- {- "LAMBDA_TASK_ROOT": function_code_abs,- "AWS_LAMBDA_FUNCTION_NAME": config["FunctionName"],- "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": str(config.get("MemorySize", 128)),- "AWS_LAMBDA_FUNCTION_VERSION": config.get("Version", "$LATEST"),- }- )- case "Image":- image_uri = config.get("ImageUri")- # We need volume mount for the layers.- if config["Layers"] and not LAMBDA_DOCKER_VOLUME_MOUNT:- return None, {- "body": {- "errorMessage": "LAMBDA_DOCKER_VOLUME_MOUNT must be set to use lambda Docker execution",- "errorType": "Runtime.MissingConfiguration",- },- "error": True,- }- if not image_uri:- return None, {- "body": {- "errorMessage": "ImageUri is required for Image package type",- "errorType": "InvalidConfiguration",- },- "error": True,- }- image = image_uri- env_vars = {}- def _wait_for_container_running(container: "Container", timeout: int = 30, interval: float = 0.2):- start = time.time()- while time.time() - start < timeout:- container.reload()- if container.status == "running":- break- time.sleep(interval)- else:- raise TimeoutError("Container did not reach running state")-- # If we have layers, we need to extract them and mount them under /opt- if config.get("Layers"):- function_layers = f"/var/task/{func_name}/{sha256}-layers"- if not os.path.exists(function_layers):- os.mkdir(function_layers)- for layer_ref in config["Layers"]:- layer_arn_str = layer_ref.get("Arn", "")- layer_zip = _resolve_layer_zip(layer_arn_str)- if layer_zip:- layer_name = function_layers + "/" + layer_arn_str.split(":")[-1]- with zipfile.ZipFile(io.BytesIO(layer_zip)) as zf:- zf.extractall(layer_name)-- mounts.append(- dict(- type="volume",- source=LAMBDA_DOCKER_VOLUME_MOUNT,- target="/opt",- volume_subpath=function_layers[1:],- read_only=True,+def _invoke_rie(container, event: dict, timeout: int) -> dict:+ """POST event to a running RIE container's HTTP endpoint."""+ import urllib.request+ max_attempts = int(timeout * 2) + 10+ for _attempt in range(max_attempts):+ time.sleep(0.5)+ container.reload()+ if container.status != "running":+ break+ try:+ networks = container.attrs.get("NetworkSettings", {}).get("Networks", {})+ # Try Docker network first (container-to-container)+ container_ip = None+ if LAMBDA_DOCKER_NETWORK:+ container_ip = networks.get(LAMBDA_DOCKER_NETWORK, {}).get("IPAddress", "")+ if container_ip:+ rie_url = f"http://{container_ip}:8080/2015-03-31/functions/function/invocations"+ else:+ ports = container.ports.get("8080/tcp") or []+ if not ports:+ continue+ rie_url = f"http://127.0.0.1:{ports[0]['HostPort']}/2015-03-31/functions/function/invocations"+ req = urllib.request.Request(+ rie_url, data=json.dumps(event).encode(),+ headers={"Content-Type": "application/json"},)- )- # Set the task root- env_vars.update(- {+ resp = urllib.request.urlopen(req, timeout=timeout)+ body = resp.read().decode("utf-8", errors="replace")+ try:+ parsed = json.loads(body)+ except json.JSONDecodeError:+ parsed = body+ logs = container.logs(stdout=True, stderr=True).decode("utf-8", errors="replace").strip()+ return {"body": parsed, "log": logs}+ except (urllib.error.URLError, ConnectionRefusedError, OSError):+ continue+ # Timed out+ stdout = container.logs(stdout=True, stderr=True).decode("utf-8", errors="replace").strip()+ return {+ "body": {"errorMessage": f"Lambda RIE failed: {stdout[:500]}", "errorType": "Runtime.ExitError"},+ "error": True, "log": stdout,+ }+++def _execute_function_docker(func: dict, event: dict) -> dict:+ """Execute a Lambda function inside a Docker container using AWS RIE.++ All runtimes (Python, Node.js, provided) use the official AWS Lambda+ Runtime Interface Emulator images. The handler is passed as CMD, and+ the event is POSTed to the RIE HTTP endpoint on port 8080 — matching+ exact AWS Lambda execution semantics.++ Containers are kept warm between invocations (like real AWS) and reused+ when the same function+code is invoked again. Warm containers are cleaned+ up on reset() and shutdown.+ """+ config = func.get("config") or func+ runtime = config.get("Runtime", "python3.12")++ if not _docker_available:+ if runtime.startswith("python") or runtime.startswith("nodejs"):+ logger.warning("docker SDK unavailable - falling back to warm executor (node/py detected)")+ return _execute_function_warm(func, event)+ logger.warning("docker SDK unavailable - falling back to local subprocess executor")+ return _execute_function_local(func, event)++ code_zip = func.get("code_zip")+ if not code_zip:+ return {"body": {"statusCode": 200, "body": "Mock response - no code deployed"}}++ handler = config["Handler"]+ timeout = config.get("Timeout", 3)+ env_vars = config.get("Environment", {}).get("Variables", {})+ code_sha = config.get("CodeSha256", "")++ image = _docker_image_for_runtime(runtime)+ if image is None:+ return {"body": {"statusCode": 200, "body": f"Mock response - {runtime} not supported for docker execution"}}++ # Try reusing a warm container+ cache_key = _warm_container_key(config["FunctionName"], code_sha)+ warm = _get_warm_container(cache_key)+ if warm:+ try:+ result = _invoke_rie(warm, event, timeout)+ if not result.get("error"):+ logger.debug("Lambda %s: warm RIE container reused", config["FunctionName"])+ return result+ except Exception:+ pass+ # Warm container failed — kill it, start fresh+ with _warm_containers_lock:+ _warm_containers.pop(cache_key, None)+ try:+ warm.stop(timeout=2)+ warm.remove(force=True)+ except Exception:+ pass++ # Cold start — create new container+ try:+ client = docker_lib.from_env()+ except Exception as exc:+ logger.warning("Cannot connect to Docker daemon (%s) – falling back to local subprocess", exc)+ return _execute_function_local(func, event)++ # Code and layers are extracted to a persistent temp dir (not cleaned up+ # immediately) so the warm container can keep using the bind mounts.+ tmpdir = tempfile.mkdtemp(prefix="ministack-lambda-docker-")+ container = None+ try:+ code_dir = os.path.join(tmpdir, "code")+ os.makedirs(code_dir)+ with open(os.path.join(tmpdir, "code.zip"), "wb") as f:+ f.write(code_zip)+ with zipfile.ZipFile(os.path.join(tmpdir, "code.zip")) as zf:+ zf.extractall(code_dir)++ # For provided runtimes, ensure bootstrap is executable+ is_provided = runtime.startswith("provided")+ if is_provided:+ bootstrap_path = os.path.join(code_dir, "bootstrap")+ if os.path.exists(bootstrap_path):+ os.chmod(bootstrap_path, 0o755)++ # Extract layers+ layers_dirs: list[str] = []+ for layer_ref in config.get("Layers", []):+ layer_arn_str = layer_ref if isinstance(layer_ref, str) else layer_ref.get("Arn", "")+ layer_zip = _resolve_layer_zip(layer_arn_str)+ if layer_zip:+ layer_dir = os.path.join(tmpdir, f"layer_{len(layers_dirs)}")+ os.makedirs(layer_dir)+ with open(os.path.join(tmpdir, f"layer_{len(layers_dirs)}.zip"), "wb") as lf:+ lf.write(layer_zip)+ with zipfile.ZipFile(os.path.join(tmpdir, f"layer_{len(layers_dirs)}.zip")) as lzf:+ lzf.extractall(layer_dir)+ layers_dirs.append(layer_dir)++ # Build mounts — RIE expects code at /var/task, layers at /opt+ host_code_dir = LAMBDA_DOCKER_VOLUME_MOUNT or code_dir+ mounts = [+ docker_lib.types.Mount("/var/task", host_code_dir, type="bind", read_only=True),+ ]+ if is_provided:+ mounts.append(docker_lib.types.Mount("/var/runtime", host_code_dir, type="bind", read_only=True))+ for idx, ld in enumerate(layers_dirs):+ mounts.append(docker_lib.types.Mount(f"/opt/layer_{idx}", ld, type="bind", read_only=True))++ # Build environment+ container_env: dict[str, str] = {"AWS_DEFAULT_REGION": REGION,"AWS_REGION": REGION,"AWS_ACCESS_KEY_ID": os.environ.get("AWS_ACCESS_KEY_ID", "test"),"AWS_SECRET_ACCESS_KEY": os.environ.get("AWS_SECRET_ACCESS_KEY", "test"),+ "AWS_SESSION_TOKEN": os.environ.get("AWS_SESSION_TOKEN", ""),+ "AWS_LAMBDA_FUNCTION_NAME": config["FunctionName"],+ "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": str(config["MemorySize"]),+ "AWS_LAMBDA_FUNCTION_VERSION": config.get("Version", "$LATEST"),"AWS_LAMBDA_LOG_STREAM_NAME": new_uuid(),+ "_LAMBDA_FUNCTION_ARN": config.get("FunctionArn", ""),+ "_LAMBDA_TIMEOUT": str(timeout),}- )- if endpoint := _get_aws_endpoint_url():- env_vars["AWS_ENDPOINT_URL"] = endpoint- container_port = "8080"- # mounts is not documented, therefore use kwargs.- run_kwargs: dict = {- "image": image,- "environment": env_vars,- "mounts": mounts,- "mem_limit": f"{config['MemorySize']}m",- "detach": True,- "stdin_open": False,- }- if running_ministack_in_container:- run_kwargs["network"] = LAMBDA_DOCKER_NETWORK- else:- run_kwargs["network_mode"] = "host"- run_kwargs["ports"] = {f"{container_port}/tcp": None}+ if is_provided:+ container_env["LAMBDA_TASK_ROOT"] = "/var/task"+ container_env["_HANDLER"] = handler+ if layers_dirs:+ container_env["_LAMBDA_LAYERS_DIRS"] = ":".join(+ f"/opt/layer_{i}" for i in range(len(layers_dirs))+ )+ container_env.update(env_vars)+ endpoint = _normalize_endpoint_url(os.environ.get("AWS_ENDPOINT_URL", ""))+ if not endpoint:+ endpoint = _normalize_endpoint_url(env_vars.get("AWS_ENDPOINT_URL", ""))+ if not endpoint:+ endpoint = _normalize_endpoint_url(env_vars.get("LOCALSTACK_HOSTNAME", ""))+ if endpoint:+ container_env["AWS_ENDPOINT_URL"] = endpoint++ # RIE: pass handler as CMD, invoke via HTTP on port 8080+ cmd = ["bootstrap"] if is_provided else [handler]+ run_kwargs: dict = {+ "image": image,+ "command": cmd,+ "environment": container_env,+ "mounts": mounts,+ "ports": {"8080/tcp": None},+ "detach": True,+ "stdin_open": False,+ "labels": {"ministack": "lambda"},+ }+ if LAMBDA_DOCKER_NETWORK:+ run_kwargs["network"] = LAMBDA_DOCKER_NETWORK- client = docker_lib.from_env()- container: "Container" = client.containers.run(- command=[config["Handler"]], name=f"lambda_{random.randbytes(8).hex()}", **run_kwargs- )- _wait_for_container_running(container)- return container, _invoke_function_in_running_container(container, event, async_mode=async_mode, tries=tries)+ container = client.containers.run(**run_kwargs)+ result = _invoke_rie(container, event, timeout)+ if not result.get("error"):+ # Success — cache the warm container for reuse+ _cache_warm_container(cache_key, container)+ container = None # prevent finally from killing it+ logger.info("Lambda %s: cold start RIE container cached for reuse", config["FunctionName"])+ return result-def _invoke_function_in_running_container(- container: "Container", event: dict, async_mode: bool = True, tries=5-) -> dict:- import urllib.request- import urllib.error-- container_port = "8080"- running_ministack_in_container = os.path.exists("/run/.containerenv") or os.path.exists("/.dockerenv")- localstack_hostname = os.getenv("LOCALSTACK_HOSTNAME") or "localhost"- if running_ministack_in_container:- host_name = container.name- host_port = container_port- else:- ports = container.ports[container_port]- host_port = ports[0]["HostPort"]- host_name = localstack_hostname- headers = {- "Content-Type": "application/json",- }- if async_mode:- headers["X-Amz-Invocation-Type"] = "Event"- try:- for retry in range(tries):- time.sleep(0.1 * (2**retry))+ except Exception as exc:+ if "timed out" in str(exc).lower() or "read timed out" in str(exc).lower():+ return {+ "body": {"errorMessage": f"Task timed out after {timeout}.00 seconds", "errorType": "Runtime.ExitError"},+ "error": True, "log": "",+ }+ logger.error("Lambda docker execution error: %s", exc)+ return {"body": {"errorMessage": str(exc), "errorType": type(exc).__name__}, "error": True, "log": ""}+ finally:+ # Only clean up if we're NOT caching this container+ if container is not None:try:- rie_url = f"http://{host_name}:{host_port}/2015-03-31/functions/function/invocations"- req = urllib.request.Request(rie_url, data=json.dumps(event).encode(), headers=headers)- resp = urllib.request.urlopen(req)- body = resp.read().decode("utf-8", errors="replace")- try:- ret = {"body": json.loads(body)}- except json.JSONDecodeError:- ret = {"body": body}-- if async_mode:- logs = container.logs(stdout=True, stderr=True).decode("utf-8", errors="replace").strip()- ret["log"] = logs- logger.debug("Invocation successful for container %s: %s", container.name, ret)- return ret- except (urllib.error.URLError, ConnectionRefusedError, OSError):- logger.debug("Failed to connect to RIE endpoint, retrying... (%d)", retry + 1)- else:- raise Exception("Failed to connect to RIE endpoint after multiple retries")- except (urllib.error.URLError, ConnectionRefusedError, OSError):- stdout = container.logs(stdout=True, stderr=True).decode("utf-8", errors="replace").strip()- return {- "body": {"errorMessage": f"provided runtime failed: {stdout[:500]}", "errorType": "Runtime.ExitError"},- "error": True,- "log": stdout,- }+ container.stop(timeout=2)+ except Exception:+ pass+ try:+ container.remove(force=True)+ except Exception:+ pass# ---------------------------------------------------------------------------@@ -1219,14 +1338,19 @@ def _invoke_function_in_running_container(# ----------------------------------------------------------------------------def _execute_function(func: dict, event: dict, async_mode: bool = True) -> dict:+def _execute_function(func: dict, event: dict) -> dict:"""Dispatch to warm worker pool (Python + Node.js) or Docker executor."""+ config = func.get("config") or func++ # Image-based Lambda — always Docker+ if config.get("PackageType") == "Image" and config.get("ImageUri"):+ return _execute_function_image(func, event)++ runtime = config.get("Runtime", "python3.12")if LAMBDA_EXECUTOR == "docker":- return _execute_function_docker_rie(func, event, async_mode=async_mode) # type: ignore+ return _execute_function_docker(func, event)- config: LambdaConfigType = func["config"]- runtime = config.get("Runtime", "python3.9")# provided runtimes (Go, Rust, etc.) — use native binary executionif runtime.startswith("provided"):return _execute_function_provided(func, event)@@ -1261,6 +1385,7 @@ def _execute_function_warm(func: dict, event: dict) -> dict:}except Exception as e:logger.error("Warm worker execution error for %s: %s", func_name, e)+ invalidate_worker(func_name)return {"body": {"errorMessage": str(e), "errorType": type(e).__name__},"error": True,@@ -1268,6 +1393,116 @@ def _execute_function_warm(func: dict, event: dict) -> dict:}+# ---------------------------------------------------------------------------+# Function execution – Docker Image mode (PackageType: Image)+# ---------------------------------------------------------------------------+++def _execute_function_image(func: dict, event: dict) -> dict:+ """Execute a Lambda function from a user-provided Docker image.++ The image must contain the Lambda Runtime Interface Client (RIC) or+ Runtime Interface Emulator (RIE) listening on port 8080.+ """+ if not _docker_available:+ return {"body": {"errorMessage": "Docker is required for Image-based Lambda functions", "errorType": "Runtime.DockerUnavailable"}, "error": True}++ config = func.get("config") or func+ image_uri = config.get("ImageUri", "")+ timeout = config.get("Timeout", 30)+ env_vars = config.get("Environment", {}).get("Variables", {})++ try:+ client = docker_lib.from_env()+ except Exception as exc:+ return {"body": {"errorMessage": f"Cannot connect to Docker: {exc}", "errorType": "Runtime.DockerError"}, "error": True}++ # Use image locally if available, otherwise pull+ try:+ client.images.get(image_uri)+ logger.info("Using local Lambda image: %s", image_uri)+ except docker_lib.errors.ImageNotFound:+ try:+ logger.info("Pulling Lambda image: %s", image_uri)+ client.images.pull(image_uri)+ except Exception as exc:+ return {"body": {"errorMessage": f"Failed to pull image {image_uri}: {exc}", "errorType": "Runtime.ImagePullError"}, "error": True}++ container_env = {+ "AWS_DEFAULT_REGION": REGION,+ "AWS_REGION": REGION,+ "AWS_ACCESS_KEY_ID": os.environ.get("AWS_ACCESS_KEY_ID", "test"),+ "AWS_SECRET_ACCESS_KEY": os.environ.get("AWS_SECRET_ACCESS_KEY", "test"),+ "AWS_LAMBDA_FUNCTION_NAME": config["FunctionName"],+ "AWS_LAMBDA_FUNCTION_MEMORY_SIZE": str(config.get("MemorySize", 128)),+ "AWS_LAMBDA_FUNCTION_VERSION": config.get("Version", "$LATEST"),+ "AWS_LAMBDA_LOG_STREAM_NAME": new_uuid(),+ }+ container_env.update(env_vars)+ # Override AWS_ENDPOINT_URL *after* function env vars so the+ # Lambda container always calls back to this MiniStack instance.+ endpoint = _normalize_endpoint_url(os.environ.get("AWS_ENDPOINT_URL", ""))+ if not endpoint:+ endpoint = _normalize_endpoint_url(env_vars.get("AWS_ENDPOINT_URL", ""))+ if endpoint:+ container_env["AWS_ENDPOINT_URL"] = endpoint++ run_kwargs = {+ "image": image_uri,+ "environment": container_env,+ "ports": {"8080/tcp": None},+ "detach": True,+ }+ if LAMBDA_DOCKER_NETWORK:+ run_kwargs["network"] = LAMBDA_DOCKER_NETWORK++ container = client.containers.run(**run_kwargs)+ try:+ import time as _time+ import urllib.request+ for _attempt in range(int(timeout * 2) + 10):+ _time.sleep(0.5)+ container.reload()+ if container.status != "running":+ break+ try:+ if LAMBDA_DOCKER_NETWORK:+ networks = container.attrs.get("NetworkSettings", {}).get("Networks", {})+ container_ip = networks.get(LAMBDA_DOCKER_NETWORK, {}).get("IPAddress", "")+ if container_ip:+ rie_url = f"http://{container_ip}:8080/2015-03-31/functions/function/invocations"+ else:+ continue+ else:+ ports = container.ports.get("8080/tcp") or []+ if not ports:+ continue+ rie_url = f"http://127.0.0.1:{ports[0]['HostPort']}/2015-03-31/functions/function/invocations"+ req = urllib.request.Request(rie_url, data=json.dumps(event).encode(),+ headers={"Content-Type": "application/json"})+ resp = urllib.request.urlopen(req, timeout=timeout)+ body = resp.read().decode("utf-8", errors="replace")+ try:+ parsed = json.loads(body)+ except json.JSONDecodeError:+ parsed = body+ logs = container.logs(stdout=True, stderr=True).decode("utf-8", errors="replace").strip()+ return {"body": parsed, "log": logs}+ except (urllib.error.URLError, ConnectionRefusedError, OSError):+ continue+ stdout = container.logs(stdout=True, stderr=True).decode("utf-8", errors="replace").strip()+ return {"body": {"errorMessage": f"Image Lambda failed: {stdout[:500]}", "errorType": "Runtime.ExitError"}, "error": True, "log": stdout}+ finally:+ try:+ container.stop(timeout=2)+ except Exception:+ pass+ try:+ container.remove(force=True)+ except Exception:+ pass++def _execute_function_provided(func: dict, event: dict) -> dict:"""Execute a provided-runtime Lambda (Go/Rust binary) via a minimal Lambda Runtime API."""config = func.get("config") or func@@ -1281,7 +1516,6 @@ def _execute_function_provided(func: dict, event: dict) -> dict:try:import http.serverimport socketserver-with tempfile.TemporaryDirectory() as tmpdir:# Extract bootstrap binaryzip_path = os.path.join(tmpdir, "code.zip")@@ -1331,7 +1565,8 @@ def _execute_function_provided(func: dict, event: dict) -> dict:if "/runtime/invocation/next" in self.path:self.send_response(200)self.send_header("Lambda-Runtime-Aws-Request-Id", request_id)- self.send_header("Lambda-Runtime-Deadline-Ms", str(int((time.time() + timeout) * 1000)))+ self.send_header("Lambda-Runtime-Deadline-Ms",+ str(int((time.time() + timeout) * 1000)))self.send_header("Content-Type", "application/json")self.end_headers()self.wfile.write(event_json.encode())@@ -1430,11 +1665,7 @@ def _execute_function_provided(func: dict, event: dict) -> dict:try:_, stderr_out = proc.communicate(timeout=5)if stderr_out:- logger.info(- "Lambda %s stderr: %s",- config.get("FunctionName", "?"),- stderr_out.decode("utf-8", errors="replace")[:500],- )+ logger.info("Lambda %s stderr: %s", config.get("FunctionName", "?"), stderr_out.decode("utf-8", errors="replace")[:500])except Exception:passif result_holder["error"]:@@ -1447,13 +1678,7 @@ def _execute_function_provided(func: dict, event: dict) -> dict:proc.kill()stdout, stderr = proc.communicate(timeout=5)logs = (stdout.decode("utf-8", errors="replace") + stderr.decode("utf-8", errors="replace")).strip()- return {- "body": {- "errorMessage": f"Lambda timed out after {timeout}s: {logs[:500]}",- "errorType": "Runtime.ExitError",- },- "error": True,- }+ return {"body": {"errorMessage": f"Lambda timed out after {timeout}s: {logs[:500]}", "errorType": "Runtime.ExitError"}, "error": True}finally:server.shutdown()@@ -1522,13 +1747,7 @@ def _execute_function_local(func: dict, event: dict) -> dict:os.symlink(src, dst)if "." not in handler:- return {- "body": {- "errorMessage": f"Invalid handler format: {handler}",- "errorType": "Runtime.InvalidEntrypoint",- },- "error": True,- }+ return {"body": {"errorMessage": f"Invalid handler format: {handler}", "errorType": "Runtime.InvalidEntrypoint"}, "error": True}module_name, func_name = handler.rsplit(".", 1)if is_node:@@ -2598,7 +2817,8 @@ def _poll_loop():def _poll_sqs():from ministack.services import sqs as _sqs- for esm in list(_esms.values()):+ for (acct_id, _esm_key), esm in list(_esms._data.items()):+ _request_account_id.set(acct_id)if not esm.get("Enabled", True):continuesource_arn = esm.get("EventSourceArn", "")@@ -2625,24 +2845,22 @@ def _poll_sqs():records = []for msg in batch:first_recv = msg.get("first_receive_at") or now- records.append(- {- "messageId": msg["id"],- "receiptHandle": msg["receipt_handle"],- "body": msg["body"],- "attributes": {- "ApproximateReceiveCount": str(msg.get("receive_count", 1)),- "SentTimestamp": str(int(msg["sent_at"] * 1000)),- "SenderId": get_account_id(),- "ApproximateFirstReceiveTimestamp": str(int(first_recv * 1000)),- },- "messageAttributes": msg.get("message_attributes", {}),- "md5OfBody": msg.get("md5_body") or msg.get("md5") or "",- "eventSource": "aws:sqs",- "eventSourceARN": source_arn,- "awsRegion": REGION,- }- )+ records.append({+ "messageId": msg["id"],+ "receiptHandle": msg["receipt_handle"],+ "body": msg["body"],+ "attributes": {+ "ApproximateReceiveCount": str(msg.get("receive_count", 1)),+ "SentTimestamp": str(int(msg["sent_at"] * 1000)),+ "SenderId": get_account_id(),+ "ApproximateFirstReceiveTimestamp": str(int(first_recv * 1000)),+ },+ "messageAttributes": msg.get("message_attributes", {}),+ "md5OfBody": msg.get("md5_body") or msg.get("md5") or "",+ "eventSource": "aws:sqs",+ "eventSourceARN": source_arn,+ "awsRegion": REGION,+ })event = {"Records": records}result = _execute_function(_functions[func_name], event)@@ -2654,11 +2872,7 @@ def _poll_sqs():esm["LastProcessingResult"] = "FAILED"logger.warning("ESM: Lambda %s failed processing SQS batch from %s (errorType=%s errorMessage=%s)\n%s",- func_name,- queue_name,- err_type,- err_msg,- result.get("log", ""),+ func_name, queue_name, err_type, err_msg, result.get("log", ""),)else:# Check for ReportBatchItemFailures — partial batch response@@ -2689,13 +2903,8 @@ def _poll_sqs():n_failed = len(batch) - len(succeeded)if n_failed:esm["LastProcessingResult"] = f"OK - {len(succeeded)} records, {n_failed} partial failures"- logger.info(- "ESM: Lambda %s processed %d SQS messages from %s (%d partial failures)",- func_name,- len(succeeded),- queue_name,- n_failed,- )+ logger.info("ESM: Lambda %s processed %d SQS messages from %s (%d partial failures)",+ func_name, len(succeeded), queue_name, n_failed)else:esm["LastProcessingResult"] = f"OK - {len(batch)} records"logger.info("ESM: Lambda %s processed %d SQS messages from %s", func_name, len(batch), queue_name)@@ -2707,7 +2916,8 @@ def _poll_sqs():def _poll_kinesis():from ministack.services import kinesis as _kin- for esm in list(_esms.values()):+ for (acct_id, _esm_key), esm in list(_esms._data.items()):+ _request_account_id.set(acct_id)if not esm.get("Enabled", True):continuesource_arn = esm.get("EventSourceArn", "")@@ -2742,7 +2952,7 @@ def _poll_kinesis():continuepos = positions[shard_id]- raw_records = shard["records"][pos : pos + batch_size]+ raw_records = shard["records"][pos:pos + batch_size]if not raw_records:continue@@ -2760,24 +2970,22 @@ def _poll_kinesis():else:data_b64 = base64.b64encode(str(data_val).encode("utf-8")).decode("ascii")- records.append(- {- "kinesis": {- "kinesisSchemaVersion": "1.0",- "partitionKey": r["PartitionKey"],- "sequenceNumber": r["SequenceNumber"],- "data": data_b64,- "approximateArrivalTimestamp": r["ApproximateArrivalTimestamp"],- },- "eventSource": "aws:kinesis",- "eventVersion": "1.0",- "eventID": f"{shard_id}:{r['SequenceNumber']}",- "eventName": "aws:kinesis:record",- "invokeIdentityArn": f"arn:aws:iam::{get_account_id()}:role/lambda-role",- "awsRegion": REGION,- "eventSourceARN": source_arn,- }- )+ records.append({+ "kinesis": {+ "kinesisSchemaVersion": "1.0",+ "partitionKey": r["PartitionKey"],+ "sequenceNumber": r["SequenceNumber"],+ "data": data_b64,+ "approximateArrivalTimestamp": r["ApproximateArrivalTimestamp"],+ },+ "eventSource": "aws:kinesis",+ "eventVersion": "1.0",+ "eventID": f"{shard_id}:{r['SequenceNumber']}",+ "eventName": "aws:kinesis:record",+ "invokeIdentityArn": f"arn:aws:iam::{get_account_id()}:role/lambda-role",+ "awsRegion": REGION,+ "eventSourceARN": source_arn,+ })event = {"Records": records}result = _execute_function(_functions[func_name], event)@@ -2789,12 +2997,7 @@ def _poll_kinesis():esm["LastProcessingResult"] = "FAILED"logger.warning("ESM: Lambda %s failed processing Kinesis batch from %s/%s (errorType=%s errorMessage=%s)\n%s",- func_name,- stream_name,- shard_id,- err_type,- err_msg,- result.get("log", ""),+ func_name, stream_name, shard_id, err_type, err_msg, result.get("log", ""),)else:positions[shard_id] = pos + len(raw_records)@@ -2804,10 +3007,7 @@ def _poll_kinesis():logger.info("ESM: Lambda %s output:\n%s", func_name, log_output)logger.info("ESM: Lambda %s processed %d Kinesis records from %s/%s",- func_name,- len(raw_records),- stream_name,- shard_id,+ func_name, len(raw_records), stream_name, shard_id,)@@ -2818,7 +3018,8 @@ def _poll_dynamodb_streams():if not stream_records:return- for esm in list(_esms.values()):+ for (acct_id, _esm_key), esm in list(_esms._data.items()):+ _request_account_id.set(acct_id)if not esm.get("Enabled", True):continuesource_arn = esm.get("EventSourceArn", "")@@ -2846,7 +3047,7 @@ def _poll_dynamodb_streams():pos = _dynamodb_stream_positions[esm_id]batch_size = esm.get("BatchSize", 100)- batch = table_records[pos : pos + batch_size]+ batch = table_records[pos:pos + batch_size]if not batch:continue@@ -2860,11 +3061,7 @@ def _poll_dynamodb_streams():esm["LastProcessingResult"] = "FAILED"logger.warning("ESM: Lambda %s failed processing DynamoDB stream batch from %s (errorType=%s errorMessage=%s)\n%s",- func_name,- table_name,- err_type,- err_msg,- result.get("log", ""),+ func_name, table_name, err_type, err_msg, result.get("log", ""),)else:with _dynamodb_stream_positions_lock:@@ -2875,9 +3072,7 @@ def _poll_dynamodb_streams():logger.info("ESM: Lambda %s output:\n%s", func_name, log_output)logger.info("ESM: Lambda %s processed %d DynamoDB stream records from %s",- func_name,- len(batch),- table_name,+ func_name, len(batch), table_name,)@@ -2955,5 +3150,5 @@ def reset():_function_urls.clear()_kinesis_positions.clear()_dynamodb_stream_positions.clear()- _containers.clear()+ _cleanup_warm_containers()lambda_runtime.reset()diff --git a/ministack/services/rds.py b/ministack/services/rds.pyindex 3e6c2a0e..8f1fa352 100644--- a/ministack/services/rds.py+++ b/ministack/services/rds.py@@ -29,6 +29,7 @@ import datetimeimport loggingimport osimport socket+import threadingimport timefrom urllib.parse import parse_qsfrom xml.sax.saxutils import escape as _esc@@ -62,8 +63,12 @@ _ministack_network = None# ── Persistence ────────────────────────────────────────────def get_state():+ instances = copy.deepcopy(_instances)+ # Strip Docker container IDs (not restorable across restarts)+ for key in list(instances._data):+ instances._data[key].pop("_docker_container_id", None)state = {- "instances": {},+ "instances": instances,"clusters": copy.deepcopy(_clusters),"subnet_groups": copy.deepcopy(_subnet_groups),"param_groups": copy.deepcopy(_param_groups),@@ -75,10 +80,6 @@ def get_state():"tags": copy.deepcopy(_tags),"port_counter": _port_counter[0],}- for name, inst in _instances.items():- i = copy.deepcopy(inst)- i.pop("_docker_container_id", None)- state["instances"][name] = ireturn state@@ -96,10 +97,19 @@ def restore_state(data):_tags.update(data.get("tags", {}))if "port_counter" in data:_port_counter[0] = data["port_counter"]- for name, inst in data.get("instances", {}).items():- inst["_docker_container_id"] = None- inst["DBInstanceStatus"] = "available"- _instances[name] = inst+ instances_data = data.get("instances", {})+ if isinstance(instances_data, AccountScopedDict):+ # New format: AccountScopedDict with full multi-account data+ for key, inst in list(instances_data._data.items()):+ inst["_docker_container_id"] = None+ inst["DBInstanceStatus"] = "available"+ _instances._data[key] = inst+ else:+ # Legacy format: plain dict keyed by instance name+ for name, inst in instances_data.items():+ inst["_docker_container_id"] = None+ inst["DBInstanceStatus"] = "available"+ _instances[name] = inst_restored = load_state("rds")@@ -152,10 +162,14 @@ def _wait_for_port(host, port, timeout=60):return False+_port_lock = threading.Lock()++def _next_port():- port = _port_counter[0]- _port_counter[0] += 1- return port+ with _port_lock:+ port = _port_counter[0]+ _port_counter[0] += 1+ return port# ---------------------------------------------------------------------------@@ -285,24 +299,30 @@ def _create_db_instance(p):if container_ip:internal_host = container_ipinternal_port = container_port- if _wait_for_port(container_ip, container_port):- logger.info(- "RDS: %s container for %s ready at "- "%s:%s (network %s)", engine, db_id,- container_ip, container_port, ms_network)- else:- logger.warning(- "RDS: %s container for %s at %s:%s "- "not ready after timeout", engine,- db_id, container_ip, container_port)+ def _bg_wait(cip=container_ip, cport=container_port,+ eng=engine, did=db_id, net=ms_network):+ if _wait_for_port(cip, cport):+ logger.info(+ "RDS: %s container for %s ready at "+ "%s:%s (network %s)", eng, did,+ cip, cport, net)+ else:+ logger.warning(+ "RDS: %s container for %s at %s:%s "+ "not ready after timeout", eng,+ did, cip, cport)+ threading.Thread(target=_bg_wait, daemon=True).start()else:logger.info("RDS: started %s container for %s on port %s",engine, db_id, host_port)else:- logger.info(- "RDS: started %s container for %s on port %s",- engine, db_id, host_port)+ def _bg_wait_port(hp=host_port, eng=engine, did=db_id):+ if _wait_for_port("127.0.0.1", hp):+ logger.info("RDS: %s container for %s ready on port %s", eng, did, hp)+ else:+ logger.warning("RDS: %s container for %s on port %s not ready after timeout", eng, did, hp)+ threading.Thread(target=_bg_wait_port, daemon=True).start()except Exception as e:logger.warning("RDS: Docker failed for %s: %s", db_id, e)diff --git a/ministack/services/route53.py b/ministack/services/route53.pyindex c1af05a1..0540a391 100644--- a/ministack/services/route53.py+++ b/ministack/services/route53.py@@ -85,15 +85,14 @@ if _restored:def reset():- global _zones, _records, _changes, _health_checks, _tags, _caller_refs, _hc_caller_refswith _lock:- _zones = {}- _records = {}- _changes = {}- _health_checks = {}- _tags = {}- _caller_refs = {}- _hc_caller_refs = {}+ _zones.clear()+ _records.clear()+ _changes.clear()+ _health_checks.clear()+ _tags.clear()+ _caller_refs.clear()+ _hc_caller_refs.clear()# ─── ID generators ────────────────────────────────────────────────────────────diff --git a/ministack/services/s3.py b/ministack/services/s3.pyindex 60ba3b27..1256ee76 100644--- a/ministack/services/s3.py+++ b/ministack/services/s3.py@@ -2844,7 +2844,8 @@ def _complete_multipart_upload()root = Element("CompleteMultipartUploadResult", xmlns=S3_NS)- SubElement(root, "Location").text = f"http://s3.amazonaws.com/{bucket_name}/{key}"+ s3_host = os.environ.get("MINISTACK_HOST", os.environ.get("AWS_ENDPOINT_URL", "http://localhost:4566"))+ SubElement(root, "Location").text = f"{s3_host}/{bucket_name}/{key}"SubElement(root, "Bucket").text = bucket_nameSubElement(root, "Key").text = keySubElement(root, "ETag").text = final_etagdiff --git a/ministack/services/sns.py b/ministack/services/sns.pyindex 257724f4..5f770812 100644--- a/ministack/services/sns.py+++ b/ministack/services/sns.py@@ -396,7 +396,7 @@ def _list_subscriptions(params):f"<Owner>{sub.get('owner', get_account_id())}</Owner>"f"<TopicArn>{sub['topic_arn']}</TopicArn>"f"<Protocol>{sub['protocol']}</Protocol>"- f"<Endpoint>{sub['endpoint']}</Endpoint>"+ f"<Endpoint>{_xml_escape(sub['endpoint'])}</Endpoint>""</member>")next_token_xml = ""@@ -419,7 +419,7 @@ def _list_subscriptions_by_topic(params):f"<Owner>{sub.get('owner', get_account_id())}</Owner>"f"<TopicArn>{topic_arn}</TopicArn>"f"<Protocol>{sub['protocol']}</Protocol>"- f"<Endpoint>{sub['endpoint']}</Endpoint>"+ f"<Endpoint>{_xml_escape(sub['endpoint'])}</Endpoint>""</member>")return _xml(200, "ListSubscriptionsByTopicResponse",diff --git a/ministack/services/ssm.py b/ministack/services/ssm.pyindex 8cf38667..407c8449 100644--- a/ministack/services/ssm.py+++ b/ministack/services/ssm.py@@ -213,9 +213,7 @@ def _get_parameters_by_path(data):continueif not name.startswith(path_prefix) and not (name.startswith(path) and path == "/"):continue- if path == "/":- matches = True- elif recursive:+ if recursive:matches = Trueelse:suffix = name[len(path_prefix):]diff --git a/ministack/services/sts.py b/ministack/services/sts.pyindex 7d3d9872..d4ab1284 100644--- a/ministack/services/sts.py+++ b/ministack/services/sts.py@@ -96,13 +96,14 @@ async def handle_request(method, path, headers, body, query_params):if not assumed_arn.endswith(f"/{session}"):assumed_arn = f"{assumed_arn}/{session}"role_id = "AROA" + new_uuid().replace("-", "")[:17].upper()+ provider = _p(params, "ProviderId") or "sts.amazonaws.com"if use_json:return json_response({"Credentials": {"AccessKeyId": access_key, "SecretAccessKey": secret_key, "SessionToken": session_token, "Expiration": time.time() + duration},"AssumedRoleUser": {"AssumedRoleId": f"{role_id}:{session}", "Arn": assumed_arn},"SubjectFromWebIdentityToken": "test-subject","Audience": "sts.amazonaws.com",- "Provider": "accounts.google.com",+ "Provider": provider,})return _xml(200, "AssumeRoleWithWebIdentityResponse",f"<AssumeRoleWithWebIdentityResult>"@@ -118,7 +119,7 @@ async def handle_request(method, path, headers, body, query_params):f"</AssumedRoleUser>"f"<SubjectFromWebIdentityToken>test-subject</SubjectFromWebIdentityToken>"f"<Audience>sts.amazonaws.com</Audience>"- f"<Provider>accounts.google.com</Provider>"+ f"<Provider>{provider}</Provider>"f"</AssumeRoleWithWebIdentityResult>",ns="sts")diff --git a/ministack/services/transfer.py b/ministack/services/transfer.pyindex 2495eb84..6aacee50 100644--- a/ministack/services/transfer.py+++ b/ministack/services/transfer.py@@ -201,7 +201,7 @@ def _list_servers(data):if next_token:for i, s in enumerate(all_servers):if s["ServerId"] == next_token:- start = i+ start = i + 1breakpage = all_servers[start:start + max_results]@@ -332,7 +332,7 @@ def _list_users(data):if next_token:for i, u in enumerate(server_users):if u["UserName"] == next_token:- start = i+ start = i + 1breakpage = server_users[start:start + max_results]diff --git a/pyproject.toml b/pyproject.tomlindex b2b1ca89..19a29325 100644--- a/pyproject.toml+++ b/pyproject.toml@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"[project]name = "ministack"-version = "1.2.19"+version = "1.2.20"description = "Free, open-source local AWS emulator — drop-in LocalStack replacement"readme = "README.md"license = { file = "LICENSE" }diff --git a/tests/conftest.py b/tests/conftest.pyindex db7faaa6..89d6c14f 100644--- a/tests/conftest.py+++ b/tests/conftest.py@@ -292,3 +292,7 @@ def autoscaling():@pytest.fixture(scope="session")def transfer():return make_client("transfer")+[email protected](scope="session")+def eks():+ return make_client("eks")diff --git a/tests/test_dynamodb.py b/tests/test_dynamodb.pyindex 4263876f..b8987685 100644--- a/tests/test_dynamodb.py+++ b/tests/test_dynamodb.py@@ -1473,3 +1473,20 @@ def test_partiql_select_where_number(ddb):assert len(resp["Items"]) == 1assert resp["Items"][0]["pk"]["S"] == "n2"++def test_dynamodb_stream_arn_stable(ddb):+ """LatestStreamArn should be stable across DescribeTable calls."""+ tname = f"stream-stable-{_uuid_mod.uuid4().hex[:8]}"+ ddb.create_table(+ TableName=tname,+ KeySchema=[{"AttributeName": "pk", "KeyType": "HASH"}],+ AttributeDefinitions=[{"AttributeName": "pk", "AttributeType": "S"}],+ BillingMode="PAY_PER_REQUEST",+ StreamSpecification={"StreamEnabled": True, "StreamViewType": "NEW_AND_OLD_IMAGES"},+ )+ desc1 = ddb.describe_table(TableName=tname)["Table"]+ desc2 = ddb.describe_table(TableName=tname)["Table"]+ assert desc1["LatestStreamArn"] == desc2["LatestStreamArn"]+ assert desc1["LatestStreamLabel"] == desc2["LatestStreamLabel"]+ ddb.delete_table(TableName=tname)+diff --git a/tests/test_ec2.py b/tests/test_ec2.pyindex 181a6016..ed56a632 100644--- a/tests/test_ec2.py+++ b/tests/test_ec2.py@@ -1358,3 +1358,31 @@ def test_ec2_authorize_sg_ingress_ipv6(ec2):rules = resp.get("SecurityGroupRules", [])assert len(rules) >= 1assert rules[0]["CidrIpv6"] == "::/0"+++def test_ec2_terminate_unknown_instance(ec2):+ """TerminateInstances with a non-existent ID should return InvalidInstanceID.NotFound."""+ with pytest.raises(ClientError) as exc:+ ec2.terminate_instances(InstanceIds=["i-nonexistent0000000"])+ assert exc.value.response["Error"]["Code"] == "InvalidInstanceID.NotFound"+++def test_ec2_stop_unknown_instance(ec2):+ """StopInstances with a non-existent ID should return InvalidInstanceID.NotFound."""+ with pytest.raises(ClientError) as exc:+ ec2.stop_instances(InstanceIds=["i-nonexistent0000000"])+ assert exc.value.response["Error"]["Code"] == "InvalidInstanceID.NotFound"+++def test_ec2_vpc_cidr_block_association_set(ec2):+ """CreateVpc and DescribeVpcs should include cidrBlockAssociationSet."""+ vpc = ec2.create_vpc(CidrBlock="10.99.0.0/16")["Vpc"]+ assocs = vpc.get("CidrBlockAssociationSet", [])+ assert len(assocs) >= 1+ assert assocs[0]["CidrBlock"] == "10.99.0.0/16"+ assert assocs[0]["CidrBlockState"]["State"] == "associated"++ # DescribeVpcs should also include it+ desc = ec2.describe_vpcs(VpcIds=[vpc["VpcId"]])["Vpcs"][0]+ assert len(desc.get("CidrBlockAssociationSet", [])) >= 1+ ec2.delete_vpc(VpcId=vpc["VpcId"])diff --git a/tests/test_eks.py b/tests/test_eks.pynew file mode 100644index 00000000..2ea624bc--- /dev/null+++ b/tests/test_eks.py@@ -0,0 +1,234 @@+"""+Integration tests for EKS service emulator.+Tests cluster CRUD, nodegroup CRUD, tags, and CloudFormation provisioning.+k3s Docker container tests require Docker socket access.+"""+import json+import time+import uuid+import pytest+import boto3+from botocore.exceptions import ClientError++ENDPOINT = "http://localhost:4566"+REGION = "us-east-1"++[email protected](scope="module")+def eks():+ return boto3.client("eks", endpoint_url=ENDPOINT,+ aws_access_key_id="test", aws_secret_access_key="test",+ region_name=REGION)++[email protected](scope="module")+def cfn():+ return boto3.client("cloudformation", endpoint_url=ENDPOINT,+ aws_access_key_id="test", aws_secret_access_key="test",+ region_name=REGION)+++def _uid():+ return uuid.uuid4().hex[:8]+++# ---------------------------------------------------------------------------+# Cluster CRUD+# ---------------------------------------------------------------------------++def test_eks_create_describe_delete_cluster(eks):+ name = f"test-cluster-{_uid()}"+ resp = eks.create_cluster(+ name=name,+ version="1.30",+ roleArn="arn:aws:iam::000000000000:role/eks-role",+ resourcesVpcConfig={"subnetIds": ["subnet-1", "subnet-2"]},+ )+ cluster = resp["cluster"]+ assert cluster["name"] == name+ assert cluster["status"] == "ACTIVE"+ assert cluster["version"] == "1.30"+ assert "arn" in cluster+ assert f"cluster/{name}" in cluster["arn"]+ assert "endpoint" in cluster+ assert "certificateAuthority" in cluster+ assert "identity" in cluster+ assert "oidc" in cluster["identity"]++ # Describe+ resp = eks.describe_cluster(name=name)+ assert resp["cluster"]["name"] == name+ assert resp["cluster"]["status"] == "ACTIVE"+ assert resp["cluster"]["endpoint"] == cluster["endpoint"]++ # Delete+ resp = eks.delete_cluster(name=name)+ assert resp["cluster"]["name"] == name++ # Verify gone+ with pytest.raises(ClientError) as exc:+ eks.describe_cluster(name=name)+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"+++def test_eks_create_duplicate_cluster(eks):+ name = f"dup-cluster-{_uid()}"+ eks.create_cluster(name=name, roleArn="arn:aws:iam::000000000000:role/r",+ resourcesVpcConfig={})+ with pytest.raises(ClientError) as exc:+ eks.create_cluster(name=name, roleArn="arn:aws:iam::000000000000:role/r",+ resourcesVpcConfig={})+ assert exc.value.response["Error"]["Code"] == "ResourceInUseException"+ eks.delete_cluster(name=name)+++def test_eks_list_clusters(eks):+ name = f"list-cluster-{_uid()}"+ eks.create_cluster(name=name, roleArn="arn:aws:iam::000000000000:role/r",+ resourcesVpcConfig={})+ resp = eks.list_clusters()+ assert name in resp["clusters"]+ eks.delete_cluster(name=name)+++def test_eks_delete_nonexistent_cluster(eks):+ with pytest.raises(ClientError) as exc:+ eks.delete_cluster(name="nonexistent-cluster-xyz")+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"+++# ---------------------------------------------------------------------------+# Nodegroup CRUD+# ---------------------------------------------------------------------------++def test_eks_create_describe_delete_nodegroup(eks):+ cluster = f"ng-cluster-{_uid()}"+ eks.create_cluster(name=cluster, roleArn="arn:aws:iam::000000000000:role/r",+ resourcesVpcConfig={})+ ng_name = f"ng-{_uid()}"+ resp = eks.create_nodegroup(+ clusterName=cluster,+ nodegroupName=ng_name,+ scalingConfig={"minSize": 1, "maxSize": 3, "desiredSize": 2},+ instanceTypes=["t3.large"],+ nodeRole="arn:aws:iam::000000000000:role/node-role",+ subnets=["subnet-1"],+ diskSize=50,+ )+ ng = resp["nodegroup"]+ assert ng["nodegroupName"] == ng_name+ assert ng["clusterName"] == cluster+ assert ng["status"] == "ACTIVE"+ assert ng["scalingConfig"]["desiredSize"] == 2+ assert ng["instanceTypes"] == ["t3.large"]+ assert ng["diskSize"] == 50+ assert "nodegroupArn" in ng++ # Describe+ resp = eks.describe_nodegroup(clusterName=cluster, nodegroupName=ng_name)+ assert resp["nodegroup"]["nodegroupName"] == ng_name++ # List+ resp = eks.list_nodegroups(clusterName=cluster)+ assert ng_name in resp["nodegroups"]++ # Delete+ resp = eks.delete_nodegroup(clusterName=cluster, nodegroupName=ng_name)+ assert resp["nodegroup"]["status"] == "DELETING"++ # Verify gone+ with pytest.raises(ClientError) as exc:+ eks.describe_nodegroup(clusterName=cluster, nodegroupName=ng_name)+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"++ eks.delete_cluster(name=cluster)+++def test_eks_nodegroup_nonexistent_cluster(eks):+ with pytest.raises(ClientError) as exc:+ eks.create_nodegroup(clusterName="no-such-cluster", nodegroupName="ng1",+ nodeRole="arn:aws:iam::000000000000:role/r",+ subnets=["subnet-1"])+ assert exc.value.response["Error"]["Code"] == "ResourceNotFoundException"+++def test_eks_delete_cluster_cascades_nodegroups(eks):+ cluster = f"cascade-{_uid()}"+ eks.create_cluster(name=cluster, roleArn="arn:aws:iam::000000000000:role/r",+ resourcesVpcConfig={})+ for i in range(3):+ eks.create_nodegroup(clusterName=cluster, nodegroupName=f"ng-{i}",+ nodeRole="arn:aws:iam::000000000000:role/r",+ subnets=["subnet-1"])+ resp = eks.list_nodegroups(clusterName=cluster)+ assert len(resp["nodegroups"]) == 3++ eks.delete_cluster(name=cluster)++ with pytest.raises(ClientError):+ eks.list_nodegroups(clusterName=cluster)+++# ---------------------------------------------------------------------------+# Tags+# ---------------------------------------------------------------------------++def test_eks_tag_cluster(eks):+ name = f"tag-cluster-{_uid()}"+ eks.create_cluster(name=name, roleArn="arn:aws:iam::000000000000:role/r",+ resourcesVpcConfig={}, tags={"env": "test"})+ arn = eks.describe_cluster(name=name)["cluster"]["arn"]++ resp = eks.list_tags_for_resource(resourceArn=arn)+ assert resp["tags"]["env"] == "test"++ eks.tag_resource(resourceArn=arn, tags={"team": "platform"})+ resp = eks.list_tags_for_resource(resourceArn=arn)+ assert resp["tags"]["team"] == "platform"+ assert resp["tags"]["env"] == "test"++ eks.untag_resource(resourceArn=arn, tagKeys=["env"])+ resp = eks.list_tags_for_resource(resourceArn=arn)+ assert "env" not in resp["tags"]+ assert resp["tags"]["team"] == "platform"++ eks.delete_cluster(name=name)+++# ---------------------------------------------------------------------------+# CloudFormation+# ---------------------------------------------------------------------------++def test_eks_cfn_cluster(cfn, eks):+ uid = _uid()+ cluster_name = f"cfn-eks-{uid}"+ template = json.dumps({+ "AWSTemplateFormatVersion": "2010-09-09",+ "Resources": {+ "Cluster": {+ "Type": "AWS::EKS::Cluster",+ "Properties": {+ "Name": cluster_name,+ "Version": "1.30",+ "RoleArn": "arn:aws:iam::000000000000:role/eks-role",+ "ResourcesVpcConfig": {+ "subnetIds": ["subnet-1", "subnet-2"],+ },+ },+ },+ },+ })+ stack_name = f"eks-stack-{uid}"+ cfn.create_stack(StackName=stack_name, TemplateBody=template)+ time.sleep(3)++ stack = cfn.describe_stacks(StackName=stack_name)["Stacks"][0]+ assert stack["StackStatus"] == "CREATE_COMPLETE"++ # Verify via EKS API+ resp = eks.describe_cluster(name=cluster_name)+ assert resp["cluster"]["name"] == cluster_name+ assert resp["cluster"]["status"] == "ACTIVE"++ cfn.delete_stack(StackName=stack_name)+ time.sleep(2)diff --git a/tests/test_kms.py b/tests/test_kms.pyindex 29adee1b..663b22e0 100644--- a/tests/test_kms.py+++ b/tests/test_kms.py@@ -89,7 +89,7 @@ def test_kms_sign_and_verify_pkcs1(kms_client):MessageType="RAW",SigningAlgorithm="RSASSA_PKCS1_V1_5_SHA_256",)- assert sign_resp["KeyId"] == key_id+ assert key_id in sign_resp["KeyId"] # KeyId in response is the full ARNassert sign_resp["SigningAlgorithm"] == "RSASSA_PKCS1_V1_5_SHA_256"assert len(sign_resp["Signature"]) > 0@@ -132,14 +132,16 @@ def test_kms_verify_wrong_message(kms_client):MessageType="RAW",SigningAlgorithm="RSASSA_PKCS1_V1_5_SHA_256",)- verify_resp = kms_client.verify(- KeyId=key_id,- Message=b"tampered",- MessageType="RAW",- Signature=sign_resp["Signature"],- SigningAlgorithm="RSASSA_PKCS1_V1_5_SHA_256",- )- assert verify_resp["SignatureValid"] is False+ # Real AWS raises KMSInvalidSignatureException on invalid signature+ import pytest+ with pytest.raises(kms_client.exceptions.KMSInvalidSignatureException):+ kms_client.verify(+ KeyId=key_id,+ Message=b"tampered",+ MessageType="RAW",+ Signature=sign_resp["Signature"],+ SigningAlgorithm="RSASSA_PKCS1_V1_5_SHA_256",+ )def test_kms_jwt_signing_flow(kms_client):"""Sign a JWT-style header.payload string and verify the signature."""@@ -180,7 +182,7 @@ def test_kms_encrypt_decrypt_roundtrip(kms_client):plaintext = b"sensitive document content"enc_resp = kms_client.encrypt(KeyId=key_id, Plaintext=plaintext)- assert enc_resp["KeyId"] == key_id+ assert key_id in enc_resp["KeyId"]dec_resp = kms_client.decrypt(CiphertextBlob=enc_resp["CiphertextBlob"])assert dec_resp["Plaintext"] == plaintext@@ -205,7 +207,7 @@ def test_kms_generate_data_key_aes_256(kms_client):key_id = key["KeyMetadata"]["KeyId"]resp = kms_client.generate_data_key(KeyId=key_id, KeySpec="AES_256")- assert resp["KeyId"] == key_id+ assert key_id in resp["KeyId"]assert len(resp["Plaintext"]) == 32assert resp["CiphertextBlob"]@@ -238,7 +240,7 @@ def test_kms_generate_data_key_without_plaintext(kms_client):resp = kms_client.generate_data_key_without_plaintext(KeyId=key_id, KeySpec="AES_256")- assert resp["KeyId"] == key_id+ assert key_id in resp["KeyId"]assert resp["CiphertextBlob"]assert "Plaintext" not in resp@@ -247,7 +249,7 @@ def test_kms_get_public_key(kms_client):key_id = key["KeyMetadata"]["KeyId"]resp = kms_client.get_public_key(KeyId=key_id)- assert resp["KeyId"] == key_id+ assert key_id in resp["KeyId"]assert resp["KeySpec"] == "RSA_2048"assert resp["PublicKey"]@@ -442,7 +444,7 @@ def test_kms_ecc_sign_and_verify(kms_client):MessageType="RAW",SigningAlgorithm="ECDSA_SHA_256",)- assert sign_resp["KeyId"] == key_id+ assert key_id in sign_resp["KeyId"] # KeyId in response is the full ARNassert sign_resp["SigningAlgorithm"] == "ECDSA_SHA_256"assert len(sign_resp["Signature"]) > 0@@ -465,21 +467,22 @@ def test_kms_ecc_verify_wrong_message(kms_client):MessageType="RAW",SigningAlgorithm="ECDSA_SHA_256",)- verify_resp = kms_client.verify(- KeyId=key_id,- Message=b"tampered",- MessageType="RAW",- Signature=sign_resp["Signature"],- SigningAlgorithm="ECDSA_SHA_256",- )- assert verify_resp["SignatureValid"] is False+ import pytest+ with pytest.raises(kms_client.exceptions.KMSInvalidSignatureException):+ kms_client.verify(+ KeyId=key_id,+ Message=b"tampered",+ MessageType="RAW",+ Signature=sign_resp["Signature"],+ SigningAlgorithm="ECDSA_SHA_256",+ )def test_kms_ecc_get_public_key(kms_client):key = kms_client.create_key(KeySpec="ECC_SECG_P256K1", KeyUsage="SIGN_VERIFY")key_id = key["KeyMetadata"]["KeyId"]resp = kms_client.get_public_key(KeyId=key_id)- assert resp["KeyId"] == key_id+ assert key_id in resp["KeyId"]assert resp["KeySpec"] == "ECC_SECG_P256K1"assert resp["PublicKey"]assert "ECDSA_SHA_256" in resp["SigningAlgorithms"]@@ -568,16 +571,17 @@ def test_kms_ecc_sign_verify_digest_mode(kms_client):)assert verify_resp["SignatureValid"] is True- # Wrong digest should fail+ # Wrong digest should fail with KMSInvalidSignatureException+ import pytestwrong_digest = hashlib.sha256(b"different message").digest()- verify_resp = kms_client.verify(- KeyId=key_id,- Message=wrong_digest,- MessageType="DIGEST",- Signature=sign_resp["Signature"],- SigningAlgorithm="ECDSA_SHA_256",- )- assert verify_resp["SignatureValid"] is False+ with pytest.raises(kms_client.exceptions.KMSInvalidSignatureException):+ kms_client.verify(+ KeyId=key_id,+ Message=wrong_digest,+ MessageType="DIGEST",+ Signature=sign_resp["Signature"],+ SigningAlgorithm="ECDSA_SHA_256",+ )def test_kms_ecc_sign_via_alias(kms_client):"""Sign and verify using an alias instead of key ID."""@@ -609,3 +613,23 @@ def test_kms_key_rotation_with_period(kms_client):assert status["KeyRotationEnabled"] is Trueassert status["RotationPeriodInDays"] == 180kms_client.schedule_key_deletion(KeyId=key_id, PendingWindowInDays=7)+++def test_kms_pending_deletion_blocks_encrypt(kms_client):+ """Encrypt on a PendingDeletion key should raise KMSInvalidStateException."""+ import pytest+ key = kms_client.create_key()+ key_id = key["KeyMetadata"]["KeyId"]+ kms_client.schedule_key_deletion(KeyId=key_id, PendingWindowInDays=7)+ with pytest.raises(kms_client.exceptions.KMSInvalidStateException):+ kms_client.encrypt(KeyId=key_id, Plaintext=b"test")+++def test_kms_disabled_key_blocks_encrypt(kms_client):+ """Encrypt on a disabled key should raise DisabledException."""+ import pytest+ key = kms_client.create_key()+ key_id = key["KeyMetadata"]["KeyId"]+ kms_client.disable_key(KeyId=key_id)+ with pytest.raises(kms_client.exceptions.DisabledException):+ kms_client.encrypt(KeyId=key_id, Plaintext=b"test")diff --git a/tests/test_lambda.py b/tests/test_lambda.pyindex 2f7052e5..824057c9 100644--- a/tests/test_lambda.py+++ b/tests/test_lambda.py@@ -1110,6 +1110,31 @@ def test_lambda_layer_publish(lam):assert resp["Version"] == 1assert "my-test-layer" in resp["LayerVersionArn"]+def test_lambda_layer_publish_from_s3(lam, s3):+ """PublishLayerVersion with S3Bucket/S3Key. Contributed by @Baptiste-Garcin (#356)."""+ import zipfile, io++ buf = io.BytesIO()+ with zipfile.ZipFile(buf, "w") as z:+ z.writestr("s3layer.py", "# layer from s3")+ zip_bytes = buf.getvalue()++ bucket = "layer-bucket"+ key = "layers/my-layer.zip"+ s3.create_bucket(Bucket=bucket)+ s3.put_object(Bucket=bucket, Key=key, Body=zip_bytes)++ resp = lam.publish_layer_version(+ LayerName="s3-layer",+ Description="Layer from S3",+ Content={"S3Bucket": bucket, "S3Key": key},+ CompatibleRuntimes=["python3.12"],+ )+ assert resp["Version"] == 1+ assert "s3-layer" in resp["LayerVersionArn"]+ assert resp["Content"]["CodeSize"] == len(zip_bytes)+ assert resp["Content"]["CodeSha256"]+def test_lambda_layer_get_version(lam):resp = lam.get_layer_version(LayerName="my-test-layer", VersionNumber=1)assert resp["Version"] == 1diff --git a/tests/test_ssm.py b/tests/test_ssm.pyindex dd90b86f..64214b6a 100644--- a/tests/test_ssm.py+++ b/tests/test_ssm.py@@ -238,3 +238,14 @@ def test_ssm_secure_string_not_decrypted_by_default(ssm):assert resp["Parameter"]["Value"] != "mysecret"resp2 = ssm.get_parameter(Name="/qa/ssm/secure", WithDecryption=True)assert resp2["Parameter"]["Value"] == "mysecret"+++def test_ssm_get_parameters_by_path_root_non_recursive(ssm):+ """GetParametersByPath with Path=/ and Recursive=False should only return top-level params."""+ ssm.put_parameter(Name="/toplevel", Value="top", Type="String", Overwrite=True)+ ssm.put_parameter(Name="/nested/deep", Value="deep", Type="String", Overwrite=True)++ resp = ssm.get_parameters_by_path(Path="/", Recursive=False)+ names = [p["Name"] for p in resp["Parameters"]]+ assert "/toplevel" in names+ assert "/nested/deep" not in names