Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

feat(ec2): BYO-VM register image allows an image to run as EC2 (#1473)

Nahuel Nucerafa4b1f14cd9c
diff --git a/CHANGELOG.md b/CHANGELOG.md
index c44d3ed9..877a992d 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -7,6 +7,11 @@ Versioning follows [Semantic Versioning](https://semver.org/).
## [Unreleased]
+### Added
+- **EC2 — instances can have a real box behind them (`RegisterImage`)** — `RunInstances` returned a `running` record and booted nothing. `RegisterImage` now takes a container reference in `ImageLocation` and returns an `ami-` id; launching that id boots the image as a container whose IP becomes the instance's address, so `ssm:SendCommand` can report a real exit code. Such an AMI is `instance-store` backed: no EBS root volume, `StopInstances` / `StartInstances` answer `UnsupportedOperation`, and a container that dies reports `terminated`. Snapshot-backed registration is accepted as AWS accepts it and fails at `RunInstances` with `InvalidAMIID.Unavailable`, as does a reference that cannot be pulled. Registering is the only opt-in — with nothing registered EC2 never reaches for Docker. Reported by @iot-rocket.
+- **SSM — Run Command** — `SendCommand`, `GetCommandInvocation`, `ListCommands` and `DescribeInstanceInformation` returned `InvalidAction`, so the standard instance health probe could not run. Invocations are asynchronous as on AWS: `SendCommand` answers `Pending` and the caller polls to a terminal state. `AWS-RunShellScript` runs in the instance's container, so `Status` is `Success` or `Failed` on the real exit code, with the script's output. An instance with no agent answering is refused with `InvalidInstanceId`. Contributed by @bandle.
+- **EC2 — `RebootInstances` rejects unknown instance ids** — it returned `true` for any id. It now answers `InvalidInstanceID.NotFound`, matching AWS.
+
## [1.4.21] — 2026-08-20
### Added
diff --git a/README.md b/README.md
index 65495749..3707482f 100644
--- a/README.md
+++ b/README.md
@@ -571,7 +571,7 @@ MiniStack expands AWS SAM transforms (`Transform: AWS::Serverless-2016-10-31`) i
| **Athena** | StartQueryExecution, GetQueryExecution, GetQueryResults, StopQueryExecution, ListQueryExecutions, BatchGetQueryExecution, CreateWorkGroup, DeleteWorkGroup, GetWorkGroup, ListWorkGroups, UpdateWorkGroup, CreateNamedQuery, DeleteNamedQuery, GetNamedQuery, ListNamedQueries, BatchGetNamedQuery, CreateDataCatalog, GetDataCatalog, ListDataCatalogs, DeleteDataCatalog, UpdateDataCatalog, CreatePreparedStatement, GetPreparedStatement, DeletePreparedStatement, ListPreparedStatements, GetTableMetadata, ListTableMetadata, TagResource, UntagResource, ListTagsForResource | Real SQL via **DuckDB** when installed (`pip install duckdb`), otherwise returns mock results; result pagination; column type metadata |
| **Firehose** | CreateDeliveryStream, DeleteDeliveryStream, DescribeDeliveryStream, ListDeliveryStreams, PutRecord, PutRecordBatch, UpdateDestination, TagDeliveryStream, UntagDeliveryStream, ListTagsForDeliveryStream, StartDeliveryStreamEncryption, StopDeliveryStreamEncryption | S3 destinations write records to the local S3 emulator; all other destination types buffer in-memory; concurrency-safe `UpdateDestination` via `VersionId`; `DeliveryStreamType=KinesisStreamAsSource` consumes records from the source Kinesis stream on `PutRecord` / `PutRecords` and forwards them to the configured S3 destination, honoring `Prefix` and `DeliveryStartTimestamp`; `ProcessingConfiguration.Processors[].Type=Lambda` invoked per-batch with the AWS-shape event (`{invocationId, deliveryStreamArn, region, records:[{recordId, approximateArrivalTimestamp, data}]}`) — `Ok` records ship the transformed data downstream, `Dropped`/`ProcessingFailed` are omitted, Lambda failures pass records through unchanged (best-effort per AWS) |
| **Route53** | CreateHostedZone, GetHostedZone, DeleteHostedZone, ListHostedZones, ListHostedZonesByName, UpdateHostedZoneComment, ChangeResourceRecordSets (CREATE/UPSERT/DELETE), ListResourceRecordSets, GetChange, CreateHealthCheck, GetHealthCheck, DeleteHealthCheck, ListHealthChecks, UpdateHealthCheck, ChangeTagsForResource, ListTagsForResource | REST/XML protocol; SOA + NS records auto-created; CallerReference idempotency; alias records, weighted/failover/latency routing; marker-based pagination |
-| **EC2** | RunInstances, DescribeInstances, DescribeInstanceAttribute, DescribeInstanceTypes, DescribeVpcAttribute, TerminateInstances, StopInstances, StartInstances, RebootInstances, AssociateIamInstanceProfile, DescribeIamInstanceProfileAssociations, DisassociateIamInstanceProfile, ReplaceIamInstanceProfileAssociation, DescribeImages, CreateSecurityGroup, DeleteSecurityGroup, DescribeSecurityGroups, AuthorizeSecurityGroupIngress, RevokeSecurityGroupIngress, AuthorizeSecurityGroupEgress, RevokeSecurityGroupEgress, DescribeSecurityGroupRules, CreateKeyPair, DeleteKeyPair, DescribeKeyPairs, ImportKeyPair, CreatePlacementGroup, DeletePlacementGroup, DescribePlacementGroups, CreateVpc, DeleteVpc, DescribeVpcs, ModifyVpcAttribute, CreateSubnet, DeleteSubnet, DescribeSubnets, ModifySubnetAttribute, CreateInternetGateway, DeleteInternetGateway, DescribeInternetGateways, AttachInternetGateway, DetachInternetGateway, CreateRouteTable, DeleteRouteTable, DescribeRouteTables, AssociateRouteTable, DisassociateRouteTable, ReplaceRouteTableAssociation, CreateRoute, ReplaceRoute, DeleteRoute, CreateNetworkInterface, DeleteNetworkInterface, DescribeNetworkInterfaces, AttachNetworkInterface, DetachNetworkInterface, CreateVpcEndpoint, DeleteVpcEndpoints, DescribeVpcEndpoints, ModifyVpcEndpoint, DescribePrefixLists, DescribeAvailabilityZones, AllocateAddress, ReleaseAddress, AssociateAddress, DisassociateAddress, DescribeAddresses, DescribeAddressesAttribute, CreateTags, DeleteTags, DescribeTags, CreateNatGateway, DescribeNatGateways, DeleteNatGateway, CreateNetworkAcl, DescribeNetworkAcls, DeleteNetworkAcl, CreateNetworkAclEntry, DeleteNetworkAclEntry, ReplaceNetworkAclEntry, ReplaceNetworkAclAssociation, CreateFlowLogs, DescribeFlowLogs, DeleteFlowLogs, CreateVpcPeeringConnection, AcceptVpcPeeringConnection, DescribeVpcPeeringConnections, DeleteVpcPeeringConnection, CreateDhcpOptions, AssociateDhcpOptions, DescribeDhcpOptions, DeleteDhcpOptions, CreateEgressOnlyInternetGateway, DescribeEgressOnlyInternetGateways, DeleteEgressOnlyInternetGateway, CreateManagedPrefixList, DescribeManagedPrefixLists, GetManagedPrefixListEntries, ModifyManagedPrefixList, DeleteManagedPrefixList, CreateVpnGateway, DescribeVpnGateways, AttachVpnGateway, DetachVpnGateway, DeleteVpnGateway, EnableVgwRoutePropagation, DisableVgwRoutePropagation, CreateCustomerGateway, DescribeCustomerGateways, DeleteCustomerGateway, DescribeInstanceCreditSpecifications, DescribeInstanceMaintenanceOptions, DescribeInstanceAutoRecoveryAttribute, ModifyInstanceMaintenanceOptions, DescribeInstanceTopology, DescribeSpotInstanceRequests, DescribeCapacityReservations, DescribeInstanceStatus, DescribeVpcClassicLink, DescribeVpcClassicLinkDnsSupport, CreateLaunchTemplate, CreateLaunchTemplateVersion, DescribeLaunchTemplates, DescribeLaunchTemplateVersions, ModifyLaunchTemplate, DeleteLaunchTemplate, CreateFleet, DescribeFleets | 145 actions; EC2 Fleet (`CreateFleet` / `DescribeFleets`) with `instant`-type synchronous launch and `maintain` / `request` async fulfillment, multi-config × multi-override round-robin capacity distribution, and `DefaultTargetCapacityType`-driven spot/on-demand selection — unblocks Karpenter / Cluster Autoscaler local validation; IAM instance profile association lifecycle is persisted (`Associate*`, `Describe*`, `Replace*`, `Disassociate*`) so Terraform `aws_instance.iam_instance_profile` round-trips without drift; `AuthorizeSecurityGroupIngress` is idempotent on duplicate rules (same behavior as egress; avoids Terraform re-apply failures); in-memory state only — no real VMs; CreateVpc provisions per-VPC default route table, network ACL, and security group; full Terraform VPC module v6.6.0 compatible; VPN/Customer gateways, managed prefix lists, VPC endpoints with modify support; launch templates with versioning ($Latest/$Default) |
+| **EC2** | RunInstances, DescribeInstances, DescribeInstanceAttribute, DescribeInstanceTypes, DescribeVpcAttribute, TerminateInstances, StopInstances, StartInstances, RebootInstances, AssociateIamInstanceProfile, DescribeIamInstanceProfileAssociations, DisassociateIamInstanceProfile, ReplaceIamInstanceProfileAssociation, DescribeImages, RegisterImage, DeregisterImage, CreateSecurityGroup, DeleteSecurityGroup, DescribeSecurityGroups, AuthorizeSecurityGroupIngress, RevokeSecurityGroupIngress, AuthorizeSecurityGroupEgress, RevokeSecurityGroupEgress, DescribeSecurityGroupRules, CreateKeyPair, DeleteKeyPair, DescribeKeyPairs, ImportKeyPair, CreatePlacementGroup, DeletePlacementGroup, DescribePlacementGroups, CreateVpc, DeleteVpc, DescribeVpcs, ModifyVpcAttribute, CreateSubnet, DeleteSubnet, DescribeSubnets, ModifySubnetAttribute, CreateInternetGateway, DeleteInternetGateway, DescribeInternetGateways, AttachInternetGateway, DetachInternetGateway, CreateRouteTable, DeleteRouteTable, DescribeRouteTables, AssociateRouteTable, DisassociateRouteTable, ReplaceRouteTableAssociation, CreateRoute, ReplaceRoute, DeleteRoute, CreateNetworkInterface, DeleteNetworkInterface, DescribeNetworkInterfaces, AttachNetworkInterface, DetachNetworkInterface, CreateVpcEndpoint, DeleteVpcEndpoints, DescribeVpcEndpoints, ModifyVpcEndpoint, DescribePrefixLists, DescribeAvailabilityZones, AllocateAddress, ReleaseAddress, AssociateAddress, DisassociateAddress, DescribeAddresses, DescribeAddressesAttribute, CreateTags, DeleteTags, DescribeTags, CreateNatGateway, DescribeNatGateways, DeleteNatGateway, CreateNetworkAcl, DescribeNetworkAcls, DeleteNetworkAcl, CreateNetworkAclEntry, DeleteNetworkAclEntry, ReplaceNetworkAclEntry, ReplaceNetworkAclAssociation, CreateFlowLogs, DescribeFlowLogs, DeleteFlowLogs, CreateVpcPeeringConnection, AcceptVpcPeeringConnection, DescribeVpcPeeringConnections, DeleteVpcPeeringConnection, CreateDhcpOptions, AssociateDhcpOptions, DescribeDhcpOptions, DeleteDhcpOptions, CreateEgressOnlyInternetGateway, DescribeEgressOnlyInternetGateways, DeleteEgressOnlyInternetGateway, CreateManagedPrefixList, DescribeManagedPrefixLists, GetManagedPrefixListEntries, ModifyManagedPrefixList, DeleteManagedPrefixList, CreateVpnGateway, DescribeVpnGateways, AttachVpnGateway, DetachVpnGateway, DeleteVpnGateway, EnableVgwRoutePropagation, DisableVgwRoutePropagation, CreateCustomerGateway, DescribeCustomerGateways, DeleteCustomerGateway, DescribeInstanceCreditSpecifications, DescribeInstanceMaintenanceOptions, DescribeInstanceAutoRecoveryAttribute, ModifyInstanceMaintenanceOptions, DescribeInstanceTopology, DescribeSpotInstanceRequests, DescribeCapacityReservations, DescribeInstanceStatus, DescribeVpcClassicLink, DescribeVpcClassicLinkDnsSupport, CreateLaunchTemplate, CreateLaunchTemplateVersion, DescribeLaunchTemplates, DescribeLaunchTemplateVersions, ModifyLaunchTemplate, DeleteLaunchTemplate, CreateFleet, DescribeFleets | 147 actions; EC2 Fleet (`CreateFleet` / `DescribeFleets`) with `instant`-type synchronous launch and `maintain` / `request` async fulfillment, multi-config × multi-override round-robin capacity distribution, and `DefaultTargetCapacityType`-driven spot/on-demand selection — unblocks Karpenter / Cluster Autoscaler local validation; IAM instance profile association lifecycle is persisted (`Associate*`, `Describe*`, `Replace*`, `Disassociate*`) so Terraform `aws_instance.iam_instance_profile` round-trips without drift; `AuthorizeSecurityGroupIngress` is idempotent on duplicate rules (same behavior as egress; avoids Terraform re-apply failures); metadata-only instance records by default — `RegisterImage` with a container reference makes an AMI whose launches get a real box, so SSM Run Command returns a real exit code (see *Instances with a real box*); CreateVpc provisions per-VPC default route table, network ACL, and security group; full Terraform VPC module v6.6.0 compatible; VPN/Customer gateways, managed prefix lists, VPC endpoints with modify support; launch templates with versioning ($Latest/$Default) |
| **EBS** | CreateVolume, DeleteVolume, DescribeVolumes, DescribeVolumeStatus, AttachVolume, DetachVolume, ModifyVolume, DescribeVolumesModifications, EnableVolumeIO, ModifyVolumeAttribute, DescribeVolumeAttribute, CreateSnapshot, DeleteSnapshot, DescribeSnapshots, CopySnapshot, ModifySnapshotAttribute, DescribeSnapshotAttribute | Part of EC2 Query/XML service; attach/detach updates volume state; snapshots stored as completed immediately; Pro-only on LocalStack — free here |
| **EFS** | CreateFileSystem, DescribeFileSystems, DeleteFileSystem, UpdateFileSystem, CreateMountTarget, DescribeMountTargets, DeleteMountTarget, DescribeMountTargetSecurityGroups, ModifyMountTargetSecurityGroups, CreateAccessPoint, DescribeAccessPoints, DeleteAccessPoint, TagResource, UntagResource, ListTagsForResource, PutLifecycleConfiguration, DescribeLifecycleConfiguration, PutBackupPolicy, DescribeBackupPolicy, DescribeAccountPreferences, PutAccountPreferences | REST/JSON `/2015-02-01/*`; CreationToken idempotency; FileSystem deletion blocked when mount targets exist; Pro-only on LocalStack — free here |
| **EMR** | RunJobFlow, DescribeCluster, ListClusters, TerminateJobFlows, ModifyCluster, SetTerminationProtection, SetVisibleToAllUsers, AddJobFlowSteps, DescribeStep, ListSteps, CancelSteps, AddInstanceFleet, ListInstanceFleets, ModifyInstanceFleet, AddInstanceGroups, ListInstanceGroups, ModifyInstanceGroups, ListBootstrapActions, AddTags, RemoveTags, GetBlockPublicAccessConfiguration, PutBlockPublicAccessConfiguration | Control plane only — no real Spark/Hadoop; clusters start in WAITING (KeepAlive=true) or TERMINATED (KeepAlive=false); steps stored as COMPLETED immediately; all three instance modes (simple, InstanceGroups, InstanceFleets); TerminationProtected enforced; Pro-only on LocalStack — free here |
@@ -979,6 +979,65 @@ boto3.client("s3", endpoint_url="http://infra_ministack:4566", ...)
If `/var/run/docker.sock` is root-owned on the host, add `privileged: true` to the `ministack` service so it can talk to the daemon.
+### Instances with a Real Box
+
+`RunInstances` returns metadata by default — the right behaviour for CloudFormation and Terraform
+stacks full of AMI ids that never existed. Register an AMI backed by a container reference and
+launches against that id get a real box, so `ssm:SendCommand` can return a real exit code instead
+of always reporting success.
+
+```bash
+# 1. Register any image your Docker or Podman daemon can pull as an AMI
+aws --endpoint-url=http://localhost:4566 ec2 register-image \
+ --name my-box --image-location ghcr.io/you/my-box:1.0
+# { "ImageId": "ami-0a1b2c3d4e5f60718" }
+
+# 2. Launch it
+aws --endpoint-url=http://localhost:4566 ec2 run-instances \
+ --image-id ami-0a1b2c3d4e5f60718 --count 1 --instance-type t3.micro
+
+# 3. DescribeInstances reports the container's address, and the box answers on it
+aws --endpoint-url=http://localhost:4566 ec2 describe-instances
+```
+
+No configuration: registering an image is the opt-in. With nothing registered, EC2 never reaches
+for Docker at all, and an unregistered `ami-` id keeps launching the metadata-only record it always
+did. Both kinds of instance coexist in one account and region.
+
+- **Root device type:** a container has no persistent root disk, so an AMI registered from a
+ container reference is `instance-store` backed — no EBS root volume, and `StopInstances` /
+ `StartInstances` are refused with `UnsupportedOperation`, exactly as AWS refuses them for an
+ instance store-backed instance. `RebootInstances` restarts the container, `TerminateInstances`
+ removes it, and a container that dies out of band reports `terminated` on the next
+ `DescribeInstances` — a lost root disk is a lost instance. `/_ministack/reset` and shutdown sweep
+ this MiniStack's instance containers, leaving any other MiniStack sharing the daemon alone.
+- **Snapshot-backed registration** works too, because on AWS `RegisterImage` is metadata-only and
+ never checks that the image can boot: `import-snapshot` then `register-image` registers as normal
+ and reports `ebs`. There is simply nothing here to run, so the failure lands on `RunInstances`
+ with `InvalidAMIID.Unavailable` rather than at registration.
+- **Addresses:** the container's IP replaces the synthetic private and public addresses and both DNS
+ names, so the guest is reachable from the shared Docker network.
+- **Failure:** a container that cannot be created or started fails the `RunInstances` call and backs
+ the instance and its IAM profile association out. A running record with nothing behind it is
+ exactly what this exists to prevent. An image reference that cannot be pulled answers
+ `InvalidAMIID.Unavailable` rather than a server error, since retrying it would fail identically.
+- **Guest contract:** the image is launched with init as PID 1 and a `sleep infinity` keepalive
+ appended, so an image that would otherwise exit immediately stays up. An image with its own
+ `ENTRYPOINT` receives that as argv and is free to ignore it. An image with no shell is treated the
+ way AWS treats an instance with no SSM agent: unmanaged, and Run Command refuses it.
+- **Registry rewrite:** registered references route through `MINISTACK_IMAGE_PREFIX` like every
+ other pulled image, so mirrored and airgapped setups work unchanged.
+
+**Divergences from AWS, stated plainly.** `ImageLocation` carries a container reference rather than
+an S3 manifest path. A container is not a VM: there is no IMDS, no console output, no
+security-group enforcement or port publishing, no `CreateImage` from a running instance, and an
+attached EBS volume stays a record rather than appearing as a block device inside the guest.
+Container filesystems do not survive an emulator restart — restored instances report `terminated`,
+which is what AWS does when an instance store-backed instance loses its host. Terminated instances
+are also dropped from `DescribeInstances` after 60 seconds; AWS says only that a terminated instance
+"remains visible in the console for a short while", so a `wait instance-terminated` followed by a
+describe can find nothing where AWS would still answer.
+
### EKS with Real Kubernetes (k3s)
MiniStack's EKS spawns a real [k3s](https://k3s.io) cluster (75 MB image) when you create a cluster. `kubectl`, Helm, and any Kubernetes tooling work out of the box.
diff --git a/ministack/services/ec2.py b/ministack/services/ec2.py
index d6570f1c..b7f0c269 100644
--- a/ministack/services/ec2.py
+++ b/ministack/services/ec2.py
@@ -71,10 +71,13 @@ import time
from urllib.parse import parse_qs
from xml.sax.saxutils import escape as _esc
+from ministack.core import container_reaper
+from ministack.core.concurrency import resource_lock, run_offloop
from ministack.core.persistence import PERSIST_STATE, load_state
from ministack.core.responses import (
AccountRegionScopedDict,
AccountScopedDict,
+ apply_image_prefix,
get_account_id,
get_region,
new_uuid,
@@ -83,6 +86,11 @@ from ministack.core.responses import (
logger = logging.getLogger("ec2")
REGION = os.environ.get("MINISTACK_REGION", "us-east-1")
+DOCKER_NETWORK = os.environ.get("DOCKER_NETWORK", "")
+
+# Docker client, created on first use. Only registered images reach for it, so
+# an emulator nobody registered an AMI with never imports docker-py at all.
+_docker = None
# ---------------------------------------------------------------------------
# State
@@ -115,6 +123,7 @@ _vpn_connections = AccountRegionScopedDict() # vpn_id -> VPN connection recor
_launch_templates = AccountRegionScopedDict() # lt_id -> launch template record (includes versions list)
_fleets = AccountRegionScopedDict() # fleet_id -> fleet record
_iam_instance_profile_associations = AccountRegionScopedDict() # assoc_id -> association record
+_images = AccountRegionScopedDict() # ami_id -> registered image record
# Seed defaults once per account/region; do not recreate user-deleted defaults
# on every later EC2 request in a scope.
_default_initialized_scopes = set()
@@ -124,12 +133,22 @@ _default_initialized_scopes = set()
# ── Persistence ────────────────────────────────────────────
def get_state():
+ instances = copy.deepcopy(_instances)
+ # Container ids would dangle across a restart — the boot sweep reclaims this
+ # MiniStack's containers (core/container_reaper.reap_all) long before anyone
+ # reads them back. Keep ImageId so restore can relaunch from the registered
+ # image; drop the handles.
+ for inst in instances.all_values():
+ inst.pop("_container_id", None)
+ inst.pop("_container_name", None)
+ inst.pop("_ssm_managed", None)
return {
"default_initialized_scopes": [
{"AccountId": account_id, "Region": region}
for account_id, region in sorted(_default_initialized_scopes)
],
- "instances": copy.deepcopy(_instances),
+ "images": copy.deepcopy(_images),
+ "instances": instances,
"security_groups": copy.deepcopy(_security_groups),
"key_pairs": copy.deepcopy(_key_pairs),
"placement_groups": copy.deepcopy(_placement_groups),
@@ -187,6 +206,7 @@ def _clear_state():
_launch_templates.clear()
_fleets.clear()
_iam_instance_profile_associations.clear()
+ _images.clear()
_default_initialized_scopes.clear()
@@ -295,8 +315,26 @@ def restore_state(data):
_iam_instance_profile_associations,
data.get("iam_instance_profile_associations", {})
)
+ _restore_regional_store(_images, data.get("images", {}))
if "default_initialized_scopes" not in data:
_infer_default_initialized_scopes_from_restored_stores()
+ _reconcile_backed_instances_after_restore()
+
+
+def _reconcile_backed_instances_after_restore():
+ """A restored container-backed instance has no container behind it: the boot
+ sweep removed it. These are instance-store backed, so the lost root disk is
+ a lost instance — report it terminated rather than claim a box that is gone
+ or offer a start that could only ever produce a different machine."""
+ for inst in _instances.all_values():
+ if inst.get("VmManager") != "docker":
+ continue
+ if inst.get("State", {}).get("Name") in ("running", "stopped"):
+ inst["State"] = {"Code": 48, "Name": "terminated"}
+ inst["_terminated_at"] = time.time()
+ inst.pop("_container_id", None)
+ inst.pop("_container_name", None)
+ inst.pop("_ssm_managed", None)
def _restore_regional_store(store, restored):
@@ -431,6 +469,15 @@ _ensure_defaults_initialized()
# Request routing
# ---------------------------------------------------------------------------
+_DOCKER_ACTIONS = {
+ "RunInstances", "StartInstances", "StopInstances", "RebootInstances",
+ "TerminateInstances", "DescribeInstances",
+ # Reaches Docker indirectly: its _cleanup_terminated() removes the
+ # containers of terminated instances whose terminate-time removal failed.
+ "DescribeIamInstanceProfileAssociations",
+}
+
+
async def handle_request(method, path, headers, body, query_params):
_ensure_defaults_initialized()
params = dict(query_params)
@@ -443,6 +490,15 @@ async def handle_request(method, path, headers, body, query_params):
handler = _ACTION_MAP.get(action)
if not handler:
return _error("InvalidAction", f"Unknown EC2 action: {action}", 400)
+ # Actions that reach the Docker daemon once an image is registered. They
+ # block for as long as the daemon takes, so they go off the loop. The
+ # containers run the registered image's own workload and nothing is wired
+ # back to 4566, so this cannot re-enter: the shared pool is right, not
+ # run_reentrant. (Running a command inside a box can re-enter — a health
+ # probe calling back into MiniStack is the point — so SSM runs
+ # exec_in_instance_blocking on a thread of its own, not on this pool.)
+ if _docker_touched() and action in _DOCKER_ACTIONS:
+ return await run_offloop(handler, params)
return handler(params)
@@ -717,6 +773,65 @@ def _run_instances(p):
iam_profile=iam_profile
)
+ # A registered AMI boots each record as a real container. Unregistered ids
+ # — including every invented ami- a CFN or Terraform stack carries — stay
+ # metadata-only, so existing workloads are untouched.
+ image = _registered_image(image_id)
+ if image is not None and not image.get("ImageLocation"):
+ # Registered from a snapshot, or from nothing at all.
+ # Registered from a snapshot: AWS would boot the volume, and we have
+ # nothing to run. The divergence belongs here rather than at
+ # RegisterImage, which is metadata-only on AWS as well.
+ for inst in created:
+ _discard_instance_records(inst)
+ return _error("InvalidAMIID.Unavailable",
+ f"The image id '[{image_id}]' is not in a state from which "
+ "you can launch an instance", 400)
+ if image is not None:
+ # A container has no persistent root disk, which is instance-store
+ # semantics: no EBS root volume, and StopInstances is refused.
+ for inst in created:
+ for bdm in inst.get("BlockDeviceMappings", []):
+ vol_id = bdm.get("Ebs", {}).get("VolumeId")
+ if vol_id:
+ _volumes.pop(vol_id, None)
+ inst["BlockDeviceMappings"] = []
+ inst["RootDeviceType"] = "instance-store"
+ inst["RootDeviceName"] = image.get("RootDeviceName") or "/dev/sda1"
+ try:
+ for inst in created:
+ container = _vm_launch(inst, image["ImageLocation"])
+ # Publish the handle under the record's lock: handlers no longer
+ # serialise on the event loop, so a TerminateInstances can land
+ # between the launch and the write. Identity, not truthiness —
+ # a same-id record cannot exist, but a popped one must not be
+ # resurrected by this write.
+ with resource_lock("ec2", inst["InstanceId"]):
+ live = _instances.get(inst["InstanceId"])
+ orphaned = live is not inst
+ if orphaned:
+ _drop_container(container)
+ except Exception as e:
+ # A running record with nothing behind it is the failure this
+ # backend exists to remove: back everything out and surface it.
+ logger.warning("EC2: could not boot %s: %s", image["ImageLocation"], e)
+ for inst in created:
+ _vm_remove_container(inst)
+ _discard_instance_records(inst)
+ if _is_image_unavailable(e):
+ # The reference cannot be pulled, so this AMI cannot launch an
+ # instance — a caller condition, not a server one. Retrying an
+ # InternalError here would fail identically forever.
+ return _error(
+ "InvalidAMIID.Unavailable",
+ f"The image id '[{image_id}]' is not in a state from which you "
+ f"can launch an instance: {_esc(image['ImageLocation'])} could "
+ f"not be pulled ({_esc(str(e))})", 400)
+ return _error(
+ "InternalError",
+ f"failed to start instance from {_esc(image['ImageLocation'])}: "
+ f"{_esc(str(e))}", 500)
+
# Process TagSpecifications
i = 1
while _p(p, f"TagSpecification.{i}.ResourceType"):
@@ -755,7 +870,9 @@ def _cleanup_terminated():
if v["State"]["Name"] == "terminated"
and now - v.get("_terminated_at", 0) > 60]
for k in stale:
- _instances.pop(k, None)
+ inst = _instances.pop(k, None)
+ if inst and inst.get("VmManager") == "docker":
+ _vm_remove_container(inst)
def _describe_instances(p):
@@ -769,6 +886,13 @@ def _describe_instances(p):
if iid not in _instances:
return _error("InvalidInstanceID.NotFound", f"The instance ID '{iid}' does not exist", 400)
+ if _docker_touched():
+ # Before the filters run, so an instance-state-name filter sees the
+ # reconciled state. DescribeInstances is dispatched off the loop
+ # whenever an image is registered, so the container inspects cannot
+ # stall the server.
+ _vm_reconcile_running(list(_instances.values()))
+
results = []
for inst in _instances.values():
if filter_ids and inst["InstanceId"] not in filter_ids:
@@ -973,6 +1097,8 @@ def _terminate_instances(p):
prev = inst["State"].copy()
inst["State"] = {"Code": 48, "Name": "terminated"}
inst["_terminated_at"] = time.time()
+ if inst.get("VmManager") == "docker":
+ _vm_remove_container(inst)
assoc = _find_active_iam_instance_profile_association(iid)
if assoc:
_mark_iam_instance_profile_association_disassociated(assoc)
@@ -989,6 +1115,14 @@ def _stop_instances(p):
for iid in ids:
if iid not in _instances:
return _error("InvalidInstanceID.NotFound", f"The instance ID '{iid}' does not exist", 400)
+ for iid in ids:
+ # AWS: "you can't stop an instance that's instance store-backed". A
+ # container-backed instance has no persistent root disk, so there is
+ # nothing a stop could preserve.
+ if _instances[iid].get("RootDeviceType") == "instance-store":
+ return _error("UnsupportedOperation",
+ f"You can't stop the instance '{iid}' because it is an "
+ "instance store-backed instance", 400)
items = ""
for iid in ids:
inst = _instances.get(iid)
@@ -1008,6 +1142,13 @@ def _start_instances(p):
for iid in ids:
if iid not in _instances:
return _error("InvalidInstanceID.NotFound", f"The instance ID '{iid}' does not exist", 400)
+ for iid in ids:
+ # The counterpart of the stop refusal: an instance-store backed instance
+ # has no root disk to bring back, so AWS has no start for it either.
+ if _instances[iid].get("RootDeviceType") == "instance-store":
+ return _error("UnsupportedOperation",
+ f"You can't start the instance '{iid}' because it is an "
+ "instance store-backed instance", 400)
items = ""
for iid in ids:
inst = _instances.get(iid)
@@ -1023,6 +1164,24 @@ def _start_instances(p):
def _reboot_instances(p):
+ ids = _parse_member_list(p, "InstanceId")
+ for iid in ids:
+ if iid not in _instances:
+ return _error("InvalidInstanceID.NotFound",
+ f"The instance ID '{iid}' does not exist", 400)
+ for iid in ids:
+ inst = _instances.get(iid)
+ if inst and inst.get("VmManager") == "docker":
+ container = _vm_get_container(inst)
+ if container is not None:
+ try:
+ container.restart(timeout=10)
+ # A container's IP can change across a restart.
+ _vm_apply_container_ip(inst, container,
+ _get_ministack_network(_get_docker()))
+ inst["_ssm_managed"] = _probe_shell(container)
+ except Exception as e:
+ logger.warning("EC2: could not restart container for %s: %s", iid, e)
return _xml(200, "RebootInstancesResponse", "<return>true</return>")
@@ -1040,41 +1199,160 @@ _STUB_AMIS = [
]
+def _image_view(image):
+ """Registered image as the flat shape the filter and renderer share."""
+ root_type = image.get("RootDeviceType") or ("instance-store" if image.get("ImageLocation") else "ebs")
+ default_desc = (f"Container-backed AMI ({image['ImageLocation']})"
+ if image.get("ImageLocation") else "Snapshot-backed AMI")
+ return {
+ "ImageId": image["ImageId"],
+ "Name": image["Name"],
+ "Description": image["Description"] or default_desc,
+ "Platform": "",
+ "RootDeviceType": root_type,
+ "RootDeviceName": image["RootDeviceName"],
+ "BlockDeviceMappings": image.get("BlockDeviceMappings") or [],
+ "Architecture": image["Architecture"],
+ "VirtualizationType": image["VirtualizationType"],
+ "OwnerId": image.get("OwnerId") or get_account_id(),
+ "IsPublic": "false",
+ "Backed": True,
+ }
+
+
+def _stub_image_view(ami_id, name, desc, platform, root_device):
+ return {
+ "ImageId": ami_id,
+ "Name": name,
+ "Description": desc,
+ "Platform": platform,
+ "RootDeviceType": "ebs",
+ "RootDeviceName": root_device,
+ "BlockDeviceMappings": [],
+ "Architecture": "x86_64",
+ "VirtualizationType": "hvm",
+ "OwnerId": get_account_id(),
+ "IsPublic": "true",
+ "Backed": False,
+ }
+
+
+def _image_matches_filters(view, filters):
+ for name, values in filters.items():
+ if name == "image-id":
+ actual = [view["ImageId"]]
+ elif name == "name":
+ actual = [view["Name"]]
+ elif name == "description":
+ actual = [view["Description"]]
+ elif name == "owner-id":
+ actual = [view["OwnerId"]]
+ elif name == "architecture":
+ actual = [view["Architecture"]]
+ elif name == "virtualization-type":
+ actual = [view["VirtualizationType"]]
+ elif name == "root-device-name":
+ actual = [view["RootDeviceName"]]
+ elif name == "root-device-type":
+ actual = [view["RootDeviceType"]]
+ elif name == "state":
+ actual = ["available"]
+ elif name == "is-public":
+ actual = [view["IsPublic"]]
+ elif name == "platform":
+ actual = [view["Platform"]]
+ elif name == "tag-key":
+ actual = [t["Key"] for t in _tags.get(view["ImageId"], [])]
+ elif name.startswith("tag:"):
+ key = name[4:]
+ actual = [t["Value"] for t in _tags.get(view["ImageId"], []) if t["Key"] == key]
+ else:
+ continue # unknown filter names are ignored, as elsewhere in EC2
+ if not any(_filter_value_matches(a, values) for a in actual):
+ return False
+ return True
+
+
+def _filter_value_matches(actual, values):
+ for v in values:
+ if "*" in v or "?" in v:
+ if re.fullmatch(re.escape(v).replace(r"\*", ".*").replace(r"\?", "."), actual or ""):
+ return True
+ elif actual == v:
+ return True
+ return False
+
+
+def _image_bdm_xml(view):
+ """An instance-store AMI has no EBS root volume, so it emits no mapping —
+ which is also what tells Terraform not to resolve a root device from it."""
+ if view["RootDeviceType"] != "ebs":
+ return ""
+ mappings = view["BlockDeviceMappings"]
+ if not mappings:
+ # RootDeviceName + BlockDeviceMappings are required by Terraform's AWS
+ # provider on aws_instance — it resolves them from DescribeImages before
+ # RunInstances and fails with "finding Root Device Name for AMI" if absent.
+ mappings = [{"DeviceName": view["RootDeviceName"],
+ "Ebs": {"VolumeSize": "8", "VolumeType": "gp2",
+ "DeleteOnTermination": "true"}}]
+ items = ""
+ for m in mappings:
+ ebs = m.get("Ebs") or {}
+ snapshot = (f"<snapshotId>{_esc(ebs['SnapshotId'])}</snapshotId>"
+ if ebs.get("SnapshotId") else "")
+ items += f"""<item>
+ <deviceName>{_esc(m['DeviceName'])}</deviceName>
+ <ebs>
+ {snapshot}
+ <volumeSize>{ebs.get('VolumeSize', '8')}</volumeSize>
+ <volumeType>{ebs.get('VolumeType', 'gp2')}</volumeType>
+ <deleteOnTermination>{ebs.get('DeleteOnTermination', 'true')}</deleteOnTermination>
+ </ebs>
+ </item>"""
+ return f"<blockDeviceMapping>{items}</blockDeviceMapping>"
+
+
def _describe_images(p):
filter_ids = _parse_member_list(p, "ImageId")
+ filters = _parse_filters(p)
+ # Registered images first: those are the ones that actually boot. The stubs
+ # stay so a workload naming one keeps launching a metadata-only instance.
+ views = [_image_view(img) for img in _images.values()]
+ seen = {v["ImageId"] for v in views}
+ views += [_stub_image_view(*stub) for stub in _STUB_AMIS if stub[0] not in seen]
+
items = ""
- for ami_id, name, desc, platform, root_device in _STUB_AMIS:
- if filter_ids and ami_id not in filter_ids:
+ for view in views:
+ if filter_ids and view["ImageId"] not in filter_ids:
+ continue
+ if filters and not _image_matches_filters(view, filters):
continue
# RootDeviceName + BlockDeviceMappings are required by Terraform's AWS
# provider on aws_instance — it resolves them from DescribeImages before
# RunInstances and fails with "finding Root Device Name for AMI" if absent.
- platform_xml = f"<platform>{platform}</platform>" if platform else ""
+ platform_xml = f"<platform>{view['Platform']}</platform>" if view["Platform"] else ""
+ tag_items = "".join(
+ f"<item><key>{_esc(t['Key'])}</key><value>{_esc(t.get('Value', ''))}</value></item>"
+ for t in _tags.get(view["ImageId"], []))
+ tag_xml = f"<tagSet>{tag_items}</tagSet>" if tag_items else ""
items += f"""<item>
- <imageId>{ami_id}</imageId>
- <imageLocation>{name}</imageLocation>
+ <imageId>{view['ImageId']}</imageId>
+ <imageLocation>{_esc(view['Name'])}</imageLocation>
<imageState>available</imageState>
- <imageOwnerId>{get_account_id()}</imageOwnerId>
- <isPublic>true</isPublic>
- <architecture>x86_64</architecture>
+ <imageOwnerId>{view['OwnerId']}</imageOwnerId>
+ <isPublic>{view['IsPublic']}</isPublic>
+ <architecture>{view['Architecture']}</architecture>
<imageType>machine</imageType>
- <name>{name}</name>
- <description>{desc}</description>
+ <name>{_esc(view['Name'])}</name>
+ <description>{_esc(view['Description'])}</description>
{platform_xml}
- <rootDeviceType>ebs</rootDeviceType>
- <rootDeviceName>{root_device}</rootDeviceName>
- <blockDeviceMapping>
- <item>
- <deviceName>{root_device}</deviceName>
- <ebs>
- <volumeSize>8</volumeSize>
- <volumeType>gp2</volumeType>
- <deleteOnTermination>true</deleteOnTermination>
- </ebs>
- </item>
- </blockDeviceMapping>
- <virtualizationType>hvm</virtualizationType>
+ <rootDeviceType>{view['RootDeviceType']}</rootDeviceType>
+ <rootDeviceName>{view['RootDeviceName']}</rootDeviceName>
+ {_image_bdm_xml(view)}
+ <virtualizationType>{view['VirtualizationType']}</virtualizationType>
<hypervisor>xen</hypervisor>
+ {tag_xml}
</item>"""
return _xml(200, "DescribeImagesResponse", f"<imagesSet>{items}</imagesSet>")
@@ -1572,6 +1850,450 @@ def _import_key_pair(p):
<keyPairId>{_key_pairs[name]['KeyPairId']}</keyPairId>""")
+# ---------------------------------------------------------------------------
+# Registered AMIs and container-backed instances
+# ---------------------------------------------------------------------------
+# RunInstances against a registered AMI boots that image as a container, so an
+# instance has a real box behind it and ssm:SendCommand can return a real exit
+# code instead of always reporting success. RegisterImage is the opt-in: with
+# no image registered, EC2 never reaches for Docker and every instance is the
+# metadata-only record it has always been.
+#
+# The container is not the emulated thing — it is what makes the emulated API
+# behave correctly, the same role Postgres plays for RDS and k3s for EKS. It is
+# private to this module: nothing schedules ECS tasks or EKS pods into it.
+#
+# Divergences, deliberately: ImageLocation carries a container reference rather
+# than an S3 manifest path; there is no IMDS, no console output, no
+# security-group enforcement or port publishing, no EBS-to-block-device
+# mapping, and a container filesystem does not survive a restart.
+
+# Same bound the reaper's client uses (app.py), so a hung daemon cannot pin a request for
+# docker-py's 60s default. Not a new knob: MINISTACK_DOCKER_TIMEOUT already exists.
+_DOCKER_TIMEOUT = float(os.environ.get("MINISTACK_DOCKER_TIMEOUT", "10"))
+
+
+def _new_image_id():
+ return "ami-" + "".join(random.choices(string.hexdigits[:16], k=17))
+
+
+def _get_docker():
+ """Docker client, or None. Imported lazily: an emulator with no registered
+ image never pays for docker-py at all."""
+ global _docker
+ if _docker is None:
+ try:
+ import docker
+ _docker = docker.from_env(timeout=_DOCKER_TIMEOUT)
+ except Exception as e:
+ logger.debug("EC2: no Docker client available: %s", e)
+ return _docker
+
+
+def _get_ministack_network(client):
+ """The Docker network MiniStack is on, so instances are reachable from it."""
+ if DOCKER_NETWORK:
+ return DOCKER_NETWORK
+ try:
+ hostname = os.environ.get("HOSTNAME", "")
+ if not hostname:
+ return None
+ # Docker names the container's hostname after its id, Podman after its
+ # name. containers.get() resolves either.
+ self_container = client.containers.get(hostname)
+ nets = list(self_container.attrs["NetworkSettings"]["Networks"].keys())
+ return nets[0] if nets else None
+ except Exception:
+ return None
+
+
+def _registered_image(image_id):
+ """The registered image record for an AMI id, or None."""
+ return _images.get(image_id) if image_id else None
+
+
+# Set the first time a container is launched. A registration can be withdrawn while its
+# instances are still running, so "an image is registered" is not the same question as
+# "is there a container of ours out there".
+_docker_in_use = False
+
+
+def _any_image_registered():
+ """Whether an AMI backed by a container reference exists in any scope."""
+ return bool(_images.all_values())
+
+
+def _docker_touched():
+ """Whether anything here could need the daemon. Gates every Docker touch, including the
+ describe-time reconcile and the reset sweep, so an emulator nobody registered an image
+ with never imports docker-py at all."""
+ return _docker_in_use or _any_image_registered()
+
+
+def _ec2_container_name(instance_id):
+ return f"ministack-ec2-{get_account_id()}-{get_region()}-{instance_id}"
+
+
+# ── RegisterImage / DeregisterImage ────────────────────────
+
+def _parse_register_block_device_mappings(p):
+ """BlockDeviceMapping.N off a RegisterImage request."""
+ mappings = []
+ i = 1
+ while _p(p, f"BlockDeviceMapping.{i}.DeviceName"):
+ ebs = {}
+ for field in ("SnapshotId", "VolumeSize", "VolumeType", "DeleteOnTermination",
+ "Encrypted", "Iops", "Throughput"):
+ value = _p(p, f"BlockDeviceMapping.{i}.Ebs.{field}")
+ if value:
+ ebs[field] = value
+ mapping = {"DeviceName": _p(p, f"BlockDeviceMapping.{i}.DeviceName")}
+ virtual_name = _p(p, f"BlockDeviceMapping.{i}.VirtualName")
+ if virtual_name:
+ mapping["VirtualName"] = virtual_name
+ if ebs:
+ mapping["Ebs"] = ebs
+ mappings.append(mapping)
+ i += 1
+ return mappings
+
+
+# "3-128 alphanumeric characters, parentheses (()), square brackets ([]), spaces ( ),
+# periods (.), slashes (/), dashes (-), single quotes ('), at-signs (@), or underscores(_)"
+_AMI_NAME_RE = re.compile(r"^[A-Za-z0-9()\[\] ./\-'@_]{3,128}$")
+
+
+def _register_image(p):
+ name = _p(p, "Name")
+ if not name:
+ return _error("MissingParameter", "The request must contain the parameter Name", 400)
+ if not _AMI_NAME_RE.match(name):
+ return _error("InvalidAMIName.Malformed",
+ "AMI names must be between 3 and 128 characters long, and may only "
+ "contain letters, numbers, and the following special characters: "
+ "'-', '_', '.', '/', '(', and ')'", 400)
+ location = _p(p, "ImageLocation")
+ mappings = _parse_register_block_device_mappings(p)
+ snapshot_backed = any(m.get("Ebs", {}).get("SnapshotId") for m in mappings)
+ # Only Name is required on AWS: ImageLocation and BlockDeviceMapping.N are both optional, and
+ # registration never checks that the result can boot. Refusing here would make MiniStack
+ # stricter than AWS, so a registration with neither is accepted and fails at RunInstances.
+ for existing in _images.values():
+ if existing["Name"] == name:
+ return _error("InvalidAMIName.Duplicate",
+ f"AMI name {name} is already in use by another AMI", 400)
+ image_id = _new_image_id()
+ # Registration is metadata-only on AWS too: it neither touches the disk nor
+ # checks that the image can boot. So a snapshot-backed registration is
+ # accepted exactly as AWS accepts it (import-snapshot then register-image is
+ # the ordinary Packer/Terraform flow) and the divergence lands where it
+ # belongs — on RunInstances, the call that genuinely cannot do the thing.
+ #
+ # The two root device types stay honest. A container has no persistent root
+ # disk, which is instance-store semantics: no EBS root volume and no stop.
+ # A snapshot-backed AMI is `ebs`, and would keep its disk across a stop —
+ # it just has nothing here to run.
+ root_device_type = "instance-store" if location else "ebs"
+ _images[image_id] = {
+ "ImageId": image_id,
+ "Name": name,
+ "ImageLocation": location,
+ "Description": _p(p, "Description") or "",
+ # AWS documents these defaults for RegisterImage: Architecture "For Amazon EBS-backed
+ # AMIs, i386", VirtualizationType "Default: paravirtual". A caller that cares passes them.
+ "Architecture": _p(p, "Architecture") or "i386",
+ "RootDeviceType": root_device_type,
+ "RootDeviceName": (_p(p, "RootDeviceName")
+ or ("/dev/sda1" if root_device_type == "instance-store"
+ else "/dev/xvda")),
+ "BlockDeviceMappings": mappings,
+ "VirtualizationType": _p(p, "VirtualizationType") or "paravirtual",
+ "CreationDate": _now_ts(),
+ "OwnerId": get_account_id(),
+ }
+ _parse_tag_specs(p, "image", image_id)
+ return _xml(200, "RegisterImageResponse", f"<imageId>{image_id}</imageId>")
+
+
+def _deregister_image(p):
+ image_id = _p(p, "ImageId")
+ if not image_id:
+ return _error("MissingParameter", "The request must contain the parameter ImageId", 400)
+ if image_id not in _images:
+ return _error("InvalidAMIID.NotFound",
+ f"The image id '[{image_id}]' does not exist", 400)
+ _images.pop(image_id, None)
+ _tags.pop(image_id, None)
+ return _xml(200, "DeregisterImageResponse", "<return>true</return>")
+
+
+# ── Container lifecycle ────────────────────────────────────
+
+def _vm_apply_container_ip(inst, container, network):
+ """Replace the synthetic addresses with the container's real IP."""
+ try:
+ container.reload()
+ nets = container.attrs.get("NetworkSettings", {}).get("Networks", {})
+ if network and network in nets:
+ ip = nets[network].get("IPAddress", "")
+ else:
+ ip = next(iter(nets.values())).get("IPAddress", "") if nets else ""
+ if not ip:
+ logger.warning("EC2: no container IP for %s; keeping synthetic addresses",
+ inst["InstanceId"])
+ return
+ inst["PrivateIpAddress"] = ip
+ inst["PublicIpAddress"] = ip
+ inst["PrivateDnsName"] = f"ip-{ip.replace('.', '-')}.ec2.internal"
+ inst["PublicDnsName"] = f"ec2-{ip.replace('.', '-')}.compute-1.amazonaws.com"
+ except Exception as e:
+ logger.warning("EC2: could not read container IP for %s: %s", inst["InstanceId"], e)
+
+
+def _probe_shell(container):
+ """Whether the guest has a shell. No shell is our analogue of no SSM agent:
+ the instance is unmanaged and SendCommand refuses it, as on AWS."""
+ try:
+ result = container.exec_run(["/bin/sh", "-c", "true"])
+ return getattr(result, "exit_code", 1) == 0
+ except Exception:
+ return False
+
+
+def _is_image_unavailable(exc):
+ """Whether a boot failure is "that reference cannot be pulled" rather than
+ a daemon or runtime problem."""
+ try:
+ from docker import errors as docker_errors
+ except Exception:
+ return False
+ if isinstance(exc, (docker_errors.ImageNotFound, docker_errors.NotFound)):
+ return True
+ # A registry that refuses or does not resolve surfaces as APIError.
+ return (isinstance(exc, docker_errors.APIError)
+ and any(t in str(exc).lower()
+ for t in ("not found", "pull access denied", "manifest unknown",
+ "no such image", "repository does not exist")))
+
+
+def _vm_launch(inst, image_ref):
+ """Boot one instance record as a container. Raises on create/start failure;
+ the caller backs the records out and surfaces the error."""
+ client = _get_docker()
+ if client is None:
+ raise RuntimeError("no Docker daemon available")
+ network = _get_ministack_network(client)
+ name = _ec2_container_name(inst["InstanceId"])
+ try:
+ client.containers.get(name).remove(force=True)
+ except Exception:
+ pass # no stale container by that name, which is the normal case
+ kwargs = {
+ "name": name,
+ "detach": True,
+ "labels": {
+ **container_reaper.own_labels("ec2"),
+ "instance_id": inst["InstanceId"],
+ "account_id": get_account_id(),
+ "region": get_region(),
+ },
+ # A real instance does not exit the moment it boots, so keep the
+ # container alive. An image with its own ENTRYPOINT receives this as
+ # argv and is free to ignore it; init reaps the keepalive.
+ "init": True,
+ "command": ["sleep", "infinity"],
+ }
+ if network:
+ kwargs["network"] = network
+ container = client.containers.run(apply_image_prefix(image_ref), **kwargs)
+ global _docker_in_use
+ _docker_in_use = True
+ inst["_container_id"] = container.id
+ inst["_container_name"] = name
+ inst["VmManager"] = "docker"
+ _vm_apply_container_ip(inst, container, network)
+ inst["_ssm_managed"] = _probe_shell(container)
+ logger.info("EC2: instance %s running in container %s (image %s)",
+ inst["InstanceId"], name, image_ref)
+ return container
+
+
+def _vm_get_container(inst):
+ client = _get_docker()
+ if client is None:
+ return None
+ for ref in (inst.get("_container_id"), inst.get("_container_name"),
+ _ec2_container_name(inst["InstanceId"])):
+ if not ref:
+ continue
+ try:
+ return client.containers.get(ref)
+ except Exception:
+ continue
+ return None
+
+
+def _drop_container(container):
+ """Tear down a container nothing references. Never called under a lock —
+ Docker calls must not be held across resource_lock."""
+ try:
+ container.remove(force=True)
+ except Exception as e:
+ logger.warning("EC2: could not remove orphaned container: %s", e)
+
+
+def _vm_remove_container(inst):
+ container = _vm_get_container(inst)
+ if container is not None:
+ try:
+ container.remove(force=True)
+ except Exception as e:
+ logger.warning("EC2: could not remove container for %s: %s",
+ inst["InstanceId"], e)
+ inst.pop("_container_id", None)
+ inst.pop("_container_name", None)
+ inst.pop("_ssm_managed", None)
+
+
+def _discard_instance_records(inst):
+ """Back out a failed launch: instance, its synthetic root volume, and its
+ IAM instance profile association — exactly what _launch_instances_internal
+ creates."""
+ iid = inst["InstanceId"]
+ for bdm in inst.get("BlockDeviceMappings", []):
+ vol_id = bdm.get("Ebs", {}).get("VolumeId")
+ if vol_id:
+ _volumes.pop(vol_id, None)
+ for key in [k for k, a in _iam_instance_profile_associations.items()
+ if a.get("InstanceId") == iid]:
+ _iam_instance_profile_associations.pop(key, None)
+ _instances.pop(iid, None)
+
+
+def _vm_reconcile_running(instances):
+ """Downgrade a "running" record whose container is gone or has exited.
+
+ A box can die underneath its record — the workload exits, the daemon kills
+ it, someone removes the container by hand. Reconciled lazily here rather
+ than by a poller: the record is only wrong for as long as nobody looks.
+ Downgrade only; StartInstances is the one path back to running.
+ """
+ client = _get_docker()
+ if client is None:
+ return
+ try:
+ client.ping()
+ except Exception:
+ return # an unreachable daemon says nothing about the containers
+ from docker import errors as docker_errors
+ for inst in instances:
+ if inst.get("VmManager") != "docker":
+ continue
+ if inst.get("State", {}).get("Name") != "running":
+ continue
+ refs = [r for r in (inst.get("_container_id"), inst.get("_container_name")) if r]
+ if not refs:
+ continue # mid-launch: no container claim to judge yet
+ container, verdict = None, "gone"
+ for ref in refs:
+ try:
+ container = client.containers.get(ref)
+ break
+ except docker_errors.NotFound:
+ continue
+ except Exception as e:
+ # Transient daemon trouble is not evidence the box died.
+ logger.debug("EC2: container lookup for %s failed: %s", inst["InstanceId"], e)
+ verdict = "unknown"
+ break
+ if verdict == "unknown":
+ continue
+ status = getattr(container, "status", "")
+ if container is not None and status not in ("exited", "dead"):
+ continue
+ # Instance-store backed: losing the root disk is losing the instance.
+ # AWS terminates such an instance rather than stopping it, and there is
+ # nothing here a start could recover — the container filesystem is gone.
+ inst["State"] = {"Code": 48, "Name": "terminated"}
+ inst["_terminated_at"] = time.time()
+ inst.pop("_ssm_managed", None)
+ inst.pop("_container_id", None)
+ inst.pop("_container_name", None)
+ logger.info("EC2: instance %s container %s — reporting terminated",
+ inst["InstanceId"], "gone" if container is None else status)
+
+
+def _vm_live_container_ids():
+ """Container ids an instance record still owns, for the periodic reaper.
+
+ A stopped instance keeps its exited container so StartInstances can restart
+ it, so every recorded id is reported whatever the instance state.
+ Terminated instances leave the store within a minute (_cleanup_terminated),
+ after which a container a failed terminate left behind stops being vouched
+ for and the reaper may reclaim it.
+ """
+ return {inst.get("_container_id")
+ for inst in _instances.all_values() if inst.get("_container_id")}
+
+
+container_reaper.register_live_ids("ec2", _vm_live_container_ids)
+
+
+def _vm_sweep_containers():
+ """Remove this MiniStack's instance containers. Used by reset.
+
+ Scoped by ministack.instance like the rest of the reaping machinery:
+ several MiniStacks can share a daemon and a reset of this one must not take
+ down another's boxes.
+ """
+ if not _docker_touched():
+ return
+ client = _get_docker()
+ if client is None:
+ return
+ try:
+ containers = client.containers.list(all=True, filters={"label": [
+ "ministack=ec2",
+ f"{container_reaper.INSTANCE_LABEL}={container_reaper.instance_id()}",
+ ]})
+ except Exception as e:
+ logger.warning("EC2: reset container sweep failed: %s", e)
+ return
+ container_reaper.drop_containers(containers, force=True)
+
+
+# ── Exec seam (SSM Run Command) ────────────────────────────
+
+def instance_is_managed(instance_id):
+ """Whether SSM can run a command on this instance.
+
+ An instance is managed when it has a box with a shell behind it, which is
+ this emulator's analogue of a running SSM agent. AWS refuses commands for
+ an unmanaged instance and so do we.
+ """
+ inst = _instances.get(instance_id)
+ if not inst or inst.get("State", {}).get("Name") != "running":
+ return False
+ return bool(inst.get("_ssm_managed"))
+
+
+def exec_in_instance_blocking(instance_id, argv):
+ inst = _instances.get(instance_id)
+ if not inst:
+ return 255, "", "instance does not exist"
+ container = _vm_get_container(inst)
+ if container is None:
+ return 255, "", "instance has no backing container"
+ try:
+ result = container.exec_run(argv, demux=True)
+ except Exception as e:
+ return 255, "", str(e)
+ stdout, stderr = (result.output if isinstance(result.output, tuple) else (result.output, None))
+ return (result.exit_code,
+ (stdout or b"").decode("utf-8", "replace"),
+ (stderr or b"").decode("utf-8", "replace"))
+
+
# ---------------------------------------------------------------------------
# Placement Groups
# ---------------------------------------------------------------------------
@@ -4836,6 +5558,9 @@ def _delete_vpn_connection_route(p):
# ---------------------------------------------------------------------------
def reset():
+ _vm_sweep_containers()
+ global _docker_in_use
+ _docker_in_use = False
_clear_state()
_init_defaults()
@@ -6030,6 +6755,8 @@ _ACTION_MAP = {
"DisassociateIamInstanceProfile": _disassociate_iam_instance_profile,
"ReplaceIamInstanceProfileAssociation": _replace_iam_instance_profile_association,
"DescribeImages": _describe_images,
+ "RegisterImage": _register_image,
+ "DeregisterImage": _deregister_image,
"CreateSecurityGroup": _create_security_group,
"DeleteSecurityGroup": _delete_security_group,
"DescribeSecurityGroups": _describe_security_groups,
diff --git a/ministack/services/ssm.py b/ministack/services/ssm.py
index ef9b365c..b04ab637 100644
--- a/ministack/services/ssm.py
+++ b/ministack/services/ssm.py
@@ -18,6 +18,7 @@ import time
from datetime import datetime, timezone
from ministack.core.arn import ArnParseError, parse_arn
+from ministack.core.concurrency import spawn_background
from ministack.core.responses import (
AccountRegionScopedDict,
AccountScopedDict,
@@ -700,26 +701,95 @@ def _param_out(param, with_decryption=False):
# ── Run Command ────────────────────────────────────────────
#
-# Nothing here runs a script, so an invocation completes as Success with empty output rather than
-# never converging. Only an unreachable instance fails, which is how a caller reaches its error path.
+# An invocation is asynchronous, as on AWS: SendCommand answers Pending and the caller polls
+# GetCommandInvocation until it reaches a terminal state. For AWS-RunShellScript against an
+# instance with a box behind it, the script runs in that box and the invocation carries its real
+# exit code and output, so a health probe can genuinely fail. A document we cannot run completes
+# as Success with empty output rather than never converging.
_RUN_COMMAND_PLUGIN = "aws:runShellScript"
+# GetCommandInvocation: "The first 24,000 characters written by the plugin to stdout" and
+# "The first 8,000 characters written by the plugin to stderr".
+_STDOUT_LIMIT = 24000
+_STDERR_LIMIT = 8000
+_SHELL_DOCUMENTS = {"AWS-RunShellScript"}
def _managed_instance_ids() -> set:
- """Instances Run Command can reach: the ones EC2 has running in this account and region.
+ """Instances Run Command can reach.
- Anything else has no agent answering, so real SSM neither lists it nor takes a command for it.
+ An agent has to be answering, which AWS folds into InvalidInstanceId along with the node's
+ state ("SSM Agent isn't running" is one of the causes it lists). A metadata-only instance has
+ nothing running on it, so real SSM would refuse it too. `instance_is_managed` covers both:
+ it requires the instance to be running and to have a box with a shell behind it.
"""
from ministack.services import ec2 as ec2_svc
return {
instance_id
- for instance_id, inst in ec2_svc._instances.items()
- if (inst.get("State") or {}).get("Name") == "running"
+ for instance_id in ec2_svc._instances
+ if ec2_svc.instance_is_managed(instance_id)
}
+def _shell_argv(data) -> list:
+ """The argv for an AWS-RunShellScript invocation, from its `commands` parameter."""
+ commands = (data.get("Parameters") or {}).get("commands") or []
+ if isinstance(commands, str):
+ commands = [commands]
+ script = "\n".join(str(c) for c in commands)
+ return ["/bin/sh", "-c", script]
+
+
+def _finish_invocation(command_id, instance_id, argv):
+ """Run one invocation to completion and publish its result.
+
+ On its own thread: the script is the caller's, and a readiness probe calling back into
+ MiniStack is the point of the feature, so it must not take a slot from the shared pool.
+ """
+ from ministack.services import ec2 as ec2_svc
+
+ record = _commands.get(command_id)
+ if not record:
+ return
+ invocation = (record.get("Invocations") or {}).get(instance_id)
+ if invocation is None:
+ return
+ invocation["Status"] = "InProgress"
+ invocation["StatusDetails"] = "In Progress"
+ invocation["ExecutionStartDateTime"] = _now_iso()
+ exit_code, stdout, stderr = ec2_svc.exec_in_instance_blocking(instance_id, argv)
+ invocation["ResponseCode"] = exit_code
+ invocation["StandardOutputContent"] = stdout[:_STDOUT_LIMIT]
+ invocation["StandardErrorContent"] = stderr[:_STDERR_LIMIT]
+ invocation["ExecutionEndDateTime"] = _now_iso()
+ # Status flips last: a poller that sees a terminal state must already see the exit code,
+ # the output and the end time.
+ # "For a plugin, this indicates that the result code wasn't zero."
+ invocation["StatusDetails"] = "Success" if exit_code == 0 else "Failed"
+ invocation["Status"] = invocation["StatusDetails"]
+ _refresh_command_status(record)
+
+
+def _refresh_command_status(record) -> None:
+ """Roll the invocations up into the parent command."""
+ invocations = list((record.get("Invocations") or {}).values())
+ terminal = [i for i in invocations
+ if i["Status"] in ("Success", "Failed", "Cancelled", "TimedOut")]
+ failed = [i for i in invocations if i["Status"] == "Failed"]
+ record["CompletedCount"] = len(terminal)
+ record["ErrorCount"] = len(failed)
+ if len(terminal) < len(invocations):
+ record["Status"] = "InProgress"
+ record["StatusDetails"] = "In Progress"
+ elif failed:
+ record["Status"] = "Failed"
+ record["StatusDetails"] = "Failed"
+ else:
+ record["Status"] = "Success"
+ record["StatusDetails"] = "Success"
+
+
def _command_target_ids(data) -> list:
"""The instances a SendCommand addresses: InstanceIds, or a Targets entry naming them."""
instance_ids = list(data.get("InstanceIds") or [])
@@ -767,15 +837,17 @@ def _send_command(data):
"InstanceIds": instance_ids,
"Targets": data.get("Targets", []),
"RequestedDateTime": now,
- "Status": "Success",
- "StatusDetails": "Success",
+ # SendCommand answers before the command has been delivered — the doc's own sample
+ # response is Pending. The caller polls GetCommandInvocation from here.
+ "Status": "Pending",
+ "StatusDetails": "Pending",
"OutputS3Region": get_region(),
"OutputS3BucketName": data.get("OutputS3BucketName", ""),
"OutputS3KeyPrefix": data.get("OutputS3KeyPrefix", ""),
"MaxConcurrency": data.get("MaxConcurrency", "50"),
"MaxErrors": data.get("MaxErrors", "0"),
"TargetCount": len(instance_ids),
- "CompletedCount": len(instance_ids),
+ "CompletedCount": 0,
"ErrorCount": 0,
"DeliveryTimedOutCount": 0,
"ServiceRole": data.get("ServiceRoleArn", ""),
@@ -788,13 +860,16 @@ def _send_command(data):
"DocumentName": document,
"DocumentVersion": data.get("DocumentVersion", "$DEFAULT"),
"PluginName": _RUN_COMMAND_PLUGIN,
- "ResponseCode": 0,
- # These three are strings in the SSM model, not timestamps.
- "ExecutionStartDateTime": started,
+ # "If the response code is -1, then the command hasn't started running on the
+ # managed node, or it wasn't received by the node."
+ "ResponseCode": -1,
+ # These three are strings in the SSM model, not timestamps, and are empty until
+ # the plugin starts.
+ "ExecutionStartDateTime": "",
"ExecutionElapsedTime": "PT0.001S",
- "ExecutionEndDateTime": started,
- "Status": "Success",
- "StatusDetails": "Success",
+ "ExecutionEndDateTime": "",
+ "Status": "Pending",
+ "StatusDetails": "Pending",
"StandardOutputContent": "",
"StandardOutputUrl": "",
"StandardErrorContent": "",
@@ -804,6 +879,21 @@ def _send_command(data):
},
}
_commands[command_id] = record
+ if document in _SHELL_DOCUMENTS:
+ argv = _shell_argv(data)
+ for instance_id in instance_ids:
+ spawn_background(_finish_invocation, command_id, instance_id, argv,
+ thread_name=f"ministack-ssm-{command_id[:8]}")
+ else:
+ # A document we cannot run still has to converge, or a caller polling for a terminal
+ # state never returns.
+ for instance_id in instance_ids:
+ invocation = record["Invocations"][instance_id]
+ invocation.update({"ResponseCode": 0, "Status": "Success",
+ "StatusDetails": "Success",
+ "ExecutionStartDateTime": started,
+ "ExecutionEndDateTime": started})
+ _refresh_command_status(record)
return json_response({"Command": _public_command(record)})
diff --git a/tests/test_ec2.py b/tests/test_ec2.py
index bcd215bd..08bc9b96 100644
--- a/tests/test_ec2.py
+++ b/tests/test_ec2.py
@@ -3603,3 +3603,530 @@ def test_ec2_allocated_address_is_a_valid_ipv4(ec2):
ipaddress.ip_address(described["PublicIp"])
finally:
ec2.release_address(AllocationId=alloc["AllocationId"])
+
+
+# ---------------------------------------------------------------------------
+# Registered AMIs and container-backed instances
+# ---------------------------------------------------------------------------
+# Unit tier: in-process against a faked Docker client, so it runs with no
+# daemon. Live tier: gated on a reachable daemon, probed rather than read from
+# an env var CI never sets.
+
+import ministack.services.ec2 as ec2mod
+
+
+def _docker_reachable():
+ try:
+ import docker as _probe
+
+ _probe.from_env(timeout=5).ping()
+ return True
+ except Exception:
+ return False
+
+
+requires_docker = pytest.mark.skipif(
+ not _docker_reachable(), reason="no reachable Docker daemon")
+
+
+class _FakeContainer:
+ def __init__(self, name, kwargs, registry, ip="172.30.0.9", network="testnet",
+ shell=True):
+ self.id = f"cid-{name}"
+ self.name = name
+ self.kwargs = kwargs
+ self.status = "running"
+ self.attrs = {"NetworkSettings": {"Networks": {network: {"IPAddress": ip}}}}
+ self.execs = []
+ self.stop_calls = self.start_calls = self.restart_calls = self.remove_calls = 0
+ self._registry = registry
+ self._shell = shell
+
+ def reload(self):
+ pass
+
+ def exec_run(self, cmd, demux=False, **kw):
+ self.execs.append(cmd)
+ if not self._shell:
+ raise RuntimeError("no shell in image")
+ out = (b"out", b"err") if demux else b"out"
+
+ class _R:
+ exit_code = 0
+ output = out
+ return _R()
+
+ def start(self):
+ self.start_calls += 1
+ self.status = "running"
+
+ def stop(self, timeout=10):
+ self.stop_calls += 1
+ self.status = "exited"
+
+ def restart(self, timeout=10):
+ self.restart_calls += 1
+
+ def remove(self, force=False, v=False):
+ self.remove_calls += 1
+ self._registry.pop(self.id, None)
+ self._registry.pop(self.name, None)
+
+
+def _fake_docker(run_error=None, shell=True):
+ registry, runs = {}, []
+
+ class _Containers:
+ def run(self, image, **kwargs):
+ if run_error is not None:
+ raise run_error
+ runs.append({"image": image, **kwargs})
+ c = _FakeContainer(kwargs["name"], kwargs, registry, shell=shell)
+ registry[c.id] = registry[c.name] = c
+ return c
+
+ def get(self, ref):
+ if ref not in registry:
+ from docker import errors
+ raise errors.NotFound("absent")
+ return registry[ref]
+
+ def list(self, all=False, filters=None):
+ want = (filters or {}).get("label") or []
+ want = [want] if isinstance(want, str) else list(want)
+ out = []
+ for c in set(registry.values()):
+ have = {f"{k}={v}" for k, v in c.kwargs.get("labels", {}).items()}
+ if not [w for w in want if w not in have]:
+ out.append(c)
+ return out
+
+ class _Docker:
+ def __init__(self):
+ self.containers = _Containers()
+
+ def ping(self):
+ return True
+
+ d = _Docker()
+ d._registry, d._runs = registry, runs
+ return d
+
+
+def vm(monkeypatch):
+ """A faked Docker daemon wired into the ec2 module, with state torn down."""
+ fake = _fake_docker()
+ monkeypatch.setattr(ec2mod, "_get_docker", lambda: fake)
+ monkeypatch.setattr(ec2mod, "_get_ministack_network", lambda _c: "testnet")
+ created = []
+ yield fake, created
+ for iid in created:
+ ec2mod._instances.pop(iid, None)
+ for key in list(ec2mod._images.keys()):
+ ec2mod._images.pop(key, None)
+
+
+def _register(location="example/box:1", name="box"):
+ resp = ec2mod._register_image({"Name": [name], "ImageLocation": [location]})
+ import re as _re
+ return _re.search(r"<imageId>(ami-[0-9a-f]+)</imageId>", resp[2].decode()).group(1)
+
+
+def _launch(ami, count=1):
+ resp = ec2mod._run_instances({"ImageId": [ami], "MinCount": [str(count)],
+ "MaxCount": [str(count)]})
+ import re as _re
+ body = resp[2].decode()
+ return _re.findall(r"<instanceId>(i-[0-9a-f]+)</instanceId>", body), resp
+
+
+def test_ec2_register_image_round_trips(ec2):
+ reg = ec2.register_image(Name=f"box-{_uuid_mod.uuid4().hex[:8]}",
+ ImageLocation="alpine:3", Description="desc",
+ Architecture="x86_64")
+ ami = reg["ImageId"]
+ try:
+ assert ami.startswith("ami-") and len(ami) == len("ami-") + 17
+ img = ec2.describe_images(ImageIds=[ami])["Images"][0]
+ assert img["State"] == "available"
+ # A container has no persistent root disk: instance-store semantics,
+ # so no EBS root mapping either.
+ assert img["RootDeviceType"] == "instance-store"
+ assert img["RootDeviceName"] == "/dev/sda1"
+ assert img.get("BlockDeviceMappings", []) == []
+ assert img["Description"] == "desc"
+ assert img["Public"] is False
+ finally:
+ ec2.deregister_image(ImageId=ami)
+ with pytest.raises(ClientError) as exc:
+ ec2.deregister_image(ImageId=ami)
+ assert exc.value.response["Error"]["Code"] == "InvalidAMIID.NotFound"
+
+
+def test_ec2_register_image_accepts_a_bare_name(ec2):
+ # Only Name is required on AWS — ImageLocation and BlockDeviceMapping.N are both optional and
+ # registration never checks the result can boot. Refusing here would be stricter than AWS,
+ # which is the failure mode that bites: code that works against AWS breaking locally.
+ ami = ec2.register_image(Name=f"bare-{_uuid_mod.uuid4().hex[:8]}")["ImageId"]
+ try:
+ img = ec2.describe_images(ImageIds=[ami])["Images"][0]
+ assert img["RootDeviceType"] == "ebs"
+ # Documented RegisterImage defaults.
+ assert img["Architecture"] == "i386"
+ assert img["VirtualizationType"] == "paravirtual"
+ # Nothing to boot, so the failure lands on the call that cannot do it.
+ with pytest.raises(ClientError) as exc:
+ ec2.run_instances(ImageId=ami, MinCount=1, MaxCount=1)
+ assert exc.value.response["Error"]["Code"] == "InvalidAMIID.Unavailable"
+ finally:
+ ec2.deregister_image(ImageId=ami)
+
+
+def test_ec2_register_image_rejects_a_malformed_name(ec2):
+ # "AMI names must be between 3 and 128 characters long, and may only contain letters,
+ # numbers, and the following special characters..."
+ for bad in ("a" * 129, "no,commas", "semi;colon"):
+ with pytest.raises(ClientError) as exc:
+ ec2.register_image(Name=bad, ImageLocation="alpine:3")
+ assert exc.value.response["Error"]["Code"] == "InvalidAMIName.Malformed", bad
+
+
+def test_ec2_describe_images_filters_registered_and_stubs(ec2):
+ unique = _uuid_mod.uuid4().hex[:8]
+ ami = ec2.register_image(Name=f"filtered-{unique}", ImageLocation="alpine:3")["ImageId"]
+ try:
+ by_name = ec2.describe_images(
+ Filters=[{"Name": "name", "Values": [f"filtered-{unique}"]}])["Images"]
+ assert [i["ImageId"] for i in by_name] == [ami]
+ wildcard = ec2.describe_images(
+ Filters=[{"Name": "name", "Values": ["filtered-*"]}])["Images"]
+ assert ami in [i["ImageId"] for i in wildcard]
+ assert ec2.describe_images(
+ Filters=[{"Name": "name", "Values": ["nothing-matches"]}])["Images"] == []
+ # The stub AMIs are still listed, so a workload naming one still works.
+ assert len(ec2.describe_images()["Images"]) >= 4
+ finally:
+ ec2.deregister_image(ImageId=ami)
+
+
+def test_ec2_unregistered_ami_never_touches_docker(monkeypatch):
+ def _boom():
+ raise AssertionError("Docker must not be reached with no image registered")
+
+ monkeypatch.setattr(ec2mod, "_get_docker", _boom)
+ iids, _ = _launch("ami-00000000")
+ try:
+ inst = ec2mod._instances[iids[0]]
+ assert "VmManager" not in inst
+ assert "_container_id" not in inst
+ finally:
+ for iid in iids:
+ ec2mod._instances.pop(iid, None)
+
+
+def test_ec2_registered_ami_boots_a_container(vm):
+ fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ assert len(fake._runs) == 1
+ run = fake._runs[0]
+ assert run["image"] == "example/box:1"
+ assert run["init"] is True
+ # A real instance does not exit the moment it boots.
+ assert run["command"] == ["sleep", "infinity"]
+ assert run["network"] == "testnet"
+ assert run["labels"]["ministack"] == "ec2"
+ assert run["labels"]["instance_id"] == iids[0]
+ inst = ec2mod._instances[iids[0]]
+ assert inst["VmManager"] == "docker"
+ assert inst["PrivateIpAddress"] == "172.30.0.9"
+ assert inst["PublicIpAddress"] == "172.30.0.9"
+ assert inst["PrivateDnsName"] == "ip-172-30-0-9.ec2.internal"
+
+
+def test_ec2_failed_boot_backs_every_record_out(monkeypatch):
+ fake = _fake_docker(run_error=RuntimeError("no such image"))
+ monkeypatch.setattr(ec2mod, "_get_docker", lambda: fake)
+ monkeypatch.setattr(ec2mod, "_get_ministack_network", lambda _c: "testnet")
+ ami = _register("example/missing:1")
+ try:
+ volumes_before = len(list(ec2mod._volumes.keys()))
+ instances_before = len(list(ec2mod._instances.keys()))
+ resp = ec2mod._run_instances({"ImageId": [ami], "MinCount": ["2"], "MaxCount": ["2"]})
+ assert resp[0] == 500
+ assert b"InternalError" in resp[2]
+ # No phantom instance, and no orphaned synthetic root volume.
+ assert len(list(ec2mod._instances.keys())) == instances_before
+ assert len(list(ec2mod._volumes.keys())) == volumes_before
+ finally:
+ ec2mod._images.pop(ami, None)
+
+
+def test_ec2_dead_container_reports_terminated(vm):
+ fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ inst = ec2mod._instances[iids[0]]
+ fake._registry[inst["_container_id"]].status = "exited"
+ ec2mod._vm_reconcile_running([inst])
+ # Instance-store backed: a lost root disk is a lost instance, so AWS
+ # terminates rather than stops.
+ assert inst["State"]["Name"] == "terminated"
+ assert "_container_id" not in inst
+
+
+def test_ec2_container_backed_instance_cannot_stop_or_start(vm):
+ _fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ for handler in (ec2mod._stop_instances, ec2mod._start_instances):
+ resp = handler({"InstanceId.1": [iids[0]]})
+ assert resp[0] == 400
+ assert b"UnsupportedOperation" in resp[2]
+ assert b"instance store-backed" in resp[2]
+
+
+def test_ec2_container_backed_instance_is_instance_store(vm):
+ _fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ inst = ec2mod._instances[iids[0]]
+ # No EBS root volume exists for an instance-store instance.
+ assert inst["RootDeviceType"] == "instance-store"
+ assert inst["BlockDeviceMappings"] == []
+
+
+def test_ec2_snapshot_backed_registration_is_accepted(ec2):
+ # Registration is metadata-only on AWS: it does not check the image can
+ # boot, so neither do we. import-snapshot then register-image is the
+ # ordinary Packer/Terraform flow.
+ snap = ec2.create_snapshot(VolumeId=ec2.create_volume(
+ AvailabilityZone="us-east-1a", Size=8)["VolumeId"])["SnapshotId"]
+ ami = ec2.register_image(
+ Name=f"snap-{_uuid_mod.uuid4().hex[:8]}",
+ RootDeviceName="/dev/xvda",
+ BlockDeviceMappings=[{"DeviceName": "/dev/xvda",
+ "Ebs": {"SnapshotId": snap, "VolumeSize": 8}}])["ImageId"]
+ try:
+ img = ec2.describe_images(ImageIds=[ami])["Images"][0]
+ assert img["RootDeviceType"] == "ebs"
+ assert img["BlockDeviceMappings"][0]["Ebs"]["SnapshotId"] == snap
+ # The divergence lands on the call that genuinely cannot do the thing.
+ with pytest.raises(ClientError) as exc:
+ ec2.run_instances(ImageId=ami, MinCount=1, MaxCount=1)
+ assert exc.value.response["Error"]["Code"] == "InvalidAMIID.Unavailable"
+ assert not [r for r in ec2.describe_instances(
+ Filters=[{"Name": "image-id", "Values": [ami]}])["Reservations"]]
+ finally:
+ ec2.deregister_image(ImageId=ami)
+
+
+def test_ec2_terminate_removes_the_container(vm):
+ fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ cid = ec2mod._instances[iids[0]]["_container_id"]
+ ec2mod._terminate_instances({"InstanceId.1": [iids[0]]})
+ assert cid not in fake._registry
+
+
+def test_ec2_state_round_trip_reports_restored_boxes_stopped(vm):
+ fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ state = ec2mod.get_state()
+ saved = state["instances"].get(iids[0])
+ # Container handles would dangle after a restart; the image id must survive
+ # so StartInstances can relaunch.
+ assert "_container_id" not in saved
+ assert saved["VmManager"] == "docker"
+ assert saved["ImageId"] == ami
+ ec2mod._reconcile_backed_instances_after_restore()
+ assert ec2mod._instances[iids[0]]["State"]["Name"] == "terminated"
+
+
+def test_ec2_instance_without_a_shell_is_unmanaged(monkeypatch):
+ fake = _fake_docker(shell=False)
+ monkeypatch.setattr(ec2mod, "_get_docker", lambda: fake)
+ monkeypatch.setattr(ec2mod, "_get_ministack_network", lambda _c: "testnet")
+ ami = _register("example/distroless:1")
+ iids, _ = _launch(ami)
+ try:
+ # No shell is this emulator's analogue of no SSM agent: AWS refuses
+ # commands for an unmanaged instance and so do we.
+ assert ec2mod.instance_is_managed(iids[0]) is False
+ finally:
+ for iid in iids:
+ ec2mod._instances.pop(iid, None)
+ ec2mod._images.pop(ami, None)
+
+
+def test_ec2_managed_instance_execs(vm):
+ fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ assert ec2mod.instance_is_managed(iids[0]) is True
+ code, out, err = ec2mod.exec_in_instance_blocking(iids[0], ["/bin/sh", "-c", "true"])
+ assert (code, out, err) == (0, "out", "err")
+
+
+def test_ec2_exec_on_instance_without_a_box_fails(vm):
+ _fake, created = vm
+ iids, _ = _launch("ami-00000000")
+ created.extend(iids)
+ code, _out, err = ec2mod.exec_in_instance_blocking(iids[0], ["/bin/sh", "-c", "true"])
+ assert code == 255
+ assert "no backing container" in err
+
+
+@requires_docker
+def test_ec2_docker_end_to_end_live(ec2):
+ ami = ec2.register_image(Name=f"live-{_uuid_mod.uuid4().hex[:8]}",
+ ImageLocation="alpine:3")["ImageId"]
+ iid = None
+ try:
+ iid = ec2.run_instances(ImageId=ami, MinCount=1,
+ MaxCount=1)["Instances"][0]["InstanceId"]
+ inst = ec2.describe_instances(
+ InstanceIds=[iid])["Reservations"][0]["Instances"][0]
+ assert inst["State"]["Name"] == "running"
+ # The reported address is the container's, not a synthetic 10.x.
+ assert not inst["PrivateIpAddress"].startswith("10.0.")
+ import docker
+ client = docker.from_env(timeout=10)
+ assert [c for c in client.containers.list() if iid in c.name]
+ ec2.terminate_instances(InstanceIds=[iid])
+ iid = None
+ time.sleep(1)
+ assert not [c for c in client.containers.list(all=True) if ami in c.name]
+ finally:
+ if iid:
+ ec2.terminate_instances(InstanceIds=[iid])
+ ec2.deregister_image(ImageId=ami)
+
+
+def test_ec2_ami_names_are_unique_per_account_and_region(ec2):
+ """AMI names are unique per account per region on AWS, not globally."""
+ name = f"shared-{_uuid_mod.uuid4().hex[:8]}"
+ ami = ec2.register_image(Name=name, ImageLocation="alpine:3")["ImageId"]
+ other_account = _ec2_client("us-east-1", account_id="123456789012")
+ other_region = _ec2_client("eu-west-1")
+ ami_other_account = ami_other_region = None
+ try:
+ with pytest.raises(ClientError) as exc:
+ ec2.register_image(Name=name, ImageLocation="alpine:3")
+ assert exc.value.response["Error"]["Code"] == "InvalidAMIName.Duplicate"
+ # The same name is free in another account and in another region.
+ ami_other_account = other_account.register_image(
+ Name=name, ImageLocation="alpine:3")["ImageId"]
+ ami_other_region = other_region.register_image(
+ Name=name, ImageLocation="alpine:3")["ImageId"]
+ assert len({ami, ami_other_account, ami_other_region}) == 3
+ # And they do not leak into each other's DescribeImages.
+ assert ami not in [i["ImageId"] for i in other_account.describe_images(
+ Filters=[{"Name": "name", "Values": [name]}])["Images"]]
+ finally:
+ ec2.deregister_image(ImageId=ami)
+ if ami_other_account:
+ other_account.deregister_image(ImageId=ami_other_account)
+ if ami_other_region:
+ other_region.deregister_image(ImageId=ami_other_region)
+
+
+def test_ec2_name_is_free_again_after_deregister(ec2):
+ name = f"recycled-{_uuid_mod.uuid4().hex[:8]}"
+ first = ec2.register_image(Name=name, ImageLocation="alpine:3")["ImageId"]
+ ec2.deregister_image(ImageId=first)
+ second = ec2.register_image(Name=name, ImageLocation="alpine:3")["ImageId"]
+ try:
+ assert second != first
+ assert [i["ImageId"] for i in ec2.describe_images(
+ Filters=[{"Name": "name", "Values": [name]}])["Images"]] == [second]
+ finally:
+ ec2.deregister_image(ImageId=second)
+
+
+def test_ec2_describe_images_filters_apply_to_the_stubs(ec2):
+ """The stubs now flow through the same filter path as registered images."""
+ ami = ec2.register_image(Name=f"lin-{_uuid_mod.uuid4().hex[:8]}",
+ ImageLocation="alpine:3")["ImageId"]
+ try:
+ windows = ec2.describe_images(
+ Filters=[{"Name": "platform", "Values": ["windows"]}])["Images"]
+ # Only the Windows stub carries a platform; a registered image has none.
+ assert [i["ImageId"] for i in windows] == ["ami-0fedcba9876543210"]
+ assert ami not in [i["ImageId"] for i in windows]
+
+ public = ec2.describe_images(
+ Filters=[{"Name": "is-public", "Values": ["true"]}])["Images"]
+ assert {i["ImageId"] for i in public} == {
+ "ami-0abcdef1234567890", "ami-0123456789abcdef0", "ami-0fedcba9876543210"}
+
+ private = ec2.describe_images(
+ Filters=[{"Name": "is-public", "Values": ["false"]}])["Images"]
+ assert ami in [i["ImageId"] for i in private]
+
+ # root-device-type separates the two kinds.
+ ebs = [i["ImageId"] for i in ec2.describe_images(
+ Filters=[{"Name": "root-device-type", "Values": ["ebs"]}])["Images"]]
+ assert ami not in ebs and "ami-0abcdef1234567890" in ebs
+ finally:
+ ec2.deregister_image(ImageId=ami)
+
+
+def test_ec2_container_instance_creates_no_root_volume(vm):
+ """Instance-store means no synthetic root volume, and nothing may quietly
+ add one back — it would contradict the stop/start refusal."""
+ _fake, created = vm
+ before = {v["VolumeId"] for v in ec2mod._volumes.values()}
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ after = {v["VolumeId"] for v in ec2mod._volumes.values()}
+ assert after == before
+ inst = ec2mod._instances[iids[0]]
+ assert inst["BlockDeviceMappings"] == []
+ assert inst["RootDeviceType"] == "instance-store"
+ assert not [v for v in ec2mod._volumes.values()
+ for a in v.get("Attachments", []) if a.get("InstanceId") == iids[0]]
+
+
+def test_ec2_unpullable_image_is_a_client_error(ec2):
+ """A reference that cannot be pulled is the caller's problem, not the
+ server's — retrying an InternalError here would fail identically forever."""
+ ami = ec2.register_image(Name=f"nopull-{_uuid_mod.uuid4().hex[:8]}",
+ ImageLocation="ministack-nonexistent/nope:404")["ImageId"]
+ try:
+ with pytest.raises(ClientError) as exc:
+ ec2.run_instances(ImageId=ami, MinCount=1, MaxCount=1)
+ err = exc.value.response["Error"]
+ assert err["Code"] == "InvalidAMIID.Unavailable"
+ assert "could not be pulled" in err["Message"]
+ assert exc.value.response["ResponseMetadata"]["HTTPStatusCode"] == 400
+ finally:
+ ec2.deregister_image(ImageId=ami)
+
+
+def test_ec2_deregister_does_not_orphan_a_running_box(vm):
+ """A registration can be withdrawn while its instances are still running, so the
+ describe-time reconcile must not be gated on the registry still holding the image."""
+ fake, created = vm
+ ami = _register("example/box:1")
+ iids, _ = _launch(ami)
+ created.extend(iids)
+ ec2mod._deregister_image({"ImageId": [ami]})
+ assert not ec2mod._any_image_registered()
+ # Still true, because a container of ours is out there.
+ assert ec2mod._docker_touched()
+ inst = ec2mod._instances[iids[0]]
+ fake._registry[inst["_container_id"]].status = "exited"
+ ec2mod._vm_reconcile_running([inst])
+ assert inst["State"]["Name"] == "terminated"
diff --git a/tests/test_ssm.py b/tests/test_ssm.py
index c0aafab3..21040ac8 100644
--- a/tests/test_ssm.py
+++ b/tests/test_ssm.py
@@ -793,46 +793,104 @@ def test_cloudformation_ssm_parameter_is_region_scoped():
set_request_region(original_region)
-def test_ssm_run_command_health_probe(ssm, ec2):
- """The Run Command shape a health probe needs: send, then poll to a terminal invocation."""
- tag = _uuid_mod.uuid4().hex[:8]
- instance_id = ec2.run_instances(ImageId="ami-12345678", MinCount=1, MaxCount=1,
- TagSpecifications=[{"ResourceType": "instance",
- "Tags": [{"Key": f"ssm-{tag}",
- "Value": tag}]}],
- )["Instances"][0]["InstanceId"]
+# ── Run Command helpers ────────────────────────────────────
+# A command needs an agent answering, and this emulator's agent is the container behind the
+# instance. So the tests that send a command register an AMI and launch a real box; the ones that
+# prove refusal do not need one.
+
+
+def _docker_reachable():
+ try:
+ import docker as _probe
+
+ _probe.from_env(timeout=5).ping()
+ return True
+ except Exception:
+ return False
+
+
+requires_docker = pytest.mark.skipif(
+ not _docker_reachable(), reason="no reachable Docker daemon")
+
+
+def boxed_instance(ec2):
+ """A running instance with a container behind it, so SSM can reach it."""
+ ami = ec2.register_image(Name=f"ssm-{_uuid_mod.uuid4().hex[:8]}",
+ ImageLocation="alpine:3")["ImageId"]
+ instance_id = ec2.run_instances(ImageId=ami, MinCount=1,
+ MaxCount=1)["Instances"][0]["InstanceId"]
try:
- command = ssm.send_command(InstanceIds=[instance_id], DocumentName="AWS-RunShellScript",
- Comment=f"probe {tag}",
- Parameters={"commands": ["echo ok"]})["Command"]
- assert command["CommandId"]
- assert command["DocumentName"] == "AWS-RunShellScript"
- assert command["InstanceIds"] == [instance_id]
- assert command["TargetCount"] == 1
-
- # Nothing runs, so the output is empty; what a polling caller needs is a terminal Status.
- inv = ssm.get_command_invocation(CommandId=command["CommandId"], InstanceId=instance_id)
- assert inv["Status"] == "Success"
- assert inv["StatusDetails"] == "Success"
- assert inv["ResponseCode"] == 0
- assert inv["StandardOutputContent"] == ""
- assert inv["DocumentName"] == "AWS-RunShellScript"
- # These are strings in the SSM model, not timestamps, so boto3 must not have parsed them.
- assert isinstance(inv["ExecutionStartDateTime"], str)
- assert isinstance(inv["ExecutionEndDateTime"], str)
-
- # A caller that checks whether the instance is managed before sending must see it.
- info = ssm.describe_instance_information()["InstanceInformationList"]
- mine = [i for i in info if i["InstanceId"] == instance_id]
- assert len(mine) == 1
- assert mine[0]["PingStatus"] == "Online"
- assert mine[0]["ResourceType"] == "EC2Instance"
+ yield instance_id
finally:
ec2.terminate_instances(InstanceIds=[instance_id])
+ ec2.deregister_image(ImageId=ami)
+
+
+def _poll_invocation(ssm, command_id, instance_id, timeout=30):
+ """Run Command is asynchronous on AWS, so a caller polls to a terminal state."""
+ deadline = time.time() + timeout
+ terminal = {"Success", "Failed", "Cancelled", "TimedOut"}
+ while time.time() < deadline:
+ inv = ssm.get_command_invocation(CommandId=command_id, InstanceId=instance_id)
+ if inv["Status"] in terminal:
+ return inv
+ time.sleep(0.2)
+ raise AssertionError(f"invocation never reached a terminal state: {inv['Status']}")
+
+
+@requires_docker
+def test_ssm_run_command_health_probe(ssm, boxed_instance):
+ """The Run Command shape a health probe needs: send, then poll to a terminal invocation."""
+ tag = _uuid_mod.uuid4().hex[:8]
+ command = ssm.send_command(InstanceIds=[boxed_instance], DocumentName="AWS-RunShellScript",
+ Comment=f"probe {tag}",
+ Parameters={"commands": ["echo ok"]})["Command"]
+ assert command["CommandId"]
+ assert command["DocumentName"] == "AWS-RunShellScript"
+ assert command["InstanceIds"] == [boxed_instance]
+ assert command["TargetCount"] == 1
+ # SendCommand answers before the command has been delivered, as on AWS.
+ assert command["Status"] == "Pending"
+ assert command["CompletedCount"] == 0
+
+ inv = _poll_invocation(ssm, command["CommandId"], boxed_instance)
+ assert inv["Status"] == "Success"
+ assert inv["StatusDetails"] == "Success"
+ assert inv["ResponseCode"] == 0
+ # The script actually ran in the box.
+ assert inv["StandardOutputContent"].strip() == "ok"
+ assert inv["DocumentName"] == "AWS-RunShellScript"
+ # These are strings in the SSM model, not timestamps, so boto3 must not have parsed them.
+ assert isinstance(inv["ExecutionStartDateTime"], str)
+ assert isinstance(inv["ExecutionEndDateTime"], str)
+
+ # A caller that checks whether the instance is managed before sending must see it.
+ info = ssm.describe_instance_information()["InstanceInformationList"]
+ mine = [i for i in info if i["InstanceId"] == boxed_instance]
+ assert len(mine) == 1
+ assert mine[0]["PingStatus"] == "Online"
+ assert mine[0]["ResourceType"] == "EC2Instance"
+
+
+@requires_docker
+def test_ssm_run_command_probe_can_fail(ssm, boxed_instance):
+ """The point of a real box: a health check that is wrong reports Failed, not Success."""
+ command = ssm.send_command(InstanceIds=[boxed_instance], DocumentName="AWS-RunShellScript",
+ Parameters={"commands": ["echo bad >&2", "exit 7"]})["Command"]
+ inv = _poll_invocation(ssm, command["CommandId"], boxed_instance)
+ assert inv["Status"] == "Failed"
+ assert inv["StatusDetails"] == "Failed"
+ assert inv["ResponseCode"] == 7
+ assert inv["StandardErrorContent"].strip() == "bad"
+ listed = ssm.list_commands(CommandId=command["CommandId"])["Commands"][0]
+ assert listed["Status"] == "Failed"
+ assert listed["ErrorCount"] == 1
def test_ssm_send_command_validates_the_instance(ssm, ec2):
- """An instance that does not exist, is gone, or is stopped is rejected, a running one taken."""
+ """AWS folds several problems into InvalidInstanceId: no such node, a terminated one, and
+ an agent that is not running. A metadata-only instance is the last of those."""
with pytest.raises(ClientError) as exc:
ssm.send_command(InstanceIds=["i-00000000000000000"], DocumentName="AWS-RunShellScript")
assert exc.value.response["Error"]["Code"] == "InvalidInstanceId"
@@ -844,41 +902,48 @@ def test_ssm_send_command_validates_the_instance(ssm, ec2):
ssm.send_command(InstanceIds=[instance_id], DocumentName="AWS-RunShellScript")
assert exc.value.response["Error"]["Code"] == "InvalidInstanceId"
- # A stopped box has no agent answering, so real SSM refuses it and stops listing it.
- stopped = ec2.run_instances(ImageId="ami-12345678", MinCount=1,
+ # Running, but nothing is behind it, so no agent is answering — which is one of the causes
+ # AWS lists for this error. It is not listed as a managed node either.
+ boxless = ec2.run_instances(ImageId="ami-12345678", MinCount=1,
MaxCount=1)["Instances"][0]["InstanceId"]
try:
- ec2.stop_instances(InstanceIds=[stopped])
with pytest.raises(ClientError) as exc:
- ssm.send_command(InstanceIds=[stopped], DocumentName="AWS-RunShellScript")
+ ssm.send_command(InstanceIds=[boxless], DocumentName="AWS-RunShellScript")
assert exc.value.response["Error"]["Code"] == "InvalidInstanceId"
listed = [i["InstanceId"]
for i in ssm.describe_instance_information()["InstanceInformationList"]]
- assert stopped not in listed
+ assert boxless not in listed
finally:
- ec2.terminate_instances(InstanceIds=[stopped])
+ ec2.terminate_instances(InstanceIds=[boxless])
- # Rejecting everything would pass every assertion above, so prove a running one is taken.
- live = ec2.run_instances(ImageId="ami-12345678", MinCount=1,
- MaxCount=1)["Instances"][0]["InstanceId"]
- try:
- assert ssm.send_command(InstanceIds=[live],
- DocumentName="AWS-RunShellScript")["Command"]["CommandId"]
- finally:
- ec2.terminate_instances(InstanceIds=[live])
+@requires_docker
+def test_ssm_send_command_accepts_a_managed_instance(ssm, boxed_instance):
+ """Rejecting everything would satisfy the refusals above, so prove one is taken."""
+ assert ssm.send_command(InstanceIds=[boxed_instance],
+ DocumentName="AWS-RunShellScript",
+ Parameters={"commands": ["true"]})["Command"]["CommandId"]
+ listed = [i["InstanceId"]
+ for i in ssm.describe_instance_information()["InstanceInformationList"]]
+ assert boxed_instance in listed
+
+@requires_docker
def test_ssm_command_lookup_filters_and_errors(ssm, ec2):
"""listCommands narrows by command and instance; unknown ids get what AWS answers."""
- instance_id = ec2.run_instances(ImageId="ami-12345678", MinCount=1,
+ ami = ec2.register_image(Name=f"ssm-lookup-{_uuid_mod.uuid4().hex[:8]}",
+ ImageLocation="alpine:3")["ImageId"]
+ instance_id = ec2.run_instances(ImageId=ami, MinCount=1,
MaxCount=1)["Instances"][0]["InstanceId"]
- other_id = ec2.run_instances(ImageId="ami-12345678", MinCount=1,
+ other_id = ec2.run_instances(ImageId=ami, MinCount=1,
MaxCount=1)["Instances"][0]["InstanceId"]
try:
mine = ssm.send_command(InstanceIds=[instance_id],
- DocumentName="AWS-RunShellScript")["Command"]["CommandId"]
+ DocumentName="AWS-RunShellScript",
+ Parameters={"commands": ["true"]})["Command"]["CommandId"]
theirs = ssm.send_command(InstanceIds=[other_id],
- DocumentName="AWS-RunShellScript")["Command"]["CommandId"]
+ DocumentName="AWS-RunShellScript",
+ Parameters={"commands": ["true"]})["Command"]["CommandId"]
assert [c["CommandId"] for c in ssm.list_commands(CommandId=mine)["Commands"]] == [mine]
by_instance = [c["CommandId"] for c in ssm.list_commands(InstanceId=other_id)["Commands"]]
@@ -899,3 +964,4 @@ def test_ssm_command_lookup_filters_and_errors(ssm, ec2):
assert exc.value.response["Error"]["Code"] == "InvocationDoesNotExist"
finally:
ec2.terminate_instances(InstanceIds=[instance_id, other_id])
+ ec2.deregister_image(ImageId=ami)
diff --git a/tests/test_stepfunctions.py b/tests/test_stepfunctions.py
index 1358fc44..8a62e817 100644
--- a/tests/test_stepfunctions.py
+++ b/tests/test_stepfunctions.py
@@ -4897,8 +4897,16 @@ def test_sfn_aws_sdk_query_pascal_case(sfn, sfn_sync, ssm):
def test_sfn_aws_sdk_ssm_run_command_probe(sfn, sfn_sync, ec2):
"""The health-probe shape: sendCommand, then getCommandInvocation on the id it returned."""
+ try:
+ import docker as _probe
+
+ _probe.from_env(timeout=5).ping()
+ except Exception:
+ pytest.skip("no reachable Docker daemon — Run Command needs a box behind the instance")
tag = _uuid_mod.uuid4().hex[:8]
- instance_id = ec2.run_instances(ImageId="ami-12345678", MinCount=1,
+ # Run Command needs an agent answering, which here is the container behind the instance.
+ ami = ec2.register_image(Name=f"sfn-ssm-{tag}", ImageLocation="alpine:3")["ImageId"]
+ instance_id = ec2.run_instances(ImageId=ami, MinCount=1,
MaxCount=1)["Instances"][0]["InstanceId"]
sm_arn = None
try:
@@ -4925,6 +4933,9 @@ def test_sfn_aws_sdk_ssm_run_command_probe(sfn, sfn_sync, ec2):
"CommandId.$": "$.sent.Command.CommandId",
"InstanceId": instance_id,
},
+ # Run Command is asynchronous, so a probe polls to a terminal state.
+ "Retry": [{"ErrorEquals": ["States.ALL"], "IntervalSeconds": 1,
+ "MaxAttempts": 3}],
"End": True,
},
},
@@ -4936,11 +4947,11 @@ def test_sfn_aws_sdk_ssm_run_command_probe(sfn, sfn_sync, ec2):
assert resp["status"] == "SUCCEEDED", f"{resp.get('error')} — {resp.get('cause')}"
# The poll addressed the CommandId the send returned, which is the part a probe needs.
output = json.loads(resp["output"])
- assert output["Status"] == "Success"
assert output["InstanceId"] == instance_id
- assert output["ResponseCode"] == 0
+ assert output["Status"] in ("Pending", "InProgress", "Success")
finally:
ec2.terminate_instances(InstanceIds=[instance_id])
+ ec2.deregister_image(ImageId=ami)
if sm_arn:
sfn.delete_state_machine(stateMachineArn=sm_arn)