Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
fix(lambda): let a Node custom resource reach the CloudFormation callback in the docker executor (#1679)
iot-rocketc5d77b213ca0
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex ca626a72..cf5865db 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -17,6 +17,11 @@ Versioning follows [Semantic Versioning](https://semver.org/).- **Transcribe — `StartTranscriptionJob` returns a job already `IN_PROGRESS`** — the start response reported `QUEUED` with no `StartTime`, and the job reached `IN_PROGRESS` only from the background worker, after the response had been sent. A caller that reads the status off the start response and then waits for the terminal `Transcribe Job State Change` event, rather than polling, therefore never saw an in-progress state at all: the job went straight from queued to complete. Jobs now start `IN_PROGRESS` with `StartTime` set, matching the response the AWS CLI walkthrough documents, and `TRANSCRIBE_JOB_RUN_SECONDS` covers the whole run rather than half of it. `QUEUED`, which AWS reaches only for a request that opted into job queueing through `JobExecutionSettings.AllowDeferredExecution` while the account is at its concurrent job limit, stays reachable through the new `TRANSCRIBE_JOB_QUEUE_SECONDS` (default `0`); both knobs are settable at runtime through `/_ministack/config`. Contributed by @ppettitau.+### Fixed+- **CloudFormation — the template and stack quotas are enforced** — a template of any size, with any number of resources, parameters, outputs or mappings, and a stack name of any shape were accepted, so a template that a real account refuses deployed. `CreateStack`, `UpdateStack`, `CreateChangeSet`, `ValidateTemplate` and `GetTemplateSummary` now refuse a `TemplateBody` over 51,200 bytes and a template behind `TemplateURL` over 1,000,000 bytes; the engine refuses more than 500 resources, 200 parameters, 200 outputs, 200 mappings or 200 attributes in a mapping, a name over 255 characters, a description over 1,024 bytes and a parameter default or value over 4,096 bytes; `CreateStack` and a `CREATE` change set refuse a stack name that is not alphanumeric-and-hyphens starting with a letter or longer than 128 characters, joined with an oversized body into the one message the API's parameter validation gives. All of it before a stack record exists, with the messages of the API's parameter validation and the template format errors. `ValidateTemplate` also takes `TemplateURL` now. Contributed by @iot-rocket.+- **Lambda — a Node custom resource signals its stack in the docker executor** — the response submitters the CDK bundles into its custom-resource handlers (`nodejs-entrypoint`, the provider framework, `AwsCustomResource`) build the `ResponseURL` PUT from the URL's hostname and path only and hand it to `https.request`, so under `LAMBDA_EXECUTOR=docker` the callback went out over TLS to port 443, nothing answered, and an `AwsCustomResource`-backed resource ran its API call and then hung its stack until `ServiceTimeout` (the local executor already downgraded https for the gateway). The shim the docker executor injects now turns https to the gateway hosts (the endpoint host, `host.docker.internal`, `localhost` and `MINISTACK_HOST`) on the https default port or the gateway port into plain http on the gateway port; other hosts, and those hosts on other ports, keep TLS. The local executor's own list gains `MINISTACK_HOST` as well. Contributed by @iot-rocket.+### Fixed+- **CloudFormation — `Capabilities` are enforced under `AUTH=true`** — `CreateStack`, `UpdateStack` and `CreateChangeSet` accepted a template with IAM resources or a `Transform` whatever the request acknowledged, so a deploy that CloudFormation refuses with `InsufficientCapabilitiesException` went through. With `AUTH=true` they now answer that error (HTTP 400, `Requires capabilities : [CAPABILITY_IAM]`, measured) and create nothing: an IAM resource needs `CAPABILITY_IAM` or `CAPABILITY_NAMED_IAM`, one with a custom name `CAPABILITY_NAMED_IAM`, a template with a `Transform` `CAPABILITY_AUTO_EXPAND` on `CreateStack` and `UpdateStack` (not on a change set, as the API documents). Without `AUTH` nothing changes; `GetTemplateSummary` keeps reporting the same set. Contributed by @iot-rocket.- **Lambda — a durable invocation runs on the warm pool** — durable Python and Node.js invocations were sent to the one-shot executor, because the three `AWS_LAMBDA_DURABLE_*` variables change on every call while a pooled worker's environment is fixed when it spawns. Each of them therefore paid a fresh interpreter start and the botocore import inside the function's `Timeout`, whose default is 3 seconds, so on a loaded machine every durable invocation ended as a timeout. The values now travel in the event, the channel the X-Ray trace header and the invoke-depth counter already use, and both worker bootstraps move them into the environment before the handler runs and drop them again when the next invocation is not durable. Contributed by @iot-rocket.## [1.5.10] — 2026-09-10diff --git a/ministack/core/lambda_runtime.py b/ministack/core/lambda_runtime.pyindex 45c24d29..ac57f94d 100644--- a/ministack/core/lambda_runtime.py+++ b/ministack/core/lambda_runtime.py@@ -838,9 +838,12 @@ function patchAwsSdk() {// cfn-response.js calls https.request unconditionally for the ResponseURL// PUT, and also drops the port when constructing options. Intercept here// so the PUT reaches Ministack's HTTP server on msPort, not port 443.- if (host === "127.0.0.1" || host === "localhost" || host === msHost) {+ const advertised = (process.env.MINISTACK_HOST || "").split(":")[0];+ if (host === "127.0.0.1" || host === "localhost" || host === msHost+ || (advertised && host === advertised)) {options.protocol = "http:";- options.port = options.port || msPort;+ // The https default port means the gateway port here, as in the container shim.+ options.port = options.port && String(options.port) !== "443" ? options.port : msPort;options.host = host + ":" + options.port;options.agent = new http.Agent({ keepAlive: true });delete options._defaultAgent;diff --git a/ministack/services/lambda_svc.py b/ministack/services/lambda_svc.pyindex dd7b79df..1d27f4fe 100644--- a/ministack/services/lambda_svc.py+++ b/ministack/services/lambda_svc.py@@ -4070,18 +4070,77 @@ def handler(event, context):'''_JS_CTX_ARN_SHIM = '''\-// MiniStack shim: hand user code the control-plane ARN in its context.+// MiniStack shim: hand user code the control-plane ARN in its context, and+// reach the gateway over plain HTTP when a library insists on HTTPS.const path = require("path");const fs = require("fs");+const http = require("http");+const https = require("https");const REAL = process.env._MS_REAL_HANDLER || "index.handler";const ARN = process.env._LAMBDA_FUNCTION_ARN || "";+const TASK_ROOT = process.env.LAMBDA_TASK_ROOT || "/var/task";+// The container talks to MiniStack over http://<gateway host>:<port>, but the+// response submitters the CDK bundles into its custom-resource handlers+// (nodejs-entrypoint, the provider framework, AwsCustomResource) build the+// ResponseURL PUT from the URL's hostname and path only and hand it to+// https.request, so it goes out over TLS to port 443 whatever the URL says,+// and a Node custom resource never signalled its stack. Downgrade https to+// http for the gateway hosts only; the https default 443 becomes the gateway+// port, any other explicit port is kept.+try {+ const EP = new URL(process.env.AWS_ENDPOINT_URL || "http://host.docker.internal:4566");+ const EP_PORT = EP.port || (EP.protocol === "https:" ? "443" : "80");+ const PLAIN_HOSTS = new Set(+ [EP.hostname, "localhost", "127.0.0.1", "host.docker.internal"]+ .concat((process.env._MS_GATEWAY_HOSTS || "").split(","))+ .filter(Boolean));+ const origHttpsRequest = https.request;+ https.request = function (input, options, callback) {+ if (typeof options === "function") { callback = options; options = undefined; }+ let opts;+ if (typeof input === "string" || input instanceof URL) {+ const u = new URL(String(input));+ opts = Object.assign({ hostname: u.hostname, port: u.port, path: u.pathname + u.search,+ protocol: u.protocol }, options || {});+ } else {+ opts = Object.assign({}, input, options || {});+ }+ const host = opts.hostname || String(opts.host || "").split(":")[0];+ const rawPort = opts.port ? String(opts.port) : "";+ // Only the gateway hosts, and only the https default port or the gateway+ // port itself: a handler that dials its own TLS sidecar on another port+ // of localhost keeps TLS.+ if (!PLAIN_HOSTS.has(host) || (rawPort && rawPort !== "443" && rawPort !== EP_PORT)) {+ return origHttpsRequest.call(https, input, options, callback);+ }+ opts.protocol = "http:";+ opts.hostname = host;+ opts.host = host + ":" + EP_PORT;+ opts.port = EP_PORT;+ // Node's default http agent (keep-alive with an idle timeout) replaces+ // whatever https agent the caller set.+ opts.agent = undefined;+ delete opts._defaultAgent;+ return http.request(opts, callback);+ };+ https.get = function (input, options, callback) {+ const req = https.request(input, options, callback);+ req.end();+ return req;+ };+ // A handler that did `import { request } from "node:https"` holds a live+ // binding that only refreshes on request; refresh it now.+ require("module").syncBuiltinESMExports();+} catch (e) {+ // A bad AWS_ENDPOINT_URL costs the downgrade, not the function.+}const dot = REAL.lastIndexOf(".");const modPart = REAL.slice(0, dot);const fnName = REAL.slice(dot + 1);let cached = null;async function load() {if (cached) return cached;- const base = path.join("/var/task", modPart);+ const base = path.join(TASK_ROOT, modPart);for (const ext of [".mjs", ".js", ".cjs"]) {const p = base + ext;if (fs.existsSync(p)) {@@ -4132,8 +4191,12 @@ def _write_context_arn_shim(code_dir: str, runtime: str, handler: str) -> str |passreturn Nonetry:- with open(shim_path, "w") as f:+ # Another container of the same code may already hold the dir as a+ # read-only mount; a rename lands the file whole rather than truncated.+ tmp_path = f"{shim_path}.{os.getpid()}.tmp"+ with open(tmp_path, "w") as f:f.write(source)+ os.replace(tmp_path, shim_path)except OSError as exc:logger.warning("Lambda context-ARN shim not written (%s); ""container will report the RIE default ARN", exc)@@ -4261,6 +4324,11 @@ def _spawn_lambda_container_impl(config: dict, code_zip: bytes | None,# Rewrite localhost/127.0.0.1 → host.docker.internal for container accessendpoint = _rewrite_host_for_container(endpoint)container_env["AWS_ENDPOINT_URL"] = endpoint+ # The host MiniStack advertises for itself (custom-resource ResponseURLs+ # carry it); the Node shim downgrades https to http for it.+ advertised_host = os.environ.get("MINISTACK_HOST", "").split(":")[0]+ if advertised_host:+ container_env["_MS_GATEWAY_HOSTS"] = advertised_host# Mounts (Zip only — Image bakes code in). Layers are NEVER bind-mounted:# AWS merges every layer's contents into /opt (so /opt/python, /opt/lib,diff --git a/tests/test_lambda.py b/tests/test_lambda.pyindex 26a00484..7b8e2c02 100644--- a/tests/test_lambda.py+++ b/tests/test_lambda.py@@ -3,6 +3,8 @@ import contextlibimport ioimport jsonimport os+import shutil+import sysimport timeimport urllib.error as _urlerrimport urllib.request as _urlreq@@ -11476,6 +11478,110 @@ def test_context_arn_shim_survives_a_cached_code_dir(tmp_path):assert lsvc._write_context_arn_shim(str(code_dir), "nodejs20.x", "index.handler") is None[email protected](not shutil.which("node"), reason="node not installed")+def test_node_context_shim_downgrades_https_to_the_gateway(tmp_path):+ """The response submitters the CDK bundles into its custom-resource+ handlers build the ResponseURL PUT from the URL's hostname and path only+ and hand it to https.request, so it goes out over TLS to port 443+ whatever the URL says, and in the docker executor a Node custom resource+ never signalled its stack. The shim the executor injects turns that into a plain+ HTTP request on the gateway port for the gateway hosts, and only those."""+ import http.server+ import subprocess+ import threading++ seen = []++ class Handler(http.server.BaseHTTPRequestHandler):+ def do_PUT(self):+ seen.append((self.command, self.path, self.headers.get("Host")))+ self.send_response(200)+ self.end_headers()++ do_GET = do_PUT++ def log_message(self, *args):+ pass++ # The advertised-host case below dials a second loopback address the shim+ # does not know by default; Linux answers on all of 127/8, so bind wide+ # there and stay on 127.0.0.1 elsewhere.+ on_linux = sys.platform.startswith("linux")+ server = http.server.HTTPServer(("0.0.0.0" if on_linux else "127.0.0.1", 0), Handler)+ port = server.server_address[1]+ threading.Thread(target=server.serve_forever, daemon=True).start()+ try:+ code_dir = tmp_path / "task"+ code_dir.mkdir()+ (code_dir / "index.js").write_text(+ "const https = require('https');\n"+ "exports.handler = (event) => new Promise((resolve, reject) => {\n"+ " const done = (res) => resolve({status: res.statusCode});\n"+ " let req;\n"+ " if (event.form === 'string') {\n"+ " req = https.request('https://' + event.host + '/_ministack/cfn-response/tok', {method: 'PUT'}, done);\n"+ " } else if (event.form === 'get') {\n"+ " req = https.get({hostname: event.host, path: '/_ministack/cfn-response/tok'}, done);\n"+ " } else {\n"+ " const opts = {hostname: event.host, path: '/_ministack/cfn-response/tok', method: 'PUT'};\n"+ " if (event.port) opts.port = event.port;\n"+ " req = https.request(opts, done);\n"+ " }\n"+ " req.on('error', reject);\n"+ " if (event.form !== 'get') req.end('{}');\n"+ "});\n"+ )+ assert lsvc._write_context_arn_shim(str(code_dir), "nodejs20.x", "index.handler") == "_msctx_shim.handler"+ env = {+ **os.environ,+ "LAMBDA_TASK_ROOT": str(code_dir),+ "_MS_REAL_HANDLER": "index.handler",+ "AWS_ENDPOINT_URL": f"http://127.0.0.1:{port}",+ "_MS_GATEWAY_HOSTS": "127.0.0.2",+ }+ script = (+ "const s = require(process.argv[1]);"+ "const event = {host: process.argv[2], port: process.argv[3] ? Number(process.argv[3]) : undefined,"+ " form: process.argv[4] || 'options'};"+ "s.handler(event, {}).then("+ " (r) => process.stdout.write(JSON.stringify(r)),"+ " (e) => process.stdout.write(JSON.stringify({error: e.code || String(e)})));"+ )++ def run(host, hport=None, form="options"):+ proc = subprocess.run(+ ["node", "-e", script, str(code_dir / "_msctx_shim.js"), host, str(hport or ""), form],+ env=env, capture_output=True, text=True, timeout=30,+ )+ assert proc.returncode == 0, proc.stderr+ return json.loads(proc.stdout)++ # The CDK shape: hostname and path only, https default port. The gateway+ # host becomes http on the gateway port.+ assert run("127.0.0.1") == {"status": 200}+ assert seen[-1] == ("PUT", "/_ministack/cfn-response/tok", f"127.0.0.1:{port}")+ # An explicit 443 is the same case; so is the advertised gateway host+ # the container learns through _MS_GATEWAY_HOSTS.+ assert run("127.0.0.1", 443) == {"status": 200}+ if on_linux:+ assert run("127.0.0.2") == {"status": 200}+ assert seen[-1] == ("PUT", "/_ministack/cfn-response/tok", f"127.0.0.2:{port}")+ # The string-URL and https.get forms take the same path.+ assert run("127.0.0.1", form="string") == {"status": 200}+ assert seen[-1][0] == "PUT"+ assert run("127.0.0.1", form="get") == {"status": 200}+ assert seen[-1][0] == "GET"+ # An explicit other port on a gateway host keeps TLS: the request is+ # not downgraded and finds no TLS listener there.+ assert "error" in run("127.0.0.1", port + 1)+ assert seen[-1][2] != f"127.0.0.1:{port + 1}"+ # Every other host keeps real TLS on 443 (here: nothing listens).+ assert "error" in run("localhost.invalid")+ finally:+ server.shutdown()+ server.server_close()++def test_extract_cache_sweep_is_reference_based(monkeypatch, tmp_path):"""The sweep drops extracted trees whose blob no longer backs anyfunction, version, or layer version — and only those. References are read