Skip to content

Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.

fix(lambda): sign the local Node executor's SDK stub calls (#2354)

iot-rocket35782d1ec718
diff --git a/CHANGELOG.md b/CHANGELOG.md
index de680a4d..625dd288 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -14,6 +14,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).
- **CloudFront — VPC origins** — `CreateVpcOrigin`, `GetVpcOrigin`, `UpdateVpcOrigin`, `DeleteVpcOrigin` and `ListVpcOrigins` replace the empty `ListVpcOrigins` stub, so Terraform's `aws_cloudfront_vpc_origin` and a distribution's `vpc_origin_config` apply. As on AWS, a create, update or delete answers `Deploying`; MiniStack then finishes it at once, so the next read shows `Deployed` with the next `ETag`, or 404 after a delete. Names are unique per account, endpoints are not, and an origin a distribution names can't be deleted or changed. Validation, its order, the `TLSv1`/`TLSv1.1`/`TLSv1.2` default, `IpAddressType`, the list paging (no `Marker` or `IsTruncated`) and the IAM resources follow AWS captures. The endpoint has to be a load balancer or instance that exists in MiniStack, and viewer requests are not routed to it. Contributed by @kharabasz.
### Fixed
+- **Lambda — the local Node executor's SDK stubs sign their calls** — the `@aws-sdk/client-lambda` stub and the JSON-RPC `@aws-sdk/client-*` stubs sent their requests without credentials, so a call landed in the default account whatever account the function belongs to, and under `AUTH=true` it ran as the account root. They now sign with the function's credentials, as the real SDK does: with `AUTH` unset a call lands in the function's own account, under `AUTH=true` it runs as the execution role. Contributed by @iot-rocket.
- **IoT — `#` does not match the level above it** — a subscription or topic rule with a filter ending in `/#` (`a/c2/#`) also got messages published to the level above (`a/c2`), as the MQTT specification allows; AWS IoT Core delivers only `a/c2/` and deeper, and so does the broker now, for subscriptions and rules. A republish topic at that level is still refused, as on AWS. Contributed by @iot-rocket.
- **EventBridge — archive KMS keys** — `CreateArchive` and `UpdateArchive` dropped `KmsKeyIdentifier`, so `DescribeArchive` never returned it and Terraform re-planned `kms_key_identifier` on every run; the key is now stored and returned, an update that omits it keeps it, and an empty identifier on update clears it; `ListArchives` entries carry only the `Archive` summary fields. Contributed by @kharabasz.
- **WAF v2 — `CreateIPSet` and `CreateRuleGroup` refuse a duplicate name** — a second IP set or rule group with the name and scope of an existing one is now refused with `WAFDuplicateItemException` (`AWS WAF couldn’t perform the operation because some resource in your request is a duplicate of an existing one.`, also from `CreateWebACL` and `CreateRegexPatternSet`). Both stored the duplicate silently, so a client that retries a create after losing the response ended up with two. Contributed by @OS-kiranmalsetty.
diff --git a/ministack/core/lambda_runtime.py b/ministack/core/lambda_runtime.py
index 5eee9ddf..9efaf428 100644
--- a/ministack/core/lambda_runtime.py
+++ b/ministack/core/lambda_runtime.py
@@ -370,6 +370,38 @@ fs.write = function(fd, ...args) {
// a Lambda Layer with the actual SDK takes precedence; fall back to a stub
// that routes through AWS_ENDPOINT_URL (Ministack).
(function _installAwsSdkV3Stubs() {
+ // SigV4-sign a stub request with the function's credentials, as the real SDK does.
+ function _signedHeaders(service, method, ep, requestPath, body, headers) {
+ const crypto = require("crypto");
+ const accessKey = process.env.AWS_ACCESS_KEY_ID;
+ const secretKey = process.env.AWS_SECRET_ACCESS_KEY;
+ if (!accessKey || !secretKey) return headers;
+ const region = process.env.AWS_REGION || process.env.AWS_DEFAULT_REGION || "us-east-1";
+ const amzDate = new Date().toISOString().replace(/[-:]|\.\d{3}/g, "");
+ const scope = amzDate.slice(0, 8) + "/" + region + "/" + service + "/aws4_request";
+ const signed = Object.assign({}, headers, { host: ep.host, "x-amz-date": amzDate });
+ if (process.env.AWS_SESSION_TOKEN) signed["x-amz-security-token"] = process.env.AWS_SESSION_TOKEN;
+ const values = {};
+ for (const name of Object.keys(signed)) values[name.toLowerCase()] = String(signed[name]).trim();
+ const names = Object.keys(values).sort();
+ const encode = (v) => encodeURIComponent(v)
+ .replace(/[!'()*]/g, (c) => "%" + c.charCodeAt(0).toString(16).toUpperCase());
+ const [rawPath, rawQuery] = requestPath.split("?");
+ const query = new URLSearchParams(rawQuery || "");
+ const canonicalQuery = [...query.entries()].map(([k, v]) => encode(k) + "=" + encode(v)).sort().join("&");
+ const hash = (data) => crypto.createHash("sha256").update(data).digest("hex");
+ const hmac = (key, data) => crypto.createHmac("sha256", key).update(data).digest();
+ const canonical = [method, rawPath.split("/").map(encode).join("/"), canonicalQuery,
+ names.map((n) => n + ":" + values[n] + "\n").join(""), names.join(";"), hash(body || "")].join("\n");
+ let key = hmac("AWS4" + secretKey, amzDate.slice(0, 8));
+ for (const part of [region, service, "aws4_request"]) key = hmac(key, part);
+ const signature = crypto.createHmac("sha256", key)
+ .update(["AWS4-HMAC-SHA256", amzDate, scope, hash(canonical)].join("\n")).digest("hex");
+ signed["Authorization"] = "AWS4-HMAC-SHA256 Credential=" + accessKey + "/" + scope
+ + ", SignedHeaders=" + names.join(";") + ", Signature=" + signature;
+ return signed;
+ }
+
// ── Lambda stub (REST-based, not JSON-RPC) ─────────────────────────────
function _lambdaInvoke(params) {
const ep = new URL(process.env.AWS_ENDPOINT_URL || "http://127.0.0.1:4566");
@@ -387,14 +419,15 @@ fs.write = function(fd, ...args) {
if (process.env.__DEPTH_ENV__) {
headers["__DEPTH_HEADER__"] = process.env.__DEPTH_ENV__;
}
+ const requestPath = "/2015-03-31/functions/" + fn + "/invocations" + qs;
return new Promise((resolve, reject) => {
const req = (ep.protocol === "https:" ? https : http).request(
{
hostname: ep.hostname,
port: parseInt(ep.port || "4566", 10),
method: "POST",
- path: "/2015-03-31/functions/" + fn + "/invocations" + qs,
- headers: headers,
+ path: requestPath,
+ headers: _signedHeaders("lambda", "POST", ep, requestPath, body, headers),
},
(res) => {
const chunks = [];
@@ -657,7 +690,7 @@ fs.write = function(fd, ...args) {
"@aws-sdk/client-translate": "AWSShineFrontendService_20170701",
};
- function _jsonRpcRequest(targetPrefix, opName, params) {
+ function _jsonRpcRequest(targetPrefix, opName, params, service) {
const ep = new URL(process.env.AWS_ENDPOINT_URL || "http://127.0.0.1:4566");
const body = JSON.stringify(params || {});
return new Promise((resolve, reject) => {
@@ -667,11 +700,11 @@ fs.write = function(fd, ...args) {
port: parseInt(ep.port || "4566", 10),
method: "POST",
path: "/",
- headers: {
+ headers: _signedHeaders(service, "POST", ep, "/", body, {
"Content-Type": "application/x-amz-json-1.1",
"X-Amz-Target": targetPrefix + "." + opName,
"Content-Length": Buffer.byteLength(body),
- },
+ }),
},
(res) => {
const chunks = [];
@@ -700,12 +733,21 @@ fs.write = function(fd, ...args) {
});
}
- function _makeGenericJsonServiceModule(targetPrefix) {
+ // Signing names that differ from the package name.
+ const _SIGNING_NAMES = {
+ "sfn": "states", "cloudwatch-logs": "logs", "secrets-manager": "secretsmanager", "eventbridge": "events",
+ "dynamodb-streams": "dynamodb", "cognito-identity-provider": "cognito-idp", "emr": "elasticmapreduce",
+ "resource-groups-tagging-api": "tagging",
+ };
+
+ function _makeGenericJsonServiceModule(targetPrefix, id) {
+ const pkg = id.replace(/^@aws-sdk\/client-/, "");
+ const service = _SIGNING_NAMES[pkg] || pkg;
// Command class factory: new PutParameterCommand(params) → has _run()
function _cmdClass(opName) {
return class {
constructor(params) { this._params = params; }
- _run() { return _jsonRpcRequest(targetPrefix, opName, this._params); }
+ _run() { return _jsonRpcRequest(targetPrefix, opName, this._params, service); }
};
}
@@ -722,7 +764,7 @@ fs.write = function(fd, ...args) {
get(_, prop) {
if (typeof prop !== "string") return undefined;
const opName = prop[0].toUpperCase() + prop.slice(1);
- return (params) => _jsonRpcRequest(targetPrefix, opName, params);
+ return (params) => _jsonRpcRequest(targetPrefix, opName, params, service);
},
});
},
@@ -772,7 +814,7 @@ fs.write = function(fd, ...args) {
requireError = err;
}
const prefix = _JSON_RPC_TARGETS[id];
- if (prefix) return _makeGenericJsonServiceModule(prefix);
+ if (prefix) return _makeGenericJsonServiceModule(prefix, id);
if (
requireError.code !== "MODULE_NOT_FOUND"
|| !requireError.message.includes("'" + id + "'")
diff --git a/tests/test_lambda.py b/tests/test_lambda.py
index bf0498c1..6a9a9b5d 100644
--- a/tests/test_lambda.py
+++ b/tests/test_lambda.py
@@ -8861,6 +8861,108 @@ def test_nodejs_worker_aws_sdk_v3_stub_wire_roundtrip(lam, ssm):
pass
[email protected](os.environ.get("LAMBDA_EXECUTOR", "").lower() == "docker",
+ reason="the SDK stubs belong to the local Node executor")
+def test_nodejs_worker_aws_sdk_v3_stub_calls_the_functions_own_account():
+ """A stub call signs with the function's credentials, so it lands in the function's account."""
+ from conftest import make_client
+
+ if not shutil.which("node"):
+ pytest.skip("node not found on PATH")
+ account = "123456789012"
+ own = {name: boto3.client(name, endpoint_url=ENDPOINT, region_name="us-east-1", aws_access_key_id=account,
+ aws_secret_access_key="test") for name in ("lambda", "ssm", "iam")}
+ fname = f"sdk-account-{_uuid_mod.uuid4().hex[:8]}"
+ param_name = f"/ministack-test/{fname}"
+ code = (
+ "const { SSMClient, PutParameterCommand } = require('@aws-sdk/client-ssm');\n"
+ "exports.handler = async (event) => {\n"
+ " await new SSMClient({}).send(new PutParameterCommand({\n"
+ " Name: event.name, Value: 'v', Type: 'String', Overwrite: true }));\n"
+ " return { ok: true };\n"
+ "};\n"
+ )
+ role = own["iam"].create_role(RoleName=fname, AssumeRolePolicyDocument=json.dumps({
+ "Version": "2012-10-17", "Statement": [{
+ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]}))["Role"]
+ own["iam"].put_role_policy(RoleName=fname, PolicyName="ssm", PolicyDocument=json.dumps({
+ "Version": "2012-10-17", "Statement": [{"Effect": "Allow", "Action": "ssm:PutParameter", "Resource": "*"}]}))
+ own["lambda"].create_function(FunctionName=fname, Runtime="nodejs20.x", Role=role["Arn"],
+ Handler="index.handler", Code={"ZipFile": _make_zip_js(code, "index.js")})
+ try:
+ resp = own["lambda"].invoke(FunctionName=fname, Payload=json.dumps({"name": param_name}).encode())
+ assert json.loads(resp["Payload"].read()) == {"ok": True}
+ assert own["ssm"].get_parameter(Name=param_name)["Parameter"]["Value"] == "v"
+ with pytest.raises(ClientError):
+ make_client("ssm").get_parameter(Name=param_name)
+ finally:
+ own["lambda"].delete_function(FunctionName=fname)
+ with contextlib.suppress(ClientError):
+ own["ssm"].delete_parameter(Name=param_name)
+ own["iam"].delete_role_policy(RoleName=fname, PolicyName="ssm")
+ own["iam"].delete_role(RoleName=fname)
+
+
[email protected](os.environ.get("LAMBDA_EXECUTOR", "").lower() == "docker",
+ reason="the SDK stubs belong to the local Node executor")
+def test_nodejs_sdk_stub_calls_run_as_the_execution_role_contract(auth_enabled, lam):
+ """The Node SDK stubs sign with the function's credentials, so the execution role's policy decides."""
+ from conftest import make_client
+
+ if not shutil.which("node"):
+ pytest.skip("node not found on PATH")
+ iam, suffix = make_client("iam"), _uuid_mod.uuid4().hex[:8]
+ trust = json.dumps({"Version": "2012-10-17", "Statement": [{
+ "Effect": "Allow", "Principal": {"Service": "lambda.amazonaws.com"}, "Action": "sts:AssumeRole"}]})
+ roles = {}
+ for kind in ("allowed", "bare"):
+ roles[kind] = iam.create_role(RoleName=f"stub-{kind}-{suffix}", AssumeRolePolicyDocument=trust)["Role"]
+ iam.put_role_policy(RoleName=roles["allowed"]["RoleName"], PolicyName="calls", PolicyDocument=json.dumps({
+ "Version": "2012-10-17", "Statement": [
+ {"Effect": "Allow", "Action": ["lambda:InvokeFunction", "ssm:PutParameter"], "Resource": "*"}]}))
+ callee = f"stub-callee-{suffix}"
+ code = (
+ "const { LambdaClient, InvokeCommand } = require('@aws-sdk/client-lambda');\n"
+ "const { SSMClient, PutParameterCommand } = require('@aws-sdk/client-ssm');\n"
+ "exports.handler = async (event) => {\n"
+ " const r = await new LambdaClient({}).send(\n"
+ " new InvokeCommand({ FunctionName: event.target, Payload: '{}' }));\n"
+ " let ssm = 'ok';\n"
+ " try {\n"
+ " await new SSMClient({}).send(new PutParameterCommand({\n"
+ " Name: event.name, Value: 'v', Type: 'String', Overwrite: true }));\n"
+ " } catch (e) { ssm = e.name; }\n"
+ " return { status: r.StatusCode, ssm: ssm };\n"
+ "};\n"
+ )
+ callers = {kind: f"stub-{kind}-{suffix}" for kind in roles}
+ try:
+ lam.create_function(FunctionName=callee, Runtime="python3.12", Role=roles["allowed"]["Arn"],
+ Handler="index.handler",
+ Code={"ZipFile": _make_zip("def handler(event, context):\n return 1\n")})
+ answers = {}
+ for kind, name in callers.items():
+ lam.create_function(FunctionName=name, Runtime="nodejs20.x", Role=roles[kind]["Arn"],
+ Handler="index.handler", Code={"ZipFile": _make_zip_js(code, "index.js")})
+ resp = lam.invoke(FunctionName=name, Payload=json.dumps(
+ {"target": callee, "name": f"/stub-roles/{suffix}/{kind}"}).encode())
+ answers[kind] = json.loads(resp["Payload"].read())
+ assert answers["allowed"] == {"status": 200, "ssm": "ok"}
+ assert answers["bare"] == ({"status": 403, "ssm": "AccessDeniedException"} if auth_enabled
+ else {"status": 200, "ssm": "ok"})
+ finally:
+ for name in (callee, *callers.values()):
+ with contextlib.suppress(Exception):
+ lam.delete_function(FunctionName=name)
+ for kind in callers:
+ with contextlib.suppress(Exception):
+ make_client("ssm").delete_parameter(Name=f"/stub-roles/{suffix}/{kind}")
+ iam.delete_role_policy(RoleName=roles["allowed"]["RoleName"], PolicyName="calls")
+ for role in roles.values():
+ iam.delete_role(RoleName=role["RoleName"])
+
+
def test_lambda_ruby_4_0_runtime_maps_to_official_image():
"""Lambda Ruby 4.0 runtime support (botocore 1.42.94 added the runtime).
Maps to AWS's official Lambda Ruby 4.0 base image."""