Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
fix(lambda): records that share a code blob share its bytes on restore (#2415)
Jeff Heller5f1dd2f1811a
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex f3e75f75..baf77919 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,7 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]### Fixed+- **Lambda — restoring persisted state shares code between versions** — each function version got its own copy of its zip at boot, although versions share one blob file, so a state with many versions took gigabytes before the first request. Records that share a blob now share its bytes. Contributed by @skialpine.- **Lambda — a package that is not a zip is refused** — `CreateFunction`, `UpdateFunctionCode` and `PublishLayerVersion` accepted a package that is not a zip (an empty file included) and answered success, `UpdateFunctionCode` keeping the previous code. They now answer `InvalidParameterValueException` `Could not unzip uploaded file. Please check your file, then try to upload again.` Contributed by @skialpine.- **DynamoDB — `TransactWriteItems` follows DynamoDB's check order** — an `Update` whose `ConditionExpression` failed was cancelled with `ValidationError` when its `UpdateExpression` could not be applied to the item (for example `SET #c = #c + :one` on a missing item); it now cancels with `ConditionalCheckFailed`, as `UpdateItem` answers `ConditionalCheckFailedException` for the same request. Such an update-expression error on one member no longer hides a failed condition on another: both reasons are returned, while wrong-typed keys and index keys still cancel before any condition. A missing table answers `ResourceNotFoundException` even when another member has a `ValidationError`, and a `begins_with` condition with a number operand answers `ValidationException` instead of an internal error, as on AWS. Contributed by @jun-ut.- **ALB — instance and IP targets receive the path as the client sent it** — the data plane forwarded the decoded path, so `%20` became a space and the request was refused with 502 (`URL can't contain control characters`), and `%2F` became `/`. The path now reaches the target with its percent-encoding, as AWS sends it; rules still match the decoded path. Contributed by @skialpine.diff --git a/ministack/services/lambda_svc.py b/ministack/services/lambda_svc.pyindex 61172ca1..4e521376 100644--- a/ministack/services/lambda_svc.py+++ b/ministack/services/lambda_svc.py@@ -599,13 +599,17 @@ def _externalize_code_zip(func: dict, referenced: set[str]) -> None:# without going through CreateFunction (no real code path does this).-def _internalize_code_zip(func: dict) -> None:- """Replace a blob ref or legacy base64 string with bytes, in place."""+def _internalize_code_zip(func: dict, blobs: dict[str, bytes | None]) -> None:+ """Replace a blob ref or legacy base64 string with bytes, in place. ``blobs``+ holds what this restore has read, so records sharing a blob share its bytes."""code = func.get("code_zip")if not code:returnif isinstance(code, dict) and _BLOB_REF_KEY in code:- func["code_zip"] = _read_code_blob(code[_BLOB_REF_KEY])+ sha = code[_BLOB_REF_KEY]+ if sha not in blobs:+ blobs[sha] = _read_code_blob(sha)+ func["code_zip"] = blobs[sha]elif isinstance(code, str):# Legacy persistence shape (inline base64).func["code_zip"] = base64.b64decode(code)@@ -653,27 +657,28 @@ def load_persisted_state(data) -> None:def _restore_state(data):if data:+ blobs: dict[str, bytes | None] = {}funcs = data.get("functions", {})if isinstance(funcs, AccountRegionScopedDict):for scoped_key, func in funcs._data.items():- _internalize_code_zip(func)+ _internalize_code_zip(func, blobs)for ver in func.get("versions", {}).values():- _internalize_code_zip(ver)+ _internalize_code_zip(ver, blobs)_functions._data[scoped_key] = funcelif isinstance(funcs, AccountScopedDict):# Pre-region state. Code may be a content-addressed blob ref (#947)# or legacy inline base64 — _internalize_code_zip handles both.for (account_id, name), func in funcs._data.items():- _internalize_code_zip(func)+ _internalize_code_zip(func, blobs)for ver in func.get("versions", {}).values():- _internalize_code_zip(ver)+ _internalize_code_zip(ver, blobs)region = _region_from_function_record(func)_functions._data[(account_id, region, name)] = funcelse:for name, func in funcs.items():- _internalize_code_zip(func)+ _internalize_code_zip(func, blobs)for ver in func.get("versions", {}).values():- _internalize_code_zip(ver)+ _internalize_code_zip(ver, blobs)region = _region_from_function_record(func)_functions._data[(get_account_id(), region, name)] = func_layers.update(data.get("layers", {}))diff --git a/tests/test_lambda.py b/tests/test_lambda.pyindex 73479395..2fd1c8f2 100644--- a/tests/test_lambda.py+++ b/tests/test_lambda.py@@ -9931,6 +9931,26 @@ def test_lambda_identical_code_dedups_to_single_blob(lambda_svc_isolated):assert len(files) == 1, [f.name for f in files]+def test_lambda_restore_shares_one_bytes_object_per_blob(lambda_svc_isolated):+ """Versions and functions whose code is one blob restore to one bytes object, as they are live+ (PublishVersion reuses $LATEST's), not one copy each."""+ svc, _ = lambda_svc_isolated+ code, other = b"shared code" * 100, b"other code" * 100+ versions = {str(n): {"code_zip": code, "config": {"Version": str(n)}} for n in (1, 2, 3)}+ svc._functions["fn-a"] = _make_lambda_record("fn-a", code, versions=versions)+ svc._functions["fn-b"] = _make_lambda_record("fn-b", other, versions={"1": {"code_zip": code, "config": {}}})+ state = svc.get_state()+ svc._functions._data.clear()+ svc.load_persisted_state(state)++ a = svc._functions._data[("000000000000", svc.get_region(), "fn-a")]+ b = svc._functions._data[("000000000000", svc.get_region(), "fn-b")]+ copies = [a["code_zip"], *(v["code_zip"] for v in a["versions"].values()), b["versions"]["1"]["code_zip"]]+ assert all(c == code for c in copies)+ assert len({id(c) for c in copies}) == 1+ assert b["code_zip"] == other++def test_lambda_legacy_inline_base64_persistence_still_loads(lambda_svc_isolated):"""Pre-existing lambda.json files (written before content-addressedstorage) stored code_zip as inline base64. restore_state must accept