Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
Fix/ddb parity suite (#2410)
Nahuel Nucera0e03d86e8672
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex dd21a6ae..66f50dcc 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -12,11 +12,13 @@ Versioning follows [Semantic Versioning](https://semver.org/).- **API Gateway — an HTTP API's responses carry `apigw-requestid`, not a REST API's request ids** — a response from an HTTP API's route got `x-amzn-requestid` and `x-amz-request-id`; an HTTP API names its request id `apigw-requestid`, 16 base64 characters, which it now sends instead. REST APIs keep `x-amzn-requestid`. Contributed by @jeffreyheller.- **Lambda — publishing on create or code update leaves `$LATEST` unpublished** — `CreateFunction` and `UpdateFunctionCode` with `Publish: true` overwrote `$LATEST`'s `Version` with the new version number. `ListVersionsByFunction` then listed that number twice and no `$LATEST`, `GetFunctionConfiguration` without a qualifier reported it, and invoking `$LATEST` set `AWS_LAMBDA_FUNCTION_VERSION` to it. `$LATEST` now stays `$LATEST`, as on AWS; the create and update responses are unchanged.### Added--- **SSM — service settings** — `GetServiceSetting`, `UpdateServiceSetting` and `ResetServiceSetting`, per account and region, persisted. `SettingValue` is validated for the ids whose accepted values AWS documents, and `/ssm/parameter-store/default-parameter-tier` sets the `Tier` of a `PutParameter` that names none. Unblocks Terraform's `aws_ssm_service_setting`.+- **SSM — service settings** — `GetServiceSetting`, `UpdateServiceSetting` and `ResetServiceSetting`, per account and region, persisted. `SettingValue` is validated for the ids whose accepted values AWS documents, and `/ssm/parameter-store/default-parameter-tier` sets the `Tier` of a `PutParameter` that names none. Unblocks Terraform's `aws_ssm_service_setting`. Contributed by @skialpine.- **Auth — SigV4a signatures are verified under `AUTH=true`** — requests signed with `AWS4-ECDSA-P256-SHA256`, which SDKs use for Multi-Region Access Points, were refused for every IAM user. Header and presigned S3 signatures are now checked with the P-256 key SigV4a derives from the secret, with their date or expiry, and an `X-Amz-Region-Set` that leaves out the serving region is refused (`RegionSetMismatch` on S3, the unknown-key error elsewhere). Contributed by @iot-rocket.### Fixed+- **Lambda — publishing on create or code update leaves `$LATEST` unpublished** — `CreateFunction` and `UpdateFunctionCode` with `Publish: true` overwrote `$LATEST`'s `Version` with the new version number. `ListVersionsByFunction` then listed that number twice and no `$LATEST`, `GetFunctionConfiguration` without a qualifier reported it, and invoking `$LATEST` set `AWS_LAMBDA_FUNCTION_VERSION` to it. `$LATEST` now stays `$LATEST`, as on AWS; the create and update responses are unchanged. Contributed by @skialpine.+- **S3 — presigned URL `X-Amz-Expires`** — under `AUTH=true` a presigned URL without `X-Amz-Expires` never expired; it now answers 400 `AuthorizationQueryParametersError` "Query-string authentication version 4 requires the X-Amz-Algorithm, X-Amz-Credential, X-Amz-Signature, X-Amz-Date, X-Amz-SignedHeaders, and X-Amz-Expires parameters.", a negative value "X-Amz-Expires must be non-negative", and SigV4a presigned URLs follow the same limits.+- **S3 — `x-amz-mp-object-size` on `CompleteMultipartUpload`** — the header (`MpuObjectSize`) was ignored; a size that differs from the parts' total now answers 400 `InvalidRequest` "The provided 'x-amz-mp-object-size' header value 11 does not match what was computed: 10", a non-numeric one "Value for x-amz-mp-object-size header is invalid: 'abc'", and neither writes the object or closes the upload. Reported by @adzcodemi.- **IoT — QoS 1 redelivery follows AWS IoT** — the broker resent an unacknowledged QoS 1 message every 10 seconds with the DUP flag set; it now resends it 30 seconds after the last delivery (`IOT_RETRANSMIT_SECONDS` keeps changing that), and with DUP 0, as AWS IoT Core does, on the open connection and after a persistent session resumes. Contributed by @iot-rocket.- **Auth — requests without credentials are anonymous in every service** — under `AUTH=true` a request with no `Authorization` header and no query signature ran as the account root everywhere but S3, so unsigned STS, SQS, DynamoDB or Lambda calls were served. It is now refused in the service's own error, as on AWS: `MissingAuthenticationToken` 403 from Query and REST-XML services, `MissingAuthenticationTokenException` from JSON services, `AccessDenied` 403 from SQS, `UnrecognizedClientException` from Secrets Manager. Query credentials without `X-Amz-Algorithm` count as none. Operations the SDKs send unsigned (Cognito sign-in, `GetId`, `AssumeRoleWithWebIdentity`), SNS `ConfirmSubscription` and `Unsubscribe`, API Gateway and Function URL invokes and S3 keep their behaviour. Contributed by @iot-rocket.- **AppSync Events — `AUTH=true` checks the Event API's authorization** — under `AUTH=true` an HTTP publish, a WebSocket connection or a subscription was accepted with any API key or none unless `APPSYNC_EVENTS_ENFORCE_AUTH` was set. It now needs what the API or the channel namespace accepts, as on AWS: a key registered on the API for `API_KEY`, a SigV4 signature for `AWS_IAM`, a bearer token for Cognito and OIDC. A refused publish answers 401 `UnauthorizedException`, with AWS's "You are not authorized to make this call." for a key the API does not know. `APPSYNC_EVENTS_ENFORCE_AUTH` works as before with `AUTH` unset. Contributed by @iot-rocket.@@ -34,11 +36,15 @@ Versioning follows [Semantic Versioning](https://semver.org/).- **API Gateway — `AWS_IAM` methods refuse an `Authorization` header that is not SigV4** — under `AUTH=true` a REST API method with `AWS_IAM` authorization served a bearer token or an algorithm in lower case, and answered the SigV4a algorithm name without a signed `x-amz-region-set` or an unknown `AWS4-` algorithm with `UnrecognizedClientException`; it now answers 403 `IncompleteSignatureException` with API Gateway's message for each, while a method without `AWS_IAM` still serves such calls. Contributed by @iot-rocket.- **S3 — access point policies and unsigned requests** — an access point policy was stored but never applied, and an unsigned request through an access point was served. With `AUTH=true` a request through an access point alias or ARN is allowed on the access point by an identity policy or the access point policy (a Deny in either refuses it), and an unsigned one gets 400 `InvalidRequest`; S3 Control calls on the access point are unaffected. Contributed by @iot-rocket.- **API Gateway v2 — WebSocket connections, image functions and dropped fields** — `DeleteConnection` answered 204 but left the socket open; it now sends the client a close frame, `GetConnection` and `PostToConnection` answer 410 `GoneException`, and `$disconnect` runs after the close. A WebSocket route whose Lambda is a container image or a runtime other than Python or Node.js answered an empty 200 without running it; the function now runs. Routes keep `RouteResponseSelectionExpression` and `ModelSelectionExpression`, and stages keep `AccessLogSettings`, `ClientCertificateId` and `DeploymentId`, also from CloudFormation. Reported by @cupidchan.-- **Lambda — layers from other accounts that MiniStack doesn't hold** — a function attaching a third-party public layer, such as Datadog's `arn:aws:lambda:<region>:464622532012:layer:Datadog-Extension-ARM:<n>`, was refused with `AccessDeniedException`, where AWS attaches it. A layer version from another account that MiniStack does not hold now resolves as an empty layer with a warning in the log: `CreateFunction` and `UpdateFunctionConfiguration` attach it as `Arn` and `CodeSize` 0, `GetLayerVersionByArn` and `GetLayerVersion` by ARN return `LayerArn`, `LayerVersionArn`, `Version` and `Content.CodeSize` 0, and the layer does not run, even if a version is published under that ARN later. This replaces the name list for AWS-published extensions. A made-up ARN looks like a real one offline, so any 12-digit account, layer name and version that could exist on AWS resolve, where AWS answers `AccessDeniedException` for one that does not exist. A version of 0, a version with a leading zero and a name longer than 140 characters can't exist, and `CreateFunction`, `UpdateFunctionConfiguration` and `GetLayerVersionByArn` refuse them with that `AccessDeniedException`, as on AWS. A layer MiniStack holds still needs its grant, so a version it does not hold or grant is refused, and a layer from another region is still refused. `GetLayerVersion` now refuses a `LayerName` longer than 140 characters, a layer ARN included, with AWS's `ValidationException` before checking the version; `GetLayerVersionByArn` has no such limit, as on AWS. Contributed by @kharabasz.+- **Lambda — layers from other accounts** — attaching a layer MiniStack doesn't hold, such as Datadog's, was refused with `AccessDeniedException`; it now resolves as an empty layer (`CodeSize` 0, a warning in the log, never run), for `CreateFunction`, `UpdateFunctionConfiguration`, `GetLayerVersionByArn` and `GetLayerVersion`. A layer MiniStack holds still needs its grant, another region is still refused, and `GetLayerVersion` refuses a `LayerName` over 140 characters with `ValidationException`. Contributed by @kharabasz.- **CloudFront — `CreateFunction` tags** — the request's `Tags` were dropped, so `ListTagsForResource` on the new function's ARN was empty and Terraform re-planned `tags_all` on `aws_cloudfront_function`; they are now stored, a tag without `Value` is listed with an empty one, a refused create stores none, and `DeleteFunction` removes them so a function created again under the same name starts without them. `CreateFunction` and `UpdateFunction` answer invalid base64 code with `MalformedInput` "Invalid Base64 encoding", and `CreateFunction` now refuses what AWS refuses, before the name lookup: a missing or empty `FunctionCode`, a missing `FunctionConfig` or `Runtime`, an unknown runtime, a missing or empty `Name`, and a tag key that is missing, empty, over 128 characters or outside letters, numbers, spaces and `_ . : / = + - @` (values over 256 characters or outside the same set), listed together (at most ten) in one `MalformedXML`; and after it, `InvalidTagging` for a repeated key, an `aws:` key or more than 50 tags. Contributed by @kharabasz.- **CloudWatch — `PutMetricAlarm` and `PutCompositeAlarm` tags on create** — `Tags` were dropped when either call created an alarm, so Terraform re-planned `tags_all` on `aws_cloudwatch_metric_alarm` and `aws_cloudwatch_composite_alarm`; they are now stored over CBOR, JSON 1.0 and Query, keeping empty values and the last value of a repeated key, and they are still ignored when the call updates an existing alarm, as on AWS (use `TagResource` / `UntagResource`). Both calls now refuse what AWS refuses, before anything is stored (`PutCompositeAlarm` before its `AlarmRule` checks): a wrong JSON type in `Tags` (`MalformedInput`), a missing or empty `Key`, a missing `Value`, a key over 128 or a value over 256 characters, or a `DatapointsToAlarm` below 1, listed together (at most ten) in one `ValidationError`, on create a key or value with characters outside letters, numbers, spaces and `_ . : / = + - @` or too long in UTF-16 units, an `aws:` key on create (`PutMetricAlarm` 403 `AccessDenied`, `PutCompositeAlarm` `ValidationError` "Invalid System Tag"), and more than 50 tags on create. Query error messages are now XML-escaped. Contributed by @kharabasz.- **CloudWatch — `DatapointsToAlarm` only when set** — an alarm created by `PutMetricAlarm` or `AWS::CloudWatch::Alarm` without `DatapointsToAlarm` reported it as `EvaluationPeriods`, so Terraform planned `datapoints_to_alarm = 3 -> null`; `DescribeAlarms` now leaves it out unless the request or template sets it, the Query response includes it when set, and an update that omits it clears it. Contributed by @kharabasz.- **API Gateway v2 — `PostToConnection` sends a text frame** — data posted to a WebSocket connection reached the client as a binary frame, so a browser `WebSocket` handed `onmessage` a `Blob` where AWS gives a string; it now arrives as a text frame, as on AWS, with bytes that aren't UTF-8 replaced by U+FFFD. Contributed by @toby6578.+- **DynamoDB — PartiQL statements are parsed as DynamoDB parses them** — the regular-expression reader is replaced by a parser that answers malformed statements with AWS's messages (`Statement wasn't well formed, can't be processed: Unexpected term`, `Unexpected token after expression`, `Missing right-hand side expression of infix operator`, `Aliasing is not supported`, positions as `at <line>:<column>:<length>`), skips `--` and `/* */` comments, reads exponent and leading-dot number literals and refuses out-of-range ones `under root` with their path, and supports `ORDER BY`, `attribute_exists`, `attribute_not_exists`, `size`, `attribute_type` and `IS` in a WHERE (a false one on UPDATE or DELETE is `ConditionalCheckFailedException`), `set_add` and `set_delete`, the `FROM ... WHERE ... SET` form of UPDATE, and `EXISTS(SELECT ...)` in `ExecuteTransaction`. Number parameters are read as Java's `BigDecimal` reads them, a SELECT on a multi-attribute index is refused, overlapping OR branches on the sort key are refused, an INSERT returns an empty `Items` list and a write transaction an empty `Responses` list. Reported by @hicksy (paritysuite.org).+- **DynamoDB — numbers and duplicate set members** — a number's exponent is now read in full before the 1E-130 to 9.99E+125 limits, so `1e99999999999` is a `Number overflow` rather than unconvertible, zero written with an exponent is range-checked, and a duplicate set member is reported with its members in byte order; PutItem, UpdateItem and DeleteItem give these inside `1 validation error detected: `, the other operations bare, wherever the value sits (item, key, expression value, `Expected`, `AttributeUpdates`, `QueryFilter`). Reported by @hicksy (paritysuite.org).+- **DynamoDB — which error comes first** — TransactWriteItems, TransactGetItems, BatchWriteItem and BatchGetItem report their member errors (enums, table names, `ClientRequestToken`, empty lists) together before any table is looked up, and TransactWriteItems then checks each action in request order; PutItem, UpdateItem and DeleteItem read a request without a `Key` or with a value nested past about 62 levels in the older per-value form; legacy `Expected` conditions are validated; a KeyConditionExpression reports an operand of the wrong type in the order written and names a repeated or extra key condition; a missing RANGE equality on a multi-attribute key is named; CreateTable reports short index names together, GSIs first; SearchVectors answers a `null` member with `SerializationException`. Reported by @hicksy (paritysuite.org).+- **DynamoDB — sizes and limits** — UpdateItem's 400KB check now charges the request clause by clause (19 per SET or ADD plus its target and operands, 1 per REMOVE or DELETE plus its target) before the condition is evaluated; the expression-size and nesting-depth messages take each operation's form; `SET l[n]` past the end of a list appends instead of padding with `NULL`; a write bills an attribute a vector index reads at its name plus four bytes per dimension; BatchExecuteStatement passes `ReturnValuesOnConditionCheckFailure` to its members. Reported by @hicksy (paritysuite.org).## [1.5.31] — 2026-10-21diff --git a/ministack/core/iam_actions.py b/ministack/core/iam_actions.pyindex 1d396c13..bb59a506 100644--- a/ministack/core/iam_actions.py+++ b/ministack/core/iam_actions.py@@ -1108,10 +1108,6 @@ def extract_resource_arn(service: str, method: str, path: str,# --- Target-based (JSON body) services ---if service == "events":- try:- event_data = json.loads(body or b"{}")- except (json.JSONDecodeError, TypeError, UnicodeDecodeError):- event_data = {}buses = eventbridge_resource_arns(body, region, account_id)if buses:return buses[0]diff --git a/ministack/core/iam_evaluator.py b/ministack/core/iam_evaluator.pyindex 8e28064c..2c7f04c3 100644--- a/ministack/core/iam_evaluator.py+++ b/ministack/core/iam_evaluator.py@@ -1682,6 +1682,8 @@ def enforce(access_key_id: str, iam_action: str, service: str,result = evaluate(ctx, principal.policies, *principal.limits)found = _resource_policy(service, resource_arn)+ if found is not None and service == "s3" and "MultiRegionAccessPoint" in iam_action:+ found = None # The MRAP policy governs data requests, not S3 Control's MRAP calls.if found is None and service == "s3" and "s3:dataaccesspointarn" in context:if (iam_action in ("s3:PutBucketPolicy", "s3:DeleteBucketPolicy") and result.decision != "Deny"and resource_arn == context["s3:dataaccesspointarn"]):diff --git a/ministack/core/router.py b/ministack/core/router.pyindex a90c67ff..3a712977 100644--- a/ministack/core/router.py+++ b/ministack/core/router.py@@ -704,7 +704,6 @@ def detect_service(method: str, path: str, headers: dict, query_params: dict) ->host = headers.get("host", "")target = headers.get("x-amz-target", "")auth = headers.get("authorization", "")- content_type = headers.get("content-type", "")# 1. Check X-Amz-Target header (most reliable for JSON-based services)if target:diff --git a/ministack/core/sigv4.py b/ministack/core/sigv4.pyindex 747f89e5..c13d13e0 100644--- a/ministack/core/sigv4.py+++ b/ministack/core/sigv4.py@@ -145,9 +145,12 @@ def sigv4a_signing_key(access_key_id: str, secret: str):def sigv4a_signature_matches(access_key_id: str, secret: str, string_to_sign: str, signature: str) -> bool:"""Whether a hex DER ECDSA signature signs the string with the key SigV4a derives."""- from cryptography.exceptions import InvalidSignature- from cryptography.hazmat.primitives import hashes- from cryptography.hazmat.primitives.asymmetric import ec+ try:+ from cryptography.exceptions import InvalidSignature+ from cryptography.hazmat.primitives import hashes+ from cryptography.hazmat.primitives.asymmetric import ec+ except ImportError: # cryptography ships in the images and the [full] extra; without it nothing verifies+ return Falsetry:sigv4a_signing_key(access_key_id, secret).public_key().verify(diff --git a/ministack/services/_dynamodb_keywords.py b/ministack/services/_dynamodb_keywords.pyindex 41588e04..30be1988 100644--- a/ministack/services/_dynamodb_keywords.py+++ b/ministack/services/_dynamodb_keywords.py@@ -1,12 +1,13 @@# Copyright (c) 2026 MiniStack Contributors. SPDX-License-Identifier: MIT# Copies or substantial portions, including AI-assisted ports or rewrites, must retain this notice (see LICENSE).-"""DynamoDB AWS system keyword list.+"""DynamoDB reserved words, and the PartiQL statement parser.-These names must be escaped with `#alias` in ConditionExpression,-UpdateExpression, FilterExpression, KeyConditionExpression, and-ProjectionExpression. Sourced from the AWS DynamoDB Developer Guide.+AWS_KEYWORDS must be escaped with `#alias` in the expression parameters (AWS DynamoDB Developer Guide)."""+import re+from decimal import Decimal+AWS_KEYWORDS = frozenset(("ABORT","ABSOLUTE","ACTION","ADD","AFTER","AGENT","AGGREGATE","ALL","ALLOCATE","ALTER","ANALYZE","AND","ANY","ARCHIVE","ARE","ARRAY","AS","ASC","ASCII","ASENSITIVE","ASSERTION",@@ -69,3 +70,730 @@ AWS_KEYWORDS = frozenset(("VARIANCE","VARINT","VARYING","VIEW","VIEWS","VIRTUAL","VOID","WAIT","WHEN","WHENEVER","WHERE","WHILE","WINDOW","WITH","WITHIN","WITHOUT","WORK","WRAPPED","WRITE","YEAR","ZONE",))+++# ---------------------------------------------------------------------------+# PartiQL+# ---------------------------------------------------------------------------++NOT_WELL_FORMED = "Statement wasn't well formed, can't be processed: "+_LEFTOVER = "Unexpected token after expression"+++class PartiqlError(ValueError):+ """A statement DynamoDB refuses before it runs. ``statement_level`` errors carry the+ ``Validation failed in TransactStatements[i]: `` prefix inside a transaction."""++ def __init__(self, message, statement_level=True):+ super().__init__(message)+ self.statement_level = statement_level+++def _malformed(detail=""):+ return PartiqlError(NOT_WELL_FORMED + detail)+++_TOKEN_RE = re.compile(r"""+ (?P<ws>\s+)+ | (?P<line>--[^\n]*)+ | (?P<block>/\*.*?\*/)+ | (?P<open>/\*|\#)+ | (?P<str>'(?:[^']|'')*')+ | (?P<qid>"(?:[^"]|"")*")+ | (?P<num>(?:\d+(?:\.\d*)?|\.\d+)(?:[eE][+-]?\d+)?)+ | (?P<id>[A-Za-z_][A-Za-z0-9_$]*)+ | (?P<op><<|>>|<=|>=|<>|!=|\|\||[-+*/%=<>(),.\[\]{}:;?])+""", re.VERBOSE | re.DOTALL)+_INT32 = (-(2 ** 31), 2 ** 31 - 1)+_RESERVED = frozenset((+ "SELECT", "FROM", "WHERE", "AND", "OR", "NOT", "IN", "IS", "BETWEEN", "LIKE", "INSERT", "INTO",+ "UPDATE", "SET", "REMOVE", "DELETE", "RETURNING", "ORDER", "AS", "EXISTS", "INNER", "NULL",+ "MISSING", "TRUE", "FALSE"))+_OPERATOR_WORDS = frozenset(("AND", "OR", "IS", "LIKE", "BETWEEN", "IN"))+_COMPARATORS = frozenset(("=", "<>", "!=", "<", "<=", ">", ">="))+_WHERE_FUNCTIONS = frozenset(("attribute_exists", "attribute_not_exists", "attribute_type",+ "begins_with", "contains", "size"))+_TYPE_NAMES = {+ "NULL": "NULL", "MISSING": "MISSING", "BOOL": "BOOL", "BOOLEAN": "BOOL",+ "SMALLINT": "N", "INT": "N", "INTEGER": "N", "INT2": "N", "INT4": "N", "INT8": "N", "BIGINT": "N",+ "FLOAT": "N", "REAL": "N", "DECIMAL": "N", "NUMERIC": "N", "DOUBLE PRECISION": "N",+ "STRING": "S", "CHAR": "S", "CHARACTER": "S", "VARCHAR": "S", "CHARACTER VARYING": "S",+ "SYMBOL": "S", "CLOB": "S", "BLOB": "B", "LIST": "L", "SEXP": "L", "STRUCT": "M", "TUPLE": "M",+ "BAG": "BAG",+}+_NUM_LIMITS = (+ "Number overflow. Attempting to store a number with magnitude larger than supported range",+ "Number underflow. Attempting to store a number with magnitude smaller than supported range",+ "Attempting to store more than 38 significant digits in a Number",+)+++def _tokenize(text):+ """(kind, value, start, end) tokens; a lone `#` or an unclosed block comment is unreadable."""+ tokens, pos, n = [], 0, len(text)+ while pos < n:+ m = _TOKEN_RE.match(text, pos)+ if not m or m.lastgroup == "open":+ raise _malformed()+ kind = m.lastgroup+ if kind == "num" and text[pos] == "." and tokens and (+ tokens[-1][0] in ("id", "qid") or tokens[-1][1] in ("]", ")")):+ kind, m = "op", re.compile(r"\.").match(text, pos)+ pos = m.end()+ if kind in ("ws", "line", "block"):+ continue+ value = m.group(0)+ if kind == "num":+ _check_number_text(value)+ tokens.append((kind, value, m.start(), m.end()))+ tokens.append(("eof", "", n, n))+ return tokens+++def _check_number_text(text):+ mantissa, _, exponent = text.lower().partition("e")+ exp = int(exponent or 0)+ scale = len(mantissa.partition(".")[2]) - exp+ if not _INT32[0] <= exp <= _INT32[1] or not _INT32[0] <= scale <= _INT32[1]:+ raise _malformed()+++def number_limit_error(text):+ """Why a number literal breaks DynamoDB's number rules, or None."""+ mantissa, _, exponent = text.lower().lstrip("+-").partition("e")+ integer, _, fraction = mantissa.partition(".")+ digits = (integer + fraction).lstrip("0")+ adjusted = max(len(digits), 1) - (len(fraction) - int(exponent or 0)) - 1+ if digits and adjusted > 125:+ return _NUM_LIMITS[0]+ if digits and adjusted < -130:+ return _NUM_LIMITS[1]+ if len(digits.rstrip("0")) > 38:+ return _NUM_LIMITS[2]+ return None+++def canonical_number(text):+ d = Decimal(text)+ return "0" if d == 0 else format(d.normalize(), "f")+++def _bag_av(members, source):+ """A set literal: duplicates are reported as written, an empty bag is refused."""+ if not members:+ raise PartiqlError("Empty bags are not supported", statement_level=False)+ kinds = {next(iter(m)) for m in members}+ if len(kinds) != 1 or kinds - {"S", "N", "B"}:+ raise PartiqlError("An operand in the update expression has an incorrect data type", statement_level=False)+ kind = kinds.pop()+ values = [m[kind] for m in members]+ seen = set()+ for v in values:+ key = canonical_number(v) if kind == "N" else v+ if key in seen:+ raise PartiqlError("One or more parameter values were invalid: Input collection "+ f"[{', '.join(source)}] contains duplicates.", statement_level=False)+ seen.add(key)+ return {kind + "S": values}+++class _Parser:+ __slots__ = ("text", "tokens", "pos", "params", "next_param")++ def __init__(self, text, parameters):+ self.text = text+ self.tokens = _tokenize(text)+ self.pos = 0+ self.params = parameters or []+ self.next_param = 0++ # -- tokens ----------------------------------------------------------+ def peek(self, offset=0):+ return self.tokens[min(self.pos + offset, len(self.tokens) - 1)]++ def take(self):+ tok = self.tokens[self.pos]+ self.pos = min(self.pos + 1, len(self.tokens) - 1)+ return tok++ def word(self, tok=None):+ tok = tok or self.peek()+ return tok[1].upper() if tok[0] == "id" else None++ def at_word(self, *words):+ return self.word() in words++ def at_op(self, *ops):+ tok = self.peek()+ return tok[0] == "op" and tok[1] in ops++ def at_end(self):+ return self.peek()[0] == "eof" or self.at_op(";")++ def expect_word(self, word, detail=_LEFTOVER):+ if not self.at_word(word):+ raise _malformed(detail)+ return self.take()++ def expect_op(self, op, detail="Unexpected term"):+ if not self.at_op(op):+ raise _malformed(detail)+ return self.take()++ def position(self, start, length):+ line = self.text.count("\n", 0, start) + 1+ column = start - (self.text.rfind("\n", 0, start) + 1) + 1+ return f"{line}:{column}:{length}"++ def finish(self):+ if self.at_op(";"):+ self.take()+ if self.peek()[0] != "eof":+ raise _malformed(_LEFTOVER)++ # -- statements ------------------------------------------------------+ def statement(self):+ head = self.word()+ if head == "SELECT":+ out = self.select()+ elif head == "INSERT":+ out = self.insert()+ elif head == "UPDATE":+ out = self.update()+ elif head == "DELETE":+ out = self.delete()+ elif head == "FROM":+ out = self.from_first()+ elif head == "EXISTS" and self.peek(1)[1] == "(":+ self.take()+ self.take()+ if not self.at_word("SELECT"):+ raise _malformed("Unexpected term")+ inner = self.select(nested=True)+ self.expect_op(")", _LEFTOVER)+ out = {"op": "EXISTS", "select": inner, "table": inner["table"], "index": inner["index"]}+ else:+ return self.bare_expression()+ self.finish()+ return out++ def bare_expression(self):+ try:+ node = self.expression()+ self.finish()+ except PartiqlError:+ raise _malformed("Expected data manipulation")+ if node[0] == "is" and node[2] not in ("NULL", "MISSING"):+ raise PartiqlError("Unsupported operation: IsType")+ raise _malformed("Expected data manipulation")++ def table_path(self, missing="Unexpected term"):+ parts = []+ while True:+ tok = self.peek()+ if tok[0] == "qid":+ name = tok[1][1:-1].replace('""', '"')+ if not name:+ raise PartiqlError("Path component cannot be an empty string")+ parts.append((name, True))+ elif tok[0] == "id" and tok[1].upper() not in _RESERVED:+ parts.append((tok[1], False))+ else:+ raise _malformed(missing if not parts else "Unexpected term")+ self.take()+ if not self.at_op("."):+ break+ self.take()+ if len(parts) > 2:+ raise PartiqlError("A path may contain at most 2 components in the FROM clause")+ if len(parts) == 2 and (parts[0][1] or parts[1][1]):+ return parts[0][0], parts[1][0]+ return ".".join(p[0] for p in parts), None++ def refuse_alias(self, leftover=False):+ tok = self.peek()+ if self.at_word("AS") or (tok[0] == "id" and tok[1].upper() not in _RESERVED):+ raise _malformed(_LEFTOVER) if leftover else PartiqlError("Aliasing is not supported")++ def select(self, nested=False):+ self.take()+ projections, count = None, False+ if self.at_op("*"):+ self.take()+ else:+ projections = []+ while True:+ tok = self.peek()+ if tok[0] == "id" and self.peek(1)[1] == "(":+ if tok[1].upper() == "COUNT" and self.peek(2)[1] == "*" and self.peek(3)[1] == ")":+ for _ in range(4):+ self.take()+ count = True+ else:+ raise PartiqlError(f"Unexpected path component at {self.position(tok[2], len(tok[1]))}")+ else:+ projections.append(self.path()[1])+ if not self.at_op(","):+ break+ self.take()+ self.expect_word("FROM", "Unexpected term")+ table, index = self.table_path()+ self.refuse_alias()+ where = order = None+ if self.at_word("WHERE"):+ self.take()+ where = self.expression()+ if self.at_word("ORDER") and not nested:+ self.take()+ self.expect_word("BY", "Unexpected term")+ path = self.path()[1]+ desc = False+ if self.at_word("ASC", "DESC"):+ desc = self.take()[1].upper() == "DESC"+ order = (path, desc)+ return {"op": "SELECT", "table": table, "index": index, "projections": projections,+ "count": count, "where": where, "order": order}++ def insert(self):+ self.take()+ self.expect_word("INTO", "Unexpected term")+ table, index = self.table_path()+ if index is not None:+ raise _malformed("FROM clause may only contain a single table name")+ self.expect_word("VALUE")+ tok = self.peek()+ if self.at_op("?"):+ item = self.parameter()+ elif self.at_op("{"):+ node = self.map_or_list()+ if node[0] != "map":+ raise _malformed("Unexpected term")+ item = {key: self.literal_value(value, "root") for key, value in node[1]}+ else:+ raise _malformed("Unexpected term")+ return {"op": "INSERT", "table": table, "index": None, "item": item, "item_token": tok}++ def write_clauses(self):+ assignments, removes = [], []+ while self.at_word("SET", "REMOVE"):+ if self.take()[1].upper() == "SET":+ while True:+ target = self.path()+ self.expect_op("=", "Unexpected term")+ assignments.append((target[1], self.assignment_value(target)))+ if not self.at_op(","):+ break+ self.take()+ else:+ while True:+ removes.append(self.path()[1])+ if not self.at_op(","):+ break+ self.take()+ return assignments, removes++ def returning(self):+ if not self.at_word("RETURNING"):+ return None+ start = self.take()[3]+ end = len(self.text)+ while self.peek()[0] != "eof" and not self.at_op(";"):+ end = self.take()[3]+ return self.text[start:end].strip()++ def update(self):+ self.take()+ table, index = self.table_path("Expected identifier for simple path")+ if index is not None:+ raise PartiqlError("This operation is not supported on an index")+ self.refuse_alias()+ assignments, removes = self.write_clauses()+ if not assignments and not removes:+ raise _malformed("Unexpected term")+ where = None+ if self.at_word("WHERE"):+ self.take()+ where = self.expression()+ return {"op": "UPDATE", "table": table, "index": None, "assignments": assignments,+ "removes": removes, "where": where, "returning": self.returning(), "from_first": False}++ def from_first(self):+ self.take()+ table, index = self.table_path()+ if index is not None:+ raise PartiqlError("This operation is not supported on an index")+ self.refuse_alias()+ where = None+ if self.at_word("WHERE"):+ self.take()+ where = self.expression()+ assignments, removes = self.write_clauses()+ if not assignments and not removes:+ raise _malformed(_LEFTOVER if not self.at_end() else "Unexpected term")+ return {"op": "UPDATE", "table": table, "index": None, "assignments": assignments,+ "removes": removes, "where": where, "returning": self.returning(), "from_first": True}++ def delete(self):+ self.take()+ self.expect_word("FROM", "Expected identifier for simple path")+ table, index = self.table_path("Expected identifier for simple path")+ if index is not None:+ raise PartiqlError("This operation is not supported on an index")+ self.refuse_alias()+ where = None+ if self.at_word("WHERE"):+ self.take()+ where = self.expression()+ return {"op": "DELETE", "table": table, "index": None, "where": where, "returning": self.returning()}++ # -- values ----------------------------------------------------------+ def parameter(self):+ self.take()+ if self.next_param >= len(self.params):+ raise PartiqlError("Number of parameters in request and statement don't match.", statement_level=False)+ value = self.params[self.next_param]+ self.next_param += 1+ return value++ def path(self):+ """(start, parts, end) of a document path; a keyword after a dot is not a component."""+ tok = self.peek()+ if tok[0] == "qid":+ parts = [tok[1][1:-1].replace('""', '"')]+ elif tok[0] == "id" and tok[1].upper() not in _RESERVED:+ parts = [tok[1]]+ else:+ raise self.term_error()+ start = self.take()[2]+ end = tok[3]+ while self.at_op(".", "["):+ if self.take()[1] == ".":+ part = self.peek()+ if part[0] == "qid":+ parts.append(part[1][1:-1].replace('""', '"'))+ elif part[0] == "id" and part[1].upper() not in _RESERVED:+ parts.append(part[1])+ else:+ raise _malformed("Invalid path dot component")+ end = self.take()[3]+ else:+ part = self.take()+ if part[0] == "num" and part[1].isdigit():+ parts.append(int(part[1]))+ elif part[0] == "str":+ parts.append(part[1][1:-1].replace("''", "'"))+ else:+ raise _malformed("Unexpected term")+ end = self.expect_op("]")[3]+ return start, parts, end++ def term_error(self):+ tok = self.peek()+ word = self.word(tok)+ if tok[0] == "eof" or tok[1] == ";":+ return _malformed("Unexpected term")+ if word in _OPERATOR_WORDS:+ return _malformed("Unexpected operator")+ if word in _RESERVED:+ return _malformed("Unexpected keyword")+ return _malformed("Unexpected term")++ def map_or_list(self):+ """A literal map, list or bag, as an expression node."""+ tok = self.take()+ if tok[1] == "{":+ entries = []+ while not self.at_op("}"):+ key = self.take()+ if key[0] != "str":+ raise _malformed("Unexpected term")+ self.expect_op(":")+ entries.append((key[1][1:-1].replace("''", "'"), self.expression()))+ if not self.at_op(","):+ break+ self.take()+ self.expect_op("}")+ return ("map", entries, tok)+ close = "]" if tok[1] == "[" else ">>"+ items = []+ while not self.at_op(close):+ items.append(self.expression())+ if not self.at_op(","):+ break+ self.take()+ end = self.expect_op(close)+ kind = "list" if close == "]" else "bag"+ return (kind, items, tok, self.text[tok[3]:end[2]])++ def literal_value(self, node, path):+ """The attribute value a literal denotes; numbers are checked where they sit."""+ kind = node[0]+ if kind == "lit":+ value = node[1]+ if "N" in value and node[2] == "num":+ err = number_limit_error(value["N"])+ if err:+ raise PartiqlError(f"{err} under {path}", statement_level=False)+ return {"N": canonical_number(value["N"])}+ return value+ if kind == "map":+ return {"M": {k: self.literal_value(v, f"{path}.{k}") for k, v in node[1]}}+ if kind == "list":+ return {"L": [self.literal_value(v, f"{path}[{i}]") for i, v in enumerate(node[1])]}+ if kind == "bag":+ members = [self.literal_value(v, path) for v in node[1]]+ return _bag_av(members, [t.strip().strip("'") for t in node[3].split(",")] if node[3].strip() else [])+ if kind == "call":+ tok = node[3]+ raise PartiqlError(f"Unsupported data type: {node[1]} under key root at {self.position(tok[2], len(tok[1]))}")+ raise _malformed("Unexpected term")++ def assignment_value(self, target):+ node = self.expression()+ if node[0] == "call" and node[1] in ("set_add", "set_delete"):+ name, args, tok = node[1].upper(), node[2], node[3]+ if len(args) != 2:+ raise PartiqlError(f"{name} must have exactly two arguments at {self.position(tok[2], len(tok[1]))}")+ first, second = args+ if first[0] != "path" or first[1] != target[1]:+ raise PartiqlError(f"The first argument to {name} must equal the assignment value at "+ f"{self.position(tok[2], len(tok[1]))}")+ if second[0] not in ("bag", "lit") or (second[0] == "lit" and second[2] != "param"):+ raise PartiqlError(f"The second argument to {name} must be a value with type SET at "+ f"{self.position(first[2], first[3] - first[2])}")+ members = self.literal_value(second, "root") if second[0] == "bag" else second[1]+ return ("setfn", node[1], members)+ return self.value_tree(node)++ def value_tree(self, node):+ """Fold literal sub-trees into attribute values; paths and arithmetic stay as nodes."""+ kind = node[0]+ if kind in ("lit", "map", "list", "bag") and self.is_literal(node):+ return ("lit", self.literal_value(node, "root"), "value")+ if kind == "arith":+ return ("arith", node[1], self.value_tree(node[2]), self.value_tree(node[3]))+ if kind == "call":+ return ("call", node[1], [self.value_tree(a) for a in node[2]], node[3])+ return node++ def is_literal(self, node):+ if node[0] == "lit":+ return True+ if node[0] == "map":+ return all(self.is_literal(v) for _, v in node[1])+ if node[0] in ("list", "bag"):+ return all(self.is_literal(v) for v in node[1])+ return False++ # -- expressions -----------------------------------------------------+ def expression(self):+ node = self.conjunction()+ while self.at_word("OR"):+ self.take()+ self.require_operand()+ node = ("or", node, self.conjunction())+ return node++ def require_operand(self):+ if self.at_end():+ raise _malformed("Missing right-hand side expression of infix operator")++ def conjunction(self):+ node = self.negation()+ while self.at_word("AND"):+ self.take()+ self.require_operand()+ node = ("and", node, self.negation())+ return node++ def negation(self):+ if self.at_word("NOT"):+ self.take()+ return ("not", self.negation())+ return self.predicate()++ def predicate(self):+ left = self.additive()+ tok = self.peek()+ if tok[0] == "op" and tok[1] in _COMPARATORS:+ self.take()+ self.require_operand()+ right = self.additive()+ op = "<>" if tok[1] == "!=" else tok[1]+ left, right = self.where_operand(left), self.where_operand(right)+ if op in ("<", "<=", ">", ">="):+ _check_ordered(op, left, right)+ return ("cmp", op, left, right)+ negated = False+ if self.at_word("NOT") and self.word(self.peek(1)) in ("BETWEEN", "IN", "LIKE"):+ self.take()+ negated = True+ word = self.word()+ if word == "BETWEEN":+ self.take()+ low = self.additive()+ self.expect_word("AND", "Unexpected term")+ node = ("between", self.where_operand(left), self.where_operand(low), self.where_operand(self.additive()))+ _check_ordered("BETWEEN", node[2], node[3])+ elif word == "IN":+ self.take()+ if self.at_op("?"):+ options = ("lit", self.parameter(), "param")+ else:+ opener = self.peek()+ if opener[1] not in ("(", "["):+ raise self.term_error()+ self.take()+ close = ")" if opener[1] == "(" else "]"+ items = []+ while not self.at_op(close):+ items.append(self.where_operand(self.additive()))+ if not self.at_op(","):+ break+ self.take()+ self.expect_op(close)+ options = ("options", items)+ node = ("in", self.where_operand(left), options)+ elif word == "LIKE":+ self.take()+ node = ("like", left, self.additive())+ elif word == "IS":+ self.take()+ is_not = False+ if self.at_word("NOT"):+ self.take()+ is_not = True+ node = ("is", self.where_operand(left), self.type_name(), is_not)+ else:+ return left+ return ("not", node) if negated else node++ def type_name(self):+ first = self.word()+ if first in ("DOUBLE", "CHARACTER") and self.word(self.peek(1)) in ("PRECISION", "VARYING"):+ self.take()+ first = f"{first} {self.take()[1].upper()}"+ elif first in _TYPE_NAMES:+ self.take()+ else:+ raise _malformed("Expected type name")+ return _TYPE_NAMES[first]++ def where_operand(self, node):+ if node[0] == "lit" and node[2] == "num":+ err = number_limit_error(node[1]["N"])+ if err:+ raise PartiqlError(f"{err} under root", statement_level=False)+ return ("lit", {"N": canonical_number(node[1]["N"])}, "value")+ return node++ def additive(self):+ node = self.unary()+ while self.at_op("+", "-"):+ op = self.take()[1]+ self.require_operand()+ node = ("arith", op, node, self.unary())+ return node++ def unary(self):+ if self.at_op("-"):+ self.take()+ node = self.unary()+ if node[0] == "lit" and node[2] == "num":+ return ("lit", {"N": "-" + node[1]["N"]}, "num")+ return ("arith", "-", ("lit", {"N": "0"}, "value"), node)+ return self.primary()++ def primary(self):+ tok = self.peek()+ kind, value = tok[0], tok[1]+ if kind == "str":+ self.take()+ return ("lit", {"S": value[1:-1].replace("''", "'")}, "str")+ if kind == "num":+ self.take()+ return ("lit", {"N": value}, "num")+ if kind == "op":+ if value == "?":+ return ("lit", self.parameter(), "param")+ if value == "(":+ self.take()+ node = self.expression()+ self.expect_op(")")+ return node+ if value in ("{", "[", "<<"):+ return self.map_or_list()+ raise self.term_error()+ word = self.word(tok)+ if word in ("TRUE", "FALSE"):+ self.take()+ return ("lit", {"BOOL": word == "TRUE"}, "bool")+ if word == "NULL":+ self.take()+ return ("lit", {"NULL": True}, "null")+ if word == "MISSING":+ self.take()+ return ("missing",)+ if kind == "id" and self.peek(1)[1] == "(":+ return self.call()+ if kind in ("id", "qid"):+ start, parts, end = self.path()+ return ("path", parts, start, end)+ raise self.term_error()++ def call(self):+ tok = self.take()+ self.take()+ args = []+ while not self.at_op(")"):+ args.append(self.expression())+ if not self.at_op(","):+ break+ self.take()+ self.expect_op(")")+ return ("call", tok[1].lower(), args, tok)+++def _check_ordered(op, *operands):+ """Ordering exists only for strings, numbers and binaries."""+ for operand in operands:+ if operand[0] == "lit" and isinstance(operand[1], dict) and operand[1]:+ vtype = next(iter(operand[1]))+ if vtype not in ("S", "N", "B"):+ raise PartiqlError("Incorrect operand type for operator or function; "+ f"operator or function: {op}, operand type: {vtype}", statement_level=False)+++def check_where(parser, node):+ """Functions a WHERE clause accepts, and their operands."""+ kind = node[0]+ if kind == "call":+ name, args, tok = node[1], node[2], node[3]+ if name not in _WHERE_FUNCTIONS:+ raise PartiqlError(f"Unrecognized function: {name} at {parser.position(tok[2], len(tok[1]))}")+ if name in ("attribute_exists", "attribute_not_exists"):+ if len(args) != 1:+ raise PartiqlError("Incorrect number of operands for operator or function; "+ f"operator or function: {name}, number of operands: {len(args)}")+ if args[0][0] != "path":+ raise PartiqlError(f"Operator or function requires a document path; operator or function: {name}")+ for arg in args:+ check_where(parser, arg)+ return+ for child in node[1:]:+ if isinstance(child, tuple) and child and isinstance(child[0], str):+ check_where(parser, child)+ elif isinstance(child, list):+ for item in child:+ if isinstance(item, tuple) and item and isinstance(item[0], str):+ check_where(parser, item)+++def parse_statement(text, parameters):+ """Parse one PartiQL statement into the shape the executors run."""+ parser = _Parser(text, parameters)+ out = parser.statement()+ for clause in (out.get("where"), (out.get("select") or {}).get("where")):+ if clause is not None:+ check_where(parser, clause)+ return outdiff --git a/ministack/services/apigateway.py b/ministack/services/apigateway.pyindex f2f90c40..969eb1ec 100644--- a/ministack/services/apigateway.py+++ b/ministack/services/apigateway.py@@ -3334,10 +3334,7 @@ async def handle_websocket(scope, receive, send, api_id: str, path_override: strif body:if isinstance(body, (dict, list)):body = json.dumps(body)- if isinstance(body, bytes):- await send({"type": "websocket.send", "bytes": body})- else:- await send({"type": "websocket.send", "text": str(body)})+ await send({"type": "websocket.send", "text": str(body)})finally:close_event.set()closed_by_delete.cancel()diff --git a/ministack/services/appsync.py b/ministack/services/appsync.pyindex b85b1008..7a8caab7 100644--- a/ministack/services/appsync.py+++ b/ministack/services/appsync.py@@ -537,11 +537,6 @@ def _get_type(api_id, type_name, query_params):if api_id not in _apis:return error_response_json("NotFoundException", f"GraphQL API {api_id} not found", 404)- fmt = "SDL"- if query_params.get("format"):- fmt_val = query_params["format"]- fmt = fmt_val[0] if isinstance(fmt_val, list) else fmt_val-t = _types.get(api_id, {}).get(type_name)if not t:return error_response_json("NotFoundException", f"Type {type_name} not found", 404)@@ -2507,7 +2502,6 @@ def _ddb_scan(table, table_name, args, sub_fields):"""Scan/list items, optionally with filters and pagination."""items = []limit = args.get("limit", 100)- next_token = args.get("nextToken")count = 0for pk in sorted(table["items"].keys()):diff --git a/ministack/services/appsync_events.py b/ministack/services/appsync_events.pyindex bc940f54..fdbc432a 100644--- a/ministack/services/appsync_events.py+++ b/ministack/services/appsync_events.py@@ -115,10 +115,6 @@ def _restore_api_child_store(store.set_scoped(account_id, region, api_id, value)-def load_persisted_state(data):- return _restore_state(data)--def _restore_state(data):if not data:return@@ -129,8 +125,8 @@ def _restore_state(data):_restore_api_child_store(_api_keys, data.get("api_keys", {}))-# Same contract as apigateway.py (used by app.py persistence loader)-load_persisted_state = _restore_state+def load_persisted_state(data):+ return _restore_state(data)diff --git a/ministack/services/bedrock_agent_runtime.py b/ministack/services/bedrock_agent_runtime.pyindex 3741aa7e..da78a4f5 100644--- a/ministack/services/bedrock_agent_runtime.py+++ b/ministack/services/bedrock_agent_runtime.py@@ -232,7 +232,6 @@ def _invoke_agent(agent_id: str, alias_id: str, session_id: str, body) -> tuple:if err:return errprompt = body_obj.get("inputText", "")- end_session = bool(body_obj.get("endSession"))reply = _mock_reply(agent_id, prompt)# Stream: chunk(s) -> (optional end-of-session). No trace events are# emitted, even with enableTrace: no orchestration happens here (no modeldiff --git a/ministack/services/cloudformation/engine.py b/ministack/services/cloudformation/engine.pyindex 336e011e..e9a9f394 100644--- a/ministack/services/cloudformation/engine.py+++ b/ministack/services/cloudformation/engine.py@@ -1378,7 +1378,6 @@ def _extract_deps(resource_def: dict, all_resource_names: set) -> set:def _topological_sort(resources: dict, conditions: dict) -> list:"""Kahn's algorithm for topological sort of resources."""- all_names = set(resources.keys())# Filter out resources whose condition evaluates to falseactive = set()for name, defn in resources.items():diff --git a/ministack/services/cloudformation/provisioners.py b/ministack/services/cloudformation/provisioners.pyindex d4f5e2ec..559d276d 100644--- a/ministack/services/cloudformation/provisioners.py+++ b/ministack/services/cloudformation/provisioners.py@@ -1685,7 +1685,6 @@ def _s3_update(physical_id, old_props, new_props, stack_name):Auto-named buckets (no explicit ``BucketName``) must keep their existingphysical resource id so that ``Ref`` keeps resolving to the same bucketwhere artifacts were uploaded."""- old_name = old_props.get("BucketName")new_name = new_props.get("BucketName")if new_name and new_name != physical_id:return _s3_create(new_name, new_props, stack_name)@@ -5071,7 +5070,6 @@ def _cfn_nested_stack_delete(physical_id, props):if not child_stack:return- child_stack_id = child_stack.get("StackId", physical_id)resources = child_stack.get("_resources", {})template = child_stack.get("_template", {})res_defs = template.get("Resources", {}) if template else {}@@ -7976,7 +7974,6 @@ def _ec2_vpc_create(logical_id, props, stack_name):}_ec2_vpc_dns_attributes(_ec2._vpcs[vpc_id], props)_ec2_apply_tags(vpc_id, props)- arn = f"arn:aws:ec2:{get_region()}:{get_account_id()}:vpc/{vpc_id}"return vpc_id, {"VpcId": vpc_id, "DefaultSecurityGroup": sg_id, "DefaultNetworkAcl": acl_id}@@ -10633,8 +10630,7 @@ def _cw_metric_alarm_record(name, props):"Unit": props.get("Unit"),"AlarmConfigurationUpdatedTimestamp": int(time.time()),}- # As with PutMetricAlarm, AWS describes DatapointsToAlarm only while the- # template sets it, on create and on update.+ # AWS describes DatapointsToAlarm only while the template sets it.if dta is not None:alarm["DatapointsToAlarm"] = int(dta)return alarm@@ -11902,6 +11898,10 @@ def _rds_db_instance_fields(props):"DBInstanceClass": props.get("DBInstanceClass", "db.t3.micro"),"EngineVersion": engine_version,"AllocatedStorage": int(props.get("AllocatedStorage") or 20),+ # Set only when it turns storage autoscaling on (above AllocatedStorage).+ "MaxAllocatedStorage": (int(props["MaxAllocatedStorage"]) if props.get("MaxAllocatedStorage")+ and int(props["MaxAllocatedStorage"]) != int(props.get("AllocatedStorage") or 20)+ else None),"StorageType": props.get("StorageType", "gp2"),"PreferredBackupWindow": props.get("PreferredBackupWindow", "03:00-04:00"),"BackupRetentionPeriod": int(props.get("BackupRetentionPeriod", 1)),diff --git a/ministack/services/cloudfront.py b/ministack/services/cloudfront.pyindex 977baa96..24103be4 100644--- a/ministack/services/cloudfront.py+++ b/ministack/services/cloudfront.py@@ -840,8 +840,7 @@ _FUNCTION_TAG_LIMIT = 50def _last_child(el, tag):- """The last direct child with this local name (AWS keeps the last of a- repeated member)."""+ """The last direct child with this local name; AWS keeps the last of a repeated member."""found = Nonefor child in el:if _local_tag_name(child) == tag:@@ -850,17 +849,12 @@ def _last_child(el, tag):def _function_request_tags(el):- """CreateFunctionRequest's Tags/Items in order as (Key, Value), None for a- missing element, and AWS's parse error for a malformed tag or None. AWS- reads every child of the last Items as a tag, whatever its name, keeps the- last of a repeated Key or Value, and ignores tags placed directly under- Tags."""+ """CreateFunction's Tags/Items as (Key, Value) pairs, plus AWS's parse error or None."""tags_el = _find(el, "Tags")items_el = _last_child(tags_el, "Items") if tags_el is not None else Nonetags = []for tag_el in items_el if items_el is not None else []:- # Text in a tag without child elements is malformed, even whitespace;- # next to children it is ignored.+ # Text in a tag without children is malformed, even whitespace.if not len(tag_el) and tag_el.text is not None:return [], "Unexpected complex element termination"if any(len(child) for child in tag_el if _local_tag_name(child) in ("Key", "Value")):@@ -872,8 +866,7 @@ def _function_request_tags(el):def _function_tag_constraint_errors(tags):- """AWS's member constraints on CreateFunction tags: per tag Value, then- Key; per member length, then pattern. Value may be absent."""+ """CreateFunction tag constraint messages: Value before Key, length before pattern."""errors = []for i, (key, value) in enumerate(tags, 1):for field, text, limit in (("value", value, 256), ("key", key, 128)):@@ -892,8 +885,7 @@ def _function_tag_constraint_errors(tags):def _function_code_and_runtime_errors(el):- """The functionCode, functionConfig and runtime constraint messages, in- that order, which AWS lists before the tag messages."""+ """The functionCode, functionConfig and runtime constraint messages, in AWS's order."""errors = []code_el = _find(el, "FunctionCode")if code_el is None:@@ -926,8 +918,7 @@ def _function_name_errors(el):def _function_tagging_error(tags):- """AWS's InvalidTagging checks, after the constraints: a repeated key, then- an aws: key, then more than 50 tags."""+ """InvalidTagging: a repeated key, then an aws: key, then more than 50 tags."""seen = set()for key, _ in tags:if key in seen:@@ -948,8 +939,6 @@ def _cf_create_function(headers, body):el = _parse_body(body)if el is None:return _error("MalformedXML", "The XML document is malformed.", 400)- # Parse errors first (a malformed tag before invalid base64, wherever- # each sits in the document), then the member constraints.tags, tags_parse_error = _function_request_tags(el)if tags_parse_error:return _error("MalformedInput", tags_parse_error, 400)@@ -959,8 +948,6 @@ def _cf_create_function(headers, body):if err is not None:return errname = _text(el, "Name")- # AWS checks the members before it looks the name up: the code, the- # config and runtime, the name and the tags, in one MalformedXML.errors = _function_code_and_runtime_errors(el) + _function_name_errors(el) + _function_tag_constraint_errors(tags)if errors:errors = errors[:_MAX_LISTED_VALIDATION_ERRORS]@@ -999,8 +986,6 @@ def _cf_create_function(headers, body):# The body PublishFunction froze; LIVE serves this, not the working copy."live_body": None,}- # The validated Tags/Items, as AWS reads them; ListTagsForResource lists- # them on the ARN, and a missing Value is an empty one._tags[fn["arn"]] = [{"Key": key, "Value": value or ""} for key, value in tags]_functions[name] = fnlogger.info("CreateFunction name=%s", name)diff --git a/ministack/services/cloudwatch.py b/ministack/services/cloudwatch.pyindex d62ef414..76bd61c9 100644--- a/ministack/services/cloudwatch.py+++ b/ministack/services/cloudwatch.py@@ -526,8 +526,7 @@ async def handle_request(method, path, headers, body, query_params):logger.error("JSON decode error: %s", e)cbor_data = {}else:- # Blank values kept: "Tags.member.2.Value=" is an empty value on- # AWS, while an absent member is null.+ # "Tags.member.2.Value=" is an empty value on AWS, an absent one is null.for k, v in parse_qs(body.decode("utf-8", errors="replace"), keep_blank_values=True).items():params[k] = v@@ -1079,15 +1078,12 @@ def _put_metric_alarm(params, cbor_data, is_cbor, is_json=False):if refused:return refused- # AWS describes DatapointsToAlarm only when the request sets it (absent- # and null alike); keep it off the record otherwise.+ # AWS describes DatapointsToAlarm only when the request sets it.if alarm["DatapointsToAlarm"] is None:del alarm["DatapointsToAlarm"]_alarms[name] = alarm- # PutMetricAlarm applies Tags on creation only; on an existing alarm they- # are ignored on every protocol (TagResource and UntagResource change them).- # A key sent twice keeps its last value.+ # Tags apply on create only; a key sent twice keeps its last value.if is_new and tags:_resource_tags[alarm["AlarmArn"]] = {t["Key"]: t["Value"] for t in tags}@@ -1115,9 +1111,7 @@ _QUERY_TAG_MEMBER = re.compile(r"Tags\.member\.([1-9][0-9]*)\.(Key|Value)")def _alarm_request_tags(params, cbor_data, is_cbor, is_json=False):- """An alarm request's Tags as sent, or None when absent. Query members are- read in index order, and one sent without Key or Value keeps that member- missing (AWS reports it as null)."""+ """An alarm request's Tags, Query members in index order, or None when absent."""if is_cbor or is_json:return cbor_data.get("Tags")members = {}@@ -1129,9 +1123,7 @@ def _alarm_request_tags(params, cbor_data, is_cbor, is_json=False):def _string_member_type_error(value):- """AWS's SerializationException text for a JSON/CBOR value sent where a- String is expected ("" where AWS sends no message), or None for a string- or null."""+ """AWS's message for a non-string JSON/CBOR Key or Value, or None."""if value is None or isinstance(value, str):return Noneif isinstance(value, bool):@@ -1146,8 +1138,7 @@ def _string_member_type_error(value):def _tags_serialization_error(tags):- """The first wrong JSON type in Tags, in document order, as (code, message,- status), or None."""+ """The first wrong JSON type in Tags as (code, message, status), or None."""if tags is None:return Noneif isinstance(tags, dict):@@ -1197,9 +1188,7 @@ _MAX_LISTED_VALIDATION_ERRORS = 10def _java_hashmap_order(keys):- """Distinct keys in the iteration order of a java.util.HashMap they were- put into: String.hashCode over UTF-16 units, spread (h ^ h >>> 16), default- capacity 16 doubled past a 0.75 load, insertion order within a bucket."""+ """Keys in java.util.HashMap iteration order, as AWS walks them."""unique = list(dict.fromkeys(keys))capacity = 16while len(unique) > capacity * 0.75:@@ -1216,11 +1205,7 @@ def _java_hashmap_order(keys):def _tagset_error(tags):- """The tag-set check AWS runs on create after the member constraints, on- the tags de-duplicated by key (a later value replaces an earlier one): a- character error anywhere, then the first too-long value, then the first- too-long key, in the tag map's iteration order. Length counts UTF-16- units; a tag whose key is too long is not checked further."""+ """The create-only tag-set check on the de-duplicated tags: characters, then length."""def too_long(text, limit):return len(text.encode("utf-16-le")) // 2 > limit@@ -1244,8 +1229,7 @@ def _tagset_error(tags):def _composite_system_tag_error(tags):- """PutCompositeAlarm's check of aws: keys, after the tag-set check, in the- same tag-map order (the first aws: key in that order is named)."""+ """PutCompositeAlarm's aws: key check, in the same tag-map order."""for key in _java_hashmap_order(tag["Key"] for tag in tags or []):if key.startswith("aws:"):if ":" not in key[4:]:@@ -1257,9 +1241,7 @@ def _composite_system_tag_error(tags):def _alarm_tags_refusal(tags, datapoints, is_new, is_cbor, is_json, composite=False):- """The error AWS answers for an alarm's Tags (and PutMetricAlarm's- DatapointsToAlarm), in AWS's order, or None. A refused call stores- nothing. The two calls differ only in how they refuse an aws: key."""+ """The refusal for an alarm's Tags and DatapointsToAlarm, in AWS's order, or None."""def refuse(code, message, status):return _error(code, "" if is_cbor and code == "MalformedInput" else message, status,use_json=is_json, use_cbor=is_cbor)@@ -1268,8 +1250,7 @@ def _alarm_tags_refusal(tags, datapoints, is_new, is_cbor, is_json, composite=Fawrong_type = _tags_serialization_error(tags)if wrong_type:return refuse(*wrong_type)- # The aws: prefix is refused only when the call creates the alarm (an- # update ignores its Tags), and before the member constraints.+ # An update ignores its Tags, so only a create refuses an aws: key.if is_new and not composite and any(isinstance(t.get("Key"), str) and t["Key"].lower().startswith("aws:")for t in tags or []):return refuse("AccessDenied", 'The provided tag keys must not start with any case of "aws:"', 403)@@ -1278,7 +1259,6 @@ def _alarm_tags_refusal(tags, datapoints, is_new, is_cbor, is_json, composite=Faerrors.append(f"Value '{datapoints}' at 'datapointsToAlarm' failed to satisfy constraint: ""Member must have value greater than or equal to 1")if errors:- # AWS lists at most ten, and counts only those.errors = errors[:_MAX_LISTED_VALIDATION_ERRORS]count = f"{len(errors)} validation error{'s' if len(errors) > 1 else ''} detected"return refuse("ValidationError", f"{count}: {'; '.join(errors)}", 400)@@ -1561,8 +1541,7 @@ def _put_composite_alarm(params, cbor_data, is_cbor, is_json=False):insuff_actions.append(_p(params, f"InsufficientDataActions.member.{ii}"))ii += 1- # Tags are checked before the rule, and stored only when the call creates- # the alarm (an update ignores them), as with PutMetricAlarm.+ # Tags are checked before the rule and stored on create only.is_new = name not in _composite_alarmstags = _alarm_request_tags(params, cbor_data, is_cbor, is_json)refused = _alarm_tags_refusal(tags, None, is_new, is_cbor, is_json, composite=True)diff --git a/ministack/services/cognito.py b/ministack/services/cognito.pyindex 6c25924b..5b1e1ba7 100644--- a/ministack/services/cognito.py+++ b/ministack/services/cognito.py@@ -4330,7 +4330,6 @@ def _mfa_challenge_for_user(pool: dict, user: dict, pid: str, username: str) ->mfa_config = pool.get("MfaConfiguration", "OFF")if mfa_config == "OFF":return None- preferred = user.get("_preferred_mfa", "")enabled_mfa = user.get("_mfa_enabled", [])# OPTIONAL: only challenge if user has TOTP set upif mfa_config == "OPTIONAL" and "SOFTWARE_TOKEN_MFA" not in enabled_mfa:@@ -5218,7 +5217,6 @@ def _confirmation_code_error(action: str, data: dict):def _confirm_sign_up(data):cid = data.get("ClientId")username = data.get("Username")- code = data.get("ConfirmationCode", "")pool = Nonefor p in _user_pools.values():diff --git a/ministack/services/documentdb.py b/ministack/services/documentdb.pyindex 01b1e566..f8caf76b 100644--- a/ministack/services/documentdb.py+++ b/ministack/services/documentdb.py@@ -987,8 +987,6 @@ def _create_db_instance(params):"ProcessorFeatures": [],"DeletionProtection": deletion_protection,"AssociatedRoles": [],- "MaxAllocatedStorage": int(- _evaluate_params(params, "MaxAllocatedStorage") or str(allocated_storage)),"TagList": [],"CustomerOwnedIpEnabled": False,"ActivityStreamStatus": "stopped",@@ -1862,9 +1860,6 @@ def _instance_xml(i):<ProcessorFeatures/><DeletionProtection>{str(i.get('DeletionProtection', False)).lower()}</DeletionProtection><AssociatedRoles/>- <MaxAllocatedStorage>- {i.get('MaxAllocatedStorage', i.get('AllocatedStorage', 20))}- </MaxAllocatedStorage><TagList>{tag_xml}</TagList>{cert_xml}<CustomerOwnedIpEnabled>diff --git a/ministack/services/dynamodb.py b/ministack/services/dynamodb.pyindex 1d91f0b6..17db027d 100644--- a/ministack/services/dynamodb.py+++ b/ministack/services/dynamodb.py@@ -46,7 +46,8 @@ from ministack.core.responses import (now_iso,request_scope,)-from ministack.services._dynamodb_keywords import AWS_KEYWORDS+from ministack.services._dynamodb_keywords import AWS_KEYWORDS, PartiqlError+from ministack.services._dynamodb_keywords import parse_statement as parse_partiql_statementlogger = logging.getLogger("dynamodb")@@ -276,18 +277,13 @@ _DDB_MAX_NESTING_DEPTH = 32def _ddb_canonicalize_number(s: str) -> str | None:- """Validate a DynamoDB Number string and return its canonical form, or None- if invalid. Mirrors AWS behavior: numbers are stored as variable-precision- decimals (up to 38 significant digits, magnitude between 1E-130 and- 9.9999E+125 inclusive). AWS canonicalizes: strips leading zeros, strips- trailing zeros after the decimal point, normalizes negative zero to "0".- Returns the canonical string the caller should persist, or None if the- value is out of range / malformed.- """+ """The canonical form of a Number string, or None when DynamoDB refuses it."""if s is None:return Noneif not isinstance(s, str) or not s:return None+ if _number_error(s):+ return None# AWS rejects numbers with whitespace or underscores even though Python's# Decimal() accepts them (e.g. " 5", "5 ", "1_000").if s != s.strip() or '_' in s:@@ -323,7 +319,6 @@ def _ddb_canonicalize_number(s: str) -> str | None:return None# Canonical string: use Decimal's normalize but format without trailing zeros# and without unnecessary leading zeros.- norm = d.normalize()# Decimal's normalize() returns scientific notation for very large/small;# we want the human-readable form when reasonable. Format manually.sign_str = "-" if sign else ""@@ -347,6 +342,36 @@ _NESTING_MSG = ("Nesting Levels have exceeded supported limits: ""Attributes in the item have nested levels beyond supported limit")+# Past this leaf level the single-item operations refuse a value by a second check.+_DDB_SECOND_NESTING_LEVEL = 62+++def _nesting_level(value, depth: int = 1) -> int:+ """Level of the deepest leaf, counting the attribute itself as level 1."""+ if not isinstance(value, dict) or len(value) != 1:+ return depth+ (vtype, vval), = value.items()+ children = vval.values() if vtype == "M" and isinstance(vval, dict) else (+ vval if vtype == "L" and isinstance(vval, list) else ())+ return max((_nesting_level(v, depth + 1) for v in children), default=depth)+++def _value_nesting_error(values: dict, first_level: int) -> tuple | None:+ """ExpressionAttributeValues nested past ``first_level`` are refused in the envelope, and past+ the second limit naming the key."""+ deepest = None+ for key, value in values.items():+ level = _nesting_level(value)+ if level > _DDB_SECOND_NESTING_LEVEL:+ return error_response_json("ValidationException",+ f"ExpressionAttributeValues contains invalid value: {_NESTING_MSG} for key {key}", 400)+ if level > first_level and deepest is None:+ deepest = key+ if deepest is not None:+ return error_response_json("ValidationException", f"1 validation error detected: {_NESTING_MSG}", 400)+ return None++def _nesting_exceeded(value, depth: int = 1) -> bool:if depth > _DDB_MAX_NESTING_DEPTH:return True@@ -360,6 +385,122 @@ def _nesting_exceeded(value, depth: int = 1) -> bool:return False+_NOT_NUMERIC = "The parameter cannot be converted to a numeric value"+_NUM_OVERFLOW = "Number overflow. Attempting to store a number with magnitude larger than supported range"+_NUM_UNDERFLOW = "Number underflow. Attempting to store a number with magnitude smaller than supported range"+_NUM_DIGITS = "Attempting to store more than 38 significant digits in a Number"+_NUMBER_TEXT_RE = re.compile(r"^[+-]?(\d+\.?\d*|\.\d+)(?:[eE]([+-]?\d+))?$")+++def _number_error(text) -> str | None:+ """Why DynamoDB refuses a number: the exponent is read as a 64-bit integer before the+ magnitude (1E-130 to 9.99E+125, zero by the exponent it is written with) and the 38 digits."""+ if not isinstance(text, str):+ return f"{_NOT_NUMERIC}: {text}"+ match = _NUMBER_TEXT_RE.match(text)+ if not match:+ return f"{_NOT_NUMERIC}: {text}"+ exponent = int(match.group(2) or 0)+ if not -(2 ** 63) <= exponent < 2 ** 63:+ return f"{_NOT_NUMERIC}: {text}"+ integer, _, fraction = match.group(1).partition(".")+ digits = (integer + fraction).lstrip("0")+ # BigDecimal's precision - scale - 1; zero has a precision of 1.+ adjusted = max(len(digits), 1) - (len(fraction) - exponent) - 1+ if adjusted > _DDB_NUM_POS_MAX_EXP - 1:+ return _NUM_OVERFLOW+ if adjusted < _DDB_NUM_NEG_MIN_EXP:+ return _NUM_UNDERFLOW+ if len(digits.rstrip("0")) > _DDB_NUM_MAX_DIGITS:+ return _NUM_DIGITS+ return None+++def _duplicate_set_message(vtype: str, members: list) -> str | None:+ """A set is read into a sorted set: the message lists what was read, in byte order, then the+ repeat, then the members not yet read."""+ if vtype == "NS":+ canon = [_ddb_canonicalize_number(m) if isinstance(m, str) else m for m in members]+ if len(set(canon)) == len(canon):+ return None+ return "One or more parameter values were invalid: Input collection contains duplicates."+ def order(m):+ if vtype == "BS":+ try:+ return base64.b64decode(m) if isinstance(m, str) else bytes(m)+ except (ValueError, TypeError):+ return str(m).encode()+ return str(m).encode("utf-8")+ seen = {}+ for i, member in enumerate(members):+ k = order(member)+ if k in seen:+ shown = [seen[x] for x in sorted(seen)] + [member] + list(members[i + 1:])+ if vtype == "BS":+ return ("One or more parameter values were invalid: Input collection "+ f"{json.dumps(shown, separators=(',', ':'))}of type BS contains duplicates.")+ return ("One or more parameter values were invalid: Input collection "+ f"{json.dumps(shown, ensure_ascii=False)} contains duplicates.")+ seen[k] = member+ return None+++def _attribute_value_error(value) -> str | None:+ """The first bad number or duplicate set member in a value, in document order."""+ if not isinstance(value, dict) or len(value) != 1:+ return None+ (vtype, vval), = value.items()+ if vtype == "N":+ return _number_error(vval)+ if vtype in ("SS", "NS", "BS") and isinstance(vval, list):+ if vtype == "NS":+ for member in vval:+ err = _number_error(member)+ if err:+ return err+ return _duplicate_set_message(vtype, vval)+ if vtype == "M" and isinstance(vval, dict):+ children = vval.values()+ elif vtype == "L" and isinstance(vval, list):+ children = vval+ else:+ return None+ for child in children:+ err = _attribute_value_error(child)+ if err:+ return err+ return None+++def _request_values(data) -> list:+ """Every attribute value a single-table request carries."""+ values = []+ for field in ("Item", "Key", "ExclusiveStartKey", "ExpressionAttributeValues"):+ container = data.get(field)+ if isinstance(container, dict):+ values.extend(container.values())+ for field in ("AttributeUpdates", "Expected"):+ for spec in (data.get(field) or {}).values() if isinstance(data.get(field), dict) else ():+ if isinstance(spec, dict):+ if "Value" in spec:+ values.append(spec["Value"])+ values.extend(spec.get("AttributeValueList") or [])+ for field in ("KeyConditions", "QueryFilter"):+ for spec in (data.get(field) or {}).values() if isinstance(data.get(field), dict) else ():+ if isinstance(spec, dict):+ values.extend(spec.get("AttributeValueList") or [])+ return values+++def _values_error_response(values, envelope: bool) -> tuple | None:+ for value in values:+ err = _attribute_value_error(value)+ if err:+ return error_response_json("ValidationException",+ f"1 validation error detected: {err}" if envelope else err, 400)+ return None++def _validate_attribute_value(attr_name: str, value: dict, _depth: int = 1) -> tuple | None:"""Recursively validate a single attribute value. Returns an error responsetuple if invalid, or None if OK. May mutate `value` to canonicalize numbers.@@ -580,9 +721,40 @@ _ITEM_SIZE_MSG = "Item size has exceeded the maximum allowed size"_UPDATE_SIZE_MSG = "Item size to update has exceeded the maximum allowed size"-def _update_statement_size(item: dict, expr: str, attr_names: dict) -> int:- """UpdateItem's size: the top-level attributes it writes plus 3, 19 per SET/ADD- action (20 through a list index) and 2 per REMOVE/DELETE action."""+def _update_request_charge(expr: str, attr_values: dict, attr_names: dict, item_before: dict) -> int:+ """UpdateItem's charge for the request: 3, plus 19 and the target and operands per SET or ADD,+ 1 and the target per REMOVE or DELETE. A target costs its first name and 4 plus each later+ element; a path operand 1 and 14 plus each element; a value its size at every use; + and -+ 34 and 37, if_not_exists 39 and list_append 37 plus their operands."""+ names = attr_names or {}+ values = attr_values or {}++ def element(part):+ return len(str(part)) if isinstance(part, int) else len(part.encode("utf-8"))++ def target_cost(path):+ return element(path[0]) + sum(4 + element(p) for p in path[1:]) if path else 0++ def operand(tokens):+ if not tokens:+ return 0+ if tokens[0][0] == "VALUE_REF" and len(tokens) == 1:+ return _attribute_value_size(values.get(tokens[0][1]))+ if tokens[0][0] == "IDENT" and len(tokens) > 1 and tokens[1][0] == "LPAREN":+ close = _find_matching_paren(tokens, 1)+ args = sum(expression(arg) for arg in _split_by_comma(tokens[2:close]))+ return {"if_not_exists": 39, "list_append": 37}.get(tokens[0][1], 0) + args+ return 1 + sum(14 + element(p) for p in _parse_path_from_tokens(tokens, names))++ def expression(tokens):+ depth = 0+ for i, tok in enumerate(tokens):+ depth += tok[0] == "LPAREN"+ depth -= tok[0] == "RPAREN"+ if depth == 0 and tok[0] in ("PLUS", "MINUS") and i > 0:+ return (34 if tok[0] == "PLUS" else 37) + operand(tokens[:i]) + operand(tokens[i + 1:])+ return operand(tokens)+clauses = []for tok in _tokenize(expr):if tok[0] == "IDENT" and tok[1].upper() in ("SET", "REMOVE", "ADD", "DELETE"):@@ -590,18 +762,24 @@ def _update_statement_size(item: dict, expr: str, attr_names: dict) -> int:elif tok[0] != "EOF" and clauses:clauses[-1][1].append(tok)size = 3- written = set()for clause, tokens in clauses:for action in _split_by_comma(tokens):- path = _parse_path_from_tokens(action, attr_names or {})- if not path:- continue- size += 19 if clause in ("SET", "ADD") else 2- if clause == "SET" and any(isinstance(p, int) for p in path):- size += 1- written.add(path[0])- return size + sum(len(n.encode("utf-8")) + _attribute_value_size(item[n]) for n in written if n in item)-+ if clause == "SET":+ eq = next((i for i, t in enumerate(action) if t[0] == "EQ"), len(action))+ size += 19 + target_cost(_parse_path_from_tokens(action[:eq], names)) + expression(action[eq + 1:])+ elif clause == "ADD":+ path = _parse_path_from_tokens(action[:-1], names)+ added = values.get(action[-1][1]) if action[-1][0] == "VALUE_REF" else None+ current = _get_at_path(item_before, path) if path else None+ if isinstance(added, dict) and isinstance(current, dict):+ (atype, members), = added.items()+ if atype in ("SS", "NS", "BS") and atype in current:+ added = {atype: [m for m in members if m not in current[atype]]}+ size += 19 + target_cost(path) + _attribute_value_size(added)+ else:+ path_tokens = action[:-1] if clause == "DELETE" and action[-1][0] == "VALUE_REF" else action+ size += 1 + target_cost(_parse_path_from_tokens(path_tokens, names))+ return sizedef _key_length_error(item: dict, pk_name: str | None, sk_name: str | None) -> tuple | None:# Key attribute value length limits: partition key max 2048 bytes, sort key max 1024 bytes.@@ -1092,6 +1270,118 @@ def _validate_data_plane_table_name(name) -> tuple | None:return None+# Regions that answer differently from the rest, per the paritysuite.org split registry (2026-10-22).+_OLDER_VALIDATION_REGIONS = frozenset(("eu-south-2", "me-central-1"))+_OLDER_BATCH_GET_REGIONS = frozenset(("me-central-1",))+_ECHOING_SCAN_REGIONS = frozenset((+ "ap-northeast-1", "ap-south-2", "ap-southeast-6", "ap-southeast-7", "ca-central-1", "eu-central-2",+ "eu-south-1", "eu-south-2", "eu-west-1", "me-central-1", "us-east-1", "us-east-2", "us-west-2"))+++def _older_validation_region() -> bool:+ return get_region() in _OLDER_VALIDATION_REGIONS+++def _older_member_errors(data, op: str) -> tuple | None:+ """Where PutItem (empty table name and bad ReturnValues) and UpdateItem (bad ReturnValues and+ ReturnConsumedCapacity) still join their member errors in the older per-value form."""+ rv, rcc = data.get("ReturnValues"), data.get("ReturnConsumedCapacity")+ bad_rv = rv is not None and rv not in _UPDATE_RV_VALUES+ bad_rcc = rcc is not None and rcc not in _RETURN_CONSUMED_CAPACITY_VALUES+ if op == "put" and not (data.get("TableName") == "" and bad_rv and not bad_rcc+ and data.get("ReturnItemCollectionMetrics") is None):+ return None+ if op == "update" and not (bad_rv and bad_rcc and data.get("TableName") and "Key" in data+ and data.get("ReturnItemCollectionMetrics") is None):+ return None+ pieces = []+ rcc = data.get("ReturnConsumedCapacity")+ if rcc is not None and rcc not in _RETURN_CONSUMED_CAPACITY_VALUES:+ pieces.append(f"Value '{rcc}' at 'returnConsumedCapacity' failed to satisfy constraint: "+ "Member must satisfy enum value set: [INDEXES, TOTAL, NONE]")+ rv = data.get("ReturnValues")+ if rv is not None and rv not in _UPDATE_RV_VALUES:+ pieces.append(f"Value '{rv}' at 'returnValues' failed to satisfy constraint: {_OLDER_RV_ENUM}")+ if data.get("TableName") == "":+ pieces.append("Value '' at 'tableName' failed to satisfy constraint: Member must have length greater than or equal to 1")+ if not pieces:+ return None+ return error_response_json("ValidationException",+ f"{len(pieces)} validation error{'s' if len(pieces) > 1 else ''} detected: " + "; ".join(pieces), 400)+++def _older_layer_error(data, op: str) -> tuple | None:+ """PutItem, UpdateItem and DeleteItem read a request without a Key, or with a value nested past+ the second limit, in an older form: member errors joined in member order, before any table."""+ key_missing = op != "put" and "Key" not in data+ if not key_missing and max((_nesting_level(v) for v in _request_values(data)), default=0) <= _DDB_SECOND_NESTING_LEVEL:+ return None+ pieces = []+ rcc = data.get("ReturnConsumedCapacity")+ if rcc is not None and rcc not in _RETURN_CONSUMED_CAPACITY_VALUES:+ pieces.append(f"Value '{rcc}' at 'returnConsumedCapacity' failed to satisfy constraint: "+ "Member must satisfy enum value set: [INDEXES, TOTAL, NONE]")+ if data.get("TableName") == "":+ pieces.append("Value '' at 'tableName' failed to satisfy constraint: Member must have length greater than or equal to 1")+ rv = data.get("ReturnValues")+ if rv is not None and rv not in _UPDATE_RV_VALUES:+ pieces.append(f"Value '{rv}' at 'returnValues' failed to satisfy constraint: {_OLDER_RV_ENUM}")+ co = data.get("ConditionalOperator")+ if co is not None and co not in ("AND", "OR"):+ pieces.append(f"Value '{co}' at 'conditionalOperator' failed to satisfy constraint: "+ "Member must satisfy enum value set: [OR, AND]")+ if key_missing:+ pieces.append("Value null at 'key' failed to satisfy constraint: Member must not be null")+ if pieces:+ return error_response_json("ValidationException",+ f"{len(pieces)} validation error{'s' if len(pieces) > 1 else ''} detected: " + "; ".join(pieces), 400)+ if op == "delete" and rv in _UPDATE_RV_VALUES - _PUT_DELETE_RV_VALUES:+ return error_response_json("ValidationException", "ReturnValues can only be ALL_OLD or NONE", 400)+ return None+++def _expected_error(expected) -> str | None:+ """The first malformed legacy Expected condition."""+ if not isinstance(expected, dict):+ return None+ for name, cond in expected.items():+ if not isinstance(cond, dict):+ continue+ has_value, has_list = "Value" in cond, "AttributeValueList" in cond+ op = cond.get("ComparisonOperator")+ if has_value and has_list:+ return f"Value and AttributeValueList cannot be used together for Attribute: {name}"+ if cond.get("Exists") is True and not has_value and not op:+ return f"Value must be provided when Exists is true for Attribute: {name}"+ if cond.get("Exists") is False and has_value:+ return f"Value cannot be used when Exists is false for Attribute: {name}"+ if op and not has_value and not has_list:+ return f"Value or AttributeValueList must be used with ComparisonOperator: {op} for Attribute: {name}"+ if op == "BETWEEN" and len(cond.get("AttributeValueList") or []) != 2:+ return "Invalid number of argument(s) for the BETWEEN ComparisonOperator"+ return None+++def _single_write_request_error(data) -> tuple | None:+ """Request checks PutItem, UpdateItem and DeleteItem report inside the validation envelope."""+ for field in ("ExpressionAttributeValues", "ExpressionAttributeNames"):+ if field in data and data[field] == {}:+ return error_response_json("ValidationException", f"1 validation error detected: {field} must not be empty", 400)+ if not any(data.get(f) for f in ("ConditionExpression", "UpdateExpression")):+ for field in ("ExpressionAttributeValues", "ExpressionAttributeNames"):+ if data.get(field):+ return error_response_json("ValidationException",+ f"1 validation error detected: {field} can only be specified when using expressions", 400)+ if "ConditionExpression" in data and isinstance(data["ConditionExpression"], str) and not data["ConditionExpression"].strip():+ return error_response_json("ValidationException",+ "1 validation error detected: Invalid ConditionExpression: The expression can not be empty;", 400)+ err = _expected_error(data.get("Expected"))+ if err:+ return error_response_json("ValidationException",+ f"1 validation error detected: One or more parameter values were invalid: {err}", 400)+ return None++def _multi_validation_error(failures: list[tuple]) -> tuple | None:"""AWS returns ONE ValidationException with all simultaneously-invalidparameters concatenated:@@ -1126,6 +1416,15 @@ def _check_per_op_param_enums(data: dict, rv_allowed: set | None, first_only: bo_TABLE_NAME_RE = re.compile(r"^[A-Za-z0-9_.\-]+$")_INDEX_NAME_RE = re.compile(r"^[A-Za-z0-9_.\-]+$")+++def _malformed_table_reference(name) -> bool:+ """A data-plane table reference is a table name or a table ARN."""+ if not isinstance(name, str):+ return False+ if name.startswith("arn:"):+ return len(name) > 1024+ return not 3 <= len(name) <= 255 or not _TABLE_NAME_RE.match(name)_VALID_ATTR_TYPES = {"S", "N", "B"}_VALID_KEY_TYPES = {"HASH", "RANGE"}_VALID_BILLING_MODES = {"PROVISIONED", "PAY_PER_REQUEST"}@@ -1217,6 +1516,23 @@ def _create_table(data):"The parameter 'TableName' is required but was not present in the request", 400)name = data.get("TableName")err = _validate_table_name(name)+ if err:+ return err+ # Index names are member constraints, GSIs before LSIs whatever the request's order.+ failures = []+ for field, member in (("GlobalSecondaryIndexes", "globalSecondaryIndexes"), ("LocalSecondaryIndexes", "localSecondaryIndexes")):+ for position, index in enumerate(data.get(field) or [], start=1):+ iname = index.get("IndexName") if isinstance(index, dict) else None+ if not isinstance(iname, str):+ continue+ path = f"{member}.{position}.member.indexName"+ if len(iname) < 3:+ failures.append((iname, path, "Member must have length greater than or equal to 3"))+ elif len(iname) > 255:+ failures.append((iname, path, "Member must have length less than or equal to 255"))+ elif not _INDEX_NAME_RE.match(iname):+ failures.append((iname, path, "Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+"))+ err = _multi_validation_error(failures)if err:return errvector_indexes = data.get("VectorIndexes") or []@@ -1306,7 +1622,6 @@ def _create_table(data):# Validate index Projection settings.for idx in gsis + lsis:- idx_name = idx.get("IndexName", "<unnamed>")proj = idx.get("Projection") or {}ptype = proj.get("ProjectionType", "ALL")if ptype not in _VALID_PROJECTION_TYPES:@@ -1786,6 +2101,7 @@ def _table_description(name):# ---------------------------------------------------------------------------_PUT_DELETE_RV_VALUES = {"NONE", "ALL_OLD"}+_OLDER_RV_ENUM = "Member must satisfy enum value set: [ALL_NEW, UPDATED_OLD, ALL_OLD, NONE, UPDATED_NEW]"_UPDATE_RV_VALUES = {"NONE", "ALL_OLD", "ALL_NEW", "UPDATED_OLD", "UPDATED_NEW"}_RETURN_ITEM_COLLECTION_METRICS = {"NONE", "SIZE"}_RETURN_CONSUMED_CAPACITY_VALUES = {"NONE", "TOTAL", "INDEXES"}@@ -1850,34 +2166,40 @@ def _projection_overlap_response(data) -> tuple | None:return error_response_json("ValidationException", message, 400) if message else None-def _expression_size_error(data, expression_fields: tuple) -> tuple | None:- """Each expression is capped at 4096 bytes of its raw text."""+def _expression_size_error(data, expression_fields: tuple, wording: str | dict = "figure") -> tuple | None:+ """Each expression is capped at 4096 bytes of its raw text. PutItem and UpdateItem wrap the+ sentence in the validation envelope, reads and transactions give it bare (``wording``)."""for fname in expression_fields:body = data.get(fname) or ""size = len(body.encode("utf-8")) if isinstance(body, str) else 0if size > _DDB_EXPR_MAX_BYTES:- return error_response_json("ValidationException",- f"Invalid {fname}: Expression size has exceeded the maximum allowed size; expression size: {size}", 400)+ style = wording.get(fname, "figure") if isinstance(wording, dict) else wording+ message = f"Invalid {fname}: Expression size has exceeded the maximum allowed size;"+ if style == "envelope":+ message = f"1 validation error detected: {message}"+ elif style == "figure":+ message += f" expression size: {size}"+ return error_response_json("ValidationException", message, 400)return None-def _validate_expression_attrs(data, expression_fields: tuple) -> tuple | None:- """AWS rejects ExpressionAttributeValues / ExpressionAttributeNames when- no expression field references them at all, AND when any defined alias- isn't used by any of the expressions, AND when any `:foo` or `#bar`- referenced by an expression isn't defined."""- # Expression string length limit: 4096 bytes each.- err = _expression_size_error(data, expression_fields)+def _envelope_condition_refs(err: tuple | None) -> tuple | None:+ """A single-item write reports an undefined reference in its condition inside the envelope."""+ if err and b'"Invalid ConditionExpression: An expression attribute' in err[2]:+ message = json.loads(err[2])["message"]+ return error_response_json("ValidationException", f"1 validation error detected: {message}", 400)+ return err+++def _validate_expression_attrs(data, expression_fields: tuple, size_wording: str | dict = "figure") -> tuple | None:+ """Expression size, then names and values defined but unused, or used but undefined."""+ err = _expression_size_error(data, expression_fields, size_wording)if err:return errhas_any_expr = any(data.get(f) for f in expression_fields)eav = data.get("ExpressionAttributeValues")ean = data.get("ExpressionAttributeNames")if eav and not has_any_expr:- # AWS-canonical: EAV without any expression is rejected with the- # "can only be used when..." wording. Real AWS appends the first- # expression-slot name ("<Field> is null") for write ops; for ops- # with multiple expression slots the suffix is omitted.suffix = ""if len(expression_fields) == 1:suffix = f": {expression_fields[0]} is null"@@ -1898,10 +2220,6 @@ def _validate_expression_attrs(data, expression_fields: tuple) -> tuple | None:if alias not in expr_text:return error_response_json("ValidationException",f"Value provided in ExpressionAttributeNames unused in expressions: keys: {{{alias}}}", 400)- # Inverse: every `:foo` / `#bar` in expressions must be defined.- # AWS scopes the error to the *specific* expression that contains the bad- # reference: "Invalid FilterExpression: An expression attribute value- # used in expression is not defined; attribute value: :v".for fname in expression_fields:body = data.get(fname) or ""if not body:@@ -1959,6 +2277,9 @@ def _add_item_collection_metrics(result: dict, data: dict, table: dict, item: didef _put_item(data):+ err = (_older_validation_region() and _older_member_errors(data, "put")) or _older_layer_error(data, "put")+ if err:+ return errname = _normalize_table_name(data.get("TableName"))err = _validate_data_plane_table_name(name)if err:@@ -1970,11 +2291,18 @@ def _put_item(data):if not table:return error_response_json("ResourceNotFoundException","Requested resource not found", 400)- err = _validate_expression_attrs(data, ("ConditionExpression",))+ err = _values_error_response(_request_values(data), envelope=True) or _single_write_request_error(data)+ if err:+ return err+ err = _envelope_condition_refs(_validate_expression_attrs(data, ("ConditionExpression",), "envelope"))if err:return erritem = data.get("Item", {})+ if isinstance(item, dict):+ level = max((_nesting_level(v) for v in item.values()), default=0)+ if _DDB_MAX_NESTING_DEPTH < level <= _DDB_SECOND_NESTING_LEVEL:+ return error_response_json("ValidationException", f"1 validation error detected: {_NESTING_MSG}", 400)err = _validate_item(item, table.get("pk_name"), table.get("sk_name"))if err:return err@@ -2039,6 +2367,9 @@ def _get_item(data):if not table:return error_response_json("ResourceNotFoundException","Requested resource not found", 400)+ err = _values_error_response(_request_values(data), envelope=False)+ if err:+ return errif data.get("ProjectionExpression") and data.get("AttributesToGet"):return error_response_json("ValidationException","Can not use both expression and non-expression parameters in the same request: Non-expression parameters: {AttributesToGet} Expression parameters: {ProjectionExpression}", 400)@@ -2047,7 +2378,7 @@ def _get_item(data):_pe_err = _validate_projection_expression_syntax(_pe)if _pe_err:return error_response_json("ValidationException", _pe_err, 400)- err = _validate_expression_attrs(data, ("ProjectionExpression",)) or _projection_overlap_response(data)+ err = _validate_expression_attrs(data, ("ProjectionExpression",), "bare") or _projection_overlap_response(data)if err:return err@@ -2068,10 +2399,16 @@ def _get_item(data):def _delete_item(data):+ err = _older_layer_error(data, "delete")+ if err:+ return errname = _normalize_table_name(data.get("TableName"))err = _validate_data_plane_table_name(name)if err:return err+ if data.get("ReturnValues") in _UPDATE_RV_VALUES - _PUT_DELETE_RV_VALUES:+ return error_response_json("ValidationException",+ "1 validation error detected: ReturnValues can only be ALL_OLD or NONE", 400)err = _check_per_op_param_enums(data, _PUT_DELETE_RV_VALUES)if err:return err@@ -2079,7 +2416,10 @@ def _delete_item(data):if not table:return error_response_json("ResourceNotFoundException","Requested resource not found", 400)- err = _validate_expression_attrs(data, ("ConditionExpression",))+ err = _values_error_response(_request_values(data), envelope=True) or _single_write_request_error(data)+ if err:+ return err+ err = _envelope_condition_refs(_validate_expression_attrs(data, ("ConditionExpression",)))if err:return err@@ -2127,6 +2467,9 @@ def _key_attribute_update_error(table, updated_attrs):def _update_item(data):+ err = (_older_validation_region() and _older_member_errors(data, "update")) or _older_layer_error(data, "update")+ if err:+ return errname = _normalize_table_name(data.get("TableName"))err = _validate_data_plane_table_name(name)if err:@@ -2139,6 +2482,9 @@ def _update_item(data):if not table:return error_response_json("ResourceNotFoundException","Requested resource not found", 400)+ err = _values_error_response(_request_values(data), envelope=True) or _single_write_request_error(data)+ if err:+ return err# Pre-validate UpdateExpression syntax BEFORE the unused-EAV check.# AWS reports `"Invalid UpdateExpression: Syntax error; token: <first>,# near: <first second>"` for a body that doesn't start with a clause@@ -2150,11 +2496,13 @@ def _update_item(data):_near = " ".join(_ue_tokens[:2])return error_response_json("ValidationException",f'Invalid UpdateExpression: Syntax error; token: "{_ue_tokens[0]}", near: "{_near}"', 400)- err = _validate_expression_attrs(data, ("ConditionExpression", "UpdateExpression"))+ err = _envelope_condition_refs(_validate_expression_attrs(data, ("ConditionExpression", "UpdateExpression"), "envelope"))+ if err:+ return err+ first_limit = _DDB_SECOND_NESTING_LEVEL if _older_validation_region() else _DDB_MAX_NESTING_DEPTH+ err = _value_nesting_error(data.get("ExpressionAttributeValues") or {}, first_limit)if err:return err- if any(_nesting_exceeded(v) for v in (data.get("ExpressionAttributeValues") or {}).values()):- return error_response_json("ValidationException", f"1 validation error detected: {_NESTING_MSG}", 400)key = data.get("Key", {})pk_val, sk_val, key_err = _resolve_table_key_values(table, key, allow_extra=False)@@ -2164,6 +2512,10 @@ def _update_item(data):existing = table["items"].get(pk_val, {}).get(sk_val)old_item = copy.deepcopy(existing) if existing else Noneitem = copy.deepcopy(existing) if existing else dict(key)+ update_text = data.get("UpdateExpression")+ if update_text and _update_request_charge(update_text, data.get("ExpressionAttributeValues"),+ data.get("ExpressionAttributeNames"), existing or {}) > _DDB_ITEM_MAX_BYTES:+ return error_response_json("ValidationException", _UPDATE_SIZE_MSG, 400)cond_expr = data.get("ConditionExpression")expected = data.get("Expected")@@ -2236,8 +2588,6 @@ def _update_item(data):err = _validate_item(item, table.get("pk_name"), table.get("sk_name"), _UPDATE_SIZE_MSG)if err:return err- if update_expr and _update_statement_size(item, update_expr, ean) > _DDB_ITEM_MAX_BYTES:- return error_response_json("ValidationException", _UPDATE_SIZE_MSG, 400)# Validate GSI/LSI key attribute types and values after the update is applied.err = _validate_index_key_values(table, item, update_expr=bool(data.get("UpdateExpression")))if err:@@ -2271,6 +2621,59 @@ def _update_item(data):# Query / Scan# ---------------------------------------------------------------------------+_KEY_OPERAND_TYPES = {"begins_with": {"S", "B"}}+++def _key_condition_clauses(tokens) -> list:+ """Top-level AND-separated clauses; the AND inside BETWEEN stays in its clause."""+ clauses, current, depth, in_between = [], [], 0, False+ for tok in tokens:+ if tok[0] == "EOF":+ break+ depth += tok[0] == "LPAREN"+ depth -= tok[0] == "RPAREN"+ is_and = tok[0] == "IDENT" and tok[1].upper() == "AND" and depth == 0+ if is_and and in_between:+ in_between = False+ elif is_and:+ clauses.append(current)+ current = []+ continue+ elif tok[0] == "IDENT" and tok[1].upper() == "BETWEEN":+ in_between = True+ current.append(tok)+ if current:+ clauses.append(current)+ return clauses+++def _key_condition_operand_error(key_cond: str, eav: dict) -> tuple | None:+ """Clauses in written order: a value the request never defined, or an operand no key type fits."""+ for clause in _key_condition_clauses(_tokenize(key_cond)):+ idents = {tok[1].lower() for tok in clause if tok[0] == "IDENT"}+ if "begins_with" in idents:+ op, allowed = "begins_with", _KEY_OPERAND_TYPES["begins_with"]+ elif "between" in idents:+ op, allowed = "BETWEEN", {"S", "N", "B"}+ else:+ ordering = next((tok[1] for tok in clause if tok[0] in ("LT", "LE", "GT", "GE")), None)+ op, allowed = ordering, ({"S", "N", "B"} if ordering else None)+ for tok in clause:+ if tok[0] != "VALUE_REF":+ continue+ value = (eav or {}).get(tok[1])+ if value is None:+ return error_response_json("ValidationException",+ "Invalid KeyConditionExpression: An expression attribute value used in expression is not defined; "+ f"attribute value: {tok[1]}", 400)+ vtype = next(iter(value)) if isinstance(value, dict) and value else None+ if allowed and vtype not in allowed:+ return error_response_json("ValidationException",+ "Invalid KeyConditionExpression: Incorrect operand type for operator or function; "+ f"operator or function: {op}, operand type: {vtype}", 400)+ return None++def _query(data):name = _normalize_table_name(data.get("TableName"))err = _validate_data_plane_table_name(name)@@ -2279,10 +2682,20 @@ def _query(data):err = _check_per_op_param_enums(data, None)if err:return err+ if data.get("ConditionalOperator") is not None and data["ConditionalOperator"] not in ("AND", "OR"):+ return error_response_json("ValidationException",+ "Failed to satisfy constraint: Member must satisfy enum value set: [ALL, OR]", 400)+ if data.get("KeyConditionExpression"):+ err = _key_condition_operand_error(data["KeyConditionExpression"], data.get("ExpressionAttributeValues"))+ if err:+ return errtable = _tables.get(name)if not table:return error_response_json("ResourceNotFoundException","Requested resource not found", 400)+ err = _values_error_response(_request_values(data), envelope=False)+ if err:+ return err# Non-existent IndexName → ValidationException (not ResourceNotFoundException).idx_req = data.get("IndexName")if idx_req:@@ -2308,7 +2721,8 @@ def _query(data):return error_response_json("ValidationException", _err, 400)except Exception:pass- err = (_validate_expression_attrs(data, ("KeyConditionExpression", "FilterExpression", "ProjectionExpression"))+ err = (_validate_expression_attrs(data, ("KeyConditionExpression", "FilterExpression", "ProjectionExpression"),+ {"FilterExpression": "bare", "ProjectionExpression": "bare"})or _projection_overlap_response(data))if err:return err@@ -2401,6 +2815,20 @@ def _query(data):return error_response_json("ValidationException","The provided starting key is invalid", 400)+ if key_cond:+ named = []+ for clause in _key_condition_clauses(_tokenize(key_cond)):+ ref = next((t for t in clause if t[0] in ("IDENT", "NAME_REF") and t[1].lower() not in _DDB_EXPR_FUNCTIONS+ and t[1].upper() not in ("AND", "BETWEEN")), None)+ if ref:+ named.append(ean.get(ref[1], ref[1]) if ref[0] == "NAME_REF" else ref[1])+ if len(set(named)) < len(named):+ return error_response_json("ValidationException",+ "Invalid KeyConditionExpression: KeyConditionExpressions must only contain one condition per key", 400)+ key_count = len([n for n in hash_names + range_names if n])+ if len(named) > key_count:+ return error_response_json("ValidationException",+ f"The number of query conditions ({len(named)}) exceeds the number of key attributes defined in the schema ({key_count})", 400)# Every partition-key attribute needs an equality condition; a# multi-attribute partition is the tuple of their values.pk_vals = []@@ -2457,9 +2885,9 @@ def _query(data):f"Query condition missed key schema element: {resolved}", 400)# Multi-attribute sort keys are queried left to right without gaps,# and only the last one queried may use a range condition.- if len(range_names) > 1 and not _multi_sort_condition_ok(kce_tokens, ean, range_names):- return error_response_json("ValidationException",- "Query key condition not supported", 400)+ multi_err = _multi_sort_condition_error(kce_tokens, ean, range_names) if len(range_names) > 1 else None+ if multi_err:+ return error_response_json("ValidationException", multi_err, 400)# Key-condition operands are validated like key values themselves:# an empty string/binary operand is rejected with the same error AWS# raises for empty key attribute values.@@ -2592,6 +3020,10 @@ def _scan(data):err = _check_per_op_param_enums(data, None)if err:return err+ scan_filter = data.get("ScanFilter")+ if data.get("ConditionalOperator") and isinstance(scan_filter, dict) and len(scan_filter) < 2:+ return error_response_json("ValidationException",+ "ConditionalOperator can only be used when Filter or Expected has two or more elements", 400)table = _tables.get(name)if not table:return error_response_json("ResourceNotFoundException",@@ -2628,7 +3060,7 @@ def _scan(data):if _t and _t not in ("S", "B"):return error_response_json("ValidationException",f"Invalid FilterExpression: Incorrect operand type for operator or function; operator or function: begins_with, operand type: {_t}", 400)- err = _validate_expression_attrs(data, ("FilterExpression", "ProjectionExpression")) or _projection_overlap_response(data)+ err = _validate_expression_attrs(data, ("FilterExpression", "ProjectionExpression"), {"FilterExpression": "bare"}) or _projection_overlap_response(data)if err:return err@@ -2646,9 +3078,11 @@ def _scan(data):select = "SPECIFIC_ATTRIBUTES" if (data.get("ProjectionExpression") or data.get("AttributesToGet")) else "ALL_ATTRIBUTES"# Limit must be > 0 when provided (AWS rejects Limit=0).+ echoing = get_region() in _ECHOING_SCAN_REGIONSif limit is not None and int(limit) <= 0:+ member = f"Value '{limit}' at 'limit'" if echoing else "Value at 'Limit'"return error_response_json("ValidationException",- "1 validation error detected: Value at 'Limit' failed to satisfy constraint: Member must have value greater than or equal to 1", 400)+ f"1 validation error detected: {member} failed to satisfy constraint: Member must have value greater than or equal to 1", 400)if data.get("ProjectionExpression") and data.get("AttributesToGet"):return error_response_json("ValidationException","Can not use both expression and non-expression parameters in the same request: Non-expression parameters: {AttributesToGet} Expression parameters: {ProjectionExpression}", 400)@@ -2673,11 +3107,13 @@ def _scan(data):# envelope with the lowercase 'segment' slot and "greater than or# equal to 0" floor — distinct from the segment>=totalSegments error.if seg < 0:+ member = f"Value '{seg}' at 'segment'" if echoing else "Value at 'Segment'"return error_response_json("ValidationException",- "1 validation error detected: Value at 'Segment' failed to satisfy constraint: Member must have value greater than or equal to 0", 400)+ f"1 validation error detected: {member} failed to satisfy constraint: Member must have value greater than or equal to 0", 400)if seg >= ts:+ detail = f": Segment: {seg} is not less than TotalSegments: {ts}" if echoing else ""return error_response_json("ValidationException",- "The Segment parameter is zero-based and must be less than parameter TotalSegments", 400)+ f"The Segment parameter is zero-based and must be less than parameter TotalSegments{detail}", 400)# Select validation per AWS (messages measured against real DynamoDB by# paritysuite — Scan reuses the Query-worded IndexName message verbatim).if "Select" in data and data.get("ProjectionExpression"):@@ -2695,7 +3131,8 @@ def _scan(data):"ALL_PROJECTED_ATTRIBUTES can be used only when Querying using an IndexName", 400)if select == "SPECIFIC_ATTRIBUTES" and not data.get("ProjectionExpression") and not data.get("AttributesToGet"):return error_response_json("ValidationException",- "1 validation error detected: Must specify the AttributesToGet or ProjectionExpression when choosing to get SPECIFIC_ATTRIBUTES", 400)+ ("" if echoing else "1 validation error detected: ")+ + "Must specify the AttributesToGet or ProjectionExpression when choosing to get SPECIFIC_ATTRIBUTES", 400)# ConsistentRead on a GSI is invalid.if index_name and data.get("ConsistentRead"):_, _, is_gsi_scan = _resolve_index_keys(table, index_name)@@ -2750,8 +3187,8 @@ def _scan(data):if index_name:required.update(idx_hashes + idx_ranges)if not required.issubset(esk.keys()):- return error_response_json("ValidationException",- "The provided starting key is invalid", 400)+ return error_response_json("ValidationException", "The provided starting key is invalid" + (+ ": The provided key element does not match the schema" if echoing else ""), 400)if index_name:# For index scans, use index-aware ordering (same as Query pagination).all_items = _apply_exclusive_start_key(@@ -2825,20 +3262,25 @@ def _execute_statement(data):table_name = parsed["table"]table = _tables.get(table_name)if not table:- return error_response_json("ResourceNotFoundException",- f"Requested resource not found: Table: {table_name} not found", 400)+ message = "Requested resource not found"+ if not parsed.get("from_first"):+ message += f": Table: {table_name} not found"+ return error_response_json("ResourceNotFoundException", message, 400)if op == "SELECT":if parsed.get("index"):status, headers, body = _partiql_select_index(table, parsed, data)else:+ overlap = _pq_overlap_error(table, parsed.get("where"))+ if overlap:+ return error_response_json("ValidationException", overlap, 400)status, headers, body = _partiql_select(table, parsed)elif op == "INSERT":status, headers, body = _partiql_insert(table, parsed)elif op == "UPDATE":- status, headers, body = _partiql_update(table, parsed)+ status, headers, body = _partiql_update(table, parsed, data)elif op == "DELETE":- status, headers, body = _partiql_delete(table, parsed)+ status, headers, body = _partiql_delete(table, parsed, data)else:return error_response_json("ValidationException", f"Unsupported PartiQL operation: {op}", 400)# Attach ConsumedCapacity per AWS when ReturnConsumedCapacity != NONE.@@ -2864,6 +3306,9 @@ def _partiql_select_index(table, parsed, data):unprojected attributes; capacity lands on the index arm; NextTokens arebound to the index that minted them."""index_name = parsed["index"]+ refusal = _pq_multi_attribute_index_error(table, index_name)+ if refusal:+ return error_response_json("ValidationException", refusal, 400)gsis = {i.get("IndexName"): (i, True) for i in table.get("GlobalSecondaryIndexes", []) or []}lsis = {i.get("IndexName"): (i, False) for i in table.get("LocalSecondaryIndexes", []) or []}idx, is_gsi = (gsis.get(index_name) or lsis.get(index_name) or (None, None))@@ -3002,6 +3447,10 @@ def _partiql_select(table, parsed):filtered = [it for it in all_items if parsed["where_fn"](it)]else:filtered = all_items+ if parsed.get("order"):+ parts, descending = parsed["order"]+ filtered.sort(key=lambda it: _ddb_comparable(_partiql_get_path(it, parts))[1] if _partiql_get_path(it, parts) else "",+ reverse=descending)projections = parsed.get("projections")if projections:@@ -3030,6 +3479,8 @@ def _partiql_select(table, parsed):def _partiql_insert(table, parsed):item = parsed.get("item", {})+ if isinstance(item, dict) and set(item) == {"M"}:+ item = item["M"]if not item:return error_response_json("ValidationException", "INSERT requires a value list", 400)pk_val = _extract_key_val(item.get(table["pk_name"]))@@ -3045,75 +3496,24 @@ def _partiql_insert(table, parsed):if _item_size_bytes(item) > _DDB_ITEM_MAX_BYTES:return error_response_json("ValidationException", _ITEM_SIZE_MSG, 400)_set_item(table, pk_val, sk_val, item)- return json_response({})+ return json_response({"Items": []})def _partiql_key_target(table, parsed):- """Return (pk_key, sk_key, non_key_fn, error_resp).-- AWS PartiQL UPDATE/DELETE require equality conditions on every primary key- attribute. Non-key clauses act as a conditional check on the targeted item- — failure → ConditionalCheckFailedException, not a silent no-op.- """- conditions = parsed.get("conditions") or []- pk_attr = table.get("pk_name")- sk_attr = table.get("sk_name")-- pk_typed = None- sk_typed = None- rest = []- for attr, op, val in conditions:+ """(pk, sk, condition, error): the item every key attribute's equality names, and the WHERE as+ a condition on it (a failing condition is ConditionalCheckFailedException)."""+ pk_typed = sk_typed = None+ pk_attr, sk_attr = table.get("pk_name"), table.get("sk_name")+ for attr, op, val in parsed.get("conditions") or []:if attr == pk_attr and op == "=":pk_typed = valelif sk_attr and attr == sk_attr and op == "=":sk_typed = val- else:- rest.append((attr, op, val))-if pk_typed is None or (sk_attr and sk_typed is None):return None, None, None, error_response_json(- "ValidationException",- "Where clause does not contain a mandatory equality on all key attributes",- 400,- )-- def non_key_fn(item):- # Each predicate is a fail-fast check: on a pass we fall through to the- # next condition rather than returning, so every predicate in a- # multi-condition WHERE is evaluated (not just the first special op).- for attr, op, val in rest:- if op == "begins_with":- iv = item.get(attr)- if iv is None or "S" not in iv:- return False- prefix = val.get("S", "") if isinstance(val, dict) else str(val)- if not iv["S"].startswith(prefix):- return False- elif op == "not_begins_with":- iv = item.get(attr)- if iv is not None and "S" in iv:- prefix = val.get("S", "") if isinstance(val, dict) else str(val)- if iv["S"].startswith(prefix):- return False- elif op == "is_missing":- if item.get(attr) is not None:- return False- elif op == "is_not_missing":- if item.get(attr) is None:- return False- elif op == "IN":- iv = item.get(attr)- if iv is None or not any(_ddb_equals(iv, v) for v in val):- return False- else:- if not _compare_ddb(item.get(attr), op, val):- return False- return True-- pk_key = _extract_key_val(pk_typed)- sk_key = _extract_key_val(sk_typed) if sk_attr else "__no_sort__"- return pk_key, sk_key, non_key_fn, None-+ "ValidationException", "Where clause does not contain a mandatory equality on all key attributes", 400)+ where_fn = parsed.get("where_fn") or (lambda item: True)+ return _extract_key_val(pk_typed), (_extract_key_val(sk_typed) if sk_attr else "__no_sort__"), where_fn, Nonedef _partiql_parse_path(raw):"""Parse a PartiQL document path: 'profile.sub' -> ["profile", "sub"],@@ -3250,105 +3650,70 @@ def _partiql_modified_items(item_before, item_after, paths, want_old):return [frag] if frag else []-def _build_partiql_returning(item_before, item_after, returning_clause):- """Build the Items list for a RETURNING clause response.-- Supported: ALL OLD *, ALL NEW *, MODIFIED OLD *, MODIFIED NEW *- Returns (items_list, error_or_None)- """- if not returning_clause:- return [], None- r = returning_clause.upper().strip()- if r == "ALL OLD *":- return [item_before] if item_before else [], None- elif r == "ALL NEW *":- return [item_after] if item_after else [], None- elif r == "MODIFIED NEW *":- if not item_before or not item_after:- return [], None- changed = {k: v for k, v in item_after.items()- if k not in item_before or item_before[k] != v}- return [changed] if changed else [], None- elif r == "MODIFIED OLD *":- if not item_before or not item_after:- return [], None- changed = {k: item_before[k] for k in item_before- if k not in item_after or item_after[k] != item_before[k]}- return [changed] if changed else [], None- else:- return None, f"Invalid returning clause: {returning_clause}"+def _pq_set_function(name, current, members):+ """set_add / set_delete on the set an attribute holds; None removes the attribute."""+ if not isinstance(members, dict) or next(iter(members), None) not in _PQ_SET_TYPES:+ raise ValueError("An operand in the update expression has an incorrect data type")+ mtype = next(iter(members))+ if current is not None and next(iter(current)) != mtype:+ raise ValueError("An operand in the update expression has an incorrect data type")+ have = list(current[mtype]) if current else []+ if name == "set_add":+ return {mtype: have + [m for m in members[mtype] if m not in have]}+ if current is None:+ return None+ kept = [m for m in have if m not in members[mtype]]+ return {mtype: kept} if kept else None-def _partiql_update(table, parsed):- set_attrs = parsed.get("set_attrs", {})- remove_attrs = parsed.get("remove_attrs", [])+def _partiql_update(table, parsed, data=None):returning = parsed.get("returning")-- if not parsed.get("where_fn") and not parsed.get("conditions"):- return error_response_json("ValidationException",- "UPDATE requires a WHERE clause", 400)- if not set_attrs and not remove_attrs:- return error_response_json("ValidationException",- "UPDATE requires SET or REMOVE clause", 400)-- # Validate RETURNING clauseif returning:valid_returning = {"ALL OLD *", "ALL NEW *", "MODIFIED OLD *", "MODIFIED NEW *"}if returning.upper().strip() not in valid_returning:return error_response_json("ValidationException",f"Invalid returning clause: RETURNING {returning} is not supported", 400)-- pk_key, sk_key, non_key_fn, err = _partiql_key_target(table, parsed)+ pk_key, sk_key, condition, err = _partiql_key_target(table, parsed)if err:return err-item = table["items"].get(pk_key, {}).get(sk_key)# UPDATE on a missing key → ConditionalCheckFailedException (not upsert).if item is None:return _conditional_check_failed({}, None)- if not non_key_fn(item):- return _conditional_check_failed({}, item)-+ try:+ holds = condition(item)+ except ValueError as exc:+ return error_response_json("ValidationException", str(exc), 400)+ if not holds:+ return _conditional_check_failed(data or {}, item)item_before = copy.deepcopy(item)work = copy.deepcopy(item)applied_paths = []- for attr, val in set_attrs.items():- # Handle arithmetic: {"__partiql_arith": {"attr": "n", "op": "+", "val": {"N": "1"}}}- if isinstance(val, dict) and "__partiql_arith" in val:- arith = val["__partiql_arith"]- src_attr = arith["attr"]- op = arith["op"]- operand = arith["val"]- cur = work.get(src_attr)- if cur and "N" in cur and "N" in operand:- from decimal import Decimal- result_n = Decimal(cur["N"]) + Decimal(operand["N"]) if op == "+" else Decimal(cur["N"]) - Decimal(operand["N"])- work[attr] = {"N": str(result_n)}+ try:+ for parts, value in parsed.get("assignments") or []:+ if value[0] == "setfn":+ result = _pq_set_function(value[1], _partiql_get_path(work, parts), value[2])+ if result is None:+ _partiql_remove_path(work, parts)+ applied_paths.append(parts)+ continueelse:- work[attr] = operand # fallback: just set to operand- applied_paths.append([attr])- continue- try:- parts = _partiql_parse_path(attr)- except ValueError as exc:- return error_response_json("ValidationException", str(exc), 400)- err_msg = _partiql_set_path(work, parts, val)- if err_msg:- return error_response_json("ValidationException", err_msg, 400)- applied_paths.append(parts)- for attr in remove_attrs:- try:- parts = _partiql_parse_path(attr)- except ValueError as exc:- return error_response_json("ValidationException", str(exc), 400)+ result = _pq_eval(value, work)+ if result is None:+ return error_response_json("ValidationException", _PARTIQL_BAD_PATH, 400)+ err_msg = _partiql_set_path(work, parts, result)+ if err_msg:+ return error_response_json("ValidationException", err_msg, 400)+ applied_paths.append(parts)+ except ValueError as exc:+ return error_response_json("ValidationException", str(exc), 400)+ for parts in parsed.get("removes") or []:_partiql_remove_path(work, parts)applied_paths.append(parts)if _item_size_bytes(work) > _DDB_ITEM_MAX_BYTES:return error_response_json("ValidationException", _UPDATE_SIZE_MSG, 400)- # Nothing failed — commit the working copy._set_item(table, pk_key, sk_key, work)item_after = work-if returning:r = returning.upper().strip()if r in ("MODIFIED OLD *", "MODIFIED NEW *"):@@ -3361,12 +3726,8 @@ def _partiql_update(table, parsed):return json_response({"Items": items_list})return json_response({"Items": []})--def _partiql_delete(table, parsed):+def _partiql_delete(table, parsed, data=None):returning = parsed.get("returning")- if not parsed.get("where_fn") and not parsed.get("conditions"):- return error_response_json("ValidationException",- "DELETE requires a WHERE clause", 400)# Validate RETURNING clause — DELETE only supports ALL OLD *if returning:@@ -3388,9 +3749,12 @@ def _partiql_delete(table, parsed):return json_response({"Items": []})return json_response({"Items": []})- # Non-key predicate fails → ConditionalCheckFailed- if not non_key_fn(item):- return _conditional_check_failed({}, item)+ try:+ holds = non_key_fn(item)+ except ValueError as exc:+ return error_response_json("ValidationException", str(exc), 400)+ if not holds:+ return _conditional_check_failed(data or {}, item)item_before = copy.deepcopy(item) if returning else None_remove_item(table, pk_key, sk_key)@@ -3400,6 +3764,25 @@ def _partiql_delete(table, parsed):return json_response({"Items": []})+def _pq_select_key_error(table, parsed) -> bool:+ """A batch or transaction SELECT reads one item: equality on every key attribute, no functions."""+ if parsed.get("index"):+ return True+ eq_attrs = {attr for attr, op, _ in parsed.get("conditions") or [] if op == "="}+ need = {table.get("pk_name")} | ({table.get("sk_name")} if table.get("sk_name") else set())+ return not need.issubset(eq_attrs) or _pq_has_call(parsed.get("where"))+++def _pq_multi_attribute_index_error(table, index_name):+ for idx in (table.get("GlobalSecondaryIndexes") or []) + (table.get("LocalSecondaryIndexes") or []):+ if idx.get("IndexName") == index_name:+ kinds = [k.get("KeyType") for k in idx.get("KeySchema") or []]+ if kinds.count("HASH") > 1 or kinds.count("RANGE") > 1:+ return ("PartiQL statements cannot be executed on tables with multi-attribute HASH keys "+ "or multi-attribute RANGE keys.")+ return None++# ---------------------------------------------------------------------------# PartiQL — BatchExecuteStatement# ---------------------------------------------------------------------------@@ -3444,27 +3827,18 @@ def _batch_execute_statement(data):try:parsed = _parse_partiql(raw_stmt, stmt.get("Parameters", []))except ValueError as exc:- msg = str(exc)- if msg.startswith(("Unsupported PartiQL statement", "Could not parse")):- msg = "Statement wasn't well formed, can't be processed: Expected data manipulation"- responses.append({"Error": {"Code": "ValidationError", "Message": msg}})+ responses.append({"Error": {"Code": "ValidationError", "Message": str(exc)}})continueif parsed["op"] == "SELECT":- # A batch SELECT must name the full table primary key with equality,- # and an index-qualified read is not reachable from a batch at all.- key_err = Nonetbl = _tables.get(parsed["table"])- if parsed.get("index"):- key_err = True- elif tbl is not None:- eq_attrs = {attr for attr, op_, _ in (parsed.get("conditions") or []) if op_ == "="}- need = {tbl.get("pk_name")} | ({tbl.get("sk_name")} if tbl.get("sk_name") else set())- key_err = not need.issubset(eq_attrs)- if key_err:- responses.append({"Error": {"Code": "ValidationError",- "Message": "Select statements within BatchExecuteStatement must specify the primary key in the where clause"}})+ refusal = _pq_multi_attribute_index_error(tbl, parsed.get("index")) if tbl else None+ if refusal or (tbl is not None and _pq_select_key_error(tbl, parsed)):+ responses.append({"Error": {"Code": "ValidationError", "Message": refusal or+ "Select statements within BatchExecuteStatement must specify the primary key in the where clause."}})continuesub = {"Statement": raw_stmt, "Parameters": stmt.get("Parameters", [])}+ if stmt.get("ReturnValuesOnConditionCheckFailure"):+ sub["ReturnValuesOnConditionCheckFailure"] = stmt["ReturnValuesOnConditionCheckFailure"]status, _, body = _execute_statement(sub)try:payload = json.loads(body)@@ -3489,6 +3863,8 @@ def _batch_execute_statement(data):elif short.endswith("Exception"):short = short[: -len("Exception")]err_entry: dict = {"Error": {"Code": short, "Message": err_msg}}+ if short == "ConditionalCheckFailed" and payload.get("Item"):+ err_entry["Error"]["Item"] = payload["Item"]# TableName is echoed only on a member that ran and failed during# execution — not on one rejected before it reached its table.if short in ("ConditionalCheckFailed", "DuplicateItem"):@@ -3559,17 +3935,30 @@ def _execute_transaction(data):if _re2.search(r'\s+RETURNING\s+', statement, _re2.IGNORECASE):return error_response_json("ValidationException",f"Validation failed in TransactStatements[{stmt_index}]: RETURNING clause is not supported in ExecuteTransaction.", 400)+ prefix = f"Validation failed in TransactStatements[{stmt_index}]: "try:parsed = _parse_partiql(statement, parameters)+ except PartiqlError as e:+ return error_response_json("ValidationException", (prefix if e.statement_level else "") + str(e), 400)except ValueError as e:return error_response_json("ValidationException", str(e), 400)if parsed.get("op") == "SELECT" and parsed.get("index"):return error_response_json("ValidationException",- f"Validation failed in TransactStatements[{stmt_index}]: Reads on indices are not supported within transactions.", 400)+ f"{prefix}Reads on indices are not supported within transactions.", 400)table = _tables.get(parsed["table"])if not table:return error_response_json("ResourceNotFoundException",f"Requested resource not found: Table: {parsed['table']} not found", 400)+ if parsed["op"] == "SELECT" and _pq_select_key_error(table, parsed):+ return error_response_json("ValidationException",+ f"{prefix}Select statements within ExecuteTransaction must specify the primary key in the where clause.", 400)+ if parsed["op"] == "EXISTS":+ inner = parsed["select"]+ key_only = all(op == "=" and attr in (table.get("pk_name"), table.get("sk_name"))+ for attr, op, _ in inner.get("conditions") or [])+ if _pq_select_key_error(table, {**inner, "where": None}) or (key_only and not _pq_has_call(inner.get("where"))):+ return error_response_json("ValidationException",+ f"{prefix}EXISTS() must contain a single item read with additional condition", 400)parsed_list.append((parsed, table))# Apply each statement; collect any failures to roll back. Take ONE@@ -3584,6 +3973,14 @@ def _execute_transaction(data):responses = []failure = Nonefor idx, (parsed, table) in enumerate(parsed_list):+ if parsed["op"] == "EXISTS":+ inner = parsed["select"]+ pk_key, sk_key, condition, _err = _partiql_key_target(table, inner)+ found = table["items"].get(pk_key, {}).get(sk_key)+ if found is None or not condition(found):+ failure = (idx, "ConditionalCheckFailed", "The conditional request failed")+ break+ continueif parsed["op"] == "SELECT":status, _, body = _partiql_select(table, parsed)elif parsed["op"] == "INSERT":@@ -3629,7 +4026,8 @@ def _execute_transaction(data):}).encode("utf-8")return 400, {"Content-Type": "application/x-amz-json-1.0", "x-amzn-errortype": "TransactionCanceledException"}, body- result = {"Responses": responses}+ reads = any(parsed["op"] == "SELECT" for parsed, _ in parsed_list)+ result = {"Responses": responses if reads else []}# Per-table item sizes: a transactional write costs 2 x ceil(size/1KB) WCU,# a same-token replay 2 x ceil(size/4KB) RCU (measured, paritysuite).txn_sizes: dict = {}@@ -3658,177 +4056,261 @@ def _execute_transaction(data):return json_response(result)-def _parse_partiql(statement, parameters):- """Minimal PartiQL parser for DynamoDB statements."""- s = statement.strip().rstrip(";").strip()- upper = s.upper()-- if upper.startswith("SELECT"):- return _parse_partiql_select(s, parameters)- elif upper.startswith("INSERT"):- return _parse_partiql_insert(s, parameters)- elif upper.startswith("UPDATE"):- return _parse_partiql_update(s, parameters)- elif upper.startswith("DELETE"):- return _parse_partiql_delete(s, parameters)- else:- raise ValueError(f"Unsupported PartiQL statement: {s[:20]}")+_PARAM_NUMBER_RE = re.compile(r"^[+-]?(\d+\.?\d*|\.\d+)(?:[eE]([+-]?\d+))?$")+_INT32_MIN, _INT32_MAX = -(2 ** 31), 2 ** 31 - 1-def _partiql_from_path_error(s):- """AWS checks the FROM path before resolving it: no empty component, at most two."""- import re- m = re.search(r'\bFROM\s+((?:"[^"]*"|[A-Za-z0-9_\-]+)(?:\s*\.\s*(?:"[^"]*"|[A-Za-z0-9_\-]+))*)', s, re.IGNORECASE)- if not m:+def _wrap_int32(n: int) -> int:+ return (n + 2 ** 31) % 2 ** 32 - 2 ** 31+++def _partiql_number_param_error(text) -> str | None:+ """A number parameter is read as Java's BigDecimal reads a string: an exponent or scale outside+ a 32-bit integer cannot be converted, and the magnitude check wraps in 32-bit arithmetic."""+ unconvertible = "The parameter cannot be converted to a numeric value"+ if not isinstance(text, str) or text == "":+ return unconvertible+ match = _PARAM_NUMBER_RE.match(text)+ if not match:+ return f"{unconvertible}: {text}"+ exponent = int(match.group(2) or 0)+ mantissa = match.group(1)+ fraction = len(mantissa.split(".", 1)[1]) if "." in mantissa else 0+ scale = fraction - exponent+ if not _INT32_MIN <= exponent <= _INT32_MAX or not _INT32_MIN <= scale <= _INT32_MAX:+ return f"{unconvertible}: {text}"+ digits = mantissa.replace(".", "").lstrip("0")+ if not digits:return None- components = re.findall(r'"[^"]*"|[A-Za-z0-9_\-]+', m.group(1))- if any(c == '""' for c in components):- return "Path component cannot be an empty string"- if len(components) > 2:- return "A path may contain at most 2 components in the FROM clause"+ wrapped = _wrap_int32(len(digits) - scale)+ if wrapped != len(digits) - scale:+ if wrapped - 1 < _DDB_NUM_NEG_MIN_EXP:+ return "Number underflow. Attempting to store a number with magnitude smaller than supported range"+ return "Number overflow. Attempting to store a number with magnitude larger than supported range"return None-def _parse_partiql_select(s, parameters):- import re- path_error = _partiql_from_path_error(s)- if path_error:- raise ValueError(path_error)- # SELECT <projections> FROM <table> [WHERE <condition>]- m = re.match(- r'SELECT\s+(.*?)\s+FROM\s+("?[A-Za-z0-9_.\-]+"?(?:\s*\.\s*"[A-Za-z0-9_.\-]+")?)(?:\s+WHERE\s+(.+))?$',- s, re.IGNORECASE | re.DOTALL,- )- if not m:- raise ValueError(f"Could not parse SELECT statement: {s}")-- proj_str = m.group(1).strip()- from_str = m.group(2).strip()- where_str = m.group(3)- # FROM "table"."index" — the index qualifier must be quoted, so a plain- # table name containing dots keeps its existing meaning.- index_name = None- qm = re.match(r'^"?([A-Za-z0-9_.\-]+?)"?\s*\.\s*"([A-Za-z0-9_.\-]+)"$', from_str)- if '"' in from_str and qm:- table_name = qm.group(1)- index_name = qm.group(2)- else:- table_name = from_str.strip('"')+def _partiql_parameter_error(value, level: int = 1) -> str | None:+ """The first problem in a parameter, in document order: too deep, or a number that cannot be read."""+ if level > _DDB_MAX_NESTING_DEPTH:+ return _NESTING_MSG+ if not isinstance(value, dict) or len(value) != 1:+ return None+ (vtype, vval), = value.items()+ if vtype == "N":+ return _partiql_number_param_error(vval)+ if vtype == "NS" and isinstance(vval, list):+ return next(filter(None, (_partiql_number_param_error(n) for n in vval)), None)+ children = vval.values() if vtype == "M" and isinstance(vval, dict) else (+ vval if vtype == "L" and isinstance(vval, list) else ())+ return next(filter(None, (_partiql_parameter_error(v, level + 1) for v in children)), None)- projections = None- if proj_str != "*":- projections = [p.strip().strip('"') for p in proj_str.split(",")]- where_fn, conditions = (_build_partiql_where(where_str, parameters)- if where_str else (None, []))+def _parse_partiql(statement, parameters):+ """Parse a PartiQL statement into the shape the executors run."""+ for parameter in parameters or []:+ err = _partiql_parameter_error(parameter)+ if err:+ raise ValueError(err)+ parsed = parse_partiql_statement(statement, parameters)+ where = parsed.get("where")+ parsed["where_fn"] = (lambda item, _w=where: _pq_truthy(_pq_eval(_w, item))) if where is not None else None+ parsed["conditions"] = _pq_conditions(where)+ if parsed.get("projections") is not None:+ parsed["projections"] = [_pq_path_text(parts) for parts in parsed["projections"]]+ if parsed["op"] == "EXISTS":+ inner = parsed["select"]+ inner["where_fn"] = ((lambda item, _w=inner["where"]: _pq_truthy(_pq_eval(_w, item)))+ if inner.get("where") is not None else None)+ inner["conditions"] = _pq_conditions(inner.get("where"))+ return parsed- return {"op": "SELECT", "table": table_name, "index": index_name,- "projections": projections, "where_fn": where_fn, "conditions": conditions}+def _pq_path_text(parts):+ return "".join(f"[{p}]" if isinstance(p, int) else (f".{p}" if i else p) for i, p in enumerate(parts))-def _parse_partiql_insert(s, parameters):- import re- # INSERT INTO <table> VALUE { ... }- if re.search(r'INTO\s+"[^"]+"\s*\.\s*"', s, re.IGNORECASE):- raise ValueError(- "Statement wasn't well formed, can't be processed: FROM clause may only contain a single table name")- m = re.match(- r"INSERT\s+INTO\s+\"?([A-Za-z0-9_.\-]+)\"?\s+VALUE\s+(.+)$",- s, re.IGNORECASE | re.DOTALL,- )- if not m:- raise ValueError(f"Could not parse INSERT statement: {s}")-- table_name = m.group(1).strip()- value_str = m.group(2).strip()- item = _parse_partiql_value(value_str, parameters)- if not isinstance(item, dict) or not all(isinstance(v, dict) for v in item.values()):- raise ValueError("INSERT VALUE must be a map of DynamoDB-typed attributes")- return {"op": "INSERT", "table": table_name, "item": item}---def _parse_partiql_update(s, parameters):- import re- # UPDATE <table> SET <assignments> [REMOVE <paths>] WHERE <condition> [RETURNING <clause>]- # Strip RETURNING clause first (it's always at the end).- returning = None- ret_match = re.search(r'\s+RETURNING\s+(.+)$', s, re.IGNORECASE)- if ret_match:- returning = ret_match.group(1).strip()- s = s[:ret_match.start()].strip()-- if re.match(r'UPDATE\s+"[^"]+"\s*\.\s*"', s, re.IGNORECASE):- raise ValueError("This operation is not supported on an index")- # Try SET ... WHERE form first.- m = re.match(- r"UPDATE\s+\"?([A-Za-z0-9_.\-]+)\"?\s+SET\s+(.+?)\s+WHERE\s+(.+)$",- s, re.IGNORECASE | re.DOTALL,- )- if m:- table_name = m.group(1).strip()- set_str = m.group(2).strip()- where_str = m.group(3).strip()- remove_str = None- # Check for embedded REMOVE clause within SET string (SET x=v REMOVE y WHERE ...)- rem_in_set = re.search(r'\s+REMOVE\s+(.+)$', set_str, re.IGNORECASE)- if rem_in_set:- remove_str = rem_in_set.group(1).strip()- set_str = set_str[:rem_in_set.start()].strip()- set_attrs = {}- param_idx = [0]- for assignment in _split_top_level(set_str, ','):- parts = assignment.split("=", 1)- if len(parts) != 2:- raise ValueError(f"Invalid SET assignment: {assignment}")- attr = parts[0].strip().strip('"')- val_str = parts[1].strip()- set_attrs[attr] = _parse_partiql_literal(val_str, parameters, param_idx)- where_fn, conditions = _build_partiql_where(where_str, parameters, param_idx)- return {"op": "UPDATE", "table": table_name, "set_attrs": set_attrs,- "remove_attrs": [r.strip() for r in remove_str.split(",")] if remove_str else [],- "where_fn": where_fn, "conditions": conditions, "returning": returning}-- # Try REMOVE ... WHERE form.- m = re.match(- r"UPDATE\s+\"?([A-Za-z0-9_.\-]+)\"?\s+REMOVE\s+(.+?)\s+WHERE\s+(.+)$",- s, re.IGNORECASE | re.DOTALL,- )- if m:- table_name = m.group(1).strip()- remove_str = m.group(2).strip()- where_str = m.group(3).strip()- param_idx = [0]- where_fn, conditions = _build_partiql_where(where_str, parameters, param_idx)- return {"op": "UPDATE", "table": table_name, "set_attrs": {},- "remove_attrs": [r.strip() for r in remove_str.split(",")],- "where_fn": where_fn, "conditions": conditions, "returning": returning}-- raise ValueError(f"Could not parse UPDATE statement: {s}")---def _parse_partiql_delete(s, parameters):- import re- # Strip RETURNING clause first.- returning = None- ret_match = re.search(r'\s+RETURNING\s+(.+)$', s, re.IGNORECASE)- if ret_match:- returning = ret_match.group(1).strip()- s = s[:ret_match.start()].strip()-- if re.match(r'DELETE\s+FROM\s+"[^"]+"\s*\.\s*"', s, re.IGNORECASE):- raise ValueError("This operation is not supported on an index")- # DELETE FROM <table> WHERE <condition>- m = re.match(- r"DELETE\s+FROM\s+\"?([A-Za-z0-9_.\-]+)\"?\s+WHERE\s+(.+)$",- s, re.IGNORECASE | re.DOTALL,- )- if not m:- raise ValueError(f"Could not parse DELETE statement: {s}")- table_name = m.group(1).strip()- where_str = m.group(2).strip()- where_fn, conditions = _build_partiql_where(where_str, parameters)- return {"op": "DELETE", "table": table_name, "where_fn": where_fn,- "conditions": conditions, "returning": returning}++def _pq_truthy(value):+ return isinstance(value, dict) and value.get("BOOL") is True+++def _pq_bool(flag):+ return {"BOOL": bool(flag)}+++_PQ_SET_TYPES = ("SS", "NS", "BS")+++def _pq_eval(node, item):+ """A PartiQL expression's value on an item; a missing path is None."""+ kind = node[0]+ if kind == "lit":+ return node[1]+ if kind == "path":+ return _partiql_get_path(item, node[1])+ if kind == "missing":+ return None+ if kind == "and":+ return _pq_bool(_pq_truthy(_pq_eval(node[1], item)) and _pq_truthy(_pq_eval(node[2], item)))+ if kind == "or":+ return _pq_bool(_pq_truthy(_pq_eval(node[1], item)) or _pq_truthy(_pq_eval(node[2], item)))+ if kind == "not":+ return _pq_bool(not _pq_truthy(_pq_eval(node[1], item)))+ if kind == "cmp":+ return _pq_bool(_compare_ddb(_pq_eval(node[2], item), node[1], _pq_eval(node[3], item)))+ if kind == "between":+ value = _pq_eval(node[1], item)+ return _pq_bool(_compare_ddb(value, ">=", _pq_eval(node[2], item))+ and _compare_ddb(value, "<=", _pq_eval(node[3], item)))+ if kind == "in":+ value = _pq_eval(node[1], item)+ options = node[2]+ candidates = ([_pq_eval(o, item) for o in options[1]] if options[0] == "options"+ else (options[1] or {}).get("L") or [])+ return _pq_bool(value is not None and any(_ddb_equals(value, c) for c in candidates))+ if kind == "is":+ value, wanted, negated = _pq_eval(node[1], item), node[2], node[3]+ if wanted == "MISSING":+ result = value is None+ elif wanted == "NULL":+ result = value is None or value == {"NULL": True}+ elif value is None:+ result = False+ else:+ vtype = next(iter(value))+ result = vtype in _PQ_SET_TYPES if wanted == "BAG" else vtype == wanted+ return _pq_bool(result != negated)+ if kind == "call":+ return _pq_call(node[1], [_pq_eval(a, item) for a in node[2]], node[2])+ if kind == "arith":+ left, right = _pq_eval(node[2], item), _pq_eval(node[3], item)+ if not (left and right and "N" in left and "N" in right):+ raise ValueError("An operand in the update expression has an incorrect data type")+ total = Decimal(left["N"]) + Decimal(right["N"]) if node[1] == "+" else Decimal(left["N"]) - Decimal(right["N"])+ return {"N": _ddb_canonicalize_number(str(total)) or str(total)}+ if kind == "map":+ return {"M": {k: _pq_eval(v, item) for k, v in node[1]}}+ if kind == "list":+ return {"L": [_pq_eval(v, item) for v in node[1]]}+ if kind == "like":+ raise ValueError("Unsupported operation: Like")+ raise ValueError(f"Unsupported operation: {kind}")+++def _pq_call(name, args, nodes):+ first = args[0] if args else None+ if name == "attribute_exists":+ return _pq_bool(first is not None)+ if name == "attribute_not_exists":+ return _pq_bool(first is None)+ second = args[1] if len(args) > 1 else None+ if name == "begins_with":+ if first is None or second is None:+ return _pq_bool(False)+ for t in ("S", "B"):+ if t in first and t in second:+ return _pq_bool(first[t].startswith(second[t]))+ return _pq_bool(False)+ if name == "contains":+ if first is None or second is None:+ return _pq_bool(False)+ if "S" in first and "S" in second:+ return _pq_bool(second["S"] in first["S"])+ vtype = next(iter(first))+ if vtype in _PQ_SET_TYPES:+ member = second.get(vtype[0])+ return _pq_bool(member is not None and member in first[vtype])+ if vtype == "L":+ return _pq_bool(any(_ddb_equals(v, second) for v in first["L"]))+ return _pq_bool(False)+ if name == "size":+ if first is None:+ return None+ vtype, value = next(iter(first.items()))+ if vtype == "S":+ return {"N": str(len(value))}+ if vtype == "B":+ return {"N": str(len(base64.b64decode(value)) if isinstance(value, str) else len(value))}+ if vtype in ("L", "M") or vtype in _PQ_SET_TYPES:+ return {"N": str(len(value))}+ return None+ if name == "attribute_type":+ return _pq_bool(first is not None and second is not None and second.get("S") == next(iter(first)))+ raise ValueError(f"Unsupported operation: {name}")+++def _pq_conditions(where):+ """Top-level AND conjuncts on a top-level attribute, as (attr, op, value) for key targeting."""+ if where is None:+ return []+ conjuncts, stack = [], [where]+ while stack:+ node = stack.pop()+ if node[0] == "and":+ stack.extend((node[2], node[1]))+ elif node[0] == "or":+ return []+ else:+ conjuncts.append(node)+ out = []++ def attr_of(node):+ return node[1][0] if node[0] == "path" and len(node[1]) == 1 else None++ flip = {"<": ">", ">": "<", "<=": ">=", ">=": "<=", "=": "=", "<>": "<>"}+ for node in conjuncts:+ negated = node[0] == "not"+ inner = node[1] if negated else node+ prefix = "NOT:" if negated else ""+ if inner[0] == "cmp":+ left, right = inner[2], inner[3]+ if attr_of(left) and right[0] == "lit":+ out.append((attr_of(left), prefix + inner[1], right[1]))+ elif attr_of(right) and left[0] == "lit":+ out.append((attr_of(right), prefix + flip[inner[1]], left[1]))+ elif inner[0] == "between" and not negated and attr_of(inner[1]) and inner[2][0] == inner[3][0] == "lit":+ out.append((attr_of(inner[1]), ">=", inner[2][1]))+ out.append((attr_of(inner[1]), "<=", inner[3][1]))+ elif inner[0] == "in" and not negated and attr_of(inner[1]):+ options = inner[2]+ if options[0] == "options" and all(o[0] == "lit" for o in options[1]):+ out.append((attr_of(inner[1]), "IN", [o[1] for o in options[1]]))+ elif inner[0] == "call" and inner[1] == "begins_with" and len(inner[2]) == 2 \+ and attr_of(inner[2][0]) and inner[2][1][0] == "lit":+ out.append((attr_of(inner[2][0]), "not_begins_with" if negated else "begins_with", inner[2][1][1]))+ elif inner[0] == "is" and not negated and inner[2] == "MISSING" and attr_of(inner[1]):+ out.append((attr_of(inner[1]), "is_not_missing" if inner[3] else "is_missing", None))+ return out+++def _pq_has_call(where) -> bool:+ if where is None:+ return False+ if where[0] == "call":+ return True+ return any(isinstance(c, tuple) and c and isinstance(c[0], str) and _pq_has_call(c) for c in where[1:])+++def _pq_overlap_error(table, where):+ """OR branches overlapping on the sort key: one branch bounds it, another filters a non-key attribute."""+ sk = table.get("sk_name")+ if not sk or where is None or where[0] != "or":+ return None+ branches, stack = [], [where]+ while stack:+ node = stack.pop()+ if node[0] == "or":+ stack.extend((node[2], node[1]))+ else:+ branches.append(_pq_conditions(node) if node[0] != "or" else [])+ keys = {table.get("pk_name"), sk}+ positive = [any(a == sk and not op.startswith("NOT:") and op not in ("<>", "not_begins_with")+ for a, op, _ in b) for b in branches]+ filtered = [any(a not in keys for a, _, _ in b) for b in branches]+ for i, b in enumerate(branches):+ if positive[i] and any(filtered[j] and not positive[j] for j in range(len(branches)) if j != i):+ return "Overlapping conditions with range keys are not supported in where clause"+ return Nonedef _eval_partiql_pred(item, attr, op, val):@@ -3856,329 +4338,6 @@ def _eval_partiql_pred(item, attr, op, val):return _compare_ddb(item_val, op, val)-def _build_partiql_where(where_str, parameters, param_idx=None):- """Build a predicate function + structural conditions list for a PartiQL- WHERE. Top-level OR splits into branches (DNF); the conditions list is- only populated for the single-branch form, which is what key targeting- and batch key validation consume."""- if not where_str or not where_str.strip():- return None, []- if param_idx is None:- param_idx = [0]-- branches = []- for branch_str in _split_top_level_by_or(where_str):- b = branch_str.strip()- # Strip one pair of parens wrapping the whole branch.- if b.startswith("(") and b.endswith(")"):- depth = 0- wraps = True- for k, ch in enumerate(b):- if ch == "(":- depth += 1- elif ch == ")":- depth -= 1- if depth == 0 and k != len(b) - 1:- wraps = False- break- if wraps:- b = b[1:-1].strip()- branches.append(_parse_partiql_conditions(b, parameters, param_idx))-- conditions = branches[0] if len(branches) == 1 else []-- def where_fn(item):- return any(all(_eval_partiql_pred(item, attr, op, val) for attr, op, val in branch)- for branch in branches)-- return where_fn, conditions---def _split_top_level_by_or(s):- """Split a WHERE string on OR at the top level (respecting parens/quotes)."""- parts = []- depth = 0- in_str = None- current = []- i = 0- s_up = s.upper()- while i < len(s):- ch = s[i]- if in_str:- current.append(ch)- if ch == in_str:- in_str = None- i += 1- continue- if ch in ("'", '"'):- in_str = ch- current.append(ch)- i += 1- continue- if ch in ('(', '[', '{'):- depth += 1- current.append(ch)- i += 1- continue- if ch in (')', ']', '}'):- depth -= 1- current.append(ch)- i += 1- continue- if depth == 0 and s_up[i:i+3] == ' OR' and (i + 3 >= len(s) or s[i+3] == ' '):- parts.append("".join(current))- current = []- i += 3- continue- current.append(ch)- i += 1- if current:- parts.append("".join(current))- return [p.strip() for p in parts if p.strip()]--def _parse_partiql_conditions(where_str, parameters, param_idx):- """Parse WHERE conditions joined by AND. Returns list of (attr, op, ddb_value)."""- import re- conditions = []- # Split on AND (case-insensitive, word boundary) — but not inside parens.- parts = _split_top_level_by_and(where_str)- for part in parts:- part = part.strip()- # begins_with(attr, val)- m = re.match(r'begins_with\s*\(\s*"?([A-Za-z0-9_.\-]+)"?\s*,\s*(.+)\s*\)\s*$', part, re.IGNORECASE)- if m:- attr = m.group(1)- val = _parse_partiql_literal(m.group(2).strip(), parameters, param_idx)- conditions.append((attr, "begins_with", val))- continue- # NOT begins_with(attr, val)- m = re.match(r'NOT\s+begins_with\s*\(\s*"?([A-Za-z0-9_.\-]+)"?\s*,\s*(.+)\s*\)\s*$', part, re.IGNORECASE)- if m:- attr = m.group(1)- val = _parse_partiql_literal(m.group(2).strip(), parameters, param_idx)- conditions.append((attr, "not_begins_with", val))- continue- # attr IS MISSING- m = re.match(r'"?([A-Za-z0-9_.\-]+)"?\s+IS\s+MISSING\s*$', part, re.IGNORECASE)- if m:- conditions.append((m.group(1), "is_missing", None))- continue- # attr IS NOT MISSING- m = re.match(r'"?([A-Za-z0-9_.\-]+)"?\s+IS\s+NOT\s+MISSING\s*$', part, re.IGNORECASE)- if m:- conditions.append((m.group(1), "is_not_missing", None))- continue- # attr IN ('v1', 'v2', ...)- m = re.match(r'"?([A-Za-z0-9_.\-]+)"?\s+IN\s*\[(.+)\]\s*$', part, re.IGNORECASE)- if not m:- m = re.match(r'"?([A-Za-z0-9_.\-]+)"?\s+IN\s*\((.+)\)\s*$', part, re.IGNORECASE)- if m:- attr = m.group(1)- vals = [_parse_partiql_literal(v.strip(), parameters, param_idx)- for v in _split_top_level(m.group(2), ',')]- conditions.append((attr, "IN", vals))- continue- # NOT <comparison> — negated predicate (begins_with has its own form).- mnot = re.match(r'NOT\s+("?[A-Za-z0-9_.\-]+"?\s*(?:=|<>|!=|<=|>=|<|>).+)$', part, re.IGNORECASE)- if mnot and not re.match(r'NOT\s+begins_with', part, re.IGNORECASE):- inner = _parse_partiql_conditions(mnot.group(1), parameters, param_idx)- for attr_i, op_i, val_i in inner:- conditions.append((attr_i, f"NOT:{op_i}", val_i))- continue- # attr BETWEEN lo AND hi (the inner AND is re-joined by the splitter)- m = re.match(r'"?([A-Za-z0-9_.\-]+)"?\s+BETWEEN\s+(.+?)\s+AND\s+(.+)$', part, re.IGNORECASE)- if m:- attr = m.group(1)- lo = _parse_partiql_literal(m.group(2).strip(), parameters, param_idx)- hi = _parse_partiql_literal(m.group(3).strip(), parameters, param_idx)- for bound in (lo, hi):- bt = _ddb_type(bound)- if bt and bt not in ("S", "N", "B"):- raise ValueError(- f"Incorrect operand type for operator or function; operator or function: BETWEEN, operand type: {bt}")- conditions.append((attr, ">=", lo))- conditions.append((attr, "<=", hi))- continue- # Standard comparison- m = re.match(r'"?([A-Za-z0-9_.\-]+)"?\s*(=|<>|!=|<=|>=|<|>)\s*(.+)$', part)- if not m:- raise ValueError(f"Could not parse WHERE condition: {part}")- attr = m.group(1)- op = m.group(2)- if op == '!=':- op = '<>'- val_str = m.group(3).strip()- val = _parse_partiql_literal(val_str, parameters, param_idx)- if op in ("<", "<=", ">", ">="):- # Ordering only exists for S / N / B operands; AWS rejects the- # statement naming the operator as written (measured, paritysuite).- vt = _ddb_type(val)- if vt and vt not in ("S", "N", "B"):- raise ValueError(- f"Incorrect operand type for operator or function; operator or function: {op}, operand type: {vt}")- conditions.append((attr, op, val))- return conditions---def _split_top_level_by_and(s):- """Split a WHERE string on AND at the top level (respecting parens/quotes)."""- # Use the existing _split_top_level approach but with AND as delimiter- parts = []- depth = 0- in_str = None- current = []- i = 0- s_up = s.upper()- while i < len(s):- ch = s[i]- if in_str:- current.append(ch)- if ch == in_str:- in_str = None- i += 1- continue- if ch in ("'", '"'):- in_str = ch- current.append(ch)- i += 1- continue- if ch in ('(', '[', '{'):- depth += 1- current.append(ch)- i += 1- continue- if ch in (')', ']', '}'):- depth -= 1- current.append(ch)- i += 1- continue- # Check for AND keyword at depth 0 — but the AND that separates a- # BETWEEN's bounds belongs to the BETWEEN, not the conjunction.- if depth == 0 and s_up[i:i+4] == ' AND' and (i + 4 >= len(s) or s[i+4] == ' '):- cur_up = "".join(current).upper()- between_pending = (' BETWEEN ' in cur_up- and ' AND ' not in cur_up.split(' BETWEEN ', 1)[1] + ' ')- if not between_pending:- parts.append("".join(current))- current = []- i += 4 # skip " AND"- continue- current.append(ch)- i += 1- if current:- parts.append("".join(current))- return [p.strip() for p in parts if p.strip()]---def _parse_partiql_literal(val_str, parameters, param_idx=None):- """Parse a PartiQL literal or ? parameter reference into a DynamoDB typed value."""- if param_idx is None:- param_idx = [0]- val_str = val_str.strip()-- if val_str == "?":- if param_idx[0] >= len(parameters):- raise ValueError("Not enough parameters for ? placeholders")- val = parameters[param_idx[0]]- param_idx[0] += 1- return val-- # String literal- if (val_str.startswith("'") and val_str.endswith("'")) or \- (val_str.startswith('"') and val_str.endswith('"')):- return {"S": val_str[1:-1]}-- # Boolean- if val_str.upper() == "TRUE":- return {"BOOL": True}- if val_str.upper() == "FALSE":- return {"BOOL": False}-- # NULL- if val_str.upper() == "NULL":- return {"NULL": True}-- # Number- try:- Decimal(val_str)- return {"N": val_str}- except (InvalidOperation, ValueError):- pass-- # Arithmetic expression: attr +/- number (e.g. "n + 1")- arith_m = re.match(r'^"?([A-Za-z0-9_.\-]+)"?\s*([+\-])\s*(.+)$', val_str)- if arith_m:- left_attr = arith_m.group(1)- operator = arith_m.group(2)- right_str = arith_m.group(3).strip()- right_val = _parse_partiql_literal(right_str, parameters, param_idx)- return {"__partiql_arith": {"attr": left_attr, "op": operator, "val": right_val}}-- raise ValueError(f"Cannot parse PartiQL value: {val_str}")---def _parse_partiql_value(val_str, parameters, param_idx=None):- """Parse a PartiQL VALUE map like {'attr': val, ...} into a DynamoDB item."""- if param_idx is None:- param_idx = [0]- val_str = val_str.strip()-- if val_str == "?":- if param_idx[0] >= len(parameters):- raise ValueError("Not enough parameters for ? placeholders")- val = parameters[param_idx[0]]- param_idx[0] += 1- return val-- # Parse DynamoDB JSON-style map: { 'key' : value, ... }- if not val_str.startswith("{") or not val_str.endswith("}"):- raise ValueError(f"Expected a map value, got: {val_str}")-- inner = val_str[1:-1].strip()- result = {}- for pair in _split_top_level(inner, ','):- pair = pair.strip()- if not pair:- continue- kv = pair.split(":", 1)- if len(kv) != 2:- raise ValueError(f"Invalid key-value pair: {pair}")- key = kv[0].strip().strip("'\"")- val = _parse_partiql_literal(kv[1].strip(), parameters, param_idx)- result[key] = val- return result---def _split_top_level(s, delimiter):- """Split string by delimiter, respecting nested braces/parens/quotes."""- parts = []- depth = 0- current = []- in_str = None- for ch in s:- if in_str:- current.append(ch)- if ch == in_str:- in_str = None- elif ch in ("'", '"'):- in_str = ch- current.append(ch)- elif ch in ('(', '{', '['):- depth += 1- current.append(ch)- elif ch in (')', '}', ']'):- depth -= 1- current.append(ch)- elif ch == delimiter and depth == 0:- parts.append("".join(current))- current = []- else:- current.append(ch)- if current:- parts.append("".join(current))- return parts--# ---------------------------------------------------------------------------# Batch operations# ---------------------------------------------------------------------------@@ -4204,6 +4363,9 @@ def _batch_write_item(data):if not request_items:return error_response_json("ValidationException","1 validation error detected: Value at 'RequestItems' failed to satisfy constraint: Member must have length greater than or equal to 1", 400)+ err = _batch_members_error(data, request_items, write=True)+ if err:+ return err# Total request count cap (25 per BatchWriteItem call).total = sum(len(v) for v in request_items.values())if total > _DDB_BATCH_WRITE_MAX:@@ -4230,7 +4392,8 @@ def _batch_write_item(data):# must not leave earlier members written.if "PutRequest" in req:item = req["PutRequest"].get("Item", {})- err = _validate_item(item, table.get("pk_name"), table.get("sk_name"))+ err = (_values_error_response(item.values() if isinstance(item, dict) else (), envelope=False)+ or _validate_item(item, table.get("pk_name"), table.get("sk_name")))if err:return err_, _, key_err = _resolve_table_key_values(table, item, allow_extra=True)@@ -4244,6 +4407,9 @@ def _batch_write_item(data):_extract_key_val(item.get(table.get("sk_name") or "")) if table.get("sk_name") else None)elif "DeleteRequest" in req:key = req["DeleteRequest"].get("Key", {})+ err = _values_error_response(key.values() if isinstance(key, dict) else (), envelope=False)+ if err:+ return err_, _, key_err = _resolve_table_key_values(table, key, allow_extra=False)if key_err:return key_err@@ -4315,8 +4481,13 @@ def _batch_write_item(data):def _batch_get_item(data):request_items = data.get("RequestItems")if not request_items:+ if get_region() in _OLDER_BATCH_GET_REGIONS:+ return error_response_json("ValidationException", "The requestItems parameter is required for BatchGetItem", 400)return error_response_json("ValidationException","1 validation error detected: Value at 'RequestItems' failed to satisfy constraint: Member must have length greater than or equal to 1", 400)+ err = _batch_members_error(data, request_items, write=False)+ if err:+ return err# Per-table key cap (100 per BatchGetItem call, per-table path in the error).for _bg_table_name, _bg_cfg in request_items.items():_bg_keys = _bg_cfg.get("Keys", [])@@ -4328,6 +4499,11 @@ def _batch_get_item(data):if _normalize_table_name(table_name) not in _tables:return error_response_json("ResourceNotFoundException","Requested resource not found", 400)+ for cfg in request_items.values():+ for key in (cfg.get("Keys") or []) if isinstance(cfg, dict) else ():+ err = _values_error_response(key.values() if isinstance(key, dict) else (), envelope=False)+ if err:+ return err# Duplicate-key rejection.for table_name, cfg in request_items.items():table = _tables[_normalize_table_name(table_name)]@@ -4392,17 +4568,14 @@ def _transact_write_items(data):if len(items_list) > _DDB_TXN_WRITE_MAX_ITEMS:return error_response_json("ValidationException",f"1 validation error detected: Value '[{', '.join(repr(i) for i in items_list)}]' at 'transactItems' failed to satisfy constraint: Member must have length less than or equal to {_DDB_TXN_WRITE_MAX_ITEMS}", 400)- # An oversized expression in any member rejects the whole request up front.+ err = _transact_write_members_error(data, items_list)+ if err:+ return err+ # Each action is read in request order, before any table is looked up.for _member in items_list:- for _op in (_member.values() if isinstance(_member, dict) else []):- if isinstance(_op, dict):- err = _expression_size_error(_op, ("ConditionExpression", "UpdateExpression"))- if err:- return err- values = list((_op.get("ExpressionAttributeValues") or {}).values())- values += list((_op.get("Item") or {}).values()) if isinstance(_op.get("Item"), dict) else []- if any(_nesting_exceeded(v) for v in values):- return error_response_json("ValidationException", _NESTING_MSG, 400)+ err = _transact_write_action_error(*_extract_transact_op(_member))+ if err:+ return err# 4MB total payload cap.try:if len(json.dumps(data).encode("utf-8")) > _DDB_TXN_MAX_BYTES:@@ -4438,16 +4611,7 @@ def _transact_write_items(data):return json_response(replay)return error_response_json("IdempotentParameterMismatchException","Request token already in use for another request with a different payload", 400)- # Member validation across the transaction. AWS validates every member- # before applying anything, but splits the failure into two shapes- # (verified against real DynamoDB):- # * up-front input errors -> top-level ValidationException (Phase 0):- # empty-string/binary key values, malformed item attrs, item size,- # and duplicate target keys.- # * per-item semantic errors -> TransactionCanceledException with a- # positional ValidationError reason (Phase 1): wrong-typed keys and- # update-expression type errors.- # Nothing is applied in either case.+ # Up-front input errors are a ValidationException; wrong-typed keys and update type errors cancel.seen_targets = set()for transact in items_list:op_type, op = _extract_transact_op(transact)@@ -4537,6 +4701,8 @@ def _transact_write_items(data):val_reasons[idx] = imsgelif _item_size_bytes(probe) > _DDB_ITEM_MAX_BYTES:val_reasons[idx] = _UPDATE_SIZE_MSG+ elif any(_nesting_exceeded(v) for v in probe.values()):+ val_reasons[idx] = "Nesting Levels have exceeded supported limits"if val_reasons:return _transact_validation_cancel_response(len(items_list), val_reasons)@@ -4696,6 +4862,15 @@ def _transact_get_items(data):if len(items_list) > _DDB_TXN_GET_MAX_ITEMS:return error_response_json("ValidationException",f"1 validation error detected: Value '[{', '.join(repr(i) for i in items_list)}]' at 'transactItems' failed to satisfy constraint: Member must have length less than or equal to {_DDB_TXN_GET_MAX_ITEMS}", 400)+ failures = _transact_member_enum_failures(data)+ for idx, member in enumerate(items_list, 1):+ name = (member.get("Get") or {}).get("TableName") if isinstance(member, dict) else None+ if _malformed_table_reference(name):+ failures.append((name, f"transactItems.{idx}.member.get.tableName",+ "Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+"))+ err = _multi_validation_error(failures)+ if err:+ return err# AWS pre-validates ProjectionExpression syntax + reserved keywords at the# request level for TransactGetItems — surfaces as a ValidationException,# not via the cancellation channel.@@ -4705,6 +4880,10 @@ def _transact_get_items(data):_err = _validate_projection_expression_syntax(_pe)if _err:return error_response_json("ValidationException", _err, 400)+ for transact in items_list:+ err = _values_error_response(((transact.get("Get") or {}).get("Key") or {}).values(), envelope=False)+ if err:+ return err# Per-action ValidationError (e.g. empty Key) surfaces as# TransactionCanceledException with reason "ValidationError" per AWS._cancel_reasons = []@@ -4797,6 +4976,98 @@ def _transact_get_items(data):return json_response(result)+_TXN_TOKEN_TOO_LONG = ("1 validation error detected: Value at 'ClientRequestToken' failed to satisfy constraint: "+ "Member must have length less than or equal to 36")+++def _transact_member_enum_failures(data) -> list:+ failures = []+ rcc = data.get("ReturnConsumedCapacity")+ if rcc is not None and rcc not in _RETURN_CONSUMED_CAPACITY_VALUES:+ failures.append((rcc, "returnConsumedCapacity", "Member must satisfy enum value set: [INDEXES, TOTAL, NONE]"))+ ricm = data.get("ReturnItemCollectionMetrics")+ if ricm is not None and ricm not in _RETURN_ITEM_COLLECTION_METRICS:+ failures.append((ricm, "returnItemCollectionMetrics", "Member must satisfy enum value set: [SIZE, NONE]"))+ return failures+++def _transact_write_members_error(data, items_list) -> tuple | None:+ """Member constraints, reported together before the actions are read."""+ crt = data.get("ClientRequestToken")+ if isinstance(crt, str) and len(crt) > 36:+ return error_response_json("ValidationException", _TXN_TOKEN_TOO_LONG, 400)+ failures = _transact_member_enum_failures(data)+ for idx, member in enumerate(items_list, 1):+ op_type, op = _extract_transact_op(member) if isinstance(member, dict) else (None, None)+ if op is None:+ failures.append(("", "transactItems",+ "TransactWriteRequest should contain Delete or Put or Update or ConditionCheck"))+ continue+ name = op.get("TableName") if isinstance(op, dict) else None+ if _malformed_table_reference(name):+ path = f"transactItems.{idx}.member.{op_type[0].lower() + op_type[1:]}.tableName"+ failures.append((name, path, "Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+"))+ return _multi_validation_error(failures)+++def _transact_write_action_error(op_type, op) -> tuple | None:+ """An action's own checks: expressions, then names, depth and values."""+ if not isinstance(op, dict):+ return None+ err = _expression_size_error(op, ("ConditionExpression", "UpdateExpression"),+ {"ConditionExpression": "figure", "UpdateExpression": "bare"})+ if err:+ return err+ for slot in ("ConditionExpression", "UpdateExpression"):+ if slot in op and isinstance(op[slot], str) and not op[slot].strip():+ return error_response_json("ValidationException", f"Invalid {slot}: The expression can not be empty;", 400)+ eav, ean = op.get("ExpressionAttributeValues") or {}, op.get("ExpressionAttributeNames") or {}+ if op.get("ConditionExpression"):+ try:+ _evaluate_condition(op["ConditionExpression"], {}, eav, ean)+ except ValueError as exc:+ return error_response_json("ValidationException", str(exc), 400)+ fields = ("ConditionExpression", "UpdateExpression") if op_type == "Update" else ("ConditionExpression",)+ err = _validate_expression_attrs(op, fields)+ if err:+ return err+ item = op.get("Item") if isinstance(op.get("Item"), dict) else {}+ if "" in item:+ return error_response_json("ValidationException", "Empty attribute name", 400)+ for container in (item, op.get("Key") if isinstance(op.get("Key"), dict) else {}):+ if any(_nesting_exceeded(v) for v in container.values()):+ return error_response_json("ValidationException", _NESTING_MSG, 400)+ # A value one level past the item cap still reaches the condition.+ for key, value in eav.items():+ level = _nesting_level(value)+ if level > _DDB_SECOND_NESTING_LEVEL - 1:+ return error_response_json("ValidationException",+ f"ExpressionAttributeValues contains invalid value: {_NESTING_MSG} for key {key}", 400)+ if level > _DDB_MAX_NESTING_DEPTH + 1:+ return error_response_json("ValidationException", _NESTING_MSG, 400)+ return _values_error_response(_request_values(op), envelope=False)+++def _batch_members_error(data, request_items, write: bool) -> tuple | None:+ """A malformed table name in RequestItems and the enums, reported together."""+ pieces = []+ if any(not isinstance(n, str) or _malformed_table_reference(n) for n in request_items):+ echoed = json.dumps(request_items, separators=(",", ":"), ensure_ascii=False)+ pieces.append(f"Value '{echoed}' at 'requestItems' failed to satisfy constraint: Map keys must satisfy constraint: "+ "[Member must have length less than or equal to 255, Member must have length greater than or equal to 3, "+ "Member must satisfy regular expression pattern: [a-zA-Z0-9_.-]+]")+ for name, entry in request_items.items():+ members = entry if write else (entry.get("Keys") if isinstance(entry, dict) else None)+ if isinstance(members, list) and not members:+ path = f"RequestItems.{name}.member" + ("" if write else ".Keys")+ pieces.append(f"Value at '{path}' failed to satisfy constraint: Member must have length greater than or equal to 1")+ pieces += [f"Value '{v}' at '{path}' failed to satisfy constraint: {c}" for v, path, c in _transact_member_enum_failures(data)]+ if not pieces:+ return None+ return error_response_json("ValidationException",+ f"{len(pieces)} validation error{'s' if len(pieces) > 1 else ''} detected: " + "; ".join(pieces), 400)++def _extract_transact_op(transact):for op_type in ("ConditionCheck", "Put", "Delete", "Update"):if op_type in transact:@@ -6546,6 +6817,132 @@ def _check_redundant_parens(tokens, slot: str = "ConditionExpression") -> str |return None+_COND_TOKEN_RE = re.compile(+ r"\s*(?:(?P<op><>|<=|>=|[=<>(),.\[\]])|(?P<name>#?[A-Za-z0-9_]+)|(?P<value>:[A-Za-z0-9_]+)|(?P<other>\S))")+_COND_COMPARATORS = {"=", "<>", "<", "<=", ">", ">="}+_COND_KEYWORDS = {"AND", "OR", "NOT", "BETWEEN", "IN"}+++class _ConditionSyntaxError(Exception):+ pass+++def _condition_syntax_error(expr: str, slot: str) -> str | None:+ """Grammar check of a condition, in one pass. Reports the token DynamoDB stops at and the text+ from the token before it to the token after it; any other fault is left to the evaluator."""+ tokens = []+ for m in _COND_TOKEN_RE.finditer(expr):+ if m.group("other"):+ return None+ group = "op" if m.group("op") else ("value" if m.group("value") else "name")+ text = m.group(group)+ kind = text.upper() if group == "name" and text.upper() in _COND_KEYWORDS else group+ tokens.append((kind, text, m.start(group), m.end(group)))+ pos = 0++ def peek(offset=0):+ i = pos + offset+ return tokens[i] if i < len(tokens) else ("EOF", "<EOF>", len(expr), len(expr))++ def fail():+ start = tokens[pos - 1][2] if pos > 0 else (tokens[pos][2] if pos < len(tokens) else len(expr))+ end = tokens[pos + 1][3] if pos + 1 < len(tokens) else (tokens[pos][3] if pos < len(tokens) else len(expr))+ raise _ConditionSyntaxError(f'Invalid {slot}: Syntax error; token: "{peek()[1]}", near: "{expr[start:end]}"')++ def take(kind=None, text=None):+ nonlocal pos+ tok = peek()+ if (kind and tok[0] != kind) or (text and tok[1] != text):+ fail()+ pos += 1+ return tok++ def path():+ if peek()[0] != "name" or peek()[1][0].isdigit():+ fail()+ take()+ while peek()[1] in (".", "["):+ if take()[1] == ".":+ take("name")+ else:+ if not peek()[1].isdigit():+ fail()+ take()+ take("op", "]")++ def operand():+ tok = peek()+ if tok[0] == "value":+ take()+ elif tok[0] == "name" and peek(1)[1] == "(":+ take()+ arguments()+ else:+ path()++ def arguments():+ take("op", "(")+ operand()+ while peek()[1] == ",":+ take()+ operand()+ take("op", ")")++ def atom():+ tok = peek()+ if tok[1] == "(":+ take()+ disjunction()+ take("op", ")")+ return+ if tok[0] == "name" and peek(1)[1] == "(" and tok[1].lower() != "size":+ take()+ arguments()+ return+ operand()+ nxt = peek()+ if nxt[1] in _COND_COMPARATORS:+ take()+ operand()+ elif nxt[0] == "BETWEEN":+ take()+ operand()+ take("AND")+ operand()+ elif nxt[0] == "IN":+ take()+ arguments()+ else:+ fail()++ def negation():+ if peek()[0] == "NOT":+ take()+ negation()+ else:+ atom()++ def conjunction():+ negation()+ while peek()[0] == "AND":+ take()+ negation()++ def disjunction():+ conjunction()+ while peek()[0] == "OR":+ take()+ conjunction()++ try:+ disjunction()+ if pos < len(tokens):+ fail()+ except _ConditionSyntaxError as exc:+ return str(exc)+ return None++def _evaluate_condition(expr, item, attr_values, attr_names,slot: str = "ConditionExpression"):"""Evaluate a DynamoDB expression. ``slot`` controls the "Invalid <X>:"@@ -6559,7 +6956,7 @@ def _evaluate_condition(expr, item, attr_values, attr_names,err = _check_redundant_parens(tokens, slot)if err:raise ValueError(err)- err = _check_reserved_keyword_usage(tokens)+ err = _check_reserved_keyword_usage(tokens) or _condition_syntax_error(expr, slot)if err:raise ValueError(err)# AWS validates BETWEEN bounds at parse time for ConditionExpression too:@@ -6939,9 +7336,11 @@ def _set_at_path(item, path_parts, value):if isinstance(part, int):if isinstance(item, dict) and "L" in item:lst = item["L"]- while len(lst) <= part:- lst.append({"NULL": True})- lst[part] = value+ # An index past the end appends the value.+ if part >= len(lst):+ lst.append(value)+ else:+ lst[part] = valueelse:if isinstance(item, dict):if "M" in item:@@ -7577,7 +7976,26 @@ def _vector_score(fn: str, query: list, vector: list) -> float:return _to_f32(score) or 0.0+_SEARCH_VECTORS_TYPES = {"TableName": "a string", "IndexName": "a string", "SearchVector": "a sequence", "TopK": "i32"}+++def _search_vectors_shape_error(data) -> tuple | None:+ """SearchVectors reads its body with a strict deserialiser before any member is validated."""+ body = json.dumps(data, separators=(",", ":"))+ for key, value in data.items():+ if value is None and key in _SEARCH_VECTORS_TYPES:+ column = body.index(f'"{key}":null') + len(f'"{key}":null')+ return error_response_json("SerializationException",+ f"invalid type: null, expected {_SEARCH_VECTORS_TYPES[key]} at line 1 column {column}", 400)+ if "TableName" not in data:+ return error_response_json("ValidationException", f"missing field `TableName` at line 1 column {len(body)}", 400)+ return None++def _search_vectors(data):+ err = _search_vectors_shape_error(data)+ if err:+ return errname = _normalize_table_name(data.get("TableName"))err = _validate_data_plane_table_name(name)if err:@@ -7855,9 +8273,9 @@ def _index_order_value(item, name, type_hint):return _sort_key_value(item.get(name), type_hint)-def _multi_sort_condition_ok(tokens, ean, range_names):- """Sort-key attributes of a multi-attribute key must be queried left to- right without skipping any, with equality on all but the last."""+def _multi_sort_condition_error(tokens, ean, range_names) -> str | None:+ """Sort-key attributes of a multi-attribute key are queried left to right, with equality on all+ but the last one named."""ops = {}i = 0while i < len(tokens):@@ -7874,9 +8292,16 @@ def _multi_sort_condition_ok(tokens, ean, range_names):ops.setdefault(name, []).append("eq" if eq else "range")i += 1used = [n for n in range_names if n in ops]- if used != range_names[:len(used)] or any(len(ops[n]) != 1 for n in used):- return False- return all(ops[n] == ["eq"] for n in used[:-1])+ if any(len(ops[n]) != 1 for n in used):+ return "Query key condition not supported"+ if not used:+ return None+ last = used[-1]+ lacking = [n for n in range_names[:range_names.index(last)] if ops.get(n) != ["eq"]]+ if lacking:+ return (f"RANGE key attributes {', '.join(lacking)} must have equality conditions specified in the query "+ f"because a condition is present on key attribute {last}")+ return Nonedef _index_order_keys(table, sk_name):@@ -8409,21 +8834,7 @@ def _evaluate_legacy_filter(item, scan_filter):def _evaluate_expected(item, expected, conditional_operator="AND"):- """Evaluate legacy ``Expected`` conditions on an item.-- Each key in *expected* is an attribute name. The value is one of:-- 1. ``{"ComparisonOperator": "...", "AttributeValueList": [...]}``- – full comparison form.- 2. ``{"Exists": true/false}``- – shorthand for NOT_NULL / NULL.- 3. ``{"Value": <attr>}``- – shorthand for ``{"ComparisonOperator": "EQ", "AttributeValueList": [<attr>]}``.- 4. ``{"Exists": false}``- – attribute must *not* exist.-- *conditional_operator* is ``"AND"`` (default) or ``"OR"``.- """+ """Evaluate legacy ``Expected`` conditions (comparison, Exists or Value shorthand) on an item."""results = []for attr_name, cond in expected.items():item_val = item.get(attr_name)@@ -8612,6 +9023,18 @@ def _index_write_units(table, idx, old_item, new_item):return max(_capacity_kb(_item_size_bytes(ov[1])), _capacity_kb(_item_size_bytes(nv[1])))+def _billed_item_bytes(table: dict, item: dict) -> int:+ """A write bills an attribute a vector index reads at its name plus four bytes per dimension."""+ size = _item_size_bytes(item)+ dimensions = {(vix.get("VectorAttribute") or {}).get("AttributeName"): vix.get("Dimensions")+ for vix in table.get("VectorIndexes") or []}+ for attr, dims in dimensions.items():+ values = (item.get(attr) or {}).get("L") if attr else None+ if isinstance(values, list) and len(values) == dims:+ size += 4 * dims - _attribute_value_size(item[attr])+ return size++def _add_consumed_capacity(result, data, table_name, write=False, old_item=None,new_item=None, index_name=None):rc = data.get("ReturnConsumedCapacity", "NONE")@@ -8621,8 +9044,8 @@ def _add_consumed_capacity(result, data, table_name, write=False, old_item=None,gsi_units: dict = {}lsi_units: dict = {}if write:- old_size = _item_size_bytes(old_item) if old_item else 0- new_size = _item_size_bytes(new_item) if new_item else 0+ old_size = _billed_item_bytes(table, old_item) if old_item else 0+ new_size = _billed_item_bytes(table, new_item) if new_item else 0table_units = _capacity_kb(max(old_size, new_size))for idx in table.get("GlobalSecondaryIndexes", []) or []:u = _index_write_units(table, idx, old_item, new_item)diff --git a/ministack/services/ec2.py b/ministack/services/ec2.pyindex 9a0f42cf..02d63dfa 100644--- a/ministack/services/ec2.py+++ b/ministack/services/ec2.py@@ -3984,7 +3984,6 @@ def _modify_volume_attribute(p):def _describe_volume_attribute(p):vol_id = _p(p, "VolumeId")- attribute = _p(p, "Attribute") or "autoEnableIO"return _xml(200, "DescribeVolumeAttributeResponse", f"""<volumeId>{vol_id}</volumeId><autoEnableIO><value>false</value></autoEnableIO>""")diff --git a/ministack/services/ecs.py b/ministack/services/ecs.pyindex 16e62e3f..68740677 100644--- a/ministack/services/ecs.py+++ b/ministack/services/ecs.py@@ -2945,7 +2945,6 @@ def _execute_command(data):def _list_account_settings(data):name = data.get("name", "")- effective = data.get("effectiveSettings", False)settings = []all_names = ["serviceLongArnFormat", "taskLongArnFormat",diff --git a/ministack/services/efs.py b/ministack/services/efs.pyindex cf22fb1d..7f335b26 100644--- a/ministack/services/efs.py+++ b/ministack/services/efs.py@@ -541,7 +541,6 @@ def _create_access_point(body):ap_id = _ap_id()arn = f"arn:aws:elasticfilesystem:{get_region()}:{get_account_id()}:access-point/{ap_id}"- now = _now_iso()tags = body.get("Tags", [])record = {diff --git a/ministack/services/eks.py b/ministack/services/eks.pyindex d31147e1..cdf61b08 100644--- a/ministack/services/eks.py+++ b/ministack/services/eks.py@@ -687,8 +687,6 @@ def _create_cluster(body, creator_arn=None):# Spawn k3s containerendpoint = ""- ca_data = ""- container_id = Noneport = _next_port()# Build cluster record immediately (status CREATING) and return fast.diff --git a/ministack/services/elasticache.py b/ministack/services/elasticache.pyindex 01a4703a..0550ba1a 100644--- a/ministack/services/elasticache.py+++ b/ministack/services/elasticache.py@@ -300,6 +300,7 @@ def _ensure_live_containers_locked():host, port, cid = _spawn_redis_container(name=f"ministack-elasticache-{account_id}-{region}-{name}",engine=engine, engine_version=version,+ transit=cl.get("TransitEncryptionEnabled", False), auth_token=_record_auth(cl),labels={**container_reaper.own_labels("elasticache"),"cluster_id": name,@@ -332,6 +333,7 @@ def _ensure_live_containers_locked():_, _, cid = _spawn_redis_container(name=f"ministack-elasticache-rg-{account_id}-{region}-{rg_id}-{ng_id}",engine=engine, engine_version=engine_version,+ transit=rg.get("TransitEncryptionEnabled", False), auth_token=_record_auth(rg),labels={**container_reaper.own_labels("elasticache"), "rg_id": rg_id,"node_group": ng_id, "account_id": account_id,@@ -516,7 +518,57 @@ def _image_is_local(docker_client, image: str) -> bool:return False-def _spawn_redis_container(name, engine, engine_version, labels):+_NODE_TLS_DIR = "/ministack-elasticache-tls"+# The certificate names the container's address, known only once it runs, so the+# server waits for the material; `ready` is the archive's last member.+_NODE_TLS_WRAPPER = (+ f"d={_NODE_TLS_DIR}; "+ 'while [ ! -e "$d/ready" ]; do sleep 0.1; done; '+ 'chown -R "$(stat -c %u:%g /data)" "$d"; '+ 'exec docker-entrypoint.sh "$@"'+)+++def _node_server_args(port, transit, auth_token):+ args = []+ if transit:+ args += ["--port", "0", "--tls-port", str(port),+ "--tls-cert-file", f"{_NODE_TLS_DIR}/server.crt",+ "--tls-key-file", f"{_NODE_TLS_DIR}/server.key",+ "--tls-ca-cert-file", f"{_NODE_TLS_DIR}/ca.crt",+ "--tls-auth-clients", "no", "--tls-replication", "yes"]+ if auth_token:+ args += ["--requirepass", auth_token, "--masterauth", auth_token]+ return args+++def _node_cli_args(transit, auth_token):+ args = ["--tls", "--cacert", f"{_NODE_TLS_DIR}/ca.crt", "-h", "127.0.0.1"] if transit else []+ if auth_token:+ args += ["-a", auth_token, "--no-auth-warning"]+ return args+++def _install_node_tls(container, name, container_ip):+ from ministack.core.x509_utils import sign_leaf_certificate++ candidates = ("localhost", _MINISTACK_HOST, name, "127.0.0.1", "::1", container_ip)+ dns_names = [n for n in dict.fromkeys(candidates) if n and not _is_ip(n)]+ ips = [n for n in dict.fromkeys(candidates) if n and _is_ip(n)]+ ca_pem, ca_key = _ensure_serverless_ca()+ cert_pem, key_pem, _public = sign_leaf_certificate(+ ca_pem, ca_key, common_name=dns_names[0], san_dns=dns_names, san_ips=ips)+ if not container.put_archive("/", _serverless_tls_archive(ca_pem, cert_pem, key_pem, "")):+ raise RuntimeError("Docker rejected the TLS material")+++def _record_auth(record):+ """The token a node starts with: the required one, or none while it is optional."""+ tokens = record.get("_auth_tokens") or []+ return tokens[-1] if tokens and not record.get("_auth_optional") else None+++def _spawn_redis_container(name, engine, engine_version, labels, transit=False, auth_token=None):"""Start a redis/valkey/memcached container.Returns ``(host, port, container_id)``. On any failure (docker unavailable,@@ -535,6 +587,9 @@ def _spawn_redis_container(name, engine, engine_version, labels):endpoint_port = host_portimage, container_port = _engine_image_and_port(engine, engine_version)+ redis_like = engine in ("redis", "valkey")+ tls = transit and redis_like+ container = Nonetry:run_kwargs = dict(@@ -544,13 +599,22 @@ def _spawn_redis_container(name, engine, engine_version, labels):labels=labels,volumes={},)+ server_args = _node_server_args(container_port, tls, auth_token if redis_like else None)+ if server_args:+ run_kwargs["command"] = server_args+ if tls:+ run_kwargs["entrypoint"] = ["sh", "-c", _NODE_TLS_WRAPPER, "ministack-elasticache-tls"]if DOCKER_NETWORK:run_kwargs["network"] = DOCKER_NETWORKcontainer = docker_client.containers.run(**run_kwargs)+ container_ip = ""if DOCKER_NETWORK:container.reload()networks = container.attrs.get("NetworkSettings", {}).get("Networks", {})container_ip = networks.get(DOCKER_NETWORK, {}).get("IPAddress", "")+ if tls:+ _install_node_tls(container, name, container_ip)+ if DOCKER_NETWORK:if container_ip:endpoint_host = container_ipendpoint_port = container_port@@ -563,10 +627,12 @@ def _spawn_redis_container(name, engine, engine_version, labels):return endpoint_host, endpoint_port, container.idexcept Exception as e:logger.warning("ElastiCache: Docker failed for %s: %s", name, e)+ if container is not None:+ _teardown_containers(docker_client, [container.id])return REDIS_DEFAULT_HOST, default_port, None-def _spawn_redis_cluster_node(name, engine, engine_version, labels):+def _spawn_redis_cluster_node(name, engine, engine_version, labels, transit=False, auth_token=None):"""Spawn a redis/valkey container with cluster-mode enabled.Requires DOCKER_NETWORK to be set so nodes can reach each other on the@@ -589,7 +655,10 @@ def _spawn_redis_cluster_node(name, engine, engine_version, labels):"--port", str(port),"--appendonly", "no","--protected-mode", "no",+ *_node_server_args(port, transit, auth_token),]+ if transit:+ cmd += ["--tls-cluster", "yes"]try:run_kwargs = dict(image=image,@@ -599,6 +668,8 @@ def _spawn_redis_cluster_node(name, engine, engine_version, labels):labels=labels,network=DOCKER_NETWORK,)+ if transit:+ run_kwargs["entrypoint"] = ["sh", "-c", _NODE_TLS_WRAPPER, "ministack-elasticache-tls"]container = docker_client.containers.run(**run_kwargs)container.reload()networks = container.attrs.get("NetworkSettings", {}).get("Networks", {})@@ -611,6 +682,12 @@ def _spawn_redis_cluster_node(name, engine, engine_version, labels):passlogger.warning("ElastiCache: cluster node %s has no IP on network %s", name, DOCKER_NETWORK)return None, None, None+ if transit:+ try:+ _install_node_tls(container, name, container_ip)+ except Exception:+ _teardown_containers(docker_client, [container.id])+ raiselogger.info("ElastiCache: started cluster node %s at %s:%s", name, container_ip, port)return container_ip, port, container.idexcept Exception as e:@@ -636,7 +713,7 @@ def _wait_redis_ready(container, timeout=15, cli_args=()):return False-def _bootstrap_redis_cluster(bootstrap_container, node_addrs, replicas_per_shard):+def _bootstrap_redis_cluster(bootstrap_container, node_addrs, replicas_per_shard, cli_args=()):"""Run ``redis-cli --cluster create`` inside one of the nodes.``node_addrs`` is a list of ``"ip:port"`` strings with primaries first,@@ -644,7 +721,7 @@ def _bootstrap_redis_cluster(bootstrap_container, node_addrs, replicas_per_shard``--cluster-replicas N`` is set)."""cmd = (- ["redis-cli", "--cluster", "create"]+ ["redis-cli", *cli_args, "--cluster", "create"]+ node_addrs+ ["--cluster-replicas", str(replicas_per_shard), "--cluster-yes"])@@ -690,7 +767,8 @@ def _teardown_containers(docker_client, container_ids):logger.warning("ElastiCache: cleanup failed for %s: %s", cid, e)-def _build_real_cluster_rg(rg_id, engine, engine_version, num_node_groups, replicas_per_shard):+def _build_real_cluster_rg(rg_id, engine, engine_version, num_node_groups, replicas_per_shard,+ transit=False, auth_token=None):"""Spawn cluster-enabled nodes and run ``redis-cli --cluster create``.Returns ``(node_groups, container_ids)`` on success, or@@ -730,6 +808,7 @@ def _build_real_cluster_rg(rg_id, engine, engine_version, num_node_groups, repliengine=engine,engine_version=engine_version,labels={**common_labels, "node_group": ng_id, "role": "primary"},+ transit=transit, auth_token=auth_token,)if not cid:return None, container_ids@@ -751,17 +830,19 @@ def _build_real_cluster_rg(rg_id, engine, engine_version, num_node_groups, repli"role": "replica","replica_idx": str(r),},+ transit=transit, auth_token=auth_token,)if not cid:return None, container_idscontainer_ids.append(cid)replicas.append({"ng_id": ng_id, "replica_idx": r, "ip": ip, "port": port, "cid": cid})+ cli_args = _node_cli_args(transit, auth_token)# Wait for every node to accept connections before bootstrapping.for node in primaries + replicas:try:container = docker_client.containers.get(node["cid"])- if not _wait_redis_ready(container):+ if not _wait_redis_ready(container, cli_args=cli_args):logger.warning("ElastiCache: cluster node %s never reported PONG", node["cid"])return None, container_idsexcept Exception as e:@@ -776,10 +857,10 @@ def _build_real_cluster_rg(rg_id, engine, engine_version, num_node_groups, replilogger.warning("ElastiCache: bootstrap container lookup failed: %s", e)return None, container_ids- if not _bootstrap_redis_cluster(bootstrap, addrs, replicas_per_shard):+ if not _bootstrap_redis_cluster(bootstrap, addrs, replicas_per_shard, cli_args=cli_args):return None, container_ids- if not _wait_cluster_ok(bootstrap):+ if not _wait_cluster_ok(bootstrap, cli_args=cli_args):logger.warning("ElastiCache: cluster %s did not reach state:ok", rg_id)return None, container_ids@@ -989,6 +1070,11 @@ def _create_cache_cluster(p):if cluster_id in _clusters:return _error("CacheClusterAlreadyExists", f"Cluster {cluster_id} already exists", 400)+ transit = _p(p, "TransitEncryptionEnabled", "false").lower() == "true"+ auth_token = _p(p, "AuthToken")+ if auth_token and not transit:+ return _error("InvalidParameterValue",+ "The AUTH token is only supported when encryption-in-transit is enabled", 400)arn = _arn_cluster(cluster_id)account_id = get_account_id()@@ -997,6 +1083,8 @@ def _create_cache_cluster(p):name=f"ministack-elasticache-{account_id}-{region}-{cluster_id}",engine=engine,engine_version=engine_version,+ transit=transit,+ auth_token=auth_token or None,labels={**container_reaper.own_labels("elasticache"),"cluster_id": cluster_id,@@ -1039,6 +1127,11 @@ def _create_cache_cluster(p):"SnapshotRetentionLimit": int(_p(p, "SnapshotRetentionLimit") or "0"),"SnapshotWindow": _p(p, "SnapshotWindow") or "05:00-06:00","PreferredMaintenanceWindow": _p(p, "PreferredMaintenanceWindow") or "sun:05:00-sun:06:00",+ "AuthTokenEnabled": bool(auth_token),+ "TransitEncryptionEnabled": transit,+ "TransitEncryptionMode": (_p(p, "TransitEncryptionMode") or "required") if transit else None,+ "AtRestEncryptionEnabled": False,+ "_auth_tokens": [auth_token] if auth_token else [],"CacheNodes": [{"CacheNodeId": f"{i:04d}",@@ -1239,6 +1332,16 @@ def _create_replication_group(p):validation_error = _validate_create_replication_group_request(p)if validation_error:return validation_error+ transit = _p(p, "TransitEncryptionEnabled", "false").lower() == "true"+ transit_mode = (_p(p, "TransitEncryptionMode") or "required") if transit else None+ auth_token = _p(p, "AuthToken")+ if auth_token and not transit:+ return _error("InvalidParameterValue",+ "The AUTH token is only supported when encryption-in-transit is enabled", 400)+ if auth_token and transit_mode == "preferred":+ return _error("InvalidParameterCombination",+ "Transit encryption preferred is not supported with access control enabled clusters.", 400)+ at_rest = _p(p, "AtRestEncryptionEnabled", "true" if engine == "valkey" else "false").lower() == "true"# AWS rejects NumNodeGroups=2: cluster-mode-enabled requires the redis-# cluster minimum of 3 masters; cluster-mode-disabled requires 1.@@ -1281,6 +1384,7 @@ def _create_replication_group(p):if use_real_cluster:node_groups, container_ids = _build_real_cluster_rg(rg_id, engine, engine_version, num_node_groups, replicas_per_node_group,+ transit=transit, auth_token=auth_token or None,)if node_groups is None:# Bootstrap failed — clean up partial state and fall back.@@ -1300,6 +1404,8 @@ def _create_replication_group(p):name=f"ministack-elasticache-rg-{account_id}-{region}-{rg_id}-{ng_id}",engine=engine,engine_version=engine_version,+ transit=transit,+ auth_token=auth_token or None,labels={**container_reaper.own_labels("elasticache"),"rg_id": rg_id,@@ -1371,9 +1477,11 @@ def _create_replication_group(p):"CacheSubnetGroupName": subnet_group,"PreferredMaintenanceWindow": maintenance_window,"SecurityGroups": security_groups,- "AuthTokenEnabled": _p(p, "AuthToken") != "",- "TransitEncryptionEnabled": _p(p, "TransitEncryptionEnabled", "false").lower() == "true",- "AtRestEncryptionEnabled": _p(p, "AtRestEncryptionEnabled", "false").lower() == "true",+ "AuthTokenEnabled": bool(auth_token),+ "TransitEncryptionEnabled": transit,+ "TransitEncryptionMode": transit_mode,+ "AtRestEncryptionEnabled": at_rest,+ "_auth_tokens": [auth_token] if auth_token else [],"AutoMinorVersionUpgrade": _p(p, "AutoMinorVersionUpgrade", "true").lower() == "true","AutomaticFailover": "enabled" if _p(p, "AutomaticFailoverEnabled", "false").lower() == "true" else "disabled","MultiAZ": "enabled" if _p(p, "MultiAZEnabled", "false").lower() == "true" else "disabled",@@ -1415,9 +1523,10 @@ def _create_replication_group(p):"SnapshotWindow": snapshot_window,"PreferredMaintenanceWindow": maintenance_window,"LogDeliveryConfigurations": log_delivery_configs,- "AtRestEncryptionEnabled": _p(p, "AtRestEncryptionEnabled", "false").lower() == "true",- "AuthTokenEnabled": _p(p, "AuthToken") != "",- "TransitEncryptionEnabled": _p(p, "TransitEncryptionEnabled", "false").lower() == "true",+ "AtRestEncryptionEnabled": at_rest,+ "AuthTokenEnabled": bool(auth_token),+ "TransitEncryptionEnabled": transit,+ "TransitEncryptionMode": transit_mode,"CacheNodes": [{"CacheNodeId": m.get("CacheNodeId", "0001"),@@ -1508,6 +1617,11 @@ def _modify_replication_group(p):rg["EngineVersion"] = _p(p, "EngineVersion")if _p(p, "CacheParameterGroupName"):rg["CacheParameterGroupName"] = _p(p, "CacheParameterGroupName")+ if _p(p, "AuthToken") or _p(p, "AuthTokenUpdateStrategy") == "DELETE":+ if not rg.get("TransitEncryptionEnabled"):+ return _error("InvalidParameterValue",+ "The AUTH token modification is only supported when encryption-in-transit is enabled", 400)+ _update_auth_token(rg, _p(p, "AuthToken"), _p(p, "AuthTokenUpdateStrategy") or "ROTATE")user_group_ids_to_add = _extract_configs(p, "UserGroupIdsToAdd", ("member",))user_group_ids_to_remove = _extract_configs(p, "UserGroupIdsToRemove", ("member",))@@ -1528,6 +1642,40 @@ def _modify_replication_group(p):f"<ModifyReplicationGroupResult><ReplicationGroup>{_rg_xml(rg)}</ReplicationGroup></ModifyReplicationGroupResult>")+def _update_auth_token(rg, token, strategy):+ """ROTATE keeps the newest previous token beside the new one (none before: the+ token is optional), SET leaves only the new one, DELETE removes AUTH."""+ tokens = rg.get("_auth_tokens") or []+ login = tokens[-1] if tokens else None+ if strategy == "DELETE":+ rg["_auth_tokens"], rg["_auth_optional"] = [], False+ rules = ["resetpass", "nopass"]+ elif strategy == "SET":+ rg["_auth_tokens"], rg["_auth_optional"] = [token], False+ rules = ["resetpass", f">{token}"]+ else:+ optional = rg.get("_auth_optional") or not rg.get("AuthTokenEnabled")+ rg["_auth_tokens"], rg["_auth_optional"] = tokens[-1:] + [token], optional+ rules = ["resetpass", "nopass"] if optional else ["resetpass", *(f">{t}" for t in rg["_auth_tokens"])]+ rg["AuthTokenEnabled"] = strategy != "DELETE"+ for cluster_id in rg.get("MemberClusters") or []:+ if cluster_id in _clusters:+ _clusters[cluster_id]["AuthTokenEnabled"] = rg["AuthTokenEnabled"]+ docker_client = _get_docker()+ if not docker_client:+ return+ new_login = rg["_auth_tokens"][-1] if rg["_auth_tokens"] else None+ for cid in rg.get("_docker_container_ids") or []:+ try:+ container = docker_client.containers.get(cid)+ container.exec_run(["redis-cli", *_node_cli_args(True, login), "-p", "6379",+ "ACL", "SETUSER", "default", "on", *rules])+ container.exec_run(["redis-cli", *_node_cli_args(True, new_login), "-p", "6379",+ "CONFIG", "SET", "masterauth", new_login or ""])+ except Exception as e:+ logger.warning("ElastiCache: AUTH token update failed on %s: %s", cid, e)++def _increase_replica_count(p):rg_id = _p(p, "ReplicationGroupId")rg = _replication_groups.get(rg_id)@@ -1583,63 +1731,70 @@ def _decrease_replica_count(p):# ---- Subnet Groups -----def _create_subnet_group(p):- name = _p(p, "CacheSubnetGroupName")- desc = _p(p, "CacheSubnetGroupDescription") or ""- arn = _arn_subnet_group(name)+def _resolve_subnets(p):+ """Subnets and VPC of the SubnetIds in the request, or None when one is unknown or they span VPCs."""+ from ministack.services import ec2- subnets = []- idx = 1- while _p(p, f"SubnetIds.member.{idx}"):+ ec2._ensure_defaults_initialized()+ subnets, vpc_ids = [], set()+ for subnet_id in _extract_configs(p, "SubnetIds", ("SubnetIdentifier", "member")):+ subnet = ec2._subnets.get(subnet_id)+ if not subnet:+ return None+ vpc_ids.add(subnet["VpcId"])subnets.append({- "SubnetIdentifier": _p(p, f"SubnetIds.member.{idx}"),- "SubnetAvailabilityZone": {"Name": f"{get_region()}{'abcdef'[(idx - 1) % 6]}"},+ "SubnetIdentifier": subnet_id,+ "SubnetAvailabilityZone": {"Name": subnet.get("AvailabilityZone", f"{get_region()}a")},})- idx += 1+ if len(vpc_ids) != 1:+ return None+ return subnets, vpc_ids.pop()+++def _subnet_group_xml(sg):+ subnets_xml = "".join(+ f"<Subnet><SubnetIdentifier>{s['SubnetIdentifier']}</SubnetIdentifier>"+ f"<SubnetAvailabilityZone><Name>{s['SubnetAvailabilityZone']['Name']}</Name></SubnetAvailabilityZone></Subnet>"+ for s in sg.get("Subnets", [])+ )+ return (+ f"<CacheSubnetGroupName>{sg['CacheSubnetGroupName']}</CacheSubnetGroupName>"+ f"<CacheSubnetGroupDescription>{xml_escape(sg.get('CacheSubnetGroupDescription', ''))}</CacheSubnetGroupDescription>"+ f"<VpcId>{sg.get('VpcId', '')}</VpcId>"+ f"<Subnets>{subnets_xml}</Subnets>"+ f"<ARN>{sg.get('ARN', '')}</ARN>"+ )++def _create_subnet_group(p):+ name = _p(p, "CacheSubnetGroupName")+ if name in _subnet_groups:+ return _error("CacheSubnetGroupAlreadyExists", f"Cache subnet group {name} already exists.", 400)+ resolved = _resolve_subnets(p)+ if resolved is None:+ return _error("InvalidSubnet", "The requested subnet is invalid, or multiple subnets were requested "+ "that are not all in a common VPC.", 400)+ subnets, vpc_id = resolved+ arn = _arn_subnet_group(name)_subnet_groups[name] = {"CacheSubnetGroupName": name,- "CacheSubnetGroupDescription": desc,- "VpcId": "vpc-00000000",+ "CacheSubnetGroupDescription": _p(p, "CacheSubnetGroupDescription") or "",+ "VpcId": vpc_id,"Subnets": subnets,"ARN": arn,}- _tags[arn] = []- subnets_xml = "".join(- f"<Subnet><SubnetIdentifier>{s['SubnetIdentifier']}</SubnetIdentifier>"- f"<SubnetAvailabilityZone><Name>{s['SubnetAvailabilityZone']['Name']}</Name></SubnetAvailabilityZone>"- f"</Subnet>" for s in subnets- )+ _tags[arn] = _extract_tags(p)return _xml(200, "CreateCacheSubnetGroupResponse",- f"<CreateCacheSubnetGroupResult><CacheSubnetGroup>"- f"<CacheSubnetGroupName>{name}</CacheSubnetGroupName>"- f"<CacheSubnetGroupDescription>{desc}</CacheSubnetGroupDescription>"- f"<Subnets>{subnets_xml}</Subnets>"- f"<ARN>{arn}</ARN>"+ f"<CreateCacheSubnetGroupResult><CacheSubnetGroup>{_subnet_group_xml(_subnet_groups[name])}"f"</CacheSubnetGroup></CreateCacheSubnetGroupResult>")def _describe_subnet_groups(p):name = _p(p, "CacheSubnetGroupName")if name and name not in _subnet_groups:- return _error("CacheSubnetGroupNotFoundFault", f"Cache subnet group {name} not found.", 404)- groups = [_subnet_groups[name]] if name and name in _subnet_groups else list(_subnet_groups.values())- members = ""- for g in groups:- # SubnetList.member.locationName = "Subnet"- subnets_xml = "".join(- f"<Subnet><SubnetIdentifier>{s['SubnetIdentifier']}</SubnetIdentifier>"- f"<SubnetAvailabilityZone><Name>{s['SubnetAvailabilityZone']['Name']}</Name></SubnetAvailabilityZone></Subnet>"- for s in g.get("Subnets", [])- )- # CacheSubnetGroups.member.locationName = "CacheSubnetGroup"- members += (- f"<CacheSubnetGroup><CacheSubnetGroupName>{g['CacheSubnetGroupName']}</CacheSubnetGroupName>"- f"<CacheSubnetGroupDescription>{g.get('CacheSubnetGroupDescription', '')}</CacheSubnetGroupDescription>"- f"<VpcId>{g.get('VpcId', '')}</VpcId>"- f"<Subnets>{subnets_xml}</Subnets>"- f"<ARN>{g.get('ARN', '')}</ARN></CacheSubnetGroup>"- )+ return _error("CacheSubnetGroupNotFoundFault", f"Cache subnet group {name} not found.", 400)+ groups = [_subnet_groups[name]] if name else list(_subnet_groups.values())+ members = "".join(f"<CacheSubnetGroup>{_subnet_group_xml(g)}</CacheSubnetGroup>" for g in groups)return _xml(200, "DescribeCacheSubnetGroupsResponse",f"<DescribeCacheSubnetGroupsResult><CacheSubnetGroups>{members}</CacheSubnetGroups></DescribeCacheSubnetGroupsResult>")@@ -1647,7 +1802,11 @@ def _describe_subnet_groups(p):def _delete_subnet_group(p):name = _p(p, "CacheSubnetGroupName")if name not in _subnet_groups:- return _error("CacheSubnetGroupNotFoundFault", f"Cache subnet group {name} not found.", 404)+ return _error("CacheSubnetGroupNotFoundFault", f"Cache subnet group {name} not found.", 400)+ in_use = any(r.get("CacheSubnetGroupName") == name+ for r in list(_clusters.values()) + list(_replication_groups.values()))+ if in_use:+ return _error("CacheSubnetGroupInUse", f"Cache subnet group {name} is currently in use.", 400)sg = _subnet_groups.pop(name, None)if sg:_tags.pop(sg.get("ARN", ""), None)@@ -1658,28 +1817,17 @@ def _modify_subnet_group(p):name = _p(p, "CacheSubnetGroupName")sg = _subnet_groups.get(name)if not sg:- return _error("CacheSubnetGroupNotFoundFault", f"Subnet group {name} not found", 404)-+ return _error("CacheSubnetGroupNotFoundFault", f"Cache subnet group {name} not found.", 400)+ if _extract_configs(p, "SubnetIds", ("SubnetIdentifier", "member")):+ resolved = _resolve_subnets(p)+ if resolved is None:+ return _error("InvalidSubnet", "The requested subnet is invalid, or multiple subnets were requested "+ "that are not all in a common VPC.", 400)+ sg["Subnets"], sg["VpcId"] = resolvedif _p(p, "CacheSubnetGroupDescription"):sg["CacheSubnetGroupDescription"] = _p(p, "CacheSubnetGroupDescription")-- subnets = []- idx = 1- while _p(p, f"SubnetIds.member.{idx}"):- subnets.append({- "SubnetIdentifier": _p(p, f"SubnetIds.member.{idx}"),- "SubnetAvailabilityZone": {"Name": f"{get_region()}{'abcdef'[(idx - 1) % 6]}"},- })- idx += 1- if subnets:- sg["Subnets"] = subnets-- arn = sg.get("ARN", _arn_subnet_group(name))return _xml(200, "ModifyCacheSubnetGroupResponse",- f"<ModifyCacheSubnetGroupResult><CacheSubnetGroup>"- f"<CacheSubnetGroupName>{name}</CacheSubnetGroupName>"- f"<CacheSubnetGroupDescription>{sg.get('CacheSubnetGroupDescription', '')}</CacheSubnetGroupDescription>"- f"<ARN>{arn}</ARN>"+ f"<ModifyCacheSubnetGroupResult><CacheSubnetGroup>{_subnet_group_xml(sg)}"f"</CacheSubnetGroup></ModifyCacheSubnetGroupResult>")@@ -2866,7 +3014,6 @@ def _log_delivery_configs_xml(configs):def _cluster_xml_inner(c):"""Render cluster fields — no wrapping element."""- ep = c.get("_endpoint", {})nodes_xml = ""# CacheNodeList.member.locationName = "CacheNode"az = c.get("PreferredAvailabilityZone", "")@@ -2924,10 +3071,19 @@ def _cluster_xml_inner(c):f"<SnapshotRetentionLimit>{c.get('SnapshotRetentionLimit', 0)}</SnapshotRetentionLimit>"f"<SnapshotWindow>{c.get('SnapshotWindow', '')}</SnapshotWindow>"f"<LogDeliveryConfigurations>{log_delivery_configs_xml}</LogDeliveryConfigurations>"+ f"<AuthTokenEnabled>{str(c.get('AuthTokenEnabled', False)).lower()}</AuthTokenEnabled>"+ f"<TransitEncryptionEnabled>{str(c.get('TransitEncryptionEnabled', False)).lower()}</TransitEncryptionEnabled>"+ f"{_transit_mode_xml(c)}"+ f"<AtRestEncryptionEnabled>{str(c.get('AtRestEncryptionEnabled', False)).lower()}</AtRestEncryptionEnabled>"f"<CacheNodes>{nodes_xml}</CacheNodes>")+def _transit_mode_xml(record):+ mode = record.get("TransitEncryptionMode")+ return f"<TransitEncryptionMode>{mode}</TransitEncryptionMode>" if mode else ""++def _cluster_xml(c):"""For list contexts (DescribeCacheClusters), wrap each item in<CacheCluster> — that's the AWS-spec locationName for@@ -2996,6 +3152,7 @@ def _rg_xml(rg):f"<ClusterMode>{rg.get('ClusterMode', 'enabled' if rg.get('ClusterEnabled', False) else 'disabled')}</ClusterMode>"f"<AuthTokenEnabled>{str(rg.get('AuthTokenEnabled', False)).lower()}</AuthTokenEnabled>"f"<TransitEncryptionEnabled>{str(rg.get('TransitEncryptionEnabled', False)).lower()}</TransitEncryptionEnabled>"+ f"{_transit_mode_xml(rg)}"f"<AtRestEncryptionEnabled>{str(rg.get('AtRestEncryptionEnabled', False)).lower()}</AtRestEncryptionEnabled>"f"<SnapshotRetentionLimit>{rg.get('SnapshotRetentionLimit', 0)}</SnapshotRetentionLimit>"f"<SnapshotWindow>{rg.get('SnapshotWindow', '')}</SnapshotWindow>"diff --git a/ministack/services/kms.py b/ministack/services/kms.pyindex 00acae02..42a2391a 100644--- a/ministack/services/kms.py+++ b/ministack/services/kms.py@@ -821,7 +821,6 @@ def _verify(data):public_key.verify(signature, message, pad, utils.Prehashed(hash_algo))else:public_key.verify(signature, message, pad, hash_algo)- valid = Trueexcept InvalidSignature:return error_response_json("KMSInvalidSignatureException",diff --git a/ministack/services/lambda_durable.py b/ministack/services/lambda_durable.pyindex bb10de2d..73e3aa71 100644--- a/ministack/services/lambda_durable.py+++ b/ministack/services/lambda_durable.py@@ -773,7 +773,6 @@ def _apply_update(rec: dict, upd: dict) -> None:# ChainedInvokeStarted event is on the log before a fast child can# append its ChainedInvokeSucceeded (history order matters to# replaying SDKs).- ci_opts = upd.get("ChainedInvokeOptions") or {}elif op_type == "EXECUTION":details = existing.setdefault("ExecutionDetails", {})if rec.get("InputPayload"):diff --git a/ministack/services/lambda_svc.py b/ministack/services/lambda_svc.pyindex fd729211..ba1086d8 100644--- a/ministack/services/lambda_svc.py+++ b/ministack/services/lambda_svc.py@@ -1371,7 +1371,6 @@ def invoke_durable_resume(function_name: str, durable_arn: str, original_event:func = _functions.get(canonical)if not func:return- config = func.get("config") or func# Set the durable context to the SAME ARN/token so the SDK reads the# accumulated operations as InitialExecutionState (replay path)._durable_ctx.set({@@ -1731,21 +1730,14 @@ def _layer_access_denied(layer_arn: str):)-# The shape of a layer version ARN that can exist on AWS: a 12-digit account, a-# name of at most 140 of [a-zA-Z0-9-_], and a version from 1 with no leading zero.+# A layer version ARN that can exist on AWS._POSSIBLE_LAYER_VERSION_ARN_RE = re.compile(r"arn:aws:lambda:[a-z0-9-]+:[0-9]{12}:layer:[a-zA-Z0-9-_]{1,140}:[1-9][0-9]*")def _resolve_cross_account_layer(layer_arn: str, spec, name_and_version):- """Another account's layer. A layer MiniStack holds answers through its grant,- so a version it does not hold or does not grant is denied, as on AWS. A layer- it does not hold resolves as an empty version: on AWS it may be a public- vendor or AWS extension, and offline a real ARN looks like a made-up one, so- any account and version that could exist resolve. Its bytes are not- available, so it has CodeSize 0, leaves out what only the publisher knows,- and does not run. An ARN that cannot name a layer version on AWS is denied."""+ """Another account's layer: through its grant if MiniStack holds it, else an empty version."""if spec.region != get_region():return _layer_access_denied(layer_arn)layer_name, version = name_and_version@@ -5868,8 +5860,7 @@ def _resolve_layer_zip(attachment: str | dict) -> bytes | None:"""Return attached content, including versions deleted since attachment."""if isinstance(attachment, dict):if attachment.get("CodeSize") == 0:- # Attached as an empty layer MiniStack did not hold; a version- # published under that ARN later must not load without its grant.+ # An empty layer never loads content published under its ARN later.return Noneattachment = attachment.get("Arn", "")try:@@ -6479,9 +6470,7 @@ def _list_layer_versions(layer_name: str, query_params: dict):def _layer_name_too_long(layer_name: str):- """GetLayerVersion's LayerName limit. It covers the whole value, a layer ARN- included, and is checked before the version. GetLayerVersionByArn has no- LayerName member and no such limit."""+ """GetLayerVersion's 140-character LayerName limit, checked before the version."""if len(layer_name) <= 140:return Nonereturn error_response_json(diff --git a/ministack/services/mq.py b/ministack/services/mq.pyindex 5785c1b4..1703cb29 100644--- a/ministack/services/mq.py+++ b/ministack/services/mq.py@@ -509,7 +509,6 @@ def _create_broker(body: dict) -> tuple:engine_version = body.get("engineVersion") or engine_versions[0]deployment_mode = str(body.get("deploymentMode", "SINGLE_INSTANCE")).upper()host_instance_type = body.get("hostInstanceType", "mq.m5.large")- storage_type = body.get("storageType", "EBS")# Create broker in statebroker_id = new_uuid()diff --git a/ministack/services/msk.py b/ministack/services/msk.pyindex 84ed9731..cae63d63 100644--- a/ministack/services/msk.py+++ b/ministack/services/msk.py@@ -327,7 +327,6 @@ def _delete_cluster(arn: str) -> tuple:validation_error = _validate_existing_cluster_arn(arn)if validation_error:return validation_error- info = _clusters.get(arn)_clusters.pop(arn, None)_scram_secrets.pop(arn, None)_tags.pop(arn, None)diff --git a/ministack/services/mwaa.py b/ministack/services/mwaa.pyindex 6f9dc2c3..89f3b5b1 100644--- a/ministack/services/mwaa.py+++ b/ministack/services/mwaa.py@@ -670,7 +670,7 @@ async def _invoke_rest_api(method, path, headers, body, query_params):"RestApiStatusCode": resp.status_code,"RestApiResponse": payload,})- except Exception as e:+ except Exception:logger.exception("MWAA InvokeRestApi failure")# AWS doesn't leak the underlying Python exception text. The# InternalServerException message is opaque to the caller.diff --git a/ministack/services/rds.py b/ministack/services/rds.pyindex 53ffca36..fe39f6bb 100644--- a/ministack/services/rds.py+++ b/ministack/services/rds.py@@ -5033,7 +5033,6 @@ def _create_db_instance_impl(p):"ProcessorFeatures": [],"DeletionProtection": _p(p, "DeletionProtection") == "true","AssociatedRoles": [],- "MaxAllocatedStorage": int(_p(p, "MaxAllocatedStorage") or str(allocated_storage)),"TagList": [],"CustomerOwnedIpEnabled": False,"ActivityStreamStatus": "stopped",@@ -5052,6 +5051,9 @@ def _create_db_instance_impl(p):"_internal_port": internal_port,"_MasterUserPassword": master_pass,}+ # Only a request that turns storage autoscaling on sets the threshold.+ if _p(p, "MaxAllocatedStorage"):+ instance["MaxAllocatedStorage"] = int(_p(p, "MaxAllocatedStorage"))if parent and reader_launch:# A replicating reader keeps its own endpoint and container fields;# the marker excludes it from shared-container aliasing and makes@@ -5557,6 +5559,9 @@ def _modify_db_instance(p):instance[instance_key] = valelse:pending[instance_key] = val+ # A threshold equal to the allocated storage turns storage autoscaling off.+ if instance.get("MaxAllocatedStorage") == instance.get("AllocatedStorage"):+ instance.pop("MaxAllocatedStorage", None)if new_pass:old_pass = instance.get("_MasterUserPassword", "password")@@ -5782,6 +5787,9 @@ def _create_read_replica(p):arn = f"arn:aws:rds:{get_region()}:{get_account_id()}:db:{replica_id}"replica = dict(source)+ replica.pop("MaxAllocatedStorage", None)+ if _p(p, "MaxAllocatedStorage"):+ replica["MaxAllocatedStorage"] = int(_p(p, "MaxAllocatedStorage"))replica.update({"DBInstanceIdentifier": replica_id,"DBInstanceArn": arn,@@ -9606,7 +9614,7 @@ def _instance_xml(i):<ProcessorFeatures/><DeletionProtection>{str(i.get('DeletionProtection',False)).lower()}</DeletionProtection><AssociatedRoles/>- <MaxAllocatedStorage>{i.get('MaxAllocatedStorage',i.get('AllocatedStorage',20))}</MaxAllocatedStorage>+ {f"<MaxAllocatedStorage>{i['MaxAllocatedStorage']}</MaxAllocatedStorage>" if i.get('MaxAllocatedStorage') else ""}<TagList>{tag_xml}</TagList>{cert_xml}<CustomerOwnedIpEnabled>{str(i.get('CustomerOwnedIpEnabled',False)).lower()}</CustomerOwnedIpEnabled>diff --git a/ministack/services/s3.py b/ministack/services/s3.pyindex 0063d78b..55e41199 100644--- a/ministack/services/s3.py+++ b/ministack/services/s3.py@@ -2236,9 +2236,15 @@ def _presign_time_error(amz_date: str, expires: str, path: str) -> tuple | None:except ValueError:return _error("AuthorizationQueryParametersError","X-Amz-Date must be in the ISO8601 Long Format \"yyyyMMdd'T'HHmmss'Z'\"", 400, path)- if expires and not expires.isdigit():+ if not expires:+ return _error("AuthorizationQueryParametersError",+ "Query-string authentication version 4 requires the X-Amz-Algorithm, X-Amz-Credential, "+ "X-Amz-Signature, X-Amz-Date, X-Amz-SignedHeaders, and X-Amz-Expires parameters.", 400, path)+ if expires.startswith("-") and expires[1:].isdigit():+ return _error("AuthorizationQueryParametersError", "X-Amz-Expires must be non-negative", 400, path)+ if not expires.isdigit():return _error("AuthorizationQueryParametersError", "X-Amz-Expires should be a number", 400, path)- if expires and int(expires) > _PRESIGN_MAX_EXPIRES:+ if int(expires) > _PRESIGN_MAX_EXPIRES:return _error("AuthorizationQueryParametersError","X-Amz-Expires must be less than a week (in seconds); that is, the given X-Amz-Expires "f"must be less than {_PRESIGN_MAX_EXPIRES} seconds", 400, path)@@ -2260,6 +2266,11 @@ def _verify_presigned_sigv4(method, path, headers, query_params):if not signature:return None # not a presigned URLalgorithm = _qp(query_params, "X-Amz-Algorithm", "") or _qp(query_params, "x-amz-algorithm", "")+ if algorithm == "AWS4-ECDSA-P256-SHA256":+ # The IAM layer verifies a SigV4a signature; its query parameters follow the same rules.+ return _presign_time_error(_qp(query_params, "X-Amz-Date", "") or _qp(query_params, "x-amz-date", ""),+ _qp(query_params, "X-Amz-Expires", "") or _qp(query_params, "x-amz-expires", ""),+ path)if algorithm != "AWS4-HMAC-SHA256":return None # only SigV4 presigned URLs are verified@@ -8626,6 +8637,24 @@ def _complete_multipart_upload(# functionality) for the completed multipart object.part_records.append({"PartNumber": pn, "Size": len(stored["body"])})+ # A declared x-amz-mp-object-size that differs from the parts' total is refused.+ declared_size = (headers or {}).get("x-amz-mp-object-size")+ if declared_size is not None:+ try:+ declared = int(declared_size)+ except ValueError:+ return _error("InvalidRequest", f"Value for x-amz-mp-object-size header is invalid: '{declared_size}'",+ 400, f"/{bucket_name}/{key}")+ actual_size = sum(len(part_body) for part_body in combined_parts)+ if declared != actual_size:+ return _error(+ "InvalidRequest",+ f"The provided 'x-amz-mp-object-size' header value {declared_size} does not match "+ f"what was computed: {actual_size}",+ 400,+ f"/{bucket_name}/{key}",+ )+composite = _composite_checksum(checksum_algorithm, part_digests)if checksum_algorithm:requested = (headers or {}).get(f"x-amz-checksum-{checksum_algorithm.lower()}")diff --git a/ministack/services/sns.py b/ministack/services/sns.pyindex f1ed8ab4..5477b8f6 100644--- a/ministack/services/sns.py+++ b/ministack/services/sns.py@@ -676,7 +676,6 @@ def _confirm_subscription(params):topic, err = _get_topic(_p(params, "TopicArn"))if err is not None:return err- topic_arn = topic["arn"]if not token:return _error("InvalidParameterException", "Token is required", 400)diff --git a/ministack/services/ssm.py b/ministack/services/ssm.pyindex 6df29fef..c8358fe2 100644--- a/ministack/services/ssm.py+++ b/ministack/services/ssm.py@@ -43,8 +43,7 @@ _tags = AccountRegionScopedDict()_commands = AccountRegionScopedDict()_service_settings = AccountRegionScopedDict()-# SettingIds are the botocore ssm model's list; (default value, default LastModifiedDate) observed on AWS-# (GetServiceSetting, us-east-1, 2026-10-10), except high-throughput-enabled (ResetServiceSetting API reference example).+# The botocore model's SettingIds with the (value, LastModifiedDate) AWS reports before any update._SERVICE_SETTING_DEFAULTS = {"/ssm/parameter-store/high-throughput-enabled": ("false", 1555532571.138),"/ssm/parameter-store/default-parameter-tier": ("Standard", 1565131339.38),@@ -57,8 +56,7 @@ _SERVICE_SETTING_DEFAULTS = {"/ssm/appmanager/appmanager-enabled": ("False", 1604983265.572),"/ssm/opsinsights/opscenter": ("Disabled", 1622077352.094),}-# Accepted SettingValues per the UpdateServiceSetting API reference; an id missing here takes any string.-# The error message is observed for default-parameter-tier; its format for the other ids is inferred.+# SettingValues the UpdateServiceSetting API reference lists; an id not here takes any string._SERVICE_SETTING_VALUES = {"/ssm/parameter-store/high-throughput-enabled": ("true", "false"),"/ssm/parameter-store/default-parameter-tier": ("Standard", "Advanced", "Intelligent-Tiering"),@@ -384,7 +382,7 @@ def _put_parameter(data):"LastModifiedDate": now,"DataType": data.get("DataType", "text"),"Description": data.get("Description", existing.get("Description", "") if existing else ""),- "Tier": data.get("Tier") or _default_tier(data, value),+ "Tier": data.get("Tier") or _default_tier(data, value, existing),"AllowedPattern": data.get("AllowedPattern", ""),"Policies": data.get("Policies", []),"Labels": [],@@ -1136,12 +1134,15 @@ def _service_setting_out(setting_id):}-def _default_tier(data, value):- """Tier of a parameter put without one: the account and region's default-parameter-tier setting."""+def _default_tier(data, value, existing):+ """Tier of a put without one: the default-parameter-tier setting; an Advanced parameter stays Advanced."""+ if existing and existing.get("Tier") == "Advanced":+ return "Advanced"setting = _service_settings.get(_DEFAULT_TIER_SETTING)tier = setting["value"] if setting else "Standard"if tier == "Intelligent-Tiering":- return "Advanced" if len(value.encode()) > 4096 or data.get("Policies") else "Standard"+ needs_advanced = len(value.encode()) > 4096 or data.get("Policies") or len(_parameters) > 10000+ return "Advanced" if needs_advanced else "Standard"return tierdiff --git a/tests/test_cfn.py b/tests/test_cfn.pyindex 142a6252..273cd1a2 100644--- a/tests/test_cfn.py+++ b/tests/test_cfn.py@@ -36336,7 +36336,7 @@ def _ec_cache_stack(s, *, rg_description="sessions", num_cache_clusters=2,"Subnets": {"Type": "AWS::ElastiCache::SubnetGroup", "Properties": {"CacheSubnetGroupName": f"subnets-{s}","Description": subnet_description,- "SubnetIds": ["subnet-aaa", "subnet-bbb"],+ "SubnetIds": ["subnet-00000001", "subnet-00000002"],}},"Params": {"Type": "AWS::ElastiCache::ParameterGroup", "Properties": {"CacheParameterGroupFamily": "redis7",@@ -36411,7 +36411,7 @@ def test_cfn_elasticache_stack_create_and_delete(cfn, ec):try:groups = ec.describe_cache_subnet_groups(CacheSubnetGroupName=f"subnets-{s}")subnets = groups["CacheSubnetGroups"][0]["Subnets"]- assert [n["SubnetIdentifier"] for n in subnets] == ["subnet-aaa", "subnet-bbb"]+ assert [n["SubnetIdentifier"] for n in subnets] == ["subnet-00000001", "subnet-00000002"]assert _ec_params(ec, out["Params"])["maxmemory-policy"] == "allkeys-lru"user = ec.describe_users(UserId=f"app-{s}")["Users"][0]@@ -36518,9 +36518,9 @@ def test_cfn_elasticache_generated_cluster_replaced_on_create_only_change(cfn, ereturn {"Resources": {"A": {"Type": "AWS::ElastiCache::SubnetGroup", "Properties": {- "Description": "a", "SubnetIds": ["subnet-a"]}},+ "Description": "a", "SubnetIds": ["subnet-00000001"]}},"B": {"Type": "AWS::ElastiCache::SubnetGroup", "Properties": {- "Description": "b", "SubnetIds": ["subnet-b"]}},+ "Description": "b", "SubnetIds": ["subnet-00000002"]}},"C": {"Type": "AWS::ElastiCache::CacheCluster", "Properties": {"Engine": "redis", "CacheNodeType": "cache.t3.micro", "NumCacheNodes": 1,"CacheSubnetGroupName": {"Ref": subnet_ref}, "SnapshotRetentionLimit": retention}},diff --git a/tests/test_dynamodb.py b/tests/test_dynamodb.pyindex 861885f1..7e954f45 100644--- a/tests/test_dynamodb.py+++ b/tests/test_dynamodb.py@@ -5494,7 +5494,7 @@ def test_dynamodb_execute_transaction_multi_insert(ddb):{"Statement": f'INSERT INTO "{name}" VALUE {{\'pk\': ?}}', "Parameters": [{"S": "a"}]},{"Statement": f'INSERT INTO "{name}" VALUE {{\'pk\': ?}}', "Parameters": [{"S": "b"}]},])- assert len(r["Responses"]) == 2+ assert r["Responses"] == []for pk in ("a", "b"):got = ddb.get_item(TableName=name, Key={"pk": {"S": pk}})assert "Item" in got@@ -7616,7 +7616,9 @@ def test_dynamodb_gsi_multi_attribute_keys(ddb):("a = :a AND b = :b AND s1 > :s1 AND s2 = :s2", {"s1": "a", "s2": "1"})):with pytest.raises(ClientError) as exc:query(cond, a="x", b="y", **vals)- assert exc.value.response["Error"]["Message"] == "Query key condition not supported"+ assert exc.value.response["Error"]["Message"] == (+ "RANGE key attributes s1 must have equality conditions specified in the query "+ "because a condition is present on key attribute s2")finally:ddb.delete_table(TableName=name)diff --git a/tests/test_elasticache.py b/tests/test_elasticache.pyindex f0dd92dd..9a05c181 100644--- a/tests/test_elasticache.py+++ b/tests/test_elasticache.py@@ -128,12 +128,12 @@ def test_elasticache_modify_subnet_group(ec):ec.create_cache_subnet_group(CacheSubnetGroupName="test-mod-ecsg",CacheSubnetGroupDescription="Test EC SG",- SubnetIds=["subnet-aaa"],+ SubnetIds=["subnet-00000001"],)ec.modify_cache_subnet_group(CacheSubnetGroupName="test-mod-ecsg",CacheSubnetGroupDescription="Updated EC SG",- SubnetIds=["subnet-bbb"],+ SubnetIds=["subnet-00000002"],)resp = ec.describe_cache_subnet_groups(CacheSubnetGroupName="test-mod-ecsg")assert resp["CacheSubnetGroups"][0]["CacheSubnetGroupDescription"] == "Updated EC SG"@@ -609,12 +609,12 @@ def test_elasticache_subnet_and_param_groups_are_region_scoped():east.create_cache_subnet_group(CacheSubnetGroupName=subnet_name,CacheSubnetGroupDescription="east subnet",- SubnetIds=["subnet-east"],+ SubnetIds=["subnet-00000001"],)west.create_cache_subnet_group(CacheSubnetGroupName=subnet_name,CacheSubnetGroupDescription="west subnet",- SubnetIds=["subnet-west"],+ SubnetIds=["subnet-00000001"],)east.create_cache_parameter_group(CacheParameterGroupName=param_name,@@ -1093,7 +1093,7 @@ def test_delete_cache_subnet_group(ec):ec.create_cache_subnet_group(CacheSubnetGroupName=name,CacheSubnetGroupDescription="To be deleted",- SubnetIds=["subnet-aaa"],+ SubnetIds=["subnet-00000001"],)# verify existsresp = ec.describe_cache_subnet_groups(CacheSubnetGroupName=name)@@ -1108,6 +1108,44 @@ def test_delete_cache_subnet_group(ec):assert "CacheSubnetGroupNotFoundFault" in str(exc.value)+def test_cache_subnet_group_keeps_subnets_and_tags(ec):+ name = f"sg-keep-{_uid()}"+ group = ec.create_cache_subnet_group(+ CacheSubnetGroupName=name,+ CacheSubnetGroupDescription="two subnets",+ SubnetIds=["subnet-00000001", "subnet-00000002"],+ Tags=[{"Key": "env", "Value": "dev"}],+ )["CacheSubnetGroup"]+ try:+ described = ec.describe_cache_subnet_groups(CacheSubnetGroupName=name)["CacheSubnetGroups"][0]+ for g in (group, described):+ assert g["VpcId"] == "vpc-00000001"+ assert [(s["SubnetIdentifier"], s["SubnetAvailabilityZone"]["Name"]) for s in g["Subnets"]] == [+ ("subnet-00000001", "us-east-1a"), ("subnet-00000002", "us-east-1b")]+ assert ec.list_tags_for_resource(ResourceName=group["ARN"])["TagList"] == [{"Key": "env", "Value": "dev"}]++ with pytest.raises(ClientError) as exc:+ ec.create_cache_subnet_group(CacheSubnetGroupName=name, CacheSubnetGroupDescription="again",+ SubnetIds=["subnet-00000001"])+ assert exc.value.response["Error"]["Code"] == "CacheSubnetGroupAlreadyExists"+ assert exc.value.response["ResponseMetadata"]["HTTPStatusCode"] == 400++ modified = ec.modify_cache_subnet_group(+ CacheSubnetGroupName=name, SubnetIds=["subnet-00000003"])["CacheSubnetGroup"]+ assert modified["VpcId"] == "vpc-00000001"+ assert [s["SubnetIdentifier"] for s in modified["Subnets"]] == ["subnet-00000003"]++ with pytest.raises(ClientError) as exc:+ ec.modify_cache_subnet_group(CacheSubnetGroupName=name, SubnetIds=["subnet-doesnotexist"])+ assert exc.value.response["Error"]["Code"] == "InvalidSubnet"+ finally:+ ec.delete_cache_subnet_group(CacheSubnetGroupName=name)++ with pytest.raises(ClientError) as exc:+ ec.describe_cache_subnet_groups(CacheSubnetGroupName=name)+ assert exc.value.response["ResponseMetadata"]["HTTPStatusCode"] == 400++# ---------------------------------------------------------------------------# 8. ResetCacheParameterGroup# ---------------------------------------------------------------------------@@ -1822,7 +1860,7 @@ def test_elasticache_restore_state_marks_clusters_for_respawn(monkeypatch):spawned = []- def fake_spawn(name, engine, engine_version, labels):+ def fake_spawn(name, engine, engine_version, labels, **_):spawned.append({"name": name, "engine": engine,"engine_version": engine_version, "labels": labels})return ("ministack-elasticache", 6379, f"cid-{name}")@@ -1865,7 +1903,7 @@ def test_elasticache_restore_state_wipes_stale_replication_group_container_ids(mset_request_account_id("000000000000")_ec.reset()monkeypatch.setattr(_ec, "_spawn_redis_container",- lambda name, engine, engine_version, labels: ("rg-host", 6379, f"cid-{name}"))+ lambda name, engine, engine_version, labels, **_: ("rg-host", 6379, f"cid-{name}"))_ec.load_persisted_state({"replication_groups": {@@ -1893,7 +1931,7 @@ def test_elasticache_restore_state_respawns_same_names_per_region(monkeypatch):_ec.reset()spawned = []- def fake_spawn(name, engine, engine_version, labels):+ def fake_spawn(name, engine, engine_version, labels, **_):spawned.append({"name": name, "labels": labels})return (f"{labels['region']}-host", 6379, f"cid-{name}")@@ -2297,3 +2335,85 @@ def test_serverless_ca_persists_across_restart(monkeypatch):monkeypatch.setattr(ec, "_serverless_ca", None)ec.load_persisted_state(state)assert ec.serverless_ca_cert_pem() == pem+++def test_replication_group_auth_token_needs_transit_encryption(ec):+ rg_id = f"auth-{_uid()}"+ for kwargs, code, message in (+ ({"TransitEncryptionEnabled": False}, "InvalidParameterValue",+ "The AUTH token is only supported when encryption-in-transit is enabled"),+ ({"TransitEncryptionEnabled": True, "TransitEncryptionMode": "preferred"}, "InvalidParameterCombination",+ "Transit encryption preferred is not supported with access control enabled clusters."),+ ):+ with pytest.raises(ClientError) as exc:+ ec.create_replication_group(ReplicationGroupId=rg_id, ReplicationGroupDescription="t", Engine="redis",+ NumCacheClusters=1, AuthToken="a-very-long-auth-token-123", **kwargs)+ assert exc.value.response["Error"]["Code"] == code+ assert exc.value.response["Error"]["Message"] == message++ plain = ec.create_replication_group(ReplicationGroupId=rg_id, ReplicationGroupDescription="t",+ Engine="redis", NumCacheClusters=1)["ReplicationGroup"]+ try:+ assert plain["TransitEncryptionEnabled"] is False and "TransitEncryptionMode" not in plain+ member = ec.describe_cache_clusters(CacheClusterId=plain["MemberClusters"][0])["CacheClusters"][0]+ assert (member["AuthTokenEnabled"], member["TransitEncryptionEnabled"], member["AtRestEncryptionEnabled"]) == (+ False, False, False)+ with pytest.raises(ClientError) as exc:+ ec.modify_replication_group(ReplicationGroupId=rg_id, AuthToken="a-very-long-auth-token-123",+ AuthTokenUpdateStrategy="ROTATE")+ assert exc.value.response["Error"]["Message"] == (+ "The AUTH token modification is only supported when encryption-in-transit is enabled")+ finally:+ ec.delete_replication_group(ReplicationGroupId=rg_id)+++def _redis_session(endpoint, ca_file, *commands):+ import socket+ import ssl+ ctx = ssl.create_default_context(cafile=ca_file)+ deadline = time.time() + 30+ while True:+ try:+ with socket.create_connection((endpoint["Address"], endpoint["Port"]), timeout=5) as raw:+ with ctx.wrap_socket(raw, server_hostname=endpoint["Address"]) as conn:+ replies = []+ for parts in commands:+ conn.sendall(f"*{len(parts)}\r\n".encode()+ + b"".join(f"${len(p)}\r\n{p}\r\n".encode() for p in parts))+ replies.append(conn.recv(65536))+ return replies+ except (OSError, ssl.SSLError):+ if time.time() > deadline:+ raise+ time.sleep(0.3)++[email protected]_plane+def test_replication_group_serves_tls_and_requires_auth_token(ec, tmp_path):+ rg_id = f"tls-{_uid()}"+ first, second = "first-auth-token-0123", "second-auth-token-4567"+ rg = ec.create_replication_group(ReplicationGroupId=rg_id, ReplicationGroupDescription="t", Engine="redis",+ NumCacheClusters=1, TransitEncryptionEnabled=True,+ AtRestEncryptionEnabled=True, AuthToken=first)["ReplicationGroup"]+ try:+ assert (rg["TransitEncryptionEnabled"], rg["TransitEncryptionMode"], rg["AuthTokenEnabled"]) == (+ True, "required", True)+ member = ec.describe_cache_clusters(CacheClusterId=rg["MemberClusters"][0])["CacheClusters"][0]+ assert (member["TransitEncryptionEnabled"], member["TransitEncryptionMode"],+ member["AuthTokenEnabled"], member["AtRestEncryptionEnabled"]) == (True, "required", True, True)++ endpoint = rg["NodeGroups"][0]["PrimaryEndpoint"]+ ca_file = _serverless_ca_file(tmp_path)+ assert _redis_session(endpoint, ca_file, ["PING"])[0].startswith(b"-NOAUTH")+ assert _redis_session(endpoint, ca_file, ["AUTH", first], ["PING"]) == [b"+OK\r\n", b"+PONG\r\n"]++ ec.modify_replication_group(ReplicationGroupId=rg_id, AuthToken=second,+ AuthTokenUpdateStrategy="ROTATE", ApplyImmediately=True)+ for token in (first, second):+ assert _redis_session(endpoint, ca_file, ["AUTH", token])[0] == b"+OK\r\n"+ ec.modify_replication_group(ReplicationGroupId=rg_id, AuthToken=second,+ AuthTokenUpdateStrategy="SET", ApplyImmediately=True)+ assert _redis_session(endpoint, ca_file, ["AUTH", first])[0].startswith(b"-WRONGPASS")+ assert _redis_session(endpoint, ca_file, ["AUTH", second])[0] == b"+OK\r\n"+ finally:+ ec.delete_replication_group(ReplicationGroupId=rg_id)diff --git a/tests/test_rds.py b/tests/test_rds.pyindex 2c6ad37b..80fcb23f 100644--- a/tests/test_rds.py+++ b/tests/test_rds.py@@ -4818,6 +4818,31 @@ def test_rds_create_read_replica(rds):rds.delete_db_instance(DBInstanceIdentifier="rr-source", SkipFinalSnapshot=True)+def test_rds_max_allocated_storage_only_with_storage_autoscaling(rds):+ """MaxAllocatedStorage is set only when storage autoscaling is on; a value equal to AllocatedStorage turns it off."""+ plain = f"rds-noauto-{_uuid_mod.uuid4().hex[:8]}"+ scaled = f"rds-auto-{_uuid_mod.uuid4().hex[:8]}"+ replica = f"rds-auto-rr-{_uuid_mod.uuid4().hex[:8]}"+ common = dict(DBInstanceClass="db.t3.micro", Engine="postgres", MasterUsername="admin",+ MasterUserPassword="secret123", AllocatedStorage=20)+ describe = lambda iid: rds.describe_db_instances(DBInstanceIdentifier=iid)["DBInstances"][0] # noqa: E731+ try:+ assert "MaxAllocatedStorage" not in rds.create_db_instance(DBInstanceIdentifier=plain, **common)["DBInstance"]+ assert "MaxAllocatedStorage" not in describe(plain)+ rds.create_db_instance(DBInstanceIdentifier=scaled, MaxAllocatedStorage=100, **common)+ assert describe(scaled)["MaxAllocatedStorage"] == 100+ rds.create_db_instance_read_replica(DBInstanceIdentifier=replica, SourceDBInstanceIdentifier=scaled)+ assert "MaxAllocatedStorage" not in describe(replica)+ rds.modify_db_instance(DBInstanceIdentifier=scaled, MaxAllocatedStorage=20, ApplyImmediately=True)+ assert "MaxAllocatedStorage" not in describe(scaled)+ finally:+ for iid in (replica, scaled, plain):+ try:+ rds.delete_db_instance(DBInstanceIdentifier=iid, SkipFinalSnapshot=True)+ except ClientError:+ pass++def test_rds_create_read_replica_source_not_found(rds):"""CreateDBInstanceReadReplica fails when the source instance does not exist."""with pytest.raises(ClientError) as exc:diff --git a/tests/test_s3.py b/tests/test_s3.pyindex caee477e..5b57f17c 100644--- a/tests/test_s3.py+++ b/tests/test_s3.py@@ -4603,6 +4603,9 @@ def test_s3_presign_verifies_in_both_auth_modes(monkeypatch, auth_enabled, bad_t(3600, 840, None, None, None),(3600, 960, None, 403, "Request is not yet valid"),(3600, 0, ("X-Amz-Expires", "abc"), 400, "X-Amz-Expires should be a number"),+ (3600, 0, ("X-Amz-Expires", "-1"), 400, "X-Amz-Expires must be non-negative"),+ (3600, 0, ("X-Amz-Expires", None), 400, "requires the X-Amz-Algorithm, X-Amz-Credential, X-Amz-Signature, "+ "X-Amz-Date, X-Amz-SignedHeaders, and X-Amz-Expires parameters."),(3600, 0, ("X-Amz-Date", "2026"), 400, "X-Amz-Date must be in the ISO8601 Long Format"),])def test_s3_presign_time_limits_under_auth(monkeypatch, auth_enabled, expires, skew, change, status, code):@@ -4627,7 +4630,9 @@ def test_s3_presign_time_limits_under_auth(monkeypatch, auth_enabled, expires, surl = urlsplit(client.generate_presigned_url("get_object", Params={"Bucket": "test-bucket", "Key": "object"}, ExpiresIn=expires))query = parse_qs(url.query)- if change:+ if change and change[1] is None:+ query.pop(change[0])+ elif change:query[change[0]] = [change[1]]error = s3_svc._verify_presigned_sigv4("GET", url.path, {"host": url.netloc}, query)@@ -4635,7 +4640,7 @@ def test_s3_presign_time_limits_under_auth(monkeypatch, auth_enabled, expires, sif auth_enabled and status:assert error[0] == status and code in error[2].decode()elif change:- assert b"SignatureDoesNotMatch" in error[2]+ assert error[0] == 403 # SignatureDoesNotMatch, or Request has expired for a negative X-Amz-Expireselse:assert error is None@@ -5837,6 +5842,46 @@ def test_s3_put_object_with_explicit_sha256_value_validated(s3):s3.delete_bucket(Bucket=bucket)+def test_s3_complete_multipart_upload_checks_mpu_object_size(s3):+ bucket = "mpu-object-size-bucket"+ s3.create_bucket(Bucket=bucket)+ body = b"x" * 10+ uid = s3.create_multipart_upload(Bucket=bucket, Key="obj")["UploadId"]+ etag = s3.upload_part(Bucket=bucket, Key="obj", UploadId=uid, PartNumber=1, Body=body)["ETag"]+ parts = {"Parts": [{"PartNumber": 1, "ETag": etag}]}++ for declared in (11, 9, 0):+ with pytest.raises(ClientError) as exc:+ s3.complete_multipart_upload(Bucket=bucket, Key="obj", UploadId=uid,+ MultipartUpload=parts, MpuObjectSize=declared)+ error = exc.value.response+ assert error["ResponseMetadata"]["HTTPStatusCode"] == 400+ assert error["Error"]["Code"] == "InvalidRequest"+ assert error["Error"]["Message"] == (+ f"The provided 'x-amz-mp-object-size' header value {declared} does not match what was computed: 10")++ def _invalid_size(request, **_):+ request.headers["x-amz-mp-object-size"] = "abc"++ s3.meta.events.register_first("before-sign.s3.CompleteMultipartUpload", _invalid_size)+ try:+ with pytest.raises(ClientError) as exc:+ s3.complete_multipart_upload(Bucket=bucket, Key="obj", UploadId=uid, MultipartUpload=parts)+ finally:+ s3.meta.events.unregister("before-sign.s3.CompleteMultipartUpload", _invalid_size)+ assert exc.value.response["Error"]["Code"] == "InvalidRequest"+ assert exc.value.response["Error"]["Message"] == "Value for x-amz-mp-object-size header is invalid: 'abc'"++ with pytest.raises(ClientError) as exc:+ s3.head_object(Bucket=bucket, Key="obj")+ assert exc.value.response["ResponseMetadata"]["HTTPStatusCode"] == 404+ assert [(p["PartNumber"], p["Size"]) for p in s3.list_parts(Bucket=bucket, Key="obj", UploadId=uid)["Parts"]] == [+ (1, 10)]++ s3.complete_multipart_upload(Bucket=bucket, Key="obj", UploadId=uid, MultipartUpload=parts, MpuObjectSize=10)+ assert s3.head_object(Bucket=bucket, Key="obj")["ContentLength"] == 10++def test_s3_multipart_checksum_sha256_composite(s3):"""A checksummed multipart upload carries its parts' checksums through toa composite on the completed object: the part echoes its own, thediff --git a/tests/test_ssm.py b/tests/test_ssm.pyindex 40c6ecd3..7ee025f3 100644--- a/tests/test_ssm.py+++ b/tests/test_ssm.py@@ -1260,7 +1260,7 @@ def test_ssm_default_parameter_tier_setting(service_settings):tiers = set(_ssm_model().shape_for("ParameterTier").enum)big = "x" * 4097name = f"/tier/{_uuid_mod.uuid4().hex[:8]}"- cases = {"Standard": ("Standard", "Standard"), "Advanced": ("Advanced", "Advanced"),+ cases = {"Standard": ("Standard", None), "Advanced": ("Advanced", "Advanced"),"Intelligent-Tiering": ("Standard", "Advanced")}try:for default_tier, (small_tier, big_tier) in cases.items():@@ -1268,12 +1268,15 @@ def test_ssm_default_parameter_tier_setting(service_settings):put = ssm.put_parameter(Name=name, Value="v", Type="String", Overwrite=True)listed = ssm.describe_parameters(ParameterFilters=[{"Key": "Name", "Values": [name]}])["Parameters"][0]assert (put["Tier"], listed["Tier"]) == (small_tier, small_tier)+ assert {put["Tier"], listed["Tier"]} <= tiersssm.delete_parameter(Name=name)- assert ssm.put_parameter(Name=name, Value=big, Type="String")["Tier"] == big_tier- ssm.delete_parameter(Name=name)- assert ssm.put_parameter(Name=name, Value=big, Type="String", Tier="Standard")["Tier"] == "Standard"- ssm.delete_parameter(Name=name)- assert {put["Tier"], listed["Tier"], big_tier} <= tiers+ if big_tier:+ assert ssm.put_parameter(Name=name, Value=big, Type="String")["Tier"] == big_tier+ ssm.delete_parameter(Name=name)+ # An Advanced parameter stays Advanced when overwritten without a Tier.+ ssm.put_parameter(Name=name, Value="v", Type="String", Tier="Advanced")+ ssm.update_service_setting(SettingId=_TIER, SettingValue="Standard")+ assert ssm.put_parameter(Name=name, Value="w", Type="String", Overwrite=True)["Tier"] == "Advanced"finally:ssm.delete_parameters(Names=[name])