Ministack: Free, open-source local AWS emulator - 60+ services, Terraform compatible, real databases. Free forever. MIT licensed.
Python awsaws-emulatoraws-localaws-sdkdevtoolsdockerdynamodbec2emulatorlambdalocalstacklocalstack-alternativeministackmock-awsopen-sourcepythons3sqsterraform
fix(eventbridge): resolve dynamic API destination HTTP parameters (#1775)
Dan Garlandb0fb3d16a6c8
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 9a43bbf3..2421f893 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -8,6 +8,8 @@ Versioning follows [Semantic Versioning](https://semver.org/).## [Unreleased]### Fixed+- **EventBridge — dynamic HTTP parameters for API destinations** — `HttpParameters` JSON paths now resolve against the original event before input transformation, including array indexes and wildcards. Literal values, connection precedence and stored target configuration are preserved. Verified against AWS EventBridge using a MongoDB partner event bus and an HTTPS request recorder.+- **RDS — StopDBCluster/StartDBCluster no longer rewrites the cluster endpoint from its DNS name to a raw container address** — restarting a cluster's shared container reported the container's IP in place of the `*.rds.amazonaws.com` name handed out at creation, even though the name still resolved to the restarted container. Connection strings kept working by accident or broke outright once the address was reused, and hostname-verified TLS could not match the address. The restart now reports the same registered network alias the first launch does, while the internal address and readiness probe keep using the refreshed container IP. The reader endpoint, in turn, only derives its `cluster-ro-` name when that name was actually registered on the backing container.- **IAM — three calls are authorized against the action and resource AWS uses** — a Lambda Function URL invoke is checked against its function ARN, alias qualifier included, and supplies `lambda:FunctionUrlAuthType`, so the policy `grantInvokeUrl` writes matches. The WebSocket `@connections` API asks for `execute-api:ManageConnections` instead of `execute-api:Invoke`. `iot-jobs-data` operations are authorized under `iotjobsdata:`, except `StartCommandExecution`, which stays on `iot:`. A policy written for the old action names stops matching, as on AWS. Contributed by @iot-rocket.diff --git a/README.md b/README.mdindex 701a9480..a0210860 100644--- a/README.md+++ b/README.md@@ -415,7 +415,7 @@ subnet = ec2.create_subnet(| Service | Operations | Notes ||---------|-----------|-------|| **SSM Parameter Store** | PutParameter, GetParameter, GetParameters, GetParametersByPath, DeleteParameter, DeleteParameters, DescribeParameters, GetParameterHistory, LabelParameterVersion, AddTagsToResource, RemoveTagsFromResource, ListTagsForResource | Supports String, SecureString, StringList |-| **EventBridge** | CreateEventBus, UpdateEventBus, DeleteEventBus, ListEventBuses, DescribeEventBus, PutRule, DeleteRule, ListRules, DescribeRule, EnableRule, DisableRule, PutTargets, RemoveTargets, ListTargetsByRule, ListRuleNamesByTarget, PutEvents, TestEventPattern, TagResource, UntagResource, ListTagsForResource, CreateArchive, DeleteArchive, DescribeArchive, UpdateArchive, ListArchives, PutPermission, RemovePermission, CreateConnection, DescribeConnection, DeleteConnection, UpdateConnection, DeauthorizeConnection, ListConnections, CreateApiDestination, DescribeApiDestination, DeleteApiDestination, UpdateApiDestination, ListApiDestinations, StartReplay, DescribeReplay, ListReplays, CancelReplay, CreateEndpoint, DeleteEndpoint, DescribeEndpoint, ListEndpoints, UpdateEndpoint, ActivateEventSource, DeactivateEventSource, DescribeEventSource, CreatePartnerEventSource, DeletePartnerEventSource, DescribePartnerEventSource, ListPartnerEventSources, ListPartnerEventSourceAccounts, ListEventSources, PutPartnerEvents | Lambda target dispatch on PutEvents; S3 EventBridge notifications; archives capture matching events and `StartReplay` re-dispatches them to the destination bus in a background thread; API destination targets are **invoked over HTTP** on PutEvents — connection auth (Basic, API Key, OAuth client-credentials with token caching and 401/407 refresh), connection/target header/query/body parameter merging with connection precedence, `*` path parameters, and the AWS default headers (`User-Agent: Amazon/EventBridge/ApiDestinations`, default `Content-Type: application/json; charset=utf-8`), which connection and target parameters cannot override; redirects are not followed (a `3xx` is a non-retryable delivery failure), only `http(s)` invocation and OAuth authorization endpoints are accepted, `HttpMethod` and `AuthorizationType` are validated against their enums, and a connection that is not `AUTHORIZED` is not invoked (`DeauthorizeConnection` clears its stored credentials) — the 24h retry pipeline, rate limiting, and DLQs are not modeled; SaaS/partner APIs are otherwise control-plane stubs. Event patterns are **validated** — `PutRule`, `TestEventPattern`, `CreateArchive` and `UpdateArchive` reject a malformed pattern with `InvalidEventPatternException` carrying AWS's own reason text — and evaluate `prefix`/`suffix` (plain or case-insensitive via a nested `equals-ignore-case`), `wildcard` (AWS semantics — only `*` is special, `\` escapes — with catastrophic backtracking ruled out), `equals-ignore-case` (AWS's per-character case alternation), `exactly`, `cidr` (IPv4/IPv6, host bits floored, `/32` and `/128` refused as AWS refuses them), `numeric` (as the grammar it is: one comparison or a lower-then-upper bound), `exists` (leaf nodes only, so an object or an empty array does not exist), `anything-but` (a literal, a list, or a nested `prefix`/`suffix`/`wildcard`/`equals-ignore-case`, singly or as a list), and `$or` at any field position — expanded into flat alternatives at rule creation, so AWS's per-leaf last-write-wins collision rule and its 1000-combination cap both hold. All ten envelope fields are filterable, an unmodelled key is a path the event lacks rather than an error, `resources` is an intersection, array values are matched per element (with AWS's array-consistency rule), and values compare as JSON so a string never equals a number. An operand or key the matcher cannot evaluate never matches, rather than raising or matching everything. Not modeled: the 4096-character pattern limit, the wildcard-complexity refusal, and the 256-leaf rule-size cap |+| **EventBridge** | CreateEventBus, UpdateEventBus, DeleteEventBus, ListEventBuses, DescribeEventBus, PutRule, DeleteRule, ListRules, DescribeRule, EnableRule, DisableRule, PutTargets, RemoveTargets, ListTargetsByRule, ListRuleNamesByTarget, PutEvents, TestEventPattern, TagResource, UntagResource, ListTagsForResource, CreateArchive, DeleteArchive, DescribeArchive, UpdateArchive, ListArchives, PutPermission, RemovePermission, CreateConnection, DescribeConnection, DeleteConnection, UpdateConnection, DeauthorizeConnection, ListConnections, CreateApiDestination, DescribeApiDestination, DeleteApiDestination, UpdateApiDestination, ListApiDestinations, StartReplay, DescribeReplay, ListReplays, CancelReplay, CreateEndpoint, DeleteEndpoint, DescribeEndpoint, ListEndpoints, UpdateEndpoint, ActivateEventSource, DeactivateEventSource, DescribeEventSource, CreatePartnerEventSource, DeletePartnerEventSource, DescribePartnerEventSource, ListPartnerEventSources, ListPartnerEventSourceAccounts, ListEventSources, PutPartnerEvents | Lambda target dispatch on PutEvents; S3 EventBridge notifications; archives capture matching events and `StartReplay` re-dispatches them to the destination bus in a background thread; API destination targets are **invoked over HTTP** on PutEvents — connection auth (Basic, API Key, OAuth client-credentials with token caching and 401/407 refresh), connection/target header/query/body parameter merging with connection precedence, `*` path parameters, dynamic JSON-path values for target HTTP paths/headers/query strings resolved from the original event before body transformation, and the AWS default headers (`User-Agent: Amazon/EventBridge/ApiDestinations`, default `Content-Type: application/json; charset=utf-8`), which connection and target parameters cannot override; redirects are not followed (a `3xx` is a non-retryable delivery failure), only `http(s)` invocation and OAuth authorization endpoints are accepted, `HttpMethod` and `AuthorizationType` are validated against their enums, and a connection that is not `AUTHORIZED` is not invoked (`DeauthorizeConnection` clears its stored credentials) — the 24h retry pipeline, rate limiting, and DLQs are not modeled; SaaS/partner APIs are otherwise control-plane stubs. Event patterns are **validated** — `PutRule`, `TestEventPattern`, `CreateArchive` and `UpdateArchive` reject a malformed pattern with `InvalidEventPatternException` carrying AWS's own reason text — and evaluate `prefix`/`suffix` (plain or case-insensitive via a nested `equals-ignore-case`), `wildcard` (AWS semantics — only `*` is special, `\` escapes — with catastrophic backtracking ruled out), `equals-ignore-case` (AWS's per-character case alternation), `exactly`, `cidr` (IPv4/IPv6, host bits floored, `/32` and `/128` refused as AWS refuses them), `numeric` (as the grammar it is: one comparison or a lower-then-upper bound), `exists` (leaf nodes only, so an object or an empty array does not exist), `anything-but` (a literal, a list, or a nested `prefix`/`suffix`/`wildcard`/`equals-ignore-case`, singly or as a list), and `$or` at any field position — expanded into flat alternatives at rule creation, so AWS's per-leaf last-write-wins collision rule and its 1000-combination cap both hold. All ten envelope fields are filterable, an unmodelled key is a path the event lacks rather than an error, `resources` is an intersection, array values are matched per element (with AWS's array-consistency rule), and values compare as JSON so a string never equals a number. An operand or key the matcher cannot evaluate never matches, rather than raising or matching everything. Not modeled: the 4096-character pattern limit, the wildcard-complexity refusal, and the 256-leaf rule-size cap || **Kinesis** | CreateStream, DeleteStream, DescribeStream, DescribeStreamSummary, ListStreams, ListShards, PutRecord, PutRecords, GetShardIterator, GetRecords, IncreaseStreamRetentionPeriod, DecreaseStreamRetentionPeriod, MergeShards, SplitShard, UpdateShardCount, StartStreamEncryption, StopStreamEncryption, EnableEnhancedMonitoring, DisableEnhancedMonitoring, RegisterStreamConsumer, DeregisterStreamConsumer, ListStreamConsumers, DescribeStreamConsumer, AddTagsToStream, RemoveTagsFromStream, ListTagsForStream | Partition key → shard routing; AWS limits enforced (1 MB/record, 500 records/batch, 5 MB payload, 256-char partition key) || **CloudWatch Metrics** | PutMetricData, GetMetricStatistics, GetMetricData, ListMetrics, PutMetricAlarm, PutCompositeAlarm, DescribeAlarms, DescribeAlarmsForMetric, DescribeAlarmHistory, DeleteAlarms, SetAlarmState, EnableAlarmActions, DisableAlarmActions, TagResource, UntagResource, ListTagsForResource, PutDashboard, GetDashboard, DeleteDashboards, ListDashboards | CBOR and JSON protocol || **SES** | SendEmail, SendRawEmail, SendTemplatedEmail, SendBulkTemplatedEmail, VerifyEmailIdentity, VerifyEmailAddress, VerifyDomainIdentity, VerifyDomainDkim, ListIdentities, ListVerifiedEmailAddresses, GetIdentityVerificationAttributes, GetIdentityDkimAttributes, DeleteIdentity, GetSendQuota, GetSendStatistics, SetIdentityNotificationTopic, SetIdentityFeedbackForwardingEnabled, CreateConfigurationSet, DeleteConfigurationSet, DescribeConfigurationSet, ListConfigurationSets, CreateTemplate, GetTemplate, UpdateTemplate, DeleteTemplate, ListTemplates | Emails stored in-memory, not sent |diff --git a/ministack/services/eventbridge.py b/ministack/services/eventbridge.pyindex e10cb324..d0cba865 100644--- a/ministack/services/eventbridge.py+++ b/ministack/services/eventbridge.py@@ -2258,7 +2258,10 @@ def _invoke_target(target, event, rule, view=None):returnif spec.service == "events" and spec.resource.startswith("api-destination/"):- _dispatch_to_api_destination(spec, event_payload, target)+ http_parameters = _resolve_http_parameters(+ target.get("HttpParameters") or {}, view.event_with_detail()+ )+ _dispatch_to_api_destination(spec, event_payload, {**target, "HttpParameters": http_parameters})elif spec.service == "events":_dispatch_to_event_bus(spec, event, rule, event_path, target_input_payload)elif spec.service == "states":@@ -2605,6 +2608,58 @@ def _connection_params_map(params) -> dict:return out+_HTTP_JSON_PATH = re.compile(r"\$(?:\.[A-Za-z0-9_/*-]+|\[(?:[0-9]+|\*)\])*")+_HTTP_JSON_PATH_TOKEN = re.compile(r"\.([A-Za-z0-9_/*-]+)|\[([0-9]+|\*)\]")+++def _resolve_http_parameter(value, event):+ """Resolve a whole JSON path; literal strings are not interpolated.++ AWS uses the original event, renders missing matches as an empty string,+ and strips JSON quotes from structured values in HTTP parameters.+ """+ if not _HTTP_JSON_PATH.fullmatch(value):+ return value+ nodes = [event]+ multiple = False+ for field, index in _HTTP_JSON_PATH_TOKEN.findall(value):+ selected = []+ wildcard = field == "*" or index == "*"+ multiple = multiple or wildcard+ for node in nodes:+ if wildcard:+ if isinstance(node, dict):+ selected.extend(node.values())+ elif isinstance(node, list):+ selected.extend(node)+ elif field and isinstance(node, dict) and field in node:+ selected.append(node[field])+ elif index and isinstance(node, list) and int(index) < len(node):+ selected.append(node[int(index)])+ nodes = selected+ if not nodes:+ return ""+ result = nodes if multiple else nodes[0]+ if isinstance(result, str):+ return result+ return json.dumps(result, separators=(",", ":"), ensure_ascii=False).replace('"', "")+++def _resolve_http_parameters(parameters, event):+ # Construct fresh containers: targets are reused by subsequent events and+ # concurrent deliveries. Header/query names are always literal.+ result = dict(parameters)+ for field in ("HeaderParameters", "QueryStringParameters"):+ if field in parameters:+ result[field] = {key: _resolve_http_parameter(value, event)+ for key, value in parameters[field].items()}+ if "PathParameterValues" in parameters:+ result["PathParameterValues"] = [+ _resolve_http_parameter(value, event) for value in parameters["PathParameterValues"]+ ]+ return result++def _apply_path_parameters(url: str, values) -> str:"""Populate ``*`` path wildcards in the invocation endpoint from thetarget's PathParameterValues, in order — the same contract as realdiff --git a/tests/test_eventbridge.py b/tests/test_eventbridge.pyindex d011596d..6c9f2946 100644--- a/tests/test_eventbridge.py+++ b/tests/test_eventbridge.py@@ -5630,3 +5630,94 @@ def test_eventbridge_dotted_key_rule_delivers_to_sqs(eb, sqs):assert len(msgs.get("Messages", [])) == 1body = json.loads(msgs["Messages"][0]["Body"])assert body["detail"]["name"] == "dotted.ping"++[email protected]("input_mode", ["Input", "InputPath", "InputTransformer"])[email protected]("topic_path", ["topicKey", "topicKeys[0]"])+def test_eventbridge_api_destination_dynamic_http_parameters_use_original_event(eb, input_mode, topic_path):+ """AWS resolves HttpParameters before body transformation (verified on a real bus)."""+ from urllib.parse import parse_qs, unquote, urlsplit++ server, captured = _start_api_dest_capture_server()+ try:+ prefix = "$.detail.fullDocument.payload"+ http_parameters = {+ "PathParameterValues": [prefix + "." + topic_path],+ "HeaderParameters": {+ "X-Subscriber": prefix + ".subscriberId",+ "X-Missing": prefix + ".missing",+ "X-Precedence": prefix + ".subscriberId",+ "X-Literal": "literal-$.detail.value",+ "X-Count": prefix + ".count",+ "X-Active": prefix + ".active",+ "X-Null": prefix + ".nil",+ "X-Object": prefix + ".object",+ "X-Topics": prefix + ".topicKeys[*]",+ },+ "QueryStringParameters": {+ "subscriber": prefix + ".subscriberId",+ "missing": prefix + ".missing",+ "precedence": prefix + ".subscriberId",+ "literal": "fixed",+ "topics": prefix + ".topicKeys[*]",+ },+ }+ body_options = {+ "Input": {"Input": '{"constant":true}'},+ "InputPath": {"InputPath": prefix + ".body"},+ "InputTransformer": {"InputTransformer": {+ "InputPathsMap": {"subscriber": prefix + ".subscriberId"},+ "InputTemplate": '{"subscriptions":[{"subscriberId":<subscriber>}]}'+ }},+ }+ slug = "dynamic-" + input_mode.lower() + ("-index" if "[" in topic_path else "-field")+ bus, source = _api_dest_pipeline(+ eb, slug,+ f"http://127.0.0.1:{server.server_address[1]}/v2/topics/*/subscriptions",+ "API_KEY", {+ "ApiKeyAuthParameters": {"ApiKeyName": "X-Key", "ApiKeyValue": "test"},+ "InvocationHttpParameters": {+ "HeaderParameters": [{"Key": "X-Precedence", "Value": "connection"}],+ "QueryStringParameters": [{"Key": "precedence", "Value": "connection"}],+ },+ },+ target_extras={"HttpParameters": http_parameters, **body_options[input_mode]},+ )+ # Reuse the same target for different events: resolution must not mutate it.+ for i in range(2):+ subscriber = f"alice-{i}"+ topic = f"venue:{i}"+ payload = {+ "subscriberId": subscriber, "topicKey": topic, "topicKeys": [topic, "artist:2"],+ "count": 42, "active": False, "nil": None, "object": {"key": "value"},+ "body": {"selected": True},+ }+ assert eb.put_events(Entries=[{+ "Source": source, "DetailType": "MongoDB Database Trigger",+ "Detail": json.dumps({"fullDocument": {"payload": payload}}), "EventBusName": bus,+ }])["FailedEntryCount"] == 0+ assert _wait_until(lambda: len(captured) > i)+ request = captured[i]+ url = urlsplit(request["path"])+ assert unquote(url.path) == f"/v2/topics/{topic}/subscriptions"+ assert parse_qs(url.query, keep_blank_values=True) == {+ "subscriber": [subscriber], "missing": [""], "precedence": ["connection"],+ "literal": ["fixed"], "topics": [f"[{topic},artist:2]"],+ }+ expected_headers = {+ "x-subscriber": subscriber, "x-missing": "", "x-precedence": "connection",+ "x-literal": "literal-$.detail.value", "x-count": "42", "x-active": "false",+ "x-null": "null", "x-object": "{key:value}", "x-topics": f"[{topic},artist:2]",+ }+ for name, value in expected_headers.items():+ assert request["headers"][name] == value+ expected_body = {+ "Input": {"constant": True}, "InputPath": {"selected": True},+ "InputTransformer": {"subscriptions": [{"subscriberId": subscriber}]},+ }[input_mode]+ assert json.loads(request["body"]) == expected_body+ assert eb.list_targets_by_rule(+ Rule="qa-eb-apidest-" + slug + "-rule", EventBusName=bus,+ )["Targets"][0]["HttpParameters"] == http_parameters+ finally:+ server.shutdown()