Light, fluffy, and always free - The AWS Local Emulator alternative
feat(cloudformation): CDK Provider-framework custom resources and changeset/SSM param fidelity (#2688)
Sam Fakhreddine08e532ab1af8
diff --git a/docs/services/cloudformation.md b/docs/services/cloudformation.mdindex 5b507bbeb0..e5301fd6c6 100644--- a/docs/services/cloudformation.md+++ b/docs/services/cloudformation.md@@ -8,13 +8,13 @@<!-- floci:actions:start -->| Action | Description || --- | --- |-| `DescribeStacks` | Get stack status and outputs |+| `DescribeStacks` | Get stack status, parameters, and outputs || `CreateStack` | Deploy a CloudFormation template || `UpdateStack` | Update an existing stack || `DeleteStack` | Delete a stack and its resources || `UpdateTerminationProtection` | - || `CreateChangeSet` | Create a change set |-| `DescribeChangeSet` | Get change set details (no computed diff/preview) |+| `DescribeChangeSet` | Get change set details with a computed Add/Modify/Remove resource diff || `ExecuteChangeSet` | Apply a change set || `DeleteChangeSet` | Delete a change set || `ListChangeSets` | List change sets for a stack |@@ -194,6 +194,51 @@ Resources provisioned by `CreateStack` / `UpdateStack` land in the **caller's ac(determined from the request's access key — see [Multi-Account Isolation](../configuration/multi-account.md)).Deleting the stack removes them from that same account.+Stacks, change sets, and **exports are scoped to the caller account**. `ListExports` returns only the+exports created by that account's stacks, and `Fn::ImportValue` resolves against the same account's+export table — so two accounts can each own an export with the same name without colliding, mirroring+how CloudFormation isolates exports per account and region. This is what lets a multi-account LZA+deployment reuse identical export names across its member accounts.++## Template Parameters++Parameters passed to `CreateStack` / `UpdateStack` (and echoed back by `DescribeStacks`) are resolved+before provisioning:++- **`String`, `Number`, `CommaDelimitedList`, and `List<...>`** parameter types are accepted and+ substituted into resource properties via `Ref`.+- **`AWS::SSM::Parameter::Value<String>`-typed parameters** are resolved at deploy time: the value the+ caller supplies is treated as an SSM parameter *name*, and floci substitutes the current value of+ that SSM parameter into the template. This is the pattern CDK uses to pull bootstrap values (image+ tags, bucket names) from Parameter Store.+- `AWS::SSM::Parameter::Value<List<String>>` and `AWS::SSM::Parameter::Name` typed parameters are+ **not yet** resolved — they are passed through as their literal input.++The `AWS::SSM::Parameter` **resource** type exposes `Value`, `Type`, and `Name` attributes through+`Ref` / `Fn::GetAtt` so downstream resources can consume a parameter the same stack creates.++## Conditions++Template `Conditions` are evaluated before provisioning. A resource whose `Condition` evaluates to+**false is skipped** — it is never provisioned, does not appear in `DescribeStackResources`, and its+`Ref` / `Fn::GetAtt` are not required to resolve. This matches CloudFormation and is required for the+condition-heavy templates CDK and LZA emit.++## Custom Resources and the CDK Provider Framework++`AWS::CloudFormation::CustomResource` and `Custom::*` resources are backed by their `ServiceToken`+Lambda. floci supports two shapes:++- **Direct custom resources** — floci invokes the handler Lambda with the CloudFormation custom-resource+ event and waits for it to `PUT` its `SUCCESS`/`FAILED` result to the response URL. Inline `ZipFile`+ handlers get the `cfn-response` / `cfnresponse` module bundled in (see the Lambda row in+ [Supported Resource Types](#supported-resource-types)), so Solutions-style handlers work unmodified.+- **CDK Provider framework** — when the `ServiceToken` points at a CDK `framework.onEvent` function,+ floci drives the asynchronous provider protocol: `onEvent` starts the work and `framework.isComplete`+ is polled (via Step Functions [`Retry`](step-functions.md)) until it reports done, at which point the+ ResponseURL callback fires. The wait is bounded by an async custom-resource timeout (3 minutes by+ default); a resource that never completes fails the stack rather than hanging.+## Deletion PoliciesA resource's [`DeletionPolicy`](https://docs.aws.amazon.com/AWSCloudFormation/latest/TemplateReference/aws-attribute-deletionpolicy.html)diff --git a/docs/services/step-functions.md b/docs/services/step-functions.mdindex 2935502d31..0f5a366c5c 100644--- a/docs/services/step-functions.md+++ b/docs/services/step-functions.md@@ -101,6 +101,11 @@ and each retrier keeps its own attempt counter. `Retry` is evaluated before `Cat`$$.State.RetryCount` increments per attempt. Attempt counts, defaults, and backofftiming were verified against real AWS Step Functions.+A state with no `Retry` array runs exactly once. This is what makes CDK-generated provider-framework+workflows converge: the `framework-isComplete-task` throws on every not-yet-complete poll and relies on+`Retry` to poll again until the custom resource reports done — see+[CloudFormation custom resources](cloudformation.md#custom-resources-and-the-cdk-provider-framework).+`JitterStrategy` supports `NONE` (the default) and `FULL`. `FULL` draws the delayuniformly between zero and the computed delay, as on AWS. One deviation. The delaybetween attempts is capped at 30 seconds, the same cap Floci applies to `Wait` states,diff --git a/src/main/java/io/github/hectorvent/floci/core/common/CustomResourceLiveness.java b/src/main/java/io/github/hectorvent/floci/core/common/CustomResourceLiveness.javanew file mode 100644index 0000000000..38539c4350--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/core/common/CustomResourceLiveness.java@@ -0,0 +1,74 @@+package io.github.hectorvent.floci.core.common;++import java.nio.charset.StandardCharsets;+import java.util.Optional;++/**+ * Sink for "this custom resource is still making progress" signals.+ *+ * <p>Lives here rather than in the cloudformation package so the Lambda service can report liveness+ * without depending on CloudFormation: the dependency already runs the other way (the provisioner+ * invokes Lambdas), and pointing it back would make the two packages mutually dependent.+ *+ * <p>The signal is observable generically because the provisioner embeds its correlation token in+ * the {@code ResponseURL} it puts on every event, and the CDK provider framework echoes that same+ * event into each {@code framework.isComplete} poll. So a poll arriving at the Lambda service is+ * proof of progress on a specific pending resource, with nothing here needing to know what the+ * resource does.+ */+public interface CustomResourceLiveness {++ /** Path segment the provisioner uses for its callback URL; the token follows it. */+ String RESPONSE_PATH = "/cfn-response/";++ /** Records progress for {@code token}. Unknown tokens are ignored. */+ void touch(String token);++ /**+ * Extracts the callback token from an invoke payload, if it carries one.+ *+ * @return the token, or empty for the overwhelming majority of invokes, which are ordinary+ * Lambda calls with no custom-resource event in them+ */+ byte[] RESPONSE_PATH_ASCII = RESPONSE_PATH.getBytes(StandardCharsets.US_ASCII);++ static Optional<String> tokenIn(byte[] payload) {+ if (payload == null || payload.length == 0) {+ return Optional.empty();+ }+ // Scans the raw bytes for the ASCII marker instead of materializing the whole payload as a+ // String: this runs on every Lambda invoke once liveness is wired in, and the overwhelming+ // majority of invokes carry no callback URL at all, so the common case should cost nothing+ // beyond the scan itself.+ int start = indexOfMarker(payload);+ if (start < 0) {+ return Optional.empty();+ }+ start += RESPONSE_PATH_ASCII.length;+ int end = start;+ // The token is a UUID; stop at whatever JSON punctuation closes the URL.+ while (end < payload.length && isTokenByte(payload[end])) {+ end++;+ }+ return end > start ? Optional.of(new String(payload, start, end - start, StandardCharsets.US_ASCII))+ : Optional.empty();+ }++ private static int indexOfMarker(byte[] payload) {+ byte[] marker = RESPONSE_PATH_ASCII;+ outer:+ for (int i = 0; i <= payload.length - marker.length; i++) {+ for (int j = 0; j < marker.length; j++) {+ if (payload[i + j] != marker[j]) {+ continue outer;+ }+ }+ return i;+ }+ return -1;+ }++ private static boolean isTokenByte(byte b) {+ return (b >= '0' && b <= '9') || (b >= 'a' && b <= 'z') || (b >= 'A' && b <= 'Z') || b == '-';+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.javaindex ee51bff2a0..9a22f1dbd4 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationQueryHandler.java@@ -141,7 +141,8 @@ public class CloudFormationQueryHandler {String stackName = params.getFirst("StackName");String templateBody = params.getFirst("TemplateBody");String templateUrl = params.getFirst("TemplateURL");- Map<String, String> parameters = extractParameters(params);+ Map<String, String> parameters =+ extractParameters(params, cfnService.currentParameters(stackName, region));List<String> capabilities = extractList(params, "Capabilities.member.");ChangeSet cs = cfnService.createChangeSet(stackName, "update-" + UUID.randomUUID().toString().substring(0, 8),@@ -206,7 +207,8 @@ public class CloudFormationQueryHandler {String changeSetType = params.getFirst("ChangeSetType");String templateBody = params.getFirst("TemplateBody");String templateUrl = params.getFirst("TemplateURL");- Map<String, String> parameters = extractParameters(params);+ Map<String, String> parameters =+ extractParameters(params, cfnService.currentParameters(stackName, region));List<String> capabilities = extractList(params, "Capabilities.member.");Map<String, String> tags = extractTags(params);@@ -234,7 +236,8 @@ public class CloudFormationQueryHandler {String changeSetName = params.getFirst("ChangeSetName");try {ChangeSet cs = cfnService.describeChangeSet(stackName, changeSetName, region);- String xml = new XmlBuilder()+ List<CloudFormationService.ResourceChange> changes = cfnService.computeChangeSetChanges(cs, region);+ XmlBuilder xml = new XmlBuilder().start("DescribeChangeSetResponse", CF_NS).start("DescribeChangeSetResult").elem("ChangeSetId", cs.getChangeSetId())@@ -243,13 +246,26 @@ public class CloudFormationQueryHandler {.elem("StackName", cs.getStackName()).elem("Status", cs.getStatus()).elem("ExecutionStatus", cs.getExecutionStatus())- .raw("<Changes/>")- .elem("CreationTime", ISO.format(cs.getCreationTime()))- .end("DescribeChangeSetResult")- .raw(AwsQueryResponse.responseMetadata())- .end("DescribeChangeSetResponse")- .build();- return Response.ok(xml).type("text/xml").build();+ .start("Changes");+ for (CloudFormationService.ResourceChange change : changes) {+ xml.start("member")+ .elem("Type", "Resource")+ .start("ResourceChange")+ .elem("Action", change.action())+ .elem("LogicalResourceId", change.logicalResourceId())+ .elem("PhysicalResourceId", change.physicalResourceId())+ .elem("ResourceType", change.resourceType())+ .elem("Replacement", change.replacement())+ .raw("<Scope/><Details/>")+ .end("ResourceChange")+ .end("member");+ }+ xml.end("Changes")+ .elem("CreationTime", ISO.format(cs.getCreationTime()))+ .end("DescribeChangeSetResult")+ .raw(AwsQueryResponse.responseMetadata())+ .end("DescribeChangeSetResponse");+ return Response.ok(xml.build()).type("text/xml").build();} catch (AwsException e) {return xmlError(e.getErrorCode(), e.getMessage(), e.getHttpStatus());}@@ -581,6 +597,13 @@ public class CloudFormationQueryHandler {xml.elem("member", cap);}xml.end("Capabilities");+ xml.start("Parameters");+ s.getParameters().forEach((k, v) ->+ xml.start("member")+ .elem("ParameterKey", k)+ .elem("ParameterValue", v)+ .end("member"));+ xml.end("Parameters");xml.start("Outputs");s.getOutputs().forEach((k, v) -> {xml.start("member")@@ -626,14 +649,28 @@ public class CloudFormationQueryHandler {}private Map<String, String> extractParameters(MultivaluedMap<String, String> params) {+ return extractParameters(params, Map.of());+ }++ /**+ * @param previousParameters the stack's currently deployed parameters, consulted when a member+ * sets {@code UsePreviousValue=true} instead of a {@code ParameterValue}+ */+ private Map<String, String> extractParameters(+ MultivaluedMap<String, String> params, Map<String, String> previousParameters) {Map<String, String> result = new HashMap<>();int i = 1;while (true) {String key = params.getFirst("Parameters.member." + i + ".ParameterKey");- String value = params.getFirst("Parameters.member." + i + ".ParameterValue");if (key == null) {break;}+ String value;+ if (Boolean.parseBoolean(params.getFirst("Parameters.member." + i + ".UsePreviousValue"))) {+ value = previousParameters.get(key);+ } else {+ value = params.getFirst("Parameters.member." + i + ".ParameterValue");+ }result.put(key, value != null ? value : "");i++;}@@ -747,10 +784,12 @@ public class CloudFormationQueryHandler {private Response updateStackSet(MultivaluedMap<String, String> params) {try {+ String stackSetName = params.getFirst("StackSetName");+ Map<String, String> previousParameters = stackSetService.describeStackSet(stackSetName).getParameters();StackSetOperation op = stackSetService.updateStackSet(- params.getFirst("StackSetName"),+ stackSetName,cfnService.resolveTemplateBody(params.getFirst("TemplateBody"), params.getFirst("TemplateURL")),- extractParameters(params),+ extractParameters(params, previousParameters),extractList(params, "Capabilities.member."),extractTags(params),params.getFirst("Description"));diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationResourceProvisioner.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationResourceProvisioner.javaindex 5b43c33b82..5767ce0755 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationResourceProvisioner.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationResourceProvisioner.java@@ -3157,6 +3157,9 @@ public class CloudFormationResourceProvisioner {}ssmService.putParameter(name, value, type, null, true, region);r.setPhysicalId(name);+ r.getAttributes().put("Name", name);+ r.getAttributes().put("Type", type);+ r.getAttributes().put("Value", value);}// ── KMS ───────────────────────────────────────────────────────────────────@@ -6404,13 +6407,13 @@ public class CloudFormationResourceProvisioner {"Custom resource handler errored (" + result.getFunctionError() + "): " + body, 400);}- return customResourceResponseStore.await(token, CR_RESPONSE_TIMEOUT);+ return customResourceResponseStore.await(token, CR_RESPONSE_TIMEOUT, serviceToken, region);} catch (AwsException e) {throw e;} catch (TimeoutException e) {throw new AwsException("CustomResourceTimeout","Timed out waiting for custom resource " + logicalId- + " to PUT its response to ResponseURL", 504);+ + " to PUT its response to ResponseURL: " + e.getMessage(), 504);} catch (Exception e) {throw new AwsException("CustomResourceFailed","Failed to invoke custom resource " + logicalId + ": " + e.getMessage(), 500);diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.javaindex 46eecc101f..abf20fcb20 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationService.java@@ -17,6 +17,7 @@ import io.github.hectorvent.floci.services.cloudformation.model.StackResource;import io.github.hectorvent.floci.services.cloudformation.model.TemplateSummary;import io.github.hectorvent.floci.services.cloudformation.provisioners.CfnRollback;import io.github.hectorvent.floci.services.s3.S3Service;+import io.github.hectorvent.floci.services.ssm.SsmService;import com.fasterxml.jackson.core.type.TypeReference;import com.fasterxml.jackson.databind.JsonNode;import com.fasterxml.jackson.databind.ObjectMapper;@@ -62,6 +63,7 @@ public class CloudFormationService implements ResourceProvider {private final CloudFormationResourceProvisioner provisioner;private final S3Service s3Service;+ private final SsmService ssmService;private final ObjectMapper objectMapper;private final EmulatorConfig config;private final RegionResolver regionResolver;@@ -78,11 +80,12 @@ public class CloudFormationService implements ResourceProvider {@Injectpublic CloudFormationService(CloudFormationResourceProvisioner provisioner, S3Service s3Service,- ObjectMapper objectMapper, EmulatorConfig config,+ SsmService ssmService, ObjectMapper objectMapper, EmulatorConfig config,RegionResolver regionResolver, Clock clock,StorageFactory storageFactory) {this.provisioner = provisioner;this.s3Service = s3Service;+ this.ssmService = ssmService;this.objectMapper = objectMapper;this.config = config;this.regionResolver = regionResolver;@@ -151,6 +154,16 @@ public class CloudFormationService implements ResourceProvider {.toList();}+ /**+ * The stack's current parameter values, or an empty map if it does not exist (yet). Used to+ * resolve {@code UsePreviousValue} on an update before the stack lookup that+ * {@code createChangeSet}/{@code executeChangeSet} would otherwise perform.+ */+ public Map<String, String> currentParameters(String stackName, String region) {+ Stack stack = resolveStack(stackName, region);+ return stack != null ? stack.getParameters() : Map.of();+ }+// ── CreateChangeSet ───────────────────────────────────────────────────────public ChangeSet createChangeSet(String stackName, String changeSetName, String changeSetType,@@ -300,6 +313,152 @@ public class CloudFormationService implements ResourceProvider {return cs;}+ /**+ * Computes the per-resource changes a change set would apply, by diffing its template against+ * the stack's currently deployed template. CREATE-type change sets (and stacks that have never+ * executed a template) report every resource with a truthy {@code Condition} as an Add.+ */+ public List<ResourceChange> computeChangeSetChanges(ChangeSet cs, String region) {+ Stack stack = getStackOrThrow(cs.getStackName(), region);+ try {+ JsonNode newTemplate = parseTemplate(cs.getTemplateBody());+ if (samTransformProcessor.hasSamTransform(newTemplate)) {+ // The deployed stack's template is always the SAM-expanded form (see+ // executeTemplate); comparing the change set's raw SAM source against it would+ // report every SAM-generated resource as Add/Remove even on a no-op update.+ newTemplate = samTransformProcessor.expandSamTemplate(newTemplate);+ }+ JsonNode newResources = newTemplate.path("Resources");+ boolean createType = "CREATE".equalsIgnoreCase(cs.getChangeSetType())+ || stack.getTemplateBody() == null;+ JsonNode oldResources = createType+ ? objectMapper.createObjectNode()+ : parseTemplate(stack.getTemplateBody()).path("Resources");++ Map<String, String> oldParams = stack.getParameters() != null+ ? stack.getParameters() : Map.of();+ // Prefer the SSM-resolved values captured by the last executeTemplate run; fall back to+ // the raw parameters for stacks persisted before resolvedParameters existed.+ Map<String, String> oldResolvedParams = stack.getResolvedParameters() != null+ && !stack.getResolvedParameters().isEmpty()+ ? stack.getResolvedParameters() : oldParams;+ // A parameter omitted from the update falls back to the template's Default when+ // ExecuteChangeSet actually runs it, so the preview must resolve the same defaults or+ // it will under-report changes to resources that depend on that fallback value.+ Map<String, String> newParams = resolveDefaultParameters(newTemplate,+ cs.getParameters() != null ? cs.getParameters() : Map.of());+ // ExecuteChangeSet also resolves AWS::SSM::Parameter::Value<String> parameters against+ // the live Parameter Store before applying resource changes, and the stored SSM value+ // can drift between deploys even when the referencing parameter name is unchanged. Diff+ // on the resolved values, like execution does, so the preview agrees with what actually+ // gets applied. A preview must not fail harder than the operation it previews though: if+ // the referenced SSM parameter is missing, fall back to the unresolved values here and+ // let ExecuteChangeSet raise that ValidationError when it actually resolves them.+ Map<String, String> ssmResolvedNewParams;+ try {+ ssmResolvedNewParams = resolveSsmParameters(newTemplate, newParams, region);+ } catch (AwsException e) {+ ssmResolvedNewParams = newParams;+ }+ final Map<String, String> newResolvedParams = ssmResolvedNewParams;+ Set<String> changedParams = new HashSet<>();+ newResolvedParams.forEach((k, v) -> {+ if (!Objects.equals(v, oldResolvedParams.get(k))) {+ changedParams.add(k);+ }+ });+ // A parameter that was deployed but is omitted from this update with no template+ // Default is dropped entirely by resolveDefaultParameters (matching what+ // ExecuteChangeSet does); flag its disappearance too, not just a value change.+ oldResolvedParams.keySet().forEach(k -> {+ if (!newResolvedParams.containsKey(k)) {+ changedParams.add(k);+ }+ });++ // A resource whose Condition depends on a changed parameter can flip from excluded to+ // included (or back) even when its own definition text is unchanged; ExecuteChangeSet+ // applies that as an Add or Remove (see hasRemovedOrConditionFalseResources /+ // deleteRemovedOrConditionFalseResources), so the preview must evaluate Conditions too+ // rather than only scanning each resource's own Ref/Sub usage. A resource is "active" in+ // the deployed stack precisely when it's present in stack.getResources() - the same+ // ground truth execution uses - so no separate old-conditions evaluation is needed.+ Map<String, Boolean> newConditions = resolveConditions(+ newTemplate, newResolvedParams, null, region, regionResolver.getAccountId());+ Set<String> deployedIds = stack.getResources().keySet();++ List<ResourceChange> changes = new ArrayList<>();+ newResources.fields().forEachRemaining(e -> {+ String logicalId = e.getKey();+ JsonNode newDef = e.getValue();+ String resourceType = newDef.path("Type").asText();+ JsonNode oldDef = oldResources.get(logicalId);+ String newConditionName = newDef.path("Condition").asText(null);+ boolean newActive = newConditionName == null+ || newConditions.getOrDefault(newConditionName, false);+ if (oldDef == null) {+ if (newActive) {+ changes.add(new ResourceChange("Add", logicalId, null, resourceType, null));+ }+ return;+ }+ boolean wasDeployed = deployedIds.contains(logicalId);+ if (wasDeployed && !newActive) {+ // Condition flipped true -> false: the definition text is unchanged, but+ // ExecuteChangeSet deletes the resource (see deleteRemovedOrConditionFalseResources).+ changes.add(new ResourceChange("Remove", logicalId,+ resourcePhysicalId(stack, logicalId), oldDef.path("Type").asText(), null));+ } else if (!wasDeployed && newActive) {+ // Condition flipped false -> true: never created before, ExecuteChangeSet creates+ // it now.+ changes.add(new ResourceChange("Add", logicalId, null, resourceType, null));+ } else if (newActive+ && (!oldDef.equals(newDef) || referencesAnyParameter(newDef, changedParams))) {+ boolean typeChanged = !oldDef.path("Type").asText().equals(resourceType);+ changes.add(new ResourceChange("Modify", logicalId,+ resourcePhysicalId(stack, logicalId), resourceType,+ typeChanged ? "True" : "False"));+ }+ });+ oldResources.fields().forEachRemaining(e -> {+ if (!newResources.has(e.getKey())) {+ changes.add(new ResourceChange("Remove", e.getKey(),+ resourcePhysicalId(stack, e.getKey()),+ e.getValue().path("Type").asText(), null));+ }+ });+ return changes;+ } catch (AwsException e) {+ throw e;+ } catch (Exception e) {+ throw new AwsException("ValidationError",+ "Unable to compute changes for change set " + cs.getChangeSetName()+ + ": " + e.getMessage(), 400);+ }+ }++ /** True if a resource's definition references any of the given (changed) parameter names. */+ private boolean referencesAnyParameter(JsonNode resourceDef, Set<String> parameterNames) {+ if (parameterNames.isEmpty()) {+ return false;+ }+ String json = resourceDef.toString();+ for (String name : parameterNames) {+ if (json.contains("\"Ref\":\"" + name + "\"") || json.contains("${" + name + "}")) {+ return true;+ }+ }+ return false;+ }++ public record ResourceChange(String action, String logicalResourceId, String physicalResourceId,+ String resourceType, String replacement) {}++ private String resourcePhysicalId(Stack stack, String logicalId) {+ StackResource resource = stack.getResources().get(logicalId);+ return resource != null ? resource.getPhysicalId() : null;+ }+// ── ExecuteChangeSet ──────────────────────────────────────────────────────public Future<?> executeChangeSet(String stackName, String changeSetName, String region) {@@ -629,6 +788,43 @@ public class CloudFormationService implements ResourceProvider {return resolved;}+ /**+ * Substitutes {@code AWS::SSM::Parameter::Value<String>}-typed parameter values — which carry+ * an SSM parameter <em>name</em> — with the value stored in Parameter Store for the stack's+ * account and region, as real CloudFormation does before template processing. Missing+ * parameters fail the stack operation with the real AWS ValidationError. The related types+ * {@code AWS::SSM::Parameter::Value<List<String>>} and {@code AWS::SSM::Parameter::Name} are+ * not resolved and pass through verbatim.+ */+ private Map<String, String> resolveSsmParameters(JsonNode template, Map<String, String> params, String region) {+ JsonNode paramDefs = template.path("Parameters");+ if (!paramDefs.isObject()) {+ return params;+ }+ Map<String, String> resolved = new HashMap<>(params);+ List<String> missing = new ArrayList<>();+ paramDefs.fields().forEachRemaining(e -> {+ if (!"AWS::SSM::Parameter::Value<String>".equals(e.getValue().path("Type").asText())) {+ return;+ }+ String parameterName = resolved.get(e.getKey());+ if (parameterName == null || parameterName.isBlank()) {+ return;+ }+ try {+ resolved.put(e.getKey(), ssmService.getParameter(parameterName, region).getValue());+ } catch (AwsException ex) {+ missing.add(parameterName);+ }+ });+ if (!missing.isEmpty()) {+ throw new AwsException("ValidationError",+ "Unable to fetch parameters [" + String.join(",", missing)+ + "] from parameter store for this account", 400);+ }+ return resolved;+ }+private void executeTemplate(Stack stack, String templateBody, Map<String, String> params,boolean isCreate, String region, String accountId) {StackUpdateSnapshot previousState = snapshotForUpdate(stack);@@ -649,7 +845,11 @@ public class CloudFormationService implements ResourceProvider {stack.setTemplateBody(templateBody);// Merge default parameter values from the template with caller-supplied params- Map<String, String> resolvedParams = resolveDefaultParameters(template, params);+ Map<String, String> givenParams = resolveDefaultParameters(template, params);+ stack.getParameters().clear();+ stack.getParameters().putAll(givenParams);+ Map<String, String> resolvedParams = resolveSsmParameters(template, givenParams, region);+ stack.setResolvedParameters(new LinkedHashMap<>(resolvedParams));// Resolve conditions firstMap<String, Boolean> conditions = resolveConditions(template, resolvedParams, stack, region, accountId);@@ -1069,6 +1269,14 @@ public class CloudFormationService implements ResourceProvider {List<String> rollbackFailures = rollbackUpdatedResources(stack, previousState.resources(), attemptedResourceIds, region);+ // Parameters are independent of resource-rollback outcome - always restore them to the last+ // successfully deployed values, even when resource rollback itself fails and the stack lands+ // in UPDATE_ROLLBACK_FAILED, so DescribeStacks and later change-set previews don't keep+ // serving the failed update's attempted values.+ stack.getParameters().clear();+ stack.getParameters().putAll(previousState.parameters());+ stack.getResolvedParameters().clear();+ stack.getResolvedParameters().putAll(previousState.resolvedParameters());if (rollbackFailures.isEmpty()) {stack.setTemplateBody(previousState.templateBody());}@@ -1214,6 +1422,8 @@ public class CloudFormationService implements ResourceProvider {private StackUpdateSnapshot snapshotForUpdate(Stack stack) {return new StackUpdateSnapshot(stack.getTemplateBody(),+ new LinkedHashMap<>(stack.getParameters()),+ new LinkedHashMap<>(stack.getResolvedParameters()),new LinkedHashMap<>(stack.getOutputs()),new LinkedHashMap<>(stack.getExports()),new LinkedHashMap<>(stack.getOutputExportNames()),@@ -1244,6 +1454,8 @@ public class CloudFormationService implements ResourceProvider {private record StackUpdateSnapshot(String templateBody,+ Map<String, String> parameters,+ Map<String, String> resolvedParameters,Map<String, String> outputs,Map<String, String> exports,Map<String, String> outputExportNames,diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStore.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStore.javaindex 0f5745f9e7..0156337d37 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStore.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStore.java@@ -1,7 +1,9 @@package io.github.hectorvent.floci.services.cloudformation;import com.fasterxml.jackson.databind.JsonNode;+import io.github.hectorvent.floci.core.common.CustomResourceLiveness;import jakarta.enterprise.context.ApplicationScoped;+import jakarta.inject.Inject;import org.jboss.logging.Logger;import java.time.Duration;@@ -10,56 +12,126 @@ import java.util.concurrent.CompletableFuture;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.TimeUnit;import java.util.concurrent.TimeoutException;+import java.util.concurrent.atomic.AtomicLong;/**- * Correlates a CloudFormation custom-resource Lambda invocation with the response the- * Lambda PUTs back to its {@code ResponseURL}.+ * Holds the futures that custom-resource ResponseURL callbacks complete.*- * <p>The AWS custom-resource protocol is callback-based: the backing Lambda does not return- * its result, it HTTP PUTs a response document to a pre-signed {@code ResponseURL}. Floci hands- * the Lambda a {@code ResponseURL} pointing at {@link CfnResponseController}, which feeds the- * body back here keyed by a per-invocation token.- *- * <p>Because Floci invokes the Lambda synchronously (RequestResponse) and the handler PUTs- * before returning, the future is normally already completed by the time the provisioner awaits- * it — the timeout only guards a PUT that lands just after the container returns.+ * <p>The wait budget is an <em>idle</em> timeout, not a total one. A CDK provider-framework waiter+ * drives {@code framework.isComplete} on a fixed cadence and PUTs only once the work is done, so the+ * total time is a property of the work rather than of the emulator — {@code+ * Custom::CreateOrganizationAccounts} creates one account per poll, meaning a 15-account org takes+ * five times as long as a 3-account one. Budgeting total time therefore needs re-tuning per config+ * set, and overshooting guillotines a Lambda that was succeeding. Budgeting idleness needs no+ * knowledge of the work at all: each poll is liveness, so progress buys unbounded wall-clock while a+ * resource that has genuinely stopped still fails on schedule.*/@ApplicationScoped-public class CustomResourceResponseStore {-+public class CustomResourceResponseStore implements CustomResourceLiveness {private static final Logger LOG = Logger.getLogger(CustomResourceResponseStore.class);+ private final ConcurrentHashMap<String, Pending> pending = new ConcurrentHashMap<>();+ private final ProviderFrameworkDetector providerFrameworkDetector;- private final ConcurrentHashMap<String, CompletableFuture<JsonNode>> pending = new ConcurrentHashMap<>();+ @Inject+ CustomResourceResponseStore(ProviderFrameworkDetector providerFrameworkDetector) {+ this.providerFrameworkDetector = providerFrameworkDetector;+ }++ /** A waiting caller: the future its callback completes, and when it last saw progress. */+ private static final class Pending {+ private final CompletableFuture<JsonNode> future = new CompletableFuture<>();+ // nanoTime, not wall-clock: this measures an elapsed interval and must not jump with NTP.+ private final AtomicLong lastActivity = new AtomicLong(System.nanoTime());+ }/** Registers a fresh token and returns it. Call before invoking the Lambda. */public String register() {String token = UUID.randomUUID().toString();- pending.put(token, new CompletableFuture<>());+ pending.put(token, new Pending());return token;}/** Completes the future for {@code token} with the PUT body. No-op for unknown/expired tokens. */public void complete(String token, JsonNode response) {- CompletableFuture<JsonNode> future = pending.get(token);- if (future != null) {- future.complete(response);+ Pending entry = pending.get(token);+ if (entry != null) {+ entry.future.complete(response);} else {LOG.debugv("Received custom-resource response for unknown token {0}", token);}}/**- * Waits up to {@code timeout} for the Lambda's response, then discards the token.+ * Records liveness for {@code token}, resetting its idle budget.+ *+ * <p>Unknown tokens are ignored by design: polls routinely outlive the await that registered+ * them (after a timeout or a rollback), and that tail is normal rather than an error.+ */+ @Override+ public void touch(String token) {+ Pending entry = pending.get(token);+ if (entry != null) {+ entry.lastActivity.set(System.nanoTime());+ }+ }++ /**+ * Waits for the Lambda's response on the budget that suits the handler behind {@code serviceToken},+ * then discards the token.+ *+ * <p>A CDK Provider-framework {@code framework.onEvent} handler PUTs asynchronously via its waiter+ * state machine and gets a longer idle budget than {@code synchronousIdleTimeout}; see {@link+ * ProviderFrameworkDetector#responseTimeout}.+ *+ * @param synchronousIdleTimeout the budget for a handler that PUTs during its own invocation+ * @throws TimeoutException if no response and no liveness arrive within the chosen budget+ */+ public JsonNode await(String token, Duration synchronousIdleTimeout, String serviceToken, String region)+ throws TimeoutException {+ return await(token,+ providerFrameworkDetector.responseTimeout(serviceToken, region, synchronousIdleTimeout));+ }++ /**+ * Waits for the Lambda's response, then discards the token.*- * @throws TimeoutException if no response arrives in time+ * @param idleTimeout how long to tolerate <em>no</em> progress; each {@link #touch} restarts it,+ * so a resource that keeps polling is never cut off+ * @throws TimeoutException if no response and no liveness arrive within {@code idleTimeout}*/- public JsonNode await(String token, Duration timeout) throws TimeoutException {- CompletableFuture<JsonNode> future = pending.get(token);- if (future == null) {+ public JsonNode await(String token, Duration idleTimeout) throws TimeoutException {+ Pending entry = pending.get(token);+ if (entry == null) {throw new IllegalStateException("No pending custom-resource token: " + token);}+ long idleNanos = idleTimeout.toNanos();+ // The budget measures idleness of the WAIT, so it starts when the caller begins waiting rather+ // than when the token was registered. Registration happens before the invoke, and a synchronous+ // handler PUTs from inside the invoke the caller is blocked on: charging that execution time+ // here would leave a slow handler — a cold container start, or two of them nested — with a+ // budget already spent by the time await was entered. A handler that just returned is not idle.+ entry.lastActivity.set(System.nanoTime());try {- return future.get(timeout.toMillis(), TimeUnit.MILLISECONDS);+ while (true) {+ long remaining = idleNanos - (System.nanoTime() - entry.lastActivity.get());+ if (remaining <= 0) {+ // A response that has already arrived is a response, whatever the clock says. The+ // future outranks the budget: complete() can land before await is ever entered, and+ // discarding a delivered response as a timeout fails a resource that succeeded.+ if (entry.future.isDone()) {+ return entry.future.get();+ }+ throw new TimeoutException("No custom-resource activity for " + idleTimeout);+ }+ try {+ return entry.future.get(remaining, TimeUnit.NANOSECONDS);+ } catch (TimeoutException e) {+ // This slice elapsed. A concurrent touch may have pushed lastActivity forward,+ // in which case the next slice is positive again and the wait continues; if+ // nothing touched, the guard above fails on the very next pass.+ LOG.tracev("Custom-resource wait slice elapsed for token {0}; rechecking liveness", token);+ }+ }} catch (TimeoutException e) {throw e;} catch (Exception e) {diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/ProviderFrameworkDetector.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/ProviderFrameworkDetector.javanew file mode 100644index 0000000000..3185104925--- /dev/null+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/ProviderFrameworkDetector.java@@ -0,0 +1,98 @@+package io.github.hectorvent.floci.services.cloudformation;++import io.github.hectorvent.floci.services.lambda.LambdaService;+import io.github.hectorvent.floci.services.lambda.model.LambdaFunction;+import jakarta.enterprise.context.ApplicationScoped;+import jakarta.inject.Inject;+import org.jboss.logging.Logger;++import java.time.Duration;+import java.util.Map;++/**+ * Decides how long a custom-resource ResponseURL callback may stay idle, by recognising the CDK+ * Provider framework's {@code framework.onEvent} Lambda from its environment.+ */+@ApplicationScoped+public class ProviderFrameworkDetector {++ private static final Logger LOG = Logger.getLogger(ProviderFrameworkDetector.class);++ /**+ * Environment variables the CDK Provider framework sets on {@code framework.onEvent} only when an+ * {@code isComplete} handler is configured — i.e. when the two-phase async waiter is in play. Their+ * presence distinguishes a Provider-framework onEvent (which PUTs asynchronously via the waiter)+ * from an onEvent-only provider or a plain single-Lambda handler (which PUT synchronously).+ */+ private static final String CR_USER_IS_COMPLETE_ENV = "USER_IS_COMPLETE_FUNCTION_ARN";+ private static final String CR_WAITER_STATE_MACHINE_ENV = "WAITER_STATE_MACHINE_ARN";++ /**+ * How long to wait when the ServiceToken is a CDK Provider-framework {@code framework.onEvent}+ * Lambda: it does not PUT itself but returns after starting the waiter state machine, so the+ * ResponseURL callback arrives asynchronously once {@code framework.isComplete} reports done (or+ * {@code framework.onTimeout} reports failure).+ *+ * <p>This is an <em>idle</em> budget, not a total one: every waiter poll resets it (see {@link+ * CustomResourceResponseStore#touch}). Total time here is a property of the work rather than of+ * the emulator — {@code Custom::CreateOrganizationAccounts} creates one account per poll, so 15+ * accounts take five times as long as three — and a total budget would need re-tuning for every+ * config set while guillotining Lambdas that were succeeding. Measuring idleness still fails a+ * genuinely hung resource cleanly instead of hanging. Non-final so tests can shorten it.+ */+ private Duration asyncCustomResourceTimeout = Duration.ofMinutes(3);++ private final LambdaService lambdaService;++ @Inject+ ProviderFrameworkDetector(LambdaService lambdaService) {+ this.lambdaService = lambdaService;+ }++ /**+ * The idle budget to allow the handler behind {@code serviceToken}.+ *+ * <p>{@code framework.onEvent} returns without PUTting (it starts the waiter state machine, which+ * drives {@code framework.isComplete} on a Retry cadence until it PUTs). So the callback lands+ * asynchronously and needs a longer wait than the synchronous single-Lambda pattern. onEvent-only+ * providers and plain handlers PUT during the invoke, so their await returns immediately+ * regardless of this budget.+ *+ * @param synchronousTimeout the budget for a handler that PUTs during its own invocation+ */+ Duration responseTimeout(String serviceToken, String region, Duration synchronousTimeout) {+ return isProviderFrameworkOnEvent(serviceToken, region)+ ? asyncCustomResourceTimeout : synchronousTimeout;+ }++ /**+ * True when the ServiceToken resolves to a CDK Provider-framework {@code framework.onEvent} Lambda,+ * i.e. one whose environment carries both {@code USER_IS_COMPLETE_FUNCTION_ARN} and+ * {@code WAITER_STATE_MACHINE_ARN}. The framework sets these only when a two-phase async waiter is+ * configured, so their presence means the ResponseURL callback will arrive asynchronously via the+ * waiter rather than synchronously during the invoke. Best-effort: an unresolvable function (or one+ * with no environment) is treated as a synchronous handler.+ */+ boolean isProviderFrameworkOnEvent(String serviceToken, String region) {+ try {+ LambdaFunction fn = lambdaService.getFunction(region, serviceToken);+ Map<String, String> env = fn != null ? fn.getEnvironment() : null;+ if (env == null) {+ return false;+ }+ String isComplete = env.get(CR_USER_IS_COMPLETE_ENV);+ String waiter = env.get(CR_WAITER_STATE_MACHINE_ENV);+ return isComplete != null && !isComplete.isBlank()+ && waiter != null && !waiter.isBlank();+ } catch (RuntimeException e) {+ LOG.debugv("Could not read environment for custom-resource handler {0}: {1}",+ serviceToken, e.getMessage());+ return false;+ }+ }++ /** Test hook: shortens the async-callback wait so the timeout bound can be exercised quickly. */+ void setAsyncCustomResourceTimeoutForTesting(Duration timeout) {+ this.asyncCustomResourceTimeout = timeout;+ }+}diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/model/Stack.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/model/Stack.javaindex d4fadc3078..3aea14e64b 100644--- a/src/main/java/io/github/hectorvent/floci/services/cloudformation/model/Stack.java+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/model/Stack.java@@ -20,6 +20,10 @@ public class Stack {private String originalTemplateBody;private List<String> capabilities = new ArrayList<>();private Map<String, String> parameters = new LinkedHashMap<>();+ // Parameters after AWS::SSM::Parameter::Value<String> resolution, as last applied by+ // executeTemplate — used to detect drift in the live SSM value between deploys even when the+ // referencing parameter (name) is unchanged, so change-set previews agree with execution.+ private Map<String, String> resolvedParameters = new LinkedHashMap<>();private Map<String, String> outputs = new LinkedHashMap<>();private Map<String, String> exports = new LinkedHashMap<>();// Maps output key to its export name (when Export.Name is defined on an output)@@ -52,6 +56,8 @@ public class Stack {public void setCapabilities(List<String> capabilities) { this.capabilities = capabilities; }public Map<String, String> getParameters() { return parameters; }public void setParameters(Map<String, String> parameters) { this.parameters = parameters; }+ public Map<String, String> getResolvedParameters() { return resolvedParameters; }+ public void setResolvedParameters(Map<String, String> resolvedParameters) { this.resolvedParameters = resolvedParameters; }public Map<String, String> getOutputs() { return outputs; }public void setOutputs(Map<String, String> outputs) { this.outputs = outputs; }public Map<String, String> getExports() { return exports; }diff --git a/src/main/java/io/github/hectorvent/floci/services/lambda/LambdaService.java b/src/main/java/io/github/hectorvent/floci/services/lambda/LambdaService.javaindex 6861c774ce..73df4b0258 100644--- a/src/main/java/io/github/hectorvent/floci/services/lambda/LambdaService.java+++ b/src/main/java/io/github/hectorvent/floci/services/lambda/LambdaService.java@@ -4,6 +4,7 @@ import com.fasterxml.jackson.core.type.TypeReference;import io.github.hectorvent.floci.config.EmulatorConfig;import io.github.hectorvent.floci.core.common.AwsArnUtils;import io.github.hectorvent.floci.core.common.AwsException;+import io.github.hectorvent.floci.core.common.CustomResourceLiveness;import io.github.hectorvent.floci.core.common.RegionResolver;import io.github.hectorvent.floci.core.storage.StorageBackedMap;import io.github.hectorvent.floci.core.storage.StorageFactory;@@ -85,6 +86,8 @@ public class LambdaService implements ResourceProvider {private final StorageFactory storageFactory;private final LambdaLayerService layerService;private final Ec2Service ec2Service;+ /** Null in the constructors tests use, exactly as the other optional collaborators above are. */+ private final CustomResourceLiveness customResourceLiveness;private Map<String, Integer> versionCounters = new ConcurrentHashMap<>();private Map<String, FunctionEventInvokeConfig> eventInvokeConfigs = new ConcurrentHashMap<>();/**@@ -155,6 +158,7 @@ public class LambdaService implements ResourceProvider {this.storageFactory = storageFactory;this.layerService = null;this.ec2Service = null;+ this.customResourceLiveness = null;}@Inject@@ -175,7 +179,9 @@ public class LambdaService implements ResourceProvider {DynamoDbStreamsEventSourcePoller dynamodbStreamsPoller,StorageFactory storageFactory,LambdaLayerService layerService,- Ec2Service ec2Service) {+ Ec2Service ec2Service,+ CustomResourceLiveness customResourceLiveness) {+ this.customResourceLiveness = customResourceLiveness;this.functionStore = functionStore;this.executorService = executorService;this.concurrencyLimiter = concurrencyLimiter;@@ -766,6 +772,7 @@ public class LambdaService implements ResourceProvider {} else {fn = resolveInvokeTarget(region, name, qualifier);}+ reportCustomResourceLiveness(payload);InvokeResult result = executorService.invoke(fn, payload, type);result.setExecutedVersion(fn.getVersion());return result;@@ -782,6 +789,21 @@ public class LambdaService implements ResourceProvider {return result;}+ /**+ * Reports that a pending custom resource is still making progress, if this payload belongs to+ * one. A CDK provider-framework waiter re-invokes {@code framework.isComplete} on a cadence and+ * echoes the original event -- including its ResponseURL -- into every poll, so a poll landing+ * here is proof of liveness for that resource's callback token. Resetting the idle budget on it+ * keeps a long-but-progressing resource (12 org accounts at one per poll) from being cut off+ * mid-success, without CloudFormation needing to know how much work is left.+ */+ private void reportCustomResourceLiveness(byte[] payload) {+ if (customResourceLiveness == null) {+ return;+ }+ CustomResourceLiveness.tokenIn(payload).ifPresent(customResourceLiveness::touch);+ }+private LambdaFunction resolveInvokeTarget(String region, String name, String qualifier) {if (qualifier == null || qualifier.equals("$LATEST")) {return functionStore.get(region, name)diff --git a/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java b/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.javaindex 72313da043..d8aad8aedb 100644--- a/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java+++ b/src/main/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutor.java@@ -4,6 +4,7 @@ import io.github.hectorvent.floci.config.EmulatorConfig;import io.github.hectorvent.floci.core.common.AwsArnUtils;import io.github.hectorvent.floci.core.common.AwsException;import io.github.hectorvent.floci.core.common.AwsErrorResponse;+import io.github.hectorvent.floci.core.common.CustomResourceLiveness;import io.github.hectorvent.floci.core.common.RequestContext;import io.github.hectorvent.floci.core.common.XmlParser;import io.github.hectorvent.floci.services.cloudformation.CloudFormationQueryHandler;@@ -180,6 +181,7 @@ public class AslExecutor {private final Instance<StepFunctionsService> sfnService;private final WebClient webClient;private final EmulatorConfig config;+ private final CustomResourceLiveness customResourceLiveness;private final Map<String, MockedTestCase> activeMocks = new ConcurrentHashMap<>();private final ExecutorService executor = Executors.newCachedThreadPool(r -> {Thread t = new Thread(r, "sfn-executor");@@ -196,7 +198,9 @@ public class AslExecutor {EventBridgeHandler eventBridgeHandler, SchedulerService schedulerService,SchedulerController schedulerController,ObjectMapper objectMapper, JsonataEvaluator jsonataEvaluator,- Instance<StepFunctionsService> sfnService, EmulatorConfig config, Vertx vertx) {+ Instance<StepFunctionsService> sfnService, EmulatorConfig config, Vertx vertx,+ CustomResourceLiveness customResourceLiveness) {+ this.customResourceLiveness = customResourceLiveness;this.lambdaExecutor = lambdaExecutor;this.functionStore = functionStore;this.dynamoDbService = dynamoDbService;@@ -726,6 +730,21 @@ public class AslExecutor {return fn;}+ /**+ * Reports that a pending custom resource is still making progress, if this payload belongs to+ * one. The Step Functions Task path drives a CDK provider-framework waiter's {@code+ * framework.isComplete} polls straight through {@link LambdaExecutorService}, bypassing {@link+ * io.github.hectorvent.floci.services.lambda.LambdaService#invoke} and the liveness hook it+ * carries -- so this poll would otherwise never reset the resource's idle budget in {@link+ * io.github.hectorvent.floci.services.cloudformation.CustomResourceResponseStore}.+ */+ private void reportCustomResourceLiveness(byte[] payload) {+ if (customResourceLiveness == null) {+ return;+ }+ CustomResourceLiveness.tokenIn(payload).ifPresent(customResourceLiveness::touch);+ }+private JsonNode invokeResource(String resource, JsonNode input, StateMachine sm, String taskToken) throws Exception {// Support Lambda resources: direct ARN or optimized integrationString functionName = null;@@ -759,8 +778,9 @@ public class AslExecutor {"Lambda function not found: " + functionName);}- String payloadStr = objectMapper.writeValueAsString(lambdaPayload);- InvokeResult result = lambdaExecutor.invoke(fn, payloadStr.getBytes(), InvocationType.RequestResponse);+ byte[] payloadBytes = objectMapper.writeValueAsString(lambdaPayload).getBytes();+ reportCustomResourceLiveness(payloadBytes);+ InvokeResult result = lambdaExecutor.invoke(fn, payloadBytes, InvocationType.RequestResponse);if (result.getFunctionError() != null) {throw new FailStateException("Lambda.AWSLambdaException", result.getFunctionError());diff --git a/src/test/java/io/github/hectorvent/floci/core/common/CustomResourceLivenessTest.java b/src/test/java/io/github/hectorvent/floci/core/common/CustomResourceLivenessTest.javanew file mode 100644index 0000000000..37b3591bbf--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/core/common/CustomResourceLivenessTest.java@@ -0,0 +1,64 @@+package io.github.hectorvent.floci.core.common;++import org.junit.jupiter.api.Test;++import java.nio.charset.StandardCharsets;+import java.util.Optional;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;++/**+ * Token extraction is the one part of the heartbeat that can fail silently: if it never matches, the+ * idle budget simply never resets and the original fixed-timeout bug is back, invisibly. So the+ * payloads here are shaped like the ones actually observed on the wire rather than invented.+ */+class CustomResourceLivenessTest {++ private static final String TOKEN = "6ef77fe2-38c8-41b0-bec6-68879687fbc8";++ private Optional<String> tokenIn(String payload) {+ return CustomResourceLiveness.tokenIn(payload.getBytes(StandardCharsets.UTF_8));+ }++ @Test+ void findsTheTokenInAProviderFrameworkPoll() {+ // Verbatim shape of the isComplete poll captured from a Prepare-stage run.+ String payload = "{\"RequestType\":\"Create\","+ + "\"ServiceToken\":\"arn:aws:lambda:us-east-1:000000000000:function:framework-onEvent\","+ + "\"ResponseURL\":\"http://localhost.floci.io:4566/cfn-response/" + TOKEN + "\","+ + "\"StackId\":\"arn:aws:cloudformation:us-east-1:000000000000:stack/AWSAccelerator-PrepareStack/abc\","+ + "\"RequestId\":\"req-1\",\"LogicalResourceId\":\"CreateAccounts\","+ + "\"ResourceType\":\"Custom::CreateOrganizationAccounts\","+ + "\"ResourceProperties\":{\"ServiceToken\":\"arn:aws:lambda:...\",\"uuid\":\"72f7b8a8\"}}";++ assertEquals(Optional.of(TOKEN), tokenIn(payload));+ }++ @Test+ void findsTheTokenWhenTheUrlIsTheLastFieldWithNoTrailingPunctuation() {+ assertEquals(Optional.of(TOKEN),+ tokenIn("http://localhost.floci.io:4566/cfn-response/" + TOKEN));+ }++ @Test+ void findsTheTokenAcrossJsonWhitespaceAndSingleQuotedRenderings() {+ assertEquals(Optional.of(TOKEN),+ tokenIn("{ \"ResponseURL\" : 'http://host:4566/cfn-response/" + TOKEN + "' }"));+ }++ @Test+ void ignoresOrdinaryInvokesThatCarryNoCallbackUrl() {+ assertTrue(tokenIn("{\"Records\":[{\"body\":\"hello\"}]}").isEmpty());+ assertTrue(CustomResourceLiveness.tokenIn(new byte[0]).isEmpty());+ assertTrue(CustomResourceLiveness.tokenIn(null).isEmpty());+ }++ @Test+ void ignoresATruncatedUrlWithNoTokenAfterThePath() {+ // The framework's own logging redacts ResponseURL to "..."; that rendering must not produce+ // a bogus token that would touch nothing and mask the real signal.+ assertTrue(tokenIn("{\"ResponseURL\":\"...\"}").isEmpty());+ assertTrue(tokenIn("http://host:4566/cfn-response/").isEmpty());+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationCdkDeployLoopIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationCdkDeployLoopIntegrationTest.javanew file mode 100644index 0000000000..244e1ec234--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationCdkDeployLoopIntegrationTest.java@@ -0,0 +1,267 @@+package io.github.hectorvent.floci.services.cloudformation;++import io.quarkus.test.junit.QuarkusTest;+import org.junit.jupiter.api.Test;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.not;++/**+ * End-to-end checks for the CloudFormation responses CDK's deploy loop depends on:+ * DescribeChangeSet must report a real Add/Modify/Remove diff (an empty Changes list makes CDK+ * skip ExecuteChangeSet on updates), Fn::GetAtt [Param, Value] on an SSM parameter must resolve+ * to the stored value (CDK reads its bootstrap version that way), and DescribeStacks must echo+ * the stack's parameters (CDK's parameter merge treats a missing list as an empty set).+ */+@QuarkusTest+class CloudFormationCdkDeployLoopIntegrationTest {++ private static final String CFN_AUTH =+ "AWS4-HMAC-SHA256 Credential=test/20260808/us-east-1/cloudformation/aws4_request";++ @Test+ void describeChangeSet_createType_reportsAllResourcesAsAdd() {+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "cs-diff-create-stack-" + suffix;++ String template = """+ {+ "Resources": {+ "Bucket": {+ "Type": "AWS::S3::Bucket",+ "Properties": {"BucketName": "cs-diff-create-bucket-%s"}+ },+ "Queue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-create-queue-%s"}+ }+ }+ }+ """.formatted(suffix, suffix);++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "create-changeset")+ .formParam("ChangeSetType", "CREATE")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<Id>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "create-changeset")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(not(containsString("<Changes/>")))+ .body(containsString("<ResourceChange><Action>Add</Action>"+ + "<LogicalResourceId>Bucket</LogicalResourceId>"+ + "<ResourceType>AWS::S3::Bucket</ResourceType>"+ + "<Scope/><Details/></ResourceChange>"))+ .body(containsString("<ResourceChange><Action>Add</Action>"+ + "<LogicalResourceId>Queue</LogicalResourceId>"+ + "<ResourceType>AWS::SQS::Queue</ResourceType>"+ + "<Scope/><Details/></ResourceChange>"));+ }++ @Test+ void describeChangeSet_updateType_reportsAddModifyRemove() {+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "cs-diff-update-stack-" + suffix;++ String initialTemplate = """+ {+ "Resources": {+ "KeepQueue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-keep-%s"}+ },+ "ModQueue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-mod-%s", "DelaySeconds": 0}+ },+ "OldQueue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-old-%s"}+ }+ }+ }+ """.formatted(suffix, suffix, suffix);++ String updatedTemplate = """+ {+ "Resources": {+ "KeepQueue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-keep-%s"}+ },+ "ModQueue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-mod-%s", "DelaySeconds": 5}+ },+ "NewQueue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-new-%s"}+ }+ }+ }+ """.formatted(suffix, suffix, suffix);++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", initialTemplate)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "update-changeset")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", updatedTemplate)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<Id>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "update-changeset")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<ResourceChange><Action>Add</Action>"+ + "<LogicalResourceId>NewQueue</LogicalResourceId>"+ + "<ResourceType>AWS::SQS::Queue</ResourceType>"+ + "<Scope/><Details/></ResourceChange>"))+ .body(containsString("<Action>Modify</Action>"+ + "<LogicalResourceId>ModQueue</LogicalResourceId>"+ + "<PhysicalResourceId>"))+ .body(containsString("<Replacement>False</Replacement>"))+ .body(containsString("<Action>Remove</Action>"+ + "<LogicalResourceId>OldQueue</LogicalResourceId>"+ + "<PhysicalResourceId>"))+ .body(not(containsString("<LogicalResourceId>KeepQueue</LogicalResourceId>")));+ }++ @Test+ void getAttOnSsmParameter_resolvesValueInOutputs() {+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "ssm-getatt-stack-" + suffix;++ String template = """+ {+ "Resources": {+ "CdkBootstrapVersion": {+ "Type": "AWS::SSM::Parameter",+ "Properties": {+ "Name": "/ssm-getatt-test/%s/version",+ "Type": "String",+ "Value": "21"+ }+ }+ },+ "Outputs": {+ "BootstrapVersion": {"Value": {"Fn::GetAtt": ["CdkBootstrapVersion", "Value"]}}+ }+ }+ """.formatted(suffix);++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"))+ .body(containsString("<OutputKey>BootstrapVersion</OutputKey><OutputValue>21</OutputValue>"))+ .body(not(containsString("CdkBootstrapVersion.Value")));+ }++ @Test+ void describeStacks_includesMergedStackParameters() {+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "params-echo-stack-" + suffix;++ String template = """+ {+ "Parameters": {+ "Env": {"Type": "String"},+ "Tier": {"Type": "String", "Default": "standard"}+ },+ "Resources": {+ "Queue": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "params-echo-queue-%s"}+ }+ }+ }+ """.formatted(suffix);++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Env")+ .formParam("Parameters.member.1.ParameterValue", "dev")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .header("Authorization", CFN_AUTH)+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"))+ .body(containsString("<ParameterKey>Env</ParameterKey><ParameterValue>dev</ParameterValue>"))+ .body(containsString("<ParameterKey>Tier</ParameterKey><ParameterValue>standard</ParameterValue>"));+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationChangeSetDiffIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationChangeSetDiffIntegrationTest.javanew file mode 100644index 0000000000..689c2414fc--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationChangeSetDiffIntegrationTest.java@@ -0,0 +1,711 @@+package io.github.hectorvent.floci.services.cloudformation;++import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import org.junit.jupiter.api.AfterEach;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;++import java.util.ArrayList;+import java.util.List;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.not;++/**+ * Covers three ways {@code computeChangeSetChanges} previously misreported a change set's diff:+ * ignoring parameter-only updates, comparing a SAM stack's raw template against its expanded+ * deployed template, and hardcoding {@code Replacement=False} even when a Type change forces a+ * replacement.+ */+@QuarkusTest+class CloudFormationChangeSetDiffIntegrationTest {++ private static final String SSM_CONTENT_TYPE = "application/x-amz-json-1.1";++ private final List<String> stacksToDelete = new ArrayList<>();++ @BeforeAll+ static void configureRestAssured() {+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @AfterEach+ void deleteStacks() {+ for (String stackName : stacksToDelete) {+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DeleteStack")+ .formParam("StackName", stackName)+ .when()+ .post("/");+ }+ stacksToDelete.clear();+ }++ @Test+ void parameterOnlyUpdate_isReportedAsAChange() {+ String stackName = "cs-diff-param-only-stack";+ stacksToDelete.add(stackName);++ String template = """+ {+ "Parameters": {+ "Suffix": {"Type": "String"}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cs-diff-${Suffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "one")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Same template, only the parameter value changes.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "param-only-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "two")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "param-only-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>Q</LogicalResourceId>"))+ .body(containsString("<Action>Modify</Action>"));+ }++ @Test+ void usePreviousValueParameter_isNotReportedAsAChange() {+ String stackName = "cs-diff-use-previous-value-stack";+ stacksToDelete.add(stackName);++ String template = """+ {+ "Parameters": {+ "Suffix": {"Type": "String"}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cs-diff-upv-${Suffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "one")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Same template, and the parameter says UsePreviousValue instead of resubmitting its value.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "use-previous-value-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.UsePreviousValue", "true")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "use-previous-value-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(not(containsString("<Action>Modify</Action>")));+ }++ @Test+ void omittedParameterFallingBackToDefault_isReportedAsAChange() {+ String stackName = "cs-diff-omitted-param-default-stack";+ stacksToDelete.add(stackName);++ String template = """+ {+ "Parameters": {+ "Suffix": {"Type": "String", "Default": "fallback"}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cs-diff-omit-${Suffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "override")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Same template, but the update omits Suffix entirely - real CloudFormation falls back to+ // the template Default ("fallback"), which differs from the deployed value ("override").+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "omitted-param-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "omitted-param-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>Q</LogicalResourceId>"))+ .body(containsString("<Action>Modify</Action>"));+ }++ @Test+ void omittedParameterWithNoDefault_isReportedAsAChange() {+ String stackName = "cs-diff-omitted-param-no-default-stack";+ stacksToDelete.add(stackName);++ String template = """+ {+ "Parameters": {+ "Suffix": {"Type": "String"}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cs-diff-omit-nodefault-${Suffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "override")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Same template, but the update omits Suffix entirely and the template has no Default -+ // resolveDefaultParameters drops the parameter, and ExecuteChangeSet would apply the+ // resulting (missing) value. The preview must flag the resource as changed too.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "omitted-param-no-default-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "omitted-param-no-default-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>Q</LogicalResourceId>"))+ .body(containsString("<Action>Modify</Action>"));+ }++ @Test+ void missingSsmParameterInPreview_doesNotFailTheDescribe() {+ String stackName = "cs-diff-ssm-missing-stack";+ stacksToDelete.add(stackName);+ String deployedSsmParamName = "/cfn/test/cs-diff-ssm-missing-deployed";++ given()+ .header("X-Amz-Target", "AmazonSSM.PutParameter")+ .contentType(SSM_CONTENT_TYPE)+ .body("""+ {+ "Name": "%s",+ "Value": "one",+ "Type": "String",+ "Overwrite": true+ }+ """.formatted(deployedSsmParamName))+ .when()+ .post("/")+ .then()+ .statusCode(200);++ String template = """+ {+ "Parameters": {+ "QueueSuffix": {"Type": "AWS::SSM::Parameter::Value<String>"}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cs-diff-ssm-missing-${QueueSuffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "QueueSuffix")+ .formParam("Parameters.member.1.ParameterValue", deployedSsmParamName)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // The update points QueueSuffix at an SSM parameter that was never put. ExecuteChangeSet+ // would fail resolving it, but DescribeChangeSet is only a preview - it must not 400 just+ // because the current value can't be resolved.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "ssm-missing-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "QueueSuffix")+ .formParam("Parameters.member.1.ParameterValue", "/cfn/test/cs-diff-ssm-missing-never-put")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "ssm-missing-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200);+ }++ @Test+ void samStackNoOpUpdate_doesNotReportSpuriousChanges() {+ String stackName = "cs-diff-sam-noop-stack";+ stacksToDelete.add(stackName);++ String template = """+ AWSTemplateFormatVersion: '2010-09-09'+ Transform: AWS::Serverless-2016-10-31+ Resources:+ HelloFunction:+ Type: AWS::Serverless::Function+ Properties:+ FunctionName: cs-diff-sam-func+ Handler: index.handler+ Runtime: nodejs22.x+ InlineCode: |+ exports.handler = async () => ({ statusCode: 200, body: 'ok' });+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Byte-identical re-submission of the same SAM source: the deployed template is the+ // *expanded* form, so diffing the raw SAM source against it must not report the+ // SAM-generated IAM role as removed or the function as modified.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "sam-noop-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Capabilities.member.1", "CAPABILITY_IAM")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "sam-noop-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(not(containsString("<Action>Remove</Action>")))+ .body(not(containsString("<Action>Modify</Action>")));+ }++ @Test+ void resourceTypeChange_reportsReplacementTrue() {+ String stackName = "cs-diff-replacement-stack";+ stacksToDelete.add(stackName);++ String initialTemplate = """+ {+ "Resources": {+ "R": {+ "Type": "AWS::SNS::Topic",+ "Properties": {"TopicName": "cs-diff-repl-topic"}+ }+ }+ }+ """;+ // Same logical id, different resource Type: AWS always forces a replacement here.+ String replacedTemplate = """+ {+ "Resources": {+ "R": {+ "Type": "AWS::SQS::Queue",+ "Properties": {"QueueName": "cs-diff-repl-queue"}+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", initialTemplate)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "replacement-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", replacedTemplate)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "replacement-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<Replacement>True</Replacement>"));+ }++ @Test+ void ssmParameterValueDrift_isReportedAsAChange() {+ String stackName = "cs-diff-ssm-drift-stack";+ stacksToDelete.add(stackName);+ String ssmParamName = "/cfn/test/cs-diff-ssm-drift-suffix";++ given()+ .header("X-Amz-Target", "AmazonSSM.PutParameter")+ .contentType(SSM_CONTENT_TYPE)+ .body("""+ {+ "Name": "%s",+ "Value": "one",+ "Type": "String",+ "Overwrite": true+ }+ """.formatted(ssmParamName))+ .when()+ .post("/")+ .then()+ .statusCode(200);++ String template = """+ {+ "Parameters": {+ "QueueSuffix": {"Type": "AWS::SSM::Parameter::Value<String>"}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cs-diff-ssm-${QueueSuffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "QueueSuffix")+ .formParam("Parameters.member.1.ParameterValue", ssmParamName)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // The SSM parameter's value changes in Parameter Store; the referencing template parameter+ // (the SSM parameter *name*) is resubmitted unchanged. Real CloudFormation re-resolves the+ // SSM value at CreateChangeSet time, so ExecuteChangeSet would apply "two" to the queue —+ // the preview must report that Modify, not compare the unchanged raw parameter name.+ given()+ .header("X-Amz-Target", "AmazonSSM.PutParameter")+ .contentType(SSM_CONTENT_TYPE)+ .body("""+ {+ "Name": "%s",+ "Value": "two",+ "Type": "String",+ "Overwrite": true+ }+ """.formatted(ssmParamName))+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "ssm-drift-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "QueueSuffix")+ .formParam("Parameters.member.1.ParameterValue", ssmParamName)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "ssm-drift-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>Q</LogicalResourceId>"))+ .body(containsString("<Action>Modify</Action>"));+ }++ @Test+ void parameterChangeFlippingConditionToTrue_isReportedAsAnAdd() {+ String stackName = "cs-diff-condition-add-stack";+ stacksToDelete.add(stackName);++ String template = """+ {+ "Parameters": {+ "Env": {"Type": "String"}+ },+ "Conditions": {+ "IsProd": {"Fn::Equals": [{"Ref": "Env"}, "prod"]}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Condition": "IsProd",+ "Properties": {"QueueName": "cs-diff-condition-add-queue"}+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Env")+ .formParam("Parameters.member.1.ParameterValue", "dev")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Q's definition text is unchanged, but Env flips IsProd from false to true - Q was never+ // created, and ExecuteChangeSet would create it now.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "condition-add-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Env")+ .formParam("Parameters.member.1.ParameterValue", "prod")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "condition-add-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>Q</LogicalResourceId>"))+ .body(containsString("<Action>Add</Action>"));+ }++ @Test+ void parameterChangeFlippingConditionToFalse_isReportedAsARemove() {+ String stackName = "cs-diff-condition-remove-stack";+ stacksToDelete.add(stackName);++ String template = """+ {+ "Parameters": {+ "Env": {"Type": "String"}+ },+ "Conditions": {+ "IsProd": {"Fn::Equals": [{"Ref": "Env"}, "prod"]}+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Condition": "IsProd",+ "Properties": {"QueueName": "cs-diff-condition-remove-queue"}+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Env")+ .formParam("Parameters.member.1.ParameterValue", "prod")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // Q's definition text is unchanged, but Env flips IsProd from true to false - Q was created,+ // and ExecuteChangeSet would delete it now.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "condition-remove-cs")+ .formParam("ChangeSetType", "UPDATE")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "Env")+ .formParam("Parameters.member.1.ParameterValue", "dev")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeChangeSet")+ .formParam("StackName", stackName)+ .formParam("ChangeSetName", "condition-remove-cs")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<LogicalResourceId>Q</LogicalResourceId>"))+ .body(containsString("<Action>Remove</Action>"));+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationServiceRollbackTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationServiceRollbackTest.javaindex df3b1cc453..ef5bbecd39 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationServiceRollbackTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationServiceRollbackTest.java@@ -10,6 +10,7 @@ import io.github.hectorvent.floci.core.storage.StorageFactory;import io.github.hectorvent.floci.services.cloudformation.model.Stack;import io.github.hectorvent.floci.services.cloudformation.model.StackResource;import io.github.hectorvent.floci.services.s3.S3Service;+import io.github.hectorvent.floci.services.ssm.SsmService;import org.junit.jupiter.api.BeforeEach;import org.junit.jupiter.api.Test;@@ -45,6 +46,7 @@ class CloudFormationServiceRollbackTest {service = new CloudFormationService(provisioner,mock(S3Service.class),+ mock(SsmService.class),new ObjectMapper(),config,mock(RegionResolver.class),diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationSsmParameterIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationSsmParameterIntegrationTest.javanew file mode 100644index 0000000000..99b0afe022--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationSsmParameterIntegrationTest.java@@ -0,0 +1,139 @@+package io.github.hectorvent.floci.services.cloudformation;++import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.not;++@QuarkusTest+class CloudFormationSsmParameterIntegrationTest {++ private static final String SSM_CONTENT_TYPE = "application/x-amz-json-1.1";++ @BeforeAll+ static void configureRestAssured() {+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @Test+ void createStack_resolvesSsmTypedParameterValue() {+ given()+ .header("X-Amz-Target", "AmazonSSM.PutParameter")+ .contentType(SSM_CONTENT_TYPE)+ .body("""+ {+ "Name": "/cfn/test/queue-suffix",+ "Value": "orders-primary",+ "Type": "String",+ "Overwrite": true+ }+ """)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ String template = """+ {+ "Parameters": {+ "QueueSuffix": {+ "Type": "AWS::SSM::Parameter::Value<String>",+ "Default": "/cfn/test/queue-suffix"+ }+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": { "Fn::Sub": "cfn-ssm-${QueueSuffix}" }+ }+ }+ },+ "Outputs": {+ "ResolvedSuffix": { "Value": { "Ref": "QueueSuffix" } }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", "ssm-param-stack")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", "ssm-param-stack")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"))+ .body(containsString("<OutputValue>orders-primary</OutputValue>"))+ .body(not(containsString("<OutputValue>/cfn/test/queue-suffix</OutputValue>")));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "GetQueueUrl")+ .formParam("QueueName", "cfn-ssm-orders-primary")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("cfn-ssm-orders-primary"));+ }++ @Test+ void createStack_missingSsmParameterFailsWithValidationError() {+ String template = """+ {+ "Parameters": {+ "MissingParam": {+ "Type": "AWS::SSM::Parameter::Value<String>",+ "Default": "/cfn/test/does-not-exist"+ }+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": { "Fn::Sub": "cfn-ssm-missing-${MissingParam}" }+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", "ssm-missing-param-stack")+ .formParam("TemplateBody", template)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", "ssm-missing-param-stack")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_FAILED</StackStatus>"))+ .body(containsString(+ "Unable to fetch parameters [/cfn/test/does-not-exist] from parameter store for this account"));+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationStackSetsIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationStackSetsIntegrationTest.javaindex ffb8bb7eba..6f3b0e3b33 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationStackSetsIntegrationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationStackSetsIntegrationTest.java@@ -345,6 +345,50 @@ class CloudFormationStackSetsIntegrationTest {.body(containsString("<Action>UPDATE</Action>"));}+ @Test+ void updateStackSetWithUsePreviousValuePreservesTheDeployedParameter() {+ // UpdateStackSet, unlike UpdateStack, resolves its Parameters through the single-arg+ // extractParameters() overload that has no previous-value map to consult. A member+ // with UsePreviousValue=true and no ParameterValue must still resolve to the StackSet's+ // currently deployed value, not an empty string that clobbers it.+ String setName = "upv-" + UUID.randomUUID().toString().substring(0, 8);+ String q1 = "upv-q1-" + UUID.randomUUID().toString().substring(0, 8);+ String original = "orig-" + UUID.randomUUID().toString().substring(0, 8);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStackSet")+ .formParam("StackSetName", setName)+ .formParam("TemplateBody", queueTemplate(q1))+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", original)+ .header("Authorization", auth(ADMIN, "cloudformation"))+ .when().post("/")+ .then().statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "UpdateStackSet")+ .formParam("StackSetName", setName)+ .formParam("TemplateBody", queueTemplate(q1))+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.UsePreviousValue", "true")+ .header("Authorization", auth(ADMIN, "cloudformation"))+ .when().post("/")+ .then().statusCode(200)+ .body(containsString("<OperationId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStackSet")+ .formParam("StackSetName", setName)+ .header("Authorization", auth(ADMIN, "cloudformation"))+ .when().post("/")+ .then().statusCode(200)+ .body(containsString("<ParameterValue>" + original + "</ParameterValue>"))+ .body(not(containsString("<ParameterValue></ParameterValue>")));+ }+@Testvoid stackSetErrorPaths() {String setName = "errset-" + UUID.randomUUID().toString().substring(0, 8);diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationUpdateRollbackParametersTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationUpdateRollbackParametersTest.javanew file mode 100644index 0000000000..da694aa683--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationUpdateRollbackParametersTest.java@@ -0,0 +1,204 @@+package io.github.hectorvent.floci.services.cloudformation;++import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;+import io.quarkus.test.junit.QuarkusTest;+import org.junit.jupiter.api.BeforeAll;+import org.junit.jupiter.api.Test;++import static io.restassured.RestAssured.given;+import static org.hamcrest.Matchers.containsString;+import static org.hamcrest.Matchers.not;++/**+ * When an update references a missing SSM parameter, resolution fails before any resource is+ * touched — but {@code executeTemplate} used to overwrite the stack's live parameter map with the+ * attempted (unresolved) values first. After the rollback, {@code DescribeStacks} must still+ * report the last successfully deployed parameter values, not the failed update's inputs.+ */+@QuarkusTest+class CloudFormationUpdateRollbackParametersTest {++ private static final String SSM_CONTENT_TYPE = "application/x-amz-json-1.1";++ @BeforeAll+ static void configureRestAssured() {+ RestAssuredJsonUtils.configureAwsContentTypes();+ }++ @Test+ void failedSsmParameterUpdate_retainsLastSuccessfulParameterValues() {+ given()+ .header("X-Amz-Target", "AmazonSSM.PutParameter")+ .contentType(SSM_CONTENT_TYPE)+ .body("""+ {+ "Name": "/cfn/test/rollback-suffix",+ "Value": "orders-primary",+ "Type": "String",+ "Overwrite": true+ }+ """)+ .when()+ .post("/")+ .then()+ .statusCode(200);++ String template = """+ {+ "Parameters": {+ "QueueSuffix": {+ "Type": "AWS::SSM::Parameter::Value<String>"+ }+ },+ "Resources": {+ "Q": {+ "Type": "AWS::SQS::Queue",+ "Properties": {+ "QueueName": {"Fn::Sub": "cfn-rollback-${QueueSuffix}"}+ }+ }+ }+ }+ """;++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", "ssm-rollback-stack")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "QueueSuffix")+ .formParam("Parameters.member.1.ParameterValue", "/cfn/test/rollback-suffix")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackId>"));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", "ssm-rollback-stack")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>CREATE_COMPLETE</StackStatus>"))+ .body(containsString("<ParameterValue>/cfn/test/rollback-suffix</ParameterValue>"));++ // Update references an SSM parameter name that was never put — resolution fails and the+ // update must roll back before any resource change is committed.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "UpdateStack")+ .formParam("StackName", "ssm-rollback-stack")+ .formParam("TemplateBody", template)+ .formParam("Parameters.member.1.ParameterKey", "QueueSuffix")+ .formParam("Parameters.member.1.ParameterValue", "/cfn/test/does-not-exist")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", "ssm-rollback-stack")+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>UPDATE_ROLLBACK_COMPLETE</StackStatus>"))+ .body(containsString("<ParameterValue>/cfn/test/rollback-suffix</ParameterValue>"))+ .body(not(containsString("<ParameterValue>/cfn/test/does-not-exist</ParameterValue>")));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DeleteStack")+ .formParam("StackName", "ssm-rollback-stack")+ .when()+ .post("/");+ }++ /**+ * When resource rollback itself also fails (not just the template update), the stack lands in+ * UPDATE_ROLLBACK_FAILED rather than UPDATE_ROLLBACK_COMPLETE. Parameter restoration must not be+ * skipped in that path — the last successfully deployed values are independent of whether the+ * resource rollback succeeded.+ */+ @Test+ void failedUpdateWithFailedResourceRollback_stillRetainsLastSuccessfulParameterValues() {+ String suffix = Long.toString(System.nanoTime(), 36);+ String stackName = "rollback-failed-params-stack-" + suffix;+ String parameterName = "/cfn/test/rollback-failed-params-" + suffix;+ String template = """+ {+ "Parameters": {+ "Suffix": {"Type": "String"}+ },+ "Resources": {+ "Parameter": {+ "Type": "AWS::SSM::Parameter",+ "Properties": {+ "Name": "%s",+ "Type": "String",+ "Value": {"Fn::Sub": "value-${Suffix}"}+ }+ }+ %s+ }+ }+ """;+ String initialTemplate = template.formatted(parameterName, "");+ String failingTemplate = template.formatted(parameterName, """+ ,+ "BadSecret": {+ "Type": "AWS::SecretsManager::Secret",+ "DependsOn": "Parameter",+ "Properties": {+ "Name": "rollback-failed-params-secret-%s",+ "SecretString": "explicit",+ "GenerateSecretString": {"PasswordLength": 32}+ }+ }+ """.formatted(suffix));++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "CreateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", initialTemplate)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "one")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ // The update changes Suffix (which the Parameter resource's Value depends on) and also+ // introduces BadSecret, which fails to create. Rollback must then also fail: rollback isn't+ // implemented for AWS::SSM::Parameter, so the stack lands in UPDATE_ROLLBACK_FAILED.+ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "UpdateStack")+ .formParam("StackName", stackName)+ .formParam("TemplateBody", failingTemplate)+ .formParam("Parameters.member.1.ParameterKey", "Suffix")+ .formParam("Parameters.member.1.ParameterValue", "two")+ .when()+ .post("/")+ .then()+ .statusCode(200);++ given()+ .contentType("application/x-www-form-urlencoded")+ .formParam("Action", "DescribeStacks")+ .formParam("StackName", stackName)+ .when()+ .post("/")+ .then()+ .statusCode(200)+ .body(containsString("<StackStatus>UPDATE_ROLLBACK_FAILED</StackStatus>"))+ .body(containsString("<ParameterValue>one</ParameterValue>"))+ .body(not(containsString("<ParameterValue>two</ParameterValue>")));+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProviderFrameworkTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProviderFrameworkTest.javanew file mode 100644index 0000000000..977b75a50a--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProviderFrameworkTest.java@@ -0,0 +1,196 @@+package io.github.hectorvent.floci.services.cloudformation;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.core.common.docker.ContainerReachableEndpoint;+import io.github.hectorvent.floci.services.cloudformation.model.StackResource;+import io.github.hectorvent.floci.services.lambda.LambdaService;+import io.github.hectorvent.floci.services.lambda.model.InvocationType;+import io.github.hectorvent.floci.services.lambda.model.InvokeResult;+import io.github.hectorvent.floci.services.lambda.model.LambdaFunction;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.time.Duration;+import java.util.Map;+import java.util.function.Function;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.ArgumentMatchers.any;+import static org.mockito.ArgumentMatchers.eq;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.when;++/**+ * Hermetic coverage for the CDK Provider-framework (two-phase async) custom-resource path. The+ * backing {@code framework.onEvent} Lambda is mocked so that — like the real framework — it returns+ * WITHOUT PUTting and the ResponseURL callback lands asynchronously (as the waiter state machine's+ * {@code framework.isComplete} / {@code framework.onTimeout} would drive it). No Docker / Step+ * Functions involved: this isolates the provisioner's detect-and-wait logic.+ */+class CustomResourceProviderFrameworkTest {++ private static final String SERVICE_TOKEN =+ "arn:aws:lambda:us-east-1:000000000000:function:framework-onEvent";+ private static final String WAITER_ARN =+ "arn:aws:states:us-east-1:000000000000:stateMachine:waiter";++ private final ObjectMapper mapper = new ObjectMapper();+ private LambdaService lambdaService;+ private ProviderFrameworkDetector detector;+ private CustomResourceResponseStore store;+ private CloudFormationResourceProvisioner provisioner;++ @BeforeEach+ void setUp() {+ lambdaService = mock(LambdaService.class);+ detector = new ProviderFrameworkDetector(lambdaService);+ store = new CustomResourceResponseStore(detector);+ ContainerReachableEndpoint endpoint = mock(ContainerReachableEndpoint.class);+ when(endpoint.baseUrl()).thenReturn("http://floci:4566");++ provisioner = new CloudFormationResourceProvisioner(+ null, null, null, null, lambdaService, null, null, null, null, null,+ null, null, null, null, null, null, mapper, store, endpoint, null, null, null, null, null, null, null, null, null, null, null, null,+ null, null,+ new io.github.hectorvent.floci.services.cloudformation.provisioners.CloudFormationResourceRegistry(java.util.List.of()),+ mock(io.github.hectorvent.floci.config.EmulatorConfig.class));+ }++ private CloudFormationTemplateEngine engine() {+ return new CloudFormationTemplateEngine("000000000000", "us-east-1", "my-stack",+ "stack/id", Map.of(), Map.of(), Map.of(), Map.of(), Map.of(), mapper,+ (Function<String, String>) name -> null);+ }++ private ObjectNode props() {+ ObjectNode props = mapper.createObjectNode();+ props.put("ServiceToken", SERVICE_TOKEN);+ return props;+ }++ /** Registers the ServiceToken as a framework.onEvent Lambda: env carries the two async markers. */+ private void stubFrameworkOnEvent() {+ LambdaFunction fn = new LambdaFunction();+ fn.setFunctionName("framework-onEvent");+ fn.setFunctionArn(SERVICE_TOKEN);+ fn.setEnvironment(Map.of(+ "USER_ON_EVENT_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:onEvent",+ "USER_IS_COMPLETE_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:isComplete",+ "WAITER_STATE_MACHINE_ARN", WAITER_ARN));+ when(lambdaService.getFunction(eq("us-east-1"), eq(SERVICE_TOKEN))).thenReturn(fn);+ }++ /** Registers the ServiceToken as an onEvent-ONLY provider: no isComplete/waiter markers. */+ private void stubOnEventOnly() {+ LambdaFunction fn = new LambdaFunction();+ fn.setFunctionName("framework-onEvent");+ fn.setFunctionArn(SERVICE_TOKEN);+ fn.setEnvironment(Map.of(+ "USER_ON_EVENT_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:onEvent"));+ when(lambdaService.getFunction(eq("us-east-1"), eq(SERVICE_TOKEN))).thenReturn(fn);+ }++ /** onEvent returns without PUTting; a background PUT lands after {@code delayMs}, as the waiter would. */+ private void stubDeferredPut(String status, String physicalId, Map<String, String> data, long delayMs) {+ when(lambdaService.invoke(any(), eq(SERVICE_TOKEN), any(), eq(InvocationType.RequestResponse)))+ .thenAnswer(inv -> {+ String token = tokenOf(inv.getArgument(2));+ Thread t = new Thread(() -> {+ try {+ Thread.sleep(delayMs);+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ return;+ }+ ObjectNode response = mapper.createObjectNode();+ response.put("Status", status);+ if (physicalId != null) {+ response.put("PhysicalResourceId", physicalId);+ }+ if (data != null) {+ ObjectNode dataNode = response.putObject("Data");+ data.forEach(dataNode::put);+ }+ if (!"SUCCESS".equals(status)) {+ response.put("Reason", "onTimeout: resource never became complete");+ }+ store.complete(token, response);+ }, "deferred-put");+ t.setDaemon(true);+ t.start();+ return new InvokeResult(200, null, "null".getBytes(), null, "req-1");+ });+ }++ private String tokenOf(byte[] payload) throws Exception {+ JsonNode event = mapper.readTree(payload);+ String responseUrl = event.get("ResponseURL").asText();+ return responseUrl.substring(responseUrl.lastIndexOf('/') + 1);+ }++ @Test+ void frameworkOnEventAwaitsAsyncPutAndPropagatesData() {+ stubFrameworkOnEvent();+ stubDeferredPut("SUCCESS", "account-provisioned", Map.of("AccountId", "111122223333"), 150);++ StackResource r = provisioner.provision("CreateAccounts", "Custom::CreateOrganizationAccounts",+ props(), engine(), "us-east-1", "000000000000", "my-stack");++ assertEquals("CREATE_COMPLETE", r.getStatus());+ assertEquals("account-provisioned", r.getPhysicalId());+ assertEquals("111122223333", r.getAttributes().get("AccountId"));+ }++ @Test+ void onEventOnlyProviderStillSucceedsViaSynchronousPut() {+ stubOnEventOnly();+ // Synchronous PUT during the invoke, exactly as an onEvent-only framework.onEvent does.+ when(lambdaService.invoke(any(), eq(SERVICE_TOKEN), any(), eq(InvocationType.RequestResponse)))+ .thenAnswer(inv -> {+ String token = tokenOf(inv.getArgument(2));+ ObjectNode response = mapper.createObjectNode();+ response.put("Status", "SUCCESS");+ response.put("PhysicalResourceId", "moved");+ store.complete(token, response);+ return new InvokeResult(200, null, "null".getBytes(), null, "req-1");+ });++ StackResource r = provisioner.provision("MoveAccounts", "Custom::MoveAccounts",+ props(), engine(), "us-east-1", "000000000000", "my-stack");++ assertEquals("CREATE_COMPLETE", r.getStatus());+ assertEquals("moved", r.getPhysicalId());+ }++ @Test+ void asyncPutThatNeverArrivesFailsCleanlyWithinBudget() {+ detector.setAsyncCustomResourceTimeoutForTesting(Duration.ofMillis(300));+ stubFrameworkOnEvent();+ // onEvent returns without PUTting and nothing ever calls back.+ when(lambdaService.invoke(any(), eq(SERVICE_TOKEN), any(), eq(InvocationType.RequestResponse)))+ .thenReturn(new InvokeResult(200, null, "null".getBytes(), null, "req-1"));++ StackResource r = provisioner.provision("CreateAccounts", "Custom::CreateOrganizationAccounts",+ props(), engine(), "us-east-1", "000000000000", "my-stack");++ assertEquals("CREATE_FAILED", r.getStatus());+ assertTrue(r.getStatusReason().contains("Timed out"),+ "expected a bounded timeout failure, got: " + r.getStatusReason());+ }++ @Test+ void waiterOnTimeoutFailurePropagatesAsResourceFailure() {+ stubFrameworkOnEvent();+ stubDeferredPut("FAILED", null, null, 150);++ StackResource r = provisioner.provision("CreateAccounts", "Custom::CreateOrganizationAccounts",+ props(), engine(), "us-east-1", "000000000000", "my-stack");++ assertEquals("CREATE_FAILED", r.getStatus());+ assertTrue(r.getStatusReason().contains("onTimeout"),+ "expected the onTimeout FAILED reason to surface, got: " + r.getStatusReason());+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProvisionerTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProvisionerTest.javaindex c75a0d9a53..32c6e4265f 100644--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProvisionerTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceProvisionerTest.java@@ -42,7 +42,7 @@ class CustomResourceProvisionerTest {@BeforeEachvoid setUp() {lambdaService = mock(LambdaService.class);- store = new CustomResourceResponseStore();+ store = new CustomResourceResponseStore(new ProviderFrameworkDetector(lambdaService));ContainerReachableEndpoint endpoint = mock(ContainerReachableEndpoint.class);when(endpoint.baseUrl()).thenReturn("http://floci:4566");diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStoreTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStoreTest.javanew file mode 100644index 0000000000..bf9fd971e3--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CustomResourceResponseStoreTest.java@@ -0,0 +1,180 @@+package io.github.hectorvent.floci.services.cloudformation;++import com.fasterxml.jackson.databind.JsonNode;+import com.fasterxml.jackson.databind.ObjectMapper;+import com.fasterxml.jackson.databind.node.ObjectNode;+import io.github.hectorvent.floci.services.lambda.LambdaService;+import org.junit.jupiter.api.Test;++import java.time.Duration;+import java.util.concurrent.TimeoutException;++import static org.junit.jupiter.api.Assertions.assertDoesNotThrow;+import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertThrows;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.Mockito.mock;++/**+ * The await budget is an IDLE timeout, not a total one.+ *+ * <p>A CDK provider-framework waiter drives {@code framework.isComplete} on a fixed cadence and only+ * PUTs to the ResponseURL once the underlying work finishes. How long that takes is a function of+ * the work, not of the emulator: {@code Custom::CreateOrganizationAccounts} creates exactly one+ * account per poll, so a 15-account org needs ~15 polls where a 6-account org needs ~3. A total-time+ * budget therefore has to be re-tuned every time a config set grows, and when it is exceeded the+ * Lambda is guillotined mid-success — the accounts exist, but the resource reports CREATE_FAILED and+ * the stack rolls back.+ *+ * <p>Measuring idleness instead makes the budget proportional to the work without the CloudFormation+ * layer needing to know what the work IS: every poll is liveness, so progress buys unbounded+ * wall-clock while a genuinely hung resource still dies on schedule.+ */+class CustomResourceResponseStoreTest {++ private final CustomResourceResponseStore store = new CustomResourceResponseStore(+ new ProviderFrameworkDetector(mock(LambdaService.class)));+ private final ObjectMapper mapper = new ObjectMapper();++ private ObjectNode success() {+ ObjectNode node = mapper.createObjectNode();+ node.put("Status", "SUCCESS");+ return node;+ }++ /** Polls {@code count} times at 100ms, then optionally PUTs. Mirrors the waiter's cadence. */+ private Thread waiterPolling(String token, int count, boolean thenComplete) {+ Thread t = new Thread(() -> {+ try {+ for (int i = 0; i < count; i++) {+ Thread.sleep(100);+ store.touch(token);+ }+ if (thenComplete) {+ store.complete(token, success());+ }+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ });+ t.setDaemon(true);+ t.start();+ return t;+ }++ @Test+ void awaitOutlivesItsBudgetWhileTouchesKeepArriving() throws Exception {+ String token = store.register();+ // ~900ms of polling against a 300ms budget: three times over, the way 15 accounts run three+ // times past a budget sized for 6.+ Thread waiter = waiterPolling(token, 9, true);++ JsonNode response = store.await(token, Duration.ofMillis(300));++ assertEquals("SUCCESS", response.get("Status").asText());+ waiter.join(2_000);+ }++ @Test+ void awaitStillTimesOutOnceTheTouchesStop() throws Exception {+ String token = store.register();+ // Three polls, then the resource hangs. The budget must still fire from the LAST poll.+ Thread waiter = waiterPolling(token, 3, false);++ assertThrows(TimeoutException.class, () -> store.await(token, Duration.ofMillis(300)));+ waiter.join(2_000);+ }++ @Test+ void awaitTimesOutWhenNothingEverHappens() {+ String token = store.register();++ assertThrows(TimeoutException.class, () -> store.await(token, Duration.ofMillis(150)));+ }++ @Test+ void awaitReturnsAResponseThatArrivedBeforeTheBudgetWasChecked() throws Exception {+ // A synchronous handler PUTs from inside the invoke the provisioner is blocked on, so its+ // response can already be here by the time await is entered. Whatever the clock says, a+ // response that arrived is a response — the future outranks the budget.+ String token = store.register();+ store.complete(token, success());++ JsonNode response = store.await(token, Duration.ZERO);++ assertEquals("SUCCESS", response.get("Status").asText());+ }++ @Test+ void awaitBudgetsIdlenessFromWhenTheCallerStartsWaitingNotFromRegistration() throws Exception {+ String token = store.register();+ // The token is registered before the invoke, and a slow handler — a cold container start, or+ // two of them nested — can burn longer than the whole budget before await is ever reached.+ // That is the handler working, not the resource going idle, so it must not be charged here.+ Thread.sleep(400);++ Thread waiter = new Thread(() -> {+ try {+ Thread.sleep(100);+ store.complete(token, success());+ } catch (InterruptedException e) {+ Thread.currentThread().interrupt();+ }+ });+ waiter.setDaemon(true);+ waiter.start();++ JsonNode response = store.await(token, Duration.ofMillis(300));++ assertEquals("SUCCESS", response.get("Status").asText());+ waiter.join(2_000);+ }++ @Test+ void touchForAnUnknownTokenIsIgnored() {+ // Polls can outlive their await (timeout, rollback). Liveness for a token nobody is waiting+ // on is not an error — it is the normal tail of a resource that already failed.+ assertDoesNotThrow(() -> store.touch("no-such-token"));+ }++ @Test+ void toleratedWaitSliceTimeoutIsLogged() throws Exception {+ // Every slice of the poll loop below the outer budget hits an internal TimeoutException that+ // is intentionally swallowed and retried. Per the repo's logging convention, a caught+ // exception that is deliberately tolerated must still be logged (at trace level, since this+ // is the normal, expected tail of every wait) rather than disappearing silently.+ java.util.logging.Logger julLogger =+ java.util.logging.Logger.getLogger(CustomResourceResponseStore.class.getName());+ julLogger.setLevel(java.util.logging.Level.ALL);+ java.util.List<java.util.logging.LogRecord> records = new java.util.ArrayList<>();+ java.util.logging.Handler handler = new java.util.logging.Handler() {+ @Override+ public void publish(java.util.logging.LogRecord record) {+ records.add(record);+ }++ @Override+ public void flush() {+ }++ @Override+ public void close() {+ }+ };+ handler.setLevel(java.util.logging.Level.ALL);+ julLogger.addHandler(handler);+ try {+ String token = store.register();+ Thread waiter = waiterPolling(token, 9, true);++ store.await(token, Duration.ofMillis(300));+ waiter.join(2_000);+ } finally {+ julLogger.removeHandler(handler);+ }++ assertTrue(records.stream().anyMatch(r -> r.getMessage() != null+ && r.getMessage().toLowerCase().contains("wait slice")),+ "expected a log record for a tolerated wait-slice timeout, got: " + records);+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/ProviderFrameworkDetectorTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/ProviderFrameworkDetectorTest.javanew file mode 100644index 0000000000..ed5a217fc5--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/ProviderFrameworkDetectorTest.java@@ -0,0 +1,119 @@+package io.github.hectorvent.floci.services.cloudformation;++import io.github.hectorvent.floci.services.lambda.LambdaService;+import io.github.hectorvent.floci.services.lambda.model.LambdaFunction;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.time.Duration;+import java.util.Map;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.junit.jupiter.api.Assertions.assertFalse;+import static org.junit.jupiter.api.Assertions.assertTrue;+import static org.mockito.ArgumentMatchers.any;+import static org.mockito.ArgumentMatchers.eq;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.when;++/**+ * The two-phase CDK Provider framework is recognised from the {@code framework.onEvent} Lambda's+ * environment: only that shape PUTs asynchronously, and only it earns the longer idle budget.+ */+class ProviderFrameworkDetectorTest {++ private static final String REGION = "us-east-1";+ private static final String SERVICE_TOKEN =+ "arn:aws:lambda:us-east-1:000000000000:function:framework-onEvent";+ private static final Duration SYNCHRONOUS = Duration.ofSeconds(10);++ private LambdaService lambdaService;+ private ProviderFrameworkDetector detector;++ @BeforeEach+ void setUp() {+ lambdaService = mock(LambdaService.class);+ detector = new ProviderFrameworkDetector(lambdaService);+ }++ private void stubEnvironment(Map<String, String> env) {+ LambdaFunction fn = new LambdaFunction();+ fn.setFunctionName("framework-onEvent");+ fn.setFunctionArn(SERVICE_TOKEN);+ fn.setEnvironment(env);+ when(lambdaService.getFunction(eq(REGION), eq(SERVICE_TOKEN))).thenReturn(fn);+ }++ private Map<String, String> frameworkEnv() {+ return Map.of(+ "USER_ON_EVENT_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:onEvent",+ "USER_IS_COMPLETE_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:isComplete",+ "WAITER_STATE_MACHINE_ARN", "arn:aws:states:us-east-1:000000000000:stateMachine:waiter");+ }++ @Test+ void bothWaiterMarkersPresentMeansProviderFramework() {+ stubEnvironment(frameworkEnv());++ assertTrue(detector.isProviderFrameworkOnEvent(SERVICE_TOKEN, REGION));+ }++ @Test+ void onEventOnlyProviderIsNotProviderFramework() {+ stubEnvironment(Map.of(+ "USER_ON_EVENT_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:onEvent"));++ assertFalse(detector.isProviderFrameworkOnEvent(SERVICE_TOKEN, REGION));+ }++ @Test+ void isCompleteWithoutWaiterIsNotProviderFramework() {+ stubEnvironment(Map.of(+ "USER_IS_COMPLETE_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:isComplete"));++ assertFalse(detector.isProviderFrameworkOnEvent(SERVICE_TOKEN, REGION));+ }++ @Test+ void blankMarkerIsNotProviderFramework() {+ stubEnvironment(Map.of(+ "USER_IS_COMPLETE_FUNCTION_ARN", "arn:aws:lambda:us-east-1:000000000000:function:isComplete",+ "WAITER_STATE_MACHINE_ARN", " "));++ assertFalse(detector.isProviderFrameworkOnEvent(SERVICE_TOKEN, REGION));+ }++ @Test+ void functionWithNoEnvironmentIsNotProviderFramework() {+ stubEnvironment(null);++ assertFalse(detector.isProviderFrameworkOnEvent(SERVICE_TOKEN, REGION));+ }++ /** Detection is best-effort: an unresolvable ServiceToken must not fail the resource. */+ @Test+ void unresolvableFunctionIsTreatedAsSynchronousHandler() {+ when(lambdaService.getFunction(any(), any()))+ .thenThrow(new RuntimeException("ResourceNotFoundException"));++ assertFalse(detector.isProviderFrameworkOnEvent(SERVICE_TOKEN, REGION));+ assertEquals(SYNCHRONOUS, detector.responseTimeout(SERVICE_TOKEN, REGION, SYNCHRONOUS));+ }++ @Test+ void providerFrameworkGetsTheAsyncBudgetAndOthersTheSynchronousOne() {+ stubEnvironment(frameworkEnv());+ assertEquals(Duration.ofMinutes(3), detector.responseTimeout(SERVICE_TOKEN, REGION, SYNCHRONOUS));++ stubEnvironment(Map.of("USER_ON_EVENT_FUNCTION_ARN", "arn:aws:lambda:us-east-1:0:function:onEvent"));+ assertEquals(SYNCHRONOUS, detector.responseTimeout(SERVICE_TOKEN, REGION, SYNCHRONOUS));+ }++ @Test+ void testSeamShortensTheAsyncBudget() {+ stubEnvironment(frameworkEnv());+ detector.setAsyncCustomResourceTimeoutForTesting(Duration.ofMillis(300));++ assertEquals(Duration.ofMillis(300), detector.responseTimeout(SERVICE_TOKEN, REGION, SYNCHRONOUS));+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/lambda/LambdaArnInvocationAccountTest.java b/src/test/java/io/github/hectorvent/floci/services/lambda/LambdaArnInvocationAccountTest.javaindex 8a5008fc64..9e9ae6f115 100644--- a/src/test/java/io/github/hectorvent/floci/services/lambda/LambdaArnInvocationAccountTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/lambda/LambdaArnInvocationAccountTest.java@@ -67,6 +67,7 @@ class LambdaArnInvocationAccountTest {null,null,null,+ null,null);InvokeResult result = service.invokeArn(functionArn, "{}".getBytes(), InvocationType.Event);@@ -127,6 +128,7 @@ class LambdaArnInvocationAccountTest {null,null,null,+ null,null);InvokeResult versionResult = service.invokeArn(@@ -265,6 +267,7 @@ class LambdaArnInvocationAccountTest {null,null,null,+ null,null);}}diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorArrayParamsTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorArrayParamsTest.javaindex dd745b7882..070038e6b2 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorArrayParamsTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorArrayParamsTest.java@@ -19,7 +19,7 @@ class AslExecutorArrayParamsTest {private AslExecutor newExecutor() {return new AslExecutor(null, null, null, null, null, null, null, null,- null, null, null, null, null, mapper, null, null, null, null);+ null, null, null, null, null, mapper, null, null, null, null, null);}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCatchTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCatchTest.javaindex 08d6f9e334..03fe2aea8a 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCatchTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCatchTest.java@@ -93,7 +93,7 @@ class AslExecutorCatchTest {mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),objectMapper,new JsonataEvaluator(objectMapper),- mock(Instance.class), mock(EmulatorConfig.class), vertx);+ mock(Instance.class), mock(EmulatorConfig.class), vertx, null);}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCustomResourceLivenessTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCustomResourceLivenessTest.javanew file mode 100644index 0000000000..054c6256f2--- /dev/null+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorCustomResourceLivenessTest.java@@ -0,0 +1,172 @@+package io.github.hectorvent.floci.services.stepfunctions;++import com.fasterxml.jackson.databind.ObjectMapper;+import io.github.hectorvent.floci.config.EmulatorConfig;+import io.github.hectorvent.floci.core.common.CustomResourceLiveness;+import io.github.hectorvent.floci.services.dynamodb.DynamoDbJsonHandler;+import io.github.hectorvent.floci.services.dynamodb.DynamoDbService;+import io.github.hectorvent.floci.services.lambda.LambdaExecutorService;+import io.github.hectorvent.floci.services.lambda.LambdaFunctionStore;+import io.github.hectorvent.floci.services.lambda.model.InvocationType;+import io.github.hectorvent.floci.services.lambda.model.InvokeResult;+import io.github.hectorvent.floci.services.lambda.model.LambdaFunction;+import io.github.hectorvent.floci.services.s3.S3Service;+import io.github.hectorvent.floci.services.sqs.SqsJsonHandler;+import io.github.hectorvent.floci.services.stepfunctions.model.Execution;+import io.github.hectorvent.floci.services.stepfunctions.model.HistoryEvent;+import io.github.hectorvent.floci.services.stepfunctions.model.StateMachine;+import io.quarkus.test.junit.QuarkusTest;+import io.vertx.mutiny.core.Vertx;+import jakarta.enterprise.inject.Instance;+import jakarta.inject.Inject;+import org.junit.jupiter.api.BeforeEach;+import org.junit.jupiter.api.Test;++import java.util.ArrayList;+import java.util.List;+import java.util.Optional;++import static org.junit.jupiter.api.Assertions.assertEquals;+import static org.mockito.ArgumentMatchers.any;+import static org.mockito.ArgumentMatchers.eq;+import static org.mockito.Mockito.mock;+import static org.mockito.Mockito.never;+import static org.mockito.Mockito.verify;+import static org.mockito.Mockito.when;++/**+ * A CDK provider-framework waiter's {@code framework.isComplete} poll is itself a Lambda Task in+ * the state machine Step Functions runs on our behalf, so it lands in {@link+ * AslExecutor#invokeResource} rather than {@link+ * io.github.hectorvent.floci.services.lambda.LambdaService#invoke}. Greptile/Copilot both flagged+ * that this path never told {@link CustomResourceLiveness} about the poll, so a long-but-progressing+ * custom resource's idle budget kept ticking down as if no polls were arriving at all.+ */+@QuarkusTest+class AslExecutorCustomResourceLivenessTest {++ private static final String REGION = "us-east-2";+ private static final String ACCOUNT = "000000000000";+ private static final String FUNCTION_NAME = "framework-onEvent";+ private static final String FUNCTION_ARN =+ "arn:aws:lambda:%s:%s:function:%s".formatted(REGION, ACCOUNT, FUNCTION_NAME);+ private static final String TOKEN = "6ef77fe2-38c8-41b0-bec6-68879687fbc8";++ private final ObjectMapper objectMapper = new ObjectMapper();+ private CustomResourceLiveness customResourceLiveness;+ private AslExecutor executor;++ @Inject+ Vertx vertx;++ @BeforeEach+ void setUp() throws Exception {+ LambdaExecutorService lambdaExecutor = mock(LambdaExecutorService.class);+ LambdaFunctionStore functionStore = mock(LambdaFunctionStore.class);+ LambdaFunction function = new LambdaFunction();+ function.setFunctionName(FUNCTION_NAME);+ function.setFunctionArn(FUNCTION_ARN);+ customResourceLiveness = mock(CustomResourceLiveness.class);++ when(functionStore.get(REGION, FUNCTION_NAME)).thenReturn(Optional.of(function));+ when(lambdaExecutor.invoke(eq(function), any(byte[].class), eq(InvocationType.RequestResponse)))+ .thenReturn(new InvokeResult(200, null, "{}".getBytes(), null, "poll-request"));++ executor = new AslExecutor(+ lambdaExecutor,+ functionStore,+ mock(DynamoDbService.class),+ mock(DynamoDbJsonHandler.class),+ mock(SqsJsonHandler.class),+ mock(io.github.hectorvent.floci.services.cloudformation.CloudFormationQueryHandler.class),+ mock(io.github.hectorvent.floci.services.ec2.Ec2Service.class),+ mock(S3Service.class),+ mock(io.github.hectorvent.floci.services.ecs.EcsService.class),+ mock(io.github.hectorvent.floci.services.ecs.EcsJsonHandler.class),+ mock(io.github.hectorvent.floci.services.eventbridge.EventBridgeHandler.class),+ mock(io.github.hectorvent.floci.services.scheduler.SchedulerService.class),+ mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),+ objectMapper,+ new JsonataEvaluator(objectMapper),+ mock(Instance.class), mock(EmulatorConfig.class), vertx, customResourceLiveness);+ }++ @Test+ void waiterPollThroughDirectFunctionArnTouchesTheEmbeddedCallbackToken() throws Exception {+ run("""+ {+ "StartAt": "Poll",+ "States": {+ "Poll": {+ "Type": "Task",+ "Resource": "%s",+ "End": true+ }+ }+ }+ """.formatted(FUNCTION_ARN),+ "{\"RequestType\":\"Create\",\"ResponseURL\":\"http://host:4566/cfn-response/" + TOKEN + "\"}");++ verify(customResourceLiveness).touch(TOKEN);+ }++ @Test+ void waiterPollThroughOptimizedIntegrationTouchesTheEmbeddedCallbackToken() throws Exception {+ run("""+ {+ "StartAt": "Poll",+ "States": {+ "Poll": {+ "Type": "Task",+ "Resource": "arn:aws:states:::lambda:invoke",+ "Parameters": {"FunctionName": "%s", "Payload.$": "$"},+ "End": true+ }+ }+ }+ """.formatted(FUNCTION_ARN),+ "{\"RequestType\":\"Create\",\"ResponseURL\":\"http://host:4566/cfn-response/" + TOKEN + "\"}");++ verify(customResourceLiveness).touch(TOKEN);+ }++ @Test+ void ordinaryLambdaInvokeWithNoCallbackUrlNeverTouchesLiveness() throws Exception {+ run("""+ {+ "StartAt": "Poll",+ "States": {+ "Poll": {+ "Type": "Task",+ "Resource": "%s",+ "End": true+ }+ }+ }+ """.formatted(FUNCTION_ARN),+ "{\"Records\":[{\"body\":\"hello\"}]}");++ verify(customResourceLiveness, never()).touch(org.mockito.ArgumentMatchers.anyString());+ }++ private void run(String definition, String input) {+ StateMachine stateMachine = new StateMachine();+ stateMachine.setName("waiter-liveness");+ stateMachine.setStateMachineArn(+ "arn:aws:states:%s:%s:stateMachine:waiter-liveness".formatted(REGION, ACCOUNT));+ stateMachine.setRoleArn("arn:aws:iam::%s:role/test-role".formatted(ACCOUNT));+ stateMachine.setDefinition(definition);++ Execution execution = new Execution();+ execution.setName("waiter-liveness-execution");+ execution.setExecutionArn("arn:aws:states:%s:%s:execution:waiter-liveness:run-1"+ .formatted(REGION, ACCOUNT));+ execution.setStateMachineArn(stateMachine.getStateMachineArn());+ execution.setInput(input);++ List<HistoryEvent> history = new ArrayList<>();+ executor.executeSync(stateMachine, execution, history, (updated, events) -> {+ });+ assertEquals("SUCCEEDED", execution.getStatus());+ }+}diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEcsRunTaskModeTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEcsRunTaskModeTest.javaindex 78d3bc25be..f603f272ec 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEcsRunTaskModeTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorEcsRunTaskModeTest.java@@ -76,6 +76,7 @@ class AslExecutorEcsRunTaskModeTest {new JsonataEvaluator(objectMapper),mock(Instance.class),mock(EmulatorConfig.class),+ null,null);}diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorErrorTerminationTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorErrorTerminationTest.javaindex 4e42b7857a..79fa37e612 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorErrorTerminationTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorErrorTerminationTest.java@@ -87,7 +87,8 @@ class AslExecutorErrorTerminationTest {mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),objectMapper,new CrashingJsonataEvaluator(objectMapper),- mock(Instance.class), mock(EmulatorConfig.class), vertx);+ mock(Instance.class), mock(EmulatorConfig.class), vertx,+ mock(io.github.hectorvent.floci.core.common.CustomResourceLiveness.class));}private static final String PASS_WITH_JSONATA_OUTPUT = """diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorHttpInvokeTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorHttpInvokeTest.javaindex 4d51ec9ef9..68becf0ee5 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorHttpInvokeTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorHttpInvokeTest.java@@ -176,7 +176,8 @@ class AslExecutorHttpInvokeTest {new JsonataEvaluator(objectMapper),mock(Instance.class),emulatorConfig,- vertx);+ vertx,+ null);}private void recordRequest(final RoutingContext ctx, String body) {diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorIntrinsicContextTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorIntrinsicContextTest.javaindex 5ab5404878..824faaed90 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorIntrinsicContextTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorIntrinsicContextTest.java@@ -23,7 +23,7 @@ class AslExecutorIntrinsicContextTest {/** Only ObjectMapper is exercised by the path/intrinsic resolution methods under test. */private AslExecutor newExecutor() {return new AslExecutor(null, null, null, null, null, null, null, null,- null, null, null, null, null, mapper, null, null, null, null);+ null, null, null, null, null, mapper, null, null, null, null, null);}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorJsonMergeTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorJsonMergeTest.javaindex d9d12037d7..32bf400c18 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorJsonMergeTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorJsonMergeTest.java@@ -55,6 +55,7 @@ class AslExecutorJsonMergeTest {new JsonataEvaluator(mapper),mock(Instance.class),mock(EmulatorConfig.class),+ null,null);}diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorLambdaInvokeResultTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorLambdaInvokeResultTest.javaindex 1bb328834b..2aa8306934 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorLambdaInvokeResultTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorLambdaInvokeResultTest.java@@ -91,7 +91,7 @@ class AslExecutorLambdaInvokeResultTest {mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),objectMapper,new JsonataEvaluator(objectMapper),- mock(Instance.class), mock(EmulatorConfig.class), vertx);+ mock(Instance.class), mock(EmulatorConfig.class), vertx, null);}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorMockedResponsesTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorMockedResponsesTest.javaindex 3b872e6535..63793084b9 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorMockedResponsesTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorMockedResponsesTest.java@@ -64,7 +64,7 @@ class AslExecutorMockedResponsesTest {mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),objectMapper,new JsonataEvaluator(objectMapper),- mock(Instance.class), mock(EmulatorConfig.class), vertx);+ mock(Instance.class), mock(EmulatorConfig.class), vertx, null);}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorPathIntrinsicsTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorPathIntrinsicsTest.javaindex cada8a66b8..aab9644436 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorPathIntrinsicsTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorPathIntrinsicsTest.java@@ -49,6 +49,7 @@ class AslExecutorPathIntrinsicsTest {new JsonataEvaluator(mapper),mock(Instance.class),mock(EmulatorConfig.class),+ null,null);}diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorResultWriterTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorResultWriterTest.javaindex 61abe6f70b..8e4cc4fd79 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorResultWriterTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorResultWriterTest.java@@ -75,6 +75,7 @@ class AslExecutorResultWriterTest {new JsonataEvaluator(mapper),sfnService,mock(EmulatorConfig.class),+ null,null);sm = new StateMachine();diff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorRetryTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorRetryTest.javaindex 494a8104e5..9b6f7c1e50 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorRetryTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorRetryTest.java@@ -81,7 +81,7 @@ class AslExecutorRetryTest {mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),objectMapper,new JsonataEvaluator(objectMapper),- mock(Instance.class), mock(EmulatorConfig.class), vertx);+ mock(Instance.class), mock(EmulatorConfig.class), vertx, null);}@Testdiff --git a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorTerminalStateTest.java b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorTerminalStateTest.javaindex c61dd21ef8..77485f8235 100644--- a/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorTerminalStateTest.java+++ b/src/test/java/io/github/hectorvent/floci/services/stepfunctions/AslExecutorTerminalStateTest.java@@ -65,7 +65,7 @@ class AslExecutorTerminalStateTest {mock(io.github.hectorvent.floci.services.scheduler.SchedulerController.class),objectMapper,new JsonataEvaluator(objectMapper),- mock(Instance.class), mock(EmulatorConfig.class), vertx);+ mock(Instance.class), mock(EmulatorConfig.class), vertx, null);}@Test