Skip to content

armgit / floci

Public Imported from GitHub
0

Light, fluffy, and always free - The AWS Local Emulator alternative

feat(cloudformation): provision Scheduler schedules (#4834)

XuJianf6c8ebd5a0d1
diff --git a/docs/services/cloudformation.md b/docs/services/cloudformation.md
index 93144da26c..faffdf1914 100644
--- a/docs/services/cloudformation.md
+++ b/docs/services/cloudformation.md
@@ -178,7 +178,7 @@ cross-resource references.
| Cognito | `UserPool` (`ProviderURL` is the local issuer of the tokens Floci mints, `<base-url>/<pool id>`), `UserPoolClient`, `UserPoolDomain`, `UserPoolGroup`, `UserPoolResourceServer`, `UserPoolUser`, `UserPoolUserToGroupAttachment` |
| ACM | `Certificate` |
| EventBridge | `Rule`, `EventBus`, `EventBusPolicy`, `ApiDestination`, `Archive` (`KmsKeyIdentifier` is ignored) |
-| EventBridge Scheduler | `ScheduleGroup` |
+| EventBridge Scheduler | `Schedule`, `ScheduleGroup` |
| Transfer Family | `Server` (management plane only; Ref returns the server ARN; Arn, ServerId and State attributes supported; AS2 managed egress IPs are not modeled; Domain replacement and IdentityProviderType changes are not supported), `User` (management plane only; Ref returns the user ARN; Arn attribute supported; Policy and PosixProfile are not supported; a ServerId or UserName change replaces the user) |
| Backup | `BackupVault` |
| GuardDuty | `MalwareProtectionPlan` (management plane only: no scan runs, so the plan stays ACTIVE; Ref returns the plan id; a BucketName change is rejected) |
@@ -212,6 +212,33 @@ Set `floci.services.cloudformation.allow-stub-unsupported-resource-types` to `fa
resource instead: it reaches `CREATE_FAILED` and the stack rolls back. Use it in a pipeline that
must not pass over a resource it never got.
+## EventBridge Scheduler Schedules
+
+`AWS::Scheduler::Schedule` creates a real schedule. `Ref` returns its name and `Fn::GetAtt Arn`
+returns its ARN. A group move preserves the name, removes the old group address, and restores the
+original configuration if the stack update rolls back.
+
+`Fn::If` values that select `AWS::NoValue` omit optional schedule properties and nested target
+or time-window fields. `ScheduleExpression`, `Target`, `FlexibleTimeWindow`, and their required
+fields still must be present and valid.
+
+Floci uses its main engine's bounded replacement-cleanup policy for committed updates and
+`DeleteStack`: each displaced address has up to three cleanup attempts, and exhausted entries are
+dropped. A failed cleanup leaves the first stack deletion in `DELETE_FAILED`, even if the current
+schedule was deleted; a later retry does not retry the dropped addresses. Schedules left after
+exhausted cleanup require manual deletion through the Scheduler API. A failed current-schedule
+delete preserves pending recovery state for a later deletion retry. This is Floci's shared cleanup
+policy, not a claim that AWS guarantees three attempts for `DeleteStack`.
+`UpdateReplacePolicy: Retain` keeps a name replacement's old schedule, but does not keep a group
+move's old address or an orphan created by a failed update.
+
+Schedules are addressed by group and name, as in the Scheduler API. If another client removes a
+managed schedule and recreates the same address, an in-place stack update applies the template to
+the current schedule there. Stack deletion and replacement cleanup delete the current schedule
+at a tracked address, and rollback restores its saved configuration at that address.
+A historical cleanup entry with attempts remaining can still affect a new occupant of its
+address. Dropping exhausted debt prevents later retries, but does not establish resource ownership.
+
## EventBridge Event Buses
`AWS::Events::EventBus` creates a real custom EventBridge bus. `Name` is required, `Ref` returns
diff --git a/docs/services/scheduler.md b/docs/services/scheduler.md
index 277f982f63..654da00bab 100644
--- a/docs/services/scheduler.md
+++ b/docs/services/scheduler.md
@@ -120,6 +120,38 @@ and `Message`, Step Functions `stateMachineArn` and `input`, EventBridge
launch type, group, and network configuration. Non-JSON input of a universal
target is encoded as a JSON string field.
+## CloudFormation
+
+`AWS::Scheduler::Schedule` creates a real schedule. `Ref` returns its name and
+`Fn::GetAtt Arn` returns its Scheduler ARN. An omitted `Name` is generated once
+and kept on updates. Expression, state, dates, timezone, time window, KMS key and
+target changes update the schedule in place. A `Name` change creates a replacement;
+`GroupName` changes keep the same `Ref` and move the schedule to the requested group.
+The group must already exist or be declared as an `AWS::Scheduler::ScheduleGroup`.
+
+The displaced schedule stays until the stack update commits, then is deleted.
+`UpdateReplacePolicy: Retain` keeps a schedule displaced by a name replacement;
+it does not retain the old address of a group move. A failed stack update restores
+the original schedule configuration and removes a replacement or group-move destination.
+During a group move the old schedule is disabled before creating the destination,
+so the stored schedules are not both enabled. A rollback disables the destination
+before restoring the original state. A name replacement can briefly leave both
+schedules enabled until commit, following the existing replacement lifecycle.
+
+If restoring a failed update also fails, the original configuration snapshot is
+kept. A later schedule update retries that restoration before taking a new snapshot
+or applying its desired configuration. If Floci skips the failed schedule during an
+otherwise unrelated stack update, cleanup preserves the snapshot and leaves the
+stack in `UPDATE_COMPLETE_CLEANUP_IN_PROGRESS` with an explicit reason. Further
+updates are blocked, and `DeleteStack` is the available recovery path. This is a
+Floci recovery limitation; AWS requires `ContinueUpdateRollback` to complete a failed
+rollback before accepting another stack update.
+
+Target parameters are limited to those the Scheduler API currently models:
+retry/dead-letter configuration, SQS, ECS and EventBridge parameters. Kinesis and
+SageMaker pipeline parameters are rejected instead of silently discarded.
+Invocation support and time-window limitations are the same as the Scheduler API.
+
## Configuration
| Variable | Default | Description |
diff --git a/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/SchedulerScheduleCfnProvisioner.java b/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/SchedulerScheduleCfnProvisioner.java
new file mode 100644
index 0000000000..5e8f6cbccf
--- /dev/null
+++ b/src/main/java/io/github/hectorvent/floci/services/cloudformation/provisioners/SchedulerScheduleCfnProvisioner.java
@@ -0,0 +1,457 @@
+package io.github.hectorvent.floci.services.cloudformation.provisioners;
+
+import com.fasterxml.jackson.core.JsonProcessingException;
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.MapperFeature;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.json.JsonMapper;
+import com.fasterxml.jackson.databind.node.ArrayNode;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.services.cloudformation.model.StackResource;
+import io.github.hectorvent.floci.services.scheduler.SchedulerService;
+import io.github.hectorvent.floci.services.scheduler.model.FlexibleTimeWindow;
+import io.github.hectorvent.floci.services.scheduler.model.Schedule;
+import io.github.hectorvent.floci.services.scheduler.model.ScheduleRequest;
+import io.github.hectorvent.floci.services.scheduler.model.Target;
+import jakarta.enterprise.context.ApplicationScoped;
+import jakarta.inject.Inject;
+import org.jboss.logging.Logger;
+
+import java.time.Instant;
+import java.time.format.DateTimeParseException;
+import java.util.Map;
+import java.util.Set;
+
+/** Provisions schedules through the same Scheduler service used by its REST API. */
+@ApplicationScoped
+public class SchedulerScheduleCfnProvisioner implements CfnResourceProvisioner {
+
+ private static final String TYPE = "AWS::Scheduler::Schedule";
+ private static final Logger LOG = Logger.getLogger(SchedulerScheduleCfnProvisioner.class);
+ private static final String GROUP_ATTR = "FlociSchedulerGroupName";
+ private static final String NAME_MODE_ATTR = "FlociSchedulerNameMode";
+ private static final String NAME_REPLACEMENT_ATTR = "__FlociSchedulerNameReplacement";
+ private static final String SNAPSHOT_ATTR = "__FlociSchedulerUpdateSnapshot";
+ private static final ObjectMapper MAPPER = JsonMapper.builder()
+ .enable(MapperFeature.ACCEPT_CASE_INSENSITIVE_PROPERTIES)
+ .addModule(new JavaTimeModule())
+ .build();
+
+ private final SchedulerService schedulerService;
+
+ @Inject
+ public SchedulerScheduleCfnProvisioner(SchedulerService schedulerService) {
+ this.schedulerService = schedulerService;
+ }
+
+ @Override
+ public Set<String> resourceTypes() {
+ return Set.of(TYPE);
+ }
+
+ @Override
+ public void provision(StackResource resource, JsonNode props, ProvisionContext ctx) {
+ readCleanup(resource);
+ if (ctx.isUpdate() && resource.getAttributes().containsKey(SNAPSHOT_ATTR)) {
+ try {
+ rollbackUpdate(resource);
+ ctx = new ProvisionContext(ctx.engine(), ctx.region(), ctx.accountId(), ctx.stackName(),
+ resource.getPhysicalId(), ctx.progress(), ctx.priorProperties());
+ resource.getAttributes().remove(CfnRollback.UPDATE_ROLLBACK_FAILURE_ATTR);
+ resource.getAttributes().remove(CfnRollback.UPDATE_ROLLBACK_RESTORED_ATTR);
+ } catch (RuntimeException failure) {
+ resource.getAttributes().put(CfnRollback.UPDATE_ROLLBACK_FAILURE_ATTR, failure.getMessage());
+ throw new IllegalStateException("Could not finish the previous schedule update rollback", failure);
+ }
+ }
+ Map<String, String> attributesBefore = Map.copyOf(resource.getAttributes());
+ String explicitName = optional(props, "Name", ctx);
+ if (explicitName != null && explicitName.isBlank()) {
+ throw validation("Name must not be blank");
+ }
+ boolean named = explicitName != null && !explicitName.isBlank();
+ boolean removedName = ctx.isUpdate() && !named
+ && "explicit".equals(attributesBefore.get(NAME_MODE_ATTR));
+ String name = removedName ? ctx.generatePhysicalName(resource.getLogicalId(), 64, false)
+ : ctx.stablePhysicalName(explicitName, resource.getLogicalId(), 64, false);
+ String explicitGroup = optional(props, "GroupName", ctx);
+ if (explicitGroup != null && explicitGroup.isBlank()) {
+ throw validation("GroupName must not be blank");
+ }
+ String group = explicitGroup == null ? "default" : explicitGroup;
+ ScheduleRequest request = request(props, name, group, ctx);
+ String priorGroup = attributesBefore.getOrDefault(GROUP_ATTR, "default");
+ boolean sameAddress = ctx.reusesPriorEntity(name) && priorGroup.equals(group);
+ boolean groupMove = ctx.reusesPriorEntity(name) && !priorGroup.equals(group);
+
+ Schedule schedule;
+ ObjectNode moveSnapshot = null;
+ if (sameAddress) {
+ Schedule current = schedulerService.getSchedule(name, group, ctx.region());
+ snapshot(resource, current, ctx.region(), null, null);
+ try {
+ schedule = schedulerService.updateSchedule(request, ctx.region());
+ } catch (RuntimeException failure) {
+ restoreAfterFailure(resource, failure);
+ throw failure;
+ }
+ } else if (groupMove) {
+ Schedule current = schedulerService.getSchedule(name, priorGroup, ctx.region());
+ moveSnapshot = snapshot(resource, current, ctx.region(), name, group);
+ ScheduleRequest paused = requestFrom(current);
+ paused.setState("DISABLED");
+ try {
+ // Keep a moving schedule from firing at both addresses while the stack update
+ // waits for its other resources. The original is restored if creation fails.
+ schedulerService.updateSchedule(paused, ctx.region());
+ schedule = schedulerService.createSchedule(request, ctx.region());
+ } catch (RuntimeException failure) {
+ restoreAfterFailure(resource, failure);
+ throw failure;
+ }
+ } else {
+ schedule = schedulerService.createSchedule(request, ctx.region());
+ }
+ if (!sameAddress) {
+ retireDisplaced(resource, address(schedule.getGroupName(), schedule.getName()));
+ }
+ resource.setPhysicalId(schedule.getName());
+ resource.getAttributes().put("Arn", schedule.getArn());
+ resource.getAttributes().put(GROUP_ATTR, schedule.getGroupName());
+ resource.getAttributes().put(NAME_MODE_ATTR, named ? "explicit" : "generated");
+ resource.getAttributes().put(NAME_REPLACEMENT_ATTR,
+ Boolean.toString(ctx.isUpdate() && !ctx.priorPhysicalId().equals(name)));
+
+ // Ref is the name, but deletion needs both name and group. The cleanup helper's private
+ // identity includes the group so a group move with an unchanged Ref is still tracked.
+ ProvisionContext cleanupContext = new ProvisionContext(ctx.engine(), ctx.region(), ctx.accountId(),
+ ctx.stackName(), ctx.isUpdate() ? address(priorGroup, ctx.priorPhysicalId()) : null, ctx.progress(),
+ ctx.priorProperties());
+ ReplacementCleanup.record(cleanupResource(resource), cleanupContext, attributesBefore);
+ if (moveSnapshot != null) {
+ moveSnapshot.put("destinationCreated", true);
+ resource.getAttributes().put(SNAPSHOT_ATTR, moveSnapshot.toString());
+ }
+ }
+
+ private static ObjectNode snapshot(StackResource resource, Schedule current, String region,
+ String destinationName, String destinationGroup) {
+ ObjectNode snapshot = MAPPER.createObjectNode().put("region", region);
+ snapshot.set("request", MAPPER.valueToTree(requestFrom(current)));
+ if (destinationName != null) {
+ snapshot.put("destinationName", destinationName);
+ snapshot.put("destinationGroup", destinationGroup);
+ snapshot.put("destinationCreated", false);
+ }
+ resource.getAttributes().put(SNAPSHOT_ATTR, snapshot.toString());
+ return snapshot;
+ }
+
+ private static ScheduleRequest request(JsonNode props, String name, String group, ProvisionContext ctx) {
+ ScheduleRequest request = new ScheduleRequest();
+ request.setName(name);
+ request.setGroupName(group);
+ String expression = optional(props, "ScheduleExpression", ctx);
+ if (expression == null || expression.isBlank()) {
+ throw validation("requires ScheduleExpression");
+ }
+ request.setScheduleExpression(expression);
+ request.setFlexibleTimeWindow(requiredObject(props, "FlexibleTimeWindow", FlexibleTimeWindow.class, ctx));
+ request.setTarget(requiredObject(props, "Target", Target.class, ctx));
+ request.setDescription(optional(props, "Description", ctx));
+ request.setScheduleExpressionTimezone(optional(props, "ScheduleExpressionTimezone", ctx));
+ String state = optional(props, "State", ctx);
+ if (state != null && !Set.of("ENABLED", "DISABLED").contains(state)) {
+ throw validation("State must be ENABLED or DISABLED");
+ }
+ request.setState(state);
+ request.setKmsKeyArn(optional(props, "KmsKeyArn", ctx));
+ request.setStartDate(date(props, "StartDate", ctx));
+ request.setEndDate(date(props, "EndDate", ctx));
+ return request;
+ }
+
+ private static <T> T requiredObject(JsonNode props, String property, Class<T> type, ProvisionContext ctx) {
+ JsonNode resolved = props == null ? null : ctx.engine().resolveNodeOmittingNoValue(props.get(property));
+ if (resolved == null || !resolved.isObject()) {
+ throw validation("requires an object for " + property);
+ }
+ try {
+ return MAPPER.treeToValue(resolved, type);
+ } catch (JsonProcessingException e) {
+ throw validation("has invalid or unsupported " + property + " properties: " + e.getOriginalMessage());
+ }
+ }
+
+ private static String optional(JsonNode props, String property, ProvisionContext ctx) {
+ JsonNode resolved = props == null ? null : ctx.engine().resolveNodeOmittingNoValue(props.get(property));
+ if (resolved == null || resolved.isNull() || resolved.isMissingNode()) {
+ return null;
+ }
+ if (!resolved.isValueNode()) {
+ throw validation("requires a scalar for " + property);
+ }
+ return resolved.asText();
+ }
+
+ private static Instant date(JsonNode props, String property, ProvisionContext ctx) {
+ String value = optional(props, property, ctx);
+ if (value == null) {
+ return null;
+ }
+ try {
+ return Instant.parse(value);
+ } catch (DateTimeParseException e) {
+ throw validation(property + " must be a UTC date-time");
+ }
+ }
+
+ private static AwsException validation(String message) {
+ return new AwsException("ValidationError", TYPE + " " + message, 400);
+ }
+
+ private static ScheduleRequest requestFrom(Schedule schedule) {
+ ScheduleRequest request = new ScheduleRequest();
+ request.setName(schedule.getName());
+ request.setGroupName(schedule.getGroupName());
+ request.setScheduleExpression(schedule.getScheduleExpression());
+ request.setScheduleExpressionTimezone(schedule.getScheduleExpressionTimezone());
+ request.setFlexibleTimeWindow(schedule.getFlexibleTimeWindow());
+ request.setTarget(schedule.getTarget());
+ request.setDescription(schedule.getDescription());
+ request.setState(schedule.getState());
+ request.setActionAfterCompletion(schedule.getActionAfterCompletion());
+ request.setKmsKeyArn(schedule.getKmsKeyArn());
+ request.setStartDate(schedule.getStartDate());
+ request.setEndDate(schedule.getEndDate());
+ return request;
+ }
+
+ private void restoreAfterFailure(StackResource resource, RuntimeException failure) {
+ try {
+ rollbackUpdate(resource);
+ resource.getAttributes().put(CfnRollback.UPDATE_ROLLBACK_RESTORED_ATTR, "true");
+ } catch (RuntimeException restoreFailure) {
+ resource.getAttributes().put(CfnRollback.UPDATE_ROLLBACK_FAILURE_ATTR, restoreFailure.getMessage());
+ if (restoreFailure != failure) {
+ failure.addSuppressed(restoreFailure);
+ }
+ }
+ }
+
+ @Override
+ public void delete(StackResource resource, String region) {
+ deleteAddress(address(resource.getAttributes().getOrDefault(GROUP_ATTR, "default"),
+ resource.getPhysicalId()), region);
+ UpdateCleanupResult result = ReplacementCleanup.complete(cleanupResource(resource),
+ (type, id, cleanupRegion) -> deleteAddress(id, cleanupRegion));
+ if (result.applicable() && !result.complete()) {
+ throw new IllegalStateException(result.failureReason());
+ }
+ resource.getAttributes().remove(SNAPSHOT_ATTR);
+ }
+
+ private void deleteAddress(String physicalId, String region) {
+ int separator = physicalId.indexOf('/');
+ String group = physicalId.substring(0, separator);
+ String name = physicalId.substring(separator + 1);
+ CfnDeletes.safeDelete("Scheduler schedule", physicalId,
+ () -> schedulerService.deleteSchedule(name, group, region), "ResourceNotFoundException");
+ }
+
+ private static String address(String group, String name) {
+ return group + "/" + name;
+ }
+
+ private static StackResource cleanupResource(StackResource resource) {
+ StackResource cleanup = new StackResource();
+ cleanup.setLogicalId(resource.getLogicalId());
+ cleanup.setResourceType(resource.getResourceType());
+ cleanup.setPhysicalId(address(resource.getAttributes().getOrDefault(GROUP_ATTR, "default"),
+ resource.getPhysicalId()));
+ cleanup.setAttributes(resource.getAttributes());
+ // GroupName is mutable on CloudFormation. Retain applies to a name replacement only,
+ // otherwise a group move would leave the old schedule firing indefinitely.
+ if (Boolean.parseBoolean(resource.getAttributes().get(NAME_REPLACEMENT_ATTR))) {
+ cleanup.setUpdateReplacePolicy(resource.getUpdateReplacePolicy());
+ }
+ return cleanup;
+ }
+
+ @Override
+ public boolean hasReplacementUpdate(StackResource resource) {
+ return ReplacementCleanup.hasReplacement(resource);
+ }
+
+ @Override
+ public String updateCleanupPhysicalId(StackResource resource) {
+ return ReplacementCleanup.cleanupPhysicalId(cleanupResource(resource));
+ }
+
+ @Override
+ public UpdateCleanupResult completeUpdate(StackResource resource) {
+ if ("UPDATE_FAILED".equals(resource.getStatus()) && retainsFailedUpdateState(resource)) {
+ throw new IllegalStateException("Schedule rollback is still pending; its original configuration "
+ + "cannot be discarded by another resource's update cleanup");
+ }
+ resource.getAttributes().remove(SNAPSHOT_ATTR);
+ UpdateCleanupResult result = ReplacementCleanup.complete(cleanupResource(resource),
+ (type, id, region) -> deleteAddress(id, region));
+ if (!result.applicable() || result.complete()) {
+ resource.getAttributes().remove(NAME_REPLACEMENT_ATTR);
+ }
+ return result;
+ }
+
+ @Override
+ public UpdateCleanupResult completeDeleteCleanup(StackResource resource) {
+ UpdateCleanupResult result = ReplacementCleanup.complete(cleanupResource(resource),
+ (type, id, region) -> deleteAddress(id, region));
+ return result;
+ }
+
+ @Override
+ public void clearDeleteCleanup(StackResource resource) {
+ ReplacementCleanup.clear(resource);
+ if (!ReplacementCleanup.hasReplacement(resource)) {
+ resource.getAttributes().remove(NAME_REPLACEMENT_ATTR);
+ }
+ }
+
+ @Override
+ public void clearUpdate(StackResource resource) {
+ resource.getAttributes().remove(SNAPSHOT_ATTR);
+ ReplacementCleanup.clear(resource);
+ resource.getAttributes().remove(NAME_REPLACEMENT_ATTR);
+ }
+
+ @Override
+ public void mergeFailedUpdateResourceTracking(StackResource previous, StackResource attempted) {
+ ObjectNode source = readCleanup(attempted);
+ if (source != null) {
+ String currentAddress = cleanupResource(previous).getPhysicalId();
+ for (JsonNode entry : source.path("displaced")) {
+ String physicalId = entry.path("physicalId").asText();
+ if (physicalId.equals(currentAddress)) {
+ continue;
+ }
+ // The attempted update can recreate an address whose earlier cleanup was exhausted.
+ // Preserve that attempt's current retry record when the engine restores the old resource.
+ retireDisplaced(previous, physicalId);
+ }
+ }
+ ReplacementCleanup.mergeDisplaced(cleanupResource(previous), cleanupResource(attempted));
+ }
+
+ private static ObjectNode readCleanup(StackResource resource) {
+ String raw = resource.getAttributes().get(CfnRollback.REPLACEMENT_CLEANUP_ATTR);
+ if (raw == null) {
+ return null;
+ }
+ try {
+ JsonNode node = MAPPER.readTree(raw);
+ if (!node.isObject() || (node.has("displaced") && !node.path("displaced").isArray())) {
+ throw new IllegalStateException("Invalid schedule replacement cleanup");
+ }
+ for (JsonNode entry : node.path("displaced")) {
+ if (!entry.isObject()) {
+ throw new IllegalStateException("Invalid schedule replacement cleanup");
+ }
+ }
+ return (ObjectNode) node;
+ } catch (JsonProcessingException e) {
+ throw new IllegalStateException("Could not read schedule replacement cleanup", e);
+ }
+ }
+
+ private static void retireDisplaced(StackResource resource, String physicalId) {
+ ObjectNode cleanup = readCleanup(resource);
+ if (cleanup == null) {
+ return;
+ }
+ ArrayNode remaining = MAPPER.createArrayNode();
+ for (JsonNode entry : cleanup.path("displaced")) {
+ if (!physicalId.equals(entry.path("physicalId").asText())) {
+ remaining.add(entry);
+ }
+ }
+ cleanup.set("displaced", remaining);
+ if (remaining.isEmpty() && !cleanup.has("priorPhysicalId")) {
+ resource.getAttributes().remove(CfnRollback.REPLACEMENT_CLEANUP_ATTR);
+ } else {
+ resource.getAttributes().put(CfnRollback.REPLACEMENT_CLEANUP_ATTR, cleanup.toString());
+ }
+ }
+
+ @Override
+ public boolean rollbackUpdate(StackResource resource) {
+ JsonNode snapshot = readSnapshot(resource);
+ if (snapshot != null && snapshot.path("destinationCreated").asBoolean(false)) {
+ // Disable the move's destination before restoring an enabled original, even if deleting
+ // the destination fails. The cleanup helper retains that disabled orphan's address.
+ pauseDestination(snapshot);
+ }
+ StackResource cleanup = cleanupResource(resource);
+ RuntimeException cleanupFailure = null;
+ try {
+ ReplacementCleanup.rollback(cleanup, (type, id, region) -> deleteAddress(id, region));
+ } catch (RuntimeException failure) {
+ cleanupFailure = failure;
+ } finally {
+ String restoredAddress = cleanup.getPhysicalId();
+ resource.setPhysicalId(restoredAddress.substring(restoredAddress.indexOf('/') + 1));
+ }
+ if (snapshot != null) {
+ try {
+ ScheduleRequest request = MAPPER.treeToValue(snapshot.get("request"), ScheduleRequest.class);
+ schedulerService.updateSchedule(request, snapshot.path("region").asText());
+ resource.getAttributes().remove(SNAPSHOT_ATTR);
+ } catch (JsonProcessingException | RuntimeException restoreFailure) {
+ if (cleanupFailure != null && cleanupFailure != restoreFailure) {
+ restoreFailure.addSuppressed(cleanupFailure);
+ }
+ throw new IllegalStateException("Could not restore schedule update snapshot", restoreFailure);
+ }
+ }
+ if (cleanupFailure != null) {
+ throw cleanupFailure;
+ }
+ return true;
+ }
+
+ @Override
+ public boolean retainsFailedUpdateState(StackResource resource) {
+ return resource.getAttributes().containsKey(SNAPSHOT_ATTR);
+ }
+
+ private static JsonNode readSnapshot(StackResource resource) {
+ String raw = resource.getAttributes().get(SNAPSHOT_ATTR);
+ if (raw == null) {
+ return null;
+ }
+ try {
+ return MAPPER.readTree(raw);
+ } catch (JsonProcessingException e) {
+ throw new IllegalStateException("Could not read schedule update snapshot", e);
+ }
+ }
+
+ private void pauseDestination(JsonNode snapshot) {
+ String name = snapshot.path("destinationName").asText();
+ String group = snapshot.path("destinationGroup").asText();
+ String region = snapshot.path("region").asText();
+ try {
+ ScheduleRequest request = requestFrom(schedulerService.getSchedule(name, group, region));
+ request.setState("DISABLED");
+ schedulerService.updateSchedule(request, region);
+ } catch (AwsException e) {
+ if (!"ResourceNotFoundException".equals(e.getErrorCode())) {
+ throw e;
+ }
+ LOG.debugv("Schedule move destination {0}/{1} already gone during rollback", group, name);
+ }
+ }
+
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixture.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixture.java
index d4bc04ce26..8a967cf751 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixture.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixture.java
@@ -97,6 +97,7 @@ import io.github.hectorvent.floci.services.cloudformation.provisioners.RedshiftC
import io.github.hectorvent.floci.services.cloudformation.provisioners.RedshiftServerlessCfnProvisioner;
import io.github.hectorvent.floci.services.cloudformation.provisioners.Route53CfnProvisioner;
import io.github.hectorvent.floci.services.cloudformation.provisioners.S3CfnProvisioner;
+import io.github.hectorvent.floci.services.cloudformation.provisioners.SchedulerScheduleCfnProvisioner;
import io.github.hectorvent.floci.services.cloudformation.provisioners.SchedulerScheduleGroupCfnProvisioner;
import io.github.hectorvent.floci.services.cloudformation.provisioners.SecretTargetAttachmentCfnProvisioner;
import io.github.hectorvent.floci.services.cloudformation.provisioners.SecretsManagerCfnProvisioner;
@@ -463,6 +464,7 @@ final class CfnProvisionerFixture {
}
if (schedulerService != null) {
discovered.add(new SchedulerScheduleGroupCfnProvisioner(schedulerService));
+ discovered.add(new SchedulerScheduleCfnProvisioner(schedulerService));
}
if (transferService != null) {
discovered.add(new TransferCfnProvisioner(transferService));
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixtureTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixtureTest.java
index d34adf9952..a0e28bb9b8 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixtureTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CfnProvisionerFixtureTest.java
@@ -7,6 +7,7 @@ import io.github.hectorvent.floci.services.cloudformation.provisioners.CloudForm
import io.github.hectorvent.floci.services.cloudformation.provisioners.ProvisionContext;
import io.github.hectorvent.floci.services.cloudwatch.logs.CloudWatchLogsService;
import io.github.hectorvent.floci.services.ec2.Ec2Service;
+import io.github.hectorvent.floci.services.scheduler.SchedulerService;
import io.github.hectorvent.floci.services.sns.SnsService;
import org.junit.jupiter.api.Test;
@@ -61,6 +62,15 @@ class CfnProvisionerFixtureTest {
assertTrue(serves(fixture, "AWS::Logs::LogGroup"));
}
+ @Test
+ void schedulerWiresBothSchedulesAndGroups() {
+ CfnProvisionerFixture.Builder fixture = CfnProvisionerFixture.builder()
+ .scheduler(mock(SchedulerService.class));
+
+ assertTrue(serves(fixture, "AWS::Scheduler::Schedule"));
+ assertTrue(serves(fixture, "AWS::Scheduler::ScheduleGroup"));
+ }
+
@Test
void anUnnamedServiceIsNotWired() {
CfnProvisionerFixture.Builder fixture = CfnProvisionerFixture.builder()
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationSchedulerScheduleIntegrationTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationSchedulerScheduleIntegrationTest.java
new file mode 100644
index 0000000000..2d28008c68
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/CloudFormationSchedulerScheduleIntegrationTest.java
@@ -0,0 +1,1090 @@
+package io.github.hectorvent.floci.services.cloudformation;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.core.common.XmlParser;
+import io.github.hectorvent.floci.services.cloudformation.model.StackResource;
+import io.github.hectorvent.floci.services.scheduler.ScheduleInvoker;
+import io.github.hectorvent.floci.services.scheduler.SchedulerService;
+import io.github.hectorvent.floci.services.scheduler.model.ScheduleRequest;
+import io.github.hectorvent.floci.services.sqs.SqsService;
+import io.github.hectorvent.floci.services.sqs.model.Message;
+import io.github.hectorvent.floci.services.sqs.model.Queue;
+import io.github.hectorvent.floci.testing.RestAssuredJsonUtils;
+import io.quarkus.test.junit.QuarkusTest;
+import io.quarkus.test.junit.mockito.InjectSpy;
+import io.restassured.response.Response;
+import io.restassured.specification.RequestSpecification;
+import jakarta.inject.Inject;
+import org.junit.jupiter.api.BeforeAll;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+
+import java.time.Duration;
+import java.time.Instant;
+import java.util.List;
+import java.util.Locale;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicBoolean;
+
+import static io.restassured.RestAssured.given;
+import static org.awaitility.Awaitility.await;
+import static org.hamcrest.Matchers.containsString;
+import static org.hamcrest.Matchers.equalTo;
+import static org.junit.jupiter.api.Assertions.assertAll;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.ArgumentMatchers.argThat;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.doAnswer;
+import static org.mockito.Mockito.doCallRealMethod;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+
+@QuarkusTest
+class CloudFormationSchedulerScheduleIntegrationTest {
+
+ private static final String AUTH = "AWS4-HMAC-SHA256 Credential=test/20261001/us-east-1/cloudformation/aws4_request";
+ private static final String TEMPLATE = """
+ {
+ "Parameters":{"Name":{"Type":"String"},"Group":{"Type":"String","Default":"default"},
+ "Expression":{"Type":"String","Default":"rate(5 minutes)"},
+ "Payload":{"Type":"String","Default":"first"}},
+ "Resources":{
+ "Queue":{"Type":"AWS::SQS::Queue"},
+ "Schedule":{"Type":"AWS::Scheduler::Schedule","UpdateReplacePolicy":"%s","Properties":{
+ %s
+ "GroupName":{"Ref":"Group"},"ScheduleExpression":{"Ref":"Expression"},
+ "State":"DISABLED","FlexibleTimeWindow":{"Mode":"OFF"},
+ "Target":{"Arn":{"Fn::GetAtt":["Queue","Arn"]},
+ "RoleArn":{"Fn::Sub":"arn:${AWS::Partition}:iam::${AWS::AccountId}:role/scheduler"},
+ "Input":{"Fn::Join":["",["payload:",{"Ref":"Payload"}]]}}
+ }}%s
+ },
+ "Outputs":{
+ "ScheduleRef":{"Value":{"Ref":"Schedule"}},
+ "ScheduleArn":{"Value":{"Fn::GetAtt":["Schedule","Arn"]}},
+ "QueueUrl":{"Value":{"Ref":"Queue"}}
+ }
+ }
+ """;
+ private static final String NAME = "\"Name\":{\"Ref\":\"Name\"},";
+ private static final String FAILURE = """
+ ,"BadSecret":{"Type":"AWS::SecretsManager::Secret","DependsOn":"Schedule",
+ "Properties":{"SecretString":"explicit","GenerateSecretString":{"PasswordLength":32}}}
+ """;
+
+ @InjectSpy
+ SchedulerService scheduler;
+
+ @Inject
+ CloudFormationService cloudFormation;
+
+ @Inject
+ ScheduleInvoker invoker;
+
+ @Inject
+ SqsService sqs;
+
+ @BeforeAll
+ static void configureRestAssured() {
+ RestAssuredJsonUtils.configureAwsContentTypes();
+ }
+
+ @Test
+ void stackCreatesRealScheduleResolvesIntrinsicsUpdatesAndDeletes() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-" + suffix;
+ String name = "schedule-" + suffix;
+ cloudFormation(stack, "CreateStack", template("Delete", NAME, ""), Map.of("Name", name));
+ Map<String, String> outputs = outputs(stack, "CREATE_COMPLETE");
+ assertEquals(name, outputs.get("ScheduleRef"));
+ String arn = getSchedule(name, "default").then().statusCode(200)
+ .body("Target.Input", equalTo("payload:first"))
+ .body("State", equalTo("DISABLED"))
+ .extract().path("Arn");
+ assertEquals(arn, outputs.get("ScheduleArn"));
+ assertTrue(arn.endsWith("schedule/default/" + name));
+
+ invoker.invoke(scheduler.getSchedule(name, "default", "us-east-1"), Instant.now());
+ List<Message> messages = sqs.receiveMessage(outputs.get("QueueUrl"), 1, 0, 0, "us-east-1");
+ assertEquals(1, messages.size());
+ assertEquals("payload:first", messages.getFirst().getBody());
+
+ cloudFormation(stack, "UpdateStack", template("Delete", NAME, ""),
+ Map.of("Name", name, "Expression", "rate(10 minutes)", "Payload", "second"));
+ assertEquals(name, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(10 minutes)"))
+ .body("Target.Input", equalTo("payload:second"));
+ deleteStack(stack);
+ getSchedule(name, "default").then().statusCode(404);
+ }
+
+ @Test
+ void updateRestoresTemplateAfterScheduleWasRecreatedAtItsAddress() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-recreated-update-" + suffix;
+ String name = "recreated-update-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ String template = externalQueueTemplate("", queue);
+ cloudFormation(stack, "CreateStack", template, Map.of("Name", name));
+ String arn = outputs(stack, "CREATE_COMPLETE").get("ScheduleArn");
+ recreateScheduleViaApi(name, "default");
+ getSchedule(name, "default").then().statusCode(200).body("Arn", equalTo(arn))
+ .body("ScheduleExpression", equalTo("rate(30 minutes)"))
+ .body("Target.Input", equalTo("out-of-band"));
+ try {
+ cloudFormation(stack, "UpdateStack", template,
+ Map.of("Name", name, "Expression", "rate(10 minutes)", "Payload", "updated"));
+ assertEquals(arn, outputs(stack, "UPDATE_COMPLETE").get("ScheduleArn"));
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(10 minutes)"))
+ .body("Target.Input", equalTo("payload:updated"))
+ .body("State", equalTo("DISABLED"));
+ cloudFormation(stack, "UpdateStack", template, Map.of("Name", name));
+ assertEquals(name, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(5 minutes)"))
+ .body("Target.Input", equalTo("payload:first"));
+ } finally {
+ deleteStack(stack);
+ given().delete("/schedules/" + name);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void stackDeletionDeletesScheduleRecreatedAtItsAddress() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-recreated-delete-" + suffix;
+ String name = "recreated-delete-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ String arn = outputs(stack, "CREATE_COMPLETE").get("ScheduleArn");
+ recreateScheduleViaApi(name, "default");
+ getSchedule(name, "default").then().statusCode(200).body("Arn", equalTo(arn));
+ try {
+ deleteStack(stack);
+ getSchedule(name, "default").then().statusCode(404);
+ } finally {
+ given().delete("/schedules/" + name);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void groupMoveKeepsNameAndDeletesOldAddressEvenWithRetain() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-move-" + suffix;
+ String name = "move-" + suffix;
+ String group = "group-" + suffix;
+ createGroup(group);
+ cloudFormation(stack, "CreateStack", template("Retain", NAME, ""), Map.of("Name", name));
+ Map<String, String> previous = outputs(stack, "CREATE_COMPLETE");
+
+ cloudFormation(stack, "UpdateStack", template("Retain", NAME, ""), Map.of("Name", name, "Group", group));
+ Map<String, String> updated = outputs(stack, "UPDATE_COMPLETE");
+
+ assertEquals(name, updated.get("ScheduleRef"));
+ assertTrue(updated.get("ScheduleArn").endsWith("/" + group + "/" + name));
+ assertFalse(previous.get("ScheduleArn").equals(updated.get("ScheduleArn")));
+ getSchedule(name, "default").then().statusCode(404);
+ getSchedule(name, group).then().statusCode(200);
+ deleteStack(stack);
+ getSchedule(name, group).then().statusCode(404);
+ deleteGroup(group);
+ }
+
+ @Test
+ void failedGroupMoveRestoresOriginalScheduleAndRemovesDestination() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-move-rb-" + suffix;
+ String name = "move-rb-" + suffix;
+ String group = "group-rb-" + suffix;
+ createGroup(group);
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ Map<String, String> before = outputs(stack, "CREATE_COMPLETE");
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Group", group, "Payload", "changed"));
+ Map<String, String> after = outputs(stack, "UPDATE_ROLLBACK_COMPLETE");
+
+ assertEquals(before.get("ScheduleRef"), after.get("ScheduleRef"));
+ assertEquals(before.get("ScheduleArn"), after.get("ScheduleArn"));
+ getSchedule(name, "default").then().statusCode(200).body("Target.Input", equalTo("payload:first"));
+ getSchedule(name, group).then().statusCode(404);
+ deleteStack(stack);
+ deleteGroup(group);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void failedInPlaceUpdateRestoresExpressionAndTarget() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-update-rb-" + suffix;
+ String name = "update-rb-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ outputs(stack, "CREATE_COMPLETE");
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Expression", "rate(10 minutes)", "Payload", "changed"));
+ outputs(stack, "UPDATE_ROLLBACK_COMPLETE");
+
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(5 minutes)"))
+ .body("Target.Input", equalTo("payload:first"));
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void nameReplacementCleansOldScheduleAndCanBeRolledBack() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-name-" + suffix;
+ String oldName = "old-" + suffix;
+ String newName = "new-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", oldName));
+ outputs(stack, "CREATE_COMPLETE");
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate("", queue), Map.of("Name", newName));
+ assertEquals(newName, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ getSchedule(oldName, "default").then().statusCode(404);
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue), Map.of("Name", oldName));
+ assertEquals(newName, outputs(stack, "UPDATE_ROLLBACK_COMPLETE").get("ScheduleRef"));
+ getSchedule(oldName, "default").then().statusCode(404);
+ getSchedule(newName, "default").then().statusCode(200);
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void generatedNameStaysStableAndLaterCreateFailureRollsItBack() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-unnamed-" + suffix;
+ cloudFormation(stack, "CreateStack", template("Delete", "", ""), Map.of("Name", "unused"));
+ String generated = outputs(stack, "CREATE_COMPLETE").get("ScheduleRef");
+ assertTrue(generated.startsWith(stack + "-Schedule-"));
+ assertTrue(generated.length() <= 64);
+ cloudFormation(stack, "UpdateStack", template("Delete", "", ""),
+ Map.of("Name", "unused", "Payload", "second"));
+ assertEquals(generated, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ deleteStack(stack);
+
+ String failedStack = "cfn-schedule-create-rb-" + suffix;
+ cloudFormation(failedStack, "CreateStack", template("Delete", NAME, FAILURE), Map.of("Name", generated));
+ outputs(failedStack, "ROLLBACK_COMPLETE");
+ getSchedule(generated, "default").then().statusCode(404);
+ deleteStack(failedStack);
+ }
+
+ @Test
+ void anotherFailedUpdateAfterRollbackFailureRestoresTheOriginalSchedule() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-retry-" + suffix;
+ String name = "retry-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ prepareFailedRollback(stack, name, queue);
+ allowOriginalRestore(name);
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Expression", "rate(15 minutes)", "Payload", "third"));
+ outputs(stack, "UPDATE_ROLLBACK_COMPLETE");
+
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(5 minutes)"))
+ .body("Target.Input", equalTo("payload:first"));
+ assertFalse(scheduleResource(stack).getAttributes().containsKey("__FlociSchedulerUpdateSnapshot"));
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void anotherUpdateCannotMutateTheScheduleWhileItsOriginalRestoreStillFails() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-retry-block-" + suffix;
+ String name = "retry-block-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ prepareFailedRollback(stack, name, queue);
+ String snapshot = scheduleResource(stack).getAttributes().get("__FlociSchedulerUpdateSnapshot");
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate("", queue),
+ Map.of("Name", name, "Expression", "rate(15 minutes)", "Payload", "third"));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(10 minutes)"))
+ .body("Target.Input", equalTo("payload:changed"));
+ assertEquals(snapshot, scheduleResource(stack).getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+ deleteStack(stack);
+ getSchedule(name, "default").then().statusCode(404);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void anUpdateThatSkipsTheScheduleCannotClearItsFailedRollback() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-skipped-" + suffix;
+ String name = "skipped-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ prepareFailedRollback(stack, name, queue);
+ String snapshot = scheduleResource(stack).getAttributes().get("__FlociSchedulerUpdateSnapshot");
+ String unchanged = externalQueueTemplate("", queue);
+ cloudFormation(stack, "UpdateStack", unchanged, Map.of("Name", name));
+
+ await().atMost(Duration.ofSeconds(10)).untilAsserted(() ->
+ given().contentType("application/x-www-form-urlencoded").header("Authorization", AUTH)
+ .formParam("Action", "DescribeStacks").formParam("StackName", stack).post("/")
+ .then().statusCode(200)
+ .body(containsString("<StackStatus>UPDATE_COMPLETE_CLEANUP_IN_PROGRESS</StackStatus>"))
+ .body(containsString("Schedule rollback is still pending")));
+ assertEquals(snapshot, scheduleResource(stack).getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+ assertEquals(unchanged, cloudFormation.describeStacks(stack, "us-east-1").getFirst().getOriginalTemplateBody());
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(10 minutes)"))
+ .body("Target.Input", equalTo("payload:changed"));
+ given().contentType("application/x-www-form-urlencoded").header("Authorization", AUTH)
+ .formParam("Action", "UpdateStack").formParam("StackName", stack)
+ .formParam("TemplateBody", unchanged).post("/").then().statusCode(400)
+ .body(containsString("UPDATE_COMPLETE_CLEANUP_IN_PROGRESS"));
+ deleteStack(stack);
+ getSchedule(name, "default").then().statusCode(404);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void failedStackDeletionKeepsThePendingScheduleSnapshotForDeletionRetry() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-delete-retry-" + suffix;
+ String name = "delete-retry-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ prepareFailedRollback(stack, name, queue);
+ String snapshot = scheduleResource(stack).getAttributes().get("__FlociSchedulerUpdateSnapshot");
+ doThrow(new AwsException("InternalServerException", "schedule deletion unavailable", 500))
+ .doCallRealMethod().when(scheduler).deleteSchedule(name, "default", "us-east-1");
+
+ cloudFormation(stack, "DeleteStack", null, Map.of());
+ outputs(stack, "DELETE_FAILED");
+
+ assertEquals(snapshot, scheduleResource(stack).getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(10 minutes)"))
+ .body("Target.Input", equalTo("payload:changed"));
+
+ deleteStack(stack);
+ getSchedule(name, "default").then().statusCode(404);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void boundedStackDeletionAbandonsAnExhaustedGroupMoveOrphanWithoutAFourthAttempt() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-orphan-delete-" + suffix;
+ String name = "orphan-delete-" + suffix;
+ String group = "orphan-group-" + suffix;
+ createGroup(group);
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ outputs(stack, "CREATE_COMPLETE");
+ doThrow(new AwsException("InternalServerException", "orphan deletion unavailable", 500))
+ .when(scheduler).deleteSchedule(name, group, "us-east-1");
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Group", group));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+ getSchedule(name, "default").then().statusCode(200);
+ getSchedule(name, group).then().statusCode(200).body("State", equalTo("DISABLED"));
+
+ boolean deleted = false;
+ clearInvocations(scheduler);
+ try {
+ cloudFormation(stack, "DeleteStack", null, Map.of());
+ outputs(stack, "DELETE_FAILED");
+ assertAll(
+ () -> assertFalse(scheduleResource(stack).getAttributes().containsKey("__FlociReplacementCleanup")),
+ () -> verify(scheduler, times(3)).deleteSchedule(name, group, "us-east-1"),
+ () -> verify(scheduler, times(1)).deleteSchedule(name, "default", "us-east-1"),
+ () -> getSchedule(name, group).then().statusCode(200).body("State", equalTo("DISABLED")));
+
+ doCallRealMethod().when(scheduler).deleteSchedule(name, group, "us-east-1");
+ deleteStack(stack);
+ deleted = true;
+ getSchedule(name, "default").then().statusCode(404);
+ getSchedule(name, group).then().statusCode(200);
+ verify(scheduler, times(3)).deleteSchedule(name, group, "us-east-1");
+ assertTrue(sqs.getQueueAttributes(queue.getQueueUrl(), List.of("QueueArn"), "us-east-1")
+ .get("QueueArn").endsWith(":external-" + suffix));
+ } finally {
+ doCallRealMethod().when(scheduler).deleteSchedule(name, group, "us-east-1");
+ if (!deleted) {
+ deleteStack(stack);
+ }
+ given().queryParam("groupName", group).delete("/schedules/" + name);
+ deleteGroup(group);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void boundedDeletionRetryPreservesAnotherStacksScheduleAtAnExhaustedAddress() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-old-owner-" + suffix;
+ String secondStack = "cfn-schedule-new-owner-" + suffix;
+ String name = "recreated-" + suffix;
+ String group = "recreated-group-" + suffix;
+ createGroup(group);
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ outputs(stack, "CREATE_COMPLETE");
+ doThrow(new AwsException("InternalServerException", "orphan deletion unavailable", 500))
+ .when(scheduler).deleteSchedule(name, group, "us-east-1");
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Group", group));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+ cloudFormation(stack, "DeleteStack", null, Map.of());
+ outputs(stack, "DELETE_FAILED");
+ boolean secondCreated = false;
+ boolean deleted = false;
+ try {
+ doCallRealMethod().when(scheduler).deleteSchedule(name, group, "us-east-1");
+ given().queryParam("groupName", group).delete("/schedules/" + name).then().statusCode(200);
+ cloudFormation(secondStack, "CreateStack", externalQueueTemplate("", queue),
+ Map.of("Name", name, "Group", group, "Payload", "second-stack"));
+ outputs(secondStack, "CREATE_COMPLETE");
+ secondCreated = true;
+ getSchedule(name, group).then().statusCode(200)
+ .body("Target.Input", equalTo("payload:second-stack"));
+ clearInvocations(scheduler);
+
+ deleteStack(stack);
+ deleted = true;
+ assertAll(
+ () -> getSchedule(name, group).then().statusCode(200)
+ .body("Target.Input", equalTo("payload:second-stack")),
+ () -> verify(scheduler, never()).deleteSchedule(name, group, "us-east-1"),
+ () -> outputs(secondStack, "CREATE_COMPLETE"),
+ () -> getSchedule(name, "default").then().statusCode(404));
+ } finally {
+ doCallRealMethod().when(scheduler).deleteSchedule(name, group, "us-east-1");
+ if (secondCreated) {
+ deleteStack(secondStack);
+ }
+ if (!deleted) {
+ deleteStack(stack);
+ }
+ given().queryParam("groupName", group).delete("/schedules/" + name);
+ deleteGroup(group);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void repeatedFailedReplacementRestartsCleanupAtTheSameAddress() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-reused-debt-" + suffix;
+ String originalName = "original-debt-" + suffix;
+ String replacementName = "replacement-debt-" + suffix;
+ String replacementAddress = "default/" + replacementName;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", originalName));
+ outputs(stack, "CREATE_COMPLETE");
+ doThrow(new AwsException("InternalServerException", "replacement deletion unavailable", 500))
+ .when(scheduler).deleteSchedule(replacementName, "default", "us-east-1");
+
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", replacementName, "Payload", "first-failed"));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+ Instant firstCreation = scheduler.getSchedule(replacementName, "default", "us-east-1").getCreationDate();
+ doCallRealMethod().when(scheduler).deleteSchedule(replacementName, "default", "us-east-1");
+ given().delete("/schedules/" + replacementName).then().statusCode(200);
+ doThrow(new AwsException("InternalServerException", "replacement deletion still unavailable", 500))
+ .when(scheduler).deleteSchedule(replacementName, "default", "us-east-1");
+
+ // Change both resources so the engine actually retries the replacement and its failing dependent.
+ String repeatedFailure = FAILURE.replace("\"SecretString\":\"explicit\"",
+ "\"SecretString\":\"explicit-second\"");
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(repeatedFailure, queue),
+ Map.of("Name", replacementName, "Payload", "second-failed"));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+ assertFalse(firstCreation.equals(scheduler.getSchedule(replacementName, "default", "us-east-1").getCreationDate()));
+ StackResource resource = scheduleResource(stack);
+ assertEquals(originalName, resource.getPhysicalId());
+ JsonNode displaced = new ObjectMapper().readTree(resource.getAttributes().get("__FlociReplacementCleanup"))
+ .path("displaced");
+ assertEquals(1, displaced.size());
+ assertEquals(replacementAddress, displaced.get(0).path("physicalId").asText());
+ assertEquals(0, displaced.get(0).path("cleanupAttempts").asInt());
+ assertFalse(displaced.get(0).path("retainable").asBoolean());
+ getSchedule(originalName, "default").then().statusCode(200).body("Target.Input", equalTo("payload:first"));
+ getSchedule(replacementName, "default").then().statusCode(200)
+ .body("Target.Input", equalTo("payload:second-failed"));
+
+ doCallRealMethod().when(scheduler).deleteSchedule(replacementName, "default", "us-east-1");
+ deleteStack(stack);
+ getSchedule(originalName, "default").then().statusCode(404);
+ getSchedule(replacementName, "default").then().statusCode(404);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"Delete", "Retain"})
+ void nameReplacementHandlesTheRecreatedOldAddressAccordingToPolicy(String policy) throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-recreated-replace-" + suffix;
+ String originalName = "recreated-source-" + suffix;
+ String replacementName = "recreated-destination-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ String template = externalQueueTemplate("", queue)
+ .replace("\"UpdateReplacePolicy\":\"Delete\"", "\"UpdateReplacePolicy\":\"" + policy + "\"");
+ cloudFormation(stack, "CreateStack", template, Map.of("Name", originalName));
+ outputs(stack, "CREATE_COMPLETE");
+ recreateScheduleViaApi(originalName, "default");
+ try {
+ cloudFormation(stack, "UpdateStack", template, Map.of("Name", replacementName));
+ assertEquals(replacementName, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ if ("Retain".equals(policy)) {
+ getSchedule(originalName, "default").then().statusCode(200).body("Target.Input", equalTo("out-of-band"));
+ } else {
+ getSchedule(originalName, "default").then().statusCode(404);
+ }
+ deleteStack(stack);
+ getSchedule(replacementName, "default").then().statusCode(404);
+ } finally {
+ given().delete("/schedules/" + originalName);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void nameReplacementWorksAfterTheOldAddressWasDeleted() {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-gone-replace-" + suffix;
+ String originalName = "gone-source-" + suffix;
+ String replacementName = "gone-destination-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ String template = externalQueueTemplate("", queue);
+ cloudFormation(stack, "CreateStack", template, Map.of("Name", originalName));
+ outputs(stack, "CREATE_COMPLETE");
+ given().delete("/schedules/" + originalName).then().statusCode(200);
+ cloudFormation(stack, "UpdateStack", template, Map.of("Name", replacementName));
+ assertEquals(replacementName, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ getSchedule(originalName, "default").then().statusCode(404);
+ getSchedule(replacementName, "default").then().statusCode(200).body("Target.Input", equalTo("payload:first"));
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+
+ @Test
+ void replacementCleanupDeletesTheAddressRecreatedWhileDeletionStarts() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-recreate-cleanup-" + suffix;
+ String originalName = "cleanup-source-" + suffix;
+ String replacementName = "cleanup-destination-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ String template = externalQueueTemplate("", queue);
+ cloudFormation(stack, "CreateStack", template, Map.of("Name", originalName));
+ outputs(stack, "CREATE_COMPLETE");
+ AtomicBoolean recreateBeforeDelete = new AtomicBoolean(true);
+ doAnswer(invocation -> {
+ if (recreateBeforeDelete.compareAndSet(true, false)) {
+ recreateScheduleViaApi(originalName, "default");
+ }
+ return invocation.callRealMethod();
+ }).when(scheduler).deleteSchedule(originalName, "default", "us-east-1");
+ try {
+ cloudFormation(stack, "UpdateStack", template, Map.of("Name", replacementName));
+ assertEquals(replacementName, outputs(stack, "UPDATE_COMPLETE").get("ScheduleRef"));
+ assertFalse(recreateBeforeDelete.get());
+ getSchedule(originalName, "default").then().statusCode(404);
+ getSchedule(replacementName, "default").then().statusCode(200);
+ deleteStack(stack);
+ getSchedule(replacementName, "default").then().statusCode(404);
+ } finally {
+ doCallRealMethod().when(scheduler).deleteSchedule(originalName, "default", "us-east-1");
+ given().delete("/schedules/" + originalName);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void pendingSnapshotRestoresTheRecreatedAddressBeforeAnotherUpdate() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-snapshot-owner-" + suffix;
+ String name = "snapshot-foreign-" + suffix;
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ prepareFailedRollback(stack, name, queue);
+ allowOriginalRestore(name);
+ recreateScheduleViaApi(name, "default");
+
+ try {
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate("", queue),
+ Map.of("Name", name, "Expression", "rate(15 minutes)", "Payload", "new"));
+ outputs(stack, "UPDATE_COMPLETE");
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(15 minutes)"))
+ .body("Target.Input", equalTo("payload:new"));
+ assertFalse(scheduleResource(stack).getAttributes().containsKey("__FlociSchedulerUpdateSnapshot"));
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Expression", "rate(20 minutes)", "Payload", "failed"));
+ outputs(stack, "UPDATE_ROLLBACK_COMPLETE");
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(15 minutes)"))
+ .body("Target.Input", equalTo("payload:new"));
+ deleteStack(stack);
+ getSchedule(name, "default").then().statusCode(404);
+ } finally {
+ given().delete("/schedules/" + name);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void pendingGroupMoveDisablesTheRecreatedDestinationBeforeCleanup() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-move-owner-" + suffix;
+ String name = "move-foreign-" + suffix;
+ String group = "move-foreign-group-" + suffix;
+ createGroup(group);
+ Queue queue = sqs.createQueue("external-" + suffix, Map.of(), "us-east-1");
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ outputs(stack, "CREATE_COMPLETE");
+ doThrow(new AwsException("InternalServerException", "orphan deletion unavailable", 500))
+ .when(scheduler).deleteSchedule(name, group, "us-east-1");
+ doCallRealMethod().doThrow(new AwsException("InternalServerException", "original restore unavailable", 500))
+ .when(scheduler).updateSchedule(argThat(request -> name.equals(request.getName())
+ && "default".equals(request.getGroupName())
+ && "rate(5 minutes)".equals(request.getScheduleExpression())), eq("us-east-1"));
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Group", group));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+ assertTrue(scheduleResource(stack).getAttributes().containsKey("__FlociSchedulerUpdateSnapshot"));
+ doCallRealMethod().when(scheduler).deleteSchedule(name, group, "us-east-1");
+ allowOriginalRestore(name);
+ recreateScheduleViaApi(name, group, "ENABLED");
+ AtomicBoolean pausedRecreatedDestination = new AtomicBoolean(false);
+ doAnswer(invocation -> {
+ ScheduleRequest request = invocation.getArgument(0);
+ assertEquals("DISABLED", request.getState());
+ assertEquals("out-of-band", request.getTarget().getInput());
+ pausedRecreatedDestination.set(true);
+ return invocation.callRealMethod();
+ }).when(scheduler).updateSchedule(argThat(request -> name.equals(request.getName())
+ && group.equals(request.getGroupName())), eq("us-east-1"));
+
+ try {
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate("", queue),
+ Map.of("Name", name, "Expression", "rate(15 minutes)", "Payload", "new"));
+ outputs(stack, "UPDATE_COMPLETE");
+ assertTrue(pausedRecreatedDestination.get());
+ getSchedule(name, group).then().statusCode(404);
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(15 minutes)"))
+ .body("Target.Input", equalTo("payload:new"));
+ assertFalse(scheduleResource(stack).getAttributes().containsKey("__FlociSchedulerUpdateSnapshot"));
+ deleteStack(stack);
+ } finally {
+ given().queryParam("groupName", group).delete("/schedules/" + name);
+ deleteGroup(group);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void conditionalNoValueScalarsCreateGeneratedScheduleWithDefaults() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-no-value-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ String conditional = conditionalScheduleTemplate(queue, true, true, false);
+ try {
+ cloudFormation(stack, "CreateStack", conditional,
+ Map.of("Name", "unused-" + suffix, "UseOptions", "false"));
+ Map<String, String> output = outputs(stack, "CREATE_COMPLETE");
+ String generated = output.get("ScheduleRef");
+ assertFalse(generated.equals("unused-" + suffix));
+ JsonNode schedule = new ObjectMapper().readTree(getSchedule(generated, "default")
+ .then().statusCode(200).extract().asString());
+ assertEquals(output.get("ScheduleArn"), schedule.path("Arn").asText());
+ assertEquals("default", schedule.path("GroupName").asText());
+ assertEquals("ENABLED", schedule.path("State").asText());
+ assertOptionalScalarsAbsent(schedule);
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void conditionalNoValueNestedTargetAndWindowPropertiesAreOmitted() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-nested-no-value-" + suffix;
+ String name = "nested-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ try {
+ cloudFormation(stack, "CreateStack", conditionalScheduleTemplate(queue, false, false, true),
+ Map.of("Name", name, "UseOptions", "false"));
+ Map<String, String> output = outputs(stack, "CREATE_COMPLETE");
+ JsonNode schedule = new ObjectMapper().readTree(getSchedule(name, "default")
+ .then().statusCode(200).extract().asString());
+ assertEquals(name, output.get("ScheduleRef"));
+ assertEquals(output.get("ScheduleArn"), schedule.path("Arn").asText());
+ assertEquals("OFF", schedule.path("FlexibleTimeWindow").path("Mode").asText());
+ assertFalse(schedule.path("FlexibleTimeWindow").has("MaximumWindowInMinutes"));
+ assertFalse(schedule.path("Target").has("RetryPolicy"));
+ assertFalse(schedule.path("Target").has("DeadLetterConfig"));
+ assertFalse(schedule.path("Target").has("Input"));
+ assertEquals(sqs.getQueueAttributes(queue.getQueueUrl(), List.of("QueueArn"), "us-east-1")
+ .get("QueueArn"), schedule.path("Target").path("Arn").asText());
+ assertTrue(schedule.path("Target").path("RoleArn").asText().endsWith(":role/scheduler"));
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void conditionalUpdatesRemoveAndRestoreOptionalPropertiesInPlace() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-conditional-update-" + suffix;
+ String name = "conditional-update-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ String conditional = conditionalScheduleTemplate(queue, false, true, true);
+ try {
+ cloudFormation(stack, "CreateStack", conditional, Map.of("Name", name, "UseOptions", "true"));
+ Map<String, String> created = outputs(stack, "CREATE_COMPLETE");
+ JsonNode before = new ObjectMapper().readTree(getSchedule(name, "default")
+ .then().statusCode(200).extract().asString());
+ assertEquals("conditional", before.path("Description").asText());
+ assertEquals(15, before.path("FlexibleTimeWindow").path("MaximumWindowInMinutes").asInt());
+ assertEquals(2, before.path("Target").path("RetryPolicy").path("MaximumRetryAttempts").asInt());
+
+ cloudFormation(stack, "UpdateStack", conditional, Map.of("Name", name, "UseOptions", "false"));
+ assertEquals(created, outputs(stack, "UPDATE_COMPLETE"));
+ JsonNode omitted = new ObjectMapper().readTree(getSchedule(name, "default")
+ .then().statusCode(200).extract().asString());
+ assertEquals("ENABLED", omitted.path("State").asText());
+ assertOptionalScalarsAbsent(omitted);
+ assertFalse(omitted.path("Target").has("Input"));
+ assertFalse(omitted.path("Target").has("RetryPolicy"));
+ assertFalse(omitted.path("Target").has("DeadLetterConfig"));
+ assertEquals("OFF", omitted.path("FlexibleTimeWindow").path("Mode").asText());
+ assertFalse(omitted.path("FlexibleTimeWindow").has("MaximumWindowInMinutes"));
+
+ cloudFormation(stack, "UpdateStack", conditional, Map.of("Name", name, "UseOptions", "true"));
+ assertEquals(created, outputs(stack, "UPDATE_COMPLETE"));
+ JsonNode restored = new ObjectMapper().readTree(getSchedule(name, "default")
+ .then().statusCode(200).extract().asString());
+ assertEquals(before.path("Target"), restored.path("Target"));
+ assertEquals(before.path("FlexibleTimeWindow"), restored.path("FlexibleTimeWindow"));
+ for (String field : List.of("Description", "State", "ScheduleExpressionTimezone", "StartDate",
+ "EndDate", "KmsKeyArn")) {
+ assertEquals(before.get(field), restored.get(field), field);
+ }
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @Test
+ void conditionalAddressUpdatesReplaceAndCleanOldSchedules() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-conditional-address-" + suffix;
+ String name = "conditional-address-" + suffix;
+ String group = "conditional-group-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ createGroup(group);
+ String conditional = conditionalScheduleTemplate(queue, true, false, false);
+ try {
+ cloudFormation(stack, "CreateStack", conditional,
+ Map.of("Name", name, "Group", group, "UseOptions", "true"));
+ Map<String, String> named = outputs(stack, "CREATE_COMPLETE");
+ getSchedule(name, group).then().statusCode(200).body("Arn", equalTo(named.get("ScheduleArn")));
+
+ cloudFormation(stack, "UpdateStack", conditional,
+ Map.of("Name", name, "Group", group, "UseOptions", "false"));
+ Map<String, String> generated = outputs(stack, "UPDATE_COMPLETE");
+ String generatedName = generated.get("ScheduleRef");
+ assertFalse(name.equals(generatedName));
+ getSchedule(name, group).then().statusCode(404);
+ getSchedule(generatedName, "default").then().statusCode(200)
+ .body("Arn", equalTo(generated.get("ScheduleArn")));
+
+ cloudFormation(stack, "UpdateStack", conditional,
+ Map.of("Name", name, "Group", group, "UseOptions", "false", "Payload", "second"));
+ assertEquals(generated, outputs(stack, "UPDATE_COMPLETE"));
+ getSchedule(generatedName, "default").then().statusCode(200).body("Target.Input", equalTo("payload:second"));
+
+ cloudFormation(stack, "UpdateStack", conditional,
+ Map.of("Name", name, "Group", group, "UseOptions", "true"));
+ assertEquals(named, outputs(stack, "UPDATE_COMPLETE"));
+ getSchedule(generatedName, "default").then().statusCode(404);
+ getSchedule(name, group).then().statusCode(200);
+ } finally {
+ deleteStack(stack);
+ getSchedule(name, group).then().statusCode(404);
+ deleteGroup(group);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"Target", "FlexibleTimeWindow", "ScheduleExpression", "Target.Arn", "Target.RoleArn",
+ "FlexibleTimeWindow.Mode"})
+ void requiredPropertiesSelectedAsNoValueStillFail(String property) throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-required-no-value-" + suffix;
+ String name = "required-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ ObjectMapper mapper = new ObjectMapper();
+ ObjectNode root = (ObjectNode) mapper.readTree(conditionalScheduleTemplate(queue, false, false, false));
+ ObjectNode props = (ObjectNode) root.path("Resources").path("Schedule").path("Properties");
+ String[] path = property.split("\\.");
+ ObjectNode parent = path.length == 1 ? props : (ObjectNode) props.get(path[0]);
+ String field = path[path.length - 1];
+ parent.set(field, conditionalOption(parent.get(field)));
+ try {
+ cloudFormation(stack, "CreateStack", root.toString(), Map.of("Name", name, "UseOptions", "false"));
+ CfnStackWaits.StackState state = CfnStackWaits.awaitTerminal(stack);
+ assertEquals("ROLLBACK_COMPLETE", state.status(), property);
+ assertTrue(state.reason().toLowerCase(Locale.ROOT).contains(field.toLowerCase(Locale.ROOT)), state.reason());
+ getSchedule(name, "default").then().statusCode(404);
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"Name", "GroupName", "State", "StartDate", "EndDate", "ScheduleExpression"})
+ void literalEmptyInvalidScalarsAreNotTreatedAsNoValue(String property) throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-empty-control-" + suffix;
+ String name = "empty-control-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ ObjectMapper mapper = new ObjectMapper();
+ ObjectNode root = (ObjectNode) mapper.readTree(externalQueueTemplate("", queue));
+ ((ObjectNode) root.path("Resources").path("Schedule").path("Properties")).put(property, "");
+ try {
+ cloudFormation(stack, "CreateStack", root.toString(), Map.of("Name", name));
+ CfnStackWaits.StackState state = CfnStackWaits.awaitTerminal(stack);
+ assertEquals("ROLLBACK_COMPLETE", state.status(), property);
+ assertTrue(state.reason().contains(property), state.reason());
+ getSchedule(name, "default").then().statusCode(404);
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"Name", "GroupName", "Description", "Target.Input"})
+ void conditionalObjectAndArrayScalarsAreRejected(String property) throws Exception {
+ ObjectMapper mapper = new ObjectMapper();
+ for (JsonNode invalid : List.of(mapper.createObjectNode(), mapper.createArrayNode())) {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-shape-control-" + suffix;
+ String name = "shape-control-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ ObjectNode root = (ObjectNode) mapper.readTree(conditionalScheduleTemplate(queue, false, false, false));
+ ObjectNode props = (ObjectNode) root.path("Resources").path("Schedule").path("Properties");
+ String[] path = property.split("\\.");
+ ObjectNode parent = path.length == 1 ? props : (ObjectNode) props.get(path[0]);
+ parent.set(path[path.length - 1], conditionalOption(invalid));
+ try {
+ cloudFormation(stack, "CreateStack", root.toString(), Map.of("Name", name, "UseOptions", "true"));
+ assertEquals("ROLLBACK_COMPLETE", CfnStackWaits.awaitTerminal(stack).status(), property + invalid);
+ getSchedule(name, "default").then().statusCode(404);
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+ }
+
+ @Test
+ void literalEmptyDescriptionAndInputRemainPresent() throws Exception {
+ String suffix = Long.toString(System.nanoTime(), 36);
+ String stack = "cfn-schedule-empty-allowed-" + suffix;
+ String name = "empty-allowed-" + suffix;
+ Queue queue = sqs.createQueue("conditional-" + suffix, Map.of(), "us-east-1");
+ ObjectMapper mapper = new ObjectMapper();
+ ObjectNode root = (ObjectNode) mapper.readTree(externalQueueTemplate("", queue));
+ ObjectNode props = (ObjectNode) root.path("Resources").path("Schedule").path("Properties");
+ props.put("Description", "");
+ ((ObjectNode) props.get("Target")).put("Input", "");
+ try {
+ cloudFormation(stack, "CreateStack", root.toString(), Map.of("Name", name));
+ outputs(stack, "CREATE_COMPLETE");
+ JsonNode schedule = mapper.readTree(getSchedule(name, "default").then().statusCode(200).extract().asString());
+ assertTrue(schedule.has("Description"));
+ assertEquals("", schedule.path("Description").asText());
+ assertTrue(schedule.path("Target").has("Input"));
+ assertEquals("", schedule.path("Target").path("Input").asText());
+ } finally {
+ deleteStack(stack);
+ sqs.deleteQueue(queue.getQueueUrl(), "us-east-1");
+ }
+ }
+
+ private static void assertOptionalScalarsAbsent(JsonNode schedule) {
+ for (String field : List.of("Description", "ScheduleExpressionTimezone", "StartDate", "EndDate", "KmsKeyArn")) {
+ assertFalse(schedule.has(field), field);
+ }
+ }
+
+ private String conditionalScheduleTemplate(Queue queue, boolean conditionalAddress,
+ boolean scalarOptions, boolean nestedOptions) throws Exception {
+ ObjectMapper mapper = new ObjectMapper();
+ ObjectNode root = (ObjectNode) mapper.readTree(externalQueueTemplate("", queue));
+ ((ObjectNode) root.path("Parameters")).putObject("UseOptions").put("Type", "String").put("Default", "true");
+ root.putObject("Conditions").set("UseOptions", mapper.readTree("""
+ {"Fn::Equals":[{"Ref":"UseOptions"},"true"]}
+ """));
+ ObjectNode props = (ObjectNode) root.path("Resources").path("Schedule").path("Properties");
+ props.put("ScheduleExpression", "at(2100-01-01T00:00:00)");
+ if (conditionalAddress) {
+ props.set("Name", conditionalOption(mapper.readTree("{\"Ref\":\"Name\"}")));
+ props.set("GroupName", conditionalOption(mapper.readTree("{\"Ref\":\"Group\"}")));
+ }
+ if (scalarOptions) {
+ for (Map.Entry<String, String> field : Map.of("Description", "conditional", "State", "DISABLED",
+ "ScheduleExpressionTimezone", "UTC", "StartDate", "2099-01-01T00:00:00Z",
+ "EndDate", "2101-01-01T00:00:00Z").entrySet()) {
+ props.set(field.getKey(), conditionalOption(mapper.valueToTree(field.getValue())));
+ }
+ props.set("KmsKeyArn", conditionalOption(mapper.readTree("""
+ {"Fn::Sub":"arn:${AWS::Partition}:kms:${AWS::Region}:${AWS::AccountId}:key/conditional"}
+ """)));
+ }
+ if (nestedOptions) {
+ ObjectNode window = (ObjectNode) props.get("FlexibleTimeWindow");
+ window.set("Mode", mapper.readTree("{\"Fn::If\":[\"UseOptions\",\"FLEXIBLE\",\"OFF\"]}"));
+ window.set("MaximumWindowInMinutes", conditionalOption(mapper.valueToTree(15)));
+ ObjectNode target = (ObjectNode) props.get("Target");
+ target.set("Input", conditionalOption(target.get("Input")));
+ target.set("RetryPolicy", conditionalOption(mapper.readTree("{\"MaximumRetryAttempts\":2}")));
+ target.set("DeadLetterConfig", conditionalOption(mapper.createObjectNode().set("Arn", target.get("Arn"))));
+ }
+ return root.toString();
+ }
+
+ private static JsonNode conditionalOption(JsonNode value) {
+ ObjectMapper mapper = new ObjectMapper();
+ ObjectNode noValue = mapper.createObjectNode().put("Ref", "AWS::NoValue");
+ return mapper.createObjectNode().set("Fn::If", mapper.createArrayNode().add("UseOptions").add(value).add(noValue));
+ }
+
+ private void prepareFailedRollback(String stack, String name, Queue queue) {
+ cloudFormation(stack, "CreateStack", externalQueueTemplate("", queue), Map.of("Name", name));
+ outputs(stack, "CREATE_COMPLETE");
+ doThrow(new AwsException("InternalServerException", "original restore unavailable", 500))
+ .when(scheduler).updateSchedule(argThat(request -> name.equals(request.getName())
+ && "rate(5 minutes)".equals(request.getScheduleExpression())), eq("us-east-1"));
+ cloudFormation(stack, "UpdateStack", externalQueueTemplate(FAILURE, queue),
+ Map.of("Name", name, "Expression", "rate(10 minutes)", "Payload", "changed"));
+ outputs(stack, "UPDATE_ROLLBACK_FAILED");
+ getSchedule(name, "default").then().statusCode(200)
+ .body("ScheduleExpression", equalTo("rate(10 minutes)"));
+ assertTrue(scheduleResource(stack).getAttributes().containsKey("__FlociSchedulerUpdateSnapshot"));
+ }
+
+ private void allowOriginalRestore(String name) {
+ doCallRealMethod().when(scheduler).updateSchedule(argThat(request -> name.equals(request.getName())
+ && "rate(5 minutes)".equals(request.getScheduleExpression())), eq("us-east-1"));
+ }
+
+ private StackResource scheduleResource(String stack) {
+ return cloudFormation.describeStacks(stack, "us-east-1").getFirst().getResources().get("Schedule");
+ }
+
+ private static String template(String policy, String name, String failure) {
+ return TEMPLATE.formatted(policy, name, failure);
+ }
+
+ private String externalQueueTemplate(String failure, Queue queue) {
+ // An externally managed target keeps these failure-path tests focused on the schedule's
+ // own rollback contract rather than a sibling queue's update lifecycle.
+ try {
+ ObjectNode template = (ObjectNode) new ObjectMapper().readTree(template("Delete", NAME, failure));
+ ObjectNode resources = (ObjectNode) template.get("Resources");
+ resources.remove("Queue");
+ ObjectNode target = (ObjectNode) resources.path("Schedule").path("Properties").path("Target");
+ target.put("Arn", sqs.getQueueAttributes(queue.getQueueUrl(), List.of("QueueArn"), "us-east-1")
+ .get("QueueArn"));
+ ((ObjectNode) template.path("Outputs").path("QueueUrl")).put("Value", queue.getQueueUrl());
+ return template.toString();
+ } catch (Exception e) {
+ throw new IllegalStateException("Could not prepare external-target template", e);
+ }
+ }
+
+ private static void cloudFormation(String stack, String action, String template, Map<String, String> parameters) {
+ RequestSpecification request = given().contentType("application/x-www-form-urlencoded")
+ .header("Authorization", AUTH).formParam("Action", action).formParam("StackName", stack);
+ if (template != null) {
+ request.formParam("TemplateBody", template);
+ }
+ int index = 1;
+ for (Map.Entry<String, String> parameter : parameters.entrySet()) {
+ request.formParam("Parameters.member." + index + ".ParameterKey", parameter.getKey());
+ request.formParam("Parameters.member." + index + ".ParameterValue", parameter.getValue());
+ index++;
+ }
+ request.post("/").then().statusCode(200);
+ }
+
+ private static Map<String, String> outputs(String stack, String expectedStatus) {
+ assertEquals(expectedStatus, CfnStackWaits.awaitTerminal(stack).status());
+ String body = given().contentType("application/x-www-form-urlencoded").header("Authorization", AUTH)
+ .formParam("Action", "DescribeStacks").formParam("StackName", stack)
+ .post("/").then().statusCode(200).extract().asString();
+ return XmlParser.extractPairs(body, "Outputs", "OutputKey", "OutputValue");
+ }
+
+ private static Response getSchedule(String name, String group) {
+ return given().queryParam("groupName", group).get("/schedules/" + name);
+ }
+
+ private static void recreateScheduleViaApi(String name, String group) throws Exception {
+ recreateScheduleViaApi(name, group, "DISABLED");
+ }
+
+ private static void recreateScheduleViaApi(String name, String group, String state) throws Exception {
+ ObjectMapper mapper = new ObjectMapper();
+ JsonNode original = mapper.readTree(getSchedule(name, group).then().statusCode(200).extract().asString());
+ ObjectNode target = original.path("Target").deepCopy();
+ target.put("Input", "out-of-band");
+ ObjectNode request = mapper.createObjectNode().put("GroupName", group)
+ .put("ScheduleExpression", "rate(30 minutes)").put("State", state);
+ request.set("Target", target);
+ request.set("FlexibleTimeWindow", mapper.createObjectNode().put("Mode", "OFF"));
+ given().queryParam("groupName", group).delete("/schedules/" + name).then().statusCode(200);
+ given().contentType("application/json").body(request.toString()).post("/schedules/" + name)
+ .then().statusCode(200);
+ }
+
+ private static void createGroup(String group) {
+ given().contentType("application/json").body("{}").post("/schedule-groups/" + group)
+ .then().statusCode(200);
+ }
+
+ private static void deleteGroup(String group) {
+ given().delete("/schedule-groups/" + group).then().statusCode(200);
+ }
+
+ private static void deleteStack(String stack) {
+ cloudFormation(stack, "DeleteStack", null, Map.of());
+ CfnStackWaits.awaitStackDeleted(stack);
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/ReplacementCleanupTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/ReplacementCleanupTest.java
index 9b04445d4d..648029b6c4 100644
--- a/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/ReplacementCleanupTest.java
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/ReplacementCleanupTest.java
@@ -190,6 +190,27 @@ class ReplacementCleanupTest {
assertFalse(ReplacementCleanup.hasReplacement(previous));
}
+ @Test
+ void committedUpdateStillStopsAfterThreeFailuresAndAbandonsTheExhaustedEntity() {
+ StackResource r = resource("subnet-prior");
+ List<String> attempted = new ArrayList<>();
+ ReplacementCleanup.recordOrphan(r, "subnet-replacement", r.getResourceType(), "us-east-1");
+ ReplacementCleanup.Deleter unavailable = (type, id, region) -> {
+ attempted.add(id);
+ throw new IllegalStateException("still in use");
+ };
+
+ for (int attempt = 1; attempt <= 4; attempt++) {
+ UpdateCleanupResult result = ReplacementCleanup.complete(r, unavailable);
+ assertFalse(result.complete());
+ assertEquals(Math.min(attempt, 3), result.attempts());
+ }
+ assertEquals(3, attempted.size());
+ ReplacementCleanup.clear(r);
+ assertFalse(ReplacementCleanup.hasReplacement(r));
+ assertFalse(ReplacementCleanup.complete(r, unavailable).applicable());
+ }
+
@Test
void mergingNothingLeavesTheTargetUntouched() {
StackResource previous = resource("subnet-prior");
diff --git a/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/SchedulerScheduleCfnProvisionerTest.java b/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/SchedulerScheduleCfnProvisionerTest.java
new file mode 100644
index 0000000000..ee560a4a61
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/cloudformation/provisioners/SchedulerScheduleCfnProvisionerTest.java
@@ -0,0 +1,1048 @@
+package io.github.hectorvent.floci.services.cloudformation.provisioners;
+
+import com.fasterxml.jackson.databind.JsonNode;
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.databind.node.ObjectNode;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.services.cloudformation.CloudFormationTemplateEngine;
+import io.github.hectorvent.floci.services.cloudformation.model.StackResource;
+import io.github.hectorvent.floci.services.scheduler.SchedulerService;
+import io.github.hectorvent.floci.services.scheduler.model.Schedule;
+import io.github.hectorvent.floci.services.scheduler.model.ScheduleRequest;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.params.ParameterizedTest;
+import org.junit.jupiter.params.provider.ValueSource;
+import org.mockito.ArgumentCaptor;
+import org.mockito.InOrder;
+
+import java.time.Instant;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.concurrent.atomic.AtomicReference;
+
+import static org.junit.jupiter.api.Assertions.assertAll;
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertFalse;
+import static org.junit.jupiter.api.Assertions.assertNotEquals;
+import static org.junit.jupiter.api.Assertions.assertNull;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+import static org.junit.jupiter.api.Assertions.assertTrue;
+import static org.mockito.ArgumentMatchers.any;
+import static org.mockito.ArgumentMatchers.argThat;
+import static org.mockito.ArgumentMatchers.eq;
+import static org.mockito.Mockito.clearInvocations;
+import static org.mockito.Mockito.doNothing;
+import static org.mockito.Mockito.doThrow;
+import static org.mockito.Mockito.inOrder;
+import static org.mockito.Mockito.mock;
+import static org.mockito.Mockito.never;
+import static org.mockito.Mockito.times;
+import static org.mockito.Mockito.verify;
+import static org.mockito.Mockito.verifyNoInteractions;
+import static org.mockito.Mockito.when;
+
+class SchedulerScheduleCfnProvisionerTest {
+
+ private static final String REGION = "us-east-1";
+ private final ObjectMapper mapper = new ObjectMapper();
+ private final SchedulerService scheduler = mock(SchedulerService.class);
+ private final SchedulerScheduleCfnProvisioner provisioner = new SchedulerScheduleCfnProvisioner(scheduler);
+
+ @BeforeEach
+ void returnCreatedSchedules() {
+ Map<String, Schedule> created = new HashMap<>();
+ when(scheduler.getSchedule(any(), any(), eq(REGION))).thenAnswer(inv ->
+ created.get(inv.getArgument(1) + "/" + inv.getArgument(0)));
+ when(scheduler.createSchedule(any(), eq(REGION))).thenAnswer(inv -> {
+ Schedule result = schedule(inv.getArgument(0));
+ created.put(result.getGroupName() + "/" + result.getName(), result);
+ return result;
+ });
+ when(scheduler.updateSchedule(any(), eq(REGION))).thenAnswer(inv -> schedule(inv.getArgument(0)));
+ }
+
+ @Test
+ void mapsPropertiesAndKeepsNestedTargetValues() throws Exception {
+ ObjectNode props = properties("schedule", "group");
+ props.put("Description", "description");
+ props.put("ScheduleExpressionTimezone", "Europe/Paris");
+ props.put("StartDate", "2030-01-01T00:00:00Z");
+ props.put("EndDate", "2031-01-01T00:00:00Z");
+ props.put("KmsKeyArn", "arn:aws:kms:us-east-1:000000000000:key/test");
+ props.put("State", "DISABLED");
+ ObjectNode target = (ObjectNode) props.get("Target");
+ target.putObject("RetryPolicy").put("MaximumRetryAttempts", 3).put("MaximumEventAgeInSeconds", 600);
+ target.putObject("DeadLetterConfig").put("Arn", "arn:aws:sqs:us-east-1:000000000000:dlq");
+ target.putObject("SqsParameters").put("MessageGroupId", "orders");
+ target.putObject("EcsParameters").put("TaskDefinitionArn", "task")
+ .putObject("NetworkConfiguration").putObject("AwsVpcConfiguration")
+ .putArray("Subnets").add("subnet-a").add("subnet-b");
+ StackResource resource = resource();
+
+ provisioner.provision(resource, props, context(null));
+
+ ArgumentCaptor<ScheduleRequest> request = ArgumentCaptor.forClass(ScheduleRequest.class);
+ verify(scheduler).createSchedule(request.capture(), eq(REGION));
+ assertEquals(Instant.parse("2030-01-01T00:00:00Z"), request.getValue().getStartDate());
+ assertEquals(Instant.parse("2031-01-01T00:00:00Z"), request.getValue().getEndDate());
+ assertEquals("description", request.getValue().getDescription());
+ assertEquals("Europe/Paris", request.getValue().getScheduleExpressionTimezone());
+ assertEquals(props.get("KmsKeyArn").asText(), request.getValue().getKmsKeyArn());
+ assertEquals("DISABLED", request.getValue().getState());
+ assertEquals(3, request.getValue().getTarget().getRetryPolicy().getMaximumRetryAttempts());
+ assertEquals("orders", request.getValue().getTarget().getSqsParameters().getMessageGroupId());
+ assertEquals(2, request.getValue().getTarget().getEcsParameters().getNetworkConfiguration()
+ .getAwsvpcConfiguration().getSubnets().size());
+ assertEquals("schedule", resource.getPhysicalId());
+ assertEquals("arn:aws:scheduler:us-east-1:000000000000:schedule/group/schedule",
+ resource.getAttributes().get("Arn"));
+ }
+
+ @Test
+ void unnamedScheduleKeepsItsGeneratedNameOnUpdate() throws Exception {
+ ObjectNode props = properties(null, null);
+ StackResource resource = resource();
+ provisioner.provision(resource, props, context(null));
+ String generated = resource.getPhysicalId();
+ assertTrue(generated.startsWith("stack-Schedule-"));
+ assertTrue(generated.length() <= 64);
+ when(scheduler.getSchedule(generated, "default", REGION))
+ .thenReturn(schedule(request(generated, "default", "rate(5 minutes)")));
+
+ props.put("ScheduleExpression", "rate(10 minutes)");
+ provisioner.provision(resource, props, context(generated));
+ provisioner.completeUpdate(resource);
+
+ assertEquals(generated, resource.getPhysicalId());
+ verify(scheduler, times(1)).createSchedule(any(), eq(REGION));
+ verify(scheduler).updateSchedule(any(), eq(REGION));
+ assertFalse(resource.getAttributes().containsKey("__FlociSchedulerUpdateSnapshot"));
+ }
+
+ @Test
+ void removingExplicitNameGeneratesReplacementAndDefersOldDeletion() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("old", "default"), context(null));
+
+ provisioner.provision(resource, properties(null, "default"), context("old"));
+
+ String generated = resource.getPhysicalId();
+ assertNotEquals("old", generated);
+ assertTrue(generated.startsWith("stack-Schedule-"));
+ assertTrue(generated.length() <= 64);
+ assertTrue(resource.getAttributes().get("Arn").endsWith("/default/" + generated));
+ verify(scheduler).createSchedule(argThat(r -> generated.equals(r.getName())), eq(REGION));
+ verify(scheduler, never()).deleteSchedule("old", "default", REGION);
+ assertEquals("default/old", provisioner.updateCleanupPhysicalId(resource));
+
+ assertTrue(provisioner.completeUpdate(resource).complete());
+
+ verify(scheduler).deleteSchedule("old", "default", REGION);
+ verify(scheduler, never()).deleteSchedule(generated, "default", REGION);
+ verify(scheduler, never()).updateSchedule(any(), eq(REGION));
+ }
+
+ @Test
+ void nameReplacementWaitsForCommitAndSupportsRollback() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("old", "a"), context(null));
+ provisioner.provision(resource, properties("new", "b"), context("old"));
+
+ verify(scheduler, never()).deleteSchedule("old", "a", REGION);
+ assertEquals("a/old", provisioner.updateCleanupPhysicalId(resource));
+ assertTrue(provisioner.rollbackUpdate(resource));
+
+ assertEquals("old", resource.getPhysicalId());
+ assertTrue(resource.getAttributes().get("Arn").endsWith("/a/old"));
+ verify(scheduler).deleteSchedule("new", "b", REGION);
+ verify(scheduler, never()).deleteSchedule("old", "a", REGION);
+ assertEquals("a", resource.getAttributes().get("FlociSchedulerGroupName"));
+ assertFalse(resource.getAttributes().containsKey("__FlociReplacementCleanup"));
+ }
+
+ @Test
+ void inPlaceUpdateUsesRecreatedScheduleAtItsAddressAndRollsBackTheObservedConfiguration() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ Schedule recreated = schedule(request("same", "a", "rate(30 minutes)"));
+ recreated.setState("DISABLED");
+ when(scheduler.getSchedule("same", "a", REGION)).thenReturn(recreated);
+
+ provisioner.provision(resource, properties("same", "a"), context("same"));
+
+ verify(scheduler).updateSchedule(argThat(request -> "rate(5 minutes)".equals(request.getScheduleExpression())
+ && "same".equals(request.getName()) && "a".equals(request.getGroupName())), eq(REGION));
+ assertTrue(provisioner.rollbackUpdate(resource));
+ verify(scheduler).updateSchedule(argThat(request -> "rate(30 minutes)".equals(request.getScheduleExpression())
+ && "DISABLED".equals(request.getState())), eq(REGION));
+ assertEquals("same", resource.getPhysicalId());
+ assertFalse(provisioner.retainsFailedUpdateState(resource));
+ }
+
+ @Test
+ void nameReplacementCreatesDestinationWhenTheOldAddressIsAlreadyGone() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("old", "a"), context(null));
+ doThrow(new AwsException("ResourceNotFoundException", "current schedule absent", 404))
+ .when(scheduler).deleteSchedule("old", "a", REGION);
+
+ provisioner.provision(resource, properties("new", "b"), context("old"));
+
+ assertEquals("new", resource.getPhysicalId());
+ assertTrue(provisioner.completeUpdate(resource).complete());
+ verify(scheduler).createSchedule(argThat(request -> "new".equals(request.getName())), eq(REGION));
+ verify(scheduler).deleteSchedule("old", "a", REGION);
+ verify(scheduler, never()).getSchedule("old", "a", REGION);
+ }
+
+ @Test
+ void committedRetainReplacementKeepsTheOldAddressAndClearsCleanupTracking() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("old", "a"), context(null));
+ resource.setUpdateReplacePolicy("Retain");
+ provisioner.provision(resource, properties("new", "b"), context("old"));
+ JsonNode cleanup = mapper.readTree(resource.getAttributes().get("__FlociReplacementCleanup"));
+ assertEquals("a/old", cleanup.path("priorPhysicalId").asText());
+ assertTrue(cleanup.path("displaced").get(0).path("retainable").asBoolean());
+ assertNull(provisioner.updateCleanupPhysicalId(resource));
+
+ assertTrue(provisioner.completeUpdate(resource).complete());
+ provisioner.clearUpdate(resource);
+
+ assertFalse(resource.getAttributes().containsKey("__FlociReplacementCleanup"));
+ assertEquals("new", resource.getPhysicalId());
+ assertEquals("b", resource.getAttributes().get("FlociSchedulerGroupName"));
+ verify(scheduler, never()).deleteSchedule("old", "a", REGION);
+ }
+
+ @Test
+ void failedReplacementMergeKeepsTheOrphansAddressForDeletionRetry() throws Exception {
+ StackResource previous = resource();
+ provisioner.provision(previous, properties("old", "a"), context(null));
+ StackResource attempted = resource();
+ attempted.setPhysicalId(previous.getPhysicalId());
+ attempted.setAttributes(new HashMap<>(previous.getAttributes()));
+ provisioner.provision(attempted, properties("new", "b"), context("old"));
+ doThrow(new AwsException("InternalServerException", "replacement deletion unavailable", 500))
+ .when(scheduler).deleteSchedule("new", "b", REGION);
+ assertThrows(AwsException.class, () -> provisioner.rollbackUpdate(attempted));
+
+ provisioner.mergeFailedUpdateResourceTracking(previous, attempted);
+
+ assertEquals("old", previous.getPhysicalId());
+ assertEquals("b/new", provisioner.updateCleanupPhysicalId(previous));
+ assertFalse(provisioner.completeDeleteCleanup(previous).complete());
+ assertEquals("b/new", provisioner.updateCleanupPhysicalId(previous));
+ doNothing().when(scheduler).deleteSchedule("new", "b", REGION);
+ assertTrue(provisioner.completeDeleteCleanup(previous).complete());
+ verify(scheduler, times(3)).deleteSchedule("new", "b", REGION);
+ verify(scheduler, never()).deleteSchedule("old", "a", REGION);
+ }
+
+ @Test
+ void recreatingAnOldOrphanStartsFreshCleanupAndCanMergeTheFailedAttempt() throws Exception {
+ StackResource previous = resource();
+ provisioner.provision(previous, properties("current", "default"), context(null));
+ ReplacementCleanup.recordOrphan(previous, "default/reused", previous.getResourceType(), REGION);
+ ObjectNode oldCleanup = (ObjectNode) mapper.readTree(previous.getAttributes().get("__FlociReplacementCleanup"));
+ ((ObjectNode) oldCleanup.path("displaced").get(0)).put("cleanupAttempts", 3);
+ previous.getAttributes().put("__FlociReplacementCleanup", oldCleanup.toString());
+ StackResource attempted = resource();
+ attempted.setPhysicalId(previous.getPhysicalId());
+ attempted.setAttributes(new HashMap<>(previous.getAttributes()));
+ provisioner.provision(attempted, properties("reused", "default"), context("current"));
+ doThrow(new AwsException("InternalServerException", "replacement delete unavailable", 500))
+ .when(scheduler).deleteSchedule("reused", "default", REGION);
+ assertThrows(AwsException.class, () -> provisioner.rollbackUpdate(attempted));
+
+ JsonNode newCleanup = mapper.readTree(attempted.getAttributes().get("__FlociReplacementCleanup"));
+ assertEquals(0, newCleanup.path("displaced").get(0).path("cleanupAttempts").asInt());
+ provisioner.mergeFailedUpdateResourceTracking(previous, attempted);
+ JsonNode merged = mapper.readTree(previous.getAttributes().get("__FlociReplacementCleanup"));
+ assertEquals(1, merged.path("displaced").size());
+ assertEquals("default/reused", merged.path("displaced").get(0).path("physicalId").asText());
+ assertEquals(0, merged.path("displaced").get(0).path("cleanupAttempts").asInt());
+ doNothing().when(scheduler).deleteSchedule("reused", "default", REGION);
+ assertTrue(provisioner.completeDeleteCleanup(previous).complete());
+ verify(scheduler, times(2)).deleteSchedule("reused", "default", REGION);
+ }
+
+ @Test
+ void mergingCopiedCleanupDebtPreservesItsExhaustedAttemptCount() throws Exception {
+ StackResource previous = resource();
+ provisioner.provision(previous, properties("current", "default"), context(null));
+ ReplacementCleanup.recordOrphan(previous, "default/unrelated", previous.getResourceType(), REGION);
+ ObjectNode cleanup = (ObjectNode) mapper.readTree(previous.getAttributes().get("__FlociReplacementCleanup"));
+ ((ObjectNode) cleanup.path("displaced").get(0)).put("cleanupAttempts", 3);
+ previous.getAttributes().put("__FlociReplacementCleanup", cleanup.toString());
+ StackResource attempted = resource();
+ attempted.setPhysicalId(previous.getPhysicalId());
+ attempted.setAttributes(new HashMap<>(previous.getAttributes()));
+
+ provisioner.mergeFailedUpdateResourceTracking(previous, attempted);
+
+ JsonNode merged = mapper.readTree(previous.getAttributes().get("__FlociReplacementCleanup"));
+ assertEquals(1, merged.path("displaced").size());
+ assertEquals("default/unrelated", merged.path("displaced").get(0).path("physicalId").asText());
+ assertEquals(3, merged.path("displaced").get(0).path("cleanupAttempts").asInt());
+ verify(scheduler, never()).deleteSchedule(any(), any(), eq(REGION));
+ }
+
+ @Test
+ void malformedCleanupRejectsReplacementBeforeCreatingADestination() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "default"), context(null));
+ resource.getAttributes().put("__FlociReplacementCleanup", "{\"displaced\":{}}");
+ Map<String, String> before = Map.copyOf(resource.getAttributes());
+ ObjectNode desired = properties("new", "default");
+
+ assertThrows(IllegalStateException.class, () -> provisioner.provision(resource, desired, context("current")));
+
+ assertEquals("current", resource.getPhysicalId());
+ assertEquals(before, resource.getAttributes());
+ verify(scheduler, never()).createSchedule(argThat(request -> "new".equals(request.getName())), eq(REGION));
+ verify(scheduler, never()).updateSchedule(any(), eq(REGION));
+ verify(scheduler, never()).deleteSchedule(any(), any(), eq(REGION));
+ }
+
+ @Test
+ void malformedCleanupRejectsInPlaceUpdateBeforeWritingTheSchedule() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "default"), context(null));
+ resource.getAttributes().put("__FlociReplacementCleanup", "{\"displaced\":{}}");
+ Map<String, String> before = Map.copyOf(resource.getAttributes());
+ ObjectNode desired = properties("current", "default").put("ScheduleExpression", "rate(10 minutes)");
+
+ assertThrows(IllegalStateException.class, () -> provisioner.provision(resource, desired, context("current")));
+
+ assertEquals("current", resource.getPhysicalId());
+ assertEquals(before, resource.getAttributes());
+ verify(scheduler, times(1)).createSchedule(any(), eq(REGION));
+ verify(scheduler, never()).updateSchedule(any(), eq(REGION));
+ verify(scheduler, never()).deleteSchedule(any(), any(), eq(REGION));
+ }
+
+ @Test
+ void malformedCleanupDoesNotPartiallyChangeThePreviousResourceOnMerge() throws Exception {
+ StackResource previous = resource();
+ provisioner.provision(previous, properties("current", "default"), context(null));
+ ReplacementCleanup.recordOrphan(previous, "default/orphan", previous.getResourceType(), REGION);
+ StackResource attempted = resource();
+ attempted.setPhysicalId(previous.getPhysicalId());
+ attempted.setAttributes(new HashMap<>(previous.getAttributes()));
+ attempted.getAttributes().put("__FlociReplacementCleanup", "{\"displaced\":{}}");
+ Map<String, String> before = Map.copyOf(previous.getAttributes());
+
+ assertThrows(IllegalStateException.class, () ->
+ provisioner.mergeFailedUpdateResourceTracking(previous, attempted));
+
+ assertEquals(before, previous.getAttributes());
+ verify(scheduler, never()).deleteSchedule(any(), any(), eq(REGION));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"1", "true", "null", "\"invalid\"", "[]"})
+ void malformedCleanupEntryRejectsCreateBeforeAnyServiceCall(String entry) throws Exception {
+ StackResource resource = resource();
+ appendMalformedCleanupEntry(resource, entry);
+
+ assertMalformedCleanupRejectedWithoutMutation(resource, properties("new", "default"), context(null));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"1", "true", "null", "\"invalid\"", "[]"})
+ void malformedCleanupEntryRejectsReplacementBeforeAnyServiceCall(String entry) throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "default"), context(null));
+ appendMalformedCleanupEntry(resource, entry);
+
+ assertMalformedCleanupRejectedWithoutMutation(resource, properties("new", "default"), context("current"));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"1", "true", "null", "\"invalid\"", "[]"})
+ void malformedCleanupEntryRejectsInPlaceUpdateBeforeAnyServiceCall(String entry) throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "default"), context(null));
+ appendMalformedCleanupEntry(resource, entry);
+
+ assertMalformedCleanupRejectedWithoutMutation(resource,
+ properties("current", "default").put("ScheduleExpression", "rate(10 minutes)"), context("current"));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"1", "true", "null", "\"invalid\"", "[]"})
+ void malformedCleanupEntryPreservesPendingRollbackBeforeAnotherUpdate(String entry) throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "default"), context(null));
+ provisioner.provision(resource,
+ properties("current", "default").put("ScheduleExpression", "rate(10 minutes)"), context("current"));
+ assertTrue(provisioner.retainsFailedUpdateState(resource));
+ appendMalformedCleanupEntry(resource, entry);
+
+ assertMalformedCleanupRejectedWithoutMutation(resource,
+ properties("current", "default").put("ScheduleExpression", "rate(15 minutes)"), context("current"));
+ assertTrue(provisioner.retainsFailedUpdateState(resource));
+ }
+
+ @ParameterizedTest
+ @ValueSource(strings = {"1", "true", "null", "\"invalid\"", "[]"})
+ void malformedCleanupEntryRejectsMergeWithoutChangingExistingDebt(String entry) throws Exception {
+ StackResource previous = resource();
+ provisioner.provision(previous, properties("current", "default"), context(null));
+ ReplacementCleanup.recordOrphan(previous, "default/orphan", previous.getResourceType(), REGION);
+ StackResource attempted = resource();
+ attempted.setPhysicalId(previous.getPhysicalId());
+ attempted.setAttributes(new HashMap<>(previous.getAttributes()));
+ appendMalformedCleanupEntry(attempted, entry);
+ Map<String, String> previousBefore = Map.copyOf(previous.getAttributes());
+ Map<String, String> attemptedBefore = Map.copyOf(attempted.getAttributes());
+ clearInvocations(scheduler);
+
+ assertAll(
+ () -> assertThrows(IllegalStateException.class,
+ () -> provisioner.mergeFailedUpdateResourceTracking(previous, attempted)),
+ () -> assertEquals(previousBefore, previous.getAttributes()),
+ () -> assertEquals(attemptedBefore, attempted.getAttributes()),
+ () -> verifyNoInteractions(scheduler));
+ }
+
+ private void appendMalformedCleanupEntry(StackResource resource, String entry) throws Exception {
+ ReplacementCleanup.recordOrphan(resource, "default/orphan", resource.getResourceType(), REGION);
+ ObjectNode cleanup = (ObjectNode) mapper.readTree(resource.getAttributes().get("__FlociReplacementCleanup"));
+ ((ObjectNode) cleanup.path("displaced").get(0)).put("cleanupAttempts", 3)
+ .put("cleanupFailureReason", "historical deletion failed");
+ cleanup.withArray("displaced").add(mapper.readTree(entry));
+ resource.getAttributes().put("__FlociReplacementCleanup", cleanup.toString());
+ resource.getAttributes().put("ExistingAttribute", "unchanged");
+ }
+
+ private void assertMalformedCleanupRejectedWithoutMutation(StackResource resource, ObjectNode desired,
+ ProvisionContext ctx) {
+ Map<String, String> before = Map.copyOf(resource.getAttributes());
+ String physicalIdBefore = resource.getPhysicalId();
+ clearInvocations(scheduler);
+
+ assertAll(
+ () -> assertThrows(IllegalStateException.class, () -> provisioner.provision(resource, desired, ctx)),
+ () -> assertEquals(physicalIdBefore, resource.getPhysicalId()),
+ () -> assertEquals(before, resource.getAttributes()),
+ () -> verifyNoInteractions(scheduler));
+ }
+
+ @Test
+ void groupMoveKeepsRefAndDoesNotHonorReplacementRetain() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ resource.setUpdateReplacePolicy("Retain");
+ Schedule previous = schedule(request("same", "a", "rate(5 minutes)"));
+ previous.setState("ENABLED");
+ when(scheduler.getSchedule("same", "a", REGION)).thenReturn(previous);
+ provisioner.provision(resource, properties("same", "b"), context("same"));
+
+ InOrder order = inOrder(scheduler);
+ order.verify(scheduler).updateSchedule(argThat(r -> "a".equals(r.getGroupName())
+ && "DISABLED".equals(r.getState())), eq(REGION));
+ order.verify(scheduler).createSchedule(argThat(r -> "b".equals(r.getGroupName())), eq(REGION));
+
+ assertEquals("same", resource.getPhysicalId());
+ assertTrue(provisioner.completeUpdate(resource).complete());
+ provisioner.clearUpdate(resource);
+
+ verify(scheduler).deleteSchedule("same", "a", REGION);
+ verify(scheduler, never()).deleteSchedule("same", "b", REGION);
+ }
+
+ @Test
+ void failedGroupDestinationCreationRestoresOriginalEnabledState() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ Schedule previous = schedule(request("same", "a", "rate(5 minutes)"));
+ previous.setState("ENABLED");
+ when(scheduler.getSchedule("same", "a", REGION)).thenReturn(previous);
+ when(scheduler.createSchedule(argThat(r -> "b".equals(r.getGroupName())), eq(REGION)))
+ .thenThrow(new AwsException("ConflictException", "destination owned elsewhere", 409));
+
+ assertThrows(AwsException.class,
+ () -> provisioner.provision(resource, properties("same", "b"), context("same")));
+
+ ArgumentCaptor<ScheduleRequest> requests = ArgumentCaptor.forClass(ScheduleRequest.class);
+ verify(scheduler, times(2)).updateSchedule(requests.capture(), eq(REGION));
+ assertEquals("DISABLED", requests.getAllValues().getFirst().getState());
+ assertEquals("ENABLED", requests.getAllValues().getLast().getState());
+ verify(scheduler, never()).deleteSchedule("same", "b", REGION);
+ assertEquals("same", resource.getPhysicalId());
+ }
+
+ @Test
+ void failedGroupRollbackDeleteRestoresOriginalAndKeepsDisabledOrphanForCleanup() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ Schedule original = schedule(request("same", "a", "rate(5 minutes)"));
+ original.setState("ENABLED");
+ when(scheduler.getSchedule("same", "a", REGION)).thenReturn(original);
+ provisioner.provision(resource, properties("same", "b"), context("same"));
+ Schedule destination = schedule(request("same", "b", "rate(5 minutes)"));
+ destination.setState("ENABLED");
+ when(scheduler.getSchedule("same", "b", REGION)).thenReturn(destination);
+ doThrow(new AwsException("InternalServerException", "delete unavailable", 500)).doNothing()
+ .when(scheduler).deleteSchedule("same", "b", REGION);
+
+ assertThrows(AwsException.class, () -> provisioner.rollbackUpdate(resource));
+
+ assertEquals("same", resource.getPhysicalId());
+ assertTrue(resource.getAttributes().get("Arn").endsWith("/a/same"));
+ InOrder order = inOrder(scheduler);
+ order.verify(scheduler).updateSchedule(argThat(r -> "b".equals(r.getGroupName())
+ && "DISABLED".equals(r.getState())), eq(REGION));
+ order.verify(scheduler).deleteSchedule("same", "b", REGION);
+ order.verify(scheduler).updateSchedule(argThat(r -> "a".equals(r.getGroupName())
+ && "ENABLED".equals(r.getState())), eq(REGION));
+ assertTrue(provisioner.completeUpdate(resource).complete());
+ verify(scheduler, times(2)).deleteSchedule("same", "b", REGION);
+ verify(scheduler, never()).deleteSchedule("same", "a", REGION);
+ }
+
+ @Test
+ void failedGroupMoveWithPriorOrphanNeverTouchesAnExternalDestination() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ Schedule original = schedule(request("same", "a", "rate(5 minutes)"));
+ original.setState("ENABLED");
+ when(scheduler.getSchedule("same", "a", REGION)).thenReturn(original);
+ provisioner.provision(resource, properties("same", "b"), context("same"));
+ Schedule orphan = schedule(request("same", "b", "rate(5 minutes)"));
+ orphan.setState("ENABLED");
+ when(scheduler.getSchedule("same", "b", REGION)).thenReturn(orphan);
+ doThrow(new AwsException("InternalServerException", "delete unavailable", 500)).doNothing()
+ .when(scheduler).deleteSchedule("same", "b", REGION);
+ assertThrows(AwsException.class, () -> provisioner.rollbackUpdate(resource));
+ assertEquals("b/same", provisioner.updateCleanupPhysicalId(resource));
+
+ Schedule external = schedule(request("same", "c", "rate(1 minute)"));
+ external.setState("ENABLED");
+ when(scheduler.getSchedule("same", "c", REGION)).thenReturn(external);
+ when(scheduler.createSchedule(argThat(r -> "c".equals(r.getGroupName())), eq(REGION)))
+ .thenThrow(new AwsException("ConflictException", "destination owned elsewhere", 409));
+
+ assertThrows(AwsException.class,
+ () -> provisioner.provision(resource, properties("same", "c"), context("same")));
+
+ verify(scheduler, never()).getSchedule("same", "c", REGION);
+ verify(scheduler, never()).updateSchedule(argThat(r -> "c".equals(r.getGroupName())), eq(REGION));
+ verify(scheduler, never()).deleteSchedule("same", "c", REGION);
+ ArgumentCaptor<ScheduleRequest> requests = ArgumentCaptor.forClass(ScheduleRequest.class);
+ verify(scheduler, times(4)).updateSchedule(argThat(r -> "a".equals(r.getGroupName())), eq(REGION));
+ verify(scheduler, times(5)).updateSchedule(requests.capture(), eq(REGION));
+ assertEquals("a", requests.getAllValues().getLast().getGroupName());
+ assertEquals("ENABLED", requests.getAllValues().getLast().getState());
+ assertEquals("same", resource.getPhysicalId());
+ assertEquals("a", resource.getAttributes().get("FlociSchedulerGroupName"));
+ assertEquals("b/same", provisioner.updateCleanupPhysicalId(resource));
+
+ assertTrue(provisioner.completeUpdate(resource).complete());
+ verify(scheduler, times(2)).deleteSchedule("same", "b", REGION);
+ verify(scheduler, never()).deleteSchedule("same", "a", REGION);
+ }
+
+ @Test
+ void aFailedEagerRestoreKeepsTheSnapshotForRollbackRetry() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ Schedule original = schedule(request("same", "a", "rate(5 minutes)"));
+ original.setState("ENABLED");
+ when(scheduler.getSchedule("same", "a", REGION)).thenReturn(original);
+ when(scheduler.createSchedule(argThat(r -> "b".equals(r.getGroupName())), eq(REGION)))
+ .thenThrow(new AwsException("ConflictException", "destination exists", 409));
+ when(scheduler.updateSchedule(argThat(r -> "a".equals(r.getGroupName())
+ && "ENABLED".equals(r.getState())), eq(REGION)))
+ .thenThrow(new AwsException("InternalServerException", "restore failed", 500))
+ .thenAnswer(inv -> schedule(inv.getArgument(0)));
+
+ assertThrows(AwsException.class,
+ () -> provisioner.provision(resource, properties("same", "b"), context("same")));
+
+ assertTrue(provisioner.retainsFailedUpdateState(resource));
+ assertTrue(resource.getAttributes().containsKey(CfnRollback.UPDATE_ROLLBACK_FAILURE_ATTR));
+ assertTrue(provisioner.rollbackUpdate(resource));
+ assertFalse(provisioner.retainsFailedUpdateState(resource));
+ verify(scheduler, times(2)).updateSchedule(argThat(r -> "a".equals(r.getGroupName())
+ && "ENABLED".equals(r.getState())), eq(REGION));
+ }
+
+ @Test
+ void aNewUpdateRecoversPendingRollbackBeforeTakingAnotherSnapshot() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ AtomicReference<Schedule> current = trackScheduleUpdates("same", "a");
+ provisioner.provision(resource, properties("same", "a").put("ScheduleExpression", "rate(10 minutes)"),
+ context("same"));
+ when(scheduler.updateSchedule(argThat(r -> "rate(5 minutes)".equals(r.getScheduleExpression())), eq(REGION)))
+ .thenThrow(new AwsException("InternalServerException", "restore failed", 500))
+ .thenAnswer(inv -> {
+ Schedule restored = schedule(inv.getArgument(0));
+ current.set(restored);
+ return restored;
+ });
+
+ assertThrows(IllegalStateException.class, () -> provisioner.rollbackUpdate(resource));
+ assertTrue(provisioner.retainsFailedUpdateState(resource));
+ assertEquals("rate(10 minutes)", current.get().getScheduleExpression());
+
+ provisioner.provision(resource, properties("same", "a").put("ScheduleExpression", "rate(15 minutes)"),
+ context("same"));
+ assertEquals("rate(15 minutes)", current.get().getScheduleExpression());
+ assertTrue(provisioner.rollbackUpdate(resource));
+
+ assertEquals("rate(5 minutes)", current.get().getScheduleExpression());
+ assertFalse(provisioner.retainsFailedUpdateState(resource));
+ }
+
+ @Test
+ void anUnrecoverableSnapshotBlocksANewUpdateWithoutLosingTheOriginalState() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ AtomicReference<Schedule> current = trackScheduleUpdates("same", "a");
+ provisioner.provision(resource, properties("same", "a").put("ScheduleExpression", "rate(10 minutes)"),
+ context("same"));
+ when(scheduler.updateSchedule(argThat(r -> "rate(5 minutes)".equals(r.getScheduleExpression())), eq(REGION)))
+ .thenThrow(new AwsException("InternalServerException", "restore failed", 500))
+ .thenThrow(new AwsException("InternalServerException", "restore still unavailable", 500))
+ .thenAnswer(inv -> {
+ Schedule restored = schedule(inv.getArgument(0));
+ current.set(restored);
+ return restored;
+ });
+ assertThrows(IllegalStateException.class, () -> provisioner.rollbackUpdate(resource));
+ String snapshot = resource.getAttributes().get("__FlociSchedulerUpdateSnapshot");
+
+ assertThrows(IllegalStateException.class,
+ () -> provisioner.provision(resource,
+ properties("same", "a").put("ScheduleExpression", "rate(15 minutes)"), context("same")));
+
+ assertEquals(snapshot, resource.getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+ assertEquals("rate(10 minutes)", current.get().getScheduleExpression());
+ verify(scheduler, never()).updateSchedule(argThat(r -> "rate(15 minutes)".equals(r.getScheduleExpression())),
+ eq(REGION));
+ provisioner.provision(resource, properties("same", "a").put("ScheduleExpression", "rate(15 minutes)"),
+ context("same"));
+ assertTrue(provisioner.rollbackUpdate(resource));
+ assertEquals("rate(5 minutes)", current.get().getScheduleExpression());
+ }
+
+ @Test
+ void cleanupOfAnotherResourceCannotDiscardAFailedScheduleRollback() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ trackScheduleUpdates("same", "a");
+ provisioner.provision(resource, properties("same", "a").put("ScheduleExpression", "rate(10 minutes)"),
+ context("same"));
+ when(scheduler.updateSchedule(argThat(r -> "rate(5 minutes)".equals(r.getScheduleExpression())), eq(REGION)))
+ .thenThrow(new AwsException("InternalServerException", "restore failed", 500));
+ assertThrows(IllegalStateException.class, () -> provisioner.rollbackUpdate(resource));
+ resource.setStatus("UPDATE_FAILED");
+ String snapshot = resource.getAttributes().get("__FlociSchedulerUpdateSnapshot");
+
+ IllegalStateException failure = assertThrows(IllegalStateException.class,
+ () -> provisioner.completeUpdate(resource));
+
+ assertTrue(failure.getMessage().contains("Schedule rollback is still pending"));
+ assertEquals(snapshot, resource.getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+ assertTrue(provisioner.retainsFailedUpdateState(resource));
+ }
+
+ @Test
+ void deleteCleanupPreservesPendingSnapshotUntilTheScheduleIsActuallyDeleted() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "a"), context(null));
+ trackScheduleUpdates("same", "a");
+ provisioner.provision(resource, properties("same", "a").put("ScheduleExpression", "rate(10 minutes)"),
+ context("same"));
+ when(scheduler.updateSchedule(argThat(r -> "rate(5 minutes)".equals(r.getScheduleExpression())), eq(REGION)))
+ .thenThrow(new AwsException("InternalServerException", "restore failed", 500));
+ assertThrows(IllegalStateException.class, () -> provisioner.rollbackUpdate(resource));
+ resource.setStatus("UPDATE_FAILED");
+ String snapshot = resource.getAttributes().get("__FlociSchedulerUpdateSnapshot");
+
+ provisioner.completeDeleteCleanup(resource);
+ provisioner.clearDeleteCleanup(resource);
+ assertEquals(snapshot, resource.getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+ doThrow(new AwsException("InternalServerException", "delete failed", 500)).doNothing()
+ .when(scheduler).deleteSchedule("same", "a", REGION);
+ assertThrows(AwsException.class, () -> provisioner.delete(resource, REGION));
+ assertEquals(snapshot, resource.getAttributes().get("__FlociSchedulerUpdateSnapshot"));
+
+ provisioner.delete(resource, REGION);
+
+ assertFalse(provisioner.retainsFailedUpdateState(resource));
+ verify(scheduler, times(1)).updateSchedule(
+ argThat(r -> "rate(5 minutes)".equals(r.getScheduleExpression())), eq(REGION));
+ }
+
+ @Test
+ void deleteCleanupStillReportsDisplacedScheduleFailureAndHonorsRetain() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("old", "a"), context(null));
+ provisioner.provision(resource, properties("new", "b"), context("old"));
+ doThrow(new AwsException("InternalServerException", "delete unavailable", 500)).doNothing()
+ .when(scheduler).deleteSchedule("old", "a", REGION);
+
+ UpdateCleanupResult failed = provisioner.completeDeleteCleanup(resource);
+
+ assertFalse(failed.complete());
+ assertEquals(1, failed.attempts());
+ assertEquals("delete unavailable", failed.failureReason());
+ assertEquals("a/old", provisioner.updateCleanupPhysicalId(resource));
+ assertTrue(provisioner.completeDeleteCleanup(resource).complete());
+ verify(scheduler, times(2)).deleteSchedule("old", "a", REGION);
+ verify(scheduler, never()).deleteSchedule("new", "b", REGION);
+
+ StackResource retained = resource();
+ retained.setUpdateReplacePolicy("Retain");
+ provisioner.provision(retained, properties("retained-old", "a"), context(null));
+ provisioner.provision(retained, properties("retained-new", "b"), context("retained-old"));
+ assertTrue(provisioner.completeDeleteCleanup(retained).complete());
+ verify(scheduler, never()).deleteSchedule("retained-old", "a", REGION);
+ }
+
+ @Test
+ void boundedDeleteCleanupDoesNotMakeAFourthAttemptDuringPhysicalDeletion() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "a"), context(null));
+ ReplacementCleanup.recordOrphan(resource, "b/orphan-one", resource.getResourceType(), REGION);
+ ReplacementCleanup.recordOrphan(resource, "c/orphan-two", resource.getResourceType(), REGION);
+ doThrow(new AwsException("InternalServerException", "delete unavailable", 500))
+ .when(scheduler).deleteSchedule("orphan-one", "b", REGION);
+ doThrow(new AwsException("InternalServerException", "delete unavailable", 500))
+ .when(scheduler).deleteSchedule("orphan-two", "c", REGION);
+
+ for (int attempt = 1; attempt <= 3; attempt++) {
+ UpdateCleanupResult failed = provisioner.completeDeleteCleanup(resource);
+ assertFalse(failed.complete());
+ assertEquals(attempt, failed.attempts());
+ }
+ provisioner.clearDeleteCleanup(resource);
+
+ assertAll(
+ () -> assertFalse(provisioner.hasReplacementUpdate(resource)),
+ () -> assertNull(provisioner.updateCleanupPhysicalId(resource)),
+ () -> provisioner.delete(resource, REGION),
+ () -> verify(scheduler, times(3)).deleteSchedule("orphan-one", "b", REGION),
+ () -> verify(scheduler, times(3)).deleteSchedule("orphan-two", "c", REGION),
+ () -> verify(scheduler, times(1)).deleteSchedule("current", "a", REGION));
+ }
+
+ @Test
+ void boundedDeleteCleanupRetainsTheOldNameAndAbandonsExhaustedOrphans() throws Exception {
+ StackResource resource = resource();
+ resource.setUpdateReplacePolicy("Retain");
+ provisioner.provision(resource, properties("retained-old", "a"), context(null));
+ provisioner.provision(resource, properties("current", "b"), context("retained-old"));
+ ReplacementCleanup.recordOrphan(resource, "b/current", resource.getResourceType(), REGION);
+ ReplacementCleanup.recordOrphan(resource, "c/orphan", resource.getResourceType(), REGION);
+ doThrow(new AwsException("InternalServerException", "delete unavailable", 500))
+ .when(scheduler).deleteSchedule("orphan", "c", REGION);
+
+ for (int attempt = 1; attempt <= 3; attempt++) {
+ assertFalse(provisioner.completeDeleteCleanup(resource).complete());
+ }
+ provisioner.clearDeleteCleanup(resource);
+ assertAll(
+ () -> assertFalse(resource.getAttributes().containsKey("__FlociSchedulerNameReplacement")),
+ () -> assertFalse(provisioner.hasReplacementUpdate(resource)),
+ () -> assertNull(provisioner.updateCleanupPhysicalId(resource)),
+ () -> verify(scheduler, never()).deleteSchedule("current", "b", REGION),
+ () -> provisioner.delete(resource, REGION),
+ () -> verify(scheduler, times(3)).deleteSchedule("orphan", "c", REGION),
+ () -> verify(scheduler, times(1)).deleteSchedule("current", "b", REGION),
+ () -> verify(scheduler, never()).deleteSchedule("retained-old", "a", REGION));
+ }
+
+ @Test
+ void boundedDeleteCleanupSkipsAnAlreadyExhaustedAddress() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "a"), context(null));
+ ReplacementCleanup.recordOrphan(resource, "b/orphan", resource.getResourceType(), REGION);
+ setCleanupAttempts(resource, "b/orphan", 3, "historical deletion failed");
+ Map<String, String> before = Map.copyOf(resource.getAttributes());
+ doThrow(new AwsException("InternalServerException", "unexpected fourth attempt", 500))
+ .when(scheduler).deleteSchedule("orphan", "b", REGION);
+ clearInvocations(scheduler);
+
+ UpdateCleanupResult result = provisioner.completeDeleteCleanup(resource);
+
+ assertAll(
+ () -> assertFalse(result.complete()),
+ () -> assertEquals("b/orphan", result.previousPhysicalId()),
+ () -> assertEquals(3, result.attempts()),
+ () -> assertEquals("historical deletion failed", result.failureReason()),
+ () -> assertEquals(before, resource.getAttributes()),
+ () -> verifyNoInteractions(scheduler));
+ }
+
+ @Test
+ void boundedDeleteCleanupClearsOnlyExhaustedDebtAndPreservesTheSnapshot() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "a"), context(null));
+ provisioner.provision(resource,
+ properties("current", "a").put("ScheduleExpression", "rate(10 minutes)"), context("current"));
+ String snapshot = resource.getAttributes().get("__FlociSchedulerUpdateSnapshot");
+ assertTrue(provisioner.retainsFailedUpdateState(resource));
+ ReplacementCleanup.recordOrphan(resource, "b/exhausted", resource.getResourceType(), REGION);
+ ReplacementCleanup.recordOrphan(resource, "c/pending", resource.getResourceType(), "eu-west-1");
+ setCleanupAttempts(resource, "b/exhausted", 3, "exhausted failure");
+ setCleanupAttempts(resource, "c/pending", 1, "pending failure");
+ resource.getAttributes().put("ExistingAttribute", "unchanged");
+ clearInvocations(scheduler);
+
+ provisioner.clearDeleteCleanup(resource);
+
+ JsonNode debt = mapper.readTree(resource.getAttributes().get("__FlociReplacementCleanup"))
+ .path("displaced");
+ assertAll(
+ () -> assertEquals(1, debt.size()),
+ () -> assertEquals("c/pending", debt.get(0).path("physicalId").asText()),
+ () -> assertEquals(1, debt.get(0).path("cleanupAttempts").asInt()),
+ () -> assertEquals("pending failure", debt.get(0).path("cleanupFailureReason").asText()),
+ () -> assertEquals("eu-west-1", debt.get(0).path("region").asText()),
+ () -> assertEquals(resource.getResourceType(), debt.get(0).path("resourceType").asText()),
+ () -> assertFalse(debt.get(0).path("retainable").asBoolean()),
+ () -> assertEquals(snapshot, resource.getAttributes().get("__FlociSchedulerUpdateSnapshot")),
+ () -> assertEquals("unchanged", resource.getAttributes().get("ExistingAttribute")),
+ () -> verifyNoInteractions(scheduler));
+ }
+
+ @Test
+ void boundedDeleteCleanupUsesEachRemainingBudgetAndRegion() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "a"), context(null));
+ ReplacementCleanup.recordOrphan(resource, "b/exhausted", resource.getResourceType(), REGION);
+ ReplacementCleanup.recordOrphan(resource, "c/pending", resource.getResourceType(), "eu-west-1");
+ ReplacementCleanup.recordOrphan(resource, "d/fresh", resource.getResourceType(), REGION);
+ setCleanupAttempts(resource, "b/exhausted", 3, "historical failure");
+ setCleanupAttempts(resource, "c/pending", 1, "earlier failure");
+ doThrow(new AwsException("InternalServerException", "pending failure", 500))
+ .when(scheduler).deleteSchedule("pending", "c", "eu-west-1");
+ doThrow(new AwsException("InternalServerException", "fresh failure", 500))
+ .when(scheduler).deleteSchedule("fresh", "d", REGION);
+ clearInvocations(scheduler);
+
+ UpdateCleanupResult result = provisioner.completeDeleteCleanup(resource);
+ JsonNode debt = mapper.readTree(resource.getAttributes().get("__FlociReplacementCleanup"))
+ .path("displaced");
+
+ assertAll(
+ () -> assertFalse(result.complete()),
+ () -> assertEquals("d/fresh", result.previousPhysicalId()),
+ () -> assertEquals(1, result.attempts()),
+ () -> assertEquals("fresh failure", result.failureReason()),
+ () -> assertEquals(3, debt.size()),
+ () -> assertEquals(3, debt.get(0).path("cleanupAttempts").asInt()),
+ () -> assertEquals("historical failure", debt.get(0).path("cleanupFailureReason").asText()),
+ () -> assertEquals(2, debt.get(1).path("cleanupAttempts").asInt()),
+ () -> assertEquals("pending failure", debt.get(1).path("cleanupFailureReason").asText()),
+ () -> assertEquals(1, debt.get(2).path("cleanupAttempts").asInt()),
+ () -> verify(scheduler, never()).deleteSchedule("exhausted", "b", REGION),
+ () -> verify(scheduler, times(1)).deleteSchedule("pending", "c", "eu-west-1"),
+ () -> verify(scheduler, times(1)).deleteSchedule("fresh", "d", REGION),
+ () -> verify(scheduler, never()).deleteSchedule("current", "a", REGION));
+ }
+
+ @Test
+ void boundedPhysicalDeleteDoesNotRetryExhaustedDebtWithoutTheCleanupHook() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("current", "a"), context(null));
+ provisioner.provision(resource,
+ properties("current", "a").put("ScheduleExpression", "rate(10 minutes)"), context("current"));
+ String snapshot = resource.getAttributes().get("__FlociSchedulerUpdateSnapshot");
+ ReplacementCleanup.recordOrphan(resource, "b/orphan", resource.getResourceType(), REGION);
+ setCleanupAttempts(resource, "b/orphan", 3, "historical deletion failed");
+ doThrow(new AwsException("InternalServerException", "unexpected fourth attempt", 500))
+ .when(scheduler).deleteSchedule("orphan", "b", REGION);
+ clearInvocations(scheduler);
+
+ IllegalStateException failure = assertThrows(IllegalStateException.class,
+ () -> provisioner.delete(resource, REGION));
+
+ assertAll(
+ () -> assertEquals("historical deletion failed", failure.getMessage()),
+ () -> assertEquals(snapshot, resource.getAttributes().get("__FlociSchedulerUpdateSnapshot")),
+ () -> verify(scheduler, times(1)).deleteSchedule("current", "a", REGION),
+ () -> verify(scheduler, never()).deleteSchedule("orphan", "b", REGION));
+ }
+
+ @Test
+ void nameReplacementHonorsRetain() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("old", "a"), context(null));
+ resource.setUpdateReplacePolicy("Retain");
+ provisioner.provision(resource, properties("new", "b"), context("old"));
+
+ assertTrue(provisioner.completeUpdate(resource).complete());
+
+ verify(scheduler, never()).deleteSchedule("old", "a", REGION);
+ }
+
+ @Test
+ void inPlaceRollbackRestoresPriorDatesStateAndTarget() throws Exception {
+ StackResource resource = resource();
+ ObjectNode oldProps = properties("same", "default");
+ provisioner.provision(resource, oldProps, context(null));
+ Schedule old = schedule(request("same", "default", "rate(5 minutes)"));
+ old.setStartDate(Instant.parse("2030-01-01T00:00:00Z"));
+ old.setState("DISABLED");
+ when(scheduler.getSchedule("same", "default", REGION)).thenReturn(old);
+ ObjectNode newProps = properties("same", "default").put("ScheduleExpression", "rate(10 minutes)");
+ provisioner.provision(resource, newProps, context("same"));
+
+ assertTrue(provisioner.rollbackUpdate(resource));
+
+ ArgumentCaptor<ScheduleRequest> requests = ArgumentCaptor.forClass(ScheduleRequest.class);
+ verify(scheduler, times(2)).updateSchedule(requests.capture(), eq(REGION));
+ ScheduleRequest restored = requests.getAllValues().get(1);
+ assertEquals("rate(5 minutes)", restored.getScheduleExpression());
+ assertEquals("DISABLED", restored.getState());
+ assertEquals(old.getStartDate(), restored.getStartDate());
+ }
+
+ @Test
+ void missingRequiredPropertiesFailBeforeCreating() throws Exception {
+ for (String property : new String[] {"FlexibleTimeWindow", "Target", "ScheduleExpression"}) {
+ ObjectNode props = properties("same", "default");
+ props.remove(property);
+ AwsException failure = assertThrows(AwsException.class,
+ () -> provisioner.provision(resource(), props, context(null)));
+ assertEquals("ValidationError", failure.getErrorCode());
+ }
+ verify(scheduler, never()).createSchedule(any(), eq(REGION));
+ }
+
+ @Test
+ void requiredObjectNullAndInvalidShapesFailBeforeCreating() throws Exception {
+ for (String property : new String[] {"FlexibleTimeWindow", "Target"}) {
+ for (JsonNode invalid : new JsonNode[] {mapper.nullNode(), mapper.createArrayNode(),
+ mapper.valueToTree("not-an-object"), mapper.valueToTree(false)}) {
+ ObjectNode props = properties("same", "default");
+ props.set(property, invalid);
+ assertThrows(AwsException.class, () -> provisioner.provision(resource(), props, context(null)),
+ property + ": " + invalid);
+ }
+ }
+ verify(scheduler, never()).createSchedule(any(), eq(REGION));
+ }
+
+ @Test
+ void unsupportedTargetParametersAreRejectedInsteadOfDiscarded() throws Exception {
+ ObjectNode props = properties("same", "default");
+ ((ObjectNode) props.get("Target")).putObject("KinesisParameters").put("PartitionKey", "key");
+ assertThrows(AwsException.class, () -> provisioner.provision(resource(), props, context(null)));
+ verify(scheduler, never()).createSchedule(any(), eq(REGION));
+ }
+
+ @Test
+ void blankExplicitNameOrGroupIsRejectedInsteadOfUsingADefault() throws Exception {
+ for (String property : new String[] {"Name", "GroupName"}) {
+ ObjectNode props = properties("same", "default").put(property, " ");
+ assertThrows(AwsException.class, () -> provisioner.provision(resource(), props, context(null)));
+ }
+ verify(scheduler, never()).createSchedule(any(), eq(REGION));
+ }
+
+ @Test
+ void invalidDateOrStateFailsBeforeMutatingTheExistingSchedule() throws Exception {
+ for (Map.Entry<String, String> bad : Map.of("StartDate", "tomorrow", "State", "ACTIVE").entrySet()) {
+ ObjectNode props = properties("same", "default").put(bad.getKey(), bad.getValue());
+ assertThrows(AwsException.class, () -> provisioner.provision(resource(), props, context("same")));
+ }
+ verify(scheduler, never()).updateSchedule(any(), eq(REGION));
+ }
+
+ @Test
+ void deleteUsesTheStoredGroupAndToleratesOnlyNotFound() throws Exception {
+ StackResource resource = resource();
+ provisioner.provision(resource, properties("same", "custom"), context(null));
+ doThrow(new AwsException("ResourceNotFoundException", "gone", 404))
+ .when(scheduler).deleteSchedule("same", "custom", REGION);
+ provisioner.delete(resource, REGION);
+ doThrow(new AwsException("InternalServerException", "failed", 500))
+ .when(scheduler).deleteSchedule("same", "custom", REGION);
+ assertThrows(AwsException.class, () -> provisioner.delete(resource, REGION));
+ }
+
+ private void setCleanupAttempts(StackResource resource, String physicalId, int attempts, String reason)
+ throws Exception {
+ ObjectNode cleanup = (ObjectNode) mapper.readTree(resource.getAttributes().get("__FlociReplacementCleanup"));
+ for (JsonNode entry : cleanup.path("displaced")) {
+ if (physicalId.equals(entry.path("physicalId").asText())) {
+ ((ObjectNode) entry).put("cleanupAttempts", attempts).put("cleanupFailureReason", reason);
+ }
+ }
+ resource.getAttributes().put("__FlociReplacementCleanup", cleanup.toString());
+ }
+
+ private ObjectNode properties(String name, String group) throws Exception {
+ ObjectNode props = (ObjectNode) mapper.readTree("""
+ {"ScheduleExpression":"rate(5 minutes)","FlexibleTimeWindow":{"Mode":"OFF"},
+ "Target":{"Arn":"arn:aws:sqs:us-east-1:000000000000:queue",
+ "RoleArn":"arn:aws:iam::000000000000:role/scheduler","Input":"payload"}}
+ """);
+ if (name != null) {
+ props.put("Name", name);
+ }
+ if (group != null) {
+ props.put("GroupName", group);
+ }
+ return props;
+ }
+
+ private AtomicReference<Schedule> trackScheduleUpdates(String name, String group) {
+ AtomicReference<Schedule> current = new AtomicReference<>(schedule(request(name, group, "rate(5 minutes)")));
+ when(scheduler.getSchedule(name, group, REGION)).thenAnswer(inv -> current.get());
+ when(scheduler.updateSchedule(any(), eq(REGION))).thenAnswer(inv -> {
+ Schedule updated = schedule(inv.getArgument(0));
+ current.set(updated);
+ return updated;
+ });
+ return current;
+ }
+
+ private ProvisionContext context(String prior) {
+ CloudFormationTemplateEngine engine = mock(CloudFormationTemplateEngine.class);
+ when(engine.resolveNode(any())).thenAnswer(inv -> inv.getArgument(0));
+ when(engine.resolveNodeOmittingNoValue(any())).thenAnswer(inv -> inv.getArgument(0));
+ when(engine.resolve(any())).thenAnswer(inv -> {
+ JsonNode node = inv.getArgument(0);
+ return node == null ? null : node.asText();
+ });
+ return new ProvisionContext(engine, REGION, "000000000000", "stack", prior);
+ }
+
+ private static StackResource resource() {
+ StackResource resource = new StackResource();
+ resource.setLogicalId("Schedule");
+ resource.setResourceType("AWS::Scheduler::Schedule");
+ resource.setAttributes(new HashMap<>());
+ return resource;
+ }
+
+ private static ScheduleRequest request(String name, String group, String expression) {
+ ScheduleRequest request = new ScheduleRequest();
+ request.setName(name);
+ request.setGroupName(group);
+ request.setScheduleExpression(expression);
+ return request;
+ }
+
+ private static Schedule schedule(ScheduleRequest request) {
+ Schedule schedule = new Schedule();
+ schedule.setName(request.getName());
+ schedule.setGroupName(request.getGroupName());
+ schedule.setArn("arn:aws:scheduler:us-east-1:000000000000:schedule/"
+ + request.getGroupName() + "/" + request.getName());
+ schedule.setScheduleExpression(request.getScheduleExpression());
+ schedule.setTarget(request.getTarget());
+ schedule.setFlexibleTimeWindow(request.getFlexibleTimeWindow());
+ return schedule;
+ }
+}
diff --git a/src/test/java/io/github/hectorvent/floci/services/scheduler/SchedulerAddressTest.java b/src/test/java/io/github/hectorvent/floci/services/scheduler/SchedulerAddressTest.java
new file mode 100644
index 0000000000..9e0dea7456
--- /dev/null
+++ b/src/test/java/io/github/hectorvent/floci/services/scheduler/SchedulerAddressTest.java
@@ -0,0 +1,100 @@
+package io.github.hectorvent.floci.services.scheduler;
+
+import com.fasterxml.jackson.databind.ObjectMapper;
+import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
+import io.github.hectorvent.floci.core.common.AwsException;
+import io.github.hectorvent.floci.core.common.RegionResolver;
+import io.github.hectorvent.floci.core.storage.InMemoryStorage;
+import io.github.hectorvent.floci.services.scheduler.model.FlexibleTimeWindow;
+import io.github.hectorvent.floci.services.scheduler.model.Schedule;
+import io.github.hectorvent.floci.services.scheduler.model.ScheduleRequest;
+import io.github.hectorvent.floci.services.scheduler.model.Target;
+import org.junit.jupiter.api.Test;
+
+import java.util.Map;
+
+import static org.junit.jupiter.api.Assertions.assertEquals;
+import static org.junit.jupiter.api.Assertions.assertThrows;
+
+class SchedulerAddressTest {
+
+ private static final String REGION = "us-east-1";
+
+ @Test
+ void updateAndDeleteAddressTheRecreatedSchedule() throws Exception {
+ SchedulerService service = service(new InMemoryStorage<>());
+ Schedule original = service.createSchedule(request("schedule", "default", "first"), REGION);
+ service.deleteSchedule("schedule", "default", REGION);
+ Schedule recreated = service.createSchedule(request("schedule", "default", "external"), REGION);
+ assertEquals(original.getArn(), recreated.getArn());
+
+ Schedule updated = service.updateSchedule(request("schedule", "default", "template"), REGION);
+
+ assertEquals(recreated.getArn(), updated.getArn());
+ assertEquals(recreated.getCreationDate(), updated.getCreationDate());
+ assertEquals("template", service.getSchedule("schedule", "default", REGION).getTarget().getInput());
+ ObjectMapper mapper = new ObjectMapper().registerModule(new JavaTimeModule());
+ Schedule restored = mapper.readValue(mapper.writeValueAsString(updated), Schedule.class);
+ assertEquals(updated.getArn(), restored.getArn());
+ assertEquals(updated.getCreationDate(), restored.getCreationDate());
+ assertEquals("template", restored.getTarget().getInput());
+ service.deleteSchedule("schedule", "default", REGION);
+ AwsException missing = assertThrows(AwsException.class, () -> service.getSchedule("schedule", "default", REGION));
+ assertEquals("ResourceNotFoundException", missing.getErrorCode());
+ }
+
+ @Test
+ void theSameNameInAnotherGroupIsUnaffectedByUpdateAndDeletion() {
+ SchedulerService service = service(new InMemoryStorage<>());
+ service.createScheduleGroup("other", Map.of(), REGION);
+ service.createSchedule(request("same", "default", "default"), REGION);
+ service.createSchedule(request("same", "other", "other"), REGION);
+
+ service.updateSchedule(request("same", "default", "changed"), REGION);
+ service.deleteSchedule("same", "default", REGION);
+
+ assertEquals("other", service.getSchedule("same", "other", REGION).getTarget().getInput());
+ service.deleteSchedule("same", "other", REGION);
+ assertThrows(AwsException.class, () -> service.getSchedule("same", "other", REGION));
+ }
+
+ @Test
+ void aPersistedScheduleCanBeUpdatedAndDeletedByItsAddress() throws Exception {
+ InMemoryStorage<String, Schedule> store = new InMemoryStorage<>();
+ Schedule persisted = new ObjectMapper().readValue("""
+ {"name":"persisted","groupName":"default","state":"DISABLED",
+ "arn":"arn:aws:scheduler:us-east-1:000000000000:schedule/default/persisted"}
+ """, Schedule.class);
+ store.put("schedule:" + REGION + ":default:persisted", persisted);
+ SchedulerService service = service(store);
+
+ Schedule updated = service.updateSchedule(request("persisted", "default", "updated"), REGION);
+
+ assertEquals(persisted.getArn(), updated.getArn());
+ assertEquals("updated", service.getSchedule("persisted", "default", REGION).getTarget().getInput());
+ service.deleteSchedule("persisted", "default", REGION);
+ assertThrows(AwsException.class, () -> service.getSchedule("persisted", "default", REGION));
+ }
+
+ private static SchedulerService service(InMemoryStorage<String, Schedule> store) {
+ return new SchedulerService(new InMemoryStorage<>(), store,
+ new RegionResolver(REGION, "000000000000"));
+ }
+
+ private static ScheduleRequest request(String name, String group, String input) {
+ ScheduleRequest request = new ScheduleRequest();
+ request.setName(name);
+ request.setGroupName(group);
+ request.setScheduleExpression("rate(5 minutes)");
+ request.setState("DISABLED");
+ FlexibleTimeWindow window = new FlexibleTimeWindow();
+ window.setMode("OFF");
+ request.setFlexibleTimeWindow(window);
+ Target target = new Target();
+ target.setArn("arn:aws:sqs:us-east-1:000000000000:queue");
+ target.setRoleArn("arn:aws:iam::000000000000:role/scheduler");
+ target.setInput(input);
+ request.setTarget(target);
+ return request;
+ }
+}
diff --git a/src/test/resources/cloudformation/supported-resource-types.tsv b/src/test/resources/cloudformation/supported-resource-types.tsv
index e476c6de93..47ae84a79e 100644
--- a/src/test/resources/cloudformation/supported-resource-types.tsv
+++ b/src/test/resources/cloudformation/supported-resource-types.tsv
@@ -166,6 +166,7 @@ AWS::SNS::TopicPolicy SnsCfnProvisioner
AWS::SQS::Queue SqsCfnProvisioner
AWS::SQS::QueuePolicy SqsCfnProvisioner
AWS::SSM::Parameter SsmCfnProvisioner
+AWS::Scheduler::Schedule SchedulerScheduleCfnProvisioner
AWS::Scheduler::ScheduleGroup SchedulerScheduleGroupCfnProvisioner
AWS::SecretsManager::Secret SecretsManagerCfnProvisioner
AWS::SecretsManager::SecretTargetAttachment SecretTargetAttachmentCfnProvisioner