Skip to content

Your favorite shift management application!

fix(ci): wait out Komodo's busy state before deleting a preview stack (#221)

panteL3b8f513a4919
diff --git a/docs/PR_PREVIEWS.md b/docs/PR_PREVIEWS.md
index 0bcbd67..ab56313 100644
--- a/docs/PR_PREVIEWS.md
+++ b/docs/PR_PREVIEWS.md
@@ -208,6 +208,8 @@ KOMODO_URL=… KOMODO_API_KEY=… KOMODO_API_SECRET=… PR_NUMBER=<n> \
node scripts/preview-stack.mjs destroy
```
+Komodo answers `/execute/DestroyStack` before the compose run behind it has finished and then refuses the delete with `Stack busy`, which leaves the containers down but the stack row standing. `scripts/preview-stack.mjs` retries the delete for up to two minutes for that reason; a row that is still there after a teardown run usually means the destroy itself never completed, and the container log in Komodo says why.
+
This is safe to run even if the stack is already gone — `scripts/preview-stack.mjs` checks for it first and does nothing if it can't find it. If Komodo is behind Cloudflare, add `PREVIEW_CF_BYPASS_TOKEN=…` to that line as well (see [Cloudflare in Front](#cloudflare-in-front)).
Note that a health check that times out during deploy deliberately leaves the stack running instead of tearing it down: the whole point is to keep the container's logs inspectable in Komodo while you figure out what went wrong. The same is true if the seed step fails. In both cases nothing removes the stack automatically — use the command above once you're done investigating.
diff --git a/scripts/preview-stack.mjs b/scripts/preview-stack.mjs
index 832d442..d0715f6 100644
--- a/scripts/preview-stack.mjs
+++ b/scripts/preview-stack.mjs
@@ -241,6 +241,27 @@ async function deploy({ dryRun }) {
console.log(`Stack ${name} deployed.`);
}
+const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
+
+/**
+ * `/execute/DestroyStack` returns before the compose run behind it has finished,
+ * and Komodo refuses to delete a stack while one is in flight ("Stack busy").
+ * Polling the delete is simpler than polling for a state Komodo spells
+ * differently across versions.
+ */
+async function deleteStack(id, name) {
+ const deadline = Date.now() + 120_000;
+ for (;;) {
+ try {
+ return await komodo("/write/DeleteStack", { id });
+ } catch (error) {
+ if (!/busy/i.test(error.bodyText || "") || Date.now() >= deadline) throw error;
+ console.log(`Stack ${name} is still busy — waiting to delete it.`);
+ await sleep(5000);
+ }
+ }
+}
+
async function destroy({ dryRun }) {
const name = `bettershift-pr-${process.env.PR_NUMBER}`;
@@ -264,7 +285,7 @@ async function destroy({ dryRun }) {
if (failure) {
console.error(`${failure}\nDeleting the stack entry anyway.`);
}
- await komodo("/write/DeleteStack", { id: stackId(existing, name) });
+ await deleteStack(stackId(existing, name), name);
console.log(`Stack ${name} destroyed and deleted.`);
}