Skip to content

Free, Open Source, Self-Hosted WhatsApp API Gateway

fix(release): keep the staging image when a gate failed on a good build

Yudhi Armyndharis906b4819eabb
diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml
index f18d1f11c..b156ba041 100644
--- a/.github/workflows/release.yml
+++ b/.github/workflows/release.yml
@@ -821,14 +821,16 @@ jobs:
# with it: v0.10.5 published a GitHub Release while `latest`, `0.10` and `0.10.5` all vanished
# from the registry, so `docker pull` failed for everyone until they were restored.
#
- # This lives in its own `if: always()` job — not inside promote — because the failure path is
- # exactly when cleanup matters most: when boot-smoke or image-scan fails, promote is skipped and
- # the staging tag would otherwise leak. On the happy path the release tags share the staging
- # version, so the step below does not run at all (the deliberate retention documented above): the
- # package-versions API is eventually consistent, and a read that does not list the new release
- # tags yet would count the staging tag alone and delete the version just published. The same
- # holds for a promote that failed after re-pointing some tags, so the step runs only when promote
- # never ran, which is the one case where the staging tag is alone and gets removed.
+ # This lives in its own `if: always()` job — not inside promote — because the docker job can fail
+ # after pushing the staging tag, which would otherwise leak. On the happy path the release tags
+ # share the staging version, so the step below does not run at all (the deliberate retention
+ # documented above): the package-versions API is eventually consistent, and a read that does not
+ # list the new release tags yet would count the staging tag alone and delete the version just
+ # published. The same holds for a promote that failed after re-pointing some tags, so the step runs
+ # only when promote never ran. It also keeps the tag when the docker job succeeded and a gate
+ # (boot-smoke or image-scan) failed: "Re-run failed jobs" pulls that tag again, and with it deleted
+ # a flaky scan could only be recovered by rebuilding everything. That leaves one stray tag behind
+ # when the release is abandoned instead.
cleanup-staging:
name: Clean up the staging tag
runs-on: ubuntu-latest
@@ -841,8 +843,9 @@ jobs:
- name: Remove the staging tag (only when it is alone on its version)
# Only when promote never ran: once it has started re-pointing tags, the version can carry
# release tags that a stale read does not list yet. A failed or cancelled promote leaves the
- # stray staging tag behind, the same retention as the happy path.
- if: needs.promote.result == 'skipped'
+ # stray staging tag behind, the same retention as the happy path. And only when the docker job
+ # did not succeed: a gate that failed on top of a good staging image is re-run against it.
+ if: needs.promote.result == 'skipped' && needs.docker.result != 'success'
# Best-effort: a failed tidy-up must not turn an otherwise successful release red.
continue-on-error: true
env:
diff --git a/src/common/release-staging-cleanup.spec.ts b/src/common/release-staging-cleanup.spec.ts
index 2dc43cbe8..d8a960780 100644
--- a/src/common/release-staging-cleanup.spec.ts
+++ b/src/common/release-staging-cleanup.spec.ts
@@ -27,6 +27,15 @@ describe('release staging-tag cleanup', () => {
const deleting = (job?.steps ?? []).filter(step => /-X DELETE/.test(step.run ?? ''));
// Vacuity guard: a renamed job or a reworded command would leave nothing to check.
expect(deleting.length).toBeGreaterThan(0);
- for (const step of deleting) expect(step.if).toBe("needs.promote.result == 'skipped'");
+ for (const step of deleting) expect(step.if).toContain("needs.promote.result == 'skipped'");
+ });
+
+ // "Re-run failed jobs" runs boot-smoke and image-scan again against the staging tag. A gate that failed
+ // on a transient fault (the vulnerability database download, a QEMU timeout) could never pass on a
+ // re-run once cleanup had deleted the image it pulls, and only a rebuild of everything recovered it.
+ it('keeps the staging image a re-run of a failed gate pulls', () => {
+ const deleting = (job?.steps ?? []).filter(step => /-X DELETE/.test(step.run ?? ''));
+ expect(deleting.length).toBeGreaterThan(0);
+ for (const step of deleting) expect(step.if).toContain("needs.docker.result != 'success'");
});
});