Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Pre-resolve all CI alias combos in cache generator (#75258)

Nemanja Glumac6134f7a7ca59
diff --git a/.github/workflows/cache-generator.yml b/.github/workflows/cache-generator.yml
index e4416d2b4cd..396aa56f906 100644
--- a/.github/workflows/cache-generator.yml
+++ b/.github/workflows/cache-generator.yml
@@ -68,10 +68,47 @@ jobs:
- name: Prepare backend
uses: ./.github/actions/prepare-backend
- # Add or remove aliases to affect the cache size and its contents.
- # Adjust to taste, then redeploy this workflow.
+ # Pre-resolve every Clojure dependency classpath CI actually uses, so the
+ # saved cache is a SUPERSET of what each job resolves at runtime.
+ #
+ # Why a single `-P` is not enough: tools.deps selects dependency *versions*
+ # globally per classpath, so different alias combinations can resolve the
+ # same transitive lib to different versions (e.g. `org.clojure/data.json`
+ # 2.4.0 vs 2.5.x, `org.ow2.asm/asm` 9.2 vs newer). Resolving only one combo
+ # leaves the others to fetch their version-deltas straight from Maven
+ # Central / Clojars on every run -> intermittent 403s. See DEV-2094.
+ #
+ # Keep this list in sync with the `clojure` invocations across .github/.
+ # Each entry notes its consumer. We resolve the FULL alias strings (not a
+ # hand-reduced set) on purpose: if a currently dep-less alias later gains a
+ # dependency, this still covers it instead of silently drifting back into a
+ # cache gap.
- name: Pre-resolve Clojure dependencies
- run: clojure -A:build:dev:ee:ee-dev:drivers:drivers-dev:cljs -P
+ shell: bash
+ run: |
+ set -euo pipefail
+ combos=(
+ ":build:dev:ee:ee-dev:drivers:drivers-dev:cljs" # build + uberjar (combined classpath)
+ ":cljs" # shadow-cljs (build:cljs); resolves :cljs alone -> covers asm:9.2
+ ":dev:ee:ee-dev:drivers:drivers-dev:test:eastwood" # backend.yml (eastwood + eastwood/test)
+ ":dev:ci:ee:ee-dev:drivers:drivers-dev:test" # test-driver.yml (ee) -> covers data.json:2.4.0
+ ":dev:ci:oss:oss-dev:drivers:drivers-dev:test" # test-driver.yml (oss)
+ ":dev:ci:test:ee:ee-dev" # backend.yml matrix (ee) + generative-query-test.yml
+ ":dev:ci:test:oss:oss-dev" # backend.yml matrix (oss)
+ ":dev:ci:ee:ee-dev:test:cloverage" # backend-cloverage.yml
+ ":dev:drivers:drivers-dev:test" # reset-cloud-test-dataset.yml + presto script
+ ":dev:drivers:build:build-dev:build-test:ci" # build-scripts.yml
+ ":ee:drivers:check" # backend.yml (check; git dep athos/clj-check)
+ ":ee:doc" # docs-generate-base.yml
+ ":kondo" # resolve-backport-conflicts.yml (replace-deps)
+ ":run:ee" # cross-branch-migration-test
+ ":dev:ee:migrate" # cross-branch-migration-test
+ )
+ for combo in "${combos[@]}"; do
+ echo "::group::clojure -P -A${combo}"
+ clojure -P -A"${combo}"
+ echo "::endgroup::"
+ done
- name: Update M2 cache
uses: ./.github/actions/update-cache