The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Ratchet config-level kondo suppressions too (#78217)
Chris Truter42cf92479ad2
diff --git a/.clj-kondo/ratchets.edn b/.clj-kondo/ratchets.ednindex 62810f2dedc..3ec039209bb 100644--- a/.clj-kondo/ratchets.edn+++ b/.clj-kondo/ratchets.edn@@ -1,9 +1,10 @@-;; Per-linter budgets for inline `:clj-kondo/ignore` forms.-;; metabase.core.kondo-ratchet-test fails when the budgets drift from the actual counts, or when an+;; Budgets for kondo suppressions: inline `:clj-kondo/ignore` forms per linter (:ignore-counts),+;; and config-level waivers in .clj-kondo/config.edn (:config-counts -- :off switches and :exclude+;; entries). metabase.core.kondo-ratchet-test fails when either drifts from reality, or when an;; ignore outside :comment-exempt lacks an explanatory comment directly above or trailing on its line.;; `./bin/mage fix-kondo-ratchets` lowers budgets and drops stale exemptions; local test runs do it-;; automatically. Raising a budget, adding one for a new linter (`--seed`), or widening the-;; exemptions is a hand edit to defend in your PR.+;; automatically. Raising a budget, adding one (`--seed` for inline, by hand for config), or+;; widening the exemptions is a hand edit to defend in your PR.;; :all is the vector-less ignore form, which suppresses every linter on the next form.{:ignore-counts {:aliased-namespace-symbol 5:case-symbol-test 1@@ -49,6 +50,26 @@:unused-namespace 2:unused-private-var 9:unused-value 1}+ :config-counts {:constant-condition 1+ :deprecated-namespace 7+ :deprecated-var 16+ :discouraged-namespace 14+ :discouraged-var 14+ :inline-def 1+ :main-without-gen-class 1+ :metabase/defsetting-must-specify-export 1+ :metabase/defsetting-namespace 1+ :metabase/prefer-with-dynamic-fn-redefs 3+ :missing-docstring 1+ :private-call 1+ :redundant-ignore 1+ :redundant-primitive-coercion 1+ :refer-all 1+ :type-mismatch 1+ :unexpected-recur 1+ :unresolved-symbol 19+ :unresolved-var 1+ :unused-referred-var 4}:comment-exempt #{:aliased-namespace-symbol:case-symbol-test:clojure-lsp/unused-public-vardiff --git a/.github/workflows/kondo-ratchets-self-heal.yml b/.github/workflows/kondo-ratchets-self-heal.ymlindex 432f89da2d4..8309c901c43 100644--- a/.github/workflows/kondo-ratchets-self-heal.yml+++ b/.github/workflows/kondo-ratchets-self-heal.yml@@ -17,6 +17,8 @@ on:- "**.clj"- "**.cljc"- "**.cljs"+ # config-level suppressions are ratcheted too, so removing one here can leave a stale budget+ - ".clj-kondo/config.edn"concurrency:group: ${{ github.workflow }}-${{ github.event.pull_request.number }}diff --git a/CLAUDE.md b/CLAUDE.mdindex 6dbd9c15776..25309165b16 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -80,8 +80,9 @@ modules need reordering) are printed as `WARNING:` lines for you to resolve by h## Kondo Ignore Ratchets`.clj-kondo/ratchets.edn` records, per linter, how many inline `:clj-kondo/ignore` forms the backend source-tree may contain. `metabase.core.kondo-ratchet-test` fails when the budgets drift from the actual counts,-in either direction. Prefer fixing the underlying warning over adding an ignore.+tree may contain, and how many config-level suppressions (`:off` switches and `:exclude` entries in+`.clj-kondo/config.edn`) exist. `metabase.core.kondo-ratchet-test` fails when either budget drifts from the+actual counts, in either direction. Prefer fixing the underlying warning over adding an ignore.Budget too high (you removed ignores): a local run of the test tightens the file for you — commit thechange. PRs labelled `kondo-ratchets-self-healing` get the lowered budgets committed to the branch by CI.diff --git a/dev/src/dev/kondo_ratchet.clj b/dev/src/dev/kondo_ratchet.cljindex e662108c113..f111eba9217 100644--- a/dev/src/dev/kondo_ratchet.clj+++ b/dev/src/dev/kondo_ratchet.clj@@ -263,39 +263,101 @@(remove still-needed)exempt)))+(def ^:private kondo-config-file+ ".clj-kondo/config.edn")++(defn- excluded-items+ "How many items an `:exclude` value waives: sequential entries count each, and a map's values count+ their elements when sequential (`{compojure.core [GET POST]}` is 2) or 1 otherwise (a scoping map like+ `{some.var {:namespaces [...]}}` excludes one var)."+ [excl]+ (cond+ (map? excl) (reduce + (map #(if (sequential? %) (count %) 1) (vals excl)))+ (coll? excl) (count excl)+ :else 0))++(defn- suppressed-in+ "How many warnings one linter's config map waives: 1 for a `{:level :off}` switch, one per excluded+ item, and one per scoped `:off` nested under it at any depth. Scopes nest arbitrarily deep --+ `:discouraged-java-method` keys by class and then by method -- so counting only the first level+ would let an `:off` hide one map further down."+ [cfg]+ (if-not (map? cfg)+ 0+ (+ (if (= (:level cfg) :off) 1 0)+ (excluded-items (:exclude cfg))+ (reduce + 0 (map suppressed-in (vals (dissoc cfg :level :exclude)))))))++(defn config-suppressions+ "Per-linter counts of config-level suppressions in `config` (default: `.clj-kondo/config.edn`):+ top-level `:linters` entries, `:config-in-comment`, and every `:config-in-ns` / `:config-in-call`+ group. Scoped groups count too, or a linter could be switched off inside one and never show up.+ Entries that add discouragements or turn linters on count nothing — only weakening counts."+ ([]+ (config-suppressions (edn/read-string (slurp kondo-config-file))))+ ([config]+ (let [counts (fn [linters-map]+ (into {}+ (for [[linter cfg] linters-map+ :let [n (suppressed-in cfg)]+ :when (pos? n)]+ [linter n])))]+ (apply merge-with ++ (counts (:linters config))+ (counts (get-in config [:config-in-comment :linters]))+ (for [scope [:config-in-ns :config-in-call]+ [_k cfg] (get config scope)]+ (counts (:linters cfg)))))))++(defn config-drift+ "Linters whose config-suppression count differs from its budget (absent = 0, either side).+ Returns `{linter {:recorded _, :actual _}}`."+ [recorded actual]+ (sorted-by-str+ (for [linter (into (set (keys actual)) (keys recorded))+ :let [budget (get recorded linter 0)+ n (get actual linter 0)]+ :when (not= budget n)]+ [linter {:recorded budget, :actual n}])))+(defn read-ratchets"Parsed contents of [[ratchets-file]], with empty defaults when the file doesn't exist."[]- (merge {:ignore-counts {}, :comment-exempt #{}}+ (merge {:ignore-counts {}, :config-counts {}, :comment-exempt #{}}(when (.exists (io/file ratchets-file))(edn/read-string (slurp ratchets-file)))))(def ^:private header- (str ";; Per-linter budgets for inline `" ignore-marker "` forms.\n"- ";; metabase.core.kondo-ratchet-test fails when the budgets drift from the actual counts, or when an\n"+ (str ";; Budgets for kondo suppressions: inline `" ignore-marker "` forms per linter (:ignore-counts),\n"+ ";; and config-level waivers in .clj-kondo/config.edn (:config-counts -- :off switches and :exclude\n"+ ";; entries). metabase.core.kondo-ratchet-test fails when either drifts from reality, or when an\n"";; ignore outside :comment-exempt lacks an explanatory comment directly above or trailing on its line.\n"";; `./bin/mage fix-kondo-ratchets` lowers budgets and drops stale exemptions; local test runs do it\n"- ";; automatically. Raising a budget, adding one for a new linter (`--seed`), or widening the\n"- ";; exemptions is a hand edit to defend in your PR.\n"+ ";; automatically. Raising a budget, adding one (`--seed` for inline, by hand for config), or\n"+ ";; widening the exemptions is a hand edit to defend in your PR.\n"";; :all is the vector-less ignore form, which suppresses every linter on the next form.\n"))+(defn- render-counts+ [counts indent]+ (if (empty? counts)+ "{}"+ (let [entries (sort-by (comp str first) counts)+ width (apply max (map (comp count str first) entries))]+ (str "{"+ (str/join (str "\n" indent)+ (for [[linter n] entries]+ (format (str "%-" width "s %d") (str linter) n)))+ "}"))))+(defn render- "Text of the ratchets file for the `{:ignore-counts _, :comment-exempt _}` map `ratchets`.+ "Text of the ratchets file for the `{:ignore-counts _, :config-counts _, :comment-exempt _}` map.Byte-stable: [[fix!]] idempotency and the file-hygiene test depend on it."- [{:keys [ignore-counts comment-exempt]}]+ [{:keys [ignore-counts config-counts comment-exempt]}](let [counts-indent (apply str (repeat (count "{:ignore-counts {") \space))exempt-indent (apply str (repeat (count " :comment-exempt #{") \space))](str header- "{:ignore-counts "- (if (empty? ignore-counts)- "{}"- (let [entries (sort-by (comp str first) ignore-counts)- width (apply max (map (comp count str first) entries))]- (str "{"- (str/join (str "\n" counts-indent)- (for [[linter n] entries]- (format (str "%-" width "s %d") (str linter) n)))- "}")))+ "{:ignore-counts " (render-counts ignore-counts counts-indent)+ "\n :config-counts " (render-counts config-counts counts-indent)"\n :comment-exempt "(if (empty? comment-exempt)"#{}"@@ -326,7 +388,7 @@(defn change-report"The lines [[fix!]] prints: lowered/dropped/seeded budgets, dropped exemptions, plus warnings foranything over budget."- [{:keys [ignore-counts comment-exempt]} occurrences seeded]+ [{:keys [ignore-counts config-counts comment-exempt]} occurrences config-actual seeded](let [actual (actual-counts occurrences)](concat(for [linter seeded@@ -345,6 +407,12 @@(for [[linter n] (sort-by (comp str first) (apply dissoc actual (concat seeded (keys ignore-counts))))](format "WARNING: %s has %d ignores but no budget entry -- seed one with `./bin/mage fix-kondo-ratchets --seed %s`"linter n linter))+ (for [[linter {:keys [recorded actual]}] (config-drift config-counts config-actual)]+ (cond+ (zero? actual) (format "dropped config %s (no suppressions left)" linter)+ (< actual recorded) (format "lowered config %s %d -> %d" linter recorded actual)+ :else (format "WARNING: config suppressions for %s are over budget (%d recorded, %d actual) -- remove one from .clj-kondo/config.edn or raise the budget by hand"+ linter recorded actual)))(for [linter (stale-exemptions comment-exempt occurrences)](format "unexempted %s (all its ignores are justified now)" linter)))))@@ -355,15 +423,17 @@([](fix! nil))([{:keys [seed]}]- (let [{:keys [ignore-counts comment-exempt] :as ratchets} (read-ratchets)- occurrences (scan)- seeded (if seed [(keyword (str/replace-first seed #"^:" ""))] [])- actual (actual-counts occurrences)- text (render {:ignore-counts (lowered-counts ignore-counts actual seeded)- :comment-exempt (reduce disj comment-exempt (stale-exemptions comment-exempt occurrences))})- file (io/file ratchets-file)- old (when (.exists file) (slurp file))]- (run! println (change-report ratchets occurrences seeded))+ (let [{:keys [ignore-counts config-counts comment-exempt] :as ratchets} (read-ratchets)+ occurrences (scan)+ seeded (if seed [(keyword (str/replace-first seed #"^:" ""))] [])+ actual (actual-counts occurrences)+ config-actual (config-suppressions)+ text (render {:ignore-counts (lowered-counts ignore-counts actual seeded)+ :config-counts (lowered-counts config-counts config-actual [])+ :comment-exempt (reduce disj comment-exempt (stale-exemptions comment-exempt occurrences))})+ file (io/file ratchets-file)+ old (when (.exists file) (slurp file))]+ (run! println (change-report ratchets occurrences config-actual seeded))(if (= old text)(println "unchanged")(do (spit file text)diff --git a/test/metabase/core/kondo_ratchet_test.clj b/test/metabase/core/kondo_ratchet_test.cljindex 5b5b45d49d9..c7f4f016ab2 100644--- a/test/metabase/core/kondo_ratchet_test.clj+++ b/test/metabase/core/kondo_ratchet_test.clj@@ -66,6 +66,15 @@(is (= #{}(kondo-ratchet/stale-exemptions comment-exempt (tree-scan)))))))+(deftest ^:parallel config-budgets-match-actual-test+ (testing (str "\nConfig-level suppression budgets in " kondo-ratchet/ratchets-file " must match\n"+ ".clj-kondo/config.edn (:off switches and :exclude entries, per linter).\n"+ "Budget too low: remove the new config suppression, or raise the budget by hand and\n"+ "defend it in the PR. Budget too high: run `./bin/mage fix-kondo-ratchets`.")+ (is (= {}+ (kondo-ratchet/config-drift (:config-counts (kondo-ratchet/read-ratchets))+ (kondo-ratchet/config-suppressions))))))+(deftest ^:parallel ratchets-file-normalized-test(testing (str "\n" kondo-ratchet/ratchets-file " should be sorted and aligned exactly as the generator"" writes it.\nAfter a hand edit, run `./bin/mage fix-kondo-ratchets` to normalize the formatting.")@@ -214,18 +223,21 @@(deftest ^:parallel render-test(testing "keys come out sorted, values aligned, and the text round-trips losslessly"(let [ratchets {:ignore-counts {:discouraged-var 3, :all 1, :metabase/modules 2}+ :config-counts {:unresolved-symbol 18, :inline-def 1}:comment-exempt #{:metabase/modules :discouraged-var}}text (kondo-ratchet/render ratchets)](is (str/ends-with? text (str "{:ignore-counts {:all 1\n"" :discouraged-var 3\n"" :metabase/modules 2}\n"+ " :config-counts {:inline-def 1\n"+ " :unresolved-symbol 18}\n"" :comment-exempt #{:discouraged-var\n"" :metabase/modules}}\n")))(is (= ratchets (edn/read-string text)))(is (= text (kondo-ratchet/render (edn/read-string text))))))(testing "empty ratchets"- (is (str/ends-with? (kondo-ratchet/render {:ignore-counts {}, :comment-exempt #{}})- "{:ignore-counts {}\n :comment-exempt #{}}\n"))))+ (is (str/ends-with? (kondo-ratchet/render {:ignore-counts {}, :config-counts {}, :comment-exempt #{}})+ "{:ignore-counts {}\n :config-counts {}\n :comment-exempt #{}}\n"))))(deftest ^:parallel lowered-counts-test(is (= {:lower 3, :over-budget 5}@@ -281,6 +293,40 @@(kondo-ratchet/stale-exemptions #{:a :b} occurrences))":a still has an unjustified ignore; :b's are all justified so its exemption is stale")))+(deftest ^:parallel config-suppressions-test+ (is (= {:redundant-ignore 1+ :unresolved-symbol 5+ :discouraged-java-method 1+ :missing-docstring 2+ :discouraged-var 1+ :unused-referred-var 4+ :deprecated-var 1}+ (kondo-ratchet/config-suppressions+ '{:linters {:redundant-ignore {:level :off}+ :unresolved-symbol {:exclude [a b c]}+ :unused-referred-var {:exclude {compojure.core [GET DELETE POST PUT]}}+ :deprecated-var {:exclude {some.ns/old-var {:namespaces [caller.*]}}}+ :discouraged-var {clojure.core/println {:message "no"}}+ ;; nests two deep: class -> method+ :discouraged-java-method {java.lang.System {exit {:level :off}+ gc {:level :error}}}+ :equals-true {:level :warning}}+ :config-in-comment {:linters {:unresolved-symbol {:level :off}}}+ :config-in-ns {tests {:linters {:missing-docstring {:level :off}+ :discouraged-var {clojure.core/println {:level :off}}}}+ lib {:linters {:missing-docstring {:level :off}}}}+ :config-in-call {some.ns/with-thing {:linters {:unresolved-symbol {:level :off}}}}}))+ "an :off is 1, :exclude items count each (map values count their elements; a scoping map is one+ var), per-var re-allows count at any nesting depth, discouragements and enablements count+ nothing; groups, :config-in-comment and :config-in-call sum per linter"))++(deftest ^:parallel config-drift-test+ (is (= {:gone {:recorded 2, :actual 0}+ :new {:recorded 0, :actual 1}+ :up {:recorded 1, :actual 3}}+ (kondo-ratchet/config-drift {:gone 2, :same 5, :up 1}+ {:same 5, :new 1, :up 3}))))+(deftest ^:parallel change-report-test(let [occurrences (concat (for [[linter n] {:lower 3, :over 7, :new 9, :same 4}i (range n)]@@ -291,9 +337,14 @@"dropped :gone (no ignores left)""lowered :lower 5 -> 3""WARNING: :over is over budget (5 recorded, 7 actual) -- remove ignores, or accept them all with `--seed :over`"+ "dropped config :cfg-gone (no suppressions left)"+ "lowered config :cfg-lower 4 -> 2"+ "WARNING: config suppressions for :cfg-over are over budget (1 recorded, 3 actual) -- remove one from .clj-kondo/config.edn or raise the budget by hand""unexempted :polite (all its ignores are justified now)"](kondo-ratchet/change-report {:ignore-counts {:lower 5, :over 5, :gone 5, :same 4, :polite 1}+ :config-counts {:cfg-lower 4, :cfg-over 1, :cfg-gone 2, :cfg-same 6}:comment-exempt #{:lower :polite}}occurrences+ {:cfg-lower 2, :cfg-over 3, :cfg-same 6}[:new :void]))- "an untouched budget (:same) and a still-needed exemption (:lower) earn no line")))+ "untouched budgets (:same, :cfg-same) and a still-needed exemption (:lower) earn no line")))