The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Read settings without the cache while reconciling their encryption (#81491)
Alexander Polyankin0c54590106e3
diff --git a/src/metabase/settings/models/setting.clj b/src/metabase/settings/models/setting.cljindex a8df93fa705..57cda655e8f 100644--- a/src/metabase/settings/models/setting.clj+++ b/src/metabase/settings/models/setting.clj@@ -1740,24 +1740,30 @@there) or re-writes as plaintext every row its registry knew as `:encryption :no`, after the one-shot encryptionmigrations have already run -- and such a row would otherwise fail the strict read and take the whole settingscache down with it. Works around the standard getters/setters deliberately: values are rewritten byte-identical- modulo encryption. No-op when MB_ENCRYPTION_SECRET_KEY is not set."+ modulo encryption. No-op when MB_ENCRYPTION_SECRET_KEY is not set.++ Runs with the settings cache disabled: any setting consulted while this runs (e.g. `read-only-mode`, which the+ cloud-migration DML guard reads on every write this issues) is read directly from the DB. Restoring the cache+ strictly decrypts every row -- including the very rows this function exists to repair -- so going through it here+ would fail the repair on exactly the state it is repairing."[](when (encryption/default-encryption-enabled?)- (let [{encrypting true, plaintext false} (group-by (comp boolean encrypts?) (vals @registered-settings))]- (t2/with-transaction [_conn]- (doseq [{v :value k :key}- (t2/select :setting {:for :update :where [:and- [:in :key (map setting-name plaintext)]- ;; these are *definitely* decrypted already, let's not bother looking- [:not [:in :value ["true" "false"]]]]})- :when (encryption/decryptable-string? v)]- (t2/update! :setting :key k {:value (encryption/decrypt v)}))- (doseq [{v :value k :key}- (t2/select :setting {:for :update :where [:and- [:in :key (map setting-name encrypting)]- [:!= :value nil]]})- :when (not (encryption/decryptable-string? v))]- (t2/update! :setting :key k {:value (encryption/encrypt v)}))))))+ (binding [config/*disable-setting-cache* true]+ (let [{encrypting true, plaintext false} (group-by (comp boolean encrypts?) (vals @registered-settings))]+ (t2/with-transaction [_conn]+ (doseq [{v :value k :key}+ (t2/select :setting {:for :update :where [:and+ [:in :key (map setting-name plaintext)]+ ;; these are *definitely* decrypted already, let's not bother looking+ [:not [:in :value ["true" "false"]]]]})+ :when (encryption/decryptable-string? v)]+ (t2/update! :setting :key k {:value (encryption/decrypt v)}))+ (doseq [{v :value k :key}+ (t2/select :setting {:for :update :where [:and+ [:in :key (map setting-name encrypting)]+ [:!= :value nil]]})+ :when (not (encryption/decryptable-string? v))]+ (t2/update! :setting :key k {:value (encryption/encrypt v)})))))))(defn- maybe-encrypt [setting-model];; In tests, sometimes we need to insert/update settings that don't have definitions in the code and therefore can'tdiff --git a/test/metabase/settings/models/setting_test.clj b/test/metabase/settings/models/setting_test.cljindex e516ca2d954..a53d7092807 100644--- a/test/metabase/settings/models/setting_test.clj+++ b/test/metabase/settings/models/setting_test.clj@@ -7,6 +7,7 @@[medley.core :as m][metabase.app-db.connection :as mdb.connection][metabase.app-db.core :as mdb]+ [metabase.cloud-migration.models.cloud-migration :as cloud-migration][metabase.config.core :as config][metabase.settings.models.setting :as setting :refer [defsetting]][metabase.settings.models.setting.cache :as setting.cache]@@ -22,6 +23,12 @@(:import(clojure.lang ExceptionInfo)))+(set! *warn-on-reflection* true)++;; side-effect require: registers the DML build guard exercised by+;; [[migrate-encrypted-settings!-does-not-depend-on-settings-cache-test]]+(comment cloud-migration/keep-me)+(use-fixtures :once (fixtures/initialize :db));; ## TEST SETTINGS DEFINITIONS@@ -1781,6 +1788,27 @@(setting/migrate-encrypted-settings!)(is (not= "foobar" (actual-value-in-db :test-never-encrypted-setting)))))))+(deftest migrate-encrypted-settings!-does-not-depend-on-settings-cache-test+ ;; The cloud-migration read-only-mode guard (a `t2.pipeline/build :before` method registered when+ ;; `metabase.cloud-migration.models.cloud-migration` loads -- required above so this holds in a targeted test run+ ;; too) runs on every DML statement and reads a setting. On a fresh JVM that read triggers a full strict+ ;; settings-cache restore, which fails on the very plaintext row this function exists to repair -- so the repair+ ;; itself must never go through the cache. Regression test for the chicken-and-egg startup crash.+ (mt/with-temp-empty-app-db [_conn :h2]+ (mdb/setup-db! :create-sample-content? false)+ (encryption-test/with-secret-key "ABCDEFGH12345678"+ (t2/insert! :setting {:key "toucan-name" :value "Lenny"})+ (binding [config/*disable-setting-cache* false]+ ;; Simulate a fresh JVM. `setting.cache/cache*` is the atom holding this app DB's in-memory settings map; nil+ ;; means never populated, so the next cached read must do the full (strictly decrypting) restore. And+ ;; `last-update-check` is the AtomicLong nanotime of the last staleness check: zeroing it defeats the+ ;; one-minute throttle that otherwise skips the check entirely in a warm test JVM.+ (reset! (#'setting.cache/cache*) nil)+ (.set ^java.util.concurrent.atomic.AtomicLong @#'setting.cache/last-update-check 0)+ (setting/migrate-encrypted-settings!))+ (is (encryption/decryptable-string? (actual-value-in-db :toucan-name)))+ (is (= "Lenny" (encryption/decrypt (actual-value-in-db :toucan-name)))))))+(deftest migrate-encrypted-settings!-encrypts-strict-settings;; raw :setting (not :model/Setting) throughout: the model's before-insert would encrypt the value, and these tests;; need genuinely plaintext rows at rest. Isolated app DB for the same reason as [[migrate-encrypted-settings!-works]].