The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Make app DB authoritative for table existence in table-candidates (#75588)
Chris Truter6d7cbe822cdb
diff --git a/src/metabase/models/serialization/resolve/mp.clj b/src/metabase/models/serialization/resolve/mp.cljindex 954273339b5..45eec8a3561 100644--- a/src/metabase/models/serialization/resolve/mp.clj+++ b/src/metabase/models/serialization/resolve/mp.clj@@ -76,17 +76,19 @@(not (false? (:active %)))))))(defn- matching-tables-via-app-db- "Return active tables matching the `(db-id, schema, table-name)` triple by direct application-DB+ "Return all tables matching the `(db-id, schema, table-name)` triple by direct application-DBquery — same shape `resolve.db/import-table-fk` has always used. Bypasses the metadataprovider entirely.- Filters `:active true` in the WHERE clause so stale FKs to inactive tables don't resolve.+ Returns inactive rows too: the app DB is authoritative for *existence*, and+ [[table-candidates]] needs to tell \"no row at all\" (fall back to the provider) apart from+ \"only inactive rows\" (an authoritative miss). It does the `:active` filtering itself.Defective `(db_id, schema, name)` duplicates (allowed by the`001_update_migrations.yaml` `is_defective_duplicate` carve-out for pre-constraint rows)return more than one candidate so [[find-table]] can raise `:ambiguous-table`."[db-id schema table-name]- (t2/select :metadata/table :db_id db-id :schema schema :name table-name :active true))+ (t2/select :metadata/table :db_id db-id :schema schema :name table-name))(defn- app-db-backed-provider?"True when `metadata-provider` is part of the production app-DB-backed wrapper chain@@ -127,23 +129,28 @@:path path}))(defn- table-candidates- "Resolve `(schema, table-name)` against `metadata-provider`. Tries- [[matching-tables-via-app-db]] first for app-DB-backed providers and falls back to- [[matching-tables-via-provider]] on empty.-- The fallback covers three cases: vanilla mocks (no `CachedMetadataProvider`, skip the- app-DB attempt entirely), wrapped mocks with synthetic db ids that don't exist as- `metabase_database` rows, and the rare production case where the app DB and the metadata- provider's cache disagree about whether a table exists (e.g. provider holds a cached row- for a table that was just deleted, or vice versa). In that last case the provider's view- becomes authoritative, since the alternative is a confusing `:unknown-table` raised for a- table the caller can plainly see via `entity_details`.-- Returns only active tables — both helpers drop inactive (`:active false`) rows."+ "Resolve `(schema, table-name)` against `metadata-provider`, returning only active tables.++ For app-DB-backed providers the app DB is authoritative for existence. If+ [[matching-tables-via-app-db]] finds any row for the triple, we return just the active ones:+ an inactive-only match is an authoritative 0-candidate miss, NOT a reason to fall back. The+ fallback must not run here — the provider's by-name cache can hold a stale `:active true`+ row from before the table was deleted / re-uploaded (BOT-739-adjacent), and falling back+ would resurrect it.++ We fall back to [[matching-tables-via-provider]] only when the app DB has no row for the+ triple at all (or the provider isn't app-DB-backed): vanilla mocks (no+ `CachedMetadataProvider`, skip the app-DB attempt entirely), wrapped mocks with synthetic db+ ids that don't exist as `metabase_database` rows, and the rare production case where the+ provider's cache holds a table the app DB has fully dropped. In that last case the+ provider's view becomes authoritative, since the alternative is a confusing `:unknown-table`+ raised for a table the caller can plainly see via `entity_details`."[metadata-provider db-id schema table-name]- (or (when (and db-id (app-db-backed-provider? metadata-provider))- (seq (matching-tables-via-app-db db-id schema table-name)))- (matching-tables-via-provider metadata-provider schema table-name)))+ (if-let [app-db-rows (and db-id+ (app-db-backed-provider? metadata-provider)+ (seq (matching-tables-via-app-db db-id schema table-name)))]+ (filter :active app-db-rows)+ (matching-tables-via-provider metadata-provider schema table-name)))(defn- find-table"Resolve `[db-name, schema, table-name]` to a `:metadata/table` or throw with context.diff --git a/test/metabase/models/serialization/resolve/mp_test.clj b/test/metabase/models/serialization/resolve/mp_test.cljindex 97ed6a82d9b..516b5135709 100644--- a/test/metabase/models/serialization/resolve/mp_test.clj+++ b/test/metabase/models/serialization/resolve/mp_test.clj@@ -14,7 +14,8 @@[metabase.models.serialization.resolve :as resolve][metabase.models.serialization.resolve.mp :as resolve.mp][metabase.test :as mt]- [metabase.test.fixtures :as fixtures]))+ [metabase.test.fixtures :as fixtures]+ [toucan2.core :as t2]))(set! *warn-on-reflection* true)@@ -169,6 +170,31 @@(is (= (str/replace msg #"\[.*?\]" "[FK]")(str/replace never-existed #"\[.*?\]" "[FK]"))))))))))))+(deftest import-table-fk-inactive-after-cache-warmed-test+ (testing "a table cached while active, then marked inactive, must NOT resolve via the stale cache"+ ;; The dangerous case the app-DB-existence check guards against: the cached metadata provider+ ;; warms its by-name cache with the `:active true` row, then the table is marked inactive+ ;; (deleted / re-uploaded upload). The cache still surfaces the stale active row, so+ ;; `table-candidates` MUST consult the app DB for existence — find the inactive row — and treat+ ;; it as a 0-candidate miss rather than falling back to the stale cache.+ (mt/with-temp [:model/Database db {:name (str "Uploads " (random-uuid)) :engine :h2}+ :model/Table gone {:name "metabot2_cafef00dbabe01" :schema "PUBLIC"+ :db_id (:id db) :active true}]+ (let [mp (lib-be/application-database-metadata-provider (:id db))+ r (resolve.mp/import-resolver mp)+ by-name #(lib.metadata.protocols/metadatas+ mp {:lib/type :metadata/table :name #{"metabot2_cafef00dbabe01"}})]+ (testing "warm the provider's by-name cache while the table is still active"+ (is (= [true] (mapv :active (by-name))))+ (is (= (:id gone) (resolve/import-table-fk r [(:name db) "PUBLIC" "metabot2_cafef00dbabe01"]))))+ (t2/update! :model/Table (:id gone) {:active false})+ (testing "the cache still surfaces the stale ACTIVE row by name"+ (is (= [true] (mapv :active (by-name)))))+ (testing "...but import-table-fk misses: the app DB sees the inactive row, no stale-cache fallback"+ (is (thrown-with-msg?+ clojure.lang.ExceptionInfo #"No table found matching portable FK"+ (resolve/import-table-fk r [(:name db) "PUBLIC" "metabot2_cafef00dbabe01"]))))))))+(deftest matching-tables-via-provider-drops-inactive-test(testing "matching-tables-via-provider filters the inactive rows the provider surfaces by name";; Isolates the provider-side filter. By-name `metadatas` lookups skip the provider's SQL