The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Fix agent portable-FK resolution when two tables share a name across schemas (#74931)
Mihael Konjevićd5d356d3eca6
diff --git a/src/metabase/models/serialization/resolve/mp.clj b/src/metabase/models/serialization/resolve/mp.cljindex 5bcfdb5ae98..5a08cd42871 100644--- a/src/metabase/models/serialization/resolve/mp.clj+++ b/src/metabase/models/serialization/resolve/mp.clj@@ -37,6 +37,7 @@index, an in-memory test fixture, or a snapshot) can be supplied to make import usablewithout an application database."(:require+ [metabase.app-db.core :as mdb][metabase.lib.metadata :as lib.metadata][metabase.lib.metadata.protocols :as lib.metadata.protocols][metabase.models.serialization :as serdes]@@ -54,17 +55,45 @@(defn- db-name [metadata-provider](:name (lib.metadata/database metadata-provider)))-(defn- matching-tables- "Return all tables in `metadata-provider` that match `table-name`, optionally filtered by `schema`- (where `schema` may be nil for schemaless databases).+(defn- matching-tables-via-provider+ "Return all tables matching `table-name` in `metadata-provider`, post-filtered by `schema`+ (`schema` may be `nil` for schemaless databases).- Uses `metadatas-by-name`-style query via `metadatas` to avoid O(tables) scans on large providers."+ Used as a fallback for mock and checker providers. NOT safe for production: the+ application's cached metadata provider keys its by-name cache on `[type :name k]` with+ `:schema` dropped, so two warehouse tables sharing a `name` across schemas collapse to+ whichever one wrote the cache entry last. The schema post-filter here then yields 0+ candidates for the loser's schema. App-DB-backed callers must use+ [[matching-tables-via-app-db]]."[metadata-provider schema table-name](->> (lib.metadata.protocols/metadatas metadata-provider{:lib/type :metadata/table:name #{table-name}})(filter #(= (:schema %) schema))))+(defn- matching-tables-via-app-db+ "Return all tables matching the `(db-id, schema, table-name)` triple by direct application-DB+ query — same shape `resolve.db/import-table-fk` has always used. Bypasses the metadata+ provider entirely.++ Defective `(db_id, schema, name)` duplicates (allowed by the+ `001_update_migrations.yaml` `is_defective_duplicate` carve-out for pre-constraint rows)+ return more than one candidate so [[find-table]] can raise `:ambiguous-table`."+ [db-id schema table-name]+ (t2/select :metadata/table :db_id db-id :schema schema :name table-name))++(defn- app-db-backed-provider?+ "True when `metadata-provider` is part of the production app-DB-backed wrapper chain+ (`InvocationTracker → CachedProxyMetadataProvider → UncachedApplicationDatabaseMetadataProvider`).++ Mocks built via `lib.tu/mock-metadata-provider` don't extend `CachedMetadataProvider` and+ correctly return `false` here. The rare test that wraps a mock in `cached-metadata-provider`+ will return `true` — callers must treat the app-DB result as authoritative only when it's+ non-empty and otherwise fall back to the provider so that wrapped-mock case still resolves."+ [metadata-provider]+ (and (lib.metadata.protocols/cached-metadata-provider? metadata-provider)+ (mdb/db-is-set-up?)))+(defn- unknown-table-ex-info"Build an agent-facing `:unknown-table` error for a miss on portable FK`[db schema name]`.@@ -86,6 +115,23 @@:agent-error? true:path path}))+(defn- table-candidates+ "Resolve `(schema, table-name)` against `metadata-provider`. Tries+ [[matching-tables-via-app-db]] first for app-DB-backed providers and falls back to+ [[matching-tables-via-provider]] on empty.++ The fallback covers three cases: vanilla mocks (no `CachedMetadataProvider`, skip the+ app-DB attempt entirely), wrapped mocks with synthetic db ids that don't exist as+ `metabase_database` rows, and the rare production case where the app DB and the metadata+ provider's cache disagree about whether a table exists (e.g. provider holds a cached row+ for a table that was just deleted, or vice versa). In that last case the provider's view+ becomes authoritative, since the alternative is a confusing `:unknown-table` raised for a+ table the caller can plainly see via `entity_details`."+ [metadata-provider db-id schema table-name]+ (or (when (and db-id (app-db-backed-provider? metadata-provider))+ (seq (matching-tables-via-app-db db-id schema table-name)))+ (matching-tables-via-provider metadata-provider schema table-name)))+(defn- find-table"Resolve `[db-name, schema, table-name]` to a `:metadata/table` or throw with context.@@ -94,7 +140,7 @@re-hallucinating the same bad path. All error branches are marked`:agent-error? true` so the outer tool wrapper relays the message verbatim."[metadata-provider [path-db-name path-schema path-table-name :as path]]- (let [current-db (db-name metadata-provider)]+ (let [{current-db :name, current-db-id :id} (lib.metadata/database metadata-provider)](when-not (= path-db-name current-db)(throw (ex-info (tru "Portable table FK references database {0}, but metadata provider is for {1}."(pr-str path-db-name)@@ -103,20 +149,20 @@:error :unknown-table:agent-error? true:path path- :expected-db current-db}))))- (let [candidates (matching-tables metadata-provider path-schema path-table-name)]- (case (count candidates)- 0 (throw (unknown-table-ex-info metadata-provider path))- 1 (first candidates)- ;; Deliberately do NOT enumerate the matching `[schema name id]` tuples — the metadata- ;; provider is un-sandboxed, so a leaked candidate list could surface tables the caller- ;; cannot otherwise see (parity with the `unknown-table-ex-info` strip above).- (throw (ex-info (tru "Ambiguous portable table FK {0}: {1} candidates. Call `entity_details` with entity-type `database` to list available tables and retry with a more specific portable FK."- (pr-str path) (count candidates))- {:status-code 400- :error :ambiguous-table- :agent-error? true- :path path})))))+ :expected-db current-db})))+ (let [candidates (table-candidates metadata-provider current-db-id path-schema path-table-name)]+ (case (count candidates)+ 0 (throw (unknown-table-ex-info metadata-provider path))+ 1 (first candidates)+ ;; Deliberately do NOT enumerate the matching `[schema name id]` tuples — the metadata+ ;; provider is un-sandboxed, so a leaked candidate list could surface tables the caller+ ;; cannot otherwise see (parity with the `unknown-table-ex-info` strip above).+ (throw (ex-info (tru "Ambiguous portable table FK {0}: {1} candidates. Call `entity_details` with entity-type `database` to list available tables and retry with a more specific portable FK."+ (pr-str path) (count candidates))+ {:status-code 400+ :error :ambiguous-table+ :agent-error? true+ :path path}))))))(defn- find-field"Resolve a field path `[db schema table field …]` to a `:metadata/column` or throw with context.diff --git a/test/metabase/models/serialization/resolve/mp_test.clj b/test/metabase/models/serialization/resolve/mp_test.cljindex bcf0a319f24..7466f9a43af 100644--- a/test/metabase/models/serialization/resolve/mp_test.clj+++ b/test/metabase/models/serialization/resolve/mp_test.clj@@ -116,6 +116,31 @@(is (= 30 (resolve/import-table-fk r ["DW" "RAW" "ORDERS"])))(is (= 31 (resolve/import-table-fk r ["DW" "CLEAN" "ORDERS"]))))))+(deftest import-table-fk-cache-collision-test+ (testing "two real tables sharing a name across schemas: app-DB-backed resolver must not drop either"+ ;; Reproduces a production failure where `query` / `construct_query` returned+ ;; `:unknown-table` for a portable FK that `entity_details` had just emitted, then+ ;; verifies the fix.+ ;;+ ;; Production wraps every `application-database-metadata-provider` with+ ;; `cached-metadata-provider`. The cached provider's by-name cache key drops `:schema`+ ;; and stores at most one metadata per requested name, so when two warehouse tables+ ;; share a `name` across schemas, `(p/metadatas mp {:name #{n}})` returns at most one+ ;; row -- the schema post-filter in `find-table` then yields 0 candidates for the+ ;; schema that didn't win the cache write.+ ;;+ ;; The fix in [[resolve.mp/find-table]] bypasses the metadata provider for app-DB-backed+ ;; lookups and queries `metabase_table` directly with schema in the WHERE clause, the+ ;; same shape `metabase.models.serialization.resolve.db/import-table-fk` has always+ ;; used.+ (mt/with-temp [:model/Database db {:name (str "DW " (random-uuid)) :engine :h2}+ :model/Table raw-orders {:name "ORDERS" :schema "RAW" :db_id (:id db)}+ :model/Table clean-orders {:name "ORDERS" :schema "CLEAN" :db_id (:id db)}]+ (let [mp (lib-be/application-database-metadata-provider (:id db))+ r (resolve.mp/import-resolver mp)]+ (is (= (:id raw-orders) (resolve/import-table-fk r [(:name db) "RAW" "ORDERS"])))+ (is (= (:id clean-orders) (resolve/import-table-fk r [(:name db) "CLEAN" "ORDERS"])))))))+(deftest ^:parallel import-table-fk-error-test(testing "unknown table name → :unknown-table, agent-error?, 400, no info leak"(let [r (resolve.mp/import-resolver mp-simple)]