Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Collapse uniform blocked view-data table permissions to db-level rows (#77130)

Alexander Polyankin21f2c65b15d3
diff --git a/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml b/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml
new file mode 100644
index 00000000000..25532bec019
--- /dev/null
+++ b/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml
@@ -0,0 +1,138 @@
+## Collapse redundant table-level data_permissions rows into db-level rows.
+##
+## A bug in the new-table default-permissions path ("going granular") deleted a group's db-level row and
+## wrote one row per table even when the db-level row was already `blocked` and every table's new value was
+## `blocked`. On instances with many databases and groups this ballooned data_permissions into tens of
+## millions of rows encoding no information beyond their db-level equivalents, which in turn caused
+## OOMs in code paths that read a user's or the whole instance's permission rows (#76077).
+##
+## Two steps, both restricted to perms/view-data and semantically lossless:
+## 1. For each (group_id, db_id) with no db-level view-data row whose table-level view-data rows ALL
+## carry the value 'blocked' — including rows referencing inactive tables, so collapsing never
+## discards a value a reactivated table would otherwise pick back up — and cover every active table
+## of the database: insert the equivalent db-level row.
+## 2. Drop every table-level view-data row whose (group_id, db_id, perm_value) matches a db-level
+## row — such rows are redundant no matter how they came to be. Because that can be tens of millions
+## of rows, this is done by staging the surviving rows, truncating the table, and reloading them,
+## rather than a row-by-row DELETE (which scales super-linearly); see the second changeSet.
+##
+## Restricted to view-data / blocked because:
+## - only the least-permissive value is safe to collapse at all: table-level rows are ignored for
+## inactive tables (they read as least-permissive), while a db-level row applies to inactive tables
+## too, so collapsing any other value would silently upgrade inactive tables;
+## - the permissions graph API intentionally reports other perm types' table-level rows as granular
+## even when uniform (#73520), so collapsing them would change what the admin UI displays;
+## - the going-granular bug only ever produced view-data rows, so this still covers the explosion.
+##
+## Combos with partial table coverage, mixed values, or a uniform non-blocked value are intentionally
+## left untouched.
+##
+## Lives in 062 so the changeSet runs on upgrades to v62 and later (the fix is not backported earlier).
+
+databaseChangeLog:
+ - logicalFilePath: migrations/062_update_migrations.yaml
+
+ - changeSet:
+ id: v62.2026-07-07T00:00:00
+ author: ranquild
+ comment: Insert db-level data_permissions rows for uniform full-coverage blocked table-level view-data permissions (#76077)
+ changes:
+ - sql:
+ sql: >-
+ INSERT INTO data_permissions (group_id, perm_type, db_id, perm_value)
+ SELECT dp.group_id, dp.perm_type, dp.db_id, MIN(dp.perm_value)
+ FROM data_permissions dp
+ LEFT JOIN metabase_table mt ON mt.id = dp.table_id
+ JOIN (SELECT db_id, COUNT(*) AS n FROM metabase_table WHERE active = TRUE GROUP BY db_id) tc
+ ON tc.db_id = dp.db_id
+ WHERE dp.perm_type = 'perms/view-data'
+ AND dp.table_id IS NOT NULL
+ AND NOT EXISTS (
+ SELECT 1 FROM data_permissions dbp
+ WHERE dbp.group_id = dp.group_id
+ AND dbp.db_id = dp.db_id
+ AND dbp.perm_type = dp.perm_type
+ AND dbp.table_id IS NULL)
+ GROUP BY dp.group_id, dp.db_id, dp.perm_type, tc.n
+ HAVING COUNT(DISTINCT dp.perm_value) = 1
+ AND MIN(dp.perm_value) = 'blocked'
+ AND COUNT(DISTINCT CASE WHEN mt.active = TRUE THEN dp.table_id END) = tc.n;
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
+
+ # The rows to keep are every data_permissions row EXCEPT the table-level view-data rows that a db-level
+ # row of the same (group, db, value) has made redundant — the anti-join below. Deleting the redundant rows
+ # directly scales super-linearly on a table ballooned to tens of millions of rows (each delete maintains
+ # six index trees), so instead of a DELETE we stage the few thousand survivors, empty the table, and
+ # reload them. Migrations run before the instance serves traffic, so the brief window in which
+ # data_permissions is empty is never observed; the survivors persist in the staging table until the reload
+ # finishes (each step is its own changeSet, so a failure resumes from the incomplete step with the staging
+ # table still present), and because the table is only truncated — never dropped — its indexes and foreign
+ # keys stay in place. The steps are split into one changeSet each because the migration linter allows only
+ # a single change per changeSet.
+ - changeSet:
+ id: v62.2026-07-07T00:00:01
+ author: ranquild
+ comment: Create the staging table that will hold the surviving data_permissions rows (#76077)
+ changes:
+ # A createTable change (rather than a raw CREATE TABLE ... AS SELECT) so Liquibase renders it per
+ # engine and, on MySQL/MariaDB, appends the ENGINE/CHARACTER SET/COLLATE clause the app-db expects.
+ - createTable:
+ tableName: data_permissions_collapse_keep
+ remarks: Transient staging table holding the data_permissions rows to keep while the table is collapsed; dropped at the end of this migration
+ columns:
+ - column: {name: id, type: int, remarks: Copy of data_permissions.id}
+ - column: {name: group_id, type: int, remarks: Copy of data_permissions.group_id}
+ - column: {name: perm_type, type: varchar(64), remarks: Copy of data_permissions.perm_type}
+ - column: {name: db_id, type: int, remarks: Copy of data_permissions.db_id}
+ - column: {name: schema_name, type: varchar(254), remarks: Copy of data_permissions.schema_name}
+ - column: {name: table_id, type: int, remarks: Copy of data_permissions.table_id}
+ - column: {name: perm_value, type: varchar(64), remarks: Copy of data_permissions.perm_value}
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
+
+ - changeSet:
+ id: v62.2026-07-07T00:00:02
+ author: ranquild
+ comment: Stage the surviving data_permissions rows before collapsing the redundant ones (#76077)
+ changes:
+ - sql:
+ sql: >-
+ INSERT INTO data_permissions_collapse_keep (id, group_id, perm_type, db_id, schema_name, table_id, perm_value)
+ SELECT dp.id, dp.group_id, dp.perm_type, dp.db_id, dp.schema_name, dp.table_id, dp.perm_value
+ FROM data_permissions dp
+ LEFT JOIN data_permissions dbp
+ ON dp.perm_type = 'perms/view-data' AND dp.table_id IS NOT NULL
+ AND dbp.group_id = dp.group_id AND dbp.db_id = dp.db_id
+ AND dbp.perm_type = dp.perm_type AND dbp.perm_value = dp.perm_value
+ AND dbp.table_id IS NULL
+ WHERE dbp.id IS NULL;
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
+
+ - changeSet:
+ id: v62.2026-07-07T00:00:03
+ author: ranquild
+ comment: Empty data_permissions before reloading only the surviving rows (#76077)
+ changes:
+ - sql:
+ sql: TRUNCATE TABLE data_permissions;
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
+
+ - changeSet:
+ id: v62.2026-07-07T00:00:04
+ author: ranquild
+ comment: Reload the surviving data_permissions rows (#76077)
+ changes:
+ - sql:
+ sql: >-
+ INSERT INTO data_permissions (id, group_id, perm_type, db_id, schema_name, table_id, perm_value)
+ SELECT id, group_id, perm_type, db_id, schema_name, table_id, perm_value
+ FROM data_permissions_collapse_keep;
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
+
+ - changeSet:
+ id: v62.2026-07-07T00:00:05
+ author: ranquild
+ comment: Drop the staging table used to collapse data_permissions (#76077)
+ changes:
+ - sql:
+ sql: DROP TABLE data_permissions_collapse_keep;
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
diff --git a/src/metabase/permissions/models/data_permissions.clj b/src/metabase/permissions/models/data_permissions.clj
index 89f1fb177f4..f12fff8a150 100644
--- a/src/metabase/permissions/models/data_permissions.clj
+++ b/src/metabase/permissions/models/data_permissions.clj
@@ -1176,14 +1176,19 @@
[{:keys [db-id db-level-idx all-db-tables batch-table-ids]}
[[group-id perm-type] entries]]
(let [db-perm (get db-level-idx [group-id perm-type])
- any-blocked? (and db-perm (some #(= :blocked (:actual-value %)) entries))
+ ;; Only go granular when a batch table needs `:blocked` and the DB-level row has some *other*
+ ;; value: a `:blocked` DB-level row already covers the new tables, and expanding it would write
+ ;; one redundant row per table (see #76077, where this ballooned data_permissions to 46M rows).
+ go-granular? (and db-perm
+ (not= :blocked (:perm_value db-perm))
+ (some #(= :blocked (:actual-value %)) entries))
mk (fn [v table-id schema]
(mk-perm-row group-id perm-type v db-id table-id schema))]
(cond
;; Going-granular fires once for the whole `(group, perm-type)`:
;; delete the DB-level row, expand non-batch tables to the old value,
;; then write each batch table at its actual-value.
- any-blocked?
+ go-granular?
(let [expansion-value (case (:perm_value db-perm)
:query-builder-and-native :query-builder
(:perm_value db-perm))]
@@ -1229,8 +1234,8 @@
(defn set-default-table-permissions!
"Set default permissions for a newly-created table across all relevant
groups. Handles three cases per `(group, perm-type)`:
- - Group has DB-level perm matching default → no-op
- - Group has DB-level perm but new table needs `:blocked` → going-granular
+ - Group has DB-level perm covering the default (including `:blocked` covering `:blocked`) → no-op
+ - Group has non-`:blocked` DB-level perm but new table needs `:blocked` → going-granular
- Group has no DB-level perm → simple insert
`group-perm-defaults` is a seq of `{:group-id :perm-type :default-value}`
diff --git a/test/metabase/app_db/schema_migrations_test.clj b/test/metabase/app_db/schema_migrations_test.clj
index 9dd4f82003d..3f0d427c763 100644
--- a/test/metabase/app_db/schema_migrations_test.clj
+++ b/test/metabase/app_db/schema_migrations_test.clj
@@ -2888,6 +2888,94 @@
:created_at :%now}
attrs)))
+(deftest collapse-uniform-table-permissions-migration-test
+ (testing "Migrations v62.2026-07-07T00:00:00 through v62.2026-07-07T00:00:05:
+ uniform full-coverage :blocked view-data table rows collapse to a db-level row (#76077)"
+ (impl/test-migrations ["v62.2026-07-07T00:00:00" "v62.2026-07-07T00:00:05"] [migrate!]
+ (let [group-id (t2/insert-returning-pk! :permissions_group {:name "collapse-test-group"})
+ db! (fn [db-name]
+ (t2/insert-returning-pk! :metabase_database
+ {:details "{}"
+ :created_at :%now
+ :updated_at :%now
+ :engine "h2"
+ :is_sample false
+ :name db-name}))
+ table! (fn [db-id active?]
+ (t2/insert-returning-pk! :metabase_table
+ {:db_id db-id
+ :name (mt/random-name)
+ :schema "PUBLIC"
+ :created_at :%now
+ :updated_at :%now
+ :active active?}))
+ perm! (fn [db-id table-id perm-type perm-value]
+ (t2/insert! :data_permissions
+ {:group_id group-id
+ :db_id db-id
+ :table_id table-id
+ :schema_name (when table-id "PUBLIC")
+ :perm_type perm-type
+ :perm_value perm-value}))
+ rows (fn [db-id perm-type]
+ (t2/select-fn-set (juxt :table_id :perm_value) :data_permissions
+ :group_id group-id :db_id db-id :perm_type perm-type))
+ ;; db-1: uniform :blocked covering all active tables (+ a :blocked row for an inactive table)
+ ;; → collapses
+ db-1 (db! "collapse-uniform")
+ t1a (table! db-1 true)
+ t1b (table! db-1 true)
+ t1-off (table! db-1 false)
+ ;; db-2: partial coverage → untouched
+ db-2 (db! "collapse-partial")
+ t2a (table! db-2 true)
+ _t2b (table! db-2 true)
+ ;; db-3: mixed values → untouched
+ db-3 (db! "collapse-mixed")
+ t3a (table! db-3 true)
+ t3b (table! db-3 true)
+ ;; db-4: uniform but :unrestricted → untouched
+ db-4 (db! "collapse-unrestricted")
+ t4a (table! db-4 true)
+ ;; db-5: uniform :no create-queries (not view-data) → untouched
+ db-5 (db! "collapse-other-type")
+ t5a (table! db-5 true)
+ ;; db-6: active tables uniformly :blocked, but an INACTIVE table's row differs → untouched;
+ ;; the differing row must survive so a reactivated table picks its value back up
+ db-6 (db! "collapse-inactive-differs")
+ t6a (table! db-6 true)
+ t6-off (table! db-6 false)
+ ;; db-7: table row duplicating an equal-value db-level row → duplicate deleted
+ db-7 (db! "collapse-dup-of-db-level")
+ t7a (table! db-7 true)]
+ (perm! db-1 t1a "perms/view-data" "blocked")
+ (perm! db-1 t1b "perms/view-data" "blocked")
+ (perm! db-1 t1-off "perms/view-data" "blocked")
+ (perm! db-2 t2a "perms/view-data" "blocked")
+ (perm! db-3 t3a "perms/view-data" "blocked")
+ (perm! db-3 t3b "perms/view-data" "unrestricted")
+ (perm! db-4 t4a "perms/view-data" "unrestricted")
+ (perm! db-5 t5a "perms/create-queries" "no")
+ (perm! db-6 t6a "perms/view-data" "blocked")
+ (perm! db-6 t6-off "perms/view-data" "unrestricted")
+ (perm! db-7 nil "perms/view-data" "unrestricted")
+ (perm! db-7 t7a "perms/view-data" "unrestricted")
+ (migrate!)
+ (testing "uniform :blocked collapses to a single db-level row, dropping the inactive table's row too"
+ (is (= #{[nil "blocked"]} (rows db-1 "perms/view-data"))))
+ (testing "partial coverage untouched"
+ (is (= #{[t2a "blocked"]} (rows db-2 "perms/view-data"))))
+ (testing "mixed values untouched"
+ (is (= #{[t3a "blocked"] [t3b "unrestricted"]} (rows db-3 "perms/view-data"))))
+ (testing "uniform non-blocked value untouched"
+ (is (= #{[t4a "unrestricted"]} (rows db-4 "perms/view-data"))))
+ (testing "non-view-data perm types untouched"
+ (is (= #{[t5a "no"]} (rows db-5 "perms/create-queries"))))
+ (testing "a differing value on an inactive table's row blocks the collapse"
+ (is (= #{[t6a "blocked"] [t6-off "unrestricted"]} (rows db-6 "perms/view-data"))))
+ (testing "table rows duplicating an equal-value db-level row are deleted"
+ (is (= #{[nil "unrestricted"]} (rows db-7 "perms/view-data"))))))))
+
(defn- collection-entity-id
[collection-id]
(str/trim (t2/select-one-fn :entity_id :collection :id collection-id)))
diff --git a/test/metabase/permissions/models/data_permissions_test.clj b/test/metabase/permissions/models/data_permissions_test.clj
index 86faaa7047a..4e9bb4b83ac 100644
--- a/test/metabase/permissions/models/data_permissions_test.clj
+++ b/test/metabase/permissions/models/data_permissions_test.clj
@@ -896,6 +896,30 @@
:group_id group-id :db_id db-id
:table_id table-id-3 :perm_type :perms/view-data))))))))
+(deftest set-default-table-permissions!-blocked-db-stays-collapsed-test
+ (testing "A :blocked DB-level row already covers a new :blocked table, so we don't go granular (#76077)"
+ (mt/with-temp [:model/Database {db-id :id} {}
+ :model/PermissionsGroup {group-id :id} {}
+ :model/Table {table-id-1 :id} {:db_id db-id :schema "PUBLIC" :active true}]
+ (data-perms/set-database-permission! group-id db-id :perms/view-data :blocked)
+ (mt/with-temp [:model/Table {table-id-2 :id} {:db_id db-id :schema "PUBLIC" :active true}]
+ (t2/delete! :model/DataPermissions :group_id group-id :table_id table-id-2 :perm_type :perms/view-data)
+ (mt/with-dynamic-fn-redefs [data-perms/new-table-view-data-permission-levels
+ (fn [_db-id group-ids] (zipmap group-ids (repeat :blocked)))]
+ (data-perms/set-default-table-permissions!
+ {:id table-id-2 :db_id db-id :schema "PUBLIC"}
+ [{:group-id group-id :perm-type :perms/view-data :default-value :unrestricted}]))
+ (testing "the DB-level :blocked row remains and no per-table rows were written"
+ (is (= :blocked (t2/select-one-fn :perm_value :model/DataPermissions
+ :group_id group-id :db_id db-id :table_id nil
+ :perm_type :perms/view-data)))
+ (is (zero? (t2/count :model/DataPermissions
+ :group_id group-id :db_id db-id :perm_type :perms/view-data
+ :table_id [:not= nil]))))
+ (testing "effective permission for the new table is still :blocked"
+ (is (= :blocked (data-perms/table-permission-for-groups #{group-id} :perms/view-data db-id table-id-2)))
+ (is (= :blocked (data-perms/table-permission-for-groups #{group-id} :perms/view-data db-id table-id-1))))))))
+
(deftest set-default-table-permissions!-simple-insert-test
(testing "When group is already table-granular, new table gets a simple insert"
(mt/with-temp [:model/Database {db-id :id} {}