Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Replace timeout-prone permissions-collapse migration with aggregate-first query (#77727)

Alexander Polyankinfcd561b654d7
diff --git a/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml b/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml
index 25532bec019..0f2ddef2d46 100644
--- a/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml
+++ b/resources/migrations/062/20260707_collapse_uniform_table_permissions.yaml
@@ -1,81 +1,141 @@
-## Collapse redundant table-level data_permissions rows into db-level rows.
+## Collapse redundant table-level data_permissions rows into db-level rows (#76077).
##
-## A bug in the new-table default-permissions path ("going granular") deleted a group's db-level row and
-## wrote one row per table even when the db-level row was already `blocked` and every table's new value was
-## `blocked`. On instances with many databases and groups this ballooned data_permissions into tens of
-## millions of rows encoding no information beyond their db-level equivalents, which in turn caused
-## OOMs in code paths that read a user's or the whole instance's permission rows (#76077).
-##
-## Two steps, both restricted to perms/view-data and semantically lossless:
-## 1. For each (group_id, db_id) with no db-level view-data row whose table-level view-data rows ALL
-## carry the value 'blocked' — including rows referencing inactive tables, so collapsing never
-## discards a value a reactivated table would otherwise pick back up — and cover every active table
-## of the database: insert the equivalent db-level row.
-## 2. Drop every table-level view-data row whose (group_id, db_id, perm_value) matches a db-level
-## row — such rows are redundant no matter how they came to be. Because that can be tens of millions
-## of rows, this is done by staging the surviving rows, truncating the table, and reloading them,
-## rather than a row-by-row DELETE (which scales super-linearly); see the second changeSet.
-##
-## Restricted to view-data / blocked because:
-## - only the least-permissive value is safe to collapse at all: table-level rows are ignored for
-## inactive tables (they read as least-permissive), while a db-level row applies to inactive tables
-## too, so collapsing any other value would silently upgrade inactive tables;
-## - the permissions graph API intentionally reports other perm types' table-level rows as granular
-## even when uniform (#73520), so collapsing them would change what the admin UI displays;
-## - the going-granular bug only ever produced view-data rows, so this still covers the explosion.
-##
-## Combos with partial table coverage, mixed values, or a uniform non-blocked value are intentionally
-## left untouched.
-##
-## Lives in 062 so the changeSet runs on upgrades to v62 and later (the fix is not backported earlier).
+## A bug wrote one 'blocked' view-data row per table even when the db-level row was already 'blocked',
+## ballooning data_permissions into millions of redundant rows. Two steps:
+## 1. For each (group, db) with no db-level view-data row, whose table-level view-data rows are all
+## 'blocked' and cover every active table: insert the equivalent db-level row.
+## 2. Drop every table-level view-data row made redundant by an equal-value db-level row.
+## Only view-data/blocked is safe to collapse; all other combos are left untouched.
databaseChangeLog:
- logicalFilePath: migrations/062_update_migrations.yaml
+ # Step 1 originally shipped as v62.2026-07-07T00:00:00, which timed out on large instances (its plan
+ # was O(rows x databases) on postgres). The changeSets below replace it under new ids; this DELETE just
+ # removes the retired changeSet's stale changelog entry from instances that already ran it. For them the
+ # replacements re-collapse already-collapsed data, which is a cheap no-op.
- changeSet:
- id: v62.2026-07-07T00:00:00
+ id: v62.2026-07-14T00:00:00
author: ranquild
+ comment: Retire the timeout-prone original of the collapse changeSets below (#76077)
+ changes:
+ - sql:
+ sql: DELETE FROM DATABASECHANGELOG WHERE id = 'v62.2026-07-07T00:00:00';
+ rollback: # not necessary
+
+ # Step 1. Aggregate to (group_id, db_id) first, then join. Coverage is counted in rows, not DISTINCT
+ # tables — equivalent unless duplicate rows exist, which the app never writes. Postgres runs the
+ # staged, parallel version in the changeSets below instead.
+ - changeSet:
+ id: v62.2026-07-14T00:00:01
+ author: ranquild
+ dbms: mysql,mariadb,h2
comment: Insert db-level data_permissions rows for uniform full-coverage blocked table-level view-data permissions (#76077)
changes:
- sql:
sql: >-
INSERT INTO data_permissions (group_id, perm_type, db_id, perm_value)
- SELECT dp.group_id, dp.perm_type, dp.db_id, MIN(dp.perm_value)
- FROM data_permissions dp
- LEFT JOIN metabase_table mt ON mt.id = dp.table_id
+ SELECT agg.group_id, 'perms/view-data', agg.db_id, 'blocked'
+ FROM (
+ SELECT dp.group_id, dp.db_id,
+ MIN(dp.perm_value) AS min_value,
+ MAX(dp.perm_value) AS max_value,
+ COUNT(*) AS covered_rows,
+ COUNT(it.id) AS inactive_rows
+ FROM data_permissions dp
+ LEFT JOIN metabase_table it ON it.id = dp.table_id AND it.active = FALSE
+ WHERE dp.perm_type = 'perms/view-data'
+ AND dp.table_id IS NOT NULL
+ GROUP BY dp.group_id, dp.db_id
+ ) agg
JOIN (SELECT db_id, COUNT(*) AS n FROM metabase_table WHERE active = TRUE GROUP BY db_id) tc
- ON tc.db_id = dp.db_id
- WHERE dp.perm_type = 'perms/view-data'
- AND dp.table_id IS NOT NULL
+ ON tc.db_id = agg.db_id
+ WHERE agg.min_value = 'blocked'
+ AND agg.max_value = 'blocked'
+ AND agg.covered_rows - agg.inactive_rows = tc.n
AND NOT EXISTS (
SELECT 1 FROM data_permissions dbp
- WHERE dbp.group_id = dp.group_id
- AND dbp.db_id = dp.db_id
- AND dbp.perm_type = dp.perm_type
- AND dbp.table_id IS NULL)
- GROUP BY dp.group_id, dp.db_id, dp.perm_type, tc.n
- HAVING COUNT(DISTINCT dp.perm_value) = 1
- AND MIN(dp.perm_value) = 'blocked'
- AND COUNT(DISTINCT CASE WHEN mt.active = TRUE THEN dp.table_id END) = tc.n;
+ WHERE dbp.group_id = agg.group_id
+ AND dbp.db_id = agg.db_id
+ AND dbp.perm_type = 'perms/view-data'
+ AND dbp.table_id IS NULL);
rollback: # not necessary; the collapsed representation is understood by all versions >= 49
- # The rows to keep are every data_permissions row EXCEPT the table-level view-data rows that a db-level
- # row of the same (group, db, value) has made redundant — the anti-join below. Deleting the redundant rows
- # directly scales super-linearly on a table ballooned to tens of millions of rows (each delete maintains
- # six index trees), so instead of a DELETE we stage the few thousand survivors, empty the table, and
- # reload them. Migrations run before the instance serves traffic, so the brief window in which
- # data_permissions is empty is never observed; the survivors persist in the staging table until the reload
- # finishes (each step is its own changeSet, so a failure resumes from the incomplete step with the staging
- # table still present), and because the table is only truncated — never dropped — its indexes and foreign
- # keys stay in place. The steps are split into one changeSet each because the migration linter allows only
- # a single change per changeSet.
+ # Step 1 for postgres. CREATE TABLE AS gets a parallel plan (INSERT ... SELECT and DISTINCT
+ # aggregates do not), so the one heavy pass over data_permissions runs on all available workers.
+ - changeSet:
+ id: v62.2026-07-14T00:00:02
+ author: ranquild
+ dbms: postgresql
+ comment: Stage per group-and-database aggregates of table-level view-data permissions (#76077)
+ changes:
+ - sql:
+ sql: >-
+ SET LOCAL max_parallel_workers_per_gather = 8;
+ CREATE TABLE data_permissions_collapse_agg AS
+ SELECT dp.group_id, dp.db_id,
+ COUNT(*) AS covered_rows,
+ COUNT(it.id) AS inactive_rows,
+ MIN(dp.perm_value) AS min_value,
+ MAX(dp.perm_value) AS max_value
+ FROM data_permissions dp
+ LEFT JOIN metabase_table it ON it.id = dp.table_id AND it.active = FALSE
+ WHERE dp.perm_type = 'perms/view-data'
+ AND dp.table_id IS NOT NULL
+ GROUP BY dp.group_id, dp.db_id;
+ rollback: # not necessary; the staging table is dropped at the end of this migration
+
+ - changeSet:
+ id: v62.2026-07-14T00:00:03
+ author: ranquild
+ dbms: postgresql
+ comment: Analyze the staged collapse aggregates (#76077)
+ changes:
+ - sql:
+ sql: ANALYZE data_permissions_collapse_agg;
+ rollback: # not necessary
+
+ # NOT IN evaluates as one hashed subplan instead of an index probe per staged row.
+ - changeSet:
+ id: v62.2026-07-14T00:00:04
+ author: ranquild
+ dbms: postgresql
+ comment: Insert db-level data_permissions rows for uniform full-coverage blocked table-level view-data permissions (#76077)
+ changes:
+ - sql:
+ sql: >-
+ INSERT INTO data_permissions (group_id, perm_type, db_id, perm_value)
+ SELECT agg.group_id, 'perms/view-data', agg.db_id, 'blocked'
+ FROM data_permissions_collapse_agg agg
+ JOIN (SELECT db_id, COUNT(*) AS n FROM metabase_table WHERE active = TRUE GROUP BY db_id) tc
+ ON tc.db_id = agg.db_id
+ WHERE agg.min_value = 'blocked'
+ AND agg.max_value = 'blocked'
+ AND agg.covered_rows - agg.inactive_rows = tc.n
+ AND (agg.group_id, agg.db_id) NOT IN (
+ SELECT dbp.group_id, dbp.db_id FROM data_permissions dbp
+ WHERE dbp.perm_type = 'perms/view-data'
+ AND dbp.table_id IS NULL);
+ rollback: # not necessary; the collapsed representation is understood by all versions >= 49
+
+ - changeSet:
+ id: v62.2026-07-14T00:00:05
+ author: ranquild
+ dbms: postgresql
+ comment: Drop the staged collapse aggregates (#76077)
+ changes:
+ - sql:
+ sql: DROP TABLE data_permissions_collapse_agg;
+ rollback: # not necessary
+
+ # Step 2. A row-by-row DELETE of millions of rows is too slow, so: stage the surviving rows,
+ # truncate data_permissions, reload. One changeSet per step, so a failed run resumes where it stopped.
- changeSet:
id: v62.2026-07-07T00:00:01
author: ranquild
comment: Create the staging table that will hold the surviving data_permissions rows (#76077)
changes:
- # A createTable change (rather than a raw CREATE TABLE ... AS SELECT) so Liquibase renders it per
- # engine and, on MySQL/MariaDB, appends the ENGINE/CHARACTER SET/COLLATE clause the app-db expects.
+ # createTable (not raw SQL) so Liquibase adds MySQL/MariaDB's ENGINE/charset clauses.
- createTable:
tableName: data_permissions_collapse_keep
remarks: Transient staging table holding the data_permissions rows to keep while the table is collapsed; dropped at the end of this migration
diff --git a/test/metabase/app_db/schema_migrations_test.clj b/test/metabase/app_db/schema_migrations_test.clj
index 3f0d427c763..0ab9a2eab3b 100644
--- a/test/metabase/app_db/schema_migrations_test.clj
+++ b/test/metabase/app_db/schema_migrations_test.clj
@@ -2889,9 +2889,9 @@
attrs)))
(deftest collapse-uniform-table-permissions-migration-test
- (testing "Migrations v62.2026-07-07T00:00:00 through v62.2026-07-07T00:00:05:
+ (testing "Migrations v62.2026-07-14T00:00:00 through v62.2026-07-07T00:00:05:
uniform full-coverage :blocked view-data table rows collapse to a db-level row (#76077)"
- (impl/test-migrations ["v62.2026-07-07T00:00:00" "v62.2026-07-07T00:00:05"] [migrate!]
+ (impl/test-migrations ["v62.2026-07-14T00:00:00" "v62.2026-07-07T00:00:05"] [migrate!]
(let [group-id (t2/insert-returning-pk! :permissions_group {:name "collapse-test-group"})
db! (fn [db-name]
(t2/insert-returning-pk! :metabase_database
@@ -2947,7 +2947,19 @@
t6-off (table! db-6 false)
;; db-7: table row duplicating an equal-value db-level row → duplicate deleted
db-7 (db! "collapse-dup-of-db-level")
- t7a (table! db-7 true)]
+ t7a (table! db-7 true)
+ ;; db-8: duplicate row alongside full coverage → conservatively NOT collapsed: coverage is
+ ;; row-counted, so the duplicate makes the count exceed the active-table count (the app never
+ ;; writes duplicate rows; leaving them uncollapsed preserves the current state)
+ db-8 (db! "collapse-dup-full-coverage")
+ t8a (table! db-8 true)
+ t8b (table! db-8 true)
+ ;; db-9: duplicate row exactly offsetting a MISSING table row → collapsed; lossless because a
+ ;; table with no row in a group with no db-level row already reads as the least permissive
+ ;; value, blocked, which is what the inserted db-level row says
+ db-9 (db! "collapse-dup-masks-missing")
+ t9a (table! db-9 true)
+ _t9b (table! db-9 true)]
(perm! db-1 t1a "perms/view-data" "blocked")
(perm! db-1 t1b "perms/view-data" "blocked")
(perm! db-1 t1-off "perms/view-data" "blocked")
@@ -2960,6 +2972,11 @@
(perm! db-6 t6-off "perms/view-data" "unrestricted")
(perm! db-7 nil "perms/view-data" "unrestricted")
(perm! db-7 t7a "perms/view-data" "unrestricted")
+ (perm! db-8 t8a "perms/view-data" "blocked")
+ (perm! db-8 t8a "perms/view-data" "blocked")
+ (perm! db-8 t8b "perms/view-data" "blocked")
+ (perm! db-9 t9a "perms/view-data" "blocked")
+ (perm! db-9 t9a "perms/view-data" "blocked")
(migrate!)
(testing "uniform :blocked collapses to a single db-level row, dropping the inactive table's row too"
(is (= #{[nil "blocked"]} (rows db-1 "perms/view-data"))))
@@ -2974,7 +2991,11 @@
(testing "a differing value on an inactive table's row blocks the collapse"
(is (= #{[t6a "blocked"] [t6-off "unrestricted"]} (rows db-6 "perms/view-data"))))
(testing "table rows duplicating an equal-value db-level row are deleted"
- (is (= #{[nil "unrestricted"]} (rows db-7 "perms/view-data"))))))))
+ (is (= #{[nil "unrestricted"]} (rows db-7 "perms/view-data"))))
+ (testing "a duplicate row alongside full coverage conservatively blocks the collapse"
+ (is (= #{[t8a "blocked"] [t8b "blocked"]} (rows db-8 "perms/view-data"))))
+ (testing "a duplicate row offsetting a missing table row collapses losslessly"
+ (is (= #{[nil "blocked"]} (rows db-9 "perms/view-data"))))))))
(defn- collection-entity-id
[collection-id]