The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Enable native pivot support for hive-like drivers (#80333)
metamben6ca347e9e2dc
diff --git a/docs/developers-guide/driver-changelog.md b/docs/developers-guide/driver-changelog.mdindex c88b5edc4a4..e3b2a6e621e 100644--- a/docs/developers-guide/driver-changelog.md+++ b/docs/developers-guide/driver-changelog.md@@ -30,6 +30,9 @@ title: Driver interface changelogform for the pivot-grouping bitmask. The default emits `GROUPING(exprs...)` (the Postgres/Oracle/Snowflakemulti-arg extension); drivers whose SQL dialect uses a different function or shape override this method.+- `:native-pivot-tables` is now enabled for `:hive-like` drivers.+ Hive-family dialects synthesise the pivot-grouping bitmask from single-arg `GROUPING(x)` calls.+- Index Manager: drivers can now read and create table indexes, in the broad sense (secondary indexes, sort keys,distribution keys, clustering, etc.). New driver feature flags:diff --git a/modules/drivers/hive-like/src/metabase/driver/hive_like.clj b/modules/drivers/hive-like/src/metabase/driver/hive_like.cljindex 8a28ba6051e..4ede71146eb 100644--- a/modules/drivers/hive-like/src/metabase/driver/hive_like.clj+++ b/modules/drivers/hive-like/src/metabase/driver/hive_like.clj@@ -9,6 +9,7 @@[metabase.driver.sql-jdbc.execute :as sql-jdbc.execute][metabase.driver.sql-jdbc.execute.legacy-impl :as sql-jdbc.legacy][metabase.driver.sql-jdbc.sync :as sql-jdbc.sync]+ [metabase.driver.sql.pivot :as sql.pivot][metabase.driver.sql.query-processor :as sql.qp][metabase.driver.sql.util :as sql.u][metabase.util :as u]@@ -24,10 +25,39 @@:parent #{:sql-jdbc ::sql-jdbc.legacy/use-legacy-classes-for-read-and-set}:abstract? true)-(doseq [[feature supported?] {:now true- :datetime-diff true}]+(doseq [[feature supported?] {:now true+ :datetime-diff true+ :native-pivot-tables true}](defmethod driver/database-supports? [:hive-like feature] [_driver _feature _db] supported?))+;; Hive-family dialects support `GROUPING SETS` and single-arg `GROUPING(x)`, but their `GROUPING_ID(a, b, ...)`+;; requires the arg order to exactly match the `GROUP BY` column order — and `[[metabase.driver.sql.pivot]]`+;; reverses the args so `bit 0 = first breakout` (the Postgres/Oracle/SQL Server convention). To keep+;; that convention without tripping Spark's `GROUPING_ID_COLUMN_MISMATCH` check, synthesize the bitmask from+;; single-arg `GROUPING(x)` calls instead.+(defmethod sql.pivot/pivot-grouping-hsql :hive-like+ [_driver exprs]+ (sql.pivot/synthesise-grouping-bitmask exprs))++;; Spark's analyzer rejects join-qualified column refs in ORDER BY after `GROUP BY GROUPING SETS`, even+;; when the same refs are valid in the SELECT list — Postgres/Oracle/SQL Server tolerate this via lenient+;; re-resolution through the SELECT, Spark doesn't. Delegate to the `:sql` implementation and then rewrite+;; the ORDER BY as 1-based positional references, which Spark accepts and which is standard SQL.+(defmethod sql.qp/apply-top-level-clause [:hive-like :pivot]+ [driver clause honeysql-form {:keys [breakout] :as pivot-ctx}]+ (let [result ((get-method sql.qp/apply-top-level-clause [:sql :pivot])+ driver clause honeysql-form pivot-ctx)+ ;; SELECT layout after the parent runs: [breakout-1..n, pivot-grouping, aggs...].+ pivot-grouping-pos (inc (count breakout))+ breakout-positions (range 1 pivot-grouping-pos)+ ;; The parent installs `GROUP BY [[::sql.pivot/grouping-sets set1 set2 ...]]`; more than one set+ ;; means the pivot-grouping bitmask varies per row and needs to lead the ORDER BY.+ multi-set? (some? (nnext (first (:group-by result))))]+ (assoc result :order-by+ (into (if multi-set? [[[:inline pivot-grouping-pos] :asc]] [])+ (map (fn [p] [[:inline p] :asc]))+ breakout-positions))))+(defmethod driver/escape-alias :hive-like[driver s];; replace question marks inside aliases with `_QMARK_`, otherwise Spark SQL will interpret them as JDBC parameter