The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Migrate usages of `driver.common.parameters.dates` and `.ops` to QP (#74253)
Cam Saul114acd798e3c
diff --git a/.clj-kondo/config.edn b/.clj-kondo/config.ednindex ecad9d49b72..6f67888c4b3 100644--- a/.clj-kondo/config.edn+++ b/.clj-kondo/config.edn@@ -813,9 +813,7 @@clojure.core/eval {:message "Don't use eval outside of tests"}}:discouraged-namespace- {metabase.legacy-mbql.jvm-util {:message "Avoid legacy MBQL utils, use Lib instead"}- metabase.legacy-mbql.normalize {:message "Avoid legacy MBQL normalization, use MBQL 5 + lib/normalize instead"}- metabase.legacy-mbql.predicates {:message "Avoid legacy MBQL utils, use Lib instead"}+ {metabase.legacy-mbql.normalize {:message "Avoid legacy MBQL normalization, use MBQL 5 + lib/normalize instead"}metabase.legacy-mbql.schema {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}metabase.legacy-mbql.schema.helpers {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}metabase.legacy-mbql.schema.macros {:message "Avoid the legacy MBQL schema, use MBQL 5 + lib.schema instead"}@@ -867,9 +865,7 @@legacy-mbql-namespaces{:linters{:discouraged-namespace- {metabase.legacy-mbql.jvm-util {:level :off}- metabase.legacy-mbql.normalize {:level :off}- metabase.legacy-mbql.predicates {:level :off}+ {metabase.legacy-mbql.normalize {:level :off}metabase.legacy-mbql.schema {:level :off}metabase.legacy-mbql.schema.helpers {:level :off}metabase.legacy-mbql.schema.macros {:level :off}@@ -889,9 +885,7 @@;; don't warn about the legacy MBQL namespaces being deprecated or discouraged in driver code now, we don't have a;; real alternative YET so it's not a helpful warning.:discouraged-namespace- {metabase.legacy-mbql.jvm-util {:level :off}- metabase.legacy-mbql.normalize {:level :off}- metabase.legacy-mbql.predicates {:level :off}+ {metabase.legacy-mbql.normalize {:level :off}metabase.legacy-mbql.schema {:level :off}metabase.legacy-mbql.schema.helpers {:level :off}metabase.legacy-mbql.schema.macros {:level :off}@@ -903,9 +897,7 @@[metabase.driver.common.parameters.datesmetabase.driver.common.parameters.operatorsmetabase.driver.common.parameters.values- metabase.legacy-mbql.jvm-utilmetabase.legacy-mbql.normalize- metabase.legacy-mbql.predicatesmetabase.legacy-mbql.schemametabase.legacy-mbql.schema.helpersmetabase.legacy-mbql.schema.macrosdiff --git a/docs/developers-guide/driver-changelog.md b/docs/developers-guide/driver-changelog.mdindex 3e41713a805..f6617444881 100644--- a/docs/developers-guide/driver-changelog.md+++ b/docs/developers-guide/driver-changelog.md@@ -4,6 +4,31 @@ title: Driver interface changelog# Driver Interface Changelog+## Metabase 0.62.0++- `sql.params.substitution/field->clause`, `to-clause`, `desugar-filter-clause`, `wrap-value-literals-in-mbql`, and+ `date-string->filter`, introduced in 0.61.0, have been removed; they are no longer necessary. They have been+ replaced by a single method, `sql.params.substitution/->honeysql`, which compiles an MBQL 5 clause to HoneySQL with+ the given options.++- The `metabase.driver.commmon.parameters` and `metabase.driver.commmon.parameters.parse` namespaces, deprecated in+ 0.57.0, have been removed. Please use the Lib implementations instead. Relevant functions are aliased in+ `metabase.lib.core`, for example `metabase.lib.core/parse-parameters`, `metabase.lib.core/parsed-parameter`, and+ `metabase.lib.core/parsed-parameter?`.++- The `metabase.driver.common.parameters.dates` and `metabase.driver.common.parameters.operators` namespaces,+ deprecated in 0.57.0, have been removed. Use the equivalent QP namespaces instead:+ `metabase.query-processor.parameters.dates` and `metabase.query-processor.parameters.operators`, respectively. These+ namespaces return MBQL 5 clauses rather than MBQL 4; use `metabase.lib.core/->legacy-MBQL` if needed until your+ driver has been fully updated to MBQL 5.++- `metabase.driver.sql.parameters.substitution/align-temporal-unit-with-param-type`, deprecated in 0.49.0, has been+ removed.++- `metabase.driver-api.core/desugar-filter-clause`, `metabase.driver-api.core/negate-filter-clause`, and+ `metabase.driver-api.core/simplify-compound-filter`, deprecated in 0.57.0, have been removed; use the+ `metabase.lib.core` versions instead. The new versions operate on MBQL 5 instead of MBQL 4.+## Metabase 0.61.0- Added the following driver multimethods to support MBQL5 compilation migration:@@ -35,14 +60,6 @@ title: Driver interface changelog`metabase.driver.sql-jdbc/set-role-statement`, which takes an additional `java.sql.Connection` parameter, so you usethe connection to call `quote_ident()` or similar for identifier quoting/escaping purposes.-- The `metabase.driver.commmon.parameters` and `metabase.driver.commmon.parameters.parse` namespaces, deprecated in- 0.57.0, have been removed. Please use the Lib implementations instead. Relevant functions are aliased in- `metabase.lib.core`, for example `metabase.lib.core/parse-parameters`, `metabase.lib.core/parsed-parameter`, and- `metabase.lib.core/parsed-parameter?`.--- `metabase.driver.sql.parameters.substitution/align-temporal-unit-with-param-type`, deprecated in 0.49.0, has been- removed.-## Metabase 0.60.0- Added `validate-impersonated-query` multimethod. This is used for drivers to perform validation on impersonated native queries.diff --git a/modules/drivers/druid/src/metabase/driver/druid/query_processor.clj b/modules/drivers/druid/src/metabase/driver/druid/query_processor.cljindex 27e80a6918b..b5b0d111edc 100644--- a/modules/drivers/druid/src/metabase/driver/druid/query_processor.clj+++ b/modules/drivers/druid/src/metabase/driver/druid/query_processor.clj@@ -5,12 +5,13 @@[metabase.driver-api.core :as driver-api][metabase.driver.common :as driver.common][metabase.driver.druid.js :as druid.js]+ [metabase.lib.core :as lib][metabase.util :as u][metabase.util.date-2 :as u.date][metabase.util.i18n :refer [tru]][metabase.util.log :as log][metabase.util.malli :as mu]- [metabase.util.performance :refer [every? mapv some get-in]]))+ [metabase.util.performance :refer [every? get-in mapv some]]))(set! *warn-on-reflection* true)@@ -308,9 +309,9 @@(-> (driver-api/replace filter-clause[_ [:field _ {:temporal-unit &truthy}] & _]nil)- ;; TODO (Cam 8/18/25) -- I am 90% sure this is serving no useful purpose.- #_{:clj-kondo/ignore [:deprecated-var]}- driver-api/simplify-compound-filter+ lib/->mbql5+ lib/simplify-compound-filter+ lib/->legacy-MBQLparse-filter*))(mu/defn- add-datetime-units* :- driver-api/mbql.schema.DateTimeValue@@ -431,7 +432,11 @@;; without checking first because some filters like string `:contains` can't be negated without using a `:not`;; filter and we don't want to stack overflow(when (seq (filter-clause->intervals subclause))- (filter-clause->intervals (driver-api/negate-filter-clause subclause))))+ (-> subclause+ lib/->mbql5+ lib/negate-boolean-expression+ lib/->legacy-MBQL+ filter-clause->intervals)))(defn- compile-intervals"Compile the interval pairs generated by `filter-clause->intervals` into the format expected by Druid (`min/max`diff --git a/modules/drivers/druid/test/metabase/driver/druid/query_processor_test.clj b/modules/drivers/druid/test/metabase/driver/druid/query_processor_test.cljindex 107aee1cdb3..489c3433a69 100644--- a/modules/drivers/druid/test/metabase/driver/druid/query_processor_test.clj+++ b/modules/drivers/druid/test/metabase/driver/druid/query_processor_test.clj@@ -10,6 +10,8 @@[metabase.driver :as driver][metabase.driver.common.table-rows-sample :as table-rows-sample][metabase.driver.druid.query-processor :as druid.qp]+ [metabase.lib.test-metadata :as meta]+ [metabase.lib.test-util.macros :as lib.tu.macros][metabase.query-processor.compile :as qp.compile][metabase.query-processor.test :as qp][metabase.query-processor.timeseries-test.util :as tqpt]@@ -87,7 +89,14 @@[:or[:= [:field 1 nil] "toucan"][:= [:field 2 nil] "threecan"]]))- ":or filters with no temporal filters should return nil"))))+ ":or filters with no temporal filters should return nil"))+ (testing :not+ (is (= ["2015-10-04T00:00:00Z/2015-10-11T00:00:00Z"]+ (filter-clause->intervals+ [:not+ [:or+ [:< dt-field (str->absolute-dt "2015-10-04T00:00:00Z")]+ [:>= dt-field (str->absolute-dt "2015-10-11T00:00:00Z")]]]))))))(defn- do-query->native [query](driver/with-driver :druid@@ -636,24 +645,21 @@(mt/first-row (qp/process-query query))))))))))))(deftest ^:parallel parse-filter-test- (mt/test-driver :druid- (testing "parse-filter should generate the correct filter clauses"- (tqpt/with-flattened-dbdef- (mt/with-metadata-provider (mt/id)- (tools.macro/macrolet [(parse-filter [filter-clause]- `(#'druid.qp/parse-filter (mt/$ids ~'checkins ~filter-clause)))]- (testing "normal non-compound filters should work as expected"- (is (= {:type :selector, :dimension "venue_price", :value 2}- (parse-filter [:= $venue_price [:value 2 {:base_type :type/Integer}]]))))- (testing "temporal filters should get stripped out"- (is (= nil- (parse-filter [:>= !default.timestamp [:absolute-datetime #t "2015-09-01T00:00Z[UTC]" :default]])))- (is (= {:type :selector, :dimension "venue_category_name", :value "Mexican"}- (parse-filter- [:and- [:= $venue_category_name [:value "Mexican" {:base_type :type/Text}]]-- [:< !default.timestamp [:absolute-datetime #t "2015-10-01T00:00Z[UTC]" :default]]]))))))))))+ (testing "parse-filter should generate the correct filter clauses"+ (mt/with-metadata-provider meta/metadata-provider+ (tools.macro/macrolet [(parse-filter [filter-clause]+ `(#'druid.qp/parse-filter (lib.tu.macros/$ids :checkins ~filter-clause)))]+ (testing "normal non-compound filters should work as expected"+ (is (= {:type :selector, :dimension "PRICE", :value 2}+ (parse-filter [:= $venues.price [:value 2 {:base_type :type/Integer}]]))))+ (testing "temporal filters should get stripped out"+ (is (= nil+ (parse-filter [:>= !checkins.date [:absolute-datetime #t "2015-09-01T00:00Z[UTC]" :default]])))+ (is (= {:type :selector, :dimension "NAME", :value "Mexican"}+ (parse-filter+ [:and+ [:= $categories.name [:value "Mexican" {:base_type :type/Text}]]+ [:< !checkins.date [:absolute-datetime #t "2015-10-01T00:00Z[UTC]" :default]]]))))))))(deftest ^:parallel multiple-filters-test(mt/test-driver :druiddiff --git a/modules/drivers/mongo/src/metabase/driver/mongo/parameters.clj b/modules/drivers/mongo/src/metabase/driver/mongo/parameters.cljindex ac42aa54b84..d07969cff92 100644--- a/modules/drivers/mongo/src/metabase/driver/mongo/parameters.clj+++ b/modules/drivers/mongo/src/metabase/driver/mongo/parameters.clj@@ -4,11 +4,11 @@[clojure.string :as str][java-time.api :as t][metabase.driver-api.core :as driver-api]- [metabase.driver.common.parameters.dates :as params.dates]- [metabase.driver.common.parameters.operators :as params.ops][metabase.driver.common.parameters.values :as params.values][metabase.driver.mongo.query-processor :as mongo.qp][metabase.lib.core :as lib]+ [metabase.query-processor.parameters.dates :as params.dates]+ [metabase.query-processor.parameters.operators :as params.ops][metabase.util :as u][metabase.util.date-2 :as u.date][metabase.util.i18n :refer [tru]]@@ -129,7 +129,6 @@(let [no-value? (= (:value v) lib/parsed-param-no-value-placeholder)](cond(params.ops/operator? (get-in v [:value :type]))- #_{:clj-kondo/ignore [:deprecated-var]}(let [param (:value v)field-name (if (str/blank? (:alias v))(mongo.qp/field->name (:field v) ".")@@ -140,11 +139,24 @@[:field field-name{:base-type (get-in v [:field :base-type])}]])params.ops/to-clause- ;; desugar only impacts :does-not-contain -> [:not [:contains ... but it prevents- ;; an optimization of [:= 'field 1 2 3] -> [:in 'field [1 2 3]] since that- ;; desugars to [:or [:= 'field 1] ...].- driver-api/desugar-filter-clause- driver-api/wrap-value-literals-in-mbql+ ;; desugar only impacts+ ;;+ ;; :does-not-contain -> [:not [:contains ...]]+ ;;+ ;; but it prevents an optimization of+ ;;+ ;; [:= <field> 1 2 3] -> [:in <field> [1 2 3]]+ ;;+ ;; since that desugars to+ ;;+ ;; [:or [:= <field> 1] ...].+ lib/desugar-filter-clause+ driver-api/wrap-value-literals-in-mbql5+ ;;+ ;; TODO (Cam 2026-05-14) -- remove once we convert the MongoDB driver to use MBQL+ ;; 5 natively+ ;;+ lib/->legacy-MBQLmongo.qp/compile-filterjson/encode)][(conj acc compiled-clause) missing])diff --git a/modules/drivers/mongo/src/metabase/driver/mongo/query_processor.clj b/modules/drivers/mongo/src/metabase/driver/mongo/query_processor.cljindex 684a9795a44..2778e47bd18 100644--- a/modules/drivers/mongo/src/metabase/driver/mongo/query_processor.clj+++ b/modules/drivers/mongo/src/metabase/driver/mongo/query_processor.clj@@ -23,6 +23,7 @@$strcasecmp $subtract $sum$toBool $toLower $unwind $year]][metabase.driver.util :as driver.u]+ [metabase.lib.core :as lib][metabase.util :as u][metabase.util.date-2 :as u.date][metabase.util.i18n :refer [tru]]@@ -950,7 +951,10 @@ function(bin) {driver-api/dispatch-by-clause-name-or-class)(defmethod negate :default [clause]- (driver-api/negate-filter-clause clause))+ (-> clause+ lib/->mbql5+ lib/negate-boolean-expression+ lib/->legacy-MBQL))(defmethod negate :and [[_ & subclauses]] (apply vector :or (map negate subclauses)))(defmethod negate :or [[_ & subclauses]] (apply vector :and (map negate subclauses)))diff --git a/modules/drivers/mongo/test/metabase/driver/mongo/parameters_test.clj b/modules/drivers/mongo/test/metabase/driver/mongo/parameters_test.cljindex de40e9bf5d6..7962891ff30 100644--- a/modules/drivers/mongo/test/metabase/driver/mongo/parameters_test.clj+++ b/modules/drivers/mongo/test/metabase/driver/mongo/parameters_test.clj@@ -176,12 +176,13 @@(#'mongo.params/substitute {"id" (field-filter "id" :number [1 2 3])}["[{$match: " (lib/parsed-param "id") "}]"])))))-(deftest ^:parallel field-filter-test-3- (testing "single date"- (is (= (to-bson [{:$match {:$and [{"date" {:$gte (ISODate "2019-12-08")}}- {"date" {:$lt (ISODate "2019-12-09")}}]}}])- (#'mongo.params/substitute {"date" (field-filter "date" :date/single "2019-12-08")}- ["[{$match: " (lib/parsed-param "date") "}]"])))))+(deftest field-filter-test-3+ (mt/with-temporary-setting-values [report-timezone "UTC"]+ (testing "single date"+ (is (= (to-bson [{:$match {:$and [{"date" {:$gte (ISODate "2019-12-08")}}+ {"date" {:$lt (ISODate "2019-12-09")}}]}}])+ (#'mongo.params/substitute {"date" (field-filter "date" :date/single "2019-12-08")}+ ["[{$match: " (lib/parsed-param "date") "}]"]))))))(deftest ^:parallel field-filter-test-4(testing "parameter not supplied"@@ -282,28 +283,29 @@:content (to-bson {"price" {:$gt 2}})}}["[{$match: " (lib/parsed-param "snippet: high price") "}]"]))))))-(deftest ^:parallel e2e-field-filter-test+(deftest e2e-field-filter-test(mt/test-driver :mongo- (testing "date ranges"- (is (= [[295 "2014-03-01T00:00:00Z" 7 97]- [642 "2014-03-02T00:00:00Z" 8 9]- [775 "2014-03-01T00:00:00Z" 4 13]]- (mt/rows- (qp/process-query- (mt/query checkins- {:type :native- :native {:query (json/encode- [{:$match (json/raw-json-generator "{{date}}")}- {:$sort {:_id 1}}])- :collection "checkins"- :template-tags {"date" {:name "date"- :display-name "Date"- :type :dimension- :widget-type :date/all-options- :dimension $date}}}- :parameters [{:type :date/range- :target [:dimension [:template-tag "date"]]- :value "2014-03-01~2014-03-02"}]}))))))))+ (mt/with-temporary-setting-values [report-timezone "UTC"]+ (testing "date ranges"+ (is (= [[295 "2014-03-01T00:00:00Z" 7 97]+ [642 "2014-03-02T00:00:00Z" 8 9]+ [775 "2014-03-01T00:00:00Z" 4 13]]+ (mt/rows+ (qp/process-query+ (mt/query checkins+ {:type :native+ :native {:query (json/encode+ [{:$match (json/raw-json-generator "{{date}}")}+ {:$sort {:_id 1}}])+ :collection "checkins"+ :template-tags {"date" {:name "date"+ :display-name "Date"+ :type :dimension+ :widget-type :date/all-options+ :dimension $date}}}+ :parameters [{:type :date/range+ :target [:dimension [:template-tag "date"]]+ :value "2014-03-01~2014-03-02"}]})))))))))(deftest ^:parallel e2e-field-filter-test-2(mt/test-driver :mongo@@ -327,24 +329,25 @@:target [:dimension [:template-tag "id"]]:value "1,2,3"}]}))))))))-(deftest ^:parallel e2e-field-filter-test-3+(deftest e2e-field-filter-test-3(mt/test-driver :mongo- (testing "param not supplied"- (is (= [[1 "2014-04-07T00:00:00Z" 5 12]]- (mt/rows- (qp/process-query- (mt/query checkins- {:type :native- :native {:query (json/encode- [{:$match (json/raw-json-generator "{{date}}")}- {:$sort {:_id 1}}- {:$limit 1}])- :collection "checkins"- :template-tags {"date" {:name "date"- :display-name "Date"- :type :dimension- :widget-type :date/all-options- :dimension $date}}}}))))))))+ (mt/with-temporary-setting-values [report-timezone "UTC"]+ (testing "param not supplied"+ (is (= [[1 "2014-04-07T00:00:00Z" 5 12]]+ (mt/rows+ (qp/process-query+ (mt/query checkins+ {:type :native+ :native {:query (json/encode+ [{:$match (json/raw-json-generator "{{date}}")}+ {:$sort {:_id 1}}+ {:$limit 1}])+ :collection "checkins"+ :template-tags {"date" {:name "date"+ :display-name "Date"+ :type :dimension+ :widget-type :date/all-options+ :dimension $date}}}})))))))))(deftest ^:parallel e2e-field-filter-test-4(mt/test-driver :mongodiff --git a/src/metabase/driver/common/parameters/dates.clj b/src/metabase/driver/common/parameters/dates.cljdeleted file mode 100644index 5ac77266363..00000000000--- a/src/metabase/driver/common/parameters/dates.clj+++ /dev/null@@ -1,228 +0,0 @@-(ns metabase.driver.common.parameters.dates- "Shared code for handling datetime parameters, used by both MBQL and native params implementations.-- DEPRECATED: This namespace will be removed in the near future, in favor- of [[metabase.query-processor.parameters.dates]] (which uses Lib/MBQL 5).-- TODO (Cam 9/30/25) -- a ton of stuff in this namespace is an exact duplicate of the version- in [[metabase.query-processor.parameters.dates]], we should remove the version here to encourage migration to that- namespace."- {:deprecated "0.57.0"}- (:refer-clojure :exclude [every? get-in])- (:require- [clojure.string :as str]- [java-time.api :as t]- [metabase.legacy-mbql.schema :as mbql.s]- [metabase.legacy-mbql.util :as mbql.u]- [metabase.lib.schema.id :as lib.schema.id]- [metabase.query-processor.error-type :as qp.error-type]- [metabase.query-processor.parameters.dates :as qp.parameters.dates]- [metabase.util.i18n :refer [tru]]- [metabase.util.malli :as mu]- [metabase.util.performance :refer [every? get-in]]- [potemkin :as p]))--(set! *warn-on-reflection* true)--;; Both in MBQL and SQL parameter substitution a field value is compared to a date range, either relative or absolute.-;; Currently the field value is casted to a day (ignoring the time of day), so the ranges should have the same-;; granularity level.-;;-;; See https://github.com/metabase/metabase/pull/4607#issuecomment-290884313 how we could support-;; hour/minute granularity in field parameter queries.--;; For parsing date strings and producing either a date range (for raw SQL parameter substitution) or a MBQL clause--;; Decoders consist of:-;; 1) Parser which tries to parse the date parameter string-;; 2) Range decoder which takes the parser output and produces a date range relative to the given datetime-;; 3) Filter decoder which takes the parser output and produces a mbql clause for a given mbql field reference--(defn- with-temporal-unit-if-field- [clause unit]- ;; legacy usages -- use Lib in new code going forward- #_{:clj-kondo/ignore [:deprecated-var]}- (cond-> clause- (mbql.u/is-clause? :field clause) (mbql.u/with-temporal-unit unit)))--(def ^:private relative-date-string-decoders- [{:parser #(= % "today")- :range (fn [_ dt]- (let [dt-res (t/local-date dt)]- {:start dt-res- :end dt-res- :unit :day}))- :filter (fn [_ field-clause]- [:= (with-temporal-unit-if-field field-clause :day) [:relative-datetime :current]])}-- {:parser #(= % "yesterday")- :range (fn [_ dt]- (let [dt-res (t/local-date dt)]- {:start (t/minus dt-res (t/days 1))- :end (t/minus dt-res (t/days 1))- :unit :day}))- :filter (fn [_ field-clause]- [:= (with-temporal-unit-if-field field-clause :day) [:relative-datetime -1 :day]])}-- ;; Adding a tilde (~) at the end of a past<n><unit>s filter means we should include the current day/etc.- ;; e.g. past30days = past 30 days, not including partial data for today ({:include-current false})- ;; past30days~ = past 30 days, *including* partial data for today ({:include-current true}).- ;; Adding a -from-<n><unit>s suffix at the end of the filter means we want to offset the range in the- ;; case of past filters into the past, in the case of next filters into the future.- ;; The implementation below uses the fact that if the relative suffix is not empty, then the- ;; include-current flag is true.- {:parser (#'qp.parameters.dates/regex->parser (re-pattern (str #"past([0-9]+)" @#'qp.parameters.dates/temporal-units-regex #"s" @#'qp.parameters.dates/relative-suffix-regex))- [:int-value :unit :relative-suffix :int-value-1 :unit-1])- :range (fn [{:keys [unit int-value unit-range to-period relative-suffix unit-1 int-value-1]} dt]- (let [dt-offset (cond-> dt- unit-1 (t/minus ((get-in @#'qp.parameters.dates/operations-by-date-unit [unit-1 :to-period]) int-value-1)))- dt-resolution (#'qp.parameters.dates/maybe-reduce-resolution unit dt-offset)]- (unit-range (t/minus dt-resolution (to-period int-value))- (t/minus dt-resolution (to-period (if (#'qp.parameters.dates/include-current? relative-suffix) 0 1))))))-- :filter (fn [{:keys [unit int-value relative-suffix unit-1 int-value-1]} field-clause]- (if unit-1- [:relative-time-interval- field-clause- (- int-value)- (keyword unit)- (- int-value-1)- (keyword unit-1)]- [:time-interval- field-clause- (- int-value)- (keyword unit)- {:include-current (#'qp.parameters.dates/include-current? relative-suffix)}]))}-- {:parser (#'qp.parameters.dates/regex->parser (re-pattern (str #"next([0-9]+)" @#'qp.parameters.dates/temporal-units-regex #"s" @#'qp.parameters.dates/relative-suffix-regex))- [:int-value :unit :relative-suffix :int-value-1 :unit-1])- :range (fn [{:keys [unit int-value unit-range to-period relative-suffix unit-1 int-value-1]} dt]- (let [dt-offset (cond-> dt- unit-1 (t/plus ((get-in @#'qp.parameters.dates/operations-by-date-unit [unit-1 :to-period]) int-value-1)))- dt-resolution (#'qp.parameters.dates/maybe-reduce-resolution unit dt-offset)]- (unit-range (t/plus dt-resolution (to-period (if (#'qp.parameters.dates/include-current? relative-suffix) 0 1)))- (t/plus dt-resolution (to-period int-value)))))- :filter (fn [{:keys [unit int-value relative-suffix unit-1 int-value-1]} field-clause]- (if unit-1- [:relative-time-interval- field-clause- int-value- (keyword unit)- int-value-1- (keyword unit-1)]- [:time-interval- field-clause- int-value- (keyword unit)- {:include-current (#'qp.parameters.dates/include-current? relative-suffix)}]))}-- {:parser (#'qp.parameters.dates/regex->parser (re-pattern (str #"last" @#'qp.parameters.dates/temporal-units-regex))- [:unit])- :range (fn [{:keys [unit unit-range to-period]} dt]- (let [last-unit (t/minus (#'qp.parameters.dates/maybe-reduce-resolution unit dt) (to-period 1))]- (unit-range last-unit last-unit)))- :filter (fn [{:keys [unit]} field-clause]- [:time-interval field-clause :last (keyword unit)])}-- {:parser (#'qp.parameters.dates/regex->parser (re-pattern (str #"this" @#'qp.parameters.dates/temporal-units-regex))- [:unit])- :range (fn [{:keys [unit unit-range]} dt]- (let [dt-adj (#'qp.parameters.dates/maybe-reduce-resolution unit dt)]- (unit-range dt-adj dt-adj)))- :filter (fn [{:keys [unit]} field-clause]- [:time-interval field-clause :current (keyword unit)])}])--;; TODO - using `range->filter` so much below seems silly. Why can't we just bucket the field and use `:=` clauses?-(defn- range->filter- [{:keys [start end]} field-clause]- [:between- (with-temporal-unit-if-field field-clause :day)- (#'qp.parameters.dates/->iso-8601-date start)- (#'qp.parameters.dates/->iso-8601-date end)])--(def ^:private absolute-date-string-decoders- ;; year and month- [{:parser (#'qp.parameters.dates/regex->parser #"([0-9]{4}-[0-9]{2})" [:date])- :range (fn [{:keys [date]} _]- (#'qp.parameters.dates/month-range date date))- :filter (fn [{:keys [date]} field-clause]- (range->filter (#'qp.parameters.dates/month-range date date) field-clause))}- ;; quarter year- {:parser (#'qp.parameters.dates/regex->parser #"(Q[1-4]{1})-([0-9]{4})" [:quarter :year])- :range (fn [{:keys [quarter year]} _]- (#'qp.parameters.dates/absolute-quarter-range quarter (Integer/parseInt year)))- :filter (fn [{:keys [quarter year]} field-clause]- (range->filter (#'qp.parameters.dates/absolute-quarter-range quarter (Integer/parseInt year))- field-clause))}- ;; single day- {:parser (#'qp.parameters.dates/regex->parser #"([0-9-T:]+)" [:date])- :range (fn [{:keys [date]} _]- {:start date :end date :unit (#'qp.parameters.dates/absolute-date->unit date)})- :filter (fn [{:keys [date]} field-clause]- (let [unit (#'qp.parameters.dates/absolute-date->unit date)- iso8601str (case unit- :day (#'qp.parameters.dates/->iso-8601-date date)- :minute (#'qp.parameters.dates/->iso-8601-date-time date))]- [:= (with-temporal-unit-if-field field-clause unit) iso8601str]))}- ;; day range- {:parser (#'qp.parameters.dates/regex->parser #"([0-9-T]+)~([0-9-T]+)" [:date-1 :date-2])- :range (fn [{:keys [date-1 date-2]} _]- {:start date-1 :end date-2 :unit (#'qp.parameters.dates/absolute-date->unit date-1)})- :filter (fn [{:keys [date-1 date-2]} field-clause]- [:between- (with-temporal-unit-if-field field-clause :day)- (#'qp.parameters.dates/->iso-8601-date date-1)- (#'qp.parameters.dates/->iso-8601-date date-2)])}- ;; datetime range- {:parser (#'qp.parameters.dates/regex->parser #"([0-9-T:]+)~([0-9-T:]+)" [:date-1 :date-2])- :range (fn [{:keys [date-1 date-2]} _]- {:start date-1, :end date-2 :unit (#'qp.parameters.dates/absolute-date->unit date-1)})- :filter (fn [{:keys [date-1 date-2]} field-clause]- [:between- (with-temporal-unit-if-field field-clause :default)- (#'qp.parameters.dates/->iso-8601-date-time date-1)- (#'qp.parameters.dates/->iso-8601-date-time date-2)])}- ;; before day- {:parser (#'qp.parameters.dates/regex->parser #"~([0-9-T:]+)" [:date])- :range (fn [{:keys [date]} _]- {:end date :unit (#'qp.parameters.dates/absolute-date->unit date)})- :filter (fn [{:keys [date]} field-clause]- [:< (with-temporal-unit-if-field field-clause :day) (#'qp.parameters.dates/->iso-8601-date date)])}- ;; after day- {:parser (#'qp.parameters.dates/regex->parser #"([0-9-T:]+)~" [:date])- :range (fn [{:keys [date]} _]- {:start date :unit (#'qp.parameters.dates/absolute-date->unit date)})- :filter (fn [{:keys [date]} field-clause]- [:> (with-temporal-unit-if-field field-clause :day) (#'qp.parameters.dates/->iso-8601-date date)])}- ;; exclusions- {:parser (#'qp.parameters.dates/regex->parser qp.parameters.dates/date-exclude-regex [:unit :exclusions])- :filter (fn [{:keys [unit exclusions]} field-clause]- (let [unit (keyword unit)- exclusions (map (partial #'qp.parameters.dates/excluded-datetime unit)- (str/split exclusions #"-"))]- (when (and (seq exclusions) (every? some? exclusions))- (into [:!= (with-temporal-unit-if-field field-clause (#'qp.parameters.dates/excluded-temporal-unit unit))]- exclusions))))}])--(def ^:private all-date-string-decoders- (concat relative-date-string-decoders absolute-date-string-decoders))--(mu/defn date-string->filter :- ::mbql.s/Filter- "Takes a string description of a *date* (not datetime) range such as 'lastmonth' or '2016-07-15~2016-08-6', or- an absolute date *or datetime* string, and returns a corresponding MBQL filter clause for a given field reference."- [date-string :- :string- field :- [:or ::lib.schema.id/field ::mbql.s/FieldOrExpressionRef]]- #_{:clj-kondo/ignore [:deprecated-var]}- (or (#'qp.parameters.dates/execute-decoders all-date-string-decoders :filter (mbql.u/wrap-field-id-if-needed field) date-string)- (throw (ex-info (tru "Don''t know how to parse date string {0}" (pr-str date-string))- {:type qp.error-type/invalid-parameter- :date-string date-string}))))--(p/import-vars- [qp.parameters.dates- date-str->datetime-range- not-single-date-type?- date-exclude-regex- date-type?- date-string->range- exclusion-date-type])diff --git a/src/metabase/driver/common/parameters/operators.clj b/src/metabase/driver/common/parameters/operators.cljdeleted file mode 100644index dbe1144a435..00000000000--- a/src/metabase/driver/common/parameters/operators.clj+++ /dev/null@@ -1,86 +0,0 @@-(ns metabase.driver.common.parameters.operators- "This namespace handles parameters that are operators.-- {:type :number/between- :target [:dimension- [:field- 26- {:source-field 5}]]- :value [3 5]}-- DEPRECATED: use [[metabase.query-processor.parameters.operators]] going forward."- {:deprecated "0.57.0"}- (:refer-clojure :exclude [get-in])- (:require- [metabase.legacy-mbql.schema :as mbql.s]- [metabase.legacy-mbql.util :as mbql.u]- [metabase.lib.schema.parameter :as lib.schema.parameter]- [metabase.query-processor.error-type :as qp.error-type]- [metabase.query-processor.parameters.operators :as qp.params.ops]- [metabase.util.i18n :refer [tru]]- [metabase.util.malli :as mu]- [metabase.util.performance :refer [get-in]]- [potemkin :as p]))--(p/import-vars- [qp.params.ops- operator?])--(defn- operator-options-fn- [param-type]- (get-in lib.schema.parameter/types [param-type :options-fn]- ;; Default is to conj on the end if options are provided.- (fn [clause options]- (cond-> clause- options (conj options)))))--(mu/defn- verify-type-and-arity- [field param-type param-value]- (letfn [(maybe-arity-error [n]- (when (not= n (count param-value))- (throw (ex-info (format "Operations Invalid arity: expected %s but received %s"- n (count param-value))- {:param-type param-type- :param-value param-value- :field-id (second field)- :type qp.error-type/invalid-parameter}))))]- (condp = (#'qp.params.ops/operator-arity param-type)- :unary- (maybe-arity-error 1)-- :binary- (maybe-arity-error 2)-- :variadic- (when-not (sequential? param-value)- (throw (ex-info (tru "Invalid values provided for operator: {0}" param-type)- {:param-type param-type- :param-value param-value- :field-id (second field)- :type qp.error-type/invalid-parameter})))-- (throw (ex-info (tru "Unrecognized operation: {0}" param-type)- {:param-type param-type- :param-value param-value- :field-id (second field)- :type qp.error-type/invalid-parameter})))))--(mu/defn to-clause :- ::mbql.s/Filter- "Convert an operator style parameter into an mbql clause. Will also do arity checks and throws an ex-info with- `:type qp.error-type/invalid-parameter` if arity is incorrect."- [param :- ::lib.schema.parameter/parameter]- (let [{param-type :type, [a b :as param-value] :value, [_ field] :target, options :options} (#'qp.params.ops/normalize-param param)]- (verify-type-and-arity field param-type param-value)- #_{:clj-kondo/ignore [:deprecated-var]}- (let [field' (mbql.u/wrap-field-id-if-needed field)- opts-fn (operator-options-fn param-type)]- (case (#'qp.params.ops/operator-arity param-type)- :binary (opts-fn [(keyword (name param-type)) field' a b] options)- :unary (opts-fn [(keyword (name param-type)) field' a] options)- :variadic (opts-fn (into [(keyword (name param-type)) field'] param-value) options)- #_else- (throw (ex-info (format "Unrecognized operator: %s" param-type)- {:param-type param-type- :param-value param-value- :field-id (second field)- :type qp.error-type/invalid-parameter}))))))diff --git a/src/metabase/driver/sql/parameters/substitution.clj b/src/metabase/driver/sql/parameters/substitution.cljindex e3a315ab042..d68069a7efc 100644--- a/src/metabase/driver/sql/parameters/substitution.clj+++ b/src/metabase/driver/sql/parameters/substitution.clj@@ -9,13 +9,12 @@(:refer-clojure :exclude [not-empty mapv])(:require[clojure.string :as str]- [medley.core :as m][metabase.driver :as driver][metabase.driver-api.core :as driver-api]- ^{:clj-kondo/ignore [:deprecated-namespace]} [metabase.driver.common.parameters.dates :as params.dates]- ^{:clj-kondo/ignore [:deprecated-namespace]} [metabase.driver.common.parameters.operators :as params.ops][metabase.driver.sql.query-processor :as sql.qp][metabase.lib.core :as lib]+ [metabase.query-processor.parameters.dates :as params.dates]+ [metabase.query-processor.parameters.operators :as params.ops][metabase.util :as u][metabase.util.date-2 :as u.date][metabase.util.honey-sql-2 :as h2x]@@ -51,10 +50,20 @@{:sql-string sql-string:param-values param-seq})+(defmulti ->honeysql+ "Compile `mbql-5-clause` to Honey SQL, converting to legacy MBQL first if needed."+ {:added "0.62.0", :arglists '([driver mbql-5-clause])}+ driver/dispatch-on-initialized-driver+ :hierarchy #'driver/hierarchy)++(defmethod ->honeysql :default+ [driver mbql-5-clause]+ (sql.qp/->honeysql driver (lib/->legacy-MBQL mbql-5-clause)))+(defn- honeysql->prepared-stmt-subs"Convert X to a replacement snippet info map by passing it to HoneySQL's `format` function."[driver x]- (let [honeysql (sql.qp/->honeysql driver x)+ (let [honeysql (->honeysql driver x)[snippet & args] (sql.qp/format-honeysql driver honeysql)](make-stmt-subs snippet args)))@@ -249,26 +258,9 @@{:replacement-snippet snippet:prepared-statement-args args}))-(defmulti field->clause- "Returns an MBQL field clause with the given options."- {:added "0.61.0" :arglists '([driver field other-opts])}- driver/dispatch-on-initialized-driver- :hierarchy #'driver/hierarchy)--(mu/defmethod field->clause :default :- driver-api/mbql.schema.field- [_driver field other-opts]- (driver-api/normalize- driver-api/mbql.schema.field- [:field- (:id field)- (merge {:base-type (:base-type field)- driver-api/qp.add.source-table (:table-id field)- ::compiling-field-filter? true}- other-opts)]))--(mu/defn- field->field-filter-clause :- [:or driver-api/mbql.schema.field :mbql.clause/field]+(mu/defn- field->field-ref :- :mbql.clause/field[driver :- :keyword- field :- driver-api/schema.metadata.column+ col :- driver-api/schema.metadata.columnparam-type :- driver-api/schema.parameter.typevalue];; The [[metabase.query-processor.middleware.parameters/substitute-parameters]] QP middleware actually happens before@@ -277,15 +269,21 @@;; middleware would work either because we don't know what Field this parameter actually refers to until we resolve;; the parameter. There's probably _some_ way to structure things that would make this "duplicate" call unneeded, but;; I haven't figured out what that is yet- (field->clause driver field (m/assoc-some {} :temporal-unit (align-temporal-unit-with-param-type-and-value driver field param-type value))))+ (let [temporal-unit (align-temporal-unit-with-param-type-and-value driver col param-type value)]+ (-> (cond-> col+ temporal-unit (lib/with-temporal-bucket temporal-unit))+ lib/ref+ (lib/update-options assoc+ driver-api/qp.add.source-table (:table-id col)+ ::compiling-field-filter? true))))(mu/defn- field->identifier :- driver-api/schema.common.non-blank-string"Return an appropriate snippet to represent this `field` in SQL given its param type.For non-date Fields, this is just a quoted identifier; for dates, the SQL includes appropriately bucketing based onthe `param-type`."[driver field param-type value]- (->> (field->field-filter-clause driver field param-type value)- (sql.qp/->honeysql driver)+ (->> (field->field-ref driver field param-type value)+ (->honeysql driver)(honeysql->replacement-snippet-info driver):replacement-snippet))@@ -300,45 +298,6 @@(-> (lib/parsed-date-time-range-param start end)->datetime-replacement-snippet-info))))-(defmulti to-clause- "Helper to dispatch to `params.ops/to-clause` for `:sql` or `qp.params.ops/to-clause` for `:sql-mbql5`."- {:added "0.61.0" :arglists '([driver param])}- driver/dispatch-on-initialized-driver- :hierarchy #'driver/hierarchy)--(defmethod to-clause :default [_driver param] (params.ops/to-clause param))--(defmulti desugar-filter-clause- "Helper to dispatch to `driver-api/desugar-filter-clause` for `:sql` or `lib/desugar-filter-clause` for `:sql-mbql5`."- {:added "0.61.0" :arglists '([driver filter-clause])}- driver/dispatch-on-initialized-driver- :hierarchy #'driver/hierarchy)--(defmethod desugar-filter-clause :default [_driver filter-clause]- #_{:clj-kondo/ignore [:deprecated-var]}- (driver-api/desugar-filter-clause filter-clause))--(defmulti wrap-value-literals-in-mbql- "Helper to dispatch to `driver-api/wrap-value-literals-in-mbql` for `:sql`- or `driver-api/wrap-value-literals-in-mbql5` for `:sql-mbql5`."- {:added "0.61.0" :arglists '([driver mbql])}- driver/dispatch-on-initialized-driver- :hierarchy #'driver/hierarchy)--(defmethod wrap-value-literals-in-mbql :default- [_driver mbql]- #_{:clj-kondo/ignore [:deprecated-var]}- (driver-api/wrap-value-literals-in-mbql mbql))--(defmulti date-string->filter- "Helper to dispatch to `params.dates/date-string->filter` for `:sql`- or `qp.params.dates/date-string->filter` for `:sql-mbql5`."- {:added "0.61.0" :arglists '([driver date-string field])}- driver/dispatch-on-initialized-driver- :hierarchy #'driver/hierarchy)--(defmethod date-string->filter :default [_driver date-string field] (params.dates/date-string->filter date-string field))-(mu/defn- field-filter->replacement-snippet-info :- ::param-snippet-info"Return `[replacement-snippet & prepared-statement-args]` appropriate for a field filter parameter."[driver {{param-type :type, value :value, :as params} :value, field :field, :as field-filter}]@@ -346,25 +305,25 @@(letfn [(prepend-field [x](update x :replacement-snippet(partial str (field->identifier driver field param-type value) " ")))- (->honeysql [form]- (sql.qp/->honeysql driver form))]+ (->honeysql* [form]+ (->honeysql driver form))](cond(params.ops/operator? param-type)#_{:clj-kondo/ignore [:deprecated-var]}- (->> (assoc params :target [:dimension (field->field-filter-clause driver field param-type value)])- (to-clause driver)- (desugar-filter-clause driver)- (wrap-value-literals-in-mbql driver)- ->honeysql+ (->> (assoc params :target [:dimension (field->field-ref driver field param-type value)])+ params.ops/to-clause+ lib/desugar-filter-clause+ driver-api/wrap-value-literals-in-mbql5+ ->honeysql*(honeysql->replacement-snippet-info driver))(params.dates/exclusion-date-type param-type value)- (let [field-clause (field->field-filter-clause driver field param-type value)]+ (let [field-ref (field->field-ref driver field param-type value)]#_{:clj-kondo/ignore [:deprecated-var]}- (->> (date-string->filter driver value field-clause)- (desugar-filter-clause driver)- (wrap-value-literals-in-mbql driver)- ->honeysql+ (->> (params.dates/date-string->filter value field-ref)+ lib/desugar-filter-clause+ driver-api/wrap-value-literals-in-mbql5+ ->honeysql*(honeysql->replacement-snippet-info driver)));; Special handling for `FieldFilter`s on `:type/DateTime` fields. DateTime range is always generated.@@ -387,11 +346,11 @@(field-filter->equals-clause-sql driver value)))))(defn- replace-alias- [driver field alias replacement-snippet-info]+ [driver col alias replacement-snippet-info](if (str/blank? alias)replacement-snippet-info- (let [[old-name] (->> (field->clause driver field nil)- (sql.qp/->honeysql driver)+ (let [[old-name] (->> (lib/ref col)+ (->honeysql driver)(sql.qp/format-honeysql driver))](update replacement-snippet-info :replacement-snippet str/replace old-name alias))))@@ -429,9 +388,11 @@(mu/defmethod ->replacement-snippet-info [:sql :metabase.lib.parameters.parse.types/temporal-unit][driver :- :keyword{:keys [value field alias]} :- :metabase.lib.parameters.parse.types/temporal-unit]- (let [replacement-snippet-info (->> (field->clause driver field (when (not= value lib/parsed-param-no-value-placeholder)- {:temporal-unit (keyword value)}))- (sql.qp/->honeysql driver)+ (let [replacement-snippet-info (->> (cond-> field+ (not= value lib/parsed-param-no-value-placeholder)+ (lib/with-temporal-bucket (keyword value)))+ lib/ref+ (->honeysql driver)(honeysql->replacement-snippet-info driver))](replace-alias driver field alias replacement-snippet-info)))diff --git a/src/metabase/driver/sql_mbql5.clj b/src/metabase/driver/sql_mbql5.cljindex 85dfc0b928e..a2754920f60 100644--- a/src/metabase/driver/sql_mbql5.clj+++ b/src/metabase/driver/sql_mbql5.clj@@ -12,12 +12,10 @@[metabase.lib.options :as lib.options][metabase.lib.schema.common :as lib.schema.common][metabase.lib.util :as lib.util]- [metabase.query-processor.parameters.dates :as qp.params.dates]- [metabase.query-processor.parameters.operators :as qp.params.ops][metabase.query-processor.util.add-alias-info :as add][metabase.util.honey-sql-2 :as h2x][metabase.util.malli :as mu]- [metabase.util.performance :refer [mapv get-in]]))+ [metabase.util.performance :refer [get-in mapv]]))(driver/register! :sql-mbql5, :parent :sql, :abstract? true)@@ -178,22 +176,6 @@(defmethod sql.qp/breakout-options-index :sql-mbql5 [_driver] 1)-(mu/defmethod sql.params.substitution/field->clause :sql-mbql5 :- :mbql.clause/field- [driver field other-opts]- (sql.qp/mbql-clause-with-opts driver :field- (merge {:base-type (:base-type field)- driver-api/qp.add.source-table (:table-id field)- ::sql.params.substitution/compiling-field-filter? true}- other-opts)- (:id field)))--(defmethod sql.params.substitution/to-clause :sql-mbql5 [_driver param] (qp.params.ops/to-clause param))--(defmethod sql.params.substitution/desugar-filter-clause :sql-mbql5 [_driver filter-clause] (lib/desugar-filter-clause filter-clause))--(defmethod sql.params.substitution/wrap-value-literals-in-mbql :sql-mbql5- [_driver mbql]- #_{:clj-kondo/ignore [:deprecated-var]}- (driver-api/wrap-value-literals-in-mbql5 mbql))--(defmethod sql.params.substitution/date-string->filter :sql-mbql5 [_driver date-string field] (qp.params.dates/date-string->filter date-string field))+(defmethod sql.params.substitution/->honeysql :sql-mbql5+ [driver mbql-5-clause]+ (sql.qp/->honeysql driver mbql-5-clause))diff --git a/src/metabase/driver_api/core.clj b/src/metabase/driver_api/core.cljindex 91d91fb1c88..808c09a5625 100644--- a/src/metabase/driver_api/core.clj+++ b/src/metabase/driver_api/core.clj@@ -125,14 +125,11 @@logger/level-enabled?mbql.u/aggregation-at-indexmbql.u/assoc-field-options- mbql.u/desugar-filter-clausembql.u/expression-with-namembql.u/field-options- mbql.u/negate-filter-clausembql.u/normalize-tokenmbql.u/query->max-rows-limitmbql.u/query->source-table-id- mbql.u/simplify-compound-filtermbql.u/update-field-optionsmdb/clob->strmdb/data-sourcediff --git a/src/metabase/legacy_mbql/jvm_util.clj b/src/metabase/legacy_mbql/jvm_util.cljdeleted file mode 100644index 6f1a8721cc5..00000000000--- a/src/metabase/legacy_mbql/jvm_util.clj+++ /dev/null@@ -1,22 +0,0 @@-(ns metabase.legacy-mbql.jvm-util- "This namespace contains functionality that is not compatible with js, hence can not be stored in corresponding- cljc ns, ie. [[metabase.legacy-mbql.util]]."- {:deprecated "0.57.0"}- (:require- [metabase.lib.filter.desugar.jvm :as lib.filter.desugar.jvm]- [metabase.util.match :as match]))--(defn desugar-host-and-domain- "Unwrap host and domain."- {:deprecated "0.57.0"}- [expression]- (match/replace- expression- [:host column]- (&recur [:regex-match-first column (str lib.filter.desugar.jvm/host-regex)])- [:domain column]- (&recur [:regex-match-first column (str lib.filter.desugar.jvm/domain-regex)])- [:subdomain column]- (&recur [:regex-match-first column (str lib.filter.desugar.jvm/subdomain-regex)])- [:path column]- (&recur [:regex-match-first column (str lib.filter.desugar.jvm/path-regex)])))diff --git a/src/metabase/legacy_mbql/predicates.cljc b/src/metabase/legacy_mbql/predicates.cljcdeleted file mode 100644index f1b7e2a0641..00000000000--- a/src/metabase/legacy_mbql/predicates.cljc+++ /dev/null@@ -1,26 +0,0 @@-(ns metabase.legacy-mbql.predicates- "Predicate functions for checking whether something is a valid instance of a given MBQL clause."- {:deprecated "0.57.0"}- (:require- [metabase.legacy-mbql.schema :as mbql.s]- [metabase.util.malli.registry :as mr]))--(defn Field?- "Is this a valid Field clause?"- [x]- ((mr/validator ::mbql.s/FieldOrExpressionRef) x))--(defn Filter?- "Is this a valid `:filter` clause?"- [x]- ((mr/validator ::mbql.s/Filter) x))--(defn Emptyable?- "Is this a valid Emptyable clause?"- [x]- ((mr/validator ::mbql.s/Emptyable) x))--(defn FieldOrExpressionDef?- "Is this a something that is valid as a top-level expression definition?"- [x]- ((mr/validator ::mbql.s/FieldOrExpressionDef) x))diff --git a/src/metabase/legacy_mbql/util.cljc b/src/metabase/legacy_mbql/util.cljcindex 753ba3bca09..19625fde216 100644--- a/src/metabase/legacy_mbql/util.cljc+++ b/src/metabase/legacy_mbql/util.cljc@@ -3,12 +3,9 @@DEPRECATED: Use [[metabase.lib.core]] for MBQL manipulation in all new code."{:deprecated "0.57.0"}- (:refer-clojure :exclude [some mapv every? not-empty get-in #?(:clj for)])+ (:refer-clojure :exclude [every? not-empty get-in #?(:clj for)])(:require- #?@(:clj- [[metabase.legacy-mbql.jvm-util :as mbql.jvm-u]])[clojure.string :as str]- [metabase.legacy-mbql.predicates :as mbql.preds][metabase.legacy-mbql.schema :as mbql.s][metabase.lib.schema.common :as lib.schema.common][metabase.lib.schema.id :as lib.schema.id]@@ -17,9 +14,7 @@[metabase.util.i18n :as i18n][metabase.util.log :as log][metabase.util.malli :as mu]- [metabase.util.match :as match]- [metabase.util.performance :refer [some mapv every? not-empty get-in #?(:clj for)]]- [metabase.util.time :as u.time]))+ [metabase.util.performance :refer [every? not-empty get-in #?(:clj for)]]))(defn mbql-clause?"True if `x` is an MBQL clause (a sequence with a keyword as its first arg).@@ -46,19 +41,6 @@((set k-or-ks) (first x))(= k-or-ks (first x)))))-(defn check-clause- "Returns `x` if it's an instance of a clause defined by keyword(s) `k-or-ks`-- (check-clause :count [:count 10]) ; => [:count 10]- (check-clause? #{:+ :- :* :/} [:+ 10 20]) ; -> [:+ 10 20]- (check-clause :sum [:count 10]) ; => nil-- DEPRECATED: use [[metabase.lib.core/clause-of-type?]] going forward"- {:deprecated "0.57.0"}- [k-or-ks x]- (when (is-clause? k-or-ks x)- x))-(mu/defn normalize-token :- [:or :keyword :string]"Convert a string or keyword in various cases (`lisp-case`, `snake_case`, or `SCREAMING_SNAKE_CASE`) to a lisp-casedkeyword.@@ -81,468 +63,8 @@(str/replace \_ \-)keyword))))-(defn- combine-compound-filters-of-type- {:deprecated "0.57.0"}- [compound-type subclauses]- (mapcat #(match/match-one %- [#{compound-type} & args] args- _ [%])- subclauses))--(declare simplify-compound-filter)--(defn- simplify-and-or-filter- {:deprecated "0.57.0"}- [op args]- #_{:clj-kondo/ignore [:deprecated-var]}- (let [args (distinct (filter some? args))]- (case (count args)- ;; an empty filter, toss it- 0 nil- ;; single arg, unwrap it- 1 (simplify-compound-filter (first args))- (if (some (partial is-clause? op) args)- ;; clause of the same type embedded, faltten it- (recur op (combine-compound-filters-of-type op args))- ;; simplify the arguments- (let [simplified (map simplify-compound-filter args)]- (if (= simplified args)- ;; no change, we can stop- (into [op] args)- ;; there is a change, we might be able to simplify even further- (recur op simplified)))))))--(defn simplify-compound-filter- "Simplify compound `:and`, `:or`, and `:not` compound filters, combining or eliminating them where possible. This- also fixes theoretically disallowed compound filters like `:and` with only a single subclause, and eliminates `nils`- and duplicate subclauses from the clauses.-- DEPRECATED: This will be removed in the near future.- Use [[metabase.lib.filter.simplify-compound/simplify-compound-filter]] going forward."- {:deprecated "0.57.0"}- [x]- #_{:clj-kondo/ignore [:deprecated-var]}- (cond- ;; look for filters in the values- (map? x) (update-vals x simplify-compound-filter)- (seq? x) (recur (vec x))- ;; not a map and not vector, leave it as is- (not (vector? x)) x- ;; an empty filter, toss it- (not (some some? x)) nil- :else (let [[op & [farg :as args]] x]- (case op- :not (if-not (seqable? farg)- x- (case (first farg)- ;; double negation, eliminate both- :not (recur (second farg))- ;; use de Morgan's law to push the negation down- :and (simplify-and-or-filter :or (map #(vector :not %) (rest farg)))- :or (simplify-and-or-filter :and (map #(vector :not %) (rest farg)))- x))- :and (simplify-and-or-filter :and args)- :or (simplify-and-or-filter :or args)- ;; simplify the elements of the vector- (mapv simplify-compound-filter x)))))--(mu/defn combine-filter-clauses :- ::mbql.s/Filter- "Combine two filter clauses into a single clause in a way that minimizes slapping a bunch of `:and`s together if- possible.-- DEPRECATED: This will be removed in the near future. Use lib utils going forward."- {:deprecated "0.57.0"}- [filter-clause & more-filter-clauses]- #_{:clj-kondo/ignore [:deprecated-var]}- (simplify-compound-filter (cons :and (cons filter-clause more-filter-clauses))))--(defn desugar-inside- "Rewrite `:inside` filter clauses as a pair of `:between` clauses."- {:deprecated "0.57.0"}- [m]- (match/replace m- [:inside lat-field lon-field lat-max lon-min lat-min lon-max]- [:and- [:between lat-field lat-min lat-max]- [:between lon-field lon-min lon-max]]))--(defn desugar-is-null-and-not-null- "Rewrite `:is-null` and `:not-null` filter clauses as simpler `:=` and `:!=`, respectively."- {:deprecated "0.57.0"}- [m]- (match/replace m- [:is-null field] [:= field nil]- [:not-null field] [:!= field nil]))-(declare field-options)-(defn- emptyable?- {:deprecated "0.57.0"}- [clause]- (if (is-clause? #{:field :expression :aggregation} clause)- (-> clause- field-options- :base-type- (isa? :metabase.lib.schema.expression/emptyable))- (mbql.preds/Emptyable? clause)))--(defn- desugar-is-empty-and-not-empty- "Rewrite `:is-empty` and `:not-empty` filter clauses as simpler `:=` and `:!=`, respectively.-- If `:not-empty` is called on `:metabase.lib.schema.expression/emptyable` type, expand check for empty string. For- non-`emptyable` types act as `:is-null`. If field has nil base type it is considered not emptyable expansion wise."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:is-empty clause]- (if (emptyable? clause)- [:or [:= clause nil] [:= clause ""]]- [:= clause nil])-- [:not-empty clause]- (if (emptyable? clause)- [:and [:!= clause nil] [:!= clause ""]]- [:!= clause nil])))--(defn- replace-field-or-expression- "Replace a field or expression inside :time-interval"- {:deprecated "0.57.0"}- [m unit]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:field id-or-name opts]- [:field id-or-name (assoc opts :temporal-unit unit)]-- [:expression & _]- (let [[_expression expression-name opts] &match]- [:expression expression-name (assoc opts :temporal-unit unit)])))--(defn- desugar-time-interval- "Rewrite `:time-interval` filter clauses as simpler ones like `:=` or `:between`."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:time-interval field-or-expression n unit] (&recur [:time-interval field-or-expression n unit nil])-- ;; replace current/last/next with corresponding value of n and recur- [:time-interval field-or-expression :current unit options] (&recur [:time-interval field-or-expression 0 unit options])- [:time-interval field-or-expression :last unit options] (&recur [:time-interval field-or-expression -1 unit options])- [:time-interval field-or-expression :next unit options] (&recur [:time-interval field-or-expression 1 unit options])-- [:time-interval field-or-expression (n :guard #{-1}) unit {:include-current &truthy}]- [:between- (replace-field-or-expression field-or-expression unit)- [:relative-datetime n unit]- [:relative-datetime 0 unit]]-- [:time-interval field-or-expression (n :guard #{1}) unit {:include-current &truthy}]- [:between- (replace-field-or-expression field-or-expression unit)- [:relative-datetime 0 unit]- [:relative-datetime n unit]]-- [:time-interval field-or-expression (n :guard #{-1 0 1}) unit _]- [:= (replace-field-or-expression field-or-expression unit) [:relative-datetime n unit]]-- [:time-interval field-or-expression (n :guard neg?) unit {:include-current &truthy}]- [:between- (replace-field-or-expression field-or-expression unit)- [:relative-datetime n unit]- [:relative-datetime 0 unit]]-- [:time-interval field-or-expression (n :guard neg?) unit _]- [:between- (replace-field-or-expression field-or-expression unit)- [:relative-datetime n unit]- [:relative-datetime -1 unit]]-- [:time-interval field-or-expression n unit {:include-current &truthy}]- [:between- (replace-field-or-expression field-or-expression unit)- [:relative-datetime 0 unit]- [:relative-datetime n unit]]-- [:time-interval field-or-expression n unit _]- [:between- (replace-field-or-expression field-or-expression unit)- [:relative-datetime 1 unit]- [:relative-datetime n unit]]))--(defn desugar-relative-time-interval- "Transform `:relative-time-interval` to `:and` expression."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace- m- [:relative-time-interval col value bucket offset-value offset-bucket]- (let [col-default-bucket (cond-> col (and (vector? col) (= 3 (count col)))- (update 2 assoc :temporal-unit :default))- offset [:interval offset-value offset-bucket]- lower-bound (if (neg? value)- [:relative-datetime value bucket]- [:relative-datetime 1 bucket])- upper-bound (if (neg? value)- [:relative-datetime 0 bucket]- [:relative-datetime (inc value) bucket])- lower-with-offset [:+ lower-bound offset]- upper-with-offset [:+ upper-bound offset]]- [:and- [:>= col-default-bucket lower-with-offset]- [:< col-default-bucket upper-with-offset]])))--(defn desugar-during- "Transform a `:during` expression to an `:and` expression."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace- m- [:during col value unit]- (let [col-default-bucket (cond-> col- (and (vector? col) (= 3 (count col)))- (update 2 assoc :temporal-unit :default))- lower-bound (u.time/truncate value unit)- upper-bound (u.time/add lower-bound unit 1)]- [:and- [:>= col-default-bucket lower-bound]- [:< col-default-bucket upper-bound]])))--(defn desugar-if- "Transform a `:if` expression to an `:case` expression."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:if & args]- (into [:case] args)))--(defn desugar-in- "Transform `:in` and `:not-in` expressions to `:=` and `:!=` expressions."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:in & args]- (into [:=] args)-- [:not-in & args]- (into [:!=] args)))--(defn- desugar-does-not-contain- "Rewrite `:does-not-contain` filter clauses as simpler `[:not [:contains ...]]` clauses.-- Note that [[desugar-multi-argument-comparisons]] will have already desugared any 3+ argument `:does-not-contain` to- several `[:and [:does-not-contain ...] [:does-not-contain ...] ...]` clauses, which then get rewritten here into- `[:and [:not [:contains ...]] [:not [:contains ...]]]`."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:does-not-contain & args]- [:not (into [:contains] args)]))--(defn- desugar-multi-argument-comparisons- "`:=`, `!=`, `:contains`, `:does-not-contain`, `:starts-with` and `:ends-with` clauses with more than 2 args- automatically get rewritten as compound filters.-- [:= field x y] -> [:or [:= field x] [:= field y]]- [:!= field x y] -> [:and [:!= field x] [:!= field y]]- [:does-not-contain field x y] -> [:and [:does-not-contain field x] [:does-not-contain field y]]-- Note that the optional options map is in different positions for `:contains`, `:does-not-contain`, `:starts-with` and- `:ends-with` depending on the number of arguments. 2-argument forms use the legacy style `[:contains field x opts]`.- Multi-argument forms use MBQL 5 style with the options at index 1, **even if there are no options**:- `[:contains {} field x y z]`."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [:= field x y & more]- (apply vector :or (for [x (concat [x y] more)]- [:= field x]))-- [:!= field x y & more]- (apply vector :and (for [x (concat [x y] more)]- [:!= field x]))-- [(op :guard #{:contains :does-not-contain :starts-with :ends-with})- (opts :guard map?)- field x y & more]- (let [tail (when (seq opts) [opts])]- (apply vector- (if (= op :does-not-contain) :and :or)- (for [x (concat [x y] more)]- (into [op field x] tail))))))--(defn desugar-current-relative-datetime- "Replace `relative-datetime` clauses like `[:relative-datetime :current]` with `[:relative-datetime 0 <unit>]`.- `<unit>` is inferred from the `:field` the clause is being compared to (if any), otherwise falls back to `default.`"- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [clause field & (args :guard (some #{[:relative-datetime :current]} args))]- (let [temporal-unit (or (match/match-one field- [:field _ {:temporal-unit temporal-unit}] temporal-unit)- :default)]- (into [clause field] (match/replace args- [:relative-datetime :current]- [:relative-datetime 0 temporal-unit])))))--(def ^{:deprecated "0.57.0"} temporal-extract-ops->unit- "Mapping from the sugar syntax to extract datetime to the unit."- {[:get-year nil] :year-of-era- [:get-quarter nil] :quarter-of-year- [:get-month nil] :month-of-year- ;; default get-week mode is iso- [:get-week nil] :week-of-year-iso- [:get-week :iso] :week-of-year-iso- [:get-week :us] :week-of-year-us- [:get-week :instance] :week-of-year-instance- [:get-day nil] :day-of-month- [:get-day-of-week nil] :day-of-week- [:get-day-of-week :iso] :day-of-week-iso- [:get-hour nil] :hour-of-day- [:get-minute nil] :minute-of-hour- [:get-second nil] :second-of-minute})--(def ^:private ^{:deprecated "0.57.0"} temporal-extract-ops- #_{:clj-kondo/ignore [:deprecated-var]}- (->> (keys temporal-extract-ops->unit)- (map first)- set))--(defn- desugar-temporal-extract- "Replace datetime extractions clauses like `[:get-year field]` with `[:temporal-extract field :year]`."- {:deprecated "0.57.0"}- [m]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace m- [(op :guard temporal-extract-ops) field & args]- [:temporal-extract field (temporal-extract-ops->unit [op (first args)])]))--(defn- desugar-divide-with-extra-args- {:deprecated "0.57.0"}- [expression]- (match/replace expression- [:/ x y z & more]- (&recur (into [:/ [:/ x y]] (cons z more)))))--(defn- temporal-case-expression- "Creates a `:case` expression with a condition for each value of the given unit."- {:deprecated "0.57.0"}- [column unit n]- (let [user-locale #?(:clj (i18n/user-locale)- :cljs nil)]- [:case- (vec (for [raw-value (range 1 (inc n))]- [[:= column raw-value] (u.time/format-unit raw-value unit user-locale)]))- {:default ""}]))--(defn- desugar-temporal-names- "Given an expression like `[:month-name column]`, transforms this into a `:case` expression, which matches the input- numbers and transforms them into names.-- Uses the user's locale rather than the site locale, so the results will depend on the runner of the query, not just- the query itself. Filtering should be done based on the number, rather than the name."- {:deprecated "0.57.0"}- [expression]- #_{:clj-kondo/ignore [:deprecated-var]}- (match/replace expression- [:month-name column]- (&recur (temporal-case-expression column :month-of-year 12))- [:quarter-name column]- (&recur (temporal-case-expression column :quarter-of-year 4))- [:day-name column]- (&recur (temporal-case-expression column :day-of-week 7))))--(mu/defn desugar-expression :- ::mbql.s/FieldOrExpressionDef- "Rewrite various 'syntactic sugar' expressions like `:/` with more than two args into something simpler for drivers- to compile."- {:deprecated "0.57.0"}- [expression :- ::mbql.s/FieldOrExpressionDef]- #_{:clj-kondo/ignore [:deprecated-var]}- ;; The `mbql.jvm-u/desugar-host-and-domain` is implemented only for jvm because regexes are not compatible with- ;; Safari.- (let [desugar-host-and-domain* #?(:clj mbql.jvm-u/desugar-host-and-domain- :cljs (fn [x]- (log/warn "`desugar-host-and-domain` implemented only on JVM.")- x))]- (-> expression- desugar-divide-with-extra-args- desugar-host-and-domain*- desugar-temporal-names)))--(defn- maybe-desugar-expression- {:deprecated "0.57.0"}- [clause]- #_{:clj-kondo/ignore [:deprecated-var]}- (cond-> clause- (mbql.preds/FieldOrExpressionDef? clause) desugar-expression))--(mu/defn desugar-filter-clause :- ::mbql.s/Filter- "Rewrite various 'syntatic sugar' filter clauses like `:time-interval` and `:inside` as simpler, logically- equivalent clauses. This can be used to simplify the number of filter clauses that need to be supported by anything- that needs to enumerate all the possible filter types (such as driver query processor implementations, or the- implementation [[negate-filter-clause]] below.)-- DEPRECATED: This will be removed in a future release. Use [[metabase.lib.core/desugar-filter-clause]] instead going- forward."- {:deprecated "0.57.0"}- [filter-clause :- ::mbql.s/Filter]- #_{:clj-kondo/ignore [:deprecated-var]}- (-> filter-clause- desugar-current-relative-datetime- desugar-in- desugar-multi-argument-comparisons- desugar-does-not-contain- desugar-time-interval- desugar-relative-time-interval- desugar-is-null-and-not-null- desugar-is-empty-and-not-empty- desugar-inside- simplify-compound-filter- desugar-temporal-extract- desugar-during- desugar-if- maybe-desugar-expression))--(defmulti ^:private negate*- {:arglists '([mbql-clause]), :deprecated "0.57.0"}- first)--(defmethod negate* :not [[_ subclause]] subclause)-(defmethod negate* :and [[_ & subclauses]] (into [:or] (map negate* subclauses)))-(defmethod negate* :or [[_ & subclauses]] (into [:and] (map negate* subclauses)))-(defmethod negate* := [[_ field value]] [:!= field value])-(defmethod negate* :!= [[_ field value]] [:= field value])-(defmethod negate* :> [[_ field value]] [:<= field value])-(defmethod negate* :< [[_ field value]] [:>= field value])-(defmethod negate* :>= [[_ field value]] [:< field value])-(defmethod negate* :<= [[_ field value]] [:> field value])--(defmethod negate* :between- [[_ field min-value max-value]]- [:or [:< field min-value] [:> field max-value]])--(defmethod negate* :contains [clause] [:not clause])-(defmethod negate* :starts-with [clause] [:not clause])-(defmethod negate* :ends-with [clause] [:not clause])--(mu/defn negate-filter-clause :- ::mbql.s/Filter- "Return the logical compliment of an MBQL filter clause, generally without using `:not` (except for the string- filter clause types). Useful for generating highly optimized filter clauses and for drivers that do not support- top-level `:not` filter clauses.-- Deprecated: use [[metabase.lib.core/negate-boolean-expression]] going forward."- {:deprecated "0.57.0"}- [filter-clause :- ::mbql.s/Filter]- #_{:clj-kondo/ignore [:deprecated-var]}- (-> filter-clause desugar-filter-clause negate* simplify-compound-filter))-(mu/defn query->source-table-id :- [:maybe ::lib.schema.id/table]"Return the source Table ID associated with `query`, if applicable; handles nested queries as well. If `query` is`nil`, returns `nil`.@@ -709,34 +231,3 @@(do(log/warnf "%s is not a valid temporal unit for %s; not adding to clause %s" unit base-type (pr-str clause))clause)))--(defn referenced-field-ids- "Find all the `:field` references with integer IDs in `coll`, which can be a full MBQL query, a snippet of MBQL, or a- sequence of those things; return a set of Field IDs. Includes Fields referenced indirectly via `:source-field`.- Returns `nil` if no IDs are found.-- DEPRECATED: Use [[metabase.lib.core/all-field-ids]] going forward."- {:deprecated "0.57.0"}- [coll]- (not-empty- (into #{}- (comp cat (filter some?))- (match/match-many coll- [:field (id :guard integer?) opts]- [id (:source-field opts)]))))--(defn wrap-field-id-if-needed- "Wrap a raw Field ID in a `:field` clause if needed.-- DEPRECATED: this operates on legacy MBQL. Use Lib for MBQL manipulation going forward."- {:deprecated "0.57.0"}- [field-id-or-form]- (cond- (mbql-clause? field-id-or-form)- field-id-or-form-- (integer? field-id-or-form)- [:field field-id-or-form nil]-- :else- field-id-or-form))diff --git a/src/metabase/query_processor/middleware/wrap_value_literals.clj b/src/metabase/query_processor/middleware/wrap_value_literals.cljindex 8da92e2626e..faa8cea0fa1 100644--- a/src/metabase/query_processor/middleware/wrap_value_literals.clj+++ b/src/metabase/query_processor/middleware/wrap_value_literals.clj@@ -276,11 +276,12 @@;;;;;; Tangentially-related nonsense not used by the middleware;;;+;;; TODO (Cam 2026-05-14) -- move this into Lib or somewhere else since this is just a raw MBQL transformation that+;;; only seems to be used by driver-specific parameter compilation code(defn- type-info-no-query"This is like [[type-info*]] but specifically for supporting the legacy/deprecated [[wrap-value-literals-in-mbql]]function."- {:deprecated "0.57.0"}[clause](let [expr-type (lib.schema.expression/type-of-resolved clause)](merge@@ -306,15 +307,11 @@[:value {:base_type :type/Text, \"foo\":semantic_type nil,:database_type \"VARCHAR\",- :name \"description\"}]]]-- DEPRECATED: This is for legacy compatibility and should not be used in new code."- {:deprecated "0.57.0"}+ :name \"description\"}]]]"[mbql :- [:cat :keyword [:* :any]]](-> mbqllib/->mbql5(as-> $mbql (binding [*type-info* (fn [_query _path clause]- #_{:clj-kondo/ignore [:deprecated-var]}(type-info-no-query clause))](-> (lib.walk/walk-clauses*[$mbql]diff --git a/src/metabase/query_processor/parameters/dates.clj b/src/metabase/query_processor/parameters/dates.cljindex a69ff783100..4677afa601c 100644--- a/src/metabase/query_processor/parameters/dates.clj+++ b/src/metabase/query_processor/parameters/dates.clj@@ -411,18 +411,18 @@(parser-result-decoder parser-result decoder-param)))decoders))-(def ^:private TemporalUnit+(mr/def ::temporal-unit(into [:enum] u.date/add-units))-(def ^:private TemporalRange+(mr/def ::temporal-range[:map[:start {:optional true} (lib.schema.common/instance-of-class Temporal)][:end {:optional true} (lib.schema.common/instance-of-class Temporal)]- [:unit TemporalUnit]])+ [:unit ::temporal-unit]])-(mu/defn- adjust-inclusive-range-if-needed :- [:maybe TemporalRange]+(mu/defn- adjust-inclusive-range-if-needed :- [:maybe ::temporal-range]"Make an inclusive date range exclusive as needed."- [temporal-range :- [:maybe TemporalRange] {:keys [inclusive-start? inclusive-end?]}]+ [temporal-range :- [:maybe ::temporal-range] {:keys [inclusive-start? inclusive-end?]}](-> temporal-range(m/update-existing :start #(if inclusive-start?%diff --git a/test/metabase/driver/common/parameters/dates_test.clj b/test/metabase/driver/common/parameters/dates_test.cljdeleted file mode 100644index dd5d9019d8b..00000000000--- a/test/metabase/driver/common/parameters/dates_test.clj+++ /dev/null@@ -1,137 +0,0 @@-(ns metabase.driver.common.parameters.dates-test- (:require- [clojure.test :refer :all]- ^{:clj-kondo/ignore [:deprecated-namespace]} [metabase.driver.common.parameters.dates :as params.dates]))--(deftest ^:parallel date-string->filter-test- (testing "year and month"- (is (= [:between- [:field "field" {:base-type :type/DateTime, :temporal-unit :day}]- "2019-04-01"- "2019-04-30"]- (params.dates/date-string->filter "2019-04" [:field "field" {:base-type :type/DateTime}]))))- (testing "quarter year"- (is (= [:between- [:field "field" {:base-type :type/DateTime, :temporal-unit :day}]- "2019-04-01"- "2019-06-30"]- (params.dates/date-string->filter "Q2-2019" [:field "field" {:base-type :type/DateTime}]))))- (testing "single day"- (is (= [:=- [:field "field" {:base-type :type/DateTime, :temporal-unit :day}]- "2019-04-01"]- (params.dates/date-string->filter "2019-04-01" [:field "field" {:base-type :type/DateTime}]))))- (testing "single minute"- (is (= [:=- [:field "field" {:base-type :type/DateTime, :temporal-unit :minute}]- "2019-04-01T09:33:00"]- (params.dates/date-string->filter "2019-04-01T09:33:00" [:field "field" {:base-type :type/DateTime}]))))- (testing "day range"- (is (= [:between- [:field "field" {:base-type :type/DateTime, :temporal-unit :day}]- "2019-04-01"- "2019-04-03"]- (params.dates/date-string->filter "2019-04-01~2019-04-03" [:field "field" {:base-type :type/DateTime}]))))- (testing "datetime range"- (is (= [:between- [:field "field" {:base-type :type/DateTime, :temporal-unit :default}]- "2019-04-01T12:30:00"- "2019-04-03T16:30:00"]- (params.dates/date-string->filter "2019-04-01T12:30~2019-04-03T16:30" [:field "field" {:base-type :type/DateTime}]))))- (testing "after day"- (is (= [:>- [:field "field" {:base-type :type/DateTime, :temporal-unit :day}]- "2019-04-01"]- (params.dates/date-string->filter "2019-04-01~" [:field "field" {:base-type :type/DateTime}]))))- (testing "relative (past) exclusive"- (is (= [:time-interval- [:field "field" {:base-type :type/DateTime}]- -3- :day- {:include-current false}]- (params.dates/date-string->filter "past3days" [:field "field" {:base-type :type/DateTime}]))))- (testing "relative (past) inclusive"- (is (= [:time-interval [:field "field" {:base-type :type/DateTime}]- -3- :day- {:include-current true}]- (params.dates/date-string->filter "past3days~" [:field "field" {:base-type :type/DateTime}]))))- (testing "relative (next) exclusive"- (is (= [:time-interval [:field "field" {:base-type :type/DateTime}]- 3- :day- {:include-current false}]- (params.dates/date-string->filter "next3days" [:field "field" {:base-type :type/DateTime}]))))- (testing "relative (next) inclusive"- (is (= [:time-interval [:field "field" {:base-type :type/DateTime}]- 3- :day- {:include-current true}]- (params.dates/date-string->filter "next3days~" [:field "field" {:base-type :type/DateTime}]))))- (testing "quarters (#21083)"- (is (= [:time-interval [:field "field" {:base-type :type/DateTime}] -30 :quarter {:include-current false}]- (params.dates/date-string->filter "past30quarters" [:field "field" {:base-type :type/DateTime}]))))- (testing "relative (past) with starting from "- (is (= [:relative-time-interval- [:field "field" {:base-type :type/DateTime}]- -3- :day- -3- :year]- (params.dates/date-string->filter "past3days-from-3years" [:field "field" {:base-type :type/DateTime}]))))- (testing "relative (next) with starting from"- (is (= [:relative-time-interval- [:field "field" {:base-type :type/DateTime}]- 7- :hour- 13- :month]- (params.dates/date-string->filter "next7hours-from-13months" [:field "field" {:base-type :type/DateTime}]))))- (testing "exclusions"- (testing "hours"- (is (= [:!=- [:field "field" {:base-type :type/DateTime, :temporal-unit :hour-of-day}]- 0]- (params.dates/date-string->filter "exclude-hours-0" [:field "field" {:base-type :type/DateTime}])))- (is (= [:!=- [:field "field" {:base-type :type/DateTime, :temporal-unit :hour-of-day}]- 0- 23]- (params.dates/date-string->filter "exclude-hours-0-23" [:field "field" {:base-type :type/DateTime}])))- (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Don't know how to parse date string \"exclude-hours-\""- (params.dates/date-string->filter "exclude-hours-" [:field "field" {:base-type :type/DateTime}])))- (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Don't know how to parse date string \"exclude-hours-24-3\""- (params.dates/date-string->filter "exclude-hours-24-3" [:field "field" {:base-type :type/DateTime}]))))- (testing "quarters"- (is (= [:!=- [:field "field" {:base-type :type/DateTime, :temporal-unit :quarter-of-year}]- 1]- (params.dates/date-string->filter "exclude-quarters-1" [:field "field" {:base-type :type/DateTime}])))- (is (= [:!=- [:field "field" {:base-type :type/DateTime, :temporal-unit :quarter-of-year}]- 2- 3- 4]- (params.dates/date-string->filter "exclude-quarters-2-3-4" [:field "field" {:base-type :type/DateTime}])))- (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Don't know how to parse date string \"exclude-quarters-Q1\""- (params.dates/date-string->filter "exclude-quarters-Q1" [:field "field" {:base-type :type/DateTime}]))))- (testing "days (with default start-of-week = Sunday: Sun=1, Mon=2, Tue=3, Wed=4, Thu=5, Fri=6, Sat=7)"- (is (= [:!=- [:field "field" {:base-type :type/DateTime, :temporal-unit :day-of-week}]- 6- 3]- (params.dates/date-string->filter "exclude-days-Fri-Tue" [:field "field" {:base-type :type/DateTime}])))- (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Don't know how to parse date string \"exclude-days-Friday\""- (params.dates/date-string->filter "exclude-days-Friday" [:field "field" {:base-type :type/DateTime}]))))- (testing "months"- (is (= [:!=- [:field "field" {:base-type :type/DateTime, :temporal-unit :month-of-year}]- 12- 4- 9]- (params.dates/date-string->filter "exclude-months-Dec-Apr-Sep" [:field "field" {:base-type :type/DateTime}])))- (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Don't know how to parse date string \"exclude-months-April\""- (params.dates/date-string->filter "exclude-months-April" [:field "field" {:base-type :type/DateTime}]))))- (testing "minutes"- (is (thrown-with-msg? clojure.lang.ExceptionInfo #"Don't know how to parse date string \"exclude-minutes-15-30\""- (params.dates/date-string->filter "exclude-minutes-15-30" [:field "field" {:base-type :type/DateTime}]))))))diff --git a/test/metabase/driver/common/parameters/operators_test.clj b/test/metabase/driver/common/parameters/operators_test.cljdeleted file mode 100644index af42a40b7cf..00000000000--- a/test/metabase/driver/common/parameters/operators_test.clj+++ /dev/null@@ -1,123 +0,0 @@-(ns metabase.driver.common.parameters.operators-test- (:require- [clojure.test :refer :all]- ^{:clj-kondo/ignore [:deprecated-namespace]} [metabase.driver.common.parameters.operators :as params.ops]- [metabase.query-processor.error-type :as qp.error-type]))--(deftest ^:parallel to-clause-test- (testing "number operations"- (is (= [:= [:field 26 {:source-field 5}] 3]- (params.ops/to-clause {:type :number/=- :target [:dimension- [:field- 26- {:source-field 5}]]- :value [3]})))))--(deftest ^:parallel to-clause-test-2- (testing "number operations"- (is (= [:between [:field 26 {:source-field 5}] 3 9]- (params.ops/to-clause {:type :number/between- :target [:dimension- [:field- 26- {:source-field 5}]]- :value [3 9]})))))--(deftest ^:parallel to-clause-test-3- (testing "number operations"- (testing "equality is variadic"- (is (= [:= [:field 26 {:source-field 5}] 3 4 5]- (params.ops/to-clause {:type :number/=- :target [:dimension- [:field- 26- {:source-field 5}]]- :value [3 4 5]}))))))--(deftest ^:parallel to-clause-test-4- (testing "string operations"- (is (= [:starts-with [:field 26 {:source-field 5}] "foo"]- (params.ops/to-clause {:type :string/starts-with- :target [:dimension- [:field- 26- {:source-field 5}]]- :value ["foo"]})))))--(deftest ^:parallel to-clause-test-4b- (testing "string operations"- (testing "with options"- (is (= [:starts-with [:field 26 {:source-field 5}] "foo" {:case-insensitive false}]- (params.ops/to-clause {:type :string/starts-with- :target [:dimension- [:field- 26- {:source-field 5}]]- :value ["foo"]- :options {:case-insensitive false}}))))))--(deftest ^:parallel to-clause-test-4c- (testing "string operations"- (testing "with multiple arguments"- (is (= [:starts-with {} [:field 26 {:source-field 5}] "foo" "bar"]- (params.ops/to-clause {:type :string/starts-with- :target [:dimension- [:field- 26- {:source-field 5}]]- :value ["foo" "bar"]}))))))--(deftest ^:parallel to-clause-test-4d- (testing "string operations"- (testing "with multiple arguments"- (testing "with options"- (is (= [:starts-with {:case-insensitive false} [:field 26 {:source-field 5}] "foo" "bar"]- (params.ops/to-clause {:type :string/starts-with- :target [:dimension- [:field- 26- {:source-field 5}]]- :value ["foo" "bar"]- :options {:case-insensitive false}})))))))--(deftest ^:parallel to-clause-test-5- (testing "string operations"- (is (= [:does-not-contain [:field 26 {:source-field 5}] "foo"]- (params.ops/to-clause {:type :string/does-not-contain- :target [:dimension- [:field- 26- {:source-field 5}]]- :value ["foo"]})))))--(deftest ^:parallel to-clause-test-6- (testing "arity errors"- (letfn [(f [op values]- (try- (let [result (params.ops/to-clause {:type op- :target [:dimension- [:field- 26- {:source-field 5}]]- :value values})]- (is (not result) "Did not throw"))- (catch Exception e- (ex-data e))))]- (doseq [[op values] [[:number/>= [2 4]]- [:number/between [1 2 3]]]]- (is (=? {:param-type op- :param-value values- :field-id pos-int?- :type qp.error-type/invalid-parameter}- (f op values)))))))--(deftest ^:parallel to-clause-test-7- (testing "between normalization"- (are [values op] (= [op [:field 26 {:source-field 5}] 1]- (params.ops/to-clause {:type :number/between- :target [:dimension [:field 26 {:source-field 5}]]- :value values}))- [1] :>=- [1 nil] :>=- [nil 1] :<=)))diff --git a/test/metabase/driver/sql/parameters/substitution_test.clj b/test/metabase/driver/sql/parameters/substitution_test.cljindex 1f112e3de50..038639ea602 100644--- a/test/metabase/driver/sql/parameters/substitution_test.clj+++ b/test/metabase/driver/sql/parameters/substitution_test.clj@@ -16,13 +16,13 @@(set! *warn-on-reflection* true)-(deftest ^:parallel field->field-filter-clause-test+(deftest ^:parallel field->field-ref-test(is (=? [:field- (meta/id :venues :id){:base-type :type/BigInteger:metabase.query-processor.util.add-alias-info/source-table (meta/id :venues)- :metabase.driver.sql.parameters.substitution/compiling-field-filter? true}]- (#'sql.params.substitution/field->field-filter-clause+ :metabase.driver.sql.parameters.substitution/compiling-field-filter? true}+ (meta/id :venues :id)]+ (#'sql.params.substitution/field->field-ref:h2(meta/field-metadata :venues :id):number/=diff --git a/test/metabase/legacy_mbql/jvm_util_test.clj b/test/metabase/legacy_mbql/jvm_util_test.cljdeleted file mode 100644index ccdbdb7b080..00000000000--- a/test/metabase/legacy_mbql/jvm_util_test.clj+++ /dev/null@@ -1,20 +0,0 @@-(ns metabase.legacy-mbql.jvm-util-test- {:clj-kondo/config '{:linters {:deprecated-var {:level :off}}}}- (:require- [clojure.test :as t]- [metabase.legacy-mbql.util :as mbql.u]- [metabase.lib.filter.desugar.jvm :as lib.filter.desugar.jvm]))--(t/deftest ^:parallel desugar-host-and-domain-test- (t/is (= [:regex-match-first [:field 1 nil] (str @#'lib.filter.desugar.jvm/host-regex)]- (mbql.u/desugar-expression [:host [:field 1 nil]]))- "`host` should desugar to a `regex-match-first` clause with the host regex")- (t/is (= [:regex-match-first [:field 1 nil] (str @#'lib.filter.desugar.jvm/domain-regex)]- (mbql.u/desugar-expression [:domain [:field 1 nil]]))- "`domain` should desugar to a `regex-match-first` clause with the domain regex")- (t/is (= [:regex-match-first [:field 1 nil] (str @#'lib.filter.desugar.jvm/subdomain-regex)]- (mbql.u/desugar-expression [:subdomain [:field 1 nil]]))- "`subdomain` should desugar to a `regex-match-first` clause with the subdomain regex")- (t/is (= [:regex-match-first [:field 1 nil] (str @#'lib.filter.desugar.jvm/path-regex)]- (mbql.u/desugar-expression [:path [:field 1 nil]]))- "`path` should desugar to a `regex-match-first` clause with the path regex"))diff --git a/test/metabase/legacy_mbql/util_test.cljc b/test/metabase/legacy_mbql/util_test.cljcindex ee099c4c28c..869937c24aa 100644--- a/test/metabase/legacy_mbql/util_test.cljc+++ b/test/metabase/legacy_mbql/util_test.cljc@@ -12,588 +12,6 @@#?(:cljs (comment metabase.test-runner.assert-exprs.approximately-equal/keep-me))-(t/deftest ^:parallel simplify-compound-filter-test- (t/is (= [:= [:field 1 nil] 2]- (mbql.u/simplify-compound-filter [:and [:= [:field 1 nil] 2]]))- "can `simplify-compound-filter` fix `and` or `or` with only one arg?")- (t/is (= [:and- [:= [:field 1 nil] 2]- [:= [:field 3 nil] 4]- [:= [:field-id 5] 6]]- (mbql.u/simplify-compound-filter [:and- [:= [:field 1 nil] 2]- [:and- [:= [:field 3 nil] 4]- [:and- [:= [:field-id 5] 6]]]]))- "can `simplify-compound-filter` unnest nested `and`s or `or`s?")- (t/is (= [:and [:= [:field 1 nil] 2] [:= [:field 3 nil] 4]]- (mbql.u/simplify-compound-filter [:and [:= [:field 1 nil] 2] [:= [:field 3 nil] 4] [:= [:field 1 nil] 2]]))- "can `simplify-compound-filter` remove duplicates?")- (t/is (= [:= [:field 1 nil] 2]- (mbql.u/simplify-compound-filter [:not [:not [:= [:field 1 nil] 2]]]))- "can `simplify-compound-filter` eliminate `not` inside a `not`?")- (t/testing "removing empty/nil filter clauses"- (t/is (= nil- (mbql.u/simplify-compound-filter nil))- "does `simplify-compound-filter` return `nil` for empty filter clauses?")-- (t/is (= nil- (mbql.u/simplify-compound-filter [])))-- (t/is (= nil- (mbql.u/simplify-compound-filter [nil nil nil])))-- (t/is (= nil- (mbql.u/simplify-compound-filter [:and nil nil])))-- (t/is (= nil- (mbql.u/simplify-compound-filter [:and nil [:and nil nil nil] nil])))- (t/is (= [:= [:field 1 nil] 2]- (mbql.u/simplify-compound-filter [:and nil [:and nil [:= [:field 1 nil] 2] nil] nil])))- (t/is (= [:and- [:= [:field 1 nil] 2]- [:= [:field 3 nil] 4]- [:= [:field-id 5] 6]- [:= [:field-id 7] 8]- [:= [:field-id 9] 10]]- (mbql.u/simplify-compound-filter- [:and- nil- [:= [:field 1 nil] 2]- [:and- [:= [:field 3 nil] 4]]- nil- [:and- [:and- [:and- [:= [:field-id 5] 6]- nil- nil]- [:= [:field-id 7] 8]- [:= [:field-id 9] 10]]]]))))- (t/is (= {:aggregation [[:share [:and- [:= [:field 1 nil] 2]- [:= [:field 3 nil] 4]- [:= [:field-id 5] 6]- [:= [:field-id 7] 8]- [:= [:field-id 9] 10]]]]}- (mbql.u/simplify-compound-filter- {:aggregation [[:share [:and- nil- [:= [:field 1 nil] 2]- [:and- [:= [:field 3 nil] 4]]- nil- [:and- [:and- [:and- [:= [:field-id 5] 6]- nil- nil]- [:= [:field-id 7] 8]- [:= [:field-id 9] 10]]]]]]}))- "`simplify-compound-filter` should also work with more complex structures")- (t/testing "Check that `simplify-compound-filter` can apply de Morgan's law on `:not`"- (t/testing ":and clauses"- (t/is (= [:or- [:not [:= [:field 1 nil] 2]]- [:not [:= [:field 2 nil] 3]]]- (mbql.u/simplify-compound-filter [:not [:and- [:= [:field 1 nil] 2]- [:= [:field 2 nil] 3]]]))))- (t/testing ":or clauses"- (t/is (= [:and- [:not [:= [:field 1 nil] 2]]- [:not [:= [:field 2 nil] 3]]]- (mbql.u/simplify-compound-filter [:not [:or- [:= [:field 1 nil] 2]- [:= [:field 2 nil] 3]]]))- "Check that `simplify-compound-filter` can apply de Morgan's law on `:not` over `:or`")))- (t/testing "check that `simplify-compound-filter` doesn't remove `nil` from filters where it's being used as the value"- (t/is (= [:= [:field 1 nil] nil]- (mbql.u/simplify-compound-filter [:= [:field 1 nil] nil])))- (t/is (= [:= [:field 1 nil] nil]- (mbql.u/simplify-compound-filter [:and nil [:= [:field 1 nil] nil]])))))--(t/deftest ^:parallel combine-filter-clauses-test- (t/is (= [:and [:= [:field 1 nil] 100] [:= [:field 2 nil] 200]]- (mbql.u/combine-filter-clauses- [:= [:field 1 nil] 100]- [:= [:field 2 nil] 200]))- "Should be able to combine non-compound clauses")- (t/is (= [:and- [:= [:field 1 nil] 100]- [:= [:field 2 nil] 200]- [:= [:field 3 nil] 300]]- (mbql.u/combine-filter-clauses- [:= [:field 1 nil] 100]- [:and- [:= [:field 2 nil] 200]- [:= [:field 3 nil] 300]]))- "Should be able to combine into an exisiting compound clause")- (t/is (= [:and- [:= [:field 1 nil] 100]- [:= [:field 2 nil] 200]- [:= [:field 3 nil] 300]- [:= [:field 4 nil] 300]]- (mbql.u/combine-filter-clauses- [:and- [:= [:field 1 nil] 100]- [:= [:field 2 nil] 200]]- [:and- [:= [:field 3 nil] 300]- [:= [:field 4 nil] 300]]))- "Should be able to combine multiple compound clauses"))--(t/deftest ^:parallel desugar-time-interval-test- (t/is (= [:between- [:field 1 {:temporal-unit :month}]- [:relative-datetime 1 :month]- [:relative-datetime 2 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] 2 :month]))- "`time-interval` with value > 1 or < -1 should generate a `between` clause")- (t/is (= [:between- [:field 1 {:temporal-unit :month}]- [:relative-datetime 0 :month]- [:relative-datetime 2 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] 2 :month {:include-current true}]))- "test the `include-current` option -- interval should start or end at `0` instead of `1`")- (t/is (= [:=- [:field 1 {:temporal-unit :month}]- [:relative-datetime 1 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] 1 :month]))- "`time-interval` with value = 1 should generate an `=` clause")- (t/is (= [:=- [:field 1 {:temporal-unit :week}]- [:relative-datetime -1 :week]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] -1 :week]))- "`time-interval` with value = -1 should generate an `=` clause")- (t/testing "`include-current` option"- (t/is (= [:between- [:field 1 {:temporal-unit :month}]- [:relative-datetime 0 :month]- [:relative-datetime 1 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] 1 :month {:include-current true}]))- "interval with value = 1 should generate a `between` clause")- (t/is (= [:between- [:field 1 {:temporal-unit :day}]- [:relative-datetime -1 :day]- [:relative-datetime 0 :day]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] -1 :day {:include-current true}]))- "`include-current` option -- interval with value = 1 should generate a `between` clause"))- (t/is (= [:=- [:field 1 {:temporal-unit :week}]- [:relative-datetime 0 :week]]- (mbql.u/desugar-filter-clause [:time-interval [:field 1 nil] :current :week]))- "keywords like `:current` should work correctly"))--(t/deftest ^:parallel desugar-time-interval-with-expression-test- (t/is (= [:between- [:expression "CC" {:temporal-unit :month}]- [:relative-datetime 1 :month]- [:relative-datetime 2 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] 2 :month]))- "`time-interval` with value > 1 or < -1 should generate a `between` clause")- (t/is (= [:between- [:expression "CC" {:temporal-unit :month}]- [:relative-datetime 0 :month]- [:relative-datetime 2 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] 2 :month {:include-current true}]))- "test the `include-current` option -- interval should start or end at `0` instead of `1`")- (t/is (= [:=- [:expression "CC" {:temporal-unit :month}]- [:relative-datetime 1 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] 1 :month]))- "`time-interval` with value = 1 should generate an `=` clause")- (t/is (= [:=- [:expression "CC" {:temporal-unit :week}]- [:relative-datetime -1 :week]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] -1 :week]))- "`time-interval` with value = -1 should generate an `=` clause")- (t/testing "`include-current` option"- (t/is (= [:between- [:expression "CC" {:temporal-unit :month}]- [:relative-datetime 0 :month]- [:relative-datetime 1 :month]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] 1 :month {:include-current true}]))- "interval with value = 1 should generate a `between` clause")- (t/is (= [:between- [:expression "CC" {:temporal-unit :day}]- [:relative-datetime -1 :day]- [:relative-datetime 0 :day]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] -1 :day {:include-current true}]))- "`include-current` option -- interval with value = 1 should generate a `between` clause"))- (t/is (= [:=- [:expression "CC" {:temporal-unit :week}]- [:relative-datetime 0 :week]]- (mbql.u/desugar-filter-clause [:time-interval [:expression "CC"] :current :week]))- "keywords like `:current` should work correctly"))--(t/deftest ^:parallel desugar-relative-time-interval-negative-test- (t/testing "Desugaring relative-date-time produces expected [:and [:>=..] [:<..]] expression"- (let [value -10- bucket :day- offset-value -8- offset-bucket :week- exp-offset [:interval offset-value offset-bucket]]- (t/testing "expression reference is transformed correctly"- (let [expr-ref [:expression "cc"]]- (t/is (= [:and- [:>= expr-ref [:+ [:relative-datetime value bucket] exp-offset]]- [:< expr-ref [:+ [:relative-datetime 0 bucket] exp-offset]]]- (mbql.u/desugar-filter-clause- [:relative-time-interval expr-ref value bucket offset-value offset-bucket])))))- (t/testing "field reference is transformed correctly"- (let [field-ref [:field 100 nil]- exp-field-ref (update field-ref 2 assoc :temporal-unit :default)]- (t/is (= [:and- [:>= exp-field-ref [:+ [:relative-datetime value bucket] exp-offset]]- [:< exp-field-ref [:+ [:relative-datetime 0 bucket] exp-offset]]]- (mbql.u/desugar-filter-clause- [:relative-time-interval exp-field-ref value bucket offset-value offset-bucket]))))))))--(t/deftest ^:parallel desugar-during-test- (t/testing "Desugaring during filter produces expected [:and [:>=..] [:<..]] expression"- (let [value "2020-01-01T13:24:32"]- (doseq [{:keys [unit expected-lower expected-upper]}- [{:unit :second- :expected-lower "2020-01-01T13:24:32"- :expected-upper "2020-01-01T13:24:33"}- {:unit :minute- :expected-lower "2020-01-01T13:24"- :expected-upper "2020-01-01T13:25"}- {:unit :hour- :expected-lower "2020-01-01T13:00"- :expected-upper "2020-01-01T14:00"}- {:unit :day- :expected-lower "2020-01-01T00:00"- :expected-upper "2020-01-02T00:00"}- {:unit :month- :expected-lower "2020-01-01T00:00"- :expected-upper "2020-02-01T00:00"}- {:unit :year- :expected-lower "2020-01-01T00:00"- :expected-upper "2021-01-01T00:00"}]]- (t/testing (str "expression reference is transformed correctly for unit " unit)- (let [expr-ref [:expression "cc"]]- (t/is (= [:and- [:>= expr-ref expected-lower]- [:< expr-ref expected-upper]]- (mbql.u/desugar-filter-clause [:during expr-ref value unit])))))- (t/testing (str "field reference is transformed correctly for unit " unit)- (let [field-ref [:field 100 nil]- exp-field-ref (update field-ref 2 assoc :temporal-unit :default)]- (t/is (= [:and- [:>= exp-field-ref expected-lower]- [:< exp-field-ref expected-upper]]- (mbql.u/desugar-filter-clause [:during exp-field-ref value unit])))))))))--(t/deftest ^:parallel desugar-if-test- (t/testing "Desugaring if produces expected [:case ..] expression"- (t/is (= [:case [[[:< [:field 1 nil] 1] 2] [[:< [:field 3 nil] 4] 5]]]- (mbql.u/desugar-filter-clause [:if [[[:< [:field 1 nil] 1] 2] [[:< [:field 3 nil] 4] 5]]])))- (t/is (= [:case [[[:< [:field 1 nil] 1] 2]] {:default 3}]- (mbql.u/desugar-filter-clause [:if [[[:< [:field 1 nil] 1] 2]] {:default 3}])))))--(t/deftest ^:parallel desugar-in-test- (t/testing "Desugaring in and not-in produces expected [:= ..] and [:!= ..] expressions"- (t/are [expected clause] (= expected (mbql.u/desugar-filter-clause clause))- [:= [:field 1 nil] 2] [:in [:field 1 nil] 2]- [:= [:field 1 nil] [:field 2 nil]] [:in [:field 1 nil] [:field 2 nil]]- [:or [:= [:field 1 nil] 2] [:= [:field 1 nil] 3]] [:in [:field 1 nil] 2 3]- [:!= [:field 1 nil] 2] [:not-in [:field 1 nil] 2]- [:!= [:field 1 nil] [:field 2 nil]] [:not-in [:field 1 nil] [:field 2 nil]]- [:and [:!= [:field 1 nil] 2] [:!= [:field 1 nil] 3]] [:not-in [:field 1 nil] 2 3])))--(t/deftest ^:parallel desugar-relative-time-interval-positive-test- (t/testing "Desugaring relative-date-time produces expected [:and [:>=..] [:<..]] expression"- (let [value 10- bucket :day- offset-value 8- offset-bucket :week- exp-offset [:interval offset-value offset-bucket]]- (t/testing "expression reference is transformed correctly"- (let [expr-ref [:expression "cc"]]- (t/is (= [:and- [:>= expr-ref [:+ [:relative-datetime 1 bucket] exp-offset]]- [:< expr-ref [:+ [:relative-datetime (inc value) bucket] exp-offset]]]- (mbql.u/desugar-filter-clause- [:relative-time-interval expr-ref value bucket offset-value offset-bucket])))))- (t/testing "field reference is transformed correctly"- (let [field-ref [:field 100 nil]- exp-field-ref (update field-ref 2 assoc :temporal-unit :default)]- (t/is (= [:and- [:>= exp-field-ref [:+ [:relative-datetime 1 bucket] exp-offset]]- [:< exp-field-ref [:+ [:relative-datetime (inc value) bucket] exp-offset]]]- (mbql.u/desugar-filter-clause- [:relative-time-interval exp-field-ref value bucket offset-value offset-bucket]))))))))--(t/deftest ^:parallel desugar-relative-datetime-with-current-test- (t/testing "when comparing `:relative-datetime`to `:field`, it should take the temporal unit of the `:field`"- (t/is (= [:=- [:field 1 {:temporal-unit :minute}]- [:relative-datetime 0 :minute]]- (mbql.u/desugar-filter-clause- [:=- [:field 1 {:temporal-unit :minute}]- [:relative-datetime :current]]))))- (t/testing "otherwise it should just get a unit of `:default`"- (t/is (= [:=- [:field 1 nil]- [:relative-datetime 0 :default]]- (mbql.u/desugar-filter-clause- [:=- [:field 1 nil]- [:relative-datetime :current]]))))- (t/testing "we should be able to handle datetime fields even if they are nested inside another clause"- (t/is (= [:=- [:field 1 {:temporal-unit :week, :binning {:strategy :default}}]- [:relative-datetime 0 :week]]- (mbql.u/desugar-filter-clause- [:=- [:field 1 {:temporal-unit :week, :binning {:strategy :default}}]- [:relative-datetime :current]])))))--(t/deftest ^:parallel relative-datetime-current-inside-between-test- (t/testing ":relative-datetime should work inside a :between clause (#19606)\n"- (let [absolute "2022-03-11T15:48:00-08:00"- relative [:relative-datetime :current]- expected (fn [v unit]- (condp = v- absolute absolute- relative [:relative-datetime 0 unit]))]- (doseq [x [relative absolute]- y [relative absolute]- unit [:week :default]]- (t/testing (pr-str [:between [:field 1 {:temporal-unit unit}] x y])- (t/is (= [:between- [:field 1 {:temporal-unit unit}]- (expected x unit)- (expected y unit)]- (mbql.u/desugar-filter-clause- [:between- [:field 1 {:temporal-unit unit}]- x- y]))))))))--(t/deftest ^:parallel desugar-other-filter-clauses-test- (t/testing "desugaring := and :!= with extra args"- (t/is (= [:or- [:= [:field 1 nil] 2]- [:= [:field 1 nil] 3]- [:= [:field 1 nil] 4]- [:= [:field 1 nil] 5]]- (mbql.u/desugar-filter-clause [:= [:field 1 nil] 2 3 4 5]))- "= with extra args should get converted to or")- (t/is (= [:and- [:!= [:field 1 nil] 2]- [:!= [:field 1 nil] 3]- [:!= [:field 1 nil] 4]- [:!= [:field 1 nil] 5]]- (mbql.u/desugar-filter-clause [:!= [:field 1 nil] 2 3 4 5]))- "!= with extra args should get converted to or"))- (t/testing "desugaring :inside"- (t/is (= [:and- [:between [:field 1 nil] -10.0 10.0]- [:between [:field 2 nil] -20.0 20.0]]- (mbql.u/desugar-filter-clause [:inside [:field 1 nil] [:field 2 nil] 10.0 -20.0 -10.0 20.0]))))- (t/testing "desugaring :is-null"- (t/is (= [:= [:field 1 nil] nil]- (mbql.u/desugar-filter-clause [:is-null [:field 1 nil]]))))- (t/testing "desugaring :not-null"- (t/is (= [:!= [:field 1 nil] nil]- (mbql.u/desugar-filter-clause [:not-null [:field 1 nil]]))))- (t/testing "desugaring :is-empty of nil base-type"- (t/is (= [:= [:field 1 nil] nil]- (mbql.u/desugar-filter-clause [:is-empty [:field 1 nil]]))))- (t/testing "desugaring :is-empty of emptyable base-type :type/Text"- (t/is (= [:or- [:= [:field 1 {:base-type :type/Text}] nil]- [:= [:field 1 {:base-type :type/Text}] ""]]- (mbql.u/desugar-filter-clause [:is-empty [:field 1 {:base-type :type/Text}]]))))- (t/testing "desugaring :is-empty of string expression #41265"- (t/is (= [:or- [:= [:regex-match-first "foo" "bar"] nil]- [:= [:regex-match-first "foo" "bar"] ""]]- (mbql.u/desugar-filter-clause [:is-empty [:regex-match-first "foo" "bar"]]))))- (t/testing "desugaring :is-empty of not emptyable base-type :type/DateTime"- (t/is (= [:= [:field 1 {:base-type :type/DateTime}] nil]- (mbql.u/desugar-filter-clause [:is-empty [:field 1 {:base-type :type/DateTime}]]))))- (t/testing "desugaring :is-empty of :type/PostgresEnum #48022"- (t/is (= [:= [:field 1 {:base-type :type/PostgresEnum}] nil]- (mbql.u/desugar-filter-clause [:is-empty [:field 1 {:base-type :type/PostgresEnum}]]))))- (t/testing "desugaring :not-empty of nil base-type"- (t/is (= [:!= [:field 1 nil] nil]- (mbql.u/desugar-filter-clause [:not-empty [:field 1 nil]]))))- (t/testing "desugaring :not-empty of emptyable base-type :type/Text"- (t/is (= [:and- [:!= [:field 1 {:base-type :type/Text}] nil]- [:!= [:field 1 {:base-type :type/Text}] ""]]- (mbql.u/desugar-filter-clause [:not-empty [:field 1 {:base-type :type/Text}]]))))- (t/testing "desugaring :not-empty of string expression #41265"- (t/is (= [:and- [:!= [:regex-match-first "foo" "bar"] nil]- [:!= [:regex-match-first "foo" "bar"] ""]]- (mbql.u/desugar-filter-clause [:not-empty [:regex-match-first "foo" "bar"]]))))- (t/testing "desugaring :not-empty of not emptyable base-type"- (t/is (= [:!= [:field 1 {:base-type :type/DateTime}] nil]- (mbql.u/desugar-filter-clause [:not-empty [:field 1 {:base-type :type/DateTime}]]))))- (t/testing "desugaring :not-empty of :type/PostgresEnum #48022"- (t/is (= [:!= [:field 1 {:base-type :type/PostgresEnum}] nil] (mbql.u/desugar-filter-clause [:not-empty [:field 1 {:base-type :type/PostgresEnum}]])))))--(t/deftest ^:parallel desugar-does-not-contain-test- (t/testing "desugaring does-not-contain"- (t/testing "without options"- (t/is (= [:not [:contains [:field 1 nil] "ABC"]]- (mbql.u/desugar-filter-clause [:does-not-contain [:field 1 nil] "ABC"]))))- (t/testing "*with* options"- (t/is (= [:not [:contains [:field 1 nil] "ABC" {:case-sensitive false}]]- (mbql.u/desugar-filter-clause [:does-not-contain [:field 1 nil] "ABC" {:case-sensitive false}]))))- (t/testing "desugaring does-not-contain with multiple arguments"- (t/testing "without options"- (t/is (= [:and- [:not [:contains [:field 1 nil] "ABC"]]- [:not [:contains [:field 1 nil] "XYZ"]]]- (mbql.u/desugar-filter-clause [:does-not-contain {} [:field 1 nil] "ABC" "XYZ"])))- (t/is (= [:and- [:not [:contains [:field 1 nil] "ABC"]]- [:not [:contains [:field 1 nil] "XYZ"]]- [:not [:contains [:field 1 nil] "LMN"]]]- (mbql.u/desugar-filter-clause [:does-not-contain {} [:field 1 nil] "ABC" "XYZ" "LMN"]))))- (t/testing "*with* options"- (t/is (= [:and- [:not [:contains [:field 1 nil] "ABC" {:case-sensitive false}]]- [:not [:contains [:field 1 nil] "XYZ" {:case-sensitive false}]]]- (mbql.u/desugar-filter-clause- [:does-not-contain {:case-sensitive false} [:field 1 nil] "ABC" "XYZ"])))- (t/is (= [:and- [:not [:contains [:field 1 nil] "ABC" {:case-sensitive false}]]- [:not [:contains [:field 1 nil] "XYZ" {:case-sensitive false}]]- [:not [:contains [:field 1 nil] "LMN" {:case-sensitive false}]]]- (mbql.u/desugar-filter-clause- [:does-not-contain {:case-sensitive false} [:field 1 nil] "ABC" "XYZ" "LMN"])))))))--(t/deftest ^:parallel desugar-temporal-extract-test- (t/testing "desugaring :get-year, :get-month, etc"- (doseq [[[op mode] unit] @#'mbql.u/temporal-extract-ops->unit]- (t/is (= [:temporal-extract [:field 1 nil] unit]- (#'mbql.u/desugar-temporal-extract [op [:field 1 nil] mode])))-- (t/is (= [:+ [:temporal-extract [:field 1 nil] unit] 1]- (#'mbql.u/desugar-temporal-extract [:+ [op [:field 1 nil] mode] 1]))))))--(t/deftest ^:parallel desugar-divide-with-extra-args-test- (t/testing `mbql.u/desugar-expression- (t/are [expression expected] (= expected- (mbql.u/desugar-expression expression))- [:/ 1 2] [:/ 1 2]- [:/ 1 2 3] [:/ [:/ 1 2] 3]- [:/ 1 2 3 4] [:/ [:/ [:/ 1 2] 3] 4]))- (t/testing `mbql.u/desugar-filter-clause- (t/are [expression expected] (= expected- (mbql.u/desugar-filter-clause expression))- [:= 1 [:/ 1 2]] [:= 1 [:/ 1 2]]- [:= 1 [:/ 1 2 3]] [:= 1 [:/ [:/ 1 2] 3]]- [:= 1 [:/ 1 2 3 4]] [:= 1 [:/ [:/ [:/ 1 2] 3] 4]])))--(t/deftest ^:parallel negate-simple-filter-clause-test- (t/testing :=- (t/is (= [:!= [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:= [:field 1 nil] 10]))))- (t/testing :!=- (t/is (= [:= [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:!= [:field 1 nil] 10]))))- (t/testing :>- (t/is (= [:<= [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:> [:field 1 nil] 10]))))- (t/testing :<- (t/is (= [:>= [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:< [:field 1 nil] 10]))))- (t/testing :>=- (t/is (= [:< [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:>= [:field 1 nil] 10]))))- (t/testing :<=- (t/is (= [:> [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:<= [:field 1 nil] 10]))))- (t/testing :between- (t/is (= [:or- [:< [:field 1 nil] 10]- [:> [:field 1 nil] 20]]- (mbql.u/negate-filter-clause [:between [:field 1 nil] 10 20]))))- (t/testing :contains- (t/is (= [:not [:contains [:field 1 nil] "ABC"]]- (mbql.u/negate-filter-clause [:contains [:field 1 nil] "ABC"]))))- (t/testing :starts-with- (t/is (= [:not [:starts-with [:field 1 nil] "ABC"]]- (mbql.u/negate-filter-clause [:starts-with [:field 1 nil] "ABC"]))))- (t/testing :ends-with- (t/is (= [:not [:ends-with [:field 1 nil] "ABC"]]- (mbql.u/negate-filter-clause [:ends-with [:field 1 nil] "ABC"])))))--(t/deftest ^:parallel negate-compund-filter-clause-test- (t/testing :not- (t/is (= [:= [:field 1 nil] 10]- (mbql.u/negate-filter-clause [:not [:= [:field 1 nil] 10]]))- "negating `:not` should simply unwrap the clause"))- (t/testing :and- (t/is (= [:or- [:!= [:field 1 nil] 10]- [:!= [:field 2 nil] 20]]- (mbql.u/negate-filter-clause- [:and- [:= [:field 1 nil] 10]- [:= [:field 2 nil] 20]]))))- (t/testing :or- (t/is (= [:and- [:= [:field 1 nil] 10]- [:= [:field 2 nil] 20]]- (mbql.u/negate-filter-clause- [:or- [:!= [:field 1 nil] 10]- [:!= [:field 2 nil] 20]])))))--(t/deftest ^:parallel negate-syntactic-sugar-filter-clause-test- (t/testing "= with extra args"- (t/is (= [:and- [:!= [:field 1 nil] 10]- [:!= [:field 1 nil] 20]- [:!= [:field 1 nil] 30]]- (mbql.u/negate-filter-clause [:= [:field 1 nil] 10 20 30]))))- (t/testing "!= with extra args"- (t/is (= [:or- [:= [:field 1 nil] 10]- [:= [:field 1 nil] 20]- [:= [:field 1 nil] 30]]- (mbql.u/negate-filter-clause [:!= [:field 1 nil] 10 20 30]))))- (t/testing :time-interval- (t/is (= [:!=- [:field 1 {:temporal-unit :week}]- [:relative-datetime 0 :week]]- (mbql.u/negate-filter-clause [:time-interval [:field 1 nil] :current :week]))))-- (t/testing :time-interval- (t/is (= [:!=- [:expression "CC" {:temporal-unit :week}]- [:relative-datetime 0 :week]]- (mbql.u/negate-filter-clause [:time-interval [:expression "CC"] :current :week]))))-- (t/testing :is-null- (t/is (= [:!= [:field 1 nil] nil]- (mbql.u/negate-filter-clause [:is-null [:field 1 nil]]))))-- (t/testing :not-null- (t/is (= [:= [:field 1 nil] nil]- (mbql.u/negate-filter-clause [:not-null [:field 1 nil]]))))- (t/testing :inside- (t/is (= [:or- [:< [:field 1 nil] -10.0]- [:> [:field 1 nil] 10.0]- [:< [:field 2 nil] -20.0]- [:> [:field 2 nil] 20.0]]- (mbql.u/negate-filter-clause- [:inside [:field 1 nil] [:field 2 nil] 10.0 -20.0 -10.0 20.0])))))-(def ^:private query-with-some-nesting{:database 1:type :query@@ -741,94 +159,7 @@(t/is (= [:field "t" {:base-type :type/Date}](mbql.u/with-temporal-unit [:field "t" {:base-type :type/Date}] :minute)))))-(t/deftest ^:parallel desugar-time-interval-expression-test- (t/is (= [:=- [:expression "Date" {:temporal-unit :quarter}]- [:relative-datetime 0 :quarter]]- (#'mbql.u/desugar-time-interval [:time-interval [:expression "Date"] :current :quarter]))))--(t/deftest ^:parallel desugar-month-quarter-day-name-test- (t/is (= [:case [[[:= [:field 1 nil] 1] "Jan"]- [[:= [:field 1 nil] 2] "Feb"]- [[:= [:field 1 nil] 3] "Mar"]- [[:= [:field 1 nil] 4] "Apr"]- [[:= [:field 1 nil] 5] "May"]- [[:= [:field 1 nil] 6] "Jun"]- [[:= [:field 1 nil] 7] "Jul"]- [[:= [:field 1 nil] 8] "Aug"]- [[:= [:field 1 nil] 9] "Sep"]- [[:= [:field 1 nil] 10] "Oct"]- [[:= [:field 1 nil] 11] "Nov"]- [[:= [:field 1 nil] 12] "Dec"]]- {:default ""}]- (mbql.u/desugar-expression [:month-name [:field 1 nil]]))- "`month-name` should desugar to a `:case` clause with values for each month")- (t/is (= [:case [[[:= [:field 1 nil] 1] "Q1"]- [[:= [:field 1 nil] 2] "Q2"]- [[:= [:field 1 nil] 3] "Q3"]- [[:= [:field 1 nil] 4] "Q4"]]- {:default ""}]- (mbql.u/desugar-expression [:quarter-name [:field 1 nil]]))- "`quarter-name` should desugar to a `:case` clause with values for each quarter")- (t/is (= [:case [[[:= [:field 1 nil] 1] "Sunday"]- [[:= [:field 1 nil] 2] "Monday"]- [[:= [:field 1 nil] 3] "Tuesday"]- [[:= [:field 1 nil] 4] "Wednesday"]- [[:= [:field 1 nil] 5] "Thursday"]- [[:= [:field 1 nil] 6] "Friday"]- [[:= [:field 1 nil] 7] "Saturday"]]- {:default ""}]- (mbql.u/desugar-expression [:day-name [:field 1 nil]]))- "`day-name` should desugar to a `:case` clause with values for each weekday"))--#?(:clj- (t/deftest ^:synchronized desugar-month-quarter-day-name-i18n-test- (metabase.test.util.i18n/with-user-locale "es"- ;; JVM versions 17 and older for some languages (including Spanish) use eg. "oct.", while in JVMs 18+ they- ;; use "oct". I wish I were joking, but I'm not. These tests were passing on 21 and failing on 17 and 11- ;; before I made them flexible about the dot.- (t/is (=? [:case [[[:= [:field 1 nil] 1] #(#{"ene" "ene."} %)]- [[:= [:field 1 nil] 2] #(#{"feb" "feb."} %)]- [[:= [:field 1 nil] 3] #(#{"mar" "mar."} %)]- [[:= [:field 1 nil] 4] #(#{"abr" "abr."} %)]- [[:= [:field 1 nil] 5] #(#{"may" "may."} %)]- [[:= [:field 1 nil] 6] #(#{"jun" "jun."} %)]- [[:= [:field 1 nil] 7] #(#{"jul" "jul."} %)]- [[:= [:field 1 nil] 8] #(#{"ago" "ago."} %)]- [[:= [:field 1 nil] 9] #(#{"sept" "sept."} %)]- [[:= [:field 1 nil] 10] #(#{"oct" "oct."} %)]- [[:= [:field 1 nil] 11] #(#{"nov" "nov."} %)]- [[:= [:field 1 nil] 12] #(#{"dic" "dic."} %)]]- {:default ""}]- (mbql.u/desugar-expression [:month-name [:field 1 nil]]))- "`month-name` should desugar to a `:case` clause with values for each month")- (t/is (= [:case [[[:= [:field 1 nil] 1] "Q1"]- [[:= [:field 1 nil] 2] "Q2"]- [[:= [:field 1 nil] 3] "Q3"]- [[:= [:field 1 nil] 4] "Q4"]]- {:default ""}]- (mbql.u/desugar-expression [:quarter-name [:field 1 nil]]))- "`quarter-name` should desugar to a `:case` clause with values for each quarter")- (t/is (= [:case [[[:= [:field 1 nil] 1] "domingo"]- [[:= [:field 1 nil] 2] "lunes"]- [[:= [:field 1 nil] 3] "martes"]- [[:= [:field 1 nil] 4] "miércoles"]- [[:= [:field 1 nil] 5] "jueves"]- [[:= [:field 1 nil] 6] "viernes"]- [[:= [:field 1 nil] 7] "sábado"]]- {:default ""}]- (mbql.u/desugar-expression [:day-name [:field 1 nil]]))- "`day-name` should desugar to a `:case` clause with values for each weekday"))))-(t/deftest ^:parallel normalize-token-handle-types-test(t/testing "If this gets called incorrectly with a base type keyword then handle it gracefully"(t/is (= :type/Text(mbql.u/normalize-token "type/Text")))))--(t/deftest ^:parallel wrap-field-id-if-needed-test- (doseq [[x expected] {10 [:field 10 nil]- [:field 10 nil] [:field 10 nil]- [:field "name" {:base-type :type/Text}] [:field "name" {:base-type :type/Text}]}]- (t/testing x- (t/is (= expected- (mbql.u/wrap-field-id-if-needed x))))))