The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
[MBQL lib] Add user docs for filter functions in `lib.core` (#71035)
Braden Shepherdson38b071e39d11
diff --git a/src/metabase/lib/core.cljc b/src/metabase/lib/core.cljcindex eac7738baa7..5c7a0c4b69d 100644--- a/src/metabase/lib/core.cljc+++ b/src/metabase/lib/core.cljc@@ -633,6 +633,134 @@[lib.breakoutbreakouts-metadata])+;;; ## Filters+;;; Filters are boolean expressions based on a row of a query. A stage of a query only returns (or aggregates) those+;;; rows for which the filters are `true`. They correspond to a SQL `WHERE` clause, of course.+;;;+;;; MBQL has a list of filter expressions, and they are implicitly `AND`ed together. The order of filters does not+;;; matter.+;;;+;;; Note that filters are *pre-aggregation* within their stage. If you want to filter based on the results of an+;;; aggregation, such a showing only those months with 1000+ orders, you need to [[append-stage]] and add the filter+;;; to the stage *after* the aggregations. (SQL has the `HAVING` clause for this, but there's no equivalent in MBQL.)++(mu/defn filter :- ::lib.schema/query+ "Adds the given `boolean-expression` to the target stage of `a-query`.++ If `boolean-expression` is an exact duplicate of an existing filter, this silently does nothing.++ **Code Health:** Healthy. This is a core API."+ ([a-query boolean-expression] (filter a-query -1 boolean-expression))+ ([a-query :- ::lib.schema/query+ stage-number :- :int+ boolean-expression :- some?]+ (lib.filter/filter a-query stage-number boolean-expression)))++(mu/defn filters :- [:maybe [:ref ::lib.schema/filters]]+ "Returns the list of filters on the target stage of `a-query`, or nil if there are none.++ **Code Health:** Healthy. This is a core API."+ ([a-query] (filters a-query -1))+ ([a-query :- ::lib.schema/query+ stage-number :- :int]+ (lib.filter/filters a-query stage-number)))++(mu/defn filterable-columns :- [:maybe [:sequential ::lib.schema.metadata/column]]+ "Returns column metadata for all the columns that could be used for filtering on the target stage of `a-query`.++ The columns which can be used for filters are, in this order:++ 1. Columns from the *source* - a table, a card or the previous stage.+ 2. *Expressions* on this stage of the query.+ 3. Columns from explicit joins on this stage.+ 4. Columns which can be *implicitly joined* via any FKs in the above columns.++ Supports an optional third argument `options`, which can have these options:++ - `:include-sensitive-fields? true` will return fields with `:visibility-type :sensitive`. (Default false.)"+ ([a-query] (filterable-columns a-query -1))+ ([a-query stage-number] (filterable-columns a-query stage-number nil))+ ([a-query :- ::lib.schema/query+ stage-number :- :int+ options :- [:maybe [:map [:include-sensitive-fields? :boolean]]]]+ (lib.filter/filterable-columns a-query stage-number options)))++;;; ### Building filters in code+;;; These functions build filter clauses in code. They're mainly used in tests, but they are a legitimate public API+;;; to use from production code too.+;;;+;;; **Code Health:** Healthy. However, UIs should generally be using [[filter-parts]] and [[string-filter-clause]]+;;; instead of calling these.+(shared.ns/import-fns+ [lib.filter+ and+ or+ not+ = !=+ < <=+ > >=+ in not-in+ between+ inside+ is-null not-null+ is-empty not-empty+ starts-with ends-with+ contains does-not-contain+ relative-time-interval+ time-interval+ segment]) ; TODO: Move segment to sit with the `lib.segment` functions.++;;; # Working with Expressions as an AST+;;; Several parts of the FE want to work with clauses in a "white box" way, so it can render the details of the+;;; expression in a structured way, or convert the AST of the expression parser into an expression, aggregation or+;;; filter.+;;;+;;; That requirement is **directly opposed to one of Lib's primary objectives:** to encapsulate the details of MBQL+;;; queries and clauses.+;;;+;;; The solution to the dilemma is for Lib to publish a separate, non-MBQL syntax for expression ASTs, and to convert+;;; between that and MBQL clauses. That is the purpose of the [[expression-parts]] and [[filter-parts]] systems.+;;;+;;; [[expression-parts]] turns an MBQL expression (or aggregation) clause into an AST built out of nested maps. The+;;; only MBQL inside the AST is that MBQL *refs* are treated as opaque leaf nodes. See [[lib.fe-util/ExpressionParts]]+;;; for details on the schema.+;;;+;;; [[expression-clause]] is the reverse direction: turning an `ExpressionParts` AST into an MBQL clause suitable to+;;; pass to [[expression]] or [[aggregate]].++;;; ## Filter Parts+;;; In a similar vein to [[expression-parts]], [[filter-parts]] breaks down a filter expression to make it consumable+;;; without exposing MBQL internals. Filters are simpler than expressions, since they are never nested and have at most+;;; one column. See [[lib.filter/FilterParts]] for the output format.++(mu/defn filter-parts :- ::lib.filter/filter-parts+ "Exports an opaque MBQL filter clause as a transparent `::lib.filter/filter-parts` map.++ `a-query` and the target stage are required for context, since the output contains complete column metadata.++ See [[describe-filter-operator]] to get a human-readable, translated description of the filter operator keywords.++ **Code Health:** Healthy. This is deliberately public, including the output format."+ ([a-query a-filter-clause] (filter-parts a-query -1 a-filter-clause))+ ([a-query :- ::lib.schema/query+ stage-number :- :int+ a-filter-clause :- ::lib.schema.expression/boolean]+ (lib.filter/filter-parts a-query stage-number a-filter-clause)))++(mu/defn describe-filter-operator :- :string+ "Returns a human-readable, translated display name for a filter operation based on the `operator` keyword.+ (For example, the `:operator` field of a [[filter-parts]] map.)++ `variant` is optional, and controls how a few operators are displayed. Valid variants:++ - `:default` (the default, naturally) uses \"Is (not)\", \"Greater than\" and \"Less than\"+ - `:number` uses \"Equal to\" and \"Not equal to\"+ - `:temporal` uses \"After\" and \"Before\" for `:>` and `:<`"+ ([operator] (describe-filter-operator operator :default))+ ([operator :- :keyword ; TODO: (bshepherdson 2026-03-06) This could be more specific, if the list were in a schema.+ variant :- [:enum :default :number :temporal]]+ (lib.filter/describe-filter-operator operator variant)))+;;; # IGNORE ME!;;; <img src="https://i.redd.it/1b9z1mv805e61.jpg" />;;; *These are the leftovers which are not properly wrapped in user documentation yet.*@@ -713,29 +841,7 @@with-fields][metabase.lib.field.utilupdate-keys-for-col-from-previous-stage]- [lib.filter- add-filter-to-stage- filter- filters- filterable-columns- filter-parts- describe-filter-operator- and- or- not- = !=- < <=- > >=- in not-in- between- inside- is-null not-null- is-empty not-empty- starts-with ends-with- contains does-not-contain- relative-time-interval- time-interval- segment]+[metabase.lib.filter.desugardesugar-filter-clause][metabase.lib.filter.negatediff --git a/src/metabase/lib/filter.cljc b/src/metabase/lib/filter.cljcindex 2491a52c02e..44f3f3755a6 100644--- a/src/metabase/lib/filter.cljc+++ b/src/metabase/lib/filter.cljc@@ -25,6 +25,7 @@[metabase.util :as u][metabase.util.i18n :as i18n][metabase.util.malli :as mu]+ [metabase.util.malli.registry :as mr][metabase.util.number :as u.number][metabase.util.performance :as perf :refer [every? some mapv empty? #?(:clj doseq) #?(:clj for)]][metabase.util.time :as u.time]))@@ -541,7 +542,7 @@(lib.options/ensure-uuid (into [tag {} (lib.common/->op-arg column)](map lib.common/->op-arg args)))))-(def ^:private FilterParts+(mr/def ::filter-parts[:map[:lib/type [:= :mbql/filter-parts]][:operator :keyword]@@ -549,7 +550,7 @@[:column [:maybe [:ref ::lib.schema.metadata/column]]][:args [:sequential :any]]])-(mu/defn filter-parts :- FilterParts+(mu/defn filter-parts :- ::filter-parts"Return the parts of the filter clause `a-filter-clause` in query `query` at stage `stage-number`.Might obsolete [[filter-operator]]."([query a-filter-clause]diff --git a/src/metabase/query_processor/middleware/parameters/mbql.clj b/src/metabase/query_processor/middleware/parameters/mbql.cljindex 09f21940a47..caf7b59242d 100644--- a/src/metabase/query_processor/middleware/parameters/mbql.clj+++ b/src/metabase/query_processor/middleware/parameters/mbql.clj@@ -3,6 +3,7 @@(:refer-clojure :exclude [every?])(:require[metabase.lib.core :as lib]+ [metabase.lib.filter :as lib.filter][metabase.lib.metadata :as lib.metadata][metabase.lib.schema :as lib.schema][metabase.lib.schema.common :as lib.schema.common]@@ -179,5 +180,5 @@:else(let [filter-clause (or (build-filter-clause query stage-path (assoc param :value param-value))(log/warnf "build-filter-clause did not return a valid clause for param %s" (pr-str param)))- stage' (lib/add-filter-to-stage stage filter-clause)]+ stage' (lib.filter/add-filter-to-stage stage filter-clause)](recur stage' more-params))))))