Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

[QP] Properly escape SQL `LIKE` metachars for `:contains` and friends (#67665)

Braden Shepherdson3fcac725b21d
diff --git a/docs/developers-guide/driver-changelog.md b/docs/developers-guide/driver-changelog.md
index e21324c7c7f..bd6856199cb 100644
--- a/docs/developers-guide/driver-changelog.md
+++ b/docs/developers-guide/driver-changelog.md
@@ -18,6 +18,17 @@ title: Driver interface changelog
- `grant-workspace-read-access!` - Grant read access on specified tables to a workspace's isolated user.
- `check-isolation-permissions` - Test whether the database connection has sufficient permissions.
+- Added support for escaping `LIKE` metacharacters to `:sql` driver's handling of `LIKE` clauses, which are used to
+ implement the `:starts-with`, `:ends-with` and `:contains` filters. The default implementation uses backslashes to
+ escape backslashes, `%` and `_`, the `LIKE` metacharacters, when the RHS of one of these filters is a literal string.
+ Drivers which can handle `x LIKE y ESCAPE '\'` should just work. If a different way of escaping is needed, you can
+ override the new multimethod `metabase.driver.sql.query-processor/escape-like-pattern`; see `:sqlserver` which uses
+ `[%]` regex character classes. For drivers which already have backslash as the `ESCAPE` default, or don't support
+ that `ESCAPE` syntax, override the new multimethod `metabase.driver.sql.query-processor/transform-literal-like-pattern-honeysql`,
+ or as a shortcut if the override is an identity function, add the abstract driver
+ `:metabase.driver.sql.query-processor.like-escape-char-built-in/like-escape-char-built-in` as a parent of your driver.
+ See `metabase.driver.mysql` for an example of using the abstract driver.
+
## Metabase 0.58.0
- Added a `:collate` feature for drivers that support collation settings on text fields
diff --git a/modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj b/modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj
index 6472c192838..47fe881028e 100644
--- a/modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj
+++ b/modules/drivers/bigquery-cloud-sdk/src/metabase/driver/bigquery_cloud_sdk.clj
@@ -17,6 +17,7 @@
[metabase.driver.sql-jdbc.sync.describe-database :as sql-jdbc.describe-database]
[metabase.driver.sql.normalize :as driver.sql.normalize]
[metabase.driver.sql.query-processor :as sql.qp]
+ [metabase.driver.sql.query-processor.like-escape-char-built-in :as-alias like-escape-char-built-in]
[metabase.driver.sql.util :as sql.u]
[metabase.driver.sync :as driver.s]
[metabase.driver.util :as driver.u]
@@ -72,7 +73,8 @@
(set! *warn-on-reflection* true)
-(driver/register! :bigquery-cloud-sdk, :parent :sql)
+(driver/register! :bigquery-cloud-sdk, :parent #{:sql
+ ::like-escape-char-built-in/like-escape-char-built-in})
;;; +----------------------------------------------------------------------------------------------------------------+
;;; | Client |
diff --git a/modules/drivers/redshift/src/metabase/driver/redshift.clj b/modules/drivers/redshift/src/metabase/driver/redshift.clj
index ee62f01d805..d50103a0f84 100644
--- a/modules/drivers/redshift/src/metabase/driver/redshift.clj
+++ b/modules/drivers/redshift/src/metabase/driver/redshift.clj
@@ -16,6 +16,7 @@
[metabase.driver.sql-jdbc.sync :as sql-jdbc.sync]
[metabase.driver.sql-jdbc.sync.describe-database :as sql-jdbc.describe-database]
[metabase.driver.sql.query-processor :as sql.qp]
+ [metabase.driver.sql.query-processor.like-escape-char-built-in :as-alias like-escape-char-built-in]
[metabase.driver.sync :as driver.s]
[metabase.util :as u]
[metabase.util.date-2 :as u.date]
@@ -35,7 +36,8 @@
(set! *warn-on-reflection* true)
-(driver/register! :redshift, :parent #{:postgres})
+(driver/register! :redshift, :parent #{:postgres
+ ::like-escape-char-built-in/like-escape-char-built-in})
(doseq [[feature supported?] {:atomic-renames true
:connection-impersonation true
diff --git a/modules/drivers/sparksql/src/metabase/driver/sparksql.clj b/modules/drivers/sparksql/src/metabase/driver/sparksql.clj
index 47365478118..848ec408398 100644
--- a/modules/drivers/sparksql/src/metabase/driver/sparksql.clj
+++ b/modules/drivers/sparksql/src/metabase/driver/sparksql.clj
@@ -16,6 +16,7 @@
[metabase.driver.sql-jdbc.sync :as sql-jdbc.sync]
[metabase.driver.sql.parameters.substitution :as sql.params.substitution]
[metabase.driver.sql.query-processor :as sql.qp]
+ [metabase.driver.sql.query-processor.like-escape-char-built-in :as-alias sql.qp.like-built-in]
[metabase.driver.sql.util :as sql.u]
[metabase.util.honey-sql-2 :as h2x]
[metabase.util.performance :refer [select-keys every? empty? not-empty get-in]])
@@ -24,7 +25,7 @@
(set! *warn-on-reflection* true)
-(driver/register! :sparksql, :parent :hive-like)
+(driver/register! :sparksql, :parent #{:hive-like ::sql.qp.like-built-in/like-escape-char-built-in})
;;; ------------------------------------------ Custom HoneySQL Clause Impls ------------------------------------------
diff --git a/modules/drivers/sqlserver/src/metabase/driver/sqlserver.clj b/modules/drivers/sqlserver/src/metabase/driver/sqlserver.clj
index 99210c54bd3..c730dbe9165 100644
--- a/modules/drivers/sqlserver/src/metabase/driver/sqlserver.clj
+++ b/modules/drivers/sqlserver/src/metabase/driver/sqlserver.clj
@@ -20,6 +20,7 @@
[metabase.driver.sql.parameters.substitution :as sql.params.substitution]
[metabase.driver.sql.query-processor :as sql.qp]
[metabase.driver.sql.query-processor.boolean-to-comparison :as sql.qp.boolean-to-comparison]
+ [metabase.driver.sql.query-processor.like-escape-char-built-in :as like-escape-char-built-in]
[metabase.driver.sql.query-processor.util :as sql.qp.u]
[metabase.driver.sql.util :as sql.u]
[metabase.driver.util :as driver.u]
@@ -43,7 +44,7 @@
(set! *warn-on-reflection* true)
-(driver/register! :sqlserver, :parent #{:sql-jdbc})
+(driver/register! :sqlserver, :parent #{:sql-jdbc ::like-escape-char-built-in/like-escape-char-built-in})
(doseq [[feature supported?] {:case-sensitivity-string-filter-options false
:connection-impersonation true
@@ -633,6 +634,15 @@
(->> (update query :filter sql.qp.boolean-to-comparison/boolean->comparison)
(parent-method driver :filter honeysql-form))))
+;; SQL Server doesn't like backslashes as the escape character for `LIKE` clauses. Use character classes instead to
+;; escape the `LIKE` metacharacters `%` and `_`.
+(defmethod sql.qp/escape-like-pattern :sqlserver
+ [_driver like-pattern]
+ (-> like-pattern
+ (str/replace "\\" "[\\]")
+ (str/replace "%" "[%]")
+ (str/replace "_" "[_]")))
+
;; SQLServer doesn't support `TRUE`/`FALSE`; it uses `1`/`0`, respectively; convert these booleans to numbers.
(defmethod sql.qp/->honeysql [:sqlserver Boolean]
[_ bool]
diff --git a/src/metabase/driver/h2.clj b/src/metabase/driver/h2.clj
index 41b8c194a25..08ea620ca1f 100644
--- a/src/metabase/driver/h2.clj
+++ b/src/metabase/driver/h2.clj
@@ -19,6 +19,7 @@
[metabase.driver.sql-jdbc.sync :as sql-jdbc.sync]
[metabase.driver.sql.normalize :as sql.normalize]
[metabase.driver.sql.query-processor :as sql.qp]
+ [metabase.driver.sql.query-processor.like-escape-char-built-in :as like-escape-char-built-in]
[metabase.driver.util :as driver.u]
[metabase.lib.schema.id :as lib.schema.id]
[metabase.lib.schema.metadata :as lib.schema.metadata]
@@ -39,7 +40,7 @@
;; method impls live in this namespace
(comment h2.actions/keep-me)
-(driver/register! :h2, :parent :sql-jdbc)
+(driver/register! :h2, :parent #{:sql-jdbc ::like-escape-char-built-in/like-escape-char-built-in})
;;; this will prevent the H2 driver from showing up in the list of options when adding a new Database.
(defmethod driver/superseded-by :h2 [_driver] :deprecated)
diff --git a/src/metabase/driver/mysql.clj b/src/metabase/driver/mysql.clj
index ba9e27f05d1..b0a69e73c15 100644
--- a/src/metabase/driver/mysql.clj
+++ b/src/metabase/driver/mysql.clj
@@ -22,6 +22,7 @@
[metabase.driver.sql-jdbc.quoting :refer [quote-columns]]
[metabase.driver.sql-jdbc.sync :as sql-jdbc.sync]
[metabase.driver.sql.query-processor :as sql.qp]
+ [metabase.driver.sql.query-processor.like-escape-char-built-in :as like-escape-char-built-in]
[metabase.driver.sql.query-processor.util :as sql.qp.u]
[metabase.driver.sql.util :as sql.u]
[metabase.driver.util :as driver.u]
@@ -56,7 +57,7 @@
mysql.actions/keep-me
mysql.ddl/keep-me)
-(driver/register! :mysql, :parent :sql-jdbc)
+(driver/register! :mysql, :parent #{:sql-jdbc ::like-escape-char-built-in/like-escape-char-built-in})
(def ^:private ^:const min-supported-mysql-version 5.7)
(def ^:private ^:const min-supported-mariadb-version 10.2)
diff --git a/src/metabase/driver/sql/query_processor.clj b/src/metabase/driver/sql/query_processor.clj
index 233b4d09a84..28eb1dbb9c2 100644
--- a/src/metabase/driver/sql/query_processor.clj
+++ b/src/metabase/driver/sql/query_processor.clj
@@ -1552,6 +1552,42 @@
[:fn {:error/message "string value"} #(string? (second %))]]
driver-api/mbql.schema.FieldOrExpressionDef])
+(defmulti escape-like-pattern
+ "Handle escaping a literal string into a `LIKE` clause pattern which will match literally.
+
+ In particular, any `%` or `_` (the standard metacharacters) should be escaped. In some engines, that's done by a
+ preceding backslash `\\_`; in others by wrapping it in a character class `[%]`.
+
+ The default implementation for `:sql` uses backslashes, which appears to be the most common approach. Most SQL-based
+ drivers should not need to implement this."
+ {:added "0.59.0" :arglists '([driver like-pattern])}
+ driver/dispatch-on-initialized-driver
+ :hierarchy #'driver/hierarchy)
+
+(defmethod escape-like-pattern :sql [_driver like-pattern]
+ (-> like-pattern
+ (str/replace "\\" "\\\\")
+ (str/replace "_" "\\_")
+ (str/replace "%" "\\%")))
+
+(defmulti transform-literal-like-pattern-honeysql
+ "When the RHS of a `LIKE` clause is a literal pattern, some drivers require extra HoneySQL clauses, such as
+ `:escape`, added to the expression.
+
+ This driver multimethod allows SQL drivers to adjust these clauses for any MBQL expression that uses `LIKE` under
+ the hood (eg. `:starts-with`, `:contains`, `:ends-with`), with a single, simpler override.
+
+ The default is to add `ESCAPE '\\'`, adopting a common default. This is compatible with the default
+ [[escape-like-pattern]]. Drivers which have an `ESCAPE` character built in can skip editing this clause by inheriting
+ from the `:metabase.driver.sql.query-processor.like-escape-char-built-in/like-escape-char-built-in` abstract driver."
+ {:added "0.59.0" :arglists '([driver like-rhs-honeysql])}
+ driver/dispatch-on-initialized-driver
+ :hierarchy #'driver/hierarchy)
+
+(defmethod transform-literal-like-pattern-honeysql :sql
+ [_driver like-rhs-honeysql]
+ [:escape like-rhs-honeysql [:inline "\\"]])
+
(mu/defn- generate-pattern
"Generate pattern to match against in like clause. Lowercasing for case insensitive matching also happens here."
[driver
@@ -1560,8 +1596,10 @@
post
{:keys [case-sensitive] :or {case-sensitive true} :as _options}]
(if (= :value type)
- (->honeysql driver (update arg 1 #(cond-> (str pre % post)
- (not case-sensitive) u/lower-case-en)))
+ (->> (update arg 1 #(cond-> (str pre (escape-like-pattern driver %) post)
+ (not case-sensitive) u/lower-case-en))
+ (->honeysql driver)
+ (transform-literal-like-pattern-honeysql driver))
(let [expr (->honeysql driver (into [:concat] (remove nil?) [pre arg post]))]
(if case-sensitive
expr
diff --git a/src/metabase/driver/sql/query_processor/like_escape_char_built_in.clj b/src/metabase/driver/sql/query_processor/like_escape_char_built_in.clj
new file mode 100644
index 00000000000..b1f7c558f99
--- /dev/null
+++ b/src/metabase/driver/sql/query_processor/like_escape_char_built_in.clj
@@ -0,0 +1,19 @@
+(ns metabase.driver.sql.query-processor.like-escape-char-built-in
+ "In MySQL and some other databases, `LIKE` clause RHS patterns have an escape character defined by default and have
+ awkward double-escaping issues with setting a `LIKE pattern ESCAPE '\\'` to the same character.
+
+ This abstract driver overrides [[sql.qp/transform-literal-like-pattern-honeysql]] to an identity function, overriding
+ the `:default` behaviour of specifying the `ESCAPE '\\'` clause as the SQL standard suggests.
+
+ See #67667 for more context."
+ (:require
+ [metabase.driver :as driver]
+ [metabase.driver.sql.query-processor :as sql.qp]))
+
+(driver/register! ::like-escape-char-built-in, :abstract? true)
+
+(defmethod sql.qp/transform-literal-like-pattern-honeysql ::like-escape-char-built-in
+ [_driver like-rhs-honeysql]
+ like-rhs-honeysql)
+
+(prefer-method sql.qp/transform-literal-like-pattern-honeysql ::like-escape-char-built-in :sql)
diff --git a/test/metabase/query_processor/filter_test.clj b/test/metabase/query_processor/filter_test.clj
index f58a48b6f24..51c5886f777 100644
--- a/test/metabase/query_processor/filter_test.clj
+++ b/test/metabase/query_processor/filter_test.clj
@@ -395,6 +395,52 @@
:order-by [[:asc $id]]
:limit 3})))))))
+(mt/defdataset strings-that-need-escaping
+ [["escaping_table"
+ [{:field-name "string1", :base-type :type/Text}]
+ [["\\Backslash"]
+ ["Back\\slash"]
+ ["Backslash\\"]
+ ["_Underscore"]
+ ["Under_score"]
+ ["Underscore_"]
+ ["%ile"]
+ ["per%cent"]
+ ["100%"]
+ ["_many%\\escapes"]
+ ["more%___%esca\\pes%"]
+ ["%backslash\\%before_percent_"]
+ ["\\backslash\\_before%underscore\\"]]]])
+
+;; TODO: (Braden 2026-01-08) This set can be expanded further!
+;; - :athena, :vertica, :clickhouse, :presto all support these clauses but have issues loading the test dataset.
+;; - :mongo supports `:starts-with` etc. but is not SQL; its regex-based fuzzy matching could be upgraded.
+(doseq [driver #{:h2 :sqlserver :mysql :maria :postgres :oracle :sqlite}]
+ (defmethod driver/database-supports? [driver :test/escaping-like-metacharacters]
+ [_driver _feature _database]
+ true))
+
+(defmethod driver/database-supports? [:default :test/escaping-like-metacharacters]
+ [_driver _feature _database]
+ false)
+
+(deftest ^:parallel starts-with-escaped-char-in-literal-test
+ (mt/test-drivers (mt/normal-drivers-with-feature :test/escaping-like-metacharacters)
+ (mt/dataset strings-that-need-escaping
+ (doseq [[lit exp] {"\\" [[1 "\\Backslash"]
+ [13 "\\backslash\\_before%underscore\\"]]
+ "_" [[4 "_Underscore"]
+ [10 "_many%\\escapes"]]
+ "%" [[7 "%ile"]
+ [12 "%backslash\\%before_percent_"]]}]
+ (testing (str "literal argument starts with " lit)
+ (is (=? exp
+ (mt/formatted-rows
+ [int identity]
+ (mt/run-mbql-query escaping_table
+ {:filter [:starts-with $string1 lit]
+ :limit 3})))))))))
+
;;; --------------------------------------------------- ends-with ----------------------------------------------------
(deftest ^:parallel ends-with-test
@@ -473,6 +519,23 @@
:order-by [[:asc $id]]
:limit 3})))))))
+(deftest ^:parallel ends-with-escaped-char-in-literal-test
+ (mt/test-drivers (mt/normal-drivers-with-feature :test/escaping-like-metacharacters)
+ (mt/dataset strings-that-need-escaping
+ (doseq [[lit exp] {"\\" [[3 "Backslash\\"]
+ [13 "\\backslash\\_before%underscore\\"]]
+ "_" [[6 "Underscore_"]
+ [12 "%backslash\\%before_percent_"]]
+ "%" [[9 "100%"]
+ [11 "more%___%esca\\pes%"]]}]
+ (testing (str "literal argument ends with " lit)
+ (is (=? exp
+ (mt/formatted-rows
+ [int identity]
+ (mt/run-mbql-query escaping_table
+ {:filter [:ends-with $string1 lit]
+ :limit 3})))))))))
+
;;; ---------------------------------------------------- contains ----------------------------------------------------
(deftest ^:parallel contains-test
@@ -547,6 +610,39 @@
:order-by [[:asc $id]]
:limit 3})))))))
+(deftest ^:parallel contains-with-escaped-char-in-literal-test
+ (mt/test-drivers (mt/normal-drivers-with-feature :test/escaping-like-metacharacters)
+ (mt/dataset strings-that-need-escaping
+ (doseq [[lit exp] {"\\" [[1 "\\Backslash"]
+ [2 "Back\\slash"]
+ [3 "Backslash\\"]
+ [10 "_many%\\escapes"]
+ [11 "more%___%esca\\pes%"]
+ [12 "%backslash\\%before_percent_"]
+ [13 "\\backslash\\_before%underscore\\"]]
+ "_" [[4 "_Underscore"]
+ [5 "Under_score"]
+ [6 "Underscore_"]
+ [10 "_many%\\escapes"]
+ [11 "more%___%esca\\pes%"]
+ [12 "%backslash\\%before_percent_"]
+ [13 "\\backslash\\_before%underscore\\"]]
+ "%" [[7 "%ile"]
+ [8 "per%cent"]
+ [9 "100%"]
+ [10 "_many%\\escapes"]
+ [11 "more%___%esca\\pes%"]
+ [12 "%backslash\\%before_percent_"]
+ [13 "\\backslash\\_before%underscore\\"]]}]
+ (testing (str "literal argument contains " lit)
+ (is (=? exp
+ (mt/formatted-rows
+ [int identity]
+ (mt/run-mbql-query escaping_table
+ {:filter [:contains $string1 lit]
+ :order-by [[:asc $id]]
+ :limit 8})))))))))
+
;;; +----------------------------------------------------------------------------------------------------------------+
;;; | NESTED AND/OR CLAUSES |
;;; +----------------------------------------------------------------------------------------------------------------+