The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Point a dangling aggregation ref at the stage's only aggregation(#85123)
lbrdnk92f8c25fbb0b
diff --git a/src/metabase/agent_lib/representations/repair.clj b/src/metabase/agent_lib/representations/repair.cljindex 5df978cfd57..ebc767e9bb9 100644--- a/src/metabase/agent_lib/representations/repair.clj+++ b/src/metabase/agent_lib/representations/repair.clj@@ -916,8 +916,8 @@;;;;;; `metabase.metabot.agent.links/->frontend-query` converts a structured query that does not;;; normalize to valid MBQL 5 to legacy MBQL. If that conversion fails too, as it does for a query-;;; stripped of its `:lib/*` keys whose aggregation ref no longer resolves, the raw MBQL 5 query goes-;;; out instead. In legacy MBQL, JSON turns the unit into a string, and the legacy schema cannot+;;; stripped of its `:lib/*` keys whose aggregation ref points at one of several aggregations, the+;;; raw MBQL 5 query goes out instead. In legacy MBQL, JSON turns the unit into a string, and the legacy schema cannot;;; decode it for `absolute-datetime` / `during`, so the frontend drops the clause (BOT-1972). A;;; `temporal-unit` on a ref decodes correctly (`legacy-json-round-trip-temporal-filters-test`). The;;; query builder also writes the hoisted form.diff --git a/src/metabase/metabot/agent/links.clj b/src/metabase/metabot/agent/links.cljindex 7bc6583e5d4..4094afaff89 100644--- a/src/metabase/metabot/agent/links.clj+++ b/src/metabase/metabot/agent/links.clj@@ -7,6 +7,7 @@[metabase.lib.core :as lib][metabase.lib.schema][metabase.measures.db :as measures.db]+ [metabase.metabot.query-repair :as query-repair][metabase.segments.db :as segments.db][metabase.system.core :as system][metabase.util.json :as json]@@ -38,20 +39,16 @@[query]#_{:clj-kondo/ignore [:discouraged-var]}(if (and (map? query)- ;; `:stages` alone (no `:lib/type`) also counts: a query round-tripped through- ;; the frontend's viewing context (`user_is_viewing` / `chart_configs`) comes- ;; back MBQL 5 but stripped of its internal `:lib/*` keys. Guarding on- ;; `:lib/type` alone let such a query fall through unconverted, leaking raw- ;; MBQL 5 into the `/question#<base64>` hash and crashing the frontend with- ;; "Stage 0 does not exist" (BOT-1604 follow-up).+ ;; `:stages` alone (no `:lib/type`) also counts: a query stripped of its internal `:lib/*` keys is+ ;; still MBQL 5. Guarding on `:lib/type` alone let such a query fall through unconverted, leaking raw+ ;; MBQL 5 into the `/question#<base64>` hash and crashing the frontend with "Stage 0 does not exist"+ ;; (BOT-1604 follow-up).(or (:lib/type query) (:stages query)))(try- (lib/->legacy-MBQL (lib/normalize :metabase.lib.schema/query query))+ (lib/->legacy-MBQL (query-repair/normalize-query query))(catch Exception e- ;; Normalizing a `:lib/*`-stripped query can mint fresh `:lib/uuid`s that don't- ;; match positional aggregation/expression refs embedded elsewhere in the query- ;; (e.g. an order-by on the query's own aggregation), which fails conversion.- ;; Fall back to the raw query rather than failing the whole agent turn over a link.+ ;; A ref that [[query-repair/normalize-query]] cannot repoint, such as one of several aggregations in a+ ;; `:lib/*`-stripped query, fails conversion. Fall back to the raw query rather than failing the whole agent turn over a link.(log/warn e "Failed to convert MBQL 5 query to legacy MBQL for link resolution")query))query))@@ -64,12 +61,14 @@Native-only queries use legacy MBQL because the embedding SDK opens its SQL editor only when`dataset_query.type` is `\"native\"`.- A query with `:stages` but no `:lib/type` stays in MBQL 5 if it normalizes to a valid query.- All other values go through [[->legacy-mbql]]."+ A query with `:stages` but no `:lib/type` stays in MBQL 5 if it normalizes to a valid query; see+ [[query-repair/normalize-query]] for the refs it repairs on the way. Conversation state repairs such a query as it+ is read, so this is the last guard, for a query that reaches a link by another way. All other values go through+ [[->legacy-mbql]]."[query](or (when (and (map? query) (or (:lib/type query) (:stages query)))(try- (let [normalized (lib/normalize :metabase.lib.schema/query query)]+ (let [normalized (query-repair/normalize-query query)](when (and (not (lib/native-only-query? normalized))(or (:lib/type query) (mr/validate :metabase.lib.schema/query normalized)))(cond-> (lib/prepare-for-serialization normalized)diff --git a/src/metabase/metabot/query_repair.clj b/src/metabase/metabot/query_repair.cljnew file mode 100644index 00000000000..1155af5179f--- /dev/null+++ b/src/metabase/metabot/query_repair.clj@@ -0,0 +1,77 @@+(ns metabase.metabot.query-repair+ "Repair of structured queries that lost their `:lib/*` keys.++ A query in conversation state persisted before the viewing context kept its `:lib/*` keys (BOT-2410), or one a+ client sends without them, has no `:lib/type` and no `:lib/uuid`s. Normalization gives each aggregation a fresh+ `:lib/uuid`, so a ref to an aggregation, such as an order-by on it, names a uuid that no aggregation carries. When+ the stage has one aggregation the ref can only mean that one, and it is pointed there.++ [[repair-stripped-query]] runs on every query of conversation state as it is normalized, so every tool reads a+ valid MBQL 5 query. [[normalize-query]] is the same repair for a query on its way to the frontend, where the link+ path keeps its own fallback."+ (:require+ [metabase.lib.core :as lib]+ [metabase.lib.schema :as lib.schema]+ [metabase.util.log :as log]+ [metabase.util.malli.registry :as mr]))++(set! *warn-on-reflection* true)++(declare repoint-dangling-aggregation-refs-in-stage)++(defn- repoint-in-joins+ [stage]+ (cond-> stage+ (sequential? (:joins stage))+ (update :joins (partial mapv (fn [join]+ (cond-> join+ (sequential? (:stages join))+ (update :stages (partial mapv repoint-dangling-aggregation-refs-in-stage))))))))++(defn- repoint-dangling-aggregation-refs-in-stage+ "Point every `[:aggregation opts uuid]` ref in `stage` that names none of the stage's aggregations at the stage's+ only aggregation, then do the same in the stages of its joins. With several aggregations the target is unknown, so+ the refs stay as they are.++ [[lib.schema/ref-errors-for-stage]] finds the dangling refs, with the same rule the query schema validates by. It+ skips `:joins`, which are walked here, and namespaced keys."+ [stage]+ (let [uuids (into #{} (map (comp :lib/uuid second)) (:aggregation stage))+ stage (if (= 1 (count uuids))+ (let [sole (first uuids)]+ (reduce (fn [stage [path clause]]+ (cond-> stage+ (and (= :aggregation (first clause))+ (= 3 (count clause)))+ (assoc-in (conj path 2) sole)))+ stage+ (lib.schema/ref-errors-for-stage stage)))+ stage)]+ (repoint-in-joins stage)))++(defn normalize-query+ "Normalize `query` as MBQL 5 and repoint its dangling aggregation refs; see+ [[repoint-dangling-aggregation-refs-in-stage]]. Throws when `query` does not normalize. The result is not+ validated: a ref that could mean any of several aggregations stays dangling."+ [query]+ (let [normalized (lib/normalize ::lib.schema/query query)]+ (cond-> normalized+ (vector? (:stages normalized))+ (update :stages (partial mapv repoint-dangling-aggregation-refs-in-stage)))))++(defn repair-stripped-query+ "Repair `query` for conversation state when it is a structured query without its `:lib/type`: normalize it and+ repoint its dangling aggregation refs. Any other value is returned unchanged, and so is a query whose repair is+ not a valid query, or does not normalize at all: state must read even when it holds a query no tool can run, and+ the tools' own guards word the refusal."+ [query]+ (if (and (map? query) (:stages query) (not (:lib/type query)))+ (try+ (let [repaired (normalize-query query)]+ (if (mr/validate ::lib.schema/query repaired)+ repaired+ query))+ (catch Exception e+ (log/warn e "Failed to repair a query without :lib/type in conversation state")+ query))+ query))diff --git a/src/metabase/metabot/schema.clj b/src/metabase/metabot/schema.cljindex 8c79fe6bb50..1222087ec33 100644--- a/src/metabase/metabot/schema.clj+++ b/src/metabase/metabot/schema.clj@@ -5,6 +5,7 @@[metabase.lib.schema.common :as lib.schema.common][metabase.lib.schema.id :as lib.schema.id][metabase.lib.util :as lib.util]+ [metabase.metabot.query-repair :as query-repair][metabase.metabot.schema.v2 :as schema.v2][metabase.util :as u][metabase.util.malli.registry :as mr]@@ -34,8 +35,11 @@[:string {:decode/normalize lib.schema.common/normalize-string-key}])(mr/def ::query- "An MBQL 5 or legacy query in any state of normalization, as it round-trips through tool-call JSON or persisted turn state."- ::lib.util/query-like)+ "An MBQL 5 or legacy query in any state of normalization, as it round-trips through tool-call JSON or persisted turn+ state. Normalizing repairs a structured query that lost its `:lib/*` keys, so every tool reads a valid MBQL 5 query;+ see [[query-repair/repair-stripped-query]]."+ [:schema {:decode/normalize query-repair/repair-stripped-query}+ ::lib.util/query-like])(mr/def ::todo"One todo item of persisted turn state; only `:id` is guaranteed, since state can hold a partial itemdiff --git a/test/metabase/metabot/agent/links_test.clj b/test/metabase/metabot/agent/links_test.cljindex 2d96c589ad4..c4ec8c28613 100644--- a/test/metabase/metabot/agent/links_test.clj+++ b/test/metabase/metabot/agent/links_test.clj@@ -7,12 +7,14 @@[metabase.lib.test-metadata :as meta][metabase.lib.test-util :as lib.tu][metabase.metabot.agent.links :as links]+ [metabase.metabot.query-repair-test :as query-repair-test][metabase.metabot.test-util :as test-util][metabase.metabot.tools.charts :as charts][metabase.metabot.tools.shared :as shared][metabase.test :as mt][metabase.util :as u]- [metabase.util.json :as json]))+ [metabase.util.json :as json]+ [metabase.util.malli.registry :as mr]))(defn- decode-question-url"Decode a `/question#<base64>` URL into its JSON-decoded pseudo-card map."@@ -42,19 +44,8 @@:base-type :type/DateTime}]]}}(links/->legacy-mbql rehydrated))))))-(defn- strip-lib-keys- "Recursively remove all namespaced `:lib/*` keys from a query, mimicking how a- pMBQL query that round-trips through the frontend's viewing context comes back:- structurally pMBQL (`:stages`, pMBQL-order field clauses, `:effective-type`s)- but stripped of its internal `:lib/type`, `:lib/uuid`, and `:lib/metadata`."- [x]- (cond- (map? x) (into {} (keep (fn [[k v]]- (when-not (and (keyword? k) (= "lib" (namespace k)))- [k (strip-lib-keys v)])))- x)- (sequential? x) (mapv strip-lib-keys x)- :else x))+(def ^:private strip-lib-keys query-repair-test/strip-lib-keys)+(def ^:private stripped-aggregated-query query-repair-test/stripped-aggregated-query)(deftest ^:parallel ->legacy-mbql-lib-type-less-pmbql-test(testing "converts pMBQL stripped of :lib/* keys (frontend viewing-context round-trip) to legacy MBQL"@@ -74,17 +65,19 @@(is (map? (:query legacy)))(is (some? (:database legacy)))))))-(deftest ^:parallel ->legacy-mbql-stripped-query-with-aggregation-ref-falls-back-test- (testing "falls back to the raw query instead of throwing when the stripped query's positional aggregation ref can't be resolved after normalize"- ;; Regression: `strip-lib-keys` drops the aggregation's `:lib/uuid`, but an- ;; `[:aggregation {} <uuid>]` ref elsewhere in the query (e.g. an order-by on the- ;; query's own aggregation) still points at the old uuid. `normalize` mints a fresh- ;; uuid, the ref lookup misses, and conversion throws — this used to take down the- ;; whole agent turn instead of just producing a broken link.- (let [q (-> (lib.tu/venues-query) (lib/aggregate (lib/count)))- q2 (lib/order-by q (lib/aggregation-ref q 0) :desc)- stripped (strip-lib-keys q2)]+(deftest ^:parallel ->legacy-mbql-stripped-query-with-aggregation-ref-test+ (testing "points the stripped query's aggregation ref at the stage's only aggregation, then converts (BOT-2410)"+ ;; `normalize` mints a fresh `:lib/uuid` for the aggregation, so the ref in the order-by names a uuid that no+ ;; aggregation carries. With one aggregation the ref can only mean that one.+ (let [stripped (stripped-aggregated-query (lib/count))+ legacy (links/->legacy-mbql stripped)](is (not (contains? stripped :lib/type)))+ (is (= :query (:type legacy)))+ (is (= [[:desc [:aggregation 0]]] (-> legacy :query :order-by)))))+ (testing "falls back to the raw query instead of throwing when the ref could mean any of several aggregations"+ ;; Conversion throws on the dangling ref. This used to take down the whole agent turn instead of just+ ;; producing a broken link.+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))](is (= stripped (links/->legacy-mbql stripped))))))(deftest ^:parallel pseudo-card->link-keeps-mbql5-temporal-filters-test@@ -115,12 +108,25 @@(is (= (strip (lib/filters query))(strip (-> dataset-query lib/normalize :stages first :filters)))))))))))-(deftest ^:parallel ->frontend-query-stripped-query-with-aggregation-ref-goes-out-raw-test- (testing "a query without :lib/* keys goes out as the raw MBQL 5 query when normalization invalidates its aggregation ref"- ;; Normalization assigns a new UUID to the aggregation, so the aggregation ref no longer resolves,- ;; and ->legacy-mbql cannot convert the query either.- (let [q (-> (lib.tu/venues-query) (lib/aggregate (lib/count)))- stripped (strip-lib-keys (lib/order-by q (lib/aggregation-ref q 0) :desc))]+(deftest ^:parallel ->frontend-query-stripped-query-with-aggregation-ref-test+ (testing "a query without :lib/* keys goes out as valid MBQL 5 with its aggregation ref pointed at the stage's only aggregation (BOT-2410)"+ (let [sent (links/->frontend-query (stripped-aggregated-query (lib/count)))+ stage (-> sent :stages first)]+ (is (= :mbql/query (:lib/type sent)))+ (is (mr/validate :metabase.lib.schema/query sent))+ (is (= (-> stage :aggregation first second :lib/uuid)+ (-> stage :order-by first (nth 2) (nth 2))))))+ (testing "a dangling ref in the stage of a join is repaired the same way"+ (let [sent (links/->frontend-query (query-repair-test/stripped-query-with-aggregated-join))+ join-stage (-> sent :stages first :joins first :stages first)]+ (is (= :mbql/query (:lib/type sent)))+ (is (mr/validate :metabase.lib.schema/query sent))+ (is (= (-> join-stage :aggregation first second :lib/uuid)+ (-> join-stage :order-by first (nth 2) (nth 2))))))+ (testing "goes out as the raw MBQL 5 query when the ref could mean any of several aggregations"+ ;; Normalization gives each aggregation a new uuid, the ref resolves to none of them, and ->legacy-mbql cannot+ ;; convert the query either.+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))](is (= stripped (links/->frontend-query stripped))))))(deftest ^:parallel ->frontend-query-sends-native-queries-as-legacy-testdiff --git a/test/metabase/metabot/agent/memory_test.clj b/test/metabase/metabot/agent/memory_test.cljindex 10cfe92a06c..623d2143a32 100644--- a/test/metabase/metabot/agent/memory_test.clj+++ b/test/metabase/metabot/agent/memory_test.clj@@ -2,8 +2,11 @@(:require[clojure.string :as str][clojure.test :refer :all]+ [metabase.lib.core :as lib]+ [metabase.lib.test-metadata :as meta][metabase.metabot.agent.links :as links][metabase.metabot.agent.memory :as memory]+ [metabase.metabot.query-repair-test :as query-repair-test][metabase.metabot.schema :as metabot.schema][metabase.metabot.tools.shared :as shared][metabase.util.malli.registry :as mr]))@@ -22,6 +25,24 @@(is (= {:queries {} :charts {} :todos [] :link-registry {}}(memory/get-state mem))))))+(deftest ^:parallel initialize-repairs-stripped-queries-test+ (testing "a structured query without its :lib/* keys is read as valid MBQL 5, wherever state holds it (BOT-2410)"+ ;; Conversation state persisted before the viewing context kept the `:lib/*` keys holds such queries. Every tool+ ;; reads them through this normalization, and `metabot_message` normalizes its `:state` the same way on read.+ (let [stripped (query-repair-test/stripped-aggregated-query (lib/count))+ mem (memory/initialize [] {:queries {"q1" stripped}+ :charts {"c1" {:chart_id "c1", :query_id "q1", :queries [stripped]}}})+ valid? (fn [query]+ (and (= :mbql/query (:lib/type query))+ (mr/validate :metabase.lib.schema/query query)))]+ (is (valid? (memory/find-query mem "q1")))+ (is (valid? (-> (memory/find-chart mem "c1") :queries first)))))+ (testing "a query the repair cannot make valid stays as it was"+ (let [stripped (query-repair-test/stripped-aggregated-query+ (lib/count) (lib/sum (meta/field-metadata :venues :price)))+ mem (memory/initialize [] {:queries {"q1" stripped}})]+ (is (= stripped (memory/find-query mem "q1"))))))+(deftest ^:parallel initialize-normalizes-state-registry-keys-test(let [query {:database 1 :nested/value "preserved"}chart {:chart_id "c1"diff --git a/test/metabase/metabot/api_test.clj b/test/metabase/metabot/api_test.cljindex 9120706bcdf..dca0a8f3ff1 100644--- a/test/metabase/metabot/api_test.clj+++ b/test/metabase/metabot/api_test.clj@@ -5,6 +5,7 @@[clojure.test :refer :all][compojure.response][metabase.api.common :as mb.api]+ [metabase.api.macros :as api.macros][metabase.config.core :as config][metabase.lib.convert :as lib.convert][metabase.lib.core :as lib]@@ -34,6 +35,7 @@[metabase.test.server.handler :as test.server.handler][metabase.util :as u][metabase.util.json :as json]+ [metabase.util.malli.registry :as mr][toucan2.core :as t2]))(set! *warn-on-reflection* true)@@ -1561,6 +1563,24 @@(let [result (first (upgrade-viewing-queries [{:type "question" :query (legacy-query)}]))](is (nil? (:chart_configs result))))))+(deftest ^:parallel upgrade-viewing-queries-keeps-lib-uuids-test+ (testing "an MBQL 5 query keeps its :lib/uuids through the request decoder, so a ref to its aggregation still resolves (BOT-2410)"+ (let [q (let [q (-> (lib/query meta/metadata-provider (meta/table-metadata :venues))+ (lib/aggregate (lib/count)))]+ (lib/order-by q (lib/aggregation-ref q 0) :desc))+ agg-uuid (-> q :stages first :aggregation first second :lib/uuid)+ ;; the query as the frontend sends it: serialized, after a JSON round trip+ sent (-> q lib/prepare-for-serialization json/encode json/decode+kw)+ body (api.macros/decode-and-validate-params+ :body+ [:map [:context ::metabot.context/context]]+ {:context {:user_is_viewing [{:type "adhoc" :query sent}]}})+ [item] (#'api/upgrade-viewing-queries (get-in body [:context :user_is_viewing]))+ stage (-> item :query :stages first)]+ (is (= agg-uuid (-> stage :aggregation first second :lib/uuid)))+ (is (= agg-uuid (-> stage :order-by first (nth 2) (nth 2))))+ (is (mr/validate :metabase.lib.schema/query (:query item))))))+(deftest upgrade-viewing-queries-mixed-items-test(let [lq (legacy-query)items [{:type "adhoc" :query lq}diff --git a/test/metabase/metabot/query_repair_test.clj b/test/metabase/metabot/query_repair_test.cljnew file mode 100644index 00000000000..2d6b6982dad--- /dev/null+++ b/test/metabase/metabot/query_repair_test.clj@@ -0,0 +1,95 @@+(ns metabase.metabot.query-repair-test+ (:require+ [clojure.test :refer :all]+ [metabase.lib.core :as lib]+ [metabase.lib.test-metadata :as meta]+ [metabase.lib.test-util :as lib.tu]+ [metabase.metabot.query-repair :as query-repair]+ [metabase.util.malli.registry :as mr]))++(defn strip-lib-keys+ "Recursively remove all namespaced `:lib/*` keys from a query: the shape of a query a client sends without lib's+ internal keys, or of one in conversation state persisted before the viewing context kept them (BOT-2410).+ Structurally MBQL 5 (`:stages`, MBQL 5 field clauses, `:effective-type`s) but without `:lib/type`, `:lib/uuid`,+ and `:lib/metadata`."+ [x]+ (cond+ (map? x) (into {} (keep (fn [[k v]]+ (when-not (and (keyword? k) (= "lib" (namespace k)))+ [k (strip-lib-keys v)])))+ x)+ (sequential? x) (mapv strip-lib-keys x)+ :else x))++(defn stripped-aggregated-query+ "A venues query with `aggregations`, ordered by the first of them, stripped of its `:lib/*` keys. The order-by's+ `[:aggregation {} <uuid>]` ref names a uuid no aggregation carries any more."+ [& aggregations]+ (let [q (reduce lib/aggregate (lib.tu/venues-query) aggregations)]+ (strip-lib-keys (lib/order-by q (lib/aggregation-ref q 0) :desc))))++(defn stripped-query-with-aggregated-join+ "A venues query joined to a categories query that counts and orders by its count, stripped of its `:lib/*` keys.+ The dangling aggregation ref sits in the join's stage, not in the query's own."+ []+ (let [source (-> (lib/query meta/metadata-provider (meta/table-metadata :categories))+ (lib/aggregate (lib/count)))+ source (lib/order-by source (lib/aggregation-ref source 0) :desc)+ join (-> (lib/join-clause source [(lib/= (meta/field-metadata :venues :category-id)+ (meta/field-metadata :categories :id))])+ (lib/with-join-alias "Cat"))]+ (strip-lib-keys (lib/join (lib.tu/venues-query) join))))++(defn- order-by-ref-uuid [stage]+ (-> stage :order-by first (nth 2) (nth 2)))++(defn- sole-aggregation-uuid [stage]+ (-> stage :aggregation first second :lib/uuid))++(deftest ^:parallel normalize-query-repoints-sole-aggregation-ref-test+ (testing "a dangling ref in a stage with one aggregation is pointed at that aggregation"+ (let [repaired (query-repair/normalize-query (stripped-aggregated-query (lib/count)))+ stage (-> repaired :stages first)]+ (is (= :mbql/query (:lib/type repaired)))+ (is (= (sole-aggregation-uuid stage) (order-by-ref-uuid stage)))+ (is (mr/validate :metabase.lib.schema/query repaired))))+ (testing "a dangling ref in a stage with several aggregations stays as it is"+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))+ repaired (query-repair/normalize-query stripped)+ stage (-> repaired :stages first)]+ (is (= (order-by-ref-uuid (-> stripped :stages first)) (order-by-ref-uuid stage)))+ (is (not (mr/validate :metabase.lib.schema/query repaired)))))+ (testing "a query whose refs resolve is unchanged by the repair"+ (let [q (lib/aggregate (lib.tu/venues-query) (lib/count))+ q (lib/order-by q (lib/aggregation-ref q 0) :desc)]+ (is (= (lib/normalize :metabase.lib.schema/query q) (query-repair/normalize-query q))))))++(deftest ^:parallel normalize-query-repoints-refs-in-join-stages-test+ (testing "a dangling ref in the stage of a join is pointed at that stage's only aggregation"+ (let [repaired (query-repair/normalize-query (stripped-query-with-aggregated-join))+ join-stage (-> repaired :stages first :joins first :stages first)]+ (is (= (sole-aggregation-uuid join-stage) (order-by-ref-uuid join-stage)))+ (is (mr/validate :metabase.lib.schema/query repaired)))))++(deftest ^:parallel repair-stripped-query-test+ (testing "a structured query without :lib/type comes back as valid MBQL 5"+ (let [repaired (query-repair/repair-stripped-query (stripped-aggregated-query (lib/count)))]+ (is (= :mbql/query (:lib/type repaired)))+ (is (mr/validate :metabase.lib.schema/query repaired))))+ (testing "a query the repair cannot make valid is returned as it was"+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))]+ (is (= stripped (query-repair/repair-stripped-query stripped)))))+ (testing "a query that does not normalize is returned as it was"+ (let [broken {:database 1, :stages "not stages"}]+ (is (= broken (query-repair/repair-stripped-query broken)))))+ (testing "anything else is untouched"+ (let [mbql-5 (lib.tu/venues-query)+ legacy {:database 1, :type :query, :query {:source-table 2}}]+ (doseq [[shape value] {"MBQL 5" mbql-5+ "MBQL 5 with string type" (assoc (strip-lib-keys mbql-5) :lib/type "mbql/query")+ "legacy" legacy+ "nil" nil+ "a string" "SELECT 1"+ "an empty map" {}}]+ (testing shape+ (is (= value (query-repair/repair-stripped-query value))))))))diff --git a/test/metabase/metabot/tools/run_query_test.clj b/test/metabase/metabot/tools/run_query_test.cljindex e0621c11fbb..cc72a3d13a8 100644--- a/test/metabase/metabot/tools/run_query_test.clj+++ b/test/metabase/metabot/tools/run_query_test.clj@@ -11,7 +11,9 @@[metabase.lib.metadata :as lib.metadata][metabase.llm.test-util :as llm.tu][metabase.metabot.agent.core :as agent]+ [metabase.metabot.agent.memory :as memory][metabase.metabot.config :as metabot.config]+ [metabase.metabot.query-repair-test :as query-repair-test][metabase.metabot.scope :as scope][metabase.metabot.self.openrouter :as openrouter][metabase.metabot.test-util :as mut]@@ -174,6 +176,22 @@(run-tool! {"q1" (venues-by-id)} {:query_id "q1"}))(is (=? {:context :metabot, :executed-by (mt/user->id :rasta)} @info))))+(deftest run-query-runs-stripped-query-from-state-test+ (testing "a query in conversation state without its :lib/* keys runs once the state is read (BOT-2410)"+ ;; State persisted before the viewing context kept the `:lib/*` keys holds such queries. The repair happens where+ ;; the agent reads the state, in `memory/initialize`, so the memory goes through it here as it does in the loop.+ (let [mp (mt/metadata-provider)+ q (-> (lib/query mp (lib.metadata/table mp (mt/id :venues)))+ (lib/aggregate (lib/count)))+ stripped (query-repair-test/strip-lib-keys (lib/order-by q (lib/aggregation-ref q 0) :desc))]+ (mt/with-temp [:model/MetabotConversation {conversation-id :id} {:user_id (mt/user->id :rasta)}]+ (mt/with-temporary-setting-values [metabot-query-execution-enabled? true]+ (let [{:keys [output structured-output]}+ (call-tool! (assoc (memory/initialize [] {:queries {"q1" stripped}}) :conversation-id conversation-id)+ {:query_id "q1"})]+ (is (=? {:query-id "q1", :returned 1} structured-output))+ (is (= ["| Count |" "| --- |" "| 100 |"] (data-lines output)))))))))+(deftest run-query-refusals-test(testing "nothing runs while an admin has query execution turned off"(mt/with-temporary-setting-values [metabot-query-execution-enabled? false]