Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Point a dangling aggregation ref at the stage's only aggregation(#85123)

lbrdnk92f8c25fbb0b
diff --git a/src/metabase/agent_lib/representations/repair.clj b/src/metabase/agent_lib/representations/repair.clj
index 5df978cfd57..ebc767e9bb9 100644
--- a/src/metabase/agent_lib/representations/repair.clj
+++ b/src/metabase/agent_lib/representations/repair.clj
@@ -916,8 +916,8 @@
;;;
;;; `metabase.metabot.agent.links/->frontend-query` converts a structured query that does not
;;; normalize to valid MBQL 5 to legacy MBQL. If that conversion fails too, as it does for a query
-;;; stripped of its `:lib/*` keys whose aggregation ref no longer resolves, the raw MBQL 5 query goes
-;;; out instead. In legacy MBQL, JSON turns the unit into a string, and the legacy schema cannot
+;;; stripped of its `:lib/*` keys whose aggregation ref points at one of several aggregations, the
+;;; raw MBQL 5 query goes out instead. In legacy MBQL, JSON turns the unit into a string, and the legacy schema cannot
;;; decode it for `absolute-datetime` / `during`, so the frontend drops the clause (BOT-1972). A
;;; `temporal-unit` on a ref decodes correctly (`legacy-json-round-trip-temporal-filters-test`). The
;;; query builder also writes the hoisted form.
diff --git a/src/metabase/metabot/agent/links.clj b/src/metabase/metabot/agent/links.clj
index 7bc6583e5d4..4094afaff89 100644
--- a/src/metabase/metabot/agent/links.clj
+++ b/src/metabase/metabot/agent/links.clj
@@ -7,6 +7,7 @@
[metabase.lib.core :as lib]
[metabase.lib.schema]
[metabase.measures.db :as measures.db]
+ [metabase.metabot.query-repair :as query-repair]
[metabase.segments.db :as segments.db]
[metabase.system.core :as system]
[metabase.util.json :as json]
@@ -38,20 +39,16 @@
[query]
#_{:clj-kondo/ignore [:discouraged-var]}
(if (and (map? query)
- ;; `:stages` alone (no `:lib/type`) also counts: a query round-tripped through
- ;; the frontend's viewing context (`user_is_viewing` / `chart_configs`) comes
- ;; back MBQL 5 but stripped of its internal `:lib/*` keys. Guarding on
- ;; `:lib/type` alone let such a query fall through unconverted, leaking raw
- ;; MBQL 5 into the `/question#<base64>` hash and crashing the frontend with
- ;; "Stage 0 does not exist" (BOT-1604 follow-up).
+ ;; `:stages` alone (no `:lib/type`) also counts: a query stripped of its internal `:lib/*` keys is
+ ;; still MBQL 5. Guarding on `:lib/type` alone let such a query fall through unconverted, leaking raw
+ ;; MBQL 5 into the `/question#<base64>` hash and crashing the frontend with "Stage 0 does not exist"
+ ;; (BOT-1604 follow-up).
(or (:lib/type query) (:stages query)))
(try
- (lib/->legacy-MBQL (lib/normalize :metabase.lib.schema/query query))
+ (lib/->legacy-MBQL (query-repair/normalize-query query))
(catch Exception e
- ;; Normalizing a `:lib/*`-stripped query can mint fresh `:lib/uuid`s that don't
- ;; match positional aggregation/expression refs embedded elsewhere in the query
- ;; (e.g. an order-by on the query's own aggregation), which fails conversion.
- ;; Fall back to the raw query rather than failing the whole agent turn over a link.
+ ;; A ref that [[query-repair/normalize-query]] cannot repoint, such as one of several aggregations in a
+ ;; `:lib/*`-stripped query, fails conversion. Fall back to the raw query rather than failing the whole agent turn over a link.
(log/warn e "Failed to convert MBQL 5 query to legacy MBQL for link resolution")
query))
query))
@@ -64,12 +61,14 @@
Native-only queries use legacy MBQL because the embedding SDK opens its SQL editor only when
`dataset_query.type` is `\"native\"`.
- A query with `:stages` but no `:lib/type` stays in MBQL 5 if it normalizes to a valid query.
- All other values go through [[->legacy-mbql]]."
+ A query with `:stages` but no `:lib/type` stays in MBQL 5 if it normalizes to a valid query; see
+ [[query-repair/normalize-query]] for the refs it repairs on the way. Conversation state repairs such a query as it
+ is read, so this is the last guard, for a query that reaches a link by another way. All other values go through
+ [[->legacy-mbql]]."
[query]
(or (when (and (map? query) (or (:lib/type query) (:stages query)))
(try
- (let [normalized (lib/normalize :metabase.lib.schema/query query)]
+ (let [normalized (query-repair/normalize-query query)]
(when (and (not (lib/native-only-query? normalized))
(or (:lib/type query) (mr/validate :metabase.lib.schema/query normalized)))
(cond-> (lib/prepare-for-serialization normalized)
diff --git a/src/metabase/metabot/query_repair.clj b/src/metabase/metabot/query_repair.clj
new file mode 100644
index 00000000000..1155af5179f
--- /dev/null
+++ b/src/metabase/metabot/query_repair.clj
@@ -0,0 +1,77 @@
+(ns metabase.metabot.query-repair
+ "Repair of structured queries that lost their `:lib/*` keys.
+
+ A query in conversation state persisted before the viewing context kept its `:lib/*` keys (BOT-2410), or one a
+ client sends without them, has no `:lib/type` and no `:lib/uuid`s. Normalization gives each aggregation a fresh
+ `:lib/uuid`, so a ref to an aggregation, such as an order-by on it, names a uuid that no aggregation carries. When
+ the stage has one aggregation the ref can only mean that one, and it is pointed there.
+
+ [[repair-stripped-query]] runs on every query of conversation state as it is normalized, so every tool reads a
+ valid MBQL 5 query. [[normalize-query]] is the same repair for a query on its way to the frontend, where the link
+ path keeps its own fallback."
+ (:require
+ [metabase.lib.core :as lib]
+ [metabase.lib.schema :as lib.schema]
+ [metabase.util.log :as log]
+ [metabase.util.malli.registry :as mr]))
+
+(set! *warn-on-reflection* true)
+
+(declare repoint-dangling-aggregation-refs-in-stage)
+
+(defn- repoint-in-joins
+ [stage]
+ (cond-> stage
+ (sequential? (:joins stage))
+ (update :joins (partial mapv (fn [join]
+ (cond-> join
+ (sequential? (:stages join))
+ (update :stages (partial mapv repoint-dangling-aggregation-refs-in-stage))))))))
+
+(defn- repoint-dangling-aggregation-refs-in-stage
+ "Point every `[:aggregation opts uuid]` ref in `stage` that names none of the stage's aggregations at the stage's
+ only aggregation, then do the same in the stages of its joins. With several aggregations the target is unknown, so
+ the refs stay as they are.
+
+ [[lib.schema/ref-errors-for-stage]] finds the dangling refs, with the same rule the query schema validates by. It
+ skips `:joins`, which are walked here, and namespaced keys."
+ [stage]
+ (let [uuids (into #{} (map (comp :lib/uuid second)) (:aggregation stage))
+ stage (if (= 1 (count uuids))
+ (let [sole (first uuids)]
+ (reduce (fn [stage [path clause]]
+ (cond-> stage
+ (and (= :aggregation (first clause))
+ (= 3 (count clause)))
+ (assoc-in (conj path 2) sole)))
+ stage
+ (lib.schema/ref-errors-for-stage stage)))
+ stage)]
+ (repoint-in-joins stage)))
+
+(defn normalize-query
+ "Normalize `query` as MBQL 5 and repoint its dangling aggregation refs; see
+ [[repoint-dangling-aggregation-refs-in-stage]]. Throws when `query` does not normalize. The result is not
+ validated: a ref that could mean any of several aggregations stays dangling."
+ [query]
+ (let [normalized (lib/normalize ::lib.schema/query query)]
+ (cond-> normalized
+ (vector? (:stages normalized))
+ (update :stages (partial mapv repoint-dangling-aggregation-refs-in-stage)))))
+
+(defn repair-stripped-query
+ "Repair `query` for conversation state when it is a structured query without its `:lib/type`: normalize it and
+ repoint its dangling aggregation refs. Any other value is returned unchanged, and so is a query whose repair is
+ not a valid query, or does not normalize at all: state must read even when it holds a query no tool can run, and
+ the tools' own guards word the refusal."
+ [query]
+ (if (and (map? query) (:stages query) (not (:lib/type query)))
+ (try
+ (let [repaired (normalize-query query)]
+ (if (mr/validate ::lib.schema/query repaired)
+ repaired
+ query))
+ (catch Exception e
+ (log/warn e "Failed to repair a query without :lib/type in conversation state")
+ query))
+ query))
diff --git a/src/metabase/metabot/schema.clj b/src/metabase/metabot/schema.clj
index 8c79fe6bb50..1222087ec33 100644
--- a/src/metabase/metabot/schema.clj
+++ b/src/metabase/metabot/schema.clj
@@ -5,6 +5,7 @@
[metabase.lib.schema.common :as lib.schema.common]
[metabase.lib.schema.id :as lib.schema.id]
[metabase.lib.util :as lib.util]
+ [metabase.metabot.query-repair :as query-repair]
[metabase.metabot.schema.v2 :as schema.v2]
[metabase.util :as u]
[metabase.util.malli.registry :as mr]
@@ -34,8 +35,11 @@
[:string {:decode/normalize lib.schema.common/normalize-string-key}])
(mr/def ::query
- "An MBQL 5 or legacy query in any state of normalization, as it round-trips through tool-call JSON or persisted turn state."
- ::lib.util/query-like)
+ "An MBQL 5 or legacy query in any state of normalization, as it round-trips through tool-call JSON or persisted turn
+ state. Normalizing repairs a structured query that lost its `:lib/*` keys, so every tool reads a valid MBQL 5 query;
+ see [[query-repair/repair-stripped-query]]."
+ [:schema {:decode/normalize query-repair/repair-stripped-query}
+ ::lib.util/query-like])
(mr/def ::todo
"One todo item of persisted turn state; only `:id` is guaranteed, since state can hold a partial item
diff --git a/test/metabase/metabot/agent/links_test.clj b/test/metabase/metabot/agent/links_test.clj
index 2d96c589ad4..c4ec8c28613 100644
--- a/test/metabase/metabot/agent/links_test.clj
+++ b/test/metabase/metabot/agent/links_test.clj
@@ -7,12 +7,14 @@
[metabase.lib.test-metadata :as meta]
[metabase.lib.test-util :as lib.tu]
[metabase.metabot.agent.links :as links]
+ [metabase.metabot.query-repair-test :as query-repair-test]
[metabase.metabot.test-util :as test-util]
[metabase.metabot.tools.charts :as charts]
[metabase.metabot.tools.shared :as shared]
[metabase.test :as mt]
[metabase.util :as u]
- [metabase.util.json :as json]))
+ [metabase.util.json :as json]
+ [metabase.util.malli.registry :as mr]))
(defn- decode-question-url
"Decode a `/question#<base64>` URL into its JSON-decoded pseudo-card map."
@@ -42,19 +44,8 @@
:base-type :type/DateTime}]]}}
(links/->legacy-mbql rehydrated))))))
-(defn- strip-lib-keys
- "Recursively remove all namespaced `:lib/*` keys from a query, mimicking how a
- pMBQL query that round-trips through the frontend's viewing context comes back:
- structurally pMBQL (`:stages`, pMBQL-order field clauses, `:effective-type`s)
- but stripped of its internal `:lib/type`, `:lib/uuid`, and `:lib/metadata`."
- [x]
- (cond
- (map? x) (into {} (keep (fn [[k v]]
- (when-not (and (keyword? k) (= "lib" (namespace k)))
- [k (strip-lib-keys v)])))
- x)
- (sequential? x) (mapv strip-lib-keys x)
- :else x))
+(def ^:private strip-lib-keys query-repair-test/strip-lib-keys)
+(def ^:private stripped-aggregated-query query-repair-test/stripped-aggregated-query)
(deftest ^:parallel ->legacy-mbql-lib-type-less-pmbql-test
(testing "converts pMBQL stripped of :lib/* keys (frontend viewing-context round-trip) to legacy MBQL"
@@ -74,17 +65,19 @@
(is (map? (:query legacy)))
(is (some? (:database legacy)))))))
-(deftest ^:parallel ->legacy-mbql-stripped-query-with-aggregation-ref-falls-back-test
- (testing "falls back to the raw query instead of throwing when the stripped query's positional aggregation ref can't be resolved after normalize"
- ;; Regression: `strip-lib-keys` drops the aggregation's `:lib/uuid`, but an
- ;; `[:aggregation {} <uuid>]` ref elsewhere in the query (e.g. an order-by on the
- ;; query's own aggregation) still points at the old uuid. `normalize` mints a fresh
- ;; uuid, the ref lookup misses, and conversion throws — this used to take down the
- ;; whole agent turn instead of just producing a broken link.
- (let [q (-> (lib.tu/venues-query) (lib/aggregate (lib/count)))
- q2 (lib/order-by q (lib/aggregation-ref q 0) :desc)
- stripped (strip-lib-keys q2)]
+(deftest ^:parallel ->legacy-mbql-stripped-query-with-aggregation-ref-test
+ (testing "points the stripped query's aggregation ref at the stage's only aggregation, then converts (BOT-2410)"
+ ;; `normalize` mints a fresh `:lib/uuid` for the aggregation, so the ref in the order-by names a uuid that no
+ ;; aggregation carries. With one aggregation the ref can only mean that one.
+ (let [stripped (stripped-aggregated-query (lib/count))
+ legacy (links/->legacy-mbql stripped)]
(is (not (contains? stripped :lib/type)))
+ (is (= :query (:type legacy)))
+ (is (= [[:desc [:aggregation 0]]] (-> legacy :query :order-by)))))
+ (testing "falls back to the raw query instead of throwing when the ref could mean any of several aggregations"
+ ;; Conversion throws on the dangling ref. This used to take down the whole agent turn instead of just
+ ;; producing a broken link.
+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))]
(is (= stripped (links/->legacy-mbql stripped))))))
(deftest ^:parallel pseudo-card->link-keeps-mbql5-temporal-filters-test
@@ -115,12 +108,25 @@
(is (= (strip (lib/filters query))
(strip (-> dataset-query lib/normalize :stages first :filters)))))))))))
-(deftest ^:parallel ->frontend-query-stripped-query-with-aggregation-ref-goes-out-raw-test
- (testing "a query without :lib/* keys goes out as the raw MBQL 5 query when normalization invalidates its aggregation ref"
- ;; Normalization assigns a new UUID to the aggregation, so the aggregation ref no longer resolves,
- ;; and ->legacy-mbql cannot convert the query either.
- (let [q (-> (lib.tu/venues-query) (lib/aggregate (lib/count)))
- stripped (strip-lib-keys (lib/order-by q (lib/aggregation-ref q 0) :desc))]
+(deftest ^:parallel ->frontend-query-stripped-query-with-aggregation-ref-test
+ (testing "a query without :lib/* keys goes out as valid MBQL 5 with its aggregation ref pointed at the stage's only aggregation (BOT-2410)"
+ (let [sent (links/->frontend-query (stripped-aggregated-query (lib/count)))
+ stage (-> sent :stages first)]
+ (is (= :mbql/query (:lib/type sent)))
+ (is (mr/validate :metabase.lib.schema/query sent))
+ (is (= (-> stage :aggregation first second :lib/uuid)
+ (-> stage :order-by first (nth 2) (nth 2))))))
+ (testing "a dangling ref in the stage of a join is repaired the same way"
+ (let [sent (links/->frontend-query (query-repair-test/stripped-query-with-aggregated-join))
+ join-stage (-> sent :stages first :joins first :stages first)]
+ (is (= :mbql/query (:lib/type sent)))
+ (is (mr/validate :metabase.lib.schema/query sent))
+ (is (= (-> join-stage :aggregation first second :lib/uuid)
+ (-> join-stage :order-by first (nth 2) (nth 2))))))
+ (testing "goes out as the raw MBQL 5 query when the ref could mean any of several aggregations"
+ ;; Normalization gives each aggregation a new uuid, the ref resolves to none of them, and ->legacy-mbql cannot
+ ;; convert the query either.
+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))]
(is (= stripped (links/->frontend-query stripped))))))
(deftest ^:parallel ->frontend-query-sends-native-queries-as-legacy-test
diff --git a/test/metabase/metabot/agent/memory_test.clj b/test/metabase/metabot/agent/memory_test.clj
index 10cfe92a06c..623d2143a32 100644
--- a/test/metabase/metabot/agent/memory_test.clj
+++ b/test/metabase/metabot/agent/memory_test.clj
@@ -2,8 +2,11 @@
(:require
[clojure.string :as str]
[clojure.test :refer :all]
+ [metabase.lib.core :as lib]
+ [metabase.lib.test-metadata :as meta]
[metabase.metabot.agent.links :as links]
[metabase.metabot.agent.memory :as memory]
+ [metabase.metabot.query-repair-test :as query-repair-test]
[metabase.metabot.schema :as metabot.schema]
[metabase.metabot.tools.shared :as shared]
[metabase.util.malli.registry :as mr]))
@@ -22,6 +25,24 @@
(is (= {:queries {} :charts {} :todos [] :link-registry {}}
(memory/get-state mem))))))
+(deftest ^:parallel initialize-repairs-stripped-queries-test
+ (testing "a structured query without its :lib/* keys is read as valid MBQL 5, wherever state holds it (BOT-2410)"
+ ;; Conversation state persisted before the viewing context kept the `:lib/*` keys holds such queries. Every tool
+ ;; reads them through this normalization, and `metabot_message` normalizes its `:state` the same way on read.
+ (let [stripped (query-repair-test/stripped-aggregated-query (lib/count))
+ mem (memory/initialize [] {:queries {"q1" stripped}
+ :charts {"c1" {:chart_id "c1", :query_id "q1", :queries [stripped]}}})
+ valid? (fn [query]
+ (and (= :mbql/query (:lib/type query))
+ (mr/validate :metabase.lib.schema/query query)))]
+ (is (valid? (memory/find-query mem "q1")))
+ (is (valid? (-> (memory/find-chart mem "c1") :queries first)))))
+ (testing "a query the repair cannot make valid stays as it was"
+ (let [stripped (query-repair-test/stripped-aggregated-query
+ (lib/count) (lib/sum (meta/field-metadata :venues :price)))
+ mem (memory/initialize [] {:queries {"q1" stripped}})]
+ (is (= stripped (memory/find-query mem "q1"))))))
+
(deftest ^:parallel initialize-normalizes-state-registry-keys-test
(let [query {:database 1 :nested/value "preserved"}
chart {:chart_id "c1"
diff --git a/test/metabase/metabot/api_test.clj b/test/metabase/metabot/api_test.clj
index 9120706bcdf..dca0a8f3ff1 100644
--- a/test/metabase/metabot/api_test.clj
+++ b/test/metabase/metabot/api_test.clj
@@ -5,6 +5,7 @@
[clojure.test :refer :all]
[compojure.response]
[metabase.api.common :as mb.api]
+ [metabase.api.macros :as api.macros]
[metabase.config.core :as config]
[metabase.lib.convert :as lib.convert]
[metabase.lib.core :as lib]
@@ -34,6 +35,7 @@
[metabase.test.server.handler :as test.server.handler]
[metabase.util :as u]
[metabase.util.json :as json]
+ [metabase.util.malli.registry :as mr]
[toucan2.core :as t2]))
(set! *warn-on-reflection* true)
@@ -1561,6 +1563,24 @@
(let [result (first (upgrade-viewing-queries [{:type "question" :query (legacy-query)}]))]
(is (nil? (:chart_configs result))))))
+(deftest ^:parallel upgrade-viewing-queries-keeps-lib-uuids-test
+ (testing "an MBQL 5 query keeps its :lib/uuids through the request decoder, so a ref to its aggregation still resolves (BOT-2410)"
+ (let [q (let [q (-> (lib/query meta/metadata-provider (meta/table-metadata :venues))
+ (lib/aggregate (lib/count)))]
+ (lib/order-by q (lib/aggregation-ref q 0) :desc))
+ agg-uuid (-> q :stages first :aggregation first second :lib/uuid)
+ ;; the query as the frontend sends it: serialized, after a JSON round trip
+ sent (-> q lib/prepare-for-serialization json/encode json/decode+kw)
+ body (api.macros/decode-and-validate-params
+ :body
+ [:map [:context ::metabot.context/context]]
+ {:context {:user_is_viewing [{:type "adhoc" :query sent}]}})
+ [item] (#'api/upgrade-viewing-queries (get-in body [:context :user_is_viewing]))
+ stage (-> item :query :stages first)]
+ (is (= agg-uuid (-> stage :aggregation first second :lib/uuid)))
+ (is (= agg-uuid (-> stage :order-by first (nth 2) (nth 2))))
+ (is (mr/validate :metabase.lib.schema/query (:query item))))))
+
(deftest upgrade-viewing-queries-mixed-items-test
(let [lq (legacy-query)
items [{:type "adhoc" :query lq}
diff --git a/test/metabase/metabot/query_repair_test.clj b/test/metabase/metabot/query_repair_test.clj
new file mode 100644
index 00000000000..2d6b6982dad
--- /dev/null
+++ b/test/metabase/metabot/query_repair_test.clj
@@ -0,0 +1,95 @@
+(ns metabase.metabot.query-repair-test
+ (:require
+ [clojure.test :refer :all]
+ [metabase.lib.core :as lib]
+ [metabase.lib.test-metadata :as meta]
+ [metabase.lib.test-util :as lib.tu]
+ [metabase.metabot.query-repair :as query-repair]
+ [metabase.util.malli.registry :as mr]))
+
+(defn strip-lib-keys
+ "Recursively remove all namespaced `:lib/*` keys from a query: the shape of a query a client sends without lib's
+ internal keys, or of one in conversation state persisted before the viewing context kept them (BOT-2410).
+ Structurally MBQL 5 (`:stages`, MBQL 5 field clauses, `:effective-type`s) but without `:lib/type`, `:lib/uuid`,
+ and `:lib/metadata`."
+ [x]
+ (cond
+ (map? x) (into {} (keep (fn [[k v]]
+ (when-not (and (keyword? k) (= "lib" (namespace k)))
+ [k (strip-lib-keys v)])))
+ x)
+ (sequential? x) (mapv strip-lib-keys x)
+ :else x))
+
+(defn stripped-aggregated-query
+ "A venues query with `aggregations`, ordered by the first of them, stripped of its `:lib/*` keys. The order-by's
+ `[:aggregation {} <uuid>]` ref names a uuid no aggregation carries any more."
+ [& aggregations]
+ (let [q (reduce lib/aggregate (lib.tu/venues-query) aggregations)]
+ (strip-lib-keys (lib/order-by q (lib/aggregation-ref q 0) :desc))))
+
+(defn stripped-query-with-aggregated-join
+ "A venues query joined to a categories query that counts and orders by its count, stripped of its `:lib/*` keys.
+ The dangling aggregation ref sits in the join's stage, not in the query's own."
+ []
+ (let [source (-> (lib/query meta/metadata-provider (meta/table-metadata :categories))
+ (lib/aggregate (lib/count)))
+ source (lib/order-by source (lib/aggregation-ref source 0) :desc)
+ join (-> (lib/join-clause source [(lib/= (meta/field-metadata :venues :category-id)
+ (meta/field-metadata :categories :id))])
+ (lib/with-join-alias "Cat"))]
+ (strip-lib-keys (lib/join (lib.tu/venues-query) join))))
+
+(defn- order-by-ref-uuid [stage]
+ (-> stage :order-by first (nth 2) (nth 2)))
+
+(defn- sole-aggregation-uuid [stage]
+ (-> stage :aggregation first second :lib/uuid))
+
+(deftest ^:parallel normalize-query-repoints-sole-aggregation-ref-test
+ (testing "a dangling ref in a stage with one aggregation is pointed at that aggregation"
+ (let [repaired (query-repair/normalize-query (stripped-aggregated-query (lib/count)))
+ stage (-> repaired :stages first)]
+ (is (= :mbql/query (:lib/type repaired)))
+ (is (= (sole-aggregation-uuid stage) (order-by-ref-uuid stage)))
+ (is (mr/validate :metabase.lib.schema/query repaired))))
+ (testing "a dangling ref in a stage with several aggregations stays as it is"
+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))
+ repaired (query-repair/normalize-query stripped)
+ stage (-> repaired :stages first)]
+ (is (= (order-by-ref-uuid (-> stripped :stages first)) (order-by-ref-uuid stage)))
+ (is (not (mr/validate :metabase.lib.schema/query repaired)))))
+ (testing "a query whose refs resolve is unchanged by the repair"
+ (let [q (lib/aggregate (lib.tu/venues-query) (lib/count))
+ q (lib/order-by q (lib/aggregation-ref q 0) :desc)]
+ (is (= (lib/normalize :metabase.lib.schema/query q) (query-repair/normalize-query q))))))
+
+(deftest ^:parallel normalize-query-repoints-refs-in-join-stages-test
+ (testing "a dangling ref in the stage of a join is pointed at that stage's only aggregation"
+ (let [repaired (query-repair/normalize-query (stripped-query-with-aggregated-join))
+ join-stage (-> repaired :stages first :joins first :stages first)]
+ (is (= (sole-aggregation-uuid join-stage) (order-by-ref-uuid join-stage)))
+ (is (mr/validate :metabase.lib.schema/query repaired)))))
+
+(deftest ^:parallel repair-stripped-query-test
+ (testing "a structured query without :lib/type comes back as valid MBQL 5"
+ (let [repaired (query-repair/repair-stripped-query (stripped-aggregated-query (lib/count)))]
+ (is (= :mbql/query (:lib/type repaired)))
+ (is (mr/validate :metabase.lib.schema/query repaired))))
+ (testing "a query the repair cannot make valid is returned as it was"
+ (let [stripped (stripped-aggregated-query (lib/count) (lib/sum (meta/field-metadata :venues :price)))]
+ (is (= stripped (query-repair/repair-stripped-query stripped)))))
+ (testing "a query that does not normalize is returned as it was"
+ (let [broken {:database 1, :stages "not stages"}]
+ (is (= broken (query-repair/repair-stripped-query broken)))))
+ (testing "anything else is untouched"
+ (let [mbql-5 (lib.tu/venues-query)
+ legacy {:database 1, :type :query, :query {:source-table 2}}]
+ (doseq [[shape value] {"MBQL 5" mbql-5
+ "MBQL 5 with string type" (assoc (strip-lib-keys mbql-5) :lib/type "mbql/query")
+ "legacy" legacy
+ "nil" nil
+ "a string" "SELECT 1"
+ "an empty map" {}}]
+ (testing shape
+ (is (= value (query-repair/repair-stripped-query value))))))))
diff --git a/test/metabase/metabot/tools/run_query_test.clj b/test/metabase/metabot/tools/run_query_test.clj
index e0621c11fbb..cc72a3d13a8 100644
--- a/test/metabase/metabot/tools/run_query_test.clj
+++ b/test/metabase/metabot/tools/run_query_test.clj
@@ -11,7 +11,9 @@
[metabase.lib.metadata :as lib.metadata]
[metabase.llm.test-util :as llm.tu]
[metabase.metabot.agent.core :as agent]
+ [metabase.metabot.agent.memory :as memory]
[metabase.metabot.config :as metabot.config]
+ [metabase.metabot.query-repair-test :as query-repair-test]
[metabase.metabot.scope :as scope]
[metabase.metabot.self.openrouter :as openrouter]
[metabase.metabot.test-util :as mut]
@@ -174,6 +176,22 @@
(run-tool! {"q1" (venues-by-id)} {:query_id "q1"}))
(is (=? {:context :metabot, :executed-by (mt/user->id :rasta)} @info))))
+(deftest run-query-runs-stripped-query-from-state-test
+ (testing "a query in conversation state without its :lib/* keys runs once the state is read (BOT-2410)"
+ ;; State persisted before the viewing context kept the `:lib/*` keys holds such queries. The repair happens where
+ ;; the agent reads the state, in `memory/initialize`, so the memory goes through it here as it does in the loop.
+ (let [mp (mt/metadata-provider)
+ q (-> (lib/query mp (lib.metadata/table mp (mt/id :venues)))
+ (lib/aggregate (lib/count)))
+ stripped (query-repair-test/strip-lib-keys (lib/order-by q (lib/aggregation-ref q 0) :desc))]
+ (mt/with-temp [:model/MetabotConversation {conversation-id :id} {:user_id (mt/user->id :rasta)}]
+ (mt/with-temporary-setting-values [metabot-query-execution-enabled? true]
+ (let [{:keys [output structured-output]}
+ (call-tool! (assoc (memory/initialize [] {:queries {"q1" stripped}}) :conversation-id conversation-id)
+ {:query_id "q1"})]
+ (is (=? {:query-id "q1", :returned 1} structured-output))
+ (is (= ["| Count |" "| --- |" "| 100 |"] (data-lines output)))))))))
+
(deftest run-query-refusals-test
(testing "nothing runs while an admin has query execution turned off"
(mt/with-temporary-setting-values [metabot-query-execution-enabled? false]