The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
UXW-4967: two tier research catalog (#79109)
John Swansona2bc99107e55
diff --git a/resources/metabot/prompts/system/explorations.selmer b/resources/metabot/prompts/system/explorations.selmerindex c79d263b425..fa85ddaea2d 100644--- a/resources/metabot/prompts/system/explorations.selmer+++ b/resources/metabot/prompts/system/explorations.selmer@@ -19,7 +19,7 @@ A research plan is made of one or more **groups**. Each group is one of:- Use the `set_research_name` tool to give the research project a relevant name based on the user's question. You only need to call this tool once.- Work in two steps. First **discover** which metrics are relevant: use the `search` tool with `"entity_types": ["metric"]` to find them by topic. This is the best way to surface relevant metrics when the instance has many — it ranks results by relevance (semantically, when available).-- Then **get valid ids** from `get_research_candidates`, which lists the research-eligible metrics and, for each, its candidate dimensions (with ids), plus the dimension groups and the metrics they can slice. You must take every metric and dimension id you pass to `add_research_groups` from this catalog. Pass `q` to narrow the catalog, but note `q` is a plain substring filter on names — use `search` (not `q`) when you don't yet know which metrics are relevant.+- Then **get valid ids**: `list_research_metrics` lists the available metrics (no dimensions); `get_research_candidates` returns the candidate dimensions for chosen metrics — pass `metric_ids` (up to 20) and/or `q`. Each metric there lists its own `dimensions` (the `id` to use, and the `group` it belongs to), and each entry in `dimension_groups` describes one group and the `metric_ids` it can slice. You must take every metric and dimension id you pass to `add_research_groups` from `get_research_candidates`. On both tools `q` is a plain substring filter on metric and dimension names — use `search` (not `q`) when you don't yet know which metrics are relevant.- If you're unsure whether a metric is relevant, you can use `read_resource` to examine it. `metabase://metric/{id}/dimensions/{dimension_id}` returns sample values for a dimension.- Try to identify multiple relevant metrics and decompose the problem. For example, if the user's question is related to revenue, also consider metrics related to churn, page views, conversion, pricing, etc.- Once you've decided on the plan, call `add_research_groups` once with all the groups, and explain your rationale to the user. No need to ask for confirmation - the user can easily add or remove groups, metrics, and dimensions themselves.diff --git a/src/metabase/explorations/core.clj b/src/metabase/explorations/core.cljindex 7363de3859f..2b188206a63 100644--- a/src/metabase/explorations/core.clj+++ b/src/metabase/explorations/core.clj@@ -8,5 +8,8 @@exploration-dataexploration-data->apiresearch-candidates+ research-candidates-max-metrics+ research-metric-index+ research-metric-index-max-metricsresearch-groupsmin-interestingness])diff --git a/src/metabase/explorations/impl.clj b/src/metabase/explorations/impl.cljindex 3c3a0a63e19..3f38b821070 100644--- a/src/metabase/explorations/impl.clj+++ b/src/metabase/explorations/impl.clj@@ -251,39 +251,68 @@(assoc :dimension_ids (mapv :id (:dimensions m)))(dissoc :dimensions)))+(defn- matching-metrics+ "`metrics` restricted to those matching search `q`; all of them+ when `q` is blank."+ [q metrics]+ (if (str/blank? q)+ metrics+ (let [q-lower (u/lower-case-en q)]+ (filterv #(metric-matches-search? % q-lower) metrics))))++(defn- catalog-metrics+ "Permission-filtered metric Cards with their synced `:dimensions` inlined and each dimension+ annotated with its Field's interestingness, restricted to `:metric-ids` (or all visible when+ nil). Dimension targets are not resolved and `:result_column_name` is not computed."+ [{:keys [metric-ids]}]+ (let [library-ids (or (library-metrics-collection-ids) #{})+ card-ids (accessible-metric-ids metric-ids library-ids)+ cards (sync-missing-dimensions! (load-metric-cards card-ids))+ ;; Filter dimensions by user permissions for all metrics at once (one set of queries+ ;; for the whole batch, rather than per metric).+ permitted (metrics/filter-dimensions-for-user-batch cards)]+ (->> permitted+ (mapv #(assoc % :in_library (contains? library-ids (:collection_id %))))+ (metrics/annotate-dimensions-with-field-data [:dimension_interestingness]))))++(defn- resolve-metric-queries+ "Drop each metric's dimensions whose target doesn't resolve against its own query and compute+ `:result_column_name`. This is the expensive half of [[hydrated-metrics]] — every metric pays+ a query build, and `lib/breakoutable-columns` on top of that."+ [metrics]+ (let [resolve-breakoutable (make-breakoutable-resolver)]+ (mapv (fn [m]+ ;; Build the metric's query once and reuse it for resolving dimension targets and+ ;; computing the result column name. Breakoutable columns (the dominant cost) are+ ;; shared across metrics that query the same table.+ (let [query (metric-query m)+ breakoutable (resolve-breakoutable m query)]+ (-> m+ (filter-resolvable-dimensions query breakoutable)+ (assoc :result_column_name (result-column-name query))+ ;; dataset_query was only needed to build `query`.+ (dissoc :dataset_query))))+ metrics)))+(defn- hydrated-metrics"Permission-filtered, interestingness-annotated metric Cards with their candidate dimensionsinlined as `:dimensions`, restricted to `:metric-ids` (or all visible when nil) and optionallyfiltered by search `:q`. Shared by [[exploration-data]], [[research-candidates]], and[[research-groups]]."- [{:keys [metric-ids q]}]+ [{:keys [q] :as opts}](lib-be/with-metadata-provider-cache- (let [library-ids (or (library-metrics-collection-ids) #{})- card-ids (accessible-metric-ids metric-ids library-ids)- cards (sync-missing-dimensions! (load-metric-cards card-ids))- ;; Filter dimensions by user permissions for all metrics at once (one set of queries- ;; for the whole batch, rather than per metric).- permitted (metrics/filter-dimensions-for-user-batch cards)- resolve-breakoutable (make-breakoutable-resolver)- hydrated (->> permitted- (mapv (fn [m]- ;; Build the metric's query once and reuse it for resolving- ;; dimension targets and computing the result column name.- ;; Breakoutable columns (the dominant cost) are shared across- ;; metrics that query the same table.- (let [query (metric-query m)- breakoutable (resolve-breakoutable m query)]- (-> m- (filter-resolvable-dimensions query breakoutable)- (assoc :result_column_name (result-column-name query)- :in_library (contains? library-ids (:collection_id m)))- ;; dataset_query was only needed to build `query`.- (dissoc :dataset_query)))))- (metrics/annotate-dimensions-with-field-data [:dimension_interestingness]))]- (if (str/blank? q)- hydrated- (let [q-lower (u/lower-case-en q)]- (filterv #(metric-matches-search? % q-lower) hydrated))))))+ (->> (catalog-metrics opts)+ resolve-metric-queries+ (matching-metrics q))))++(defn- index-metrics+ "[[hydrated-metrics]] minus [[resolve-metric-queries]], for callers that need only a metric's+ identity and its dimension names. Skipping target resolution means a dimension the metric's+ query can't actually break out on is still counted here — over-inclusive by design, since the+ caller ([[research-metric-index]]) surfaces no dimension the user could act on."+ [{:keys [q] :as opts}]+ (lib-be/with-metadata-provider-cache+ (matching-metrics q (catalog-metrics opts))))(defn- candidate-dimension?"Whether a dimension is surfaced as a research candidate: it scored at or above@@ -352,33 +381,144 @@"Metric fields surfaced to Metabot in the research catalog."[:id :name :description :result_column_name])-(def ^:private llm-dimension-cols- "Dimension fields surfaced to Metabot in the research catalog."- [:id :name :display-name :effective-type :semantic-type :dimension-interestingness])+(def research-candidates-max-metrics+ "Maximum metrics a single `get_research_candidates` response details. Explicit `:metric-ids`+ requests are capped to this at the tool layer; `:q` requests matching more are truncated (with+ a marker) so a broad search term can't recreate the unbounded catalog dump this bound exists+ to prevent."+ 20)++(def research-metric-index-max-metrics+ "Maximum rows a single `list_research_metrics` response lists. Index rows are slim, but an+ unfiltered index on an instance with thousands of metrics is still a prompt-straining blob+ (measured ~130 bytes/row); above this the index is truncated and stamped with a marker so the+ model knows to narrow with `q`."+ 500)++(def ^:private catalog-description-max-length+ "Character budget for a metric description in either research tool's payload — enough for scent,+ short enough that a handful of essay-length descriptions can't dominate a response. A model+ that needs the whole thing can `read_resource` the metric."+ 150)++(defn- truncate-description+ [description]+ (when-let [d (some-> description str/trim not-empty)]+ (if (> (count d) catalog-description-max-length)+ (str (subs d 0 catalog-description-max-length) "…")+ d)))++(defn- metric-interestingness+ "Ranking score for a metric: the max interestingness across its candidate dimensions, or nil+ when none scored. Callers pass metrics already restricted to their candidate dimensions (see+ [[with-candidate-dimensions]]), so a metric whose every dimension scored below+ [[min-interestingness]] scores nil here rather than its best sub-threshold dimension."+ [m]+ (some->> (keep :dimension-interestingness (:dimensions m)) seq (apply max)))++(defn- rank-metrics+ "Order `metrics` library-first then by interestingness (unscored last) — the order both+ research-tool caps clip in, so the best-curated content survives truncation."+ [metrics]+ (vec (sort-by (fn [m]+ [(if (:in_library m) 0 1)+ (- (or (metric-interestingness m) -1))])+ metrics)))++(defn research-metric-index+ "Slim research catalog index for the `list_research_metrics` Metabot tool:+ `{:metrics [{:id :name :description :in_library} ...]}` — one row per metric the user can+ read, truncated `:description`, no dimensions. The model shortlists metric ids here, then+ fetches their dimension detail via [[research-candidates]]. `:q` filters like+ [[exploration-data]]'s — a case-insensitive substring match on metric names and dimension+ display names.++ Rows are always ordered by [[rank-metrics]], so a metric doesn't move just because the+ instance grew past the cap. More than [[research-metric-index-max-metrics]] matches are+ truncated and stamped `{:truncated true :shown <n> :matched <m>}` so the model knows to narrow+ with `:q`. A metric with no candidate dimensions still gets a row — it can anchor a group on+ its own — but ranks below every metric that has one."+ [opts]+ (let [ranked (rank-metrics (mapv with-candidate-dimensions (index-metrics opts)))+ capped (vec (take research-metric-index-max-metrics ranked))+ rows (mapv (fn [m]+ {:id (:id m)+ :name (:name m)+ :description (truncate-description (:description m))+ :in_library (:in_library m)})+ capped)]+ (cond-> {:metrics rows}+ (> (count ranked) (count capped))+ (assoc :truncated true+ :shown (count capped)+ :matched (count ranked)))))++(defn- llm-dimension-groups+ "`groups` (from [[group-dimensions]] over `metrics`) in the `get_research_candidates` shape: the+ descriptive fields stated once, plus the ids of the metrics the group can slice. The per-metric+ dimension ids live on the metrics themselves (see [[llm-metric-dimensions]]).++ `:effective_type`/`:semantic_type` appear only when every dimension in the group agrees on+ them."+ [metrics groups]+ (let [dim->metrics (dimension-id->metric-ids metrics)]+ (mapv (fn [g]+ (let [dims (:dimensions g)+ types (when (= 1 (count (distinct (map (juxt :effective-type :semantic-type) dims))))+ (first dims))]+ (cond-> {:name (:name g)+ :interestingness (:dimension_interestingness g)+ :metric_ids (vec (sort (into #{} (mapcat #(dim->metrics (:id %))) dims)))}+ types (assoc :effective_type (:effective-type types)+ :semantic_type (:semantic-type types)))))+ groups)))++(defn- llm-metric-dimensions+ "One metric's candidate dimensions in the `get_research_candidates` shape: the `:id` to echo to+ `add_research_groups`, tagged with the `:group` whose descriptive fields describe it. `:name`+ appears only when this metric calls the dimension something other than its group name — renames+ are common, but repeating the group name on every member is most of the payload."+ [m dim->group-name]+ (mapv (fn [d]+ (let [group-name (dim->group-name (:id d))+ own-name (dimension-display-name d)]+ (cond-> {:id (:id d) :group group-name}+ (not= own-name group-name) (assoc :name own-name))))+ (:dimensions m)))(defn research-candidates- "Metabot-facing research catalog: each metric with its candidate dimensions inlined- (`:id`, `:name`, `:dimension_interestingness`, ...), plus dimension groups annotated with the- dimension ids they bundle and the metric ids they can slice. Shaped for the- `get_research_candidates` tool so the LLM can pick valid metric/dimension ids before authoring- groups; the FE does not consume this. Accepts the same `:metric-ids`/`:q` options as- [[exploration-data]]."- [opts]- (let [filtered (mapv with-candidate-dimensions (hydrated-metrics opts))- dim->metrics (dimension-id->metric-ids filtered)]- {:metrics (mapv (fn [m]- (assoc (select-keys m llm-metric-cols)- :dimensions (mapv #(select-keys % llm-dimension-cols)- (:dimensions m))))- filtered)- :dimension_groups (mapv (fn [g]- (let [dim-ids (mapv :id (:dimensions g))]- {:name (:name g)- :dimension_interestingness (:dimension_interestingness g)- :dimension_ids dim-ids- :metric_ids (into [] (distinct)- (mapcat dim->metrics dim-ids))}))- (group-dimensions filtered))}))+ "Metabot-facing research catalog detail for the `get_research_candidates` tool: the requested+ metrics, each carrying the dimension ids it can be sliced by, plus the dimension groups those+ ids belong to. Descriptive dimension fields are stated once per group rather than once per+ metric. The FE does not consume this.++ Accepts the same `:metric-ids`/`:q` options as [[exploration-data]]; callers pass at least one+ (the tool layer enforces it). An explicit `:metric-ids` request returns exactly those metrics+ (the tool caps the id count), stamping any that didn't come back — unknown or unreadable — as+ `:missing_metric_ids`. A `:q` request matching more than [[research-candidates-max-metrics]]+ metrics is truncated in [[rank-metrics]] order and stamped+ `{:truncated true :shown <n> :matched <m>}` so the model knows to narrow the search."+ [{:keys [metric-ids] :as opts}]+ (let [filtered (mapv with-candidate-dimensions (hydrated-metrics opts))+ capped (if (seq metric-ids)+ filtered+ (vec (take research-candidates-max-metrics (rank-metrics filtered))))+ groups (group-dimensions capped)+ dim-group (into {} (for [g groups, d (:dimensions g)] [(:id d) (:name g)]))+ missing (vec (distinct (remove (set (map :id capped)) metric-ids)))]+ (cond-> {:metrics (mapv (fn [m]+ (-> (select-keys m llm-metric-cols)+ (update :description truncate-description)+ (assoc :dimensions (llm-metric-dimensions m dim-group))))+ capped)+ :dimension_groups (llm-dimension-groups capped groups)}+ (> (count filtered) (count capped))+ (assoc :truncated true+ :shown (count capped)+ :matched (count filtered))++ (seq missing)+ (assoc :missing_metric_ids missing))))(defn research-groups"Validate Metabot's chosen research groups and return the FE picker payload for them.diff --git a/src/metabase/metabot/agent/profiles.clj b/src/metabase/metabot/agent/profiles.cljindex 48ac1b9b48e..fcec859e8ab 100644--- a/src/metabase/metabot/agent/profiles.clj+++ b/src/metabase/metabot/agent/profiles.clj@@ -237,6 +237,7 @@:skills? false:tools [#'tools/search-tool#'tools/read-resource-tool+ #'tools/list-research-metrics-tool#'tools/get-research-candidates-tool#'tools/add-research-groups-tool#'tools/remove-from-research-plan-tooldiff --git a/src/metabase/metabot/tools.clj b/src/metabase/metabot/tools.cljindex 3de5d1cf39c..05397fb5c4e 100644--- a/src/metabase/metabot/tools.clj+++ b/src/metabase/metabot/tools.clj@@ -53,6 +53,7 @@document-construct-sql-chart-tooldocument-construct-model-chart-tool][tools.explorations+ list-research-metrics-toolget-research-candidates-tooladd-research-groups-toolremove-from-research-plan-tooldiff --git a/src/metabase/metabot/tools/explorations.clj b/src/metabase/metabot/tools/explorations.cljindex 99614b65f2a..4f95b64e6a7 100644--- a/src/metabase/metabot/tools/explorations.clj+++ b/src/metabase/metabot/tools/explorations.clj@@ -68,19 +68,54 @@[context]{:research_plan (format-research-plan context)})-(def ^:private get-research-candidates-schema+(def ^:private list-research-metrics-schema[:map {:closed true}[:q {:optional true} [:maybe :string]]])+(mu/defn ^{:tool-name "list_research_metrics"+ :scope scope/agent-explorations-read}+ list-research-metrics-tool+ "List the metrics available for research: one slim row per metric (id, name, description,+ in_library — a quality signal), best first. Dimensions are not included; pass metric ids from+ this index to `get_research_candidates` to see their candidate dimensions. Pass `q` to filter+ by a case-insensitive substring of a metric or dimension name — e.g. `q: \"region\"` returns+ the metrics that have a Region-like dimension. More than 500 matches are truncated to the top+ 500 with `truncated: true` — narrow with `q`."+ [{:keys [q]} :- list-research-metrics-schema]+ {:output (json/encode (explorations/research-metric-index {:q q}))})++(def ^:private get-research-candidates-schema+ [:map {:closed true}+ [:q {:optional true} [:maybe :string]]+ [:metric_ids {:optional true} [:maybe [:sequential :int]]]])+(mu/defn ^{:tool-name "get_research_candidates":scope scope/agent-explorations-read}get-research-candidates-tool- "List the metrics and dimensions available for research. Each metric lists its candidate- dimensions (id, name, interestingness); each dimension group lists the dimension ids it bundles- and the metric ids it can slice. Use this to choose valid metric and dimension ids before- calling `add_research_groups`. Pass `q` to filter by a search term."- [{:keys [q]} :- get-research-candidates-schema]- {:output (json/encode (explorations/research-candidates {:q q}))})+ "Get the candidate dimensions for chosen research metrics. Pass `metric_ids` (up to 20, from+ `list_research_metrics`) and/or `q` (a case-insensitive substring of a metric or dimension+ name) — at least one is required. Each metric lists the `dimensions` it can be sliced by: the+ `id` to pass to `add_research_groups`, the `group` it belongs to, and a `name` only when this+ metric calls it something other than the group name. Each entry in `dimension_groups` states+ that group's types and interestingness once, plus the `metric_ids` it can slice. Every metric+ and dimension id you pass to `add_research_groups` must come from this tool. Requested ids you+ can't see come back as `missing_metric_ids`; a too-broad `q` returns the top matches with+ `truncated: true` — narrow the search or pass explicit `metric_ids`."+ [{:keys [q metric_ids]} :- get-research-candidates-schema]+ (cond+ (and (str/blank? q) (empty? metric_ids))+ {:output (str "Error: pass metric_ids (up to " explorations/research-candidates-max-metrics+ ", from list_research_metrics) and/or q (a search term). This tool reports"+ " dimensions for metrics you have already chosen; use list_research_metrics or"+ " search to choose them.")}++ (> (count metric_ids) explorations/research-candidates-max-metrics)+ {:output (str "Error: pass at most " explorations/research-candidates-max-metrics+ " metric_ids per call (got " (count metric_ids)+ "). Split the request into multiple calls.")}++ :else+ {:output (json/encode (explorations/research-candidates {:metric-ids metric_ids :q q}))}))(def ^:private add-research-groups-schema[:map {:closed true}diff --git a/test/metabase/explorations/impl_test.clj b/test/metabase/explorations/impl_test.cljindex 21f4a1ed0c3..36641f39ba0 100644--- a/test/metabase/explorations/impl_test.clj+++ b/test/metabase/explorations/impl_test.clj@@ -1,5 +1,6 @@(ns metabase.explorations.impl-test(:require+ [clojure.string :as str][clojure.test :refer :all][metabase.explorations.impl :as explorations.impl][metabase.lib.core :as lib]@@ -7,6 +8,7 @@[metabase.test :as mt][metabase.test.fixtures :as fixtures][metabase.util :as u]+ [metabase.util.json :as json][toucan2.core :as t2]))(use-fixtures :once (fixtures/initialize :db :test-users))@@ -119,29 +121,217 @@:dimension-interestingness interestingness :sources sources})(def ^:private synthetic-metrics- [{:id 1 :name "Revenue" :description "rev" :result_column_name "count"+ [{:id 1 :name "Revenue" :description "rev" :result_column_name "count" :in_library true:dimensions [(dim "region-1" "Region" 0.9 [region-source])(dim "plan-1" "Plan" 0.5 [plan-source])]}- {:id 2 :name "Churn" :description "churn" :result_column_name "count"+ {:id 2 :name "Churn" :description "churn" :result_column_name "count" :in_library false:dimensions [(dim "region-2" "Region" 0.9 [region-source])]}])+(defn- synthetic-hydrated+ "Stand-in for the metric-loading functions over [[synthetic-metrics]], honouring the same+ `:metric-ids` restriction and `:q` match the real ones do — so tests can exercise filtering,+ and an id no metric has behaves like one the user can't read."+ [{:keys [metric-ids q]}]+ (cond->> synthetic-metrics+ (seq metric-ids) (filterv (comp (set metric-ids) :id))+ (not (str/blank? q)) (filterv #(@#'explorations.impl/metric-matches-search?+ % (u/lower-case-en q)))))+(defmacro ^:private with-synthetic-metrics [& body]- `(with-redefs [explorations.impl/hydrated-metrics (fn [~'_] synthetic-metrics)]+ `(mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics synthetic-hydrated+ explorations.impl/index-metrics synthetic-hydrated]~@body))+(deftest research-metric-index-test+ (with-synthetic-metrics+ (testing "slim rows only — id/name/description/in_library, no dimensions"+ (is (= {:metrics [{:id 1 :name "Revenue" :description "rev" :in_library true}+ {:id 2 :name "Churn" :description "churn" :in_library false}]}+ (explorations.impl/research-metric-index {}))))))++(deftest research-metric-index-truncates-descriptions-test+ (let [long-desc (apply str (repeat 40 "long desc "))+ stub [{:id 1 :name "M" :description long-desc :in_library false :dimensions []}]]+ (mt/with-dynamic-fn-redefs [explorations.impl/index-metrics (fn [_] stub)]+ (let [[{:keys [description]}] (:metrics (explorations.impl/research-metric-index {}))]+ (is (= (inc @#'explorations.impl/catalog-description-max-length) (count description)))+ (is (str/ends-with? description "…"))))))++(deftest research-metric-index-cap-test+ ;; 505 matches: ids 0-4 in the library, the rest not.+ (let [many (vec (for [i (range 505)]+ {:id i :name (str "M" i) :description nil+ :in_library (< i 5)+ :dimensions [(dim (str "d" i) (str "D" i) (+ 0.1 (/ i 1000.0))+ [{:source i}])]}))]+ (mt/with-dynamic-fn-redefs [explorations.impl/index-metrics (fn [_] many)]+ (let [{:keys [metrics truncated shown matched]} (explorations.impl/research-metric-index {})]+ (testing "an over-cap index is truncated and stamped"+ (is (true? truncated))+ (is (= 500 shown))+ (is (= 505 matched))+ (is (= 500 (count metrics))))+ (testing "library metrics survive the cut, ranked first"+ (is (= [4 3 2 1 0] (mapv :id (take 5 metrics)))))))))++(deftest research-metric-index-ranks-below-the-cap-too-test+ (testing "an under-cap index is ranked the same way an over-cap one is, so a metric doesn't+ move just because the instance grew past the cap"+ (let [few (vec (for [i (range 6)]+ {:id i :name (str "M" i) :description nil+ :in_library (< i 2)+ :dimensions [(dim (str "d" i) (str "D" i) (+ 0.1 (/ i 100.0))+ [{:source i}])]}))]+ (mt/with-dynamic-fn-redefs [explorations.impl/index-metrics (fn [_] few)]+ (let [{:keys [metrics truncated]} (explorations.impl/research-metric-index {})]+ (is (nil? truncated))+ (is (= [1 0 5 4 3 2] (mapv :id metrics))))))))++(deftest research-metric-index-ranks-on-candidate-dimensions-test+ (testing "a metric whose only dimension scores below min-interestingness has no candidates, so+ it ranks with the dimension-less metrics rather than above them"+ ;; Input order matters: `bare` is listed first, so it only stays ahead of `sub` if `sub`+ ;; scored nothing. Ranking on raw (unfiltered) dimensions would lift `sub` above it.+ (let [bare {:id 2 :name "Bare" :description nil :in_library false :dimensions []}+ sub {:id 1 :name "Sub" :description nil :in_library false+ :dimensions [(dim "d1" "D1" 0.09 [{:source 1}])]}+ real {:id 3 :name "Real" :description nil :in_library false+ :dimensions [(dim "d3" "D3" 0.2 [{:source 3}])]}]+ (mt/with-dynamic-fn-redefs [explorations.impl/index-metrics (fn [_] [bare sub real])]+ (is (= [3 2 1] (mapv :id (:metrics (explorations.impl/research-metric-index {})))))))))+(deftest research-candidates-test(with-synthetic-metrics- (let [{:keys [metrics dimension_groups]} (explorations.impl/research-candidates {})]- (testing "each metric inlines its candidate dimensions with ids + interestingness"- (is (= [{:id 1 :name "Revenue"} {:id 2 :name "Churn"}]- (mapv #(select-keys % [:id :name]) metrics)))- (is (= ["region-1" "plan-1"] (mapv :id (:dimensions (first metrics)))))- (is (= 0.9 (:dimension-interestingness (first (:dimensions (first metrics)))))))- (testing "dimension groups carry their member dimension ids and the metrics they slice"- (let [by-name (u/index-by :name dimension_groups)]- (is (= #{"region-1" "region-2"} (set (:dimension_ids (get by-name "Region")))))- (is (= #{1 2} (set (:metric_ids (get by-name "Region")))))- (is (= #{1} (set (:metric_ids (get by-name "Plan"))))))))))+ (let [{:keys [metrics dimension_groups truncated]} (explorations.impl/research-candidates+ {:metric-ids [1 2]})]+ (testing "each metric carries the dimension ids it can be sliced by, tagged with their group"+ (is (= [{:id 1 :name "Revenue" :description "rev" :result_column_name "count"+ :dimensions [{:id "region-1" :group "Region"} {:id "plan-1" :group "Plan"}]}+ {:id 2 :name "Churn" :description "churn" :result_column_name "count"+ :dimensions [{:id "region-2" :group "Region"}]}]+ metrics)))+ (testing "groups state the descriptive fields once, and which metrics they slice"+ (is (= [{:name "Region" :effective_type "type/Text" :semantic_type nil+ :interestingness 0.9 :metric_ids [1 2]}+ {:name "Plan" :effective_type "type/Text" :semantic_type nil+ :interestingness 0.5 :metric_ids [1]}]+ dimension_groups)))+ (testing "an explicit metric-ids request is not truncated"+ (is (nil? truncated))))))++(deftest research-candidates-renamed-dimension-test+ (testing "a dimension a metric renamed carries that metric's name; ones matching their group+ don't repeat it"+ (let [stub [{:id 1 :name "Revenue" :description nil :result_column_name "count"+ :in_library false :dimensions [(dim "region-1" "Region" 0.9 [region-source])]}+ {:id 2 :name "Churn" :description nil :result_column_name "count"+ :in_library false :dimensions [(dim "region-2" "Territory" 0.9 [region-source])]}]]+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] stub)]+ (let [{:keys [metrics]} (explorations.impl/research-candidates {:metric-ids [1 2]})]+ (is (= [[{:id "region-1" :group "Region"}]+ [{:id "region-2" :group "Region" :name "Territory"}]]+ (mapv :dimensions metrics))))))))++(deftest research-candidates-two-dimensions-in-one-group-test+ (testing "a metric with two dimensions in the same group keeps both — neither is dropped"+ (let [stub [{:id 1 :name "Revenue" :description nil :result_column_name "count"+ :in_library false+ :dimensions [(dim "region-a" "Region" 0.9 [region-source])+ (dim "region-b" "Region (billing)" 0.9 [region-source])]}]]+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] stub)]+ (let [{:keys [metrics dimension_groups]} (explorations.impl/research-candidates+ {:metric-ids [1]})]+ (is (= ["region-a" "region-b"] (mapv :id (:dimensions (first metrics)))))+ (is (= [[1]] (mapv :metric_ids dimension_groups))))))))++(deftest research-candidates-heterogeneous-group-test+ (testing "when a group's dimensions don't agree on their types (group-by-source unions+ transitively, so a group can span Fields) nothing is stated at the group level"+ (let [shared {:source 1}+ stub [{:id 1 :name "Revenue" :description nil :result_column_name "count"+ :in_library false+ :dimensions [(dim "d1" "Region" 0.9 [shared])+ (assoc (dim "d2" "Signup" 0.9 [shared])+ :effective-type "type/DateTime")]}]]+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] stub)]+ (let [groups (:dimension_groups (explorations.impl/research-candidates {:metric-ids [1]}))]+ (is (= 1 (count groups)))+ (is (not (contains? (first groups) :effective_type)))+ (is (not (contains? (first groups) :semantic_type))))))))++(deftest research-candidates-truncates-descriptions-test+ (testing "descriptions are capped here too — 20 essay-length ones would otherwise be most of+ the payload"+ (let [stub [{:id 1 :name "M" :description (apply str (repeat 40 "long desc "))+ :result_column_name "count" :in_library false :dimensions []}]]+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] stub)]+ (let [[{:keys [description]}] (:metrics (explorations.impl/research-candidates+ {:metric-ids [1]}))]+ (is (= (inc @#'explorations.impl/catalog-description-max-length) (count description))))))))++(deftest research-candidates-metric-ids-and-q-test+ (with-synthetic-metrics+ (testing "metric_ids and q compose — q narrows the requested metrics rather than replacing them"+ (is (= [1] (mapv :id (:metrics (explorations.impl/research-candidates+ {:metric-ids [1 2] :q "plan"}))))))))++(deftest research-candidates-missing-metric-ids-test+ (with-synthetic-metrics+ (testing "ids the user can't see are reported rather than silently dropped"+ (let [{:keys [metrics missing_metric_ids]} (explorations.impl/research-candidates+ {:metric-ids [1 999 1000]})]+ (is (= [1] (mapv :id metrics)))+ (is (= [999 1000] missing_metric_ids))))+ (testing "nothing is stamped when every requested id came back"+ (is (nil? (:missing_metric_ids (explorations.impl/research-candidates+ {:metric-ids [1 2]})))))))++(deftest research-candidates-q-truncation-test+ ;; 25 matches: ids 0-4 in the library, the rest not; interestingness rises with id.+ (let [many (vec (for [i (range 25)]+ {:id i :name (str "M" i) :description nil :result_column_name "count"+ :in_library (< i 5)+ :dimensions [(dim (str "d" i) (str "D" i) (+ 0.1 (/ i 100.0))+ [{:source i}])]}))]+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] many)]+ (let [{:keys [metrics truncated shown matched]} (explorations.impl/research-candidates+ {:q "m"})]+ (testing "a q match beyond the cap is truncated and stamped"+ (is (true? truncated))+ (is (= 20 shown))+ (is (= 25 matched))+ (is (= 20 (count metrics))))+ (testing "library metrics rank first, then by interestingness"+ (is (= [4 3 2 1 0] (mapv :id (take 5 metrics))))+ (is (= 24 (:id (nth metrics 5)))))))))++;;; The whole point of the two-tier split (UXW-4967) is that neither research tool can hand the+;;; LLM an unbounded blob. These guard that: the payloads are a function of the caps, not of how+;;; many metrics the instance has. Sizes are generous ceilings — they catch a return to+;;; per-metric-inlined dimensions (which ran ~1KB/metric over the whole catalog), not drift.++(defn- realistic-metrics+ "`n` metrics shaped like real ones: uuid dimension ids, prose descriptions, `d` dimensions each+ drawn from a small pool of shared groups (as same-table metrics really do share Fields)."+ [n d]+ (vec (for [i (range n)]+ {:id i :name (str "Metric number " i) :in_library false :result_column_name "sum"+ :description (apply str (repeat 30 "some prose describing this metric. "))+ :dimensions (vec (for [j (range d)]+ (dim (str (random-uuid)) (str "Dimension " j) (+ 0.1 (/ j 100.0))+ [{:source j}])))})))++(deftest research-metric-index-payload-is-bounded-test+ (mt/with-dynamic-fn-redefs [explorations.impl/index-metrics (fn [_] (realistic-metrics 5000 8))]+ (let [payload (explorations.impl/research-metric-index {})]+ (is (true? (:truncated payload)))+ (is (> 120000 (count (json/encode payload)))))))++(deftest research-candidates-payload-is-bounded-test+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] (realistic-metrics 5000 8))]+ (let [payload (explorations.impl/research-candidates {:q "metric"})]+ (is (true? (:truncated payload)))+ (is (> 20000 (count (json/encode payload)))))))(deftest research-groups-metric-anchored-test(with-synthetic-metrics@@ -224,7 +414,7 @@(deftest exploration-data-no-dangling-dimension-ids-test(testing "metric :dimension_ids and :dimension_groups apply the same interestingness filter"- (with-redefs [explorations.impl/hydrated-metrics (fn [_] threshold-metrics)]+ (mt/with-dynamic-fn-redefs [explorations.impl/hydrated-metrics (fn [_] threshold-metrics)](let [{:keys [metrics dimension_groups]} (explorations.impl/exploration-data {})metric-dim-ids (set (:dimension_ids (first metrics)))group-dim-ids (into #{} (mapcat #(map :id (:dimensions %))) dimension_groups)]diff --git a/test/metabase/metabot/tools/explorations_test.clj b/test/metabase/metabot/tools/explorations_test.cljindex c0e34fcd5fb..bb280a2ff4a 100644--- a/test/metabase/metabot/tools/explorations_test.clj+++ b/test/metabase/metabot/tools/explorations_test.clj@@ -11,7 +11,8 @@[metabase.util.json :as json]))(def ^:private exploration-tool-vars- [#'tools.explorations/get-research-candidates-tool+ [#'tools.explorations/list-research-metrics-tool+ #'tools.explorations/get-research-candidates-tool#'tools.explorations/add-research-groups-tool#'tools.explorations/remove-from-research-plan-tool#'tools.explorations/set-exploration-name-tool@@ -39,6 +40,23 @@[result](json/decode+kw (:output result)))+(deftest ^:parallel get-research-candidates-requires-a-filter-test+ (testing "an argument-less call is rejected with an instructive error (the unfiltered catalog is+ the unbounded payload this tool's contract exists to prevent)"+ (doseq [args [{} {:q nil} {:q ""} {:metric_ids nil} {:metric_ids []} {:q " " :metric_ids []}]]+ (testing (pr-str args)+ (let [{:keys [output]} (tools.explorations/get-research-candidates-tool args)]+ (is (str/starts-with? output "Error:"))+ (is (str/includes? output "list_research_metrics")))))))++(deftest ^:parallel get-research-candidates-caps-metric-ids-test+ (testing "more than the per-call cap of metric_ids is rejected, naming the cap and the count"+ (let [{:keys [output]} (tools.explorations/get-research-candidates-tool+ {:metric_ids (vec (range 21))})]+ (is (str/starts-with? output "Error:"))+ (is (str/includes? output "at most 20"))+ (is (str/includes? output "got 21")))))+(deftest ^:parallel remove-from-research-plan-tool-test(testing "echoes the block ids the agent asked to remove (pure-echo; the FE applies them)"(is (= {:block_ids ["metric:42" "dim:7"] :members nil :timeline_ids nil}