The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Let Metabot add model-based charts to documents (#82990)
Andrei Simionescuffa7eecd6c4c
diff --git a/resources/metabot/prompts/system/document-generate-content.selmer b/resources/metabot/prompts/system/document-generate-content.selmerindex d1bc3002db7..0965071b27d 100644--- a/resources/metabot/prompts/system/document-generate-content.selmer+++ b/resources/metabot/prompts/system/document-generate-content.selmer@@ -14,6 +14,9 @@ IF using an existing model, identify the relevant model using the `list_availablTHEN use the `document_construct_model_chart` tool (IF a relevant model is found) to generate the chart.OR use the `document_schema_collect` tool (IF a relevant model is NOT found) as a first step to generating a SQL chart.+`document_construct_model_chart` queries the model as its `source-card`: load the `construct-notebook-query-core`+and `construct-notebook-query-advanced` skills before writing its `query`.+## IF using custom SQL:IF writing custom SQL, FIRST use the `document_schema_collect` to get the schema information for thedatabase you will be querying against@@ -44,3 +47,5 @@ you to `document_construct_sql_chart`.- For date columns, extract year with `YEAR(column_name)` and month with `MONTH(column_name)` rather than EXTRACT().{% include "metabot/prompts/shared/prompt_snippets/field-references.selmer" %}++{% include "metabot/prompts/shared/skills.selmer" %}diff --git a/resources/metabot/skills/construct-notebook-query-advanced.md b/resources/metabot/skills/construct-notebook-query-advanced.mdindex ec2d0b80436..1270d3d884d 100644--- a/resources/metabot/skills/construct-notebook-query-advanced.md+++ b/resources/metabot/skills/construct-notebook-query-advanced.md@@ -2,7 +2,7 @@id: construct-notebook-query-advancedtitle: Construct notebook query — advanceddescription: Joins (explicit and implicit), multi-stage queries, querying saved questions/models with source-card, metrics, measures, segments, expressions, and aggregation references in construct_notebook_query — load when a query needs any of these.-tools: [construct_notebook_query]+tools: [construct_notebook_query, document_construct_model_chart]priority: 50---# Construct Query Reference — Advanceddiff --git a/resources/metabot/skills/construct-notebook-query-core.md b/resources/metabot/skills/construct-notebook-query-core.mdindex 6798d846b08..7d92e70ae9e 100644--- a/resources/metabot/skills/construct-notebook-query-core.md+++ b/resources/metabot/skills/construct-notebook-query-core.md@@ -2,7 +2,7 @@id: construct-notebook-query-coretitle: Construct notebook query — coredescription: Building or editing a notebook (MBQL) query with construct_notebook_query — load before your first query so you get the clause shape, field references, and the rules/anti-patterns right.-tools: [construct_notebook_query]+tools: [construct_notebook_query, document_construct_model_chart]priority: 60---# Construct Query Reference — Core@@ -19,6 +19,8 @@ Return:> The Slackbot variant of this tool has a different contract: `reasoning` is **required**, `title` is optional and there's no `description`; it uses `display` (a Slack-specific visualization-type enum) instead of `visualization`. See the Slackbot system prompt for its exact argument set.+> `document_construct_model_chart` takes the same `query`, with `name` instead of `title` and a required `viz_settings` instead of `visualization`.+## Minimal example — count of orders by month```jsondiff --git a/resources/metabot/skills/construct-notebook-query-operators.md b/resources/metabot/skills/construct-notebook-query-operators.mdindex a9a8408ec34..92657ef32e0 100644--- a/resources/metabot/skills/construct-notebook-query-operators.md+++ b/resources/metabot/skills/construct-notebook-query-operators.md@@ -2,7 +2,7 @@id: construct-notebook-query-operatorstitle: Construct notebook query — operator catalogsdescription: The full catalog of aggregation, filter, expression, and temporal-unit operators for construct_notebook_query — load when you need the exact name/arity of a specific operator or the list of temporal units.-tools: [construct_notebook_query]+tools: [construct_notebook_query, document_construct_model_chart]priority: 40---# Construct Query Reference — Operator catalogsdiff --git a/src/metabase/metabot/tools/construct.clj b/src/metabase/metabot/tools/construct.cljindex 950bb5532ef..b7309d58005 100644--- a/src/metabase/metabot/tools/construct.clj+++ b/src/metabase/metabot/tools/construct.clj@@ -97,6 +97,13 @@::mr/deliberately-open true:description "An LLM-authored MBQL 5 query, structurally unvalidated here; real validation happens at the entry-point boundaries."}])+(def LLMQueryArgument+ "Schema for the `:query` argument of `construct_notebook_query` and the tools that reuse it.++ Validation stays a fully open, property-less `:map` (the repair layer fixes LLM shortcuts); the `:json-schema`+ override only changes what the LLM sees. See [[construct-notebook-query-json-schema]]."+ (mu/with LLMExternalQuery {:json-schema construct-notebook-query-json-schema}))+(def ^:private construct-notebook-query-args-schema"Args schema for `construct_notebook_query`.@@ -113,9 +120,7 @@omits `:source_entity` and `:referenced_entities` — the query body is self-describing."[:map {:closed true}[:reasoning {:optional true} :string]- ;; Validation stays a fully open, property-less `:map` (the repair layer fixes LLM shortcuts); the- ;; `:json-schema` override only changes what the LLM sees. See [[construct-notebook-query-json-schema]].- [:query (mu/with LLMExternalQuery {:json-schema construct-notebook-query-json-schema})]+ [:query LLMQueryArgument][:visualization {:optional true} construct-visualization-schema][:title :string][:description :string]])diff --git a/src/metabase/metabot/tools/document.clj b/src/metabase/metabot/tools/document.cljindex f9144339caa..a4c1511fca6 100644--- a/src/metabase/metabot/tools/document.clj+++ b/src/metabase/metabot/tools/document.clj@@ -176,16 +176,10 @@(metabot.tools.u/handle-agent-error e))))(def ^:private model-chart-schema- "Schema for `document_construct_model_chart`. Mirrors `construct_notebook_query`'s- representations format: `:query` is a YAML string in MBQL 5 representations format.-- Per `repr-plan.md` step 13, `:source_entity` is no longer part of the contract — the YAML- query is self-describing (carries `database:` at the top level and full portable FK paths- everywhere else)."[:map {:closed true}[:name :string][:description :string]- [:query :string]+ [:query construct-tools/LLMQueryArgument][:viz_settings [:map {:closed true}[:chart_type chart-type-enum]]]])@@ -198,6 +192,8 @@(let [chart-type (get viz_settings :chart_type)result (construct-tools/construct-notebook-query-tool{:query query+ :title name+ :description description:visualization {:chart_type chart-type}})structured (or (:structured-output result) (:structured_output result))query-id (:query-id structured)diff --git a/test/metabase/metabot/agent/prompts_test.clj b/test/metabase/metabot/agent/prompts_test.cljindex ae6a446cf44..81ff091669b 100644--- a/test/metabase/metabot/agent/prompts_test.clj+++ b/test/metabase/metabot/agent/prompts_test.clj@@ -3,7 +3,11 @@[clojure.java.io :as io][clojure.string :as str][clojure.test :refer :all]- [metabase.metabot.agent.prompts :as prompts]))+ [metabase.api-scope.core :as api-scope]+ [metabase.metabot.agent.profiles :as profiles]+ [metabase.metabot.agent.prompts :as prompts]+ [metabase.metabot.scope :as scope]+ [metabase.metabot.skills :as skills]))(deftest ^:parallel load-system-prompt-template-test(testing "loads internal.selmer template"@@ -177,6 +181,16 @@(is (not (str/includes? content "{% include"))"unresolved {% include %} tags mean rendering failed and the raw template was returned"))))+(deftest ^:parallel build-system-message-content-document-skills-test+ (testing "the document prompt lists the notebook query skills its model chart tool needs"+ (binding [scope/*current-user-scope* api-scope/unrestricted]+ (let [profile (profiles/get-profile :document-generate-content)+ content (prompts/build-system-message-content profile {} (profiles/profile->tools profile []) [])]+ (doseq [skill-id [:construct-notebook-query-core+ :construct-notebook-query-advanced+ :construct-notebook-query-operators]]+ (is (str/includes? content (:description (skills/get-skill skill-id)))))))))+(deftest ^:parallel build-system-message-content-test-9(testing "renders sql querying template with literal model syntax"(let [profile {:prompt-template "sql-querying-only.selmer"}diff --git a/test/metabase/metabot/api/document_test.clj b/test/metabase/metabot/api/document_test.cljindex ba929f2b992..97953c9d71f 100644--- a/test/metabase/metabot/api/document_test.clj+++ b/test/metabase/metabot/api/document_test.clj@@ -1,7 +1,10 @@(ns metabase.metabot.api.document-test(:require+ [clojure.string :as str][clojure.test :refer :all][metabase.analytics.snowplow-test :as snowplow-test]+ [metabase.lib.core :as lib]+ [metabase.lib.metadata :as lib.metadata][metabase.llm.test-util :as llm.tu][metabase.metabot.scope :as scope][metabase.metabot.self.openrouter :as openrouter]@@ -118,6 +121,51 @@:draft_card {:name "Orders by day"}}response))))))))+(deftest generate-content-model-chart-round-trip-test+ (testing "the model loads the query skills, then charts a model by its column names"+ (let [mp (mt/metadata-provider)]+ (mt/with-temp [:model/Card model {:type :model+ :dataset_query (lib/query mp (lib.metadata/table mp (mt/id :orders)))}]+ (mt/with-temporary-setting-values [llm-providers llm.tu/default-connections+ llm-metabot-provider test-provider]+ (let [requests (atom [])+ responses [[{:type :tool-input+ :id "t1"+ :function "load_skill"+ :arguments {:ids ["construct-notebook-query-core" "construct-notebook-query-advanced"]}}]+ [{:type :tool-input+ :id "t2"+ :function "document_construct_model_chart"+ :arguments {:name "Orders by month"+ :description "Count of orders per month."+ :query {"lib/type" "mbql/query"+ "stages" [{"lib/type" "mbql.stage/mbql"+ "source-card" (:entity_id model)+ "aggregation" [["count" {}]]+ "breakout" [["field" {"temporal-unit" "month"}+ "CREATED_AT"]]}]}+ :viz_settings {:chart_type "line"}}}]]]+ (mt/with-dynamic-fn-redefs [openrouter/openrouter+ (fn [request]+ (let [n (count (swap! requests conj request))]+ (mut/mock-llm-response+ (get responses (dec n) [{:type :text :text "Chart created"}]))))]+ (let [response (mt/user-http-request :crowberto+ :post 200 "metabot/document/generate-content"+ {:instructions "Chart orders by month"})+ [first-request second-request] @requests]+ (is (some #{"load_skill"} (map :tool-name (:tools first-request))))+ (is (some #(and (string? %)+ (str/includes? % "<skill id=\"construct-notebook-query-advanced\">"))+ (tree-seq coll? seq (:input second-request)))+ "the skill body reaches the model before it writes the query")+ (is (=? {:error nil+ :draft_card {:name "Orders by month"+ :display "line"+ :database_id (mt/id)+ :dataset_query {:stages [{:source-card (:id model)}]}}}+ response))))))))))+(deftest generate-content-snowplow-test(mt/with-temporary-setting-values [llm-providers llm.tu/default-connectionsllm-metabot-provider test-provider]diff --git a/test/metabase/metabot/skills_test.clj b/test/metabase/metabot/skills_test.cljindex d98f93cf790..3e2f5687988 100644--- a/test/metabase/metabot/skills_test.clj+++ b/test/metabase/metabot/skills_test.clj@@ -60,7 +60,7 @@(is (= "construct-notebook-query-core" (name (:id s))))(is (string? (:title s)))(is (str/includes? (:description s) "construct_notebook_query"))- (is (= ["construct_notebook_query"] (:tools s)))+ (is (= ["construct_notebook_query" "document_construct_model_chart"] (:tools s)))(is (str/includes? (:body s) "Universal clause shape"));; the YAML frontmatter delimiter must not leak into the body(is (not (str/starts-with? (:body s) "---"))))))diff --git a/test/metabase/metabot/tools/document_test.clj b/test/metabase/metabot/tools/document_test.cljindex f1e9f42476a..d282fc377a6 100644--- a/test/metabase/metabot/tools/document_test.clj+++ b/test/metabase/metabot/tools/document_test.clj@@ -1,6 +1,8 @@(ns metabase.metabot.tools.document-test(:require[clojure.test :refer :all]+ [metabase.lib.core :as lib]+ [metabase.lib.metadata :as lib.metadata][metabase.metabot.table-utils :as table-utils][metabase.metabot.tools.construct :as construct-tools][metabase.metabot.tools.document :as document-tools]@@ -118,31 +120,32 @@(is (re-find #"missing_table" (:output result)))))))(deftest document-construct-model-chart-tool-test- (testing "builds chart draft payload from model query"- (mt/with-dynamic-fn-redefs [construct-tools/construct-notebook-query-tool- (fn [_]- {:structured-output {:query-id "3"- :query {:database 1- :type "query"}}})]- ;; `construct-notebook-query-tool` is stubbed above, so the YAML string is an opaque- ;; placeholder — it only needs to be a string.- (let [result (document-tools/document-construct-model-chart-tool- {:name "Test Name"- :description "Test Desc"- :query ""- :viz_settings {:chart_type "bar"}})- structured (:structured-output result)]- (is (some? structured)- "structured-output present -> a successful, terminal-eligible call")- (is (= "document_construct_model_chart" (:tool structured)))- (is (= "Test Name" (:name structured)))- (is (= "Test Desc" (:description structured)))- (is (= "bar" (:display structured)))- (is (= "bar" (:chart_type structured)))- (is (= :chart-draft (:result-type structured)))- (is (= {:database 1- :type "query"}- (:dataset_query structured)))))))+ (testing "builds chart draft payload from a query on a model"+ (let [mp (mt/metadata-provider)]+ (mt/with-current-user (mt/user->id :crowberto)+ (mt/with-temp [:model/Card model {:type :model+ :dataset_query (lib/query mp (lib.metadata/table mp (mt/id :orders)))}]+ (let [result (document-tools/document-construct-model-chart-tool+ {:name "Test Name"+ :description "Test Desc"+ :query {:lib/type "mbql/query"+ :stages [{:lib/type "mbql.stage/mbql"+ :source-card (:entity_id model)+ :aggregation [["count" {}]]+ :breakout [["field" {:temporal-unit "month"} "CREATED_AT"]]}]}+ :viz_settings {:chart_type "bar"}})+ structured (:structured-output result)]+ (is (some? structured)+ "structured-output present -> a successful, terminal-eligible call")+ (is (= "document_construct_model_chart" (:tool structured)))+ (is (= "Test Name" (:name structured)))+ (is (= "Test Desc" (:description structured)))+ (is (= "bar" (:display structured)))+ (is (= "bar" (:chart_type structured)))+ (is (= :chart-draft (:result-type structured)))+ (is (=? {:database (mt/id)+ :stages [{:source-card (:id model)}]}+ (:dataset_query structured)))))))))(deftest document-construct-model-chart-new-chart-types-test(testing "the tool schema accepts newly added chart types"@@ -155,7 +158,7 @@(let [result (document-tools/document-construct-model-chart-tool{:name "Test Name":description "Test Desc"- :query ""+ :query {:lib/type "mbql/query" :stages []}:viz_settings {:chart_type chart-type}})structured (:structured-output result)](is (= chart-type (:display structured)))diff --git a/test/metabase/metabot/tools_test.clj b/test/metabase/metabot/tools_test.cljindex fe791bbf8a5..f51934f75da 100644--- a/test/metabase/metabot/tools_test.clj+++ b/test/metabase/metabot/tools_test.clj@@ -95,6 +95,8 @@(is (contains? tools "list_available_fields"))(is (contains? tools "get_field_values"))(is (contains? tools "document_construct_model_chart"))+ (is (contains? tools "load_skill")+ "the model chart tool's query skills make load_skill available")(testing "both SQL tools need the SQL capability, like the tools they delegate to"(is (not (contains? tools "document_construct_sql_chart")))(is (contains? sql-tools "document_construct_sql_chart")))