Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

BOT-2010: Post Metabot's Slack answer as a markdown block (#80652)

Paolo Rascunàa9f25e8645cb
diff --git a/resources/metabot/prompts/system/slackbot.selmer b/resources/metabot/prompts/system/slackbot.selmer
index 95339385d1e..155bf622578 100644
--- a/resources/metabot/prompts/system/slackbot.selmer
+++ b/resources/metabot/prompts/system/slackbot.selmer
@@ -19,17 +19,17 @@ Be direct and concise. Get users to insights efficiently while providing transpa
**Important: You cannot create inline visualizations, alerts, or dashboard subscriptions.** You do not have the ability to render visualizations or create alerts. Help users find existing dashboards and questions through search instead.
{% endif %}
-**Slack Markdown**: Use Slack-specific formatting exclusively:
+**Slack Markdown**: Slack renders standard markdown for text styling, but keeps its own syntax for links and mentions. Follow both sets of rules below exactly.
**Text Formatting**:
-- `*text*` for bold text (single asterisks only)
+- `**text**` for bold text (double asterisks — a single asterisk renders as italic, not bold)
- `_text_` for italic text (underscores only)
-- `~text~` for strikethrough text
+- `~~text~~` for strikethrough text (double tildes — a single tilde does not render at all)
- `` `text` `` for inline code
-- ```text``` or ```language\ncode\n``` for code blocks
+- ```` ```\ntext\n``` ```` or ```` ```language\ncode\n``` ```` for code blocks
**Lists**:
-- `• Item` or `- Item` for bullet lists
+- `- Item` for bullet lists
- `1. Item` for numbered lists
- Indent with spaces for nested lists
- NEVER add extra blank lines after list items - use single line breaks only
@@ -45,8 +45,8 @@ Be direct and concise. Get users to insights efficiently while providing transpa
- Use the metabase:// protocol for all Metabase entity links: `<metabase://dashboard/123|Dashboard Name>`
- This applies to dashboards, questions, metrics, models, tables, charts, and queries
-**Don't Use**: These standard markdown formats don't work in Slack:
-- `**text**` (double asterisks for bold)
+**Don't Use**: Never use these formats:
+- `*text*` (single asterisks — renders as italic, not bold)
- `__text__` (double underscores for bold)
- `[text](url)` (standard markdown links)
- `# Headers` (markdown headers)
@@ -391,8 +391,8 @@ For multiple files, list each model on its own line:
```
Your data has been uploaded:
-• 📦 <metabase://model/{id1}|{display_name1}>
-• 📦 <metabase://model/{id2}|{display_name2}>
+- 📦 <metabase://model/{id1}|{display_name1}>
+- 📦 <metabase://model/{id2}|{display_name2}>
{initial_analysis}
```
diff --git a/src/metabase/slackbot/channel.clj b/src/metabase/slackbot/channel.clj
index 8b4556b846f..20140218ccd 100644
--- a/src/metabase/slackbot/channel.clj
+++ b/src/metabase/slackbot/channel.clj
@@ -29,20 +29,30 @@
See https://docs.slack.dev/reference/block-kit/blocks/section-block."
3000)
+(def markdown-text-limit
+ "Slack documents this as \"the cumulative limit for all `markdown` blocks in a single payload\" in characters.
+ Today Slack enforces the figure per block rather than cumulatively.
+ Budgeting cumulatively is a deliberate choice: enforcement can catch up with the documentation
+ without notice, which would turn messages that fit today into rejections.
+ See https://docs.slack.dev/reference/block-kit/blocks/markdown-block
+ https://github.com/metabase/metabase/pull/80652#discussion_r3861967932."
+ 12000)
+
(defn- final-text-blocks
"Build the leading text block(s) for a finalized non-streaming Slack message."
[text]
(if (str/blank? text)
[]
- [{:type "section"
- :text {:type "mrkdwn"
- :text text}}]))
+ ;; Slack expands one `markdown` block into several as it renders: a heading becomes a `header`,
+ ;; a thematic break a `divider`, a table a `table` -- and those count against the 50-block limit.
+ [{:type "markdown"
+ :text text}]))
(defn- truncation-notice
"Copy explaining that an answer was cut short.
`url` points at the Metabase instance; nil leaves the sentence standing without a link."
[url]
- (str "_This answer was longer than " section-text-limit " characters. That is too long to post in "
+ (str "_This answer was longer than " markdown-text-limit " characters. That is too long to post in "
"Slack, so I cut it short._\n\n"
"Ask a narrower question so the answer comes back smaller"
(if url
@@ -126,7 +136,7 @@
;; `final-text` doubles as the message's `:text` -- Slack's notification preview, and
;; what `streaming/thread->history` replays to the model as the assistant's own words.
;; The notice stays out of it, so the model is never told it said this.
- final-text (u.str/elide answer section-text-limit)
+ final-text (u.str/elide answer markdown-text-limit)
;; Derived rather than re-tested: `elide` returns `answer` itself when it fits, so
;; the notice cannot disagree with whether a cut actually happened.
truncated? (not= final-text answer)
@@ -138,7 +148,7 @@
final-text :blocks final-blocks)]
(when truncated?
(log/infof "[slackbot] channel answer truncated (answer_length=%d limit=%d)"
- (count answer) section-text-limit)
+ (count answer) markdown-text-limit)
(analytics/inc! :metabase-slackbot/responses-truncated))
(when-let [res-ts (:ts res)]
(metabot.persistence/set-response-slack-msg-id! assistant-msg-id res-ts))
diff --git a/test/metabase/metabot/agent/prompt_gating_test.clj b/test/metabase/metabot/agent/prompt_gating_test.clj
index 1ca2a9e62ba..727ca4146f2 100644
--- a/test/metabase/metabot/agent/prompt_gating_test.clj
+++ b/test/metabase/metabot/agent/prompt_gating_test.clj
@@ -213,6 +213,22 @@
(is (not (re-find #"You cannot build custom queries" rendered)))
(is (not (re-find #"You cannot create inline visualizations" rendered))))))
+(deftest ^:parallel slackbot-prompt-teaches-standard-markdown-test
+ ;; The channel answer goes in a `markdown` block, which renders standard markdown rather than
+ ;; Slack's `mrkdwn`. Measured against the API: `*one*` comes back italic and `~one~` is not
+ ;; parsed at all, so the old dialect would silently lose every bold and strikethrough.
+ (testing "the prompt teaches the dialect the answer block actually renders (BOT-2010)"
+ (let [rendered (render-slackbot-template all-yes-perms)]
+ (testing "standard markdown is prescribed"
+ (is (re-find #"`\*\*text\*\*` for bold text" rendered))
+ (is (re-find #"`~~text~~` for strikethrough text" rendered)))
+ (testing "and the Slack-only dialect is not"
+ (is (not (re-find #"`\*text\*` for bold text" rendered)))
+ (is (not (re-find #"`~text~` for strikethrough text" rendered))))
+ (testing "Slack's link syntax is kept -- it does render inside a markdown block (measured)"
+ (is (re-find #"<https://example\.com\|Link Text>" rendered))
+ (is (re-find #"metabase://dashboard/123\|Dashboard Name" rendered))))))
+
;;; ──────────────────────────────────────────────────────────────────
;;; Custom system prompt injection
;;; ──────────────────────────────────────────────────────────────────
diff --git a/test/metabase/slackbot/api_test.clj b/test/metabase/slackbot/api_test.clj
index 33bfbff4cab..2a008d3b4fb 100644
--- a/test/metabase/slackbot/api_test.clj
+++ b/test/metabase/slackbot/api_test.clj
@@ -225,14 +225,15 @@
:timeout-ms 5000})
(is (= 1 (count @post-calls)))
(let [blocks (:blocks (first @post-calls))]
- (testing "no section block is over the limit -- unlike the untruncated answer"
- (is (nil? (tu/oversized-section-error blocks)))
- (is (some? (tu/oversized-section-error [{:type "section"
- :text {:type "mrkdwn" :text tu/oversized-answer}}]))
+ (testing "no block is over its limit -- unlike the untruncated answer"
+ (is (nil? (tu/oversized-block-error blocks)))
+ ;; The control has to name the block type the answer actually goes in, or it would
+ ;; assert a section rule no longer in play and pass whatever the code does.
+ (is (some? (tu/oversized-block-error [{:type "markdown" :text tu/oversized-answer}]))
"the answer really is past the limit, so the case under test is the real one"))
(testing "the answer is cut to the limit, and the message says why"
- (is (= tu/slack-section-text-limit
- (count (get-in (first blocks) [:text :text]))))
+ (is (= tu/slack-markdown-text-limit
+ (count (:text (first blocks)))))
(is (some (fn [block]
(and (= "context" (:type block))
(str/includes? (get-in block [:elements 0 :text])
diff --git a/test/metabase/slackbot/channel_test.clj b/test/metabase/slackbot/channel_test.clj
index 7736829ff00..5429a803eb7 100644
--- a/test/metabase/slackbot/channel_test.clj
+++ b/test/metabase/slackbot/channel_test.clj
@@ -111,13 +111,15 @@
(deftest ^:parallel truncation-notice-test
(testing "the notice points at the instance, and says what to do about the cut"
- (is (= (str "_This answer was longer than 3000 characters. That is too long to post in Slack, "
+ (is (= (str "_This answer was longer than " slackbot.tu/slack-markdown-text-limit
+ " characters. That is too long to post in Slack, "
"so I cut it short._\n\n"
"Ask a narrower question so the answer comes back smaller"
", or head to <https://metabase.example.com|Metabase> and try again.")
(#'slackbot.channel/truncation-notice test-site-url))))
(testing "an instance with no site URL still gets a sentence, just without the link"
- (is (= (str "_This answer was longer than 3000 characters. That is too long to post in Slack, "
+ (is (= (str "_This answer was longer than " slackbot.tu/slack-markdown-text-limit
+ " characters. That is too long to post in Slack, "
"so I cut it short._\n\n"
"Ask a narrower question so the answer comes back smaller.")
(#'slackbot.channel/truncation-notice nil))))
@@ -130,15 +132,14 @@
(mt/with-temporary-setting-values [site-url test-site-url]
(let [{:keys [text blocks backfill]} (send-channel-response! slackbot.tu/oversized-answer)
[answer-block notice-block] blocks
- section-text (get-in answer-block [:text :text])]
- (testing "no section block is over the limit -- so Slack no longer rejects the message"
- (is (nil? (slackbot.tu/oversized-section-error blocks))))
- (testing "the answer rides one mrkdwn section block, cut to the limit"
- (is (= "section" (:type answer-block)))
- (is (= "mrkdwn" (get-in answer-block [:text :type])))
- (is (= slackbot.tu/slack-section-text-limit (count section-text)))
- (is (str/ends-with? section-text "..."))
- (is (str/starts-with? slackbot.tu/oversized-answer (str/replace section-text #"\.\.\.$" ""))
+ answer-text (:text answer-block)]
+ (testing "no block is over its limit -- so Slack no longer rejects the message"
+ (is (nil? (slackbot.tu/oversized-block-error blocks))))
+ (testing "the answer goes in one markdown block, cut to the limit (BOT-2010)"
+ (is (= "markdown" (:type answer-block)))
+ (is (= slackbot.tu/slack-markdown-text-limit (count answer-text)))
+ (is (str/ends-with? answer-text "..."))
+ (is (str/starts-with? slackbot.tu/oversized-answer (str/replace answer-text #"\.\.\.$" ""))
"what survives the cut is a genuine prefix of the answer"))
(testing "the notice rides in a context block of the same message, linking to the instance"
(is (= "context" (:type notice-block)))
@@ -148,10 +149,10 @@
(is (not (str/includes? (get-in notice-block [:elements 0 :text]) "/metabot/conversation/"))
"the link is the home page, not a conversation we cannot continue on the web"))
(testing "the visualizations and the feedback buttons still ride the message"
- (is (= ["section" "context" "section" "table" "context_actions"]
+ (is (= ["markdown" "context" "section" "table" "context_actions"]
(mapv :type blocks))))
(testing "the notice stays out of `:text`, which `thread->history` replays back to the model"
- (is (= section-text text))
+ (is (= answer-text text))
(is (not (str/includes? text "too long to post in Slack"))))
(testing "one message is posted, and its ts is persisted -- it carries the feedback buttons"
(is (= {:msg-id 42 :slack-msg-id "1700000000.000002"} backfill)))))))
@@ -171,10 +172,10 @@
(testing "an answer within the limit is posted whole, with no notice"
(let [answer "Orders peaked in March. *1,204* of them."
{:keys [text blocks]} (send-channel-response! answer)]
- (is (nil? (slackbot.tu/oversized-section-error blocks)))
- (is (= answer (get-in (first blocks) [:text :text])))
+ (is (nil? (slackbot.tu/oversized-block-error blocks)))
+ (is (= answer (:text (first blocks))))
(is (= answer text))
- (is (= ["section" "section" "table" "context_actions"] (mapv :type blocks))
+ (is (= ["markdown" "section" "table" "context_actions"] (mapv :type blocks))
"no context block, because nothing was cut"))))
;; Not ^:parallel: `with-prometheus-system!` redefs a process-global var.
diff --git a/test/metabase/slackbot/streaming_test.clj b/test/metabase/slackbot/streaming_test.clj
index a44243252b4..aabc8aa7f8a 100644
--- a/test/metabase/slackbot/streaming_test.clj
+++ b/test/metabase/slackbot/streaming_test.clj
@@ -173,9 +173,9 @@
"revenue-by-month"
"Revenue by month"
huge-link)]
- (is (nil? (tu/oversized-section-error blocks))
+ (is (nil? (tu/oversized-block-error blocks))
"Slack no longer rejects the whole message")
- (is (some? (tu/oversized-section-error
+ (is (some? (tu/oversized-block-error
[{:type "section"
:text {:type "mrkdwn"
:text (str "📊 <https://metabase.example.com" huge-link "|Revenue by month>")}}]))
diff --git a/test/metabase/slackbot/test_util.clj b/test/metabase/slackbot/test_util.clj
index 0f77effa041..6e00edc1a68 100644
--- a/test/metabase/slackbot/test_util.clj
+++ b/test/metabase/slackbot/test_util.clj
@@ -52,17 +52,19 @@
"Alias for [[metabase.slackbot.channel/section-text-limit]]."
slackbot.channel/section-text-limit)
+(def slack-markdown-text-limit
+ "Alias for [[metabase.slackbot.channel/markdown-text-limit]]."
+ slackbot.channel/markdown-text-limit)
+
(def oversized-answer
- "An answer comfortably past [[slack-section-text-limit]].
+ "An answer comfortably past [[slack-markdown-text-limit]].
Numbered line by line, so a truncated copy can be checked against the original."
- (str/join "\n" (map #(format "Line %04d of a rather long answer." %) (range 200))))
+ (str/join "\n" (map #(format "Line %04d of a rather long answer." %) (range 400))))
-(defn oversized-section-error
- "The `chat.postMessage` rejection Slack returns for an oversized `section` block in `blocks`.
- Nil when every block is within [[slack-section-text-limit]]. Models that one rule -- the one
- BOT-1606 is about -- not Block Kit validation at large."
+(defn- oversized-section-error
+ "Returns Slack's `invalid_blocks` rejection for an over-long `section`, or nil when every one fits.
+ Slack names the offending block and its limit, which the messages here reproduce."
[blocks]
- ;; The mocked client accepts anything, so tests that care about this rule come through here.
(when-let [idx (first (keep-indexed (fn [idx block]
(when (and (= "section" (:type block))
(> (count (get-in block [:text :text] ""))
@@ -76,6 +78,28 @@
(format "[ERROR] must be less than %d characters [json-pointer:/blocks/%d/text/text]"
(inc slack-section-text-limit) idx)]}}))
+(defn- oversized-markdown-error
+ "Returns Slack's `msg_too_long` rejection for over-long `markdown` text, or nil when it fits.
+ The budget is charged across every `markdown` block at once, matching what
+ [[slack-markdown-text-limit]] documents rather than the per-block enforcement seen today."
+ [blocks]
+ (let [total (->> blocks
+ (filter #(= "markdown" (:type %)))
+ (map #(count (:text % "")))
+ (reduce + 0))]
+ (when (> total slack-markdown-text-limit)
+ ;; A bare error, carrying none of the per-block detail the `section` case does (measured).
+ {:ok false :error "msg_too_long"})))
+
+(defn oversized-block-error
+ "Returns the `chat.postMessage` rejection Slack would send for `blocks`, or nil when it accepts them.
+ Models two rules only, the `section` and `markdown` text limits BOT-1606 and BOT-2010 are about,
+ not Block Kit validation at large."
+ [blocks]
+ ;; The mocked client accepts anything, so tests that care about these rules come through here.
+ (or (oversized-section-error blocks)
+ (oversized-markdown-error blocks)))
+
(defmacro with-ensure-encryption
"Use the existing encryption key if one is configured, otherwise set a test key.
Avoids conflicts with encrypted settings in the DB that were written with the real key."