The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Fix parsing of :ERROR chunks in metabase-enterprise.metabot-v3.util/aisdk->messages (#67674)
applebyfbdf7574ede4
diff --git a/enterprise/backend/src/metabase_enterprise/metabot_v3/util.clj b/enterprise/backend/src/metabase_enterprise/metabot_v3/util.cljindex ce712edbaa6..8774ea73396 100644--- a/enterprise/backend/src/metabase_enterprise/metabot_v3/util.clj+++ b/enterprise/backend/src/metabase_enterprise/metabot_v3/util.clj@@ -49,11 +49,11 @@(mapcat (fn [block](let [type (ffirst block)](case type- :TEXT [{:role role+ (:TEXT+ :ERROR) [{:role role:_type type:content (transduce (map second) str block)}]- (:DATA- :ERROR) (map #(-> (second %) (assoc :_type type)) block)+ :DATA (map #(-> (second %) (assoc :_type type)) block):TOOL_CALL [{:role role:_type type:tool_calls (map (fn [[_ v]]diff --git a/enterprise/backend/test/metabase_enterprise/metabot_v3/aisdkstream3.txt b/enterprise/backend/test/metabase_enterprise/metabot_v3/aisdkstream3.txtnew file mode 100644index 00000000000..ce34a12d97e--- /dev/null+++ b/enterprise/backend/test/metabase_enterprise/metabot_v3/aisdkstream3.txt@@ -0,0 +1,2 @@+3:"litellm.ServiceUnavailableError: litellm.MidStreamFallbackError: litellm.APIConnectionError: APIConnectionError: OpenrouterException - Connection closed. Original exception: APIConnectionError: litellm.APIConnectionError: APIConnectionError: OpenrouterException - Connection closed."+d:{"finishReason":"error","usage":{"promptTokens":0,"completionTokens":0}}diff --git a/enterprise/backend/test/metabase_enterprise/metabot_v3/aisdkstream4.txt b/enterprise/backend/test/metabase_enterprise/metabot_v3/aisdkstream4.txtnew file mode 100644index 00000000000..0ccd3af818b--- /dev/null+++ b/enterprise/backend/test/metabase_enterprise/metabot_v3/aisdkstream4.txt@@ -0,0 +1,14 @@+0:"I'll help you show"+0:" trends in user"+0:" monthly spending by user ID. Let me"+0:" first"+0:" explore"+0:" the available data to fin"+0:"d the"+0:" right"+0:" source"+0:"."+9:{"toolCallId":"toolu_bdrk_01GPPfFCzqWpZhsDdGMi9mYi","toolName":"search","args":"{\"semantic_queries\": [\"user spending trends over time\",\"monthly spending by user\"], \"keyword_queries\": [\"spending\",\"user\",\"monthly\",\"trend\"], \"entity_types\": [\"metric\",\"model\",\"table\"]}"}+a:{"toolCallId":"toolu_bdrk_01GPPfFCzqWpZhsDdGMi9mYi","result":"\n<result>\nHere are the search results:\n<search-results>\n<metric id=\"77\", name=\"User monthly spending trend\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nMonthly spending patterns by user showing total spend, transaction count, and spending tier. Enables identification of high spenders and spending behavior changes.\n\n</metric>\n \n\n<metric id=\"78\", name=\"Category spending concentration\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nSpending distribution across expense categories with transaction volumes and unique merchant counts. Identifies major cost drivers and category trends.\n\n</metric>\n \n\n<metric id=\"82\", name=\"User activity level distribution\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nDistribution of users by activity level with lifetime spend metrics. Identifies active vs inactive cardholders and engagement patterns.\n\n</metric>\n \n\n<metric id=\"79\", name=\"Department budget performance\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nDepartment-level spending aggregated monthly with approved and pending expense breakdowns. Critical for budget tracking and departmental cost management.\n\n</metric>\n \n\n<metric id=\"81\", name=\"Transaction size distribution\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nDistribution of transactions by size category (Small/Medium/Large) with merchant and category breakdowns. Helps understand spending patterns.\n\n</metric>\n \n\n<metric id=\"87\", name=\"User cards per transaction ratio\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nAverage number of different cards used per user per month. Indicates card usage diversity and potential card consolidation opportunities.\n\n</metric>\n \n\n<metric id=\"89\", name=\"Department user spending intensity\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nAverage spending per active user within each department. Identifies departments with high per-capita spending.\n\n</metric>\n \n\n<metric id=\"85\", name=\"Average transaction value by category\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nAverage transaction amount by spending category using pre-aggregated monthly data. Identifies typical spending levels per category.\n\n</metric>\n \n\n<metric id=\"97\", name=\"Weekly scheduling capacity trend\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nTrend analysis of weekly scheduling capacity utilization. Shows how users are ramping up or down their availability.\n\n</metric>\n \n\n<table id=\"151\", name=\"int_brex_spending_by_department_facts\" database_id=\"2\" database_engine=\"unknown\" fully_qualified_name=\"brex_enriched.int_brex_spending_by_department_facts\">\n\n### Description\nNone\n\n\n\nBefore using any field in comparisons, filters, or conditions, you should check their metadata and sample values first, using the uri `metabase://table/{table.id}/fields/{field_id}`.\nThis will help avoid errors due to unexpected data types or formats.\n</table>\n\n<metric id=\"93\", name=\"User expense reporting rate\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nNumber of expense reports submitted per user per month. Measures expense reporting activity and compliance.\n\n</metric>\n \n\n<metric id=\"95\", name=\"Category user adoption rate\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nNumber of unique users making purchases in each spending category. Indicates category usage breadth across the organization.\n\n</metric>\n \n\n<metric id=\"88\", name=\"Category merchant diversity\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nNumber of unique merchants per spending category. Measures vendor diversity and potential for vendor consolidation.\n\n</metric>\n \n\n<metric id=\"99\", name=\"Multi event user percentage\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nPercentage of users hosting multiple event types. Shows product stickiness and feature adoption.\n\n</metric>\n \n\n<metric id=\"86\", name=\"Quarterly transaction volume\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nTotal number of transactions per quarter. Tracks overall company transaction activity trends.\n\n</metric>\n \n\n<metric id=\"76\", name=\"Card Utilization Rate\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nPercentage of credit limit being utilized across active cards. Measures credit usage efficiency and potential need for limit adjustments.\n\n</metric>\n \n\n<metric id=\"80\", name=\"Expense approval cycle time\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nAverage days from expense submission to approval by department. Measures approval workflow efficiency and identifies bottlenecks.\n\n</metric>\n \n\n<metric id=\"113\", name=\"Click count\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nNumber of email link clicks. Tracks engagement trends and identifies high-activity periods for email campaigns.\n\n</metric>\n \n\n<metric id=\"92\", name=\"Card transaction volume per card\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nAverage number of transactions per card using pre-aggregated card data. Identifies heavily used vs underutilized cards.\n\n</metric>\n \n\n<metric id=\"116\", name=\"Open to click conversion rate\" is_verified=\"False\">\n\nThe metric is stored in the following collection:\n<collection name=\"Metrics\">\n</collection>\n\n\n### Metric Description\nPercentage of opened emails that resulted in at least one click. Measures content relevance for engaged users.\n\n</metric>\n </search-results>\n</result>\n<instructions>\nSearch results ranked by relevance (highest first).\n\nWhen using results:\n- Prioritize verified dashboards, questions, and metrics over raw data exploration when they match the user's request\n- Check available fields or metadata of promising results before recommending them to ensure they contain needed data\n- **Be proactive with clear matches**: When the top result(s) have names or descriptions that clearly match the user's request, use them to achieve the users goal first, then list alternatives at the end. Don't ask for confirmation when there's an obvious best choice.\n- **Collection metadata**: Results may include collection information (name, description, authority level) that can help you determine whether the result is relevant to the user's query. Use this context to assess relevance.\n- **Present options for ambiguous results**: When search returns multiple plausible options without a clear best match, present them and ask the user to choose\n- Reference results using the metabase protocol link format: [display name](metabase://type/id)\n\nExamples: [Customer Metrics](metabase://metric/42), [Sales Dashboard](metabase://dashboard/158)\n\n</instructions>"}+3:"litellm.Timeout: Timeout Error: OpenrouterException - litellm.Timeout: Connection timed out. Timeout passed=600.0, time taken=600.018 seconds"+d:{"finishReason":"error","usage":{"openrouter/anthropic/claude-haiku-4.5":{"prompt":10458,"completion":117,"costs":0.011043},"promptTokens":10458,"completionTokens":117}}diff --git a/enterprise/backend/test/metabase_enterprise/metabot_v3/api_test.clj b/enterprise/backend/test/metabase_enterprise/metabot_v3/api_test.cljindex 22dff4094a4..5ce5bae8b06 100644--- a/enterprise/backend/test/metabase_enterprise/metabot_v3/api_test.clj+++ b/enterprise/backend/test/metabase_enterprise/metabot_v3/api_test.clj@@ -22,7 +22,7 @@(deftest agent-streaming-test(mt/with-premium-features #{:metabot-v3}- (let [mock-response (client-test/make-mock-stream-response+ (let [mock-response (client-test/make-mock-text-stream-response["Hello", " from", " streaming!"]{"some-model" {:prompt 12 :completion 3}})conversation-id (str (random-uuid))diff --git a/enterprise/backend/test/metabase_enterprise/metabot_v3/client_test.clj b/enterprise/backend/test/metabase_enterprise/metabot_v3/client_test.cljindex 67d2d470289..78bc120516e 100644--- a/enterprise/backend/test/metabase_enterprise/metabot_v3/client_test.clj+++ b/enterprise/backend/test/metabase_enterprise/metabot_v3/client_test.clj@@ -21,11 +21,17 @@(set! *warn-on-reflection* true)-(mu/defn make-mock-stream-response [chunks usage :- ::metabot-v3.client.schema/usage]+(mu/defn- make-mock-stream-response* [part-prefix finish-reason chunks usage :- ::metabot-v3.client.schema/usage](str (->> chunks- (map #(str "0:" (json/encode %) "\n"))+ (map #(str part-prefix (json/encode %) "\n"))(str/join))- "d:" (json/encode {:finishReason "stop" :usage usage})))+ "d:" (json/encode {:finishReason finish-reason :usage usage})))++(mu/defn make-mock-text-stream-response [chunks usage :- ::metabot-v3.client.schema/usage]+ (make-mock-stream-response* "0:" "stop" chunks usage))++(mu/defn make-mock-error-stream-response [chunks usage :- ::metabot-v3.client.schema/usage]+ (make-mock-stream-response* "3:" "error" chunks usage))(defn mock-post! [^String body & [{:keys [delay-ms]:or {delay-ms 0}}]]@@ -60,7 +66,7 @@(deftest client-test(mt/with-premium-features #{:metabot-v3}(let [input ["a1" "a2" "a3"]- mock-response (make-mock-stream-response input {"some-model" {:prompt 8 :completion 2}})+ mock-response (make-mock-text-stream-response input {"some-model" {:prompt 8 :completion 2}})cid (str (random-uuid))req {:context {:some "context"}:message {:role :user :content "stuff"}@@ -80,6 +86,29 @@{:_type :FINISH_MESSAGE :finish_reason "stop"}](metabot-v3.util/aisdk->messages :assistant (str/split-lines body))))))))))+(deftest client-error-test+ (mt/with-premium-features #{:metabot-v3}+ (let [input ["oof " "bad " "error!"]+ mock-response (make-mock-error-stream-response input {"some-model" {:prompt 8 :completion 2}})+ cid (str (random-uuid))+ req {:context {:some "context"}+ :message {:role :user :content "stuff"}+ :history []+ :profile-id "test-profile"+ :conversation-id cid+ :session-id "test-session"+ :state {:some "state"}}]+ (mt/with-dynamic-fn-redefs [http/post (mock-post! mock-response)]+ (mt/with-current-user (mt/user->id :crowberto)+ (let [res (metabot-v3.client/streaming-request req)+ body (consume-streaming-response res)]+ (is (instance? StreamingResponse res))+ (is (= "text/event-stream; charset=utf-8" (:content-type (.options ^StreamingResponse res))))+ (is (string? body))+ (is (=? [{:_type :ERROR :content "oof bad error!"}+ {:_type :FINISH_MESSAGE :finish_reason "error"}]+ (metabot-v3.util/aisdk->messages :assistant (str/split-lines body))))))))))+(deftest streaming-request-error-excludes-headers-test(testing "When streaming-request gets an error response, the exception should not include headers"(mt/with-premium-features #{:metabot-v3}diff --git a/enterprise/backend/test/metabase_enterprise/metabot_v3/util_test.clj b/enterprise/backend/test/metabase_enterprise/metabot_v3/util_test.cljindex ced32b65509..e262991770d 100644--- a/enterprise/backend/test/metabase_enterprise/metabot_v3/util_test.clj+++ b/enterprise/backend/test/metabase_enterprise/metabot_v3/util_test.clj@@ -4,7 +4,7 @@[clojure.test :refer :all][metabase-enterprise.metabot-v3.util :as metabot.u]))-(deftest aisdk-line-parse-test+(deftest ^:parallel aisdk-line-parse-test(testing "We should be able to parse AI SDK streaming format to ai-service format 1"(is (=? [{:role "assistant":_type :TOOL_CALL@@ -64,5 +64,39 @@:usage {}}](metabot.u/aisdk->messages "assistant"(-> (io/resource "metabase_enterprise/metabot_v3/aisdkstream2.txt")+ io/reader+ line-seq)))))+ (testing "We should be able to parse AI SDK streaming format with error messages 3"+ (is (=? [{:role "assistant"+ :_type :ERROR+ :content #"litellm.ServiceUnavailableError: litellm.MidStreamFallbackError:.*Connection closed."}+ {:finish_reason "error"+ :_type :FINISH_MESSAGE+ :usage {}}]+ (metabot.u/aisdk->messages "assistant"+ (-> (io/resource "metabase_enterprise/metabot_v3/aisdkstream3.txt")+ io/reader+ line-seq)))))+ (testing "We should be able to parse AI SDK streaming format with error messages 4"+ (is (=? [{:role "assistant"+ :_type :TEXT+ :content #"(?s)I'll help.*find the right source."}+ {:role "assistant"+ :_type :TOOL_CALL+ :tool_calls [{:id "toolu_bdrk_01GPPfFCzqWpZhsDdGMi9mYi"+ :name "search"+ :arguments #"\{\"semantic_queries\":.*\}"}]}+ {:role "tool"+ :_type :TOOL_RESULT+ :tool_call_id "toolu_bdrk_01GPPfFCzqWpZhsDdGMi9mYi"+ :content #"(?s)\n<result>.*</instructions>"}+ {:role "assistant"+ :_type :ERROR+ :content #"litellm.Timeout: Timeout Error: OpenrouterException.*seconds"}+ {:finish_reason "error"+ :_type :FINISH_MESSAGE+ :usage {}}]+ (metabot.u/aisdk->messages "assistant"+ (-> (io/resource "metabase_enterprise/metabot_v3/aisdkstream4.txt")io/readerline-seq))))))