The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
Derive the mini model only from what a connection was listed as serving (#83139)
Maksym Yakubych190a31be8215
diff --git a/src/metabase/llm/api/provider.clj b/src/metabase/llm/api/provider.cljindex ce4d29840f5..f53dbe1d082 100644--- a/src/metabase/llm/api/provider.clj+++ b/src/metabase/llm/api/provider.clj@@ -277,14 +277,19 @@for a type that probes more than its credentials, by exercising the model it will run on. Throws a 400 carryingthe provider's own message when the credentials are rejected.- Returns the model listing and `:connection-info`: whatever the probe determined about the connection, for the- caller to store on it. A probe records the model it exercised as `:probed-model`."+ Returns the model listing and `:connection-info`: whatever the listing determined about the connection, for the+ caller to store on it. A probe records the model it exercised as `:probed-model`, and every listing records the+ type's mini model when it includes it as `:mini-model`."[conn config model](when-not (llm.provider/managed-type? (:type conn))- (let [{:keys [error] :as listed} (list-connection-models* conn config model true)]+ (let [{:keys [error models] :as listed} (list-connection-models* conn config model true)](when error(throw (ex-info error {:status-code 400 :api-error true})))- listed)))+ (update listed :connection-info merge (llm.provider/served-mini-model (:type conn) models)))))++(defn- with-connection-info+ [config connection-info]+ (without-blank-values (merge config connection-info)))(defn- seed-models-cache!"Cache the listing that verified `conn` under its post-save config and selected model. Call this only after any@@ -448,7 +453,7 @@:config config}](llm.provider/validate-config! type config)(let [{:keys [connection-info] :as listed} (verify-credentials! conn config model)- conn (update conn :config merge connection-info)+ conn (update conn :config with-connection-info connection-info)had-usable-model? (metabot-has-a-usable-model?)](llm.provider/set-connections! (conj (llm.provider/stored-connections) conn))(when-not had-usable-model?@@ -497,8 +502,8 @@(llm.provider/validate-config! (:type merged) effective)(let [{:keys [connection-info] :as listed}(verify-credentials! merged effective (or model (selected-model conn-key)))- merged (update merged :config merge connection-info)- effective (merge effective connection-info)]+ merged (update merged :config with-connection-info connection-info)+ effective (with-connection-info effective connection-info)](llm.provider/set-connections! (assoc stored idx merged))(follow-edited-connection-model! (assoc merged :config effective) model)(seed-models-cache! (assoc merged :config effective) listed)diff --git a/src/metabase/llm/provider.clj b/src/metabase/llm/provider.cljindex 6d3ce6bafe1..d992aa62926 100644--- a/src/metabase/llm/provider.clj+++ b/src/metabase/llm/provider.clj@@ -480,12 +480,24 @@(defn mini-model"The fastest and cheapest model `type-name` serves — what short utility calls such as conversation titles run on- when no model has been picked for them. Returns nil for the types that have no cheaper tier to fall back to: the- ones whose connection names the single model it serves rather than picking from a catalog, and the managed- provider, which serves one benchmarked model."+ when no model has been picked for them and the connection's listing includes it. Returns nil for the types that+ have no cheaper tier to fall back to: the ones whose connection names the single model it serves rather than+ picking from a catalog, and the managed provider, which serves one benchmarked model."[type-name](:mini-model (provider-type type-name)))+(defn served-mini-model+ "The `:config` entry recording `type-name`'s [[mini-model]] when `listed-models` includes it, and nil when it does+ not."+ [type-name listed-models]+ (let [model (mini-model type-name)]+ {:mini-model (when (some #(= model (:id %)) listed-models) model)}))++(defn connection-mini-model+ "The [[mini-model]] `conn`'s listing included when it was last saved, or nil."+ [conn]+ (get-in conn [:config :mini-model]))+;;; -------------------------------------------------- Validation --------------------------------------------------(defn connection-modeldiff --git a/src/metabase/metabot/settings.clj b/src/metabase/metabot/settings.cljindex 42b17bbf1c2..1d3feca59a2 100644--- a/src/metabase/metabot/settings.clj+++ b/src/metabase/metabot/settings.clj@@ -194,13 +194,13 @@(llm.provider/canonical-model-ref new-value))))(defn- mini-model-ref- "The model reference for the fastest model of the connection `model-ref` names, or nil when that connection's- provider type has no such model or the connection names the one model it serves."+ "The model reference for the mini model the connection `model-ref` names was listed as serving, or nil when it+ names the one model it serves."[model-ref]- (let [conn-key (llm.provider/model-ref->connection-key model-ref)- {:keys [type config]} (llm.provider/connection conn-key)]+ (let [conn-key (llm.provider/model-ref->connection-key model-ref)+ {:keys [type config] :as conn} (llm.provider/connection conn-key)](when-let [model (and (not (llm.provider/connection-model type config))- (llm.provider/mini-model type))]+ (llm.provider/connection-mini-model conn))](str conn-key "/" model))))(defn explicit-mini-model@@ -214,10 +214,9 @@(defn- -llm-mini-model"Quick background tasks — naming a conversation, and whatever short, high-volume calls come next — do not need the- model Metabot chats on, so with nothing stored this resolves to the fastest model of the- connection [[llm-metabot-provider]] names. Connections that name the single model they serve, and those whose- provider type has no such model like the managed provider, fall through to the Metabot model itself, so this always- names a model as long as Metabot does."+ model Metabot chats on, so with nothing stored this resolves to the fastest model the+ connection [[llm-metabot-provider]] names was listed as serving. Connections with no such model fall through to+ the Metabot model itself, so this always names a model as long as Metabot does."[](or (explicit-mini-model)(let [metabot-ref (llm-metabot-provider)]diff --git a/test/metabase/llm/api/provider_test.clj b/test/metabase/llm/api/provider_test.cljindex 4c1894bc279..c0e9fb5d101 100644--- a/test/metabase/llm/api/provider_test.clj+++ b/test/metabase/llm/api/provider_test.clj@@ -293,6 +293,56 @@{:type "openai" :config {:api-key "sk-valid"}})(is (= "anthropic/claude-opus-4-8" (metabot.settings/llm-metabot-provider))))))))+(deftest create-records-whether-the-listing-offered-the-mini-model-test+ (testing "a connection whose listing includes its type's mini model runs quick tasks on it"+ (mt/with-dynamic-fn-redefs [metabot.self/list-models+ (fn [& _] {:models [{:id "claude-sonnet-4-6" :display_name "Claude Sonnet 4.6"}+ {:id "claude-haiku-4-5-20251001" :display_name "Claude Haiku 4.5"}]})]+ (mt/with-temporary-setting-values [llm-providers []]+ (mt/with-temporary-raw-setting-values [llm-metabot-provider nil+ llm-mini-model nil]+ (mt/user-http-request :crowberto :post 200 "llm/providers"+ {:type "anthropic" :config {:api-key "sk-ant-valid"}})+ (is (= {:api-key "sk-ant-valid" :mini-model "claude-haiku-4-5-20251001"} (stored-config "anthropic")))+ (is (= "anthropic/claude-haiku-4-5-20251001" (metabot.settings/llm-mini-model)))))))+ (testing "one whose listing leaves it out runs them on the Metabot model, not on a guess the account cannot serve"+ (mt/with-dynamic-fn-redefs [metabot.self/list-models+ (fn [& _] {:models [{:id "claude-sonnet-4-6" :display_name "Claude Sonnet 4.6"}]})]+ (mt/with-temporary-setting-values [llm-providers []]+ (mt/with-temporary-raw-setting-values [llm-metabot-provider nil+ llm-mini-model nil]+ (mt/user-http-request :crowberto :post 200 "llm/providers"+ {:type "anthropic" :config {:api-key "sk-ant-valid"}})+ (is (= {:api-key "sk-ant-valid"} (stored-config "anthropic")))+ (is (nil? (setting/db-stored-value :llm-mini-model)))+ (is (= "anthropic/claude-sonnet-4-6" (metabot.settings/llm-mini-model))))))))++(deftest update-relists-the-mini-model-test+ (testing "editing a connection asks its listing again, and the form echoing the old answer back does not keep it"+ (mt/with-dynamic-fn-redefs [metabot.self/list-models+ (fn [& _] {:models [{:id "claude-sonnet-4-6" :display_name "Claude Sonnet 4.6"}]})]+ (mt/with-temporary-setting-values [llm-providers [(connection "anthropic" "anthropic"+ {:api-key "sk-ant-stored"+ :mini-model "claude-haiku-4-5-20251001"})]]+ (mt/with-temporary-raw-setting-values [llm-metabot-provider "anthropic/claude-sonnet-4-6"+ llm-mini-model nil]+ (is (= [:api-key]+ (keys (:config (mt/user-http-request :crowberto :put 200 "llm/providers/anthropic"+ {:config {:api-key "sk-ant-rotated"+ :mini-model "claude-haiku-4-5-20251001"}})))))+ (is (= {:api-key "sk-ant-rotated"} (stored-config "anthropic")))+ (is (= "anthropic/claude-sonnet-4-6" (metabot.settings/llm-mini-model)))))))+ (testing "and records the mini model once the account serves it"+ (mt/with-dynamic-fn-redefs [metabot.self/list-models+ (fn [& _] {:models [{:id "claude-sonnet-4-6" :display_name "Claude Sonnet 4.6"}+ {:id "claude-haiku-4-5-20251001" :display_name "Claude Haiku 4.5"}]})]+ (mt/with-temporary-setting-values [llm-providers [(connection "anthropic" "anthropic" {:api-key "sk-ant-stored"})]]+ (mt/with-temporary-raw-setting-values [llm-metabot-provider "anthropic/claude-sonnet-4-6"+ llm-mini-model nil]+ (mt/user-http-request :crowberto :put 200 "llm/providers/anthropic" {:name "Anthropic (prod)"})+ (is (= {:api-key "sk-ant-stored" :mini-model "claude-haiku-4-5-20251001"} (stored-config "anthropic")))+ (is (= "anthropic/claude-haiku-4-5-20251001" (metabot.settings/llm-mini-model))))))))+(deftest create-vllm-connection-adopts-the-model-its-probe-exercised-test(testing (str "A vLLM server serves whatever the operator loaded, so there is no default model to select: ""connecting adopts the model the connect-time probe actually ran the agent-loop contract "@@ -1110,7 +1160,8 @@(mt/with-temporary-raw-setting-values [llm-metabot-provider "openai/gpt-5.4"llm-mini-model "anthropic/claude-haiku-4-5"](mt/with-temporary-setting-values [llm-providers [(connection "anthropic" "anthropic" {:api-key "sk-ant-stored"})- (connection "openai" "openai" {:api-key "sk-stored"})]]+ (connection "openai" "openai" {:api-key "sk-stored"+ :mini-model "gpt-5.4-mini"})]](mt/user-http-request :crowberto :delete 204 "llm/providers/anthropic")(is (nil? (setting/db-stored-value :llm-mini-model)))(is (= "openai/gpt-5.4-mini" (metabot.settings/llm-mini-model)))))))diff --git a/test/metabase/llm/provider_test.clj b/test/metabase/llm/provider_test.cljindex 1e70927a214..b48c349a880 100644--- a/test/metabase/llm/provider_test.clj+++ b/test/metabase/llm/provider_test.clj@@ -752,3 +752,25 @@(testing "every type other than the managed one is always available"(is (true? (llm.provider/type-available? "anthropic")))(is (false? (llm.provider/type-available? "evilai")))))++(deftest ^:parallel served-mini-model-test+ (testing "a listing that includes the type's mini model records it on the connection"+ (is (= {:mini-model "claude-haiku-4-5-20251001"}+ (llm.provider/served-mini-model "anthropic" [{:id "claude-sonnet-4-6" :display_name "Claude Sonnet 4.6"}+ {:id "claude-haiku-4-5-20251001" :display_name "Claude Haiku 4.5"}]))))+ (testing "one that leaves it out records nothing, so an earlier answer is retired rather than kept"+ (is (= {:mini-model nil}+ (llm.provider/served-mini-model "anthropic" [{:id "claude-sonnet-4-6" :display_name "Claude Sonnet 4.6"}])))+ (is (= {:mini-model nil} (llm.provider/served-mini-model "anthropic" []))))+ (testing "a type with no mini model has nothing to record"+ (is (= {:mini-model nil}+ (llm.provider/served-mini-model "azure" [{:id "openai/gpt-4.1" :display_name "gpt-4.1"}])))))++(deftest ^:parallel connection-mini-model-test+ (testing "reads the model the connection's listing recorded"+ (is (= "claude-haiku-4-5-20251001"+ (llm.provider/connection-mini-model {:key "anthropic" :type "anthropic"+ :config {:api-key "sk-ant" :mini-model "claude-haiku-4-5-20251001"}}))))+ (testing "and nothing for a connection no listing has answered for, whatever its type would offer"+ (is (nil? (llm.provider/connection-mini-model {:key "anthropic" :type "anthropic" :config {:api-key "sk-ant"}})))+ (is (nil? (llm.provider/connection-mini-model nil)))))diff --git a/test/metabase/metabot/settings_test.clj b/test/metabase/metabot/settings_test.cljindex e9ac10c855a..162291e5ac4 100644--- a/test/metabase/metabot/settings_test.clj+++ b/test/metabase/metabot/settings_test.clj@@ -16,7 +16,7 @@{:key conn-key :type type :name conn-key :config config}))(def ^:private configured-anthropic- (connection "anthropic" "anthropic" {:api-key "sk-ant-test"}))+ (connection "anthropic" "anthropic" {:api-key "sk-ant-test" :mini-model "claude-haiku-4-5-20251001"}))(def ^:private configured-google(connection "google" "google" {:oauth-access-token "ya29.test" :project-id "my-project"}))@@ -458,10 +458,10 @@(metabot.settings/llm-metabot-provider! "metabase/")))))))(deftest llm-mini-model-defaults-to-the-metabot-connections-mini-model-test- (testing "with nothing stored, quick tasks run on the fastest model of the connection Metabot uses"+ (testing "with nothing stored, quick tasks run on the cheaper model the connection Metabot uses was listed as serving"(mt/with-temporary-raw-setting-values [llm-mini-model nil](with-connections [configured-anthropic- (connection "openai" "openai" {:api-key "sk-openai"})]+ (connection "openai" "openai" {:api-key "sk-openai" :mini-model "gpt-5.4-mini"})](with-selected-model "anthropic/claude-sonnet-4-6"(is (= "anthropic/claude-haiku-4-5-20251001" (metabot.settings/llm-mini-model))))(testing "including a second connection of the same type, which keeps its own key"@@ -470,6 +470,11 @@(deftest llm-mini-model-falls-back-to-the-metabot-model-test(mt/with-temporary-raw-setting-values [llm-mini-model nil]+ (testing "a connection whose listing left out the cheaper model its type is known for falls through to the model+ Metabot itself uses, rather than to a guess the account cannot serve"+ (with-connections [(connection "anthropic" "anthropic" {:api-key "sk-ant-test"})]+ (with-selected-model "anthropic/claude-sonnet-4-6"+ (is (= "anthropic/claude-sonnet-4-6" (metabot.settings/llm-mini-model))))))(testing "provider types with no mini model fall through to the model Metabot itself uses"(with-connections [(connection "azure" "azure" {:api-key "azure-key":base-url "https://my-resource.services.ai.azure.com/openai"})]@@ -482,10 +487,11 @@(with-selected-model "google/endpoints/1234567890123456789"(is (= "google/endpoints/1234567890123456789" (metabot.settings/llm-mini-model))))))(testing "so does a Bedrock connection that names its model, while one without a model ID keeps its mini model"- (with-connections [(connection "bedrock" "bedrock" {:model-id "eu.anthropic.claude-sonnet-4-6"})]+ (with-connections [(connection "bedrock" "bedrock" {:model-id "eu.anthropic.claude-sonnet-4-6"+ :mini-model "anthropic.claude-haiku-4-5"})](with-selected-model "bedrock/eu.anthropic.claude-sonnet-4-6"(is (= "bedrock/eu.anthropic.claude-sonnet-4-6" (metabot.settings/llm-mini-model)))))- (with-connections [(connection "bedrock" "bedrock")]+ (with-connections [(connection "bedrock" "bedrock" {:mini-model "anthropic.claude-haiku-4-5"})](with-selected-model "bedrock/anthropic.claude-opus-4-8"(is (= "bedrock/anthropic.claude-haiku-4-5" (metabot.settings/llm-mini-model))))))(testing "so does a model reference naming a connection that does not exist"