The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
docs - script to generate page for supported AI providers (#81101)
Jeff Bruemmere8b8ad1a8895
diff --git a/.clj-kondo/config/modules/config.edn b/.clj-kondo/config/modules/config.ednindex 237ace5914f..46d0480c488 100644--- a/.clj-kondo/config/modules/config.edn+++ b/.clj-kondo/config/modules/config.edn@@ -578,6 +578,8 @@auth-identityclassloaderconfig+ llm+ metabotmodelsquery-processorsearchdiff --git a/.github/CODEOWNERS b/.github/CODEOWNERSindex 701447689c1..e60ba0f9fc1 100644--- a/.github/CODEOWNERS+++ b/.github/CODEOWNERS@@ -42,6 +42,7 @@ docs/api.json # auto-generateddocs/configuring-metabase/environment-variables.md # auto-generateddocs/configuring-metabase/config-template.md # auto-generateddocs/installation-and-operation/commands.md # auto-generated+docs/ai/providers.md # auto-generateddocs/embedding/sdk/api # auto-generateddocs/embedding/eajs/snippets # auto-generateddiff --git a/.github/workflows/docs-generate-base.yml b/.github/workflows/docs-generate-base.ymlindex 3cf73325657..d8e44fe0e9a 100644--- a/.github/workflows/docs-generate-base.yml+++ b/.github/workflows/docs-generate-base.yml@@ -75,6 +75,14 @@ jobs:clojure -M:ee:doc command-documentationcontinue-on-error: false+ - name: Generate AI provider documentation+ id: generate_ai_provider_docs+ run: |+ set -e # Exit on any error+ echo "Generating AI provider documentation..."+ clojure -M:ee:doc ai-providers-documentation+ continue-on-error: false+- name: Generate usage analytics documentationrun: ./bin/generate-usage-analytics-docs.bb@@ -164,6 +172,7 @@ jobs:- \`clojure -M:ee:doc config-template\`- \`clojure -M:ee:doc api-documentation\`- \`clojure -M:ee:doc command-documentation\`+ - \`clojure -M:ee:doc ai-providers-documentation\`- \`./bin/generate-usage-analytics-docs.bb\`- \`bun run embedding-sdk:docs:generate\`- \`bun run embedding-eajs:docs:generate\`diff --git a/.prettierignore b/.prettierignoreindex 09371d9da2d..8d74c98d7ee 100644--- a/.prettierignore+++ b/.prettierignore@@ -1,5 +1,6 @@frontend/src/cljsdocs/configuring-metabase/environment-variables.md+docs/ai/providers.mddocs/api/*docs/api-documentation.mdfrontend/src/embedding-sdk-shared/.storybook/msw-public/mockServiceWorker.jsdiff --git a/docs/ai/settings.md b/docs/ai/settings.mdindex ff531a9abee..f0bbec4ffdb 100644--- a/docs/ai/settings.md+++ b/docs/ai/settings.md@@ -70,7 +70,7 @@ To enable AI features with your own API key:1. Go to **Admin > AI**.2. Select your **Provider**.-3. Enter your **API key**. The **Get or manage keys in [provider]** link opens your provider's key management page in a new tab.+3. Enter your **API key**. The **Where do I find this?** link opens your provider's key management page in a new tab.4. Click **Connect**.5. Select a **Model** from the dropdown. Available models are fetched from the provider using your API key.diff --git a/docs/developers-guide/devenv.md b/docs/developers-guide/devenv.mdindex 37a612c74d7..da6c06bf8aa 100644--- a/docs/developers-guide/devenv.md+++ b/docs/developers-guide/devenv.md@@ -332,13 +332,13 @@ As in any clojure.test project, you can also run unit tests from the REPL. Some```clojure;; run a single test with clojure.test-some-ns=> (clojure.test/run-test metabase.util-test/add-period-test)+some-ns=> (clojure.test/run-test metabase.util-test/index-by-test)Testing metabase.util-test-Ran 1 tests containing 4 assertions.+Ran 1 tests containing 3 assertions.0 failures, 0 errors.-{:test 1, :pass 4, :fail 0, :error 0, :type :summary}+{:test 1, :pass 3, :fail 0, :error 0, :type :summary};; run all tests in the namespacesome-ns=> (clojure.test/run-tests 'metabase.util-test)diff --git a/docs/installation-and-operation/commands.md b/docs/installation-and-operation/commands.mdindex f7c9114ec97..ddc923c0e70 100644--- a/docs/installation-and-operation/commands.md+++ b/docs/installation-and-operation/commands.md@@ -18,6 +18,10 @@ java --add-opens java.base/java.nio=ALL-UNNAMED -jar metabase.jar helpMetabase will print out the help text for available commands.+## `ai-providers-documentation`++Generates a markdown file listing the AI providers Metabase can connect to, the credentials each one needs, and the models each one offers. This is written to a file called `docs/ai/providers.md`.+## `api-documentation`Generate an HTML file and a JSON file for Scalar docs for the Metabase API.diff --git a/src/metabase/cmd/ai_provider_dox.clj b/src/metabase/cmd/ai_provider_dox.cljnew file mode 100644index 00000000000..2976f6ad8ae--- /dev/null+++ b/src/metabase/cmd/ai_provider_dox.clj@@ -0,0 +1,282 @@+(ns metabase.cmd.ai-provider-dox+ "Generate the reference page for the AI providers Metabase can connect to, by running++ clojure -M:ee:doc ai-providers-documentation++ The page is written from [[metabase.llm.provider/provider-types]]."+ (:require+ [clojure.string :as str]+ [metabase.cmd.common :as cmd.common]+ [metabase.cmd.markdown :as md]+ [metabase.llm.provider :as llm.provider]+ [metabase.metabot.self :as metabot.self]))++(set! *warn-on-reflection* true)++(def ^:private output-path "docs/ai/providers.md")++(def ^:private intro-resource "metabase/cmd/resources/ai-provider-intro.md")++(def ^:private provider-notes-resources+ "Hand-written prose appended to a provider type's section, keyed by type. Only the managed provider has an entry: its+ section is otherwise a models table, and what an admin needs to know about it — that it's a Metabase Cloud offering,+ how it's billed, how it authenticates — is prose rather than anything the registry holds."+ {"metabase" "metabase/cmd/resources/ai-provider-metabase.md"})++(def ^:private dynamic-catalog-types+ "Provider types that serve whatever models the operator loaded, so there is no list to publish."+ #{"vllm"})++(def ^:private max-enumerated-options+ "Above this many `:options`, a field's choices are pointed at rather than listed. Bedrock's dozens of regions are+ noise on a docs page, and are right there in the dropdown."+ 6)++;;;; Credential fields++(defn- label+ "The `:label` of a registry entry — a provider type, a credential field, or one of a field's options — as a plain+ string."+ [x]+ ;; `str` runs a `deferred-tru` through `MessageFormat`, which is how a doubled `''` in the registry reaches the+ ;; page as a single quote+ (str (:label x)))++(defn- field-at+ "The field `k` names among `fields`. Throws when there is none."+ [fields k]+ ;; a `:show-when`, `:requires`, or `:required-any` left pointing at a renamed key would otherwise render as an+ ;; empty `****`+ (or (some #(when (= k (:key %)) %) fields)+ (throw (ex-info (str "No credential field named " (pr-str k))+ {:field k :known-fields (mapv :key fields)}))))++(defn- field-labels+ "The fields `field-keys` name among `fields`, as bolded labels joined with `and` —+ `**Access key ID** and **API key**`."+ [fields field-keys]+ (str/join " and " (map #(md/bold (label (field-at fields %))) field-keys)))++(defn- option-label+ "The label a field's form control shows for the stored `value`, or `value` itself when no `:options` entry claims it."+ [{:keys [options]} value]+ (or (some #(when (= value (:value %)) (label %)) options)+ value))++(defn- field-qualifier+ "The parenthetical after a field's name, or nil when there is nothing to say."+ [{:keys [required? advanced?]}]+ ;; the defaults get no marker: spelling out `(optional)` would stutter against the `:help` text several+ ;; fields open with+ (when-let [notes (seq (cond-> []+ required? (conj "required")+ advanced? (conj "advanced")))]+ (str "(" (str/join ", " notes) ")")))++(defn- field-condition-sentence+ "The condition that reveals a field, or nil when it is always shown."+ [{:keys [show-when]} {:keys [fields]}]+ (when-let [{:keys [field value]} show-when]+ (let [controlling (field-at fields field)]+ (str "Only when " (md/bold (label controlling))+ " is " (md/bold (option-label controlling value)) "."))))++(defn- field-requires-sentence+ "The siblings a field cannot be set without, or nil when it stands on its own. The registry keys `:requires` by+ field, so this reads the map off the provider rather than anything hung off the field itself."+ [{:keys [key]} {:keys [requires fields]}]+ (when-let [dependency-keys (seq (get requires key))]+ (str "Only together with " (field-labels fields dependency-keys) ".")))++(defn- field-hosted-sentence+ "What Metabase Cloud asks of a field that a self-hosted Metabase does not, or nil when the two agree."+ [{:keys [help hosted-help hosted-required?]}]+ (if hosted-help+ ;; nil on a page generated with a hosting token, where the registry has already folded `:hosted-help` into+ ;; `:help` and repeating it would stutter+ (when (not= (str help) (str hosted-help))+ (md/sentence hosted-help))+ (when hosted-required?+ "Required on Metabase Cloud.")))++(defn- field-options-sentence+ "The choices a `:select` or `:segmented` field offers: listed when there are at most [[max-enumerated-options]] of+ them, and otherwise pointed at. Nil when the field has no `:options`."+ [{:keys [options]}]+ (when (seq options)+ (if (<= (count options) max-enumerated-options)+ (str "One of: " (str/join ", " (map #(md/code (label %)) options)) ".")+ ;; deliberately not a count: the long lists come from bundled SDKs, and would churn this page on every bump+ (str "Pick one from the dropdown in " (md/bold "Admin > AI") "."))))++(defn- field-default-sentence+ "The value a field starts at, or nil when it has no `:default`."+ [{:keys [default] :as field}]+ (when default+ ;; a field with `:options` stores one value but displays another, so show what the form shows+ (str "Defaults to " (md/code (option-label field default)) ".")))++(defn- field-env-var-sentence+ "The environment variable that can stand in for filling the field in, or nil when none configures it. Phrased as an+ alternative rather than an instruction: the admin form is how a connection is normally set up, and a Metabase Cloud+ admin can't set these themselves at all."+ [{:keys [key]} {:keys [env-vars]}]+ (when-let [env-var (get env-vars key)]+ (str "You can also set it with the environment variable " (md/code env-var) ".")))++(defn- field-entry+ "One credential field as a Markdown bullet, rendered against its `provider`: the type's `:requires` map, the+ environment variables that configure it, and its sibling fields."+ [{:keys [help docs-url] :as field} provider]+ (md/sentences+ [;; what the field is called, carrying its qualifier: `**API key** (required).`+ (str (md/sentences [(md/bold (label field)) (field-qualifier field)]) ".")+ (field-condition-sentence field provider)+ (field-requires-sentence field provider)+ (md/sentence help)+ ;; straight after `:help`, which on Bedrock is the keyless mode this qualifies+ (field-hosted-sentence field)+ (field-options-sentence field)+ (field-default-sentence field)+ ;; the words the admin form's own link uses, so the page and the UI agree; the `?` terminates it, so no period+ (when docs-url+ (md/link "Where do I find this?" docs-url))+ (field-env-var-sentence field provider)]))++;;;; Models++(defn- known-models-table+ "A `{model-id {:display-name ... :context-window ...}}` allow-list as a table, ordered by model ID."+ [models]+ ;; the context window column is dropped when nothing in the table publishes one — DeepSeek would otherwise get a+ ;; column of dashes, which reads as missing data rather than as a column that doesn't apply+ (let [sorted (sort-by key models)+ windows? (some (comp :context-window val) sorted)+ row (fn [[model-id {:keys [display-name context-window]}]]+ (cond-> [(or display-name model-id) (md/code model-id)]+ windows? (conj (if context-window (md/thousands context-window) "—"))))]+ (md/table (cond-> ["Model" "Model ID"]+ windows? (conj "Context window (tokens)"))+ (map row sorted))))++(defn- fixed-models-table+ "A registry-pinned `[{:id ... :display_name ...} ...]` catalog as a table, ordered by ID."+ [models]+ (md/table ["Model" "Model ID"]+ (for [{:keys [id display_name]} (sort-by :id models)]+ [(or display_name id) (md/code id)])))++(defn- models-markdown+ "Where a provider type's models come from, rendered as the body of its models block: an adapter allow-list, a+ registry-pinned catalog, deployments the admin names, or a catalog only the running server knows. Throws when the+ type matches none of them."+ [{:keys [type models model-fields fields] :as provider}]+ (let [known (metabot.self/known-models type)+ provider-label (label provider)]+ (cond+ (seq known)+ (known-models-table known)++ (seq models)+ (fixed-models-table models)++ (seq model-fields)+ (str "Whichever model your deployment serves. " provider-label+ " serves the deployments you create, not a fixed catalog, so there's no list to pick from — Metabase "+ "works out the model from " (field-labels fields model-fields) " instead.")++ (contains? dynamic-catalog-types type)+ (str "Metabase lists whichever models your " provider-label " server is serving, so what you can "+ "pick depends on how you started it.")++ :else+ (throw (ex-info (str "No model source for provider type " (pr-str type)+ ". Add it to metabase.metabot.self/known-models, give it a :models or :model-fields key"+ " in the registry, or name it in dynamic-catalog-types.")+ {:provider type+ :registry-keys (vec (keys provider))})))))++;;;; Sections++(defn- provider-facts+ "The bullets under a provider's heading: how to name it in settings, and what it runs by default."+ [{:keys [type default-model mini-model managed? singleton?]}]+ (md/bullets+ [(str "Provider key: " (md/code type))+ (when default-model+ (str "Default model: " (md/code default-model)))+ (when mini-model+ (str "Model for short tasks like naming a conversation: " (md/code mini-model)))+ ;; each fact carries its own guard, so a singleton type that isn't managed still says so+ (md/sentences+ [(when managed? "Managed by Metabase, so there's nothing to configure.")+ (when singleton? "You can only connect one.")])]))++(defn- required-any-sentence+ "The `:required-any` credential groups spelled out, or nil for a type that has none."+ [{:keys [required-any fields] :as provider}]+ ;; the per-field `(optional)` markers on their own would read as though none of the credentials were needed+ (when (seq required-any)+ (str (label provider) " needs either "+ (str/join ", or " (map #(field-labels fields %) required-any))+ ".")))++(defn- credentials-section+ "What an admin has to enter to connect, and which combinations of it are enough."+ [{:keys [type fields managed?] :as provider}]+ (md/paragraphs+ [(cond+ (seq fields)+ (md/labeled-block "Credentials:" (md/bullets (map #(field-entry % provider) fields)))++ ;; only the managed provider has nothing to enter, so it gets no block at all — what it does instead of taking+ ;; credentials is prose, and lives in its `provider-notes-resources` entry. Saying so about any other type would+ ;; tell an admin their credentials are Metabase's problem when they are not+ managed?+ nil++ :else+ (throw (ex-info (str "No credential fields for provider type " (pr-str type)+ ". Give it a :fields key in the registry, or mark it :managed?.")+ {:provider type})))+ (required-any-sentence provider)]))++(defn- provider-section+ "One provider's section of the page: heading, facts, models, credentials, then whatever `notes` — a `{type markdown}`+ map of hand-written prose — holds for this type."+ [{:keys [type] :as registry-entry} notes]+ ;; one map carries the whole section: the registry entry, plus the environment variables that configure it+ (let [provider (assoc registry-entry :env-vars (llm.provider/connection-env-vars type))]+ (md/paragraphs+ [(md/heading 2 (label provider))+ (provider-facts provider)+ (md/labeled-block "Supported models:" (models-markdown provider))+ (credentials-section provider)+ (get notes type)])))++(defn- document-markdown+ "The whole page: the `intro` resource, then a section per registered provider type, each carrying whatever prose+ `notes` holds for it."+ [intro provider-types notes]+ (when (empty? provider-types)+ (throw (ex-info "No provider types found; metabase.llm.provider's registry likely moved or changed shape" {})))+ (md/document (cons intro (map #(provider-section % notes) provider-types))))++;;;; Entry point++(defn generate-dox!+ "Write the AI provider reference to `path`, defaulting to `docs/ai/providers.md`. Returns+ `{:path ... :providers n}`."+ ([]+ (generate-dox! output-path))+ ([path]+ (printf "Generating AI provider documentation in %s\n" path)+ (let [provider-types (llm.provider/provider-types)+ n (count provider-types)]+ (cmd.common/write-doc-file! path (document-markdown (cmd.common/load-resource! intro-resource)+ provider-types+ (update-vals provider-notes-resources+ cmd.common/load-resource!)))+ (printf "Wrote %s (%d providers)\n" path n)+ (println "Done.")+ {:path path :providers n})))diff --git a/src/metabase/cmd/command_dox.clj b/src/metabase/cmd/command_dox.cljindex bf650e2b404..2bf82b930fe 100644--- a/src/metabase/cmd/command_dox.clj+++ b/src/metabase/cmd/command_dox.clj@@ -8,18 +8,11 @@java --add-opens java.base/java.nio=ALL-UNNAMED -jar metabase.jar command-documentation"(:require[clojure.string :as str]- [metabase.cmd.common :as cmd.common]))+ [metabase.cmd.common :as cmd.common]+ [metabase.cmd.markdown :as md]))(set! *warn-on-reflection* true)-(def ^:private section-separator- "Separator between major sections in generated documentation"- "\n\n")--(def ^:private document-ending- "Trailing newline at end of document"- "\n")-(defn- command?"Returns true if the var has :command metadata"[[_symb varr]]@@ -50,19 +43,18 @@(str/join " | ")))(defn- format-option- "Format a single option spec as a markdown list item"+ "Format a single option spec as the contents of a Markdown bullet. The `- ` prefix is [[md/bullets]]' job."[[short-opt long-opt desc & _]](let [opt-str (->> [short-opt long-opt](remove str/blank?)(str/join ", "))]- (format "- `%s` - %s" opt-str desc)))+ (str (md/code opt-str) " - " desc)))(defn- format-options- "Format the options section for a command"+ "Format the options section for a command. Nil when the command takes none, so it drops out of the section."[arg-spec](when (seq arg-spec)- (str "\n\nOptions:\n\n"- (str/join "\n" (map format-option arg-spec)))))+ (md/labeled-block "Options:" (md/bullets (map format-option arg-spec)))))(defn- normalize-whitespace"Normalize whitespace in a string by replacing multiple spaces and newlines with single spaces"@@ -71,29 +63,20 @@str/trim(str/replace #"\s+" " ")))-(defn- format-description- "Format a docstring as markdown description"- [doc]- (some-> doc- normalize-whitespace- (->> (str "\n\n"))))-(defn- format-command"Generate markdown documentation for a single command"[[symb varr]](let [{:keys [doc arg-spec arglists]} (meta varr)- command-name (name symb)- heading (str "## `" (format-arglists command-name arglists) "`")]- (str heading- (format-description doc)- (format-options arg-spec))))+ command-name (name symb)]+ ;; a command with no docstring or no options contributes nothing, rather than a stray blank line+ (md/paragraphs [(md/heading 2 (md/code (format-arglists command-name arglists)))+ (some-> doc normalize-whitespace)+ (format-options arg-spec)])))(defn- generate-commands-section"Generate the commands section of the documentation"[]- (->> (command-vars)- (map format-command)- (str/join section-separator)))+ (md/paragraphs (map format-command (command-vars))))(defn- header-section"Generate the header section of the documentation"@@ -108,10 +91,9 @@(defn- generate-documentation"Generate the complete commands documentation"[]- (str/join section-separator- [(header-section)- (generate-commands-section)- (str (footer-section) document-ending)]))+ (md/document [(header-section)+ (generate-commands-section)+ (footer-section)]))(defn generate-dox!"Generates CLI command documentation and writes it to docs/installation-and-operation/commands.md"diff --git a/src/metabase/cmd/core.clj b/src/metabase/cmd/core.cljindex 23da694b716..a3d4fa3ffa4 100644--- a/src/metabase/cmd/core.clj+++ b/src/metabase/cmd/core.clj@@ -165,6 +165,13 @@(classloader/require 'metabase.cmd.config-file-gen)((resolve 'metabase.cmd.config-file-gen/generate-config-file-doc!)))+(defn ^:command ai-providers-documentation+ "Generates a markdown file listing the AI providers Metabase can connect to, the credentials each one needs, and the+ models each one offers. This is written to a file called `docs/ai/providers.md`."+ []+ (classloader/require 'metabase.cmd.ai-provider-dox)+ ((resolve 'metabase.cmd.ai-provider-dox/generate-dox!)))+(defn ^:command command-documentation"Generates a markdown file containing documentation for all CLI commands. This is written to a file called`docs/installation-and-operation/commands.md`."diff --git a/src/metabase/cmd/env_var_dox.clj b/src/metabase/cmd/env_var_dox.cljindex fcdedbbea71..13515c51f7e 100644--- a/src/metabase/cmd/env_var_dox.clj+++ b/src/metabase/cmd/env_var_dox.clj@@ -7,9 +7,9 @@[clojure.tools.namespace.find :as ns.find][clojure.tools.reader.edn :as edn][metabase.cmd.common :as cmd.common]+ [metabase.cmd.markdown :as md][metabase.query-processor.middleware.constraints :as qp.constraints]- [metabase.settings.core :as setting]- [metabase.util :as u]))+ [metabase.settings.core :as setting]))(defn prep-settings"Used to return a map from the registered settings atom."@@ -59,11 +59,12 @@[env-var](let [d (:default (handle-defaults-set-elsewhere env-var))](str "Default: "- (cond- (false? d) "`false`"- (nil? d) "`null`"- (keyword? d) (str "`" (name d) "`")- :else (str "`" d "`")))))+ (md/code+ (cond+ (false? d) "false"+ (nil? d) "null"+ (keyword? d) (name d)+ :else d)))))(defn- format-prefix"Formats an environment variable name with the 'MB_' prefix@@ -74,44 +75,40 @@(str base-name " [DEPRECATED]")base-name)))-(defn- format-heading- "Takes an integer and a string and creates a Markdown heading of level n."- [n s]- (str (apply str (take n (repeat "#"))) " `" s "`"))-(defn- format-description"Helper function to specify description format for environment variable docs."[env-var]- (->> ((:description env-var))- u/add-period- ;; Drop brackets used to create source code links- (#(str/replace % #"\[\[|\]\]" ""))))+ ;; `some->` so a setting with a blank description drops out of its section rather than contributing a bare "."+ (some-> ((:description env-var))+ md/sentence+ ;; Drop brackets used to create source code links+ (str/replace #"\[\[|\]\]" "")))(defn- format-deprecated"Tags a deprecated environment variable."[env-var](when-let [deprecated (:deprecated env-var)]- (if (string? deprecated)- (str "> DEPRECATED: " deprecated)- "> DEPRECATED")))+ (md/blockquote (if (string? deprecated)+ (str "DEPRECATED: " deprecated)+ "DEPRECATED"))))(def paid-message"Used to mark an env var that requires a paid plan."- "> Only available on Metabase [Pro](https://www.metabase.com/product/pro) and [Enterprise](https://www.metabase.com/product/enterprise) plans.")+ (md/blockquote (str "Only available on Metabase [Pro](https://www.metabase.com/product/pro) and "+ "[Enterprise](https://www.metabase.com/product/enterprise) plans.")))(defn- format-paid"Does the variable require a paid license?"[env-var]- (if (nil? (:feature env-var))- ""+ (when (:feature env-var)paid-message))(defn- format-export"Whether the variable is exported in serialization settings."[env-var]- (if (true? (:export? env-var))- (str "[Exported as](../installation-and-operation/serialization.md): `" (:munged-name env-var) "`.")- ""))+ (when (true? (:export? env-var))+ (str (md/link "Exported as" "../installation-and-operation/serialization.md")+ ": " (md/code (:munged-name env-var)) ".")))(defn- format-doc"Includes additional documentation for an environment variable, if it exists.@@ -138,40 +135,24 @@(not (settable-in-config-file? env-var))"Environment variable only: you can't set this in the Admin settings or in a [configuration file](./config-file.md)."- (= (:visibility env-var) :internal)- ""-- :else- (str "[Configuration file name](./config-file.md): `" (:munged-name env-var) "`")))--(defn- list-item- "Create a list item for an entry, like `- Default: 100`."- [entry]- (if (or (str/blank? entry)- (nil? entry))- ""- (str "- " entry)))--(defn- format-list- "Used to format metadata as a list."- [entries]- (str/join "\n" (remove str/blank? (map list-item entries))))+ ;; an internal setting has no config-file name to give, and nothing else to say either+ (not= (:visibility env-var) :internal)+ (str (md/link "Configuration file name" "./config-file.md") ": " (md/code (:munged-name env-var)))))(defn- format-env-var-entry"Preps a doc entry for an environment variable as a Markdown section."[env-var]- (str/join "\n\n" (remove str/blank?- [(format-heading 3 (format-prefix env-var))- (format-paid env-var)- (format-deprecated env-var)- ;; metadata we should format as a list- ;; Like `- Default: 100`- (format-list [(format-type env-var)- (format-default env-var)- (format-export env-var)- (format-config-name env-var)])- (format-description env-var)- (format-doc env-var)])))+ (md/paragraphs+ [(md/heading 3 (md/code (format-prefix env-var)))+ (format-paid env-var)+ (format-deprecated env-var)+ ;; metadata we should format as a list, like `- Default: 100`+ (md/bullets [(format-type env-var)+ (format-default env-var)+ (format-export env-var)+ (format-config-name env-var)])+ (format-description env-var)+ (format-doc env-var)]));;;; Filter functions@@ -220,19 +201,19 @@(defn- format-intro"Exists just so we can write the intro in Markdown."[]- (str (cmd.common/load-resource! "metabase/cmd/resources/env-var-intro.md") "\n\n"))+ (cmd.common/load-resource! "metabase/cmd/resources/env-var-intro.md"))(defn- non-defsetting-env-vars"Retrieves environment variables not specified via `defsetting`."[]- (str "\n\n" (cmd.common/load-resource! "metabase/cmd/resources/other-env-vars.md") "\n"))+ (cmd.common/load-resource! "metabase/cmd/resources/other-env-vars.md"))(defn prep-dox"Preps the environment variable docs for printing."[]- (apply str (format-intro)- (str/join "\n\n" (format-env-var-docs (get-settings)))- (non-defsetting-env-vars)))+ (md/document (concat [(format-intro)]+ (format-env-var-docs (get-settings))+ [(non-defsetting-env-vars)])))(defn generate-dox!"Prints the generated environment variable docs to a file."diff --git a/src/metabase/cmd/markdown.clj b/src/metabase/cmd/markdown.cljnew file mode 100644index 00000000000..b55e97cb2be--- /dev/null+++ b/src/metabase/cmd/markdown.clj@@ -0,0 +1,102 @@+(ns metabase.cmd.markdown+ "Markdown primitives shared by the documentation generators in [[metabase.cmd]].++ Everything here is a pure string function: the generators read their own source of truth — the settings registry,+ the LLM provider registry, the CLI command vars — and lean on these to render it. Resource IO lives in+ [[metabase.cmd.common]] instead."+ (:require+ [clojure.string :as str]))++(set! *warn-on-reflection* true)++(defn code+ "`s` as inline code."+ [s]+ (str "`" s "`"))++(defn bold+ "`s` in bold."+ [s]+ (str "**" s "**"))++(defn link+ "`text` as a link to `url`."+ [text url]+ (str "[" text "](" url ")"))++(defn heading+ "`s` as a level-`n` heading. Callers that want a backticked heading compose the two: `(heading 3 (code s))`."+ [n s]+ (str (str/join (repeat n "#")) " " s))++(defn blockquote+ "`s` as a blockquote. Every line is prefixed, so a quote that runs to several lines stays one quote rather than+ breaking out into body text after the first."+ [s]+ (str/join "\n" (map #(str "> " %) (str/split-lines (str s)))))++(defn thousands+ "`1000000` -> `\"1,000,000\"`."+ [n]+ ;; not `format`, whose grouping separator follows the default locale and so would differ between a laptop and CI+ (str/replace (str n) #"(\d)(?=(\d{3})+$)" "$1,"))++(defn sentence+ "`s` as a sentence: forced out of i18n and terminated with a period. Nil when there is nothing to say, so a field+ whose text is blank contributes no stray `.` to its bullet.++ Knows about the endings docs prose runs into: a trailing `:` reads as a dangling lead-in once the text stands on+ its own, so it becomes a period, and a fenced code block is left exactly as it is."+ [s]+ (let [text (str s)]+ (cond+ (str/blank? text) nil+ (#{\. \? \!} (last text)) text+ (str/ends-with? text "```") text+ (str/ends-with? text ":") (str (subs text 0 (dec (count text))) ".")+ :else (str text "."))))++(defn- parts+ "`xs` stringified and trimmed, with the empties dropped — a nil `:help` leaves no gap where it was, and a resource+ carrying its own trailing newline opens no extra blank line."+ [xs]+ (into [] (comp (map #(str/trim (str %))) (remove str/blank?)) xs))++(defn sentences+ "Join the non-blank parts into one run of prose, a space between them."+ [xs]+ (str/join " " (parts xs)))++(defn paragraphs+ "Join the non-blank parts with blank lines between them."+ [xs]+ (str/join "\n\n" (parts xs)))++(defn bullets+ "Join the non-blank items as a Markdown list."+ [xs]+ (str/join "\n" (map #(str "- " %) (parts xs))))++(defn document+ "`xs` as a finished page: joined as [[paragraphs]] and terminated with exactly one newline."+ [xs]+ (str (paragraphs xs) "\n"))++(defn labeled-block+ "A `label` line, then `body` beneath it — `Options:` over its list, `Credentials:` over its bullets."+ [label body]+ (paragraphs [label body]))++(defn table+ "A Markdown table. `rows` is a sequence of already-rendered cell vectors."+ [headers rows]+ ;; columns are padded to a common width: the rendered page is what reviewers read in a PR diff, and a ragged table+ ;; is hard to scan there even though it renders identically+ (let [widths (apply mapv (fn [& cells] (apply max (map count cells))) headers rows)+ row->line (fn [cells]+ (str "| " (str/join " | " (map #(format (str "%-" %2 "s") %1) cells widths)) " |"))]+ (str/join "\n"+ ;; the separator is just another row — its dashes are already exactly column width, so it pads to itself+ (list* (row->line headers)+ (row->line (map #(str/join (repeat % "-")) widths))+ (map row->line rows)))))diff --git a/src/metabase/cmd/resources/ai-provider-intro.md b/src/metabase/cmd/resources/ai-provider-intro.mdnew file mode 100644index 00000000000..97f70047732--- /dev/null+++ b/src/metabase/cmd/resources/ai-provider-intro.md@@ -0,0 +1,18 @@+---+title: Supported AI providers+summary: The AI providers Metabase can connect to, the credentials each one needs, and the models each one offers.+---++# Supported AI providers++_This documentation was generated from source by running:_++```+clojure -M:ee:doc ai-providers-documentation+```++To power [Metabot](./metabot.md), you can connect Metabase to one of the providers below with your own credentials, or let Metabase manage the AI for you. For how to set up a connection, check out [AI settings](./settings.md).++You set up a provider in **Admin > AI**. If you self-host Metabase, you can set a provider's credentials with environment variables instead; each credential below lists the variable that sets it. You don't have to use environment variables. On Metabase Cloud, [contact support](https://www.metabase.com/help-premium) if you want environment variables set for your instance.++If you want Metabase to support a provider or model that isn't listed here, let us know by submitting a [feature request](../troubleshooting-guide/requesting-new-features.md).diff --git a/src/metabase/cmd/resources/ai-provider-metabase.md b/src/metabase/cmd/resources/ai-provider-metabase.mdnew file mode 100644index 00000000000..068cbf47034--- /dev/null+++ b/src/metabase/cmd/resources/ai-provider-metabase.md@@ -0,0 +1,5 @@+On Metabase Cloud, you can have Metabase manage the AI for you. Metabase selects benchmarked, cost-effective models, so this is a good option if you don't have a preferred AI provider, or if you want to manage your AI costs through Metabase. You'll be charged based on token usage, on top of your Metabase Cloud subscription. See [Pricing](https://www.metabase.com/pricing).++Metabase authenticates this connection with your instance's license token, so there's no API key to enter.++For how to connect and disconnect, see [AI settings](./settings.md#metabase-ai-service).diff --git a/src/metabase/llm/provider.clj b/src/metabase/llm/provider.cljindex 6b95bf472b4..76201de5967 100644--- a/src/metabase/llm/provider.clj+++ b/src/metabase/llm/provider.clj@@ -542,6 +542,21 @@:settings {:base-url {:setting :llm-vllm-api-base-url :credential? true}:api-key {:setting :llm-vllm-api-key}}}})+(defn connection-env-vars+ "The environment variables that configure a connection of `type-name`, as `{config-field \"MB_LLM_...\"}`.++ Returns nil for a type no per-provider variable configures — the managed provider, which holds no credentials of+ its own, is the only one today. Setting these is the supported way to configure a single connection without writing+ JSON into [[metabase.llm.settings/llm-providers]]."+ [type-name]+ (when-let [{:keys [settings]} (get single-provider-settings type-name)]+ (not-empty+ (into {}+ (keep (fn [{field-key :key}]+ (when-let [setting-kw (get-in settings [field-key :setting])]+ [field-key (setting/env-var-name setting-kw)])))+ (:fields (provider-type type-name))))))+(defn- env-supplied-fields"The `:config` fields the environment supplies for one [[single-provider-settings]] group:`{:config {field value} :vars {field \"MB_...\"} :credential-set? bool}`."diff --git a/src/metabase/metabot/self.clj b/src/metabase/metabot/self.cljindex 13ad71696a3..6999359231b 100644--- a/src/metabase/metabot/self.clj+++ b/src/metabase/metabot/self.clj@@ -70,6 +70,43 @@(throw (ex-info (str "Unknown LLM provider: " provider){:provider provider}))))+(defn- normalize-known-model+ "Coerce one adapter's `supported-models` value into `{:display-name ... :context-window ...}`. Most adapters store a+ map already; DeepSeek stores the display name on its own. Anything else throws, so an adapter that invents a third+ shape fails loudly instead of quietly documenting a model with no name."+ [provider model-id value]+ (cond+ (map? value) value+ (string? value) {:display-name value}+ :else (throw (ex-info (str "Unrecognized supported-models entry for " provider)+ {:provider provider :model model-id :value value}))))++(defn known-models+ "The models `provider`'s adapter is willing to offer, as `{model-id {:display-name ... :context-window ...}}`.++ This is the allow-list [[list-models]] intersects with the provider's live catalog, so a model listed here is+ available only if the connection's credentials can actually reach it. Returns nil for the provider types that have+ no allow-list: `azure`, whose model is the deployment name the admin gives it, `vllm`, which serves whatever the+ operator loaded, and `google` and `metabase`, whose catalogs are fixed in [[metabase.llm.provider]] instead."+ [provider]+ ;; a `case` like [[resolve-adapter]], so a new adapter that forgets to register here throws rather than reading as+ ;; a provider that simply has no models+ (when-let [models (case provider+ "anthropic" claude/supported-models+ "bedrock" bedrock/supported-models+ "deepseek" deepseek/supported-models+ "mistral" mistral/supported-models+ "moonshot" moonshot/supported-models+ "openai" openai/supported-models+ "openrouter" openrouter/supported-models+ "zai" zai/supported-models+ ("azure" "google" "metabase" "vllm") nil+ (throw (ex-info (str "Unknown LLM provider: " provider)+ {:provider provider})))]+ (into {}+ (map (fn [[model-id value]] [model-id (normalize-known-model provider model-id value)]))+ models)))+(defn- parse-provider-model"Resolve a `connection-key/model` string into the adapter, model, and credentials needed to serve it.Throws a 400 when the string names a connection that is not configured, so a stalediff --git a/src/metabase/metabot/self/bedrock.clj b/src/metabase/metabot/self/bedrock.cljindex f465d0512d0..100fcf8961b 100644--- a/src/metabase/metabot/self/bedrock.clj+++ b/src/metabase/metabot/self/bedrock.clj@@ -171,7 +171,7 @@(catch Exception e(core/rethrow-api-error! "bedrock" bedrock-error-msg e))))-(def ^:private supported-models+(def supported-models"Bedrock models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the mantle `/v1/models` catalog.Excludes `openai.gpt-oss*`, which are not invokable through the mantle `/openai/v1` routes.diff --git a/src/metabase/metabot/self/claude.clj b/src/metabase/metabot/self/claude.cljindex 136e763f896..433c7f442e6 100644--- a/src/metabase/metabot/self/claude.clj+++ b/src/metabase/metabot/self/claude.clj@@ -336,7 +336,7 @@529 (tru "Anthropic API is overloaded and is asking us to wait")(tru "Anthropic API error (HTTP {0})" status))))-(def ^:private supported-models+(def supported-models"Anthropic chat models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the account's `/v1/models` catalog."{"claude-fable-5" {:display-name "Claude Fable 5" :max-tokens 128000 :context-window 1000000}diff --git a/src/metabase/metabot/self/deepseek.clj b/src/metabase/metabot/self/deepseek.cljindex 8650036d157..5b7400afd2f 100644--- a/src/metabase/metabot/self/deepseek.clj+++ b/src/metabase/metabot/self/deepseek.clj@@ -33,7 +33,7 @@(def ^:private messages-path "/anthropic/v1/messages")-(def ^:private supported-models+(def supported-models"DeepSeek models offered in the Metabot model picker, as a map of model id -> display name.`list-models` returns the intersection of this map with the `/models` catalog."{"deepseek-v4-flash" "DeepSeek V4 Flash"diff --git a/src/metabase/metabot/self/mistral.clj b/src/metabase/metabot/self/mistral.cljindex 8c90fb6c446..cc0d283a10e 100644--- a/src/metabase/metabot/self/mistral.clj+++ b/src/metabase/metabot/self/mistral.clj@@ -34,7 +34,7 @@500 (tru "Mistral returned an internal server error")(tru "Mistral API error (HTTP {0})" status))))-(def ^:private supported-models+(def supported-models"Mistral models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the `/models` catalog."{"mistral-medium-3-5" {:display-name "Mistral Medium 3.5" :context-window 262144}})diff --git a/src/metabase/metabot/self/moonshot.clj b/src/metabase/metabot/self/moonshot.cljindex 941f7a879ee..ec79ab7b899 100644--- a/src/metabase/metabot/self/moonshot.clj+++ b/src/metabase/metabot/self/moonshot.clj@@ -40,7 +40,7 @@500 (tru "Moonshot returned an internal server error")(tru "Moonshot API error (HTTP {0})" status))))-(def ^:private supported-models+(def supported-models"Moonshot models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the `/models` catalog.diff --git a/src/metabase/metabot/self/openai.clj b/src/metabase/metabot/self/openai.cljindex 6f3e0e91b57..e2f7a31312f 100644--- a/src/metabase/metabot/self/openai.clj+++ b/src/metabase/metabot/self/openai.clj@@ -250,7 +250,7 @@500 (tru "OpenAI API is not working but not saying why")(tru "OpenAI API error (HTTP {0})" status))))-(def ^:private supported-models+(def supported-models"OpenAI chat models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the account's `/v1/models` catalog."{"gpt-5.6-sol" {:display-name "GPT-5.6 Sol" :context-window 922000}diff --git a/src/metabase/metabot/self/openrouter.clj b/src/metabase/metabot/self/openrouter.cljindex 92b8b6a0d52..8a4e070d24f 100644--- a/src/metabase/metabot/self/openrouter.clj+++ b/src/metabase/metabot/self/openrouter.clj@@ -41,7 +41,7 @@503 (tru "OpenRouter service is unavailable")(tru "OpenRouter API error (HTTP {0})" status))))-(def ^:private supported-models+(def supported-models"OpenRouter models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the `/v1/models` catalog.Mirrors the models whitelisted for the direct anthropic and openai providers; note thatdiff --git a/src/metabase/metabot/self/zai.clj b/src/metabase/metabot/self/zai.cljindex 14a26971e5f..5edf2af3fc6 100644--- a/src/metabase/metabot/self/zai.clj+++ b/src/metabase/metabot/self/zai.clj@@ -34,7 +34,7 @@500 (tru "Z.AI returned an internal server error")(tru "Z.AI API error (HTTP {0})" status))))-(def ^:private supported-models+(def supported-models"Z.AI models offered in the Metabot model picker, keyed by model id.`list-models` returns the intersection of this map with the `/models` catalog."{"glm-5.2" {:display-name "GLM-5.2" :context-window 1048576}})diff --git a/src/metabase/util.cljc b/src/metabase/util.cljcindex 8c181202acc..14dc70c5cc4 100644--- a/src/metabase/util.cljc+++ b/src/metabase/util.cljc@@ -186,19 +186,6 @@%)m))-(defn add-period- "Fixes strings that don't terminate in a period; also accounts for strings- that end in `:` and triple backticks (e.g., if a string ends in codeblock).- Used for formatting docs."- [s]- (let [text (str s)]- (cond- (str/blank? text) text- (#{\. \? \!} (last text)) text- (str/ends-with? text "```") text- (str/ends-with? text ":") (str (subs text 0 (dec (count text))) ".")- :else (str text "."))))-(defn trimmed-string"`value` trimmed of surrounding whitespace, or nil when it is not a string or has nothing left once trimmed."^String [value]diff --git a/test/metabase/cmd/ai_provider_dox_test.clj b/test/metabase/cmd/ai_provider_dox_test.cljnew file mode 100644index 00000000000..fcd60555096--- /dev/null+++ b/test/metabase/cmd/ai_provider_dox_test.clj@@ -0,0 +1,292 @@+(ns metabase.cmd.ai-provider-dox-test+ (:require+ [clojure.string :as str]+ [clojure.test :refer :all]+ [metabase.cmd.ai-provider-dox :as ai-provider-dox]+ [metabase.llm.provider :as llm.provider]+ [metabase.metabot.self :as metabot.self]+ [metabase.test :as mt]+ [metabase.util.i18n :refer [deferred-tru]]))++(set! *warn-on-reflection* true)++(def ^:private api-key-field+ {:key :api-key+ :label (deferred-tru "API key")+ :type :password+ :required? true+ :docs-url "https://example.com/keys"})++(def ^:private base-url-field+ {:key :base-url+ :label (deferred-tru "API base URL")+ :type :text+ :advanced? true+ :default "https://example.com"})++;;; Bedrock's shape since BOT-1918: the key pair is optional on a self-hosted Metabase, which can sign with the AWS+;;; default credentials chain instead, required on Metabase Cloud, and half a pair authenticates nothing.++(def ^:private access-key-field+ {:key :access-key-id+ :label (deferred-tru "Access key ID")+ :type :password+ :help (deferred-tru "Leave the keys blank to authenticate with the AWS default credentials chain.")+ :hosted-required? true+ :hosted-help (deferred-tru "On Metabase Cloud, Bedrock always authenticates with your own AWS keys.")})++(def ^:private secret-key-field+ {:key :secret-access-key+ :label (deferred-tru "Secret access key")+ :type :password+ :hosted-required? true})++(defn- provider+ "The per-provider map the renderers take: the registry entry's own keys, plus the environment variables that+ configure a connection of that type."+ [fields & {:keys [requires env-vars]}]+ {:fields fields :requires requires :env-vars env-vars})++(defn- registry-entry+ "The live registry entry for `type-name`, which is what the section renderers take."+ [type-name]+ (first (filter #(= type-name (:type %)) (llm.provider/provider-types))))++(deftest ^:parallel field-at-test+ (testing "a `:show-when` or `:required-any` left pointing at a renamed key fails loudly"+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"No credential field named :nope"+ (#'ai-provider-dox/field-at [api-key-field] :nope)))))++(deftest ^:parallel field-qualifier-test+ (testing "only the facts that aren't the default get a marker"+ (is (= "(required)" (#'ai-provider-dox/field-qualifier api-key-field)))+ (is (= "(advanced)" (#'ai-provider-dox/field-qualifier base-url-field)))+ (is (= "(required, advanced)"+ (#'ai-provider-dox/field-qualifier {:required? true :advanced? true}))))+ (testing "a plain optional field gets no parenthetical, so `:help` opening with \"Optional\" doesn't stutter"+ (is (nil? (#'ai-provider-dox/field-qualifier {})))))++(deftest ^:parallel field-options-sentence-test+ (testing "a short list of choices is spelled out"+ (is (= "One of: `OpenAI`, `Anthropic`."+ (#'ai-provider-dox/field-options-sentence+ {:options [{:value "openai" :label "OpenAI"} {:value "anthropic" :label "Anthropic"}]}))))+ (testing "a long list is pointed at instead — but still says the field is a picker"+ (let [sentence (#'ai-provider-dox/field-options-sentence+ {:options (for [n (range (inc @#'ai-provider-dox/max-enumerated-options))]+ {:value (str n) :label (str "Region " n)})})]+ (is (= "Pick one from the dropdown in **Admin > AI**." sentence))+ (is (not (str/includes? sentence "Region 0")))))+ (testing "a field with no choices says nothing"+ (is (nil? (#'ai-provider-dox/field-options-sentence api-key-field)))))++(deftest ^:parallel field-default-sentence-test+ (testing "a plain default is shown as stored"+ (is (= "Defaults to `https://example.com`." (#'ai-provider-dox/field-default-sentence base-url-field))))+ (testing "a default with `:options` is shown as the label the form displays, not the stored value"+ (is (= "Defaults to `OpenAI`."+ (#'ai-provider-dox/field-default-sentence+ {:default "openai"+ :options [{:value "openai" :label "OpenAI"} {:value "anthropic" :label "Anthropic"}]}))))+ (testing "no default, nothing to say"+ (is (nil? (#'ai-provider-dox/field-default-sentence api-key-field)))))++(deftest ^:parallel field-requires-sentence-test+ (let [fields [access-key-field secret-key-field api-key-field]+ requires {:access-key-id [:secret-access-key]+ :secret-access-key [:access-key-id :api-key]}+ bedrock (provider fields :requires requires)]+ (testing "a field that needs one sibling names it"+ (is (= "Only together with **Secret access key**."+ (#'ai-provider-dox/field-requires-sentence access-key-field bedrock))))+ (testing "a field that needs several names them all"+ (is (= "Only together with **Access key ID** and **API key**."+ (#'ai-provider-dox/field-requires-sentence secret-key-field bedrock))))+ (testing "a field that stands on its own says nothing"+ (is (nil? (#'ai-provider-dox/field-requires-sentence api-key-field bedrock))))+ (testing "a `:requires` left pointing at a renamed key fails loudly"+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"No credential field named :nope"+ (#'ai-provider-dox/field-requires-sentence+ access-key-field (assoc bedrock :requires {:access-key-id [:nope]})))))))++(deftest ^:parallel field-hosted-sentence-test+ (testing "a field whose help changes on Metabase Cloud says how"+ (is (= "On Metabase Cloud, Bedrock always authenticates with your own AWS keys."+ (#'ai-provider-dox/field-hosted-sentence access-key-field))))+ (testing "a field Metabase Cloud merely requires says that much"+ (is (= "Required on Metabase Cloud."+ (#'ai-provider-dox/field-hosted-sentence secret-key-field))))+ (testing "generated with a hosting token the registry has already folded the text into `:help`, so don't stutter"+ (is (nil? (#'ai-provider-dox/field-hosted-sentence+ (assoc access-key-field :help (:hosted-help access-key-field))))))+ (testing "a field Metabase Cloud treats no differently says nothing"+ (is (nil? (#'ai-provider-dox/field-hosted-sentence api-key-field)))))++(deftest ^:parallel field-entry-test+ (testing "a required field names itself, links to its docs in the admin form's own words, and gives its env var"+ (is (= (str "**API key** (required). [Where do I find this?](https://example.com/keys) "+ "You can also set it with the environment variable `MB_LLM_EXAMPLE_API_KEY`.")+ (#'ai-provider-dox/field-entry+ api-key-field (provider [api-key-field] :env-vars {:api-key "MB_LLM_EXAMPLE_API_KEY"})))))+ (testing "a field with no environment variable simply omits it"+ (is (= "**API base URL** (advanced). Defaults to `https://example.com`."+ (#'ai-provider-dox/field-entry base-url-field (provider [base-url-field])))))+ (testing "a `:show-when` field says which choice reveals it, by the label the form shows for that choice"+ (let [method {:key :auth-method+ :label (deferred-tru "Authentication method")+ :type :segmented+ :options [{:value "key" :label (deferred-tru "Service account key")}+ {:value "oauth" :label (deferred-tru "OAuth token")}]}+ keyfile {:key :service-account-key+ :label (deferred-tru "Service account key file")+ :type :file+ :show-when {:field :auth-method :value "key"}}]+ (is (= "**Service account key file**. Only when **Authentication method** is **Service account key**."+ (#'ai-provider-dox/field-entry keyfile (provider [method keyfile]))))))+ (testing "a field optional here, required on Metabase Cloud, and useless without its partner says all three"+ (let [fields [access-key-field secret-key-field]]+ (is (= (str "**Access key ID**. Only together with **Secret access key**. "+ "Leave the keys blank to authenticate with the AWS default credentials chain. "+ "On Metabase Cloud, Bedrock always authenticates with your own AWS keys. "+ "You can also set it with the environment variable `MB_LLM_BEDROCK_ACCESS_KEY_ID`.")+ (#'ai-provider-dox/field-entry+ access-key-field+ (provider fields+ :requires {:access-key-id [:secret-access-key]}+ :env-vars {:access-key-id "MB_LLM_BEDROCK_ACCESS_KEY_ID"})))))))++(deftest ^:parallel known-models-table-test+ (testing "an adapter allow-list becomes a table with context windows"+ (let [markdown (#'ai-provider-dox/known-models-table+ {"claude-haiku" {:display-name "Claude Haiku 4.5" :context-window 200000}})]+ (is (str/includes? markdown "| Context window (tokens) |"))+ (is (str/includes? markdown "| 200,000"))))+ (testing "a model with no context window still gets a row, dashed out under its neighbours"+ (is (str/includes? (#'ai-provider-dox/known-models-table+ {"deepseek-v4-flash" {:display-name "DeepSeek V4 Flash"}+ "deepseek-v4-pro" {:display-name "DeepSeek V4 Pro" :context-window 131072}})+ "| —")))+ (testing "a provider that publishes no context windows at all loses the column, rather than showing dashes"+ (let [markdown (#'ai-provider-dox/known-models-table+ {"deepseek-v4-pro" {:display-name "DeepSeek V4 Pro"}})]+ (is (str/includes? markdown "| Model "))+ (is (not (str/includes? markdown "Context window")))+ (is (not (str/includes? markdown "—"))))))++(deftest ^:parallel fixed-models-table-test+ (testing "a registry-pinned catalog becomes a table with no context window column"+ (let [markdown (#'ai-provider-dox/fixed-models-table+ [{:id "google/gemini-3.5-flash" :display_name "gemini-3.5-flash"}])]+ (is (str/includes? markdown "| Model "))+ (is (not (str/includes? markdown "Context window"))))))++(deftest ^:parallel models-markdown-test+ (testing "each provider type renders from the source its models really come from"+ (is (str/includes? (#'ai-provider-dox/models-markdown (registry-entry "anthropic"))+ "| Context window (tokens) |"))+ (is (str/includes? (#'ai-provider-dox/models-markdown (registry-entry "google"))+ "| Model "))+ (is (str/includes? (#'ai-provider-dox/models-markdown (registry-entry "vllm"))+ "whichever models your vLLM server is serving")))+ (testing "Azure explains that the model comes from the deployment instead, naming the fields by their labels"+ ;; answer first: the block sits under a "Supported models:" label, so it leads with what you get rather than+ ;; with the catalog it hasn't got+ (is (str/starts-with? (#'ai-provider-dox/models-markdown (registry-entry "azure"))+ "Whichever model your deployment serves."))+ (is (str/includes? (#'ai-provider-dox/models-markdown (registry-entry "azure"))+ "**Model provider** and **Deployment name**")))+ (testing "a deployment field left pointing at a renamed key fails loudly"+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"No credential field named :nope"+ (#'ai-provider-dox/models-markdown+ (assoc (registry-entry "azure") :model-fields [:nope])))))+ (testing "a provider added to the registry but wired up nowhere fails loudly"+ ;; two guards stand behind this, and a brand-new type hits the first one: `known-models` refuses to answer for a+ ;; provider its `case` doesn't name, rather than shrugging and returning nil+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"Unknown LLM provider"+ (#'ai-provider-dox/models-markdown {:type "brand-new"})))))++(deftest models-markdown-unwired-provider-test+ (testing "a provider known to have no allow-list, and no other model source either, fails loudly too"+ ;; the second guard: registering a type as having no allow-list isn't enough to document it+ (mt/with-dynamic-fn-redefs [metabot.self/known-models (constantly nil)]+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"No model source for provider type"+ (#'ai-provider-dox/models-markdown {:type "brand-new"}))))))++(deftest ^:parallel provider-section-test+ (testing "a section heads with the provider's label, then runs facts, models, and credentials in that order"+ (let [markdown (#'ai-provider-dox/provider-section (registry-entry "anthropic") nil)]+ (is (str/starts-with? markdown "## Anthropic\n\n"))+ (is (< (str/index-of markdown "Provider key:")+ (str/index-of markdown "Supported models:")+ (str/index-of markdown "Credentials:")))+ (testing "the facts and the per-field environment variable both come off the registry entry"+ (is (str/includes? markdown "Default model: `claude-sonnet-4-6`"))+ (is (str/includes? markdown+ "You can also set it with the environment variable `MB_LLM_ANTHROPIC_API_KEY`.")))))+ (testing "a field's references to its siblings are rendered as labels, not keys"+ (let [markdown (#'ai-provider-dox/provider-section (registry-entry "google") nil)]+ (is (str/includes? markdown+ (str "needs either **Service account key file**, or "+ "**OAuth access token** and **Project ID**.")))+ (is (str/includes? markdown "Only when **Authentication method** is **Service account key**."))+ (testing "a field with no `:show-when` carries no condition of its own"+ (let [project-id (->> (str/split-lines markdown)+ (filter #(str/starts-with? % "- **Project ID**"))+ first)]+ (is (some? project-id))+ (is (not (str/includes? project-id "Only when")))))))+ (testing "the managed provider has no credentials list to label, and closes on its hand-written prose instead"+ (let [markdown (#'ai-provider-dox/provider-section (registry-entry "metabase")+ {"metabase" "There's no API key to enter."})]+ (is (not (str/includes? markdown "Credentials:")))+ (is (str/ends-with? markdown "\n\nThere's no API key to enter."))))+ (testing "a type with no prose of its own picks up nobody else's"+ (is (not (str/includes? (#'ai-provider-dox/provider-section (registry-entry "anthropic")+ {"metabase" "There's no API key to enter."})+ "There's no API key to enter.")))))++(deftest ^:parallel document-markdown-requires-providers-test+ (testing "an empty registry means the source moved, not that Metabase supports nothing"+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"No provider types found"+ (#'ai-provider-dox/document-markdown "# Intro" [] nil)))))++(deftest generate-dox-test+ (mt/with-temp-file [path]+ (let [{:keys [providers]} (ai-provider-dox/generate-dox! path)+ markdown (slurp path)]+ (testing "reports what it wrote"+ (is (= (count (llm.provider/provider-types)) providers)))+ (testing "the intro template is included"+ (is (str/starts-with? markdown "---\ntitle: Supported AI providers")))+ (testing "every provider in the registry gets a section"+ ;; the point of the page: a provider added to the registry can't quietly go undocumented+ (doseq [{:keys [label type]} (llm.provider/provider-types)]+ (is (str/includes? markdown (str "## " label))+ (str "no section for " type))+ (is (str/includes? markdown (str "Provider key: `" type "`"))+ (str "no provider key for " type))))+ (testing "each section runs provider facts, then models, then credentials"+ (let [section (second (str/split markdown #"\n## Anthropic\n"))]+ (is (< (str/index-of section "Provider key:")+ (str/index-of section "Supported models:")+ (str/index-of section "Credentials:")))))+ (testing "per-provider environment variables make it onto the page"+ (is (str/includes? markdown "`MB_LLM_ANTHROPIC_API_KEY`")))+ (testing "environment variables read as an alternative to the admin form, not as the way to configure"+ (is (str/includes? markdown+ "You can also set it with the environment variable `MB_LLM_ANTHROPIC_API_KEY`."))+ (is (not (str/includes? markdown "Set it with `MB"))))+ (testing "the managed provider's prose is inlined, rather than the section being a link elsewhere"+ (let [section (second (str/split markdown #"\n## Metabase AI service\n"))]+ (is (some? section))+ (is (str/includes? section "there's no API key to enter"))+ (is (str/includes? section "[Pricing](https://www.metabase.com/pricing)"))))+ (testing "the page ends with exactly one newline"+ (is (str/ends-with? markdown "\n"))+ (is (not (str/ends-with? markdown "\n\n")))))))diff --git a/test/metabase/cmd/command_dox_test.clj b/test/metabase/cmd/command_dox_test.cljindex ee058e65558..05b3a3ab80d 100644--- a/test/metabase/cmd/command_dox_test.clj+++ b/test/metabase/cmd/command_dox_test.clj@@ -27,27 +27,27 @@(#'sut/format-arglists "command" '([arg1] [arg1 arg2])))))))(deftest ^:parallel format-option-test- (testing "format-option handles option specs correctly"+ (testing "format-option returns the bullet contents; the `- ` prefix is md/bullets' job"(testing "short and long option"- (is (= "- `-k, --keep-existing` - Do not delete target file"+ (is (= "`-k, --keep-existing` - Do not delete target file"(#'sut/format-option ["-k" "--keep-existing" "Do not delete target file"]))))(testing "long option only"- (is (= "- `--full-stacktrace` - Output full stacktraces"+ (is (= "`--full-stacktrace` - Output full stacktraces"(#'sut/format-option [nil "--full-stacktrace" "Output full stacktraces"]))))(testing "short option only"- (is (= "- `-e` - Continue on error"+ (is (= "`-e` - Continue on error"(#'sut/format-option ["-e" nil "Continue on error"]))))(testing "option with additional metadata is ignored"- (is (= "- `-c, --collection` - Export only specified IDs"+ (is (= "`-c, --collection` - Export only specified IDs"(#'sut/format-option ["-c" "--collection" "Export only specified IDs" :id :collection-ids]))))(testing "empty string for short option should be ignored"- (is (= "- `--full-stacktrace` - Output full stacktraces"+ (is (= "`--full-stacktrace` - Output full stacktraces"(#'sut/format-option ["" "--full-stacktrace" "Output full stacktraces"]))))(testing "empty string for long option should be ignored"- (is (= "- `-e` - Continue on error"+ (is (= "`-e` - Continue on error"(#'sut/format-option ["-e" "" "Continue on error"]))))(testing "both options as empty strings should handle gracefully"- (is (= "- `` - Some description"+ (is (= "`` - Some description"(#'sut/format-option ["" "" "Some description"]))))))(deftest ^:parallel format-options-testdiff --git a/test/metabase/cmd/env_var_dox_test.clj b/test/metabase/cmd/env_var_dox_test.cljindex c67a0adbd4c..25c725c9e19 100644--- a/test/metabase/cmd/env_var_dox_test.clj+++ b/test/metabase/cmd/env_var_dox_test.clj@@ -97,7 +97,7 @@:setter :none:visibility :internal})))))(testing "Internal settings with a setter get no config file line"- (is (= "" (#'sut/format-config-name {:munged-name "internal-setting"+ (is (nil? (#'sut/format-config-name {:munged-name "internal-setting":visibility :internal}))))(testing "Settable, non-internal settings get their config file name"(is (= "[Configuration file name](./config-file.md): `admin-email`"diff --git a/test/metabase/cmd/markdown_test.clj b/test/metabase/cmd/markdown_test.cljnew file mode 100644index 00000000000..cf30064a486--- /dev/null+++ b/test/metabase/cmd/markdown_test.clj@@ -0,0 +1,83 @@+(ns metabase.cmd.markdown-test+ (:require+ [clojure.test :refer :all]+ [metabase.cmd.markdown :as md]))++(set! *warn-on-reflection* true)++(deftest ^:parallel sentence-test+ (testing "text that doesn't terminate itself gets a period"+ (is (= "Only needed for temporary credentials."+ (md/sentence "Only needed for temporary credentials"))))+ (testing "text that already terminates is left alone, whichever mark it ends on"+ (is (= "Defaults to global." (md/sentence "Defaults to global.")))+ (is (= "Which model?" (md/sentence "Which model?")))+ (is (= "This one doesn't need a period!" (md/sentence "This one doesn't need a period!"))))+ (testing "the endings `sentence` knows about, which a docs page runs into"+ ;; a trailing colon reads as a dangling lead-in once the text is a+ ;; bullet of its own, and a fenced code block must not pick up a period after its closing fence+ (is (= "Pick one of." (md/sentence "Pick one of:")))+ (is (= "Run:\n```\nls\n```" (md/sentence "Run:\n```\nls\n```"))))+ (testing "nothing to say, nothing rendered — a blank `:help` must not become a bare `.`"+ (is (nil? (md/sentence nil)))+ (is (nil? (md/sentence " ")))))++(deftest ^:parallel joining-test+ (testing "blank parts drop out rather than leaving a gap where they were"+ (is (= "One. Two." (md/sentences ["One." nil "" "Two."])))+ (is (= "One.\n\nTwo." (md/paragraphs ["One." nil "" "Two."])))+ (is (= "- One\n- Two" (md/bullets ["One" nil "" "Two"]))))+ (testing "each paragraph is trimmed, so a resource carrying its own trailing newline opens no extra blank line"+ (is (= "Intro\n\nBody" (md/paragraphs ["Intro\n" "\nBody\n"])))+ (testing "only at the ends: a table or a bullet list keeps the newlines inside it"+ (is (= "- One\n- Two\n\nBody" (md/paragraphs ["- One\n- Two\n" "Body"])))))+ (testing "nothing to join is the empty string, so a caller can pass the result straight to `paragraphs`"+ (is (= "" (md/sentences [])))+ (is (= "" (md/paragraphs [nil ""])))+ (is (= "" (md/bullets [nil ""])))))++(deftest ^:parallel inline-test+ (is (= "`MB_ADMIN_EMAIL`" (md/code "MB_ADMIN_EMAIL")))+ (is (= "**API key**" (md/bold "API key")))+ (is (= "[Where to find this](https://example.com/keys)"+ (md/link "Where to find this" "https://example.com/keys")))+ (testing "headings carry no formatting of their own, so a caller composes what it wants"+ (is (= "## Anthropic" (md/heading 2 "Anthropic")))+ (is (= "### `MB_ADMIN_EMAIL`" (md/heading 3 (md/code "MB_ADMIN_EMAIL"))))))++(deftest ^:parallel thousands-test+ (testing "large numbers are grouped without leaning on the default locale"+ (is (= "1,000,000" (md/thousands 1000000)))+ (is (= "200,000" (md/thousands 200000)))+ (is (= "512" (md/thousands 512)))))++(deftest ^:parallel blockquote-test+ (is (= "> DEPRECATED" (md/blockquote "DEPRECATED")))+ (testing "every line is prefixed, so a quote that runs on stays one quote instead of falling out into body text"+ (is (= "> Only on Pro.\n> Ask your admin." (md/blockquote "Only on Pro.\nAsk your admin.")))))++(deftest ^:parallel document-test+ (testing "a finished page ends in exactly one newline"+ (is (= "One.\n\nTwo.\n" (md/document ["One." nil "Two."]))))+ (testing "a part carrying its own trailing newline is trimmed by [[md/paragraphs]] rather than by its caller"+ (is (= "Intro\n\nBody\n" (md/document ["Intro\n" "Body"])))))++(deftest ^:parallel labeled-block-test+ (testing "a label line over the block it introduces"+ (is (= "Options:\n\n- `-e` - Continue on error"+ (md/labeled-block "Options:" (md/bullets ["`-e` - Continue on error"])))))+ (testing "an empty body leaves the label on its own rather than trailing a blank line"+ (is (= "Options:" (md/labeled-block "Options:" "")))))++(deftest ^:parallel table-test+ (testing "columns are padded to a common width so the raw file stays scannable in a diff"+ (is (= (str "| Model | Model ID |\n"+ "| ---------------- | -------- |\n"+ "| Claude Haiku 4.5 | `haiku` |\n"+ "| GPT | `gpt` |")+ (md/table ["Model" "Model ID"]+ [["Claude Haiku 4.5" "`haiku`"] ["GPT" "`gpt`"]]))))+ (testing "a table with no rows still renders its header"+ (is (= (str "| Model |\n"+ "| ----- |")+ (md/table ["Model"] [])))))diff --git a/test/metabase/metabot/self_test.clj b/test/metabase/metabot/self_test.cljindex 752ca62d276..18e694a2f1c 100644--- a/test/metabase/metabot/self_test.clj+++ b/test/metabase/metabot/self_test.clj@@ -2024,3 +2024,24 @@"a warn with provider and status is still emitted for server-side debugging")(is (not (str/includes? (:message entry) secret))"the secret-bearing body never appears in the warn log")))))++(deftest known-models-normalization-test+ (testing "adapters that key model id to a map are passed through"+ (let [models (self/known-models "anthropic")]+ (is (seq models))+ (is (every? (comp :display-name val) models))))+ (testing "DeepSeek keys model id straight to a display name, and is normalized to the same shape"+ (let [models (self/known-models "deepseek")]+ (is (seq models))+ (is (every? (comp string? :display-name val) models))))+ (testing "the types with no allow-list return nil rather than an empty map"+ (doseq [provider ["azure" "google" "vllm" "metabase"]]+ (is (nil? (self/known-models provider)) provider)))+ (testing "an unregistered provider throws instead of reading as one with no models"+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"Unknown LLM provider"+ (self/known-models "brand-new"))))+ (testing "an entry that is neither a map nor a string throws"+ (is (thrown-with-msg? clojure.lang.ExceptionInfo+ #"Unrecognized supported-models entry"+ (#'self/normalize-known-model "anthropic" "some-model" 42)))))diff --git a/test/metabase/util_test.cljc b/test/metabase/util_test.cljcindex 306d82f3ad1..ff18186e00d 100644--- a/test/metabase/util_test.cljc+++ b/test/metabase/util_test.cljc@@ -15,16 +15,6 @@#?(:clj (set! *warn-on-reflection* true))-(deftest ^:parallel add-period-test- (is (= "This sentence needs a period."- (u/add-period "This sentence needs a period")))- (is (= "This sentence doesn't need a period!"- (u/add-period "This sentence doesn't need a period!")))- (is (= "What about this one?"- (u/add-period "What about this one?")))- (is (= " "- (u/add-period " "))))-(deftest ^:parallel url?-test(are [s expected] (= expected(u/url? s))