The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:
TypeScript analyticsbibusiness-intelligencebusinessintelligenceclojuredashboarddatadata-analysisdata-visualizationdatabasemetabasemysqlpostgrespostgresqlreportingslacksql-editorvisualization
docs - ai usage analytics (#73662)
Jeff Bruemmer178d36f7d0d1
diff --git a/.github/workflows/docs-generate-base.yml b/.github/workflows/docs-generate-base.ymlindex 392808ab9b8..d4ab41f1084 100644--- a/.github/workflows/docs-generate-base.yml+++ b/.github/workflows/docs-generate-base.yml@@ -1,4 +1,3 @@-# .github/workflows/reusable-doc-update.ymlname: Reusable - Update Documentation# This workflow is designed to be called by other workflows.@@ -34,6 +33,11 @@ jobs:- name: Prepare backend environmentuses: ./.github/actions/prepare-backend+ - name: Set up Babashka+ uses: turtlequeue/setup-babashka@2d4df498c7a578b4f3e906283f9af913590bdec7 # v1.7.0+ with:+ babashka-version: 1.12.197+- name: Generate environment variables documentationid: generate_env_docsrun: |@@ -68,6 +72,9 @@ jobs:clojure -M:ee:doc command-documentationcontinue-on-error: false+ - name: Generate usage analytics documentation+ run: ./bin/generate-usage-analytics-docs.bb+- name: Check for changesid: check_changesrun: |@@ -137,6 +144,8 @@ jobs:- \`clojure -M:ee:doc environment-variables-documentation\`- \`clojure -M:ee:doc config-template\`- \`clojure -M:ee:doc api-documentation\`+ - \`clojure -M:ee:doc command-documentation\`+ - \`./bin/generate-usage-analytics-docs.bb\`Please review the changes and merge if they look correct." \--label "Type:Documentation" \diff --git a/bin/generate-usage-analytics-docs.bb b/bin/generate-usage-analytics-docs.bbnew file mode 100755index 00000000000..d8d63a71911--- /dev/null+++ b/bin/generate-usage-analytics-docs.bb@@ -0,0 +1,359 @@+#!/usr/bin/env bb++(ns generate-usage-analytics-docs+ "Generate a markdown reference for the Usage Analytics collection by walking+ the shipped serdes YAML files. Also extracts a few categorical column-value+ lists from their canonical sources (Malli schemas, event derives, SQL views).++ Run from the repo root:++ ./bin/generate-usage-analytics-docs.bb++ Output: docs/usage-and-performance-tools/usage-analytics-reference.md"+ (:require [babashka.fs :as fs]+ [clj-yaml.core :as yaml]+ [clojure.java.io :as io]+ [clojure.string :as str]))++(def ^:private config+ {:yaml-dir "resources/instance_analytics/collections/main/usage_analytics"+ :template "bin/templates/usage-analytics-reference-intro.md"+ :output "docs/usage-and-performance-tools/usage-analytics-reference.md"+ :title "Usage analytics reference"+ :sources {:query-sources "src/metabase/lib/schema/info.cljc"+ :audit-log-events "src/metabase/audit_app/events/audit_log.clj"+ :content-views "resources/migrations/instance_analytics_views/content"+ :view-log-views "resources/migrations/instance_analytics_views/view_log"}})++;; ---------------------------------------------------------------------------+;; Categorical-column rename/exclusion config+;; ---------------------------------------------------------------------------++(def ^:private audit-topic-renames+ "Topic renames applied by the v_audit_log SQL view's CASE statement.+ Keep in sync with resources/migrations/instance_analytics_views/audit_log/v*/mysql-audit_log.sql."+ {"pulse-create" "subscription-create"+ "pulse-delete" "subscription-delete"})++(def ^:private audit-topic-exclusions+ "Topics filtered out of the v_audit_log SQL view's WHERE clause."+ #{"card-read" "card-query" "dashboard-read" "dashboard-query"})++(def ^:private content-card-types+ "Values from report_card.type yielded by the `type AS entity_type` UNION+ arm in the v_content SQL view."+ ["model" "question"])++(def ^:private view-log-entity-types+ "Entity types written to the view_log table by event handlers in+ src/metabase/view_log/events/view_log.clj. Hardcoded because the+ v_view_log SQL view exposes them via `model AS entity_type` (no SQL+ literals to extract). Update when new model types start being+ view-logged."+ ["card" "collection" "dashboard" "table"])++;; ---------------------------------------------------------------------------+;; Shared helpers+;; ---------------------------------------------------------------------------++(defn- non-empty!+ "Returns `coll`, or throws when empty so a moved/renamed source surfaces loudly."+ [label coll]+ (when (empty? coll)+ (throw (ex-info (str "Extracted 0 " label "; source likely moved or changed shape")+ {:extractor label})))+ coll)++(defn- latest-vN-dir+ "Return the directory under `parent` with the highest numeric vN suffix."+ [parent]+ (let [dirs (->> (fs/list-dir parent)+ (filter fs/directory?)+ (filter #(re-matches #"v\d+" (fs/file-name %))))]+ (when (empty? dirs)+ (throw (ex-info (str "No vN directories under " parent) {:parent parent})))+ (->> dirs+ (sort-by #(Long/parseLong (subs (fs/file-name %) 1)))+ last)))++;; ---------------------------------------------------------------------------+;; YAML loading + classification+;; ---------------------------------------------------------------------------++(defn- yaml-file? [p]+ (boolean (re-find #"\.ya?ml$" (str p))))++(defn- read-yaml+ "Parse a YAML file into `{:path ... :doc ...}`."+ [path]+ (try+ {:path (str path)+ :doc (yaml/parse-string (slurp (str path)))}+ (catch Exception e+ (throw (ex-info (str "Failed to parse YAML: " path)+ {:path (str path) :cause (.getMessage e)} e)))))++(defn- serdes-model+ "Read the entity kind from a YAML's serdes/meta block (last entry wins —+ nested entities such as DashboardCards put their parent first)."+ [doc]+ (some-> doc :serdes/meta last :model))++(defn- dashboard? [doc] (= "Dashboard" (serdes-model doc)))+(defn- card? [doc] (= "Card" (serdes-model doc)))+(defn- model? [doc] (and (card? doc) (= "model" (some-> doc :type name))))++;; ---------------------------------------------------------------------------+;; Collectors+;; ---------------------------------------------------------------------------++(defn- top-level-yamls+ "Return `{:path :doc}` maps for every YAML directly in the usage-analytics+ collection (excluding files inside dashboard subdirectories)."+ [yaml-dir]+ (->> (fs/list-dir yaml-dir)+ (filter #(and (fs/regular-file? %) (yaml-file? %)))+ (map read-yaml)+ (sort-by :path)))++(defn- dashboard-card-dir+ "Sibling directory next to a dashboard YAML that holds the Card YAMLs+ referenced by the dashboard's inline DashboardCards."+ [dashboard-path]+ (fs/path (fs/parent dashboard-path)+ (str/replace (fs/file-name dashboard-path) #"\.ya?ml$" "")))++(defn- dashboard-card-names+ "Return card display names from a dashboard's sibling subdir, ordered by filename."+ [dashboard-path]+ (let [dir (dashboard-card-dir dashboard-path)]+ (when (fs/directory? dir)+ (->> (fs/list-dir dir)+ (filter yaml-file?)+ (map read-yaml)+ (sort-by :path)+ (keep (comp :name :doc))))))++(defn- model-columns+ "Display names of every column the model exposes."+ [doc]+ (keep :display_name (:result_metadata doc)))++(defn- dashboard-entries [yamls]+ (for [{:keys [path doc]} yamls+ :when (dashboard? doc)]+ {:name (:name doc)+ :description (:description doc)+ :cards (dashboard-card-names path)}))++(defn- model-entries [yamls]+ (for [{:keys [doc]} yamls+ :when (model? doc)]+ {:name (:name doc)+ :description (:description doc)+ :columns (model-columns doc)}))++;; ---------------------------------------------------------------------------+;; Categorical-column extractors+;; ---------------------------------------------------------------------------++(defn- query-sources+ "Read the [:enum ...] body of `::context` in info.cljc and return its+ keyword names as sorted strings."+ [src]+ (let [content (slurp src)+ m (re-find #"(?s)\(mr/def\s+::context.*?\[:enum(.*?)\]\)" content)]+ (when-not m+ (throw (ex-info (str "Could not find (mr/def ::context [:enum ...]) in " src)+ {:source src})))+ (->> (re-seq #":([a-z][a-z0-9-]*(?:/[a-z][a-z0-9-]*)?)" (second m))+ (map second)+ (remove #(str/includes? % "/"))+ distinct+ sort+ vec+ (non-empty! "query sources"))))++(defn- audit-log-topics+ "Grep `(derive :event/<topic> ::<parent>)` from the audit-log event file,+ then apply the v_audit_log SQL view's rename and exclusion rules so the+ list matches what users see in the analytics model."+ [src]+ (let [content (slurp src)+ topics (->> (re-seq #"\(derive\s+:event/([a-z0-9-]+)\s+::[a-z0-9-]+\)" content)+ (map second)+ distinct)]+ (->> topics+ (remove audit-topic-exclusions)+ (map #(get audit-topic-renames % %))+ distinct+ sort+ vec+ (non-empty! "audit log topics"))))++(defn- content-entity-types+ "Pull `'X' AS entity_type` literals from the latest mysql-content.sql view,+ plus the report_card.type values when the view's CASE-on-type arm is+ present."+ [views-dir]+ (let [sql-file (str (fs/path (latest-vN-dir views-dir) "mysql-content.sql"))+ content (slurp sql-file)+ literals (->> (re-seq #"(?i)'([a-z_]+)'\s+as\s+entity_type" content)+ (map second))+ has-card-arm? (and (re-find #"(?i)type\s+as\s+entity_type" content)+ (re-find #"(?i)report_card" content))]+ (->> (concat literals (when has-card-arm? content-card-types))+ distinct+ sort+ vec+ (non-empty! "content entity types"))))++(defn- assert-view-log-sql-shape!+ "Throws if the v_view_log SQL view stops using `model AS entity_type` —+ the assumption that lets us hardcode [[view-log-entity-types]]."+ [views-dir]+ (let [sql-file (str (fs/path (latest-vN-dir views-dir) "mysql-view_log.sql"))]+ (when-not (re-find #"(?i)model\s+as\s+entity_type" (slurp sql-file))+ (throw (ex-info "v_view_log SQL no longer uses `model AS entity_type`. Re-derive view-log-entity-types manually."+ {:source sql-file})))))++(defn- categorical-sections [{:keys [sources]}]+ [{:name "Activity log topics"+ :description "The Topic column on the [Activity log](#activity-log) model takes one of:"+ :values-fn #(audit-log-topics (:audit-log-events sources))}+ {:name "Query log query sources"+ :description "The Query Source column on the [Query log](#query-log) model takes one of:"+ :values-fn #(query-sources (:query-sources sources))}+ {:name "Content entity types"+ :description "The Entity Type column on the [Content](#content) model takes one of:"+ :values-fn #(content-entity-types (:content-views sources))}+ {:name "View log entity types"+ :description "The Entity Type column on the [View log](#view-log) model takes one of:"+ :values-fn (constantly view-log-entity-types)}])++(defn- categorical-entries [sections]+ (mapv (fn [{:keys [name description values-fn]}]+ {:name name+ :description description+ :values (values-fn)})+ sections))++;; ---------------------------------------------------------------------------+;; Markdown rendering+;; ---------------------------------------------------------------------------++(defn- bullet-list [items]+ (str/join "\n" (map #(str "- " %) items)))++(defn- section-header [level text]+ (str (apply str (repeat level "#")) " " text))++(defn- labelled-list-renderer+ "Returns a fn that renders items as `<label>:\\n\\n<bullets>`, falling back+ to `empty-label` when the seq is empty."+ [label empty-label]+ (fn [items]+ (if (seq items)+ (str label ":\n\n" (bullet-list items))+ empty-label)))++(defn- bare-list+ "Render `items` as a plain bullet list, or `nil` when empty."+ [items]+ (when (seq items)+ (bullet-list items)))++(defn- entity-markdown+ "Render one `###` subsection from `{:name :description :items :render-items}`."+ [{:keys [name description items render-items]}]+ (str/join "\n\n"+ (remove str/blank?+ [(section-header 3 name)+ description+ (render-items items)])))++(defn- entities-markdown+ [{:keys [items-key render-items]} entities]+ (->> entities+ (map (fn [e]+ (entity-markdown {:name (:name e)+ :description (:description e)+ :items (items-key e)+ :render-items render-items})))+ (str/join "\n\n")))++(def ^:private dashboard-section+ {:items-key :cards+ :render-items (labelled-list-renderer "Cards" "_No cards found._")})++(def ^:private model-section+ {:items-key :columns+ :render-items (labelled-list-renderer "Columns" "_No columns found._")})++(def ^:private categorical-section+ {:items-key :values+ :render-items bare-list})++(defn- intro-markdown+ "Substitute `{{title}}` into `template-content`."+ [template-content title]+ (str/replace template-content "{{title}}" title))++(defn- document-markdown+ [{:keys [intro dashboards models categorical]}]+ (str (str/join "\n\n"+ (remove str/blank?+ [(str/trimr intro)+ "## Dashboards"+ (entities-markdown dashboard-section dashboards)+ "## Models"+ (entities-markdown model-section models)+ "## Categorical column values"+ "Some columns in the models above hold one of a fixed set of values."+ (entities-markdown categorical-section categorical)]))+ "\n"))++;; ---------------------------------------------------------------------------+;; Entry point+;; ---------------------------------------------------------------------------++(defn- generate! [{:keys [yaml-dir template output title sources] :as cfg}]+ (when-not (fs/directory? yaml-dir)+ (throw (ex-info (str "YAML directory does not exist: " yaml-dir)+ {:yaml-dir yaml-dir+ :cwd (str (fs/cwd))})))+ (assert-view-log-sql-shape! (:view-log-views sources))+ (let [yamls (top-level-yamls yaml-dir)+ dashboards (dashboard-entries yamls)+ models (model-entries yamls)+ categorical (categorical-entries (categorical-sections cfg))+ intro (intro-markdown (slurp template) title)+ content (document-markdown+ {:intro intro+ :dashboards dashboards+ :models models+ :categorical categorical})]+ (io/make-parents output)+ (spit output content)+ (println (format "Wrote %s" output))+ (println (format " %d dashboards, %d models, %d categorical columns (%s values)"+ (count dashboards)+ (count models)+ (count categorical)+ (str/join ", " (map (comp count :values) categorical))))+ {:dashboards (count dashboards)+ :models (count models)+ :categorical (count categorical)+ :output output}))++(defn -main [& _args]+ (try+ (generate! config)+ (catch Exception e+ (println "Error:" (.getMessage e))+ (when-let [data (ex-data e)]+ (println " data:" (pr-str data)))+ (System/exit 1))))++(when (= *file* (System/getProperty "babashka.file"))+ (-main))diff --git a/bin/templates/usage-analytics-reference-intro.md b/bin/templates/usage-analytics-reference-intro.mdnew file mode 100644index 00000000000..8c032abf82c--- /dev/null+++ b/bin/templates/usage-analytics-reference-intro.md@@ -0,0 +1,15 @@+---+title: {{title}}+summary: "A list of the dashboards and models included in the Usage Analytics collection."+---++# {{title}}++_This page is generated by `bin/generate-usage-analytics-docs.bb`._++Here is a list of all of the dashboards and models included in the Usage Analytics collection, as well as the list of topics and entity types found in the models.++For an overview of access, retention, and custom reports, see [Usage analytics](usage-analytics.md).++For more on AI usage data, see [AI usage auditing](../ai/usage-auditing.md).+diff --git a/docs/README.md b/docs/README.mdindex dc108429604..bb6349d99de 100644--- a/docs/README.md+++ b/docs/README.md@@ -152,6 +152,7 @@ Metabase's reference documentation.- [Metabot](./ai/metabot.md)- [AI settings](./ai/settings.md)- [AI usage controls](./ai/usage-controls.md)+- [AI usage auditing](./ai/usage-auditing.md)- [AI customization](./ai/customization.md)- [AI system prompts](./ai/system-prompts.md)- [Agent API](./ai/agent-api.md)diff --git a/docs/ai/start.md b/docs/ai/start.mdindex 9349f3e9e3a..2591093e3e1 100644--- a/docs/ai/start.md+++ b/docs/ai/start.md@@ -37,3 +37,8 @@ Connect MCP-compatible AI clients to your Metabase.## [Metabot in Slack](./metabot-slack.md)Chat with Metabot directly in Slack — ask questions, get charts, and manage subscriptions without leaving Slack.++## [AI usage auditing](./usage-auditing.md)++See how people are using the AI features in your Metabase.+diff --git a/docs/ai/usage-auditing.md b/docs/ai/usage-auditing.mdnew file mode 100644index 00000000000..f020c101e03--- /dev/null+++ b/docs/ai/usage-auditing.md@@ -0,0 +1,148 @@+---+title: AI usage auditing+summary: "See token and message counts, chat logs, and more."+---++# AI usage auditing++{% include plans-blockquote.html feature="AI usage auditing" %}++_Admin > AI > Usage auditing_++Admins can get an overview of human-robot interactions across Metabase, from high-level stats like total token counts down to visibility into actual conversations. These interactions include the in-product Metabot chat sidebar, Metabot conversations in [Documents](../documents/start.md), [Slack chats](./metabot-slack.md), and [inline SQL editing](./metabot.md#inline-sql-editing).++The usage auditing section includes:++- [Stats](#stats): aggregate charts across all Metabot activity.+- [Conversations](#conversations): a filterable list of every conversation, with a detail view for each.++You can also build your own questions on top of the [Usage Analytics views](#building-custom-reports) that back these reports.++## Stats++_Admin > AI > Usage auditing > Stats_++The Stats page shows aggregate Metabot activity over a date range you choose, defaulting to the previous 30 days.++### Filters++- **Date range**: the time window the charts cover.+- **User**: limit to a single person (or **All users**).+- **Group**: limit to a single [group](../people-and-groups/managing.md) (or **All groups**).+- **Tenant**: limit to a single [tenant](../embedding/tenants.md). Only shows up if tenants are enabled.++### Metrics++Pick what you want to count:++- **Conversations**: one row per distinct Metabot chat, across every surface listed above (chat sidebar, Documents, Slack, inline SQL). Not MCP conversations.+- **Tokens**: total tokens (input and output) consumed by LLM calls.+- **Messages**: every message exchanged, both from people and from Metabot.++For each metric, you'll see the same set of charts:++- **By time**: a time-series chart that buckets by hour or day depending on the date range you filter. Defaults to day.+- **By source**: _where_ in Metabase the request came from. See [Sources and profiles](#sources-and-profiles).+- **By profile**: _which_ Metabot persona answered. See [Sources and profiles](#sources-and-profiles).+- **Users with most ...**, **Groups with most ...**, **IP addresses with most ...**: top-N rankings.+- **Tenants with most ...**: only shown when tenants are enabled.++You can drill through in the **By day**, **Groups with most ...**, **Users with most ...**, or **Tenants with most ...** charts to see the [Conversations list](#conversations) with the corresponding filter applied. The **By source**, **By profile**, and **IP addresses with most ...** charts are display-only.++## Sources and profiles++- **Source** is _where_ the request came from in Metabase.+- **Profile** is _which agent persona_ answered it.++They often line up (a conversation that started in Slack is handled by the Slackbot profile), but they don't have to. A conversation started from the Metabot chat sidebar might be handled by the **Internal**, **NLQ**, or **SQL** profile depending on what the person asked.++The Conversations admin page only shows **Profile**. Source is visible in the **Stats** charts and in the [Usage Analytics models](#building-custom-reports) used to build custom reports.++### Sources++Each conversation is tagged with a source, i.e. where in Metabase the conversation took place. The **By source** chart in [Stats](#stats) groups conversations by a human-readable `source_name`, and the [AI Usage Log](#building-custom-reports) model exposes both `source_name` and a raw `source` ID (e.g. `metabot_agent`, `oss-sql-gen`, `document_generate_content`) for custom reports. Conversations Metabase couldn't classify show up as `(empty)` on the chart.++| Source name | Where it comes from |+| ------------------- | ----------------------------------------------------------------------------------------------------------------------- |+| `Metabot` | The Metabot chat sidebar inside Metabase. |+| `Documents` | Content generation inside [Documents](../documents/start.md). |+| `Suggested Prompts` | Background generation of suggested prompts. |+| `Slackbot` | Conversations that started in [Slack](./metabot-slack.md). |+| `SQL` | [Inline SQL editing](./metabot.md#inline-sql-editing) in the native editor. |+| `Unknown` | A conversation Metabase couldn't classify (distinct from no-source conversations, which appear as `(empty)` on charts). |++### Profiles++A profile is the configuration Metabot uses for a conversation: which prompt, which tools, and what it's allowed to do. The Conversations admin page, the **By profile** chart in [Stats](#stats), and the **Metabot Conversations** model (see [Building custom reports](#building-custom-reports)) all show the human-readable profile name. The [AI Usage Log](#building-custom-reports) model exposes the raw `profile_id` instead (e.g. `internal`, `transforms_codegen`, `embedding_next`).++| Profile | What it does |+| -------------------- | ------------------------------------------------------------------------------------------------------------------------- |+| `Internal` | The default Metabot in the chat sidebar. Can build query-builder questions _and_ write SQL. |+| `NLQ` | Natural-language querying only. Always returns a [query builder](../questions/query-builder/editor.md) result, never SQL. |+| `SQL` | SQL writing only. Used by [inline SQL editing](./metabot.md#inline-sql-editing) and similar surfaces. |+| `Slackbot` | The persona behind [Metabot in Slack](./metabot-slack.md). |+| `Embedding` | The Metabot persona used inside [embedded Metabase](../embedding/start.md). |+| `Transforms codegen` | Generates [transform](../data-studio/transforms/transforms-overview.md), SQL, or Python. |+| `Documents` | Generates content inside [Documents](../documents/start.md). |++## Conversations++_Admin > AI > Usage auditing > Conversations_++The Conversations page lists every Metabot conversation Metabase has on file, newest first.++### Filters++- **Date range**, **User**, **Group**, **Tenant**: same as the [Stats filters](#filters).++### Columns++Each row shows:++- **User**: who started the conversation.+- **Profile**: which Metabot persona answered.+- **Date**: when the conversation started.+- **Messages**: total messages, including both sides.+- **Tokens**: total LLM tokens spent.+- **Queries**: how many queries (SQL or query-builder) Metabot generated during the conversation.+- **Searches**: how many search-tool calls Metabot made.+- **IP**: the IP address the request came from.++You can sort by Date, Messages, or Tokens. Click any row to open the [conversation detail](#conversation-detail).++## Conversation detail++The detail view is a full audit of a single conversation. It includes:++- **Header**: start date, the person who chatted with Metabot, the profile Metabot used, the person's groups (including whether they're an admin), and tenant if applicable. From the **...** menu next to the person's name you can jump to all of their conversations, or to their account details.+- **Stat tiles**: Messages, Total tokens, Queries run, Searches.+- **Feedback** (if any): thumbs-up or thumbs-down and comments. The agent response that triggered the feedback is shown alongside.+- **Conversation transcript**: the full message-by-message exchange. Tool calls (search calls, query construction, etc.) are inlined. You can click "View" to open a modal with the info.+- **Queries generated**: every SQL or [query builder](../questions/query-builder/editor.md) (MBQL) query Metabot wrote during the conversation, with the referenced tables listed underneath. Hit **Visit** on a query to open the item in a new tab and run it yourself. Transform code-gen queries are shown read-only and can't be re-run from here.++### The `/inspect` shortcut++If you're an admin chatting with Metabot, type `/inspect` in the chat to jump straight from the current conversation to its detail page in Usage auditing.++## Building custom reports++Three [Usage Analytics](../usage-and-performance-tools/usage-analytics.md) models back the Usage auditing pages.++- [AI Usage Log](../usage-and-performance-tools/usage-analytics-reference.md#ai-usage-log): one row per LLM call.+- [Metabot Conversations](../usage-and-performance-tools/usage-analytics-reference.md#metabot-conversations): one row per conversation.+- [Metabot Messages](../usage-and-performance-tools/usage-analytics-reference.md#metabot-messages): one row per message.++Save your custom questions in the [Custom reports](../usage-and-performance-tools/usage-analytics.md#custom-reports-collection) sub-collection so the reports inherit the right permissions.++## What isn't tracked++[MCP](./mcp.md) activity isn't included in Usage auditing. MCP requests don't go through Metabot's conversation pipeline, so they don't generate conversations or token rows.++## Further reading++- [AI usage controls](./usage-controls.md)+- [AI settings](./settings.md)+- [Metabot](./metabot.md)+- [Metabot in Slack](./metabot-slack.md)+- [Usage analytics](../usage-and-performance-tools/usage-analytics.md)+- [Permissions overview](../permissions/start.md)diff --git a/docs/configuring-metabase/caching.md b/docs/configuring-metabase/caching.mdindex 53b67bf8006..1f42059a6d0 100644--- a/docs/configuring-metabase/caching.md+++ b/docs/configuring-metabase/caching.md@@ -97,7 +97,7 @@ For example, let's say you have a dashboard with a category filter (default valu### Automatic caching logs-To view the queries Metabase ran to automatically refresh the cache, check out the [query log](../usage-and-performance-tools/usage-analytics.md#query-log-model) and filter the `Query source` by `cache-refresh`.+To view the queries Metabase ran to automatically refresh the cache, check out the [query log](../usage-and-performance-tools/usage-analytics-reference.md#query-log) and filter the `Query source` by `cache-refresh`.## Set caching policies for dashboards, questions, and databasesdiff --git a/docs/questions/alerts.md b/docs/questions/alerts.mdindex 20f97bd2dfd..e7478da5e50 100644--- a/docs/questions/alerts.md+++ b/docs/questions/alerts.md@@ -142,7 +142,7 @@ Alerts will continue to work _even if the person who set up the alert no longer{% include plans-blockquote.html feature="Usage analytics" %}-Admins can view a list of all alerts and dashboard subscriptions that people have set up in your Metabase in the **Usage analytics** collection. See [Usage analytics](../usage-and-performance-tools/usage-analytics.md#alerts-model).+Admins can view a list of all alerts and dashboard subscriptions that people have set up in your Metabase in the **Usage analytics** collection. See the [Alerts model](../usage-and-performance-tools/usage-analytics-reference.md#alerts) in the Usage analytics reference.## How permissions work with alertsdiff --git a/docs/usage-and-performance-tools/usage-analytics-reference.md b/docs/usage-and-performance-tools/usage-analytics-reference.mdnew file mode 100644index 00000000000..992b6021a4a--- /dev/null+++ b/docs/usage-and-performance-tools/usage-analytics-reference.md@@ -0,0 +1,702 @@+---+title: Usage analytics reference+summary: "A list of the dashboards and models included in the Usage Analytics collection."+---++# Usage analytics reference++_This page is generated by `bin/generate-usage-analytics-docs.bb`._++Here is a list of all of the dashboards and models included in the Usage Analytics collection, as well as the list of topics and entity types found in the models.++For an overview of access, retention, and custom reports, see [Usage analytics](usage-analytics.md).++For more on AI usage data, see [AI usage auditing](../ai/usage-auditing.md).++## Dashboards++### Content with cobwebs++Dashboards and questions that you could consider archiving.++Cards:++- Dashboards without recent views+- Questions that don't belong to a dashboard+- Questions without recent views++### Dashboard overview++Information on the performance, views, activity, and more for a particular dashboard.++Cards:++- Dashboard metadata+- Dashboard views per month+- Most active people on this dashboard+- Questions in this dashboard+- Recent activity on dashboard+- Subscriptions on this dashboard++### Dashboard subscriptions and alerts++All active notifications++Cards:++- Active dashboard subscriptions+- Alerts++### Embedding Usage++General information about embedding usage, including views across embedded dashboards and questions++Cards:++- Active tenants+- Authenticated views+- Guest views+- Most viewed authenticated embeds+- Most viewed guest embeds+- Most viewed public links+- Public link views+- Tenants with most views+- Unique users+- Views by auth method+- Views by embedding client+- Views by embedding host+- Views by user agent++### Metabase metrics++General information about people viewing and creating dashboards, questions, subscriptions, and alerts.++Cards:++- Active users last week+- Alerts and subscriptions created last week+- Dashboards created last week+- Most active creators+- Most active viewers+- Most viewed cards+- Most viewed dashboards+- Question views last week+- Question views per week+- Questions created last week+- Weekly active users++### Most viewed content++Most viewed dashboards, questions, and tables in your Metabase.++Cards:++- Most viewed dashboards+- Most viewed models+- Most viewed questions+- Most viewed tables++### Performance overview++Question, dashboard and database performance.++Cards:++- 50th and 90th percentile of query running time (seconds)+- 90th percentile of query run time (seconds)+- Dashboards consuming most resources+- Dashboards with more questions in the same tab+- Questions consuming most resources+- Slowest dashboards+- Slowest questions+- Total number of queries+- Total time looking for field values per day+- Total time spent on sync per day+- Users consuming most resources+- Weekly cache hit rate++### Person overview++See what someone's been up to in your Metabase.++Cards:++- Active alerts+- Last downloads+- Last queries+- Last viewed dashboards+- Last viewed models+- Last viewed questions+- Last viewed tables+- Member of+- Most viewed dashboards+- Most viewed models+- Most viewed questions+- Most viewed tables+- Person detail+- Question views per month+- Questions created per month+- Recent activity++### Question overview++Views, performance, activity, and other data for a particular question.++Cards:++- Alerts on this question+- Dashboards with this question+- Last activity on question+- Last downloads+- Most active people on this question+- Question metadata+- Question views per month++## Models++### Activity log++Things that happened in your Metabase.++Columns:++- ID+- Topic+- Timestamp+- End Timestamp+- User ID+- Entity Type+- Entity ID+- Entity Qualified ID+- Details++### AI Usage Log++One row per LLM round-trip with per-call tokens, source, model, profile, and more.++Columns:++- Usage Log ID+- Created At+- Source+- Source Name+- Model+- Profile ID+- Prompt Tokens+- Completion Tokens+- Total Tokens+- Conversation ID+- User ID+- User Qualified ID+- User Display Name+- Group Name+- IP Address+- Tenant ID+- Request ID++### Alerts++All alerts, both active and archived.++Columns:++- Entity ID+- Entity Qualified ID+- Created At+- Updated At+- Creator ID+- Card ID+- Card Qualified ID+- Alert Condition+- Schedule Type+- Schedule Day+- Schedule Hour+- Archived+- Recipient Type+- Recipients+- Recipient External++### Content++All Metabase content, including questions, models, dashboards, events, and collections.++Columns:++- Entity ID+- Entity Qualified ID+- Entity Type+- Created At+- Updated At+- Creator ID+- Name+- Description+- Collection ID+- Made Public By User+- Is Embedding Enabled+- Is Verified+- Archived+- Action Type+- Action Model ID+- Collection Is Official+- Collection Is Personal+- Question Viz Type+- Question Database ID+- Question Is Native+- Event Timestamp++### Dashboard cards++Records of all dashboard cards (including text cards).++Columns:++- Entity ID+- Entity Qualified ID+- Dashboard Qualified ID+- Dashboardtab ID+- Card Qualified ID+- Created At+- Updated At+- Visualization Settings+- Parameter Mappings++### Dashboard subscriptions++Which subscriptions are active, who created them, who's subscribed to them, when they're sent, and more.++Columns:++- Entity ID+- Entity Qualified ID+- Created At+- Updated At+- Creator ID+- Archived+- Dashboard Qualified ID+- Schedule Type+- Schedule Day+- Schedule Hour+- Recipient Type+- Recipients+- Recipient External+- Parameters++### Databases++Information about your connected data sources.++Columns:++- Entity ID+- Entity Qualified ID+- Created At+- Updated At+- Name+- Description+- Database Type+- Metadata Sync Schedule+- Cache Field Values Schedule+- Timezone+- Is On Demand+- Auto Run Queries+- Cache Ttl+- Creator ID+- Db Version++### Fields++All fields from all connected data sources.++Columns:++- Entity ID+- Entity Qualified ID+- Created At+- Updated At+- Name+- Display Name+- Description+- Base Type+- Visibility Type+- Fk Target Field ID+- Has Field Values+- Active+- Table ID++### Group Members++Metabase group membership++Columns:++- User ID+- Group ID+- Group Name++### Metabot Conversations++One row per Metabot conversation, with message counts, token totals, owning user/group, source, and tenant.++Columns:++- Conversation ID+- Created At+- User ID+- Summary+- User Display Name+- Message Count+- User Message Count+- Assistant Message Count+- Total Tokens+- Prompt Tokens+- Completion Tokens+- Last Message At+- Profile ID+- Profile Name+- Group Name+- Source+- Source Name+- IP Address+- Tenant ID+- Tenant Name+- Model++### Metabot Messages++One row per non-deleted Metabot message, with role, profile, tokens, user, and the owning conversation.++Columns:++- Message ID+- Conversation ID+- Created At+- Role+- Profile ID+- Total Tokens+- User ID+- Slack Msg ID+- Channel ID++### People++Everyone who's ever had an account in your Metabase, including both active and deactivated accounts.++Columns:++- User ID+- Entity Qualified ID+- Type+- First Name+- Last Name+- Full Name+- Date Joined+- Last Login+- Updated At+- Is Admin+- Is Active+- Sso Source+- Locale+- Tenant ID++### Query log++Information about all queries Metabase ran across all dashboards++Columns:++- Entity ID+- Started At+- Running Time Seconds+- Result Rows+- Is Native+- Query Source+- Error+- User ID+- Card ID+- Card Qualified ID+- Dashboard ID+- Dashboard Qualified ID+- Pulse ID+- Database ID+- Database Qualified ID+- Cache Hit+- Action ID+- Action Qualified ID+- Query+- Embedding Client+- Is Preview+- Embedding SDK Package Version+- Metabase Version+- Auth Method+- Is Sandboxed+- Is Impersonated+- Is Db Routed+- Parameters+- Tenant ID+- Embedding Hostname+- Embedding Path+- User Agent+- Sanitized User Agent+- IP Address++### System tasks++Describes the last 14 days of Metabase internal processes tasks++Columns:++- ID+- Task+- Status+- Database Qualified ID+- Started At+- Ended At+- Duration Seconds+- Details++### Tables++List of all tables across all connected data sources.++Columns:++- Entity ID+- Entity Qualified ID+- Created At+- Updated At+- Name+- Display Name+- Description+- Active+- Database ID+- Schema+- Is Upload+- Entity Type+- Visibility Type+- Estimated Row Count+- View Count+- Owner Email+- Owner User ID+- Database ID++### Task Runs++Tracks job executions++Columns:++- ID+- Run Type+- Entity Type+- Entity ID+- Entity Qualified ID+- Started At+- Ended At+- Duration Seconds+- Status+- Process UUID+- Updated At++### Tenants++All tenants in this Metabase instance++Columns:++- Tenant ID+- Entity Qualified ID+- Name+- Slug+- Is Active+- Attributes+- Created At+- Updated At+- Tenant Collection ID+- Tenant Collection Qualified ID++### View log++Each row describes a question, model, table or dashboard view++Columns:++- ID+- Timestamp+- User ID+- Entity Type+- Entity ID+- Entity Qualified ID+- Embedding Client+- Auth Method+- Is Preview+- Metabase Version+- Embedding SDK Package Version+- Tenant ID+- Embedding Hostname+- Embedding Path+- User Agent+- Sanitized User Agent+- IP Address+- Context++## Categorical column values++Some columns in the models above hold one of a fixed set of values.++### Activity log topics++The Topic column on the [Activity log](#activity-log) model takes one of:++- action-v2-execute+- alert-create+- alert-delete+- alert-send+- alert-unsubscribe+- alert-update+- api-key-create+- api-key-delete+- api-key-regenerate+- api-key-update+- cache-config-update+- card-create+- card-delete+- card-public-link-created+- card-public-link-deleted+- card-update+- channel-create+- channel-update+- cloud-add-on-purchase+- comment-create+- comment-delete+- comment-update+- create-permission-failure+- dashboard-add-cards+- dashboard-create+- dashboard-delete+- dashboard-public-link-created+- dashboard-public-link-deleted+- dashboard-remove-cards+- database-create+- database-delete+- database-discard-field-values+- database-manual-scan+- database-manual-sync+- database-update+- document-create+- document-delete+- document-update+- glossary-create+- glossary-delete+- glossary-update+- group-create+- group-delete+- group-membership-create+- group-membership-delete+- group-membership-update+- group-update+- install+- measure-create+- measure-delete+- measure-update+- notification-create+- notification-unsubscribe+- notification-unsubscribe-ex+- notification-unsubscribe-undo-ex+- notification-update+- password-reset-initiated+- password-reset-successful+- remote-sync-create-branch+- remote-sync-export+- remote-sync-import+- remote-sync-settings-update+- remote-sync-stash+- security-advisory-acknowledge+- security-advisory-match+- segment-create+- segment-delete+- segment-update+- setting-update+- subscription-create+- subscription-delete+- subscription-send+- subscription-unsubscribe+- subscription-unsubscribe-undo+- subscription-update+- table-manual-scan+- table-manual-sync+- table-publish+- table-unpublish+- tenant-create+- tenant-update+- transform-create+- transform-delete+- transform-inspect-discover+- transform-inspect-lens+- transform-run-start+- update-permission-failure+- update-transform+- upload-append+- upload-create+- upload-replace+- user-deactivated+- user-invited+- user-joined+- user-reactivated+- user-update+- write-permission-failure++### Query log query sources++The Query Source column on the [Query log](#query-log) model takes one of:++- action+- ad-hoc+- agent+- cache-refresh+- collection+- csv-download+- dashboard+- dashboard-subscription+- embedded-csv-download+- embedded-dashboard+- embedded-json-download+- embedded-question+- embedded-xlsx-download+- json-download+- map-tiles+- public-csv-download+- public-dashboard+- public-json-download+- public-question+- public-xlsx-download+- pulse+- question+- slackbot+- table-grid+- table-rows-sample+- transform-inspector+- xlsx-download++### Content entity types++The Entity Type column on the [Content](#content) model takes one of:++- action+- collection+- dashboard+- document+- event+- glossary+- model+- question+- transform++### View log entity types++The Entity Type column on the [View log](#view-log) model takes one of:++- card+- collection+- dashboard+- tablediff --git a/docs/usage-and-performance-tools/usage-analytics.md b/docs/usage-and-performance-tools/usage-analytics.mdindex 68dc6c8e625..d71728b59ce 100644--- a/docs/usage-and-performance-tools/usage-analytics.md+++ b/docs/usage-and-performance-tools/usage-analytics.md@@ -44,9 +44,9 @@ Metabase will take you to the relevant usage dashboard and plug in the item's ID## How long Metabase keeps usage data-By default, Metabase will keep the data about [activity](#activity-log-model), [views](#view-log-model), and [query execution](#query-log-model) for **720 days**. Twice a day, Metabase will delete rows older than this threshold. You can change this limit by adjusting the environment variable [`MB_AUDIT_MAX_RETENTION_DAYS`](../configuring-metabase/environment-variables.md#mb_audit_max_retention_days).+By default, Metabase will keep the data about [activity](./usage-analytics-reference.md#activity-log), [views](./usage-analytics-reference.md#view-log), and [query execution](./usage-analytics-reference.md#query-log) for **720 days**. Twice a day, Metabase will delete rows older than this threshold. You can change this limit by adjusting the environment variable [`MB_AUDIT_MAX_RETENTION_DAYS`](../configuring-metabase/environment-variables.md#mb_audit_max_retention_days).-If you're on the Metabase Open Source Edition, or on the [Metabase Cloud Starter plan](https://www.metabase.com/pricing/), Metabase doesn't collect [Activity](#activity-log-model) and [View](#view-log-model) data. If you upgrade to a Pro or Enterprise plan, either self-hosted or Cloud, you'll only see View and Activity data in Usage Analytics _starting from the time when you upgraded_.+If you're on the Metabase Open Source Edition, or on the [Metabase Cloud Starter plan](https://www.metabase.com/pricing/), Metabase doesn't collect [Activity](./usage-analytics-reference.md#activity-log) and [View](./usage-analytics-reference.md#view-log) data. If you upgrade to a Pro or Enterprise plan, either self-hosted or Cloud, you'll only see View and Activity data in Usage Analytics _starting from the time when you upgraded_.## Creating custom reports@@ -58,358 +58,14 @@ While you _can_ save custom questions, models, and dashboards wherever you likeThere is one thing to know about the Custom reports collection: its metadata resets whenever Metabase restarts. While you are able to temporarily rename the Custom reports collection, or give it a description or an Official badge, Metabase will drop this collection's metadata when it restarts. But rest assured that Metabase will preserve any questions, models, events, or dashboards that you add to the Custom reports collection.-## Dashboards+## Dashboards and models-The Usage analytics collection includes a set of read-only dashboards.+The Usage analytics collection ships a set of read-only dashboards and models that you can browse from the **collections** sidebar.-### Metabase metrics dashboard+For dashboards and models specifically about Metabot and LLM usage, see [AI usage auditing](../ai/usage-auditing.md).-General information about people viewing and creating dashboards, questions, subscriptions, and alerts. Cards include:+See also the [Usage analytics reference](./usage-analytics-reference.md).-- Active users last week-- Question views last week-- Questions created last week-- Dashboards created last week-- Alerts and subscriptions created last week-- Weekly active users-- Question views per week-- Most active users-- Most active creators-- Most viewed dashboards-- Most viewed cards+### Performance overview on MySQL or MariaDB-### Most viewed content dashboard--View the most relevant content in your Metabase. Cards include:--- Most viewed dashboards-- Most viewed questions-- Most viewed tables--### Person overview dashboard--See what someone's been up to in your Metabase. Cards include:--- Member of-- Active alerts-- Questions created per month-- Question views per month-- Most viewed dashboards-- Most viewed questions-- Last viewed dashboards-- Last viewed questions-- Last viewed tables-- Recent activity-- Last queries--### Dashboard overview dashboard--Information about dashboards, questions, models, and tables. Cards include:--- Dashboard metadata-- Dashboard views per month-- Question performance-- Most active people on this dashboard-- Questions in this dashboard-- Most active people on this dashboard-- Questions in this dashboard-- Recent activity on dashboard-- Subscriptions on this dashboard--### Question overview dashboard--Views, performance, activity, and other data for a particular question. Cards include:--- Question metadata-- Question views per month-- Question performance-- Most active people on this question-- Dashboards with this question-- Last activity on this question-- Alerts on this question--### Performance overview dashboard--Question, dashboard and database performance. Cards include:--- Slowest dashboards-- Dashboards consuming most resources-- Slowest questions-- Questions consuming the most resources-- Dashboards with more questions in the same tab-- Users consuming the most resources--> If you're using MySQL or MariaDB as your application database, the Performance overview dashboard won't display results for the cards displaying the 50th and 90th percentile query running times, because MySQL and MariaDB don't support the [Percentile aggregation](../questions/query-builder/expressions-list.md#percentile). We recommend using PostgreSQL as your application database.--### Content with cobwebs dashboard--Dashboards and questions that you could consider archiving. Cards include:--- Dashboards without recent reviews-- Questions without recent reviews-- Questions that don't belong to a dashboard--## Models--The Usage analytics collection includes a bunch of useful models based on Metabase's application database.--## Activity log model--Each row of this model describes one event of a particular topic. Fields include:--- ID-- Topic-- Timestamp-- End Timestamp-- User ID-- Model-- Model ID-- Details--The topics include:--- alert-create-- alert-delete-- card-create-- card-delete-- card-update-- dashboard-add-cards-- dashboard-create-- dashboard-delete-- dashboard-remove-cards-- install-- metric-create-- metric-delete-- metric-update-- segment-create-- segment-delete-- segment-update-- setting-update-- subscription-create-- subscription-delete-- user-joined--## View log model--Tracks views cards (which includes models), dashboards, and tables. Fields include:--- ID-- Timestamp-- User ID-- Entity Type (card, dashboard, or table)-- Entity ID-- Entity Qualified ID--## Query log model--Information about all queries Metabase ran across all dashboards. Fields include:--- Entity ID-- Started At-- Running Time Seconds-- Result Rows-- Is Native-- Query Source-- Error-- User ID-- Card ID-- Card Qualified ID-- Dashboard ID-- Dashboard Qualified ID-- Pulse ID-- Database ID-- Database Qualified ID-- Cache Hit-- Action ID--Query sources include:--- action-- ad-hoc-- collection-- csv-download-- dashboard-- embedded-dashboard-- embedded-csv-download-- embedded-json-download-- embedded-question-- embedded-xlsx-download-- json-download-- map-tiles-- metabot (experimental)-- public-dashboard-- public-question-- pulse (which includes dashboard subscriptions and alerts)-- question-- xlsx-download--## Alerts model--All alerts, both active and archived.--- Entity ID-- Entity Qualified ID-- Created At-- Updated At-- Creator ID-- Card ID-- Card Qualified ID-- Alert Condition-- Schedule Type-- Schedule Day-- Schedule Hour-- Archived-- Recipient Type-- Recipients-- Recipient External--### Content model--Questions, dashboards, models, events, and collections.--- Entity ID-- Entity Qualified ID-- Entity Type-- Created At-- Updated At-- Creator ID-- Name-- Description-- Collection ID-- Made Public By User-- Is Embedding Enabled-- Archived-- Action Type-- Action Model ID-- Collection Is Official-- Collection Is Personal-- Question Viz Type-- Question Database ID-- Question Is Native-- Event Timestamp--Entity types include:--- action-- collection-- dashboard-- event-- model-- question--## People model--Everyone in your Metabase, including deactivated accounts. Fields include:--- User ID-- First Name-- Last Name-- Full Name-- Date Joined-- Last Login-- Updated At-- Is Admin-- Is Active-- SSO Source-- Locale--## Dashboard subscriptions model--Which subscriptions are active, who created them, who's subscribed to them, when they're sent, and more.--- Entity ID-- Entity Qualified ID-- Created At-- Updated At-- Creator ID-- Archived-- Dashboard Qualified ID-- Schedule Type-- Schedule Day-- Schedule Hour-- Recipient Type-- Recipients-- Recipient External-- Parameters--## Dashboard cards model--Each row is a dashboard card: either a question card or a text card. Fields include:--- ID-- Dashboard ID-- Dashboardtab ID-- Question ID-- Created At-- Updated At-- Size X-- Size Y-- Visualization Settings-- Parameter Mappings--## Databases model--Information about your connected data sources. Fields include:--- Entity ID-- Entity Qualified ID-- Created At-- Updated At-- Name-- Description-- Database Type-- Metadata Sync Schedule-- Cache Field Values Schedule-- Timezone-- Is On Demand-- Auto Run Queries-- Cache Ttl-- Creator ID-- Db Version--## Tables model--List of all tables across all connected data sources. Fields include:--- Entity ID-- Entity Qualified ID-- Created At-- Updated At-- Name-- Display Name-- Description-- Active-- Database ID-- Schema-- Is Upload--## Fields model--All fields from all connected data sources. Fields include:--- Entity ID-- Entity Qualified ID-- Created At-- Updated At-- Name-- Display Name-- Description-- Base Type-- Visibility Type-- Fk Target Field ID-- Has Field Values-- Active-- Table ID--## System tasks model--Describes the last 14 days of Metabase internal processes tasks.--- ID-- Task-- Database Qualified ID-- Started At-- Ended At-- Duration Seconds-- Details+If you're using MySQL or MariaDB as your application database, the Performance overview dashboard won't display results for the cards showing 50th and 90th percentile query running times, because MySQL and MariaDB don't support the [Percentile aggregation](../questions/query-builder/expressions-list.md#percentile). We recommend using PostgreSQL as your application database.