Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

docs: transform addons (#74379)

Alex Yaroshfbde6064769a
diff --git a/docs/data-modeling/model-persistence.md b/docs/data-modeling/model-persistence.md
index 8060ec7147c..9f3ce2f35f2 100644
--- a/docs/data-modeling/model-persistence.md
+++ b/docs/data-modeling/model-persistence.md
@@ -4,7 +4,7 @@ title: Model persistence
# Model persistence
-> Prefer [Transforms](../data-studio/transforms/transforms-overview.md) instead of model persistence. Model persistence remains supported for now, but it will be deprecated in future versions of Metabase. You can [convert models to transforms in bulk](../data-studio/transforms/transforms-overview.md#convert-models-to-transforms).
+> Prefer [Transforms](../data-studio/transforms/transforms-overview.md) instead of model persistence. Model persistence remains supported for now, but it will be deprecated in future versions of Metabase. You can [convert models to transforms in bulk](../data-studio/transforms/query-transforms.md#convert-models-to-transforms).
Metabase can persist the results of your models so that your models (and the questions based on those models) load faster.
diff --git a/docs/data-modeling/models.md b/docs/data-modeling/models.md
index ccbdf421ecd..e658380f51a 100644
--- a/docs/data-modeling/models.md
+++ b/docs/data-modeling/models.md
@@ -6,7 +6,7 @@ redirect_from:
# Models
-> Consider using [Transforms](../data-studio/transforms/transforms-overview.md) instead of models. You can [convert models to transforms in bulk](../data-studio/transforms/transforms-overview.md#convert-models-to-transforms).
+> Consider using [Transforms](../data-studio/transforms/transforms-overview.md) instead of models. You can [convert models to transforms in bulk](../data-studio/transforms/query-transforms.md#convert-models-to-transforms).
Models curate data from another table or tables from the same database to anticipate the kinds of questions people will ask of the data. You can think of them as derived tables, or a special kind of saved question meant to be used as the starting point for new questions. You can base a model on a SQL or query builder question, which means you can include custom, calculated columns in your model.
@@ -224,7 +224,7 @@ See [Model persistence](./model-persistence.md)
If you're an admin, you can convert existing models to transforms one at a time. Conversion creates a transform from the model's query, runs it to produce the output table, and then updates all questions and dashboards that used the model to use the transform's table instead. The model itself becomes a saved question.
-See [Convert existing models to transforms](../data-studio/transforms/transforms-overview.md#convert-models-to-transforms).
+See [Convert existing models to transforms](../data-studio/transforms/query-transforms.md#convert-models-to-transforms).
## Further reading
diff --git a/docs/data-studio/images/disable-all-jobs.png b/docs/data-studio/images/disable-all-jobs.png
new file mode 100644
index 00000000000..61bc13e838c
Binary files /dev/null and b/docs/data-studio/images/disable-all-jobs.png differ
diff --git a/docs/data-studio/images/jobs-schedule.png b/docs/data-studio/images/jobs-schedule.png
new file mode 100644
index 00000000000..57329bd6cac
Binary files /dev/null and b/docs/data-studio/images/jobs-schedule.png differ
diff --git a/docs/data-studio/transforms/addons.md b/docs/data-studio/transforms/addons.md
new file mode 100644
index 00000000000..dc51ee53a46
--- /dev/null
+++ b/docs/data-studio/transforms/addons.md
@@ -0,0 +1,104 @@
+---
+title: Transform add-ons
+summary: Metabase transforms come in two flavors - basic transforms for basic query-based functionality, and advanced transforms for Python workflows, transform inspector, and other functionality.
+---
+
+# Transform add-ons
+
+At a glance:
+
+[**Basic transforms**](#basic-transforms):
+
+- Run [query-based transforms](query-transforms.md)
+- Schedule transform [jobs](jobs-and-runs.md).
+- (Pro/Enterprise only) Configure [permissions for transforms](transforms-overview.md#permissions-for-transforms).
+
+[**Advanced transforms**](#advanced-transforms):
+
+- Run [query-based](query-transforms.md) and [Python transforms](python-transforms.md);
+- Schedule transform [jobs](jobs-and-runs.md).
+- (Pro/Enterprise only) Configure [permissions for transforms](transforms-overview.md#permissions-for-transforms).
+- [Writable connection](../../databases/writable-connection.md): separate database connection used for write operations.
+- [Transform inspector](transform-inspector.md).
+
+Availability and pricing depends on your plan and hosting method.
+
+## Basic transforms
+
+With basic transforms, you can:
+
+- Write and run [query-based transforms](query-transforms.md) (but not Python transforms).
+- [Schedule and run jobs](jobs-and-runs.md).
+- (Pro/Enterprise only) Configure [permissions for transforms](transforms-overview.md#permissions-for-transforms).
+
+### Enable basic transforms
+
+- **Self-hosted Metabases**: Basic transform functionality is included on self-hosted Metabases by default. Just log into your Metabase, [enable transforms](../transforms/transforms-overview.md#enable-transforms), and you're good to go.
+
+- **Metabase Cloud**: Basic transform functionality on Metabase Cloud - Starter, Pro, or Enterprise - comes with an additional small fee per successful transform run, see [Pricing](https://www.metabase.com/pricing).
+
+ Only people logged in with an email of a [Metabase Store admins](../../cloud/accounts-and-billing.md#add-people-to-manage-your-metabase-store-account) (not just Metabase _instance_ admins) can enable basic transforms. To enable Basic transforms on Metabase Cloud, see [Enable transforms](./transforms-overview.md#enable-transforms).
+
+### Cancel basic transforms
+
+Once basic transforms are enabled on your Metabase Cloud instance, they can't be disabled.
+
+## Advanced transforms
+
+Advanced transforms include:
+
+- Everything in [basic transforms](#basic-transforms).
+- [Python transforms](python-transforms.md) for more flexible data processing.
+- [Writable connection](../../databases/writable-connection.md): separate database connection used for write operations.
+- [Transform inspector](transform-inspector.md).
+
+You can buy the Advanced transforms add-on for:
+
+- Any Metabase Cloud instance (Starter, Pro, Enterprise)
+- Self-hosted Pro or Enterprise instances.
+
+ Currently, you can't use Advanced transforms functionality on Open Source self-hosted plans.
+
+The Advanced transforms add-on comes with an additional charge per successful transform run (compare to [Basic transforms](#basic-transforms)). See [Pricing](https://www.metabase.com/pricing).
+
+### Enable Advanced transforms
+
+To enable Advanced transforms functionality, you need to have [Basic transforms](#basic-transforms) already, see [Enable basic transforms](#enable-basic-transforms).
+
+There are two ways to enable Advanced transforms:
+
+- **From your Metabase instance**: you can navigate to a feature requiring advanced transforms (like Python transforms or transform inspector), and follow the prompts to upgrade.
+
+ To enable Advanced transforms from your Metabase instance, you need to be logged into the instance with the same email as a [Metabase Store admin](../../cloud/accounts-and-billing.md#add-people-to-manage-your-metabase-store-account), because Advanced transform incur an additional charge.
+
+- **From [Metabase Store](https://store.metabase.com)**:
+
+ 1. Log into [Metabase Store](https://store.metabase.com) (your Metabase Store account might be different from your Metabase instance account).
+ 2. Click on **Manage plan** next to the instance where you'd like to add Advanced transforms.
+ 3. Under **Manage Add-ons**, find **Advanced transforms** and click **Upgrade**.
+
+Once you upgrade to Advanced transforms:
+
+- You get access to Advanced transforms features/
+- All your existing query-based transforms will be charged at Advanced transforms rate.
+
+### Cancel Advanced transforms
+
+You can downgrade from Advanced transforms to [Basic transforms](#basic-transforms).
+
+1. Log into [Metabase Store](https://store.metabase.com) (your Metabase Store account might be different from your Metabase instance account).
+2. Click on **Manage plan** next to the instance with Advanced transforms.
+3. Under **Manage Add-ons**, find **Advanced transforms**, click **three dos** and select **Downgrade to basic**.
+
+Once you downgrade from advanced transforms:
+
+- Your Python transforms will no longer run and will be removed.
+- Transforms and other write features will stop using the writable connection (if you had one configured.)
+
+## How billing works for transforms
+
+Unless you're on an Enterprise plan, transforms - either basic (on Metabase Cloud) or advanced - are billed based on the number of successful runs. See [Jobs and runs](jobs-and-runs.md) for more on transform runs, and [Pricing](https://www.metabase.com/pricing) for up-to-date information on pricing.
+
+When you upgrade from basic to advanced transforms, _all_ your transforms will be billed at advanced transforms rate.
+
+If you're on an Enterprise plan and have questions about billing, [contact us](https://www.metabase.com/help-premium).
diff --git a/docs/data-studio/transforms/jobs-and-runs.md b/docs/data-studio/transforms/jobs-and-runs.md
index 5de28bba303..3ebb6229080 100644
--- a/docs/data-studio/transforms/jobs-and-runs.md
+++ b/docs/data-studio/transforms/jobs-and-runs.md
@@ -33,16 +33,76 @@ _Data Studio > Jobs_
Jobs run one or more transforms on schedule based on transform tags.
-To see all jobs, go to **Data Studio** and click on the **Jobs** at the bottom of the left sidebar.
-
-To create a new job, go to **Data Studio > Jobs**, and click on the **+ New** button in the top right.
-
Jobs have two components: schedule and tags.
- **Schedule** determines when the job will be executed: daily, hourly, etc. You can specify a custom cron schedule (e.g. "Every weekday at 9:05 AM"). The times are given in your Metabase's system timezone.
- **Tags** determine _which_ transforms a job runs, not when the job runs. For example, you can create a `Weekdays` tag, add that tag to a few transforms, then create a job that runs all the transforms with the `Weekdays` tag every weekday at 9:05AM.
-Job can use multiple tags, and will run all transforms that have _any_ of those tags. For example, you can have a job "Weekend job" that is scheduled run at noon on Saturdays and Sundays that picks up all transforms tagged either "Saturday", "Sunday", or "Weekend".
+Jobs will run all transforms tagged with any of the tags, plus any transforms that the tagged transforms depend on, see [Jobs will run all dependent transforms](#jobs-will-run-all-dependent-transforms).
+
+You can see which transforms a job will run and in which order on the job's page.
+
+### See all jobs
+
+To see all jobs, go to **Data Studio** and click the **Jobs** at the bottom of the left sidebar.
+
+### Create a job
+
+To create a new job:
+
+1. Go to **Data Studio > Jobs**
+2. Click the **+ New** button in the top right.
+3. Specify the schedule: select one of the built-in schedules or use cron syntax to specify a custom schedule,
+
+ ![Jobs schedule](../images/jobs-schedule.png)
+
+ Job can use multiple tags, and will run all transforms that have _any_ of those tags. For example, you can have a job "Weekend job" that is scheduled run at noon on Saturdays and Sundays that picks up all transforms tagged either "Saturday", "Sunday", or "Weekend".
+
+### Disable jobs
+
+You can disable jobs without deleting them. Unlike deletion (which is permanent), disabling a job just means it won't run until you re-enable it. This is useful when you want to temporarily stop transforms from running - for example, for debugging purposes. This way you don't your lose configuration settings like tags and schedules.
+
+To disable a specific job:
+
+1. Go to **Data studio > Jobs**.
+2. Find the job you want to disable and click the **three dots** icon to the right of the job's name.
+3. Select **Disable**
+
+To disable all jobs:
+
+1. Go to **Data studio > Jobs**.
+2. Click the **three dots** icon above the table with all the jobs, and select **Disable all**.
+
+ ![Disable all jobs](../images/disable-all-jobs.png)
+
+Even if you disable all jobs, new jobs will still be created enabled by default.
+
+### Re-enable jobs
+
+If you [disabled any jobs](#disable-jobs), you can later re-enable them:
+
+To re-enable a specific job:
+
+1. Go to **Data studio > Jobs**.
+2. Find the job you want to re-enable and click the **three dots** icon to the right of the job's name.
+3. Select **Re-enable**
+
+To re-enable all jobs:
+
+1. Go to **Data studio > Jobs**.
+2. Click the **three dots** icon above the table with all the jobs, and select **Re-enable all**.
+
+### Delete a job
+
+Deleting a job will not delete any transforms.
+
+Deleted jobs can't be restored. If you want to temporarily stop a job from running, consider disabling the job instead.
+
+To delete a job:
+
+1. Go to **Data Studio > Jobs**.
+2. Find the job you want to delete and click the **three dots** icon to the right of the job's name.
+3. Select **Delete**.
## Jobs will run all dependent transforms
diff --git a/docs/data-studio/transforms/python-runner.md b/docs/data-studio/transforms/python-runner.md
index c0024289ada..0bde1cdc1e0 100644
--- a/docs/data-studio/transforms/python-runner.md
+++ b/docs/data-studio/transforms/python-runner.md
@@ -5,7 +5,7 @@ summary: Configure self-hosted Python execution environment to run Python transf
# Python runner
-> Self-hosted Python transforms require self-hosted Pro or Enterprise plan with the **Advanced transforms** add-on.
+> Self-hosted Python transforms require self-hosted Pro or Enterprise plan with the [Advanced transforms add-on](addons.md).
To run Python transforms from a self-hosted Metabase, you'll need to configure a separate self-hosted execution environment. If you're using Metabase Cloud, you'll need to buy the Transforms add-on.
@@ -95,7 +95,7 @@ docker run -d \
### Metabase environment variables
-These settings can also be configured in the Metabase UI at **Admin** > **Settings** > **Python Runner** (`/admin/settings/python-runner`). Note that environment variables take precedence over UI settings.
+These settings can also be configured in the Metabase UI at **Admin** > **Settings** > **Python Runner**. Note that environment variables take precedence over UI settings.
| Variable | Description |
| ----------------------------------------- | ---------------------------------------------------------------------------------- |
@@ -108,8 +108,6 @@ These settings can also be configured in the Metabase UI at **Admin** > **Settin
| `MB_PYTHON_STORAGE_S_3_SECRET_KEY` | S3 secret key |
| `MB_PYTHON_STORAGE_S_3_PATH_STYLE_ACCESS` | (Optional) Set to `true` for S3-compatible services like MinIO or LocalStack |
----
-
## Using Docker Compose
Docker Compose simplifies managing multiple containers. Below are example configurations for different scenarios.
diff --git a/docs/data-studio/transforms/python-transforms.md b/docs/data-studio/transforms/python-transforms.md
index aff313289ee..4d1de8d50cd 100644
--- a/docs/data-studio/transforms/python-transforms.md
+++ b/docs/data-studio/transforms/python-transforms.md
@@ -5,7 +5,7 @@ summary: Use Python to wrangle your data in Metabase and write the results back
# Python transforms
-> Python transforms require the **Transforms add-on**.
+> Python transforms require the [Advanced transforms add-on](addons.md)
Use Python to write [transforms](transforms-overview.md).
diff --git a/docs/data-studio/transforms/query-transforms.md b/docs/data-studio/transforms/query-transforms.md
index 976ce6eb835..e8b05f1290e 100644
--- a/docs/data-studio/transforms/query-transforms.md
+++ b/docs/data-studio/transforms/query-transforms.md
@@ -5,7 +5,7 @@ summary: Create Metabase questions and SQL queries to transform your data and wr
# Query-based transforms
-> On Metabase Cloud, you need the **Transforms** add-on to run query-based transforms.
+> On Metabase Cloud, you need the [**Basic transforms** add-on](addons.md) to run query-based transforms.
With query-based transforms, you can write a query in SQL or Metabase's query builder, and then write the results of the query back into the database on schedule.
@@ -23,23 +23,24 @@ For general information about Metabase transforms, see [Transforms](transforms-o
## Create a query-based transform
-On Metabase Cloud, you need the **Transforms** add-on to create query-based transforms.
+On Metabase Cloud, you need the [**Basic transforms** add-on](addons.md) to create query-based transforms.
-1. Go to **Data studio > Transforms**.
+1. [Enable transforms](transforms-overview.md#enable-transforms).
+2. Go to **Data studio > Transforms**.
-2. Click **+ New** and pick "Query builder", "SQL", or "Copy of existing question".
+3. Click **+ New** and pick "Query builder", "SQL", or "Copy of existing question".
Currently, you can't convert between different transform types (like converting a query builder transform to a SQL-based transform, or a SQL transform into a Python transform). If you want to change your transform built with the query builder into a SQL transform, you'll need to create a new transform with the same target and tags, and delete the old transform.
-3. Write your transform query as you would normally write a query in Metabase. See [Query builder](../../questions/query-builder/editor.md) and [SQL editor](../../questions/native-editor/writing-sql.md) documentation for more information.
+4. Write your transform query as you would normally write a query in Metabase. See [Query builder](../../questions/query-builder/editor.md) and [SQL editor](../../questions/native-editor/writing-sql.md) documentation for more information.
Not all databases support transforms, see [Databases that support transforms](transforms-overview.md#databases-that-support-transforms).
-4. To test your transform, press the **Run** button at the bottom of the editor.
+5. To test your transform, press the **Run** button at the bottom of the editor.
Previewing a query transform in the editor will _not_ write the result of the transform back to the database.
-5. Click **Save** in the top right corner and fill out the transform information:
+6. Click **Save** in the top right corner and fill out the transform information:
- **Name** (required): The name of the transform.
- **Schema** (required): Target schema for your transform. This schema can be different from the schema of the source table(s). You create a new schema by typing its name in this field. You can only transform data _within_ a database; you can't write from one database to another.
@@ -47,7 +48,7 @@ On Metabase Cloud, you need the **Transforms** add-on to create query-based tran
- **Folder** (optional): The folder where the transform should live. Click on the field to pick a different folder or create a new one.
- **Incremental transformation** (optional): see [Incremental query transforms](#incremental-query-transforms)
-6. Optionally, assign tags to your transforms. Tags are used by [jobs](jobs-and-runs.md) to run transforms on schedule.
+7. Optionally, assign tags to your transforms. Tags are used by [jobs](jobs-and-runs.md) to run transforms on schedule.
## Variables in SQL transforms
@@ -85,8 +86,6 @@ Parameters in transforms must either:
The reason transform variables must have a default value (or be optional) is that transforms run on a schedule, so there's no way to pass a value to the variable when the job runs the transform.
-The incremental `{%raw%}[[WHERE id > {{checkpoint}}]]{% endraw %}` pattern shown in [Incremental query transforms](#incremental-query-transforms) is an example of this an optional variable in practice. See also [optional variables](../../questions/native-editor/optional-variables.md).
-
## Run a query transform
See [Run a transform](transforms-overview.md#run-a-transform). You'll see logs for a transform run on the transform's page.
@@ -127,8 +126,8 @@ To make this transform incrementally load the data based on new values of `order
1. Add a table variable, for example `{{orders_var}}` replacing `orders` in the `FROM` statement;
2. In the table variable settings, connect the table variable to the `orders` table;
3. Replace other references to the table in your query with either:
- - The name of the table variable (if you have "Emit table alias" toggled on in variable's setting).
- - Your own handcrafted alias for the variable.
+ - The name of the table variable (if you have "Emit table alias" toggled on in variable's setting).
+ - Your own handcrafted alias for the variable.
So your query will look like this:
@@ -152,3 +151,36 @@ To make a query transform incremental:
1. Go to the transform's page in **Data studio > Transforms**.
2. Switch to **Settings** tab.
3. In **Column to check for new values**, select the column in one of the source tables that Metabase should check to determine which values are new. Only some columns are eligible. See [prerequisites for incremental transforms](./transforms-overview.md#prerequisites-for-incremental-transforms).
+
+## Convert models to transforms
+
+{% include plans-blockquote.html feature="Converting models to transforms"%}
+
+If you have models you'd like to migrate to transforms, Metabase can convert them for you. When you convert a model, Metabase:
+
+1. Creates a new transform based on the model's query.
+2. Runs the transform to create the output table in your database.
+3. Replaces every question, dashboard, and other item that used the model with the transform's output table.
+4. Converts the original model to a question.
+
+You must be an admin to convert models, and the model's database must [support transforms](transforms-overview.md#databases-that-support-transforms).
+
+To convert a model into a transform:
+
+1. Review [Replacing data sources](../dependencies/replace-data-sources.md) docs for overview and limitations of the process.
+2. Open **Data Studio** and select **Transforms** in the sidebar.
+3. Click **Tools > Migrate models**.
+4. Find the model you want to convert and click it to open its details panel. The panel shows the model's name, database, and collection, and a list of items that depend on it.
+5. Click **Convert to a transform**.
+6. Fill out the transform settings. See [Create a transform](#create-a-query-based-transform) for the overview of settings.
+7. Click **Convert to a transform**.
+
+Metabase runs the conversion in the background. A status indicator at the bottom of the screen shows progress.
+
+If the transform run fails, Metabase stops and leaves the model unchanged—nothing is replaced.
+
+If the transform runs successfully, but the source swap fails afterward, the transform and its output table are kept. The model is left unchanged. You can complete the migration manually using [Replace data sources](../dependencies/replace-data-sources.md) to point the remaining content from the model to the transform's output table.
+
+Once conversion completes, all content that previously queried the model now queries the transform's output table. The original model just becomes a question.
+
+Newly created tables will be created with default permissions and will _not_ inherit the model's permissions. As an alternative, consider manually creating and running the transform first, setting up the permissions, then using [Replace data sources](../dependencies/replace-data-sources.md) to swap the model for the transform's output table once you configured permissions.
diff --git a/docs/data-studio/transforms/transform-inspector.md b/docs/data-studio/transforms/transform-inspector.md
index 24b76950cd9..386400e568c 100644
--- a/docs/data-studio/transforms/transform-inspector.md
+++ b/docs/data-studio/transforms/transform-inspector.md
@@ -5,7 +5,7 @@ summary: Analyze how your transforms process data by inspecting input and output
# Transform inspector
-> Transform inspector requires the **Advanced transforms** add-on.
+> Transform inspector requires the [Advanced transforms add-on](addons.md).
_Data Studio > Transforms > [transform name] > Inspect_
diff --git a/docs/data-studio/transforms/transforms-overview.md b/docs/data-studio/transforms/transforms-overview.md
index 13bd2aa139f..d0e50daaa9e 100644
--- a/docs/data-studio/transforms/transforms-overview.md
+++ b/docs/data-studio/transforms/transforms-overview.md
@@ -36,7 +36,7 @@ Currently, Metabase can create transforms on the following databases:
You can't create transforms on databases that have [Database routing](../../permissions/database-routing.md) enabled, or on Metabase's Sample Database.
-Transforms will create tables in your database, so the database user you use for your connection must have appropriate privileges. See [Database users, roles, and privileges](../../databases/users-roles-privileges.md). We suggest using a [Writable connection](../../databases/writable-connection.md) option for your database.
+Transforms will create tables in your database, so the database user you use for your connection must have appropriate privileges. See [Database users, roles, and privileges](../../databases/users-roles-privileges.md). We suggest using a [Writable connection](../../databases/writable-connection.md) for your database.
## Types of transforms
@@ -47,22 +47,39 @@ Metabase supports two types of transforms: query-based transforms and Python tra
## Permissions for transforms
-If you are running Metabase Open Source/Starter, Admins (and only Admins) can see and run transforms.
+Permission configuration for transform depends on your plan.
-Metabase Pro/Enterprise comes with additional permission controls for transforms:
+- **Metabase Open Source/Starter**: Admins (and only Admins) can see and run transforms.
-- To **see** the list of transforms on your instance, people need to be able to access Data Studio, so they need to be either an Admin or a member of the special [Data Analyst group](../../people-and-groups/managing.md).
-- To **execute** transforms on a database, people additionally need to have the [Transform permissions](../../permissions/data.md) for that database.
+- **Metabase Pro/Enterprise** comes with additional permission controls for transforms: a special [Data Analysts](../../people-and-groups/managing.md) group for non-Admins with potential transform access, and granular transform permissions for each database:
+
+ - To **see** the list of transforms on your instance, people need to be able to access Data Studio, so they need to be either an Admin or a member of the special [Data Analyst group](../../people-and-groups/managing.md).
+ - To **execute** transforms on a database, people need to be either Admins on belong to the special [Data Analyst group](../../people-and-groups/managing.md). Additionally people need to have the [Transform permissions](../../permissions/data.md) for that database.
+
+## Enable transforms
+
+Before you can start writing transforms, you'll need to enable transforms in your Metabase instance.
+
+If you are on a Metabase Cloud plan, only people logged in with an email of a [Metabase Store admins](../../cloud/accounts-and-billing.md#add-people-to-manage-your-metabase-store-account) (not just Metabase _instance_ admins) can enable basic transforms, because transforms incur a cost per run on Metabase Cloud.
+
+To enable transforms:
+
+1. Navigate to [**Data Studio**](../overview.md) by click the **grid icon** in top right corner of your Metabase and selecing **Data Studio**.
+2. In Data Studio, click **Transforms** in the right sidebar.
+3. If the transforms have not been enabled on your instance yet, you'll see a prompt to enable them. You can do just that.
+
+Once you've enabled transforms, you can [configure permissions](#permissions-for-transforms) and start [creating transforms](#create-a-transform).
## See all transforms
_Data Studio > Transforms_
+See [permissions needed to see transforms](#permissions-for-transforms).
+
You can see all your Metabase's transforms:
-1. Make sure you have [appropriate permissions to see transforms](#permissions-for-transforms).
-2. Click the **grid** icon on top right and go to **Data Studio**.
-3. In the left sidebar, select **Transforms**.
+1. Click the **grid** icon on top right and go to **Data Studio**.
+2. In the left sidebar, select **Transforms**.
![Transforms](../images/transforms.png)
@@ -70,11 +87,13 @@ You can see all your Metabase's transforms:
_Data Studio > Transforms_
-If you're using remote sync, you won't be able to create transforms if your instance is in ["read-only" sync mode](../../installation-and-operation/remote-sync.md).
+> If you're using remote sync, you won't be able to create transforms if your instance is in ["read-only" sync mode](../../installation-and-operation/remote-sync.md).
+
+See [permissions needed to create transforms](#permissions-for-transforms).
To create a transform:
-1. Make sure you have [appropriate permissions for creating transforms](#permissions-for-transforms).
+1. [Enable transforms](#enable-transforms).
2. Click the **grid** icon on top right and go to **Data Studio**.
3. In the left sidebar, select **Transforms**.
4. Click **+ New** and select a source for your transform.
@@ -111,7 +130,7 @@ To create a transform:
## Use Metabot to generate code for transforms
-> Code generation for transforms requires [Metabot to be enabled](../../ai/settings.md#enable-ai-features) and the **Transforms** add-on.
+> Code generation for transforms requires [AI features](../../ai/settings.md#enable-ai-features).
You can ask Metabot to generate a new SQL or Python-based transform, or edit an existing transform.
@@ -145,12 +164,13 @@ If you're using remote sync, you won't be able to edit transforms if your instan
_Data Studio > Transforms > Definition_
+See [permissions to edit transforms](#permissions-for-transforms).
+
To edit the transform's query or script:
-1. Make sure you have [permissions to edit transforms](../../permissions/data.md).
-2. Go to **Data Studio > Transforms**.
-3. Find the transform you'd like to edit and click on **Edit definition** above the transform definition.
-4. Edit the query or script.
+1. Go to **Data Studio > Transforms**.
+2. Find the transform you'd like to edit and click on **Edit definition** above the transform definition.
+3. Edit the query or script.
See [query-based transforms](query-transforms.md) and [Python transforms](python-transforms.md) for more information. You can [use Metabot](#use-metabot-to-generate-code-for-transforms) to help edit your transform.
@@ -168,11 +188,11 @@ To edit transform's target table, i.e., the table where the query results are wr
You can run a transform manually or schedule the transform using tags and jobs.
-Running a transform for the first time will create and sync the table created by the transform, and you'll be able to edit the table's [metadata](../../data-modeling/metadata-editing.md) and [permissions](../../permissions/data.md). Subsequent runs will drop and recreate the table, unless you use [Incremental transforms](#incremental-transforms).
+- To run a transform manually, visit the transform in **Data Studio > Transforms > Runs** and click **Run**.
-To run a transform manually, visit the transform in **Data Studio > Transforms > Runs** and click **Run**.
+- To schedule a transform, you'll need to assign one or more tags to it, then create a [scheduled job](./jobs-and-runs.md) that picks up those tags.
-To schedule a transform, you'll need to assign one or more tags to it, then create a [scheduled job](./jobs-and-runs.md) that picks up those tags.
+Running a transform for the first time will create and sync the table created by the transform, and you'll be able to edit the table's [metadata](../../data-modeling/metadata-editing.md) and [permissions](../../permissions/data.md). Subsequent runs will drop and recreate the table, unless you use [Incremental transforms](#incremental-transforms).
You can see the time and status of the latest transform run on the transform's page, or in the [Runs view](./jobs-and-runs.md). The time of the run is given in the system's timezone.
@@ -182,6 +202,8 @@ For Python transforms, you'll also see the transform's execution logs.
_Data Studio > Transforms > [transform name] > Inspect_
+> Transform inspector requires the [Advanced transforms add-on](addons.md)
+
The [transform inspector](./transform-inspector.md) lets you poke at the input and outputs of your transform.
## Transform dependencies
@@ -200,7 +222,7 @@ On Metabase Pro or Enterprise plans, you can see the transform dependencies grap
If a job includes a transform that depends on a table created by another transform, then the job will run all the tagged transforms and their dependencies, even if they lack tags, see [Jobs and runs](jobs-and-runs.md) for more information.
-### Incremental transforms
+## Incremental transforms
_Data Studio > Transforms > Settings_
@@ -220,6 +242,8 @@ Incremental transforms work differently for query-based transforms and Python tr
_Admin > General > Remote sync_
+{% include plans-blockquote.html feature="Versioning transforms" %}
+
You can check your transforms into git with [Remote Sync](../../installation-and-operation/remote-sync.md). If you enable transform sync, Metabase will serialize transforms as YAML files and push them to your specified GitHub repo branch.
To enable git sync of transforms:
@@ -241,34 +265,4 @@ Transforms are similar to models with model persistence turned on, but there are
Use models to enable non-admins to create their own datasets within Metabase, and to add context like field descriptions and semantic types. Use transforms to create persisted datasets in your database and reuse them across Metabase. In future versions of Metabase, model persistence will be deprecated in favor of transforms.
-## Convert models to transforms
-
-If you have models you'd like to migrate to transforms, Metabase can convert them for you. When you convert a model, Metabase:
-
-1. Creates a new transform based on the model's query.
-2. Runs the transform to create the output table in your database.
-3. Replaces every question, dashboard, and other item that used the model with the transform's output table.
-4. Converts the original model to a question.
-
-You must be an admin to convert models, and the model's database must [support transforms](#databases-that-support-transforms).
-
-Before converting models into transforms, review [Replacing data sources](../dependencies/replace-data-sources.md) docs for overview and limitations of the process.
-
-To convert a model into a transform:
-
-1. Open **Data Studio** and select **Transforms** in the sidebar.
-2. Click **Tools > Migrate models**.
-3. Find the model you want to convert and click it to open its details panel. The panel shows the model's name, database, and collection, and a list of items that depend on it.
-4. Click **Convert to a transform**.
-5. Fill out the transform settings. See [Create a transform](#create-a-transform) for the overview of settings.
-6. Click **Convert to a transform**.
-
-Metabase runs the conversion in the background. A status indicator at the bottom of the screen shows progress.
-
-If the transform run fails, Metabase stops and leaves the model unchanged—nothing is replaced.
-
-If the transform runs successfully but the source swap fails afterward, the transform and its output table are kept. The model is left unchanged. You can complete the migration manually using [Replace data sources](../dependencies/replace-data-sources.md) to point remaining content from the model to the transform's output table.
-
-Once conversion completes, all content that previously queried the model now queries the transform's output table, and the model becomes a saved question.
-
-Newly created tables will be created with default permissions and will _not_ inherit the model's permissions. As an alternative, consider manually creating and running the transform first, setting up the permissions, then using [Replace data sources](../dependencies/replace-data-sources.md) to swap the model for the transform's output table once you configured permissions.
+On Metabase Pro/Enterprise plans, you can convert Metabase models to transforms in bulk, see [Convert models to transforms](query-transforms.md#convert-models-to-transforms)
diff --git a/docs/databases/writable-connection.md b/docs/databases/writable-connection.md
index f51a5902878..ac98bca720e 100644
--- a/docs/databases/writable-connection.md
+++ b/docs/databases/writable-connection.md
@@ -5,9 +5,9 @@ redirect_from:
- /docs/latest/databases/writeable-connection
---
-## Writable connection
+# Writable connection
-> Writable connection requires the **Advanced transforms (SQL + Python)** add-on
+> Writable connection requires the [Advanced transforms add-on](../data-studio/transforms/addons.md)
_Admin > Databases > Writable connection_