Skip to content

The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.

Standardize external replication (ETL) docs (#46875)

Riccardo Busettif14d49dd2d8b
diff --git a/apps/docs/content/guides/database/postgres/roles.mdx b/apps/docs/content/guides/database/postgres/roles.mdx
index e5271f84df9..3286c786e49 100644
--- a/apps/docs/content/guides/database/postgres/roles.mdx
+++ b/apps/docs/content/guides/database/postgres/roles.mdx
@@ -113,7 +113,15 @@ Used by the Auth middleware to connect to the database and run migration. Access
### `supabase_etl_admin`
-Used by [Replication powered by Supabase ETL](/docs/guides/database/replication) to replicate database changes to external destinations. Has read-all access and replication privileges for change data capture, bypasses Row Level Security, and can write to the `etl` schema.
+`supabase_etl_admin` is used by [Database replication](/docs/guides/database/replication) through Supabase ETL.
+
+This role:
+
+- Replicates database changes to destination systems
+- Has read-all access
+- Has replication privileges for change data capture, bypasses Row Level Security
+- Can create event triggers
+- Can write to the `etl` schema
### `dashboard_user`
diff --git a/apps/docs/content/guides/database/replication.mdx b/apps/docs/content/guides/database/replication.mdx
index 53a92e048be..8ff0b26278f 100644
--- a/apps/docs/content/guides/database/replication.mdx
+++ b/apps/docs/content/guides/database/replication.mdx
@@ -1,7 +1,7 @@
---
id: 'replication'
title: 'Database replication'
-description: 'Compare read replicas, external replication, and manual replication.'
+description: 'Compare read replicas, external replication (ETL), and manual replication.'
subtitle: 'An introduction to database replication and change data capture.'
sidebar_label: 'Overview'
---
@@ -18,7 +18,7 @@ You might use database replication for:
## Replication methods
-Supabase supports three replication methods. Choose based on whether you need another Supabase Postgres database, a managed pipeline to an external system, or full control over your own logical replication setup.
+Supabase supports three replication methods. Choose based on whether you need another Supabase Postgres database, a managed pipeline to a destination system, or full control over your own logical replication setup.
### Read replicas
@@ -26,21 +26,21 @@ Read replicas are additional Supabase Postgres databases kept in sync with your
- [Set up read replicas](/docs/guides/platform/read-replicas)
-### External replication
+### External replication (ETL) [#external-replication]
<Admonition type="caution" title="Private Alpha">
-External replication is currently in private alpha. Access is limited and features may change.
+External replication (ETL) is currently in private alpha. Managed pipelines run through Supabase ETL. Access is limited and features may change.
</Admonition>
-External replication is powered by [Supabase ETL](https://github.com/supabase/etl). It uses Postgres logical replication under the hood and provides a managed Dashboard workflow for replicating data from Supabase Postgres to external data systems.
+External replication (ETL) is powered by [Supabase ETL](https://github.com/supabase/etl). It uses Postgres logical replication under the hood and provides a managed Dashboard workflow for replicating data from Supabase Postgres to destination systems. In this context, "external" means outside the source database. Destinations can be Supabase-managed or third-party systems as support expands.
-- [Set up external replication](/docs/guides/database/replication/external-replication-setup)
+- [Set up external replication (ETL)](/docs/guides/database/replication/external-replication-setup)
#### Supported destinations
-External replication currently supports BigQuery as the managed destination. We are working on new destinations, and this table will be updated as support expands.
+External replication (ETL) currently supports BigQuery as the managed destination. We are working on new destinations, and this table will be updated as support expands.
| Destination | Insert | Update | Delete | Truncate | Schema change | Description |
| ------------------------------------------------------ | ------------ | ------------ | ------------ | ------------ | ------------- | ------------------------------------------------------------------- |
@@ -48,7 +48,7 @@ External replication currently supports BigQuery as the managed destination. We
### Manual replication
-Manual replication uses the same underlying Postgres logical replication features as external replication, but you configure and operate the pieces yourself. Use this path when you want to connect tools such as Airbyte, Estuary, Fivetran, Materialize, Stitch, AWS DMS, or another system that supports Postgres logical replication.
+Manual replication uses the same underlying Postgres logical replication features as external replication (ETL), but you configure and operate the pieces yourself. Use this path when you want to connect tools such as Airbyte, Estuary, Fivetran, Materialize, Stitch, AWS DMS, or another system that supports Postgres logical replication.
- [Set up manual replication](/docs/guides/database/replication/manual-replication-setup)
diff --git a/apps/docs/content/guides/database/replication/bigquery.mdx b/apps/docs/content/guides/database/replication/bigquery.mdx
index b224b2185cb..12f1ae14fc9 100644
--- a/apps/docs/content/guides/database/replication/bigquery.mdx
+++ b/apps/docs/content/guides/database/replication/bigquery.mdx
@@ -1,14 +1,14 @@
---
id: 'bigquery-destination'
title: 'BigQuery destination'
-description: 'Configure BigQuery as an external replication destination.'
+description: 'Configure BigQuery as an external replication (ETL) destination.'
subtitle: 'Replicate Supabase Postgres tables to BigQuery.'
sidebar_label: 'BigQuery'
---
<Admonition type="caution" title="Private Alpha">
-External replication is currently in private alpha. Access is limited and features may change.
+External replication (ETL) is currently in private alpha. Managed pipelines run through Supabase ETL. Access is limited and features may change.
</Admonition>
@@ -90,9 +90,9 @@ Once configured, replication to BigQuery:
## Source table requirements
-BigQuery replication requires each source table to have a primary key, and the publication must include the primary-key columns. External replication declares those columns as the BigQuery destination primary key so BigQuery CDC can apply `UPSERT` and `DELETE` rows.
+BigQuery replication requires each source table to have a primary key, and the publication must include the primary-key columns. Supabase ETL declares those columns as the BigQuery destination primary key so BigQuery change data capture (CDC) can apply `UPSERT` and `DELETE` rows.
-BigQuery primary keys are `NOT ENFORCED`, and BigQuery CDC supports composite primary keys with up to 16 columns. Your source primary key must stay unique and non-null because BigQuery uses it to match CDC rows.
+BigQuery primary keys are `NOT ENFORCED`, and BigQuery change data capture (CDC) supports composite primary keys with up to 16 columns. Your source primary key must stay unique and non-null because BigQuery uses it to match CDC rows.
Source tables must also use a BigQuery-compatible Postgres `REPLICA IDENTITY` setting. Most tables can keep the Postgres default, as long as they have a primary key and all primary-key columns are included in the publication.
@@ -100,13 +100,13 @@ Source tables must also use a BigQuery-compatible Postgres `REPLICA IDENTITY` se
| ------------------------------------------------ | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `REPLICA IDENTITY DEFAULT` with a primary key | Supported | Recommended for most tables. BigQuery uses the replicated source primary key to apply upserts and deletes. |
| `REPLICA IDENTITY FULL` | Supported | Recommended for tables with large `text`, `jsonb`, `bytea`, or other values that Postgres may store out-of-line using TOAST, especially when those rows update. |
-| `REPLICA IDENTITY USING INDEX` | Not supported | BigQuery CDC rows are keyed by the source primary key, not by an alternative unique index. |
+| `REPLICA IDENTITY USING INDEX` | Not supported | BigQuery change data capture (CDC) rows are keyed by the source primary key, not by an alternative unique index. |
| `REPLICA IDENTITY NOTHING` | Not supported | Updates and deletes do not include enough row identity for BigQuery to apply them safely. |
| `REPLICA IDENTITY DEFAULT` without a primary key | Not supported | BigQuery requires a source primary key. |
For a general explanation of how replica identity affects update and delete events, see [How does replica identity affect updates and deletes?](/docs/guides/database/replication/external-replication-faq#how-does-replica-identity-affect-updates-and-deletes).
-For updates, Postgres does not always send a complete old row through logical replication. It can also mark unchanged toasted values as `unchanged toast` instead of resending the value. BigQuery CDC upserts require a complete new row because omitted columns are not preserved in the destination. The replication pipeline can reconstruct a complete update when the old row image contains the missing value, which is reliable with `REPLICA IDENTITY FULL`.
+For updates, Postgres does not always send a complete old row through logical replication. It can also mark unchanged toasted values as `unchanged toast` instead of resending the value. BigQuery change data capture (CDC) upserts require a complete new row because omitted columns are not preserved in the destination. The replication pipeline can reconstruct a complete update when the old row image contains the missing value, which is reliable with `REPLICA IDENTITY FULL`.
If a BigQuery pipeline fails with an error about a partial update row, set `REPLICA IDENTITY FULL` on the affected source table and restart the pipeline. Changing replica identity only affects new WAL records, so a retained update that was written before the change may still need to be skipped by recreating the pipeline or re-copying the affected table.
@@ -145,7 +145,7 @@ This structure handles table truncations while maintaining query compatibility.
## Schema change support
-Schema change support for BigQuery is currently in beta. External replication supports a limited set of schema changes while the feature is developed further.
+Schema change support for BigQuery is currently in beta. Supabase ETL supports a limited set of schema changes for BigQuery while the feature is developed further.
Supported schema changes:
@@ -162,7 +162,7 @@ Unsupported or limited schema changes:
- Filling existing rows for `ADD COLUMN ... DEFAULT`
- Unsupported default expressions
-BigQuery requires added columns to be nullable. When a replicated `ADD COLUMN` includes a default, external replication can apply supported default metadata for future rows, but BigQuery does not backfill existing rows through that DDL. Existing destination rows remain `NULL` unless you run a separate backfill.
+BigQuery requires added columns to be nullable. When a replicated `ADD COLUMN` includes a default, external replication (ETL) can apply supported default metadata for future rows, but BigQuery does not backfill existing rows through that DDL. Existing destination rows remain `NULL` unless you run a separate backfill.
Supported defaults are best-effort translations to BigQuery SQL. Unsupported defaults are skipped with a warning instead of failing replication.
@@ -177,4 +177,4 @@ Supported defaults are best-effort translations to BigQuery SQL. Unsupported def
## Additional resources
- [BigQuery documentation](https://cloud.google.com/bigquery/docs) - Official Google BigQuery documentation
-- [BigQuery change data capture](https://cloud.google.com/bigquery/docs/change-data-capture) - BigQuery CDC requirements and limitations
+- [BigQuery change data capture](https://cloud.google.com/bigquery/docs/change-data-capture) - BigQuery change data capture (CDC) requirements and limitations
diff --git a/apps/docs/content/guides/database/replication/external-replication-faq.mdx b/apps/docs/content/guides/database/replication/external-replication-faq.mdx
index b511cb07198..6c56a465e64 100644
--- a/apps/docs/content/guides/database/replication/external-replication-faq.mdx
+++ b/apps/docs/content/guides/database/replication/external-replication-faq.mdx
@@ -1,26 +1,26 @@
---
id: 'external-replication-faq'
-title: 'External replication FAQ'
-description: 'Frequently asked questions about external replication.'
-subtitle: 'Common questions and answers about external replication.'
+title: 'External replication (ETL) FAQ'
+description: 'Frequently asked questions about external replication (ETL).'
+subtitle: 'Common questions and answers about external replication (ETL).'
sidebar_label: 'FAQ'
---
<Admonition type="caution" title="Private Alpha">
-External replication is currently in private alpha. Access is limited and features may change.
+External replication (ETL) is currently in private alpha. Managed pipelines run through Supabase ETL. Access is limited and features may change.
</Admonition>
## What destinations are supported?
-External replication currently supports **BigQuery** as the managed destination. See the [BigQuery destination guide](/docs/guides/database/replication/bigquery) for configuration details.
+External replication (ETL) currently supports **BigQuery** as the managed destination. See the [BigQuery destination guide](/docs/guides/database/replication/bigquery) for configuration details.
-We are working on new destinations. Availability may continue to vary based on the planned roll-out strategy.
+We are working on new destinations. "External" means outside the source database, so supported destinations can be Supabase-managed or third-party systems as support expands. Availability may continue to vary based on the planned roll-out strategy.
-## Which plans support external replication?
+## Which plans support external replication (ETL)? [#which-plans-support-external-replication]
-External replication is available on the Pro, Team, and Enterprise plans.
+External replication (ETL) is available on the Pro, Team, and Enterprise plans.
{/* supa-mdx-lint-disable-next-line Rule001HeadingCase */}
@@ -30,11 +30,11 @@ We are currently working on a new Supabase Warehouse product designed to address
As a result, managed replication to Analytics Buckets is no longer available. Right now, **BigQuery** is the only supported managed destination, and we are actively working on expanding capabilities.
-## What does external replication install in the database?
+## What does external replication (ETL) install in the database? [#what-does-external-replication-install-in-the-database]
-When you enable external replication, Supabase installs database objects that help track replication state and support schema changes:
+When you enable external replication (ETL), Supabase installs database objects that help Supabase ETL track replication state and support schema changes:
-- An event trigger that runs on every `ALTER TABLE` statement. External replication uses this to support schema change handling.
+- An event trigger that runs on every `ALTER TABLE` statement. Supabase ETL uses this to support schema change handling.
- A set of tables in the `etl` schema. These tables track replication state for your pipelines.
The replication state tables are not updated very often, especially after the initial copy phase is complete.
@@ -51,15 +51,15 @@ Supported schema changes:
- Dropping a `NOT NULL` constraint
- Setting or dropping supported column default metadata
-External replication does not currently support changing column data types, adding `NOT NULL` with `SET NOT NULL`, or filling existing destination rows for `ADD COLUMN ... DEFAULT`. Supported defaults are applied as destination metadata for future rows where BigQuery can represent them. See [BigQuery schema change support](/docs/guides/database/replication/bigquery#schema-change-support) for details.
+External replication (ETL) does not currently support changing column data types, adding `NOT NULL` with `SET NOT NULL`, or filling existing destination rows for `ADD COLUMN ... DEFAULT`. Supported defaults are applied as destination metadata for future rows where BigQuery can represent them. See [BigQuery schema change support](/docs/guides/database/replication/bigquery#schema-change-support) for details.
-## What happens when you disable external replication?
+## What happens when you disable external replication (ETL)? [#what-happens-when-you-disable-external-replication]
-Disabling external replication removes the database objects that were installed in your source database, including the replication state tables in the `etl` schema and the DDL event trigger.
+Disabling external replication (ETL) removes the database objects that were installed in your source database, including the replication state tables in the `etl` schema and the DDL event trigger.
-You must delete all external replication pipelines before the disable action is available. For the Dashboard steps, see [Disabling external replication](/docs/guides/database/replication/external-replication-setup#disabling-external-replication).
+To remove external replication (ETL) from a project, delete all replication pipelines first. After all pipelines are deleted, the disable action becomes available. For the Dashboard steps, see [Disabling external replication (ETL)](/docs/guides/database/replication/external-replication-setup#disabling-external-replication).
-Disabling external replication stops Supabase from managing external replication for the project. It does not delete tables or data that were already written to your destination.
+Disabling external replication (ETL) stops Supabase from managing replication for the project. It does not delete tables or data that were already written to your destination.
## Why is a table not being replicated?
@@ -67,13 +67,15 @@ Common reasons:
- **Missing primary key**: Tables must have a primary key to be replicated (Postgres logical replication requirement)
- **Not in publication**: Ensure the table is included in your Postgres publication
-- **Unsupported data types**: Tables with custom data types are not supported
+- **Generated columns**: Generated columns are skipped during replication
Check your publication settings and verify your table meets the requirements.
+Custom data types replicate as strings. Check that your destination can interpret those string values correctly.
+
## Why are partitioned tables replicated as separate tables?
-Postgres controls this with the publication's `publish_via_partition_root` setting. If the setting is `false`, or if you created the publication manually with SQL and did not set it, Postgres publishes changes from the leaf partitions. External replication then creates destination tables for those leaf partitions. If `publish_via_partition_root = true`, Postgres publishes changes as the partition root, so external replication treats the partition hierarchy as the published partition root.
+Postgres controls this with the publication's `publish_via_partition_root` setting. If the setting is `false`, or if you created the publication manually with SQL and did not set it, Postgres publishes changes from the leaf partitions. External replication (ETL) then creates destination tables for those leaf partitions. If `publish_via_partition_root = true`, Postgres publishes changes as the partition root, so the partition hierarchy is treated as the published partition root.
Publications created from the Dashboard replication flow use `publish_via_partition_root = true`.
@@ -169,7 +171,7 @@ Check the [**Database > Replication**](/dashboard/project/_/database/replication
3. Ensure all tables show **Live** state (actively replicating)
4. Monitor replication lag metrics
-See the [external replication monitoring guide](/docs/guides/database/replication/external-replication-monitoring) for comprehensive monitoring instructions.
+See the [external replication (ETL) monitoring guide](/docs/guides/database/replication/external-replication-monitoring) for comprehensive monitoring instructions.
## How to stop or pause replication
@@ -183,13 +185,13 @@ Stopping replication causes changes to queue up in the WAL.
## What happens if a project becomes inactive?
-If your project becomes inactive, external replication stops any running pipelines and does not automatically resume them after the project is restarted.
+If your project becomes inactive, external replication (ETL) stops any running pipelines and does not automatically resume them after the project is restarted.
After restarting the project, restart each replication pipeline manually from the [**Database > Replication**](/dashboard/project/_/database/replication) section of the Dashboard.
## What happens after a downgrade to the free plan?
-When a project is downgraded to the Free Plan, all external replication pipelines for that project are deleted.
+When a project is downgraded to the Free Plan, all external replication (ETL) pipelines for that project are deleted.
## What happens if a table is deleted at the destination?
@@ -243,6 +245,6 @@ Navigate to the [**Logs > Replication**](/dashboard/project/_/logs/explorer) sec
If you need assistance:
-1. Check the [external replication setup guide](/docs/guides/database/replication/external-replication-setup) and [external replication monitoring guide](/docs/guides/database/replication/external-replication-monitoring)
+1. Check the [external replication (ETL) setup guide](/docs/guides/database/replication/external-replication-setup) and [external replication (ETL) monitoring guide](/docs/guides/database/replication/external-replication-monitoring)
2. Review this FAQ for common issues
3. Contact support with your error details and logs
diff --git a/apps/docs/content/guides/database/replication/external-replication-monitoring.mdx b/apps/docs/content/guides/database/replication/external-replication-monitoring.mdx
index eb22ca4c65c..789607ce586 100644
--- a/apps/docs/content/guides/database/replication/external-replication-monitoring.mdx
+++ b/apps/docs/content/guides/database/replication/external-replication-monitoring.mdx
@@ -1,18 +1,18 @@
---
id: 'external-replication-monitoring'
-title: 'External replication monitoring'
-description: 'Monitor the status and health of your external replication pipelines.'
+title: 'External replication (ETL) monitoring'
+description: 'Monitor the status and health of your external replication (ETL) pipelines.'
subtitle: 'Track replication status, view logs, and troubleshoot issues.'
sidebar_label: 'Monitoring'
---
<Admonition type="caution" title="Private Alpha">
-External replication is currently in private alpha. Access is limited and features may change.
+External replication (ETL) is currently in private alpha. Managed pipelines run through Supabase ETL. Access is limited and features may change.
</Admonition>
-After setting up external replication, you can monitor the status and health of your replication pipelines directly from the Dashboard. The pipeline is the active Postgres replication process that continuously streams changes from your database to your destination.
+After setting up external replication (ETL), you can monitor the status and health of your replication pipelines directly from the Dashboard. The pipeline is the active Postgres replication process that continuously streams changes from your database to your destination.
### Viewing pipeline status
@@ -69,7 +69,7 @@ The detailed status page shows:
| **Connected** | Whether the pipeline's replication slot is active and currently being used. | `Not connected` while the pipeline should be running usually means you should check pipeline status and logs. |
| **Slot status** | How safely Postgres is keeping the WAL files the pipeline still needs. | `Unreserved` and `Lost` require action. See [Slot statuses](#slot-statuses). |
-External replication uses one main pipeline replication slot for ongoing changes. During the initial copy phase, it can also create temporary table-sync replication slots. These temporary slots let multiple tables copy in parallel, make large table copies faster, and allow individual tables to be retried or copied again without restarting the whole pipeline.
+External replication (ETL) uses one main pipeline replication slot for ongoing changes. During the initial copy phase, it can also create temporary table-sync replication slots. These temporary slots let multiple tables copy in parallel, make large table copies faster, and allow individual tables to be retried or copied again without restarting the whole pipeline.
Temporary table-sync slots show the same kind of lag and slot health metrics while they are active. After a table finishes copying and catches up, its temporary slot is removed and ongoing changes continue through the main pipeline slot. For overall replication health, focus first on the main pipeline slot.
@@ -265,9 +265,9 @@ If you notice issues with your replication:
4. **Verify publication**: Ensure your Postgres publication is properly configured
5. **Monitor replication lag**: High lag may indicate performance issues
-For more troubleshooting tips, see the [external replication FAQ](/docs/guides/database/replication/external-replication-faq).
+For more troubleshooting tips, see the [external replication (ETL) FAQ](/docs/guides/database/replication/external-replication-faq).
### Next steps
-- [Set up external replication](/docs/guides/database/replication/external-replication-setup)
-- [View external replication FAQ](/docs/guides/database/replication/external-replication-faq)
+- [Set up external replication (ETL)](/docs/guides/database/replication/external-replication-setup)
+- [View external replication (ETL) FAQ](/docs/guides/database/replication/external-replication-faq)
diff --git a/apps/docs/content/guides/database/replication/external-replication-setup.mdx b/apps/docs/content/guides/database/replication/external-replication-setup.mdx
index f82edd4f457..849560ba439 100644
--- a/apps/docs/content/guides/database/replication/external-replication-setup.mdx
+++ b/apps/docs/content/guides/database/replication/external-replication-setup.mdx
@@ -1,26 +1,26 @@
---
id: 'external-replication-setup'
-title: 'Set up external replication'
-description: 'Set up external replication using Postgres logical replication.'
-subtitle: 'Configure publications and destinations for external replication.'
+title: 'Set up external replication (ETL)'
+description: 'Set up external replication (ETL) using Postgres logical replication.'
+subtitle: 'Configure publications and destinations for external replication (ETL).'
sidebar_label: 'Setting up'
---
<Admonition type="caution" title="Private Alpha">
-External replication is currently in private alpha. Access is limited and features may change.
+External replication (ETL) is currently in private alpha. Managed pipelines run through Supabase ETL. Access is limited and features may change.
</Admonition>
-External replication is powered by [Supabase ETL](https://github.com/supabase/etl) and uses **Postgres logical replication** to stream changes from your database to external data systems. It provides a managed interface through the Dashboard to configure and monitor replication pipelines.
+External replication (ETL) is powered by [Supabase ETL](https://github.com/supabase/etl) and uses **Postgres logical replication** to stream changes from your database to destination systems. It provides a managed interface through the Dashboard to configure and monitor replication pipelines. In this context, "external" means outside the source database; destinations can be Supabase-managed or third-party systems as support expands.
## Setup overview
-External replication requires two main components: a **Postgres publication** (defines what to replicate) and a **destination** (where data is sent). Follow these steps to set up your replication pipeline.
+External replication (ETL) requires two main components: a **Postgres publication** (defines what to replicate) and a **destination** (where data is sent). Supabase ETL runs the managed pipeline that reads from the publication and writes to the destination. Follow these steps to set up your replication pipeline.
<Admonition type="tip">
-If you already have a Postgres publication set up, you can skip to [Step 2: Enable external replication](#step-2-enable-external-replication).
+If you already have a Postgres publication set up, you can skip to [Step 2: Enable external replication (ETL)](#step-2-enable-external-replication).
</Admonition>
@@ -90,7 +90,7 @@ for table orders where (created_at > '2024-01-01');
##### Partitioned tables
-External replication follows Postgres publication semantics for partitioned tables. The `publish_via_partition_root` publication setting controls whether changes from partitions are emitted as the partition root or as the leaf partitions.
+External replication (ETL) follows Postgres publication semantics for partitioned tables. The `publish_via_partition_root` publication setting controls whether changes from partitions are emitted as the partition root or as the leaf partitions.
| Publication setting | What gets replicated | Destination shape |
| ------------------------------------------ | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------- |
@@ -118,7 +118,7 @@ Use `publish_via_partition_root = true` when you want analytics queries to read
Publications created from the Dashboard replication flow use `publish_via_partition_root = true`. If you create or alter a publication manually with SQL, set this option explicitly so the destination shape matches what you expect.
-On Postgres 15 and newer, row filters on partition publications apply during both the initial copy and streaming phases. External replication uses the row filter attached to the effective publication table entry: the published partition root when `publish_via_partition_root = true`, and the published leaf relation when `publish_via_partition_root = false`.
+On Postgres 15 and newer, row filters on partition publications apply during both the initial copy and streaming phases. External replication (ETL) uses the row filter attached to the effective publication table entry: the published partition root when `publish_via_partition_root = true`, and the published leaf relation when `publish_via_partition_root = false`.
<Admonition type="note">
@@ -133,17 +133,17 @@ After creating a publication via SQL, you can view it in the Dashboard:
1. Navigate to the [**Database > Publications**](/dashboard/project/_/database/publications) section of the Dashboard
2. You'll see all your publications listed with their tables
-### Step 2: Enable external replication
+### Step 2: Enable external replication (ETL) [#step-2-enable-external-replication]
-Before creating an external replication pipeline, enable external replication for your project:
+Before creating an external replication (ETL) pipeline, enable it for your project:
1. Navigate to the [**Database > Replication**](/dashboard/project/_/database/replication) section of the Dashboard
2. Click **Add destination** to show the replication side panel
-3. Select an external replication destination, such as **BigQuery**
-4. Click **Enable external replication**
+3. Select an external replication (ETL) destination, such as **BigQuery**
+4. Click **Enable external replication (ETL)**
<Image
- alt="Enable external replication"
+ alt="Enable external replication (ETL)"
src="/docs/img/database/replication/replication-enable-replication.png"
width={3560}
height={2156}
@@ -151,7 +151,7 @@ Before creating an external replication pipeline, enable external replication fo
### Step 3: Configure a destination
-Once external replication is enabled and you have a Postgres publication, configure a destination. The destination is where your replicated data will be stored, while the pipeline is the active Postgres replication process that continuously streams changes from your database to that destination.
+Once external replication (ETL) is enabled and you have a Postgres publication, configure a destination. The destination is where your replicated data will be stored, while the pipeline is the active Postgres replication process that continuously streams changes from your database to that destination.
#### Choose and configure your destination
@@ -190,14 +190,14 @@ Your replication pipeline now starts copying data from your database to your des
After creating a destination, the replication pipeline starts and appears in the destinations list. You can monitor the pipeline's status and performance from the Dashboard.
<Image
- alt="External replication destinations list"
+ alt="External replication (ETL) destinations list"
src="/docs/img/database/replication/replication-destinations-list.png"
width={3560}
height={2146}
/>
-For comprehensive monitoring instructions including pipeline states, metrics, and logs, see the [external replication monitoring guide](/docs/guides/database/replication/external-replication-monitoring).
+For comprehensive monitoring instructions including pipeline states, metrics, and logs, see the [external replication (ETL) monitoring guide](/docs/guides/database/replication/external-replication-monitoring).
### Managing your pipeline
@@ -219,18 +219,18 @@ Available actions:
- **Edit destination**: Modify destination settings like credentials or advanced options
- **Delete**: Remove the destination and permanently stop replication
-### Disabling external replication
+### Disabling external replication (ETL) [#disabling-external-replication]
-To turn off external replication for a project, delete all external replication pipelines first. After all pipelines are removed, open the three-dot actions menu on the Replication page and click **Disable external replication**.
+To turn off external replication (ETL) for a project, delete all replication pipelines first. After all pipelines are removed, open the three-dot actions menu on the Replication page and click **Disable external replication (ETL)**.
<Image
- alt="Disable external replication from the Replication page actions menu"
+ alt="Disable external replication (ETL) from the Replication page actions menu"
src="/docs/img/database/replication/replication-disable-external-replication.png"
width={3560}
height={2156}
/>
-For cleanup details, see [What happens when you disable external replication?](/docs/guides/database/replication/external-replication-faq#what-happens-when-you-disable-external-replication).
+For cleanup details, see [What happens when you disable external replication (ETL)?](/docs/guides/database/replication/external-replication-faq#what-happens-when-you-disable-external-replication).
### Adding or removing tables
@@ -272,7 +272,7 @@ If your Postgres publication uses `FOR ALL TABLES` or `FOR TABLES IN SCHEMA`, ne
<Admonition type="note">
-When a table is deleted at the destination, the behavior depends on the destination. In general, the pipeline tries to recreate the table so replication can continue. To permanently delete a table, pause the pipeline first or remove it from the publication before deleting. See the [external replication FAQ](/docs/guides/database/replication/external-replication-faq#what-happens-if-a-table-is-deleted-at-the-destination) for details.
+When a table is deleted at the destination, the behavior depends on the destination. In general, the pipeline tries to recreate the table so replication can continue. To permanently delete a table, pause the pipeline first or remove it from the publication before deleting. See the [external replication (ETL) FAQ](/docs/guides/database/replication/external-replication-faq#what-happens-if-a-table-is-deleted-at-the-destination) for details.
</Admonition>
@@ -282,13 +282,13 @@ Schema change support depends on the destination. BigQuery is currently the only
### How it works
-Once configured, external replication:
+Once configured, external replication (ETL):
1. **Captures** changes from your Postgres database using Postgres publications and logical replication
-2. **Streams** the changes through the replication pipeline
+2. **Streams** the changes through Supabase ETL
3. **Loads** the data to your destination
-External replication automatically optimizes how changes are delivered to the destination. The replication pipeline currently performs data extraction and loading only, without transformation - your data is replicated as-is to the destination.
+External replication (ETL) automatically optimizes how changes are delivered to the destination. Supabase ETL currently performs data extraction and loading only, without transformation - your data is replicated as-is to the destination.
### Troubleshooting
@@ -299,14 +299,16 @@ If you encounter issues during setup:
- **Pipeline failed to start**: Check the error message in the status view for specific details
- **No data being replicated**: Verify your Postgres publication includes the correct tables and event types
-For more troubleshooting help, see the [external replication FAQ](/docs/guides/database/replication/external-replication-faq).
+For more troubleshooting help, see the [external replication (ETL) FAQ](/docs/guides/database/replication/external-replication-faq).
### Limitations
-External replication has the following limitations:
+External replication (ETL) has the following limitations:
- **Primary keys required**: Tables must have primary keys (Postgres logical replication requirement)
-- **Custom data types**: Not supported
+- **Custom data types**: Custom values replicate as strings. Check that your destination can interpret those string values correctly.
+- **Generated columns**: Generated columns are skipped. Use triggers to store derived values in regular columns if you need them in the destination.
+- **Replica identity**: `REPLICA IDENTITY FULL` is strongly recommended. Updates and deletes need enough row data to apply changes correctly at the destination.
- **Schema changes**: Currently in beta and limited to BigQuery
- **No data transformation**: Data is replicated as-is without transformation
- **Data duplicates**: Duplicates can occur when stopping a pipeline if your database has transactions that take longer than a few minutes to complete. See [Can data duplicates occur during pipeline operations?](/docs/guides/database/replication/external-replication-faq#can-data-duplicates-occur-during-pipeline-operations) for details
@@ -316,5 +318,5 @@ Destination-specific limitations, such as BigQuery's row size limits, are docume
### Next steps
- [Set up BigQuery](/docs/guides/database/replication/bigquery)
-- [Monitor external replication](/docs/guides/database/replication/external-replication-monitoring)
-- [View external replication FAQ](/docs/guides/database/replication/external-replication-faq)
+- [Monitor external replication (ETL)](/docs/guides/database/replication/external-replication-monitoring)
+- [View external replication (ETL) FAQ](/docs/guides/database/replication/external-replication-faq)
diff --git a/apps/docs/content/guides/database/replication/manual-replication-setup.mdx b/apps/docs/content/guides/database/replication/manual-replication-setup.mdx
index f2dccb25e8a..8d5076fedc2 100644
--- a/apps/docs/content/guides/database/replication/manual-replication-setup.mdx
+++ b/apps/docs/content/guides/database/replication/manual-replication-setup.mdx
@@ -6,11 +6,11 @@ subtitle: 'Set up replication with Airbyte, Estuary, Fivetran, and other tools.'
sidebar_label: 'Setting up'
---
-This guide covers setting up **manual logical replication** using external tools. If you prefer a managed solution, read the [external replication setup guide](/docs/guides/database/replication/external-replication-setup) instead.
+This guide covers setting up **manual logical replication** using your own tools. If you prefer a managed solution through Supabase ETL, read the [external replication (ETL) setup guide](/docs/guides/database/replication/external-replication-setup) instead.
<Admonition type="note">
-This guide is for replicating data to external systems using your own tools. For deploying read-only databases across multiple regions, see [read replicas](/docs/guides/platform/read-replicas) instead.
+This guide is for replicating data to destination systems using your own tools. For deploying read-only databases across multiple regions, see [read replicas](/docs/guides/platform/read-replicas) instead.
</Admonition>
@@ -21,7 +21,7 @@ To set up replication, the following is recommended:
- Instance size of XL or greater
- [IPv4 add-on](/docs/guides/platform/ipv4-address) enabled
-To create a replication slot, you will need to use the `postgres` user and follow the instructions in the [external replication setup guide](/docs/guides/database/postgres/setup-replication-external).
+To create a replication slot, you will need to use the `postgres` user and follow the instructions in the [logical replication example](/docs/guides/database/postgres/setup-replication-external).
<Admonition type="note">
@@ -29,7 +29,7 @@ If you are running Postgres 17 or higher, you can create a new user and grant th
</Admonition>
-If you are replicating to an external system and using any of the tools below, check their documentation first. Additional information is provided where the setup with Supabase can vary.
+If you are replicating to a destination system and using any of the tools below, check their documentation first. Additional information is provided where the setup with Supabase can vary.
<Tabs
scrollable
@@ -77,7 +77,7 @@ Materialize has the following [documentation](https://materialize.com/docs/sql/c
You can follow those steps with the following modifications:
-1. Follow the steps in the [external replication setup guide](/docs/guides/database/postgres/setup-replication-external) to create a publication slot
+1. Follow the steps in the [logical replication example](/docs/guides/database/postgres/setup-replication-external) to create a publication slot
</TabPanel>
diff --git a/apps/docs/content/guides/getting-started/features.mdx b/apps/docs/content/guides/getting-started/features.mdx
index 184be241615..8dcb43769e0 100644
--- a/apps/docs/content/guides/getting-started/features.mdx
+++ b/apps/docs/content/guides/getting-started/features.mdx
@@ -34,7 +34,7 @@ Encrypt sensitive data and store secrets using our Postgres extension, Supabase
### Replication
-Automatically replicate your database to external destinations like data warehouses and analytics platforms. Read [the external replication documentation](/docs/guides/database/replication/external-replication-setup) for more details.
+Automatically replicate your database to destination systems like data warehouses and analytics platforms with external replication (ETL), powered by Supabase ETL. Read [the documentation](/docs/guides/database/replication/external-replication-setup) for more details.
## Platform
diff --git a/apps/docs/content/guides/storage/analytics/creating-analytics-buckets.mdx b/apps/docs/content/guides/storage/analytics/creating-analytics-buckets.mdx
index 249b722cc4b..2dbb3f27a6a 100644
--- a/apps/docs/content/guides/storage/analytics/creating-analytics-buckets.mdx
+++ b/apps/docs/content/guides/storage/analytics/creating-analytics-buckets.mdx
@@ -13,7 +13,7 @@ Analytics buckets use [Apache Iceberg](https://iceberg.apache.org/), an open-tab
<Admonition type="note" title="About replication">
-Analytics Buckets are still available, but managed replication into Analytics Buckets through Supabase ETL is no longer supported. If you need managed replication today, use [external replication](/docs/guides/database/replication/external-replication-setup) with **BigQuery**. If you want to use Analytics Buckets, bring your own ingestion pipeline.
+Analytics Buckets are still available, but managed replication into Analytics Buckets through Supabase ETL is no longer supported. If you need managed replication today, use [external replication (ETL)](/docs/guides/database/replication/external-replication-setup) with **BigQuery**. If you want to use Analytics Buckets, bring your own ingestion pipeline.
</Admonition>
diff --git a/apps/docs/content/guides/storage/analytics/query-with-postgres.mdx b/apps/docs/content/guides/storage/analytics/query-with-postgres.mdx
index 938c6e1d5df..41c296a0999 100644
--- a/apps/docs/content/guides/storage/analytics/query-with-postgres.mdx
+++ b/apps/docs/content/guides/storage/analytics/query-with-postgres.mdx
@@ -9,7 +9,7 @@ This is made possible by the [Iceberg Foreign Data Wrapper](/docs/guides/databas
<Admonition type="note" title="About ingestion">
-Managed replication into Analytics Buckets through Supabase ETL is no longer supported. This guide assumes your Analytics Bucket is being populated by your own ingestion pipeline.
+Managed replication into Analytics Buckets through Supabase ETL is no longer supported. This guide assumes your Analytics Bucket is being populated by your own ingestion pipeline. If you need managed replication today, use [external replication (ETL)](/docs/guides/database/replication/external-replication-setup) with **BigQuery**.
</Admonition>