The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.
TypeScript aialternativeauthdatabasedenoembeddingsexamplefirebasenextjsoauth2pgvectorpostgispostgrespostgresqlpostgrestrealtimesupabasevectorswebsockets
fix(pipelines): clarify DuckLake destination setup (#51013)
Danny White614344149399
diff --git a/apps/docs/content/guides/database/replication/pipelines/ducklake.mdx b/apps/docs/content/guides/database/replication/pipelines/ducklake.mdxindex f2c31fcdc92..917f68bcaef 100644--- a/apps/docs/content/guides/database/replication/pipelines/ducklake.mdx+++ b/apps/docs/content/guides/database/replication/pipelines/ducklake.mdx@@ -18,7 +18,7 @@ Insert-only tables don't require a primary key or replica identity. Updates and## Prepare DuckLake resources [#understand-the-ducklake-components]-Prepare a Postgres catalog, object storage, and a compatible query engine:+DuckLake stores metadata in a Postgres catalog and data in object storage. With **Select Supabase projects**, choose a project for the catalog, then a project and bucket for storage. The catalog and storage can use the same project. Pipelines creates the connection credentials. With **Enter connection details**, you provide the catalog URL and storage credentials. You also need a compatible query engine to read the replicated data.| Component | Purpose || ---------------- | ----------------------------------------------------------------------------------------------------------------------------------- |@@ -30,56 +30,54 @@ You can query replicated tables, but treat them and their underlying catalog and## Configure DuckLake as a destination [#choose-a-configuration-mode]-Choose a mode below for its resource requirements and configuration steps.+Follow [Set up Pipelines](/docs/guides/database/replication/pipelines#setup-overview). In **Add pipeline**, select **DuckLake**, enter a **Pipeline name**, choose a **Publication**, and set **Initial sync**. Then choose how to configure the catalog and storage.-Follow [Set up Pipelines](/docs/guides/database/replication/pipelines#setup-overview), select **DuckLake**, then choose **Use Supabase** or **Custom parameters** for the catalog and storage.+### Select Supabase projects [#use-supabase]-### Use Supabase--Use this mode to back the DuckLake with Supabase projects. Pipelines provisions its own catalog and object-storage credentials when you create the destination.+Choose Supabase projects for the Postgres catalog and Storage bucket. Pipelines creates the connection credentials when you create the destination.Before you begin:- Choose active, healthy, non-branch projects from the same organization for the catalog and storage. You can use the same project for both.- Make sure your organization role can administer SQL in the catalog project and Storage in the storage project.-- Create a private standard Storage bucket, or create one from the destination form.-- Choose a metadata schema unique to this DuckLake. Use only letters, numbers, and underscores.+- Choose a Files bucket for DuckLake data. You can create a private bucket from the destination form.+- Choose an unused metadata schema name for this DuckLake. Pipelines creates the schema and its tables. Use only letters, numbers, and underscores. If the catalog project already uses the default `ducklake` schema for Warehouse, enter a different name.- Keep catalog and storage in the same region when possible, near the [managed pipeline region](/docs/guides/database/replication/pipelines#region).To configure the destination:-1. Select **Use Supabase**.-2. Choose the **Catalog project**, **Pool size**, and **Metadata schema**. Pool size allows `1` to `6` concurrent DuckDB connections; the default is `4`.-3. Choose the **Storage project** and private **Bucket**.-4. Click **Create and start pipeline** and complete the validation and cost confirmations.+1. Select **Select Supabase projects** under **Configuration method**.+2. Choose the **Catalog project** and **Metadata schema**.+3. Choose the **Storage project** and **Bucket**. Select **New bucket** at the bottom of the bucket list if you haven't created one yet.+4. Optionally adjust **Pool size** under **Advanced settings**. It allows 1 to 6 concurrent DuckDB connections; the default is 4.+5. Click **Start pipeline**. Review any validation warnings, then confirm the estimated cost with **Create and start pipeline**.Credential-provisioning warnings are expected before creation: catalog and Storage credentials are provisioned when you save the destination. Review the selected resources before proceeding.-### Custom parameters+### Enter connection details [#custom-parameters]-Use this mode with a Postgres catalog and S3-compatible object storage that you control.+Use this mode to connect an existing Postgres database and S3-compatible object storage. You provide their connection details and credentials.Prepare the following resources:-1. A Postgres database reachable from managed Pipelines. Create a dedicated user that can create and modify the DuckLake metadata schema and its tables.+1. A Postgres database reachable from managed Pipelines. Create a dedicated user with permission to create the DuckLake metadata schema and its tables.2. An S3-compatible bucket and a dedicated prefix for this DuckLake.3. Object-storage credentials that can list, read, write, and delete objects under that prefix. Delete access is required for managed file cleanup.-Use a new catalog metadata schema and data prefix for each destination. Reusing an existing schema or prefix can mix the metadata or files of different DuckLakes.+Choose an unused metadata schema name and a new data prefix for each destination. Pipelines creates the schema and its tables. Initialising a DuckLake catalog in that schema beforehand triggers a validation warning; reusing its schema or prefix can mix metadata or files from different DuckLakes.-Configure these fields in the destination form:+Select **Enter connection details** under **Configuration method**, then configure these fields:-- **Catalog URL**: A `postgres://` or `postgresql://` connection URL, including credentials and an `sslmode` appropriate for your provider+- **Catalog URL**: A `postgres://` or `postgresql://` URL for your existing database, including credentials. If your provider requires TLS, append `?sslmode=require` after the database name, or `&sslmode=require` if the URL already has query parameters. Keep the database name from your provider's URL; it doesn't need to be `ducklake_catalog`.- **Data path**: An `s3://<bucket>/<prefix>` URL-- **Pool size**: From `1` to `6`; the default is `4`- **S3 access key ID** and **S3 secret access key**: A credential pair for the data path+- **S3 endpoint**: A publicly reachable provider endpoint without `http://` or `https://`- **S3 region**: The storage provider's region-- **S3 endpoint**: The provider endpoint without `http://` or `https://`-- **S3 URL style**: `path` for Supabase Storage and many S3-compatible providers, or `vhost` for virtual-host-style addressing-- **Use SSL**: Keep enabled for production endpoints+- **S3 URL style**: Path style if the bucket is in the URL path, or virtual-host style if the bucket is in the hostname+- **Use SSL**: Choose **On** for HTTPS, or **Off** if your provider requires HTTP- **Metadata schema**: A unique Postgres schema for DuckLake metadata, using only letters, numbers, and underscores-Click **Create and start pipeline** and complete the validation and cost confirmations.+Optionally adjust **Pool size** under **Advanced settings**. It allows 1 to 6 concurrent DuckDB connections; the default is 4. Click **Start pipeline**, review the validation results, then confirm the estimated cost with **Create and start pipeline**.The catalog URL and storage credentials are stored as secrets and aren't returned after creation. When editing the destination, leave a secret field empty to keep its stored value, or enter a new value to replace it.@@ -95,7 +93,7 @@ A source `TRUNCATE` truncates the DuckLake table. A [table restart](/docs/guidesConnect DuckDB with its `ducklake` extension, or another compatible engine, to the same catalog and storage path. Query through the catalog; reading raw Parquet files can miss inlined changes, delete files, and the current snapshot.-For **Use Supabase** mode, create separate read credentials for the selected catalog and Storage projects. The writer credentials generated for Pipelines aren't exposed. For **Custom parameters**, use separate read-only credentials when your catalog and storage provider support them.+If you selected Supabase projects, create separate read credentials for the catalog and Storage projects. The writer credentials generated for Pipelines aren't exposed. If you entered connection details, use separate read-only credentials when your catalog and storage provider support them.After attaching the catalog under an alias such as `my_ducklake`, source schemas and tables are available as qualified DuckLake tables:@@ -163,14 +161,14 @@ For type changes, unsupported changes, and interrupted schema changes, see the s## Troubleshooting-| Issue | Resolution |-| ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- |-| Credential-provisioning warnings | Expected in [Use Supabase](#use-supabase) mode before saving. Review the selected resources. |-| Catalog or storage validation fails | Check [custom parameters](#custom-parameters), credentials, connectivity, and permissions for the configured prefix. Local `file://` paths are unsupported. |-| Metadata schema exists | Choose a new schema, unless intentionally reusing the same DuckLake and its corresponding data path. |-| Inserts work but updates or deletes fail | Check [replica identity and published columns](#source-table-requirements). |-| Queries omit changes or deleted rows | [Query through the catalog](#query-the-destination) with credentials for both catalog and storage. |-| A schema change fails | Review [supported changes](#schema-change-support). Don't modify catalog tables or files manually. |+| Issue | Resolution |+| ---------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| Credential-provisioning warnings | Expected when you [select Supabase projects](#use-supabase) before saving. Review the selected resources. |+| Catalog or storage validation fails | Check the [connection details](#custom-parameters), credentials, connectivity, and permissions for the configured prefix. Local `file://` paths are unsupported. |+| Metadata schema exists | Choose a new schema, unless intentionally reusing the same DuckLake and its corresponding data path. |+| Inserts work but updates or deletes fail | Check [replica identity and published columns](#source-table-requirements). |+| Queries omit changes or deleted rows | [Query through the catalog](#query-the-destination) with credentials for both catalog and storage. |+| A schema change fails | Review [supported changes](#schema-change-support). Don't modify catalog tables or files manually. |Use [pipeline monitoring](/docs/guides/database/replication/pipelines-monitoring) to inspect errors. For unresolved failures, [contact support](/dashboard/support/new) with the pipeline ID and error details.diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AdvancedSettings.tsx b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AdvancedSettings.tsxindex 4c706a8ca98..996cb4366ee 100644--- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AdvancedSettings.tsx+++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/AdvancedSettings.tsx@@ -23,6 +23,7 @@ import { DestinationType } from '../DestinationPanel.types'import { TableOptions } from './BigQuery/TableOptions'import {DEFAULT_CONNECTION_POOL_SIZE,+ DEFAULT_DUCKLAKE_POOL_SIZE,DEFAULT_MAX_COPY_CONNECTIONS_PER_TABLE,DEFAULT_MAX_FILL_MS,DEFAULT_MAX_TABLE_SYNC_WORKERS,@@ -55,11 +56,39 @@ export const AdvancedSettings = ({<div className="flex flex-col items-start gap-0.5"><span className="text-sm font-medium">Advanced settings</span><span className="text-sm text-foreground-lighter font-normal">- Customize how the pipeline syncs and replicates data.+ {type === 'DuckLake'+ ? 'Adjust catalog connections and replication settings.'+ : 'Customize how the pipeline syncs and replicates data.'}</span></div></AccordionTrigger><AccordionContent className="pb-0! pt-3 [&>div]:flex [&>div]:flex-col [&>div]:gap-y-4 [&>div]:px-5">+ {type === 'DuckLake' && (+ <FormField+ control={form.control}+ name="ducklakePoolSize"+ render={({ field }) => (+ <FormItemLayout+ layout="horizontal"+ label="Pool size"+ description="Maximum concurrent connections this pipeline opens to the catalog. Choose 1 to 6."+ >+ <FormControl>+ <Input+ {...field}+ type="number"+ min={1}+ max={6}+ value={field.value ?? ''}+ onChange={handleNumberChange(field)}+ placeholder={String(DEFAULT_DUCKLAKE_POOL_SIZE)}+ />+ </FormControl>+ </FormItemLayout>+ )}+ />+ )}+<FormFieldcontrol={form.control}name="maxFillMs"@@ -107,7 +136,7 @@ export const AdvancedSettings = ({step={1}value={field.value ?? ''}onChange={handleNumberChange(field)}- placeholder={`Default: ${DEFAULT_MAX_TABLE_SYNC_WORKERS}`}+ placeholder={String(DEFAULT_MAX_TABLE_SYNC_WORKERS)}/><InputGroupAddon align="inline-end"><InputGroupText>workers</InputGroupText>@@ -136,7 +165,7 @@ export const AdvancedSettings = ({step={1}value={field.value ?? ''}onChange={handleNumberChange(field)}- placeholder={`Default: ${DEFAULT_MAX_COPY_CONNECTIONS_PER_TABLE}`}+ placeholder={String(DEFAULT_MAX_COPY_CONNECTIONS_PER_TABLE)}/><InputGroupAddon align="inline-end"><InputGroupText>connections</InputGroupText>@@ -201,7 +230,7 @@ export const AdvancedSettings = ({step={1}value={field.value ?? ''}onChange={handleNumberChange(field)}- placeholder={`Default: ${DEFAULT_CONNECTION_POOL_SIZE}`}+ placeholder={String(DEFAULT_CONNECTION_POOL_SIZE)}/><InputGroupAddon align="inline-end"><InputGroupText>connections</InputGroupText>diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.test.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.test.tsindex 85d29ea363a..d21f4478114 100644--- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.test.ts+++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.test.ts@@ -311,6 +311,16 @@ const baseClickHouseFormData = {}describe('DestinationForm.utils DuckLake', () => {+ it('uses the default pool size when the advanced field is cleared', () => {+ const config = buildDestinationConfigForValidation({+ projectRef: 'project-ref',+ selectedType: 'DuckLake',+ data: { ...baseDucklakeFormData, ducklakePoolSize: '' },+ })++ expect(config).toMatchObject({ ducklake: { poolSize: undefined } })+ })+it('builds DuckLake validation config with required fields trimmed and blank optionals removed', () => {const config = buildDestinationConfigForValidation({projectRef: 'project-ref',diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.tsindex 4f3fb97e1e2..007201431d8 100644--- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.ts+++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationForm.utils.ts@@ -332,7 +332,7 @@ const buildDucklakeConfig = (catalogProjectRef: normalizeRequiredString(data.ducklakeCatalogProjectRef),storageProjectRef: normalizeRequiredString(data.ducklakeStorageProjectRef),bucket: normalizeRequiredString(data.ducklakeStorageBucket),- poolSize: data.ducklakePoolSize,+ poolSize: data.ducklakePoolSize === '' ? undefined : data.ducklakePoolSize,metadataSchema: normalizeOptionalString(data.ducklakeMetadataSchema),}return supabaseConfig@@ -341,7 +341,7 @@ const buildDucklakeConfig = (const manualConfig: DucklakeManualDestinationConfig = {catalogUrl: data.ducklakeCatalogUrl ?? '',dataPath: data.ducklakeDataPath ?? '',- poolSize: data.ducklakePoolSize,+ poolSize: data.ducklakePoolSize === '' ? undefined : data.ducklakePoolSize,s3AccessKeyId: normalizeRequiredString(data.ducklakeS3AccessKeyId),s3SecretAccessKey: normalizeRequiredString(data.ducklakeS3SecretAccessKey),s3Region: normalizeRequiredString(data.ducklakeS3Region),diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationFormFieldCopy.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationFormFieldCopy.tsindex 10fde3b4e7f..d264964e23e 100644--- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationFormFieldCopy.ts+++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DestinationFormFieldCopy.ts@@ -25,29 +25,29 @@ export const ANALYTICS_BUCKET_NAMESPACE_FIELD_COPY = {export const DUCKLAKE_CATALOG_PROJECT_FIELD_COPY = {label: 'Catalog project',- description:- "Pipelines connects to this project's Postgres instance to store the DuckLake catalog.",+ description: 'Postgres project that stores this DuckLake’s metadata.',} as constexport const DUCKLAKE_STORAGE_PROJECT_FIELD_COPY = {label: 'Storage project',- description: 'The project whose object storage holds the DuckLake data files.',+ description: 'Supabase project that stores the DuckLake data files.',} as constexport const DUCKLAKE_BUCKET_FIELD_COPY = {label: 'Bucket',- description: 'The bucket in which DuckLake data files will be stored.',+ description: 'Files bucket for DuckLake data.',} as constexport const DUCKLAKE_CATALOG_URL_FIELD_COPY = {label: 'Catalog URL',- createDescription: 'A Postgres connection string for the DuckLake catalog.',+ createDescription:+ 'Postgres URL for an existing database. Add TLS settings to the URL if required.',editDescription: 'Stored catalog URL is hidden. Enter a new URL to replace it.',} as constexport const DUCKLAKE_DATA_PATH_FIELD_COPY = {label: 'Data path',- description: 'An S3 path where DuckLake data files will be written.',+ description: 'S3 path for DuckLake data files.',} as constexport const SNOWFLAKE_ACCOUNT_ID_FIELD_COPY = {diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/DuckLake.schema.ts b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/DuckLake.schema.tsindex 50a9ace4a91..e78683d1e3e 100644--- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/DuckLake.schema.ts+++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/DuckLake.schema.ts@@ -12,10 +12,14 @@ export const DuckLakeFormSchema = z.object({ducklakeCatalogUrl: z.string().optional(),ducklakeDataPath: z.string().optional(),ducklakePoolSize: z- .number()- .int()- .min(1, 'Pool size must be greater than 0.')- .max(6, 'Pool size must be 6 or less.')+ .union([+ z.literal(''),+ z+ .number()+ .int()+ .min(1, 'Pool size must be greater than 0.')+ .max(6, 'Pool size must be 6 or less.'),+ ]).optional(),ducklakeS3AccessKeyId: z.string().optional(),ducklakeS3SecretAccessKey: z.string().optional(),diff --git a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/Fields.tsx b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/Fields.tsxindex ea5231e0743..7fedb790004 100644--- a/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/Fields.tsx+++ b/apps/studio/components/interfaces/Database/Replication/DestinationPanel/DestinationForm/DuckLake/Fields.tsx@@ -1,12 +1,21 @@-import { Check, Database, Eye, EyeOff, Plus, SlidersHorizontal } from 'lucide-react'+import { Check, Eye, EyeOff, Plus } from 'lucide-react'import { useMemo, useState } from 'react'import { useWatch, type UseFormReturn } from 'react-hook-form'import { toast } from 'sonner'import {Button,cn,+ ComboboxTrigger,+ Command,+ CommandEmpty,+ CommandGroup,+ CommandInput,+ CommandItem,+ CommandList,+ CommandSeparator,Dialog,DialogContent,+ DialogDescription,DialogFooter,DialogHeader,DialogSection,@@ -15,6 +24,12 @@ import {FormControl,FormField,Input,+ Popover,+ PopoverContent,+ PopoverTrigger,+ RadioGroupStacked,+ RadioGroupStackedItem,+ ScrollArea,Select,SelectContent,SelectGroup,@@ -26,7 +41,7 @@ import { Input as PasswordInput } from 'ui-patterns/DataInputs/Input'import { FormItemLayout } from 'ui-patterns/form/FormItemLayout/FormItemLayout'import { SelectionListState } from 'ui-patterns/SelectionListState'-import { DEFAULT_DUCKLAKE_POOL_SIZE, STORED_SECRET_PLACEHOLDER } from '../DestinationForm.constants'+import { STORED_SECRET_PLACEHOLDER } from '../DestinationForm.constants'import type { DestinationPanelSchemaType } from '../DestinationForm.schema'import {DUCKLAKE_BUCKET_FIELD_COPY,@@ -55,16 +70,15 @@ import { PROJECT_STATUS } from '@/lib/constants'const DUCKLAKE_MODE_OPTIONS = [{value: DUCKLAKE_MODE_SUPABASE,- icon: Database,- label: 'Use Supabase',+ label: 'Select Supabase projects',description:- 'Create or use a DuckLake backed by your Supabase projects. Catalog and storage are managed for you.',+ 'Choose projects for the Postgres catalog and Storage bucket. They can be the same project; Pipelines creates credentials.',},{value: DUCKLAKE_MODE_CUSTOM,- icon: SlidersHorizontal,- label: 'Custom parameters',- description: 'Bring your own Postgres catalog and S3-compatible object storage credentials.',+ label: 'Enter connection details',+ description:+ 'Provide a Postgres catalog URL and S3-compatible storage details and credentials.',},] as const@@ -76,44 +90,20 @@ const DuckLakeModeSelector = ({onChange: (value: DucklakeMode) => void}) => {return (- <div- role="radiogroup"- aria-label="DuckLake configuration mode"- className="grid grid-cols-2 gap-3"+ <RadioGroupStacked+ aria-label="Configuration method"+ value={value}+ onValueChange={(nextValue) => onChange(nextValue as DucklakeMode)}>- {DUCKLAKE_MODE_OPTIONS.map((option) => {- const Icon = option.icon- const selected = value === option.value- return (- <button- key={option.value}- type="button"- tabIndex={0}- role="radio"- aria-checked={selected}- onClick={() => onChange(option.value)}- className={cn(- 'relative flex flex-col gap-y-3 rounded-md border p-4 text-left transition',- 'hover:border-control-hover',- selected ? 'border-control-hover bg-surface-300' : 'border-default bg-surface-100'- )}- >- <div className="flex items-start justify-between">- <Icon size={18} strokeWidth={1.5} className="text-foreground-light" />- {selected ? (- <Check size={16} className="text-primary" />- ) : (- <span className="h-4 w-4 rounded-full border border-strong" />- )}- </div>- <div className="flex flex-col gap-y-1">- <span className="text-sm text-foreground">{option.label}</span>- <span className="text-xs text-foreground-light">{option.description}</span>- </div>- </button>- )- })}- </div>+ {DUCKLAKE_MODE_OPTIONS.map((option) => (+ <RadioGroupStackedItem+ key={option.value}+ value={option.value}+ label={option.label}+ description={option.description}+ />+ ))}+ </RadioGroupStacked>)}@@ -146,6 +136,11 @@ const DuckLakeSupabaseFields = ({ form }: { form: UseFormReturn<DestinationPanel() => new Map(projects.map((project) => [project.ref, project])),[projects])+ const storageProjectName =+ projectsByRef.get(ducklakeStorageProjectRef ?? '')?.name ??+ (ducklakeStorageProjectRef === sourceProject?.ref+ ? sourceProject?.name+ : ducklakeStorageProjectRef)const regionForRef = (ref?: string) => {if (!ref) return undefined@@ -194,7 +189,7 @@ const DuckLakeSupabaseFields = ({ form }: { form: UseFormReturn<DestinationPanel<div className="flex flex-col gap-y-1"><p className="text-sm font-medium text-foreground">Catalog</p><p className="text-sm text-foreground-light">- The selected project's Postgres database is used as the DuckLake catalog.+ DuckLake metadata is stored in the selected project’s Postgres database.</p></div>@@ -223,31 +218,6 @@ const DuckLakeSupabaseFields = ({ form }: { form: UseFormReturn<DestinationPanel)}/>- <FormField- control={form.control}- name="ducklakePoolSize"- render={({ field }) => (- <FormItemLayout- layout="horizontal"- label="Pool size"- description="Number of concurrent DuckDB connections to the catalog."- >- <FormControl>- <Input- type="number"- min={1}- max={6}- value={field.value ?? ''}- placeholder={`Default: ${DEFAULT_DUCKLAKE_POOL_SIZE}`}- onChange={(event) =>- field.onChange(event.target.value === '' ? undefined : Number(event.target.value))- }- />- </FormControl>- </FormItemLayout>- )}- />-<FormFieldcontrol={form.control}name="ducklakeMetadataSchema"@@ -255,7 +225,7 @@ const DuckLakeSupabaseFields = ({ form }: { form: UseFormReturn<DestinationPanel<FormItemLayoutlayout="horizontal"label="Metadata schema"- description="Schema used for DuckLake metadata tables in the catalog's Postgres."+ description="New schema where DuckLake metadata will be stored."><FormControl><Input {...field} placeholder="ducklake" value={field.value ?? ''} />@@ -309,29 +279,25 @@ const DuckLakeSupabaseFields = ({ form }: { form: UseFormReturn<DestinationPanellabel={DUCKLAKE_BUCKET_FIELD_COPY.label}description={DUCKLAKE_BUCKET_FIELD_COPY.description}>- <div className="flex items-center gap-x-2">- <div className="grow">- <FormControl>- <BucketSelection form={form} value={field.value} onChange={field.onChange} />- </FormControl>- </div>- <Button- type="button"- icon={<Plus />}- disabled={!ducklakeStorageProjectRef}- onClick={() => setShowNewBucketDialog(true)}- >- New bucket- </Button>- </div>+ <FormControl>+ <BucketSelection+ form={form}+ value={field.value}+ onChange={field.onChange}+ onCreateBucket={() => setShowNewBucketDialog(true)}+ />+ </FormControl></FormItemLayout>)}/><Dialog open={showNewBucketDialog} onOpenChange={setShowNewBucketDialog}>- <DialogContent>+ <DialogContent size="small"><DialogHeader>- <DialogTitle>Create a new file bucket</DialogTitle>+ <DialogTitle>New bucket</DialogTitle>+ <DialogDescription>+ Creates a private bucket in the selected Storage project ({storageProjectName}).+ </DialogDescription></DialogHeader><DialogSectionSeparator /><DialogSection className="flex flex-col gap-y-2">@@ -409,7 +375,7 @@ const DuckLakeCustomFields = ({placeholder={editMode? STORED_SECRET_PLACEHOLDER- : 'postgres://user:pass@host:5432/ducklake_catalog'+ : 'postgresql://user:password@host:5432/database'}onChange={(event) => field.onChange(event.target.value)}actions={@@ -445,39 +411,12 @@ const DuckLakeCustomFields = ({</FormItemLayout>)}/>-- <FormField- control={form.control}- name="ducklakePoolSize"- render={({ field }) => (- <FormItemLayout- layout="horizontal"- label="Pool size"- description="Number of concurrent DuckDB connections to use."- >- <FormControl>- <Input- type="number"- min={1}- max={6}- value={field.value ?? ''}- placeholder={`Default: ${DEFAULT_DUCKLAKE_POOL_SIZE}`}- onChange={(event) =>- field.onChange(- event.target.value === '' ? undefined : Number(event.target.value)- )- }- />- </FormControl>- </FormItemLayout>- )}- /></div><div className="flex flex-col gap-y-1"><p className="text-sm font-medium text-foreground">Object storage</p><p className="text-sm text-foreground-light">- Optional credentials and endpoint settings for S3-compatible storage providers.+ Connection settings and credentials for your S3-compatible object storage.</p></div>@@ -498,8 +437,13 @@ const DuckLakeCustomFields = ({<FormControl><Input{...field}- placeholder={editMode ? STORED_SECRET_PLACEHOLDER : 'my-access-key'}+ placeholder={editMode ? STORED_SECRET_PLACEHOLDER : undefined}value={field.value ?? ''}+ autoComplete="off"+ data-1p-ignore+ data-lpignore="true"+ data-form-type="other"+ data-bwignore/></FormControl></FormItemLayout>@@ -518,41 +462,49 @@ const DuckLakeCustomFields = ({? 'Stored secret access key is hidden. Enter a new secret to replace it.': 'Required secret access key for the object storage provider.'}- className="relative"><FormControl>- <Input+ <PasswordInput{...field}type={showSecretAccessKey && !editMode ? 'text' : 'password'}- placeholder={editMode ? STORED_SECRET_PLACEHOLDER : 'my-secret-key'}+ placeholder={editMode ? STORED_SECRET_PLACEHOLDER : undefined}value={field.value ?? ''}- />- </FormControl>- {!editMode && (- <Button- aria-label={- showSecretAccessKey ? 'Hide secret access key' : 'Show secret access key'+ autoComplete="off"+ actions={+ !editMode && (+ <Button+ className="w-7"+ aria-label={+ showSecretAccessKey+ ? 'Hide S3 secret access key'+ : 'Show S3 secret access key'+ }+ icon={showSecretAccessKey ? <Eye /> : <EyeOff />}+ onClick={() => setShowSecretAccessKey(!showSecretAccessKey)}+ />+ )}- icon={showSecretAccessKey ? <Eye /> : <EyeOff />}- className="w-7 absolute right-6 top-[4px]"- onClick={() => setShowSecretAccessKey(!showSecretAccessKey)}/>- )}+ </FormControl></FormItemLayout>)}/><FormFieldcontrol={form.control}- name="ducklakeS3Region"+ name="ducklakeS3Endpoint"render={({ field }) => (<FormItemLayoutlayout="horizontal"- label="S3 region"- description="Required region for the object storage provider."+ label="S3 endpoint"+ description="Public address of your storage provider, without the HTTP or HTTPS prefix."><FormControl>- <Input {...field} placeholder="us-east-1" value={field.value ?? ''} />+ <Input+ {...field}+ placeholder="s3.us-east-1.amazonaws.com"+ value={field.value ?? ''}+ /></FormControl></FormItemLayout>)}@@ -560,15 +512,15 @@ const DuckLakeCustomFields = ({<FormFieldcontrol={form.control}- name="ducklakeS3Endpoint"+ name="ducklakeS3Region"render={({ field }) => (<FormItemLayoutlayout="horizontal"- label="S3 endpoint"- description="Required endpoint without the protocol scheme, for example `127.0.0.1:5000/s3`."+ label="S3 region"+ description="Required region for the object storage provider."><FormControl>- <Input {...field} placeholder="127.0.0.1:5000/s3" value={field.value ?? ''} />+ <Input {...field} placeholder="us-east-1" value={field.value ?? ''} /></FormControl></FormItemLayout>)}@@ -581,14 +533,26 @@ const DuckLakeCustomFields = ({<FormItemLayoutlayout="horizontal"label="S3 URL style"- description="Choose `path` for MinIO/Supabase-style endpoints or `vhost` for AWS-style virtual host addressing."+ description="Controls where the bucket name appears in requests to your storage provider."><FormControl><Select value={field.value ?? 'path'} onValueChange={field.onChange}>- <SelectTrigger>{field.value ?? 'path'}</SelectTrigger>+ <SelectTrigger>+ {field.value === 'vhost' ? 'Virtual-host style' : 'Path style'}+ </SelectTrigger><SelectContent>- <SelectItem value="path">path</SelectItem>- <SelectItem value="vhost">vhost</SelectItem>+ <SelectItem value="path" className="[&>span]:top-2.5">+ <p>Path style</p>+ <p className="text-foreground-lighter">+ Bucket name appears in the URL path.+ </p>+ </SelectItem>+ <SelectItem value="vhost" className="[&>span]:top-2.5">+ <p>Virtual-host style</p>+ <p className="text-foreground-lighter">+ Bucket name appears in the hostname.+ </p>+ </SelectItem></SelectContent></Select></FormControl>@@ -603,17 +567,25 @@ const DuckLakeCustomFields = ({<FormItemLayoutlayout="horizontal"label="Use SSL"- description="Whether to use SSL when connecting to the S3-compatible endpoint."+ description="Controls whether connections to your storage provider use HTTPS."><FormControl><Selectvalue={field.value === false ? 'false' : 'true'}onValueChange={(value) => field.onChange(value === 'true')}>- <SelectTrigger>{field.value === false ? 'false' : 'true'}</SelectTrigger>+ <SelectTrigger>{field.value === false ? 'Off' : 'On'}</SelectTrigger><SelectContent>- <SelectItem value="true">true</SelectItem>- <SelectItem value="false">false</SelectItem>+ <SelectItem value="true" className="[&>span]:top-2.5">+ <p>On</p>+ <p className="text-foreground-lighter">Encrypts the connection with HTTPS.</p>+ </SelectItem>+ <SelectItem value="false" className="[&>span]:top-2.5">+ <p>Off</p>+ <p className="text-foreground-lighter">+ Uses HTTP if your provider requires it.+ </p>+ </SelectItem></SelectContent></Select></FormControl>@@ -637,7 +609,11 @@ const DuckLakeCustomFields = ({<FormItemLayoutlayout="horizontal"label="Metadata schema"- description="Schema used for DuckLake metadata tables in Postgres."+ description={+ editMode+ ? 'Schema containing this destination’s DuckLake metadata tables.'+ : 'New schema where DuckLake metadata will be stored.'+ }><FormControl><Input {...field} placeholder="ducklake" value={field.value ?? ''} />@@ -666,20 +642,19 @@ export const DuckLakeFields = ({return (<div className="flex flex-col gap-y-6 p-5">- <p className="text-sm font-medium text-foreground">DuckLake settings</p>-- {!editMode && (- <div className="flex flex-col gap-y-3">- <p className="text-xs uppercase tracking-wider text-foreground-lighter">- How should this DuckLake be configured?- </p>- <DuckLakeModeSelector- value={effectiveMode}- onChange={(value) =>- form.setValue('ducklakeMode', value, { shouldValidate: true, shouldDirty: true })- }- />- </div>+ {editMode ? (+ <p className="text-sm font-medium text-foreground">DuckLake settings</p>+ ) : (+ <FormItemLayout layout="horizontal" label="Configuration method">+ <FormControl>+ <DuckLakeModeSelector+ value={effectiveMode}+ onChange={(value) =>+ form.setValue('ducklakeMode', value, { shouldValidate: true, shouldDirty: true })+ }+ />+ </FormControl>+ </FormItemLayout>)}{effectiveMode === DUCKLAKE_MODE_SUPABASE ? (@@ -776,11 +751,15 @@ const BucketSelection = ({form,value,onChange,+ onCreateBucket,}: {form: UseFormReturn<DestinationPanelSchemaType>value: string | undefinedonChange: (value: string) => void+ onCreateBucket: () => void}) => {+ const [searchTerm, setSearchTerm] = useState('')+ const [isDropdownOpen, setIsDropdownOpen] = useState(false)const ducklakeStorageProjectRef = useWatch({control: form.control,name: 'ducklakeStorageProjectRef',@@ -805,6 +784,10 @@ const BucketSelection = ({[bucketsData])const isBucketsErrorVisible = isMetadataListErrorVisible(isErrorBuckets, buckets.length)+ const isBucketsLoading = isMetadataListLoading(+ isPendingBuckets || isFetchingBuckets,+ buckets.length+ )const { handleOpenChange: handleRefreshBucketsOnOpen } = useRefreshOnOpen({isEnabled: !!ducklakeStorageProjectRef,refetch: refetchBuckets,@@ -812,41 +795,94 @@ const BucketSelection = ({if (!ducklakeStorageProjectRef) {return (- <Select disabled>- <SelectTrigger>Select a storage project first</SelectTrigger>- </Select>+ <ComboboxTrigger size="small" disabled>+ Select a storage project first+ </ComboboxTrigger>)}return (- <Select- value={value || ''}- onOpenChange={handleRefreshBucketsOnOpen}- onValueChange={(e) => {- if (e) onChange(e)+ <Popover+ modal={false}+ open={isDropdownOpen}+ onOpenChange={(open) => {+ setIsDropdownOpen(open)+ handleRefreshBucketsOnOpen(open)+ if (!open) setSearchTerm('')}}>- <SelectTrigger>{value || 'Select a bucket'}</SelectTrigger>- <SelectContent>- <SelectGroup>- <SelectionListState- isLoading={isMetadataListLoading(isPendingBuckets || isFetchingBuckets, buckets.length)}- isError={isBucketsErrorVisible}- isEmpty={- !isMetadataListLoading(isPendingBuckets || isFetchingBuckets, buckets.length) &&- !isBucketsErrorVisible &&- buckets.length === 0- }- emptyLabel="No buckets available"- errorLabel="Unable to load buckets"+ <PopoverTrigger asChild>+ <ComboboxTrigger+ size="small"+ aria-expanded={isDropdownOpen}+ data-state={isDropdownOpen ? 'open' : 'closed'}+ className={cn(!value && 'text-foreground-muted')}+ >+ {value || 'Select a bucket'}+ </ComboboxTrigger>+ </PopoverTrigger>+ <PopoverContent sameWidthAsTrigger className="p-0" align="start" side="bottom">+ <Command>+ <CommandInput+ placeholder="Find bucket..."+ className="text-xs"+ value={searchTerm}+ onValueChange={setSearchTerm}/>- {buckets.map((bucket) => (- <SelectItem key={bucket.id} value={bucket.id}>- {bucket.name}- </SelectItem>- ))}- </SelectGroup>- </SelectContent>- </Select>+ <CommandList>+ {!isBucketsLoading && !isBucketsErrorVisible && buckets.length > 0 && (+ <CommandEmpty>No buckets found</CommandEmpty>+ )}+ <SelectionListState+ isLoading={isBucketsLoading}+ isError={isBucketsErrorVisible}+ isEmpty={!isBucketsLoading && !isBucketsErrorVisible && buckets.length === 0}+ emptyLabel="No buckets available"+ errorLabel="Unable to load buckets"+ skeletonVariant="command"+ />+ {buckets.length > 0 && (+ <CommandGroup>+ <ScrollArea+ className={buckets.length > 7 ? 'h-[210px]' : ''}+ onWheel={(event) => event.stopPropagation()}+ >+ {buckets.map((bucket) => (+ <CommandItem+ key={bucket.id}+ value={bucket.name}+ className="cursor-pointer flex items-center justify-between gap-x-2 w-full"+ onSelect={() => {+ onChange(bucket.id)+ setIsDropdownOpen(false)+ }}+ >+ <span>{bucket.name}</span>+ {value === bucket.id && (+ <Check className="text-primary" strokeWidth={2} size={13} />+ )}+ </CommandItem>+ ))}+ </ScrollArea>+ </CommandGroup>+ )}+ <CommandSeparator />+ <CommandGroup forceMount>+ <CommandItem+ forceMount+ className="cursor-pointer w-full"+ onSelect={() => {+ setIsDropdownOpen(false)+ onCreateBucket()+ }}+ >+ <Plus size={14} strokeWidth={1.5} className="mr-2" />+ New bucket+ </CommandItem>+ </CommandGroup>+ </CommandList>+ </Command>+ </PopoverContent>+ </Popover>)}