Skip to content

The Postgres development platform. Supabase gives you a dedicated Postgres database to build your web, mobile, and AI applications.

feat(log drains): add s3 form FE-2101 (#41031)

Jordi Enricf17f049e5e24
diff --git a/apps/docs/content/guides/telemetry/log-drains.mdx b/apps/docs/content/guides/telemetry/log-drains.mdx
index b1eee214ef9..bbf6f5f876d 100644
--- a/apps/docs/content/guides/telemetry/log-drains.mdx
+++ b/apps/docs/content/guides/telemetry/log-drains.mdx
@@ -12,12 +12,13 @@ You can read about the initial announcement [here](/blog/log-drains) and vote fo
The following table lists the supported destinations and the required setup configuration:
-| Destination | Transport Method | Configuration |
-| --------------------- | ---------------- | ------------------------------------------------- |
-| Generic HTTP endpoint | HTTP | URL <br /> HTTP Version <br/> Gzip <br /> Headers |
-| DataDog | HTTP | API Key <br /> Region |
-| Loki | HTTP | URL <br /> Headers |
-| Sentry | HTTP | DSN |
+| Destination | Transport Method | Configuration |
+| --------------------- | ---------------- | -------------------------------------------------------------------------------------- |
+| Generic HTTP endpoint | HTTP | URL <br /> HTTP Version <br/> Gzip <br /> Headers |
+| Datadog | HTTP | API Key <br /> Region |
+| Loki | HTTP | URL <br /> Headers |
+| Sentry | HTTP | DSN |
+| Amazon S3 | AWS SDK | S3 Bucket <br/> Region <br/> Access Key ID <br/> Secret Access Key <br/> Batch Timeout |
HTTP requests are batched with a max of 250 logs or 1 second intervals, whichever happens first. Logs are compressed via Gzip if the destination supports it.
@@ -136,13 +137,13 @@ Deno.serve(async (req) => {
</Accordion>
-## DataDog logs
+## Datadog logs
-Logs sent to DataDog have the name of the log source set on the `service` field of the event and the source set to `Supabase`. Logs are gzipped before they are sent to DataDog.
+Logs sent to Datadog have the name of the log source set on the `service` field of the event and the source set to `Supabase`. Logs are gzipped before they are sent to Datadog.
The payload message is a JSON string of the raw log event, prefixed with the event timestamp.
-To setup DataDog log drain, generate a DataDog API key [here](https://app.datadoghq.com/organization-settings/api-keys) and the location of your DataDog site.
+To setup Datadog log drain, generate a Datadog API key [here](https://app.datadoghq.com/organization-settings/api-keys) and the location of your Datadog site.
<Accordion
type="default"
@@ -153,9 +154,9 @@ To setup DataDog log drain, generate a DataDog API key [here](https://app.datado
id="walkthrough"
>
- 1. Generate API Key in [DataDog dashboard](https://app.datadoghq.com/organization-settings/api-keys)
+ 1. Generate API Key in [Datadog dashboard](https://app.datadoghq.com/organization-settings/api-keys)
2. Create log drain in [Supabase dashboard](/dashboard/project/_/settings/log-drains)
- 3. Watch for events in the [DataDog Logs page](https://app.datadoghq.com/logs)
+ 3. Watch for events in the [Datadog Logs page](https://app.datadoghq.com/logs)
</AccordionItem>
@@ -211,6 +212,24 @@ All fields from the log event are attached as attributes to the Sentry log, whic
If you are self-hosting Sentry, Sentry Logs are only supported in self-hosted version [25.9.0](https://github.com/getsentry/self-hosted/releases/tag/25.9.0) and later.
+## Amazon S3
+
+Logs are written to an existing S3 bucket that you own.
+
+Required configuration when creating an S3 Log Drain:
+
+- S3 Bucket: the name of an existing S3 bucket.
+- Region: the AWS region where the bucket is located.
+- Access Key ID: used for authentication.
+- Secret Access Key: used for authentication.
+- Batch Timeout (ms): maximum time to wait before flushing a batch. Recommended 2000–5000ms.
+
+<Admonition type="note">
+
+Ensure the AWS account tied to the Access Key ID has permissions to write to the specified S3 bucket.
+
+</Admonition>
+
## Pricing
For a detailed breakdown of how charges are calculated, refer to [Manage Log Drain usage](/docs/guides/platform/manage-your-usage/log-drains).
diff --git a/apps/studio/components/interfaces/LogDrains/LogDrainDestinationSheetForm.tsx b/apps/studio/components/interfaces/LogDrains/LogDrainDestinationSheetForm.tsx
index f612b62df4f..e3fa362be18 100644
--- a/apps/studio/components/interfaces/LogDrains/LogDrainDestinationSheetForm.tsx
+++ b/apps/studio/components/interfaces/LogDrains/LogDrainDestinationSheetForm.tsx
@@ -91,6 +91,14 @@ const formUnion = z.discriminatedUnion('type', [
}),
z.object({
type: z.literal('s3'),
+ s3_bucket: z.string().min(1, { message: 'Bucket name is required' }),
+ storage_region: z.string().min(1, { message: 'Region is required' }),
+ access_key_id: z.string().min(1, { message: 'Access Key ID is required' }),
+ secret_access_key: z.string().min(1, { message: 'Secret Access Key is required' }),
+ batch_timeout: z.coerce
+ .number()
+ .int({ message: 'Batch timeout must be an integer' })
+ .min(1, { message: 'Batch timeout must be a positive integer' }),
}),
z.object({
type: z.literal('axiom'),
@@ -120,7 +128,6 @@ function LogDrainFormItem({
formControl,
placeholder,
type,
- defaultValue,
}: {
value: string
label: string
@@ -128,7 +135,6 @@ function LogDrainFormItem({
placeholder?: string
description?: ReactNode
type?: string
- defaultValue?: string
}) {
return (
<FormField_Shadcn_
@@ -137,12 +143,7 @@ function LogDrainFormItem({
render={({ field }) => (
<FormItemLayout layout="horizontal" label={label} description={description || ''}>
<FormControl_Shadcn_>
- <Input_Shadcn_
- defaultValue={defaultValue}
- type={type || 'text'}
- placeholder={placeholder}
- {...field}
- />
+ <Input_Shadcn_ type={type || 'text'} placeholder={placeholder} {...field} />
</FormControl_Shadcn_>
</FormItemLayout>
)}
@@ -178,6 +179,7 @@ export function LogDrainDestinationSheetForm({
const DEFAULT_HEADERS = mode === 'create' ? CREATE_DEFAULT_HEADERS : defaultConfig?.headers || {}
const sentryEnabled = useFlag('SentryLogDrain')
+ const s3Enabled = useFlag('S3logdrain')
const { ref } = useParams()
const { data: logDrains } = useLogDrainsQuery({
@@ -203,6 +205,11 @@ export function LogDrainDestinationSheetForm({
username: defaultConfig?.username || '',
password: defaultConfig?.password || '',
dsn: defaultConfig?.dsn || '',
+ s3_bucket: defaultConfig?.s3_bucket || '',
+ storage_region: defaultConfig?.storage_region || '',
+ access_key_id: defaultConfig?.access_key_id || '',
+ secret_access_key: defaultConfig?.secret_access_key || '',
+ batch_timeout: defaultConfig?.batch_timeout ?? 3000,
},
})
@@ -290,10 +297,7 @@ export function LogDrainDestinationSheetForm({
form.handleSubmit(onSubmit)(e)
track('log_drain_save_button_clicked', {
- destination: form.getValues('type') as Exclude<
- LogDrainType,
- 'elastic' | 'postgres' | 'bigquery' | 'clickhouse' | 's3' | 'axiom'
- >,
+ destination: form.getValues('type'),
})
}}
>
@@ -327,18 +331,20 @@ export function LogDrainDestinationSheetForm({
{LOG_DRAIN_TYPES.find((t) => t.value === type)?.name}
</SelectTrigger_Shadcn_>
<SelectContent_Shadcn_>
- {LOG_DRAIN_TYPES.filter((t) => t.value !== 'sentry' || sentryEnabled).map(
- (type) => (
- <SelectItem_Shadcn_
- value={type.value}
- key={type.value}
- id={type.value}
- className="text-left"
- >
- {type.name}
- </SelectItem_Shadcn_>
- )
- )}
+ {LOG_DRAIN_TYPES.filter(
+ (t) =>
+ (t.value !== 'sentry' || sentryEnabled) &&
+ (t.value !== 's3' || s3Enabled)
+ ).map((type) => (
+ <SelectItem_Shadcn_
+ value={type.value}
+ key={type.value}
+ id={type.value}
+ className="text-left"
+ >
+ {type.name}
+ </SelectItem_Shadcn_>
+ ))}
</SelectContent_Shadcn_>
</Select_Shadcn_>
</FormItemLayout>
@@ -517,6 +523,49 @@ export function LogDrainDestinationSheetForm({
/>
</div>
)}
+ {type === 's3' && (
+ <div className="grid gap-4 px-content">
+ <LogDrainFormItem
+ value="s3_bucket"
+ label="S3 Bucket"
+ placeholder="my-log-bucket"
+ formControl={form.control}
+ description="The name of an existing S3 bucket."
+ />
+ <LogDrainFormItem
+ value="storage_region"
+ label="Region"
+ placeholder="us-east-1"
+ formControl={form.control}
+ description="AWS region where the bucket is located."
+ />
+ <LogDrainFormItem
+ value="access_key_id"
+ label="Access Key ID"
+ placeholder="AKIA..."
+ formControl={form.control}
+ />
+ <LogDrainFormItem
+ type="password"
+ value="secret_access_key"
+ label="Secret Access Key"
+ placeholder="••••••••••••••••"
+ formControl={form.control}
+ />
+ <LogDrainFormItem
+ type="number"
+ value="batch_timeout"
+ label="Batch Timeout (ms)"
+ placeholder="3000"
+ formControl={form.control}
+ description="Recommended 2000–5000ms."
+ />
+ <p className="text-xs text-foreground-lighter">
+ Ensure the account tied to the Access Key ID can write to the specified
+ bucket.
+ </p>
+ </div>
+ )}
<FormMessage_Shadcn_ />
</div>
</form>
diff --git a/apps/studio/components/interfaces/LogDrains/LogDrains.constants.tsx b/apps/studio/components/interfaces/LogDrains/LogDrains.constants.tsx
index ea38250a168..49d11a49812 100644
--- a/apps/studio/components/interfaces/LogDrains/LogDrains.constants.tsx
+++ b/apps/studio/components/interfaces/LogDrains/LogDrains.constants.tsx
@@ -1,6 +1,6 @@
import { components } from 'api-types'
import { Datadog, Grafana, Sentry } from 'icons'
-import { BracesIcon } from 'lucide-react'
+import { BracesIcon, Cloud } from 'lucide-react'
const iconProps = {
height: 24,
@@ -30,6 +30,12 @@ export const LOG_DRAIN_TYPES = [
'Loki is an open-source log aggregation system designed to store and query logs from multiple sources',
icon: <Grafana {...iconProps} fill="currentColor" strokeWidth={0} />,
},
+ {
+ value: 's3',
+ name: 'Amazon S3',
+ description: 'Forward logs to an S3 bucket',
+ icon: <Cloud {...iconProps} />,
+ },
{
value: 'sentry',
name: 'Sentry',
diff --git a/apps/studio/components/interfaces/LogDrains/LogDrains.tsx b/apps/studio/components/interfaces/LogDrains/LogDrains.tsx
index 6bc48cfaa35..8eba92aba9c 100644
--- a/apps/studio/components/interfaces/LogDrains/LogDrains.tsx
+++ b/apps/studio/components/interfaces/LogDrains/LogDrains.tsx
@@ -56,6 +56,7 @@ export function LogDrains({
}
)
const sentryEnabled = useFlag('SentryLogDrain')
+ const s3Enabled = useFlag('S3logdrain')
const hasLogDrains = !!logDrains?.length
const { mutate: deleteLogDrain } = useDeleteLogDrainMutation({
@@ -90,7 +91,9 @@ export function LogDrains({
return (
<>
<div className="grid lg:grid-cols-2 gap-4">
- {LOG_DRAIN_TYPES.filter((t) => t.value !== 'sentry' || sentryEnabled).map((src) => (
+ {LOG_DRAIN_TYPES.filter(
+ (t) => (t.value !== 'sentry' || sentryEnabled) && (t.value !== 's3' || s3Enabled)
+ ).map((src) => (
<LogDrainsCard
key={src.value}
title={src.name}
@@ -195,10 +198,7 @@ export function LogDrains({
if (selectedLogDrain && ref) {
deleteLogDrain({ token: selectedLogDrain.token, projectRef: ref })
track('log_drain_confirm_button_submitted', {
- destination: selectedLogDrain.type as Exclude<
- LogDrainType,
- 'elastic' | 'postgres' | 'bigquery' | 'clickhouse' | 's3' | 'axiom'
- >,
+ destination: selectedLogDrain.type,
})
}
}}
diff --git a/packages/common/telemetry-constants.ts b/packages/common/telemetry-constants.ts
index c07fbd00e08..76b386b9fe0 100644
--- a/packages/common/telemetry-constants.ts
+++ b/packages/common/telemetry-constants.ts
@@ -2493,7 +2493,17 @@ export interface LogDrainSaveButtonClickedEvent {
/**
* Type of the destination saved
*/
- destination: 'webhook' | 'datadog' | 'loki' | 'sentry'
+ destination:
+ | 'postgres'
+ | 'bigquery'
+ | 'clickhouse'
+ | 'webhook'
+ | 'datadog'
+ | 'elastic'
+ | 'loki'
+ | 'sentry'
+ | 's3'
+ | 'axiom'
}
groups: TelemetryGroups
}
@@ -2511,7 +2521,17 @@ export interface LogDrainConfirmButtonSubmittedEvent {
/**
* Type of the destination confirmed
*/
- destination: 'webhook' | 'datadog' | 'loki' | 'sentry'
+ destination:
+ | 'postgres'
+ | 'bigquery'
+ | 'clickhouse'
+ | 'webhook'
+ | 'datadog'
+ | 'elastic'
+ | 'loki'
+ | 'sentry'
+ | 's3'
+ | 'axiom'
}
groups: TelemetryGroups
}
diff --git a/packages/ui-patterns/src/form/Layout/FormLayout.tsx b/packages/ui-patterns/src/form/Layout/FormLayout.tsx
index a92a60f449d..13319daf8ae 100644
--- a/packages/ui-patterns/src/form/Layout/FormLayout.tsx
+++ b/packages/ui-patterns/src/form/Layout/FormLayout.tsx
@@ -299,23 +299,16 @@ export const FormLayout = React.forwardRef<
) => {
const flex = layout === 'flex' || layout === 'flex-row-reverse'
const hasLabel = Boolean(label || beforeLabel || afterLabel)
- const renderError = isReactForm && !hideMessage && (
- <motion.div
- initial={{ opacity: 0, y: -10 }}
- animate={{ opacity: 1, y: 0 }}
- exit={{ opacity: 0, y: -10 }}
- transition={{ duration: 0.3, ease: 'easeInOut' }}
- className={cn(layout === 'flex-row-reverse' ? 'mt-0' : 'mt-2')}
- >
+ const renderError =
+ isReactForm && !hideMessage ? (
<FormMessage_Shadcn_
className={cn(
- 'mt-2 transition-opacity duration-300 ease-in-out',
+ 'mt-2 transition-all duration-300 ease-in-out',
layout === 'flex-row-reverse' && 'mt-0'
)}
- data-formlayout-id={'message'}
+ data-formlayout-id="message"
/>
- </motion.div>
- )
+ ) : null
const renderDescription =
description && isReactForm ? (