🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
feat(blob-storage): add an outbound host allowlist for blob storage endpoints (#18907)
Niklas Semmlerfb154e8f8785
diff --git a/.agents/skills/security-review/references/outbound-url-validation.md b/.agents/skills/security-review/references/outbound-url-validation.mdindex b24a567553..73cfc511ba 100644--- a/.agents/skills/security-review/references/outbound-url-validation.md+++ b/.agents/skills/security-review/references/outbound-url-validation.md@@ -152,17 +152,20 @@ Every outbound-URL surface MUST apply **all three** of:the AWS transport (VPC endpoints resolve to private IPs). When the list isset, that provider checks the host the transport calls:`bedrock-runtime.<region>.<partition>`, or the host of- `AWS_ENDPOINT_URL_BEDROCK_RUNTIME` / `AWS_ENDPOINT_URL` when set. Blob storage- has no positive host list. For external media S3 virtual-hosted addressing,- connect-time validation also trusts `{bucket}.{allowlisted endpoint host}`- for that bucket.+ `AWS_ENDPOINT_URL_BEDROCK_RUNTIME` / `AWS_ENDPOINT_URL` when set. For+ external media S3 virtual-hosted addressing, connect-time validation also+ trusts `{bucket}.{allowlisted endpoint host}` for that bucket. Blob storage+ accepts any bucket-shaped `{bucket}.{listed host}` (`allowedBucketHostParents`),+ so all blob integrations share one pooled agent policy. Those names still get+ the private-address block; only exact listed hosts skip it, because the+ parent zone's DNS or the bucket name decides where they resolve.- `LANGFUSE_OUTBOUND_BLOCKED_IP_SEGMENTS` adds CIDR ranges to the built-inprivate-address block for outbound URL checks that already run. Blobstorage stays opt-in, so these ranges do not by themselves turn blobvalidation on. Per-surface exemptions still win over those ranges.- Blob storage endpoint validation is **opt-in** today on self-hosted (thehelper is a no-op until the operator configures one of the exemption env- vars). There is a `TODO(next major)` in `blobStorageEndpointValidation.ts`+ vars or `LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS`). There is a `TODO(next major)` in `blobStorageEndpointValidation.ts`to flip the default; until then, do not rely on blob storage validationfor new surfaces — wire your own wrapper that defaults to strict.diff --git a/.env.dev.example b/.env.dev.exampleindex 4fd32efcfb..69c503c75d 100644--- a/.env.dev.example+++ b/.env.dev.example@@ -300,6 +300,11 @@ LANGFUSE_AI_FEATURES_PROJECT_ID=7a88fb47-b4e2-43b8-a06c-a5ce950dc53a# LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_HOST=localhost# LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IPS=127.0.0.1,::1# LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IP_SEGMENTS=127.0.0.0/8+# Self-hosted only: when set, blob storage endpoints must use one of these hosts.+# S3 exports without a custom endpoint need s3.<region>.amazonaws.com listed+# (region "auto" also needs s3.us-east-1.amazonaws.com). Host credentials via+# web identity or an assumed role also need sts.<region>.amazonaws.com.+# LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS=# Self-hosted only: exempt internal hosts/IPs for external media storage endpoints.# LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_WHITELISTED_HOST=localhost# LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_WHITELISTED_IPS=127.0.0.1,::1diff --git a/.env.prod.example b/.env.prod.exampleindex 5f722b7c00..2c4aba1274 100644--- a/.env.prod.example+++ b/.env.prod.example@@ -525,6 +525,11 @@ LANGFUSE_ENABLE_BLOB_STORAGE_FILE_LOG=true# LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_HOST=# LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IPS=# LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IP_SEGMENTS=+# Self-hosted only: when set, blob storage endpoints must use one of these hosts.+# S3 exports without a custom endpoint need s3.<region>.amazonaws.com listed+# (region "auto" also needs s3.us-east-1.amazonaws.com). Host credentials via+# web identity or an assumed role also need sts.<region>.amazonaws.com.+# LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS=# Self-hosted only: exempt internal hosts/IPs for external media storage endpoints.# LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_WHITELISTED_HOST=# LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_WHITELISTED_IPS=diff --git a/docker-compose.yml b/docker-compose.ymlindex fbb337f305..2608261fc8 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -35,6 +35,7 @@ services:# Empty defaults leave egress policy unchanged until an operator sets them.LANGFUSE_LLM_CONNECTION_ALLOWED_HOSTS: ${LANGFUSE_LLM_CONNECTION_ALLOWED_HOSTS:-}LANGFUSE_WEBHOOK_ALLOWED_HOSTS: ${LANGFUSE_WEBHOOK_ALLOWED_HOSTS:-}+ LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS: ${LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS:-}LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_ALLOWED_HOSTS: ${LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_ALLOWED_HOSTS:-}LANGFUSE_OUTBOUND_BLOCKED_IP_SEGMENTS: ${LANGFUSE_OUTBOUND_BLOCKED_IP_SEGMENTS:-}CLICKHOUSE_MIGRATION_URL: ${CLICKHOUSE_MIGRATION_URL:-clickhouse://clickhouse:9000}diff --git a/packages/shared/src/env.ts b/packages/shared/src/env.tsindex d08ea8611a..ac1d16cb05 100644--- a/packages/shared/src/env.ts+++ b/packages/shared/src/env.ts@@ -565,6 +565,19 @@ const EnvSchema = z.object({.transform((s) =>s ? s.split(",").map((s) => s.toLowerCase().trim()) : [],),+ // When non-empty, blob storage export endpoints must use one of these hosts,+ // and blob endpoint validation is on. Listed hosts may resolve to private+ // addresses. Bucket names under a listed host ({bucket}.{host}) are allowed+ // too, but still may not resolve to a private address unless it is in+ // LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IPS / _IP_SEGMENTS; private+ // S3-compatible stores can use path-style instead. S3 exports without a+ // custom endpoint connect to s3.<region>.amazonaws.com, which must then be+ // listed; region "auto" first connects to s3.us-east-1.amazonaws.com. S3+ // exports using host credentials (web identity or an assumed role) also+ // call sts.<region>.amazonaws.com through the same check, so list it too.+ // GCS exports always connect to Google and are not checked. Ignored on+ // Langfuse Cloud.+ LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS: optionalCommaSeparatedLowercase,LANGFUSE_EXTERNAL_MEDIA_STORAGE_ENDPOINT_WHITELISTED_IPS: z.string().optional()@@ -623,7 +636,8 @@ const EnvSchema = z.object({),// Extra CIDR ranges merged into the private-address block for outbound URL// checks that already run. Blob storage stays opt-in: these ranges apply- // there only when blob endpoint validation is already enabled. Exemptions+ // there only when a blob endpoint exemption or allowed-hosts list turns+ // blob endpoint validation on. Exemptions// in the per-surface *_WHITELISTED_* lists still win.LANGFUSE_OUTBOUND_BLOCKED_IP_SEGMENTS: optionalCommaSeparatedLowercase.refine((segments) => segments.every(isCidr),diff --git a/packages/shared/src/server/outbound-url/connection.ts b/packages/shared/src/server/outbound-url/connection.tsindex 814568a146..bb2b5bca7c 100644--- a/packages/shared/src/server/outbound-url/connection.ts+++ b/packages/shared/src/server/outbound-url/connection.ts@@ -264,5 +264,8 @@ function normalizeWhitelist(ips: [...whitelist.ips].sort(),ip_ranges: [...whitelist.ip_ranges].sort(),allowedHosts: [...(whitelist.allowedHosts ?? [])].sort(),+ allowedBucketHostParents: [+ ...(whitelist.allowedBucketHostParents ?? []),+ ].sort(),};}diff --git a/packages/shared/src/server/outbound-url/validation.ts b/packages/shared/src/server/outbound-url/validation.tsindex 9a38b2e8ce..976133aee8 100644--- a/packages/shared/src/server/outbound-url/validation.ts+++ b/packages/shared/src/server/outbound-url/validation.ts@@ -37,6 +37,10 @@ export class OutboundUrlValidationError extends Error {export const DNS_LOOKUP_FAILED_MESSAGE_PREFIX = "DNS lookup failed for";const HOST_NOT_IN_ALLOWLIST_MESSAGE ="Host is not in the outbound URL allowlist";+/** Bucket names the AWS SDK virtual-hosts as `{bucket}.{endpoint host}`. */+const DNS_COMPATIBLE_BUCKET_NAME = /^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/;+const IP_SHAPED_BUCKET_NAME = /(?:\d+\.){3}\d+/;+const ADJACENT_DOTS = /\.\./;export interface OutboundUrlValidationWhitelist {hosts: string[];@@ -45,6 +49,12 @@ export interface OutboundUrlValidationWhitelist {// Positive host list. Empty or omitted keeps the default-allow behavior:// any public host is accepted, and `hosts` only exempts private targets.allowedHosts?: string[];+ // Listed hosts whose S3 virtual-hosted names `{bucket}.{host}` also pass+ // the positive list. Unlike exact entries, those names still get the+ // private-address block: whoever controls the parent zone's DNS, or picks+ // the bucket name, decides where they resolve. Only read when+ // `allowedHosts` is non-empty.+ allowedBucketHostParents?: string[];}export interface ValidateOutboundUrlHostOptions {@@ -212,16 +222,36 @@ export function validateOutboundResolvedIp({function enforcePositiveAllowlist(hostname: string,whitelist: OutboundUrlValidationWhitelist,-): "allowed" | "not-configured" {+): "allowed" | "bucket-host" | "not-configured" {const allowedHosts = whitelist.allowedHosts ?? [];if (allowedHosts.length === 0) return "not-configured";if (allowedHosts.includes(hostname)) return "allowed";+ if (isBucketHostOfAllowedParent(hostname, whitelist)) return "bucket-host";throw new OutboundUrlValidationError("host-not-allowed",HOST_NOT_IN_ALLOWLIST_MESSAGE,);}+function isBucketHostOfAllowedParent(+ hostname: string,+ whitelist: OutboundUrlValidationWhitelist,+): boolean {+ return (whitelist.allowedBucketHostParents ?? []).some(+ (parent) =>+ hostname.endsWith(`.${parent}`) &&+ isDnsCompatibleBucketName(hostname.slice(0, -parent.length - 1)),+ );+}++export function isDnsCompatibleBucketName(bucketName: string): boolean {+ return (+ DNS_COMPATIBLE_BUCKET_NAME.test(bucketName) &&+ !IP_SHAPED_BUCKET_NAME.test(bucketName) &&+ !ADJACENT_DOTS.test(bucketName)+ );+}+function assertValidUrlEncoding(urlString: string): void {// This intentionally checks encoding validity only. Do not parse or validate// the decoded result: decoding the whole URL can turn encoded data into URLdiff --git a/packages/shared/src/server/services/blobStorageEndpointValidation.test.ts b/packages/shared/src/server/services/blobStorageEndpointValidation.test.tsindex a86bed11e9..cccacfd6af 100644--- a/packages/shared/src/server/services/blobStorageEndpointValidation.test.ts+++ b/packages/shared/src/server/services/blobStorageEndpointValidation.test.ts@@ -1,7 +1,14 @@-import { afterEach, describe, it, expect } from "vitest";-import { OutboundUrlValidationError, parseOutboundUrl } from "../outbound-url";+import dns from "node:dns/promises";+import { afterEach, describe, it, expect, vi } from "vitest";+import {+ OutboundUrlValidationError,+ parseOutboundUrl,+ validateOutboundResolvedIp,+} from "../outbound-url";import {assertGcsBlobStorageBucketAllowed,+ blobStorageEndpointConnectionValidationOptions,+ validateBlobStorageBucketHost,validateBlobStorageEndpoint,} from "./blobStorageEndpointValidation";import { env } from "../../env";@@ -62,6 +69,136 @@ describe("validateBlobStorageEndpoint", () => {}),).resolves.toBeUndefined();});++ it("rejects a host outside a configured allowed-hosts list", async () => {+ await expect(+ validateBlobStorageEndpoint("https://blob.example.com", {+ hosts: [],+ ips: [],+ ip_ranges: [],+ allowedHosts: ["minio.internal"],+ }),+ ).rejects.toMatchObject({+ code: "host-not-allowed",+ message: expect.stringContaining(+ "LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS",+ ),+ });+ });++ it("accepts a listed host even when it is a private address", async () => {+ await expect(+ validateBlobStorageEndpoint("http://10.0.0.1:9000", {+ hosts: [],+ ips: [],+ ip_ranges: [],+ allowedHosts: ["10.0.0.1"],+ }),+ ).resolves.toBeUndefined();+ });+});++describe("validateBlobStorageBucketHost", () => {+ const allowlist = {+ hosts: [],+ ips: [],+ ip_ranges: [],+ allowedHosts: ["minio.internal"],+ };+ const target = {+ type: "S3_COMPATIBLE" as const,+ endpoint: "http://minio.internal:9000",+ bucketName: "exports",+ forcePathStyle: false,+ };++ afterEach(() => {+ vi.restoreAllMocks();+ });++ it("rejects a virtual-hosted bucket name that resolves to a private address", async () => {+ vi.spyOn(dns, "resolve4").mockResolvedValue(["10.0.0.5"]);+ vi.spyOn(dns, "resolve6").mockRejectedValue(new Error("ENODATA"));+ vi.spyOn(dns, "lookup").mockResolvedValue([+ { address: "10.0.0.5", family: 4 },+ ] as never);++ await expect(+ validateBlobStorageBucketHost(target, allowlist),+ ).rejects.toMatchObject({+ code: "blocked-ip",+ message: expect.stringContaining("Force Path Style"),+ });+ });++ it.each([+ ["path-style is forced", { forcePathStyle: true }],+ ["the endpoint is an IP literal", { endpoint: "http://10.0.0.1:9000" }],+ ["a dotted bucket has a short label", { bucketName: "exports.v2" }],+ ])("skips the check when %s", async (_, override) => {+ const resolve4 = vi.spyOn(dns, "resolve4");++ await expect(+ validateBlobStorageBucketHost(+ { ...target, ...override },+ { ...allowlist, allowedHosts: ["minio.internal", "10.0.0.1"] },+ ),+ ).resolves.toBeUndefined();+ expect(resolve4).not.toHaveBeenCalled();+ });+});++describe("blobStorageEndpointConnectionValidationOptions", () => {+ const allowlist = {+ hosts: [],+ ips: [],+ ip_ranges: [],+ allowedHosts: ["minio.internal", "s3.us-east-1.amazonaws.com"],+ };+ const connectAs =+ (hostname: string, ip = "52.216.0.1") =>+ () =>+ validateOutboundResolvedIp({+ hostname,+ ip,+ whitelist:+ blobStorageEndpointConnectionValidationOptions(allowlist)!.whitelist!,+ logContext: "test",+ });++ it("is off when no list is configured", () => {+ expect(+ blobStorageEndpointConnectionValidationOptions({+ hosts: [],+ ips: [],+ ip_ranges: [],+ allowedHosts: [],+ }),+ ).toBeUndefined();+ });++ it("trusts virtual-hosted bucket names under a listed host", () => {+ expect(connectAs("exports.minio.internal")).not.toThrow();+ });++ it("keeps the private-address block for bucket names, but not for exact listed hosts", () => {+ expect(connectAs("minio.internal", "10.0.0.5")).not.toThrow();+ expect(connectAs("exports.minio.internal", "10.0.0.5")).toThrow(+ expect.objectContaining({ code: "blocked-ip" }),+ );+ });++ it("rejects names that are not a bucket under a listed host", () => {+ for (const hostname of [+ "exports.s3.eu-west-1.amazonaws.com",+ "evilminio.internal",+ "10.0.0.5.minio.internal",+ ]) {+ expect(connectAs(hostname)).toThrow(+ expect.objectContaining({ code: "host-not-allowed" }),+ );+ }+ });});describe("assertGcsBlobStorageBucketAllowed", () => {diff --git a/packages/shared/src/server/services/blobStorageEndpointValidation.ts b/packages/shared/src/server/services/blobStorageEndpointValidation.tsindex a5251fc071..a1be747778 100644--- a/packages/shared/src/server/services/blobStorageEndpointValidation.ts+++ b/packages/shared/src/server/services/blobStorageEndpointValidation.ts@@ -1,20 +1,28 @@/* eslint-disable @typescript-eslint/no-use-before-define */+import { type BlobStorageIntegrationType } from "@prisma/client";import { env } from "../../env";import {+ isDnsCompatibleBucketName,type OutboundUrlConnectionValidationOptions,type OutboundUrlValidationWhitelist,OutboundUrlValidationError,parseOutboundUrl,+ resolveHost,+ validateOutboundResolvedIp,validateOutboundUrlHost,} from "../outbound-url";+import { isIPAddress } from "../webhooks/ipBlocking";export const BLOB_STORAGE_ENDPOINT_VALIDATION_LOG_CONTEXT ="Blob storage endpoint";+const S3_BUCKET_LABEL = /^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$/;+const STRICT_BLOB_STORAGE_ENDPOINT_WHITELIST: OutboundUrlValidationWhitelist = {hosts: [],ips: [],ip_ranges: [],+ allowedHosts: [],};function isLangfuseCloudEndpointValidationEnabled(): boolean {@@ -33,6 +41,7 @@ export function blobStorageEndpointWhitelistFromEnv(): OutboundUrlValidationWhithosts: env.LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_HOST || [],ips: env.LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IPS || [],ip_ranges: env.LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IP_SEGMENTS || [],+ allowedHosts: env.LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS,};}@@ -83,13 +92,41 @@ export async function validateBlobStorageEndpoint(if (error instanceof OutboundUrlValidationError) {throw new OutboundUrlValidationError(error.code,- `${error.message}${getSelfHostedWhitelistGuidance()}`,+ `${error.message}${getSelfHostedWhitelistGuidance(error.code)}`,);}throw error;}}+export type BlobStorageBucketHostTarget = {+ type: BlobStorageIntegrationType;+ endpoint: string | null | undefined;+ bucketName: string;+ forcePathStyle: boolean | null | undefined;+};++/**+ * With an allowed-hosts list, a listed endpoint host may be private, but the+ * `{bucket}.{host}` name S3 connects to without path-style still gets the+ * private-address block. Check that name up front so the integration is+ * rejected with guidance instead of disabled after its first export.+ */+export async function validateBlobStorageBucketHost(+ target: BlobStorageBucketHostTarget,+ whitelist: OutboundUrlValidationWhitelist = blobStorageEndpointWhitelistFromEnv(),+): Promise<void> {+ const connectionWhitelist =+ blobStorageEndpointConnectionValidationOptions(whitelist)?.whitelist;+ const bucketHost = virtualHostedBucketHost(+ target,+ connectionWhitelist?.allowedHosts ?? [],+ );+ if (!connectionWhitelist || !bucketHost) return;++ await assertBucketHostAddressesAllowed(bucketHost, connectionWhitelist);+}+// Rejection of a keyless GCS export. The worker classifies it by name as a// bucket fault, so the integration is disabled instead of retried.class GcsBucketNotAllowedError extends Error {@@ -138,7 +175,15 @@ export function blobStorageEndpointConnectionValidationOptions(}return {- whitelist: effectiveWhitelist,+ // S3 connects to `{bucket}.{endpoint host}` unless path-style is forced,+ // and follows region redirects to `{bucket}.s3.<region>.amazonaws.com`.+ // A rule over all listed hosts, rather than the integration's own bucket,+ // keeps one pooled agent policy per deployment; the bucket name is the+ // project owner's choice either way.+ whitelist: {+ ...effectiveWhitelist,+ allowedBucketHostParents: effectiveWhitelist.allowedHosts,+ },logContext: BLOB_STORAGE_ENDPOINT_VALIDATION_LOG_CONTEXT,};}@@ -164,12 +209,92 @@ function isBlobStorageEndpointValidationEnabled(return (whitelist.hosts.length > 0 ||whitelist.ips.length > 0 ||- whitelist.ip_ranges.length > 0+ whitelist.ip_ranges.length > 0 ||+ (whitelist.allowedHosts?.length ?? 0) > 0);}-function getSelfHostedWhitelistGuidance(): string {+function getSelfHostedWhitelistGuidance(+ code: OutboundUrlValidationError["code"],+): string {if (isLangfuseCloudEndpointValidationEnabled()) return "";+ if (code === "host-not-allowed") {+ return " Add the host to LANGFUSE_BLOB_STORAGE_ENDPOINT_ALLOWED_HOSTS.";+ }+return " For self-hosted deployments with internal blob storage endpoints, configure LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_HOST, LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IPS, or LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IP_SEGMENTS.";}++/** virtualHostedBucketHost returns the `{bucket}.{host}` name the S3 SDK connects to, if it virtual-hosts this target under a listed host. */+function virtualHostedBucketHost(+ target: BlobStorageBucketHostTarget,+ allowedHosts: string[],+): string | undefined {+ if (+ allowedHosts.length === 0 ||+ !target.endpoint ||+ target.forcePathStyle ||+ target.type === "AZURE_BLOB_STORAGE" ||+ target.type === "GOOGLE_CLOUD_STORAGE"+ ) {+ return undefined;+ }++ const url = parseOutboundUrl(target.endpoint);+ const bucketName = target.bucketName.trim();+ if (+ !allowedHosts.includes(url.hostname) ||+ isIpLiteralHostname(url.hostname) ||+ !isSdkVirtualHostableBucket(bucketName, url.protocol === "https:")+ ) {+ return undefined;+ }+ return `${bucketName}.${url.hostname}`;+}++// The SDK addresses IP endpoints path-style.+function isIpLiteralHostname(hostname: string): boolean {+ return hostname.startsWith("[") || isIPAddress(hostname);+}++// Mirrors the S3 endpoint ruleset: over HTTPS a dotted bucket stays on the+// endpoint host so the certificate matches; over HTTP every label must be a+// 3-63 character bucket label.+function isSdkVirtualHostableBucket(+ bucketName: string,+ https: boolean,+): boolean {+ if (!isDnsCompatibleBucketName(bucketName)) return false;+ if (!bucketName.includes(".")) return true;+ return (+ !https &&+ bucketName.split(".").every((label) => S3_BUCKET_LABEL.test(label))+ );+}++async function assertBucketHostAddressesAllowed(+ bucketHost: string,+ whitelist: OutboundUrlValidationWhitelist,+): Promise<void> {+ // No hostname blocklist here: sockets are only checked by resolved address,+ // so names like `{bucket}.minio.internal` must not be rejected by suffix.+ try {+ for (const ip of await resolveHost(bucketHost)) {+ validateOutboundResolvedIp({+ hostname: bucketHost,+ ip,+ whitelist,+ logContext: BLOB_STORAGE_ENDPOINT_VALIDATION_LOG_CONTEXT,+ });+ }+ } catch (error) {+ if (error instanceof OutboundUrlValidationError) {+ throw new OutboundUrlValidationError(+ error.code,+ `${error.message} for ${bucketHost}. S3 connects to the bucket as a subdomain of the endpoint unless Force Path Style is on. Turn on Force Path Style, or add the address to LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IPS or LANGFUSE_BLOB_STORAGE_ENDPOINT_WHITELISTED_IP_SEGMENTS.`,+ );+ }+ throw error;+ }+}diff --git a/packages/shared/src/server/services/s3VirtualHostedAllowlist.ts b/packages/shared/src/server/services/s3VirtualHostedAllowlist.tsindex a8d1f2b4e3..ce79565cf6 100644--- a/packages/shared/src/server/services/s3VirtualHostedAllowlist.ts+++ b/packages/shared/src/server/services/s3VirtualHostedAllowlist.ts@@ -1,15 +1,11 @@/* eslint-disable @typescript-eslint/no-use-before-define */import { BLOB_STORAGE_REGION_REGEX } from "../../utils/stringChecks";import {+ isDnsCompatibleBucketName,type OutboundUrlValidationWhitelist,parseOutboundUrl,} from "../outbound-url";-/** Bucket names the AWS SDK virtual-hosts as `{bucket}.{endpoint host}`. */-const DNS_COMPATIBLE_BUCKET_NAME = /^[a-z0-9][a-z0-9.-]{1,61}[a-z0-9]$/;-const IP_SHAPED_BUCKET_NAME = /(?:\d+\.){3}\d+/;-const ADJACENT_DOTS = /\.\./;-export type S3VirtualHostTarget = {endpoint?: string | null;bucketName?: string | null;@@ -41,14 +37,6 @@ export function withS3VirtualHostedBucket(};}-function isDnsCompatibleBucketName(bucketName: string): boolean {- return (- DNS_COMPATIBLE_BUCKET_NAME.test(bucketName) &&- !IP_SHAPED_BUCKET_NAME.test(bucketName) &&- !ADJACENT_DOTS.test(bucketName)- );-}-function virtualHostedNames(target: S3VirtualHostTarget,bucketName: string,diff --git a/web/src/features/blobstorage-integration/blobstorage-integration-router.ts b/web/src/features/blobstorage-integration/blobstorage-integration-router.tsindex 8503c5c223..c59951d9a0 100644--- a/web/src/features/blobstorage-integration/blobstorage-integration-router.ts+++ b/web/src/features/blobstorage-integration/blobstorage-integration-router.ts@@ -23,6 +23,7 @@ import {QueueJobs,StorageServiceFactory,blobStorageEndpointConnectionValidationOptions,+ validateBlobStorageBucketHost,validateBlobStorageEndpoint,assertGcsBlobStorageBucketAllowed,} from "@langfuse/shared/src/server";@@ -393,6 +394,12 @@ export const blobStorageIntegrationRouter = createTRPCRouter({if (endpoint) {await validateBlobStorageEndpoint(endpoint);+ await validateBlobStorageBucketHost({+ type,+ endpoint,+ bucketName,+ forcePathStyle,+ });}const isGcs = type === BlobStorageIntegrationType.GOOGLE_CLOUD_STORAGE;// GCS: stored secret = customer service account JSON key; none = thediff --git a/web/src/features/blobstorage-integration/service.ts b/web/src/features/blobstorage-integration/service.tsindex c030f14037..45e29580ed 100644--- a/web/src/features/blobstorage-integration/service.ts+++ b/web/src/features/blobstorage-integration/service.ts@@ -16,6 +16,7 @@ import { encrypt } from "@langfuse/shared/encryption";import { env } from "@/src/env.mjs";import {assertGcsBlobStorageBucketAllowed,+ validateBlobStorageBucketHost,validateBlobStorageEndpoint,} from "@langfuse/shared/src/server";@@ -138,6 +139,12 @@ export async function upsertBlobStorageIntegration(params: {if (endpoint) {try {await validateBlobStorageEndpoint(endpoint);+ await validateBlobStorageBucketHost({+ type: data.type,+ endpoint,+ bucketName: data.bucketName,+ forcePathStyle: data.forcePathStyle,+ });} catch (error) {throw new InvalidRequestError(`Invalid blob storage endpoint: ${error instanceof Error ? error.message : "Endpoint validation failed"}`,diff --git a/worker/src/features/blobstorage/handleBlobStorageIntegrationProjectJob.ts b/worker/src/features/blobstorage/handleBlobStorageIntegrationProjectJob.tsindex b53649863a..6f2004028a 100644--- a/worker/src/features/blobstorage/handleBlobStorageIntegrationProjectJob.ts+++ b/worker/src/features/blobstorage/handleBlobStorageIntegrationProjectJob.ts@@ -30,6 +30,7 @@ import {createModelCache,blobStorageEndpointConnectionValidationOptions,assertGcsBlobStorageBucketAllowed,+ validateBlobStorageBucketHost,validateBlobStorageEndpoint,dispatchProjectNotification,} from "@langfuse/shared/src/server";@@ -1275,6 +1276,7 @@ export const handleBlobStorageIntegrationProjectJob = async (// DNS-rebinding defense for each SDK connection.if (blobStorageIntegration.endpoint) {await validateBlobStorageEndpoint(blobStorageIntegration.endpoint);+ await validateBlobStorageBucketHost(blobStorageIntegration);}// Process the export based on the integration configuration