Skip to content

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

feat(ai-gateway): add an analytics tab to the AI Gateway settings (#19417)

Tobias Wochinger07548ebfa5da
diff --git a/.agents/skills/seed-test-data/SKILL.md b/.agents/skills/seed-test-data/SKILL.md
index 29d7acbf0f..a6d1dd7fcf 100644
--- a/.agents/skills/seed-test-data/SKILL.md
+++ b/.agents/skills/seed-test-data/SKILL.md
@@ -48,6 +48,7 @@ this.
| Session messages carrying Langfuse media references (inline image, several refs in one message, bare refs in a content array, link-only payload) | `pnpm run seed -- session-shapes --shape media` — uploads the image/audio/pdf fixtures to MinIO and links them to the observation, so the inline chip and the "Media" strip both resolve (LFE-14815, LFE-9577) |
| Many traces for list/filter performance | `pnpm run seed -- many-traces --count 100000 --days 14` |
| Long-window v4 traffic with cost/latency/token OUTLIERS (outlier chart strip, LFE-14451) | `pnpm run seed -- outlier-traffic --days 90` (diurnal base load + deterministic spikes + hour-long ×8-latency incidents; root AGENT + GENERATION carrying cost + TOOL per trace; v4 on by default) |
+| AI Gateway traffic for the org settings Analytics tab (cost per user / API key / model) | `pnpm run seed -- gateway-traffic --days 30` (writes into the org's gateway ingestion project; creates gateway keys bound to org members up to `--keys` plus one service key bound to no user, one GENERATION per request attributed like gateway ingestion does (`service_name = langfuse-ai-gateway`, gateway key id in `ingestion_api_key`), plus a few SDK events the tab must exclude; v4 on by default) |
| Scores with spaces in the name (filter/grammar testing) | `pnpm run seed -- scored-traces --traces 24 --v4` |
| Custom model definitions reachable from a trace (price editor entry points) | `pnpm run seed -- custom-models --v4` (a tiered model with a condition-gated second tier and one usage type priced at 0, a single-tier model, and a generation whose model matches no definition so its badge opens the create dialog) |
| Many project-owned evaluators for gallery infinite-scroll testing | `NEXTAUTH_URL=https://pr-<N>.preview.langfuse.com pnpm run seed -- evaluator-gallery --count 200` (uses the default synthetic seed API key and reconciles deterministic evaluator names; also works against local seeded environments when `NEXTAUTH_URL` is omitted) |
diff --git a/packages/shared/scripts/seeder/scenarios/gateway-traffic.ts b/packages/shared/scripts/seeder/scenarios/gateway-traffic.ts
new file mode 100644
index 0000000000..b3cb81b469
--- /dev/null
+++ b/packages/shared/scripts/seeder/scenarios/gateway-traffic.ts
@@ -0,0 +1,469 @@
+import { prisma } from "../../../src/db";
+import {
+ OrganizationId,
+ SystemRoleId,
+ UserId,
+} from "../../../src/features/rbac/types";
+import {
+ createApiKey,
+ createEvent,
+ createEventsCh,
+ EventRecordInsertType,
+} from "../../../src/server";
+import {
+ AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY,
+ AI_GATEWAY_SERVICE_NAME,
+} from "../../../src/features/ai-gateway/constants";
+import {
+ AI_GATEWAY_INSTRUMENTATION_SCOPE_NAME,
+ AI_GATEWAY_KEY_ID_METADATA_KEY,
+} from "../../../src/server/otel/OtelIngestionProcessor";
+import { GEN_INPUT_PRICE, GEN_OUTPUT_PRICE } from "./payload";
+import { jitter, utcDayStartMs } from "./rng";
+import {
+ chunk,
+ ScenarioContext,
+ ScenarioDefinition,
+ SeedError,
+ SeedSummary,
+} from "./types";
+import { countRows, escapeLike } from "./verify";
+
+/**
+ * AI Gateway traffic in the organization's gateway ingestion project, built
+ * for the AI Gateway analytics settings tab. Each request is one root
+ * GENERATION event with the gateway service name, the gateway key id as its
+ * ingestion API key, and the key's bound user as user ID and owner metadata,
+ * the shape gateway ingestion writes. A few SDK events land in the same project; analytics must ignore
+ * them.
+ *
+ * Writes go to the ingestion project of the --project's organization, not to
+ * --project itself. Missing gateway keys are created (bound round-robin to
+ * organization members) until the organization has --keys of them. One extra
+ * service key, bound to no user, sends requests whose user ID the caller set.
+ */
+
+const MODELS = [
+ { name: "gpt-5.4-mini", rate: 1 },
+ { name: "claude-haiku-4-5", rate: 1.5 },
+ { name: "gpt-5.6-sol", rate: 6 },
+];
+
+const SDK_EVENTS_PER_DAY = 3;
+
+const ensureGatewayKeys = async (params: {
+ organizationId: string;
+ keyCount: number;
+ idPrefix: string;
+}) => {
+ const existing = await prisma.gatewayApiKeyAssociation.findMany({
+ where: {
+ apiKey: {
+ orgId: params.organizationId,
+ scope: "ORGANIZATION",
+ userId: { not: null },
+ },
+ },
+ select: { apiKey: { select: { id: true, userId: true } } },
+ orderBy: { apiKeyId: "asc" },
+ });
+ const keys = existing.map(({ apiKey }) => apiKey);
+ if (keys.length >= params.keyCount) return keys;
+
+ const members = await prisma.organizationMembership.findMany({
+ where: { orgId: params.organizationId },
+ select: { userId: true },
+ orderBy: { createdAt: "asc" },
+ });
+ if (members.length === 0) {
+ throw new SeedError(
+ "The gateway organization has no members to bind keys to",
+ "run `pnpm run dx` to seed the default organization and its members",
+ );
+ }
+
+ for (let i = keys.length; i < params.keyCount; i++) {
+ const userId = members[i % members.length].userId;
+ const keyMaterial = `${params.idPrefix}-${params.organizationId}-${i}`;
+ const publicKey = `pk-lf-gateway-${keyMaterial}`;
+ const found = await prisma.apiKey.findUnique({
+ where: { publicKey },
+ select: { id: true, userId: true, orgId: true },
+ });
+ if (found && found.orgId !== params.organizationId) {
+ throw new SeedError(
+ `Seeded gateway key ${publicKey} belongs to another organization`,
+ "pass a different --id-prefix",
+ );
+ }
+ const key: { id: string; userId: string | null } = found ?? {
+ id: (
+ await createApiKey(prisma, {
+ tenant: OrganizationId(params.organizationId),
+ scope: UserId(userId),
+ role: SystemRoleId("AI_GATEWAY"),
+ createdBy: UserId(userId),
+ name: `Seeded team key ${i}`,
+ predefinedKeys: {
+ publicKey,
+ secretKey: `sk-lf-gateway-${keyMaterial}`,
+ },
+ })
+ ).id,
+ userId,
+ };
+ await prisma.gatewayApiKeyAssociation.upsert({
+ where: { apiKeyId: key.id },
+ create: { apiKeyId: key.id, metadata: { team: `team-${i}` } },
+ update: {},
+ });
+ keys.push(key);
+ }
+ return keys;
+};
+
+/** A key issued to no user, as the gateway's service-account keys are. */
+const ensureServiceKey = async (params: {
+ organizationId: string;
+ idPrefix: string;
+}) => {
+ const keyMaterial = `${params.idPrefix}-${params.organizationId}-service`;
+ const publicKey = `pk-lf-gateway-${keyMaterial}`;
+ const found = await prisma.apiKey.findUnique({
+ where: { publicKey },
+ select: { id: true, orgId: true },
+ });
+ if (found && found.orgId !== params.organizationId) {
+ throw new SeedError(
+ `Seeded gateway key ${publicKey} belongs to another organization`,
+ "pass a different --id-prefix",
+ );
+ }
+ const id =
+ found?.id ??
+ (
+ await createApiKey(prisma, {
+ tenant: OrganizationId(params.organizationId),
+ scope: OrganizationId(params.organizationId),
+ role: SystemRoleId("AI_GATEWAY"),
+ createdBy: "system",
+ name: "Seeded service key",
+ predefinedKeys: {
+ publicKey,
+ secretKey: `sk-lf-gateway-${keyMaterial}`,
+ },
+ })
+ ).id;
+ await prisma.gatewayApiKeyAssociation.upsert({
+ where: { apiKeyId: id },
+ create: { apiKeyId: id, metadata: { team: "service" } },
+ update: {},
+ });
+ return { id, userId: null };
+};
+
+const generationCost = (
+ usageInput: number,
+ usageOutput: number,
+ rate: number,
+) => {
+ const usage = {
+ input: usageInput,
+ output: usageOutput,
+ total: usageInput + usageOutput,
+ };
+ const inputCost = usageInput * GEN_INPUT_PRICE * rate;
+ const outputCost = usageOutput * GEN_OUTPUT_PRICE * rate;
+ const cost = {
+ input: inputCost,
+ output: outputCost,
+ total: inputCost + outputCost,
+ };
+ return {
+ provided_usage_details: usage,
+ usage_details: usage,
+ provided_cost_details: cost,
+ cost_details: cost,
+ };
+};
+
+const toGenerationEvent = (params: {
+ projectId: string;
+ environment: string;
+ id: string;
+ name: string;
+ userId: string | null;
+ gatewayKeyId: string | null;
+ keyOwnerUserId: string | null;
+ model: (typeof MODELS)[number];
+ usageInput: number;
+ usageOutput: number;
+ startTime: number;
+ durationMs: number;
+}): EventRecordInsertType => {
+ const metadata: Record<string, string> = params.gatewayKeyId
+ ? {
+ [AI_GATEWAY_KEY_ID_METADATA_KEY]: params.gatewayKeyId,
+ ...(params.keyOwnerUserId
+ ? {
+ [AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY]:
+ params.keyOwnerUserId,
+ }
+ : {}),
+ }
+ : { scenario: "gateway-traffic" };
+ const metadataNames = Object.keys(metadata).sort();
+ return createEvent({
+ project_id: params.projectId,
+ trace_id: params.id,
+ span_id: params.id,
+ id: params.id,
+ parent_span_id: "",
+ is_app_root: true,
+ name: params.name,
+ trace_name: params.name,
+ type: "GENERATION",
+ environment: params.environment,
+ level: "DEFAULT",
+ status_message: null,
+ version: null,
+ release: null,
+ tags: [],
+ user_id: params.userId,
+ session_id: null,
+ public: false,
+ bookmarked: false,
+ input: JSON.stringify({
+ messages: [{ role: "user", content: params.name }],
+ }),
+ output: "Seeded completion",
+ provided_model_name: params.model.name,
+ model_id: null,
+ model_parameters: "{}",
+ ...generationCost(params.usageInput, params.usageOutput, params.model.rate),
+ prompt_id: null,
+ prompt_name: null,
+ prompt_version: null,
+ metadata_names: metadataNames,
+ metadata_values: metadataNames.map((name) => metadata[name]),
+ ...(params.gatewayKeyId
+ ? {
+ ingestion_api_key: params.gatewayKeyId,
+ service_name: AI_GATEWAY_SERVICE_NAME,
+ scope_name: AI_GATEWAY_INSTRUMENTATION_SCOPE_NAME,
+ }
+ : {}),
+ start_time: params.startTime,
+ end_time: params.startTime + params.durationMs,
+ completion_start_time: null,
+ created_at: Date.now(),
+ updated_at: Date.now(),
+ event_ts: Date.now(),
+ event_bytes: 0,
+ source: "otel",
+ });
+};
+
+const run = async (
+ ctx: ScenarioContext,
+ params: Record<string, string | number | boolean>,
+): Promise<SeedSummary> => {
+ const startedAt = Date.now();
+ const days = params["days"] as number;
+ const requestsPerDay = params["requests-per-day"] as number;
+ const keyCount = params["keys"] as number;
+
+ if (days < 1 || days > 366) {
+ throw new SeedError(
+ `--days must be between 1 and 366, got ${days}`,
+ "pass the window the analytics should cover, e.g. --days 30",
+ );
+ }
+ if (requestsPerDay < 1 || keyCount < 1) {
+ throw new SeedError(
+ "--requests-per-day and --keys must be >= 1",
+ "pass positive values, e.g. --requests-per-day 40 --keys 4",
+ );
+ }
+
+ const project = await prisma.project.findUnique({
+ where: { id: ctx.projectId },
+ select: {
+ orgId: true,
+ organization: {
+ select: {
+ gatewayConfig: { select: { defaultIngestionProjectId: true } },
+ },
+ },
+ },
+ });
+ const ingestionProjectId =
+ project?.organization.gatewayConfig?.defaultIngestionProjectId;
+ if (!project || !ingestionProjectId) {
+ throw new SeedError(
+ `No AI Gateway ingestion project for the organization of project ${ctx.projectId}`,
+ "run `pnpm run dx` (it seeds the gateway config) or pick an ingestion project in Organization settings > AI Gateway > Setup",
+ );
+ }
+
+ const gatewayEvents = days * requestsPerDay;
+ const sdkEvents = days * SDK_EVENTS_PER_DAY;
+ const counts = {
+ gatewayEvents,
+ sdkEvents,
+ events: gatewayEvents + sdkEvents,
+ };
+ const links = [
+ `${ctx.baseUrl}/organization/${project.orgId}/settings/ai-gateway-analytics`,
+ ];
+ const summary = {
+ scenario: "gateway-traffic",
+ target: "clickhouse" as const,
+ params,
+ projectId: ingestionProjectId,
+ environment: ctx.environment,
+ traceIds: Array.from(
+ { length: Math.min(requestsPerDay, 5) },
+ (_, i) => `${ctx.idPrefix}-d1-r${i}`,
+ ),
+ sessionIds: [],
+ counts,
+ links,
+ };
+
+ if (ctx.dryRun) {
+ return {
+ ...summary,
+ verified: {},
+ dryRun: true,
+ durationMs: Date.now() - startedAt,
+ };
+ }
+
+ const keys = [
+ await ensureServiceKey({
+ organizationId: project.orgId,
+ idPrefix: ctx.idPrefix,
+ }),
+ ...(await ensureGatewayKeys({
+ organizationId: project.orgId,
+ keyCount,
+ idPrefix: ctx.idPrefix,
+ })),
+ ];
+ ctx.log(
+ `building ${gatewayEvents} gateway requests over ${days} day(s) across ${keys.length} key(s)`,
+ );
+
+ const dayStartToday = utcDayStartMs();
+ const events: EventRecordInsertType[] = [];
+ for (let d = 1; d <= days; d++) {
+ const dayStart = dayStartToday - d * 86_400_000;
+ for (let i = 0; i < requestsPerDay + SDK_EVENTS_PER_DAY; i++) {
+ const isGateway = i < requestsPerDay;
+ const row = isGateway ? i : i - requestsPerDay;
+ // Keyed by (day, kind, row) only: start_time is in the events ORDER BY,
+ // so it must not shift when --requests-per-day changes.
+ const g = (d * 2 + (isGateway ? 0 : 1)) * 1_000_000 + row;
+ // sqrt of a uniform pick skews toward later keys, so the per-key and
+ // per-user bars differ in length.
+ const key =
+ keys[
+ Math.floor(
+ Math.sqrt(jitter(ctx.seed, g * 7 + 1, keys.length ** 2 - 1)),
+ )
+ ];
+ events.push(
+ toGenerationEvent({
+ projectId: ingestionProjectId,
+ environment: ctx.environment,
+ id: isGateway
+ ? `${ctx.idPrefix}-d${d}-r${row}`
+ : `${ctx.idPrefix}-d${d}-sdk${row}`,
+ name: isGateway ? "chat.completions" : "sdk-generation",
+ // A service key's caller sets the user ID itself.
+ userId: isGateway
+ ? (key.userId ?? `app-user-${jitter(ctx.seed, g * 29 + 7, 2)}`)
+ : "sdk-user",
+ gatewayKeyId: isGateway ? key.id : null,
+ keyOwnerUserId: isGateway ? key.userId : null,
+ model: MODELS[jitter(ctx.seed, g * 11 + 2, MODELS.length - 1)],
+ usageInput: 300 + jitter(ctx.seed, g * 13 + 3, 4000),
+ usageOutput: 100 + jitter(ctx.seed, g * 17 + 4, 1500),
+ startTime: dayStart + jitter(ctx.seed, g * 19 + 5, 86_399_000),
+ durationMs: 400 + jitter(ctx.seed, g * 23 + 6, 3000),
+ }),
+ );
+ }
+ }
+
+ ctx.log(`inserting ${events.length} events`);
+ for (const batch of chunk(events, 500)) {
+ await createEventsCh(batch);
+ }
+
+ const verified = {
+ events: await countRows(
+ "events_core",
+ `project_id = {projectId: String} AND trace_id LIKE {prefix: String} AND is_deleted = 0`,
+ {
+ projectId: ingestionProjectId,
+ prefix: `${escapeLike(ctx.idPrefix)}-d%`,
+ },
+ "uniqExact(span_id)",
+ ),
+ };
+ // Exact, because rows left by an earlier run with more --days or
+ // --requests-per-day under the same prefix would inflate the analytics.
+ if (verified.events !== counts.events) {
+ throw new SeedError(
+ `Readback mismatch: expected ${counts.events} events, found ${verified.events}`,
+ verified.events > counts.events
+ ? "an earlier run with larger --days or --requests-per-day left rows under this prefix; pass a fresh --id-prefix"
+ : undefined,
+ );
+ }
+
+ return {
+ ...summary,
+ verified,
+ dryRun: false,
+ durationMs: Date.now() - startedAt,
+ };
+};
+
+export const gatewayTrafficScenario: ScenarioDefinition = {
+ name: "gateway-traffic",
+ description:
+ "AI Gateway requests in the organization's gateway ingestion project: one GENERATION per request attributed to the gateway service, its gateway key, and the key's user, plus a few SDK events. Creates gateway keys up to --keys, plus one service key bound to no user. Built for the AI Gateway analytics settings tab.",
+ supportsV4: true,
+ flags: [
+ {
+ flag: "days",
+ type: "number",
+ default: 30,
+ description: "spread requests over the past N days (starting yesterday)",
+ },
+ {
+ flag: "requests-per-day",
+ type: "number",
+ default: 40,
+ description: "gateway requests per day",
+ },
+ {
+ flag: "keys",
+ type: "number",
+ default: 4,
+ description:
+ "gateway keys to spread requests over (missing ones are created)",
+ },
+ {
+ flag: "v4",
+ type: "boolean",
+ default: true,
+ description:
+ "check the v4 events tables before writing (the scenario only writes events)",
+ },
+ ],
+ run,
+};
diff --git a/packages/shared/scripts/seeder/scenarios/index.ts b/packages/shared/scripts/seeder/scenarios/index.ts
index 2c303d79c8..f99e9647f0 100644
--- a/packages/shared/scripts/seeder/scenarios/index.ts
+++ b/packages/shared/scripts/seeder/scenarios/index.ts
@@ -7,6 +7,7 @@ import { deepChainScenario } from "./deep-chain";
import { evaluatorGalleryScenario } from "./evaluator-gallery";
import { experimentFiltersScenario } from "./experiment-filters";
import { experimentIoScenario } from "./experiment-io";
+import { gatewayTrafficScenario } from "./gateway-traffic";
import { incidentSessionScenario } from "./incident-session";
import { longSessionScenario } from "./long-session";
import { manyTracesScenario } from "./many-traces";
@@ -34,6 +35,7 @@ export const scenarios: Record<string, ScenarioDefinition> = {
"evaluator-gallery": evaluatorGalleryScenario,
"experiment-io": experimentIoScenario,
"experiment-filters": experimentFiltersScenario,
+ "gateway-traffic": gatewayTrafficScenario,
"long-session": longSessionScenario,
"many-traces": manyTracesScenario,
"outlier-traffic": outlierTrafficScenario,
diff --git a/packages/shared/src/features/ai-gateway/constants.ts b/packages/shared/src/features/ai-gateway/constants.ts
index f5c2fb947f..a836bbc927 100644
--- a/packages/shared/src/features/ai-gateway/constants.ts
+++ b/packages/shared/src/features/ai-gateway/constants.ts
@@ -3,3 +3,10 @@
* gateway uploaded carry it as their `service_name`.
*/
export const AI_GATEWAY_SERVICE_NAME = "langfuse-ai-gateway";
+
+/**
+ * Metadata key holding the user a gateway API key is bound to. The gateway
+ * always overwrites `langfuse.gateway.*` metadata, so callers cannot set it.
+ */
+export const AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY =
+ "langfuse.gateway.api_key.owner.user_id";
diff --git a/packages/shared/src/features/query/dataModel.ts b/packages/shared/src/features/query/dataModel.ts
index b483d28347..35416085b7 100644
--- a/packages/shared/src/features/query/dataModel.ts
+++ b/packages/shared/src/features/query/dataModel.ts
@@ -14,6 +14,10 @@ import {
} from "../../eventsTable";
import { LISTABLE_SCORE_TYPES } from "../../domain/scores";
import { EvalExecutionMetadataKey } from "../evals/evalExecutionMetadata";
+import {
+ AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY,
+ AI_GATEWAY_SERVICE_NAME,
+} from "../ai-gateway/constants";
// The data model defines all available dimensions, measures, and the timeDimension for a given view.
// Make sure to update web/src/features/dashboard/lib/dashboardUiTableToViewMapping.ts if you make changes
@@ -1325,6 +1329,29 @@ export const eventsObservationsView: ViewDeclarationType = {
description: "Identifier of the session triggering the observation.",
highCardinality: true,
},
+ aiGatewayKeyId: {
+ // Ingestion only writes the gateway service name on uploads the gateway
+ // authenticated, and then sets ingestion_api_key to the gateway key id.
+ sql: `if(events_observations.service_name = '${AI_GATEWAY_SERVICE_NAME}', nullIf(events_observations.ingestion_api_key, ''), NULL)`,
+ alias: "aiGatewayKeyId",
+ type: "string",
+ description:
+ "Identifier of the AI Gateway API key that produced the observation. Empty for traffic that did not pass through the gateway.",
+ highCardinality: true,
+ uiHidden: true,
+ },
+ aiGatewayKeyOwnerUserId: {
+ // The span's user_id is not used: a key without an owner lets its caller
+ // set any user ID. The owner ID comes from metadata the gateway writes,
+ // so it outlives the key's deletion.
+ sql: `if(events_observations.service_name = '${AI_GATEWAY_SERVICE_NAME}', nullIf(events_observations.metadata_values[indexOf(events_observations.metadata_names, '${AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY}')], ''), NULL)`,
+ alias: "aiGatewayKeyOwnerUserId",
+ type: "string",
+ description:
+ "Identifier of the user an AI Gateway key is bound to. Empty for service keys and for traffic that did not pass through the gateway.",
+ highCardinality: true,
+ uiHidden: true,
+ },
tags: {
sql: "events_observations.tags",
alias: "tags",
diff --git a/web/src/__tests__/organization-settings-pages.clienttest.tsx b/web/src/__tests__/organization-settings-pages.clienttest.tsx
index b3481da44e..2f06a0fd9a 100644
--- a/web/src/__tests__/organization-settings-pages.clienttest.tsx
+++ b/web/src/__tests__/organization-settings-pages.clienttest.tsx
@@ -115,6 +115,7 @@ vi.mock("@/src/utils/api", () => ({
}));
vi.mock("@/src/features/ai-gateway", () => ({
+ GatewayAnalyticsPage: () => null,
GatewayApiKeysPage: () => null,
GatewayModelsPage: () => null,
GatewayNotDeployedNotice: () => null,
@@ -285,6 +286,7 @@ describe("useOrganizationSettingsPages", () => {
"ai-gateway",
"ai-gateway-models",
"ai-gateway-api-keys",
+ "ai-gateway-analytics",
]);
expect(gatewayPages.every((page) => page.show === true)).toBe(true);
expect(gatewayPages.every((page) => !page.disabledReason)).toBe(true);
@@ -313,6 +315,7 @@ describe("useOrganizationSettingsPages", () => {
["ai-gateway", false],
["ai-gateway-models", true],
["ai-gateway-api-keys", true],
+ ["ai-gateway-analytics", true],
]);
});
diff --git a/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts b/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts
index 59ce3908a9..0b6c236d43 100644
--- a/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts
+++ b/web/src/__tests__/server/ai-gateway-control-plane.servertest.ts
@@ -997,6 +997,69 @@ describe("AI gateway control plane", () => {
);
});
+ it("names organization keys and members for managers, plain organization keys only for key admins", async () => {
+ const owner = await prepare();
+ const other = await prepare();
+ const member = await createCallerForOrganizationRole(owner, Role.MEMBER);
+ const admin = await createCallerForOrganizationRole(owner, Role.ADMIN);
+ const memberKey = await member.aiGateway.createApiKey({
+ orgId: owner.org.id,
+ name: "Member key",
+ metadata: {},
+ });
+ const serviceKey = await owner.caller.aiGateway.createApiKey({
+ orgId: owner.org.id,
+ name: "Service key",
+ metadata: {},
+ issuedAs: "serviceAccount",
+ });
+ const plainOrgKey = await createApiKey(prisma, {
+ tenant: OrganizationId(owner.org.id),
+ scope: OrganizationId(owner.org.id),
+ role: SystemRoleId("ADMIN"),
+ createdBy: UserId(owner.user.id),
+ });
+ await createApiKey(prisma, {
+ tenant: OrganizationId(owner.org.id),
+ scope: OrganizationId(owner.org.id),
+ role: SystemRoleId("ADMIN"),
+ createdBy: UserId(owner.user.id),
+ isInAppAgentKey: true,
+ });
+ await other.caller.aiGateway.createApiKey({
+ orgId: other.org.id,
+ name: "Other org key",
+ metadata: {},
+ });
+
+ const ownerLabels = await owner.caller.aiGateway.getAnalyticsLabels({
+ orgId: owner.org.id,
+ });
+ expect(ownerLabels.apiKeys.map((key) => key.id).sort()).toEqual(
+ [memberKey.id, serviceKey.id, plainOrgKey.id].sort(),
+ );
+ const byId = new Map(ownerLabels.apiKeys.map((key) => [key.id, key]));
+ expect(byId.get(memberKey.id)).toMatchObject({ name: "Member key" });
+ expect(byId.get(serviceKey.id)).toMatchObject({ name: "Service key" });
+ expect(ownerLabels.users.map((user) => user.name).sort()).toEqual([
+ "Gateway admin",
+ "Gateway member",
+ "Gateway tester",
+ ]);
+
+ // An admin manages the gateway but cannot list organization API keys.
+ const adminLabels = await admin.aiGateway.getAnalyticsLabels({
+ orgId: owner.org.id,
+ });
+ expect(adminLabels.apiKeys.map((key) => key.id).sort()).toEqual(
+ [memberKey.id, serviceKey.id].sort(),
+ );
+
+ await expect(
+ member.aiGateway.getAnalyticsLabels({ orgId: owner.org.id }),
+ ).rejects.toMatchObject({ code: "FORBIDDEN" });
+ });
+
it("creates, lists, and revokes only associated organization keys", async () => {
const { caller, org } = await prepare();
const expiresAt = new Date(Date.now() + 30 * 24 * 60 * 60 * 1000);
diff --git a/web/src/__tests__/server/queryBuilder.servertest.ts b/web/src/__tests__/server/queryBuilder.servertest.ts
index 34a1a5f668..755291139f 100644
--- a/web/src/__tests__/server/queryBuilder.servertest.ts
+++ b/web/src/__tests__/server/queryBuilder.servertest.ts
@@ -5,6 +5,10 @@ import {
validateQuery,
type QueryType,
} from "@langfuse/shared/query";
+import {
+ AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY,
+ AI_GATEWAY_SERVICE_NAME,
+} from "@langfuse/shared";
import { env } from "@/src/env.mjs";
import {
createTrace,
@@ -4441,6 +4445,82 @@ describe("queryBuilder", () => {
).rejects.toThrow("Only one pairExpand dimension is supported per query");
});
});
+
+ describe("AI gateway attribution (v2)", () => {
+ const itWithEventsTable =
+ env.LANGFUSE_MIGRATION_V4_ALLOW_PREVIEW_OPT_IN === "true" ? it : it.skip;
+
+ itWithEventsTable(
+ "attributes user-tied keys to their owner and leaves service keys out",
+ async () => {
+ const projectId = randomUUID();
+ const gatewayEvent = (params: {
+ keyId: string;
+ ownerUserId?: string;
+ userId?: string;
+ }) =>
+ createEvent({
+ project_id: projectId,
+ type: "GENERATION",
+ start_time: Date.now() * 1000,
+ service_name: AI_GATEWAY_SERVICE_NAME,
+ ingestion_api_key: params.keyId,
+ user_id: params.userId ?? null,
+ metadata_names: params.ownerUserId
+ ? [AI_GATEWAY_KEY_OWNER_USER_ID_METADATA_KEY]
+ : [],
+ metadata_values: params.ownerUserId ? [params.ownerUserId] : [],
+ });
+ await createEventsCh([
+ gatewayEvent({ keyId: "key-a1", ownerUserId: "user-a" }),
+ // A second key of the same owner, e.g. one revoked since.
+ gatewayEvent({ keyId: "key-a2", ownerUserId: "user-a" }),
+ // A service key whose caller claims to be user-a.
+ gatewayEvent({ keyId: "service-key", userId: "user-a" }),
+ // SDK traffic into the same project.
+ createEvent({
+ project_id: projectId,
+ type: "GENERATION",
+ start_time: Date.now() * 1000,
+ user_id: "user-a",
+ ingestion_api_key: "pk-lf-sdk",
+ }),
+ ]);
+
+ const result = await executeQuery(
+ projectId,
+ {
+ view: "observations",
+ dimensions: [{ field: "aiGatewayKeyOwnerUserId" }],
+ metrics: [{ measure: "count", aggregation: "count" }],
+ filters: [
+ {
+ column: "aiGatewayKeyOwnerUserId",
+ operator: "is not null",
+ value: "",
+ type: "null",
+ },
+ ],
+ timeDimension: null,
+ fromTimestamp: new Date(Date.now() - 86400000).toISOString(),
+ toTimestamp: new Date(Date.now() + 86400000).toISOString(),
+ orderBy: null,
+ },
+ "v2",
+ true,
+ );
+
+ expect(
+ Object.fromEntries(
+ result.map((row) => [
+ String(row["aiGatewayKeyOwnerUserId"]),
+ Number(row["count_count"]),
+ ]),
+ ),
+ ).toEqual({ "user-a": 2 });
+ },
+ );
+ });
});
describe("validateQuery", () => {
diff --git a/web/src/components/design-system/internal/charts/ChartTooltip.tsx b/web/src/components/design-system/internal/charts/ChartTooltip.tsx
index c2d7ddfcb6..23c4916cc6 100644
--- a/web/src/components/design-system/internal/charts/ChartTooltip.tsx
+++ b/web/src/components/design-system/internal/charts/ChartTooltip.tsx
@@ -447,7 +447,7 @@ export function ChartTooltip({
<div
ref={refs.setFloating}
role="tooltip"
- className="border-border/50 bg-background pointer-events-none z-50 grid min-w-32 gap-1.5 rounded-lg border px-2.5 py-1.5 text-xs shadow-xl"
+ className="ph-no-capture border-border/50 bg-background pointer-events-none z-50 grid min-w-32 gap-1.5 rounded-lg border px-2.5 py-1.5 text-xs shadow-xl"
style={floatingStyles}
>
{activeTooltip.heading ? (
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/GatewayAnalyticsPage.tsx b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/GatewayAnalyticsPage.tsx
new file mode 100644
index 0000000000..4fa01aa101
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/GatewayAnalyticsPage.tsx
@@ -0,0 +1,108 @@
+import { ChartColumn } from "@langfuse/icons";
+
+import Header from "@/src/components/layouts/header";
+import { Alert } from "@/src/components/design-system/Alert/Alert";
+import { Button } from "@/src/components/design-system/Button/Button";
+import { Skeleton } from "@/src/components/ui/skeleton";
+import { GatewayAnalyticsDashboard } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsDashboard/GatewayAnalyticsDashboard";
+import { GatewaySetupEmptyState } from "@/src/features/ai-gateway/components/GatewaySetupEmptyState/GatewaySetupEmptyState";
+import { useGatewayConfig } from "@/src/features/ai-gateway/components/GatewaySetupPage/hooks/useGatewayConfig";
+import { useLangfuseV4WriteMode } from "@/src/features/organizations/hooks";
+
+/** Managers only; the settings route sends them to Setup until ingestion is configured. */
+export function GatewayAnalyticsPage({
+ organizationId,
+}: {
+ organizationId: string;
+}) {
+ const configState = useGatewayConfig(organizationId);
+ const v4WriteMode = useLangfuseV4WriteMode();
+
+ // Widgets run v2 queries, which legacy write mode does not serve.
+ if (v4WriteMode === "legacy") {
+ return <AnalyticsNeedEvents />;
+ }
+ if (configState.status === "pending") {
+ return <AnalyticsSkeleton />;
+ }
+ if (configState.status === "error") {
+ return <AnalyticsLoadError retry={configState.retry} />;
+ }
+
+ const project = configState.config?.defaultIngestionProject;
+ if (
+ configState.config?.ingestionMode === "NONE" ||
+ !project ||
+ project.deletedAt
+ ) {
+ return <AnalyticsUnavailable organizationId={organizationId} />;
+ }
+ return (
+ <GatewayAnalyticsDashboard
+ organizationId={organizationId}
+ projectId={project.id}
+ />
+ );
+}
+
+function AnalyticsUnavailable({ organizationId }: { organizationId: string }) {
+ return (
+ <div className="flex flex-col gap-4">
+ <Header title="Gateway analytics" />
+ <GatewaySetupEmptyState
+ icon={ChartColumn}
+ title="Gateway requests are not recorded"
+ description="Analytics are built from the traces the gateway writes to its ingestion project. Turn on usage or full ingestion in Setup to see cost and usage here."
+ actions={[
+ {
+ type: "link",
+ label: "Open Setup",
+ href: `/organization/${organizationId}/settings/ai-gateway`,
+ },
+ ]}
+ />
+ </div>
+ );
+}
+
+function AnalyticsNeedEvents() {
+ return (
+ <div className="flex flex-col gap-4">
+ <Header title="Gateway analytics" />
+ <GatewaySetupEmptyState
+ icon={ChartColumn}
+ title="Analytics are not available on this deployment"
+ description="Gateway analytics read from the events tables, which are not written while LANGFUSE_MIGRATION_V4_WRITE_MODE is legacy."
+ />
+ </div>
+ );
+}
+
+function AnalyticsLoadError({ retry }: { retry: () => void }) {
+ return (
+ <div className="flex flex-col gap-4">
+ <Header title="Gateway analytics" />
+ <Alert variant="destructive">
+ <Alert.Title>Gateway configuration could not be loaded</Alert.Title>
+ <Alert.Description>
+ Analytics need the gateway&apos;s ingestion project.
+ </Alert.Description>
+ </Alert>
+ <div>
+ <Button text="Retry" variant="secondary" onClick={retry} />
+ </div>
+ </div>
+ );
+}
+
+function AnalyticsSkeleton() {
+ return (
+ <div
+ className="flex flex-col gap-4"
+ data-testid="gateway-analytics-loading"
+ >
+ <Skeleton className="h-7 w-48" />
+ <Skeleton className="h-80 w-full" />
+ </div>
+ );
+}
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsDashboard/GatewayAnalyticsDashboard.tsx b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsDashboard/GatewayAnalyticsDashboard.tsx
new file mode 100644
index 0000000000..9d272ffb58
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsDashboard/GatewayAnalyticsDashboard.tsx
@@ -0,0 +1,70 @@
+import { useMemo, useState } from "react";
+
+import { TimeRangePicker } from "@/src/components/date-picker";
+import Header from "@/src/components/layouts/header";
+import { GatewayAnalyticsWidgetCard } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsWidgetCard/GatewayAnalyticsWidgetCard";
+import { GATEWAY_ANALYTICS_WIDGETS } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/constants/gatewayAnalyticsWidgets";
+import { useGatewayAnalyticsLabels } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/hooks/useGatewayAnalyticsLabels";
+import {
+ toAbsoluteTimeRange,
+ type TimeRange,
+} from "@/src/utils/date-range-utils";
+
+const TIME_RANGE_PRESETS = [
+ "last1Day",
+ "last7Days",
+ "last30Days",
+ "last90Days",
+ "last1Year",
+] as const;
+
+export function GatewayAnalyticsDashboard({
+ organizationId,
+ projectId,
+}: {
+ organizationId: string;
+ projectId: string;
+}) {
+ const [timeRange, setTimeRange] = useState<TimeRange>({
+ range: "last30Days",
+ });
+ const absoluteTimeRange = useMemo(
+ () => toAbsoluteTimeRange(timeRange),
+ [timeRange],
+ );
+ const labels = useGatewayAnalyticsLabels(organizationId);
+
+ return (
+ <div className="flex flex-col gap-4">
+ <Header
+ title="Gateway analytics"
+ help={{
+ description:
+ "Cost and usage of requests made through the AI Gateway, read from its ingestion project.",
+ }}
+ actionButtons={
+ <TimeRangePicker
+ timeRange={timeRange}
+ onTimeRangeChange={setTimeRange}
+ timeRangePresets={TIME_RANGE_PRESETS}
+ />
+ }
+ />
+ {absoluteTimeRange && (
+ <div className="grid grid-cols-1 gap-3 lg:grid-cols-6">
+ {GATEWAY_ANALYTICS_WIDGETS.map((widget) => (
+ <GatewayAnalyticsWidgetCard
+ key={widget.id}
+ projectId={projectId}
+ widget={widget}
+ timeRange={absoluteTimeRange}
+ dimensionLabelMap={
+ widget.dimensionLabels && labels[widget.dimensionLabels]
+ }
+ />
+ ))}
+ </div>
+ )}
+ </div>
+ );
+}
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsWidgetCard/GatewayAnalyticsWidgetCard.tsx b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsWidgetCard/GatewayAnalyticsWidgetCard.tsx
new file mode 100644
index 0000000000..7e8b204aff
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/components/GatewayAnalyticsWidgetCard/GatewayAnalyticsWidgetCard.tsx
@@ -0,0 +1,75 @@
+import { useMemo } from "react";
+
+import { buildGatewayAnalyticsQuery } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery";
+import type { GatewayAnalyticsWidget } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget";
+import { DashboardCard } from "@/src/features/dashboard/components/cards/DashboardCard";
+import { WidgetContent } from "@/src/features/widgets";
+import { NoDataOrLoading } from "@/src/components/NoDataOrLoading";
+import { cn } from "@/src/utils/tailwind";
+
+const SIZE_CLASSES: Record<GatewayAnalyticsWidget["size"], string> = {
+ small: "h-40",
+ medium: "h-96 lg:col-span-3",
+ full: "h-96 lg:col-span-6",
+};
+
+// A breakdown bar chart draws nothing but empty axes when there are no rows,
+// unlike the time-series and number tiles that already fail into a "No data"
+// card themselves. Hand it one explicitly. Stable node so the chart memo holds.
+const NO_DATA_PLACEHOLDER = <NoDataOrLoading isLoading={false} />;
+
+const BAR_CHART_TYPES = new Set<GatewayAnalyticsWidget["chartType"]>([
+ "HORIZONTAL_BAR",
+ "VERTICAL_BAR",
+]);
+
+export function GatewayAnalyticsWidgetCard({
+ projectId,
+ widget,
+ timeRange,
+ dimensionLabelMap,
+}: {
+ projectId: string;
+ widget: GatewayAnalyticsWidget;
+ timeRange: { from: Date; to: Date };
+ dimensionLabelMap?: ReadonlyMap<string, string>;
+}) {
+ const query = useMemo(
+ () => buildGatewayAnalyticsQuery(widget, timeRange),
+ [widget, timeRange],
+ );
+
+ return (
+ <DashboardCard
+ className={cn(
+ // Series labels can be member names, emails and key notes.
+ "ph-no-capture min-w-0 lg:col-span-2",
+ SIZE_CLASSES[widget.size],
+ )}
+ title={widget.name}
+ description={widget.description}
+ // WidgetContent renders its own loading state.
+ isLoading={false}
+ cardContentClassName="min-h-0"
+ >
+ <WidgetContent
+ projectId={projectId}
+ query={query}
+ version="v2"
+ chartType={widget.chartType}
+ chartConfig={widget.chartConfig}
+ metrics={widget.metrics}
+ dimensions={widget.dimensions}
+ view={widget.view}
+ schedulerId={`ai-gateway-analytics:${widget.id}`}
+ layoutHint="compact"
+ dimensionLabelMap={dimensionLabelMap}
+ emptyState={
+ BAR_CHART_TYPES.has(widget.chartType)
+ ? NO_DATA_PLACEHOLDER
+ : undefined
+ }
+ />
+ </DashboardCard>
+ );
+}
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/constants/gatewayAnalyticsWidgets.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/constants/gatewayAnalyticsWidgets.ts
new file mode 100644
index 0000000000..9ede87ab99
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/constants/gatewayAnalyticsWidgets.ts
@@ -0,0 +1,113 @@
+import type { GatewayAnalyticsWidget } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget";
+
+const BREAKDOWN_ROW_LIMIT = 20;
+
+/** Per-user breakdowns leave out service keys, which have no bound owner. */
+const KEY_OWNER_PRESENT_FILTER: GatewayAnalyticsWidget["filters"][number] = {
+ column: "aiGatewayKeyOwnerUserId",
+ operator: "is not null",
+ value: "",
+ type: "null",
+};
+
+export const GATEWAY_ANALYTICS_WIDGETS: readonly GatewayAnalyticsWidget[] = [
+ {
+ id: "total-cost",
+ name: "Total cost",
+ description: "Cost of all gateway requests",
+ view: "observations",
+ dimensions: [],
+ metrics: [{ measure: "totalCost", agg: "sum" }],
+ filters: [],
+ chartType: "NUMBER",
+ chartConfig: { type: "NUMBER" },
+ size: "small",
+ },
+ {
+ id: "requests",
+ name: "Requests",
+ description: "Observations recorded by the gateway",
+ view: "observations",
+ dimensions: [],
+ metrics: [{ measure: "count", agg: "count" }],
+ filters: [],
+ chartType: "NUMBER",
+ chartConfig: { type: "NUMBER" },
+ size: "small",
+ },
+ {
+ id: "total-tokens",
+ name: "Tokens",
+ description: "Input and output tokens",
+ view: "observations",
+ dimensions: [],
+ metrics: [{ measure: "totalTokens", agg: "sum" }],
+ filters: [],
+ chartType: "NUMBER",
+ chartConfig: { type: "NUMBER" },
+ size: "small",
+ },
+ {
+ id: "cost-over-time",
+ name: "Cost over time",
+ description: "Cost of all gateway requests over time",
+ view: "observations",
+ dimensions: [],
+ metrics: [{ measure: "totalCost", agg: "sum" }],
+ filters: [],
+ chartType: "BAR_TIME_SERIES",
+ chartConfig: { type: "BAR_TIME_SERIES" },
+ size: "full",
+ },
+ {
+ id: "cost-by-user",
+ name: "Cost by user",
+ description: `Top ${BREAKDOWN_ROW_LIMIT} users by cost; service keys are excluded`,
+ view: "observations",
+ dimensions: [{ field: "aiGatewayKeyOwnerUserId" }],
+ metrics: [{ measure: "totalCost", agg: "sum" }],
+ filters: [KEY_OWNER_PRESENT_FILTER],
+ chartType: "HORIZONTAL_BAR",
+ chartConfig: { type: "HORIZONTAL_BAR", row_limit: BREAKDOWN_ROW_LIMIT },
+ dimensionLabels: "users",
+ size: "medium",
+ },
+ {
+ id: "requests-per-user",
+ name: "Requests per user",
+ description: `Top ${BREAKDOWN_ROW_LIMIT} users by requests; service keys are excluded`,
+ view: "observations",
+ dimensions: [{ field: "aiGatewayKeyOwnerUserId" }],
+ metrics: [{ measure: "count", agg: "count" }],
+ filters: [KEY_OWNER_PRESENT_FILTER],
+ chartType: "HORIZONTAL_BAR",
+ chartConfig: { type: "HORIZONTAL_BAR", row_limit: BREAKDOWN_ROW_LIMIT },
+ dimensionLabels: "users",
+ size: "medium",
+ },
+ {
+ id: "cost-by-api-key",
+ name: "Cost by API key",
+ description: `Top ${BREAKDOWN_ROW_LIMIT} gateway keys by cost`,
+ view: "observations",
+ dimensions: [{ field: "aiGatewayKeyId" }],
+ metrics: [{ measure: "totalCost", agg: "sum" }],
+ filters: [],
+ chartType: "HORIZONTAL_BAR",
+ chartConfig: { type: "HORIZONTAL_BAR", row_limit: BREAKDOWN_ROW_LIMIT },
+ dimensionLabels: "apiKeys",
+ size: "medium",
+ },
+ {
+ id: "cost-by-model",
+ name: "Cost by model",
+ description: `Top ${BREAKDOWN_ROW_LIMIT} models by cost`,
+ view: "observations",
+ dimensions: [{ field: "providedModelName" }],
+ metrics: [{ measure: "totalCost", agg: "sum" }],
+ filters: [],
+ chartType: "HORIZONTAL_BAR",
+ chartConfig: { type: "HORIZONTAL_BAR", row_limit: BREAKDOWN_ROW_LIMIT },
+ size: "medium",
+ },
+];
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery.clienttest.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery.clienttest.ts
new file mode 100644
index 0000000000..12194f48a6
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery.clienttest.ts
@@ -0,0 +1,59 @@
+import { describe, expect, it } from "vitest";
+import { query as querySchema, validateQuery } from "@langfuse/shared/query";
+
+import { GATEWAY_ANALYTICS_WIDGETS } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/constants/gatewayAnalyticsWidgets";
+import { buildGatewayAnalyticsQuery } from "./buildGatewayAnalyticsQuery";
+
+const timeRange = {
+ from: new Date("2026-01-01T00:00:00.000Z"),
+ to: new Date("2026-01-31T00:00:00.000Z"),
+};
+
+const widget = (id: string) => {
+ const found = GATEWAY_ANALYTICS_WIDGETS.find((w) => w.id === id);
+ if (!found) throw new Error(`Unknown widget ${id}`);
+ return found;
+};
+
+describe("buildGatewayAnalyticsQuery", () => {
+ it.each(GATEWAY_ANALYTICS_WIDGETS.map((w) => [w.id, w] as const))(
+ "builds a valid v2 query scoped to gateway traffic for %s",
+ (_id, w) => {
+ const query = buildGatewayAnalyticsQuery(w, timeRange);
+
+ expect(w.chartConfig.type).toBe(w.chartType);
+ expect(querySchema.safeParse(query).success).toBe(true);
+ expect(validateQuery(query, "v2")).toEqual({ valid: true });
+ expect(query.filters[0]).toEqual({
+ column: "aiGatewayKeyId",
+ operator: "is not null",
+ value: "",
+ type: "null",
+ });
+ },
+ );
+
+ it("ranks breakdowns by their first metric and caps the rows", () => {
+ const query = buildGatewayAnalyticsQuery(
+ widget("cost-by-api-key"),
+ timeRange,
+ );
+
+ expect(query.dimensions).toEqual([{ field: "aiGatewayKeyId" }]);
+ expect(query.orderBy).toEqual([
+ { field: "sum_totalCost", direction: "desc" },
+ ]);
+ expect(query.chartConfig?.row_limit).toBe(20);
+ expect(query.timeDimension).toBeNull();
+ });
+
+ it("buckets time series automatically without ordering", () => {
+ const query = buildGatewayAnalyticsQuery(
+ widget("cost-over-time"),
+ timeRange,
+ );
+
+ expect(query.timeDimension).toEqual({ granularity: "auto" });
+ expect(query.orderBy).toBeNull();
+ });
+});
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery.ts
new file mode 100644
index 0000000000..43e476df99
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/buildGatewayAnalyticsQuery.ts
@@ -0,0 +1,60 @@
+import type { FilterCondition } from "@langfuse/shared";
+import {
+ buildWidgetOrderBy,
+ isV2BreakdownChart,
+ toQueryChartConfig,
+ type QueryType,
+} from "@langfuse/shared/query";
+import { isTimeSeriesChart } from "@/src/features/widgets";
+import type { GatewayAnalyticsWidget } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget";
+
+/**
+ * The ingestion project can also receive SDK traffic; only observations
+ * ingestion attributed to a gateway key count as gateway usage.
+ */
+const GATEWAY_TRAFFIC_FILTER: FilterCondition = {
+ column: "aiGatewayKeyId",
+ operator: "is not null",
+ value: "",
+ type: "null",
+};
+
+/** Top-N cap for breakdowns that do not set their own row limit. */
+const DEFAULT_BREAKDOWN_ROW_LIMIT = 100;
+
+export function buildGatewayAnalyticsQuery(
+ widget: GatewayAnalyticsWidget,
+ timeRange: { from: Date; to: Date },
+): QueryType {
+ const chartType = widget.chartType;
+ const isTimeSeries = isTimeSeriesChart(chartType);
+ const needsTopN = isV2BreakdownChart({
+ version: "v2",
+ hasDimension: widget.dimensions.length > 0,
+ isTimeSeries,
+ chartType,
+ });
+ const metrics = widget.metrics.map((metric) => ({
+ measure: metric.measure,
+ aggregation: metric.agg,
+ }));
+
+ return {
+ view: widget.view,
+ dimensions: widget.dimensions,
+ metrics,
+ filters: [GATEWAY_TRAFFIC_FILTER, ...widget.filters],
+ timeDimension: isTimeSeries ? { granularity: "auto" } : null,
+ fromTimestamp: timeRange.from.toISOString(),
+ toTimestamp: timeRange.to.toISOString(),
+ orderBy: buildWidgetOrderBy({
+ chartType,
+ sortState: null,
+ needsTopN,
+ firstMetric: metrics[0],
+ }),
+ chartConfig: toQueryChartConfig(widget.chartConfig, {
+ defaultRowLimit: needsTopN ? DEFAULT_BREAKDOWN_ROW_LIMIT : undefined,
+ }),
+ };
+}
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap.clienttest.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap.clienttest.ts
new file mode 100644
index 0000000000..a76c9bb419
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap.clienttest.ts
@@ -0,0 +1,37 @@
+import { describe, expect, it } from "vitest";
+
+import { toLabelMap } from "./toLabelMap";
+
+describe("toLabelMap", () => {
+ it("prefers the name, then the detail, then the id", () => {
+ expect(
+ toLabelMap([
+ { id: "a", name: "Alice", detail: "[email protected]" },
+ { id: "b", name: null, detail: "[email protected]" },
+ { id: "c", name: null, detail: null },
+ ]),
+ ).toEqual(
+ new Map([
+ ["a", "Alice"],
+ ["b", "[email protected]"],
+ ["c", "c"],
+ ]),
+ );
+ });
+
+ it("keeps entries that share a label apart, since charts group by label", () => {
+ expect(
+ toLabelMap([
+ { id: "k1", name: "ci", detail: "sk-gw-...1111" },
+ { id: "k2", name: "ci", detail: "sk-gw-...2222" },
+ { id: "k3", name: "prod", detail: "sk-gw-...3333" },
+ ]),
+ ).toEqual(
+ new Map([
+ ["k1", "ci (sk-gw-...1111)"],
+ ["k2", "ci (sk-gw-...2222)"],
+ ["k3", "prod"],
+ ]),
+ );
+ });
+});
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap.ts
new file mode 100644
index 0000000000..8d26b0fc07
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap.ts
@@ -0,0 +1,28 @@
+type LabelEntry = { id: string; name: string | null; detail: string | null };
+
+/**
+ * Maps IDs to display labels. A chart groups series by label, so labels shared
+ * by several IDs get their detail appended to keep the bars apart. A `Map`, as
+ * IDs are caller-controlled and may be `__proto__` or `constructor`.
+ */
+export function toLabelMap(
+ entries: readonly LabelEntry[],
+): ReadonlyMap<string, string> {
+ const baseLabels = entries.map((entry) => ({
+ entry,
+ label: entry.name || entry.detail || entry.id,
+ }));
+ const counts = new Map<string, number>();
+ for (const { label } of baseLabels) {
+ counts.set(label, (counts.get(label) ?? 0) + 1);
+ }
+
+ return new Map(
+ baseLabels.map(({ entry, label }) => {
+ const isDuplicate = (counts.get(label) ?? 0) > 1;
+ const suffix =
+ entry.detail && entry.detail !== label ? entry.detail : entry.id;
+ return [entry.id, isDuplicate ? `${label} (${suffix})` : label];
+ }),
+ );
+}
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/hooks/useGatewayAnalyticsLabels.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/hooks/useGatewayAnalyticsLabels.ts
new file mode 100644
index 0000000000..79b3b6e36c
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/hooks/useGatewayAnalyticsLabels.ts
@@ -0,0 +1,22 @@
+import { useMemo } from "react";
+
+import { toLabelMap } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/fns/toLabelMap";
+import type { GatewayAnalyticsLabelSource } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget";
+import { api } from "@/src/utils/api";
+
+/** Label maps per source; empty until loaded, so charts fall back to raw IDs. */
+export function useGatewayAnalyticsLabels(
+ organizationId: string,
+): Record<GatewayAnalyticsLabelSource, ReadonlyMap<string, string>> {
+ const { data } = api.aiGateway.getAnalyticsLabels.useQuery({
+ orgId: organizationId,
+ });
+
+ return useMemo(
+ () => ({
+ apiKeys: toLabelMap(data?.apiKeys ?? []),
+ users: toLabelMap(data?.users ?? []),
+ }),
+ [data],
+ );
+}
diff --git a/web/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget.ts b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget.ts
new file mode 100644
index 0000000000..d40391619c
--- /dev/null
+++ b/web/src/features/ai-gateway/components/GatewayAnalyticsPage/types/gatewayAnalyticsWidget.ts
@@ -0,0 +1,11 @@
+import type { WidgetExportSource } from "@/src/features/widgets";
+
+/** Which name lookup turns the first dimension's IDs into readable labels. */
+export type GatewayAnalyticsLabelSource = "apiKeys" | "users";
+
+/** A saved dashboard widget's configuration plus its place on the page. */
+export type GatewayAnalyticsWidget = WidgetExportSource & {
+ id: string;
+ dimensionLabels?: GatewayAnalyticsLabelSource;
+ size: "small" | "medium" | "full";
+};
diff --git a/web/src/features/ai-gateway/index.ts b/web/src/features/ai-gateway/index.ts
index 6882ad0854..993fb95c6e 100644
--- a/web/src/features/ai-gateway/index.ts
+++ b/web/src/features/ai-gateway/index.ts
@@ -1,3 +1,4 @@
+export { GatewayAnalyticsPage } from "@/src/features/ai-gateway/components/GatewayAnalyticsPage/GatewayAnalyticsPage";
export { GatewayApiKeysPage } from "@/src/features/ai-gateway/components/GatewayApiKeysPage/GatewayApiKeysPage";
export { GatewayModelsPage } from "@/src/features/ai-gateway/components/GatewayModelsPage/GatewayModelsPage";
export { GatewayNotDeployedNotice } from "@/src/features/ai-gateway/components/GatewayNotDeployedNotice/GatewayNotDeployedNotice";
diff --git a/web/src/features/ai-gateway/server/analytics/gatewayAnalyticsLabelsRepository.ts b/web/src/features/ai-gateway/server/analytics/gatewayAnalyticsLabelsRepository.ts
new file mode 100644
index 0000000000..808995a50e
--- /dev/null
+++ b/web/src/features/ai-gateway/server/analytics/gatewayAnalyticsLabelsRepository.ts
@@ -0,0 +1,62 @@
+import type { PrismaClient } from "@langfuse/shared/src/db";
+
+type LabelUser = { id: string; name: string | null; email: string | null };
+
+/**
+ * Names for the IDs gateway analytics group by: organization API keys and the
+ * users traffic is attributed to (organization members and key owners).
+ */
+export class GatewayAnalyticsLabelsRepository {
+ constructor(private readonly prisma: PrismaClient) {}
+
+ /**
+ * Gateway keys are always included. Plain organization keys can also call
+ * the gateway, but only callers who may list organization API keys get them.
+ */
+ async getLabels(params: {
+ organizationId: string;
+ includePlainOrganizationKeys: boolean;
+ }) {
+ const [keys, memberships] = await Promise.all([
+ this.prisma.apiKey.findMany({
+ where: {
+ orgId: params.organizationId,
+ scope: "ORGANIZATION",
+ isInAppAgentKey: false,
+ ...(!params.includePlainOrganizationKeys && {
+ gatewayAssociation: { isNot: null },
+ }),
+ },
+ select: {
+ id: true,
+ note: true,
+ displaySecretKey: true,
+ user: { select: { id: true, name: true, email: true } },
+ },
+ }),
+ this.prisma.organizationMembership.findMany({
+ where: { orgId: params.organizationId },
+ select: { user: { select: { id: true, name: true, email: true } } },
+ }),
+ ]);
+
+ const users = new Map<string, LabelUser>();
+ for (const { user } of memberships) users.set(user.id, user);
+ for (const apiKey of keys) {
+ if (apiKey.user) users.set(apiKey.user.id, apiKey.user);
+ }
+
+ return {
+ apiKeys: keys.map((apiKey) => ({
+ id: apiKey.id,
+ name: apiKey.note,
+ detail: apiKey.displaySecretKey,
+ })),
+ users: [...users.values()].map((user) => ({
+ id: user.id,
+ name: user.name,
+ detail: user.email,
+ })),
+ };
+ }
+}
diff --git a/web/src/features/ai-gateway/server/gatewaySetupRedirect.ts b/web/src/features/ai-gateway/server/gatewaySetupRedirect.ts
index dbd747a37d..14bb9b2aa3 100644
--- a/web/src/features/ai-gateway/server/gatewaySetupRedirect.ts
+++ b/web/src/features/ai-gateway/server/gatewaySetupRedirect.ts
@@ -8,10 +8,11 @@ import { GatewayConfigService } from "@/src/features/ai-gateway/server/config/ga
const SETUP_GATED_SETTINGS_SLUGS = new Set([
"ai-gateway-models",
"ai-gateway-api-keys",
+ "ai-gateway-analytics",
]);
/**
- * Gateway managers cannot open model or API key settings before an ingestion
+ * Gateway managers cannot open model, API key or analytics settings before an ingestion
* project is configured; they are sent to the setup page.
* Other members are not redirected because their Models and API key pages
* explain the state.
diff --git a/web/src/features/ai-gateway/server/router.ts b/web/src/features/ai-gateway/server/router.ts
index 798f492c43..9a2d1a4dd3 100644
--- a/web/src/features/ai-gateway/server/router.ts
+++ b/web/src/features/ai-gateway/server/router.ts
@@ -32,6 +32,7 @@ import {
MAX_MEMBER_SEARCH_LENGTH,
MAX_TEAM_INVITES,
} from "@/src/features/ai-gateway/constants/gatewayTeamInvites";
+import { GatewayAnalyticsLabelsRepository } from "./analytics/gatewayAnalyticsLabelsRepository";
import { GatewayConfigService } from "./config/gatewayConfigService";
import { GatewayTeamService } from "./team/gatewayTeamService";
import {
@@ -139,6 +140,21 @@ export const aiGatewayRouter = createTRPCRouter({
return new GatewayConfigService(ctx.prisma).getSetupStatus(input.orgId);
}),
+ /** Names for the API key and user IDs that gateway analytics break down by. */
+ getAnalyticsLabels: protectedOrganizationProcedure
+ .input(organizationInput)
+ .query(async ({ input, ctx }) => {
+ requireGatewayAdmin({ session: ctx.session, orgId: input.orgId });
+ return new GatewayAnalyticsLabelsRepository(ctx.prisma).getLabels({
+ organizationId: input.orgId,
+ includePlainOrganizationKeys: hasOrganizationAccess({
+ session: ctx.session,
+ organizationId: input.orgId,
+ scope: "organization:CRUD_apiKeys",
+ }),
+ });
+ }),
+
/** What a key owner needs to point a client at the gateway; no connection details. */
getClientSetup: protectedOrganizationProcedure
.input(organizationInput)
diff --git a/web/src/features/organizations/OrganizationSettingsPage.tsx b/web/src/features/organizations/OrganizationSettingsPage.tsx
index aaadde78dd..64e30387f6 100644
--- a/web/src/features/organizations/OrganizationSettingsPage.tsx
+++ b/web/src/features/organizations/OrganizationSettingsPage.tsx
@@ -26,6 +26,7 @@ import { OrgAuditLogsSettingsPage } from "@/src/ee/features/audit-log-viewer";
import { useV4UpgradeUiFlag } from "@/src/features/v4-migration/useV4UpgradeUiEnabled";
import { OrganizationFeaturePreviewsSettings } from "@/src/features/feature-flags/components/OrganizationFeaturePreviewsSettings";
import {
+ GatewayAnalyticsPage,
GatewayApiKeysPage,
GatewayModelsPage,
GatewayNotDeployedNotice,
@@ -365,6 +366,15 @@ const getOrganizationSettingsPages = ({
show: showAiGatewayApiKeys,
disabledReason: gatewaySetupDisabledReason,
},
+ {
+ title: "Analytics",
+ slug: "ai-gateway-analytics",
+ section: "AI Gateway",
+ cmdKKeywords: ["gateway", "analytics", "cost", "usage", "consumption"],
+ content: <GatewayAnalyticsPage organizationId={organization.id} />,
+ show: showAiGatewayManagement,
+ disabledReason: gatewaySetupDisabledReason,
+ },
];
function OrganizationMembersSettings({ orgId }: { orgId: string }) {
diff --git a/web/src/features/widgets/components/InlineWidget.tsx b/web/src/features/widgets/components/InlineWidget.tsx
index 1752eec1de..c30a27da4e 100644
--- a/web/src/features/widgets/components/InlineWidget.tsx
+++ b/web/src/features/widgets/components/InlineWidget.tsx
@@ -78,6 +78,11 @@ export interface WidgetContentProps {
* Optional presentation-only labels for entity_dimension values.
*/
entityDimensionLabelMap?: Record<string, string>;
+ /**
+ * Optional presentation-only labels for the first dimension's values, e.g.
+ * names for ID breakdowns. Values without a label keep their raw value.
+ */
+ dimensionLabelMap?: ReadonlyMap<string, string>;
/**
* Hide x-axis tick labels on a categorical (entity-name) axis; the full name
* stays in the hover tooltip. Off by default. Opt in on entity-dimension
@@ -146,6 +151,7 @@ export function WidgetContent({
onSortChange,
className,
entityDimensionLabelMap,
+ dimensionLabelMap,
hideXAxisLabels,
colorBarsByCategory,
legendPosition,
@@ -276,7 +282,7 @@ export function WidgetContent({
if (val === null || val === undefined || val === "") {
seriesDimension = "n/a";
} else if (typeof val === "string") {
- seriesDimension = val;
+ seriesDimension = dimensionLabelMap?.get(val) ?? val;
} else if (Array.isArray(val)) {
seriesDimension = val.join(", ");
} else {
@@ -331,6 +337,7 @@ export function WidgetContent({
metrics,
dimensions,
entityDimensionLabelMap,
+ dimensionLabelMap,
]);
// Chart presentation (label and formatter)
diff --git a/web/src/features/widgets/index.ts b/web/src/features/widgets/index.ts
index ff3fcc2f52..9b9692ef56 100644
--- a/web/src/features/widgets/index.ts
+++ b/web/src/features/widgets/index.ts
@@ -16,7 +16,10 @@ export type {
MissingBucketValue,
} from "@/src/features/widgets/chart-library/chart-props";
export { parseChartTimestamp } from "@/src/features/widgets/chart-library/prepareTimeAxis";
-export { formatMetric } from "@/src/features/widgets/chart-library/utils";
+export {
+ formatMetric,
+ isTimeSeriesChart,
+} from "@/src/features/widgets/chart-library/utils";
export type { DashboardPlacement } from "@/src/features/widgets/components/DashboardGrid";
export { useClipboardWidgetProbe } from "@/src/features/widgets/hooks/useClipboardWidgetProbe";
export { pushDownForInsertion } from "@/src/features/widgets/utils/grid-placement";