AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
feat(ai): ai usage read api for the usage page (#16981)
Louai Boumedienefa4eb3c8bd40
diff --git a/brain/knowledge/ai-intelligence/ai-providers.md b/brain/knowledge/ai-intelligence/ai-providers.mdindex 1bf929b059f..13218449c5d 100644--- a/brain/knowledge/ai-intelligence/ai-providers.md+++ b/brain/knowledge/ai-intelligence/ai-providers.md@@ -75,6 +75,9 @@ after a file without chat keys it seeds the chat list from the live flow list.**Specific model** — a model a builder picks directly from one key (`{provider, configId, model}`), as opposed to a tier. The admin switch `platform_configuration.aiSpecificModelsVisible` (Tiers tab, "Visible to builders") hides them from builders. _Avoid_: "Advanced" (an early mockup word for the same section)### Gotchas+- **The AI usage read API (`/v1/ai-usage/*`) works in UTC days and scales today's partial day inside the SQL.** Every previous-range sum is multiplied by a weight that is below 1 only on the day matching today, so totals, cards, lists and related items all compare the same way. Scaling after the query, in TypeScript, can't reach per-item numbers and makes the views disagree.+- **Usage tracking starts per platform, at its copy cursor.** `trackingSince` is the cursor's `startedAt`, or the platform's creation if that is later. A period that starts earlier hides its change %, so a fresh platform never shows fake growth. A platform without a cursor has no tracking start.+- **Usage ids are not always entity ids.** `no-project` is platform chat, `activepieces-credits` and `unknown-<provider>` are keys with no config, and a model id is `base64url(provider|modelId)`. Lists stop at 1000 rows. `ACTION_RUN` rows have no `flowId`, so the flows card never adds up to the flow-steps slice.- **AI usage analytics come from Autumn events, not our own stream** (decision 000053). Every Autumn `track` for an AI call carries a `usage` block v1 (`ai/usage/ai-usage-block.ts`). The block is built inside `tryCatch`: if pricing fails, the billed track still goes out without it, so billing never depends on analytics.- **Usage-only calls send a value-0 `apCredits` track with only `{ usage }`.** That covers BYOK chat and agent turns, compaction, memory, personalization, own-key embeddings and managed calls without a cost. Source is `ai_usage`, with no top-level `projectId`, so the billing page's `groupBy projectId` isn't crowded. The value is a literal 0: `chargeFor` would return `toolCalls`.- **`AI_USAGE_COPY` reads a delayed, non-overlapping window.** Each run reads `[copiedUntil, min(now − 2 min, copiedUntil + 10 min))`. The 2 minutes cover Autumn's 10–20 s visibility lag. The cursor moves with a compare-and-swap on epoch milliseconds in the same transaction as the upserts. The run's end time is fixed once, at its start: chasing `now` loops on tiny windows until the budget runs out. A window that needs more than 3 pages is halved and retried (down to 1 s, which reads uncapped), and the run's 40 s budget is only checked between windows: aborting mid-window would re-read the same busy window forever. The smaller window is saved in Redis per platform for an hour, so a run that spends its budget on a capped read still leaves the next run a smaller window instead of the full 10 minutes again. A full-size window that reads in one page doubles back toward 10 minutes (the short leftover window at the end of a run never grows it, or a busy platform would grow and re-halve every run), and the Redis entry is deleted at full size, so one busy spell does not leave a platform on tiny windows. A bad event must be skipped at parse time (token counts are integers, the block is run through `sanitizeObjectForPostgresql`): one row Postgres rejects fails the whole window's write and pins that platform's cursor until someone fixes it.diff --git a/brain/knowledge/platform-editions-ee/projects.md b/brain/knowledge/platform-editions-ee/projects.mdindex b269323847f..78743b69936 100644--- a/brain/knowledge/platform-editions-ee/projects.md+++ b/brain/knowledge/platform-editions-ee/projects.md@@ -24,6 +24,7 @@ A **Project** is the workspace within a platform where flows, connections, table### Gotchas- `projectHooks.postCreate` is where EE creates the associated `ProjectPlan`, sets piece filters, and auto-subscribes an alert receiver (owner email for personal, `context.alertReceiverEmail` for team).- Soft-deleted projects stay in DB; a background job hard-deletes them.+- **Raw SQL on `project` must add `"deleted" IS NULL` itself.** TypeORM adds it only to repository queries, so a raw `databaseConnection().query` or a join through `project` returns soft-deleted projects, names included. Usage name lookups (`ai-usage-names.ts`) hit this.- **A default project is a flag on the project**, `project.isDefault` (see decision 000051). The database only allows it on team projects (`chk_project_is_default_team`). Admins set it per project with `POST /v1/projects/:id` and `isDefault`; only platform admins and API keys can, and a plan without project roles gets `FEATURE_DISABLED` (checked in the controller; the service only takes `enforceDefaultGuard`). There is no cap on how many defaults a platform has; the platform response reads them in cursor pages of 100, so no single query is unbounded. The platform response lists the live team projects with the flag as `defaultProjectIds`, oldest first. The order matters, because the invite dialog preselects the first default.- The web project collection never sends `isDefault` when it saves other project settings, so a stale cache can't undo another admin's change. Defaults are only changed through `projectCollectionUtils.useSetDefaultProjects`.- A soft-deleted project keeps its flag until the hard delete; reads skip deleted rows. A default project can't be deleted from the app while it is a default: remove it on Roles & Access first (see decision 000049). API-key deletes, SCIM group deletes, and plans without project roles still delete it. The app check is part of the soft delete (`AND "isDefault" = false`), not a read before it, so a project another admin makes a default mid-delete is still refused.diff --git a/bun.lock b/bun.lockindex 9fa8ecda440..cdcadc673ce 100644--- a/bun.lock+++ b/bun.lock@@ -167,7 +167,7 @@},"packages/core/shared": {"name": "@activepieces/shared",- "version": "0.210.0",+ "version": "0.211.0","dependencies": {"@activepieces/core-execution": "workspace:*","@activepieces/core-formula": "workspace:*",diff --git a/packages/core/shared/package.json b/packages/core/shared/package.jsonindex 8a3d23eb257..72e263deb08 100644--- a/packages/core/shared/package.json+++ b/packages/core/shared/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/shared",- "version": "0.211.0",+ "version": "0.212.0","type": "commonjs","sideEffects": false,"main": "./dist/src/index.js",diff --git a/packages/core/shared/src/index.ts b/packages/core/shared/src/index.tsindex 4ad7640fa62..fd66b29587a 100755--- a/packages/core/shared/src/index.ts+++ b/packages/core/shared/src/index.ts@@ -33,6 +33,7 @@ export * from './lib/management/project-role/project-role.request'export * from './lib/management/invitations'export * from './lib/management/analytics'export * from './lib/management/ai-providers'+export * from './lib/management/ai-usage'export * from './lib/management/ai-tools'export * from './lib/management/template'diff --git a/packages/core/shared/src/lib/management/ai-usage/index.ts b/packages/core/shared/src/lib/management/ai-usage/index.tsnew file mode 100644index 00000000000..6107de1b7cc--- /dev/null+++ b/packages/core/shared/src/lib/management/ai-usage/index.ts@@ -0,0 +1,154 @@+import { AIProviderName } from '@activepieces/core-utils'+import { z } from 'zod'++export const AI_USAGE_PERIODS = ['1h', '6h', '12h', '24h', '7d', '30d', '90d', 'this-month', 'last-month'] as const+export const AI_USAGE_SCOPES = ['all', 'production', 'testing'] as const+export const AI_USAGE_DIMENSIONS = ['project', 'flow', 'agent', 'user', 'key', 'model'] as const+export const AI_USAGE_SOURCES = ['flow-steps', 'agent-steps', 'agents', 'chat', 'knowledge-base'] as const++export const AiUsagePeriod = z.enum(AI_USAGE_PERIODS)+export type AiUsagePeriod = z.infer<typeof AiUsagePeriod>++export const AiUsageScope = z.enum(AI_USAGE_SCOPES)+export type AiUsageScope = z.infer<typeof AiUsageScope>++export const AiUsageDimension = z.enum(AI_USAGE_DIMENSIONS)+export type AiUsageDimension = z.infer<typeof AiUsageDimension>++export const AiUsageSource = z.enum(AI_USAGE_SOURCES)+export type AiUsageSource = z.infer<typeof AiUsageSource>++export const AI_USAGE_RELATED_DIMENSIONS: Record<AiUsageDimension, AiUsageDimension[]> = {+ project: ['flow', 'agent', 'user', 'model', 'key'],+ flow: ['model', 'key'],+ agent: ['user', 'model', 'key'],+ user: ['agent', 'project', 'model'],+ key: ['project', 'flow', 'model'],+ model: ['project', 'flow', 'agent', 'key'],+}++export const AI_USAGE_SOURCE_DIMENSIONS: AiUsageDimension[] = ['project', 'agent']++export const AI_USAGE_NO_PROJECT_ID = 'no-project'+export const AI_USAGE_CREDITS_KEY_ID = 'activepieces-credits'+export const AI_USAGE_UNKNOWN_KEY_PREFIX = 'unknown-'++export const AiUsageQuery = z.object({+ period: AiUsagePeriod,+ scope: AiUsageScope,+})+export type AiUsageQuery = z.infer<typeof AiUsageQuery>++export const AiUsageItemsQuery = AiUsageQuery.extend({+ dimension: AiUsageDimension,+})+export type AiUsageItemsQuery = z.infer<typeof AiUsageItemsQuery>++export const AiUsageDetailParams = z.object({+ dimension: AiUsageDimension,+ id: z.string().min(1).max(512),+})+export type AiUsageDetailParams = z.infer<typeof AiUsageDetailParams>++export const AiUsageHistory = z.object({+ trackingSince: z.string().nullable(),+ hasAnyUsage: z.boolean(),+ coveredDays: z.number(),+ periodDays: z.number(),+ currentCoverage: z.number(),+ comparable: z.boolean(),+})+export type AiUsageHistory = z.infer<typeof AiUsageHistory>++const AiUsageRange = z.object({+ start: z.string(),+ end: z.string(),+})++export const AiUsageRanges = z.object({+ current: AiUsageRange,+ previous: AiUsageRange,+})+export type AiUsageRanges = z.infer<typeof AiUsageRanges>++export const AiUsageTotals = z.object({+ spendUsd: z.number(),+ previousSpendUsd: z.number(),+ calls: z.number(),+ previousCalls: z.number(),+ tokens: z.number(),+ previousTokens: z.number(),+ cacheShare: z.number(),+ managedCredits: z.number(),+})+export type AiUsageTotals = z.infer<typeof AiUsageTotals>++export const AiUsageSeriesPoint = z.object({+ label: z.string(),+ spendUsd: z.number().nullable(),+ calls: z.number().nullable(),+ tokens: z.number().nullable(),+ previousSpendUsd: z.number().nullable(),+ previousCalls: z.number().nullable(),+ previousTokens: z.number().nullable(),+})+export type AiUsageSeriesPoint = z.infer<typeof AiUsageSeriesPoint>++export const AiUsageItem = z.object({+ dimension: AiUsageDimension,+ id: z.string(),+ name: z.string().nullable(),+ parent: z.string().optional(),+ provider: z.enum(AIProviderName).optional(),+ flowCount: z.number().optional(),+ agentCount: z.number().optional(),+ spendUsd: z.number(),+ previousSpendUsd: z.number(),+ calls: z.number(),+ tokens: z.number(),+ comparable: z.boolean(),+ sharedUsd: z.number().optional(),+ inputPrice: z.number().optional(),+ outputPrice: z.number().optional(),+})+export type AiUsageItem = z.infer<typeof AiUsageItem>++export const AiUsageSourceSpend = z.object({+ source: AiUsageSource,+ spendUsd: z.number(),+})+export type AiUsageSourceSpend = z.infer<typeof AiUsageSourceSpend>++export const AiUsageOverview = z.object({+ history: AiUsageHistory,+ ranges: AiUsageRanges,+ totals: AiUsageTotals,+ series: z.array(AiUsageSeriesPoint),+ bySource: z.array(AiUsageSourceSpend),+ top: z.record(AiUsageDimension, z.array(AiUsageItem)),+})+export type AiUsageOverview = z.infer<typeof AiUsageOverview>++export const AiUsageItemList = z.object({+ history: AiUsageHistory,+ ranges: AiUsageRanges,+ items: z.array(AiUsageItem),+ count: z.number(),+ itemsSpendUsd: z.number(),+ totalSpendUsd: z.number(),+ truncated: z.boolean(),+})+export type AiUsageItemList = z.infer<typeof AiUsageItemList>++export const AiUsageDetail = z.object({+ history: AiUsageHistory,+ ranges: AiUsageRanges,+ item: AiUsageItem,+ series: z.array(AiUsageSeriesPoint),+ related: z.partialRecord(AiUsageDimension, z.array(AiUsageItem)),+ bySource: z.array(AiUsageSourceSpend).optional(),+ runs: z.number().optional(),+ reportedUsd: z.number().optional(),+ estimatedUsd: z.number().optional(),+})+export type AiUsageDetail = z.infer<typeof AiUsageDetail>diff --git a/packages/server/api/src/app/ai/usage/ai-usage-dimension.ts b/packages/server/api/src/app/ai/usage/ai-usage-dimension.tsnew file mode 100644index 00000000000..7158c1794b4--- /dev/null+++ b/packages/server/api/src/app/ai/usage/ai-usage-dimension.ts@@ -0,0 +1,106 @@+import { AIProviderName, AiUsageOrigin, isNil } from '@activepieces/core-utils'+import { AI_USAGE_CREDITS_KEY_ID, AI_USAGE_NO_PROJECT_ID, AI_USAGE_UNKNOWN_KEY_PREFIX, AiUsageDimension, AiUsageSource } from '@activepieces/shared'++function providerOf(value: string): AIProviderName | null {+ return PROVIDERS.find((provider) => provider === value) ?? null+}++function keyFilterOf(id: string): DimensionCondition[] | null {+ if (id === AI_USAGE_CREDITS_KEY_ID) {+ return [{ column: 'provider', value: AIProviderName.ACTIVEPIECES }, { column: 'providerConfigId', value: null }]+ }+ if (id.startsWith(AI_USAGE_UNKNOWN_KEY_PREFIX)) {+ const provider = providerOf(id.slice(AI_USAGE_UNKNOWN_KEY_PREFIX.length))+ return isNil(provider) ? null : [{ column: 'provider', value: provider }, { column: 'providerConfigId', value: null }]+ }+ return [{ column: 'providerConfigId', value: id }]+}++function modelFilterOf(id: string): DimensionCondition[] | null {+ const decoded = Buffer.from(id, 'base64url').toString('utf8')+ const separator = decoded.indexOf(MODEL_ID_SEPARATOR)+ if (separator <= 0 || separator === decoded.length - 1) {+ return null+ }+ const provider = providerOf(decoded.slice(0, separator))+ return isNil(provider) ? null : [{ column: 'provider', value: provider }, { column: 'modelId', value: decoded.slice(separator + 1) }]+}++function conditionsOf({ dimension, id }: { dimension: AiUsageDimension, id: string }): DimensionCondition[] | null {+ switch (dimension) {+ case 'project':+ return [{ column: 'projectId', value: id === AI_USAGE_NO_PROJECT_ID ? null : id }]+ case 'flow':+ return [{ column: 'flowId', value: id }]+ case 'agent':+ return [{ column: 'agentId', value: id }]+ case 'user':+ return [{ column: 'userId', value: id }]+ case 'key':+ return keyFilterOf(id)+ case 'model':+ return modelFilterOf(id)+ }+}++const PROVIDERS = Object.values(AIProviderName)+const NUL = '\u0000'+const MODEL_ID_SEPARATOR = '|'++export const aiUsageDimension = {+ providerOf,+ sourceOf(origin: AiUsageOrigin): AiUsageSource {+ switch (origin) {+ case AiUsageOrigin.FLOW_STEP:+ case AiUsageOrigin.ACTION_RUN:+ return 'flow-steps'+ case AiUsageOrigin.FLOW_AGENT_STEP:+ return 'agent-steps'+ case AiUsageOrigin.AGENT:+ case AiUsageOrigin.AGENT_BUILDER:+ return 'agents'+ case AiUsageOrigin.CHAT:+ return 'chat'+ case AiUsageOrigin.KNOWLEDGE_BASE:+ return 'knowledge-base'+ }+ },+ idOf({ dimension, keys }: { dimension: AiUsageDimension, keys: DimensionKeys }): string | null {+ switch (dimension) {+ case 'project':+ return keys.projectId ?? AI_USAGE_NO_PROJECT_ID+ case 'flow':+ return keys.flowId+ case 'agent':+ return keys.agentId+ case 'user':+ return keys.userId+ case 'key':+ if (!isNil(keys.providerConfigId)) {+ return keys.providerConfigId+ }+ return keys.provider === AIProviderName.ACTIVEPIECES ? AI_USAGE_CREDITS_KEY_ID : `${AI_USAGE_UNKNOWN_KEY_PREFIX}${keys.provider}`+ case 'model':+ return Buffer.from(`${keys.provider}${MODEL_ID_SEPARATOR}${keys.modelId}`, 'utf8').toString('base64url')+ }+ },+ filterOf({ dimension, id }: { dimension: AiUsageDimension, id: string }): DimensionCondition[] | null {+ const conditions = conditionsOf({ dimension, id })+ return conditions?.some((condition) => condition.value?.includes(NUL)) ? null : conditions+ },+}++export type DimensionKeys = {+ projectId: string | null+ flowId: string | null+ agentId: string | null+ userId: string | null+ provider: string | null+ providerConfigId: string | null+ modelId: string | null+}++export type DimensionCondition = {+ column: 'projectId' | 'flowId' | 'agentId' | 'userId' | 'provider' | 'providerConfigId' | 'modelId'+ value: string | null+}diff --git a/packages/server/api/src/app/ai/usage/ai-usage-names.ts b/packages/server/api/src/app/ai/usage/ai-usage-names.tsnew file mode 100644index 00000000000..dff0ceb6701--- /dev/null+++ b/packages/server/api/src/app/ai/usage/ai-usage-names.ts@@ -0,0 +1,54 @@+import { isNil } from '@activepieces/core-utils'+import { AiUsageDimension } from '@activepieces/shared'+import { databaseConnection } from '../../database/database-connection'++async function namesOf({ platformId, dimension, ids }: { platformId: string, dimension: AiUsageDimension, ids: string[] }): Promise<NamedEntity[]> {+ const sql = NAME_QUERIES[dimension]+ if (isNil(sql) || ids.length === 0) {+ return []+ }+ return databaseConnection().query<NamedEntity[]>(sql, [platformId, ids])+}++const NAME_QUERIES: Partial<Record<AiUsageDimension, string>> = {+ project: `SELECT p."id", p."displayName" AS "name", NULL AS "parent", NULL AS "provider"+ FROM "project" p WHERE p."platformId" = $1 AND p."deleted" IS NULL AND p."id" = ANY($2)`,+ flow: `SELECT DISTINCT ON (fv."flowId") fv."flowId" AS "id", fv."displayName" AS "name", p."displayName" AS "parent", NULL AS "provider"+ FROM "flow_version" fv+ JOIN "flow" f ON f."id" = fv."flowId"+ JOIN "project" p ON p."id" = f."projectId"+ WHERE p."platformId" = $1 AND p."deleted" IS NULL AND fv."flowId" = ANY($2)+ ORDER BY fv."flowId", fv."created" DESC`,+ agent: `SELECT a."id", a."displayName" AS "name", p."displayName" AS "parent", NULL AS "provider"+ FROM "agent" a JOIN "project" p ON p."id" = a."projectId"+ WHERE p."platformId" = $1 AND p."deleted" IS NULL AND a."id" = ANY($2)`,+ user: `SELECT u."id", nullif(trim(concat_ws(' ', ui."firstName", ui."lastName")), '') AS "name", ui."email" AS "parent", NULL AS "provider"+ FROM "user" u JOIN "user_identity" ui ON ui."id" = u."identityId"+ WHERE u."platformId" = $1 AND u."id" = ANY($2)`,+ key: `SELECT k."id", k."displayName" AS "name", NULL AS "parent", k."provider"+ FROM "ai_provider" k WHERE k."platformId" = $1 AND k."id" = ANY($2)`,+}++export const aiUsageNames = {+ async lookup({ platformId, ids }: { platformId: string, ids: Partial<Record<AiUsageDimension, string[]>> }): Promise<NameBook> {+ const entries = await Promise.all(Object.entries(ids).map(async ([dimension, dimensionIds]) => {+ const named = await namesOf({ platformId, dimension: AiUsageDimension.parse(dimension), ids: dimensionIds ?? [] })+ return named.map((entity): [string, NamedEntity] => [`${dimension}:${entity.id}`, entity])+ }))+ const book = new Map(entries.flat())+ return {+ of: ({ dimension, id }) => book.get(`${dimension}:${id}`),+ }+ },+}++export type NamedEntity = {+ id: string+ name: string | null+ parent: string | null+ provider: string | null+}++export type NameBook = {+ of: (params: { dimension: AiUsageDimension, id: string }) => NamedEntity | undefined+}diff --git a/packages/server/api/src/app/ai/usage/ai-usage-period.ts b/packages/server/api/src/app/ai/usage/ai-usage-period.tsnew file mode 100644index 00000000000..6a821614ffc--- /dev/null+++ b/packages/server/api/src/app/ai/usage/ai-usage-period.ts@@ -0,0 +1,153 @@+import { AiUsagePeriod } from '@activepieces/shared'++function subDay({ now, spanMinutes, stepMinutes, ttlSeconds }: SubDayParams): ResolvedPeriod {+ const span = spanMinutes * MINUTE_MS+ const end = Math.floor(now.getTime() / FIVE_MINUTES_MS) * FIVE_MINUTES_MS + FIVE_MINUTES_MS+ const start = end - span+ const slotCount = spanMinutes / stepMinutes+ return {+ table: 'ai_usage_5m',+ stepMs: stepMinutes * MINUTE_MS,+ current: { start: new Date(start), end: new Date(end) },+ previous: { start: new Date(start - span), end: new Date(start) },+ slotCount,+ previousSlotCount: slotCount,+ scale: null,+ ttlSeconds,+ }+}++function lastDays({ now, days, ttlSeconds }: { now: Date, days: number, ttlSeconds: number }): ResolvedPeriod {+ const today = dayStart(now)+ const currentStart = today - (days - 1) * DAY_MS+ return daily({+ now,+ current: { start: currentStart, end: today + DAY_MS },+ previous: { start: currentStart - days * DAY_MS, end: currentStart },+ ttlSeconds,+ })+}++function thisMonth(now: Date): ResolvedPeriod {+ const today = dayStart(now)+ const monthStart = Date.UTC(now.getUTCFullYear(), now.getUTCMonth(), 1)+ const previousStart = Date.UTC(now.getUTCFullYear(), now.getUTCMonth() - 1, 1)+ const daysSoFar = (today - monthStart) / DAY_MS + 1+ const previousDays = Math.min(daysSoFar, (monthStart - previousStart) / DAY_MS)+ return daily({+ now,+ current: { start: monthStart, end: today + DAY_MS },+ previous: { start: previousStart, end: previousStart + previousDays * DAY_MS },+ ttlSeconds: THIS_MONTH_TTL_SECONDS,+ })+}++function lastMonth(now: Date): ResolvedPeriod {+ const year = now.getUTCFullYear()+ const month = now.getUTCMonth()+ return daily({+ now,+ current: { start: Date.UTC(year, month - 1, 1), end: Date.UTC(year, month, 1) },+ previous: { start: Date.UTC(year, month - 2, 1), end: Date.UTC(year, month - 1, 1) },+ ttlSeconds: LAST_MONTH_TTL_SECONDS,+ })+}++function daily({ now, current, previous, ttlSeconds }: DailyParams): ResolvedPeriod {+ const slotCount = (current.end - current.start) / DAY_MS+ return {+ table: 'ai_usage_daily',+ stepMs: DAY_MS,+ current: { start: new Date(current.start), end: new Date(current.end) },+ previous: { start: new Date(previous.start), end: new Date(previous.end) },+ slotCount,+ previousSlotCount: Math.min(slotCount, (previous.end - previous.start) / DAY_MS),+ scale: scaleOf({ now, current, previous }),+ ttlSeconds,+ }+}++function scaleOf({ now, current, previous }: Pick<DailyParams, 'now' | 'current' | 'previous'>): ResolvedPeriod['scale'] {+ const today = dayStart(now)+ if (current.end <= now.getTime()) {+ return null+ }+ const day = previous.start + (today - current.start)+ if (day >= previous.end) {+ return null+ }+ return { day: new Date(day), fraction: (now.getTime() - today) / DAY_MS }+}++function dayStart(date: Date): number {+ return Date.UTC(date.getUTCFullYear(), date.getUTCMonth(), date.getUTCDate())+}++const MINUTE_MS = 60 * 1000+const FIVE_MINUTES_MS = 5 * MINUTE_MS+const DAY_MS = 24 * 60 * MINUTE_MS+const THIS_MONTH_TTL_SECONDS = 600+const LAST_MONTH_TTL_SECONDS = 3600++export const aiUsagePeriod = {+ resolve({ period, now }: { period: AiUsagePeriod, now: Date }): ResolvedPeriod {+ switch (period) {+ case '1h':+ return subDay({ now, spanMinutes: 60, stepMinutes: 5, ttlSeconds: 60 })+ case '6h':+ return subDay({ now, spanMinutes: 6 * 60, stepMinutes: 15, ttlSeconds: 60 })+ case '12h':+ return subDay({ now, spanMinutes: 12 * 60, stepMinutes: 30, ttlSeconds: 120 })+ case '24h':+ return subDay({ now, spanMinutes: 24 * 60, stepMinutes: 60, ttlSeconds: 120 })+ case '7d':+ return lastDays({ now, days: 7, ttlSeconds: 300 })+ case '30d':+ return lastDays({ now, days: 30, ttlSeconds: 600 })+ case '90d':+ return lastDays({ now, days: 90, ttlSeconds: 900 })+ case 'this-month':+ return thisMonth(now)+ case 'last-month':+ return lastMonth(now)+ }+ },+ labelOf({ resolved, slot }: { resolved: ResolvedPeriod, slot: number }): string {+ const iso = new Date(resolved.current.start.getTime() + slot * resolved.stepMs).toISOString()+ return resolved.table === 'ai_usage_daily' ? iso.slice(0, 10) : iso+ },+ dayOf(date: Date): string {+ return date.toISOString().slice(0, 10)+ },+ DAY_MS,+}++export type ResolvedPeriod = {+ table: 'ai_usage_5m' | 'ai_usage_daily'+ stepMs: number+ current: TimeRange+ previous: TimeRange+ slotCount: number+ previousSlotCount: number+ scale: { day: Date, fraction: number } | null+ ttlSeconds: number+}++export type TimeRange = {+ start: Date+ end: Date+}++type SubDayParams = {+ now: Date+ spanMinutes: number+ stepMinutes: number+ ttlSeconds: number+}++type DailyParams = {+ now: Date+ current: { start: number, end: number }+ previous: { start: number, end: number }+ ttlSeconds: number+}diff --git a/packages/server/api/src/app/ai/usage/ai-usage-read.service.ts b/packages/server/api/src/app/ai/usage/ai-usage-read.service.tsnew file mode 100644index 00000000000..73f10dee41e--- /dev/null+++ b/packages/server/api/src/app/ai/usage/ai-usage-read.service.ts@@ -0,0 +1,525 @@+import { AIProviderName, AiUsageEnvironment, AiUsageOrigin, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'+import { modelCatalog } from '@activepieces/server-utils'+import { AI_USAGE_RELATED_DIMENSIONS, AI_USAGE_SOURCE_DIMENSIONS, AIProviderModelMetadata, AiUsageDetail, AiUsageDimension, AiUsageHistory, AiUsageItem, AiUsageItemList, AiUsageOverview, AiUsagePeriod, AiUsageScope, AiUsageSeriesPoint, AiUsageSource, AiUsageSourceSpend, AiUsageTotals } from '@activepieces/shared'+import { FastifyBaseLogger } from 'fastify'+import { transaction } from '../../core/db/transaction'+import { databaseConnection } from '../../database/database-connection'+import { distributedStore } from '../../database/redis-connections'+import { platformService } from '../../platform/platform.service'+import { aiUsageDimension, DimensionCondition, DimensionKeys } from './ai-usage-dimension'+import { aiUsageNames, NameBook } from './ai-usage-names'+import { aiUsagePeriod, ResolvedPeriod } from './ai-usage-period'++function maskOf({ set, columns }: { set: SetName, columns: GroupColumn[] }): number {+ return columns.reduce((mask, column, index) => SETS[set].includes(column) ? mask : mask | (1 << (columns.length - 1 - index)), 0)+}++function groupColumnsOf(sets: SetName[]): GroupColumn[] {+ return GROUP_COLUMNS.filter((column) => sets.some((set) => SETS[set].includes(column)))+}++function buildQuery({ platformId, resolved, scope, conditions, sets, rankedSets, alwaysSets, topN }: QueryParams): BuiltQuery {+ const values: unknown[] = []+ const param = (value: unknown, cast: string): string => {+ values.push(value)+ return `$${values.length}::${cast}`+ }+ const daily = resolved.table === 'ai_usage_daily'+ const time = daily ? '"day"' : '"bucket"'+ const timeCast = daily ? 'date' : 'timestamptz'+ const timeValue = (date: Date): string => daily ? aiUsagePeriod.dayOf(date) : date.toISOString()+ const platform = param(platformId, 'varchar')+ const previousStart = param(timeValue(resolved.previous.start), timeCast)+ const previousEnd = param(timeValue(resolved.previous.end), timeCast)+ const currentStart = param(timeValue(resolved.current.start), timeCast)+ const currentEnd = param(timeValue(resolved.current.end), timeCast)+ const rangeStart = `CASE WHEN ${time} >= ${currentStart} THEN ${currentStart} ELSE ${previousStart} END`+ const rawSlot = daily ? `(${time} - ${rangeStart})` : `floor(extract(epoch FROM ${time} - ${rangeStart}) / ${param(resolved.stepMs / 1000, 'float8')})::int`+ const slot = `least(${rawSlot}, ${param(resolved.slotCount - 1, 'int')})`+ const weight = !daily || isNil(resolved.scale)+ ? '1'+ : `CASE WHEN ${time} < ${currentStart} AND ${time} = ${param(aiUsagePeriod.dayOf(resolved.scale.day), 'date')} THEN ${param(resolved.scale.fraction, 'float8')} ELSE 1 END`+ const environment = SCOPE_ENVIRONMENT[scope]+ const scopeFilter = isNil(environment) ? '' : `AND "environment" = ${param(environment, 'varchar')}`+ const dimensionFilter = conditions.map(({ column, value }) => isNil(value) ? `AND "${column}" IS NULL` : `AND "${column}" = ${param(value, 'varchar')}`).join(' ')+ const columns = groupColumnsOf(sets)+ const quoted = columns.map((column) => `"${column}"`)+ const maskFor = (set: SetName): number => maskOf({ set, columns })+ const requiredKeyDrops = sets.flatMap((set) => {+ const required = REQUIRED_KEY[set]+ return isNil(required) ? [] : [`NOT (g = ${maskFor(set)} AND "${required}" IS NULL)`]+ })+ const projectCounts = sets.includes('project')+ ? `, count(*) FILTER (WHERE g = ${maskFor('flow')} AND "calls" > 0) OVER (PARTITION BY "projectId")::int AS "flowCount",+ count(*) FILTER (WHERE g = ${maskFor('projectAgent')} AND "calls" > 0) OVER (PARTITION BY "projectId")::int AS "agentCount"`+ : ''+ const groupingSets = sets.map((set) => `(${SETS[set].map((column) => `"${column}"`).join(', ')})`).join(', ')+ const tieBreak = quoted.filter((column) => column !== '"slot"' && column !== '"origin"')+ const sql = `+ WITH base AS (+ SELECT "projectId", "flowId", "agentId", "userId", "provider", "providerConfigId", "modelId", "origin",+ "spendUsd", "calls", "inputTokens", "outputTokens", "cacheReadTokens", "managedCredits", "runs", "reportedCostUsd", "estimatedCostUsd",+ ${time} >= ${currentStart} AS cur,+ ${weight} AS w,+ ${slot} AS "slot"+ FROM "${resolved.table}"+ WHERE "platformId" = ${platform}+ AND ((${time} >= ${previousStart} AND ${time} < ${previousEnd}) OR (${time} >= ${currentStart} AND ${time} < ${currentEnd}))+ ${scopeFilter}+ ${dimensionFilter}+ ), agg AS (+ SELECT GROUPING(${quoted.join(', ')})::int AS g, ${quoted.join(', ')},+ coalesce(sum("spendUsd") FILTER (WHERE cur), 0)::float8 AS "spend",+ coalesce(sum("spendUsd" * w) FILTER (WHERE NOT cur), 0)::float8 AS "previousSpend",+ coalesce(sum("calls") FILTER (WHERE cur), 0)::float8 AS "calls",+ coalesce(sum("calls" * w) FILTER (WHERE NOT cur), 0)::float8 AS "previousCalls",+ coalesce(sum("inputTokens" + "outputTokens") FILTER (WHERE cur), 0)::float8 AS "tokens",+ coalesce(sum(("inputTokens" + "outputTokens") * w) FILTER (WHERE NOT cur), 0)::float8 AS "previousTokens",+ coalesce(sum("inputTokens") FILTER (WHERE cur), 0)::float8 AS "inputTokens",+ coalesce(sum("cacheReadTokens") FILTER (WHERE cur), 0)::float8 AS "cacheReadTokens",+ coalesce(sum("managedCredits") FILTER (WHERE cur), 0)::float8 AS "managedCredits",+ coalesce(sum("runs") FILTER (WHERE cur), 0)::float8 AS "runs",+ coalesce(sum("reportedCostUsd") FILTER (WHERE cur), 0)::float8 AS "reportedUsd",+ coalesce(sum("estimatedCostUsd") FILTER (WHERE cur), 0)::float8 AS "estimatedUsd",+ coalesce(sum("spendUsd") FILTER (WHERE cur AND "origin" = '${AiUsageOrigin.FLOW_AGENT_STEP}' AND "flowId" IS NOT NULL AND "agentId" IS NOT NULL), 0)::float8 AS "sharedUsd"+ FROM base+ GROUP BY GROUPING SETS (${groupingSets})+ ), ranked AS (+ SELECT agg.*,+ row_number() OVER (PARTITION BY g ORDER BY "spend" DESC, "calls" DESC${tieBreak.length > 0 ? `, ${tieBreak.join(', ')}` : ''})::int AS "rank",+ count(*) FILTER (WHERE "calls" > 0) OVER (PARTITION BY g)::int AS "setCount",+ sum("spend") OVER (PARTITION BY g)::float8 AS "setSpend"+ ${projectCounts}+ FROM agg+ ${requiredKeyDrops.length > 0 ? `WHERE ${requiredKeyDrops.join(' AND ')}` : ''}+ )+ SELECT ranked.*, (SELECT EXISTS (SELECT 1 FROM "ai_usage_daily" WHERE "platformId" = ${platform})) AS "hasAnyUsage"+ FROM ranked+ WHERE g = ANY(${param(alwaysSets.map(maskFor), 'int[]')})+ OR (g = ANY(${param(rankedSets.map(maskFor), 'int[]')}) AND "rank" <= ${param(topN, 'int')} AND "calls" > 0)+ ORDER BY g, "rank"`+ return { sql, values, maskFor }+}++async function queryRows(query: BuiltQuery): Promise<UsageRow[]> {+ return transaction(async (entityManager) => {+ await entityManager.query(`SET LOCAL statement_timeout = ${STATEMENT_TIMEOUT_MS}`)+ const rows: UsageRow[] = await entityManager.query(query.sql, query.values)+ return rows+ })+}++async function trackingStartOf(platformId: string): Promise<number | null> {+ const rows: { startedAtMs: string }[] = await databaseConnection().query(+ 'SELECT (extract(epoch FROM "startedAt") * 1000)::bigint AS "startedAtMs" FROM "ai_usage_copy_cursor" WHERE "platformId" = $1',+ [platformId],+ )+ return isNil(rows[0]) ? null : Number(rows[0].startedAtMs)+}++function historyOf({ resolved, since, hasAnyUsage, now }: { resolved: ResolvedPeriod, since: number | null, hasAnyUsage: boolean, now: number }): AiUsageHistory {+ const start = resolved.current.start.getTime()+ const end = Math.min(resolved.current.end.getTime(), now)+ const periodDays = Math.max(1, Math.round((resolved.current.end.getTime() - start) / aiUsagePeriod.DAY_MS))+ if (isNil(since)) {+ return { trackingSince: null, hasAnyUsage, coveredDays: periodDays, periodDays, currentCoverage: 1, comparable: false }+ }+ const coveredMs = Math.max(0, end - Math.max(start, since))+ const coveredDays = Math.min(periodDays, Math.ceil((resolved.current.end.getTime() - Math.max(start, since)) / aiUsagePeriod.DAY_MS))+ return {+ trackingSince: new Date(since).toISOString(),+ hasAnyUsage,+ coveredDays: Math.max(0, coveredDays),+ periodDays,+ currentCoverage: end > start ? Math.min(1, coveredMs / (end - start)) : 1,+ comparable: since <= resolved.previous.start.getTime(),+ }+}++function totalsOf(row: UsageRow | undefined): AiUsageTotals {+ const totals = row ?? EMPTY_ROW+ return {+ spendUsd: totals.spend,+ previousSpendUsd: totals.previousSpend,+ calls: totals.calls,+ previousCalls: totals.previousCalls,+ tokens: totals.tokens,+ previousTokens: totals.previousTokens,+ cacheShare: totals.inputTokens > 0 ? totals.cacheReadTokens / totals.inputTokens : 0,+ managedCredits: totals.managedCredits,+ }+}++function seriesOf({ rows, resolved, since }: { rows: UsageRow[], resolved: ResolvedPeriod, since: number | null }): AiUsageSeriesPoint[] {+ const bySlot = new Map(rows.map((row) => [row.slot, row]))+ const untracked = (slotStart: number): boolean => !isNil(since) && slotStart + resolved.stepMs <= since+ return Array.from({ length: resolved.slotCount }, (_, slot) => {+ const row = bySlot.get(slot) ?? EMPTY_ROW+ const currentMissing = untracked(resolved.current.start.getTime() + slot * resolved.stepMs)+ const previousMissing = slot >= resolved.previousSlotCount || untracked(resolved.previous.start.getTime() + slot * resolved.stepMs)+ return {+ label: aiUsagePeriod.labelOf({ resolved, slot }),+ spendUsd: currentMissing ? null : row.spend,+ calls: currentMissing ? null : row.calls,+ tokens: currentMissing ? null : row.tokens,+ previousSpendUsd: previousMissing ? null : row.previousSpend,+ previousCalls: previousMissing ? null : row.previousCalls,+ previousTokens: previousMissing ? null : row.previousTokens,+ }+ })+}++function bySourceOf(rows: UsageRow[]): AiUsageSourceSpend[] {+ const spendBySource = rows.reduce((totals, row) => {+ const origin = ORIGINS.find((value) => value === row.origin)+ if (isNil(origin)) {+ return totals+ }+ const source = aiUsageDimension.sourceOf(origin)+ return new Map([...totals, [source, (totals.get(source) ?? 0) + row.spend]])+ }, new Map<AiUsageSource, number>())+ return [...spendBySource.entries()]+ .filter(([, spendUsd]) => spendUsd > 0)+ .sort(([, a], [, b]) => b - a)+ .map(([source, spendUsd]) => ({ source, spendUsd }))+}++function itemOf({ dimension, row, names, prices, comparable }: ItemParams): AiUsageItem | null {+ const keys = keysOf(row)+ const id = aiUsageDimension.idOf({ dimension, keys })+ if (isNil(id)) {+ return null+ }+ const named = names.of({ dimension, id })+ const providerName = keys.provider ?? named?.provider ?? null+ const provider = isNil(providerName) ? null : aiUsageDimension.providerOf(providerName)+ const price = dimension === 'model' && !isNil(provider) && !isNil(keys.modelId) ? prices({ provider, modelId: keys.modelId }) : undefined+ return {+ dimension,+ id,+ name: dimension === 'model' ? keys.modelId : named?.name ?? null,+ ...(isNil(named?.parent) ? {} : { parent: named.parent }),+ ...((dimension === 'key' || dimension === 'model') && !isNil(provider) ? { provider } : {}),+ ...(dimension === 'project' ? { flowCount: row.flowCount ?? 0, agentCount: row.agentCount ?? 0 } : {}),+ spendUsd: row.spend,+ previousSpendUsd: row.previousSpend,+ calls: row.calls,+ tokens: row.tokens,+ comparable: comparable && row.previousSpend > 0,+ ...((dimension === 'flow' || dimension === 'agent') ? { sharedUsd: row.sharedUsd } : {}),+ ...(isNil(price?.inputCostPerMillionTokens) ? {} : { inputPrice: price.inputCostPerMillionTokens }),+ ...(isNil(price?.outputCostPerMillionTokens) ? {} : { outputPrice: price.outputCostPerMillionTokens }),+ }+}++function keysOf(row: UsageRow): DimensionKeys {+ return {+ projectId: row.projectId ?? null,+ flowId: row.flowId ?? null,+ agentId: row.agentId ?? null,+ userId: row.userId ?? null,+ provider: row.provider ?? null,+ providerConfigId: row.providerConfigId ?? null,+ modelId: row.modelId ?? null,+ }+}++function idsToName({ groups, extra }: { groups: { dimension: AiUsageDimension, rows: UsageRow[] }[], extra?: { dimension: AiUsageDimension, id: string } }): Partial<Record<AiUsageDimension, string[]>> {+ const pairs = [+ ...groups.flatMap(({ dimension, rows }) => rows.map((row) => ({ dimension, id: aiUsageDimension.idOf({ dimension, keys: keysOf(row) }) }))),+ ...(isNil(extra) ? [] : [extra]),+ ]+ return pairs.reduce<Partial<Record<AiUsageDimension, string[]>>>((ids, { dimension, id }) => isNil(id) ? ids : { ...ids, [dimension]: [...(ids[dimension] ?? []), id] }, {})+}++async function priceBook(): Promise<ItemParams['prices']> {+ const catalog = await modelCatalog.load()+ return ({ provider, modelId }) => catalog.lookup({ provider, modelId })+}++async function cachedView<T>({ key, ttlSeconds, compute }: { key: string, ttlSeconds: number, compute: () => Promise<T> }): Promise<T> {+ const { data: hit } = await tryCatch(() => distributedStore.get<T>(key))+ if (!isNil(hit)) {+ return hit+ }+ const value = await compute()+ await tryCatch(() => distributedStore.put(key, value, ttlSeconds))+ return value+}++function cacheKeyOf({ platformId, endpoint, dimension, id, period, scope, resolved }: CacheKeyParams): string {+ return `${CACHE_PREFIX}:${platformId}:${endpoint}:${dimension ?? '*'}:${id ?? '*'}:${period}:${scope}:${resolved.current.start.toISOString()}:${resolved.current.end.toISOString()}`+}++function setsWithHelpers(sets: SetName[]): SetName[] {+ const helpers: SetName[] = sets.includes('project') ? ['flow', 'projectAgent'] : []+ return [...new Set([...sets, ...helpers])]+}++async function sinceOf({ platformId, log }: { platformId: string, log: FastifyBaseLogger }): Promise<number | null> {+ const [first, platform] = await Promise.all([trackingStartOf(platformId), platformService(log).getOneOrThrow(platformId)])+ return isNil(first) ? null : Math.max(first, new Date(platform.created).getTime())+}++function keysOfItem(conditions: DimensionCondition[] | null): Partial<UsageRow> {+ const values = Object.fromEntries((conditions ?? []).map(({ column, value }) => [column, value]))+ return {+ projectId: values['projectId'] ?? null,+ flowId: values['flowId'] ?? null,+ agentId: values['agentId'] ?? null,+ userId: values['userId'] ?? null,+ provider: values['provider'] ?? null,+ providerConfigId: values['providerConfigId'] ?? null,+ modelId: values['modelId'] ?? null,+ }+}++function rangesOf(resolved: ResolvedPeriod): AiUsageOverview['ranges'] {+ return {+ current: { start: resolved.current.start.toISOString(), end: resolved.current.end.toISOString() },+ previous: { start: resolved.previous.start.toISOString(), end: resolved.previous.end.toISOString() },+ }+}++const GROUP_COLUMNS = ['slot', 'origin', 'projectId', 'flowId', 'agentId', 'userId', 'provider', 'providerConfigId', 'modelId'] as const++const SETS: Record<SetName, readonly GroupColumn[]> = {+ total: [],+ slot: ['slot'],+ origin: ['origin'],+ project: ['projectId'],+ flow: ['projectId', 'flowId'],+ projectAgent: ['projectId', 'agentId'],+ agent: ['agentId'],+ user: ['userId'],+ key: ['provider', 'providerConfigId'],+ model: ['provider', 'modelId'],+}++const REQUIRED_KEY: Partial<Record<SetName, GroupColumn>> = {+ flow: 'flowId',+ projectAgent: 'agentId',+ agent: 'agentId',+ user: 'userId',+}++const DIMENSION_SETS: Record<AiUsageDimension, SetName> = {+ project: 'project',+ flow: 'flow',+ agent: 'agent',+ user: 'user',+ key: 'key',+ model: 'model',+}++const SCOPE_ENVIRONMENT: Record<AiUsageScope, AiUsageEnvironment | null> = {+ all: null,+ production: AiUsageEnvironment.PRODUCTION,+ testing: AiUsageEnvironment.TESTING,+}++const EMPTY_ROW: UsageRow = {+ g: 0,+ spend: 0,+ previousSpend: 0,+ calls: 0,+ previousCalls: 0,+ tokens: 0,+ previousTokens: 0,+ inputTokens: 0,+ cacheReadTokens: 0,+ managedCredits: 0,+ runs: 0,+ reportedUsd: 0,+ estimatedUsd: 0,+ sharedUsd: 0,+ rank: 0,+ setCount: 0,+ setSpend: 0,+ hasAnyUsage: false,+}++const ORIGINS = Object.values(AiUsageOrigin)+const AI_USAGE_DIMENSIONS_IN_ORDER: AiUsageDimension[] = ['project', 'flow', 'agent', 'user', 'key', 'model']+const TOP_SIZE = 5+const LIST_LIMIT = 1000+const LIST_AND_DETAIL_MAX_TTL_SECONDS = 5 * 60+const STATEMENT_TIMEOUT_MS = 5000+const CACHE_PREFIX = 'ai-usage:v1'++export const aiUsageReadService = (log: FastifyBaseLogger) => ({+ async overview({ platformId, period, scope }: { platformId: string, period: AiUsagePeriod, scope: AiUsageScope }): Promise<AiUsageOverview> {+ const now = new Date()+ const resolved = aiUsagePeriod.resolve({ period, now })+ return cachedView({+ key: cacheKeyOf({ platformId, endpoint: 'overview', period, scope, resolved }),+ ttlSeconds: resolved.ttlSeconds,+ compute: async () => {+ const rankedSets: SetName[] = ['project', 'flow', 'agent', 'user', 'key', 'model']+ const query = buildQuery({ platformId, resolved, scope, conditions: [], sets: setsWithHelpers(['total', 'slot', 'origin', ...rankedSets]), rankedSets, alwaysSets: ['total', 'slot', 'origin'], topN: TOP_SIZE })+ const [rows, since, prices] = await Promise.all([queryRows(query), sinceOf({ platformId, log }), priceBook()])+ const setRows = (set: SetName): UsageRow[] => rows.filter((row) => row.g === query.maskFor(set))+ const history = historyOf({ resolved, since, hasAnyUsage: rows[0]?.hasAnyUsage ?? false, now: now.getTime() })+ const groups = AI_USAGE_DIMENSIONS_IN_ORDER.map((dimension) => ({ dimension, rows: setRows(DIMENSION_SETS[dimension]) }))+ const names = await aiUsageNames.lookup({ platformId, ids: idsToName({ groups }) })+ const itemsOf = (dimension: AiUsageDimension): AiUsageItem[] => setRows(DIMENSION_SETS[dimension])+ .flatMap((row) => itemOf({ dimension, row, names, prices, comparable: history.comparable }) ?? [])+ return {+ history,+ ranges: rangesOf(resolved),+ totals: totalsOf(setRows('total')[0]),+ series: seriesOf({ rows: setRows('slot'), resolved, since }),+ bySource: bySourceOf(setRows('origin')),+ top: {+ project: itemsOf('project'),+ flow: itemsOf('flow'),+ agent: itemsOf('agent'),+ user: itemsOf('user'),+ key: itemsOf('key'),+ model: itemsOf('model'),+ },+ }+ },+ })+ },+ async items({ platformId, period, scope, dimension }: { platformId: string, period: AiUsagePeriod, scope: AiUsageScope, dimension: AiUsageDimension }): Promise<AiUsageItemList> {+ const now = new Date()+ const resolved = aiUsagePeriod.resolve({ period, now })+ return cachedView({+ key: cacheKeyOf({ platformId, endpoint: 'items', dimension, period, scope, resolved }),+ ttlSeconds: Math.min(resolved.ttlSeconds, LIST_AND_DETAIL_MAX_TTL_SECONDS),+ compute: async () => {+ const set = DIMENSION_SETS[dimension]+ const query = buildQuery({ platformId, resolved, scope, conditions: [], sets: setsWithHelpers(['total', set]), rankedSets: [set], alwaysSets: ['total'], topN: LIST_LIMIT })+ const [rows, since, prices] = await Promise.all([queryRows(query), sinceOf({ platformId, log }), priceBook()])+ const listRows = rows.filter((row) => row.g === query.maskFor(set))+ const history = historyOf({ resolved, since, hasAnyUsage: rows[0]?.hasAnyUsage ?? false, now: now.getTime() })+ const names = await aiUsageNames.lookup({ platformId, ids: idsToName({ groups: [{ dimension, rows: listRows }] }) })+ const items = listRows.flatMap((row) => itemOf({ dimension, row, names, prices, comparable: history.comparable }) ?? [])+ const count = listRows[0]?.setCount ?? 0+ return {+ history,+ ranges: rangesOf(resolved),+ items,+ count,+ itemsSpendUsd: listRows[0]?.setSpend ?? 0,+ totalSpendUsd: rows.find((row) => row.g === query.maskFor('total'))?.spend ?? 0,+ truncated: count > items.length,+ }+ },+ })+ },+ async detail({ platformId, period, scope, dimension, id }: { platformId: string, period: AiUsagePeriod, scope: AiUsageScope, dimension: AiUsageDimension, id: string }): Promise<AiUsageDetail> {+ const now = new Date()+ const resolved = aiUsagePeriod.resolve({ period, now })+ return cachedView({+ key: cacheKeyOf({ platformId, endpoint: 'detail', dimension, id, period, scope, resolved }),+ ttlSeconds: Math.min(resolved.ttlSeconds, LIST_AND_DETAIL_MAX_TTL_SECONDS),+ compute: async () => {+ const conditions = aiUsageDimension.filterOf({ dimension, id })+ const related = AI_USAGE_RELATED_DIMENSIONS[dimension]+ const rankedSets = related.map((relatedDimension) => DIMENSION_SETS[relatedDimension])+ const withSource = AI_USAGE_SOURCE_DIMENSIONS.includes(dimension)+ const alwaysSets: SetName[] = withSource ? ['total', 'slot', 'origin'] : ['total', 'slot']+ const query = buildQuery({ platformId, resolved, scope, conditions: conditions ?? [], sets: setsWithHelpers([...alwaysSets, ...rankedSets]), rankedSets, alwaysSets, topN: TOP_SIZE })+ const [rows, since, prices] = await Promise.all([isNil(conditions) ? Promise.resolve([]) : queryRows(query), sinceOf({ platformId, log }), priceBook()])+ const setRows = (set: SetName): UsageRow[] => rows.filter((row) => row.g === query.maskFor(set))+ const history = historyOf({ resolved, since, hasAnyUsage: rows[0]?.hasAnyUsage ?? false, now: now.getTime() })+ const groups = related.map((relatedDimension) => ({ dimension: relatedDimension, rows: setRows(DIMENSION_SETS[relatedDimension]) }))+ const names = await aiUsageNames.lookup({ platformId, ids: idsToName({ groups, ...spreadIfDefined('extra', isNil(conditions) ? undefined : { dimension, id }) }) })+ const totalRow = setRows('total')[0] ?? EMPTY_ROW+ const itemRow: UsageRow = {+ ...totalRow,+ ...keysOfItem(conditions),+ flowCount: setRows('flow')[0]?.setCount ?? 0,+ agentCount: setRows('agent')[0]?.setCount ?? 0,+ }+ const item = itemOf({ dimension, row: itemRow, names, prices, comparable: history.comparable }) ?? { dimension, id, name: null, spendUsd: 0, previousSpendUsd: 0, calls: 0, tokens: 0, comparable: false }+ return {+ history,+ ranges: rangesOf(resolved),+ item: { ...item, id },+ series: seriesOf({ rows: setRows('slot'), resolved, since }),+ related: Object.fromEntries(groups.map(({ dimension: relatedDimension, rows: relatedRows }) => [+ relatedDimension,+ relatedRows.flatMap((row) => itemOf({ dimension: relatedDimension, row, names, prices, comparable: history.comparable }) ?? []),+ ])),+ ...(withSource ? { bySource: bySourceOf(setRows('origin')) } : {}),+ ...(dimension === 'flow' ? { runs: totalRow.runs } : {}),+ ...(dimension === 'key' ? { reportedUsd: totalRow.reportedUsd, estimatedUsd: totalRow.estimatedUsd } : {}),+ }+ },+ })+ },+})++type GroupColumn = typeof GROUP_COLUMNS[number]++type SetName = 'total' | 'slot' | 'origin' | 'project' | 'flow' | 'projectAgent' | 'agent' | 'user' | 'key' | 'model'++type QueryParams = {+ platformId: string+ resolved: ResolvedPeriod+ scope: AiUsageScope+ conditions: DimensionCondition[]+ sets: SetName[]+ rankedSets: SetName[]+ alwaysSets: SetName[]+ topN: number+}++type BuiltQuery = {+ sql: string+ values: unknown[]+ maskFor: (set: SetName) => number+}++type UsageRow = {+ g: number+ slot?: number | null+ origin?: string | null+ projectId?: string | null+ flowId?: string | null+ agentId?: string | null+ userId?: string | null+ provider?: string | null+ providerConfigId?: string | null+ modelId?: string | null+ spend: number+ previousSpend: number+ calls: number+ previousCalls: number+ tokens: number+ previousTokens: number+ inputTokens: number+ cacheReadTokens: number+ managedCredits: number+ runs: number+ reportedUsd: number+ estimatedUsd: number+ sharedUsd: number+ rank: number+ setCount: number+ setSpend: number+ flowCount?: number+ agentCount?: number+ hasAnyUsage: boolean+}++type ItemParams = {+ dimension: AiUsageDimension+ row: UsageRow+ names: NameBook+ prices: (params: { provider: AIProviderName, modelId: string }) => AIProviderModelMetadata | undefined+ comparable: boolean+}++type CacheKeyParams = {+ platformId: string+ endpoint: 'overview' | 'items' | 'detail'+ dimension?: AiUsageDimension+ id?: string+ period: AiUsagePeriod+ scope: AiUsageScope+ resolved: ResolvedPeriod+}diff --git a/packages/server/api/src/app/ai/usage/ai-usage.controller.ts b/packages/server/api/src/app/ai/usage/ai-usage.controller.tsnew file mode 100644index 00000000000..dd5dd674cc3--- /dev/null+++ b/packages/server/api/src/app/ai/usage/ai-usage.controller.ts@@ -0,0 +1,64 @@+import { AiUsageDetail, AiUsageDetailParams, AiUsageItemList, AiUsageItemsQuery, AiUsageOverview, AiUsageQuery, PrincipalType } from '@activepieces/shared'+import { RateLimitOptions } from '@fastify/rate-limit'+import { FastifyRequest } from 'fastify'+import { FastifyPluginAsyncZod } from 'fastify-type-provider-zod'+import { StatusCodes } from 'http-status-codes'+import { securityAccess } from '../../core/security/authorization/fastify-security'+import { aiProviderService } from '../ai-provider-service'+import { aiUsageReadService } from './ai-usage-read.service'++export const aiUsageController: FastifyPluginAsyncZod = async (app) => {+ app.get('/overview', GetOverview, async (request) => {+ return aiUsageReadService(request.log).overview({ platformId: request.principal.platform.id, ...request.query })+ })+ app.get('/items', ListItems, async (request) => {+ return aiUsageReadService(request.log).items({ platformId: request.principal.platform.id, ...request.query })+ })+ app.get('/items/:dimension/:id', GetDetail, async (request) => {+ return aiUsageReadService(request.log).detail({ platformId: request.principal.platform.id, ...request.query, ...request.params })+ })+}++function assertPlan(request: FastifyRequest): Promise<void> {+ return aiProviderService(request.log).assertAiProvidersPlan(request.principal.platform.id)+}++const RATE_LIMIT: RateLimitOptions = {+ max: 60,+ timeWindow: '1 minute',+ hook: 'preHandler',+ keyGenerator: (request) => `ai-usage:${request.principal.id}`,+}++const READ_CONFIG = {+ security: securityAccess.platformAdminOnly([PrincipalType.USER]),+ rateLimit: RATE_LIMIT,+}++const GetOverview = {+ config: READ_CONFIG,+ preHandler: assertPlan,+ schema: {+ querystring: AiUsageQuery,+ response: { [StatusCodes.OK]: AiUsageOverview },+ },+}++const ListItems = {+ config: READ_CONFIG,+ preHandler: assertPlan,+ schema: {+ querystring: AiUsageItemsQuery,+ response: { [StatusCodes.OK]: AiUsageItemList },+ },+}++const GetDetail = {+ config: READ_CONFIG,+ preHandler: assertPlan,+ schema: {+ params: AiUsageDetailParams,+ querystring: AiUsageQuery,+ response: { [StatusCodes.OK]: AiUsageDetail },+ },+}diff --git a/packages/server/api/src/app/ai/usage/ai-usage.module.ts b/packages/server/api/src/app/ai/usage/ai-usage.module.tsnew file mode 100644index 00000000000..d333cfac78f--- /dev/null+++ b/packages/server/api/src/app/ai/usage/ai-usage.module.ts@@ -0,0 +1,6 @@+import { FastifyPluginAsyncZod } from 'fastify-type-provider-zod'+import { aiUsageController } from './ai-usage.controller'++export const aiUsageModule: FastifyPluginAsyncZod = async (app) => {+ await app.register(aiUsageController, { prefix: '/v1/ai-usage' })+}diff --git a/packages/server/api/src/app/app.ts b/packages/server/api/src/app/app.tsindex 712981b5c70..4f9d2dcc2a3 100644--- a/packages/server/api/src/app/app.ts+++ b/packages/server/api/src/app/app.ts@@ -18,6 +18,7 @@ import { aiUsageHooks } from './ai/ai-usage-hooks'import { aiUsageService } from './ai/ai-usage-service'import { platformModelTierModule } from './ai/platform-model-tier.module'import { aiUsageJobs } from './ai/usage/ai-usage-jobs'+import { aiUsageModule } from './ai/usage/ai-usage.module'import { platformAnalyticsModule } from './analytics/platform-analytics.module'import { setPlatformOAuthService } from './app-connection/app-connection-service/oauth2'import { appConnectionModule } from './app-connection/app-connection.module'@@ -399,6 +400,7 @@ export const setupApp = async (app: FastifyInstance): Promise<FastifyInstance> =publishHooksFactory.set(eeFlowPublishHook)billingProvider.set(autumnBillingProvider)await aiUsageJobs.register(app.log)+ await app.register(aiUsageModule)resumePageHooks.set((log) => ({ getTheme: (params) => appearanceHelper.getTheme({ ...params, log }) }))verifiedDomainHooks.set(authnSsoSamlService)flowPublishHooks.set((log) => ({ assertReferencesResolve: assertAgentsResolveInProject, findMissingRequiredActions: (params) => pieceSetRequiredActions(log).findMissing(params) }))@@ -445,6 +447,7 @@ export const setupApp = async (app: FastifyInstance): Promise<FastifyInstance> =publishHooksFactory.set(eeFlowPublishHook)billingProvider.set(autumnBillingProvider)await aiUsageJobs.register(app.log)+ await app.register(aiUsageModule)resumePageHooks.set((log) => ({ getTheme: (params) => appearanceHelper.getTheme({ ...params, log }) }))flowPublishHooks.set((log) => ({ assertReferencesResolve: assertAgentsResolveInProject, findMissingRequiredActions: (params) => pieceSetRequiredActions(log).findMissing(params) }))aiUsageHooks.set(agentConversationCreditsHooks)diff --git a/packages/server/api/test/integration/ee/ai-usage/ai-usage-plan.test.ts b/packages/server/api/test/integration/ee/ai-usage/ai-usage-plan.test.tsnew file mode 100644index 00000000000..dd5b4a32644--- /dev/null+++ b/packages/server/api/test/integration/ee/ai-usage/ai-usage-plan.test.ts@@ -0,0 +1,35 @@+import { FastifyInstance } from 'fastify'+import { StatusCodes } from 'http-status-codes'+import { createTestContext } from '../../../helpers/test-context'+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'++let app: FastifyInstance++beforeAll(async () => {+ app = await setupTestEnvironment()+})++afterAll(async () => {+ await teardownTestEnvironment()+})++describe('AI usage plan gate', () => {+ it('refuses every usage view when the plan has no AI providers', async () => {+ const ctx = await createTestContext(app, { plan: { aiProvidersEnabled: false } })+ const query = { period: '30d', scope: 'production' }++ const overview = await ctx.get('/v1/ai-usage/overview', query)+ const items = await ctx.get('/v1/ai-usage/items', { ...query, dimension: 'flow' })+ const detail = await ctx.get('/v1/ai-usage/items/project/no-project', query)++ expect(overview.statusCode).toBe(StatusCodes.PAYMENT_REQUIRED)+ expect(items.statusCode).toBe(StatusCodes.PAYMENT_REQUIRED)+ expect(detail.statusCode).toBe(StatusCodes.PAYMENT_REQUIRED)+ })++ it('serves usage views when the plan has AI providers', async () => {+ const ctx = await createTestContext(app, { plan: { aiProvidersEnabled: true } })+ const overview = await ctx.get('/v1/ai-usage/overview', { period: '30d', scope: 'production' })+ expect(overview.statusCode).toBe(StatusCodes.OK)+ })+})diff --git a/packages/server/api/test/integration/ee/ai-usage/ai-usage-read.test.ts b/packages/server/api/test/integration/ee/ai-usage/ai-usage-read.test.tsnew file mode 100644index 00000000000..4cd107b605b--- /dev/null+++ b/packages/server/api/test/integration/ee/ai-usage/ai-usage-read.test.ts@@ -0,0 +1,307 @@+import { AIProviderName, AiUsageEnvironment, AiUsageOrigin, apId } from '@activepieces/core-utils'+import { AiUsageDetail, AiUsageItemList, AiUsageOverview, DefaultProjectRole, FlowVersionState } from '@activepieces/shared'+import { FastifyInstance } from 'fastify'+import { StatusCodes } from 'http-status-codes'+import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'+import { aiUsageDimension } from '../../../../src/app/ai/usage/ai-usage-dimension'+import { aiUsagePeriod } from '../../../../src/app/ai/usage/ai-usage-period'+import { databaseConnection } from '../../../../src/app/database/database-connection'+import { db } from '../../../helpers/db'+import { createMockFlow, createMockFlowVersion, createMockProject } from '../../../helpers/mocks'+import { createMemberContext, createTestContext, TestContext } from '../../../helpers/test-context'+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'++let app: FastifyInstance+let ctx: TestContext+let seriesCounter = 0++beforeAll(async () => {+ app = await setupTestEnvironment()+}, 300_000)++afterAll(async () => {+ await teardownTestEnvironment()+})++beforeEach(async () => {+ ctx = await createTestContext(app, { plan: { aiProvidersEnabled: true } })+ await db.update('platform', ctx.platform.id, { created: '2025-01-01T00:00:00.000Z' })+ await databaseConnection().query(+ 'INSERT INTO "ai_usage_copy_cursor" ("platformId", "copiedUntil", "startedAt") VALUES ($1, now(), $2)',+ [ctx.platform.id, '2025-06-01T00:00:00.000Z'],+ )+})++async function seed(row: SeedRow): Promise<void> {+ seriesCounter += 1+ const table = 'bucket' in row ? 'ai_usage_5m' : 'ai_usage_daily'+ const time = 'bucket' in row ? row.bucket : row.day+ await databaseConnection().query(+ `INSERT INTO "${table}" ("${'bucket' in row ? 'bucket' : 'day'}", "seriesId", "platformId", "projectId", "flowId", "agentId", "userId", "provider", "providerConfigId", "modelId", "origin", "environment",+ "calls", "runs", "inputTokens", "outputTokens", "cacheReadTokens", "spendUsd", "reportedCostUsd", "estimatedCostUsd", "managedCredits")+ VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20, $21)`,+ [+ time, String(seriesCounter).padStart(32, '0'), row.platformId ?? ctx.platform.id, row.projectId === undefined ? ctx.project.id : row.projectId,+ row.flowId ?? null, row.agentId ?? null, row.userId ?? null, row.provider ?? AIProviderName.OPENAI, row.providerConfigId ?? null, row.modelId ?? 'gpt-5',+ row.origin ?? AiUsageOrigin.FLOW_STEP, row.environment ?? AiUsageEnvironment.PRODUCTION,+ row.calls ?? 1, row.runs ?? 0, row.inputTokens ?? 0, row.outputTokens ?? 0, row.cacheReadTokens ?? 0,+ row.spendUsd ?? 0, row.reportedCostUsd ?? 0, row.estimatedCostUsd ?? row.spendUsd ?? 0, row.managedCredits ?? 0,+ ],+ )+}++async function saveFlow({ projectId, displayName }: { projectId: string, displayName: string }): Promise<string> {+ const flow = createMockFlow({ projectId })+ await db.save('flow', flow)+ await db.save('flow_version', createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.DRAFT, displayName }))+ return flow.id+}++async function getJson<T>({ url, query }: { url: string, query: Record<string, string> }): Promise<T> {+ const response = await ctx.get(url, query)+ expect(response.statusCode).toBe(StatusCodes.OK)+ return response.json()+}++function daysOf(period: '7d' | '30d' | 'last-month'): { current: string, previous: string, previousLast: string, scaleDay: string | null } {+ const resolved = aiUsagePeriod.resolve({ period, now: new Date() })+ return {+ current: aiUsagePeriod.dayOf(new Date(resolved.current.start.getTime() + aiUsagePeriod.DAY_MS)),+ previous: aiUsagePeriod.dayOf(resolved.previous.start),+ previousLast: aiUsagePeriod.dayOf(new Date(resolved.previous.end.getTime() - aiUsagePeriod.DAY_MS)),+ scaleDay: resolved.scale ? aiUsagePeriod.dayOf(resolved.scale.day) : null,+ }+}++function sum(values: (number | null)[]): number {+ return values.reduce<number>((total, value) => total + (value ?? 0), 0)+}++async function awayFromFiveMinuteEdge(): Promise<void> {+ const untilEdgeMs = FIVE_MINUTES_MS - (Date.now() % FIVE_MINUTES_MS)+ if (untilEdgeMs < EDGE_MARGIN_MS) {+ await new Promise((resolve) => setTimeout(resolve, untilEdgeMs + 100))+ }+}++describe('AI usage read API', () => {+ async function seedPlatform(): Promise<{ invoiceFlowId: string, quietFlowId: string, agentId: string }> {+ const { current, previous } = daysOf('30d')+ const invoiceFlowId = await saveFlow({ projectId: ctx.project.id, displayName: 'Invoice flow' })+ const quietFlowId = await saveFlow({ projectId: ctx.project.id, displayName: 'Quiet flow' })+ const agentId = apId()+ await seed({ day: current, flowId: invoiceFlowId, spendUsd: 10, reportedCostUsd: 6, estimatedCostUsd: 4, calls: 2, runs: 1, inputTokens: 100, outputTokens: 50, cacheReadTokens: 40 })+ await seed({ day: current, flowId: invoiceFlowId, agentId, origin: AiUsageOrigin.FLOW_AGENT_STEP, provider: AIProviderName.ACTIVEPIECES, modelId: 'openai/gpt-5', spendUsd: 5, calls: 1, runs: 1, managedCredits: 50 })+ await seed({ day: current, projectId: null, userId: ctx.user.id, origin: AiUsageOrigin.CHAT, provider: AIProviderName.ANTHROPIC, modelId: 'claude-sonnet-5', spendUsd: 3, calls: 1 })+ await seed({ day: current, flowId: quietFlowId, modelId: 'unpriced-model', spendUsd: 0, calls: 3 })+ await seed({ day: current, flowId: invoiceFlowId, environment: AiUsageEnvironment.TESTING, spendUsd: 7, calls: 1 })+ await seed({ day: previous, flowId: invoiceFlowId, spendUsd: 4, calls: 1 })+ return { invoiceFlowId, quietFlowId, agentId }+ }++ it('overview adds up across totals, series, sources and every card', async () => {+ const { invoiceFlowId, quietFlowId, agentId } = await seedPlatform()+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'production' } })++ expect(overview.history).toMatchObject({ hasAnyUsage: true, comparable: true })+ expect(overview.totals).toMatchObject({ spendUsd: 18, previousSpendUsd: 4, calls: 7, tokens: 150, cacheShare: 0.4, managedCredits: 50 })+ expect(sum(overview.series.map((point) => point.spendUsd))).toBeCloseTo(18)+ expect(sum(overview.series.map((point) => point.previousSpendUsd))).toBeCloseTo(4)+ expect(overview.bySource).toEqual([+ { source: 'flow-steps', spendUsd: 10 },+ { source: 'agent-steps', spendUsd: 5 },+ { source: 'chat', spendUsd: 3 },+ ])++ expect(overview.top.project.map((item) => [item.id, item.name, item.spendUsd, item.flowCount, item.agentCount])).toEqual([+ [ctx.project.id, ctx.project.displayName, 15, 2, 1],+ ['no-project', null, 3, 0, 0],+ ])+ expect(sum(overview.top.project.map((item) => item.spendUsd))).toBe(overview.totals.spendUsd)++ expect(overview.top.flow.map((item) => [item.id, item.name, item.parent, item.spendUsd, item.sharedUsd])).toEqual([+ [invoiceFlowId, 'Invoice flow', ctx.project.displayName, 15, 5],+ [quietFlowId, 'Quiet flow', ctx.project.displayName, 0, 0],+ ])+ expect(overview.top.agent.map((item) => [item.id, item.spendUsd, item.sharedUsd])).toEqual([[agentId, 5, 5]])+ expect(overview.top.user.map((item) => [item.id, item.parent, item.spendUsd])).toEqual([[ctx.user.id, ctx.userIdentity.email, 3]])+ expect(overview.top.key.map((item) => [item.id, item.provider, item.spendUsd])).toEqual([+ ['unknown-openai', AIProviderName.OPENAI, 10],+ ['activepieces-credits', AIProviderName.ACTIVEPIECES, 5],+ ['unknown-anthropic', AIProviderName.ANTHROPIC, 3],+ ])+ expect(sum(overview.top.key.map((item) => item.spendUsd))).toBe(overview.totals.spendUsd)+ expect(overview.top.model.map((item) => item.name)).toEqual(['gpt-5', 'openai/gpt-5', 'claude-sonnet-5', 'unpriced-model'])+ })++ it('follows the scope', async () => {+ await seedPlatform()+ const all = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'all' } })+ const testing = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'testing' } })+ expect(all.totals.spendUsd).toBe(25)+ expect(testing.totals.spendUsd).toBe(7)+ })++ it('lists every item of a dimension with the list and platform totals', async () => {+ const { invoiceFlowId } = await seedPlatform()+ const list = await getJson<AiUsageItemList>({ url: '/v1/ai-usage/items', query: { dimension: 'flow', period: '30d', scope: 'production' } })+ expect(list).toMatchObject({ count: 2, itemsSpendUsd: 15, totalSpendUsd: 18, truncated: false })+ expect(list.items[0]).toMatchObject({ id: invoiceFlowId, previousSpendUsd: 4, comparable: true })+ })++ it('details a flow with its runs and related models and keys', async () => {+ const { invoiceFlowId } = await seedPlatform()+ const detail = await getJson<AiUsageDetail>({ url: `/v1/ai-usage/items/flow/${invoiceFlowId}`, query: { period: '30d', scope: 'production' } })+ expect(detail.item).toMatchObject({ id: invoiceFlowId, name: 'Invoice flow', spendUsd: 15, sharedUsd: 5 })+ expect(detail.runs).toBe(2)+ expect(sum((detail.related.model ?? []).map((item) => item.spendUsd))).toBe(15)+ expect(sum(detail.series.map((point) => point.spendUsd))).toBeCloseTo(15)+ expect(detail.bySource).toBeUndefined()+ })++ it('splits a key into reported and estimated cost', async () => {+ await seedPlatform()+ const detail = await getJson<AiUsageDetail>({ url: '/v1/ai-usage/items/key/unknown-openai', query: { period: '30d', scope: 'production' } })+ expect(detail.item).toMatchObject({ provider: AIProviderName.OPENAI, spendUsd: 10 })+ expect(detail).toMatchObject({ reportedUsd: 6, estimatedUsd: 4 })+ expect(sum((detail.related.project ?? []).map((item) => item.spendUsd))).toBe(10)+ })++ it('details platform chat under the no-project id', async () => {+ await seedPlatform()+ const detail = await getJson<AiUsageDetail>({ url: '/v1/ai-usage/items/project/no-project', query: { period: '30d', scope: 'production' } })+ expect(detail.item).toMatchObject({ id: 'no-project', name: null, spendUsd: 3 })+ expect(detail.bySource).toEqual([{ source: 'chat', spendUsd: 3 }])+ })++ it('returns zeros for an id it cannot decode, and for another platform\'s item without its name', async () => {+ const other = await createTestContext(app)+ const secretFlowId = await saveFlow({ projectId: other.project.id, displayName: 'Secret flow' })+ await seed({ platformId: other.platform.id, projectId: other.project.id, day: daysOf('30d').current, flowId: secretFlowId, spendUsd: 99 })++ const foreign = await getJson<AiUsageDetail>({ url: `/v1/ai-usage/items/flow/${secretFlowId}`, query: { period: '30d', scope: 'all' } })+ expect(foreign.item).toMatchObject({ id: secretFlowId, name: null, spendUsd: 0 })+ const garbage = await getJson<AiUsageDetail>({ url: '/v1/ai-usage/items/model/%2A%2A', query: { period: '30d', scope: 'all' } })+ expect(garbage.item).toMatchObject({ spendUsd: 0, calls: 0 })+ const nulModel = await getJson<AiUsageDetail>({ url: '/v1/ai-usage/items/model/b3BlbmFpfAA', query: { period: '30d', scope: 'all' } })+ expect(nulModel.item).toMatchObject({ spendUsd: 0, calls: 0 })+ const nulFlow = await getJson<AiUsageDetail>({ url: '/v1/ai-usage/items/flow/abc%00', query: { period: '30d', scope: 'all' } })+ expect(nulFlow.item).toMatchObject({ spendUsd: 0, calls: 0 })+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'all' } })+ expect(overview.totals.spendUsd).toBe(0)+ })++ it('keeps a deleted project\'s spend but not its name', async () => {+ const gone = createMockProject({ platformId: ctx.platform.id, ownerId: ctx.user.id, displayName: 'Closed team', deleted: new Date().toISOString() })+ await db.save('project', gone)+ await seed({ day: daysOf('30d').current, projectId: gone.id, spendUsd: 2 })+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'production' } })+ expect(overview.top.project.map((item) => [item.id, item.name, item.spendUsd])).toEqual([[gone.id, null, 2]])+ })++ it('scales the partial previous day the same way for totals and items', async () => {+ const { current, scaleDay } = daysOf('7d')+ expect(scaleDay).not.toBeNull()+ await seed({ day: scaleDay ?? '', spendUsd: 100, calls: 10 })+ await seed({ day: current, spendUsd: 1 })+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '7d', scope: 'production' } })+ const fraction = overview.totals.previousSpendUsd / 100+ expect(fraction).toBeGreaterThanOrEqual(0)+ expect(fraction).toBeLessThanOrEqual(1)+ expect(overview.top.project[0].previousSpendUsd).toBeCloseTo(overview.totals.previousSpendUsd, 6)+ expect(sum(overview.series.map((point) => point.previousSpendUsd))).toBeCloseTo(overview.totals.previousSpendUsd, 6)+ })++ it('keeps every previous-month day in the series of last month', async () => {+ const { previous, previousLast } = daysOf('last-month')+ await seed({ day: previous, spendUsd: 1 })+ await seed({ day: previousLast, spendUsd: 2 })+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: 'last-month', scope: 'production' } })+ expect(overview.totals.previousSpendUsd).toBe(3)+ expect(sum(overview.series.map((point) => point.previousSpendUsd))).toBe(3)+ })++ it.each(['1h', '6h', '12h', '24h'] as const)('reads five-minute rows into the %s chart slots and detail', async (period) => {+ const flowId = await saveFlow({ projectId: ctx.project.id, displayName: 'Hourly flow' })+ await awayFromFiveMinuteEdge()+ const resolved = aiUsagePeriod.resolve({ period, now: new Date() })+ await seed({ bucket: new Date(resolved.current.start.getTime() + resolved.stepMs).toISOString(), flowId, spendUsd: 6, calls: 2 })+ await seed({ bucket: resolved.current.start.toISOString(), flowId, spendUsd: 1 })+ await seed({ bucket: resolved.previous.start.toISOString(), flowId, spendUsd: 4 })+ await seed({ bucket: new Date(resolved.previous.start.getTime() - resolved.stepMs).toISOString(), flowId, spendUsd: 50 })++ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period, scope: 'production' } })+ expect(overview.totals).toMatchObject({ spendUsd: 7, previousSpendUsd: 4, calls: 3 })+ expect(overview.series).toHaveLength(resolved.slotCount)+ expect(overview.series.slice(0, 2).map((point) => point.spendUsd)).toEqual([1, 6])+ expect(overview.series[0].previousSpendUsd).toBe(4)+ expect(sum(overview.series.map((point) => point.spendUsd))).toBe(7)++ const detail = await getJson<AiUsageDetail>({ url: `/v1/ai-usage/items/flow/${flowId}`, query: { period, scope: 'production' } })+ expect(detail.item).toMatchObject({ id: flowId, name: 'Hourly flow', spendUsd: 7, previousSpendUsd: 4 })+ expect(sum(detail.series.map((point) => point.spendUsd))).toBe(7)+ })++ it('shows the page at zero for a platform that never used AI', async () => {+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: 'this-month', scope: 'production' } })+ expect(overview.history.hasAnyUsage).toBe(false)+ expect(overview.totals.spendUsd).toBe(0)+ expect(overview.series.every((point) => point.spendUsd === 0)).toBe(true)+ expect(overview.top.project).toEqual([])+ })++ it('serves a repeated view from the cache', async () => {+ const { current } = daysOf('30d')+ await seed({ day: current, spendUsd: 1 })+ const first = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'production' } })+ await seed({ day: current, spendUsd: 5 })+ const second = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '30d', scope: 'production' } })+ expect(second.totals.spendUsd).toBe(first.totals.spendUsd)+ })++ it('starts tracking at the platform\'s copy cursor, so an earlier period is not compared', async () => {+ await databaseConnection().query('UPDATE "ai_usage_copy_cursor" SET "startedAt" = now() WHERE "platformId" = $1', [ctx.platform.id])+ await seed({ day: daysOf('7d').current, spendUsd: 1 })+ const overview = await getJson<AiUsageOverview>({ url: '/v1/ai-usage/overview', query: { period: '7d', scope: 'production' } })+ expect(overview.history.comparable).toBe(false)+ expect(overview.history.currentCoverage).toBeLessThan(1)+ })++ it('is for platform admins only', async () => {+ const member = await createMemberContext(app, ctx, { projectRole: DefaultProjectRole.ADMIN })+ const response = await member.get('/v1/ai-usage/overview', { period: '30d', scope: 'production' })+ expect(response.statusCode).toBe(StatusCodes.FORBIDDEN)+ })++ it('encodes model ids that the detail endpoint reads back', async () => {+ await seedPlatform()+ const id = aiUsageDimension.idOf({ dimension: 'model', keys: { projectId: null, flowId: null, agentId: null, userId: null, provider: AIProviderName.ANTHROPIC, providerConfigId: null, modelId: 'claude-sonnet-5' } }) ?? ''+ const detail = await getJson<AiUsageDetail>({ url: `/v1/ai-usage/items/model/${id}`, query: { period: '30d', scope: 'production' } })+ expect(detail.item).toMatchObject({ name: 'claude-sonnet-5', provider: AIProviderName.ANTHROPIC, spendUsd: 3 })+ })+})++const FIVE_MINUTES_MS = 5 * 60 * 1000+const EDGE_MARGIN_MS = 5000++type SeedRow = ({ day: string } | { bucket: string }) & {+ platformId?: string+ projectId?: string | null+ flowId?: string+ agentId?: string+ userId?: string+ provider?: AIProviderName+ providerConfigId?: string+ modelId?: string+ origin?: AiUsageOrigin+ environment?: AiUsageEnvironment+ calls?: number+ runs?: number+ inputTokens?: number+ outputTokens?: number+ cacheReadTokens?: number+ spendUsd?: number+ reportedCostUsd?: number+ estimatedCostUsd?: number+ managedCredits?: number+}diff --git a/packages/server/api/test/unit/app/ai/ai-usage-period.test.ts b/packages/server/api/test/unit/app/ai/ai-usage-period.test.tsnew file mode 100644index 00000000000..3cea1baa7e9--- /dev/null+++ b/packages/server/api/test/unit/app/ai/ai-usage-period.test.ts@@ -0,0 +1,99 @@+import { AIProviderName, AiUsageOrigin } from '@activepieces/core-utils'+import { describe, expect, it } from 'vitest'+import { aiUsageDimension } from '../../../../src/app/ai/usage/ai-usage-dimension'+import { aiUsagePeriod } from '../../../../src/app/ai/usage/ai-usage-period'++function iso(value: string): Date {+ return new Date(value)+}++describe('aiUsagePeriod.resolve', () => {+ it('ends sub-day periods at the current 5-minute bucket and compares with the span just before', () => {+ const resolved = aiUsagePeriod.resolve({ period: '24h', now: iso('2026-10-26T10:07:30Z') })+ expect(resolved.table).toBe('ai_usage_5m')+ expect(resolved.current).toEqual({ start: iso('2026-10-25T10:10:00Z'), end: iso('2026-10-26T10:10:00Z') })+ expect(resolved.previous).toEqual({ start: iso('2026-10-24T10:10:00Z'), end: iso('2026-10-25T10:10:00Z') })+ expect(resolved.slotCount).toBe(24)+ expect(resolved.scale).toBeNull()+ expect(aiUsagePeriod.labelOf({ resolved, slot: 1 })).toBe('2026-10-25T11:10:00.000Z')+ })++ it('counts last N days including today and scales the matching previous day by the elapsed fraction', () => {+ const resolved = aiUsagePeriod.resolve({ period: '7d', now: iso('2026-10-26T06:00:00Z') })+ expect(resolved.table).toBe('ai_usage_daily')+ expect(resolved.current).toEqual({ start: iso('2026-10-20T00:00:00Z'), end: iso('2026-10-27T00:00:00Z') })+ expect(resolved.previous).toEqual({ start: iso('2026-10-13T00:00:00Z'), end: iso('2026-10-20T00:00:00Z') })+ expect(resolved.scale).toEqual({ day: iso('2026-10-19T00:00:00Z'), fraction: 0.25 })+ expect(aiUsagePeriod.labelOf({ resolved, slot: 6 })).toBe('2026-10-26')+ })++ it('compares this month with the same days of last month', () => {+ const resolved = aiUsagePeriod.resolve({ period: 'this-month', now: iso('2026-10-20T12:00:00Z') })+ expect(resolved.current).toEqual({ start: iso('2026-10-01T00:00:00Z'), end: iso('2026-10-21T00:00:00Z') })+ expect(resolved.previous).toEqual({ start: iso('2026-09-01T00:00:00Z'), end: iso('2026-09-21T00:00:00Z') })+ expect(resolved.slotCount).toBe(20)+ expect(resolved.previousSlotCount).toBe(20)+ expect(resolved.scale).toEqual({ day: iso('2026-09-20T00:00:00Z'), fraction: 0.5 })+ })++ it('clamps this month at the end of a shorter previous month and stops scaling a finished day', () => {+ const resolved = aiUsagePeriod.resolve({ period: 'this-month', now: iso('2026-03-30T12:00:00Z') })+ expect(resolved.previous).toEqual({ start: iso('2026-02-01T00:00:00Z'), end: iso('2026-03-01T00:00:00Z') })+ expect(resolved.slotCount).toBe(30)+ expect(resolved.previousSlotCount).toBe(28)+ expect(resolved.scale).toBeNull()+ })++ it('handles a leap February', () => {+ const resolved = aiUsagePeriod.resolve({ period: 'this-month', now: iso('2028-03-31T00:00:00Z') })+ expect(resolved.previousSlotCount).toBe(29)+ })++ it('compares last month with the month before and folds a longer previous month into the last slot', () => {+ const resolved = aiUsagePeriod.resolve({ period: 'last-month', now: iso('2026-10-05T00:00:00Z') })+ expect(resolved.current).toEqual({ start: iso('2026-09-01T00:00:00Z'), end: iso('2026-10-01T00:00:00Z') })+ expect(resolved.previous).toEqual({ start: iso('2026-08-01T00:00:00Z'), end: iso('2026-09-01T00:00:00Z') })+ expect(resolved.slotCount).toBe(30)+ expect(resolved.previousSlotCount).toBe(30)+ expect(resolved.scale).toBeNull()+ })++ it('rolls last month over the year boundary', () => {+ const resolved = aiUsagePeriod.resolve({ period: 'last-month', now: iso('2027-01-03T00:00:00Z') })+ expect(resolved.current).toEqual({ start: iso('2026-12-01T00:00:00Z'), end: iso('2027-01-01T00:00:00Z') })+ expect(resolved.previous.start).toEqual(iso('2026-11-01T00:00:00Z'))+ })+})++describe('aiUsageDimension', () => {+ it('round-trips model ids that contain slashes and colons', () => {+ const id = aiUsageDimension.idOf({ dimension: 'model', keys: { projectId: null, flowId: null, agentId: null, userId: null, provider: AIProviderName.OPENROUTER, providerConfigId: null, modelId: 'anthropic/claude-sonnet-5:beta' } })+ expect(id).not.toBeNull()+ expect(aiUsageDimension.filterOf({ dimension: 'model', id: id ?? '' })).toEqual([+ { column: 'provider', value: AIProviderName.OPENROUTER },+ { column: 'modelId', value: 'anthropic/claude-sonnet-5:beta' },+ ])+ })++ it('maps keys without a config to credits or an unknown key of that provider', () => {+ const keys = { projectId: null, flowId: null, agentId: null, userId: null, providerConfigId: null, modelId: null }+ expect(aiUsageDimension.idOf({ dimension: 'key', keys: { ...keys, provider: AIProviderName.ACTIVEPIECES } })).toBe('activepieces-credits')+ expect(aiUsageDimension.filterOf({ dimension: 'key', id: 'unknown-openai' })).toEqual([+ { column: 'provider', value: AIProviderName.OPENAI },+ { column: 'providerConfigId', value: null },+ ])+ })++ it('rejects ids that do not decode to a known provider', () => {+ expect(aiUsageDimension.filterOf({ dimension: 'model', id: 'not-base64-at-all' })).toBeNull()+ expect(aiUsageDimension.filterOf({ dimension: 'key', id: 'unknown-nope' })).toBeNull()+ })++ it('maps the null project to the no-project id', () => {+ expect(aiUsageDimension.filterOf({ dimension: 'project', id: 'no-project' })).toEqual([{ column: 'projectId', value: null }])+ })++ it('gives every origin a source', () => {+ expect(Object.values(AiUsageOrigin).map((origin) => aiUsageDimension.sourceOf(origin))).not.toContain(undefined)+ })+})