Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

refactor(ai): name and type the AI billing path, end to end (#15540)

Abdul70790deaafaf
diff --git a/CLAUDE.md b/CLAUDE.md
index 4ef35617243..3bb2019f434 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -47,6 +47,7 @@ Open-source AI-first workflow automation platform. Self-hosted or cloud. 400+ pi
- **No type casting** — Do not use `as SomeType` to force types. If you encounter an unnecessary cast, remove it.
- **No deprecated APIs** — Before using any library method or export, check its JSDoc. If it carries a `@deprecated` tag, use the recommended replacement instead. Examples: prefer `z.enum` over `z.nativeEnum`.
- **Go-style error handling** — Use `tryCatch` / `tryCatchSync` from `@activepieces/shared`
+- **Never write `void` in front of a promise** — When a promise is deliberately not awaited, end the chain with `.catch()` instead. `void somePromise()` is not our convention: it reads as a marker for the author rather than for the next reader, and it hides the question that matters — who handles a rejection. `@typescript-eslint/no-floating-promises` is an error in `server/api` and every package extending it (`server/worker`), so a bare statement will not lint — `.catch(() => undefined)` is the minimum, `tryCatch` where a failure changes what happens next, `rejectedPromiseHandler` in `server/api` where it only needs logging. Existing `void` usages predate this rule; do not sweep them, but do not add new ones. (This is about promises only — `void reply.header(...)` on Fastify's chainable reply is a different thing and is fine.)
- **Zod error messages must be i18n keys** — Every `.min()`, `.refine()`, `.superRefine()`, etc. that surfaces a user-facing message must pass a string that exists as a key in `packages/web/public/locales/en/translation.json`. For common messages (e.g. required fields) use the `formErrors` constant from `@activepieces/shared`. Add a new translation key if none fits; never use raw English sentences that are not in the translation file.
- **`@activepieces/shared` version bump** — Any change to `packages/core/shared` must be accompanied by a version bump in `packages/core/shared/package.json`: bump the **patch** version for non-breaking additions or fixes, bump the **minor** version for new exports or behaviour changes after you check if it has already been bumped in the current branch or not
- **Helper functions** — Define non-exported helpers outside of const declarations
diff --git a/bun.lock b/bun.lock
index be1c3748d09..2a003be168e 100644
--- a/bun.lock
+++ b/bun.lock
@@ -164,7 +164,7 @@
},
"packages/core/shared": {
"name": "@activepieces/shared",
- "version": "0.172.0",
+ "version": "0.173.0",
"dependencies": {
"@activepieces/core-execution": "workspace:*",
"@activepieces/core-formula": "workspace:*",
diff --git a/docs/install/reference/breaking-changes.mdx b/docs/install/reference/breaking-changes.mdx
index c828764cd74..57db79da402 100644
--- a/docs/install/reference/breaking-changes.mdx
+++ b/docs/install/reference/breaking-changes.mdx
@@ -236,6 +236,30 @@ Nothing on a normal upgrade. Every flow is moved to the AI piece version that ru
If you serve pieces from your own registry, make sure it carries the AI piece at `0.11.0` or later. A flow left on an earlier version would still try to fetch its provider credentials over this endpoint and fail.
+#### AI steps are billed on what each call costs, not on a fixed price per model
+
+The managed Activepieces AI provider used to charge a fixed number of credits per model per step, taken from a table calibrated at roughly a thousand tokens per call. A short call was charged about right; a long one was charged the same as a short one while costing us many times more.
+
+Every managed AI call is now billed on the dollar cost the provider reports for that call, converted at `AP_AI_CREDIT_USD_VALUE` (default `0.0005`, so one credit is a twentieth of a cent of model spend). A call on your own API key is billed a flat one credit per model call. Tool calls stay at one credit each.
+
+This changes what the same flow costs in both directions. A short prompt on an expensive model gets much cheaper; a long document through Extract Structured Data gets dearer. The credit cost shown in the model picker and in the credits breakdown is gone, because there is no longer a fixed number to show.
+
+A step or a chat turn on your own API key costs the same as it did: one credit, plus one per tool call. An agent still counts as one credit however many times it goes back to the model.
+
+#### What you need to do
+
+Nothing to configure. Self-hosters who want a different conversion rate can set `AP_AI_CREDIT_USD_VALUE` to the dollar value of one credit.
+
+#### AI steps now consume credits on your own API key too
+
+`POST /v1/ai/execute` refuses to start an AI step when the platform is out of credits, whatever provider the step uses. Previously only the managed Activepieces provider was gated, so a platform that had exhausted its credits kept running AI steps as long as they used its own key.
+
+Credits are only enforced where a billing provider is configured, so a Community self-host is unaffected.
+
+#### What you need to do
+
+Nothing, unless you run AI steps on your own provider key on a platform that sits at zero credits. Those steps now fail with a credits error instead of running. Top up, or raise the plan's credit allowance.
+
#### Files passed to an AI step are capped at `AP_MAX_FILE_SIZE_MB`
Extract Structured Data and Generate Image used to read file inputs straight from the flow's memory, so no Activepieces-side size limit applied. Those files are now stored and fetched by the worker that runs the model, which puts them under the same per-file limit as every other file the engine writes: `AP_MAX_FILE_SIZE_MB`, 25 MB by default.
diff --git a/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.test.ts b/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.test.ts
index a6bb82cc832..49a6aed3d0c 100644
--- a/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.test.ts
+++ b/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.test.ts
@@ -1,3 +1,4 @@
+import { aiProviderCredentials, AIProviderName } from '@activepieces/core-utils'
import { describe, expect, it } from 'vitest'
import { createCloudflareGatewayModel } from './create-cloudflare-gateway-model'
@@ -7,25 +8,24 @@ function identify(model: unknown): ModelIdentity {
return model as ModelIdentity
}
-const auth = { apiKey: 'cf-key' }
-const config = { accountId: 'acct', gatewayId: 'gw' }
+const credentials = aiProviderCredentials({ provider: AIProviderName.CLOUDFLARE_GATEWAY, auth: { apiKey: 'cf-key' }, config: { accountId: 'acct', gatewayId: 'gw' } })
describe('createCloudflareGatewayModel', () => {
describe('compat routing', () => {
it('strips the submodel prefix from the model id', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'openai/gpt-4o' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'openai/gpt-4o' }))
expect(model.modelId).toBe('gpt-4o')
})
it('sends the gateway authorization header and no metadata header', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'openai/gpt-4o' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'openai/gpt-4o' }))
const headers = model.config?.headers?.() ?? {}
expect(headers['cf-aig-authorization']).toBe('Bearer cf-key')
expect(headers['cf-aig-metadata']).toBeUndefined()
})
it('ignores the submodel prefix and stays on the compat endpoint', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'anthropic/claude-sonnet-4' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'anthropic/claude-sonnet-4' }))
expect(model.modelId).toBe('claude-sonnet-4')
expect(model.provider).toContain('cloudflare')
})
@@ -33,36 +33,35 @@ describe('createCloudflareGatewayModel', () => {
describe('submodel routing', () => {
it('routes an openai submodel to the gateway openai endpoint with the stripped id', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'openai/gpt-4o', routing: 'submodel' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'openai/gpt-4o', routing: 'submodel' }))
expect(model.modelId).toBe('gpt-4o')
expect(model.provider).toContain('openai')
})
it('falls back to the compat endpoint with the full model id for an unmapped prefix', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'mystery/some-model', routing: 'submodel' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'mystery/some-model', routing: 'submodel' }))
expect(model.modelId).toBe('mystery/some-model')
expect(model.provider).toContain('cloudflare')
})
it('falls back to compat when a vertex submodel has no vertex project configured', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'google-vertex-ai/publishers/google/models/gemini-2.0', routing: 'submodel' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'google-vertex-ai/publishers/google/models/gemini-2.0', routing: 'submodel' }))
expect(model.provider).toContain('cloudflare')
})
it('routes an anthropic submodel through the gateway wrapper', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'anthropic/claude-sonnet-4', routing: 'submodel' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'anthropic/claude-sonnet-4', routing: 'submodel' }))
expect(model.modelId).toBe('claude-sonnet-4')
})
it('routes a google-ai-studio submodel through the gateway wrapper', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'google-ai-studio/gemini-2.0-flash', routing: 'submodel' }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'google-ai-studio/gemini-2.0-flash', routing: 'submodel' }))
expect(model.modelId).toBe('gemini-2.0-flash')
})
it('serialises step metadata into the gateway metadata header', () => {
const model = identify(createCloudflareGatewayModel({
- auth,
- config,
+ credentials,
modelId: 'mystery/some-model',
routing: 'submodel',
metadata: { projectId: 'p1', flowId: 'f1', runId: 'r1' },
@@ -74,13 +73,13 @@ describe('createCloudflareGatewayModel', () => {
describe('image models', () => {
it('builds an image model for an openai submodel', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'openai/dall-e-3', routing: 'submodel', isImage: true }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'openai/dall-e-3', routing: 'submodel', isImage: true }))
expect(model.modelId).toBe('dall-e-3')
expect(model.provider).toContain('openai')
})
it('builds a compat image model for an unmapped prefix', () => {
- const model = identify(createCloudflareGatewayModel({ auth, config, modelId: 'mystery/some-image-model', routing: 'submodel', isImage: true }))
+ const model = identify(createCloudflareGatewayModel({ credentials, modelId: 'mystery/some-image-model', routing: 'submodel', isImage: true }))
expect(model.modelId).toBe('mystery/some-image-model')
expect(model.provider).toContain('cloudflare')
})
diff --git a/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.ts b/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.ts
index ab30fad8f12..1e00332ef95 100644
--- a/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.ts
+++ b/packages/core/ai-providers/src/lib/create-cloudflare-gateway-model.ts
@@ -1,5 +1,5 @@
-import { isNil, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils'
-import { BaseAIProviderAuthConfig, CloudflareGatewayProviderConfig, splitCloudflareGatewayModelId } from '@activepieces/core-piece-types'
+import { AiProviderCredentials, AIProviderName, isNil, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils'
+import { splitCloudflareGatewayModelId } from '@activepieces/core-piece-types'
import { createAnthropic } from '@ai-sdk/anthropic'
import { createGoogleGenerativeAI } from '@ai-sdk/google'
import { createOpenAI } from '@ai-sdk/openai'
@@ -9,9 +9,11 @@ import { createAiGateway } from 'ai-gateway-provider'
export function createCloudflareGatewayModel(params: CreateCloudflareGatewayModelParams & { isImage: true }): ImageModel
export function createCloudflareGatewayModel(params: CreateCloudflareGatewayModelParams & { isImage?: false }): LanguageModel
-export function createCloudflareGatewayModel({ auth, config, modelId, isImage = false, openaiResponsesModel = false, routing = 'compat', metadata, onOutcome }: CreateCloudflareGatewayModelParams): ImageModel | LanguageModel {
- const { apiKey } = auth as BaseAIProviderAuthConfig
- const { accountId, gatewayId, vertexProject, vertexRegion } = config as CloudflareGatewayProviderConfig
+export function createCloudflareGatewayModel({ credentials, modelId, isImage = false, openaiResponsesModel = false, routing = 'compat', metadata, onOutcome }: CreateCloudflareGatewayModelParams): ImageModel | LanguageModel {
+ const { apiKey } = credentials.auth
+ const { vertexProject, vertexRegion } = credentials.config
+ const accountId = credentials.config.accountId ?? ''
+ const gatewayId = credentials.config.gatewayId ?? ''
const { provider: providerPrefix, model: actualModelId, publisher } = splitCloudflareGatewayModelId(modelId)
const headers = {
'cf-aig-authorization': `Bearer ${apiKey}`,
@@ -90,8 +92,7 @@ export type CloudflareGatewayMetadata = {
}
export type CreateCloudflareGatewayModelParams = {
- auth: Record<string, unknown>
- config: Record<string, unknown>
+ credentials: Extract<AiProviderCredentials, { provider: AIProviderName.CLOUDFLARE_GATEWAY }>
modelId: string
isImage?: boolean
openaiResponsesModel?: boolean
diff --git a/packages/core/ai-providers/src/lib/create-language-model.test.ts b/packages/core/ai-providers/src/lib/create-language-model.test.ts
index efc9c99da17..5da38ca71f2 100644
--- a/packages/core/ai-providers/src/lib/create-language-model.test.ts
+++ b/packages/core/ai-providers/src/lib/create-language-model.test.ts
@@ -1,4 +1,4 @@
-import { AIProviderName } from '@activepieces/core-utils'
+import { aiProviderCredentials, AIProviderName } from '@activepieces/core-utils'
import { AIProviderConfig, AIProviderModelType, VertexProviderConfig } from '@activepieces/core-piece-types'
import { describe, expect, it } from 'vitest'
import { buildOpenAICompatibleHeaders, createLanguageModel } from './create-language-model'
@@ -58,9 +58,7 @@ const configFor: Partial<Record<AIProviderName, unknown>> = {
}
const buildFor = (provider: AIProviderName, options?: Record<string, unknown>) => createLanguageModel({
- provider,
- auth: authFor[provider] ?? { apiKey: 'test-key' },
- config: configFor[provider] ?? {},
+ credentials: aiProviderCredentials({ provider, auth: authFor[provider] ?? { apiKey: 'test-key' }, config: configFor[provider] ?? {} }),
modelId: 'some-model-id',
options,
})
@@ -119,9 +117,7 @@ describe('createLanguageModel', () => {
it('sends Model Garden Claude ids to the Vertex Anthropic client, not the Gemini one', () => {
const anthropicOnVertex = createLanguageModel({
- provider: AIProviderName.VERTEX,
- auth: authFor[AIProviderName.VERTEX],
- config: configFor[AIProviderName.VERTEX],
+ credentials: aiProviderCredentials({ provider: AIProviderName.VERTEX, auth: authFor[AIProviderName.VERTEX], config: configFor[AIProviderName.VERTEX] }),
modelId: 'claude-sonnet-4-6',
})
@@ -131,9 +127,7 @@ describe('createLanguageModel', () => {
it('sends Model Garden MaaS ids to the Vertex MaaS client', () => {
const build = (modelId: string) => identify(createLanguageModel({
- provider: AIProviderName.VERTEX,
- auth: authFor[AIProviderName.VERTEX],
- config: configFor[AIProviderName.VERTEX],
+ credentials: aiProviderCredentials({ provider: AIProviderName.VERTEX, auth: authFor[AIProviderName.VERTEX], config: configFor[AIProviderName.VERTEX] }),
modelId,
})).provider
@@ -146,9 +140,7 @@ describe('createLanguageModel', () => {
it('keeps the custom provider on chat completions unless apiStyle asks for responses', () => {
const responsesConfig = { ...(configFor[AIProviderName.CUSTOM] as Record<string, unknown>), apiStyle: 'responses' }
const model = createLanguageModel({
- provider: AIProviderName.CUSTOM,
- auth: { apiKey: 'test-key' },
- config: responsesConfig,
+ credentials: aiProviderCredentials({ provider: AIProviderName.CUSTOM, auth: { apiKey: 'test-key' }, config: responsesConfig }),
modelId: 'openai.gpt-oss-120b',
})
expect(identify(buildFor(AIProviderName.CUSTOM)).provider).toBe('openai-compatible.chat')
@@ -158,9 +150,7 @@ describe('createLanguageModel', () => {
it('drops the SDK default Authorization when the custom provider authenticates with another header', async () => {
const model = createLanguageModel({
- provider: AIProviderName.CUSTOM,
- auth: { apiKey: 'secret-key' },
- config: { apiKeyHeader: 'x-api-key', baseUrl: 'https://example.test/v1', models: [], apiStyle: 'responses' },
+ credentials: aiProviderCredentials({ provider: AIProviderName.CUSTOM, auth: { apiKey: 'secret-key' }, config: { apiKeyHeader: 'x-api-key', baseUrl: 'https://example.test/v1', models: [], apiStyle: 'responses' } }),
modelId: 'some-model-id',
})
const { headers, fetch: patchedFetch } = identifyCustom(model).config
@@ -177,9 +167,7 @@ describe('createLanguageModel', () => {
it('keeps the Authorization header when that is the custom provider\'s own api key header', () => {
const model = createLanguageModel({
- provider: AIProviderName.CUSTOM,
- auth: { apiKey: 'Bearer bedrock-key' },
- config: { apiKeyHeader: 'Authorization', baseUrl: 'https://bedrock-mantle.us-east-1.api.aws/v1', models: [], apiStyle: 'responses' },
+ credentials: aiProviderCredentials({ provider: AIProviderName.CUSTOM, auth: { apiKey: 'Bearer bedrock-key' }, config: { apiKeyHeader: 'Authorization', baseUrl: 'https://bedrock-mantle.us-east-1.api.aws/v1', models: [], apiStyle: 'responses' } }),
modelId: 'openai.gpt-oss-120b',
})
const { headers, fetch: patchedFetch } = identifyCustom(model).config
@@ -190,15 +178,17 @@ describe('createLanguageModel', () => {
it('keeps an Authorization the admin supplied through custom headers', async () => {
const model = createLanguageModel({
- provider: AIProviderName.CUSTOM,
- auth: { apiKey: 'secret-key' },
- config: {
- apiKeyHeader: 'x-api-key',
- baseUrl: 'https://gateway.test/v1',
- models: [],
- apiStyle: 'responses',
- defaultHeaders: { Authorization: 'Bearer gateway-token' },
- },
+ credentials: aiProviderCredentials({
+ provider: AIProviderName.CUSTOM,
+ auth: { apiKey: 'secret-key' },
+ config: {
+ apiKeyHeader: 'x-api-key',
+ baseUrl: 'https://gateway.test/v1',
+ models: [],
+ apiStyle: 'responses',
+ defaultHeaders: { Authorization: 'Bearer gateway-token' },
+ },
+ }),
modelId: 'some-model-id',
})
const { headers, fetch: patchedFetch } = identifyCustom(model).config
@@ -215,9 +205,7 @@ describe('createLanguageModel', () => {
it('forwards OpenRouter web-search plugin settings onto the model', () => {
const openRouterSettings = { plugins: [{ id: 'web', max_results: 5 }] }
const model = createLanguageModel({
- provider: AIProviderName.OPENROUTER,
- auth: { apiKey: 'test-key' },
- config: {},
+ credentials: aiProviderCredentials({ provider: AIProviderName.OPENROUTER, auth: { apiKey: 'test-key' }, config: {} }),
modelId: 'anthropic/claude',
options: { openRouterSettings },
})
@@ -226,9 +214,7 @@ describe('createLanguageModel', () => {
it('refuses to build Cloudflare Gateway (caller-specific)', () => {
expect(() => createLanguageModel({
- provider: AIProviderName.CLOUDFLARE_GATEWAY,
- auth: { apiKey: 'test-key' },
- config: { accountId: 'a', gatewayId: 'g', models: [] },
+ credentials: aiProviderCredentials({ provider: AIProviderName.CLOUDFLARE_GATEWAY, auth: { apiKey: 'test-key' }, config: { accountId: 'a', gatewayId: 'g', models: [] } }),
modelId: 'openai/gpt-4',
})).toThrow()
})
@@ -268,13 +254,13 @@ describe('resolved endpoint, credentials and headers', () => {
const urlOf = (cfg: ResolvedConfig): string => cfg.url({ path: '/chat/completions', modelId: 'm' })
it('sends OpenAI to the OpenAI endpoint with a bearer credential', () => {
- const cfg = configOf(createLanguageModel({ provider: AIProviderName.OPENAI, auth: { apiKey: 'SECRET' }, config: {}, modelId: 'm' }))
+ const cfg = configOf(createLanguageModel({ credentials: aiProviderCredentials({ provider: AIProviderName.OPENAI, auth: { apiKey: 'SECRET' }, config: {} }), modelId: 'm' }))
expect(urlOf(cfg)).toContain('https://api.openai.com')
expect(headersOf(cfg)['authorization']).toBe('Bearer SECRET')
})
it('encodes the Azure resource name and api version into the URL', () => {
- const cfg = configOf(createLanguageModel({ provider: AIProviderName.AZURE, auth: { apiKey: 'SECRET' }, config: { resourceName: 'myres', apiVersion: '2024-08-01' }, modelId: 'm' }))
+ const cfg = configOf(createLanguageModel({ credentials: aiProviderCredentials({ provider: AIProviderName.AZURE, auth: { apiKey: 'SECRET' }, config: { resourceName: 'myres', apiVersion: '2024-08-01' } }), modelId: 'm' }))
const url = urlOf(cfg)
expect(url).toContain('myres')
expect(url).toContain('api-version=2024-08-01')
@@ -283,9 +269,7 @@ describe('resolved endpoint, credentials and headers', () => {
it('attaches the caller metadata headers to managed OpenRouter traffic', () => {
const cfg = configOf(createLanguageModel({
- provider: AIProviderName.ACTIVEPIECES,
- auth: { apiKey: 'SECRET' },
- config: {},
+ credentials: aiProviderCredentials({ provider: AIProviderName.ACTIVEPIECES, auth: { apiKey: 'SECRET' }, config: {} }),
modelId: 'anthropic/claude',
options: { extraHeaders: { 'x-ap-platform-id': 'plat', 'x-ap-conversation-id': 'conv' } },
}))
@@ -296,7 +280,7 @@ describe('resolved endpoint, credentials and headers', () => {
})
it('leaves OpenRouter headers untouched when no metadata is passed', () => {
- const cfg = configOf(createLanguageModel({ provider: AIProviderName.OPENROUTER, auth: { apiKey: 'SECRET' }, config: {}, modelId: 'anthropic/claude' }))
+ const cfg = configOf(createLanguageModel({ credentials: aiProviderCredentials({ provider: AIProviderName.OPENROUTER, auth: { apiKey: 'SECRET' }, config: {} }), modelId: 'anthropic/claude' }))
const headers = headersOf(cfg)
expect(headers['x-ap-platform-id']).toBeUndefined()
expect(headers['Authorization']).toBe('Bearer SECRET')
@@ -304,9 +288,7 @@ describe('resolved endpoint, credentials and headers', () => {
it('points Custom at its base URL and applies header precedence end to end', () => {
const cfg = configOf(createLanguageModel({
- provider: AIProviderName.CUSTOM,
- auth: { apiKey: 'SECRET' },
- config: { apiKeyHeader: 'x-api-key', baseUrl: 'https://custom.test/v1', defaultHeaders: { 'x-shared': 'from-default' }, models: [] },
+ credentials: aiProviderCredentials({ provider: AIProviderName.CUSTOM, auth: { apiKey: 'SECRET' }, config: { apiKeyHeader: 'x-api-key', baseUrl: 'https://custom.test/v1', defaultHeaders: { 'x-shared': 'from-default' }, models: [] } }),
modelId: 'm',
options: { extraHeaders: { 'x-ap-project-id': 'proj', 'x-shared': 'from-extra' } },
}))
diff --git a/packages/core/ai-providers/src/lib/create-language-model.ts b/packages/core/ai-providers/src/lib/create-language-model.ts
index e7d4a32b95a..e0f81fbdf8e 100644
--- a/packages/core/ai-providers/src/lib/create-language-model.ts
+++ b/packages/core/ai-providers/src/lib/create-language-model.ts
@@ -1,5 +1,5 @@
-import { AIProviderName, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils'
-import { AzureProviderConfig, BaseAIProviderAuthConfig, BedrockProviderAuthConfig, BedrockProviderConfig, OPENAI_COMPATIBLE_VENDOR_BASE_URLS, OpenAICompatibleProviderConfig, VertexProviderAuthConfig, VertexProviderConfig } from '@activepieces/core-piece-types'
+import { AiProviderCredentials, AIProviderName, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils'
+import { OPENAI_COMPATIBLE_VENDOR_BASE_URLS } from '@activepieces/core-piece-types'
import { createAmazonBedrock } from '@ai-sdk/amazon-bedrock'
import { createVertex } from '@ai-sdk/google-vertex'
import { createVertexAnthropic } from '@ai-sdk/google-vertex/anthropic'
@@ -17,46 +17,46 @@ const VERTEX_ANTHROPIC_PREFIX = 'claude'
const MISTRAL_BASE_URL = 'https://api.mistral.ai/v1'
const AUTHORIZATION_HEADER = 'authorization'
-export function createLanguageModel({ provider, auth, config, modelId, options = {} }: CreateLanguageModelParams): LanguageModel {
+export function createLanguageModel({ credentials, modelId, options = {} }: CreateLanguageModelParams): LanguageModel {
const observed = spreadIfDefined('fetch', observedProviderFetch(options.onOutcome))
- switch (provider) {
+ switch (credentials.provider) {
case AIProviderName.OPENAI: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
const client = createOpenAI({ apiKey, ...observed })
return options.openaiResponsesModel ? client.responses(modelId) : client.chat(modelId)
}
case AIProviderName.ANTHROPIC: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
return createAnthropic({ apiKey, ...observed })(modelId)
}
case AIProviderName.GOOGLE: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
return createGoogleGenerativeAI({ apiKey, ...observed })(modelId)
}
case AIProviderName.AZURE: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
- const { resourceName, apiVersion } = config as AzureProviderConfig
+ const { apiKey } = credentials.auth
+ const { resourceName, apiVersion } = credentials.config
return createAzure({ resourceName, apiKey, apiVersion, ...observed }).chat(modelId)
}
case AIProviderName.BEDROCK: {
- const { accessKeyId, secretAccessKey } = auth as BedrockProviderAuthConfig
- const { region } = config as BedrockProviderConfig
+ const { accessKeyId, secretAccessKey } = credentials.auth
+ const { region } = credentials.config
return createAmazonBedrock({ region, accessKeyId, secretAccessKey, ...observed })(modelId)
}
case AIProviderName.VERTEX: {
- const { serviceAccountJson } = auth as VertexProviderAuthConfig
- const { project, region } = config as VertexProviderConfig
- const vertexSettings = { project, location: region, googleAuthOptions: { credentials: parseServiceAccount(serviceAccountJson) }, ...observed }
+ const { serviceAccountJson } = credentials.auth
+ const { project, region } = credentials.config
+ const vertexSettings = { project, location: region, googleAuthOptions: { credentials: parseServiceAccount(serviceAccountJson ?? '') }, ...observed }
return vertexClientFor({ modelId })(vertexSettings)(modelId)
}
case AIProviderName.CUSTOM: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
- const { apiKeyHeader, baseUrl, defaultHeaders, apiStyle } = config as OpenAICompatibleProviderConfig
- const headers = buildOpenAICompatibleHeaders({ apiKeyHeader, apiKey, defaultHeaders, extraHeaders: options.extraHeaders })
+ const { apiKey } = credentials.auth
+ const { apiKeyHeader, baseUrl, defaultHeaders, apiStyle } = credentials.config
+ const headers = buildOpenAICompatibleHeaders({ apiKeyHeader: apiKeyHeader ?? AUTHORIZATION_HEADER, apiKey: apiKey ?? '', defaultHeaders, extraHeaders: options.extraHeaders })
if (apiStyle === 'responses') {
return createOpenAI({
baseURL: baseUrl,
- apiKey,
+ apiKey: apiKey ?? '',
headers,
...observed,
...spreadIfDefined('fetch', stripDefaultAuthorization({
@@ -67,17 +67,17 @@ export function createLanguageModel({ provider, auth, config, modelId, options =
}
return createOpenAICompatible({
name: 'openai-compatible',
- baseURL: baseUrl,
+ baseURL: baseUrl ?? '',
headers,
...observed,
}).chatModel(modelId)
}
case AIProviderName.MISTRAL: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
if (options.mistralViaOpenRouter) {
return createOpenRouterChatModel({ apiKey, modelId, options })
}
- return createOpenAICompatible({ name: 'mistral', baseURL: MISTRAL_BASE_URL, apiKey, ...observed }).chatModel(modelId)
+ return createOpenAICompatible({ name: 'mistral', baseURL: MISTRAL_BASE_URL, apiKey: apiKey ?? '', ...observed }).chatModel(modelId)
}
case AIProviderName.XAI:
case AIProviderName.DEEPSEEK:
@@ -85,57 +85,57 @@ export function createLanguageModel({ provider, auth, config, modelId, options =
case AIProviderName.QWEN:
case AIProviderName.MINIMAX:
case AIProviderName.MOONSHOT: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
return createOpenAICompatible({
- name: provider,
- baseURL: OPENAI_COMPATIBLE_VENDOR_BASE_URLS[provider],
- apiKey,
+ name: credentials.provider,
+ baseURL: OPENAI_COMPATIBLE_VENDOR_BASE_URLS[credentials.provider],
+ apiKey: apiKey ?? '',
...observed,
}).chatModel(modelId)
}
case AIProviderName.OPENROUTER:
case AIProviderName.ACTIVEPIECES: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
return createOpenRouterChatModel({ apiKey, modelId, options })
}
case AIProviderName.CLOUDFLARE_GATEWAY:
throw new Error('Cloudflare Gateway routing is caller-specific and is not handled by the shared language-model factory')
default: {
- const exhaustiveCheck: never = provider
+ const exhaustiveCheck: never = credentials
throw new Error(`Unsupported provider: ${exhaustiveCheck}`)
}
}
}
-export function createImageModel({ provider, auth, config, modelId, options = {} }: CreateImageModelParams): ImageModel | undefined {
+export function createImageModel({ credentials, modelId, options = {} }: CreateImageModelParams): ImageModel | undefined {
const observed = spreadIfDefined('fetch', observedProviderFetch(options.onOutcome))
- switch (provider) {
+ switch (credentials.provider) {
case AIProviderName.OPENAI: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
+ const { apiKey } = credentials.auth
return createOpenAI({ apiKey, ...observed }).imageModel(modelId)
}
case AIProviderName.AZURE: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
- const { resourceName, apiVersion } = config as AzureProviderConfig
+ const { apiKey } = credentials.auth
+ const { resourceName, apiVersion } = credentials.config
return createAzure({ resourceName, apiKey, apiVersion, ...observed }).imageModel(modelId)
}
case AIProviderName.BEDROCK: {
- const { accessKeyId, secretAccessKey } = auth as BedrockProviderAuthConfig
- const { region } = config as BedrockProviderConfig
+ const { accessKeyId, secretAccessKey } = credentials.auth
+ const { region } = credentials.config
return createAmazonBedrock({ region, accessKeyId, secretAccessKey, ...observed }).imageModel(modelId)
}
case AIProviderName.VERTEX: {
- const { serviceAccountJson } = auth as VertexProviderAuthConfig
- const { project, region } = config as VertexProviderConfig
- return createVertex({ project, location: region, googleAuthOptions: { credentials: parseServiceAccount(serviceAccountJson) }, ...observed }).imageModel(modelId)
+ const { serviceAccountJson } = credentials.auth
+ const { project, region } = credentials.config
+ return createVertex({ project, location: region, googleAuthOptions: { credentials: parseServiceAccount(serviceAccountJson ?? '') }, ...observed }).imageModel(modelId)
}
case AIProviderName.CUSTOM: {
- const { apiKey } = auth as BaseAIProviderAuthConfig
- const { apiKeyHeader, baseUrl, defaultHeaders } = config as OpenAICompatibleProviderConfig
+ const { apiKey } = credentials.auth
+ const { apiKeyHeader, baseUrl, defaultHeaders } = credentials.config
return createOpenAICompatible({
name: 'openai-compatible',
- baseURL: baseUrl,
- headers: buildOpenAICompatibleHeaders({ apiKeyHeader, apiKey, defaultHeaders, extraHeaders: options.extraHeaders }),
+ baseURL: baseUrl ?? '',
+ headers: buildOpenAICompatibleHeaders({ apiKeyHeader: apiKeyHeader ?? AUTHORIZATION_HEADER, apiKey: apiKey ?? '', defaultHeaders, extraHeaders: options.extraHeaders }),
...observed,
}).imageModel(modelId)
}
@@ -181,12 +181,12 @@ function stripDefaultAuthorization({ headers, delegate }: {
}
function createOpenRouterChatModel({ apiKey, modelId, options }: {
- apiKey: string
+ apiKey: string | undefined
modelId: string
options: LanguageModelOptions
}): LanguageModel {
return createOpenRouter({
- apiKey,
+ apiKey: apiKey ?? '',
...spreadIfDefined('headers', options.extraHeaders),
...spreadIfDefined('fetch', observedProviderFetch(options.onOutcome)),
}).chat(modelId, options.openRouterSettings) as LanguageModel
@@ -214,9 +214,7 @@ export type LanguageModelOptions = {
}
export type CreateImageModelParams = {
- provider: AIProviderName
- auth: unknown
- config: unknown
+ credentials: AiProviderCredentials
modelId: string
options?: ImageModelOptions
}
@@ -227,9 +225,7 @@ export type ImageModelOptions = {
}
export type CreateLanguageModelParams = {
- provider: AIProviderName
- auth: unknown
- config: unknown
+ credentials: AiProviderCredentials
modelId: string
options?: LanguageModelOptions
}
diff --git a/packages/core/execution/src/lib/workers/job-data.ts b/packages/core/execution/src/lib/workers/job-data.ts
index f8fa01b0a25..47a804fc1cd 100644
--- a/packages/core/execution/src/lib/workers/job-data.ts
+++ b/packages/core/execution/src/lib/workers/job-data.ts
@@ -1,6 +1,6 @@
import { z } from 'zod'
-import { AIProviderName, isNil } from '@activepieces/core-utils'
+import { AIProviderName, AiStepAction, isNil } from '@activepieces/core-utils'
import { ResumeReason, StreamStepProgress, TriggerHookType, TriggerPayload } from '../engine'
import { ExecutionType } from '../flow-run/execution/execution-output'
import { RunEnvironment } from '../flow-run/flow-run'
@@ -370,14 +370,6 @@ export const EventDestinationJobData = z.object({
export type EventDestinationJobData = z.infer<typeof EventDestinationJobData>
-export enum AiStepAction {
- ASK_AI = 'ASK_AI',
- SUMMARIZE_TEXT = 'SUMMARIZE_TEXT',
- CLASSIFY_TEXT = 'CLASSIFY_TEXT',
- EXTRACT_STRUCTURED_DATA = 'EXTRACT_STRUCTURED_DATA',
- GENERATE_IMAGE = 'GENERATE_IMAGE',
-}
-
export const AiStepWebSearchOptions = z.object({
maxUses: z.number().optional(),
includeSources: z.boolean().optional(),
diff --git a/packages/core/execution/src/lib/workers/worker-contract.ts b/packages/core/execution/src/lib/workers/worker-contract.ts
index a95a6acd25b..6a907bfb38b 100644
--- a/packages/core/execution/src/lib/workers/worker-contract.ts
+++ b/packages/core/execution/src/lib/workers/worker-contract.ts
@@ -1,4 +1,4 @@
-import { ActivepiecesAiBilling, AIProviderName } from '@activepieces/core-utils'
+import { ActivepiecesAiBilling, AiChargeBasis, AIProviderName, AiProviderCredentials } from '@activepieces/core-utils'
import { AgentPieceToolMetadata, PiecePackage } from '@activepieces/core-piece-types'
import { StreamStepProgress } from '../engine/engine-operation'
import { GetFlowVersionForWorkerRequest, UploadRunLogsRequest } from '../engine/requests'
@@ -152,10 +152,8 @@ export type AgentAiToolsConfig = {
}
export type AgentConfigResponse = {
- provider: string
+ credentials: AiProviderCredentials
providerConfigId: string
- auth: Record<string, unknown>
- providerConfig: Record<string, unknown>
modelId: string
fastModelId: string
systemPrompt: string
@@ -365,9 +363,7 @@ export type PersonalizationConfigResponse =
| { claimed: false }
| {
claimed: true
- provider: string
- auth: Record<string, unknown>
- providerConfig: Record<string, unknown>
+ credentials: AiProviderCredentials
modelId: string
fastModelId: string
user: { firstName: string, lastName: string, email: string }
@@ -422,11 +418,8 @@ export type ResolveAiProviderRequest = {
providerConfigId?: string
}
-export type ResolveAiProviderResponse = {
- provider: AIProviderName
+export type ResolveAiProviderResponse = AiProviderCredentials & {
providerConfigId: string
- auth: Record<string, unknown>
- config: Record<string, unknown>
}
export type SaveFlowStepFileRequest = {
@@ -445,6 +438,8 @@ export type ReportAiUsageRequest = {
usage: AiUsageCharge
toolCalls?: number
generationId?: string
+ inputTokens?: number
+ outputTokens?: number
requestId?: string
flowRun?: AiUsageFlowRunContext
chat?: AiUsageChatContext
@@ -453,7 +448,6 @@ export type ReportAiUsageRequest = {
export type AiUsageFlowRunContext = {
flowId: string
flowRunId: string
- environment?: string
}
export type AiUsageChatContext = {
@@ -463,8 +457,8 @@ export type AiUsageChatContext = {
}
export type AiUsageCharge =
- | { type: 'observed-cost', costUsd: number }
- | { type: 'flat-credits', credits: number }
+ | { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: number }
+ | { type: AiChargeBasis.FIXED_CREDITS, credits: number }
export type ReadFlowStepFileRequest = {
projectId: string
diff --git a/packages/core/execution/test/workers/ai-job-data.test.ts b/packages/core/execution/test/workers/ai-job-data.test.ts
index ae1a5e4f2b8..f829ba2377b 100644
--- a/packages/core/execution/test/workers/ai-job-data.test.ts
+++ b/packages/core/execution/test/workers/ai-job-data.test.ts
@@ -1,5 +1,6 @@
+import { AiStepAction } from '@activepieces/core-utils'
import { describe, expect, it } from 'vitest'
-import { AiStepAction, ExecuteAiJobData, WorkerJobType } from '../../src/index'
+import { ExecuteAiJobData, WorkerJobType } from '../../src/index'
const base = {
schemaVersion: 1,
diff --git a/packages/core/piece-types/src/lib/ai-providers.ts b/packages/core/piece-types/src/lib/ai-providers.ts
index 3c082bf0ee9..3d43a882f8b 100644
--- a/packages/core/piece-types/src/lib/ai-providers.ts
+++ b/packages/core/piece-types/src/lib/ai-providers.ts
@@ -378,6 +378,14 @@ export const DEFAULT_CHAT_TIER_ID = 'smart' as const
export type ActivepiecesChatTier = typeof ACTIVEPIECES_CHAT_TIERS[number]
+export const ACTIVEPIECES_IMAGE_TIERS = [
+ { id: 'fast', label: 'Fast', modelId: 'google/gemini-3.1-flash-lite-image' },
+ { id: 'smart', label: 'Expert', modelId: 'google/gemini-3.1-flash-image' },
+ { id: 'premium', label: 'Heavy', modelId: 'google/gemini-3-pro-image' },
+] as const
+
+export type ActivepiecesImageTier = typeof ACTIVEPIECES_IMAGE_TIERS[number]
+
export const AI_PROVIDER_CAPABILITIES: Record<AIProviderName, AIProviderCapabilities> = {
[AIProviderName.OPENAI]: buildProviderCapabilities(AIProviderName.OPENAI),
[AIProviderName.ANTHROPIC]: buildProviderCapabilities(AIProviderName.ANTHROPIC),
diff --git a/packages/core/shared/package.json b/packages/core/shared/package.json
index 6a26621c547..7d780eccb1c 100644
--- a/packages/core/shared/package.json
+++ b/packages/core/shared/package.json
@@ -1,6 +1,6 @@
{
"name": "@activepieces/shared",
- "version": "0.172.0",
+ "version": "0.173.0",
"type": "commonjs",
"sideEffects": false,
"main": "./dist/src/index.js",
diff --git a/packages/core/shared/src/lib/form-errors.ts b/packages/core/shared/src/lib/form-errors.ts
index a99fa4e5932..458ac4e895b 100644
--- a/packages/core/shared/src/lib/form-errors.ts
+++ b/packages/core/shared/src/lib/form-errors.ts
@@ -1,11 +1 @@
-export const formErrors = {
- required: 'required',
- invalidGitRepoSlug: 'invalidGitRepoSlug',
- invalidGitRepoBranch: 'invalidGitRepoBranch',
- invalidGitRepoRemoteUrl: 'invalidGitRepoRemoteUrl',
- invalidExternalId: 'invalidExternalId',
- invalidFileName: 'invalidFileName',
- messageRequiresContentOrFiles: 'messageRequiresContentOrFiles',
- agentConfigTooLarge: 'agentConfigTooLarge',
- invalidGcpResourceId: 'invalidGcpResourceId',
-} as const
+export { formErrors, SAFE_EXTERNAL_ID_PATTERN } from '@activepieces/core-utils'
diff --git a/packages/core/shared/src/lib/management/ai-providers/index.ts b/packages/core/shared/src/lib/management/ai-providers/index.ts
index 8ecd9f0465c..fc936e92aea 100644
--- a/packages/core/shared/src/lib/management/ai-providers/index.ts
+++ b/packages/core/shared/src/lib/management/ai-providers/index.ts
@@ -1,155 +1,7 @@
-import { AiProviderKeyStatus, AIProviderName, BaseModelSchema } from '@activepieces/core-utils'
+import { ActivePiecesProviderAuthConfig, ActivePiecesProviderConfig, AIProviderAuthConfig, AIProviderConfig, AiProviderCredentials, AiProviderKeyStatus, AIProviderModelType, AIProviderName, AnthropicProviderAuthConfig, AnthropicProviderConfig, AzureProviderAuthConfig, AzureProviderConfig, BaseAIProviderAuthConfig, BaseModelSchema, BedrockProviderAuthConfig, BedrockProviderConfig, CloudflareGatewayProviderAuthConfig, CloudflareGatewayProviderConfig, GoogleProviderAuthConfig, GoogleProviderConfig, MistralProviderAuthConfig, MistralProviderConfig, OpenAICompatibleProviderAuthConfig, OpenAICompatibleProviderConfig, OpenAiCompatibleVendorConfig, OpenAIProviderAuthConfig, OpenAIProviderConfig, OpenRouterProviderAuthConfig, OpenRouterProviderConfig, VertexProviderAuthConfig, VertexProviderConfig } from '@activepieces/core-utils'
import { z } from 'zod'
-import { formErrors } from '../../form-errors'
-export enum AIProviderModelType {
- IMAGE = 'image',
- TEXT = 'text',
-}
-
-export const BaseAIProviderAuthConfig = z.object({
- apiKey: z.string(),
-})
-export type BaseAIProviderAuthConfig = z.infer<typeof BaseAIProviderAuthConfig>
-
-export const VertexProviderAuthConfig = z.object({
- serviceAccountJson: z.string().min(1),
-})
-export type VertexProviderAuthConfig = z.infer<typeof VertexProviderAuthConfig>
-
-export const AnthropicProviderAuthConfig = BaseAIProviderAuthConfig
-export type AnthropicProviderAuthConfig = z.infer<typeof AnthropicProviderAuthConfig>
-
-export const ActivePiecesProviderAuthConfig = BaseAIProviderAuthConfig.extend({
- apiKeyHash: z.string(),
-})
-export type ActivePiecesProviderAuthConfig = z.infer<typeof ActivePiecesProviderAuthConfig>
-
-export const OpenAICompatibleProviderAuthConfig = BaseAIProviderAuthConfig
-export type OpenAICompatibleProviderAuthConfig = z.infer<typeof OpenAICompatibleProviderAuthConfig>
-
-export const CloudflareGatewayProviderAuthConfig = BaseAIProviderAuthConfig
-export type CloudflareGatewayProviderAuthConfig = z.infer<typeof CloudflareGatewayProviderAuthConfig>
-
-export const AzureProviderAuthConfig = BaseAIProviderAuthConfig
-export type AzureProviderAuthConfig = z.infer<typeof AzureProviderAuthConfig>
-
-export const GoogleProviderAuthConfig = BaseAIProviderAuthConfig
-export type GoogleProviderAuthConfig = z.infer<typeof GoogleProviderAuthConfig>
-
-export const OpenAIProviderAuthConfig = BaseAIProviderAuthConfig
-export type OpenAIProviderAuthConfig = z.infer<typeof OpenAIProviderAuthConfig>
-
-export const OpenRouterProviderAuthConfig = BaseAIProviderAuthConfig
-export type OpenRouterProviderAuthConfig = z.infer<typeof OpenRouterProviderAuthConfig>
-
-export const BedrockProviderAuthConfig = z.object({
- accessKeyId: z.string().min(1),
- secretAccessKey: z.string().min(1),
-})
-export type BedrockProviderAuthConfig = z.infer<typeof BedrockProviderAuthConfig>
-
-export const MistralProviderAuthConfig = BaseAIProviderAuthConfig
-export type MistralProviderAuthConfig = z.infer<typeof MistralProviderAuthConfig>
-
-export const AnthropicProviderConfig = z.object({})
-export type AnthropicProviderConfig = z.infer<typeof AnthropicProviderConfig>
-
-export const ActivePiecesProviderConfig = z.object({})
-export type ActivePiecesProviderConfig = z.infer<typeof ActivePiecesProviderConfig>
-
-export const ProviderModelConfig = z.object({
- modelId: z.string(),
- modelName: z.string(),
- modelType: z.nativeEnum(AIProviderModelType),
-})
-export type ProviderModelConfig = z.infer<typeof ProviderModelConfig>
-
-export const OpenAICompatibleProviderConfig = z.object({
- apiKeyHeader: z.string(),
- baseUrl: z.string(),
- models: z.array(ProviderModelConfig),
- defaultHeaders: z.record(z.string(), z.string()).optional(),
- apiStyle: z.enum(['chat', 'responses']).optional(),
-})
-export type OpenAICompatibleProviderConfig = z.infer<typeof OpenAICompatibleProviderConfig>
-
-
-export const CloudflareGatewayProviderConfig = z.object({
- accountId: z.string(),
- gatewayId: z.string(),
- models: z.array(ProviderModelConfig),
- vertexProject: z.string().optional(),
- vertexRegion: z.string().optional(),
-})
-export type CloudflareGatewayProviderConfig = z.infer<typeof CloudflareGatewayProviderConfig>
-
-export const AzureProviderConfig = z.object({
- resourceName: z.string(),
- apiVersion: z.preprocess(
- (v) => (typeof v === 'string' && v.trim().length === 0 ? undefined : v),
- z.string().optional(),
- ),
-})
-export type AzureProviderConfig = z.infer<typeof AzureProviderConfig>
-
-export const GoogleProviderConfig = z.object({})
-export type GoogleProviderConfig = z.infer<typeof GoogleProviderConfig>
-
-export const OpenAIProviderConfig = z.object({})
-export type OpenAIProviderConfig = z.infer<typeof OpenAIProviderConfig>
-
-export const OpenRouterProviderConfig = z.object({})
-export type OpenRouterProviderConfig = z.infer<typeof OpenRouterProviderConfig>
-
-export const BedrockProviderConfig = z.object({
- region: z.string().min(1),
-})
-export type BedrockProviderConfig = z.infer<typeof BedrockProviderConfig>
-
-export const VertexProviderConfig = z.object({
- project: z.string().regex(/^[a-z0-9][a-z0-9-]{0,62}$/, formErrors.invalidGcpResourceId),
- region: z.string().regex(/^[a-z0-9][a-z0-9-]{0,62}$/, formErrors.invalidGcpResourceId),
- models: z.array(ProviderModelConfig),
-})
-export type VertexProviderConfig = z.infer<typeof VertexProviderConfig>
-
-export const MistralProviderConfig = z.object({})
-export type MistralProviderConfig = z.infer<typeof MistralProviderConfig>
-
-export const OpenAiCompatibleVendorConfig = z.object({})
-export type OpenAiCompatibleVendorConfig = z.infer<typeof OpenAiCompatibleVendorConfig>
-
-export const AIProviderAuthConfig = z.union([
- AnthropicProviderAuthConfig,
- AzureProviderAuthConfig,
- GoogleProviderAuthConfig,
- OpenAIProviderAuthConfig,
- OpenRouterProviderAuthConfig,
- CloudflareGatewayProviderAuthConfig,
- OpenAICompatibleProviderAuthConfig,
- ActivePiecesProviderAuthConfig,
- BedrockProviderAuthConfig,
- VertexProviderAuthConfig,
- MistralProviderAuthConfig,
-])
-export type AIProviderAuthConfig = z.infer<typeof AIProviderAuthConfig>
-// Order matters, put schemas with required fields first, empty ones last. This is to avoid empty objects matching any object.
-export const AIProviderConfig = z.union([
- OpenAICompatibleProviderConfig,
- CloudflareGatewayProviderConfig,
- AzureProviderConfig,
- VertexProviderConfig,
- BedrockProviderConfig,
- AnthropicProviderConfig,
- GoogleProviderConfig,
- OpenAIProviderConfig,
- OpenRouterProviderConfig,
- ActivePiecesProviderConfig,
- MistralProviderConfig,
- OpenAiCompatibleVendorConfig,
-])
-export type AIProviderConfig = z.infer<typeof AIProviderConfig>
+export { ActivePiecesProviderAuthConfig, ActivePiecesProviderConfig, AIProviderAuthConfig, AIProviderConfig, AiProviderCredentials, aiProviderCredentials, AIProviderModelType, AnthropicProviderAuthConfig, AnthropicProviderConfig, AzureProviderAuthConfig, AzureProviderConfig, BaseAIProviderAuthConfig, BedrockProviderAuthConfig, BedrockProviderConfig, CloudflareGatewayProviderAuthConfig, CloudflareGatewayProviderConfig, GoogleProviderAuthConfig, GoogleProviderConfig, MistralProviderAuthConfig, MistralProviderConfig, OpenAICompatibleProviderAuthConfig, OpenAICompatibleProviderConfig, OpenAiCompatibleVendorConfig, OpenAIProviderAuthConfig, OpenAIProviderConfig, OpenRouterProviderAuthConfig, OpenRouterProviderConfig, ProviderModelConfig, VertexProviderAuthConfig, VertexProviderConfig } from '@activepieces/core-utils'
const ProviderConfigUnion = z.discriminatedUnion('provider', [
z.object({
@@ -293,7 +145,7 @@ export const ProjectAIProviderKey = z.object({
export type ProjectAIProviderKey = z.infer<typeof ProjectAIProviderKey>
export const ProjectAIProvider = z.object({
- provider: z.nativeEnum(AIProviderName),
+ provider: z.enum(AIProviderName),
name: z.string(),
enabledForChat: z.boolean(),
keys: z.array(ProjectAIProviderKey),
@@ -338,14 +190,11 @@ export type UpdateAIProviderRequest = z.infer<typeof UpdateAIProviderRequest>
export const GetProviderConfigResponse = z.object({
- provider: z.nativeEnum(AIProviderName),
configId: z.string(),
- config: AIProviderConfig,
- auth: AIProviderAuthConfig,
platformId: z.string(),
modelScope: AiProviderModelScope,
modelIds: z.array(z.string()),
-})
+}).and(AiProviderCredentials)
export type GetProviderConfigResponse = z.infer<typeof GetProviderConfigResponse>
@@ -455,11 +304,12 @@ export {
AI_PROVIDER_ENTITY_TYPES,
ALLOWED_CHAT_MODELS_BY_PROVIDER,
ACTIVEPIECES_CHAT_TIERS,
+ ACTIVEPIECES_IMAGE_TIERS,
DEFAULT_CHAT_TIER_ID,
AI_PROVIDER_CAPABILITIES,
OPENAI_COMPATIBLE_VENDOR_BASE_URLS,
aiProviderUtils,
} from '@activepieces/core-piece-types'
-export type { ActivepiecesChatTier, AIProviderCapabilities, AIWebSearchMode, OpenAiCompatibleVendor } from '@activepieces/core-piece-types'
+export type { ActivepiecesChatTier, ActivepiecesImageTier, AIProviderCapabilities, AIWebSearchMode, OpenAiCompatibleVendor } from '@activepieces/core-piece-types'
export const AI_PIECE_COST_BILLING_VERSION = '0.11.0'
diff --git a/packages/core/utils/src/index.ts b/packages/core/utils/src/index.ts
index 60ad2d7bca5..961246e206e 100644
--- a/packages/core/utils/src/index.ts
+++ b/packages/core/utils/src/index.ts
@@ -10,6 +10,8 @@ export * from './lib/seek-page'
export * from './lib/locale'
export * from './lib/permission'
export * from './lib/activepieces-ai-billing'
+export * from './lib/ai-provider-credentials'
+export * from './lib/ai-step'
export * from './lib/mustache-utils'
export * from './lib/friendly-piece-error'
export * from './lib/color'
diff --git a/packages/core/utils/src/lib/activepieces-ai-billing.ts b/packages/core/utils/src/lib/activepieces-ai-billing.ts
index 43961088177..b45eb6b1dfc 100644
--- a/packages/core/utils/src/lib/activepieces-ai-billing.ts
+++ b/packages/core/utils/src/lib/activepieces-ai-billing.ts
@@ -1,20 +1,32 @@
+import { AiProviderCredentials } from './ai-provider-credentials'
import { AIProviderName } from './permission'
+import { isNil } from './utils'
-export enum ActivepiecesAiBillingScope {
- PLATFORM = 'platform',
- PROJECT = 'project',
- CONVERSATION = 'conversation',
+export function aiChargeFor({ credentials, turnAlreadyCharged = false }: AiChargeForParams): AiCharge | undefined {
+ if (credentials.provider !== AIProviderName.ACTIVEPIECES) {
+ return turnAlreadyCharged ? undefined : { basis: AiChargeBasis.FIXED_CREDITS }
+ }
+ const { apiKey } = credentials.auth
+ return { basis: AiChargeBasis.PROVIDER_REPORTED_COST, managedApiKey: isNil(apiKey) || apiKey.length === 0 ? undefined : apiKey }
+}
+
+export enum ActivepiecesAiConsumerSource {
+ AI_STEP_IN_FLOW = 'ai-step-in-flow',
+ CHAT = 'chat',
+}
+
+export enum AiChargeBasis {
+ PROVIDER_REPORTED_COST = 'provider-reported-cost',
+ FIXED_CREDITS = 'fixed-credits',
}
export type ActivepiecesAiBilling =
- | { scope: ActivepiecesAiBillingScope.PLATFORM, platformId: string }
- | { scope: ActivepiecesAiBillingScope.PROJECT, platformId: string, projectId: string, flowRun?: ActivepiecesAiFlowRun }
- | { scope: ActivepiecesAiBillingScope.CONVERSATION, platformId: string, projectId: string | null, conversationId: string, chat?: ActivepiecesAiChat }
+ | { source: ActivepiecesAiConsumerSource.AI_STEP_IN_FLOW, platformId: string, projectId: string, flowRun: ActivepiecesAiFlowRun }
+ | { source: ActivepiecesAiConsumerSource.CHAT, platformId: string, projectId: string | null, conversationId: string, chat?: ActivepiecesAiChat }
export type ActivepiecesAiFlowRun = {
flowId: string
flowRunId: string
- environment?: string
}
export type ActivepiecesAiChat = {
@@ -23,9 +35,24 @@ export type ActivepiecesAiChat = {
tier: string
}
-export type ActivepiecesAiCall =
- | { charge: 'observed-cost', generationId: string, costUsd: number, inputTokens?: number, outputTokens?: number }
- | { charge: 'flat-credits', credits: number, generationId?: string }
+export type AiCharge =
+ | { basis: AiChargeBasis.PROVIDER_REPORTED_COST, managedApiKey?: string }
+ | { basis: AiChargeBasis.FIXED_CREDITS }
+
+export type AiChargeForParams = {
+ credentials: AiProviderCredentials
+ turnAlreadyCharged?: boolean
+}
+
+export type AiCallTokens = {
+ inputTokens?: number
+ outputTokens?: number
+}
+
+export type ActivepiecesAiCall = AiCallTokens & (
+ | { charge: AiChargeBasis.PROVIDER_REPORTED_COST, generationId?: string, costUsd: number }
+ | { charge: AiChargeBasis.FIXED_CREDITS, credits: number, generationId?: string }
+)
export type ActivepiecesAiCostEvent = {
billing: ActivepiecesAiBilling
diff --git a/packages/core/utils/src/lib/ai-provider-credentials.ts b/packages/core/utils/src/lib/ai-provider-credentials.ts
new file mode 100644
index 00000000000..c9bcf5bb054
--- /dev/null
+++ b/packages/core/utils/src/lib/ai-provider-credentials.ts
@@ -0,0 +1,185 @@
+import { z } from 'zod'
+import { formErrors } from './form-errors'
+import { AIProviderName } from './permission'
+
+export enum AIProviderModelType {
+ IMAGE = 'image',
+ TEXT = 'text',
+}
+
+export const BaseAIProviderAuthConfig = z.object({
+ apiKey: z.string(),
+})
+export type BaseAIProviderAuthConfig = z.infer<typeof BaseAIProviderAuthConfig>
+
+export const VertexProviderAuthConfig = z.object({
+ serviceAccountJson: z.string().min(1),
+})
+export type VertexProviderAuthConfig = z.infer<typeof VertexProviderAuthConfig>
+
+export const AnthropicProviderAuthConfig = BaseAIProviderAuthConfig
+export type AnthropicProviderAuthConfig = z.infer<typeof AnthropicProviderAuthConfig>
+
+export const ActivePiecesProviderAuthConfig = BaseAIProviderAuthConfig.extend({
+ apiKeyHash: z.string(),
+})
+export type ActivePiecesProviderAuthConfig = z.infer<typeof ActivePiecesProviderAuthConfig>
+
+export const OpenAICompatibleProviderAuthConfig = BaseAIProviderAuthConfig
+export type OpenAICompatibleProviderAuthConfig = z.infer<typeof OpenAICompatibleProviderAuthConfig>
+
+export const CloudflareGatewayProviderAuthConfig = BaseAIProviderAuthConfig
+export type CloudflareGatewayProviderAuthConfig = z.infer<typeof CloudflareGatewayProviderAuthConfig>
+
+export const AzureProviderAuthConfig = BaseAIProviderAuthConfig
+export type AzureProviderAuthConfig = z.infer<typeof AzureProviderAuthConfig>
+
+export const GoogleProviderAuthConfig = BaseAIProviderAuthConfig
+export type GoogleProviderAuthConfig = z.infer<typeof GoogleProviderAuthConfig>
+
+export const OpenAIProviderAuthConfig = BaseAIProviderAuthConfig
+export type OpenAIProviderAuthConfig = z.infer<typeof OpenAIProviderAuthConfig>
+
+export const OpenRouterProviderAuthConfig = BaseAIProviderAuthConfig
+export type OpenRouterProviderAuthConfig = z.infer<typeof OpenRouterProviderAuthConfig>
+
+export const BedrockProviderAuthConfig = z.object({
+ accessKeyId: z.string().min(1),
+ secretAccessKey: z.string().min(1),
+})
+export type BedrockProviderAuthConfig = z.infer<typeof BedrockProviderAuthConfig>
+
+export const MistralProviderAuthConfig = BaseAIProviderAuthConfig
+export type MistralProviderAuthConfig = z.infer<typeof MistralProviderAuthConfig>
+
+export const AnthropicProviderConfig = z.object({})
+export type AnthropicProviderConfig = z.infer<typeof AnthropicProviderConfig>
+
+export const ActivePiecesProviderConfig = z.object({})
+export type ActivePiecesProviderConfig = z.infer<typeof ActivePiecesProviderConfig>
+
+export const ProviderModelConfig = z.object({
+ modelId: z.string(),
+ modelName: z.string(),
+ modelType: z.enum(AIProviderModelType),
+})
+export type ProviderModelConfig = z.infer<typeof ProviderModelConfig>
+
+export const OpenAICompatibleProviderConfig = z.object({
+ apiKeyHeader: z.string(),
+ baseUrl: z.string(),
+ models: z.array(ProviderModelConfig),
+ defaultHeaders: z.record(z.string(), z.string()).optional(),
+ apiStyle: z.enum(['chat', 'responses']).optional(),
+})
+export type OpenAICompatibleProviderConfig = z.infer<typeof OpenAICompatibleProviderConfig>
+
+
+export const CloudflareGatewayProviderConfig = z.object({
+ accountId: z.string(),
+ gatewayId: z.string(),
+ models: z.array(ProviderModelConfig),
+ vertexProject: z.string().optional(),
+ vertexRegion: z.string().optional(),
+})
+export type CloudflareGatewayProviderConfig = z.infer<typeof CloudflareGatewayProviderConfig>
+
+export const AzureProviderConfig = z.object({
+ resourceName: z.string(),
+ apiVersion: z.preprocess(
+ (v) => (typeof v === 'string' && v.trim().length === 0 ? undefined : v),
+ z.string().optional(),
+ ),
+})
+export type AzureProviderConfig = z.infer<typeof AzureProviderConfig>
+
+export const GoogleProviderConfig = z.object({})
+export type GoogleProviderConfig = z.infer<typeof GoogleProviderConfig>
+
+export const OpenAIProviderConfig = z.object({})
+export type OpenAIProviderConfig = z.infer<typeof OpenAIProviderConfig>
+
+export const OpenRouterProviderConfig = z.object({})
+export type OpenRouterProviderConfig = z.infer<typeof OpenRouterProviderConfig>
+
+export const BedrockProviderConfig = z.object({
+ region: z.string().min(1),
+})
+export type BedrockProviderConfig = z.infer<typeof BedrockProviderConfig>
+
+export const VertexProviderConfig = z.object({
+ project: z.string().regex(/^[a-z0-9][a-z0-9-]{0,62}$/, formErrors.invalidGcpResourceId),
+ region: z.string().regex(/^[a-z0-9][a-z0-9-]{0,62}$/, formErrors.invalidGcpResourceId),
+ models: z.array(ProviderModelConfig),
+})
+export type VertexProviderConfig = z.infer<typeof VertexProviderConfig>
+
+export const MistralProviderConfig = z.object({})
+export type MistralProviderConfig = z.infer<typeof MistralProviderConfig>
+
+export const OpenAiCompatibleVendorConfig = z.object({})
+export type OpenAiCompatibleVendorConfig = z.infer<typeof OpenAiCompatibleVendorConfig>
+
+export const AIProviderAuthConfig = z.union([
+ AnthropicProviderAuthConfig,
+ AzureProviderAuthConfig,
+ GoogleProviderAuthConfig,
+ OpenAIProviderAuthConfig,
+ OpenRouterProviderAuthConfig,
+ CloudflareGatewayProviderAuthConfig,
+ OpenAICompatibleProviderAuthConfig,
+ ActivePiecesProviderAuthConfig,
+ BedrockProviderAuthConfig,
+ VertexProviderAuthConfig,
+ MistralProviderAuthConfig,
+])
+export type AIProviderAuthConfig = z.infer<typeof AIProviderAuthConfig>
+// Order matters, put schemas with required fields first, empty ones last. This is to avoid empty objects matching any object.
+export const AIProviderConfig = z.union([
+ OpenAICompatibleProviderConfig,
+ CloudflareGatewayProviderConfig,
+ AzureProviderConfig,
+ VertexProviderConfig,
+ BedrockProviderConfig,
+ AnthropicProviderConfig,
+ GoogleProviderConfig,
+ OpenAIProviderConfig,
+ OpenRouterProviderConfig,
+ ActivePiecesProviderConfig,
+ MistralProviderConfig,
+ OpenAiCompatibleVendorConfig,
+])
+export type AIProviderConfig = z.infer<typeof AIProviderConfig>
+
+export const AiProviderCredentials = z.discriminatedUnion('provider', [
+ z.object({ provider: z.literal(AIProviderName.OPENAI), config: OpenAIProviderConfig.partial(), auth: OpenAIProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.OPENROUTER), config: OpenRouterProviderConfig.partial(), auth: OpenRouterProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.ANTHROPIC), config: AnthropicProviderConfig.partial(), auth: AnthropicProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.AZURE), config: AzureProviderConfig.partial(), auth: AzureProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.GOOGLE), config: GoogleProviderConfig.partial(), auth: GoogleProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.CLOUDFLARE_GATEWAY), config: CloudflareGatewayProviderConfig.partial(), auth: CloudflareGatewayProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.CUSTOM), config: OpenAICompatibleProviderConfig.partial(), auth: OpenAICompatibleProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.ACTIVEPIECES), config: ActivePiecesProviderConfig.partial(), auth: ActivePiecesProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.BEDROCK), config: BedrockProviderConfig.partial(), auth: BedrockProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.VERTEX), config: VertexProviderConfig.partial(), auth: VertexProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.MISTRAL), config: MistralProviderConfig.partial(), auth: MistralProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.XAI), config: OpenAiCompatibleVendorConfig.partial(), auth: BaseAIProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.DEEPSEEK), config: OpenAiCompatibleVendorConfig.partial(), auth: BaseAIProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.ZAI), config: OpenAiCompatibleVendorConfig.partial(), auth: BaseAIProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.QWEN), config: OpenAiCompatibleVendorConfig.partial(), auth: BaseAIProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.MINIMAX), config: OpenAiCompatibleVendorConfig.partial(), auth: BaseAIProviderAuthConfig.partial() }),
+ z.object({ provider: z.literal(AIProviderName.MOONSHOT), config: OpenAiCompatibleVendorConfig.partial(), auth: BaseAIProviderAuthConfig.partial() }),
+])
+export type AiProviderCredentials = z.infer<typeof AiProviderCredentials>
+
+export function aiProviderCredentials({ provider, auth, config }: { provider: AIProviderName, auth: unknown, config: unknown }): AiProviderCredentials {
+ const parsed = AiProviderCredentials.safeParse({ provider, auth, config })
+ if (parsed.success) {
+ return parsed.data
+ }
+ const authOnly = AiProviderCredentials.safeParse({ provider, auth, config: {} })
+ if (authOnly.success) {
+ return authOnly.data
+ }
+ return AiProviderCredentials.parse({ provider, auth: {}, config: {} })
+}
diff --git a/packages/core/utils/src/lib/ai-step.ts b/packages/core/utils/src/lib/ai-step.ts
new file mode 100644
index 00000000000..3da62518caf
--- /dev/null
+++ b/packages/core/utils/src/lib/ai-step.ts
@@ -0,0 +1,7 @@
+export enum AiStepAction {
+ ASK_AI = 'ASK_AI',
+ SUMMARIZE_TEXT = 'SUMMARIZE_TEXT',
+ CLASSIFY_TEXT = 'CLASSIFY_TEXT',
+ EXTRACT_STRUCTURED_DATA = 'EXTRACT_STRUCTURED_DATA',
+ GENERATE_IMAGE = 'GENERATE_IMAGE',
+}
diff --git a/packages/core/utils/src/lib/form-errors.ts b/packages/core/utils/src/lib/form-errors.ts
index 28e3739e92a..d4f1545d740 100644
--- a/packages/core/utils/src/lib/form-errors.ts
+++ b/packages/core/utils/src/lib/form-errors.ts
@@ -6,6 +6,8 @@ export const formErrors = {
invalidExternalId: 'invalidExternalId',
invalidFileName: 'invalidFileName',
messageRequiresContentOrFiles: 'messageRequiresContentOrFiles',
+ agentConfigTooLarge: 'agentConfigTooLarge',
+ invalidGcpResourceId: 'invalidGcpResourceId',
} as const
export const SAFE_EXTERNAL_ID_PATTERN = /^(?!\.{1,2}$)[A-Za-z0-9._-]{1,128}$/
diff --git a/packages/core/utils/test/ai-provider-credentials.test.ts b/packages/core/utils/test/ai-provider-credentials.test.ts
new file mode 100644
index 00000000000..004dc77353c
--- /dev/null
+++ b/packages/core/utils/test/ai-provider-credentials.test.ts
@@ -0,0 +1,39 @@
+import { describe, expect, it } from 'vitest'
+import { aiProviderCredentials, AIProviderModelType } from '../src/lib/ai-provider-credentials'
+import { AIProviderName } from '../src/lib/permission'
+
+const customConfig = {
+ apiKeyHeader: 'Authorization',
+ baseUrl: 'https://example.com/v1',
+ models: [{ modelId: 'gpt-4o', modelName: 'GPT-4o', modelType: AIProviderModelType.TEXT }],
+}
+
+describe('aiProviderCredentials', () => {
+ it('returns a well-formed row unchanged', () => {
+ const credentials = aiProviderCredentials({
+ provider: AIProviderName.CUSTOM,
+ auth: { apiKey: 'sk-live' },
+ config: customConfig,
+ })
+ expect(credentials).toEqual({ provider: AIProviderName.CUSTOM, auth: { apiKey: 'sk-live' }, config: customConfig })
+ })
+
+ it('keeps the key of a row whose config predates a tightened rule, rather than failing the read', () => {
+ const credentials = aiProviderCredentials({
+ provider: AIProviderName.VERTEX,
+ auth: { serviceAccountJson: '{"type":"service_account"}' },
+ config: { project: 'my-project', region: 'US-CENTRAL1', models: [] },
+ })
+ expect(credentials.auth).toEqual({ serviceAccountJson: '{"type":"service_account"}' })
+ expect(credentials.config).toEqual({})
+ })
+
+ it('returns an empty row when the auth itself cannot be parsed', () => {
+ const credentials = aiProviderCredentials({
+ provider: AIProviderName.CUSTOM,
+ auth: { apiKey: 42 },
+ config: customConfig,
+ })
+ expect(credentials).toEqual({ provider: AIProviderName.CUSTOM, auth: {}, config: {} })
+ })
+})
diff --git a/packages/pieces/community/ai/src/index.ts b/packages/pieces/community/ai/src/index.ts
index 9434d9d9ca1..b4149c5bb07 100644
--- a/packages/pieces/community/ai/src/index.ts
+++ b/packages/pieces/community/ai/src/index.ts
@@ -12,7 +12,7 @@ import { runAgent } from "./lib/actions/agents/run-agent";
export const ai = createPiece({
displayName: "AI",
auth: PieceAuth.None(),
- minimumSupportedRelease: '0.91.0',
+ minimumSupportedRelease: '0.92.0',
categories: [
PieceCategory.ARTIFICIAL_INTELLIGENCE,
PieceCategory.UNIVERSAL_AI,
diff --git a/packages/pieces/community/ai/src/lib/actions/image/generate-image.ts b/packages/pieces/community/ai/src/lib/actions/image/generate-image.ts
index 1fdb5f3e25c..a27cac35a51 100644
--- a/packages/pieces/community/ai/src/lib/actions/image/generate-image.ts
+++ b/packages/pieces/community/ai/src/lib/actions/image/generate-image.ts
@@ -1,5 +1,6 @@
import {
AIProviderName,
+ AiStepAction,
ApFile,
createAction,
DynamicPropsValue,
@@ -143,7 +144,7 @@ export const generateImageAction = createAction({
const result = await runOnWorker({
context,
buildRequest: async () => ({
- action: 'GENERATE_IMAGE',
+ action: AiStepAction.GENERATE_IMAGE,
provider,
...spreadIfDefined('providerConfigId', configId),
modelId: context.propsValue.model,
diff --git a/packages/pieces/community/ai/src/lib/actions/text/ask-ai.ts b/packages/pieces/community/ai/src/lib/actions/text/ask-ai.ts
index 773ef92e11e..3ff0d873e33 100644
--- a/packages/pieces/community/ai/src/lib/actions/text/ask-ai.ts
+++ b/packages/pieces/community/ai/src/lib/actions/text/ask-ai.ts
@@ -1,4 +1,4 @@
-import { createAction, isNil, Property, spreadIfDefined } from '@activepieces/pieces-framework';
+import { AiStepAction, createAction, isNil, Property, spreadIfDefined } from '@activepieces/pieces-framework';
import { runOnWorker } from '../../common/ai-step';
import { aiProps, aiProviderSelection } from '../../common/props';
import { buildWebSearchOptionsProperty, sanitizeWebSearchOptions, usesNativeWebSearchTools } from '../../common/web-search';
@@ -60,7 +60,7 @@ export const askAI = createAction({
const result = await runOnWorker({
context,
buildRequest: async () => ({
- action: 'ASK_AI',
+ action: AiStepAction.ASK_AI,
provider,
...spreadIfDefined('providerConfigId', configId),
modelId: context.propsValue.model,
diff --git a/packages/pieces/community/ai/src/lib/actions/text/summarize-text.ts b/packages/pieces/community/ai/src/lib/actions/text/summarize-text.ts
index b85d4b48f89..2e64f0a1f2e 100644
--- a/packages/pieces/community/ai/src/lib/actions/text/summarize-text.ts
+++ b/packages/pieces/community/ai/src/lib/actions/text/summarize-text.ts
@@ -1,4 +1,4 @@
-import { createAction, Property, spreadIfDefined } from '@activepieces/pieces-framework';
+import { AiStepAction, createAction, Property, spreadIfDefined } from '@activepieces/pieces-framework';
import { runOnWorker } from '../../common/ai-step';
import { aiProps, aiProviderSelection } from '../../common/props';
@@ -34,7 +34,7 @@ export const summarizeText = createAction({
const result = await runOnWorker({
context,
buildRequest: async () => ({
- action: 'SUMMARIZE_TEXT',
+ action: AiStepAction.SUMMARIZE_TEXT,
provider,
...spreadIfDefined('providerConfigId', configId),
modelId: context.propsValue.model,
diff --git a/packages/pieces/community/ai/src/lib/actions/utility/classify-text.ts b/packages/pieces/community/ai/src/lib/actions/utility/classify-text.ts
index 714559bb1e5..40b7a8ad008 100644
--- a/packages/pieces/community/ai/src/lib/actions/utility/classify-text.ts
+++ b/packages/pieces/community/ai/src/lib/actions/utility/classify-text.ts
@@ -1,4 +1,4 @@
-import { createAction, Property, spreadIfDefined } from '@activepieces/pieces-framework';
+import { AiStepAction, createAction, Property, spreadIfDefined } from '@activepieces/pieces-framework';
import { runOnWorker } from '../../common/ai-step';
import { aiProps, aiProviderSelection } from '../../common/props';
@@ -29,7 +29,7 @@ export const classifyText = createAction({
const result = await runOnWorker({
context,
buildRequest: async () => ({
- action: 'CLASSIFY_TEXT',
+ action: AiStepAction.CLASSIFY_TEXT,
provider,
...spreadIfDefined('providerConfigId', configId),
modelId: context.propsValue.model,
diff --git a/packages/pieces/community/ai/src/lib/actions/utility/extract-structured-data.ts b/packages/pieces/community/ai/src/lib/actions/utility/extract-structured-data.ts
index cfef9bb5269..6d84673ad44 100644
--- a/packages/pieces/community/ai/src/lib/actions/utility/extract-structured-data.ts
+++ b/packages/pieces/community/ai/src/lib/actions/utility/extract-structured-data.ts
@@ -1,4 +1,4 @@
-import { ApFile, createAction, PieceAuth, Property, spreadIfDefined } from '@activepieces/pieces-framework';
+import { AiStepAction, ApFile, createAction, PieceAuth, Property, spreadIfDefined } from '@activepieces/pieces-framework';
import Ajv from 'ajv';
import mime from 'mime-types';
import { runOnWorker, uploadAiFiles } from '../../common/ai-step';
@@ -140,7 +140,7 @@ export const extractStructuredData = createAction({
const result = await runOnWorker({
context,
buildRequest: async () => ({
- action: 'EXTRACT_STRUCTURED_DATA',
+ action: AiStepAction.EXTRACT_STRUCTURED_DATA,
provider,
...spreadIfDefined('providerConfigId', configId),
modelId: context.propsValue.model,
diff --git a/packages/pieces/community/ai/src/lib/common/ai-step.ts b/packages/pieces/community/ai/src/lib/common/ai-step.ts
index 10eaba85803..7104bc78e25 100644
--- a/packages/pieces/community/ai/src/lib/common/ai-step.ts
+++ b/packages/pieces/community/ai/src/lib/common/ai-step.ts
@@ -1,5 +1,5 @@
import { AuthenticationType, httpClient, HttpMethod } from '@activepieces/pieces-common';
-import { ActionContext, ApFile, ExecutionType, InputPropertyMap, isNil, spreadIfDefined } from '@activepieces/pieces-framework';
+import { ActionContext, AiStepAction, ApFile, ExecutionType, InputPropertyMap, isNil, spreadIfDefined } from '@activepieces/pieces-framework';
export async function runOnWorker(params: {
context: AiStepContext;
@@ -91,13 +91,6 @@ export type AiStepContext = Pick<
'server' | 'flows' | 'files' | 'run' | 'executionType'
> & { resumePayload?: { body?: unknown } };
-export type AiStepAction =
- | 'ASK_AI'
- | 'SUMMARIZE_TEXT'
- | 'CLASSIFY_TEXT'
- | 'EXTRACT_STRUCTURED_DATA'
- | 'GENERATE_IMAGE';
-
export type AiStepFile = {
fileId: string;
mimeType?: string;
diff --git a/packages/pieces/community/ai/src/lib/common/props.ts b/packages/pieces/community/ai/src/lib/common/props.ts
index 8879ff6dad5..af514dd0db8 100644
--- a/packages/pieces/community/ai/src/lib/common/props.ts
+++ b/packages/pieces/community/ai/src/lib/common/props.ts
@@ -1,12 +1,13 @@
-import { ACTIVEPIECES_CHAT_TIERS, PieceAuth, Property } from '@activepieces/pieces-framework';
+import { ACTIVEPIECES_CHAT_TIERS, ACTIVEPIECES_IMAGE_TIERS, PieceAuth, Property } from '@activepieces/pieces-framework';
import { httpClient, HttpMethod } from '@activepieces/pieces-common';
import { isNil } from '@activepieces/pieces-framework';
import { AIProviderModel, AIProviderName, ProjectAIProvider } from '@activepieces/pieces-framework';
type AIModelType = 'text' | 'image';
-function managedModelLabel(modelId: string): string | undefined {
- return ACTIVEPIECES_CHAT_TIERS.find((tier) => tier.modelId === modelId)?.label;
+function managedModelLabel({ modelId, modelType }: { modelId: string; modelType: AIModelType }): string | undefined {
+ const tiers = modelType === 'image' ? ACTIVEPIECES_IMAGE_TIERS : ACTIVEPIECES_CHAT_TIERS;
+ return tiers.find((tier) => tier.modelId === modelId)?.label;
}
async function listProviders(ctx: {
@@ -120,9 +121,9 @@ export const aiProps = <T extends AIModelType>({
disabled: false,
options: allModels
.filter(model => model.type === modelType)
- .filter(model => provider !== AIProviderName.ACTIVEPIECES || managedModelLabel(model.id) !== undefined)
+ .filter(model => provider !== AIProviderName.ACTIVEPIECES || managedModelLabel({ modelId: model.id, modelType }) !== undefined)
.map(model => ({
- label: provider === AIProviderName.ACTIVEPIECES ? (managedModelLabel(model.id) ?? model.name) : model.name,
+ label: provider === AIProviderName.ACTIVEPIECES ? (managedModelLabel({ modelId: model.id, modelType }) ?? model.name) : model.name,
value: model.id,
})),
};
diff --git a/packages/pieces/framework/src/index.ts b/packages/pieces/framework/src/index.ts
index ae03d46c53f..10026dfc3f7 100644
--- a/packages/pieces/framework/src/index.ts
+++ b/packages/pieces/framework/src/index.ts
@@ -18,6 +18,7 @@ export {
startCase,
tryCatch,
AIProviderName,
+ AiStepAction,
} from '@activepieces/core-utils';
export type { SeekPage } from '@activepieces/core-utils';
@@ -107,6 +108,7 @@ export {
PARENT_RUN_ID_HEADER,
FAIL_PARENT_ON_FAILURE_HEADER,
ACTIVEPIECES_CHAT_TIERS,
+ ACTIVEPIECES_IMAGE_TIERS,
DEFAULT_CHAT_TIER_ID,
} from '@activepieces/core-piece-types';
export type {
diff --git a/packages/server/api/src/app/ai/ai-credits.ts b/packages/server/api/src/app/ai/ai-credits.ts
index 42e1599ab75..0585b5dc471 100644
--- a/packages/server/api/src/app/ai/ai-credits.ts
+++ b/packages/server/api/src/app/ai/ai-credits.ts
@@ -1,3 +1,4 @@
+import { AiChargeBasis } from '@activepieces/core-utils'
import { AiUsageCharge } from '@activepieces/shared'
import { system } from '../helper/system/system'
import { AppSystemProp } from '../helper/system/system-props'
@@ -7,14 +8,14 @@ export function chargeFor({ usage, toolCalls = 0 }: { usage: AiUsageCharge, tool
}
function creditsForModelCall(usage: AiUsageCharge): number {
- if (usage.type === 'flat-credits') {
+ if (usage.type === AiChargeBasis.FIXED_CREDITS) {
return usage.credits
}
return usage.costUsd / dollarsPerCredit()
}
export function costBasisOf(usage: AiUsageCharge): CostBasis | undefined {
- if (usage.type !== 'observed-cost') {
+ if (usage.type !== AiChargeBasis.PROVIDER_REPORTED_COST) {
return undefined
}
return { costUsd: usage.costUsd, creditUsdValue: dollarsPerCredit() }
diff --git a/packages/server/api/src/app/ai/ai-key-health-reporter.ts b/packages/server/api/src/app/ai/ai-key-health-reporter.ts
new file mode 100644
index 00000000000..99ddf373657
--- /dev/null
+++ b/packages/server/api/src/app/ai/ai-key-health-reporter.ts
@@ -0,0 +1,10 @@
+import { aiProviderKeyHealth } from '@activepieces/server-utils'
+import { FastifyBaseLogger } from 'fastify'
+import { aiProviderService } from './ai-provider-service'
+
+export function installAiKeyHealthReporter(log: FastifyBaseLogger): void {
+ aiProviderKeyHealth.setReporter(({ platformId, providerConfigId, signal }) => {
+ aiProviderService(log).recordKeyObservation({ platformId, providerId: providerConfigId, signal })
+ .catch((error) => log.warn({ error, aiProvider: { id: providerConfigId } }, '[aiKeyHealthReporter] Could not record key status'))
+ })
+}
diff --git a/packages/server/api/src/app/ai/ai-provider-service.ts b/packages/server/api/src/app/ai/ai-provider-service.ts
index 0216ea62e3e..82706fb738e 100644
--- a/packages/server/api/src/app/ai/ai-provider-service.ts
+++ b/packages/server/api/src/app/ai/ai-provider-service.ts
@@ -1,6 +1,6 @@
import { ActivepiecesError, AiProviderKeyStatus, AIProviderName, apId, classifyProviderOutcome, ErrorCode, isNil, PlatformId, ProviderOutcomeSignal, spreadIfDefined, spreadIfNotUndefined, toProviderOutcomeSignal, tryCatch, unique } from '@activepieces/core-utils'
import { modelCatalog } from '@activepieces/server-utils'
-import { ActivePiecesProviderAuthConfig, AI_PROVIDER_ENTITY_TYPES, AIProviderAuthConfig, AIProviderConfig, AIProviderModel, AiProviderProjectScope, AIProviderWithoutSensitiveData, CreateAIProviderRequest, GetProviderConfigResponse, ProjectAIProvider, UpdateAIProviderRequest } from '@activepieces/shared'
+import { ActivePiecesProviderAuthConfig, AI_PROVIDER_ENTITY_TYPES, AIProviderAuthConfig, AIProviderConfig, aiProviderCredentials, AIProviderModel, AiProviderProjectScope, AIProviderWithoutSensitiveData, CreateAIProviderRequest, GetProviderConfigResponse, ProjectAIProvider, UpdateAIProviderRequest } from '@activepieces/shared'
import { FastifyBaseLogger } from 'fastify'
import cron from 'node-cron'
import { repoFactory } from '../core/db/repo-factory'
@@ -162,7 +162,7 @@ export const aiProviderService = (log: FastifyBaseLogger) => ({
return null
}
const auth = await decryptRowAuth({ aiProvider: chatProvider, platformId })
- return { provider: chatProvider.provider, configId: chatProvider.id, auth, config: chatProvider.config, platformId, modelScope: chatProvider.modelScope, modelIds: chatProvider.modelIds }
+ return { ...aiProviderCredentials({ provider: chatProvider.provider, auth, config: chatProvider.config }), configId: chatProvider.id, platformId, modelScope: chatProvider.modelScope, modelIds: chatProvider.modelIds }
},
async keyServesScope({ platformId, provider, configId, resolvedFor, target }: { platformId: PlatformId, provider?: AIProviderName, configId?: string, resolvedFor: ProviderScope, target: ProviderScope }): Promise<boolean> {
@@ -243,7 +243,7 @@ export const aiProviderService = (log: FastifyBaseLogger) => ({
async getConfigOrThrow({ platformId, provider, scope, configId }: { platformId: PlatformId, provider: AIProviderName, scope: ProviderScope, configId?: string }): Promise<GetProviderConfigResponse> {
const aiProvider = await resolveRowForScope({ platformId, provider, scope, configId })
const auth = await decryptRowAuth({ aiProvider, platformId })
- return { provider: aiProvider.provider, configId: aiProvider.id, auth, config: aiProvider.config, platformId, modelScope: aiProvider.modelScope, modelIds: aiProvider.modelIds }
+ return { ...aiProviderCredentials({ provider: aiProvider.provider, auth, config: aiProvider.config }), configId: aiProvider.id, platformId, modelScope: aiProvider.modelScope, modelIds: aiProvider.modelIds }
},
async getOrCreateActivePiecesProviderAuthConfig(platformId: PlatformId): Promise<ActivePiecesProviderAuthConfig> {
await ensureManagedProviderRow({ platformId })
@@ -454,8 +454,7 @@ async function decryptRowAuth({ aiProvider, platformId }: { aiProvider: AIProvid
if (aiProvider.provider === AIProviderName.ACTIVEPIECES) {
const doesHaveKeys = !isNil(auth) && 'apiKey' in auth && !isNil(auth.apiKey) && auth.apiKey !== ''
if (!doesHaveKeys) {
- const { auth: activePiecesAuth } = await enrichWithKeysIfNeeded(aiProvider, platformId)
- return activePiecesAuth
+ return enrichWithKeysIfNeeded(aiProvider, platformId)
}
}
return auth
@@ -478,14 +477,14 @@ async function isActivepiecesAiProviderHidden({ platformId, log }: { platformId:
return shouldHideActivepiecesAiProvider({ platformId, log })
}
-async function enrichWithKeysIfNeeded(aiProvider: AIProviderSchema, platformId: PlatformId): Promise<GetProviderConfigResponse> {
+async function enrichWithKeysIfNeeded(aiProvider: AIProviderSchema, platformId: PlatformId): Promise<ActivePiecesProviderAuthConfig> {
const { key, data } = await openRouterApi.createKey({
name: `Platform ${platformId}`,
limit: MANAGED_OPENROUTER_KEY_MONTHLY_LIMIT_USD,
limit_reset: MANAGED_OPENROUTER_KEY_LIMIT_RESET,
})
const rawAuth: ActivePiecesProviderAuthConfig = { apiKey: key, apiKeyHash: data.hash }
- const savedAiProvider = await aiProviderRepo().save({
+ await aiProviderRepo().save({
id: aiProvider.id,
platformId,
provider: AIProviderName.ACTIVEPIECES,
@@ -493,7 +492,7 @@ async function enrichWithKeysIfNeeded(aiProvider: AIProviderSchema, platformId:
config: {},
auth: await encryptUtils.encryptObject(rawAuth),
})
- return { provider: savedAiProvider.provider, configId: savedAiProvider.id, auth: rawAuth, config: savedAiProvider.config, platformId, modelScope: savedAiProvider.modelScope, modelIds: savedAiProvider.modelIds }
+ return rawAuth
}
diff --git a/packages/server/api/src/app/ai/ai-rpc-handlers.ts b/packages/server/api/src/app/ai/ai-rpc-handlers.ts
index 139a0cc9493..5976758774e 100644
--- a/packages/server/api/src/app/ai/ai-rpc-handlers.ts
+++ b/packages/server/api/src/app/ai/ai-rpc-handlers.ts
@@ -21,10 +21,8 @@ export const aiRpcHandlers = (log: FastifyBaseLogger) => ({
...spreadIfDefined('configId', input.providerConfigId),
})
return {
- provider: config.provider,
+ ...config,
providerConfigId: config.configId,
- auth: config.auth as Record<string, unknown>,
- config: config.config as Record<string, unknown>,
}
},
diff --git a/packages/server/api/src/app/ai/ai-usage-service.ts b/packages/server/api/src/app/ai/ai-usage-service.ts
index e6f0c51bfbd..f6876ece603 100644
--- a/packages/server/api/src/app/ai/ai-usage-service.ts
+++ b/packages/server/api/src/app/ai/ai-usage-service.ts
@@ -1,4 +1,4 @@
-import { ActivepiecesAiBillingScope, isNil, tryCatch } from '@activepieces/core-utils'
+import { ActivepiecesAiConsumerSource, AiCallTokens, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'
import { isAppSumoCreditedPlan, ReportAiUsageRequest } from '@activepieces/shared'
import { FastifyBaseLogger } from 'fastify'
import { platformPlanService } from '../ee/platform/platform-plan/platform-plan.service'
@@ -38,7 +38,7 @@ export const aiUsageService = (log: FastifyBaseLogger) => ({
})
async function addConversationCredits({ input, credits, log }: { input: ReportAiUsageRequest, credits: number, log: FastifyBaseLogger }): Promise<void> {
- if (input.billing.scope !== ActivepiecesAiBillingScope.CONVERSATION) {
+ if (input.billing.source !== ActivepiecesAiConsumerSource.CHAT) {
return
}
const { conversationId } = input.billing
@@ -49,7 +49,7 @@ async function addConversationCredits({ input, credits, log }: { input: ReportAi
}
function chatEventOf({ input, toolCalls, costBasis }: PropertiesParams): { credits: ChatCreditConsumptionProperties, appSumo: ChatAppSumoConsumptionProperties } | undefined {
- if (input.billing.scope !== ActivepiecesAiBillingScope.CONVERSATION || isNil(input.chat)) {
+ if (input.billing.source !== ActivepiecesAiConsumerSource.CHAT || isNil(input.chat)) {
return undefined
}
const { conversationId } = input.billing
@@ -68,6 +68,7 @@ function chatEventOf({ input, toolCalls, costBasis }: PropertiesParams): { credi
model: input.modelId,
tier,
...costBasis,
+ ...tokensOf(input),
},
appSumo: { platformId: input.billing.platformId, projectId, conversationId, turnIndex, tier },
}
@@ -79,7 +80,7 @@ function billingProperties({ input, toolCalls, costBasis }: PropertiesParams): A
projectId: projectIdOf(input),
flowId: input.flowRun?.flowId ?? OUTSIDE_A_FLOW,
flowRunId: input.flowRun?.flowRunId ?? input.requestId ?? OUTSIDE_A_FLOW,
- environment: input.flowRun?.environment ?? UNKNOWN_ENVIRONMENT,
+ environment: UNKNOWN_ENVIRONMENT,
messages: MESSAGES_PER_MODEL_CALL,
toolCalls,
breakdown: [{
@@ -89,13 +90,18 @@ function billingProperties({ input, toolCalls, costBasis }: PropertiesParams): A
toolCalls,
}],
...costBasis,
+ ...tokensOf(input),
}
}
-function projectIdOf(input: ReportAiUsageRequest): string {
- if (input.billing.scope === ActivepiecesAiBillingScope.PLATFORM) {
- return PROJECTLESS_CHAT
+function tokensOf(input: ReportAiUsageRequest): AiCallTokens {
+ return {
+ ...spreadIfDefined('inputTokens', input.inputTokens),
+ ...spreadIfDefined('outputTokens', input.outputTokens),
}
+}
+
+function projectIdOf(input: ReportAiUsageRequest): string {
return input.billing.projectId ?? PROJECTLESS_CHAT
}
diff --git a/packages/server/api/src/app/app.ts b/packages/server/api/src/app/app.ts
index 0609e82c7b6..df82eb46e4c 100644
--- a/packages/server/api/src/app/app.ts
+++ b/packages/server/api/src/app/app.ts
@@ -1,6 +1,6 @@
import { isNil, spreadIfDefined } from '@activepieces/core-utils'
import { PieceMetadata } from '@activepieces/pieces-framework'
-import { apVersionUtil, onCallService, UNKNOWN_VERSION, wideEvent } from '@activepieces/server-utils'
+import { aiCostReporter, apVersionUtil, onCallService, UNKNOWN_VERSION, wideEvent } from '@activepieces/server-utils'
import { AddAllowedEmbedOriginsRequestBody, ApEdition, ApEnvironment, AppConnectionWithoutSensitiveData, ApplicationEventName, ConnectionDeletedEvent, ConnectionUpsertedEvent, Flow, FlowActivatedEvent, FlowCreatedEvent, FlowDeactivatedEvent, FlowDeletedEvent, FlowPiecesRevertedEvent, FlowPiecesUpgradedEvent, FlowPublishedEvent, FlowRun, FlowRunFinishedEvent, FlowRunRetriedEvent, FlowRunStartedEvent, FlowUpdatedEvent, Folder, FolderCreatedEvent, FolderDeletedEvent, FolderUpdatedEvent, GitRepoWithoutSensitiveData, ProjectMember, ProjectRelease, ProjectReleaseEvent, ProjectRoleEvent, ProjectWithLimits, SigningKeyEvent, SignUpEvent, Template, UserEmailVerifiedEvent, UserInvitation, UserPasswordResetEvent, UserSignedInEvent, UserWithMetaInformation } from '@activepieces/shared'
import replyFrom from '@fastify/reply-from'
import swagger from '@fastify/swagger'
@@ -10,10 +10,12 @@ import { jsonSchemaTransform, jsonSchemaTransformObject } from 'fastify-type-pro
import Mustache from 'mustache'
import { globalRegistry } from 'zod/v4/core'
import { agentsModule } from './agents/agents-module'
+import { installAiKeyHealthReporter } from './ai/ai-key-health-reporter'
import { aiProviderService } from './ai/ai-provider-service'
import { aiProviderModule } from './ai/ai-provider.module'
import { aiToolConfigModule } from './ai/ai-tool-config.module'
import { aiUsageHooks } from './ai/ai-usage-hooks'
+import { aiUsageService } from './ai/ai-usage-service'
import { platformAnalyticsModule } from './analytics/platform-analytics.module'
import { setPlatformOAuthService } from './app-connection/app-connection-service/oauth2'
import { appConnectionModule } from './app-connection/app-connection.module'
@@ -457,6 +459,12 @@ The application started on ${await domainHelper.getPublicApiUrl({ path: '' })},
const pieces = process.env.AP_DEV_PIECES
assertReleaseReadable(app.log)
+ aiCostReporter.install({
+ log: app.log,
+ report: (request) => aiUsageService(app.log).report(request),
+ pageWebhookUrl: () => system.get(AppSystemProp.PAGE_ONCALL_WEBHOOK),
+ })
+ installAiKeyHealthReporter(app.log)
systemSnapshot.start({ log: app.log })
await migrateQueuesAndRunConsumers(app)
app.log.info('Queues migrated and consumers run')
diff --git a/packages/server/api/src/app/ee/agent/agent-conversation-credits.ts b/packages/server/api/src/app/ee/agent/agent-conversation-credits.ts
index f15b7823379..231b9a3bed5 100644
--- a/packages/server/api/src/app/ee/agent/agent-conversation-credits.ts
+++ b/packages/server/api/src/app/ee/agent/agent-conversation-credits.ts
@@ -5,11 +5,15 @@ import { agentHelpers } from './agent-helpers'
export const agentConversationCreditsHooks = (_log: FastifyBaseLogger): AiUsageHooks => ({
async addConversationCredits({ conversationId, credits }: AddConversationCreditsParams): Promise<void> {
+ const wholeCredits = Math.round(credits)
+ if (wholeCredits === 0) {
+ return
+ }
await agentHelpers.conversationRepo()
.createQueryBuilder()
.update(AgentConversationEntity)
.set({ aiCredits: () => 'COALESCE("aiCredits", 0) + :credits' })
- .setParameter('credits', credits)
+ .setParameter('credits', wholeCredits)
.where('id = :conversationId', { conversationId })
.execute()
},
diff --git a/packages/server/api/src/app/ee/agent/agent-helpers.ts b/packages/server/api/src/app/ee/agent/agent-helpers.ts
index b8a5cbd016b..657a5ae8903 100644
--- a/packages/server/api/src/app/ee/agent/agent-helpers.ts
+++ b/packages/server/api/src/app/ee/agent/agent-helpers.ts
@@ -1,5 +1,5 @@
import { ExecuteAgentRunJobData } from '@activepieces/core-execution'
-import { ActivepiecesError, AIProviderName, apId, ErrorCode, isNil, ProviderOutcomeReporter, spreadIfDefined, tryCatch, unique } from '@activepieces/core-utils'
+import { ActivepiecesAiBilling, ActivepiecesError, AIProviderName, apId, ErrorCode, isNil, spreadIfDefined, tryCatch, unique } from '@activepieces/core-utils'
import { aiUtils } from '@activepieces/server-utils'
import { AgentConfig, AgentConversation, AgentConversationStatus, AI_PROVIDER_ENTITY_TYPES, GetAgentMemoryResponse, GetProviderConfigResponse, Project, ProjectType, UserMemory } from '@activepieces/shared'
import { SharedV3ProviderOptions } from '@ai-sdk/provider'
@@ -145,25 +145,15 @@ async function assertRunProviderConfigured({ platformId, provider, providerConfi
}
}
-function reportKeyOutcome({ platformId, providerId, log }: { platformId: string, providerId: string, log: FastifyBaseLogger }): ProviderOutcomeReporter {
- return async (signal) => {
- const { error } = await tryCatch(() => aiProviderService(log).recordKeyObservation({ platformId, providerId, signal }))
- if (!isNil(error)) {
- log.warn({ error, aiProvider: { id: providerId } }, '[agentHelpers#reportKeyOutcome] Could not record key status')
- }
- }
-}
-
async function resolveTierModel({ platformId, tierId, provider, providerConfigId, scope, log }: { platformId: string, tierId: string, provider?: AIProviderName, providerConfigId?: string, scope: ProviderScope, log: FastifyBaseLogger }): Promise<{ model: LanguageModel, modelId: string, provider: AIProviderName }> {
const providerConfig = await resolveRunProvider({ platformId, scope, log, ...spreadIfDefined('provider', provider), ...spreadIfDefined('providerConfigId', providerConfigId) })
const modelId = agentModelResolution.resolveModelIdForProvider({ provider: providerConfig.provider, selectedModel: tierId, config: providerConfig.config, modelScope: providerConfig.modelScope, modelIds: providerConfig.modelIds })
return {
model: aiUtils.createModel({
- provider: providerConfig.provider,
- auth: providerConfig.auth,
- config: providerConfig.config,
+ credentials: providerConfig,
modelId,
- onOutcome: reportKeyOutcome({ platformId, providerId: providerConfig.configId, log }),
+ platformId,
+ providerConfigId: providerConfig.configId,
}),
modelId,
provider: providerConfig.provider,
@@ -175,13 +165,13 @@ async function resolveFastModel({ platformId, provider, providerConfigId, scope,
}
-async function resolveEmbeddingModel({ platformId, provider, providerConfigId, scope, log }: { platformId: string, provider?: AIProviderName, providerConfigId?: string, scope: ProviderScope, log: FastifyBaseLogger }): Promise<{ model: EmbeddingModel, providerOptions: SharedV3ProviderOptions }> {
+async function resolveEmbeddingModel({ platformId, provider, providerConfigId, scope, billing, log }: { platformId: string, provider?: AIProviderName, providerConfigId?: string, scope: ProviderScope, billing?: ActivepiecesAiBilling, log: FastifyBaseLogger }): Promise<{ model: EmbeddingModel, providerOptions: SharedV3ProviderOptions }> {
const providerConfig = await resolveRunProvider({ platformId, scope, log, ...spreadIfDefined('provider', provider), ...spreadIfDefined('providerConfigId', providerConfigId) })
return aiUtils.createEmbeddingModel({
- provider: providerConfig.provider,
- auth: providerConfig.auth,
- config: providerConfig.config,
- onOutcome: reportKeyOutcome({ platformId, providerId: providerConfig.configId, log }),
+ credentials: providerConfig,
+ platformId,
+ providerConfigId: providerConfig.configId,
+ ...spreadIfDefined('billing', billing),
})
}
diff --git a/packages/server/api/src/app/ee/agent/agent-model-resolution.ts b/packages/server/api/src/app/ee/agent/agent-model-resolution.ts
index 7cb3577190c..65f17175d08 100644
--- a/packages/server/api/src/app/ee/agent/agent-model-resolution.ts
+++ b/packages/server/api/src/app/ee/agent/agent-model-resolution.ts
@@ -1,5 +1,5 @@
import { ActivepiecesError, AIProviderName, ErrorCode, isNil, tryCatchSync } from '@activepieces/core-utils'
-import { ACTIVEPIECES_CHAT_TIERS, AI_PROVIDER_ENTITY_TYPES, AIProviderConfig, AiProviderModelScope, AIProviderModelType, aiProviderUtils, DEFAULT_CHAT_TIER_ID } from '@activepieces/shared'
+import { ACTIVEPIECES_CHAT_TIERS, AI_PROVIDER_ENTITY_TYPES, AiProviderCredentials, AiProviderModelScope, AIProviderModelType, aiProviderUtils, DEFAULT_CHAT_TIER_ID } from '@activepieces/shared'
function findTier({ tierId }: { tierId: string | null }) {
return ACTIVEPIECES_CHAT_TIERS.find((t) => t.id === tierId)
@@ -11,8 +11,8 @@ function resolveTier({ tierId }: { tierId: string | null }) {
// An admin-listed catalog is the whole truth about what a key exposes, so an empty one means the key
// serves no text model - not that we may fall back to a curated id it was never configured for.
-function manualTextModelCatalog({ config }: { config?: AIProviderConfig }): string[] | undefined {
- if (isNil(config) || !('models' in config)) {
+function manualTextModelCatalog({ config }: { config?: AiProviderCredentials['config'] }): string[] | undefined {
+ if (isNil(config) || !('models' in config) || isNil(config.models)) {
return undefined
}
return config.models.filter((model) => model.modelType === AIProviderModelType.TEXT).map((model) => model.modelId)
@@ -51,7 +51,7 @@ function resolveNamedModelId({ provider, modelName, modelScope, modelIds }: { pr
return requested
}
-function resolveModelIdForProvider({ provider, selectedModel, config, modelScope, modelIds }: { provider: AIProviderName, selectedModel: string | null, config?: AIProviderConfig, modelScope?: AiProviderModelScope, modelIds?: string[] }): string {
+function resolveModelIdForProvider({ provider, selectedModel, config, modelScope, modelIds }: { provider: AIProviderName, selectedModel: string | null, config?: AiProviderCredentials['config'], modelScope?: AiProviderModelScope, modelIds?: string[] }): string {
const catalog = manualTextModelCatalog({ config })
if (!isNil(catalog)) {
return pickAllowedModel({ provider, selectedModel, candidates: catalog, modelScope, modelIds })
@@ -87,7 +87,7 @@ function resolveModelIdForAnalytics({ provider, selectedModel }: { provider: AIP
return aiProviderUtils.isCuratedChatModelId({ modelId: selectedModel }) ? selectedModel : null
}
-function resolveFastModelId({ provider, config, modelScope, modelIds }: { provider: AIProviderName, config?: AIProviderConfig, modelScope?: AiProviderModelScope, modelIds?: string[] }): string {
+function resolveFastModelId({ provider, config, modelScope, modelIds }: { provider: AIProviderName, config?: AiProviderCredentials['config'], modelScope?: AiProviderModelScope, modelIds?: string[] }): string {
return resolveModelIdForProvider({ provider, selectedModel: FAST_TIER_ID, config, modelScope, modelIds })
}
diff --git a/packages/server/api/src/app/ee/agent/personalization/chat-personalization-service.ts b/packages/server/api/src/app/ee/agent/personalization/chat-personalization-service.ts
index 2b956a5b308..8a9aeef2297 100644
--- a/packages/server/api/src/app/ee/agent/personalization/chat-personalization-service.ts
+++ b/packages/server/api/src/app/ee/agent/personalization/chat-personalization-service.ts
@@ -254,9 +254,7 @@ export const chatPersonalizationService = (log: FastifyBaseLogger) => ({
const webSearch = enabledTools.data?.webSearch ?? null
return {
claimed: true,
- provider: provider.provider,
- auth: provider.auth,
- providerConfig: provider.config ?? {},
+ credentials: provider,
modelId: agentHelpers.resolveModelIdForProvider({ provider: providerName, selectedModel: null, config: provider.config, modelScope: provider.modelScope, modelIds: provider.modelIds }),
fastModelId: agentHelpers.resolveFastModelId({ provider: providerName, config: provider.config, modelScope: provider.modelScope, modelIds: provider.modelIds }),
user: { firstName: user.firstName, lastName: user.lastName, email: user.email },
diff --git a/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts b/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts
index 8ea04b9a3ea..73bab9ce314 100644
--- a/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts
+++ b/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts
@@ -200,9 +200,7 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({
log.debug({ estimatedTokens, willCompact, messageCount: llmHistory.length, systemPromptLength: systemPromptText.length }, '[agentRpc#getAgentConfig] Compaction decision')
if (willCompact) {
const model = aiUtils.createModel({
- provider: providerConfig.provider,
- auth: providerConfig.auth as Record<string, unknown>,
- config: providerConfig.config as Record<string, unknown>,
+ credentials: providerConfig,
modelId: resolvedModelId,
})
compactionState = await agentCompaction.compactMessages({
@@ -239,10 +237,8 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({
log.debug({ systemPrompt: systemPromptText, guideNames: Object.keys(guides) }, '[agentRpc#getAgentConfig] System prompt assembled')
return {
- provider: providerConfig.provider,
+ credentials: providerConfig,
providerConfigId: providerConfig.configId,
- auth: providerConfig.auth as Record<string, unknown>,
- providerConfig: providerConfig.config as Record<string, unknown>,
modelId: resolvedModelId,
fastModelId: agentHelpers.resolveFastModelId({ provider: providerConfig.provider, config: providerConfig.config, modelScope: providerConfig.modelScope, modelIds: providerConfig.modelIds }),
systemPrompt: systemPromptText,
diff --git a/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts b/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts
index 834bb4a6f4c..6acd40c44a7 100644
--- a/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts
+++ b/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts
@@ -1,4 +1,4 @@
-import { ActivepiecesError, ErrorCode, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'
+import { ActivepiecesAiConsumerSource, ActivepiecesError, ErrorCode, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'
import { aiUtils } from '@activepieces/server-utils'
import { AGENT_SELF_EDIT_TOOLS, AGENT_SURFACE_TOOLS, AgentActionOutcome, AgentRunSource, agentToolClassification, ExecuteAgentToolRequest, ExecuteAgentToolResponse, ExecuteFlowToolRequest, ExecuteFlowToolResponse, ExecuteKnowledgeBaseToolRequest, ExecuteKnowledgeBaseToolResponse, ExecutePieceToolRequest, ExecutePieceToolResponse, FlowActionType, flowStructureUtil } from '@activepieces/shared'
import { embed } from 'ai'
@@ -75,7 +75,14 @@ export const toolExecutionRpc = (log: FastifyBaseLogger) => ({
log.warn({ conversation: { id: input.conversationId }, project: { id: projectId }, knowledgeBaseFile: { id: input.knowledgeBaseFileId } }, '[agentRpc#executeKnowledgeBaseTool] The file has no searchable text, so the search was not run')
return { result: `"${file.displayName}" is attached but has never been indexed, so its text cannot be searched and you have not read any of it. Tell the user exactly that. Do not say the file does not contain what they asked for, and do not suggest re-uploading it: that will not index it either.` }
}
- const { model, providerOptions } = await agentHelpers.resolveEmbeddingModel({ platformId, scope: { type: 'project', projectId }, log, ...spreadIfDefined('provider', input.provider), ...spreadIfDefined('providerConfigId', input.providerConfigId) })
+ const { model, providerOptions } = await agentHelpers.resolveEmbeddingModel({
+ platformId,
+ scope: { type: 'project', projectId },
+ billing: { source: ActivepiecesAiConsumerSource.CHAT, platformId, projectId, conversationId: input.conversationId },
+ log,
+ ...spreadIfDefined('provider', input.provider),
+ ...spreadIfDefined('providerConfigId', input.providerConfigId),
+ })
const { embedding } = await embed({ model, value: input.query, providerOptions })
const results = await knowledgeBaseService(log).search({
projectId,
diff --git a/packages/server/api/src/app/flows/flow-version/migrations/migrate-v26-ai-piece-cost-billing.ts b/packages/server/api/src/app/flows/flow-version/migrations/migrate-v26-ai-piece-cost-billing.ts
index 2a1a86a6723..a7d46a36ba6 100644
--- a/packages/server/api/src/app/flows/flow-version/migrations/migrate-v26-ai-piece-cost-billing.ts
+++ b/packages/server/api/src/app/flows/flow-version/migrations/migrate-v26-ai-piece-cost-billing.ts
@@ -1,3 +1,4 @@
+import { extractMustacheTokens } from '@activepieces/core-utils'
import { AgentPieceProps, AI_PIECE_COST_BILLING_VERSION, AI_PIECE_NAME, FlowActionType, flowStructureUtil, FlowVersion } from '@activepieces/shared'
import semver from 'semver'
import { Migration } from '.'
@@ -23,10 +24,23 @@ export const migrateV26AiPieceCostBilling: Migration = {
}
function withAgentMaxSteps({ actionName, input }: { actionName?: string, input: Record<string, unknown> }): Record<string, unknown> {
- if (actionName !== AGENT_ACTION_NAME || typeof input[AgentPieceProps.MAX_STEPS] === 'number') {
+ const stored = input[AgentPieceProps.MAX_STEPS]
+ if (actionName !== AGENT_ACTION_NAME || typeof stored === 'number' || holdsExpression(stored)) {
return input
}
- return { ...input, [AgentPieceProps.MAX_STEPS]: DEFAULT_MAX_STEPS }
+ return { ...input, [AgentPieceProps.MAX_STEPS]: storedMaxSteps(stored) ?? DEFAULT_MAX_STEPS }
+}
+
+function holdsExpression(value: unknown): boolean {
+ return typeof value === 'string' && extractMustacheTokens(value).length > 0
+}
+
+function storedMaxSteps(value: unknown): number | undefined {
+ if (typeof value !== 'string') {
+ return undefined
+ }
+ const parsed = Number(value.trim())
+ return Number.isInteger(parsed) && parsed > 0 ? parsed : undefined
}
function isBelowCostBillingVersion(pieceVersion: string): boolean {
diff --git a/packages/server/api/src/app/platform/billing-provider.ts b/packages/server/api/src/app/platform/billing-provider.ts
index f809bbdb742..c384ccdeb61 100644
--- a/packages/server/api/src/app/platform/billing-provider.ts
+++ b/packages/server/api/src/app/platform/billing-provider.ts
@@ -192,6 +192,8 @@ export type AiCreditConsumptionProperties = FlowRunCreditConsumptionProperties &
breakdown: CreditEventBreakdownEntry[]
costUsd?: number
creditUsdValue?: number
+ inputTokens?: number
+ outputTokens?: number
}
export type McpCallCreditConsumptionProperties = {
@@ -212,6 +214,8 @@ export type ChatCreditConsumptionProperties = CreditConsumptionPropertiesBase &
tier: string
costUsd?: number
creditUsdValue?: number
+ inputTokens?: number
+ outputTokens?: number
}
export type ChatAppSumoConsumptionProperties = CreditConsumptionPropertiesBase & {
diff --git a/packages/server/api/test/integration/ce/agent/agent-run-worker-handoff.test.ts b/packages/server/api/test/integration/ce/agent/agent-run-worker-handoff.test.ts
index dc24cbfd988..8521c22db82 100644
--- a/packages/server/api/test/integration/ce/agent/agent-run-worker-handoff.test.ts
+++ b/packages/server/api/test/integration/ce/agent/agent-run-worker-handoff.test.ts
@@ -60,7 +60,7 @@ describe('community edition: the worker picking up an agent step', () => {
source: AgentRunSource.FLOW_STEP,
})
- expect(config.provider).toBe(AIProviderName.OPENAI)
+ expect(config.credentials.provider).toBe(AIProviderName.OPENAI)
expect(config.modelId).toBe('gpt-4o')
expect(config.systemPrompt.length).toBeGreaterThan(0)
})
diff --git a/packages/server/api/test/unit/app/ai/ai-credits.test.ts b/packages/server/api/test/unit/app/ai/ai-credits.test.ts
index 7042161bafe..5e660794e63 100644
--- a/packages/server/api/test/unit/app/ai/ai-credits.test.ts
+++ b/packages/server/api/test/unit/app/ai/ai-credits.test.ts
@@ -1,3 +1,4 @@
+import { AiChargeBasis } from '@activepieces/core-utils'
import { AiUsageCharge } from '@activepieces/shared'
import { describe, expect, it, vi } from 'vitest'
@@ -11,69 +12,69 @@ const { chargeFor, costBasisOf } = await import('../../../../src/app/ai/ai-credi
describe('chargeFor', () => {
it('converts an observed dollar cost at the configured rate', () => {
- expect(chargeFor({ usage: { type: 'observed-cost', costUsd: 0.0005 } })).toBe(1)
- expect(chargeFor({ usage: { type: 'observed-cost', costUsd: 0.005 } })).toBe(10)
+ expect(chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.0005 } })).toBe(1)
+ expect(chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.005 } })).toBe(10)
})
it('keeps a small call fractional rather than rounding it up to a whole credit', () => {
- expect(chargeFor({ usage: { type: 'observed-cost', costUsd: 0.00037 } })).toBeCloseTo(0.74, 6)
+ expect(chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.00037 } })).toBeCloseTo(0.74, 6)
})
it('charges nothing for a genuinely free model, which reports a real cost of zero', () => {
- expect(chargeFor({ usage: { type: 'observed-cost', costUsd: 0 } })).toBe(0)
+ expect(chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0 } })).toBe(0)
})
it('scales with the size of the call, which is the whole point of billing on cost', () => {
- const small = chargeFor({ usage: { type: 'observed-cost', costUsd: 0.0004 } })
- const large = chargeFor({ usage: { type: 'observed-cost', costUsd: 0.004 } })
+ const small = chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.0004 } })
+ const large = chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.004 } })
expect(large).toBeCloseTo(small * 10, 6)
})
it('passes flat credits straight through, which is how a BYOK call bills one per model call', () => {
- expect(chargeFor({ usage: { type: 'flat-credits', credits: 1 } })).toBe(1)
+ expect(chargeFor({ usage: { type: AiChargeBasis.FIXED_CREDITS, credits: 1 } })).toBe(1)
})
it('passes a flat charge of several credits through unchanged', () => {
- expect(chargeFor({ usage: { type: 'flat-credits', credits: 4 } })).toBe(4)
+ expect(chargeFor({ usage: { type: AiChargeBasis.FIXED_CREDITS, credits: 4 } })).toBe(4)
})
it('adds one credit per tool call on top of what the model call cost', () => {
- expect(chargeFor({ usage: { type: 'observed-cost', costUsd: 0.0005 }, toolCalls: 3 })).toBe(4)
+ expect(chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.0005 }, toolCalls: 3 })).toBe(4)
})
it('charges the tool calls a BYOK turn made alongside its flat credit', () => {
- expect(chargeFor({ usage: { type: 'flat-credits', credits: 1 }, toolCalls: 2 })).toBe(3)
+ expect(chargeFor({ usage: { type: AiChargeBasis.FIXED_CREDITS, credits: 1 }, toolCalls: 2 })).toBe(3)
})
it('charges for tool calls even when the model call itself cost nothing', () => {
- expect(chargeFor({ usage: { type: 'observed-cost', costUsd: 0 }, toolCalls: 2 })).toBe(2)
+ expect(chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0 }, toolCalls: 2 })).toBe(2)
})
it('refuses a rate of zero rather than billing an unbounded number of credits', () => {
rate.value = 0
- expect(() => chargeFor({ usage: { type: 'observed-cost', costUsd: 0.0042 } })).toThrow(/greater than zero/)
+ expect(() => chargeFor({ usage: { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.0042 } })).toThrow(/greater than zero/)
rate.value = 0.0005
})
})
describe('costBasisOf', () => {
it('records the dollars the call cost and the rate they were converted at', () => {
- expect(costBasisOf({ type: 'observed-cost', costUsd: 0.005 })).toEqual({ costUsd: 0.005, creditUsdValue: 0.0005 })
+ expect(costBasisOf({ type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.005 })).toEqual({ costUsd: 0.005, creditUsdValue: 0.0005 })
})
it('records the rate in force at the time, so a later rate change cannot rewrite what was charged', () => {
rate.value = 0.001
- expect(costBasisOf({ type: 'observed-cost', costUsd: 0.005 })).toEqual({ costUsd: 0.005, creditUsdValue: 0.001 })
+ expect(costBasisOf({ type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.005 })).toEqual({ costUsd: 0.005, creditUsdValue: 0.001 })
rate.value = 0.0005
})
it('reports a basis a reader can divide back into the credits that were billed', () => {
- const usage: AiUsageCharge = { type: 'observed-cost', costUsd: 0.0037 }
+ const usage: AiUsageCharge = { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: 0.0037 }
const basis = costBasisOf(usage)
expect(basis!.costUsd / basis!.creditUsdValue).toBeCloseTo(chargeFor({ usage }), 6)
})
it('records nothing for a flat charge, which never went through a dollar rate', () => {
- expect(costBasisOf({ type: 'flat-credits', credits: 1 })).toBeUndefined()
+ expect(costBasisOf({ type: AiChargeBasis.FIXED_CREDITS, credits: 1 })).toBeUndefined()
})
})
diff --git a/packages/server/api/test/unit/app/ee/agent/agent-fast-model-provider.test.ts b/packages/server/api/test/unit/app/ee/agent/agent-fast-model-provider.test.ts
index e72faef3da8..6183238c989 100644
--- a/packages/server/api/test/unit/app/ee/agent/agent-fast-model-provider.test.ts
+++ b/packages/server/api/test/unit/app/ee/agent/agent-fast-model-provider.test.ts
@@ -40,7 +40,7 @@ describe('resolveFastModel', () => {
expect(mockGetChatProvider).not.toHaveBeenCalled()
expect(mockGetConfigOrThrow).toHaveBeenCalledWith({ platformId, provider: AIProviderName.OPENROUTER, scope })
- expect(model).toMatchObject({ provider: AIProviderName.OPENROUTER, modelId: 'anthropic/claude-haiku-4.5' })
+ expect(model).toMatchObject({ credentials: { provider: AIProviderName.OPENROUTER }, modelId: 'anthropic/claude-haiku-4.5' })
})
it('asks for the pinned key when the run names one', async () => {
@@ -60,6 +60,6 @@ describe('resolveFastModel', () => {
const model = await agentHelpers.resolveFastModel({ platformId, scope, log })
- expect(model).toMatchObject({ provider: AIProviderName.ANTHROPIC, modelId: 'claude-haiku-4-5' })
+ expect(model).toMatchObject({ credentials: { provider: AIProviderName.ANTHROPIC }, modelId: 'claude-haiku-4-5' })
})
})
diff --git a/packages/server/api/test/unit/app/flows/flow-version/migrate-v26-ai-piece-cost-billing.test.ts b/packages/server/api/test/unit/app/flows/flow-version/migrate-v26-ai-piece-cost-billing.test.ts
index 310b10ba830..7a25691ffbe 100644
--- a/packages/server/api/test/unit/app/flows/flow-version/migrate-v26-ai-piece-cost-billing.test.ts
+++ b/packages/server/api/test/unit/app/flows/flow-version/migrate-v26-ai-piece-cost-billing.test.ts
@@ -114,6 +114,26 @@ describe('migrateV26AiPieceCostBilling', () => {
expect(input).toEqual({ maxSteps: 5 })
})
+ it('keeps a max steps saved as text, since the engine reads it as a number too', async () => {
+ const input = await migratedInputOf(withAiStep({ pieceVersion: '0.4.5', actionName: 'run_agent', input: { maxSteps: '3' } }))
+ expect(input).toEqual({ maxSteps: 3 })
+ })
+
+ it('ignores the padding around a max steps saved as text', async () => {
+ const input = await migratedInputOf(withAiStep({ pieceVersion: '0.4.5', actionName: 'run_agent', input: { maxSteps: '10\n' } }))
+ expect(input).toEqual({ maxSteps: 10 })
+ })
+
+ it('keeps a max steps the flow computes at run time, since the engine resolves the expression to a number', async () => {
+ const input = await migratedInputOf(withAiStep({ pieceVersion: '0.4.5', actionName: 'run_agent', input: { maxSteps: '{{trigger.maxSteps}}' } }))
+ expect(input).toEqual({ maxSteps: '{{trigger.maxSteps}}' })
+ })
+
+ it.each(['0', '-5', '3.7', 'abc', ''])('falls back to the default when the saved max steps %s is not a whole positive number', async (maxSteps) => {
+ const input = await migratedInputOf(withAiStep({ pieceVersion: '0.4.5', actionName: 'run_agent', input: { maxSteps } }))
+ expect(input).toEqual({ maxSteps: 20 })
+ })
+
it('leaves the input of a non-agent AI action untouched', async () => {
const input = await migratedInputOf(withAiStep({ pieceVersion: '0.4.5', actionName: 'askAi', input: { prompt: 'hi' } }))
expect(input).toEqual({ prompt: 'hi' })
diff --git a/packages/server/utils/src/activepieces-ai-cost.ts b/packages/server/utils/src/activepieces-ai-cost.ts
index 00e6b00de1e..cf8c91594ef 100644
--- a/packages/server/utils/src/activepieces-ai-cost.ts
+++ b/packages/server/utils/src/activepieces-ai-cost.ts
@@ -1,123 +1,164 @@
-import { ActivepiecesAiBilling, ActivepiecesAiCall, ActivepiecesAiCostReporter, AIProviderName, isNil, spreadIfDefined } from '@activepieces/core-utils'
-import { isJSONObject, LanguageModelV4GenerateResult, LanguageModelV4StreamPart, SharedV4ProviderMetadata } from '@ai-sdk/provider'
-import { LanguageModel, wrapLanguageModel } from 'ai'
+import { ActivepiecesAiBilling, ActivepiecesAiCall, ActivepiecesAiConsumerSource, ActivepiecesAiCostReporter, AiCallTokens, AiCharge, AiChargeBasis, AIProviderName, isNil, spreadIfDefined } from '@activepieces/core-utils'
+import { EmbeddingModelV4Result, isJSONObject, LanguageModelV4GenerateResult, LanguageModelV4StreamPart, LanguageModelV4StreamResult, LanguageModelV4Usage, SharedV4ProviderMetadata } from '@ai-sdk/provider'
+import { EmbeddingModel, LanguageModel, wrapEmbeddingModel, wrapLanguageModel } from 'ai'
import { z } from 'zod'
+import { lookupGeneration } from './openrouter-generation'
-export function billedLanguageModel({ model, provider, modelId, billing, ownKeyCredit = 'per-call' }: BilledLanguageModelParams): LanguageModel {
- if (isNil(billing) || typeof model === 'string') {
+export function billedLanguageModel({ model, provider, modelId, billing, charge }: BilledLanguageModelParams): LanguageModel {
+ const context = billingContextOf({ provider, modelId, billing, charge })
+ if (isNil(context) || typeof model === 'string') {
return model
}
- if (provider !== AIProviderName.ACTIVEPIECES) {
- return ownKeyCredit === 'per-call' ? billedAtAFlatRate({ model, provider, modelId, billing }) : model
- }
return wrapLanguageModel({
model,
middleware: {
- wrapGenerate: async ({ doGenerate }) => {
- const result = await doGenerate()
- reportUsage({ billing, provider, modelId, call: toGeneratedCall(result) })
- return result
- },
- wrapStream: async ({ doStream }) => {
- const { stream, ...rest } = await doStream()
- return { ...rest, stream: stream.pipeThrough(streamCostObserver({ billing, provider, modelId })) }
- },
+ wrapGenerate: ({ doGenerate }) => billGenerate({ doGenerate, context }),
+ wrapStream: ({ doStream }) => billStream({ doStream, context }),
},
})
}
-function billedAtAFlatRate({ model, provider, modelId, billing }: BilledAtAFlatRateParams): LanguageModel {
- return wrapLanguageModel({
+export function billedEmbeddingModel({ model, provider, modelId, billing }: BilledEmbeddingModelParams): EmbeddingModel {
+ if (isNil(billing) || provider !== AIProviderName.ACTIVEPIECES || typeof model === 'string' || model.specificationVersion !== 'v4') {
+ return model
+ }
+ const context: BillingContext = { basis: AiChargeBasis.PROVIDER_REPORTED_COST, billing, provider, modelId }
+ return wrapEmbeddingModel({
model,
middleware: {
- wrapGenerate: async ({ doGenerate }) => {
- const result = await doGenerate()
- reportFlatCredits({ billing, provider, modelId, generationId: result.response?.id })
- return result
- },
- wrapStream: async ({ doStream }) => {
- const { stream, ...rest } = await doStream()
- return { ...rest, stream: stream.pipeThrough(flatCreditObserver({ billing, provider, modelId })) }
- },
+ wrapEmbed: ({ doEmbed }) => billEmbed({ doEmbed, context }),
},
})
}
-export function reportFlatCredits({ billing, provider, modelId, generationId }: ReportFlatCreditsParams): void {
- if (isNil(reporter)) {
- unreportedCalls += 1
- return
- }
- reporter({ billing, provider, modelId, call: { charge: 'flat-credits', credits: FLAT_CREDITS_PER_MODEL_CALL, ...spreadIfDefined('generationId', generationId) } })
+export function reportFixedCredits({ billing, provider, modelId, generationId }: ReportFixedCreditsParams): void {
+ report({
+ context: { basis: AiChargeBasis.FIXED_CREDITS, billing, provider, modelId },
+ call: createFixedCreditChargeInfo({ generationId }),
+ })
}
-export function observedEmbeddingFetch({ provider, modelId, billing, wrapped }: ObservedEmbeddingFetchParams): typeof globalThis.fetch | undefined {
- if (provider !== AIProviderName.ACTIVEPIECES || isNil(billing)) {
- return wrapped
+function billingContextOf({ provider, modelId, billing, charge }: BillingContextParams): BillingContext | undefined {
+ if (isNil(billing) || isNil(charge)) {
+ return undefined
}
- const send = wrapped ?? fetch
- return async (input, init) => {
- const response = await send(input, init)
- void readEmbeddingCall(response)
- .then((call) => reportUsage({ billing, provider, modelId, call }))
- .catch(() => undefined)
- return response
+ if (charge.basis === AiChargeBasis.FIXED_CREDITS) {
+ return { basis: AiChargeBasis.FIXED_CREDITS, billing, provider, modelId }
}
+ return { basis: AiChargeBasis.PROVIDER_REPORTED_COST, billing, provider, modelId, apiKey: charge.managedApiKey }
+}
+
+async function billGenerate({ doGenerate, context }: BillGenerateParams): Promise<LanguageModelV4GenerateResult> {
+ const result = await doGenerate()
+ const generationId = result.response?.id
+ report({ context, generationId, call: createBillingInfo({ context, generationId, providerMetadata: result.providerMetadata, usage: result.usage }) })
+ return result
+}
+
+async function billStream({ doStream, context }: BillStreamParams): Promise<LanguageModelV4StreamResult> {
+ const { stream, ...rest } = await doStream()
+ return { ...rest, stream: stream.pipeThrough(billedStream(context)) }
}
-function toGeneratedCall(result: LanguageModelV4GenerateResult): ActivepiecesAiCall | undefined {
- return toCall({ generationId: result.response?.id, providerMetadata: result.providerMetadata })
+async function billEmbed({ doEmbed, context }: BillEmbedParams): Promise<EmbeddingModelV4Result> {
+ const result = await doEmbed()
+ const generationId = parseOrUndefined(EmbeddingResponseBody, result.response?.body)?.id
+ report({ context, generationId, call: createEmbeddingBillingInfo({ result, generationId }) })
+ return result
}
-function streamCostObserver({ billing, provider, modelId }: ObserverParams): TransformStream<LanguageModelV4StreamPart, LanguageModelV4StreamPart> {
- let generationId: string | undefined
- let reported = false
+function billedStream(context: BillingContext): TransformStream<LanguageModelV4StreamPart, LanguageModelV4StreamPart> {
+ const state: StreamState = { reported: false }
return new TransformStream({
- transform: (chunk, controller) => {
- if (chunk.type === 'response-metadata') {
- generationId = chunk.id ?? generationId
- }
- if (chunk.type === 'finish' && !reported) {
- reported = true
- reportUsage({ billing, provider, modelId, call: toCall({ generationId, providerMetadata: chunk.providerMetadata }) })
- }
- controller.enqueue(chunk)
- },
+ transform: (chunk, controller) => trackChunk({ chunk, controller, state, context }),
+ flush: () => settleStream({ state, context }),
+ cancel: () => settleStream({ state, context }),
})
}
-function flatCreditObserver({ billing, provider, modelId }: ObserverParams): TransformStream<LanguageModelV4StreamPart, LanguageModelV4StreamPart> {
- let generationId: string | undefined
- let reported = false
- return new TransformStream({
- transform: (chunk, controller) => {
- if (chunk.type === 'response-metadata') {
- generationId = chunk.id ?? generationId
- }
- if (chunk.type === 'finish' && !reported) {
- reported = true
- reportFlatCredits({ billing, provider, modelId, generationId })
- }
- controller.enqueue(chunk)
+function trackChunk({ chunk, controller, state, context }: TrackChunkParams): void {
+ if (chunk.type === 'response-metadata') {
+ state.generationId = chunk.id ?? state.generationId
+ }
+ if (chunk.type === 'finish' && !state.reported) {
+ const call = createBillingInfo({ context, generationId: state.generationId, providerMetadata: chunk.providerMetadata, usage: chunk.usage })
+ if (!isNil(call)) {
+ state.reported = true
+ report({ context, generationId: state.generationId, call })
+ }
+ }
+ controller.enqueue(chunk)
+}
+
+function settleStream({ state, context }: SettleStreamParams): void {
+ if (state.reported) {
+ return
+ }
+ state.reported = true
+ if (context.basis === AiChargeBasis.FIXED_CREDITS) {
+ report({ context, generationId: state.generationId, call: createFixedCreditChargeInfo({ generationId: state.generationId }) })
+ return
+ }
+ recoverCostAndReport({ state, context }).catch(() => report({ context, generationId: state.generationId, call: undefined }))
+}
+
+async function recoverCostAndReport({ state, context }: SettleStreamParams): Promise<void> {
+ const { generationId } = state
+ const { apiKey } = context
+ if (isNil(generationId) || isNil(apiKey)) {
+ report({ context, generationId, call: undefined })
+ return
+ }
+ const generation = await lookupGeneration({ apiKey, generationId })
+ if (isNil(generation)) {
+ report({ context, generationId, call: undefined })
+ return
+ }
+ report({
+ context,
+ generationId,
+ call: {
+ charge: AiChargeBasis.PROVIDER_REPORTED_COST,
+ generationId,
+ costUsd: generation.costUsd,
+ inputTokens: generation.inputTokens,
+ outputTokens: generation.outputTokens,
},
})
}
-function toCall({ generationId, providerMetadata }: { generationId: string | undefined, providerMetadata: SharedV4ProviderMetadata | undefined }): ActivepiecesAiCall | undefined {
- const usage = openRouterUsageOf(providerMetadata)
- if (isNil(generationId) || isNil(usage) || isNil(usage.cost)) {
+function createBillingInfo({ context, generationId, providerMetadata, usage }: CreateBillingInfoParams): ActivepiecesAiCall | undefined {
+ const tokens = tokensOf(usage)
+ if (context.basis === AiChargeBasis.FIXED_CREDITS) {
+ return { ...createFixedCreditChargeInfo({ generationId }), ...tokens }
+ }
+ const costUsd = openRouterCostOf(providerMetadata)
+ if (isNil(costUsd)) {
return undefined
}
+ return { charge: AiChargeBasis.PROVIDER_REPORTED_COST, generationId, costUsd, ...tokens }
+}
+
+function createEmbeddingBillingInfo({ result, generationId }: CreateEmbeddingBillingInfoParams): ActivepiecesAiCall | undefined {
+ const costUsd = openRouterCostOf(result.providerMetadata)
+ if (isNil(costUsd)) {
+ return undefined
+ }
+ return { charge: AiChargeBasis.PROVIDER_REPORTED_COST, generationId, costUsd, inputTokens: result.usage?.tokens }
+}
+
+function createFixedCreditChargeInfo({ generationId }: { generationId?: string }): ActivepiecesAiCall {
+ return { charge: AiChargeBasis.FIXED_CREDITS, credits: FIXED_CREDITS_PER_MODEL_CALL, generationId }
+}
+
+function tokensOf(usage: LanguageModelV4Usage | undefined): AiCallTokens {
return {
- charge: 'observed-cost',
- generationId,
- costUsd: usage.cost,
- ...(isNil(usage.promptTokens) ? {} : { inputTokens: usage.promptTokens }),
- ...(isNil(usage.completionTokens) ? {} : { outputTokens: usage.completionTokens }),
+ inputTokens: usage?.inputTokens?.total,
+ outputTokens: usage?.outputTokens?.total,
}
}
-function openRouterUsageOf(providerMetadata: SharedV4ProviderMetadata | undefined): OpenRouterUsage | undefined {
+function openRouterCostOf(providerMetadata: SharedV4ProviderMetadata | undefined): number | undefined {
const openRouter = providerMetadata?.['openrouter']
if (isNil(openRouter)) {
return undefined
@@ -126,23 +167,7 @@ function openRouterUsageOf(providerMetadata: SharedV4ProviderMetadata | undefine
if (!isJSONObject(usage)) {
return undefined
}
- return parseOrUndefined(OpenRouterUsage, usage)
-}
-
-async function readEmbeddingCall(response: Response): Promise<ActivepiecesAiCall | undefined> {
- if (!response.ok) {
- return undefined
- }
- const body = parseOrUndefined(EmbeddingCostResponse, await response.clone().json())
- if (isNil(body) || isNil(body.usage.cost)) {
- return undefined
- }
- return {
- charge: 'observed-cost',
- generationId: body.id,
- costUsd: body.usage.cost,
- ...(isNil(body.usage.prompt_tokens) ? {} : { inputTokens: body.usage.prompt_tokens }),
- }
+ return parseOrUndefined(OpenRouterUsage, usage)?.cost
}
function parseOrUndefined<T extends z.ZodType>(schema: T, value: unknown): z.infer<T> | undefined {
@@ -150,87 +175,180 @@ function parseOrUndefined<T extends z.ZodType>(schema: T, value: unknown): z.inf
return parsed.success ? parsed.data : undefined
}
-function reportUsage({ billing, provider, modelId, call }: ReportUsageParams): void {
+function report({ context, call, generationId }: ReportParams): void {
if (isNil(call)) {
- unreportedCalls += 1
+ recordUnbilledCall({ context, generationId, reason: isNil(generationId) ? UnbilledCallReason.NO_GENERATION_ID : UnbilledCallReason.PROVIDER_REPORTED_NO_COST })
return
}
if (isNil(reporter)) {
- unreportedCalls += 1
+ recordUnbilledCall({ context, generationId: call.generationId ?? generationId, reason: UnbilledCallReason.NO_REPORTER_INSTALLED })
return
}
- reporter({ billing, provider, modelId, call })
+ reporter({ billing: context.billing, provider: context.provider, modelId: context.modelId, call })
}
-const OpenRouterUsage = z.object({
- cost: z.number().optional(),
- promptTokens: z.number().optional(),
- completionTokens: z.number().optional(),
-})
+function recordUnbilledCall({ context, generationId, reason }: RecordUnbilledCallParams): void {
+ const unbilled: UnbilledCall = {
+ reason,
+ provider: context.provider,
+ modelId: context.modelId,
+ source: context.billing.source,
+ platformId: context.billing.platformId,
+ ...billingIdsOf(context.billing),
+ generationId,
+ }
+ unbilledCalls = [...unbilledCalls, unbilled]
+}
+
+function billingIdsOf(billing: ActivepiecesAiBilling): UnbilledCallOwner {
+ switch (billing.source) {
+ case ActivepiecesAiConsumerSource.AI_STEP_IN_FLOW:
+ return { projectId: billing.projectId, flowRunId: billing.flowRun.flowRunId }
+ case ActivepiecesAiConsumerSource.CHAT:
+ return { conversationId: billing.conversationId, ...spreadIfDefined('projectId', billing.projectId) }
+ }
+}
+
+function drainUnbilledCalls(): UnbilledCall[] {
+ const drained = unbilledCalls
+ unbilledCalls = []
+ return drained
+}
-const EmbeddingCostResponse = z.object({
+const EmbeddingResponseBody = z.object({
id: z.string().min(1),
- usage: z.object({
- cost: z.number().optional(),
- prompt_tokens: z.number().optional(),
- }),
+})
+
+const OpenRouterUsage = z.object({
+ cost: z.number().optional(),
})
let reporter: ActivepiecesAiCostReporter | undefined
-let unreportedCalls = 0
+let unbilledCalls: UnbilledCall[] = []
-const FLAT_CREDITS_PER_MODEL_CALL = 1
+const FIXED_CREDITS_PER_MODEL_CALL = 1
export const activepiecesAiCost = {
setReporter: (next: ActivepiecesAiCostReporter): void => {
reporter = next
},
- unreportedCallCount: (): number => unreportedCalls,
+ drainUnbilledCalls,
billedLanguageModel,
- observedEmbeddingFetch,
- reportFlatCredits,
+ billedEmbeddingModel,
+ reportFixedCredits,
}
-type OpenRouterUsage = z.infer<typeof OpenRouterUsage>
+type BillingContext = {
+ basis: AiChargeBasis
+ billing: ActivepiecesAiBilling
+ provider: AIProviderName
+ modelId: string
+ apiKey?: string
+}
-type BilledLanguageModelParams = {
- model: LanguageModel
+type BillingContextParams = {
provider: AIProviderName
modelId: string
billing: ActivepiecesAiBilling | undefined
- ownKeyCredit?: OwnKeyCredit
+ charge: AiCharge | undefined
}
-export type OwnKeyCredit = 'per-call' | 'charged-with-the-turn'
-
-type ObservedEmbeddingFetchParams = {
+type BilledLanguageModelParams = {
+ model: LanguageModel
provider: AIProviderName
modelId: string
billing: ActivepiecesAiBilling | undefined
- wrapped: typeof globalThis.fetch | undefined
+ charge: AiCharge | undefined
}
-type BilledAtAFlatRateParams = {
- model: Exclude<LanguageModel, string>
+type BilledEmbeddingModelParams = {
+ model: EmbeddingModel
provider: AIProviderName
modelId: string
- billing: ActivepiecesAiBilling
+ billing: ActivepiecesAiBilling | undefined
}
-type ReportFlatCreditsParams = {
+type ReportFixedCreditsParams = {
billing: ActivepiecesAiBilling
provider: AIProviderName
modelId: string
generationId?: string
}
-type ObserverParams = {
- billing: ActivepiecesAiBilling
- provider: AIProviderName
- modelId: string
+type StreamState = {
+ reported: boolean
+ generationId?: string
}
-type ReportUsageParams = ObserverParams & {
+type BillGenerateParams = {
+ doGenerate: () => PromiseLike<LanguageModelV4GenerateResult>
+ context: BillingContext
+}
+
+type BillStreamParams = {
+ doStream: () => PromiseLike<LanguageModelV4StreamResult>
+ context: BillingContext
+}
+
+type BillEmbedParams = {
+ doEmbed: () => PromiseLike<EmbeddingModelV4Result>
+ context: BillingContext
+}
+
+type TrackChunkParams = {
+ chunk: LanguageModelV4StreamPart
+ controller: TransformStreamDefaultController<LanguageModelV4StreamPart>
+ state: StreamState
+ context: BillingContext
+}
+
+type SettleStreamParams = {
+ state: StreamState
+ context: BillingContext
+}
+
+type CreateBillingInfoParams = {
+ context: BillingContext
+ generationId: string | undefined
+ providerMetadata: SharedV4ProviderMetadata | undefined
+ usage: LanguageModelV4Usage | undefined
+}
+
+type ReportParams = {
+ context: BillingContext
call: ActivepiecesAiCall | undefined
+ generationId?: string
+}
+
+type CreateEmbeddingBillingInfoParams = {
+ result: EmbeddingModelV4Result
+ generationId: string | undefined
+}
+
+type RecordUnbilledCallParams = {
+ context: BillingContext
+ generationId: string | undefined
+ reason: UnbilledCallReason
+}
+
+type UnbilledCallOwner = {
+ projectId?: string
+ flowRunId?: string
+ conversationId?: string
+}
+
+export enum UnbilledCallReason {
+ PROVIDER_REPORTED_NO_COST = 'provider-reported-no-cost',
+ NO_GENERATION_ID = 'no-generation-id',
+ NO_REPORTER_INSTALLED = 'no-reporter-installed',
+}
+
+export type UnbilledCall = UnbilledCallOwner & {
+ reason: UnbilledCallReason
+ provider: AIProviderName
+ modelId: string
+ source: ActivepiecesAiConsumerSource
+ platformId: string
+ generationId?: string
}
diff --git a/packages/server/utils/src/ai-cost-reporter.ts b/packages/server/utils/src/ai-cost-reporter.ts
new file mode 100644
index 00000000000..29191a7a1bd
--- /dev/null
+++ b/packages/server/utils/src/ai-cost-reporter.ts
@@ -0,0 +1,50 @@
+import { apId, isNil, tryCatch } from '@activepieces/core-utils'
+import { ReportAiUsageRequest } from '@activepieces/shared'
+import { activepiecesAiCost } from './activepieces-ai-cost'
+import { aiUsageReportOf } from './ai-usage-report'
+import { onCallService } from './on-call.service'
+
+function install({ log, report, pageWebhookUrl }: InstallAiCostReporterParams): void {
+ activepiecesAiCost.setReporter((event) => {
+ tryCatch(() => report(aiUsageReportOf({ event, idempotencyKey: `ai:${apId()}` }))).then(({ error }) => {
+ if (isNil(error)) {
+ return
+ }
+ log.error({ error, provider: event.provider, model: { id: event.modelId }, generation: { id: event.call.generationId } }, '[aiCostReporter] An AI call went unbilled')
+ }).catch(() => undefined)
+ })
+ startUnbilledCallWatch({ log, pageWebhookUrl })
+}
+
+function startUnbilledCallWatch({ log, pageWebhookUrl }: Omit<InstallAiCostReporterParams, 'report'>): void {
+ const timer = setInterval(() => {
+ const unbilledCalls = activepiecesAiCost.drainUnbilledCalls()
+ if (unbilledCalls.length === 0) {
+ return
+ }
+ const details = { unbilledCallCount: unbilledCalls.length, unbilledCalls }
+ log.error(details, '[aiCostReporter] AI calls ran without a charge to bill')
+ onCallService(log, pageWebhookUrl()).page({
+ code: 'AI_CALLS_WENT_UNBILLED',
+ message: 'AI calls ran without a charge to bill. Either a managed call reported no cost, or a call was made before the cost reporter was installed.',
+ params: details,
+ }).catch(() => undefined)
+ }, UNBILLED_CHECK_INTERVAL_MS)
+ timer.unref()
+}
+
+const UNBILLED_CHECK_INTERVAL_MS = 5 * 60 * 1000
+
+export const aiCostReporter = {
+ install,
+}
+
+export type AiCostReporterLogger = {
+ error: (obj: Record<string, unknown>, msg: string) => void
+}
+
+export type InstallAiCostReporterParams = {
+ log: AiCostReporterLogger
+ report: (request: ReportAiUsageRequest) => Promise<void>
+ pageWebhookUrl: () => string | undefined
+}
diff --git a/packages/server/utils/src/ai-provider-key-health.ts b/packages/server/utils/src/ai-provider-key-health.ts
new file mode 100644
index 00000000000..adb3ff11a49
--- /dev/null
+++ b/packages/server/utils/src/ai-provider-key-health.ts
@@ -0,0 +1,30 @@
+import { isNil, ProviderOutcomeReporter, ProviderOutcomeSignal } from '@activepieces/core-utils'
+
+export function keyHealthReporterFor({ platformId, providerConfigId }: KeyHealthTarget): ProviderOutcomeReporter | undefined {
+ const sink = reporter
+ if (isNil(sink) || isNil(platformId) || isNil(providerConfigId)) {
+ return undefined
+ }
+ return (signal) => sink({ platformId, providerConfigId, signal })
+}
+
+let reporter: KeyHealthReporter | undefined
+
+export const aiProviderKeyHealth = {
+ setReporter: (next: KeyHealthReporter): void => {
+ reporter = next
+ },
+}
+
+export type KeyHealthObservation = {
+ platformId: string
+ providerConfigId: string
+ signal: ProviderOutcomeSignal
+}
+
+export type KeyHealthReporter = (observation: KeyHealthObservation) => void
+
+export type KeyHealthTarget = {
+ platformId?: string
+ providerConfigId?: string
+}
diff --git a/packages/server/utils/src/ai-usage-report.ts b/packages/server/utils/src/ai-usage-report.ts
new file mode 100644
index 00000000000..733ce1f0a74
--- /dev/null
+++ b/packages/server/utils/src/ai-usage-report.ts
@@ -0,0 +1,34 @@
+import { ActivepiecesAiCall, ActivepiecesAiConsumerSource, ActivepiecesAiCostEvent, AiChargeBasis } from '@activepieces/core-utils'
+import { AiUsageCharge, ReportAiUsageRequest } from '@activepieces/shared'
+
+export function aiUsageReportOf({ event, idempotencyKey }: AiUsageReportParams): ReportAiUsageRequest {
+ const { billing, provider, modelId, call } = event
+ const baseProps: ReportAiUsageRequest = {
+ billing,
+ provider,
+ modelId,
+ idempotencyKey,
+ usage: chargeOf(call),
+ generationId: call.generationId,
+ inputTokens: call.inputTokens,
+ outputTokens: call.outputTokens,
+ }
+ switch (billing.source) {
+ case ActivepiecesAiConsumerSource.AI_STEP_IN_FLOW:
+ return { ...baseProps, flowRun: billing.flowRun }
+ case ActivepiecesAiConsumerSource.CHAT:
+ return { ...baseProps, chat: billing.chat }
+ }
+}
+
+function chargeOf(call: ActivepiecesAiCall): AiUsageCharge {
+ if (call.charge === AiChargeBasis.FIXED_CREDITS) {
+ return { type: AiChargeBasis.FIXED_CREDITS, credits: call.credits }
+ }
+ return { type: AiChargeBasis.PROVIDER_REPORTED_COST, costUsd: call.costUsd }
+}
+
+export type AiUsageReportParams = {
+ event: ActivepiecesAiCostEvent
+ idempotencyKey: string
+}
diff --git a/packages/server/utils/src/ai-utils.ts b/packages/server/utils/src/ai-utils.ts
index c2ec61f4274..525637153de 100644
--- a/packages/server/utils/src/ai-utils.ts
+++ b/packages/server/utils/src/ai-utils.ts
@@ -1,60 +1,60 @@
-import { ActivepiecesAiBilling, AIProviderName, isNil, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils';
+import { ActivepiecesAiBilling, aiChargeFor, AIProviderName, isNil, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils';
import { CloudflareGatewayMetadata, createCloudflareGatewayModel, createImageModel, createLanguageModel } from '@activepieces/ai-providers';
-import { AI_PROVIDER_CAPABILITIES, AIWebSearchMode, BaseAIProviderAuthConfig, getEffectiveProviderAndModel } from '@activepieces/shared';
-import { createAnthropic } from '@ai-sdk/anthropic'
+import { AI_PROVIDER_CAPABILITIES, AiProviderCredentials, AIWebSearchMode, getEffectiveProviderAndModel } from '@activepieces/shared';
+import { anthropic } from '@ai-sdk/anthropic'
import { createAzure } from '@ai-sdk/azure'
-import { createGoogleGenerativeAI } from '@ai-sdk/google'
-import { createOpenAI } from '@ai-sdk/openai'
+import { createGoogleGenerativeAI, google } from '@ai-sdk/google'
+import { createOpenAI, openai } from '@ai-sdk/openai'
import { SharedV3ProviderOptions } from '@ai-sdk/provider'
import { createOpenRouter, OpenRouterChatSettings } from '@openrouter/ai-sdk-provider'
import { EmbeddingModel, ImageModel, LanguageModel, ToolSet } from 'ai'
-import { billedLanguageModel, OwnKeyCredit } from './activepieces-ai-cost'
+import { billedEmbeddingModel, billedLanguageModel } from './activepieces-ai-cost'
+import { keyHealthReporterFor } from './ai-provider-key-health'
const DEFAULT_WEB_SEARCH_RESULTS = 5
const MIN_OPENROUTER_WEB_SEARCH_RESULTS = 1
const MAX_OPENROUTER_WEB_SEARCH_RESULTS = 10
const REPORT_WHAT_THE_CALL_COST = { usage: { include: true } } as const
+const EMBEDDING_REPORTS_WHAT_IT_COST = { extraBody: { usage: { include: true } } } as const
const NATIVE_WEB_SEARCH_TOOLS: Record<string, (params: NativeWebSearchToolParams) => ToolSet> = {
- [AIProviderName.ANTHROPIC]: ({ auth: { apiKey }, options }) => ({
- web_search: createAnthropic({ apiKey }).tools.webSearch_20250305({
+ [AIProviderName.ANTHROPIC]: ({ options }) => ({
+ web_search: anthropic.tools.webSearch_20250305({
maxUses: options.maxUses ?? DEFAULT_WEB_SEARCH_RESULTS,
...spreadIfDefined('userLocation', buildUserLocation(options)),
...spreadIfDefined('allowedDomains', allowedSearchDomains(options)),
...spreadIfDefined('blockedDomains', blockedSearchDomains(options)),
}),
}),
- [AIProviderName.OPENAI]: ({ auth: { apiKey }, options }) => ({
- web_search_preview: createOpenAI({ apiKey }).tools.webSearchPreview({
+ [AIProviderName.OPENAI]: ({ options }) => ({
+ web_search_preview: openai.tools.webSearchPreview({
...spreadIfDefined('searchContextSize', options.searchContextSize),
...spreadIfDefined('userLocation', buildUserLocation(options)),
}),
}),
- [AIProviderName.GOOGLE]: ({ auth: { apiKey } }) => ({ google_search: createGoogleGenerativeAI({ apiKey }).tools.googleSearch({}) }),
+ [AIProviderName.GOOGLE]: () => ({ google_search: google.tools.googleSearch({}) }),
}
function supportsWebSearch(provider: AIProviderName): boolean {
return AI_PROVIDER_CAPABILITIES[provider].webSearch !== undefined
}
-function buildWebSearchTools({ provider, model, auth, options = {} }: {
+function buildWebSearchTools({ provider, model, options = {} }: {
provider: AIProviderName
model?: string
- auth: Record<string, unknown>
options?: WebSearchOptions
}): ToolSet {
const { provider: searchProvider } = getEffectiveProviderAndModel({ provider, model })
- return NATIVE_WEB_SEARCH_TOOLS[searchProvider ?? provider]?.({ auth: auth as BaseAIProviderAuthConfig, options }) ?? {}
+ return NATIVE_WEB_SEARCH_TOOLS[searchProvider ?? provider]?.({ options }) ?? {}
}
function webSearchModeOf(provider: AIProviderName): AIWebSearchMode | undefined {
return AI_PROVIDER_CAPABILITIES[provider].webSearch
}
-function buildWebSearchToolsOrThrow({ provider, model, auth, webSearchEnabled, options = {} }: {
+function buildWebSearchToolsOrThrow({ provider, model, webSearchEnabled, options = {} }: {
provider: AIProviderName
model?: string
- auth: Record<string, unknown>
webSearchEnabled: boolean
options?: WebSearchOptions
}): ToolSet {
@@ -69,7 +69,7 @@ function buildWebSearchToolsOrThrow({ provider, model, auth, webSearchEnabled, o
if (isNil(NATIVE_WEB_SEARCH_TOOLS[resolvedProvider])) {
throw new Error(`Provider ${resolvedProvider} is not supported for web search`)
}
- return buildWebSearchTools({ provider, model, auth, options })
+ return buildWebSearchTools({ provider, model, options })
}
function buildUserLocation(options: WebSearchOptions): UserLocation | undefined {
@@ -116,28 +116,31 @@ function openRouterWebSearchResults(options?: WebSearchOptions): number {
)
}
-function createModel({ provider, auth, config, modelId, metadata, flowStep, billing, ownKeyCredit, openaiResponsesModel = false, webSearchEnabled = false, webSearchOptions, onOutcome }: {
- provider: AIProviderName
- auth: Record<string, unknown>
- config: Record<string, unknown>
+function createModel({ credentials, modelId, metadata, flowStep, billing, turnAlreadyCharged, openaiResponsesModel = false, webSearchEnabled = false, webSearchOptions, platformId, providerConfigId }: {
+ credentials: AiProviderCredentials
modelId: string
metadata?: ChatModelMetadata
flowStep?: FlowStepMetadata
billing?: ActivepiecesAiBilling
- ownKeyCredit?: OwnKeyCredit
+ turnAlreadyCharged?: boolean
openaiResponsesModel?: boolean
webSearchEnabled?: boolean
webSearchOptions?: WebSearchOptions
- onOutcome?: ProviderOutcomeReporter
+ platformId?: string
+ providerConfigId?: string
}): LanguageModel {
- const model = buildModel({ provider, auth, config, modelId, metadata, flowStep, openaiResponsesModel, webSearchEnabled, webSearchOptions, onOutcome })
- return billedLanguageModel({ model, provider, modelId, billing, ...spreadIfDefined('ownKeyCredit', ownKeyCredit) })
+ const model = buildModel({ credentials, modelId, metadata, flowStep, openaiResponsesModel, webSearchEnabled, webSearchOptions, onOutcome: keyHealthReporterFor({ platformId, providerConfigId }) })
+ return billedLanguageModel({
+ model,
+ provider: credentials.provider,
+ modelId,
+ billing,
+ charge: aiChargeFor({ credentials, turnAlreadyCharged }),
+ })
}
-function buildModel({ provider, auth, config, modelId, metadata, flowStep, openaiResponsesModel, webSearchEnabled, webSearchOptions, onOutcome }: {
- provider: AIProviderName
- auth: Record<string, unknown>
- config: Record<string, unknown>
+function buildModel({ credentials, modelId, metadata, flowStep, openaiResponsesModel, webSearchEnabled, webSearchOptions, onOutcome }: {
+ credentials: AiProviderCredentials
modelId: string
metadata?: ChatModelMetadata
flowStep?: FlowStepMetadata
@@ -146,10 +149,10 @@ function buildModel({ provider, auth, config, modelId, metadata, flowStep, opena
webSearchOptions?: WebSearchOptions
onOutcome?: ProviderOutcomeReporter
}): LanguageModel {
- if (provider === AIProviderName.CLOUDFLARE_GATEWAY) {
+ const { provider } = credentials
+ if (credentials.provider === AIProviderName.CLOUDFLARE_GATEWAY) {
return createCloudflareGatewayModel({
- auth,
- config,
+ credentials,
modelId,
openaiResponsesModel,
routing: isNil(flowStep) ? 'compat' : 'submodel',
@@ -158,9 +161,7 @@ function buildModel({ provider, auth, config, modelId, metadata, flowStep, opena
})
}
return createLanguageModel({
- provider,
- auth,
- config,
+ credentials,
modelId,
options: {
openRouterSettings: openRouterModelSettings({ provider, webSearchEnabled, options: webSearchOptions }),
@@ -191,29 +192,21 @@ function flowStepMetadataHeaders(flowStep?: FlowStepMetadata): Record<string, st
}
}
-function createModelForImages({ provider, auth, config, modelId, flowStep }: {
- provider: AIProviderName
- auth: Record<string, unknown>
- config: Record<string, unknown>
+function createModelForImages({ credentials, modelId, flowStep }: {
+ credentials: AiProviderCredentials
modelId: string
flowStep?: FlowStepMetadata
}): ImageModel | undefined {
- if (provider === AIProviderName.CLOUDFLARE_GATEWAY) {
+ if (credentials.provider === AIProviderName.CLOUDFLARE_GATEWAY) {
return createCloudflareGatewayModel({
- auth,
- config,
+ credentials,
modelId,
isImage: true,
routing: isNil(flowStep) ? 'compat' : 'submodel',
...spreadIfDefined('metadata', cloudflareGatewayMetadata(flowStep)),
})
}
- return createImageModel({ provider, auth, config, modelId })
-}
-
-function readStringField(source: Record<string, unknown>, key: string): string {
- const value = source[key]
- return typeof value === 'string' ? value : ''
+ return createImageModel({ credentials, modelId })
}
function toStorageEmbedding(embedding: number[]): number[] {
@@ -225,36 +218,49 @@ function toStorageEmbedding(embedding: number[]): number[] {
return magnitude === 0 ? truncated : truncated.map((value) => value / magnitude)
}
-function createEmbeddingModel({ provider, auth, config, onOutcome }: {
- provider: AIProviderName
- auth: Record<string, unknown>
- config: Record<string, unknown>
- onOutcome?: ProviderOutcomeReporter
+function createEmbeddingModel({ credentials, billing, platformId, providerConfigId }: {
+ credentials: AiProviderCredentials
+ billing?: ActivepiecesAiBilling
+ platformId?: string
+ providerConfigId?: string
}): { model: EmbeddingModel, providerOptions: SharedV3ProviderOptions } {
- const embeddingModelId = AI_PROVIDER_CAPABILITIES[provider].defaultEmbeddingModel
+ const embeddingModelId = AI_PROVIDER_CAPABILITIES[credentials.provider].defaultEmbeddingModel
if (isNil(embeddingModelId)) {
- throw new Error(`Provider ${provider} does not support knowledge base search`)
+ throw new Error(`Provider ${credentials.provider} does not support knowledge base search`)
}
- const apiKey = readStringField(auth, 'apiKey')
- const fetch = observedProviderFetch(onOutcome)
- switch (provider) {
+ const built = buildEmbeddingModel({ credentials, embeddingModelId, fetch: observedProviderFetch(keyHealthReporterFor({ platformId, providerConfigId })) })
+ return {
+ model: billedEmbeddingModel({ model: built.model, provider: credentials.provider, modelId: embeddingModelId, billing }),
+ providerOptions: built.providerOptions,
+ }
+}
+
+function buildEmbeddingModel({ credentials, embeddingModelId, fetch }: {
+ credentials: AiProviderCredentials
+ embeddingModelId: string
+ fetch: typeof globalThis.fetch | undefined
+}): { model: EmbeddingModel, providerOptions: SharedV3ProviderOptions } {
+ switch (credentials.provider) {
case AIProviderName.OPENAI:
- return { model: createOpenAI({ apiKey, ...spreadIfDefined('fetch', fetch) }).embeddingModel(embeddingModelId), providerOptions: OPENAI_EMBEDDING_PROVIDER_OPTIONS }
+ return { model: createOpenAI({ apiKey: credentials.auth.apiKey, ...spreadIfDefined('fetch', fetch) }).embeddingModel(embeddingModelId), providerOptions: OPENAI_EMBEDDING_PROVIDER_OPTIONS }
case AIProviderName.GOOGLE:
- return { model: createGoogleGenerativeAI({ apiKey, ...spreadIfDefined('fetch', fetch) }).textEmbeddingModel(embeddingModelId), providerOptions: {} }
+ return { model: createGoogleGenerativeAI({ apiKey: credentials.auth.apiKey, ...spreadIfDefined('fetch', fetch) }).textEmbeddingModel(embeddingModelId), providerOptions: {} }
case AIProviderName.AZURE: {
- const resourceName = readStringField(config, 'resourceName')
- const apiVersion = readStringField(config, 'apiVersion')
+ const { resourceName, apiVersion } = credentials.config
return {
- model: createAzure({ resourceName, apiKey, ...spreadIfDefined('apiVersion', apiVersion || undefined), ...spreadIfDefined('fetch', fetch) }).embeddingModel(embeddingModelId),
+ model: createAzure({ resourceName: resourceName ?? '', apiKey: credentials.auth.apiKey, ...spreadIfDefined('apiVersion', apiVersion), ...spreadIfDefined('fetch', fetch) }).embeddingModel(embeddingModelId),
providerOptions: OPENAI_EMBEDDING_PROVIDER_OPTIONS,
}
}
case AIProviderName.ACTIVEPIECES:
case AIProviderName.OPENROUTER:
- return { model: createOpenRouter({ apiKey, ...spreadIfDefined('fetch', fetch) }).textEmbeddingModel(embeddingModelId), providerOptions: OPENROUTER_EMBEDDING_PROVIDER_OPTIONS }
+ return {
+ model: createOpenRouter({ apiKey: credentials.auth.apiKey, ...spreadIfDefined('fetch', fetch) })
+ .textEmbeddingModel(embeddingModelId, credentials.provider === AIProviderName.ACTIVEPIECES ? EMBEDDING_REPORTS_WHAT_IT_COST : undefined),
+ providerOptions: OPENROUTER_EMBEDDING_PROVIDER_OPTIONS,
+ }
default:
- throw new Error(`Provider ${provider} does not support knowledge base search`)
+ throw new Error(`Provider ${credentials.provider} does not support knowledge base search`)
}
}
@@ -294,7 +300,6 @@ export const aiUtils = {
}
type NativeWebSearchToolParams = {
- auth: BaseAIProviderAuthConfig
options: WebSearchOptions
}
diff --git a/packages/server/utils/src/index.ts b/packages/server/utils/src/index.ts
index 1e2a6701d4a..851cefaa3a5 100644
--- a/packages/server/utils/src/index.ts
+++ b/packages/server/utils/src/index.ts
@@ -28,6 +28,10 @@ export type { SsrfAgents } from './safe-http'
export { DEFAULT_MCP_MAX_RESPONSE_BYTES, DEFAULT_MCP_TIMEOUT_MS, mcpTransport } from './mcp-transport'
export { systemUsage } from './system-usage'
export { activepiecesAiCost } from './activepieces-ai-cost'
+export { aiUsageReportOf } from './ai-usage-report'
+export { aiCostReporter } from './ai-cost-reporter'
+export { aiProviderKeyHealth } from './ai-provider-key-health'
+export type { KeyHealthObservation } from './ai-provider-key-health'
export { agentAiUtils } from './agent-ai-utils'
export type { ContentPartLike } from './agent-ai-utils'
export { aiUtils, EMBEDDING_DIMENSIONS } from './ai-utils'
diff --git a/packages/server/utils/src/openrouter-generation.ts b/packages/server/utils/src/openrouter-generation.ts
new file mode 100644
index 00000000000..4da9cb1e6f7
--- /dev/null
+++ b/packages/server/utils/src/openrouter-generation.ts
@@ -0,0 +1,52 @@
+import { isNil, tryCatch } from '@activepieces/core-utils'
+import { z } from 'zod'
+import { safeHttp } from './safe-http'
+
+export async function lookupGeneration({ apiKey, generationId }: LookupGenerationParams): Promise<Generation | undefined> {
+ const { data: response, error } = await tryCatch(() => safeHttp.axios.get<unknown>(GENERATION_URL, {
+ params: { id: generationId },
+ timeout: LOOKUP_TIMEOUT_MS,
+ headers: { Authorization: `Bearer ${apiKey}` },
+ }))
+ if (!isNil(error) || isNil(response)) {
+ return undefined
+ }
+ const parsed = GenerationResponse.safeParse(response.data)
+ return parsed.success ? generationOf(parsed.data.data) : undefined
+}
+
+function generationOf(data: GenerationData): Generation {
+ return {
+ costUsd: data.total_cost,
+ inputTokens: data.native_tokens_prompt ?? data.tokens_prompt ?? undefined,
+ outputTokens: data.native_tokens_completion ?? data.tokens_completion ?? undefined,
+ }
+}
+
+const GENERATION_URL = 'https://openrouter.ai/api/v1/generation'
+const LOOKUP_TIMEOUT_MS = 10_000
+
+const GenerationData = z.object({
+ total_cost: z.number(),
+ tokens_prompt: z.number().nullish(),
+ tokens_completion: z.number().nullish(),
+ native_tokens_prompt: z.number().nullish(),
+ native_tokens_completion: z.number().nullish(),
+})
+
+const GenerationResponse = z.object({
+ data: GenerationData,
+})
+
+export type LookupGenerationParams = {
+ apiKey: string
+ generationId: string
+}
+
+export type Generation = {
+ costUsd: number
+ inputTokens?: number
+ outputTokens?: number
+}
+
+type GenerationData = z.infer<typeof GenerationData>
diff --git a/packages/server/utils/test/activepieces-ai-cost.test.ts b/packages/server/utils/test/activepieces-ai-cost.test.ts
index 800fd81fb3a..c7818812f46 100644
--- a/packages/server/utils/test/activepieces-ai-cost.test.ts
+++ b/packages/server/utils/test/activepieces-ai-cost.test.ts
@@ -1,14 +1,27 @@
-import { ActivepiecesAiBilling, ActivepiecesAiBillingScope, ActivepiecesAiCostEvent, AIProviderName } from '@activepieces/core-utils'
-import { LanguageModelV4 } from '@ai-sdk/provider'
-import { afterEach, beforeEach, describe, expect, it } from 'vitest'
-import { activepiecesAiCost } from '../src/activepieces-ai-cost'
+import { ActivepiecesAiBilling, ActivepiecesAiConsumerSource, ActivepiecesAiCostEvent, aiChargeFor, AiChargeBasis, AIProviderName, aiProviderCredentials } from '@activepieces/core-utils'
+import { EmbeddingModelV4, LanguageModelV4 } from '@ai-sdk/provider'
+import { EmbeddingModel, LanguageModel } from 'ai'
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'
+import { activepiecesAiCost, UnbilledCallReason } from '../src/activepieces-ai-cost'
+import { Generation } from '../src/openrouter-generation'
+
+const generationLookup = vi.hoisted(() => ({
+ answer: async (): Promise<Generation | undefined> => undefined,
+}))
+
+vi.mock('../src/openrouter-generation', () => ({
+ lookupGeneration: () => generationLookup.answer(),
+}))
const BILLING: ActivepiecesAiBilling = {
- scope: ActivepiecesAiBillingScope.PROJECT,
+ source: ActivepiecesAiConsumerSource.AI_STEP_IN_FLOW,
platformId: 'platform-1',
projectId: 'project-1',
+ flowRun: { flowId: 'flow-1', flowRunId: 'run-1' },
}
+const MANAGED_KEY = 'sk-or-managed'
+
const reported: ActivepiecesAiCostEvent[] = []
function modelReturning(result: Record<string, unknown>): LanguageModelV4 {
@@ -22,6 +35,17 @@ function modelReturning(result: Record<string, unknown>): LanguageModelV4 {
}
}
+function embeddingModelReturning(result: Record<string, unknown>): EmbeddingModelV4 {
+ return {
+ specificationVersion: 'v4',
+ provider: 'test',
+ modelId: 'test-embedding',
+ maxEmbeddingsPerCall: undefined,
+ supportsParallelCalls: true,
+ doEmbed: async () => result as unknown as Awaited<ReturnType<EmbeddingModelV4['doEmbed']>>,
+ }
+}
+
async function generateWith(model: ReturnType<typeof activepiecesAiCost.billedLanguageModel>): Promise<void> {
if (typeof model === 'string') {
throw new Error('expected a model object')
@@ -29,8 +53,71 @@ async function generateWith(model: ReturnType<typeof activepiecesAiCost.billedLa
await model.doGenerate({ prompt: [] })
}
+async function embedWith(model: EmbeddingModel): Promise<void> {
+ if (typeof model === 'string') {
+ throw new Error('expected a model object')
+ }
+ await model.doEmbed({ values: ['hello'] })
+}
+
+function streamingModel({ provider, apiKey, chunks = [{ type: 'response-metadata', id: 'gen-stream' }] }: {
+ provider: AIProviderName
+ apiKey?: string
+ chunks?: Record<string, unknown>[]
+}): LanguageModel {
+ const raw: LanguageModelV4 = {
+ specificationVersion: 'v4',
+ provider: 'test',
+ modelId: 'test-model',
+ supportedUrls: {},
+ doGenerate: async () => ({}) as unknown as Awaited<ReturnType<LanguageModelV4['doGenerate']>>,
+ doStream: async () => ({ stream: sourceOf(chunks) }),
+ }
+ return activepiecesAiCost.billedLanguageModel({
+ model: raw,
+ provider,
+ modelId: 'anthropic/claude-sonnet-5',
+ billing: BILLING,
+ charge: chargeFor({ provider, apiKey }),
+ })
+}
+
+function chargeFor({ provider, apiKey, turnAlreadyCharged }: { provider: AIProviderName, apiKey?: string, turnAlreadyCharged?: boolean }) {
+ return aiChargeFor({ credentials: aiProviderCredentials({ provider, auth: { apiKey }, config: {} }), turnAlreadyCharged })
+}
+
+function sourceOf(chunks: Record<string, unknown>[]): ReadableStream {
+ const source = new TransformStream()
+ providerSide = source.writable.getWriter()
+ pendingChunks = chunks
+ return source.readable
+}
+
+async function streamThen(model: LanguageModel, act: (handles: { reader: ReadableStreamDefaultReader, writer: WritableStreamDefaultWriter }) => Promise<unknown>): Promise<void> {
+ if (typeof model === 'string') {
+ throw new Error('expected a model object')
+ }
+ const { stream } = await model.doStream({ prompt: [] })
+ const reader = stream.getReader()
+ const writer = providerSide
+ for (const chunk of pendingChunks) {
+ const delivered = reader.read()
+ await writer.write(chunk)
+ await delivered
+ }
+ await act({ reader, writer })
+}
+
+function settled(): Promise<void> {
+ return new Promise((resolve) => setTimeout(resolve, 20))
+}
+
+let providerSide: WritableStreamDefaultWriter
+let pendingChunks: Record<string, unknown>[] = []
+
beforeEach(() => {
reported.length = 0
+ generationLookup.answer = async () => undefined
activepiecesAiCost.setReporter((event) => reported.push(event))
})
@@ -44,20 +131,21 @@ describe('what a model call reports back for billing', () => {
model: modelReturning({
content: [],
finishReason: 'stop',
- usage: {},
+ usage: { inputTokens: { total: 100 }, outputTokens: { total: 20 } },
response: { id: 'gen-abc' },
- providerMetadata: { openrouter: { usage: { cost: 0.0042, promptTokens: 100, completionTokens: 20 } } },
+ providerMetadata: { openrouter: { usage: { cost: 0.0042 } } },
}),
provider: AIProviderName.ACTIVEPIECES,
modelId: 'anthropic/claude-sonnet-5',
billing: BILLING,
+ charge: chargeFor({ provider: AIProviderName.ACTIVEPIECES }),
})
await generateWith(model)
expect(reported).toHaveLength(1)
expect(reported[0].call).toEqual({
- charge: 'observed-cost',
+ charge: AiChargeBasis.PROVIDER_REPORTED_COST,
generationId: 'gen-abc',
costUsd: 0.0042,
inputTokens: 100,
@@ -65,33 +153,70 @@ describe('what a model call reports back for billing', () => {
})
})
- it('bills a call on the customer own key at one flat credit, whatever it cost us', async () => {
+ it('bills a call on the customer own key at one fixed credit, whatever it cost us', async () => {
const model = activepiecesAiCost.billedLanguageModel({
- model: modelReturning({ content: [], finishReason: 'stop', usage: {}, response: { id: 'resp-1' } }),
+ model: modelReturning({
+ content: [],
+ finishReason: 'stop',
+ usage: { inputTokens: { total: 7 }, outputTokens: { total: 3 } },
+ response: { id: 'resp-1' },
+ }),
provider: AIProviderName.OPENAI,
modelId: 'gpt-5',
billing: BILLING,
+ charge: chargeFor({ provider: AIProviderName.OPENAI }),
})
await generateWith(model)
expect(reported).toHaveLength(1)
- expect(reported[0].call).toEqual({ charge: 'flat-credits', credits: 1, generationId: 'resp-1' })
+ expect(reported[0].call).toEqual({
+ charge: AiChargeBasis.FIXED_CREDITS,
+ credits: 1,
+ generationId: 'resp-1',
+ inputTokens: 7,
+ outputTokens: 3,
+ })
})
it('counts a managed call whose cost never arrived, instead of quietly billing nothing', async () => {
- const before = activepiecesAiCost.unreportedCallCount()
+ activepiecesAiCost.drainUnbilledCalls()
const model = activepiecesAiCost.billedLanguageModel({
model: modelReturning({ content: [], finishReason: 'stop', usage: {}, response: { id: 'gen-xyz' } }),
provider: AIProviderName.ACTIVEPIECES,
modelId: 'anthropic/claude-sonnet-5',
billing: BILLING,
+ charge: chargeFor({ provider: AIProviderName.ACTIVEPIECES }),
})
await generateWith(model)
expect(reported).toHaveLength(0)
- expect(activepiecesAiCost.unreportedCallCount()).toBe(before + 1)
+ expect(activepiecesAiCost.drainUnbilledCalls()).toHaveLength(1)
+ })
+
+ it('says which project and model lost the charge, so the page names something to chase', async () => {
+ activepiecesAiCost.drainUnbilledCalls()
+ const model = activepiecesAiCost.billedLanguageModel({
+ model: modelReturning({ content: [], finishReason: 'stop', usage: {}, response: { id: 'gen-xyz' } }),
+ provider: AIProviderName.ACTIVEPIECES,
+ modelId: 'anthropic/claude-sonnet-5',
+ billing: BILLING,
+ charge: chargeFor({ provider: AIProviderName.ACTIVEPIECES }),
+ })
+
+ await generateWith(model)
+
+ expect(activepiecesAiCost.drainUnbilledCalls()).toEqual([{
+ reason: UnbilledCallReason.PROVIDER_REPORTED_NO_COST,
+ provider: AIProviderName.ACTIVEPIECES,
+ modelId: 'anthropic/claude-sonnet-5',
+ source: ActivepiecesAiConsumerSource.AI_STEP_IN_FLOW,
+ platformId: 'platform-1',
+ projectId: 'project-1',
+ flowRunId: 'run-1',
+ generationId: 'gen-xyz',
+ }])
})
it('leaves a model alone when nothing is being billed for it', async () => {
@@ -101,6 +226,7 @@ describe('what a model call reports back for billing', () => {
provider: AIProviderName.ACTIVEPIECES,
modelId: 'anthropic/claude-sonnet-5',
billing: undefined,
+ charge: chargeFor({ provider: AIProviderName.ACTIVEPIECES }),
})
expect(model).toBe(raw)
@@ -114,7 +240,7 @@ describe('what a model call reports back for billing', () => {
provider: AIProviderName.OPENAI,
modelId: 'gpt-5',
billing: BILLING,
- ownKeyCredit: 'charged-with-the-turn',
+ charge: chargeFor({ provider: AIProviderName.OPENAI, turnAlreadyCharged: true }),
})
await generateWith(model)
@@ -134,19 +260,245 @@ describe('what a model call reports back for billing', () => {
provider: AIProviderName.ACTIVEPIECES,
modelId: 'anthropic/claude-sonnet-5',
billing: BILLING,
- ownKeyCredit: 'charged-with-the-turn',
+ charge: chargeFor({ provider: AIProviderName.ACTIVEPIECES, turnAlreadyCharged: true }),
})
await generateWith(model)
expect(reported).toHaveLength(1)
- expect(reported[0].call).toEqual({ charge: 'observed-cost', generationId: 'gen-def', costUsd: 0.001 })
+ expect(reported[0].call).toEqual({ charge: AiChargeBasis.PROVIDER_REPORTED_COST, generationId: 'gen-def', costUsd: 0.001 })
+ })
+
+ it('bills an image model call that cannot be wrapped at the same fixed credit', () => {
+ activepiecesAiCost.reportFixedCredits({ billing: BILLING, provider: AIProviderName.OPENAI, modelId: 'dall-e-3' })
+
+ expect(reported).toHaveLength(1)
+ expect(reported[0].call).toEqual({ charge: AiChargeBasis.FIXED_CREDITS, credits: 1 })
+ })
+})
+
+describe('an embedding call on the managed provider', () => {
+ it('reports what the call cost, so knowledge base search is not served for free', async () => {
+ const model = activepiecesAiCost.billedEmbeddingModel({
+ model: embeddingModelReturning({
+ embeddings: [[0.1]],
+ usage: { tokens: 8 },
+ providerMetadata: { openrouter: { usage: { cost: 0.000012 } } },
+ response: { body: { id: 'embed-1' } },
+ warnings: [],
+ }),
+ provider: AIProviderName.ACTIVEPIECES,
+ modelId: 'openai/text-embedding-3-small',
+ billing: BILLING,
+ })
+
+ await embedWith(model)
+
+ expect(reported).toHaveLength(1)
+ expect(reported[0].call).toEqual({
+ charge: AiChargeBasis.PROVIDER_REPORTED_COST,
+ generationId: 'embed-1',
+ costUsd: 0.000012,
+ inputTokens: 8,
+ })
+ })
+
+ it('leaves a customer own-key embedding alone, we are not paying for it', async () => {
+ const raw = embeddingModelReturning({
+ embeddings: [[0.1]],
+ usage: { tokens: 8 },
+ providerMetadata: { openrouter: { usage: { cost: 0.5 } } },
+ response: { body: { id: 'embed-2' } },
+ warnings: [],
+ })
+ const model = activepiecesAiCost.billedEmbeddingModel({
+ model: raw,
+ provider: AIProviderName.OPENAI,
+ modelId: 'text-embedding-3-small',
+ billing: BILLING,
+ })
+
+ expect(model).toBe(raw)
+ await embedWith(model)
+ expect(reported).toHaveLength(0)
+ })
+
+ it('bills on the cost even when the response carried no id, because OpenRouter marks it optional', async () => {
+ const model = activepiecesAiCost.billedEmbeddingModel({
+ model: embeddingModelReturning({
+ embeddings: [[0.1]],
+ usage: { tokens: 8 },
+ providerMetadata: { openrouter: { usage: { cost: 0.0004 } } },
+ response: { body: { object: 'list' } },
+ warnings: [],
+ }),
+ provider: AIProviderName.ACTIVEPIECES,
+ modelId: 'openai/text-embedding-3-small',
+ billing: BILLING,
+ })
+
+ await embedWith(model)
+
+ expect(reported).toHaveLength(1)
+ expect(reported[0].call).toEqual({
+ charge: AiChargeBasis.PROVIDER_REPORTED_COST,
+ generationId: undefined,
+ costUsd: 0.0004,
+ inputTokens: 8,
+ })
+ })
+
+ it('counts the call as unbilled when the provider did not say what it cost', async () => {
+ activepiecesAiCost.drainUnbilledCalls()
+ const model = activepiecesAiCost.billedEmbeddingModel({
+ model: embeddingModelReturning({
+ embeddings: [[0.1]],
+ usage: { tokens: 8 },
+ response: { body: { id: 'embed-3' } },
+ warnings: [],
+ }),
+ provider: AIProviderName.ACTIVEPIECES,
+ modelId: 'openai/text-embedding-3-small',
+ billing: BILLING,
+ })
+
+ await embedWith(model)
+
+ expect(reported).toHaveLength(0)
+ expect(activepiecesAiCost.drainUnbilledCalls()).toHaveLength(1)
+ })
+
+ it('hands the embeddings back untouched, so billing cannot change what search sees', async () => {
+ const model = activepiecesAiCost.billedEmbeddingModel({
+ model: embeddingModelReturning({
+ embeddings: [[0.25, 0.5]],
+ usage: { tokens: 4 },
+ providerMetadata: { openrouter: { usage: { cost: 0.1 } } },
+ response: { body: { id: 'embed-4' } },
+ warnings: [],
+ }),
+ provider: AIProviderName.ACTIVEPIECES,
+ modelId: 'openai/text-embedding-3-small',
+ billing: BILLING,
+ })
+
+ if (typeof model === 'string') {
+ throw new Error('expected a model object')
+ }
+ const { embeddings } = await model.doEmbed({ values: ['hello'] })
+
+ expect(embeddings).toEqual([[0.25, 0.5]])
+ })
+})
+
+describe('a stream that never reaches its finish chunk', () => {
+ it('bills a managed call on the cost the provider reports when the client walks away mid-stream', async () => {
+ generationLookup.answer = async () => ({ costUsd: 0.0031, inputTokens: 40, outputTokens: 9 })
+ const model = streamingModel({ provider: AIProviderName.ACTIVEPIECES, apiKey: MANAGED_KEY })
+
+ await streamThen(model, ({ reader }) => reader.cancel(new Error('client went away')))
+ await settled()
+
+ expect(reported).toHaveLength(1)
+ expect(reported[0].call).toEqual({
+ charge: AiChargeBasis.PROVIDER_REPORTED_COST,
+ generationId: 'gen-stream',
+ costUsd: 0.0031,
+ inputTokens: 40,
+ outputTokens: 9,
+ })
})
- it('bills an image model call that cannot be wrapped at the same flat credit', () => {
- activepiecesAiCost.reportFlatCredits({ billing: BILLING, provider: AIProviderName.OPENAI, modelId: 'dall-e-3' })
+ it('bills a managed call the same way when the provider errors part-way through', async () => {
+ generationLookup.answer = async () => ({ costUsd: 0.0007 })
+ const model = streamingModel({ provider: AIProviderName.ACTIVEPIECES, apiKey: MANAGED_KEY })
+
+ await streamThen(model, ({ writer }) => writer.abort(new Error('provider blew up')).catch(() => undefined))
+ await settled()
expect(reported).toHaveLength(1)
- expect(reported[0].call).toEqual({ charge: 'flat-credits', credits: 1 })
+ expect(reported[0].call).toEqual({ charge: AiChargeBasis.PROVIDER_REPORTED_COST, generationId: 'gen-stream', costUsd: 0.0007 })
+ })
+
+ it('counts the call as unbilled when the provider cannot tell us what it cost, so the alarm fires', async () => {
+ activepiecesAiCost.drainUnbilledCalls()
+ const model = streamingModel({ provider: AIProviderName.ACTIVEPIECES, apiKey: MANAGED_KEY })
+
+ await streamThen(model, ({ reader }) => reader.cancel(new Error('client went away')))
+ await settled()
+
+ expect(reported).toHaveLength(0)
+ expect(activepiecesAiCost.drainUnbilledCalls()).toHaveLength(1)
+ })
+
+ it('counts the call as unbilled when the stream died before naming a generation to ask about', async () => {
+ activepiecesAiCost.drainUnbilledCalls()
+ generationLookup.answer = async () => ({ costUsd: 0.5 })
+ const model = streamingModel({ provider: AIProviderName.ACTIVEPIECES, apiKey: MANAGED_KEY, chunks: [] })
+
+ await streamThen(model, ({ reader }) => reader.cancel(new Error('client went away')))
+ await settled()
+
+ expect(reported).toHaveLength(0)
+ expect(activepiecesAiCost.drainUnbilledCalls()).toHaveLength(1)
+ })
+
+ it('counts the call as unbilled when there is no managed key to ask with', async () => {
+ activepiecesAiCost.drainUnbilledCalls()
+ generationLookup.answer = async () => ({ costUsd: 0.5 })
+ const model = streamingModel({ provider: AIProviderName.ACTIVEPIECES })
+
+ await streamThen(model, ({ reader }) => reader.cancel(new Error('client went away')))
+ await settled()
+
+ expect(reported).toHaveLength(0)
+ expect(activepiecesAiCost.drainUnbilledCalls()).toHaveLength(1)
+ })
+
+ it('still charges a customer own-key stream its fixed credit, the call was made either way', async () => {
+ const model = streamingModel({ provider: AIProviderName.OPENAI })
+
+ await streamThen(model, ({ reader }) => reader.cancel(new Error('client went away')))
+ await settled()
+
+ expect(reported).toHaveLength(1)
+ expect(reported[0].call).toEqual({ charge: AiChargeBasis.FIXED_CREDITS, credits: 1, generationId: 'gen-stream' })
+ })
+
+ it('bills a finished stream once, and does not ask the provider again as it closes', async () => {
+ let asked = 0
+ generationLookup.answer = async () => {
+ asked += 1
+ return { costUsd: 9.99 }
+ }
+ const model = streamingModel({
+ provider: AIProviderName.ACTIVEPIECES,
+ apiKey: MANAGED_KEY,
+ chunks: [
+ { type: 'response-metadata', id: 'gen-stream' },
+ { type: 'finish', finishReason: 'stop', usage: { inputTokens: { total: 12 }, outputTokens: { total: 5 } }, providerMetadata: { openrouter: { usage: { cost: 0.002 } } } },
+ ],
+ })
+
+ await streamThen(model, async ({ reader, writer }) => {
+ await writer.close()
+ for (;;) {
+ const { done } = await reader.read()
+ if (done) {
+ return
+ }
+ }
+ })
+ await settled()
+
+ expect(asked).toBe(0)
+ expect(reported).toHaveLength(1)
+ expect(reported[0].call).toEqual({
+ charge: AiChargeBasis.PROVIDER_REPORTED_COST,
+ generationId: 'gen-stream',
+ costUsd: 0.002,
+ inputTokens: 12,
+ outputTokens: 5,
+ })
})
})
diff --git a/packages/server/utils/test/ai-utils.test.ts b/packages/server/utils/test/ai-utils.test.ts
index 35e0cd87a81..7016d3bb66d 100644
--- a/packages/server/utils/test/ai-utils.test.ts
+++ b/packages/server/utils/test/ai-utils.test.ts
@@ -1,9 +1,10 @@
import { AIProviderName } from '@activepieces/core-utils'
+import { aiProviderCredentials } from '@activepieces/shared'
import { describe, expect, it } from 'vitest'
import { aiUtils, WebSearchOptions } from '../src/ai-utils'
function anthropicSearchArgs(options: WebSearchOptions): Record<string, unknown> {
- const tools = aiUtils.buildWebSearchTools({ provider: AIProviderName.ANTHROPIC, auth: { apiKey: 'key' }, options })
+ const tools = aiUtils.buildWebSearchTools({ provider: AIProviderName.ANTHROPIC, options })
return toArgs(tools['web_search'])
}
@@ -14,7 +15,7 @@ function toArgs(tool: unknown): Record<string, unknown> {
describe('aiUtils.buildWebSearchTools', () => {
it('defaults to five uses when the caller passes no options, as the agent does', () => {
- const tools = aiUtils.buildWebSearchTools({ provider: AIProviderName.ANTHROPIC, auth: { apiKey: 'key' } })
+ const tools = aiUtils.buildWebSearchTools({ provider: AIProviderName.ANTHROPIC })
expect(toArgs(tools['web_search'])).toEqual({ maxUses: 5 })
})
@@ -47,7 +48,6 @@ describe('aiUtils.buildWebSearchTools', () => {
it('gives OpenAI the Responses-API preview tool with its own options', () => {
const tools = aiUtils.buildWebSearchTools({
provider: AIProviderName.OPENAI,
- auth: { apiKey: 'key' },
options: { searchContextSize: 'high', userLocationCountry: 'JO' },
})
expect(toArgs(tools['web_search_preview']))
@@ -58,7 +58,6 @@ describe('aiUtils.buildWebSearchTools', () => {
const tools = aiUtils.buildWebSearchTools({
provider: AIProviderName.CLOUDFLARE_GATEWAY,
model: 'anthropic/claude-sonnet-4-6',
- auth: { apiKey: 'key' },
options: { maxUses: 3 },
})
expect(toArgs(tools['web_search'])).toEqual({ maxUses: 3 })
@@ -68,18 +67,22 @@ describe('aiUtils.buildWebSearchTools', () => {
expect(aiUtils.buildWebSearchTools({
provider: AIProviderName.CLOUDFLARE_GATEWAY,
model: 'mistral/mistral-large',
- auth: { apiKey: 'key' },
})).toEqual({})
})
it('returns no tools for a provider that has no native web search', () => {
- expect(aiUtils.buildWebSearchTools({ provider: AIProviderName.MISTRAL, auth: { apiKey: 'key' } })).toEqual({})
+ expect(aiUtils.buildWebSearchTools({ provider: AIProviderName.MISTRAL })).toEqual({})
+ })
+
+ it('returns no tools for a key-less provider, which can never reach a native search tool', () => {
+ expect(aiUtils.buildWebSearchTools({ provider: AIProviderName.VERTEX })).toEqual({})
+ expect(aiUtils.buildWebSearchTools({ provider: AIProviderName.BEDROCK })).toEqual({})
})
})
describe('aiUtils.buildWebSearchToolsOrThrow', () => {
it('fails the way the piece does when the provider has no web search', () => {
- expect(() => aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.AZURE, auth: { apiKey: 'key' }, webSearchEnabled: true }))
+ expect(() => aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.AZURE, webSearchEnabled: true }))
.toThrow('Provider azure is not supported for web search')
})
@@ -87,21 +90,19 @@ describe('aiUtils.buildWebSearchToolsOrThrow', () => {
expect(() => aiUtils.buildWebSearchToolsOrThrow({
provider: AIProviderName.CLOUDFLARE_GATEWAY,
model: 'mistral/mistral-large',
- auth: { apiKey: 'key' },
webSearchEnabled: true,
})).toThrow('Provider cloudflare-gateway is not supported for web search')
})
it('returns no tools without throwing for a plugin provider, which searches through the model instead', () => {
- expect(aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.ACTIVEPIECES, auth: { apiKey: 'key' }, webSearchEnabled: true })).toEqual({})
- expect(aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.OPENROUTER, auth: { apiKey: 'key' }, webSearchEnabled: true })).toEqual({})
+ expect(aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.ACTIVEPIECES, webSearchEnabled: true })).toEqual({})
+ expect(aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.OPENROUTER, webSearchEnabled: true })).toEqual({})
})
it('builds the same tool as the total variant when the provider is supported', () => {
const strict = aiUtils.buildWebSearchToolsOrThrow({
provider: AIProviderName.CLOUDFLARE_GATEWAY,
model: 'anthropic/claude-sonnet-4-6',
- auth: { apiKey: 'key' },
webSearchEnabled: true,
options: { maxUses: 3 },
})
@@ -109,7 +110,7 @@ describe('aiUtils.buildWebSearchToolsOrThrow', () => {
})
it('stays silent for an unsupported provider when web search is switched off', () => {
- expect(aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.AZURE, auth: { apiKey: 'key' }, webSearchEnabled: false })).toEqual({})
+ expect(aiUtils.buildWebSearchToolsOrThrow({ provider: AIProviderName.AZURE, webSearchEnabled: false })).toEqual({})
})
})
@@ -118,9 +119,7 @@ describe('aiUtils.createModelForImages', () => {
it('builds an image model for a Cloudflare Gateway model, as the piece does', () => {
const model = aiUtils.createModelForImages({
- provider: AIProviderName.CLOUDFLARE_GATEWAY,
- auth: { apiKey: 'key' },
- config: gatewayConfig,
+ credentials: aiProviderCredentials({ provider: AIProviderName.CLOUDFLARE_GATEWAY, auth: { apiKey: 'key' }, config: gatewayConfig }),
modelId: 'openai/dall-e-3',
})
@@ -129,9 +128,7 @@ describe('aiUtils.createModelForImages', () => {
it('builds an image model for a gateway submodel it has no dedicated branch for', () => {
const model = aiUtils.createModelForImages({
- provider: AIProviderName.CLOUDFLARE_GATEWAY,
- auth: { apiKey: 'key' },
- config: gatewayConfig,
+ credentials: aiProviderCredentials({ provider: AIProviderName.CLOUDFLARE_GATEWAY, auth: { apiKey: 'key' }, config: gatewayConfig }),
modelId: 'workers-ai/flux',
})
@@ -140,9 +137,7 @@ describe('aiUtils.createModelForImages', () => {
it('returns nothing for a provider with no image model, so the caller can fall back to text', () => {
expect(aiUtils.createModelForImages({
- provider: AIProviderName.GOOGLE,
- auth: { apiKey: 'key' },
- config: {},
+ credentials: aiProviderCredentials({ provider: AIProviderName.GOOGLE, auth: { apiKey: 'key' }, config: {} }),
modelId: 'gemini-2.5-flash',
})).toBeUndefined()
})
@@ -150,7 +145,7 @@ describe('aiUtils.createModelForImages', () => {
describe('what the managed provider is asked to send back', () => {
function settingsFor({ provider, webSearchEnabled = false }: { provider: AIProviderName, webSearchEnabled?: boolean }): Record<string, unknown> | undefined {
- const model = aiUtils.createModel({ provider, auth: { apiKey: 'key' }, config: {}, modelId: 'anthropic/claude-sonnet-5', webSearchEnabled })
+ const model = aiUtils.createModel({ credentials: aiProviderCredentials({ provider, auth: { apiKey: 'key' }, config: {} }), modelId: 'anthropic/claude-sonnet-5', webSearchEnabled })
return (model as unknown as { settings?: Record<string, unknown> }).settings
}
diff --git a/packages/server/worker/src/lib/execute/jobs/ai/ai-cost-reporter.ts b/packages/server/worker/src/lib/execute/jobs/ai/ai-cost-reporter.ts
deleted file mode 100644
index 8fa7957bbc9..00000000000
--- a/packages/server/worker/src/lib/execute/jobs/ai/ai-cost-reporter.ts
+++ /dev/null
@@ -1,66 +0,0 @@
-import { ActivepiecesAiBilling, ActivepiecesAiBillingScope, ActivepiecesAiCall, apId, isNil, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'
-import { activepiecesAiCost, onCallService } from '@activepieces/server-utils'
-import { AiUsageCharge, AiUsageChatContext, AiUsageFlowRunContext, WorkerToApiContract } from '@activepieces/shared'
-import { FastifyBaseLogger } from 'fastify'
-import { workerSettings } from '../../../config/worker-settings'
-
-export function installAiCostReporter({ apiClient, log }: {
- apiClient: WorkerToApiContract
- log: FastifyBaseLogger
-}): void {
- activepiecesAiCost.setReporter(({ billing, provider, modelId, call }) => {
- void tryCatch(() => apiClient.reportAiUsage({
- billing,
- provider,
- modelId,
- idempotencyKey: `ai:${apId()}`,
- usage: chargeFor(call),
- ...spreadIfDefined('generationId', call.generationId),
- ...spreadIfDefined('flowRun', flowRunOf(billing)),
- ...spreadIfDefined('chat', chatOf(billing)),
- })).then(({ error }) => {
- if (isNil(error)) {
- return
- }
- log.error({ error, provider, model: modelId, generationId: call.generationId }, '[aiCostReporter] An AI call went unbilled')
- })
- })
- startUnbilledCallWatch(log)
-}
-
-function startUnbilledCallWatch(log: FastifyBaseLogger): void {
- let lastSeen = 0
- const timer = setInterval(() => {
- const unbilled = activepiecesAiCost.unreportedCallCount()
- if (unbilled <= lastSeen) {
- return
- }
- const since = unbilled - lastSeen
- lastSeen = unbilled
- log.error({ unbilledCallCount: unbilled, sinceLastCheck: since }, '[aiCostReporter] AI calls ran without a charge to bill')
- const { data: settings } = tryCatchSync(() => workerSettings.getSettings())
- void onCallService(log, settings?.PAGE_ONCALL_WEBHOOK).page({
- code: 'AI_CALLS_WENT_UNBILLED',
- message: 'AI calls ran without a charge to bill. Either a managed call reported no cost, or a call was made before the cost reporter was installed.',
- params: { unbilledCallCount: unbilled, sinceLastCheck: since },
- })
- }, UNBILLED_CHECK_INTERVAL_MS)
- timer.unref()
-}
-
-function flowRunOf(billing: ActivepiecesAiBilling): AiUsageFlowRunContext | undefined {
- return billing.scope === ActivepiecesAiBillingScope.PROJECT ? billing.flowRun : undefined
-}
-
-function chatOf(billing: ActivepiecesAiBilling): AiUsageChatContext | undefined {
- return billing.scope === ActivepiecesAiBillingScope.CONVERSATION ? billing.chat : undefined
-}
-
-function chargeFor(call: ActivepiecesAiCall): AiUsageCharge {
- if (call.charge === 'flat-credits') {
- return { type: 'flat-credits', credits: call.credits }
- }
- return { type: 'observed-cost', costUsd: call.costUsd }
-}
-
-const UNBILLED_CHECK_INTERVAL_MS = 5 * 60 * 1000
diff --git a/packages/server/worker/src/lib/execute/jobs/ai/execute-ai.ts b/packages/server/worker/src/lib/execute/jobs/ai/execute-ai.ts
index afb54d8c69f..fa34064c188 100644
--- a/packages/server/worker/src/lib/execute/jobs/ai/execute-ai.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ai/execute-ai.ts
@@ -1,4 +1,4 @@
-import { ActivepiecesAiBilling, ActivepiecesAiBillingScope, AIProviderName, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'
+import { ActivepiecesAiBilling, ActivepiecesAiConsumerSource, AIProviderName, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'
import { aiUtils, FlowStepMetadata } from '@activepieces/server-utils'
import { AiStepAction, ClassifyTextJobData, EngineResponseStatus, ExecuteAiJobData, getEffectiveProviderAndModel, ResolveAiProviderResponse, WorkerJobType } from '@activepieces/shared'
import { generateText, ModelMessage, stepCountIs } from 'ai'
@@ -72,15 +72,14 @@ async function callTheModel({ ctx, data }: { ctx: JobContext, data: ExecuteAiJob
}
async function runTextStep({ data, resolved, flowStep, billing }: { data: ExecuteAiJobData, resolved: ResolveAiProviderResponse, flowStep: FlowStepMetadata, billing: ActivepiecesAiBilling }): Promise<unknown> {
- const { provider, auth, config } = resolved
+ const credentials = resolved
+ const { provider } = credentials
const webSearchEnabled = data.webSearch?.enabled ?? false
const webSearchOptions = data.webSearch?.options
const { provider: effectiveProvider } = getEffectiveProviderAndModel({ provider, model: data.modelId })
- const tools = aiUtils.buildWebSearchToolsOrThrow({ provider, model: data.modelId, auth, webSearchEnabled, options: webSearchOptions })
+ const tools = aiUtils.buildWebSearchToolsOrThrow({ provider, model: data.modelId, webSearchEnabled, options: webSearchOptions })
const model = aiUtils.createModel({
- provider,
- auth,
- config,
+ credentials,
modelId: data.modelId,
flowStep,
billing,
@@ -104,7 +103,7 @@ async function runTextStep({ data, resolved, flowStep, billing }: { data: Execut
function billingFor(data: ExecuteAiJobData): ActivepiecesAiBilling {
return {
- scope: ActivepiecesAiBillingScope.PROJECT,
+ source: ActivepiecesAiConsumerSource.AI_STEP_IN_FLOW,
platformId: data.platformId,
projectId: data.projectId,
flowRun: { flowId: data.flowId, flowRunId: data.flowRunId },
diff --git a/packages/server/worker/src/lib/execute/jobs/ai/extract-structured-data.ts b/packages/server/worker/src/lib/execute/jobs/ai/extract-structured-data.ts
index 32adef7c701..56537cc43ac 100644
--- a/packages/server/worker/src/lib/execute/jobs/ai/extract-structured-data.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ai/extract-structured-data.ts
@@ -12,9 +12,7 @@ export async function extractStructuredData({ data, resolved, flowStep, billing,
files: ResolvedAiFile[]
}): Promise<unknown> {
const model = aiUtils.createModel({
- provider: resolved.provider,
- auth: resolved.auth,
- config: resolved.config,
+ credentials: resolved,
modelId: data.modelId,
flowStep,
billing,
diff --git a/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts b/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts
index c3e5705976d..3d979eb01ce 100644
--- a/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts
@@ -57,7 +57,7 @@ async function getGeneratedImage({ data, resolved, inputImages, flowStep, billin
: prompt,
providerOptions: { [resolvedProvider]: { ...stripLegacyImageField(data.advancedOptions) } } as Parameters<typeof generateImage>[0]['providerOptions'],
})
- activepiecesAiCost.reportFlatCredits({ billing, provider: resolved.provider, modelId: data.modelId })
+ activepiecesAiCost.reportFixedCredits({ billing, provider: resolved.provider, modelId: data.modelId })
return image
})
}
@@ -71,12 +71,12 @@ function createImageCapableModel({ resolved, modelId, flowStep, billing }: {
if (!AI_PROVIDER_CAPABILITIES[resolved.provider].supportsImageGeneration) {
throw new Error(`Provider ${resolved.provider} does not support image models`)
}
- const { provider, auth, config } = resolved
- const imageModel = aiUtils.createModelForImages({ provider, auth, config, modelId, flowStep })
+ const credentials = resolved
+ const imageModel = aiUtils.createModelForImages({ credentials, modelId, flowStep })
if (!isNil(imageModel)) {
return { kind: 'image', model: imageModel }
}
- return { kind: 'language', model: aiUtils.createModel({ provider, auth, config, modelId, flowStep, billing }) }
+ return { kind: 'language', model: aiUtils.createModel({ credentials, modelId, flowStep, billing }) }
}
async function generateImageUsingGenerateText({ model, prompt, inputImages }: {
diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts
index 740a0c233e2..ee569479d54 100644
--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts
@@ -1,4 +1,4 @@
-import { ActivepiecesAiBilling, ActivepiecesAiBillingScope, AIProviderName, ErrorCode, formatPieceError, isNil, isObject, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'
+import { ActivepiecesAiBilling, ActivepiecesAiConsumerSource, AIProviderName, ErrorCode, formatPieceError, isNil, isObject, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'
import { agentAiUtils, aiUtils } from '@activepieces/server-utils'
import { AgentEvent, AgentEventType, AgentKnowledgeBaseTool, AgentMcpTool, AgentOutputField, AgentPhase, AgentPieceTool, AgentResult, AgentRunSource, AgentTool, AgentToolType, EngineResponseStatus, ExecuteAgentRunJobData, MAX_AGENT_TURN_WALL_CLOCK_MS, PersistedAgentMessage, PersistedAgentPart, PersistedAgentRole, ResolvedAgentFlowTool, WorkerJobType } from '@activepieces/shared'
import { createUIMessageStream, generateText, ModelMessage, streamText, ToolSet, toUIMessageStream } from 'ai'
@@ -75,13 +75,13 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
}
progressSequence += 1
const sequence = progressSequence
- void tryCatch(async () => {
+ tryCatch(async () => {
const { error } = await tryCatch(() => ctx.apiClient.updateFlowStepProgress({ conversationId, flowRunId, output, sequence }))
if (!isNil(error) && !warnedOnProgress) {
warnedOnProgress = true
log.warn({ error, flowRun: { id: flowRunId } }, '[executeAgentRun] Could not push step progress; the builder timeline may lag')
}
- })
+ }).catch(() => undefined)
}
const reportFinal = (output: AgentResult) => pushProgress(output)
const reportProgress = (uiParts: PersistedAgentPart[]) =>
@@ -102,7 +102,8 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
...spreadIfDefined('discoveryOnly', discoveryOnly),
})
- const provider = config.provider as AIProviderName
+ const { credentials } = config
+ const { provider } = credentials
runProvider = provider
runModelId = config.modelId
source = config.source
@@ -119,23 +120,23 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
&& !untrackedSearchWouldBeat
&& aiUtils.supportsWebSearch(provider)
const billing: ActivepiecesAiBilling = {
- scope: ActivepiecesAiBillingScope.CONVERSATION,
+ source: ActivepiecesAiConsumerSource.CHAT,
platformId,
projectId: projectId ?? null,
conversationId,
chat: { userId, turnIndex: config.previousUiMessages.length, tier: config.tier.id },
}
const model = aiUtils.createModel({
- provider, auth: config.auth, config: config.providerConfig, modelId: config.modelId,
+ credentials, modelId: config.modelId,
metadata: { platformId, conversationId, runId },
webSearchEnabled: webSearchActive,
billing,
- ownKeyCredit: 'charged-with-the-turn',
+ turnAlreadyCharged: true,
})
const fastModel = aiUtils.createModel({
- provider, auth: config.auth, config: config.providerConfig, modelId: config.fastModelId,
+ credentials, modelId: config.fastModelId,
billing,
- ownKeyCredit: 'charged-with-the-turn',
+ turnAlreadyCharged: true,
})
log.info({ provider, model: { id: config.modelId }, tier: { id: config.tier.id }, dryRun: dryRun ?? false, tavilySearchActive, webSearchActive }, '[executeAgentRun] Chat config loaded')
@@ -185,11 +186,11 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
// timestamp, so a slow-but-live turn is never reclaimed as stale by either the
// client stale-check or the server's getConversationOrThrow stale-recovery.
const sendHeartbeat = () => {
- void tryCatch(() => ctx.apiClient.sendAgentEvent({
+ tryCatch(() => ctx.apiClient.sendAgentEvent({
userId, conversationId, runId,
event: { type: AgentEventType.CHUNK, data: [] },
- }))
- void tryCatch(() => ctx.apiClient.heartbeatAgentConversation({ conversationId, runId }))
+ })).catch(() => undefined)
+ tryCatch(() => ctx.apiClient.heartbeatAgentConversation({ conversationId, runId })).catch(() => undefined)
}
heartbeatInterval = setInterval(sendHeartbeat, HEARTBEAT_INTERVAL_MS)
@@ -199,7 +200,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
const webTools: ToolSet = dryRun ? {} : {
...agentWorkerTools.createWebTools({ taintState }),
...(aiTools.webSearch ? agentWorkerTools.createSearchTools({ webSearch: aiTools.webSearch, taintState }) : {}),
- ...(webSearchActive ? agentWorkerTools.wrapToolsWithTaint({ tools: aiUtils.buildWebSearchTools({ provider, auth: config.auth }), taintState }) : {}),
+ ...(webSearchActive ? agentWorkerTools.wrapToolsWithTaint({ tools: aiUtils.buildWebSearchTools({ provider }), taintState }) : {}),
...(aiTools.webScraping ? agentWorkerTools.createScrapeTools({ scraping: aiTools.webScraping, taintState }) : {}),
...(aiTools.imageGeneration && !discoveryOnly ? agentWorkerTools.createImageTools({
imageGeneration: aiTools.imageGeneration,
@@ -273,7 +274,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
},
}),
onProgress: ({ uiParts, responseMessages }) => {
- void retryWithBackoff({
+ retryWithBackoff({
fn: () => ctx.apiClient.updateAgentProgress({
conversationId,
runId,
@@ -285,7 +286,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
}),
maxAttempts: 2,
log,
- })
+ }).catch(() => undefined)
reportProgress(uiParts)
},
},
@@ -341,7 +342,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge
outputTokens: totalOutputTokens,
...spreadIfDefined('cacheReadTokens', usage?.inputTokenDetails?.cacheReadTokens),
...spreadIfDefined('cacheWriteTokens', usage?.inputTokenDetails?.cacheWriteTokens),
- provider: config.provider,
+ provider: config.credentials.provider,
finishReason: turn.finishReason,
truncatedAfterRetries,
}, 'Chat message completed')
diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-personalization-research.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-personalization-research.ts
index 002e0fa2495..098398efca2 100644
--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-personalization-research.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-personalization-research.ts
@@ -1,4 +1,4 @@
-import { AIProviderName, isNil, tryCatch } from '@activepieces/core-utils'
+import { isNil, tryCatch } from '@activepieces/core-utils'
import { aiUtils } from '@activepieces/server-utils'
import { EngineResponseStatus, ExecutePersonalizationResearchJobData, WorkerJobType } from '@activepieces/shared'
import { FireAndForgetJobResult, JobContext, JobHandler, JobResultKind } from '../../../types'
@@ -74,9 +74,9 @@ async function runResearch({ data, config, progress, log }: {
progress: ProgressFn
log: JobContext['log']
}): Promise<ResearchOutput | null> {
- const provider = config.provider as AIProviderName
+ const { credentials } = config
const fastModel = aiUtils.createModel({
- provider, auth: config.auth, config: config.providerConfig, modelId: config.fastModelId,
+ credentials, modelId: config.fastModelId,
})
if (data.scope === 'user') {
@@ -137,7 +137,7 @@ async function runResearch({ data, config, progress, log }: {
const gathered = searchBlocks ?? []
const digest = gathered.length > 0
? `${groundwork}\n\n${gathered.map((block) => block.block).join('\n\n')}`
- : await fallbackResearch({ provider, auth: config.auth, providerConfig: config.providerConfig, fastModelId: config.fastModelId, companyRef, role, groundwork, log })
+ : await fallbackResearch({ credentials, fastModelId: config.fastModelId, companyRef, role, groundwork, log })
await progress({ phase: 'understanding', message: `Studying how ${homepage?.siteName ?? companyLabel} runs behind the scenes…` })
const profilePromise = generateProfile({ model: fastModel, domain, companyText, digest, role, user: config.user, log })
diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/personalization-generate.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/personalization-generate.ts
index 2f42aac8491..3265c3a0454 100644
--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/personalization-generate.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/personalization-generate.ts
@@ -1,5 +1,6 @@
-import { AIProviderName, isNil, tryCatch } from '@activepieces/core-utils'
+import { isNil, tryCatch } from '@activepieces/core-utils'
import { aiUtils } from '@activepieces/server-utils'
+import { AiProviderCredentials } from '@activepieces/shared'
import { generateObject, generateText, LanguageModel, stepCountIs } from 'ai'
import { z } from 'zod'
import { JobContext } from '../../../types'
@@ -7,10 +8,8 @@ import { CARDS_SCHEMA, MAX_DISPLAY_NAME_CHARS, MAX_USE_CASES, MIN_USE_CASES, Per
import { aOrAn, cleanCards, cleanProfile } from './personalization-shaping'
import { delayWithJitter } from './run-agent-turn'
-export async function fallbackResearch({ provider, auth, providerConfig, fastModelId, companyRef, role, groundwork, log }: {
- provider: AIProviderName
- auth: Record<string, unknown>
- providerConfig: Record<string, unknown>
+export async function fallbackResearch({ credentials, fastModelId, companyRef, role, groundwork, log }: {
+ credentials: AiProviderCredentials
fastModelId: string
companyRef: string
role: string | null
@@ -18,11 +17,12 @@ export async function fallbackResearch({ provider, auth, providerConfig, fastMod
log: JobContext['log']
}): Promise<string> {
const groundworkBlock = groundwork.length > 0 ? groundwork : '(no grounding facts could be gathered)'
+ const { provider } = credentials
if (!aiUtils.supportsWebSearch(provider)) {
return groundworkBlock
}
- const model = aiUtils.createModel({ provider, auth, config: providerConfig, modelId: fastModelId, webSearchEnabled: true })
- const nativeTools = aiUtils.buildWebSearchTools({ provider, auth })
+ const model = aiUtils.createModel({ credentials, modelId: fastModelId, webSearchEnabled: true })
+ const nativeTools = aiUtils.buildWebSearchTools({ provider })
const { data, error } = await tryCatch(() => generateText({
model,
abortSignal: AbortSignal.timeout(FALLBACK_RESEARCH_TIMEOUT_MS),
diff --git a/packages/server/worker/src/lib/worker.ts b/packages/server/worker/src/lib/worker.ts
index e4d190d7ab5..1b8f06385e8 100644
--- a/packages/server/worker/src/lib/worker.ts
+++ b/packages/server/worker/src/lib/worker.ts
@@ -2,7 +2,7 @@ import { createServer } from 'http'
import os from 'os'
import { ActivepiecesError, isNil, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'
import { ACTION_RUN_CACHE_ACTIVE_WINDOW_MS, ACTION_RUN_CACHE_FIRST_SWEEP_DELAY_MS, ACTION_RUN_CACHE_SWEEP_INTERVAL_MS, actionRunCache, cacheUtils, createResolver, createSandboxRuntime, Runtime } from '@activepieces/sandbox'
-import { createLogger, systemUsage, wideEvent } from '@activepieces/server-utils'
+import { aiCostReporter, createLogger, systemUsage, wideEvent } from '@activepieces/server-utils'
import { ApEdition, ApiToWorkerContract, ConsumeJobRequest, createNotifyServer, createRpcClient, EngineResponseStatus, ExecutionMode, JobData, LONG_RUNNING_RPC_METHODS, SandboxInformation, WebsocketServerEvent, WorkerJobType, WorkerMachineHealthcheckRequest, WorkerProps, WorkerSettingsResponse, WorkerToApiContract } from '@activepieces/shared'
import { nanoid } from 'nanoid'
import { io, Socket } from 'socket.io-client'
@@ -11,7 +11,6 @@ import { getApiUrl, system, WorkerSystemProp } from './config/configs'
import { logger } from './config/logger'
import { workerSettings } from './config/worker-settings'
import { getHandler } from './execute/job-registry'
-import { installAiCostReporter } from './execute/jobs/ai/ai-cost-reporter'
import { JobContext, JobResult, JobResultKind } from './execute/types'
import { sandboxConfig } from './runtime/sandbox-config'
import { VERSION_MISMATCH_POLL_PAUSE_MS, versionChecker } from './utils/version-checker'
@@ -72,7 +71,11 @@ export const worker = {
})
const apiClient = createRpcClient<WorkerToApiContract>(socket, rpcTimeoutMsFor)
- installAiCostReporter({ apiClient, log: logger })
+ aiCostReporter.install({
+ log: logger,
+ report: (request) => apiClient.reportAiUsage(request),
+ pageWebhookUrl: () => tryCatchSync(() => workerSettings.getSettings()).data?.PAGE_ONCALL_WEBHOOK,
+ })
socket.on('connect', async () => {
logger.info('Connected to API server via Socket.IO')
diff --git a/packages/server/worker/test/lib/agent-eval/core/llm-judge.ts b/packages/server/worker/test/lib/agent-eval/core/llm-judge.ts
index 18d0ad5e7a6..d9348455ef8 100644
--- a/packages/server/worker/test/lib/agent-eval/core/llm-judge.ts
+++ b/packages/server/worker/test/lib/agent-eval/core/llm-judge.ts
@@ -1,5 +1,5 @@
import { aiUtils } from '@activepieces/server-utils'
-import { AIProviderName } from '@activepieces/core-utils';
+import { aiProviderCredentials, AIProviderName } from '@activepieces/core-utils';
import { generateText } from 'ai'
const JUDGE_SYSTEM = 'You are a strict binary evaluator of an AI assistant transcript. You judge exactly ONE dimension against ONE pass criterion. Be conservative: if the criterion is not clearly met, answer FAIL. Respond with "PASS" or "FAIL" on the first line, then a single short sentence explaining why on the second line. Output nothing else.'
@@ -9,7 +9,7 @@ function createJudge({ provider, modelId, auth }: {
modelId: string
auth: Record<string, unknown>
}): { judge: (params: { dimension: string, rubric: string, transcript: string }) => Promise<JudgeVerdict> } {
- const model = aiUtils.createModel({ provider, auth, config: {}, modelId })
+ const model = aiUtils.createModel({ credentials: aiProviderCredentials({ provider, auth, config: {} }), modelId })
const judge = async ({ dimension, rubric, transcript }: { dimension: string, rubric: string, transcript: string }): Promise<JudgeVerdict> => {
const { text } = await generateText({
diff --git a/packages/server/worker/test/lib/agent-eval/core/runner.ts b/packages/server/worker/test/lib/agent-eval/core/runner.ts
index 64f0ecdfd03..04b3cc601a3 100644
--- a/packages/server/worker/test/lib/agent-eval/core/runner.ts
+++ b/packages/server/worker/test/lib/agent-eval/core/runner.ts
@@ -1,5 +1,5 @@
import { aiUtils } from '@activepieces/server-utils'
-import { tryCatch } from '@activepieces/core-utils';
+import { aiProviderCredentials, tryCatch } from '@activepieces/core-utils';
import { AgentPhase, PersistedAgentPartType } from '@activepieces/shared';
import { hasToolCall, isLoopFinished, ModelMessage, ToolSet } from 'ai'
import { ChatEvalFixture } from './fixture'
@@ -107,7 +107,7 @@ async function runTurn({ fixture, systemPrompt, guides, auth }: { fixture: ChatE
const replay = replayExecutor.create({ recordedToolCalls: fixture.recordedToolCalls })
const phaseState: { phase: AgentPhase } = { phase: 'discovery' }
const tools = buildEvalToolSet({ replay, guides: guides ?? evalPrompts.loadGuides(), phaseState })
- const model = aiUtils.createModel({ provider: fixture.model.provider, auth, config: {}, modelId: fixture.model.modelId })
+ const model = aiUtils.createModel({ credentials: aiProviderCredentials({ provider: fixture.model.provider, auth, config: {} }), modelId: fixture.model.modelId })
const messages: ModelMessage[] = [
...fixture.initialMessages,
...fixture.userTurns.map((content) => ({ role: 'user' as const, content })),