Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix: Activepieces AI credits now only run the models offered in the picker (#15928)

Hazem Adel7e08a33ace73
diff --git a/brain/knowledge/ai-intelligence/ai-providers.md b/brain/knowledge/ai-intelligence/ai-providers.md
index 167ec72035f..5eabcb51f81 100644
--- a/brain/knowledge/ai-intelligence/ai-providers.md
+++ b/brain/knowledge/ai-intelligence/ai-providers.md
@@ -69,6 +69,7 @@ enqueued. `AP_MODEL_TIERS_URL` overrides the URL. The file is published from the
decision 000041. The console publishes both lists and both fast pointers on every Publish, and on its first load
after a file without chat keys it seeds the chat list from the live flow list.
### Gotchas
+- **The managed key only runs models the tiers or the chat allow-list permit, enforced in `aiUtils.createModel`.** The managed provider is OpenRouter underneath, so before this guard a crafted step naming any OpenRouter model ran on our key; the picker hid it but nothing denied it. `assertManagedModelAllowed` (`server/utils/src/ai-utils.ts`) checks `managedChatModelIds()` plus both surfaces' published tier `modelId`s and throws a `VALIDATION` error. Image models are checked too, in `createModel` (`imageGeneration: true`) and `createModelForImages`: only `ACTIVEPIECES_IMAGE_TIERS` model ids, plus the id the chat path passes as `adminChosenImageModelId` (the server-chosen Capabilities model). The worker now calls `modelTierCatalog.warmUp()` at start so a cold worker does not judge a newly published tier against the bundled list.
- **A catalog change takes up to ~2 days to reach a dropdown, through three caches in series.** CDN edge (`--cache-control max-age=3600`, 1h) → the server's in-memory catalog (`CATALOG_TTL_MS`, 24h) → `modelsCache` (flushed by the nightly `cron.schedule('0 0 * * *')`). So "I republished but the UI still shows the old price" is expected, not a bug; restarting the API short-circuits the two in-process layers. The edge TTL is deliberately 1h and not the 604800 that `publish-embed-sdk.yml` uses — that path is version-stamped, ours is a stable key rewritten in place, so a week-long edge cache would pin stale prices for a week.
- **`Sync AI Model Catalog` is the only workflow that proves the CDN S3 credentials still work — a green `Publish Embed SDK` proves nothing.** Both read the same `CDN_S3_ACCESS_KEY_ID` / `CDN_S3_SECRET_ACCESS_KEY` / `CDN_S3_BUCKET` / `CDN_S3_ENDPOINT` secrets, but `publish-embed-sdk.yml` guards its upload job on `needs.check.outputs.publish == 'true'`, which is false whenever the version in `packages/ee/embed-sdk/package.json` is already on the CDN. Its daily cron then finishes in about nine seconds with the upload skipped and the run still marked success, so the key can rot for weeks behind a green tick — that is how the model-catalog cron came to fail with `InvalidAccessKeyId` on three consecutive Mondays while the embed SDK looked healthy. To date a dead key, find the last embed-SDK run whose `publish` job actually ran (`gh run list --workflow=publish-embed-sdk.yml`, look for a duration in minutes rather than seconds) and compare it against the first model-catalog failure. To test a rotated key, dispatch `sync-model-catalog.yml` — it uploads unconditionally.
- **Publishing is an overwrite of one S3 key, and it only ever happens on the Monday cron or a manual `workflow_dispatch`.** Never on merge, deploy or release. There is no versioning and no history: last write wins and the previous contents are gone, which is why the object carries `generatedAt` — `curl -s https://cdn.activepieces.com/ai/model-catalog.json | jq .generatedAt` is the only way to tell how fresh what you are serving is. Editing the generator changes nothing in production until someone dispatches the workflow.
diff --git a/packages/server/utils/src/ai-utils.ts b/packages/server/utils/src/ai-utils.ts
index eaf9c61bc7a..055b39c7646 100644
--- a/packages/server/utils/src/ai-utils.ts
+++ b/packages/server/utils/src/ai-utils.ts
@@ -1,6 +1,6 @@
-import { ActivepiecesAiBilling, aiChargeFor, AIProviderName, isNil, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils';
+import { ActivepiecesAiBilling, aiChargeFor, ActivepiecesError, AIProviderName, ErrorCode, isNil, observedProviderFetch, ProviderOutcomeReporter, spreadIfDefined } from '@activepieces/core-utils';
import { CloudflareGatewayMetadata, createCloudflareGatewayModel, createImageModel, createLanguageModel } from '@activepieces/ai-providers';
-import { AI_PROVIDER_CAPABILITIES, AiProviderCredentials, getEffectiveProviderAndModel } from '@activepieces/shared';
+import { ACTIVEPIECES_IMAGE_TIERS, AI_PROVIDER_CAPABILITIES, AiProviderCredentials, aiProviderUtils, getEffectiveProviderAndModel } from '@activepieces/shared';
import { anthropic } from '@ai-sdk/anthropic'
import { createAzure } from '@ai-sdk/azure'
import { createGoogleGenerativeAI, google } from '@ai-sdk/google'
@@ -10,6 +10,7 @@ import { createOpenRouter, OpenRouterChatSettings } from '@openrouter/ai-sdk-pro
import { EmbeddingModel, generateText, ImageModel, LanguageModel, ToolSet } from 'ai'
import { billedEmbeddingModel, billedLanguageModel } from './activepieces-ai-cost'
import { keyHealthReporterFor } from './ai-provider-key-health'
+import { modelTierCatalog } from './model-tier-catalog'
const DEFAULT_WEB_SEARCH_RESULTS = 5
const MIN_OPENROUTER_WEB_SEARCH_RESULTS = 1
@@ -132,7 +133,30 @@ function openRouterWebSearchResults(options?: WebSearchOptions): number {
)
}
-function createModel({ credentials, modelId, metadata, flowStep, billing, turnAlreadyCharged, openaiResponsesModel = false, webSearchEnabled = false, webSearchOptions, platformId, providerConfigId }: {
+function managedModelIds(): string[] {
+ return [
+ ...aiProviderUtils.managedChatModelIds(),
+ ...(['flow', 'chat'] as const).flatMap((surface) => modelTierCatalog.current(surface).tiers.map((tier) => tier.modelId)),
+ ]
+}
+
+function managedImageModelIds(): string[] {
+ return ACTIVEPIECES_IMAGE_TIERS.map((tier) => tier.modelId)
+}
+
+function assertManagedModelAllowed({ provider, modelId, image, adminChosenModelId }: { provider: AIProviderName, modelId: string, image: boolean, adminChosenModelId?: string }): void {
+ const allowed = image ? [...managedImageModelIds(), ...(isNil(adminChosenModelId) ? [] : [adminChosenModelId])] : managedModelIds()
+ if (provider !== AIProviderName.ACTIVEPIECES || allowed.includes(modelId)) {
+ return
+ }
+ const message = `The model "${modelId}" is not available on Activepieces AI credits. Choose one of the listed models or connect your own AI provider key.`
+ throw new ActivepiecesError({
+ code: ErrorCode.VALIDATION,
+ params: { message },
+ }, message)
+}
+
+function createModel({ credentials, modelId, metadata, flowStep, billing, turnAlreadyCharged, openaiResponsesModel = false, webSearchEnabled = false, webSearchOptions, platformId, providerConfigId, imageGeneration = false, adminChosenImageModelId }: {
credentials: AiProviderCredentials
modelId: string
metadata?: ChatModelMetadata
@@ -144,7 +168,10 @@ function createModel({ credentials, modelId, metadata, flowStep, billing, turnAl
webSearchOptions?: WebSearchOptions
platformId?: string
providerConfigId?: string
+ imageGeneration?: boolean
+ adminChosenImageModelId?: string
}): LanguageModel {
+ assertManagedModelAllowed({ provider: credentials.provider, modelId, image: imageGeneration, ...spreadIfDefined('adminChosenModelId', adminChosenImageModelId) })
const model = buildModel({ credentials, modelId, metadata, flowStep, openaiResponsesModel, webSearchEnabled, webSearchOptions, onOutcome: keyHealthReporterFor({ platformId, providerConfigId }) })
return billedLanguageModel({
model,
@@ -208,11 +235,13 @@ function flowStepMetadataHeaders(flowStep?: FlowStepMetadata): Record<string, st
}
}
-function createModelForImages({ credentials, modelId, flowStep }: {
+function createModelForImages({ credentials, modelId, flowStep, adminChosenImageModelId }: {
credentials: AiProviderCredentials
modelId: string
flowStep?: FlowStepMetadata
+ adminChosenImageModelId?: string
}): ImageModel | undefined {
+ assertManagedModelAllowed({ provider: credentials.provider, modelId, image: true, ...spreadIfDefined('adminChosenModelId', adminChosenImageModelId) })
if (credentials.provider === AIProviderName.CLOUDFLARE_GATEWAY) {
return createCloudflareGatewayModel({
credentials,
diff --git a/packages/server/utils/test/ai-utils-published-tiers.test.ts b/packages/server/utils/test/ai-utils-published-tiers.test.ts
new file mode 100644
index 00000000000..1a198fc7604
--- /dev/null
+++ b/packages/server/utils/test/ai-utils-published-tiers.test.ts
@@ -0,0 +1,42 @@
+import { AIProviderName } from '@activepieces/core-utils'
+import { aiProviderCredentials } from '@activepieces/shared'
+import { beforeEach, describe, expect, it, vi } from 'vitest'
+
+const get = vi.fn()
+
+vi.mock('../src/safe-http', () => ({
+ safeHttp: {
+ get retryingAxios() {
+ return { get }
+ },
+ },
+}))
+
+const PUBLISHED_ONLY_MODEL = 'acme/brand-new-model'
+
+const PUBLISHED = {
+ version: 1,
+ publishedAt: '2026-09-29T00:00:00.000Z',
+ publishedBy: '[email protected]',
+ tiers: [{ id: 'fast', label: 'Fast', modelId: PUBLISHED_ONLY_MODEL, thinkingBudget: 5_000 }],
+ defaultTierId: 'fast',
+}
+
+describe('aiUtils.createModel on the managed key with a published tier file', () => {
+ beforeEach(() => {
+ get.mockReset()
+ process.env['AP_MODEL_TIERS_URL'] = 'https://tiers.test/pricing.json'
+ vi.resetModules()
+ })
+
+ it('runs a model only the published tiers permit once the file is loaded', async () => {
+ get.mockResolvedValue({ data: PUBLISHED })
+ const { modelTierCatalog } = await import('../src/model-tier-catalog')
+ const { aiUtils } = await import('../src/ai-utils')
+ const managed = aiProviderCredentials({ provider: AIProviderName.ACTIVEPIECES, auth: { apiKey: 'sk-managed' }, config: {} })
+
+ await modelTierCatalog.warmUp()
+
+ expect(() => aiUtils.createModel({ credentials: managed, modelId: PUBLISHED_ONLY_MODEL })).not.toThrow()
+ })
+})
diff --git a/packages/server/utils/test/ai-utils.test.ts b/packages/server/utils/test/ai-utils.test.ts
index 7016d3bb66d..f28a66213d8 100644
--- a/packages/server/utils/test/ai-utils.test.ts
+++ b/packages/server/utils/test/ai-utils.test.ts
@@ -1,5 +1,5 @@
import { AIProviderName } from '@activepieces/core-utils'
-import { aiProviderCredentials } from '@activepieces/shared'
+import { ACTIVEPIECES_CHAT_TIERS, ACTIVEPIECES_IMAGE_TIERS, aiProviderCredentials } from '@activepieces/shared'
import { describe, expect, it } from 'vitest'
import { aiUtils, WebSearchOptions } from '../src/ai-utils'
@@ -143,9 +143,39 @@ describe('aiUtils.createModelForImages', () => {
})
})
+describe('aiUtils.createModel on the managed Activepieces key', () => {
+ const managed = aiProviderCredentials({ provider: AIProviderName.ACTIVEPIECES, auth: { apiKey: 'sk-managed' }, config: {} })
+
+ it('refuses a model that no published tier or allow-list entry permits', () => {
+ expect(() => aiUtils.createModel({ credentials: managed, modelId: 'perplexity/sonar-deep-research' }))
+ .toThrow(/not available on Activepieces AI credits/)
+ })
+
+ it('runs a model the tiers permit', () => {
+ expect(() => aiUtils.createModel({ credentials: managed, modelId: ACTIVEPIECES_CHAT_TIERS[0].modelId })).not.toThrow()
+ })
+
+ it('refuses an image model outside the managed image tiers', () => {
+ expect(() => aiUtils.createModelForImages({ credentials: managed, modelId: 'openai/gpt-5-image' }))
+ .toThrow(/not available on Activepieces AI credits/)
+ expect(() => aiUtils.createModel({ credentials: managed, modelId: 'openai/gpt-5-image', imageGeneration: true }))
+ .toThrow(/not available on Activepieces AI credits/)
+ })
+
+ it('runs the managed default image model and an image model the admin chose', () => {
+ expect(() => aiUtils.createModelForImages({ credentials: managed, modelId: ACTIVEPIECES_IMAGE_TIERS[0].modelId })).not.toThrow()
+ expect(() => aiUtils.createModelForImages({ credentials: managed, modelId: 'openai/gpt-5-image', adminChosenImageModelId: 'openai/gpt-5-image' })).not.toThrow()
+ })
+
+ it('leaves a customer-supplied OpenRouter key unrestricted', () => {
+ const own = aiProviderCredentials({ provider: AIProviderName.OPENROUTER, auth: { apiKey: 'sk-own' }, config: {} })
+ expect(() => aiUtils.createModel({ credentials: own, modelId: 'perplexity/sonar-deep-research' })).not.toThrow()
+ })
+})
+
describe('what the managed provider is asked to send back', () => {
function settingsFor({ provider, webSearchEnabled = false }: { provider: AIProviderName, webSearchEnabled?: boolean }): Record<string, unknown> | undefined {
- const model = aiUtils.createModel({ credentials: aiProviderCredentials({ provider, auth: { apiKey: 'key' }, config: {} }), modelId: 'anthropic/claude-sonnet-5', webSearchEnabled })
+ const model = aiUtils.createModel({ credentials: aiProviderCredentials({ provider, auth: { apiKey: 'key' }, config: {} }), modelId: ACTIVEPIECES_CHAT_TIERS[0].modelId, webSearchEnabled })
return (model as unknown as { settings?: Record<string, unknown> }).settings
}
diff --git a/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts b/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts
index 259d9a2e17d..1e141744db2 100644
--- a/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ai/generate-image.ts
@@ -24,7 +24,7 @@ export async function generateImageStep({ ctx, data, resolved, flowStep, billing
return url
}
-export async function getGeneratedImage({ credentials, modelId, prompt, advancedOptions, inputImages, flowStep, billing, turnAlreadyCharged = false, aspectRatio, abortSignal }: {
+export async function getGeneratedImage({ credentials, modelId, prompt, advancedOptions, inputImages, flowStep, billing, turnAlreadyCharged = false, aspectRatio, abortSignal, adminChosenImageModelId }: {
credentials: AiProviderCredentials
modelId: string
prompt: string
@@ -35,8 +35,9 @@ export async function getGeneratedImage({ credentials, modelId, prompt, advanced
turnAlreadyCharged?: boolean
aspectRatio?: `${number}:${number}`
abortSignal?: AbortSignal
+ adminChosenImageModelId?: string
}): Promise<GeneratedImage> {
- const model = createImageCapableModel({ credentials, modelId, flowStep, billing, turnAlreadyCharged })
+ const model = createImageCapableModel({ credentials, modelId, flowStep, billing, turnAlreadyCharged, adminChosenImageModelId })
const { provider: effectiveProvider } = getEffectiveProviderAndModel({ provider: credentials.provider, model: modelId })
const resolvedProvider = effectiveProvider ?? credentials.provider
const hasInputImages = inputImages.length > 0
@@ -71,21 +72,22 @@ export function imageBytesOf(image: GeneratedImage): Buffer {
return !isNil(image.base64) && image.base64.length > 0 ? Buffer.from(image.base64, 'base64') : Buffer.from(image.uint8Array)
}
-function createImageCapableModel({ credentials, modelId, flowStep, billing, turnAlreadyCharged }: {
+function createImageCapableModel({ credentials, modelId, flowStep, billing, turnAlreadyCharged, adminChosenImageModelId }: {
credentials: AiProviderCredentials
modelId: string
flowStep?: FlowStepMetadata
billing: ActivepiecesAiBilling
turnAlreadyCharged: boolean
+ adminChosenImageModelId?: string
}): ImageCapableModel {
if (!AI_PROVIDER_CAPABILITIES[credentials.provider].supportsImageGeneration) {
throw new Error(`Provider ${credentials.provider} does not support image models`)
}
- const imageModel = aiUtils.createModelForImages({ credentials, modelId, flowStep })
+ const imageModel = aiUtils.createModelForImages({ credentials, modelId, flowStep, ...spreadIfDefined('adminChosenImageModelId', adminChosenImageModelId) })
if (!isNil(imageModel)) {
return { kind: 'image', model: imageModel }
}
- return { kind: 'language', model: aiUtils.createModel({ credentials, modelId, flowStep, billing, turnAlreadyCharged }) }
+ return { kind: 'language', model: aiUtils.createModel({ credentials, modelId, flowStep, billing, turnAlreadyCharged, imageGeneration: true, ...spreadIfDefined('adminChosenImageModelId', adminChosenImageModelId) }) }
}
async function generateImageUsingGenerateText({ model, prompt, inputImages, aspectRatio, abortSignal }: {
diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/tools/web-media-tools.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/tools/web-media-tools.ts
index e9c2b3de9d1..edc30ba9545 100644
--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/tools/web-media-tools.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/tools/web-media-tools.ts
@@ -253,6 +253,7 @@ export function providerImageGenerator({ credentials, modelId, billing }: { cred
turnAlreadyCharged: true,
aspectRatio: ASPECT_RATIO_BY_ASPECT[aspectRatio],
abortSignal: signal,
+ adminChosenImageModelId: modelId,
})
return {
bytes: imageBytesOf(image),
diff --git a/packages/server/worker/src/lib/worker.ts b/packages/server/worker/src/lib/worker.ts
index ae2cf31557d..8f226747bec 100644
--- a/packages/server/worker/src/lib/worker.ts
+++ b/packages/server/worker/src/lib/worker.ts
@@ -2,7 +2,7 @@ import { createServer } from 'http'
import os from 'os'
import { ActivepiecesError, isNil, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'
import { ACTION_RUN_CACHE_ACTIVE_WINDOW_MS, ACTION_RUN_CACHE_FIRST_SWEEP_DELAY_MS, ACTION_RUN_CACHE_SWEEP_INTERVAL_MS, actionRunCache, cacheUtils, createResolver, createSandboxRuntime, Runtime } from '@activepieces/sandbox'
-import { aiCostReporter, createLogger, modelCatalog, systemUsage, wideEvent } from '@activepieces/server-utils'
+import { aiCostReporter, createLogger, modelCatalog, modelTierCatalog, systemUsage, wideEvent } from '@activepieces/server-utils'
import { ApEdition, ApiToWorkerContract, ConsumeJobRequest, createNotifyServer, createRpcClient, EngineResponseStatus, ExecutionMode, JobData, LONG_RUNNING_RPC_METHODS, SandboxInformation, WebsocketServerEvent, WorkerJobType, WorkerMachineHealthcheckRequest, WorkerProps, WorkerSettingsResponse, WorkerToApiContract } from '@activepieces/shared'
import { nanoid } from 'nanoid'
import { io, Socket } from 'socket.io-client'
@@ -62,6 +62,7 @@ export const worker = {
async start({ apiUrl, socketUrl, workerToken, withHealthServer = false }: WorkerStartParams): Promise<void> {
versionChecker.assertReleaseReadable()
modelCatalog.load().catch(() => undefined)
+ await modelTierCatalog.warmUp()
const workerGroupId = system.get(WorkerSystemProp.WORKER_GROUP_ID)
const projectWorker = system.getBoolean(WorkerSystemProp.PROJECT_WORKER) ?? true
socket = io(socketUrl.url, {