AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(ai): model dropdowns offer only models a step can call (#15497)
Othman Abu Ajamiehfecc1e0a369c
diff --git a/brain/knowledge/ai-intelligence/ai-providers.md b/brain/knowledge/ai-intelligence/ai-providers.mdindex 9718d93288f..0f4148f782b 100644--- a/brain/knowledge/ai-intelligence/ai-providers.md+++ b/brain/knowledge/ai-intelligence/ai-providers.md@@ -191,6 +191,9 @@ the URL. The file is published from the console in `ap-analytics`; see decision- **The AI Center providers page renders its whole state machine off one query, so every branch needs an `isLoading` gate above it.** `ProvidersTab` derives the empty state, the grouped list *and* the `?config=<id>` detail view from `useAiProviderConfigs()`. While that query is pending, `data` is `undefined` — `configs` is `[]`, so the empty "Connect your first provider" screen paints for a frame, and a hard refresh on a `?config=` URL finds no matching row and falls through to the list before flipping to the detail page. The parent `SuspenseWrapper` does not cover it: the route chunk and the suspense-backed projects collection resolve first, then the component mounts with no providers yet. Any new branch added to this page belongs below the `if (isLoading) return <ProvidersSkeleton />` gate, not beside it.- **A save bar in the AI Center needs a full-height column, not `fixed` and not bare `sticky`.** The page sits inside *two* nested scrollers — `PlatformLayout`'s `flex flex-col flex-1 overflow-auto` and the AI Center's own `flex-1 overflow-auto` column — so `fixed inset-x-0` centres the bar on the viewport, roughly a sidebar-width left of the content column it belongs to. Swapping it to `sticky bottom-4` fixes the centring for free but pins nothing when the content is shorter than the viewport: the bar just lands wherever the content ends, mid-page. The working shape is `min-h-full` + `flex flex-col` on the tab-content wrapper, `flex-1` on the `TabsContent`, `grow` on the panel root, then `sticky bottom-4 mt-auto` on the bar — `mt-auto` handles the short page, `sticky` handles the long one. `min-h-full` resolves only because every ancestor up to `SidebarInset h-full` has a definite height; that chain is what makes the whole thing work, so check it before reusing this elsewhere.- **A chat *tier* is our own idea, so it has no meaning on a provider we ship no curated model list for — resolve it by asking the key what it offers.** `ACTIVEPIECES_CHAT_TIERS` (fast/smart/premium) only maps to a real id through `ALLOWED_CHAT_MODELS_BY_PROVIDER`, which covers Anthropic, OpenAI, Google, Vertex and Activepieces. Everything else — Bedrock, Azure, Mistral, xAI, DeepSeek and the other OpenAI-compatible vendors — has an empty candidate list. The resolver used to invent an id from the tier (strip the vendor prefix, dots to dashes: `anthropic/claude-haiku-4.5` -> `claude-haiku-4-5`), which Bedrock rejected as "The provided model identifier is invalid" (GIT-1840, Pylon 5836). #15355 removed the invention and left a throw in its place, which was worse: every agent turn on those providers died with `ENTITY_NOT_FOUND: this AI provider key allows no text model a chat turn can run on` before the configured model handled anything (Pylon 6069). **The missing ingredient was never a smarter guess, it was the key's own model list.** `agentHelpers.resolveModelId` is now the single resolver for every surface: it tries our catalogue first and, only when that has nothing to say, reads `aiProviderService.listModels` (process-cached, cleared nightly by cron, already filtered by the key's `modelScope`), preferring a model whose id *contains* the tier's `nativeModelId` — Bedrock spells them `eu.anthropic.claude-sonnet-4-6-...`, so a tier still selects the model it names wherever the vendor hosts it, and otherwise degrades to the first text model the way `pickAllowedModel` already does for an admin-listed catalogue. Picking from a list the provider returned is the opposite of the fabrication #15355 removed. **Two traps, both learned the hard way.** First, an agent surface and generic chat are not symmetric: an agent *carries* a real model id (its picker lists `listModels` output unfiltered, precisely because these providers have no `ALLOWED_CHAT_MODELS_BY_PROVIDER` entry) while chat carries only a tier name, because `ChatModelSelector` falls back to tier options when `getCuratedChatModels` returns `undefined`. Second, **being listed proves the key can see a model, not that the account can invoke it** — so where a model is already settled the fast round reuses it rather than choosing again (`getAgentConfig` passes the model the turn resolved; the tool RPC gets the run's own model from the worker, `modelId` on `executePieceTool`). One more trap, found in review on 2026-09-27: a stored name is not the model the run ran on. The worker stamps the resolved *tier id* on an agentless inline run's conversation, so reading it back as a model hands a bare `smart` to a catalogue-less own-key provider, and no rule on the string can save it, because a deployment may be named `smart` too. So nothing is read off the conversation or the agent for the fast round any more: the worker says which model the turn resolved, `modelId` on `ExecutePieceToolRequest` (core-execution), and the API borrows exactly that. The field is required, not optional, because the dispatch gate keeps worker and app on one release, so an older worker never reaches a newer API. Resolving the fast tier independently once picked a Haiku the account could not invoke and failed a turn whose configured Sonnet worked fine. A borrowed model still has to pass the key's current `modelScope`, or resolution falls through to the list, which applies it. Nothing crosses the worker, so a rolling deploy has no version-skew window.+- **Non-chat models are filtered out centrally in `fetchModels`, not per provider strategy.** OpenAI's `/v1/models` returns `whisper-1`, `tts-1`, `gpt-4o-transcribe`, `text-embedding-3-*`, `omni-moderation-latest` and `sora-*`, and every strategy that cannot tell them apart typed them `TEXT` — so they were offered in Ask AI / Summarize / Classify / Extract and died at run time on chat completion (#14496). One `aiProviderUtils.isChatModelId` denylist now runs inside `fetchModels`, which is the single choke point behind both the project route and the admin `configs/:id/models` route, so a new provider is covered without opting in. Three things about it are load-bearing: it tests **only `TEXT`-typed rows**, so image models are untouched; it **never judges a name the customer chose** — admin-typed configs (`'models' in config` — VERTEX, CUSTOM, CLOUDFLARE_GATEWAY) skip it because an admin who typed an id meant it, Azure skips it (`modelIdsAreCustomerNamed` on the strategy) because it lists deployment names like `support-voice-agent` and filters on `deployment.model` itself, and a fine-tune is judged by its base model only (`ft:<base>:<org>:<label>:…`, Azure `<base>.ft-…`) so an org called `voiceflow` or a `content-moderation` label does not hide a working chat model; and `validateConnection` **bypasses `fetchModels` entirely**, so a key that lists nothing but whisper still validates as a working key. The rule is a denylist on purpose — an unknown id is treated as chat, so a self-hoster's own model on a custom endpoint never disappears, at the cost of a new modality leaking until a pattern is added. Because pieces may not import `@activepieces/core-*` (eslint `no-restricted-imports`), the openai piece's `isLLM` reaches the same ruleset through a `packages/pieces/framework/src/index.ts` re-export. Still leaking, knowingly: vendor-branded generative models on the five compat vendors that publish no modality data (`cogview-*`, `wan*`, `music-*`, `video-01`), whose real ids nobody could verify without five paid keys — xAI is handled structurally, see the next bullet.+- **xAI is the one compat vendor asked what its models output, because it is the one that answers.** `listVendorModels` hits `/v1/language-models` for XAI first and keeps a model only when `output_modalities` includes `text`; the OpenAI-shaped `/v1/models` that the other five vendors use carries no modality field at all, which is why `grok-2-image-1212` and the Grok Imagine image/video models were offered as chat. A model that declares no modalities falls back to the id denylist. Every failure mode degrades to the old path rather than emptying the picker: a 404, a changed shape, or an empty `models` array all fall through to `/v1/models`, and because that fallback is what authenticates, a bad key still fails validation with the vendor-prefixed error. The endpoint pair (`/language-models`, `/image-generation-models`) was already known in-repo — `packages/pieces/community/grok-xai` calls both. Adding a vendor to `VENDORS_DECLARING_MODALITIES` is the whole opt-in; do it only with a real `/models` response in hand, not from recollection of a catalogue.+- **Google is filtered on `supportedGenerationMethods`, and that is what keeps Imagen out of the image picker.** A model survives only if the field is absent or includes `generateContent`, which is the only way to exclude `aqa` (`generateAnswer`) and `gemini-embedding-001` (`embedContent`) — no name pattern catches them. The side effect is deliberate: `imagen-*` was classified `IMAGE` (because `'imagen'.includes('image')`) and offered in Generate Image, but the AI piece reaches Google image generation through `generateText`/`:generateContent` (there is no `google` case in `buildNativeImageModel`), and Imagen only answers `:predict` — so those options always failed at run time. Gemini image models do expose `generateContent` and are unaffected; Vertex is where Imagen actually works, and it is admin-typed so the filter never sees it.- `packages/server/api/src/app/ai/` — backend module: provider controller, service, entity, module, plus the sibling ai-tool-config files- `packages/server/api/src/app/ai/providers/` — per-vendor strategies keyed by `AIProviderName`- `packages/server/api/src/app/platform/billing-provider.ts` — `assertCreditsAndAppSumoNotExceeded` credit gatediff --git a/bun.lock b/bun.lockindex fbed33b8620..e86d7eb9bea 100644--- a/bun.lock+++ b/bun.lock@@ -152,7 +152,7 @@},"packages/core/piece-types": {"name": "@activepieces/core-piece-types",- "version": "0.11.2",+ "version": "0.12.0","dependencies": {"@activepieces/core-utils": "workspace:*","tslib": "2.6.2",@@ -6552,7 +6552,7 @@},"packages/pieces/community/openai": {"name": "@activepieces/piece-openai",- "version": "0.11.0",+ "version": "0.11.1","dependencies": {"@activepieces/core-piece-types": "workspace:*","@activepieces/core-utils": "workspace:*",diff --git a/packages/core/piece-types/package.json b/packages/core/piece-types/package.jsonindex 674dd6f8e4e..16b520733a8 100644--- a/packages/core/piece-types/package.json+++ b/packages/core/piece-types/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/core-piece-types",- "version": "0.11.2",+ "version": "0.12.0","type": "commonjs","main": "./dist/src/index.js","scripts": {diff --git a/packages/core/piece-types/src/lib/ai-providers.test.ts b/packages/core/piece-types/src/lib/ai-providers.test.tsindex 10532834a1b..f8bfb92ba49 100644--- a/packages/core/piece-types/src/lib/ai-providers.test.ts+++ b/packages/core/piece-types/src/lib/ai-providers.test.ts@@ -148,3 +148,99 @@ describe('tier native model ids', () => {}})})++describe('aiProviderUtils.isChatModelId', () => {+ const isChat = (modelId: string) => aiProviderUtils.isChatModelId({ modelId })++ it.each([+ 'whisper-1',+ 'canary-whisper',+ 'tts-1',+ 'tts-1-hd-1106',+ 'gpt-4o-mini-tts',+ 'gpt-4o-transcribe',+ 'qwen-tts',+ ])('rejects %s, which speaks audio rather than chat', (modelId) => {+ expect(isChat(modelId)).toBe(false)+ })++ it.each([+ 'text-embedding-3-small',+ 'text-embedding-3-large',+ 'text-embedding-v3',+ 'gemini-embedding-001',+ 'embedding-3',+ 'bge-reranker-v2-m3',+ ])('rejects %s, which returns vectors the chat actions cannot read', (modelId) => {+ expect(isChat(modelId)).toBe(false)+ })++ it.each([+ 'omni-moderation-latest',+ 'text-moderation-stable',+ 'sora-2',+ 'sora',+ 'codex-mini-latest',+ 'computer-use-preview',+ 'babbage-002',+ 'davinci-002',+ 'gpt-4o-realtime-preview',+ 'gpt-4o-audio-preview',+ 'glm-4-voice',+ 'speech-01-turbo',+ ])('rejects %s, which needs an endpoint other than chat completion', (modelId) => {+ expect(isChat(modelId)).toBe(false)+ })++ it.each([+ 'gpt-image-1',+ 'gpt-image-2',+ 'dall-e-3',+ 'dall-e-2',+ 'chatgpt-image-latest',+ ])('rejects %s, so an image model never lands in a text dropdown', (modelId) => {+ expect(isChat(modelId)).toBe(false)+ })++ it.each([+ 'gpt-4o',+ 'gpt-4.1-mini',+ 'o3',+ 'claude-sonnet-4-6',+ 'gemini-2.5-pro',+ 'deepseek-chat',+ 'kimi-k2',+ ])('accepts %s', (modelId) => {+ expect(isChat(modelId)).toBe(true)+ })++ it.each([+ 'ft:gpt-4o-2024-08-06:acme:support:9xYz',+ 'ft:gpt-4o-2024-08-06:acme:content-moderation:9xYz',+ 'ft:gpt-4o-mini-2024-07-18:voiceflow::AbCd',+ 'ft:gpt-4o-mini-2024-07-18:acme:tts-helper:AbCd',+ 'ft:gpt-4o-mini-2024-07-18:personal::AbCd:ckpt-step-100',+ 'ft:open-mistral-7b:voice-bot:20240514:7e773925',+ 'gpt-35-turbo-0613.ft-b044a9d3cf9c4228b5d393567f693b83',+ 'gpt-4o-mini-2024-07-18.ft-0ab3f80e-voice-agent',+ ])('accepts the fine-tune %s, judged by its base model rather than the name its owner chose', (modelId) => {+ expect(isChat(modelId)).toBe(true)+ })++ it('rejects a fine-tune whose base model cannot chat', () => {+ expect(isChat('ft:babbage-002:acme::AbCd')).toBe(false)+ })++ it('accepts a model id nobody has seen, so a self-hoster keeps their own model', () => {+ expect(isChat('my-company-llm-v2')).toBe(true)+ })++ it('accepts an image-reading chat model, which the bare image rule used to eat', () => {+ expect(isChat('gpt-4o-image-input')).toBe(true)+ })++ it('ignores case and surrounding space, so a provider echoing an odd id still filters', () => {+ expect(isChat(' WHISPER-1 ')).toBe(false)+ expect(isChat(' GPT-4O ')).toBe(true)+ })+})diff --git a/packages/core/piece-types/src/lib/ai-providers.ts b/packages/core/piece-types/src/lib/ai-providers.tsindex eb7480dfa7e..9a3c097b1f6 100644--- a/packages/core/piece-types/src/lib/ai-providers.ts+++ b/packages/core/piece-types/src/lib/ai-providers.ts@@ -304,6 +304,67 @@ function isCuratedChatModelId({ modelId }: { modelId: string }): boolean {return curatedChatModelIds().includes(modelId)}+function isChatModelId({ modelId }: { modelId: string }): boolean {+ const baseModelId = fineTuneBaseModelId({ modelId: modelId.trim().toLowerCase() })+ if (NON_CHAT_MODEL_IDS.includes(baseModelId)) {+ return false+ }+ if (NON_CHAT_MODEL_ID_PREFIXES.some((prefix) => baseModelId.startsWith(prefix))) {+ return false+ }+ if (NON_CHAT_MODEL_ID_FRAGMENTS.some((fragment) => baseModelId.includes(fragment))) {+ return false+ }+ const idTokens = baseModelId.split(MODEL_ID_TOKEN_SEPARATOR)+ return !NON_CHAT_MODEL_ID_TOKENS.some((token) => idTokens.includes(token))+}++function fineTuneBaseModelId({ modelId }: { modelId: string }): string {+ if (modelId.startsWith(FINE_TUNE_PREFIX)) {+ return modelId.split(FINE_TUNE_SEGMENT_SEPARATOR)[1] ?? modelId+ }+ const azureFineTuneStart = modelId.indexOf(AZURE_FINE_TUNE_INFIX)+ return azureFineTuneStart > 0 ? modelId.slice(0, azureFineTuneStart) : modelId+}++const FINE_TUNE_PREFIX = 'ft:'++const FINE_TUNE_SEGMENT_SEPARATOR = ':'++const AZURE_FINE_TUNE_INFIX = '.ft-'++const NON_CHAT_MODEL_IDS = ['babbage-002', 'davinci-002', 'sora']++const NON_CHAT_MODEL_ID_PREFIXES = [+ 'text-embedding-',+ 'text-moderation-',+ 'omni-moderation-',+ 'tts-',+ 'whisper-',+ 'dall-e-',+ 'sora-',+ 'computer-use-',+ 'codex-',+ 'gpt-image-',+ 'chatgpt-image-',+]++const NON_CHAT_MODEL_ID_FRAGMENTS = [+ 'realtime',+ 'audio',+ 'transcribe',+ 'whisper',+ 'embed',+ 'rerank',+ 'moderation',+ 'speech',+ 'voice',+]++const NON_CHAT_MODEL_ID_TOKENS = ['tts', 'asr']++const MODEL_ID_TOKEN_SEPARATOR = /[-_.:/]/+const DEFAULT_MAX_CONTEXT_TOKENS = 128_000const PROVIDER_MAX_CONTEXT_TOKENS: Partial<Record<AIProviderName, number>> = {@@ -422,6 +483,7 @@ export const aiProviderUtils = {managedChatModelIds,isManagedChatModelId,canDisableReasoning,+ isChatModelId,}export const AI_PROVIDER_ENTITY_TYPES = {diff --git a/packages/pieces/community/openai/package.json b/packages/pieces/community/openai/package.jsonindex d2409c17f3b..29446269847 100644--- a/packages/pieces/community/openai/package.json+++ b/packages/pieces/community/openai/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/piece-openai",- "version": "0.11.0",+ "version": "0.11.1","main": "./dist/src/index.js","types": "./dist/src/index.d.ts","dependencies": {diff --git a/packages/pieces/community/openai/src/lib/common/common.ts b/packages/pieces/community/openai/src/lib/common/common.tsindex 2e1c010dd44..5635d11ed42 100644--- a/packages/pieces/community/openai/src/lib/common/common.ts+++ b/packages/pieces/community/openai/src/lib/common/common.ts@@ -1,3 +1,4 @@+import { aiProviderUtils } from '@activepieces/pieces-framework';import { encoding_for_model } from 'tiktoken';export const baseUrl = 'https://api.openai.com/v1';@@ -205,46 +206,5 @@ export const modelTokenLimit = (model: string) => {}};-// Models that aren't chat-completion-capable text LLMs and should be filtered-// out of the chat-model dropdowns (ask_chatgpt, extract-structured-data, etc.).-const notLLMExactIds = [- 'gpt-4o-realtime-preview-2024-10-01',- 'gpt-4o-realtime-preview',- 'babbage-002',- 'davinci-002',- 'tts-1-hd-1106',- 'whisper-1',- 'canary-whisper',- 'canary-tts',- 'tts-1',- 'tts-1-hd',- 'tts-1-1106',- 'dall-e-3',- 'dall-e-2',- 'gpt-image-1',- 'gpt-image-2',-];--const notLLMPrefixes = [- 'text-embedding-',- 'text-moderation-',- 'omni-moderation-',- 'tts-',- 'whisper-',- 'dall-e-',- 'sora-',- 'computer-use-',- 'codex-',-];--export const isLLM = (modelId: string): boolean => {- if (notLLMExactIds.includes(modelId)) return false;- if (notLLMPrefixes.some((p) => modelId.startsWith(p))) return false;- if (modelId.includes('realtime')) return false;- if (modelId.includes('audio')) return false;- if (modelId.includes('transcribe')) return false;- if (modelId.includes('image')) return false;- return true;-};--export const notLLMs = notLLMExactIds;+export const isLLM = (modelId: string): boolean =>+ aiProviderUtils.isChatModelId({ modelId });diff --git a/packages/pieces/framework/src/index.ts b/packages/pieces/framework/src/index.tsindex 10026dfc3f7..4fcfb2a394e 100644--- a/packages/pieces/framework/src/index.ts+++ b/packages/pieces/framework/src/index.ts@@ -110,6 +110,7 @@ export {ACTIVEPIECES_CHAT_TIERS,ACTIVEPIECES_IMAGE_TIERS,DEFAULT_CHAT_TIER_ID,+ aiProviderUtils,} from '@activepieces/core-piece-types';export type {McpAuthConfig,diff --git a/packages/server/api/src/app/ai/ai-provider-service.ts b/packages/server/api/src/app/ai/ai-provider-service.tsindex 940eac9eed6..52482bdfcd8 100644--- a/packages/server/api/src/app/ai/ai-provider-service.ts+++ b/packages/server/api/src/app/ai/ai-provider-service.ts@@ -1,6 +1,6 @@import { ActivepiecesError, AiProviderKeyStatus, AIProviderName, apId, classifyProviderOutcome, ErrorCode, isNil, PlatformId, ProviderOutcomeSignal, spreadIfDefined, spreadIfNotUndefined, toProviderOutcomeSignal, tryCatch, unique } from '@activepieces/core-utils'import { modelCatalog, modelTierCatalog } from '@activepieces/server-utils'-import { ActivePiecesProviderAuthConfig, AI_PROVIDER_ENTITY_TYPES, AIProviderAuthConfig, AIProviderConfig, aiProviderCredentials, AIProviderModel, AiProviderProjectScope, AIProviderWithoutSensitiveData, CreateAIProviderRequest, GetProviderConfigResponse, ProjectAIProvider, UpdateAIProviderRequest } from '@activepieces/shared'+import { ActivePiecesProviderAuthConfig, AI_PROVIDER_ENTITY_TYPES, AIProviderAuthConfig, AIProviderConfig, aiProviderCredentials, AIProviderModel, AIProviderModelType, AiProviderProjectScope, aiProviderUtils, AIProviderWithoutSensitiveData, CreateAIProviderRequest, GetProviderConfigResponse, ProjectAIProvider, UpdateAIProviderRequest } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import cron from 'node-cron'import { repoFactory } from '../core/db/repo-factory'@@ -440,7 +440,8 @@ async function fetchModels({ aiProvider, platformId, log }: { aiProvider: AIProvthrow error}const catalog = await modelCatalog.load()- modelsCache.set(cacheKey, data.map(model => ({+ const offerableModels = appliesChatModelIdRule({ provider, config }) ? data.filter(model => model.type !== AIProviderModelType.TEXT || aiProviderUtils.isChatModelId({ modelId: model.id })) : data+ modelsCache.set(cacheKey, offerableModels.map(model => ({id: model.id,name: model.name,type: model.type,@@ -450,6 +451,10 @@ async function fetchModels({ aiProvider, platformId, log }: { aiProvider: AIProvreturn modelsCache.get(cacheKey)!}+function appliesChatModelIdRule({ provider, config }: { provider: AIProviderName, config: AIProviderConfig }): boolean {+ return !('models' in config) && aiProviders[provider].modelIdsAreCustomerNamed !== true+}+async function decryptRowAuth({ aiProvider, platformId }: { aiProvider: AIProviderSchema, platformId: PlatformId }): Promise<AIProviderAuthConfig> {const auth = await encryptUtils.decryptObject<AIProviderAuthConfig>(aiProvider.auth)if (aiProvider.provider === AIProviderName.ACTIVEPIECES) {diff --git a/packages/server/api/src/app/ai/providers/ai-provider.ts b/packages/server/api/src/app/ai/providers/ai-provider.tsindex ec726e72050..ee08f3d453c 100644--- a/packages/server/api/src/app/ai/providers/ai-provider.ts+++ b/packages/server/api/src/app/ai/providers/ai-provider.ts@@ -5,6 +5,7 @@ export const MODEL_LIST_TIMEOUT_MS = 15_000export type AIProviderStrategy<T extends AIProviderAuthConfig, C extends AIProviderConfig> = {name: string+ modelIdsAreCustomerNamed?: booleanlistModels(authConfig: T, config: C): Promise<AIProviderModel[]>validateConnection(authConfig: T, config: C, log: FastifyBaseLogger): Promise<void>}diff --git a/packages/server/api/src/app/ai/providers/azure-provider.ts b/packages/server/api/src/app/ai/providers/azure-provider.tsindex 07cbfe55f05..501161ae457 100644--- a/packages/server/api/src/app/ai/providers/azure-provider.ts+++ b/packages/server/api/src/app/ai/providers/azure-provider.ts@@ -1,10 +1,11 @@import { httpClient, HttpMethod } from '@activepieces/pieces-common'-import { AIProviderModel, AIProviderModelType, AzureProviderAuthConfig, AzureProviderConfig } from '@activepieces/shared'+import { AIProviderModel, AIProviderModelType, aiProviderUtils, AzureProviderAuthConfig, AzureProviderConfig, isNil } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import { AIProviderStrategy } from './ai-provider'export const azureProvider: AIProviderStrategy<AzureProviderAuthConfig, AzureProviderConfig> = {name: 'Azure OpenAI',+ modelIdsAreCustomerNamed: true,async validateConnection(authConfig: AzureProviderAuthConfig, config: AzureProviderConfig, _log: FastifyBaseLogger): Promise<void> {await azureProvider.listModels(authConfig, config)},@@ -27,16 +28,23 @@ export const azureProvider: AIProviderStrategy<AzureProviderAuthConfig, AzureProconst { data } = res.body- return data.map((deployment: AzureDeployment) => ({- id: deployment.id,- name: deployment.id,- type: AIProviderModelType.TEXT,- }))+ return data+ .filter((deployment: AzureDeployment) => deploysChatModel(deployment))+ .map((deployment: AzureDeployment) => ({+ id: deployment.id,+ name: deployment.id,+ type: AIProviderModelType.TEXT,+ }))},}+function deploysChatModel(deployment: AzureDeployment): boolean {+ return isNil(deployment.model) || aiProviderUtils.isChatModelId({ modelId: deployment.model })+}+const AZURE_DEPLOYMENTS_API_VERSION = '2023-03-15-preview'type AzureDeployment = {id: string+ model?: string}diff --git a/packages/server/api/src/app/ai/providers/google-provider.ts b/packages/server/api/src/app/ai/providers/google-provider.tsindex 93cfd880765..b1046ea3e8f 100644--- a/packages/server/api/src/app/ai/providers/google-provider.ts+++ b/packages/server/api/src/app/ai/providers/google-provider.ts@@ -1,5 +1,5 @@import { httpClient, HttpMethod } from '@activepieces/pieces-common'-import { AIProviderModel, AIProviderModelType, GoogleProviderAuthConfig, GoogleProviderConfig } from '@activepieces/shared'+import { AIProviderModel, AIProviderModelType, GoogleProviderAuthConfig, GoogleProviderConfig, isNil } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import { AIProviderStrategy, MODEL_LIST_TIMEOUT_MS } from './ai-provider'@@ -18,21 +18,30 @@ export const googleProvider: AIProviderStrategy<GoogleProviderAuthConfig, Google'Content-Type': 'application/json',},})- return res.body.models.map((model: GoogleModel) => ({- id: stripModelsPrefix(model.name),- name: model.displayName,- type: model.name.includes('image') ? AIProviderModelType.IMAGE : AIProviderModelType.TEXT,- }))+ return res.body.models+ .filter((model: GoogleModel) => supportsGenerateContent(model))+ .map((model: GoogleModel) => ({+ id: stripModelsPrefix(model.name),+ name: model.displayName,+ type: model.name.includes('image') ? AIProviderModelType.IMAGE : AIProviderModelType.TEXT,+ }))},}const GOOGLE_MODEL_PREFIX = 'models/'+const GENERATE_CONTENT_METHOD = 'generateContent'+function stripModelsPrefix(modelName: string): string {return modelName.startsWith(GOOGLE_MODEL_PREFIX) ? modelName.slice(GOOGLE_MODEL_PREFIX.length) : modelName}+function supportsGenerateContent(model: GoogleModel): boolean {+ return isNil(model.supportedGenerationMethods) || model.supportedGenerationMethods.includes(GENERATE_CONTENT_METHOD)+}+type GoogleModel = {name: stringdisplayName: string+ supportedGenerationMethods?: string[]}\ No newline at end of filediff --git a/packages/server/api/src/app/ai/providers/openai-compatible-vendor.ts b/packages/server/api/src/app/ai/providers/openai-compatible-vendor.tsindex d7dd38d4071..485091111d6 100644--- a/packages/server/api/src/app/ai/providers/openai-compatible-vendor.ts+++ b/packages/server/api/src/app/ai/providers/openai-compatible-vendor.ts@@ -1,6 +1,7 @@import { OPENAI_COMPATIBLE_VENDOR_BASE_URLS, OpenAiCompatibleVendor } from '@activepieces/core-piece-types'+import { AIProviderName } from '@activepieces/core-utils'import { safeHttp } from '@activepieces/server-utils'-import { AIProviderModel, AIProviderModelType, BaseAIProviderAuthConfig, isNil, OpenAiCompatibleVendorConfig, tryCatch } from '@activepieces/shared'+import { AIProviderModel, AIProviderModelType, aiProviderUtils, BaseAIProviderAuthConfig, isNil, OpenAiCompatibleVendorConfig, tryCatch } from '@activepieces/shared'import { AIProviderStrategy } from './ai-provider'export function openAiCompatibleVendor({ name, provider }: {@@ -23,15 +24,18 @@ async function listVendorModels({ authConfig, provider, name }: {provider: OpenAiCompatibleVendorname: string}): Promise<AIProviderModel[]> {- const baseUrl = OPENAI_COMPATIBLE_VENDOR_BASE_URLS[provider]+ const declaredModels = VENDORS_DECLARING_MODALITIES.includes(provider)+ ? await listModelsByDeclaredModality({ authConfig, provider })+ : undefined+ if (!isNil(declaredModels)) {+ return declaredModels+ }+const { data: response, error } = await tryCatch(() => safeHttp.axios.request<OpenAiCompatibleModelsResponse>({method: 'GET',- url: `${baseUrl.replace(/\/+$/, '')}/models`,+ url: vendorEndpoint({ provider, path: 'models' }),timeout: REQUEST_TIMEOUT_MS,- headers: {- 'Authorization': `Bearer ${authConfig.apiKey}`,- 'Content-Type': 'application/json',- },+ headers: vendorHeaders({ authConfig }),}))if (!isNil(error) || isNil(response)) {@@ -45,8 +49,63 @@ async function listVendorModels({ authConfig, provider, name }: {}))}+async function listModelsByDeclaredModality({ authConfig, provider }: {+ authConfig: BaseAIProviderAuthConfig+ provider: OpenAiCompatibleVendor+}): Promise<AIProviderModel[] | undefined> {+ const { data: response, error } = await tryCatch(() => safeHttp.axios.request<VendorLanguageModelsResponse>({+ method: 'GET',+ url: vendorEndpoint({ provider, path: 'language-models' }),+ timeout: REQUEST_TIMEOUT_MS,+ headers: vendorHeaders({ authConfig }),+ }))++ const declaredModels = response?.data.models+ if (!isNil(error) || isNil(declaredModels) || declaredModels.length === 0) {+ return undefined+ }++ return declaredModels+ .filter((model) => outputsText(model))+ .map((model) => ({+ id: model.id,+ name: model.id,+ type: AIProviderModelType.TEXT,+ }))+}++function outputsText(model: VendorLanguageModel): boolean {+ return isNil(model.output_modalities)+ ? aiProviderUtils.isChatModelId({ modelId: model.id })+ : model.output_modalities.includes(TEXT_OUTPUT_MODALITY)+}++function vendorEndpoint({ provider, path }: { provider: OpenAiCompatibleVendor, path: string }): string {+ return `${OPENAI_COMPATIBLE_VENDOR_BASE_URLS[provider].replace(/\/+$/, '')}/${path}`+}++function vendorHeaders({ authConfig }: { authConfig: BaseAIProviderAuthConfig }): Record<string, string> {+ return {+ 'Authorization': `Bearer ${authConfig.apiKey}`,+ 'Content-Type': 'application/json',+ }+}+const REQUEST_TIMEOUT_MS = 15_000+const TEXT_OUTPUT_MODALITY = 'text'++const VENDORS_DECLARING_MODALITIES: OpenAiCompatibleVendor[] = [AIProviderName.XAI]+type OpenAiCompatibleModelsResponse = {data?: { id: string }[]}++type VendorLanguageModel = {+ id: string+ output_modalities?: string[]+}++type VendorLanguageModelsResponse = {+ models?: VendorLanguageModel[]+}diff --git a/packages/server/api/test/integration/ce/ai-provider/chat-model-filter.test.ts b/packages/server/api/test/integration/ce/ai-provider/chat-model-filter.test.tsnew file mode 100644index 00000000000..8f331e1c9b4--- /dev/null+++ b/packages/server/api/test/integration/ce/ai-provider/chat-model-filter.test.ts@@ -0,0 +1,119 @@+import { AIProviderName } from '@activepieces/core-utils'+import { AIProviderModelType } from '@activepieces/shared'+import { FastifyInstance } from 'fastify'+import { StatusCodes } from 'http-status-codes'+import { vi } from 'vitest'+import { mockAndSaveAIProvider } from '../../../helpers/mocks'+import { createTestContext, TestContext } from '../../../helpers/test-context'+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'++const { mockSendRequest } = vi.hoisted(() => ({ mockSendRequest: vi.fn() }))++vi.mock('@activepieces/pieces-common', async (importOriginal) => {+ const original = await importOriginal<typeof import('@activepieces/pieces-common')>()+ return {+ ...original,+ httpClient: { ...original.httpClient, sendRequest: mockSendRequest },+ }+})++let app: FastifyInstance | null = null+let ctx: TestContext++beforeAll(async () => {+ app = await setupTestEnvironment({ fresh: true })+})++afterAll(async () => {+ await teardownTestEnvironment()+})++beforeEach(async () => {+ ctx = await createTestContext(app!)+ mockSendRequest.mockReset()+})++describe('GET /v1/ai-providers/:provider/models', () => {+ it('offers only the openai models a chat step can call', async () => {+ mockSendRequest.mockResolvedValue({+ body: {+ data: [+ { id: 'gpt-4o' },+ { id: 'ft:gpt-4o-2024-08-06:acme:support:9xYz' },+ { id: 'ft:gpt-4o-2024-08-06:acme:content-moderation:9xYz' },+ { id: 'gpt-image-1' },+ { id: 'chatgpt-image-latest' },+ { id: 'whisper-1' },+ { id: 'tts-1' },+ { id: 'gpt-4o-mini-tts' },+ { id: 'gpt-4o-transcribe' },+ { id: 'text-embedding-3-small' },+ { id: 'omni-moderation-latest' },+ { id: 'sora-2' },+ ].map((model) => ({ ...model, object: 'model', created: 0, owned_by: 'openai' })),+ },+ })+ await mockAndSaveAIProvider({+ platformId: ctx.platform.id,+ provider: AIProviderName.OPENAI,+ displayName: 'OpenAI',+ })++ const response = await ctx.get('/v1/ai-providers/openai/models', { projectId: ctx.project.id })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ const models: { id: string, type: string }[] = response?.json()+ expect(models.map((model) => model.id)).toEqual([+ 'gpt-4o',+ 'ft:gpt-4o-2024-08-06:acme:support:9xYz',+ 'ft:gpt-4o-2024-08-06:acme:content-moderation:9xYz',+ 'gpt-image-1',+ ])+ expect(models.find((model) => model.id === 'gpt-image-1')?.type).toBe(AIProviderModelType.IMAGE)+ })++ it('offers an azure deployment by the model it runs, whatever the customer named it', async () => {+ mockSendRequest.mockResolvedValue({+ body: {+ data: [+ { id: 'support-voice-agent', model: 'gpt-4o', status: 'succeeded' },+ { id: 'realtime-support-bot', model: 'gpt-4.1', status: 'succeeded' },+ { id: 'prod-1', model: 'whisper', status: 'succeeded' },+ { id: 'prod-2', model: 'text-embedding-3-small', status: 'succeeded' },+ ],+ },+ })+ await mockAndSaveAIProvider({+ platformId: ctx.platform.id,+ provider: AIProviderName.AZURE,+ displayName: 'Azure',+ config: { resourceName: 'my-resource' },+ })++ const response = await ctx.get(`/v1/ai-providers/${AIProviderName.AZURE}/models`, { projectId: ctx.project.id })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(response?.json().map((model: { id: string }) => model.id)).toEqual(['support-voice-agent', 'realtime-support-bot'])+ })++ it('keeps every model an admin typed by hand, whatever it is for', async () => {+ await mockAndSaveAIProvider({+ platformId: ctx.platform.id,+ provider: AIProviderName.CUSTOM,+ displayName: 'Self-hosted gateway',+ config: {+ baseUrl: 'https://gateway.example.com/v1',+ apiKeyHeader: 'Authorization',+ models: [+ { modelId: 'my-company-llm-v2', modelName: 'House LLM', modelType: AIProviderModelType.TEXT },+ { modelId: 'whisper-1', modelName: 'Whisper', modelType: AIProviderModelType.TEXT },+ ],+ },+ })++ const response = await ctx.get(`/v1/ai-providers/${AIProviderName.CUSTOM}/models`, { projectId: ctx.project.id })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(response?.json().map((model: { id: string }) => model.id)).toEqual(['my-company-llm-v2', 'whisper-1'])+ })+})diff --git a/packages/server/api/test/unit/app/ai/providers/azure-provider.test.ts b/packages/server/api/test/unit/app/ai/providers/azure-provider.test.tsindex f7c7233c327..79e1ae777bd 100644--- a/packages/server/api/test/unit/app/ai/providers/azure-provider.test.ts+++ b/packages/server/api/test/unit/app/ai/providers/azure-provider.test.ts@@ -32,6 +32,35 @@ describe('azureProvider.listModels', () => {])})+ it('judges a deployment by the model it runs, not the name the customer gave it', async () => {+ mockSendRequest.mockResolvedValue({+ body: {+ data: [+ { id: 'support-voice-agent', model: 'gpt-4o', status: 'succeeded', object: 'deployment' },+ { id: 'content-moderation-gpt4o', model: 'gpt-4.1', status: 'succeeded', object: 'deployment' },+ { id: 'prod-1', model: 'whisper', status: 'succeeded', object: 'deployment' },+ { id: 'prod-2', model: 'text-embedding-3-small', status: 'succeeded', object: 'deployment' },+ { id: 'prod-3', model: 'dall-e-3', status: 'succeeded', object: 'deployment' },+ { id: 'prod-4', model: 'sora', status: 'succeeded', object: 'deployment' },+ ],+ },+ })++ const models = await azureProvider.listModels({ apiKey: 'test-key' }, { resourceName: 'my-resource' })++ expect(models.map((model) => model.id)).toEqual(['support-voice-agent', 'content-moderation-gpt4o'])+ })++ it('keeps a deployment that does not say which model it runs, so an unfamiliar response is not emptied', async () => {+ mockSendRequest.mockResolvedValue({+ body: { data: [{ id: 'prod-voice', status: 'succeeded', object: 'deployment' }] },+ })++ const models = await azureProvider.listModels({ apiKey: 'test-key' }, { resourceName: 'my-resource' })++ expect(models.map((model) => model.id)).toEqual(['prod-voice'])+ })+it('lists deployments with the legacy api-version even when a newer one is configured', async () => {await azureProvider.listModels({ apiKey: 'test-key' }, { resourceName: 'my-resource', apiVersion: '2024-10-21' })diff --git a/packages/server/api/test/unit/app/ai/providers/google-provider.test.ts b/packages/server/api/test/unit/app/ai/providers/google-provider.test.tsindex ee2c01cf73a..8a68621c533 100644--- a/packages/server/api/test/unit/app/ai/providers/google-provider.test.ts+++ b/packages/server/api/test/unit/app/ai/providers/google-provider.test.ts@@ -54,6 +54,51 @@ describe('googleProvider.listModels', () => {)})+ it('drops a model that cannot generate content, so aqa and embeddings leave the picker', async () => {+ mockSendRequest.mockResolvedValue({+ body: {+ models: [+ { name: 'models/gemini-2.5-flash', displayName: 'Gemini 2.5 Flash', supportedGenerationMethods: ['generateContent'] },+ { name: 'models/gemini-embedding-001', displayName: 'Gemini Embedding', supportedGenerationMethods: ['embedContent'] },+ { name: 'models/aqa', displayName: 'Attributed Question Answering', supportedGenerationMethods: ['generateAnswer'] },+ ],+ },+ })++ const models = await googleProvider.listModels({ apiKey: 'test-key' }, {})++ expect(models.map((model) => model.id)).toEqual(['gemini-2.5-flash'])+ })++ it('drops imagen, which the image action reaches through generateContent and imagen never answers', async () => {+ mockSendRequest.mockResolvedValue({+ body: {+ models: [+ { name: 'models/imagen-3.0-generate-002', displayName: 'Imagen 3', supportedGenerationMethods: ['predict'] },+ { name: 'models/gemini-2.5-flash-image', displayName: 'Nano Banana', supportedGenerationMethods: ['generateContent'] },+ ],+ },+ })++ const models = await googleProvider.listModels({ apiKey: 'test-key' }, {})++ expect(models).toEqual([+ { id: 'gemini-2.5-flash-image', name: 'Nano Banana', type: AIProviderModelType.IMAGE },+ ])+ })++ it('keeps a model that declares no methods at all, so an unfamiliar response is not emptied', async () => {+ mockSendRequest.mockResolvedValue({+ body: {+ models: [{ name: 'models/gemini-4-pro', displayName: 'Gemini 4 Pro' }],+ },+ })++ const models = await googleProvider.listModels({ apiKey: 'test-key' }, {})++ expect(models.map((model) => model.id)).toEqual(['gemini-4-pro'])+ })+it('still classifies image models by their name', async () => {mockSendRequest.mockResolvedValue({body: {diff --git a/packages/server/api/test/unit/app/ai/providers/openai-compatible-vendor.test.ts b/packages/server/api/test/unit/app/ai/providers/openai-compatible-vendor.test.tsindex 77de22342e7..40aa721968b 100644--- a/packages/server/api/test/unit/app/ai/providers/openai-compatible-vendor.test.ts+++ b/packages/server/api/test/unit/app/ai/providers/openai-compatible-vendor.test.ts@@ -14,20 +14,26 @@ const respondWith = (ids: string[]) => {mockRequest.mockResolvedValue({ data: { data: ids.map((id) => ({ id })) } })}+const respondWithDeclaredModalities = (models: { id: string, output_modalities?: string[] }[]) => {+ mockRequest.mockResolvedValue({ data: { models } })+}+const requestedUrl = () => mockRequest.mock.calls[0][0].url+const requestedUrls = () => mockRequest.mock.calls.map((call) => call[0].url)+describe('openAiCompatibleVendor', () => {beforeEach(() => {mockRequest.mockReset()})it('requests the vendor endpoint for the provider', async () => {- respondWith(['grok-4.1-fast'])- const vendor = openAiCompatibleVendor({ name: 'xAI', provider: AIProviderName.XAI })+ respondWith(['kimi-k2'])+ const vendor = openAiCompatibleVendor({ name: 'Moonshot AI', provider: AIProviderName.MOONSHOT })await vendor.listModels({ apiKey: 'k' }, {})- expect(requestedUrl()).toBe('https://api.x.ai/v1/models')+ expect(requestedUrl()).toBe('https://api.moonshot.ai/v1/models')})it('only ever requests a hardcoded vendor host, so no admin input can redirect it', async () => {@@ -77,3 +83,85 @@ describe('openAiCompatibleVendor', () => {await expect(vendor.validateConnection({ apiKey: 'bad' }, {})).rejects.toThrow(/\[MiniMax\].*401 Unauthorized/)})})++describe('openAiCompatibleVendor (xAI declared modalities)', () => {+ const xai = () => openAiCompatibleVendor({ name: 'xAI', provider: AIProviderName.XAI })++ beforeEach(() => {+ mockRequest.mockReset()+ })++ it('asks xAI what each model outputs instead of guessing from its id', async () => {+ respondWithDeclaredModalities([{ id: 'grok-4', output_modalities: ['text'] }])++ await xai().listModels({ apiKey: 'k' }, {})++ expect(requestedUrl()).toBe('https://api.x.ai/v1/language-models')+ })++ it('drops a model that cannot answer with text, so grok image and video stay out of chat dropdowns', async () => {+ respondWithDeclaredModalities([+ { id: 'grok-4', output_modalities: ['text'] },+ { id: 'grok-2-vision-1212', output_modalities: ['text'] },+ { id: 'grok-2-image-1212', output_modalities: ['image'] },+ { id: 'grok-imagine-v0.9', output_modalities: ['image', 'video'] },+ ])++ const models = await xai().listModels({ apiKey: 'k' }, {})++ expect(models).toEqual([+ { id: 'grok-4', name: 'grok-4', type: AIProviderModelType.TEXT },+ { id: 'grok-2-vision-1212', name: 'grok-2-vision-1212', type: AIProviderModelType.TEXT },+ ])+ })++ it('falls back to the id ruleset for a model that declares no modalities', async () => {+ respondWithDeclaredModalities([+ { id: 'grok-4' },+ { id: 'grok-4-tts' },+ ])++ const models = await xai().listModels({ apiKey: 'k' }, {})++ expect(models.map((model) => model.id)).toEqual(['grok-4'])+ })++ it('falls back to the openai-shaped endpoint when the modality endpoint is unusable', async () => {+ mockRequest.mockRejectedValueOnce(new Error('404 Not Found'))+ mockRequest.mockResolvedValueOnce({ data: { data: [{ id: 'grok-4' }] } })++ const models = await xai().listModels({ apiKey: 'k' }, {})++ expect(requestedUrls()).toEqual([+ 'https://api.x.ai/v1/language-models',+ 'https://api.x.ai/v1/models',+ ])+ expect(models).toEqual([{ id: 'grok-4', name: 'grok-4', type: AIProviderModelType.TEXT }])+ })++ it('falls back rather than emptying the dropdown when the modality endpoint returns nothing', async () => {+ mockRequest.mockResolvedValueOnce({ data: { models: [] } })+ mockRequest.mockResolvedValueOnce({ data: { data: [{ id: 'grok-4' }] } })++ const models = await xai().listModels({ apiKey: 'k' }, {})++ expect(models.map((model) => model.id)).toEqual(['grok-4'])+ })++ it('still rejects a bad key, since validation falls through to the endpoint that authenticates', async () => {+ mockRequest.mockRejectedValue(new Error('401 Unauthorized'))++ await expect(xai().validateConnection({ apiKey: 'bad' }, {})).rejects.toThrow(/\[xAI\].*401 Unauthorized/)+ })++ it('only ever requests hardcoded x.ai hosts, whichever endpoint answers', async () => {+ mockRequest.mockRejectedValueOnce(new Error('404 Not Found'))+ mockRequest.mockResolvedValueOnce({ data: { data: [] } })++ await xai().listModels({ apiKey: 'k' }, {})++ for (const url of requestedUrls()) {+ expect(new URL(url).host).toBe('api.x.ai')+ }+ })+})