AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
feat(ai): run agent and chat turns on platform tiers with step-boundary fallback (#16093)
Louai Boumedienef05889be258b
diff --git a/brain/knowledge/ai-intelligence/ai-agents.md b/brain/knowledge/ai-intelligence/ai-agents.mdindex f3942579d85..d554a59c8ed 100644--- a/brain/knowledge/ai-intelligence/ai-agents.md+++ b/brain/knowledge/ai-intelligence/ai-agents.md@@ -19,7 +19,7 @@ A flow step type (the `run_agent` action of `@activepieces/piece-ai`) that runs| `AGENT_BUILDER` | `conversation.modelName` | tier id || `FLOW_STEP` | `agent.published.modelName`, or the step's own `input.aiProviderModel.model` | same as `AGENT` |-Three things follow. `run_agent` has two modes: pointed at a saved agent (`agentId` set) the step sends no model and the agent row decides; inline, it sends `input.aiProviderModel.model`. An agent row carries two configs: agent-page chat reads `agent.draft`, a `run_agent` step reads `agent.published` (via `resolvePublishedAgent`), so the same agent can already run different models through different doors. The fast model crosses every door: `firstStepUsesFastModel` (`run-agent-turn.ts`) runs the first step of almost every turn on the fast tier, flow steps included, whatever the user picked. `execute-agent-run.ts` persists `config.tier.id` onto `conversation.modelName` when the job carried no `modelName`, so conversations already hold tier ids today (inline `run_agent`, eval runs, every `AGENT_BUILDER`), read back by `chat-tool-billing.ts` and `chat-analytics-sync.ts`. **A persisted tier id is not a pin.** `pinnedModelOf` (`rpc/rpc-shared.ts`) ignores `conversation.modelName` for a conversation with an agent: that value is the resolved default, and once the console removes that tier it would reach a catalogue-less own-key provider as a literal model id. A saved agent's run model is the agent's own half (draft for AGENT, published for FLOW_STEP); only an agentless inline step reads `conversation.modelName`, which is the step's configured model.+Three things follow. `run_agent` has two modes: pointed at a saved agent (`agentId` set) the step sends no model and the agent row decides; inline, it sends `input.aiProviderModel.model`. An agent row carries two configs: agent-page chat reads `agent.draft`, a `run_agent` step reads `agent.published` (via `resolvePublishedAgent`), so the same agent can already run different models through different doors. The fast model crosses every door: `firstStepUsesFastModel` (`run-agent-turn.ts`) runs the first step of almost every turn on the fast tier, flow steps included, whatever the user picked. `execute-agent-run.ts` persists `config.tier.id` onto `conversation.modelName` when the job carried no `modelName`, so conversations already hold tier ids today (inline `run_agent`, eval runs, every `AGENT_BUILDER`), read back by `chat-tool-billing.ts` and `chat-analytics-sync.ts`. **A persisted tier id is not a pin.** `getAgentConfig` picks `modelName ?? conversation.modelName`, so a job that carries no model reads the persisted value back (`pinnedModelOf` no longer exists). A saved agent's run model is the agent's own half (draft for AGENT, published for FLOW_STEP), copied onto the job by `jobFieldsFromConfig`.### Tool types (AgentTool discriminated union)@@ -73,13 +73,17 @@ Three things follow. `run_agent` has two modes: pointed at a saved agent (`agent- **Reasoning cannot be *disabled* on a reasoning-native endpoint, and the managed catalogue is full of models nobody chose.** `buildProviderOptions` (`server/utils/src/agent-provider-options.ts`) expresses `disableThinking` as `reasoning: { enabled: false }` on the `ACTIVEPIECES`/`OPENROUTER` branch, and `prepareStep` sets `disableThinking` for the first step *and the whole discovery phase*. OpenRouter forwards the flag, and Gemini 3.x, GPT-6 Astra and Claude Fable 5.1 reject it outright: `Reasoning is mandatory for this endpoint and cannot be disabled.` The step dies before its first token. Four things this taught, in order of how easy each is to get wrong:- **Swapping in `{ effort: 'minimal' }` unconditionally is a regression, not a fix.** OpenRouter maps `minimal` to `low` for Anthropic, which is 20% of `max_tokens`, so the *thinking-disabled* first step on the Fast tier would get a bigger reasoning allowance (7,400) than its thinking-*enabled* step asks for (5,000). Every default tier is Anthropic, so that degrades every working customer to fix the broken ones. The suppression is gated on `aiProviderUtils.isCuratedChatModelId` instead: a model we picked keeps `{ enabled: false }`, anything else gets `{ effort: 'minimal' }`.- **`getCuratedChatModels` returns `undefined` for `ACTIVEPIECES` on purpose, so the managed provider serves the entire live OpenRouter catalogue.** Every failing model in the incident (`google/gemini-3.8-flash`, `openai/gpt-6-astra`, `openai/gpt-6-astra-pro`, `anthropic/claude-fable-5.1`) is absent from `ALLOWED_CHAT_MODELS_BY_PROVIDER`. Only a React `.filter()` narrows the picker to the three tiers; `modelName` is a free `z.string()` server-side and `agent-config-rpc` uses it verbatim for `FLOW_STEP` and `AGENT`. Hiding is not denying.- - **The curated list is a guess about capability, so the code learns.** A rejection calls `noteReasoningIsMandatory`, which records the id **only if it is curated** so a caller-supplied model id cannot grow the set, and the existing one-shot `shouldRetryStream` re-issues the turn with a minimal budget. `gemini-3.7-flash` is curated *and* reasoning-native, which is exactly the case the list alone would miss.+ - **The curated list is a fixed guess; nothing learns from a rejection any more.** `aiProviderUtils.canDisableReasoning` (`core/piece-types/src/lib/ai-providers.ts`) is a static check, and `shouldRetryStream` re-runs the identical request once. A curated model that turns out to be reasoning-native fails until the list is fixed.- **`satisfies` against the vendor's own reasoning type is near-worthless here.** `{ effort: 'none' }`, `{ enabled: false, max_tokens: n }` and `{ enabled: false, effort: 'minimal' }` all satisfy it, because the vendor supports disabling and it is the *endpoints* that do not. The guard has to be a closed union of the three directives we actually send.- **A tool's *property keys* are user text too, so they go through `mcp-tool-input.ts`.** Field labels a customer typed into the MCP trigger (`Email Sender`) are not valid schema keys on Anthropic. `modelKeyByPropertyName` sanitizes them with `mcpToolNameUtils.toProviderSafeIdentifier`, keeping them unique, with a positional fallback and a length cap. `toFlowPayload` maps the model's args back to the original labels, so `{{trigger['Email Sender']}}` keeps working. The MCP server and agent flow tools both use it; build any new flow-as-tool schema through it.- **Do not raise `ai` past 7.0.69 — from `[email protected]` the SDK stops auto-executing tools, and it fails silently.** Bisected on 2026-09-21 while trying to reach `experimental_evaluate`. On 7.0.44 (what the repo pins) and every version through 7.0.69, a tool call executes normally. From **7.0.70** onward, including 7.0.103–7.0.107, `generateText`/`streamText` still *recognise* the call — `staticToolCalls=1`, a `tool-call` part is emitted with correctly parsed input — but `execute` is never invoked and `toolResults` stays `0`. No error, no warning. The changelog entry for 7.0.70 is `a828527: Prevent automatic tool execution when a model call ends with an unsafe finish reason`, but the gate fires for every `finishReason` tried, `tool-calls` and `stop` included. Reproduced in a clean project outside the monorepo, on both `MockLanguageModelV3` and `MockLanguageModelV4`, on zod 3 and zod 4; `needsApproval: false` and `toolApproval: () => false` do not restore it, and neither does emitting the full `tool-input-start`/`delta`/`end` lifecycle before the `tool-call`. The alarm is `packages/server/worker/test/lib/agent-eval/agent-tool-call-repair.test.ts` — 4 of its 5 tests go red, and they look like a *repair* regression, which is the wrong trail: a plain valid tool call does not execute either. **Consequence: `experimental_evaluate` (first shipped in 7.0.103) cannot be reached without breaking the agent**, so anything needing typed evaluation calls the gateway's `/v4/ai/evaluation-model` endpoint over `safeHttp` instead of taking the bump. Re-test the table above before any future `ai` upgrade; if a release fixes it, that is the one to jump to.+- **A platform tier turn falls back per step, not per turn.** When a step fails on a provider error (`isFallbackWorthy`) before sending content, the turn continues on the next candidate from the steps already done, the way `continue_truncation` does. Tool results stay in the messages, so nothing runs twice. A fast step that sent nothing is dropped on **any** error, not only fallback-worthy ones: the Fast tier can sit on a smaller context window than the main tier (Haiku 200K under a Gemini 1M tier), and once the history outgrows it the fast model answers 400 "prompt is too long", which must not sink a turn the main model can still take. For the same reason compaction sizes its budget over the tier's candidates **and** the Fast tier, not the main tier alone. After any switch the turn stays on that candidate and has no fast model. A step that already sent text never falls back, and is not retried on the same model either (`shouldRetryStream` also reads the drain). Piece tools follow the model **running the current step** (`onStepModel` from `prepareStep`), so a fast step's tool calls use the fast key. The worker sends `answeredBy` on save, and billing labels the turn with it. Search and image stay on the pair they were resolved with, and knowledge-base search stays on the chat key, because the chunks were embedded with it.+- **"Did the failing step send anything" is read off the UI stream, and `producedVisibleOutput` cannot answer it.** `uiParts` only grows in `onStepEnd`, after chunks have already reached the client. The UI stream sends `start`/`start-step` before any content. When the model call itself throws, there is no `start-step` at all, only `error`. A stream error arrives **before** `finish-step` (`text-delta, error, finish-step`). `trackStepContent` therefore resets on `start-step`/`finish-step` and freezes its answer at the first `error` chunk. Drive it through the real `toUIMessageStream` in tests, because a mocked drain hides the ordering.+- **`AgentConfigResponse.tier` is always the console credits tier.** A platform tier run fills `platformTier`, `candidates` and `fastCandidate`. The single `credentials`/`modelId` fields equal `candidates[0]`. On a tier run the worker sends no `modelName` to `saveAgentMessages`, and the RPC ignores one on a tier row, because a model name clears the tier (the last write wins).+### Key filesEntry point: `runAgent`, the createAction in the `ai` piece registered in `packages/pieces/community/ai/src/index.ts`.diff --git a/brain/knowledge/ai-intelligence/ai-providers.md b/brain/knowledge/ai-intelligence/ai-providers.mdindex 318b4b2593e..34c0be786c9 100644--- a/brain/knowledge/ai-intelligence/ai-providers.md+++ b/brain/knowledge/ai-intelligence/ai-providers.md@@ -78,6 +78,7 @@ after a file without chat keys it seeds the chat list from the live flow list.- **The live-name unique index is `lower("name")`, which `EntitySchema` cannot express.** The entity declares it `synchronize: false` (so `check-migrations` skips it), which means test DBs built by `synchronize` don't have it. The case-insensitive check that tests rely on is the service's `assertNameIsFree`, which is race-free because it runs under the `platform` row lock; the index is the backstop.- **`lockPlatform` takes `FOR NO KEY UPDATE`, not `FOR UPDATE`.** Every FK insert that references `platform` (projects, users, …) takes `FOR KEY SHARE` on the platform row, which `FOR UPDATE` blocks until commit. `FOR NO KEY UPDATE` serialises tier writes just the same without stalling those inserts.- **The tier grant lives in one function: `aiModelCandidates.resolve` (via `platformModelTierService.getForRun`) is the only path that hands out a key without `rowAllowsScope`.** The `configId`s come from a live tier on the caller's platform, never from the caller; keep it that way. A specific pick still goes through `getConfigOrThrow` with the project scope.+- **Agent and chat runs read the tier grant from the conversation row, never from the worker.** CHAT and AGENT_BUILDER take `modelTierId` from the row (set by create/update). AGENT and FLOW_STEP take it from the job and write it onto the row **after** `acquireStreamingLock`, including null. Tool RPCs then call `aiModelCandidates.grantedEntryConfig`, which honours a worker-sent `providerConfigId` only as an exact `{configId, modelId}` entry of that tier or of the Fast tier. A tier row also skips `assertProjectSwitchKeepsKey`. Any other `providerConfigId` goes through the normal scope check.- **A platform tier step falls back only on provider failures, decided by `isFallbackWorthy` on a signal from `aiProviderSignal.fromError`.** 401/402/403/404/408/429/5xx, network errors (ai 7 throws them as `APICallError` with **no status** and `isRetryable: true`) and billing bodies (Anthropic sends low credit as a **400**) fall back; any other 4xx and our own errors (classify mismatch, no structured data, web search unsupported) stop. No fallback once a web-search step finished, since it was already billed.- **Worker-built models report no key health through `createModel`: no reporter is installed in the worker.** Tier steps report explicitly via the `reportAiKeyOutcome` RPC, and only provider signals (`fromProvider`) plus recoveries of non-active keys. Reporting our own errors would demote healthy keys to `unreachable`. The API never writes a worker-reported status for most providers: `confirmReportedOutcome` runs in the background and only triggers a `recheck` of the real key (once per key per 10s) when the report disagrees with the stored status. The exception is providers flagged `validationSkipsModelEndpoint` (CUSTOM, Vertex): their check never reaches the model, so a recheck would always pass, and the report is recorded as-is. Any direct `aiProviderHealth.record` from concurrent callers must pass `expectVersion` (the `statusVersion` it read), or an older report landing late overwrites a newer one. The worker token is shared and a shared worker serves every platform, so a report can't be bound to a platform; a forged one can at most cause a recheck. Don't pass `platformId`/`providerConfigId` to `createModel` in the worker, or a future reporter double-counts.- **`ExecuteAiJobData` is a zod schema that strips unknown keys** (`JobData.parse` in `worker.ts` and `job-broker.ts`). A field added only to the TS type silently never reaches the worker; add it to `AiStepJobBase`.diff --git a/brain/knowledge/decisions/000044-platform-tiers-are-platform-owned-and-grant-key-access.md b/brain/knowledge/decisions/000044-platform-tiers-are-platform-owned-and-grant-key-access.mdindex f5df4c7fd19..11b2ff7e603 100644--- a/brain/knowledge/decisions/000044-platform-tiers-are-platform-owned-and-grant-key-access.md+++ b/brain/knowledge/decisions/000044-platform-tiers-are-platform-owned-and-grant-key-access.md@@ -17,4 +17,5 @@ If a tier were limited by each key's project scope, the same tier would run diff- The grant is security-sensitive. The runtime must only honour a `configId` that a live tier on the same platform contains, and never a client-sent `configId`.- A key a live tier uses cannot be deleted, and its `modelScope` cannot drop a tier's model. `aiProviderService` refuses both.- A deleted tier stays as a soft-deleted row with a `replacedBy` pointer, so stored refs keep resolving without rewriting any flow.+- On agent and chat runs the grant also covers the platform's **Fast** tier, because the first step runs on it. Fallback there is per step: a step that fails before sending content continues on the next candidate.- Platform tiers are unrelated to the console **credits tiers** (`ModelTier`, `pricing.json`). They share only the word "tier".diff --git a/brain/knowledge/engineering/ci-pr-review-hygiene.md b/brain/knowledge/engineering/ci-pr-review-hygiene.mdindex ec968423190..137583b7138 100644--- a/brain/knowledge/engineering/ci-pr-review-hygiene.md+++ b/brain/knowledge/engineering/ci-pr-review-hygiene.md@@ -24,7 +24,7 @@ Enforcement is the **`Codeowners review` repository ruleset** (active on the def## Gotchas- **When merging main, `rerere` can replay a stale `bun.lock` resolution.** "Staged 'bun.lock' using previous resolution" means git reused an old answer, which can carry workspace versions that no longer match any `package.json` (seen: core-utils `0.8.6` in the lockfile against `0.8.5` in the manifest). If the branch adds no dependencies, take main's `bun.lock` and edit only the branch's own workspace `version` lines. Expect `bun install` to then rewrite a few lines for pieces the branch never touched: piece PRs on `main` often bump a `package.json` without its `bun.lock` entry (seen 2026-10-05: grist, microsoft-todo, ticktick, zendesk). Those lines are main catching up, not your change; keep them, because the lock should match the manifests.- **Any change to `bun.lock` runs every package's tests in `build-test`, all 700+ pieces included.** The `changes` job in `ci.yml` sets `all=true` when a changed file is outside `packages/`, `docs/`, `brain/`, `.agents/` and `.claude/` and is not a `*.md` file (so root Markdown alone does not count), and `tools/scripts/test-filters.ts` then adds every package with a `test` script **except `api`** (it lists with `--filter=!api`). API tests run in the separate `api-tests` matrix job (`api (ce)`, `api (cloud)`, `api (ee, unit, migrations)`), so look there for their results, not in `build-test`. Without that, a piece only runs when its own files or `pieces/framework` / `pieces/common` change; a `core-utils` change alone does not pull pieces in. Any workspace version bump edits `bun.lock`, so this hits most PRs that bump a package. A CPU-heavy piece test can then fail on the starved runner with no change of yours: an Odoo polling test that takes ~350ms locally passed vitest's 5s default in 3 of 3 runs, reruns included. Don't keep rerunning: give the heavy test an explicit timeout (third `it` argument). Even a test-only change needs that piece's patch bump (and its `bun.lock` line): `validate-publishable-packages` diffs the whole package folder against `origin/main`.-- **After the base PR of a stack merges, merging main into the next PR silently keeps the base PR's old lines.** The merge base predates the base PR, so code the base PR dropped during review (fields, routes, columns) looks like a branch addition and survives with no conflict. Check with `git diff --name-only origin/main`: every file must be one the stacked PR itself changed. If not, save the stacked PR's own patch first (`git diff --binary <base-PR tip it was built on> <head before the merge> > pr.patch`; without `--binary`, an image or other binary file is written as "Binary files differ" and `read-tree` deletes it with no way back), then `git read-tree -u --reset origin/main` and `git apply --3way pr.patch`, and commit. `read-tree` also deletes files that exist only on the branch (a file the base PR added, then dropped), which `git checkout origin/main -- .` would leave behind. Use `origin/main`, not `MERGE_HEAD`: a merge with no conflicts commits itself and `MERGE_HEAD` is gone. Then retarget it (`gh pr edit <n> --base main`): a PR still pointed at the merged base branch shows every main change since then (seen: 1,285 files for a 28-file PR). No rebase or force push needed.+- **After the base PR of a stack merges, merging main into the next PR silently keeps the base PR's old lines.** The merge base predates the base PR, so code the base PR dropped during review (fields, routes, columns) looks like a branch addition and survives with no conflict. Check with `git diff --name-only origin/main`: every file must be one the stacked PR itself changed. If not, save the stacked PR's own patch first (`git diff --binary <base-PR tip it was built on> <head before the merge> > pr.patch`; without `--binary`, an image or other binary file is written as "Binary files differ" and `read-tree` deletes it with no way back), then `git read-tree -u --reset origin/main` and `git apply --3way pr.patch`, and commit. `read-tree` also deletes files that exist only on the branch (a file the base PR added, then dropped), which `git checkout origin/main -- .` would leave behind. Use `origin/main`, not `MERGE_HEAD`: a merge with no conflicts commits itself and `MERGE_HEAD` is gone. **The rebuild is not finished until `origin/main` is an ancestor.** `read-tree` + `apply` makes a commit whose *tree* is main + patch, but the merge base with main is still the old one, so GitHub's PR diff counts every main change since then as the PR's (seen: 731 files for a 43-file PR). Follow it with `git merge origin/main`; every conflict is the same change on both sides, so `git diff --name-only --diff-filter=U | xargs git checkout --ours --` is the answer, and `git diff --cached HEAD --stat` must print nothing before you commit (the tree does not change, only the ancestry). A branch cut from a stacked branch with `git checkout -b` inherits its **upstream**, so `git status -sb` reads `feat/pr-4...origin/feat/pr-3` and a plain `git push` would push PR 4's commits onto PR 3's remote; run `git branch --unset-upstream` right after cutting it (seen 2026-10-05 on `feat/platform-tiers-admin-ui`). Then retarget it (`gh pr edit <n> --base main`): a PR still pointed at the merged base branch shows every main change since then (seen: 1,285 files for a 28-file PR). No rebase or force push needed.- **The tests bucket keys off a path *segment*, so a feature folder named `test-step` is not mistaken for tests.** `packages/web/src/app/builder/test-step/` and `latest-test.ts` stay in `packages/web`; the regex requires `test/` as a whole segment or a `.test.`/`.spec.` suffix. The one deliberate overlap is `packages/pieces/framework/src/lib/test/index.ts` — shipped `createMockActionContext` helpers, counted as tests, and exempt either way because pieces are exempt.- **A job-level `env:` in `ci.yml` does not reach anything turbo runs.** Turbo 2.x defaults to `--env-mode=strict`: a task's process only receives variables named in `globalPassThroughEnv` / `passThroughEnv` / `env`, plus turbo's own system list. Everything else is dropped silently, so the task falls back to whatever `packages/server/api/.env.tests` set (dotenv there runs without `override`, so a real env var would otherwise win). Symptom: the workflow clearly sets the variable and the test still uses the old value. Add the name to `globalPassThroughEnv` in `turbo.json` — that is why `AP_DB_TYPE` and `AP_POSTGRES_*` are already listed. Jobs that call `npx vitest` directly, like `tool-search-postgres`, are unaffected, which is what makes the difference easy to miss. Verify with a throwaway test that `console.log`s the variable through `npx turbo run <task> --filter=<pkg>`.- **A GitHub service container is not reachable by its name from a non-container job.** `services:` attaches the container to a docker network with a `--network-alias`, but the runner itself is on the host, so the alias does not resolve (`getaddrinfo EAI_AGAIN redis`) and only the published `ports:` mapping on `localhost` works. `.env.tests` pins `AP_REDIS_HOST=redis` for `docker-compose`, so any test that opens a real connection needs `localhost` forced past it.diff --git a/bun.lock b/bun.lockindex f79006079c7..60ae67471b8 100644--- a/bun.lock+++ b/bun.lock@@ -122,7 +122,7 @@},"packages/core/execution": {"name": "@activepieces/core-execution",- "version": "0.26.0",+ "version": "0.26.1","dependencies": {"@activepieces/core-piece-types": "workspace:*","@activepieces/core-utils": "workspace:*",@@ -167,7 +167,7 @@},"packages/core/shared": {"name": "@activepieces/shared",- "version": "0.194.0",+ "version": "0.195.0","dependencies": {"@activepieces/core-execution": "workspace:*","@activepieces/core-formula": "workspace:*",diff --git a/packages/core/execution/package.json b/packages/core/execution/package.jsonindex bb1a0317885..449d9e2ce2c 100644--- a/packages/core/execution/package.json+++ b/packages/core/execution/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/core-execution",- "version": "0.26.0",+ "version": "0.26.1","type": "commonjs","main": "./dist/src/index.js","scripts": {diff --git a/packages/core/execution/src/lib/workers/job-data.ts b/packages/core/execution/src/lib/workers/job-data.tsindex d2950f86a32..cfd6c5872dc 100644--- a/packages/core/execution/src/lib/workers/job-data.ts+++ b/packages/core/execution/src/lib/workers/job-data.ts@@ -328,6 +328,7 @@ export const ExecuteAgentRunJobData = z.object({provider: z.enum(AIProviderName).optional(),providerConfigId: z.string().optional(),modelName: z.string().nullable(),+ modelTierId: z.string().optional(),files: z.array(z.object({name: z.string(),mimeType: z.string(),diff --git a/packages/core/execution/src/lib/workers/worker-contract.ts b/packages/core/execution/src/lib/workers/worker-contract.tsindex 4efc1920953..c76a223607c 100644--- a/packages/core/execution/src/lib/workers/worker-contract.ts+++ b/packages/core/execution/src/lib/workers/worker-contract.ts@@ -138,6 +138,7 @@ export type GetAgentConfigRequest = {projectId?: string | nulluserMessage: stringmodelName: string | null+ modelTierId?: stringfiles?: Array<{ name: string, mimeType: string, data: string }>promptOverride?: AgentPromptOverridedryRun?: boolean@@ -178,6 +179,13 @@ export type AgentConfigResponse = {agentsAvailable: booleanuserEmail: stringsource: AgentRunSource+ platformTier?: { id: string, name: string }+ candidates?: AgentModelCandidate[]+ fastCandidate?: AgentModelCandidate+}++export type AgentModelCandidate = AiModelCandidate & {+ thinkingBudget: number}export type SaveAgentMessagesRequest = {@@ -187,9 +195,15 @@ export type SaveAgentMessagesRequest = {uiMessages: unknown[]title?: stringmodelName?: string+ answeredBy?: AgentTurnModelfailure?: { message: string, userMessage?: string }}+export type AgentTurnModel = {+ provider: AIProviderName+ modelId: string+}+export type SaveAgentFileRequest = {platformId: stringprojectId?: stringdiff --git a/packages/core/shared/package.json b/packages/core/shared/package.jsonindex 21514f7212e..5348e9290d3 100644--- a/packages/core/shared/package.json+++ b/packages/core/shared/package.json@@ -1,6 +1,6 @@{"name": "@activepieces/shared",- "version": "0.194.0",+ "version": "0.195.0","type": "commonjs","sideEffects": false,"main": "./dist/src/index.js",diff --git a/packages/core/shared/src/lib/ee/agent/agent.ts b/packages/core/shared/src/lib/ee/agent/agent.tsindex cc8bfb55b99..ccbbc9cfbba 100644--- a/packages/core/shared/src/lib/ee/agent/agent.ts+++ b/packages/core/shared/src/lib/ee/agent/agent.ts@@ -44,6 +44,7 @@ const AgentConfig = z.object({provider: Nullable(z.enum(AIProviderName)),providerConfigId: Nullable(ApId),modelName: Nullable(z.string().max(MAX_AGENT_NAME_LENGTH)),+ modelTierId: Nullable(ApId),maxSteps: z.number().int().positive().max(MAX_AGENT_STEP_BUDGET).default(DEFAULT_AGENT_MAX_STEPS),tools: z.array(AgentTool).max(MAX_AGENT_TOOLS).default([]),structuredOutput: z.array(AgentOutputField).max(MAX_AGENT_OUTPUT_FIELDS).default([]),diff --git a/packages/core/shared/src/lib/ee/agent/index.ts b/packages/core/shared/src/lib/ee/agent/index.tsindex 83ad2418794..c2a4785a089 100644--- a/packages/core/shared/src/lib/ee/agent/index.ts+++ b/packages/core/shared/src/lib/ee/agent/index.ts@@ -1,5 +1,5 @@import { AgentPromptOverride, AgentRunSource } from '@activepieces/core-execution'-import { BaseModelSchema, Nullable } from '@activepieces/core-utils'+import { ApId, BaseModelSchema, Nullable } from '@activepieces/core-utils'import { z } from 'zod'import { formErrors } from '../../form-errors'import { MAX_AGENT_TEXT_LENGTH } from './agent'@@ -205,6 +205,7 @@ export const AgentConversation = z.object({source: z.enum(AgentRunSource),title: Nullable(z.string()),modelName: Nullable(z.string()),+ modelTierId: Nullable(ApId),status: z.nativeEnum(AgentConversationStatus).default(AgentConversationStatus.IDLE),activeRunId: Nullable(z.string()),messages: z.array(z.record(z.string(), z.unknown())).default([]),@@ -223,6 +224,7 @@ export type AgentRunListItem = z.infer<typeof AgentRunListItem>export const CreateAgentConversationRequest = z.object({title: z.optional(Nullable(z.string())),modelName: z.optional(Nullable(z.string())),+ modelTierId: z.optional(Nullable(ApId)),agentId: z.optional(z.string()),builder: z.optional(z.boolean()),projectId: z.optional(z.string()),@@ -232,6 +234,7 @@ export type CreateAgentConversationRequest = z.infer<typeof CreateAgentConversatexport const UpdateAgentConversationRequest = z.object({title: z.optional(Nullable(z.string())),modelName: z.optional(Nullable(z.string())),+ modelTierId: z.optional(Nullable(ApId)),})export type UpdateAgentConversationRequest = z.infer<typeof UpdateAgentConversationRequest>diff --git a/packages/core/shared/src/lib/management/ai-providers/platform-model-tier.ts b/packages/core/shared/src/lib/management/ai-providers/platform-model-tier.tsindex f3ae8caf5c5..8318318ccf9 100644--- a/packages/core/shared/src/lib/management/ai-providers/platform-model-tier.ts+++ b/packages/core/shared/src/lib/management/ai-providers/platform-model-tier.ts@@ -43,6 +43,7 @@ export const PlatformModelTier = z.object({position: z.number().int(),entries: z.array(PlatformModelTierEntry),isDefault: z.boolean(),+ isFast: z.boolean(),thinkingBudget: Nullable(z.number().int()),deleted: Nullable(DateOrString),replacedBy: Nullable(ApId),@@ -55,6 +56,7 @@ export const PlatformModelTierSummary = z.object({description: Nullable(z.string()),position: z.number().int(),isDefault: z.boolean(),+ isFast: z.boolean(),mainModel: Nullable(z.object({provider: z.enum(AIProviderName),modelId: z.string(),@@ -77,6 +79,7 @@ export const UpdatePlatformModelTierRequest = z.object({entries: TierEntries.optional(),thinkingBudget: Nullable(TierThinkingBudget),isDefault: z.literal(true).optional(),+ isFast: z.literal(true).optional(),})export const ReorderPlatformModelTiersRequest = z.object({diff --git a/packages/server/api/src/app/ai/ai-model-candidates.ts b/packages/server/api/src/app/ai/ai-model-candidates.tsindex 5f687544075..8ee546de98e 100644--- a/packages/server/api/src/app/ai/ai-model-candidates.ts+++ b/packages/server/api/src/app/ai/ai-model-candidates.ts@@ -1,8 +1,8 @@-import { ActivepiecesError, AIProviderName, ErrorCode, isNil, PlatformId, tryCatch } from '@activepieces/core-utils'-import { ResolveAiModelCandidatesResponse } from '@activepieces/shared'+import { ActivepiecesError, AiProviderKeyStatus, AIProviderName, ErrorCode, isNil, PlatformId, tryCatch } from '@activepieces/core-utils'+import { GetProviderConfigResponse, ResolveAiModelCandidatesResponse } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import { aiProviderService } from './ai-provider-service'-import { platformModelTierService } from './platform-model-tier-service'+import { platformModelTierService, TierForRun } from './platform-model-tier-service'export const aiModelCandidates = (log: FastifyBaseLogger) => ({async firstCandidate({ platformId, tierId }: { platformId: PlatformId, tierId: string }): Promise<FirstCandidate> {@@ -10,26 +10,75 @@ export const aiModelCandidates = (log: FastifyBaseLogger) => ({return { provider: first.key.provider, providerConfigId: first.key.id, modelId: first.modelId }},- async resolve({ platformId, tierId }: { platformId: PlatformId, tierId: string }): Promise<ResolveAiModelCandidatesResponse> {- const { tier, entries } = await platformModelTierService.getForRun({ platformId, id: tierId })- const decrypted = await Promise.all(entries.map(async ({ modelId, key }) => {- const { data: credentials, error } = await tryCatch(() => aiProviderService(log).credentialsForTierKey({ platformId, key }))- if (!isNil(error) || isNil(credentials)) {- log.warn({ error, aiProvider: { id: key.id }, platformTier: { id: tier.id } }, '[aiModelCandidates] Skipping a tier entry whose key cannot be read')- return []- }- return [{ ...credentials, providerConfigId: key.id, modelId, status: key.status }]+ async firstCandidates({ platformId, tierIds }: { platformId: PlatformId, tierIds: string[] }): Promise<Map<string, FirstCandidate | null>> {+ const tiers = await platformModelTierService.getManyForRun({ platformId, ids: tierIds })+ return new Map([...tiers].map(([tierId, forRun]) => {+ const first = forRun?.entries[0]+ return [tierId, isNil(first) ? null : { provider: first.key.provider, providerConfigId: first.key.id, modelId: first.modelId }]}))- const candidates = decrypted.flat()+ },++ async resolve({ platformId, tierId }: { platformId: PlatformId, tierId: string }): Promise<ResolveAiModelCandidatesResponse> {+ const { tier, candidates } = await this.resolveConfigs({ platformId, tierId })+ return {+ tierName: tier.name,+ candidates: candidates.map(({ config, modelId, status }) => ({ ...config, providerConfigId: config.configId, modelId, status })),+ }+ },++ async resolveConfigs({ platformId, tierId }: { platformId: PlatformId, tierId: string }): Promise<TierConfigs> {+ const forRun = await platformModelTierService.getForRun({ platformId, id: tierId })+ const candidates = await readableCandidates({ platformId, forRun, log })if (candidates.length === 0) {- throw new ActivepiecesError({ code: ErrorCode.VALIDATION, params: { message: `No model in tier "${tier.name}" can run` } })+ throw new ActivepiecesError({ code: ErrorCode.VALIDATION, params: { message: `No model in tier "${forRun.tier.name}" can run` } })+ }+ return { tier: forRun.tier, candidates }+ },++ async resolveFastConfig({ platformId }: { platformId: PlatformId }): Promise<TierConfigs | null> {+ const forRun = await platformModelTierService.getFastForRun({ platformId })+ if (isNil(forRun)) {+ return null}- return { tierName: tier.name, candidates }+ const candidates = await readableCandidates({ platformId, forRun, log })+ return candidates.length === 0 ? null : { tier: forRun.tier, candidates }+ },++ async grantedEntryConfig({ platformId, tierId, configId, modelId }: { platformId: PlatformId, tierId: string, configId: string, modelId: string }): Promise<GetProviderConfigResponse | null> {+ const [run, fast] = await Promise.all([+ platformModelTierService.getForRun({ platformId, id: tierId }),+ platformModelTierService.getFastForRun({ platformId }),+ ])+ const entry = [...run.entries, ...(fast?.entries ?? [])].find((candidate) => candidate.key.id === configId && candidate.modelId === modelId)+ return isNil(entry) ? null : aiProviderService(log).configForTierKey({ platformId, key: entry.key })},})+async function readableCandidates({ platformId, forRun, log }: { platformId: PlatformId, forRun: TierForRun, log: FastifyBaseLogger }): Promise<TierConfigCandidate[]> {+ const read = await Promise.all(forRun.entries.map(async ({ modelId, key }) => {+ const { data: config, error } = await tryCatch(() => aiProviderService(log).configForTierKey({ platformId, key }))+ if (!isNil(error) || isNil(config)) {+ log.warn({ error, aiProvider: { id: key.id }, platformTier: { id: forRun.tier.id } }, '[aiModelCandidates] Skipping a tier entry whose key cannot be read')+ return []+ }+ return [{ config, modelId, status: key.status }]+ }))+ return read.flat()+}+export type FirstCandidate = {provider: AIProviderNameproviderConfigId: stringmodelId: string}++export type TierConfigCandidate = {+ config: GetProviderConfigResponse+ modelId: string+ status: AiProviderKeyStatus+}++export type TierConfigs = {+ tier: TierForRun['tier']+ candidates: TierConfigCandidate[]+}diff --git a/packages/server/api/src/app/ai/ai-provider-service.ts b/packages/server/api/src/app/ai/ai-provider-service.tsindex 479aba38e45..0ed8ac75cf7 100644--- a/packages/server/api/src/app/ai/ai-provider-service.ts+++ b/packages/server/api/src/app/ai/ai-provider-service.ts@@ -1,4 +1,4 @@-import { ActivepiecesError, AiProviderCredentials, AiProviderKeyStatus, AIProviderName, apId, classifyProviderOutcome, ErrorCode, isNil, PlatformId, ProviderOutcomeSignal, spreadIfDefined, spreadIfNotUndefined, toProviderOutcomeSignal, tryCatch, unique } from '@activepieces/core-utils'+import { ActivepiecesError, AiProviderKeyStatus, AIProviderName, apId, classifyProviderOutcome, ErrorCode, isNil, PlatformId, ProviderOutcomeSignal, spreadIfDefined, spreadIfNotUndefined, toProviderOutcomeSignal, tryCatch, unique } from '@activepieces/core-utils'import { modelCatalog, modelTierCatalog } from '@activepieces/server-utils'import { ActivePiecesProviderAuthConfig, AI_PROVIDER_ENTITY_TYPES, AIProviderAuthConfig, AIProviderConfig, aiProviderCredentials, AIProviderModel, AIProviderModelType, AiProviderProjectScope, aiProviderUtils, AIProviderWithoutSensitiveData, CreateAIProviderRequest, GetProviderConfigResponse, ProjectAIProvider, UpdateAIProviderRequest } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'@@ -289,9 +289,9 @@ export const aiProviderService = (log: FastifyBaseLogger) => ({const auth = await decryptRowAuth({ aiProvider, platformId, log })return { ...aiProviderCredentials({ provider: aiProvider.provider, auth, config: aiProvider.config }), configId: aiProvider.id, platformId, modelScope: aiProvider.modelScope, modelIds: aiProvider.modelIds }},- async credentialsForTierKey({ platformId, key }: { platformId: PlatformId, key: AIProviderSchema }): Promise<AiProviderCredentials> {+ async configForTierKey({ platformId, key }: { platformId: PlatformId, key: AIProviderSchema }): Promise<GetProviderConfigResponse> {const auth = await decryptRowAuth({ aiProvider: key, platformId, log })- return aiProviderCredentials({ provider: key.provider, auth, config: key.config })+ return { ...aiProviderCredentials({ provider: key.provider, auth, config: key.config }), configId: key.id, platformId, modelScope: key.modelScope, modelIds: key.modelIds }},async getOrCreateActivePiecesProviderAuthConfig(platformId: PlatformId): Promise<ActivePiecesProviderAuthConfig> {await ensureManagedProviderRow({ platformId })diff --git a/packages/server/api/src/app/ai/platform-model-tier-entity.ts b/packages/server/api/src/app/ai/platform-model-tier-entity.tsindex 7b032035167..e28a0d9a7ec 100644--- a/packages/server/api/src/app/ai/platform-model-tier-entity.ts+++ b/packages/server/api/src/app/ai/platform-model-tier-entity.ts@@ -40,6 +40,11 @@ export const PlatformModelTierEntity = new EntitySchema<PlatformModelTierSchema>nullable: false,default: false,},+ isFast: {+ type: Boolean,+ nullable: false,+ default: false,+ },thinkingBudget: {type: Number,nullable: true,@@ -71,6 +76,12 @@ export const PlatformModelTierEntity = new EntitySchema<PlatformModelTierSchema>unique: true,where: '"isDefault" = true AND "deleted" IS NULL',},+ {+ name: 'idx_platform_model_tier_platform_fast',+ columns: ['platformId'],+ unique: true,+ where: '"isFast" = true AND "deleted" IS NULL',+ },],relations: {platform: {diff --git a/packages/server/api/src/app/ai/platform-model-tier-service.ts b/packages/server/api/src/app/ai/platform-model-tier-service.tsindex 3ef06df1cab..557f6ff640c 100644--- a/packages/server/api/src/app/ai/platform-model-tier-service.ts+++ b/packages/server/api/src/app/ai/platform-model-tier-service.ts@@ -1,4 +1,4 @@-import { ActivepiecesError, AIProviderName, apId, ErrorCode, isNil, PlatformId, spreadIfDefined, spreadIfNotUndefined, unique } from '@activepieces/core-utils'+import { ActivepiecesError, AIProviderName, apId, ErrorCode, isNil, PlatformId, spreadIfDefined, spreadIfNotUndefined, tryCatch, unique } from '@activepieces/core-utils'import { AiProviderModelScope, CreatePlatformModelTierRequest, PlatformModelTier, PlatformModelTierEntry, PlatformModelTierSummary, UpdatePlatformModelTierRequest } from '@activepieces/shared'import { EntityManager, In } from 'typeorm'import { repoFactory } from '../core/db/repo-factory'@@ -48,6 +48,7 @@ export const platformModelTierService = {entries: request.entries,thinkingBudget: request.thinkingBudget ?? null,isDefault: isFirstTier,+ isFast: isFirstTier,deleted: null,replacedBy: null,})@@ -67,6 +68,9 @@ export const platformModelTierService = {if (request.isDefault === true && !tier.isDefault) {await tierRepo(manager).update({ platformId, isDefault: true }, { isDefault: false })}+ if (request.isFast === true && !tier.isFast) {+ await tierRepo(manager).update({ platformId, isFast: true }, { isFast: false })+ }await tierRepo(manager).update({ platformId, id }, {...spreadIfDefined('name', request.name),...spreadIfDefined('emoji', request.emoji),@@ -74,6 +78,7 @@ export const platformModelTierService = {...spreadIfDefined('entries', request.entries),...spreadIfNotUndefined('thinkingBudget', request.thinkingBudget),...spreadIfDefined('isDefault', request.isDefault),+ ...spreadIfDefined('isFast', request.isFast),})return getLiveOrThrow({ manager, platformId, id })}))@@ -113,7 +118,7 @@ export const platformModelTierService = {const tier = await getLiveOrThrow({ manager, platformId, id })if (isNil(replacedBy)) {await assertLastTierCanGo({ manager, platformId })- await tierRepo(manager).update({ platformId, id }, { isDefault: false })+ await tierRepo(manager).update({ platformId, id }, { isDefault: false, isFast: false })await tierRepo(manager).softDelete({ platformId, id })return}@@ -122,10 +127,13 @@ export const platformModelTierService = {'UPDATE "platform_model_tier" SET "replacedBy" = $1 WHERE "platformId" = $2 AND ("id" = $3 OR "replacedBy" = $3)',[replacedBy, platformId, id],)- await tierRepo(manager).update({ platformId, id }, { isDefault: false })+ await tierRepo(manager).update({ platformId, id }, { isDefault: false, isFast: false })await tierRepo(manager).softDelete({ platformId, id })- if (tier.isDefault) {- await tierRepo(manager).update({ platformId, id: replacedBy }, { isDefault: true })+ if (tier.isDefault || tier.isFast) {+ await tierRepo(manager).update({ platformId, id: replacedBy }, {+ ...(tier.isDefault ? { isDefault: true } : {}),+ ...(tier.isFast ? { isFast: true } : {}),+ })}})},@@ -138,26 +146,36 @@ export const platformModelTierService = {}},+ async getFastForRun({ platformId }: { platformId: PlatformId }): Promise<TierForRun | null> {+ const fastTier = await tierRepo().findOneBy({ platformId, isFast: true })+ if (isNil(fastTier)) {+ return null+ }+ const { data } = await tryCatch(() => platformModelTierService.getForRun({ platformId, id: fastTier.id }))+ return data ?? null+ },+async getForRun({ platformId, id }: { platformId: PlatformId, id: string }): Promise<TierForRun> {const tier = await followReplacements({ platformId, id })const configIds = unique(tier.entries.map((entry) => entry.configId))const keys = await aiProviderRepo().findBy({ platformId, id: In(configIds) })- const keyById = new Map(keys.map((key) => [key.id, key]))- const runnable = tier.entries.flatMap((entry) => {- const key = keyById.get(entry.configId)- if (isNil(key) || key.provider === AIProviderName.ACTIVEPIECES || !scopeAllows({ modelScope: key.modelScope, modelIds: key.modelIds, modelId: entry.modelId })) {- return []- }- return [{ modelId: entry.modelId, key }]- })- if (runnable.length === 0) {+ const entries = runnableEntries({ tier, keyById: new Map(keys.map((key) => [key.id, key])) })+ if (entries.length === 0) {throw new ActivepiecesError({ code: ErrorCode.VALIDATION, params: { message: `No model in tier "${tier.name}" can run` } })}- const healthyFirst = [- ...runnable.filter((entry) => entry.key.status === 'active'),- ...runnable.filter((entry) => entry.key.status !== 'active'),- ]- return { tier, entries: healthyFirst }+ return { tier, entries }+ },++ async getManyForRun({ platformId, ids }: { platformId: PlatformId, ids: string[] }): Promise<Map<string, TierForRun | null>> {+ const liveByRequestedId = await followReplacementsInBulk({ platformId, ids: unique(ids) })+ const liveTiers = unique([...liveByRequestedId.values()].filter((tier): tier is PlatformModelTier => !isNil(tier)))+ const configIds = unique(liveTiers.flatMap((tier) => tier.entries.map((entry) => entry.configId)))+ const keys = configIds.length === 0 ? [] : await aiProviderRepo().findBy({ platformId, id: In(configIds) })+ const keyById = new Map(keys.map((key) => [key.id, key]))+ return new Map([...liveByRequestedId].map(([requestedId, tier]) => {+ const entries = isNil(tier) ? [] : runnableEntries({ tier, keyById })+ return [requestedId, isNil(tier) || entries.length === 0 ? null : { tier, entries }]+ }))},async assertKeyScopeKeepsTiers({ manager, platformId, configId, modelScope, modelIds }: AssertKeyScopeParams): Promise<void> {@@ -185,6 +203,41 @@ async function lockPlatform({ manager, platformId }: { manager: EntityManager, p}}+function runnableEntries({ tier, keyById }: { tier: PlatformModelTier, keyById: Map<string, AIProviderSchema> }): TierForRun['entries'] {+ const runnable = tier.entries.flatMap((entry) => {+ const key = keyById.get(entry.configId)+ if (isNil(key) || key.provider === AIProviderName.ACTIVEPIECES || !scopeAllows({ modelScope: key.modelScope, modelIds: key.modelIds, modelId: entry.modelId })) {+ return []+ }+ return [{ modelId: entry.modelId, key }]+ })+ return [+ ...runnable.filter((entry) => entry.key.status === 'active'),+ ...runnable.filter((entry) => entry.key.status !== 'active'),+ ]+}++async function followReplacementsInBulk({ platformId, ids }: { platformId: PlatformId, ids: string[] }): Promise<Map<string, PlatformModelTier | null>> {+ const loaded = new Map<string, PlatformModelTier>()+ let missing = ids+ for (let hop = 0; hop <= MAX_REPLACEMENT_HOPS && missing.length > 0; hop++) {+ const tiers = await tierRepo().find({ where: { platformId, id: In(missing) }, withDeleted: true })+ tiers.forEach((tier) => loaded.set(tier.id, tier))+ missing = unique(tiers.flatMap((tier) => isNil(tier.deleted) || isNil(tier.replacedBy) || loaded.has(tier.replacedBy) ? [] : [tier.replacedBy]))+ }+ const liveFor = (id: string): PlatformModelTier | null => {+ let current = loaded.get(id)+ for (let hop = 0; hop <= MAX_REPLACEMENT_HOPS && !isNil(current); hop++) {+ if (isNil(current.deleted)) {+ return current+ }+ current = isNil(current.replacedBy) ? undefined : loaded.get(current.replacedBy)+ }+ return null+ }+ return new Map(ids.map((id) => [id, liveFor(id)]))+}+async function followReplacements({ platformId, id }: { platformId: PlatformId, id: string }): Promise<PlatformModelTier> {let currentId = idfor (let hop = 0; hop <= MAX_REPLACEMENT_HOPS; hop++) {@@ -288,6 +341,7 @@ function toSummary({ tier, providerByConfigId }: { tier: PlatformModelTier, provdescription: tier.description ?? null,position: tier.position,isDefault: tier.isDefault,+ isFast: tier.isFast,mainModel: isNil(main) || isNil(provider) ? null : { provider, modelId: main.modelId },fallbackCount: Math.max(tier.entries.length - 1, 0),}diff --git a/packages/server/api/src/app/database/migration/postgres/1864000000000-AddPlatformTierFastAndConversationTier.ts b/packages/server/api/src/app/database/migration/postgres/1864000000000-AddPlatformTierFastAndConversationTier.tsnew file mode 100644index 00000000000..6d576d1dd75--- /dev/null+++ b/packages/server/api/src/app/database/migration/postgres/1864000000000-AddPlatformTierFastAndConversationTier.ts@@ -0,0 +1,37 @@+import { QueryRunner } from 'typeorm'+import { Migration } from '../../migration'++export class AddPlatformTierFastAndConversationTier1864000000000 implements Migration {+ name = 'AddPlatformTierFastAndConversationTier1864000000000'+ breaking = false+ release = '0.93.0'++ public async up(queryRunner: QueryRunner): Promise<void> {+ await queryRunner.query(`+ ALTER TABLE "platform_model_tier"+ ADD "isFast" boolean NOT NULL DEFAULT false+ `)+ await queryRunner.query(`+ UPDATE "platform_model_tier" SET "isFast" = true WHERE "isDefault" = true AND "deleted" IS NULL+ `)+ await queryRunner.query(`+ CREATE UNIQUE INDEX "idx_platform_model_tier_platform_fast" ON "platform_model_tier" ("platformId") WHERE "isFast" = true AND "deleted" IS NULL+ `)+ await queryRunner.query(`+ ALTER TABLE "agent_conversation"+ ADD "modelTierId" character varying(21)+ `)+ }++ public async down(queryRunner: QueryRunner): Promise<void> {+ await queryRunner.query(`+ ALTER TABLE "agent_conversation" DROP COLUMN "modelTierId"+ `)+ await queryRunner.query(`+ DROP INDEX "idx_platform_model_tier_platform_fast"+ `)+ await queryRunner.query(`+ ALTER TABLE "platform_model_tier" DROP COLUMN "isFast"+ `)+ }+}diff --git a/packages/server/api/src/app/database/postgres-connection.ts b/packages/server/api/src/app/database/postgres-connection.tsindex 65bce6bb27d..5218f65bd28 100644--- a/packages/server/api/src/app/database/postgres-connection.ts+++ b/packages/server/api/src/app/database/postgres-connection.ts@@ -451,6 +451,7 @@ import { AddPlatformModelTier1859000000000 } from './migration/postgres/18590000import { AddPieceSetRequiredActions1860000000000 } from './migration/postgres/1860000000000-AddPieceSetRequiredActions'import { AddEventDestinationEnabledAndFormat1862000000000 } from './migration/postgres/1862000000000-AddEventDestinationEnabledAndFormat'import { AddEventDestinationHeaders1863000000000 } from './migration/postgres/1863000000000-AddEventDestinationHeaders'+import { AddPlatformTierFastAndConversationTier1864000000000 } from './migration/postgres/1864000000000-AddPlatformTierFastAndConversationTier'const getSslConfig = (): boolean | TlsOptions => {const useSsl = system.get(AppSystemProp.POSTGRES_USE_SSL)@@ -917,6 +918,7 @@ export const getMigrations = (): (new () => Migration)[] => {AddPieceSetRequiredActions1860000000000,AddEventDestinationEnabledAndFormat1862000000000,AddEventDestinationHeaders1863000000000,+ AddPlatformTierFastAndConversationTier1864000000000,]return migrations}diff --git a/packages/server/api/src/app/ee/agent/agent-conversation-controller.ts b/packages/server/api/src/app/ee/agent/agent-conversation-controller.tsindex 7f7a358bb2a..074915871f3 100644--- a/packages/server/api/src/app/ee/agent/agent-conversation-controller.ts+++ b/packages/server/api/src/app/ee/agent/agent-conversation-controller.ts@@ -114,7 +114,9 @@ export const agentConversationController: FastifyPluginAsyncZod = async (app) =>const isBuilder = conversation.source === AgentRunSource.AGENT_BUILDER// resolveRunProvider and the assertion below both fall through to the platform's chat// provider when no provider is named. An agent answers on its own model or it does not run.- if (!isNil(agent) && !isBuilder && (isNil(agentConfig?.provider) || isNil(agentConfig?.modelName))) {+ const runsOnAgentModel = !isNil(agentConfig) && !isBuilder+ const runTierId = runsOnAgentModel ? agentConfig.modelTierId ?? null : conversation.modelTierId ?? null+ if (runsOnAgentModel && isNil(runTierId) && (isNil(agentConfig.provider) || isNil(agentConfig.modelName))) {throw new ActivepiecesError({code: ErrorCode.VALIDATION,params: { message: 'Pick a model for this agent before talking to it' },@@ -133,6 +135,7 @@ export const agentConversationController: FastifyPluginAsyncZod = async (app) =>platformId,log,scope: runScope,+ modelTierId: runTierId,...spreadIfDefined('provider', agentConfig?.provider ?? undefined),...spreadIfDefined('providerConfigId', agentConfig?.providerConfigId ?? undefined),})diff --git a/packages/server/api/src/app/ee/agent/agent-conversation-entity.ts b/packages/server/api/src/app/ee/agent/agent-conversation-entity.tsindex 2f40d1c919b..47269b50927 100644--- a/packages/server/api/src/app/ee/agent/agent-conversation-entity.ts+++ b/packages/server/api/src/app/ee/agent/agent-conversation-entity.ts@@ -50,6 +50,10 @@ export const AgentConversationEntity = new EntitySchema<AgentConversationWithReltype: String,nullable: true,},+ modelTierId: {+ ...ApIdSchema,+ nullable: true,+ },status: {type: String,nullable: false,diff --git a/packages/server/api/src/app/ee/agent/agent-conversation-service.ts b/packages/server/api/src/app/ee/agent/agent-conversation-service.tsindex c27520bb6f1..69d976b77fd 100644--- a/packages/server/api/src/app/ee/agent/agent-conversation-service.ts+++ b/packages/server/api/src/app/ee/agent/agent-conversation-service.ts@@ -3,6 +3,7 @@ import { Agent, AgentConversation, AgentConversationStatus, AgentHistoryMessage,import { ModelMessage } from 'ai'import { FastifyBaseLogger } from 'fastify'import { EntityManager, In } from 'typeorm'+import { platformModelTierService } from '../../ai/platform-model-tier-service'import { transaction } from '../../core/db/transaction'import { databaseConnection } from '../../database/database-connection'import { FlowRunEntity } from '../../flows/flow-run/flow-run-entity'@@ -55,12 +56,24 @@ async function projectStillHoldingAgent({ agentId, authorisedProjectId, entityMareturn locked.projectId}+async function modelChoiceFrom({ platformId, modelName, modelTierId }: { platformId: string, modelName: string | null | undefined, modelTierId: string | null | undefined }): Promise<ModelChoice> {+ if (!isNil(modelTierId)) {+ await platformModelTierService.getForRun({ platformId, id: modelTierId })+ return { modelTierId, modelName: null }+ }+ if (!isNil(modelName)) {+ return { modelName, modelTierId: null }+ }+ return modelTierId === null ? { modelTierId: null } : {}+}+export const agentConversationService = (log: FastifyBaseLogger) => ({async createConversation({ platformId, userId, request, id }: CreateConversationParams): Promise<AgentConversation> {const agent = isNil(request.agentId)? null: await agentService(log).getOneOrThrowByPlatform({ id: request.agentId, platformId, userId })const builder = request.builder === true+ const modelChoice = await modelChoiceFrom({ platformId, modelName: request.modelName, modelTierId: request.modelTierId })const builderProjectId = builder? await resolveBuilderProject({ agent, requestedProjectId: request.projectId, platformId, userId, log }): null@@ -74,7 +87,8 @@ export const agentConversationService = (log: FastifyBaseLogger) => ({agentId: agent?.id ?? null,source: builder ? AgentRunSource.AGENT_BUILDER : isNil(agent) ? AgentRunSource.CHAT : AgentRunSource.AGENT,title: request.title ?? null,- modelName: request.modelName ?? null,+ modelName: modelChoice.modelName ?? null,+ modelTierId: modelChoice.modelTierId ?? null,messages: [],}))log.info({ conversation: { id: conversation.id }, platform: { id: platformId }, user: { id: userId } }, '[agentConversationService] Conversation created')@@ -206,7 +220,7 @@ export const agentConversationService = (log: FastifyBaseLogger) => ({const conversation = await this.getConversationOrThrow({ id, platformId, userId })const updates = {...spreadIfDefined('title', request.title),- ...spreadIfDefined('modelName', request.modelName),+ ...await modelChoiceFrom({ platformId, modelName: request.modelName, modelTierId: request.modelTierId }),}if (Object.keys(updates).length > 0) {@@ -327,3 +341,8 @@ type SetMessageFeedbackParams = ConversationIdentifier & {messageIndex: numberrequest: SetAgentMessageFeedbackRequest}++type ModelChoice = {+ modelName?: string | null+ modelTierId?: string | null+}diff --git a/packages/server/api/src/app/ee/agent/agent-helpers.ts b/packages/server/api/src/app/ee/agent/agent-helpers.tsindex 3047b5fddfb..7a5540c5a58 100644--- a/packages/server/api/src/app/ee/agent/agent-helpers.ts+++ b/packages/server/api/src/app/ee/agent/agent-helpers.ts@@ -8,6 +8,7 @@ import { EmbeddingModel, LanguageModel } from 'ai'import { FastifyBaseLogger } from 'fastify'import { Repository } from 'typeorm'import { z } from 'zod'+import { aiModelCandidates } from '../../ai/ai-model-candidates'import { aiProviderService, ProviderScope } from '../../ai/ai-provider-service'import { repoFactory } from '../../core/db/repo-factory'import { transaction } from '../../core/db/transaction'@@ -129,7 +130,11 @@ async function resolveChatProvider({ platformId, scope, log }: { platformId: strreturn chatProvider}-async function assertRunProviderConfigured({ platformId, provider, providerConfigId, scope, log }: { platformId: string, provider?: AIProviderName | null, providerConfigId?: string | null, scope: ProviderScope, log: FastifyBaseLogger }): Promise<void> {+async function assertRunProviderConfigured({ platformId, provider, providerConfigId, modelTierId, scope, log }: { platformId: string, provider?: AIProviderName | null, providerConfigId?: string | null, modelTierId?: string | null, scope: ProviderScope, log: FastifyBaseLogger }): Promise<void> {+ if (!isNil(modelTierId)) {+ await aiModelCandidates(log).firstCandidate({ platformId, tierId: modelTierId })+ return+ }if (isNil(provider)) {const chatProvider = await aiProviderService(log).getChatProviderName({ platformId, scope })if (isNil(chatProvider)) {@@ -178,14 +183,16 @@ async function resolveModelId({ platformId, providerConfig, selectedModel, surfareturn picked.id}-async function resolveImageModelId({ platformId, providerConfig, scope, log }: { platformId: string, providerConfig: GetProviderConfigResponse, scope: ProviderScope, log: FastifyBaseLogger }): Promise<string | undefined> {+async function resolveImageModelId({ platformId, providerConfig, scope, grantedByTier = false, log }: { platformId: string, providerConfig: GetProviderConfigResponse, scope: ProviderScope, grantedByTier?: boolean, log: FastifyBaseLogger }): Promise<string | undefined> {const { provider, configId } = providerConfigconst preferred = AI_PROVIDER_CAPABILITIES[provider].defaultImageModelif (isNil(preferred) || provider === AIProviderName.ACTIVEPIECES) {return preferred}const listed = await Promise.race([- tryCatch(() => aiProviderService(log).listModels({ platformId, provider, scope, configId })),+ tryCatch(() => grantedByTier+ ? aiProviderService(log).listModelsForConfig({ platformId, configId })+ : aiProviderService(log).listModels({ platformId, provider, scope, configId })),delay(IMAGE_MODEL_LOOKUP_TIMEOUT_MS, null),])if (isNil(listed) || isNil(listed.data)) {@@ -336,6 +343,7 @@ function jobFieldsFromConfig({ config }: { config: AgentConfig }): AgentJobConfimodelName: config.modelName ?? null,...spreadIfDefined('provider', config.provider ?? undefined),...spreadIfDefined('providerConfigId', config.providerConfigId ?? undefined),+ ...spreadIfDefined('modelTierId', config.modelTierId ?? undefined),promptOverride: { system: config.instructions },}}@@ -458,6 +466,6 @@ export const agentHelpers = {saveUserMemory,}-type AgentJobConfigFields = Pick<ExecuteAgentRunJobData, 'tools' | 'structuredOutput' | 'maxSteps' | 'modelName' | 'provider' | 'providerConfigId' | 'promptOverride'>+type AgentJobConfigFields = Pick<ExecuteAgentRunJobData, 'tools' | 'structuredOutput' | 'maxSteps' | 'modelName' | 'provider' | 'providerConfigId' | 'modelTierId' | 'promptOverride'>export type StreamingLockResult = 'acquired' | 'busy' | 'superseded'diff --git a/packages/server/api/src/app/ee/agent/agent-model-tier.ts b/packages/server/api/src/app/ee/agent/agent-model-tier.tsnew file mode 100644index 00000000000..4a0f905ce35--- /dev/null+++ b/packages/server/api/src/app/ee/agent/agent-model-tier.ts@@ -0,0 +1,75 @@+import { isNil, PlatformId, tryCatch } from '@activepieces/core-utils'+import { ModelTierSurface } from '@activepieces/server-utils'+import { AgentConversation, AgentModelCandidate } from '@activepieces/shared'+import { FastifyBaseLogger } from 'fastify'+import { aiModelCandidates, FirstCandidate, TierConfigCandidate, TierConfigs } from '../../ai/ai-model-candidates'+import { agentModelResolution } from './agent-model-resolution'++export const agentModelTier = (log: FastifyBaseLogger) => ({+ async resolveRun({ platformId, tierId, surface }: { platformId: PlatformId, tierId: string, surface: ModelTierSurface }): Promise<AgentTierRun> {+ const [run, fast] = await Promise.all([+ aiModelCandidates(log).resolveConfigs({ platformId, tierId }),+ aiModelCandidates(log).resolveFastConfig({ platformId }),+ ])+ const consoleBudget = agentModelResolution.resolveTier({ tierId: null, surface }).thinkingBudget+ const candidates = withBudget({ configs: run, consoleBudget })+ const fastIsThisTier = isNil(fast) || fast.tier.id === run.tier.id+ return {+ tier: { id: run.tier.id, name: run.tier.name },+ candidates,+ fast: fastIsThisTier ? null : withBudget({ configs: fast, consoleBudget })[0],+ }+ },++ async mainModelOf({ conversation }: { conversation: Pick<AgentConversation, 'platformId' | 'modelTierId'> }): Promise<FirstCandidate | null> {+ const tierId = conversation.modelTierId+ if (isNil(tierId)) {+ return null+ }+ const { data } = await tryCatch(() => aiModelCandidates(log).firstCandidates({ platformId: conversation.platformId, tierIds: [tierId] }))+ return data?.get(tierId) ?? null+ },++ async mainModelsOf({ conversations }: { conversations: Pick<AgentConversation, 'platformId' | 'modelTierId'>[] }): Promise<Map<string, FirstCandidate | null>> {+ const tierIdsByPlatform = new Map<string, string[]>()+ conversations.forEach(({ platformId, modelTierId }) => {+ if (!isNil(modelTierId)) {+ tierIdsByPlatform.set(platformId, [...(tierIdsByPlatform.get(platformId) ?? []), modelTierId])+ }+ })+ const perPlatform = await Promise.all([...tierIdsByPlatform].map(async ([platformId, tierIds]): Promise<[string, FirstCandidate | null][]> => {+ const { data, error } = await tryCatch(() => aiModelCandidates(log).firstCandidates({ platformId, tierIds }))+ if (!isNil(error) || isNil(data)) {+ log.warn({ error, platform: { id: platformId }, tierCount: tierIds.length }, '[agentModelTier] Could not resolve the tiers of a platform in one pass, each conversation will look its tier up on its own')+ return []+ }+ return tierIds.map((tierId) => [tierCacheKey({ platformId, tierId }), data.get(tierId) ?? null])+ }))+ return new Map(perPlatform.flat())+ },+})++function tierCacheKey({ platformId, tierId }: { platformId: string, tierId: string }): string {+ return `${platformId}:${tierId}`+}++function withBudget({ configs, consoleBudget }: { configs: TierConfigs, consoleBudget: number }): AgentTierCandidate[] {+ const thinkingBudget = configs.tier.thinkingBudget ?? consoleBudget+ return configs.candidates.map((candidate) => ({ ...candidate, thinkingBudget }))+}++function toWorkerCandidate({ config, modelId, status, thinkingBudget }: AgentTierCandidate): AgentModelCandidate {+ return { ...config, providerConfigId: config.configId, modelId, status, thinkingBudget }+}++export const agentTierCandidates = { toWorkerCandidate, tierCacheKey }++export type AgentTierCandidate = TierConfigCandidate & {+ thinkingBudget: number+}++export type AgentTierRun = {+ tier: { id: string, name: string }+ candidates: AgentTierCandidate[]+ fast: AgentTierCandidate | null+}diff --git a/packages/server/api/src/app/ee/agent/agent-run-controller.ts b/packages/server/api/src/app/ee/agent/agent-run-controller.tsindex 59032293318..45750ddd225 100644--- a/packages/server/api/src/app/ee/agent/agent-run-controller.ts+++ b/packages/server/api/src/app/ee/agent/agent-run-controller.ts@@ -49,6 +49,7 @@ export const agentRunController: FastifyPluginAsyncZod = async (app) => {modelName: request.body.modelName ?? null,...spreadIfDefined('provider', request.body.provider),...spreadIfDefined('providerConfigId', providerConfigId),+ ...spreadIfDefined('modelTierId', request.body.modelTierId),}: agentHelpers.jobFieldsFromConfig({ config: linked.config })const { tools, structuredOutput, provider } = runFields@@ -71,7 +72,7 @@ export const agentRunController: FastifyPluginAsyncZod = async (app) => {})}const flowTools = await agentHelpers.resolveFlowTools({ projectId, tools: tools ?? [], log: request.log })- await agentHelpers.assertRunProviderConfigured({ platformId: platform.id, provider, providerConfigId: runFields.providerConfigId, scope: agentHelpers.runScopeOrThrow({ projectId }), log: request.log })+ await agentHelpers.assertRunProviderConfigured({ platformId: platform.id, provider, providerConfigId: runFields.providerConfigId, modelTierId: runFields.modelTierId, scope: agentHelpers.runScopeOrThrow({ projectId }), log: request.log })await assertCreditsAndAppSumoNotExceeded({ platformId: platform.id, log: request.log })const { ownerId } = await projectService(request.log).getOneOrThrow(projectId)@@ -184,6 +185,7 @@ const StartAgentRunRequest = z.object({modelName: z.string().optional(),provider: z.enum(AIProviderName).optional(),providerConfigId: z.string().optional(),+ modelTierId: z.optional(ApId),})const StartAgentRunResponse = z.object({diff --git a/packages/server/api/src/app/ee/agent/agent-service.ts b/packages/server/api/src/app/ee/agent/agent-service.tsindex da6769b20ed..8a3170ea5ea 100644--- a/packages/server/api/src/app/ee/agent/agent-service.ts+++ b/packages/server/api/src/app/ee/agent/agent-service.ts@@ -4,6 +4,7 @@ import { ActivepiecesError, AIProviderName, apId, ApId, connectionTemplate, Cursimport { Agent, AgentConfig, AgentFlowTool, AgentKnowledgeBaseTool, AgentListSort, AgentMoveLoss, AgentMoveLossKind, AgentMovePreview, AgentRunSource, AgentSummary, agentUtils, AgentVisibility, CreateAgentRequest, DefaultProjectRole, Project, ProjectType, UpdateAgentRequest } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import { Brackets, EntityManager, In, SelectQueryBuilder } from 'typeorm'+import { platformModelTierService } from '../../ai/platform-model-tier-service'import { appConnectionService } from '../../app-connection/app-connection-service/app-connection-service'import { repoFactory } from '../../core/db/repo-factory'import { transaction } from '../../core/db/transaction'@@ -35,7 +36,7 @@ const AGENT_MOVED_AWAY = 'That agent has just been moved somewhere else. Reloadexport const agentService = (log: FastifyBaseLogger) => ({async create({ platformId, projectId, ownerId, request }: CreateParams): Promise<Agent> {const visibility = request.visibility ?? AgentVisibility.PROJECT- const draft = await withDefaultModel({ draft: request.draft, platformId, projectId, log })+ const draft = await withDefaultModel({ draft: await withTierOrModel({ draft: request.draft, platformId }), platformId, projectId, log })return agentRepo().save({id: apId(),projectId,@@ -119,7 +120,7 @@ export const agentService = (log: FastifyBaseLogger) => ({projectId,log,})- const draft = isNil(request.draft) ? agent.draft : sanitizeObjectForPostgresql(request.draft)+ const draft = isNil(request.draft) ? agent.draft : sanitizeObjectForPostgresql(await withTierOrModel({ draft: request.draft, platformId }))await getFolderIdFromRequest({ projectId, folderId: request.folderId ?? undefined, folderName: undefined, log })const goingLive = goLive && agentUtils.isPublishable(draft)const published = goingLive ? draft : agent.published@@ -422,13 +423,21 @@ async function isProjectAdministrator({ projectId, userId, log }: { projectId: Preturn role?.name === DefaultProjectRole.ADMIN}+async function withTierOrModel({ draft, platformId }: { draft: AgentConfig, platformId: PlatformId }): Promise<AgentConfig> {+ if (isNil(draft.modelTierId)) {+ return draft+ }+ await platformModelTierService.getForRun({ platformId, id: draft.modelTierId })+ return { ...draft, provider: null, providerConfigId: null, modelName: null }+}+async function withDefaultModel({ draft, platformId, projectId, log }: {draft: AgentConfigplatformId: PlatformIdprojectId: ProjectIdlog: FastifyBaseLogger}): Promise<AgentConfig> {- if (!isNil(draft.modelName)) {+ if (!isNil(draft.modelName) || !isNil(draft.modelTierId)) {return draft}const provider = await agentHelpers.resolveChatProviderName({ platformId, projectId, log })diff --git a/packages/server/api/src/app/ee/agent/chat-analytics-sync.ts b/packages/server/api/src/app/ee/agent/chat-analytics-sync.tsindex 2d5b8646d36..04a5c8cb3f1 100644--- a/packages/server/api/src/app/ee/agent/chat-analytics-sync.ts+++ b/packages/server/api/src/app/ee/agent/chat-analytics-sync.ts@@ -1,6 +1,7 @@import { AIProviderName, chunk, isNil, tryCatch } from '@activepieces/core-utils'import { AgentConversation, AgentRunSource, ApEdition, PersistedAgentMessage, PersistedAgentPartType, PersistedToolCallStatus } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'+import { FirstCandidate } from '../../ai/ai-model-candidates'import { isNotOneOfTheseEditions } from '../../database/database-common'import { rejectedPromiseHandler } from '../../helper/promise-handler'import { system } from '../../helper/system/system'@@ -9,6 +10,7 @@ import { platformService } from '../../platform/platform.service'import { userService } from '../../user/user-service'import { platformPlanRepo } from '../platform/platform-plan/platform-plan.service'import { agentHelpers } from './agent-helpers'+import { agentModelTier, agentTierCandidates } from './agent-model-tier'import { agentHistory } from './history/agent-history'const CONSOLE_TELEMETRY_URL = 'https://console.activepieces.com/api/chat-analytics/external/sync'@@ -49,13 +51,13 @@ async function syncConversations({ conversations, log }: {const platformIds = [...new Set(conversations.map((c) => c.platformId))]const licenseKeyByPlatform = await resolveLicenseKeysByPlatform({ platformIds })- const { userCache, platformCache, providerCache } = await resolveLookups({ conversations, log })+ const { userCache, platformCache, providerCache, tierCache } = await resolveLookups({ conversations, log })let pushed = 0let skipped = 0let failed = 0for (const batch of chunk(conversations, BATCH_SIZE)) {- const result = await pushBatch({ conversations: batch, licenseKeyByPlatform, log, userCache, platformCache, providerCache })+ const result = await pushBatch({ conversations: batch, licenseKeyByPlatform, log, userCache, platformCache, providerCache, tierCache })pushed += result.pushedskipped += result.skippedif (!result.success) {@@ -100,29 +102,32 @@ async function resolveLookups({ conversations, log }: {const uniquePlatformIds = [...new Set(conversations.map((c) => c.platformId))]const uniqueScopes = [...new Map(conversations.map((c) => [chatProviderCacheKey(c), c])).values()]- const [userEntries, platformNameEntries, providerEntries] = await Promise.all([+ const [userEntries, platformNameEntries, providerEntries, tierCache] = await Promise.all([Promise.all(uniqueUserIds.map(async (userId): Promise<[string, string | null]> => [userId, await resolveUserEmail({ userId, log })])),Promise.all(uniquePlatformIds.map(async (platformId): Promise<[string, string | null]> => [platformId, await resolvePlatformName({ platformId, log })])),Promise.all(uniqueScopes.map(async (conversation): Promise<[string, AIProviderName | null]> => [chatProviderCacheKey(conversation),await resolveProviderName({ platformId: conversation.platformId, projectId: conversation.projectId ?? null, log }),])),+ agentModelTier(log).mainModelsOf({ conversations }),])return {userCache: new Map(userEntries),platformCache: new Map(platformNameEntries),providerCache: new Map(providerEntries),+ tierCache,}}-async function pushBatch({ conversations, licenseKeyByPlatform, log, userCache, platformCache, providerCache }: {+async function pushBatch({ conversations, licenseKeyByPlatform, log, userCache, platformCache, providerCache, tierCache }: {conversations: AgentConversation[]licenseKeyByPlatform: Map<string, string>log: FastifyBaseLoggeruserCache?: Map<string, string | null>platformCache?: Map<string, string | null>providerCache?: Map<string, AIProviderName | null>+ tierCache?: Map<string, FirstCandidate | null>}): Promise<{ pushed: number, skipped: number, success: boolean }> {const secret = system.get(AppSystemProp.CONSOLE_API_SECRET_KEY)if (isNil(secret)) {@@ -132,7 +137,7 @@ async function pushBatch({ conversations, licenseKeyByPlatform, log, userCache,const payloads: Record<string, unknown>[] = []for (const conversation of conversations) {const licenseKey = licenseKeyByPlatform.get(conversation.platformId) ?? null- const payloadResult = await tryCatch(() => toSyncPayload({ conversation, licenseKey, log, userCache, platformCache, providerCache }))+ const payloadResult = await tryCatch(() => toSyncPayload({ conversation, licenseKey, log, userCache, platformCache, providerCache, tierCache }))if (payloadResult.error) {log.error({ conversation: { id: conversation.id }, error: String(payloadResult.error) }, 'Failed to build sync payload for conversation, skipping')continue@@ -168,17 +173,20 @@ async function pushBatch({ conversations, licenseKeyByPlatform, log, userCache,return { pushed: payloads.length, skipped, success: true }}-async function toSyncPayload({ conversation, licenseKey, log, userCache, platformCache, providerCache }: {+async function toSyncPayload({ conversation, licenseKey, log, userCache, platformCache, providerCache, tierCache }: {conversation: AgentConversationlicenseKey: string | nulllog: FastifyBaseLoggeruserCache?: Map<string, string | null>platformCache?: Map<string, string | null>providerCache?: Map<string, AIProviderName | null>+ tierCache?: Map<string, FirstCandidate | null>}): Promise<Record<string, unknown>> {const userEmail = userCache?.get(conversation.userId) ?? await resolveUserEmail({ userId: conversation.userId, log })const platformName = platformCache?.get(conversation.platformId) ?? await resolvePlatformName({ platformId: conversation.platformId, log })- const provider = providerCache?.get(chatProviderCacheKey(conversation)) ?? await resolveProviderName({ platformId: conversation.platformId, projectId: conversation.projectId ?? null, log })+ const tierKey = agentTierCandidates.tierCacheKey({ platformId: conversation.platformId, tierId: conversation.modelTierId ?? '' })+ const tierModel = tierCache?.has(tierKey) ? tierCache.get(tierKey) ?? null : await agentModelTier(log).mainModelOf({ conversation })+ const provider = tierModel?.provider ?? providerCache?.get(chatProviderCacheKey(conversation)) ?? await resolveProviderName({ platformId: conversation.platformId, projectId: conversation.projectId ?? null, log })const messages = agentHistory.resolveMessages({ conversation, log })@@ -190,7 +198,7 @@ async function toSyncPayload({ conversation, licenseKey, log, userCache, platforuserId: conversation.userId,userEmail,title: conversation.title,- modelName: agentHelpers.resolveModelIdForAnalytics({ selectedModel: conversation.modelName ?? null, provider, surface: 'chat' }),+ modelName: tierModel?.modelId ?? agentHelpers.resolveModelIdForAnalytics({ selectedModel: conversation.modelName ?? null, provider, surface: 'chat' }),provider,messages,messageCount: messages.length,@@ -245,4 +253,5 @@ type ConversationLookups = {userCache: Map<string, string | null>platformCache: Map<string, string | null>providerCache: Map<string, AIProviderName | null>+ tierCache: Map<string, FirstCandidate | null>}diff --git a/packages/server/api/src/app/ee/agent/chat-tool-billing.ts b/packages/server/api/src/app/ee/agent/chat-tool-billing.tsindex 9cfe562c477..97a0162af2e 100644--- a/packages/server/api/src/app/ee/agent/chat-tool-billing.ts+++ b/packages/server/api/src/app/ee/agent/chat-tool-billing.ts@@ -1,11 +1,12 @@import { isNil, spreadIfDefined } from '@activepieces/core-utils'-import { AgentConversation, chatBilling, ChatToolCall, isAppSumoCreditedPlan, PersistedAgentMessage, PersistedAgentPartType, PersistedAgentRole, PersistedToolCallStatus } from '@activepieces/shared'+import { AgentConversation, AgentTurnModel, chatBilling, ChatToolCall, isAppSumoCreditedPlan, PersistedAgentMessage, PersistedAgentPartType, PersistedAgentRole, PersistedToolCallStatus } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import { LicenseKeyPostHogEvents } from '../../helper/telemetry.utils'import { trackBillingAndSendTelemetry } from '../../platform/billing-and-telemetry'import { CreditUsageSource } from '../../platform/billing-provider'import { platformPlanService } from '../platform/platform-plan/platform-plan.service'import { agentHelpers } from './agent-helpers'+import { agentModelTier } from './agent-model-tier'import { agentHistory } from './history/agent-history'function latestTurnToolCalls({ messages }: { messages: PersistedAgentMessage[] }): ChatToolCall[] {@@ -22,17 +23,18 @@ function countBillableToolCallsInLatestTurn({ messages }: { messages: PersistedAreturn latestTurnToolCalls({ messages }).filter(chatBilling.isFlatBilledToolCall).length}-async function chargeForLatestTurn({ conversation, runId, log }: ChargeForLatestTurnParams): Promise<void> {+async function chargeForLatestTurn({ conversation, runId, answeredBy, log }: ChargeForLatestTurnParams): Promise<void> {const messages = agentHistory.resolveMessages({ conversation, log })const turnIndex = messages.filter((message) => message.role === PersistedAgentRole.USER).lengthconst idempotencyScope = runId ?? turnIndex- const provider = await agentHelpers.resolveChatProviderName({+ const tierModel = isNil(conversation.modelTierId) ? null : answeredBy ?? await agentModelTier(log).mainModelOf({ conversation })+ const provider = tierModel?.provider ?? await agentHelpers.resolveChatProviderName({platformId: conversation.platformId,projectId: conversation.projectId ?? null,log,})const surface = agentHelpers.surfaceOf({ source: conversation.source })- const model = agentHelpers.resolveModelIdForAnalytics({ selectedModel: conversation.modelName ?? null, provider, surface })+ const model = tierModel?.modelId ?? agentHelpers.resolveModelIdForAnalytics({ selectedModel: conversation.modelName ?? null, provider, surface })const tier = agentHelpers.resolveTier({ tierId: conversation.modelName ?? null, surface })const platformPlan = await platformPlanService(log).getOrCreateForPlatform(conversation.platformId)@@ -93,5 +95,6 @@ const PROJECTLESS_CHAT = 'chat'type ChargeForLatestTurnParams = {conversation: AgentConversationrunId?: string+ answeredBy?: AgentTurnModellog: FastifyBaseLogger}diff --git a/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts b/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.tsindex cbc27a07ecf..3322a20bdb2 100644--- a/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts+++ b/packages/server/api/src/app/ee/agent/rpc/agent-config-rpc.ts@@ -1,12 +1,13 @@-import { ActivepiecesError, ErrorCode, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'-import { agentAiUtils, aiUtils } from '@activepieces/server-utils'-import { AgentConfigResponse, AgentRunSource, GetAgentConfigRequest, GetEnabledAiToolsResponse, PersistedAgentMessage, PersistedAgentPartType, PersistedAgentRole } from '@activepieces/shared'-import { ModelMessage } from 'ai'+import { ActivepiecesError, AIProviderName, ErrorCode, isFallbackWorthy, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'+import { agentAiUtils, aiProviderSignal, aiUtils } from '@activepieces/server-utils'+import { AgentConfigResponse, AgentConversation, AgentRunSource, aiProviderUtils, GetAgentConfigRequest, GetEnabledAiToolsResponse, GetProviderConfigResponse, PersistedAgentMessage, PersistedAgentPartType, PersistedAgentRole } from '@activepieces/shared'+import { LanguageModel, ModelMessage } from 'ai'import { FastifyBaseLogger } from 'fastify'import { agentApprovalGate } from '.././agent-approval-gate'import { agentCompaction } from '.././agent-compaction'import { buildAttachmentNote, buildUserContentWithFiles, persistAgentAttachments } from '.././agent-file-utils'import { agentHelpers } from '.././agent-helpers'+import { agentModelTier, agentTierCandidates, AgentTierRun } from '.././agent-model-tier'import { agentMcp } from '.././mcp/agent-mcp'import { chatPersonalizationService } from '.././personalization/chat-personalization-service'import { agentPrompt } from '.././prompt/agent-prompt'@@ -86,7 +87,12 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({// selection above, which narrows to what the owner can still see in chat.const runProjectId = isFlowStep ? conversation.projectId ?? null : selectedProjectIdconst runScope = agentHelpers.runScopeOrThrow({ projectId: runProjectId })- const providerConfig = await agentHelpers.resolveRunProvider({ platformId, log, scope: runScope, ...spreadIfDefined('provider', input.provider), ...spreadIfDefined('providerConfigId', input.providerConfigId) })+ const surface = agentHelpers.surfaceOf({ source: requestedSource })+ const modelTierId = tierIdForRun({ conversation, requestedTierId: input.modelTierId ?? null })+ const tierRun = isNil(modelTierId) ? null : await agentModelTier(log).resolveRun({ platformId, tierId: modelTierId, surface })+ const providerConfig = isNil(tierRun)+ ? await agentHelpers.resolveRunProvider({ platformId, log, scope: runScope, ...spreadIfDefined('provider', input.provider), ...spreadIfDefined('providerConfigId', input.providerConfigId) })+ : tierRun.candidates[0].configconst attachmentRefs = files && files.length > 0 && !isNil(selectedProjectId)? await persistAgentAttachments({ files, projectId: selectedProjectId, platformId, conversationId, log })@@ -94,7 +100,6 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({const userContent = await buildUserContentWithFiles({ text: userMessage, files, attachmentNote: buildAttachmentNote(attachmentRefs) })const aiTools: GetEnabledAiToolsResponse = dryRun ? {} : enabledAiTools- const surface = agentHelpers.surfaceOf({ source: requestedSource })const chosen = dryRun? { search: null, image: null }: await chosenProviders.resolveForRun({ platformId, choices: providerChoices, surface, scope: runScope, log })@@ -108,7 +113,7 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({const generatesImagesOnProvider = actingRun && isNil(aiTools.imageGeneration)const imageModelId = !generatesImagesOnProvider? undefined- : chosen.image?.modelId ?? await agentHelpers.resolveImageModelId({ platformId, providerConfig, scope: runScope, log })+ : chosen.image?.modelId ?? await agentHelpers.resolveImageModelId({ platformId, providerConfig, scope: runScope, grantedByTier: !isNil(tierRun), log })const lock = await agentHelpers.acquireStreamingLock({ conversationId, ...spreadIfDefined('runId', input.runId) })if (lock === 'superseded') {@@ -129,14 +134,23 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({if (!dryRun && isNil(validCandidateProjectId) && !isNil(selectedProjectId)) {await agentHelpers.conversationRepo().update(conversationId, { projectId: selectedProjectId })}+ if (!dryRun && (conversation.modelTierId ?? null) !== modelTierId) {+ await agentHelpers.conversationRepo().update(conversationId, { modelTierId })+ }const selectedModel = modelName ?? conversation.modelName ?? nullconst namesItsOwnModel = surface === 'flow'- const tier = agentHelpers.resolveRunTier({ provider: providerConfig.provider, modelName: modelName ?? null, selectedModel, surface })- const resolvedModelId = namesItsOwnModel && !isNil(modelName)- ? agentHelpers.resolveNamedModelId({ provider: providerConfig.provider, modelName, surface, modelScope: providerConfig.modelScope, modelIds: providerConfig.modelIds, log })- : await agentHelpers.resolveModelId({ platformId, providerConfig, selectedModel, surface, scope: runScope, log })- const fastModelId = await agentHelpers.resolveFastModelId({ platformId, providerConfig, surface, scope: runScope, fallbackModelId: resolvedModelId, log })+ const tier = isNil(tierRun)+ ? agentHelpers.resolveRunTier({ provider: providerConfig.provider, modelName: modelName ?? null, selectedModel, surface })+ : { ...agentHelpers.resolveRunTier({ provider: providerConfig.provider, modelName: null, selectedModel: null, surface }), thinkingBudget: tierRun.candidates[0].thinkingBudget, modelId: tierRun.candidates[0].modelId }+ const resolvedModelId = !isNil(tierRun)+ ? tierRun.candidates[0].modelId+ : namesItsOwnModel && !isNil(modelName)+ ? agentHelpers.resolveNamedModelId({ provider: providerConfig.provider, modelName, surface, modelScope: providerConfig.modelScope, modelIds: providerConfig.modelIds, log })+ : await agentHelpers.resolveModelId({ platformId, providerConfig, selectedModel, surface, scope: runScope, log })+ const fastModelId = isNil(tierRun)+ ? await agentHelpers.resolveFastModelId({ platformId, providerConfig, surface, scope: runScope, fallbackModelId: resolvedModelId, log })+ : resolvedModelId// Inject an inventory of the project's existing connections into context so the agent// never has to *guess* an app name to find out what's connected. Without this, discovery@@ -204,27 +218,35 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({})await agentApprovalGate.clearCancel({ conversationId })- const reservedOutputTokens = await agentAiUtils.affordableOutputTokens({ provider: providerConfig.provider, modelIds: [resolvedModelId, fastModelId], thinkingBudget: tier.thinkingBudget })+ const compactionModels = isNil(tierRun)+ ? [{ config: providerConfig, modelId: resolvedModelId, thinkingBudget: tier.thinkingBudget }]+ : tierRun.candidates+ const everyModelOfTheTurn = isNil(tierRun) ? compactionModels : [...tierRun.candidates, ...(isNil(tierRun.fast) ? [] : [tierRun.fast])]+ const contextProvider = smallestContextProvider({ configs: everyModelOfTheTurn.map((candidate) => candidate.config) })+ const reservedOutputTokens = isNil(tierRun)+ ? await agentAiUtils.affordableOutputTokens({ provider: providerConfig.provider, modelIds: [resolvedModelId, fastModelId], thinkingBudget: tier.thinkingBudget })+ : Math.max(...await Promise.all(everyModelOfTheTurn.map((candidate) => agentAiUtils.affordableOutputTokens({ provider: candidate.config.provider, modelIds: [candidate.modelId], thinkingBudget: candidate.thinkingBudget }))))const reservedTokens = reservedOutputTokens + TOOL_SCHEMA_TOKEN_ESTIMATEconst payloadReservedTokens = reservedTokens + agentAiUtils.estimateTokenCount({ messages: [], systemPromptLength: systemPromptText.length })const estimatedTokens = agentCompaction.estimateTokenCount({ messages: llmHistory, systemPromptLength: systemPromptText.length })let compactionState = { summary: conversation.summary ?? null, summarizedUpToIndex: conversation.summarizedUpToIndex ?? null }- const willCompact = agentCompaction.shouldCompact({ estimatedTokens, provider: providerConfig.provider, messageCount: llmHistory.length, reservedTokens })+ const willCompact = agentCompaction.shouldCompact({ estimatedTokens, provider: contextProvider, messageCount: llmHistory.length, reservedTokens })log.debug({ estimatedTokens, willCompact, messageCount: llmHistory.length, systemPromptLength: systemPromptText.length }, '[agentRpc#getAgentConfig] Compaction decision')if (willCompact) {- const model = aiUtils.createModel({- credentials: providerConfig,- modelId: resolvedModelId,- })- compactionState = await agentCompaction.compactMessages({- messages: llmHistory,- existingSummary: compactionState.summary,- summarizedUpToIndex: compactionState.summarizedUpToIndex,- provider: providerConfig.provider,- reservedTokens: payloadReservedTokens,- model,+ const previous = compactionState+ compactionState = await onFirstModelThatAnswers({+ models: compactionModels,log,+ run: (model) => agentCompaction.compactMessages({+ messages: llmHistory,+ existingSummary: previous.summary,+ summarizedUpToIndex: previous.summarizedUpToIndex,+ provider: contextProvider,+ reservedTokens: payloadReservedTokens,+ model,+ log,+ }),})await agentHelpers.conversationRepo().update(conversationId, {summary: compactionState.summary,@@ -237,7 +259,7 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({messages: llmHistory,summary: compactionState.summary,summarizedUpToIndex: compactionState.summarizedUpToIndex,- provider: providerConfig.provider,+ provider: contextProvider,reservedTokens: payloadReservedTokens,})@@ -247,6 +269,7 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({model: { id: resolvedModelId },provider: providerConfig.provider,tier: { id: tier.id, surface },+ ...spreadIfDefined('platformTier', tierRun?.tier),project: selectedProjectId ? { id: selectedProjectId } : undefined,webSearchAvailable,}, '[agentRpc#getAgentConfig] Chat config resolved')@@ -276,9 +299,45 @@ export const agentConfigRpc = (log: FastifyBaseLogger) => ({agentsAvailable,userEmail: runUserEmail,source: conversation.source,+ ...tierFieldsOf({ tierRun }),}},})+function tierIdForRun({ conversation, requestedTierId }: { conversation: AgentConversation, requestedTierId: string | null }): string | null {+ const choiceLivesOnConversation = conversation.source === AgentRunSource.CHAT || conversation.source === AgentRunSource.AGENT_BUILDER+ return choiceLivesOnConversation ? conversation.modelTierId ?? null : requestedTierId+}++function smallestContextProvider({ configs }: { configs: GetProviderConfigResponse[] }): AIProviderName {+ return configs.reduce((smallest, config) => aiProviderUtils.getMaxContextTokens({ provider: config.provider }) < aiProviderUtils.getMaxContextTokens({ provider: smallest.provider }) ? config : smallest).provider+}++async function onFirstModelThatAnswers<T>({ models, run, log }: { models: { config: GetProviderConfigResponse, modelId: string }[], run: (model: LanguageModel) => Promise<T>, log: FastifyBaseLogger }): Promise<T> {+ for (const [index, candidate] of models.entries()) {+ const attempt = await tryCatch(() => run(aiUtils.createModel({ credentials: candidate.config, modelId: candidate.modelId })))+ if (attempt.error === null) {+ return attempt.data+ }+ const isLast = index === models.length - 1+ if (isLast || !isFallbackWorthy(aiProviderSignal.fromError(attempt.error))) {+ throw attempt.error+ }+ log.warn({ error: attempt.error, candidateIndex: index, model: { id: candidate.modelId } }, '[agentRpc#getAgentConfig] Compaction model failed, trying the next one in the tier')+ }+ throw new ActivepiecesError({ code: ErrorCode.VALIDATION, params: { message: 'No model is left to compact this conversation' } })+}++function tierFieldsOf({ tierRun }: { tierRun: AgentTierRun | null }): Pick<AgentConfigResponse, 'platformTier' | 'candidates' | 'fastCandidate'> {+ if (isNil(tierRun)) {+ return {}+ }+ return {+ platformTier: tierRun.tier,+ candidates: tierRun.candidates.map(agentTierCandidates.toWorkerCandidate),+ ...spreadIfDefined('fastCandidate', isNil(tierRun.fast) ? undefined : agentTierCandidates.toWorkerCandidate(tierRun.fast)),+ }+}+const TOOL_SCHEMA_TOKEN_ESTIMATE = 12_000diff --git a/packages/server/api/src/app/ee/agent/rpc/conversation-rpc.ts b/packages/server/api/src/app/ee/agent/rpc/conversation-rpc.tsindex 4bbc696d458..4f6f162455a 100644--- a/packages/server/api/src/app/ee/agent/rpc/conversation-rpc.ts+++ b/packages/server/api/src/app/ee/agent/rpc/conversation-rpc.ts@@ -75,7 +75,7 @@ export const conversationRpc = (log: FastifyBaseLogger) => ({updates.messages = input.messagesupdates.uiMessages = sanitizeObjectForPostgresql(input.uiMessages)if (input.title) updates.title = input.title- if (input.modelName) updates.modelName = input.modelName+ if (input.modelName && isNil(stored?.modelTierId)) updates.modelName = input.modelName}const failureAppend = isNil(input.failure) ? null : failureAppendFor({ stored: stored?.uiMessages ?? [], failure: input.failure })if (!isNil(failureAppend)) {@@ -118,7 +118,7 @@ export const conversationRpc = (log: FastifyBaseLogger) => ({const conversation = await agentHelpers.conversationRepo().findOneBy({ id: input.conversationId })if (conversation) {chatAnalyticsTelemetry(log).sendConversationUpdate({ conversation })- rejectedPromiseHandler(chatToolBilling.chargeForLatestTurn({ conversation, runId: input.runId, log }), log)+ rejectedPromiseHandler(chatToolBilling.chargeForLatestTurn({ conversation, runId: input.runId, log, ...spreadIfDefined('answeredBy', input.answeredBy) }), log)}}},@@ -162,7 +162,7 @@ export const conversationRpc = (log: FastifyBaseLogger) => ({params: { message: 'A flow-step agent run cannot move to another project' },})}- if (!isNil(conversation)) {+ if (!isNil(conversation) && isNil(conversation.modelTierId)) {await agentHelpers.assertProjectSwitchKeepsKey({platformId: conversation.platformId,fromProjectId: conversation.projectId ?? null,diff --git a/packages/server/api/src/app/ee/agent/rpc/rpc-shared.ts b/packages/server/api/src/app/ee/agent/rpc/rpc-shared.tsindex b4bbe731efc..7081d0c2cb6 100644--- a/packages/server/api/src/app/ee/agent/rpc/rpc-shared.ts+++ b/packages/server/api/src/app/ee/agent/rpc/rpc-shared.ts@@ -66,6 +66,7 @@ export async function configuredToolConversationOrThrow({ conversationId }: { coplatformId: conversation.platformId,userId: conversation.userId,source: conversation.source,+ modelTierId: conversation.modelTierId ?? null,...spreadIfDefined('agent', isNil(conversation.agentId) ? undefined : {id: conversation.agentId,...spreadIfDefined('displayName', conversation.agent?.displayName),@@ -200,6 +201,7 @@ export type ConfiguredToolRun = {platformId: stringuserId: stringsource: AgentRunSource+ modelTierId: string | nullagent?: { id: string, displayName?: string }}diff --git a/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts b/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.tsindex d1bac09ef71..d6f293ed72d 100644--- a/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts+++ b/packages/server/api/src/app/ee/agent/rpc/tool-execution-rpc.ts@@ -2,24 +2,25 @@ import { isDeepStrictEqual } from 'node:util'import { ActivepiecesAiConsumerSource, ActivepiecesError, ErrorCode, isNil, spreadIfDefined, tryCatch } from '@activepieces/core-utils'import { aiUtils } from '@activepieces/server-utils'import { AGENT_SELF_EDIT_TOOLS, AGENT_SURFACE_TOOLS, AgentActionOutcome, AgentRunSource, agentToolClassification, ExecuteAgentToolRequest, ExecuteAgentToolResponse, ExecuteFlowToolRequest, ExecuteFlowToolResponse, ExecuteKnowledgeBaseToolRequest, ExecuteKnowledgeBaseToolResponse, ExecutePieceToolRequest, ExecutePieceToolResponse, FlowActionType, flowStructureUtil } from '@activepieces/shared'-import { embed } from 'ai'+import { embed, LanguageModel } from 'ai'import { FastifyBaseLogger } from 'fastify'import { agentApprovalGate } from '.././agent-approval-gate'import { agentHelpers } from '.././agent-helpers'import { executeCrossProjectTool } from '.././tools/agent-tools'import { pieceToolRunner } from '.././tools/piece-tool-runner'+import { aiModelCandidates } from '../../../ai/ai-model-candidates'import { flowService } from '../../../flows/flow/flow.service'import { flowRunService } from '../../../flows/flow-run/flow-run-service'import { knowledgeBaseService } from '../../../knowledge-base/knowledge-base.service'import { extractMcpTriggerInput, resolveRunnableFlow, runFlowAsTool } from '../../../mcp/mcp-server-builder'-import { byteLengthOf, CONFIGURED_TOOL_SOURCES, configuredToolConversationOrThrow, confinedRunFor, connectionForConfiguredTool, markTurnAsHavingRead, outcomeOfToolResult, pinConnectionToAgent, recordAgentAction, recordAgentFlowToolUse, turnHasRead } from './rpc-shared'+import { byteLengthOf, CONFIGURED_TOOL_SOURCES, configuredToolConversationOrThrow, ConfiguredToolRun, confinedRunFor, connectionForConfiguredTool, markTurnAsHavingRead, outcomeOfToolResult, pinConnectionToAgent, recordAgentAction, recordAgentFlowToolUse, turnHasRead } from './rpc-shared'export const toolExecutionRpc = (log: FastifyBaseLogger) => ({async executePieceTool(input: ExecutePieceToolRequest): Promise<ExecutePieceToolResponse> {const configuredRun = await configuredToolConversationOrThrow({ conversationId: input.conversationId })const { projectId, platformId } = configuredRun- const model = await agentHelpers.resolveFastModel({ platformId, surface: agentHelpers.surfaceOf({ source: configuredRun.source }), scope: { type: 'project', projectId }, log, ...spreadIfDefined('provider', input.provider), ...spreadIfDefined('providerConfigId', input.providerConfigId), fallbackModelId: input.modelId })+ const model = await pieceToolModel({ configuredRun, input, log })const piece = { pieceName: input.piece.pieceName, actionName: input.piece.actionName, ...spreadIfDefined('pieceVersion', input.piece.pieceVersion) }const connection = await connectionForConfiguredTool({ piece: input.piece, projectId, platformId, log })const { data: resolved, error: resolveError } = await tryCatch(() => pieceToolRunner.resolveInput({@@ -321,6 +322,17 @@ const UNATTENDED_FORBIDDEN_TOOLS = ['ap_run_code', 'ap_execute_action', 'ap_explconst KNOWLEDGE_BASE_SEARCH_LIMIT = 5const KNOWLEDGE_BASE_SIMILARITY_THRESHOLD = 0.5+async function pieceToolModel({ configuredRun, input, log }: { configuredRun: ConfiguredToolRun, input: ExecutePieceToolRequest, log: FastifyBaseLogger }): Promise<LanguageModel> {+ const { projectId, platformId, modelTierId } = configuredRun+ const granted = isNil(modelTierId) || isNil(input.providerConfigId)+ ? null+ : await aiModelCandidates(log).grantedEntryConfig({ platformId, tierId: modelTierId, configId: input.providerConfigId, modelId: input.modelId })+ if (!isNil(granted)) {+ return aiUtils.createModel({ credentials: granted, modelId: input.modelId, platformId, providerConfigId: granted.configId })+ }+ return agentHelpers.resolveFastModel({ platformId, surface: agentHelpers.surfaceOf({ source: configuredRun.source }), scope: { type: 'project', projectId }, log, ...spreadIfDefined('provider', input.provider), ...spreadIfDefined('providerConfigId', input.providerConfigId), fallbackModelId: input.modelId })+}+async function flowOfRun({ flowRunId, projectId, log }: { flowRunId: string, projectId: string, log: FastifyBaseLogger }): Promise<{ id: string, runId: string } | undefined> {const { data: flowRun, error } = await tryCatch(() => flowRunService(log).getOneOrThrow({ id: flowRunId, projectId }))if (isNil(flowRun)) {diff --git a/packages/server/api/test/integration/ce/ai-provider/ai-model-candidates.test.ts b/packages/server/api/test/integration/ce/ai-provider/ai-model-candidates.test.tsindex 395c62047ec..8b7cd1f57cc 100644--- a/packages/server/api/test/integration/ce/ai-provider/ai-model-candidates.test.ts+++ b/packages/server/api/test/integration/ce/ai-provider/ai-model-candidates.test.ts@@ -3,6 +3,7 @@ import { AiStepAction, PlatformModelTier, PrincipalType } from '@activepieces/shimport { FastifyInstance } from 'fastify'import { StatusCodes } from 'http-status-codes'import { vi } from 'vitest'+import { aiModelCandidates } from '../../../../src/app/ai/ai-model-candidates'import { aiRpcHandlers } from '../../../../src/app/ai/ai-rpc-handlers'import { databaseConnection } from '../../../../src/app/database/database-connection'import { generateMockToken } from '../../../helpers/auth'@@ -92,6 +93,24 @@ describe('tier candidates', () => {expect(candidates.map((candidate) => candidate.modelId)).toEqual(['new-model'])})+ it('resolves many tiers at once, following replacements and skipping removed or foreign ones', async () => {+ const key = await seedKey({ testCtx: ctx })+ const live = await createTier({ testCtx: ctx, name: 'Live', entries: [{ configId: key.id, modelId: 'live-model' }] })+ const old = await createTier({ testCtx: ctx, name: 'Old', entries: [{ configId: key.id, modelId: 'old-model' }] })+ const gone = await createTier({ testCtx: ctx, name: 'Gone', entries: [{ configId: key.id, modelId: 'gone-model' }] })+ await ctx.delete(`${TIERS}/${old.id}`, { replacedBy: live.id })+ await ctx.delete(`${TIERS}/${gone.id}`, { replacedBy: live.id })+ const otherCtx = await createTestContext(app!)+ const foreign = await createTier({ testCtx: otherCtx, name: 'Foreign', entries: [{ configId: (await seedKey({ testCtx: otherCtx })).id, modelId: 'foreign-model' }] })++ const found = await aiModelCandidates(app!.log).firstCandidates({ platformId: ctx.platform.id, tierIds: [live.id, old.id, foreign.id, apId()] })++ expect(found.get(live.id)?.modelId).toBe('live-model')+ expect(found.get(old.id)?.modelId).toBe('live-model')+ expect(found.get(foreign.id)).toBeNull()+ expect(found.size).toBe(4)+ })+it('fails a tier that was removed without a replacement', async () => {const key = await seedKey({ testCtx: ctx })const tier = await createTier({ testCtx: ctx, entries: [{ configId: key.id, modelId: 'gpt-4o' }] })diff --git a/packages/server/api/test/integration/ce/ai-provider/platform-model-tier.test.ts b/packages/server/api/test/integration/ce/ai-provider/platform-model-tier.test.tsindex 77ecdb6883e..7c3a9bf6f4c 100644--- a/packages/server/api/test/integration/ce/ai-provider/platform-model-tier.test.ts+++ b/packages/server/api/test/integration/ce/ai-provider/platform-model-tier.test.ts@@ -31,8 +31,8 @@ describe('Platform model tiers API', () => {const first = await createTier({ testCtx: ctx, body: tierBody({ configId: key.id, name: 'Fast' }) })const second = await createTier({ testCtx: ctx, body: tierBody({ configId: key.id, name: 'Expert' }) })- expect(first).toMatchObject({ isDefault: true, position: 0, emoji: '⚡' })- expect(second).toMatchObject({ isDefault: false, position: 1 })+ expect(first).toMatchObject({ isDefault: true, isFast: true, position: 0, emoji: '⚡' })+ expect(second).toMatchObject({ isDefault: false, isFast: false, position: 1 })})it('rejects bad entry counts and duplicate entries', async () => {@@ -160,6 +160,19 @@ describe('Platform model tiers API', () => {expect(tiers.find((tier) => tier.id === second.id)).toMatchObject({ isDefault: true, name: 'B2' })})+ it('moves the fast pointer on its own', async () => {+ const key = await seedKey({ testCtx: ctx })+ const first = await createTier({ testCtx: ctx, body: tierBody({ configId: key.id, name: 'A' }) })+ const second = await createTier({ testCtx: ctx, body: tierBody({ configId: key.id, name: 'B' }) })++ const response = await ctx.post(`${TIERS}/${second.id}`, { isFast: true })+ const tiers = await listAdmin({ testCtx: ctx })++ expect(response.statusCode).toBe(StatusCodes.OK)+ expect(tiers.find((tier) => tier.id === first.id)).toMatchObject({ isDefault: true, isFast: false })+ expect(tiers.find((tier) => tier.id === second.id)).toMatchObject({ isDefault: false, isFast: true })+ })+it('reorders only with the exact live set', async () => {const key = await seedKey({ testCtx: ctx })const a = await createTier({ testCtx: ctx, body: tierBody({ configId: key.id, name: 'A' }) })@@ -225,9 +238,9 @@ describe('Platform model tiers API', () => {expect(first.statusCode).toBe(StatusCodes.NO_CONTENT)expect(second.statusCode).toBe(StatusCodes.NO_CONTENT)- expect(oldA).toMatchObject({ replacedBy: c.id, isDefault: false })+ expect(oldA).toMatchObject({ replacedBy: c.id, isDefault: false, isFast: false })expect(live.map((tier) => tier.id)).toEqual([c.id])- expect(live[0]).toMatchObject({ isDefault: true })+ expect(live[0]).toMatchObject({ isDefault: true, isFast: true })})it('refuses a bad replacement', async () => {diff --git a/packages/server/api/test/integration/ee/agent/agent-tier-runtime.test.ts b/packages/server/api/test/integration/ee/agent/agent-tier-runtime.test.tsnew file mode 100644index 00000000000..da6b824c8d6--- /dev/null+++ b/packages/server/api/test/integration/ee/agent/agent-tier-runtime.test.ts@@ -0,0 +1,185 @@+import { AIProviderName, apId, ErrorCode } from '@activepieces/core-utils'+import { AgentConversation, AgentIcon, AgentRunSource, ColorName, WorkerJobType } from '@activepieces/shared'+import { FastifyInstance } from 'fastify'+import { StatusCodes } from 'http-status-codes'+import { afterAll, afterEach, beforeAll, describe, expect, it, vi } from 'vitest'+import { aiModelCandidates } from '../../../../src/app/ai/ai-model-candidates'+import { agentRpcHandlers } from '../../../../src/app/ee/agent/agent-rpc-handlers'+import * as jobQueueModule from '../../../../src/app/workers/job-queue/job-queue'+import { db } from '../../../helpers/db'+import { createMockProject, mockAndSaveAIProvider } from '../../../helpers/mocks'+import { createTestContext, TestContext } from '../../../helpers/test-context'+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'++let app: FastifyInstance++const originalJobQueue = jobQueueModule.jobQueue++beforeAll(async () => {+ app = await setupTestEnvironment()+})++afterAll(async () => {+ await teardownTestEnvironment()+})++afterEach(() => {+ vi.restoreAllMocks()+})++describe('a chat on a platform tier', () => {+ it('runs on a tier key that is scoped away from the project, because the tier grants it', async () => {+ const { ctx, tier, keyId } = await tierOnScopedAwayKey()+ const conversation = await createConversation({ ctx, body: { modelTierId: tier.id } })++ const config = await getConfig({ ctx, conversationId: conversation.id })++ expect(config.candidates?.map((candidate) => candidate.providerConfigId)).toEqual([keyId])+ expect(config.providerConfigId).toBe(keyId)+ expect(config.platformTier).toEqual({ id: tier.id, name: 'Expert' })+ })++ it('cannot reach the same key as a specific model, because the key does not serve the project', async () => {+ const { ctx, keyId } = await tierOnScopedAwayKey()+ const conversation = await createConversation({ ctx, body: {} })++ await expect(getConfig({ ctx, conversationId: conversation.id, provider: AIProviderName.OPENAI, providerConfigId: keyId })).rejects.toMatchObject({ error: { code: ErrorCode.ENTITY_NOT_FOUND } })+ })++ it('refuses a tier from another platform', async () => {+ const { tier } = await tierOnScopedAwayKey()+ const otherCtx = await context()++ const response = await otherCtx.post(CONVERSATIONS_URL, { modelTierId: tier.id })++ expect(response.statusCode).toBe(StatusCodes.NOT_FOUND)+ })++ it('keeps one model choice: a tier clears the model name and a model name clears the tier', async () => {+ const { ctx, tier } = await tierOnScopedAwayKey()+ const conversation = await createConversation({ ctx, body: { modelName: 'expert', modelTierId: tier.id } })++ const pickedModel = await ctx.post(`${CONVERSATIONS_URL}/${conversation.id}`, { modelName: 'fast' })+ const pickedTier = await ctx.post(`${CONVERSATIONS_URL}/${conversation.id}`, { modelTierId: tier.id })++ expect(conversation).toMatchObject({ modelTierId: tier.id, modelName: null })+ expect(pickedModel.json()).toMatchObject({ modelName: 'fast', modelTierId: null })+ expect(pickedTier.json()).toMatchObject({ modelTierId: tier.id, modelName: null })+ })++ it('keeps the tier when the worker saves the turn with a model name', async () => {+ const { ctx, tier } = await tierOnScopedAwayKey()+ const conversation = await createConversation({ ctx, body: { modelTierId: tier.id } })++ await agentRpcHandlers(app.log).saveAgentMessages({ conversationId: conversation.id, messages: [{ role: 'user', content: 'hi' }], uiMessages: [], modelName: 'smart' })++ const stored = await db.findOneByOrFail<AgentConversation>('agent_conversation', { id: conversation.id })+ expect(stored).toMatchObject({ modelTierId: tier.id, modelName: null })+ })++ it('moves to another project on a tier key that serves neither project', async () => {+ const { ctx, tier, keyId } = await tierOnScopedAwayKey()+ const conversation = await createConversation({ ctx, body: { modelTierId: tier.id } })+ await db.update('agent_conversation', conversation.id, { projectId: ctx.project.id })+ const target = createMockProject({ ownerId: ctx.user.id, platformId: ctx.platform.id })+ await db.save('project', target)++ await agentRpcHandlers(app.log).updateProjectContext({ conversationId: conversation.id, projectId: target.id, provider: AIProviderName.OPENAI, providerConfigId: keyId })++ const stored = await db.findOneByOrFail<AgentConversation>('agent_conversation', { id: conversation.id })+ expect(stored.projectId).toBe(target.id)+ })+})++describe('the tier grant for tools', () => {+ it('grants a model of the tier on its own key, and nothing else', async () => {+ const { ctx, tier, keyId } = await tierOnScopedAwayKey()+ const otherKey = await mockAndSaveAIProvider({ platformId: ctx.platform.id, provider: AIProviderName.OPENAI })+ const grant = (configId: string, modelId: string) => aiModelCandidates(app.log).grantedEntryConfig({ platformId: ctx.platform.id, tierId: tier.id, configId, modelId })++ expect((await grant(keyId, TIER_MODEL))?.configId).toBe(keyId)+ expect(await grant(keyId, 'another-model')).toBeNull()+ expect(await grant(otherKey.id, TIER_MODEL)).toBeNull()+ })+})++describe('an agent on a platform tier', () => {+ it('keeps the tier on create instead of filling in a default model', async () => {+ const { ctx, tier } = await tierOnScopedAwayKey()++ const agent = await createAgent({ ctx, draft: { modelTierId: tier.id, provider: AIProviderName.OPENAI, modelName: 'gpt-4o' } })++ expect(agent.draft).toMatchObject({ modelTierId: tier.id, provider: null, providerConfigId: null, modelName: null })+ })++ it('can be talked to, and hands the tier to the worker', async () => {+ const { ctx, tier } = await tierOnScopedAwayKey()+ const add = vi.fn()+ vi.spyOn(jobQueueModule, 'jobQueue').mockImplementation((log) => ({ ...originalJobQueue(log), add }))+ const agent = await createAgent({ ctx, draft: { modelTierId: tier.id } })+ const conversation = await createConversation({ ctx, body: { agentId: agent.id } })++ const response = await ctx.post(`${CONVERSATIONS_URL}/${conversation.id}/messages`, { content: 'hello' })++ expect(response.statusCode).toBe(StatusCodes.OK)+ const job = add.mock.calls.map(([call]) => call).find((call) => call.data.jobType === WorkerJobType.EXECUTE_AGENT_RUN)+ expect(job.data.modelTierId).toBe(tier.id)+ })++ it('records the tier a flow step ran on, so its tools are granted the same keys', async () => {+ const { ctx, tier } = await tierOnScopedAwayKey()+ const conversationId = apId()++ await agentRpcHandlers(app.log).getAgentConfig({ conversationId, platformId: ctx.platform.id, projectId: ctx.project.id, userId: ctx.user.id, userMessage: 'sweep the inbox', modelName: null, modelTierId: tier.id, source: AgentRunSource.FLOW_STEP })++ const stored = await db.findOneByOrFail<AgentConversation>('agent_conversation', { id: conversationId })+ expect(stored.modelTierId).toBe(tier.id)+ })+})++async function context(): Promise<TestContext> {+ return createTestContext(app, { plan: { agentsEnabled: true, chatEnabled: true, aiProvidersEnabled: true } })+}++async function tierOnScopedAwayKey(): Promise<{ ctx: TestContext, tier: { id: string }, keyId: string }> {+ const ctx = await context()+ const key = await mockAndSaveAIProvider({ platformId: ctx.platform.id, provider: AIProviderName.OPENAI })+ await db.update('ai_provider', key.id, { projectScope: 'selected', projectIds: [] })+ const response = await ctx.post('/v1/platform-model-tiers', { name: 'Expert', emoji: '🧠', description: null, entries: [{ configId: key.id, modelId: TIER_MODEL }] })+ expect(response.statusCode).toBe(StatusCodes.OK)+ return { ctx, tier: response.json(), keyId: key.id }+}++async function createConversation({ ctx, body }: { ctx: TestContext, body: Record<string, unknown> }): Promise<AgentConversation> {+ const response = await ctx.post(CONVERSATIONS_URL, body)+ expect(response.statusCode).toBe(StatusCodes.CREATED)+ return response.json()+}++async function createAgent({ ctx, draft }: { ctx: TestContext, draft: Record<string, unknown> }): Promise<{ id: string, draft: Record<string, unknown> }> {+ const response = await ctx.post('/v1/agents', {+ projectId: ctx.project.id,+ displayName: 'Inbox sorter',+ icon: AgentIcon.MAIL,+ color: ColorName.BLUE,+ draft: { instructions: 'Sort unread mail.', provider: null, modelName: null, maxSteps: 5, tools: [], structuredOutput: [], ...draft },+ })+ expect(response.statusCode).toBe(StatusCodes.CREATED)+ return response.json()+}++async function getConfig({ ctx, conversationId, provider, providerConfigId }: { ctx: TestContext, conversationId: string, provider?: AIProviderName, providerConfigId?: string }): ReturnType<ReturnType<typeof agentRpcHandlers>['getAgentConfig']> {+ return agentRpcHandlers(app.log).getAgentConfig({+ conversationId,+ platformId: ctx.platform.id,+ userId: ctx.user.id,+ userMessage: 'hello',+ modelName: null,+ dryRun: true,+ ...(provider ? { provider } : {}),+ ...(providerConfigId ? { providerConfigId } : {}),+ })+}++const CONVERSATIONS_URL = '/v1/agents/conversations'+const TIER_MODEL = 'gpt-4o'diff --git a/packages/server/api/test/unit/app/ee/agent/agent-rpc-handlers.test.ts b/packages/server/api/test/unit/app/ee/agent/agent-rpc-handlers.test.tsindex ca937f6c8cc..a9064a83614 100644--- a/packages/server/api/test/unit/app/ee/agent/agent-rpc-handlers.test.ts+++ b/packages/server/api/test/unit/app/ee/agent/agent-rpc-handlers.test.ts@@ -187,7 +187,7 @@ describe('agentRpcHandlers.updateAgentProgress — incremental LLM message persi})})-async function callSaveChatMessages(input: { conversationId: string, runId?: string, messages: unknown[], uiMessages: unknown[] }): Promise<void> {+async function callSaveChatMessages(input: { conversationId: string, runId?: string, messages: unknown[], uiMessages: unknown[], answeredBy?: { provider: string, modelId: string } }): Promise<void> {await agentRpcHandlers(noopLogger as never).saveAgentMessages(input as never)}@@ -264,6 +264,15 @@ describe('agentRpcHandlers.saveAgentMessages — billing a row the run no longerexpect(mockTrack.mock.calls[0][0]).toMatchObject({ runId: 'run-1' })})+ it('bills a platform tier turn against the model that answered it', async () => {+ mockExecute.mockResolvedValue({ raw: [{ id: 'conv-1' }] })+ mockFindOneBy.mockResolvedValue({ id: 'conv-1', messages: [{ role: 'user' }] })++ await callSaveChatMessages({ conversationId: 'conv-1', runId: 'run-1', messages: [{ role: 'user' }, { role: 'assistant' }], uiMessages: [{ role: 'assistant' }], answeredBy: { provider: 'openai', modelId: 'gpt-4o' } })++ expect(mockTrack.mock.calls[0][0]).toMatchObject({ answeredBy: { provider: 'openai', modelId: 'gpt-4o' } })+ })+it('does not bill when the write returned nothing, on any driver', async () => {mockExecute.mockResolvedValue({})mockFindOneBy.mockResolvedValue({ id: 'conv-1', messages: [{ role: 'user' }] })diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.tsindex 8976944d054..a8185d04842 100644--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/execute-agent-run.ts@@ -1,14 +1,14 @@import { ActivepiecesAiBilling, ActivepiecesAiConsumerSource, AIProviderName, ErrorCode, formatPieceError, isNil, isObject, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'import { agentAiUtils, aiUtils } from '@activepieces/server-utils'-import { AgentEvent, AgentEventType, AgentKnowledgeBaseTool, AgentMcpTool, AgentOutputField, AgentPhase, AgentPieceTool, AgentResult, AgentRunSource, AgentTool, AgentToolType, AiProviderCredentials, apErrorOf, EngineResponseStatus, ExecuteAgentRunJobData, MAX_AGENT_TURN_WALL_CLOCK_MS, PersistedAgentMessage, PersistedAgentMessageSchema, PersistedAgentPart, PersistedAgentPartType, PersistedAgentRole, ResolvedAgentFlowTool, WorkerJobType } from '@activepieces/shared'-import { createUIMessageStream, generateText, ModelMessage, streamText, ToolSet, toUIMessageStream } from 'ai'+import { AgentConfigResponse, AgentEvent, AgentEventType, AgentKnowledgeBaseTool, AgentMcpTool, AgentModelCandidate, AgentOutputField, AgentPhase, AgentPieceTool, AgentResult, AgentRunSource, AgentTool, AgentToolType, AiProviderCredentials, apErrorOf, EngineResponseStatus, ExecuteAgentRunJobData, MAX_AGENT_TURN_WALL_CLOCK_MS, PersistedAgentMessage, PersistedAgentMessageSchema, PersistedAgentPart, PersistedAgentPartType, PersistedAgentRole, ResolvedAgentFlowTool, SaveAgentMessagesRequest, WorkerJobType } from '@activepieces/shared'+import { createUIMessageStream, generateText, LanguageModel, ModelMessage, streamText, ToolSet, toUIMessageStream } from 'ai'import { FireAndForgetJobResult, JobContext, JobHandler, JobResultKind } from '../../../types'import { toResolvedAiFile } from '../../ai/ai-files'import { agentMcpClient, McpConnection } from './agent-mcp-client'import { stepResultFrom } from './agent-step-result'import { agentToolPolicy } from './agent-tool-policy'import { agentWorkerTools, GateDecision, TaintState } from './agent-worker-tools'-import { classifyAgentRunError, delayWithJitter, firstStepUsesFastModel, isTransientFailureText, runAgentTurn } from './run-agent-turn'+import { classifyAgentRunError, delayWithJitter, drainOf, firstStepUsesFastModel, isTransientFailureText, NO_STEP_CONTENT, runAgentTurn, StreamDrain, trackStepContent, TurnModel } from './run-agent-turn'const BATCH_SIZE = 10const BATCH_FLUSH_MS = 50@@ -65,6 +65,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForgelet heartbeatInterval: NodeJS.Timeout | undefinedlet runProvider: string | undefinedlet runModelId: string | undefined+ let tierFailureMessage: string | undefinedlet answer: AgentResult | undefinedlet configLoaded = falseconst structured: { output?: Record<string, unknown> } = {}@@ -98,6 +99,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge...spreadIfDefined('flowRunId', flowRunId),...spreadIfDefined('provider', data.provider),...spreadIfDefined('providerConfigId', data.providerConfigId),+ ...spreadIfDefined('modelTierId', data.modelTierId),...spreadIfDefined('projectId', projectId),...spreadIfDefined('promptOverride', promptOverride),...spreadIfDefined('dryRun', dryRun),@@ -135,6 +137,12 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForgebilling,turnAlreadyCharged: true,})+ const turnModels = turnModelsFor({ config, model, billing, metadata: { platformId, conversationId, runId } })+ const fastTurnModel: TurnModel | undefined = isNil(config.platformTier)+ ? { model: fastModel, provider, modelId: config.fastModelId, thinkingBudget: config.tier.thinkingBudget }+ : isNil(config.fastCandidate) ? undefined : candidateTurnModel({ candidate: config.fastCandidate, billing })+ let answering: TurnModel = turnModels[0]+ const runKey = (): RunKey => ({ provider: answering.provider, providerConfigId: answering.key?.providerConfigId ?? config.providerConfigId, modelId: answering.modelId })log.info({ provider, model: { id: config.modelId }, tier: { id: config.tier.id }, dryRun: dryRun ?? false, tavilySearchActive, providerSearchActive }, '[executeAgentRun] Chat config loaded')@@ -216,9 +224,8 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge}const allTools = buildToolSet({- provider,- providerConfigId: config.providerConfigId,- modelId: config.modelId,+ runKey,+ knowledgeBaseKey: isNil(config.platformTier) ? { provider, providerConfigId: config.providerConfigId } : {},ctx, eventEmitter, log, phaseState, taintState, mcpToolSet, webTools,projects: config.projects, projectId, conversationId, runId, ...spreadIfDefined('flowRunId', flowRunId), platformId, userId, userEmail: config.userEmail,guides: config.guides, dryRun: dryRun ?? false, discoveryOnly: discoveryOnly ?? false,@@ -254,22 +261,34 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForgereturn runAgentTurn({...spreadIfDefined('stepCeiling', data.maxSteps),creditsLeft: (pendingCredits) => ctx.apiClient.agentCreditsLeft({ platformId, conversationId, pendingCredits }),- model,- fastModel: firstStepUsesFastModel({ source, dryRun, runsASavedAgent: !isNil(data.promptOverride) }) ? fastModel : undefined,- provider,+ models: turnModels,+ fastModel: firstStepUsesFastModel({ source, dryRun, runsASavedAgent: !isNil(data.promptOverride) }) ? fastTurnModel : undefined,+ onStepModel: (stepModel) => {+ if (isNil(stepModel.key)) {+ return+ }+ answering = stepModel+ runProvider = stepModel.provider+ runModelId = stepModel.modelId+ },+ onModelOutcome: ({ turnModel, signal }) => {+ if (isNil(turnModel.key)) {+ return+ }+ ctx.apiClient.reportAiKeyOutcome({ platformId, providerConfigId: turnModel.key.providerConfigId, signal }).catch(() => undefined)+ },systemPrompt: config.systemPrompt,messages: config.messages as ModelMessage[],tools: mergedTools,allToolNames,tier: config.tier,- modelId: config.modelId,- ...spreadIfDefined('fastModelId', dryRun ? undefined : config.fastModelId),phaseState,abortSignal: abortController.signal,log,sinks: {drainStream: (result) => streamChunksToClient({result, ctx, userId, conversationId, runId, log,+ holdErrorChunks: !isNil(config.platformTier),abortSignal: abortController.signal,onStreamIdle: (reason) => {const fields = { conversation: { id: conversationId }, idleMs: STREAM_IDLE_REPORT_MS, reason }@@ -318,6 +337,7 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge...(config.previousUiMessages as PersistedAgentMessage[]),...(uiParts.length > 0 ? [{ role: PersistedAgentRole.ASSISTANT, parts: uiParts, thinkingDurationMs, tainted: taintState.readInThisReply() }] : []),],+ ...answeredByOf({ config, answeredBy: turn.answeredBy }),}const { error: cancelSaveError } = await tryCatch(() => ctx.apiClient.saveAgentMessages(cancelSavePayload))if (cancelSaveError) {@@ -338,6 +358,9 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge}if (streamError) {+ tierFailureMessage = turn.allModelsFailed && !isNil(config.platformTier)+ ? `All ${turnModels.length} models in tier ${config.platformTier.name} failed (${turnModels.map((candidate) => candidate.modelId).join(', ')}). Last error: ${apErrorOf(streamError)?.message ?? formatPieceError(streamError).message}`+ : undefinedthrow streamError}@@ -350,7 +373,8 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForgeoutputTokens: totalOutputTokens,...spreadIfDefined('cacheReadTokens', usage?.inputTokenDetails?.cacheReadTokens),...spreadIfDefined('cacheWriteTokens', usage?.inputTokenDetails?.cacheWriteTokens),- provider: config.credentials.provider,+ provider: turn.answeredBy.provider,+ model: { id: turn.answeredBy.modelId },finishReason: turn.finishReason,truncatedAfterRetries,}, 'Chat message completed')@@ -365,7 +389,8 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge...(uiParts.length > 0 ? [{ role: PersistedAgentRole.ASSISTANT, parts: uiParts, thinkingDurationMs, tainted: taintState.readInThisReply() }] : []),],...spreadIfDefined('title', autoTitle),- ...spreadIfDefined('modelName', isNil(data.modelName) ? config.tier.id : undefined),+ ...spreadIfDefined('modelName', isNil(data.modelName) && isNil(config.platformTier) ? config.tier.id : undefined),+ ...answeredByOf({ config, answeredBy: turn.answeredBy }),}await retryWithBackoff({ fn: () => ctx.apiClient.saveAgentMessages(savePayload), description: 'Saving the transcript', throwOnExhausted: true, log })@@ -409,11 +434,13 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForgeconst errorMessage = apErrorOf(err)?.message ?? formatPieceError(err).messageconst isCreditError = errorClass === 'credit'// "User not found" is OpenRouter refusing a key, and reads like a missing account.- const clientMessage = !isCreditError && isTransientFailureText(errorMessage)- ? 'The AI provider is temporarily unavailable. Please try again in a moment.'- : isCreditError || isNil(runProvider)- ? errorMessage- : `${runProvider}${isNil(runModelId) ? '' : ` (${runModelId})`}: ${errorMessage}`+ const clientMessage = !isNil(tierFailureMessage)+ ? tierFailureMessage+ : !isCreditError && isTransientFailureText(errorMessage)+ ? 'The AI provider is temporarily unavailable. Please try again in a moment.'+ : isCreditError || isNil(runProvider)+ ? errorMessage+ : `${runProvider}${isNil(runModelId) ? '' : ` (${runModelId})`}: ${errorMessage}`const failedResult = { ...(answer ?? stepResultFrom({ prompt: userMessage, uiParts: [], timestamp: new Date().toISOString(), tools: reportedTools, structuredOutput: structured.output, failure: clientMessage })), failure: clientMessage }reportFinal(failedResult)const { error: releaseError } = await tryCatch(() => releaseFlowStep({ ctx, conversationId, flowRunId, waitpointId, output: failedResult, source, log }))@@ -458,6 +485,32 @@ export const executeAgentRunJob: JobHandler<ExecuteAgentRunJobData, FireAndForge},}+function answeredByOf({ config, answeredBy }: { config: AgentConfigResponse, answeredBy: TurnModel }): Pick<SaveAgentMessagesRequest, 'answeredBy'> {+ return isNil(config.platformTier) ? {} : { answeredBy: { provider: answeredBy.provider, modelId: answeredBy.modelId } }+}++function turnModelsFor({ config, model, billing, metadata }: { config: AgentConfigResponse, model: LanguageModel, billing: ActivepiecesAiBilling, metadata: ModelMetadata }): [TurnModel, ...TurnModel[]] {+ const single: TurnModel = { model, provider: config.credentials.provider, modelId: config.modelId, thinkingBudget: config.tier.thinkingBudget }+ if (isNil(config.candidates) || config.candidates.length === 0) {+ return [single]+ }+ const [first, ...rest] = config.candidates+ return [+ { ...candidateTurnModel({ candidate: first, billing }), model },+ ...rest.map((candidate) => candidateTurnModel({ candidate, billing, metadata })),+ ]+}++function candidateTurnModel({ candidate, billing, metadata }: { candidate: AgentModelCandidate, billing: ActivepiecesAiBilling, metadata?: ModelMetadata }): TurnModel {+ return {+ model: aiUtils.createModel({ credentials: candidate, modelId: candidate.modelId, billing, turnAlreadyCharged: true, ...spreadIfDefined('metadata', metadata) }),+ provider: candidate.provider,+ modelId: candidate.modelId,+ thinkingBudget: candidate.thinkingBudget,+ key: { providerConfigId: candidate.providerConfigId, status: candidate.status },+ }+}+function incompleteReason({ truncatedAfterRetries, budgetExceeded, creditsExhausted }: { truncatedAfterRetries: boolean, budgetExceeded: boolean, creditsExhausted: boolean }): string | undefined {if (truncatedAfterRetries) {return 'The response reached the output limit before the agent finished'@@ -534,11 +587,10 @@ function pickImageGenerator({ falApiKey, imageModelId, credentials, billing, rea}}-function buildToolSet({ ctx, eventEmitter, log, phaseState, taintState, mcpToolSet, webTools, projects, projectId, conversationId, flowRunId, runId, platformId, userId, userEmail, guides, dryRun, discoveryOnly, emailEnabled, agentsAvailable, abortSignal, source, provider, providerConfigId, modelId, configuredPieceTools, configuredFlowTools, configuredKnowledgeBaseTools, structuredOutput, captureStructured }: {+function buildToolSet({ ctx, eventEmitter, log, phaseState, taintState, mcpToolSet, webTools, projects, projectId, conversationId, flowRunId, runId, platformId, userId, userEmail, guides, dryRun, discoveryOnly, emailEnabled, agentsAvailable, abortSignal, source, runKey, knowledgeBaseKey, configuredPieceTools, configuredFlowTools, configuredKnowledgeBaseTools, structuredOutput, captureStructured }: {ctx: JobContext- provider: AIProviderName- providerConfigId: string- modelId: string+ runKey: () => RunKey+ knowledgeBaseKey: { provider?: AIProviderName, providerConfigId?: string }eventEmitter: ReturnType<typeof agentWorkerTools.createEventEmitter>log: JobContext['log']phaseState: { phase: AgentPhase }@@ -634,7 +686,7 @@ function buildToolSet({ ctx, eventEmitter, log, phaseState, taintState, mcpToolSconst selectedConnectionByPiece = new Map<string, string>()const localTools = agentWorkerTools.createLocalTools({onSetProjectContext: async (projectId) => {- const { error } = await tryCatch(() => ctx.apiClient.updateProjectContext({ conversationId, runId, projectId, provider, providerConfigId }))+ const { error } = await tryCatch(() => ctx.apiClient.updateProjectContext({ conversationId, runId, projectId, provider: runKey().provider, providerConfigId: runKey().providerConfigId }))if (!isNil(error)) {log.warn({ error, conversation: { id: conversationId }, project: projectId ? { id: projectId } : undefined }, '[executeAgentRun] Refused a project switch')return { success: false, error: 'Could not switch to that project on this chat — the AI provider key this chat runs on is not available there. Start a new chat in that project instead.' }@@ -744,7 +796,7 @@ function buildToolSet({ ctx, eventEmitter, log, phaseState, taintState, mcpToolS// answer, and an agent that asks an empty room reads the silence as a refusal and stops.const configuredTools = agentWorkerTools.createConfiguredPieceTools({tools: dryRun || discoveryOnly ? [] : configuredPieceTools,- runPieceTool: ({ toolName, instruction, piece }) => ctx.apiClient.executePieceTool({ conversationId, ...spreadIfDefined('runId', runId), toolName, instruction, piece, provider, providerConfigId, modelId, ...spreadIfDefined('flowRunId', flowRunId) }),+ runPieceTool: ({ toolName, instruction, piece }) => ctx.apiClient.executePieceTool({ conversationId, ...spreadIfDefined('runId', runId), toolName, instruction, piece, ...runKey(), ...spreadIfDefined('flowRunId', flowRunId) }),taintState,eventEmitter,log,@@ -758,7 +810,7 @@ function buildToolSet({ ctx, eventEmitter, log, phaseState, taintState, mcpToolSconst knowledgeBaseTools = agentWorkerTools.createConfiguredKnowledgeBaseTools({taintState,tools: dryRun || discoveryOnly ? [] : configuredKnowledgeBaseTools,- runKnowledgeBaseTool: ({ toolName, knowledgeBaseFileId, query }) => ctx.apiClient.executeKnowledgeBaseTool({ conversationId, ...spreadIfDefined('runId', runId), toolName, knowledgeBaseFileId, query, provider, providerConfigId }),+ runKnowledgeBaseTool: ({ toolName, knowledgeBaseFileId, query }) => ctx.apiClient.executeKnowledgeBaseTool({ conversationId, ...spreadIfDefined('runId', runId), toolName, knowledgeBaseFileId, query, ...spreadIfDefined('provider', knowledgeBaseKey.provider), ...spreadIfDefined('providerConfigId', knowledgeBaseKey.providerConfigId) }),log,})const completionTool = structuredOutput.length === 0@@ -785,7 +837,7 @@ function buildToolSet({ ctx, eventEmitter, log, phaseState, taintState, mcpToolS})}-async function streamChunksToClient({ result, ctx, userId, conversationId, runId, log, abortSignal, onStreamIdle }: {+async function streamChunksToClient({ result, ctx, userId, conversationId, runId, log, abortSignal, onStreamIdle, holdErrorChunks }: {result: ReturnType<typeof streamText>ctx: JobContextuserId: string@@ -794,8 +846,10 @@ async function streamChunksToClient({ result, ctx, userId, conversationId, runIdlog: JobContext['log']abortSignal: AbortSignalonStreamIdle: (reason: 'idle' | 'tool' | 'reasoning') => void-}): Promise<void> {+ holdErrorChunks: boolean+}): Promise<StreamDrain> {let chunkBuffer: unknown[] = []+ let stepContent = NO_STEP_CONTENTlet flushTimer: ReturnType<typeof setTimeout> | null = nullconst flushChunks = async () => {@@ -860,6 +914,10 @@ async function streamChunksToClient({ result, ctx, userId, conversationId, runIdelse if (chunkType === 'reasoning-end') {reasoningInFlight = false}+ stepContent = trackStepContent({ state: stepContent, chunkType })+ if (holdErrorChunks && chunkType === 'error') {+ continue+ }chunkBuffer.push(chunk)if (chunkBuffer.length >= BATCH_SIZE) {if (flushTimer) {@@ -887,6 +945,7 @@ async function streamChunksToClient({ result, ctx, userId, conversationId, runId}if (flushTimer) clearTimeout(flushTimer)await flushChunks()+ return drainOf({ state: stepContent })}async function generateTitleIfFirstTurn({ model, userMessage, previousUiMessages, log, conversationId, abortSignal }: {@@ -956,3 +1015,11 @@ async function retryWithBackoff({ fn, maxAttempts = RETRY_MAX_ATTEMPTS, descript}type ImageToolParams = Parameters<typeof agentWorkerTools.createImageTools>[0]++type ModelMetadata = Parameters<typeof aiUtils.createModel>[0]['metadata']++type RunKey = {+ provider: AIProviderName+ providerConfigId: string+ modelId: string+}diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.tsindex 41dd461f8e3..51520d5f65f 100644--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts@@ -1,5 +1,5 @@-import { AIProviderName, ErrorCode, formatPieceError, isNil, isObject, isProviderBillingError, isTransientProviderError, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'-import { agentAiUtils, ContentPartLike } from '@activepieces/server-utils'+import { AiProviderKeyStatus, AIProviderName, ErrorCode, formatPieceError, isFallbackWorthy, isNil, isObject, isProviderBillingError, isTransientProviderError, ProviderOutcomeSignal, spreadIfDefined, tryCatch, tryCatchSync, unique } from '@activepieces/core-utils'+import { agentAiUtils, aiProviderSignal, ContentPartLike } from '@activepieces/server-utils'import { AgentPhase, AgentRunSource, agentToolClassification, agentToolPhases, AI_PROVIDER_ENTITY_TYPES, aiProviderUtils, apErrorOf, CHAT_CREDITS_PER_TOOL_CALL, chatBilling, ChatToolCall, PersistedAgentPart, TASK_COMPLETION_TOOL_NAME } from '@activepieces/shared'import { APICallError, generateText, isLoopFinished, isStepCount, LanguageModel, LanguageModelUsage, ModelMessage, NoSuchToolError, RetryError, StepResultPerformance, StopCondition, streamText, ToolChoice, ToolExecutionOptions, ToolSet } from 'ai'@@ -10,6 +10,10 @@ const MAX_IDENTICAL_TOOL_FAILURES = 2const IN_LOOP_COMPACTION_THRESHOLD = 0.6const RUNAWAY_TURN_CONTEXT_MULTIPLE = 90const STREAM_RETRY_BASE_DELAY_MS = 1_000+const SDK_RETRIES = 3+const SDK_RETRIES_BEFORE_FALLBACK = 1+const MIN_THINKING_BUDGET = 1_024+const CONTENT_CHUNK_TYPES = new Set(['text-delta', 'reasoning-delta', 'tool-input-start', 'tool-input-available', 'source-url', 'source-document', 'file'])const USER_FAULT_STATUS_CODES = new Set([401, 403, 404])const MODEL_UNAVAILABLE_PATTERNS = [/\bis deprecated\b/i, /no longer (available|supported)/i, /\bmodel_not_found\b/i, /\bunknown model\b/i, /\bdecommissioned\b/i]const USER_CONFIG_ENTITY_TYPES = new Set<string>(Object.values(AI_PROVIDER_ENTITY_TYPES))@@ -41,8 +45,25 @@ export function shouldRetryStream({ producedVisibleOutput, streamRetries }: {return !producedVisibleOutput && streamRetries < MAX_STREAM_RETRIES}-export async function runAgentTurn({ model, fastModel, provider, systemPrompt, messages, tools, allToolNames, tier, modelId, fastModelId, phaseState, abortSignal, log, sinks, stopWhen, stepCeiling, creditsLeft }: RunAgentTurnParams): Promise<AgentTurnResult> {- const drainStream = sinks?.drainStream ?? (async () => {})+export function trackStepContent({ state, chunkType }: { state: StepContentState, chunkType: string | undefined }): StepContentState {+ if (!isNil(state.sentWhenFailed)) {+ return state+ }+ if (chunkType === 'error') {+ return { ...state, sentWhenFailed: state.sentInStep }+ }+ if (chunkType === 'start-step' || chunkType === 'finish-step') {+ return { ...state, sentInStep: false }+ }+ return { ...state, sentInStep: state.sentInStep || (!isNil(chunkType) && CONTENT_CHUNK_TYPES.has(chunkType)) }+}++export function drainOf({ state }: { state: StepContentState }): StreamDrain {+ return { lastStepSentContent: state.sentWhenFailed ?? state.sentInStep }+}++export async function runAgentTurn({ models, fastModel, systemPrompt, messages, tools, allToolNames, tier, phaseState, abortSignal, log, sinks, stopWhen, stepCeiling, creditsLeft, onModelOutcome, onStepModel }: RunAgentTurnParams): Promise<AgentTurnResult> {+ const drainStream = sinks?.drainStream ?? (async () => undefined)const onProgress = sinks?.onProgress ?? (() => {})const baseStopCondition = stopWhen ?? isLoopFinished()let creditsExhausted = false@@ -52,7 +73,7 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mif (isNil(creditsLeft)) {return false}- const pendingCredits = chatBilling.creditsForTurn({ provider, toolCalls: [...earlierAttemptToolCalls, ...completedToolCalls(steps)] }).total+ const pendingCredits = chatBilling.creditsForTurn({ provider: current().provider, toolCalls: [...earlierAttemptToolCalls, ...completedToolCalls(steps)] }).totalconst { data: left, error } = await tryCatch(() => creditsLeft(pendingCredits))if (error) {log.warn({ error }, 'Credit check failed mid-turn, letting the turn continue')@@ -67,7 +88,16 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mcreditsRanOut,]const guardedTools = wrapToolsWithFailureGuard({ tools, log })- const maxTurnTokens = runawayTokenCeiling(provider)+ const maxTurnTokens = runawayTokenCeiling(models[0].provider)+ const runsATier = models.some((candidate) => !isNil(candidate.key))+ const peers = [...models, ...(isNil(fastModel) ? [] : [fastModel])]+ const ranked = await Promise.all(models.map((turnModel) => withOutputBudget({ turnModel, peers })))+ let fast = isNil(fastModel) ? undefined : await withOutputBudget({ turnModel: fastModel, peers })+ let modelIndex = 0+ let lastStepUsedFast = false+ const failedModelIds = new Set<string>()+ const current = (): BudgetedTurnModel => ranked[modelIndex]+ const hasNextModel = (): boolean => modelIndex < ranked.length - 1const uiParts: PersistedAgentPart[] = []const toolCalls: AgentTurnToolCall[] = []@@ -80,7 +110,6 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mlet continuations = 0let emptyContinuations = 0let streamRetries = 0- let lastStepModelId = modelIdlet truncatedAfterRetries = falselet usage: LanguageModelUsage | undefinedlet totalInputTokens = 0@@ -88,16 +117,13 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mlet lastFinishReason = ''let budgetExceeded = false- const maxOutputTokens = await agentAiUtils.affordableOutputTokens({ provider, modelIds: [modelId, fastModelId], thinkingBudget: tier.thinkingBudget })- const maxOutputTokensWithoutThinking = agentAiUtils.clampOutputTokens({ thinkingBudget: 0, ceilings: [maxOutputTokens] })-const runStreamAttempt = (attemptMessages: ModelMessage[]): ReturnType<typeof streamText> => streamText({- model,- maxRetries: 3,- maxOutputTokens,+ model: current().model,+ maxRetries: runsATier && hasNextModel() ? SDK_RETRIES_BEFORE_FALLBACK : SDK_RETRIES,+ maxOutputTokens: current().budget.maxOutputTokens,abortSignal,- instructions: agentAiUtils.buildSystemPromptWithCaching({ systemPrompt, provider }),- messages: agentAiUtils.stripThinkingBlocks(attemptMessages, provider),+ instructions: agentAiUtils.buildSystemPromptWithCaching({ systemPrompt, provider: current().provider }),+ messages: agentAiUtils.stripThinkingBlocks(attemptMessages, current().provider),tools: guardedTools,stopWhen: loopStopCondition,telemetry: agentAiUtils.buildTelemetry({ functionId: 'agent-turn' }),@@ -127,20 +153,24 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, m? (forcesCompletion ? { type: 'tool', toolName: TASK_COMPLETION_TOOL_NAME } : 'none'): undefinedconst disableThinking = isFirstStep || phaseState.phase === 'discovery' || forcesCompletion- const usesFastModel = isFirstStep && !isNil(fastModel)+ const stepModel = isFirstStep && !isNil(fast) ? fast : current()+ const usesFastModel = stepModel !== current()+ const stepBudget = stepModel.budgetconst phaseTools = agentToolPhases.activeToolsForPhase({ phase: phaseState.phase, allToolNames }).filter((name) => paidToolsAffordable || !chatBilling.isPaidTool(name))const activeTools = forcesCompletion ? [TASK_COMPLETION_TOOL_NAME] : phaseTools- const boundedContext = boundContextForStep({ baseMessages: attemptMessages, steps, systemPrompt, provider })+ const boundedContext = boundContextForStep({ baseMessages: attemptMessages, steps, systemPrompt, provider: stepModel.provider })const stepContext = isLastAllowedStep? { messages: [...(boundedContext.messages ?? currentMessages), FINAL_STEP_MESSAGE] }: boundedContext- lastStepModelId = usesFastModel ? fastModelId ?? modelId : modelId+ lastStepUsedFast = usesFastModel+ onStepModel?.(stepModel)+ const thinkingOff = disableThinking || stepBudget.thinkingBudget === 0return {- ...(usesFastModel ? { model: fastModel } : {}),+ ...(usesFastModel ? { model: stepModel.model } : {}),activeTools,...spreadIfDefined('toolChoice', toolChoice),- maxOutputTokens: disableThinking ? maxOutputTokensWithoutThinking : maxOutputTokens,- providerOptions: agentAiUtils.buildProviderOptions({ provider, tier, modelId: lastStepModelId, disableThinking }),+ maxOutputTokens: thinkingOff ? stepBudget.maxOutputTokensWithoutThinking : stepBudget.maxOutputTokens,+ providerOptions: agentAiUtils.buildProviderOptions({ provider: stepModel.provider, tier: { id: tier.id, thinkingBudget: stepBudget.thinkingBudget }, modelId: stepModel.modelId, disableThinking: thinkingOff }),...stepContext,}},@@ -152,7 +182,7 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mlog.warn({ toolName: toolCall.toolName, error }, 'Repairing malformed tool call')const { data: repaired } = await tryCatch(async () => {const { text } = await generateText({- model,+ model: current().model,abortSignal,telemetry: agentAiUtils.buildTelemetry({ functionId: 'agent-tool-repair' }),prompt: `Fix this malformed JSON tool call for "${toolCall.toolName}". The error was: ${error.message}\n\nOriginal input:\n${toolCall.input}\n\nReturn ONLY the corrected JSON input, nothing else.`,@@ -203,7 +233,7 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mconst uiPartsCountBefore = uiParts.lengthcurrentAttemptMessages = []const result = runStreamAttempt(llmMessages)- await drainStream(result)+ const drained = await drainStream(result)const producedVisibleOutput = uiParts.length > uiPartsCountBefore// On abort/error we leave the loop WITHOUT reaching the clean-exit pushes below, so fold// this attempt's completed steps in here — otherwise the turn's work is lost from the@@ -213,7 +243,29 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mbreak}if (streamError) {- if (shouldRetryStream({ producedVisibleOutput, streamRetries })) {+ const signal = aiProviderSignal.fromError(streamError)+ const failedModel = lastStepUsedFast && !isNil(fast) ? fast : current()+ reportOutcome({ turnModel: failedModel, signal, onModelOutcome })+ const failedFast = failedModel === fast+ if (!failedFast) {+ failedModelIds.add(failedModel.modelId)+ }+ const sentNothing = runsATier && drained?.lastStepSentContent === false+ const dropsFast = failedFast && sentNothing+ const movesOn = !failedFast && sentNothing && isFallbackWorthy(signal) && hasNextModel()+ if (dropsFast || movesOn) {+ if (!failedFast) {+ modelIndex+++ }+ log.warn({ error: streamError, failedModel: { id: failedModel.modelId, fast: failedFast }, model: { id: current().modelId }, candidateIndex: modelIndex }, 'Chat model failed before sending anything — continuing the turn on the next model')+ fast = undefined+ accumulatedResponseMessages.push(...currentAttemptMessages)+ llmMessages = [...llmMessages, ...currentAttemptMessages]+ streamError = null+ streamRetries = 0+ continue+ }+ if (shouldRetryStream({ producedVisibleOutput: producedVisibleOutput || drained?.lastStepSentContent === true, streamRetries })) {streamRetries++log.warn({ streamRetries, error: streamError }, 'Chat stream failed before any visible output — retrying the turn')streamError = null@@ -239,7 +291,7 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mtotalInputTokens += attemptUsage.inputTokens ?? 0totalOutputTokens += attemptUsage.outputTokens ?? 0lastFinishReason = finishReason- logTurnPerformance({ performance: finalStep.performance, modelId: tier.modelId, stepCount: steps.length, log })+ logTurnPerformance({ performance: finalStep.performance, modelId: current().modelId, stepCount: steps.length, log })if (creditsExhausted) {accumulatedResponseMessages.push(...stepMessages)@@ -280,6 +332,13 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mllmMessages = [...llmMessages, ...sanitizedTail, { role: 'user', content: EMPTY_OUTPUT_NUDGE }]}+ if (isNil(streamError) && !abortSignal.aborted && current().key?.status !== 'active') {+ reportOutcome({ turnModel: current(), signal: { statusCode: 200 }, onModelOutcome })+ }+ if (modelIndex > 0) {+ log.info({ candidateIndex: modelIndex, model: { id: current().modelId } }, 'A fallback model answered the chat turn')+ }+return {accumulatedResponseMessages,uiParts,@@ -293,7 +352,32 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, mtotalInputTokens,totalOutputTokens,toolCalls,+ answeredBy: current(),+ allModelsFailed: !isNil(streamError) && runsATier && failedModelIds.size === unique(ranked.map((candidate) => candidate.modelId)).length,+ }+}++async function withOutputBudget({ turnModel, peers }: { turnModel: TurnModel, peers: TurnModel[] }): Promise<BudgetedTurnModel> {+ const sameProviderIds = unique(peers.filter((peer) => peer.provider === turnModel.provider).map((peer) => peer.modelId))+ const maxOutputTokens = await agentAiUtils.affordableOutputTokens({ provider: turnModel.provider, modelIds: sameProviderIds, thinkingBudget: turnModel.thinkingBudget })+ const roomForThinking = Math.min(turnModel.thinkingBudget, maxOutputTokens - MIN_THINKING_BUDGET)+ const thinkingBudget = isNil(turnModel.key) ? turnModel.thinkingBudget : roomForThinking < MIN_THINKING_BUDGET ? 0 : roomForThinking+ return {+ ...turnModel,+ budget: {+ maxOutputTokens,+ maxOutputTokensWithoutThinking: agentAiUtils.clampOutputTokens({ thinkingBudget: 0, ceilings: [maxOutputTokens] }),+ thinkingBudget,+ },+ }+}++function reportOutcome({ turnModel, signal, onModelOutcome }: { turnModel: TurnModel, signal: ProviderOutcomeSignal, onModelOutcome?: (outcome: { turnModel: TurnModel, signal: ProviderOutcomeSignal }) => void }): void {+ const comesFromProvider = signal.fromProvider === true || signal.statusCode === 200+ if (isNil(turnModel.key) || isNil(onModelOutcome) || !comesFromProvider) {+ return}+ onModelOutcome({ turnModel, signal })}// timeToFirstOutputMs is the number the fast-model-first-step swap in prepareStep exists to buy;@@ -493,21 +577,45 @@ export type AgentTurnToolCall = {}export type AgentTurnSinks = {- drainStream?: (result: ReturnType<typeof streamText>) => Promise<void>+ drainStream?: (result: ReturnType<typeof streamText>) => Promise<StreamDrain | undefined>onProgress?: (progress: { uiParts: PersistedAgentPart[], responseMessages: ModelMessage[] }) => void}-export type RunAgentTurnParams = {+export type StreamDrain = {+ lastStepSentContent: boolean+}++export type StepContentState = {+ sentInStep: boolean+ sentWhenFailed: boolean | null+}++export const NO_STEP_CONTENT: StepContentState = { sentInStep: false, sentWhenFailed: null }++export type TurnModel = {model: LanguageModel- fastModel?: LanguageModelprovider: AIProviderName+ modelId: string+ thinkingBudget: number+ key?: { providerConfigId: string, status: AiProviderKeyStatus }+}++type BudgetedTurnModel = TurnModel & {+ budget: {+ maxOutputTokens: number+ maxOutputTokensWithoutThinking: number+ thinkingBudget: number+ }+}++export type RunAgentTurnParams = {+ models: [TurnModel, ...TurnModel[]]+ fastModel?: TurnModelsystemPrompt: stringmessages: ModelMessage[]tools: ToolSetallToolNames: string[]tier: { id: string, thinkingBudget: number, modelId: string }- modelId: string- fastModelId?: stringphaseState: { phase: AgentPhase }abortSignal: AbortSignallog: AgentTurnLogger@@ -515,6 +623,8 @@ export type RunAgentTurnParams = {stopWhen?: StopCondition<ToolSet> | Array<StopCondition<ToolSet>>stepCeiling?: numbercreditsLeft?: (pendingCredits: number) => Promise<number | null>+ onModelOutcome?: (outcome: { turnModel: TurnModel, signal: ProviderOutcomeSignal }) => void+ onStepModel?: (turnModel: TurnModel) => void}export type AgentTurnResult = {@@ -530,6 +640,8 @@ export type AgentTurnResult = {totalInputTokens: numbertotalOutputTokens: numbertoolCalls: AgentTurnToolCall[]+ answeredBy: TurnModel+ allModelsFailed: boolean}type AgentRunErrorClass = 'credit' | 'user' | 'provider' | 'internal'diff --git a/packages/server/worker/test/lib/agent-eval/agent-tool-call-repair.test.ts b/packages/server/worker/test/lib/agent-eval/agent-tool-call-repair.test.tsindex 65aa1793a54..e56aef5ee56 100644--- a/packages/server/worker/test/lib/agent-eval/agent-tool-call-repair.test.ts+++ b/packages/server/worker/test/lib/agent-eval/agent-tool-call-repair.test.ts@@ -41,8 +41,7 @@ async function turnWhereTheModelRepairsWith(repairText: string): Promise<unknown})await runAgentTurn({- model,- provider: AIProviderName.ANTHROPIC,+ models: [{ model, provider: AIProviderName.ANTHROPIC, modelId: TIER.modelId, thinkingBudget: TIER.thinkingBudget }],systemPrompt: 'You are a test agent.',messages: [{ role: 'user', content: 'search the web' }],tools: {@@ -57,7 +56,6 @@ async function turnWhereTheModelRepairsWith(repairText: string): Promise<unknown} as never,allToolNames: ['ap_web_search'],tier: TIER,- modelId: TIER.modelId,phaseState: { phase: 'discovery' },abortSignal: new AbortController().signal,log: silentLog,diff --git a/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts b/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.tsindex 3d24b60827b..0f754e86f51 100644--- a/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts+++ b/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts@@ -100,8 +100,7 @@ async function runTurn({ search, creditsLeft, stepCeiling = 20, model = alwaysSe? {}: { updateTaskStatus: tool({ description: 'report the result', inputSchema: z.object({ output: z.object({ summary: z.string() }) }), execute: async (input) => { completion(input); return SEARCH_RESULT } }) }return runAgentTurn({- model,- provider: AIProviderName.ANTHROPIC,+ models: [{ model, provider: AIProviderName.ANTHROPIC, modelId: TIER.modelId, thinkingBudget: TIER.thinkingBudget }],systemPrompt: 'You are a test agent.',messages: [{ role: 'user', content: 'research this' }],tools: {@@ -111,12 +110,11 @@ async function runTurn({ search, creditsLeft, stepCeiling = 20, model = alwaysSe},allToolNames: ['ap_web_search', 'ap_fetch_url', ...Object.keys(completionTools)],tier: TIER,- modelId: TIER.modelId,phaseState: { phase },abortSignal: new AbortController().signal,log: SILENT_LOG,...spreadIfDefined('stepCeiling', stepCeiling ?? undefined),- ...(drainsStream ? { sinks: { drainStream: (result) => result.consumeStream() } } : {}),+ ...(drainsStream ? { sinks: { drainStream: async (result) => { await result.consumeStream(); return undefined } } } : {}),creditsLeft,})}diff --git a/packages/server/worker/test/lib/agent-eval/agent-turn-provider-options.test.ts b/packages/server/worker/test/lib/agent-eval/agent-turn-provider-options.test.tsindex 185cb5d9515..69456ebe156 100644--- a/packages/server/worker/test/lib/agent-eval/agent-turn-provider-options.test.ts+++ b/packages/server/worker/test/lib/agent-eval/agent-turn-provider-options.test.ts@@ -26,14 +26,12 @@ async function capturedProviderOptions({ provider, modelId }: { provider: AIProv})await runAgentTurn({- model,- provider,+ models: [{ model, provider, modelId: modelId, thinkingBudget: TIER.thinkingBudget }],systemPrompt: 'You are a test agent.',messages: [{ role: 'user', content: 'hello' }],tools: {},allToolNames: [],tier: TIER,- modelId,phaseState: { phase: 'discovery' },abortSignal: new AbortController().signal,log: silentLog,diff --git a/packages/server/worker/test/lib/agent-eval/core/runner.ts b/packages/server/worker/test/lib/agent-eval/core/runner.tsindex 1f7218a6f14..c84556c8fb3 100644--- a/packages/server/worker/test/lib/agent-eval/core/runner.ts+++ b/packages/server/worker/test/lib/agent-eval/core/runner.ts@@ -201,14 +201,12 @@ async function runTurn({ fixture, systemPrompt, guides, auth }: { fixture: ChatEconst capturedErrors: unknown[] = []const { data: result, error: turnError } = await tryCatch(() => runAgentTurn({- model,- provider: fixture.model.provider,+ models: [{ model, provider: fixture.model.provider, modelId: fixture.model.tier.modelId, thinkingBudget: fixture.model.tier.thinkingBudget }],systemPrompt: systemPrompt ?? evalPrompts.loadSystemPrompt(),messages,tools,allToolNames: Object.keys(tools),tier: fixture.model.tier,- modelId: fixture.model.tier.modelId,phaseState,abortSignal: new AbortController().signal,log: {diff --git a/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-fallback.test.ts b/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-fallback.test.tsnew file mode 100644index 00000000000..3a2f852f1de--- /dev/null+++ b/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-fallback.test.ts@@ -0,0 +1,252 @@+import { AIProviderName, isObject } from '@activepieces/core-utils'+import { SharedV3ProviderOptions } from '@ai-sdk/provider'+import { APICallError, streamText, tool, toUIMessageStream } from 'ai'+import { convertArrayToReadableStream, MockLanguageModelV3 } from 'ai/test'+import { describe, expect, it, vi } from 'vitest'+import { z } from 'zod'+import { drainOf, NO_STEP_CONTENT, runAgentTurn, StepContentState, StreamDrain, trackStepContent, TurnModel } from '../../../../../../src/lib/execute/jobs/ee/agent/run-agent-turn'++describe('a platform tier chat turn', () => {+ it('answers on the next model when the main model fails before sending anything', async () => {+ const onStepModel = vi.fn()++ const turn = await runTurn({ models: [tierModel({ modelId: 'main', model: failingModel({ statusCode: 503 }) }), tierModel({ modelId: 'backup', model: answeringModel('from backup') })], onStepModel })++ expect(turn.streamError).toBeNull()+ expect(turn.answeredBy.modelId).toBe('backup')+ expect(onStepModel).toHaveBeenLastCalledWith(expect.objectContaining({ modelId: 'backup' }))+ expect(textOf(turn)).toBe('from backup')+ })++ it('continues on the next model after the fast step already answered and ran a tool, without running it again', async () => {+ const search = vi.fn(async () => ({ content: [{ type: 'text', text: 'found' }] }))+ const backup = answeringModel('done')++ const turn = await runTurn({+ models: [tierModel({ modelId: 'main', model: failingModel({ statusCode: 503 }) }), tierModel({ modelId: 'backup', model: backup })],+ fastModel: tierModel({ modelId: 'fast', model: searchingModel() }),+ search,+ })++ expect(turn.streamError).toBeNull()+ expect(turn.answeredBy.modelId).toBe('backup')+ expect(search).toHaveBeenCalledTimes(1)+ expect(textOf(turn)).toBe('Looking.done')+ expect(JSON.stringify(backup.doStreamCalls[0].prompt)).toContain('found')+ })++ it('does not fall back once the failing step already sent text to the user', async () => {+ const backup = answeringModel('from backup')++ const turn = await runTurn({ models: [tierModel({ modelId: 'main', model: textThenFailingModel() }), tierModel({ modelId: 'backup', model: backup })] })++ expect(turn.streamError).not.toBeNull()+ expect(backup.doStreamCalls).toHaveLength(0)+ })++ it('tells tools which model runs each step, so a fast step\'s tool calls use the fast key', async () => {+ const onStepModel = vi.fn()++ await runTurn({+ models: [tierModel({ modelId: 'main', model: answeringModel('done') })],+ fastModel: tierModel({ modelId: 'fast', model: searchingModel() }),+ onStepModel,+ })++ expect(onStepModel.mock.calls.map(([turnModel]) => turnModel.modelId)).toEqual(['fast', 'main'])+ })++ it('does not retry a step that already sent text, even on the last model', async () => {+ const main = textThenFailingModel()++ const turn = await runTurn({ models: [tierModel({ modelId: 'main', model: main })] })++ expect(turn.streamError).not.toBeNull()+ expect(main.doStreamCalls).toHaveLength(1)+ })++ it('drops a failing fast model and answers on the same main model', async () => {+ const turn = await runTurn({+ models: [tierModel({ modelId: 'main', model: answeringModel('from main') }), tierModel({ modelId: 'backup', model: answeringModel('from backup') })],+ fastModel: tierModel({ modelId: 'fast', model: failingModel({ statusCode: 429 }) }),+ })++ expect(turn.streamError).toBeNull()+ expect(turn.answeredBy.modelId).toBe('main')+ expect(textOf(turn)).toBe('from main')+ })++ it('drops a fast model that rejects the request outright, since the main model may still take it', async () => {+ const turn = await runTurn({+ models: [tierModel({ modelId: 'main', model: answeringModel('from main') })],+ fastModel: tierModel({ modelId: 'fast', model: failingModel({ statusCode: 400 }) }),+ })++ expect(turn.streamError).toBeNull()+ expect(textOf(turn)).toBe('from main')+ })++ it('fails on a bad request instead of trying the next model', async () => {+ const backup = answeringModel('from backup')++ const turn = await runTurn({ models: [tierModel({ modelId: 'main', model: failingModel({ statusCode: 400 }) }), tierModel({ modelId: 'backup', model: backup })] })++ expect(turn.streamError).not.toBeNull()+ expect(backup.doStreamCalls).toHaveLength(0)+ expect(turn.allModelsFailed).toBe(false)+ })++ it('says every model failed when the whole tier is down', async () => {+ const turn = await runTurn({ models: [tierModel({ modelId: 'main', model: failingModel({ statusCode: 503 }) }), tierModel({ modelId: 'backup', model: failingModel({ statusCode: 503 }) })] })++ expect(turn.allModelsFailed).toBe(true)+ })++ it('builds the provider options for the model that runs, not the one that failed', async () => {+ const sent: SharedV3ProviderOptions[] = []+ const backup = answeringModel('from backup', sent)++ await runTurn({+ models: [+ tierModel({ modelId: 'claude', model: failingModel({ statusCode: 503 }), provider: AIProviderName.ANTHROPIC }),+ tierModel({ modelId: 'gpt', model: backup, provider: AIProviderName.OPENAI }),+ ],+ })++ expect(sent[0]).not.toHaveProperty('anthropic')+ })++ it('reports a provider failure for the key that failed', async () => {+ const onModelOutcome = vi.fn()++ await runTurn({ models: [tierModel({ modelId: 'main', model: failingModel({ statusCode: 503 }) }), tierModel({ modelId: 'backup', model: answeringModel('ok') })], onModelOutcome })++ expect(onModelOutcome).toHaveBeenCalledWith(expect.objectContaining({ turnModel: expect.objectContaining({ modelId: 'main' }), signal: expect.objectContaining({ statusCode: 503 }) }))+ })++ it('leaves a single specific model on today\'s path, with no fallback', async () => {+ const turn = await runTurn({ models: [{ model: failingModel({ statusCode: 503 }), provider: AIProviderName.OPENAI, modelId: 'only', thinkingBudget: 0 }] })++ expect(turn.streamError).not.toBeNull()+ expect(turn.allModelsFailed).toBe(false)+ })+})++describe('what counts as content sent in the step that failed', () => {+ it('ignores the start markers the stream sends before any content', () => {+ expect(sentContentAfter(['start', 'error'])).toBe(false)+ })++ it('counts a step that failed after a step that finished as clean', () => {+ expect(sentContentAfter(['start', 'start-step', 'text-delta', 'tool-output-available', 'finish-step', 'error'])).toBe(false)+ })++ it('counts text sent before the error, even though the step is closed after it', () => {+ expect(sentContentAfter(['start', 'start-step', 'text-start', 'text-delta', 'error', 'finish-step', 'finish'])).toBe(true)+ })+})++async function runTurn({ models, fastModel, search, onStepModel, onModelOutcome }: {+ models: [TurnModel, ...TurnModel[]]+ fastModel?: TurnModel+ search?: () => Promise<unknown>+ onStepModel?: (turnModel: TurnModel) => void+ onModelOutcome?: (outcome: unknown) => void+}): ReturnType<typeof runAgentTurn> {+ return runAgentTurn({+ models,+ ...(fastModel ? { fastModel } : {}),+ systemPrompt: 'You are a test agent.',+ messages: [{ role: 'user', content: 'hello' }],+ tools: {+ ap_web_search: tool({ description: 'search the web', inputSchema: z.object({ query: z.string() }), execute: search ?? (async () => ({ content: [] })) }),+ },+ allToolNames: ['ap_web_search'],+ tier: TIER,+ phaseState: { phase: 'discovery' },+ abortSignal: new AbortController().signal,+ log: SILENT_LOG,+ sinks: { drainStream: drainLikeTheWorker },+ ...(onStepModel ? { onStepModel } : {}),+ ...(onModelOutcome ? { onModelOutcome } : {}),+ })+}++async function drainLikeTheWorker(result: ReturnType<typeof streamText>): Promise<StreamDrain> {+ let state: StepContentState = NO_STEP_CONTENT+ const reader = toUIMessageStream({ stream: result.stream }).getReader()+ for (let next = await reader.read(); !next.done; next = await reader.read()) {+ const chunkType = isObject(next.value) && typeof next.value['type'] === 'string' ? next.value['type'] : undefined+ state = trackStepContent({ state, chunkType })+ }+ return drainOf({ state })+}++function sentContentAfter(chunkTypes: string[]): boolean {+ return drainOf({ state: chunkTypes.reduce((state, chunkType) => trackStepContent({ state, chunkType }), NO_STEP_CONTENT) }).lastStepSentContent+}++function tierModel({ modelId, model, provider = AIProviderName.OPENAI }: { modelId: string, model: MockLanguageModelV3, provider?: AIProviderName }): TurnModel {+ return { model, provider, modelId, thinkingBudget: 0, key: { providerConfigId: `key-${modelId}`, status: 'active' } }+}++function failingModel({ statusCode }: { statusCode: number }): MockLanguageModelV3 {+ return new MockLanguageModelV3({+ doStream: async () => {+ throw new APICallError({ message: `HTTP ${statusCode}`, url: 'https://llm.example/v1', requestBodyValues: {}, statusCode, isRetryable: false })+ },+ })+}++function answeringModel(text: string, sentOptions?: SharedV3ProviderOptions[]): MockLanguageModelV3 {+ return new MockLanguageModelV3({+ doStream: async ({ providerOptions }) => {+ sentOptions?.push(providerOptions ?? {})+ return {+ stream: convertArrayToReadableStream([+ { type: 'stream-start' as const, warnings: [] },+ { type: 'text-start' as const, id: 't' },+ { type: 'text-delta' as const, id: 't', delta: text },+ { type: 'text-end' as const, id: 't' },+ { type: 'finish' as const, finishReason: 'stop' as const, usage: USAGE },+ ]),+ }+ },+ })+}++function searchingModel(): MockLanguageModelV3 {+ return new MockLanguageModelV3({+ doStream: async () => ({+ stream: convertArrayToReadableStream([+ { type: 'stream-start' as const, warnings: [] },+ { type: 'text-start' as const, id: 't' },+ { type: 'text-delta' as const, id: 't', delta: 'Looking.' },+ { type: 'text-end' as const, id: 't' },+ { type: 'tool-call' as const, toolCallId: 'call-1', toolName: 'ap_web_search', input: '{"query":"x"}' },+ { type: 'finish' as const, finishReason: 'tool-calls' as const, usage: USAGE },+ ]),+ }),+ })+}++function textThenFailingModel(): MockLanguageModelV3 {+ return new MockLanguageModelV3({+ doStream: async () => ({+ stream: convertArrayToReadableStream([+ { type: 'stream-start' as const, warnings: [] },+ { type: 'text-start' as const, id: 't' },+ { type: 'text-delta' as const, id: 't', delta: 'Half an ans' },+ { type: 'error' as const, error: new APICallError({ message: 'HTTP 503', url: 'https://llm.example/v1', requestBodyValues: {}, statusCode: 503, isRetryable: false }) },+ ]),+ }),+ })+}++function textOf(turn: Awaited<ReturnType<typeof runAgentTurn>>): string {+ return turn.uiParts.map((part) => isObject(part) && typeof part['text'] === 'string' ? part['text'] : '').join('')+}++const TIER = { id: 'fast', thinkingBudget: 0, modelId: 'main' }+const USAGE = { inputTokens: 1, outputTokens: 1, totalTokens: 2 }+const SILENT_LOG = { debug: () => undefined, info: () => undefined, warn: () => undefined, error: () => undefined }diff --git a/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-history.test.ts b/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-history.test.tsindex 3e7d1c8eede..3f898525f53 100644--- a/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-history.test.ts+++ b/packages/server/worker/test/lib/execute/jobs/ee/agent/run-agent-turn-history.test.ts@@ -10,8 +10,7 @@ describe('the history a turn leaves behind', () => {const progress: ModelMessage[][] = []const turn = await runAgentTurn({- model: updateThenConfirm(),- provider: AIProviderName.ANTHROPIC,+ models: [{ model: updateThenConfirm(), provider: AIProviderName.ANTHROPIC, modelId: TIER.modelId, thinkingBudget: TIER.thinkingBudget }],systemPrompt: 'You are a test agent.',messages: [{ role: 'user', content: 'Change the instructions of New agent to: Reply in Arabic.' }],tools: {@@ -19,13 +18,12 @@ describe('the history a turn leaves behind', () => {},allToolNames: ['ap_update_agent'],tier: TIER,- modelId: TIER.modelId,phaseState: { phase: 'build' },abortSignal: new AbortController().signal,log: SILENT_LOG,creditsLeft: async () => 100,sinks: {- drainStream: (result) => result.consumeStream(),+ drainStream: async (result) => { await result.consumeStream(); return undefined },onProgress: ({ responseMessages }) => progress.push(responseMessages),},})@@ -39,8 +37,7 @@ describe('the history a turn leaves behind', () => {let saves = 0const turn = await runAgentTurn({- model: keepsUpdating(),- provider: AIProviderName.ANTHROPIC,+ models: [{ model: keepsUpdating(), provider: AIProviderName.ANTHROPIC, modelId: TIER.modelId, thinkingBudget: TIER.thinkingBudget }],systemPrompt: 'You are a test agent.',messages: [{ role: 'user', content: 'Update New agent three times.' }],tools: {@@ -58,12 +55,11 @@ describe('the history a turn leaves behind', () => {},allToolNames: ['ap_update_agent'],tier: TIER,- modelId: TIER.modelId,phaseState: { phase: 'build' },abortSignal: controller.signal,log: SILENT_LOG,creditsLeft: async () => 100,- sinks: { drainStream: (result) => result.consumeStream() },+ sinks: { drainStream: async (result) => { await result.consumeStream(); return undefined } },})const toolCalls = turn.accumulatedResponseMessages.flatMap((message) => typeof message.content === 'string'