AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
icon: 🤖
AI Agents
A flow step type (the run_agent action of @activepieces/piece-ai) that runs an LLM-driven autonomous loop. Given a prompt, tools, an AI provider/model, and optional structured-output fields, it runs a ReAct-style loop (up to maxSteps) where the model can call any configured tool before producing a final answer.
How it works
- A step either carries its own configuration in the flow version's step settings —
settings.inputholdsagentTools,structuredOutput,prompt,maxStepsandaiProviderModel({ provider, model, configId }) — or stores theexternalIdof a saved Agent (agenttable,ee/agent/agent-entity.ts) underagentIdand carries nothing else. A linked step is resolved at run start, so improving the agent improves the next run of every flow using it. The two are exclusive: a step that sends both is refused. - Configured entirely in the Flow Builder (
web/src/app/builder/step-settings/agent-settings/); a test panel runs a single agent step.AgentTimelinerendersAgentStepBlock[]from the output as markdown blocks + expandable tool-call cards. - One engine, four doors. Chat, agent chat, the agent builder and
run_agentare not four systems. Each creates anagent_conversationrow and runs the sameEXECUTE_AGENT_RUNworker job, and the worker calls back overgetAgentConfig(ee/agent/rpc/agent-config-rpc.ts) to ask which model, tools and instructions to use. The assistant chat is an agent turn with no agent row. Which tier list a door resolves against followsnamesItsOwnModelin that RPC: FLOW_STEP and AGENT name their own model and read the flow tiers; CHAT and AGENT_BUILDER read the chat tiers. An agent therefore resolves the same model everywhere it is used: configured, chatted with, or dropped in a flow.
Door (AgentRunSource) |
Model comes from | Stored as |
|---|---|---|
CHAT |
conversation.modelName, picked in the UI |
tier id |
AGENT |
agent.draft.modelName; no picker is shown while chatting (ai-chat-box.tsx hides it when agentId is set) |
tier id on the managed provider; concrete id on an own key or a row saved before ENG-754 |
AGENT_BUILDER |
conversation.modelName |
tier id |
FLOW_STEP |
agent.published.modelName, or the step's own input.aiProviderModel.model |
same as AGENT |
Three things follow. run_agent has two modes: pointed at a saved agent (agentId set) the step sends no model and the agent row decides; inline, it sends input.aiProviderModel.model. An agent row carries two configs: agent-page chat reads agent.draft, a run_agent step reads agent.published (via resolvePublishedAgent), so the same agent can already run different models through different doors. The fast model crosses every door: firstStepUsesFastModel (run-agent-turn.ts) runs the first step of almost every turn on the fast tier, flow steps included, whatever the user picked. execute-agent-run.ts persists config.tier.id onto conversation.modelName when the job carried no modelName, so conversations already hold tier ids today (inline run_agent, eval runs, every AGENT_BUILDER), read back by chat-tool-billing.ts and chat-analytics-sync.ts. A persisted tier id is not a pin. getAgentConfig picks modelName ?? conversation.modelName, so a job that carries no model reads the persisted value back (pinnedModelOf no longer exists). A saved agent's run model is the agent's own half (draft for AGENT, published for FLOW_STEP), copied onto the job by jobFieldsFromConfig.
Tool types (AgentTool discriminated union)
- PIECE — a specific piece action (
pieceName/pieceVersion/actionName); can carrypredefinedInputlocking certain fields. - FLOW — calls another flow by
externalFlowId, executed as a child run. - MCP — connects to an external MCP server (SSE / StreamableHTTP / SimpleHTTP; None/Bearer/ApiKey/Headers auth).
- KNOWLEDGE_BASE — semantic search over a KB file/table (cosine similarity, 768-dim embeddings).
- PredefinedInputsStructure — per-field
AGENT_DECIDE/CHOOSE_YOURSELF/LEAVE_EMPTYbaked into the tool so the agent knows which inputs it controls.
Gotchas
PostHog drops any AI span without
$lib. Our PostHog project has a "Filter Out Plugin" transformation that keeps an event only if$libis set and isn't Segment, and it drops events where the property is undefined. OTel spans don't carry$lib, soaiObservability'senrichSpansets it on every span. Without it, the export still returns 200 and the event only shows up insystem.ingestion_warningsasevent_dropped_by_transformation.PostHog AI tracing is per call, never global.
agentAiUtils.buildTelemetryadds the PostHog integration (aiObservability, Cloud only) to chat and agent calls, and that's why flow AI steps aren't traced. CallingregisterTelemetryglobally would trace everygenerateTextin the process, including flow steps. A call that skipsbuildTelemetry, or doesn't passtrace, shows up in PostHog with no user or conversation.A chat's project is chosen inside
getAgentConfig, not on the job. A new chat's first turn is enqueued with no project; the API picks one, saves it, and resolves the key for it. Anything the worker records per project (usagebilling.projectId) must useconfig.projectIdfrom the response, or the first turn lands under no project.Chat image attachments must be gated on the model's
supportsVision, not just the provider. The managed key serves the whole OpenRouter catalogue, including text-only models likez-ai/glm-5.3. Sending them an image part gets a404 No endpoints found that support image input, which the classifier counts asinternalon the managed key, so it pages. The image is saved inconversation.messages, so every later turn in that conversation fails too.dropImagesForTextOnlyModels(ee/agent/agent-file-utils.ts) replaces image parts in the payload sent to the LLM with a note telling the model it cannot see the image, ingetAgentConfig, when any model that reads the conversation (the run candidates, plus the fast model outside chat) is cataloguedsupportsVision: false. Saved history keeps the image.A chat stores its own model pick, and a chat saved before that pins its key on its next turn.
agent_conversationholds a tier (modelTierId) or a key + model (providerConfigId+modelName); a Credits pick is the managed row id plus a credits tier id. A row with onlymodelName(old web client, or any chat from before 0.93.0) runs on today's Default-AI-key path once, thenagentChatModel.legacyPinstores the key and the model it resolved to. A published tier slug (fast/smart/premium) stays a slug on every key, or an own-keypremiumchat would think at the default budget from turn 2; an unknown slash-free name on an Activepieces/OpenRouter key stores the tier that actually ran, because the pinned path has no fallback. The pin is saved under aFOR SHARElock on the key row and only if the key still serves the chat, and key delete removes the row before it unpins, so a key deleted mid-turn is never pinned. The write is conditional on the row still being unpinned, so a model change made during the turn wins. A chat with nothing stored gets the Default tier, else the Default AI key path, else the picker's first option, and stores it.Only a pinned chat fails on an unknown model. A pinned pick whose model or key went away fails with "The model this chat uses isn't available anymore" (S11). A legacy row keeps the old silent fallback (
curated[0]/ default tier), so an upgrade never breaks a chat. Deleting a key, or narrowing its projects or models, unpins the chats it no longer serves (aiUsageHooks.unpinConversationsFromKey), and they re-pin on the next turn. The worker'smodelNamewrite-back skips pinned rows: it sends the tier slug back, which would replace a real model id. A chat run reads its model and key from the freshly loaded conversation, never from the job'smodelName: the job carries the model from enqueue time, and a pick changed while the message waited would pair an old model with a new key. Billing labels a turn by the worker'sansweredBy(sent on every turn), not by the conversation's current pick, which can change mid-turn. A pinned Activepieces/OpenRouter tier slug never falls back: an unpublished slug, or one whose model the key'smodelIdsexcludes, fails with S11. A key pick needs a chat with a project: the offer check (aiModelOptionsService.list) is per project, and the run path passes own-key names straight through.Deleting an agent locks its row before it checks for published flows.
agentService.deletetakesFOR UPDATEon the row (lockedAgentInProjectOrThrow) and only then runs theNOT EXISTSdelete as a fresh statement. Publishing takesFOR SHAREinassertAgentsResolveInProject. A bareDELETE ... WHERE NOT EXISTSis not enough: if it blocks on a publisher'sFOR SHARE, it wakes with the snapshot it started with and deletes an agent the publish just linked. The integration tests run on PGlite, one connection, so the race cannot be reproduced there; the tests assert the lock modes are requested instead.A managed agent row or Run Agent step stores a tier id (
smart) from ENG-754 on; nothing converts the rows saved before.AIModelSelector(web/src/features/agents/ai-model/) lists the published flow tiers by id for the managed provider and never calls/modelsfor it; a blank picker pre-fills the publishedflow.defaultTierIdonce the tiers query settles (it used to take the first tier, Fast), andwithDefaultModel(agent-service.ts) seeds the same default tier id on create. Own-key providers still store concrete ids:resolveNamedModelIdpasses their name through verbatim, so a tier id there would reach the provider as a model name. An older row keeps its concrete id: it runs that model (slash rule), shows the raw id in the picker, and does not follow a re-point until the ENG-753 migration.A flow-surface run takes its thinking budget from the tier it names, not the default tier.
resolveRunTier(agent-model-resolution.ts) buildsconfig.tierfor FLOW_STEP and AGENT runs: a managed tier id gives that tier'sthinkingBudget,idandmodelId; a concrete id, an own-key model, a tier the file dropped, or no model at all gives the flow default, as every flow-surface run did before ENG-754. So a Heavy agent thinks at Heavy's budget and reportspremiuminchat.tier, while a Heavy row saved asanthropic/claude-opus-4.8still thinks at Expert's until it is migrated.A chat turn's data lives in jsonb, so recording something per turn needs no migration.
agent_conversation.messagesis ajsonbcolumn and each entry is aPersistedAgentMessageSchema, which already carries optional per-turn fields (thinkingDurationMs,feedback). Adding another optional one is a zod field plus a@activepieces/sharedpatch bump — no column, no migration, no rollback flag. The conversation row itself only hasmodelName, which is why "record it on the conversation" is usually the wrong instinct: a user can switch tier mid-conversation, so anything that describes one turn belongs on the message, not the row.But the server does not build those messages, so "no migration" does not mean "cheap".
messagesarrives already assembled from the worker over RPC (saveAgentMessagesinrpc/conversation-rpc.ts), and the server only persists it and bills off it. Putting a new per-turn field on a message therefore means changing the RPC request schema and the worker that fills it, not editing a server file. Costing this off the jsonb shape alone gave an estimate of about an hour for work that is really a change to the engine contract. Check who authors the object before you price a change to it.A turn's
inputTokenscounts only the main chat. Task and researcher tokens go intaskInputTokensandtaskOutputTokenson the sameChat message completedlog. They're summed bytaskTokensOffrom thetokenseachap_run_taskorap_deep_researchresult carries, which is kept out of what the model reads. On a managed key, task tokens bill at cost because tasks reuse the turn's billing-wrapped models. On an own key, the turn pays one message credit plus the paid calls the tasks report inbilledToolCalls.CHAT_HIDDEN_TOOL_NAMESis load-bearing for the MCP Activity feed, not just for chat UX. The chat reaches the MCP server as an ordinary HTTP client, andmcp_activityrecordsap_run_action. The only thing keeping chat activity out of that feed isap_run_actionsitting inCHAT_HIDDEN_TOOL_NAMES(core/shared/src/lib/ee/agent/tool-phases.ts), filtered worker-side inagent-mcp-client.ts. Remove it and the chat starts writing rows into a tab meant for external MCP clients, with no test failing. See the MCP Server page for the enforcement that was designed and deliberately not built.Gated by
platform.plan.agentsEnabled; when off, the step type is hidden from the piece selector. Off by default on Community, on for Cloud plans that include it.External MCP tools are validated server-side via
POST /v1/projects/:projectId/agent-tools/mcp/validate— a JSON-RPCinitialize→notifications/initialized→tools/listhandshake returning tool names. Outbound call routes throughapAxioswithssrf-agents.tsrejecting private/loopback/link-local/meta IPs (allow ranges viaAP_SSRF_ALLOW_LIST, CIDR). All error paths collapse to one generic message to avoid leaking reachability.That validator lives under
agents/(validating a server the agent connects to), deliberately separate from themcp/module which exposes Activepieces itself as an MCP server (opposite direction).Shared types live in two packages on purpose:
core/piece-types/src/lib/agents.ts(zod/mini, for pieces) andcore/execution/src/lib/agents/(plainzod, for server/web).AgentResultisprompt,steps[],status, optionalstructuredOutput.The enums and pure functions have exactly one home:
core/piece-types/src/lib/agents.ts. Do not re-declareAgentToolType,McpAuthType,buildAuthHeaders,TASK_COMPLETION_TOOL_NAME, ormcpToolNameUtilsincore-execution— re-export them. They used to be duplicated byte-for-byte across both packages, which was silently load-bearing: ifcreateToolNamedrifted, the tool namesmigrate-v16persisted would stop matching runtime names and every piece/flow/MCP call on a migrated flow would degrade toToolCallType.UNKNOWN.mcp-tool-name-util.test.tsasserts both entry points resolve to the same object, so a re-fork fails the test rather than shipping.The four
core/execution/src/lib/agents/files are not uniform.mcp-tool-name-util.tsandmcp.tsare pure re-export shims (1 and 6 lines).index.tsandtools.tsre-export the canonical enums and functions but still own the execution-side plain-zodschema definitions —tools.tsdeclares theAgentToolunion and theMcpAuth*schemas,index.tsdeclaresAgentOutputField,MarkdownContentBlock,ToolCallContentBlockandAgentStepBlock. Adding a field to one of those schemas means editing it there and in thezod/minitwin inagents.ts.A flow-step run must not reuse chat's resolution logic. Four separate production failures came from this one assumption while moving the step server-side, each looking like its own bug.
resolveChatProvidermade a step need Chat's provider configured before it would run at all, so an instance that never uses Chat could not run an agent step — and it bit twice, becauseresolveFastModelreached the same helper underneath, so every configured piece tool failed with a bareENTITY_NOT_FOUNDlong after the main model had been fixed. Grep for the transitive callers, not just the direct ones.resolveModelIdForProvidertreats its argument as a tier id and falls back to the tier default when it is not in the curated chat list — a step configured forclaude-sonnet-4.5silently ran4.6, because a step names a concrete model while chat names a tier. And the chat tool set reaches an unattended run, where a tool that asks the user a question is worse than useless: the agent opened a connection picker, read the empty answer as a refusal, and stopped. When a value crosses between the two surfaces, check what it means on each side, not just that the types line up.A worker RPC failure carries
{ code, entityType }now, but still no stack. The envelope incore/execution/src/lib/engine/rpc.tsused to serializeerror.messagealone, so three unrelated causes (conversation gone, no chat-enabled provider, pinned provider has no row) all arrived as the same bareENTITY_NOT_FOUND.apErrorOfnow also ships anActivepiecesError's code and entity type, which the client re-attaches to the thrown error — read it withapErrorOf(error), never by parsing the message. Deliberately not the wholeparams: it is typedunknown, and socket.io JSON-encodes this ack from inside acatchwhere nothing handles a throw, so one cyclic or BigInt-bearing params object would send no ack at all and stall the caller for the full 60s RPC timeout (the engine side wouldprocess.exit(4)on the unhandled rejection).rpc.test.tspins this with a cyclic params case and a JSON-round-tripping fake socket — keep the projection narrow.A failed agent run is a user's misconfiguration far more often than our bug, and only our bugs belong in the failed set.
EXECUTE_AGENT_RUNre-threw on everything except credit exhaustion, so ~5,900 unrecoverable user-config failures accumulated in the BullMQ failed set over one 30-day retention window (REDIS_FAILED_JOB_RETENTION_DAYS) and buried the real bugs.classifyAgentRunError(run-agent-turn.ts) splits them, and a user-class failure returnsEngineResponseStatus.USER_FAILURE, whichjob-broker.completeJobcompletes exactly likeOKwhile naming the outcome. Four things it gets deliberately right, each of which is a way to get it wrong:- The user-fault statuses are an allow-list (401/403/404), not
!APICallError.isRetryable. The SDK calls every 4xx non-retryable, so the tempting one-liner blames the user for a 400 from an illegal generated tool name or a 413 from a prompt still over the window after compaction — requests we built, and exactly the laundering the split exists to prevent. - The managed
activepiecesprovider is never user-fault on auth. It runs on our own OpenRouter key, so a 401 there fails every platform at once and must page. - Credit is read from a status or the specific
insufficient_quotamarker, never loose patterns over a response body. OpenAI signals billing exhaustion as a retryable 429 with the marker in the body, so credit is checked before the retryable verdict — but scanning a body forcredits/402made a provider 500 whose HTML error page said "credits" complete as a billing failure and hide a real outage. ENTITY_NOT_FOUNDcounts only for an AI-providerentityType, andVALIDATIONcounts for nothing. A bare not-found is our bug; theVALIDATIONthat reaches this surface is the conversation concurrency lock, and a conversation stuckSTREAMINGis a state worth keeping visible. So a user-config refusal thrown fromgetAgentConfigmust beENTITY_NOT_FOUND/AIProviderwith its text asActivepiecesError's second argument: text left inparamsnever crosses the RPC, which is how a step naming a model the Activepieces credits do not serve failed as a bare internalVALIDATIONon every scheduled run. The billing text heuristic reads provider text only and skips any error carrying our own code, becauseisProviderBillingErrormatched our own "Activepieces AI credits" wording; our credit exhaustion is alwaysQUOTA_EXCEEDED. Those bad models came fromap_list_ai_models, which offered the AI builder the whole OpenRouter catalog for Run Agent steps; it now lists only what the resolver accepts, whilelistModelskeeps the full catalog because Ask AI and image lookups need it.- A 400 from the
customprovider is user-fault. That endpoint is a gateway the customer chose, and it has its own request rules. For example, OpenCode Go rejects any client but its own withmissing x-opencode-session. Built-in providers keep 400 internal, so a tool-name bug still pages there (#16821). A completed job stores noerrorMessage, so thewarnlog carryingagentRun.errorClassis the only remaining record.
- The user-fault statuses are an allow-list (401/403/404), not
A tool name is user text on four paths, and only the worker sees all of them.
createToolNameis applied by the flow-tool dialog and the piece-tool stores, but a knowledge-base name was stored as typed and the AI piece'stoolNameis freeShortText. That string becomes the AI-SDKToolSetkey verbatim, which is how a name earned a 400 from Anthropic — our request, never the user's fault, which is why widening the status allow-list to 400 would have been the wrong fix.mcpToolNameUtils.toValidToolNameis the guard, applied byagentToolPolicy.withValidNamesinexecute-agent-run— the one place the flow-step, chat and eval enqueue paths converge (agent-conversation-controllervalidates tool names not at all;agent-run-controllerchecks only the reserved prefix and duplicates, and does it on the raw names, so it cannot see a collision the rewrite creates). Four things it has to get right:- The pattern is the intersection of every provider we ship, not the one from the error we happened to see. Anthropic's
^[a-zA-Z0-9_.-]{1,64}$is the loosest: OpenAI and Bedrock reject., and Gemini requires a leading letter or underscore. Guarding with Anthropic's rule leaveshandbook.pdf— the obvious name for a knowledge base file — still failing everywhere else. The guard is^[a-zA-Z_][a-zA-Z0-9_-]{0,63}$. - It rewrites only a name that already fails, because
createToolNameis not idempotent and re-running it would break the namesmigrate-v16persisted.toValidToolNamere-checks its own output and re-derives from a prefixed source whencreateToolNamereturns a leading digit. - It dedupes, because sanitising converges.
Company DocsandCompany docsmap to one key, and every name with no[a-z0-9_-]at all used to hash identically — two CJK-named tools became the same key.Object.fromEntriesis last-wins, so one tool vanished from the toolset with no error and answered from the wrong source.createToolNamenow hashes the original when the sanitised form is empty, andwithValidNamesis a list→list function holding atakenset. - MCP tools are left alone, because
agent-mcp-clientalready derives a sanitised key from${toolName}_${name}. Server-sidetoolNameis log-only —executePieceTool/executeFlowTool/executeKnowledgeBaseToolroute onpiece,flowIdandknowledgeBaseFileId— so rewriting it breaks no lookup.stepResultFrommust be passed the knowledge-base tools too, or itsToolCallType.KNOWLEDGE_BASEbranch is unreachable and the card shows the rewritten key instead of the file name.
- The pattern is the intersection of every provider we ship, not the one from the error we happened to see. Anthropic's
A retired model is user config, and only a marker in the message says so. A provider 400 stays
internalby default; one whose message matchesMODEL_UNAVAILABLE_PATTERNSisuser. Two deliberate narrowings, both learned the hard way: the body is never scanned, because any 400 carrying an HTML error page that says "deprecated" in its footer would launder our own outage; and a retired model on the managedactivepieceskey is ours, sinceresolveModelIdForProvidersubstitutescuratedModels[0]for anything uncurated — a stale constant in our repo failing every platform at once must page, not read as "the customer picked a bad model". That substitution still classifies asuseron a BYO key, which is the residual gap.The curated chat lists rot silently and nothing checks them, but "the ticket said it's deprecated" is not evidence.
ALLOWED_CHAT_MODELS_BY_PROVIDER(core/piece-types/src/lib/ai-providers.ts) is the only thing deciding what the pickers offer; the models.dev catalog is metadata keyed by id and adds or removes nothing. Check a suspected-dead id against models.dev before deleting it — of the four ENG-466 named, onlygrok-4.1-fasthad actually gone; the Gemini 2.5 pair was live, current and the cheapest Google option. Removing a live model is not a cleanup:resolveModelIdForProviderfalls through tocuratedModels[0], so a BYO customer pinned to Flash would have silently moved to a Pro preview at roughly five times the token price, on their own key, with no notice and no migration. Three things move together when editing a list:CHAT_MODEL_LABELS(a curated id with no label failsai-providers.test.ts),MANAGED_MODEL_WEIGHTSinflow-run-ai-usage-tracker.ts(its?? 2default is below the table's floor of 6, so a forgotten managed model under-bills — everyx-ai/*id does today), and the order, sincecuratedModels[0]is both the picker's first row and the fallback for every unrecognised selection.An agent's tools reach the worker in two shapes, and only one of them can actually run a flow.
ExecuteAgentRunJobDatacarriestools(the rawAgentTool[]) andflowTools(resolved:flowId,flowVersionId, JSON input schema). The worker builds its flow tool set fromflowToolsalone —data.toolsis only filtered for PIECE, MCP and KNOWLEDGE_BASE, so theAgentToolType.FLOWentries in it are inert. An enqueue site that fills onlytoolstherefore drops every flow tool with no error and no log: the Configure panel still lists them, the chat footer still names them, and the model is simply told it has no such tool. This is how saved-agent chat shipped for three releases while the same agent worked inside aRun Agentstep, which resolved them. Both enqueue paths now go throughagentHelpers.resolveFlowTools, which takes the whole tool array and filters internally, so no caller can do half the job. Resolve against the project the run will execute in —executeFlowToollooks the flow up in the conversation's project and refuses anything else, so resolving elsewhere hands the model aflowIdthe RPC then rejects.Whatever enqueues an agent run must pre-check the same thing the worker resolves. The chat route asked "is any provider enabled for chat" while the worker looked up the run's pinned provider, and the flow-step route checked nothing at all — so a run enqueued fine and could only fail. Both now call
agentHelpers.assertRunProviderConfigured, which mirrors the worker's lookup. A pre-check that answers a different question than the worker is worse than none: it makes the failure look impossible.Everything the agent job does before its try/catch has no recovery.
getAgentConfigused to run outside it, so a config failure sent no error to the chat client and never calledreleaseFlowStep— the flow run sat PAUSED untilAP_PAUSED_FLOW_TIMEOUT_DAYS. Anything added above that block needs its own failure path, or a paused run leaks.Build the unattended tool set as an allow-list. Removing chat tools by name failed three times running — display tools, then build-plan and phase tools, then
ap_discover_action_authandap_load_guide, which live with the local tools and so survived a filter written by tool group. Grouping tracks where a tool was constructed, not whether it assumes someone is reading. A flow step gets exactly what it is listed: its configured piece actions, the public-web readers, and the structured-output tool. Anything added to chat later stays out by default.A separate zod-free
agent-primitives.tsholding those values was tried and folded back — don't re-create it. It bought no isolation:core-executionimports the@activepieces/core-piece-typesbarrel, which re-exportsagents.ts, sozod/minicomes along whatever the values live in.Only the zod schemas stay duplicated — the
zodvszod/minisplit is a real bundle-size decision, and a schema drift breaks loudly where a function drift did not.The system prompt has to be gated by the same source rule as the tool set, not just the tools. Listing tools per surface is only half of it: an agent run still got chat's connection-inventory, memory and email notes appended, so the model was told to use
ap_remember,ap_list_connectionsand connection cards it did not have — and acted on it, opening an account picker for a piece tool whose connection its owner had already pinned, where the pick is read and discarded. A note naming a tool the surface cannot reach is worse than a missing note.agentSurfaceNotes.buildRunNotesnow decides notes byAgentRunSourcein one place, and a unit test asserts no chat-only tool name appears in anAGENTorFLOW_STEPprompt.A piece action's output is stringified into the tool result at the producer, so that is the only place it can be shortened with its shape intact.
formatPieceActionRunResultJSON-stringifies the whole output into one text blob; past that point a consumer trying to fit it in context can only cut a prefix, and the prefix of a Gmail search is DKIM headers. A five-email search reached the model as 2KB ofReceived:lines with every subject gone, while two rounds of improving the worker's array-finding heuristic changed nothing, because there was no array left to find. Shrink where the data is still structured, and measure the wrapped form so JSON escaping is counted rather than guessed.A test run of an agent step waits three hours before it admits nothing answered.
AGENT_STEP_TIMEOUT_MSis 3h and the same value is used whether or notstepNameToTestis set, so if the job never reaches a worker — a version-skewed worker idling by design is the usual cause — testing the step looks like a hang and eventually fails withThe agent did not report a result before this step timed out. Check for a running, version-matched worker before debugging the step's config.In
agents.tsthe enums must stay above the schemas that use them. A TS enum compiles to a hoistedvarplus a deferred IIFE, so a schema evaluatingz.literal(AgentToolType.PIECE)at module load before the enum block has run readsundefined.tsccatches it (TS2450: Enum used before its declaration), but only if you build — it is easy to introduce while reordering the file to satisfy the "exported types and constants at the end" convention.Reasoning cannot be disabled on a reasoning-native endpoint, and the managed catalogue is full of models nobody chose.
buildProviderOptions(server/utils/src/agent-provider-options.ts) expressesdisableThinkingasreasoning: { enabled: false }on theACTIVEPIECES/OPENROUTERbranch, andprepareStepsetsdisableThinkingfor the first step and the whole discovery phase. OpenRouter forwards the flag, and Gemini 3.x, GPT-6 Astra and Claude Fable 5.1 reject it outright:Reasoning is mandatory for this endpoint and cannot be disabled.The step dies before its first token. Four things this taught, in order of how easy each is to get wrong:- Swapping in
{ effort: 'minimal' }unconditionally is a regression, not a fix. OpenRouter mapsminimaltolowfor Anthropic, which is 20% ofmax_tokens, so the thinking-disabled first step on the Fast tier would get a bigger reasoning allowance (7,400) than its thinking-enabled step asks for (5,000). Every default tier is Anthropic, so that degrades every working customer to fix the broken ones. The suppression is gated onaiProviderUtils.isCuratedChatModelIdinstead: a model we picked keeps{ enabled: false }, anything else gets{ effort: 'minimal' }. getCuratedChatModelsreturnsundefinedforACTIVEPIECESon purpose, so the managed provider serves the entire live OpenRouter catalogue. Every failing model in the incident (google/gemini-3.8-flash,openai/gpt-6-astra,openai/gpt-6-astra-pro,anthropic/claude-fable-5.1) is absent fromALLOWED_CHAT_MODELS_BY_PROVIDER. Only a React.filter()narrows the picker to the three tiers;modelNameis a freez.string()server-side andagent-config-rpcuses it verbatim forFLOW_STEPandAGENT. Hiding is not denying.- The curated list is a fixed guess; nothing learns from a rejection any more.
aiProviderUtils.canDisableReasoning(core/piece-types/src/lib/ai-providers.ts) is a static check, andshouldRetryStreamre-runs the identical request once. A curated model that turns out to be reasoning-native fails until the list is fixed. satisfiesagainst the vendor's own reasoning type is near-worthless here.{ effort: 'none' },{ enabled: false, max_tokens: n }and{ enabled: false, effort: 'minimal' }all satisfy it, because the vendor supports disabling and it is the endpoints that do not. The guard has to be a closed union of the three directives we actually send.
- Swapping in
A tool's property keys are user text too, so they go through
mcp-tool-input.ts. Field labels a customer typed into the MCP trigger (Email Sender) are not valid schema keys on Anthropic.modelKeyByPropertyNamesanitizes them withmcpToolNameUtils.toProviderSafeIdentifier, keeping them unique, with a positional fallback and a length cap.toFlowPayloadmaps the model's args back to the original labels, so{{trigger['Email Sender']}}keeps working. The MCP server and agent flow tools both use it; build any new flow-as-tool schema through it.Do not raise
aipast 7.0.69 — from[email protected]the SDK stops auto-executing tools, and it fails silently. Bisected on 2026-09-21 while trying to reachexperimental_evaluate. On 7.0.44 (what the repo pins) and every version through 7.0.69, a tool call executes normally. From 7.0.70 onward, including 7.0.103–7.0.107,generateText/streamTextstill recognise the call —staticToolCalls=1, atool-callpart is emitted with correctly parsed input — butexecuteis never invoked andtoolResultsstays0. No error, no warning. The changelog entry for 7.0.70 isa828527: Prevent automatic tool execution when a model call ends with an unsafe finish reason, but the gate fires for everyfinishReasontried,tool-callsandstopincluded. Reproduced in a clean project outside the monorepo, on bothMockLanguageModelV3andMockLanguageModelV4, on zod 3 and zod 4;needsApproval: falseandtoolApproval: () => falsedo not restore it, and neither does emitting the fulltool-input-start/delta/endlifecycle before thetool-call. The alarm ispackages/server/worker/test/lib/agent-eval/agent-tool-call-repair.test.ts— 4 of its 5 tests go red, and they look like a repair regression, which is the wrong trail: a plain valid tool call does not execute either. Consequence:experimental_evaluate(first shipped in 7.0.103) cannot be reached without breaking the agent, so anything needing typed evaluation calls the gateway's/v4/ai/evaluation-modelendpoint oversafeHttpinstead of taking the bump. Re-test the table above before any futureaiupgrade; if a release fixes it, that is the one to jump to.A platform tier turn falls back per step, not per turn. When a step fails on a provider error (
isFallbackWorthy) before sending content, the turn continues on the next candidate from the steps already done, the waycontinue_truncationdoes. Tool results stay in the messages, so nothing runs twice. A fast step that sent nothing is dropped on any error, not only fallback-worthy ones: the Fast tier can sit on a smaller context window than the main tier (Haiku 200K under a Gemini 1M tier), and once the history outgrows it the fast model answers 400 "prompt is too long", which must not sink a turn the main model can still take. For the same reason compaction sizes its budget over the tier's candidates and the Fast tier, not the main tier alone. After any switch the turn stays on that candidate and has no fast model. A step that already sent text never falls back, and is not retried on the same model either (shouldRetryStreamalso reads the drain). Piece tools follow the model running the current step (onStepModelfromprepareStep), so a fast step's tool calls use the fast key. The worker sendsansweredByon save, and billing labels the turn with it.answeredByis the model of the turn's last step, so a turn that ends on the fast step is billed under the fast tier's key, not the main one. Search and image stay on the pair they were resolved with, and knowledge-base search stays on the chat key, because the chunks were embedded with it."Did the failing step send anything" is read off the UI stream, and
producedVisibleOutputcannot answer it.uiPartsonly grows inonStepEnd, after chunks have already reached the client. The UI stream sendsstart/start-stepbefore any content. When the model call itself throws, there is nostart-stepat all, onlyerror. A stream error arrives beforefinish-step(text-delta, error, finish-step).trackStepContenttherefore resets onstart-step/finish-stepand freezes its answer at the firsterrorchunk. Drive it through the realtoUIMessageStreamin tests, because a mocked drain hides the ordering.AgentConfigResponse.tieris always the console credits tier. A platform tier run fillsplatformTier,candidatesandfastCandidate. The singlecredentials/modelIdfields equalcandidates[0]. On a tier run the worker sends nomodelNametosaveAgentMessages, and the RPC ignores one on a tier row, because a model name clears the tier (the last write wins).A thinking-only final step triggers a reply nudge.
decideLoopAction(run-agent-turn.ts) decides thecontinue_emptynudge from the final step only (stepEndsWithReply), up toMAX_EMPTY_CONTINUATIONSnudges per turn. Counting every part the attempt produced, as it used to, let earlier tool chips and reasoning hide a last step that was only thinking, so the user saw "Thought for N seconds" and no reply.shouldRetryStreamstill uses the raw part count, since a stream retry is about the attempt, not its last step.A tool dialog that copies
toolsinto state must remount on every open. The dialogs stay mounted and only flip an open flag in their zustand store, souseState(tools...)keeps the first snapshot. The flow-tool dialog saved that snapshot over the agent's flow tools, which brought back flows removed since (AI-120). Key the dialog content on the open flag, as the flow-tool and knowledge-base dialogs do, or build the result from the livetoolsat save time, as the MCP dialog does.
Key files
Entry point: runAgent, the createAction in the ai piece registered in packages/pieces/community/ai/src/index.ts.
packages/pieces/community/ai/src/lib/actions/agents/— the agent loop itself:runAgent, tool construction, output builderpackages/core/piece-types/src/lib/agents.ts—AgentToolType,AgentPieceProps,AgentStepBlock, tool zod schemas; re-exported throughpieces-frameworkpackages/core/execution/src/lib/agents/— execution-side agent types, tool schemas, MCP tool-name helperspackages/web/src/features/agents/— all agent UI: tool dialogs and stores,AgentTimeline,AIModelSelector,SUPPORTED_AI_PROVIDERS, structured outputpackages/web/src/app/builder/step-settings/agent-settings/— builder panel for configuring an agent steppackages/web/src/app/builder/test-step/agent-test-step/— test panel for running one agent steppackages/server/api/src/app/agents/—agentsModule, the/agent-toolsroute, and the external MCP tool validatorpackages/server/api/src/app/flows/flow-version/migrations/— the agent step migrations (v7, v8, v14, v15, v16)packages/core/utils/src/lib/ssrf-ip-classifier.tsandpackages/server/utils/src/safe-http.ts— the SSRF guard on outbound calls
Paths verified 2026-07-17. An earlier version pointed at packages/core/shared/src/lib/automation/agents/; those types now live in packages/core/piece-types/src/lib/agents.ts and packages/core/execution/src/lib/agents/.
- Read a chat or saved-agent tool name only through
agentToolSkills.effectiveToolCall. Those surfaces run with skills (agentToolSkills.surfaceFor), so most calls arrive asap_lazy_tool({ tool, input })and any check ontoolNamesees the wrapper. Billing (completedToolCalls), the build-phase switch,buildStepParts,collapseStaleToolOutputspinning and the live stream (unwrapLazyToolChunk) all unwrap first. A new name-keyed check that skips this silently treats every deferred tool asap_lazy_tool. Never changetoolsoractiveToolsmid-conversation either, or the prompt cache is rebuilt from scratch. - The catalog of deferred tools is rendered in the worker, not the API. Configured and MCP tool names are only final after
agentToolPolicy.withValidNames, so a catalog the API built from raw names would point the model at tools that do not exist. ap_load_skilltakes the skill's name, not its guide topic. The build skill isflow_buildingand its guide topic isbuild_flow; a prompt that tells the model to loadbuild_flowgets the call rejected and the build guide never arrives. Name skills byAGENT_SKILLS[].namein every prompt, including the task-agent prompt.- Every agent turn fetches the model catalog over the network unless it is already cached. Production warms it at worker start; tests do not, so an eval that runs a turn waits on the CDN and times out at random when it is slow. Worker tests point
AP_MODEL_CATALOG_URLat an.invalidhost invitest.setup.ts, so run any new agent-turn test under the worker's vitest config. - A tool whose arguments the model guesses must be sent directly, not behind
ap_lazy_tool. Through the wrapper the provider no longer checks a tool's input, so a model that never loaded the schema guesses it: the connection picker was called withlabelinstead ofdisplayName, which cost retries and crashed the web card that readpiecefrom the input before the dispatcher rejected it. The connection cards, auth discovery, the question card, piece props and dropdown resolution are therefore inCORE_TOOL_NAMES; only rarely used tools stay deferred. - Paid tools are never sent directly in skills mode; they always go through
ap_lazy_tool.buildSkillSurfacedrops everychatBilling.isPaidToolname fromcoreToolNames, even for a small tool set, andCORE_TOOL_NAMESholds none (a test pins it). An unaffordable call is then refused in the wrapper'svalidate, which neither runs nor bills it. Hiding the tool fromactiveToolsonce credits ran low changed the tool list mid-turn and lost the prompt cache, and refusing insideexecuteby returning{ error }still counted as a completed paid call, so billing charged for work that never ran. - A rule that must hold in every chat belongs in the system prompt, not a guide. Guides arrive only when their skill is loaded. When "never publish without a yes" lived only in
build_flow, a follow-up chat ("Nice, thanks.") published the flow and called it live. - A note that applies to one message goes in that user message, not the system prompt. The onboarding first-message note used to sit in the system prompt, so it vanished on message two and invalidated the whole prompt cache.
agentSurfaceNotes.onboardingNotenow rides on the first user message instead.
Knowledge base gotchas
Upload and query embed with one key: the Embeddings capability, else the Default AI key (
knowledgeBaseService.embeddingModelFor). The query ignores the run key the worker sends. Changing that key does not re-embed anything: files uploaded before keep the old provider's vectors (same 768 dimensions, so nothing errors) and match poorly until they are uploaded again.An upload is embedded when it arrives, or refused.
knowledgeBaseService.uploadFilechunks and embeds the file in memory before writing anything, then saves the blob and writes the file record and its chunks in one transaction (the blob is deleted if that fails). It refuses withKNOWLEDGE_BASE_NEEDS_AI_PROVIDERwhen the project has no chat provider andKNOWLEDGE_BASE_FILE_HAS_NO_TEXTwhen nothing searchable comes out; the upload dialog maps both to translated messages. Embeddings bill like any other managed-provider call. Files uploaded before this fix are indexed on the agent's first search of them (embedMissingChunks, run only from the agent's knowledge-base tool, never from the read-only/searchroute). If that fails, the agent says the file could not be indexed, and uploading it again shows why.knowledge_base_chunkis created by a migration that records itself as run even when pgvector is absent. A database that gains pgvector later never gets the table, because the migration is already marked complete. Deleting its row frommigrationsreplays it safely, since the DDL isCREATE TABLE IF NOT EXISTS.Embeddings are stored at a fixed 768 dimensions, and most models do not return that.
text-embedding-3-smallanswers 1536, and thedimensionsprovider option is namespaced underopenai, so the OpenRouter and managed paths never see it.aiUtils.toStorageEmbeddingtruncates and re-normalises instead, which is what the option does server-side and works whatever the provider returns. This only holds for Matryoshka-trained models — adding a model that is not one will truncate badly and silently.Every agent write builds untouched fields from the locked row.
agentService.updatetakeslockedAgentInProjectOrThrowand reads draft, published, visibility and share list from that row, likemoveandeditDraftTools. Values read before the lock are only for permission checks; saving them back undoes a concurrent unpublish, draft edit or share revoke.Saving a saved agent publishes it.
POST /v1/agents/:idsetsgoLive: trueunless the body says otherwise, so an ordinary save copies the draft over the published snapshot. There is no separate publish step in the UI, deliberately: two versions with no history means nobody can say which one a linked flow runs. The consequence is easy to trip over in tests and callers — anything that needsdraftto differ frompublishedhas to write the row directly (db.update('agent', id, { draft })) or passgoLive: false, which is what the Test tab uses to stage a change it can run without shipping it. A test that edited the draft through the API to prove "a flow runs the published copy" was quietly moving the copy it was asserting about, and it only started failing when the save-publishes change merged from another branch. The Automations page hits the same trap: renaming an agent or filing it into a folder is a save, so those calls passgoLive: false; without it, moving an agent into a folder silently ships its staged draft.Agent folder counts are computed in the browser, not in
folder.servicelist.agent.folderIdexists, butFolderDtoonly counts flows and tables. The Automations page adds agents from the visibility-filtered/v1/agentslist. A server-sideCOUNT(*)would countRESTRICTEDagents the viewer can't see, so the count would disagree with the rows shown.ap_add_agent_toolswill save a tool with no connection pinned.connectionExternalIdis optional, so a tool the AI adds without one carries nopredefinedInput.authfor good. The visible symptom is a connection picker card on every conversation with that agent, which reads as the card being broken or the credential expiring — it is neither. The agent has nothing to use, so it asks, and the answer only ever lands on the run (__store_selected_connectionwrites a map the agent tool set does not read), so the next conversation asks again. Fixing the card is the wrong end: pin a connection when the tool is created, and write a chosen or repaired one back intodraft.toolsviaeditDraftTools. The same gap shows up one turn apart inside a single conversation:selectedConnectionByPieceis rebuilt empty per run inexecute-agent-run.ts, and the picker's only skip check readspiecesTheAuthorGaveAnAccount, which is built empty unlesssource === AgentRunSource.AGENT. So on the chat surface the card re-asked for an account the user had picked ninety seconds earlier. The durable copy was already being written; nothing read it back. The picker now asks the server for it through__get_selected_connectionand seeds the in-memory map from the answer, which fixes auth injection across turns too. Scope that skip toap_show_connection_pickerand nothing else:ap_show_connection_requiredis the reconnect card (status: 'error'), so refusing it because a selection exists would make a broken connection unrepairable, which is the one case the prompt insists must always be fixable inline. The picker takesswitchAccountfor when the user wants a different account. Leave the olderaccountAlreadyChosenForrefusal alone while you are in there: it fires only when a saved agent runs on a connection its author pinned, and it deliberately blocks both switching and reconnecting, because the card offers "use a different account" and the person chatting with someone else's agent should not be able to repoint that credential. Compose the two guards at the call site rather than nesting one inside the other, or a reviewer reads the pre-existing rule as part of the new one.An agent asked to change itself will send back the entire system prompt as its new brief.
ap_update_agentreplaces the instructions rather than patching them, so the note tells the agent to read its current brief first. On an agent run that brief is the top of the system prompt and every run note is appended under it, and the model reads "your current instructions above" as the whole message: a 183 character brief came back as 3,156, carrying the capabilities note, the connection guidance and the self-edit note itself, which would then be appended again on the next run and grow every turn.agentSurfaceNotes.stripRunNotescuts the payload at the first run-note heading and both write paths go through it. It needs a heading on its own line and two of them present, because a hand-written brief may legitimately contain one of those strings and truncating it silently is worse than the pollution.The taint flag must be set where the tool's
executeis built, not where the group is assembled.TaintStateis one mutable object per job, and the two readers are the action-preview gate and the refusal to change a saved agent after a read. Three paths reached the model without marking, each because it joined the tool set past the policy: the provider's native web search, the flow step's MCP tools (merged inrun(stepMcpToolSet)), and the conversation MCP set (throughagentMcpClient.withToolTimeouts). Set it inwithToolTimeoutsand the configured-tool factories, before awaiting, so a same-batch call cannot slip through. Provider search never reaches the chat model directly: plugin search has no tool call at all, and native Anthropic/Google search has noexecute, so neither can be wrapped. Without a Tavily key,ap_web_searchruns the provider's own search in a sub-call on the fast model (createProviderSearchTools) and taints like any other tool. See decision 000041. But marking every MCP tool was too wide, and it broke agent building outright.ap_list_connectionsandap_research_piecesare MCP-served and return only the platform's own catalog, yet they tainted, soap_add_agent_toolrefused the write its own description had just told the model to prepare ("Look them up first"). Following the instructions correctly was what broke it.agentToolPhases.taintsTurnnow exempts a short catalog-only list (ap_research_pieces,ap_search_actions,ap_search_triggers,ap_list_connections) and unknown names still taint, so a new tool is never silently exempt. It lives intool-phases.tsbesideBUILD_ONLY_TOOL_NAMESandCHAT_HIDDEN_TOOL_NAMESbecause it classifies the same name space; a near-twin already exists server-side asPLATFORM_LEVEL_TOOL_NAMESin the MCP registry, differing only by+ap_list_connectionsand-ap_get_piece_props. Keep anything that can reach a third party off the list however metadata-ish it looks:ap_get_piece_propsandap_resolve_property_*run a piece's dynamic-props function against its API when givenauth, andap_list_ai_modelscan fetch a provider's model list over the network. The two layers are not symmetric, and it matters which way you reconcile them. The server's Redis flag is set by three RPCs only, so it is blind to MCP and web reads:ap_find_records,ap_list_runsand everymcp__connector__*call never reachmarkTurnAsHavingRead. The worker is the only mark for those and has to stay wider, so the server is the tighter list for catalog tools, not the model to copy wholesale, and narrowing the worker to match it would reopen the hole. A server-owned mark is reachable in principle, contrary to what this note used to claim: the agent's MCP server is the API (${frontendUrl}/mcp/platform) and the worker already sendsx-ap-conversation-idon every MCP request, so marking in the MCP tool-call handler would work given arunIdalongside it. Nobody has done it, but do not cite impossibility as the reason to keep both copies.Build the agent before reading the user's data, not after. A turn that reads the user's mail, rows or records may not then change a saved agent, and the model's instinct is to look at the real data first so the instructions fit what is actually there. It then ends the turn describing an agent it cannot create and asking the user to say "create it". The order has to be: stand it up from what the user said plus the piece catalog, add its tools, then explore and refine with
ap_update_agenton the next turn. That rule lives in three places that must agree, since the model only ever sees whichever one it is reading: theap_create_agentandap_add_agent_tooldescriptions insession-tools.ts, the agent paragraph inchat-system-prompt.md, and the refusal string itself, which should teach the ordering rather than just apologise. Treat it as a nudge, not a fix. Measured over repeated runs of the same request it holds sometimes and not others: one run created the agent first and published it with its Gmail tools in a single turn, the next reached forap_explore_datato read the real labels first and was refused exactly as before. Hiding the agent-surface tools once tainted does not help either, it only turns the refusal into an absence and the agent still is not created. The deterministic fix, unimplemented, is to let a tainted turn make the change behind the approval cardwaitForApprovalalready provides: a human sees the diff before it lands, which keeps the property the guard exists for while removing the dead end. Do not instead make the guard advisory on the first offence, that hands an injection one free write. Agent mutations all return aurlfromafterDraftChange; the<links>block has to tell the model to use it, or a successful build ends with no way to open what was built.A step only persists the inputs its pinned piece version declares.
validatePropsinflow-version-validator-util.tsbuildscleanInputfromObject.keys(propsSchema.shape), so any key the pinned version does not know is dropped on save with no error. This bit the agent picker:AgentLinkrendered on every Run Agent step, butagentIdonly exists from@activepieces/[email protected], so on an older pinned step linking looked like it worked and was gone on reload. Anything that writes a new prop from a custom component has to gate on that prop being in the step'sselectedAction.props, not on a plan flag. Two dev-environment corollaries:packages/webconsumes@activepieces/core-piece-typesfrom its built dist, so a branch that adds an enum member needs that package rebuilt before web typechecks, and the API caches dev piece metadata in memory, so a piece version bump needs the API restarted before the builder pins the new version. You cannot fake the new version locally to test the prop: inserting the row intopiece_metadataby hand works for a few minutes and thenpieceSyncServicereconciles the table against the published registry and deletes a version that is not there, so the builder goes back to pinning the old one. A prop added to a piece is only exercisable end to end in the builder once that piece version is actually published, which is why the linked-step behaviour is pinned byagent-run-link.test.tsat the API level rather than by a browser pass.A queue failure reading
RPC [getAgentConfig] handler threw: VALIDATIONhides the real reason. The thrownError's message is only the error code, so BullMQ'sfailedReasonshows the same string for every validation throw in the config path. The full text does cross the RPC in__rpcApError: the worker reads it withapErrorOf(err)?.messageand writes it to the conversation's saved failure, the client error event and the flow step output, and the API logs it asRPC handler threw. Read one of those, not the queue. Group a backlog byplatformIdandmodelNamefromjob.databefore calling it one incident: the Sep 13 to Oct 1 2026 wave of 28.7k was 3 platforms.Unit tests do not run on the shipped tier list.
test/unit/app/ee/agent/model-tier-fixture.tspublishes chat tiersfast/smart/turbo(nopremium) and different model ids, so a test that hard-codespremiumor a shipped model id fails on CI. Read ids from the fixture ormodelTierCatalog.current(surface).