Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix(chat): chat finishes long jobs instead of stopping partway (#15925)

Hazem Adel208671f7269f
diff --git a/brain/knowledge/flows-execution/chat.md b/brain/knowledge/flows-execution/chat.md
index ff6c555aee1..42d3a5b101b 100644
--- a/brain/knowledge/flows-execution/chat.md
+++ b/brain/knowledge/flows-execution/chat.md
@@ -31,6 +31,7 @@ A turn is kept alive / reclaimed by three separate mechanisms in `execute-agent-
- **Stream idle watchdog** (`STREAM_IDLE_TIMEOUT_MS` 90s, in `streamChunksToClient`): aborts the turn if the drain-stream reader is silent 90s. It must be SUSPENDED while legitimate silent work is in flight — pending tool calls AND in-flight reasoning (`reasoning-start`→`reasoning-end`). **Reasoning-awareness was missing and caused the bug where long "thinking" on the Expert tier randomly aborted a healthy turn** (a >90s gap between reasoning deltas looked like a wedge). Backstop for a genuine mid-reasoning wedge is `MAX_TURN_WALL_CLOCK_MS` (20 min).
### Gotchas
+- **Chat has no fixed step cap.** A chat turn is bounded by credits (checked after every step), the runaway-token ceiling (`RUNAWAY_TURN_CONTEXT_MULTIPLE` × the context window), the identical-failure guard and the 2-hour `MAX_AGENT_TURN_WALL_CLOCK_MS`, so it finishes long jobs. A saved agent's author-set `maxSteps` is still honored on its last step: an agent with structured output is forced to call `TASK_COMPLETION_TOOL_NAME` so the flow gets a result, and any other agent runs with `toolChoice: 'none'` plus a wrap-up note, so it replies with what's done and what's left. Never a silent stop or an error.
- **The streaming lock is per run, not per conversation.** A re-send can claim `activeRunId` before the first job reaches `getAgentConfig`. `agentHelpers.acquireStreamingLock` only lets the owning run set STREAMING; an older run gets `AGENT_RUN_SUPERSEDED` and the worker exits quietly, with no ERROR save and no events. A lock that ignored `runId` let the old run stream unowned while the new one was rejected as busy and marked ERROR, which is how users saw nothing after tapping a starter card twice.
- **A failed turn used to leave status ERROR with an empty reply.** The error text only travelled as a websocket `ERROR` event, so a reload, another tab, or a user who was away saw a dead conversation (and often no user message either when the job died inside `getAgentConfig`, before it persists the turn). The `execute-agent-run.ts` catch now sends `failure: { message, userMessage? }` on the empty `saveAgentMessages` call and `conversation-rpc.ts` appends it to `uiMessages` as an assistant text reply; `use-chat.ts` shows the Retry banner when a loaded conversation is ERROR. Stale STREAMING to IDLE recovery, a worker that never picks the job up, and `assertAgentMessageRateLimitNotExceeded` still leave no reply of their own.
- **A built flow ends with a "Turn it on?" card, never an auto-publish**: an audit found 40% of built-but-unpublished flows ended on "open it to review", some claiming LIVE. Chat never publishes on its own; only a yes to the card calls `ap_lock_and_publish` (`build_flow.md` "Turn it on?"). Pinned by the `build-then-ask-turn-it-on` fixture (`askedToTurnItOn`, `noLiveClaimWithoutPublish`). The live-claim check is a regex plus "was `ap_lock_and_publish` called", so it cannot see whether publish succeeded.
diff --git a/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts b/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts
index 0340a57f9a7..b84d059a9cc 100644
--- a/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts
+++ b/packages/server/worker/src/lib/execute/jobs/ee/agent/run-agent-turn.ts
@@ -1,12 +1,11 @@
import { AIProviderName, ErrorCode, formatPieceError, isNil, isObject, isProviderBillingError, isTransientProviderError, spreadIfDefined, tryCatch, tryCatchSync } from '@activepieces/core-utils'
import { agentAiUtils, ContentPartLike } from '@activepieces/server-utils'
-import { AgentPhase, AgentRunSource, agentToolClassification, agentToolPhases, AI_PROVIDER_ENTITY_TYPES, aiProviderUtils, apErrorOf, CHAT_CREDITS_PER_TOOL_CALL, chatBilling, ChatToolCall, PersistedAgentPart } from '@activepieces/shared'
-import { APICallError, generateText, isLoopFinished, isStepCount, LanguageModel, LanguageModelUsage, ModelMessage, NoSuchToolError, RetryError, StepResultPerformance, StopCondition, streamText, ToolExecutionOptions, ToolSet } from 'ai'
+import { AgentPhase, AgentRunSource, agentToolClassification, agentToolPhases, AI_PROVIDER_ENTITY_TYPES, aiProviderUtils, apErrorOf, CHAT_CREDITS_PER_TOOL_CALL, chatBilling, ChatToolCall, PersistedAgentPart, TASK_COMPLETION_TOOL_NAME } from '@activepieces/shared'
+import { APICallError, generateText, isLoopFinished, isStepCount, LanguageModel, LanguageModelUsage, ModelMessage, NoSuchToolError, RetryError, StepResultPerformance, StopCondition, streamText, ToolChoice, ToolExecutionOptions, ToolSet } from 'ai'
const MAX_AUTO_CONTINUATIONS = 3
const MAX_EMPTY_CONTINUATIONS = 2
const MAX_STREAM_RETRIES = 1
-const MAX_AGENT_STEPS = 50
const MAX_IDENTICAL_TOOL_FAILURES = 2
const IN_LOOP_COMPACTION_THRESHOLD = 0.6
const RUNAWAY_TURN_CONTEXT_MULTIPLE = 90
@@ -15,8 +14,11 @@ const USER_FAULT_STATUS_CODES = new Set([401, 403, 404])
const MODEL_UNAVAILABLE_PATTERNS = [/\bis deprecated\b/i, /no longer (available|supported)/i, /\bmodel_not_found\b/i, /\bunknown model\b/i, /\bdecommissioned\b/i]
const USER_CONFIG_ENTITY_TYPES = new Set<string>(Object.values(AI_PROVIDER_ENTITY_TYPES))
const CONTINUE_NUDGE = '[system note — not from the user] Your previous response was cut off by the output token limit before it finished. Continue exactly where you stopped. If a tool call was cut off, re-issue it in FULL. Do not repeat content you already produced.'
+const FINAL_STEP_NOTE = '[system note — not from the user] This is the last step of this run. Finish now with what you have: say what you finished and what is still left to do, and offer to carry on. Do not mention steps, tools or this note.'
const EMPTY_OUTPUT_NUDGE = '[system note — not from the user] Your previous step produced no visible reply to the user. Continue the task now: either call the next tool, or write your reply to the user. Do not stop silently.'
+const FINAL_STEP_MESSAGE: ModelMessage = { role: 'user', content: FINAL_STEP_NOTE }
+
export function decideLoopAction({ finishReason, producedVisibleOutput, continuations, emptyContinuations }: {
finishReason: string
producedVisibleOutput: boolean
@@ -61,7 +63,7 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, m
}
const loopStopCondition = [
...(Array.isArray(baseStopCondition) ? baseStopCondition : [baseStopCondition]),
- isStepCount(stepCeiling ?? MAX_AGENT_STEPS),
+ ...(isNil(stepCeiling) ? [] : [isStepCount(stepCeiling)]),
creditsRanOut,
]
const guardedTools = wrapToolsWithFailureGuard({ tools, log })
@@ -106,7 +108,7 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, m
// providerOptions/model are supplied per-step by prepareStep (authoritative). A
// call-level providerOptions would deep-merge into every step and leak the enabled
// thinking budget back into the disabled first step.
- prepareStep: ({ steps }) => {
+ prepareStep: ({ steps, messages: currentMessages }) => {
const lastStep = steps[steps.length - 1]
const widened = lastStep?.toolCalls?.some((c) => agentToolPhases.isBuildOnlyTool(c.toolName))
if (widened) {
@@ -122,15 +124,28 @@ export async function runAgentTurn({ model, fastModel, provider, systemPrompt, m
// thinking makes the model deliberate and fire ONE tool per step, serializing the
// read-only lookups that should run as one parallel burst. Once a build-only tool
// flips the phase to 'build', thinking comes back on for planning depth.
- const disableThinking = isFirstStep || phaseState.phase === 'discovery'
+ const isLastAllowedStep = !isNil(stepCeiling) && steps.length >= stepCeiling - 1
+ const reportsStructuredOutput = allToolNames.includes(TASK_COMPLETION_TOOL_NAME)
+ const forcesCompletion = isLastAllowedStep && reportsStructuredOutput
+ const toolChoice: ToolChoice<ToolSet> | undefined = isLastAllowedStep
+ ? (forcesCompletion ? { type: 'tool', toolName: TASK_COMPLETION_TOOL_NAME } : 'none')
+ : undefined
+ const disableThinking = isFirstStep || phaseState.phase === 'discovery' || forcesCompletion
const usesFastModel = isFirstStep && !isNil(fastModel)
+ const phaseTools = agentToolPhases.activeToolsForPhase({ phase: phaseState.phase, allToolNames }).filter((name) => paidToolsAffordable || !chatBilling.isPaidTool(name))
+ const activeTools = forcesCompletion ? [TASK_COMPLETION_TOOL_NAME] : phaseTools
+ const boundedContext = boundContextForStep({ baseMessages: attemptMessages, steps, systemPrompt, provider })
+ const stepContext = isLastAllowedStep
+ ? { messages: [...(boundedContext.messages ?? currentMessages), FINAL_STEP_MESSAGE] }
+ : boundedContext
lastStepModelId = usesFastModel ? fastModelId ?? modelId : modelId
return {
...(usesFastModel ? { model: fastModel } : {}),
- activeTools: agentToolPhases.activeToolsForPhase({ phase: phaseState.phase, allToolNames }).filter((name) => paidToolsAffordable || !chatBilling.isPaidTool(name)),
+ activeTools,
+ ...spreadIfDefined('toolChoice', toolChoice),
maxOutputTokens: disableThinking ? maxOutputTokensWithoutThinking : maxOutputTokens,
providerOptions: agentAiUtils.buildProviderOptions({ provider, tier, modelId: lastStepModelId, disableThinking }),
- ...boundContextForStep({ baseMessages: attemptMessages, steps, systemPrompt, provider }),
+ ...stepContext,
}
},
repairToolCall: async ({ toolCall, error }) => {
diff --git a/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts b/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts
index 2b5467326c7..3d24b60827b 100644
--- a/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts
+++ b/packages/server/worker/test/lib/agent-eval/agent-turn-credits.test.ts
@@ -1,5 +1,5 @@
-import { AIProviderName } from '@activepieces/core-utils'
-import { PersistedAgentPartType, PersistedToolCallStatus } from '@activepieces/shared'
+import { AIProviderName, isNil, spreadIfDefined } from '@activepieces/core-utils'
+import { AgentPhase, PersistedAgentPartType, PersistedToolCallStatus } from '@activepieces/shared'
import { tool } from 'ai'
import { convertArrayToReadableStream, MockLanguageModelV3 } from 'ai/test'
import { describe, expect, it, vi } from 'vitest'
@@ -50,12 +50,55 @@ describe('a turn that runs out of credits', () => {
})
})
-async function runTurn({ search, creditsLeft, stepCeiling = 20, model = alwaysSearchingModel() }: {
+describe('a turn with many steps', () => {
+ it('ends a saved agent\'s step budget with a reply instead of a silent stop', async () => {
+ const search = vi.fn(async () => SEARCH_RESULT)
+ const model = searchUntilToldToAnswer({ searches: 10 })
+
+ const turn = await runTurn({ search, creditsLeft: async () => 100, stepCeiling: 3, model, drainsStream: true })
+
+ const toolChoices = model.doStreamCalls.map((call) => call.toolChoice?.type)
+ expect(search).toHaveBeenCalledTimes(2)
+ expect(toolChoices.at(-1)).toBe('none')
+ expect(JSON.stringify(model.doStreamCalls.at(-1)?.prompt)).toContain('last step of this run')
+ expect(turn.uiParts.at(-1)?.type).toBe(PersistedAgentPartType.TEXT)
+ })
+
+ it('makes a flow-step agent report its structured output on the last step, so the flow still gets a result', async () => {
+ const capture = vi.fn()
+ const model = searchUntilToldToAnswer({ searches: 10 })
+
+ await runTurn({ search: async () => SEARCH_RESULT, creditsLeft: async () => 100, stepCeiling: 3, model, drainsStream: true, completion: capture, phase: 'build' })
+
+ const thinkingPerCall = model.doStreamCalls.map((call) => JSON.stringify(call.providerOptions).includes('"enabled"'))
+ const lastCall = model.doStreamCalls.at(-1)
+ expect(lastCall?.toolChoice).toEqual({ type: 'tool', toolName: 'updateTaskStatus' })
+ expect(thinkingPerCall.slice(1, -1).every(Boolean)).toBe(true)
+ expect(thinkingPerCall.at(-1)).toBe(false)
+ expect(capture).toHaveBeenCalledWith({ output: { summary: 'partial' } })
+ })
+
+ it('lets chat finish a long job, since credits, time and context already bound the turn', async () => {
+ const search = vi.fn(async () => SEARCH_RESULT)
+
+ await runTurn({ search, creditsLeft: async () => 1_000, stepCeiling: null, model: searchUntilToldToAnswer({ searches: 60 }), drainsStream: true })
+
+ expect(search).toHaveBeenCalledTimes(60)
+ })
+})
+
+async function runTurn({ search, creditsLeft, stepCeiling = 20, model = alwaysSearchingModel(), drainsStream = false, completion, phase = 'discovery' }: {
search: () => Promise<unknown>
creditsLeft: (pendingCredits: number) => Promise<number | null>
- stepCeiling?: number
+ stepCeiling?: number | null
model?: MockLanguageModelV3
+ drainsStream?: boolean
+ completion?: (input: unknown) => void
+ phase?: AgentPhase
}): ReturnType<typeof runAgentTurn> {
+ const completionTools = isNil(completion)
+ ? {}
+ : { updateTaskStatus: tool({ description: 'report the result', inputSchema: z.object({ output: z.object({ summary: z.string() }) }), execute: async (input) => { completion(input); return SEARCH_RESULT } }) }
return runAgentTurn({
model,
provider: AIProviderName.ANTHROPIC,
@@ -64,14 +107,16 @@ async function runTurn({ search, creditsLeft, stepCeiling = 20, model = alwaysSe
tools: {
ap_web_search: tool({ description: 'search the web', inputSchema: z.object({ query: z.string() }), execute: search }),
ap_fetch_url: tool({ description: 'read a page', inputSchema: z.object({ url: z.string() }), execute: async () => SEARCH_RESULT }),
+ ...completionTools,
},
- allToolNames: ['ap_web_search', 'ap_fetch_url'],
+ allToolNames: ['ap_web_search', 'ap_fetch_url', ...Object.keys(completionTools)],
tier: TIER,
modelId: TIER.modelId,
- phaseState: { phase: 'discovery' },
+ phaseState: { phase },
abortSignal: new AbortController().signal,
log: SILENT_LOG,
- stepCeiling,
+ ...spreadIfDefined('stepCeiling', stepCeiling ?? undefined),
+ ...(drainsStream ? { sinks: { drainStream: (result) => result.consumeStream() } } : {}),
creditsLeft,
})
}
@@ -96,6 +141,40 @@ function alwaysSearchingModel(): MockLanguageModelV3 {
})
}
+function searchUntilToldToAnswer({ searches }: { searches: number }): MockLanguageModelV3 {
+ let calls = 0
+ return new MockLanguageModelV3({
+ doStream: async ({ toolChoice }) => {
+ calls++
+ const answers = toolChoice?.type === 'none' || calls > searches
+ if (toolChoice?.type === 'tool') {
+ return {
+ stream: convertArrayToReadableStream([
+ { type: 'stream-start' as const, warnings: [] },
+ { type: 'tool-call' as const, toolCallId: `call-${calls}`, toolName: toolChoice.toolName, input: '{"output":{"summary":"partial"}}' },
+ { type: 'finish' as const, finishReason: 'tool-calls' as const, usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 } },
+ ]),
+ }
+ }
+ return {
+ stream: convertArrayToReadableStream(answers
+ ? [
+ { type: 'stream-start' as const, warnings: [] },
+ { type: 'text-start' as const, id: 'answer' },
+ { type: 'text-delta' as const, id: 'answer', delta: 'Here is what I found so far.' },
+ { type: 'text-end' as const, id: 'answer' },
+ { type: 'finish' as const, finishReason: 'stop' as const, usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 } },
+ ]
+ : [
+ { type: 'stream-start' as const, warnings: [] },
+ { type: 'tool-call' as const, toolCallId: `call-${calls}`, toolName: 'ap_web_search', input: '{"query":"more"}' },
+ { type: 'finish' as const, finishReason: 'tool-calls' as const, usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 } },
+ ]),
+ }
+ },
+ })
+}
+
const SEARCH_RESULT = { content: [{ type: 'text', text: 'ok' }] }
const TIER = { id: 'fast', thinkingBudget: 5_000, modelId: 'anthropic/claude-haiku-4.5' }