AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(chat): long chats no longer fail a turn while older messages are being summarized (#16085)
Hazem Adelb38970ad62e6
diff --git a/brain/knowledge/flows-execution/chat.md b/brain/knowledge/flows-execution/chat.mdindex c24a0c5fcf2..14e6baff8e7 100644--- a/brain/knowledge/flows-execution/chat.md+++ b/brain/knowledge/flows-execution/chat.md@@ -41,6 +41,7 @@ A turn is kept alive / reclaimed by three separate mechanisms in `execute-agent-- **Write-check gate**: before a live `ap_test_flow`, `__flow_write_check` RPC flags write/destructive PIECE steps; read-only flows run ungated; gate fails open on RPC error.- **Chat follows the plan on Cloud and Enterprise.** `chatVisibility` returns `plan.chatEnabled` on both, and Community and embedded sessions never see chat. Opening chat to everyone on Cloud means granting `CHAT_ENABLED` on every Autumn plan, which `toPlatformPlanFlags` then writes to `chatEnabled`; the code has no Cloud override. The 200-user rollout cap (`CLOUD_CHAT_ROLLOUT_CAP`) and its grandfathering were removed when chat went public on 1 Oct 2026, so a beta user whose plan lacks the grant loses chat. It cached its closed state in Redis under `chat-rollout:closed` with no expiry, which is why raising the env var never reopened it; that key is now ignored and safe to delete.- **Flow correctness is 100% prompt/guide-driven — nothing in code enforces it.** The "#1 silent bug" ("Class A"): the agent frames a *recurring* automation as a *one-time task* and omits any anti-reprocessing step, so run N+1 redoes run N's work (re-pays, re-sends). It's a design-time reasoning gap, not a testing gap — `ap_test_flow` runs ONCE, so a single test looks perfect; the bug only shows on the 2nd run. Fix lives in the prompt (`chat-system-prompt.md` `<decision_framework>` + `build_flow.md` "Recurring flows must not reprocess") + capability eval fixtures with a `recurring_avoids_reprocessing` judge dimension. The platform already has every primitive (Tables New-Record webhook, polling `DedupeStrategy`, `_dedupe_key`, Store, update/delete-record); the agent just wasn't reaching for them. Watch the `build_flow.md` "don't over-build" bias — it once actively discouraged the fix.+- **Compaction runs inside the 60s `getAgentConfig` RPC, so it must stay bounded.** The summarizer call is capped at `SUMMARY_OUTPUT_RESERVE_TOKENS` and aborted after `COMPACTION_TIMEOUT_MS` (35s). On failure the previous summary is kept and `buildCompactedPayload` still fits the history to the budget, flagging trimmed messages to the model. An unbounded summary once ran for 2.5 min and failed the turn as `RPC [getAgentConfig] failed (timeout: 60000ms)`.- **The eval replay harness drifts silently when a worker tool's signature changes.** `buildEvalToolSet` (`agent-eval/core/runner.ts`) hand-builds the tool set with stubs, and nothing type-checks `test/lib/agent-eval/` in CI, so a new required param never fails a build. Until Oct 2026 the eval passed no `taintState` to `createCrossProjectTools` (every data tool threw, transcripts showed `TOOL_RESULT: null`, and fixtures like the injection one never saw their data), `onSetProjectContext` returned void, and approvals lacked `outcome` so all counted as declined. When you change a tool factory's params, update the eval stubs too, and treat `TOOL_RESULT: null` in an eval transcript as a harness bug, not agent behaviour. Check with `npx tsc -p packages/server/worker/test/lib/agent-eval/cli/tsconfig.json --noEmit`.- **An eval verdict is a majority over repeats, graded by a different model.** LLM output is random, so one run proves little (two full runs of the same commit 13 seconds apart, Oct 2026, scored 55/78 vs 54/78 repeats but disagreed on 7 of 26 fixtures): `CHAT_EVAL_REPEATS` (nightly 3) runs each fixture N times; judge dimensions pass on a majority, but a deterministic assertion must hold on **every** run (one run that emails the attacker is a real failure, not noise). The report shows `passes/runs` and the first *failing* run's checks; `runVerdicts` in the JSON keeps every run. The judge is `CHAT_EVAL_JUDGE_MODEL` (default an Opus slug), never the agent's own model, which went easy on itself. The judge's transcript carries each tool call's input and result, not just its name, so it can see where an email went. The report's "expected-label match" only compares the judge to each fixture's `expectedLabel`, so an agent miss counts as a judge error; judge accuracy is the separate "judge vs human labels" line, from person-reviewed cases in `agent-eval/calibration/`. A model-written label measures Claude against Claude, so labels carrying `"labelledBy": "claude"` are excluded from it and only feed a separate "draft estimate" line until a person reviews them. Each run is published (public) to `cdn.activepieces.com/ai/evals/runs/`. Nightly runs cover gate fixtures only; a Sunday 07:00 UTC run covers all fixtures, and only those full runs feed the Config Console's **Agent score** (share of repeats passed), whose "+N pts" compares just the fixtures present in both runs so adding a fixture never fakes a gain. Prefer a deterministic assertion (`noToolArgMatches`, `neverCalledTool`) over a judge dimension wherever code can see the behaviour.- **The context budget subtracts what the request reserves.** Anthropic/OpenRouter count tool schemas and the reserved `max_tokens` against the 200k window, so `getAgentConfig` passes `reservedTokens` (the clamped output window from `agentAiUtils.affordableOutputTokens`, the same helper the worker uses for `maxOutputTokens`, plus `TOOL_SCHEMA_TOKEN_ESTIMATE = 12_000`) into `agentCompaction`. The threshold, the fit check and the recent window are all measured against `maxContext - reservedTokens`, and the recent window is sized by estimated tokens, not message count. The tool-schema figure is a constant because the tools are built in the worker and the API never sees their definitions. The worker sets `maxOutputTokens` per step in `prepareStep`, so a thinking-disabled step (round one, discovery phase) does not reserve `thinkingBudget`. The system prompt's size is added to the reserve for the window and fit check, and each message is capped at 20k chars in the summary request so the summarizer itself cannot overflow.diff --git a/packages/server/api/src/app/ee/agent/agent-compaction.ts b/packages/server/api/src/app/ee/agent/agent-compaction.tsindex 889d0dd5bfa..4a89cb1a2db 100644--- a/packages/server/api/src/app/ee/agent/agent-compaction.ts+++ b/packages/server/api/src/app/ee/agent/agent-compaction.ts@@ -1,6 +1,6 @@import { readFileSync } from 'node:fs'import path from 'node:path'-import { ActivepiecesError, AIProviderName, ErrorCode } from '@activepieces/core-utils'+import { ActivepiecesError, AIProviderName, ErrorCode, tryCatch } from '@activepieces/core-utils'import { agentAiUtils } from '@activepieces/server-utils'import { aiProviderUtils } from '@activepieces/shared'import { generateText, LanguageModel, ModelMessage } from 'ai'@@ -12,6 +12,7 @@ const CHARS_PER_TOKEN_ESTIMATE = 4const MIN_MESSAGES_BEFORE_COMPACTION = 6const MAX_TOOL_RESULT_CHARS_FOR_SUMMARY = 2_000const SUMMARY_OUTPUT_RESERVE_TOKENS = 4_000+const COMPACTION_TIMEOUT_MS = 35_000const COMPACTION_SYSTEM_PROMPT = readFileSync(path.resolve('packages/server/api/src/assets/prompts/chat-compaction-prompt.md'),@@ -115,14 +116,20 @@ async function compactMessages({ messages, existingSummary, summarizedUpToIndex,hadExistingSummary: !!existingSummary,}, 'Compacting chat messages')- const { text: summary } = await generateText({+ const { data, error } = await tryCatch(() => generateText({model,instructions: COMPACTION_SYSTEM_PROMPT,telemetry: agentAiUtils.buildTelemetry({ functionId: 'agent-compaction' }),prompt: contentToSummarize,- })+ maxOutputTokens: SUMMARY_OUTPUT_RESERVE_TOKENS,+ abortSignal: AbortSignal.timeout(COMPACTION_TIMEOUT_MS),+ }))+ if (error) {+ log.warn({ error }, 'Compaction failed or timed out, keeping previous summary')+ return { summary: existingSummary ?? '', summarizedUpToIndex: startIndex }+ }- return { summary, summarizedUpToIndex: newCutoffIndex }+ return { summary: data.text, summarizedUpToIndex: newCutoffIndex }}function buildCompactedPayload({ messages, summary, summarizedUpToIndex, provider, reservedTokens }: {@@ -132,18 +139,14 @@ function buildCompactedPayload({ messages, summary, summarizedUpToIndex, provideprovider: AIProviderNamereservedTokens: number}): ModelMessage[] {- if (!summary || summarizedUpToIndex === null) {- return messages- }-- const recentMessages = messages.slice(summarizedUpToIndex)- const summaryText = `[Previous conversation summary]\n${summary}\n[End of summary — conversation continues below]`+ const recentMessages = summary ? messages.slice(summarizedUpToIndex ?? 0) : messages+ const summaryBlock = summary ? `[Previous conversation summary]\n${summary}\n[End of summary — conversation continues below]` : ''const budget = contextBudget({ provider, reservedTokens })const threshold = budget * COMPACTION_THRESHOLDconst recentTokens = recentMessages.map((m) => tokensIn(JSON.stringify(m)))- let runningTokens = tokensIn(JSON.stringify(summaryText)) + recentTokens.reduce((a, b) => a + b, 0)+ let runningTokens = tokensIn(JSON.stringify(summaryBlock)) + recentTokens.reduce((a, b) => a + b, 0)let startIdx = 0while (@@ -154,7 +157,19 @@ function buildCompactedPayload({ messages, summary, summarizedUpToIndex, providestartIdx++}+ if (runningTokens > budget) {+ throw new ActivepiecesError({+ code: ErrorCode.CHAT_CONTEXT_LIMIT_EXCEEDED,+ params: {},+ })+ }+const trimmedRecent = recentMessages.slice(startIdx)+ const omittedNote = startIdx > 0 ? `[${startIdx} earlier messages were left out to fit the context window]` : ''+ const summaryText = [omittedNote, summaryBlock].filter(Boolean).join('\n')+ if (!summaryText) {+ return trimmedRecent+ }// Anthropic rejects consecutive same-role messages, so merge the summary// into the first message when it is already a 'user' turn.@@ -169,12 +184,6 @@ function buildCompactedPayload({ messages, summary, summarizedUpToIndex, provide...trimmedRecent.slice(1),]: [{ role: 'user', content: summaryText }, ...trimmedRecent]- if (runningTokens > budget) {- throw new ActivepiecesError({- code: ErrorCode.CHAT_CONTEXT_LIMIT_EXCEEDED,- params: {},- })- }return finalPayload}diff --git a/packages/server/api/test/unit/app/agent/agent-compaction.test.ts b/packages/server/api/test/unit/app/agent/agent-compaction.test.tsindex 38fd7293c23..34a589b96e6 100644--- a/packages/server/api/test/unit/app/agent/agent-compaction.test.ts+++ b/packages/server/api/test/unit/app/agent/agent-compaction.test.ts@@ -96,7 +96,25 @@ describe('agentCompaction.buildCompactedPayload', () => {provider: AIProviderName.ANTHROPIC,reservedTokens: 0,})- expect(result).toBe(messages)+ expect(result).toEqual(messages)+ })++ it('fits an oversized history with no summary into the budget and says what was left out', () => {+ const largeMessages: ModelMessage[] = Array.from({ length: 10 }, (_, i) => ({+ role: i % 2 === 0 ? 'user' as const : 'assistant' as const,+ content: `Message ${i}: ${'x'.repeat(200_000)}`,+ }))+ const result = agentCompaction.buildCompactedPayload({+ messages: largeMessages,+ summary: null,+ summarizedUpToIndex: null,+ provider: AIProviderName.ANTHROPIC,+ reservedTokens: 0,+ })+ expect(Math.ceil(JSON.stringify(result).length / 4)).toBeLessThanOrEqual(200_000)+ expect(result[0].role).toBe('user')+ expect(String(result[0].content)).toMatch(/\[\d+ earlier messages were left out to fit the context window\]/)+ expect(result.at(-1)).toBe(largeMessages.at(-1))})it('prepends summary and keeps only recent messages', () => {@@ -137,6 +155,7 @@ describe('agentCompaction.buildCompactedPayload', () => {// Should have trimmed some recent messagesexpect(result.length).toBeLessThan(6) // less than 1 summary + 5 recentexpect(result[0].content).toContain('[Previous conversation summary]')+ expect(String(result[0].content)).toMatch(/\[\d+ earlier messages were left out to fit the context window\]/)})it('throws CHAT_CONTEXT_LIMIT_EXCEEDED when even minimal payload is too large', () => {@@ -311,4 +330,33 @@ describe('agentCompaction with reserved tokens', () => {const request = vi.mocked(generateText).mock.calls.at(-1)?.[0]expect(String(request?.prompt)).toContain('THE-LAST-DETAIL')})++ it('caps the summary length and bounds how long it may run', async () => {+ await agentCompaction.compactMessages({+ messages: makeMessages(40, 20_000),+ existingSummary: null,+ summarizedUpToIndex: null,+ provider: AIProviderName.ANTHROPIC,+ reservedTokens: RESERVED_TOKENS,+ model: summaryModel,+ log: silentLog,+ })+ const request = vi.mocked(generateText).mock.calls.at(-1)?.[0]+ expect(request?.maxOutputTokens).toBe(4_000)+ expect(request?.abortSignal).toBeInstanceOf(AbortSignal)+ })++ it('keeps the previous summary when the summarizer times out, so the turn still runs', async () => {+ vi.mocked(generateText).mockRejectedValueOnce(new DOMException('The operation was aborted due to timeout', 'TimeoutError'))+ const result = await agentCompaction.compactMessages({+ messages: makeMessages(40, 20_000),+ existingSummary: 'earlier summary',+ summarizedUpToIndex: 4,+ provider: AIProviderName.ANTHROPIC,+ reservedTokens: RESERVED_TOKENS,+ model: summaryModel,+ log: silentLog,+ })+ expect(result).toEqual({ summary: 'earlier summary', summarizedUpToIndex: 4 })+ })})