AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(chat): a tool call that fails shows as failed and is no longer charged (#16952)
Hazem Adelb7be720aae54
diff --git a/brain/knowledge/platform-editions-ee/ee-platform-plans-billing.md b/brain/knowledge/platform-editions-ee/ee-platform-plans-billing.mdindex 8a89370f89e..ebfca3458dc 100644--- a/brain/knowledge/platform-editions-ee/ee-platform-plans-billing.md+++ b/brain/knowledge/platform-editions-ee/ee-platform-plans-billing.md@@ -41,7 +41,7 @@ Billing and entitlements are powered by [Autumn](https://useautumn.com). Each pl- `AutumnFeatureId` (`platform.model.ts`) is a three-way contract: each value must equal BOTH the `platform_plan` column name (the projection writes them verbatim via `mapAutumnFeaturesToPlatformPlan` and forwards them as Autumn `featureId`s) AND the feature id configured in the Autumn dashboard. Renaming any one side silently breaks projection or metering for that feature. One deliberate exception: feature id `teamProjectsLimit` projects onto `plan.billedTeamProjectsLimit` (decision 000019).- Every path that admits a **production** run must go through `shouldBlockRunOnCredits` / `assertRunCreditsNotExceeded` (`billing-provider.ts`) — there are four entry points (sync webhook, worker `submitPayloads`, manual trigger, retry), and the gate was originally added to only the first two, so a zero-credit platform could replay its whole `QUOTA_EXCEEDED` backlog and get every run executed for free. Webhook/polling/manual-trigger admit a `QUOTA_EXCEEDED` run instead of running it (the builder renders the out-of-credits message); retry throws `QUOTA_EXCEEDED` (402) so single retry, `bulkRetry`, and the MCP `ap_retry_run` tool all refuse. Testing runs are never gated here — their AI spend is gated at the AI proxy instead.- **`AP_EDITION=ee` short-circuits all four run gates** — `shouldBlockRunOnCredits` returns `false` before touching the provider, so a self-hosted EE box does zero billing I/O on run admission. This is a latency stopgap, not policy, and the cost it was dodging is now much smaller: since the cache-only rewrite the gate costs two Redis reads per admission bounded at 25 ms, with no lock and no inline Autumn call on any path. What remains before the branch can go is an in-process TTL cache so an unenrolled platform costs nothing per run (decision 000020). Chat and managed-AI gates are unaffected and still run on every edition.-- Only `PersistedToolCallStatus.COMPLETED` tool calls are billable (`chatToolBilling.countBillableToolCallsInLatestTurn`). `ERROR` means the call never returned a result at all, so there is nothing to charge for; a tool that ran and returned a `❌ …` failure message is `COMPLETED` and *is* billed, because the third-party work happened. This count was telemetry-only before credits — treat any change to it as a pricing change.+- Only `PersistedToolCallStatus.COMPLETED` tool calls are billable (`chatToolBilling.countBillableToolCallsInLatestTurn`). `ERROR` means the call gave no result: the SDK refused it, it was never answered, or the tool threw (an MCP error or timeout, saved with its `errorText` since AI-111, 2026-10-27). None of those are charged. A tool that ran and returned a `❌ …` failure message is `COMPLETED` and *is* billed, because the third-party work happened. This count was telemetry-only before credits — treat any change to it as a pricing change.- **Autumn's auto top-up lands *after* the `track` response returns, so `trackCredits` caches a balance that is already wrong.** Verified against the sandbox (2026-07-29): a `track` that crosses the threshold returns `remaining` at its *pre*-top-up value, and the top-up appears only on a subsequent `getCustomer`. Since `trackCredits` writes `response.balance` verbatim with a fresh `syncedAt`, an exhausting run pins `remaining: 0` in Redis and `isCreditsStale` would suppress the refetch for `CREDITS_REFETCH_PERIOD_MS` (180s) — a funded platform with working auto-recharge gated for three minutes. Two guards close this: `scheduleCreditsCacheMaintenance` refreshes whenever the cached balance is stale **or would actually block** (so unenforced plans sitting at zero never trigger a call), fired through `rejectedPromiseHandler` and debounced to one `getCustomer` per platform per `CUSTOMER_STATE_REFRESH_DEBOUNCE_SECONDS` (15s) by `runOnceWithin`; and `/consumable-product-topups/auto-topup` now calls `refreshEntitlements` after `configureAutoTopUp`, like its four sibling routes always did. Do not try to derive that debounce from the cache's own `syncedAt` — `trackCredits` stamps it fresh, so "recently written" and "recently verified against Autumn" are different facts and conflating them disables the guard. The refresh is **background**, so the request that first sees the stale zero is still blocked and a topped-up platform keeps producing `QUOTA_EXCEEDED` runs for up to that debounce window; re-verifying inline is what this deliberately gave up, because it cost a RedLock acquire (retrying every 200 ms for up to its full 15s TTL when contended) plus an Autumn round-trip, on the webhook path. Related evaluation semantics, same verification: a `getCustomer` read does **not** trigger evaluation; a `billing_controls` mutation **does** (if already below threshold); one evaluation grants exactly `quantity` once and does not loop to clear the threshold; `track` with `value: 0` is accepted, deducts nothing, and is not a usable way to force evaluation. Note `value` defaults to **1** when omitted.- With `overage_allowed: false` a balance **floors at zero and never goes negative** — tracking 60500 against a `remaining` of 54990 deducted only 54990 and silently dropped the excess (`usage` capped at `granted`). Combined with post-run bulk AI metering (decision 000016), a single expensive run against a nearly-empty balance under-bills by the overflow instead of carrying it.- **Never write `platform_plan` by loading the row and spreading it into `save()`** — `platformPlanService.update()` and `setAutumnCredentials()` both did (`save({ ...platformPlan, ...changes })`) and it cost an activation. TypeORM `save()` re-`SELECT`s the row and diffs your object against it, so a column *another request committed* between your read and the save is indistinguishable from one you edited on purpose, and your stale value wins. A `refreshEntitlements` → `update()` (no lock, fires on any plan read) overlapping `activateLicense`'s `setAutumnCredentials` (holds the enroll lock, which the refresh never takes) reverted `autumnCustomerId`/`autumnApiKey` to the pre-activation customer while keeping the new `licenseKey` — the platform then reads entitlements and meters credits against an orphaned free customer, and `ensureEnrolled` early-returns on any non-nil customer id so it never self-heals. Both now use targeted `repo().update({ platformId }, changes)`, which cannot write a column its caller didn't name. Two traps when working on this: the window is **not** the slow `getCustomer` call (those creds only build the client and are never written back) but the ~1ms gap between `update()`'s own `findOneByOrFail` and `save()`'s internal reload; and because `save()`'s diff emits a *narrow* `UPDATE` whenever nothing raced, a test that mutates credentials before calling `update()` passes on the broken code too — `plan-update-column-isolation.test.ts` forces the interleave by spying on the shared repository's `findOneByOrFail` to commit the activation mid-call.diff --git a/packages/server/utils/src/agent-ai-utils.ts b/packages/server/utils/src/agent-ai-utils.tsindex 31f4b179b43..3bbd4c59046 100644--- a/packages/server/utils/src/agent-ai-utils.ts+++ b/packages/server/utils/src/agent-ai-utils.ts@@ -1,4 +1,4 @@-import { AIProviderName, isNil, spreadIfDefined } from '@activepieces/core-utils';+import { AIProviderName, isNil, spreadIfDefined, toError } from '@activepieces/core-utils';import { agentPersistenceUtils, agentToolClassification, agentToolSkills, PersistedAgentPart, PersistedAgentPartType, PersistedToolCallStatus } from '@activepieces/shared';import { agentProviderOptions } from './agent-provider-options'import { ModelMessage, TelemetryOptions } from 'ai'@@ -157,6 +157,7 @@ function toRecord(value: unknown): Record<string, unknown> {type ContentPartLike = {type: stringinvalid?: boolean+ error?: unknowntext?: stringtoolCallId?: stringtoolName?: string@@ -221,6 +222,7 @@ function buildStepParts({ content }: {break}const result = part.toolCallId ? resultMap.get(part.toolCallId) : undefined+ const threw = result?.type === 'tool-error'const rawOutput = result?.output ? agentPersistenceUtils.unwrapToolOutput(result.output) : undefinedconst title = typeof input['title'] === 'string' ? input['title'] : undefinedconst description = typeof input['description'] === 'string' ? input['description'] : undefined@@ -232,7 +234,8 @@ function buildStepParts({ content }: {...spreadIfDefined('description', description),input,output: rawOutput,- status: result && part.invalid !== true ? PersistedToolCallStatus.COMPLETED : PersistedToolCallStatus.ERROR,+ status: result && !threw && part.invalid !== true ? PersistedToolCallStatus.COMPLETED : PersistedToolCallStatus.ERROR,+ ...spreadIfDefined('errorText', threw && !isNil(result.error) ? toError(result.error).message : undefined),})if (toolName === 'ap_execute_action' && typeof rawOutput === 'object' && rawOutput !== null && 'batchProgress' in rawOutput) {parts.push({diff --git a/packages/server/utils/test/agent-ai-utils.test.ts b/packages/server/utils/test/agent-ai-utils.test.tsindex 5fa7a5dde86..a9dc499a5e0 100644--- a/packages/server/utils/test/agent-ai-utils.test.ts+++ b/packages/server/utils/test/agent-ai-utils.test.ts@@ -145,24 +145,24 @@ describe('collectStepMessages', () => {})describe('buildStepParts — tool call status', () => {- it('bills a call that ran, even one that then threw, and never one the SDK refused to run', () => {+ it('saves a call that threw as an error with its message, and one the SDK refused to run as an error', () => {const content = [{ type: 'tool-call', toolCallId: 'ran', toolName: 'ap_web_search', input: { query: 'a' } },{ type: 'tool-result', toolCallId: 'ran', toolName: 'ap_web_search', output: { type: 'json', value: { answer: 'ok' } } },{ type: 'tool-call', toolCallId: 'threw', toolName: 'mcp__crm__create_record', input: { name: 'b' } },- { type: 'tool-error', toolCallId: 'threw', toolName: 'mcp__crm__create_record' },+ { type: 'tool-error', toolCallId: 'threw', toolName: 'mcp__crm__create_record', error: new Error('MCP tool timed out') },{ type: 'tool-call', toolCallId: 'refused', toolName: 'ap_web_search', input: { query: 'c' }, invalid: true },{ type: 'tool-error', toolCallId: 'refused', toolName: 'ap_web_search' },{ type: 'tool-call', toolCallId: 'unanswered', toolName: 'ap_web_search', input: { query: 'd' } },]- const statuses = agentAiUtils.buildStepParts({ content }).flatMap((part) => part.type === PersistedAgentPartType.TOOL_CALL ? [[part.toolCallId, part.status]] : [])+ const statuses = agentAiUtils.buildStepParts({ content }).flatMap((part) => part.type === PersistedAgentPartType.TOOL_CALL ? [[part.toolCallId, part.status, part.errorText]] : [])expect(statuses).toEqual([- ['ran', PersistedToolCallStatus.COMPLETED],- ['threw', PersistedToolCallStatus.COMPLETED],- ['refused', PersistedToolCallStatus.ERROR],- ['unanswered', PersistedToolCallStatus.ERROR],+ ['ran', PersistedToolCallStatus.COMPLETED, undefined],+ ['threw', PersistedToolCallStatus.ERROR, 'MCP tool timed out'],+ ['refused', PersistedToolCallStatus.ERROR, undefined],+ ['unanswered', PersistedToolCallStatus.ERROR, undefined],])})})