Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix(webhook): failed webhook authentication now shows a failed run and stops sync flows (#14954)

Bartosz Majewski83677ba5d405
diff --git a/brain/knowledge/eventing-webhooks/webhooks.md b/brain/knowledge/eventing-webhooks/webhooks.md
index 905be3e104a..a61bfcdaf25 100644
--- a/brain/knowledge/eventing-webhooks/webhooks.md
+++ b/brain/knowledge/eventing-webhooks/webhooks.md
@@ -37,6 +37,7 @@ Webhooks are the primary entry point for event-driven flow execution from outsid
- **`workerHandlerId` and `httpRequestId` are persisted in exactly one place: `waitpoint.workerHandlerId` and `waitpoint.httpRequestId`, and only for paused runs.** `flow_run` has no such column and no migration ever added one, so for a plain sync run the two ids live only in the BullMQ job payload. That is why answering a waiting caller from the app side means carrying them back on the request rather than looking them up. A useful consequence of the waitpoint columns: an async resume inherits them (`resume-service.ts`), so a resumed run that then fails answers the original sync caller too.
- **Splitting a sync timeout into engine-class and worker-class tells you who can still answer.** Joining timed-out sync runs to their worker `job.execute` event over 3h gave 5,685 with outcome `success` (the engine reported the terminal status itself) against 141 `failed` (137 `SANDBOX_INTERNAL_ERROR`, the rest RPC timeout or socket exit). Note no `flowRun.status` attribute is shipped to ClickHouse, so separating a genuinely failed run from a silent success inside that 5,685 needs a Postgres query on `flow_run.status`.
- **The sync response reaches the caller before the run row says it failed.** `uploadRunLog` only *enqueues* the status onto `runsMetadataQueue`; the Postgres write happens later in that queue's worker, behind a distributed lock. The 500 is published in the same call, so a caller that gets its 500 and immediately reads the run over the API can still see `RUNNING`. Do not assert the persisted status straight after a sync response in a test: poll for it. An e2e assertion written that way failed on exactly this while the response itself was already correct.
+- **A trigger that rejects a request must throw, never return `[]`.** On async, `[]` just drops the event (no run, counted COMPLETED). On `/sync` it used to mark the trigger SUCCEEDED with an undefined payload and run every downstream step, so a Catch Webhook with a wrong secret still executed the flow (GIT-1749). `runOrReturnPayload` now fails the run on empty output. A throw gives a FAILED run with the reason on `/sync` (500), and a FAILED trigger-run count plus a worker warn on async. Async callers still get 200 because the API answers before any piece code runs; a real 401 is GIT-628.
- **A retry can never answer a sync caller.** Flow jobs run `attempts: 2` with exponential backoff starting at eight minutes (`job-queue.ts`), so the second attempt lands long after any 30s caller is gone. Publishing a 500 the moment a run fails therefore costs nothing, and waiting for a retry to maybe succeed would buy nothing.
### Editions
diff --git a/packages/pieces/core/webhook/package.json b/packages/pieces/core/webhook/package.json
index 6b5a23246bc..9d2e6b7e134 100644
--- a/packages/pieces/core/webhook/package.json
+++ b/packages/pieces/core/webhook/package.json
@@ -1,6 +1,6 @@
{
"name": "@activepieces/piece-webhook",
- "version": "0.1.41",
+ "version": "0.1.42",
"main": "./dist/src/index.js",
"types": "./dist/src/index.d.ts",
"scripts": {
diff --git a/packages/pieces/core/webhook/src/lib/triggers/catch-hook.ts b/packages/pieces/core/webhook/src/lib/triggers/catch-hook.ts
index 452cce5b09b..652f8a74a1a 100644
--- a/packages/pieces/core/webhook/src/lib/triggers/catch-hook.ts
+++ b/packages/pieces/core/webhook/src/lib/triggers/catch-hook.ts
@@ -34,7 +34,7 @@ If it takes more than {{webhookTimeoutSeconds}} seconds, it will return a 408 Re
To return data, add an Webhook step to your flow with the Return Response action.
`;
-enum AuthType {
+export enum AuthType {
NONE = 'none',
BASIC = 'basic',
HEADER = 'header',
@@ -47,7 +47,7 @@ export const catchWebhook = createTrigger({
description:
'Receive incoming HTTP/webhooks using any HTTP method such as GET, POST, PUT, DELETE, etc.',
aiMetadata: {
- description: 'Fires once for every request delivered to the unique webhook URL generated for this flow, under any HTTP method. Use it as the entry point for any system that can call a URL; Authentication must be set explicitly to None, Basic credentials, a shared header value, or an HMAC signature over the raw body, and requests that fail verification are silently dropped without starting a run. Appending /sync to the URL makes the caller wait for a Return Response action, while /test generates sample data without triggering the published flow.',
+ description: 'Fires once for every request delivered to the unique webhook URL generated for this flow, under any HTTP method. Use it as the entry point for any system that can call a URL; Authentication must be set explicitly to None, Basic credentials, a shared header value, or an HMAC signature over the raw body, and requests that fail verification are rejected and recorded as a failed run instead of starting the flow. Appending /sync to the URL makes the caller wait for a Return Response action, while /test generates sample data without triggering the published flow.',
},
props: {
liveMarkdown: Property.MarkDown({
@@ -199,7 +199,9 @@ export const catchWebhook = createTrigger({
context.payload.rawBody
);
if (!verified) {
- return [];
+ throw new Error(
+ `Webhook authentication failed: request did not pass the configured ${authenticationType} check`
+ );
}
return [context.payload];
},
diff --git a/packages/pieces/core/webhook/test/auth-rejection.test.ts b/packages/pieces/core/webhook/test/auth-rejection.test.ts
new file mode 100644
index 00000000000..f862c814806
--- /dev/null
+++ b/packages/pieces/core/webhook/test/auth-rejection.test.ts
@@ -0,0 +1,102 @@
+/// <reference types="vitest/globals" />
+
+import { createHmac } from 'crypto';
+import { createMockActionContext, TriggerStrategy } from '@activepieces/pieces-framework';
+import { AuthType, catchWebhook } from '../src/lib/triggers/catch-hook';
+
+const rawBody = JSON.stringify({ event: 'push' });
+const basic = (password: string) =>
+ `Basic ${Buffer.from(`user:${password}`).toString('base64')}`;
+
+const authCases: AuthCase[] = [
+ {
+ authType: AuthType.BASIC,
+ authFields: { username: 'user', password: 'pass' },
+ goodHeaders: { authorization: basic('pass') },
+ badHeaders: { authorization: basic('wrong') },
+ },
+ {
+ authType: AuthType.HEADER,
+ authFields: { headerName: 'x-secret', headerValue: 'good' },
+ goodHeaders: { 'x-secret': 'good' },
+ badHeaders: { 'x-secret': 'bad' },
+ },
+ {
+ authType: AuthType.HMAC,
+ authFields: {
+ hmacHeaderName: 'x-signature',
+ hmacSecret: 'secret',
+ hmacAlgorithm: 'sha256',
+ hmacEncoding: 'hex',
+ hmacSignaturePrefix: '',
+ },
+ goodHeaders: {
+ 'x-signature': createHmac('sha256', 'secret').update(rawBody).digest('hex'),
+ },
+ badHeaders: { 'x-signature': 'deadbeef' },
+ },
+];
+
+describe('catchWebhook.run authentication', () => {
+ test('passes the request through when no auth is configured', async () => {
+ const payload = buildPayload({});
+ await expect(
+ runCatchWebhook({ authType: AuthType.NONE, authFields: {}, payload })
+ ).resolves.toEqual([payload]);
+ });
+
+ test.each(authCases)(
+ 'passes the request through when $authType auth matches',
+ async ({ authType, authFields, goodHeaders }) => {
+ const payload = buildPayload(goodHeaders);
+ await expect(runCatchWebhook({ authType, authFields, payload })).resolves.toEqual([payload]);
+ }
+ );
+
+ test.each(authCases)(
+ 'throws instead of dropping the request when $authType auth fails',
+ async ({ authType, authFields, badHeaders }) => {
+ await expect(
+ runCatchWebhook({ authType, authFields, payload: buildPayload(badHeaders) })
+ ).rejects.toThrow('Webhook authentication failed');
+ }
+ );
+});
+
+function buildPayload(headers: Record<string, string>) {
+ return { body: {}, queryParams: {}, rawBody, headers };
+}
+
+function runCatchWebhook({
+ authType,
+ authFields,
+ payload,
+}: {
+ authType: AuthType;
+ authFields: Record<string, string>;
+ payload: ReturnType<typeof buildPayload>;
+}) {
+ if (catchWebhook.type !== TriggerStrategy.WEBHOOK) {
+ throw new Error('catchWebhook must be a webhook trigger');
+ }
+ return catchWebhook.run({
+ ...createMockActionContext<typeof catchWebhook.props>({
+ propsValue: {
+ liveMarkdown: undefined,
+ syncMarkdown: undefined,
+ testMarkdown: undefined,
+ authType,
+ authFields,
+ },
+ }),
+ webhookUrl: 'http://localhost:3000/api/v1/webhooks/flow-id',
+ payload,
+ });
+}
+
+type AuthCase = {
+ authType: AuthType;
+ authFields: Record<string, string>;
+ goodHeaders: Record<string, string>;
+ badHeaders: Record<string, string>;
+};
diff --git a/packages/server/engine/src/lib/operations/flow.operation.ts b/packages/server/engine/src/lib/operations/flow.operation.ts
index 99cf6b385e3..49846d41b17 100644
--- a/packages/server/engine/src/lib/operations/flow.operation.ts
+++ b/packages/server/engine/src/lib/operations/flow.operation.ts
@@ -193,6 +193,9 @@ async function runOrReturnPayload(input: ResolvedBeginExecuteFlowOperation, cons
},
constants,
}) as ExecuteTriggerResponse<TriggerHookType.RUN>
+ if (newPayload.output.length === 0) {
+ throw new ExecutionError('EmptyTriggerPayload', 'The trigger returned no payload, so the flow did not run', ExecutionErrorType.USER)
+ }
return newPayload.output[0] as TriggerPayload
}
diff --git a/packages/server/engine/test/operations/flow-operation-invariants.test.ts b/packages/server/engine/test/operations/flow-operation-invariants.test.ts
index 4043cedf2f6..789c8717916 100644
--- a/packages/server/engine/test/operations/flow-operation-invariants.test.ts
+++ b/packages/server/engine/test/operations/flow-operation-invariants.test.ts
@@ -586,6 +586,25 @@ describe('flow operation invariants', () => {
expect(finalCtx.verdict.failedStep.name).toBe('trigger_1')
expect(finalCtx.steps.trigger_1.errorMessage).toEqual(expect.stringContaining('toLowerCase'))
})
+
+ it('fails the run without executing any step when the trigger run() returns no payload', async () => {
+ mockSendUpdate.mockClear()
+ mockExecuteTrigger.mockResolvedValue({ output: [] })
+ const operation = makeBeginOperation({
+ flowVersion: makeFlowVersionWithTwoApprovals(),
+ triggerPayload: { type: 'inline', value: {} },
+ executeTrigger: true,
+ })
+
+ const response = await flowOperation.execute(operation)
+
+ expect(response.status).toBe(EngineResponseStatus.OK)
+ const finalCtx = mockSendUpdate.mock.calls[mockSendUpdate.mock.calls.length - 1][0].flowExecutorContext
+ expect(finalCtx.verdict.status).toBe(FlowRunStatus.FAILED)
+ expect(finalCtx.verdict.failedStep.name).toBe('trigger_1')
+ expect(finalCtx.steps.trigger_1.errorMessage).toEqual(expect.stringContaining('no payload'))
+ expect(finalCtx.steps.step_1).toBeUndefined()
+ })
})
describe('trigger success output shape', () => {
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-polling.ts b/packages/server/worker/src/lib/execute/jobs/execute-polling.ts
index e0c4943ca6a..80b57d4933e 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-polling.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-polling.ts
@@ -61,7 +61,7 @@ export const executePollingJob: JobHandler<PollingJobData, FireAndForgetJobResul
}
}
- await recordTriggerRun({ apiClient: ctx.apiClient, log: ctx.log, flowVersion, platformId: data.platformId, status: result.status })
+ await recordTriggerRun({ apiClient: ctx.apiClient, log: ctx.log, flowVersion, platformId: data.platformId, status: result.status, error: result.error })
return { kind: JobResultKind.FIRE_AND_FORGET, status: EngineResponseStatus.OK, logs: result.logs }
}
diff --git a/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts b/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts
index 3d50807926d..cc3b057727b 100644
--- a/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts
+++ b/packages/server/worker/src/lib/execute/jobs/execute-webhook.ts
@@ -147,7 +147,7 @@ export const executeWebhookJob: JobHandler<WebhookJobData, FireAndForgetJobResul
}
}
- await recordTriggerRun({ apiClient: ctx.apiClient, log: ctx.log, flowVersion, platformId: data.platformId, status: execResult.status })
+ await recordTriggerRun({ apiClient: ctx.apiClient, log: ctx.log, flowVersion, platformId: data.platformId, status: execResult.status, error: execResult.error })
return { kind: JobResultKind.FIRE_AND_FORGET, status: EngineResponseStatus.OK, logs: execResult.logs }
},
diff --git a/packages/server/worker/src/lib/execute/utils/trigger-run-recorder.ts b/packages/server/worker/src/lib/execute/utils/trigger-run-recorder.ts
index 4f50992cbfe..61ae1de6f7b 100644
--- a/packages/server/worker/src/lib/execute/utils/trigger-run-recorder.ts
+++ b/packages/server/worker/src/lib/execute/utils/trigger-run-recorder.ts
@@ -1,13 +1,17 @@
-import { tryCatch } from '@activepieces/core-utils'
+import { tryCatch, tryParseFriendlyPieceError } from '@activepieces/core-utils'
import { ApLogger } from '@activepieces/server-utils'
import { EngineResponseStatus, FlowTriggerType, FlowVersion, TriggerRunStatus, WorkerToApiContract } from '@activepieces/shared'
-export async function recordTriggerRun({ apiClient, log, flowVersion, platformId, status }: RecordTriggerRunParams): Promise<void> {
+export async function recordTriggerRun({ apiClient, log, flowVersion, platformId, status, error: triggerError }: RecordTriggerRunParams): Promise<void> {
if (flowVersion.trigger.type !== FlowTriggerType.PIECE) {
return
}
const pieceName = flowVersion.trigger.settings.pieceName
- const triggerRunStatus = status === EngineResponseStatus.OK ? TriggerRunStatus.COMPLETED : TriggerRunStatus.FAILED
+ const succeeded = status === EngineResponseStatus.OK
+ if (!succeeded) {
+ log.warn({ error: tryParseFriendlyPieceError(triggerError)?.message, status, piece: { name: pieceName }, flow: { id: flowVersion.flowId }, flowVersion: { id: flowVersion.id } }, 'Trigger run did not complete successfully')
+ }
+ const triggerRunStatus = succeeded ? TriggerRunStatus.COMPLETED : TriggerRunStatus.FAILED
const { error } = await tryCatch(() => apiClient.recordTriggerRun({ platformId, pieceName, status: triggerRunStatus }))
if (error) {
log.warn({ error: String(error), piece: { name: pieceName }, flowVersion: { id: flowVersion.id } }, 'Failed to record trigger run stats')
@@ -20,4 +24,5 @@ type RecordTriggerRunParams = {
flowVersion: FlowVersion
platformId: string
status: EngineResponseStatus
+ error?: string
}
diff --git a/packages/server/worker/test/lib/execute/utils/trigger-run-recorder.test.ts b/packages/server/worker/test/lib/execute/utils/trigger-run-recorder.test.ts
index 18d63a0c211..438502a849a 100644
--- a/packages/server/worker/test/lib/execute/utils/trigger-run-recorder.test.ts
+++ b/packages/server/worker/test/lib/execute/utils/trigger-run-recorder.test.ts
@@ -1,4 +1,5 @@
import { EngineResponseStatus, FlowTriggerType, FlowVersion, FlowVersionState, LATEST_FLOW_SCHEMA_VERSION, TriggerRunStatus, WorkerToApiContract } from '@activepieces/shared'
+import { formatPieceError } from '@activepieces/core-utils'
import { describe, expect, it, vi } from 'vitest'
import { recordTriggerRun } from '../../../../src/lib/execute/utils/trigger-run-recorder'
@@ -72,4 +73,14 @@ describe('recordTriggerRun', () => {
await expect(recordTriggerRun({ apiClient, log, flowVersion: buildPieceFlowVersion('@activepieces/piece-slack'), platformId: 'p1', status: EngineResponseStatus.OK })).resolves.toBeUndefined()
})
+
+ it('logs only the failure message with the flow id, never the raw error', async () => {
+ const apiClient = { recordTriggerRun: vi.fn(async () => undefined) } as unknown as WorkerToApiContract
+ const engineError = JSON.stringify(formatPieceError(new Error('Webhook authentication failed'), { raw: 'headers: { Authorization: Bearer SECRET123 }' }))
+
+ await recordTriggerRun({ apiClient, log, flowVersion: buildPieceFlowVersion('@activepieces/piece-webhook'), platformId: 'p1', status: EngineResponseStatus.USER_FAILURE, error: engineError })
+
+ expect(log.warn).toHaveBeenCalledWith(expect.objectContaining({ error: 'Webhook authentication failed', flow: { id: 'flow1' }, status: EngineResponseStatus.USER_FAILURE }), 'Trigger run did not complete successfully')
+ expect(JSON.stringify(vi.mocked(log.warn).mock.calls)).not.toContain('SECRET123')
+ })
})