AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(flows): polling triggers keep their place when a republish is approved (#16717)
Hazem Adel36b5775901d3
diff --git a/brain/knowledge/flows-execution/triggers.md b/brain/knowledge/flows-execution/triggers.mdindex bb4db1aaa49..1dad4fddc83 100644--- a/brain/knowledge/flows-execution/triggers.md+++ b/brain/knowledge/flows-execution/triggers.md@@ -21,7 +21,8 @@ Triggers define how and when a flow starts. The module handles registration, eve### Gotchas- **Deduplication** (polling): extracts `__DEDUPE_KEY_PROPERTY`, Redis INCR with 30s TTL — first passes, duplicates filtered; the dedupe key is stripped from returned payloads.-- **Republish preserves the polling checkpoint** (`isRepublish`): republishing a running flow does `onDisable(old) → onEnable(new)`, which used to reset `lastPoll`/`lastItem` to now and silently drop events created in between. `flowService.update` sets `isRepublish=true` only for a `LOCK_AND_PUBLISH` of an already-`ENABLED` flow whose trigger is unchanged — same piece, same trigger name, **and deep-equal `settings.input`** (`flowPublishUtils.isSameTrigger`); the flag is threaded through the ON_ENABLE job → `ExecuteTriggerOperation` → trigger context (`context.isRepublish`), and `pollingHelper.onEnable` then keeps the existing checkpoint. A fresh enable, a manual off→on toggle, a trigger swap, and any change to the trigger's props all still reset to now. The props check is not cosmetic: a checkpoint kept across a props change points at a resource that is no longer being polled, and `pollingHelper.poll` treats a `LAST_ITEM` id it cannot find in the fetched page (`findIndex → -1`) the same as "no checkpoint", emitting **every** item. Custom polling triggers that don't use `pollingHelper` can opt in by reading `context.isRepublish`.+- **Republish preserves the polling checkpoint** (`isRepublish`): republishing a running flow does `onDisable(old) → onEnable(new)`, which used to reset `lastPoll`/`lastItem` to now and silently drop events created in between. `flowService.applyStatusChangeForPublishedFlow` (the one entry point for both a direct `LOCK_AND_PUBLISH` and an EE approval) sets `isRepublish=true` only for a publish of an already-`ENABLED` flow whose trigger is unchanged — same piece, same trigger name, **and deep-equal `settings.input`** (`flowPublishUtils.isSameTrigger`); the flag is threaded through the ON_ENABLE job → `ExecuteTriggerOperation` → trigger context (`context.isRepublish`), and `pollingHelper.onEnable` then keeps the existing checkpoint. A fresh enable, a manual off→on toggle, a trigger swap, and any change to the trigger's props all still reset to now. The props check is not cosmetic: a checkpoint kept across a props change points at a resource that is no longer being polled, and `pollingHelper.poll` treats a `LAST_ITEM` id it cannot find in the fetched page (`findIndex → -1`) the same as "no checkpoint", emitting **every** item. Custom polling triggers that don't use `pollingHelper` can opt in by reading `context.isRepublish`. A new publish path must go through `applyStatusChangeForPublishedFlow` with the pre-publish flow snapshot; before ENG-938 the approval path called `applyStatusChange` itself and every approved republish reset the checkpoint.+- **A trigger only keeps its checkpoint if it forwards `isRepublish`.** `pollingHelper.onEnable(polling, context)` gets it; an object literal like `{ auth, store, propsValue }` (RSS and roughly half of polling triggers) drops it and resets to now on every republish.- **A missing timestamp permanently kills a TIMEBASED polling trigger** — `pollingHelper.poll` advances the checkpoint with `items.reduce((acc, i) => Math.max(acc, i.epochMilliSeconds), lastPoll)`, and `Math.max(n, NaN)` is `NaN`, so a single item whose date field was never requested (`dayjs(undefined).valueOf()` → `NaN`) writes `NaN` into `lastPoll`; every later poll then filters on `> NaN` → false and the trigger silently never fires again, with no error anywhere. Two guards, both needed: request the date in the API's `fields`/select mask, and drop items with an unusable date before mapping to `epochMilliSeconds`. The filter must check the raw value, not just `.isValid()` — `dayjs(undefined)` is *now* and reports valid.- **The timestamp you poll on may be client-supplied, and a future one is fatal.** Google Drive sets `modifiedTime` from the *local file mtime* on upload, not the upload moment, so it can be older than `createdTime` — or years ahead if the uploader's clock is fast. A TIMEBASED watermark is `max(epochMilliSeconds)` over the emitted items, so one future-dated row pushes `lastPoll` into the future and the trigger emits nothing until wall-clock catches up. Hold back items timestamped after `Date.now()` — they fire once the clock passes them. Do **not** clamp the watermark instead: a clamped watermark re-emits that same row on every poll forever.- The **simulate flag** lets a production source and a test source coexist independently.diff --git a/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts b/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.tsindex bc25f9ce263..ee404f67683 100644--- a/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts+++ b/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts@@ -118,8 +118,8 @@ export const flowApprovalRequestService = (log: FastifyBaseLogger) => ({}await flowExecutionCache(log).invalidate(flow.id)await flowService(log).applyStatusChangeForPublishedFlow({- id: flow.id,- projectId: approval.projectId,+ previousFlow: flow,+ publishedVersion: lockedVersion,newStatus: approval.requestedStatus,})rejectedPromiseHandler(telemetry(log).trackProject({diff --git a/packages/server/api/src/app/flows/flow/flow.service.ts b/packages/server/api/src/app/flows/flow/flow.service.tsindex 7f3592178cb..bae6ca7bf1c 100644--- a/packages/server/api/src/app/flows/flow/flow.service.ts+++ b/packages/server/api/src/app/flows/flow/flow.service.ts@@ -338,17 +338,6 @@ export const flowService = (log: FastifyBaseLogger) => ({? previousFlow ?? await this.getOnePopulatedOrThrow({ id, projectId }): undefined- let previouslyPublishedVersion: FlowVersion | undefined- if (operation.type === FlowOperationType.LOCK_AND_PUBLISH || operation.type === FlowOperationType.CHANGE_STATUS) {- const flow = await this.getOneOrThrow({- id,- projectId,- })- if (operation.type === FlowOperationType.LOCK_AND_PUBLISH && flow.status === FlowStatus.ENABLED && !isNil(flow.publishedVersionId)) {- previouslyPublishedVersion = await flowVersionService(log).getFlowVersionOrThrow({ flowId: id, versionId: flow.publishedVersionId })- }- }-switch (operation.type) {case FlowOperationType.LOCK_AND_PUBLISH: {const flow = await this.getOneOrThrow({ id, projectId })@@ -385,11 +374,7 @@ export const flowService = (log: FastifyBaseLogger) => ({platformId,flowVersionToPublish,})- const isRepublish = !isNil(previouslyPublishedVersion) && flowPublishUtils.isSameTrigger({- published: previouslyPublishedVersion.trigger,- toPublish: publishedFlow.version.trigger,- })- await applyStatusChange({ id, projectId, newStatus: requestedStatus, isRepublish }, log)+ await this.applyStatusChangeForPublishedFlow({ previousFlow: flow, publishedVersion: publishedFlow.version, newStatus: requestedStatus })break}@@ -565,8 +550,16 @@ export const flowService = (log: FastifyBaseLogger) => ({})},- async applyStatusChangeForPublishedFlow({ id, projectId, newStatus }: { id: FlowId, projectId: ProjectId, newStatus: FlowStatus }): Promise<void> {- await applyStatusChange({ id, projectId, newStatus }, log)+ async applyStatusChangeForPublishedFlow({ previousFlow, publishedVersion, newStatus }: ApplyStatusChangeForPublishedFlowParams): Promise<void> {+ const previousVersionId = previousFlow.status === FlowStatus.ENABLED ? previousFlow.publishedVersionId : null+ const previousVersion = isNil(previousVersionId)+ ? null+ : await flowVersionService(log).getFlowVersionOrThrow({ flowId: previousFlow.id, versionId: previousVersionId })+ const isRepublish = !isNil(previousVersion) && flowPublishUtils.isSameTrigger({+ published: previousVersion.trigger,+ toPublish: publishedVersion.trigger,+ })+ await applyStatusChange({ id: previousFlow.id, projectId: previousFlow.projectId, newStatus, isRepublish }, log)},async delete({ id, projectId, previousFlow, userId, ip, emitEvents = true }: DeleteParams): Promise<void> {@@ -1005,6 +998,12 @@ type SetPublishedVersionParams = {entityManager: EntityManager}+type ApplyStatusChangeForPublishedFlowParams = {+ previousFlow: Flow+ publishedVersion: FlowVersion+ newStatus: FlowStatus+}+type DeleteParams = EventEmissionParams & {id: FlowIdprojectId: ProjectIddiff --git a/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts b/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.tsnew file mode 100644index 00000000000..33d835df27d--- /dev/null+++ b/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts@@ -0,0 +1,214 @@+import {+ apId,+ DefaultProjectRole,+ FlowApprovalRequest,+ FlowApprovalRequestState,+ FlowOperationType,+ FlowStatus,+ FlowTriggerType,+ FlowVersion,+ FlowVersionState,+ PackageType,+ PieceType,+ PropertyExecutionType,+ TriggerStrategy,+ TriggerTestStrategy,+} from '@activepieces/shared'+import dayjs from 'dayjs'+import { FastifyInstance } from 'fastify'+import { StatusCodes } from 'http-status-codes'+import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'+import { db } from '../../../helpers/db'+import { createMockFlow, createMockFlowVersion, createMockPieceMetadata } from '../../../helpers/mocks'+import { createMemberContext, createTestContext } from '../../../helpers/test-context'+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'++const triggerCalls = vi.hoisted(() => {+ const calls: TriggerCall[] = []+ return calls+})++vi.mock('../../../../src/app/trigger/trigger-source/trigger-source-service', async (importOriginal) => {+ const actual = await importOriginal<typeof import('../../../../src/app/trigger/trigger-source/trigger-source-service')>()+ return {+ ...actual,+ triggerSourceService: (log: Parameters<typeof actual.triggerSourceService>[0]) => ({+ ...actual.triggerSourceService(log),+ enable: async (params: { flowVersion: { flowId: string }, simulate: boolean, isRepublish?: boolean }) => {+ if (!params.simulate) {+ triggerCalls.push({ call: 'enable', flowId: params.flowVersion.flowId, isRepublish: params.isRepublish })+ }+ return null+ },+ disable: async (params: { flowId: string, simulate: boolean }) => {+ if (!params.simulate) {+ triggerCalls.push({ call: 'disable', flowId: params.flowId })+ }+ },+ }),+ }+})++let app: FastifyInstance | null = null++beforeAll(async () => {+ app = await setupTestEnvironment()+})++afterAll(async () => {+ await teardownTestEnvironment()+})++describe('republishing a flow through an approval request', () => {+ it.each([+ {+ name: 'keeps the checkpoint of an enabled flow whose trigger is unchanged',+ flowStatus: FlowStatus.ENABLED,+ requestedStatus: FlowStatus.ENABLED,+ republishedInput: { projectId: 'AP' },+ expected: [{ call: 'disable' }, { call: 'enable', isRepublish: true }],+ },+ {+ name: 'resets the checkpoint when the trigger input changed',+ flowStatus: FlowStatus.ENABLED,+ requestedStatus: FlowStatus.ENABLED,+ republishedInput: { projectId: 'OTHER' },+ expected: [{ call: 'disable' }, { call: 'enable', isRepublish: false }],+ },+ {+ name: 'resets the checkpoint when the flow was disabled',+ flowStatus: FlowStatus.DISABLED,+ requestedStatus: FlowStatus.ENABLED,+ republishedInput: { projectId: 'AP' },+ expected: [{ call: 'enable', isRepublish: false }],+ },+ {+ name: 'only disables the old trigger when the approved publish keeps the flow disabled',+ flowStatus: FlowStatus.ENABLED,+ requestedStatus: FlowStatus.DISABLED,+ republishedInput: { projectId: 'AP' },+ expected: [{ call: 'disable' }],+ },+ ])('$name', async ({ flowStatus, requestedStatus, republishedInput, expected }) => {+ const { statusCode, calls } = await approveRepublish({ flowStatus, requestedStatus, republishedInput })++ expect(statusCode).toBe(StatusCodes.OK)+ expect(calls).toEqual(expected)+ })+})++describe('republishing a flow directly', () => {+ it('keeps the checkpoint of an enabled flow whose trigger is unchanged', async () => {+ const ctx = await createTestContext(app!)+ await db.save('piece_metadata', createMockPieceMetadata({+ name: '@activepieces/piece-jira-cloud',+ version: '0.4.1',+ triggers: {+ new_issue: {+ name: 'new_issue',+ displayName: 'New Issue',+ description: 'New Issue',+ requireAuth: false,+ props: {},+ type: TriggerStrategy.POLLING,+ sampleData: {},+ testStrategy: TriggerTestStrategy.TEST_FUNCTION,+ },+ },+ pieceType: PieceType.OFFICIAL,+ packageType: PackageType.REGISTRY,+ }))+ const flow = createMockFlow({ projectId: ctx.project.id, status: FlowStatus.ENABLED })+ await db.save('flow', flow)+ const published = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: { projectId: 'AP' } }) })+ await db.save('flow_version', published)+ await db.update('flow', flow.id, { publishedVersionId: published.id })+ const draft = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.DRAFT, valid: true, created: dayjs().add(1, 'minute').toISOString(), trigger: pollingTrigger({ input: { projectId: 'AP' } }) })+ await db.save('flow_version', draft)++ const response = await ctx.post(`/v1/flows/${flow.id}`, { type: FlowOperationType.LOCK_AND_PUBLISH, request: {} })++ expect(response?.statusCode).toBe(StatusCodes.OK)+ expect(recordedCalls({ flowId: flow.id }))+ .toEqual([{ call: 'disable' }, { call: 'enable', isRepublish: true }])+ })+})++async function approveRepublish({ flowStatus, requestedStatus, republishedInput }: ApproveRepublishParams): Promise<ApproveRepublishResult> {+ const ctx = await createTestContext(app!, { plan: { environmentsEnabled: true } })+ await db.update('project', ctx.project.id, { sensitive: true })+ const memberCtx = await createMemberContext(app!, ctx, { projectRole: DefaultProjectRole.EDITOR })++ const flow = createMockFlow({ projectId: ctx.project.id, status: flowStatus })+ await db.save('flow', flow)+ const published = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: { projectId: 'AP' } }) })+ await db.save('flow_version', published)+ await db.update('flow', flow.id, { publishedVersionId: published.id })+ const republished = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: republishedInput }) })+ await db.save('flow_version', republished)+ const now = new Date().toISOString()+ const approval: FlowApprovalRequest = {+ id: apId(),+ created: now,+ updated: now,+ flowId: flow.id,+ flowVersionId: republished.id,+ projectId: ctx.project.id,+ platformId: ctx.platform.id,+ submitterId: memberCtx.user.id,+ submittedAt: now,+ approverId: null,+ decidedAt: null,+ state: FlowApprovalRequestState.PENDING,+ requestedStatus,+ rejectionReason: null,+ }+ await db.save('flow_approval_request', approval)++ const response = await ctx.post(`/v1/flow-approval-requests/${approval.id}/approve`, {})++ return {+ statusCode: response?.statusCode,+ calls: recordedCalls({ flowId: flow.id }),+ }+}++function recordedCalls({ flowId }: { flowId: string }): Omit<TriggerCall, 'flowId'>[] {+ return triggerCalls.filter((call) => call.flowId === flowId).map(({ call, isRepublish }) => ({ call, isRepublish }))+}++function pollingTrigger({ input }: { input: Record<string, unknown> }): FlowVersion['trigger'] {+ return {+ type: FlowTriggerType.PIECE,+ settings: {+ pieceName: '@activepieces/piece-jira-cloud',+ pieceVersion: '0.4.1',+ triggerName: 'new_issue',+ input,+ propertySettings: {+ projectId: { type: PropertyExecutionType.MANUAL },+ },+ },+ valid: true,+ name: 'trigger',+ displayName: 'New Issue',+ lastUpdatedDate: new Date().toISOString(),+ }+}++type TriggerCall = {+ call: 'enable' | 'disable'+ flowId: string+ isRepublish?: boolean+}++type ApproveRepublishParams = {+ flowStatus: FlowStatus+ requestedStatus: FlowStatus+ republishedInput: Record<string, unknown>+}++type ApproveRepublishResult = {+ statusCode: number | undefined+ calls: Omit<TriggerCall, 'flowId'>[]+}