Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix(flows): polling triggers keep their place when a republish is approved (#16717)

Hazem Adel36b5775901d3
diff --git a/brain/knowledge/flows-execution/triggers.md b/brain/knowledge/flows-execution/triggers.md
index bb4db1aaa49..1dad4fddc83 100644
--- a/brain/knowledge/flows-execution/triggers.md
+++ b/brain/knowledge/flows-execution/triggers.md
@@ -21,7 +21,8 @@ Triggers define how and when a flow starts. The module handles registration, eve
### Gotchas
- **Deduplication** (polling): extracts `__DEDUPE_KEY_PROPERTY`, Redis INCR with 30s TTL — first passes, duplicates filtered; the dedupe key is stripped from returned payloads.
-- **Republish preserves the polling checkpoint** (`isRepublish`): republishing a running flow does `onDisable(old) → onEnable(new)`, which used to reset `lastPoll`/`lastItem` to now and silently drop events created in between. `flowService.update` sets `isRepublish=true` only for a `LOCK_AND_PUBLISH` of an already-`ENABLED` flow whose trigger is unchanged — same piece, same trigger name, **and deep-equal `settings.input`** (`flowPublishUtils.isSameTrigger`); the flag is threaded through the ON_ENABLE job → `ExecuteTriggerOperation` → trigger context (`context.isRepublish`), and `pollingHelper.onEnable` then keeps the existing checkpoint. A fresh enable, a manual off→on toggle, a trigger swap, and any change to the trigger's props all still reset to now. The props check is not cosmetic: a checkpoint kept across a props change points at a resource that is no longer being polled, and `pollingHelper.poll` treats a `LAST_ITEM` id it cannot find in the fetched page (`findIndex → -1`) the same as "no checkpoint", emitting **every** item. Custom polling triggers that don't use `pollingHelper` can opt in by reading `context.isRepublish`.
+- **Republish preserves the polling checkpoint** (`isRepublish`): republishing a running flow does `onDisable(old) → onEnable(new)`, which used to reset `lastPoll`/`lastItem` to now and silently drop events created in between. `flowService.applyStatusChangeForPublishedFlow` (the one entry point for both a direct `LOCK_AND_PUBLISH` and an EE approval) sets `isRepublish=true` only for a publish of an already-`ENABLED` flow whose trigger is unchanged — same piece, same trigger name, **and deep-equal `settings.input`** (`flowPublishUtils.isSameTrigger`); the flag is threaded through the ON_ENABLE job → `ExecuteTriggerOperation` → trigger context (`context.isRepublish`), and `pollingHelper.onEnable` then keeps the existing checkpoint. A fresh enable, a manual off→on toggle, a trigger swap, and any change to the trigger's props all still reset to now. The props check is not cosmetic: a checkpoint kept across a props change points at a resource that is no longer being polled, and `pollingHelper.poll` treats a `LAST_ITEM` id it cannot find in the fetched page (`findIndex → -1`) the same as "no checkpoint", emitting **every** item. Custom polling triggers that don't use `pollingHelper` can opt in by reading `context.isRepublish`. A new publish path must go through `applyStatusChangeForPublishedFlow` with the pre-publish flow snapshot; before ENG-938 the approval path called `applyStatusChange` itself and every approved republish reset the checkpoint.
+- **A trigger only keeps its checkpoint if it forwards `isRepublish`.** `pollingHelper.onEnable(polling, context)` gets it; an object literal like `{ auth, store, propsValue }` (RSS and roughly half of polling triggers) drops it and resets to now on every republish.
- **A missing timestamp permanently kills a TIMEBASED polling trigger** — `pollingHelper.poll` advances the checkpoint with `items.reduce((acc, i) => Math.max(acc, i.epochMilliSeconds), lastPoll)`, and `Math.max(n, NaN)` is `NaN`, so a single item whose date field was never requested (`dayjs(undefined).valueOf()` → `NaN`) writes `NaN` into `lastPoll`; every later poll then filters on `> NaN` → false and the trigger silently never fires again, with no error anywhere. Two guards, both needed: request the date in the API's `fields`/select mask, and drop items with an unusable date before mapping to `epochMilliSeconds`. The filter must check the raw value, not just `.isValid()` — `dayjs(undefined)` is *now* and reports valid.
- **The timestamp you poll on may be client-supplied, and a future one is fatal.** Google Drive sets `modifiedTime` from the *local file mtime* on upload, not the upload moment, so it can be older than `createdTime` — or years ahead if the uploader's clock is fast. A TIMEBASED watermark is `max(epochMilliSeconds)` over the emitted items, so one future-dated row pushes `lastPoll` into the future and the trigger emits nothing until wall-clock catches up. Hold back items timestamped after `Date.now()` — they fire once the clock passes them. Do **not** clamp the watermark instead: a clamped watermark re-emits that same row on every poll forever.
- The **simulate flag** lets a production source and a test source coexist independently.
diff --git a/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts b/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts
index bc25f9ce263..ee404f67683 100644
--- a/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts
+++ b/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts
@@ -118,8 +118,8 @@ export const flowApprovalRequestService = (log: FastifyBaseLogger) => ({
}
await flowExecutionCache(log).invalidate(flow.id)
await flowService(log).applyStatusChangeForPublishedFlow({
- id: flow.id,
- projectId: approval.projectId,
+ previousFlow: flow,
+ publishedVersion: lockedVersion,
newStatus: approval.requestedStatus,
})
rejectedPromiseHandler(telemetry(log).trackProject({
diff --git a/packages/server/api/src/app/flows/flow/flow.service.ts b/packages/server/api/src/app/flows/flow/flow.service.ts
index 7f3592178cb..bae6ca7bf1c 100644
--- a/packages/server/api/src/app/flows/flow/flow.service.ts
+++ b/packages/server/api/src/app/flows/flow/flow.service.ts
@@ -338,17 +338,6 @@ export const flowService = (log: FastifyBaseLogger) => ({
? previousFlow ?? await this.getOnePopulatedOrThrow({ id, projectId })
: undefined
- let previouslyPublishedVersion: FlowVersion | undefined
- if (operation.type === FlowOperationType.LOCK_AND_PUBLISH || operation.type === FlowOperationType.CHANGE_STATUS) {
- const flow = await this.getOneOrThrow({
- id,
- projectId,
- })
- if (operation.type === FlowOperationType.LOCK_AND_PUBLISH && flow.status === FlowStatus.ENABLED && !isNil(flow.publishedVersionId)) {
- previouslyPublishedVersion = await flowVersionService(log).getFlowVersionOrThrow({ flowId: id, versionId: flow.publishedVersionId })
- }
- }
-
switch (operation.type) {
case FlowOperationType.LOCK_AND_PUBLISH: {
const flow = await this.getOneOrThrow({ id, projectId })
@@ -385,11 +374,7 @@ export const flowService = (log: FastifyBaseLogger) => ({
platformId,
flowVersionToPublish,
})
- const isRepublish = !isNil(previouslyPublishedVersion) && flowPublishUtils.isSameTrigger({
- published: previouslyPublishedVersion.trigger,
- toPublish: publishedFlow.version.trigger,
- })
- await applyStatusChange({ id, projectId, newStatus: requestedStatus, isRepublish }, log)
+ await this.applyStatusChangeForPublishedFlow({ previousFlow: flow, publishedVersion: publishedFlow.version, newStatus: requestedStatus })
break
}
@@ -565,8 +550,16 @@ export const flowService = (log: FastifyBaseLogger) => ({
})
},
- async applyStatusChangeForPublishedFlow({ id, projectId, newStatus }: { id: FlowId, projectId: ProjectId, newStatus: FlowStatus }): Promise<void> {
- await applyStatusChange({ id, projectId, newStatus }, log)
+ async applyStatusChangeForPublishedFlow({ previousFlow, publishedVersion, newStatus }: ApplyStatusChangeForPublishedFlowParams): Promise<void> {
+ const previousVersionId = previousFlow.status === FlowStatus.ENABLED ? previousFlow.publishedVersionId : null
+ const previousVersion = isNil(previousVersionId)
+ ? null
+ : await flowVersionService(log).getFlowVersionOrThrow({ flowId: previousFlow.id, versionId: previousVersionId })
+ const isRepublish = !isNil(previousVersion) && flowPublishUtils.isSameTrigger({
+ published: previousVersion.trigger,
+ toPublish: publishedVersion.trigger,
+ })
+ await applyStatusChange({ id: previousFlow.id, projectId: previousFlow.projectId, newStatus, isRepublish }, log)
},
async delete({ id, projectId, previousFlow, userId, ip, emitEvents = true }: DeleteParams): Promise<void> {
@@ -1005,6 +998,12 @@ type SetPublishedVersionParams = {
entityManager: EntityManager
}
+type ApplyStatusChangeForPublishedFlowParams = {
+ previousFlow: Flow
+ publishedVersion: FlowVersion
+ newStatus: FlowStatus
+}
+
type DeleteParams = EventEmissionParams & {
id: FlowId
projectId: ProjectId
diff --git a/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts b/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts
new file mode 100644
index 00000000000..33d835df27d
--- /dev/null
+++ b/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts
@@ -0,0 +1,214 @@
+import {
+ apId,
+ DefaultProjectRole,
+ FlowApprovalRequest,
+ FlowApprovalRequestState,
+ FlowOperationType,
+ FlowStatus,
+ FlowTriggerType,
+ FlowVersion,
+ FlowVersionState,
+ PackageType,
+ PieceType,
+ PropertyExecutionType,
+ TriggerStrategy,
+ TriggerTestStrategy,
+} from '@activepieces/shared'
+import dayjs from 'dayjs'
+import { FastifyInstance } from 'fastify'
+import { StatusCodes } from 'http-status-codes'
+import { afterAll, beforeAll, describe, expect, it, vi } from 'vitest'
+import { db } from '../../../helpers/db'
+import { createMockFlow, createMockFlowVersion, createMockPieceMetadata } from '../../../helpers/mocks'
+import { createMemberContext, createTestContext } from '../../../helpers/test-context'
+import { setupTestEnvironment, teardownTestEnvironment } from '../../../helpers/test-setup'
+
+const triggerCalls = vi.hoisted(() => {
+ const calls: TriggerCall[] = []
+ return calls
+})
+
+vi.mock('../../../../src/app/trigger/trigger-source/trigger-source-service', async (importOriginal) => {
+ const actual = await importOriginal<typeof import('../../../../src/app/trigger/trigger-source/trigger-source-service')>()
+ return {
+ ...actual,
+ triggerSourceService: (log: Parameters<typeof actual.triggerSourceService>[0]) => ({
+ ...actual.triggerSourceService(log),
+ enable: async (params: { flowVersion: { flowId: string }, simulate: boolean, isRepublish?: boolean }) => {
+ if (!params.simulate) {
+ triggerCalls.push({ call: 'enable', flowId: params.flowVersion.flowId, isRepublish: params.isRepublish })
+ }
+ return null
+ },
+ disable: async (params: { flowId: string, simulate: boolean }) => {
+ if (!params.simulate) {
+ triggerCalls.push({ call: 'disable', flowId: params.flowId })
+ }
+ },
+ }),
+ }
+})
+
+let app: FastifyInstance | null = null
+
+beforeAll(async () => {
+ app = await setupTestEnvironment()
+})
+
+afterAll(async () => {
+ await teardownTestEnvironment()
+})
+
+describe('republishing a flow through an approval request', () => {
+ it.each([
+ {
+ name: 'keeps the checkpoint of an enabled flow whose trigger is unchanged',
+ flowStatus: FlowStatus.ENABLED,
+ requestedStatus: FlowStatus.ENABLED,
+ republishedInput: { projectId: 'AP' },
+ expected: [{ call: 'disable' }, { call: 'enable', isRepublish: true }],
+ },
+ {
+ name: 'resets the checkpoint when the trigger input changed',
+ flowStatus: FlowStatus.ENABLED,
+ requestedStatus: FlowStatus.ENABLED,
+ republishedInput: { projectId: 'OTHER' },
+ expected: [{ call: 'disable' }, { call: 'enable', isRepublish: false }],
+ },
+ {
+ name: 'resets the checkpoint when the flow was disabled',
+ flowStatus: FlowStatus.DISABLED,
+ requestedStatus: FlowStatus.ENABLED,
+ republishedInput: { projectId: 'AP' },
+ expected: [{ call: 'enable', isRepublish: false }],
+ },
+ {
+ name: 'only disables the old trigger when the approved publish keeps the flow disabled',
+ flowStatus: FlowStatus.ENABLED,
+ requestedStatus: FlowStatus.DISABLED,
+ republishedInput: { projectId: 'AP' },
+ expected: [{ call: 'disable' }],
+ },
+ ])('$name', async ({ flowStatus, requestedStatus, republishedInput, expected }) => {
+ const { statusCode, calls } = await approveRepublish({ flowStatus, requestedStatus, republishedInput })
+
+ expect(statusCode).toBe(StatusCodes.OK)
+ expect(calls).toEqual(expected)
+ })
+})
+
+describe('republishing a flow directly', () => {
+ it('keeps the checkpoint of an enabled flow whose trigger is unchanged', async () => {
+ const ctx = await createTestContext(app!)
+ await db.save('piece_metadata', createMockPieceMetadata({
+ name: '@activepieces/piece-jira-cloud',
+ version: '0.4.1',
+ triggers: {
+ new_issue: {
+ name: 'new_issue',
+ displayName: 'New Issue',
+ description: 'New Issue',
+ requireAuth: false,
+ props: {},
+ type: TriggerStrategy.POLLING,
+ sampleData: {},
+ testStrategy: TriggerTestStrategy.TEST_FUNCTION,
+ },
+ },
+ pieceType: PieceType.OFFICIAL,
+ packageType: PackageType.REGISTRY,
+ }))
+ const flow = createMockFlow({ projectId: ctx.project.id, status: FlowStatus.ENABLED })
+ await db.save('flow', flow)
+ const published = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: { projectId: 'AP' } }) })
+ await db.save('flow_version', published)
+ await db.update('flow', flow.id, { publishedVersionId: published.id })
+ const draft = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.DRAFT, valid: true, created: dayjs().add(1, 'minute').toISOString(), trigger: pollingTrigger({ input: { projectId: 'AP' } }) })
+ await db.save('flow_version', draft)
+
+ const response = await ctx.post(`/v1/flows/${flow.id}`, { type: FlowOperationType.LOCK_AND_PUBLISH, request: {} })
+
+ expect(response?.statusCode).toBe(StatusCodes.OK)
+ expect(recordedCalls({ flowId: flow.id }))
+ .toEqual([{ call: 'disable' }, { call: 'enable', isRepublish: true }])
+ })
+})
+
+async function approveRepublish({ flowStatus, requestedStatus, republishedInput }: ApproveRepublishParams): Promise<ApproveRepublishResult> {
+ const ctx = await createTestContext(app!, { plan: { environmentsEnabled: true } })
+ await db.update('project', ctx.project.id, { sensitive: true })
+ const memberCtx = await createMemberContext(app!, ctx, { projectRole: DefaultProjectRole.EDITOR })
+
+ const flow = createMockFlow({ projectId: ctx.project.id, status: flowStatus })
+ await db.save('flow', flow)
+ const published = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: { projectId: 'AP' } }) })
+ await db.save('flow_version', published)
+ await db.update('flow', flow.id, { publishedVersionId: published.id })
+ const republished = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: republishedInput }) })
+ await db.save('flow_version', republished)
+ const now = new Date().toISOString()
+ const approval: FlowApprovalRequest = {
+ id: apId(),
+ created: now,
+ updated: now,
+ flowId: flow.id,
+ flowVersionId: republished.id,
+ projectId: ctx.project.id,
+ platformId: ctx.platform.id,
+ submitterId: memberCtx.user.id,
+ submittedAt: now,
+ approverId: null,
+ decidedAt: null,
+ state: FlowApprovalRequestState.PENDING,
+ requestedStatus,
+ rejectionReason: null,
+ }
+ await db.save('flow_approval_request', approval)
+
+ const response = await ctx.post(`/v1/flow-approval-requests/${approval.id}/approve`, {})
+
+ return {
+ statusCode: response?.statusCode,
+ calls: recordedCalls({ flowId: flow.id }),
+ }
+}
+
+function recordedCalls({ flowId }: { flowId: string }): Omit<TriggerCall, 'flowId'>[] {
+ return triggerCalls.filter((call) => call.flowId === flowId).map(({ call, isRepublish }) => ({ call, isRepublish }))
+}
+
+function pollingTrigger({ input }: { input: Record<string, unknown> }): FlowVersion['trigger'] {
+ return {
+ type: FlowTriggerType.PIECE,
+ settings: {
+ pieceName: '@activepieces/piece-jira-cloud',
+ pieceVersion: '0.4.1',
+ triggerName: 'new_issue',
+ input,
+ propertySettings: {
+ projectId: { type: PropertyExecutionType.MANUAL },
+ },
+ },
+ valid: true,
+ name: 'trigger',
+ displayName: 'New Issue',
+ lastUpdatedDate: new Date().toISOString(),
+ }
+}
+
+type TriggerCall = {
+ call: 'enable' | 'disable'
+ flowId: string
+ isRepublish?: boolean
+}
+
+type ApproveRepublishParams = {
+ flowStatus: FlowStatus
+ requestedStatus: FlowStatus
+ republishedInput: Record<string, unknown>
+}
+
+type ApproveRepublishResult = {
+ statusCode: number | undefined
+ calls: Omit<TriggerCall, 'flowId'>[]
+}