AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(flows): polling triggers no longer skip new items after a republish (#16742)
Hazem Adel635e1661a2ff
diff --git a/brain/knowledge/execution-runtime/workers.md b/brain/knowledge/execution-runtime/workers.mdindex 82bfc5ca36d..a5d63d3730e 100644--- a/brain/knowledge/execution-runtime/workers.md+++ b/brain/knowledge/execution-runtime/workers.md@@ -21,6 +21,7 @@ The deep `Resolver`/`Runtime` concurrency and bundle-caching model lives on the- `concurrency === 1` primes the sandbox to the full container RAM (cgroup-aware) via `primeFullContainerMemory()`.### Gotchas+- **A local engine edit doesn't reach sandboxes until the worker restarts, unless `AP_ENVIRONMENT` is dev.** `engineInstaller` copies `dist/packages/engine/main.js` into `cache/v<N>/common/main.js` once per worker process (`installedPaths`) and only re-copies on every install in `DEVELOPMENT`. With a prod-like local env, sandboxes keep running whatever engine was built when the worker first ran a job, and killing sandboxes doesn't help. Restart the worker (touching `packages/server/worker/src/bootstrap.ts` makes `tsx watch` do it), then check the cached `main.js` for your change before testing.- **Piece bundles download over the worker's own `AP_FRONTEND_URL` (`internalApiUrl`), never the app's `PUBLIC_URL` — keep it that way.** Through 0.92.x the sandbox built `GET /v1/engine/pieces/bundle` from the `PUBLIC_URL` the app pushes over the socket, so a standalone worker ignored its own `AP_FRONTEND_URL=http://app` for this one hop. A `localhost` public URL (the docs' default for compose and `docker run -p 8080:80`) failed every uncached install with `fetch failed` (GIT-1838). A public URL that reached a *different* instance sharing the JWT secret returned 404, so custom-piece upload failed with `PIECE_BUNDLE_NOT_AVAILABLE` while the `PACKAGE_ARCHIVE` row sat intact in Postgres. Tell-tale: the bundle GET never shows up in the app's own request log. Fixed by GIT-1718.- **`AP_CONTAINER_TYPE=WORKER` moves every engine→app call onto `AP_FRONTEND_URL`.** In `WORKER_AND_APP` the engine reaches the app at `http://127.0.0.1:<port>/api/`; in `WORKER` it uses `AP_FRONTEND_URL + '/api/'` (`getApiUrl` in `worker/src/lib/config/configs.ts`), and the worker's Socket.IO connection follows it. If that URL is the public hostname, file uploads, run-log uploads, piece bundle downloads and the worker socket all cross the ingress/WAF/CDN, so WAF rules and websocket idle timeouts start failing runs that worked the day before. Point separated workers at the in-cluster app service instead. Found on Alan's fleet 2026-09-29, the day after being advised to switch to `WORKER`.- **The piece workspace pins `linker = "isolated"` in its own `bunfig.toml` — do not delete it, the engine's piece resolver depends on the layout bun chooses.** `pieceInstaller` generates a bun workspace at `<cache>/<version>/common` whose members are `pieces/<pieceName>-<pieceVersion>`, and the engine (`piece-loader.ts` → `resolveInstalledPieceEntry`) resolves a piece **only** at `pieces/<alias>/node_modules/<pieceName>`, which just the isolated linker produces. The workspace used to carry no bunfig and inherited whatever ancestor bunfig bun discovered above the cache mount (`/usr/src/app/bunfig.toml`); when bun instead uses the **hoisted** linker it writes the package as a real directory at the workspace root `node_modules/@activepieces/<name>` and leaves the member with no `node_modules` at all, so every run of that piece fails `PieceNotFoundError: Piece not found for package: <name>-<version>` while `bun install` still exits 0. It was silent and self-perpetuating — `markPiecesAsUsed` writes `ready` and `pieceCheckIfAlreadyInstalled` only looks for *a* `node_modules`, so the piece is reinstalled forever and never resolves. Sep 2026 production: 125/125 folders missing `node_modules` were exactly the 125 packages sitting hoisted at the workspace root; 7,400+ `EXECUTE_FLOW` jobs, 15+ platforms, ~60 official pieces, ~300/hr. Two things had to line up — bun 1.4.0 (Aug 31) changed the layout newly-installed members got, and removing the piece-upgrade Redis gate (#15236, deployed 2026-09-03 08:09 UTC) mass-upgraded every platform's flows to newer piece versions so almost every install was suddenly a *first-time* install. Diagnose on a worker with `for d in <cache>/v*/common/pieces/@activepieces/*/; do [ -d "$d/node_modules" ] || basename $d; done`. Fixed by pinning the linker plus a `LATEST_CACHE_VERSION` bump (v14 → v15) so the fleet rebuilds one clean workspace under the pinned layout rather than carrying a mixed one.diff --git a/brain/knowledge/flows-execution/triggers.md b/brain/knowledge/flows-execution/triggers.mdindex 1dad4fddc83..ccc03ae0bfb 100644--- a/brain/knowledge/flows-execution/triggers.md+++ b/brain/knowledge/flows-execution/triggers.md@@ -21,8 +21,8 @@ Triggers define how and when a flow starts. The module handles registration, eve### Gotchas- **Deduplication** (polling): extracts `__DEDUPE_KEY_PROPERTY`, Redis INCR with 30s TTL — first passes, duplicates filtered; the dedupe key is stripped from returned payloads.-- **Republish preserves the polling checkpoint** (`isRepublish`): republishing a running flow does `onDisable(old) → onEnable(new)`, which used to reset `lastPoll`/`lastItem` to now and silently drop events created in between. `flowService.applyStatusChangeForPublishedFlow` (the one entry point for both a direct `LOCK_AND_PUBLISH` and an EE approval) sets `isRepublish=true` only for a publish of an already-`ENABLED` flow whose trigger is unchanged — same piece, same trigger name, **and deep-equal `settings.input`** (`flowPublishUtils.isSameTrigger`); the flag is threaded through the ON_ENABLE job → `ExecuteTriggerOperation` → trigger context (`context.isRepublish`), and `pollingHelper.onEnable` then keeps the existing checkpoint. A fresh enable, a manual off→on toggle, a trigger swap, and any change to the trigger's props all still reset to now. The props check is not cosmetic: a checkpoint kept across a props change points at a resource that is no longer being polled, and `pollingHelper.poll` treats a `LAST_ITEM` id it cannot find in the fetched page (`findIndex → -1`) the same as "no checkpoint", emitting **every** item. Custom polling triggers that don't use `pollingHelper` can opt in by reading `context.isRepublish`. A new publish path must go through `applyStatusChangeForPublishedFlow` with the pre-publish flow snapshot; before ENG-938 the approval path called `applyStatusChange` itself and every approved republish reset the checkpoint.-- **A trigger only keeps its checkpoint if it forwards `isRepublish`.** `pollingHelper.onEnable(polling, context)` gets it; an object literal like `{ auth, store, propsValue }` (RSS and roughly half of polling triggers) drops it and resets to now on every republish.+- **Republish preserves the polling checkpoint** (`isRepublish`): republishing a running flow does `onDisable(old) → onEnable(new)`, which used to reset `lastPoll`/`lastItem` to now and silently drop events created in between. `flowService.isRepublish` decides it once, before anything is disabled: true only when an already-`ENABLED` flow is published to stay `ENABLED` with an unchanged trigger — same piece, **same piece version**, same trigger name, **and deep-equal `settings.input`** (`flowPublishUtils.isSameTrigger`). A piece upgrade counts as a change and starts fresh, because the new version may read its state differently. Both publish paths (direct `LOCK_AND_PUBLISH` and EE approval) pass that one value to the old version's ON_DISABLE job and to the new version's ON_ENABLE job → `ExecuteTriggerOperation` → trigger context (`context.isRepublish`). A fresh enable, a manual off→on toggle, a trigger swap, and any change to the trigger's props all still reset to now. The props check is not cosmetic: a checkpoint kept across a props change points at a resource that is no longer being polled, and `pollingHelper.poll` treats a `LAST_ITEM` id it cannot find in the fetched page (`findIndex → -1`) the same as "no checkpoint", emitting **every** item. A new publish path must call `flowService.isRepublish` with the pre-publish flow snapshot and pass the result to both `triggerSourceService.disable` and `applyStatusChangeForPublishedFlow`; before ENG-938 the approval path skipped it and every approved republish reset the checkpoint.+- **The engine keeps the checkpoint, not the piece.** On a republish of a `POLLING` trigger, `republishStore.forLifecycleHook` (`packages/server/engine/src/lib/piece-context`) makes the store read-only for both the old version's `onDisable` and the new version's `onEnable`: reads work, writes and deletes do nothing, so the state ends exactly as it was. Per-operation rules were tried and each missed a piece: Twilio clears its checkpoint in `onDisable` with `put(key, null)`, Shopify's `onEnable` deletes `lastItem` and writes new scan keys, ~38 triggers delete their checkpoint in `onDisable`. Pieces don't need to forward the flag. Webhook triggers are excluded because they must remove and re-create their webhook. The cost: a cache that `onEnable` would refresh (Zoho's field list) stays until it expires on its own (6h for Zoho). A plain off toggle or a flow delete sends no flag, so `onDisable` still cleans up there.- **A missing timestamp permanently kills a TIMEBASED polling trigger** — `pollingHelper.poll` advances the checkpoint with `items.reduce((acc, i) => Math.max(acc, i.epochMilliSeconds), lastPoll)`, and `Math.max(n, NaN)` is `NaN`, so a single item whose date field was never requested (`dayjs(undefined).valueOf()` → `NaN`) writes `NaN` into `lastPoll`; every later poll then filters on `> NaN` → false and the trigger silently never fires again, with no error anywhere. Two guards, both needed: request the date in the API's `fields`/select mask, and drop items with an unusable date before mapping to `epochMilliSeconds`. The filter must check the raw value, not just `.isValid()` — `dayjs(undefined)` is *now* and reports valid.- **The timestamp you poll on may be client-supplied, and a future one is fatal.** Google Drive sets `modifiedTime` from the *local file mtime* on upload, not the upload moment, so it can be older than `createdTime` — or years ahead if the uploader's clock is fast. A TIMEBASED watermark is `max(epochMilliSeconds)` over the emitted items, so one future-dated row pushes `lastPoll` into the future and the trigger emits nothing until wall-clock catches up. Hold back items timestamped after `Date.now()` — they fire once the clock passes them. Do **not** clamp the watermark instead: a clamped watermark re-emits that same row on every poll forever.- The **simulate flag** lets a production source and a test source coexist independently.diff --git a/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts b/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.tsindex c27ae4c9c87..918eb689b93 100644--- a/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts+++ b/packages/server/api/src/app/ee/flows/flow-approval/flow-approval-request.service.ts@@ -89,6 +89,7 @@ export const flowApprovalRequestService = (log: FastifyBaseLogger) => ({flowId: flow.id,versionId: approval.flowVersionId,})+ const isRepublish = await flowService(log).isRepublish({ previousFlow: flow, versionToPublish: lockedVersion, newStatus: approval.requestedStatus })const decidedAt = new Date().toISOString()const approverId = approverPrincipal.type === PrincipalType.SERVICE ? null : approverPrincipal.id@@ -115,13 +116,15 @@ export const flowApprovalRequestService = (log: FastifyBaseLogger) => ({projectId: flow.projectId,simulate: false,ignoreError: true,+ isRepublish,})}await flowExecutionCache(log).invalidate(flow.id)await flowService(log).applyStatusChangeForPublishedFlow({- previousFlow: flow,- publishedVersion: lockedVersion,+ id: flow.id,+ projectId: flow.projectId,newStatus: approval.requestedStatus,+ isRepublish,})rejectedPromiseHandler(telemetry(log).trackProject({projectId: approval.projectId,diff --git a/packages/server/api/src/app/flows/flow/flow-publish-utils.ts b/packages/server/api/src/app/flows/flow/flow-publish-utils.tsindex f0285c7b0c5..462b93e6c07 100644--- a/packages/server/api/src/app/flows/flow/flow-publish-utils.ts+++ b/packages/server/api/src/app/flows/flow/flow-publish-utils.ts@@ -5,6 +5,7 @@ function isSameTrigger({ published, toPublish }: IsSameTriggerParams): boolean {return published.type === FlowTriggerType.PIECE&& toPublish.type === FlowTriggerType.PIECE&& published.settings.pieceName === toPublish.settings.pieceName+ && published.settings.pieceVersion === toPublish.settings.pieceVersion&& published.settings.triggerName === toPublish.settings.triggerName&& deepEqual(published.settings.input, toPublish.settings.input)}diff --git a/packages/server/api/src/app/flows/flow/flow.service.ts b/packages/server/api/src/app/flows/flow/flow.service.tsindex c21a40a6833..9a03739ff33 100644--- a/packages/server/api/src/app/flows/flow/flow.service.ts+++ b/packages/server/api/src/app/flows/flow/flow.service.ts@@ -376,14 +376,16 @@ export const flowService = (log: FastifyBaseLogger) => ({})break}- const publishedFlow = await this.updatedPublishedVersionId({+ const isRepublish = await this.isRepublish({ previousFlow: flow, versionToPublish: flowVersionToPublish, newStatus: requestedStatus })+ await this.updatedPublishedVersionId({id,userId,projectId,platformId,flowVersionToPublish,+ isRepublish,})- await this.applyStatusChangeForPublishedFlow({ previousFlow: flow, publishedVersion: publishedFlow.version, newStatus: requestedStatus })+ await this.applyStatusChangeForPublishedFlow({ id, projectId, newStatus: requestedStatus, isRepublish })break}@@ -500,6 +502,7 @@ export const flowService = (log: FastifyBaseLogger) => ({projectId,platformId,flowVersionToPublish,+ isRepublish,}: UpdatePublishedVersionIdParams): Promise<PopulatedFlow> {const flowToUpdate = await this.getOneOrThrow({ id, projectId })@@ -509,6 +512,7 @@ export const flowService = (log: FastifyBaseLogger) => ({projectId: flowToUpdate.projectId,simulate: false,ignoreError: true,+ isRepublish,})}@@ -563,16 +567,21 @@ export const flowService = (log: FastifyBaseLogger) => ({})},- async applyStatusChangeForPublishedFlow({ previousFlow, publishedVersion, newStatus }: ApplyStatusChangeForPublishedFlowParams): Promise<void> {- const previousVersionId = previousFlow.status === FlowStatus.ENABLED ? previousFlow.publishedVersionId : null- const previousVersion = isNil(previousVersionId)- ? null- : await flowVersionService(log).getFlowVersionOrThrow({ flowId: previousFlow.id, versionId: previousVersionId })- const isRepublish = !isNil(previousVersion) && flowPublishUtils.isSameTrigger({+ async isRepublish({ previousFlow, versionToPublish, newStatus }: IsRepublishParams): Promise<boolean> {+ const previousVersionId = previousFlow.publishedVersionId+ const keepsRunning = previousFlow.status === FlowStatus.ENABLED && newStatus === FlowStatus.ENABLED+ if (!keepsRunning || isNil(previousVersionId)) {+ return false+ }+ const previousVersion = await flowVersionService(log).getFlowVersionOrThrow({ flowId: previousFlow.id, versionId: previousVersionId })+ return flowPublishUtils.isSameTrigger({published: previousVersion.trigger,- toPublish: publishedVersion.trigger,+ toPublish: versionToPublish.trigger,})- await applyStatusChange({ id: previousFlow.id, projectId: previousFlow.projectId, newStatus, isRepublish }, log)+ },++ async applyStatusChangeForPublishedFlow({ id, projectId, newStatus, isRepublish }: ApplyStatusChangeForPublishedFlowParams): Promise<void> {+ await applyStatusChange({ id, projectId, newStatus, isRepublish }, log)},async delete({ id, projectId, previousFlow, userId, ip, emitEvents = true }: DeleteParams): Promise<void> {@@ -1008,6 +1017,7 @@ type UpdatePublishedVersionIdParams = {platformId: PlatformIdprojectId: ProjectIdflowVersionToPublish: FlowVersion+ isRepublish: boolean}type SetPublishedVersionParams = {@@ -1016,10 +1026,17 @@ type SetPublishedVersionParams = {entityManager: EntityManager}-type ApplyStatusChangeForPublishedFlowParams = {+type IsRepublishParams = {previousFlow: Flow- publishedVersion: FlowVersion+ versionToPublish: FlowVersion+ newStatus: FlowStatus+}++type ApplyStatusChangeForPublishedFlowParams = {+ id: FlowId+ projectId: ProjectIdnewStatus: FlowStatus+ isRepublish: boolean}type DeleteParams = EventEmissionParams & {diff --git a/packages/server/api/src/app/trigger/trigger-source/flow-trigger-side-effect.ts b/packages/server/api/src/app/trigger/trigger-source/flow-trigger-side-effect.tsindex d763b59aced..26cf3f3b6d6 100644--- a/packages/server/api/src/app/trigger/trigger-source/flow-trigger-side-effect.ts+++ b/packages/server/api/src/app/trigger/trigger-source/flow-trigger-side-effect.ts@@ -75,7 +75,7 @@ export const flowTriggerSideEffect = (log: FastifyBaseLogger) => {if (environment === ApEnvironment.TESTING) {return}- const { flowId, flowVersionId, projectId, simulate, pieceTrigger } = params+ const { flowId, flowVersionId, projectId, simulate, pieceTrigger, isRepublish } = paramsconst platformId = await projectService(log).getPlatformId(projectId)const { error, data: engineHelperResponse } = await tryCatch(() => userInteractionWatcher.submitAndWaitForResponse<EngineResponse<ExecuteTriggerResponse<TriggerHookType.ON_DISABLE>>>({@@ -86,6 +86,7 @@ export const flowTriggerSideEffect = (log: FastifyBaseLogger) => {test: simulate,projectId,platformId,+ isRepublish,}, log),)if (!isNil(error)) {diff --git a/packages/server/api/src/app/trigger/trigger-source/trigger-source-service.ts b/packages/server/api/src/app/trigger/trigger-source/trigger-source-service.tsindex 9dd6221c084..0b494b404bf 100644--- a/packages/server/api/src/app/trigger/trigger-source/trigger-source-service.ts+++ b/packages/server/api/src/app/trigger/trigger-source/trigger-source-service.ts@@ -195,6 +195,7 @@ export const triggerSourceService = (log: FastifyBaseLogger) => {pieceTrigger,simulate,ignoreError: params.ignoreError,+ isRepublish: params.isRepublish,})log.info('[triggerSourceService#disable] Disabled flow trigger side effect')}@@ -250,6 +251,7 @@ type DisableTriggerParams = {simulate: booleanignoreError: booleantemplateId?: string+ isRepublish?: boolean}type EnableTriggerParams = {diff --git a/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts b/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.tsindex 33d835df27d..602f2f734bd 100644--- a/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts+++ b/packages/server/api/test/integration/ee/flow-approval/approved-republish-keeps-checkpoint.test.ts@@ -40,9 +40,9 @@ vi.mock('../../../../src/app/trigger/trigger-source/trigger-source-service', asy}return null},- disable: async (params: { flowId: string, simulate: boolean }) => {+ disable: async (params: { flowId: string, simulate: boolean, isRepublish?: boolean }) => {if (!params.simulate) {- triggerCalls.push({ call: 'disable', flowId: params.flowId })+ triggerCalls.push({ call: 'disable', flowId: params.flowId, isRepublish: params.isRepublish })}},}),@@ -66,14 +66,14 @@ describe('republishing a flow through an approval request', () => {flowStatus: FlowStatus.ENABLED,requestedStatus: FlowStatus.ENABLED,republishedInput: { projectId: 'AP' },- expected: [{ call: 'disable' }, { call: 'enable', isRepublish: true }],+ expected: [{ call: 'disable', isRepublish: true }, { call: 'enable', isRepublish: true }],},{name: 'resets the checkpoint when the trigger input changed',flowStatus: FlowStatus.ENABLED,requestedStatus: FlowStatus.ENABLED,republishedInput: { projectId: 'OTHER' },- expected: [{ call: 'disable' }, { call: 'enable', isRepublish: false }],+ expected: [{ call: 'disable', isRepublish: false }, { call: 'enable', isRepublish: false }],},{name: 'resets the checkpoint when the flow was disabled',@@ -87,7 +87,7 @@ describe('republishing a flow through an approval request', () => {flowStatus: FlowStatus.ENABLED,requestedStatus: FlowStatus.DISABLED,republishedInput: { projectId: 'AP' },- expected: [{ call: 'disable' }],+ expected: [{ call: 'disable', isRepublish: false }],},])('$name', async ({ flowStatus, requestedStatus, republishedInput, expected }) => {const { statusCode, calls } = await approveRepublish({ flowStatus, requestedStatus, republishedInput })@@ -98,39 +98,48 @@ describe('republishing a flow through an approval request', () => {})describe('republishing a flow directly', () => {- it('keeps the checkpoint of an enabled flow whose trigger is unchanged', async () => {+ it.each([+ {+ name: 'keeps the checkpoint of an enabled flow whose trigger is unchanged',+ draftInput: { projectId: 'AP' },+ requestedStatus: FlowStatus.ENABLED,+ expected: [{ call: 'disable', isRepublish: true }, { call: 'enable', isRepublish: true }],+ },+ {+ name: 'resets the checkpoint when the trigger input changed',+ draftInput: { projectId: 'OTHER' },+ requestedStatus: FlowStatus.ENABLED,+ expected: [{ call: 'disable', isRepublish: false }, { call: 'enable', isRepublish: false }],+ },+ {+ name: 'clears the old trigger when the publish keeps the flow disabled',+ draftInput: { projectId: 'AP' },+ requestedStatus: FlowStatus.DISABLED,+ expected: [{ call: 'disable', isRepublish: false }],+ },+ {+ name: 'resets the checkpoint when the piece version changed',+ draftInput: { projectId: 'AP' },+ draftPieceVersion: '0.4.2',+ requestedStatus: FlowStatus.ENABLED,+ expected: [{ call: 'disable', isRepublish: false }, { call: 'enable', isRepublish: false }],+ },+ ])('$name', async ({ draftInput, draftPieceVersion, requestedStatus, expected }) => {const ctx = await createTestContext(app!)- await db.save('piece_metadata', createMockPieceMetadata({- name: '@activepieces/piece-jira-cloud',- version: '0.4.1',- triggers: {- new_issue: {- name: 'new_issue',- displayName: 'New Issue',- description: 'New Issue',- requireAuth: false,- props: {},- type: TriggerStrategy.POLLING,- sampleData: {},- testStrategy: TriggerTestStrategy.TEST_FUNCTION,- },- },- pieceType: PieceType.OFFICIAL,- packageType: PackageType.REGISTRY,- }))+ await db.save('piece_metadata', jiraPollingPiece({ version: '0.4.1' }))+ await db.save('piece_metadata', jiraPollingPiece({ version: '0.4.2' }))const flow = createMockFlow({ projectId: ctx.project.id, status: FlowStatus.ENABLED })await db.save('flow', flow)const published = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.LOCKED, valid: true, trigger: pollingTrigger({ input: { projectId: 'AP' } }) })await db.save('flow_version', published)await db.update('flow', flow.id, { publishedVersionId: published.id })- const draft = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.DRAFT, valid: true, created: dayjs().add(1, 'minute').toISOString(), trigger: pollingTrigger({ input: { projectId: 'AP' } }) })+ const draft = createMockFlowVersion({ flowId: flow.id, state: FlowVersionState.DRAFT, valid: true, created: dayjs().add(1, 'minute').toISOString(), trigger: pollingTrigger({ input: draftInput, pieceVersion: draftPieceVersion }) })await db.save('flow_version', draft)- const response = await ctx.post(`/v1/flows/${flow.id}`, { type: FlowOperationType.LOCK_AND_PUBLISH, request: {} })+ const response = await ctx.post(`/v1/flows/${flow.id}`, { type: FlowOperationType.LOCK_AND_PUBLISH, request: { status: requestedStatus } })expect(response?.statusCode).toBe(StatusCodes.OK)- expect(recordedCalls({ flowId: flow.id }))- .toEqual([{ call: 'disable' }, { call: 'enable', isRepublish: true }])+ expect(recordedCalls({ flowId: flow.id })).toEqual(expected)})})@@ -177,12 +186,33 @@ function recordedCalls({ flowId }: { flowId: string }): Omit<TriggerCall, 'flowIreturn triggerCalls.filter((call) => call.flowId === flowId).map(({ call, isRepublish }) => ({ call, isRepublish }))}-function pollingTrigger({ input }: { input: Record<string, unknown> }): FlowVersion['trigger'] {+function jiraPollingPiece({ version }: { version: string }): ReturnType<typeof createMockPieceMetadata> {+ return createMockPieceMetadata({+ name: '@activepieces/piece-jira-cloud',+ version,+ triggers: {+ new_issue: {+ name: 'new_issue',+ displayName: 'New Issue',+ description: 'New Issue',+ requireAuth: false,+ props: {},+ type: TriggerStrategy.POLLING,+ sampleData: {},+ testStrategy: TriggerTestStrategy.TEST_FUNCTION,+ },+ },+ pieceType: PieceType.OFFICIAL,+ packageType: PackageType.REGISTRY,+ })+}++function pollingTrigger({ input, pieceVersion = '0.4.1' }: { input: Record<string, unknown>, pieceVersion?: string }): FlowVersion['trigger'] {return {type: FlowTriggerType.PIECE,settings: {pieceName: '@activepieces/piece-jira-cloud',- pieceVersion: '0.4.1',+ pieceVersion,triggerName: 'new_issue',input,propertySettings: {diff --git a/packages/server/engine/src/lib/helper/trigger-helper.ts b/packages/server/engine/src/lib/helper/trigger-helper.tsindex d87f84a208c..807141b6808 100644--- a/packages/server/engine/src/lib/helper/trigger-helper.ts+++ b/packages/server/engine/src/lib/helper/trigger-helper.ts@@ -6,6 +6,7 @@ import { EngineConstants, ResolvedExecuteTriggerOperation } from '../handler/conimport { FlowExecutorContext } from '../handler/context/flow-execution-context'import { createFileUploader } from '../piece-context/file-uploader'import { createFlowsContext } from '../piece-context/flows'+import { republishStore } from '../piece-context/republish-store'import { createContextStore } from '../piece-context/store'import { utils } from '../utils'import { propsProcessor } from '../variables/props-processor'@@ -164,11 +165,17 @@ export const triggerHelper = {}switch (params.hookType) {case TriggerHookType.ON_DISABLE: {- await pieceTrigger.onDisable(context)+ await pieceTrigger.onDisable({+ ...context,+ store: republishStore.forLifecycleHook({ store: context.store, isRepublish: params.isRepublish, triggerType: pieceTrigger.type }),+ })return {}}case TriggerHookType.ON_ENABLE: {- await pieceTrigger.onEnable(context)+ await pieceTrigger.onEnable({+ ...context,+ store: republishStore.forLifecycleHook({ store: context.store, isRepublish: params.isRepublish, triggerType: pieceTrigger.type }),+ })return {listeners: appListeners,scheduleOptions: pieceTrigger.type === TriggerStrategy.POLLING ? scheduleOptions : undefined,diff --git a/packages/server/engine/src/lib/piece-context/republish-store.ts b/packages/server/engine/src/lib/piece-context/republish-store.tsnew file mode 100644index 00000000000..f9be5800600--- /dev/null+++ b/packages/server/engine/src/lib/piece-context/republish-store.ts@@ -0,0 +1,25 @@+import { Store, TriggerStrategy } from '@activepieces/pieces-framework'++export const republishStore = {+ forLifecycleHook({ store, isRepublish, triggerType }: RepublishStoreParams): Store {+ const keepsPollingState = isRepublish === true && triggerType === TriggerStrategy.POLLING+ return keepsPollingState ? readOnly(store) : store+ },+}++function readOnly(store: Store): Store {+ return {+ get: (key, scope) => store.get(key, scope),+ async put<T>(_key: string, value: T): Promise<T> {+ return value+ },+ async delete(): Promise<void> {+ },+ }+}++type RepublishStoreParams = {+ store: Store+ isRepublish: boolean | undefined+ triggerType: TriggerStrategy+}diff --git a/packages/server/engine/test/helper/trigger-helper-republish.test.ts b/packages/server/engine/test/helper/trigger-helper-republish.test.tsnew file mode 100644index 00000000000..1a10de1dd32--- /dev/null+++ b/packages/server/engine/test/helper/trigger-helper-republish.test.ts@@ -0,0 +1,200 @@+import { Store, TriggerStrategy } from '@activepieces/pieces-framework'+import { ExecuteTriggerResponse, FlowVersionState, TriggerHookType } from '@activepieces/shared'+import { beforeEach, describe, expect, it, vi } from 'vitest'++const { mockGetPieceAndTriggerOrThrow, storeEntries } = vi.hoisted(() => ({+ mockGetPieceAndTriggerOrThrow: vi.fn(),+ storeEntries: new Map<string, unknown>(),+}))+vi.mock('../../src/lib/helper/piece-loader', () => ({+ pieceLoader: {+ getPieceAndTriggerOrThrow: mockGetPieceAndTriggerOrThrow,+ },+}))+vi.mock('../../src/lib/piece-context/store', () => ({+ createContextStore: (): Store => ({+ async put<T>(key: string, value: T): Promise<T> {+ storeEntries.set(key, value)+ return value+ },+ async get<T>(key: string): Promise<T | null> {+ return storeEntries.has(key) ? storeEntries.get(key) as T : null+ },+ async delete(key: string): Promise<void> {+ storeEntries.delete(key)+ },+ }),+}))++import { EngineConstants, ResolvedExecuteTriggerOperation } from '../../src/lib/handler/context/engine-constants'+import { triggerHelper } from '../../src/lib/helper/trigger-helper'++describe('a republish leaves a polling trigger\'s stored state untouched', () => {+ beforeEach(() => {+ mockGetPieceAndTriggerOrThrow.mockReset()+ storeEntries.clear()+ })++ it('keeps the checkpoint onEnable overwrites', async () => {+ storeEntries.set('lastPoll', OLD_CHECKPOINT)+ mockTrigger({ type: TriggerStrategy.POLLING, onEnable: async ({ store }) => {+ await store.put('lastPoll', NEW_CHECKPOINT)+ } })++ await executeHook({ hookType: TriggerHookType.ON_ENABLE, isRepublish: true })++ expect(storeEntries.get('lastPoll')).toBe(OLD_CHECKPOINT)+ })++ it('keeps the checkpoint onDisable deletes', async () => {+ storeEntries.set('_seenVideoIds', ['video-1'])+ mockTrigger({ type: TriggerStrategy.POLLING, onDisable: async ({ store }) => {+ await store.delete('_seenVideoIds')+ } })++ await executeHook({ hookType: TriggerHookType.ON_DISABLE, isRepublish: true })++ expect(storeEntries.get('_seenVideoIds')).toEqual(['video-1'])+ })++ it('keeps the checkpoint onDisable clears by writing null', async () => {+ storeEntries.set('_new_incoming_sms_trigger', { sid: 'message-1' })+ mockTrigger({+ type: TriggerStrategy.POLLING,+ onDisable: async ({ store }) => {+ await store.put('_new_incoming_sms_trigger', null)+ },+ onEnable: async ({ store }) => {+ const last = await store.get('_new_incoming_sms_trigger')+ if (last === null) {+ await store.put('_new_incoming_sms_trigger', { sid: 'message-now' })+ }+ },+ })++ await executeHook({ hookType: TriggerHookType.ON_DISABLE, isRepublish: true })+ await executeHook({ hookType: TriggerHookType.ON_ENABLE, isRepublish: true })++ expect(storeEntries.get('_new_incoming_sms_trigger')).toEqual({ sid: 'message-1' })+ })++ it('does not let onEnable replace the checkpoint with new scan state', async () => {+ storeEntries.set('lastItem', 'checkout-1')+ mockTrigger({ type: TriggerStrategy.POLLING, onEnable: async ({ store }) => {+ await store.put('newestCheckoutScanCursor', 'checkout-99')+ await store.put('newestCheckoutScanEnabledAt', NEW_CHECKPOINT)+ await store.delete('lastItem')+ } })++ await executeHook({ hookType: TriggerHookType.ON_ENABLE, isRepublish: true })++ expect(Object.fromEntries(storeEntries)).toEqual({ lastItem: 'checkout-1' })+ })++ it('still registers the polling schedule', async () => {+ mockTrigger({ type: TriggerStrategy.POLLING, onEnable: async ({ store, setSchedule }) => {+ await store.put('lastPoll', NEW_CHECKPOINT)+ setSchedule({ cronExpression: '*/5 * * * *' })+ } })++ const response = await executeHook({ hookType: TriggerHookType.ON_ENABLE, isRepublish: true })++ expect(response).toMatchObject({ scheduleOptions: { cronExpression: '*/5 * * * *' } })+ })+})++describe('outside a republish the store behaves normally', () => {+ beforeEach(() => {+ mockGetPieceAndTriggerOrThrow.mockReset()+ storeEntries.clear()+ })++ it('lets onEnable reset the checkpoint on a fresh enable', async () => {+ storeEntries.set('lastPoll', OLD_CHECKPOINT)+ mockTrigger({ type: TriggerStrategy.POLLING, onEnable: async ({ store }) => {+ await store.put('lastPoll', NEW_CHECKPOINT)+ } })++ await executeHook({ hookType: TriggerHookType.ON_ENABLE, isRepublish: false })++ expect(storeEntries.get('lastPoll')).toBe(NEW_CHECKPOINT)+ })++ it('lets onDisable clean up when the flow is turned off', async () => {+ storeEntries.set('_seenVideoIds', ['video-1'])+ mockTrigger({ type: TriggerStrategy.POLLING, onDisable: async ({ store }) => {+ await store.delete('_seenVideoIds')+ } })++ await executeHook({ hookType: TriggerHookType.ON_DISABLE, isRepublish: false })++ expect(storeEntries.has('_seenVideoIds')).toBe(false)+ })++ it('lets a webhook trigger replace its webhook on a republish', async () => {+ storeEntries.set('webhookId', 'old-webhook')+ mockTrigger({+ type: TriggerStrategy.WEBHOOK,+ onDisable: async ({ store }) => {+ await store.delete('webhookId')+ },+ onEnable: async ({ store }) => {+ await store.put('webhookId', 'new-webhook')+ },+ })++ await executeHook({ hookType: TriggerHookType.ON_DISABLE, isRepublish: true })+ await executeHook({ hookType: TriggerHookType.ON_ENABLE, isRepublish: true })++ expect(storeEntries.get('webhookId')).toBe('new-webhook')+ })+})++function mockTrigger({ type, onEnable, onDisable }: { type: TriggerStrategy, onEnable?: (context: HookContext) => Promise<void>, onDisable?: (context: HookContext) => Promise<void> }): void {+ mockGetPieceAndTriggerOrThrow.mockResolvedValue({+ piece: { auth: undefined },+ pieceTrigger: { type, props: {}, requireAuth: false, onEnable, onDisable },+ })+}++async function executeHook({ hookType, isRepublish }: { hookType: TriggerHookType.ON_ENABLE | TriggerHookType.ON_DISABLE, isRepublish: boolean }): Promise<ExecuteTriggerResponse<TriggerHookType>> {+ const params = {+ hookType,+ test: false,+ isRepublish,+ webhookUrl: 'http://127.0.0.1:3000/api/v1/webhooks/flow-1',+ flowVersion: {+ id: 'flow-version-1',+ flowId: 'flow-1',+ state: FlowVersionState.LOCKED,+ trigger: {+ name: 'trigger',+ type: 'PIECE_TRIGGER',+ valid: true,+ displayName: 'Polling trigger',+ settings: {+ pieceName: '@activepieces/piece-under-test',+ pieceVersion: '1.0.0',+ triggerName: 'new_item',+ input: {},+ propertySettings: {},+ },+ },+ },+ projectId: 'project-1',+ platformId: 'platform-1',+ engineToken: 'engine-token',+ internalApiUrl: 'http://127.0.0.1:3000/api/',+ publicApiUrl: 'http://127.0.0.1:4200/api/',+ timeoutInSeconds: 30,+ } as unknown as ResolvedExecuteTriggerOperation<TriggerHookType>+ return triggerHelper.executeTrigger({ params, constants: EngineConstants.fromExecuteTriggerInput(params) })+}++const OLD_CHECKPOINT = 1000+const NEW_CHECKPOINT = 2000++type HookContext = {+ store: Store+ setSchedule: (request: { cronExpression: string }) => void+}