Skip to content

AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents

fix(flows): testing a step no longer waits on the whole flow's sample data (#15713)

Hazem Adel8385e0eb8fec
diff --git a/brain/knowledge/pieces-engine/pieces.md b/brain/knowledge/pieces-engine/pieces.md
index d6ba047925f..711184ffecb 100644
--- a/brain/knowledge/pieces-engine/pieces.md
+++ b/brain/knowledge/pieces-engine/pieces.md
@@ -29,6 +29,7 @@ The metadata catalog of automation integrations ("pieces") — each a named inte
- **DynamicProperties clears its value before it knows the new schema, so the merge source must be a snapshot.** `DynamicPropertiesImplementation` re-fetches the child schema on every refresher change, clearing the form value synchronously and re-populating it in the mutation callback. The merge source for `getDefaultValueForProperties` has to be a `lastKnownValue` ref captured *before* the clear — reading `form.getValues()` in the callback sees the cleared `null` and defaults every child (GIT-1514). The snapshot must be spread-cloned: RHF `getValues(name)` hands back the live object and the clear's `setValue(...child, null)` mutates it in place. Guard the ref with `isNil` so it survives rapid successive changes, where later effect runs already observe `null`.
- `DynamicPropertiesContext` tracks loading by property name only, so two in-flight requests for the same property let the first completion clear the flag for both — briefly re-enabling Test Step while the value is still cleared.
- **`POST /v1/pieces/options` cost scales with the whole flow's sample data, not with the property being resolved.** The handler builds an `EXECUTE_PROPERTY` job whose payload carries `sampleData` — and `getSampleDataForFlow` reads *every* step's sample-data `file` row. That payload then crosses Postgres → API → Redis → worker → engine socket, so a big flow pays megabytes to resolve a trivial resolver like the HTTP piece's `authFields`: measured 7.3 MB on a 25-step flow, and a panel open fires three of these in parallel (GIT-1822). The engine only uses `sampleData` to seed step outputs so `{{step.x}}` inside the property's `input` resolves, so callers must use `getSampleDataReferencedBy` and pass the request `input` — which narrows the read to the steps that input names. When narrowing, a referenced `LOOP_ON_ITEMS` step also needs whatever its `items` expression references: `stateFromFlowVersion` derives a loop's `item` by resolving its settings against the context built so far, never from `sampleData[loopName]`. Matching step names as substrings of the serialized input over-approximates, which is the safe direction — a step reference always spells the name literally, in both the property-path and script branches of `props-resolver`.
+- **There is no static signal for whether a DynamicProperties resolver is pure, so caching its result is never safe by inspection.** Both tempting heuristics have counterexamples in this repo. "Trivial resolver" fails because purity cannot be read off the declaration; "declares no connection" fails because `packages/pieces/core/tables` `update-record` declares `auth: PieceAuth.None()` yet its `values` resolver calls `tablesCommon.createFieldProperties` → `getTableFields`, a live request to `v1/fields` authenticated with the **engine token** rather than a connection — so a column added to an Activepieces Table would not show up. Third-party resolvers that fetch live schema (Zendesk `userFieldsDynamicProp`, Supabase `table_columns`) are the same class reached through a connection. The failure mode is a silently outdated form, which the builder gives the user no way to refresh. Anything in this area (GIT-813) needs either a purity signal the piece declares, or stale-while-revalidate that always refetches and corrects within a round trip — and note that SWR re-applies the schema a second time, which is exactly the path the `lastKnownValue` snapshot above exists to protect.
- **The frontend `POST /v1/pieces/options` client only rejects for DYNAMIC.** `piecesApi.options` (`packages/web/src/features/pieces/api/`) catches DROPDOWN failures, toasts, and *resolves* with a disabled-dropdown fallback — so for dropdowns every error path wired onto that mutation is dead: `usePieceOptions`' `onError` handlers, its `retry: 1`, and the `if (error) throw error` into `DynamicPropertiesErrorBoundary`. DYNAMIC must rethrow: a swallowed failure arrives as a *successful* empty schema, which resets the property's children to defaults and gets persisted by step-settings autosave.
- **An action that can be called as an MCP tool cannot create a waitpoint.** An MCP piece tool runs the
real action through `actionRunService`, which sets `actionRunMode` on the engine, and
diff --git a/packages/server/api/src/app/flows/flow-run/flow-run-service.ts b/packages/server/api/src/app/flows/flow-run/flow-run-service.ts
index d9d089a1899..ce8560d1069 100644
--- a/packages/server/api/src/app/flows/flow-run/flow-run-service.ts
+++ b/packages/server/api/src/app/flows/flow-run/flow-run-service.ts
@@ -1,6 +1,6 @@
import { ActivepiecesError, apId, Cursor, ErrorCode, FlowId, FlowRunId, FlowVersionId, isNil, PlatformId, ProjectId, SeekPage } from '@activepieces/core-utils'
import { apDayjs, apDayjsDuration, wideEvent } from '@activepieces/server-utils'
-import { ExecuteFlowJobData, ExecutionType, ExecutioOutputFile, FileCompression, FileType, FlowRetryStrategy, FlowRun, FlowRunCountByStatus, FlowRunStatus, FlowRunWithRetryError, FlowVersion, GenericStepOutput, isFlowRunStateTerminal, JobPayload, LATEST_JOB_DATA_SCHEMA_VERSION, logSerializer, LogSliceRef, ResumeReason, RunEnvironment, RunInternalError, SampleDataFileType, StepOutput, StepOutputStatus, StepOutputType, StreamStepProgress, WorkerJobType } from '@activepieces/shared'
+import { ExecuteFlowJobData, ExecutionType, ExecutioOutputFile, FileCompression, FileType, FlowRetryStrategy, FlowRun, FlowRunCountByStatus, FlowRunStatus, FlowRunWithRetryError, flowStructureUtil, FlowVersion, GenericStepOutput, isFlowRunStateTerminal, JobPayload, LATEST_JOB_DATA_SCHEMA_VERSION, logSerializer, LogSliceRef, ResumeReason, RunEnvironment, RunInternalError, SampleDataFileType, StepOutput, StepOutputStatus, StepOutputType, StreamStepProgress, WorkerJobType } from '@activepieces/shared'
import { FastifyBaseLogger } from 'fastify'
import pLimit from 'p-limit'
import { ArrayContains, In, IsNull, Not, Repository, SelectQueryBuilder } from 'typeorm'
@@ -375,7 +375,12 @@ export const flowRunService = (log: FastifyBaseLogger) => ({
platformId: await projectService(log).getPlatformId(projectId),
executeTrigger: false,
streamStepProgress: StreamStepProgress.WEBSOCKET,
- sampleData: !isNil(stepNameToTest) ? await sampleDataService(log).getSampleDataForFlow({ projectId, flowVersion, type: SampleDataFileType.OUTPUT }) : undefined,
+ sampleData: !isNil(stepNameToTest) ? await sampleDataService(log).getSampleDataForFlow({
+ projectId,
+ flowVersion,
+ type: SampleDataFileType.OUTPUT,
+ referencedBy: settingsOfStepUnderTest({ flowVersion, stepNameToTest }),
+ }) : undefined,
}, log)
},
async startManualTrigger({ projectId, flowVersionId, triggeredBy }: StartManualTriggerParams): Promise<FlowRun> {
@@ -779,6 +784,19 @@ export function isOutsideRetentionWindow(createdTime: string, retentionDays: num
return apDayjs(createdTime).add(retentionDays, 'day').isBefore(apDayjs())
}
+function settingsOfStepUnderTest({ flowVersion, stepNameToTest }: SettingsOfStepUnderTestParams): unknown {
+ const stepUnderTest = flowStructureUtil.getStep(stepNameToTest, flowVersion.trigger)
+ if (isNil(stepUnderTest)) {
+ return undefined
+ }
+ return flowStructureUtil.getAllChildSteps(stepUnderTest).map((step) => step.settings)
+}
+
+type SettingsOfStepUnderTestParams = {
+ flowVersion: FlowVersion
+ stepNameToTest: string
+}
+
type CreateParams = {
projectId: ProjectId
flowVersionId: FlowVersionId
diff --git a/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts b/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts
index 3285393076c..673015c0807 100644
--- a/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts
+++ b/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts
@@ -124,6 +124,17 @@ describe('Sample data scope for property resolution', () => {
expect(Object.keys(sampleData).sort()).toStrictEqual(expectedStepNames)
})
+ it('reads a step under test together with the steps inside it', async () => {
+ const sampleData = await sampleDataService(app.log).getSampleDataForFlow({
+ projectId,
+ flowVersion,
+ type: SampleDataFileType.OUTPUT,
+ referencedBy: [{ items: '{{trigger.rows}}' }, { input: { url: '{{step_1.body}}' } }],
+ })
+
+ expect(Object.keys(sampleData).sort()).toStrictEqual(['step_1', 'trigger'])
+ })
+
it('still reads every step for a whole-flow read', async () => {
const sampleData = await sampleDataService(app.log).getSampleDataForFlow({
projectId,
diff --git a/packages/server/api/test/integration/ce/flows/step-run/test-step-e2e.test.ts b/packages/server/api/test/integration/ce/flows/step-run/test-step-e2e.test.ts
new file mode 100644
index 00000000000..a14380978db
--- /dev/null
+++ b/packages/server/api/test/integration/ce/flows/step-run/test-step-e2e.test.ts
@@ -0,0 +1,179 @@
+import { apDayjs } from '@activepieces/server-utils'
+import {
+ FileCompression,
+ FileType,
+ FlowActionType,
+ FlowRunStatus,
+ FlowTriggerType,
+ FlowVersionState,
+ PackageType,
+ PieceType,
+} from '@activepieces/shared'
+import { FastifyInstance } from 'fastify'
+import { worker } from '../../../../../../worker/src/lib/worker'
+import { databaseConnection } from '../../../../../src/app/database/database-connection'
+import { flowRunService } from '../../../../../src/app/flows/flow-run/flow-run-service'
+import { db } from '../../../../helpers/db'
+import { setupE2eEnvironment } from '../../../../helpers/e2e-setup'
+import {
+ createMockFile,
+ createMockFlow,
+ createMockFlowVersion,
+ createMockPieceMetadata,
+ mockAndSaveBasicSetup,
+} from '../../../../helpers/mocks'
+
+let app: FastifyInstance
+
+beforeAll(async () => {
+ const ctx = await setupE2eEnvironment()
+ app = ctx.app
+ await worker.start({
+ apiUrl: ctx.apiUrl,
+ socketUrl: { url: ctx.apiUrl, path: '/api/socket.io' },
+ workerToken: ctx.workerToken,
+ })
+ await new Promise((resolve) => setTimeout(resolve, 5000))
+}, 30_000)
+
+afterAll(async () => {
+ await worker.stop()
+ await app.close()
+}, 15_000)
+
+async function saveSampleDataFile({ projectId, platformId, output }: { projectId: string, platformId: string, output: unknown }): Promise<string> {
+ const file = createMockFile({
+ projectId,
+ platformId,
+ type: FileType.SAMPLE_DATA,
+ compression: FileCompression.NONE,
+ data: Buffer.from(JSON.stringify(output)),
+ })
+ await db.save('file', file)
+ return file.id
+}
+
+async function pollFlowRunToCompletion({ flowRunId, projectId }: { flowRunId: string, projectId: string }) {
+ const deadline = Date.now() + 120_000
+ let result = await flowRunService(app.log).getOnePopulatedOrThrow({ id: flowRunId, projectId })
+ while (
+ (result.status === FlowRunStatus.QUEUED || result.status === FlowRunStatus.RUNNING) &&
+ Date.now() < deadline
+ ) {
+ await new Promise((resolve) => setTimeout(resolve, 500))
+ result = await flowRunService(app.log).getOnePopulatedOrThrow({ id: flowRunId, projectId })
+ }
+ return result
+}
+
+describe('Test step with narrowed sample data', () => {
+ it('resolves a reference to an earlier step through the full worker round trip', async () => {
+ const { mockPlatform, mockProject } = await mockAndSaveBasicSetup()
+
+ const triggerFileId = await saveSampleDataFile({
+ projectId: mockProject.id,
+ platformId: mockPlatform.id,
+ output: { unreferenced: 'trigger payload the tested step never mentions' },
+ })
+ const firstStepFileId = await saveSampleDataFile({
+ projectId: mockProject.id,
+ platformId: mockPlatform.id,
+ output: { value: 'from-step-1' },
+ })
+
+ const mockPiece = createMockPieceMetadata({
+ name: '@activepieces/piece-webhook',
+ version: '0.1.29',
+ platformId: undefined,
+ packageType: PackageType.REGISTRY,
+ pieceType: PieceType.OFFICIAL,
+ })
+ await databaseConnection().getRepository('piece_metadata').save(mockPiece)
+
+ const mockFlow = createMockFlow({ projectId: mockProject.id })
+ await db.save('flow', mockFlow)
+
+ const stepUnderTest = {
+ type: FlowActionType.CODE,
+ name: 'step_2',
+ displayName: 'Echo',
+ valid: true,
+ lastUpdatedDate: apDayjs().toISOString(),
+ settings: {
+ sourceCode: {
+ code: 'export const code = async (inputs) => ({ echoed: inputs.value });',
+ packageJson: '{}',
+ },
+ input: { value: '{{step_1[\'output\'].value}}' },
+ errorHandlingOptions: {},
+ },
+ }
+
+ const mockFlowVersion = createMockFlowVersion({
+ flowId: mockFlow.id,
+ state: FlowVersionState.DRAFT,
+ trigger: {
+ type: FlowTriggerType.PIECE,
+ name: 'trigger',
+ displayName: 'Catch Webhook',
+ valid: true,
+ lastUpdatedDate: apDayjs().toISOString(),
+ settings: {
+ pieceName: '@activepieces/piece-webhook',
+ pieceVersion: '~0.1.29',
+ triggerName: 'catch_webhook',
+ input: {},
+ propertySettings: {},
+ sampleData: { sampleDataFileId: triggerFileId, lastTestDate: apDayjs().toISOString() },
+ },
+ nextAction: {
+ type: FlowActionType.CODE,
+ name: 'step_1',
+ displayName: 'Produce Value',
+ valid: true,
+ lastUpdatedDate: apDayjs().toISOString(),
+ settings: {
+ sourceCode: { code: 'export const code = async () => ({ value: \'never runs\' });', packageJson: '{}' },
+ input: {},
+ errorHandlingOptions: {},
+ sampleData: { sampleDataFileId: firstStepFileId, lastTestDate: apDayjs().toISOString() },
+ },
+ nextAction: stepUnderTest,
+ },
+ },
+ })
+ await db.save('flow_version', mockFlowVersion)
+
+ const flowRun = await flowRunService(app.log).test({
+ projectId: mockProject.id,
+ flowVersionId: mockFlowVersion.id,
+ stepNameToTest: 'step_2',
+ triggeredBy: undefined,
+ })
+
+ const result = await pollFlowRunToCompletion({ flowRunId: flowRun.id, projectId: mockProject.id })
+
+ expect(result.status).toBe(FlowRunStatus.SUCCEEDED)
+ expect(result.steps?.['step_2']?.output).toStrictEqual({ echoed: 'from-step-1' })
+ }, 180_000)
+
+ it('still queues a tracked run when the step under test no longer exists', async () => {
+ const { mockProject } = await mockAndSaveBasicSetup()
+ const mockFlow = createMockFlow({ projectId: mockProject.id })
+ await db.save('flow', mockFlow)
+ const mockFlowVersion = createMockFlowVersion({
+ flowId: mockFlow.id,
+ state: FlowVersionState.DRAFT,
+ })
+ await db.save('flow_version', mockFlowVersion)
+
+ const flowRun = await flowRunService(app.log).test({
+ projectId: mockProject.id,
+ flowVersionId: mockFlowVersion.id,
+ stepNameToTest: 'deleted_step',
+ triggeredBy: undefined,
+ })
+
+ expect(flowRun.id).toBeDefined()
+ }, 180_000)
+})