AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
fix(flows): testing a step no longer waits on the whole flow's sample data (#15713)
Hazem Adel8385e0eb8fec
diff --git a/brain/knowledge/pieces-engine/pieces.md b/brain/knowledge/pieces-engine/pieces.mdindex d6ba047925f..711184ffecb 100644--- a/brain/knowledge/pieces-engine/pieces.md+++ b/brain/knowledge/pieces-engine/pieces.md@@ -29,6 +29,7 @@ The metadata catalog of automation integrations ("pieces") — each a named inte- **DynamicProperties clears its value before it knows the new schema, so the merge source must be a snapshot.** `DynamicPropertiesImplementation` re-fetches the child schema on every refresher change, clearing the form value synchronously and re-populating it in the mutation callback. The merge source for `getDefaultValueForProperties` has to be a `lastKnownValue` ref captured *before* the clear — reading `form.getValues()` in the callback sees the cleared `null` and defaults every child (GIT-1514). The snapshot must be spread-cloned: RHF `getValues(name)` hands back the live object and the clear's `setValue(...child, null)` mutates it in place. Guard the ref with `isNil` so it survives rapid successive changes, where later effect runs already observe `null`.- `DynamicPropertiesContext` tracks loading by property name only, so two in-flight requests for the same property let the first completion clear the flag for both — briefly re-enabling Test Step while the value is still cleared.- **`POST /v1/pieces/options` cost scales with the whole flow's sample data, not with the property being resolved.** The handler builds an `EXECUTE_PROPERTY` job whose payload carries `sampleData` — and `getSampleDataForFlow` reads *every* step's sample-data `file` row. That payload then crosses Postgres → API → Redis → worker → engine socket, so a big flow pays megabytes to resolve a trivial resolver like the HTTP piece's `authFields`: measured 7.3 MB on a 25-step flow, and a panel open fires three of these in parallel (GIT-1822). The engine only uses `sampleData` to seed step outputs so `{{step.x}}` inside the property's `input` resolves, so callers must use `getSampleDataReferencedBy` and pass the request `input` — which narrows the read to the steps that input names. When narrowing, a referenced `LOOP_ON_ITEMS` step also needs whatever its `items` expression references: `stateFromFlowVersion` derives a loop's `item` by resolving its settings against the context built so far, never from `sampleData[loopName]`. Matching step names as substrings of the serialized input over-approximates, which is the safe direction — a step reference always spells the name literally, in both the property-path and script branches of `props-resolver`.+- **There is no static signal for whether a DynamicProperties resolver is pure, so caching its result is never safe by inspection.** Both tempting heuristics have counterexamples in this repo. "Trivial resolver" fails because purity cannot be read off the declaration; "declares no connection" fails because `packages/pieces/core/tables` `update-record` declares `auth: PieceAuth.None()` yet its `values` resolver calls `tablesCommon.createFieldProperties` → `getTableFields`, a live request to `v1/fields` authenticated with the **engine token** rather than a connection — so a column added to an Activepieces Table would not show up. Third-party resolvers that fetch live schema (Zendesk `userFieldsDynamicProp`, Supabase `table_columns`) are the same class reached through a connection. The failure mode is a silently outdated form, which the builder gives the user no way to refresh. Anything in this area (GIT-813) needs either a purity signal the piece declares, or stale-while-revalidate that always refetches and corrects within a round trip — and note that SWR re-applies the schema a second time, which is exactly the path the `lastKnownValue` snapshot above exists to protect.- **The frontend `POST /v1/pieces/options` client only rejects for DYNAMIC.** `piecesApi.options` (`packages/web/src/features/pieces/api/`) catches DROPDOWN failures, toasts, and *resolves* with a disabled-dropdown fallback — so for dropdowns every error path wired onto that mutation is dead: `usePieceOptions`' `onError` handlers, its `retry: 1`, and the `if (error) throw error` into `DynamicPropertiesErrorBoundary`. DYNAMIC must rethrow: a swallowed failure arrives as a *successful* empty schema, which resets the property's children to defaults and gets persisted by step-settings autosave.- **An action that can be called as an MCP tool cannot create a waitpoint.** An MCP piece tool runs thereal action through `actionRunService`, which sets `actionRunMode` on the engine, anddiff --git a/packages/server/api/src/app/flows/flow-run/flow-run-service.ts b/packages/server/api/src/app/flows/flow-run/flow-run-service.tsindex d9d089a1899..ce8560d1069 100644--- a/packages/server/api/src/app/flows/flow-run/flow-run-service.ts+++ b/packages/server/api/src/app/flows/flow-run/flow-run-service.ts@@ -1,6 +1,6 @@import { ActivepiecesError, apId, Cursor, ErrorCode, FlowId, FlowRunId, FlowVersionId, isNil, PlatformId, ProjectId, SeekPage } from '@activepieces/core-utils'import { apDayjs, apDayjsDuration, wideEvent } from '@activepieces/server-utils'-import { ExecuteFlowJobData, ExecutionType, ExecutioOutputFile, FileCompression, FileType, FlowRetryStrategy, FlowRun, FlowRunCountByStatus, FlowRunStatus, FlowRunWithRetryError, FlowVersion, GenericStepOutput, isFlowRunStateTerminal, JobPayload, LATEST_JOB_DATA_SCHEMA_VERSION, logSerializer, LogSliceRef, ResumeReason, RunEnvironment, RunInternalError, SampleDataFileType, StepOutput, StepOutputStatus, StepOutputType, StreamStepProgress, WorkerJobType } from '@activepieces/shared'+import { ExecuteFlowJobData, ExecutionType, ExecutioOutputFile, FileCompression, FileType, FlowRetryStrategy, FlowRun, FlowRunCountByStatus, FlowRunStatus, FlowRunWithRetryError, flowStructureUtil, FlowVersion, GenericStepOutput, isFlowRunStateTerminal, JobPayload, LATEST_JOB_DATA_SCHEMA_VERSION, logSerializer, LogSliceRef, ResumeReason, RunEnvironment, RunInternalError, SampleDataFileType, StepOutput, StepOutputStatus, StepOutputType, StreamStepProgress, WorkerJobType } from '@activepieces/shared'import { FastifyBaseLogger } from 'fastify'import pLimit from 'p-limit'import { ArrayContains, In, IsNull, Not, Repository, SelectQueryBuilder } from 'typeorm'@@ -375,7 +375,12 @@ export const flowRunService = (log: FastifyBaseLogger) => ({platformId: await projectService(log).getPlatformId(projectId),executeTrigger: false,streamStepProgress: StreamStepProgress.WEBSOCKET,- sampleData: !isNil(stepNameToTest) ? await sampleDataService(log).getSampleDataForFlow({ projectId, flowVersion, type: SampleDataFileType.OUTPUT }) : undefined,+ sampleData: !isNil(stepNameToTest) ? await sampleDataService(log).getSampleDataForFlow({+ projectId,+ flowVersion,+ type: SampleDataFileType.OUTPUT,+ referencedBy: settingsOfStepUnderTest({ flowVersion, stepNameToTest }),+ }) : undefined,}, log)},async startManualTrigger({ projectId, flowVersionId, triggeredBy }: StartManualTriggerParams): Promise<FlowRun> {@@ -779,6 +784,19 @@ export function isOutsideRetentionWindow(createdTime: string, retentionDays: numreturn apDayjs(createdTime).add(retentionDays, 'day').isBefore(apDayjs())}+function settingsOfStepUnderTest({ flowVersion, stepNameToTest }: SettingsOfStepUnderTestParams): unknown {+ const stepUnderTest = flowStructureUtil.getStep(stepNameToTest, flowVersion.trigger)+ if (isNil(stepUnderTest)) {+ return undefined+ }+ return flowStructureUtil.getAllChildSteps(stepUnderTest).map((step) => step.settings)+}++type SettingsOfStepUnderTestParams = {+ flowVersion: FlowVersion+ stepNameToTest: string+}+type CreateParams = {projectId: ProjectIdflowVersionId: FlowVersionIddiff --git a/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts b/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.tsindex 3285393076c..673015c0807 100644--- a/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts+++ b/packages/server/api/test/integration/ce/flows/step-run/sample-data-scope.test.ts@@ -124,6 +124,17 @@ describe('Sample data scope for property resolution', () => {expect(Object.keys(sampleData).sort()).toStrictEqual(expectedStepNames)})+ it('reads a step under test together with the steps inside it', async () => {+ const sampleData = await sampleDataService(app.log).getSampleDataForFlow({+ projectId,+ flowVersion,+ type: SampleDataFileType.OUTPUT,+ referencedBy: [{ items: '{{trigger.rows}}' }, { input: { url: '{{step_1.body}}' } }],+ })++ expect(Object.keys(sampleData).sort()).toStrictEqual(['step_1', 'trigger'])+ })+it('still reads every step for a whole-flow read', async () => {const sampleData = await sampleDataService(app.log).getSampleDataForFlow({projectId,diff --git a/packages/server/api/test/integration/ce/flows/step-run/test-step-e2e.test.ts b/packages/server/api/test/integration/ce/flows/step-run/test-step-e2e.test.tsnew file mode 100644index 00000000000..a14380978db--- /dev/null+++ b/packages/server/api/test/integration/ce/flows/step-run/test-step-e2e.test.ts@@ -0,0 +1,179 @@+import { apDayjs } from '@activepieces/server-utils'+import {+ FileCompression,+ FileType,+ FlowActionType,+ FlowRunStatus,+ FlowTriggerType,+ FlowVersionState,+ PackageType,+ PieceType,+} from '@activepieces/shared'+import { FastifyInstance } from 'fastify'+import { worker } from '../../../../../../worker/src/lib/worker'+import { databaseConnection } from '../../../../../src/app/database/database-connection'+import { flowRunService } from '../../../../../src/app/flows/flow-run/flow-run-service'+import { db } from '../../../../helpers/db'+import { setupE2eEnvironment } from '../../../../helpers/e2e-setup'+import {+ createMockFile,+ createMockFlow,+ createMockFlowVersion,+ createMockPieceMetadata,+ mockAndSaveBasicSetup,+} from '../../../../helpers/mocks'++let app: FastifyInstance++beforeAll(async () => {+ const ctx = await setupE2eEnvironment()+ app = ctx.app+ await worker.start({+ apiUrl: ctx.apiUrl,+ socketUrl: { url: ctx.apiUrl, path: '/api/socket.io' },+ workerToken: ctx.workerToken,+ })+ await new Promise((resolve) => setTimeout(resolve, 5000))+}, 30_000)++afterAll(async () => {+ await worker.stop()+ await app.close()+}, 15_000)++async function saveSampleDataFile({ projectId, platformId, output }: { projectId: string, platformId: string, output: unknown }): Promise<string> {+ const file = createMockFile({+ projectId,+ platformId,+ type: FileType.SAMPLE_DATA,+ compression: FileCompression.NONE,+ data: Buffer.from(JSON.stringify(output)),+ })+ await db.save('file', file)+ return file.id+}++async function pollFlowRunToCompletion({ flowRunId, projectId }: { flowRunId: string, projectId: string }) {+ const deadline = Date.now() + 120_000+ let result = await flowRunService(app.log).getOnePopulatedOrThrow({ id: flowRunId, projectId })+ while (+ (result.status === FlowRunStatus.QUEUED || result.status === FlowRunStatus.RUNNING) &&+ Date.now() < deadline+ ) {+ await new Promise((resolve) => setTimeout(resolve, 500))+ result = await flowRunService(app.log).getOnePopulatedOrThrow({ id: flowRunId, projectId })+ }+ return result+}++describe('Test step with narrowed sample data', () => {+ it('resolves a reference to an earlier step through the full worker round trip', async () => {+ const { mockPlatform, mockProject } = await mockAndSaveBasicSetup()++ const triggerFileId = await saveSampleDataFile({+ projectId: mockProject.id,+ platformId: mockPlatform.id,+ output: { unreferenced: 'trigger payload the tested step never mentions' },+ })+ const firstStepFileId = await saveSampleDataFile({+ projectId: mockProject.id,+ platformId: mockPlatform.id,+ output: { value: 'from-step-1' },+ })++ const mockPiece = createMockPieceMetadata({+ name: '@activepieces/piece-webhook',+ version: '0.1.29',+ platformId: undefined,+ packageType: PackageType.REGISTRY,+ pieceType: PieceType.OFFICIAL,+ })+ await databaseConnection().getRepository('piece_metadata').save(mockPiece)++ const mockFlow = createMockFlow({ projectId: mockProject.id })+ await db.save('flow', mockFlow)++ const stepUnderTest = {+ type: FlowActionType.CODE,+ name: 'step_2',+ displayName: 'Echo',+ valid: true,+ lastUpdatedDate: apDayjs().toISOString(),+ settings: {+ sourceCode: {+ code: 'export const code = async (inputs) => ({ echoed: inputs.value });',+ packageJson: '{}',+ },+ input: { value: '{{step_1[\'output\'].value}}' },+ errorHandlingOptions: {},+ },+ }++ const mockFlowVersion = createMockFlowVersion({+ flowId: mockFlow.id,+ state: FlowVersionState.DRAFT,+ trigger: {+ type: FlowTriggerType.PIECE,+ name: 'trigger',+ displayName: 'Catch Webhook',+ valid: true,+ lastUpdatedDate: apDayjs().toISOString(),+ settings: {+ pieceName: '@activepieces/piece-webhook',+ pieceVersion: '~0.1.29',+ triggerName: 'catch_webhook',+ input: {},+ propertySettings: {},+ sampleData: { sampleDataFileId: triggerFileId, lastTestDate: apDayjs().toISOString() },+ },+ nextAction: {+ type: FlowActionType.CODE,+ name: 'step_1',+ displayName: 'Produce Value',+ valid: true,+ lastUpdatedDate: apDayjs().toISOString(),+ settings: {+ sourceCode: { code: 'export const code = async () => ({ value: \'never runs\' });', packageJson: '{}' },+ input: {},+ errorHandlingOptions: {},+ sampleData: { sampleDataFileId: firstStepFileId, lastTestDate: apDayjs().toISOString() },+ },+ nextAction: stepUnderTest,+ },+ },+ })+ await db.save('flow_version', mockFlowVersion)++ const flowRun = await flowRunService(app.log).test({+ projectId: mockProject.id,+ flowVersionId: mockFlowVersion.id,+ stepNameToTest: 'step_2',+ triggeredBy: undefined,+ })++ const result = await pollFlowRunToCompletion({ flowRunId: flowRun.id, projectId: mockProject.id })++ expect(result.status).toBe(FlowRunStatus.SUCCEEDED)+ expect(result.steps?.['step_2']?.output).toStrictEqual({ echoed: 'from-step-1' })+ }, 180_000)++ it('still queues a tracked run when the step under test no longer exists', async () => {+ const { mockProject } = await mockAndSaveBasicSetup()+ const mockFlow = createMockFlow({ projectId: mockProject.id })+ await db.save('flow', mockFlow)+ const mockFlowVersion = createMockFlowVersion({+ flowId: mockFlow.id,+ state: FlowVersionState.DRAFT,+ })+ await db.save('flow_version', mockFlowVersion)++ const flowRun = await flowRunService(app.log).test({+ projectId: mockProject.id,+ flowVersionId: mockFlowVersion.id,+ stepNameToTest: 'deleted_step',+ triggeredBy: undefined,+ })++ expect(flowRun.id).toBeDefined()+ }, 180_000)+})