AI Agents & MCPs & AI Workflow Automation • (~400 MCP servers for AI agents) • AI Automation / AI Agent with MCPs • AI Workflows & AI Agents • MCPs for AI Agents
TypeScript ai-agentai-agent-toolsai-agentsai-agents-frameworkmcpmcp-servermcp-toolsmcpsn8n-alternativeno-code-automationworkflowworkflow-automationworkflows
ci: run the e2e matrix from the ready-for-e2e label (#16899)
Othman Abu Ajamiehd5a364540120
diff --git a/.github/workflows/e2e.yml b/.github/workflows/e2e.ymlindex 72e406f6cca..30437130a89 100644--- a/.github/workflows/e2e.yml+++ b/.github/workflows/e2e.yml@@ -1,144 +1,23 @@name: E2E Tests+# Runs the E2E matrix on a PR that carries the ready-for-e2e label.on:- pull_request_target:- types: [opened, synchronize, reopened, labeled]+ pull_request:+ types: [opened, reopened, synchronize, labeled]branches:- main- - gh-actions-test-branch- workflow_dispatch:permissions:- actions: writecontents: read- issues: write- pull-requests: write--concurrency:- group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}- cancel-in-progress: true+ packages: readjobs:- check-label:- runs-on: ubuntu-latest- outputs:- should-run-e2e: ${{ steps.check-if-pr-has-label.outputs.run-e2e }}- steps:- - name: Check if PR exists with ready-for-e2e label for this SHA- id: check-if-pr-has-label- uses: actions/github-script@v8- with:- script: |- // Always run on manual workflow dispatch- if (context.eventName === 'workflow_dispatch') {- core.setOutput('run-e2e', true);- return;- }-- let labels = [];- if (context.payload.pull_request) {- labels = context.payload.pull_request.labels;- } else {- try {- const sha = context.sha;- const { data: prs } = await github.rest.repos.listPullRequestsAssociatedWithCommit({- owner: context.repo.owner,- repo: context.repo.repo,- commit_sha: sha- });- if (prs.length === 0) {- core.setOutput('run-e2e', false);- return;- }- const pr = prs[0];- labels = pr.labels;- } catch (e) {- core.setOutput('run-e2e', false);- return;- }- }-- const labelFound = labels.map(l => l.name).includes('ready-for-e2e');- core.setOutput('run-e2e', labelFound);-- test-e2e-ee:- needs: check-label- if: needs.check-label.outputs.should-run-e2e == 'true'- uses: ./.github/workflows/tests-e2e-ee.yml-- test-e2e-ce:- needs: check-label- if: needs.check-label.outputs.should-run-e2e == 'true'- uses: ./.github/workflows/tests-e2e-ce.yml-- notify-on-completion:- needs: [check-label, test-e2e-ce, test-e2e-ee]- if: always() && needs.check-label.outputs.should-run-e2e == 'true'- runs-on: ubuntu-latest- steps:- - name: Download all artifacts- uses: actions/download-artifact@v7- with:- path: ./artifacts- pattern: "*"- merge-multiple: false-- - name: Re-upload consolidated artifacts- uses: actions/upload-artifact@v6- if: always()- with:- name: e2e-test-artifacts-all- path: ./artifacts/- retention-days: 30-- - name: Comment on PR with test results- if: always() && github.event_name == 'pull_request_target'- uses: actions/github-script@v8- with:- script: |- // Determine which tests failed- const results = {- ee: '${{ needs.test-e2e-ee.result }}',- ce: '${{ needs.test-e2e-ce.result }}'- };-- const failed = Object.entries(results).filter(([_, status]) => status === 'failure');- const passed = Object.entries(results).filter(([_, status]) => status === 'success');- const skipped = Object.entries(results).filter(([_, status]) => status === 'skipped');-- // Build status summary- let summary = "## 🧪 E2E Test Results\n\n";-- if (failed.length > 0) {- summary += "### ❌ Failed Tests\n";- failed.forEach(([edition, _]) => {- summary += `- **${edition.toUpperCase()} Edition**\n`;- });- summary += "\n";- }-- if (passed.length > 0) {- summary += "### ✅ Passed Tests\n";- passed.forEach(([edition, _]) => {- summary += `- **${edition.toUpperCase()} Edition**\n`;- });- summary += "\n";- }-- if (skipped.length > 0) {- summary += "### ⏭️ Skipped Tests\n";- skipped.forEach(([edition, _]) => {- summary += `- **${edition.toUpperCase()} Edition**\n`;- });- summary += "\n";- }-- summary += `🤖 Automated E2E test results from [workflow run ${{ github.run_id }}](https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }})`;-- // Post comment- await github.rest.issues.createComment({- issue_number: context.issue.number,- owner: context.repo.owner,- repo: context.repo.repo,- body: summary- });\ No newline at end of file+ e2e:+ if: >+ (github.event.action == 'labeled' && github.event.label.name == 'ready-for-e2e') ||+ (github.event.action != 'labeled' && contains(github.event.pull_request.labels.*.name, 'ready-for-e2e'))+ concurrency:+ group: e2e-${{ github.event.pull_request.number }}+ cancel-in-progress: true+ uses: ./.github/workflows/e2e-matrix.yml+ secrets: inheritdiff --git a/.github/workflows/tests-e2e-ce.yml b/.github/workflows/tests-e2e-ce.ymldeleted file mode 100644index e84f491234e..00000000000--- a/.github/workflows/tests-e2e-ce.yml+++ /dev/null@@ -1,109 +0,0 @@-name: E2E Tests - Community Edition--on:- workflow_call:- workflow_dispatch:--env:- AP_DEV_PIECES: store,webhook- AP_PIECES_SYNC_MODE: NONE- AP_PIECES_SOURCE: FILE- AP_API_KEY: api-key- AP_DB_TYPE: POSTGRES- AP_POSTGRES_HOST: localhost- AP_POSTGRES_PORT: 5432- AP_POSTGRES_USERNAME: postgres- AP_POSTGRES_PASSWORD: postgres- AP_POSTGRES_DATABASE: activepieces- AP_POSTGRES_USE_SSL: "false"- AP_REDIS_URL: redis://localhost:6379- AP_QUEUE_MODE: REDIS- AP_FRONTEND_URL: http://localhost:4200- AP_WEBHOOK_TIMEOUT_SECONDS: "30"- AP_TRIGGER_DEFAULT_POLL_INTERVAL: "1"- AP_EXECUTION_MODE: UNSANDBOXED- AP_TELEMETRY_ENABLED: "false"- AP_ENVIRONMENT: test- AP_JWT_SECRET: "secret"- AP_ENCRYPTION_KEY: "fdbe51af3678563560972e4a912aee6d"--jobs:- test-e2e-ce:- runs-on: depot-ubuntu-24.04-4- timeout-minutes: 30-- services:- postgres:- image: postgres:17- env:- POSTGRES_USER: postgres- POSTGRES_PASSWORD: postgres- POSTGRES_DB: activepieces- options: >-- --health-cmd pg_isready- --health-interval 10s- --health-timeout 5s- --health-retries 5- ports:- - 5432:5432- redis:- image: redis:latest- ports:- - 6379:6379- options: >-- --health-cmd "redis-cli ping"- --health-interval 10s- --health-timeout 5s- --health-retries 5-- steps:- - name: Checkout code- uses: actions/checkout@v5- with:- ref: ${{ github.event.pull_request.head.sha }}- fetch-depth: 2-- - name: Cache dependencies- uses: actions/cache@v5- with:- path: ~/.bun/install/cache- key: bun-${{ hashFiles('bun.lock', 'package.json') }}- restore-keys: bun--- - name: Setup nodejs- uses: actions/setup-node@v6- with:- node-version: 24-- - name: Setup Bun- uses: oven-sh/setup-bun@v2- with:- bun-version: latest-- - name: Install dependencies- run: bun install-- - name: Install Playwright browsers- run: npx playwright install --with-deps chromium-- - name: Run E2E tests- run: |- npm run test:e2e- env:- CI: true-- - name: Upload Playwright Report- uses: actions/upload-artifact@v6- if: always()- with:- name: playwright-report-ce- path: packages/tests-e2e/playwright-report/- retention-days: 10-- - name: Upload Test Results- uses: actions/upload-artifact@v6- if: always()- with:- name: test-results-ce- path: packages/tests-e2e/test-results/- retention-days: 10diff --git a/.github/workflows/tests-e2e-ee.yml b/.github/workflows/tests-e2e-ee.ymldeleted file mode 100644index 0f666fcca66..00000000000--- a/.github/workflows/tests-e2e-ee.yml+++ /dev/null@@ -1,114 +0,0 @@-name: E2E Tests - Enterprise Edition--on:- workflow_call:- workflow_dispatch:--env:- AP_EDITION: ee- AP_DEV_PIECES: store,webhook- AP_PIECES_SYNC_MODE: NONE- AP_PIECES_SOURCE: FILE- AP_API_KEY: api-key- AP_LICENSE_KEY: ${{ secrets.EE_LICENSE_KEY }}- AP_DB_TYPE: POSTGRES- AP_POSTGRES_HOST: localhost- AP_POSTGRES_PORT: 5432- AP_POSTGRES_USERNAME: postgres- AP_POSTGRES_PASSWORD: postgres- AP_POSTGRES_DATABASE: activepieces- AP_POSTGRES_USE_SSL: "false"- AP_JWT_SECRET: "secret"- AP_ENCRYPTION_KEY: "fdbe51af3678563560972e4a912aee6d"- AP_FRONTEND_URL: http://localhost:4200- AP_REDIS_URL: redis://localhost:6379- AP_QUEUE_MODE: REDIS- AP_EXECUTION_MODE: SANDBOX_CODE_ONLY- AP_TELEMETRY_ENABLED: "false"- NODE_ENV: test--jobs:- test-e2e-ee:- runs-on: depot-ubuntu-24.04- timeout-minutes: 30-- services:- postgres:- image: postgres:17- env:- POSTGRES_USER: postgres- POSTGRES_PASSWORD: postgres- POSTGRES_DB: activepieces- options: >-- --health-cmd pg_isready- --health-interval 10s- --health-timeout 5s- --health-retries 5- ports:- - 5432:5432- redis:- image: redis:latest- ports:- - 6379:6379- options: >-- --health-cmd "redis-cli ping"- --health-interval 10s- --health-timeout 5s- --health-retries 5-- steps:- - name: Checkout code- uses: actions/checkout@v5- with:- ref: ${{ github.event.pull_request.head.sha }}- fetch-depth: 2-- - name: Cache dependencies- uses: actions/cache@v5- with:- path: ~/.bun/install/cache- key: bun-${{ hashFiles('bun.lock', 'package.json') }}- restore-keys: bun--- - name: Setup nodejs- uses: actions/setup-node@v6- with:- node-version: 24-- - name: Setup Bun- uses: oven-sh/setup-bun@v2- with:- bun-version: latest-- - name: Install dependencies- run: bun install-- - name: Install Playwright browsers- run: npx playwright install --with-deps chromium-- - name: Run E2E tests- run: |- bun run test:e2e- env:- CI: true-- - name: Upload Playwright Report- uses: actions/upload-artifact@v6- if: always()- with:- name: playwright-report-ee- path: packages/tests-e2e/playwright-report/- retention-days: 30-- - name: Upload Test Results- uses: actions/upload-artifact@v6- if: always()- with:- name: test-results-ee- path: packages/tests-e2e/test-results/- retention-days: 30-- - name: Notify on failure- if: failure()- run: |- echo "::error::Enterprise Edition E2E tests failed. This may indicate licensing or advanced feature issues."diff --git a/brain/knowledge/engineering/e2e-tests-and-monitors.md b/brain/knowledge/engineering/e2e-tests-and-monitors.mdindex a3ada483194..cb8e81a1188 100644--- a/brain/knowledge/engineering/e2e-tests-and-monitors.md+++ b/brain/knowledge/engineering/e2e-tests-and-monitors.md@@ -16,7 +16,8 @@ One Playwright suite in `packages/tests-e2e` feeds three consumers that fail ind- **BetterStack does not read the repo — the repo pushes to it.** `.github/workflows/sync-betterstack-playwright.yml` fires on push to `main` and `PATCH`es the file's contents into the hardcoded monitor `4211060` as `playwright_script`. One-way and `main`-only: the monitor updates at *merge*, never on the PR, and any edit made in the BetterStack UI is silently overwritten by the next push.- **The BetterStack file is deliberately flat and duplicated.** BetterStack executes one self-contained script, so it cannot `require` the `pages/` objects — its sign-in is a copy. Fix the page object and you have *not* fixed the monitor; both files need the change.- **The `.flat.spec.js` runs nowhere else.** Playwright matches `*.spec.ts` and Checkly matches `**/scenarios/**/*.spec.ts`, so a broken flat file is invisible locally and in CI until it fails in BetterStack.-- **CI only runs on the `ready-for-e2e` label** (`e2e.yml` gates both edition workflows on it), which is why the suite can rot for weeks without anyone noticing.+- **PRs do not run E2E unless they touch the suite or carry `ready-for-e2e`.** `e2e-matrix.yml` runs on its own only when `packages/tests-e2e/**` or its compose files change, so an unlabeled server change merges with no E2E check and the first signal is staging. A missing check is not a pass. The label makes `e2e.yml` call the same matrix.+- **The label runs on `pull_request`, not `pull_request_target`.** A fork PR gets no `EE_LICENSE_KEY`, so its two EE configs fail; read the CE ones. Do not move the trigger back to `pull_request_target` to "fix" that: it hands PR code the secret.- **Turbo strict env mode silently strips most of `.env.e2e`.** `globalPassThroughEnv` in `turbo.json` is an allow-list, so vars not named there never reach the `serve` tasks — verify with `tr '\0' '\n' < /proc/<api-pid>/environ`. `AP_ENVIRONMENT` is among the casualties, so CI falls back to the `prod` default. Widening it to `AP_*` does forward them, but that alone broke worker→API Socket.IO auth (jobs queue up unconsumed), so the passthrough and the worker's `AP_WORKER_TOKEN` have to be sorted out together.- **`AP_DEV_PIECES` loads from `packages/pieces/**/dist`, which `npm run dev` does not build.** Only pieces that happen to be build dependencies of api/worker have a `dist`, so a default dev instance serves **0 pieces** and every spec that picks a trigger times out on the piece search. Build them explicitly: `npx turbo run build --filter=@activepieces/piece-webhook --filter=@activepieces/piece-store`.- **CE sign-up is invitation-only once a platform exists** (`INVITATION_ONLY_SIGN_UP`), so the suite's sign-up path only works on a genuinely fresh instance. Against a dev-seeded database, set `E2E_EMAIL` / `E2E_PASSWORD` instead — `global-setup.ts` prefers them and signs in rather than signing up.@@ -27,5 +28,6 @@ One Playwright suite in `packages/tests-e2e` feeds three consumers that fail ind## Key files- `packages/tests-e2e` — `playwright.config.ts` (local/CI), `checkly.config.ts` (Cloud monitors), `global-setup.ts` (provisions or signs in the seed account), `pages/` (shared page objects), `scenarios/betterstack/` (the standalone monitor script)-- `.github/workflows/e2e.yml` — the `ready-for-e2e` gate that calls the per-edition workflows+- `.github/workflows/e2e-matrix.yml` — the only E2E suite that gates a release: the built image in four CE/EE configurations+- `.github/workflows/e2e.yml` — the `ready-for-e2e` label trigger for that matrix- `.github/workflows/sync-betterstack-playwright.yml` — the push-to-`main` uploaddiff --git a/packages/tests-e2e/README.md b/packages/tests-e2e/README.mdindex b2e28ec43ce..9f94156638f 100644--- a/packages/tests-e2e/README.md+++ b/packages/tests-e2e/README.md@@ -38,10 +38,11 @@ Required environment variables:## CI-Triggered by:+PRs do not run E2E unless they change the suite or carry the `ready-for-e2e` label. Staging and the release candidate run it.-- `.github/workflows/tests-e2e-ce.yml` — CE Playwright suite-- `.github/workflows/tests-e2e-ee.yml` — EE Playwright suite+- `.github/workflows/e2e-matrix.yml` — the only E2E suite that gates a release: builds the image and runs the suite in four CE/EE configurations. Runs on PRs that touch `packages/tests-e2e/**`, and from the Cloud gate.+- `.github/workflows/e2e.yml` — runs `e2e-matrix.yml` on a PR with the `ready-for-e2e` label, and again on each push while the label stays+- `.github/workflows/continuous-delivery-stg.yml` — runs the CE specs against staging after each deploy- `.github/workflows/e2e-tests-checkly.yml` — scheduled Checkly run (optional)## Related documentation