Skip to content

armgit / spree

Public Imported from GitHub
0

Open Source Platform for DTC, B2B Commerce, Marketplaces & Omnichannel. REST APIs, TypeScript SDKs, and production-ready Next.js storefront. Self-host it. Own your data. No vendor lock-in. Zero platform fees.

Parallelize dashboard E2E tests across sharded CI jobs (#14343)

Damian Legawiec8af053ca4136
diff --git a/.github/workflows/packages.yml b/.github/workflows/packages.yml
index f6dda3d238..690d0c2d00 100644
--- a/.github/workflows/packages.yml
+++ b/.github/workflows/packages.yml
@@ -98,8 +98,16 @@ jobs:
run: pnpm test:integration
dashboard-e2e:
- name: "Dashboard - E2E"
+ name: "Dashboard - E2E (${{ matrix.shard }}/4)"
runs-on: ubuntu-latest
+ # Each shard is a full isolated stack (own Rails + SQLite + Vite), so
+ # shards can't interfere through shared server state the way parallel
+ # workers against one backend would. Playwright distributes spec files
+ # across shards; within a shard the suite stays serial (workers: 1).
+ strategy:
+ fail-fast: false
+ matrix:
+ shard: [1, 2, 3, 4]
env:
RAILS_ENV: test
DB: sqlite3
@@ -119,14 +127,25 @@ jobs:
node-version: 22
cache: pnpm
+ # Same Turbo cache as the check job — unchanged packages restore their
+ # build outputs instead of rebuilding per shard.
+ - name: Cache Turbo
+ uses: actions/cache@v4
+ with:
+ path: .turbo
+ key: turbo-${{ runner.os }}-${{ github.sha }}
+ restore-keys: |
+ turbo-${{ runner.os }}-
+
- run: pnpm install --frozen-lockfile
- # The admin SPA imports built dist from the SDKs (per the workspace
- # convention), and its vite.config resolves dashboard-core's compiled
- # Vite entry (dist/vite) — build them first so Vite resolves the
- # workspace deps.
- - name: Build SDK dependencies
- run: pnpm turbo build --filter=@spree/sdk-core --filter=@spree/admin-sdk --filter=@spree/dashboard-core
+ # Build the dashboard and its workspace deps (sdk-core, admin-sdk,
+ # dashboard-core/-ui via ^build). The E2E suite runs against the built
+ # SPA (`vite preview`, E2E_PREVIEW below): fresh browser contexts have
+ # empty caches, so the dev server would re-serve hundreds of on-demand
+ # modules on every page load in every test.
+ - name: Build dashboard packages
+ run: pnpm turbo build --filter=@spree/dashboard --cache-dir=.turbo
- name: Setup dummy Rails app
working-directory: spree/api
@@ -161,18 +180,36 @@ jobs:
- name: Run dashboard E2E tests
working-directory: packages/dashboard
- run: pnpm test:e2e
+ env:
+ E2E_PREVIEW: "1"
+ run: pnpm test:e2e --shard=${{ matrix.shard }}/4
- name: Upload Playwright report
if: failure()
uses: actions/upload-artifact@v4
with:
- name: playwright-report
+ name: playwright-report-shard-${{ matrix.shard }}
path: |
packages/dashboard/playwright-report/
packages/dashboard/test-results/
retention-days: 14
+ # Stable single-status roll-up of the shard matrix, keeping the check name
+ # branch-protection rules and status badges can pin against.
+ dashboard-e2e-result:
+ name: "Dashboard - E2E"
+ needs: dashboard-e2e
+ if: ${{ !cancelled() }}
+ runs-on: ubuntu-latest
+ steps:
+ - name: Check shard results
+ run: |
+ if [ "${{ needs.dashboard-e2e.result }}" != "success" ]; then
+ echo "One or more E2E shards did not succeed: ${{ needs.dashboard-e2e.result }}"
+ exit 1
+ fi
+ echo "All E2E shards passed."
+
release-sdk:
name: "SDK - Release"
needs: check
diff --git a/CLAUDE.md b/CLAUDE.md
index c2c3b76df5..45e4fd1afc 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -638,13 +638,15 @@ cd packages/sdk && pnpm test # SDK tests (uses MSW for HTTP mocking)
### Admin SPA E2E (Playwright)
-End-to-end tests for `packages/dashboard` live in `packages/dashboard/e2e/`. The global setup boots a real Rails test server (port 3010) + Vite dev (port 5174) once and seeds the DB; specs then exercise the SPA through a browser against that stack.
+End-to-end tests for `packages/dashboard` live in `packages/dashboard/e2e/`. The global setup boots a real Rails test server (port 3010) + Vite (port 5174) once and seeds the DB; specs then exercise the SPA through a browser against that stack. Locally Vite runs in dev mode; CI builds first and serves the bundle (`E2E_PREVIEW=1` → `vite preview`) because every test's fresh browser context re-downloads all dev-mode modules. CI also splits the suite across shard jobs (`--shard=n/m`), each with its own isolated Rails + SQLite + Vite stack — specs must stay self-contained (seed via global-setup fixtures or create your own records) and must not depend on records another spec file leaves behind.
```bash
cd packages/dashboard && pnpm test:e2e # full suite
cd packages/dashboard && pnpm test:e2e:ui # Playwright UI mode (debug)
```
+The `login(page)` helper authenticates through the API (one POST plants the refresh cookie; the SPA's boot-time silent refresh does the rest) — only `auth.spec.ts` drives the login form itself.
+
**Write UI-only assertions, like Capybara.** Drive the test through user-visible actions (fill labels, click buttons, find by role) and assert on visible UI. **Do not** reach for `page.waitForResponse(/api/...)` to wait for backend completion — it leaks API shape into tests and makes refactors painful. Playwright's `await expect(...).toBeVisible()` auto-polls until the condition is met (same as Capybara's `default_max_wait_time`), which covers virtually all cases.
```ts
diff --git a/packages/dashboard/e2e/helpers.ts b/packages/dashboard/e2e/helpers.ts
index 100658f81f..0780ae5d30 100644
--- a/packages/dashboard/e2e/helpers.ts
+++ b/packages/dashboard/e2e/helpers.ts
@@ -62,19 +62,36 @@ export function getCredentials(): E2ECredentials {
}
/**
- * Drive the admin login flow as a precondition for any spec that needs an
- * authenticated session. Returns the credentials so callers can immediately
+ * Establish an authenticated admin session as a precondition for any spec
+ * that needs one. Returns the credentials so callers can immediately
* navigate into a specific store (e.g. `/${creds.store_id}/products/options`).
*
- * Not used by `auth.spec.ts` — that spec exercises the login flow itself.
+ * Logs in through the API rather than the login form: the POST plants the
+ * refresh-token cookie in this context's cookie jar (`page.request` shares
+ * it), and the SPA's boot-time silent refresh turns that into a session on
+ * the next navigation. This skips a full SPA boot + form roundtrip per test.
+ * A shared Playwright `storageState` can't do this cheaper — refresh tokens
+ * are single-use (rotated on every refresh), so each context needs its own.
+ *
+ * Not used by `auth.spec.ts` — that spec exercises the login form itself.
*/
export async function login(page: Page): Promise<E2ECredentials> {
const creds = getCredentials()
- await page.goto('/login')
- await page.getByLabel(/email/i).fill(creds.admin_email)
- await page.getByLabel(/password/i).fill(creds.admin_password)
- await page.getByRole('button', { name: /^sign in$/i }).click()
- await expect(page).not.toHaveURL(/\/login/, { timeout: 15_000 })
+ const res = await page.request.post('/api/v3/admin/auth/login', {
+ data: { email: creds.admin_email, password: creds.admin_password },
+ })
+ if (!res.ok()) {
+ throw new Error(`API login failed with ${res.status()}: ${await res.text()}`)
+ }
+ await page.goto('/')
+ // Wait for the authenticated redirect to the store-scoped home — not just
+ // "left /login", which is trivially true at `/` while the boot refresh is
+ // still in flight. Returning early lets the spec's next `page.goto` boot a
+ // second SPA instance that refreshes with the same single-use cookie; the
+ // loser of that race gets a 401 that clears the cookie and bounces the
+ // page to /login. The redirect only fires after auth init settles, so it
+ // doubles as the refresh-complete barrier.
+ await expect(page).toHaveURL(new RegExp(`/${creds.store_id}`), { timeout: 15_000 })
return creds
}
diff --git a/packages/dashboard/playwright.config.ts b/packages/dashboard/playwright.config.ts
index dd05ca72ea..dde90af2a0 100644
--- a/packages/dashboard/playwright.config.ts
+++ b/packages/dashboard/playwright.config.ts
@@ -5,14 +5,29 @@ import { defineConfig, devices } from '@playwright/test'
const RAILS_PORT = process.env.E2E_RAILS_PORT || '3010'
const VITE_PORT = process.env.E2E_VITE_PORT || '5174'
+// `E2E_PREVIEW=1` serves the built SPA (`vite preview`) instead of the dev
+// server. Every test gets a fresh browser context with an empty HTTP cache,
+// so dev mode re-serves hundreds of on-demand-transformed modules on every
+// `page.goto` — the production bundle is a handful of files. CI builds first
+// and sets this; local runs default to `vite dev` (no build step needed).
+const PREVIEW = process.env.E2E_PREVIEW === '1'
+
export default defineConfig({
testDir: './e2e',
- // Sequential — the global Rails server + SQLite test DB are shared, and the
- // suite mutates server state (creating invitations, accepting them).
+ // Sequential per machine — the global Rails server + SQLite test DB are
+ // shared across specs, and the suite mutates server state (store settings,
+ // the admin profile, creating + accepting invitations), so concurrent
+ // workers against one backend would interfere. CI still parallelizes by
+ // sharding (`--shard=n/m`): each shard is a separate machine with its own
+ // Rails + DB + Vite stack, and specs are self-contained (fixtures come from
+ // global-setup; dynamic records use `Date.now()` suffixes).
fullyParallel: false,
workers: 1,
- reporter: process.env.CI ? 'github' : 'list',
+ // `github` gives inline PR annotations; `html` is what the failure-artifact
+ // upload ships (with the plain `github` reporter alone, playwright-report/
+ // stays empty and the artifact is useless).
+ reporter: process.env.CI ? [['github'], ['html', { open: 'never' }]] : 'list',
retries: process.env.CI ? 2 : 0,
timeout: 30_000,
expect: { timeout: 10_000 },
@@ -27,16 +42,19 @@ export default defineConfig({
projects: [{ name: 'chromium', use: { ...devices['Desktop Chrome'] } }],
// Boots Rails + the seed pipeline once before the suite starts, then Vite
- // dev pointed at that Rails. Playwright handles process lifecycle and waits
+ // pointed at that Rails. Playwright handles process lifecycle and waits
// for both to respond to a probe before any spec runs.
globalSetup: './e2e/global-setup.ts',
globalTeardown: './e2e/global-teardown.ts',
webServer: [
{
- // Vite dev — proxies `/api/*` to the test Rails per `vite.config.ts`,
+ // Vite — proxies `/api/*` to the test Rails per `vite.config.ts`,
// aimed at the test Rails port via `VITE_API_PROXY_TARGET` below.
- command: `pnpm dev --port ${VITE_PORT}`,
+ // Preview mode serves `dist/` and applies the same proxy config.
+ command: PREVIEW
+ ? `pnpm preview --port ${VITE_PORT} --strictPort`
+ : `pnpm dev --port ${VITE_PORT} --strictPort`,
url: `http://localhost:${VITE_PORT}`,
reuseExistingServer: false,
timeout: 60_000,
diff --git a/packages/dashboard/vite.config.ts b/packages/dashboard/vite.config.ts
index d9bad3a3eb..4c5486f19d 100644
--- a/packages/dashboard/vite.config.ts
+++ b/packages/dashboard/vite.config.ts
@@ -62,5 +62,14 @@ export default defineConfig(({ mode }) => {
'/rails': { target: proxyTarget, changeOrigin: true },
},
},
+ // `vite preview` inherits `server.proxy` by default, and the E2E suite
+ // depends on that when it serves the built SPA on CI — pin the proxy
+ // explicitly so a future `preview` block can't silently drop it.
+ preview: {
+ proxy: {
+ '/api': { target: proxyTarget, changeOrigin: true },
+ '/rails': { target: proxyTarget, changeOrigin: true },
+ },
+ },
}
})