Skip to content

The easy-to-use open source Business Intelligence and Embedded Analytics tool that lets everyone work with data :bar_chart:

Add E2E (questions) tests on top of the cross-version automation (#70190)

Nemanja Glumacddba65c3680d
diff --git a/.github/workflows/cross-version.yml b/.github/workflows/cross-version.yml
index 0ca02c9ed76..e056799aaa1 100644
--- a/.github/workflows/cross-version.yml
+++ b/.github/workflows/cross-version.yml
@@ -61,6 +61,12 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
+ - name: Prepare front-end environment
+ uses: ./.github/actions/prepare-frontend
+
+ - name: Prepare Cypress environment
+ uses: ./.github/actions/prepare-cypress
+
- name: Run cross-version test
env:
CURRENT_VERSION: ${{ vars.CURRENT_VERSION }}
@@ -68,6 +74,16 @@ jobs:
cd cross-version
./test.sh --source "${{ matrix.source }}" --target "${{ matrix.target }}"
+ - name: Upload Cypress Artifacts upon failure
+ uses: actions/upload-artifact@v4
+ if: failure()
+ with:
+ name: cypress-recording-${{ matrix.source }}-${{ matrix.target }}
+ path: |
+ ./cypress
+ ./logs/test.log
+ if-no-files-found: ignore
+
- name: Show logs on failure
if: failure()
run: |
@@ -87,6 +103,12 @@ jobs:
- name: Prepare release scripts
uses: ./.github/actions/prepare-release-scripts
+ - name: Prepare front-end environment
+ uses: ./.github/actions/prepare-frontend
+
+ - name: Prepare Cypress environment
+ uses: ./.github/actions/prepare-cypress
+
- name: Run cross-version test
env:
CURRENT_VERSION: ${{ vars.CURRENT_VERSION }}
@@ -94,6 +116,16 @@ jobs:
cd cross-version
./test.sh --source "${{ inputs.source_version }}" --target "${{ inputs.target_version }}"
+ - name: Upload Cypress Artifacts upon failure
+ uses: actions/upload-artifact@v4
+ if: failure()
+ with:
+ name: cypress-recording-${{ inputs.source_version }}-${{ inputs.target_version }}
+ path: |
+ ./cypress
+ ./logs/test.log
+ if-no-files-found: ignore
+
- name: Show logs on failure
if: failure()
run: |
diff --git a/cross-version/.gitignore b/cross-version/.gitignore
new file mode 100644
index 00000000000..5374f16071a
--- /dev/null
+++ b/cross-version/.gitignore
@@ -0,0 +1 @@
+.xv-*
diff --git a/cross-version/README.md b/cross-version/README.md
index a5511869d9b..3546e6a3abf 100644
--- a/cross-version/README.md
+++ b/cross-version/README.md
@@ -1,55 +1,70 @@
# Cross-Version Migration Testing
-Tests Metabase database migrations across version upgrades and downgrades.
+Tests that Metabase data survives database migrations across version upgrades and downgrades.
-## Quick Start
+Uses Cypress e2e tests to create data on a **source** version, then verify it's intact on a **target** version after migration. The application database is PostgreSQL (matching production), and state is preserved between versions via a Docker volume.
+
+## Scripts
+
+### `test.sh` — CI / production runs
+
+Runs the full migration test end-to-end. Starts source, runs `@source` specs, stops it, starts target, runs `@target` specs.
+
+```bash
+./test.sh --source v1.57.6 --target v1.58.3 # upgrade
+./test.sh --source v1.58.3 --target v1.57.6 # downgrade
+```
+
+We only test Enterprise Edition (`v1.x`) for now. Direction is inferred from version comparison.
+
+For **downgrades**, the script expects the target to refuse startup, then runs cascading `migrate down` (one step per major version) before retrying.
+
+### `dev.sh` — local development
+
+Starts a single Metabase version in Docker with H2 and opens Cypress interactively. Snapshot/restore works via `/api/testing/*` endpoints, controlled by the `CROSS_VERSION_DEV_MODE` env var.
```bash
-# Test upgrade (OSS)
-./test.sh --source v0.58.6 --target v0.58.7
+./dev.sh --version v1.57.6
+./dev.sh --version v1.58.3 --port 3001
+```
+
+## E2E test specs
-# Test downgrade (OSS)
-./test.sh --source v0.58.7 --target v0.58.6
+Specs live in `e2e/cross-version/` and are organized by major version:
-# Test Enterprise edition (use v1.x.x)
-./test.sh --source v1.58.6 --target v1.58.7
+```
+e2e/cross-version/
+ 56/ # specs for v56
+ 57/ # specs for v57
+ latest/ # specs for HEAD / newest versions
```
-Edition is inferred from version prefix: `v0.x.x` = OSS, `v1.x.x` = EE.
+Each spec file uses `@source` and `@target` tags (via `cypress-grep`) to separate setup from verification. Each folder also contains a `helpers.ts` with version-specific utility functions used by the specs.
-## Requirements
+### When to create a new versioned folder
-- Docker
-- Docker Compose v2
-- Bun (for version helpers CLI)
+The `latest/` folder is the working copy — it targets the current development branch and is used for local dev with `dev.sh`. When a Metabase release introduces a breaking change that makes existing specs fail on older versions (e.g., a renamed UI element, a changed API response), copy `latest/` into a new numbered folder (e.g., `58/`). That snapshot covers all future versions until the next breaking change requires a new folder.
-## How It Works
+In short: versioned folders are frozen snapshots, `latest/` is the living version.
-### Upgrade Test
-1. Start PostgreSQL container
-2. Start SOURCE Metabase version, wait for healthy
-3. Stop SOURCE container (PostgreSQL persists)
-4. Start TARGET version - migrations run automatically
-5. Verify health check passes
+### Spec resolution
-### Downgrade Test
-1. Start PostgreSQL container
-2. Start SOURCE (newer) Metabase version, wait for healthy
-3. Stop SOURCE container
-4. Start TARGET (older) version - expects refusal
-5. Run `migrate down` command
-6. Start TARGET version again
-7. Verify health check passes
+Both `test.sh` and `dev.sh` resolve which spec folder to use for a given version:
-## Environment Variables
+1. **Exact match** — `e2e/cross-version/{major}/` if it exists
+2. **Closest newer** — the lowest numbered folder >= the major version
+3. **Fallback** — `e2e/cross-version/latest/`
+
+## Environment variables
| Variable | Default | Description |
|----------|---------|-------------|
-| `METABASE_PORT` | 3000 | Port to expose Metabase |
-| `HEALTH_TIMEOUT` | 120 | Seconds to wait for health check |
+| `METABASE_PORT` | `3000` (`3077` in dev) | Port to expose Metabase |
+| `HEALTH_TIMEOUT` | `120` | Seconds to wait for health check |
+| `CROSS_VERSION_DEV_MODE` | — | Set automatically by `dev.sh`; enables snapshot/restore helpers in specs |
-## CI
+## Requirements
-The GitHub workflow (`.github/workflows/cross-version.yml`) supports:
-- **Manual trigger**: Specify source/target versions
-- **Nightly schedule**: Runs predefined version matrix
+- Docker and Docker Compose v2
+- Bun (for `cli.ts` version helpers)
+- Node.js with `bunx` (for Cypress)
diff --git a/cross-version/dev.sh b/cross-version/dev.sh
new file mode 100755
index 00000000000..a8dae9d95c6
--- /dev/null
+++ b/cross-version/dev.sh
@@ -0,0 +1,167 @@
+#!/usr/bin/env bash
+#
+# Interactive dev environment for writing cross-version e2e tests.
+#
+# Starts an older Metabase version in Docker with H2 and opens Cypress.
+# Uses H2 with a shared volume so snapshot/restore works via test endpoints.
+#
+# Resolves which spec folder to use:
+# e2e/cross-version/{major}/ if it exists, otherwise e2e/cross-version/latest/
+#
+# Usage:
+# ./dev.sh --version v1.57.6
+# ./dev.sh --version v1.58.7 --port 3001
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
+
+VERSION=""
+PORT="${METABASE_PORT:-3077}"
+
+RED='\033[0;31m'
+GREEN='\033[0;32m'
+NC='\033[0m'
+
+log() { echo -e "${GREEN}[xv-dev]${NC} $*"; }
+error() { echo -e "${RED}[xv-dev]${NC} $*" >&2; }
+
+usage() {
+ cat <<EOF
+Usage: $0 --version VERSION [--port PORT]
+
+Options:
+ --version VERSION Metabase version to run (e.g., v1.57.6, v1.58.7)
+ --port PORT Port to expose (default: 3077)
+ --help Show this help
+
+Examples:
+ $0 --version v1.57.6
+ $0 --version v1.58.7 --port 3001
+EOF
+ exit 1
+}
+
+while [[ $# -gt 0 ]]; do
+ case $1 in
+ --version) VERSION="$2"; shift 2 ;;
+ --port) PORT="$2"; shift 2 ;;
+ --help) usage ;;
+ *) error "Unknown option: $1"; usage ;;
+ esac
+done
+
+if [[ -z "$VERSION" ]]; then
+ error "--version is required"
+ usage
+fi
+
+# CLI wrapper for version helpers
+cli() {
+ bun "$SCRIPT_DIR/cli.ts" "$@"
+}
+
+IMAGE=$(cli image "$VERSION")
+MAJOR=$(cli major "$VERSION")
+H2_DIR="$SCRIPT_DIR/.xv-h2/v${MAJOR}"
+LOG_FILE="$SCRIPT_DIR/.xv-metabase.log"
+
+# Resolve spec folder: exact match, then closest older-version folder, then latest
+SPECS_DIR="$REPO_ROOT/e2e/cross-version/${MAJOR}"
+if [[ ! -d "$SPECS_DIR" ]]; then
+ closest=""
+ closest_dist=999
+ for dir in "$REPO_ROOT"/e2e/cross-version/[0-9]*/; do
+ v=$(basename "$dir")
+ (( v < MAJOR )) && continue
+ dist=$(( v - MAJOR ))
+ if (( dist < closest_dist )); then
+ closest_dist=$dist
+ closest="$dir"
+ fi
+ done
+ if [[ -n "$closest" ]]; then
+ SPECS_DIR="${closest%/}"
+ else
+ SPECS_DIR="$REPO_ROOT/e2e/cross-version/latest"
+ fi
+fi
+
+log "============================================"
+log "Cross-Version Dev Environment"
+log "============================================"
+log "Version: $VERSION"
+log "Image: $IMAGE"
+log "Port: $PORT"
+log "H2 dir: $H2_DIR"
+log "Logs: $LOG_FILE"
+log "Specs: $SPECS_DIR"
+log "============================================"
+
+# Clean up previous session
+rm -f "$LOG_FILE"
+
+# Start with a clean app db every time
+# H2 files are owned by the Docker container's user, so use Docker to clean up
+if [[ -d "$H2_DIR" ]]; then
+ docker run --rm -v "$(dirname "$H2_DIR"):/data" alpine rm -rf "/data/$(basename "$H2_DIR")"
+fi
+mkdir -p "$H2_DIR"
+
+cleanup() {
+ log "Stopping Metabase..."
+ docker rm -f xv-dev-metabase 2>/dev/null || true
+}
+
+trap cleanup EXIT
+
+# Remove any leftover container from a previous run (e.g., after SIGKILL)
+docker rm -f xv-dev-metabase 2>/dev/null || true
+
+log "Starting Metabase ${VERSION}..."
+docker run -d \
+ --name xv-dev-metabase \
+ -p "${PORT}:3000" \
+ -v "${H2_DIR}:/metabase.db" \
+ -e MB_ENABLE_TEST_ENDPOINTS=true \
+ -e MB_DANGEROUS_UNSAFE_ENABLE_TESTING_H2_CONNECTIONS_DO_NOT_ENABLE=true \
+ "$IMAGE"
+
+# Stream backend logs to file for debugging (docker logs xv-dev-metabase)
+docker logs -f xv-dev-metabase > "$LOG_FILE" 2>&1 &
+
+# Create snapshots dir inside the container for /api/testing/snapshot and /api/testing/restore
+docker exec xv-dev-metabase sh -c "mkdir -p /e2e/snapshots && chmod 777 /e2e/snapshots"
+
+log "Waiting for Metabase to be ready..."
+TIMEOUT=120
+START=$(date +%s)
+while true; do
+ ELAPSED=$(( $(date +%s) - START ))
+ if (( ELAPSED >= TIMEOUT )); then
+ error "Timed out after ${TIMEOUT}s"
+ docker logs xv-dev-metabase
+ exit 1
+ fi
+ if curl -sf "http://localhost:${PORT}/api/health" 2>/dev/null | grep -q '"ok"'; then
+ break
+ fi
+ sleep 2
+done
+log "Metabase is ready at http://localhost:${PORT}"
+
+log "Saving blank snapshot..."
+curl -sf -X POST "http://localhost:${PORT}/api/testing/snapshot/blank"
+
+log ""
+log "Opening Cypress..."
+cd "$REPO_ROOT"
+
+CROSS_VERSION_DEV_MODE=true \
+CYPRESS_BASE_URL="http://localhost:${PORT}" \
+ bunx cypress open \
+ --e2e \
+ --config-file "e2e/cross-version/cypress.config.js" \
+ --config "specPattern=${SPECS_DIR}/**/*.cy.spec.ts"
diff --git a/cross-version/test.sh b/cross-version/test.sh
index fa4de40a9c6..c4375d0d8d6 100755
--- a/cross-version/test.sh
+++ b/cross-version/test.sh
@@ -11,6 +11,7 @@
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
cd "$SCRIPT_DIR"
# Defaults
@@ -66,6 +67,41 @@ cli() {
bun "$SCRIPT_DIR/cli.ts" "$@"
}
+run_e2e() {
+ local phase="$1"
+ local version="$2"
+ local major specs_dir
+
+ major=$(cli major "$version")
+ specs_dir="$REPO_ROOT/e2e/cross-version/${major}"
+ if [[ ! -d "$specs_dir" ]]; then
+ local closest="" closest_dist=999
+ for dir in "$REPO_ROOT"/e2e/cross-version/[0-9]*/; do
+ local v=$(basename "$dir")
+ (( v < major )) && continue
+ local dist=$(( v - major ))
+ if (( dist < closest_dist )); then
+ closest_dist=$dist
+ closest="$dir"
+ fi
+ done
+ if [[ -n "$closest" ]]; then
+ specs_dir="${closest%/}"
+ log "No exact specs for v${major} — using closest: ${specs_dir##*/}/"
+ else
+ specs_dir="$REPO_ROOT/e2e/cross-version/latest"
+ log "No exact specs for v${major} — falling back to latest/"
+ fi
+ else
+ log "Found exact specs for v${major}"
+ fi
+
+ log "Running @${phase} e2e tests for ${version} from: e2e/cross-version/${specs_dir##*/}/"
+ CYPRESS_SPEC_PATTERN="${specs_dir}/**/*.cy.spec.ts" \
+ "$REPO_ROOT/e2e/cross-version/run.sh" --phase "$phase"
+}
+
+
wait_for_health() {
local timeout="$1"
local start_time=$(date +%s)
@@ -325,11 +361,15 @@ main() {
log "✅ SOURCE version ($SOURCE_VERSION) is healthy"
log ""
- log "Step 2: Stopping SOURCE version ($SOURCE_VERSION)..."
+ log "Step 2: Running e2e tests (@source)..."
+ run_e2e source "$SOURCE_VERSION"
+
+ log ""
+ log "Step 3: Stopping SOURCE version ($SOURCE_VERSION)..."
stop_metabase
log ""
- log "Step 3: Starting TARGET version ($TARGET_VERSION)..."
+ log "Step 4: Starting TARGET version ($TARGET_VERSION)..."
start_metabase "$target_image"
if [[ "$direction" == "upgrade" ]]; then
@@ -341,6 +381,10 @@ main() {
fi
log "✅ UPGRADE successful - TARGET version ($TARGET_VERSION) is healthy"
+ log ""
+ log "Step 5: Running e2e tests (@target)..."
+ run_e2e target "$TARGET_VERSION"
+
else
# Downgrade: should refuse to start, then we run migrate down
if ! check_downgrade_refused; then
@@ -353,9 +397,7 @@ main() {
stop_metabase
log ""
- log "============================================"
- log "Step 4: Rolling back database ($SOURCE_VERSION → $TARGET_VERSION)..."
- log "============================================"
+ log "Step 5: Rolling back database ($SOURCE_VERSION → $TARGET_VERSION)..."
if ! cascading_migrate_down "$SOURCE_VERSION" "$TARGET_VERSION"; then
error "❌ migrate down failed"
exit 1
@@ -363,7 +405,7 @@ main() {
# Try starting again
log ""
- log "Step 5: Starting TARGET version ($TARGET_VERSION) after migrate down..."
+ log "Step 6: Starting TARGET version ($TARGET_VERSION) after migrate down..."
start_metabase "$target_image"
if ! wait_for_health "$HEALTH_TIMEOUT"; then
@@ -372,6 +414,10 @@ main() {
exit 1
fi
log "✅ DOWNGRADE successful - TARGET version ($TARGET_VERSION) is healthy after migrate down"
+
+ log ""
+ log "Step 7: Running e2e tests (@target)..."
+ run_e2e target "$TARGET_VERSION"
fi
log ""
diff --git a/e2e/cross-version/57/00-setup.cy.spec.ts b/e2e/cross-version/57/00-setup.cy.spec.ts
new file mode 100644
index 00000000000..018740c996f
--- /dev/null
+++ b/e2e/cross-version/57/00-setup.cy.spec.ts
@@ -0,0 +1,99 @@
+/**
+ * Cross-version test: Instance setup
+ *
+ * This must be the first test to run. It completes the Metabase setup wizard
+ * in the source phase, and verifies the admin can authenticate in both phases.
+ *
+ * Uses credentials from e2e/support/cypress_data.js so cy.signInAsAdmin() works.
+ */
+
+import { USERS } from "e2e/support/cypress_data";
+
+const { email: ADMIN_EMAIL, password: ADMIN_PASSWORD } = USERS.admin;
+
+const { H } = cy;
+
+describe("Cross-version: Instance setup", () => {
+ it("setup: completes the setup wizard", { tags: ["@source"] }, () => {
+ H.restoreCrossVersionDev("blank");
+ cy.visit("/");
+
+ cy.request("GET", "/api/session/properties").then(({ body }) => {
+ const setupToken = body["setup-token"];
+
+ cy.log("Setup token must exist on a fresh instance");
+ expect(setupToken).to.be.a("string");
+
+ cy.request("POST", "/api/setup", {
+ token: setupToken,
+ user: {
+ email: ADMIN_EMAIL,
+ password: ADMIN_PASSWORD,
+ first_name: "Admin",
+ last_name: "CrossVersion",
+ },
+ prefs: {
+ site_name: "CrossVersion Test",
+ site_locale: "en",
+ allow_tracking: false,
+ },
+ }).then((response) => {
+ expect(response.status).to.eq(200);
+ });
+
+ cy.request("GET", "/api/user/current").then((response) => {
+ expect(response.status).to.eq(200);
+ expect(response.body.email).to.eq(ADMIN_EMAIL);
+
+ cy.log("Dismiss `it's ok to play around` modal for admin");
+ cy.request("PUT", `/api/user/${response.body.id}/modal/qbnewb`);
+ });
+
+ H.withSampleDatabase(({ ORDERS, PRODUCTS, REVIEWS }) => {
+ cy.log("Remap display values to use foreign key as Product Title");
+ cy.request("POST", `/api/field/${ORDERS.PRODUCT_ID}/dimension`, {
+ name: "Product ID",
+ type: "external",
+ human_readable_field_id: PRODUCTS.TITLE,
+ });
+
+ cy.log("Remap rating to use custom values");
+ cy.request("POST", `/api/field/${REVIEWS.RATING}/dimension`, {
+ type: "internal",
+ name: "Rating",
+ });
+
+ cy.request("POST", `/api/field/${REVIEWS.RATING}/values`, {
+ values: [
+ [1, "Awful"],
+ [2, "Unpleasant"],
+ [3, "Meh"],
+ [4, "Enjoyable"],
+ [5, "Perfecto"],
+ ],
+ });
+ });
+ });
+ });
+
+ it("verify: admin can authenticate", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/browse/databases");
+ cy.findByTestId("database-browser").contains("Sample Database").click();
+
+ cy.log("Verify REVIEWS.Rating values display custom remapping");
+ cy.findAllByRole("link").filter(":contains(Reviews)").click();
+ cy.findAllByRole("gridcell")
+ .should("contain", "Enjoyable")
+ .and("contain", "Perfecto");
+
+ cy.go("back");
+ cy.log("Verify ORDERS.ProductId FK is remapped to Product Title");
+ cy.findAllByRole("link").filter(":contains(Orders)").click();
+ cy.findAllByRole("gridcell")
+ .should("contain", "Fantastic Wool Shirt")
+ .and("contain", "Small Marble Hat");
+ H.snapshotCrossVersionDev("00-complete");
+ });
+});
diff --git a/e2e/cross-version/57/01-pivot.cy.spec.ts b/e2e/cross-version/57/01-pivot.cy.spec.ts
new file mode 100644
index 00000000000..8e96342effc
--- /dev/null
+++ b/e2e/cross-version/57/01-pivot.cy.spec.ts
@@ -0,0 +1,73 @@
+import { Q1_NAME } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - pivot", () => {
+ it("setup: creates a pivot table", { tags: ["@source"] }, () => {
+ H.restoreCrossVersionDev("00-complete");
+ cy.signIn("admin", { skipCache: true });
+
+ cy.log("-- Create a pivot table --");
+ X.startQuestionFromTable("People");
+
+ cy.log(
+ "-- Add filter: Narrow down the states that start with K (only two) --",
+ );
+ H.getNotebookStep("filter")
+ .findByText("Add filters to narrow your answer")
+ .click();
+ X.selectFromPopover("State");
+ cy.findByLabelText("Filter operator").click();
+ X.selectFromPopover("Starts with");
+ cy.findByLabelText("Filter value").type("K").blur();
+ cy.button("Add filter").click();
+
+ cy.log("-- Add aggregation --");
+ H.addSummaryField({ metric: "Count of rows" });
+
+ cy.log("-- Add breakouts --");
+ H.addSummaryGroupingField({ field: "State" });
+ H.addSummaryGroupingField({ field: "Source" });
+ H.addSummaryGroupingField({ field: "Created At" });
+
+ H.visualize();
+
+ cy.log("-- Render as pivot table --");
+ cy.intercept("POST", "/api/dataset/pivot").as("pivot");
+ H.openVizTypeSidebar();
+ H.vizTypeSidebar().within(() => {
+ cy.findAllByRole("option").filter(":contains(Pivot Table)").click();
+ cy.wait("@pivot");
+ cy.button("Done").click();
+ });
+
+ H.vizTypeSidebar().should("not.exist");
+ X.assertRowCount("155");
+
+ X.saveQuestion(Q1_NAME);
+ });
+
+ it("verify: pivot table is preserved", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+
+ cy.log(`-- ${Q1_NAME}: Assert that the pivot table viz is preserved --`);
+ cy.findAllByTestId("collection-entry-name")
+ .filter(`:contains(${Q1_NAME})`)
+ .click();
+
+ X.assertRowCount("155");
+
+ cy.findAllByTestId("pivot-table-cell")
+ .should("contain", "State")
+ .should("contain", "Created At: Month")
+ .and("contain", "Organic")
+ .and("contain", "Row totals")
+ .and("contain", "KS");
+
+ H.snapshotCrossVersionDev("01-complete");
+ });
+});
diff --git a/e2e/cross-version/57/02-joins.cy.spec.ts b/e2e/cross-version/57/02-joins.cy.spec.ts
new file mode 100644
index 00000000000..43be00c3ccf
--- /dev/null
+++ b/e2e/cross-version/57/02-joins.cy.spec.ts
@@ -0,0 +1,75 @@
+import { Q2_NAME } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - joins", () => {
+ it("setup: creates a joined question", { tags: ["@source"] }, () => {
+ H.restoreCrossVersionDev("01-complete");
+ cy.signIn("admin", { skipCache: true });
+
+ cy.log("-- Create a joined question --");
+
+ cy.visit("/");
+ H.newButton("Question").click();
+
+ X.joinTables("Orders", "Products");
+
+ cy.log("-- Filter on the joined table --");
+ H.getNotebookStep("filter")
+ .findByText("Add filters to narrow your answer")
+ .click();
+ H.popover().within(() => {
+ cy.findByRole("heading", { name: "Products" }).click();
+ cy.findByLabelText("Price").click();
+ });
+
+ H.popover().within(() => {
+ cy.findByLabelText("Filter operator").should("have.text", "Between");
+ cy.findByPlaceholderText("Min").type("0");
+ cy.findByPlaceholderText("Min").type("50").blur();
+ cy.button("Add filter").click();
+ });
+
+ cy.log("-- Add aggregation --");
+ H.addSummaryField({ metric: "Average of ..." });
+ X.selectFromPopover("Discount");
+
+ cy.log("-- Add breakouts --");
+ H.addSummaryGroupingField({ table: "Products", field: "Category" });
+
+ cy.log("-- Sort the columns --");
+ cy.findByLabelText("Sort").click();
+ X.selectFromPopover("Average of Discount");
+
+ H.visualize();
+ X.assertRowCount("4");
+
+ X.saveQuestion(Q2_NAME);
+ });
+
+ it("verify: bar chart is preserved", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+ cy.findAllByTestId("collection-entry-name")
+ .filter(`:contains(${Q2_NAME})`)
+ .click();
+
+ X.assertRowCount("4");
+
+ cy.log("-- Assert that there are four bars --");
+ H.chartPathWithFillColor("#A989C5").should("have.length", 4);
+ H.echartsContainer()
+ .find("text")
+ .should("contain", "Average of Discount")
+ .and("contain", "Products → Category")
+ .and("contain", "Doohickey")
+ .and("contain", "Gizmo")
+ .and("contain", "$0")
+ .and("contain", "$5.00");
+
+ H.snapshotCrossVersionDev("02-complete");
+ });
+});
diff --git a/e2e/cross-version/57/03-nested-model.cy.spec.ts b/e2e/cross-version/57/03-nested-model.cy.spec.ts
new file mode 100644
index 00000000000..7ee063a3a22
--- /dev/null
+++ b/e2e/cross-version/57/03-nested-model.cy.spec.ts
@@ -0,0 +1,90 @@
+import { Q2_NAME, Q3_NAME } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - nested model", () => {
+ it(
+ "setup: creates a model based on another question",
+ { tags: ["@source"] },
+ () => {
+ H.restoreCrossVersionDev("02-complete");
+ cy.signIn("admin", { skipCache: true });
+
+ cy.log("-- Create a model from a previous question --");
+
+ cy.visit("/");
+ H.newButton("Question").click();
+
+ H.modal().within(() => {
+ cy.findAllByRole("tab")
+ .should("be.visible")
+ .filter(":contains(Collections)")
+ .click();
+ });
+ cy.findAllByTestId("picker-item").filter(`:contains(${Q2_NAME})`).click();
+
+ cy.log("-- Add aggregation --");
+ H.addSummaryField({ metric: "Sum of ..." });
+ X.selectFromPopover("Average of Discount");
+
+ H.visualize();
+ cy.findByTestId("scalar-value").should("have.text", "$20.80");
+
+ X.saveQuestion(Q3_NAME);
+
+ cy.log("-- Turn question into a model --");
+ H.openQuestionActions();
+ X.selectFromPopover("Turn into a model");
+ H.modal().button("Turn this into a model").click();
+ cy.location("pathname").should("contain", "model");
+
+ cy.log("-- Edit model metadata --");
+ H.openQuestionActions();
+ X.selectFromPopover("Edit metadata");
+ H.waitForLoaderToBeRemoved();
+ cy.location("pathname").should("include", "columns");
+
+ H.renameColumn("Sum of Average of Discount", "Total Discount");
+
+ H.rightSidebar().findByDisplayValue("US Dollar").click();
+ X.selectFromPopover("Euro");
+ H.rightSidebar().findByDisplayValue("Euro").should("be.visible");
+
+ H.saveMetadataChanges();
+ cy.location("pathname").should("not.include", "columns");
+
+ cy.findByTestId("scalar-value").should("have.text", "€20.80");
+ },
+ );
+
+ it(
+ "verify: model custom metadata is preserved",
+ { tags: ["@target"] },
+ () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+
+ H.getPinnedSection()
+ .findByRole("link")
+ .should("contain", Q3_NAME)
+ .click();
+
+ cy.findByTestId("question-row-count").should(
+ "have.text",
+ "Showing 1 row",
+ );
+
+ cy.log("-- Assert that the model metadata is preserved --");
+ cy.findByTestId("scalar-value").should("have.text", "€20.80");
+ cy.findByLabelText("Switch to data").click();
+
+ cy.findByTestId("scalar-value").should("not.exist");
+ cy.findByTestId("header-cell").should("contain", "Total Discount (€)");
+
+ H.snapshotCrossVersionDev("03-complete");
+ },
+ );
+});
diff --git a/e2e/cross-version/57/04-sql.cy.spec.ts b/e2e/cross-version/57/04-sql.cy.spec.ts
new file mode 100644
index 00000000000..e6a35784f25
--- /dev/null
+++ b/e2e/cross-version/57/04-sql.cy.spec.ts
@@ -0,0 +1,123 @@
+import { Q4_SQL_NAME, SQL_QUERY } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - sql", () => {
+ it(
+ "setup: creates a sql question with a snippet and a field filter variable",
+ { tags: ["@source"] },
+ () => {
+ H.restoreCrossVersionDev("03-complete");
+ cy.signIn("admin", { setupCache: true });
+
+ cy.request("POST", "/api/native-query-snippet", {
+ content: "REVIEWS.BODY IS NOT NULL",
+ name: "Body Not Null",
+ });
+
+ cy.visit("/");
+ H.newButton("SQL query").click();
+
+ cy.log("-- Create a SQL question --");
+ H.NativeEditor.type(SQL_QUERY, {
+ focus: true,
+ allowFastSet: true,
+ }).blur();
+
+ cy.log("-- Map the variable to a field filter --");
+ cy.findByTestId("variable-type-select").click();
+ X.selectFromPopover("Field Filter");
+ H.popover().within(() => {
+ cy.findByRole("heading", { name: "Sample Database" }).should(
+ "be.visible",
+ );
+ cy.findByText("Reviews").click();
+ });
+ X.selectFromPopover("Rating");
+
+ H.NativeEditor.clickOnRun();
+ X.assertRowCount("1,112");
+
+ cy.log("-- Hide the PK column --");
+ cy.findByTestId("visualization-root")
+ .findByTestId("table-header")
+ .contains("PRODUCT_ID")
+ .click();
+ H.popover().icon("eye_crossed_out").click();
+
+ cy.log("-- Assert on the preview values in a table --");
+ cy.findByTestId("visualization-root").within(() => {
+ cy.findByTestId("table-header")
+ .should("contain", "REVIEWER")
+ .and("contain", "RATING")
+ .and("not.contain", "PRODUCT_ID");
+
+ cy.findByTestId("table-body")
+ .should("contain", "xavier")
+ .and("contain", "4")
+ .and("contain", "clement")
+ .and("contain", "5");
+ });
+
+ cy.log("-- Assert that the filter is mapped to a correct value --");
+ cy.findByTestId("parameter-widget").should("contain", "Rating");
+
+ X.saveQuestion(Q4_SQL_NAME);
+ },
+ );
+
+ it("verify: sql question can be filtered", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+
+ cy.findAllByTestId("collection-entry-name")
+ .filter(`:contains(${Q4_SQL_NAME})`)
+ .click();
+
+ cy.log("-- Assert that the columns are preserved --");
+ X.assertRowCount("1,112");
+ cy.findByTestId("visualization-root").within(() => {
+ cy.findByTestId("table-header")
+ .should("contain", "REVIEWER")
+ .and("contain", "RATING")
+ .and("not.contain", "PRODUCT_ID");
+
+ cy.findByTestId("table-body")
+ .should("contain", "xavier")
+ .and("contain", "4")
+ .and("contain", "clement")
+ .and("contain", "5");
+ });
+
+ cy.log("-- Apply the filter --");
+ cy.findByTestId("native-query-editor-container").within(() => {
+ cy.findByText("This question is written in SQL.").should("be.visible");
+
+ cy.findByTestId("parameter-widget").should("contain", "Rating").click();
+ });
+ H.popover().within(() => {
+ cy.findByText("Perfecto").click();
+ cy.findByText("Meh").click();
+
+ cy.button("Add filter").click();
+ });
+
+ cy.findByTestId("qb-header").findByLabelText("Get Answer").click();
+
+ cy.log("-- Assert that the filter values are correctly applied --");
+ X.assertRowCount("442");
+ cy.findByTestId("parameter-widget")
+ .should("contain", "Rating")
+ .and("contain", "2 selections");
+ cy.findByTestId("table-body")
+ .and("contain", "clement")
+ .and("contain", "5")
+ .should("contain", "clement")
+ .and("contain", "3")
+ .should("not.contain", "xavier")
+ .and("not.contain", "4");
+ });
+});
diff --git a/e2e/cross-version/57/helpers.ts b/e2e/cross-version/57/helpers.ts
new file mode 100644
index 00000000000..bc88185964a
--- /dev/null
+++ b/e2e/cross-version/57/helpers.ts
@@ -0,0 +1,49 @@
+const { H } = cy;
+
+export function startQuestionFromTable(table: string) {
+ cy.visit("/");
+ H.newButton("Question").click();
+ H.modal().contains(table).should("be.visible").click();
+}
+
+export function saveQuestion(name: string) {
+ cy.log(`-- Save question: ${name}`);
+ cy.intercept("POST", "/api/card").as("saveQuestion");
+ cy.findByTestId("qb-header").button("Save").click();
+ cy.findByTestId("save-question-modal").within(() => {
+ cy.findByLabelText("Name").clear().type(name);
+ cy.button("Save").click();
+ cy.wait("@saveQuestion");
+ });
+}
+
+export function assertRowCount(count: string) {
+ cy.findByTestId("question-row-count").should(
+ "have.text",
+ `Showing ${count} rows`,
+ );
+}
+
+export function selectFromPopover(item: string) {
+ H.popover().contains(item).click();
+}
+
+export function joinTables(baseTable: string, joinTable: string) {
+ H.modal().within(() => {
+ cy.findAllByRole("tab")
+ .should("be.visible")
+ .filter(":contains(Tables)")
+ .click();
+ cy.findAllByTestId("picker-item").filter(`:contains(${baseTable})`).click();
+ });
+
+ H.join();
+
+ H.modal().within(() => {
+ cy.findAllByRole("tab")
+ .should("be.visible")
+ .filter(":contains(Tables)")
+ .click();
+ cy.findAllByTestId("picker-item").filter(`:contains(${joinTable})`).click();
+ });
+}
diff --git a/e2e/cross-version/constants.ts b/e2e/cross-version/constants.ts
new file mode 100644
index 00000000000..98c03ad4f09
--- /dev/null
+++ b/e2e/cross-version/constants.ts
@@ -0,0 +1,10 @@
+export const Q1_NAME = "cv1-pivot";
+export const Q2_NAME = "cv2-joins";
+export const Q3_NAME = "cv3-nested";
+export const Q4_SQL_NAME = "cv4-sql";
+
+export const SQL_QUERY = `
+SELECT PRODUCT_ID, REVIEWER, RATING
+FROM REVIEWS
+WHERE {{snippet: Body Not Null}}
+AND {{rating}}`;
diff --git a/e2e/cross-version/cypress.config.js b/e2e/cross-version/cypress.config.js
new file mode 100644
index 00000000000..ac3d31a5d8a
--- /dev/null
+++ b/e2e/cross-version/cypress.config.js
@@ -0,0 +1,45 @@
+const { defineConfig } = require("cypress");
+
+const { defaultConfig } = require("../support/config");
+
+const isDevMode = !!process.env.CROSS_VERSION_DEV_MODE;
+const viewportWidth = 1600;
+const viewportHeight = 1200;
+
+module.exports = defineConfig({
+ e2e: {
+ ...defaultConfig,
+ specPattern: "e2e/cross-version/latest/**/*.cy.spec.ts",
+ viewportWidth,
+ viewportHeight,
+ expose: {
+ ...defaultConfig.expose,
+ CROSS_VERSION_DEV_MODE: isDevMode,
+ },
+ setupNodeEvents(on, config) {
+ // Call the default setupNodeEvents first
+ defaultConfig.setupNodeEvents(on, config);
+
+ // Set browser window size for headless mode to match viewport
+ on("before:browser:launch", (browser, launchOptions) => {
+ if (
+ (browser.name === "chrome" || browser.name === "chromium") &&
+ browser.isHeadless
+ ) {
+ launchOptions.args.push(
+ `--window-size=${viewportWidth},${viewportHeight}`,
+ );
+ }
+
+ if (browser.name === "electron" && browser.isHeadless) {
+ launchOptions.preferences.width = viewportWidth;
+ launchOptions.preferences.height = viewportHeight;
+ }
+
+ return launchOptions;
+ });
+
+ return config;
+ },
+ },
+});
diff --git a/e2e/cross-version/latest/00-setup.cy.spec.ts b/e2e/cross-version/latest/00-setup.cy.spec.ts
new file mode 100644
index 00000000000..9bcf8678c68
--- /dev/null
+++ b/e2e/cross-version/latest/00-setup.cy.spec.ts
@@ -0,0 +1,100 @@
+/**
+ * Cross-version test: Instance setup
+ *
+ * This must be the first test to run. It completes the Metabase setup wizard
+ * in the source phase, and verifies the admin can authenticate in both phases.
+ *
+ * Uses credentials from e2e/support/cypress_data.js so cy.signInAsAdmin() works.
+ */
+
+import { USERS } from "e2e/support/cypress_data";
+
+const { email: ADMIN_EMAIL, password: ADMIN_PASSWORD } = USERS.admin;
+
+const { H } = cy;
+
+describe("Cross-version: Instance setup", () => {
+ it("setup: completes the setup wizard", { tags: ["@source"] }, () => {
+ H.restoreCrossVersionDev("blank");
+ cy.visit("/");
+
+ cy.request("GET", "/api/session/properties").then(({ body }) => {
+ const setupToken = body["setup-token"];
+
+ cy.log("Setup token must exist on a fresh instance");
+ expect(setupToken).to.be.a("string");
+
+ cy.request("POST", "/api/setup", {
+ token: setupToken,
+ user: {
+ email: ADMIN_EMAIL,
+ password: ADMIN_PASSWORD,
+ first_name: "Admin",
+ last_name: "CrossVersion",
+ },
+ prefs: {
+ site_name: "CrossVersion Test",
+ site_locale: "en",
+ allow_tracking: false,
+ },
+ }).then((response) => {
+ expect(response.status).to.eq(200);
+ });
+
+ cy.request("GET", "/api/user/current").then((response) => {
+ expect(response.status).to.eq(200);
+ expect(response.body.email).to.eq(ADMIN_EMAIL);
+
+ cy.log("Dismiss `it's ok to play around` modal for admin");
+ cy.request("PUT", `/api/user/${response.body.id}/modal/qbnewb`);
+ });
+
+ H.withSampleDatabase(({ ORDERS, PRODUCTS, REVIEWS }) => {
+ cy.log("Remap display values to use foreign key as Product Title");
+ cy.request("POST", `/api/field/${ORDERS.PRODUCT_ID}/dimension`, {
+ name: "Product ID",
+ type: "external",
+ human_readable_field_id: PRODUCTS.TITLE,
+ });
+
+ cy.log("Remap rating to use custom values");
+ cy.request("POST", `/api/field/${REVIEWS.RATING}/dimension`, {
+ type: "internal",
+ name: "Rating",
+ });
+
+ cy.request("POST", `/api/field/${REVIEWS.RATING}/values`, {
+ values: [
+ [1, "Awful"],
+ [2, "Unpleasant"],
+ [3, "Meh"],
+ [4, "Enjoyable"],
+ [5, "Perfecto"],
+ ],
+ });
+ });
+ });
+ });
+
+ it("verify: admin can authenticate", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/browse/databases");
+ cy.findByTestId("database-browser").contains("Sample Database").click();
+
+ cy.log("Verify REVIEWS.Rating values display custom remapping");
+ cy.findAllByRole("link").filter(":contains(Reviews)").click();
+ cy.findAllByRole("gridcell")
+ .should("contain", "Enjoyable")
+ .and("contain", "Perfecto");
+
+ cy.go("back");
+ cy.log("Verify ORDERS.ProductId FK is remapped to Product Title");
+ cy.findAllByRole("link").filter(":contains(Orders)").click();
+ cy.findAllByRole("gridcell")
+ .should("contain", "Fantastic Wool Shirt")
+ .and("contain", "Small Marble Hat");
+
+ H.snapshotCrossVersionDev("00-complete");
+ });
+});
diff --git a/e2e/cross-version/latest/01-pivot.cy.spec.ts b/e2e/cross-version/latest/01-pivot.cy.spec.ts
new file mode 100644
index 00000000000..8e96342effc
--- /dev/null
+++ b/e2e/cross-version/latest/01-pivot.cy.spec.ts
@@ -0,0 +1,73 @@
+import { Q1_NAME } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - pivot", () => {
+ it("setup: creates a pivot table", { tags: ["@source"] }, () => {
+ H.restoreCrossVersionDev("00-complete");
+ cy.signIn("admin", { skipCache: true });
+
+ cy.log("-- Create a pivot table --");
+ X.startQuestionFromTable("People");
+
+ cy.log(
+ "-- Add filter: Narrow down the states that start with K (only two) --",
+ );
+ H.getNotebookStep("filter")
+ .findByText("Add filters to narrow your answer")
+ .click();
+ X.selectFromPopover("State");
+ cy.findByLabelText("Filter operator").click();
+ X.selectFromPopover("Starts with");
+ cy.findByLabelText("Filter value").type("K").blur();
+ cy.button("Add filter").click();
+
+ cy.log("-- Add aggregation --");
+ H.addSummaryField({ metric: "Count of rows" });
+
+ cy.log("-- Add breakouts --");
+ H.addSummaryGroupingField({ field: "State" });
+ H.addSummaryGroupingField({ field: "Source" });
+ H.addSummaryGroupingField({ field: "Created At" });
+
+ H.visualize();
+
+ cy.log("-- Render as pivot table --");
+ cy.intercept("POST", "/api/dataset/pivot").as("pivot");
+ H.openVizTypeSidebar();
+ H.vizTypeSidebar().within(() => {
+ cy.findAllByRole("option").filter(":contains(Pivot Table)").click();
+ cy.wait("@pivot");
+ cy.button("Done").click();
+ });
+
+ H.vizTypeSidebar().should("not.exist");
+ X.assertRowCount("155");
+
+ X.saveQuestion(Q1_NAME);
+ });
+
+ it("verify: pivot table is preserved", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+
+ cy.log(`-- ${Q1_NAME}: Assert that the pivot table viz is preserved --`);
+ cy.findAllByTestId("collection-entry-name")
+ .filter(`:contains(${Q1_NAME})`)
+ .click();
+
+ X.assertRowCount("155");
+
+ cy.findAllByTestId("pivot-table-cell")
+ .should("contain", "State")
+ .should("contain", "Created At: Month")
+ .and("contain", "Organic")
+ .and("contain", "Row totals")
+ .and("contain", "KS");
+
+ H.snapshotCrossVersionDev("01-complete");
+ });
+});
diff --git a/e2e/cross-version/latest/02-joins.cy.spec.ts b/e2e/cross-version/latest/02-joins.cy.spec.ts
new file mode 100644
index 00000000000..43be00c3ccf
--- /dev/null
+++ b/e2e/cross-version/latest/02-joins.cy.spec.ts
@@ -0,0 +1,75 @@
+import { Q2_NAME } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - joins", () => {
+ it("setup: creates a joined question", { tags: ["@source"] }, () => {
+ H.restoreCrossVersionDev("01-complete");
+ cy.signIn("admin", { skipCache: true });
+
+ cy.log("-- Create a joined question --");
+
+ cy.visit("/");
+ H.newButton("Question").click();
+
+ X.joinTables("Orders", "Products");
+
+ cy.log("-- Filter on the joined table --");
+ H.getNotebookStep("filter")
+ .findByText("Add filters to narrow your answer")
+ .click();
+ H.popover().within(() => {
+ cy.findByRole("heading", { name: "Products" }).click();
+ cy.findByLabelText("Price").click();
+ });
+
+ H.popover().within(() => {
+ cy.findByLabelText("Filter operator").should("have.text", "Between");
+ cy.findByPlaceholderText("Min").type("0");
+ cy.findByPlaceholderText("Min").type("50").blur();
+ cy.button("Add filter").click();
+ });
+
+ cy.log("-- Add aggregation --");
+ H.addSummaryField({ metric: "Average of ..." });
+ X.selectFromPopover("Discount");
+
+ cy.log("-- Add breakouts --");
+ H.addSummaryGroupingField({ table: "Products", field: "Category" });
+
+ cy.log("-- Sort the columns --");
+ cy.findByLabelText("Sort").click();
+ X.selectFromPopover("Average of Discount");
+
+ H.visualize();
+ X.assertRowCount("4");
+
+ X.saveQuestion(Q2_NAME);
+ });
+
+ it("verify: bar chart is preserved", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+ cy.findAllByTestId("collection-entry-name")
+ .filter(`:contains(${Q2_NAME})`)
+ .click();
+
+ X.assertRowCount("4");
+
+ cy.log("-- Assert that there are four bars --");
+ H.chartPathWithFillColor("#A989C5").should("have.length", 4);
+ H.echartsContainer()
+ .find("text")
+ .should("contain", "Average of Discount")
+ .and("contain", "Products → Category")
+ .and("contain", "Doohickey")
+ .and("contain", "Gizmo")
+ .and("contain", "$0")
+ .and("contain", "$5.00");
+
+ H.snapshotCrossVersionDev("02-complete");
+ });
+});
diff --git a/e2e/cross-version/latest/03-nested-model.cy.spec.ts b/e2e/cross-version/latest/03-nested-model.cy.spec.ts
new file mode 100644
index 00000000000..07ad93ba2fe
--- /dev/null
+++ b/e2e/cross-version/latest/03-nested-model.cy.spec.ts
@@ -0,0 +1,85 @@
+import { Q2_NAME, Q3_NAME } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - nested model", () => {
+ it(
+ "setup: creates a model based on another question",
+ { tags: ["@source"] },
+ () => {
+ H.restoreCrossVersionDev("02-complete");
+ cy.signIn("admin", { skipCache: true });
+
+ cy.log("-- Create a model from a previous question --");
+
+ cy.visit("/");
+ H.newButton("Question").click();
+
+ X.selectFromPopover("Our analytics");
+ X.selectFromPopover(Q2_NAME);
+
+ cy.log("-- Add aggregation --");
+ H.addSummaryField({ metric: "Sum of ..." });
+ X.selectFromPopover("Average of Discount");
+
+ H.visualize();
+ cy.findByTestId("scalar-value").should("have.text", "$20.80");
+
+ X.saveQuestion(Q3_NAME);
+
+ cy.log("-- Turn question into a model --");
+ H.openQuestionActions();
+ X.selectFromPopover("Turn into a model");
+ H.modal().button("Turn this into a model").click();
+ cy.location("pathname").should("contain", "model");
+
+ cy.log("-- Edit model metadata --");
+ H.openQuestionActions();
+ X.selectFromPopover("Edit metadata");
+ H.waitForLoaderToBeRemoved();
+ cy.location("pathname").should("include", "columns");
+
+ H.renameColumn("Sum of Average of Discount", "Total Discount");
+
+ H.rightSidebar().findByDisplayValue("US Dollar").click();
+ X.selectFromPopover("Euro");
+ H.rightSidebar().findByDisplayValue("Euro").should("be.visible");
+
+ H.saveMetadataChanges();
+ cy.location("pathname").should("not.include", "columns");
+
+ cy.findByTestId("scalar-value").should("have.text", "€20.80");
+ },
+ );
+
+ it(
+ "verify: model custom metadata is preserved",
+ { tags: ["@target"] },
+ () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+
+ H.getPinnedSection()
+ .findByRole("link")
+ .should("contain", Q3_NAME)
+ .click();
+
+ cy.findByTestId("question-row-count").should(
+ "have.text",
+ "Showing 1 row",
+ );
+
+ cy.log("-- Assert that the model metadata is preserved --");
+ cy.findByTestId("scalar-value").should("have.text", "€20.80");
+ cy.findByLabelText("Switch to data").click();
+
+ cy.findByTestId("scalar-value").should("not.exist");
+ cy.findByTestId("header-cell").should("contain", "Total Discount (€)");
+
+ H.snapshotCrossVersionDev("03-complete");
+ },
+ );
+});
diff --git a/e2e/cross-version/latest/04-sql.cy.spec.ts b/e2e/cross-version/latest/04-sql.cy.spec.ts
new file mode 100644
index 00000000000..e6a35784f25
--- /dev/null
+++ b/e2e/cross-version/latest/04-sql.cy.spec.ts
@@ -0,0 +1,123 @@
+import { Q4_SQL_NAME, SQL_QUERY } from "../constants";
+
+import * as X from "./helpers";
+
+const { H } = cy;
+
+describe("Cross-version questions - sql", () => {
+ it(
+ "setup: creates a sql question with a snippet and a field filter variable",
+ { tags: ["@source"] },
+ () => {
+ H.restoreCrossVersionDev("03-complete");
+ cy.signIn("admin", { setupCache: true });
+
+ cy.request("POST", "/api/native-query-snippet", {
+ content: "REVIEWS.BODY IS NOT NULL",
+ name: "Body Not Null",
+ });
+
+ cy.visit("/");
+ H.newButton("SQL query").click();
+
+ cy.log("-- Create a SQL question --");
+ H.NativeEditor.type(SQL_QUERY, {
+ focus: true,
+ allowFastSet: true,
+ }).blur();
+
+ cy.log("-- Map the variable to a field filter --");
+ cy.findByTestId("variable-type-select").click();
+ X.selectFromPopover("Field Filter");
+ H.popover().within(() => {
+ cy.findByRole("heading", { name: "Sample Database" }).should(
+ "be.visible",
+ );
+ cy.findByText("Reviews").click();
+ });
+ X.selectFromPopover("Rating");
+
+ H.NativeEditor.clickOnRun();
+ X.assertRowCount("1,112");
+
+ cy.log("-- Hide the PK column --");
+ cy.findByTestId("visualization-root")
+ .findByTestId("table-header")
+ .contains("PRODUCT_ID")
+ .click();
+ H.popover().icon("eye_crossed_out").click();
+
+ cy.log("-- Assert on the preview values in a table --");
+ cy.findByTestId("visualization-root").within(() => {
+ cy.findByTestId("table-header")
+ .should("contain", "REVIEWER")
+ .and("contain", "RATING")
+ .and("not.contain", "PRODUCT_ID");
+
+ cy.findByTestId("table-body")
+ .should("contain", "xavier")
+ .and("contain", "4")
+ .and("contain", "clement")
+ .and("contain", "5");
+ });
+
+ cy.log("-- Assert that the filter is mapped to a correct value --");
+ cy.findByTestId("parameter-widget").should("contain", "Rating");
+
+ X.saveQuestion(Q4_SQL_NAME);
+ },
+ );
+
+ it("verify: sql question can be filtered", { tags: ["@target"] }, () => {
+ cy.signIn("admin", { skipCache: true });
+
+ cy.visit("/collection/root");
+
+ cy.findAllByTestId("collection-entry-name")
+ .filter(`:contains(${Q4_SQL_NAME})`)
+ .click();
+
+ cy.log("-- Assert that the columns are preserved --");
+ X.assertRowCount("1,112");
+ cy.findByTestId("visualization-root").within(() => {
+ cy.findByTestId("table-header")
+ .should("contain", "REVIEWER")
+ .and("contain", "RATING")
+ .and("not.contain", "PRODUCT_ID");
+
+ cy.findByTestId("table-body")
+ .should("contain", "xavier")
+ .and("contain", "4")
+ .and("contain", "clement")
+ .and("contain", "5");
+ });
+
+ cy.log("-- Apply the filter --");
+ cy.findByTestId("native-query-editor-container").within(() => {
+ cy.findByText("This question is written in SQL.").should("be.visible");
+
+ cy.findByTestId("parameter-widget").should("contain", "Rating").click();
+ });
+ H.popover().within(() => {
+ cy.findByText("Perfecto").click();
+ cy.findByText("Meh").click();
+
+ cy.button("Add filter").click();
+ });
+
+ cy.findByTestId("qb-header").findByLabelText("Get Answer").click();
+
+ cy.log("-- Assert that the filter values are correctly applied --");
+ X.assertRowCount("442");
+ cy.findByTestId("parameter-widget")
+ .should("contain", "Rating")
+ .and("contain", "2 selections");
+ cy.findByTestId("table-body")
+ .and("contain", "clement")
+ .and("contain", "5")
+ .should("contain", "clement")
+ .and("contain", "3")
+ .should("not.contain", "xavier")
+ .and("not.contain", "4");
+ });
+});
diff --git a/e2e/cross-version/latest/helpers.ts b/e2e/cross-version/latest/helpers.ts
new file mode 100644
index 00000000000..a352e97ad9e
--- /dev/null
+++ b/e2e/cross-version/latest/helpers.ts
@@ -0,0 +1,40 @@
+const { H } = cy;
+
+export function startQuestionFromTable(table: string) {
+ cy.visit("/");
+ H.newButton("Question").click();
+ selectFromPopover("Sample Database");
+ selectFromPopover(table);
+}
+
+export function saveQuestion(name: string) {
+ cy.log(`-- Save question: ${name}`);
+ cy.intercept("POST", "/api/card").as("saveQuestion");
+ cy.findByTestId("qb-header").button("Save").click();
+ cy.findByTestId("save-question-modal").within(() => {
+ cy.findByLabelText("Name").clear().type(name);
+ cy.button("Save").click();
+ cy.wait("@saveQuestion");
+ });
+}
+
+export function assertRowCount(count: string) {
+ cy.findByTestId("question-row-count").should(
+ "have.text",
+ `Showing ${count} rows`,
+ );
+}
+
+export function selectFromPopover(item: string) {
+ H.popover().contains(item).should("be.visible").click();
+}
+
+export function joinTables(baseTable: string, joinTable: string) {
+ selectFromPopover("Sample Database");
+ selectFromPopover(baseTable);
+
+ H.join();
+
+ selectFromPopover("Sample Database");
+ selectFromPopover(joinTable);
+}
diff --git a/e2e/cross-version/run.sh b/e2e/cross-version/run.sh
new file mode 100755
index 00000000000..ceb91d0e4ac
--- /dev/null
+++ b/e2e/cross-version/run.sh
@@ -0,0 +1,54 @@
+#!/usr/bin/env bash
+#
+# Run cross-version e2e tests
+#
+# Called by cross-version/test.sh - assumes Metabase is already running.
+#
+# Usage:
+# ./run.sh --phase source # Run setup tests (creates data)
+# ./run.sh --phase target # Run verification tests (checks data survived)
+#
+# Environment variables:
+# CYPRESS_BASE_URL - Metabase instance URL (default: http://localhost:3000)
+#
+
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/../.." && pwd)"
+
+PHASE=""
+
+usage() {
+ echo "Usage: $0 --phase <source|target>"
+ exit 1
+}
+
+while [[ $# -gt 0 ]]; do
+ case $1 in
+ --phase) PHASE="$2"; shift 2 ;;
+ -h|--help) usage ;;
+ *) echo "Unknown option: $1"; usage ;;
+ esac
+done
+
+[[ -z "$PHASE" ]] && { echo "Error: --phase is required"; usage; }
+[[ "$PHASE" != "source" && "$PHASE" != "target" ]] && { echo "Error: --phase must be 'source' or 'target'"; usage; }
+
+cd "$PROJECT_ROOT"
+
+export CYPRESS_BASE_URL="${CYPRESS_BASE_URL:-http://localhost:3000}"
+
+CYPRESS_ARGS=(
+ --config-file "e2e/cross-version/cypress.config.js"
+ --env "grepTags=@${PHASE}"
+)
+
+if [[ -n "${CYPRESS_SPEC_PATTERN:-}" ]]; then
+ echo "[cypress] Running @${PHASE} tests from ${CYPRESS_SPEC_PATTERN}"
+ CYPRESS_ARGS+=(--config "specPattern=${CYPRESS_SPEC_PATTERN}")
+else
+ echo "[cypress] Running @${PHASE} tests against ${CYPRESS_BASE_URL}"
+fi
+
+bunx cypress run "${CYPRESS_ARGS[@]}"
diff --git a/e2e/support/helpers/e2e-setup-helpers.js b/e2e/support/helpers/e2e-setup-helpers.js
index 1f1e3139563..6517ce1de2c 100644
--- a/e2e/support/helpers/e2e-setup-helpers.js
+++ b/e2e/support/helpers/e2e-setup-helpers.js
@@ -4,6 +4,26 @@ export function snapshot(name) {
cy.request("POST", `/api/testing/snapshot/${name}`);
}
+/**
+ * Take a snapshot during cross-version test development.
+ *
+ * This is a no-op in CI / production runs. It only takes effect when
+ * the `CROSS_VERSION_DEV_MODE` env var is exposed through Cypress,
+ * allowing developers to incrementally snapshot state while iterating
+ * on cross-version test scenarios locally.
+ *
+ * @param {string} name - snapshot identifier
+ */
+export function snapshotCrossVersionDev(name) {
+ if (!Cypress.expose?.("CROSS_VERSION_DEV_MODE")) {
+ cy.log(
+ `skipping cross-version snapshot "${name}" — not running in dev mode (production runs against Postgres)`,
+ );
+ return;
+ }
+ cy.request("POST", `/api/testing/snapshot/${name}`);
+}
+
/**
*
* @param { |
@@ -30,3 +50,24 @@ export function restore(name = "default") {
return cy.request("POST", `/api/testing/restore/${name}`);
}
+
+/**
+ * Restore a snapshot during cross-version test development.
+ *
+ * This is a no-op in CI / production runs. It only takes effect when
+ * the `CROSS_VERSION_DEV_MODE` env var is exposed through Cypress,
+ * allowing developers to restore a previously-saved snapshot while
+ * iterating on cross-version test scenarios locally.
+ *
+ * @param {string} [name="blank"] - snapshot identifier to restore
+ */
+export function restoreCrossVersionDev(name = "blank") {
+ if (!Cypress.expose?.("CROSS_VERSION_DEV_MODE")) {
+ cy.log(
+ `skipping cross-version restore "${name}" — not running in dev mode (production runs against Postgres)`,
+ );
+ return;
+ }
+
+ return cy.request("POST", `/api/testing/restore/${name}`);
+}
diff --git a/e2e/support/helpers/e2e-viz-settings-helpers.js b/e2e/support/helpers/e2e-viz-settings-helpers.js
index a43b3f0587f..ea6698e26a5 100644
--- a/e2e/support/helpers/e2e-viz-settings-helpers.js
+++ b/e2e/support/helpers/e2e-viz-settings-helpers.js
@@ -28,6 +28,10 @@ export function openVizSettingsSidebar() {
cy.findByTestId("viz-settings-button").click();
}
+export function vizTypeSidebar() {
+ return cy.findByTestId("chart-type-sidebar");
+}
+
export function vizSettingsSidebar() {
return cy.findByTestId("chartsettings-sidebar");
}