Skip to content

armgit / kaneo

Public Imported from GitHub
0

⚡ Project management that doesn't become the project. Boards, Gantt charts, and GitHub integration. Self-host it or use Kaneo Cloud.

ci: share PR browser orchestration

justin0d192cb8a5ef
diff --git a/.github/workflows/browser-tests.yml b/.github/workflows/browser-tests.yml
deleted file mode 100644
index ed98b2c91..000000000
--- a/.github/workflows/browser-tests.yml
+++ /dev/null
@@ -1,111 +0,0 @@
-name: Browser tests
-
-on:
- pull_request:
-
-permissions:
- contents: read
-
-concurrency:
- group: browser-tests-${{ github.event.pull_request.number }}
- cancel-in-progress: true
-
-jobs:
- detect-pr-changes:
- permissions:
- contents: read
- pull-requests: read
- runs-on: blacksmith-4vcpu-ubuntu-2404
- timeout-minutes: 5
- outputs:
- skip_expensive: ${{ steps.classify.outputs.skip_expensive }}
- steps:
- - uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1
- with:
- persist-credentials: false
- - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
- with:
- node-version: 24.19.0
- package-manager-cache: false
- - id: classify
- name: Classify complete pull request diff
- env:
- GITHUB_TOKEN: ${{ github.token }}
- run: node scripts/ci/pr-change-classification.mjs
-
- smoke:
- needs: detect-pr-changes
- if: ${{ !cancelled() && (needs.detect-pr-changes.result != 'success' || needs.detect-pr-changes.outputs.skip_expensive != 'true') }}
- name: Chromium smoke tests
- runs-on: blacksmith-4vcpu-ubuntu-2404
- timeout-minutes: 30
- env:
- KANEO_E2E_HOST: localhost
- steps:
- - uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1
- with:
- persist-credentials: false
- - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
- with:
- version: 10.32.1
- - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
- with:
- node-version: 24.19.0
- package-manager-cache: false
- - run: pnpm --dir tests/e2e install --frozen-lockfile --ignore-scripts
-
- - run: pnpm --dir tests/e2e exec playwright install --with-deps chromium
-
- - name: Setup Blacksmith Builder
- uses: useblacksmith/setup-docker-builder@19215110ab936351210feebdfa5b440b4493e184 # v2.2.0
- with:
- cache-key: ${{ github.repository }}/kaneo
- - name: Build this revision of Kaneo
- uses: useblacksmith/build-push-action@9b0579bbec7a6cad2f171596c57e7ac1e7658850 # v2
- with:
- context: .
- file: Dockerfile.kaneo
- platforms: linux/amd64
- push: false
- load: true
- tags: kaneo-browserstack:test
-
- - name: Start disposable test instance
- run: docker compose -f tests/e2e/compose.yml up --no-build --wait --wait-timeout 180
- - name: Run browser smoke tests
- run: pnpm test:e2e
-
- - name: Collect instance logs
- if: failure()
- run: |
- mkdir -p .cache/e2e
- docker compose -f tests/e2e/compose.yml logs --no-color > .cache/e2e/instance.log
- - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
- if: always()
- with:
- name: browser-test-results
- path: .cache/e2e/
- if-no-files-found: ignore
- retention-days: 7
- - name: Stop disposable test instance
- if: always()
- run: docker compose -f tests/e2e/compose.yml down --volumes
-
- browser-result:
- if: ${{ always() }}
- needs: [detect-pr-changes, smoke]
- runs-on: blacksmith-4vcpu-ubuntu-2404
- steps:
- - uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1
- with:
- persist-credentials: false
- - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
- with:
- node-version: 24.19.0
- package-manager-cache: false
- - name: Reject failed, cancelled, and unexpected skipped checks
- env:
- CI_GATE_WORKFLOW: browser
- CI_SKIP_EXPENSIVE: ${{ needs.detect-pr-changes.outputs.skip_expensive }}
- CI_GATE_JOBS: '{"detect-pr-changes":"${{ needs.detect-pr-changes.result }}","smoke":"${{ needs.smoke.result }}"}'
- run: node scripts/ci/pr-change-classification.mjs
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 2efe0efa2..17d6fa694 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -37,6 +37,64 @@ jobs:
GITHUB_TOKEN: ${{ github.token }}
run: node scripts/ci/pr-change-classification.mjs
+ smoke:
+ needs: detect-pr-changes
+ if: ${{ !cancelled() && github.event_name == 'pull_request' && (needs.detect-pr-changes.result != 'success' || needs.detect-pr-changes.outputs.skip_expensive != 'true') }}
+ name: Chromium smoke tests
+ runs-on: blacksmith-4vcpu-ubuntu-2404
+ timeout-minutes: 30
+ env:
+ KANEO_E2E_HOST: localhost
+ steps:
+ - uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1
+ with:
+ persist-credentials: false
+ - uses: pnpm/action-setup@ea17c68df8912ef543352723c149a84f56e3d413 # v6.1.0
+ with:
+ version: 10.32.1
+ - uses: actions/setup-node@820762786026740c76f36085b0efc47a31fe5020 # v7
+ with:
+ node-version: 24.19.0
+ package-manager-cache: false
+ - run: pnpm --dir tests/e2e install --frozen-lockfile --ignore-scripts
+
+ - run: pnpm --dir tests/e2e exec playwright install --with-deps chromium
+
+ - name: Setup Blacksmith Builder
+ uses: useblacksmith/setup-docker-builder@19215110ab936351210feebdfa5b440b4493e184 # v2.2.0
+ with:
+ cache-key: ${{ github.repository }}/kaneo
+ - name: Build this revision of Kaneo
+ uses: useblacksmith/build-push-action@9b0579bbec7a6cad2f171596c57e7ac1e7658850 # v2
+ with:
+ context: .
+ file: Dockerfile.kaneo
+ platforms: linux/amd64
+ push: false
+ load: true
+ tags: kaneo-browserstack:test
+
+ - name: Start disposable test instance
+ run: docker compose -f tests/e2e/compose.yml up --no-build --wait --wait-timeout 180
+ - name: Run browser smoke tests
+ run: pnpm test:e2e
+
+ - name: Collect instance logs
+ if: failure()
+ run: |
+ mkdir -p .cache/e2e
+ docker compose -f tests/e2e/compose.yml logs --no-color > .cache/e2e/instance.log
+ - uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
+ if: always()
+ with:
+ name: browser-test-results
+ path: .cache/e2e/
+ if-no-files-found: ignore
+ retention-days: 7
+ - name: Stop disposable test instance
+ if: always()
+ run: docker compose -f tests/e2e/compose.yml down --volumes
+
peekareq-tests:
runs-on: blacksmith-4vcpu-ubuntu-2404
timeout-minutes: 5
@@ -486,7 +544,7 @@ jobs:
ci-result:
if: ${{ always() }}
- needs: [detect-pr-changes, peekareq-tests, slop-labeler-tests, quality, unit, build, integration, docker-build, workflows, storage, split-images, secret-scan]
+ needs: [detect-pr-changes, smoke, peekareq-tests, slop-labeler-tests, quality, unit, build, integration, docker-build, workflows, storage, split-images, secret-scan]
runs-on: blacksmith-4vcpu-ubuntu-2404
steps:
- uses: useblacksmith/checkout@25227e61ff9dafe400e22fa487b673eac4e4409a # v1
@@ -499,5 +557,5 @@ jobs:
- name: Reject failed, cancelled, and unexpected skipped checks
env:
CI_SKIP_EXPENSIVE: ${{ needs.detect-pr-changes.outputs.skip_expensive }}
- CI_GATE_JOBS: '{"detect-pr-changes":"${{ needs.detect-pr-changes.result }}","peekareq-tests":"${{ needs.peekareq-tests.result }}","slop-labeler-tests":"${{ needs.slop-labeler-tests.result }}","quality":"${{ needs.quality.result }}","unit":"${{ needs.unit.result }}","build":"${{ needs.build.result }}","integration":"${{ needs.integration.result }}","docker-build":"${{ needs.docker-build.result }}","workflows":"${{ needs.workflows.result }}","storage":"${{ needs.storage.result }}","split-images":"${{ needs.split-images.result }}","secret-scan":"${{ needs.secret-scan.result }}"}'
+ CI_GATE_JOBS: '{"detect-pr-changes":"${{ needs.detect-pr-changes.result }}","smoke":"${{ needs.smoke.result }}","peekareq-tests":"${{ needs.peekareq-tests.result }}","slop-labeler-tests":"${{ needs.slop-labeler-tests.result }}","quality":"${{ needs.quality.result }}","unit":"${{ needs.unit.result }}","build":"${{ needs.build.result }}","integration":"${{ needs.integration.result }}","docker-build":"${{ needs.docker-build.result }}","workflows":"${{ needs.workflows.result }}","storage":"${{ needs.storage.result }}","split-images":"${{ needs.split-images.result }}","secret-scan":"${{ needs.secret-scan.result }}"}'
run: node scripts/ci/pr-change-classification.mjs
diff --git a/scripts/ci/README.md b/scripts/ci/README.md
index 1dac10640..4feae9f9e 100644
--- a/scripts/ci/README.md
+++ b/scripts/ci/README.md
@@ -1,11 +1,11 @@
# CI checks
-`ci.yml` runs on PRs and main, and is reused by Nightly. It checks formatting,
-translations, OpenAPI, types, unit/integration tests, site links, workflow syntax,
-real S3 uploads, and all three Dockerfiles. The bundled image is loaded and run
-against disposable PostgreSQL and MinIO services for browser, realtime and
-upgrade tests. Runtime failures retain container logs, a browser screenshot and
-a Playwright trace for seven days.
+`ci.yml` runs on PRs and main, supports manual runs, and is reused by Nightly.
+It checks formatting, translations, OpenAPI, types, unit/integration tests, site links, workflow syntax, real S3 uploads, and all three Dockerfiles.
+The bundled image runs against disposable PostgreSQL and MinIO services for browser, realtime, and upgrade tests.
+Runtime failures retain container logs, a browser screenshot, and a Playwright trace for seven days.
+PRs also run the independent **Chromium smoke tests** job unless change detection confirms an allowlisted docs/site-only change.
+BrowserStack runs separately for main pushes and manual runs on main.
The disposable S3 fixture pins Bitnami's archived MinIO 2025.7.23 image by its
multi-platform digest because the upstream public registries no longer serve
@@ -25,13 +25,13 @@ the previous image. It runs the MinIO binary directly with temporary test data.
example a release commit containing `[skip ci]`), dispatch **CI** on main before
dispatching **Release**. Dry runs don't wait or publish.
-PR change detection uses the whole paginated GitHub PR-files list and verifies
-its base/head revisions and declared file count before allowing only the
-integration, storage, Docker image, and browser suites to skip for clearly
-allowlisted documentation-only changes. Any uncertainty runs the full suite.
-Both workflows have final result gates; configure these new checks as required
-in GitHub branch rules if they should block merges (workflow changes do not
-update branch protection).
+PR change detection uses the paginated GitHub PR-files list and verifies base/head revisions and the declared file count.
+Only clearly allowlisted docs/site-only PRs may skip integration, storage, Docker image, and Chromium smoke jobs.
+Uncertainty runs all PR suites.
+The PR-only smoke job skips for recognized non-PR events; other application validation still runs in full.
+`ci-result` rejects failures, cancellations, missing results, and unexpected skips.
+Require `ci-result` in branch rules to gate PRs; it replaces the retired `browser-result` check.
+Workflow changes do not configure GitHub branch rules.
The four quality checks share one job setup and dependency installation.
Each check has its own step and runs after ordinary check failures, provided installation succeeded and the job was not cancelled.
diff --git a/scripts/ci/pr-change-classification.mjs b/scripts/ci/pr-change-classification.mjs
index c019f0a52..65470c10d 100644
--- a/scripts/ci/pr-change-classification.mjs
+++ b/scripts/ci/pr-change-classification.mjs
@@ -54,18 +54,19 @@ export function classifyChanges({ eventName, files, complete, error = false }) {
return { skipExpensive: true };
}
-export function assessGate({
- workflow = "ci",
- eventName,
- skipExpensive,
- jobs,
-}) {
+export function assessGate({ eventName, skipExpensive, jobs }) {
if (!jobs || typeof jobs !== "object") return "failure";
const allowedSkipped = new Set(
skipExpensive && eventName === "pull_request"
? [...expensive, "smoke"]
: [],
);
+ if (
+ ["push", "workflow_dispatch", "workflow_call", "schedule"].includes(
+ eventName,
+ )
+ )
+ allowedSkipped.add("smoke");
for (const [name, status] of Object.entries(jobs)) {
if (
name === "secret-scan" &&
@@ -76,23 +77,21 @@ export function assessGate({
if (allowedSkipped.has(name) && status === "skipped") continue;
if (status !== "success") return "failure";
}
- const required =
- workflow === "browser"
- ? ["detect-pr-changes", "smoke"]
- : [
- "detect-pr-changes",
- "peekareq-tests",
- "slop-labeler-tests",
- "quality",
- "unit",
- "build",
- "integration",
- "docker-build",
- "workflows",
- "storage",
- "split-images",
- "secret-scan",
- ];
+ const required = [
+ "detect-pr-changes",
+ "smoke",
+ "peekareq-tests",
+ "slop-labeler-tests",
+ "quality",
+ "unit",
+ "build",
+ "integration",
+ "docker-build",
+ "workflows",
+ "storage",
+ "split-images",
+ "secret-scan",
+ ];
if (required.some((name) => !(name in jobs))) return "failure";
return "success";
}
@@ -153,7 +152,6 @@ async function main() {
if (process.env.CI_GATE_JOBS) {
const jobs = JSON.parse(process.env.CI_GATE_JOBS);
const result = assessGate({
- workflow: process.env.CI_GATE_WORKFLOW ?? "ci",
eventName: process.env.GITHUB_EVENT_NAME,
skipExpensive: process.env.CI_SKIP_EXPENSIVE === "true",
jobs,
diff --git a/scripts/ci/pr-change-classification.test.mjs b/scripts/ci/pr-change-classification.test.mjs
index a147acfa6..57e0bfd6c 100644
--- a/scripts/ci/pr-change-classification.test.mjs
+++ b/scripts/ci/pr-change-classification.test.mjs
@@ -312,132 +312,134 @@ test("non-PR events always run the full checks", () => {
}
});
-test("result gate accepts only successful jobs and deliberate expensive skips", () => {
- const ciJobs = {
- "detect-pr-changes": "success",
- "peekareq-tests": "success",
- "slop-labeler-tests": "success",
- quality: "success",
- unit: "success",
- build: "success",
+const successfulCiJobs = {
+ "detect-pr-changes": "success",
+ "peekareq-tests": "success",
+ "slop-labeler-tests": "success",
+ quality: "success",
+ unit: "success",
+ build: "success",
+ integration: "success",
+ "docker-build": "success",
+ workflows: "success",
+ storage: "success",
+ "split-images": "success",
+ "secret-scan": "success",
+ smoke: "success",
+};
+
+test("PR result gate permits only deliberate documentation suite skips", () => {
+ const jobs = {
+ ...successfulCiJobs,
integration: "skipped",
- "docker-build": "skipped",
- workflows: "success",
storage: "skipped",
+ "docker-build": "skipped",
"split-images": "skipped",
- "secret-scan": "success",
+ smoke: "skipped",
};
+ const input = { eventName: "pull_request", skipExpensive: true, jobs };
+ assert.equal(assessGate(input), "success");
+ for (const job of ["quality", "secret-scan", "detect-pr-changes", "smoke"]) {
+ for (const status of ["failure", "cancelled"]) {
+ assert.equal(
+ assessGate({ ...input, jobs: { ...jobs, [job]: status } }),
+ "failure",
+ `${job}: ${status} blocks documentation PRs`,
+ );
+ }
+ }
+ for (const job of ["quality", "secret-scan"]) {
+ assert.equal(
+ assessGate({ ...input, jobs: { ...jobs, [job]: "skipped" } }),
+ "failure",
+ );
+ }
+ assert.equal(assessGate({ ...input, skipExpensive: false }), "failure");
assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "pull_request",
- skipExpensive: true,
- jobs: ciJobs,
- }),
- "success",
- );
- assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "pull_request",
- skipExpensive: true,
- jobs: { ...ciJobs, "secret-scan": "skipped" },
- }),
+ assessGate({ ...input, jobs: { quality: "success" } }),
"failure",
);
- for (const state of ["failure", "cancelled", "skipped"]) {
+});
+
+test("runtime PR result gate requires smoke success and a present result", () => {
+ const input = {
+ eventName: "pull_request",
+ skipExpensive: false,
+ jobs: successfulCiJobs,
+ };
+ assert.equal(assessGate(input), "success");
+ for (const job of [
+ "smoke",
+ "integration",
+ "storage",
+ "docker-build",
+ "split-images",
+ ]) {
+ for (const status of ["failure", "cancelled", "skipped"]) {
+ assert.equal(
+ assessGate({ ...input, jobs: { ...successfulCiJobs, [job]: status } }),
+ "failure",
+ `${job}: ${status} blocks runtime PRs`,
+ );
+ }
+ }
+ const withoutSmoke = { ...successfulCiJobs };
+ delete withoutSmoke.smoke;
+ assert.equal(assessGate({ ...input, jobs: withoutSmoke }), "failure");
+});
+
+test("non-PR result gate allows smoke skips but keeps runtime suites required", () => {
+ for (const [eventName, skippedSecretResult] of [
+ ["push", "failure"],
+ ["workflow_dispatch", "failure"],
+ ["workflow_call", "failure"],
+ ["schedule", "success"],
+ ]) {
+ const jobs = { ...successfulCiJobs, smoke: "skipped" };
+ const input = { eventName, skipExpensive: true, jobs };
+ assert.equal(assessGate(input), "success", eventName);
assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "pull_request",
- skipExpensive: true,
- jobs: { ...ciJobs, quality: state },
- }),
- "failure",
+ assessGate({ ...input, jobs: { ...jobs, "secret-scan": "skipped" } }),
+ skippedSecretResult,
+ `${eventName}: scheduled secret-scan exception stays narrow`,
);
+ for (const status of ["failure", "cancelled"]) {
+ assert.equal(
+ assessGate({ ...input, jobs: { ...jobs, smoke: status } }),
+ "failure",
+ );
+ }
+ for (const job of [
+ "integration",
+ "storage",
+ "docker-build",
+ "split-images",
+ ]) {
+ assert.equal(
+ assessGate({ ...input, jobs: { ...jobs, [job]: "skipped" } }),
+ "failure",
+ );
+ }
+ const withoutSmoke = { ...jobs };
+ delete withoutSmoke.smoke;
+ assert.equal(assessGate({ ...input, jobs: withoutSmoke }), "failure");
+ }
+});
+
+test("unknown event context does not authorize a smoke skip", () => {
+ for (const eventName of [undefined, "unknown-event"]) {
assert.equal(
assessGate({
- workflow: "ci",
- eventName: "pull_request",
- skipExpensive: false,
- jobs: { ...ciJobs, integration: state },
+ eventName,
+ skipExpensive: true,
+ jobs: { ...successfulCiJobs, smoke: "skipped" },
}),
"failure",
);
}
- assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "push",
- skipExpensive: false,
- jobs: ciJobs,
- }),
- "failure",
- );
- const scheduled = {
- ...ciJobs,
- integration: "success",
- storage: "success",
- "docker-build": "success",
- "split-images": "success",
- "secret-scan": "skipped",
- };
- assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "schedule",
- skipExpensive: false,
- jobs: scheduled,
- }),
- "success",
- );
- assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "schedule",
- skipExpensive: false,
- jobs: { ...scheduled, integration: "skipped" },
- }),
- "failure",
- );
- assert.equal(
- assessGate({
- workflow: "ci",
- eventName: "pull_request",
- skipExpensive: true,
- jobs: { quality: "success" },
- }),
- "failure",
- );
- assert.equal(
- assessGate({
- workflow: "browser",
- eventName: "pull_request",
- skipExpensive: true,
- jobs: { "detect-pr-changes": "success", smoke: "skipped" },
- }),
- "success",
- );
});
test("result gate requires successful quality checks on every CI event", () => {
- const jobs = {
- "detect-pr-changes": "success",
- "peekareq-tests": "success",
- "slop-labeler-tests": "success",
- quality: "success",
- unit: "success",
- build: "success",
- integration: "success",
- "docker-build": "success",
- workflows: "success",
- storage: "success",
- "split-images": "success",
- "secret-scan": "success",
- };
- const withoutQuality = { ...jobs };
- delete withoutQuality.quality;
-
for (const eventName of [
"pull_request",
"push",
@@ -445,8 +447,14 @@ test("result gate requires successful quality checks on every CI event", () => {
"workflow_call",
"schedule",
]) {
- const input = { workflow: "ci", eventName, skipExpensive: false, jobs };
+ const jobs = {
+ ...successfulCiJobs,
+ smoke: eventName === "pull_request" ? "success" : "skipped",
+ };
+ const input = { eventName, skipExpensive: false, jobs };
assert.equal(assessGate(input), "success", `${eventName}: all checks pass`);
+ const withoutQuality = { ...jobs };
+ delete withoutQuality.quality;
assert.equal(
assessGate({ ...input, jobs: withoutQuality }),
"failure",
diff --git a/tests/e2e/README.md b/tests/e2e/README.md
index 94b0ce332..8601fe4d4 100644
--- a/tests/e2e/README.md
+++ b/tests/e2e/README.md
@@ -79,10 +79,10 @@ videos appear in the Automate dashboard under **Kaneo**.
3. Every push to `main` now runs **BrowserStack** automatically. To rerun it,
open **Actions → BrowserStack → Run workflow**, selecting `main`.
-**Browser tests** runs the same suite in Chromium for every pull request, including
-forks, without cloud credentials. **BrowserStack** runs Chrome on Windows and
-WebKit on macOS after a push to `main`, or a manual dispatch on `main`. Cloud runs
-are serialized to stay within the two-session limit; obsolete PR runs are cancelled.
+The CI workflow runs the independent **Chromium smoke tests** job for PRs, including forks, without cloud credentials.
+It skips that job only for qualifying docs/site-only PRs.
+**BrowserStack** runs Chrome on Windows and WebKit on macOS after a push to `main` or a manual dispatch on `main`.
+Cloud runs are serialized to stay within the two-session limit; obsolete PR runs are cancelled.
Both workflows build the checked-out revision, use a disposable database, upload
reports for seven days, and tear down the stack even after failure. BrowserStack
@@ -90,8 +90,10 @@ credentials are passed only to its credential check and cloud test step. No
production instance is contacted. Tests use synthetic accounts; invitation setup
uses the normal API without sending email because SMTP is not configured.
-Add the **Chromium smoke tests** check to branch protection when it has proven
-stable. The cloud check runs after merge and does not currently gate releases.
+**Chromium smoke tests** keeps its display name for external check consumers.
+Require `ci-result` for PR gating; the former `browser-result` gate is retired.
+Workflow edits do not configure GitHub branch rules.
+The cloud check runs after merge and does not currently gate releases.
References: