Skip to content

High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.

perf(ci): move the multi-page audit runs out of the PR path into the extended tier

Luigi Agostiebbbcbcdee66
diff --git a/TESTING.md b/TESTING.md
index 6443f3e5..8e823834 100644
--- a/TESTING.md
+++ b/TESTING.md
@@ -77,10 +77,13 @@ The map reads product paths as well as test paths:
| any other `*.sh` under `tests/e2e/scenarios/api/`, `cli/` or `infra/` | that group's extended suite |
| `Dockerfile`, `.dockerignore`, `scripts/docker-*smoke.sh`, the smoke Dockerfile and workflows | `smoke` |
| audit implementation — `internal/audit/`, `pkg/pinchtabaudit/`, the audit handlers, `internal/cli/actions/actions_audit*`, `cmd/pinchtab/cmd_audit*` | `api-extended`, `cli-extended` |
+| compare implementation — `internal/cli/actions/actions_compare*`, `cmd/pinchtab/cmd_compare*` (`internal/audit/compare.go` is already audit) | `cli-extended` |
Covering a new product area is one line in the map. `go test ./internal/devtools/` drives the script over synthetic file lists, so a mapping is verified by running it rather than by reading the YAML. Extended coverage the map does not claim still runs on `workflow_dispatch` only.
-Audit is also declared there as an *enrolled area*: every tracked audit source file must escalate the audit suites, with a short exclusion list for files that only share the word (`internal/authn/audit.go` is security audit logging). So a new file in an enrolled family cannot quietly miss the map — which is how the CLI audit implementation went unenrolled while its cobra wrapper was covered. Enrolling a second area is one entry in `enrolledAreas`.
+Audit and compare are also declared there as *enrolled areas*: every tracked source file matching an area must escalate that area's suites, with a short exclusion list for files that only share the word (`internal/authn/audit.go` is security audit logging). So a new file in an enrolled family cannot quietly miss the map — which is how the CLI audit implementation went unenrolled while its cobra wrapper was covered. Enrolling another area is one entry in `enrolledAreas`.
+
+The multi-page audit runs — `api/audit-extended.sh`, `cli/audit-cli-extended.sh`, `cli/audit-seaportal-extended.sh`, `cli/compare-extended.sh` — sit in the extended tier rather than the basic one, so a pull request pays for them only when it touches audit or compare. The PR path keeps the single-page audit signal: `api/audit-page-basic.sh`, `api/a11y-audit-basic.sh`, `api/audit-fixtures-basic.sh`, `cli/audit-auth-basic.sh` and `cli/audit-report-basic.sh`.
### Basic Suites
diff --git a/internal/devtools/e2e_escalation_test.go b/internal/devtools/e2e_escalation_test.go
index db872b5b..92b0eb84 100644
--- a/internal/devtools/e2e_escalation_test.go
+++ b/internal/devtools/e2e_escalation_test.go
@@ -37,9 +37,14 @@ func TestEscalationFollowsProductPathsAndTestPaths(t *testing.T) {
},
{
name: "audit implementation alongside a basic scenario",
- changed: []string{"internal/audit/enrich.go", "tests/e2e/scenarios/api/audit-basic.sh"},
+ changed: []string{"internal/audit/enrich.go", "tests/e2e/scenarios/api/audit-page-basic.sh"},
want: []string{"run_api_extended", "run_cli_extended"},
},
+ {
+ name: "compare implementation, which no API suite covers",
+ changed: []string{"internal/cli/actions/actions_compare.go", "cmd/pinchtab/cmd_compare.go"},
+ want: []string{"run_cli_extended"},
+ },
{
name: "api extended scenario churn",
changed: []string{"tests/e2e/scenarios/api/tabs-extended.sh"},
@@ -241,6 +246,11 @@ var enrolledAreas = []enrolledArea{
"internal/authn/audit.go": "security audit logging — AuditLog writes slog entries, a different sense of the word, covered by no audit scenario",
},
},
+ {
+ name: "compare",
+ match: `(?i)compare`,
+ suites: []string{"run_cli_extended"},
+ },
}
func TestEveryEnrolledAreaIsCoveredByTheMap(t *testing.T) {
diff --git a/scripts/ci/e2e-escalation.map b/scripts/ci/e2e-escalation.map
index ba192f81..502bb1dc 100644
--- a/scripts/ci/e2e-escalation.map
+++ b/scripts/ci/e2e-escalation.map
@@ -20,3 +20,5 @@
^internal/handlers/(a11y_)?audit api-extended cli-extended
^internal/cli/actions/actions_audit api-extended cli-extended
^cmd/pinchtab/cmd_audit api-extended cli-extended
+^internal/cli/actions/actions_compare cli-extended
+^cmd/pinchtab/cmd_compare cli-extended
diff --git a/tests/e2e/scenarios/api/audit-basic.sh b/tests/e2e/scenarios/api/audit-extended.sh
similarity index 97%
rename from tests/e2e/scenarios/api/audit-basic.sh
rename to tests/e2e/scenarios/api/audit-extended.sh
index 024baa3f..61f1c29f 100755
--- a/tests/e2e/scenarios/api/audit-basic.sh
+++ b/tests/e2e/scenarios/api/audit-extended.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-# audit-basic.sh — POST /audit multi-page site audit runs.
+# audit-extended.sh — POST /audit multi-page site audit runs.
GROUP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${GROUP_DIR}/../../helpers/api.sh"
diff --git a/tests/e2e/scenarios/cli/audit-cli-basic.sh b/tests/e2e/scenarios/cli/audit-cli-extended.sh
similarity index 97%
rename from tests/e2e/scenarios/cli/audit-cli-basic.sh
rename to tests/e2e/scenarios/cli/audit-cli-extended.sh
index 8dddd1ea..16466ed5 100755
--- a/tests/e2e/scenarios/cli/audit-cli-basic.sh
+++ b/tests/e2e/scenarios/cli/audit-cli-extended.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-# audit-cli-basic.sh — pinchtab audit CLI happy path.
+# audit-cli-extended.sh — pinchtab audit CLI happy path.
GROUP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${GROUP_DIR}/../../helpers/cli.sh"
diff --git a/tests/e2e/scenarios/cli/audit-seaportal-basic.sh b/tests/e2e/scenarios/cli/audit-seaportal-extended.sh
similarity index 97%
rename from tests/e2e/scenarios/cli/audit-seaportal-basic.sh
rename to tests/e2e/scenarios/cli/audit-seaportal-extended.sh
index 7edba3d4..f06e537d 100755
--- a/tests/e2e/scenarios/cli/audit-seaportal-basic.sh
+++ b/tests/e2e/scenarios/cli/audit-seaportal-extended.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-# audit-seaportal-basic.sh — pinchtab audit --seaportal-report ingestion.
+# audit-seaportal-extended.sh — pinchtab audit --seaportal-report ingestion.
GROUP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${GROUP_DIR}/../../helpers/cli.sh"
diff --git a/tests/e2e/scenarios/cli/compare-basic.sh b/tests/e2e/scenarios/cli/compare-extended.sh
similarity index 97%
rename from tests/e2e/scenarios/cli/compare-basic.sh
rename to tests/e2e/scenarios/cli/compare-extended.sh
index 87b235d2..65f729b9 100755
--- a/tests/e2e/scenarios/cli/compare-basic.sh
+++ b/tests/e2e/scenarios/cli/compare-extended.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-# compare-basic.sh — pinchtab compare live-vs-staging happy path.
+# compare-extended.sh — pinchtab compare live-vs-staging happy path.
GROUP_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
source "${GROUP_DIR}/../../helpers/cli.sh"
diff --git a/tests/e2e/scenarios/manifest.json b/tests/e2e/scenarios/manifest.json
index 00ba6624..ef4f459d 100644
--- a/tests/e2e/scenarios/manifest.json
+++ b/tests/e2e/scenarios/manifest.json
@@ -504,7 +504,7 @@
"errors"
]
},
- "cli/compare-basic.sh": {
+ "cli/compare-extended.sh": {
"tags": [
"compare",
"audit",