Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

ci: fix the steps that fail in a copy of the repository (#122108)

Georges-Antoine Assi4577dd02685e
diff --git a/.github/workflows/ci-e2e-playwright.yml b/.github/workflows/ci-e2e-playwright.yml
index 1e8409269619..4a515123396a 100644
--- a/.github/workflows/ci-e2e-playwright.yml
+++ b/.github/workflows/ci-e2e-playwright.yml
@@ -977,22 +977,36 @@ jobs:
echo "test_seconds=$(( $(date +%s) - start ))" >> "$GITHUB_OUTPUT"
exit "$rc"
- - name: Verify changed Playwright tests are stable
+ # The stability script diffs from the merge-base of BASE_SHA and HEAD so a PR on
+ # an older master doesn't inherit specs changed only on master. The merge-base
+ # must be reachable from *both* tips: this job checks out HEAD at depth 1, so
+ # deepen both histories (depth=1000 + blob:none mirrors the merge-base fetches
+ # in the `changes` job). If the branch point is still out of reach, the script
+ # warns and falls back to BASE_SHA — bounded by its own test-run budget guard.
+ # Its own step, so the token reaches git alone: the next step runs PR code,
+ # and the checkout persists no credentials for the same reason. The one-off
+ # header keeps the token out of .git/config; a private copy of the repository
+ # refuses an anonymous fetch.
+ - name: Deepen history for the stability check
if: steps.playwright-tests.outcome == 'success' && github.event_name == 'pull_request'
shell: bash
env:
BASE_SHA: ${{ github.event.pull_request.base.sha }}
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
+ GH_TOKEN: ${{ github.token }}
run: |
- # The script diffs from the merge-base of BASE_SHA and HEAD so a PR on an
- # older master doesn't inherit specs changed only on master. The merge-base
- # must be reachable from *both* tips: this job checks out HEAD at depth 1, so
- # deepen both histories (depth=1000 + blob:none mirrors the merge-base fetches
- # in the `changes` job). If the branch point is still out of reach, the script
- # warns and falls back to BASE_SHA — bounded by its own test-run budget guard.
- git fetch --no-tags --prune --depth=1000 --filter=blob:none origin "$BASE_SHA"
- git fetch --no-tags --depth=1000 --filter=blob:none origin "$HEAD_SHA"
- .github/scripts/verify-playwright-new-tests-and-snapshots.sh "$BASE_SHA" 10
+ auth_header="AUTHORIZATION: basic $(printf 'x-access-token:%s' "$GH_TOKEN" | base64 | tr -d '\n')"
+ git -c "http.https://github.com/.extraheader=$auth_header" \
+ fetch --no-tags --prune --depth=1000 --filter=blob:none origin "$BASE_SHA"
+ git -c "http.https://github.com/.extraheader=$auth_header" \
+ fetch --no-tags --depth=1000 --filter=blob:none origin "$HEAD_SHA"
+
+ - name: Verify changed Playwright tests are stable
+ if: steps.playwright-tests.outcome == 'success' && github.event_name == 'pull_request'
+ shell: bash
+ env:
+ BASE_SHA: ${{ github.event.pull_request.base.sha }}
+ run: .github/scripts/verify-playwright-new-tests-and-snapshots.sh "$BASE_SHA" 10
# Visual Review: create run + upload snapshots directly from the test job.
# Completion happens in handle-screenshots so the baseline lands in the same commit as PNGs.
@@ -1000,7 +1014,7 @@ jobs:
# workspace root here, so this is a separate entry. No restore-keys.
- name: Restore VR CLI build
id: restore-vr-cli
- if: always() && (github.repository != 'PostHog/posthog' || github.event.pull_request.head.repo.full_name == github.repository || github.event_name == 'push')
+ if: always() && github.repository == 'PostHog/posthog' && (github.event.pull_request.head.repo.full_name == github.repository || github.event_name == 'push')
uses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3
with:
path: |
@@ -1012,7 +1026,7 @@ jobs:
# in ci-storybook.yml.
- name: Build VR CLI
id: build-vr-cli
- if: always() && (github.repository != 'PostHog/posthog' || github.event.pull_request.head.repo.full_name == github.repository || github.event_name == 'push') && steps.restore-vr-cli.outputs.cache-hit != 'true'
+ if: always() && github.repository == 'PostHog/posthog' && (github.event.pull_request.head.repo.full_name == github.repository || github.event_name == 'push') && steps.restore-vr-cli.outputs.cache-hit != 'true'
timeout-minutes: 8
shell: bash
working-directory: products/visual_review/cli
@@ -1079,33 +1093,12 @@ jobs:
--baseline playwright/snapshots.yml \
--token "$VR_TOKEN"
- - name: Verify snapshots against the baseline offline
+ # playwright/snapshots.yml carries no hashes: Playwright baselines live in Visual
+ # Review alone, so a repository without a Visual Review run has nothing to compare
+ # its screenshots against. Say so instead of failing on the empty baseline.
+ - name: Report that Visual Review did not run
if: ${{ !cancelled() && github.repository != 'PostHog/posthog' }}
- shell: bash
- run: |
- set +e
- node products/visual_review/cli/dist/cli/src/index.js verify --partial \
- --dir playwright/__snapshots__/ \
- --baseline playwright/snapshots.yml > /tmp/vr-verify.log 2>&1
- status=$?
- set -e
- cat /tmp/vr-verify.log
- {
- echo "### Offline snapshot check"
- echo
- echo "This run has no Visual Review run, so it compares its screenshots with \`playwright/snapshots.yml\` offline."
- echo "This fallback accepts only an exact pixel match. It does not know the tolerated hashes or the quarantine of Visual Review, so a flaky snapshot can also fail it."
- echo
- echo '```'
- cat /tmp/vr-verify.log
- echo '```'
- } >> "$GITHUB_STEP_SUMMARY"
- if [ "$status" -ne 0 ] && grep -q 'Baseline mismatch' /tmp/vr-verify.log; then
- echo "::error title=Snapshots differ from the baseline::These screenshots do not match playwright/snapshots.yml. This run has no Visual Review run, so this offline fallback accepts only an exact pixel match. Update the baseline after reviewing the change. The job log lists the snapshots that differ."
- elif [ "$status" -ne 0 ]; then
- echo "::error title=Offline snapshot check failed::The offline snapshot check could not compare the screenshots. The job log shows the error."
- fi
- exit "$status"
+ run: echo "::notice title=Playwright screenshots not verified::Visual Review runs only in PostHog/posthog, and playwright/snapshots.yml holds no baseline hashes to compare against offline. The screenshots of this run are verified when the change reaches PostHog/posthog."
# ── Artifacts on failure / always ─────────────────────────────────────
- name: Capture docker logs
diff --git a/.github/workflows/ci-mcp.yml b/.github/workflows/ci-mcp.yml
index 8c6cb2f54abd..164bdbbc9aaa 100644
--- a/.github/workflows/ci-mcp.yml
+++ b/.github/workflows/ci-mcp.yml
@@ -798,7 +798,8 @@ jobs:
# A status function is required to commit regenerated snapshots even when unit-tests
# failed (a snapshot mismatch fails that job). !cancelled() rather than always() so a
# superseded run does not push a stale snapshot commit to the branch.
- if: ${{ !cancelled() && (needs.changes.outputs.mcp == 'true' || needs.changes.outputs.mcp-typescript == 'true') && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && !contains(github.actor, '[bot]') && github.actor != 'posthog-bot' && !contains(github.actor, 'github-actions') }}
+ # The commit App is installed on PostHog/posthog only; the gate reads a skipped job as a pass.
+ if: ${{ !cancelled() && github.repository == 'PostHog/posthog' && (needs.changes.outputs.mcp == 'true' || needs.changes.outputs.mcp-typescript == 'true') && github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository && !contains(github.actor, '[bot]') && github.actor != 'posthog-bot' && !contains(github.actor, 'github-actions') }}
permissions:
contents: write
pull-requests: write
diff --git a/.github/workflows/ci-security.yaml b/.github/workflows/ci-security.yaml
index f6f162e7c640..09966f3de16a 100644
--- a/.github/workflows/ci-security.yaml
+++ b/.github/workflows/ci-security.yaml
@@ -150,7 +150,7 @@ jobs:
needs: changes
if: needs.changes.outputs.python == 'true'
runs-on: ubuntu-24.04
- timeout-minutes: 30
+ timeout-minutes: 45
env:
DOCKERHUB_USERNAME: ${{ vars.DOCKERHUB_USER }}
diff --git a/.github/workflows/ci-storybook.yml b/.github/workflows/ci-storybook.yml
index 41c916e2eb7a..f899a9f4a795 100644
--- a/.github/workflows/ci-storybook.yml
+++ b/.github/workflows/ci-storybook.yml
@@ -829,9 +829,14 @@ jobs:
# bad signature, so the merge queue still catches a forged one.
# !cancelled() keeps the check after a test-runner failure. A failed CLI build
# fails this step too, so a fork PR never passes unverified.
+ # Outside PostHog/posthog the check warns instead of failing: another runner
+ # renders a third of the unchanged stories a pixel off, and Visual Review
+ # verifies the change once it reaches PostHog/posthog.
- name: Verify snapshots against the baseline offline
if: ${{ !cancelled() && ((github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name != github.repository) || github.repository != 'PostHog/posthog') }}
shell: bash
+ env:
+ BLOCKING: ${{ github.repository == 'PostHog/posthog' }}
run: |
set +e
node vr-cli/products/visual_review/cli/dist/cli/src/index.js verify --partial \
@@ -850,6 +855,10 @@ jobs:
cat /tmp/vr-verify.log
echo '```'
} >> "$GITHUB_STEP_SUMMARY"
+ if [ "$status" -ne 0 ] && [ "$BLOCKING" != "true" ]; then
+ echo "::warning title=Snapshots not verified::The offline snapshot check did not pass for this shard. This repository has no Visual Review run, and the exact-pixel fallback also flags stories that another runner renders a pixel off, so it does not block here. Visual Review verifies the change when it reaches PostHog/posthog. The job log lists the snapshots that differ."
+ exit 0
+ fi
if [ "$status" -ne 0 ] && grep -q 'Baseline mismatch' /tmp/vr-verify.log; then
echo "::error title=Snapshots differ from the baseline::This shard's screenshots do not match frontend/snapshots.yml. This run has no Visual Review run, so this offline fallback accepts only an exact pixel match, without the tolerated hashes or the quarantine of Visual Review. A maintainer must review the change and update the baseline before this PR can merge. The job log lists the snapshots that differ."
elif [ "$status" -ne 0 ]; then