:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
chore(ci): bound stalled cache restores in backend ci (#123431)
Raúl Negrón-Otero373397b03ba0
diff --git a/.depot/actions/pnpm-install/action.yml b/.depot/actions/pnpm-install/action.ymlindex c2e92050eb81..12a17f4de700 100644--- a/.depot/actions/pnpm-install/action.yml+++ b/.depot/actions/pnpm-install/action.yml@@ -1,6 +1,7 @@# Depot mirror of .github/actions/pnpm-install (same pattern as .depot/actions/setup-sqlx-cli).-# Kept equivalent to canonical except for the `inputs.install` guards and the master-gated-# "Save pnpm store" step below. actions/cache routes to Depot Cache, so shared keys do not+# Kept equivalent to canonical except for the `inputs.install` guards, the master-gated+# "Save pnpm store" step below, and the timeout-minutes on both restores, which GitHub Actions+# rejects in a composite step. actions/cache routes to Depot Cache, so shared keys do not# collide.name: Install pnpm dependenciesdescription: |@@ -55,6 +56,8 @@ runs:- name: Restore pnpm storeid: restoreif: inputs.install != 'false'+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: ${{ steps.pnpm-store.outputs.path }}@@ -75,6 +78,8 @@ runs:- name: Restore pnpm lockfile verdictid: restore-verdictif: inputs.install != 'false'+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: ${{ steps.pnpm-cache.outputs.path }}/lockfile-verified.jsonldiff --git a/.depot/actions/setup-pnpm/action.yml b/.depot/actions/setup-pnpm/action.ymlindex a0b40db31572..b0aaee6c223d 100644--- a/.depot/actions/setup-pnpm/action.yml+++ b/.depot/actions/setup-pnpm/action.yml@@ -69,6 +69,9 @@ runs:- name: Restore Node tool cacheid: node-toolcacheif: steps.node-version.outputs.version != ''+ # GitHub Actions rejects timeout-minutes in a composite step, so only this mirror bounds the restore.+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: ${{ runner.tool_cache }}/node/${{ steps.node-version.outputs.version }}diff --git a/.depot/actions/setup-python-cached/action.yml b/.depot/actions/setup-python-cached/action.ymlindex 68852daa27ba..aa8cf86b3ced 100644--- a/.depot/actions/setup-python-cached/action.yml+++ b/.depot/actions/setup-python-cached/action.yml@@ -1,3 +1,5 @@+# Depot mirror of .github/actions/setup-python-cached. Only the tool cache restore differs. It sets+# timeout-minutes, which GitHub Actions rejects in a composite step.name: Set up Pythondescription: |Wraps actions/setup-python with a pre-seeded tool cache@@ -41,6 +43,8 @@ runs:- name: Restore Python tool cacheid: python-toolcacheif: steps.python-version.outputs.version != ''+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: ${{ runner.tool_cache }}/Python/${{ steps.python-version.outputs.version }}diff --git a/.depot/actions/setup-sqlx-cli/action.yml b/.depot/actions/setup-sqlx-cli/action.ymlindex 21d8425ed8ef..3b8e226d555b 100644--- a/.depot/actions/setup-sqlx-cli/action.yml+++ b/.depot/actions/setup-sqlx-cli/action.yml@@ -1,3 +1,5 @@+# Depot mirror of .github/actions/setup-sqlx-cli. Only the cache restore differs. It sets+# timeout-minutes, which GitHub Actions rejects in a composite step.name: Setup sqlx-clidescription: |Installs sqlx-cli and adds it to PATH.@@ -20,6 +22,8 @@ runs:steps:- name: Restore sqlx-cli binary from cacheid: cache-sqlx-cli+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@9255dc7a253b0ccc959486e2bca901246202afeb # v5.0.1with:# The cache version hashes this path string, and runner.temp differs betweendiff --git a/.depot/workflows/ci-backend.yml b/.depot/workflows/ci-backend.ymlindex 1de9f907e9ab..b17e3ad6a1dc 100644--- a/.depot/workflows/ci-backend.yml+++ b/.depot/workflows/ci-backend.yml@@ -38,6 +38,13 @@# Without those plans, shards use the combined .test_durations file.# - actions/cache uses Depot Cache, which Depot CI and Depot GitHub Actions runners share.# GitHub-hosted runners use a separate GitHub Actions cache.+# - A stalled actions/cache request holds a restore for about 15 minutes and then misses. The+# client tries each request 5 times with a 3-minute socket timeout, and no input or env var+# shortens either. SEGMENT_DOWNLOAD_TIMEOUT_MINS covers only the blob download. Restores here+# and in the Depot composite mirrors set timeout-minutes and continue-on-error, so a stall+# becomes an early miss. Schema restores get 4 minutes, so one retry runs before a+# from-scratch migrate. Saves stay unbounded, because a timeout drops what they write. The uv+# cache inside astral-sh/setup-uv stays unbounded, because a timeout there kills the install.# - The env block clears GITHUB_TOKEN and OTEL_SERVICE_NAME because Depot provides them by default.# - Token steps read vars.* and secrets.* from Depot's store. If a value is missing there,# setup-action steps with continue-on-error fall back to the default GITHUB_TOKEN, a Depot@@ -780,6 +787,8 @@ jobs:enable-cache: falsecontinue-on-error: true- name: Restore test durations from cache+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: .test_durations@@ -807,6 +816,8 @@ jobs:gh run download "$run_id" --repo "$GITHUB_REPOSITORY" --name test-durations --dir . \|| echo "::warning::Timing artifact of run $run_id unavailable, sharding without durations"- name: Restore per-segment test durations from cache+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: |@@ -819,6 +830,8 @@ jobs:# The timing workflow rebuilds it from the newest hourly recording. A miss# leaves the file absent, and the events_json rows then run every listed test.if: github.event_name == 'pull_request'+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: events-schema-prune.json@@ -942,6 +955,8 @@ jobs:steps:- name: Look up schema cacheid: schema-cache+ timeout-minutes: 4+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: schema.sql.gz@@ -1151,6 +1166,8 @@ jobs:|| { echo "::group::compose launch log"; cat "${RUNNER_TEMP:-/tmp}/compose-launch.log" 2>/dev/null || true; echo "::endgroup::"; exit 1; }- name: Restore schema cache from masterif: ${{ github.event_name == 'pull_request' }}+ timeout-minutes: 4+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: schema.sql.gz@@ -1207,7 +1224,7 @@ jobs:bin/wait-for-docker temporalPYTHONFAULTHANDLER=1 timeout --signal=ABRT --kill-after=30s 3m python manage.py register_temporal_search_attributes- name: Restore test durations from cache- timeout-minutes: 7+ timeout-minutes: 2continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:@@ -1800,6 +1817,8 @@ jobs:- name: Restore master schema dump from cacheid: master-schema-cacheif: github.event_name != 'push' && steps.master-schema-key.outputs.key != ''+ timeout-minutes: 4+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: schema.sql.gz@@ -3144,6 +3163,8 @@ jobs:# Restore before setup-python so the tool cache is warm when it starts.- name: Restore Python tool cacheid: python-toolcache+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: ${{ runner.tool_cache }}/Python/${{ matrix.python-version }}@@ -3293,6 +3314,8 @@ jobs:|| { echo "::group::compose launch log"; cat "${RUNNER_TEMP:-/tmp}/compose-launch.log" 2>/dev/null || true; echo "::endgroup::"; exit 1; }- name: Restore schema cache from masterif: ${{ github.event_name == 'pull_request' }}+ timeout-minutes: 4+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: schema.sql.gz@@ -3344,6 +3367,8 @@ jobs:admin_psql -c "DROP DATABASE IF EXISTS test_posthog;" || truefi- name: Restore test durations from cache+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: .test_durations@@ -3371,6 +3396,8 @@ jobs:gh run download "$run_id" --repo "$GITHUB_REPOSITORY" --name test-durations --dir . \|| echo "::warning::Timing artifact of run $run_id unavailable, sharding without durations"- name: Restore per-segment test durations from cache+ timeout-minutes: 2+ continue-on-error: trueuses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: |diff --git a/tools/hogli-commands/hogli_commands/depot_mirrors.py b/tools/hogli-commands/hogli_commands/depot_mirrors.pyindex bc3200ebc6c2..f1917ceac52e 100644--- a/tools/hogli-commands/hogli_commands/depot_mirrors.py+++ b/tools/hogli-commands/hogli_commands/depot_mirrors.py@@ -13,7 +13,7 @@ from pathlib import Path# These mirrors carry intentional deltas that each mirror's header documents.# Every other mirror stays byte-identical to its canonical action.-DELTA_MIRRORS = frozenset({"commit-snapshots", "pnpm-install", "setup-pnpm"})+DELTA_MIRRORS = frozenset({"commit-snapshots", "pnpm-install", "setup-pnpm", "setup-python-cached", "setup-sqlx-cli"})# Depot resolves both `./.github/actions/<name>` and `./.depot/actions/<name>` from .depot/actions/<name>.LOCAL_ACTION_USE = re.compile(r"""uses:\s*['"]?\./\.(?:github|depot)/actions/([A-Za-z0-9_.-]+)""")