:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
chore(ci): point docs and comments at the depot backend workflow (#116515)
Raúl Negrón-Otero63de6a059cdc
diff --git a/.agents/skills/authoring-ci-workflows/SKILL.md b/.agents/skills/authoring-ci-workflows/SKILL.mdindex 346ca800dd7f..fb611f4fa6ab 100644--- a/.agents/skills/authoring-ci-workflows/SKILL.md+++ b/.agents/skills/authoring-ci-workflows/SKILL.md@@ -17,8 +17,8 @@ The linters own the mechanical rules (below); this skill is the **judgment calls## Before you write- Copy from a canonical file rather than from memory.- `ci-paths-filter.yml` is the smallest complete example (triggers, concurrency, timeout, app token, Depot runner);- `ci-backend.yml` is the reference for the heavy patterns (bounded-depth checkout, per-SHA concurrency, draft/ready, sharding).+ `ci-paths-filter.yml` is the smallest complete example, with triggers, concurrency, a timeout, an app token, and a Depot runner.+ Use `.depot/workflows/ci-backend.yml` for bounded-depth checkouts, per-SHA concurrency, draft/ready handling, and sharding.- Related skills — reach for these instead of duplicating them here:- `/gating-production-deploys` — any job that pushes a prod image or dispatches a Charts deploy.Read its [ordered-image reference](../gating-production-deploys/references/ordered-image-releases.md) for new monorepo publishers and shared-build consumers.@@ -38,15 +38,20 @@ The linters check the shape of an `if:`; `tools/workflow-plan` checks what it doIt evaluates every job and step condition with GitHub's own expression evaluator against a synthetic draft PR, ready PR, fork PR, merge-queue run, master push, hourly schedule, and manual dispatch, and prints which jobs run:```bash-hogli ci:plan .github/workflows/ci-backend.yml # one row per job, one column per scenario-hogli ci:plan .github/workflows/ci-backend.yml --steps changes+hogli ci:plan .depot/workflows/ci-backend.yml # one row per job, one column per scenario+hogli ci:plan .depot/workflows/ci-backend.yml --steps changeshogli test:workflows # the pinned expectations, run in ci-lint-workflows```-`tools/workflow-plan/tests/workflows.test.ts` pins the rules in this file as `runs` / `skipped` rows per scenario: drafts skip the product matrix, the queue takes the full one, forks skip telemetry, a `no-ci` draft still reports its gate, the hourly run takes the matrices and skips the PR-only checks, and a superseded run records the gate as `cancelled`.-Every job in those two workflows that carries an `if:` must be named by at least one row, which a coverage test enforces: add a conditional job and the suite fails until a row says what it should do.-When you change a condition in `ci-backend.yml` or `ci-frontend.yml`, run the suite and update the row that describes the behavior you changed; when you add a lever to another heavy suite, add rows for it.-The planner stubs the paths filter as "everything changed" and knows nothing a `run:` body produces, so a scenario stubs selector outputs by step id.+`tools/workflow-plan/tests/workflows.test.ts` pins the expected jobs and results for each scenario.+Drafts skip the product matrix; merge-queue runs use the full matrix.+Forks skip telemetry, and a `no-ci` draft still reports its gate.+Hourly runs use the matrices and skip PR-only checks.+Superseded runs record the gate as `cancelled`.+Every job with an `if:` in the backend and frontend workflows must appear in at least one expectation, or the coverage test fails.+When you change a condition in `.depot/workflows/ci-backend.yml` or `ci-frontend.yml`, update its expectation and run the suite.+Add expectations when introducing conditional jobs in another heavy suite.+The planner treats every paths filter as matched and uses stubbed step outputs for selector scripts.It does not model trigger `paths:`, concurrency, or matrix expansion beyond a cell count; see [the README](../../../tools/workflow-plan/README.md).## The dispatch budget (500 runs / 10s / repo)@@ -105,7 +110,7 @@ concurrency:Where latest-wins is genuinely right (a cache warmer), say so with `# hogli-lint: allow-master-cancel -- <reason>`.- Use `github.ref` as the fallback, never `github.run_id` — `run_id` is unique per run, so it silently gives every push its own group and dedup is lost.- Publish-on-push workflows must not let two master pushes race `:latest` / a deploy dispatch.- Key the push arm per-SHA (see `ci-backend.yml`):+ Key the push arm per-SHA (see `.depot/workflows/ci-backend.yml`):```yamlgroup: ${{ github.workflow }}-${{ github.event_name == 'push' && github.sha || github.head_ref || github.ref }}@@ -205,7 +210,12 @@ Three cases behave in ways the name does not suggest:- A **job-level** `continue-on-error: true` posts a `failure` check run even though dependents read `success` and the run goes green. Use step-level `continue-on-error` plus an explicit verdict step instead.- A **matrix that expands to zero cells** fails its dependents on GitHub Actions and posts no check run at all. Guard any `fromJSON` matrix with an `if:` that skips the job when the list is empty.-Required contexts are pinned to one app: every entry in this repo's `master` ruleset carries `integration_id: 15368`, the `github-actions` app, so a check run from any other app never satisfies one however exactly the name matches. [`/depot-ci`](../depot-ci/references/posthog-check-run-semantics.md) has the measurements, the rulesets query, and how Depot CI differs.+Most required checks in this repo's `master` ruleset are pinned to the `github-actions` app, `integration_id: 15368`.+Backend CI has two exceptions.+`Backend CI on Depot / Test results` is pinned to the PostHog tests app, `2492437`.+`Backend CI on Depot / Django Tests Pass on Depot` has no app pin because Depot posts it from more than one app.+A check from another app cannot satisfy a pinned requirement, even when its name matches.+See [`/depot-ci`](../depot-ci/references/posthog-check-run-semantics.md) for measurements and the rulesets query.## Checkout / clone — sparse first, then shallow@@ -510,9 +520,15 @@ Any _step_ that reads a filter output needs the same `schedule` arm.A lane that stops producing runs is invisible to the master-red alerter: it reads run completions, so a dropped cron reads as unreadable and drops out of evaluation rather than paging.Add every converted workflow to `SCHEDULED_GATING_WORKFLOWS` in `ci-alerts-devex.yml` in the same change, or its failures stop paging altogether.-Backend CI's hourly run is the one test suite whose cron is on Depot CI: `.depot/workflows/ci-backend.yml` has the `schedule` trigger and `.github/workflows/ci-backend.yml` has none.-GitHub has no runs or artifacts for it, so its readers go through `.github/scripts/depot_scheduled_runs.py`: the alerter names it in `DEPOT_SCHEDULED_GATING_WORKFLOW`, and `ci-backend-update-test-timing.yml` finds and downloads its artifacts from Depot.-The master run trace reporter reads the same Depot lane. Depot's public API exposes workflow and job timestamps, so those traces omit step and job-queue spans.+Backend CI runs on Depot CI from `.depot/workflows/ci-backend.yml` for same-repository pull requests, master pushes, manual runs, and the hourly schedule.+Depot stores those runs and their original artifacts; GitHub receives check results.+Use `.github/scripts/depot_backend_runs.py` to read the Depot runs and artifacts.+The consumers are:++- The alerter lists Backend CI in `DEPOT_WORKFLOWS` and reads master push and scheduled runs from the files that script writes.+- `ci-backend-update-test-timing.yml` downloads timing artifacts from Depot.+- `ci-backend-schema-mirror.yml` copies the newest master schema dump into a GitHub artifact and GitHub Actions cache every ten minutes so GitHub-hosted jobs and fork pull requests can read it.+- The master run trace reporter reads master push and scheduled runs from Depot. Depot's public API exposes workflow and job timestamps, but no step or job-queue timestamps, so those traces stop at jobs.## Backwards-compat with unrebased PRsdiff --git a/.agents/skills/debugging-ci-failures/SKILL.md b/.agents/skills/debugging-ci-failures/SKILL.mdindex aae74caaed10..ff5c4396a438 100644--- a/.agents/skills/debugging-ci-failures/SKILL.md+++ b/.agents/skills/debugging-ci-failures/SKILL.md@@ -153,7 +153,7 @@ practice that is most of the queue. So:The branch is ephemeral; the run and its logs stay on GitHub, and thewarehouse keeps its jobs under that `head_branch` (query 8 in the`investigating-ci-failures` references).- Backend CI is the exception when `CI_BACKEND_DEPOT_MERGE_QUEUE_PERCENT` hands the batch to Depot CI: the backend tests then run on Depot, and the `Django Tests Pass` check links that run.+ Backend tests run on Depot CI. The `Backend CI on Depot / Django Tests Pass on Depot` check links to the Depot run.- The PR's own checks can be green with the failing job **skipped** ornarrowed. On the PR, path filters see only that diff and the Django suite runsa selected subset; on the queue branch the diff is every carried PR's and the@@ -224,7 +224,7 @@ has pushed, so an earlier push's failure is still there.Extract these before classifying:-- Workflow name or file, e.g. `.github/workflows/ci-backend.yml`.+- Workflow name or file, e.g. `.depot/workflows/ci-backend.yml`.- Job name, e.g. `backend-tests (4/10)`.- Step name, e.g. `Run pytest`.- Failing command and the smallest useful output excerpt.@@ -247,11 +247,18 @@ token to the server entry in `.mcp.json`. To dig into one test's flakinesshistory, hand off to `fixing-flaky-tests`, which covers the `search-test` and`fix-flaky-test` tools.-#### Backend tests that ran on Depot CI+#### Backend tests on Depot CI-Depot CI runs the backend tests of every same-repo PR.-The required check is `Backend CI on Depot / Django Tests Pass on Depot`, and the `Backend CI on Depot / Test results` check lists each failed step with its log lines.-`Django Tests Pass` on GitHub Actions only prints a notice on a PR.+Use the Depot CLI for Backend CI runs and logs. `gh run` only lists GitHub Actions runs.+The required checks are `Backend CI on Depot / Test results` and `Backend CI on Depot / Django Tests Pass on Depot`.+`Test results` lists failing tests in its summary:++```bash+gh api 'repos/PostHog/posthog/commits/<sha>/check-runs?check_name=Backend%20CI%20on%20Depot%20/%20Test%20results' \+ --jq '.check_runs[0].output.summary'+```++A passing draft run does not post `Test results`. GitHub shows it as expected until the PR is ready for review, so do not wait for that check on a draft.GitHub's Re-run button does nothing on a Depot check.A `/depot retry` comment on the PR from an organization member or collaborator retries the failed and cancelled jobs of the newest run, also while the rest of the run still goes.@@ -268,7 +275,7 @@ If a `depot` command fails to authenticate, check `DEPOT_TOKEN` first: it overriWithout it, ask the user to run `! depot login --clear`.Do not run it yourself: it waits on a browser sign-in.Every PostHog engineer gets an invite to the Depot org. A user without access checks their email for it, or requests one in #team-devex.-Until then, retry with the `/depot retry` comment.+Without Depot access, use `/depot retry` to retry the run and read its `Test results` summary through the GitHub API.## Classification@@ -377,13 +384,12 @@ Excerpt:<up to 40 lines, trimmed around the failure>Classification: <class from the table>-Shadow run: <yes | no>Likely cause: <one sentence>Local repro: <exact command, or "none">Next action (needs your approval):- <push fix | rerun job | update snapshot | none>```-If the classification is `infra / runner` or a shadow run, say so and stop;+If the classification is `infra / runner`, say so and stop;do not propose a code change. For `infra / runner`, include the base rate andwhether a retry is warranted.diff --git a/.agents/skills/debugging-ci-failures/references/master-red-incident.md b/.agents/skills/debugging-ci-failures/references/master-red-incident.mdindex 860415834a5d..462fb36c4d4e 100644--- a/.agents/skills/debugging-ci-failures/references/master-red-incident.md+++ b/.agents/skills/debugging-ci-failures/references/master-red-incident.md@@ -53,9 +53,11 @@ The commands above read the same evidence.### The lane on Depot CI-`Backend CI (scheduled)` is the hourly full test matrix. It runs on Depot CI, so GitHub has no runs or logs for it.-The alert gives this lane a Depot run ID, ten characters such as `b1c2d3f4g5`, and a depot.dev URL.-The alerter reads the lane from Depot when it opens the incident, so the alerted run is the newest failing run.+`Backend CI (scheduled)` is the hourly full test matrix.+`Backend CI` without the suffix is the master push run.+Depot stores both types of runs and their logs; GitHub receives job checks.+The alert includes a Depot run ID, such as `b1c2d3f4g5`, and a depot.dev URL.+The alerter reads Depot when it opens the incident, so the alert identifies the newest failing run at that time.Depot posts a check for each job on the master commit that the run tested.Those checks give each job's conclusion and times, with no log text.diff --git a/.agents/skills/depot-ci/references/posthog-check-run-semantics.md b/.agents/skills/depot-ci/references/posthog-check-run-semantics.mdindex 6a74561ce3ed..47f913485020 100644--- a/.agents/skills/depot-ci/references/posthog-check-run-semantics.md+++ b/.agents/skills/depot-ci/references/posthog-check-run-semantics.md@@ -79,7 +79,15 @@ In the following cancelled runs, an unstarted job received a `cancelled` check w## Finding one event's Depot run-Check times and `pull_requests` did not pick out the Depot run in these cases, so the event goes into a check name. The Depot wait job's name ends with `(PR <number>, event <pull_request.updated_at>)`, and Depot renders expressions in job names into the check name. `.github/scripts/ci_backend_relay.py` builds the same name from the GitHub run's payload, reads that check, takes the Depot workflow id from its `details_url`, and reads the gate check of that workflow only. Measured 2026-09-24 on PR 105886: Depot posted check `107656013423` named `… (PR 105886, event 2026-09-24T13:33:14Z)`, and GitHub relay job `107655803961` used `EVENT_AT: 2026-09-24T13:33:14Z` and relayed a successful gate.+Check timestamps and `pull_requests` could not identify which Depot run belonged to an event in these cases.+Until October 2026, the GitHub Actions relay matched events through the Depot wait job's check name.+The name ended with `(PR <number>, event <pull_request.updated_at>)` because Depot includes evaluated job-name expressions in its check names.+The relay built the same name from the GitHub event payload, found that check, and read the Depot workflow ID from its `details_url`.+It then read the gate check for that workflow.++On 2026-09-24, Depot posted check `107656013423` for [the relay example](https://github.com/PostHog/posthog/pull/105886) with the suffix `(PR 105886, event 2026-09-24T13:33:14Z)`.+GitHub relay job `107655803961` matched it using `EVENT_AT: 2026-09-24T13:33:14Z` and reported success.+The relay no longer runs, but this example shows how a check name can identify an event when timestamps and PR associations cannot.### Racing events@@ -87,15 +95,24 @@ Two events of one commit that arrive within a second or two race the concurrency- PR 106429: GitHub Actions cancelled the run of the newer event (15:17:40Z) and kept the older one (15:17:39Z). Depot kept 15:17:40Z and cancelled Depot run `rtl932mrbc` with "Cancelled due to the workflow concurrency policy".- PR 106435: GitHub Actions kept the newer event (15:17:41Z). Depot kept the older one (15:17:40Z) and cancelled `nzb661zql2`.-- Neither cancelled Depot run posted a wait check, so each surviving GitHub relay found no run for its event and failed after the grace period, although Depot's run for the other event passed on the same commit.+- Neither canceled Depot run posted a wait check. Each surviving GitHub relay found no run for its event and failed after the grace period, although Depot's other run passed on the same commit.-GitHub Actions alone does the same. Among the 946 `ci-backend.yml` pull request runs created on 2026-09-25 between 11:30Z and 16:00Z, 56 same-commit pairs were created within 30 seconds of each other, 52 of them 0 to 2 seconds apart. GitHub kept the older run in 6 pairs, all 0 to 1 second apart. So after its grace period, the relay follows the Depot run of an event of the same pull request up to 2 seconds away (`RACING_EVENT_SECONDS`), newest first.+GitHub Actions can also keep the older event.+Among the 946 `ci-backend.yml` pull request runs created on 2026-09-25 between 11:30Z and 16:00Z, 56 same-commit pairs were created within 30 seconds of each other.+Of those, 52 pairs were created 0 to 2 seconds apart.+GitHub kept the older run in 6 pairs, all 0 to 1 second apart.+The relay therefore accepted a Depot run for another event on the same pull request within 2 seconds of its own event, checking newest first.+Readers that match events by timestamp need to allow for this race.The newest check per name tells you the verdict. It does not tell you whether the PR can merge. A cancelled run on the same head keeps its checks, and when one of them is a required check, GitHub's merge box and Trunk keep the PR blocked until that run is rerun. `/merging-prs` has the recipe.### Superseded runs-Depot's concurrency policy keeps the workflow it created last, and a Depot run gets its workflow some time after the run itself, so the policy can keep an older event. Measured 2026-09-29 on 284 `ci-backend.yml` pull request workflows: the workflow followed its run by 55 seconds at the median and 120 seconds at most, and in 11 of 165 consecutive run pairs of one pull request the older run's workflow was created first or in the same second. Two of those cancelled the event that GitHub Actions had handed off, and the relay failed with "Depot CI started no run for this event":+Depot's concurrency policy keeps the workflow it created last.+A run receives its workflow after the run itself is created, so workflow creation order can differ from event order.+On 2026-09-29, across 284 `ci-backend.yml` pull request workflows, the delay was 55 seconds at the median and 120 seconds at most.+In 11 of 165 consecutive run pairs for one pull request, the older run's workflow was created first or in the same second.+Two races canceled the event that GitHub Actions had handed off, and the relay failed with "Depot CI started no run for this event":- PR 107856: run `wq9rjmvr7z` for head `056fb05982` was created at 16:09:59Z and got its workflow at 16:11:10Z. Run `0k16ml32s1` for the newer head `b9fbb26201` was created at 16:10:18Z and got its workflow at 16:11:09Z, so Depot cancelled `0k16ml32s1`.- PR 108622: two events of `d484f7a101` at 18:29:23Z and 18:29:34Z. GitHub Actions kept the second. Depot kept the first, whose gate passed, and the relay's 2-second racing window did not reach it.@@ -132,7 +149,7 @@ Depot CI checks out the pull request merge ref (`refs/pull/<n>/merge`) and postsFrom Depot's [compatibility page](https://depot.dev/docs/ci/compatibility), not measured here:-- Fork pull requests get no run; Depot lists support as planned. The backend router keeps them on GitHub Actions, see [Pull requests from forks](../../../../docs/published/handbook/engineering/fork-pull-requests.md).+- Depot does not run fork pull requests. After review, a maintainer re-creates the pull request in the main repository. See [Pull requests from forks](../../../../docs/published/handbook/engineering/fork-pull-requests.md).- `environment:` is unsupported, and `uses:` cannot reference a workflow in another repository.- `secrets.GITHUB_TOKEN` is a GitHub App installation token, not the Actions token. GitHub Packages rejects it. The rate limit it draws on is the app installation's pool, which is shared across repos and is not the per-repo Actions bucket that `monitor-github-rate-limit.yml` watches.- Non-Depot `runs-on` labels are treated as `depot-ubuntu-latest`.diff --git a/.agents/skills/fixing-flaky-tests/SKILL.md b/.agents/skills/fixing-flaky-tests/SKILL.mdindex a0b0033f91e8..40873faeda6c 100644--- a/.agents/skills/fixing-flaky-tests/SKILL.md+++ b/.agents/skills/fixing-flaky-tests/SKILL.md@@ -37,8 +37,16 @@ Establish the timeline yourself from raw run data:```bash# Pass/fail history for the workflow on the branch under suspicion (master shown):-gh run list --repo PostHog/posthog --workflow=ci-backend.yml --branch master \+gh run list --repo PostHog/posthog --workflow=ci-frontend.yml --branch master \--status completed --limit 60 --json conclusion,headSha,createdAt,databaseId+# Backend CI's scheduled runs on master:+depot ci workflow list --repo PostHog/posthog --name 'Backend CI on Depot' --trigger schedule -n 60 --output json+```++Read a backend job's log from Depot:++```bash+depot ci logs <run-id> --workflow <workflow-id> --job <job-key>```The run-level conclusion is not enough — a red run may be failing on a _different_ job/test. Confirm it is the **same test** each time by reading the failing shard's log; and for green runs, confirm the test actually **ran and passed** in that shard (it may have been sharded elsewhere, not fixed):diff --git a/.agents/skills/setup-web-tests/SKILL.md b/.agents/skills/setup-web-tests/SKILL.mdindex 8b3c222288b5..a3326c1ab066 100644--- a/.agents/skills/setup-web-tests/SKILL.md+++ b/.agents/skills/setup-web-tests/SKILL.md@@ -78,9 +78,10 @@ Tests expect certain hostnames to resolve to localhost:echo "127.0.0.1 kafka clickhouse clickhouse-coordinator objectstorage" | sudo tee -a /etc/hosts```-## Environment Variables+## Environment variables-Tests require environment variables defined in [`.github/workflows/ci-backend.yml`](../../../.github/workflows/ci-backend.yml) (see the `env:` section at the top of the file). You can also copy `.env.example` to `.env` for local development defaults.+The workflow-level `env:` section in [`.depot/workflows/ci-backend.yml`](../../../.depot/workflows/ci-backend.yml) defines the test environment variables.+For local development defaults, copy `.env.example` to `.env`.## Additional Setup@@ -114,9 +115,10 @@ The `pytest.ini` sets:Default ignores: `--ignore=posthog/user_scripts --ignore=services/llm-gateway --ignore=common/ingestion/acceptance_tests`-## Debugging Installation Issues+## Debugging installation issues-If you encounter issues with the test setup, refer to [`.github/workflows/ci-backend.yml`](../../../.github/workflows/ci-backend.yml) for the authoritative CI configuration. This file shows:+Compare your setup with [`.depot/workflows/ci-backend.yml`](../../../.depot/workflows/ci-backend.yml) when installation fails.+It defines:- Exact Python version used in CI- System dependencies installeddiff --git a/.depot/workflows/cancel-closed-merge-queue-runs.yml b/.depot/workflows/cancel-closed-merge-queue-runs.ymlindex 63f3789ec3a3..c2bb02d72d13 100644--- a/.depot/workflows/cancel-closed-merge-queue-runs.yml+++ b/.depot/workflows/cancel-closed-merge-queue-runs.yml@@ -13,8 +13,7 @@ permissions:jobs:cancel:name: Cancel the closed batch's Backend CI runs- # Source pull requests are left alone: GitHub Actions does not cancel their relay on close,- # and the relay reports a cancelled Depot run as a failure. Trunk cancels a batch's relay itself.+ # Canceling a source PR's gate would block it if reopened until another push starts CI.if: startsWith(github.head_ref, 'trunk-merge/')runs-on: depot-ubuntu-24.04timeout-minutes: 5diff --git a/.depot/workflows/ci-backend.yml b/.depot/workflows/ci-backend.ymlindex d29ba2293cd2..748d1ec8b3b8 100644--- a/.depot/workflows/ci-backend.yml+++ b/.depot/workflows/ci-backend.yml@@ -1,87 +1,61 @@-# Backend CI. Lives under .depot/ so GitHub Actions ignores it; only Depot CI parses this-# file. Every pull request event, master push, manual run and the hourly master run start-# here, and the gate job's `Test results` check is the required one.+# Only Depot CI reads this file under .depot/. Branch protection requires the gate job's+# `Django Tests Pass on Depot` check and the `Test results` check it posts. Keep the workflow+# and gate job names in sync with the ruleset or pull requests cannot merge.#-# This file began as a copy of a GitHub Actions workflow, and the local actions it uses are-# mirrors of .github/actions (.github/workflows/ci-depot-action-mirrors.yml keeps those in-# sync). The notes below record where it differs from the GitHub Actions shape:+# Depot CI behavior this file works around:# - Depot's own checks carry no output, so the gate posts a `Test results` check with the# failure summary.-# - runs-on: depot-* labels instead of GitHub-hosted ubuntu-24.04-# - turbo-discover adds the `origin` remote before its history fetch when it is missing.-# Depot CI serves an input-less actions/checkout from a fast clone that has no remote.-# - action paths: ./.depot/actions/... instead of ./.github/actions/... (pnpm-install,-# setup-sqlx-cli, setup-python-cached, setup-uv, setup-turbo, and paths-filter mirror-# the canonical composites; any deltas are documented in each mirror's header). Every-# local action a depot workflow uses needs a mirror even when it is identical to-# canonical: Depot resolves ./.github/actions/<x> from .depot/actions/<x>, so a missing-# mirror fails the step at setup with "action manifest not found".-# - permissions: read-only in every job, so the ambient token never writes. Canonical grants-# handle-snapshots' and backend-coverage-report's github.token pull-requests:write for their-# CI report sections; here those sections post with a posthog-tests token scoped to pull-requests:write (see the header of-# .depot/actions/commit-snapshots). check-migrations mints that token before its SQL and-# risk steps, and skips each section write when the mint fails, where canonical posts the-# sections with github.token. Its Migration risk check token (checks:write only) is minted-# after those steps, as in canonical.-# - Trunk uploads and quarantine verdicts match canonical only when side effects-# are enabled. Disabled test steps retain their direct failure verdict.-# - dynamic-ci-filter: canonical asks Trunk which jobs a diff needs and records the verdict-# in that job's log. Nothing reads its outputs, so the job and its action pin are-# depot-exempt. Its fail-open step and its `no-ci` draft guard belong to that job, so-# they need no depot bump either.-# - Artifact uploads, run-scoped sharding plans, and the failing-test rollup-# require side effects, including duration writes. Depot stores these separately-# from GitHub Actions.-# - Hourly scheduled master run: only this workflow has the `schedule` trigger. Every `schedule` arm-# here is depot-only: the paths-filter skip, the `!= 'schedule'` gates on the per-commit-# checks, the events_json cron in build_django_matrix, the events schema recording and-# its upload, and POSTHOG_PYTEST_HARD_EXIT.-# - The master run's schema dump lands in Depot artifacts and Depot Cache, which GitHub-hosted-# runners and fork PRs cannot read. .github/workflows/ci-backend-schema-mirror.yml copies it-# into a GitHub artifact and GitHub Actions cache; canonical has no mirror job.-# - Superseded-run cancel: Depot's concurrency policy can keep an older event, so each event gets-# its own group and `cancel-superseded-runs` cancels older runs. See "Superseded runs" in+# - Depot resolves ./.github/actions/<x> from .depot/actions/<x>. Every local action needs a+# mirror there or setup fails with "action manifest not found". Mirrors explain intentional+# differences in their headers; .github/workflows/ci-depot-action-mirrors.yml checks the rest.+# - With no inputs, actions/checkout uses Depot's fast clone, which has no remote.+# turbo-discover must add `origin` before fetching history.+# - Every job has read-only permissions. CI report sections use a posthog-tests token scoped+# to pull-requests:write; see .depot/actions/commit-snapshots. check-migrations creates that+# token before the SQL and risk steps and skips report writes if token creation fails.+# It creates a separate checks:write token for the Migration risk check after those steps.+# - The hourly schedule skips path filtering and per-commit checks. It controls the events_json+# cadence in build_django_matrix, events schema recording and upload, and POSTHOG_PYTEST_HARD_EXIT.+# Master pushes run per-commit checks and dump the schema. Test matrices run on pull requests,+# manual runs, and the hourly schedule.+# - Master schema dumps go to Depot artifacts and Depot Cache, which GitHub-hosted+# runners and fork PRs cannot read. .github/workflows/ci-backend-schema-mirror.yml copies the+# newest dump into a GitHub artifact and GitHub Actions cache every ten minutes.+# - Depot's concurrency policy can keep an older event. Each event therefore gets its own+# group, and `cancel-superseded-runs` cancels older runs. See "Superseded runs" in# .agents/skills/depot-ci/references/posthog-check-run-semantics.md.-# - Deterministic-failure self-cancel: `depot ci cancel` with DEPOT_CI_CANCEL_TOKEN replaces-# `gh run cancel` with github.token, which only reaches GitHub Actions runs.-# - calculate-running-time: canonical's PostHog/posthog-github-action reads the run from the-# Actions API by run id, and GitHub has no run for a Depot run id. Here-# .github/scripts/capture_depot_running_time.py reads the workflow with `depot ci workflow show`-# and DEPOT_CI_CANCEL_TOKEN, then sends the same events with the same property keys and-# runner `depot-ci`. Its attempt guard sits on the steps, as canonical's does.-# - The workflow name is part of the required check's name, so renaming it means changing-# the ruleset.-# - events_json variants: the doubled matrices are gated on the test-new-events-schema label-# only; canonical also doubles them on workflow_dispatch, but a shadow dispatch runs the-# events_json rows instead, so trial runs match PR runs-# - per-segment split plans (.test_durations.core/.temporal): the restore steps mirror-# canonical but miss until the depot timing refresher saves those files to Depot Cache;-# shards fall back to the union .test_durations-# - actions/cache reads/writes route to Depot Cache, not GitHub Actions Cache,-# so shared keys with canonical (schema, uv, pnpm) do not collide-# - hypothesis constants cache: the restore and save steps mirror canonical verbatim, but the-# save only fires on master test-matrix runs and the shadow skips those, so the Depot Cache-# entry only exists after a master workflow_dispatch. PR shards run cold (they rebuild-# the pool, as canonical did before this cache existed) until then: bounded and documented.-# - GITHUB_TOKEN / OTEL_SERVICE_NAME neutralized (ambient on Depot, not on GHA)-# - GitHub App tokens: the mint steps' vars.* / secrets.* resolve from Depot's store, not-# GitHub's. A name missing there makes the setup-action mints (continue-on-error) fall-# back to the ambient GITHUB_TOKEN, the Depot Code Access installation token that every-# Depot CI job across repos shares, and fails the paths-filter mint outright.-# - Migration reports post from check-migrations, as in canonical. Do not move them to a-# pull_request_target workflow: Depot posts that workflow's job checks on another pull-# request's head commit when both events share a base commit. See "pull_request_target-# checks" in .agents/skills/depot-ci/references/posthog-check-run-semantics.md.-# - Workflow-command annotations (::error, ::warning) stay in the Depot job log. Depot CI-# does not attach them to the job's GitHub check run. The file annotations from-# `hogli lint:migration-deletions` therefore have no GitHub counterpart here; the failing-# Repo checks run carries the verdict.-# - `depot ci run` fires with event_name `api`, which the `workflow_dispatch` arms cover.-# Depot caps an inline workflow at 64 KiB, which this file exceeds, so-# trial a branch with `depot ci dispatch --workflow ci-backend.yml --ref <branch>` instead.-# - Three data-modeling tests deselected (Depot-only MinIO 403 quarantine)-# - COMPOSE_PROJECT_NAME pinned to posthog because bin/wait-for-docker filters by that-# project name and checkout directory names are not stable across CI providers+# - github.token can only cancel GitHub Actions runs. To stop after a deterministic failure,+# use `depot ci cancel` with DEPOT_CI_CANCEL_TOKEN.+# - The Actions API cannot look up a Depot run ID. calculate-running-time therefore uses+# .github/scripts/capture_depot_running_time.py to read `depot ci workflow show` with+# DEPOT_CI_CANCEL_TOKEN. It sends the same events and property keys as+# PostHog/posthog-github-action, with runner `depot-ci`. Attempt guards belong on steps+# because Depot evaluates `github.run_attempt` as empty in a job-level `if`.+# - The test-new-events-schema label doubles the matrices. A manual run uses only the+# events_json rows so trial runs cover the same schema as PR runs.+# - Per-segment split plans, .test_durations.core and .test_durations.temporal, remain absent+# from Depot Cache until .github/workflows/ci-backend-update-test-timing.yml saves them.+# Without those plans, shards use the combined .test_durations file.+# - actions/cache uses Depot Cache, which Depot CI and Depot GitHub Actions runners share.+# GitHub-hosted runners use a separate GitHub Actions cache.+# - Only master matrix runs save the hypothesis constants cache. PR shards rebuild the pool+# until the hourly run saves it.+# - The env block clears GITHUB_TOKEN and OTEL_SERVICE_NAME because Depot provides them by default.+# - Token steps read vars.* and secrets.* from Depot's store. If a value is missing there,+# setup-action steps with continue-on-error fall back to the default GITHUB_TOKEN, a Depot+# Code Access installation token shared across repositories. The paths-filter token step fails.+# - Migration reports post from check-migrations. Do not move them to a pull_request_target+# workflow: Depot posts that workflow's job checks on another pull request's head commit when+# both events share a base commit. See "pull_request_target checks" in+# .agents/skills/depot-ci/references/posthog-check-run-semantics.md.+# - Depot keeps ::error and ::warning annotations in the job log. GitHub receives the Repo+# checks verdict but no file annotations from `hogli lint:migration-deletions`.+# - `depot ci run` uses event_name `api`, which the workflow_dispatch conditions also handle.+# This file exceeds Depot's 64 KiB inline-workflow limit. To test a branch, use+# `depot ci dispatch --workflow ci-backend.yml --ref <branch>`.+# - Three data-modeling tests are quarantined because they receive MinIO 403 errors on Depot.+# - COMPOSE_PROJECT_NAME is pinned to posthog because bin/wait-for-docker filters by that+# project name and checkout directory names are not stable across CI providers.# - WAIT_FOR_DOCKER_REGISTRY_MIRROR makes bin/ci-wait-for-docker launch pull Docker Hub# images from mirrors before `compose up`. Token requests to auth.docker.io time out# intermittently from Depot CI sandboxes, and one failed pull leaves the whole stack missing.@@ -93,33 +67,18 @@# $RUNNER_TEMP/compose-launch.log. Depot ends a step's background processes and closes its# output pipes when the step shell exits, so a plain `&` launch dies before it creates any# container. The wait steps print that log when they fail.-# - Canonical baseline checkouts and comparisons use PostHog/posthog directly so fork-# dispatches compare against the authoritative upstream master branch-# - Docker Hub login reads vars.DOCKERHUB_USER / secrets.DOCKERHUB_TOKEN directly in the-# step `if:` rather than through canonical's env.DOCKERHUB_USERNAME / env.DOCKERHUB_TOKEN,-# because Depot never indirects through env:, so canonical's per-job env-scoping-# (workflow-level -> job-level, for the gha-workflow-env-secret Semgrep fix) is-# depot-exempt. Login is best-effort because these jobs only pull public images and can-# fall back to anonymous pulls, which canonical relies on too.-# - Path-filter persons-SQL is folded into migrations-# - handle-snapshots sets up Node from .nvmrc for its CI report steps, which canonical runs on-# the node that ships with the GitHub-hosted image, and fails with a rebase hint when the PR-# head predates .depot/actions/commit-snapshots-# - Test durations restored from Depot Cache (posthog-test-durations- prefix). Canonical's-# .github/workflows/ci-backend-update-test-timing.yml runs on Depot runners and writes-# that same cache, so shard counts match GitHub Actions. Committed .test_durations is the-# fallback.-# - The master push runs here, so the per-commit checks and the schema dump have no-# `github.event_name != 'push'` guard; the matrices keep theirs, as in canonical.--# This workflow runs all of our backend django tests.-#-# Shards derive from .test_durations (Amdahl), restored from Depot Cache (else committed);-# .github/workflows/ci-backend-update-test-timing.yml keeps that cache fresh.+# - Baseline checkouts and comparisons use PostHog/posthog directly, so a manual run from a+# fork compares against the upstream master branch.+# - Depot does not resolve env references in the Docker Hub login condition. Read+# vars.DOCKERHUB_USER and secrets.DOCKERHUB_TOKEN directly. Login can fail without stopping+# the job because these jobs pull public images and can use anonymous pulls.+# - handle-snapshots needs .depot/actions/commit-snapshots from the PR head. If that action+# is missing, the PR must rebase before CI can update its report.+# - Shard planning uses test durations from Depot Cache under the posthog-test-durations-+# prefix. .github/workflows/ci-backend-update-test-timing.yml writes the cache from Depot+# runners and uploads a `test-durations` artifact that shards use when the cache misses.+name: Backend CI on Depot-# Depot CI copy of canonical Backend CI. Branch protection requires the gate job's-# `Backend CI on Depot / Test results` check, posted by the PostHog tests app.-# Pull requests, master pushes, manual runs and the hourly master run all start here.on:push:branches:@@ -145,10 +104,9 @@ on:# labeled/unlabeled triggers, so a label takes effect from the next push# or from ready_for_review. Every pull request event, merge-queue PRs included,# runs here.- # The events_json leg runs wherever the legacy tests run, over the paths in- # .github/new-events-schema-targets.txt, the same as canonical, so a PR- # cannot pass here and fail the queue. The `test-new-events-schema` label runs- # the doubled matrices in place of that leg.+ # The events_json leg covers .github/new-events-schema-targets.txt wherever legacy+ # tests run, keeping schema coverage consistent between source PRs and the merge queue.+ # The `test-new-events-schema` label runs doubled matrices in place of that leg.types: [opened, synchronize, reopened, ready_for_review]concurrency:# PRs: one group per event; cancel-superseded-runs cancels stale runs. Push: per-SHA so master@@ -500,8 +458,7 @@ jobs:- 'posthog/migrations/*.py'- 'products/*/backend/migrations/*.py'- 'products/*/migrations/*.py' # Legacy structure- # persons-SQL folded into migrations here (see header deltas):- # depot strips canonical's separate persons-migration job+ # The persons SQL migrations share this filter; there is no separate persons-migration job.- 'rust/persons_migrations/*.sql'- 'rust/bin/migrate-persons'migrations_review:@@ -829,7 +786,6 @@ jobs:enable-cache: falsecontinue-on-error: true- name: Restore test durations from cache- # Restore fresh timings so shard counts match GitHub Actions; miss falls back to committed.uses: actions/cache/restore@cdf6c1fa76f9f475f3d7449005a359c84ca0f306 # v5.0.3with:path: .test_durations@@ -1157,10 +1113,6 @@ jobs:client-id: ${{ vars.GH_APP_POSTHOG_PRODUCT_TESTS_APP_ID }}private-key: ${{ secrets.GH_APP_POSTHOG_PRODUCT_TESTS_PRIVATE_KEY }}skip-token-revoke: true- # Sequential to mirror canonical. GitHub's native `parallel:` crashes the- # Actions runner when two branches write $GITHUB_PATH at once. Depot CI's- # `parallel:` is a separate implementation, but the shadow keeps the same- # step order so the timing trial measures the same work.- name: Set up pnpm and Nodeuses: ./.depot/actions/setup-pnpmwith:@@ -1746,7 +1698,7 @@ jobs:run: printf '%s' "$CHANGED_FILES" | xargs -r ./bin/hogli owners:lint --live# Migration validation.- # This job needs Docker + DB. Canonical master is checked out beside the comparison+ # This job needs Docker + DB. Master is checked out beside the comparison# branch so repository switches cannot disturb live Compose bind mounts.check-migrations:needs: [changes]@@ -3117,12 +3069,10 @@ jobs:(needs.turbo-discover.result != 'success' && needs.turbo-discover.result != 'skipped') ||fromJSON(needs.turbo-discover.outputs.selection || '{}').json_targets_files != '')# increase for tmate testing- # 45 to match canonical.timeout-minutes: 45name: Django tests – ${{ matrix.segment }}${{ matrix.compat && ' compat' || '' }}${{ matrix.json-targets && ' json-targets' || '' }} (persons-on-events ${{ (matrix.segment == 'Core' && !matrix.person-on-events && !matrix.compat) && 'on' || 'off' }}, events schema ${{ matrix.new-events-schema && 'json' || 'legacy' }}), Py ${{ matrix.python-version }}, ${{ matrix.clickhouse-server-image }} (${{matrix.group}}/${{ matrix.concurrency }})- # Match canonical's runner size. The shadow restores the same master-primed- # schema cache (turbo-discover key + check-migrations save), so shards skip- # migrations exactly like canonical once Depot's cache is warmed by a master run.+ # Shards restore the master-primed schema cache (turbo-discover key + check-migrations+ # save), so they skip migrations once a master run has warmed Depot Cache.runs-on: depot-ubuntu-24.04strategy:fail-fast: false@@ -3193,10 +3143,6 @@ jobs:with:path: ${{ runner.tool_cache }}/Python/${{ matrix.python-version }}key: python-toolcache-${{ runner.os }}-${{ runner.arch }}-${{ matrix.python-version }}- # Sequential to mirror canonical. GitHub's native `parallel:` crashes the- # Actions runner when two branches write $GITHUB_PATH at once. Depot CI's- # `parallel:` is a separate implementation, but the shadow keeps the same- # step order so the timing trial measures the same work.- name: Set up Pythonuses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0with:@@ -3553,7 +3499,7 @@ jobs:JSON_TARGETS_FILES: ${{ fromJSON(needs.turbo-discover.outputs.selection || '{}').json_targets_files }}# devex: measure core (posthog/ee) coverage on PRs to size the gap vs products. sysmon = low overhead.COVERAGE_CORE: sysmon- # Mirrors canonical: the coverage report skips narrowed runs, so+ # The coverage report skips narrowed runs, so# instrumenting one is pure overhead, and the events_json rows rerun# tests the legacy rows already measure.COV_ARGS: ${{ github.event_name == 'pull_request' && needs.turbo-discover.outputs.mode != 'selected' && !matrix.json-targets && '--cov=posthog --cov=ee --cov-config=.github/coverage-core.cfg --cov-report=' || '' }}@@ -3644,7 +3590,7 @@ jobs:elif [[ "$MODE" == "selected" ]]; then# Selected mode: run only the test files chosen by the snob shadow selector.# Sharded only when build_django_matrix sized the segment above one- # shard. One pytest retry mirrors canonical's budget.+ # shard. One pytest retry.if [[ "${{ matrix.compat }}" == "true" ]]; thentargets="$COMPAT_FILES"elif [[ "${{ matrix.person-on-events }}" == "true" ]]; then@@ -3759,7 +3705,7 @@ jobs:# MinIO on Depot-native CI only — MinIO itself is reachable with the standard test creds# (verified via boto3 LIST/CREATE/HEAD), so root cause is env-specific and not yet# isolated (tracked for follow-up). All three tests below run that materialize path; the- # two timeout tests fail downstream (hogql_table never reached). Canonical GHA still runs them.+ # two timeout tests fail downstream (hogql_table never reached).# The --deselect guards stay on both branches so a selected run can't re-enable them.if [[ "$MODE" == "selected" ]]; then# Selected mode uses the same single-retry budget as the full run.@@ -4178,7 +4124,7 @@ jobs:runs-on: depot-ubuntu-24.04timeout-minutes: 8# !cancelled() collates skipped/failed matrix jobs without reporting failure on a- # superseded run (mirrors canonical).+ # superseded run.if: >!cancelled()|| needs.repo-checks.outputs.deterministic_failure == 'true'diff --git a/.github/scripts/desktop/check-pr-backend-coupling.sh b/.github/scripts/desktop/check-pr-backend-coupling.shindex 0cc02e54f9ff..44cfba91b72f 100755--- a/.github/scripts/desktop/check-pr-backend-coupling.sh+++ b/.github/scripts/desktop/check-pr-backend-coupling.sh@@ -9,11 +9,9 @@ set -euo pipefailREPOSITORY="${REPOSITORY:?}"PR_NUMBER="${PR_NUMBER:?}"-# Deploy-relevant subset of ci-backend.yml's `backend` filter (paths that ship-# in the Django image) plus rust/** (rust services deploy separately and also-# serve clients). Tooling/test entries (pyproject.toml, uv.lock, conftest.py,-# docker-compose*, ...) are deliberately excluded so they cannot block an-# unrelated desktop PR.+# Match deployable Django paths in .depot/workflows/ci-backend.yml's `backend` filter.+# Include rust/** because Rust services deploy separately and serve desktop clients.+# Exclude tooling and tests so they cannot block an unrelated desktop PR.is_backend_path() {case "$1" in*.md | *.mdx) return 1 ;;diff --git a/.github/workflows/cd-sandbox-base-image.yml b/.github/workflows/cd-sandbox-base-image.ymlindex d335ee9e997c..554c16e26620 100644--- a/.github/workflows/cd-sandbox-base-image.yml+++ b/.github/workflows/cd-sandbox-base-image.yml@@ -294,8 +294,8 @@ jobs:env:BASE_REF: ${{ github.event.pull_request.base.ref }}IS_PR: ${{ github.event_name == 'pull_request' }}- # Manual cache-bust knob. Keep in sync with SCHEMA_CACHE_EPOCH in- # ci-backend.yml, which saves these caches on master pushes.+ # Keep SCHEMA_CACHE_EPOCH in sync with .depot/workflows/ci-backend.yml+ # so both workflows use the same schema cache keys.SCHEMA_CACHE_EPOCH: v2run: |if [ "$IS_PR" != "true" ]; then@@ -333,7 +333,7 @@ jobs:if [ "$HEAD_AGE_SECONDS" -lt 86400 ]; thenecho "prefix_key=posthog-schema-mig-${SCHEMA_CACHE_EPOCH}-" >> $GITHUB_OUTPUTfi- # Must match ci-backend.yml's save-side computation byte for byte or the key never hits.+ # Match the inputs and hashing in .depot/workflows/ci-backend.yml to find its schema dump.MIG_FILES=$(git ls-tree -r --format='%(objectname) %(path)' "$REF" \| grep -E '/migrations/[^/]+\.py$|^[0-9a-f]+ uv\.lock$' \| grep -vE '/(clickhouse|async_migrations)/migrations/' \diff --git a/.github/workflows/ci-agent-skills.yml b/.github/workflows/ci-agent-skills.ymlindex d49509161d74..149a6a436e8b 100644--- a/.github/workflows/ci-agent-skills.yml+++ b/.github/workflows/ci-agent-skills.yml@@ -125,8 +125,8 @@ jobs:env:BASE_REF: ${{ github.event.pull_request.base.ref }}IS_PUSH: ${{ github.event_name == 'push' }}- # Manual cache-bust knob. Keep in sync with SCHEMA_CACHE_EPOCH in- # ci-backend.yml, which saves these caches on master pushes.+ # Keep SCHEMA_CACHE_EPOCH in sync with .depot/workflows/ci-backend.yml+ # so both workflows use the same schema cache keys.SCHEMA_CACHE_EPOCH: v2run: |if [ "$IS_PUSH" = "true" ]; then@@ -164,7 +164,7 @@ jobs:if [ "$HEAD_AGE_SECONDS" -lt 86400 ]; thenecho "prefix_key=posthog-schema-mig-${SCHEMA_CACHE_EPOCH}-" >> $GITHUB_OUTPUTfi- # Must match ci-backend.yml's save-side computation byte for byte or the key never hits.+ # Match the inputs and hashing in .depot/workflows/ci-backend.yml to find its schema dump.MIG_FILES=$(git ls-tree -r --format='%(objectname) %(path)' "$REF" \| grep -E '/migrations/[^/]+\.py$|^[0-9a-f]+ uv\.lock$' \| grep -vE '/(clickhouse|async_migrations)/migrations/' \diff --git a/.github/workflows/ci-dagster.yml b/.github/workflows/ci-dagster.ymlindex fe461f295bcb..8fb51fa13741 100644--- a/.github/workflows/ci-dagster.yml+++ b/.github/workflows/ci-dagster.yml@@ -27,7 +27,7 @@ permissions:pull-requests: readenv:- # Manual cache-bust knob. Keep in sync with SCHEMA_CACHE_EPOCH in ci-backend.yml, which saves these caches on master pushes.+ # Keep SCHEMA_CACHE_EPOCH in sync with .depot/workflows/ci-backend.yml so schema cache keys match.SCHEMA_CACHE_EPOCH: v2SECRET_KEY: '6b01eee4f945ca25045b5aab440b953461faf08693a9abbf1166dc7c6b9772da' # unsafe - for testing onlyDATABASE_URL: 'postgres://posthog:posthog@localhost:5432/posthog'@@ -365,7 +365,7 @@ jobs:if [ "$HEAD_AGE_SECONDS" -lt 86400 ]; thenecho "prefix_key=posthog-schema-mig-${SCHEMA_CACHE_EPOCH}-" >> $GITHUB_OUTPUTfi- # Shared migration-set key. Must match ci-backend.yml's save-side computation byte for byte or it never hits.+ # Match the inputs and hashing in .depot/workflows/ci-backend.yml to find its schema dump.MIG_FILES=$(git ls-tree -r --format='%(objectname) %(path)' "$MERGE_BASE" \| grep -E '/migrations/[^/]+\.py$|^[0-9a-f]+ uv\.lock$' \| grep -vE '/(clickhouse|async_migrations)/migrations/' \diff --git a/.github/workflows/ci-e2e-playwright.yml b/.github/workflows/ci-e2e-playwright.ymlindex 957a316aa714..5b4b24e656c8 100644--- a/.github/workflows/ci-e2e-playwright.yml+++ b/.github/workflows/ci-e2e-playwright.yml@@ -314,8 +314,8 @@ jobs:env:BASE_REF: ${{ github.event.pull_request.base.ref }}IS_PR: ${{ github.event_name == 'pull_request' }}- # Manual cache-bust knob — keep in sync with SCHEMA_CACHE_EPOCH in- # ci-backend.yml, which saves these caches on master pushes.+ # Keep SCHEMA_CACHE_EPOCH in sync with .depot/workflows/ci-backend.yml+ # so both workflows use the same schema cache keys.SCHEMA_CACHE_EPOCH: v2run: |if [ "$IS_PR" != "true" ]; then@@ -357,11 +357,9 @@ jobs:if [ "$GITHUB_EVENT_NAME" != "workflow_dispatch" ] && [ "$HEAD_AGE_SECONDS" -lt 86400 ]; thenecho "prefix_key=posthog-schema-mig-${SCHEMA_CACHE_EPOCH}-" >> $GITHUB_OUTPUTfi- # Shared migration-set key: mirrors the save-side computation in- # ci-backend.yml (same inputs, hash, and prefix) so it hits whenever- # the ref's migration files match a saved master schema — far more- # stable than the per-SHA key, which needs that exact commit's cache- # to still be alive.+ # Match the inputs, hash, and prefix in .depot/workflows/ci-backend.yml.+ # The migration-set key lets refs with identical migrations share a schema+ # dump even after a per-SHA cache expires.MIG_FILES=$(git ls-tree -r --format='%(objectname) %(path)' "$REF" \| grep -E '/migrations/[^/]+\.py$|^[0-9a-f]+ uv\.lock$' \| grep -vE '/(clickhouse|async_migrations)/migrations/' \diff --git a/.github/workflows/ci-master-run-traces.yml b/.github/workflows/ci-master-run-traces.ymlindex 34d2d847cd40..95f07bc03c54 100644--- a/.github/workflows/ci-master-run-traces.yml+++ b/.github/workflows/ci-master-run-traces.yml@@ -4,8 +4,8 @@ name: Master run traces# a span per job beneath it, a span per step beneath those. Covers every workflow,# including the ones with no telemetry of their own (Container Images CD, the# container CD lane, the Rust image builds). Master pushes are scanned repo-wide;-# Backend CI's push and scheduled runs are read from Depot. Depot exposes workflow and job-# timestamps, but not step or job-queue timestamps, so those traces stop at jobs.+# Backend CI's push and scheduled runs come from Depot, which exposes workflow and job+# timestamps. Those traces stop at jobs because Depot provides no step or job-queue timestamps.## Complements .github/scripts/report_test_timings.py, which emits per-test traces# from JUnit XML under its own trace IDs. Those stay separate traces.@@ -49,10 +49,8 @@ jobs:runs-on: ubuntu-24.04timeout-minutes: 20steps:- # A cron workflow always runs the default-branch copy of this file, so the- # checked-out script is master's. The base-ref pinning that ci-backend.yml's- # report-test-timings needs guards against a PR swapping the script that- # receives the project token, which cannot happen here.+ # Scheduled runs check out master, so PR code cannot replace the reporter+ # that receives the project token.- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2with:persist-credentials: false@@ -62,10 +60,9 @@ jobs:.github/actions/setup-uvsparse-checkout-cone-mode: false- # Reads job durations from the Actions API — the same purpose as- # ci-backend.yml's calculate-running-time, so it shares that bucket rather- # than the posthog-tests app ci-alerts-devex already bursts on at merge- # peaks. A failed mint degrades to github.token instead of skipping the tick.+ # Share the telemetry app with the frontend, Storybook, and Playwright+ # calculate-running-time jobs to avoid more traffic on the posthog-tests+ # app used by ci-alerts-devex. Fall back to github.token if token creation fails.- name: Mint telemetry GitHub tokenid: app-tokencontinue-on-error: truediff --git a/.github/workflows/ci-mcp.yml b/.github/workflows/ci-mcp.ymlindex 6142d6e36655..3d0f9b254cee 100644--- a/.github/workflows/ci-mcp.yml+++ b/.github/workflows/ci-mcp.yml@@ -32,7 +32,7 @@ permissions:pull-requests: readenv:- # Manual cache-bust knob. Keep in sync with SCHEMA_CACHE_EPOCH in ci-backend.yml, which saves these caches on master pushes.+ # Keep SCHEMA_CACHE_EPOCH in sync with .depot/workflows/ci-backend.yml so schema cache keys match.SCHEMA_CACHE_EPOCH: v2RUNS_ON_INTERNAL_PR: ${{ github.event_name != 'pull_request' || github.event.pull_request.head.repo.fork == false }}@@ -560,7 +560,7 @@ jobs:if [ "$HEAD_AGE_SECONDS" -lt 86400 ]; thenecho "prefix_key=posthog-schema-mig-${SCHEMA_CACHE_EPOCH}-" >> $GITHUB_OUTPUTfi- # Shared migration-set key. Must match ci-backend.yml's save-side computation byte for byte or it never hits.+ # Match the inputs and hashing in .depot/workflows/ci-backend.yml to find its schema dump.MIG_FILES=$(git ls-tree -r --format='%(objectname) %(path)' "$MERGE_BASE" \| grep -E '/migrations/[^/]+\.py$|^[0-9a-f]+ uv\.lock$' \| grep -vE '/(clickhouse|async_migrations)/migrations/' \diff --git a/.github/workflows/ci-rust-flags-integration.yml b/.github/workflows/ci-rust-flags-integration.ymlindex f2ec72731490..a1d109f72918 100644--- a/.github/workflows/ci-rust-flags-integration.yml+++ b/.github/workflows/ci-rust-flags-integration.yml@@ -39,7 +39,7 @@ concurrency:cancel-in-progress: ${{ github.event_name == 'pull_request' }}env:- # Cache-bust knob. Keep in sync with ci-backend.yml, which saves these caches.+ # Keep SCHEMA_CACHE_EPOCH in sync with .depot/workflows/ci-backend.yml so schema cache keys match.SCHEMA_CACHE_EPOCH: v2# Django SECRET_KEY for cryptographic signing (sessions, CSRF, etc.) - test-only, not used in productionSECRET_KEY: '6b01eee4f945ca25045b5aab440b953461b81b72'@@ -164,7 +164,7 @@ jobs:if [ "$HEAD_AGE_SECONDS" -lt 86400 ]; thenecho "prefix_key=posthog-schema-mig-${SCHEMA_CACHE_EPOCH}-" >> $GITHUB_OUTPUTfi- # Must match ci-backend.yml's save-side computation byte for byte or the key never hits.+ # Match the inputs and hashing in .depot/workflows/ci-backend.yml to find its schema dump.MIG_FILES=$(git ls-tree -r --format='%(objectname) %(path)' "$MERGE_BASE" \| grep -E '/migrations/[^/]+\.py$|^[0-9a-f]+ uv\.lock$' \| grep -vE '/(clickhouse|async_migrations)/migrations/' \diff --git a/docs/internal/ci-things-already-tried.md b/docs/internal/ci-things-already-tried.mdindex a4fbd44ec961..5622a1e51374 100644--- a/docs/internal/ci-things-already-tried.md+++ b/docs/internal/ci-things-already-tried.md@@ -209,7 +209,7 @@ Two things make this look inactive:The selection was also active for draft PRs only for some time. The team wanted a stable merge queue first.[#85530](https://github.com/PostHog/posthog/pull/85530) then extended the selection to PRs that are ready for review. [#88265](https://github.com/PostHog/posthog/pull/88265) put the Django selection and the product selection in one job.-Read the comment at the top of `.github/workflows/ci-backend.yml` for the current rules.+The comment under `pull_request:` in `.depot/workflows/ci-backend.yml` explains which tests run on each type of pull request.The jest and Playwright suites made the same move later, so `ready_for_review` no longer buys a full matrix anywhere.Selection now runs on drafts and ready PRs alike in `ci-backend.yml`, `ci-frontend.yml`, `ci-storybook.yml`, and `ci-e2e-playwright.yml`, and the merge queue's `trunk-merge/**` run is the only full gate.diff --git a/posthog/management/commands/test_ch_migrations_are_safe.py b/posthog/management/commands/test_ch_migrations_are_safe.pyindex ea6e5f357edd..3e6989bf0657 100644--- a/posthog/management/commands/test_ch_migrations_are_safe.py+++ b/posthog/management/commands/test_ch_migrations_are_safe.py@@ -86,7 +86,7 @@ class Command(BaseCommand):sys.exit(1)if sys.stdin.isatty():- logger.warning("Not running migration-specific checks. See .github/workflows/ci-backend.yml for usage.")+ logger.warning("Not running migration-specific checks. See .depot/workflows/ci-backend.yml for usage.")returnmigration_path_re = re.compile(r"^[a-z]+/clickhouse/migrations/[0-9]+_[a-zA-Z_0-9]+\.py$")diff --git a/posthog/test/repo_invariants/README.md b/posthog/test/repo_invariants/README.mdindex c3d3f33b2148..55b1623ffaef 100644--- a/posthog/test/repo_invariants/README.md+++ b/posthog/test/repo_invariants/README.md@@ -1,15 +1,11 @@# Repo invariants-Tests here take the whole repo as input: the import graph after `django.setup()`,-`apps.get_models()`, signal receiver registries, repo-wide AST scans against a-baseline. Any file anywhere can break them, and diff-based test selection has no-way to pick them from a diff.+Tests here inspect the whole repository through the import graph after `django.setup()`, `apps.get_models()`, signal receiver registries, and AST scans against a baseline.+Any file can break them, so selecting tests by changed paths cannot reliably include them.-So CI runs this directory unconditionally on every backend-touching PR, including-products-only diffs and drafts, as a step of the `repo-checks` job in-`.github/workflows/ci-backend.yml`. There is no Postgres or ClickHouse in that-job, and the Core Django shards skip this directory.+The `repo-checks` job in `.depot/workflows/ci-backend.yml` therefore runs this directory on every PR that touches backend code, including products-only diffs and drafts.+That job has no Postgres or ClickHouse, and the Core Django shards skip this directory.-Rules: no database or ClickHouse fixtures. A test whose input is a bounded set of-files belongs next to the code it covers, not here. Regenerate baselines with the-commands the tests print when they fail.+Do not use database or ClickHouse fixtures here.+Put tests that inspect a bounded set of files next to the code they cover.+Regenerate baselines with the commands the failing tests print.diff --git a/products/engineering_analytics/SPEC.md b/products/engineering_analytics/SPEC.mdindex 596863db7c33..a4038dee6bb7 100644--- a/products/engineering_analytics/SPEC.md+++ b/products/engineering_analytics/SPEC.md@@ -200,7 +200,17 @@ Warehouse table (Depot source):- `depot_job_attempts`: one row per Depot CI job attempt, for the repository the Depot source names. Reads keep only that repository's rows, because a source moved to another repository keeps what it synced before. Depot CI is its own engine, so these runs never reach the GitHub tables. `logic/views/depot_ci.py` reshapes the attempts into rows of the GitHub runs and jobs contracts and unions them onto those tables, so every builder reads both engines. Each Depot workflow is one runs row. A run with one workflow decodes its id to the integer `GITHUB_RUN_ID` Depot CI gives its jobs, which joins them to the CI test spans; a run with several workflows keys each by its own decoded workflow id, so no join fans out. The PR number comes from the run's `refs/pull/<n>/merge` ref, and the run's branch and actor from that PR's snapshot (its head ref and author). A merge-queue gate run's PR is the gate PR the queue bot opened, so the run resolves through `logic/merge_queue.py` to the PR being landed, like a GitHub gate run (§6). Job rows carry no branch, because a PR snapshot lookup in the jobs builder would add two full PR scans to every jobs read; a read that joins a job to its run takes the branch through `workflow_jobs.branch`, which falls back to the run's. Depot numbers attempts per job, not per run, so a job's last attempt is listed again under each later attempt of its run, as GitHub lists a job it did not re-run; the jobs builder flags those rows as `is_rerun_copy`. Depot's API reports no `runs-on`, so every attempt is costed as the default `depot-ubuntu-24.04` sandbox. Optional, and reads degrade to GitHub-only CI when unsynced.- Depot ingestion replays seven days behind the creation-time watermark on every incremental sync and reconciles weekly. A reconciliation reads runs created up to 30 days before the previous reconciliation, the window in which GitHub Actions allows a re-run; Depot CI documents no window of its own. The first reconciliation, a full refresh, and a sync without a watermark read everything since the fixed `history_start`; a legacy source without that bound reads all available history. The durable reconciliation cursor advances only after extraction finishes and the pipeline makes the rows durable. An interrupted sync resumes after the last run it handed off and keeps the window of its first attempt, so a long reconciliation finishes across worker restarts. Completed runs do not wait behind queued or stuck runs. Late retries older than the replay window can remain stale until the next reconciliation. Replayed attempts merge by `attempt_id`; append mode is unsupported, and an existing append sync stops before extraction with instructions to change its sync method. Depot's existing two-page-size pagination workaround still cannot guarantee recovery of every same-second cursor omission.- The shared CI sources assign `ci_engine` from the warehouse source: `github_actions` or `depot_ci`. Runner provider is independent: GitHub Actions can use Depot runners. Managed job-history and job-cost views append `ci_engine`, `native_run_id`, `native_workflow_run_id`, `native_job_id`, and `native_attempt_id`. GitHub's native workflow execution ID is its run ID; Depot's is its workflow ID. These additive string IDs preserve source identity while the existing integer IDs remain compatible with test traces and API consumers. CI job-to-run joins, failure attribution, job run shares, and timeline paging qualify integer IDs with `ci_engine`. The friction view consumes the same normalized sources and resolves a Depot job's default branch through its run. Run and job APIs append engine and source-native IDs; per-run costs and UI caches qualify compatible integers with the engine. Detail, job, and log lookups accept `ci_engine`; an unqualified integer that names both engines fails with a 400. New failure logs and test traces stamp the engine. Recovery evidence groups by engine, native workflow identity, and commit SHA; historical untagged telemetry remains unknown and cannot establish cross-engine identity. Legacy logs are read only for integer IDs unique to one engine in the selected source. Depot workflows without a hand-off job remain visible, including default-branch pushes; their branch and commit come from `ref` and `sha` when no PR head is present.- - While a workflow moves to Depot CI, GitHub Actions routes each pull request to one engine, and both engines record a run of the commit. The union suppresses a successful Depot workflow whose `wait-for-handoff` job ran alone. A GitHub run is suppressed only when it succeeded, its `Hand off backend tests to Depot CI` job succeeded, its latest `Django Tests Pass` job succeeded, no synced job attempt decisively failed, and a successful Depot workflow took the hand-off for the same PR number and `head_sha`. The PR number on both sides is the run's own association, because a merge-queue gate run's `pr_number` names the landed PR while Depot's ref names the gate PR. Missing association or replacement data leaves the GitHub run visible. Failed, cancelled and unsettled GitHub relays and Depot waits stay visible. A relay that recovered retains its earlier failure history. Suppression applies to both runs and jobs; raw source rows remain available for full runner-cost accounting. Curated estimates cover the visible execution jobs; suppressed hand-off and relay overhead is excluded. These figures are not total runner spend or invoice reconciliation.+ - Historical relay runs need deduplication. During Backend CI's migration to Depot CI in 2026, GitHub Actions routed each pull request to one engine, and both engines recorded a run of the commit.+ The union suppresses a successful Depot workflow whose `wait-for-handoff` job ran alone.+ To suppress a GitHub run, the run itself, its `Hand off backend tests to Depot CI` job, and its latest `Django Tests Pass` job must all have succeeded.+ No synced job attempt may have decisively failed.+ A successful Depot workflow must also have accepted the hand-off for the same PR number and `head_sha`.+ Both sides use the run's own PR association for this comparison.+ A merge-queue run's normalized `pr_number` names the PR being landed, while Depot's ref names the temporary gate PR, so those values cannot be compared directly.+ Missing association or replacement data leaves the GitHub run visible.+ Failed, canceled, and unsettled relays and waits stay visible, and a recovered relay retains its earlier failure history.+ Suppression applies to runs and jobs; raw source rows remain available for full runner-cost accounting.+ Curated cost estimates cover visible execution jobs and exclude suppressed hand-off and relay overhead, so they cannot establish total runner spend or reconcile an invoice.The Depot source is attempt-grained. A terminal run with no executed job attempts has no row here, sothe normalized run inventory cannot establish total run completeness. Vendor pagination can omit runsdiff --git a/products/engineering_analytics/backend/logic/views/depot_ci.py b/products/engineering_analytics/backend/logic/views/depot_ci.pyindex 4c0d9f90af52..471e2f0117eb 100644--- a/products/engineering_analytics/backend/logic/views/depot_ci.py+++ b/products/engineering_analytics/backend/logic/views/depot_ci.py@@ -73,8 +73,7 @@ _DEFAULT_SANDBOX_LABELS = '["depot-ubuntu-24.04"]'# pull request association as the run's own._REPOSITORY_ID = 1-# The hand-off between the engines, as .github/workflows/ci-backend.yml and .depot/workflows/ci-backend.yml-# name its two ends.+# Historical job names identify relay and hand-off rows that the union must deduplicate._GITHUB_HANDOFF_JOB = "Hand off backend tests to Depot CI"_GITHUB_RELAY_JOB = "Django Tests Pass"_DEPOT_WAIT_JOB_KEY_SUFFIX = ":wait-for-handoff"diff --git a/tools/hogli-commands/hogli_commands/product/checks.py b/tools/hogli-commands/hogli_commands/product/checks.pyindex 8c4a1d7d301d..0ca70367af89 100644--- a/tools/hogli-commands/hogli_commands/product/checks.py+++ b/tools/hogli-commands/hogli_commands/product/checks.py@@ -1275,10 +1275,10 @@ class OrphanedTestFilesCheck(ProductCheck):_EXTERNAL_RUNNER_PREFIXES = ("dags/", # ci-dagster.yml: pytest posthog/dags products/**/dags)- # Per-product exemptions — paths that another workflow targets directly- # (e.g. ci-backend.yml's Temporal segment) rather than `backend:test`.+ # Exempt paths tested outside `backend:test`, including the Temporal tests in+ # .depot/workflows/ci-backend.yml._PRODUCT_SPECIFIC_EXEMPTIONS = {- # ci-backend.yml "Run Temporal tests" step pytest paths:+ # .depot/workflows/ci-backend.yml "Run Temporal tests" step pytest paths:"batch_exports": ("backend/tests/temporal/",),"tasks": ("backend/temporal/",),"warehouse_sources": ("backend/temporal/",),diff --git a/tools/workflow-plan/README.md b/tools/workflow-plan/README.mdindex f66668bd4d01..83134bfcb296 100644--- a/tools/workflow-plan/README.md+++ b/tools/workflow-plan/README.md@@ -6,8 +6,8 @@ The evaluator is GitHub's own [`@actions/expressions`](https://www.npmjs.com/pac## See a workflow's run plan```bash-hogli ci:plan .github/workflows/ci-backend.yml-hogli ci:plan .github/workflows/ci-backend.yml --steps changes+hogli ci:plan .depot/workflows/ci-backend.yml+hogli ci:plan .depot/workflows/ci-backend.yml --steps changes```The table has one row per job and one column per built-in scenario: `draft`, `ready`, and `fork` pull requests, a `queued` merge-queue run, a `merged` push to master, a `scheduled` run, and a `dispatched` run.@@ -23,9 +23,11 @@ Where a workflow cannot be planned without such a value, `SCRIPT_STUBS` in `src/hogli test:workflows```-`tests/workflows.test.ts` holds one row per workflow and scenario, naming the jobs that must run and the jobs that must skip.-The rows encode the rules in [the authoring skill](../../.agents/skills/authoring-ci-workflows/SKILL.md): drafts skip the product matrix, the merge queue takes the full one, forks skip telemetry, a `no-ci` draft still reports its gate, the hourly schedule runs the matrices and skips the PR-only checks, and a superseded run records its gate as `cancelled` rather than `failure`.-A coverage test keeps those rows exhaustive: every job in `ci-backend.yml` and `ci-frontend.yml` that carries an `if:` must be named by at least one row, so a new conditional job fails the suite until someone states what it should do.+`tests/workflows.test.ts` names the jobs that must run or skip for each workflow and scenario.+The expectations follow [the authoring skill](../../.agents/skills/authoring-ci-workflows/SKILL.md).+They cover draft and merge-queue matrices, fork telemetry, `no-ci` draft gates, hourly runs, and cancellation verdicts.+Every job with an `if:` in `.depot/workflows/ci-backend.yml` or `ci-frontend.yml` must appear in at least one expectation.+The coverage test fails when a new conditional job has no expectation.A smoke test also feeds every workflow through the built-in scenarios and fails on any condition the evaluator rejects.Add a row when you add or change a condition.