:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
ci(images): bind the data warehouse metadata worker to its own channel (#123678)
Stephen Schmidt600e7d5bfda2
diff --git a/.agents/skills/gating-production-deploys/references/ordered-image-releases.md b/.agents/skills/gating-production-deploys/references/ordered-image-releases.mdindex 742bb97faa3a..88e640d4c7a3 100644--- a/.agents/skills/gating-production-deploys/references/ordered-image-releases.md+++ b/.agents/skills/gating-production-deploys/references/ordered-image-releases.md@@ -4,7 +4,7 @@ Use this reference when reusing the common ordered alias helper or the shared-im## Current capability-The common alias uses `.github/scripts/ordered-image-release.sh`. `.github/actions/paths-filter` owns canonical manifest schema loading in `src/shared-image-policy.ts` and legacy projection. `.github/actions/shared-image-releases` owns `inspect`, `publish`, source publication bindings, the interval adapter and fingerprint APIs. It imports the loader and vendored matcher from paths-filter. The new action has no Depot mirror. Channels stay empty until separately reviewed activation; manifest entries name provisioned origin repositories.+The common alias uses `.github/scripts/ordered-image-release.sh`. `.github/actions/paths-filter` owns canonical manifest schema loading in `src/shared-image-policy.ts` and legacy projection. `.github/actions/shared-image-releases` owns `inspect`, `publish`, source publication bindings, the interval adapter and fingerprint APIs. It imports the loader and vendored matcher from paths-filter. The new action has no Depot mirror. Channels are added only through separately reviewed activation; manifest entries name provisioned origin repositories.## Common ordered alias@@ -33,7 +33,7 @@ Image annotations and consumer `source.repo` keep canonical `PostHog/posthog` idMove each reused filter into `.github/shared-image-releases.yaml` once. The legacy action's `shared-image-policy` input calls `loadPolicy(text)` and `projectLegacyFilters(policy, inlineYaml)`. Preserve legacy endpoint comparisons, output names, ordering and compound dispatch conditions, including Python dependency analysis and coverage-added checks.-Keep a `dataWarehouse: null` reference at that rule's original inline position. The projection fills it from the manifest while preserving every non-null inline rule. Unresolved references fail; nonempty duplicate definitions are rejected. Metadata's ten-pattern historical fixture remains distinct from the broader active `dataWarehouse` rule and is not an active channel.+Keep a `dataWarehouse: null` reference at that rule's original inline position. The projection fills it from the manifest while preserving every non-null inline rule. Unresolved references fail; nonempty duplicate definitions are rejected. `dataWarehouseMetadata` is an active channel filter, distinct from the broader `dataWarehouse` rule that still gates two legacy workers, with its `bootstrapPolicyDigest` pinned at adoption.Version 1 has `version`, `filters` and `images`; images contain `channels`. Each channel has `repository`, `filter` and optional `bootstrapPolicyDigest`. `loadPolicy` validates the closed shape, references, positive patterns and nonempty channel status qualifiers limited to `added`, `modified` and `deleted`. The vendored matcher owns extglob/negation semantics; legacy-only veto and status rules retain prior behavior. Structural validation does not authorize publication.diff --git a/.depot/actions/paths-filter/__tests__/fixtures/metadata-removal-parent.yaml b/.depot/actions/paths-filter/__tests__/fixtures/metadata-removal-parent.yamldeleted file mode 100644index d8c54853b5dc..000000000000--- a/.depot/actions/paths-filter/__tests__/fixtures/metadata-removal-parent.yaml+++ /dev/null@@ -1,11 +0,0 @@-dataWarehouseMetadata:- - 'posthog/hogql*/**'- - 'posthog/temporal/data_modeling/**'- - 'products/customer_analytics/backend/**'- - 'products/warehouse_sources/backend/temporal/data_imports/**'- - 'posthog/warehouse/**'- - 'posthog/temporal/common/**'- - 'posthog/management/commands/start_temporal_worker.py'- - 'pyproject.toml'- - 'bin/temporal-django-worker'- - 'posthog/clickhouse/migrations/**'diff --git a/.depot/actions/paths-filter/__tests__/shared-image-policy.test.ts b/.depot/actions/paths-filter/__tests__/shared-image-policy.test.tsindex a4790c5d533f..b5ebe2124466 100644--- a/.depot/actions/paths-filter/__tests__/shared-image-policy.test.ts+++ b/.depot/actions/paths-filter/__tests__/shared-image-policy.test.ts@@ -119,17 +119,15 @@ describe('shared-image policy and legacy projection', () => {expect(new Filter(projected).match(files)).toEqual(new Filter(yaml.dump({...yaml.load(inline) as object, ...policy.filters})).match(files))})- test('metadata removal-parent coverage stays distinct from the broader active warehouse rule', () => {- const historical = readFileSync(path.join(__dirname, 'fixtures/metadata-removal-parent.yaml'), 'utf8')- const current = productionPolicy()- const filter = new Filter(projectLegacyFilters(current, historical))+ test('the metadata channel filter and the broader active warehouse rule stay distinct', () => {+ const filter = new Filter(projectLegacyFilters(productionPolicy(), '{}'))const files = [- {filename: 'products/warehouse_sources/backend/temporal/data_imports/source.py', status: ChangeStatus.Modified},- {filename: 'products/warehouse_sources/backend/models.py', status: ChangeStatus.Modified},+ {filename: 'products/data_modeling/backend/views.py', status: ChangeStatus.Modified},+ {filename: 'products/warehouse_sources_queue/consumer.py', status: ChangeStatus.Modified},{filename: 'Dockerfile', status: ChangeStatus.Modified}]const result = filter.match(files)- expect(result.dataWarehouseMetadata).toEqual([files[0]])- expect(result.dataWarehouse).toEqual(files)+ expect(result.dataWarehouseMetadata).toEqual([files[0], files[2]])+ expect(result.dataWarehouse).toEqual([files[1], files[2]])})})diff --git a/.github/actions/paths-filter/__tests__/fixtures/metadata-removal-parent.yaml b/.github/actions/paths-filter/__tests__/fixtures/metadata-removal-parent.yamldeleted file mode 100644index d8c54853b5dc..000000000000--- a/.github/actions/paths-filter/__tests__/fixtures/metadata-removal-parent.yaml+++ /dev/null@@ -1,11 +0,0 @@-dataWarehouseMetadata:- - 'posthog/hogql*/**'- - 'posthog/temporal/data_modeling/**'- - 'products/customer_analytics/backend/**'- - 'products/warehouse_sources/backend/temporal/data_imports/**'- - 'posthog/warehouse/**'- - 'posthog/temporal/common/**'- - 'posthog/management/commands/start_temporal_worker.py'- - 'pyproject.toml'- - 'bin/temporal-django-worker'- - 'posthog/clickhouse/migrations/**'diff --git a/.github/actions/paths-filter/__tests__/shared-image-policy.test.ts b/.github/actions/paths-filter/__tests__/shared-image-policy.test.tsindex a4790c5d533f..b5ebe2124466 100644--- a/.github/actions/paths-filter/__tests__/shared-image-policy.test.ts+++ b/.github/actions/paths-filter/__tests__/shared-image-policy.test.ts@@ -119,17 +119,15 @@ describe('shared-image policy and legacy projection', () => {expect(new Filter(projected).match(files)).toEqual(new Filter(yaml.dump({...yaml.load(inline) as object, ...policy.filters})).match(files))})- test('metadata removal-parent coverage stays distinct from the broader active warehouse rule', () => {- const historical = readFileSync(path.join(__dirname, 'fixtures/metadata-removal-parent.yaml'), 'utf8')- const current = productionPolicy()- const filter = new Filter(projectLegacyFilters(current, historical))+ test('the metadata channel filter and the broader active warehouse rule stay distinct', () => {+ const filter = new Filter(projectLegacyFilters(productionPolicy(), '{}'))const files = [- {filename: 'products/warehouse_sources/backend/temporal/data_imports/source.py', status: ChangeStatus.Modified},- {filename: 'products/warehouse_sources/backend/models.py', status: ChangeStatus.Modified},+ {filename: 'products/data_modeling/backend/views.py', status: ChangeStatus.Modified},+ {filename: 'products/warehouse_sources_queue/consumer.py', status: ChangeStatus.Modified},{filename: 'Dockerfile', status: ChangeStatus.Modified}]const result = filter.match(files)- expect(result.dataWarehouseMetadata).toEqual([files[0]])- expect(result.dataWarehouse).toEqual(files)+ expect(result.dataWarehouseMetadata).toEqual([files[0], files[2]])+ expect(result.dataWarehouse).toEqual([files[1], files[2]])})})diff --git a/.github/shared-image-releases.yaml b/.github/shared-image-releases.yamlindex 5fea5235aefb..ec672fc4a3e7 100644--- a/.github/shared-image-releases.yaml+++ b/.github/shared-image-releases.yaml@@ -18,6 +18,32 @@ filters:- pyproject.toml- bin/temporal-django-worker- 'posthog/clickhouse/migrations/**'+ dataWarehouseMetadata:+ - 'posthog/hogql*/**'+ - 'posthog/temporal/data_modeling/**'+ - 'products/customer_analytics/backend/**'+ - 'products/warehouse_sources/backend/**'+ - 'posthog/temporal/common/**'+ - 'posthog/management/commands/start_temporal_worker.py'+ - pyproject.toml+ - bin/temporal-django-worker+ - 'posthog/clickhouse/migrations/**'+ - 'products/data_modeling/backend/**'+ - 'posthog/llm/**'+ - 'products/data_warehouse/backend/**'+ - 'posthog/personhog_client/**'+ - 'packages/personhog-proto/**'+ - 'posthog/kafka_client/**'+ - 'common/hogvm/**'+ - 'posthog/models/**'+ - 'posthog/migrations/**'+ - 'products/*/backend/migrations/**'+ - posthog/schema.py+ - 'posthog/settings/temporal.py'+ - posthog/exceptions.py+ - uv.lock+ - manage.py+ - DockerfileexperimentsRecalculationActivity:- 'products/experiments/backend/**'- 'products/experiments/stats/**'@@ -49,3 +75,7 @@ images:repository: posthog-temporal-workers/experiments-recalculation-activityfilter: experimentsRecalculationActivitybootstrapPolicyDigest: a06fc277bfb801b6da6406ac2a8d0ed82ac09c13e1ac37234c8a4e01b44b1313+ data-warehouse-metadata:+ repository: posthog-temporal-workers/data-warehouse-metadata+ filter: dataWarehouseMetadata+ bootstrapPolicyDigest: 49975342dfa3dce5f0c4767e2bc90c295a7ce12cfb5daaa99075026f20dd3620diff --git a/posthog/management/commands/start_temporal_worker.py b/posthog/management/commands/start_temporal_worker.pyindex f2a857bc443c..5d6f92120e00 100644--- a/posthog/management/commands/start_temporal_worker.py+++ b/posthog/management/commands/start_temporal_worker.py@@ -361,8 +361,9 @@ from products.wizard.backend.facade.temporal import (WORKFLOWS as WIZARD_WORKFLOWS,)-# When adding modules to a queue, also add their paths to that fleet's filter in the-# check_temporal_worker_changes step of .github/workflows/container-images-cd.yml+# When adding modules to a queue, also add their paths to that fleet's filter: the+# check_temporal_worker_changes step of .github/workflows/container-images-cd.yml, or+# .github/shared-image-releases.yaml for fleets published through their own channel_task_queue_specs = [(settings.SYNC_BATCH_EXPORTS_TASK_QUEUE,