:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Data warehouse presentation layer
The presentation layer stays source-agnostic
This applies to both data warehouse presentation layers: the source/schema views (now in
products/warehouse_sources/backend/presentation/views/: external_data_source.py,
external_data_schema.py) and the table/query-side views here. Neither may branch on a
concrete source type. No:
if source_type == ExternalDataSourceType.POSTGRES: ...
elif source.is_direct_snowflake: ...
Every such branch splits "how source X behaves" between the source class and the API layer, and grows the views without bound as new sources land. The API asks what a source supports and delegates the how.
Where the behaviour goes
Pick the mechanism by the kind of concern:
Source-domain behaviour (how to talk to the upstream system, whether it supports a feature) → the source class under
products/warehouse_sources/backend/temporal/data_imports/sources:- A boolean/value the API reads → a flag on
_BaseSourcewith a safe default (likesupports_column_selection,connection_host_fields,has_managed_hogql_schema). Branch on the flag. - Methods only some sources have (CDC, xmin, webhooks, custom manifests) → a capability
mixin the source opts into; the API dispatches with
isinstance(source, <Capability>).
- A boolean/value the API reads → a flag on
Direct-query engine behaviour (how a SQL engine resolves a table location, builds and reprojects its
DataWarehouseTable, maps columns) → an engine-keyed adapter, dispatched onsource.direct_engine(viaDIRECT_ENGINE_BY_SOURCE_TYPE/is_direct_capable), never onsource_type. The engine, not the source type, is the axis of variation, and the same engine can back more than one source. Query-side engine behaviour already lives in theposthog/hogql/direct_sql/adapter registry; the warehouse-side materialization (get_*_source_location,upsert_direct_*_table,reproject_direct_*_table,*_columns_to_dwh_columns) belongs behind the matchingdata_warehouseengine registry.
Whatever the mechanism, source-domain semantics live on the source and warehouse-domain work
(DataWarehouseTable rows, managed viewsets, hog functions) stays here, keyed off what the
source / adapter returns. Source capabilities never import data_warehouse types.
Enforcement
.github/scripts/check-dwh-source-agnostic.py is a shrink-only CI guard (run in
ci-backend.yml). It counts every ExternalDataSourceType.<NAME> / SourceType.<NAME> /
is_direct_<engine> reference across both the data_warehouse and warehouse_sources
presentation layers against a baseline (source_agnostic_baseline.txt, kept here). A new
reference fails CI; migrating a family lets you shrink the baseline:
python .github/scripts/check-dwh-source-agnostic.py --regenerate-baseline
The baseline only shrinks. It is now empty: the presentation layer is fully source-agnostic, so
the guard acts as a hard regression check — any new ExternalDataSourceType.<NAME> /
is_direct_<engine> reference here fails CI. Keep it that way.
See implementing-warehouse-sources for how to add behaviour to a source.