:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(web-analytics): read dimensional precompute tables behind a query modifier (#117041)
Lucas Ricoy30419639feb5
diff --git a/docs/internal/web-analytics-query-serving.md b/docs/internal/web-analytics-query-serving.mdindex 0ece3c60c537..38b933ae60fc 100644--- a/docs/internal/web-analytics-query-serving.md+++ b/docs/internal/web-analytics-query-serving.md@@ -124,6 +124,7 @@ The `warehouse source connect completed` event records this label after the crea| # | Strategy | Conditions | Tag || --- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ || 0 | Conversion-goal split | Conversion goal + sessions v2/v3 + events-evaluable filters; visitors from the goal-less dispatch below, conversions from goal events | `web_overview_conversion_goal_query` (+ visitors' tag) |+| 0b | Dimensional | `webQueryModifiers.useDimensionalTables` + tier-2 shape + no property access rules + READY dimensional jobs cover the range | `web_overview_dimensional_query` || 1 | Lazy precompute | Shared gate only — overview has no extra shape restrictions | `web_overview_lazy_query` || 2 | Preaggregated (deprecated) | Modifier on + no conversion goal | `web_overview_preaggregated_query` || 3 | Session-id-set | Filtered + allowlisted + preflight passes (sets `sessionIdPushdown`) | `web_overview_session_id_set_query` |@@ -134,6 +135,7 @@ The `warehouse source connect completed` event records this label after the crea| # | Strategy | Conditions | Tag || --- | -------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------- |+| 0 | Dimensional | `webQueryModifiers.useDimensionalTables` + tier-4 shape + no property access rules + READY dimensional jobs cover the range | `stats_table_dimensional*` || 1 | Paths lazy | `breakdownBy` ∈ {Page, InitialPage} + `includeBounceRate`; rejects avg-time-on-page, scroll depth, unsupported orderBy; top-K 10,000 paths/day cap for high-cardinality teams | `web_stats_paths_lazy_query` || 2 | Frustration lazy | `breakdownBy` = FrustrationMetrics; rejects unsupported orderBy | `web_stats_frustration_lazy_query` || 3 | Simple-breakdown lazy | ~18 supported breakdowns (DeviceType, Browser, OS, Country, Region, City, Viewport, Timezone, Language, ExitPage, InitialChannelType, InitialReferringDomain/URL, InitialUTM\_\*); rejects bounce rate, avg time, scroll depth | `web_stats_lazy_query` |@@ -142,6 +144,19 @@ The `warehouse source connect completed` event records this label after the crea| 6 | No-join | Unfiltered path-bounce and path-bounce+avg-time; any simple or first-pageview breakdown without bounce rate or session fields (filters and conversion goals ride the single events scan) | `stats_table_no_join_*` || 7 | Full join | Fallback per shape | `stats_table_path_bounce`, `stats_table_entry_bounce`, `stats_table_channel_type`, `stats_table_simple_breakdown`, … |+Both runners skip the dimensional tables when the rows cannot match the live result:++- The team has property access rules or the query filters out test accounts.+- The current or compare period does not start on a UTC hour boundary, or ends inside an hour before now.+- Either period starts at or before the events retention floor.+- The team reads sessions v3 and either period starts before the region's v3 backfill start (`SESSIONS_V3_COVERAGE_START_BY_REGION`). The writer does not build those days either, and the coverage check skips them.+- The team has custom channel rules and the query breaks down or filters by channel type.+- The request overrides a bounce or session modifier (`DIMENSIONAL_SEMANTIC_MODIFIER_FIELDS`) with a value the team's builds do not use.++The jobs' identity includes the team's values of those modifiers. A change to the team's bounce or session settings gives new jobs, so reads fall back until the writer rebuilds the window.++The dimensional tables subclass `EventsRetentionTable`, as `events` does. The printer floors every read of them at the events retention floor on `period_bucket`, so a direct HogQL query cannot read rows past retention either. A new table that stores aggregates of events should subclass it too.+### Traffic metrics alongside conversion goals`WebStatsTableQuery.includeTrafficMetrics` adds sessions and retains pageviews alongside conversion columns. Traffic visitors and sessions require a pageview or screenview; goal-only sessions contribute to conversions without increasing the traffic denominator. The option defaults to off, preserving existing callers.@@ -216,18 +231,19 @@ The eager and demand jobs can still overlap if either pass runs long.| `WEB_ANALYTICS_NO_JOIN_TEAM_IDS` + rollout % | Env + percentage | Unfiltered fast path — 100% on Cloud || `useWebAnalyticsPrecompute` | Per-query field | User opt-out switch (WebAnalyticsMenu toggle); explicit `false` always wins || `useWebAnalyticsPreAggregatedTables` | Query modifier | Preaggregated-tables tier (deprecated — largest existing customers only, no new enrollments) |+| `webQueryModifiers.useDimensionalTables` | Per-query field | Tries the dimensional tables first; a gap or a failure falls back to the normal dispatch |## query_type tag referenceSuffix conventions: `*_lazy_query` = bucket read (served from precompute), `*_lazy_insert` = bucket build (background only), `*_preflight` = selectivity probe.-| Family | Precompute | Live tags |-| --------------- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |-| Overview | `web_overview_lazy_query/insert`, `web_overview_preaggregated_query` | `web_overview_conversion_goal_query`, `web_overview_no_join_query`, `web_overview_session_id_set_query` (+`_preflight`), `web_overview_query` |-| Stats table | `web_stats_paths_lazy_*`, `web_stats_frustration_lazy_*`, `web_stats_lazy_*`, `stats_table_preaggregated*` | `stats_table_no_join_*` (incl. `stats_table_no_join_first_pageview_attribution`), `stats_table_session_id_set_*` (+`_preflight`), `stats_table_path_bounce[_and_avg_time]`, `stats_table_entry_bounce`, `stats_table_channel_type`, `stats_table_first_pageview_attribution`, `stats_table_frustration_metrics`, `stats_table_simple_breakdown` |-| Goals | `web_goals_lazy_query/insert` | `web_goals_query` |-| Vitals | `web_vitals_paths_lazy_query/insert` | `web_vitals_path_breakdown_query` |-| External clicks | — | `external_clicks_query` |+| Family | Precompute | Live tags |+| --------------- | -------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| Overview | `web_overview_lazy_query/insert`, `web_overview_preaggregated_query`, `web_overview_dimensional_query` | `web_overview_conversion_goal_query`, `web_overview_no_join_query`, `web_overview_session_id_set_query` (+`_preflight`), `web_overview_query` |+| Stats table | `web_stats_paths_lazy_*`, `web_stats_frustration_lazy_*`, `web_stats_lazy_*`, `stats_table_preaggregated*`, `stats_table_dimensional*` | `stats_table_no_join_*` (incl. `stats_table_no_join_first_pageview_attribution`), `stats_table_session_id_set_*` (+`_preflight`), `stats_table_path_bounce[_and_avg_time]`, `stats_table_entry_bounce`, `stats_table_channel_type`, `stats_table_first_pageview_attribution`, `stats_table_frustration_metrics`, `stats_table_simple_breakdown` |+| Goals | `web_goals_lazy_query/insert` | `web_goals_query` |+| Vitals | `web_vitals_paths_lazy_query/insert` | `web_vitals_path_breakdown_query` |+| External clicks | — | `external_clicks_query` |## Reading a slow tilediff --git a/frontend/src/lib/constants.tsx b/frontend/src/lib/constants.tsxindex a175740eabf8..ddccfc722825 100644--- a/frontend/src/lib/constants.tsx+++ b/frontend/src/lib/constants.tsx@@ -600,6 +600,7 @@ export const FEATURE_FLAGS = {WEB_ANALYTICS_PAGE_REPORTS_DIRECTORY: 'web-analytics-page-reports-directory', // owner: @jordanm-posthog #team-web-analyticsWEB_ANALYTICS_PRECOMPUTE_TOGGLE: 'web-analytics-precompute-toggle', // owner: @lricoy #team-web-analyticsWEB_ANALYTICS_PRECOMPUTED_AT: 'web-analytics-precomputed-at', // owner: @lricoy #team-web-analytics+ WEB_ANALYTICS_QUERY_MODIFIERS: 'web-analytics-query-modifiers', // owner: @lricoy #team-web-analyticsWEB_ANALYTICS_RECAP: 'web-analytics-recap', // owner: @jordanm-posthog #team-web-analyticsWEB_ANALYTICS_REGIONS_MAP: 'web-analytics-regions-map', // owner: @jordanm-posthog #team-web-analyticsWEB_ANALYTICS_REMOVE_REPLAY_TILE: 'web-analytics-remove-replay-tile', // owner: @lricoy #team-web-analytics multivariate=control,testdiff --git a/frontend/src/queries/schema.json b/frontend/src/queries/schema.jsonindex 16abd366df39..de50a6ffdd59 100644--- a/frontend/src/queries/schema.json+++ b/frontend/src/queries/schema.json@@ -36058,6 +36058,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -36315,6 +36319,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["conversionGoalId", "kind", "properties"],@@ -36520,6 +36528,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["conversionGoalId", "kind", "properties"],@@ -37220,6 +37232,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -37902,6 +37918,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -58704,6 +58724,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties", "queryType"],@@ -59020,6 +59044,16 @@},"type": "array"},+ "WebAnalyticsQueryModifiers": {+ "additionalProperties": false,+ "properties": {+ "useDimensionalTables": {+ "description": "Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise.",+ "type": "boolean"+ }+ },+ "type": "object"+ },"WebAnalyticsSampling": {"additionalProperties": false,"properties": {@@ -59126,6 +59160,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["breakdownBy", "kind", "properties"],@@ -59300,6 +59338,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -59478,6 +59520,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -59703,6 +59749,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -59891,6 +59941,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -60164,6 +60218,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -60472,6 +60530,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties"],@@ -60701,6 +60763,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["breakdownBy", "kind", "properties"],@@ -60953,6 +61019,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "metric", "percentile", "properties", "thresholds"],@@ -61168,6 +61238,10 @@"version": {"description": "version of the node, used for schema migrations","type": "number"+ },+ "webQueryModifiers": {+ "$ref": "#/definitions/WebAnalyticsQueryModifiers",+ "description": "Per-request switches for how web analytics serves this query."}},"required": ["kind", "properties", "source"],diff --git a/frontend/src/queries/schema/schema-general.ts b/frontend/src/queries/schema/schema-general.tsindex 14b392ed3245..5f6e3375e5ce 100644--- a/frontend/src/queries/schema/schema-general.ts+++ b/frontend/src/queries/schema/schema-general.ts@@ -3548,7 +3548,14 @@ export type WebAnalyticsSampling = {enabled?: booleanforceSamplingRate?: SamplingRate}+export interface WebAnalyticsQueryModifiers {+ /** Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise. */+ useDimensionalTables?: boolean+}+interface WebAnalyticsQueryBase<R extends Record<string, any>> extends DataNode<R> {+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersdateRange?: DateRangeproperties: WebAnalyticsPropertyFiltersconversionGoal?: WebAnalyticsConversionGoal | nulldiff --git a/frontend/src/scenes/web-analytics/WebAnalyticsMenu.tsx b/frontend/src/scenes/web-analytics/WebAnalyticsMenu.tsxindex ae792381bb51..3a211d1914af 100644--- a/frontend/src/scenes/web-analytics/WebAnalyticsMenu.tsx+++ b/frontend/src/scenes/web-analytics/WebAnalyticsMenu.tsx@@ -45,12 +45,19 @@ export const WebAnalyticsMenu = (): JSX.Element => {showFocusMode,tiles,useWebAnalyticsPrecompute,+ webQueryModifiers,} = useValues(webAnalyticsLogic)const { featureFlags } = useValues(featureFlagLogic)const { achievementsOptOut } = useValues(webAnalyticsAchievementsPreferencesLogic)- const { enterFocusMode, exitFocusMode, openFocusModeModal, setUseWebAnalyticsPrecompute, setTileVisibility } =- useActions(webAnalyticsLogic)+ const {+ enterFocusMode,+ exitFocusMode,+ openFocusModeModal,+ setUseWebAnalyticsPrecompute,+ setTileVisibility,+ setWebQueryModifiers,+ } = useActions(webAnalyticsLogic)const { openModal: openAchievementsModal } = useActions(webAnalyticsAchievementsLogic)const { exportTriggered } = useActions(shareNudgeLogic)@@ -121,6 +128,31 @@ export const WebAnalyticsMenu = (): JSX.Element => {</ScenePanelActionsSection></>)}+ {featureFlags[FEATURE_FLAGS.WEB_ANALYTICS_QUERY_MODIFIERS] && (+ <>+ <ScenePanelDivider />+ <ScenePanelActionsSection>+ <ScenePanelLabel title="Query modifiers" className="px-1.5">+ <Tooltip title="When on, overview and table tiles read from the dimensional pre-aggregated tables if they have data for the whole date range. Otherwise tiles load as usual. Only affects queries from this browser.">+ <ButtonPrimitive+ menuItem+ data-attr="web-analytics-dimensional-tables-toggle"+ onClick={() => {+ setWebQueryModifiers({+ useDimensionalTables: webQueryModifiers.useDimensionalTables+ ? undefined+ : true,+ })+ }}+ >+ <LemonSwitch checked={!!webQueryModifiers.useDimensionalTables} size="xsmall" />+ Use dimensional tables+ </ButtonPrimitive>+ </Tooltip>+ </ScenePanelLabel>+ </ScenePanelActionsSection>+ </>+ )}{showTileToggles && availableTiles.length > 0 && (<><ScenePanelDivider />diff --git a/frontend/src/scenes/web-analytics/WebAnalyticsModal.tsx b/frontend/src/scenes/web-analytics/WebAnalyticsModal.tsxindex c66458286ea4..8609eba9cdb8 100644--- a/frontend/src/scenes/web-analytics/WebAnalyticsModal.tsx+++ b/frontend/src/scenes/web-analytics/WebAnalyticsModal.tsx@@ -5,13 +5,13 @@ import { IconOpenInNew } from 'lib/lemon-ui/icons'import { LemonButton } from 'lib/lemon-ui/LemonButton'import { LemonModal } from 'lib/lemon-ui/LemonModal'import { addProductIntentForCrossSell } from 'lib/utils/product-intents'-import { urls } from 'scenes/urls'import { WebQuery } from 'scenes/web-analytics/tiles/WebAnalyticsTile'import { ProductIntentContext, ProductKey } from '~/queries/schema/schema-general'import { ProductTab } from './common'import { webAnalyticsDateMapping } from './constants'+import { getNewInsightUrlForQuery } from './insightsUtils'import { WebAnalyticsExport } from './WebAnalyticsExport'import { webAnalyticsLogic } from './webAnalyticsLogic'import { webAnalyticsModalLogic } from './webAnalyticsModalLogic'@@ -69,7 +69,7 @@ export const WebAnalyticsModal = (): JSX.Element | null => {<div className="flex flex-row justify-end">{modal.canOpenInsight ? (<LemonButton- to={urls.insightNew({ query: modal.query, sceneSource: 'web-analytics' })}+ to={getNewInsightUrlForQuery(modal.query)}icon={<IconOpenInNew />}size="small"type="secondary"diff --git a/frontend/src/scenes/web-analytics/WebAnalyticsSceneMenuBar.tsx b/frontend/src/scenes/web-analytics/WebAnalyticsSceneMenuBar.tsxindex d3b085cafaea..4a3f81892c15 100644--- a/frontend/src/scenes/web-analytics/WebAnalyticsSceneMenuBar.tsx+++ b/frontend/src/scenes/web-analytics/WebAnalyticsSceneMenuBar.tsx@@ -99,9 +99,16 @@ function WebAnalyticsSceneMenuBarInner(): JSX.Element {shouldFilterTestAccounts,showFocusMode,tiles,+ webQueryModifiers,} = useValues(webAnalyticsLogic)- const { enterFocusMode, exitFocusMode, openFocusModeModal, setShouldFilterTestAccounts, setTileVisibility } =- useActions(webAnalyticsLogic)+ const {+ enterFocusMode,+ exitFocusMode,+ openFocusModeModal,+ setShouldFilterTestAccounts,+ setTileVisibility,+ setWebQueryModifiers,+ } = useActions(webAnalyticsLogic)const { exportTriggered } = useActions(shareNudgeLogic)const { featureFlags } = useValues(featureFlagLogic)const { hasFileItems } = useValues(sceneFileLogic)@@ -225,6 +232,17 @@ function WebAnalyticsSceneMenuBarInner(): JSX.Element {>Filter out internal and test users</SceneMenuBarCheckboxItem>+ {featureFlags[FEATURE_FLAGS.WEB_ANALYTICS_QUERY_MODIFIERS] && (+ <SceneMenuBarCheckboxItem+ checked={!!webQueryModifiers.useDimensionalTables}+ onCheckedChange={(checked) =>+ setWebQueryModifiers({ useDimensionalTables: checked || undefined })+ }+ data-attr="web-analytics-menubar-dimensional-tables"+ >+ Use dimensional tables+ </SceneMenuBarCheckboxItem>+ )}{showTileToggles && availableTiles.length > 0 && (<SceneMenuBarSubMenu label="Visible tiles">{availableTiles.map((tileId) => (diff --git a/frontend/src/scenes/web-analytics/insightsUtils.ts b/frontend/src/scenes/web-analytics/insightsUtils.tsindex bc0caba3151a..61ced2bbe87e 100644--- a/frontend/src/scenes/web-analytics/insightsUtils.ts+++ b/frontend/src/scenes/web-analytics/insightsUtils.ts@@ -4,14 +4,20 @@ import { InsightVizNode, NodeKind, QuerySchema } from '~/queries/schema/schema-gimport { isWebAnalyticsInsightQuery } from '~/queries/utils'import { TileId, WebAnalyticsTile } from './common'+import { withoutWebQueryModifiers } from './webQueryModifiers'export const getDashboardItemId = (section: TileId, tab: string | undefined, isModal?: boolean): `new-${string}` => {return `new-AdHoc.web-analytics.${section}.${tab || 'default'}.${isModal ? 'modal' : 'default'}`}+// A saved insight runs for every viewer, so it must not keep this browser's web query modifiers.+export const getNewInsightUrlForQuery = (query: QuerySchema): string =>+ urls.insightNew({ query: withoutWebQueryModifiers(query), sceneSource: 'web-analytics' })+export const getNewInsightUrlFactory = (tiles: WebAnalyticsTile[]) => {return function getNewInsightUrl(tileId: TileId, tabId?: string): string | undefined {- const formatQueryForNewInsight = (query: QuerySchema): QuerySchema => {+ const formatQueryForNewInsight = (tileQuery: QuerySchema): QuerySchema => {+ const query = withoutWebQueryModifiers(tileQuery)if (query.kind === NodeKind.InsightVizNode) {const source = query.sourceconst editableSource =diff --git a/frontend/src/scenes/web-analytics/webAnalyticsLogic.tsx b/frontend/src/scenes/web-analytics/webAnalyticsLogic.tsxindex c89de81f1a1c..f4ef7e63c08f 100644--- a/frontend/src/scenes/web-analytics/webAnalyticsLogic.tsx+++ b/frontend/src/scenes/web-analytics/webAnalyticsLogic.tsx@@ -96,7 +96,7 @@ import {} from '~/types'import type { FeatureFlagsSet } from '../../lib/logic/featureFlagLogic'-import type { CompareFilter, CurrencyCode } from '../../queries/schema/schema-general'+import type { CompareFilter, CurrencyCode, WebAnalyticsQueryModifiers } from '../../queries/schema/schema-general'import { botAnalyticsLogic } from './botAnalyticsLogic'import {ActiveHoursTab,@@ -143,6 +143,7 @@ import { IncludeHostToggle } from './IncludeHostToggle'import { getDashboardItemId, getNewInsightUrlFactory } from './insightsUtils'import { webAnalyticsFilterLogic } from './webAnalyticsFilterLogic'import { WebAnalyticsLogicProps } from './webAnalyticsLogicProps'+import { applyWebQueryModifiers } from './webQueryModifiers'export interface DateFilterState {dateFrom: string | null@@ -197,6 +198,7 @@ export interface webAnalyticsLogicValues {currentFiltersConfig: WebAnalyticsFiltersConfigdateFilter: DateFilterStatedeviceTab: string+ effectiveWebQueryModifiers: WebAnalyticsQueryModifiersexportAllDisabledReason: string | nullfilters: {compareFilter: CompareFilter@@ -258,9 +260,11 @@ export interface webAnalyticsLogicValues {}tileVisualizations: Record<TileId, TileVisualizationOption>tiles: WebAnalyticsTile[]+ tilesWithoutQueryModifiers: WebAnalyticsTile[]useWebAnalyticsPrecompute: boolean | nullwarmablePresetShortId: string | nullwebAnalyticsFilters: WebAnalyticsPropertyFilters+ webQueryModifiers: WebAnalyticsQueryModifierswebVitalsMetricQuery: InsightVizNode<TrendsQuery>webVitalsPercentile: WebVitalsPercentilewebVitalsTab: WebVitalsMetric@@ -528,6 +532,9 @@ export interface webAnalyticsLogicActions {setUseWebAnalyticsPrecompute: (useWebAnalyticsPrecompute: boolean | null) => {useWebAnalyticsPrecompute: boolean | null}+ setWebQueryModifiers: (patch: WebAnalyticsQueryModifiers) => {+ patch: WebAnalyticsQueryModifiers+ }setWebVitalsPercentile: (percentile: WebVitalsPercentile) => {percentile: WebVitalsPercentile}@@ -683,7 +690,7 @@ export interface webAnalyticsLogicMeta {focusModeEnabled: boolean,focusModeConcerns: WebAnalyticsConcern[]) => boolean- tiles: (+ tilesWithoutQueryModifiers: (productTab: ProductTab,tabs: {activeHoursTab: string@@ -718,6 +725,14 @@ export interface webAnalyticsLogicMeta {hiddenTiles: TileId[],warmablePresetShortId: string | null) => WebAnalyticsTile[]+ effectiveWebQueryModifiers: (+ webQueryModifiers: WebAnalyticsQueryModifiers,+ featureFlags: FeatureFlagsSet+ ) => WebAnalyticsQueryModifiers+ tiles: (+ tilesWithoutQueryModifiers: WebAnalyticsTile[],+ effectiveWebQueryModifiers: WebAnalyticsQueryModifiers+ ) => WebAnalyticsTile[]getNewInsightUrl: (tiles: WebAnalyticsTile[]) => (tileId: TileId, tabId?: string | undefined) => string | undefined@@ -829,6 +844,7 @@ export const webAnalyticsLogic: LogicWrapper<webAnalyticsLogicType> = kea<webAnasetIsPathCleaningEnabled: (isPathCleaningEnabled: boolean) => ({ isPathCleaningEnabled }),setShouldFilterTestAccounts: (shouldFilterTestAccounts: boolean) => ({ shouldFilterTestAccounts }),setUseWebAnalyticsPrecompute: (useWebAnalyticsPrecompute: boolean | null) => ({ useWebAnalyticsPrecompute }),+ setWebQueryModifiers: (patch: WebAnalyticsQueryModifiers) => ({ patch }),setShouldStripQueryParams: (shouldStripQueryParams: boolean) => ({ shouldStripQueryParams }),setIncludeHostPath: (includeHostPath: boolean) => ({ includeHostPath }),setConversionGoal: (conversionGoal: WebAnalyticsConversionGoal | null) => ({ conversionGoal }),@@ -1064,6 +1080,17 @@ export const webAnalyticsLogic: LogicWrapper<webAnalyticsLogicType> = kea<webAnaclearFilters: () => false,},],+ webQueryModifiers: [+ {} as WebAnalyticsQueryModifiers,+ persistConfig,+ {+ // A key set to `undefined` in the patch is removed, so the query falls back to the team default.+ setWebQueryModifiers: (state, { patch }) =>+ Object.fromEntries(+ Object.entries({ ...state, ...patch }).filter(([, value]) => value !== undefined)+ ) as WebAnalyticsQueryModifiers,+ },+ ],useWebAnalyticsPrecompute: [// Tri-state: `null` means the user never touched the toggle, so the// backend's per-team default decides (opt-out for unrestricted teams,@@ -1727,7 +1754,7 @@ export const webAnalyticsLogic: LogicWrapper<webAnalyticsLogicType> = kea<webAna(showFocusMode: boolean, focusModeEnabled: boolean, focusModeConcerns: WebAnalyticsConcern[]): boolean =>showFocusMode && focusModeEnabled && focusModeConcerns.length > 0,],- tiles: [+ tilesWithoutQueryModifiers: [(s) => [s.productTab,s.tabs,@@ -3202,6 +3229,20 @@ export const webAnalyticsLogic: LogicWrapper<webAnalyticsLogicType> = kea<webAna)},],+ effectiveWebQueryModifiers: [+ (s) => [s.webQueryModifiers, s.featureFlags],+ (+ webQueryModifiers: WebAnalyticsQueryModifiers,+ featureFlags: FeatureFlagsSet+ ): WebAnalyticsQueryModifiers =>+ // Stored modifiers apply only while the flag is on, so turning the flag off stops sending them.+ featureFlags[FEATURE_FLAGS.WEB_ANALYTICS_QUERY_MODIFIERS] ? webQueryModifiers : {},+ ],+ tiles: [+ (s) => [s.tilesWithoutQueryModifiers, s.effectiveWebQueryModifiers],+ (tiles: WebAnalyticsTile[], effectiveWebQueryModifiers: WebAnalyticsQueryModifiers): WebAnalyticsTile[] =>+ applyWebQueryModifiers(tiles, effectiveWebQueryModifiers),+ ],getNewInsightUrl: [(s) => [s.tiles], (tiles: WebAnalyticsTile[]) => getNewInsightUrlFactory(tiles)],})),diff --git a/frontend/src/scenes/web-analytics/webQueryModifiers.test.ts b/frontend/src/scenes/web-analytics/webQueryModifiers.test.tsnew file mode 100644index 000000000000..30d31cfeffa0--- /dev/null+++ b/frontend/src/scenes/web-analytics/webQueryModifiers.test.ts@@ -0,0 +1,100 @@+import { NodeKind, QuerySchema, WebStatsBreakdown } from '~/queries/schema/schema-general'++import { TileId, WebAnalyticsTile } from './common'+import { getNewInsightUrlFactory, getNewInsightUrlForQuery } from './insightsUtils'+import { applyWebQueryModifiers, withWebQueryModifiers } from './webQueryModifiers'++const overview: QuerySchema = {+ kind: NodeKind.WebOverviewQuery,+ properties: [],+ useWebAnalyticsPrecompute: true,+}+const statsTable: QuerySchema = {+ kind: NodeKind.DataTableNode,+ source: { kind: NodeKind.WebStatsTableQuery, properties: [], breakdownBy: WebStatsBreakdown.Page },+}+const trends: QuerySchema = {+ kind: NodeKind.InsightVizNode,+ source: { kind: NodeKind.TrendsQuery, series: [] },+}+const webVitals: QuerySchema = {+ kind: NodeKind.WebVitalsQuery,+ properties: [],+ source: { kind: NodeKind.TrendsQuery, series: [] },+}++const layout = { colSpanClassName: 'md:col-span-full' as const }+const insightProps = { dashboardItemId: 'new-test' as const }++const tiles: WebAnalyticsTile[] = [+ { kind: 'query', tileId: TileId.OVERVIEW, layout, query: overview, insightProps },+ {+ kind: 'section',+ tileId: TileId.PATHS,+ layout,+ tiles: [+ {+ kind: 'tabs',+ tileId: TileId.PATHS,+ layout,+ activeTabId: 'paths',+ setTabId: () => {},+ tabs: [{ id: 'paths', title: 'Paths', linkText: 'Paths', query: statsTable, insightProps }],+ },+ ],+ },+ { kind: 'query', tileId: TileId.GRAPHS, layout, query: trends, insightProps },+ { kind: 'replay', tileId: TileId.REPLAY, layout },+ { kind: 'query', tileId: TileId.WEB_VITALS, layout, query: webVitals, insightProps },+]++describe('webQueryModifiers', () => {+ it('keeps the same tiles when there are no modifiers', () => {+ expect(applyWebQueryModifiers(tiles, {})).toBe(tiles)+ })++ it('sets the modifiers on web analytics queries only, including wrapped sources inside sections and tabs', () => {+ const [overviewTile, sectionTile, trendsTile, replayTile, webVitalsTile] = applyWebQueryModifiers(tiles, {+ useDimensionalTables: true,+ })++ expect(overviewTile).toMatchObject({+ query: { useWebAnalyticsPrecompute: true, webQueryModifiers: { useDimensionalTables: true } },+ })+ expect(sectionTile).toMatchObject({+ tiles: [{ tabs: [{ query: { source: { webQueryModifiers: { useDimensionalTables: true } } } }] }],+ })+ // TrendsQuery does not accept the field, so the backend would reject the whole tile.+ expect(trendsTile).toMatchObject({ query: { source: trends.source } })+ expect((trendsTile as { query: { source: object } }).query.source).not.toHaveProperty('webQueryModifiers')+ expect(replayTile).toBe(tiles[3])+ expect(webVitalsTile).toMatchObject({ query: { webQueryModifiers: { useDimensionalTables: true } } })+ expect((webVitalsTile as { query: { source: object } }).query.source).not.toHaveProperty('webQueryModifiers')+ })++ it.each([[TileId.OVERVIEW], [TileId.WEB_VITALS]])(+ 'leaves the modifiers out of the new insight url for %s',+ (tileId) => {+ const getNewInsightUrl = getNewInsightUrlFactory(+ applyWebQueryModifiers(tiles, { useDimensionalTables: true })+ )++ const url = decodeURIComponent(getNewInsightUrl(tileId) ?? '')++ expect(url).toContain('/insights/new')+ expect(url).not.toContain('webQueryModifiers')+ }+ )++ it.each([+ ['overview', overview],+ ['stats table', statsTable],+ ])('leaves the modifiers out of the modal new insight url for the %s', (_, query) => {+ const url = decodeURIComponent(+ getNewInsightUrlForQuery(withWebQueryModifiers(query, { useDimensionalTables: true }))+ )++ expect(url).toContain('/insights/new')+ expect(url).not.toContain('webQueryModifiers')+ })+})diff --git a/frontend/src/scenes/web-analytics/webQueryModifiers.ts b/frontend/src/scenes/web-analytics/webQueryModifiers.tsnew file mode 100644index 000000000000..f5b4846684aa--- /dev/null+++ b/frontend/src/scenes/web-analytics/webQueryModifiers.ts@@ -0,0 +1,72 @@+import { NodeKind, QuerySchema, WebAnalyticsQueryModifiers } from '~/queries/schema/schema-general'++import { WebAnalyticsTile } from './common'++// The query schemas reject unknown fields, so only kinds built on WebAnalyticsQueryBase may carry webQueryModifiers.+const WEB_ANALYTICS_QUERY_KINDS = new Set<NodeKind>([+ NodeKind.WebOverviewQuery,+ NodeKind.WebStatsTableQuery,+ NodeKind.WebExternalClicksTableQuery,+ NodeKind.WebBotsTableQuery,+ NodeKind.WebAgentAnalyticsQuery,+ NodeKind.WebGoalsQuery,+ NodeKind.WebVitalsQuery,+ NodeKind.WebVitalsPathBreakdownQuery,+ NodeKind.WebPageURLSearchQuery,+ NodeKind.WebNotableChangesQuery,+])++// These nodes run their source, so the modifiers go there. WebVitalsQuery also has a source, but it takes the+// modifiers itself and its source is a TrendsQuery, which rejects them.+const WRAPPER_QUERY_KINDS = new Set<NodeKind>([+ NodeKind.DataTableNode,+ NodeKind.InsightVizNode,+ NodeKind.DataVisualizationNode,+])++export function withWebQueryModifiers<T extends QuerySchema>(query: T, modifiers: WebAnalyticsQueryModifiers): T {+ if (WEB_ANALYTICS_QUERY_KINDS.has(query.kind)) {+ const existing = 'webQueryModifiers' in query ? query.webQueryModifiers : undefined+ return { ...query, webQueryModifiers: { ...existing, ...modifiers } }+ }+ if (WRAPPER_QUERY_KINDS.has(query.kind) && 'source' in query && query.source) {+ return { ...query, source: withWebQueryModifiers(query.source as QuerySchema, modifiers) }+ }+ return query+}++export function withoutWebQueryModifiers<T extends QuerySchema>(query: T): T {+ if ('webQueryModifiers' in query && query.webQueryModifiers) {+ const { webQueryModifiers: _, ...rest } = query+ return rest as T+ }+ if (WRAPPER_QUERY_KINDS.has(query.kind) && 'source' in query && query.source) {+ return { ...query, source: withoutWebQueryModifiers(query.source as QuerySchema) }+ }+ return query+}++export function applyWebQueryModifiers(+ tiles: WebAnalyticsTile[],+ modifiers: WebAnalyticsQueryModifiers+): WebAnalyticsTile[] {+ // Keep the same tile objects when there is nothing to apply, so no tile refetches.+ if (Object.keys(modifiers).length === 0) {+ return tiles+ }+ return tiles.map((tile): WebAnalyticsTile => {+ switch (tile.kind) {+ case 'query':+ return { ...tile, query: withWebQueryModifiers(tile.query, modifiers) }+ case 'tabs':+ return {+ ...tile,+ tabs: tile.tabs.map((tab) => ({ ...tab, query: withWebQueryModifiers(tab.query, modifiers) })),+ }+ case 'section':+ return { ...tile, tiles: applyWebQueryModifiers(tile.tiles, modifiers) }+ default:+ return tile+ }+ })+}diff --git a/frontend/src/scenes/web-analytics/webTileHeaderHooks.tsx b/frontend/src/scenes/web-analytics/webTileHeaderHooks.tsxindex db0d0421b3fb..aff52a6ac356 100644--- a/frontend/src/scenes/web-analytics/webTileHeaderHooks.tsx+++ b/frontend/src/scenes/web-analytics/webTileHeaderHooks.tsx@@ -18,6 +18,7 @@ import { TileId, WEB_ANALYTICS_DATA_COLLECTION_NODE_ID } from './common'import { shareNudgeLogic } from './shareNudgeLogic'import { downloadTableDataAsCsv, getExportAdapter } from './webAnalyticsExportUtils'import { webAnalyticsModalLogic } from './webAnalyticsModalLogic'+import { withoutWebQueryModifiers } from './webQueryModifiers'const NO_ACTIVE_TAB_INSIGHT_PROPS: InsightLogicProps = {dashboardItemId: 'new-AdHoc.web-analytics-no-active-tab-fallback',@@ -103,7 +104,7 @@ export function useWebTileOverflowMenuItems({if (!query) {return}- void copyToClipboard(JSON.stringify(query, null, 2), 'query JSON')+ void copyToClipboard(JSON.stringify(withoutWebQueryModifiers(query), null, 2), 'query JSON')},},]diff --git a/posthog/api/debug_ch_queries.py b/posthog/api/debug_ch_queries.pyindex 36e41d10837e..ad00b86e40bf 100644--- a/posthog/api/debug_ch_queries.py+++ b/posthog/api/debug_ch_queries.py@@ -187,6 +187,7 @@ _PRECOMPUTE_PRODUCTS: dict[str, dict[str, Any]] = {"eligible_live_query_types": ("web_overview_query","web_overview_preaggregated_query",+ "web_overview_dimensional_query","web_overview_no_join_query","web_overview_session_id_set_query","stats_table_main_query",@@ -201,6 +202,9 @@ _PRECOMPUTE_PRODUCTS: dict[str, dict[str, Any]] = {"stats_table_preaggregated_query","stats_table_preaggregated_path_breakdown_query","stats_table_preaggregated_entry_bounce_query",+ "stats_table_dimensional_query",+ "stats_table_dimensional_path_breakdown_query",+ "stats_table_dimensional_entry_bounce_query","web_goals_query","web_vitals_path_breakdown_query",),diff --git a/posthog/hogql/ACCESS_CONTROL.md b/posthog/hogql/ACCESS_CONTROL.mdindex 15e0a0a33ec5..7e8891113ecb 100644--- a/posthog/hogql/ACCESS_CONTROL.md+++ b/posthog/hogql/ACCESS_CONTROL.md@@ -229,17 +229,20 @@ Covering a table means moving it out of the exemptions and into the expected mapMasking rewrites `properties.<key>` reads and strips keys from a JSON blob. It does not reach a physical column that holds a copy of the same value.Several catalog tables expose such columns because reading them scans far less data than digging the value out of the blob: `events` exposes `$session_id`, `$window_id`, and `$group_0`..`$group_4`; `flag_evaluations` exposes `flag_key`, `response`, `session_id`, `request_id`, and `$group_0`..`$group_4`.+The web analytics dimensional tables `web_stats_dimensional_preaggregated` and `web_bounces_dimensional_preaggregated` copy event properties into dimension columns, such as `pathname` from `$pathname` and `country_code` from `$geoip_country_code`, and copy session properties that the sessions table builds from event properties, such as `utm_source` from `utm_source` and `entry_pathname` from `$current_url`.`events`' mirror columns are not masked: restricting the property they mirror masks the blob read and leaves the column readable.-`flag_evaluations` and `ai_events` mirror columns are masked through `mirrored_property_for_column()` in `posthog/hogql/restricted_properties.py`, consulted by `ClickHousePropertyResolver` in `posthog/hogql/transforms/clickhouse_property_resolution.py`.+`flag_evaluations`, `ai_events`, and web analytics dimensional mirror columns are masked through `mirrored_properties_for_column()` in `posthog/hogql/restricted_properties.py`, consulted by `ClickHousePropertyResolver` in `posthog/hogql/transforms/clickhouse_property_resolution.py`.The mapping is keyed by the stored column name, not the HogQL field name, so an expression field that reads a mirror column masks too: `flag_evaluations.response` reads the stored `response` column through `nullIf`, and the hidden `flag_evaluation_response` reads it bare, and both mask. The hidden `goe_N.key` on `flag_evaluations` reads the stored `$group_N` column through an expression field, and it masks too.For AI columns, `AI_PROPERTY_TO_COLUMN` in `posthog/hogql/database/schema/ai_events.py` is shared with the query rewriter.One rule for `$ai_input` masks both `events.properties.$ai_input` and `ai_events.input`, including nested reads such as `input[1].content`.The hidden length fields `input_state_length` and `output_state_length` follow the rule of their state property, so a masked state also hides whether it exists.The same rule applies when a filter or an aggregate reads the column.+A mirror of a dotted property masks when the property or any parent segment is restricted, so restricting `metadata` masks `mat_metadata_backend`, the mirror of `metadata.backend`.Nested keys in an allowed AI payload are not separate event properties: a restriction on the event property `content` does not restrict `input.content`.+A dimensional session-level column follows the rule of the event property the sessions table reads, so restricting `utm_source` masks `utm_source` there. `entry_pathname` and `end_pathname` also follow `$pathname`, which the SDK derives from the same URL. The `sessions` table itself does not mask these columns, because HogQL does not enforce session property restrictions on any table.That resolver rewrites a restricted mirror column to the same `Constant(value=None, type=StringType(nullable=True))` the source property lowers to, before the AST reaches the printer — so the mirror column and its source property are one AST node by the time comparisons and nullability are decided, not two independently masked spellings that could drift apart.-Covering `events`' remaining mirror columns, or a future catalog table's, means adding it to `_FLAG_EVALUATIONS_MIRRORED_COLUMNS`'s sibling mapping (or a new one `mirrored_property_for_column` dispatches to) rather than a print-time patch.+Covering `events`' remaining mirror columns, or a future catalog table's, means adding it to `_FLAG_EVALUATIONS_MIRRORED_COLUMNS`'s sibling mapping (or a new one `mirrored_properties_for_column` dispatches to) rather than a print-time patch.### No user: default rules applydiff --git a/posthog/hogql/database/database.py b/posthog/hogql/database/database.pyindex ffbee42bd855..f439a93d3081 100644--- a/posthog/hogql/database/database.py+++ b/posthog/hogql/database/database.py@@ -146,8 +146,10 @@ from posthog.hogql.database.schema.spans import TraceAttributesTable, TraceSpansfrom posthog.hogql.database.schema.static_cohort_people import StaticCohortPeoplefrom posthog.hogql.database.schema.system import SystemTablesfrom posthog.hogql.database.schema.web_analytics_preaggregated import (+ WebBouncesDimensionalPreAggregatedTable,WebPreAggregatedBouncesTable,WebPreAggregatedStatsTable,+ WebStatsDimensionalPreAggregatedTable,)from posthog.hogql.database.schema.web_goals_preaggregated import WebGoalsPreaggregatedTablefrom posthog.hogql.database.schema.web_overview_preaggregated import WebOverviewPreaggregatedTable@@ -409,6 +411,17 @@ ROOT_TABLES__DO_NOT_ADD_ANY_MORE: dict[str, TableNode] = {# Web analytics pre-aggregated tables (internal use only)"web_pre_aggregated_stats": TableNode(name="web_pre_aggregated_stats", table=WebPreAggregatedStatsTable()),"web_pre_aggregated_bounces": TableNode(name="web_pre_aggregated_bounces", table=WebPreAggregatedBouncesTable()),+ # Justified exceptions to the "do not add" rule: the dimensional precompute+ # tables are the v2 tables' read successors and must live in the root namespace+ # for the same reason v2 does — the web-analytics query builder qualifies columns+ # as `table.col` (single-identifier), which only resolves at the root. Namespacing+ # them under `posthog.` would require reworking that shared builder.+ "web_stats_dimensional_preaggregated": TableNode(+ name="web_stats_dimensional_preaggregated", table=WebStatsDimensionalPreAggregatedTable()+ ),+ "web_bounces_dimensional_preaggregated": TableNode(+ name="web_bounces_dimensional_preaggregated", table=WebBouncesDimensionalPreAggregatedTable()+ ),"preaggregation_results": TableNode(name="preaggregation_results", table=PreaggregationResultsTable()),"experiment_exposures_preaggregated": TableNode(name="experiment_exposures_preaggregated", table=ExperimentExposuresPreaggregatedTable()diff --git a/posthog/hogql/database/models.py b/posthog/hogql/database/models.pyindex 35ac972150eb..b72186a38079 100644--- a/posthog/hogql/database/models.py+++ b/posthog/hogql/database/models.py@@ -296,6 +296,16 @@ class Table(FieldOrTable):return asterisk+class EventsRetentionTable(Table):+ """A table of events or of rows derived from events, which the team's events retention applies to.++ The ClickHouse printer floors every scan of a subclass at `events_retention_field > now() - retention`, so+ derived rows cannot outlive the events they came from.+ """++ events_retention_field: str++_CI_INDEX_CACHE_KEY = "_case_insensitive_index_cache"diff --git a/posthog/hogql/database/schema/events.py b/posthog/hogql/database/schema/events.pyindex a44ad1ccdd38..ae5da811f846 100644--- a/posthog/hogql/database/schema/events.py+++ b/posthog/hogql/database/schema/events.py@@ -7,6 +7,7 @@ from posthog.hogql.database.lazy_join_tags import (from posthog.hogql.database.models import (DatabaseField,DateTimeDatabaseField,+ EventsRetentionTable,FieldOrTable,FieldTraverser,IntegerDatabaseField,@@ -14,7 +15,6 @@ from posthog.hogql.database.models import (StringArrayDatabaseField,StringDatabaseField,StringJSONDatabaseField,- Table,UUIDDatabaseField,VirtualTable,)@@ -99,8 +99,9 @@ class EventsGroupSubTable(VirtualTable):return "events"-class EventsTable(Table):+class EventsTable(EventsRetentionTable):description: str = "Every analytics event captured for the project. The central fact table for product analytics."+ events_retention_field: str = "timestamp"fields: dict[str, FieldOrTable] = {"uuid": UUIDDatabaseField(name="uuid", nullable=False, description="Unique identifier of this event row."),"event": StringDatabaseField(diff --git a/posthog/hogql/database/schema/flag_evaluations.py b/posthog/hogql/database/schema/flag_evaluations.pyindex 766e28f22700..2362ac7cf496 100644--- a/posthog/hogql/database/schema/flag_evaluations.py+++ b/posthog/hogql/database/schema/flag_evaluations.py@@ -5,6 +5,7 @@ from posthog.hogql import astfrom posthog.hogql.database.lazy_join_tags import GROUP_N, PERSON_DISTINCT_ID_OVERRIDES, PERSON_DISTINCT_IDS, PERSONSfrom posthog.hogql.database.models import (DateTimeDatabaseField,+ EventsRetentionTable,ExpressionField,FieldOrTable,FieldTraverser,@@ -12,7 +13,6 @@ from posthog.hogql.database.models import (LazyJoin,StringDatabaseField,StringJSONDatabaseField,- Table,UUIDDatabaseField,VirtualTable,)@@ -121,11 +121,13 @@ _ELEMENT_ARRAY_STAND_INS = ("elements_chain_texts", "elements_chain_ids", "elemeELEMENT_STAND_INS = frozenset(_ELEMENT_STRING_STAND_INS + _ELEMENT_ARRAY_STAND_INS)-class FlagEvaluationsTable(Table):+# Flag calls are events too, and a query on events can read them from here, so events retention applies.+class FlagEvaluationsTable(EventsRetentionTable):description: str = ("One row per feature flag evaluation, from the `$feature_flag_called` event. Rows are kept for 90 days. ""A project that is not sending flag-evaluation telemetry yet sees an empty table.")+ events_retention_field: str = "timestamp"fields: dict[str, FieldOrTable] = {"uuid": UUIDDatabaseField(name="uuid", nullable=False, description="Unique identifier of this row."),"event": StringDatabaseField(diff --git a/posthog/hogql/database/schema/web_analytics_preaggregated.py b/posthog/hogql/database/schema/web_analytics_preaggregated.pyindex 6a8635e42dba..5b13ef874b01 100644--- a/posthog/hogql/database/schema/web_analytics_preaggregated.py+++ b/posthog/hogql/database/schema/web_analytics_preaggregated.py@@ -1,11 +1,16 @@+from pydantic import Field++from posthog.hogql.constants import HogQLQuerySettingsfrom posthog.hogql.database.models import (BooleanDatabaseField,DatabaseField,DateTimeDatabaseField,+ EventsRetentionTable,FieldOrTable,IntegerDatabaseField,StringDatabaseField,Table,+ UUIDDatabaseField,)DEVICE_BROWSER_FIELDS = {@@ -182,3 +187,69 @@ class WebPreAggregatedBouncesTable(Table):def to_printed_hogql(self):return "web_pre_aggregated_bounces"+++# The dimensional precompute tables mirror the v2 schema but carry an extra `job_id`+# (the precomputation framework writes one generation per job). Reads filter+# `job_id IN (ready ids)` to dedup the ReplacingMergeTree generations.+WEB_DIMENSIONAL_SPECIFIC_FIELDS = {+ "job_id": UUIDDatabaseField(+ name="job_id", description="Precomputation job that wrote this row; reads filter to READY job ids."+ ),+}+++class WebStatsDimensionalPreAggregatedTable(EventsRetentionTable):+ description: str = (+ "Precomputation-framework successor to web_pre_aggregated_stats: same dimensions plus a job_id "+ "generation column, used internally by the web analytics product. Metric columns are AggregateFunction "+ "states that must be merged, and reads must filter job_id to READY jobs to avoid double-counting."+ )+ # Holds the start of each hourly bucket, so the floor drops the whole bucket that contains the boundary.+ events_retention_field: str = "period_bucket"+ # Sharded by `sipHash64(job_id)` and every read filters `job_id IN (...)`, so ClickHouse can skip+ # shards that hold none of the requested jobs.+ top_level_settings: HogQLQuerySettings | None = Field(+ default_factory=lambda: HogQLQuerySettings(optimize_skip_unused_shards=True)+ )+ fields: dict[str, FieldOrTable] = {+ **web_preaggregated_base_fields,+ **web_preaggregated_base_aggregation_fields,+ **SHARED_SCHEMA_FIELDS,+ **WEB_STATS_SPECIFIC_FIELDS,+ **WEB_DIMENSIONAL_SPECIFIC_FIELDS,+ }++ def to_printed_clickhouse(self, context):+ return "web_stats_dimensional_preaggregated"++ def to_printed_hogql(self):+ return "web_stats_dimensional_preaggregated"+++class WebBouncesDimensionalPreAggregatedTable(EventsRetentionTable):+ description: str = (+ "Precomputation-framework successor to web_pre_aggregated_bounces: same dimensions plus a job_id "+ "generation column, used internally by the web analytics product. Metric columns are AggregateFunction "+ "states that must be merged, and reads must filter job_id to READY jobs to avoid double-counting."+ )+ # Holds the start of each hourly bucket, so the floor drops the whole bucket that contains the boundary.+ events_retention_field: str = "period_bucket"+ # Sharded by `sipHash64(job_id)` and every read filters `job_id IN (...)`, so ClickHouse can skip+ # shards that hold none of the requested jobs.+ top_level_settings: HogQLQuerySettings | None = Field(+ default_factory=lambda: HogQLQuerySettings(optimize_skip_unused_shards=True)+ )+ fields: dict[str, FieldOrTable] = {+ **web_preaggregated_base_fields,+ **web_preaggregated_base_aggregation_fields,+ **SHARED_SCHEMA_FIELDS,+ **WEB_BOUNCES_SPECIFIC_FIELDS,+ **WEB_DIMENSIONAL_SPECIFIC_FIELDS,+ }++ def to_printed_clickhouse(self, context):+ return "web_bounces_dimensional_preaggregated"++ def to_printed_hogql(self):+ return "web_bounces_dimensional_preaggregated"diff --git a/posthog/hogql/database/test/test_database.py b/posthog/hogql/database/test/test_database.pyindex 62c42169247d..c94019180380 100644--- a/posthog/hogql/database/test/test_database.py+++ b/posthog/hogql/database/test/test_database.py@@ -4456,6 +4456,8 @@ class TestDatabase(BaseTest, QueryMatchingTest):"logs_kafka_metrics","web_pre_aggregated_stats","web_pre_aggregated_bounces",+ "web_stats_dimensional_preaggregated",+ "web_bounces_dimensional_preaggregated","preaggregation_results","experiment_exposures_preaggregated","experiment_metric_events_preaggregated",diff --git a/posthog/hogql/printer/base.py b/posthog/hogql/printer/base.pyindex a7500df893a7..cce08a59e372 100644--- a/posthog/hogql/printer/base.py+++ b/posthog/hogql/printer/base.py@@ -543,7 +543,7 @@ class BasePrinter(Visitor[str]):table_type: ast.TableType | ast.LazyTableType,node_type: ast.TableOrSelectType | None,) -> ast.Expr | None:- """Floor events-table scans to the team's retention window. Overridden by the ClickHouse dialect;+ """Floor scans of events and events-derived tables to the team's retention window. Overridden by the ClickHouse dialect;default no-op — the HogQL and warehouse dialects don't enforce events retention."""return Nonediff --git a/posthog/hogql/printer/clickhouse.py b/posthog/hogql/printer/clickhouse.pyindex e7546263760f..aae9ef7e6099 100644--- a/posthog/hogql/printer/clickhouse.py+++ b/posthog/hogql/printer/clickhouse.py@@ -15,6 +15,7 @@ from posthog.hogql.database.direct_sql_table import DirectSQLTablefrom posthog.hogql.database.models import (DANGEROUS_NoTeamIdCheckTable,DatabaseField,+ EventsRetentionTable,SavedQuery,StringJSONDatabaseField,StructDatabaseField,@@ -81,8 +82,10 @@ def team_id_guard_for_table(table_type: ast.TableOrSelectType, context: HogQLCon)-def retention_floor_for_table(table_type: ast.TableOrSelectType, retention_months: int) -> ast.Expr:- """Floor an events-table scan to ``timestamp > now() - toIntervalMonth(retention_months)``.+def retention_floor_for_table(+ table_type: ast.TableOrSelectType, retention_months: int, field_name: str = "timestamp"+) -> ast.Expr:+ """Floor an events-derived scan to ``<field_name> > now() - toIntervalMonth(retention_months)``.Sibling to ``team_id_guard_for_table``: a mandatory, context-derived guard added at the lowest level on theevents table, so the events-data-retention cap can't be bypassed by query-supplied date filters or modifiers.@@ -91,7 +94,7 @@ def retention_floor_for_table(table_type: ast.TableOrSelectType, retention_monthfield_table_type = _table_filter_type(table_type)return ast.CompareOperation(op=ast.CompareOperationOp.Gt,- left=ast.Field(chain=["timestamp"], type=ast.FieldType(name="timestamp", table_type=field_table_type)),+ left=ast.Field(chain=[field_name], type=ast.FieldType(name=field_name, table_type=field_table_type)),right=ast.ArithmeticOperation(op=ast.ArithmeticOperationOp.Sub,left=ast.Call(name="now", args=[]),@@ -1258,16 +1261,12 @@ class ClickHousePrinter(BasePrinter):table_type: ast.TableType | ast.LazyTableType,node_type: ast.TableOrSelectType | None,) -> ast.Expr | None:- from posthog.hogql.database.schema.events import EventsTable- from posthog.hogql.database.schema.flag_evaluations import FlagEvaluationsTable-months = self.context.events_retention_monthsif months is None or node_type is None or not isinstance(table_type, ast.TableType):return None- # flag_evaluations holds flag calls that are events too, and a query on events can read them from there.- if not isinstance(table_type.table, (EventsTable, FlagEvaluationsTable)):+ if not isinstance(table_type.table, EventsRetentionTable):return None- return retention_floor_for_table(node_type, months)+ return retention_floor_for_table(node_type, months, field_name=table_type.table.events_retention_field)def _print_table_ref(self, table_type: ast.TableType | ast.LazyTableType, node: ast.JoinExpr) -> str:table = table_type.tablediff --git a/posthog/hogql/printer/test/test_printer.py b/posthog/hogql/printer/test/test_printer.pyindex ae6ff2bfa08f..ba02f06e7875 100644--- a/posthog/hogql/printer/test/test_printer.py+++ b/posthog/hogql/printer/test/test_printer.py@@ -224,19 +224,21 @@ class TestPrinter(BaseTest):@parameterized.expand([- (12, "events", "events"),- (84, "events", "events"),- (12, "posthog.flag_evaluations", "flag_evaluations"),+ ("events", 12, "events.timestamp"),+ ("events", 84, "events.timestamp"),+ ("web_stats_dimensional_preaggregated", 12, "web_stats_dimensional_preaggregated.period_bucket"),+ ("web_bounces_dimensional_preaggregated", 12, "web_bounces_dimensional_preaggregated.period_bucket"),+ ("posthog.flag_evaluations", 12, "flag_evaluations.timestamp"),])@override_settings(EVENTS_DATA_RETENTION_ENFORCED=True)- def test_events_retention_floor_applied(self, months: int, table: str, printed_table: str):+ def test_events_retention_floor_applied(self, table: str, months: int, floored_column: str):self.team.event_retention_months = monthscontext = HogQLContext(team_id=self.team.pk, team=self.team, enable_select_queries=True)with patch("products.feature_flags.backend.facade.flags.is_flag_evaluations_table_enabled", return_value=True):sql = self._select(f"select count() from {table}", context=context)assert f"toIntervalMonth({months})" in sql- assert f"greater({printed_table}.timestamp, minus(now" in sql+ assert f"greater({floored_column}, minus(now" in sql@override_settings(EVENTS_DATA_RETENTION_ENFORCED=False)def test_events_retention_floor_not_applied_when_disabled(self):diff --git a/posthog/hogql/printer/test/test_property_access_control.py b/posthog/hogql/printer/test/test_property_access_control.pyindex b1b3339cf619..23239ae103f5 100644--- a/posthog/hogql/printer/test/test_property_access_control.py+++ b/posthog/hogql/printer/test/test_property_access_control.py@@ -2,14 +2,21 @@ import uuidfrom posthog.test.base import BaseTest, ClickhouseTestMixin, cleanup_materialized_columns, materialized+from django.test import SimpleTestCase+from parameterized import parameterizedfrom posthog.schema import HogQLQueryModifiers, PersonsOnEventsModefrom posthog.hogql.context import HogQLContext+from posthog.hogql.database.schema.web_analytics_preaggregated import (+ WebBouncesDimensionalPreAggregatedTable,+ WebStatsDimensionalPreAggregatedTable,+)from posthog.hogql.parser import parse_selectfrom posthog.hogql.printer import prepare_and_print_astfrom posthog.hogql.query import execute_hogql_query+from posthog.hogql.restricted_properties import WEB_DIMENSIONAL_MIRRORED_COLUMNSfrom posthog.constants import AvailableFeaturefrom posthog.models import PropertyDefinition@@ -18,6 +25,34 @@ from posthog.models.event.util import create_eventfrom products.access_control.backend.models.property_access_control import PropertyAccessControlfrom products.access_control.backend.property_access_control import PropertyAccessLevel+# Columns that hold no event or session property value: keys and aggregate states.+_WEB_DIMENSIONAL_UNMIRRORED_COLUMNS = frozenset(+ {+ "period_bucket",+ "team_id",+ "job_id",+ "persons_uniq_state",+ "pageviews_count_state",+ "sessions_uniq_state",+ "bounces_count_state",+ "total_session_duration_state",+ "total_session_count_state",+ }+)+++class TestWebDimensionalMirrorCoverage(SimpleTestCase):+ @parameterized.expand(+ [+ ("stats", WebStatsDimensionalPreAggregatedTable),+ ("bounces", WebBouncesDimensionalPreAggregatedTable),+ ]+ )+ def test_every_dimension_column_is_mapped_or_listed_safe(self, _name: str, table_class: type) -> None:+ dimension_columns = table_class().fields.keys() - _WEB_DIMENSIONAL_UNMIRRORED_COLUMNS++ assert dimension_columns - WEB_DIMENSIONAL_MIRRORED_COLUMNS.keys() == set()+class TestRestrictPropertiesInHogQL(BaseTest):"""Integration tests for property-level access control in HogQL query compilation."""@@ -418,6 +453,44 @@ class TestRestrictPropertiesInHogQL(BaseTest):assert "JSONExtract" not in sqlassert (printed_column or expression.split("[")[0]) not in sql+ @parameterized.expand(+ [+ ("stats_pathname", "web_stats_dimensional_preaggregated", "pathname", "$pathname"),+ ("stats_viewport", "web_stats_dimensional_preaggregated", "viewport_width", "$viewport_width"),+ ("bounces_country", "web_bounces_dimensional_preaggregated", "country_code", "$geoip_country_code"),+ ("bounces_metadata", "web_bounces_dimensional_preaggregated", "mat_metadata_backend", "metadata.backend"),+ ("stats_metadata_parent", "web_stats_dimensional_preaggregated", "mat_metadata_loggedIn", "metadata"),+ ("stats_entry_pathname", "web_stats_dimensional_preaggregated", "entry_pathname", "$pathname"),+ ("bounces_end_pathname", "web_bounces_dimensional_preaggregated", "end_pathname", "$pathname"),+ ("stats_end_pathname_url", "web_stats_dimensional_preaggregated", "end_pathname", "$current_url"),+ (+ "bounces_referring_domain",+ "web_bounces_dimensional_preaggregated",+ "referring_domain",+ "$referring_domain",+ ),+ ("stats_utm_source", "web_stats_dimensional_preaggregated", "utm_source", "utm_source"),+ ]+ )+ def test_web_dimensional_columns_use_the_source_property_rule(+ self, _name: str, table_name: str, column: str, property_name: str+ ) -> None:+ self.event_prop.name = property_name+ self.event_prop.save()+ PropertyAccessControl.objects.create(+ team=self.team,+ property_definition=self.event_prop,+ access_level=PropertyAccessLevel.NONE.value,+ )++ sql = self._compile_select(+ f"SELECT {column} AS restricted_value, host FROM {table_name} WHERE {column} = 'x' OR {column} IS NULL"+ )++ assert "NULL AS restricted_value" in sql+ assert f"{table_name}.{column}" not in sql+ assert f"{table_name}.host" in sql+def test_ai_json_content_is_not_an_event_property(self) -> None:PropertyAccessControl.objects.create(team=self.team,diff --git a/posthog/hogql/restricted_properties.py b/posthog/hogql/restricted_properties.pyindex e9e83366dcab..ba16cbec9c74 100644--- a/posthog/hogql/restricted_properties.py+++ b/posthog/hogql/restricted_properties.py@@ -12,6 +12,10 @@ from posthog.hogql.database.schema.events_with_flag_calls import EVENT_ROW_TABLEfrom posthog.hogql.database.schema.flag_evaluations import FlagEvaluationsPersonSubTable, FlagEvaluationsTablefrom posthog.hogql.database.schema.groups import GroupsTable, RawGroupsTablefrom posthog.hogql.database.schema.persons import PersonsTable, RawPersonsTable+from posthog.hogql.database.schema.web_analytics_preaggregated import (+ WebBouncesDimensionalPreAggregatedTable,+ WebStatsDimensionalPreAggregatedTable,+)from posthog.clickhouse.events_json import TEMPORARY_PROPERTIES_COLUMN, UNPARSEABLE_PROPERTIES_KEYfrom posthog.constants import GROUP_TYPES_LIMIT@@ -54,16 +58,48 @@ _AI_EVENTS_MIRRORED_COLUMNS |= {subcolumn: _AI_EVENTS_MIRRORED_COLUMNS[source] for subcolumn, source in AI_SUBCOLUMN_TO_SOURCE.items()}+# Web analytics dimensional columns, mapped to the event properties the INSERT templates in+# `web_dimensional_precompute.py` read to fill them. The session-level columns come from the sessions+# table, which copies them from event properties, so they follow the rule of that event property. The+# entry and end pathnames are `path($current_url)` there, and they also follow `$pathname` because the+# SDK derives `$pathname` from the same URL. A guard test fails when a dimension column is missing here.+WEB_DIMENSIONAL_MIRRORED_COLUMNS: dict[str, tuple[str, ...]] = {+ "host": ("$host",),+ "device_type": ("$device_type",),+ "pathname": ("$pathname",),+ "entry_pathname": ("$current_url", "$pathname"),+ "end_pathname": ("$current_url", "$pathname"),+ "browser": ("$browser",),+ "os": ("$os",),+ "viewport_width": ("$viewport_width",),+ "viewport_height": ("$viewport_height",),+ "referring_domain": ("$referring_domain",),+ "utm_source": ("utm_source",),+ "utm_medium": ("utm_medium",),+ "utm_campaign": ("utm_campaign",),+ "utm_term": ("utm_term",),+ "utm_content": ("utm_content",),+ "has_gclid": ("gclid",),+ "has_gad_source_paid_search": ("gad_source",),+ "has_fbclid": ("fbclid",),+ "country_code": ("$geoip_country_code",),+ "city_name": ("$geoip_city_name",),+ "region_code": ("$geoip_subdivision_1_code",),+ "region_name": ("$geoip_subdivision_1_name",),+ "mat_metadata_loggedIn": ("metadata.loggedIn",),+ "mat_metadata_backend": ("metadata.backend",),+}+-def mirrored_property_for_column(table_type: ast.Type, column_name: str, context: HogQLContext) -> str | None:- """The event property a typed column copies verbatim, or None when the column copies nothing.+def mirrored_properties_for_column(table_type: ast.Type, column_name: str, context: HogQLContext) -> tuple[str, ...]:+ """The event properties a typed column copies, or an empty tuple when the column copies nothing.- A caller that prints one of these columns must mask the read when the property is restricted, in a predicate+ A caller that prints one of these columns must mask the read when any of the properties is restricted, in a predicateas much as in a SELECT: an unmasked column lets a filter narrow down a value the same user cannot read.`events` exposes mirror columns of its own that nothing consults this for; see ACCESS_CONTROL.md."""if not isinstance(table_type, ast.BaseTableType):- return None+ return ()try:table = table_type.resolve_database_table(context)@@ -71,13 +107,17 @@ def mirrored_property_for_column(table_type: ast.Type, column_name: str, context# Fail-open to match restricted_property_keys_for_table_type, whose docstring explains why this is# unreachable today; log so a future table type that can raise does not silently unmask a column.logger.warning("mirrored_property_table_resolution_failed", table_type=type(table_type).__name__)- return None+ return ()+ if isinstance(table, WebStatsDimensionalPreAggregatedTable | WebBouncesDimensionalPreAggregatedTable):+ return WEB_DIMENSIONAL_MIRRORED_COLUMNS.get(column_name, ())if isinstance(table, AiEventsTable):- return _AI_EVENTS_MIRRORED_COLUMNS.get(column_name)- if isinstance(table, FlagEvaluationsTable):- return _FLAG_EVALUATIONS_MIRRORED_COLUMNS.get(column_name)- return None+ source = _AI_EVENTS_MIRRORED_COLUMNS.get(column_name)+ elif isinstance(table, FlagEvaluationsTable):+ source = _FLAG_EVALUATIONS_MIRRORED_COLUMNS.get(column_name)+ else:+ source = None+ return (source,) if source is not None else ()def restricted_feature_flag_names(keys: Iterable[str]) -> list[str]:@@ -122,7 +162,15 @@ def restricted_property_keys_for_table_type(elif isinstance(table, EventsGroupSubTable):prop_def_type = PropertyDefinition.Type.GROUPgroup_type_index = table.group_index- elif isinstance(table, (*EVENT_ROW_TABLES, AiEventsTable)):+ elif isinstance(+ table,+ (+ *EVENT_ROW_TABLES,+ AiEventsTable,+ WebStatsDimensionalPreAggregatedTable,+ WebBouncesDimensionalPreAggregatedTable,+ ),+ ):prop_def_type = PropertyDefinition.Type.EVENTelif isinstance(table, (PersonsTable, RawPersonsTable)):prop_def_type = PropertyDefinition.Type.PERSONdiff --git a/posthog/hogql/test/__snapshots__/test_resolver.ambr b/posthog/hogql/test/__snapshots__/test_resolver.ambrindex 60027a174528..eac52284c91e 100644--- a/posthog/hogql/test/__snapshots__/test_resolver.ambr+++ b/posthog/hogql/test/__snapshots__/test_resolver.ambr@@ -23,6 +23,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -2309,6 +2310,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -5325,6 +5327,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -6368,6 +6371,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -9544,6 +9548,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -11242,6 +11247,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -12942,6 +12948,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -13247,6 +13254,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -13900,6 +13908,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -14241,6 +14250,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -14586,6 +14596,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -14971,6 +14982,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -15362,6 +15374,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -15725,6 +15738,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -16089,6 +16103,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -16436,6 +16451,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -16897,6 +16913,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -17199,6 +17216,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,@@ -17523,6 +17541,7 @@table_type: {table: {description: "Every analytics event captured for the project. The central fact table for product analytics.",+ events_retention_field: "timestamp",fields: {$group_0: {description: None,diff --git a/posthog/hogql/transforms/clickhouse_property_resolution.py b/posthog/hogql/transforms/clickhouse_property_resolution.pyindex c438276ce9be..42d5624a919b 100644--- a/posthog/hogql/transforms/clickhouse_property_resolution.py+++ b/posthog/hogql/transforms/clickhouse_property_resolution.py@@ -44,7 +44,8 @@ from posthog.hogql.functions.mapping import HOGQL_COMPARISON_MAPPINGfrom posthog.hogql.printer.base import resolve_field_typefrom posthog.hogql.printer.clickhouse import AI_BLOOM_FILTER_PROPERTIES, COLUMNS_WITH_HACKY_OPTIMIZED_NULL_HANDLINGfrom posthog.hogql.restricted_properties import (- mirrored_property_for_column,+ key_read_overlaps_restriction,+ mirrored_properties_for_column,property_path_overlaps_restriction,restricted_feature_flag_names,restricted_key_path,@@ -602,11 +603,11 @@ def _map_value_read(blob: ast.Expr, key: str) -> ast.Expr:)-def _mirrored_source_property(field_type: ast.FieldType, context: HogQLContext) -> str | None:+def _mirrored_source_properties(field_type: ast.FieldType, context: HogQLContext) -> tuple[str, ...]:resolved_field = field_type.resolve_database_field(context)if not isinstance(resolved_field, DatabaseField):- return None- return mirrored_property_for_column(field_type.table_type, resolved_field.name, context)+ return ()+ return mirrored_properties_for_column(field_type.table_type, resolved_field.name, context)def feature_flag_variant_read(value: ast.Expr) -> ast.Expr:@@ -933,11 +934,11 @@ def _substitute_value_read(node: ast.PropertyAccess, context: HogQLContext) -> a# JSONDropKeys strips the key, so extracting it always yields '' which scrubs to NULL — so return that constant# directly and skip the wasted drop-then-extract. (The column resolvers also decline, so comparisons over a# restricted property never read the backing column either; their operand falls through to this same NULL.)- source_property = _mirrored_source_property(field_type, context)+ source_properties = _mirrored_source_properties(field_type, context)restricted_keys = restricted_property_keys_for_table_type(field_type.table_type, context)if (- source_property in restricted_keys- if source_property is not None+ any(source_property in restricted_keys for source_property in source_properties)+ if source_propertieselse restricted_key_path(node.keys, restricted_keys) is not None):_record_property_usage(context, None)@@ -1318,10 +1319,14 @@ class ClickHousePropertyResolver(GroupedClauseTracker, CloningVisitor):Must match what `_substitute_value_read` builds for the source property, so the mirror column andits source property become the same AST node and agree on nullability and comparison printing."""- source_property = _mirrored_source_property(field_type, self.context)- if source_property is None:+ source_properties = _mirrored_source_properties(field_type, self.context)+ if not source_properties:return None- if source_property not in restricted_property_keys_for_table_type(field_type.table_type, self.context):+ restricted_keys = restricted_property_keys_for_table_type(field_type.table_type, self.context)+ # A dotted source such as `metadata.backend` is restricted by its parent `metadata` too.+ if not any(+ key_read_overlaps_restriction(source_property, restricted_keys) for source_property in source_properties+ ):return Nonereturn ast.Constant(value=None, type=ast.StringType(nullable=True))diff --git a/posthog/schema.py b/posthog/schema.pyindex e906ca4079f0..45eb663e6cfb 100644--- a/posthog/schema.py+++ b/posthog/schema.py@@ -3672,6 +3672,20 @@ class WebAnalyticsExternalSummaryRequest(BaseModel):explicit_date: bool | None = None+class WebAnalyticsQueryModifiers(BaseModel):+ model_config = ConfigDict(+ extra="forbid",+ )+ useDimensionalTables: bool | None = Field(+ default=None,+ description=(+ "Serve web overview and stats table queries from the dimensional precompute"+ " tables when they hold ready data for the whole date range. Falls back to"+ " the normal path otherwise."+ ),+ )++class WebAnalyticsSampling(BaseModel):model_config = ConfigDict(extra="forbid",@@ -27386,6 +27400,10 @@ class WebBotsTableQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebExternalClicksTableQuery(BaseModel):@@ -27424,6 +27442,10 @@ class WebExternalClicksTableQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebGoalsQuery(BaseModel):@@ -27469,6 +27491,10 @@ class WebGoalsQuery(BaseModel):),)version: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebNotableChangesQuery(BaseModel):@@ -27506,6 +27532,10 @@ class WebNotableChangesQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebOverviewQuery(BaseModel):@@ -27550,6 +27580,10 @@ class WebOverviewQuery(BaseModel):),)version: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebPageDirectoryQueryResponse(BaseModel):@@ -27640,6 +27674,10 @@ class WebPageURLSearchQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebStatsTableQuery(BaseModel):@@ -27692,6 +27730,10 @@ class WebStatsTableQuery(BaseModel):),)version: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebVitalsItem(BaseModel):@@ -30478,6 +30520,10 @@ class MarketingAnalyticsAggregatedQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class MarketingAnalyticsAttributionPathsQuery(BaseModel):@@ -30551,6 +30597,10 @@ class MarketingAnalyticsAttributionPathsQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class MarketingAnalyticsAttributionQuery(BaseModel):@@ -30627,6 +30677,10 @@ class MarketingAnalyticsAttributionQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class MarketingAnalyticsConfig(BaseModel):@@ -30736,6 +30790,10 @@ class MarketingAnalyticsRetentionQuery(BaseModel):)useSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class MarketingAnalyticsSearchQuery(BaseModel):@@ -30809,6 +30867,10 @@ class MarketingAnalyticsTableQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class MaxInnerUniversalFiltersGroup(BaseModel):@@ -31690,6 +31752,10 @@ class WebAgentAnalyticsQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebPageDirectoryQuery(BaseModel):@@ -31735,6 +31801,10 @@ class WebPageDirectoryQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class WebVitalsPathBreakdownQuery(BaseModel):@@ -31782,6 +31852,10 @@ class WebVitalsPathBreakdownQuery(BaseModel):),)version: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class ActionsNode(BaseModel):@@ -34565,6 +34639,10 @@ class WebVitalsQuery(BaseModel):tags: QueryLogTags | None = NoneuseSessionsTable: bool | None = Noneversion: float | None = Field(default=None, description="version of the node, used for schema migrations")+ webQueryModifiers: WebAnalyticsQueryModifiers | None = Field(+ default=None,+ description="Per-request switches for how web analytics serves this query.",+ )class DocumentArtifactContent(BaseModel):diff --git a/products/dashboards/frontend/generated/api.schemas.ts b/products/dashboards/frontend/generated/api.schemas.tsindex 379c7cc097e1..a651b5e3e8b8 100644--- a/products/dashboards/frontend/generated/api.schemas.ts+++ b/products/dashboards/frontend/generated/api.schemas.ts@@ -4883,6 +4883,11 @@ export interface WebAnalyticsSamplingApi {forceSamplingRate?: SamplingRateApi | null}+export interface WebAnalyticsQueryModifiersApi {+ /** Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise. */+ useDimensionalTables?: boolean | null+}+export interface WebStatsTableQueryApi {/** Groups aggregation - not used in Web Analytics but required for type compatibility */aggregation_group_type_index?: number | null@@ -4924,6 +4929,8 @@ export interface WebStatsTableQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebAnalyticsItemKindApi = (typeof WebAnalyticsItemKindApi)[keyof typeof WebAnalyticsItemKindApi]@@ -5009,6 +5016,8 @@ export interface WebOverviewQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface ActionsPieApi {@@ -7439,6 +7448,8 @@ export interface WebExternalClicksTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebBotsBreakdownApi = (typeof WebBotsBreakdownApi)[keyof typeof WebBotsBreakdownApi]@@ -7508,6 +7519,8 @@ export interface WebBotsTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface WebGoalsQueryResponseApi {@@ -7579,6 +7592,8 @@ export interface WebGoalsQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface WebVitalsQueryApi {@@ -7622,6 +7637,8 @@ export interface WebVitalsQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebVitalsMetricApi = (typeof WebVitalsMetricApi)[keyof typeof WebVitalsMetricApi]@@ -7714,6 +7731,8 @@ export interface WebVitalsPathBreakdownQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface FiltersApi {@@ -8330,6 +8349,8 @@ export interface MarketingAnalyticsTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type MarketingAnalyticsAggregatedQueryResponseApiResults = { [key: string]: MarketingAnalyticsItemApi }@@ -8401,6 +8422,8 @@ export interface MarketingAnalyticsAggregatedQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type OrderDirection2Api = (typeof OrderDirection2Api)[keyof typeof OrderDirection2Api]diff --git a/products/desktop/packages/api-client/src/generated.ts b/products/desktop/packages/api-client/src/generated.tsindex 897c95f7d8c8..8c2b7df6db58 100644--- a/products/desktop/packages/api-client/src/generated.ts+++ b/products/desktop/packages/api-client/src/generated.ts@@ -3606,6 +3606,9 @@ export namespace Schemas {enabled: boolean | null;forceSamplingRate: SamplingRate | null;}>;+ export type WebAnalyticsQueryModifiers = Partial<{+ useDimensionalTables: boolean | null;+ }>;export type WebStatsTableQuery = {aggregation_group_type_index?: (number | null) | undefined;breakdownBy: WebStatsBreakdown;@@ -3646,6 +3649,7 @@ export namespace Schemas {useSessionsTable?: (boolean | null) | undefined;useWebAnalyticsPrecompute?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type WebAnalyticsItemKind =| "unit"@@ -3721,6 +3725,7 @@ export namespace Schemas {useSessionsTable?: (boolean | null) | undefined;useWebAnalyticsPrecompute?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type InsightActorsQuery = {breakdown?: (string | Array<string> | number | null) | undefined;@@ -7955,6 +7960,7 @@ export namespace Schemas {tags?: (QueryLogTags | null) | undefined;useSessionsTable?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type WebBotsBreakdown = "Crawler" | "Path";export type WebBotsTableQueryResponse = {@@ -8015,6 +8021,7 @@ export namespace Schemas {tags?: (QueryLogTags | null) | undefined;useSessionsTable?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type WebGoalsQueryResponse = {columns?: (Array<unknown> | null) | undefined;@@ -8081,6 +8088,7 @@ export namespace Schemas {useSessionsTable?: (boolean | null) | undefined;useWebAnalyticsPrecompute?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type WebVitalsQuery = {aggregation_group_type_index?: (number | null) | undefined;@@ -8123,6 +8131,7 @@ export namespace Schemas {tags?: (QueryLogTags | null) | undefined;useSessionsTable?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type WebVitalsMetric = "INP" | "LCP" | "CLS" | "FCP";export type WebVitalsPercentile = "p75" | "p90" | "p99";@@ -8187,6 +8196,7 @@ export namespace Schemas {useSessionsTable?: (boolean | null) | undefined;useWebAnalyticsPrecompute?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type Filters = Partial<{dateRange: DateRange | null;@@ -8459,6 +8469,7 @@ export namespace Schemas {tags?: (QueryLogTags | null) | undefined;useSessionsTable?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type MarketingAnalyticsAggregatedQueryResponse = {dataComputedAt?: (string | null) | undefined;@@ -8523,6 +8534,7 @@ export namespace Schemas {tags?: (QueryLogTags | null) | undefined;useSessionsTable?: (boolean | null) | undefined;version?: (number | null) | undefined;+ webQueryModifiers?: (WebAnalyticsQueryModifiers | null) | undefined;};export type OrderDirection2 = "ASC" | "DESC";export type ErrorTrackingPendingFingerprintIssueStateUpdate = {diff --git a/products/marketing_analytics/frontend/generated/api.schemas.ts b/products/marketing_analytics/frontend/generated/api.schemas.tsindex 21ab7c4846ce..bfa549aad94d 100644--- a/products/marketing_analytics/frontend/generated/api.schemas.ts+++ b/products/marketing_analytics/frontend/generated/api.schemas.ts@@ -1995,6 +1995,11 @@ export interface QueryLogTagsApi {scene?: string | null}+export interface WebAnalyticsQueryModifiersApi {+ /** Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise. */+ useDimensionalTables?: boolean | null+}+export interface MarketingAnalyticsTableQueryApi {/** Groups aggregation - not used in Web Analytics but required for type compatibility */aggregation_group_type_index?: number | null@@ -2041,6 +2046,8 @@ export interface MarketingAnalyticsTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface ConversionRecordingsRequestApi {diff --git a/products/persons/frontend/generated/api.schemas.ts b/products/persons/frontend/generated/api.schemas.tsindex 40bff931419c..3a9525440a03 100644--- a/products/persons/frontend/generated/api.schemas.ts+++ b/products/persons/frontend/generated/api.schemas.ts@@ -3753,6 +3753,11 @@ export interface WebAnalyticsSamplingApi {forceSamplingRate?: SamplingRateApi | null}+export interface WebAnalyticsQueryModifiersApi {+ /** Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise. */+ useDimensionalTables?: boolean | null+}+export interface WebStatsTableQueryApi {/** Groups aggregation - not used in Web Analytics but required for type compatibility */aggregation_group_type_index?: number | null@@ -3794,6 +3799,8 @@ export interface WebStatsTableQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebAnalyticsItemKindApi = (typeof WebAnalyticsItemKindApi)[keyof typeof WebAnalyticsItemKindApi]@@ -3879,6 +3886,8 @@ export interface WebOverviewQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface InsightActorsQueryApi {diff --git a/products/product_analytics/frontend/generated/api.schemas.ts b/products/product_analytics/frontend/generated/api.schemas.tsindex 2dea1faeb9c2..07ec152dc261 100644--- a/products/product_analytics/frontend/generated/api.schemas.ts+++ b/products/product_analytics/frontend/generated/api.schemas.ts@@ -3720,6 +3720,11 @@ export interface WebAnalyticsSamplingApi {forceSamplingRate?: SamplingRateApi | null}+export interface WebAnalyticsQueryModifiersApi {+ /** Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise. */+ useDimensionalTables?: boolean | null+}+export interface WebStatsTableQueryApi {/** Groups aggregation - not used in Web Analytics but required for type compatibility */aggregation_group_type_index?: number | null@@ -3761,6 +3766,8 @@ export interface WebStatsTableQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebAnalyticsItemKindApi = (typeof WebAnalyticsItemKindApi)[keyof typeof WebAnalyticsItemKindApi]@@ -3846,6 +3853,8 @@ export interface WebOverviewQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface ActionsPieApi {@@ -6276,6 +6285,8 @@ export interface WebExternalClicksTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebBotsBreakdownApi = (typeof WebBotsBreakdownApi)[keyof typeof WebBotsBreakdownApi]@@ -6345,6 +6356,8 @@ export interface WebBotsTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface WebGoalsQueryResponseApi {@@ -6416,6 +6429,8 @@ export interface WebGoalsQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface WebVitalsQueryApi {@@ -6459,6 +6474,8 @@ export interface WebVitalsQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type WebVitalsMetricApi = (typeof WebVitalsMetricApi)[keyof typeof WebVitalsMetricApi]@@ -6551,6 +6568,8 @@ export interface WebVitalsPathBreakdownQueryApi {useWebAnalyticsPrecompute?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export interface FiltersApi {@@ -7167,6 +7186,8 @@ export interface MarketingAnalyticsTableQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type MarketingAnalyticsAggregatedQueryResponseApiResults = { [key: string]: MarketingAnalyticsItemApi }@@ -7238,6 +7259,8 @@ export interface MarketingAnalyticsAggregatedQueryApi {useSessionsTable?: boolean | null/** version of the node, used for schema migrations */version?: number | null+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiersApi | null}export type ErrorTrackingOrderByApi = (typeof ErrorTrackingOrderByApi)[keyof typeof ErrorTrackingOrderByApi]diff --git a/products/web_analytics/backend/hogql_queries/stats_table.py b/products/web_analytics/backend/hogql_queries/stats_table.pyindex c7d695d971fc..e2e0ae44b0fa 100644--- a/products/web_analytics/backend/hogql_queries/stats_table.py+++ b/products/web_analytics/backend/hogql_queries/stats_table.py@@ -4,6 +4,8 @@ from dataclasses import dataclassfrom functools import cached_propertyfrom typing import Literal, Optional, Union, cast+import structlog+from posthog.schema import (CachedWebStatsTableQueryResponse,EventPropertyFilter,@@ -57,6 +59,18 @@ from products.web_analytics.backend.hogql_queries.stats_table_strategies importStatsTableQueryStrategy,)from products.web_analytics.backend.hogql_queries.web_analytics_query_runner import WebAnalyticsQueryRunner, map_columns+from products.web_analytics.backend.hogql_queries.web_dimensional_precompute import (+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE,+ WEB_STATS_DIMENSIONAL_READ_TABLE,+)+from products.web_analytics.backend.hogql_queries.web_dimensional_read import (+ DimensionalReadFailure,+ DimensionalStatsTableQueryBuilder,+ classify_dimensional_read_failure,+ dimensional_precompute_stale,+ dimensional_precomputed_at,+ ready_dimensional_job_ids,+)from products.web_analytics.backend.hogql_queries.web_lazy_precompute_common import lazy_precompute_ineligible_reasonfrom products.web_analytics.backend.hogql_queries.web_stats_frustration_lazy_precompute import (can_use_lazy_precompute as can_use_frustration_lazy_precompute,@@ -72,6 +86,8 @@ from products.web_analytics.backend.hogql_queries.web_stats_paths_lazy_precomputexecute_lazy_precomputed_read as execute_paths_lazy_precomputed_read,)+logger = structlog.get_logger(__name__)+BREAKDOWN_NULL_DISPLAY = "(none)"BREAKDOWN_REFERRER_PREFIX = "referrer:"@@ -121,35 +137,41 @@ class WebStatsTableQueryRunner(WebAnalyticsQueryRunner[WebStatsTableQueryRespons)self.preaggregated_query_builder = StatsTablePreAggregatedQueryBuilder(self)+ # Set by `_calculate` when it serves the dimensional read.+ self.dimensional_query_builder: Optional[DimensionalStatsTableQueryBuilder] = None++ def _is_preaggregated_shape(self) -> bool:+ return (+ not self.query.includeAvgTimeOnPage+ and not self.query.conversionGoal+ and self.preaggregated_query_builder.can_use_preaggregated_tables()+ )def _can_serve_preaggregated(self) -> bool:return bool(- self.modifiers- and self.modifiers.useWebAnalyticsPreAggregatedTables- and self.preaggregated_query_builder.can_use_preaggregated_tables()- and not self.query.includeAvgTimeOnPage- and not self.query.conversionGoal+ self.modifiers and self.modifiers.useWebAnalyticsPreAggregatedTables and self._is_preaggregated_shape()+ )++ def _preaggregated_strategy(+ self, builder: StatsTablePreAggregatedQueryBuilder, prefix: str+ ) -> ResolvedStatsTableStrategy:+ if self.query.breakdownBy == WebStatsBreakdown.PAGE:+ strategy = f"{prefix}_path_breakdown"+ elif self.query.breakdownBy == WebStatsBreakdown.INITIAL_PAGE and self.query.includeBounceRate:+ strategy = f"{prefix}_entry_bounce"+ else:+ strategy = prefix+ return ResolvedStatsTableStrategy(+ strategy=strategy,+ build_query=builder.get_query,+ uses_preaggregated_tables=True,)def _resolve_strategy(self) -> ResolvedStatsTableStrategy:+ if self.dimensional_query_builder is not None:+ return self._preaggregated_strategy(self.dimensional_query_builder, "stats_table_dimensional")if self._can_serve_preaggregated():- if self.query.breakdownBy == WebStatsBreakdown.PAGE:- return ResolvedStatsTableStrategy(- strategy="stats_table_preaggregated_path_breakdown",- build_query=self.preaggregated_query_builder.get_query,- uses_preaggregated_tables=True,- )- if self.query.breakdownBy == WebStatsBreakdown.INITIAL_PAGE and self.query.includeBounceRate:- return ResolvedStatsTableStrategy(- strategy="stats_table_preaggregated_entry_bounce",- build_query=self.preaggregated_query_builder.get_query,- uses_preaggregated_tables=True,- )- return ResolvedStatsTableStrategy(- strategy="stats_table_preaggregated",- build_query=self.preaggregated_query_builder.get_query,- uses_preaggregated_tables=True,- )+ return self._preaggregated_strategy(self.preaggregated_query_builder, "stats_table_preaggregated")strategy = self._get_strategy()return ResolvedStatsTableStrategy(@@ -801,6 +823,25 @@ WHERE and(return field, directiondef _calculate(self):+ self.dimensional_query_builder = None+ live_blocking_error: Exception | None = None+ if self._is_preaggregated_shape():+ job_ids = ready_dimensional_job_ids(+ self, tables=(WEB_STATS_DIMENSIONAL_READ_TABLE, WEB_BOUNCES_DIMENSIONAL_READ_TABLE)+ )+ if job_ids is not None:+ self.dimensional_query_builder = DimensionalStatsTableQueryBuilder(self, job_ids)+ try:+ return self._calculate_from_query()+ except Exception as error:+ failure = classify_dimensional_read_failure(error)+ if failure is DimensionalReadFailure.RERAISE:+ raise+ logger.exception("web_stats_dimensional_read_failed", team_id=self.team.pk)+ self.dimensional_query_builder = None+ if failure is DimensionalReadFailure.SKIP_LIVE:+ live_blocking_error = error+# Ask only the family that owns this shape. Trying all three in turn made the other two# refuse a query they could never have served, and each refusal recorded its own rejection# reason, so the last gate to run decided what the read reported.@@ -819,9 +860,18 @@ WHERE and(if lazy_result is not None:return self._build_response_from_lazy(lazy_result)+ if live_blocking_error is not None and not self._can_serve_preaggregated():+ raise live_blocking_error+ return self._calculate_from_query()++ def _calculate_from_query(self) -> WebStatsTableQueryResponse:# Preflight only when the live query will actually run as the id-set# shape — pre-aggregated serving must not pay a discarded events scan.- if self.should_use_session_id_set and not self._can_serve_preaggregated():+ if (+ self.should_use_session_id_set+ and self.dimensional_query_builder is None+ and not self._can_serve_preaggregated()+ ):self._session_id_set_selectivity_ok = self._check_session_id_set_selectivity()query = self.to_query()@@ -885,6 +935,7 @@ WHERE and(else WebAnalyticsPreComputeStrategy.LIVE)+ dimensional_job_ids = self.dimensional_query_builder.job_ids if self.dimensional_query_builder else Nonereturn WebStatsTableQueryResponse(columns=columns,results=results_mapped,@@ -894,6 +945,8 @@ WHERE and(modifiers=self.modifiers,preComputeStrategy=strategy,preComputeIneligibleReason=lazy_precompute_ineligible_reason(strategy),+ preComputeStale=dimensional_precompute_stale(dimensional_job_ids),+ preComputedAt=dimensional_precomputed_at(dimensional_job_ids),**self.paginator.response_params(),)diff --git a/products/web_analytics/backend/hogql_queries/test/test_dimensional_read_selection.py b/products/web_analytics/backend/hogql_queries/test/test_dimensional_read_selection.pynew file mode 100644index 000000000000..ac5bf1663597--- /dev/null+++ b/products/web_analytics/backend/hogql_queries/test/test_dimensional_read_selection.py@@ -0,0 +1,425 @@+import uuid+from datetime import UTC, datetime, timedelta+from zoneinfo import ZoneInfo++import time_machine+from unittest.mock import MagicMock, patch++from django.test import SimpleTestCase, override_settings++from parameterized import parameterized++from posthog.schema import (+ CustomChannelCondition,+ CustomChannelField,+ CustomChannelOperator,+ CustomChannelRule,+ EventPropertyFilter,+ FilterLogicalOperator,+ HogQLQueryModifiers,+ MaterializationMode,+ PersonsOnEventsMode,+ PropertyOperator,+ SessionPropertyFilter,+ SessionTableVersion,+ WebAnalyticsQueryModifiers,+ WebStatsBreakdown,+)++from posthog.hogql import ast++from products.analytics_platform.backend.lazy_computation.lazy_computation_executor import LazyComputationResult+from products.web_analytics.backend.hogql_queries.web_dimensional_precompute import (+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE,+ WEB_STATS_DIMENSIONAL_READ_TABLE,+ DimensionalJobIds,+ dimensional_job_ids,+)+from products.web_analytics.backend.hogql_queries.web_dimensional_read import (+ DimensionalStatsTableQueryBuilder,+ ready_dimensional_job_ids,+)++_READ = "products.web_analytics.backend.hogql_queries.web_dimensional_read"+_PRECOMPUTE = "products.web_analytics.backend.hogql_queries.web_dimensional_precompute"+_STATS_IDS = [uuid.uuid4()]+_BOUNCES_IDS = [uuid.uuid4()]+_JOB_IDS = DimensionalJobIds(stats=_STATS_IDS, bounces=_BOUNCES_IDS)+_BOTH_TABLES = (WEB_STATS_DIMENSIONAL_READ_TABLE, WEB_BOUNCES_DIMENSIONAL_READ_TABLE)+_NEW_YORK = ZoneInfo("America/New_York")+_SESSIONS_V3_FLOOR = datetime(2026, 2, 15, tzinfo=UTC)+_TEST_ACCOUNT_FILTERS = [{"key": "email", "value": "@example.com", "operator": "not_icontains", "type": "person"}]+_CUSTOM_CHANNEL_RULES = [+ CustomChannelRule(+ id="docs",+ channel_type="Docs",+ combiner=FilterLogicalOperator.AND_,+ items=[+ CustomChannelCondition(+ id="url",+ key=CustomChannelField.URL,+ op=CustomChannelOperator.ICONTAINS,+ value="/docs",+ )+ ],+ )+]+_CHANNEL_FILTER = SessionPropertyFilter(key="$channel_type", value="Docs", operator=PropertyOperator.EXACT)+++def _event_filter(key: str, value: str = "x") -> EventPropertyFilter:+ return EventPropertyFilter(key=key, value=value, operator=PropertyOperator.EXACT)+++def _session_filter(key: str, value: str = "x") -> SessionPropertyFilter:+ return SessionPropertyFilter(key=key, value=value, operator=PropertyOperator.EXACT)+++def _runner(+ opted_in: bool | None,+ compare_from: datetime | None = None,+ tz: ZoneInfo = _NEW_YORK,+ test_account_filters: list | None = None,+ date_from: datetime | None = None,+ date_to: datetime | None = None,+ compare_to: datetime | None = None,+ custom_channel_rules: list[CustomChannelRule] | None = None,+ breakdown_by: WebStatsBreakdown = WebStatsBreakdown.PAGE,+ properties: list | None = None,+ kind: str = "WebStatsTableQuery",+ modifiers: HogQLQueryModifiers | None = None,+) -> MagicMock:+ runner = MagicMock()+ runner.team.pk = 1+ runner.query.kind = kind+ runner.query.webQueryModifiers = (+ None if opted_in is None else WebAnalyticsQueryModifiers(useDimensionalTables=opted_in)+ )+ runner.query.properties = properties or []+ runner.query.breakdownBy = breakdown_by+ runner.modifiers = modifiers or HogQLQueryModifiers(customChannelTypeRules=custom_channel_rules)+ runner._test_account_filters = test_account_filters or []+ # A project-local range: the current day in New York ends after the UTC now below.+ runner.query_date_range.date_from.return_value = date_from or datetime(2026, 3, 1, tzinfo=tz)+ runner.query_date_range.date_to.return_value = date_to or datetime(2026, 3, 31, 23, 59, 59, 999999, tzinfo=tz)+ if compare_from is None:+ runner.query_compare_to_date_range = None+ else:+ runner.query_compare_to_date_range.date_from.return_value = compare_from+ runner.query_compare_to_date_range.date_to.return_value = compare_to or (+ datetime(2026, 3, 1, tzinfo=compare_from.tzinfo) - timedelta(microseconds=1)+ )+ return runner+++def _result(+ ready: bool, job_ids: list[uuid.UUID], stale: bool = False, computed_at: datetime | None = None+) -> LazyComputationResult:+ return LazyComputationResult(+ ready=ready, job_ids=job_ids if ready else [], stale=stale, computed_at=computed_at if ready else None+ )+++@time_machine.travel(datetime(2026, 3, 31, 12, tzinfo=UTC), tick=False)+class TestDimensionalReadSelection(SimpleTestCase):+ def setUp(self) -> None:+ retention_patcher = patch(f"{_READ}.events_retention_months_for_team", return_value=None)+ self.retention_months = retention_patcher.start()+ self.addCleanup(retention_patcher.stop)+ insert_modifiers_patcher = patch(f"{_READ}.dimensional_insert_modifiers", return_value=HogQLQueryModifiers())+ insert_modifiers_patcher.start()+ self.addCleanup(insert_modifiers_patcher.stop)+ flag_patcher = patch(f"{_READ}.feature_enabled_or_false", return_value=True)+ self.flag_enabled = flag_patcher.start()+ self.addCleanup(flag_patcher.stop)++ @parameterized.expand(+ [+ ("no_compare", None, _NEW_YORK, datetime(2026, 3, 1, 5, tzinfo=UTC)),+ (+ "compare_period_widens_start",+ datetime(2026, 2, 1, tzinfo=_NEW_YORK),+ _NEW_YORK,+ datetime(2026, 2, 1, 5, tzinfo=UTC),+ ),+ ("whole_hour_east_of_utc", None, ZoneInfo("Asia/Tokyo"), datetime(2026, 2, 28, 15, tzinfo=UTC)),+ ]+ )+ def test_opted_in_query_checks_the_utc_range_up_to_now(self, _name, compare_from, tz, expected_start):+ with (+ patch(f"{_READ}.team_has_property_access_rules", return_value=False),+ patch(f"{_READ}.dimensional_job_ids", return_value=_JOB_IDS) as mock_job_ids,+ ):+ assert ready_dimensional_job_ids(_runner(True, compare_from, tz=tz), _BOTH_TABLES) == _JOB_IDS++ _team, start, end = mock_job_ids.call_args.args+ assert (start, end) == (expected_start, datetime(2026, 3, 31, 12, tzinfo=UTC))+ assert start.tzinfo == UTC and end.tzinfo == UTC++ @parameterized.expand(+ [+ ("no_modifiers", _runner(None), False, _JOB_IDS),+ ("opted_out", _runner(False), False, _JOB_IDS),+ ("property_access_rules", _runner(True), True, _JOB_IDS),+ ("test_account_filters", _runner(True, test_account_filters=_TEST_ACCOUNT_FILTERS), False, _JOB_IDS),+ ("half_hour_timezone", _runner(True, tz=ZoneInfo("Asia/Kolkata")), False, _JOB_IDS),+ (+ "half_hour_timezone_in_compare_period_only",+ _runner(True, compare_from=datetime(2026, 2, 1, tzinfo=ZoneInfo("Asia/Kolkata"))),+ False,+ _JOB_IDS,+ ),+ (+ "start_inside_an_hour",+ _runner(+ True,+ date_from=datetime(2026, 3, 1, 10, 30, tzinfo=UTC),+ date_to=datetime(2026, 3, 1, 11, 59, 59, 999999, tzinfo=UTC),+ ),+ False,+ _JOB_IDS,+ ),+ (+ "past_end_inside_an_hour",+ _runner(+ True,+ date_from=datetime(2026, 3, 1, 10, tzinfo=UTC),+ date_to=datetime(2026, 3, 1, 11, 30, tzinfo=UTC),+ ),+ False,+ _JOB_IDS,+ ),+ (+ "past_end_on_an_hour_boundary",+ _runner(+ True,+ date_from=datetime(2026, 3, 1, 10, tzinfo=UTC),+ date_to=datetime(2026, 3, 1, 11, tzinfo=UTC),+ ),+ False,+ _JOB_IDS,+ ),+ (+ "compare_end_inside_an_hour",+ _runner(+ True,+ compare_from=datetime(2026, 2, 1, tzinfo=_NEW_YORK),+ compare_to=datetime(2026, 2, 28, 23, 59, 59, tzinfo=_NEW_YORK),+ ),+ False,+ _JOB_IDS,+ ),+ (+ "custom_channel_rules_with_channel_breakdown",+ _runner(+ True,+ custom_channel_rules=_CUSTOM_CHANNEL_RULES,+ breakdown_by=WebStatsBreakdown.INITIAL_CHANNEL_TYPE,+ ),+ False,+ _JOB_IDS,+ ),+ (+ "custom_channel_rules_with_channel_filter",+ _runner(True, custom_channel_rules=_CUSTOM_CHANNEL_RULES, properties=[_CHANNEL_FILTER]),+ False,+ _JOB_IDS,+ ),+ ("not_ready", _runner(True), False, None),+ ("lookup_fails", _runner(True), False, RuntimeError("postgres down")),+ (+ "overview_pathname_filter",+ _runner(True, kind="WebOverviewQuery", properties=[_event_filter("$pathname", "/pricing")]),+ False,+ _JOB_IDS,+ ),+ (+ "event_referring_domain_filter",+ _runner(True, properties=[_event_filter("$referring_domain", "example.com")]),+ False,+ _JOB_IDS,+ ),+ ("entry_gclid_value_filter", _runner(True, properties=[_session_filter("$entry_gclid")]), False, _JOB_IDS),+ (+ "entry_fbclid_value_filter",+ _runner(True, properties=[_session_filter("$entry_fbclid")]),+ False,+ _JOB_IDS,+ ),+ ("browser_version_filter", _runner(True, properties=[_event_filter("$browser_version")]), False, _JOB_IDS),+ ("os_version_filter", _runner(True, properties=[_event_filter("$os_version")]), False, _JOB_IDS),+ ("country_name_filter", _runner(True, properties=[_event_filter("$geoip_country_name")]), False, _JOB_IDS),+ ("time_zone_filter", _runner(True, properties=[_event_filter("$geoip_time_zone")]), False, _JOB_IDS),+ (+ "persons_on_events_mode_differs_from_the_build",+ _runner(True, modifiers=HogQLQueryModifiers(personsOnEventsMode=PersonsOnEventsMode.DISABLED)),+ False,+ _JOB_IDS,+ ),+ (+ "materialization_mode_differs_from_the_build",+ _runner(+ True,+ modifiers=HogQLQueryModifiers(materializationMode=MaterializationMode.LEGACY_NULL_AS_STRING),+ ),+ False,+ _JOB_IDS,+ ),+ ]+ )+ def test_query_falls_back(self, _name, runner, has_rules, job_ids):+ with (+ patch(f"{_READ}.team_has_property_access_rules", return_value=has_rules),+ patch(f"{_READ}.dimensional_job_ids", side_effect=[job_ids]),+ ):+ assert ready_dimensional_job_ids(runner, _BOTH_TABLES) is None++ def test_request_modifier_is_ignored_when_the_flag_is_off(self):+ self.flag_enabled.return_value = False+ with (+ patch(f"{_READ}.team_has_property_access_rules", return_value=False),+ patch(f"{_READ}.dimensional_job_ids", return_value=_JOB_IDS) as mock_job_ids,+ ):+ assert ready_dimensional_job_ids(_runner(True), _BOTH_TABLES) is None++ assert self.flag_enabled.call_args.args[0] == "web-analytics-query-modifiers"+ mock_job_ids.assert_not_called()++ @parameterized.expand(+ [+ ("retention_not_enforced", _runner(True, compare_from=datetime(2026, 2, 1, tzinfo=_NEW_YORK)), None, True),+ ("both_periods_inside_retention", _runner(True), 1, True),+ (+ "stats_table_pathname_filter",+ _runner(True, properties=[_event_filter("$pathname", "/pricing")]),+ None,+ True,+ ),+ (+ "compare_period_crosses_retention",+ _runner(True, compare_from=datetime(2026, 2, 1, tzinfo=_NEW_YORK)),+ 1,+ False,+ ),+ (+ "custom_channel_rules_without_channel_dependency",+ _runner(True, custom_channel_rules=_CUSTOM_CHANNEL_RULES),+ None,+ True,+ ),+ ("sessions_v3_after_the_floor", _runner(True), None, True, SessionTableVersion.V3),+ (+ "sessions_v3_compare_period_before_the_floor",+ _runner(True, compare_from=datetime(2026, 2, 1, tzinfo=_NEW_YORK)),+ None,+ False,+ SessionTableVersion.V3,+ ),+ ("sessions_v3_region_without_a_backfill", _runner(True), None, False, SessionTableVersion.V3, "EU"),+ (+ "sessions_v2_before_the_v3_floor",+ _runner(True, compare_from=datetime(2026, 2, 1, tzinfo=_NEW_YORK)),+ None,+ True,+ SessionTableVersion.V2,+ ),+ ]+ )+ def test_query_serves_only_inside_retention_sessions_coverage_and_channel_rules(+ self,+ _name,+ runner,+ retention_months,+ serves,+ session_table_version=SessionTableVersion.AUTO,+ region="US",+ ):+ self.retention_months.return_value = retention_months+ build_modifiers = HogQLQueryModifiers(sessionTableVersion=session_table_version)+ runner.modifiers.sessionTableVersion = session_table_version+ with (+ patch(f"{_READ}.team_has_property_access_rules", return_value=False),+ patch(f"{_READ}.dimensional_job_ids", return_value=_JOB_IDS),+ patch(f"{_READ}.dimensional_insert_modifiers", return_value=build_modifiers),+ patch(f"{_PRECOMPUTE}.is_cloud", return_value=True),+ patch.dict(f"{_PRECOMPUTE}.SESSIONS_V3_COVERAGE_START_BY_REGION", {"US": _SESSIONS_V3_FLOOR}, clear=True),+ override_settings(CLOUD_DEPLOYMENT=region),+ ):+ assert ready_dimensional_job_ids(runner, _BOTH_TABLES) == (_JOB_IDS if serves else None)++ @parameterized.expand(+ [+ ("both_ready", _BOTH_TABLES, _result(True, _STATS_IDS), _result(True, _BOUNCES_IDS), _JOB_IDS),+ ("stats_gap", _BOTH_TABLES, _result(False, _STATS_IDS), _result(True, _BOUNCES_IDS), None),+ ("bounces_gap", _BOTH_TABLES, _result(True, _STATS_IDS), _result(False, _BOUNCES_IDS), None),+ (+ "bounces_only_ignores_a_stats_gap",+ (WEB_BOUNCES_DIMENSIONAL_READ_TABLE,),+ _result(False, _STATS_IDS),+ _result(True, _BOUNCES_IDS),+ DimensionalJobIds(stats=None, bounces=_BOUNCES_IDS),+ ),+ (+ "stale_job_and_oldest_computed_at",+ _BOTH_TABLES,+ _result(True, _STATS_IDS, computed_at=datetime(2026, 3, 2, 10, tzinfo=UTC)),+ _result(True, _BOUNCES_IDS, stale=True, computed_at=datetime(2026, 3, 2, 9, tzinfo=UTC)),+ DimensionalJobIds(+ stats=_STATS_IDS,+ bounces=_BOUNCES_IDS,+ stale=True,+ computed_at=datetime(2026, 3, 2, 9, tzinfo=UTC),+ ),+ ),+ ]+ )+ def test_dimensional_job_ids_checks_only_the_requested_tables(+ self, _name, tables, stats_result, bounces_result, expected+ ):+ modifiers = HogQLQueryModifiers()+ with (+ patch(f"{_PRECOMPUTE}.ensure_web_stats_dimensional_precomputed", return_value=stats_result) as mock_stats,+ patch(+ f"{_PRECOMPUTE}.ensure_web_bounces_dimensional_precomputed", return_value=bounces_result+ ) as mock_bounces,+ ):+ assert (+ dimensional_job_ids(+ MagicMock(),+ datetime(2026, 3, 1, tzinfo=UTC),+ datetime(2026, 3, 2, tzinfo=UTC),+ tables=tables,+ modifiers=modifiers,+ )+ == expected+ )++ assert mock_stats.called is (WEB_STATS_DIMENSIONAL_READ_TABLE in tables)+ for mock_ensure in (mock_stats, mock_bounces):+ if mock_ensure.called:+ assert mock_ensure.call_args.kwargs["run_inserts"] is False+ assert mock_ensure.call_args.kwargs["modifiers"] is modifiers++ @parameterized.expand(+ [+ ("stats", WEB_STATS_DIMENSIONAL_READ_TABLE, _STATS_IDS),+ ("bounces", WEB_BOUNCES_DIMENSIONAL_READ_TABLE, _BOUNCES_IDS),+ ]+ )+ def test_builder_reads_dimensional_tables_filtered_to_their_job_ids(self, _name, table_name, job_ids):+ runner = _runner(True)+ runner.query_date_range.date_from.return_value = datetime(2026, 3, 1, tzinfo=UTC)+ builder = DimensionalStatsTableQueryBuilder(runner, _JOB_IDS)++ assert (builder.stats_table, builder.bounces_table) == (+ WEB_STATS_DIMENSIONAL_READ_TABLE,+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE,+ )+ job_id_filters = [+ expr+ for expr in builder._get_filters(table_name).exprs+ if isinstance(expr, ast.CompareOperation) and expr.op == ast.CompareOperationOp.In+ ]+ assert len(job_id_filters) == 1+ assert job_id_filters[0].left == ast.Field(chain=[table_name, "job_id"])+ assert job_id_filters[0].right == ast.Constant(value=[str(job_id) for job_id in job_ids])diff --git a/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_precompute.py b/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_precompute.pyindex 30d4bd9f8325..459a13ed1068 100644--- a/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_precompute.py+++ b/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_precompute.py@@ -140,6 +140,31 @@ class TestWebDimensionalPrecompute(ClickhouseTestMixin, APIBaseTest):assert jobs_after_second == jobs_after_firstassert set(second.job_ids) == set(first.job_ids)+ @parameterized.expand(+ [+ ("bounce_duration", {"bounceRateDurationSeconds": 30}),+ ("bounce_page_view_mode", {"bounceRatePageViewMode": "uniq_urls"}),+ ("sessions_join_mode", {"sessionsV2JoinMode": "string"}),+ # Never the team default, so the change is real in every persons-on-events CI mode.+ ("persons_on_events_mode", {"personsOnEventsMode": "disabled"}),+ ("materialization_mode", {"materializationMode": "legacy_null_as_string"}),+ ]+ )+ @time_machine.travel("2024-01-15T12:00:00Z", tick=False)+ def test_a_team_settings_change_misses_the_existing_jobs(self, _name: str, team_modifiers: dict):+ self._seed_two_sessions()+ built = ensure_web_bounces_dimensional_precomputed(self.team, WINDOW_START, WINDOW_END)+ assert built.ready++ self.team.modifiers = team_modifiers+ self.team.save()+ after_change = ensure_web_bounces_dimensional_precomputed(+ self.team, WINDOW_START, WINDOW_END, run_inserts=False+ )++ assert not after_change.ready+ assert after_change.job_ids == []+@parameterized.expand([("UTC",), ("Pacific/Honolulu",), ("Asia/Tokyo",)])def test_current_utc_day_refreshes_hourly_in_any_team_timezone(self, timezone: str):# 16:00 UTC is after local midnight in both non-UTC zones, so team-timezone TTL bands would@@ -297,7 +322,7 @@ class TestColdBackfillSplitsByTtlBandNotPerDay(unittest.TestCase):# DIMENSIONAL_TTL_SECONDS has 3 entries (0d / 2d / default) -> 3 INSERTs,# oldest first. NOT 90.ttls = [ttl for _s, _e, ttl in bands]- assert ttls == [90 * 86400, 86400, 3600]+ assert ttls == [90 * 86400, 86400, 45 * 60]# The bulk of history is ONE query covering many days, not one-per-day.oldest_start, oldest_end, _ = bands[0]diff --git a/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_read_parity.py b/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_read_parity.pynew file mode 100644index 000000000000..6071dc72b75f--- /dev/null+++ b/products/web_analytics/backend/hogql_queries/test/test_web_dimensional_read_parity.py@@ -0,0 +1,471 @@+from datetime import UTC, datetime++import time_machine+from posthog.test.base import _create_event, _create_person, flush_persons_and_events+from unittest.mock import patch++from django.test import override_settings++from parameterized import parameterized++from posthog.schema import (+ CustomChannelCondition,+ CustomChannelField,+ CustomChannelOperator,+ CustomChannelRule,+ DateRange,+ EventPropertyFilter,+ FilterLogicalOperator,+ HogQLQueryModifiers,+ PropertyOperator,+ SessionPropertyFilter,+ WebAnalyticsPreComputeStrategy,+ WebAnalyticsQueryModifiers,+ WebOverviewQuery,+ WebStatsBreakdown,+ WebStatsTableQuery,+ WebStatsTableQueryResponse,+)++from posthog.hogql.query import execute_hogql_query++from posthog.clickhouse.client import sync_execute+from posthog.errors import CHQueryErrorQueryWasCancelled+from posthog.exceptions import ClickHouseAtCapacity, ClickHouseQueryTimeOut+from posthog.models.utils import uuid7+from posthog.models.web_preaggregated.sql import WEB_BOUNCES_INSERT_SQL, WEB_STATS_INSERT_SQL++from products.web_analytics.backend.hogql_queries.stats_table import WebStatsTableQueryRunner+from products.web_analytics.backend.hogql_queries.test.web_preaggregated_test_base import (+ WebAnalyticsPreAggregatedTestBase,+)+from products.web_analytics.backend.hogql_queries.web_dimensional_precompute import (+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE,+ WEB_STATS_DIMENSIONAL_READ_TABLE,+ ensure_web_bounces_dimensional_precomputed,+ ensure_web_stats_dimensional_precomputed,+)+from products.web_analytics.backend.hogql_queries.web_dimensional_read import ready_dimensional_job_ids+from products.web_analytics.backend.hogql_queries.web_overview import WebOverviewQueryRunner++DIMENSIONAL_READ_TABLES = (WEB_STATS_DIMENSIONAL_READ_TABLE, WEB_BOUNCES_DIMENSIONAL_READ_TABLE)++DATE_FROM = "2024-01-01"+DATE_TO = "2024-01-02"+# A `date_to` of "2024-01-02" resolves to end-of-day, so the query spans the 01-01 and+# 01-02 daily windows. Precompute both so the read path's full-coverage gate is satisfied+# (events only land on 01-01; the 01-02 window precomputes to an empty, still-READY job).+WINDOW_START = datetime(2024, 1, 1, tzinfo=UTC)+WINDOW_END = datetime(2024, 1, 3, tzinfo=UTC)++LEGACY = HogQLQueryModifiers(useWebAnalyticsPreAggregatedTables=True)+OPT_IN = WebAnalyticsQueryModifiers(useDimensionalTables=True)+UNTAGGED_CHANNEL = HogQLQueryModifiers(+ customChannelTypeRules=[+ CustomChannelRule(+ id="untagged",+ channel_type="Untagged",+ combiner=FilterLogicalOperator.AND_,+ items=[+ CustomChannelCondition(+ id="source", key=CustomChannelField.UTM_SOURCE, op=CustomChannelOperator.IS_NOT_SET+ )+ ],+ )+ ]+)+++class TestWebDimensionalReadParity(WebAnalyticsPreAggregatedTestBase):+ def setUp(self):+ # STOP TTL MERGES before any data lands, so born-expired parts (the test's+ # frozen window is years behind the real CH clock) survive until we read.+ for table in ("sharded_web_stats_dimensional_preaggregated", "sharded_web_bounces_dimensional_preaggregated"):+ sync_execute(f"SYSTEM STOP TTL MERGES {table}")+ self.addCleanup(sync_execute, f"SYSTEM START TTL MERGES {table}")+ flag_patcher = patch(+ "products.web_analytics.backend.hogql_queries.web_dimensional_read.feature_enabled_or_false",+ return_value=True,+ )+ flag_patcher.start()+ self.addCleanup(flag_patcher.stop)+ super().setUp()++ def _pageview(self, distinct_id: str, session: str, ts: str, **props) -> None:+ _create_event(+ team=self.team,+ event="$pageview",+ distinct_id=distinct_id,+ timestamp=ts,+ properties={"$session_id": session, **props},+ )++ def _setup_test_data(self):+ # Two hosts × device types so a host+device filter is a real cross-section.+ sa, sb, sc = (str(uuid7("2024-01-01")) for _ in range(3))+ with time_machine.travel("2024-01-01T09:00:00Z", tick=False):+ for did in ("user_a", "user_b", "user_c"):+ _create_person(team_id=self.team.pk, distinct_ids=[did])+ app_desktop = {+ "$host": "app.example.com",+ "$device_type": "Desktop",+ "$browser": "Chrome",+ "$browser_version": 120,+ "$os": "Windows",+ "$viewport_width": 1920,+ "$viewport_height": 1080,+ }+ # user_a: two pageviews on app.example.com / Desktop (non-bounce).+ self._pageview("user_a", sa, "2024-01-01T10:00:00Z", **{**app_desktop, "$pathname": "/"})+ self._pageview("user_a", sa, "2024-01-01T10:03:00Z", **{**app_desktop, "$pathname": "/pricing"})+ # user_b: single pageview on app.example.com / Mobile (bounce).+ self._pageview(+ "user_b",+ sb,+ "2024-01-01T11:00:00Z",+ **{+ "$host": "app.example.com",+ "$device_type": "Mobile",+ "$browser": "Safari",+ "$os": "iOS",+ "$viewport_width": 390,+ "$viewport_height": 844,+ "$pathname": "/",+ },+ )+ # user_c: single pageview on blog.example.com / Desktop.+ self._pageview(+ "user_c",+ sc,+ "2024-01-01T12:00:00Z",+ **{+ "$host": "blog.example.com",+ "$device_type": "Desktop",+ "$browser": "Firefox",+ "$os": "Linux",+ "$viewport_width": 1280,+ "$viewport_height": 720,+ "$pathname": "/blog",+ },+ )+ flush_persons_and_events()++ # Populate v2 and dimensional over the same window. ensure_* runs at real+ # time, so the jobs' expires_at is genuinely in the future and READY.+ for table, insert_sql in (+ ("web_pre_aggregated_stats", WEB_STATS_INSERT_SQL),+ ("web_pre_aggregated_bounces", WEB_BOUNCES_INSERT_SQL),+ ):+ sync_execute(+ insert_sql(+ date_start=DATE_FROM,+ date_end=DATE_TO,+ team_ids=[self.team.pk],+ table_name=table,+ granularity="hourly",+ )+ )+ ensure_web_stats_dimensional_precomputed(self.team, WINDOW_START, WINDOW_END)+ ensure_web_bounces_dimensional_precomputed(self.team, WINDOW_START, WINDOW_END)++ def _run(+ self,+ breakdown: WebStatsBreakdown,+ modifiers: HogQLQueryModifiers | None,+ web_query_modifiers: WebAnalyticsQueryModifiers | None,+ properties=None,+ ) -> tuple[str, list, WebStatsTableQueryResponse]:+ runner = WebStatsTableQueryRunner(+ query=WebStatsTableQuery(+ dateRange=DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ properties=properties or [],+ breakdownBy=breakdown,+ limit=100,+ webQueryModifiers=web_query_modifiers,+ ),+ team=self.team,+ modifiers=modifiers,+ )+ response = runner.calculate()+ assert runner.used_preaggregated_tables, "expected the pre-aggregated path, not a live fallback"+ return runner.query_strategy(), self._sort_results(response.results), response++ def _run_live(self, breakdown: WebStatsBreakdown, properties=None) -> list:+ runner = WebStatsTableQueryRunner(+ query=WebStatsTableQuery(+ dateRange=DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ properties=properties or [],+ breakdownBy=breakdown,+ limit=100,+ ),+ team=self.team,+ )+ response = runner.calculate()+ assert not runner.used_preaggregated_tables+ return self._sort_results(response.results)++ @staticmethod+ def _counts(results: list) -> dict:+ # {breakdown_value: (visitors, views)} — anchors values so a both-routes-broken+ # case can't pass on equality alone.+ return {row[0]: (row[1][0], row[2][0]) for row in results}++ @parameterized.expand(+ [+ # No filter — Desktop: user_a (2 views) + user_c (1 view) = 2 visitors / 3 views; Mobile: user_b = 1 / 1.+ ("device_breakdown", None, {"Desktop": (2.0, 3.0), "Mobile": (1.0, 1.0)}),+ # Host + device cross-section the fixed-dimension tables uniquely serve: blog.example.com / user_c is+ # excluded, leaving only the two app.example.com sessions.+ (+ "host_and_device_filter",+ [EventPropertyFilter(key="$host", value="app.example.com", operator=PropertyOperator.EXACT)],+ {"Desktop": (1.0, 2.0), "Mobile": (1.0, 1.0)},+ ),+ ]+ )+ def test_dimensional_matches_v2(self, _name, properties, expected_counts):+ v2_strategy, v2, v2_response = self._run(WebStatsBreakdown.DEVICE_TYPE, LEGACY, None, properties=properties)+ # Only the request opt-in: the legacy modifier must not be needed to reach this path.+ dimensional_strategy, dimensional, dimensional_response = self._run(+ WebStatsBreakdown.DEVICE_TYPE, None, OPT_IN, properties=properties+ )++ assert v2_strategy == "stats_table_preaggregated"+ assert dimensional_strategy == "stats_table_dimensional"+ assert v2_response.preComputeStrategy == WebAnalyticsPreComputeStrategy.PRE_AGGREGATED+ assert dimensional_response.preComputeStrategy == WebAnalyticsPreComputeStrategy.PRE_AGGREGATED+ assert dimensional_response.preComputedAt is not None+ assert dimensional_response.preComputeStale is None+ assert dimensional == v2, f"{_name} mismatch\nv2={v2}\ndimensional={dimensional}"+ assert self._counts(dimensional) == expected_counts+ assert self._counts(self._run_live(WebStatsBreakdown.DEVICE_TYPE, properties)) == expected_counts++ @parameterized.expand(+ [+ ("legacy_on_falls_back_to_v2", LEGACY, True),+ ("legacy_off_falls_back_to_live", None, False),+ ]+ )+ def test_partial_coverage_does_not_read_dimensional(self, _name, modifiers, expect_v2):+ # Dimensional is precomputed for 2024-01-01 and 01-02 only (setUp). A window that+ # also spans 2024-01-03 is only partially covered, so the job_id filter would+ # silently drop the uncovered day. The runner must skip dimensional instead.+ runner = WebStatsTableQueryRunner(+ team=self.team,+ query=WebStatsTableQuery(+ dateRange=DateRange(date_from="2024-01-01", date_to="2024-01-03"),+ properties=[],+ breakdownBy=WebStatsBreakdown.DEVICE_TYPE,+ limit=100,+ webQueryModifiers=OPT_IN,+ ),+ modifiers=modifiers,+ )+ assert ready_dimensional_job_ids(runner, DIMENSIONAL_READ_TABLES) is None, (+ "partial coverage must not route to dimensional"+ )++ runner.calculate()+ assert runner.used_preaggregated_tables is expect_v2+ if expect_v2:+ assert runner.query_strategy() == "stats_table_preaggregated"++ @parameterized.expand(+ [+ (+ "custom_channel_breakdown",+ DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ WebStatsBreakdown.INITIAL_CHANNEL_TYPE,+ [],+ UNTAGGED_CHANNEL,+ None,+ {"Untagged": (3.0, 4.0)},+ ),+ (+ "custom_channel_filter",+ DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ WebStatsBreakdown.DEVICE_TYPE,+ [SessionPropertyFilter(key="$channel_type", value="Untagged", operator=PropertyOperator.EXACT)],+ UNTAGGED_CHANNEL,+ None,+ {"Desktop": (2.0, 3.0), "Mobile": (1.0, 1.0)},+ ),+ (+ "range_inside_hour_buckets",+ DateRange(date_from="2024-01-01T10:01:00Z", date_to="2024-01-01T11:30:00Z", explicitDate=True),+ WebStatsBreakdown.DEVICE_TYPE,+ [],+ None,+ None,+ {"Desktop": (1.0, 1.0), "Mobile": (1.0, 1.0)},+ ),+ (+ "bounce_duration_override",+ DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ WebStatsBreakdown.DEVICE_TYPE,+ [],+ HogQLQueryModifiers(bounceRateDurationSeconds=60),+ None,+ {"Desktop": (2.0, 3.0), "Mobile": (1.0, 1.0)},+ ),+ (+ "browser_version_filter",+ DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ WebStatsBreakdown.DEVICE_TYPE,+ [EventPropertyFilter(key="$browser_version", value="120", operator=PropertyOperator.EXACT)],+ None,+ None,+ {"Desktop": (1.0, 2.0)},+ ),+ (+ "range_before_retention_floor",+ DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ WebStatsBreakdown.DEVICE_TYPE,+ [],+ None,+ 1,+ {},+ ),+ ]+ )+ def test_ineligible_query_matches_live(+ self, _name, date_range, breakdown, properties, modifiers, retention_months, expected_counts+ ):+ def run(web_query_modifiers):+ runner = WebStatsTableQueryRunner(+ team=self.team,+ query=WebStatsTableQuery(+ dateRange=date_range,+ properties=properties,+ breakdownBy=breakdown,+ limit=100,+ webQueryModifiers=web_query_modifiers,+ ),+ modifiers=modifiers,+ )+ response = runner.calculate()+ return runner, self._sort_results(response.results)++ if retention_months is not None:+ self.team.event_retention_months = retention_months+ self.team.save()+ with override_settings(EVENTS_DATA_RETENTION_ENFORCED=retention_months is not None):+ opted_in_runner, opted_in = run(OPT_IN)+ _live_runner, live = run(None)+ assert ready_dimensional_job_ids(opted_in_runner, DIMENSIONAL_READ_TABLES) is None++ assert not opted_in_runner.used_preaggregated_tables+ assert opted_in == live+ assert self._counts(opted_in) == expected_counts++ @parameterized.expand(+ [+ ("stats_table", "DimensionalStatsTableQueryBuilder"),+ ("overview", "DimensionalWebOverviewQueryBuilder"),+ ]+ )+ def test_failed_dimensional_read_falls_back_to_live(self, _name, builder_name):+ self._assert_dimensional_failure(builder_name, RuntimeError("dimensional read failed"), falls_back=True)++ @parameterized.expand(+ [+ ("stats_table_at_capacity", "DimensionalStatsTableQueryBuilder", ClickHouseAtCapacity(), LEGACY),+ ("overview_at_capacity", "DimensionalWebOverviewQueryBuilder", ClickHouseAtCapacity(), LEGACY),+ (+ "overview_cancelled",+ "DimensionalWebOverviewQueryBuilder",+ CHQueryErrorQueryWasCancelled("cancelled", code=394, code_name="query_was_cancelled"),+ LEGACY,+ ),+ ("stats_table_timeout_without_v2", "DimensionalStatsTableQueryBuilder", ClickHouseQueryTimeOut(), None),+ ("overview_timeout_without_v2", "DimensionalWebOverviewQueryBuilder", ClickHouseQueryTimeOut(), None),+ ]+ )+ def test_overload_and_cancel_errors_do_not_rerun_live(self, _name, builder_name, error, modifiers):+ self._assert_dimensional_failure(builder_name, error, falls_back=False, modifiers=modifiers)++ @parameterized.expand(+ [+ ("stats_table", "DimensionalStatsTableQueryBuilder"),+ ("overview", "DimensionalWebOverviewQueryBuilder"),+ ]+ )+ def test_timeout_falls_back_to_v2(self, _name, builder_name):+ self._assert_dimensional_failure(builder_name, ClickHouseQueryTimeOut(), falls_back=True, modifiers=LEGACY)++ def _assert_dimensional_failure(+ self,+ builder_name: str,+ error: Exception,+ falls_back: bool,+ modifiers: HogQLQueryModifiers | None = None,+ ) -> None:+ def run(web_query_modifiers):+ if builder_name == "DimensionalStatsTableQueryBuilder":+ stats_runner = WebStatsTableQueryRunner(+ team=self.team,+ query=WebStatsTableQuery(+ dateRange=DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ properties=[],+ breakdownBy=WebStatsBreakdown.DEVICE_TYPE,+ limit=100,+ webQueryModifiers=web_query_modifiers,+ ),+ modifiers=modifiers,+ )+ return self._sort_results(stats_runner.calculate().results)+ overview_runner = WebOverviewQueryRunner(+ team=self.team,+ query=WebOverviewQuery(+ dateRange=DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ properties=[],+ webQueryModifiers=web_query_modifiers,+ ),+ modifiers=modifiers,+ )+ return [(item.key, item.value) for item in overview_runner.calculate().results]++ live = run(None)+ with patch(+ f"products.web_analytics.backend.hogql_queries.web_dimensional_read.{builder_name}.get_query",+ side_effect=error,+ ) as failing_query:+ if falls_back:+ assert run(OPT_IN) == live+ else:+ with self.assertRaises(type(error)):+ run(OPT_IN)++ assert failing_query.called++ def test_overview_dimensional_matches_v2(self):+ def run(modifiers, web_query_modifiers):+ runner = WebOverviewQueryRunner(+ query=WebOverviewQuery(+ dateRange=DateRange(date_from=DATE_FROM, date_to=DATE_TO),+ properties=[],+ webQueryModifiers=web_query_modifiers,+ ),+ team=self.team,+ modifiers=modifiers,+ )+ with patch(+ "products.web_analytics.backend.hogql_queries.web_overview.execute_hogql_query",+ wraps=execute_hogql_query,+ ) as mock_execute:+ response = runner.calculate()+ return [call.kwargs["query_type"] for call in mock_execute.call_args_list], response++ v2_query_types, v2 = run(LEGACY, None)+ dimensional_query_types, dimensional = run(None, OPT_IN)++ assert v2_query_types == ["web_overview_preaggregated_query"]+ assert dimensional_query_types == ["web_overview_dimensional_query"]+ assert dimensional.preComputeStrategy == WebAnalyticsPreComputeStrategy.PRE_AGGREGATED+ assert dimensional.preComputedAt is not None+ assert dimensional.preComputeStale is None+ assert [(item.key, item.value) for item in dimensional.results] == [+ (item.key, item.value) for item in v2.results+ ]+ # visitors, views, sessions: user_a has 2 pageviews, user_b and user_c have 1 each.+ assert [item.value for item in dimensional.results[:3]] == [3, 4, 3]diff --git a/products/web_analytics/backend/hogql_queries/web_dimensional_precompute.py b/products/web_analytics/backend/hogql_queries/web_dimensional_precompute.pyindex 6b35231551a4..876c12df6026 100644--- a/products/web_analytics/backend/hogql_queries/web_dimensional_precompute.py+++ b/products/web_analytics/backend/hogql_queries/web_dimensional_precompute.py@@ -5,8 +5,8 @@ These are the precomputation-framework successors to the v2 pre-aggregationWhere v2 was hand-written ClickHouse SQL run by a staging-table + partition-swapDagster ETL, these are HogQL templates fed to `ensure_precomputed`, which managesjob tracking, TTLs and the `ReplacingMergeTree` INSERTs. The scheduled-`web_dimensional_precompute` Dagster job is the only caller (the write path is-not wired into any query runner yet).+`web_dimensional_precompute` Dagster job is the only writer. Query runners only+check for READY jobs (`dimensional_job_ids`) and never build a window inline.The framework auto-prepends `team_id`/`job_id` and appends `expires_at` to thetop-level SELECT, and substitutes `{time_window_min}` / `{time_window_max}` per@@ -22,14 +22,21 @@ Parity with v2 is intentional, with two deliberate deviations:NULL, so they fit the non-nullable Int64 columns deterministically."""-from datetime import datetime+import json+from collections.abc import Collection+from datetime import UTC, datetime+from uuid import UUID-from posthog.schema import HogQLQueryModifiers+from django.conf import settings++from posthog.schema import HogQLQueryModifiers, SessionTableVersionfrom posthog.hogql import astfrom posthog.hogql.modifiers import create_default_modifiers_for_teamfrom posthog.hogql.parser import parse_expr+from posthog.cloud_utils import is_cloud+from posthog.dataclasses import frozenfrom posthog.models.team import Teamfrom posthog.models.web_preaggregated.sql import is_eu_cluster@@ -42,12 +49,19 @@ from products.analytics_platform.backend.lazy_computation.lazy_computation_execu)from products.web_analytics.backend.hogql_queries.web_analytics_lazy_precompute import SESSION_FORWARD_PAD_MINUTES+# Distributed read-table names as registered in the HogQL database schema.+WEB_STATS_DIMENSIONAL_READ_TABLE = "web_stats_dimensional_preaggregated"+WEB_BOUNCES_DIMENSIONAL_READ_TABLE = "web_bounces_dimensional_preaggregated"+# TTL tuned for "keep ~90 days warm, don't recompute every time": today's window# refreshes hourly, the last two days daily, everything older is computed once and# held for 90 days. The scheduled job re-runs over the rolling window and the-# framework only recomputes windows whose jobs have expired.+# framework only recomputes windows whose jobs have expired. Today's TTL is shorter+# than the hourly schedule so that every run finds today's job expired and rebuilds+# it. A TTL equal to the schedule period lets a run find the job fresh by seconds,+# skip it, and leave it expired until the next run.DIMENSIONAL_TTL_SECONDS: dict[str, int] = {- "0d": 60 * 60,+ "0d": 45 * 60,"2d": 24 * 60 * 60,"default": 90 * 24 * 60 * 60,}@@ -336,17 +350,61 @@ DIMENSIONAL_INSERT_SETTINGS: dict[str, int | str] = {DIMENSIONAL_SESSION_BUFFER_MINUTES = 25 * 60-def _insert_modifiers(team: Team) -> HogQLQueryModifiers:+# Modifier fields that change what the INSERTs store. The executor leaves modifiers out of the job+# hash, so `dimensional_cache_key_context` adds these values to it: a team settings change gives new+# jobs, and the old ones stop matching. A read whose values differ from the team's cannot use the jobs.+DIMENSIONAL_SEMANTIC_MODIFIER_FIELDS: tuple[str, ...] = (+ "bounceRateDurationSeconds",+ "bounceRatePageViewMode",+ "materializationMode",+ "personsOnEventsMode",+ "sessionsV2JoinMode",+ "sessionTableVersion",+)+++# The sessions v3 table holds every session only from the day its backfill reached, per Cloud region.+# A region missing here has no complete v3 history yet, so a v3 team there gets no dimensional days.+SESSIONS_V3_COVERAGE_START_BY_REGION: dict[str, datetime] = {+ "US": datetime(2026, 10, 1, tzinfo=UTC),+}+_NO_SESSIONS_COVERAGE = datetime.max.replace(tzinfo=UTC)+++def sessions_coverage_floor(modifiers: HogQLQueryModifiers) -> datetime | None:+ # A window before the floor builds as if no sessions existed: empty for the stats template, and+ # without entry dimensions or bounces for the rest. The writer must not build it and reads must+ # not serve it. Off Cloud the v3 materialized view fills the table natively, so it has no floor.+ if modifiers.sessionTableVersion != SessionTableVersion.V3 or not is_cloud():+ return None+ region = (settings.CLOUD_DEPLOYMENT or "").upper()+ return SESSIONS_V3_COVERAGE_START_BY_REGION.get(region, _NO_SESSIONS_COVERAGE)+++def dimensional_insert_modifiers(team: Team) -> HogQLQueryModifiers:return create_default_modifiers_for_team(team, HogQLQueryModifiers(sessionTimestampBufferMinutes=DIMENSIONAL_SESSION_BUFFER_MINUTES))+def dimensional_cache_key_context(modifiers: HogQLQueryModifiers) -> dict[str, str]:+ dump = modifiers.model_dump(mode="json")+ return {+ "semantic_modifiers": json.dumps(+ {field: dump.get(field) for field in DIMENSIONAL_SEMANTIC_MODIFIER_FIELDS}, sort_keys=True+ )+ }++def ensure_web_stats_dimensional_precomputed(team: Team,time_range_start: datetime,time_range_end: datetime,+ run_inserts: bool = True,+ stale_while_revalidate_seconds: float | None = None,+ modifiers: HogQLQueryModifiers | None = None,) -> LazyComputationResult:+ modifiers = modifiers or dimensional_insert_modifiers(team)return ensure_precomputed(team=team,insert_query=STATS_INSERT_TEMPLATE,@@ -356,9 +414,13 @@ def ensure_web_stats_dimensional_precomputed(table=LazyComputationTable.WEB_STATS_DIMENSIONAL_PREAGGREGATED,placeholders=_base_placeholders(),query_type="web_stats_dimensional_insert",- read_after_write=False,+ # Readers select by job_id as soon as a job turns READY, so its rows must be on the replica quorum first.+ read_after_write=True,+ run_inserts=run_inserts,+ stale_while_revalidate_seconds=stale_while_revalidate_seconds,insert_settings=DIMENSIONAL_INSERT_SETTINGS,- modifiers=_insert_modifiers(team),+ modifiers=modifiers,+ cache_key_context=dimensional_cache_key_context(modifiers),)@@ -366,7 +428,11 @@ def ensure_web_bounces_dimensional_precomputed(team: Team,time_range_start: datetime,time_range_end: datetime,+ run_inserts: bool = True,+ stale_while_revalidate_seconds: float | None = None,+ modifiers: HogQLQueryModifiers | None = None,) -> LazyComputationResult:+ modifiers = modifiers or dimensional_insert_modifiers(team)return ensure_precomputed(team=team,insert_query=BOUNCES_INSERT_TEMPLATE,@@ -376,7 +442,69 @@ def ensure_web_bounces_dimensional_precomputed(table=LazyComputationTable.WEB_BOUNCES_DIMENSIONAL_PREAGGREGATED,placeholders=_base_placeholders(),query_type="web_bounces_dimensional_insert",- read_after_write=False,+ # Readers select by job_id as soon as a job turns READY, so its rows must be on the replica quorum first.+ read_after_write=True,+ run_inserts=run_inserts,+ stale_while_revalidate_seconds=stale_while_revalidate_seconds,insert_settings=DIMENSIONAL_INSERT_SETTINGS,- modifiers=_insert_modifiers(team),+ modifiers=modifiers,+ cache_key_context=dimensional_cache_key_context(modifiers),+ )+++# The hourly writer rebuilds today's window, and the read keeps serving the previous+# generation while the rebuild runs or after a run fails. The grace stays far below+# EXPIRY_BUFFER_SECONDS, so the ClickHouse rows of a graced job still exist.+DIMENSIONAL_READ_STALE_WHILE_REVALIDATE_SECONDS = 30 * 60+++@frozen+class DimensionalJobIds:+ # None for a table the caller did not ask for.+ stats: list[UUID] | None+ bounces: list[UUID] | None+ stale: bool = False+ # The oldest `computed_at` of the served jobs.+ computed_at: datetime | None = None+++def dimensional_job_ids(+ team: Team,+ time_range_start: datetime,+ time_range_end: datetime,+ tables: Collection[str],+ modifiers: HogQLQueryModifiers,+) -> DimensionalJobIds | None:+ """Return the READY job ids of `tables` that cover the range, or None when any of them+ has a gap. Check-only: a read never builds a window, the scheduled job does.+ The framework splits jobs on UTC days, so the bounds should already be in UTC."""+ results: dict[str, LazyComputationResult] = {}+ ensure_by_table = {+ WEB_STATS_DIMENSIONAL_READ_TABLE: ensure_web_stats_dimensional_precomputed,+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE: ensure_web_bounces_dimensional_precomputed,+ }+ for table, ensure in ensure_by_table.items():+ if table not in tables:+ continue+ result = ensure(+ team,+ time_range_start,+ time_range_end,+ run_inserts=False,+ stale_while_revalidate_seconds=DIMENSIONAL_READ_STALE_WHILE_REVALIDATE_SECONDS,+ modifiers=modifiers,+ )+ if not result.ready:+ return None+ results[table] = result++ stats = results.get(WEB_STATS_DIMENSIONAL_READ_TABLE)+ bounces = results.get(WEB_BOUNCES_DIMENSIONAL_READ_TABLE)+ return DimensionalJobIds(+ stats=stats.job_ids if stats else None,+ bounces=bounces.job_ids if bounces else None,+ stale=any(result.stale for result in results.values()),+ computed_at=min(+ (result.computed_at for result in results.values() if result.computed_at is not None), default=None+ ),)diff --git a/products/web_analytics/backend/hogql_queries/web_dimensional_read.py b/products/web_analytics/backend/hogql_queries/web_dimensional_read.pynew file mode 100644index 000000000000..eeec24151c12--- /dev/null+++ b/products/web_analytics/backend/hogql_queries/web_dimensional_read.py@@ -0,0 +1,253 @@+"""Serve web overview and stats table queries from the dimensional precompute tables.++The dimensional tables have the same columns as the legacy pre-aggregated tables, so the+legacy query builders run unchanged against them. Two differences need the subclasses here:+the table names, and a `job_id IN (...)` filter. A recent window can hold several job+generations at once, and without the filter `sumMerge`/`uniqMerge` would count each+generation.+"""++from collections.abc import Collection+from datetime import UTC, datetime, timedelta+from enum import Enum, auto+from typing import TYPE_CHECKING, Union++from django.utils import timezone++import structlog+from dateutil.relativedelta import relativedelta++from posthog.schema import WebStatsBreakdown++from posthog.hogql import ast++from posthog.errors import QueryErrorCategory, classify_query_error+from posthog.models.team.event_retention import events_retention_months_for_team+from posthog.ph_client import feature_enabled_or_false++from products.access_control.backend.facade.api import team_has_property_access_rules+from products.web_analytics.backend.hogql_queries.pre_aggregated.query_builder import (+ WebAnalyticsPreAggregatedQueryBuilder,+)+from products.web_analytics.backend.hogql_queries.stats_table_pre_aggregated import StatsTablePreAggregatedQueryBuilder+from products.web_analytics.backend.hogql_queries.web_dimensional_precompute import (+ DIMENSIONAL_SEMANTIC_MODIFIER_FIELDS,+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE,+ WEB_STATS_DIMENSIONAL_READ_TABLE,+ DimensionalJobIds,+ dimensional_insert_modifiers,+ dimensional_job_ids,+ sessions_coverage_floor,+)+from products.web_analytics.backend.hogql_queries.web_overview_pre_aggregated import (+ WebOverviewPreAggregatedQueryBuilder,+)++if TYPE_CHECKING:+ from products.web_analytics.backend.hogql_queries.stats_table import WebStatsTableQueryRunner+ from products.web_analytics.backend.hogql_queries.web_overview import WebOverviewQueryRunner++logger = structlog.get_logger(__name__)++# The same flag gates the frontend toggle, so a request that sets the modifier without the flag is ignored.+WEB_ANALYTICS_QUERY_MODIFIERS_FLAG = "web-analytics-query-modifiers"++# The tables have no column for these keys, or store a session-level value that differs from what the+# live filter matches. Gclid, fbclid and gad_source are stored as booleans, so a value filter cannot match them.+_INEXACT_PROPERTY_KEYS = frozenset(+ {+ "$browser_version",+ "$os_version",+ "$geoip_country_name",+ "$geoip_time_zone",+ "$entry_gclid",+ "$entry_fbclid",+ "$entry_gad_source",+ }+)+++def ready_dimensional_job_ids(+ runner: Union["WebStatsTableQueryRunner", "WebOverviewQueryRunner"],+ tables: Collection[str],+) -> DimensionalJobIds | None:+ """The job ids to read when the query opted in and `tables` cover its whole range,+ including the compare period. None means the caller uses its next path."""+ web_query_modifiers = runner.query.webQueryModifiers+ if not (web_query_modifiers and web_query_modifiers.useDimensionalTables):+ return None+ if not _query_modifiers_flag_enabled(runner):+ return None+ if _has_inexact_property_filter(runner):+ return None+ # The tables are built without a user, so they cannot honor per-user property restrictions.+ if team_has_property_access_rules(team_id=runner.team.pk):+ return None+ # The writer builds all traffic, so the rows cannot exclude the team's test accounts.+ if runner._test_account_filters:+ return None++ # The channel breakdown and filter use the built-in channel rules, because the stored dimensions+ # cannot evaluate custom rules that match on the entry URL.+ if runner.modifiers.customChannelTypeRules and _reads_channel_type(runner):+ return None++ # A per-request override, such as a longer bounce duration, would count the stored rows differently.+ build_modifiers = dimensional_insert_modifiers(runner.team)+ if any(+ getattr(runner.modifiers, field) != getattr(build_modifiers, field)+ for field in DIMENSIONAL_SEMANTIC_MODIFIER_FIELDS+ ):+ return None++ periods = [runner.query_date_range]+ if runner.query_compare_to_date_range:+ periods.append(runner.query_compare_to_date_range)+ if not all(_fits_whole_hour_buckets(period.date_from(), period.date_to()) for period in periods):+ return None++ start = min(period.date_from() for period in periods)+ if _crosses_events_retention_floor(runner, start):+ return None+ sessions_floor = sessions_coverage_floor(build_modifiers)+ if sessions_floor is not None and start < sessions_floor:+ return None++ # A range can end after now, for example at the end of the current day in the project+ # timezone. No rows exist past now, and the writer does not build the next UTC day yet.+ end = min(runner.query_date_range.date_to(), timezone.now())+ try:+ return dimensional_job_ids(+ runner.team, start.astimezone(UTC), end.astimezone(UTC), tables=tables, modifiers=build_modifiers+ )+ except Exception:+ logger.exception("web_dimensional_job_ids_lookup_failed", team_id=runner.team.pk)+ return None+++class DimensionalReadFailure(Enum):+ # Cancelled by the user or the server, or refused for cluster capacity: no other read may run.+ RERAISE = auto()+ # Too slow or too large: the cheaper lazy and v2 reads may run, but not the live query, which+ # reads more data than the dimensional query that failed.+ SKIP_LIVE = auto()+ FALL_BACK = auto()+++def classify_dimensional_read_failure(error: Exception) -> DimensionalReadFailure:+ category = classify_query_error(error)+ if category in (QueryErrorCategory.CANCELLED, QueryErrorCategory.RATE_LIMITED):+ return DimensionalReadFailure.RERAISE+ if category == QueryErrorCategory.QUERY_PERFORMANCE_ERROR:+ return DimensionalReadFailure.SKIP_LIVE+ return DimensionalReadFailure.FALL_BACK+++def dimensional_precompute_stale(job_ids: DimensionalJobIds | None) -> bool | None:+ return True if job_ids and job_ids.stale else None+++def dimensional_precomputed_at(job_ids: DimensionalJobIds | None) -> str | None:+ if job_ids is None or job_ids.computed_at is None:+ return None+ return job_ids.computed_at.astimezone(UTC).isoformat()+++def _query_modifiers_flag_enabled(runner: Union["WebStatsTableQueryRunner", "WebOverviewQueryRunner"]) -> bool:+ team = runner.team+ user = runner.user+ return feature_enabled_or_false(+ WEB_ANALYTICS_QUERY_MODIFIERS_FLAG,+ str(user.distinct_id) if user and user.distinct_id else str(team.uuid),+ groups={"organization": str(team.organization_id), "project": str(team.id)},+ person_properties={"email": user.email} if user else None,+ group_properties={+ "organization": {"id": str(team.organization_id)},+ "project": {"id": str(team.id)},+ },+ only_evaluate_locally=True,+ send_feature_flag_events=False,+ )+++def _has_inexact_property_filter(runner: Union["WebStatsTableQueryRunner", "WebOverviewQueryRunner"]) -> bool:+ for prop in runner.query.properties:+ key = getattr(prop, "key", None)+ if key in _INEXACT_PROPERTY_KEYS:+ return True+ # The tables hold the session's initial referring domain, not the referrer of each event.+ if key == "$referring_domain" and getattr(prop, "type", None) == "event":+ return True+ # The overview maps a page filter to the entry page, which counts sessions that started on the+ # page instead of the visitors and views of it.+ if key == "$pathname" and runner.query.kind == "WebOverviewQuery":+ return True+ return False+++def _reads_channel_type(runner: Union["WebStatsTableQueryRunner", "WebOverviewQueryRunner"]) -> bool:+ if getattr(runner.query, "breakdownBy", None) == WebStatsBreakdown.INITIAL_CHANNEL_TYPE:+ return True+ return any(getattr(prop, "key", None) == "$channel_type" for prop in runner.query.properties)+++def _fits_whole_hour_buckets(date_from: datetime, date_to: datetime) -> bool:+ """Whether `period_bucket >= date_from AND period_bucket <= date_to` selects exactly the+ events in the range. Each bucket holds one UTC hour, so the start must be an hour boundary+ and the inclusive end must be the last microsecond of an hour. An end after now is also+ exact, because the bucket of the current hour holds only events up to now."""+ if not _is_utc_hour_boundary(date_from):+ return False+ return _is_utc_hour_boundary(date_to + timedelta(microseconds=1)) or date_to >= timezone.now()+++def _is_utc_hour_boundary(moment: datetime) -> bool:+ return moment.astimezone(UTC).replace(minute=0, second=0, microsecond=0) == moment+++def _crosses_events_retention_floor(+ runner: Union["WebStatsTableQueryRunner", "WebOverviewQueryRunner"], start: datetime+) -> bool:+ # READY jobs outlive the retention window. The live path floors events at+ # `timestamp > now() - retention`, so a range that starts at or before that floor+ # would show older traffic here than live does.+ retention_months = events_retention_months_for_team(runner.team, runner.team.pk)+ if retention_months is None:+ return False+ return start <= timezone.now() - relativedelta(months=retention_months)+++class _DimensionalTablesMixin(WebAnalyticsPreAggregatedQueryBuilder):+ job_ids: DimensionalJobIds++ @property+ def stats_table(self) -> str:+ return WEB_STATS_DIMENSIONAL_READ_TABLE++ @property+ def bounces_table(self) -> str:+ return WEB_BOUNCES_DIMENSIONAL_READ_TABLE++ def _get_filters(self, table_name: str, exclude_pathname: bool = False) -> ast.And:+ filters = super()._get_filters(table_name, exclude_pathname=exclude_pathname)+ job_ids = self.job_ids.stats if table_name == WEB_STATS_DIMENSIONAL_READ_TABLE else self.job_ids.bounces+ if job_ids is None:+ raise ValueError(f"No dimensional job ids were looked up for {table_name}")+ job_id_filter = ast.CompareOperation(+ op=ast.CompareOperationOp.In,+ left=ast.Field(chain=[table_name, "job_id"]),+ right=ast.Constant(value=[str(job_id) for job_id in job_ids]),+ )+ return ast.And(exprs=[*filters.exprs, job_id_filter])+++class DimensionalStatsTableQueryBuilder(_DimensionalTablesMixin, StatsTablePreAggregatedQueryBuilder):+ def __init__(self, runner: "WebStatsTableQueryRunner", job_ids: DimensionalJobIds) -> None:+ super().__init__(runner)+ self.job_ids = job_ids+++class DimensionalWebOverviewQueryBuilder(_DimensionalTablesMixin, WebOverviewPreAggregatedQueryBuilder):+ def __init__(self, runner: "WebOverviewQueryRunner", job_ids: DimensionalJobIds) -> None:+ super().__init__(runner)+ self.job_ids = job_idsdiff --git a/products/web_analytics/backend/hogql_queries/web_overview.py b/products/web_analytics/backend/hogql_queries/web_overview.pyindex 8c705a89f743..b1cc63b427fb 100644--- a/products/web_analytics/backend/hogql_queries/web_overview.py+++ b/products/web_analytics/backend/hogql_queries/web_overview.py@@ -8,6 +8,7 @@ from posthog.schema import (CachedWebOverviewQueryResponse,EventPropertyFilter,HogQLQueryModifiers,+ HogQLQueryResponse,PersonPropertyFilter,SessionTableVersion,WebAnalyticsPreComputeStrategy,@@ -24,6 +25,18 @@ from products.web_analytics.backend.hogql_queries.web_analytics_query_runner impWEB_ANALYTICS_NO_JOIN_SERVED,WebAnalyticsQueryRunner,)+from products.web_analytics.backend.hogql_queries.web_dimensional_precompute import (+ WEB_BOUNCES_DIMENSIONAL_READ_TABLE,+ DimensionalJobIds,+)+from products.web_analytics.backend.hogql_queries.web_dimensional_read import (+ DimensionalReadFailure,+ DimensionalWebOverviewQueryBuilder,+ classify_dimensional_read_failure,+ dimensional_precompute_stale,+ dimensional_precomputed_at,+ ready_dimensional_job_ids,+)from products.web_analytics.backend.hogql_queries.web_lazy_precompute_common import lazy_precompute_ineligible_reasonfrom products.web_analytics.backend.hogql_queries.web_overview_lazy_precompute import (can_use_lazy_precompute,@@ -254,6 +267,20 @@ CROSS JOIN {sessions_agg} AS sessions_aggassert isinstance(combined, ast.SelectQuery)return combined+ def get_dimensional_response(self) -> tuple[Optional[HogQLQueryResponse], Optional[DimensionalJobIds]]:+ if self.query.conversionGoal or not self.preaggregated_query_builder.can_use_preaggregated_tables():+ return None, None+ # The overview builder reads only the bounces table.+ job_ids = ready_dimensional_job_ids(self, tables=(WEB_BOUNCES_DIMENSIONAL_READ_TABLE,))+ if job_ids is None:+ return None, None+ response = self._execute_pre_aggregated(+ DimensionalWebOverviewQueryBuilder(self, job_ids),+ query_type="web_overview_dimensional_query",+ raise_errors=True,+ )+ return response, job_ids if response else None+def get_pre_aggregated_response(self):should_use_preaggregated = (self.modifiers@@ -265,6 +292,13 @@ CROSS JOIN {sessions_agg} AS sessions_aggif not should_use_preaggregated:return None+ return self._execute_pre_aggregated(+ self.preaggregated_query_builder, query_type="web_overview_preaggregated_query"+ )++ def _execute_pre_aggregated(+ self, builder: WebOverviewPreAggregatedQueryBuilder, query_type: str, raise_errors: bool = False+ ):try:# Pre-aggregated tables store data in UTC **buckets**, so we need to disable timezone conversion# to prevent HogQL from automatically converting DateTime fields to team timezone.@@ -274,8 +308,8 @@ CROSS JOIN {sessions_agg} AS sessions_aggpre_agg_modifiers.convertToProjectTimezone = Falseresponse = execute_hogql_query(- query_type="web_overview_preaggregated_query",- query=self.preaggregated_query_builder.get_query(),+ query_type=query_type,+ query=builder.get_query(),team=self.team,user=self.user,bypass_warehouse_access_control=self.bypass_warehouse_access_control,@@ -292,7 +326,9 @@ CROSS JOIN {sessions_agg} AS sessions_aggreturn responseexcept Exception as e:- logger.exception("Error getting pre-aggregated web_overview", error=e)+ if raise_errors:+ raise+ logger.exception("Error getting pre-aggregated web_overview", query_type=query_type, error=e)return Nonedef get_lazy_precomputed_row(self) -> Optional[list]:@@ -445,11 +481,25 @@ WHERE and(if self.should_split_conversion_goal:return self._calculate_split_conversion_goal()- lazy_row = self.get_lazy_precomputed_row()- if lazy_row is not None:- return self._build_response_from_row(lazy_row)-- pre_aggregated_response = self.get_pre_aggregated_response()+ live_blocking_error: Exception | None = None+ try:+ pre_aggregated_response, dimensional_job_ids = self.get_dimensional_response()+ except Exception as error:+ failure = classify_dimensional_read_failure(error)+ if failure is DimensionalReadFailure.RERAISE:+ raise+ logger.exception("web_overview_dimensional_read_failed", team_id=self.team.pk)+ pre_aggregated_response, dimensional_job_ids = None, None+ if failure is DimensionalReadFailure.SKIP_LIVE:+ live_blocking_error = error+ if not pre_aggregated_response:+ lazy_row = self.get_lazy_precomputed_row()+ if lazy_row is not None:+ return self._build_response_from_row(lazy_row)++ pre_aggregated_response = self.get_pre_aggregated_response()+ if live_blocking_error is not None and not pre_aggregated_response:+ raise live_blocking_error# Preflight only when the main query will actually run — if the# pre-aggregated builder already served the response, spending a@@ -532,6 +582,8 @@ WHERE and(dateTo=self.query_date_range.date_to_str,preComputeStrategy=strategy,preComputeIneligibleReason=lazy_precompute_ineligible_reason(strategy),+ preComputeStale=dimensional_precompute_stale(dimensional_job_ids),+ preComputedAt=dimensional_precomputed_at(dimensional_job_ids),)@cached_propertydiff --git a/products/web_analytics/dags/tests/test_web_dimensional_precompute.py b/products/web_analytics/dags/tests/test_web_dimensional_precompute.pyindex 95020dcad7aa..6d71087061b1 100644--- a/products/web_analytics/dags/tests/test_web_dimensional_precompute.py+++ b/products/web_analytics/dags/tests/test_web_dimensional_precompute.py@@ -6,13 +6,15 @@ from zoneinfo import ZoneInfoimport pytestimport time_machinefrom posthog.test.base import APIBaseTest-from unittest.mock import patch+from unittest.mock import MagicMock, patch-from django.test import SimpleTestCase+from django.test import SimpleTestCase, override_settingsimport dagsterfrom parameterized import parameterized+from posthog.schema import HogQLQueryModifiers, SessionTableVersion+from posthog.models import Organization, Teamfrom products.analytics_platform.backend.lazy_computation.lazy_computation_executor import LazyComputationResult@@ -32,6 +34,7 @@ from products.web_analytics.dags.web_dimensional_precompute import (get_selected_team_ids,precompute_window,summarize_coverage,+ team_window,web_dimensional_precompute_job,)@@ -107,6 +110,38 @@ class TestPrecomputeWindow:assert window.end == window.start + timedelta(days=PRECOMPUTE_WINDOW_DAYS)+class TestTeamWindow(SimpleTestCase):+ _WINDOW = TimeSpan(start=datetime(2026, 9, 1, tzinfo=UTC), end=datetime(2026, 11, 1, tzinfo=UTC))+ _FLOOR = datetime(2026, 10, 1, tzinfo=UTC)++ @parameterized.expand(+ [+ ("sessions_v2_keeps_the_window", SessionTableVersion.V2, "US", _WINDOW.start),+ ("sessions_v3_starts_at_the_floor", SessionTableVersion.V3, "US", _FLOOR),+ ("sessions_v3_without_a_backfill_is_empty", SessionTableVersion.V3, "EU", _WINDOW.end),+ ]+ )+ def test_window_starts_at_the_sessions_coverage_floor(self, _name, session_table_version, region, expected_start):+ with (+ patch(+ "products.web_analytics.dags.web_dimensional_precompute.dimensional_insert_modifiers",+ return_value=HogQLQueryModifiers(sessionTableVersion=session_table_version),+ ),+ patch(+ "products.web_analytics.backend.hogql_queries.web_dimensional_precompute.is_cloud", return_value=True+ ),+ patch.dict(+ "products.web_analytics.backend.hogql_queries.web_dimensional_precompute.SESSIONS_V3_COVERAGE_START_BY_REGION",+ {"US": self._FLOOR},+ clear=True,+ ),+ override_settings(CLOUD_DEPLOYMENT=region),+ ):+ window = team_window(MagicMock(), self._WINDOW)++ assert window == TimeSpan(start=expected_start, end=self._WINDOW.end)++class TestSummarizeCoverage(SimpleTestCase):@parameterized.expand([@@ -361,6 +396,21 @@ class TestCheckWebDimensionalPrecomputeCoverageOp(APIBaseTest):with time_machine.travel(_NOW, tick=False), pytest.raises(dagster.Failure, match=BOUNCES_TABLE):check_web_dimensional_precompute_coverage_op(dagster.build_op_context(), run)+ def test_team_without_sessions_history_in_the_window_is_not_checked(self):+ # The writer builds nothing for a sessions v3 team in a region the v3 backfill has not reached, so the+ # check must not report its empty window as missing days on every run.+ self.team.modifiers = {"sessionTableVersion": "v3"}+ self.team.save()++ with (+ time_machine.travel(_NOW, tick=False),+ patch(+ "products.web_analytics.backend.hogql_queries.web_dimensional_precompute.is_cloud", return_value=True+ ),+ override_settings(CLOUD_DEPLOYMENT="EU"),+ ):+ check_web_dimensional_precompute_coverage_op(dagster.build_op_context(), self._run({}))+@patch(_SINGLE_CHUNK, PRECOMPUTE_WINDOW_DAYS)def test_run_crossing_utc_midnight_checks_the_writers_day(self):# The writers finish before midnight and the check runs after it. The new UTC day has no jobdiff --git a/products/web_analytics/dags/web_dimensional_precompute.py b/products/web_analytics/dags/web_dimensional_precompute.pyindex 6700dd893aa8..b63d79c169f6 100644--- a/products/web_analytics/dags/web_dimensional_precompute.py+++ b/products/web_analytics/dags/web_dimensional_precompute.py@@ -43,8 +43,10 @@ from posthog.models import Teamfrom products.analytics_platform.backend.models.preaggregation_job import PreaggregationJobfrom products.web_analytics.backend.hogql_queries.web_dimensional_precompute import (DIMENSIONAL_TTL_SECONDS,+ dimensional_insert_modifiers,ensure_web_bounces_dimensional_precomputed,ensure_web_stats_dimensional_precomputed,+ sessions_coverage_floor,)from products.web_analytics.dags.web_preaggregated_utils import check_for_concurrent_runs@@ -311,6 +313,13 @@ def _push_coverage_gauges(coverages: list[TableCoverage]) -> None:)+def team_window(team: Team, window: TimeSpan) -> TimeSpan:+ floor = sessions_coverage_floor(dimensional_insert_modifiers(team))+ if floor is None or floor <= window.start:+ return window+ return TimeSpan(start=min(floor, window.end), end=window.end)++def _ensure_for_team(context: dagster.OpExecutionContext,team: Team,@@ -387,7 +396,7 @@ def ensure_web_dimensional_precompute_op(context: dagster.OpExecutionContext) ->if team is None:context.log.warning(f"web_dimensional_precompute_team_missing team_id={team_id}")continue- failures += _ensure_for_team(context, team, window, PRECOMPUTE_CHUNK_DAYS, sample_job_ids)+ failures += _ensure_for_team(context, team, team_window(team, window), PRECOMPUTE_CHUNK_DAYS, sample_job_ids)processed.append(team_id)context.log.info(f"web_dimensional_precompute_complete teams={len(processed)} failures={failures}")@@ -415,8 +424,16 @@ def check_web_dimensional_precompute_coverage_op().values_list("id", "query_hash")}+ teams_by_id = {team.pk: team for team in Team.objects.filter(pk__in=run.team_ids)}coverages: list[TableCoverage] = []for team_id in run.team_ids:+ team = teams_by_id.get(team_id)+ if team is None:+ continue+ window = team_window(team, run.window)+ # A team whose sessions history starts after the window has no days to build or check.+ if window.start >= window.end:+ continuefor table in (STATS_TABLE, BOUNCES_TABLE):job_id = run.sample_job_ids.get((team_id, table))coverages.append(@@ -424,7 +441,7 @@ def check_web_dimensional_precompute_coverage_op(team_id=team_id,table=table,query_hash=query_hash_by_job_id.get(job_id) if job_id is not None else None,- window=run.window,+ window=window,current_day=run.current_day,now=now,)diff --git a/services/mcp/src/api/generated.ts b/services/mcp/src/api/generated.tsindex 7db7345b8d07..de4daad9a198 100644--- a/services/mcp/src/api/generated.ts+++ b/services/mcp/src/api/generated.ts@@ -7761,6 +7761,11 @@ export namespace Schemas {forceSamplingRate?: SamplingRate | null;}+ export interface WebAnalyticsQueryModifiers {+ /** Serve web overview and stats table queries from the dimensional precompute tables when they hold ready data for the whole date range. Falls back to the normal path otherwise. */+ useDimensionalTables?: boolean | null;+ }+export interface WebStatsTableQuery {/** Groups aggregation - not used in Web Analytics but required for type compatibility */aggregation_group_type_index?: number | null;@@ -7797,6 +7802,8 @@ export namespace Schemas {useWebAnalyticsPrecompute?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type WebAnalyticsItemKind = typeof WebAnalyticsItemKind[keyof typeof WebAnalyticsItemKind];@@ -7878,6 +7885,8 @@ export namespace Schemas {useWebAnalyticsPrecompute?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface InsightActorsQuery {@@ -9937,6 +9946,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type WebBotsBreakdown = typeof WebBotsBreakdown[keyof typeof WebBotsBreakdown];@@ -10002,6 +10013,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface WebGoalsQueryResponse {@@ -10068,6 +10081,8 @@ export namespace Schemas {useWebAnalyticsPrecompute?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface WebVitalsQuery {@@ -10097,6 +10112,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type WebVitalsMetric = typeof WebVitalsMetric[keyof typeof WebVitalsMetric];@@ -10186,6 +10203,8 @@ export namespace Schemas {useWebAnalyticsPrecompute?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface Filters {@@ -10525,6 +10544,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type MarketingAnalyticsAggregatedQueryResponseResults = {[key: string]: MarketingAnalyticsItem};@@ -10591,6 +10612,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type ErrorTrackingOrderBy = typeof ErrorTrackingOrderBy[keyof typeof ErrorTrackingOrderBy];@@ -54606,6 +54629,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface MarketingAnalyticsAttributionPathRow {@@ -54697,6 +54722,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type MarketingAnalyticsRetentionInterval = typeof MarketingAnalyticsRetentionInterval[keyof typeof MarketingAnalyticsRetentionInterval];@@ -54815,6 +54842,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface MarketingAnalyticsSearchConversionGoal {@@ -55169,6 +55198,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type WebPageDirectoryMetric = typeof WebPageDirectoryMetric[keyof typeof WebPageDirectoryMetric];@@ -55298,6 +55329,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export interface PageURL {@@ -55358,6 +55391,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type WebAnalyticsExternalSummaryQueryResponseData = { [key: string]: unknown };@@ -55438,6 +55473,8 @@ export namespace Schemas {useSessionsTable?: boolean | null;/** version of the node, used for schema migrations */version?: number | null;+ /** Per-request switches for how web analytics serves this query. */+ webQueryModifiers?: WebAnalyticsQueryModifiers | null;}export type LogsOrderBy = typeof LogsOrderBy[keyof typeof LogsOrderBy];