:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
feat(sessions): ship the sessions v3 schema with more ad click ids (#104566)
Lucas Ricoy367aabc54d1a
diff --git a/frontend/src/scenes/web-analytics/contributing.md b/frontend/src/scenes/web-analytics/contributing.mdindex c027da7a4ebd..ae212399e34e 100644--- a/frontend/src/scenes/web-analytics/contributing.md+++ b/frontend/src/scenes/web-analytics/contributing.md@@ -58,6 +58,30 @@ The sessions table stores pre-computed session properties essential for web anal- Feature flag values seen during the session- Session replay presence+### How entry and exit values are chosen++Entry and exit values come from page views, not from the first or last event of any kind.++- The entry URL, entry referring domain, UTM parameters, click IDs, ad IDs and channel type inputs come from the session's first `$pageview` or `$screen`.+- The exit URL comes from the session's last `$pageview` or `$screen`.+- Other events (autocapture, `$pageleave`, custom events) set these values only when the session has no `$pageview` or `$screen`.+ The materialized view does this by moving the timestamp of every other event one year later (`pageview_prio_timestamp_min`) before it takes `argMin`.+- Device and geography values (browser, OS, viewport, country, city) come from the first event that has a value.+- An empty `gclid` or `fbclid` property does not count as a click ID, so it does not make `$entry_has_gclid` or the channel type say paid.++For the discussion behind the page view rule, see [this thread](https://posthog.slack.com/archives/C05LJK1N3CP/p1748876828748129).++One known inconsistency is accepted:++- `entry_utm_*` and `entry_referring_domain` are separate `argMin` columns over nullable values, and `argMin` skips `NULL`.+ When the entry page view has no UTM source, `$entry_utm_source` takes the value from a later event in the session.+- `entry_channel_type_properties`, which `$channel_type` reads, is one tuple that is never `NULL`, so it always holds the entry page view's values.+- So, for a session that lands without UTMs and gets one later, `$entry_utm_source` shows that later value while `$channel_type` can say Direct.++The materialized views also drop events whose timestamp is more than 31 days from the timestamp in their UUIDv7 session ID.+A client can send a session ID with a broken timestamp (for example 1970), which would put the session in its own monthly partition and stretch its duration across months.+The bound is symmetric because the client creates the session ID with its own clock while the server corrects event timestamps, so a client clock that is a few days ahead still gives a real session.+### Why session properties are central to web analyticsSession properties enable fast, attribution-focused analysis:diff --git a/posthog/api/test/test_session.py b/posthog/api/test/test_session.pyindex 68ea53776497..46d7078e7e91 100644--- a/posthog/api/test/test_session.py+++ b/posthog/api/test/test_session.py@@ -137,6 +137,11 @@ class TestSessionsAPI(APIBaseTest):"$entry_utm_source","$entry_utm_term",}+ if version == "v3":+ for ad_id in SESSION_V3_LOWER_TIER_AD_IDS:+ if "utm" in ad_id:+ expected_properties.add(f"$entry_{ad_id}")+ expected_properties.add(f"$entry_has_{ad_id}")assert actual_properties == expected_propertiesdef test_empty_search_session_properties(self):diff --git a/posthog/clickhouse/hcl/golden/dev/data.hcl b/posthog/clickhouse/hcl/golden/dev/data.hclindex 436c085a09a4..b5a186520df2 100644--- a/posthog/clickhouse/hcl/golden/dev/data.hcl+++ b/posthog/clickhouse/hcl/golden/dev/data.hcl@@ -4062,9 +4062,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -4072,13 +4069,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -4089,12 +4086,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -6599,9 +6590,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -6609,13 +6597,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -6626,12 +6614,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -6649,15 +6631,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -9019,9 +8996,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -9029,13 +9003,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -9046,12 +9020,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -10590,16 +10558,14 @@ SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/golden/dev/sessions.hcl b/posthog/clickhouse/hcl/golden/dev/sessions.hclindex 7e1fe6893464..160b935370be 100644--- a/posthog/clickhouse/hcl/golden/dev/sessions.hcl+++ b/posthog/clickhouse/hcl/golden/dev/sessions.hcl@@ -1010,9 +1010,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1020,13 +1017,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1037,12 +1034,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -1060,15 +1051,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}engine "replicated_aggregating_merge_tree" {zoo_path = "/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3"diff --git a/posthog/clickhouse/hcl/golden/local-multi/data.hcl b/posthog/clickhouse/hcl/golden/local-multi/data.hclindex 8dc5fb3408e6..6b49ca428ab1 100644--- a/posthog/clickhouse/hcl/golden/local-multi/data.hcl+++ b/posthog/clickhouse/hcl/golden/local-multi/data.hcl@@ -4421,9 +4421,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -4431,13 +4428,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -4448,9 +4445,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -6869,9 +6863,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -6879,13 +6870,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -6896,9 +6887,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -6916,15 +6904,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1- }- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -9488,9 +9471,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -9498,13 +9478,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -9515,9 +9495,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -11320,6 +11297,507 @@ SQL}}+ materialized_view "raw_sessions_v3_mv" {+ to_table = "posthog.writable_raw_sessions_v3"+ query = <<SQL+WITH+ JSONExtract(properties, 'Tuple(\n `$current_url` Nullable(String),\n `$external_click_url` Nullable(String),\n `$browser` Nullable(String),\n `$browser_version` Nullable(String),\n `$os` Nullable(String),\n `$os_version` Nullable(String),\n `$device_type` Nullable(String),\n `$viewport_width` Nullable(Int64),\n `$viewport_height` Nullable(Int64),\n `$geoip_country_code` Nullable(String),\n `$geoip_subdivision_1_code` Nullable(String),\n `$geoip_subdivision_1_name` Nullable(String),\n `$geoip_city_name` Nullable(String),\n `$geoip_time_zone` Nullable(String),\n `$referring_domain` Nullable(String),\n `utm_source` Nullable(String),\n `utm_campaign` Nullable(String),\n `utm_medium` Nullable(String),\n `utm_term` Nullable(String),\n `utm_content` Nullable(String),\n `gclid` Nullable(String),\n `gad_source` Nullable(String),\n `fbclid` Nullable(String),\n `$host` Nullable(String),\n `gclsrc` Nullable(String),\n `dclid` Nullable(String),\n `gbraid` Nullable(String),\n `wbraid` Nullable(String),\n `gad_campaignid` Nullable(String),\n `srsltid` Nullable(String),\n `msclkid` Nullable(String),\n `twclid` Nullable(String),\n `li_fat_id` Nullable(String),\n `mc_cid` Nullable(String),\n `igshid` Nullable(String),\n `ttclid` Nullable(String),\n `rdt_cid` Nullable(String),\n `epik` Nullable(String),\n `qclid` Nullable(String),\n `sccid` Nullable(String),\n `_kx` Nullable(String),\n `irclid` Nullable(String),\n `irclickid` Nullable(String),\n `utm_id` Nullable(String),\n `utm_source_platform` Nullable(String),\n `utm_creative_format` Nullable(String),\n `utm_marketing_tactic` Nullable(String),\n `yclid` Nullable(String),\n `tblci` Nullable(String),\n `dicbo` Nullable(String),\n `ndclid` Nullable(String),\n `cjevent` Nullable(String),\n `awc` Nullable(String),\n `syclid` Nullable(String),\n `_bhlid` Nullable(String),\n `rtid` Nullable(String),\n `oppref` Nullable(String),\n `bfcid` Nullable(String)\n )') AS p,+ JSONExtractString(person_properties, 'email') AS _person_email,+ tupleElement(p, '$current_url') AS _current_url,+ if((event = '$pageview') OR (event = '$screen'), left(coalesce(_current_url, ''), 16384), '') AS _attribution_url,+ tupleElement(p, '$external_click_url') AS _external_click_url,+ tupleElement(p, '$browser') AS _browser,+ tupleElement(p, '$browser_version') AS _browser_version,+ tupleElement(p, '$os') AS _os,+ tupleElement(p, '$os_version') AS _os_version,+ tupleElement(p, '$device_type') AS _device_type,+ tupleElement(p, '$viewport_width') AS _viewport_width,+ tupleElement(p, '$viewport_height') AS _viewport_height,+ tupleElement(p, '$geoip_country_code') AS _geoip_country_code,+ tupleElement(p, '$geoip_subdivision_1_code') AS _geoip_subdivision_1_code,+ tupleElement(p, '$geoip_subdivision_1_name') AS _geoip_subdivision_1_name,+ tupleElement(p, '$geoip_city_name') AS _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_time_zone') AS _geoip_time_zone,+ tupleElement(p, '$referring_domain') AS _referring_domain,+ tupleElement(p, 'utm_source') AS _utm_source,+ tupleElement(p, 'utm_campaign') AS _utm_campaign,+ tupleElement(p, 'utm_medium') AS _utm_medium,+ tupleElement(p, 'utm_term') AS _utm_term,+ tupleElement(p, 'utm_content') AS _utm_content,+ tupleElement(p, 'gclid') AS _gclid,+ tupleElement(p, 'gad_source') AS _gad_source,+ tupleElement(p, 'fbclid') AS _fbclid,+ coalesce(nullIf(tupleElement(p, 'gclsrc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gclsrc')), 2048)), '')) AS gclsrc,+ coalesce(nullIf(tupleElement(p, 'dclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dclid')), 2048)), '')) AS dclid,+ coalesce(nullIf(tupleElement(p, 'gbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gbraid')), 2048)), '')) AS gbraid,+ coalesce(nullIf(tupleElement(p, 'wbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'wbraid')), 2048)), '')) AS wbraid,+ coalesce(nullIf(tupleElement(p, 'gad_campaignid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gad_campaignid')), 2048)), '')) AS gad_campaignid,+ coalesce(nullIf(tupleElement(p, 'srsltid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'srsltid')), 2048)), '')) AS srsltid,+ coalesce(nullIf(tupleElement(p, 'msclkid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'msclkid')), 2048)), '')) AS msclkid,+ coalesce(nullIf(tupleElement(p, 'twclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'twclid')), 2048)), '')) AS twclid,+ coalesce(nullIf(tupleElement(p, 'li_fat_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'li_fat_id')), 2048)), '')) AS li_fat_id,+ coalesce(nullIf(tupleElement(p, 'mc_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'mc_cid')), 2048)), '')) AS mc_cid,+ coalesce(nullIf(tupleElement(p, 'igshid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'igshid')), 2048)), '')) AS igshid,+ coalesce(nullIf(tupleElement(p, 'ttclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ttclid')), 2048)), '')) AS ttclid,+ coalesce(nullIf(tupleElement(p, 'rdt_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rdt_cid')), 2048)), '')) AS rdt_cid,+ coalesce(nullIf(tupleElement(p, 'epik'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'epik')), 2048)), '')) AS epik,+ coalesce(nullIf(tupleElement(p, 'qclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'qclid')), 2048)), '')) AS qclid,+ coalesce(nullIf(tupleElement(p, 'sccid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'sccid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ScCid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'SsCid')), 2048)), '')) AS sccid,+ coalesce(nullIf(tupleElement(p, '_kx'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_kx')), 2048)), '')) AS _kx,+ coalesce(nullIf(tupleElement(p, 'irclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclid')), 2048)), '')) AS irclid,+ coalesce(nullIf(tupleElement(p, 'irclickid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclickid')), 2048)), '')) AS irclickid,+ coalesce(nullIf(tupleElement(p, 'utm_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_id')), 2048)), '')) AS utm_id,+ coalesce(nullIf(tupleElement(p, 'utm_source_platform'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_source_platform')), 2048)), '')) AS utm_source_platform,+ coalesce(nullIf(tupleElement(p, 'utm_creative_format'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_creative_format')), 2048)), '')) AS utm_creative_format,+ coalesce(nullIf(tupleElement(p, 'utm_marketing_tactic'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_marketing_tactic')), 2048)), '')) AS utm_marketing_tactic,+ coalesce(nullIf(tupleElement(p, 'yclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'yclid')), 2048)), '')) AS yclid,+ coalesce(nullIf(tupleElement(p, 'tblci'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'tblci')), 2048)), '')) AS tblci,+ coalesce(nullIf(tupleElement(p, 'dicbo'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dicbo')), 2048)), '')) AS dicbo,+ coalesce(nullIf(tupleElement(p, 'ndclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ndclid')), 2048)), '')) AS ndclid,+ coalesce(nullIf(tupleElement(p, 'cjevent'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'cjevent')), 2048)), '')) AS cjevent,+ coalesce(nullIf(tupleElement(p, 'awc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'awc')), 2048)), '')) AS awc,+ coalesce(nullIf(tupleElement(p, 'syclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'syclid')), 2048)), '')) AS syclid,+ coalesce(nullIf(tupleElement(p, '_bhlid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_bhlid')), 2048)), '')) AS _bhlid,+ coalesce(nullIf(tupleElement(p, 'rtid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rtid')), 2048)), '')) AS rtid,+ coalesce(nullIf(tupleElement(p, 'oppref'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'oppref')), 2048)), '')) AS oppref,+ coalesce(nullIf(tupleElement(p, 'bfcid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'bfcid')), 2048)), '')) AS bfcid,+ CAST(arrayFilter(x -> (x IS NOT NULL), [if(gclsrc IS NOT NULL, concat('gclsrc=', gclsrc), NULL), if(dclid IS NOT NULL, concat('dclid=', dclid), NULL), if(gbraid IS NOT NULL, concat('gbraid=', gbraid), NULL), if(wbraid IS NOT NULL, concat('wbraid=', wbraid), NULL), if(gad_campaignid IS NOT NULL, concat('gad_campaignid=', gad_campaignid), NULL), if(srsltid IS NOT NULL, concat('srsltid=', srsltid), NULL), if(msclkid IS NOT NULL, concat('msclkid=', msclkid), NULL), if(twclid IS NOT NULL, concat('twclid=', twclid), NULL), if(li_fat_id IS NOT NULL, concat('li_fat_id=', li_fat_id), NULL), if(mc_cid IS NOT NULL, concat('mc_cid=', mc_cid), NULL), if(igshid IS NOT NULL, concat('igshid=', igshid), NULL), if(ttclid IS NOT NULL, concat('ttclid=', ttclid), NULL), if(rdt_cid IS NOT NULL, concat('rdt_cid=', rdt_cid), NULL), if(epik IS NOT NULL, concat('epik=', epik), NULL), if(qclid IS NOT NULL, concat('qclid=', qclid), NULL), if(sccid IS NOT NULL, concat('sccid=', sccid), NULL), if(_kx IS NOT NULL, concat('_kx=', _kx), NULL), if(irclid IS NOT NULL, concat('irclid=', irclid), NULL), if(irclickid IS NOT NULL, concat('irclickid=', irclickid), NULL), if(utm_id IS NOT NULL, concat('utm_id=', utm_id), NULL), if(utm_source_platform IS NOT NULL, concat('utm_source_platform=', utm_source_platform), NULL), if(utm_creative_format IS NOT NULL, concat('utm_creative_format=', utm_creative_format), NULL), if(utm_marketing_tactic IS NOT NULL, concat('utm_marketing_tactic=', utm_marketing_tactic), NULL), if(yclid IS NOT NULL, concat('yclid=', yclid), NULL), if(tblci IS NOT NULL, concat('tblci=', tblci), NULL), if(dicbo IS NOT NULL, concat('dicbo=', dicbo), NULL), if(ndclid IS NOT NULL, concat('ndclid=', ndclid), NULL), if(cjevent IS NOT NULL, concat('cjevent=', cjevent), NULL), if(awc IS NOT NULL, concat('awc=', awc), NULL), if(syclid IS NOT NULL, concat('syclid=', syclid), NULL), if(_bhlid IS NOT NULL, concat('_bhlid=', _bhlid), NULL), if(rtid IS NOT NULL, concat('rtid=', rtid), NULL), if(oppref IS NOT NULL, concat('oppref=', oppref), NULL), if(bfcid IS NOT NULL, concat('bfcid=', bfcid), NULL)]), 'Array(String)') AS ad_ids_set,+ tupleElement(p, '$host') AS _host,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp + toIntervalYear(1)) AS pageview_prio_timestamp_min,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp - toIntervalYear(1)) AS pageview_prio_timestamp_max+SELECT+ team_id,+ `$session_id_uuid` AS session_id_v7,+ initializeAggregation('argMaxState', source_table.distinct_id, timestamp) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ inserted_at AS max_inserted_at,+ if(+ (_current_url IS NOT NULL) AND ((event = '$pageview') OR (event = '$screen')),+ [_current_url],+ []+ ) AS urls,+ initializeAggregation('argMinState', _current_url, pageview_prio_timestamp_min) AS entry_url,+ initializeAggregation('argMaxState', _current_url, pageview_prio_timestamp_max) AS end_url,+ initializeAggregation('argMaxState', _external_click_url, timestamp) AS last_external_click_url,+ initializeAggregation('argMinState', _browser, timestamp) AS browser,+ initializeAggregation('argMinState', _browser_version, timestamp) AS browser_version,+ initializeAggregation('argMinState', _os, timestamp) AS os,+ initializeAggregation('argMinState', _os_version, timestamp) AS os_version,+ initializeAggregation('argMinState', _device_type, timestamp) AS device_type,+ initializeAggregation('argMinState', _viewport_width, timestamp) AS viewport_width,+ initializeAggregation('argMinState', _viewport_height, timestamp) AS viewport_height,+ initializeAggregation('argMinState', _geoip_country_code, timestamp) AS geoip_country_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_code, timestamp) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_name, timestamp) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', _geoip_subdivision_city_name, timestamp) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', _geoip_time_zone, timestamp) AS geoip_time_zone,+ initializeAggregation('argMinState', _referring_domain, pageview_prio_timestamp_min) AS entry_referring_domain,+ initializeAggregation('argMinState', _utm_source, pageview_prio_timestamp_min) AS entry_utm_source,+ initializeAggregation('argMinState', _utm_campaign, pageview_prio_timestamp_min) AS entry_utm_campaign,+ initializeAggregation('argMinState', _utm_medium, pageview_prio_timestamp_min) AS entry_utm_medium,+ initializeAggregation('argMinState', _utm_term, pageview_prio_timestamp_min) AS entry_utm_term,+ initializeAggregation('argMinState', _utm_content, pageview_prio_timestamp_min) AS entry_utm_content,+ initializeAggregation('argMinState', _gclid, pageview_prio_timestamp_min) AS entry_gclid,+ initializeAggregation('argMinState', _gad_source, pageview_prio_timestamp_min) AS entry_gad_source,+ initializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) AS entry_fbclid,+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_fbclid,+ initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(+ _utm_source,+ _utm_medium,+ _utm_campaign,+ _referring_domain,+ nullIf(_gclid, '') IS NOT NULL,+ nullIf(_fbclid, '') IS NOT NULL,+ _gad_source+ ),+ pageview_prio_timestamp_min+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', if(event = '$pageview', uuid, NULL)) AS pageview_uniq,+ initializeAggregation('uniqState', if(event = '$autocapture', uuid, NULL)) AS autocapture_uniq,+ initializeAggregation('uniqState', if(event = '$screen', uuid, NULL)) AS screen_uniq,+ initializeAggregation(+ 'uniqUpToState(1)',+ if((event = '$pageview') OR (event = '$screen'), uuid, NULL)+ ) AS page_screen_uniq_up_to,+ event = '$autocapture' AS has_autocapture,+ arrayMap(+ (k, v) -> concat(k, '=', v),+ mapKeys(properties_group_feature_flags),+ mapValues(properties_group_feature_flags)+ ) AS flag_key_values,+ [event] AS event_names,+ if((_host IS NOT NULL) AND (_host != ''), [_host], []) AS hosts,+ if((_person_email IS NOT NULL) AND (_person_email != ''), [_person_email], []) AS emails,+ false AS has_replay_events+FROM posthog.sharded_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) >= (timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) <= (timestamp + toIntervalDay(31))))+AND+ true+SQL++ column "team_id" {+ type = "Int64"+ }+ column "session_id_v7" {+ type = "Nullable(UInt128)"+ }+ column "distinct_id" {+ type = "AggregateFunction(argMax, String, DateTime64(6, 'UTC'))"+ }+ column "distinct_ids" {+ type = "AggregateFunction(groupUniqArray, String)"+ }+ column "min_timestamp" {+ type = "DateTime64(6, 'UTC')"+ }+ column "max_timestamp" {+ type = "DateTime64(6, 'UTC')"+ }+ column "max_inserted_at" {+ type = "Nullable(DateTime64(6, 'UTC'))"+ }+ column "urls" {+ type = "Array(Nullable(String))"+ }+ column "entry_url" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "end_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "last_external_click_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "browser" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "browser_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "os" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "os_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "device_type" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "viewport_width" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC'))"+ }+ column "viewport_height" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC'))"+ }+ column "geoip_country_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_1_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_1_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_city_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_time_zone" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_referring_domain" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_campaign" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_medium" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_term" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_content" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_gclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_gad_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_fbclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_has_gclid" {+ type = "AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC'))"+ }+ column "entry_has_fbclid" {+ type = "AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC'))"+ }+ column "entry_ad_ids_set" {+ type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"+ }+ column "entry_channel_type_properties" {+ type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), UInt8, UInt8, Nullable(String)), DateTime64(6, 'UTC'))"+ }+ column "pageview_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "autocapture_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "screen_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "page_screen_uniq_up_to" {+ type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"+ }+ column "has_autocapture" {+ type = "UInt8"+ }+ column "flag_key_values" {+ type = "Array(String)"+ }+ column "event_names" {+ type = "Array(String)"+ }+ column "hosts" {+ type = "Array(Nullable(String))"+ }+ column "emails" {+ type = "Array(String)"+ }+ column "has_replay_events" {+ type = "Bool"+ }+ }++ materialized_view "raw_sessions_v3_recordings_mv" {+ to_table = "posthog.writable_raw_sessions_v3"+ query = <<SQL+WITH+ min_first_timestamp AS timestamp,+ CAST(fromUnixTimestamp64Milli(9223372036854775), 'DateTime64(6)') AS max_ts_64,+ CAST(fromUnixTimestamp64Milli(-9223372036854775), 'DateTime64(6)') AS min_ts_64,+ CAST(NULL, 'Nullable(String)') AS null_s,+ CAST(NULL, 'Nullable(Int64)') AS null_i64,+ CAST(NULL, 'Nullable(UUID)') AS null_uuid+SELECT+ team_id,+ toUInt128(accurateCast(session_id, 'UUID')) AS session_id_v7,+ fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))) AS session_timestamp,+ initializeAggregation('argMaxState', source_table.distinct_id, min_ts_64) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ fromUnixTimestamp(0) AS max_inserted_at,+ CAST([], 'Array(String)') AS urls,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS end_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS last_external_click_url,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS device_type,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_width,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_height,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_country_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_time_zone,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_referring_domain,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_campaign,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_medium,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_term,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_content,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gclid,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gad_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_fbclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_gclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_fbclid,+ initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(null_s, null_s, null_s, null_s, false, false, null_s),+ max_ts_64+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', null_uuid) AS pageview_uniq,+ initializeAggregation('uniqState', null_uuid) AS autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) AS screen_uniq,+ initializeAggregation('uniqUpToState(1)', null_uuid) AS page_screen_uniq_up_to,+ false AS has_autocapture,+ CAST([], 'Array(String)') AS flag_key_values,+ CAST([], 'Array(String)') AS event_names,+ CAST([], 'Array(String)') AS hosts,+ CAST([], 'Array(String)') AS emails,+ true AS has_replay_events+FROM posthog.sharded_session_replay_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) >= (min_first_timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) <= (min_first_timestamp + toIntervalDay(31))))+AND+ true+SQL++ column "team_id" {+ type = "Int64"+ }+ column "session_id_v7" {+ type = "UInt128"+ }+ column "session_timestamp" {+ type = "DateTime64(3)"+ }+ column "distinct_id" {+ type = "AggregateFunction(argMax, String, DateTime64(6))"+ }+ column "distinct_ids" {+ type = "AggregateFunction(groupUniqArray, String)"+ }+ column "min_timestamp" {+ type = "SimpleAggregateFunction(min, DateTime64(6, 'UTC'))"+ }+ column "max_timestamp" {+ type = "SimpleAggregateFunction(min, DateTime64(6, 'UTC'))"+ }+ column "max_inserted_at" {+ type = "DateTime"+ }+ column "urls" {+ type = "Array(String)"+ }+ column "entry_url" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "end_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6))"+ }+ column "last_external_click_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6))"+ }+ column "browser" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "browser_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "os" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "os_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "device_type" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "viewport_width" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6))"+ }+ column "viewport_height" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6))"+ }+ column "geoip_country_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_1_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_1_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_city_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_time_zone" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_referring_domain" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_campaign" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_medium" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_term" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_content" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_gclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_gad_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_fbclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_has_gclid" {+ type = "AggregateFunction(argMin, Bool, DateTime64(6))"+ }+ column "entry_has_fbclid" {+ type = "AggregateFunction(argMin, Bool, DateTime64(6))"+ }+ column "entry_ad_ids_set" {+ type = "AggregateFunction(argMin, Array(String), DateTime64(6))"+ }+ column "entry_channel_type_properties" {+ type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6))"+ }+ column "pageview_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "autocapture_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "screen_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "page_screen_uniq_up_to" {+ type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"+ }+ column "has_autocapture" {+ type = "Bool"+ }+ column "flag_key_values" {+ type = "Array(String)"+ }+ column "event_names" {+ type = "Array(String)"+ }+ column "hosts" {+ type = "Array(String)"+ }+ column "emails" {+ type = "Array(String)"+ }+ column "has_replay_events" {+ type = "Bool"+ }+ }+materialized_view "sessions_mv" {to_table = "posthog.writable_sessions"query = <<SQL@@ -11960,16 +12438,14 @@ SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/golden/local-single/all.hcl b/posthog/clickhouse/hcl/golden/local-single/all.hclindex a144006d9c37..9895dad26e39 100644--- a/posthog/clickhouse/hcl/golden/local-single/all.hcl+++ b/posthog/clickhouse/hcl/golden/local-single/all.hcl@@ -10052,9 +10052,6 @@ SQLcolumn "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -10062,13 +10059,13 @@ SQLtype = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -10079,9 +10076,6 @@ SQLcolumn "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -13703,9 +13697,6 @@ SQLcolumn "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -13713,13 +13704,13 @@ SQLtype = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -13730,9 +13721,6 @@ SQLcolumn "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -13750,15 +13738,10 @@ SQLtype = "bloom_filter()"granularity = 1}- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1- }- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -19222,9 +19205,6 @@ SQLcolumn "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -19232,13 +19212,13 @@ SQLtype = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -19249,9 +19229,6 @@ SQLcolumn "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -24577,6 +24554,507 @@ SQL}}+ materialized_view "raw_sessions_v3_mv" {+ to_table = "posthog.writable_raw_sessions_v3"+ query = <<SQL+WITH+ JSONExtract(properties, 'Tuple(\n `$current_url` Nullable(String),\n `$external_click_url` Nullable(String),\n `$browser` Nullable(String),\n `$browser_version` Nullable(String),\n `$os` Nullable(String),\n `$os_version` Nullable(String),\n `$device_type` Nullable(String),\n `$viewport_width` Nullable(Int64),\n `$viewport_height` Nullable(Int64),\n `$geoip_country_code` Nullable(String),\n `$geoip_subdivision_1_code` Nullable(String),\n `$geoip_subdivision_1_name` Nullable(String),\n `$geoip_city_name` Nullable(String),\n `$geoip_time_zone` Nullable(String),\n `$referring_domain` Nullable(String),\n `utm_source` Nullable(String),\n `utm_campaign` Nullable(String),\n `utm_medium` Nullable(String),\n `utm_term` Nullable(String),\n `utm_content` Nullable(String),\n `gclid` Nullable(String),\n `gad_source` Nullable(String),\n `fbclid` Nullable(String),\n `$host` Nullable(String),\n `gclsrc` Nullable(String),\n `dclid` Nullable(String),\n `gbraid` Nullable(String),\n `wbraid` Nullable(String),\n `gad_campaignid` Nullable(String),\n `srsltid` Nullable(String),\n `msclkid` Nullable(String),\n `twclid` Nullable(String),\n `li_fat_id` Nullable(String),\n `mc_cid` Nullable(String),\n `igshid` Nullable(String),\n `ttclid` Nullable(String),\n `rdt_cid` Nullable(String),\n `epik` Nullable(String),\n `qclid` Nullable(String),\n `sccid` Nullable(String),\n `_kx` Nullable(String),\n `irclid` Nullable(String),\n `irclickid` Nullable(String),\n `utm_id` Nullable(String),\n `utm_source_platform` Nullable(String),\n `utm_creative_format` Nullable(String),\n `utm_marketing_tactic` Nullable(String),\n `yclid` Nullable(String),\n `tblci` Nullable(String),\n `dicbo` Nullable(String),\n `ndclid` Nullable(String),\n `cjevent` Nullable(String),\n `awc` Nullable(String),\n `syclid` Nullable(String),\n `_bhlid` Nullable(String),\n `rtid` Nullable(String),\n `oppref` Nullable(String),\n `bfcid` Nullable(String)\n )') AS p,+ JSONExtractString(person_properties, 'email') AS _person_email,+ tupleElement(p, '$current_url') AS _current_url,+ if((event = '$pageview') OR (event = '$screen'), left(coalesce(_current_url, ''), 16384), '') AS _attribution_url,+ tupleElement(p, '$external_click_url') AS _external_click_url,+ tupleElement(p, '$browser') AS _browser,+ tupleElement(p, '$browser_version') AS _browser_version,+ tupleElement(p, '$os') AS _os,+ tupleElement(p, '$os_version') AS _os_version,+ tupleElement(p, '$device_type') AS _device_type,+ tupleElement(p, '$viewport_width') AS _viewport_width,+ tupleElement(p, '$viewport_height') AS _viewport_height,+ tupleElement(p, '$geoip_country_code') AS _geoip_country_code,+ tupleElement(p, '$geoip_subdivision_1_code') AS _geoip_subdivision_1_code,+ tupleElement(p, '$geoip_subdivision_1_name') AS _geoip_subdivision_1_name,+ tupleElement(p, '$geoip_city_name') AS _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_time_zone') AS _geoip_time_zone,+ tupleElement(p, '$referring_domain') AS _referring_domain,+ tupleElement(p, 'utm_source') AS _utm_source,+ tupleElement(p, 'utm_campaign') AS _utm_campaign,+ tupleElement(p, 'utm_medium') AS _utm_medium,+ tupleElement(p, 'utm_term') AS _utm_term,+ tupleElement(p, 'utm_content') AS _utm_content,+ tupleElement(p, 'gclid') AS _gclid,+ tupleElement(p, 'gad_source') AS _gad_source,+ tupleElement(p, 'fbclid') AS _fbclid,+ coalesce(nullIf(tupleElement(p, 'gclsrc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gclsrc')), 2048)), '')) AS gclsrc,+ coalesce(nullIf(tupleElement(p, 'dclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dclid')), 2048)), '')) AS dclid,+ coalesce(nullIf(tupleElement(p, 'gbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gbraid')), 2048)), '')) AS gbraid,+ coalesce(nullIf(tupleElement(p, 'wbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'wbraid')), 2048)), '')) AS wbraid,+ coalesce(nullIf(tupleElement(p, 'gad_campaignid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gad_campaignid')), 2048)), '')) AS gad_campaignid,+ coalesce(nullIf(tupleElement(p, 'srsltid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'srsltid')), 2048)), '')) AS srsltid,+ coalesce(nullIf(tupleElement(p, 'msclkid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'msclkid')), 2048)), '')) AS msclkid,+ coalesce(nullIf(tupleElement(p, 'twclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'twclid')), 2048)), '')) AS twclid,+ coalesce(nullIf(tupleElement(p, 'li_fat_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'li_fat_id')), 2048)), '')) AS li_fat_id,+ coalesce(nullIf(tupleElement(p, 'mc_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'mc_cid')), 2048)), '')) AS mc_cid,+ coalesce(nullIf(tupleElement(p, 'igshid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'igshid')), 2048)), '')) AS igshid,+ coalesce(nullIf(tupleElement(p, 'ttclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ttclid')), 2048)), '')) AS ttclid,+ coalesce(nullIf(tupleElement(p, 'rdt_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rdt_cid')), 2048)), '')) AS rdt_cid,+ coalesce(nullIf(tupleElement(p, 'epik'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'epik')), 2048)), '')) AS epik,+ coalesce(nullIf(tupleElement(p, 'qclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'qclid')), 2048)), '')) AS qclid,+ coalesce(nullIf(tupleElement(p, 'sccid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'sccid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ScCid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'SsCid')), 2048)), '')) AS sccid,+ coalesce(nullIf(tupleElement(p, '_kx'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_kx')), 2048)), '')) AS _kx,+ coalesce(nullIf(tupleElement(p, 'irclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclid')), 2048)), '')) AS irclid,+ coalesce(nullIf(tupleElement(p, 'irclickid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclickid')), 2048)), '')) AS irclickid,+ coalesce(nullIf(tupleElement(p, 'utm_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_id')), 2048)), '')) AS utm_id,+ coalesce(nullIf(tupleElement(p, 'utm_source_platform'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_source_platform')), 2048)), '')) AS utm_source_platform,+ coalesce(nullIf(tupleElement(p, 'utm_creative_format'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_creative_format')), 2048)), '')) AS utm_creative_format,+ coalesce(nullIf(tupleElement(p, 'utm_marketing_tactic'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_marketing_tactic')), 2048)), '')) AS utm_marketing_tactic,+ coalesce(nullIf(tupleElement(p, 'yclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'yclid')), 2048)), '')) AS yclid,+ coalesce(nullIf(tupleElement(p, 'tblci'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'tblci')), 2048)), '')) AS tblci,+ coalesce(nullIf(tupleElement(p, 'dicbo'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dicbo')), 2048)), '')) AS dicbo,+ coalesce(nullIf(tupleElement(p, 'ndclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ndclid')), 2048)), '')) AS ndclid,+ coalesce(nullIf(tupleElement(p, 'cjevent'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'cjevent')), 2048)), '')) AS cjevent,+ coalesce(nullIf(tupleElement(p, 'awc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'awc')), 2048)), '')) AS awc,+ coalesce(nullIf(tupleElement(p, 'syclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'syclid')), 2048)), '')) AS syclid,+ coalesce(nullIf(tupleElement(p, '_bhlid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_bhlid')), 2048)), '')) AS _bhlid,+ coalesce(nullIf(tupleElement(p, 'rtid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rtid')), 2048)), '')) AS rtid,+ coalesce(nullIf(tupleElement(p, 'oppref'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'oppref')), 2048)), '')) AS oppref,+ coalesce(nullIf(tupleElement(p, 'bfcid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'bfcid')), 2048)), '')) AS bfcid,+ CAST(arrayFilter(x -> (x IS NOT NULL), [if(gclsrc IS NOT NULL, concat('gclsrc=', gclsrc), NULL), if(dclid IS NOT NULL, concat('dclid=', dclid), NULL), if(gbraid IS NOT NULL, concat('gbraid=', gbraid), NULL), if(wbraid IS NOT NULL, concat('wbraid=', wbraid), NULL), if(gad_campaignid IS NOT NULL, concat('gad_campaignid=', gad_campaignid), NULL), if(srsltid IS NOT NULL, concat('srsltid=', srsltid), NULL), if(msclkid IS NOT NULL, concat('msclkid=', msclkid), NULL), if(twclid IS NOT NULL, concat('twclid=', twclid), NULL), if(li_fat_id IS NOT NULL, concat('li_fat_id=', li_fat_id), NULL), if(mc_cid IS NOT NULL, concat('mc_cid=', mc_cid), NULL), if(igshid IS NOT NULL, concat('igshid=', igshid), NULL), if(ttclid IS NOT NULL, concat('ttclid=', ttclid), NULL), if(rdt_cid IS NOT NULL, concat('rdt_cid=', rdt_cid), NULL), if(epik IS NOT NULL, concat('epik=', epik), NULL), if(qclid IS NOT NULL, concat('qclid=', qclid), NULL), if(sccid IS NOT NULL, concat('sccid=', sccid), NULL), if(_kx IS NOT NULL, concat('_kx=', _kx), NULL), if(irclid IS NOT NULL, concat('irclid=', irclid), NULL), if(irclickid IS NOT NULL, concat('irclickid=', irclickid), NULL), if(utm_id IS NOT NULL, concat('utm_id=', utm_id), NULL), if(utm_source_platform IS NOT NULL, concat('utm_source_platform=', utm_source_platform), NULL), if(utm_creative_format IS NOT NULL, concat('utm_creative_format=', utm_creative_format), NULL), if(utm_marketing_tactic IS NOT NULL, concat('utm_marketing_tactic=', utm_marketing_tactic), NULL), if(yclid IS NOT NULL, concat('yclid=', yclid), NULL), if(tblci IS NOT NULL, concat('tblci=', tblci), NULL), if(dicbo IS NOT NULL, concat('dicbo=', dicbo), NULL), if(ndclid IS NOT NULL, concat('ndclid=', ndclid), NULL), if(cjevent IS NOT NULL, concat('cjevent=', cjevent), NULL), if(awc IS NOT NULL, concat('awc=', awc), NULL), if(syclid IS NOT NULL, concat('syclid=', syclid), NULL), if(_bhlid IS NOT NULL, concat('_bhlid=', _bhlid), NULL), if(rtid IS NOT NULL, concat('rtid=', rtid), NULL), if(oppref IS NOT NULL, concat('oppref=', oppref), NULL), if(bfcid IS NOT NULL, concat('bfcid=', bfcid), NULL)]), 'Array(String)') AS ad_ids_set,+ tupleElement(p, '$host') AS _host,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp + toIntervalYear(1)) AS pageview_prio_timestamp_min,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp - toIntervalYear(1)) AS pageview_prio_timestamp_max+SELECT+ team_id,+ `$session_id_uuid` AS session_id_v7,+ initializeAggregation('argMaxState', source_table.distinct_id, timestamp) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ inserted_at AS max_inserted_at,+ if(+ (_current_url IS NOT NULL) AND ((event = '$pageview') OR (event = '$screen')),+ [_current_url],+ []+ ) AS urls,+ initializeAggregation('argMinState', _current_url, pageview_prio_timestamp_min) AS entry_url,+ initializeAggregation('argMaxState', _current_url, pageview_prio_timestamp_max) AS end_url,+ initializeAggregation('argMaxState', _external_click_url, timestamp) AS last_external_click_url,+ initializeAggregation('argMinState', _browser, timestamp) AS browser,+ initializeAggregation('argMinState', _browser_version, timestamp) AS browser_version,+ initializeAggregation('argMinState', _os, timestamp) AS os,+ initializeAggregation('argMinState', _os_version, timestamp) AS os_version,+ initializeAggregation('argMinState', _device_type, timestamp) AS device_type,+ initializeAggregation('argMinState', _viewport_width, timestamp) AS viewport_width,+ initializeAggregation('argMinState', _viewport_height, timestamp) AS viewport_height,+ initializeAggregation('argMinState', _geoip_country_code, timestamp) AS geoip_country_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_code, timestamp) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_name, timestamp) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', _geoip_subdivision_city_name, timestamp) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', _geoip_time_zone, timestamp) AS geoip_time_zone,+ initializeAggregation('argMinState', _referring_domain, pageview_prio_timestamp_min) AS entry_referring_domain,+ initializeAggregation('argMinState', _utm_source, pageview_prio_timestamp_min) AS entry_utm_source,+ initializeAggregation('argMinState', _utm_campaign, pageview_prio_timestamp_min) AS entry_utm_campaign,+ initializeAggregation('argMinState', _utm_medium, pageview_prio_timestamp_min) AS entry_utm_medium,+ initializeAggregation('argMinState', _utm_term, pageview_prio_timestamp_min) AS entry_utm_term,+ initializeAggregation('argMinState', _utm_content, pageview_prio_timestamp_min) AS entry_utm_content,+ initializeAggregation('argMinState', _gclid, pageview_prio_timestamp_min) AS entry_gclid,+ initializeAggregation('argMinState', _gad_source, pageview_prio_timestamp_min) AS entry_gad_source,+ initializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) AS entry_fbclid,+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_fbclid,+ initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(+ _utm_source,+ _utm_medium,+ _utm_campaign,+ _referring_domain,+ nullIf(_gclid, '') IS NOT NULL,+ nullIf(_fbclid, '') IS NOT NULL,+ _gad_source+ ),+ pageview_prio_timestamp_min+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', if(event = '$pageview', uuid, NULL)) AS pageview_uniq,+ initializeAggregation('uniqState', if(event = '$autocapture', uuid, NULL)) AS autocapture_uniq,+ initializeAggregation('uniqState', if(event = '$screen', uuid, NULL)) AS screen_uniq,+ initializeAggregation(+ 'uniqUpToState(1)',+ if((event = '$pageview') OR (event = '$screen'), uuid, NULL)+ ) AS page_screen_uniq_up_to,+ event = '$autocapture' AS has_autocapture,+ arrayMap(+ (k, v) -> concat(k, '=', v),+ mapKeys(properties_group_feature_flags),+ mapValues(properties_group_feature_flags)+ ) AS flag_key_values,+ [event] AS event_names,+ if((_host IS NOT NULL) AND (_host != ''), [_host], []) AS hosts,+ if((_person_email IS NOT NULL) AND (_person_email != ''), [_person_email], []) AS emails,+ false AS has_replay_events+FROM posthog.sharded_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) >= (timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) <= (timestamp + toIntervalDay(31))))+AND+ true+SQL++ column "team_id" {+ type = "Int64"+ }+ column "session_id_v7" {+ type = "Nullable(UInt128)"+ }+ column "distinct_id" {+ type = "AggregateFunction(argMax, String, DateTime64(6, 'UTC'))"+ }+ column "distinct_ids" {+ type = "AggregateFunction(groupUniqArray, String)"+ }+ column "min_timestamp" {+ type = "DateTime64(6, 'UTC')"+ }+ column "max_timestamp" {+ type = "DateTime64(6, 'UTC')"+ }+ column "max_inserted_at" {+ type = "Nullable(DateTime64(6, 'UTC'))"+ }+ column "urls" {+ type = "Array(Nullable(String))"+ }+ column "entry_url" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "end_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "last_external_click_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "browser" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "browser_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "os" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "os_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "device_type" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "viewport_width" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC'))"+ }+ column "viewport_height" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC'))"+ }+ column "geoip_country_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_1_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_1_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_city_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_time_zone" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_referring_domain" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_campaign" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_medium" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_term" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_content" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_gclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_gad_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_fbclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_has_gclid" {+ type = "AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC'))"+ }+ column "entry_has_fbclid" {+ type = "AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC'))"+ }+ column "entry_ad_ids_set" {+ type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"+ }+ column "entry_channel_type_properties" {+ type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), UInt8, UInt8, Nullable(String)), DateTime64(6, 'UTC'))"+ }+ column "pageview_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "autocapture_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "screen_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "page_screen_uniq_up_to" {+ type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"+ }+ column "has_autocapture" {+ type = "UInt8"+ }+ column "flag_key_values" {+ type = "Array(String)"+ }+ column "event_names" {+ type = "Array(String)"+ }+ column "hosts" {+ type = "Array(Nullable(String))"+ }+ column "emails" {+ type = "Array(String)"+ }+ column "has_replay_events" {+ type = "Bool"+ }+ }++ materialized_view "raw_sessions_v3_recordings_mv" {+ to_table = "posthog.writable_raw_sessions_v3"+ query = <<SQL+WITH+ min_first_timestamp AS timestamp,+ CAST(fromUnixTimestamp64Milli(9223372036854775), 'DateTime64(6)') AS max_ts_64,+ CAST(fromUnixTimestamp64Milli(-9223372036854775), 'DateTime64(6)') AS min_ts_64,+ CAST(NULL, 'Nullable(String)') AS null_s,+ CAST(NULL, 'Nullable(Int64)') AS null_i64,+ CAST(NULL, 'Nullable(UUID)') AS null_uuid+SELECT+ team_id,+ toUInt128(accurateCast(session_id, 'UUID')) AS session_id_v7,+ fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))) AS session_timestamp,+ initializeAggregation('argMaxState', source_table.distinct_id, min_ts_64) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ fromUnixTimestamp(0) AS max_inserted_at,+ CAST([], 'Array(String)') AS urls,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS end_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS last_external_click_url,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS device_type,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_width,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_height,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_country_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_time_zone,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_referring_domain,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_campaign,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_medium,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_term,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_content,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gclid,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gad_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_fbclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_gclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_fbclid,+ initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(null_s, null_s, null_s, null_s, false, false, null_s),+ max_ts_64+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', null_uuid) AS pageview_uniq,+ initializeAggregation('uniqState', null_uuid) AS autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) AS screen_uniq,+ initializeAggregation('uniqUpToState(1)', null_uuid) AS page_screen_uniq_up_to,+ false AS has_autocapture,+ CAST([], 'Array(String)') AS flag_key_values,+ CAST([], 'Array(String)') AS event_names,+ CAST([], 'Array(String)') AS hosts,+ CAST([], 'Array(String)') AS emails,+ true AS has_replay_events+FROM posthog.sharded_session_replay_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) >= (min_first_timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) <= (min_first_timestamp + toIntervalDay(31))))+AND+ true+SQL++ column "team_id" {+ type = "Int64"+ }+ column "session_id_v7" {+ type = "UInt128"+ }+ column "session_timestamp" {+ type = "DateTime64(3)"+ }+ column "distinct_id" {+ type = "AggregateFunction(argMax, String, DateTime64(6))"+ }+ column "distinct_ids" {+ type = "AggregateFunction(groupUniqArray, String)"+ }+ column "min_timestamp" {+ type = "SimpleAggregateFunction(min, DateTime64(6, 'UTC'))"+ }+ column "max_timestamp" {+ type = "SimpleAggregateFunction(min, DateTime64(6, 'UTC'))"+ }+ column "max_inserted_at" {+ type = "DateTime"+ }+ column "urls" {+ type = "Array(String)"+ }+ column "entry_url" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "end_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6))"+ }+ column "last_external_click_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6))"+ }+ column "browser" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "browser_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "os" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "os_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "device_type" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "viewport_width" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6))"+ }+ column "viewport_height" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6))"+ }+ column "geoip_country_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_1_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_1_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_city_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_time_zone" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_referring_domain" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_campaign" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_medium" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_term" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_content" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_gclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_gad_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_fbclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_has_gclid" {+ type = "AggregateFunction(argMin, Bool, DateTime64(6))"+ }+ column "entry_has_fbclid" {+ type = "AggregateFunction(argMin, Bool, DateTime64(6))"+ }+ column "entry_ad_ids_set" {+ type = "AggregateFunction(argMin, Array(String), DateTime64(6))"+ }+ column "entry_channel_type_properties" {+ type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6))"+ }+ column "pageview_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "autocapture_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "screen_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "page_screen_uniq_up_to" {+ type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"+ }+ column "has_autocapture" {+ type = "Bool"+ }+ column "flag_key_values" {+ type = "Array(String)"+ }+ column "event_names" {+ type = "Array(String)"+ }+ column "hosts" {+ type = "Array(String)"+ }+ column "emails" {+ type = "Array(String)"+ }+ column "has_replay_events" {+ type = "Bool"+ }+ }+materialized_view "session_replay_events_mv" {to_table = "posthog.writable_session_replay_events"query = <<SQL@@ -26298,16 +26776,14 @@ SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/golden/prod-eu/data.hcl b/posthog/clickhouse/hcl/golden/prod-eu/data.hclindex b434e375ebe2..6ba9707ca10e 100644--- a/posthog/clickhouse/hcl/golden/prod-eu/data.hcl+++ b/posthog/clickhouse/hcl/golden/prod-eu/data.hcl@@ -4672,9 +4672,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -4682,13 +4679,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -4699,12 +4696,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -6562,9 +6553,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -6572,13 +6560,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -6589,12 +6577,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -6612,15 +6594,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -9231,9 +9208,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -9241,13 +9215,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -9258,12 +9232,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -10981,16 +10949,14 @@ SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/golden/prod-eu/sessions.hcl b/posthog/clickhouse/hcl/golden/prod-eu/sessions.hclindex 2949718c96d4..8b313a626a8a 100644--- a/posthog/clickhouse/hcl/golden/prod-eu/sessions.hcl+++ b/posthog/clickhouse/hcl/golden/prod-eu/sessions.hcl@@ -1006,9 +1006,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1016,13 +1013,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1033,12 +1030,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -1056,15 +1047,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}engine "replicated_aggregating_merge_tree" {zoo_path = "/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3"diff --git a/posthog/clickhouse/hcl/golden/prod-us/data.hcl b/posthog/clickhouse/hcl/golden/prod-us/data.hclindex 0a03c13f8977..7c977c4bbce9 100644--- a/posthog/clickhouse/hcl/golden/prod-us/data.hcl+++ b/posthog/clickhouse/hcl/golden/prod-us/data.hcl@@ -4467,9 +4467,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -4477,13 +4474,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -4494,12 +4491,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -4513,9 +4504,9 @@ database "posthog" {type = "SimpleAggregateFunction(max, Bool)"}engine "distributed" {- cluster_name = "posthog"+ cluster_name = "sessions"remote_database = "posthog"- remote_table = "sharded_raw_sessions_v3"+ remote_table = "raw_sessions_v3"sharding_key = "cityHash64(session_id_v7)"}}@@ -6402,9 +6393,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -6412,13 +6400,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -6429,12 +6417,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -6452,15 +6434,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -9053,9 +9030,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -9063,13 +9037,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -9080,12 +9054,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -10766,16 +10734,14 @@ SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/golden/prod-us/sessions.hcl b/posthog/clickhouse/hcl/golden/prod-us/sessions.hclindex a9f5131a9e8b..0ab554ed0917 100644--- a/posthog/clickhouse/hcl/golden/prod-us/sessions.hcl+++ b/posthog/clickhouse/hcl/golden/prod-us/sessions.hcl@@ -1017,9 +1017,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1027,13 +1024,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1044,12 +1041,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -1067,15 +1058,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}engine "replicated_aggregating_merge_tree" {zoo_path = "/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3"diff --git a/posthog/clickhouse/hcl/roles/data/cloud/tables.hcl b/posthog/clickhouse/hcl/roles/data/cloud/tables.hclindex daa3ab21931e..8d0dbc806581 100644--- a/posthog/clickhouse/hcl/roles/data/cloud/tables.hcl+++ b/posthog/clickhouse/hcl/roles/data/cloud/tables.hcl@@ -417,9 +417,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -427,13 +424,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -444,12 +441,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -1060,9 +1051,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1070,13 +1058,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1087,12 +1075,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}diff --git a/posthog/clickhouse/hcl/roles/data/common/shared_declarations.hcl b/posthog/clickhouse/hcl/roles/data/common/shared_declarations.hclindex 29297717ed2c..6104d7810b27 100644--- a/posthog/clickhouse/hcl/roles/data/common/shared_declarations.hcl+++ b/posthog/clickhouse/hcl/roles/data/common/shared_declarations.hcl@@ -970,9 +970,6 @@ SQLcolumn "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -980,13 +977,13 @@ SQLtype = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -997,9 +994,6 @@ SQLcolumn "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -1058,16 +1052,14 @@ SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,@@ -1237,9 +1229,6 @@ SQLcolumn "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1247,13 +1236,13 @@ SQLtype = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1264,9 +1253,6 @@ SQLcolumn "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}@@ -1284,15 +1270,10 @@ SQLtype = "bloom_filter()"granularity = 1}- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1- }- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -1937,9 +1918,6 @@ SQLcolumn "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1947,13 +1925,13 @@ SQLtype = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1964,9 +1942,6 @@ SQLcolumn "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String))"}diff --git a/posthog/clickhouse/hcl/roles/data/dev/tables.hcl b/posthog/clickhouse/hcl/roles/data/dev/tables.hclindex e91470263ce3..7095450030f0 100644--- a/posthog/clickhouse/hcl/roles/data/dev/tables.hcl+++ b/posthog/clickhouse/hcl/roles/data/dev/tables.hcl@@ -1181,9 +1181,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -1191,13 +1188,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -1208,12 +1205,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -1231,15 +1222,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"diff --git a/posthog/clickhouse/hcl/roles/data/local/sql/raw_sessions_v3_mv.sql b/posthog/clickhouse/hcl/roles/data/local/sql/raw_sessions_v3_mv.sqlnew file mode 100644index 000000000000..818c0a9a6df2--- /dev/null+++ b/posthog/clickhouse/hcl/roles/data/local/sql/raw_sessions_v3_mv.sql@@ -0,0 +1,142 @@+WITH+ JSONExtract(properties, 'Tuple(\n `$current_url` Nullable(String),\n `$external_click_url` Nullable(String),\n `$browser` Nullable(String),\n `$browser_version` Nullable(String),\n `$os` Nullable(String),\n `$os_version` Nullable(String),\n `$device_type` Nullable(String),\n `$viewport_width` Nullable(Int64),\n `$viewport_height` Nullable(Int64),\n `$geoip_country_code` Nullable(String),\n `$geoip_subdivision_1_code` Nullable(String),\n `$geoip_subdivision_1_name` Nullable(String),\n `$geoip_city_name` Nullable(String),\n `$geoip_time_zone` Nullable(String),\n `$referring_domain` Nullable(String),\n `utm_source` Nullable(String),\n `utm_campaign` Nullable(String),\n `utm_medium` Nullable(String),\n `utm_term` Nullable(String),\n `utm_content` Nullable(String),\n `gclid` Nullable(String),\n `gad_source` Nullable(String),\n `fbclid` Nullable(String),\n `$host` Nullable(String),\n `gclsrc` Nullable(String),\n `dclid` Nullable(String),\n `gbraid` Nullable(String),\n `wbraid` Nullable(String),\n `gad_campaignid` Nullable(String),\n `srsltid` Nullable(String),\n `msclkid` Nullable(String),\n `twclid` Nullable(String),\n `li_fat_id` Nullable(String),\n `mc_cid` Nullable(String),\n `igshid` Nullable(String),\n `ttclid` Nullable(String),\n `rdt_cid` Nullable(String),\n `epik` Nullable(String),\n `qclid` Nullable(String),\n `sccid` Nullable(String),\n `_kx` Nullable(String),\n `irclid` Nullable(String),\n `irclickid` Nullable(String),\n `utm_id` Nullable(String),\n `utm_source_platform` Nullable(String),\n `utm_creative_format` Nullable(String),\n `utm_marketing_tactic` Nullable(String),\n `yclid` Nullable(String),\n `tblci` Nullable(String),\n `dicbo` Nullable(String),\n `ndclid` Nullable(String),\n `cjevent` Nullable(String),\n `awc` Nullable(String),\n `syclid` Nullable(String),\n `_bhlid` Nullable(String),\n `rtid` Nullable(String),\n `oppref` Nullable(String),\n `bfcid` Nullable(String)\n )') AS p,+ JSONExtractString(person_properties, 'email') AS _person_email,+ tupleElement(p, '$current_url') AS _current_url,+ if((event = '$pageview') OR (event = '$screen'), left(coalesce(_current_url, ''), 16384), '') AS _attribution_url,+ tupleElement(p, '$external_click_url') AS _external_click_url,+ tupleElement(p, '$browser') AS _browser,+ tupleElement(p, '$browser_version') AS _browser_version,+ tupleElement(p, '$os') AS _os,+ tupleElement(p, '$os_version') AS _os_version,+ tupleElement(p, '$device_type') AS _device_type,+ tupleElement(p, '$viewport_width') AS _viewport_width,+ tupleElement(p, '$viewport_height') AS _viewport_height,+ tupleElement(p, '$geoip_country_code') AS _geoip_country_code,+ tupleElement(p, '$geoip_subdivision_1_code') AS _geoip_subdivision_1_code,+ tupleElement(p, '$geoip_subdivision_1_name') AS _geoip_subdivision_1_name,+ tupleElement(p, '$geoip_city_name') AS _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_time_zone') AS _geoip_time_zone,+ tupleElement(p, '$referring_domain') AS _referring_domain,+ tupleElement(p, 'utm_source') AS _utm_source,+ tupleElement(p, 'utm_campaign') AS _utm_campaign,+ tupleElement(p, 'utm_medium') AS _utm_medium,+ tupleElement(p, 'utm_term') AS _utm_term,+ tupleElement(p, 'utm_content') AS _utm_content,+ tupleElement(p, 'gclid') AS _gclid,+ tupleElement(p, 'gad_source') AS _gad_source,+ tupleElement(p, 'fbclid') AS _fbclid,+ coalesce(nullIf(tupleElement(p, 'gclsrc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gclsrc')), 2048)), '')) AS gclsrc,+ coalesce(nullIf(tupleElement(p, 'dclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dclid')), 2048)), '')) AS dclid,+ coalesce(nullIf(tupleElement(p, 'gbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gbraid')), 2048)), '')) AS gbraid,+ coalesce(nullIf(tupleElement(p, 'wbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'wbraid')), 2048)), '')) AS wbraid,+ coalesce(nullIf(tupleElement(p, 'gad_campaignid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gad_campaignid')), 2048)), '')) AS gad_campaignid,+ coalesce(nullIf(tupleElement(p, 'srsltid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'srsltid')), 2048)), '')) AS srsltid,+ coalesce(nullIf(tupleElement(p, 'msclkid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'msclkid')), 2048)), '')) AS msclkid,+ coalesce(nullIf(tupleElement(p, 'twclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'twclid')), 2048)), '')) AS twclid,+ coalesce(nullIf(tupleElement(p, 'li_fat_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'li_fat_id')), 2048)), '')) AS li_fat_id,+ coalesce(nullIf(tupleElement(p, 'mc_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'mc_cid')), 2048)), '')) AS mc_cid,+ coalesce(nullIf(tupleElement(p, 'igshid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'igshid')), 2048)), '')) AS igshid,+ coalesce(nullIf(tupleElement(p, 'ttclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ttclid')), 2048)), '')) AS ttclid,+ coalesce(nullIf(tupleElement(p, 'rdt_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rdt_cid')), 2048)), '')) AS rdt_cid,+ coalesce(nullIf(tupleElement(p, 'epik'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'epik')), 2048)), '')) AS epik,+ coalesce(nullIf(tupleElement(p, 'qclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'qclid')), 2048)), '')) AS qclid,+ coalesce(nullIf(tupleElement(p, 'sccid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'sccid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ScCid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'SsCid')), 2048)), '')) AS sccid,+ coalesce(nullIf(tupleElement(p, '_kx'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_kx')), 2048)), '')) AS _kx,+ coalesce(nullIf(tupleElement(p, 'irclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclid')), 2048)), '')) AS irclid,+ coalesce(nullIf(tupleElement(p, 'irclickid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclickid')), 2048)), '')) AS irclickid,+ coalesce(nullIf(tupleElement(p, 'utm_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_id')), 2048)), '')) AS utm_id,+ coalesce(nullIf(tupleElement(p, 'utm_source_platform'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_source_platform')), 2048)), '')) AS utm_source_platform,+ coalesce(nullIf(tupleElement(p, 'utm_creative_format'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_creative_format')), 2048)), '')) AS utm_creative_format,+ coalesce(nullIf(tupleElement(p, 'utm_marketing_tactic'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_marketing_tactic')), 2048)), '')) AS utm_marketing_tactic,+ coalesce(nullIf(tupleElement(p, 'yclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'yclid')), 2048)), '')) AS yclid,+ coalesce(nullIf(tupleElement(p, 'tblci'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'tblci')), 2048)), '')) AS tblci,+ coalesce(nullIf(tupleElement(p, 'dicbo'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dicbo')), 2048)), '')) AS dicbo,+ coalesce(nullIf(tupleElement(p, 'ndclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ndclid')), 2048)), '')) AS ndclid,+ coalesce(nullIf(tupleElement(p, 'cjevent'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'cjevent')), 2048)), '')) AS cjevent,+ coalesce(nullIf(tupleElement(p, 'awc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'awc')), 2048)), '')) AS awc,+ coalesce(nullIf(tupleElement(p, 'syclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'syclid')), 2048)), '')) AS syclid,+ coalesce(nullIf(tupleElement(p, '_bhlid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_bhlid')), 2048)), '')) AS _bhlid,+ coalesce(nullIf(tupleElement(p, 'rtid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rtid')), 2048)), '')) AS rtid,+ coalesce(nullIf(tupleElement(p, 'oppref'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'oppref')), 2048)), '')) AS oppref,+ coalesce(nullIf(tupleElement(p, 'bfcid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'bfcid')), 2048)), '')) AS bfcid,+ CAST(arrayFilter(x -> (x IS NOT NULL), [if(gclsrc IS NOT NULL, concat('gclsrc=', gclsrc), NULL), if(dclid IS NOT NULL, concat('dclid=', dclid), NULL), if(gbraid IS NOT NULL, concat('gbraid=', gbraid), NULL), if(wbraid IS NOT NULL, concat('wbraid=', wbraid), NULL), if(gad_campaignid IS NOT NULL, concat('gad_campaignid=', gad_campaignid), NULL), if(srsltid IS NOT NULL, concat('srsltid=', srsltid), NULL), if(msclkid IS NOT NULL, concat('msclkid=', msclkid), NULL), if(twclid IS NOT NULL, concat('twclid=', twclid), NULL), if(li_fat_id IS NOT NULL, concat('li_fat_id=', li_fat_id), NULL), if(mc_cid IS NOT NULL, concat('mc_cid=', mc_cid), NULL), if(igshid IS NOT NULL, concat('igshid=', igshid), NULL), if(ttclid IS NOT NULL, concat('ttclid=', ttclid), NULL), if(rdt_cid IS NOT NULL, concat('rdt_cid=', rdt_cid), NULL), if(epik IS NOT NULL, concat('epik=', epik), NULL), if(qclid IS NOT NULL, concat('qclid=', qclid), NULL), if(sccid IS NOT NULL, concat('sccid=', sccid), NULL), if(_kx IS NOT NULL, concat('_kx=', _kx), NULL), if(irclid IS NOT NULL, concat('irclid=', irclid), NULL), if(irclickid IS NOT NULL, concat('irclickid=', irclickid), NULL), if(utm_id IS NOT NULL, concat('utm_id=', utm_id), NULL), if(utm_source_platform IS NOT NULL, concat('utm_source_platform=', utm_source_platform), NULL), if(utm_creative_format IS NOT NULL, concat('utm_creative_format=', utm_creative_format), NULL), if(utm_marketing_tactic IS NOT NULL, concat('utm_marketing_tactic=', utm_marketing_tactic), NULL), if(yclid IS NOT NULL, concat('yclid=', yclid), NULL), if(tblci IS NOT NULL, concat('tblci=', tblci), NULL), if(dicbo IS NOT NULL, concat('dicbo=', dicbo), NULL), if(ndclid IS NOT NULL, concat('ndclid=', ndclid), NULL), if(cjevent IS NOT NULL, concat('cjevent=', cjevent), NULL), if(awc IS NOT NULL, concat('awc=', awc), NULL), if(syclid IS NOT NULL, concat('syclid=', syclid), NULL), if(_bhlid IS NOT NULL, concat('_bhlid=', _bhlid), NULL), if(rtid IS NOT NULL, concat('rtid=', rtid), NULL), if(oppref IS NOT NULL, concat('oppref=', oppref), NULL), if(bfcid IS NOT NULL, concat('bfcid=', bfcid), NULL)]), 'Array(String)') AS ad_ids_set,+ tupleElement(p, '$host') AS _host,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp + toIntervalYear(1)) AS pageview_prio_timestamp_min,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp - toIntervalYear(1)) AS pageview_prio_timestamp_max+SELECT+ team_id,+ `$session_id_uuid` AS session_id_v7,+ initializeAggregation('argMaxState', source_table.distinct_id, timestamp) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ inserted_at AS max_inserted_at,+ if(+ (_current_url IS NOT NULL) AND ((event = '$pageview') OR (event = '$screen')),+ [_current_url],+ []+ ) AS urls,+ initializeAggregation('argMinState', _current_url, pageview_prio_timestamp_min) AS entry_url,+ initializeAggregation('argMaxState', _current_url, pageview_prio_timestamp_max) AS end_url,+ initializeAggregation('argMaxState', _external_click_url, timestamp) AS last_external_click_url,+ initializeAggregation('argMinState', _browser, timestamp) AS browser,+ initializeAggregation('argMinState', _browser_version, timestamp) AS browser_version,+ initializeAggregation('argMinState', _os, timestamp) AS os,+ initializeAggregation('argMinState', _os_version, timestamp) AS os_version,+ initializeAggregation('argMinState', _device_type, timestamp) AS device_type,+ initializeAggregation('argMinState', _viewport_width, timestamp) AS viewport_width,+ initializeAggregation('argMinState', _viewport_height, timestamp) AS viewport_height,+ initializeAggregation('argMinState', _geoip_country_code, timestamp) AS geoip_country_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_code, timestamp) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_name, timestamp) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', _geoip_subdivision_city_name, timestamp) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', _geoip_time_zone, timestamp) AS geoip_time_zone,+ initializeAggregation('argMinState', _referring_domain, pageview_prio_timestamp_min) AS entry_referring_domain,+ initializeAggregation('argMinState', _utm_source, pageview_prio_timestamp_min) AS entry_utm_source,+ initializeAggregation('argMinState', _utm_campaign, pageview_prio_timestamp_min) AS entry_utm_campaign,+ initializeAggregation('argMinState', _utm_medium, pageview_prio_timestamp_min) AS entry_utm_medium,+ initializeAggregation('argMinState', _utm_term, pageview_prio_timestamp_min) AS entry_utm_term,+ initializeAggregation('argMinState', _utm_content, pageview_prio_timestamp_min) AS entry_utm_content,+ initializeAggregation('argMinState', _gclid, pageview_prio_timestamp_min) AS entry_gclid,+ initializeAggregation('argMinState', _gad_source, pageview_prio_timestamp_min) AS entry_gad_source,+ initializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) AS entry_fbclid,+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_fbclid,+ initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(+ _utm_source,+ _utm_medium,+ _utm_campaign,+ _referring_domain,+ nullIf(_gclid, '') IS NOT NULL,+ nullIf(_fbclid, '') IS NOT NULL,+ _gad_source+ ),+ pageview_prio_timestamp_min+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', if(event = '$pageview', uuid, NULL)) AS pageview_uniq,+ initializeAggregation('uniqState', if(event = '$autocapture', uuid, NULL)) AS autocapture_uniq,+ initializeAggregation('uniqState', if(event = '$screen', uuid, NULL)) AS screen_uniq,+ initializeAggregation(+ 'uniqUpToState(1)',+ if((event = '$pageview') OR (event = '$screen'), uuid, NULL)+ ) AS page_screen_uniq_up_to,+ event = '$autocapture' AS has_autocapture,+ arrayMap(+ (k, v) -> concat(k, '=', v),+ mapKeys(properties_group_feature_flags),+ mapValues(properties_group_feature_flags)+ ) AS flag_key_values,+ [event] AS event_names,+ if((_host IS NOT NULL) AND (_host != ''), [_host], []) AS hosts,+ if((_person_email IS NOT NULL) AND (_person_email != ''), [_person_email], []) AS emails,+ false AS has_replay_events+FROM posthog.sharded_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) >= (timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) <= (timestamp + toIntervalDay(31))))+AND+ truediff --git a/posthog/clickhouse/hcl/roles/data/local/sql/raw_sessions_v3_recordings_mv.sql b/posthog/clickhouse/hcl/roles/data/local/sql/raw_sessions_v3_recordings_mv.sqlnew file mode 100644index 000000000000..39fa96669a0a--- /dev/null+++ b/posthog/clickhouse/hcl/roles/data/local/sql/raw_sessions_v3_recordings_mv.sql@@ -0,0 +1,66 @@+WITH+ min_first_timestamp AS timestamp,+ CAST(fromUnixTimestamp64Milli(9223372036854775), 'DateTime64(6)') AS max_ts_64,+ CAST(fromUnixTimestamp64Milli(-9223372036854775), 'DateTime64(6)') AS min_ts_64,+ CAST(NULL, 'Nullable(String)') AS null_s,+ CAST(NULL, 'Nullable(Int64)') AS null_i64,+ CAST(NULL, 'Nullable(UUID)') AS null_uuid+SELECT+ team_id,+ toUInt128(accurateCast(session_id, 'UUID')) AS session_id_v7,+ fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))) AS session_timestamp,+ initializeAggregation('argMaxState', source_table.distinct_id, min_ts_64) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ fromUnixTimestamp(0) AS max_inserted_at,+ CAST([], 'Array(String)') AS urls,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS end_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS last_external_click_url,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS device_type,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_width,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_height,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_country_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_time_zone,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_referring_domain,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_campaign,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_medium,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_term,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_content,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gclid,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gad_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_fbclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_gclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_fbclid,+ initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(null_s, null_s, null_s, null_s, false, false, null_s),+ max_ts_64+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', null_uuid) AS pageview_uniq,+ initializeAggregation('uniqState', null_uuid) AS autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) AS screen_uniq,+ initializeAggregation('uniqUpToState(1)', null_uuid) AS page_screen_uniq_up_to,+ false AS has_autocapture,+ CAST([], 'Array(String)') AS flag_key_values,+ CAST([], 'Array(String)') AS event_names,+ CAST([], 'Array(String)') AS hosts,+ CAST([], 'Array(String)') AS emails,+ true AS has_replay_events+FROM posthog.sharded_session_replay_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) >= (min_first_timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) <= (min_first_timestamp + toIntervalDay(31))))+AND+ truediff --git a/posthog/clickhouse/hcl/roles/data/local/tables.hcl b/posthog/clickhouse/hcl/roles/data/local/tables.hclindex 3543b2d9d979..58bf02876f9a 100644--- a/posthog/clickhouse/hcl/roles/data/local/tables.hcl+++ b/posthog/clickhouse/hcl/roles/data/local/tables.hcl@@ -1260,6 +1260,297 @@ database "posthog" {type = "AggregateFunction(argMin, Nullable(Float64), DateTime64(6, 'UTC'))"}}+ materialized_view "raw_sessions_v3_mv" {+ to_table = "posthog.writable_raw_sessions_v3"+ query = file("sql/raw_sessions_v3_mv.sql")++ column "team_id" {+ type = "Int64"+ }+ column "session_id_v7" {+ type = "Nullable(UInt128)"+ }+ column "distinct_id" {+ type = "AggregateFunction(argMax, String, DateTime64(6, 'UTC'))"+ }+ column "distinct_ids" {+ type = "AggregateFunction(groupUniqArray, String)"+ }+ column "min_timestamp" {+ type = "DateTime64(6, 'UTC')"+ }+ column "max_timestamp" {+ type = "DateTime64(6, 'UTC')"+ }+ column "max_inserted_at" {+ type = "Nullable(DateTime64(6, 'UTC'))"+ }+ column "urls" {+ type = "Array(Nullable(String))"+ }+ column "entry_url" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "end_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "last_external_click_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "browser" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "browser_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "os" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "os_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "device_type" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "viewport_width" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC'))"+ }+ column "viewport_height" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC'))"+ }+ column "geoip_country_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_1_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_1_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_subdivision_city_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "geoip_time_zone" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_referring_domain" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_campaign" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_medium" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_term" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_utm_content" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_gclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_gad_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_fbclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))"+ }+ column "entry_has_gclid" {+ type = "AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC'))"+ }+ column "entry_has_fbclid" {+ type = "AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC'))"+ }+ column "entry_ad_ids_set" {+ type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"+ }+ column "entry_channel_type_properties" {+ type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), UInt8, UInt8, Nullable(String)), DateTime64(6, 'UTC'))"+ }+ column "pageview_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "autocapture_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "screen_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "page_screen_uniq_up_to" {+ type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"+ }+ column "has_autocapture" {+ type = "UInt8"+ }+ column "flag_key_values" {+ type = "Array(String)"+ }+ column "event_names" {+ type = "Array(String)"+ }+ column "hosts" {+ type = "Array(Nullable(String))"+ }+ column "emails" {+ type = "Array(String)"+ }+ column "has_replay_events" {+ type = "Bool"+ }+ }++ materialized_view "raw_sessions_v3_recordings_mv" {+ to_table = "posthog.writable_raw_sessions_v3"+ query = file("sql/raw_sessions_v3_recordings_mv.sql")++ column "team_id" {+ type = "Int64"+ }+ column "session_id_v7" {+ type = "UInt128"+ }+ column "session_timestamp" {+ type = "DateTime64(3)"+ }+ column "distinct_id" {+ type = "AggregateFunction(argMax, String, DateTime64(6))"+ }+ column "distinct_ids" {+ type = "AggregateFunction(groupUniqArray, String)"+ }+ column "min_timestamp" {+ type = "SimpleAggregateFunction(min, DateTime64(6, 'UTC'))"+ }+ column "max_timestamp" {+ type = "SimpleAggregateFunction(min, DateTime64(6, 'UTC'))"+ }+ column "max_inserted_at" {+ type = "DateTime"+ }+ column "urls" {+ type = "Array(String)"+ }+ column "entry_url" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "end_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6))"+ }+ column "last_external_click_url" {+ type = "AggregateFunction(argMax, Nullable(String), DateTime64(6))"+ }+ column "browser" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "browser_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "os" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "os_version" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "device_type" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "viewport_width" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6))"+ }+ column "viewport_height" {+ type = "AggregateFunction(argMin, Nullable(Int64), DateTime64(6))"+ }+ column "geoip_country_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_1_code" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_1_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_subdivision_city_name" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "geoip_time_zone" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_referring_domain" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_campaign" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_medium" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_term" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_utm_content" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_gclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_gad_source" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_fbclid" {+ type = "AggregateFunction(argMin, Nullable(String), DateTime64(6))"+ }+ column "entry_has_gclid" {+ type = "AggregateFunction(argMin, Bool, DateTime64(6))"+ }+ column "entry_has_fbclid" {+ type = "AggregateFunction(argMin, Bool, DateTime64(6))"+ }+ column "entry_ad_ids_set" {+ type = "AggregateFunction(argMin, Array(String), DateTime64(6))"+ }+ column "entry_channel_type_properties" {+ type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6))"+ }+ column "pageview_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "autocapture_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "screen_uniq" {+ type = "AggregateFunction(uniq, Nullable(UUID))"+ }+ column "page_screen_uniq_up_to" {+ type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"+ }+ column "has_autocapture" {+ type = "Bool"+ }+ column "flag_key_values" {+ type = "Array(String)"+ }+ column "event_names" {+ type = "Array(String)"+ }+ column "hosts" {+ type = "Array(String)"+ }+ column "emails" {+ type = "Array(String)"+ }+ column "has_replay_events" {+ type = "Bool"+ }+ }+materialized_view "sessions_mv" {to_table = "posthog.writable_sessions"query = file("sql/sessions_mv.sql")diff --git a/posthog/clickhouse/hcl/roles/data/prod-eu/tables.hcl b/posthog/clickhouse/hcl/roles/data/prod-eu/tables.hclindex 306c500007f7..bd4d4c6a356c 100644--- a/posthog/clickhouse/hcl/roles/data/prod-eu/tables.hcl+++ b/posthog/clickhouse/hcl/roles/data/prod-eu/tables.hcl@@ -725,9 +725,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -735,13 +732,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -752,12 +749,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -775,15 +766,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"diff --git a/posthog/clickhouse/hcl/roles/data/prod-us/tables.hcl b/posthog/clickhouse/hcl/roles/data/prod-us/tables.hclindex 0fbcb00fa84e..62429d6afac1 100644--- a/posthog/clickhouse/hcl/roles/data/prod-us/tables.hcl+++ b/posthog/clickhouse/hcl/roles/data/prod-us/tables.hcl@@ -795,9 +795,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -805,13 +802,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -822,12 +819,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -845,15 +836,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}index "hosts_bloom_filter" {expr = "hosts"@@ -1673,6 +1659,15 @@ SQL}}+ patch_table "raw_sessions_v3" {+ engine "distributed" {+ cluster_name = "sessions"+ remote_database = "posthog"+ remote_table = "raw_sessions_v3"+ sharding_key = "cityHash64(session_id_v7)"+ }+ }+patch_table "writable_raw_sessions_v3" {engine "distributed" {cluster_name = "sessions"diff --git a/posthog/clickhouse/hcl/roles/sessions/shared/tables.hcl b/posthog/clickhouse/hcl/roles/sessions/shared/tables.hclindex 06d7130ad4f3..5f0c97e296a7 100644--- a/posthog/clickhouse/hcl/roles/sessions/shared/tables.hcl+++ b/posthog/clickhouse/hcl/roles/sessions/shared/tables.hcl@@ -388,9 +388,6 @@ database "posthog" {column "entry_has_fbclid" {type = "AggregateFunction(argMin, Bool, DateTime64(6, 'UTC'))"}- column "entry_ad_ids_map" {- type = "AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC'))"- }column "entry_ad_ids_set" {type = "AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC'))"}@@ -398,13 +395,13 @@ database "posthog" {type = "AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC'))"}column "pageview_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "autocapture_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "screen_uniq" {- type = "AggregateFunction(uniqExact, Nullable(UUID))"+ type = "AggregateFunction(uniq, Nullable(UUID))"}column "page_screen_uniq_up_to" {type = "AggregateFunction(uniqUpTo(1), Nullable(UUID))"@@ -415,12 +412,6 @@ database "posthog" {column "flag_key_values" {type = "SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String))"}- column "flag_values" {- type = "AggregateFunction(groupUniqArrayMap, Map(String, String))"- }- column "flag_keys" {- type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"- }column "event_names" {type = "SimpleAggregateFunction(groupUniqArrayArray, Array(String))"}@@ -438,15 +429,10 @@ database "posthog" {type = "bloom_filter()"granularity = 1}- index "flag_keys_bloom_filter" {- expr = "flag_keys"- type = "bloom_filter()"- granularity = 1- }- index "flag_key_values_bloom_filter" {+ index "flag_key_values_text" {expr = "flag_key_values"- type = "bloom_filter()"- granularity = 1+ type = "text(tokenizer = splitByString(['=']))"+ granularity = 100000000}engine "replicated_aggregating_merge_tree" {zoo_path = "/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3"diff --git a/posthog/clickhouse/hcl/sql/dev/data.sql b/posthog/clickhouse/hcl/sql/dev/data.sqlindex 45d354945d44..ce3aed1d9771 100644--- a/posthog/clickhouse/hcl/sql/dev/data.sql+++ b/posthog/clickhouse/hcl/sql/dev/data.sql@@ -1439,24 +1439,20 @@ CREATE TABLE posthog.sharded_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000,INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/{shard}/posthog.raw_sessions_v3', '{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192;@@ -2190,17 +2186,14 @@ CREATE TABLE posthog.writable_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -3041,17 +3034,14 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -3403,16 +3393,14 @@ CREATE VIEW posthog.raw_sessions_v3_v AS SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/sql/dev/sessions.sql b/posthog/clickhouse/hcl/sql/dev/sessions.sqlindex e8f3a288322e..f789c2ed6c8b 100644--- a/posthog/clickhouse/hcl/sql/dev/sessions.sql+++ b/posthog/clickhouse/hcl/sql/dev/sessions.sql@@ -320,24 +320,20 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3', '{shard}-{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192, storage_policy = 's3_tiered';CREATE TABLE posthog.sessions (session_id String,diff --git a/posthog/clickhouse/hcl/sql/local-multi/data.sql b/posthog/clickhouse/hcl/sql/local-multi/data.sqlindex 5bce7827aabb..035e152ffff5 100644--- a/posthog/clickhouse/hcl/sql/local-multi/data.sql+++ b/posthog/clickhouse/hcl/sql/local-multi/data.sql@@ -1436,23 +1436,20 @@ CREATE TABLE posthog.sharded_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000,INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/{shard}/posthog.raw_sessions_v3', '{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192, max_delay_to_insert = 10, parts_to_delay_insert = 250, parts_to_throw_insert = 1000;@@ -2248,16 +2245,14 @@ CREATE TABLE posthog.writable_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -2756,6 +2751,214 @@ WHEREbitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7GROUP BYteam_id, toStartOfHour(fromUnixTimestamp(intDiv(toUInt64(bitShiftRight(session_id_v7, 80)), 1000))), cityHash64(session_id_v7), session_id_v7;+CREATE MATERIALIZED VIEW posthog.raw_sessions_v3_mv TO posthog.writable_raw_sessions_v3 (team_id Int64, session_id_v7 Nullable(UInt128), distinct_id AggregateFunction(argMax, String, DateTime64(6, 'UTC')), distinct_ids AggregateFunction(groupUniqArray, String), min_timestamp DateTime64(6, 'UTC'), max_timestamp DateTime64(6, 'UTC'), max_inserted_at Nullable(DateTime64(6, 'UTC')), urls Array(Nullable(String)), entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), end_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')), last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')), browser AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), os AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), os_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), device_type AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')), viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')), geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_has_gclid AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC')), entry_has_fbclid AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC')), entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')), entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), UInt8, UInt8, Nullable(String)), DateTime64(6, 'UTC')), pageview_uniq AggregateFunction(uniq, Nullable(UUID)), autocapture_uniq AggregateFunction(uniq, Nullable(UUID)), screen_uniq AggregateFunction(uniq, Nullable(UUID)), page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)), has_autocapture UInt8, flag_key_values Array(String), event_names Array(String), hosts Array(Nullable(String)), emails Array(String), has_replay_events Bool) AS WITH+ JSONExtract(properties, 'Tuple(\n `$current_url` Nullable(String),\n `$external_click_url` Nullable(String),\n `$browser` Nullable(String),\n `$browser_version` Nullable(String),\n `$os` Nullable(String),\n `$os_version` Nullable(String),\n `$device_type` Nullable(String),\n `$viewport_width` Nullable(Int64),\n `$viewport_height` Nullable(Int64),\n `$geoip_country_code` Nullable(String),\n `$geoip_subdivision_1_code` Nullable(String),\n `$geoip_subdivision_1_name` Nullable(String),\n `$geoip_city_name` Nullable(String),\n `$geoip_time_zone` Nullable(String),\n `$referring_domain` Nullable(String),\n `utm_source` Nullable(String),\n `utm_campaign` Nullable(String),\n `utm_medium` Nullable(String),\n `utm_term` Nullable(String),\n `utm_content` Nullable(String),\n `gclid` Nullable(String),\n `gad_source` Nullable(String),\n `fbclid` Nullable(String),\n `$host` Nullable(String),\n `gclsrc` Nullable(String),\n `dclid` Nullable(String),\n `gbraid` Nullable(String),\n `wbraid` Nullable(String),\n `gad_campaignid` Nullable(String),\n `srsltid` Nullable(String),\n `msclkid` Nullable(String),\n `twclid` Nullable(String),\n `li_fat_id` Nullable(String),\n `mc_cid` Nullable(String),\n `igshid` Nullable(String),\n `ttclid` Nullable(String),\n `rdt_cid` Nullable(String),\n `epik` Nullable(String),\n `qclid` Nullable(String),\n `sccid` Nullable(String),\n `_kx` Nullable(String),\n `irclid` Nullable(String),\n `irclickid` Nullable(String),\n `utm_id` Nullable(String),\n `utm_source_platform` Nullable(String),\n `utm_creative_format` Nullable(String),\n `utm_marketing_tactic` Nullable(String),\n `yclid` Nullable(String),\n `tblci` Nullable(String),\n `dicbo` Nullable(String),\n `ndclid` Nullable(String),\n `cjevent` Nullable(String),\n `awc` Nullable(String),\n `syclid` Nullable(String),\n `_bhlid` Nullable(String),\n `rtid` Nullable(String),\n `oppref` Nullable(String),\n `bfcid` Nullable(String)\n )') AS p,+ JSONExtractString(person_properties, 'email') AS _person_email,+ tupleElement(p, '$current_url') AS _current_url,+ if((event = '$pageview') OR (event = '$screen'), left(coalesce(_current_url, ''), 16384), '') AS _attribution_url,+ tupleElement(p, '$external_click_url') AS _external_click_url,+ tupleElement(p, '$browser') AS _browser,+ tupleElement(p, '$browser_version') AS _browser_version,+ tupleElement(p, '$os') AS _os,+ tupleElement(p, '$os_version') AS _os_version,+ tupleElement(p, '$device_type') AS _device_type,+ tupleElement(p, '$viewport_width') AS _viewport_width,+ tupleElement(p, '$viewport_height') AS _viewport_height,+ tupleElement(p, '$geoip_country_code') AS _geoip_country_code,+ tupleElement(p, '$geoip_subdivision_1_code') AS _geoip_subdivision_1_code,+ tupleElement(p, '$geoip_subdivision_1_name') AS _geoip_subdivision_1_name,+ tupleElement(p, '$geoip_city_name') AS _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_time_zone') AS _geoip_time_zone,+ tupleElement(p, '$referring_domain') AS _referring_domain,+ tupleElement(p, 'utm_source') AS _utm_source,+ tupleElement(p, 'utm_campaign') AS _utm_campaign,+ tupleElement(p, 'utm_medium') AS _utm_medium,+ tupleElement(p, 'utm_term') AS _utm_term,+ tupleElement(p, 'utm_content') AS _utm_content,+ tupleElement(p, 'gclid') AS _gclid,+ tupleElement(p, 'gad_source') AS _gad_source,+ tupleElement(p, 'fbclid') AS _fbclid,+ coalesce(nullIf(tupleElement(p, 'gclsrc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gclsrc')), 2048)), '')) AS gclsrc,+ coalesce(nullIf(tupleElement(p, 'dclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dclid')), 2048)), '')) AS dclid,+ coalesce(nullIf(tupleElement(p, 'gbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gbraid')), 2048)), '')) AS gbraid,+ coalesce(nullIf(tupleElement(p, 'wbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'wbraid')), 2048)), '')) AS wbraid,+ coalesce(nullIf(tupleElement(p, 'gad_campaignid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gad_campaignid')), 2048)), '')) AS gad_campaignid,+ coalesce(nullIf(tupleElement(p, 'srsltid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'srsltid')), 2048)), '')) AS srsltid,+ coalesce(nullIf(tupleElement(p, 'msclkid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'msclkid')), 2048)), '')) AS msclkid,+ coalesce(nullIf(tupleElement(p, 'twclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'twclid')), 2048)), '')) AS twclid,+ coalesce(nullIf(tupleElement(p, 'li_fat_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'li_fat_id')), 2048)), '')) AS li_fat_id,+ coalesce(nullIf(tupleElement(p, 'mc_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'mc_cid')), 2048)), '')) AS mc_cid,+ coalesce(nullIf(tupleElement(p, 'igshid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'igshid')), 2048)), '')) AS igshid,+ coalesce(nullIf(tupleElement(p, 'ttclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ttclid')), 2048)), '')) AS ttclid,+ coalesce(nullIf(tupleElement(p, 'rdt_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rdt_cid')), 2048)), '')) AS rdt_cid,+ coalesce(nullIf(tupleElement(p, 'epik'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'epik')), 2048)), '')) AS epik,+ coalesce(nullIf(tupleElement(p, 'qclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'qclid')), 2048)), '')) AS qclid,+ coalesce(nullIf(tupleElement(p, 'sccid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'sccid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ScCid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'SsCid')), 2048)), '')) AS sccid,+ coalesce(nullIf(tupleElement(p, '_kx'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_kx')), 2048)), '')) AS _kx,+ coalesce(nullIf(tupleElement(p, 'irclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclid')), 2048)), '')) AS irclid,+ coalesce(nullIf(tupleElement(p, 'irclickid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclickid')), 2048)), '')) AS irclickid,+ coalesce(nullIf(tupleElement(p, 'utm_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_id')), 2048)), '')) AS utm_id,+ coalesce(nullIf(tupleElement(p, 'utm_source_platform'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_source_platform')), 2048)), '')) AS utm_source_platform,+ coalesce(nullIf(tupleElement(p, 'utm_creative_format'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_creative_format')), 2048)), '')) AS utm_creative_format,+ coalesce(nullIf(tupleElement(p, 'utm_marketing_tactic'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_marketing_tactic')), 2048)), '')) AS utm_marketing_tactic,+ coalesce(nullIf(tupleElement(p, 'yclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'yclid')), 2048)), '')) AS yclid,+ coalesce(nullIf(tupleElement(p, 'tblci'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'tblci')), 2048)), '')) AS tblci,+ coalesce(nullIf(tupleElement(p, 'dicbo'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dicbo')), 2048)), '')) AS dicbo,+ coalesce(nullIf(tupleElement(p, 'ndclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ndclid')), 2048)), '')) AS ndclid,+ coalesce(nullIf(tupleElement(p, 'cjevent'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'cjevent')), 2048)), '')) AS cjevent,+ coalesce(nullIf(tupleElement(p, 'awc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'awc')), 2048)), '')) AS awc,+ coalesce(nullIf(tupleElement(p, 'syclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'syclid')), 2048)), '')) AS syclid,+ coalesce(nullIf(tupleElement(p, '_bhlid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_bhlid')), 2048)), '')) AS _bhlid,+ coalesce(nullIf(tupleElement(p, 'rtid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rtid')), 2048)), '')) AS rtid,+ coalesce(nullIf(tupleElement(p, 'oppref'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'oppref')), 2048)), '')) AS oppref,+ coalesce(nullIf(tupleElement(p, 'bfcid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'bfcid')), 2048)), '')) AS bfcid,+ CAST(arrayFilter(x -> (x IS NOT NULL), [if(gclsrc IS NOT NULL, concat('gclsrc=', gclsrc), NULL), if(dclid IS NOT NULL, concat('dclid=', dclid), NULL), if(gbraid IS NOT NULL, concat('gbraid=', gbraid), NULL), if(wbraid IS NOT NULL, concat('wbraid=', wbraid), NULL), if(gad_campaignid IS NOT NULL, concat('gad_campaignid=', gad_campaignid), NULL), if(srsltid IS NOT NULL, concat('srsltid=', srsltid), NULL), if(msclkid IS NOT NULL, concat('msclkid=', msclkid), NULL), if(twclid IS NOT NULL, concat('twclid=', twclid), NULL), if(li_fat_id IS NOT NULL, concat('li_fat_id=', li_fat_id), NULL), if(mc_cid IS NOT NULL, concat('mc_cid=', mc_cid), NULL), if(igshid IS NOT NULL, concat('igshid=', igshid), NULL), if(ttclid IS NOT NULL, concat('ttclid=', ttclid), NULL), if(rdt_cid IS NOT NULL, concat('rdt_cid=', rdt_cid), NULL), if(epik IS NOT NULL, concat('epik=', epik), NULL), if(qclid IS NOT NULL, concat('qclid=', qclid), NULL), if(sccid IS NOT NULL, concat('sccid=', sccid), NULL), if(_kx IS NOT NULL, concat('_kx=', _kx), NULL), if(irclid IS NOT NULL, concat('irclid=', irclid), NULL), if(irclickid IS NOT NULL, concat('irclickid=', irclickid), NULL), if(utm_id IS NOT NULL, concat('utm_id=', utm_id), NULL), if(utm_source_platform IS NOT NULL, concat('utm_source_platform=', utm_source_platform), NULL), if(utm_creative_format IS NOT NULL, concat('utm_creative_format=', utm_creative_format), NULL), if(utm_marketing_tactic IS NOT NULL, concat('utm_marketing_tactic=', utm_marketing_tactic), NULL), if(yclid IS NOT NULL, concat('yclid=', yclid), NULL), if(tblci IS NOT NULL, concat('tblci=', tblci), NULL), if(dicbo IS NOT NULL, concat('dicbo=', dicbo), NULL), if(ndclid IS NOT NULL, concat('ndclid=', ndclid), NULL), if(cjevent IS NOT NULL, concat('cjevent=', cjevent), NULL), if(awc IS NOT NULL, concat('awc=', awc), NULL), if(syclid IS NOT NULL, concat('syclid=', syclid), NULL), if(_bhlid IS NOT NULL, concat('_bhlid=', _bhlid), NULL), if(rtid IS NOT NULL, concat('rtid=', rtid), NULL), if(oppref IS NOT NULL, concat('oppref=', oppref), NULL), if(bfcid IS NOT NULL, concat('bfcid=', bfcid), NULL)]), 'Array(String)') AS ad_ids_set,+ tupleElement(p, '$host') AS _host,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp + toIntervalYear(1)) AS pageview_prio_timestamp_min,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp - toIntervalYear(1)) AS pageview_prio_timestamp_max+SELECT+ team_id,+ `$session_id_uuid` AS session_id_v7,+ initializeAggregation('argMaxState', source_table.distinct_id, timestamp) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ inserted_at AS max_inserted_at,+ if(+ (_current_url IS NOT NULL) AND ((event = '$pageview') OR (event = '$screen')),+ [_current_url],+ []+ ) AS urls,+ initializeAggregation('argMinState', _current_url, pageview_prio_timestamp_min) AS entry_url,+ initializeAggregation('argMaxState', _current_url, pageview_prio_timestamp_max) AS end_url,+ initializeAggregation('argMaxState', _external_click_url, timestamp) AS last_external_click_url,+ initializeAggregation('argMinState', _browser, timestamp) AS browser,+ initializeAggregation('argMinState', _browser_version, timestamp) AS browser_version,+ initializeAggregation('argMinState', _os, timestamp) AS os,+ initializeAggregation('argMinState', _os_version, timestamp) AS os_version,+ initializeAggregation('argMinState', _device_type, timestamp) AS device_type,+ initializeAggregation('argMinState', _viewport_width, timestamp) AS viewport_width,+ initializeAggregation('argMinState', _viewport_height, timestamp) AS viewport_height,+ initializeAggregation('argMinState', _geoip_country_code, timestamp) AS geoip_country_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_code, timestamp) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_name, timestamp) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', _geoip_subdivision_city_name, timestamp) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', _geoip_time_zone, timestamp) AS geoip_time_zone,+ initializeAggregation('argMinState', _referring_domain, pageview_prio_timestamp_min) AS entry_referring_domain,+ initializeAggregation('argMinState', _utm_source, pageview_prio_timestamp_min) AS entry_utm_source,+ initializeAggregation('argMinState', _utm_campaign, pageview_prio_timestamp_min) AS entry_utm_campaign,+ initializeAggregation('argMinState', _utm_medium, pageview_prio_timestamp_min) AS entry_utm_medium,+ initializeAggregation('argMinState', _utm_term, pageview_prio_timestamp_min) AS entry_utm_term,+ initializeAggregation('argMinState', _utm_content, pageview_prio_timestamp_min) AS entry_utm_content,+ initializeAggregation('argMinState', _gclid, pageview_prio_timestamp_min) AS entry_gclid,+ initializeAggregation('argMinState', _gad_source, pageview_prio_timestamp_min) AS entry_gad_source,+ initializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) AS entry_fbclid,+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_fbclid,+ initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(+ _utm_source,+ _utm_medium,+ _utm_campaign,+ _referring_domain,+ nullIf(_gclid, '') IS NOT NULL,+ nullIf(_fbclid, '') IS NOT NULL,+ _gad_source+ ),+ pageview_prio_timestamp_min+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', if(event = '$pageview', uuid, NULL)) AS pageview_uniq,+ initializeAggregation('uniqState', if(event = '$autocapture', uuid, NULL)) AS autocapture_uniq,+ initializeAggregation('uniqState', if(event = '$screen', uuid, NULL)) AS screen_uniq,+ initializeAggregation(+ 'uniqUpToState(1)',+ if((event = '$pageview') OR (event = '$screen'), uuid, NULL)+ ) AS page_screen_uniq_up_to,+ event = '$autocapture' AS has_autocapture,+ arrayMap(+ (k, v) -> concat(k, '=', v),+ mapKeys(properties_group_feature_flags),+ mapValues(properties_group_feature_flags)+ ) AS flag_key_values,+ [event] AS event_names,+ if((_host IS NOT NULL) AND (_host != ''), [_host], []) AS hosts,+ if((_person_email IS NOT NULL) AND (_person_email != ''), [_person_email], []) AS emails,+ false AS has_replay_events+FROM posthog.sharded_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) >= (timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) <= (timestamp + toIntervalDay(31))))+AND+ true;+CREATE MATERIALIZED VIEW posthog.raw_sessions_v3_recordings_mv TO posthog.writable_raw_sessions_v3 (team_id Int64, session_id_v7 UInt128, session_timestamp DateTime64(3), distinct_id AggregateFunction(argMax, String, DateTime64(6)), distinct_ids AggregateFunction(groupUniqArray, String), min_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')), max_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')), max_inserted_at DateTime, urls Array(String), entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6)), end_url AggregateFunction(argMax, Nullable(String), DateTime64(6)), last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6)), browser AggregateFunction(argMin, Nullable(String), DateTime64(6)), browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6)), os AggregateFunction(argMin, Nullable(String), DateTime64(6)), os_version AggregateFunction(argMin, Nullable(String), DateTime64(6)), device_type AggregateFunction(argMin, Nullable(String), DateTime64(6)), viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6)), viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6)), geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6)), entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6)), entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6)), entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6)), pageview_uniq AggregateFunction(uniq, Nullable(UUID)), autocapture_uniq AggregateFunction(uniq, Nullable(UUID)), screen_uniq AggregateFunction(uniq, Nullable(UUID)), page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)), has_autocapture Bool, flag_key_values Array(String), event_names Array(String), hosts Array(String), emails Array(String), has_replay_events Bool) AS WITH+ min_first_timestamp AS timestamp,+ CAST(fromUnixTimestamp64Milli(9223372036854775), 'DateTime64(6)') AS max_ts_64,+ CAST(fromUnixTimestamp64Milli(-9223372036854775), 'DateTime64(6)') AS min_ts_64,+ CAST(NULL, 'Nullable(String)') AS null_s,+ CAST(NULL, 'Nullable(Int64)') AS null_i64,+ CAST(NULL, 'Nullable(UUID)') AS null_uuid+SELECT+ team_id,+ toUInt128(accurateCast(session_id, 'UUID')) AS session_id_v7,+ fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))) AS session_timestamp,+ initializeAggregation('argMaxState', source_table.distinct_id, min_ts_64) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ fromUnixTimestamp(0) AS max_inserted_at,+ CAST([], 'Array(String)') AS urls,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS end_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS last_external_click_url,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS device_type,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_width,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_height,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_country_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_time_zone,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_referring_domain,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_campaign,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_medium,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_term,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_content,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gclid,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gad_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_fbclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_gclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_fbclid,+ initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(null_s, null_s, null_s, null_s, false, false, null_s),+ max_ts_64+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', null_uuid) AS pageview_uniq,+ initializeAggregation('uniqState', null_uuid) AS autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) AS screen_uniq,+ initializeAggregation('uniqUpToState(1)', null_uuid) AS page_screen_uniq_up_to,+ false AS has_autocapture,+ CAST([], 'Array(String)') AS flag_key_values,+ CAST([], 'Array(String)') AS event_names,+ CAST([], 'Array(String)') AS hosts,+ CAST([], 'Array(String)') AS emails,+ true AS has_replay_events+FROM posthog.sharded_session_replay_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) >= (min_first_timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) <= (min_first_timestamp + toIntervalDay(31))))+AND+ true;CREATE MATERIALIZED VIEW posthog.sessions_mv TO posthog.writable_sessions (session_id String, team_id Int64, distinct_id String, min_timestamp DateTime64(6, 'UTC'), max_timestamp DateTime64(6, 'UTC'), urls Array(String), entry_url AggregateFunction(argMin, String, DateTime64(6, 'UTC')), exit_url AggregateFunction(argMax, String, DateTime64(6, 'UTC')), initial_referring_domain AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_utm_source AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_utm_campaign AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_utm_medium AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_utm_term AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_utm_content AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_gclid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_gad_source AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_gclsrc AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_dclid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_gbraid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_wbraid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_fbclid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_msclkid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_twclid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_li_fat_id AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_mc_cid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_igshid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_ttclid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_epik AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_qclid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), initial_sccid AggregateFunction(argMin, String, DateTime64(6, 'UTC')), event_count_map Map(String, UInt64), pageview_count UInt64, autocapture_count UInt64) AS SELECT`$session_id` AS session_id,team_id,@@ -3456,16 +3659,14 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -3786,16 +3987,14 @@ CREATE VIEW posthog.raw_sessions_v3_v AS SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/sql/local-single/all.sql b/posthog/clickhouse/hcl/sql/local-single/all.sqlindex 87260bcb07db..e74e7dc93895 100644--- a/posthog/clickhouse/hcl/sql/local-single/all.sql+++ b/posthog/clickhouse/hcl/sql/local-single/all.sql@@ -2682,23 +2682,20 @@ CREATE TABLE posthog.sharded_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000,INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/{shard}/posthog.raw_sessions_v3', '{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192, max_delay_to_insert = 10, parts_to_delay_insert = 250, parts_to_throw_insert = 1000;@@ -4236,16 +4233,14 @@ CREATE TABLE posthog.writable_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -5644,6 +5639,214 @@ WHEREbitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7GROUP BYteam_id, toStartOfHour(fromUnixTimestamp(intDiv(toUInt64(bitShiftRight(session_id_v7, 80)), 1000))), cityHash64(session_id_v7), session_id_v7;+CREATE MATERIALIZED VIEW posthog.raw_sessions_v3_mv TO posthog.writable_raw_sessions_v3 (team_id Int64, session_id_v7 Nullable(UInt128), distinct_id AggregateFunction(argMax, String, DateTime64(6, 'UTC')), distinct_ids AggregateFunction(groupUniqArray, String), min_timestamp DateTime64(6, 'UTC'), max_timestamp DateTime64(6, 'UTC'), max_inserted_at Nullable(DateTime64(6, 'UTC')), urls Array(Nullable(String)), entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), end_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')), last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')), browser AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), os AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), os_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), device_type AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')), viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')), geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), entry_has_gclid AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC')), entry_has_fbclid AggregateFunction(argMin, UInt8, DateTime64(6, 'UTC')), entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')), entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), UInt8, UInt8, Nullable(String)), DateTime64(6, 'UTC')), pageview_uniq AggregateFunction(uniq, Nullable(UUID)), autocapture_uniq AggregateFunction(uniq, Nullable(UUID)), screen_uniq AggregateFunction(uniq, Nullable(UUID)), page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)), has_autocapture UInt8, flag_key_values Array(String), event_names Array(String), hosts Array(Nullable(String)), emails Array(String), has_replay_events Bool) AS WITH+ JSONExtract(properties, 'Tuple(\n `$current_url` Nullable(String),\n `$external_click_url` Nullable(String),\n `$browser` Nullable(String),\n `$browser_version` Nullable(String),\n `$os` Nullable(String),\n `$os_version` Nullable(String),\n `$device_type` Nullable(String),\n `$viewport_width` Nullable(Int64),\n `$viewport_height` Nullable(Int64),\n `$geoip_country_code` Nullable(String),\n `$geoip_subdivision_1_code` Nullable(String),\n `$geoip_subdivision_1_name` Nullable(String),\n `$geoip_city_name` Nullable(String),\n `$geoip_time_zone` Nullable(String),\n `$referring_domain` Nullable(String),\n `utm_source` Nullable(String),\n `utm_campaign` Nullable(String),\n `utm_medium` Nullable(String),\n `utm_term` Nullable(String),\n `utm_content` Nullable(String),\n `gclid` Nullable(String),\n `gad_source` Nullable(String),\n `fbclid` Nullable(String),\n `$host` Nullable(String),\n `gclsrc` Nullable(String),\n `dclid` Nullable(String),\n `gbraid` Nullable(String),\n `wbraid` Nullable(String),\n `gad_campaignid` Nullable(String),\n `srsltid` Nullable(String),\n `msclkid` Nullable(String),\n `twclid` Nullable(String),\n `li_fat_id` Nullable(String),\n `mc_cid` Nullable(String),\n `igshid` Nullable(String),\n `ttclid` Nullable(String),\n `rdt_cid` Nullable(String),\n `epik` Nullable(String),\n `qclid` Nullable(String),\n `sccid` Nullable(String),\n `_kx` Nullable(String),\n `irclid` Nullable(String),\n `irclickid` Nullable(String),\n `utm_id` Nullable(String),\n `utm_source_platform` Nullable(String),\n `utm_creative_format` Nullable(String),\n `utm_marketing_tactic` Nullable(String),\n `yclid` Nullable(String),\n `tblci` Nullable(String),\n `dicbo` Nullable(String),\n `ndclid` Nullable(String),\n `cjevent` Nullable(String),\n `awc` Nullable(String),\n `syclid` Nullable(String),\n `_bhlid` Nullable(String),\n `rtid` Nullable(String),\n `oppref` Nullable(String),\n `bfcid` Nullable(String)\n )') AS p,+ JSONExtractString(person_properties, 'email') AS _person_email,+ tupleElement(p, '$current_url') AS _current_url,+ if((event = '$pageview') OR (event = '$screen'), left(coalesce(_current_url, ''), 16384), '') AS _attribution_url,+ tupleElement(p, '$external_click_url') AS _external_click_url,+ tupleElement(p, '$browser') AS _browser,+ tupleElement(p, '$browser_version') AS _browser_version,+ tupleElement(p, '$os') AS _os,+ tupleElement(p, '$os_version') AS _os_version,+ tupleElement(p, '$device_type') AS _device_type,+ tupleElement(p, '$viewport_width') AS _viewport_width,+ tupleElement(p, '$viewport_height') AS _viewport_height,+ tupleElement(p, '$geoip_country_code') AS _geoip_country_code,+ tupleElement(p, '$geoip_subdivision_1_code') AS _geoip_subdivision_1_code,+ tupleElement(p, '$geoip_subdivision_1_name') AS _geoip_subdivision_1_name,+ tupleElement(p, '$geoip_city_name') AS _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_time_zone') AS _geoip_time_zone,+ tupleElement(p, '$referring_domain') AS _referring_domain,+ tupleElement(p, 'utm_source') AS _utm_source,+ tupleElement(p, 'utm_campaign') AS _utm_campaign,+ tupleElement(p, 'utm_medium') AS _utm_medium,+ tupleElement(p, 'utm_term') AS _utm_term,+ tupleElement(p, 'utm_content') AS _utm_content,+ tupleElement(p, 'gclid') AS _gclid,+ tupleElement(p, 'gad_source') AS _gad_source,+ tupleElement(p, 'fbclid') AS _fbclid,+ coalesce(nullIf(tupleElement(p, 'gclsrc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gclsrc')), 2048)), '')) AS gclsrc,+ coalesce(nullIf(tupleElement(p, 'dclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dclid')), 2048)), '')) AS dclid,+ coalesce(nullIf(tupleElement(p, 'gbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gbraid')), 2048)), '')) AS gbraid,+ coalesce(nullIf(tupleElement(p, 'wbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'wbraid')), 2048)), '')) AS wbraid,+ coalesce(nullIf(tupleElement(p, 'gad_campaignid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gad_campaignid')), 2048)), '')) AS gad_campaignid,+ coalesce(nullIf(tupleElement(p, 'srsltid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'srsltid')), 2048)), '')) AS srsltid,+ coalesce(nullIf(tupleElement(p, 'msclkid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'msclkid')), 2048)), '')) AS msclkid,+ coalesce(nullIf(tupleElement(p, 'twclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'twclid')), 2048)), '')) AS twclid,+ coalesce(nullIf(tupleElement(p, 'li_fat_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'li_fat_id')), 2048)), '')) AS li_fat_id,+ coalesce(nullIf(tupleElement(p, 'mc_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'mc_cid')), 2048)), '')) AS mc_cid,+ coalesce(nullIf(tupleElement(p, 'igshid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'igshid')), 2048)), '')) AS igshid,+ coalesce(nullIf(tupleElement(p, 'ttclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ttclid')), 2048)), '')) AS ttclid,+ coalesce(nullIf(tupleElement(p, 'rdt_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rdt_cid')), 2048)), '')) AS rdt_cid,+ coalesce(nullIf(tupleElement(p, 'epik'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'epik')), 2048)), '')) AS epik,+ coalesce(nullIf(tupleElement(p, 'qclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'qclid')), 2048)), '')) AS qclid,+ coalesce(nullIf(tupleElement(p, 'sccid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'sccid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ScCid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'SsCid')), 2048)), '')) AS sccid,+ coalesce(nullIf(tupleElement(p, '_kx'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_kx')), 2048)), '')) AS _kx,+ coalesce(nullIf(tupleElement(p, 'irclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclid')), 2048)), '')) AS irclid,+ coalesce(nullIf(tupleElement(p, 'irclickid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclickid')), 2048)), '')) AS irclickid,+ coalesce(nullIf(tupleElement(p, 'utm_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_id')), 2048)), '')) AS utm_id,+ coalesce(nullIf(tupleElement(p, 'utm_source_platform'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_source_platform')), 2048)), '')) AS utm_source_platform,+ coalesce(nullIf(tupleElement(p, 'utm_creative_format'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_creative_format')), 2048)), '')) AS utm_creative_format,+ coalesce(nullIf(tupleElement(p, 'utm_marketing_tactic'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_marketing_tactic')), 2048)), '')) AS utm_marketing_tactic,+ coalesce(nullIf(tupleElement(p, 'yclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'yclid')), 2048)), '')) AS yclid,+ coalesce(nullIf(tupleElement(p, 'tblci'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'tblci')), 2048)), '')) AS tblci,+ coalesce(nullIf(tupleElement(p, 'dicbo'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dicbo')), 2048)), '')) AS dicbo,+ coalesce(nullIf(tupleElement(p, 'ndclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ndclid')), 2048)), '')) AS ndclid,+ coalesce(nullIf(tupleElement(p, 'cjevent'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'cjevent')), 2048)), '')) AS cjevent,+ coalesce(nullIf(tupleElement(p, 'awc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'awc')), 2048)), '')) AS awc,+ coalesce(nullIf(tupleElement(p, 'syclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'syclid')), 2048)), '')) AS syclid,+ coalesce(nullIf(tupleElement(p, '_bhlid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_bhlid')), 2048)), '')) AS _bhlid,+ coalesce(nullIf(tupleElement(p, 'rtid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rtid')), 2048)), '')) AS rtid,+ coalesce(nullIf(tupleElement(p, 'oppref'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'oppref')), 2048)), '')) AS oppref,+ coalesce(nullIf(tupleElement(p, 'bfcid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'bfcid')), 2048)), '')) AS bfcid,+ CAST(arrayFilter(x -> (x IS NOT NULL), [if(gclsrc IS NOT NULL, concat('gclsrc=', gclsrc), NULL), if(dclid IS NOT NULL, concat('dclid=', dclid), NULL), if(gbraid IS NOT NULL, concat('gbraid=', gbraid), NULL), if(wbraid IS NOT NULL, concat('wbraid=', wbraid), NULL), if(gad_campaignid IS NOT NULL, concat('gad_campaignid=', gad_campaignid), NULL), if(srsltid IS NOT NULL, concat('srsltid=', srsltid), NULL), if(msclkid IS NOT NULL, concat('msclkid=', msclkid), NULL), if(twclid IS NOT NULL, concat('twclid=', twclid), NULL), if(li_fat_id IS NOT NULL, concat('li_fat_id=', li_fat_id), NULL), if(mc_cid IS NOT NULL, concat('mc_cid=', mc_cid), NULL), if(igshid IS NOT NULL, concat('igshid=', igshid), NULL), if(ttclid IS NOT NULL, concat('ttclid=', ttclid), NULL), if(rdt_cid IS NOT NULL, concat('rdt_cid=', rdt_cid), NULL), if(epik IS NOT NULL, concat('epik=', epik), NULL), if(qclid IS NOT NULL, concat('qclid=', qclid), NULL), if(sccid IS NOT NULL, concat('sccid=', sccid), NULL), if(_kx IS NOT NULL, concat('_kx=', _kx), NULL), if(irclid IS NOT NULL, concat('irclid=', irclid), NULL), if(irclickid IS NOT NULL, concat('irclickid=', irclickid), NULL), if(utm_id IS NOT NULL, concat('utm_id=', utm_id), NULL), if(utm_source_platform IS NOT NULL, concat('utm_source_platform=', utm_source_platform), NULL), if(utm_creative_format IS NOT NULL, concat('utm_creative_format=', utm_creative_format), NULL), if(utm_marketing_tactic IS NOT NULL, concat('utm_marketing_tactic=', utm_marketing_tactic), NULL), if(yclid IS NOT NULL, concat('yclid=', yclid), NULL), if(tblci IS NOT NULL, concat('tblci=', tblci), NULL), if(dicbo IS NOT NULL, concat('dicbo=', dicbo), NULL), if(ndclid IS NOT NULL, concat('ndclid=', ndclid), NULL), if(cjevent IS NOT NULL, concat('cjevent=', cjevent), NULL), if(awc IS NOT NULL, concat('awc=', awc), NULL), if(syclid IS NOT NULL, concat('syclid=', syclid), NULL), if(_bhlid IS NOT NULL, concat('_bhlid=', _bhlid), NULL), if(rtid IS NOT NULL, concat('rtid=', rtid), NULL), if(oppref IS NOT NULL, concat('oppref=', oppref), NULL), if(bfcid IS NOT NULL, concat('bfcid=', bfcid), NULL)]), 'Array(String)') AS ad_ids_set,+ tupleElement(p, '$host') AS _host,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp + toIntervalYear(1)) AS pageview_prio_timestamp_min,+ if((event = '$pageview') OR (event = '$screen'), timestamp, timestamp - toIntervalYear(1)) AS pageview_prio_timestamp_max+SELECT+ team_id,+ `$session_id_uuid` AS session_id_v7,+ initializeAggregation('argMaxState', source_table.distinct_id, timestamp) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ inserted_at AS max_inserted_at,+ if(+ (_current_url IS NOT NULL) AND ((event = '$pageview') OR (event = '$screen')),+ [_current_url],+ []+ ) AS urls,+ initializeAggregation('argMinState', _current_url, pageview_prio_timestamp_min) AS entry_url,+ initializeAggregation('argMaxState', _current_url, pageview_prio_timestamp_max) AS end_url,+ initializeAggregation('argMaxState', _external_click_url, timestamp) AS last_external_click_url,+ initializeAggregation('argMinState', _browser, timestamp) AS browser,+ initializeAggregation('argMinState', _browser_version, timestamp) AS browser_version,+ initializeAggregation('argMinState', _os, timestamp) AS os,+ initializeAggregation('argMinState', _os_version, timestamp) AS os_version,+ initializeAggregation('argMinState', _device_type, timestamp) AS device_type,+ initializeAggregation('argMinState', _viewport_width, timestamp) AS viewport_width,+ initializeAggregation('argMinState', _viewport_height, timestamp) AS viewport_height,+ initializeAggregation('argMinState', _geoip_country_code, timestamp) AS geoip_country_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_code, timestamp) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', _geoip_subdivision_1_name, timestamp) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', _geoip_subdivision_city_name, timestamp) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', _geoip_time_zone, timestamp) AS geoip_time_zone,+ initializeAggregation('argMinState', _referring_domain, pageview_prio_timestamp_min) AS entry_referring_domain,+ initializeAggregation('argMinState', _utm_source, pageview_prio_timestamp_min) AS entry_utm_source,+ initializeAggregation('argMinState', _utm_campaign, pageview_prio_timestamp_min) AS entry_utm_campaign,+ initializeAggregation('argMinState', _utm_medium, pageview_prio_timestamp_min) AS entry_utm_medium,+ initializeAggregation('argMinState', _utm_term, pageview_prio_timestamp_min) AS entry_utm_term,+ initializeAggregation('argMinState', _utm_content, pageview_prio_timestamp_min) AS entry_utm_content,+ initializeAggregation('argMinState', _gclid, pageview_prio_timestamp_min) AS entry_gclid,+ initializeAggregation('argMinState', _gad_source, pageview_prio_timestamp_min) AS entry_gad_source,+ initializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) AS entry_fbclid,+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) AS entry_has_fbclid,+ initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(+ _utm_source,+ _utm_medium,+ _utm_campaign,+ _referring_domain,+ nullIf(_gclid, '') IS NOT NULL,+ nullIf(_fbclid, '') IS NOT NULL,+ _gad_source+ ),+ pageview_prio_timestamp_min+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', if(event = '$pageview', uuid, NULL)) AS pageview_uniq,+ initializeAggregation('uniqState', if(event = '$autocapture', uuid, NULL)) AS autocapture_uniq,+ initializeAggregation('uniqState', if(event = '$screen', uuid, NULL)) AS screen_uniq,+ initializeAggregation(+ 'uniqUpToState(1)',+ if((event = '$pageview') OR (event = '$screen'), uuid, NULL)+ ) AS page_screen_uniq_up_to,+ event = '$autocapture' AS has_autocapture,+ arrayMap(+ (k, v) -> concat(k, '=', v),+ mapKeys(properties_group_feature_flags),+ mapValues(properties_group_feature_flags)+ ) AS flag_key_values,+ [event] AS event_names,+ if((_host IS NOT NULL) AND (_host != ''), [_host], []) AS hosts,+ if((_person_email IS NOT NULL) AND (_person_email != ''), [_person_email], []) AS emails,+ false AS has_replay_events+FROM posthog.sharded_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) >= (timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) <= (timestamp + toIntervalDay(31))))+AND+ true;+CREATE MATERIALIZED VIEW posthog.raw_sessions_v3_recordings_mv TO posthog.writable_raw_sessions_v3 (team_id Int64, session_id_v7 UInt128, session_timestamp DateTime64(3), distinct_id AggregateFunction(argMax, String, DateTime64(6)), distinct_ids AggregateFunction(groupUniqArray, String), min_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')), max_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')), max_inserted_at DateTime, urls Array(String), entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6)), end_url AggregateFunction(argMax, Nullable(String), DateTime64(6)), last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6)), browser AggregateFunction(argMin, Nullable(String), DateTime64(6)), browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6)), os AggregateFunction(argMin, Nullable(String), DateTime64(6)), os_version AggregateFunction(argMin, Nullable(String), DateTime64(6)), device_type AggregateFunction(argMin, Nullable(String), DateTime64(6)), viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6)), viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6)), geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6)), geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6)), entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6)), entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6)), entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6)), entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6)), pageview_uniq AggregateFunction(uniq, Nullable(UUID)), autocapture_uniq AggregateFunction(uniq, Nullable(UUID)), screen_uniq AggregateFunction(uniq, Nullable(UUID)), page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)), has_autocapture Bool, flag_key_values Array(String), event_names Array(String), hosts Array(String), emails Array(String), has_replay_events Bool) AS WITH+ min_first_timestamp AS timestamp,+ CAST(fromUnixTimestamp64Milli(9223372036854775), 'DateTime64(6)') AS max_ts_64,+ CAST(fromUnixTimestamp64Milli(-9223372036854775), 'DateTime64(6)') AS min_ts_64,+ CAST(NULL, 'Nullable(String)') AS null_s,+ CAST(NULL, 'Nullable(Int64)') AS null_i64,+ CAST(NULL, 'Nullable(UUID)') AS null_uuid+SELECT+ team_id,+ toUInt128(accurateCast(session_id, 'UUID')) AS session_id_v7,+ fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))) AS session_timestamp,+ initializeAggregation('argMaxState', source_table.distinct_id, min_ts_64) AS distinct_id,+ initializeAggregation('groupUniqArrayState', source_table.distinct_id) AS distinct_ids,+ timestamp AS min_timestamp,+ timestamp AS max_timestamp,+ fromUnixTimestamp(0) AS max_inserted_at,+ CAST([], 'Array(String)') AS urls,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS end_url,+ initializeAggregation('argMaxState', null_s, min_ts_64) AS last_external_click_url,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser,+ initializeAggregation('argMinState', null_s, max_ts_64) AS browser_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os,+ initializeAggregation('argMinState', null_s, max_ts_64) AS os_version,+ initializeAggregation('argMinState', null_s, max_ts_64) AS device_type,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_width,+ initializeAggregation('argMinState', null_i64, max_ts_64) AS viewport_height,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_country_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_code,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_1_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_subdivision_city_name,+ initializeAggregation('argMinState', null_s, max_ts_64) AS geoip_time_zone,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_referring_domain,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_campaign,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_medium,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_term,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_utm_content,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gclid,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_gad_source,+ initializeAggregation('argMinState', null_s, max_ts_64) AS entry_fbclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_gclid,+ initializeAggregation('argMinState', false, max_ts_64) AS entry_has_fbclid,+ initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) AS entry_ad_ids_set,+ initializeAggregation(+ 'argMinState',+ tuple(null_s, null_s, null_s, null_s, false, false, null_s),+ max_ts_64+ ) AS entry_channel_type_properties,+ initializeAggregation('uniqState', null_uuid) AS pageview_uniq,+ initializeAggregation('uniqState', null_uuid) AS autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) AS screen_uniq,+ initializeAggregation('uniqUpToState(1)', null_uuid) AS page_screen_uniq_up_to,+ false AS has_autocapture,+ CAST([], 'Array(String)') AS flag_key_values,+ CAST([], 'Array(String)') AS event_names,+ CAST([], 'Array(String)') AS hosts,+ CAST([], 'Array(String)') AS emails,+ true AS has_replay_events+FROM posthog.sharded_session_replay_events AS source_table+WHERE+ (bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 15) = 7)+AND+ ((fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) >= (min_first_timestamp - toIntervalDay(31))) AND (fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) <= (min_first_timestamp + toIntervalDay(31))))+AND+ true;CREATE MATERIALIZED VIEW posthog.session_replay_events_mv TO posthog.writable_session_replay_events (session_id String, team_id Int64, distinct_id String, min_first_timestamp DateTime64(6, 'UTC'), max_last_timestamp DateTime64(6, 'UTC'), block_first_timestamps SimpleAggregateFunction(groupArrayArray, Array(DateTime64(6, 'UTC'))), block_last_timestamps SimpleAggregateFunction(groupArrayArray, Array(DateTime64(6, 'UTC'))), block_urls SimpleAggregateFunction(groupArrayArray, Array(String)), first_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), all_urls SimpleAggregateFunction(groupUniqArrayArray, Array(String)), click_count Int64, keypress_count Int64, mouse_activity_count Int64, active_milliseconds Int64, console_log_count Int64, console_warn_count Int64, console_error_count Int64, size Int64, message_count Int64, event_count Int64, snapshot_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), snapshot_library AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), snapshot_mode_v2 AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')), _timestamp Nullable(DateTime), retention_period_days SimpleAggregateFunction(max, Nullable(Int64)), is_deleted SimpleAggregateFunction(max, UInt8), ai_tags_fixed SimpleAggregateFunction(groupUniqArrayArray, Array(String)), ai_tags_freeform SimpleAggregateFunction(groupUniqArrayArray, Array(String)), ai_highlighted SimpleAggregateFunction(max, UInt8), surfacing_score SimpleAggregateFunction(max, Nullable(Float32))) AS SELECTsession_id,team_id,@@ -7335,16 +7538,14 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -8125,16 +8326,14 @@ CREATE VIEW posthog.raw_sessions_v3_v AS SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/sql/prod-eu/data.sql b/posthog/clickhouse/hcl/sql/prod-eu/data.sqlindex ce33ba61df44..491415a21438 100644--- a/posthog/clickhouse/hcl/sql/prod-eu/data.sql+++ b/posthog/clickhouse/hcl/sql/prod-eu/data.sql@@ -1181,17 +1181,14 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -1650,24 +1647,20 @@ CREATE TABLE posthog.sharded_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000,INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/{shard}/posthog.raw_sessions_v3', '{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192;@@ -2475,17 +2468,14 @@ CREATE TABLE posthog.writable_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),@@ -3080,16 +3070,14 @@ CREATE VIEW posthog.raw_sessions_v3_v AS SELECTargMinMerge(entry_fbclid) AS entry_fbclid,argMinMerge(entry_has_gclid) AS entry_has_gclid,argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,max(has_autocapture) AS has_autocapture,groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,groupUniqArrayArray(2000)(event_names) AS event_names,groupUniqArrayArray(100)(hosts) AS hosts,groupUniqArrayArray(10)(emails) AS emails,diff --git a/posthog/clickhouse/hcl/sql/prod-eu/sessions.sql b/posthog/clickhouse/hcl/sql/prod-eu/sessions.sqlindex 085954cd688b..315ecb4af203 100644--- a/posthog/clickhouse/hcl/sql/prod-eu/sessions.sql+++ b/posthog/clickhouse/hcl/sql/prod-eu/sessions.sql@@ -320,24 +320,20 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3', '{shard}-{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192, storage_policy = 's3_tiered';CREATE TABLE posthog.sessions (session_id String,diff --git a/posthog/clickhouse/hcl/sql/prod-us/data.sql b/posthog/clickhouse/hcl/sql/prod-us/data.sqlindex 1ee516670807..b1acd23b5ffb 100644--- a/posthog/clickhouse/hcl/sql/prod-us/data.sql+++ b/posthog/clickhouse/hcl/sql/prod-us/data.sql@@ -1014,6 +1014,55 @@ CREATE TABLE posthog.raw_sessions (page_screen_autocapture_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),vitals_lcp AggregateFunction(argMin, Nullable(Float64), DateTime64(6, 'UTC'))) ENGINE = Distributed('sessions', 'posthog', 'raw_sessions', cityHash64(session_id_v7));+CREATE TABLE posthog.raw_sessions_v3 (+ team_id Int64,+ session_id_v7 UInt128,+ session_timestamp DateTime64(3) MATERIALIZED fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))),+ distinct_id AggregateFunction(argMax, String, DateTime64(6, 'UTC')),+ distinct_ids AggregateFunction(groupUniqArray, String),+ min_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')),+ max_timestamp SimpleAggregateFunction(max, DateTime64(6, 'UTC')),+ max_inserted_at SimpleAggregateFunction(max, DateTime64(6, 'UTC')),+ urls SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),+ entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ end_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),+ last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),+ browser AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ os AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ os_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ device_type AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),+ viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),+ geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),+ entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),+ entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),+ entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),+ page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),+ has_autocapture SimpleAggregateFunction(max, Bool),+ flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),+ event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),+ hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),+ emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),+ has_replay_events SimpleAggregateFunction(max, Bool)+) ENGINE = Distributed('sessions', 'posthog', 'raw_sessions_v3', cityHash64(session_id_v7));CREATE TABLE posthog.session_recording_events (uuid UUID,timestamp DateTime64(6, 'UTC'),@@ -1499,24 +1548,20 @@ CREATE TABLE posthog.sharded_raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000,INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/reshard/{shard}/posthog.raw_sessions_v3', '{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192;@@ -2290,6 +2335,55 @@ CREATE TABLE posthog.writable_raw_sessions_primary (vitals_lcp AggregateFunction(argMin, Nullable(Float64), DateTime64(6, 'UTC')),initial__kx AggregateFunction(argMin, String, DateTime64(6, 'UTC'))) ENGINE = Distributed('posthog', 'posthog', 'sharded_raw_sessions', cityHash64(session_id_v7));+CREATE TABLE posthog.writable_raw_sessions_v3 (+ team_id Int64,+ session_id_v7 UInt128,+ session_timestamp DateTime64(3) MATERIALIZED fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))),+ distinct_id AggregateFunction(argMax, String, DateTime64(6, 'UTC')),+ distinct_ids AggregateFunction(groupUniqArray, String),+ min_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')),+ max_timestamp SimpleAggregateFunction(max, DateTime64(6, 'UTC')),+ max_inserted_at SimpleAggregateFunction(max, DateTime64(6, 'UTC')),+ urls SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),+ entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ end_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),+ last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),+ browser AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ os AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ os_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ device_type AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),+ viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),+ geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),+ entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),+ entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),+ entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),+ entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),+ page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),+ has_autocapture SimpleAggregateFunction(max, Bool),+ flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),+ event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),+ hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),+ emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),+ has_replay_events SimpleAggregateFunction(max, Bool)+) ENGINE = Distributed('sessions', 'posthog', 'raw_sessions_v3', cityHash64(session_id_v7));CREATE TABLE posthog.writable_session_recording_events (uuid UUID,timestamp DateTime64(6, 'UTC'),@@ -2843,6 +2937,57 @@ CREATE VIEW posthog.raw_sessions_v AS SELECTFROM posthog.raw_sessionsGROUP BYsession_id_v7, team_id;+CREATE VIEW posthog.raw_sessions_v3_v AS SELECT+ session_id_v7,+ session_timestamp,+ team_id,+ argMaxMerge(distinct_id) AS distinct_id,+ groupUniqArrayMerge(distinct_ids) AS distinct_ids,+ min(min_timestamp) AS min_timestamp,+ max(max_timestamp) AS max_timestamp,+ max(max_inserted_at) AS max_inserted_at,+ groupUniqArrayArray(2000)(urls) AS urls,+ argMinMerge(entry_url) AS entry_url,+ argMaxMerge(end_url) AS end_url,+ argMaxMerge(last_external_click_url) AS last_external_click_url,+ argMinMerge(browser) AS browser,+ argMinMerge(browser_version) AS browser_version,+ argMinMerge(os) AS os,+ argMinMerge(os_version) AS os_version,+ argMinMerge(device_type) AS device_type,+ argMinMerge(viewport_width) AS viewport_width,+ argMinMerge(viewport_height) AS viewport_height,+ argMinMerge(geoip_country_code) AS geoip_country_code,+ argMinMerge(geoip_subdivision_1_code) AS geoip_subdivision_1_code,+ argMinMerge(geoip_subdivision_1_name) AS geoip_subdivision_1_name,+ argMinMerge(geoip_subdivision_city_name) AS geoip_subdivision_city_name,+ argMinMerge(geoip_time_zone) AS geoip_time_zone,+ argMinMerge(entry_utm_source) AS entry_utm_source,+ argMinMerge(entry_utm_campaign) AS entry_utm_campaign,+ argMinMerge(entry_utm_medium) AS entry_utm_medium,+ argMinMerge(entry_utm_term) AS entry_utm_term,+ argMinMerge(entry_utm_content) AS entry_utm_content,+ argMinMerge(entry_referring_domain) AS entry_referring_domain,+ argMinMerge(entry_gclid) AS entry_gclid,+ argMinMerge(entry_gad_source) AS entry_gad_source,+ argMinMerge(entry_fbclid) AS entry_fbclid,+ argMinMerge(entry_has_gclid) AS entry_has_gclid,+ argMinMerge(entry_has_fbclid) AS entry_has_fbclid,+ argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,+ argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,+ uniqMerge(pageview_uniq) AS pageview_uniq,+ uniqMerge(autocapture_uniq) AS autocapture_uniq,+ uniqMerge(screen_uniq) AS screen_uniq,+ uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,+ max(has_autocapture) AS has_autocapture,+ groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,+ groupUniqArrayArray(2000)(event_names) AS event_names,+ groupUniqArrayArray(100)(hosts) AS hosts,+ groupUniqArrayArray(10)(emails) AS emails,+ max(has_replay_events) AS has_replay_events+FROM posthog.raw_sessions_v3+GROUP BY+ session_id_v7, session_timestamp, team_id;CREATE VIEW posthog.sessions_v AS SELECTsession_id,team_id,@@ -3175,58 +3320,6 @@ CREATE TABLE posthog.preaggregation_results (breakdown_value Array(String),uniq_exact_state AggregateFunction(uniqExact, UUID)) ENGINE = Distributed('posthog', 'posthog', 'sharded_preaggregation_results', sipHash64(job_id));-CREATE TABLE posthog.raw_sessions_v3 (- team_id Int64,- session_id_v7 UInt128,- session_timestamp DateTime64(3) MATERIALIZED fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))),- distinct_id AggregateFunction(argMax, String, DateTime64(6, 'UTC')),- distinct_ids AggregateFunction(groupUniqArray, String),- min_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')),- max_timestamp SimpleAggregateFunction(max, DateTime64(6, 'UTC')),- max_inserted_at SimpleAggregateFunction(max, DateTime64(6, 'UTC')),- urls SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),- entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- end_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),- last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),- browser AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- os AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- os_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- device_type AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),- viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),- geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),- entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),- entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),- page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),- has_autocapture SimpleAggregateFunction(max, Bool),- flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),- event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),- hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),- emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),- has_replay_events SimpleAggregateFunction(max, Bool)-) ENGINE = Distributed('posthog', 'posthog', 'sharded_raw_sessions_v3', cityHash64(session_id_v7));CREATE TABLE posthog.session_replay_embeddings (session_id String,team_id Int64,@@ -3267,58 +3360,6 @@ CREATE TABLE posthog.session_replay_events (surfacing_score SimpleAggregateFunction(max, Nullable(Float32)),snapshot_mode_v2 AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC'))) ENGINE = Distributed('posthog', 'posthog', 'sharded_session_replay_events', sipHash64(distinct_id));-CREATE TABLE posthog.writable_raw_sessions_v3 (- team_id Int64,- session_id_v7 UInt128,- session_timestamp DateTime64(3) MATERIALIZED fromUnixTimestamp64Milli(toUInt64(bitShiftRight(session_id_v7, 80))),- distinct_id AggregateFunction(argMax, String, DateTime64(6, 'UTC')),- distinct_ids AggregateFunction(groupUniqArray, String),- min_timestamp SimpleAggregateFunction(min, DateTime64(6, 'UTC')),- max_timestamp SimpleAggregateFunction(max, DateTime64(6, 'UTC')),- max_inserted_at SimpleAggregateFunction(max, DateTime64(6, 'UTC')),- urls SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),- entry_url AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- end_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),- last_external_click_url AggregateFunction(argMax, Nullable(String), DateTime64(6, 'UTC')),- browser AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- browser_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- os AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- os_version AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- device_type AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- viewport_width AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),- viewport_height AggregateFunction(argMin, Nullable(Int64), DateTime64(6, 'UTC')),- geoip_country_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_subdivision_1_code AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_subdivision_1_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_subdivision_city_name AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- geoip_time_zone AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_referring_domain AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_campaign AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_medium AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_term AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_utm_content AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_gclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_gad_source AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),- entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),- entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),- entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),- page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),- has_autocapture SimpleAggregateFunction(max, Bool),- flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),- event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),- hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),- emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),- has_replay_events SimpleAggregateFunction(max, Bool)-) ENGINE = Distributed('sessions', 'posthog', 'raw_sessions_v3', cityHash64(session_id_v7));CREATE VIEW posthog.custom_metrics AS SELECT * REPLACE(toFloat64(value) AS value)FROM posthog.custom_metrics_testUNION ALL@@ -3494,59 +3535,6 @@ SELECT_partition,'text-embedding-3-large-3072' AS model_nameFROM posthog.distributed_posthog_document_embeddings_text_embedding_3_large_3072;-CREATE VIEW posthog.raw_sessions_v3_v AS SELECT- session_id_v7,- session_timestamp,- team_id,- argMaxMerge(distinct_id) AS distinct_id,- groupUniqArrayMerge(distinct_ids) AS distinct_ids,- min(min_timestamp) AS min_timestamp,- max(max_timestamp) AS max_timestamp,- max(max_inserted_at) AS max_inserted_at,- groupUniqArrayArray(2000)(urls) AS urls,- argMinMerge(entry_url) AS entry_url,- argMaxMerge(end_url) AS end_url,- argMaxMerge(last_external_click_url) AS last_external_click_url,- argMinMerge(browser) AS browser,- argMinMerge(browser_version) AS browser_version,- argMinMerge(os) AS os,- argMinMerge(os_version) AS os_version,- argMinMerge(device_type) AS device_type,- argMinMerge(viewport_width) AS viewport_width,- argMinMerge(viewport_height) AS viewport_height,- argMinMerge(geoip_country_code) AS geoip_country_code,- argMinMerge(geoip_subdivision_1_code) AS geoip_subdivision_1_code,- argMinMerge(geoip_subdivision_1_name) AS geoip_subdivision_1_name,- argMinMerge(geoip_subdivision_city_name) AS geoip_subdivision_city_name,- argMinMerge(geoip_time_zone) AS geoip_time_zone,- argMinMerge(entry_utm_source) AS entry_utm_source,- argMinMerge(entry_utm_campaign) AS entry_utm_campaign,- argMinMerge(entry_utm_medium) AS entry_utm_medium,- argMinMerge(entry_utm_term) AS entry_utm_term,- argMinMerge(entry_utm_content) AS entry_utm_content,- argMinMerge(entry_referring_domain) AS entry_referring_domain,- argMinMerge(entry_gclid) AS entry_gclid,- argMinMerge(entry_gad_source) AS entry_gad_source,- argMinMerge(entry_fbclid) AS entry_fbclid,- argMinMerge(entry_has_gclid) AS entry_has_gclid,- argMinMerge(entry_has_fbclid) AS entry_has_fbclid,- argMinMerge(entry_ad_ids_map) AS entry_ad_ids_map,- argMinMerge(entry_ad_ids_set) AS entry_ad_ids_set,- argMinMerge(entry_channel_type_properties) AS entry_channel_type_properties,- uniqExactMerge(pageview_uniq) AS pageview_uniq,- uniqExactMerge(autocapture_uniq) AS autocapture_uniq,- uniqExactMerge(screen_uniq) AS screen_uniq,- uniqUpToMerge(1)(page_screen_uniq_up_to) AS page_screen_uniq_up_to,- max(has_autocapture) AS has_autocapture,- groupUniqArrayArray(10000)(flag_key_values) AS flag_key_values,- groupUniqArrayArray(flag_keys) AS flag_keys,- groupUniqArrayArray(2000)(event_names) AS event_names,- groupUniqArrayArray(100)(hosts) AS hosts,- groupUniqArrayArray(10)(emails) AS emails,- max(has_replay_events) AS has_replay_events-FROM posthog.raw_sessions_v3-GROUP BY- session_id_v7, session_timestamp, team_id;CREATE VIEW posthog.team_events_last_month_view AS SELECT team_id, count() AS event_countFROM posthog.eventsWHEREdiff --git a/posthog/clickhouse/hcl/sql/prod-us/sessions.sql b/posthog/clickhouse/hcl/sql/prod-us/sessions.sqlindex 59c8f4370c4a..73452c2a55bf 100644--- a/posthog/clickhouse/hcl/sql/prod-us/sessions.sql+++ b/posthog/clickhouse/hcl/sql/prod-us/sessions.sql@@ -320,24 +320,20 @@ CREATE TABLE posthog.raw_sessions_v3 (entry_fbclid AggregateFunction(argMin, Nullable(String), DateTime64(6, 'UTC')),entry_has_gclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Bool, DateTime64(6, 'UTC')),- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),entry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Bool, Bool, Nullable(String)), DateTime64(6, 'UTC')),- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),page_screen_uniq_up_to AggregateFunction(uniqUpTo(1), Nullable(UUID)),has_autocapture SimpleAggregateFunction(max, Bool),flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_values AggregateFunction(groupUniqArrayMap, Map(String, String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)),hosts SimpleAggregateFunction(groupUniqArrayArray(100), Array(String)),emails SimpleAggregateFunction(groupUniqArrayArray(10), Array(String)),has_replay_events SimpleAggregateFunction(max, Bool),INDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])) GRANULARITY 100000000) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/sessions/noshard/posthog.raw_sessions_v3', '{shard}-{replica}') ORDER BY (team_id, session_timestamp, session_id_v7) PARTITION BY toYYYYMM(session_timestamp) SETTINGS index_granularity = 8192, max_replicated_merges_in_queue = 5, parts_to_delay_insert = 1500, parts_to_throw_insert = 2000, prefer_fetch_merged_part_size_threshold = 1, prefer_fetch_merged_part_time_threshold = 0, replicated_deduplication_window = 0, replicated_deduplication_window_for_async_inserts = 0, storage_policy = 's3_tiered';CREATE TABLE posthog.sessions (session_id String,diff --git a/posthog/clickhouse/migrations/0325_sessions_v3_flag_key_values.py b/posthog/clickhouse/migrations/0325_sessions_v3_flag_key_values.pyindex 3a1244654224..ef6be6907211 100644--- a/posthog/clickhouse/migrations/0325_sessions_v3_flag_key_values.py+++ b/posthog/clickhouse/migrations/0325_sessions_v3_flag_key_values.py@@ -10,7 +10,6 @@ from posthog.models.raw_sessions.migrations_v3 import ()from posthog.models.raw_sessions.sessions_v3 import (DISTRIBUTED_RAW_SESSIONS_TABLE_V3,- RAW_SESSIONS_CREATE_OR_REPLACE_VIEW_SQL_V3,SHARDED_RAW_SESSIONS_TABLE_V3,TABLE_BASE_NAME_V3,WRITABLE_RAW_SESSIONS_TABLE_V3,@@ -42,8 +41,7 @@ operations = [sharded=True,is_alter_on_replicated_table=True,),- # hosts and emails were only ever added by hand, so the main-cluster copies never got- # them; the recreated view below selects both, so the columns must exist first.+ # hosts and emails were only ever added by hand, so the main-cluster copies never got them.run_sql_with_exceptions(ADD_HOSTS.format(table_name=SHARDED_RAW_SESSIONS_TABLE_V3()),node_roles=[NodeRole.DATA],@@ -92,10 +90,6 @@ operations = [sharded=True,is_alter_on_replicated_table=True,),- run_sql_with_exceptions(- RAW_SESSIONS_CREATE_OR_REPLACE_VIEW_SQL_V3(),- node_roles=[NodeRole.DATA],- ),]# In cloud envs the live copy is the single-shard raw_sessions_v3 MergeTree on the sessionsdiff --git a/posthog/clickhouse/migrations/0364_sessions_v3_more_ad_ids.py b/posthog/clickhouse/migrations/0364_sessions_v3_more_ad_ids.pynew file mode 100644index 000000000000..893fe51b2995--- /dev/null+++ b/posthog/clickhouse/migrations/0364_sessions_v3_more_ad_ids.py@@ -0,0 +1,77 @@+from posthog.clickhouse.client.connection import NodeRole+from posthog.clickhouse.client.migration_tools import run_sql_with_exceptions+from posthog.models.raw_sessions.migrations_v3 import (+ DROP_FLAG_KEY_VALUES_BLOOM_FILTER,+ DROP_FLAG_KEYS,+ DROP_FLAG_KEYS_BLOOM_FILTER,+)+from posthog.models.raw_sessions.sessions_v3 import (+ DISTRIBUTED_RAW_SESSIONS_TABLE_V3,+ RAW_SESSIONS_TABLE_MV_RECORDINGS_SQL_V3,+ RAW_SESSIONS_TABLE_MV_SQL_V3,+ SHARDED_RAW_SESSIONS_TABLE_V3,+ WRITABLE_RAW_SESSIONS_TABLE_V3,+)+from posthog.run_mode import run_mode++# raw_sessions_v3 has a new shape: no entry_ad_ids_map or flag_keys, the ad-ids set holds+# 'key=value' pairs, and the counts are uniq states instead of uniqExact. A shape change cannot be+# applied with ALTER, so an existing v3 table has to be rebuilt with a backfill.+#+# Cloud is applied by hand by the ClickHouse team, because the rebuild needs a backfill and+# dropping the ingestion MV stops v3 ingestion. As of 2026-10-28, prod-us is done; dev and EU are+# still to do. The declared cloud schema is the HCL in posthog/clickhouse/hcl/.+#+# Everywhere else (fresh installs, CI, the test database) this migration is required: the earlier+# v3 migrations render the current table and view definitions, but 0178 dropped the MVs, and this+# is the only migration that creates them again. Both statements are CREATE ... IF NOT EXISTS, so+# an existing MV is kept. An install that created v3 before the shape change keeps the old shape.+# To use sessionTableVersion v3 there, drop raw_sessions_v3_mv, raw_sessions_v3_recordings_mv,+# raw_sessions_v3_v, raw_sessions_v3, writable_raw_sessions_v3 and sharded_raw_sessions_v3, then+# create them again from the definitions in posthog/models/raw_sessions/sessions_v3.py.+#+# The drops come first. 0180 and 0325 replay on a fresh cluster and put flag_keys and the two+# bloom filter indexes back on top of today's base SQL, which no longer declares them, so without+# these statements a fresh install ends up with a table the declared HCL does not describe.+DROP_SUPERSEDED_FLAG_COLUMNS = [+ run_sql_with_exceptions(+ DROP_FLAG_KEYS_BLOOM_FILTER.format(table_name=SHARDED_RAW_SESSIONS_TABLE_V3()),+ node_roles=[NodeRole.DATA],+ sharded=True,+ is_alter_on_replicated_table=True,+ ),+ run_sql_with_exceptions(+ DROP_FLAG_KEY_VALUES_BLOOM_FILTER.format(table_name=SHARDED_RAW_SESSIONS_TABLE_V3()),+ node_roles=[NodeRole.DATA],+ sharded=True,+ is_alter_on_replicated_table=True,+ ),+ run_sql_with_exceptions(+ DROP_FLAG_KEYS.format(table_name=SHARDED_RAW_SESSIONS_TABLE_V3()),+ node_roles=[NodeRole.DATA],+ sharded=True,+ is_alter_on_replicated_table=True,+ ),+ run_sql_with_exceptions(+ DROP_FLAG_KEYS.format(table_name=WRITABLE_RAW_SESSIONS_TABLE_V3()),+ node_roles=[NodeRole.DATA],+ sharded=False,+ is_alter_on_replicated_table=False,+ ),+ run_sql_with_exceptions(+ DROP_FLAG_KEYS.format(table_name=DISTRIBUTED_RAW_SESSIONS_TABLE_V3()),+ node_roles=[NodeRole.DATA],+ sharded=False,+ is_alter_on_replicated_table=False,+ ),+]++operations = (+ []+ if run_mode().is_deployed_cloud+ else [+ *DROP_SUPERSEDED_FLAG_COLUMNS,+ run_sql_with_exceptions(RAW_SESSIONS_TABLE_MV_SQL_V3(), node_roles=[NodeRole.DATA]),+ run_sql_with_exceptions(RAW_SESSIONS_TABLE_MV_RECORDINGS_SQL_V3(), node_roles=[NodeRole.DATA]),+ ]+)diff --git a/posthog/clickhouse/migrations/max_migration.txt b/posthog/clickhouse/migrations/max_migration.txtindex 452e42cade07..7432923099bf 100644--- a/posthog/clickhouse/migrations/max_migration.txt+++ b/posthog/clickhouse/migrations/max_migration.txt@@ -1 +1 @@-0363_email_status+0364_sessions_v3_more_ad_idsdiff --git a/posthog/clickhouse/test/__snapshots__/test_raw_sessions_v3_model.ambr b/posthog/clickhouse/test/__snapshots__/test_raw_sessions_v3_model.ambrindex 18be1c646860..c8c3c2419ed1 100644--- a/posthog/clickhouse/test/__snapshots__/test_raw_sessions_v3_model.ambr+++ b/posthog/clickhouse/test/__snapshots__/test_raw_sessions_v3_model.ambr@@ -1,5 +1,5 @@# serializer version: 1-# name: TestRawSessionsModel.test_ad_ids_map_and_set+# name: TestRawSessionsModel.test_ad_id_property_vs_url_param_precedence_0_property_wins'''select *@@ -8,7 +8,88 @@AND team_id = 99999'''# ----# name: TestRawSessionsModel.test_autocapture_does_not_set_attribution_when_pageview_present+# name: TestRawSessionsModel.test_ad_id_property_vs_url_param_precedence_1_empty_property_falls_back+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_id_url_param_decodes_plus_as_space+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_id_url_param_with_undecodable_bytes_is_normalized+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_fall_back_to_url_params_0_gad_campaignid+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_fall_back_to_url_params_1_utm_id+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_fall_back_to_url_params_2_irclickid+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_fall_back_to_url_params_3_snapchat_canonical+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_fall_back_to_url_params_4_snapchat_sccid_cased+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_fall_back_to_url_params_5_snapchat_sscid_cased+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_ad_ids_set_collects_kv_pairs'''select *@@ -26,7 +107,16 @@AND team_id = 99999'''# ----# name: TestRawSessionsModel.test_channel_type_properties+# name: TestRawSessionsModel.test_channel_type_properties_0_click_ids_present+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_channel_type_properties_1_empty_click_ids_are_not_present'''select *@@ -44,7 +134,7 @@AND team_id = 99999'''# ----# name: TestRawSessionsModel.test_emails_are_collected+# name: TestRawSessionsModel.test_drops_events_far_from_the_session_id_timestamp_0_garbage_session_timestamp'''select *@@ -53,7 +143,7 @@AND team_id = 99999'''# ----# name: TestRawSessionsModel.test_empty_hosts_and_emails+# name: TestRawSessionsModel.test_drops_events_far_from_the_session_id_timestamp_1_session_id_over_31_days_after_event'''select *@@ -62,7 +152,25 @@AND team_id = 99999'''# ----# name: TestRawSessionsModel.test_event_names_are_collected+# name: TestRawSessionsModel.test_drops_events_far_from_the_session_id_timestamp_2_client_clock_days_ahead+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_drops_events_far_from_the_session_id_timestamp_3_late_event_within_31_days+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_emails_are_collected'''select *@@ -71,7 +179,16 @@AND team_id = 99999'''# ----# name: TestRawSessionsModel.test_flag_keys_are_collected+# name: TestRawSessionsModel.test_empty_hosts_and_emails+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_event_names_are_collected'''select *@@ -184,6 +301,42 @@AND team_id = 99999'''# ---+# name: TestRawSessionsModel.test_non_pageview_event_does_not_set_attribution_when_pageview_present_0_autocapture_first+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_non_pageview_event_does_not_set_attribution_when_pageview_present_1_pageleave_first+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_non_pageview_event_does_not_set_attribution_when_pageview_present_2_custom_event_first+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---+# name: TestRawSessionsModel.test_non_pageview_event_does_not_set_attribution_when_pageview_present_3_pageview_inserted_before_earlier_event+ '''++ select *+ from raw_sessions_v3_v+ where session_id_v7 = toUInt128(toUUID('00000000-0000-0000-0000-000000000000'))+ AND team_id = 99999+ '''+# ---# name: TestRawSessionsModel.test_select_from_sessions'''diff --git a/posthog/clickhouse/test/__snapshots__/test_schema.ambr b/posthog/clickhouse/test/__snapshots__/test_schema.ambrindex e27b3b78382e..7c927ce9fc14 100644--- a/posthog/clickhouse/test/__snapshots__/test_schema.ambr+++ b/posthog/clickhouse/test/__snapshots__/test_schema.ambr@@ -6954,8 +6954,8 @@entry_has_gclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),- -- for lower-tier ad ids, just put them in a map, and set of the ones present- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),+ -- lower-tier ad ids as a flat 'key=value' array - like flag_key_values, a per-key map+ -- state made merges much more expensiveentry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),-- channel type properties tuple - to reduce redundant reading of the timestamp when loading all of these columns@@ -6963,12 +6963,13 @@entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Boolean, Boolean, Nullable(String)), DateTime64(6, 'UTC')),-- Count pageview, autocapture, and screen events for providing totals.- -- Use uniqExact instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see+ -- Use uniq instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see-- events being inserted multiple times to be deduped later, but that can trigger multiple rows here.-- Additionally, idempotency is useful for backfilling, as we can just reinsert the same events without worrying.- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ -- uniq (not uniqExact) keeps the state small; per-session counts are tiny so it is exact in practice.+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),-- As a performance optimisation, also keep track of the uniq events for all of these combined.-- This is a much more efficient way of calculating the bounce rate, as >2 means not a bounce@@ -6977,8 +6978,8 @@-- Flags - store every seen 'key=value' pair per flag. A flat array instead of a-- groupUniqArrayMap state because per-key sub-aggregators made merges ~8-37x more expensive.+ -- Key-only searches use the text index below (tokenized on '='), so no separate keys column.flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),-- Event names - store unique event names seen in this sessionevent_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),@@ -7017,7 +7018,7 @@`$geoip_country_code` Nullable(String),`$geoip_subdivision_1_code` Nullable(String),`$geoip_subdivision_1_name` Nullable(String),- `$geoip_subdivision_city_name` Nullable(String),+ `$geoip_city_name` Nullable(String),`$geoip_time_zone` Nullable(String),`$referring_domain` Nullable(String),`utm_source` Nullable(String),@@ -7033,20 +7034,40 @@`dclid` Nullable(String),`gbraid` Nullable(String),`wbraid` Nullable(String),+ `gad_campaignid` Nullable(String),+ `srsltid` Nullable(String),`msclkid` Nullable(String),`twclid` Nullable(String),`li_fat_id` Nullable(String),`mc_cid` Nullable(String),`igshid` Nullable(String),`ttclid` Nullable(String),+ `rdt_cid` Nullable(String),`epik` Nullable(String),`qclid` Nullable(String),`sccid` Nullable(String),`_kx` Nullable(String),- `irclid` Nullable(String)+ `irclid` Nullable(String),+ `irclickid` Nullable(String),+ `utm_id` Nullable(String),+ `utm_source_platform` Nullable(String),+ `utm_creative_format` Nullable(String),+ `utm_marketing_tactic` Nullable(String),+ `yclid` Nullable(String),+ `tblci` Nullable(String),+ `dicbo` Nullable(String),+ `ndclid` Nullable(String),+ `cjevent` Nullable(String),+ `awc` Nullable(String),+ `syclid` Nullable(String),+ `_bhlid` Nullable(String),+ `rtid` Nullable(String),+ `oppref` Nullable(String),+ `bfcid` Nullable(String))') as p,JSONExtractString(person_properties, 'email') as _person_email,tupleElement(p, '$current_url') as _current_url,+ if(event = '$pageview' OR event = '$screen', left(coalesce(_current_url, ''), 16384), '') as _attribution_url,tupleElement(p, '$external_click_url') as _external_click_url,tupleElement(p, '$browser') as _browser,tupleElement(p, '$browser_version') as _browser_version,@@ -7058,7 +7079,7 @@tupleElement(p, '$geoip_country_code') as _geoip_country_code,tupleElement(p, '$geoip_subdivision_1_code') as _geoip_subdivision_1_code,tupleElement(p, '$geoip_subdivision_1_name') as _geoip_subdivision_1_name,- tupleElement(p, '$geoip_subdivision_city_name') as _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_city_name') as _geoip_subdivision_city_name,tupleElement(p, '$geoip_time_zone') as _geoip_time_zone,tupleElement(p, '$referring_domain') as _referring_domain,tupleElement(p, 'utm_source') as _utm_source,@@ -7069,54 +7090,75 @@tupleElement(p, 'gclid') as _gclid,tupleElement(p, 'gad_source') as _gad_source,tupleElement(p, 'fbclid') as _fbclid,- tupleElement(p, 'gclsrc') as gclsrc,- tupleElement(p, 'dclid') as dclid,- tupleElement(p, 'gbraid') as gbraid,- tupleElement(p, 'wbraid') as wbraid,- tupleElement(p, 'msclkid') as msclkid,- tupleElement(p, 'twclid') as twclid,- tupleElement(p, 'li_fat_id') as li_fat_id,- tupleElement(p, 'mc_cid') as mc_cid,- tupleElement(p, 'igshid') as igshid,- tupleElement(p, 'ttclid') as ttclid,- tupleElement(p, 'epik') as epik,- tupleElement(p, 'qclid') as qclid,- tupleElement(p, 'sccid') as sccid,- tupleElement(p, '_kx') as _kx,- tupleElement(p, 'irclid') as irclid,- CAST(mapFilter((k, v) -> v IS NOT NULL, map(- 'gclsrc', gclsrc,- 'dclid', dclid,- 'gbraid', gbraid,- 'wbraid', wbraid,- 'msclkid', msclkid,- 'twclid', twclid,- 'li_fat_id', li_fat_id,- 'mc_cid', mc_cid,- 'igshid', igshid,- 'ttclid', ttclid,- 'epik', epik,- 'qclid', qclid,- 'sccid', sccid,- '_kx', _kx,- 'irclid', irclid- )) AS Map(String, String)) as ad_ids_map,+ coalesce(nullIf(tupleElement(p, 'gclsrc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gclsrc')), 2048)), '')) as gclsrc,+ coalesce(nullIf(tupleElement(p, 'dclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dclid')), 2048)), '')) as dclid,+ coalesce(nullIf(tupleElement(p, 'gbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gbraid')), 2048)), '')) as gbraid,+ coalesce(nullIf(tupleElement(p, 'wbraid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'wbraid')), 2048)), '')) as wbraid,+ coalesce(nullIf(tupleElement(p, 'gad_campaignid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'gad_campaignid')), 2048)), '')) as gad_campaignid,+ coalesce(nullIf(tupleElement(p, 'srsltid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'srsltid')), 2048)), '')) as srsltid,+ coalesce(nullIf(tupleElement(p, 'msclkid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'msclkid')), 2048)), '')) as msclkid,+ coalesce(nullIf(tupleElement(p, 'twclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'twclid')), 2048)), '')) as twclid,+ coalesce(nullIf(tupleElement(p, 'li_fat_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'li_fat_id')), 2048)), '')) as li_fat_id,+ coalesce(nullIf(tupleElement(p, 'mc_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'mc_cid')), 2048)), '')) as mc_cid,+ coalesce(nullIf(tupleElement(p, 'igshid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'igshid')), 2048)), '')) as igshid,+ coalesce(nullIf(tupleElement(p, 'ttclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ttclid')), 2048)), '')) as ttclid,+ coalesce(nullIf(tupleElement(p, 'rdt_cid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rdt_cid')), 2048)), '')) as rdt_cid,+ coalesce(nullIf(tupleElement(p, 'epik'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'epik')), 2048)), '')) as epik,+ coalesce(nullIf(tupleElement(p, 'qclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'qclid')), 2048)), '')) as qclid,+ coalesce(nullIf(tupleElement(p, 'sccid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'sccid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ScCid')), 2048)), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'SsCid')), 2048)), '')) as sccid,+ coalesce(nullIf(tupleElement(p, '_kx'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_kx')), 2048)), '')) as _kx,+ coalesce(nullIf(tupleElement(p, 'irclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclid')), 2048)), '')) as irclid,+ coalesce(nullIf(tupleElement(p, 'irclickid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'irclickid')), 2048)), '')) as irclickid,+ coalesce(nullIf(tupleElement(p, 'utm_id'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_id')), 2048)), '')) as utm_id,+ coalesce(nullIf(tupleElement(p, 'utm_source_platform'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_source_platform')), 2048)), '')) as utm_source_platform,+ coalesce(nullIf(tupleElement(p, 'utm_creative_format'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_creative_format')), 2048)), '')) as utm_creative_format,+ coalesce(nullIf(tupleElement(p, 'utm_marketing_tactic'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'utm_marketing_tactic')), 2048)), '')) as utm_marketing_tactic,+ coalesce(nullIf(tupleElement(p, 'yclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'yclid')), 2048)), '')) as yclid,+ coalesce(nullIf(tupleElement(p, 'tblci'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'tblci')), 2048)), '')) as tblci,+ coalesce(nullIf(tupleElement(p, 'dicbo'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'dicbo')), 2048)), '')) as dicbo,+ coalesce(nullIf(tupleElement(p, 'ndclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'ndclid')), 2048)), '')) as ndclid,+ coalesce(nullIf(tupleElement(p, 'cjevent'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'cjevent')), 2048)), '')) as cjevent,+ coalesce(nullIf(tupleElement(p, 'awc'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'awc')), 2048)), '')) as awc,+ coalesce(nullIf(tupleElement(p, 'syclid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'syclid')), 2048)), '')) as syclid,+ coalesce(nullIf(tupleElement(p, '_bhlid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '_bhlid')), 2048)), '')) as _bhlid,+ coalesce(nullIf(tupleElement(p, 'rtid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'rtid')), 2048)), '')) as rtid,+ coalesce(nullIf(tupleElement(p, 'oppref'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'oppref')), 2048)), '')) as oppref,+ coalesce(nullIf(tupleElement(p, 'bfcid'), ''), nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, 'bfcid')), 2048)), '')) as bfcid,CAST(arrayFilter(x -> x IS NOT NULL, [- if(gclsrc IS NOT NULL, 'gclsrc', NULL),- if(dclid IS NOT NULL, 'dclid', NULL),- if(gbraid IS NOT NULL, 'gbraid', NULL),- if(wbraid IS NOT NULL, 'wbraid', NULL),- if(msclkid IS NOT NULL, 'msclkid', NULL),- if(twclid IS NOT NULL, 'twclid', NULL),- if(li_fat_id IS NOT NULL, 'li_fat_id', NULL),- if(mc_cid IS NOT NULL, 'mc_cid', NULL),- if(igshid IS NOT NULL, 'igshid', NULL),- if(ttclid IS NOT NULL, 'ttclid', NULL),- if(epik IS NOT NULL, 'epik', NULL),- if(qclid IS NOT NULL, 'qclid', NULL),- if(sccid IS NOT NULL, 'sccid', NULL),- if(_kx IS NOT NULL, '_kx', NULL),- if(irclid IS NOT NULL, 'irclid', NULL)+ if(gclsrc IS NOT NULL, concat('gclsrc=', gclsrc), NULL),+ if(dclid IS NOT NULL, concat('dclid=', dclid), NULL),+ if(gbraid IS NOT NULL, concat('gbraid=', gbraid), NULL),+ if(wbraid IS NOT NULL, concat('wbraid=', wbraid), NULL),+ if(gad_campaignid IS NOT NULL, concat('gad_campaignid=', gad_campaignid), NULL),+ if(srsltid IS NOT NULL, concat('srsltid=', srsltid), NULL),+ if(msclkid IS NOT NULL, concat('msclkid=', msclkid), NULL),+ if(twclid IS NOT NULL, concat('twclid=', twclid), NULL),+ if(li_fat_id IS NOT NULL, concat('li_fat_id=', li_fat_id), NULL),+ if(mc_cid IS NOT NULL, concat('mc_cid=', mc_cid), NULL),+ if(igshid IS NOT NULL, concat('igshid=', igshid), NULL),+ if(ttclid IS NOT NULL, concat('ttclid=', ttclid), NULL),+ if(rdt_cid IS NOT NULL, concat('rdt_cid=', rdt_cid), NULL),+ if(epik IS NOT NULL, concat('epik=', epik), NULL),+ if(qclid IS NOT NULL, concat('qclid=', qclid), NULL),+ if(sccid IS NOT NULL, concat('sccid=', sccid), NULL),+ if(_kx IS NOT NULL, concat('_kx=', _kx), NULL),+ if(irclid IS NOT NULL, concat('irclid=', irclid), NULL),+ if(irclickid IS NOT NULL, concat('irclickid=', irclickid), NULL),+ if(utm_id IS NOT NULL, concat('utm_id=', utm_id), NULL),+ if(utm_source_platform IS NOT NULL, concat('utm_source_platform=', utm_source_platform), NULL),+ if(utm_creative_format IS NOT NULL, concat('utm_creative_format=', utm_creative_format), NULL),+ if(utm_marketing_tactic IS NOT NULL, concat('utm_marketing_tactic=', utm_marketing_tactic), NULL),+ if(yclid IS NOT NULL, concat('yclid=', yclid), NULL),+ if(tblci IS NOT NULL, concat('tblci=', tblci), NULL),+ if(dicbo IS NOT NULL, concat('dicbo=', dicbo), NULL),+ if(ndclid IS NOT NULL, concat('ndclid=', ndclid), NULL),+ if(cjevent IS NOT NULL, concat('cjevent=', cjevent), NULL),+ if(awc IS NOT NULL, concat('awc=', awc), NULL),+ if(syclid IS NOT NULL, concat('syclid=', syclid), NULL),+ if(_bhlid IS NOT NULL, concat('_bhlid=', _bhlid), NULL),+ if(rtid IS NOT NULL, concat('rtid=', rtid), NULL),+ if(oppref IS NOT NULL, concat('oppref=', oppref), NULL),+ if(bfcid IS NOT NULL, concat('bfcid=', bfcid), NULL)]) AS Array(String)) as ad_ids_set,tupleElement(p, '$host') as _host,-- attribution properties from non-pageview/screen events should be deprioritized, so make the timestamp +/- 1 year so they sort last@@ -7168,21 +7210,21 @@initializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) as entry_fbclid,-- has gclid/fbclid for reading fewer bytes when calculating channel type- initializeAggregation('argMinState', _gclid IS NOT NULL, pageview_prio_timestamp_min) as entry_has_gclid,- initializeAggregation('argMinState', _fbclid IS NOT NULL, pageview_prio_timestamp_min) as entry_has_fbclid,+ -- clients send "gclid": "" and that is not a click id, so an empty value must not count as present+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) as entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) as entry_has_fbclid,-- other ad ids- initializeAggregation('argMinState', ad_ids_map, pageview_prio_timestamp_min) as entry_ad_ids_map,initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) as entry_ad_ids_set,-- channel type- initializeAggregation('argMinState', tuple(_utm_source, _utm_medium, _utm_campaign, _referring_domain, _gclid IS NOT NULL, _fbclid IS NOT NULL, _gad_source), pageview_prio_timestamp_min) as entry_channel_type_properties,+ initializeAggregation('argMinState', tuple(_utm_source, _utm_medium, _utm_campaign, _referring_domain, nullIf(_gclid, '') IS NOT NULL, nullIf(_fbclid, '') IS NOT NULL, _gad_source), pageview_prio_timestamp_min) as entry_channel_type_properties,-- counts- initializeAggregation('uniqExactState', if(event='$pageview', uuid, NULL)) as pageview_uniq,- initializeAggregation('uniqExactState', if(event='$autocapture', uuid, NULL)) as autocapture_uniq,- initializeAggregation('uniqExactState', if(event='$screen', uuid, NULL)) as screen_uniq,+ initializeAggregation('uniqState', if(event='$pageview', uuid, NULL)) as pageview_uniq,+ initializeAggregation('uniqState', if(event='$autocapture', uuid, NULL)) as autocapture_uniq,+ initializeAggregation('uniqState', if(event='$screen', uuid, NULL)) as screen_uniq,-- perfinitializeAggregation('uniqUpToState(1)', if(event='$pageview' OR event='$screen', uuid, NULL)) as page_screen_uniq_up_to,@@ -7190,7 +7232,6 @@-- flagsarrayMap((k, v) -> concat(k, '=', v), mapKeys(properties_group_feature_flags), mapValues(properties_group_feature_flags)) as flag_key_values,- mapKeys(properties_group_feature_flags) as flag_keys,-- event names[event] as event_names,@@ -7204,6 +7245,11 @@false as has_replay_eventsFROM posthog_test.sharded_events AS source_tableWHERE bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 0xF) == 7 -- has a session id and is valid uuidv7+ -- Clients can send a UUIDv7 session id with a garbage timestamp (1970, year 6339). Each one puts its session in its+ -- own monthly partition, and its events stretch the session duration across months. Keep only events within 31 days+ -- of the session id's timestamp. The bound is symmetric because the client mints the session id with its own clock+ -- while the server corrects event timestamps, so a client clock that runs days ahead is still a real session.+ AND fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) BETWEEN timestamp - INTERVAL 31 DAY AND timestamp + INTERVAL 31 DAYAND TRUE@@ -7272,16 +7318,15 @@initializeAggregation('argMinState', false, max_ts_64) as entry_has_fbclid,-- other ad ids- initializeAggregation('argMinState', CAST(map(), 'Map(String, String)'), max_ts_64) as entry_ad_ids_map,initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) as entry_ad_ids_set,-- channel typeinitializeAggregation('argMinState', tuple(null_s, null_s, null_s, null_s, false, false, null_s), max_ts_64) as entry_channel_type_properties,-- counts- initializeAggregation('uniqExactState', null_uuid) as pageview_uniq,- initializeAggregation('uniqExactState', null_uuid) as autocapture_uniq,- initializeAggregation('uniqExactState', null_uuid) as screen_uniq,+ initializeAggregation('uniqState', null_uuid) as pageview_uniq,+ initializeAggregation('uniqState', null_uuid) as autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) as screen_uniq,-- perfinitializeAggregation('uniqUpToState(1)', null_uuid) as page_screen_uniq_up_to,@@ -7289,7 +7334,6 @@-- flagsCAST([], 'Array(String)') as flag_key_values,- CAST([], 'Array(String)') as flag_keys,-- event namesCAST([], 'Array(String)') as event_names,@@ -7304,6 +7348,9 @@true as has_replay_eventsFROM posthog_test.sharded_session_replay_events AS source_tableWHERE bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 0xF) == 7 -- has a session id and is valid uuidv7+ -- same session timestamp guard as the events SELECT. accurateCastOrNull because accurateCast throws on a malformed+ -- session_id, and ClickHouse can evaluate this condition before the uuidv7 check above filters those rows out+ AND fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) BETWEEN min_first_timestamp - INTERVAL 31 DAY AND min_first_timestamp + INTERVAL 31 DAYAND TRUE@@ -9048,8 +9095,8 @@entry_has_gclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),- -- for lower-tier ad ids, just put them in a map, and set of the ones present- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),+ -- lower-tier ad ids as a flat 'key=value' array - like flag_key_values, a per-key map+ -- state made merges much more expensiveentry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),-- channel type properties tuple - to reduce redundant reading of the timestamp when loading all of these columns@@ -9057,12 +9104,13 @@entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Boolean, Boolean, Nullable(String)), DateTime64(6, 'UTC')),-- Count pageview, autocapture, and screen events for providing totals.- -- Use uniqExact instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see+ -- Use uniq instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see-- events being inserted multiple times to be deduped later, but that can trigger multiple rows here.-- Additionally, idempotency is useful for backfilling, as we can just reinsert the same events without worrying.- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ -- uniq (not uniqExact) keeps the state small; per-session counts are tiny so it is exact in practice.+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),-- As a performance optimisation, also keep track of the uniq events for all of these combined.-- This is a much more efficient way of calculating the bounce rate, as >2 means not a bounce@@ -9071,8 +9119,8 @@-- Flags - store every seen 'key=value' pair per flag. A flat array instead of a-- groupUniqArrayMap state because per-key sub-aggregators made merges ~8-37x more expensive.+ -- Key-only searches use the text index below (tokenized on '='), so no separate keys column.flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),-- Event names - store unique event names seen in this sessionevent_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),@@ -9089,8 +9137,7 @@-- IndexesINDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])),INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/77f1df52-4b43-11e9-910f-b8ca3a9b9f3e_{shard}/posthog.raw_sessions_v3', '{replica}')@@ -11544,8 +11591,8 @@entry_has_gclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),- -- for lower-tier ad ids, just put them in a map, and set of the ones present- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),+ -- lower-tier ad ids as a flat 'key=value' array - like flag_key_values, a per-key map+ -- state made merges much more expensiveentry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),-- channel type properties tuple - to reduce redundant reading of the timestamp when loading all of these columns@@ -11553,12 +11600,13 @@entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Boolean, Boolean, Nullable(String)), DateTime64(6, 'UTC')),-- Count pageview, autocapture, and screen events for providing totals.- -- Use uniqExact instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see+ -- Use uniq instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see-- events being inserted multiple times to be deduped later, but that can trigger multiple rows here.-- Additionally, idempotency is useful for backfilling, as we can just reinsert the same events without worrying.- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ -- uniq (not uniqExact) keeps the state small; per-session counts are tiny so it is exact in practice.+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),-- As a performance optimisation, also keep track of the uniq events for all of these combined.-- This is a much more efficient way of calculating the bounce rate, as >2 means not a bounce@@ -11567,8 +11615,8 @@-- Flags - store every seen 'key=value' pair per flag. A flat array instead of a-- groupUniqArrayMap state because per-key sub-aggregators made merges ~8-37x more expensive.+ -- Key-only searches use the text index below (tokenized on '='), so no separate keys column.flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),-- Event names - store unique event names seen in this sessionevent_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),@@ -14939,8 +14987,8 @@entry_has_gclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),- -- for lower-tier ad ids, just put them in a map, and set of the ones present- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),+ -- lower-tier ad ids as a flat 'key=value' array - like flag_key_values, a per-key map+ -- state made merges much more expensiveentry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),-- channel type properties tuple - to reduce redundant reading of the timestamp when loading all of these columns@@ -14948,12 +14996,13 @@entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Boolean, Boolean, Nullable(String)), DateTime64(6, 'UTC')),-- Count pageview, autocapture, and screen events for providing totals.- -- Use uniqExact instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see+ -- Use uniq instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see-- events being inserted multiple times to be deduped later, but that can trigger multiple rows here.-- Additionally, idempotency is useful for backfilling, as we can just reinsert the same events without worrying.- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ -- uniq (not uniqExact) keeps the state small; per-session counts are tiny so it is exact in practice.+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),-- As a performance optimisation, also keep track of the uniq events for all of these combined.-- This is a much more efficient way of calculating the bounce rate, as >2 means not a bounce@@ -14962,8 +15011,8 @@-- Flags - store every seen 'key=value' pair per flag. A flat array instead of a-- groupUniqArrayMap state because per-key sub-aggregators made merges ~8-37x more expensive.+ -- Key-only searches use the text index below (tokenized on '='), so no separate keys column.flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),-- Event names - store unique event names seen in this sessionevent_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),@@ -14980,8 +15029,7 @@-- IndexesINDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])),INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = ReplicatedAggregatingMergeTree('/clickhouse/tables/77f1df52-4b43-11e9-910f-b8ca3a9b9f3e_{shard}/posthog.raw_sessions_v3', '{replica}')diff --git a/posthog/clickhouse/test/test_raw_sessions_v3_model.py b/posthog/clickhouse/test/test_raw_sessions_v3_model.pyindex 160d0f6d2898..d5ee351a105b 100644--- a/posthog/clickhouse/test/test_raw_sessions_v3_model.py+++ b/posthog/clickhouse/test/test_raw_sessions_v3_model.py@@ -8,6 +8,8 @@ from posthog.test.base import (snapshot_clickhouse_queries,)+from parameterized import parameterized+from posthog.clickhouse.client import query_with_columns, sync_executefrom posthog.models.raw_sessions.sessions_v3 import (DISTRIBUTED_RAW_SESSIONS_TABLE_V3,@@ -28,10 +30,12 @@ def create_distinct_id():return f"d{distinct_id_counter}"-def create_session_id():+def create_session_id(start: str = "2024-03-08T00:00:00+00:00") -> str:+ # The sessions MV drops events that are far from the session id's own timestamp, so the id has to+ # start at the same time as the events these tests write.global session_id_countersession_id_counter += 1- return str(uuid7(random=session_id_counter))+ return str(uuid7(start, random=session_id_counter))@snapshot_clickhouse_queries@@ -373,7 +377,7 @@ class TestRawSessionsModel(ClickhouseTestMixin, BaseTest):# assert that it's close to now, allowing for a small margin of error because we're running this on CI in the cloud somewhere with preemptingassert abs((max_inserted_at - now).total_seconds()) < 10- def test_ad_ids_map_and_set(self):+ def test_ad_ids_set_collects_kv_pairs(self):distinct_id = create_distinct_id()session_id = create_session_id()@@ -391,13 +395,44 @@ class TestRawSessionsModel(ClickhouseTestMixin, BaseTest):result = self.select_by_session_id(session_id)- assert result[0]["entry_ad_ids_map"][present_ad_id] == value- assert missing_ad_id not in result[0]["entry_ad_ids_map"].keys()+ ad_ids_set = result[0]["entry_ad_ids_set"]+ assert f"{present_ad_id}={value}" in ad_ids_set+ assert not any(entry.startswith(f"{missing_ad_id}=") for entry in ad_ids_set)++ @parameterized.expand(+ [+ ("gad_campaignid", "gad_campaignid", "123456789"),+ ("utm_id", "utm_id", "spring_sale"),+ ("irclickid", "irclickid", "imp_abc"),+ ("snapchat_canonical", "sccid", "sc-lower"),+ ("snapchat_sccid_cased", "ScCid", "sc-cased"),+ ("snapchat_sscid_cased", "SsCid", "sc-alt"),+ ]+ )+ def test_ad_ids_fall_back_to_url_params(self, _name, url_param, value):+ distinct_id = create_distinct_id()+ session_id = create_session_id()++ _create_event(+ team=self.team,+ event="$pageview",+ distinct_id=distinct_id,+ properties={"$session_id": session_id, "$current_url": f"https://example.com/?{url_param}={value}"},+ timestamp="2024-03-08",+ )++ result = self.select_by_session_id(session_id)- assert present_ad_id in result[0]["entry_ad_ids_set"]- assert missing_ad_id not in result[0]["entry_ad_ids_set"]+ canonical_key = "sccid" if url_param in ("ScCid", "SsCid") else url_param+ assert f"{canonical_key}={value}" in result[0]["entry_ad_ids_set"]- def test_channel_type_properties(self):+ @parameterized.expand(+ [+ ("property_wins", "from_property", "from_property"),+ ("empty_property_falls_back", "", "from_url"),+ ]+ )+ def test_ad_id_property_vs_url_param_precedence(self, _name, property_value, expected):distinct_id = create_distinct_id()session_id = create_session_id()@@ -407,54 +442,144 @@ class TestRawSessionsModel(ClickhouseTestMixin, BaseTest):distinct_id=distinct_id,properties={"$session_id": session_id,- "gad_source": "1",- "gclid": "some_gclid",- "$referring_domain": "google.com",- "utm_campaign": "some_campaign",- "utm_medium": "some_medium",- "utm_source": "some_source",+ "ttclid": property_value,+ "$current_url": "https://example.com/?ttclid=from_url",},timestamp="2024-03-08",)result = self.select_by_session_id(session_id)- assert result[0]["entry_channel_type_properties"] == (- "some_source",- "some_medium",- "some_campaign",- "google.com",- True,- False,- "1",+ assert f"ttclid={expected}" in result[0]["entry_ad_ids_set"]++ def test_ad_id_url_param_decodes_plus_as_space(self):+ distinct_id = create_distinct_id()+ session_id = create_session_id()++ _create_event(+ team=self.team,+ event="$pageview",+ distinct_id=distinct_id,+ properties={+ "$session_id": session_id,+ "$current_url": "https://example.com/?utm_id=spring+sale",+ },+ timestamp="2024-03-08",)- def test_autocapture_does_not_set_attribution_when_pageview_present(self):+ result = self.select_by_session_id(session_id)++ # The SDK promotes this property as 'spring sale', so a fallback that kept the plus+ # would split one campaign across two values.+ assert "utm_id=spring sale" in result[0]["entry_ad_ids_set"]++ def test_ad_id_url_param_with_undecodable_bytes_is_normalized(self):distinct_id = create_distinct_id()session_id = create_session_id()_create_event(team=self.team,- event="$autocapture",+ event="$pageview",distinct_id=distinct_id,properties={"$session_id": session_id,- "$current_url": "/1",- "utm_source": "source1",+ "$current_url": "https://example.com/?ttclid=%FF",},timestamp="2024-03-08",)++ result = self.select_by_session_id(session_id)++ # Without toValidUTF8 this value comes back as bytes, which breaks text readers.+ assert "ttclid=\ufffd" in result[0]["entry_ad_ids_set"]++ @parameterized.expand(+ [+ ("click_ids_present", "some_gclid", "some_fbclid", True, True),+ ("empty_click_ids_are_not_present", "", "", False, False),+ ]+ )+ def test_channel_type_properties(self, _name, gclid, fbclid, expected_has_gclid, expected_has_fbclid):+ distinct_id = create_distinct_id()+ session_id = create_session_id()+_create_event(team=self.team,event="$pageview",distinct_id=distinct_id,properties={"$session_id": session_id,- "$current_url": "/2",- "utm_source": "source2",+ "gad_source": "1",+ "gclid": gclid,+ "fbclid": fbclid,+ "$referring_domain": "google.com",+ "utm_campaign": "some_campaign",+ "utm_medium": "some_medium",+ "utm_source": "some_source",},- timestamp="2024-03-09",+ timestamp="2024-03-08",+ )++ result = self.select_by_session_id(session_id)++ assert result[0]["entry_channel_type_properties"] == (+ "some_source",+ "some_medium",+ "some_campaign",+ "google.com",+ expected_has_gclid,+ expected_has_fbclid,+ "1",)+ assert result[0]["entry_has_gclid"] is expected_has_gclid+ assert result[0]["entry_has_fbclid"] is expected_has_fbclid++ @parameterized.expand(+ [+ ("autocapture_first", "$autocapture", False),+ ("pageleave_first", "$pageleave", False),+ ("custom_event_first", "$feature_flag_called", False),+ ("pageview_inserted_before_earlier_event", "$autocapture", True),+ ]+ )+ def test_non_pageview_event_does_not_set_attribution_when_pageview_present(+ self, _name, first_event, insert_pageview_first+ ):+ distinct_id = create_distinct_id()+ session_id = create_session_id()++ def create_first_event() -> None:+ _create_event(+ team=self.team,+ event=first_event,+ distinct_id=distinct_id,+ properties={+ "$session_id": session_id,+ "$current_url": "/1",+ "utm_source": "source1",+ },+ timestamp="2024-03-08",+ )++ def create_pageview() -> None:+ _create_event(+ team=self.team,+ event="$pageview",+ distinct_id=distinct_id,+ properties={+ "$session_id": session_id,+ "$current_url": "/2",+ "utm_source": "source2",+ },+ timestamp="2024-03-09",+ )++ if insert_pageview_first:+ create_pageview()+ create_first_event()+ else:+ create_first_event()+ create_pageview()result = self.select_by_session_id(session_id)@@ -645,6 +770,35 @@ class TestRawSessionsModel(ClickhouseTestMixin, BaseTest):k: v for k, v in result_2[0].items() if k not in {"has_replay_events", "max_inserted_at"}}+ @parameterized.expand(+ [+ ("garbage_session_timestamp", "1970-01-02T00:00:00+00:00", "2024-03-08", False),+ ("session_id_over_31_days_after_event", "2024-04-10T00:00:00+00:00", "2024-03-08", False),+ ("client_clock_days_ahead", "2024-03-11T00:00:00+00:00", "2024-03-08", True),+ ("late_event_within_31_days", "2024-03-08T00:00:00+00:00", "2024-04-07", True),+ ]+ )+ def test_drops_events_far_from_the_session_id_timestamp(self, _name, session_start, event_timestamp, kept):+ distinct_id = create_distinct_id()+ session_id = create_session_id(session_start)+ _create_event(+ team=self.team,+ event="$pageview",+ distinct_id=distinct_id,+ properties={"$current_url": "/", "$session_id": session_id},+ timestamp=event_timestamp,+ )+ produce_replay_summary(+ team_id=self.team.pk,+ distinct_id=distinct_id,+ session_id=session_id,+ first_timestamp=event_timestamp,+ last_timestamp=event_timestamp,+ ensure_analytics_event_in_session=False,+ )++ assert len(self.select_by_session_id(session_id)) == (1 if kept else 0)+def test_event_names_are_collected(self):distinct_id = create_distinct_id()session_id = create_session_id()@@ -682,38 +836,6 @@ class TestRawSessionsModel(ClickhouseTestMixin, BaseTest):assert set(result[0]["event_names"]) == {"$pageview", "$autocapture", "custom_event"}- def test_flag_keys_are_collected(self):- distinct_id = create_distinct_id()- session_id = create_session_id()-- _create_event(- team=self.team,- event="$pageview",- distinct_id=distinct_id,- properties={- "$session_id": session_id,- "$feature/flag_a": "value1",- "$feature/flag_b": "value2",- },- timestamp="2024-03-08",- )- _create_event(- team=self.team,- event="$autocapture",- distinct_id=distinct_id,- properties={- "$session_id": session_id,- "$feature/flag_a": "different_value",- "$feature/flag_c": "value3",- },- timestamp="2024-03-08",- )-- result = self.select_by_session_id(session_id)-- # Should have all unique flag keys (not values)- assert set(result[0]["flag_keys"]) == {"$feature/flag_a", "$feature/flag_b", "$feature/flag_c"}-def test_hosts_are_collected(self):distinct_id = create_distinct_id()session_id = create_session_id()diff --git a/posthog/hogql/database/schema/sessions_v3.py b/posthog/hogql/database/schema/sessions_v3.pyindex 0262f3abc834..444b41a29b66 100644--- a/posthog/hogql/database/schema/sessions_v3.py+++ b/posthog/hogql/database/schema/sessions_v3.py@@ -75,7 +75,6 @@ RAW_SESSIONS_FIELDS: dict[str, FieldOrTable] = {"entry_fbclid": DatabaseField(name="entry_fbclid", nullable=False),"entry_has_gclid": DatabaseField(name="entry_has_gclid", nullable=False),"entry_has_fbclid": DatabaseField(name="entry_has_fbclid", nullable=False),- "entry_ad_ids_map": DatabaseField(name="entry_ad_ids_map", nullable=False),"entry_ad_ids_set": DatabaseField(name="entry_ad_ids_set", nullable=False),"entry_channel_type_properties": DatabaseField(name="entry_channel_type_properties", nullable=False),"pageview_uniq": DatabaseField(name="pageview_uniq", nullable=False),@@ -156,7 +155,7 @@ LAZY_SESSIONS_FIELDS: dict[str, FieldOrTable] = {"$entry_gad_source": DatabaseField(name="$entry_gad_source"),"$entry_has_fbclid": BooleanDatabaseField(name="$entry_has_fbclid"),"$entry_has_gclid": BooleanDatabaseField(name="$entry_has_gclid"),- # "counts" - actually uses uniqExact behind the scenes+ # "counts" - actually uses uniq behind the scenes"$pageview_count": IntegerDatabaseField(name="$pageview_count"),"$autocapture_count": IntegerDatabaseField(name="$autocapture_count"),"$screen_count": IntegerDatabaseField(name="$screen_count"),@@ -195,6 +194,43 @@ for session_ad_id in SESSION_V3_LOWER_TIER_AD_IDS:LAZY_SESSIONS_FIELDS["$entry_has_" + session_ad_id] = BooleanDatabaseField(name="$entry_has_" + session_ad_id)+# entry_ad_ids_set stores flat "key=value" strings; these helpers pull one key's value or presence out of it+def ad_id_value_from_set_expr(set_expr: ast.Expr, session_ad_id: str) -> ast.Call:+ prefix = session_ad_id + "="+ # arrayFirst guarantees the prefix sits at position 0, so replaceOne strips exactly the prefix+ return ast.Call(+ name="replaceOne",+ args=[+ ast.Call(+ name="arrayFirst",+ args=[+ ast.Lambda(+ args=["x"],+ expr=ast.Call(name="startsWith", args=[ast.Field(chain=["x"]), ast.Constant(value=prefix)]),+ ),+ set_expr,+ ],+ ),+ ast.Constant(value=prefix),+ ast.Constant(value=""),+ ],+ )+++def ad_id_present_in_set_expr(set_expr: ast.Expr, session_ad_id: str) -> ast.Call:+ prefix = session_ad_id + "="+ return ast.Call(+ name="arrayExists",+ args=[+ ast.Lambda(+ args=["x"],+ expr=ast.Call(name="startsWith", args=[ast.Field(chain=["x"]), ast.Constant(value=prefix)]),+ ),+ set_expr,+ ],+ )++def get_binary_fields(table: Table) -> set[str]:return {key for key, val in table.fields.items() if val.__class__ == DatabaseField}@@ -295,9 +331,9 @@ def select_from_sessions_table_v3("$entry_has_gclid": arg_min_merge_field("entry_has_gclid"),"$entry_has_fbclid": arg_min_merge_field("entry_has_fbclid"),# the count columns here do not come from the "count" columns in the raw table, instead aggregate the uniq columns- "$pageview_count": ast.Call(name="uniqExactMerge", args=[ast.Field(chain=[table_name, "pageview_uniq"])]),- "$screen_count": ast.Call(name="uniqExactMerge", args=[ast.Field(chain=[table_name, "screen_uniq"])]),- "$autocapture_count": ast.Call(name="uniqExactMerge", args=[ast.Field(chain=[table_name, "autocapture_uniq"])]),+ "$pageview_count": ast.Call(name="uniqMerge", args=[ast.Field(chain=[table_name, "pageview_uniq"])]),+ "$screen_count": ast.Call(name="uniqMerge", args=[ast.Field(chain=[table_name, "screen_uniq"])]),+ "$autocapture_count": ast.Call(name="uniqMerge", args=[ast.Field(chain=[table_name, "autocapture_uniq"])]),"$page_screen_count_up_to": ast.Call(name="uniqUpToMerge",params=[ast.Constant(value=1)],@@ -307,7 +343,6 @@ def select_from_sessions_table_v3(name="max",args=[ast.Field(chain=[table_name, "has_autocapture"])],),- "$entry_ad_ids_map": arg_min_merge_field("entry_ad_ids_map"),"$entry_ad_ids_set": arg_min_merge_field("entry_ad_ids_set"),"$entry_channel_type_properties": arg_min_merge_field("entry_channel_type_properties"),"$has_replay_events": ast.Call(name="max", args=[ast.Field(chain=[table_name, "has_replay_events"])]),@@ -318,16 +353,11 @@ def select_from_sessions_table_v3(# Add ad ids fieldsfor session_ad_id in SESSION_V3_LOWER_TIER_AD_IDS:- aggregate_fields["$entry_" + session_ad_id] = ast.Call(- name="arrayElement",- args=[- aggregate_fields["$entry_ad_ids_map"],- ast.Constant(value=session_ad_id),- ],+ aggregate_fields["$entry_" + session_ad_id] = ad_id_value_from_set_expr(+ aggregate_fields["$entry_ad_ids_set"], session_ad_id)- aggregate_fields["$entry_has_" + session_ad_id] = ast.Call(- name="has",- args=[aggregate_fields["$entry_ad_ids_set"], ast.Constant(value=session_ad_id)],+ aggregate_fields["$entry_has_" + session_ad_id] = ad_id_present_in_set_expr(+ aggregate_fields["$entry_ad_ids_set"], session_ad_id)# Some fields are calculated from others. It'd be good to actually deduplicate common sub expressions in SQL, but@@ -601,9 +631,8 @@ SESSION_PROPERTY_TO_RAW_SESSIONS_EXPR: dict[str, ast.Expr] = {}for session_ad_id in SESSION_V3_LOWER_TIER_AD_IDS:- SESSION_PROPERTY_TO_RAW_SESSIONS_EXPR["$entry_" + session_ad_id] = ast.Call(- name="arrayElement",- args=[finalize_aggregation("entry_ad_ids_map"), ast.Constant(value=session_ad_id)],+ SESSION_PROPERTY_TO_RAW_SESSIONS_EXPR["$entry_" + session_ad_id] = ad_id_value_from_set_expr(+ finalize_aggregation("entry_ad_ids_set"), session_ad_id)diff --git a/posthog/hogql/database/schema/test/__snapshots__/test_sessions_v3.ambr b/posthog/hogql/database/schema/test/__snapshots__/test_sessions_v3.ambrindex 8560c75ea4fa..de42c3865df3 100644--- a/posthog/hogql/database/schema/test/__snapshots__/test_sessions_v3.ambr+++ b/posthog/hogql/database/schema/test/__snapshots__/test_sessions_v3.ambr@@ -60,7 +60,232 @@use_hive_partitioning=0'''# ----# name: TestSessionsV3.test_channel_type+# name: TestSessionsV3.test_channel_type_0_gad_source+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_1_gclid_from_google+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_2_empty_gclid_is_not_a_click+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_3_gad_source_without_referrer+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_4_fbclid_without_referrer+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_5_unknown_source_stripped_referrer+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_6_unknown_referrer+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_7_self_referral_with_paid_utms+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_8_self_referral_with_unknown_source+ '''+ SELECT sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_channel_type_9_msclkid_only_in_url'''SELECT sessions.`$channel_type` AS `$channel_type`FROM@@ -91,9 +316,9 @@sessions.`$autocapture_count` AS `$autocapture_count`,sessions.`$screen_count` AS `$screen_count`FROM- (SELECT uniqExactMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,- uniqExactMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,- uniqExactMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,+ (SELECT uniqMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,+ uniqMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,+ uniqMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS id,raw_sessions_v3.session_id_v7 AS session_id_v7FROM raw_sessions_v3@@ -198,9 +423,9 @@sessions.`$autocapture_count` AS `$autocapture_count`,sessions.`$screen_count` AS `$screen_count`FROM- (SELECT uniqExactMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,- uniqExactMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,- uniqExactMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,+ (SELECT uniqMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,+ uniqMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,+ uniqMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS id,raw_sessions_v3.session_id_v7 AS session_id_v7FROM raw_sessions_v3@@ -221,6 +446,151 @@use_hive_partitioning=0'''# ---+# name: TestSessionsV3.test_entry_utm_and_channel_type_across_events_0_utm_only_in_url+ '''+ SELECT sessions.`$entry_utm_source` AS `$entry_utm_source`,+ sessions.`$entry_utm_campaign` AS `$entry_utm_campaign`,+ sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT argMinMerge(raw_sessions_v3.entry_utm_source) AS `$entry_utm_source`,+ argMinMerge(raw_sessions_v3.entry_utm_campaign) AS `$entry_utm_campaign`,+ multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_entry_utm_and_channel_type_across_events_1_first_event_is_not_a_pageview+ '''+ SELECT sessions.`$entry_utm_source` AS `$entry_utm_source`,+ sessions.`$entry_utm_campaign` AS `$entry_utm_campaign`,+ sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT argMinMerge(raw_sessions_v3.entry_utm_source) AS `$entry_utm_source`,+ argMinMerge(raw_sessions_v3.entry_utm_campaign) AS `$entry_utm_campaign`,+ multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_entry_utm_and_channel_type_across_events_2_utm_added_on_second_pageview+ '''+ SELECT sessions.`$entry_utm_source` AS `$entry_utm_source`,+ sessions.`$entry_utm_campaign` AS `$entry_utm_campaign`,+ sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT argMinMerge(raw_sessions_v3.entry_utm_source) AS `$entry_utm_source`,+ argMinMerge(raw_sessions_v3.entry_utm_campaign) AS `$entry_utm_campaign`,+ multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_entry_utm_and_channel_type_across_events_3_two_campaigns_keep_the_first+ '''+ SELECT sessions.`$entry_utm_source` AS `$entry_utm_source`,+ sessions.`$entry_utm_campaign` AS `$entry_utm_campaign`,+ sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT argMinMerge(raw_sessions_v3.entry_utm_source) AS `$entry_utm_source`,+ argMinMerge(raw_sessions_v3.entry_utm_campaign) AS `$entry_utm_campaign`,+ multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_entry_utm_and_channel_type_across_events_4_sdk_session_entry_properties_are_not_read+ '''+ SELECT sessions.`$entry_utm_source` AS `$entry_utm_source`,+ sessions.`$entry_utm_campaign` AS `$entry_utm_campaign`,+ sessions.`$channel_type` AS `$channel_type`+ FROM+ (SELECT argMinMerge(raw_sessions_v3.entry_utm_source) AS `$entry_utm_source`,+ argMinMerge(raw_sessions_v3.entry_utm_campaign) AS `$entry_utm_campaign`,+ multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---# name: TestSessionsV3.test_event_sessions_where'''SELECT count() AS `count()`@@ -277,9 +647,9 @@sessions.`$autocapture_count` AS `$autocapture_count`,sessions.`$screen_count` AS `$screen_count`FROM- (SELECT uniqExactMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,- uniqExactMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,- uniqExactMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,+ (SELECT uniqMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,+ uniqMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,+ uniqMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS id,raw_sessions_v3.session_id_v7 AS session_id_v7FROM raw_sessions_v3@@ -386,6 +756,72 @@use_hive_partitioning=0'''# ---+# name: TestSessionsV3.test_replay_rows_shape_the_session_0_replay_starts_before_first_pageview+ '''+ SELECT sessions.`$start_timestamp` AS `$start_timestamp`,+ sessions.`$session_duration` AS `$session_duration`,+ sessions.`$pageview_count` AS `$pageview_count`,+ sessions.`$is_bounce` AS `$is_bounce`,+ sessions.`$has_replay_events` AS `$has_replay_events`+ FROM+ (SELECT min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')) AS `$start_timestamp`,+ dateDiff('second', min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')), max(toTimeZone(raw_sessions_v3.max_timestamp, 'UTC'))) AS `$session_duration`,+ uniqMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,+ if(equals(uniqUpToMerge(1)(raw_sessions_v3.page_screen_uniq_up_to), 0), NULL, not(or(greaterOrEquals(uniqUpToMerge(1)(raw_sessions_v3.page_screen_uniq_up_to), 2), max(raw_sessions_v3.has_autocapture), greaterOrEquals(dateDiff('second', min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')), max(toTimeZone(raw_sessions_v3.max_timestamp, 'UTC'))), 10)))) AS `$is_bounce`,+ max(raw_sessions_v3.has_replay_events) AS `$has_replay_events`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestSessionsV3.test_replay_rows_shape_the_session_1_replay_only+ '''+ SELECT sessions.`$start_timestamp` AS `$start_timestamp`,+ sessions.`$session_duration` AS `$session_duration`,+ sessions.`$pageview_count` AS `$pageview_count`,+ sessions.`$is_bounce` AS `$is_bounce`,+ sessions.`$has_replay_events` AS `$has_replay_events`+ FROM+ (SELECT min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')) AS `$start_timestamp`,+ dateDiff('second', min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')), max(toTimeZone(raw_sessions_v3.max_timestamp, 'UTC'))) AS `$session_duration`,+ uniqMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,+ if(equals(uniqUpToMerge(1)(raw_sessions_v3.page_screen_uniq_up_to), 0), NULL, not(or(greaterOrEquals(uniqUpToMerge(1)(raw_sessions_v3.page_screen_uniq_up_to), 2), max(raw_sessions_v3.has_autocapture), greaterOrEquals(dateDiff('second', min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')), max(toTimeZone(raw_sessions_v3.max_timestamp, 'UTC'))), 10)))) AS `$is_bounce`,+ max(raw_sessions_v3.has_replay_events) AS `$has_replay_events`,+ toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,+ raw_sessions_v3.session_id_v7 AS session_id_v7+ FROM raw_sessions_v3+ WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))+ GROUP BY raw_sessions_v3.session_id_v7) AS sessions+ WHERE equals(sessions.session_id, '00000000-0000-0000-0000-000000000000')+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---# name: TestSessionsV3.test_select_session_replay_session_duration'''SELECT raw_session_replay_events__session.duration AS duration@@ -480,6 +916,10 @@sessions.`$entry_has_gbraid` AS `$entry_has_gbraid`,sessions.`$entry_wbraid` AS `$entry_wbraid`,sessions.`$entry_has_wbraid` AS `$entry_has_wbraid`,+ sessions.`$entry_gad_campaignid` AS `$entry_gad_campaignid`,+ sessions.`$entry_has_gad_campaignid` AS `$entry_has_gad_campaignid`,+ sessions.`$entry_srsltid` AS `$entry_srsltid`,+ sessions.`$entry_has_srsltid` AS `$entry_has_srsltid`,sessions.`$entry_msclkid` AS `$entry_msclkid`,sessions.`$entry_has_msclkid` AS `$entry_has_msclkid`,sessions.`$entry_twclid` AS `$entry_twclid`,@@ -492,6 +932,8 @@sessions.`$entry_has_igshid` AS `$entry_has_igshid`,sessions.`$entry_ttclid` AS `$entry_ttclid`,sessions.`$entry_has_ttclid` AS `$entry_has_ttclid`,+ sessions.`$entry_rdt_cid` AS `$entry_rdt_cid`,+ sessions.`$entry_has_rdt_cid` AS `$entry_has_rdt_cid`,sessions.`$entry_epik` AS `$entry_epik`,sessions.`$entry_has_epik` AS `$entry_has_epik`,sessions.`$entry_qclid` AS `$entry_qclid`,@@ -501,7 +943,39 @@sessions.`$entry__kx` AS `$entry__kx`,sessions.`$entry_has__kx` AS `$entry_has__kx`,sessions.`$entry_irclid` AS `$entry_irclid`,- sessions.`$entry_has_irclid` AS `$entry_has_irclid`+ sessions.`$entry_has_irclid` AS `$entry_has_irclid`,+ sessions.`$entry_irclickid` AS `$entry_irclickid`,+ sessions.`$entry_has_irclickid` AS `$entry_has_irclickid`,+ sessions.`$entry_utm_id` AS `$entry_utm_id`,+ sessions.`$entry_has_utm_id` AS `$entry_has_utm_id`,+ sessions.`$entry_utm_source_platform` AS `$entry_utm_source_platform`,+ sessions.`$entry_has_utm_source_platform` AS `$entry_has_utm_source_platform`,+ sessions.`$entry_utm_creative_format` AS `$entry_utm_creative_format`,+ sessions.`$entry_has_utm_creative_format` AS `$entry_has_utm_creative_format`,+ sessions.`$entry_utm_marketing_tactic` AS `$entry_utm_marketing_tactic`,+ sessions.`$entry_has_utm_marketing_tactic` AS `$entry_has_utm_marketing_tactic`,+ sessions.`$entry_yclid` AS `$entry_yclid`,+ sessions.`$entry_has_yclid` AS `$entry_has_yclid`,+ sessions.`$entry_tblci` AS `$entry_tblci`,+ sessions.`$entry_has_tblci` AS `$entry_has_tblci`,+ sessions.`$entry_dicbo` AS `$entry_dicbo`,+ sessions.`$entry_has_dicbo` AS `$entry_has_dicbo`,+ sessions.`$entry_ndclid` AS `$entry_ndclid`,+ sessions.`$entry_has_ndclid` AS `$entry_has_ndclid`,+ sessions.`$entry_cjevent` AS `$entry_cjevent`,+ sessions.`$entry_has_cjevent` AS `$entry_has_cjevent`,+ sessions.`$entry_awc` AS `$entry_awc`,+ sessions.`$entry_has_awc` AS `$entry_has_awc`,+ sessions.`$entry_syclid` AS `$entry_syclid`,+ sessions.`$entry_has_syclid` AS `$entry_has_syclid`,+ sessions.`$entry__bhlid` AS `$entry__bhlid`,+ sessions.`$entry_has__bhlid` AS `$entry_has__bhlid`,+ sessions.`$entry_rtid` AS `$entry_rtid`,+ sessions.`$entry_has_rtid` AS `$entry_has_rtid`,+ sessions.`$entry_oppref` AS `$entry_oppref`,+ sessions.`$entry_has_oppref` AS `$entry_has_oppref`,+ sessions.`$entry_bfcid` AS `$entry_bfcid`,+ sessions.`$entry_has_bfcid` AS `$entry_has_bfcid`FROM(SELECT toString(reinterpretAsUUID(bitOr(bitShiftLeft(raw_sessions_v3.session_id_v7, 64), bitShiftRight(raw_sessions_v3.session_id_v7, 64)))) AS session_id,raw_sessions_v3.session_timestamp AS session_timestamp,@@ -527,45 +1001,83 @@argMinMerge(raw_sessions_v3.entry_fbclid) AS `$entry_fbclid`,argMinMerge(raw_sessions_v3.entry_has_fbclid) AS `$entry_has_fbclid`,argMinMerge(raw_sessions_v3.entry_has_gclid) AS `$entry_has_gclid`,- uniqExactMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,- uniqExactMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,- uniqExactMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,+ uniqMerge(raw_sessions_v3.pageview_uniq) AS `$pageview_count`,+ uniqMerge(raw_sessions_v3.autocapture_uniq) AS `$autocapture_count`,+ uniqMerge(raw_sessions_v3.screen_uniq) AS `$screen_count`,multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), 'cross-network'), 'Cross Network', or(in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), tuple('cpc', 'cpm', 'cpv', 'cpa', 'ppc', 'retargeting')), startsWith(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'paid'), tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 5), isNotNull(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'))), coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Paid Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_paid', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(ifNull(equals(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 7), ''), 'null'), '1'), 0), 'Paid Search', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Paid Video', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Paid Social', 'Paid Unknown')), and(ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null'))), or(isNull(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null'))), in(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), tuple('(direct)', 'direct', '$direct'))), not(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6))), 'Direct', coalesce(coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 1), ''), 'null')), '')), 'source'))), if(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*(([^a-df-z]|^)shop|shopping).*)$'), 'Organic Shopping', NULL), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), ''), 'medium')), coalesce(dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), ''), 'source')), dictGetOrNull('posthog_test.channel_definition_dict', 'type_if_organic', (cutToFirstSignificantSubdomain(coalesce(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '')), 'source'))), multiIf(match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 3), ''), 'null')), '^(.*video.*)$'), 'Organic Video', match(lower(nullIf(nullIf(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 2), ''), 'null')), 'push$'), 'Push', tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 6), 'Organic Social', ifNull(equals(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4), '$direct'), 0), 'Direct', isNotNull(tupleElement(argMinMerge(raw_sessions_v3.entry_channel_type_properties), 4)), 'Referral', 'Unknown'))) AS `$channel_type`,dateDiff('second', min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')), max(toTimeZone(raw_sessions_v3.max_timestamp, 'UTC'))) AS `$session_duration`,if(equals(uniqUpToMerge(1)(raw_sessions_v3.page_screen_uniq_up_to), 0), NULL, not(or(greaterOrEquals(uniqUpToMerge(1)(raw_sessions_v3.page_screen_uniq_up_to), 2), max(raw_sessions_v3.has_autocapture), greaterOrEquals(dateDiff('second', min(toTimeZone(raw_sessions_v3.min_timestamp, 'UTC')), max(toTimeZone(raw_sessions_v3.max_timestamp, 'UTC'))), 10)))) AS `$is_bounce`,arrayDistinct(arrayFlatten(groupArray(raw_sessions_v3.hosts))) AS `$hosts`,arrayDistinct(arrayFlatten(groupArray(raw_sessions_v3.emails))) AS `$emails`,max(raw_sessions_v3.has_replay_events) AS `$has_replay_events`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'gclsrc') AS `$entry_gclsrc`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'gclsrc') AS `$entry_has_gclsrc`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'dclid') AS `$entry_dclid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'dclid') AS `$entry_has_dclid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'gbraid') AS `$entry_gbraid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'gbraid') AS `$entry_has_gbraid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'wbraid') AS `$entry_wbraid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'wbraid') AS `$entry_has_wbraid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'msclkid') AS `$entry_msclkid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'msclkid') AS `$entry_has_msclkid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'twclid') AS `$entry_twclid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'twclid') AS `$entry_has_twclid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'li_fat_id') AS `$entry_li_fat_id`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'li_fat_id') AS `$entry_has_li_fat_id`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'mc_cid') AS `$entry_mc_cid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'mc_cid') AS `$entry_has_mc_cid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'igshid') AS `$entry_igshid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'igshid') AS `$entry_has_igshid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'ttclid') AS `$entry_ttclid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'ttclid') AS `$entry_has_ttclid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'epik') AS `$entry_epik`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'epik') AS `$entry_has_epik`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'qclid') AS `$entry_qclid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'qclid') AS `$entry_has_qclid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'sccid') AS `$entry_sccid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'sccid') AS `$entry_has_sccid`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), '_kx') AS `$entry__kx`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), '_kx') AS `$entry_has__kx`,- arrayElement(argMinMerge(raw_sessions_v3.entry_ad_ids_map), 'irclid') AS `$entry_irclid`,- has(argMinMerge(raw_sessions_v3.entry_ad_ids_set), 'irclid') AS `$entry_has_irclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'gclsrc='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'gclsrc=', '') AS `$entry_gclsrc`,+ arrayExists(x -> startsWith(x, 'gclsrc='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_gclsrc`,+ replaceOne(arrayFirst(x -> startsWith(x, 'dclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'dclid=', '') AS `$entry_dclid`,+ arrayExists(x -> startsWith(x, 'dclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_dclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'gbraid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'gbraid=', '') AS `$entry_gbraid`,+ arrayExists(x -> startsWith(x, 'gbraid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_gbraid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'wbraid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'wbraid=', '') AS `$entry_wbraid`,+ arrayExists(x -> startsWith(x, 'wbraid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_wbraid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'gad_campaignid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'gad_campaignid=', '') AS `$entry_gad_campaignid`,+ arrayExists(x -> startsWith(x, 'gad_campaignid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_gad_campaignid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'srsltid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'srsltid=', '') AS `$entry_srsltid`,+ arrayExists(x -> startsWith(x, 'srsltid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_srsltid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'msclkid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'msclkid=', '') AS `$entry_msclkid`,+ arrayExists(x -> startsWith(x, 'msclkid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_msclkid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'twclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'twclid=', '') AS `$entry_twclid`,+ arrayExists(x -> startsWith(x, 'twclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_twclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'li_fat_id='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'li_fat_id=', '') AS `$entry_li_fat_id`,+ arrayExists(x -> startsWith(x, 'li_fat_id='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_li_fat_id`,+ replaceOne(arrayFirst(x -> startsWith(x, 'mc_cid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'mc_cid=', '') AS `$entry_mc_cid`,+ arrayExists(x -> startsWith(x, 'mc_cid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_mc_cid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'igshid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'igshid=', '') AS `$entry_igshid`,+ arrayExists(x -> startsWith(x, 'igshid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_igshid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'ttclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'ttclid=', '') AS `$entry_ttclid`,+ arrayExists(x -> startsWith(x, 'ttclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_ttclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'rdt_cid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'rdt_cid=', '') AS `$entry_rdt_cid`,+ arrayExists(x -> startsWith(x, 'rdt_cid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_rdt_cid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'epik='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'epik=', '') AS `$entry_epik`,+ arrayExists(x -> startsWith(x, 'epik='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_epik`,+ replaceOne(arrayFirst(x -> startsWith(x, 'qclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'qclid=', '') AS `$entry_qclid`,+ arrayExists(x -> startsWith(x, 'qclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_qclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'sccid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'sccid=', '') AS `$entry_sccid`,+ arrayExists(x -> startsWith(x, 'sccid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_sccid`,+ replaceOne(arrayFirst(x -> startsWith(x, '_kx='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), '_kx=', '') AS `$entry__kx`,+ arrayExists(x -> startsWith(x, '_kx='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has__kx`,+ replaceOne(arrayFirst(x -> startsWith(x, 'irclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'irclid=', '') AS `$entry_irclid`,+ arrayExists(x -> startsWith(x, 'irclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_irclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'irclickid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'irclickid=', '') AS `$entry_irclickid`,+ arrayExists(x -> startsWith(x, 'irclickid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_irclickid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'utm_id='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'utm_id=', '') AS `$entry_utm_id`,+ arrayExists(x -> startsWith(x, 'utm_id='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_utm_id`,+ replaceOne(arrayFirst(x -> startsWith(x, 'utm_source_platform='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'utm_source_platform=', '') AS `$entry_utm_source_platform`,+ arrayExists(x -> startsWith(x, 'utm_source_platform='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_utm_source_platform`,+ replaceOne(arrayFirst(x -> startsWith(x, 'utm_creative_format='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'utm_creative_format=', '') AS `$entry_utm_creative_format`,+ arrayExists(x -> startsWith(x, 'utm_creative_format='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_utm_creative_format`,+ replaceOne(arrayFirst(x -> startsWith(x, 'utm_marketing_tactic='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'utm_marketing_tactic=', '') AS `$entry_utm_marketing_tactic`,+ arrayExists(x -> startsWith(x, 'utm_marketing_tactic='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_utm_marketing_tactic`,+ replaceOne(arrayFirst(x -> startsWith(x, 'yclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'yclid=', '') AS `$entry_yclid`,+ arrayExists(x -> startsWith(x, 'yclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_yclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'tblci='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'tblci=', '') AS `$entry_tblci`,+ arrayExists(x -> startsWith(x, 'tblci='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_tblci`,+ replaceOne(arrayFirst(x -> startsWith(x, 'dicbo='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'dicbo=', '') AS `$entry_dicbo`,+ arrayExists(x -> startsWith(x, 'dicbo='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_dicbo`,+ replaceOne(arrayFirst(x -> startsWith(x, 'ndclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'ndclid=', '') AS `$entry_ndclid`,+ arrayExists(x -> startsWith(x, 'ndclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_ndclid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'cjevent='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'cjevent=', '') AS `$entry_cjevent`,+ arrayExists(x -> startsWith(x, 'cjevent='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_cjevent`,+ replaceOne(arrayFirst(x -> startsWith(x, 'awc='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'awc=', '') AS `$entry_awc`,+ arrayExists(x -> startsWith(x, 'awc='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_awc`,+ replaceOne(arrayFirst(x -> startsWith(x, 'syclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'syclid=', '') AS `$entry_syclid`,+ arrayExists(x -> startsWith(x, 'syclid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_syclid`,+ replaceOne(arrayFirst(x -> startsWith(x, '_bhlid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), '_bhlid=', '') AS `$entry__bhlid`,+ arrayExists(x -> startsWith(x, '_bhlid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has__bhlid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'rtid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'rtid=', '') AS `$entry_rtid`,+ arrayExists(x -> startsWith(x, 'rtid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_rtid`,+ replaceOne(arrayFirst(x -> startsWith(x, 'oppref='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'oppref=', '') AS `$entry_oppref`,+ arrayExists(x -> startsWith(x, 'oppref='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_oppref`,+ replaceOne(arrayFirst(x -> startsWith(x, 'bfcid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)), 'bfcid=', '') AS `$entry_bfcid`,+ arrayExists(x -> startsWith(x, 'bfcid='), argMinMerge(raw_sessions_v3.entry_ad_ids_set)) AS `$entry_has_bfcid`,raw_sessions_v3.session_id_v7 AS session_id_v7FROM raw_sessions_v3WHERE and(equals(raw_sessions_v3.team_id, 99999), equals(raw_sessions_v3.session_timestamp, fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull('00000000-0000-0000-0000-000000000000', 'UUID')), 80)))))diff --git a/posthog/hogql/database/schema/test/test_sessions_v3.py b/posthog/hogql/database/schema/test/test_sessions_v3.pyindex 80bbe2538448..b49b8277c149 100644--- a/posthog/hogql/database/schema/test/test_sessions_v3.py+++ b/posthog/hogql/database/schema/test/test_sessions_v3.py@@ -1,5 +1,5 @@import uuid-from time import time_ns+from datetime import datetime, timedeltaimport pytestfrom posthog.test.base import (@@ -24,6 +24,7 @@ from posthog.hogql.database.schema.sessions_v3 import (from posthog.hogql.parser import parse_selectfrom posthog.hogql.query import execute_hogql_query+from posthog.session_recordings.queries.test.session_replay_sql import produce_replay_summaryfrom posthog.uuidt import uuid7from products.event_definitions.backend.models.property_definition import PropertyType@@ -130,14 +131,44 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):0, # just making sure the query runs)- def test_channel_type(self):+ @parameterized.expand(+ [+ ("gad_source", {"gad_source": "1"}, "Paid Search"),+ ("gclid_from_google", {"gclid": "abc", "$referring_domain": "www.google.com"}, "Paid Search"),+ ("empty_gclid_is_not_a_click", {"gclid": "", "$referring_domain": "www.google.com"}, "Organic Search"),+ ("gad_source_without_referrer", {"gad_source": "1", "$referring_domain": "$direct"}, "Paid Search"),+ ("fbclid_without_referrer", {"fbclid": "xyz", "$referring_domain": "$direct"}, "Organic Social"),+ (+ "unknown_source_stripped_referrer",+ {"utm_source": "newsletter-2026", "$referring_domain": "$direct"},+ "Direct",+ ),+ ("unknown_referrer", {"$referring_domain": "blog.example.org"}, "Referral"),+ (+ "self_referral_with_paid_utms",+ {"$referring_domain": "example.com", "utm_source": "google", "utm_medium": "cpc"},+ "Paid Search",+ ),+ (+ "self_referral_with_unknown_source",+ {"$referring_domain": "example.com", "utm_source": "newsletter-2026"},+ "Referral",+ ),+ (+ "msclkid_only_in_url",+ {"$current_url": "https://example.com/?msclkid=m1", "$referring_domain": "$direct"},+ "Direct",+ ),+ ]+ )+ def test_channel_type(self, _name, properties, expected_channel_type):session_id = str(uuid7())_create_event(event="$pageview",team=self.team,distinct_id="d1",- properties={"gad_source": "1", "$session_id": session_id},+ properties={**properties, "$session_id": session_id},)response = self.__execute(@@ -150,9 +181,105 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):result = (response.results or [])[0]self.assertEqual(result[0],- "Paid Search",+ expected_channel_type,+ )++ @parameterized.expand(+ [+ (+ "utm_only_in_url",+ [("$pageview", 0, {"$current_url": "https://example.com/?utm_source=newsletter&utm_campaign=fall"})],+ (None, None, "Direct"),+ ),+ (+ "first_event_is_not_a_pageview",+ [+ ("$feature_flag_called", 0, {}),+ ("$pageview", 2, {"utm_source": "google", "utm_campaign": "spring"}),+ ],+ ("google", "spring", "Organic Search"),+ ),+ (+ # entry_utm_* skip a NULL on the landing page view and take a later value, while the channel type+ # tuple keeps the landing page view's empty values, so the two disagree for this session+ "utm_added_on_second_pageview",+ [+ ("$pageview", 0, {}),+ ("$pageview", 1, {"utm_source": "google", "utm_medium": "cpc", "utm_campaign": "spring"}),+ ],+ ("google", "spring", "Direct"),+ ),+ (+ "two_campaigns_keep_the_first",+ [+ ("$pageview", 0, {"utm_source": "newsletter", "utm_campaign": "A"}),+ ("$pageview", 30, {"utm_source": "newsletter", "utm_campaign": "B"}),+ ],+ ("newsletter", "A", "Direct"),+ ),+ (+ "sdk_session_entry_properties_are_not_read",+ [("$pageview", 0, {"$session_entry_utm_source": "email"})],+ (None, None, "Direct"),+ ),+ ]+ )+ def test_entry_utm_and_channel_type_across_events(self, _name, events, expected):+ session_id = str(uuid7("2024-03-08T00:00:00+00:00"))+ start = datetime.fromisoformat("2024-03-08T00:00:00+00:00")+ for event, offset_seconds, properties in events:+ _create_event(+ event=event,+ team=self.team,+ distinct_id="d1",+ properties={"$referring_domain": "$direct", **properties, "$session_id": session_id},+ timestamp=start + timedelta(seconds=offset_seconds),+ )++ response = self.__execute(+ parse_select(+ "select $entry_utm_source, $entry_utm_campaign, $channel_type from sessions where session_id = {session_id}",+ placeholders={"session_id": ast.Constant(value=session_id)},+ ),+ )++ self.assertEqual((response.results or [])[0], expected)++ @parameterized.expand([("replay_starts_before_first_pageview", True), ("replay_only", False)])+ def test_replay_rows_shape_the_session(self, _name, with_pageview):+ session_id = str(uuid7("2024-03-08T00:00:00+00:00"))+ produce_replay_summary(+ team_id=self.team.pk,+ distinct_id="d1",+ session_id=session_id,+ first_timestamp="2024-03-08T00:00:00",+ last_timestamp="2024-03-08T00:00:00",+ ensure_analytics_event_in_session=False,+ )+ if with_pageview:+ _create_event(+ event="$pageview",+ team=self.team,+ distinct_id="d1",+ properties={"$session_id": session_id, "$current_url": "https://example.com/"},+ timestamp="2024-03-08T00:00:05",+ )++ response = self.__execute(+ parse_select(+ "select $start_timestamp, $session_duration, $pageview_count, $is_bounce, $has_replay_events from sessions where session_id = {session_id}",+ placeholders={"session_id": ast.Constant(value=session_id)},+ ),)+ start_timestamp, duration, pageview_count, is_bounce, has_replay_events = (response.results or [])[0]+ self.assertEqual(start_timestamp.isoformat()[:19], "2024-03-08T00:00:00")+ self.assertTrue(has_replay_events)+ if with_pageview:+ self.assertEqual((duration, pageview_count, is_bounce), (5, 1, True))+ else:+ self.assertEqual((duration, pageview_count, is_bounce), (0, 0, None))+def test_session_dot_channel_type(self):session_id = str(uuid7())@@ -227,7 +354,7 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):self.assertEqual(response.results or [], [(0, 0, 0)])def test_counts(self):- s1 = str(uuid7())+ s1 = str(uuid7("2023-12-02T00:00:00+00:00"))d1 = "d1"_create_event(event="$pageview",@@ -261,7 +388,7 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):self.assertEqual(response.results or [], [(1, 2, 3)])def test_idempotent_event_counts(self):- s1 = str(uuid7())+ s1 = str(uuid7("2023-12-02T00:00:00+00:00"))d1 = "d1"pageview_uuid = str(uuid.uuid4())autocapture_uuid = str(uuid.uuid4())@@ -303,7 +430,7 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):self.assertEqual(response.results or [], [(1, 1, 1)])def test_page_screen_autocapture_count_up_to(self):- time = time_ns() // (10**6)+ time = int(datetime.fromisoformat("2023-12-02T00:00:00+00:00").timestamp() * 1000)# two pageviewss1 = str(uuid7(time))@@ -407,7 +534,7 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):assert results == [(2, False), (1, True), (1, False), (2, False), (1, False), (0, True)]def test_bounce_rate(self):- time = time_ns() // (10**6)+ time = int(datetime.fromisoformat("2023-12-02T00:00:00+00:00").timestamp() * 1000)# ensure the sessions ids are sortable by giving them different time componentss1a = str(uuid7(time))s1b = str(uuid7(time + 1))@@ -517,7 +644,7 @@ class TestSessionsV3(ClickhouseTestMixin, APIBaseTest):]def test_custom_bounce_rate_duration(self):- time = time_ns() // (10**6)+ time = int(datetime.fromisoformat("2023-12-11T00:00:00+00:00").timestamp() * 1000)# ensure the sessions ids are sortable by giving them different time componentss = str(uuid7(time))@@ -694,50 +821,88 @@ class TestGetLazySessionProperties(ClickhouseTestMixin, APIBaseTest):"$end_hostname","$end_pathname","$end_timestamp",+ "$entry__bhlid","$entry__kx",+ "$entry_awc",+ "$entry_bfcid",+ "$entry_cjevent","$entry_current_url","$entry_dclid",+ "$entry_dicbo","$entry_epik","$entry_fbclid",+ "$entry_gad_campaignid","$entry_gad_source","$entry_gbraid","$entry_gclid","$entry_gclsrc",+ "$entry_has__bhlid","$entry_has__kx",+ "$entry_has_awc",+ "$entry_has_bfcid",+ "$entry_has_cjevent","$entry_has_dclid",+ "$entry_has_dicbo","$entry_has_epik","$entry_has_fbclid",+ "$entry_has_gad_campaignid","$entry_has_gbraid","$entry_has_gclid","$entry_has_gclsrc","$entry_has_igshid",+ "$entry_has_irclickid","$entry_has_irclid","$entry_has_li_fat_id","$entry_has_mc_cid","$entry_has_msclkid",+ "$entry_has_ndclid",+ "$entry_has_oppref","$entry_has_qclid",+ "$entry_has_rdt_cid",+ "$entry_has_rtid","$entry_has_sccid",+ "$entry_has_srsltid",+ "$entry_has_syclid",+ "$entry_has_tblci","$entry_has_ttclid","$entry_has_twclid",+ "$entry_has_utm_creative_format",+ "$entry_has_utm_id",+ "$entry_has_utm_marketing_tactic",+ "$entry_has_utm_source_platform","$entry_has_wbraid",+ "$entry_has_yclid","$entry_hostname","$entry_igshid",+ "$entry_irclickid","$entry_irclid","$entry_li_fat_id","$entry_mc_cid","$entry_msclkid",+ "$entry_ndclid",+ "$entry_oppref","$entry_pathname","$entry_qclid",+ "$entry_rdt_cid","$entry_referring_domain",+ "$entry_rtid","$entry_sccid",+ "$entry_srsltid",+ "$entry_syclid",+ "$entry_tblci","$entry_ttclid","$entry_twclid","$entry_utm_campaign","$entry_utm_content",+ "$entry_utm_creative_format",+ "$entry_utm_id",+ "$entry_utm_marketing_tactic","$entry_utm_medium","$entry_utm_source",+ "$entry_utm_source_platform","$entry_utm_term","$entry_wbraid",+ "$entry_yclid","$is_bounce","$last_external_click_url","$pageview_count",@@ -764,32 +929,45 @@ class TestGetLazySessionProperties(ClickhouseTestMixin, APIBaseTest):def test_source(self):results = get_lazy_session_table_properties_v3("source")self.assertEqual(- results,+ [result["name"] for result in results],[- {- "id": "$entry_utm_source",- "is_numerical": False,- "is_seen_on_filtered_events": None,- "name": "$entry_utm_source",- "property_type": PropertyType.String,- "tags": [],- },- {- "id": "$entry_gad_source",- "is_numerical": False,- "is_seen_on_filtered_events": None,- "name": "$entry_gad_source",- "property_type": PropertyType.String,- "tags": [],- },+ "$entry_utm_source",+ "$entry_gad_source",+ "$entry_utm_source_platform",+ "$entry_has_utm_source_platform",],)+ self.assertEqual(+ results[0],+ {+ "id": "$entry_utm_source",+ "is_numerical": False,+ "is_seen_on_filtered_events": None,+ "name": "$entry_utm_source",+ "property_type": PropertyType.String,+ "tags": [],+ },+ )def test_entry_utm(self):results = get_lazy_session_table_properties_v3("entry utm")self.assertEqual([result["name"] for result in results],- ["$entry_utm_source", "$entry_utm_campaign", "$entry_utm_medium", "$entry_utm_term", "$entry_utm_content"],+ [+ "$entry_utm_source",+ "$entry_utm_campaign",+ "$entry_utm_medium",+ "$entry_utm_term",+ "$entry_utm_content",+ "$entry_utm_id",+ "$entry_has_utm_id",+ "$entry_utm_source_platform",+ "$entry_has_utm_source_platform",+ "$entry_utm_creative_format",+ "$entry_has_utm_creative_format",+ "$entry_utm_marketing_tactic",+ "$entry_has_utm_marketing_tactic",+ ],)def test_can_get_values_for_all(self):diff --git a/posthog/models/raw_sessions/migrations_v3.py b/posthog/models/raw_sessions/migrations_v3.pyindex b105e0c19be5..5191c1109e5b 100644--- a/posthog/models/raw_sessions/migrations_v3.py+++ b/posthog/models/raw_sessions/migrations_v3.py@@ -6,9 +6,11 @@ DROP COLUMN IF EXISTS person_id"""+# These statements replay on fresh clusters against a table created from today's base SQL,+# so an AFTER clause may only name a column that base SQL still creates.ADD_HAS_REPLAY_EVENTS = """ALTER TABLE {table_name}-ADD COLUMN IF NOT EXISTS has_replay_events SimpleAggregateFunction(max, Boolean) AFTER flag_values;+ADD COLUMN IF NOT EXISTS has_replay_events SimpleAggregateFunction(max, Boolean);"""SPLIT_BOUNCE_RATE = """@@ -23,7 +25,7 @@ ADD COLUMN IF NOT EXISTS has_autocapture SimpleAggregateFunction(max, Boolean) AADD_EVENT_NAMES = """ALTER TABLE {table_name}-ADD COLUMN IF NOT EXISTS event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String)) AFTER flag_values+ADD COLUMN IF NOT EXISTS event_names SimpleAggregateFunction(groupUniqArrayArray, Array(String))"""@@ -35,7 +37,7 @@ ADD INDEX IF NOT EXISTS event_names_bloom_filter event_names TYPE bloom_filter()ADD_FLAG_KEYS = """ALTER TABLE {table_name}-ADD COLUMN IF NOT EXISTS flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)) AFTER flag_values+ADD COLUMN IF NOT EXISTS flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String))"""@@ -91,3 +93,21 @@ ADD_FLAG_KEY_VALUES_BLOOM_FILTER = """ALTER TABLE {table_name}ADD INDEX IF NOT EXISTS flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1"""+++DROP_FLAG_KEYS = """+ALTER TABLE {table_name}+DROP COLUMN IF EXISTS flag_keys+"""+++DROP_FLAG_KEYS_BLOOM_FILTER = """+ALTER TABLE {table_name}+DROP INDEX IF EXISTS flag_keys_bloom_filter+"""+++DROP_FLAG_KEY_VALUES_BLOOM_FILTER = """+ALTER TABLE {table_name}+DROP INDEX IF EXISTS flag_key_values_bloom_filter+"""diff --git a/posthog/models/raw_sessions/sessions_v3.py b/posthog/models/raw_sessions/sessions_v3.pyindex 7752a6d1c460..50b5c14f5e93 100644--- a/posthog/models/raw_sessions/sessions_v3.py+++ b/posthog/models/raw_sessions/sessions_v3.py@@ -24,7 +24,7 @@ table should always aggregate again on session_id (the HogQL session table willdon't need to consider this).Upgrades over v2:-* Has a property map for storing lower-tier ad ids, making it easier to add new ad ids in the future+* Stores lower-tier ad ids as a flat 'key=value' array, making it easier to add new ad ids in the future* Stores presence of ad ids separately from the value, so e.g. channel type calculations only need to read 1 bit instead of a gclid string up to 100 chars* Parses JSON only once per event rather than once per column per event, saving CPU usage* Removes a lot of deprecated fields that are no longer used@@ -148,8 +148,8 @@ CREATE TABLE IF NOT EXISTS {table_name}entry_has_gclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),entry_has_fbclid AggregateFunction(argMin, Boolean, DateTime64(6, 'UTC')),- -- for lower-tier ad ids, just put them in a map, and set of the ones present- entry_ad_ids_map AggregateFunction(argMin, Map(String, String), DateTime64(6, 'UTC')),+ -- lower-tier ad ids as a flat 'key=value' array - like flag_key_values, a per-key map+ -- state made merges much more expensiveentry_ad_ids_set AggregateFunction(argMin, Array(String), DateTime64(6, 'UTC')),-- channel type properties tuple - to reduce redundant reading of the timestamp when loading all of these columns@@ -157,12 +157,13 @@ CREATE TABLE IF NOT EXISTS {table_name}entry_channel_type_properties AggregateFunction(argMin, Tuple(Nullable(String), Nullable(String), Nullable(String), Nullable(String), Boolean, Boolean, Nullable(String)), DateTime64(6, 'UTC')),-- Count pageview, autocapture, and screen events for providing totals.- -- Use uniqExact instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see+ -- Use uniq instead of count, so that inserting events can be idempotent. This is necessary as sometimes we see-- events being inserted multiple times to be deduped later, but that can trigger multiple rows here.-- Additionally, idempotency is useful for backfilling, as we can just reinsert the same events without worrying.- pageview_uniq AggregateFunction(uniqExact, Nullable(UUID)),- autocapture_uniq AggregateFunction(uniqExact, Nullable(UUID)),- screen_uniq AggregateFunction(uniqExact, Nullable(UUID)),+ -- uniq (not uniqExact) keeps the state small; per-session counts are tiny so it is exact in practice.+ pageview_uniq AggregateFunction(uniq, Nullable(UUID)),+ autocapture_uniq AggregateFunction(uniq, Nullable(UUID)),+ screen_uniq AggregateFunction(uniq, Nullable(UUID)),-- As a performance optimisation, also keep track of the uniq events for all of these combined.-- This is a much more efficient way of calculating the bounce rate, as >2 means not a bounce@@ -171,8 +172,8 @@ CREATE TABLE IF NOT EXISTS {table_name}-- Flags - store every seen 'key=value' pair per flag. A flat array instead of a-- groupUniqArrayMap state because per-key sub-aggregators made merges ~8-37x more expensive.+ -- Key-only searches use the text index below (tokenized on '='), so no separate keys column.flag_key_values SimpleAggregateFunction(groupUniqArrayArray(10000), Array(String)),- flag_keys SimpleAggregateFunction(groupUniqArrayArray, Array(String)),-- Event names - store unique event names seen in this sessionevent_names SimpleAggregateFunction(groupUniqArrayArray(2000), Array(String)),@@ -207,8 +208,7 @@ def SHARDED_RAW_SESSIONS_TABLE_SQL_V3():-- IndexesINDEX event_names_bloom_filter event_names TYPE bloom_filter() GRANULARITY 1,- INDEX flag_key_values_bloom_filter flag_key_values TYPE bloom_filter() GRANULARITY 1,- INDEX flag_keys_bloom_filter flag_keys TYPE bloom_filter() GRANULARITY 1,+ INDEX flag_key_values_text flag_key_values TYPE text(tokenizer = splitByString(['='])),INDEX hosts_bloom_filter hosts TYPE bloom_filter() GRANULARITY 1,INDEX emails_bloom_filter emails TYPE bloom_filter() GRANULARITY 1) ENGINE = {engine}""",@@ -247,10 +247,29 @@ def ALTER_SHARDED_RAW_SESSIONS_TABLE_SETTINGS_V3():# Re-exported from the Django-free posthog.raw_sessions_v3_ad_ids module so the HogQL schema can# use it without booting Django; kept importable here for existing callers.-from posthog.raw_sessions_v3_ad_ids import SESSION_V3_LOWER_TIER_AD_IDS # noqa: E402+from posthog.raw_sessions_v3_ad_ids import SESSION_V3_LOWER_TIER_AD_IDS, session_ad_id_url_params # noqa: E402new_line = "\n"++def _ad_id_value_sql(ad_id: str) -> str:+ # The SDKs only promote a fixed set of URL params to event properties, so fall back to+ # parsing the event's URL — this is what lets a new ad id apply to backfilled history.+ # The lookups read _attribution_url, which is empty outside $pageview/$screen, so the+ # per-event string scans stay off the bulk of the ingestion hot path.+ # Form decoding, not plain: a query string spells a space as '+', and the SDK decodes it that+ # way before promoting the property, so plain decoding would split one campaign into two values.+ # left() caps what a client can pad into the URL; observed click IDs stay under ~120 chars+ # toValidUTF8 wraps the truncation because both inner steps can emit invalid UTF-8:+ # decodeURLFormComponent turns '%FF' into a raw byte, and left() counts bytes so it can cut a+ # multi-byte character in half. Stored bytes then break every reader that expects text.+ url_lookups = ", ".join(+ f"nullIf(toValidUTF8(left(decodeURLFormComponent(extractURLParameter(_attribution_url, '{param}')), 2048)), '')"+ for param in session_ad_id_url_params(ad_id)+ )+ return f"coalesce(nullIf(tupleElement(p, '{ad_id}'), ''), {url_lookups}) as {ad_id}"++# See https://kb.altinity.com/altinity-kb-queries-and-syntax/jsonextract-to-parse-many-attributes-at-a-time/# Or https://posthog.slack.com/archives/C02JQ320FV3/p1721406540313379?thread_ts=1721334861.073739&cid=C02JQ320FV3PROPERTIES = f"""@@ -267,7 +286,7 @@ PROPERTIES = f"""`$geoip_country_code` Nullable(String),`$geoip_subdivision_1_code` Nullable(String),`$geoip_subdivision_1_name` Nullable(String),- `$geoip_subdivision_city_name` Nullable(String),+ `$geoip_city_name` Nullable(String),`$geoip_time_zone` Nullable(String),`$referring_domain` Nullable(String),`utm_source` Nullable(String),@@ -283,6 +302,7 @@ PROPERTIES = f""")') as p,JSONExtractString(person_properties, 'email') as _person_email,tupleElement(p, '$current_url') as _current_url,+ if(event = '$pageview' OR event = '$screen', left(coalesce(_current_url, ''), 16384), '') as _attribution_url,tupleElement(p, '$external_click_url') as _external_click_url,tupleElement(p, '$browser') as _browser,tupleElement(p, '$browser_version') as _browser_version,@@ -294,7 +314,7 @@ PROPERTIES = f"""tupleElement(p, '$geoip_country_code') as _geoip_country_code,tupleElement(p, '$geoip_subdivision_1_code') as _geoip_subdivision_1_code,tupleElement(p, '$geoip_subdivision_1_name') as _geoip_subdivision_1_name,- tupleElement(p, '$geoip_subdivision_city_name') as _geoip_subdivision_city_name,+ tupleElement(p, '$geoip_city_name') as _geoip_subdivision_city_name,tupleElement(p, '$geoip_time_zone') as _geoip_time_zone,tupleElement(p, '$referring_domain') as _referring_domain,tupleElement(p, 'utm_source') as _utm_source,@@ -305,12 +325,9 @@ PROPERTIES = f"""tupleElement(p, 'gclid') as _gclid,tupleElement(p, 'gad_source') as _gad_source,tupleElement(p, 'fbclid') as _fbclid,-{f",{new_line}".join([f" tupleElement(p, '{ad_id}') as {ad_id}" for ad_id in SESSION_V3_LOWER_TIER_AD_IDS])},- CAST(mapFilter((k, v) -> v IS NOT NULL, map(-{f",{new_line}".join([f" '{ad_id}', {ad_id}" for ad_id in SESSION_V3_LOWER_TIER_AD_IDS])}- )) AS Map(String, String)) as ad_ids_map,+{f",{new_line}".join([f" {_ad_id_value_sql(ad_id)}" for ad_id in SESSION_V3_LOWER_TIER_AD_IDS])},CAST(arrayFilter(x -> x IS NOT NULL, [-{f",{new_line}".join([f" if({ad_id} IS NOT NULL, '{ad_id}', NULL)" for ad_id in SESSION_V3_LOWER_TIER_AD_IDS])}+{f",{new_line}".join([f" if({ad_id} IS NOT NULL, concat('{ad_id}=', {ad_id}), NULL)" for ad_id in SESSION_V3_LOWER_TIER_AD_IDS])}]) AS Array(String)) as ad_ids_set,tupleElement(p, '$host') as _host"""@@ -368,21 +385,21 @@ SELECTinitializeAggregation('argMinState', _fbclid, pageview_prio_timestamp_min) as entry_fbclid,-- has gclid/fbclid for reading fewer bytes when calculating channel type- initializeAggregation('argMinState', _gclid IS NOT NULL, pageview_prio_timestamp_min) as entry_has_gclid,- initializeAggregation('argMinState', _fbclid IS NOT NULL, pageview_prio_timestamp_min) as entry_has_fbclid,+ -- clients send "gclid": "" and that is not a click id, so an empty value must not count as present+ initializeAggregation('argMinState', nullIf(_gclid, '') IS NOT NULL, pageview_prio_timestamp_min) as entry_has_gclid,+ initializeAggregation('argMinState', nullIf(_fbclid, '') IS NOT NULL, pageview_prio_timestamp_min) as entry_has_fbclid,-- other ad ids- initializeAggregation('argMinState', ad_ids_map, pageview_prio_timestamp_min) as entry_ad_ids_map,initializeAggregation('argMinState', ad_ids_set, pageview_prio_timestamp_min) as entry_ad_ids_set,-- channel type- initializeAggregation('argMinState', tuple(_utm_source, _utm_medium, _utm_campaign, _referring_domain, _gclid IS NOT NULL, _fbclid IS NOT NULL, _gad_source), pageview_prio_timestamp_min) as entry_channel_type_properties,+ initializeAggregation('argMinState', tuple(_utm_source, _utm_medium, _utm_campaign, _referring_domain, nullIf(_gclid, '') IS NOT NULL, nullIf(_fbclid, '') IS NOT NULL, _gad_source), pageview_prio_timestamp_min) as entry_channel_type_properties,-- counts- initializeAggregation('uniqExactState', if(event='$pageview', uuid, NULL)) as pageview_uniq,- initializeAggregation('uniqExactState', if(event='$autocapture', uuid, NULL)) as autocapture_uniq,- initializeAggregation('uniqExactState', if(event='$screen', uuid, NULL)) as screen_uniq,+ initializeAggregation('uniqState', if(event='$pageview', uuid, NULL)) as pageview_uniq,+ initializeAggregation('uniqState', if(event='$autocapture', uuid, NULL)) as autocapture_uniq,+ initializeAggregation('uniqState', if(event='$screen', uuid, NULL)) as screen_uniq,-- perfinitializeAggregation('uniqUpToState(1)', if(event='$pageview' OR event='$screen', uuid, NULL)) as page_screen_uniq_up_to,@@ -390,7 +407,6 @@ SELECT-- flagsarrayMap((k, v) -> concat(k, '=', v), mapKeys(properties_group_feature_flags), mapValues(properties_group_feature_flags)) as flag_key_values,- mapKeys(properties_group_feature_flags) as flag_keys,-- event names[event] as event_names,@@ -404,6 +420,11 @@ SELECTfalse as has_replay_eventsFROM {source_table} AS source_tableWHERE bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(`$session_id`, 'UUID')), 76), 0xF) == 7 -- has a session id and is valid uuidv7+-- Clients can send a UUIDv7 session id with a garbage timestamp (1970, year 6339). Each one puts its session in its+-- own monthly partition, and its events stretch the session duration across months. Keep only events within 31 days+-- of the session id's timestamp. The bound is symmetric because the client mints the session id with its own clock+-- while the server corrects event timestamps, so a client clock that runs days ahead is still a real session.+AND fromUnixTimestamp64Milli(toUInt64(bitShiftRight(`$session_id_uuid`, 80))) BETWEEN timestamp - INTERVAL 31 DAY AND timestamp + INTERVAL 31 DAYAND {where}""".format(source_table=source_table,@@ -589,16 +610,15 @@ SELECTinitializeAggregation('argMinState', false, max_ts_64) as entry_has_fbclid,-- other ad ids- initializeAggregation('argMinState', CAST(map(), 'Map(String, String)'), max_ts_64) as entry_ad_ids_map,initializeAggregation('argMinState', CAST([], 'Array(String)'), max_ts_64) as entry_ad_ids_set,-- channel typeinitializeAggregation('argMinState', tuple(null_s, null_s, null_s, null_s, false, false, null_s), max_ts_64) as entry_channel_type_properties,-- counts- initializeAggregation('uniqExactState', null_uuid) as pageview_uniq,- initializeAggregation('uniqExactState', null_uuid) as autocapture_uniq,- initializeAggregation('uniqExactState', null_uuid) as screen_uniq,+ initializeAggregation('uniqState', null_uuid) as pageview_uniq,+ initializeAggregation('uniqState', null_uuid) as autocapture_uniq,+ initializeAggregation('uniqState', null_uuid) as screen_uniq,-- perfinitializeAggregation('uniqUpToState(1)', null_uuid) as page_screen_uniq_up_to,@@ -606,7 +626,6 @@ SELECT-- flagsCAST([], 'Array(String)') as flag_key_values,- CAST([], 'Array(String)') as flag_keys,-- event namesCAST([], 'Array(String)') as event_names,@@ -621,6 +640,9 @@ SELECTtrue as has_replay_eventsFROM {source_table} AS source_tableWHERE bitAnd(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 76), 0xF) == 7 -- has a session id and is valid uuidv7+-- same session timestamp guard as the events SELECT. accurateCastOrNull because accurateCast throws on a malformed+-- session_id, and ClickHouse can evaluate this condition before the uuidv7 check above filters those rows out+AND fromUnixTimestamp64Milli(toUInt64(bitShiftRight(toUInt128(accurateCastOrNull(session_id, 'UUID')), 80))) BETWEEN min_first_timestamp - INTERVAL 31 DAY AND min_first_timestamp + INTERVAL 31 DAYAND {where}""".format(source_table=source_table,@@ -815,15 +837,14 @@ SELECTargMinMerge(entry_has_gclid) as entry_has_gclid,argMinMerge(entry_has_fbclid) as entry_has_fbclid,- argMinMerge(entry_ad_ids_map) as entry_ad_ids_map,argMinMerge(entry_ad_ids_set) as entry_ad_ids_set,argMinMerge(entry_channel_type_properties) as entry_channel_type_properties,-- counts- uniqExactMerge(pageview_uniq) as pageview_uniq,- uniqExactMerge(autocapture_uniq) as autocapture_uniq,- uniqExactMerge(screen_uniq) as screen_uniq,+ uniqMerge(pageview_uniq) as pageview_uniq,+ uniqMerge(autocapture_uniq) as autocapture_uniq,+ uniqMerge(screen_uniq) as screen_uniq,-- perfuniqUpToMerge(1)(page_screen_uniq_up_to) as page_screen_uniq_up_to,@@ -831,7 +852,6 @@ SELECT-- flagsgroupUniqArrayArray(10000)(flag_key_values) as flag_key_values,- groupUniqArrayArray(flag_keys) as flag_keys,-- event namesgroupUniqArrayArray(2000)(event_names) as event_names,diff --git a/posthog/raw_sessions_v3_ad_ids.py b/posthog/raw_sessions_v3_ad_ids.pyindex bb5e36f4c265..a4a08e4136ec 100644--- a/posthog/raw_sessions_v3_ad_ids.py+++ b/posthog/raw_sessions_v3_ad_ids.py@@ -1,22 +1,60 @@+import re+# Django-free home for the raw-sessions-v3 lower-tier ad-id list. Kept out of# posthog.models.raw_sessions.sessions_v3 (whose package __init__ boots the Django ORM) so the# HogQL sessions_v3 schema table can build its field catalog at import without booting Django.# posthog.models.raw_sessions.sessions_v3 re-exports this for existing callers.SESSION_V3_LOWER_TIER_AD_IDS = [- "gclsrc",- "dclid",- "gbraid",- "wbraid",- "msclkid",- "twclid",- "li_fat_id",- "mc_cid",- "igshid",- "ttclid",- "epik",- "qclid",- "sccid",- "_kx",- "irclid",+ "gclsrc", # google ads 360+ "dclid", # google display ads+ "gbraid", # google ads, web to app+ "wbraid", # google ads, app to web+ "gad_campaignid", # google ads campaign id+ "srsltid", # google merchant center auto-tagging+ "msclkid", # microsoft+ "twclid", # twitter+ "li_fat_id", # linkedin+ "mc_cid", # mailchimp campaign id+ "igshid", # instagram+ "ttclid", # tiktok+ "rdt_cid", # reddit+ "epik", # pinterest+ "qclid", # quora+ "sccid", # snapchat+ "_kx", # klaviyo+ "irclid", # impact (legacy spelling, kept for property parity)+ "irclickid", # impact+ "utm_id", # GA4 campaign id+ "utm_source_platform", # GA4+ "utm_creative_format", # GA4+ "utm_marketing_tactic", # GA4+ "yclid", # yandex+ "tblci", # taboola+ "dicbo", # outbrain+ "ndclid", # nextdoor+ "cjevent", # cj affiliate+ "awc", # awin+ "syclid", # shopify+ "_bhlid", # beehiiv+ "rtid", # rokt+ "oppref", # openai ads+ "bfcid", # freebuff]++# URL query-string spellings that differ from the canonical property key. Snapchat documents+# all three casings for its click ID (https://businesshelp.snapchat.com/s/article/click-id).+SESSION_V3_AD_ID_URL_ALIASES: dict[str, list[str]] = {+ "sccid": ["sccid", "ScCid", "SsCid"],+}+++def session_ad_id_url_params(ad_id: str) -> list[str]:+ return SESSION_V3_AD_ID_URL_ALIASES.get(ad_id, [ad_id])+++# Every entry is interpolated into ClickHouse DDL as a quoted literal and a bare alias, so a+# stray quote or metacharacter in a future entry would become injected DDL. Fail at import.+for _param in SESSION_V3_LOWER_TIER_AD_IDS + [p for aliases in SESSION_V3_AD_ID_URL_ALIASES.values() for p in aliases]:+ if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", _param):+ raise ValueError(f"ad id {_param!r} is not a safe SQL identifier")