Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(hogql): restore recorded nulls in native documents (#114299)

Sandy Spicer81096e07e80a
diff --git a/docs/internal/clickhouse-utility-udfs.md b/docs/internal/clickhouse-utility-udfs.md
index ec5d7e634d72..f7dd914e4b95 100644
--- a/docs/internal/clickhouse-utility-udfs.md
+++ b/docs/internal/clickhouse-utility-udfs.md
@@ -54,7 +54,9 @@ One call per event row for the native JSON events table. It takes the raw event
| `temporary_properties_null_keys` | `Array(String)` | The same for `temporary_properties` |
| `person_properties_null_keys` | `Array(String)` | The same for `person_properties` |
-The typed JSON column cannot store `null`, so the cleaners drop null fields. The null-key arrays keep the difference between a property sent as `null` and one never sent. No reader uses them yet: queries, filters and exports treat a null property the same as a missing one, and the arrays keep the data so that a later change can restore the nulls. A path joins the object keys with `.` and writes array positions as numbers: `{"items":[{"a":null}]}` records `items.0.a`. A dot inside one key is written as `%2E`, the spelling `json_type_escape_dots_in_keys` uses, so `{"user":{"plan":null},"a.b":null}` records `user.plan` and `a%2Eb`. A path that still holds a value after duplicate handling is not recorded, so `{"a":null,"a":1}` records nothing. Nulls inside arrays are not object fields; they stay in the array. The paths of one document can use at most 256 KiB. If they need more, that document records no null keys, and its cleaned properties are unchanged. A path repeats every key above it, so without this limit a small document with a long key and many null children could produce gigabytes of paths.
+The typed JSON column cannot store `null`, so the cleaners drop null fields. The null-key arrays keep the difference between a property sent as `null` and one never sent. Whole-document reads (HogQL `SELECT properties` and `person.properties`, the events API, the native batch export source) restore recorded `null` fields in documents whose structure the cleaner preserves, using the shared builder in `posthog/hogql/printer/events_json_document.py`. Reads of one key still see a field sent as `null` as missing: `JSONHas(properties, 'plan')` returns 0 for it, and an object read on its own, such as `properties.$set`, leaves out its null fields where the legacy table keeps them. A path joins the object keys with `.` and writes array positions as numbers: `{"items":[{"a":null}]}` records `items.0.a`. A dot inside one key is written as `%2E`, the spelling `json_type_escape_dots_in_keys` uses, so `{"user":{"plan":null},"a.b":null}` records `user.plan` and `a%2Eb`. A `%` is not escaped, so a key sent with a literal `%2E` reads back with a dot, the same as the JSON column returns a stored value under that key. A path that still holds a value after duplicate handling is not recorded, so `{"a":null,"a":1}` records nothing. Nulls inside arrays are not object fields; they stay in the array. The paths of one document can use at most 256 KiB. If they need more, that document records no null keys, and its cleaned properties are unchanged. A path repeats every key above it, so without this limit a small document with a long key and many null children could produce gigabytes of paths.
+
+The native export's separate `$set`, `$set_once`, `$unset` and `$group_set` columns read from the rebuilt document too. HTTP exports send that document without overwriting those fields. Cleaner transformations can leave a recorded path pointing into a different shape: for example, an `$exception_list` object becomes an array, and null fields recorded under the original object are not restored. The reader keeps the cleaned array intact. Reconstruction expands affected objects and arrays one level at a time, grouping their null paths and preserving finished JSON fragments. Sibling arrays therefore share the traversal of their parent instead of each rebuilding it. The reader restores all recorded paths that match the stored structure; ClickHouse resource limits fail the query rather than returning a document with nulls omitted.
The function never fails. Blank input becomes `{}`. Malformed, non-object, or too-deep input comes back quarantined under `$unparseable_properties` in that document, with `{}` temporary properties and no null keys, so the calling SQL needs no `isValidJSON` guards. Rows travel as `JSONEachRow` in both directions and the executable pool reads a chunk header before each batch.
diff --git a/posthog/api/test/__snapshots__/test_query.new_events_schema.ambr b/posthog/api/test/__snapshots__/test_query.new_events_schema.ambr
index 407ac417d0a0..5d5d1a1142bc 100644
--- a/posthog/api/test/__snapshots__/test_query.new_events_schema.ambr
+++ b/posthog/api/test/__snapshots__/test_query.new_events_schema.ambr
@@ -513,28 +513,69 @@
# name: TestQuery.test_select_hogql_expressions[new_events_schema.1]
'''
/* user_id:0 request:_snapshot_ */
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`,
- events.event AS event
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`,
+ events.event AS event
FROM events_json AS events
- WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
- (SELECT events.uuid AS uuid
- FROM events_json AS events
- WHERE and(equals(events.team_id, 99999), less(events.timestamp, toDateTime64('2020-01-10 12:14:05.000000', 6, 'UTC')), greater(events.timestamp, toDateTime64('2020-01-09 12:14:00.000000', 6, 'UTC')))
- ORDER BY tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) ASC
- LIMIT 101)), and(less(events.timestamp, toDateTime64('2020-01-10 12:14:05.000000', 6, 'UTC')), greater(events.timestamp, toDateTime64('2020-01-09 12:14:00.000000', 6, 'UTC'))))
- ORDER BY tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) ASC
+ WHERE and(equals(events.team_id, 99999),
+ globalIn(events.uuid,
+ (SELECT events.uuid AS uuid
+ FROM events_json AS events
+ WHERE and(equals(events.team_id, 99999), less(events.timestamp, toDateTime64('2020-01-10 12:14:05.000000', 6, 'UTC')), greater(events.timestamp, toDateTime64('2020-01-09 12:14:00.000000', 6, 'UTC')))
+ ORDER BY tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) ASC
+ LIMIT 101)),
+ and(less(events.timestamp, toDateTime64('2020-01-10 12:14:05.000000', 6, 'UTC')), greater(events.timestamp, toDateTime64('2020-01-09 12:14:00.000000', 6, 'UTC'))))
+ ORDER BY tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) ASC
LIMIT 101
OFFSET 0 SETTINGS readonly=2,
max_execution_time=60,
diff --git a/posthog/api/test/test_event.py b/posthog/api/test/test_event.py
index 7964c0b4f458..d13044f3be96 100644
--- a/posthog/api/test/test_event.py
+++ b/posthog/api/test/test_event.py
@@ -963,7 +963,15 @@ class TestEvents(ClickhouseTestMixin, APIBaseTest):
distinct_ids=["1"],
is_identified=True,
)
- event_id = _create_event(team=self.team, event="event", distinct_id="1", timestamp=timezone.now())
+ sent_properties = {
+ "plan": None,
+ "billing": {"coupon": None, "seats": 3},
+ "items": [{"sku": "a", "note": None}],
+ "$set": {"email": None, "name": "n"},
+ }
+ event_id = _create_event(
+ team=self.team, event="event", distinct_id="1", timestamp=timezone.now(), properties=sent_properties
+ )
flush_persons_and_events()
response = self.client.get(f"/api/projects/{self.team.id}/events/{event_id}")
@@ -971,6 +979,7 @@ class TestEvents(ClickhouseTestMixin, APIBaseTest):
response_json = response.json()
assert response_json["event"] == "event"
assert response_json["person"] is None
+ assert response_json["properties"] == sent_properties
with_person_response = self.client.get(f"/api/projects/{self.team.id}/events/{event_id}?include_person=true")
assert with_person_response.status_code == status.HTTP_200_OK
diff --git a/posthog/clickhouse/events_json.py b/posthog/clickhouse/events_json.py
index 8d0d65f5a60c..39f0c1f19a0e 100644
--- a/posthog/clickhouse/events_json.py
+++ b/posthog/clickhouse/events_json.py
@@ -14,6 +14,13 @@ DISTRIBUTED_EVENTS_JSON_TABLE = "events_json"
KAFKA_EVENTS_NATIVE_JSON_TABLE = "kafka_events_json_native_json"
UNPARSEABLE_PROPERTIES_KEY = "$unparseable_properties"
TEMPORARY_PROPERTIES_COLUMN = "temporary_properties"
+# The JSON type cannot store null, so the cleaner drops each null field of a JSON column and records its path in the
+# column's companion array.
+NULL_KEYS_COLUMNS = {
+ "properties": "properties_null_keys",
+ TEMPORARY_PROPERTIES_COLUMN: "temporary_properties_null_keys",
+ "person_properties": "person_properties_null_keys",
+}
# Mirrors isTemporaryProperty in clickhouse-udfs/util/cmd/json_clean_posthog_event_properties_udf/main.go, so update both.
TEMPORARY_EVENT_PROPERTY_ROOTS = frozenset(
diff --git a/posthog/hogql/printer/clickhouse.py b/posthog/hogql/printer/clickhouse.py
index 117932ad927a..281d8a838ff1 100644
--- a/posthog/hogql/printer/clickhouse.py
+++ b/posthog/hogql/printer/clickhouse.py
@@ -52,7 +52,7 @@ from posthog.hogql.type_system import infer_tuple_access_constant_type, parse_sq
from posthog.hogql.visitor import GetFieldsTraverser, clone_expr
from posthog.clickhouse.client.escape import substitute_params
-from posthog.clickhouse.events_json import TEMPORARY_PROPERTIES_COLUMN
+from posthog.clickhouse.events_json import NULL_KEYS_COLUMNS, TEMPORARY_PROPERTIES_COLUMN
from posthog.exchange_rate_constants import EXCHANGE_RATE_DECIMAL_PRECISION, EXCHANGE_RATE_DICTIONARY_NAME
from posthog.uuidt import UUIDT
from posthog.week_start_day import WeekStartDay
@@ -691,16 +691,24 @@ class ClickHousePrinter(BasePrinter):
if resolved_field is None:
return None
+ prefix = field_sql.removesuffix(self._print_identifier(resolved_field.name))
+ null_keys_sql = f"{prefix}{self._print_identifier(NULL_KEYS_COLUMNS[resolved_field.name])}"
if resolved_field.name == "properties":
- prefix = field_sql.removesuffix(self._print_identifier("properties"))
temporary_properties_sql = f"{prefix}{self._print_identifier(TEMPORARY_PROPERTIES_COLUMN)}"
+ temporary_null_keys_sql = (
+ f"{prefix}{self._print_identifier(NULL_KEYS_COLUMNS[TEMPORARY_PROPERTIES_COLUMN])}"
+ )
serialized = event_document_sql(
- field_sql, temporary_properties_sql, self._document_feature_flags_sql(type, field_sql)
+ field_sql,
+ null_keys_sql,
+ temporary_properties_sql,
+ temporary_null_keys_sql,
+ self._document_feature_flags_sql(type, field_sql),
)
elif resolved_field.name == "person_properties":
- serialized = person_document_sql(field_sql)
+ serialized = person_document_sql(field_sql, null_keys_sql)
else:
- serialized = json_document_sql(field_sql)
+ serialized = json_document_sql(field_sql, null_keys_sql)
return serialized
def _document_feature_flags_sql(self, type: ast.FieldType, field_sql: str) -> str | None:
diff --git a/posthog/hogql/printer/events_json_document.py b/posthog/hogql/printer/events_json_document.py
index 113102cd90d6..71bf3369c1c5 100644
--- a/posthog/hogql/printer/events_json_document.py
+++ b/posthog/hogql/printer/events_json_document.py
@@ -2,12 +2,13 @@
The cleaner splits event properties at insert: each `$feature/<key>` value goes into the typed `$feature_flags` map,
`$active_feature_flags` is dropped, and the keys `is_temporary_event_property` names go into `temporary_properties`,
-which ClickHouse clears 60 days after insertion. A whole-document read puts those parts back so the document matches
-the legacy table while the parts exist. The HogQL printer and the batch export source share this builder so every
-whole-document reader returns the same document.
+which ClickHouse clears 60 days after insertion. The JSON type cannot store `null`, so the cleaner also drops every
+null field and records its path in the column's null-key array. A whole-document read puts those parts back so the
+document matches the legacy table while the parts exist. The HogQL printer and the batch export source share this
+builder so every whole-document reader returns the same document.
"""
-from collections.abc import Iterable
+from collections.abc import Callable, Iterable
from posthog.hogql.constants import (
FEATURE_FLAG_PROPERTY_PREFIX,
@@ -18,6 +19,12 @@ from posthog.hogql.escape_sql import escape_clickhouse_string
from posthog.clickhouse.events_json import EVENTS_PROPERTIES_JSON_SUBCOLUMNS, PERSON_PROPERTIES_JSON_SUBCOLUMNS
+_FEATURE_FLAGS_KEY = "$feature_flags"
+# A null-key path joins object keys and array positions with `.` and writes a dot inside one key as `%2E`. The SQL
+# spells that `%` as the escape `\x25`, because a printed query goes through %-style parameter substitution.
+_ENCODED_DOT_SQL = "'\\x252E'"
+_NO_PATHS_SQL = "CAST([], 'Array(Array(String))')"
+
def declared_array_paths(subcolumns: dict[str, str]) -> tuple[str, ...]:
"""The declared paths of a JSON column whose type is an array."""
@@ -38,18 +45,118 @@ EVENTS_PROPERTIES_DECLARED_STRING_PATHS = declared_string_paths(EVENTS_PROPERTIE
PERSON_PROPERTIES_DECLARED_STRING_PATHS = declared_string_paths(PERSON_PROPERTIES_JSON_SUBCOLUMNS)
+def _decoded_key_sql(segment_sql: str) -> str:
+ return f"replaceAll({segment_sql}, {_ENCODED_DOT_SQL}, '.')"
+
+
+def _bind_sql(values: dict[str, str], body_sql: str) -> str:
+ """`body_sql` with each name in `values` bound to its SQL value.
+
+ ClickHouse has no local variables, so this maps a lambda over one-element arrays. A value used more than once
+ is then printed and planned once.
+ """
+ names = ", ".join(values)
+ arrays = ", ".join(f"[{value}]" for value in values.values())
+ return f"arrayMap(({names}) -> {body_sql}, {arrays})[1]"
+
+
+def _runs_sql(sorted_sql: str, key: Callable[[str], str], item: str) -> str:
+ """The sorted array `sorted_sql` split into runs of items with the same `key`, which maps an item to SQL.
+
+ ClickHouse copies an array that a lambda reads from outside its parameters once per element it maps, so
+ grouping by filtering the whole array for each key costs the square of the null count. Sorting and splitting
+ where the key changes stays near linear. `item` names the lambda parameters, which must not shadow an outer one.
+ """
+ before = f"{item}_before"
+ items = f"{item}_sorted"
+ split = (
+ f"arraySplit(({item}, {before}) -> {key(item)} != {key(before)}, {items}, "
+ f"arrayPopBack(arrayPushFront({items}, {items}[1])))"
+ )
+ return _bind_sql({items: sorted_sql}, split)
+
+
+def _top_level_null_keys_sql(null_keys_sql: str) -> str:
+ """The keys of the members sent as `null`, which are the recorded paths of one segment."""
+ return f"arrayMap(null_key -> {_decoded_key_sql('null_key')}, arrayFilter(null_key -> position(null_key, '.') = 0, {null_keys_sql}))"
+
+
+def _nested_null_paths_sql(null_keys_sql: str) -> str:
+ """Each recorded path of more than one segment as its list of segments."""
+ return (
+ f"arrayMap(null_path -> arrayMap(null_segment -> {_decoded_key_sql('null_segment')}, splitByChar('.', null_path)), "
+ f"arrayFilter(null_path -> position(null_path, '.') > 0, {null_keys_sql}))"
+ )
+
+
+def _restore_nulls_sql(value_sql: str, paths_sql: str) -> str:
+ """Expand affected containers one level at a time, then join their JSON fragments.
+
+ Each node carries (JSON text, remaining null paths). Finished fragments have no paths. Processing siblings
+ together reads each container once per depth instead of rebuilding their common parent for every array.
+ ClickHouse query limits apply normally; reconstruction never discards recorded nulls to meet a work budget.
+ """
+ is_array = "null_array"
+ root_runs = _runs_sql("arraySort(null_node.2)", lambda path: f"{path}[1]", "null_tail")
+ roots = (
+ "arrayMap(null_root -> ("
+ f"if({is_array}, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], "
+ "toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), "
+ f"if({is_array}, '', null_root[1][1]), 1, '', "
+ f"arrayMap(null_path -> arrayPopFront(null_path), null_root)), {root_runs})"
+ )
+ members = (
+ f"if({is_array}, "
+ "arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, "
+ f"{_NO_PATHS_SQL}), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), "
+ f"arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, {_NO_PATHS_SQL}), JSONExtractKeysAndValuesRaw(null_node.1)))"
+ )
+ runs = _runs_sql(
+ f"arraySort(arrayConcat({members}, {roots}))", lambda entry: f"({entry}.1, {entry}.2)", "null_child"
+ )
+ runs = f"arrayFilter(null_run -> not {is_array} or null_run[1].3 = 0, {runs})"
+ children = _bind_sql(
+ {"null_children": runs},
+ "arrayFlatten(arrayMap((null_run, null_index) -> ["
+ "(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), "
+ f"{_NO_PATHS_SQL}), "
+ "(null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children)))",
+ )
+ expanded = (
+ f"arrayConcat([(if({is_array}, '[', '{{'), {_NO_PATHS_SQL})], "
+ f"{children}, [(if({is_array}, ']', '}}'), {_NO_PATHS_SQL})])"
+ )
+ expanded = _bind_sql({"null_array": "startsWith(null_node.1, '[')"}, expanded)
+ node = (
+ "multiIf(empty(null_node.2), [null_node], "
+ f"empty(null_node.2[1]), [('null', {_NO_PATHS_SQL})], "
+ "null_node.1 = '' or startsWith(null_node.1, '{') or startsWith(null_node.1, '['), "
+ f"{expanded}, [(null_node.1, {_NO_PATHS_SQL})])"
+ )
+ return _bind_sql(
+ {"null_paths": paths_sql},
+ "arrayStringConcat(arrayFold((null_nodes, null_depth) -> "
+ f"arrayFlatten(arrayMap(null_node -> {node}, null_nodes)), "
+ f"range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [({value_sql}, null_paths)]).1)",
+ )
+
+
def json_member_pairs_sql(
document_sql: str,
+ null_keys_sql: str,
*,
exclude_key: str | None = None,
declared_array_paths: Iterable[str] = (),
declared_string_paths: Iterable[str] = (),
) -> str:
- """`"key":<raw json>` strings for the members of a JSON column.
+ """`"key":<raw json>` strings for the members of a JSON column, with the fields sent as `null` put back.
The JSON type materializes a declared path on every row, as `[]` for an array and `''` for a string, so under
those keys an empty value means the key was absent and the pair is left out. An empty array or string under any
- other key was sent and stays; the cleaner already removed nulls at insert, so nothing else is filtered.
+ other key was sent and stays.
+
+ A null-key path with one segment is a member sent as `null`. A longer path rebuilds the member it starts with,
+ which is absent when every field below it was null.
"""
conditions = []
array_paths = ", ".join(escape_clickhouse_string(path) for path in declared_array_paths)
@@ -58,20 +165,46 @@ def json_member_pairs_sql(
string_paths = ", ".join(escape_clickhouse_string(path) for path in declared_string_paths)
if string_paths:
conditions.append(f"not (kv.2 = '\"\"' and kv.1 in ({string_paths}))")
+ nested_paths = _nested_null_paths_sql(null_keys_sql)
if exclude_key is not None:
conditions.append(f"kv.1 != {escape_clickhouse_string(exclude_key)}")
+ nested_paths = (
+ f"arrayFilter(null_path -> null_path[1] != {escape_clickhouse_string(exclude_key)}, {nested_paths})"
+ )
members = f"JSONExtractKeysAndValuesRaw(toJSONString({document_sql}))"
if conditions:
members = f"arrayFilter(kv -> {' and '.join(conditions)}, {members})"
- return f"arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), {members})"
+ member_pairs = f"arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), {members})"
+
+ # Each member and each root of a longer path becomes an entry (key, 0 for a member or 1 for a root, member
+ # value, paths below the root). Sorted by key, a run holds a member, its root, or the member then its root.
+ root_runs = _runs_sql(f"arraySort({nested_paths})", lambda path: f"{path}[1]", "null_path")
+ entries = (
+ "arraySort(arrayConcat("
+ f"arrayMap(kv -> (kv.1, 0, kv.2, {_NO_PATHS_SQL}), {members}), "
+ "arrayMap(null_root_run -> (null_root_run[1][1], 1, '', "
+ f"arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), {root_runs})))"
+ )
+ rebuilt_value = _restore_nulls_sql("null_key_run[1].3", "null_key_run[-1].4")
+ rebuilt_pairs = (
+ f"arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', {rebuilt_value}), "
+ f"{_runs_sql(entries, lambda entry: f'{entry}.1', 'null_entry')})"
+ )
+ null_pairs = (
+ f"arrayMap(null_key -> concat(toJSONString(null_key), ':null'), {_top_level_null_keys_sql(null_keys_sql)})"
+ )
+ # Most documents have no longer paths, so they keep their members as stored and skip the sort.
+ return f"arrayConcat(if(empty({nested_paths}), {member_pairs}, {rebuilt_pairs}), {null_pairs})"
-def feature_flag_pairs_sql(feature_flags_sql: str) -> str:
+
+def feature_flag_pairs_sql(feature_flags_sql: str, null_keys_sql: str) -> str:
"""`"$feature/<key>":<value>` pairs plus the `"$active_feature_flags":[...]` pair, from a `$feature_flags` map.
A boolean flag is stored as 'true' or 'false' and comes back as a JSON boolean. A variant comes back as a JSON
string, with the cleaner sentinels mapped to the variant names. `$active_feature_flags` lists the flags not
- evaluated to off, in map order, and is present whenever the map has an entry.
+ evaluated to off, in map order, and is present whenever the map has an entry. A flag sent as `null` is not in the
+ map; the cleaner records it under the map's null-key path and it comes back as `null`.
"""
variant_branches = ", ".join(
f"value = {escape_clickhouse_string(sentinel)}, {escape_clickhouse_string(variant)}"
@@ -88,28 +221,45 @@ def feature_flag_pairs_sql(feature_flags_sql: str) -> str:
f"toJSONString(mapKeys(mapFilter((key, value) -> value not in ({inactive_values}), {feature_flags_sql})))"
)
active_pair = f"if(empty({feature_flags_sql}), [], [concat('\"$active_feature_flags\":', {active_flags})])"
- return f"arrayConcat({flag_pairs}, {active_pair})"
+ null_flag_pairs = (
+ f"arrayMap(null_path -> concat(toJSONString(concat({prefix}, null_path[2])), ':null'), "
+ f"arrayFilter(null_path -> length(null_path) = 2 and null_path[1] = {escape_clickhouse_string(_FEATURE_FLAGS_KEY)}, "
+ f"{_nested_null_paths_sql(null_keys_sql)}))"
+ )
+ return f"arrayConcat({flag_pairs}, {active_pair}, {null_flag_pairs})"
def json_document_sql(
document_sql: str,
+ null_keys_sql: str,
*,
declared_array_paths: Iterable[str] = (),
declared_string_paths: Iterable[str] = (),
) -> str:
"""A JSON column as document text, built from its member pairs so declared defaults can be left out."""
pairs = json_member_pairs_sql(
- document_sql, declared_array_paths=declared_array_paths, declared_string_paths=declared_string_paths
+ document_sql,
+ null_keys_sql,
+ declared_array_paths=declared_array_paths,
+ declared_string_paths=declared_string_paths,
)
return f"concat('{{', arrayStringConcat({pairs}, ','), '}}')"
-def person_document_sql(person_properties_sql: str) -> str:
+def person_document_sql(person_properties_sql: str, null_keys_sql: str) -> str:
"""The person properties column as document text, without the declared string defaults."""
- return json_document_sql(person_properties_sql, declared_string_paths=PERSON_PROPERTIES_DECLARED_STRING_PATHS)
+ return json_document_sql(
+ person_properties_sql, null_keys_sql, declared_string_paths=PERSON_PROPERTIES_DECLARED_STRING_PATHS
+ )
-def event_document_sql(properties_sql: str, temporary_properties_sql: str, feature_flags_sql: str | None) -> str:
+def event_document_sql(
+ properties_sql: str,
+ properties_null_keys_sql: str,
+ temporary_properties_sql: str,
+ temporary_properties_null_keys_sql: str,
+ feature_flags_sql: str | None,
+) -> str:
"""The rebuilt document as JSON text. The caller wraps it in any restricted-key masking.
`feature_flags_sql` is the flags map to rebuild from, already filtered of restricted flags, or None when every
@@ -119,12 +269,18 @@ def event_document_sql(properties_sql: str, temporary_properties_sql: str, featu
pairs = [
json_member_pairs_sql(
properties_sql,
- exclude_key="$feature_flags",
+ properties_null_keys_sql,
+ exclude_key=_FEATURE_FLAGS_KEY,
declared_array_paths=EVENTS_PROPERTIES_DECLARED_ARRAY_PATHS,
declared_string_paths=EVENTS_PROPERTIES_DECLARED_STRING_PATHS,
)
]
if feature_flags_sql is not None:
- pairs.append(feature_flag_pairs_sql(feature_flags_sql))
- pairs.append(json_member_pairs_sql(temporary_properties_sql))
+ pairs.append(feature_flag_pairs_sql(feature_flags_sql, properties_null_keys_sql))
+ pairs.append(
+ json_member_pairs_sql(
+ temporary_properties_sql,
+ temporary_properties_null_keys_sql,
+ )
+ )
return f"concat('{{', arrayStringConcat(arrayConcat({', '.join(pairs)}), ','), '}}')"
diff --git a/posthog/hogql/printer/test/__snapshots__/test_printer.new_events_schema.ambr b/posthog/hogql/printer/test/__snapshots__/test_printer.new_events_schema.ambr
index b12987cf9c89..0ae4f9543ee5 100644
--- a/posthog/hogql/printer/test/__snapshots__/test_printer.new_events_schema.ambr
+++ b/posthog/hogql/printer/test/__snapshots__/test_printer.new_events_schema.ambr
@@ -3640,7 +3640,7 @@
"SELECT outer.event AS event FROM (SELECT inner.event AS event FROM (SELECT events.event AS event, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), '{}'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_1)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), '{}'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS inner) AS outer LIMIT 50000"
# ---
# name: TestPrinter.test_projection_pushdown_nested_subqueries_1_without_optimize_projections[new_events_schema]
- 'SELECT outer.event AS event FROM (SELECT inner.uuid AS uuid, inner.event AS event, inner.properties AS properties, inner.timestamp AS timestamp, inner.distinct_id AS distinct_id, inner.elements_chain AS elements_chain, inner.created_at AS created_at, inner.`$session_id` AS `$session_id`, inner.`$window_id` AS `$window_id`, inner.person_mode AS person_mode, inner.person_id AS person_id, inner.`$group_0` AS `$group_0`, inner.`$group_1` AS `$group_1`, inner.`$group_2` AS `$group_2`, inner.`$group_3` AS `$group_3`, inner.`$group_4` AS `$group_4`, inner.elements_chain_href AS elements_chain_href, inner.elements_chain_texts AS elements_chain_texts, inner.elements_chain_ids AS elements_chain_ids, inner.elements_chain_elements AS elements_chain_elements, inner.event_issue_id AS event_issue_id, inner.issue_id AS issue_id, inner.issue_id_v2 AS issue_id_v2, inner.issue_name AS issue_name, inner.issue_description AS issue_description, inner.issue_status AS issue_status, inner.issue_assigned_user_id AS issue_assigned_user_id, inner.issue_assigned_role_id AS issue_assigned_role_id, inner.issue_first_seen AS issue_first_seen FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS inner) AS outer LIMIT 50000'
+ 'SELECT outer.event AS event FROM (SELECT inner.uuid AS uuid, inner.event AS event, inner.properties AS properties, inner.timestamp AS timestamp, inner.distinct_id AS distinct_id, inner.elements_chain AS elements_chain, inner.created_at AS created_at, inner.`$session_id` AS `$session_id`, inner.`$window_id` AS `$window_id`, inner.person_mode AS person_mode, inner.person_id AS person_id, inner.`$group_0` AS `$group_0`, inner.`$group_1` AS `$group_1`, inner.`$group_2` AS `$group_2`, inner.`$group_3` AS `$group_3`, inner.`$group_4` AS `$group_4`, inner.elements_chain_href AS elements_chain_href, inner.elements_chain_texts AS elements_chain_texts, inner.elements_chain_ids AS elements_chain_ids, inner.elements_chain_elements AS elements_chain_elements, inner.event_issue_id AS event_issue_id, inner.issue_id AS issue_id, inner.issue_id_v2 AS issue_id_v2, inner.issue_name AS issue_name, inner.issue_description AS issue_description, inner.issue_status AS issue_status, inner.issue_assigned_user_id AS issue_assigned_user_id, inner.issue_assigned_role_id AS issue_assigned_role_id, inner.issue_first_seen AS issue_first_seen FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat(\'$feature/\', null_path[2])), \':null\'), arrayFilter(null_path -> length(null_path) = 2 and null_path[1] = \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.temporary_properties_null_keys))))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS inner) AS outer LIMIT 50000'
# ---
# name: TestPrinter.test_projection_pushdown_no_asterisk_unchanged_0_with_optimize_projections[new_events_schema]
'SELECT sub.event AS event FROM (SELECT events.event AS event, events.distinct_id AS distinct_id FROM events_json AS events WHERE equals(events.team_id, 99999)) AS sub LIMIT 50000'
@@ -3652,19 +3652,19 @@
"SELECT e.event AS event FROM (SELECT events.event AS event, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), '{}'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_1)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), '{}'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS e LEFT JOIN (SELECT person.id AS id FROM person WHERE equals(person.team_id, 99999) GROUP BY person.id HAVING and(equals(argMax(person.is_deleted, person.version), 0), less(argMax(toTimeZone(person.created_at, %(hogql_val_2)s), person.version), plus(now64(6, %(hogql_val_3)s), toIntervalDay(1)))) SETTINGS optimize_aggregation_in_order=1) AS persons ON equals(persons.id, e.person_id) LIMIT 50000"
# ---
# name: TestPrinter.test_projection_pushdown_preserves_join_columns_1_without_optimize_projections[new_events_schema]
- 'SELECT e.event AS event FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS e LEFT JOIN (SELECT person.id AS id FROM person WHERE equals(person.team_id, 99999) GROUP BY person.id HAVING and(equals(argMax(person.is_deleted, person.version), 0), less(argMax(toTimeZone(person.created_at, %(hogql_val_4)s), person.version), plus(now64(6, %(hogql_val_5)s), toIntervalDay(1)))) SETTINGS optimize_aggregation_in_order=1) AS persons ON equals(persons.id, e.person_id) LIMIT 50000'
+ 'SELECT e.event AS event FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat(\'$feature/\', null_path[2])), \':null\'), arrayFilter(null_path -> length(null_path) = 2 and null_path[1] = \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.temporary_properties_null_keys))))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS e LEFT JOIN (SELECT person.id AS id FROM person WHERE equals(person.team_id, 99999) GROUP BY person.id HAVING and(equals(argMax(person.is_deleted, person.version), 0), less(argMax(toTimeZone(person.created_at, %(hogql_val_4)s), person.version), plus(now64(6, %(hogql_val_5)s), toIntervalDay(1)))) SETTINGS optimize_aggregation_in_order=1) AS persons ON equals(persons.id, e.person_id) LIMIT 50000'
# ---
# name: TestPrinter.test_projection_pushdown_preserves_where_columns_0_with_optimize_projections[new_events_schema]
"SELECT sub.event AS event FROM (SELECT events.event AS event, events.distinct_id AS distinct_id, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), '{}'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_1)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), '{}'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS sub WHERE equals(sub.distinct_id, %(hogql_val_2)s) LIMIT 50000"
# ---
# name: TestPrinter.test_projection_pushdown_preserves_where_columns_1_without_optimize_projections[new_events_schema]
- 'SELECT sub.event AS event FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS sub WHERE equals(sub.distinct_id, %(hogql_val_4)s) LIMIT 50000'
+ 'SELECT sub.event AS event FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat(\'$feature/\', null_path[2])), \':null\'), arrayFilter(null_path -> length(null_path) = 2 and null_path[1] = \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.temporary_properties_null_keys))))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS sub WHERE equals(sub.distinct_id, %(hogql_val_4)s) LIMIT 50000'
# ---
# name: TestPrinter.test_projection_pushdown_simple_asterisk_subquery_0_with_optimize_projections[new_events_schema]
"SELECT sub.event AS event FROM (SELECT events.event AS event, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), '{}'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_1)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), '{}'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS sub LIMIT 50000"
# ---
# name: TestPrinter.test_projection_pushdown_simple_asterisk_subquery_1_without_optimize_projections[new_events_schema]
- 'SELECT sub.event AS event FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS sub LIMIT 50000'
+ 'SELECT sub.event AS event FROM (SELECT events.uuid AS uuid, events.event AS event, concat(\'{\', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), arrayFilter(kv -> not (kv.2 = \'[]\' and kv.1 in (\'$exception_functions\', \'$exception_list\', \'$exception_sources\', \'$exception_types\', \'$exception_values\', \'$mcp_listed_tool_names\')) and not (kv.2 = \'""\' and kv.1 in (\'$browser\', \'$browser_language\', \'$browser_version\', \'$config_defaults\', \'$device_type\', \'$geoip_city_name\', \'$geoip_continent_code\', \'$geoip_continent_name\', \'$geoip_country_code\', \'$geoip_country_name\', \'$geoip_subdivision_1_name\', \'$geoip_time_zone\', \'$group_0\', \'$group_1\', \'$group_2\', \'$group_3\', \'$group_4\', \'$lib\', \'$lib_version\', \'$os\', \'$os_version\', \'$session_id\', \'$timezone\', \'$window_id\')) and kv.1 != \'$feature_flags\', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat(\'$feature/\', key)), \':\', if(value in (\'true\', \'false\'), value, toJSONString(multiIf(value = \'$false\', \'false\', value = \'$true\', \'true\', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat(\'"$active_feature_flags":\', toJSONString(mapKeys(mapFilter((key, value) -> value not in (\'\', \'false\'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat(\'$feature/\', null_path[2])), \':null\'), arrayFilter(null_path -> length(null_path) = 2 and null_path[1] = \'$feature_flags\', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), \':\', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), \':\', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [(\'null\', CAST([], \'Array(Array(String))\'))], null_node.1 = \'\' or startsWith(null_node.1, \'{\') or startsWith(null_node.1, \'[\'), arrayMap((null_array) -> arrayConcat([(if(null_array, \'[\', \'{\'), CAST([], \'Array(Array(String))\'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, \'\', \',\'), if(null_array, \'\', concat(toJSONString(null_run[1].2), \':\'))), CAST([], \'Array(Array(String))\')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), \'\', 0, null_element, CAST([], \'Array(Array(String))\')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, \'\', null_root[1][1]), 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, \']\', \'}\'), CAST([], \'Array(Array(String))\'))]), [startsWith(null_node.1, \'[\')])[1], [(null_node.1, CAST([], \'Array(Array(String))\'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], \'Array(Array(String))\')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, \'\', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, \'\\x252E\', \'.\'), splitByChar(\'.\', null_path)), arrayFilter(null_path -> position(null_path, \'.\') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), \':null\'), arrayMap(null_key -> replaceAll(null_key, \'\\x252E\', \'.\'), arrayFilter(null_key -> position(null_key, \'.\') = 0, events.temporary_properties_null_keys))))), \',\'), \'}\') AS properties, toTimeZone(events.timestamp, %(hogql_val_1)s) AS timestamp, events.distinct_id AS distinct_id, events.elements_chain AS elements_chain, toTimeZone(events.created_at, %(hogql_val_2)s) AS created_at, events.properties.`$session_id` AS `$session_id`, events.properties.`$window_id` AS `$window_id`, events.person_mode AS person_mode, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS person_id, events.properties.`$group_0` AS `$group_0`, events.properties.`$group_1` AS `$group_1`, events.properties.`$group_2` AS `$group_2`, events.properties.`$group_3` AS `$group_3`, events.properties.`$group_4` AS `$group_4`, events.elements_chain_href AS elements_chain_href, events.elements_chain_texts AS elements_chain_texts, events.elements_chain_ids AS elements_chain_ids, events.elements_chain_elements AS elements_chain_elements, accurateCastOrNull(if(notEquals(toJSONString(events.properties.^`$exception_issue_id`), \'{}\'), toJSONString(events.properties.^`$exception_issue_id`), if(isNull(events.properties.`$exception_issue_id`), NULL, if(and(in(ascii(toString(events.properties.`$exception_issue_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_issue_id`)), tuple(91, 123))), toJSONString(events.properties.`$exception_issue_id`), toString(events.properties.`$exception_issue_id`)))), %(hogql_val_3)s) AS event_issue_id, events__fingerprint_issue_state.issue_id AS issue_id, events__fingerprint_issue_state.issue_id AS issue_id_v2, events__fingerprint_issue_state.issue_name AS issue_name, events__fingerprint_issue_state.issue_description AS issue_description, events__fingerprint_issue_state.issue_status AS issue_status, events__fingerprint_issue_state.assigned_user_id AS issue_assigned_user_id, events__fingerprint_issue_state.assigned_role_id AS issue_assigned_role_id, events__fingerprint_issue_state.first_seen AS issue_first_seen FROM events_json AS events LEFT OUTER JOIN (SELECT cityHash64(error_tracking_fingerprint_issue_state.fingerprint) AS fp_hash, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_id), error_tracking_fingerprint_issue_state.version), 1)) AS issue_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_name), error_tracking_fingerprint_issue_state.version), 1)) AS issue_name, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_description), error_tracking_fingerprint_issue_state.version), 1)) AS issue_description, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.issue_status), error_tracking_fingerprint_issue_state.version), 1)) AS issue_status, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_user_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_user_id, toNullable(tupleElement(argMax(tuple(error_tracking_fingerprint_issue_state.assigned_role_id), error_tracking_fingerprint_issue_state.version), 1)) AS assigned_role_id, toNullable(tupleElement(argMax(tuple(toTimeZone(error_tracking_fingerprint_issue_state.first_seen, %(hogql_val_0)s)), error_tracking_fingerprint_issue_state.version), 1)) AS first_seen FROM error_tracking_fingerprint_issue_state WHERE equals(error_tracking_fingerprint_issue_state.team_id, 99999) GROUP BY fp_hash HAVING equals(argMax(error_tracking_fingerprint_issue_state.is_deleted, error_tracking_fingerprint_issue_state.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__fingerprint_issue_state ON equals(cityHash64(if(notEquals(toJSONString(events.properties.^`$exception_fingerprint`), \'{}\'), toJSONString(events.properties.^`$exception_fingerprint`), if(isNull(events.properties.`$exception_fingerprint`), NULL, if(and(in(ascii(toString(events.properties.`$exception_fingerprint`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$exception_fingerprint`)), tuple(91, 123))), toJSONString(events.properties.`$exception_fingerprint`), toString(events.properties.`$exception_fingerprint`))))), events__fingerprint_issue_state.fp_hash) LEFT OUTER JOIN (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id FROM person_distinct_id_overrides WHERE equals(person_distinct_id_overrides.team_id, 99999) GROUP BY person_distinct_id_overrides.distinct_id HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS events__override ON equals(events.distinct_id, events__override.distinct_id) WHERE equals(events.team_id, 99999)) AS sub LIMIT 50000'
# ---
# name: TestPrinter.test_s3_tables_global_join_anonymous_tables_0_global_joins_with_optimize[new_events_schema]
"SELECT e.event AS event, ij.remote_id AS remote_id FROM events_json AS e GLOBAL INNER JOIN (SELECT remote_id AS remote_id FROM (SELECT p.id AS person_id, rt.id AS remote_id FROM (SELECT person.id AS id FROM person WHERE equals(person.team_id, 99999) GROUP BY person.id HAVING and(equals(argMax(person.is_deleted, person.version), 0), less(argMax(toTimeZone(person.created_at, %(hogql_val_0)s), person.version), plus(now64(6, %(hogql_val_1)s), toIntervalDay(1)))) SETTINGS optimize_aggregation_in_order=1) AS p LEFT JOIN (SELECT test_table.id AS id FROM (SELECT * FROM s3(%(hogql_val_2_sensitive)s, %(hogql_val_4_sensitive)s, %(hogql_val_5_sensitive)s, 'Parquet', %(hogql_val_3)s) SETTINGS optimize_move_to_prewhere = 0) AS test_table) AS rt ON equals(rt.id, p.id))) AS ij ON equals(e.event, ij.remote_id) WHERE equals(e.team_id, 99999) LIMIT 50000"
diff --git a/posthog/hogql/printer/test/test_events_json_document.py b/posthog/hogql/printer/test/test_events_json_document.py
index 16efc84a3dc4..c678d5c8c2be 100644
--- a/posthog/hogql/printer/test/test_events_json_document.py
+++ b/posthog/hogql/printer/test/test_events_json_document.py
@@ -4,60 +4,103 @@ import pytest
from django.test import SimpleTestCase
-from parameterized import parameterized
-
from posthog.hogql.printer.events_json_document import event_document_sql, person_document_sql
from posthog.clickhouse.client import sync_execute
from posthog.clickhouse.events_json import TEMPORARY_PROPERTIES_JSON_TYPE
-from posthog.models.event.sql import EVENTS_PROPERTIES_JSON_TYPE, PERSON_PROPERTIES_JSON_TYPE
+from posthog.models.event.sql import EVENTS_JSON_CLEANER, EVENTS_PROPERTIES_JSON_TYPE, PERSON_PROPERTIES_JSON_TYPE
-# A sent empty value under a dynamic key stays; the declared paths read '' or [] whether sent empty or absent,
-# so the document leaves those out.
-EVENT_DOCUMENTS = [
- ({"$browser": "Chrome", "pathname": "/docs", "empty": "", "list": [], "nested": {"child": ""}}, None),
- ({"$browser": "", "$exception_types": [], "custom_list": [], "note": ""}, {"custom_list": [], "note": ""}),
- ({}, None),
-]
-PERSON_DOCUMENTS = [
- ({"$initial_browser": "Chrome", "email": "[email protected]", "empty": "", "list": []}, None),
- ({"$initial_browser": "", "$browser": "", "list": [], "note": ""}, {"list": [], "note": ""}),
- ({}, None),
+# The event properties and person properties an SDK sent, then the rebuilt documents where they differ from what was
+# sent. A sent empty value under a dynamic key stays; the declared paths read '' or [] whether sent empty or absent,
+# so the document leaves those out. Recorded null fields must survive reconstruction regardless of document size.
+SENT_DOCUMENTS: list[tuple[dict, dict, dict | None, dict | None]] = [
+ (
+ {"$browser": "Chrome", "pathname": "/docs", "empty": "", "list": [], "nested": {"child": ""}},
+ {"$initial_browser": "Chrome", "email": "[email protected]", "empty": "", "list": []},
+ None,
+ None,
+ ),
+ (
+ {"$browser": "", "$exception_types": [], "custom_list": [], "note": ""},
+ {"$initial_browser": "", "$browser": "", "list": [], "note": ""},
+ {"custom_list": [], "note": ""},
+ {"list": [], "note": ""},
+ ),
+ ({}, {}, None, None),
+ (
+ {
+ "plan": None,
+ "a.b": None,
+ "$browser": None,
+ "$feature/beta": None,
+ "billing": {"coupon": None, "seats": 3, "card": {"brand": None}},
+ "trial": {"ends": None},
+ "$set": {"email": None, "name": "n"},
+ "$set_once": {"first": None},
+ },
+ {"team": None, "profile": {"org": {"size": None, "seats": 5}}, "geo": {"city.name": None}},
+ None,
+ None,
+ ),
+ (
+ {
+ "items": [{"sku": "a", "note": None}, {"sku": "b"}, {"gone": None}],
+ "matrix": [[{"x": None, "y": 1}]],
+ "list": [1, None],
+ "$exception_list": [{"type": "E", "value": "v", "stacktrace": {"frames": [{"lineno": None, "f": "g"}]}}],
+ "$group_set": {"modules": [{"owner": None, "name": "m1"}]},
+ },
+ {"years": {"2024": {"x": None}, "2025": 1}},
+ None,
+ None,
+ ),
+ (
+ {"container": {"matrix": [[{"x": None, "y": 1}] for _ in range(500)], "padding": "v" * 20_000}},
+ {},
+ None,
+ None,
+ ),
+ ({f"p{i}": {"missing": None} for i in range(10_000)}, {}, None, None),
]
@pytest.mark.usefixtures("clickhouse_database")
class TestEventsJsonDocument(SimpleTestCase):
- @parameterized.expand(
- [
- (
- "event",
- "properties",
- event_document_sql("properties", "temporary_properties", None),
- EVENTS_PROPERTIES_JSON_TYPE(),
- EVENT_DOCUMENTS,
- ),
- (
- "person",
- "person_properties",
- person_document_sql("person_properties"),
- PERSON_PROPERTIES_JSON_TYPE(),
- PERSON_DOCUMENTS,
- ),
- ]
- )
- def test_document_keeps_sent_empties_and_drops_declared_defaults(
- self, _name: str, column: str, document_sql: str, json_type: str, cases: list[tuple[dict, dict | None]]
- ) -> None:
+ def test_rebuilt_documents_match_what_the_sdk_sent(self) -> None:
+ documents = event_document_sql(
+ "properties",
+ "properties_null_keys",
+ "temporary_properties",
+ "temporary_properties_null_keys",
+ "properties.`$feature_flags`",
+ )
+ person_documents = person_document_sql("person_properties", "person_properties_null_keys")
rows = sync_execute(
- f"SELECT {document_sql} FROM (SELECT CAST(arrayJoin(%(documents)s), %(json_type)s) AS {column}, "
- f"CAST('{{}}', %(temporary_type)s) AS temporary_properties)",
+ f"SELECT {documents}, {person_documents} FROM ("
+ "SELECT CAST(cleaned.properties, %(event_type)s) AS properties, "
+ "CAST(cleaned.temporary_properties, %(temporary_type)s) AS temporary_properties, "
+ "CAST(cleaned.person_properties, %(person_type)s) AS person_properties, "
+ "cleaned.properties_null_keys AS properties_null_keys, "
+ "cleaned.temporary_properties_null_keys AS temporary_properties_null_keys, "
+ "cleaned.person_properties_null_keys AS person_properties_null_keys "
+ f"FROM (SELECT {EVENTS_JSON_CLEANER}(sent.1, sent.2) AS cleaned FROM (SELECT arrayJoin(%(sent)s) AS sent)))",
{
- "documents": [json.dumps(document) for document, _ in cases],
- "json_type": json_type,
+ "sent": [(json.dumps(event), json.dumps(person)) for event, person, _, _ in SENT_DOCUMENTS],
+ "event_type": EVENTS_PROPERTIES_JSON_TYPE(),
"temporary_type": TEMPORARY_PROPERTIES_JSON_TYPE,
+ "person_type": PERSON_PROPERTIES_JSON_TYPE(),
+ },
+ settings={
+ "input_format_try_infer_dates": 0,
+ "input_format_try_infer_datetimes": 0,
+ "json_type_escape_dots_in_keys": 1,
+ "type_json_skip_duplicated_paths": 1,
},
)
- assert [json.loads(row[0]) for row in rows] == [
- expected if expected is not None else document for document, expected in cases
+ assert [(json.loads(event), json.loads(person)) for event, person in rows] == [
+ (
+ event if expected_event is None else expected_event,
+ person if expected_person is None else expected_person,
+ )
+ for event, person, expected_event, expected_person in SENT_DOCUMENTS
]
diff --git a/posthog/hogql/printer/test/test_printer.py b/posthog/hogql/printer/test/test_printer.py
index 2ecfb48da712..ecbdebf7a233 100644
--- a/posthog/hogql/printer/test/test_printer.py
+++ b/posthog/hogql/printer/test/test_printer.py
@@ -1453,7 +1453,8 @@ class TestPrinter(BaseTest):
printed = self._expr(expression)
[(serialized,)] = sync_execute(
f"SELECT {printed} FROM (SELECT CAST(%(raw)s, %(json_type)s) AS properties, "
- "CAST('{}', %(temporary_type)s) AS temporary_properties) AS events",
+ "CAST('{}', %(temporary_type)s) AS temporary_properties, CAST([], 'Array(String)') AS properties_null_keys, "
+ "CAST([], 'Array(String)') AS temporary_properties_null_keys) AS events",
{
"raw": json.dumps(properties),
"json_type": EVENTS_PROPERTIES_JSON_TYPE(),
diff --git a/posthog/hogql/transforms/test/__snapshots__/test_events_predicate_pushdown.new_events_schema.ambr b/posthog/hogql/transforms/test/__snapshots__/test_events_predicate_pushdown.new_events_schema.ambr
index 042af6d4e7aa..7d31dcc4d177 100644
--- a/posthog/hogql/transforms/test/__snapshots__/test_events_predicate_pushdown.new_events_schema.ambr
+++ b/posthog/hogql/transforms/test/__snapshots__/test_events_predicate_pushdown.new_events_schema.ambr
@@ -106,7 +106,7 @@
SELECT events.event AS event, if(not(empty(events__override.distinct_id)), events__override.person_id, events.person_id) AS id, events.person_properties AS properties, events.__pd_expr_0 AS created_at, events__session.`$session_duration` AS `$session_duration`
FROM (
- SELECT toTimeZone(events.person_created_at, %(hogql_val_0)s) AS __pd_expr_0, toUInt128(accurateCastOrNull(events.properties.`$session_id`, %(hogql_val_1)s)) AS __pd_expr_1, events.distinct_id AS distinct_id, events.event AS event, events.person_id AS person_id, concat('{', arrayStringConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '""' and kv.1 in ('$browser', '$browser_language', '$browser_version', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$initial_browser', '$initial_browser_language', '$initial_browser_version', '$initial_device_type', '$initial_geoip_city_name', '$initial_geoip_continent_code', '$initial_geoip_continent_name', '$initial_geoip_country_code', '$initial_geoip_country_name', '$initial_geoip_subdivision_1_name', '$initial_geoip_time_zone', '$initial_os', '$initial_os_version', '$os', '$os_version')), JSONExtractKeysAndValuesRaw(toJSONString(events.person_properties)))), ','), '}') AS person_properties
+ SELECT toTimeZone(events.person_created_at, %(hogql_val_0)s) AS __pd_expr_0, toUInt128(accurateCastOrNull(events.properties.`$session_id`, %(hogql_val_1)s)) AS __pd_expr_1, events.distinct_id AS distinct_id, events.event AS event, events.person_id AS person_id, concat('{', arrayStringConcat(arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.person_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '""' and kv.1 in ('$browser', '$browser_language', '$browser_version', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$initial_browser', '$initial_browser_language', '$initial_browser_version', '$initial_device_type', '$initial_geoip_city_name', '$initial_geoip_continent_code', '$initial_geoip_continent_name', '$initial_geoip_country_code', '$initial_geoip_country_name', '$initial_geoip_subdivision_1_name', '$initial_geoip_time_zone', '$initial_os', '$initial_os_version', '$os', '$os_version')), JSONExtractKeysAndValuesRaw(toJSONString(events.person_properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = '' or startsWith(null_node.1, '{') or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '""' and kv.1 in ('$browser', '$browser_language', '$browser_version', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$initial_browser', '$initial_browser_language', '$initial_browser_version', '$initial_device_type', '$initial_geoip_city_name', '$initial_geoip_continent_code', '$initial_geoip_continent_name', '$initial_geoip_country_code', '$initial_geoip_country_name', '$initial_geoip_subdivision_1_name', '$initial_geoip_time_zone', '$initial_os', '$initial_os_version', '$os', '$os_version')), JSONExtractKeysAndValuesRaw(toJSONString(events.person_properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.person_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.person_properties_null_keys)))), ','), '}') AS person_properties
FROM events_json AS events
WHERE and(equals(events.team_id, 420), greaterOrEquals(events.timestamp, toDateTime64(%(hogql_val_2)s, 6, %(hogql_val_3)s)))
LIMIT 50000) AS events LEFT JOIN (
@@ -273,7 +273,7 @@
SELECT events.event AS event, events.person_properties AS properties, events__session.`$session_duration` AS `$session_duration`
FROM (
- SELECT toUInt128(accurateCastOrNull(events.properties.`$session_id`, %(hogql_val_0)s)) AS __pd_expr_0, events.event AS event, concat('{', arrayStringConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '""' and kv.1 in ('$browser', '$browser_language', '$browser_version', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$initial_browser', '$initial_browser_language', '$initial_browser_version', '$initial_device_type', '$initial_geoip_city_name', '$initial_geoip_continent_code', '$initial_geoip_continent_name', '$initial_geoip_country_code', '$initial_geoip_country_name', '$initial_geoip_subdivision_1_name', '$initial_geoip_time_zone', '$initial_os', '$initial_os_version', '$os', '$os_version')), JSONExtractKeysAndValuesRaw(toJSONString(events.person_properties)))), ','), '}') AS person_properties
+ SELECT toUInt128(accurateCastOrNull(events.properties.`$session_id`, %(hogql_val_0)s)) AS __pd_expr_0, events.event AS event, concat('{', arrayStringConcat(arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.person_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '""' and kv.1 in ('$browser', '$browser_language', '$browser_version', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$initial_browser', '$initial_browser_language', '$initial_browser_version', '$initial_device_type', '$initial_geoip_city_name', '$initial_geoip_continent_code', '$initial_geoip_continent_name', '$initial_geoip_country_code', '$initial_geoip_country_name', '$initial_geoip_subdivision_1_name', '$initial_geoip_time_zone', '$initial_os', '$initial_os_version', '$os', '$os_version')), JSONExtractKeysAndValuesRaw(toJSONString(events.person_properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = '' or startsWith(null_node.1, '{') or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '""' and kv.1 in ('$browser', '$browser_language', '$browser_version', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$initial_browser', '$initial_browser_language', '$initial_browser_version', '$initial_device_type', '$initial_geoip_city_name', '$initial_geoip_continent_code', '$initial_geoip_continent_name', '$initial_geoip_country_code', '$initial_geoip_country_name', '$initial_geoip_subdivision_1_name', '$initial_geoip_time_zone', '$initial_os', '$initial_os_version', '$os', '$os_version')), JSONExtractKeysAndValuesRaw(toJSONString(events.person_properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.person_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.person_properties_null_keys)))), ','), '}') AS person_properties
FROM events_json AS events
WHERE and(equals(events.team_id, 420), greaterOrEquals(events.timestamp, toDateTime64(%(hogql_val_1)s, 6, %(hogql_val_2)s)))
LIMIT 50000) AS events LEFT JOIN (
diff --git a/posthog/hogql/transforms/test/test_property_types.py b/posthog/hogql/transforms/test/test_property_types.py
index 228daabf56d7..d12294ab53e4 100644
--- a/posthog/hogql/transforms/test/test_property_types.py
+++ b/posthog/hogql/transforms/test/test_property_types.py
@@ -454,7 +454,7 @@ class TestNewEventsSchemaArraySubcolumns(_NewEventsSchemaArraySubcolumnsHelpers,
"'\"$active_feature_flags\":'",
f"{JSON_DROP_KEYS_CLICKHOUSE_NAME}(",
),
- ("JSONMergePatch(", f"'', {JSON_DROP_KEYS_CLICKHOUSE_NAME}("),
+ ("'{\"$feature_flags\":'", f"'', {JSON_DROP_KEYS_CLICKHOUSE_NAME}("),
),
(
"stored_document_restricted",
@@ -1085,7 +1085,10 @@ class TestNewEventsSchemaArraySubcolumnsClickhouse(_NewEventsSchemaArraySubcolum
SELECT 1 AS team_id, 'synthetic' AS event,
CAST(%(document)s, %(event_type)s) AS properties,
CAST(%(document)s, %(person_type)s) AS person_properties,
- CAST('{}', %(temporary_type)s) AS temporary_properties
+ CAST('{}', %(temporary_type)s) AS temporary_properties,
+ CAST([], 'Array(String)') AS properties_null_keys,
+ CAST([], 'Array(String)') AS temporary_properties_null_keys,
+ CAST([], 'Array(String)') AS person_properties_null_keys
) """
+ printed,
{
@@ -1124,7 +1127,10 @@ class TestNewEventsSchemaArraySubcolumnsClickhouse(_NewEventsSchemaArraySubcolum
SELECT 1 AS team_id, '$identify' AS event,
CAST(%(document)s, %(event_type)s) AS properties,
CAST('{}', %(person_type)s) AS person_properties,
- CAST(%(temporary_document)s, %(temporary_type)s) AS temporary_properties
+ CAST(%(temporary_document)s, %(temporary_type)s) AS temporary_properties,
+ CAST([], 'Array(String)') AS properties_null_keys,
+ CAST([], 'Array(String)') AS temporary_properties_null_keys,
+ CAST([], 'Array(String)') AS person_properties_null_keys
) """
+ printed,
{
@@ -2464,6 +2470,8 @@ class TestEventsSchemaPropertyParity(ClickhouseTestMixin, BaseTest):
"$unset": ["old_key"],
"$sdk_debug_replay_flushed_size": 42,
"$browser": "Firefox",
+ "coupon": None,
+ "cart": {"items": 2, "note": None},
},
)
flush_persons_and_events()
@@ -2512,7 +2520,10 @@ class TestEventsSchemaPropertyParity(ClickhouseTestMixin, BaseTest):
"(SELECT properties.$set.email FROM (SELECT * FROM events WHERE uuid = {uuid})) "
f"FROM events WHERE uuid = '{event_uuid}'"
).replace("{uuid}", f"'{event_uuid}'")
- restricted_plan = {RestrictedProperty(name="$set.plan", property_type=PropertyDefinition.Type.EVENT)}
+ restricted_keys = {
+ RestrictedProperty(name=name, property_type=PropertyDefinition.Type.EVENT)
+ for name in ("$set.plan", "coupon")
+ }
documents: dict[tuple[bool, bool], tuple[Any, ...]] = {}
for use_new_events_schema in (False, True):
for with_restriction in (False, True):
@@ -2520,7 +2531,7 @@ class TestEventsSchemaPropertyParity(ClickhouseTestMixin, BaseTest):
team_id=self.team.pk,
enable_select_queries=True,
use_new_events_schema=use_new_events_schema,
- restricted_properties=restricted_plan if with_restriction else set(),
+ restricted_properties=restricted_keys if with_restriction else set(),
)
response = execute_hogql_query(document_query, team=self.team, context=context)
assert response.results is not None
@@ -2540,6 +2551,8 @@ class TestEventsSchemaPropertyParity(ClickhouseTestMixin, BaseTest):
"$unset": ["old_key"],
"$sdk_debug_replay_flushed_size": 42,
"$browser": "Firefox",
+ "coupon": None,
+ "cart": {"items": 2, "note": None},
},
{
"$set": {"email": "[email protected]", "plan": {"tier": "pro"}},
@@ -2547,11 +2560,14 @@ class TestEventsSchemaPropertyParity(ClickhouseTestMixin, BaseTest):
"$unset": ["old_key"],
"$sdk_debug_replay_flushed_size": 42,
"$browser": "Firefox",
+ "coupon": None,
+ "cart": {"items": 2, "note": None},
},
- ["$browser", "$sdk_debug_replay_flushed_size", "$set", "$set_once", "$unset"],
+ ["$browser", "$sdk_debug_replay_flushed_size", "$set", "$set_once", "$unset", "cart", "coupon"],
)
assert documents[(True, True)][0]["$set"] == {"email": "[email protected]"}
+ assert "coupon" not in documents[(True, True)][0]
# One call per registered JSON function, so a newly registered function fails here until native reads the
# moved key. isValidJSON and JSONArrayLength take no key path.
diff --git a/posthog/hogql_queries/ai/test/__snapshots__/test_event_taxonomy_query_runner.new_events_schema.ambr b/posthog/hogql_queries/ai/test/__snapshots__/test_event_taxonomy_query_runner.new_events_schema.ambr
index 136209e366bb..c9147b133dbf 100644
--- a/posthog/hogql_queries/ai/test/__snapshots__/test_event_taxonomy_query_runner.new_events_schema.ambr
+++ b/posthog/hogql_queries/ai/test/__snapshots__/test_event_taxonomy_query_runner.new_events_schema.ambr
@@ -5,27 +5,35 @@
arrayMap(item -> item.3, arraySlice(reverse(arraySort(item -> tuple(item.1, item.2, item.3), groupArray(tuple(value_count, latest_seen, value)))), 1, 5)) AS
values,
count(DISTINCT value) AS total_count
- FROM
- (SELECT key,
- value,
- count() AS value_count,
- max(timestamp) AS latest_seen
- FROM
- (SELECT JSONExtractKeysAndValues(concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), 'String') AS kv,
- toTimeZone(events.timestamp, 'UTC') AS timestamp
- FROM events_json AS events
- WHERE and(equals(events.team_id, 99999), greaterOrEquals(events.timestamp, toDateTime64(minus(now64(6, 'UTC'), toIntervalDay(30)), 6, 'UTC')), equals(events.event, 'event1'))
- ORDER BY toTimeZone(events.timestamp, 'UTC') DESC
- LIMIT 100) ARRAY
- JOIN (kv).1 AS key,
- (kv).2 AS value
- WHERE and(not(match(key, '(\\$set|\\$time|\\$set_once|\\$sent_at|distinct_id|\\$ip|\\$feature\\/|^\\$feature_flags$|^\\$active_feature_flags$|\\$feature_enrollment\\/|\\$feature_interaction\\/|\\$product_tour|__|survey_dismiss|survey_responded|phjs|partial_filter_chosen|changed_action|window-id|changed_event|partial_filter)')), isNotNull(value), ifNull(notEquals(value, ''), 1))
- GROUP BY key,
- value)
+ FROM (
+ SELECT key,
+ value,
+ count() AS value_count,
+ max(timestamp) AS latest_seen
+ FROM (
+ SELECT JSONExtractKeysAndValues(concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), 'String') AS kv, toTimeZone(events.timestamp, 'UTC') AS timestamp
+ FROM events_json AS events
+ WHERE and(equals(events.team_id, 99999), greaterOrEquals(events.timestamp, toDateTime64(minus(now64(6, 'UTC'), toIntervalDay(30)), 6, 'UTC')), equals(events.event, 'event1'))
+ ORDER BY toTimeZone(events.timestamp, 'UTC') DESC
+ LIMIT 100) ARRAY
+ JOIN (kv).1 AS key, (kv).2 AS value
+ WHERE and(not(match(key, '(\\$set|\\$time|\\$set_once|\\$sent_at|distinct_id|\\$ip|\\$feature\\/|^\\$feature_flags$|^\\$active_feature_flags$|\\$feature_enrollment\\/|\\$feature_interaction\\/|\\$product_tour|__|survey_dismiss|survey_responded|phjs|partial_filter_chosen|changed_action|window-id|changed_event|partial_filter)')), isNotNull(value), ifNull(notEquals(value, ''), 1))
+ GROUP BY key, value)
GROUP BY key
ORDER BY total_count DESC,
key ASC
@@ -93,27 +101,35 @@
arrayMap(item -> item.3, arraySlice(reverse(arraySort(item -> tuple(item.1, item.2, item.3), groupArray(tuple(value_count, latest_seen, value)))), 1, 5)) AS
values,
count(DISTINCT value) AS total_count
- FROM
- (SELECT key,
- value,
- count() AS value_count,
- max(timestamp) AS latest_seen
- FROM
- (SELECT JSONExtractKeysAndValues(concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), 'String') AS kv,
- toTimeZone(events.timestamp, 'UTC') AS timestamp
- FROM events_json AS events
- WHERE and(equals(events.team_id, 99999), greaterOrEquals(events.timestamp, toDateTime64(minus(now64(6, 'UTC'), toIntervalDay(30)), 6, 'UTC')), equals(events.event, '$pageview'))
- ORDER BY toTimeZone(events.timestamp, 'UTC') DESC
- LIMIT 100) ARRAY
- JOIN (kv).1 AS key,
- (kv).2 AS value
- WHERE and(not(match(key, '(\\$set|\\$time|\\$set_once|\\$sent_at|distinct_id|\\$ip|\\$feature\\/|^\\$feature_flags$|^\\$active_feature_flags$|\\$feature_enrollment\\/|\\$feature_interaction\\/|\\$product_tour|__|survey_dismiss|survey_responded|phjs|partial_filter_chosen|changed_action|window-id|changed_event|partial_filter)')), isNotNull(value), ifNull(notEquals(value, ''), 1))
- GROUP BY key,
- value)
+ FROM (
+ SELECT key,
+ value,
+ count() AS value_count,
+ max(timestamp) AS latest_seen
+ FROM (
+ SELECT JSONExtractKeysAndValues(concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), 'String') AS kv, toTimeZone(events.timestamp, 'UTC') AS timestamp
+ FROM events_json AS events
+ WHERE and(equals(events.team_id, 99999), greaterOrEquals(events.timestamp, toDateTime64(minus(now64(6, 'UTC'), toIntervalDay(30)), 6, 'UTC')), equals(events.event, '$pageview'))
+ ORDER BY toTimeZone(events.timestamp, 'UTC') DESC
+ LIMIT 100) ARRAY
+ JOIN (kv).1 AS key, (kv).2 AS value
+ WHERE and(not(match(key, '(\\$set|\\$time|\\$set_once|\\$sent_at|distinct_id|\\$ip|\\$feature\\/|^\\$feature_flags$|^\\$active_feature_flags$|\\$feature_enrollment\\/|\\$feature_interaction\\/|\\$product_tour|__|survey_dismiss|survey_responded|phjs|partial_filter_chosen|changed_action|window-id|changed_event|partial_filter)')), isNotNull(value), ifNull(notEquals(value, ''), 1))
+ GROUP BY key, value)
GROUP BY key
ORDER BY total_count DESC,
key ASC
diff --git a/posthog/hogql_queries/ai/test/__snapshots__/test_trace_query_runner.new_events_schema.ambr b/posthog/hogql_queries/ai/test/__snapshots__/test_trace_query_runner.new_events_schema.ambr
index 2b8b32f91302..edceaad9b65f 100644
--- a/posthog/hogql_queries/ai/test/__snapshots__/test_trace_query_runner.new_events_schema.ambr
+++ b/posthog/hogql_queries/ai/test/__snapshots__/test_trace_query_runner.new_events_schema.ambr
@@ -17,36 +17,31 @@
argMinIf(deduped.input_state, deduped.timestamp, equals(deduped.event, '$ai_trace')) AS input_state,
argMinIf(deduped.output_state, deduped.timestamp, equals(deduped.event, '$ai_trace')) AS output_state,
ifNull(argMinIf(ifNull(nullIf(deduped.span_name, ''), nullIf(deduped.trace_name, '')), deduped.timestamp, equals(deduped.event, '$ai_trace')), argMin(ifNull(nullIf(deduped.span_name, ''), nullIf(deduped.trace_name, '')), deduped.timestamp)) AS trace_name
- FROM
- (SELECT __ai_events_fallback.uuid AS uuid,
- __ai_events_fallback.event AS event,
- toTimeZone(__ai_events_fallback.timestamp, 'UTC') AS timestamp,
- __ai_events_fallback.distinct_id AS distinct_id,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(__ai_events_fallback.properties.`$feature_flags`), mapValues(__ai_events_fallback.properties.`$feature_flags`)), if(empty(__ai_events_fallback.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), __ai_events_fallback.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.temporary_properties)))), ','), '}') AS properties,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), if(isNull(__ai_events_fallback.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_id`), toString(__ai_events_fallback.properties.`$ai_trace_id`)))) AS trace_id,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), if(isNull(__ai_events_fallback.properties.`$ai_session_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_session_id`), toString(__ai_events_fallback.properties.`$ai_session_id`)))) AS session_id,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), if(isNull(__ai_events_fallback.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_parent_id`), toString(__ai_events_fallback.properties.`$ai_parent_id`)))) AS parent_id,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), if(isNull(__ai_events_fallback.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_span_name`), toString(__ai_events_fallback.properties.`$ai_span_name`)))) AS span_name,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), if(isNull(__ai_events_fallback.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_name`), toString(__ai_events_fallback.properties.`$ai_trace_name`)))) AS trace_name,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_latency`, 'Float64'), 'Float64') AS latency,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_tokens`, 'Float64'), 'Float64') AS input_tokens,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_tokens`, 'Float64'), 'Float64') AS output_tokens,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_cost_usd`, 'Float64'), 'Float64') AS input_cost_usd,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_cost_usd`, 'Float64'), 'Float64') AS output_cost_usd,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64') AS total_cost_usd,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input`), if(isNull(__ai_events_fallback.properties.`$ai_input`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input`))), '')) AS input,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output`), if(isNull(__ai_events_fallback.properties.`$ai_output`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output`))), '')) AS output,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), if(isNull(__ai_events_fallback.properties.`$ai_output_choices`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_choices`))), '')) AS output_choices,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), if(isNull(__ai_events_fallback.properties.`$ai_input_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input_state`))), '')) AS input_state,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), if(isNull(__ai_events_fallback.properties.`$ai_output_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_state`))), '')) AS output_state,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_tools`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_tools`), if(isNull(__ai_events_fallback.properties.`$ai_tools`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_tools`))), '')) AS tools
- FROM events_json AS __ai_events_fallback
- WHERE and(equals(__ai_events_fallback.team_id, 99999), in(__ai_events_fallback.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-11-30 23:50:00', 'UTC'))), lessOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-12-08 00:10:00', 'UTC'))), equals(CAST(__ai_events_fallback.properties.`$ai_trace_id`, 'Nullable(String)'), 'trace1'), ifNull(equals(CAST(__ai_events_fallback.properties.foo, 'Nullable(String)'), 'barz'), 0)))
- LIMIT 1 BY __ai_events_fallback.uuid) AS deduped
+ FROM (
+ SELECT __ai_events_fallback.uuid AS uuid,
+ __ai_events_fallback.event AS event,
+ toTimeZone(__ai_events_fallback.timestamp, 'UTC') AS timestamp,
+ __ai_events_fallback.distinct_id AS distinct_id,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, __ai_events_fallback.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(__ai_events_fallback.properties.`$feature_flags`), mapValues(__ai_events_fallback.properties.`$feature_flags`)), if(empty(__ai_events_fallback.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), __ai_events_fallback.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, __ai_events_fallback.temporary_properties_null_keys))))), ','), '}') AS properties, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), if(isNull(__ai_events_fallback.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_id`), toString(__ai_events_fallback.properties.`$ai_trace_id`)))) AS trace_id, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), if(isNull(__ai_events_fallback.properties.`$ai_session_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_session_id`), toString(__ai_events_fallback.properties.`$ai_session_id`)))) AS session_id, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), if(isNull(__ai_events_fallback.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_parent_id`), toString(__ai_events_fallback.properties.`$ai_parent_id`)))) AS parent_id, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), if(isNull(__ai_events_fallback.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_span_name`), toString(__ai_events_fallback.properties.`$ai_span_name`)))) AS span_name, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), if(isNull(__ai_events_fallback.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_name`), toString(__ai_events_fallback.properties.`$ai_trace_name`)))) AS trace_name, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_latency`, 'Float64'), 'Float64') AS latency, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_tokens`, 'Float64'), 'Float64') AS input_tokens, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_tokens`, 'Float64'), 'Float64') AS output_tokens, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_cost_usd`, 'Float64'), 'Float64') AS input_cost_usd, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_cost_usd`, 'Float64'), 'Float64') AS output_cost_usd, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64') AS total_cost_usd, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input`), if(isNull(__ai_events_fallback.properties.`$ai_input`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input`))), '')) AS input, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output`), if(isNull(__ai_events_fallback.properties.`$ai_output`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output`))), '')) AS output, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), if(isNull(__ai_events_fallback.properties.`$ai_output_choices`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_choices`))), '')) AS output_choices, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), if(isNull(__ai_events_fallback.properties.`$ai_input_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input_state`))), '')) AS input_state, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), if(isNull(__ai_events_fallback.properties.`$ai_output_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_state`))), '')) AS output_state, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_tools`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_tools`), if(isNull(__ai_events_fallback.properties.`$ai_tools`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_tools`))), '')) AS tools
+ FROM events_json AS __ai_events_fallback
+ WHERE and(equals(__ai_events_fallback.team_id, 99999), in(__ai_events_fallback.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-11-30 23:50:00', 'UTC'))), lessOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-12-08 00:10:00', 'UTC'))), equals(CAST(__ai_events_fallback.properties.`$ai_trace_id`, 'Nullable(String)'), 'trace1'), ifNull(equals(CAST(__ai_events_fallback.properties.foo, 'Nullable(String)'), 'barz'), 0)))
+ LIMIT 1 BY __ai_events_fallback.uuid) AS deduped
GROUP BY deduped.trace_id
LIMIT 1 SETTINGS readonly=2,
max_execution_time=60,
@@ -82,36 +77,31 @@
argMinIf(deduped.input_state, deduped.timestamp, equals(deduped.event, '$ai_trace')) AS input_state,
argMinIf(deduped.output_state, deduped.timestamp, equals(deduped.event, '$ai_trace')) AS output_state,
ifNull(argMinIf(ifNull(nullIf(deduped.span_name, ''), nullIf(deduped.trace_name, '')), deduped.timestamp, equals(deduped.event, '$ai_trace')), argMin(ifNull(nullIf(deduped.span_name, ''), nullIf(deduped.trace_name, '')), deduped.timestamp)) AS trace_name
- FROM
- (SELECT __ai_events_fallback.uuid AS uuid,
- __ai_events_fallback.event AS event,
- toTimeZone(__ai_events_fallback.timestamp, 'UTC') AS timestamp,
- __ai_events_fallback.distinct_id AS distinct_id,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(__ai_events_fallback.properties.`$feature_flags`), mapValues(__ai_events_fallback.properties.`$feature_flags`)), if(empty(__ai_events_fallback.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), __ai_events_fallback.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.temporary_properties)))), ','), '}') AS properties,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), if(isNull(__ai_events_fallback.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_id`), toString(__ai_events_fallback.properties.`$ai_trace_id`)))) AS trace_id,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), if(isNull(__ai_events_fallback.properties.`$ai_session_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_session_id`), toString(__ai_events_fallback.properties.`$ai_session_id`)))) AS session_id,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), if(isNull(__ai_events_fallback.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_parent_id`), toString(__ai_events_fallback.properties.`$ai_parent_id`)))) AS parent_id,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), if(isNull(__ai_events_fallback.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_span_name`), toString(__ai_events_fallback.properties.`$ai_span_name`)))) AS span_name,
- if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), if(isNull(__ai_events_fallback.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_name`), toString(__ai_events_fallback.properties.`$ai_trace_name`)))) AS trace_name,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_latency`, 'Float64'), 'Float64') AS latency,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_tokens`, 'Float64'), 'Float64') AS input_tokens,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_tokens`, 'Float64'), 'Float64') AS output_tokens,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_cost_usd`, 'Float64'), 'Float64') AS input_cost_usd,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_cost_usd`, 'Float64'), 'Float64') AS output_cost_usd,
- accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64') AS total_cost_usd,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input`), if(isNull(__ai_events_fallback.properties.`$ai_input`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input`))), '')) AS input,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output`), if(isNull(__ai_events_fallback.properties.`$ai_output`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output`))), '')) AS output,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), if(isNull(__ai_events_fallback.properties.`$ai_output_choices`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_choices`))), '')) AS output_choices,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), if(isNull(__ai_events_fallback.properties.`$ai_input_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input_state`))), '')) AS input_state,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), if(isNull(__ai_events_fallback.properties.`$ai_output_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_state`))), '')) AS output_state,
- JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_tools`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_tools`), if(isNull(__ai_events_fallback.properties.`$ai_tools`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_tools`))), '')) AS tools
- FROM events_json AS __ai_events_fallback
- WHERE and(equals(__ai_events_fallback.team_id, 99999), in(__ai_events_fallback.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-11-30 23:50:00', 'UTC'))), lessOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-12-08 00:10:00', 'UTC'))), equals(CAST(__ai_events_fallback.properties.`$ai_trace_id`, 'Nullable(String)'), 'trace1'), ifNull(equals(CAST(__ai_events_fallback.person_properties.bar, 'Nullable(String)'), 'foo'), 0)))
- LIMIT 1 BY __ai_events_fallback.uuid) AS deduped
+ FROM (
+ SELECT __ai_events_fallback.uuid AS uuid,
+ __ai_events_fallback.event AS event,
+ toTimeZone(__ai_events_fallback.timestamp, 'UTC') AS timestamp,
+ __ai_events_fallback.distinct_id AS distinct_id,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, __ai_events_fallback.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(__ai_events_fallback.properties.`$feature_flags`), mapValues(__ai_events_fallback.properties.`$feature_flags`)), if(empty(__ai_events_fallback.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), __ai_events_fallback.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(__ai_events_fallback.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, __ai_events_fallback.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, __ai_events_fallback.temporary_properties_null_keys))))), ','), '}') AS properties, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_id`), if(isNull(__ai_events_fallback.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_id`), toString(__ai_events_fallback.properties.`$ai_trace_id`)))) AS trace_id, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_session_id`), if(isNull(__ai_events_fallback.properties.`$ai_session_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_session_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_session_id`), toString(__ai_events_fallback.properties.`$ai_session_id`)))) AS session_id, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_parent_id`), if(isNull(__ai_events_fallback.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_parent_id`), toString(__ai_events_fallback.properties.`$ai_parent_id`)))) AS parent_id, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_span_name`), if(isNull(__ai_events_fallback.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_span_name`), toString(__ai_events_fallback.properties.`$ai_span_name`)))) AS span_name, if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_trace_name`), if(isNull(__ai_events_fallback.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(__ai_events_fallback.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(__ai_events_fallback.properties.`$ai_trace_name`), toString(__ai_events_fallback.properties.`$ai_trace_name`)))) AS trace_name, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_latency`, 'Float64'), 'Float64') AS latency, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_tokens`, 'Float64'), 'Float64') AS input_tokens, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_tokens`, 'Float64'), 'Float64') AS output_tokens, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_input_cost_usd`, 'Float64'), 'Float64') AS input_cost_usd, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_output_cost_usd`, 'Float64'), 'Float64') AS output_cost_usd, accurateCastOrNull(accurateCastOrNull(__ai_events_fallback.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64') AS total_cost_usd, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input`), if(isNull(__ai_events_fallback.properties.`$ai_input`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input`))), '')) AS input, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output`), if(isNull(__ai_events_fallback.properties.`$ai_output`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output`))), '')) AS output, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_choices`), if(isNull(__ai_events_fallback.properties.`$ai_output_choices`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_choices`))), '')) AS output_choices, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_input_state`), if(isNull(__ai_events_fallback.properties.`$ai_input_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_input_state`))), '')) AS input_state, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_output_state`), if(isNull(__ai_events_fallback.properties.`$ai_output_state`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_output_state`))), '')) AS output_state, JSONExtractRaw(ifNull(if(notEquals(toJSONString(__ai_events_fallback.properties.^`$ai_tools`), '{}'), toJSONString(__ai_events_fallback.properties.^`$ai_tools`), if(isNull(__ai_events_fallback.properties.`$ai_tools`), NULL, toJSONString(__ai_events_fallback.properties.`$ai_tools`))), '')) AS tools
+ FROM events_json AS __ai_events_fallback
+ WHERE and(equals(__ai_events_fallback.team_id, 99999), in(__ai_events_fallback.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-11-30 23:50:00', 'UTC'))), lessOrEquals(__ai_events_fallback.timestamp, assumeNotNull(toDateTime('2024-12-08 00:10:00', 'UTC'))), equals(CAST(__ai_events_fallback.properties.`$ai_trace_id`, 'Nullable(String)'), 'trace1'), ifNull(equals(CAST(__ai_events_fallback.person_properties.bar, 'Nullable(String)'), 'foo'), 0)))
+ LIMIT 1 BY __ai_events_fallback.uuid) AS deduped
GROUP BY deduped.trace_id
LIMIT 1 SETTINGS readonly=2,
max_execution_time=60,
diff --git a/posthog/hogql_queries/ai/test/__snapshots__/test_traces_query_runner.new_events_schema.ambr b/posthog/hogql_queries/ai/test/__snapshots__/test_traces_query_runner.new_events_schema.ambr
index 967e6159dcad..91501ad1f9a2 100644
--- a/posthog/hogql_queries/ai/test/__snapshots__/test_traces_query_runner.new_events_schema.ambr
+++ b/posthog/hogql_queries/ai/test/__snapshots__/test_traces_query_runner.new_events_schema.ambr
@@ -14,17 +14,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-13 23:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 01:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace1', 'trace2')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -92,17 +104,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-14 23:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 02:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_project_1', 'trace_org_b', 'trace_org_a')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -170,17 +194,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-14 23:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 00:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_org_a')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -248,17 +284,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 00:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 01:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_org_b')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -326,17 +374,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 01:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 02:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_project_1')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -435,17 +495,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 05:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 10:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_10', 'trace_9', 'trace_8', 'trace_7', 'trace_6')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -513,17 +585,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 01:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 10:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_10', 'trace_9', 'trace_8', 'trace_7', 'trace_6', 'trace_5', 'trace_4', 'trace_3', 'trace_2')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -591,17 +675,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-14 23:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2025-01-15 10:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace_10', 'trace_9', 'trace_8', 'trace_7', 'trace_6', 'trace_5', 'trace_4', 'trace_3', 'trace_2', 'trace_1', 'trace_0')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -669,17 +765,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-11-30 23:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-12-01 00:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace1')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -747,17 +855,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-11-30 23:50:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-12-01 00:10:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace1')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -856,17 +976,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-11-30 23:55:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-12-01 00:15:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace1')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
@@ -934,17 +1066,29 @@
round(sumIf(accurateCastOrNull(events.properties.`$ai_request_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS request_cost,
round(sumIf(accurateCastOrNull(events.properties.`$ai_web_search_cost_usd`, 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS web_search_cost,
round(sumIf(accurateCastOrNull(accurateCastOrNull(events.properties.`$ai_total_cost_usd`, 'Float64'), 'Float64'), in(events.event, tuple('$ai_generation', '$ai_embedding'))), 10) AS total_cost,
- arrayDistinct(arraySort(x -> x.3, groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
- argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
- ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
- countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
- any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
- arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
+ arrayDistinct(arraySort(x -> x.3,
+ groupArrayIf(tuple(events.uuid, events.event, toTimeZone(events.timestamp, 'UTC'), concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}')), or(in(events.event, tuple('$ai_metric', '$ai_feedback')), equals(toString(if(notEquals(toJSONString(events.properties.^`$ai_parent_id`), '{}'), toJSONString(events.properties.^`$ai_parent_id`), if(isNull(events.properties.`$ai_parent_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_parent_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_parent_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_parent_id`), toString(events.properties.`$ai_parent_id`))))), toString(if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`)))))))))) AS events,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_input_state`), '{}'), toJSONString(events.properties.^`$ai_input_state`), if(isNull(events.properties.`$ai_input_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_input_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_input_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_input_state`), toString(events.properties.`$ai_input_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS input_state,
+ argMinIf(if(notEquals(toJSONString(events.properties.^`$ai_output_state`), '{}'), toJSONString(events.properties.^`$ai_output_state`), if(isNull(events.properties.`$ai_output_state`), NULL, if(and(in(ascii(toString(events.properties.`$ai_output_state`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_output_state`)), tuple(91, 123))), toJSONString(events.properties.`$ai_output_state`), toString(events.properties.`$ai_output_state`)))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')) AS output_state,
+ ifNull(argMinIf(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'), equals(events.event, '$ai_trace')), argMin(ifNull(if(notEquals(toJSONString(events.properties.^`$ai_span_name`), '{}'), toJSONString(events.properties.^`$ai_span_name`), if(isNull(events.properties.`$ai_span_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_span_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_span_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_span_name`), toString(events.properties.`$ai_span_name`)))), if(notEquals(toJSONString(events.properties.^`$ai_trace_name`), '{}'), toJSONString(events.properties.^`$ai_trace_name`), if(isNull(events.properties.`$ai_trace_name`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_name`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_name`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_name`), toString(events.properties.`$ai_trace_name`))))), toTimeZone(events.timestamp, 'UTC'))) AS trace_name,
+ countIf(or(isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_error`), '{}'), toJSONString(events.properties.^`$ai_error`), if(isNull(events.properties.`$ai_error`), NULL, if(and(in(ascii(toString(events.properties.`$ai_error`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_error`)), tuple(91, 123))), toJSONString(events.properties.`$ai_error`), toString(events.properties.`$ai_error`))))), equals(CAST(events.properties.`$ai_is_error`, 'Nullable(String)'), 'true'))) AS error_count,
+ any(if(notEquals(toJSONString(events.properties.^ai_support_impersonated), '{}'), toJSONString(events.properties.^ai_support_impersonated), if(isNull(events.properties.ai_support_impersonated), NULL, if(and(in(ascii(toString(events.properties.ai_support_impersonated)), tuple(91, 123)), in(ascii(toJSONString(events.properties.ai_support_impersonated)), tuple(91, 123))), toJSONString(events.properties.ai_support_impersonated), toString(events.properties.ai_support_impersonated))))) AS is_support_trace,
+ arrayFilter(x -> ifNull(notEquals(x, ''), 1), arrayDistinct(splitByChar(',', arrayStringConcat(groupArrayIf(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), and(equals(events.event, '$ai_generation'), isNotNull(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ifNull(notEquals(toString(if(notEquals(toJSONString(events.properties.^`$ai_tools_called`), '{}'), toJSONString(events.properties.^`$ai_tools_called`), if(isNull(events.properties.`$ai_tools_called`), NULL, if(and(in(ascii(toString(events.properties.`$ai_tools_called`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_tools_called`)), tuple(91, 123))), toJSONString(events.properties.`$ai_tools_called`), toString(events.properties.`$ai_tools_called`))))), ''), 1))), ',')))) AS tools
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), and(in(events.event, tuple('$ai_span', '$ai_generation', '$ai_embedding', '$ai_metric', '$ai_feedback', '$ai_trace')), and(greaterOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-11-30 23:52:00', 'UTC'))), lessOrEquals(events.timestamp, assumeNotNull(toDateTime('2024-12-01 00:12:00', 'UTC'))))), in(CAST(events.properties.`$ai_trace_id`, 'Nullable(String)'), tuple('trace1')))
GROUP BY if(notEquals(toJSONString(events.properties.^`$ai_trace_id`), '{}'), toJSONString(events.properties.^`$ai_trace_id`), if(isNull(events.properties.`$ai_trace_id`), NULL, if(and(in(ascii(toString(events.properties.`$ai_trace_id`)), tuple(91, 123)), in(ascii(toJSONString(events.properties.`$ai_trace_id`)), tuple(91, 123))), toJSONString(events.properties.`$ai_trace_id`), toString(events.properties.`$ai_trace_id`))))
diff --git a/posthog/hogql_queries/test/__snapshots__/test_events_query_runner.new_events_schema.ambr b/posthog/hogql_queries/test/__snapshots__/test_events_query_runner.new_events_schema.ambr
index 030cc0b7b23f..2e707593fcdf 100644
--- a/posthog/hogql_queries/test/__snapshots__/test_events_query_runner.new_events_schema.ambr
+++ b/posthog/hogql_queries/test/__snapshots__/test_events_query_runner.new_events_schema.ambr
@@ -1,11 +1,25 @@
# serializer version: 1
# name: TestEventsQueryRunner.test_absolute_date_range[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
@@ -261,11 +275,25 @@
# ---
# name: TestEventsQueryRunner.test_cursor_with_star_select[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
@@ -293,11 +321,25 @@
# ---
# name: TestEventsQueryRunner.test_element_chain_property_filter[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
@@ -325,11 +367,25 @@
# ---
# name: TestEventsQueryRunner.test_presorted_events_table[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
@@ -357,11 +413,25 @@
# ---
# name: TestEventsQueryRunner.test_presorted_events_table_multiple_order_by[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
@@ -390,11 +460,25 @@
# ---
# name: TestEventsQueryRunner.test_presorted_events_table_order_by_event[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
@@ -422,11 +506,25 @@
# ---
# name: TestEventsQueryRunner.test_presorted_events_table_order_by_property[new_events_schema]
'''
- SELECT tuple(events.uuid, events.event, concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties)))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
+ SELECT tuple(events.uuid,
+ events.event,
+ concat('{',
+ arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(events.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(events.properties.`$feature_flags`), mapValues(events.properties.`$feature_flags`)), if(empty(events.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), events.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(events.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, events.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, events.temporary_properties_null_keys))))), ','), '}'), toTimeZone(events.timestamp, 'UTC'), events.team_id, events.distinct_id, events.elements_chain, toTimeZone(events.created_at, 'UTC'), events.person_mode) AS `tuple(uuid, event, properties, toTimeZone(timestamp, 'UTC'), team_id, distinct_id, elements_chain, toTimeZone(created_at, 'UTC'), person_mode)`
FROM events_json AS events
WHERE and(equals(events.team_id, 99999), globalIn(events.uuid,
(SELECT events.uuid AS uuid
diff --git a/posthog/hogql_queries/test/__snapshots__/test_sessions_timeline_query_runner.new_events_schema.ambr b/posthog/hogql_queries/test/__snapshots__/test_sessions_timeline_query_runner.new_events_schema.ambr
index f9eb441a24a9..45e95f51d9e9 100644
--- a/posthog/hogql_queries/test/__snapshots__/test_sessions_timeline_query_runner.new_events_schema.ambr
+++ b/posthog/hogql_queries/test/__snapshots__/test_sessions_timeline_query_runner.new_events_schema.ambr
@@ -11,44 +11,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 1001)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 1001)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
@@ -89,44 +95,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 1001)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 1001)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
@@ -167,44 +179,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 3)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 3)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
@@ -245,44 +263,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 1001)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 1001)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
@@ -323,44 +347,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 1001)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 1001)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
@@ -401,44 +431,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), equals(if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id), 'd63e1ecf-f92f-1e50-71f9-501e16e9aebe'))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 1001)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), equals(if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id), 'd63e1ecf-f92f-1e50-71f9-501e16e9aebe'))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 1001)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
@@ -479,44 +515,50 @@
first_value(e.uuid) OVER (PARTITION BY tuple(e.person_id, e.session_id_flip_index)
ORDER BY toInt64(e.timestamp) ASC RANGE BETWEEN 1800 PRECEDING AND CURRENT ROW) AS informal_session_uuid,
dateDiff('s', sre.start_time, sre.end_time) AS recording_duration_s
- FROM
- (SELECT uuid AS uuid,
- person_id AS person_id,
- timestamp AS timestamp,
- event AS event,
- properties AS properties,
- distinct_id AS distinct_id,
- elements_chain AS elements_chain,
- session_id AS session_id,
- sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
- and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
- ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
- FROM
- (SELECT event_source.uuid AS uuid,
- if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
- toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
- event_source.event AS event,
- concat('{', arrayStringConcat(arrayConcat(arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
- and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
- and not (kv.2 = '""'
- and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
- and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))])), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties)))), ','), '}') AS properties,
- event_source.distinct_id AS distinct_id,
- event_source.elements_chain AS elements_chain,
- event_source.properties.`$session_id` AS session_id,
- lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
- FROM events_json AS event_source
- LEFT OUTER JOIN
- (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
- person_distinct_id_overrides.distinct_id AS distinct_id
- FROM person_distinct_id_overrides
- WHERE equals(person_distinct_id_overrides.team_id, 99999)
- GROUP BY person_distinct_id_overrides.distinct_id
- HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
- WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
- ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
- LIMIT 1001)) AS e
+ FROM (
+ SELECT uuid AS uuid,
+ person_id AS person_id,
+ timestamp AS timestamp,
+ event AS event,
+ properties AS properties,
+ distinct_id AS distinct_id,
+ elements_chain AS elements_chain,
+ session_id AS session_id,
+ sum(if(ifNull(equals(session_id, prev_session_id), isNull(session_id)
+ and isNull(prev_session_id)), 0, 1)) OVER (PARTITION BY person_id
+ ORDER BY timestamp ASC ROWS UNBOUNDED PRECEDING) AS session_id_flip_index
+ FROM (
+ SELECT event_source.uuid AS uuid,
+ if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id) AS person_id,
+ toTimeZone(event_source.timestamp, 'UTC') AS timestamp,
+ event_source.event AS event,
+ concat('{', arrayStringConcat(arrayConcat(arrayConcat(if(empty(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys)))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), arrayFilter(kv -> not (kv.2 = '[]'
+ and kv.1 in ('$exception_functions', '$exception_list', '$exception_sources', '$exception_types', '$exception_values', '$mcp_listed_tool_names'))
+ and not (kv.2 = '""'
+ and kv.1 in ('$browser', '$browser_language', '$browser_version', '$config_defaults', '$device_type', '$geoip_city_name', '$geoip_continent_code', '$geoip_continent_name', '$geoip_country_code', '$geoip_country_name', '$geoip_subdivision_1_name', '$geoip_time_zone', '$group_0', '$group_1', '$group_2', '$group_3', '$group_4', '$lib', '$lib_version', '$os', '$os_version', '$session_id', '$timezone', '$window_id'))
+ and kv.1 != '$feature_flags', JSONExtractKeysAndValuesRaw(toJSONString(event_source.properties)))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayFilter(null_path -> null_path[1] != '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.properties_null_keys)))), arrayConcat(arrayMap((key, value) -> concat(toJSONString(concat('$feature/', key)), ':', if(value in ('true', 'false'), value, toJSONString(multiIf(value = '$false', 'false', value = '$true', 'true', value)))), mapKeys(event_source.properties.`$feature_flags`), mapValues(event_source.properties.`$feature_flags`)), if(empty(event_source.properties.`$feature_flags`), [], [concat('"$active_feature_flags":', toJSONString(mapKeys(mapFilter((key, value) -> value not in ('', 'false'), event_source.properties.`$feature_flags`))))]), arrayMap(null_path -> concat(toJSONString(concat('$feature/', null_path[2])), ':null'), arrayFilter(null_path -> length(null_path) = 2
+ and null_path[1] = '$feature_flags', arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.properties_null_keys))))), arrayConcat(if(empty(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys))), arrayMap(kv -> concat(toJSONString(kv.1), ':', kv.2), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_key_run -> concat(toJSONString(null_key_run[1].1), ':', arrayMap((null_paths) -> arrayStringConcat(arrayFold((null_nodes, null_depth) -> arrayFlatten(arrayMap(null_node -> multiIf(empty(null_node.2), [null_node], empty(null_node.2[1]), [('null', CAST([], 'Array(Array(String))'))], null_node.1 = ''
+ or startsWith(null_node.1, '{')
+ or startsWith(null_node.1, '['), arrayMap((null_array) -> arrayConcat([(if(null_array, '[', '{'), CAST([], 'Array(Array(String))'))], arrayMap((null_children) -> arrayFlatten(arrayMap((null_run, null_index) -> [(concat(if(null_index = 1, '', ','), if(null_array, '', concat(toJSONString(null_run[1].2), ':'))), CAST([], 'Array(Array(String))')), (null_run[1].4, null_run[-1].5)], null_children, arrayEnumerate(null_children))), [arrayFilter(null_run -> not null_array
+ or null_run[1].3 = 0, arrayMap((null_child_sorted) -> arraySplit((null_child, null_child_before) -> (null_child.1, null_child.2) != (null_child_before.1, null_child_before.2), null_child_sorted, arrayPopBack(arrayPushFront(null_child_sorted, null_child_sorted[1]))), [arraySort(arrayConcat(if(null_array, arrayMap((null_element, null_index) -> (toInt64(null_index - 1), '', 0, null_element, CAST([], 'Array(Array(String))')), JSONExtractArrayRaw(null_node.1), arrayEnumerate(JSONExtractArrayRaw(null_node.1))), arrayMap(kv -> (toInt64(0), kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(null_node.1))), arrayMap(null_root -> (if(null_array, if(toString(toInt64OrNull(null_root[1][1])) = null_root[1][1], toInt64OrZero(null_root[1][1]), toInt64(-1)), toInt64(0)), if(null_array, '', null_root[1][1]), 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root)), arrayMap((null_tail_sorted) -> arraySplit((null_tail, null_tail_before) -> null_tail[1] != null_tail_before[1], null_tail_sorted, arrayPopBack(arrayPushFront(null_tail_sorted, null_tail_sorted[1]))), [arraySort(null_node.2)])[1])))])[1])])[1], [(if(null_array, ']', '}'), CAST([], 'Array(Array(String))'))]), [startsWith(null_node.1, '[')])[1], [(null_node.1, CAST([], 'Array(Array(String))'))]), null_nodes)), range(arrayMax(arrayMap(null_path -> length(null_path), null_paths)) + 1), [(null_key_run[1].3, null_paths)]).1), [null_key_run[-1].4])[1]), arrayMap((null_entry_sorted) -> arraySplit((null_entry, null_entry_before) -> null_entry.1 != null_entry_before.1, null_entry_sorted, arrayPopBack(arrayPushFront(null_entry_sorted, null_entry_sorted[1]))), [arraySort(arrayConcat(arrayMap(kv -> (kv.1, 0, kv.2, CAST([], 'Array(Array(String))')), JSONExtractKeysAndValuesRaw(toJSONString(event_source.temporary_properties))), arrayMap(null_root_run -> (null_root_run[1][1], 1, '', arrayMap(null_path -> arrayPopFront(null_path), null_root_run)), arrayMap((null_path_sorted) -> arraySplit((null_path, null_path_before) -> null_path[1] != null_path_before[1], null_path_sorted, arrayPopBack(arrayPushFront(null_path_sorted, null_path_sorted[1]))), [arraySort(arrayMap(null_path -> arrayMap(null_segment -> replaceAll(null_segment, '\x252E', '.'), splitByChar('.', null_path)), arrayFilter(null_path -> position(null_path, '.') > 0, event_source.temporary_properties_null_keys)))])[1])))])[1])), arrayMap(null_key -> concat(toJSONString(null_key), ':null'), arrayMap(null_key -> replaceAll(null_key, '\x252E', '.'), arrayFilter(null_key -> position(null_key, '.') = 0, event_source.temporary_properties_null_keys))))), ','), '}') AS properties, event_source.distinct_id AS distinct_id, event_source.elements_chain AS elements_chain, event_source.properties.`$session_id` AS session_id, lagInFrame(event_source.properties.`$session_id`, 1) OVER (PARTITION BY if(not(empty(event_source__override.distinct_id)), event_source__override.person_id, event_source.person_id)
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') ASC) AS prev_session_id
+ FROM events_json AS event_source
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id, person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS event_source__override ON equals(event_source.distinct_id, event_source__override.distinct_id)
+ WHERE and(equals(event_source.team_id, 99999), greater(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), less(event_source.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')))
+ ORDER BY toTimeZone(event_source.timestamp, 'UTC') DESC
+ LIMIT 1001)) AS e
LEFT JOIN
(SELECT session_replay_events.start_time AS start_time,
session_replay_events.end_time AS end_time,
diff --git a/products/batch_exports/backend/temporal/batch_exports.py b/products/batch_exports/backend/temporal/batch_exports.py
index 805d24c19b51..2b9546f3e7fa 100644
--- a/products/batch_exports/backend/temporal/batch_exports.py
+++ b/products/batch_exports/backend/temporal/batch_exports.py
@@ -234,35 +234,6 @@ def select_events_query_template(
return SELECT_FROM_EVENTS_VIEW
-def reads_native_events_source(
- *,
- use_new_events_schema: bool,
- team_id: int,
- interval_start: str | None,
- interval_end: str,
- is_backfill: bool = False,
- backfill_details: BackfillDetails | None = None,
-) -> bool:
- """Whether a run reads the native events source instead of a legacy events table.
-
- Only the native source projects the `$unset` and `$group_set` columns, so a caller that asks for
- them on a run routed to a legacy table builds a query ClickHouse cannot resolve. The routing is
- wider than `is_backfill`: a backfill over recent data still reads `distributed_events_recent`.
- """
- if not use_new_events_schema:
- return False
- return (
- select_events_query_template(
- team_id=team_id,
- interval_start=interval_start,
- interval_end=interval_end,
- is_backfill=is_backfill,
- backfill_details=backfill_details,
- )
- is SELECT_FROM_EVENTS_VIEW_BACKFILL
- )
-
-
def iter_records(
client: ClickHouseClient,
team_id: int,
diff --git a/products/batch_exports/backend/temporal/destinations/http_batch_export.py b/products/batch_exports/backend/temporal/destinations/http_batch_export.py
index ac0251c22f9d..29d40bfc6d27 100644
--- a/products/batch_exports/backend/temporal/destinations/http_batch_export.py
+++ b/products/batch_exports/backend/temporal/destinations/http_batch_export.py
@@ -32,7 +32,6 @@ from products.batch_exports.backend.temporal.batch_exports import (
execute_batch_export_insert_activity,
get_data_interval,
iter_records,
- reads_native_events_source,
start_batch_export_run,
)
from products.batch_exports.backend.temporal.filters import compose_filters_clause
@@ -43,12 +42,6 @@ from products.batch_exports.backend.temporal.utils import handle_non_retryable_e
NON_RETRYABLE_ERROR_TYPES = ("NonRetryableResponseError", "InvalidDestinationURLError")
LOGGER = get_logger(__name__)
-_NATIVE_MUTATION_PROPERTIES = {
- "$set": "set",
- "$set_once": "set_once",
- "$unset": "unset",
- "$group_set": "group_set",
-}
class RetryableResponseError(Exception):
@@ -89,9 +82,9 @@ async def raise_for_status(response: aiohttp.ClientResponse):
raise NonRetryableResponseError(response.status, text)
-def http_default_fields(reads_native_source: bool = False) -> list[BatchExportField]:
+def http_default_fields() -> list[BatchExportField]:
"""Return default fields used in HTTP batch export, currently supporting only migrations."""
- fields = [
+ return [
BatchExportField(expression="uuid", alias="uuid"),
BatchExportField(expression="timestamp", alias="timestamp"),
BatchExportField(expression="_inserted_at", alias="_inserted_at"),
@@ -100,9 +93,6 @@ def http_default_fields(reads_native_source: bool = False) -> list[BatchExportFi
BatchExportField(expression="distinct_id", alias="distinct_id"),
BatchExportField(expression="elements_chain", alias="elements_chain"),
]
- if reads_native_source:
- fields.extend(BatchExportField(expression=alias, alias=alias) for alias in _NATIVE_MUTATION_PROPERTIES.values())
- return fields
class HeartbeatDetails:
@@ -222,17 +212,7 @@ async def insert_into_http_activity(inputs: HttpInsertInputs) -> BatchExportResu
is_backfill = inputs.get_is_backfill()
- # Only the native source projects the mutation columns; a legacy table carries the same keys
- # in its `properties`.
- native_source = reads_native_events_source(
- use_new_events_schema=use_native_schema,
- team_id=inputs.team_id,
- interval_start=interval_start,
- interval_end=inputs.data_interval_end,
- is_backfill=is_backfill,
- backfill_details=inputs.backfill_details,
- )
- fields = http_default_fields(native_source)
+ fields = http_default_fields()
columns = [field["alias"] for field in fields]
filters = model.filters
@@ -330,9 +310,6 @@ async def insert_into_http_activity(inputs: HttpInsertInputs) -> BatchExportResu
properties = row["properties"]
properties = json.loads(properties) if properties else {}
- for property_name, column in _NATIVE_MUTATION_PROPERTIES.items():
- if value := row.get(column):
- properties[property_name] = json.loads(value)
properties["$geoip_disable"] = True
if row["event"] == "$autocapture" and row["elements_chain"] is not None:
diff --git a/products/batch_exports/backend/temporal/sql/events.py b/products/batch_exports/backend/temporal/sql/events.py
index a440f686b372..492bf477aeb7 100644
--- a/products/batch_exports/backend/temporal/sql/events.py
+++ b/products/batch_exports/backend/temporal/sql/events.py
@@ -485,20 +485,29 @@ SETTINGS
# `toJSONString` prints a stored dotted key as `a%2Eb` unless the query sets `json_type_escape_dots_in_keys`,
# which is why the native query's SETTINGS carry it.
-_PROPERTIES_DOCUMENT = event_document_sql("properties", "temporary_properties", "properties.`$feature_flags`")
-_PERSON_PROPERTIES_DOCUMENT = person_document_sql("person_properties")
+_PROPERTIES_DOCUMENT = event_document_sql(
+ "properties",
+ "properties_null_keys",
+ "temporary_properties",
+ "temporary_properties_null_keys",
+ "properties.`$feature_flags`",
+)
+_PERSON_PROPERTIES_DOCUMENT = person_document_sql("person_properties", "person_properties_null_keys")
SERIALIZED_EVENTS_JSON_SOURCE = """(
- SELECT * REPLACE (
- toString(uuid) AS uuid,
- toString(person_id) AS person_id,
- __PROPERTIES_DOCUMENT__ AS properties,
- __PERSON_PROPERTIES_DOCUMENT__ AS person_properties
- ),
- nullIf(toJSONString(temporary_properties.^`$set`), '{}') AS set,
- nullIf(toJSONString(temporary_properties.^`$set_once`), '{}') AS set_once,
- nullIf(toJSONString(temporary_properties.^`$unset`), '[]') AS unset,
- nullIf(toJSONString(temporary_properties.^`$group_set`), '{}') AS group_set
- FROM events_json
+ SELECT *,
+ nullIf(JSONExtractRaw(properties, '$set'), '') AS set,
+ nullIf(JSONExtractRaw(properties, '$set_once'), '') AS set_once,
+ nullIf(JSONExtractRaw(properties, '$unset'), '') AS unset,
+ nullIf(JSONExtractRaw(properties, '$group_set'), '') AS group_set
+ FROM (
+ SELECT * REPLACE (
+ toString(uuid) AS uuid,
+ toString(person_id) AS person_id,
+ __PROPERTIES_DOCUMENT__ AS properties,
+ __PERSON_PROPERTIES_DOCUMENT__ AS person_properties
+ )
+ FROM events_json
+ )
)""".replace("__PROPERTIES_DOCUMENT__", _PROPERTIES_DOCUMENT).replace(
"__PERSON_PROPERTIES_DOCUMENT__", _PERSON_PROPERTIES_DOCUMENT
)
diff --git a/products/batch_exports/backend/tests/temporal/destinations/test_http_batch_export_workflow.py b/products/batch_exports/backend/tests/temporal/destinations/test_http_batch_export_workflow.py
index fb31557ae5f8..0d8565dd1cff 100644
--- a/products/batch_exports/backend/tests/temporal/destinations/test_http_batch_export_workflow.py
+++ b/products/batch_exports/backend/tests/temporal/destinations/test_http_batch_export_workflow.py
@@ -9,7 +9,6 @@ import unittest.mock
from django.conf import settings
from django.test import override_settings
-import pyarrow as pa
import pytest_asyncio
from aioresponses import aioresponses
from temporalio import activity
@@ -251,33 +250,32 @@ async def test_insert_into_http_activity_inserts_data_into_http_endpoint(
)
+@override_settings(CLICKHOUSE_HOGQL_USE_NEW_EVENTS_SCHEMA=True, UNCONSTRAINED_TIMESTAMP_TEAM_IDS=[])
async def test_insert_into_http_activity_preserves_mutation_properties_when_using_native_events(
- activity_environment, http_config
+ clickhouse_client, activity_environment, http_config
):
- """Native ingestion strips the mutation keys out of `properties`, so the export has to put them
- back from the columns the native source carries.
- """
- interval_end = dt.datetime(2026, 1, 2, tzinfo=dt.UTC)
- event_time = interval_end - dt.timedelta(hours=1)
- record_batch = pa.RecordBatch.from_pylist(
- [
- {
- "uuid": str(uuid4()),
- "timestamp": event_time,
- "_inserted_at": event_time,
- "event": "$pageview",
- "properties": "{}",
- "distinct_id": "person-1",
- "elements_chain": None,
- "set": '{"email":"[email protected]"}',
- "set_once": '{"first_seen":"2026-01-01"}',
- "unset": '["old_property"]',
- "group_set": '{"company":{"name":"Example"}}',
- }
- ]
+ interval_end = dt.datetime.now(tz=dt.UTC).replace(microsecond=0) - dt.timedelta(days=10)
+ interval_start = interval_end - dt.timedelta(hours=1)
+ team_id = randint(1, 1000000)
+ properties = {
+ "$set": {"email": "[email protected]", "phone": None},
+ "$set_once": {"referrer": None},
+ "$unset": ["old_property"],
+ "$group_set": {"name": "Example", "owner": None},
+ }
+ await generate_test_events_in_clickhouse(
+ client=clickhouse_client,
+ team_id=team_id,
+ start_time=interval_start,
+ end_time=interval_end,
+ count=1,
+ count_outside_range=0,
+ count_other_team=0,
+ properties=properties,
+ table="sharded_events",
)
insert_inputs = HttpInsertInputs(
- team_id=randint(1, 1000000),
+ team_id=team_id,
data_interval_start=None,
data_interval_end=interval_end.isoformat(),
backfill_details=BackfillDetails(
@@ -289,27 +287,12 @@ async def test_insert_into_http_activity_preserves_mutation_properties_when_usin
)
mock_server = MockServer()
- with (
- unittest.mock.patch(
- "products.batch_exports.backend.temporal.destinations.http_batch_export.use_new_events_schema",
- return_value=True,
- ),
- unittest.mock.patch(
- "products.batch_exports.backend.temporal.destinations.http_batch_export.iter_records",
- return_value=iter([record_batch]),
- ),
- aioresponses(passthrough=[settings.CLICKHOUSE_HTTP_URL]) as responses,
- ):
+ with aioresponses(passthrough=[settings.CLICKHOUSE_HTTP_URL]) as responses:
responses.post(TEST_URL, status=200, callback=mock_server.post, repeat=True)
await activity_environment.run(insert_into_http_activity, insert_inputs)
- assert mock_server.records[0]["properties"] == {
- "$geoip_disable": True,
- "$set": {"email": "[email protected]"},
- "$set_once": {"first_seen": "2026-01-01"},
- "$unset": ["old_property"],
- "$group_set": {"company": {"name": "Example"}},
- }
+ assert len(mock_server.records) == 1
+ assert mock_server.records[0]["properties"] == {**properties, "$geoip_disable": True}
@pytest.mark.parametrize("is_backfill", [False, True], ids=["scheduled", "backfill"])
diff --git a/products/batch_exports/backend/tests/temporal/test_batch_exports.py b/products/batch_exports/backend/tests/temporal/test_batch_exports.py
index 6f8eb359b3a6..ce0f5a83fc96 100644
--- a/products/batch_exports/backend/tests/temporal/test_batch_exports.py
+++ b/products/batch_exports/backend/tests/temporal/test_batch_exports.py
@@ -9,13 +9,7 @@ from django.test import override_settings
from posthog.temporal.tests.utils.events import generate_test_events_in_clickhouse
-from products.batch_exports.backend.service import BackfillDetails
-from products.batch_exports.backend.temporal.batch_exports import (
- DataInterval,
- get_data_interval,
- iter_records,
- reads_native_events_source,
-)
+from products.batch_exports.backend.temporal.batch_exports import DataInterval, get_data_interval, iter_records
pytestmark = [pytest.mark.asyncio, pytest.mark.django_db]
@@ -409,33 +403,3 @@ def test_data_interval_rejects_reversed_bounds() -> None:
end = dt.datetime(2023, 8, 1, tzinfo=dt.UTC)
with pytest.raises(ValueError, match="start"):
DataInterval(start=end + dt.timedelta(seconds=1), end=end)
-
-
- "start_ago,end_ago,is_backfill,backfill_ago,expected",
- [
- (dt.timedelta(hours=2), dt.timedelta(hours=1), False, None, False),
- (dt.timedelta(days=61), dt.timedelta(days=60), False, None, False),
- (dt.timedelta(days=61), dt.timedelta(days=60), True, dt.timedelta(days=61), True),
- (dt.timedelta(hours=2), dt.timedelta(hours=1), True, dt.timedelta(hours=2), False),
- ],
- ids=["scheduled-recent", "scheduled-old", "backfill-old", "backfill-recent"],
-)
-def test_reads_native_events_source_tracks_query_routing(start_ago, end_ago, is_backfill, backfill_ago, expected):
- now = dt.datetime.now(tz=dt.UTC)
- start, end = now - start_ago, now - end_ago
- backfill_details = (
- BackfillDetails(backfill_id=None, start_at=(now - backfill_ago).isoformat(), end_at=end.isoformat())
- if backfill_ago is not None
- else None
- )
- arguments = {
- "team_id": 1,
- "interval_start": start.isoformat(),
- "interval_end": end.isoformat(),
- "is_backfill": is_backfill,
- "backfill_details": backfill_details,
- }
-
- assert reads_native_events_source(use_new_events_schema=True, **arguments) is expected
- assert reads_native_events_source(use_new_events_schema=False, **arguments) is False
diff --git a/products/batch_exports/backend/tests/temporal/test_events_sql.py b/products/batch_exports/backend/tests/temporal/test_events_sql.py
index a08d65d148a4..a0cd9293c5d3 100644
--- a/products/batch_exports/backend/tests/temporal/test_events_sql.py
+++ b/products/batch_exports/backend/tests/temporal/test_events_sql.py
@@ -20,12 +20,11 @@ def test_native_events_export_query_refuses_stale_replicas() -> None:
assert "fallback_to_stale_replicas_for_distributed_queries=0" in query
-def test_native_events_export_query_keeps_empty_object_literals() -> None:
- query = ClickHouseClient().prepare_query(native_events_export_query("event"), QUERY_PARAMETERS)
+def test_native_events_export_query_keeps_empty_object_literals_in_custom_fields() -> None:
+ query = ClickHouseClient().prepare_query(native_events_export_query("'{}' AS properties"), QUERY_PARAMETERS)
assert "'{0}'" not in query
- for key in ("$set", "$set_once", "$group_set"):
- assert f"temporary_properties.^`{key}`), '{{}}')" in query
+ assert "'{}'" in query
# Without these settings `toJSONString(properties)` prints a stored dotted key as `a%2Eb` and a `/` as `\/`.
diff --git a/products/batch_exports/backend/tests/temporal/test_record_batch_model.py b/products/batch_exports/backend/tests/temporal/test_record_batch_model.py
index d0d0d85b400f..a914cb2d5307 100644
--- a/products/batch_exports/backend/tests/temporal/test_record_batch_model.py
+++ b/products/batch_exports/backend/tests/temporal/test_record_batch_model.py
@@ -813,8 +813,10 @@ async def test_custom_export_backfill_runs_without_legacy_events_tables(
f"CREATE TABLE events_json (uuid UUID, person_id UUID, team_id Int64, event String, distinct_id String, "
f"timestamp DateTime64(6), inserted_at DateTime64(6), created_at DateTime64(6), elements_chain String, "
f"properties {EVENTS_PROPERTIES_JSON_TYPE()}, person_properties {PERSON_PROPERTIES_JSON_TYPE()}, "
- "temporary_properties JSON) ENGINE = Memory"
+ "temporary_properties JSON, properties_null_keys Array(String), "
+ "temporary_properties_null_keys Array(String), person_properties_null_keys Array(String)) ENGINE = Memory"
)
+ # The null-key columns hold what the cleaner records for fields the SDK sent as null.
row = {
"uuid": str(uuid7()),
"person_id": str(uuid7()),
@@ -829,10 +831,14 @@ async def test_custom_export_backfill_runs_without_legacy_events_tables(
"$browser": "Firefox",
"amount": 2.5,
"person": {"properties": "event value"},
+ "cart": {"items": 2},
"$feature_flags": {"named-false": "$false", "named-true": "$true", "some-feature": "true"},
},
"person_properties": {"email": "[email protected]"},
- "temporary_properties": {"$set": {"email": "[email protected]"}},
+ "temporary_properties": {"$set": {"email": "[email protected]"}, "$unset": ["old_property"]},
+ "properties_null_keys": ["coupon", "cart.note", "$feature_flags.retired-flag"],
+ "temporary_properties_null_keys": ["$set.phone", "$set_once.referrer", "$group_set.owner"],
+ "person_properties_null_keys": ["nickname"],
}
await client.execute_query(
"INSERT INTO events_json FORMAT JSONEachRow\n"
@@ -866,7 +872,10 @@ async def test_custom_export_backfill_runs_without_legacy_events_tables(
interval_end="2024-01-02 00:00:00",
include_events=["purchase"],
fields=fields
- + [{"expression": key, "alias": key} for key in ("properties", "person_properties", "set")],
+ + [
+ {"expression": key, "alias": key}
+ for key in ("properties", "person_properties", "set", "set_once", "unset", "group_set")
+ ],
filters_str=predicate,
extra_query_parameters=values,
is_backfill=True,
@@ -884,14 +893,22 @@ async def test_custom_export_backfill_runs_without_legacy_events_tables(
assert rows[0]["named_true"] == "true"
exported_properties = json.loads(rows[0]["properties"])
assert exported_properties["$browser"] == "Firefox"
- assert exported_properties["$set"] == {"email": "[email protected]"}
+ assert exported_properties["$set"] == {"email": "[email protected]", "phone": None}
assert "$feature_flags" not in exported_properties
assert exported_properties["$feature/some-feature"] is True
assert exported_properties["$feature/named-false"] == "false"
assert exported_properties["$feature/named-true"] == "true"
assert exported_properties["$active_feature_flags"] == ["named-false", "named-true", "some-feature"]
- assert json.loads(rows[0]["person_properties"])["email"] == "[email protected]"
- assert json.loads(rows[0]["set"]) == {"email": "[email protected]"}
+ assert {key: exported_properties[key] for key in ("coupon", "cart", "$feature/retired-flag")} == {
+ "coupon": None,
+ "cart": {"items": 2, "note": None},
+ "$feature/retired-flag": None,
+ }
+ assert json.loads(rows[0]["person_properties"]) == {"email": "[email protected]", "nickname": None}
+ assert json.loads(rows[0]["set"]) == {"email": "[email protected]", "phone": None}
+ assert json.loads(rows[0]["set_once"]) == {"referrer": None}
+ assert json.loads(rows[0]["unset"]) == ["old_property"]
+ assert json.loads(rows[0]["group_set"]) == {"owner": None}
assert rows[0]["_inserted_at"] == dt.datetime(2024, 1, 1, 12, tzinfo=dt.UTC)
finally:
await clickhouse_client.execute_query(f"DROP DATABASE {database}")