:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Python ab-testingai-analyticsanalyticscdpdata-warehouseexperimentsfeature-flagsjavascriptproduct-analyticspythonreactsession-replaysurveystypescriptweb-analytics
fix(funnels): align drop-off recordings with optional steps (#117315)
posthog[bot]f155b3e1020c
diff --git a/funnel-udf/Cross.toml b/funnel-udf/Cross.tomlnew file mode 100644index 000000000000..66635e3a5196--- /dev/null+++ b/funnel-udf/Cross.toml@@ -0,0 +1,7 @@+# The default images follow cross's main branch, so a rebuild can silently raise the glibc version that+# ClickHouse hosts need to run the binaries. These images link against glibc 2.23.+[target.x86_64-unknown-linux-gnu]+image = "ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.5"++[target.aarch64-unknown-linux-gnu]+image = "ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5"diff --git a/funnel-udf/README.md b/funnel-udf/README.mdindex edbe18f2d5cb..e1f78a544ee6 100644--- a/funnel-udf/README.md+++ b/funnel-udf/README.md@@ -10,21 +10,27 @@## Deployment workflow-The `user_defined_function.xml` deployed to ClickHouse is now pulled automatically from the `master` branch of this repo by ansible (`posthog-cloud-infra/ansible/roles/clickhouse/tasks/setup_udfs.yml`).-It downloads `posthog/user_scripts/latest_user_defined_function.xml` and deploys it as ClickHouse's `user_defined_function.xml`.+Cloud ClickHouse does not read the `master` branch of this repo.+Ansible (`posthog-cloud-infra/ansible/roles/clickhouse/tasks/setup_udfs.yml`) downloads this repo at the commit pinned in `posthog_udfs_revision` (`ansible/roles/clickhouse/defaults/main.yml`).+It copies that commit's `posthog/user_scripts` to the ClickHouse nodes, installs its `latest_user_defined_function.xml` as ClickHouse's `user_defined_function.xml`, and reloads the functions.+A merge here changes nothing in cloud ClickHouse until the pin moves.For revertible cloud deploys:1. Develop using the binary files at the top level of `user_scripts` (see section above), with schema defined in `docker/clickhouse/user_defined_function.xml`2. When ready to deploy, increment the version in `posthog/udf_versioner.py` and run it. This generates versioned binaries and `latest_user_defined_function.xml`-3. Land a PR with the updated `user_scripts` folder — include both the new version folder (e.g. `v12/`) and the regenerated `latest_user_defined_function.xml`. **Do not** include the `UDF_VERSION` bump in this PR — that goes in a separate PR (step 5)-4. The binaries and XML should be automatically deployed to ClickHouse on the next config run. Verify in metabase by running `SELECT aggregate_funnel_vXX()`. An "invalid arguments" error means the function is registered. An "unknown function" error means the deploy hasn't taken effect yet. **Make sure to do this for both EU and US**-5. Once the deploy is confirmed, land a separate PR that bumps `UDF_VERSION` in `posthog/udf_versioner.py` to the new version. This switches the posthog query builder to use the new UDF+3. Land a PR with the updated `user_scripts` folder — include both the new version folder (e.g. `v12/`) and the regenerated `latest_user_defined_function.xml`. **Do not** include the `UDF_VERSION` bump in this PR — that goes in a separate PR (step 6). Check that git keeps the wrappers, binaries and Python scripts in the version folder executable (mode `100755`), because local, CI and self-hosted ClickHouse run them directly+4. In `posthog-cloud-infra`, bump `posthog_udfs_revision` to a `master` commit that includes the PR from step 3. Merging that change deploys the pinned `user_scripts` to every ClickHouse environment at once. The bump also ships every other `user_scripts` change since the previous pin, and the unversioned utility UDFs take effect immediately, so list those changes in the infra PR+5. Verify in metabase by running `SELECT aggregate_funnel_vXX()`. An "invalid arguments" error means the function is registered. An "unknown function" error means the deploy hasn't taken effect yet. Also run a valid funnel query against the new version. Registration alone does not check the launcher or binary. **Make sure to do this for both EU and US**+6. Once the deploy is confirmed, land a separate PR that bumps `UDF_VERSION` in `posthog/udf_versioner.py` to the new version. This switches the posthog query builder to use the new UDF## Design decisions+- Each replay slot holds the events of its own step, optional steps included. Optional matches never change funnel counts, because the duplicate-event check only sees required steps.+- Unversioned binaries use the current event layout. Replay queries must use that layout even while cloud queries select an older UDF version.- We tried writing the UDFs in Python, but this cause ClickHouse nodes to crash as the garbage collector could not keep up. As such we're now writing the functions in Rust.- We're using `cross-rs` to compile the binaries - it's a cross platfrom rust compiler.+- `Cross.toml` pins the build images, which link against glibc 2.23. Check the highest version a new binary needs with `objdump -T <binary> | grep -o 'GLIBC_[0-9.]*' | sort -uV | tail -1`.## Troubleshootingdiff --git a/funnel-udf/src/steps.rs b/funnel-udf/src/steps.rsindex 00bed1cbc99c..a165613cb33e 100644--- a/funnel-udf/src/steps.rs+++ b/funnel-udf/src/steps.rs@@ -14,6 +14,11 @@ pub struct EnteredTimestamp {pub timings: Vec<f64>,pub uuids: Vec<Uuid>,pub steps: u32,+ // The event of every step on the path, optional steps included, in step order. Only the replay slots read it.+ // `uuids` keeps the required steps only, because the duplicate-event check reads it and an optional step must+ // not change which paths count.+ #[serde(default)]+ pub step_uuids: Vec<Uuid>,}#[derive(Debug, Clone, Deserialize)]@@ -65,6 +70,7 @@ pub const DEFAULT_ENTERED_TIMESTAMP: EnteredTimestamp = EnteredTimestamp {timings: vec![],uuids: vec![],steps: 0,+ step_uuids: vec![],};pub fn run(args: &Args) -> Vec<Result> {@@ -143,6 +149,7 @@ impl AggregateFunnelRow {timings: vec![],uuids: vec![],steps: 0,+ step_uuids: vec![],};if events_with_same_timestamp.len() == 1 {@@ -209,11 +216,11 @@ impl AggregateFunnelRow {.filter(|i| **i <= final_index as i8).count();- for i in 0..(final_index - optional_count) {- //if event_uuids[i].len() >= MAX_REPLAY_EVENTS && !event_uuids[i].contains(&final_value.uuids[i]) {+ let matched_steps = (0..final_index).filter(|i| final_value.steps & (1 << i) != 0);+ for (step_index, uuid) in matched_steps.zip(&final_value.step_uuids) {// Always put the actual event uuids first, we use it to extract timestamps// This might create duplicates, but that's fine (we can remove it in clickhouse)- vars.event_uuids[i].insert(0, final_value.uuids[i].clone());+ vars.event_uuids[step_index].insert(0, *uuid);}self.results.push(Result(@@ -301,18 +308,21 @@ impl AggregateFunnelRow {t.push(event.timestamp);u.push(event.uuid);}+ let mut s = previous_step.step_uuids.clone();+ s.push(event.uuid);let new_entered_timestamp = EnteredTimestamp {timestamp: previous_step.timestamp,excluded: previous_step.excluded,timings: t,uuids: u,steps: previous_step.steps | (1 << (step - 1)),+ step_uuids: s,};if !previous_step.excluded {vars.entered_timestamp[step] = new_entered_timestamp.clone();- if vars.event_uuids[previous_step_index].len() < MAX_REPLAY_EVENTS - 1 {- vars.event_uuids[previous_step_index].push(event.uuid);+ if vars.event_uuids[step - 1].len() < MAX_REPLAY_EVENTS - 1 {+ vars.event_uuids[step - 1].push(event.uuid);}}@@ -337,3 +347,138 @@ impl AggregateFunnelRow {}}}++#[cfg(test)]+mod tests {+ use super::*;+ use rstest::rstest;++ #[rstest]+ #[case(vec![1, 4])]+ #[case(vec![1, 2, 4])]+ #[case(vec![1, 2, 3, 4])]+ fn matched_events_keep_their_funnel_step(#[case] visited_steps: Vec<i8>) {+ let args = Args {+ num_steps: 5,+ conversion_window_limit: 3600,+ breakdown_attribution_type: "first_touch".to_string(),+ funnel_order_type: "ordered".to_string(),+ prop_vals: vec![PropVal::Int(0)],+ optional_steps: vec![2, 3],+ value: visited_steps+ .iter()+ .map(|&step| Event {+ timestamp: f64::from(step),+ uuid: Uuid::from_u128(step as u128),+ breakdown: PropVal::Int(0),+ steps: vec![step],+ })+ .collect(),+ };++ let results = run(&args);+ assert_eq!(results.len(), 1);+ let result = &results[0];+ assert_eq!(result.0, 1);+ assert_eq!(result.2, vec![3.0]);+ for step in 1..=args.num_steps {+ if visited_steps.contains(&(step as i8)) {+ assert_eq!(result.3[step - 1][0], Uuid::from_u128(step as u128));+ assert!(result.3[step - 1]+ .iter()+ .all(|uuid| *uuid == Uuid::from_u128(step as u128)));+ } else {+ assert!(result.3[step - 1].is_empty());+ }+ }+ }++ #[test]+ fn optional_step_starts_with_the_event_from_the_selected_path() {+ let args = Args {+ num_steps: 3,+ conversion_window_limit: 3600,+ breakdown_attribution_type: "first_touch".to_string(),+ funnel_order_type: "ordered".to_string(),+ prop_vals: vec![PropVal::Int(0)],+ optional_steps: vec![2],+ value: [1, 2, 1, 2, 3]+ .iter()+ .enumerate()+ .map(|(index, &step)| Event {+ timestamp: (index + 1) as f64,+ uuid: Uuid::from_u128((index + 1) as u128),+ breakdown: PropVal::Int(0),+ steps: vec![step],+ })+ .collect(),+ };++ let results = run(&args);+ assert_eq!(results[0].2, vec![2.0]);+ for (step, uuid) in [3, 4, 5].iter().enumerate() {+ assert_eq!(results[0].3[step][0], Uuid::from_u128(*uuid));+ }+ }++ // Each event is (timestamp, uuid, steps). The expected counts are the ones the previous UDF version returns.+ #[rstest]+ #[case::optional_and_next_required(+ 3,+ vec![2],+ vec![(1.0, 1, vec![1]), (2.0, 2, vec![2, 3]), (2.0, 3, vec![1])],+ 1,+ 0b111,+ vec![3, 2, 2]+ )]+ #[case::two_optionals_and_next_required(+ 4,+ vec![2, 3],+ vec![(1.0, 1, vec![1]), (2.0, 2, vec![2, 3, 4]), (2.0, 3, vec![1])],+ 1,+ 0b1111,+ vec![3, 2, 2, 2]+ )]+ #[case::optional_and_later_required(+ 4,+ vec![2],+ vec![(1.0, 1, vec![1]), (2.0, 10, vec![2, 4]), (2.0, 11, vec![3])],+ 2,+ 0b1111,+ vec![1, 10, 11, 10]+ )]+ fn optional_step_matches_do_not_change_counts(+ #[case] num_steps: usize,+ #[case] optional_steps: Vec<i8>,+ #[case] events: Vec<(f64, u128, Vec<i8>)>,+ #[case] step_reached: i8,+ #[case] steps_bitfield: u32,+ #[case] first_event_per_step: Vec<u128>,+ #[values("ordered", "strict")] funnel_order_type: &str,+ ) {+ let args = Args {+ num_steps,+ conversion_window_limit: 3600,+ breakdown_attribution_type: "first_touch".to_string(),+ funnel_order_type: funnel_order_type.to_string(),+ prop_vals: vec![PropVal::Int(0)],+ optional_steps,+ value: events+ .into_iter()+ .map(|(timestamp, uuid, steps)| Event {+ timestamp,+ uuid: Uuid::from_u128(uuid),+ breakdown: PropVal::Int(0),+ steps,+ })+ .collect(),+ };++ let results = run(&args);+ assert_eq!(results[0].0, step_reached);+ assert_eq!(results[0].4, steps_bitfield);+ for (step, uuid) in first_event_per_step.iter().enumerate() {+ assert_eq!(results[0].3[step][0], Uuid::from_u128(*uuid));+ }+ }+}diff --git a/funnel-udf/src/unordered_steps.rs b/funnel-udf/src/unordered_steps.rsindex cec710ee0822..79c2dfee48a3 100644--- a/funnel-udf/src/unordered_steps.rs+++ b/funnel-udf/src/unordered_steps.rs@@ -197,6 +197,7 @@ impl AggregateFunnelRowUnordered {timings,uuids,steps: 0,+ step_uuids: vec![],},);}diff --git a/posthog/user_scripts/aggregate_funnel_aarch64 b/posthog/user_scripts/aggregate_funnel_aarch64index 1d95a94b7ea5..43dc41faeca3 100755Binary files a/posthog/user_scripts/aggregate_funnel_aarch64 and b/posthog/user_scripts/aggregate_funnel_aarch64 differdiff --git a/posthog/user_scripts/aggregate_funnel_x86_64 b/posthog/user_scripts/aggregate_funnel_x86_64index e758982049d9..82c05e9c94ad 100755Binary files a/posthog/user_scripts/aggregate_funnel_x86_64 and b/posthog/user_scripts/aggregate_funnel_x86_64 differdiff --git a/posthog/user_scripts/latest_user_defined_function.xml b/posthog/user_scripts/latest_user_defined_function.xmlindex 639e7db7f770..42b7688a7cd9 100644--- a/posthog/user_scripts/latest_user_defined_function.xml+++ b/posthog/user_scripts/latest_user_defined_function.xml@@ -1,5 +1,5 @@<functions>- <!-- Version: v12. Generated at: 2026-09-10T10:15:07.735801+00:00+ <!-- Version: v13. Generated at: 2026-10-27T10:48:41.604645+00:00This file is autogenerated by udf_versioner.py. Do not edit this, only edit the version at docker/clickhouse/user_defined_function.xml--><function><type>executable_pool</type><name>aggregate_funnel_v11</name>@@ -916,6 +916,601 @@ This file is autogenerated by udf_versioner.py. Do not edit this, only edit the<command>v12/json_drop_keys_udf {keys_parameter:Array(String)}</command></function>+ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_v13</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_v13</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_v13</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_json_v13</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --json --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_json_v13</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --json --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_json_v13</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --json --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_json_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --json --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_json_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --json --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_json_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --json --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_test_v13</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_test_v13</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel_array_trends_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable</type>+ <name>JSONDropKeys_v13</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>v13/json_drop_keys_udf {keys_parameter:Array(String)}</command>+ </function>+<function><type>executable_pool</type><name>JSONCleanPostHogEvent</name>@@ -996,6 +1591,7 @@ This file is autogenerated by udf_versioner.py. Do not edit this, only edit the<stderr_reaction>throw</stderr_reaction><deterministic>true</deterministic></function>+<function><type>executable_pool</type><name>JSONDropKeysPool</name>diff --git a/posthog/user_scripts/v13/aggregate_funnel b/posthog/user_scripts/v13/aggregate_funnelnew file mode 100755index 000000000000..b7e180ec43c7--- /dev/null+++ b/posthog/user_scripts/v13/aggregate_funnel@@ -0,0 +1,2 @@+#!/bin/bash+. "$(dirname "$0")/run_udf.sh"diff --git a/posthog/user_scripts/v13/aggregate_funnel_aarch64 b/posthog/user_scripts/v13/aggregate_funnel_aarch64new file mode 100755index 000000000000..43dc41faeca3Binary files /dev/null and b/posthog/user_scripts/v13/aggregate_funnel_aarch64 differdiff --git a/posthog/user_scripts/v13/aggregate_funnel_array_trends_test.py b/posthog/user_scripts/v13/aggregate_funnel_array_trends_test.pynew file mode 100755index 000000000000..97da07a2e72d--- /dev/null+++ b/posthog/user_scripts/v13/aggregate_funnel_array_trends_test.py@@ -0,0 +1,16 @@+#!/usr/bin/python3++import os+import sys+import json++if __name__ == "__main__":+ # ClickHouse starts this script with copies of its open client connections. A retained copy keeps a connection+ # open after the server closes it, so the client's next request on that connection never gets a response.+ os.closerange(3, os.sysconf("SC_OPEN_MAX"))+ for line in sys.stdin:+ try:+ print(json.dumps({"result": line})) # noqa: T201+ except Exception as e:+ print(json.dumps({"result": json.dumps(str(e))}), end="\n") # noqa: T201+ sys.stdout.flush()diff --git a/posthog/user_scripts/v13/aggregate_funnel_test.py b/posthog/user_scripts/v13/aggregate_funnel_test.pynew file mode 100755index 000000000000..1473a11dfd6f--- /dev/null+++ b/posthog/user_scripts/v13/aggregate_funnel_test.py@@ -0,0 +1,16 @@+#!/usr/bin/python3+import os+import sys+import json+import traceback++if __name__ == "__main__":+ # ClickHouse starts this script with copies of its open client connections. A retained copy keeps a connection+ # open after the server closes it, so the client's next request on that connection never gets a response.+ os.closerange(3, os.sysconf("SC_OPEN_MAX"))+ for line in sys.stdin:+ try:+ print(json.dumps({"result": line})) # noqa: T201+ except Exception as e:+ print(json.dumps({"result": json.dumps(str(e) + traceback.format_exc())}), end="\n") # noqa: T201+ sys.stdout.flush()diff --git a/posthog/user_scripts/v13/aggregate_funnel_x86_64 b/posthog/user_scripts/v13/aggregate_funnel_x86_64new file mode 100755index 000000000000..82c05e9c94adBinary files /dev/null and b/posthog/user_scripts/v13/aggregate_funnel_x86_64 differdiff --git a/posthog/user_scripts/v13/json_drop_keys_udf b/posthog/user_scripts/v13/json_drop_keys_udfnew file mode 100755index 000000000000..b7e180ec43c7--- /dev/null+++ b/posthog/user_scripts/v13/json_drop_keys_udf@@ -0,0 +1,2 @@+#!/bin/bash+. "$(dirname "$0")/run_udf.sh"diff --git a/posthog/user_scripts/v13/json_drop_keys_udf_aarch64 b/posthog/user_scripts/v13/json_drop_keys_udf_aarch64new file mode 100755index 000000000000..b1a06be34547Binary files /dev/null and b/posthog/user_scripts/v13/json_drop_keys_udf_aarch64 differdiff --git a/posthog/user_scripts/v13/json_drop_keys_udf_x86_64 b/posthog/user_scripts/v13/json_drop_keys_udf_x86_64new file mode 100755index 000000000000..91ac16dd6e10Binary files /dev/null and b/posthog/user_scripts/v13/json_drop_keys_udf_x86_64 differdiff --git a/posthog/user_scripts/v13/run_udf.sh b/posthog/user_scripts/v13/run_udf.shnew file mode 100644index 000000000000..b4c59bc14cda--- /dev/null+++ b/posthog/user_scripts/v13/run_udf.sh@@ -0,0 +1,36 @@+# shellcheck shell=bash+# Sourced by every wrapper in this directory. The wrapper's file name is the base name of the binary it starts, so+# every wrapper is the same two lines. Each version directory carries its own copy of this file, because a deployed+# version must not depend on files outside it.++# ClickHouse starts a wrapper with copies of its open files, including client connections. A connection stays open+# while any copy of it is open, so a long-running UDF process keeps a connection open after the server closes it,+# and the next request on that connection never gets a response. Close every inherited descriptor except stdin,+# stdout and stderr before anything else. The wrappers run under bash because sh cannot close a descriptor above 9.+for fd_path in /proc/$$/fd/*; do+ fd=${fd_path##*/}+ if (( fd > 2 )); then+ exec {fd}>&-+ fi+done++DIR_NAME=$(dirname "$0")+UDF_NAME=$(basename "$0")++TEMP_DIR=$(mktemp -d -p /tmp "${UDF_NAME}_XXXXXXXXXX")+trap 'rm -rf "$TEMP_DIR"; exit' 0++case $(uname -m) in+aarch64)+ EXECUTABLE_NAME="${UDF_NAME}_aarch64"+ ;;+*)+ EXECUTABLE_NAME="${UDF_NAME}_x86_64"+ ;;+esac++# A deploy replaces the binaries in user_scripts while pooled processes still run them. Each process runs a private+# copy, so the replacement never touches a file a process is executing.+cp "$DIR_NAME/$EXECUTABLE_NAME" "$TEMP_DIR/$EXECUTABLE_NAME"+chmod +x "$TEMP_DIR/$EXECUTABLE_NAME"+"$TEMP_DIR/$EXECUTABLE_NAME" "$@"diff --git a/posthog/user_scripts/v13/user_defined_function.xml b/posthog/user_scripts/v13/user_defined_function.xmlnew file mode 100644index 000000000000..42b7688a7cd9--- /dev/null+++ b/posthog/user_scripts/v13/user_defined_function.xml@@ -0,0 +1,1613 @@+<functions>+ <!-- Version: v13. Generated at: 2026-10-27T10:48:41.604645+00:00+This file is autogenerated by udf_versioner.py. Do not edit this, only edit the version at docker/clickhouse/user_defined_function.xml--><function>+ <type>executable_pool</type>+ <name>aggregate_funnel_v11</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel steps</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_v11</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel steps</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_v11</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel steps</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_test_v11</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_v11</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel trends</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_v11</name>++ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel trends</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_v11</name>++ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel trends</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_test_v11</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <command>v11/aggregate_funnel_array_trends_test.py</command>+ <lifetime>600</lifetime>+ </function>+<function>+ <type>executable_pool</type>+ <name>aggregate_funnel_v12</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel steps --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_v12</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel steps --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_v12</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel steps --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_v12</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel trends --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_v12</name>+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel trends --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_v12</name>+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel trends --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_json_v12</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel steps --json --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_json_v12</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel steps --json --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_json_v12</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel steps --json --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_json_v12</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel trends --json --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_json_v12</name>+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel trends --json --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_json_v12</name>+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel trends --json --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_test_v12</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_test_v12</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v12/aggregate_funnel_array_trends_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable</type>+ <name>JSONDropKeys_v12</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>v12/json_drop_keys_udf {keys_parameter:Array(String)}</command>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_v13</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_v13</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_v13</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>RowBinaryWithNamesAndTypes</format>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_json_v13</name>+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --json --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_json_v13</name>+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --json --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_json_v13</name>+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel steps --json --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_trends_json_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Nullable(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --json --variant=plain</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_json_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --json --variant=array</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_cohort_trends_json_v13</name>+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(UInt64)</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel trends --json --variant=cohort</command>+ <lifetime>600</lifetime>+ </function>++++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_test_v13</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Int8)</type>+ <name>optional_steps</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable_pool</type>+ <name>aggregate_funnel_array_trends_test_v13</name>+ <return_type>String</return_type>+ <return_name>result</return_name>+ <argument>+ <type>UInt8</type>+ <name>from_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>to_step</name>+ </argument>+ <argument>+ <type>UInt8</type>+ <name>num_steps</name>+ </argument>+ <argument>+ <type>UInt64</type>+ <name>conversion_window_limit</name>+ </argument>+ <argument>+ <type>String</type>+ <name>breakdown_attribution_type</name>+ </argument>+ <argument>+ <type>String</type>+ <name>funnel_order_type</name>+ </argument>+ <argument>+ <type>Array(Array(String))</type>+ <name>prop_vals</name>+ </argument>+ <argument>+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>+ <name>value</name>+ </argument>+ <format>JSONEachRow</format>+ <stderr_reaction>throw</stderr_reaction>+ <command>v13/aggregate_funnel_array_trends_test.py</command>+ <lifetime>600</lifetime>+ </function>++ <function>+ <type>executable</type>+ <name>JSONDropKeys_v13</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>v13/json_drop_keys_udf {keys_parameter:Array(String)}</command>+ </function>++ <function>+ <type>executable_pool</type>+ <name>JSONCleanPostHogEvent</name>+ <return_type>Tuple(properties String, temporary_properties String, person_properties String, properties_null_keys Array(String), temporary_properties_null_keys Array(String), person_properties_null_keys Array(String))</return_type>+ <argument>+ <type>String</type>+ <name>properties</name>+ </argument>+ <argument>+ <type>String</type>+ <name>person_properties</name>+ </argument>+ <format>JSONEachRow</format>+ <command>json_clean_posthog_event_properties_udf --chunked --event</command>+ <send_chunk_header>true</send_chunk_header>+ </function>++ <function>+ <type>executable_pool</type>+ <name>JSONCleanPostHogEventProperties</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>json_clean_posthog_event_properties_udf --chunked</command>+ <send_chunk_header>true</send_chunk_header>+ </function>++ <function>+ <type>executable_pool</type>+ <name>JSONCleanPostHogPersonProperties</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>json_clean_posthog_event_properties_udf --chunked --person-properties</command>+ <send_chunk_header>true</send_chunk_header>+ </function>++ <function>+ <type>executable_pool</type>+ <name>JSONCleanPostHogTemporaryProperties</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>json_clean_posthog_event_properties_udf --chunked --temporary-properties</command>+ <send_chunk_header>true</send_chunk_header>+ </function>++ <function>+ <type>executable_pool</type>+ <name>JSONStripEmptyStringsAndNulls</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <format>Raw</format>+ <command>json_strip_empty_strings_and_nulls_udf --chunked</command>+ <send_chunk_header>true</send_chunk_header>+ </function>+ <function>+ <type>executable_pool</type>+ <name>decompress</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ </argument>+ <argument>+ <type>String</type>+ </argument>+ <format>RowBinary</format>+ <command>decompress_udf</command>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <deterministic>true</deterministic>+ </function>++ <function>+ <type>executable_pool</type>+ <name>JSONDropKeysPool</name>+ <return_type>String</return_type>+ <argument>+ <type>String</type>+ <name>json</name>+ </argument>+ <argument>+ <type>Array(String)</type>+ <name>keys</name>+ </argument>+ <format>RowBinary</format>+ <command>json_drop_keys_udf --row-binary</command>+ <send_chunk_header>true</send_chunk_header>+ <stderr_reaction>throw</stderr_reaction>+ <deterministic>true</deterministic>+ </function>+</functions>\ No newline at end of filediff --git a/products/product_analytics/backend/hogql_queries/funnels/funnel.py b/products/product_analytics/backend/hogql_queries/funnels/funnel.pyindex 3ef6b4c1281a..9798d4bd2592 100644--- a/products/product_analytics/backend/hogql_queries/funnels/funnel.py+++ b/products/product_analytics/backend/hogql_queries/funnels/funnel.py@@ -6,9 +6,11 @@ from posthog.schema import BreakdownAttributionType, BreakdownType, StepOrderValfrom posthog.hogql import astfrom posthog.hogql.constants import DEFAULT_RETURNED_ROWS, HogQLQuerySettings+from posthog.hogql.functions.udfs import UDFSfrom posthog.hogql.parser import parse_expr, parse_selectfrom posthog.hogql_queries.utils.breakdowns import NOT_IN_COHORT_ID+from posthog.udf_versioner import UDF_VERSIONfrom posthog.utils import DATERANGE_MAPfrom products.product_analytics.backend.hogql_queries.funnels.base import (@@ -356,18 +358,8 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):conditions.append(parse_expr(f"bitTest(steps_bitfield, {funnelStep - 1})"))else:# For dropoff at step N, check that user completed the prior REQUIRED step but not step N- # With optional steps, we need to find the last required step before the dropoff steptarget_step_index = abs(funnelStep) - 1 # 0-indexed step we're checking dropoff at-- # Find the last required step before target_step_index- prior_required_step_index = None- for i in range(target_step_index - 1, -1, -1):- if not getattr(self.context.query.series[i], "optionalInFunnel", False):- prior_required_step_index = i- break-- if prior_required_step_index is None:- raise ValueError("Missing prior required step in actors query")+ prior_required_step_index = self._prior_required_step_index(target_step_index)# User completed the prior required step but not the target stepconditions.append(parse_expr(f"bitTest(steps_bitfield, {prior_required_step_index})"))@@ -392,16 +384,76 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):and self.context.actorsQuery is not Noneand self.context.actorsQuery.includeRecordings):+ uses_step_event_slots = self._uses_step_event_slots()if self.context.includeFinalMatchingEvents:+ required_step_indexes = self._required_step_indexes()+ # step_reached counts required steps only. UDF v13+ keeps matched events in their own step slot.+ if uses_step_event_slots and len(required_step_indexes) < len(self.context.query.series):+ required_step_slots = ",".join(str(i + 1) for i in required_step_indexes)+ return [+ parse_expr(+ f"matched_events_array[[{required_step_slots}][step_reached + 1]] as matching_events"+ )+ ]# Always returns the user's final step of the funnel, 1 indexedreturn [parse_expr("matched_events_array[step_reached + 1] as matching_events")]absolute_actors_step = self._absolute_actors_stepif absolute_actors_step is None:raise ValueError("Missing funnelStep actors query property")+ funnelStep = self.context.actorsQuery.funnelStep+ if uses_step_event_slots and funnelStep is not None and funnelStep < 0:+ # The drop-off condition matches on the prior required step. Add the optional steps after it that the+ # actor completed, because those events are the closest to the drop-off.+ target_step_index = abs(funnelStep) - 1+ prior_required_step_index = self._prior_required_step_index(target_step_index)+ step_events: list[ast.Expr] = [parse_expr(f"matched_events_array[{prior_required_step_index + 1}]")]+ step_events.extend(+ parse_expr(f"if(bitTest(steps_bitfield, {i}), matched_events_array[{i + 1}], [])")+ for i in self._optional_step_indexes_between(prior_required_step_index, target_step_index)+ )+ if len(step_events) > 1:+ return [+ ast.Alias(+ alias="matching_events",+ expr=ast.Call(name="arrayDistinct", args=[ast.Call(name="arrayConcat", args=step_events)]),+ )+ ]+ absolute_actors_step = prior_required_step_indexreturn [parse_expr(f"matched_events_array[{absolute_actors_step + 1}] as matching_events")]return []+ def _uses_step_event_slots(self) -> bool:+ return UDFS["aggregate_funnel"].clickhouse_name == "aggregate_funnel" or UDF_VERSION >= 13++ def _required_step_indexes(self) -> list[int]:+ return [+ i for i, series in enumerate(self.context.query.series) if not getattr(series, "optionalInFunnel", False)+ ]++ def _prior_required_step_index(self, target_step_index: int) -> int:+ prior_required_step_indexes = [i for i in self._required_step_indexes() if i < target_step_index]+ if not prior_required_step_indexes:+ raise ValueError("Missing prior required step in actors query")+ return prior_required_step_indexes[-1]++ def _optional_step_indexes_between(self, prior_required_step_index: int, target_step_index: int) -> list[int]:+ # Iterate the series rather than a range from the request, so an out-of-range funnelStep stays cheap+ return [i for i in range(len(self.context.query.series)) if prior_required_step_index < i < target_step_index]++ def _last_completed_step_slot(self, target_step_index: int) -> str:+ """The 1-indexed matched_events_array slot of the last step the actor completed before the target step."""+ prior_required_step_index = self._prior_required_step_index(target_step_index)+ # The slot of a skipped optional step is empty or holds events from another attempt, so check the path bitfield+ # and fall back through the optional steps to the prior required step+ branches = [+ f"bitTest(steps_bitfield, {i}), {i + 1}"+ for i in reversed(self._optional_step_indexes_between(prior_required_step_index, target_step_index))+ ]+ if not branches:+ return str(prior_required_step_index + 1)+ return f"multiIf({', '.join(branches)}, {prior_required_step_index + 1})"+def _get_timestamp_outer_select(self) -> list[ast.Expr]:"""Returns timestamp selectors for the target step and optionally the preceding step.@@ -409,11 +461,19 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):"""target_step = self._absolute_actors_step- if target_step is None:+ if target_step is None or self.context.actorsQuery is None or self.context.actorsQuery.funnelStep is None:return []# We pull timestamps from matched_events_array and SQL arrays are 1-indexedtarget_step += 1+ target_slot = str(target_step)+ preceding_slot = str(target_step - 1)+ funnelStep = self.context.actorsQuery.funnelStep+ if self._uses_step_event_slots():+ if funnelStep < 0:+ target_slot = self._last_completed_step_slot(abs(funnelStep) - 1)+ elif self.context.includePrecedingTimestamp and funnelStep > 1:+ preceding_slot = self._last_completed_step_slot(funnelStep - 1)final_step = self.context.max_stepsfirst_step = 1@@ -423,12 +483,12 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):raise ValueError("Cannot request preceding step timestamp if target funnel step is the first step")return [- parse_expr(f"matched_events_array[{target_step}][1].1 AS max_timestamp"),- parse_expr(f"matched_events_array[{target_step - 1}][1].1 AS min_timestamp"),+ parse_expr(f"matched_events_array[{target_slot}][1].1 AS max_timestamp"),+ parse_expr(f"matched_events_array[{preceding_slot}][1].1 AS min_timestamp"),]elif self.context.includeTimestamp:return [- parse_expr(f"matched_events_array[{target_step}][1].1 AS timestamp"),+ parse_expr(f"matched_events_array[{target_slot}][1].1 AS timestamp"),parse_expr(f"nullIf(matched_events_array[{final_step}][1].1, 0) AS final_timestamp"),parse_expr(f"matched_events_array[{first_step}][1].1 as first_timestamp"),]diff --git a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambrindex badfc4d1ab1a..26457583f10b 100644--- a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr+++ b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr@@ -260,3 +260,91 @@use_hive_partitioning=0'''# ---+# name: TestFunnelPersons.test_funnel_person_recordings.6+ '''+ SELECT source.id,+ source.id AS id,+ source.matching_events AS matching_events+ FROM+ (SELECT aggregation_target AS actor_id,+ arrayDistinct(arrayConcat(matched_events_array[2], if(bitTest(steps_bitfield, 2), matched_events_array[3], []))) AS matching_events,+ actor_id AS id+ FROM+ (SELECT arraySort(t -> t.1, groupArray(tuple(accurateCastOrNull(timestamp, 'Float64'), uuid, '', arrayFilter(x -> ifNull(notEquals(x, 0), 1), [multiply(1, step_0), multiply(2, step_1), multiply(3, step_2), multiply(4, step_3)])))) AS events_array,+ [''] AS prop,+ arrayJoin(aggregate_funnel(4, 604800, 'first_touch', 'ordered', prop, [3], arrayFilter((x, x_before, x_after) -> not(and(lessOrEquals(length(x.4), 1), equals(x.4, x_before.4), equals(x.4, x_after.4), equals(x.3, x_before.3), equals(x.3, x_after.3), ifNull(greater(x.1, x_before.1), 0), ifNull(less(x.1, x_after.1), 0))), events_array, arrayRotateRight(events_array, 1), arrayRotateLeft(events_array, 1)))) AS af_tuple,+ af_tuple.1 AS step_reached,+ plus(af_tuple.1, 1) AS steps,+ af_tuple.2 AS breakdown,+ af_tuple.3 AS timings,+ af_tuple.4 AS matched_event_uuids_array_array,+ groupArray(tuple(timestamp, uuid, `$session_id`, `$window_id`)) AS user_events,+ mapFromArrays(arrayMap(x -> x.2, user_events), user_events) AS user_events_map,+ arrayMap(matched_event_uuids_array -> arrayMap(event_uuid -> user_events_map[event_uuid], arrayDistinct(matched_event_uuids_array)), matched_event_uuids_array_array) AS matched_events_array,+ af_tuple.5 AS steps_bitfield,+ aggregation_target AS aggregation_target+ FROM+ (SELECT toTimeZone(e.timestamp, 'UTC') AS timestamp,+ if(not(empty(e__override.distinct_id)), e__override.person_id, e.person_id) AS aggregation_target,+ e.uuid AS uuid,+ e.`$session_id` AS `$session_id`,+ e.`$window_id` AS `$window_id`,+ if(equals(e.event, 'step one'), 1, 0) AS step_0,+ if(equals(e.event, 'step two'), 1, 0) AS step_1,+ if(equals(e.event, 'optional step'), 1, 0) AS step_2,+ if(equals(e.event, 'step three'), 1, 0) AS step_3+ FROM events AS e+ LEFT OUTER JOIN+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,+ person_distinct_id_overrides.distinct_id AS distinct_id+ FROM person_distinct_id_overrides+ WHERE equals(person_distinct_id_overrides.team_id, 99999)+ GROUP BY person_distinct_id_overrides.distinct_id+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS e__override ON equals(e.distinct_id, e__override.distinct_id)+ WHERE and(equals(e.team_id, 99999), and(and(greaterOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), lessOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC'))), in(e.event, tuple('optional step', 'step one', 'step three', 'step two'))), or(equals(step_0, 1), equals(step_1, 1), equals(step_2, 1), equals(step_3, 1))))+ GROUP BY aggregation_target+ HAVING ifNull(greaterOrEquals(step_reached, 0), 0))+ WHERE and(bitTest(steps_bitfield, 1), not(bitTest(steps_bitfield, 3)))+ ORDER BY aggregation_target ASC SETTINGS join_algorithm='auto') AS source+ ORDER BY source.id ASC+ LIMIT 101+ OFFSET 0 SETTINGS optimize_aggregation_in_order=1,+ join_algorithm='auto',+ readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---+# name: TestFunnelPersons.test_funnel_person_recordings.7+ '''+ SELECT session_replay_events.session_id AS session_id,+ min(toTimeZone(session_replay_events.min_first_timestamp, 'UTC')) AS start_time,+ max(session_replay_events.retention_period_days) AS retention_period_days,+ plus(dateTrunc('DAY', start_time), toIntervalDay(coalesce(retention_period_days, 30))) AS expiry_time+ FROM session_replay_events+ WHERE and(equals(session_replay_events.team_id, 99999), in(session_replay_events.session_id, ['s2']))+ GROUP BY session_replay_events.session_id+ HAVING and(greaterOrEquals(expiry_time, toDateTime64('2021-01-02 00:00:00.000000', 6, 'UTC')), equals(max(session_replay_events.is_deleted), 0))+ LIMIT 100 SETTINGS readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0+ '''+# ---diff --git a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambrindex 42ca06a63631..113d35a22cde 100644--- a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr+++ b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr@@ -129,6 +129,72 @@output_format_json_escape_forward_slashes=0'''# ---+# name: TestFunnelPersons.test_funnel_person_recordings[new_events_schema.3]+ '''+ SELECT source.id,+ source.id AS id,+ source.matching_events AS matching_events+ FROM+ (SELECT aggregation_target AS actor_id,+ arrayDistinct(arrayConcat(matched_events_array[2], if(bitTest(steps_bitfield, 2), matched_events_array[3], []))) AS matching_events,+ actor_id AS id+ FROM+ (SELECT arraySort(t -> t.1, groupArray(tuple(accurateCastOrNull(timestamp, 'Float64'), uuid, '', arrayFilter(x -> ifNull(notEquals(x, 0), 1), [multiply(1, step_0), multiply(2, step_1), multiply(3, step_2), multiply(4, step_3)])))) AS events_array,+ [''] AS prop,+ arrayJoin(aggregate_funnel(4, 604800, 'first_touch', 'ordered', prop, [3], arrayFilter((x, x_before, x_after) -> not(and(lessOrEquals(length(x.4), 1), equals(x.4, x_before.4), equals(x.4, x_after.4), equals(x.3, x_before.3), equals(x.3, x_after.3), ifNull(greater(x.1, x_before.1), 0), ifNull(less(x.1, x_after.1), 0))), events_array, arrayRotateRight(events_array, 1), arrayRotateLeft(events_array, 1)))) AS af_tuple,+ af_tuple.1 AS step_reached,+ plus(af_tuple.1, 1) AS steps,+ af_tuple.2 AS breakdown,+ af_tuple.3 AS timings,+ af_tuple.4 AS matched_event_uuids_array_array,+ groupArray(tuple(timestamp, uuid, `$session_id`, `$window_id`)) AS user_events,+ mapFromArrays(arrayMap(x -> x.2, user_events), user_events) AS user_events_map,+ arrayMap(matched_event_uuids_array -> arrayMap(event_uuid -> user_events_map[event_uuid], arrayDistinct(matched_event_uuids_array)), matched_event_uuids_array_array) AS matched_events_array,+ af_tuple.5 AS steps_bitfield,+ aggregation_target AS aggregation_target+ FROM+ (SELECT toTimeZone(e.timestamp, 'UTC') AS timestamp,+ if(not(empty(e__override.distinct_id)), e__override.person_id, e.person_id) AS aggregation_target,+ e.uuid AS uuid,+ e.properties.`$session_id` AS `$session_id`,+ e.properties.`$window_id` AS `$window_id`,+ if(equals(e.event, 'step one'), 1, 0) AS step_0,+ if(equals(e.event, 'step two'), 1, 0) AS step_1,+ if(equals(e.event, 'optional step'), 1, 0) AS step_2,+ if(equals(e.event, 'step three'), 1, 0) AS step_3+ FROM events_json AS e+ LEFT OUTER JOIN+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,+ person_distinct_id_overrides.distinct_id AS distinct_id+ FROM person_distinct_id_overrides+ WHERE equals(person_distinct_id_overrides.team_id, 99999)+ GROUP BY person_distinct_id_overrides.distinct_id+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS e__override ON equals(e.distinct_id, e__override.distinct_id)+ WHERE and(equals(e.team_id, 99999), and(and(greaterOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), lessOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC'))), in(e.event, tuple('optional step', 'step one', 'step three', 'step two'))), or(equals(step_0, 1), equals(step_1, 1), equals(step_2, 1), equals(step_3, 1))))+ GROUP BY aggregation_target+ HAVING ifNull(greaterOrEquals(step_reached, 0), 0))+ WHERE and(bitTest(steps_bitfield, 1), not(bitTest(steps_bitfield, 3)))+ ORDER BY aggregation_target ASC SETTINGS join_algorithm='auto') AS source+ ORDER BY source.id ASC+ LIMIT 101+ OFFSET 0 SETTINGS optimize_aggregation_in_order=1,+ join_algorithm='auto',+ readonly=2,+ max_execution_time=60,+ allow_experimental_object_type=1,+ max_ast_elements=4000000,+ max_expanded_ast_elements=4000000,+ max_bytes_before_external_group_by=0,+ transform_null_in=1,+ optimize_min_equality_disjunction_chain_length=4294967295,+ optimize_rewrite_aggregate_function_with_if=0,+ optimize_min_inequality_conjunction_chain_length=4294967295,+ allow_experimental_join_condition=1,+ use_hive_partitioning=0,+ json_type_escape_dots_in_keys=1,+ output_format_json_escape_forward_slashes=0+ '''+# ---# name: TestFunnelPersons.test_funnel_person_recordings[new_events_schema]'''SELECT source.id,diff --git a/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py b/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.pyindex 9bd0145149e3..9c61cae65f79 100644--- a/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py+++ b/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py@@ -1,4 +1,4 @@-from datetime import datetime, timedelta+from datetime import UTC, datetime, timedeltafrom typing import Optionalfrom uuid import UUID@@ -12,6 +12,7 @@ from posthog.test.base import (flush_persons_and_events,snapshot_clickhouse_queries,)+from unittest.mock import patchfrom django.test import override_settingsfrom django.utils import timezone@@ -33,6 +34,10 @@ from posthog.schema import (FunnelVizType,)+from posthog.hogql.functions.udfs import UDFS+from posthog.hogql.query import execute_hogql_query++from posthog.clickhouse.workload import Workloadfrom posthog.hogql_queries.actors_query_runner import ActorsQueryRunnerfrom posthog.models.event.util import bulk_create_eventsfrom posthog.models.team.team import Team@@ -435,6 +440,40 @@ class TestFunnelPersons(ClickhouseTestMixin, APIBaseTest):],)+ # Dropoff after a skipped optional step, with recording from the prior required step+ query_with_optional_step = FunnelsQuery(+ series=[+ EventsNode(event="step one"),+ EventsNode(event="step two"),+ EventsNode(event="optional step", optionalInFunnel=True),+ EventsNode(event="step three"),+ ],+ interval="day",+ dateRange=DateRange(date_from="2021-01-01", date_to="2021-01-08"),+ funnelsFilter=FunnelsFilter(funnelWindowIntervalUnit="day", funnelWindowInterval=7),+ )+ with (+ patch("products.product_analytics.backend.hogql_queries.funnels.funnel.UDF_VERSION", 13),+ patch.object(UDFS["aggregate_funnel"], "clickhouse_name", "aggregate_funnel_v13"),+ ):+ results = get_actors(query_with_optional_step, self.team, funnel_step=-4, include_recordings=True)+ self.assertEqual(results[0][0], p1.uuid)+ self.assertEqual(+ list(results[0][2]),+ [+ {+ "session_id": "s2",+ "events": [+ {+ "uuid": UUID("21111111-1111-1111-1111-111111111111"),+ "timestamp": timezone.now() + timedelta(days=1),+ "window_id": "w2",+ }+ ],+ }+ ],+ )+def test_parses_step_breakdown_correctly(self):person1 = _create_person(distinct_ids=["person1"],@@ -602,6 +641,106 @@ class TestFunnelPersons(ClickhouseTestMixin, APIBaseTest):self.assertCountEqual(set(results[0][1]["distinct_ids"]), {"person1", "anon1"})self.assertCountEqual(set(results[1][1]["distinct_ids"]), {"person2", "anon2"})+ @parameterized.expand([(12, False), (12, True), (13, False), (13, True)])+ @time_machine.travel("2021-05-02", tick=False)+ def test_optional_funnel_step_recordings(self, udf_version: int, visit_optional_steps: bool) -> None:+ with (+ patch("products.product_analytics.backend.hogql_queries.funnels.funnel.UDF_VERSION", udf_version),+ patch.object(UDFS["aggregate_funnel"], "clickhouse_name", f"aggregate_funnel_v{udf_version}"),+ ):+ optional_steps = {2, 4}+ visited_steps = [1, 3]+ if visit_optional_steps:+ visited_steps = [1, 2, 3, 4]+ journeys_for(+ {+ "replay-user": [+ {+ "event": f"step {step}",+ "timestamp": datetime(2021, 5, 1, 0, step),+ "event_uuid": str(UUID(int=step)),+ "properties": {"$session_id": str(UUID(f"01800000-0000-7000-8000-{step:012d}"))},+ }+ for step in visited_steps+ ]+ },+ self.team,+ )+ for step in visited_steps:+ produce_replay_summary(+ team_id=self.team.pk,+ session_id=str(UUID(f"01800000-0000-7000-8000-{step:012d}")),+ distinct_id="replay-user",+ first_timestamp=datetime(2021, 5, 1, 0, step, tzinfo=UTC),+ )+ query = FunnelsQuery(+ series=[+ EventsNode(event=f"step {step}", optionalInFunnel=step in optional_steps) for step in range(1, 6)+ ],+ dateRange=DateRange(date_from="2021-05-01", date_to="2021-05-02"),+ )+ for funnel_step in [3, -5]:+ with self.subTest(funnel_step=funnel_step):+ results = get_actors(query, self.team, funnel_step=funnel_step, include_recordings=True)+ self.assertEqual(len(results), 1)+ expected_steps = [3]+ if udf_version == 12:+ if not visit_optional_steps:+ expected_steps = []+ elif funnel_step == -5:+ expected_steps = [4]+ elif visit_optional_steps and funnel_step == -5:+ expected_steps = [3, 4]+ self.assertCountEqual(+ [+ (recording["session_id"], [event["uuid"] for event in recording["events"]])+ for recording in results[0][2]+ ],+ [+ (str(UUID(f"01800000-0000-7000-8000-{step:012d}")), [UUID(int=step)])+ for step in expected_steps+ ],+ )++ if udf_version != 12:+ runner = FunnelsQueryRunner(query=query, team=self.team)+ runner.context.actorsQuery = FunnelsActorsQuery(source=query, funnelStep=-5, includeRecordings=True)+ runner.context.includeFinalMatchingEvents = True+ response = execute_hogql_query(+ runner.to_actors_query(),+ team=self.team,+ query_type="test_funnel_final_recordings",+ workload=Workload.ONLINE,+ )+ self.assertEqual(len(response.results), 1)+ self.assertEqual(response.results[0][1][0][1], UUID(int=3))++ last_step_before_drop_off = 4 if visit_optional_steps else 3+ self.assertEqual(+ self._funnel_actor_timestamp(query, funnel_step=-5, column="timestamp"),+ datetime(2021, 5, 1, 0, last_step_before_drop_off, tzinfo=UTC),+ )+ last_step_before_step_3 = 2 if visit_optional_steps else 1+ self.assertEqual(+ self._funnel_actor_timestamp(query, funnel_step=3, column="min_timestamp"),+ datetime(2021, 5, 1, 0, last_step_before_step_3, tzinfo=UTC),+ )++ def _funnel_actor_timestamp(self, query: FunnelsQuery, funnel_step: int, column: str) -> datetime:+ runner = FunnelsQueryRunner(query=query, team=self.team)+ runner.context.actorsQuery = FunnelsActorsQuery(source=query, funnelStep=funnel_step)+ runner.context.includeTimestamp = column == "timestamp"+ runner.context.includePrecedingTimestamp = column == "min_timestamp"+ response = execute_hogql_query(+ runner.to_actors_query(),+ team=self.team,+ query_type="test_funnel_paths_timestamps",+ workload=Workload.ONLINE,+ )+ assert response.columns is not None+ self.assertEqual(len(response.results), 1)+ return response.results[0][response.columns.index(column)]+def test_optional_funnel_step_actors_query(self):journeys_for({