Skip to content

:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.

fix(funnels): align drop-off recordings with optional steps (#117315)

posthog[bot]f155b3e1020c
diff --git a/funnel-udf/Cross.toml b/funnel-udf/Cross.toml
new file mode 100644
index 000000000000..66635e3a5196
--- /dev/null
+++ b/funnel-udf/Cross.toml
@@ -0,0 +1,7 @@
+# The default images follow cross's main branch, so a rebuild can silently raise the glibc version that
+# ClickHouse hosts need to run the binaries. These images link against glibc 2.23.
+[target.x86_64-unknown-linux-gnu]
+image = "ghcr.io/cross-rs/x86_64-unknown-linux-gnu:0.2.5"
+
+[target.aarch64-unknown-linux-gnu]
+image = "ghcr.io/cross-rs/aarch64-unknown-linux-gnu:0.2.5"
diff --git a/funnel-udf/README.md b/funnel-udf/README.md
index edbe18f2d5cb..e1f78a544ee6 100644
--- a/funnel-udf/README.md
+++ b/funnel-udf/README.md
@@ -10,21 +10,27 @@
## Deployment workflow
-The `user_defined_function.xml` deployed to ClickHouse is now pulled automatically from the `master` branch of this repo by ansible (`posthog-cloud-infra/ansible/roles/clickhouse/tasks/setup_udfs.yml`).
-It downloads `posthog/user_scripts/latest_user_defined_function.xml` and deploys it as ClickHouse's `user_defined_function.xml`.
+Cloud ClickHouse does not read the `master` branch of this repo.
+Ansible (`posthog-cloud-infra/ansible/roles/clickhouse/tasks/setup_udfs.yml`) downloads this repo at the commit pinned in `posthog_udfs_revision` (`ansible/roles/clickhouse/defaults/main.yml`).
+It copies that commit's `posthog/user_scripts` to the ClickHouse nodes, installs its `latest_user_defined_function.xml` as ClickHouse's `user_defined_function.xml`, and reloads the functions.
+A merge here changes nothing in cloud ClickHouse until the pin moves.
For revertible cloud deploys:
1. Develop using the binary files at the top level of `user_scripts` (see section above), with schema defined in `docker/clickhouse/user_defined_function.xml`
2. When ready to deploy, increment the version in `posthog/udf_versioner.py` and run it. This generates versioned binaries and `latest_user_defined_function.xml`
-3. Land a PR with the updated `user_scripts` folder — include both the new version folder (e.g. `v12/`) and the regenerated `latest_user_defined_function.xml`. **Do not** include the `UDF_VERSION` bump in this PR — that goes in a separate PR (step 5)
-4. The binaries and XML should be automatically deployed to ClickHouse on the next config run. Verify in metabase by running `SELECT aggregate_funnel_vXX()`. An "invalid arguments" error means the function is registered. An "unknown function" error means the deploy hasn't taken effect yet. **Make sure to do this for both EU and US**
-5. Once the deploy is confirmed, land a separate PR that bumps `UDF_VERSION` in `posthog/udf_versioner.py` to the new version. This switches the posthog query builder to use the new UDF
+3. Land a PR with the updated `user_scripts` folder — include both the new version folder (e.g. `v12/`) and the regenerated `latest_user_defined_function.xml`. **Do not** include the `UDF_VERSION` bump in this PR — that goes in a separate PR (step 6). Check that git keeps the wrappers, binaries and Python scripts in the version folder executable (mode `100755`), because local, CI and self-hosted ClickHouse run them directly
+4. In `posthog-cloud-infra`, bump `posthog_udfs_revision` to a `master` commit that includes the PR from step 3. Merging that change deploys the pinned `user_scripts` to every ClickHouse environment at once. The bump also ships every other `user_scripts` change since the previous pin, and the unversioned utility UDFs take effect immediately, so list those changes in the infra PR
+5. Verify in metabase by running `SELECT aggregate_funnel_vXX()`. An "invalid arguments" error means the function is registered. An "unknown function" error means the deploy hasn't taken effect yet. Also run a valid funnel query against the new version. Registration alone does not check the launcher or binary. **Make sure to do this for both EU and US**
+6. Once the deploy is confirmed, land a separate PR that bumps `UDF_VERSION` in `posthog/udf_versioner.py` to the new version. This switches the posthog query builder to use the new UDF
## Design decisions
+- Each replay slot holds the events of its own step, optional steps included. Optional matches never change funnel counts, because the duplicate-event check only sees required steps.
+- Unversioned binaries use the current event layout. Replay queries must use that layout even while cloud queries select an older UDF version.
- We tried writing the UDFs in Python, but this cause ClickHouse nodes to crash as the garbage collector could not keep up. As such we're now writing the functions in Rust.
- We're using `cross-rs` to compile the binaries - it's a cross platfrom rust compiler.
+- `Cross.toml` pins the build images, which link against glibc 2.23. Check the highest version a new binary needs with `objdump -T <binary> | grep -o 'GLIBC_[0-9.]*' | sort -uV | tail -1`.
## Troubleshooting
diff --git a/funnel-udf/src/steps.rs b/funnel-udf/src/steps.rs
index 00bed1cbc99c..a165613cb33e 100644
--- a/funnel-udf/src/steps.rs
+++ b/funnel-udf/src/steps.rs
@@ -14,6 +14,11 @@ pub struct EnteredTimestamp {
pub timings: Vec<f64>,
pub uuids: Vec<Uuid>,
pub steps: u32,
+ // The event of every step on the path, optional steps included, in step order. Only the replay slots read it.
+ // `uuids` keeps the required steps only, because the duplicate-event check reads it and an optional step must
+ // not change which paths count.
+ #[serde(default)]
+ pub step_uuids: Vec<Uuid>,
}
#[derive(Debug, Clone, Deserialize)]
@@ -65,6 +70,7 @@ pub const DEFAULT_ENTERED_TIMESTAMP: EnteredTimestamp = EnteredTimestamp {
timings: vec![],
uuids: vec![],
steps: 0,
+ step_uuids: vec![],
};
pub fn run(args: &Args) -> Vec<Result> {
@@ -143,6 +149,7 @@ impl AggregateFunnelRow {
timings: vec![],
uuids: vec![],
steps: 0,
+ step_uuids: vec![],
};
if events_with_same_timestamp.len() == 1 {
@@ -209,11 +216,11 @@ impl AggregateFunnelRow {
.filter(|i| **i <= final_index as i8)
.count();
- for i in 0..(final_index - optional_count) {
- //if event_uuids[i].len() >= MAX_REPLAY_EVENTS && !event_uuids[i].contains(&final_value.uuids[i]) {
+ let matched_steps = (0..final_index).filter(|i| final_value.steps & (1 << i) != 0);
+ for (step_index, uuid) in matched_steps.zip(&final_value.step_uuids) {
// Always put the actual event uuids first, we use it to extract timestamps
// This might create duplicates, but that's fine (we can remove it in clickhouse)
- vars.event_uuids[i].insert(0, final_value.uuids[i].clone());
+ vars.event_uuids[step_index].insert(0, *uuid);
}
self.results.push(Result(
@@ -301,18 +308,21 @@ impl AggregateFunnelRow {
t.push(event.timestamp);
u.push(event.uuid);
}
+ let mut s = previous_step.step_uuids.clone();
+ s.push(event.uuid);
let new_entered_timestamp = EnteredTimestamp {
timestamp: previous_step.timestamp,
excluded: previous_step.excluded,
timings: t,
uuids: u,
steps: previous_step.steps | (1 << (step - 1)),
+ step_uuids: s,
};
if !previous_step.excluded {
vars.entered_timestamp[step] = new_entered_timestamp.clone();
- if vars.event_uuids[previous_step_index].len() < MAX_REPLAY_EVENTS - 1 {
- vars.event_uuids[previous_step_index].push(event.uuid);
+ if vars.event_uuids[step - 1].len() < MAX_REPLAY_EVENTS - 1 {
+ vars.event_uuids[step - 1].push(event.uuid);
}
}
@@ -337,3 +347,138 @@ impl AggregateFunnelRow {
}
}
}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+ use rstest::rstest;
+
+ #[rstest]
+ #[case(vec![1, 4])]
+ #[case(vec![1, 2, 4])]
+ #[case(vec![1, 2, 3, 4])]
+ fn matched_events_keep_their_funnel_step(#[case] visited_steps: Vec<i8>) {
+ let args = Args {
+ num_steps: 5,
+ conversion_window_limit: 3600,
+ breakdown_attribution_type: "first_touch".to_string(),
+ funnel_order_type: "ordered".to_string(),
+ prop_vals: vec![PropVal::Int(0)],
+ optional_steps: vec![2, 3],
+ value: visited_steps
+ .iter()
+ .map(|&step| Event {
+ timestamp: f64::from(step),
+ uuid: Uuid::from_u128(step as u128),
+ breakdown: PropVal::Int(0),
+ steps: vec![step],
+ })
+ .collect(),
+ };
+
+ let results = run(&args);
+ assert_eq!(results.len(), 1);
+ let result = &results[0];
+ assert_eq!(result.0, 1);
+ assert_eq!(result.2, vec![3.0]);
+ for step in 1..=args.num_steps {
+ if visited_steps.contains(&(step as i8)) {
+ assert_eq!(result.3[step - 1][0], Uuid::from_u128(step as u128));
+ assert!(result.3[step - 1]
+ .iter()
+ .all(|uuid| *uuid == Uuid::from_u128(step as u128)));
+ } else {
+ assert!(result.3[step - 1].is_empty());
+ }
+ }
+ }
+
+ #[test]
+ fn optional_step_starts_with_the_event_from_the_selected_path() {
+ let args = Args {
+ num_steps: 3,
+ conversion_window_limit: 3600,
+ breakdown_attribution_type: "first_touch".to_string(),
+ funnel_order_type: "ordered".to_string(),
+ prop_vals: vec![PropVal::Int(0)],
+ optional_steps: vec![2],
+ value: [1, 2, 1, 2, 3]
+ .iter()
+ .enumerate()
+ .map(|(index, &step)| Event {
+ timestamp: (index + 1) as f64,
+ uuid: Uuid::from_u128((index + 1) as u128),
+ breakdown: PropVal::Int(0),
+ steps: vec![step],
+ })
+ .collect(),
+ };
+
+ let results = run(&args);
+ assert_eq!(results[0].2, vec![2.0]);
+ for (step, uuid) in [3, 4, 5].iter().enumerate() {
+ assert_eq!(results[0].3[step][0], Uuid::from_u128(*uuid));
+ }
+ }
+
+ // Each event is (timestamp, uuid, steps). The expected counts are the ones the previous UDF version returns.
+ #[rstest]
+ #[case::optional_and_next_required(
+ 3,
+ vec![2],
+ vec![(1.0, 1, vec![1]), (2.0, 2, vec![2, 3]), (2.0, 3, vec![1])],
+ 1,
+ 0b111,
+ vec![3, 2, 2]
+ )]
+ #[case::two_optionals_and_next_required(
+ 4,
+ vec![2, 3],
+ vec![(1.0, 1, vec![1]), (2.0, 2, vec![2, 3, 4]), (2.0, 3, vec![1])],
+ 1,
+ 0b1111,
+ vec![3, 2, 2, 2]
+ )]
+ #[case::optional_and_later_required(
+ 4,
+ vec![2],
+ vec![(1.0, 1, vec![1]), (2.0, 10, vec![2, 4]), (2.0, 11, vec![3])],
+ 2,
+ 0b1111,
+ vec![1, 10, 11, 10]
+ )]
+ fn optional_step_matches_do_not_change_counts(
+ #[case] num_steps: usize,
+ #[case] optional_steps: Vec<i8>,
+ #[case] events: Vec<(f64, u128, Vec<i8>)>,
+ #[case] step_reached: i8,
+ #[case] steps_bitfield: u32,
+ #[case] first_event_per_step: Vec<u128>,
+ #[values("ordered", "strict")] funnel_order_type: &str,
+ ) {
+ let args = Args {
+ num_steps,
+ conversion_window_limit: 3600,
+ breakdown_attribution_type: "first_touch".to_string(),
+ funnel_order_type: funnel_order_type.to_string(),
+ prop_vals: vec![PropVal::Int(0)],
+ optional_steps,
+ value: events
+ .into_iter()
+ .map(|(timestamp, uuid, steps)| Event {
+ timestamp,
+ uuid: Uuid::from_u128(uuid),
+ breakdown: PropVal::Int(0),
+ steps,
+ })
+ .collect(),
+ };
+
+ let results = run(&args);
+ assert_eq!(results[0].0, step_reached);
+ assert_eq!(results[0].4, steps_bitfield);
+ for (step, uuid) in first_event_per_step.iter().enumerate() {
+ assert_eq!(results[0].3[step][0], Uuid::from_u128(*uuid));
+ }
+ }
+}
diff --git a/funnel-udf/src/unordered_steps.rs b/funnel-udf/src/unordered_steps.rs
index cec710ee0822..79c2dfee48a3 100644
--- a/funnel-udf/src/unordered_steps.rs
+++ b/funnel-udf/src/unordered_steps.rs
@@ -197,6 +197,7 @@ impl AggregateFunnelRowUnordered {
timings,
uuids,
steps: 0,
+ step_uuids: vec![],
},
);
}
diff --git a/posthog/user_scripts/aggregate_funnel_aarch64 b/posthog/user_scripts/aggregate_funnel_aarch64
index 1d95a94b7ea5..43dc41faeca3 100755
Binary files a/posthog/user_scripts/aggregate_funnel_aarch64 and b/posthog/user_scripts/aggregate_funnel_aarch64 differ
diff --git a/posthog/user_scripts/aggregate_funnel_x86_64 b/posthog/user_scripts/aggregate_funnel_x86_64
index e758982049d9..82c05e9c94ad 100755
Binary files a/posthog/user_scripts/aggregate_funnel_x86_64 and b/posthog/user_scripts/aggregate_funnel_x86_64 differ
diff --git a/posthog/user_scripts/latest_user_defined_function.xml b/posthog/user_scripts/latest_user_defined_function.xml
index 639e7db7f770..42b7688a7cd9 100644
--- a/posthog/user_scripts/latest_user_defined_function.xml
+++ b/posthog/user_scripts/latest_user_defined_function.xml
@@ -1,5 +1,5 @@
<functions>
- <!-- Version: v12. Generated at: 2026-09-10T10:15:07.735801+00:00
+ <!-- Version: v13. Generated at: 2026-10-27T10:48:41.604645+00:00
This file is autogenerated by udf_versioner.py. Do not edit this, only edit the version at docker/clickhouse/user_defined_function.xml--><function>
<type>executable_pool</type>
<name>aggregate_funnel_v11</name>
@@ -916,6 +916,601 @@ This file is autogenerated by udf_versioner.py. Do not edit this, only edit the
<command>v12/json_drop_keys_udf {keys_parameter:Array(String)}</command>
</function>
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_v13</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_v13</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_v13</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_json_v13</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --json --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_json_v13</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --json --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_json_v13</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --json --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_json_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --json --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_json_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --json --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_json_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --json --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_test_v13</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_test_v13</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel_array_trends_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable</type>
+ <name>JSONDropKeys_v13</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>v13/json_drop_keys_udf {keys_parameter:Array(String)}</command>
+ </function>
+
<function>
<type>executable_pool</type>
<name>JSONCleanPostHogEvent</name>
@@ -996,6 +1591,7 @@ This file is autogenerated by udf_versioner.py. Do not edit this, only edit the
<stderr_reaction>throw</stderr_reaction>
<deterministic>true</deterministic>
</function>
+
<function>
<type>executable_pool</type>
<name>JSONDropKeysPool</name>
diff --git a/posthog/user_scripts/v13/aggregate_funnel b/posthog/user_scripts/v13/aggregate_funnel
new file mode 100755
index 000000000000..b7e180ec43c7
--- /dev/null
+++ b/posthog/user_scripts/v13/aggregate_funnel
@@ -0,0 +1,2 @@
+#!/bin/bash
+. "$(dirname "$0")/run_udf.sh"
diff --git a/posthog/user_scripts/v13/aggregate_funnel_aarch64 b/posthog/user_scripts/v13/aggregate_funnel_aarch64
new file mode 100755
index 000000000000..43dc41faeca3
Binary files /dev/null and b/posthog/user_scripts/v13/aggregate_funnel_aarch64 differ
diff --git a/posthog/user_scripts/v13/aggregate_funnel_array_trends_test.py b/posthog/user_scripts/v13/aggregate_funnel_array_trends_test.py
new file mode 100755
index 000000000000..97da07a2e72d
--- /dev/null
+++ b/posthog/user_scripts/v13/aggregate_funnel_array_trends_test.py
@@ -0,0 +1,16 @@
+#!/usr/bin/python3
+
+import os
+import sys
+import json
+
+if __name__ == "__main__":
+ # ClickHouse starts this script with copies of its open client connections. A retained copy keeps a connection
+ # open after the server closes it, so the client's next request on that connection never gets a response.
+ os.closerange(3, os.sysconf("SC_OPEN_MAX"))
+ for line in sys.stdin:
+ try:
+ print(json.dumps({"result": line})) # noqa: T201
+ except Exception as e:
+ print(json.dumps({"result": json.dumps(str(e))}), end="\n") # noqa: T201
+ sys.stdout.flush()
diff --git a/posthog/user_scripts/v13/aggregate_funnel_test.py b/posthog/user_scripts/v13/aggregate_funnel_test.py
new file mode 100755
index 000000000000..1473a11dfd6f
--- /dev/null
+++ b/posthog/user_scripts/v13/aggregate_funnel_test.py
@@ -0,0 +1,16 @@
+#!/usr/bin/python3
+import os
+import sys
+import json
+import traceback
+
+if __name__ == "__main__":
+ # ClickHouse starts this script with copies of its open client connections. A retained copy keeps a connection
+ # open after the server closes it, so the client's next request on that connection never gets a response.
+ os.closerange(3, os.sysconf("SC_OPEN_MAX"))
+ for line in sys.stdin:
+ try:
+ print(json.dumps({"result": line})) # noqa: T201
+ except Exception as e:
+ print(json.dumps({"result": json.dumps(str(e) + traceback.format_exc())}), end="\n") # noqa: T201
+ sys.stdout.flush()
diff --git a/posthog/user_scripts/v13/aggregate_funnel_x86_64 b/posthog/user_scripts/v13/aggregate_funnel_x86_64
new file mode 100755
index 000000000000..82c05e9c94ad
Binary files /dev/null and b/posthog/user_scripts/v13/aggregate_funnel_x86_64 differ
diff --git a/posthog/user_scripts/v13/json_drop_keys_udf b/posthog/user_scripts/v13/json_drop_keys_udf
new file mode 100755
index 000000000000..b7e180ec43c7
--- /dev/null
+++ b/posthog/user_scripts/v13/json_drop_keys_udf
@@ -0,0 +1,2 @@
+#!/bin/bash
+. "$(dirname "$0")/run_udf.sh"
diff --git a/posthog/user_scripts/v13/json_drop_keys_udf_aarch64 b/posthog/user_scripts/v13/json_drop_keys_udf_aarch64
new file mode 100755
index 000000000000..b1a06be34547
Binary files /dev/null and b/posthog/user_scripts/v13/json_drop_keys_udf_aarch64 differ
diff --git a/posthog/user_scripts/v13/json_drop_keys_udf_x86_64 b/posthog/user_scripts/v13/json_drop_keys_udf_x86_64
new file mode 100755
index 000000000000..91ac16dd6e10
Binary files /dev/null and b/posthog/user_scripts/v13/json_drop_keys_udf_x86_64 differ
diff --git a/posthog/user_scripts/v13/run_udf.sh b/posthog/user_scripts/v13/run_udf.sh
new file mode 100644
index 000000000000..b4c59bc14cda
--- /dev/null
+++ b/posthog/user_scripts/v13/run_udf.sh
@@ -0,0 +1,36 @@
+# shellcheck shell=bash
+# Sourced by every wrapper in this directory. The wrapper's file name is the base name of the binary it starts, so
+# every wrapper is the same two lines. Each version directory carries its own copy of this file, because a deployed
+# version must not depend on files outside it.
+
+# ClickHouse starts a wrapper with copies of its open files, including client connections. A connection stays open
+# while any copy of it is open, so a long-running UDF process keeps a connection open after the server closes it,
+# and the next request on that connection never gets a response. Close every inherited descriptor except stdin,
+# stdout and stderr before anything else. The wrappers run under bash because sh cannot close a descriptor above 9.
+for fd_path in /proc/$$/fd/*; do
+ fd=${fd_path##*/}
+ if (( fd > 2 )); then
+ exec {fd}>&-
+ fi
+done
+
+DIR_NAME=$(dirname "$0")
+UDF_NAME=$(basename "$0")
+
+TEMP_DIR=$(mktemp -d -p /tmp "${UDF_NAME}_XXXXXXXXXX")
+trap 'rm -rf "$TEMP_DIR"; exit' 0
+
+case $(uname -m) in
+aarch64)
+ EXECUTABLE_NAME="${UDF_NAME}_aarch64"
+ ;;
+*)
+ EXECUTABLE_NAME="${UDF_NAME}_x86_64"
+ ;;
+esac
+
+# A deploy replaces the binaries in user_scripts while pooled processes still run them. Each process runs a private
+# copy, so the replacement never touches a file a process is executing.
+cp "$DIR_NAME/$EXECUTABLE_NAME" "$TEMP_DIR/$EXECUTABLE_NAME"
+chmod +x "$TEMP_DIR/$EXECUTABLE_NAME"
+"$TEMP_DIR/$EXECUTABLE_NAME" "$@"
diff --git a/posthog/user_scripts/v13/user_defined_function.xml b/posthog/user_scripts/v13/user_defined_function.xml
new file mode 100644
index 000000000000..42b7688a7cd9
--- /dev/null
+++ b/posthog/user_scripts/v13/user_defined_function.xml
@@ -0,0 +1,1613 @@
+<functions>
+ <!-- Version: v13. Generated at: 2026-10-27T10:48:41.604645+00:00
+This file is autogenerated by udf_versioner.py. Do not edit this, only edit the version at docker/clickhouse/user_defined_function.xml--><function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_v11</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel steps</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_v11</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel steps</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_v11</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel steps</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_test_v11</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_v11</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel trends</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_v11</name>
+
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel trends</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_v11</name>
+
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel trends</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_test_v11</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>v11/aggregate_funnel_array_trends_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+<function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_v12</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel steps --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_v12</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel steps --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_v12</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel steps --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_v12</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel trends --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_v12</name>
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel trends --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_v12</name>
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel trends --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_json_v12</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel steps --json --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_json_v12</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel steps --json --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_json_v12</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel steps --json --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_json_v12</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel trends --json --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_json_v12</name>
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel trends --json --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_json_v12</name>
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel trends --json --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_test_v12</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_test_v12</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v12/aggregate_funnel_array_trends_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable</type>
+ <name>JSONDropKeys_v12</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>v12/json_drop_keys_udf {keys_parameter:Array(String)}</command>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_v13</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_v13</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_v13</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>RowBinaryWithNamesAndTypes</format>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_json_v13</name>
+ <return_type>Array(Tuple(Int8, Nullable(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --json --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_json_v13</name>
+ <return_type>Array(Tuple(Int8, UInt64, Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --json --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_json_v13</name>
+ <return_type>Array(Tuple(Int8, Array(String), Array(Float64), Array(Array(UUID)), UInt32))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel steps --json --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_trends_json_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Nullable(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Nullable(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --json --variant=plain</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_json_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, Array(String), UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --json --variant=array</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_cohort_trends_json_v13</name>
+ <return_type>Array(Tuple(UInt64, Int8, UInt64, UUID))</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(UInt64)</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, UInt64, Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel trends --json --variant=cohort</command>
+ <lifetime>600</lifetime>
+ </function>
+
+
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_test_v13</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Int8)</type>
+ <name>optional_steps</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UUID, Nullable(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>aggregate_funnel_array_trends_test_v13</name>
+ <return_type>String</return_type>
+ <return_name>result</return_name>
+ <argument>
+ <type>UInt8</type>
+ <name>from_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>to_step</name>
+ </argument>
+ <argument>
+ <type>UInt8</type>
+ <name>num_steps</name>
+ </argument>
+ <argument>
+ <type>UInt64</type>
+ <name>conversion_window_limit</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>breakdown_attribution_type</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>funnel_order_type</name>
+ </argument>
+ <argument>
+ <type>Array(Array(String))</type>
+ <name>prop_vals</name>
+ </argument>
+ <argument>
+ <type>Array(Tuple(Nullable(Float64), UInt64, UUID, Array(String), Array(Int8)))</type>
+ <name>value</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <stderr_reaction>throw</stderr_reaction>
+ <command>v13/aggregate_funnel_array_trends_test.py</command>
+ <lifetime>600</lifetime>
+ </function>
+
+ <function>
+ <type>executable</type>
+ <name>JSONDropKeys_v13</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>v13/json_drop_keys_udf {keys_parameter:Array(String)}</command>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>JSONCleanPostHogEvent</name>
+ <return_type>Tuple(properties String, temporary_properties String, person_properties String, properties_null_keys Array(String), temporary_properties_null_keys Array(String), person_properties_null_keys Array(String))</return_type>
+ <argument>
+ <type>String</type>
+ <name>properties</name>
+ </argument>
+ <argument>
+ <type>String</type>
+ <name>person_properties</name>
+ </argument>
+ <format>JSONEachRow</format>
+ <command>json_clean_posthog_event_properties_udf --chunked --event</command>
+ <send_chunk_header>true</send_chunk_header>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>JSONCleanPostHogEventProperties</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>json_clean_posthog_event_properties_udf --chunked</command>
+ <send_chunk_header>true</send_chunk_header>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>JSONCleanPostHogPersonProperties</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>json_clean_posthog_event_properties_udf --chunked --person-properties</command>
+ <send_chunk_header>true</send_chunk_header>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>JSONCleanPostHogTemporaryProperties</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>json_clean_posthog_event_properties_udf --chunked --temporary-properties</command>
+ <send_chunk_header>true</send_chunk_header>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>JSONStripEmptyStringsAndNulls</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>Raw</format>
+ <command>json_strip_empty_strings_and_nulls_udf --chunked</command>
+ <send_chunk_header>true</send_chunk_header>
+ </function>
+ <function>
+ <type>executable_pool</type>
+ <name>decompress</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ </argument>
+ <argument>
+ <type>String</type>
+ </argument>
+ <format>RowBinary</format>
+ <command>decompress_udf</command>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <deterministic>true</deterministic>
+ </function>
+
+ <function>
+ <type>executable_pool</type>
+ <name>JSONDropKeysPool</name>
+ <return_type>String</return_type>
+ <argument>
+ <type>String</type>
+ <name>json</name>
+ </argument>
+ <argument>
+ <type>Array(String)</type>
+ <name>keys</name>
+ </argument>
+ <format>RowBinary</format>
+ <command>json_drop_keys_udf --row-binary</command>
+ <send_chunk_header>true</send_chunk_header>
+ <stderr_reaction>throw</stderr_reaction>
+ <deterministic>true</deterministic>
+ </function>
+</functions>
\ No newline at end of file
diff --git a/products/product_analytics/backend/hogql_queries/funnels/funnel.py b/products/product_analytics/backend/hogql_queries/funnels/funnel.py
index 3ef6b4c1281a..9798d4bd2592 100644
--- a/products/product_analytics/backend/hogql_queries/funnels/funnel.py
+++ b/products/product_analytics/backend/hogql_queries/funnels/funnel.py
@@ -6,9 +6,11 @@ from posthog.schema import BreakdownAttributionType, BreakdownType, StepOrderVal
from posthog.hogql import ast
from posthog.hogql.constants import DEFAULT_RETURNED_ROWS, HogQLQuerySettings
+from posthog.hogql.functions.udfs import UDFS
from posthog.hogql.parser import parse_expr, parse_select
from posthog.hogql_queries.utils.breakdowns import NOT_IN_COHORT_ID
+from posthog.udf_versioner import UDF_VERSION
from posthog.utils import DATERANGE_MAP
from products.product_analytics.backend.hogql_queries.funnels.base import (
@@ -356,18 +358,8 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):
conditions.append(parse_expr(f"bitTest(steps_bitfield, {funnelStep - 1})"))
else:
# For dropoff at step N, check that user completed the prior REQUIRED step but not step N
- # With optional steps, we need to find the last required step before the dropoff step
target_step_index = abs(funnelStep) - 1 # 0-indexed step we're checking dropoff at
-
- # Find the last required step before target_step_index
- prior_required_step_index = None
- for i in range(target_step_index - 1, -1, -1):
- if not getattr(self.context.query.series[i], "optionalInFunnel", False):
- prior_required_step_index = i
- break
-
- if prior_required_step_index is None:
- raise ValueError("Missing prior required step in actors query")
+ prior_required_step_index = self._prior_required_step_index(target_step_index)
# User completed the prior required step but not the target step
conditions.append(parse_expr(f"bitTest(steps_bitfield, {prior_required_step_index})"))
@@ -392,16 +384,76 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):
and self.context.actorsQuery is not None
and self.context.actorsQuery.includeRecordings
):
+ uses_step_event_slots = self._uses_step_event_slots()
if self.context.includeFinalMatchingEvents:
+ required_step_indexes = self._required_step_indexes()
+ # step_reached counts required steps only. UDF v13+ keeps matched events in their own step slot.
+ if uses_step_event_slots and len(required_step_indexes) < len(self.context.query.series):
+ required_step_slots = ",".join(str(i + 1) for i in required_step_indexes)
+ return [
+ parse_expr(
+ f"matched_events_array[[{required_step_slots}][step_reached + 1]] as matching_events"
+ )
+ ]
# Always returns the user's final step of the funnel, 1 indexed
return [parse_expr("matched_events_array[step_reached + 1] as matching_events")]
absolute_actors_step = self._absolute_actors_step
if absolute_actors_step is None:
raise ValueError("Missing funnelStep actors query property")
+ funnelStep = self.context.actorsQuery.funnelStep
+ if uses_step_event_slots and funnelStep is not None and funnelStep < 0:
+ # The drop-off condition matches on the prior required step. Add the optional steps after it that the
+ # actor completed, because those events are the closest to the drop-off.
+ target_step_index = abs(funnelStep) - 1
+ prior_required_step_index = self._prior_required_step_index(target_step_index)
+ step_events: list[ast.Expr] = [parse_expr(f"matched_events_array[{prior_required_step_index + 1}]")]
+ step_events.extend(
+ parse_expr(f"if(bitTest(steps_bitfield, {i}), matched_events_array[{i + 1}], [])")
+ for i in self._optional_step_indexes_between(prior_required_step_index, target_step_index)
+ )
+ if len(step_events) > 1:
+ return [
+ ast.Alias(
+ alias="matching_events",
+ expr=ast.Call(name="arrayDistinct", args=[ast.Call(name="arrayConcat", args=step_events)]),
+ )
+ ]
+ absolute_actors_step = prior_required_step_index
return [parse_expr(f"matched_events_array[{absolute_actors_step + 1}] as matching_events")]
return []
+ def _uses_step_event_slots(self) -> bool:
+ return UDFS["aggregate_funnel"].clickhouse_name == "aggregate_funnel" or UDF_VERSION >= 13
+
+ def _required_step_indexes(self) -> list[int]:
+ return [
+ i for i, series in enumerate(self.context.query.series) if not getattr(series, "optionalInFunnel", False)
+ ]
+
+ def _prior_required_step_index(self, target_step_index: int) -> int:
+ prior_required_step_indexes = [i for i in self._required_step_indexes() if i < target_step_index]
+ if not prior_required_step_indexes:
+ raise ValueError("Missing prior required step in actors query")
+ return prior_required_step_indexes[-1]
+
+ def _optional_step_indexes_between(self, prior_required_step_index: int, target_step_index: int) -> list[int]:
+ # Iterate the series rather than a range from the request, so an out-of-range funnelStep stays cheap
+ return [i for i in range(len(self.context.query.series)) if prior_required_step_index < i < target_step_index]
+
+ def _last_completed_step_slot(self, target_step_index: int) -> str:
+ """The 1-indexed matched_events_array slot of the last step the actor completed before the target step."""
+ prior_required_step_index = self._prior_required_step_index(target_step_index)
+ # The slot of a skipped optional step is empty or holds events from another attempt, so check the path bitfield
+ # and fall back through the optional steps to the prior required step
+ branches = [
+ f"bitTest(steps_bitfield, {i}), {i + 1}"
+ for i in reversed(self._optional_step_indexes_between(prior_required_step_index, target_step_index))
+ ]
+ if not branches:
+ return str(prior_required_step_index + 1)
+ return f"multiIf({', '.join(branches)}, {prior_required_step_index + 1})"
+
def _get_timestamp_outer_select(self) -> list[ast.Expr]:
"""
Returns timestamp selectors for the target step and optionally the preceding step.
@@ -409,11 +461,19 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):
"""
target_step = self._absolute_actors_step
- if target_step is None:
+ if target_step is None or self.context.actorsQuery is None or self.context.actorsQuery.funnelStep is None:
return []
# We pull timestamps from matched_events_array and SQL arrays are 1-indexed
target_step += 1
+ target_slot = str(target_step)
+ preceding_slot = str(target_step - 1)
+ funnelStep = self.context.actorsQuery.funnelStep
+ if self._uses_step_event_slots():
+ if funnelStep < 0:
+ target_slot = self._last_completed_step_slot(abs(funnelStep) - 1)
+ elif self.context.includePrecedingTimestamp and funnelStep > 1:
+ preceding_slot = self._last_completed_step_slot(funnelStep - 1)
final_step = self.context.max_steps
first_step = 1
@@ -423,12 +483,12 @@ class FunnelUDF(FunnelUDFMixin, FunnelBase):
raise ValueError("Cannot request preceding step timestamp if target funnel step is the first step")
return [
- parse_expr(f"matched_events_array[{target_step}][1].1 AS max_timestamp"),
- parse_expr(f"matched_events_array[{target_step - 1}][1].1 AS min_timestamp"),
+ parse_expr(f"matched_events_array[{target_slot}][1].1 AS max_timestamp"),
+ parse_expr(f"matched_events_array[{preceding_slot}][1].1 AS min_timestamp"),
]
elif self.context.includeTimestamp:
return [
- parse_expr(f"matched_events_array[{target_step}][1].1 AS timestamp"),
+ parse_expr(f"matched_events_array[{target_slot}][1].1 AS timestamp"),
parse_expr(f"nullIf(matched_events_array[{final_step}][1].1, 0) AS final_timestamp"),
parse_expr(f"matched_events_array[{first_step}][1].1 as first_timestamp"),
]
diff --git a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr
index badfc4d1ab1a..26457583f10b 100644
--- a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr
+++ b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.ambr
@@ -260,3 +260,91 @@
use_hive_partitioning=0
'''
# ---
+# name: TestFunnelPersons.test_funnel_person_recordings.6
+ '''
+ SELECT source.id,
+ source.id AS id,
+ source.matching_events AS matching_events
+ FROM
+ (SELECT aggregation_target AS actor_id,
+ arrayDistinct(arrayConcat(matched_events_array[2], if(bitTest(steps_bitfield, 2), matched_events_array[3], []))) AS matching_events,
+ actor_id AS id
+ FROM
+ (SELECT arraySort(t -> t.1, groupArray(tuple(accurateCastOrNull(timestamp, 'Float64'), uuid, '', arrayFilter(x -> ifNull(notEquals(x, 0), 1), [multiply(1, step_0), multiply(2, step_1), multiply(3, step_2), multiply(4, step_3)])))) AS events_array,
+ [''] AS prop,
+ arrayJoin(aggregate_funnel(4, 604800, 'first_touch', 'ordered', prop, [3], arrayFilter((x, x_before, x_after) -> not(and(lessOrEquals(length(x.4), 1), equals(x.4, x_before.4), equals(x.4, x_after.4), equals(x.3, x_before.3), equals(x.3, x_after.3), ifNull(greater(x.1, x_before.1), 0), ifNull(less(x.1, x_after.1), 0))), events_array, arrayRotateRight(events_array, 1), arrayRotateLeft(events_array, 1)))) AS af_tuple,
+ af_tuple.1 AS step_reached,
+ plus(af_tuple.1, 1) AS steps,
+ af_tuple.2 AS breakdown,
+ af_tuple.3 AS timings,
+ af_tuple.4 AS matched_event_uuids_array_array,
+ groupArray(tuple(timestamp, uuid, `$session_id`, `$window_id`)) AS user_events,
+ mapFromArrays(arrayMap(x -> x.2, user_events), user_events) AS user_events_map,
+ arrayMap(matched_event_uuids_array -> arrayMap(event_uuid -> user_events_map[event_uuid], arrayDistinct(matched_event_uuids_array)), matched_event_uuids_array_array) AS matched_events_array,
+ af_tuple.5 AS steps_bitfield,
+ aggregation_target AS aggregation_target
+ FROM
+ (SELECT toTimeZone(e.timestamp, 'UTC') AS timestamp,
+ if(not(empty(e__override.distinct_id)), e__override.person_id, e.person_id) AS aggregation_target,
+ e.uuid AS uuid,
+ e.`$session_id` AS `$session_id`,
+ e.`$window_id` AS `$window_id`,
+ if(equals(e.event, 'step one'), 1, 0) AS step_0,
+ if(equals(e.event, 'step two'), 1, 0) AS step_1,
+ if(equals(e.event, 'optional step'), 1, 0) AS step_2,
+ if(equals(e.event, 'step three'), 1, 0) AS step_3
+ FROM events AS e
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
+ person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS e__override ON equals(e.distinct_id, e__override.distinct_id)
+ WHERE and(equals(e.team_id, 99999), and(and(greaterOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), lessOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC'))), in(e.event, tuple('optional step', 'step one', 'step three', 'step two'))), or(equals(step_0, 1), equals(step_1, 1), equals(step_2, 1), equals(step_3, 1))))
+ GROUP BY aggregation_target
+ HAVING ifNull(greaterOrEquals(step_reached, 0), 0))
+ WHERE and(bitTest(steps_bitfield, 1), not(bitTest(steps_bitfield, 3)))
+ ORDER BY aggregation_target ASC SETTINGS join_algorithm='auto') AS source
+ ORDER BY source.id ASC
+ LIMIT 101
+ OFFSET 0 SETTINGS optimize_aggregation_in_order=1,
+ join_algorithm='auto',
+ readonly=2,
+ max_execution_time=60,
+ allow_experimental_object_type=1,
+ max_ast_elements=4000000,
+ max_expanded_ast_elements=4000000,
+ max_bytes_before_external_group_by=0,
+ transform_null_in=1,
+ optimize_min_equality_disjunction_chain_length=4294967295,
+ optimize_rewrite_aggregate_function_with_if=0,
+ optimize_min_inequality_conjunction_chain_length=4294967295,
+ allow_experimental_join_condition=1,
+ use_hive_partitioning=0
+ '''
+# ---
+# name: TestFunnelPersons.test_funnel_person_recordings.7
+ '''
+ SELECT session_replay_events.session_id AS session_id,
+ min(toTimeZone(session_replay_events.min_first_timestamp, 'UTC')) AS start_time,
+ max(session_replay_events.retention_period_days) AS retention_period_days,
+ plus(dateTrunc('DAY', start_time), toIntervalDay(coalesce(retention_period_days, 30))) AS expiry_time
+ FROM session_replay_events
+ WHERE and(equals(session_replay_events.team_id, 99999), in(session_replay_events.session_id, ['s2']))
+ GROUP BY session_replay_events.session_id
+ HAVING and(greaterOrEquals(expiry_time, toDateTime64('2021-01-02 00:00:00.000000', 6, 'UTC')), equals(max(session_replay_events.is_deleted), 0))
+ LIMIT 100 SETTINGS readonly=2,
+ max_execution_time=60,
+ allow_experimental_object_type=1,
+ max_ast_elements=4000000,
+ max_expanded_ast_elements=4000000,
+ max_bytes_before_external_group_by=0,
+ transform_null_in=1,
+ optimize_min_equality_disjunction_chain_length=4294967295,
+ optimize_rewrite_aggregate_function_with_if=0,
+ optimize_min_inequality_conjunction_chain_length=4294967295,
+ allow_experimental_join_condition=1,
+ use_hive_partitioning=0
+ '''
+# ---
diff --git a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr
index 42ca06a63631..113d35a22cde 100644
--- a/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr
+++ b/products/product_analytics/backend/hogql_queries/funnels/test/__snapshots__/test_funnel_persons.new_events_schema.ambr
@@ -129,6 +129,72 @@
output_format_json_escape_forward_slashes=0
'''
# ---
+# name: TestFunnelPersons.test_funnel_person_recordings[new_events_schema.3]
+ '''
+ SELECT source.id,
+ source.id AS id,
+ source.matching_events AS matching_events
+ FROM
+ (SELECT aggregation_target AS actor_id,
+ arrayDistinct(arrayConcat(matched_events_array[2], if(bitTest(steps_bitfield, 2), matched_events_array[3], []))) AS matching_events,
+ actor_id AS id
+ FROM
+ (SELECT arraySort(t -> t.1, groupArray(tuple(accurateCastOrNull(timestamp, 'Float64'), uuid, '', arrayFilter(x -> ifNull(notEquals(x, 0), 1), [multiply(1, step_0), multiply(2, step_1), multiply(3, step_2), multiply(4, step_3)])))) AS events_array,
+ [''] AS prop,
+ arrayJoin(aggregate_funnel(4, 604800, 'first_touch', 'ordered', prop, [3], arrayFilter((x, x_before, x_after) -> not(and(lessOrEquals(length(x.4), 1), equals(x.4, x_before.4), equals(x.4, x_after.4), equals(x.3, x_before.3), equals(x.3, x_after.3), ifNull(greater(x.1, x_before.1), 0), ifNull(less(x.1, x_after.1), 0))), events_array, arrayRotateRight(events_array, 1), arrayRotateLeft(events_array, 1)))) AS af_tuple,
+ af_tuple.1 AS step_reached,
+ plus(af_tuple.1, 1) AS steps,
+ af_tuple.2 AS breakdown,
+ af_tuple.3 AS timings,
+ af_tuple.4 AS matched_event_uuids_array_array,
+ groupArray(tuple(timestamp, uuid, `$session_id`, `$window_id`)) AS user_events,
+ mapFromArrays(arrayMap(x -> x.2, user_events), user_events) AS user_events_map,
+ arrayMap(matched_event_uuids_array -> arrayMap(event_uuid -> user_events_map[event_uuid], arrayDistinct(matched_event_uuids_array)), matched_event_uuids_array_array) AS matched_events_array,
+ af_tuple.5 AS steps_bitfield,
+ aggregation_target AS aggregation_target
+ FROM
+ (SELECT toTimeZone(e.timestamp, 'UTC') AS timestamp,
+ if(not(empty(e__override.distinct_id)), e__override.person_id, e.person_id) AS aggregation_target,
+ e.uuid AS uuid,
+ e.properties.`$session_id` AS `$session_id`,
+ e.properties.`$window_id` AS `$window_id`,
+ if(equals(e.event, 'step one'), 1, 0) AS step_0,
+ if(equals(e.event, 'step two'), 1, 0) AS step_1,
+ if(equals(e.event, 'optional step'), 1, 0) AS step_2,
+ if(equals(e.event, 'step three'), 1, 0) AS step_3
+ FROM events_json AS e
+ LEFT OUTER JOIN
+ (SELECT argMax(person_distinct_id_overrides.person_id, person_distinct_id_overrides.version) AS person_id,
+ person_distinct_id_overrides.distinct_id AS distinct_id
+ FROM person_distinct_id_overrides
+ WHERE equals(person_distinct_id_overrides.team_id, 99999)
+ GROUP BY person_distinct_id_overrides.distinct_id
+ HAVING equals(argMax(person_distinct_id_overrides.is_deleted, person_distinct_id_overrides.version), 0) SETTINGS optimize_aggregation_in_order=1) AS e__override ON equals(e.distinct_id, e__override.distinct_id)
+ WHERE and(equals(e.team_id, 99999), and(and(greaterOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC')), lessOrEquals(e.timestamp, toDateTime64('explicit_redacted_timestamp', 6, 'UTC'))), in(e.event, tuple('optional step', 'step one', 'step three', 'step two'))), or(equals(step_0, 1), equals(step_1, 1), equals(step_2, 1), equals(step_3, 1))))
+ GROUP BY aggregation_target
+ HAVING ifNull(greaterOrEquals(step_reached, 0), 0))
+ WHERE and(bitTest(steps_bitfield, 1), not(bitTest(steps_bitfield, 3)))
+ ORDER BY aggregation_target ASC SETTINGS join_algorithm='auto') AS source
+ ORDER BY source.id ASC
+ LIMIT 101
+ OFFSET 0 SETTINGS optimize_aggregation_in_order=1,
+ join_algorithm='auto',
+ readonly=2,
+ max_execution_time=60,
+ allow_experimental_object_type=1,
+ max_ast_elements=4000000,
+ max_expanded_ast_elements=4000000,
+ max_bytes_before_external_group_by=0,
+ transform_null_in=1,
+ optimize_min_equality_disjunction_chain_length=4294967295,
+ optimize_rewrite_aggregate_function_with_if=0,
+ optimize_min_inequality_conjunction_chain_length=4294967295,
+ allow_experimental_join_condition=1,
+ use_hive_partitioning=0,
+ json_type_escape_dots_in_keys=1,
+ output_format_json_escape_forward_slashes=0
+ '''
+# ---
# name: TestFunnelPersons.test_funnel_person_recordings[new_events_schema]
'''
SELECT source.id,
diff --git a/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py b/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py
index 9bd0145149e3..9c61cae65f79 100644
--- a/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py
+++ b/products/product_analytics/backend/hogql_queries/funnels/test/test_funnel_persons.py
@@ -1,4 +1,4 @@
-from datetime import datetime, timedelta
+from datetime import UTC, datetime, timedelta
from typing import Optional
from uuid import UUID
@@ -12,6 +12,7 @@ from posthog.test.base import (
flush_persons_and_events,
snapshot_clickhouse_queries,
)
+from unittest.mock import patch
from django.test import override_settings
from django.utils import timezone
@@ -33,6 +34,10 @@ from posthog.schema import (
FunnelVizType,
)
+from posthog.hogql.functions.udfs import UDFS
+from posthog.hogql.query import execute_hogql_query
+
+from posthog.clickhouse.workload import Workload
from posthog.hogql_queries.actors_query_runner import ActorsQueryRunner
from posthog.models.event.util import bulk_create_events
from posthog.models.team.team import Team
@@ -435,6 +440,40 @@ class TestFunnelPersons(ClickhouseTestMixin, APIBaseTest):
],
)
+ # Dropoff after a skipped optional step, with recording from the prior required step
+ query_with_optional_step = FunnelsQuery(
+ series=[
+ EventsNode(event="step one"),
+ EventsNode(event="step two"),
+ EventsNode(event="optional step", optionalInFunnel=True),
+ EventsNode(event="step three"),
+ ],
+ interval="day",
+ dateRange=DateRange(date_from="2021-01-01", date_to="2021-01-08"),
+ funnelsFilter=FunnelsFilter(funnelWindowIntervalUnit="day", funnelWindowInterval=7),
+ )
+ with (
+ patch("products.product_analytics.backend.hogql_queries.funnels.funnel.UDF_VERSION", 13),
+ patch.object(UDFS["aggregate_funnel"], "clickhouse_name", "aggregate_funnel_v13"),
+ ):
+ results = get_actors(query_with_optional_step, self.team, funnel_step=-4, include_recordings=True)
+ self.assertEqual(results[0][0], p1.uuid)
+ self.assertEqual(
+ list(results[0][2]),
+ [
+ {
+ "session_id": "s2",
+ "events": [
+ {
+ "uuid": UUID("21111111-1111-1111-1111-111111111111"),
+ "timestamp": timezone.now() + timedelta(days=1),
+ "window_id": "w2",
+ }
+ ],
+ }
+ ],
+ )
+
def test_parses_step_breakdown_correctly(self):
person1 = _create_person(
distinct_ids=["person1"],
@@ -602,6 +641,106 @@ class TestFunnelPersons(ClickhouseTestMixin, APIBaseTest):
self.assertCountEqual(set(results[0][1]["distinct_ids"]), {"person1", "anon1"})
self.assertCountEqual(set(results[1][1]["distinct_ids"]), {"person2", "anon2"})
+ @parameterized.expand([(12, False), (12, True), (13, False), (13, True)])
+ @time_machine.travel("2021-05-02", tick=False)
+ def test_optional_funnel_step_recordings(self, udf_version: int, visit_optional_steps: bool) -> None:
+ with (
+ patch("products.product_analytics.backend.hogql_queries.funnels.funnel.UDF_VERSION", udf_version),
+ patch.object(UDFS["aggregate_funnel"], "clickhouse_name", f"aggregate_funnel_v{udf_version}"),
+ ):
+ optional_steps = {2, 4}
+ visited_steps = [1, 3]
+ if visit_optional_steps:
+ visited_steps = [1, 2, 3, 4]
+ journeys_for(
+ {
+ "replay-user": [
+ {
+ "event": f"step {step}",
+ "timestamp": datetime(2021, 5, 1, 0, step),
+ "event_uuid": str(UUID(int=step)),
+ "properties": {"$session_id": str(UUID(f"01800000-0000-7000-8000-{step:012d}"))},
+ }
+ for step in visited_steps
+ ]
+ },
+ self.team,
+ )
+ for step in visited_steps:
+ produce_replay_summary(
+ team_id=self.team.pk,
+ session_id=str(UUID(f"01800000-0000-7000-8000-{step:012d}")),
+ distinct_id="replay-user",
+ first_timestamp=datetime(2021, 5, 1, 0, step, tzinfo=UTC),
+ )
+ query = FunnelsQuery(
+ series=[
+ EventsNode(event=f"step {step}", optionalInFunnel=step in optional_steps) for step in range(1, 6)
+ ],
+ dateRange=DateRange(date_from="2021-05-01", date_to="2021-05-02"),
+ )
+ for funnel_step in [3, -5]:
+ with self.subTest(funnel_step=funnel_step):
+ results = get_actors(query, self.team, funnel_step=funnel_step, include_recordings=True)
+ self.assertEqual(len(results), 1)
+ expected_steps = [3]
+ if udf_version == 12:
+ if not visit_optional_steps:
+ expected_steps = []
+ elif funnel_step == -5:
+ expected_steps = [4]
+ elif visit_optional_steps and funnel_step == -5:
+ expected_steps = [3, 4]
+ self.assertCountEqual(
+ [
+ (recording["session_id"], [event["uuid"] for event in recording["events"]])
+ for recording in results[0][2]
+ ],
+ [
+ (str(UUID(f"01800000-0000-7000-8000-{step:012d}")), [UUID(int=step)])
+ for step in expected_steps
+ ],
+ )
+
+ if udf_version != 12:
+ runner = FunnelsQueryRunner(query=query, team=self.team)
+ runner.context.actorsQuery = FunnelsActorsQuery(source=query, funnelStep=-5, includeRecordings=True)
+ runner.context.includeFinalMatchingEvents = True
+ response = execute_hogql_query(
+ runner.to_actors_query(),
+ team=self.team,
+ query_type="test_funnel_final_recordings",
+ workload=Workload.ONLINE,
+ )
+ self.assertEqual(len(response.results), 1)
+ self.assertEqual(response.results[0][1][0][1], UUID(int=3))
+
+ last_step_before_drop_off = 4 if visit_optional_steps else 3
+ self.assertEqual(
+ self._funnel_actor_timestamp(query, funnel_step=-5, column="timestamp"),
+ datetime(2021, 5, 1, 0, last_step_before_drop_off, tzinfo=UTC),
+ )
+ last_step_before_step_3 = 2 if visit_optional_steps else 1
+ self.assertEqual(
+ self._funnel_actor_timestamp(query, funnel_step=3, column="min_timestamp"),
+ datetime(2021, 5, 1, 0, last_step_before_step_3, tzinfo=UTC),
+ )
+
+ def _funnel_actor_timestamp(self, query: FunnelsQuery, funnel_step: int, column: str) -> datetime:
+ runner = FunnelsQueryRunner(query=query, team=self.team)
+ runner.context.actorsQuery = FunnelsActorsQuery(source=query, funnelStep=funnel_step)
+ runner.context.includeTimestamp = column == "timestamp"
+ runner.context.includePrecedingTimestamp = column == "min_timestamp"
+ response = execute_hogql_query(
+ runner.to_actors_query(),
+ team=self.team,
+ query_type="test_funnel_paths_timestamps",
+ workload=Workload.ONLINE,
+ )
+ assert response.columns is not None
+ self.assertEqual(len(response.results), 1)
+ return response.results[0][response.columns.index(column)]
+
def test_optional_funnel_step_actors_query(self):
journeys_for(
{