:hedgehog: PostHog is the leading platform for building self-driving products. Our developer tools – AI observability, analytics, session replay, flags, experiments, error tracking, logs, and more – capture all the context agents need to diagnose problems, uncover opportunities, and ship fixes. Steer it all from Slack, web, desktop, or the MCP.
Funnel UDFs
Development workflow
- Building the binaries:
- Exit flox:
exit - Switch to the subfolder:
cd funnel-udf - Execute the build script:
./build.sh - Restart the clickhouse docker container
- Exit flox:
Deployment workflow
Cloud ClickHouse does not read the master branch of this repo.
Ansible (posthog-cloud-infra/ansible/roles/clickhouse/tasks/setup_udfs.yml) downloads this repo at the commit pinned in posthog_udfs_revision (ansible/roles/clickhouse/defaults/main.yml).
It copies that commit's posthog/user_scripts to the ClickHouse nodes, installs its latest_user_defined_function.xml as ClickHouse's user_defined_function.xml, and reloads the functions.
A merge here changes nothing in cloud ClickHouse until the pin moves.
For revertible cloud deploys:
- Develop using the binary files at the top level of
user_scripts(see section above), with schema defined indocker/clickhouse/user_defined_function.xml - When ready to deploy, increment the version in
posthog/udf_versioner.pyand run it. This generates versioned binaries andlatest_user_defined_function.xml - Land a PR with the updated
user_scriptsfolder — include both the new version folder (e.g.v12/) and the regeneratedlatest_user_defined_function.xml. Do not include theUDF_VERSIONbump in this PR — that goes in a separate PR (step 6). Check that git keeps the wrappers, binaries and Python scripts in the version folder executable (mode100755), because local, CI and self-hosted ClickHouse run them directly - In
posthog-cloud-infra, bumpposthog_udfs_revisionto amastercommit that includes the PR from step 3. Merging that change deploys the pinneduser_scriptsto every ClickHouse environment at once. The bump also ships every otheruser_scriptschange since the previous pin, and the unversioned utility UDFs take effect immediately, so list those changes in the infra PR - Verify in metabase by running
SELECT aggregate_funnel_vXX(). An "invalid arguments" error means the function is registered. An "unknown function" error means the deploy hasn't taken effect yet. Also run a valid funnel query against the new version. Registration alone does not check the launcher or binary. Make sure to do this for both EU and US - Once the deploy is confirmed, land a separate PR that bumps
UDF_VERSIONinposthog/udf_versioner.pyto the new version. This switches the posthog query builder to use the new UDF
Design decisions
- Each replay slot holds the events of its own step, optional steps included. Optional matches never change funnel counts, because the duplicate-event check only sees required steps.
- Unversioned binaries use the current event layout. Replay queries must use that layout even while cloud queries select an older UDF version.
- We tried writing the UDFs in Python, but this cause ClickHouse nodes to crash as the garbage collector could not keep up. As such we're now writing the functions in Rust.
- We're using
cross-rsto compile the binaries - it's a cross platfrom rust compiler. Cross.tomlpins the build images, which link against glibc 2.23. Check the highest version a new binary needs withobjdump -T <binary> | grep -o 'GLIBC_[0-9.]*' | sort -uV | tail -1.
Troubleshooting
Running ./build.sh fails in flox environment
If you're using flox for development, you'll have to exit out of the environment first (nix os isn't supported).
Error:
0: could not determine os in target triplet
1: unsupported os in target, abi: "1.82.0", system: "rustc"