Skip to content

OpenPanel is an open-source web and product analytics platform, an open-source alternative to Mixpanel with optional self-hosting.

ralph(M39-003): Bounce rate: drop FINAL AND rewrite sign-aware (never a plain drop)

ralphc31cc593375d
diff --git a/packages/core/src/modules/overview/src/overview.sql.proof.md b/packages/core/src/modules/overview/src/overview.sql.proof.md
index 90c39cc4a..55704e3c1 100644
--- a/packages/core/src/modules/overview/src/overview.sql.proof.md
+++ b/packages/core/src/modules/overview/src/overview.sql.proof.md
@@ -107,6 +107,47 @@
`verdict` `topGenericQuery` 47–51 → 18 ms, `topEntryExitQuery` 61–86 → 26–30 ms, series
52–56 → 24 ms.
+- **Amended by M39-003 (2026-09-16): the `metricsWithPageFilterQuery` bounce rate drops `FINAL` and
+ is now sign-weighted.** `countIf(is_bounce = 1 AND sign = 1) * 100. / countIf(sign = 1) FROM sessions
+ FINAL WHERE sign = 1` became `sumIf(sign, is_bounce = 1) * 100. / sum(sign) FROM sessions` with no
+ `sign` filter, plus `HAVING sum(sign) > 0` after `WITH ROLLUP`. The old form is not linear in
+ `sign`, so dropping `FINAL` alone counts the stale `+1` of every uncollapsed pair. That is the
+ "naive drop" control below.
+ **The two checks M38-002 left open.** Both were run on a scratch `openpanel_m39003` database
+ (`sessions`/`events` created `AS openpanel.*`, merges stopped, seeded in separate parts), with a
+ bucket and a whole window whose sessions all cancel:
+ 1. *Division by zero.* A bucket with `sum(sign) = 0` evaluates to `0 * 100. / 0` = **`nan`**, not an
+ error. Without `HAVING`, a net-zero day came out as a `bounce_rate = nan` row. A net-zero window
+ came out as `overall_bounce_rate = nan` on every row, and `getMetricsWithPageFilter`'s
+ `?? 0` would have passed that `nan` through. `FINAL` had produced no such rows.
+ 2. *`WITH ROLLUP` on net zero.* ClickHouse emits no rollup row for an empty input, but it does
+ emit one (`nan`) for rows that sum to zero. `HAVING sum(sign) > 0` is applied to the rollup row
+ as well (verified), so both cases return to `FINAL`'s shape: the bucket is absent, and with no
+ rollup row `overall_bounce_rate` is `NULL`, which is the "no data" signal
+ `metrics-page-filter.test.ts` pins. `sum(sign)` equals `FINAL`'s surviving `+1` count whenever
+ every `-1` has its `+1`, which is the `session-buffer.ts` invariant.
+ **Proof**, run 2026-09-16 21:57–22:05 UTC, read-only on `openpanel`, `wait_end_of_query=1`,
+ `use_query_cache=0`, `use_query_condition_cache=0`, `session_timezone=UTC`. The statements were
+ rendered by the real builder, and the old form is the same text with the previous `sessionAgg`
+ put back. `count(), sum(sipHash64(formatRow('TSV', *)))` was compared for the whole statement
+ **and** for `session_stats` alone (the join hides session-only buckets): **158 of 158 hashes
+ identical over 79 cases.** Cases: `chatpaper`, `verdict`, `bayse`, `earlysalary-production` and
+ `website-8103`, each at 30 d/day, with no filter and with `path=/`; 7 d/hour; 8 w/week with
+ `country=US`; 2 mo/month; 1 d/minute. `e2e-sessions` (the box's 21 unmerged `-1` rows) at
+ day/week/hour, a 1 h minute window, the 1 s window holding the `-1` rows, and `country=SE`. The
+ scratch net-zero-day project (pending pairs, a pair whose replacement moved day and path, a
+ cancelled-only day) and net-zero-window project, at day/hour/week/month × no filter /
+ matching / non-matching / other path, plus minute and single-day windows.
+ **Control:** the naive drop (`FINAL` removed, `countIf`/`WHERE sign = 1` kept) **differed** on 5 of
+ 6 unmerged cases (e2e-sessions day and both minute windows; the scratch project day, with 3 → 4
+ rows, and minute, with 5 → 7 rows). The probe therefore does detect the wrong answer.
+ **Cost**, warm, 3 runs each, 2026-09-16 22:05–22:06 UTC, load 1.4–2.4, 30 d/day, shipped → new.
+ `session_agg` alone: `chatpaper` 139–159 → 45–49 ms (3,571,254 → 3,448,379 rows, 103 → 73 MiB);
+ `chatpaper` with `path=/` 203–228 → 87–89 ms; `verdict` 40–44 → 14–19; `bayse` 28–33 → 11–12;
+ `earlysalary-production` 33–34 → 14–15. Whole statement: `chatpaper` 2,737–2,829 → 2,616–2,623 ms,
+ `path=/` 734–741 → 611–623; `verdict` 63–66 → 40–43; `bayse` 416–421 → 397–404;
+ `earlysalary-production` 507–513 → 493–509. The `events` side dominates the whole statement.
+
- **Date**: 2026-09-04. **Data**: local prod-copy `openpanel` (319,499,000 events, static).
Project: `skills-directory` (293,597 events, 75,727 sessions, 2026-07-01..2026-08-25).
Window used throughout: `2026-08-01 00:00:00`..`2026-08-08 00:00:00`, UTC.
diff --git a/packages/core/src/modules/overview/src/overview.sql.ts b/packages/core/src/modules/overview/src/overview.sql.ts
index 36fe7964b..f4961ba68 100644
--- a/packages/core/src/modules/overview/src/overview.sql.ts
+++ b/packages/core/src/modules/overview/src/overview.sql.ts
@@ -243,17 +243,20 @@ export function metricsWithPageFilterQuery(
WITH ROLLUP
`;
+ // Sign-weighted instead of `FINAL`: an uncollapsed (+1, V)/(-1, V) pair nets
+ // to 0. `HAVING` drops buckets (and the rollup row) whose sessions all
+ // cancelled — `FINAL` never produced those rows, and they would read `nan`.
const sessionAgg = sql`
SELECT
${dateBucket} AS date,
- round((countIf(is_bounce = 1 AND sign = 1) * 100.) / countIf(sign = 1), 2) AS bounce_rate
- FROM sessions FINAL
- WHERE sign = 1
- AND project_id = ${sql.string(input.projectId)}
+ round((sumIf(sign, is_bounce = 1) * 100.) / sum(sign), 2) AS bounce_rate
+ FROM sessions
+ WHERE project_id = ${sql.string(input.projectId)}
AND ${dateRangeWhere('created_at', input.startDate, input.endDate)}
${rawWhere(input.rawSessionFilterWhere)}
GROUP BY date
WITH ROLLUP
+ HAVING sum(sign) > 0
ORDER BY date ASC
`;