Skip to content

OpenPanel is an open-source web and product analytics platform, an open-source alternative to Mixpanel with optional self-hosting.

ralph(M12-001): SQL migration recipe + the P12 grep gate: docs/SQL_MIGRATION_RECIPE.md with one worked conversion executed against ClickHouse, tooling/gates/p12-grep-gates.sh with the baseline recorded

ralphbe029da17b54
diff --git a/docs/SQL_MIGRATION_RECIPE.md b/docs/SQL_MIGRATION_RECIPE.md
new file mode 100644
index 000000000..6ca7a2a5e
--- /dev/null
+++ b/docs/SQL_MIGRATION_RECIPE.md
@@ -0,0 +1,631 @@
+# SQL migration recipe — `sqlstring` / clix → the `sql` tag
+
+The eight P12 conversion tasks after this one share one procedure, written here
+so it is not re-derived eight times. ADR-013 decision 21 is the target:
+`sqlstring` = 0, clix = 0, no builder, all three definer files deleted.
+
+The gate is `tooling/gates/p12-grep-gates.sh`. Run `--report` before and after
+every conversion; the wave is finished when `--assert` exits 0.
+
+## Baseline 448fa219a68bcc6929b6a11276fbcb414511e8f9
+
+`bash tooling/gates/p12-grep-gates.sh --report`, run 2026-09-07 on this box.
+Reproduced verbatim. This commit adds only `tooling/gates/` and `docs/`, neither
+of which the gate scans, so the table is byte-identical at the parent and at
+this commit.
+
+```
+ sqlstring clix sql-builder file
+----------------------------------------------------------------------------
+ 1 0 0 apps/start/package.json
+ 2 0 0 packages/core/package.json
+ 3 0 0 packages/core/src/buffers/group-buffer.ts
+ 3 0 0 packages/core/src/buffers/profile-backfill-buffer.ts
+ 2 0 0 packages/core/src/buffers/profile-buffer.ts
+ 13 0 0 packages/core/src/modules/chart/src/field-resolution.ts
+ 2 0 0 packages/core/src/modules/chart/src/filter-cast.ts
+ 53 0 0 packages/core/src/modules/chart/src/filter-where.ts
+ 13 0 0 packages/core/src/modules/chart/src/table-filter-where.ts
+ 49 0 0 packages/core/src/modules/cohort/cohort.service.ts
+ 0 13 0 packages/core/src/modules/insight/src/cached-clix.ts
+ 0 2 0 packages/core/src/modules/insight/src/engine.ts
+ 0 11 0 packages/core/src/modules/insight/src/legacy-scan.ts
+ 0 5 0 packages/core/src/modules/insight/src/modules/devices.module.ts
+ 0 5 0 packages/core/src/modules/insight/src/modules/entry-pages.module.ts
+ 0 5 0 packages/core/src/modules/insight/src/modules/geo.module.ts
+ 0 5 0 packages/core/src/modules/insight/src/modules/page-trends.module.ts
+ 0 5 0 packages/core/src/modules/insight/src/modules/referrers.module.ts
+ 0 4 0 packages/core/src/modules/insight/src/referrer-spikes.ts
+ 0 1 0 packages/core/src/modules/insight/src/types.ts
+ 0 2 0 packages/core/src/modules/mcp/src/tools/analytics/property-values.ts
+ 10 0 5 packages/core/src/modules/organization/organization.service.ts
+ 2 0 0 packages/core/src/modules/profile/src/profile.sql.ts
+ 2 2 0 packages/core/src/modules/project/project.service.ts
+ 8 7 0 packages/core/src/modules/realtime/realtime.service.ts
+ 0 7 0 packages/core/src/modules/widget/widget.rpc.ts
+ 0 2 0 packages/core/src/v1-compat.ts
+ 0 1 1 packages/db/index.ts
+ 2 0 0 packages/db/package.json
+ 3 0 0 packages/db/src/clickhouse/client.ts
+----------------------------------------------------------------------------
+ 168 77 6 TOTAL
+```
+
+Read the totals as three separate deletions:
+
+- **`sqlstring` 168** — 6 files hold 133 of them: `filter-where.ts` (53),
+ `cohort.service.ts` (49), `table-filter-where.ts` (13), `field-resolution.ts`
+ (13), `organization.service.ts` (10), `realtime.service.ts` (8). Every one is
+ `escape()`; there is no `format()` and no `escapeId()` in the tree.
+- **clix 77** — the insight modules (5 × 5 through `createCachedClix`),
+ `legacy-scan.ts` (11), `cached-clix.ts` (13, the wrapper itself),
+ `realtime.service.ts` (7), `widget.rpc.ts` (7), `referrer-spikes.ts` (4),
+ `property-values.ts` (2), `project.service.ts` (2), `engine.ts` (2),
+ `types.ts` (1), `v1-compat.ts` (2), `packages/db/index.ts` (1).
+- **sql-builder 6** — `organization.service.ts` (5 sites) plus the
+ `packages/db/index.ts` re-export.
+
+The three files that DEFINE the builders are excluded from the scan and checked
+separately by `--assert`: `packages/db/src/clickhouse/query-builder.ts`, its
+`.test.ts`, and `packages/db/src/sql-builder.ts`. `packages/db/index.ts` still
+re-exports the first and third with a comment saying they are dead per ADR-013
+and blocked on exactly this conversion; that line is the last one to go.
+
+Comment-only mentions do not count. `funnel.sql.ts`, `retention.sql.ts`,
+`compiled.ts`, `mcp/src/tools/shared.ts`, `event|profile|session/src/filter-clauses.ts`
+and `overview-raw-where.sql.test.ts` all name `sqlstring` or `clix` in prose and
+score 0 — that is correct, and it is what makes 0 reachable without editing
+history.
+
+## The `sql` tag as it is
+
+`packages/db/src/clickhouse/sql.ts`. Every exported member, with the line it is
+defined on:
+
+| Export | Line | What it is |
+|---|---|---|
+| `sql` (the tag) | 261 (`tag` at 258) | `` sql`…${slot}…` `` → `SqlFragment`. Slots are typed `SqlFragment \| SqlParam`, so a bare string or number in a slot is a **compile-time error**. There is deliberately no `sql.raw()`. |
+| `sql.empty` | 262 (`EMPTY_FRAGMENT` 206) | A fragment that renders to nothing. The identity element for conditionals — `cond ? sql\`…\` : sql.empty`. |
+| `sql.param(type, value)` | 264 (`param` 162) | Escape hatch for a ClickHouse type with no shortcut. `sql.param('Int64', -1)`, `sql.param('DateTime', …)`. |
+| `sql.string(v)` | 265 | `{pN:String}` |
+| `sql.uint64(v)` | 266 | `{pN:UInt64}` — `number \| bigint \| string` |
+| `sql.float64(v)` | 267 | `{pN:Float64}` |
+| `sql.date(v)` | 268 (`toClickhouseDate` 200) | `{pN:Date}`. A `Date` is truncated to `YYYY-MM-DD` **in JS** — ClickHouse rejects a unix timestamp for `Date`, and the driver would send one. |
+| `sql.dateTime64(v, precision = 3)` | 269 | `{pN:DateTime64(3)}` |
+| `sql.uuid(v)` | 273 | `{pN:UUID}` |
+| `sql.bool(v)` | 274 | `{pN:Bool}` |
+| `sql.array(itemType, values)` | 275 | `{pN:Array(<itemType>)}` |
+| `sql.map(keyType, valueType, v)` | 277 | `{pN:Map(<k>,<v>)}`; accepts a `Map` or a plain object |
+| `sql.nullable(innerType, v)` | 284 | `{pN:Nullable(<inner>)}` — the only correct binding for a value that may be `null` |
+| `sql.identifier(name)` | 286 | `{pN:Identifier}`, bound server-side. Positional limits below. |
+| `sql.id(name, allowed?)` | 288 (`assertSafeIdentifier` 166) | Validated identifier **inlined as text**, for positions `{x:Identifier}` cannot reach. Throws `SqlIdentifierError`; never falls back to interpolation (ADR-013 R3). Pass `allowed` wherever the call site has a closed column set. |
+| `sql.join(fragments, separator?)` | 293 | Composes fragments. `separator` is a closed union (`SQL_JOIN_SEPARATORS`, line 151): `', '`, `' AND '`, `' OR '`, `' UNION ALL '`, `' '`, `'\n'`. Empty input → `sql.empty`. |
+| `SqlFragment` (class) | 87 | `strings` + `slots`; `new SqlFragment([text], [])` is the one text bridge (see `compiledText` below). |
+| `SqlFragment#toStatement()` | 92 | `{ query, query_params }`. Placeholder names are assigned **here**, from one counter, which is why nesting and reuse never collide (ADR-013 R2). |
+| `toStatement(query)` | 132 | Accepts `string \| SqlFragment`, so `chQuery` has one code path. |
+| `isSqlFragment(v)` | 110 | Type guard. |
+| `SqlIdentifierError` | 138 | What `sql.id` throws. |
+| `SqlParam` / `SqlParamValue` / `SqlParamType` / `SqlSlot` / `SqlStatement` / `SqlJoinSeparator` | 61 / 46 / 58 / 68 / 71 / 159 | Types. |
+
+`{name:Identifier}` does not work everywhere. The measured matrix is a comment
+in `packages/db/src/clickhouse/sql.clickhouse.test.ts:241-269` (ClickHouse
+26.1.3.52, 2026-09-02) — SELECT / alias / function argument / WHERE / GROUP BY /
+ORDER BY / FROM / JOIN / FROM-inside-a-CTE all work; a qualified
+`'alias.column'` in ONE param and a `SETTINGS` value both FAIL. Do not re-derive
+it; read it.
+
+## Idiom mapping
+
+`v` is the V1 value. `n` is a column name. Every V2 form below composes; none of
+them is a special case.
+
+| # | V1 | V2 | Note |
+|---|---|---|---|
+| 1 | `` `${n} = ${sqlstring.escape(v)}` `` | ``sql`${sql.id(n, ALLOWED)} = ${sql.string(v)}` `` | The default. `escape` produces `'…'`; the tag produces `{p1:String}` + a param. |
+| 2 | `` `${n} = ${sqlstring.escape(String(v).trim())}` `` on a numeric column | ``sql`${col} = ${sql.uint64(v)}` `` **only if** the column is unsigned and the V1 text had no `toFloat64` wrapper | See "numbers" below. When V1 wrote `toFloat64(${n}) > toFloat64('5')`, keep the wrapper and bind `sql.string(v)` — the cast is the behaviour. |
+| 3 | `` `${n} > ${sqlstring.escape(v)}` `` where `v` may be negative | ``sql`${col} > ${sql.param('Int64', v)}` `` | `sql.uint64(-1)` throws `BAD_QUERY_PARAMETER` (code 457) at the server. Measured 2026-09-07. |
+| 4 | fractional | `sql.float64(v)` | |
+| 5 | `` `toDateTime('${d}')` `` / `` toDateTime(${sqlstring.escape(d)}) `` | ``sql`toDateTime(${sql.string(d)})` `` | **Keep `toDateTime(...)` and bind the string.** This is what every landed M7 conversion did (`chart.sql.ts`, `funnel.sql.ts`, `retention.sql.ts`). Do not "improve" it to `sql.dateTime64` — that changes the emitted expression and its parse rules. |
+| 6 | `toDate('${d}')` on a calendar day | `sql.date(d)` where the expression is genuinely a `Date` column comparison; otherwise `toDate(${sql.string(d)})` | `sql.date` truncates a JS `Date` to `YYYY-MM-DD` in JS (line 200). |
+| 7 | `sqlstring.escape(true)` → `true` | `sql.bool(v)` → `{p:Bool}` | |
+| 8 | `sqlstring.escape(null)` → **`NULL`** | `sql.nullable('String', v)` | `sql.string(null as any)` sends `null`, which ClickHouse parses as **`''`** for `String`. Measured 2026-09-07: `SELECT {p1:String}` with `p1=null` returns `""`, while `{p1:Nullable(String)}` returns `null` and `isNull()` = 1. Getting this wrong turns `IS NULL` into `= ''` silently. |
+| 9 | `` `${n} IN (${v.map(escape).join(', ')})` `` | ``sql`${col} IN ${sql.array('String', v)}` `` | **Drop the parentheses.** `IN {p:Array(String)}` and `IN ({p:Array(String)})` both work (measured), but the array form is the idiom in every landed conversion. |
+| 10 | `IN ()` on an empty list | `IN ${sql.array('String', [])}` | Both return 0 rows; ClickHouse accepts a literal `IN ()`. Measured 2026-09-07. An empty array is a legal param — it does not need a `sql.empty` guard for correctness, only for readability. |
+| 11 | `` `${n} LIKE ${sqlstring.escape(`%${v}%`)}` `` | ``sql`${col} LIKE ${sql.string(`%${v}%`)}` `` | **Build the `%` in JS, on the value, exactly as V1 did.** Do not move `%` into the SQL text as `'%' \|\| {p}`; that is a different expression. |
+| 12 | `%` / `_` inside `v` | unchanged | Neither `sqlstring.escape` nor a bound param escapes LIKE metacharacters. A user value of `a%b` is a wildcard on both sides. Preserve the defect; do not fix it during a conversion. |
+| 13 | `` `${n} = ${escape(v)}` `` where `n` came from user input | ``sql`${sql.id(n, ALLOWED)} = …` `` — **`sql.id` only** | Never `compiledText(n)`, never a template hole. `sql.id` throws on anything that is not a bare or once-qualified identifier (≤64 chars, `[A-Za-z_][A-Za-z0-9_]*`). Pass the closed set as `allowed` whenever one exists — `EVENT_TOP_LEVEL_COLUMNS`, `PROFILE_COLUMNS`, `SESSION_LIST_COLUMNS` are already used this way. |
+| 14 | a dynamic column list joined with `', '` | `sql.join(cols.map((c) => sql.id(c, ALLOWED)))` | Default separator is `', '`. For a WHERE conjunction use `sql.join(parts, ' AND ')`; for a disjunction `' OR '`. |
+| 15 | a nested builder / sub-select spliced as text | a nested `SqlFragment` in a slot | Nesting is free: names are assigned once at `toStatement()`, depth-first, from one counter. Three levels deep is tested (`sql.test.ts`, "auto-named params (R2)"). |
+| 16 | `properties[${escape(k)}]` | ``sql`properties[${sql.string(k)}]` `` | A map **key** is a value, not an identifier. |
+
+### Numbers: which ClickHouse type, and why
+
+There is no "the number type". Pick by the column and by what V1 emitted:
+
+- **`UInt64`** (`sql.uint64`) for counts, limits, offsets, `range(n)` bounds and
+ `INTERVAL n` operands — anything that cannot be negative. This is 50 of the 55
+ numeric bindings already in core (`retention.sql.ts:97,202,233,243`,
+ `sankey.sql.ts:115,175,192`, `chart.sql.ts:620`). It **throws at the server**
+ on a negative value (measured: `Value -1 cannot be parsed as UInt64 … code
+ 457`), which is the point — a limit that went negative is a bug, not a row.
+- **`Int64`** via `sql.param('Int64', v)` when the value is a signed offset or a
+ user-supplied comparand that may be negative. There is no `sql.int64`
+ shortcut; `sql.param` is the sanctioned form (line 264).
+- **`Float64`** (`sql.float64`) for fractional values —
+ `event.sql.ts:167` and `session.sql.ts:124` bind a lookback in days this way.
+- **`String`** when V1 wrapped the value in a cast. `filter-where.ts:489` emits
+ ``toFloat64(${name}) > toFloat64('5')`` — a *quoted* comparand fed to
+ `toFloat64`. The V2 form is
+ ``sql`toFloat64(${col}) > toFloat64(${sql.string(v)})` ``. Binding `Float64`
+ there deletes the cast and changes the comparison for a non-numeric input,
+ which is exactly the kind of silent drift the per-query proof exists to catch.
+
+One precision note, measured: `sql.uint64('9007199254740993')` binds correctly
+but comes back through `format: 'JSON'` as `9007199254740992`. That is the
+JSON response, not the parameter. If a converted query returns an id above 2^53,
+compare it as a string.
+
+### The `compiledText` seam
+
+`packages/core/src/modules/chart/src/compiled.ts` is the ONE place the chart
+module splices pre-compiled SQL text into a fragment:
+
+- `compiledText(text)` (line 14) — `new SqlFragment([text], [])`. Not a
+ `sql.raw()`: it is a named export in one file, so the conversion's endpoint is
+ "this file has no callers left", which is a grep.
+- `compiledTextWithProfileRefs(text, keys)` (line 23) — the same, after running
+ `rewriteProfilePropertyRefs`.
+
+**What crosses it today:** the output of `field-resolution.ts` and
+`filter-where.ts` — V1's field resolver and filter compiler, both of which still
+render `sqlstring`-escaped text. ADR-013 left the shared compilers alone ("two
+behaviours, not two builders"), so M7 bridged them rather than rewriting them.
+Those two files are 66 of the baseline's 168 `sqlstring` lines.
+
+**Why it disappears:** once `getEventFiltersWhereClause` and the field resolver
+return `SqlFragment` instead of `string`, their output is already a fragment and
+there is nothing to bridge. Every caller that today writes
+`compiledText(getEventFiltersWhereClause(...))` writes the fragment directly.
+`compiled.ts` is then deleted whole, and with it the last legitimate
+`new SqlFragment([text], [])` in core. The conversion order therefore matters:
+**convert the compilers, then delete the seam** — not the other way round.
+
+## clix to sql
+
+A clix chain is a fragment sum. `packages/db/src/clickhouse/query-builder.ts`
+defines the methods being replaced (`select`/`from` 117, `where` 150,
+`groupBy` 205, `orderBy` 237, `limit` 246, `offset` 253, `having` 211,
+`with` 266, joins 297-345, `settings` 261, `toSQL` 570, `execute` 552).
+
+```ts
+// V1
+const rows = await clix(ch, timezone)
+ .select<Row>(['name', 'count() as count'])
+ .from(TABLE_NAMES.events)
+ .where('project_id', '=', projectId)
+ .where('created_at', '>=', clix.datetime(start))
+ .groupBy(['name'])
+ .orderBy('count', 'DESC')
+ .limit(10)
+ .execute();
+
+// V2
+const statement = sql`
+ SELECT ${sql.join([sql.id('name'), sql`count() as count`])}
+ FROM ${sql.id(TABLE_NAMES.events)}
+ WHERE project_id = ${sql.string(projectId)}
+ AND created_at >= toDateTime(${sql.string(start)})
+ GROUP BY name
+ ORDER BY count DESC
+ LIMIT ${sql.uint64(10)}
+`;
+const rows = await chQuery<Row>(deps, statement, { session_timezone: timezone });
+```
+
+Rules that fall out of the shape:
+
+- **Clause order becomes textual, so write the whole statement.** clix let
+ `.where()` calls accumulate in any order; a fragment does not. Collect the
+ conditions into an array and `sql.join(conditions, ' AND ')`, with
+ `sql.empty` for the absent ones — that is what `event.sql.ts`,
+ `profile.sql.ts` and `group.sql.ts` do.
+- **`.if()` / `.endIf()` has no successor and needs none.** clix's `_skipNext`
+ (`query-builder.ts:578-589`) silently swallows the next clause when an
+ `endIf()` is missing. A ternary into `sql.empty` cannot.
+- **`.rawWhere()` / `.rawJoin()` / `.rawHaving()` / `clix.exp()` are already raw
+ text** — those are the easy ones. They become the literal parts of a template,
+ with any value in them pulled out into a param.
+- **`.settings({ session_timezone })` does not become SQL text.** A `SETTINGS`
+ value cannot be a bound param (matrix, above), and clix itself had already
+ moved it to `clickhouse_settings` (`query-builder.ts:556` is the commented-out
+ text form, `:562` the live one). Pass it as `chQuery`'s settings argument.
+- **`.execute()` becomes `chQuery(deps, statement, settings)`** — one code path
+ for `string` and `SqlFragment` via `toStatement` (sql.ts:132), keeping
+ `withRetry` and round-robin (ADR-013 R1).
+
+### `createCachedClix`
+
+`packages/core/src/modules/insight/src/cached-clix.ts` wraps `clix` so that
+`execute()` memoises on `sha256(query.toSQL() + '|' + timezone)` into a
+caller-supplied `Map` (lines 22-55). It is used by the five insight modules
+(`devices`, `entry-pages`, `geo`, `page-trends`, `referrers`) plus `engine.ts`,
+which is 27 of the baseline's 77 clix lines.
+
+The replacement keys on the **rendered statement plus its params**, because two
+statements with identical text and different params are different queries — the
+whole point of binding:
+
+```ts
+// modules/insight/src/cached-query.ts (the shape; write it in the task that needs it)
+export function createStatementCache(cache?: Map<string, unknown>) {
+ return async function run<T>(
+ deps: ServiceDeps,
+ fragment: SqlFragment,
+ settings?: ClickHouseSettings
+ ): Promise<T[]> {
+ const { query, query_params } = fragment.toStatement();
+ if (!cache) {
+ return chQuery<T>(deps, fragment, settings);
+ }
+ const key = createHash('sha256')
+ .update(`${query}|${JSON.stringify(query_params)}|${settings?.session_timezone ?? 'UTC'}`)
+ .digest('hex');
+ const hit = cache.get(key);
+ if (hit !== undefined) {
+ return hit as T[];
+ }
+ const rows = await chQuery<T>(deps, fragment, settings);
+ cache.set(key, rows);
+ return rows;
+ };
+}
+```
+
+Three things that must hold, and are worth asserting in the task's test:
+
+1. **`query_params` is serialized deterministically.** `toStatement` assigns
+ `p1, p2, …` in render order from one counter, so the same fragment always
+ produces the same names in the same order and `JSON.stringify` is stable. Do
+ not sort the keys — two fragments that differ only in param *order* are
+ different queries.
+2. **The timezone stays in the key.** clix folded it in (`|${queryTimezone}`,
+ line 35) and it defaulted to `'UTC'` (line 24). Keep both, including the
+ default, or a cache built under one project's timezone serves another's.
+3. **The cache is per-request, not global** — `createCachedClix` takes the `Map`
+ from its caller and that does not change. A module-level cache would be a
+ cross-tenant read.
+
+## Traps
+
+Every one of these is a way to write a conversion that typechecks, passes
+locally, and is wrong.
+
+### 1. Date-shaped substrings — clix re-quoted them, binding cannot
+
+clix's `escapeDate` regex-replaced any `\d{4}-\d{2}-\d{2}` substring **anywhere**
+in a SELECT expression or JOIN condition with a quoted literal; `clix.exp()`
+existed to opt out. A bound param does not do this, and that is a **behaviour
+change per conversion, not a bug fix to celebrate silently**.
+
+Evidence: `packages/db/src/clickhouse/sql.test.ts:221-250`, the
+`describe('injection (ported from query-builder.test.ts)')` block — in
+particular the case at `:236` named "leaves a path containing a date substring
+intact (the escapeDate bug)", asserting
+`/blog/2024-03-17-supabase-activity-scheduler` arrives at
+`query_params.p1` verbatim.
+
+Consequence for a conversion: if a V1 expression relied on `escapeDate` to quote
+something for it, the V2 text must quote it explicitly. The side-by-side proof
+is what surfaces this — the two texts will differ visibly.
+
+### 2. `IN` vs `GLOBAL IN` — never rewritten, in either direction
+
+Read `/home/deploy/rewrite-openpanel/docs/ENVIRONMENT.md` before touching any
+query on a `Distributed` table. On one node the two are **indistinguishable**;
+production Cloud is 2 shards × 2 replicas, where a plain `IN (subquery)` runs
+per-shard against that shard's local data and silently returns wrong answers.
+`docs/ENVIRONMENT.md` calls it "the single most dangerous edit in this repo".
+
+**The rule for this wave: a conversion changes the binding of values and nothing
+about the distribution semantics.** If V1 wrote `IN`, V2 writes `IN`. If V1 wrote
+`GLOBAL IN`, V2 writes `GLOBAL IN`. Whether a given site *should* be `GLOBAL IN`
+is a separate, pre-existing question — `retention.sql.ts:14` and
+`sankey.sql.ts:14-16` are the precedent: the M7 conversions recorded the question
+in a comment and did not answer it. Do the same.
+
+The tag does not obstruct either form — `sql.test.ts:143` and
+`sql.clickhouse.test.ts:393,410` pin `GLOBAL IN`, `GLOBAL JOIN`, `PREWHERE`,
+`FINAL`, `LIMIT BY`, `WITH FILL`, `ARRAY JOIN` and a trailing `SETTINGS` clause
+(ADR-013 R4). `packages/db/src/clickhouse/client.ts:118` sets
+`distributed_product_mode: 'allow'` globally, which is what keeps today's plain
+joins from erroring.
+
+### 3. `session_timezone` — clix always sent `'UTC'`
+
+`clix(client, timezone)` defaults to `'UTC'` when no timezone is passed
+(`query-builder.ts:696-697`) and sends it as
+`clickhouse_settings.session_timezone` (`:562`). Almost every clix call site
+passes no timezone, so almost every clix query ran under `session_timezone =
+'UTC'` — including queries whose date expressions would otherwise have used the
+server default.
+
+A converted query that just drops the setting changes every `toStartOfDay`,
+`toDate` and `toDateTime` in it. The landed conversions pin it explicitly:
+
+```ts
+// clix always sent `session_timezone: 'UTC'`; the two queries converted from
+// clix keep sending it so their result sets stay identical.
+const CLIX_SESSION_TIMEZONE = { session_timezone: 'UTC' } as const;
+```
+
+— `packages/core/src/modules/session/session.service.ts:178-180`, and the
+byte-identical comment at `event.service.ts:94-96` and
+`profile.service.ts:66-68`. Copy that constant into any module you convert off
+clix. (`chart/src/run-query.ts:21` documents the other half: where a real
+project timezone was passed, it still travels as `session_timezone`.)
+
+Also, from the identifier matrix: a `SETTINGS` value **cannot** be a bound
+param, of any type. It goes through `chQuery`'s settings argument or nowhere.
+
+### 4. `rewriteProfilePropertyRefs` ran over finished text
+
+`packages/core/src/modules/chart/src/field-resolution.ts:457-467` rewrites
+`profile.properties['<key>']` → `` `profile.properties.<key>` `` by string
+`split`/`join` over the **whole finished query**. V1 could do that because
+everything was text.
+
+The reason it survives the conversion at all is stated at
+`compiled.ts:18-22`: bound params were never matched by it — an escaped literal
+cannot contain `['`, so the rewrite only ever hit generated column expressions,
+never values. That is why applying it per compiled piece before splicing
+(`compiledTextWithProfileRefs`, line 23) is provably the same rewrite.
+
+**The trap:** it is a textual pass. If a conversion moves a
+`profile.properties['x']` reference out of `field-resolution.ts`'s output and
+into a `sql` template's literal part, the rewrite will still match it — and if a
+conversion moves one into a *param*, it will silently stop matching. When
+converting the field resolver, the rewrite must be applied to the fragment's
+literal parts or folded into the resolver itself; it cannot be left as a pass
+over `toStatement().query`, because by then the params are gone and the CTE
+column names would no longer line up.
+
+### 5. Nullable columns and `null` values
+
+`sqlstring.escape(null)` produces the SQL keyword `NULL`. `sql.string(v)` with
+`v === null` produces `{p:String}` bound to `null`, which ClickHouse parses as
+the **empty string**. Measured 2026-09-07 against local ClickHouse 26.1.3.52:
+
+```
+SELECT {p1:String} AS v p1=null -> [{"v":""}]
+SELECT {p1:Nullable(String)} AS v p1=null -> [{"v":null}] isNull() = 1
+```
+
+So: any value whose type is `T | null | undefined` binds through
+`sql.nullable('T', v)`, not `sql.string`. If the V1 branch tested for null
+before escaping and emitted `IS NULL` / `= ''` instead (as `filter-where.ts:420-425`
+does for `isNull`/`isNotNull`), keep that branch — do not collapse it into a
+nullable param.
+
+### 6. Empty arrays
+
+`sql.array('String', [])` renders `{p:Array(String)}` bound to `[]`. Measured:
+ClickHouse accepts it and `n IN {p:Array(String)}` returns 0 rows, matching
+V1's literal `IN ()`, which it also accepts and which also returns 0. So an
+empty list is not a correctness hazard — but it is a *semantic* one wherever V1
+had an early return. `filter-where.ts:67-69` returns without emitting a clause
+when `cohortIds.length === 0`; converting that to `IN ${sql.array('String', [])}`
+would turn "no filter" into "match nothing". Preserve the guard.
+
+## Side-by-side proof
+
+M7 produced ten `*.proof.md` files by one method. This is that method, written
+as a procedure. It is not optional: `CLAUDE.md` requires that a replaced query
+be run against local ClickHouse with the old and new result sets shown side by
+side, and "looks equivalent" is explicitly not evidence.
+
+### Procedure
+
+1. **Get the PRE-change file.** `git show HEAD:<path> > /tmp/v1-<name>.ts`, or
+ `git worktree add /tmp/v1 HEAD` if the V1 code needs its imports to resolve.
+ Import V1 and V2 **side by side in one process** — that is what makes "the
+ same inputs" checkable rather than asserted. Do not retype the V1 text by
+ hand; generate it by calling the V1 function.
+2. **Execute both through one `@clickhouse/client`**, `format: 'JSON'`, the same
+ `session_timezone` for both (trap 3 — if V1 appended
+ `SETTINGS session_timezone = '<tz>'` to its text, pass that same value as
+ `clickhouse_settings.session_timezone` for both sides). V1 goes as `query`;
+ V2 goes as `query` + `query_params`.
+3. **Compare `data` row for row and `meta` column for column.** Compare as
+ **sets** where V1 has no total `ORDER BY` — `ORDER BY date ASC WITH FILL`
+ leaves rows sharing a `date` in arbitrary order, and three V1-vs-V1 runs of
+ the same case produced three different orders. State in the proof which cases
+ were compared as sets and why. Never sort away a difference you have not
+ explained.
+4. **Record row counts on both sides, for every case.** A case that returns
+ **zero rows proves nothing** — it proves the two texts are both syntactically
+ valid. Every case in a proof needs a non-zero row count, or an explicit note
+ saying which fixture would have produced one and why it does not exist here
+ (the `chart.sql.proof.md` cohort cases are the precedent: `cohort_members` is
+ empty in the prod-copy, so those cases are stated as 0-row statement
+ equivalence with positive-row coverage moved to `openpanel_test`).
+5. **A difference is either fixed or explained as a V1 defect reproduced
+ identically.** `IDENTICAL ERROR` — both sides failing with the same ClickHouse
+ error — is a passing case and must be listed. A difference you cannot explain
+ is a failed conversion.
+6. **Reproducibility.** Every statement in the proof must be runnable with
+ `curl 'http://127.0.0.1:8123/?database=openpanel'`, binding V2's params as
+ `param_pN=`. Write the params out.
+
+### Proof-file template
+
+Save as `<module>/src/<name>.sql.proof.md`. The four header lines are mandatory.
+
+````markdown
+# <name>.sql.ts — V1 → V2 result-set proof (<TASK-ID>)
+
+<One paragraph: which V1 file and commit sha, how both sides were executed,
+what was compared (`data` + `meta`), and which cases were compared as sets
+because V1 has no total ORDER BY.>
+
+- **Date**: YYYY-MM-DD.
+- **Data**: local prod-copy `openpanel` (<N> events). Projects: <ids and their
+ timezones>. Name every fixture that is empty here, and say where its
+ positive-row coverage lives instead.
+- **Machine**: single-node ClickHouse <version> on a <N>-vCPU box — timings are
+ directional only; production is 2 shards × 2 replicas
+ (`docs/ENVIRONMENT.md`). <One sentence on whether the statements contain
+ `IN (subquery)` on a Distributed table, and that no `IN`/`GLOBAL IN` was
+ changed.>
+- **Verdict**: N cases, all IDENTICAL.
+ <or: N cases, all IDENTICAL; M of the N are IDENTICAL ERROR — V1 defects
+ reproduced byte-for-byte, listed below, not fixed here.>
+
+## <case name>
+
+**statement** — IDENTICAL; rows V1/V2 = <n>/<n>; rows_read V1/V2 = <n>/<n>;
+wall V1/V2 = <n> ms / <n> ms; clickhouse_settings: session_timezone=<tz> (both).
+
+```sql
+-- V1
+<text>
+-- V2
+<text>
+-- V2 params: {"p1": …}
+```
+````
+
+`rows_read` differing between the two sides is normal and is a perf note, not a
+semantic one: ClickHouse's index analysis treats a bound `{p:String}` differently
+from an inline literal for a `created_at` range. Record it; do not chase it.
+
+## Worked example
+
+One real `sqlstring.escape()` site, converted and executed. Documentation only —
+`git diff --stat` for this task shows `tooling/gates/` and `docs/` and nothing
+else.
+
+**Site:** `packages/core/src/modules/chart/src/filter-where.ts:439-446`, the
+`contains` branch for a top-level column:
+
+```ts
+case 'contains': {
+ where[id] = `(${value
+ .map(
+ (val) =>
+ `${name} LIKE ${sqlstring.escape(`%${String(val).trim()}%`)}`
+ )
+ .join(' OR ')})`;
+ break;
+}
+```
+
+**V2** — idiom 11 for the `%` wrapping, idiom 13 for the column, idiom 14 for
+the disjunction:
+
+```ts
+const where = sql`(${sql.join(
+ value.map(
+ (val) =>
+ sql`${sql.id(name, [...EVENT_TOP_LEVEL_COLUMNS])} LIKE ${sql.string(`%${String(val).trim()}%`)}`
+ ),
+ ' OR '
+)})`;
+```
+
+**Inputs** — `filter = { name: 'path', operator: 'contains', value: ['/blog', "o'brien-%_"] }`,
+`projectId = 'secure-privacy'`, window `2026-07-01 00:00:00` … `2026-07-08 00:00:00`.
+The second value carries a quote (the escaping case) and a `%` and a `_` (the
+LIKE-metacharacter case, trap idiom 12).
+
+V1 text, produced by calling the real `getEventFiltersWhereClause`:
+
+```sql
+(path LIKE '%/blog%' OR path LIKE '%o\'brien-%_%')
+```
+
+V2 fragment, rendered:
+
+```sql
+(path LIKE {p1:String} OR path LIKE {p2:String})
+-- params: {"p1":"%/blog%","p2":"%o'brien-%_%"}
+```
+
+Spliced into the same surrounding count, both sides were executed against local
+prod-copy `openpanel` through one `@clickhouse/client` with
+`session_timezone: 'UTC'`, `format: 'JSON'`:
+
+```sql
+-- V1
+SELECT count() AS c, uniq(profile_id) AS u FROM events e
+WHERE project_id = 'secure-privacy'
+ AND created_at >= toDateTime('2026-07-01 00:00:00')
+ AND created_at < toDateTime('2026-07-08 00:00:00')
+ AND (path LIKE '%/blog%' OR path LIKE '%o\'brien-%_%')
+
+-- V2
+SELECT count() AS c, uniq(profile_id) AS u FROM events e
+WHERE project_id = {p1:String}
+ AND created_at >= toDateTime({p2:String})
+ AND created_at < toDateTime({p3:String})
+ AND (path LIKE {p4:String} OR path LIKE {p5:String})
+-- V2 params: {"p1":"secure-privacy","p2":"2026-07-01 00:00:00","p3":"2026-07-08 00:00:00","p4":"%/blog%","p5":"%o'brien-%_%"}
+```
+
+**statement** — IDENTICAL; rows V1/V2 = 1/1; matched events V1/V2 = **60795/60795**;
+distinct profiles V1/V2 = **19723/19723**; `meta` V1/V2 =
+`[{"name":"c","type":"UInt64"},{"name":"u","type":"UInt64"}]` on both sides;
+rows_read V1/V2 = 90057/90057; wall V1/V2 = 33 ms / 31 ms; clickhouse_settings:
+`session_timezone=UTC` (both).
+
+- **Date**: 2026-09-07. **Data**: local prod-copy `openpanel`, 319,850,223 events.
+- **Machine**: single-node ClickHouse 26.1.3.52 on this box — timings are
+ directional only; production is 2 shards × 2 replicas. The statement contains
+ no `IN (subquery)`, so no `IN`/`GLOBAL IN` question arises.
+- **Verdict**: 1 case, IDENTICAL.
+
+Reproduce either side with `curl`:
+
+```bash
+curl -s 'http://127.0.0.1:8123/?database=openpanel&session_timezone=UTC&default_format=JSONCompact' \
+ --data-binary "SELECT count() AS c, uniq(profile_id) AS u FROM events e WHERE project_id = 'secure-privacy' AND created_at >= toDateTime('2026-07-01 00:00:00') AND created_at < toDateTime('2026-07-08 00:00:00') AND (path LIKE '%/blog%' OR path LIKE '%o\\'brien-%_%')"
+
+curl -s -G 'http://127.0.0.1:8123/' \
+ --data-urlencode 'database=openpanel' --data-urlencode 'session_timezone=UTC' \
+ --data-urlencode 'default_format=JSONCompact' \
+ --data-urlencode 'param_p1=secure-privacy' \
+ --data-urlencode 'param_p2=2026-07-01 00:00:00' \
+ --data-urlencode 'param_p3=2026-07-08 00:00:00' \
+ --data-urlencode 'param_p4=%/blog%' \
+ --data-urlencode "param_p5=%o'brien-%_%" \
+ --data-urlencode "query=SELECT count() AS c, uniq(profile_id) AS u FROM events e WHERE project_id = {p1:String} AND created_at >= toDateTime({p2:String}) AND created_at < toDateTime({p3:String}) AND (path LIKE {p4:String} OR path LIKE {p5:String})"
+```
+
+Both return `[[60795, 19723]]`.
+
+Two things this one case already demonstrates, and which the eight conversion
+tasks should expect:
+
+- The quote in `o'brien` is handled on both sides — V1 by `sqlstring`'s
+ backslash escape, V2 by never putting the value in the text at all.
+- The `%` and `_` inside that value are LIKE wildcards on **both** sides. That
+ is trap 12: the conversion preserves the defect. `%o'brien-%_%` matches
+ nothing here, so its contribution to the count is zero on both sides — which
+ is why the case pairs it with `/blog`, whose 60,795 matching rows are what
+ makes the proof mean something.
+
+## Checklist per conversion task
+
+1. `bash tooling/gates/p12-grep-gates.sh --report | tail -3` — record the before.
+2. Convert. Values become params; identifiers become `sql.id`/`sql.identifier`;
+ structure becomes `sql.join`. No `IN`/`GLOBAL IN` change. Keep
+ `CLIX_SESSION_TIMEZONE` where clix implied it.
+3. Write the proof file from the template. Every case has a non-zero row count
+ or a stated reason.
+4. `bash tooling/gates/p12-grep-gates.sh --report | tail -3` — the numbers went
+ down, and nothing else moved.
+5. `pnpm run typecheck` and `cd packages/core && bun test`.
+6. When all three totals reach 0, delete `query-builder.ts`,
+ `query-builder.test.ts`, `sql-builder.ts` and their `packages/db/index.ts`
+ re-exports, drop `sqlstring` and `@types/sqlstring` from the three manifests,
+ add `sql.ts` to the barrel (the comment at `sql.ts:21-23` says it joins then),
+ and `--assert` exits 0.
diff --git a/tooling/gates/p12-grep-gates.sh b/tooling/gates/p12-grep-gates.sh
new file mode 100755
index 000000000..b70143e7d
--- /dev/null
+++ b/tooling/gates/p12-grep-gates.sh
@@ -0,0 +1,195 @@
+#!/usr/bin/env bash
+#
+# P12 grep gates — ADR-013 decision 21: sqlstring = 0, clix = 0, no builder.
+#
+# --report per-file counts + a TOTAL line; always exits 0.
+# --assert exits 1 while anything is left; exits 0 only when all three
+# totals are 0 AND the three definer files are gone.
+#
+# Columns
+# sqlstring every mention of the `sqlstring` identifier: `import`,
+# `require`, `sqlstring.escape(...)` calls, and the
+# `"sqlstring"` / `"@types/sqlstring"` package.json entries.
+# clix `clix(`, `clix as`, `createCachedClix`, `clixCached`, and any
+# import/require/export whose specifier ends in `query-builder`.
+# sql-builder `createSqlBuilder`, and any import/require/export whose
+# specifier ends in `sql-builder`.
+#
+# Scanned: packages/*/src/**, packages/*/index.ts, apps/*/src/**, and every
+# workspace package.json — source and manifests only (.ts .tsx .mts .cts .js
+# .jsx .mjs .cjs .json), which keeps generated binaries and the `*.proof.md`
+# result-set records out of a count that is supposed to reach zero. Never
+# node_modules/, dist/, .git/, or the three files that DEFINE what the gate is
+# counting (they are the deletion target, not a violation) —
+# packages/db/src/clickhouse/query-builder.ts, its .test.ts, and
+# packages/db/src/sql-builder.ts.
+#
+# COMMENT-ONLY MENTIONS DO NOT COUNT. Each line is passed through a small lexer
+# that removes `/* ... */` (across lines) and `// ...` to end of line before the
+# patterns are applied, so `// clix is dead` or a JSDoc paragraph naming
+# `createSqlBuilder` scores 0. Two deliberate limits, both stated so the number
+# is not read as more than it is: `://` is never treated as a comment (URLs
+# survive), and the lexer does not track string literals, so a `//` inside a
+# string truncates the rest of that line. Neither can turn a comment into a hit;
+# both could in principle hide one, which is why --assert also requires the
+# three definer files to be absent rather than trusting the count alone. JSON
+# has no comments, so package.json is never passed through the lexer — an
+# exports-map entry like "./modules/*.constants" would otherwise open a block
+# comment that swallows the dependency list.
+#
+# A count is matching LINES, not occurrences: one line with two
+# `sqlstring.escape(...)` calls scores 1. The gate's target is 0, so the
+# distinction only affects how big the number looks on the way down.
+#
+# Output is deterministic — no sha, no timestamp — so two runs diff cleanly.
+
+set -uo pipefail
+
+REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
+cd "$REPO_ROOT" || exit 1
+
+readonly DEFINER_FILES=(
+ 'packages/db/src/clickhouse/query-builder.ts'
+ 'packages/db/src/clickhouse/query-builder.test.ts'
+ 'packages/db/src/sql-builder.ts'
+)
+
+readonly SCANNED_EXTENSIONS='\.(ts|tsx|mts|cts|js|jsx|mjs|cjs|json)$'
+
+readonly COLUMN_WIDTH=12
+readonly TABLE_RULE_WIDTH=76
+
+usage() {
+ echo "usage: $(basename "$0") --report | --assert" >&2
+}
+
+# packages/*/src, packages/*/index.ts, apps/*/src, every workspace package.json.
+collect_files() {
+ {
+ find packages apps -type d \( -name node_modules -o -name dist -o -name .git \) -prune \
+ -o -type f -path '*/src/*' -print || true
+ find packages -mindepth 2 -maxdepth 3 -type f -name index.ts -print || true
+ # docker/data holds the local Postgres/ClickHouse volumes and is unreadable.
+ find . -type d \( -name node_modules -o -name dist -o -name .git -o -path './docker/data' \) -prune \
+ -o -type f -name package.json -print || true
+ } 2>/dev/null |
+ sed 's|^\./||' |
+ grep -E "$SCANNED_EXTENSIONS" |
+ grep -v -e '/node_modules/' -e '^node_modules/' \
+ -e '/dist/' -e '^dist/' -e '/\.git/' -e '^\.git/' |
+ grep -v -x -F -e "${DEFINER_FILES[0]}" -e "${DEFINER_FILES[1]}" -e "${DEFINER_FILES[2]}" |
+ sort -u
+}
+
+# Emits "<file>\t<line-number>\t<column>\t<stripped line>" for every matching
+# line, once per column it matches. One awk pass over every file.
+scan() {
+ collect_files | tr '\n' '\0' | xargs -0 -r awk '
+ FNR == 1 { inblock = 0; is_json = (FILENAME ~ /\.json$/) }
+
+ {
+ line = is_json ? $0 : strip($0)
+ if (line ~ /(^|[^A-Za-z0-9_$])sqlstring([^A-Za-z0-9_$]|$)/)
+ emit("sqlstring", line)
+ if (line ~ /(^|[^A-Za-z0-9_$])clix\(/ ||
+ line ~ /(^|[^A-Za-z0-9_$])clix[[:space:]]+as[[:space:]]/ ||
+ line ~ /createCachedClix/ ||
+ line ~ /clixCached/ ||
+ line ~ /(from|import|require)[[:space:]]*\(?[[:space:]]*["'"'"'][^"'"'"']*query-builder["'"'"']/)
+ emit("clix", line)
+ if (line ~ /createSqlBuilder/ ||
+ line ~ /(from|import|require)[[:space:]]*\(?[[:space:]]*["'"'"'][^"'"'"']*sql-builder["'"'"']/)
+ emit("sql-builder", line)
+ }
+
+ function emit(column, text) {
+ sub(/^[[:space:]]+/, "", text)
+ sub(/[[:space:]]+$/, "", text)
+ printf "%s\t%d\t%s\t%s\n", FILENAME, FNR, column, text
+ }
+
+ # Removes block and line comments. See the header for its two known limits.
+ function strip(line, out, i, n, one, two, prev) {
+ out = ""; i = 1; n = length(line)
+ while (i <= n) {
+ one = substr(line, i, 1); two = substr(line, i, 2)
+ if (inblock) {
+ if (two == "*/") { inblock = 0; i += 2 } else { i += 1 }
+ continue
+ }
+ if (two == "/*") { inblock = 1; i += 2; continue }
+ if (two == "//") {
+ prev = (i > 1) ? substr(line, i - 1, 1) : ""
+ if (prev != ":") break
+ out = out two; i += 2; continue
+ }
+ out = out one; i += 1
+ }
+ return out
+ }
+ '
+}
+
+print_report() {
+ scan | awk -v w="$COLUMN_WIDTH" -v rule="$TABLE_RULE_WIDTH" '
+ { count[$1 "\t" $3]++; files[$1] = 1; total[$3]++ }
+ END {
+ printf "%*s%*s%*s %s\n", w, "sqlstring", w, "clix", w, "sql-builder", "file"
+ for (i = 0; i < rule; i++) printf "-"
+ printf "\n"
+ n = 0
+ for (f in files) ordered[++n] = f
+ asort_paths(ordered, n)
+ for (i = 1; i <= n; i++) {
+ f = ordered[i]
+ printf "%*d%*d%*d %s\n", w, count[f "\tsqlstring"] + 0, \
+ w, count[f "\tclix"] + 0, w, count[f "\tsql-builder"] + 0, f
+ }
+ for (i = 0; i < rule; i++) printf "-"
+ printf "\n"
+ printf "%*d%*d%*d %s\n", w, total["sqlstring"] + 0, w, total["clix"] + 0, \
+ w, total["sql-builder"] + 0, "TOTAL"
+ }
+ function asort_paths(a, len, i, j, tmp) {
+ for (i = 2; i <= len; i++) {
+ tmp = a[i]; j = i - 1
+ while (j > 0 && a[j] > tmp) { a[j + 1] = a[j]; j-- }
+ a[j + 1] = tmp
+ }
+ }
+ '
+}
+
+assert_clean() {
+ local violations=0
+ local hits
+ hits="$(scan)"
+
+ if [[ -n "$hits" ]]; then
+ echo "FAIL: builder / sqlstring references remain (ADR-013 decision 21 wants 0)."
+ echo "$hits" | sort | awk -F'\t' '{ printf " [%s] %s:%s: %s\n", $3, $1, $2, $4 }'
+ violations=1
+ fi
+
+ for definer in "${DEFINER_FILES[@]}"; do
+ if [[ -e "$definer" ]]; then
+ echo "FAIL: definer file still exists: $definer"
+ violations=1
+ fi
+ done
+
+ if [[ "$violations" -ne 0 ]]; then
+ echo
+ print_report
+ return 1
+ fi
+
+ echo "OK: sqlstring = 0, clix = 0, sql-builder = 0; all three definer files are gone."
+ return 0
+}
+
+case "${1-}" in
+ --report) print_report ;;
+ --assert) assert_clean || exit 1 ;;
+ *) usage; exit 2 ;;
+esac