Skip to content

πŸͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

feat(query-ast): clean Kysely base for the ClickHouse query AST (#16687)

Niklas Semmler8521ece6c8d5
diff --git a/.codespellrc b/.codespellrc
index 9d7fda74e2..6fcafd986c 100644
--- a/.codespellrc
+++ b/.codespellrc
@@ -1,4 +1,4 @@
[codespell]
skip = .git,*.pdf,*.svg,package-lock.json,*.prisma,pnpm-lock.yaml,./worker/src/__tests__/chatml/framework-traces
-ignore-words-list = afterall,vertx,notIn,alue,allTime,rouge
+ignore-words-list = afterall,vertx,notIn,alue,allTime,rouge,froms
diff --git a/packages/shared/AGENTS.md b/packages/shared/AGENTS.md
index 13ed785621..d18d5e3ad5 100644
--- a/packages/shared/AGENTS.md
+++ b/packages/shared/AGENTS.md
@@ -28,7 +28,10 @@
recording/diff harness that captures the current SQL at the
`src/server/repositories/clickhouse.ts` exec seam and normalizes it via
`clickhouse format` for snapshot comparison. Every migrated call site is
- proven against its baseline here.
+ proven against its baseline here. The Kysely ClickHouse dialect (ARRAY JOIN /
+ LIMIT BY / metadata indexOf nodes, `ExecutionContext` tenancy injection,
+ typed selection, virtual views, catalog parity) lives under
+ `src/server/query-ast/kysely/`.
- Postgres schema: `prisma/schema.prisma`
- Prisma migrations: `prisma/migrations/*`
- ClickHouse migrations: `clickhouse/migrations/{clustered,unclustered}/*`
diff --git a/packages/shared/package.json b/packages/shared/package.json
index d28ce20a24..4bccbbe9f8 100644
--- a/packages/shared/package.json
+++ b/packages/shared/package.json
@@ -193,6 +193,7 @@
"ioredis": "^5.10.1",
"ipaddr.js": "^2.2.0",
"jsonpath-plus": "10.3.0",
+ "kysely": "0.28.17",
"langfuse": "3.38.4",
"lodash": "^4.18.1",
"lossless-json": "^4.1.1",
diff --git a/packages/shared/src/server/query-ast/README.md b/packages/shared/src/server/query-ast/README.md
index 9860d76e2d..e1e57352d0 100644
--- a/packages/shared/src/server/query-ast/README.md
+++ b/packages/shared/src/server/query-ast/README.md
@@ -9,16 +9,21 @@ sugar, semantic field metadata, `FilterState` embedding) will land under
## What's here today
-Only the golden-SQL harness β€” the thing every later migration is tested
-against:
-
- `goldenHarness.ts` β€” captures SQL at the `repositories/clickhouse.ts` exec
seam in test mode, then normalizes it (`clickhouse format` + positional param
names) for snapshot comparison. No ClickHouse server needed.
-- `environments.golden.test.ts` β€” baselines the current SQL shapes of
- `environments.getEnvironmentsForProject` (both write-mode branches Γ— the
- timestamp bound). This is the tier-0 exit target for the spike: once
- `environments.ts` compiles from the AST, it must reproduce these snapshots.
+- `environments.golden.test.ts` β€” baselines `environments.getEnvironmentsForProject`
+ (both write-mode branches Γ— the timestamp bound).
+- `kysely/` β€” compile-only ClickHouse dialect on Kysely 0.28. Real
+ `OperationNode`s for ARRAY JOIN, LIMIT BY, and metadata `indexOf`
+ subscripts; a mandatory tenancy injection pass keyed on `ExecutionContext`;
+ schema-typed selection (TS + runtime validation); virtual views as WITH
+ CTEs; catalog parity. Library-specific code lives only in this folder.
+- `kysely/schema.ts` β€” the physical table registry: one `defineTable`
+ declaration per relation drives all three downstream views β€” the Kysely row
+ types (`ClickHouseDatabase`), the runtime column-type map the type-check pass
+ consults (`COLUMN_DATA_TYPES`), and the set the tenancy pass scopes
+ (`TENANTED_TABLES`) β€” instead of three hand-maintained tables that drift.
Regenerate baselines with `-u` after an intentional SQL change:
@@ -41,3 +46,51 @@ against the new binary in the same PR, or the golden tests drift.
The CI step is intentionally **non-blocking**: a drift surfaces as a warning
annotation but never fails the pipeline (`|| echo "::warning::"`). Promote it to
a required check once it has proven stable.
+
+## Using it
+
+- **Compile only through `compileClickhouseQuery(query, ctx)`.** It is the one
+ supported path from a builder to `{ sql, params }`; the repository layer hands
+ that to `queryClickhouse`. `ctx` (an `ExecutionContext` carrying `projectId`)
+ is required β€” omitting it is a compile-time type error and an empty one throws.
+- **Never filter `project_id` yourself.** The compile step injects
+ `project_id = {projectId}` into every tenanted relation, so call sites pass
+ only `{ projectId }` (see `repositories/environments.ts`).
+- **ClickHouse-only clauses use `$call(helper())`** β€” not fluent builder
+ methods, so they compose inside CTEs, subqueries, and views. See the recipes
+ below.
+
+### Recipes
+
+**Write an ARRAY JOIN** (`arrayJoin` + `mapKeys`/`mapValues` from `./kysely/extensions`):
+
+```ts
+db.selectFrom("observations")
+ .select("environment")
+ .$call(arrayJoin({ cost_key: mapKeys("cost_details"), cost: mapValues("cost_details") }));
+// … array join mapKeys(cost_details) as cost_key, mapValues(cost_details) as cost
+```
+
+**Write a LIMIT BY** (`limitBy`):
+
+```ts
+db.selectFrom("events_core")
+ .select(["span_id", "project_id"])
+ .orderBy("event_ts", "desc")
+ .$call(limitBy({ count: 1, columns: ["span_id", "project_id"] }));
+// … order by event_ts desc limit 1 by span_id, project_id
+```
+
+**Select a metadata value** (`metadataValue` β€” lowers `metadata[key]` to a bound `indexOf` subscript):
+
+```ts
+db.selectFrom("events_core as e")
+ .select((eb) => [metadataValue("e", "my_key").as("my_val")])
+ .where((eb) => eb(metadataValue("e", "my_key"), ">", 2));
+// select metadata_values[indexof(e.metadata_names, {p:String})] as my_val …
+```
+
+For how ARRAY JOIN / LIMIT BY / metadata nodes are implemented without forking
+Kysely, how tenancy injection works internally, the escape hatches, and the
+Kysely-internals upgrade hazard, see the developer guide in
+[`kysely/README.md`](kysely/README.md).
diff --git a/packages/shared/src/server/query-ast/executionContext.ts b/packages/shared/src/server/query-ast/executionContext.ts
new file mode 100644
index 0000000000..7f00f76053
--- /dev/null
+++ b/packages/shared/src/server/query-ast/executionContext.ts
@@ -0,0 +1,8 @@
+/**
+ * Compile-time tenancy scope. Every ClickHouse query compiled through
+ * {@link compileClickhouseQuery} must carry one of these; the tenancy
+ * injection pass keys off `projectId`.
+ */
+export type ExecutionContext = {
+ projectId: string;
+};
diff --git a/packages/shared/src/server/query-ast/goldenHarness.ts b/packages/shared/src/server/query-ast/goldenHarness.ts
index 62af4d9e08..2412481fa8 100644
--- a/packages/shared/src/server/query-ast/goldenHarness.ts
+++ b/packages/shared/src/server/query-ast/goldenHarness.ts
@@ -1,4 +1,6 @@
import { spawnSync } from "child_process";
+import { homedir } from "os";
+import { join } from "path";
import { Readable } from "stream";
import type * as ClickhouseModule from "../repositories/clickhouse";
@@ -134,8 +136,31 @@ export type GoldenQuery = {
};
// The `clickhouse` multi-binary's `format` subcommand. Not `clickhouse-local`,
-// which forces local mode and won't accept `format` as a subcommand.
-const clickhouseBin = "clickhouse";
+// which forces local mode and won't accept `format` as a subcommand. Prefer
+// CLICKHOUSE_BIN, then PATH, then a user-local install β€” the PATH binary in
+// some agent VMs is a docker wrapper that cannot run `format`.
+function resolveClickhouseBin(): string {
+ const candidates = [
+ process.env.CLICKHOUSE_BIN,
+ "clickhouse",
+ join(homedir(), ".local/bin/clickhouse"),
+ ].filter((bin): bin is string => Boolean(bin));
+
+ for (const bin of candidates) {
+ try {
+ const res = spawnSync(bin, ["format"], {
+ input: "SELECT 1",
+ encoding: "utf8",
+ });
+ if (res.status === 0) return bin;
+ } catch {
+ // try the next candidate
+ }
+ }
+ return "clickhouse";
+}
+
+const clickhouseBin = resolveClickhouseBin();
let formatAvailability: boolean | undefined;
@@ -164,7 +189,7 @@ export function clickhouseFormatAvailable(): boolean {
* which `clickhouse-local` provides β€” no server, no state. Throws on a parse
* error so malformed SQL fails the test instead of comparing raw.
*/
-function formatSql(sql: string): string {
+export function formatSql(sql: string): string {
const res = spawnSync(clickhouseBin, ["format"], {
input: sql,
encoding: "utf8",
@@ -179,6 +204,78 @@ function formatSql(sql: string): string {
return res.stdout.trimEnd();
}
+let localAvailability: boolean | undefined;
+
+/** Whether `clickhouse local` can execute a query on this machine. */
+export function clickhouseLocalAvailable(): boolean {
+ if (localAvailability !== undefined) return localAvailability;
+ try {
+ const res = spawnSync(clickhouseBin, ["local", "--query", "SELECT 1"], {
+ encoding: "utf8",
+ });
+ localAvailability = res.status === 0;
+ } catch {
+ localAvailability = false;
+ }
+ return localAvailability;
+}
+
+const NAMED_PARAM = /\{([A-Za-z_][A-Za-z0-9_]*)\s*:\s*([^}]+)\}/g;
+
+function sqlLiteral(value: unknown, type: string): string {
+ const t = type.trim();
+ if (t.startsWith("Array")) {
+ if (!Array.isArray(value)) {
+ throw new Error(`Expected array bind for ${t}, got ${typeof value}`);
+ }
+ return `[${value.map((item) => sqlLiteral(item, "String")).join(", ")}]`;
+ }
+ if (t.startsWith("DateTime")) {
+ const iso =
+ value instanceof Date
+ ? value.toISOString().replace("T", " ").replace("Z", "")
+ : String(value);
+ return `'${iso}'`;
+ }
+ if (typeof value === "number" || typeof value === "bigint") {
+ return String(value);
+ }
+ if (typeof value === "boolean") return value ? "1" : "0";
+ const escaped = String(value).replace(/\\/g, "\\\\").replace(/'/g, "\\'");
+ return `'${escaped}'`;
+}
+
+/** Replace `{name:Type}` placeholders with ClickHouse literals for local exec. */
+export function substituteNamedParams(
+ sql: string,
+ params: Record<string, unknown>,
+): string {
+ return sql.replace(NAMED_PARAM, (_match, name: string, type: string) => {
+ if (!(name in params)) {
+ throw new Error(`Missing bind '${name}' for local execution`);
+ }
+ return sqlLiteral(params[name], type);
+ });
+}
+
+/**
+ * Run SQL through `clickhouse local` (no server). Throws on a non-zero status
+ * so an analyzer/parse error fails the test instead of comparing stdout.
+ */
+export function executeClickhouseLocal(sql: string): string {
+ const res = spawnSync(clickhouseBin, ["local", "--query", sql], {
+ encoding: "utf8",
+ maxBuffer: 16 * 1024 * 1024,
+ });
+ if (res.error) throw res.error;
+ if (res.status !== 0) {
+ throw new Error(
+ `clickhouse local failed (status ${res.status}): ${res.stderr?.trim()}\n--- SQL ---\n${sql}`,
+ );
+ }
+ return (res.stdout ?? "").trimEnd();
+}
+
const PARAM_PLACEHOLDER = /\{\s*([A-Za-z_][A-Za-z0-9_]*)\s*:/g;
/**
diff --git a/packages/shared/src/server/query-ast/kysely/README.md b/packages/shared/src/server/query-ast/kysely/README.md
new file mode 100644
index 0000000000..148a395afd
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/README.md
@@ -0,0 +1,143 @@
+# kysely/ β€” compile-only ClickHouse dialect: patterns & gotchas
+
+This folder adapts Kysely to emit ClickHouse SQL. Several patterns here are
+deliberately unusual β€” this is the guide for anyone **developing** the query
+builder. For how to **use** it from a call site, see [`../README.md`](../README.md).
+
+## How ClickHouse clauses land (no fork)
+
+Kysely's `OperationNodeKind` union is closed, so a first-class node kind would
+need a fork. Instead each ClickHouse-only construct rides as an extra field or a
+special-cased node, and the compiler/transformer are overridden to emit and
+preserve it:
+
+| Clause | How it lands | Fork? |
+| ------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----- |
+| ARRAY JOIN | Plugin attaches `ArrayJoinNode` as an extra field on `SelectQueryNode`. `ClickHouseOperationNodeTransformer` preserves it. `ClickHouseQueryCompiler.visitSelectQuery` emits it after JOINs / before WHERE. | no |
+| LIMIT BY | Plugin attaches `LimitByNode` the same way. Compiler emits it after ORDER BY / before LIMIT. | no |
+| metadata `indexOf` | Helper builds an `ArrayIndexNode` whose index child is a `FunctionNode` (`indexOf`) over a bound `ValueNode` key. Transformer + compiler special-case the node. No plugin. | no |
+| Virtual view | Plugin rewrites `selectFrom(viewName)` into a WITH CTE. Outer types only expose the view's selected columns. | no |
+
+These are real node objects whose children are traced Kysely
+`FunctionNode`/`ColumnNode`/`ValueNode`/`IdentifierNode` values β€” not `RawNode`
+string splices. Any plugin using the default `OperationNodeTransformer` would
+drop them, so ours overrides `transformSelectQuery` to keep them. Upstream is
+not patched.
+
+## Kysely never runs β€” it only compiles
+
+`getClickhouseKysely()` returns a `Kysely` instance wired to a `DummyDriver`
+with SQLite adapter/introspector stand-ins (`dialect.ts`). Nothing ever
+connects or executes. The only supported output path is
+`compileClickhouseQuery(query, ctx)` β†’ `{ sql, params }`, which the repository
+layer hands to `queryClickhouse`. Do **not** call `.execute()` / `.compile()`
+directly: the compiler refuses any tree that did not go through the tenancy
+pass (see `../README.md`).
+
+## Tenancy injection β€” how the choke point works
+
+`compileClickhouseQuery(query, ctx)` is the only supported compile path, and it
+is where tenancy is enforced:
+
+1. A missing/empty `ExecutionContext` throws (`QueryCompileError`) β€” `ctx` is a
+ required parameter, so omitting it is also a compile-time type error.
+2. `TenancyInjectionPlugin` walks every FROM/JOIN and injects
+ `project_id = {projectId}` on each tenanted physical table, unless the tree
+ already carries a predicate that _proves_ that scope: the equality's value
+ must equal the context project, and β€” when more than one tenanted relation is
+ in scope β€” the column must be table-qualified. A qualified predicate covers a
+ relation only when the qualifier matches its alias (if aliased) or its table
+ name (if not) β€” so `scores AS traces` joined to `traces AS t` still scopes
+ both. It then identity-stamps the tree (`WeakSet`); a copied
+ `langfuseTenancy` property is not a valid stamp.
+3. `ClickHouseQueryCompiler` refuses to emit SQL unless that identity stamp is
+ present, so `qb.compile()` without the plugin also fails.
+4. Raw-SQL table sources (`selectFrom(sql\`...\`)`) and raw fragments embedding a
+ `SELECT`/`FROM`/`JOIN` in SELECT/WHERE throw `UnscopedRelationError`. Kysely's
+ own keyword fragments (`asc`/`desc`) are not relations.
+
+So query bodies here never filter `project_id` by hand β€” it is redundant, and
+forgetting it is impossible. Call sites like `repositories/environments.ts` pass
+only `{ projectId }`.
+
+## ClickHouse-only clauses use `$call(helper())`, not builder methods
+
+ARRAY JOIN and LIMIT BY are applied through curried helpers, invoked with
+Kysely's public `$call`:
+
+```ts
+db.selectFrom("observations")
+ .select("environment")
+ .$call(arrayJoin({ cost_key: mapKeys("cost_details"), cost: mapValues("cost_details") }))
+
+db.selectFrom("events_core")
+ .select(["span_id", "project_id"])
+ .orderBy("event_ts", "desc")
+ .$call(limitBy({ count: 1, columns: ["span_id", "project_id"] }))
+```
+
+**Why `$call(...)` and not a fluent `.arrayJoin(...)` method:** a real method
+would have to exist on _every_ builder instance, including the native Kysely
+builders that `.with((qb) => …)`, subquery, and `defineView` callbacks hand you.
+Getting that requires either forking Kysely's whole builder graph or globally
+mutating its prototype via an internal `kysely/dist/...` import (blocked here by
+Kysely's `exports` map under NodeNext). A curried helper is a plain function of
+a builder, so it works in any of those positions β€” which is exactly why ARRAY
+JOIN / LIMIT BY compose inside CTEs, subqueries, and views. `composition.test.ts`
+locks that property in.
+
+**`arrayJoin` widens the row type.** Each `{ alias: arrayExpr }` entry is added
+to the builder's output row, so an outer query over a CTE body can reference the
+produced column and a typo on the alias is a compile error. The element _value_
+type is `unknown`: Kysely's `Expression<T>` hides its type argument from
+inference. Precise value types would need a branded array-expression wrapper on
+`mapKeys` / `mapValues` / etc. β€” not done yet. `types.assert.ts` pins the
+widening behavior.
+
+**`arrayJoin` (clause) β‰  `arrayJoin()` (function).** ClickHouse has both. The
+helper here builds the ARRAY JOIN _clause_. The row-expanding SELECT _function_
+is just `eb.fn("arrayJoin", [...])`.
+
+## Escape hatches and their cost
+
+- `sql.ref("alias")` β€” the only way to reference a SELECT alias that is not a
+ schema column (ClickHouse allows `GROUP BY`/expression reuse of aliases;
+ Kysely's types do not model it). It is fully untyped β€” a typo reaches
+ ClickHouse unchecked. Use sparingly.
+- `eb.fn("ch_function", [...])` β€” arbitrary ClickHouse functions. The name is an
+ unchecked string and the return type defaults to `unknown`; no arity or
+ return checking.
+- New columns β€” add them to the table registry in `schema.ts` (one `defineTable`
+ entry per relation) as queries need them. That single declaration drives the
+ row types, the runtime column-type map, and the tenanted-table set.
+
+## The one Kysely-internals coupling (upgrade hazard)
+
+`compiler.ts` wraps Kysely's **private** `visitNode` / `nodeStack` to dispatch
+`ArrayIndexNode` (for `metadata[key]`), because it is not one of Kysely's closed
+`OperationNode` kinds. Kysely is pinned to **0.28.17** for this reason. Re-verify
+this hack on any Kysely bump; it is the single place that reaches past Kysely's
+documented surface. Everything else (plugins, dialect, transformer overrides)
+uses public or documented-protected API.
+
+## Types are asserted at compile time
+
+`types.assert.ts` holds `tsc`-only assertions (schema typing, view opacity,
+arrayJoin widening, limitBy preservation). It is never run; `schema.test.ts`
+anchors it so it stays in the build graph. `@ts-expect-error` lines there must
+stay live.
+
+## Verifying changes
+
+```
+CLICKHOUSE_BIN=clickhouse pnpm --filter @langfuse/shared run test src/server/query-ast
+```
+
+`catalog.golden.test.ts` and the other `*.golden.test.ts` suites assert
+`compile(AST) ≑ referenceSQL` after `clickhouse format`, so they need a local
+`clickhouse` binary and otherwise `describe.skip`. CI's non-blocking
+SQL-equivalence step installs the pinned binary and runs every `*.golden.test.ts`
+(it selects them by that name), so a suite must carry the `.golden.test.ts`
+suffix to run there. `composition.test.ts` asserts on raw compiler output and
+runs everywhere. After an intentional SQL change, regenerate golden baselines
+with `-u` (see `../README.md`).
diff --git a/packages/shared/src/server/query-ast/kysely/catalog.golden.test.ts b/packages/shared/src/server/query-ast/kysely/catalog.golden.test.ts
new file mode 100644
index 0000000000..cf83074d08
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/catalog.golden.test.ts
@@ -0,0 +1,100 @@
+import { describe, expect, it } from "vitest";
+
+import {
+ clickhouseFormatAvailable,
+ clickhouseLocalAvailable,
+ executeClickhouseLocal,
+ formatSql,
+ normalizeParams,
+ substituteNamedParams,
+} from "../goldenHarness";
+import { CATALOG, CATALOG_PROJECT_ID } from "./catalog";
+import { compileClickhouseQuery } from "./compile";
+
+const describeWithClickhouse = clickhouseFormatAvailable()
+ ? describe
+ : describe.skip;
+
+const describeWithClickhouseLocal = clickhouseLocalAvailable()
+ ? describe
+ : describe.skip;
+
+if (!clickhouseFormatAvailable()) {
+ console.warn(
+ "[catalog] `clickhouse format` unavailable β€” skipping catalog parity tests.",
+ );
+}
+
+if (!clickhouseLocalAvailable()) {
+ console.warn(
+ "[catalog] `clickhouse local` unavailable β€” skipping JOIN execution tests.",
+ );
+}
+
+describeWithClickhouse("catalog parity", () => {
+ for (const entry of CATALOG) {
+ it(`${entry.id} (tier ${entry.tier}) compile(AST) ≑ referenceSQL`, () => {
+ const compiled = compileClickhouseQuery(entry.build(), {
+ projectId: CATALOG_PROJECT_ID,
+ });
+
+ const left = normalizeParams(formatSql(compiled.sql), compiled.params);
+ const right = normalizeParams(
+ formatSql(entry.referenceSql),
+ compiled.params,
+ );
+
+ expect(left.sql).toBe(right.sql);
+ });
+ }
+});
+
+const JOIN_MEMORY_DDL = `
+ CREATE TABLE observations (
+ environment String,
+ project_id String,
+ start_time DateTime64(3),
+ trace_id String,
+ cost_details Map(String, Float64),
+ usage_details Map(String, Float64)
+ ) ENGINE = Memory;
+ CREATE TABLE traces (
+ environment String,
+ project_id String,
+ timestamp DateTime64(3),
+ id String
+ ) ENGINE = Memory;
+`;
+
+describeWithClickhouseLocal("catalog JOIN execution", () => {
+ it("join_observations_traces is valid ClickHouse with table-qualified project_id on both sides", () => {
+ const entry = CATALOG.find((e) => e.id === "join_observations_traces");
+ expect(entry).toBeDefined();
+ if (!entry) return;
+
+ const compiled = compileClickhouseQuery(entry.build(), {
+ projectId: CATALOG_PROJECT_ID,
+ });
+ const refs = compiled.sql.match(/[\w.]*project_id/gi) ?? [];
+ expect(refs.length).toBeGreaterThanOrEqual(3);
+ for (const ref of refs) {
+ expect(ref).toMatch(/^(o|t)\.project_id$/i);
+ }
+
+ const executable = substituteNamedParams(compiled.sql, compiled.params);
+ executeClickhouseLocal(`${JOIN_MEMORY_DDL}\n${executable}`);
+
+ const explained = executeClickhouseLocal(
+ `${JOIN_MEMORY_DDL}\nEXPLAIN QUERY TREE ${executable}`,
+ );
+ expect(explained).toMatch(/table_name: default\.observations/);
+ expect(explained).toMatch(/table_name: default\.traces/);
+ // Injected t.project_id and WHERE o.project_id resolve to different tables.
+ expect(explained).toMatch(
+ /column_name: project_id, result_type: String, source_id: 3/,
+ );
+ expect(explained).toMatch(
+ /column_name: project_id, result_type: String, source_id: 5/,
+ );
+ });
+});
diff --git a/packages/shared/src/server/query-ast/kysely/catalog.ts b/packages/shared/src/server/query-ast/kysely/catalog.ts
new file mode 100644
index 0000000000..f9c9ad5ebd
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/catalog.ts
@@ -0,0 +1,217 @@
+import type { ClickhouseCompilable } from "./compile";
+import { getClickhouseKysely } from "./dialect";
+import { arrayJoin, limitBy, mapKeys, mapValues } from "./extensions";
+
+type CatalogTier = 0 | 1 | 2 | 3 | 4;
+
+export type CatalogEntry = {
+ id: string;
+ tier: CatalogTier;
+ /** Hand-written reference SQL the compiler must match after `clickhouse format`. */
+ referenceSql: string;
+ build: () => ClickhouseCompilable;
+};
+
+const PROJECT_ID = "golden-project";
+const FROM_TS = new Date("2026-01-01T00:00:00.000Z");
+
+function db() {
+ return getClickhouseKysely();
+}
+
+/**
+ * Catalog of reference query shapes, one entry per complexity tier.
+ * Reference SQL is the intended ClickHouse shape, not a dump of Kysely's
+ * first draft β€” `compile(AST) ≑ referenceSQL` after `clickhouse format`.
+ *
+ * No entry filters `project_id`: the reference SQL shows the `project_id`
+ * predicate the mandatory tenancy pass injects, so these also prove injection.
+ */
+export const CATALOG: CatalogEntry[] = [
+ {
+ id: "environments_events_core",
+ tier: 0,
+ referenceSql: `
+ SELECT DISTINCT environment
+ FROM events_core
+ WHERE project_id = {p1:String}
+ `,
+ build: () =>
+ db().selectFrom("events_core").select("environment").distinct(),
+ },
+ {
+ id: "environments_legacy_union",
+ tier: 0,
+ referenceSql: `
+ SELECT DISTINCT environment
+ FROM traces
+ WHERE project_id = {p1:String}
+ UNION ALL
+ SELECT DISTINCT environment
+ FROM observations
+ WHERE project_id = {p1:String}
+ `,
+ build: () =>
+ db()
+ .selectFrom("traces")
+ .select("environment")
+ .distinct()
+ .unionAll(
+ db().selectFrom("observations").select("environment").distinct(),
+ ),
+ },
+ {
+ id: "environments_scores_in_array",
+ tier: 0,
+ referenceSql: `
+ SELECT DISTINCT environment
+ FROM scores
+ WHERE (project_id = {p1:String}) AND (data_type IN ({p2:Array(String)}))
+ `,
+ build: () =>
+ db()
+ .selectFrom("scores")
+ .select("environment")
+ .distinct()
+ .where("data_type", "in", [
+ "NUMERIC",
+ "BOOLEAN",
+ "CATEGORICAL",
+ "TEXT",
+ ]),
+ },
+ {
+ id: "events_core_time_bound",
+ tier: 1,
+ referenceSql: `
+ SELECT DISTINCT environment
+ FROM events_core
+ WHERE (project_id = {p1:String}) AND (start_time >= {p2:DateTime64(3)})
+ `,
+ build: () =>
+ db()
+ .selectFrom("events_core")
+ .select("environment")
+ .distinct()
+ .where("start_time", ">=", FROM_TS),
+ },
+ {
+ id: "group_by_count",
+ tier: 1,
+ referenceSql: `
+ SELECT environment, count(*) AS n
+ FROM events_core
+ WHERE project_id = {p1:String}
+ GROUP BY environment
+ `,
+ build: () =>
+ db()
+ .selectFrom("events_core")
+ .select((eb) => ["environment", eb.fn.countAll().as("n")])
+ .groupBy("environment"),
+ },
+ {
+ id: "join_observations_traces",
+ tier: 2,
+ referenceSql: `
+ SELECT o.environment
+ FROM observations AS o
+ INNER JOIN traces AS t ON (o.trace_id = t.id) AND (o.project_id = t.project_id) AND (t.project_id = {p1:String})
+ WHERE o.project_id = {p1:String}
+ `,
+ build: () =>
+ db()
+ .selectFrom("observations as o")
+ .innerJoin("traces as t", (join) =>
+ join
+ .onRef("o.trace_id", "=", "t.id")
+ .onRef("o.project_id", "=", "t.project_id"),
+ )
+ .select("o.environment"),
+ },
+ {
+ id: "cte_trace_ids",
+ tier: 2,
+ referenceSql: `
+ WITH traces_cte AS (
+ SELECT id
+ FROM traces
+ WHERE project_id = {p1:String}
+ )
+ SELECT o.environment
+ FROM observations AS o
+ WHERE (o.project_id = {p1:String}) AND (o.trace_id IN (
+ SELECT id
+ FROM traces_cte
+ ))
+ `,
+ build: () =>
+ db()
+ .with("traces_cte", (qb) => qb.selectFrom("traces").select("id"))
+ .selectFrom("observations as o")
+ .select("o.environment")
+ .where("o.trace_id", "in", (eb) =>
+ eb.selectFrom("traces_cte").select("id"),
+ ),
+ },
+ {
+ id: "array_join_cost_details",
+ tier: 3,
+ referenceSql: `
+ SELECT environment
+ FROM observations
+ ARRAY JOIN mapKeys(cost_details) AS cost_key, mapValues(cost_details) AS cost
+ WHERE project_id = {p1:String}
+ `,
+ build: () =>
+ db()
+ .selectFrom("observations")
+ .select("environment")
+ .$call(
+ arrayJoin({
+ cost_key: mapKeys("cost_details"),
+ cost: mapValues("cost_details"),
+ }),
+ ),
+ },
+ {
+ id: "limit_by_dedup",
+ tier: 3,
+ referenceSql: `
+ SELECT span_id, project_id
+ FROM events_core
+ WHERE project_id = {p1:String}
+ ORDER BY event_ts DESC
+ LIMIT 1 BY span_id, project_id
+ `,
+ build: () =>
+ db()
+ .selectFrom("events_core")
+ .select(["span_id", "project_id"])
+ .orderBy("event_ts", "desc")
+ .$call(limitBy({ count: 1, columns: ["span_id", "project_id"] })),
+ },
+ {
+ id: "window_rank",
+ tier: 4,
+ referenceSql: `
+ SELECT span_id, rank() OVER (PARTITION BY trace_id ORDER BY start_time DESC) AS rk
+ FROM events_core
+ WHERE project_id = {p1:String}
+ `,
+ build: () =>
+ db()
+ .selectFrom("events_core")
+ .select((eb) => [
+ "span_id",
+ eb.fn
+ .agg("rank")
+ .over((ob) =>
+ ob.partitionBy("trace_id").orderBy("start_time", "desc"),
+ )
+ .as("rk"),
+ ]),
+ },
+];
+
+export const CATALOG_PROJECT_ID = PROJECT_ID;
diff --git a/packages/shared/src/server/query-ast/kysely/compile.ts b/packages/shared/src/server/query-ast/kysely/compile.ts
new file mode 100644
index 0000000000..88f5840fa4
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/compile.ts
@@ -0,0 +1,51 @@
+import {
+ createQueryId,
+ type KyselyPlugin,
+ type RootOperationNode,
+} from "kysely";
+
+import type { ExecutionContext } from "../executionContext";
+import { ClickHouseQueryCompiler } from "./compiler";
+import { TenancyInjectionPlugin, requireExecutionContext } from "./tenancy";
+import { validateTypeCompatibility } from "./typecheck";
+
+export type CompiledClickhouseQuery = {
+ sql: string;
+ params: Record<string, unknown>;
+};
+
+/**
+ * Anything Kysely will give us a SelectQueryNode for, including plugin-wrapped
+ * builders. Kept structural so UNION / `$if` chains stay assignable.
+ */
+export type ClickhouseCompilable = {
+ withPlugin(plugin: KyselyPlugin): ClickhouseCompilable;
+ toOperationNode(): RootOperationNode;
+};
+
+/**
+ * The choke point. Every query that should produce ClickHouse SQL goes
+ * through here: the `ctx` parameter is required, so a caller cannot compile
+ * without a tenancy scope β€” omitting it is a compile-time type error. Tenancy
+ * is injected from that scope, and the dialect compiler refuses to emit SQL
+ * unless the tree was identity-stamped by that pass (a copied property is not
+ * enough). The runtime `requireExecutionContext` guard remains as
+ * defense-in-depth for the one case the type cannot express (an empty
+ * `projectId`) and for callers reaching this through an `any` boundary.
+ */
+export function compileClickhouseQuery(
+ query: ClickhouseCompilable,
+ ctx: ExecutionContext,
+): CompiledClickhouseQuery {
+ const scope = requireExecutionContext(ctx);
+ const stamped = query
+ .withPlugin(new TenancyInjectionPlugin(scope))
+ .toOperationNode();
+ const compiler = new ClickHouseQueryCompiler();
+ validateTypeCompatibility(stamped);
+ const compiled = compiler.compileQuery(stamped, createQueryId());
+ return {
+ sql: compiled.sql,
+ params: { ...compiler.namedParameters },
+ };
+}
diff --git a/packages/shared/src/server/query-ast/kysely/compiler.ts b/packages/shared/src/server/query-ast/kysely/compiler.ts
new file mode 100644
index 0000000000..02fc6dbb79
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/compiler.ts
@@ -0,0 +1,324 @@
+import {
+ CreateTableNode,
+ CreateViewNode,
+ DefaultQueryCompiler,
+ InsertQueryNode,
+ ParensNode,
+ SetOperationNode,
+ ValueNode,
+ type OperationNode,
+ type PrimitiveValueListNode,
+ type RootOperationNode,
+ type SelectQueryNode,
+ type ValueListNode,
+} from "kysely";
+import type { QueryId } from "kysely";
+
+import {
+ ArrayJoinNode,
+ ArrayIndexNode,
+ type ClickHouseSelectQueryNode,
+ type LimitByNode,
+} from "./nodes";
+import { assertTenancyStamped } from "./tenancy";
+
+const ARRAY_JOIN_SQL: Record<ArrayJoinNode["variant"], string> = {
+ default: "array join",
+ left: "left array join",
+ inner: "inner array join",
+};
+
+function canonicalParamValue(value: unknown): string {
+ if (value instanceof Date) return `d:${value.toISOString()}`;
+ if (Array.isArray(value)) {
+ return `a:${JSON.stringify(value)}`;
+ }
+ return `${typeof value}:${String(value)}`;
+}
+
+function inferClickHouseType(value: unknown): string {
+ if (value instanceof Date) return "DateTime64(3)";
+ if (Array.isArray(value)) {
+ const inner = value.length === 0 ? "String" : inferClickHouseType(value[0]);
+ return `Array(${inner})`;
+ }
+ if (typeof value === "string") return "String";
+ if (typeof value === "boolean") return "UInt8";
+ if (typeof value === "number") {
+ return Number.isInteger(value) ? "Int64" : "Float64";
+ }
+ if (typeof value === "bigint") return "Int64";
+ return "String";
+}
+
+/**
+ * ClickHouse SQL compiler. Identifier quoting is omitted so `clickhouse format`
+ * matches the golden snapshots (unquoted identifiers). Parameters are named
+ * `{pN:Type}` binds, interned by (type, value) so repeated values collapse to a
+ * single placeholder, matching the named `{projectId:String}` style once the
+ * golden harness's param normalizer runs.
+ *
+ * Example β€” a UNION whose two branches both filter `project_id = 'p'`:
+ *
+ * ... WHERE project_id = {p1:String} ... UNION ALL ... WHERE project_id = {p1:String}
+ *
+ * One `{p1:String}` bind is emitted and reused across both branches, not
+ * separate `{p1:String}` / `{p2:String}` bound to the same value.
+ *
+ * ARRAY JOIN and LIMIT BY ride as extra fields on the select node as real
+ * {@link ArrayJoinNode} / {@link LimitByNode} operation nodes.
+ */
+export class ClickHouseQueryCompiler extends DefaultQueryCompiler {
+ namedParameters: Record<string, unknown> = {};
+ private intern = new Map<string, string>();
+
+ constructor() {
+ super();
+ // ArrayIndexNode is a custom node kind Kysely's dispatch cannot reach and
+ // Kysely has no public hook to register one, so we patch its private
+ // visitNode/nodeStack to route the node to visitArrayIndex. Kysely is
+ // pinned to 0.28.17 for this; see kysely/README.md ("upgrade hazard").
+ const parentVisit = this.visitNode.bind(this);
+ // Anti-pattern: cast through `unknown` once to reach Kysely's private
+ // `visitNode` / `nodeStack` (not part of the public type). Deliberate, to
+ // avoid a fork.
+ const self = this as unknown as {
+ visitNode: (node: OperationNode) => void;
+ nodeStack: OperationNode[];
+ };
+ self.visitNode = (node: OperationNode) => {
+ if (ArrayIndexNode.is(node)) {
+ self.nodeStack.push(node);
+ this.visitArrayIndex(node);
+ self.nodeStack.pop();
+ return;
+ }
+ parentVisit(node);
+ };
+ }
+
+ compileQuery(node: RootOperationNode, queryId: QueryId) {
+ this.namedParameters = {};
+ this.intern = new Map();
+ assertTenancyStamped(node);
+ return super.compileQuery(node, queryId);
+ }
+
+ // Emit identifiers unquoted. `clickhouse format` normalizes to unquoted
+ // identifiers, so returning empty wrappers is what keeps raw compiler output
+ // byte-comparable with the golden snapshots. (Column/table names in this
+ // codebase are plain identifiers, so dropping the quotes is safe here.)
+ protected override getLeftIdentifierWrapper(): string {
+ return "";
+ }
+
+ protected override getRightIdentifierWrapper(): string {
+ return "";
+ }
+
+ protected override appendValue(parameter: unknown): void {
+ this.append(this.bindValue(parameter));
+ }
+
+ /**
+ * ClickHouse `IN` takes a single `Array(T)` bind, not a parenthesized list of
+ * scalar binds. So `col IN (1, 2, 3)` compiles to `col IN ({p:Array(Int64)})`
+ * β€” one array parameter β€” which is also the shape the existing production SQL
+ * (e.g. the scores list query) already relies on.
+ */
+ protected override visitPrimitiveValueList(
+ node: PrimitiveValueListNode,
+ ): void {
+ this.append("(");
+ this.append(
+ this.bindValue([...node.values], inferClickHouseType([...node.values])),
+ );
+ this.append(")");
+ }
+
+ protected override visitValueList(node: ValueListNode): void {
+ const values = node.values.map((v) =>
+ ValueNode.is(v) ? v.value : undefined,
+ );
+ if (values.every((v) => v !== undefined)) {
+ this.append("(");
+ this.append(this.bindValue(values, inferClickHouseType(values)));
+ this.append(")");
+ return;
+ }
+ super.visitValueList(node);
+ }
+
+ /**
+ * Wholesale copy of Kysely's `DefaultQueryCompiler.visitSelectQuery` (kept in
+ * sync with the pinned 0.28.17) with two ClickHouse-only insertions:
+ * - the ARRAY JOIN block, emitted after JOINs and before WHERE (line ~193)
+ * - the LIMIT BY block, emitted before LIMIT (line ~225)
+ * Both read extra fields off {@link ClickHouseSelectQueryNode}. Every other
+ * clause mirrors the parent verbatim. We override the whole method rather
+ * than call `super` because the parent emits clauses in a fixed order and
+ * offers no hook to inject a clause between JOIN and WHERE.
+ */
+ protected override visitSelectQuery(node: SelectQueryNode): void {
+ const wrapInParens =
+ this.parentNode !== undefined &&
+ !ParensNode.is(this.parentNode) &&
+ !InsertQueryNode.is(this.parentNode) &&
+ !CreateTableNode.is(this.parentNode) &&
+ !CreateViewNode.is(this.parentNode) &&
+ !SetOperationNode.is(this.parentNode);
+
+ if (this.parentNode === undefined && node.explain) {
+ this.visitNode(node.explain);
+ this.append(" ");
+ }
+
+ if (wrapInParens) {
+ this.append("(");
+ }
+
+ if (node.with) {
+ this.visitNode(node.with);
+ this.append(" ");
+ }
+
+ this.append("select");
+
+ if (node.distinctOn) {
+ this.append(" ");
+ this.compileDistinctOn(node.distinctOn);
+ }
+
+ if (node.frontModifiers?.length) {
+ this.append(" ");
+ this.compileList(node.frontModifiers, " ");
+ }
+
+ if (node.top) {
+ this.append(" ");
+ this.visitNode(node.top);
+ }
+
+ if (node.selections) {
+ this.append(" ");
+ this.compileList(node.selections);
+ }
+
+ if (node.from) {
+ this.append(" ");
+ this.visitNode(node.from);
+ }
+
+ if (node.joins) {
+ this.append(" ");
+ this.compileList(node.joins, " ");
+ }
+
+ const chNode = node as ClickHouseSelectQueryNode;
+ if (chNode.arrayJoins?.length) {
+ for (const arrayJoin of chNode.arrayJoins) {
+ this.append(" ");
+ this.visitArrayJoin(arrayJoin);
+ }
+ }
+
+ if (node.where) {
+ this.append(" ");
+ this.visitNode(node.where);
+ }
+
+ if (node.groupBy) {
+ this.append(" ");
+ this.visitNode(node.groupBy);
+ }
+
+ if (node.having) {
+ this.append(" ");
+ this.visitNode(node.having);
+ }
+
+ if (node.setOperations) {
+ this.append(" ");
+ this.compileList(node.setOperations, " ");
+ }
+
+ if (node.orderBy) {
+ this.append(" ");
+ this.visitNode(node.orderBy);
+ }
+
+ if (chNode.limitBy) {
+ this.append(" ");
+ this.visitLimitBy(chNode.limitBy);
+ }
+
+ if (node.limit) {
+ this.append(" ");
+ this.visitNode(node.limit);
+ }
+
+ if (node.offset) {
+ this.append(" ");
+ this.visitNode(node.offset);
+ }
+
+ if (node.fetch) {
+ this.append(" ");
+ this.visitNode(node.fetch);
+ }
+
+ if (node.endModifiers?.length) {
+ this.append(" ");
+ this.compileList(this.sortSelectModifiers([...node.endModifiers]), " ");
+ }
+
+ if (wrapInParens) {
+ this.append(")");
+ }
+ }
+
+ private visitArrayIndex(node: ArrayIndexNode): void {
+ this.visitNode(node.array);
+ this.append("[");
+ this.visitNode(node.index);
+ this.append("]");
+ }
+
+ private visitArrayJoin(node: ArrayJoinNode): void {
+ this.append(ARRAY_JOIN_SQL[node.variant]);
+ this.append(" ");
+ node.items.forEach((item, index) => {
+ if (index > 0) this.append(", ");
+ this.visitNode(item.expression);
+ this.append(" as ");
+ this.visitNode(item.alias);
+ });
+ }
+
+ private visitLimitBy(node: LimitByNode): void {
+ this.append("limit ");
+ this.visitNode(node.count);
+ this.append(" by ");
+ this.compileList(node.columns);
+ }
+
+ private bindValue(value: unknown, type?: string): string {
+ // `type` is the caller-known ClickHouse type; when omitted we infer the
+ // widest canonical type from the JS value (Int64 / Float64 / String /
+ // DateTime64(3)). Inference only types the bind *placeholder* β€” ClickHouse
+ // still coerces it to the compared column's type β€” so the widening cases we
+ // emit are safe. Pass `type` explicitly where exact width or precision
+ // matters (e.g. a Decimal column, where inferred Float64 could round).
+ const inferred = type ?? inferClickHouseType(value);
+ const key = `${inferred}:${canonicalParamValue(value)}`;
+ const existing = this.intern.get(key);
+ if (existing) {
+ return `{${existing}:${inferred}}`;
+ }
+ this.addParameter(value);
+ const name = `p${this.intern.size + 1}`;
+ this.intern.set(key, name);
+ this.namedParameters[name] = value;
+ return `{${name}:${inferred}}`;
+ }
+}
diff --git a/packages/shared/src/server/query-ast/kysely/composition.test.ts b/packages/shared/src/server/query-ast/kysely/composition.test.ts
new file mode 100644
index 0000000000..5435aa0bf5
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/composition.test.ts
@@ -0,0 +1,126 @@
+import { describe, expect, it } from "vitest";
+
+import { compileClickhouseQuery } from "./compile";
+import { getClickhouseKysely } from "./dialect";
+import { arrayJoin, limitBy, mapKeys, mapValues } from "./extensions";
+import { defineView, fromView } from "./views";
+
+// Raw compiler output (no `clickhouse format` binary needed), so these run
+// unconditionally in CI. The raw SQL is lowercase; match case-insensitively.
+const ctx = { projectId: "p" };
+
+const costArrayJoin = () =>
+ arrayJoin({
+ cost_key: mapKeys("cost_details"),
+ cost: mapValues("cost_details"),
+ });
+
+const dedup = () => limitBy({ count: 1, columns: ["span_id", "project_id"] });
+
+describe("ARRAY JOIN composition across nesting positions", () => {
+ it("emits ARRAY JOIN at the top level (baseline)", () => {
+ const db = getClickhouseKysely();
+ const qb = db
+ .selectFrom("observations")
+ .select("environment")
+ .$call(costArrayJoin());
+
+ const { sql } = compileClickhouseQuery(qb, ctx);
+ expect(sql).toMatch(/array join/i);
+ expect(sql).toMatch(/mapkeys/i);
+ expect(sql).toMatch(/mapvalues/i);
+ });
+
+ it("keeps ARRAY JOIN when applied inside a CTE body", () => {
+ const db = getClickhouseKysely();
+ const qb = db
+ .with("aj", (qb) =>
+ qb
+ .selectFrom("observations")
+ .select("environment")
+ .$call(costArrayJoin()),
+ )
+ .selectFrom("aj")
+ .select("environment");
+
+ const { sql } = compileClickhouseQuery(qb, ctx);
+ expect(sql).toMatch(/with aj as/i);
+ // The ARRAY JOIN must live inside the CTE body and not leak into the outer
+ // query. Split on the outer query (the `) select ... from aj` that closes
+ // the CTE): the clause must appear before the split and be absent after it.
+ const outerStart = sql.toLowerCase().search(/\)\s*select[\s\S]*from aj/i);
+ expect(outerStart).toBeGreaterThan(-1);
+ const cteBody = sql.slice(0, outerStart);
+ const outerQuery = sql.slice(outerStart);
+ expect(cteBody).toMatch(/array join/i);
+ expect(outerQuery).not.toMatch(/array join/i);
+ });
+
+ it("keeps ARRAY JOIN when applied inside a subquery", () => {
+ const db = getClickhouseKysely();
+ const qb = db
+ .selectFrom(
+ db
+ .selectFrom("observations")
+ .select("environment")
+ .$call(costArrayJoin())
+ .as("sub"),
+ )
+ .select("environment");
+
+ const { sql } = compileClickhouseQuery(qb, ctx);
+ // Outer query has no ARRAY JOIN; presence proves the subquery clause held.
+ expect(sql).toMatch(/array join/i);
+ });
+
+ it("keeps ARRAY JOIN when applied inside a virtual-view body", () => {
+ const db = getClickhouseKysely();
+ const view = defineView("aj_view")<{ environment: string }>(() =>
+ db
+ .selectFrom("observations")
+ .select("environment")
+ .$call(costArrayJoin()),
+ );
+
+ const qb = fromView(view).select("environment");
+
+ const { sql } = compileClickhouseQuery(qb, ctx);
+ // Outer query only selects from the view alias; presence proves the view
+ // body's ARRAY JOIN composed into the emitted CTE.
+ expect(sql).toMatch(/array join/i);
+ expect(sql).toMatch(/with aj_view as/i);
+ });
+});
+
+describe("LIMIT BY composition across nesting positions", () => {
+ it("emits LIMIT BY at the top level (baseline)", () => {
+ const db = getClickhouseKysely();
+ const qb = db
+ .selectFrom("events_core")
+ .select(["span_id", "project_id"])
+ .orderBy("event_ts", "desc")
+ .$call(dedup());
+
+ const { sql } = compileClickhouseQuery(qb, ctx);
+ expect(sql).toMatch(/limit 1 by/i);
+ });
+
+ it("keeps LIMIT BY when applied inside a CTE body", () => {
+ const db = getClickhouseKysely();
+ const qb = db
+ .with("lb", (qb) =>
+ qb
+ .selectFrom("events_core")
+ .select(["span_id", "project_id"])
+ .orderBy("event_ts", "desc")
+ .$call(dedup()),
+ )
+ .selectFrom("lb")
+ .select("span_id");
+
+ const { sql } = compileClickhouseQuery(qb, ctx);
+ // Outer query has no LIMIT BY; presence proves the CTE body's clause held.
+ expect(sql).toMatch(/limit 1 by/i);
+ expect(sql).toMatch(/with lb as/i);
+ });
+});
diff --git a/packages/shared/src/server/query-ast/kysely/dialect.ts b/packages/shared/src/server/query-ast/kysely/dialect.ts
new file mode 100644
index 0000000000..8b85c5bebd
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/dialect.ts
@@ -0,0 +1,42 @@
+import {
+ DummyDriver,
+ Kysely,
+ SqliteAdapter,
+ SqliteIntrospector,
+ type Dialect,
+} from "kysely";
+
+import { ClickHouseQueryCompiler } from "./compiler";
+import type { ClickHouseDatabase } from "./schema";
+
+/**
+ * Compile-only ClickHouse dialect. Queries are never executed through Kysely;
+ * the existing `queryClickhouse` seam remains the exec path. DummyDriver
+ * throws if anyone tries to `.execute()`.
+ */
+export class ClickHouseCompileDialect implements Dialect {
+ createDriver() {
+ return new DummyDriver();
+ }
+
+ createQueryCompiler() {
+ return new ClickHouseQueryCompiler();
+ }
+
+ createAdapter() {
+ return new SqliteAdapter();
+ }
+
+ createIntrospector(db: Kysely<any>) {
+ return new SqliteIntrospector(db);
+ }
+}
+
+let compileDb: Kysely<ClickHouseDatabase> | undefined;
+
+export function getClickhouseKysely(): Kysely<ClickHouseDatabase> {
+ compileDb ??= new Kysely<ClickHouseDatabase>({
+ dialect: new ClickHouseCompileDialect(),
+ });
+ return compileDb;
+}
diff --git a/packages/shared/src/server/query-ast/kysely/errors.ts b/packages/shared/src/server/query-ast/kysely/errors.ts
new file mode 100644
index 0000000000..e7479512e7
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/errors.ts
@@ -0,0 +1,20 @@
+export class QueryCompileError extends Error {
+ constructor(message: string) {
+ super(message);
+ this.name = "QueryCompileError";
+ }
+}
+
+export class UnscopedRelationError extends QueryCompileError {
+ constructor(message: string) {
+ super(message);
+ this.name = "UnscopedRelationError";
+ }
+}
+
+export class TypeCompatibilityError extends QueryCompileError {
+ constructor(message: string) {
+ super(message);
+ this.name = "TypeCompatibilityError";
+ }
+}
diff --git a/packages/shared/src/server/query-ast/kysely/extensions.test.ts b/packages/shared/src/server/query-ast/kysely/extensions.test.ts
new file mode 100644
index 0000000000..a96246165e
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/extensions.test.ts
@@ -0,0 +1,148 @@
+import { describe, expect, it } from "vitest";
+
+import { compileClickhouseQuery } from "./compile";
+import { getClickhouseKysely } from "./dialect";
+import { TypeCompatibilityError } from "./errors";
+import { metadataValue } from "./extensions";
+import { ArrayIndexNode } from "./nodes";
+import { TenancyInjectionPlugin } from "./tenancy";
+import { defineView, fromView } from "./views";
+
+const ctx = { projectId: "proj-1" };
+
+describe("schema-typed selection (condition 7)", () => {
+ it("accepts sum() over a numeric column and emits SQL", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select((eb) => [eb.fn.sum("total_cost").as("s")]);
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ expect(compiled.toLowerCase()).toContain("sum(total_cost)");
+ expect(compiled.toLowerCase()).toContain("project_id");
+ });
+
+ it("rejects sum() over a String column at the runtime validation pass", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select((eb) => [eb.fn.sum("environment").as("s")]);
+
+ expect(() => compileClickhouseQuery(qb, ctx)).toThrow(
+ TypeCompatibilityError,
+ );
+ expect(() => compileClickhouseQuery(qb, ctx)).toThrow(/environment/);
+ });
+});
+
+describe("metadata indexOf access (condition 7b)", () => {
+ // These tests use a two-layer strategy. This first test inspects the AST
+ // itself (serialized to JSON) to prove `metadata[key]` is *lowered* to real
+ // operation nodes β€” an `ArrayIndexNode` subscript around an `indexOf(...)`
+ // `FunctionNode` β€” and never a `RawNode` (a raw SQL string). The sibling
+ // tests below assert the SQL the compiler then emits. Separating them
+ // isolates "the builder produced the right tree" (composable, escaped,
+ // parameter-bound) from "the compiler rendered that tree correctly".
+ it("lowers metadata[key] to ArrayIndexNode + indexOf FunctionNode, not RawNode", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core as e")
+ .select((_eb) => [metadataValue("e", "a").as("a")])
+ .where((eb) => eb(metadataValue("e", "a"), ">", 2));
+
+ const node = qb
+ .withPlugin(new TenancyInjectionPlugin(ctx))
+ .toOperationNode();
+ const json = JSON.stringify(node);
+ expect(json).toContain('"kind":"ArrayIndexNode"');
+ expect(json).toContain('"func":"indexOf"');
+ expect(json).not.toMatch(/"kind":"RawNode"/);
+
+ // `.as("a")` wraps the expression in an AliasNode, so the first selection's
+ // inner node must be that AliasNode rather than the bare ArrayIndexNode.
+ const selections = (
+ node as { selections?: { selection?: { kind: string } }[] }
+ ).selections;
+ expect(selections?.[0]?.selection?.kind).toBe("AliasNode");
+ });
+
+ it("emits subscript + bound key param in SELECT and WHERE", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core as e")
+ .select((_eb) => [metadataValue("e", "a").as("a")])
+ .where((eb) => eb(metadataValue("e", "a"), ">", 2));
+
+ const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);
+ const lower = compiled.toLowerCase();
+ expect(lower).toContain("metadata_values[indexof(e.metadata_names");
+ expect(lower).toContain("as a");
+ expect(Object.values(params)).toContain("a");
+ expect(Object.values(params)).toContain(2);
+ expect(compiled).not.toMatch(/metadata_names,\s*'a'/);
+ });
+
+ it("emits the same lowering in HAVING", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core as e")
+ .select("environment")
+ .groupBy("environment")
+ .having((eb) => eb(metadataValue("e", "a"), ">", 2));
+
+ const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);
+ expect(compiled.toLowerCase()).toContain("having");
+ expect(compiled.toLowerCase()).toContain(
+ "metadata_values[indexof(e.metadata_names",
+ );
+ expect(Object.values(params)).toContain("a");
+ });
+});
+
+describe("virtual view (condition 8)", () => {
+ const environmentsView = defineView("environments_view")<{
+ environment: string;
+ }>(() =>
+ getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("environment")
+ .distinct(),
+ );
+
+ it("treats the view as a table and emits a WITH CTE", () => {
+ const qb = fromView(environmentsView)
+ .select("environment")
+ .where("environment", "=", "production");
+
+ const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);
+ const lower = compiled.toLowerCase();
+ expect(lower).toContain("with environments_view as");
+ expect(lower).toContain("from events_core");
+ expect(lower).toContain("from environments_view");
+ expect(lower).toContain("environment =");
+ expect(Object.values(params)).toContain("proj-1");
+ expect(Object.values(params)).toContain("production");
+ });
+
+ it("injects tenancy on the inner physical table, not the view alias", () => {
+ const qb = fromView(environmentsView).select("environment");
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ const lower = compiled.toLowerCase();
+ expect(lower).toMatch(
+ /with environments_view as \([\s\S]*project_id[\s\S]*\)[\s\S]*from environments_view/,
+ );
+ const afterView = lower.split("from environments_view")[1] ?? "";
+ expect(afterView).not.toContain("project_id");
+ });
+});
+
+describe("ArrayIndexNode identity", () => {
+ it("is a real node kind", () => {
+ expect(
+ ArrayIndexNode.is(
+ ArrayIndexNode.create(
+ {
+ kind: "ColumnNode",
+ column: { kind: "IdentifierNode", name: "x" },
+ } as never,
+ { kind: "ValueNode", value: 1 } as never,
+ ),
+ ),
+ ).toBe(true);
+ });
+});
diff --git a/packages/shared/src/server/query-ast/kysely/extensions.ts b/packages/shared/src/server/query-ast/kysely/extensions.ts
new file mode 100644
index 0000000000..dcb64d2c9a
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/extensions.ts
@@ -0,0 +1,230 @@
+/**
+ * ClickHouse-specific query-builder extensions for the Kysely dialect.
+ *
+ * Kysely has no native representation for several ClickHouse constructs, so
+ * this file supplies them in three forms:
+ * - Plugins ({@link ArrayJoinPlugin}, {@link LimitByPlugin}) that attach a
+ * custom operation node (`ArrayJoinNode` / `LimitByNode`) onto the select
+ * node during query transformation; the ClickHouse compiler renders them.
+ * - `$call` helpers ({@link arrayJoin}, {@link limitBy}) that apply those
+ * plugins while extending the builder's output row type, so produced
+ * aliases are type-checked and misspelled references are compile errors.
+ * - Typed expression helpers ({@link mapKeys}, {@link mapValues},
+ * {@link metadataValue}) that build ClickHouse map/array function calls and
+ * `metadata[key]` subscripts as real, parameter-bound nodes β€” never raw SQL
+ * strings β€” so they escape and compose like any other expression.
+ */
+import {
+ ColumnNode,
+ ExpressionWrapper,
+ FunctionNode,
+ IdentifierNode,
+ ReferenceNode,
+ TableNode,
+ ValueNode,
+ type Expression,
+ type KyselyPlugin,
+ type OperationNode,
+ type PluginTransformQueryArgs,
+ type PluginTransformResultArgs,
+ type RootOperationNode,
+ type SelectQueryBuilder,
+} from "kysely";
+
+import {
+ ArrayIndexNode,
+ ArrayJoinNode,
+ LimitByNode,
+ type ArrayJoinVariant,
+ type ClickHouseSelectQueryNode,
+} from "./nodes";
+import type { ClickHouseDatabase } from "./schema";
+import { ClickHouseOperationNodeTransformer } from "./transformer";
+
+type OperationNodeSource = OperationNode | { toOperationNode(): OperationNode };
+
+function toNode(source: OperationNodeSource): OperationNode {
+ if (
+ "toOperationNode" in source &&
+ typeof source.toOperationNode === "function"
+ ) {
+ return source.toOperationNode();
+ }
+ return source as OperationNode;
+}
+
+type ArrayJoinItem = {
+ expression: OperationNodeSource;
+ as: string;
+};
+
+/** Plugin that attaches an {@link ArrayJoinNode} onto the select node. */
+class ArrayJoinPlugin implements KyselyPlugin {
+ constructor(
+ private readonly items: ReadonlyArray<ArrayJoinItem>,
+ private readonly variant: ArrayJoinVariant = "default",
+ ) {}
+
+ transformQuery(args: PluginTransformQueryArgs): RootOperationNode {
+ if (args.node.kind !== "SelectQueryNode") return args.node;
+ const transformer = new ClickHouseOperationNodeTransformer();
+ const node = transformer.transformNode(
+ args.node,
+ ) as ClickHouseSelectQueryNode;
+ const arrayJoinNode = ArrayJoinNode.create(
+ this.items.map((item) => ({
+ expression: toNode(item.expression),
+ alias: IdentifierNode.create(item.as),
+ })),
+ this.variant,
+ );
+ return {
+ ...node,
+ arrayJoins: [...(node.arrayJoins ?? []), arrayJoinNode],
+ } as RootOperationNode;
+ }
+
+ async transformResult(args: PluginTransformResultArgs) {
+ return args.result;
+ }
+}
+
+type LimitBySpec = {
+ count: number;
+ columns: ReadonlyArray<string>;
+};
+
+/** Plugin that attaches a {@link LimitByNode} onto the select node. */
+class LimitByPlugin implements KyselyPlugin {
+ constructor(private readonly spec: LimitBySpec) {}
+
+ transformQuery(args: PluginTransformQueryArgs): RootOperationNode {
+ if (args.node.kind !== "SelectQueryNode") return args.node;
+ const transformer = new ClickHouseOperationNodeTransformer();
+ const node = transformer.transformNode(
+ args.node,
+ ) as ClickHouseSelectQueryNode;
+ return {
+ ...node,
+ limitBy: LimitByNode.create(
+ ValueNode.createImmediate(this.spec.count),
+ this.spec.columns.map(columnRef),
+ ),
+ } as RootOperationNode;
+ }
+
+ async transformResult(args: PluginTransformResultArgs) {
+ return args.result;
+ }
+}
+
+// Build a `table.column` reference straight from its parts. Prefer this
+// whenever the caller already holds the table and column separately.
+function qualifiedColumn(table: string, column: string): OperationNode {
+ return ReferenceNode.create(
+ ColumnNode.create(column),
+ TableNode.create(table),
+ );
+}
+
+// For callers whose column arrives as a single, possibly dotted string β€” e.g.
+// the `limitBy` column list. Kysely only parses `"table.column"` at the
+// expression-builder layer (`eb.ref`) and does not export its string-reference
+// parser, and these plugins build raw OperationNodes with no ExpressionBuilder
+// in scope, so we split the dotted name here rather than route through a string.
+function columnRef(name: string): OperationNode {
+ const parts = name.split(".");
+ return parts.length === 2
+ ? qualifiedColumn(parts[0], parts[1])
+ : ColumnNode.create(name);
+}
+
+function arrayFunction<T>(fn: string, column: string): Expression<T[]> {
+ return new ExpressionWrapper<
+ ClickHouseDatabase,
+ keyof ClickHouseDatabase,
+ T[]
+ >(FunctionNode.create(fn, [columnRef(column)]) as unknown as OperationNode);
+}
+
+/** `mapKeys(map)` as a typed array expression (element type defaults to string). */
+export function mapKeys<K = string>(column: string): Expression<K[]> {
+ return arrayFunction<K>("mapKeys", column);
+}
+
+/** `mapValues(map)` as a typed array expression (element type defaults to string). */
+export function mapValues<V = string>(column: string): Expression<V[]> {
+ return arrayFunction<V>("mapValues", column);
+}
+
+/**
+ * Lower `metadata[key]` to a traced array-subscript + `indexOf` node.
+ * The key is a bound `ValueNode`, not a SQL literal.
+ */
+export function metadataValue(
+ tableAlias: string,
+ key: string,
+): ExpressionWrapper<ClickHouseDatabase, "events_core", string | number> {
+ return new ExpressionWrapper(
+ ArrayIndexNode.create(
+ qualifiedColumn(tableAlias, "metadata_values"),
+ FunctionNode.create("indexOf", [
+ qualifiedColumn(tableAlias, "metadata_names"),
+ ValueNode.create(key),
+ ]),
+ ) as unknown as OperationNode,
+ );
+}
+
+// Extract the element type `T` from an array-typed `Expression<T[]>`. `arrayJoin`
+// uses it to give each produced alias the array's *element* type in the
+// builder's output row (ARRAY JOIN yields one row per element, so a
+// `mapValues(...)` expression of `V[]` surfaces the alias as `V`, not `V[]`).
+// Resolves to `never` for a non-array expression.
+type ElementOf<E> =
+ E extends Expression<infer A>
+ ? A extends ReadonlyArray<infer T>
+ ? T
+ : never
+ : never;
+
+/**
+ * ARRAY JOIN clause as a `$call` step: `qb.$call(arrayJoin({ alias: expr }))`.
+ * Each entry's alias is added to the builder's output row type, so consumers of
+ * the result (e.g. an outer query selecting from a CTE body) can reference the
+ * produced columns and typos on the alias name are compile errors. The element
+ * value type is opaque (Kysely's Expression hides its type arg), so aliases
+ * surface as `unknown`. The alias must still be projected in a `select` to
+ * appear in the emitted SQL. Distinct from the `arrayJoin()` SELECT function.
+ */
+export function arrayJoin<
+ Items extends { [K in keyof Items]: Expression<ReadonlyArray<unknown>> },
+>(items: Items, variant: ArrayJoinVariant = "default") {
+ const list: ArrayJoinItem[] = Object.entries(items).map(
+ ([as, expression]) => ({
+ expression: expression as OperationNodeSource,
+ as,
+ }),
+ );
+ return <DB, TB extends keyof DB, O>(
+ qb: SelectQueryBuilder<DB, TB, O>,
+ ): SelectQueryBuilder<
+ DB,
+ TB,
+ O & { [K in keyof Items]: ElementOf<Items[K]> }
+ > =>
+ qb.withPlugin(
+ new ArrayJoinPlugin(list, variant),
+ ) as unknown as SelectQueryBuilder<
+ DB,
+ TB,
+ O & { [K in keyof Items]: ElementOf<Items[K]> }
+ >;
+}
+
+/** LIMIT BY clause as a `$call` step: `qb.$call(limitBy({ count, columns }))`. */
+export function limitBy(spec: LimitBySpec) {
+ return <DB, TB extends keyof DB, O>(
+ qb: SelectQueryBuilder<DB, TB, O>,
+ ): SelectQueryBuilder<DB, TB, O> => qb.withPlugin(new LimitByPlugin(spec));
+}
diff --git a/packages/shared/src/server/query-ast/kysely/nodes.test.ts b/packages/shared/src/server/query-ast/kysely/nodes.test.ts
new file mode 100644
index 0000000000..7b911e5350
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/nodes.test.ts
@@ -0,0 +1,288 @@
+import { createQueryId, sql, type KyselyPlugin } from "kysely";
+import { describe, expect, it } from "vitest";
+
+import { compileClickhouseQuery } from "./compile";
+import { ClickHouseQueryCompiler } from "./compiler";
+import { getClickhouseKysely } from "./dialect";
+import { QueryCompileError, UnscopedRelationError } from "./errors";
+import { arrayJoin, limitBy, mapKeys, mapValues } from "./extensions";
+import {
+ ArrayJoinNode,
+ LimitByNode,
+ isClickHouseSelectQueryNode,
+} from "./nodes";
+import { TenancyInjectionPlugin } from "./tenancy";
+
+const ctx = { projectId: "proj-1" };
+
+describe("tenancy injection", () => {
+ it("requires an ExecutionContext at compile time", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("traces")
+ .select("environment")
+ .where("project_id", "=", "proj-1");
+
+ // The mandatory `ctx` parameter is the primary guard: omitting it is a
+ // type error. The `@ts-expect-error` both documents and enforces that
+ // contract, and the runtime throw is the defense-in-depth backstop for
+ // callers that reach this through an `any` boundary.
+ // @ts-expect-error - ctx is required
+ expect(() => compileClickhouseQuery(qb)).toThrow(QueryCompileError);
+ // @ts-expect-error - ctx is required
+ expect(() => compileClickhouseQuery(qb)).toThrow(/no tenancy scope/);
+ });
+
+ it("refuses a context with an empty projectId", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("traces")
+ .select("environment")
+ .where("project_id", "=", "proj-1");
+
+ // Type-valid but semantically empty: the type system cannot express
+ // "non-empty string", so the runtime check earns its keep here.
+ expect(() => compileClickhouseQuery(qb, { projectId: "" })).toThrow(
+ /no tenancy scope/,
+ );
+ });
+
+ it("refuses kysely.compile() when the tenancy pass did not run", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("traces")
+ .select("environment")
+ .where("project_id", "=", "proj-1");
+
+ expect(() => qb.compile()).toThrow(QueryCompileError);
+ });
+
+ it("injects project_id when the builder omitted it", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("environment");
+
+ const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);
+ expect(compiled.toLowerCase()).toContain("project_id");
+ expect(Object.values(params)).toContain("proj-1");
+ });
+
+ it("does not duplicate project_id when the builder already scoped the table", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("environment")
+ .where("project_id", "=", "proj-1");
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ const matches = compiled.match(/project_id/gi) ?? [];
+ expect(matches).toHaveLength(1);
+ });
+
+ it("rejects a raw-SQL table source as an unscoped escape hatch", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom(sql<{ environment: string }>`traces`.as("t"))
+ .select("environment");
+
+ expect(() => compileClickhouseQuery(qb, ctx)).toThrow(
+ UnscopedRelationError,
+ );
+ });
+
+ it("rejects aliased raw SQL that would introduce an unscoped relation", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom(sql<{ environment: string }>`(SELECT * FROM traces)`.as("t"))
+ .selectAll();
+
+ expect(() => compileClickhouseQuery(qb, ctx)).toThrow(
+ UnscopedRelationError,
+ );
+ });
+
+ it("rejects raw SQL fragments in SELECT and WHERE, not only FROM/JOIN", () => {
+ const selectRaw = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select(
+ sql<string>`(SELECT environment FROM traces LIMIT 1)`.as("stolen"),
+ );
+
+ const whereRaw = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("environment")
+ .where(sql<boolean>`id IN (SELECT id FROM traces)`);
+
+ expect(() => compileClickhouseQuery(selectRaw, ctx)).toThrow(
+ UnscopedRelationError,
+ );
+ expect(() => compileClickhouseQuery(whereRaw, ctx)).toThrow(
+ UnscopedRelationError,
+ );
+ });
+
+ it("does not treat a copied langfuseTenancy property as a valid stamp", () => {
+ const qb = getClickhouseKysely().selectFrom("traces").select("environment");
+ const forged = {
+ ...qb.toOperationNode(),
+ langfuseTenancy: { projectId: "proj-1" },
+ };
+ const compiler = new ClickHouseQueryCompiler();
+ expect(() => compiler.compileQuery(forged, createQueryId())).toThrow(
+ QueryCompileError,
+ );
+
+ const fakeStamp: KyselyPlugin = {
+ transformQuery(args) {
+ return {
+ ...args.node,
+ langfuseTenancy: { projectId: "forged" },
+ };
+ },
+ async transformResult(args) {
+ return args.result;
+ },
+ };
+ const stampedByProperty = getClickhouseKysely()
+ .withPlugin(fakeStamp)
+ .selectFrom("traces")
+ .select("environment");
+ expect(() => stampedByProperty.compile()).toThrow(QueryCompileError);
+ });
+
+ it("qualifies every project_id on a two-table JOIN", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("observations as o")
+ .innerJoin("traces as t", (join) =>
+ join
+ .onRef("o.trace_id", "=", "t.id")
+ .onRef("o.project_id", "=", "t.project_id"),
+ )
+ .select("o.environment")
+ .where("o.project_id", "=", "proj-1");
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ const refs = compiled.match(/[\w.]*project_id/gi) ?? [];
+ expect(refs.length).toBeGreaterThanOrEqual(3);
+ for (const ref of refs) {
+ expect(ref).toMatch(/^(o|t)\.project_id$/i);
+ }
+ expect(compiled).toMatch(/o\.project_id/i);
+ expect(compiled).toMatch(/t\.project_id/i);
+ });
+
+ it("injects the context project even when the builder scoped a different project", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("environment")
+ .where("project_id", "=", "other-project");
+
+ const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);
+ // The builder's `other-project` predicate does not scope the request's
+ // project, so the pass must still inject `project_id = proj-1`.
+ expect(Object.values(params)).toContain("proj-1");
+ const matches = compiled.match(/project_id/gi) ?? [];
+ expect(matches.length).toBeGreaterThanOrEqual(2);
+ });
+
+ it("does not let one unqualified project_id cover multiple tenanted JOIN relations", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("observations as o")
+ .innerJoin("traces as t", (join) => join.onRef("o.trace_id", "=", "t.id"))
+ .select("o.environment")
+ // Unqualified, and ambiguous across two tenanted tables β€” must not be
+ // treated as scoping either relation.
+ .where("project_id", "=", "proj-1");
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ expect(compiled).toMatch(/o\.project_id/i);
+ expect(compiled).toMatch(/t\.project_id/i);
+ });
+
+ it("scopes an aliased relation whose alias collides with another table's physical name", () => {
+ // `scores AS traces` aliases one tenanted table to the *physical name* of
+ // another tenanted table (`traces`). A qualified `traces.project_id`
+ // predicate scopes the alias only; the physically-`traces` relation
+ // (aliased `t`) must still get its own injected predicate. Regression for
+ // an alias/table-name collision that previously left `t` unscoped.
+ const qb = getClickhouseKysely()
+ .selectFrom("scores as traces")
+ .innerJoin("traces as t", (join) => join.onRef("traces.id", "=", "t.id"))
+ .select("t.environment")
+ .where("traces.project_id", "=", "proj-1");
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ expect(compiled).toMatch(/t\.project_id/i);
+ const matches = compiled.match(/project_id/gi) ?? [];
+ expect(matches.length).toBeGreaterThanOrEqual(2);
+ });
+});
+
+describe("ARRAY JOIN and LIMIT BY nodes", () => {
+ it("attaches a real ArrayJoinNode, not a RawNode", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("observations")
+ .select("environment")
+ .where("project_id", "=", "proj-1")
+ .$call(
+ arrayJoin({
+ cost_key: mapKeys("cost_details"),
+ cost: mapValues("cost_details"),
+ }),
+ );
+
+ const node = qb
+ .withPlugin(new TenancyInjectionPlugin(ctx))
+ .toOperationNode();
+ expect(isClickHouseSelectQueryNode(node)).toBe(true);
+ if (!isClickHouseSelectQueryNode(node)) return;
+ expect(node.arrayJoins).toHaveLength(1);
+ const arrayJoinNode = node.arrayJoins![0];
+ expect(ArrayJoinNode.is(arrayJoinNode)).toBe(true);
+ expect(arrayJoinNode.items).toHaveLength(2);
+ expect(arrayJoinNode.items[0].expression.kind).toBe("FunctionNode");
+ expect(arrayJoinNode.items[0].alias.kind).toBe("IdentifierNode");
+ expect(JSON.stringify(node)).not.toMatch(/"kind":"RawNode"/);
+ });
+
+ it("emits ARRAY JOIN from the node, not a string splice", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("observations")
+ .select("environment")
+ .where("project_id", "=", "proj-1")
+ .$call(arrayJoin({ cost_key: mapKeys("cost_details") }));
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ expect(compiled.toLowerCase()).toContain("array join");
+ expect(compiled.toLowerCase()).toContain(
+ "mapkeys(cost_details) as cost_key",
+ );
+ });
+
+ it("attaches a real LimitByNode", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("span_id")
+ .where("project_id", "=", "proj-1")
+ .$call(limitBy({ count: 1, columns: ["span_id", "project_id"] }));
+
+ const node = qb
+ .withPlugin(new TenancyInjectionPlugin(ctx))
+ .toOperationNode();
+ expect(isClickHouseSelectQueryNode(node)).toBe(true);
+ if (!isClickHouseSelectQueryNode(node)) return;
+ expect(node.limitBy).toBeDefined();
+ expect(LimitByNode.is(node.limitBy!)).toBe(true);
+ expect(node.limitBy!.columns).toHaveLength(2);
+ expect(node.limitBy!.count.kind).toBe("ValueNode");
+ });
+
+ it("emits LIMIT BY before a plain LIMIT", () => {
+ const qb = getClickhouseKysely()
+ .selectFrom("events_core")
+ .select("span_id")
+ .where("project_id", "=", "proj-1")
+ .limit(50)
+ .$call(limitBy({ count: 1, columns: ["span_id"] }));
+
+ const { sql: compiled } = compileClickhouseQuery(qb, ctx);
+ const lower = compiled.toLowerCase();
+ expect(lower).toContain("limit ");
+ expect(lower).toContain(" by ");
+ expect(lower.indexOf(" by ")).toBeLessThan(lower.lastIndexOf("limit"));
+ });
+});
diff --git a/packages/shared/src/server/query-ast/kysely/nodes.ts b/packages/shared/src/server/query-ast/kysely/nodes.ts
new file mode 100644
index 0000000000..8b08f21423
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/nodes.ts
@@ -0,0 +1,114 @@
+import {
+ type IdentifierNode,
+ type OperationNode,
+ type SelectQueryNode,
+} from "kysely";
+
+/**
+ * ClickHouse `ARRAY JOIN` / `LEFT ARRAY JOIN` / `INNER ARRAY JOIN`.
+ *
+ * Not a Kysely `OperationNode` kind β€” Kysely's `kind` union is closed, so a
+ * real custom kind would collapse to `never`. These nodes live as extra
+ * fields on {@link ClickHouseSelectQueryNode}. Children (`expression`,
+ * `alias`) are traced Kysely nodes, not raw SQL.
+ */
+export type ArrayJoinVariant = "default" | "left" | "inner";
+
+type ArrayJoinItemNode = {
+ readonly expression: OperationNode;
+ readonly alias: IdentifierNode;
+};
+
+export type ArrayJoinNode = {
+ readonly kind: "ArrayJoinNode";
+ readonly variant: ArrayJoinVariant;
+ readonly items: ReadonlyArray<ArrayJoinItemNode>;
+};
+
+export const ArrayJoinNode = {
+ is(node: { kind: string }): node is ArrayJoinNode {
+ return node.kind === "ArrayJoinNode";
+ },
+ create(
+ items: ReadonlyArray<ArrayJoinItemNode>,
+ variant: ArrayJoinVariant = "default",
+ ): ArrayJoinNode {
+ return Object.freeze({
+ kind: "ArrayJoinNode",
+ variant,
+ items: Object.freeze([...items]),
+ });
+ },
+};
+
+/**
+ * ClickHouse `LIMIT n BY col1, col2, ...`.
+ *
+ * Distinct from Kysely's `LimitNode` (`LIMIT n`). ClickHouse places LIMIT BY
+ * after ORDER BY and before a plain LIMIT.
+ */
+export type LimitByNode = {
+ readonly kind: "LimitByNode";
+ readonly count: OperationNode;
+ readonly columns: ReadonlyArray<OperationNode>;
+};
+
+export const LimitByNode = {
+ is(node: { kind: string }): node is LimitByNode {
+ return node.kind === "LimitByNode";
+ },
+ create(
+ count: OperationNode,
+ columns: ReadonlyArray<OperationNode>,
+ ): LimitByNode {
+ return Object.freeze({
+ kind: "LimitByNode",
+ count,
+ columns: Object.freeze([...columns]),
+ });
+ },
+};
+
+/**
+ * ClickHouse array subscript `arr[index]`. Children are traced nodes
+ * (typically a column ref and an `indexOf(...)` FunctionNode), not raw SQL.
+ *
+ * Same closed-`kind` constraint as ARRAY JOIN: not a Kysely visitor-map
+ * kind. The dialect compiler and transformer special-case this object.
+ */
+export type ArrayIndexNode = {
+ readonly kind: "ArrayIndexNode";
+ readonly array: OperationNode;
+ readonly index: OperationNode;
+};
+
+export const ArrayIndexNode = {
+ is(node: { kind: string }): node is ArrayIndexNode {
+ return node.kind === "ArrayIndexNode";
+ },
+ create(array: OperationNode, index: OperationNode): ArrayIndexNode {
+ return Object.freeze({
+ kind: "ArrayIndexNode",
+ array,
+ index,
+ });
+ },
+};
+
+/**
+ * Select query extended with ClickHouse-only clauses.
+ * Extra fields are ignored by Kysely's default compiler/transformer; our
+ * dialect compiler and {@link ClickHouseOperationNodeTransformer} are the
+ * ones that read them. Tenancy is stamped by identity (WeakSet), not a
+ * copyable property on this node.
+ */
+export type ClickHouseSelectQueryNode = SelectQueryNode & {
+ readonly arrayJoins?: ReadonlyArray<ArrayJoinNode>;
+ readonly limitBy?: LimitByNode;
+};
+
+export function isClickHouseSelectQueryNode(
+ node: OperationNode,
+): node is ClickHouseSelectQueryNode {
+ return node.kind === "SelectQueryNode";
+}
diff --git a/packages/shared/src/server/query-ast/kysely/schema.test.ts b/packages/shared/src/server/query-ast/kysely/schema.test.ts
new file mode 100644
index 0000000000..815486231c
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/schema.test.ts
@@ -0,0 +1,36 @@
+import { describe, expect, it } from "vitest";
+
+import { COLUMN_DATA_TYPES, TENANTED_TABLES } from "./schema";
+import { schemaTypeAssertions, extensionTypeAssertions } from "./types.assert";
+
+describe("table registry derivation", () => {
+ it("derives the tenanted table set from the registry", () => {
+ // Only relations in the registry are scoped. `events_full` is just as
+ // project-gated as `events_core` and would be `tenant: true` if modeled β€”
+ // it is absent only because the builder targets `events_core` (the MV
+ // projection) and never selects `events_full`. Since it is not in the row
+ // type and raw sources are rejected, it is unreachable, so it needs no
+ // entry. Add one (tenant defaults true) the day a query selects it.
+ expect([...TENANTED_TABLES].sort()).toEqual([
+ "events_core",
+ "observations",
+ "scores",
+ "traces",
+ ]);
+ });
+
+ it("derives coarse runtime column types for the type-check pass", () => {
+ expect(COLUMN_DATA_TYPES.total_cost).toBe("number");
+ expect(COLUMN_DATA_TYPES.environment).toBe("string");
+ expect(COLUMN_DATA_TYPES.timestamp).toBe("date");
+ expect(COLUMN_DATA_TYPES.metadata_names).toBe("array");
+ expect(COLUMN_DATA_TYPES.cost_details).toBe("map");
+ });
+
+ // The assertions themselves are compile-time (`tsc` is the test); this only
+ // anchors the file so its `@ts-expect-error` checks stay in the build graph.
+ it("keeps the compile-time type assertions in the build graph", () => {
+ expect(typeof schemaTypeAssertions).toBe("function");
+ expect(typeof extensionTypeAssertions).toBe("function");
+ });
+});
diff --git a/packages/shared/src/server/query-ast/kysely/schema.ts b/packages/shared/src/server/query-ast/kysely/schema.ts
new file mode 100644
index 0000000000..0fd812f599
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/schema.ts
@@ -0,0 +1,140 @@
+/**
+ * Single source of truth for the physical ClickHouse relations the Kysely
+ * compiler targets. One table declaration drives all three downstream views:
+ *
+ * - `ClickHouseDatabase` β€” the Kysely row types (column autocomplete and typed
+ * comparisons).
+ * - `COLUMN_DATA_TYPES` β€” the coarse runtime type map the aggregate type-check
+ * pass consults (`typecheck.ts`).
+ * - `TENANTED_TABLES` β€” the relations the tenancy pass must scope
+ * (`tenancy.ts`).
+ *
+ * Deriving all three from one declaration keeps them from drifting apart.
+ *
+ * Column sets cover the relations the compiler targets, not a full schema
+ * dump. Physical tuning metadata (partition / sort / dedup keys) is not
+ * modeled here; dedup lowering will extend this same declaration.
+ */
+
+/**
+ * ClickHouse column types the registry understands. Each maps to a TS row type
+ * (for Kysely) and a coarse runtime category (for the type-check pass).
+ */
+export type ChColumnType =
+ | "String"
+ | "Float"
+ | "DateTime"
+ | "Array(String)"
+ | "Map(String, Float)";
+
+type TsTypeOfColumn<T extends ChColumnType> = T extends "String"
+ ? string
+ : T extends "Float"
+ ? number
+ : T extends "DateTime"
+ ? Date
+ : T extends "Array(String)"
+ ? string[]
+ : T extends "Map(String, Float)"
+ ? Record<string, number>
+ : never;
+
+/** Coarse runtime category the aggregate type-check pass reasons about. */
+export type ColumnDataType = "string" | "number" | "date" | "array" | "map";
+
+const RUNTIME_TYPE: Record<ChColumnType, ColumnDataType> = {
+ String: "string",
+ Float: "number",
+ DateTime: "date",
+ "Array(String)": "array",
+ "Map(String, Float)": "map",
+};
+
+function defineTable<const Cols extends Record<string, ChColumnType>>(spec: {
+ columns: Cols;
+ /**
+ * Whether the tenancy pass must inject a `project_id` scope. Defaults to
+ * `true`: every relation modeled here is project-gated, and defaulting on is
+ * fail-closed β€” a table added without this flag is still scoped. Set `false`
+ * only for a genuinely global relation.
+ */
+ tenant?: boolean;
+}) {
+ return { columns: spec.columns, tenant: spec.tenant ?? true };
+}
+
+const TABLE_REGISTRY = {
+ traces: defineTable({
+ columns: {
+ environment: "String",
+ project_id: "String",
+ timestamp: "DateTime",
+ id: "String",
+ },
+ }),
+ observations: defineTable({
+ columns: {
+ environment: "String",
+ project_id: "String",
+ start_time: "DateTime",
+ trace_id: "String",
+ cost_details: "Map(String, Float)",
+ usage_details: "Map(String, Float)",
+ },
+ }),
+ events_core: defineTable({
+ columns: {
+ environment: "String",
+ project_id: "String",
+ start_time: "DateTime",
+ span_id: "String",
+ trace_id: "String",
+ event_ts: "DateTime",
+ type: "String",
+ total_cost: "Float",
+ metadata_names: "Array(String)",
+ metadata_values: "Array(String)",
+ },
+ }),
+ scores: defineTable({
+ columns: {
+ environment: "String",
+ project_id: "String",
+ timestamp: "DateTime",
+ data_type: "String",
+ },
+ }),
+} as const;
+
+type RowOf<Cols extends Record<string, ChColumnType>> = {
+ [C in keyof Cols]: TsTypeOfColumn<Cols[C]>;
+};
+
+/** Derived Kysely row types. */
+export type ClickHouseDatabase = {
+ [T in keyof typeof TABLE_REGISTRY]: RowOf<
+ (typeof TABLE_REGISTRY)[T]["columns"]
+ >;
+};
+
+/**
+ * Derived coarse runtime column-type map (column name -> category), consumed by
+ * the aggregate type-check pass. Column names are consistent across tables in
+ * this schema, so a flat map is unambiguous.
+ */
+export const COLUMN_DATA_TYPES: Record<string, ColumnDataType> =
+ Object.fromEntries(
+ Object.values(TABLE_REGISTRY).flatMap((table) =>
+ Object.entries(table.columns).map(([name, chType]) => [
+ name,
+ RUNTIME_TYPE[chType],
+ ]),
+ ),
+ );
+
+/** Derived set of relations the tenancy pass must scope. */
+export const TENANTED_TABLES = new Set<string>(
+ Object.entries(TABLE_REGISTRY)
+ .filter(([, spec]) => spec.tenant)
+ .map(([name]) => name),
+);
diff --git a/packages/shared/src/server/query-ast/kysely/tenancy.ts b/packages/shared/src/server/query-ast/kysely/tenancy.ts
new file mode 100644
index 0000000000..60dbd7fb83
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/tenancy.ts
@@ -0,0 +1,345 @@
+import {
+ AliasNode,
+ AndNode,
+ BinaryOperationNode,
+ ColumnNode,
+ IdentifierNode,
+ JoinNode,
+ OperatorNode,
+ OrNode,
+ ParensNode,
+ RawNode,
+ ReferenceNode,
+ SelectQueryNode,
+ TableNode,
+ ValueNode,
+ WhereNode,
+ type KyselyPlugin,
+ type OperationNode,
+ type PluginTransformQueryArgs,
+ type PluginTransformResultArgs,
+ type QueryId,
+ type RootOperationNode,
+} from "kysely";
+
+import type { ExecutionContext } from "../executionContext";
+import { QueryCompileError, UnscopedRelationError } from "./errors";
+import { type ClickHouseSelectQueryNode } from "./nodes";
+import { TENANTED_TABLES } from "./schema";
+import { ClickHouseOperationNodeTransformer } from "./transformer";
+
+const PROJECT_ID_COLUMN = "project_id";
+
+/**
+ * Identity-based stamp. A copied `langfuseTenancy` property on a cloned node
+ * is not enough β€” only trees that actually went through
+ * {@link TenancyInjectionPlugin} are in this set.
+ */
+const TENANCY_STAMPED = new WeakSet<object>();
+
+type Relation =
+ | { kind: "table"; tableName: string; alias?: string }
+ | { kind: "subquery"; node: SelectQueryNode }
+ | { kind: "raw" }
+ | { kind: "other" };
+
+/**
+ * Mandatory tenancy injection. Attaches `project_id = {projectId}` to every
+ * tenanted physical relation that does not already have that predicate, then
+ * stamps the tree so {@link ClickHouseQueryCompiler} will compile it.
+ *
+ * Any `RawNode` whose SQL fragments introduce a relation (`SELECT` / `FROM` /
+ * `JOIN`) is rejected β€” not only FROM/JOIN table sources. Kysely's own
+ * keyword fragments (`asc` / `desc`) are not relations and are allowed.
+ */
+export class TenancyInjectionPlugin implements KyselyPlugin {
+ constructor(private readonly ctx: ExecutionContext) {
+ if (!ctx?.projectId) {
+ throw new QueryCompileError(
+ "ExecutionContext.projectId is required for tenancy injection",
+ );
+ }
+ }
+
+ transformQuery(args: PluginTransformQueryArgs): RootOperationNode {
+ const transformer = new TenancyInjectionTransformer(this.ctx);
+ const injected = transformer.transformNode(args.node) as RootOperationNode;
+ rejectUnscopedRawSql(injected);
+ return stampTenancy(injected);
+ }
+
+ async transformResult(args: PluginTransformResultArgs) {
+ return args.result;
+ }
+}
+
+class TenancyInjectionTransformer extends ClickHouseOperationNodeTransformer {
+ constructor(private readonly ctx: ExecutionContext) {
+ super();
+ }
+
+ protected override transformSelectQuery(
+ node: SelectQueryNode,
+ queryId?: QueryId,
+ ): SelectQueryNode {
+ const withChildren = super.transformSelectQuery(
+ node,
+ queryId,
+ ) as ClickHouseSelectQueryNode;
+ return injectSelect(withChildren, this.ctx, collectCteNames(withChildren));
+ }
+}
+
+function collectCteNames(node: SelectQueryNode): Set<string> {
+ const names = new Set<string>();
+ for (const expr of node.with?.expressions ?? []) {
+ names.add(tableNameOf(expr.name.table));
+ }
+ return names;
+}
+
+function injectSelect(
+ node: ClickHouseSelectQueryNode,
+ ctx: ExecutionContext,
+ cteNames: Set<string>,
+): ClickHouseSelectQueryNode {
+ const fromRelations = (node.from?.froms ?? []).map(describeRelation);
+ const joinRelations = (node.joins ?? []).map((join) => ({
+ join,
+ relation: describeRelation(join.table),
+ }));
+
+ for (const relation of fromRelations) {
+ assertRelationAllowed(relation);
+ }
+ for (const { relation } of joinRelations) {
+ assertRelationAllowed(relation);
+ }
+
+ const isTenantedTable = (
+ r: Relation,
+ ): r is Extract<Relation, { kind: "table" }> =>
+ r.kind === "table" &&
+ TENANTED_TABLES.has(r.tableName) &&
+ !cteNames.has(r.tableName);
+
+ const tenantedFrom = fromRelations.filter(isTenantedTable);
+ const tenantedJoinCount = joinRelations.filter(({ relation }) =>
+ isTenantedTable(relation),
+ ).length;
+
+ // When more than one tenanted relation is in scope, an unqualified
+ // `project_id = …` predicate is ambiguous β€” it cannot be proven to scope a
+ // *specific* relation β€” so a table-qualified reference is required before a
+ // relation counts as already covered. With a single tenanted relation an
+ // unqualified reference is unambiguous and accepted.
+ const requireQualified = tenantedFrom.length + tenantedJoinCount > 1;
+
+ let where = node.where;
+ for (const table of tenantedFrom) {
+ if (
+ !predicateCovers(where?.where, table, ctx.projectId, requireQualified)
+ ) {
+ const predicate = projectIdPredicate(table, ctx.projectId);
+ // Prepend: the tenancy scope is the leading WHERE predicate, so its bound
+ // value keeps a stable (first) parameter position regardless of the other
+ // predicates a caller wrote.
+ where = where
+ ? WhereNode.create(AndNode.create(predicate, where.where))
+ : WhereNode.create(predicate);
+ }
+ }
+
+ const joins = joinRelations.map(({ join, relation }) => {
+ if (!isTenantedTable(relation)) {
+ return join;
+ }
+ const onExpr = join.on?.on;
+ if (
+ predicateCovers(onExpr, relation, ctx.projectId, requireQualified) ||
+ predicateCovers(where?.where, relation, ctx.projectId, requireQualified)
+ ) {
+ return join;
+ }
+ const predicate = projectIdPredicate(relation, ctx.projectId);
+ if (!join.on) {
+ return JoinNode.createWithOn(join.joinType, join.table, predicate);
+ }
+ return JoinNode.cloneWithOn(join, predicate);
+ });
+
+ return {
+ ...node,
+ ...(where ? { where } : {}),
+ ...(joins.length ? { joins } : {}),
+ };
+}
+
+function assertRelationAllowed(relation: Relation): void {
+ if (relation.kind === "raw") {
+ throw new UnscopedRelationError(
+ "Raw SQL table sources are rejected: they can introduce an unscoped relation that the tenancy pass cannot prove. Use a traced table reference instead.",
+ );
+ }
+}
+
+function describeRelation(node: OperationNode): Relation {
+ if (RawNode.is(node)) {
+ return { kind: "raw" };
+ }
+ if (AliasNode.is(node)) {
+ const inner = describeRelation(node.node);
+ if (inner.kind === "table") {
+ const alias = identifierName(node.alias);
+ return { kind: "table", tableName: inner.tableName, alias };
+ }
+ return inner;
+ }
+ if (TableNode.is(node)) {
+ return { kind: "table", tableName: tableNameOf(node) };
+ }
+ if (SelectQueryNode.is(node)) {
+ return { kind: "subquery", node };
+ }
+ return { kind: "other" };
+}
+
+function tableNameOf(node: TableNode): string {
+ return node.table.identifier.name;
+}
+
+function identifierName(node: OperationNode): string | undefined {
+ if (IdentifierNode.is(node)) return node.name;
+ return undefined;
+}
+
+function projectIdPredicate(
+ table: Extract<Relation, { kind: "table" }>,
+ projectId: string,
+): OperationNode {
+ const column = ColumnNode.create(PROJECT_ID_COLUMN);
+ const left = table.alias
+ ? ReferenceNode.create(column, TableNode.create(table.alias))
+ : column;
+ return BinaryOperationNode.create(
+ left,
+ OperatorNode.create("="),
+ ValueNode.create(projectId),
+ );
+}
+
+/**
+ * True only when `expr` provably constrains `table` to `projectId`. Both halves
+ * matter: the left operand must be `table`'s `project_id` column (qualified when
+ * {@link requireQualified}, see {@link injectSelect}) *and* the right operand
+ * must be the literal `projectId` from the {@link ExecutionContext}. A predicate
+ * such as `project_id = <someOtherProject>` or `o.project_id = t.project_id`
+ * does not scope the relation to the request's project, so it is not covered and
+ * the pass injects the correct predicate.
+ */
+function predicateCovers(
+ expr: OperationNode | undefined,
+ table: Extract<Relation, { kind: "table" }>,
+ projectId: string,
+ requireQualified: boolean,
+): boolean {
+ if (!expr) return false;
+ if (AndNode.is(expr)) {
+ return (
+ predicateCovers(expr.left, table, projectId, requireQualified) ||
+ predicateCovers(expr.right, table, projectId, requireQualified)
+ );
+ }
+ if (OrNode.is(expr)) {
+ return (
+ predicateCovers(expr.left, table, projectId, requireQualified) &&
+ predicateCovers(expr.right, table, projectId, requireQualified)
+ );
+ }
+ if (ParensNode.is(expr)) {
+ return predicateCovers(expr.node, table, projectId, requireQualified);
+ }
+ if (!BinaryOperationNode.is(expr)) return false;
+ if (!OperatorNode.is(expr.operator) || expr.operator.operator !== "=") {
+ return false;
+ }
+ return (
+ isProjectIdColumn(expr.leftOperand, table, requireQualified) &&
+ isProjectIdValue(expr.rightOperand, projectId)
+ );
+}
+
+function isProjectIdColumn(
+ node: OperationNode,
+ table: Extract<Relation, { kind: "table" }>,
+ requireQualified: boolean,
+): boolean {
+ if (ColumnNode.is(node)) {
+ // Bare column with no table qualifier: accept only when unambiguous.
+ return !requireQualified && node.column.name === PROJECT_ID_COLUMN;
+ }
+ if (ReferenceNode.is(node) && ColumnNode.is(node.column)) {
+ if (node.column.column.name !== PROJECT_ID_COLUMN) return false;
+ if (!node.table) return !requireQualified;
+ const referenced = tableNameOf(node.table);
+ // Once a relation is aliased, its physical table name is no longer a valid
+ // SQL qualifier for it β€” only the alias is. Accepting the physical name
+ // here would let one relation's predicate "cover" a *different* relation
+ // whose physical name collides with this alias (e.g. `scores AS traces`
+ // joined to `traces AS t`), leaving the second relation unscoped. Mirror
+ // projectIdPredicate: alias when present, else the table name.
+ return table.alias
+ ? referenced === table.alias
+ : referenced === table.tableName;
+ }
+ return false;
+}
+
+function isProjectIdValue(node: OperationNode, projectId: string): boolean {
+ return ValueNode.is(node) && node.value === projectId;
+}
+
+export function requireExecutionContext(
+ ctx: ExecutionContext | undefined,
+): ExecutionContext {
+ if (!ctx?.projectId) {
+ throw new QueryCompileError(
+ "ExecutionContext is required: a query with no tenancy scope cannot compile.",
+ );
+ }
+ return ctx;
+}
+
+function stampTenancy<T extends object>(node: T): T {
+ TENANCY_STAMPED.add(node);
+ return node;
+}
+
+export function assertTenancyStamped(node: RootOperationNode): void {
+ if (!TENANCY_STAMPED.has(node)) {
+ throw new QueryCompileError(
+ "Refusing to compile: the tenancy injection pass was not applied. Compile through compileClickhouseQuery() with an ExecutionContext.",
+ );
+ }
+}
+
+const RAW_RELATION_SQL = /\b(?:from|join|select)\b/i;
+
+function rejectUnscopedRawSql(node: unknown): void {
+ if (!node || typeof node !== "object") return;
+ if (Array.isArray(node)) {
+ for (const item of node) rejectUnscopedRawSql(item);
+ return;
+ }
+ if (RawNode.is(node as OperationNode)) {
+ const fragments = (node as RawNode).sqlFragments;
+ if (fragments.some((fragment) => RAW_RELATION_SQL.test(fragment))) {
+ throw new UnscopedRelationError(
+ "Raw SQL fragments that introduce a relation (SELECT/FROM/JOIN) are rejected in any position: they can bypass tenancy injection. Use traced table and column references instead.",
+ );
+ }
+ }
+ for (const value of Object.values(node)) {
+ rejectUnscopedRawSql(value);
+ }
+}
diff --git a/packages/shared/src/server/query-ast/kysely/transformer.ts b/packages/shared/src/server/query-ast/kysely/transformer.ts
new file mode 100644
index 0000000000..732b6ee430
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/transformer.ts
@@ -0,0 +1,84 @@
+import {
+ OperationNodeTransformer,
+ type OperationNode,
+ type QueryId,
+ type SelectQueryNode,
+} from "kysely";
+
+import {
+ ArrayJoinNode,
+ ArrayIndexNode,
+ LimitByNode,
+ type ClickHouseSelectQueryNode,
+} from "./nodes";
+
+/**
+ * Transformer that knows the ClickHouse extra fields on SelectQueryNode.
+ *
+ * Kysely's default {@link OperationNodeTransformer.transformSelectQuery}
+ * reconstructs a SelectQueryNode from known slots and drops anything else β€”
+ * so without this override, ARRAY JOIN / LIMIT BY / the tenancy stamp would
+ * vanish the moment any plugin walked the tree.
+ */
+export class ClickHouseOperationNodeTransformer extends OperationNodeTransformer {
+ protected override transformNodeImpl<T extends OperationNode>(
+ node: T,
+ queryId?: QueryId,
+ ): T {
+ if (ArrayIndexNode.is(node)) {
+ // Anti-pattern: cast through `unknown`. ArrayIndexNode is a custom kind
+ // outside Kysely's closed OperationNode union, so `transformArrayIndex`'s
+ // return type cannot be proven assignable to the generic `T` the base
+ // method is parameterized over. The `.is` guard above establishes at
+ // runtime that `T` is ArrayIndexNode here, so the cast is sound.
+ return this.transformArrayIndex(node) as unknown as T;
+ }
+ return super.transformNodeImpl(node, queryId);
+ }
+
+ protected transformArrayIndex(node: ArrayIndexNode): ArrayIndexNode {
+ return ArrayIndexNode.create(
+ this.transformNode(node.array),
+ this.transformNode(node.index),
+ );
+ }
+
+ protected override transformSelectQuery(
+ node: SelectQueryNode,
+ queryId?: QueryId,
+ ): SelectQueryNode {
+ const transformed = super.transformSelectQuery(
+ node,
+ queryId,
+ ) as ClickHouseSelectQueryNode;
+ const extra = node as ClickHouseSelectQueryNode;
+ return {
+ ...transformed,
+ ...(extra.arrayJoins
+ ? {
+ arrayJoins: extra.arrayJoins.map((n) => this.transformArrayJoin(n)),
+ }
+ : {}),
+ ...(extra.limitBy
+ ? { limitBy: this.transformLimitBy(extra.limitBy) }
+ : {}),
+ } as ClickHouseSelectQueryNode;
+ }
+
+ protected transformArrayJoin(node: ArrayJoinNode): ArrayJoinNode {
+ return ArrayJoinNode.create(
+ node.items.map((item) => ({
+ expression: this.transformNode(item.expression),
+ alias: this.transformNode(item.alias) as typeof item.alias,
+ })),
+ node.variant,
+ );
+ }
+
+ protected transformLimitBy(node: LimitByNode): LimitByNode {
+ return LimitByNode.create(
+ this.transformNode(node.count),
+ node.columns.map((column) => this.transformNode(column)),
+ );
+ }
+}
diff --git a/packages/shared/src/server/query-ast/kysely/typecheck.ts b/packages/shared/src/server/query-ast/kysely/typecheck.ts
new file mode 100644
index 0000000000..2a4034ba7b
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/typecheck.ts
@@ -0,0 +1,61 @@
+import {
+ AggregateFunctionNode,
+ ColumnNode,
+ ReferenceNode,
+ type OperationNode,
+} from "kysely";
+
+import { TypeCompatibilityError } from "./errors";
+import { COLUMN_DATA_TYPES } from "./schema";
+
+// Kysely already type-checks comparisons against the schema; `sum()` / `avg()`
+// do not constrain the argument column, so this pass catches those. Column data
+// types come from the table registry in `schema.ts`.
+
+const NUMERIC_AGGREGATES = new Set(["sum", "avg"]);
+
+/**
+ * Runtime validation pass: numeric aggregates over non-numeric columns.
+ * Walks every node object (including ClickHouse extra fields).
+ */
+export function validateTypeCompatibility(root: OperationNode): void {
+ walk(root, (node) => {
+ if (!AggregateFunctionNode.is(node)) return;
+ if (!NUMERIC_AGGREGATES.has(node.func.toLowerCase())) return;
+ for (const arg of node.aggregated) {
+ const column = columnNameOf(arg);
+ if (!column) continue;
+ const dataType = COLUMN_DATA_TYPES[column];
+ if (dataType && dataType !== "number") {
+ throw new TypeCompatibilityError(
+ `Aggregate ${node.func}() is not compatible with ${dataType} column "${column}"`,
+ );
+ }
+ }
+ });
+}
+
+function columnNameOf(node: OperationNode): string | undefined {
+ if (ColumnNode.is(node)) return node.column.name;
+ if (ReferenceNode.is(node) && ColumnNode.is(node.column)) {
+ return node.column.column.name;
+ }
+ return undefined;
+}
+
+function walk(value: unknown, visit: (node: OperationNode) => void): void {
+ if (!value || typeof value !== "object") return;
+ if (
+ "kind" in value &&
+ typeof (value as { kind: unknown }).kind === "string"
+ ) {
+ visit(value as OperationNode);
+ }
+ for (const child of Object.values(value)) {
+ if (Array.isArray(child)) {
+ for (const item of child) walk(item, visit);
+ } else {
+ walk(child, visit);
+ }
+ }
+}
diff --git a/packages/shared/src/server/query-ast/kysely/types.assert.ts b/packages/shared/src/server/query-ast/kysely/types.assert.ts
new file mode 100644
index 0000000000..4083d53e50
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/types.assert.ts
@@ -0,0 +1,88 @@
+import { type InferResult } from "kysely";
+
+import { getClickhouseKysely } from "./dialect";
+import { arrayJoin, limitBy, mapKeys, mapValues } from "./extensions";
+import { defineView, fromView } from "./views";
+
+/**
+ * Compile-time assertions that the typed schema and virtual views enforce their
+ * guarantees at the type layer. Never called at runtime; `tsc` is the test, and
+ * every `@ts-expect-error` must stay a real (unused-error-free) error.
+ *
+ * Which layer catches what:
+ * - DateTime vs Int comparison: TS types (Kysely OperandValueExpression)
+ * - unknown column name: TS types
+ * - view exposing only declared columns: TS types
+ * - sum() over a String column: NOT caught by TS (Kysely's sum() argument
+ * is any ReferenceExpression) β€” caught by the runtime validation pass
+ * - column-name auto-complete: yes, from ClickHouseDatabase schema keys
+ */
+export function schemaTypeAssertions(): void {
+ const db = getClickhouseKysely();
+
+ db.selectFrom("events_core")
+ .select("environment")
+ // @ts-expect-error DateTime column compared to Int
+ .where("start_time", "=", 123);
+
+ db.selectFrom("events_core")
+ .select("environment")
+ .where("start_time", ">=", new Date("2026-01-01T00:00:00.000Z"));
+
+ db.selectFrom("events_core").select((eb) => eb.fn.sum("total_cost").as("s"));
+
+ // Intentionally valid at the TS layer β€” Kysely does not constrain sum()
+ // to numeric columns. The runtime pass in typecheck.ts rejects this.
+ db.selectFrom("events_core").select((eb) => eb.fn.sum("environment").as("s"));
+
+ db.selectFrom("events_core")
+ // @ts-expect-error unknown column is not in the schema
+ .select("not_a_column");
+
+ const view = defineView("environments_view")<{ environment: string }>(() =>
+ db.selectFrom("events_core").select("environment").distinct(),
+ );
+ fromView(view).select("environment");
+ fromView(view)
+ // @ts-expect-error view is a black box: inner columns are not exposed
+ .select("span_id");
+}
+
+/**
+ * Compile-time assertions that the `$call` extension helpers infer correctly:
+ * `arrayJoin` widens the output row type with each declared alias, and `limitBy`
+ * preserves it. `tsc` is the test.
+ */
+export function extensionTypeAssertions(): void {
+ const db = getClickhouseKysely();
+
+ const _widened = db
+ .selectFrom("observations")
+ .select("environment")
+ .$call(
+ arrayJoin({
+ cost_key: mapKeys("cost_details"),
+ cost: mapValues("cost_details"),
+ }),
+ );
+ type Widened = InferResult<typeof _widened>[number];
+ // Each arrayJoin alias is added to the output row, so it is referenceable
+ // downstream (e.g. an outer query over a CTE body) and a typo on the alias
+ // name is a compile error (indexing an absent key fails). The element value
+ // type is opaque β€” Kysely's Expression hides its type arg from inference β€”
+ // so it surfaces as unknown.
+ type _Key = Widened["cost_key"];
+ type _Cost = Widened["cost"];
+ // @ts-expect-error an alias the arrayJoin did not declare is absent
+ type _Missing = Widened["not_joined"];
+
+ // limitBy preserves the row type: it declares no new columns.
+ const _limited = db
+ .selectFrom("events_core")
+ .select("span_id")
+ .$call(limitBy({ count: 1, columns: ["span_id"] }));
+ type Limited = InferResult<typeof _limited>[number];
+ type _Span = Limited["span_id"];
+ // @ts-expect-error limitBy adds no columns
+ type _NoExtra = Limited["cost_key"];
+}
diff --git a/packages/shared/src/server/query-ast/kysely/views.ts b/packages/shared/src/server/query-ast/kysely/views.ts
new file mode 100644
index 0000000000..103b80dc39
--- /dev/null
+++ b/packages/shared/src/server/query-ast/kysely/views.ts
@@ -0,0 +1,110 @@
+import {
+ AliasNode,
+ CommonTableExpressionNameNode,
+ CommonTableExpressionNode,
+ Kysely,
+ TableNode,
+ WithNode,
+ type KyselyPlugin,
+ type OperationNode,
+ type PluginTransformQueryArgs,
+ type PluginTransformResultArgs,
+ type RootOperationNode,
+ type SelectQueryBuilder,
+ type SelectQueryNode,
+} from "kysely";
+
+import type { ClickhouseCompilable } from "./compile";
+import { ClickHouseCompileDialect } from "./dialect";
+import { ClickHouseOperationNodeTransformer } from "./transformer";
+
+export type ClickHouseView<N extends string, O> = {
+ name: N;
+ build: () => ClickhouseCompilable;
+ /** Phantom: the view's exposed row type. */
+ readonly $row?: O;
+};
+
+/**
+ * Name a compilable query as a black-box relation. The outer query can only
+ * see `O` (the view's selected columns), never the inner physical schema.
+ * Implemented as a plugin that rewrites `selectFrom(viewName)` into a WITH CTE;
+ * tenancy injection still walks the CTE body.
+ */
+export function defineView<N extends string>(name: N) {
+ return <O>(build: () => ClickhouseCompilable): ClickHouseView<N, O> => ({
+ name,
+ build,
+ });
+}
+
+class VirtualViewPlugin implements KyselyPlugin {
+ private readonly views: Map<string, ClickHouseView<string, unknown>>;
+
+ constructor(
+ views: ClickHouseView<string, unknown> | ClickHouseView<string, unknown>[],
+ ) {
+ const list = Array.isArray(views) ? views : [views];
+ this.views = new Map(list.map((view) => [view.name, view]));
+ }
+
+ transformQuery(args: PluginTransformQueryArgs): RootOperationNode {
+ if (args.node.kind !== "SelectQueryNode") return args.node;
+ const transformer = new ClickHouseOperationNodeTransformer();
+ const node = transformer.transformNode(args.node) as SelectQueryNode;
+
+ const froms = node.from?.froms ?? [];
+ const viewNames = froms
+ .map(tableNameOfFrom)
+ .filter((name): name is string => Boolean(name && this.views.has(name)));
+
+ if (viewNames.length === 0) return node;
+
+ let withNode = node.with;
+ for (const name of viewNames) {
+ const def = this.views.get(name)!;
+ const inner = def.build().toOperationNode();
+ const cte = CommonTableExpressionNode.create(
+ CommonTableExpressionNameNode.create(name),
+ inner,
+ );
+ withNode = withNode
+ ? WithNode.cloneWithExpression(withNode, cte)
+ : WithNode.create(cte);
+ }
+
+ return { ...node, with: withNode };
+ }
+
+ async transformResult(args: PluginTransformResultArgs) {
+ return args.result;
+ }
+}
+
+function tableNameOfFrom(node: OperationNode): string | undefined {
+ if (TableNode.is(node)) {
+ return node.table.identifier.name;
+ }
+ if (AliasNode.is(node)) {
+ return tableNameOfFrom(node.node);
+ }
+ return undefined;
+}
+
+/**
+ * Open a query against a named view as if it were a table. Only the view's
+ * exposed columns are in the type.
+ */
+export function fromView<N extends string, O>(
+ view: ClickHouseView<N, O>,
+): SelectQueryBuilder<Record<N, O>, N, Record<string, never>> {
+ type DB = Record<N, O>;
+ const db = new Kysely<DB>({ dialect: new ClickHouseCompileDialect() });
+ return db
+ .withPlugin(new VirtualViewPlugin(view as ClickHouseView<string, unknown>))
+ .selectFrom(view.name) as SelectQueryBuilder<
+ Record<N, O>,
+ N,
+ Record<string, never>
+ >;
+}
diff --git a/packages/shared/src/server/repositories/environments.ts b/packages/shared/src/server/repositories/environments.ts
index 42d80140b9..ffbd67982d 100644
--- a/packages/shared/src/server/repositories/environments.ts
+++ b/packages/shared/src/server/repositories/environments.ts
@@ -1,5 +1,8 @@
import { LISTABLE_SCORE_TYPES } from "../../domain/scores";
import { env } from "../../env";
+import type { ExecutionContext } from "../query-ast/executionContext";
+import { compileClickhouseQuery } from "../query-ast/kysely/compile";
+import { getClickhouseKysely } from "../query-ast/kysely/dialect";
import { queryClickhouse } from "./clickhouse";
export type EnvironmentFilterProps = {
@@ -11,54 +14,69 @@ export const getEnvironmentsForProject = async (
props: EnvironmentFilterProps,
): Promise<{ environment: string }[]> => {
const { projectId, fromTimestamp } = props;
+ const writeMode = env.LANGFUSE_MIGRATION_V4_WRITE_MODE;
+ const db = getClickhouseKysely();
+ // Tenancy is applied at compile time: compileClickhouseQuery injects
+ // `project_id = {projectId}` into every tenanted relation and refuses to
+ // compile an unscoped query, so the query bodies below carry no project_id
+ // filter of their own.
+ const ctx: ExecutionContext = { projectId };
- // In dual and events_only write modes all tracing data lands in the events
- // tables: a single events_core scan covers traces and observations and is
- // the only populated source under events_only. Scores keep their own table
- // in every write mode. The events read may be routed to a dedicated
- // ClickHouse service (CLICKHOUSE_EVENTS_READ_ONLY_URL), so it cannot share
- // a query with the scores read.
- const tracingEnvironmentsPromise =
- env.LANGFUSE_MIGRATION_V4_WRITE_MODE === "legacy"
- ? queryClickhouse<{ environment: string }>({
- query: `
- (
- SELECT distinct environment
- FROM traces
- WHERE project_id = {projectId: String}
- ${fromTimestamp ? "AND timestamp >= {fromTimestamp: DateTime64(3)}" : ""}
- ) UNION ALL (
- SELECT distinct environment
- FROM observations
- WHERE project_id = {projectId: String}
- ${fromTimestamp ? "AND start_time >= {fromTimestamp: DateTime64(3)}" : ""}
- )
- `,
- params: { projectId, fromTimestamp },
- tags: { projectId, route: "environments.getEnvironmentsForProject" },
- preferredClickhouseService: "ReadOnly",
- })
- : queryClickhouse<{ environment: string }>({
- query: `
- SELECT distinct environment
- FROM events_core
- WHERE project_id = {projectId: String}
- ${fromTimestamp ? "AND start_time >= {fromTimestamp: DateTime64(3)}" : ""}
- `,
- params: { projectId, fromTimestamp },
- tags: { projectId, route: "environments.getEnvironmentsForProject" },
- preferredClickhouseService: "EventsReadOnly",
- });
+ // In dual/events_only write modes all tracing data lands in events_core (the
+ // only populated source under events_only), so one scan covers traces and
+ // observations. Legacy still reads the two separate tables and unions them.
+ const tracingQuery =
+ writeMode === "legacy"
+ ? db
+ .selectFrom("traces")
+ .select("environment")
+ .distinct()
+ .$if(fromTimestamp != null, (qb) =>
+ qb.where("timestamp", ">=", fromTimestamp!),
+ )
+ .unionAll(
+ db
+ .selectFrom("observations")
+ .select("environment")
+ .distinct()
+ .$if(fromTimestamp != null, (qb) =>
+ qb.where("start_time", ">=", fromTimestamp!),
+ ),
+ )
+ : db
+ .selectFrom("events_core")
+ .select("environment")
+ .distinct()
+ .$if(fromTimestamp != null, (qb) =>
+ qb.where("start_time", ">=", fromTimestamp!),
+ );
+
+ const scoresQuery = db
+ .selectFrom("scores")
+ .select("environment")
+ .distinct()
+ .where("data_type", "in", [...LISTABLE_SCORE_TYPES])
+ .$if(fromTimestamp != null, (qb) =>
+ qb.where("timestamp", ">=", fromTimestamp!),
+ );
+
+ const tracing = compileClickhouseQuery(tracingQuery, ctx);
+ const scores = compileClickhouseQuery(scoresQuery, ctx);
+
+ // The events read may route to a dedicated ClickHouse service
+ // (CLICKHOUSE_EVENTS_READ_ONLY_URL) while scores always read the primary, so
+ // the two cannot share one query.
+ const tracingEnvironmentsPromise = queryClickhouse<{ environment: string }>({
+ query: tracing.sql,
+ params: tracing.params,
+ tags: { projectId, route: "environments.getEnvironmentsForProject" },
+ preferredClickhouseService:
+ writeMode === "legacy" ? "ReadOnly" : "EventsReadOnly",
+ });
const scoreEnvironmentsPromise = queryClickhouse<{ environment: string }>({
- query: `
- SELECT distinct environment
- FROM scores
- WHERE project_id = {projectId: String}
- AND data_type IN ({dataTypes: Array(String)})
- ${fromTimestamp ? "AND timestamp >= {fromTimestamp: DateTime64(3)}" : ""}
- `,
- params: { projectId, fromTimestamp, dataTypes: LISTABLE_SCORE_TYPES },
+ query: scores.sql,
+ params: scores.params,
tags: { projectId, route: "environments.getEnvironmentsForProject" },
preferredClickhouseService: "ReadOnly",
});
@@ -67,7 +85,8 @@ export const getEnvironmentsForProject = async (
await Promise.all([tracingEnvironmentsPromise, scoreEnvironmentsPromise])
).flat();
- // Always add default environment to list
+ // "default" always exists as a selectable environment even when no rows have
+ // been written under it, so it is not guaranteed to appear in the scan.
results.push({ environment: "default" });
return Array.from(new Set(results.map((e) => e.environment))).map(
diff --git a/pnpm-lock.yaml b/pnpm-lock.yaml
index 579771d605..5837a3dd22 100644
--- a/pnpm-lock.yaml
+++ b/pnpm-lock.yaml
@@ -380,6 +380,9 @@ importers:
jsonpath-plus:
specifier: 10.3.0
version: 10.3.0
+ kysely:
+ specifier: 0.28.17
+ version: 0.28.17
langfuse:
specifier: 3.38.4
version: 3.38.4
@@ -8659,6 +8662,10 @@ packages:
resolution: {integrity: sha512-Xq9nH7KlWZmXAtodXDDRE7vs6DU1gTU8zYDHDiWLSip45Egwq3plLHzPn27NgvzL2r1LMPC1vdqh98sQxtqj4A==}
+ resolution: {integrity: sha512-nbD8lB9EB3wNdMhOCdx5Li8DxnLbvKByylRLcJ1h+4SkrowVeECAyZlyiKMThF7xFdRz0jSQ2MoJr+wXux2y0Q==}
+ engines: {node: '>=20.0.0'}
+
resolution: {integrity: sha512-zBKVmQN/1oT5VWZUBYlWzvokIlkC/6mnpgr/2atMyTeAm+jR3ia7w2iJMjlrF5/oG8ukO1s8+LDRCzJpF1QeEA==}
engines: {node: '>=18'}
@@ -19584,6 +19591,8 @@ snapshots:
+
dependencies:
mustache: 4.2.0
diff --git a/turbo.json b/turbo.json
index 433fed3eb7..ce122ceaa7 100644
--- a/turbo.json
+++ b/turbo.json
@@ -5,7 +5,8 @@
"globalEnv": [
"NEXT_PUBLIC_LANGFUSE_BLOB_EXPORT_CUTOFF",
"NEXT_PUBLIC_LANGFUSE_BLOB_EXPORTER_CUTOFF",
- "NEXT_PUBLIC_LANGFUSE_ANALYTICS_EXPORTER_CUTOFF"
+ "NEXT_PUBLIC_LANGFUSE_ANALYTICS_EXPORTER_CUTOFF",
+ "CLICKHOUSE_BIN"
],
"envMode": "loose",
"tasks": {