🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
fix(query-ast): take the last value of a duplicated metadata key in metadataValue (#18808)
Niklas Semmler2c13b3fa2efd
diff --git a/packages/shared/AGENTS.md b/packages/shared/AGENTS.mdindex d03b0e40a0..456d721fbe 100644--- a/packages/shared/AGENTS.md+++ b/packages/shared/AGENTS.md@@ -47,7 +47,7 @@- Code evaluator dispatcher/error contract: `src/server/evals/codeEvalDispatcherTypes.ts`. Keep provider mappings, user-visible messages, and worker terminal-outcome classification aligned when adding an error code.- Dashboard/monitor query feature (data model + server-only builder/executor): `src/features/query/*`- Query-builder AST (server half, WIP): `src/server/query-ast/*` — the Kysely- ClickHouse dialect (ARRAY JOIN / LIMIT BY / FINAL / metadata indexOf nodes,+ ClickHouse dialect (ARRAY JOIN / LIMIT BY / FINAL / metadata subscript nodes,`ExecutionContext` tenancy injection, per-table dedup lowering, virtual views,catalog parity). Compile only through `compileClickhouseQuery` in`src/server/query-ast/compile.ts`. SQL correctness is proven by a golden-SQLdiff --git a/packages/shared/src/server/query-ast/README.md b/packages/shared/src/server/query-ast/README.mdindex ce6f798df1..bed030cc7c 100644--- a/packages/shared/src/server/query-ast/README.md+++ b/packages/shared/src/server/query-ast/README.md@@ -8,7 +8,7 @@ metadata, `FilterState` embedding) will land under`packages/shared/src/query-ast/`.This folder is a compile-only ClickHouse dialect on Kysely 0.28: real-`OperationNode`s for ARRAY JOIN, LIMIT BY, FINAL, and metadata `indexOf`+`OperationNode`s for ARRAY JOIN, LIMIT BY, FINAL, and metadata mapsubscripts; a mandatory tenancy injection pass keyed on `ExecutionContext`;per-table dedup lowering from the registry; schema-typed selection; virtualviews as WITH CTEs; catalog parity.@@ -136,13 +136,13 @@ db.selectFrom("scores as s")// … from scores as s final```-**Select a metadata value** (`metadataValue` — lowers `metadata[key]` to a bound `indexOf` subscript):+**Select a metadata value** (`metadataValue` — lowers `metadata[key]` to a bound subscript of `metadataMap`, so the last duplicate key wins):```tsdb.selectFrom("events_core as e").select((eb) => [metadataValue("e", "my_key").as("my_val")])- .where((eb) => eb(metadataValue("e", "my_key"), ">", 2));-// select metadata_values[indexof(e.metadata_names, {p:String})] as my_val …+ .where((eb) => eb(metadataValue("e", "my_key"), "=", "x"));+// select mapfromarrays(arrayreverse(e.metadata_names), arrayreverse(e.metadata_values))[{p:String}] as my_val …```## Developing the dialect@@ -162,7 +162,7 @@ preserve it:| ARRAY JOIN | Plugin attaches `ArrayJoinNode` as an extra field on `SelectQueryNode`. `ClickHouseOperationNodeTransformer` preserves it. `ClickHouseQueryCompiler.visitSelectQuery` emits it after JOINs / before WHERE. | no || LIMIT BY | Plugin attaches `LimitByNode` the same way. Compiler emits it after ORDER BY / before LIMIT. | no || FINAL | Plugin wraps a FROM/JOIN table expression in `FinalTableNode`. Compiler emits `final` after that expression (`scores as s final`). `$call(useFinal(["scores"]))`. Tables declared `dedup: none` throw. | no |-| metadata `indexOf` | Helper builds an `ArrayIndexNode` whose index child is a `FunctionNode` (`indexOf`) over a bound `ValueNode` key. Transformer + compiler special-case the node. No plugin. | no |+| metadata subscript | Helper builds an `ArrayIndexNode` over the `metadataMap` `FunctionNode`, indexed by a bound `ValueNode` key. Transformer + compiler special-case the node. No plugin. | no || Virtual view | Plugin rewrites `selectFrom(viewName)` into a WITH CTE. Outer types only expose the view's selected columns. | no |These are real node objects whose children are traced Kyselydiff --git a/packages/shared/src/server/query-ast/catalog.golden.test.ts b/packages/shared/src/server/query-ast/catalog.golden.test.tsindex 22bb737d25..f9485f6c6b 100644--- a/packages/shared/src/server/query-ast/catalog.golden.test.ts+++ b/packages/shared/src/server/query-ast/catalog.golden.test.ts@@ -161,3 +161,22 @@ describeWithClickhouseLocal("catalog tier 5-7 execution", () => {});}});++describeWithClickhouseLocal("catalog metadata semantics", () => {+ it("metadata_group_by takes the last value of a duplicated metadata key", () => {+ const entry = CATALOG.find((e) => e.id === "metadata_group_by");+ expect(entry).toBeDefined();+ if (!entry) return;++ const compiled = compileClickhouseQuery(entry.build(), {+ projectId: CATALOG_PROJECT_ID,+ });+ const executable = substituteNamedParams(compiled.sql, compiled.params);+ const output = executeClickhouseLocal(`${CATALOG_MEMORY_DDL}+ INSERT INTO events_core (project_id, metadata_names, metadata_values)+ VALUES ('${CATALOG_PROJECT_ID}', ['model', 'other', 'model'], ['first', 'x', 'last']);+ ${executable}`);++ expect(output).toBe("last\t1");+ });+});diff --git a/packages/shared/src/server/query-ast/extensions.test.ts b/packages/shared/src/server/query-ast/extensions.test.tsindex 6ca2018da5..a0fd7c1845 100644--- a/packages/shared/src/server/query-ast/extensions.test.ts+++ b/packages/shared/src/server/query-ast/extensions.test.ts@@ -46,26 +46,26 @@ describe("schema-typed selection (condition 7)", () => {});});-describe("metadata indexOf access (condition 7b)", () => {+describe("metadata subscript access (condition 7b)", () => {// These tests use a two-layer strategy. This first test inspects the AST// itself (serialized to JSON) to prove `metadata[key]` is *lowered* to real- // operation nodes — an `ArrayIndexNode` subscript around an `indexOf(...)`+ // operation nodes — an `ArrayIndexNode` subscript over a `mapFromArrays(...)`// `FunctionNode` — and never a `RawNode` (a raw SQL string). The sibling// tests below assert the SQL the compiler then emits. Separating them// isolates "the builder produced the right tree" (composable, escaped,// parameter-bound) from "the compiler rendered that tree correctly".- it("lowers metadata[key] to ArrayIndexNode + indexOf FunctionNode, not RawNode", () => {+ it("lowers metadata[key] to ArrayIndexNode + mapFromArrays FunctionNode, not RawNode", () => {const qb = getClickhouseKysely().selectFrom("events_core as e").select((_eb) => [metadataValue("e", "a").as("a")])- .where((eb) => eb(metadataValue("e", "a"), ">", 2));+ .where((eb) => eb(metadataValue("e", "a"), "=", "x"));const node = qb.withPlugin(new TenancyInjectionPlugin(ctx)).toOperationNode();const json = JSON.stringify(node);expect(json).toContain('"kind":"ArrayIndexNode"');- expect(json).toContain('"func":"indexOf"');+ expect(json).toContain('"func":"mapFromArrays"');expect(json).not.toMatch(/"kind":"RawNode"/);// `.as("a")` wraps the expression in an AliasNode, so the first selection's@@ -80,15 +80,17 @@ describe("metadata indexOf access (condition 7b)", () => {const qb = getClickhouseKysely().selectFrom("events_core as e").select((_eb) => [metadataValue("e", "a").as("a")])- .where((eb) => eb(metadataValue("e", "a"), ">", 2));+ .where((eb) => eb(metadataValue("e", "a"), "=", "x"));const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);const lower = compiled.toLowerCase();- expect(lower).toContain("metadata_values[indexof(e.metadata_names");+ expect(lower).toContain(+ "mapfromarrays(arrayreverse(e.metadata_names), arrayreverse(e.metadata_values))[",+ );expect(lower).toContain("as a");expect(Object.values(params)).toContain("a");- expect(Object.values(params)).toContain(2);- expect(compiled).not.toMatch(/metadata_names,\s*'a'/);+ expect(Object.values(params)).toContain("x");+ expect(compiled).not.toMatch(/\['a'\]/);});it("emits the same lowering in HAVING", () => {@@ -96,12 +98,12 @@ describe("metadata indexOf access (condition 7b)", () => {.selectFrom("events_core as e").select("environment").groupBy("environment")- .having((eb) => eb(metadataValue("e", "a"), ">", 2));+ .having((eb) => eb(metadataValue("e", "a"), "=", "x"));const { sql: compiled, params } = compileClickhouseQuery(qb, ctx);expect(compiled.toLowerCase()).toContain("having");expect(compiled.toLowerCase()).toContain(- "metadata_values[indexof(e.metadata_names",+ "mapfromarrays(arrayreverse(e.metadata_names), arrayreverse(e.metadata_values))[",);expect(Object.values(params)).toContain("a");});diff --git a/packages/shared/src/server/query-ast/extensions.ts b/packages/shared/src/server/query-ast/extensions.tsindex 5e698c34ea..141ef0a373 100644--- a/packages/shared/src/server/query-ast/extensions.ts+++ b/packages/shared/src/server/query-ast/extensions.ts@@ -468,21 +468,21 @@ export function traceSessionId(): ChExpression<string> {}/**- * Lower `metadata[key]` to a traced array-subscript + `indexOf` node.- * The key is a bound `ValueNode`, not a SQL literal.+ * Lower `metadata[key]` to a subscript of {@link metadataMap}, so the last+ * entry of a duplicated key wins. The key is a bound `ValueNode`, not a SQL+ * literal.*/export function metadataValue(tableAlias: string,key: string,-): ExpressionWrapper<ClickHouseDatabase, "events_core", string | number> {- return new ExpressionWrapper(- ArrayIndexNode.create(- qualifiedColumn(tableAlias, "metadata_values"),- FunctionNode.create("indexOf", [- qualifiedColumn(tableAlias, "metadata_names"),- ValueNode.create(key),- ]),- ) as unknown as OperationNode,+): ChExpression<string> {+ const column = (name: string) =>+ new ExpressionWrapper<ClickHouseDatabase, "events_core", string[]>(+ qualifiedColumn(tableAlias, name),+ );+ return mapValue<string>(+ metadataMap(column("metadata_names"), column("metadata_values")),+ key,);}diff --git a/packages/shared/src/server/query-ast/fixtures/catalog.ts b/packages/shared/src/server/query-ast/fixtures/catalog.tsindex 54d78d9922..761ff1addc 100644--- a/packages/shared/src/server/query-ast/fixtures/catalog.ts+++ b/packages/shared/src/server/query-ast/fixtures/catalog.ts@@ -333,12 +333,12 @@ export const CATALOG: CatalogEntry[] = [},{// Metadata-map access in projection and GROUP BY. `metadata[key]` lowers to- // the `metadata_values[indexOf(metadata_names, {key})]` array-pair subscript- // (ArrayIndexNode), aliased and then grouped by that alias.+ // a subscript of the reversed name/value map (ArrayIndexNode), so the last+ // duplicate key wins, aliased and then grouped by that alias.id: "metadata_group_by",tier: 6,referenceSql: `- SELECT e.metadata_values[indexOf(e.metadata_names, {p1:String})] AS model, count(*) AS n+ SELECT mapFromArrays(arrayReverse(e.metadata_names), arrayReverse(e.metadata_values))[{p1:String}] AS model, count(*) AS nFROM events_core AS eWHERE e.project_id = {p2:String}GROUP BY modeldiff --git a/packages/shared/src/server/query-ast/nodes.ts b/packages/shared/src/server/query-ast/nodes.tsindex f5ef78b6ae..9f2f5c4493 100644--- a/packages/shared/src/server/query-ast/nodes.ts+++ b/packages/shared/src/server/query-ast/nodes.ts@@ -71,7 +71,7 @@ export const LimitByNode = {/*** ClickHouse array subscript `arr[index]`. Children are traced nodes- * (typically a column ref and an `indexOf(...)` FunctionNode), not raw SQL.+ * (typically a `mapFromArrays(...)` FunctionNode and a bound key), not raw SQL.** Same closed-`kind` constraint as ARRAY JOIN: not a Kysely visitor-map* kind. The dialect compiler and transformer special-case this object.