🐸 Rybbit - open-source and privacy-friendly alternative to Google Analytics that is 10x more intuitive.
Custom queries (#1006)
Goldflag94594fe4ca8b
diff --git a/SECURITY_SCAN_REPORT.md b/SECURITY_SCAN_REPORT.mdnew file mode 100644index 000000000..cd69459b7--- /dev/null+++ b/SECURITY_SCAN_REPORT.md@@ -0,0 +1,280 @@+# Rybbit Security Review — Final Report++## Executive Summary++This review consolidates 25 verified raw findings into **16 distinct issues** (overlapping reports merged).++| Severity | Count |+|----------|-------|+| High | 7 |+| Medium | 4 |+| Low | 5 |+| Info | 1 (latent) |++The High-severity findings cluster into two dominant themes:++1. **ClickHouse SQL injection** reachable on public or API-key-scoped analytics endpoints, enabling cross-tenant data exfiltration.+2. **CSRF / credentialed cross-site access** created by all-origins CORS reflection (`callback(null, true)` + `credentials: true`) combined with `SameSite=None` session cookies and no CSRF protection on the custom Fastify routes.++### Top 3 to fix first++1. **Fix the ClickHouse SQL injection vectors** (`getSqlParam` url_param:/utm_ branch and `getJourneys` stepFilters). These bypass schema validation and standard escaping, and are reachable with only public-site or scoped-key access. Validate against the allow-list before interpolation and use `SqlString.escape` for map keys / string literals.+2. **Lock down CORS + cookie CSRF posture.** Replace origin reflection with a trusted-origin allow-list, stop combining `credentials:true` with reflected origins, set `SameSite=Lax` (or add CSRF tokens / strict Origin checks if `None` is genuinely required for embedding), and populate `trustedOrigins` from the production `BASE_URL`.+3. **Authorize the GSC OAuth callback.** Add `getUserHasAdminAccessToSite(req, siteId)` after resolving `siteId` from `state`, and replace the plaintext `state=<siteId>` with a signed/server-stored single-use nonce binding `{userId, siteId}`.++---++## HIGH Severity++### H-1. ClickHouse SQL injection via unvalidated `parameter` on `/sites/:siteId/metric`+**CWE-89 (Injection)** — *Merged from findings #2 and #8.*++- **Sink:** `server/src/api/analytics/utils/getFilterStatement.ts:62-71`+- **Source:** `server/src/api/analytics/getMetric.ts:56` (reads `parameter` from querystring), `:354` (`getSqlParam(parameter)`)+- **Route:** `server/src/index.ts:268` — `fastify.get("/sites/:siteId/metric", publicSite, getMetric)`, no querystring schema++`getSqlParam()` builds a ClickHouse map accessor by raw string-templating the parameter key with no escaping. For `url_param:<name>` it returns `url_parameters['${paramName}']` and for `utm_<x>` it returns `url_parameters['${utm}']`. **Both branches `return` before the validating `filterParamSchema.parse(parameter)` at line 98**, so they never hit the enum allow-list. The result is interpolated directly into SELECT and WHERE positions in `getMetric.ts`. Because the route has no Fastify querystring schema and uses `publicSite`, `parameter` arrives fully attacker-controlled.++PoC payload breaks out of the map-key string literal:+```+parameter=url_param:x'] FROM events WHERE site_id=1 UNION SELECT password,1,1 FROM secret--+```++- **Impact:** A public-site or API/private-link-key requester can inject arbitrary SQL into the analytics query, enabling cross-tenant exfiltration (UNION dumping of other sites' events) and resource-exhausting queries. No authenticated session required for public sites.+- **Remediation:** Move `filterParamSchema.parse(parameter)` to the **top** of `getSqlParam` so the url_param:/utm_ branches cannot bypass it, OR escape the map key as `url_parameters[${SqlString.escape(paramName)}]` — matching the existing `feature_flag` branch (line 58) which already does `feature_flags[${SqlString.escape(key)}]`. Additionally add a Fastify querystring schema to the `/metric` route for edge validation.++---++### H-2. ClickHouse SQL injection in `/sites/:siteId/journeys` via `stepFilters`+**CWE-89 (Injection)** — *Merged from findings #3 and #14.*++- **File:** `server/src/api/analytics/getJourneys.ts:42-66` (parse + sink at line 64)+- **Route:** `server/src/index.ts:290` (`publicSite`)++`stepFilters` is `JSON.parse`d with **no Zod schema** into a path map. The exact-match branch builds `journey[${stepIndex}] = '${path.replace(/'/g, "''")}'` using only MySQL-style quote-doubling. ClickHouse also honors backslash escapes inside string literals, so `''` doubling is the wrong escaping. A path of `\' OR 1=1 OR journey[1]='` produces:+```+journey[1] = '\'' OR 1=1 OR journey[1]='''+```+The `\'` is an escaped quote, the doubled sequence closes the literal early, and the trailing ` OR 1=1 OR ...` executes as raw SQL. (The wildcard branch runs `patternToRegex` first, which doubles backslashes, so it is not affected.) Separately, a non-string JSON value (e.g. `{"0": 5}`) reaching `path.includes("*")` throws a 500 (trivial DoS).++- **Impact:** SQL injection into the journeys HAVING clause → arbitrary SQL / cross-tenant data access on any public or keyed site.+- **Remediation:** Validate `stepFilters` with a Zod schema (numeric keys → string values, with length caps) before use, and replace the hand-rolled `.replace(...)` with `SqlString.escape(path)` (and on the regex string), matching `getFunnel`/`getFilterStatement` conventions.++---++### H-3. GSC OAuth callback: IDOR + missing CSRF nonce+**CWE-639 (Broken Access Control)** — *Merged from findings #1 and #4 (plaintext-token aspect tracked in M-3 / L-3).*++- **File:** `server/src/api/gsc/callback.ts:36-112`+- **Initiator:** `server/src/api/gsc/connect.ts:20,34` (checks access, sets `state = numericSiteId.toString()`)++`gscCallback` reads the target `siteId` from the OAuth `state` query parameter (`const siteId = Number(state)`, line 36). The **only** authorization check is that a valid session exists (lines 42-45). It never calls `getUserHasAccessToSite` / `getUserHasAdminAccessToSite` for that `siteId` before writing to `gscConnections` (insert/update at lines 88-112). The `state` is an unsigned plaintext `siteId` with no nonce/HMAC, so the callback's trust in it is misplaced.++Any authenticated user can complete the OAuth flow with **their own** Google account and pass `state=<victim siteId>`, causing the server to write the attacker's Google access/refresh tokens into the victim site's `gscConnections` row (overwriting an existing connection via the UPDATE branch).++- **Impact:** (a) Connection hijack / data injection — the victim's dashboard surfaces the attacker's Search Console data; (b) DoS — clobber a legitimate connection; (c) OAuth CSRF / connection-fixation due to the unbound `state`.+- **Remediation:** After resolving `siteId` from `state`, call `getUserHasAdminAccessToSite(req, siteId)` and reject with 403 if false. Replace the plaintext `state` with a signed/HMAC'd or server-stored single-use nonce encoding both `siteId` and the initiating `userId`, verified on callback.++---++### H-4. Permissive CORS: all origins reflected with credentials enabled+**CWE-942** — *Merged from findings #5 and #11.*++- **File:** `server/src/index.ts:211-218`++`@fastify/cors` is registered with an origin callback that unconditionally calls `callback(null, true)` for every origin, combined with `credentials: true` and all state-changing methods. This reflects the requesting Origin into `Access-Control-Allow-Origin` while setting `Access-Control-Allow-Credentials: true`, on every authenticated route (sites, organizations, stripe, admin) — not just the intentionally-public tracking endpoints.++```js+server.register(cors, {+ origin: (_origin, callback) => { callback(null, true); },+ methods: ["GET", "POST", "PUT", "DELETE", "OPTIONS", "PATCH"],+ credentials: true,+});+```++- **Impact:** Any malicious site a logged-in user visits can issue credentialed cross-origin requests and **read the JSON responses** (Allow-Origin reflects their origin), exfiltrating analytics/org data. This compounds with H-5 below for state-changing CSRF.+- **Remediation:** Do not reflect arbitrary origins when `credentials:true`. Maintain an allow-list (production dashboard URL + localhost in dev) and only echo the Origin if listed. For genuinely public ingestion endpoints (`/api/track`, `/api/identify`, `/session-replay/record`, embed-stats, tracking-config, feature-flags/evaluate) keep `credentials:false` with scoped/wildcard CORS.++---++### H-5. Session cookies `SameSite=None` with no CSRF token on custom routes+**CWE-352 (CSRF)** — *finding #6.*++- **File:** `server/src/lib/auth.ts:253-259`++better-auth's `defaultCookieAttributes.sameSite` is `'none'` in production, so the session cookie is attached to all cross-site requests. Combined with H-4 (open CORS) and the absence of any CSRF-token middleware on the custom Fastify route groups, every authenticated state-changing endpoint is reachable cross-site with the victim's cookie. `trustedOrigins` is hardcoded to `['http://localhost:3002']` and excludes the production `BASE_URL`, so better-auth's own origin protection is misconfigured for production.++- **Impact:** CSRF against all authenticated custom routes (delete sites, change subscription, add/remove org members) without per-request token validation.+- **Remediation:** Set `SameSite=Lax` (or `Strict`) unless cross-site embedding genuinely requires `None`; if `None` is required, add CSRF-token validation or strict Origin/Referer checks on all state-changing custom routes. Populate `trustedOrigins` from the production `BASE_URL`.++> **Note:** H-4 and H-5 are mutually reinforcing and should be fixed together. Neither alone fully closes the cross-site attack surface.++---++### H-6. AppSumo webhook has no signature/secret verification+**CWE-345** — *finding #7.*++- **File:** `server/src/api/as/webhook.ts:52-101`; route `server/src/index.ts:423`++`POST /api/as/webhook` is registered with no auth middleware. The handler only runs `validateWebhookPayload` (presence of `license_key`/`event` + event allow-list). There is **no HMAC/shared-secret verification** against the request body, unlike the Stripe webhook (`constructEvent` with `STRIPE_WEBHOOK_SECRET`). The handler then writes/updates `appsumo.licenses`, including activating/upgrading licenses and transferring an `organization_id` onto an attacker-chosen `license_key`. The upgrade/downgrade handlers fall back to "any license with an organization" when `prev_license_key` is unknown.++- **Impact:** Any unauthenticated party reaching the endpoint can forge purchase/activate/upgrade/downgrade/deactivate events — creating an "active" license bound to a real organization (subscription-tier escalation) or deactivating a victim's license.+- **Remediation:** Verify an AppSumo-provided signature/HMAC over the raw request body using a shared secret (mirroring the Stripe pattern) before processing; reject unsigned/invalid requests with 401.++---++### H-7. Session replay recordings served under the public-site access gate+**CWE-200 (Sensitive Data Exposure)** — *finding #23.*++- **File:** `server/src/index.ts:328-329`++`GET /sites/:siteId/session-replay/list` and `GET /sites/:siteId/session-replay/:sessionId` are registered with the `publicSite` preHandler. `allowPublicSiteAccess` (`server/src/lib/auth-middleware.ts:153-165`) grants access whenever `site.public` is true (or a matching `x-private-key`/API key is supplied) **without any authenticated session**. `getSessionReplayEvents` then returns the full reconstructed rrweb event stream — the complete recorded DOM of a visitor's session, routinely containing page content, URLs, and non-masked form input.++- **Impact:** Any time a site is flagged `public` (intending to share an aggregate dashboard), or for anyone holding a private-link key, all stored session replays become listable and fully downloadable/replayable unauthenticated. An attacker can enumerate sessions and harvest captured PII / page content.+- **Remediation:** Gate these endpoints behind authenticated site access (`authSite` / `requireSiteAccess`) even when the aggregate dashboard is public — matching the existing `authSite` gate already on the DELETE replay route (`index.ts:330`). Replays are categorically more sensitive than aggregate metrics.++---++## MEDIUM Severity++### M-1. Org admin can create/assign the higher `owner` role, bypassing role hierarchy+**CWE-269 (Privilege Escalation)** — *Merged from findings #9 and #10.*++- **Files:** `server/src/api/user/createUserInOrganization.ts:50-112`; `server/src/api/user/addUserToOrganization.ts:34-86`+- **Routes:** registered with `authOnly` (`index.ts:364`)++Both `POST /organizations/:organizationId/users` and `POST /organizations/:organizationId/members` self-authorize on `role === 'admin' OR 'owner'`, then accept an arbitrary `role` of `admin|member|owner` and insert directly into the `member` table. The better-auth org plugin treats `owner` as the highest role and would not let an admin mint an owner — but these custom routes sidestep that hierarchy. A mere admin can create a new credentialed `owner` account (password hashed via `ctx.password.hash`) or promote a confederate to `owner`.++- **Impact:** An org admin self-escalates to owner-equivalent control (remove other owners/admins, delete/transfer the org, billing).+- **Remediation:** Disallow assigning a role higher than the caller's own. An `admin` caller may create/assign only `member`/`admin`, never `owner`. Restrict `owner` creation to existing owners (or system admins). Centralize the check in both endpoints.++---++### M-2. Insecure default secrets in `.env.example` and compose defaults+**CWE-1188** — *finding #12.*++- **File:** `.env.example:7,22,26` (and `docker-compose*.yml` fallbacks)++Ships `BETTER_AUTH_SECRET=insecure-secret`, `CLICKHOUSE_PASSWORD=frog`, `POSTGRES_PASSWORD=frog`. Compose files bake in `${POSTGRES_PASSWORD:-frog}`, `${CLICKHOUSE_PASSWORD:-frog}`, `${REDIS_PASSWORD:-changeme}`. An operator who copies the example or relies on compose defaults runs with a publicly known auth-signing secret (forgeable session tokens → account takeover) and guessable DB credentials. (`setup.sh` does generate a random secret, but the example/compose paths remain a foot-gun.)++- **Remediation:** Do not ship a usable default for `BETTER_AUTH_SECRET` — leave it empty and fail startup if unset. Remove weak password defaults from compose fallbacks; require operators to set them.++---++### M-3. Cloud docker-compose publishes datastores to all interfaces with weak default credentials+**CWE-1188** — *finding #25.*++- **File:** `docker-compose.cloud.yml:49-81`++The cloud variant maps Postgres `5432`, ClickHouse `8123/9000`, and Redis `6379` to the host **without** binding to `127.0.0.1` (the non-cloud compose binds backend/client to localhost). Credentials default to `frog`/`changeme`; a comment on the Redis port states it is "Exposed to internet for remote connections", and ClickHouse `network.xml` binds `listen_host 0.0.0.0`.++- **Impact:** On any host where these ports are not firewalled, the entire analytics dataset and app metadata (users, orgs, billing) are directly reachable; with default creds an attacker gets full read/write without touching the application.+- **Remediation:** Bind datastore host port mappings to `127.0.0.1` (or remove host port mappings and rely on the internal docker network). Require strong generated passwords; do not expose Redis to the internet.++---++### M-4. GSC OAuth tokens stored in plaintext+**CWE-312** — *Merged from findings #13 and #19 (the CSRF/IDOR aspects of these reports are covered by H-3).*++- **File:** `server/src/api/gsc/callback.ts:88-112`++The callback writes Google `access_token` and `refresh_token` directly into the `gscConnections` Postgres table with no encryption at rest. Refresh tokens are long-lived credentials. On token-exchange failure the full Google error body is logged (line 71), which can include grant details.++- **Impact:** A Postgres compromise (or any SQL-read access — see M-3's exposed Postgres port) yields usable long-lived Google refresh tokens for every connected site.+- **Remediation:** Encrypt OAuth tokens at rest (application-level envelope encryption with a KMS-managed key, or pgcrypto). Avoid logging raw token-exchange error bodies.++---++## LOW Severity++### L-1. API key accepted via query string+**CWE-598** — *finding #15.*++- **File:** `server/src/lib/auth-utils.ts:242-243,316-317`++`checkApiKey`/`getUserIdFromRequest` accept the key from `?api_key=` as an alternative to the `Authorization: Bearer` header. Query-string secrets are captured in access logs, reverse-proxy logs (Caddy), browser history, and the Referer header. The same scoped key grants read and admin actions.++- **Remediation:** Accept keys only via the `Authorization` header. If query keys must remain for compatibility, document as testing-only and strip them from all logging/Referer.++---++### L-2. Private-link key has 48-bit entropy and is compared non-constant-time+**CWE-208** — *finding #16.*++- **File:** `server/src/lib/auth-utils.ts:352`; generation at `server/src/api/sites/updateSitePrivateLinkConfig.ts:36`++The `x-private-key` header is compared with `===` (not constant-time), and the key is generated as `crypto.randomBytes(6).toString('hex')` — only 48 bits / 12 hex chars. The low entropy is the material weakness; the non-constant-time compare is a minor timing oracle. This key guards unauthenticated read access to a non-public site's full analytics.++- **Remediation:** Generate with substantially more entropy (e.g. `crypto.randomBytes(24)`) and compare with `crypto.timingSafeEqual` on equal-length buffers.++---++### L-3. Sensitive data logged: full user record on signup, AppSumo payload/license keys+**CWE-532** — *Merged from findings #17, #18, and #20.*++- **Files:** `server/src/lib/auth.ts:264` (`console.log(u)` on every signup — dumps email/name/PII, bypassing the structured logger); `server/src/api/as/webhook.ts:66,84,108-161` (logs full payload + `license_key`/`prev_license_key` at info level). Fastify logger is configured at `debug` level (`index.ts:178-182`).++`license_key` is the bearer secret granting subscription tier (and the webhook is forgeable per H-6), so logging it persists replayable secrets. The user object exposes PII.++- **Remediation:** Remove `console.log(u)` (or log only a non-PII id at debug). Do not log full webhook payloads; redact/hash `license_key`/`prev_license_key`. Set production log level to info/warn.++---++### L-4. No security response headers (HSTS/CSP/X-Frame-Options/etc.)+**CWE-693** — *finding #21.*++- **File:** `server/src/index.ts:211-225`; `Caddyfile`++No `@fastify/helmet` (or Caddy `header` directives) is registered. No HSTS, CSP, X-Content-Type-Options, X-Frame-Options/frame-ancestors, or Referrer-Policy. Given the dashboard renders session-replay content and a public iframe widget, the absence of CSP/frame controls reduces defense-in-depth against XSS and clickjacking.++- **Remediation:** Register `@fastify/helmet` (or add Caddy `header` directives) for HSTS, a restrictive CSP, `nosniff`, frame-ancestors, and Referrer-Policy. Scope frame-ancestors for the intended embed widget.++---++### L-5. `@fastify/rate-limit` declared but never registered+**CWE-770** — *finding #22.*++- **File:** `package.json:39` (dependency); `server/src/index.ts:268,290,306,327,442-443` (unprotected routes)++`@fastify/rate-limit` is a declared dependency but `fastify.register(rateLimit, ...)` is never called. The only throttling is inside better-auth's apiKey plugin (API-key requests only). High-volume public endpoints — `POST /api/track`, `/api/identify`, `/session-replay/record/:siteId`, `/site/:siteId/feature-flags/evaluate` — and the better-auth login routes have no app-level rate limit, each accepting up to the 10MB global body limit. Self-host login is captcha-free.++- **Impact:** Unauthenticated event-flooding / ClickHouse data pollution, feature-flag-evaluation abuse (each triggers geo lookup + Postgres query), and credential-stuffing/brute-force against login with no throttle.+- **Remediation:** Register `@fastify/rate-limit` globally and/or apply per-route `config.rateLimit` to public ingestion and auth endpoints, keyed by IP. Tighten limits in cloud vs self-host.++---++## INFO (Latent — not currently exploitable)++### I-1. Session/geo data interpolated into innerHTML without HTML-escaping in globe tooltips+**CWE-79 (XSS)** — *finding #24.*++- **File:** `client/src/app/[site]/globe/utils/timelineTooltipBuilder.ts:63-108`; sinks at `useOpenLayersTimelineLayer.ts:280,417`, `useOpenLayersCoordinatesLayer.ts:199-210`++`buildTooltipHTML()` interpolates `session.city`, `country`, `browser`, `operating_system`, `device_type`, `referrerText`, `pageviews/events`, and `session_id` into an HTML string with **no escaping**, assigned via `innerHTML`. All values originate from data ingested through the public `/api/track` endpoint.++- **Status:** **Not provably exploitable today** — the directly attacker-controlled free-form fields are normalized upstream before reaching the sink: `referrer` is reduced to a hostname (`extractDomain`) or a fixed channel enum, `user_id` is rendered through `renderToStaticMarkup` (escaped), and city/browser/os/device come from MaxMind geo and UA-parser enum tables. The risk is that escaping is entirely absent, so safety depends on every current and future source field staying server-normalized — fragile against a new field, an import mapper, or a UA-parser edge case.+- **Remediation:** HTML-escape every interpolated value (reuse the `escapeHTML` helper already implemented in `client/src/app/widget/[siteId]/route.ts:267`, or DOMPurify), and prefer building DOM nodes with `textContent` over string concatenation.++---++## Appendix: Deduplication Map++| Report ID(s) | Consolidated as |+|--------------|-----------------|+| #2, #8 | H-1 (metric SQL injection) |+| #3, #14 | H-2 (journeys SQL injection) |+| #1, #4 | H-3 (GSC callback IDOR/CSRF) |+| #5, #11 | H-4 (permissive CORS) |+| #6 | H-5 (SameSite=None CSRF) |+| #7 | H-6 (AppSumo webhook) |+| #23 | H-7 (public session replays) |+| #9, #10 | M-1 (owner-role escalation) |+| #12 | M-2 (default secrets) |+| #25 | M-3 (exposed datastores) |+| #13, #19 | M-4 (plaintext GSC tokens) |+| #15 | L-1 (API key in query) |+| #16 | L-2 (private-link key entropy) |+| #17, #18, #20 | L-3 (sensitive logging) |+| #21 | L-4 (missing security headers) |+| #22 | L-5 (no rate limiting) |+| #24 | I-1 (globe tooltip XSS, latent) |\ No newline at end of filediff --git a/client/messages/cs.json b/client/messages/cs.jsonindex 53279ca14..196abbda5 100644--- a/client/messages/cs.json+++ b/client/messages/cs.json@@ -74,6 +74,7 @@"AA5h7P": "Výkon","LlNItA": "","XHLYdJ": "Cíle",+ "qj71j1": "","9d/EEp": "Produktová analytika","5yUsUe": "Přehrávání","puhcpn": "Trychtýře",@@ -107,7 +108,6 @@"Z5dM3R": "Odkazující stránka","Qp1beM": "Cesta","9a9+ww": "",- "qj71j1": "","y1Z3or": "Jazyk","TE4fIS": "","lnaWo/": "Region",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Metrika Web Vitals pro měření výkonu webu.","O8lxEI": "Podívej se na dokumentaci Web Vitals od Google pro prahové hodnoty.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Přehrávání relací je vypnuto","Tb0/AL": "Přehrávání relací zvětší analytický skript","CRjhOx": "8× více",diff --git a/client/messages/de.json b/client/messages/de.jsonindex c60b38da5..59be82840 100644--- a/client/messages/de.json+++ b/client/messages/de.json@@ -74,6 +74,7 @@"AA5h7P": "Leistung","LlNItA": "","XHLYdJ": "Ziele",+ "qj71j1": "Abfrage","9d/EEp": "Produktanalyse","5yUsUe": "Wiedergabe","puhcpn": "Trichter",@@ -107,7 +108,6 @@"Z5dM3R": "Referrer","Qp1beM": "Pfad","9a9+ww": "Titel",- "qj71j1": "Abfrage","y1Z3or": "Sprache","TE4fIS": "Stadt","lnaWo/": "Region",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Web Vitals-Metrik zur Messung der Website-Leistung.","O8lxEI": "Prüfen Sie die Web Vitals-Dokumentation von Google für Schwellenwerte.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Session Replay ist deaktiviert","Tb0/AL": "Session Replay macht das Analytics-Skript","CRjhOx": "8x größer",diff --git a/client/messages/en.json b/client/messages/en.jsonindex 7783f8e76..aa36f3e8b 100644--- a/client/messages/en.json+++ b/client/messages/en.json@@ -74,6 +74,7 @@"AA5h7P": "Performance","LlNItA": "Bots","XHLYdJ": "Goals",+ "qj71j1": "Query","9d/EEp": "Product Analytics","5yUsUe": "Replay","puhcpn": "Funnels",@@ -107,7 +108,6 @@"Z5dM3R": "Referrer","Qp1beM": "Path","9a9+ww": "Title",- "qj71j1": "Query","y1Z3or": "Language","TE4fIS": "City","lnaWo/": "Region",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Web Vitals metric for measuring website performance.","O8lxEI": "Check Google's Web Vitals documentation for thresholds.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "What do you want to query?",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Session Replay is Disabled","Tb0/AL": "Session replay will make the analytics script","CRjhOx": "8x larger",diff --git a/client/messages/es.json b/client/messages/es.jsonindex 19285c379..f5db901af 100644--- a/client/messages/es.json+++ b/client/messages/es.json@@ -74,6 +74,7 @@"AA5h7P": "Rendimiento","LlNItA": "","XHLYdJ": "Objetivos",+ "qj71j1": "Consulta","9d/EEp": "Analíticas de Producto","5yUsUe": "Repetición","puhcpn": "Embudos",@@ -107,7 +108,6 @@"Z5dM3R": "Referidor","Qp1beM": "Ruta","9a9+ww": "Título",- "qj71j1": "Consulta","y1Z3or": "Idioma","TE4fIS": "Ciudad","lnaWo/": "Región",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Métrica de Web Vitals para medir el rendimiento del sitio web.","O8lxEI": "Consulta la documentación de Web Vitals de Google para los umbrales.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "La repetición de sesión está desactivada","Tb0/AL": "La repetición de sesión hará que el script de analíticas sea","CRjhOx": "8x más grande",diff --git a/client/messages/fr.json b/client/messages/fr.jsonindex cd6cce129..33d494eea 100644--- a/client/messages/fr.json+++ b/client/messages/fr.json@@ -74,6 +74,7 @@"AA5h7P": "Performance","LlNItA": "","XHLYdJ": "Objectifs",+ "qj71j1": "Requête","9d/EEp": "Analyse produit","5yUsUe": "Replay","puhcpn": "Entonnoirs",@@ -107,7 +108,6 @@"Z5dM3R": "Référent","Qp1beM": "Chemin","9a9+ww": "Titre",- "qj71j1": "Requête","y1Z3or": "Langue","TE4fIS": "Ville","lnaWo/": "Région",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Métrique Web Vitals pour mesurer la performance du site web.","O8lxEI": "Consultez la documentation Web Vitals de Google pour les seuils.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Le Session Replay est désactivé","Tb0/AL": "Le Session Replay rendra le script analytique","CRjhOx": "8x plus grand",diff --git a/client/messages/it.json b/client/messages/it.jsonindex 9db38a142..ef41e0567 100644--- a/client/messages/it.json+++ b/client/messages/it.json@@ -74,6 +74,7 @@"AA5h7P": "Prestazioni","LlNItA": "","XHLYdJ": "Obiettivi",+ "qj71j1": "Query","9d/EEp": "Analisi del Prodotto","5yUsUe": "Replay","puhcpn": "Funnel",@@ -107,7 +108,6 @@"Z5dM3R": "Referrer","Qp1beM": "Percorso","9a9+ww": "Titolo",- "qj71j1": "Query","y1Z3or": "Lingua","TE4fIS": "Città","lnaWo/": "Regione",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Metrica Web Vitals per misurare le prestazioni del sito web.","O8lxEI": "Controlla la documentazione Web Vitals di Google per le soglie.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "La riproduzione della sessione è Disabilitata","Tb0/AL": "La riproduzione della sessione renderà lo script di analisi","CRjhOx": "8x più grande",diff --git a/client/messages/ja.json b/client/messages/ja.jsonindex 52152b504..3e2685760 100644--- a/client/messages/ja.json+++ b/client/messages/ja.json@@ -74,6 +74,7 @@"AA5h7P": "パフォーマンス","LlNItA": "","XHLYdJ": "目標",+ "qj71j1": "クエリ","9d/EEp": "プロダクトアナリティクス","5yUsUe": "リプレイ","puhcpn": "ファネル",@@ -107,7 +108,6 @@"Z5dM3R": "リファラー","Qp1beM": "パス","9a9+ww": "タイトル",- "qj71j1": "クエリ","y1Z3or": "言語","TE4fIS": "都市","lnaWo/": "地域",@@ -683,6 +683,15 @@"4Urwt8": "ウェブ バイタル","/Vcvr8": "Webサイトパフォーマンス測定用のWeb Vitals指標。","O8lxEI": "しきい値についてはGoogleのWeb Vitalsドキュメントをご確認ください。",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "セッションリプレイは無効です","Tb0/AL": "セッションリプレイを有効にすると、アナリティクススクリプトが","CRjhOx": "8倍大きく",diff --git a/client/messages/ko.json b/client/messages/ko.jsonindex 60b9e1e59..c08d38698 100644--- a/client/messages/ko.json+++ b/client/messages/ko.json@@ -74,6 +74,7 @@"AA5h7P": "성능","LlNItA": "","XHLYdJ": "목표",+ "qj71j1": "쿼리","9d/EEp": "제품 분석","5yUsUe": "리플레이","puhcpn": "퍼널",@@ -107,7 +108,6 @@"Z5dM3R": "리퍼러","Qp1beM": "경로","9a9+ww": "제목",- "qj71j1": "쿼리","y1Z3or": "언어","TE4fIS": "도시","lnaWo/": "지역",@@ -683,6 +683,15 @@"4Urwt8": "웹 바이탈","/Vcvr8": "웹사이트 성능을 측정하는 Web Vitals 지표.","O8lxEI": "임계값은 Google의 Web Vitals 문서를 확인하세요.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "세션 리플레이가 비활성화됨","Tb0/AL": "세션 리플레이는 분석 스크립트를","CRjhOx": "8배 더 크게",diff --git a/client/messages/pl.json b/client/messages/pl.jsonindex fb408974b..bcd633d93 100644--- a/client/messages/pl.json+++ b/client/messages/pl.json@@ -74,6 +74,7 @@"AA5h7P": "Wydajność","LlNItA": "","XHLYdJ": "Cele",+ "qj71j1": "Zapytanie","9d/EEp": "Analityka produktowa","5yUsUe": "Odtwarzanie","puhcpn": "Lejki",@@ -107,7 +108,6 @@"Z5dM3R": "Źródło odesłania","Qp1beM": "Ścieżka","9a9+ww": "Tytuł",- "qj71j1": "Zapytanie","y1Z3or": "Język","TE4fIS": "Miasto","lnaWo/": "Region",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Metryka Web Vitals do mierzenia wydajności strony.","O8lxEI": "Sprawdź dokumentację Google Web Vitals, aby poznać progi.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Odtwarzanie sesji jest wyłączone","Tb0/AL": "Odtwarzanie sesji spowoduje, że skrypt analityczny będzie","CRjhOx": "8x większy",diff --git a/client/messages/pt.json b/client/messages/pt.jsonindex 08600c4fc..b62bbf82d 100644--- a/client/messages/pt.json+++ b/client/messages/pt.json@@ -74,6 +74,7 @@"AA5h7P": "Desempenho","LlNItA": "","XHLYdJ": "Metas",+ "qj71j1": "Consulta","9d/EEp": "Análise de Produto","5yUsUe": "Replay","puhcpn": "Funis",@@ -107,7 +108,6 @@"Z5dM3R": "Referenciador","Qp1beM": "Caminho","9a9+ww": "Título",- "qj71j1": "Consulta","y1Z3or": "Idioma","TE4fIS": "Cidade","lnaWo/": "Região",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Métrica Web Vitals para medir o desempenho do site.","O8lxEI": "Consulte a documentação de Web Vitals do Google para os limites.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Replay de Sessão Desativado","Tb0/AL": "O replay de sessão tornará o script de análise","CRjhOx": "8x maior",diff --git a/client/messages/uk.json b/client/messages/uk.jsonindex 1b4423c60..b6f24a7e8 100644--- a/client/messages/uk.json+++ b/client/messages/uk.json@@ -74,6 +74,7 @@"AA5h7P": "Продуктивність","LlNItA": "","XHLYdJ": "Цілі",+ "qj71j1": "","9d/EEp": "Продуктова аналітика","5yUsUe": "Відтворення","puhcpn": "Воронки",@@ -107,7 +108,6 @@"Z5dM3R": "Джерело переходу","Qp1beM": "Шлях","9a9+ww": "",- "qj71j1": "","y1Z3or": "Мова","TE4fIS": "","lnaWo/": "Регіон",@@ -683,6 +683,15 @@"4Urwt8": "Web Vital","/Vcvr8": "Метрика Web Vitals для вимірювання продуктивності вебсайту.","O8lxEI": "Перевірте документацію Google Web Vitals для отримання порогових значень.",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "Відтворення сеансів вимкнено","Tb0/AL": "Відтворення сеансів зробить аналітичний скрипт","CRjhOx": "у 8 разів більшим",diff --git a/client/messages/zh.json b/client/messages/zh.jsonindex 136df25a0..b6f3477e6 100644--- a/client/messages/zh.json+++ b/client/messages/zh.json@@ -74,6 +74,7 @@"AA5h7P": "性能","LlNItA": "","XHLYdJ": "目标",+ "qj71j1": "查询","9d/EEp": "产品分析","5yUsUe": "会话回放","puhcpn": "漏斗",@@ -107,7 +108,6 @@"Z5dM3R": "来源 URL","Qp1beM": "路径","9a9+ww": "标题",- "qj71j1": "查询","y1Z3or": "语言","TE4fIS": "城市","lnaWo/": "地区",@@ -683,6 +683,15 @@"4Urwt8": "Web Vitals","/Vcvr8": "用于衡量网站性能的 Web Vitals 指标。","O8lxEI": "检查 Google 的 Web Vitals 文档了解阈值。",+ "k1sXXZ": "Failed to generate query",+ "SyINSq": "Failed to run query",+ "Vn7Wtu": "",+ "Pc+tM3": "Generate",+ "yaMHMB": "Results",+ "u3e151": "{count} rows",+ "0eYAOP": "Not run",+ "uDXLhQ": "No rows returned",+ "ICdFJO": "Run a query","4h8Ert": "会话回放已禁用","Tb0/AL": "会话回放将使分析脚本","CRjhOx": "大 8 倍",diff --git a/client/src/api/analytics/endpoints/customQuery.ts b/client/src/api/analytics/endpoints/customQuery.tsnew file mode 100644index 000000000..8fc0ac7cb--- /dev/null+++ b/client/src/api/analytics/endpoints/customQuery.ts@@ -0,0 +1,48 @@+import { authedFetch } from "../../utils";++export type CustomQueryRow = Record<string, unknown>;++export type RunCustomQueryResponse = {+ data: CustomQueryRow[];+ meta: {+ queryId: string;+ rowCount: number;+ maxExecutionTimeSeconds: number;+ maxRows: number;+ };+};++export type GenerateCustomQueryResponse = {+ query: string;+};++export type CustomQueryGenerationMessage = {+ role: "user" | "assistant";+ content: string;+};++export type GenerateCustomQueryRequest = {+ prompt: string;+ currentSiteId?: number;+ currentQuery?: string;+ history?: CustomQueryGenerationMessage[];+};++export function runCustomQuery(organizationId: string, query: string) {+ return authedFetch<RunCustomQueryResponse>(`/organizations/${organizationId}/analytics/query`, undefined, {+ method: "POST",+ data: { query },+ });+}++export function generateCustomQuery(organizationId: string, data: GenerateCustomQueryRequest, signal?: AbortSignal) {+ return authedFetch<GenerateCustomQueryResponse>(+ `/organizations/${organizationId}/analytics/query/generate`,+ undefined,+ {+ method: "POST",+ data,+ signal,+ }+ );+}diff --git a/client/src/api/analytics/endpoints/index.ts b/client/src/api/analytics/endpoints/index.tsindex 61fd9ee1b..bc75e18e6 100644--- a/client/src/api/analytics/endpoints/index.ts+++ b/client/src/api/analytics/endpoints/index.ts@@ -42,6 +42,16 @@ export type {SiteEventCountParams,} from "./events";+// Custom query endpoints+export { generateCustomQuery, runCustomQuery } from "./customQuery";+export type {+ CustomQueryGenerationMessage,+ CustomQueryRow,+ GenerateCustomQueryRequest,+ GenerateCustomQueryResponse,+ RunCustomQueryResponse,+} from "./customQuery";+// Errors endpointsexport { fetchErrorNames, fetchErrorEvents, fetchErrorBucketed } from "./errors";export type {diff --git a/client/src/api/analytics/hooks/useCustomQuery.ts b/client/src/api/analytics/hooks/useCustomQuery.tsnew file mode 100644index 000000000..51de5c0f0--- /dev/null+++ b/client/src/api/analytics/hooks/useCustomQuery.ts@@ -0,0 +1,29 @@+import { useMutation } from "@tanstack/react-query";+import { CustomQueryGenerationMessage, generateCustomQuery, runCustomQuery } from "../endpoints/customQuery";++export function useRunCustomQuery() {+ return useMutation({+ mutationFn: ({ organizationId, query }: { organizationId: string; query: string }) =>+ runCustomQuery(organizationId, query),+ });+}++export function useGenerateCustomQuery() {+ return useMutation({+ mutationFn: ({+ organizationId,+ prompt,+ currentSiteId,+ currentQuery,+ history,+ signal,+ }: {+ organizationId: string;+ prompt: string;+ currentSiteId?: number;+ currentQuery?: string;+ history?: CustomQueryGenerationMessage[];+ signal?: AbortSignal;+ }) => generateCustomQuery(organizationId, { prompt, currentSiteId, currentQuery, history }, signal),+ });+}diff --git a/client/src/app/[site]/[privateKey]/query/page.tsx b/client/src/app/[site]/[privateKey]/query/page.tsxnew file mode 100644index 000000000..f5dcbb998--- /dev/null+++ b/client/src/app/[site]/[privateKey]/query/page.tsx@@ -0,0 +1,2 @@+// Re-export the query page for private link routes+export { default } from "../../query/page";diff --git a/client/src/app/[site]/components/Sidebar/Sidebar.tsx b/client/src/app/[site]/components/Sidebar/Sidebar.tsxindex 5c7dd48a8..218ca9e5f 100644--- a/client/src/app/[site]/components/Sidebar/Sidebar.tsx+++ b/client/src/app/[site]/components/Sidebar/Sidebar.tsx@@ -4,6 +4,7 @@ import {Bot,ChartColumnDecreasing,Code,+ Database,File,Flag,FlaskConical,@@ -124,6 +125,12 @@ function SidebarContent() {icon={<Code className="w-4 h-4" />}/></div>+ <SidebarComponents.Item+ label={t("Query")}+ active={isActiveTab("query")}+ href={getTabPath("query")}+ icon={<Database className="w-4 h-4" />}+ /><SidebarComponents.SectionHeader>{t("Product Analytics")}</SidebarComponents.SectionHeader><div className="hidden md:block">{!isMobileSite && !subscription?.planName?.startsWith("appsumo") && !isSubscriptionLoading && (diff --git a/client/src/app/[site]/journeys/page.tsx b/client/src/app/[site]/journeys/page.tsxindex ac2e91d1c..78215d55f 100644--- a/client/src/app/[site]/journeys/page.tsx+++ b/client/src/app/[site]/journeys/page.tsx@@ -54,9 +54,11 @@ export default function JourneysPage() {<DisabledOverlay message="User Journeys" featurePath="journeys"><div className="container mx-auto p-2 md:p-4"><SubHeader availableFilters={JOURNEY_PAGE_FILTERS} />- <div className="flex items-center gap-6 my-2">+ <div className="flex items-center gap-6 my-1"><div className="flex items-center gap-3 w-[180px]">- <span className="text-sm text-neutral-600 dark:text-neutral-300 whitespace-nowrap">{t("{steps} steps", { steps: String(steps) })}</span>+ <span className="text-sm text-neutral-600 dark:text-neutral-300 whitespace-nowrap">+ {t("{steps} steps", { steps: String(steps) })}+ </span><Slidervalue={[steps]}onValueChange={([value]) => setSteps(value)}@@ -91,17 +93,19 @@ export default function JourneysPage() {</div></div>)}- <div className="flex flex-col gap-1 mb-4 mt-4">+ <div className="flex flex-col gap-1 my-2"><div className="flex gap-1">{Array.from({ length: steps }, (_, i) => (<divkey={i}- className="flex-1 h-10 bg-neutral-50 dark:bg-neutral-850 flex items-center text-neutral-900 dark:text-white text-sm relative px-3"+ className="flex-1 h-8 bg-neutral-50 dark:bg-neutral-850 flex items-center text-neutral-900 dark:text-white text-sm relative px-2"style={{clipPath: "polygon(0 0, 10px 50%, 0 100%, calc(100% - 10px) 100%, 100% 50%, calc(100% - 10px) 0)",}}>- <span className="ml-2 whitespace-nowrap text-neutral-700 dark:text-neutral-200">{t("Step {number}", { number: String(i + 1) })}</span>+ <span className="ml-2 whitespace-nowrap text-neutral-700 dark:text-neutral-200">+ {t("Step {number}", { number: String(i + 1) })}+ </span><InputWithSuggestionssuggestions={pathSuggestions}placeholder={t("Path filter")}@@ -115,7 +119,7 @@ export default function JourneysPage() {}setStepFilters(newFilters);}}- className="h-7 bg-white dark:bg-neutral-900 border-neutral-100 dark:border-neutral-800 text-neutral-900 dark:text-white placeholder:text-neutral-400 dark:placeholder:text-neutral-500 ml-3 mr-3"+ className="h-6 bg-white dark:bg-neutral-900 border-neutral-100 dark:border-neutral-800 text-neutral-900 dark:text-white placeholder:text-neutral-400 dark:placeholder:text-neutral-500 ml-2 mr-2"/></div>))}diff --git a/client/src/app/[site]/layout.tsx b/client/src/app/[site]/layout.tsxindex 4bc2b2254..8aa35c970 100644--- a/client/src/app/[site]/layout.tsx+++ b/client/src/app/[site]/layout.tsx@@ -70,7 +70,8 @@ export default function SiteLayout({ children }: { children: React.ReactNode })!pathname.includes("/realtime") &&!pathname.includes("/replay") &&!pathname.includes("/globe") &&- !pathname.includes("/api-playground") && <Footer disabled={embed} />}+ !pathname.includes("/api-playground") &&+ !pathname.includes("/query") && <Footer disabled={embed} />}</div></div></div>diff --git a/client/src/app/[site]/query/components/QueryEditor.tsx b/client/src/app/[site]/query/components/QueryEditor.tsxnew file mode 100644index 000000000..de779a1b4--- /dev/null+++ b/client/src/app/[site]/query/components/QueryEditor.tsx@@ -0,0 +1,146 @@+"use client";++import { AlignLeft, Loader2, Play } from "lucide-react";+import { useExtracted } from "next-intl";+import { useTheme } from "next-themes";+import { useEffect, useRef, useState } from "react";+import { Light as SyntaxHighlighter } from "react-syntax-highlighter";+import sql from "react-syntax-highlighter/dist/esm/languages/hljs/sql";+import { vs, vs2015 } from "react-syntax-highlighter/dist/esm/styles/hljs";+import { Button } from "../../../../components/ui/button";+import { cn } from "../../../../lib/utils";++SyntaxHighlighter.registerLanguage("sql", sql);++type QueryEditorProps = {+ value: string;+ disabled: boolean;+ isRunning: boolean;+ onChange: (value: string) => void;+ onFormat: () => void;+ onRun: () => void;+};++export function QueryEditor({ value, disabled, isRunning, onChange, onFormat, onRun }: QueryEditorProps) {+ const t = useExtracted();+ const { resolvedTheme } = useTheme();+ const [isDark, setIsDark] = useState(false);+ const highlightRef = useRef<HTMLDivElement>(null);+ const lineNumberRef = useRef<HTMLDivElement>(null);+ const lineCount = Math.max(1, value.split("\n").length);++ useEffect(() => {+ setIsDark(resolvedTheme === "dark" || document.documentElement.classList.contains("dark"));+ }, [resolvedTheme]);++ const handleScroll = (event: React.UIEvent<HTMLTextAreaElement>) => {+ const { scrollLeft, scrollTop } = event.currentTarget;+ if (highlightRef.current) {+ highlightRef.current.style.transform = `translate(${-scrollLeft}px, ${-scrollTop}px)`;+ }+ if (lineNumberRef.current) {+ lineNumberRef.current.style.transform = `translateY(${-scrollTop}px)`;+ }+ };++ return (+ <div className="flex min-h-[280px] flex-col overflow-hidden rounded-lg border border-neutral-150 bg-white shadow-sm dark:border-neutral-850 dark:bg-neutral-900">+ <div className="flex h-10 items-center justify-between border-b border-neutral-150 bg-neutral-50 px-3 dark:border-neutral-800 dark:bg-neutral-950">+ <div className="flex items-center gap-2">+ <div className="h-2 w-2 rounded-full bg-emerald-500" />+ <div className="text-sm font-medium text-neutral-900 dark:text-neutral-100">{t("Query")}</div>+ <div className="rounded border border-neutral-200 bg-white px-1.5 py-0.5 text-[11px] font-medium text-neutral-500 dark:border-neutral-800 dark:bg-neutral-900 dark:text-neutral-400">+ SQL+ </div>+ </div>+ <div className="flex items-center gap-1.5">+ <Button+ type="button"+ size="smIcon"+ variant="ghost"+ onClick={onFormat}+ disabled={disabled || !value.trim()}+ title="Format query"+ aria-label="Format query"+ >+ <AlignLeft className="h-4 w-4" />+ </Button>+ <Button size="sm" onClick={onRun} disabled={disabled || !value.trim()}>+ {isRunning ? <Loader2 className="h-4 w-4 animate-spin" /> : <Play className="h-4 w-4" />}+ {t("Run")}+ </Button>+ </div>+ </div>++ <div className="grid min-h-0 flex-1 grid-cols-[42px_minmax(0,1fr)] bg-[#fbfcfd] dark:bg-[#090d16]">+ <div className="relative overflow-hidden border-r border-neutral-150 bg-neutral-50 dark:border-neutral-800 dark:bg-neutral-950">+ <div+ ref={lineNumberRef}+ className="select-none px-2 py-2.5 text-right font-mono text-[11px] leading-[18px] text-neutral-400 dark:text-neutral-600"+ aria-hidden="true"+ >+ {Array.from({ length: lineCount }, (_, index) => (+ <div key={index} className="h-[18px]">+ {index + 1}+ </div>+ ))}+ </div>+ </div>+ <div className="relative min-h-[240px] overflow-hidden">+ <div className="pointer-events-none absolute inset-0 overflow-hidden">+ <div+ ref={highlightRef}+ className="min-w-full px-3 py-2.5 font-mono text-[12px] leading-[18px]"+ style={{ width: "max-content" }}+ >+ <SyntaxHighlighter+ key={isDark ? "sql-dark" : "sql-light"}+ language="sql"+ style={isDark ? vs2015 : vs}+ customStyle={{+ margin: 0,+ padding: 0,+ background: "transparent",+ color: isDark ? "#dcdcdc" : "#111827",+ fontSize: "12px",+ lineHeight: "18px",+ overflow: "visible",+ whiteSpace: "pre",+ }}+ codeTagProps={{+ style: {+ fontFamily: "inherit",+ fontSize: "12px",+ lineHeight: "18px",+ whiteSpace: "pre",+ color: isDark ? "#dcdcdc" : "#111827",+ },+ }}+ >+ {value || " "}+ </SyntaxHighlighter>+ </div>+ </div>+ <textarea+ value={value}+ onChange={event => onChange(event.target.value)}+ onScroll={handleScroll}+ disabled={disabled}+ spellCheck={false}+ wrap="off"+ className={cn(+ "absolute inset-0 min-h-[240px] resize-none overflow-auto border-0 bg-transparent px-3 py-2.5 font-mono text-[12px] leading-[18px] text-transparent outline-none caret-neutral-900",+ "selection:bg-blue-500/20 placeholder:text-neutral-400 focus:ring-0 disabled:cursor-not-allowed disabled:opacity-60",+ "dark:caret-neutral-100 dark:selection:bg-blue-400/25 dark:placeholder:text-neutral-600"+ )}+ />+ </div>+ </div>++ <div className="flex h-7 items-center justify-between border-t border-neutral-150 bg-neutral-50 px-3 text-[11px] text-neutral-500 dark:border-neutral-800 dark:bg-neutral-950 dark:text-neutral-500">+ <span>{lineCount} lines</span>+ <span>{value.length.toLocaleString()} chars</span>+ </div>+ </div>+ );+}diff --git a/client/src/app/[site]/query/components/QueryPromptForm.tsx b/client/src/app/[site]/query/components/QueryPromptForm.tsxnew file mode 100644index 000000000..095508d1b--- /dev/null+++ b/client/src/app/[site]/query/components/QueryPromptForm.tsx@@ -0,0 +1,51 @@+"use client";++import { Loader2, Sparkles, X } from "lucide-react";+import { useExtracted } from "next-intl";+import type { FormEvent } from "react";+import { Button } from "../../../../components/ui/button";+import { Input } from "../../../../components/ui/input";++type QueryPromptFormProps = {+ prompt: string;+ canUseQuery: boolean;+ isBusy: boolean;+ isGenerating: boolean;+ onPromptChange: (prompt: string) => void;+ onGenerate: (event: FormEvent<HTMLFormElement>) => void;+ onCancelGenerate: () => void;+};++export function QueryPromptForm({+ prompt,+ canUseQuery,+ isBusy,+ isGenerating,+ onPromptChange,+ onGenerate,+ onCancelGenerate,+}: QueryPromptFormProps) {+ const t = useExtracted();++ return (+ <form onSubmit={onGenerate} className="flex flex-col gap-2 md:flex-row">+ <Input+ value={prompt}+ onChange={event => onPromptChange(event.target.value)}+ placeholder={t("What do you want to query?")}+ disabled={!canUseQuery || isBusy}+ className="md:flex-1"+ />+ <Button type="submit" disabled={!canUseQuery || !prompt.trim() || isBusy} className="md:w-auto">+ {isGenerating ? <Loader2 className="h-4 w-4 animate-spin" /> : <Sparkles className="h-4 w-4" />}+ {t("Generate")}+ </Button>+ {isGenerating && (+ <Button type="button" variant="outline" onClick={onCancelGenerate} className="md:w-auto">+ <X className="h-4 w-4" />+ {t("Cancel")}+ </Button>+ )}+ </form>+ );+}diff --git a/client/src/app/[site]/query/components/QueryTabs.tsx b/client/src/app/[site]/query/components/QueryTabs.tsxnew file mode 100644index 000000000..5330f5fc1--- /dev/null+++ b/client/src/app/[site]/query/components/QueryTabs.tsx@@ -0,0 +1,72 @@+"use client";++import { AlertCircle, Loader2, Plus, X } from "lucide-react";+import { Button } from "../../../../components/ui/button";+import { cn } from "../../../../lib/utils";+import type { QueryTab } from "../types";++type QueryTabsProps = {+ tabs: QueryTab[];+ activeTabId?: string;+ runningTabIds: Set<string>;+ generatingTabIds: Set<string>;+ onSelectTab: (tabId: string) => void;+ onCloseTab: (tabId: string) => void;+ onAddTab: () => void;+};++export function QueryTabs({+ tabs,+ activeTabId,+ runningTabIds,+ generatingTabIds,+ onSelectTab,+ onCloseTab,+ onAddTab,+}: QueryTabsProps) {+ return (+ <div className="flex items-center gap-1 overflow-x-auto rounded-lg border border-neutral-150 bg-neutral-50 p-1 dark:border-neutral-850 dark:bg-neutral-950">+ {tabs.map((tab, index) => {+ const isActive = tab.id === activeTabId;+ const tabIsBusy = runningTabIds.has(tab.id) || generatingTabIds.has(tab.id);++ return (+ <button+ key={tab.id}+ type="button"+ onClick={() => onSelectTab(tab.id)}+ className={cn(+ "group flex h-8 min-w-[120px] items-center justify-between gap-2 rounded-md border px-2 text-left text-xs transition-colors",+ isActive+ ? "border-neutral-200 bg-white text-neutral-900 shadow-sm dark:border-neutral-750 dark:bg-neutral-900 dark:text-neutral-100"+ : "border-transparent text-neutral-600 hover:bg-white/70 hover:text-neutral-900 dark:text-neutral-400 dark:hover:bg-neutral-900/70 dark:hover:text-neutral-100"+ )}+ >+ <span className="flex min-w-0 items-center gap-1.5">+ {tabIsBusy && <Loader2 className="h-3 w-3 shrink-0 animate-spin text-neutral-400" />}+ {tab.resultError && !tabIsBusy && <AlertCircle className="h-3 w-3 shrink-0 text-red-500" />}+ <span className="truncate">{tab.name || `Query ${index + 1}`}</span>+ </span>+ {tabs.length > 1 && (+ <span+ role="button"+ tabIndex={-1}+ onClick={event => {+ event.stopPropagation();+ onCloseTab(tab.id);+ }}+ className="rounded p-0.5 text-neutral-400 opacity-70 hover:bg-neutral-100 hover:text-neutral-700 group-hover:opacity-100 dark:hover:bg-neutral-800 dark:hover:text-neutral-200"+ aria-label="Close query tab"+ >+ <X className="h-3 w-3" />+ </span>+ )}+ </button>+ );+ })}+ <Button type="button" size="smIcon" variant="ghost" onClick={onAddTab} className="shrink-0" aria-label="New query">+ <Plus className="h-4 w-4" />+ </Button>+ </div>+ );+}diff --git a/client/src/app/[site]/query/components/ResultsPanel.tsx b/client/src/app/[site]/query/components/ResultsPanel.tsxnew file mode 100644index 000000000..c0649b829--- /dev/null+++ b/client/src/app/[site]/query/components/ResultsPanel.tsx@@ -0,0 +1,59 @@+"use client";++import { AlertCircle } from "lucide-react";+import { useExtracted } from "next-intl";+import type { CustomQueryRow } from "../../../../api/analytics/endpoints";+import type { QueryTab, SortState } from "../types";+import { ResultsTable } from "./ResultsTable";++type ResultsPanelProps = {+ activeTab?: QueryTab;+ columns: string[];+ rows: CustomQueryRow[];+ sort: SortState;+ onSortChange: (sort: SortState) => void;+};++export function ResultsPanel({ activeTab, columns, rows, sort, onSortChange }: ResultsPanelProps) {+ const t = useExtracted();++ return (+ <div className="flex min-h-0 flex-1 flex-col overflow-hidden rounded-lg border border-neutral-150 bg-white dark:border-neutral-850 dark:bg-neutral-900">+ <div className="flex h-10 items-center justify-between border-b border-neutral-100 px-3 dark:border-neutral-850">+ <div className="text-sm font-medium text-neutral-900 dark:text-neutral-100">{t("Results")}</div>+ <div className="text-xs text-neutral-500 dark:text-neutral-400">+ {activeTab?.resultError ? (+ <span className="inline-flex items-center gap-1 text-red-600 dark:text-red-400">+ <AlertCircle className="h-3.5 w-3.5" />+ {t("Error")}+ </span>+ ) : activeTab?.hasRun ? (+ t("{count} rows", { count: activeTab.rows.length.toLocaleString() })+ ) : (+ t("Not run")+ )}+ </div>+ </div>++ {activeTab?.resultError ? (+ <div className="flex min-h-0 flex-1 items-start justify-center overflow-auto p-4">+ <div className="w-full rounded-md border border-red-300 bg-red-50 p-4 dark:border-red-800 dark:bg-red-950/40">+ <div className="flex items-center gap-2 text-sm font-medium text-red-700 dark:text-red-300">+ <AlertCircle className="h-4 w-4" />+ {t("Error")}+ </div>+ <pre className="mt-3 max-h-[320px] overflow-auto whitespace-pre-wrap break-words rounded border border-red-200 bg-white/70 p-3 font-mono text-xs leading-5 text-red-800 dark:border-red-900/70 dark:bg-red-950/50 dark:text-red-100">+ {activeTab.resultError}+ </pre>+ </div>+ </div>+ ) : columns.length > 0 ? (+ <ResultsTable columns={columns} rows={rows} sort={sort} onSortChange={onSortChange} />+ ) : (+ <div className="flex min-h-0 flex-1 items-center justify-center text-sm text-neutral-500 dark:text-neutral-400">+ {activeTab?.hasRun ? t("No rows returned") : t("Run a query")}+ </div>+ )}+ </div>+ );+}diff --git a/client/src/app/[site]/query/components/ResultsTable.tsx b/client/src/app/[site]/query/components/ResultsTable.tsxnew file mode 100644index 000000000..5979872a8--- /dev/null+++ b/client/src/app/[site]/query/components/ResultsTable.tsx@@ -0,0 +1,110 @@+"use client";++import { useVirtualizer } from "@tanstack/react-virtual";+import { useRef } from "react";+import type { CustomQueryRow } from "../../../../api/analytics/endpoints";+import { TableSortIndicator } from "../../../../components/ui/table";+import type { SortState } from "../types";+import { formatCellValue, getNextSortState } from "../utils";++type ResultsTableProps = {+ columns: string[];+ rows: CustomQueryRow[];+ sort: SortState;+ onSortChange: (sort: SortState) => void;+};++export function ResultsTable({ columns, rows, sort, onSortChange }: ResultsTableProps) {+ const scrollContainerRef = useRef<HTMLDivElement>(null);+ const headerScrollRef = useRef<HTMLDivElement>(null);+ const rowVirtualizer = useVirtualizer({+ count: rows.length,+ getScrollElement: () => scrollContainerRef.current,+ estimateSize: () => 26,+ overscan: 16,+ });+ const gridTemplateColumns = `repeat(${columns.length}, minmax(160px, 1fr))`;+ const minWidth = `${columns.length * 180}px`;++ const handleBodyScroll = (event: React.UIEvent<HTMLDivElement>) => {+ if (headerScrollRef.current) {+ headerScrollRef.current.scrollLeft = event.currentTarget.scrollLeft;+ }+ };++ return (+ <div className="flex min-h-0 flex-1 flex-col">+ <div+ ref={headerScrollRef}+ className="shrink-0 overflow-hidden border-b border-neutral-100 dark:border-neutral-800"+ >+ <div+ role="row"+ className="grid min-h-7 bg-neutral-50 text-[11px] font-medium text-neutral-500 dark:bg-neutral-850 dark:text-neutral-400"+ style={{ gridTemplateColumns, minWidth }}+ >+ {columns.map(column => {+ const sortDirection = sort?.column === column ? sort.direction : undefined;++ return (+ <div+ key={column}+ role="columnheader"+ aria-sort={sortDirection === "asc" ? "ascending" : sortDirection === "desc" ? "descending" : "none"}+ className="min-w-0 border-r border-neutral-100 last:border-r-0 dark:border-neutral-800"+ >+ <button+ type="button"+ onClick={() => {+ onSortChange(getNextSortState(sort, column));+ scrollContainerRef.current?.scrollTo({ top: 0 });+ }}+ className="flex h-7 w-full items-center justify-between gap-2 px-2 text-left hover:bg-neutral-100 dark:hover:bg-neutral-800"+ >+ <span className="truncate">{column}</span>+ <TableSortIndicator sortDirection={sortDirection} className="shrink-0" />+ </button>+ </div>+ );+ })}+ </div>+ </div>++ <div ref={scrollContainerRef} className="min-h-0 flex-1 overflow-auto" onScroll={handleBodyScroll}>+ <div+ className="relative bg-white dark:bg-neutral-900"+ style={{ height: rowVirtualizer.getTotalSize(), minWidth }}+ >+ {rowVirtualizer.getVirtualItems().map(virtualRow => {+ const row = rows[virtualRow.index];+ if (!row) return null;++ return (+ <div+ key={virtualRow.key}+ role="row"+ className="absolute left-0 grid min-h-[26px] w-full border-b border-neutral-100 bg-white text-[11px] transition-colors hover:bg-neutral-0 dark:border-neutral-800 dark:bg-neutral-900 dark:hover:bg-neutral-800/20"+ style={{+ gridTemplateColumns,+ minWidth,+ transform: `translateY(${virtualRow.start}px)`,+ }}+ >+ {columns.map(column => (+ <div+ key={column}+ role="cell"+ className="min-w-0 truncate whitespace-nowrap border-r border-neutral-100 px-1.5 py-1 font-mono last:border-r-0 dark:border-neutral-800"+ title={formatCellValue(row[column])}+ >+ {formatCellValue(row[column])}+ </div>+ ))}+ </div>+ );+ })}+ </div>+ </div>+ </div>+ );+}diff --git a/client/src/app/[site]/query/page.tsx b/client/src/app/[site]/query/page.tsxnew file mode 100644index 000000000..0a1d1a2eb--- /dev/null+++ b/client/src/app/[site]/query/page.tsx@@ -0,0 +1,222 @@+"use client";++import { useExtracted } from "next-intl";+import { useParams } from "next/navigation";+import type { FormEvent } from "react";+import { useEffect, useMemo, useRef, useState } from "react";+import { useGetSite } from "../../../api/admin/hooks/useSites";+import { useGenerateCustomQuery, useRunCustomQuery } from "../../../api/analytics/hooks/useCustomQuery";+import { useSetPageTitle } from "../../../hooks/useSetPageTitle";+import { QueryEditor } from "./components/QueryEditor";+import { QueryPromptForm } from "./components/QueryPromptForm";+import { QueryTabs } from "./components/QueryTabs";+import { ResultsPanel } from "./components/ResultsPanel";+import type { QueryTab } from "./types";+import { createQueryTab, formatQuery, getColumns, getErrorMessage, isAbortError, sortRows } from "./utils";++export default function QueryPage() {+ useSetPageTitle("Query");+ const t = useExtracted();+ const params = useParams<{ site: string }>();+ const siteId = Number(params.site);+ const { data: siteMetadata, isLoading: isLoadingSite } = useGetSite(siteId);+ const organizationId = siteMetadata?.organizationId;++ const [tabs, setTabs] = useState<QueryTab[]>(() => [createQueryTab(1)]);+ const [activeTabId, setActiveTabId] = useState(() => tabs[0]?.id);+ const [runningTabIds, setRunningTabIds] = useState<Set<string>>(() => new Set());+ const [generatingTabIds, setGeneratingTabIds] = useState<Set<string>>(() => new Set());+ const generateAbortControllersRef = useRef<Map<string, AbortController>>(new Map());++ const runMutation = useRunCustomQuery();+ const generateMutation = useGenerateCustomQuery();+ const activeTab = tabs.find(tab => tab.id === activeTabId) ?? tabs[0];+ const columns = useMemo(() => getColumns(activeTab?.rows ?? []), [activeTab?.rows]);+ const activeSort = activeTab?.sort && columns.includes(activeTab.sort.column) ? activeTab.sort : null;+ const sortedRows = useMemo(() => sortRows(activeTab?.rows ?? [], activeSort), [activeTab?.rows, activeSort]);+ const activeTabIsRunning = activeTab ? runningTabIds.has(activeTab.id) : false;+ const activeTabIsGenerating = activeTab ? generatingTabIds.has(activeTab.id) : false;+ const activeTabIsBusy = activeTabIsRunning || activeTabIsGenerating;++ const updateTab = (tabId: string, updates: Partial<QueryTab>) => {+ setTabs(currentTabs => currentTabs.map(tab => (tab.id === tabId ? { ...tab, ...updates } : tab)));+ };++ const updateActiveTab = (updates: Partial<QueryTab>) => {+ if (!activeTab) return;+ updateTab(activeTab.id, updates);+ };++ const setTabRunning = (tabId: string, isRunning: boolean) => {+ setRunningTabIds(currentIds => {+ const nextIds = new Set(currentIds);+ if (isRunning) {+ nextIds.add(tabId);+ } else {+ nextIds.delete(tabId);+ }+ return nextIds;+ });+ };++ const setTabGenerating = (tabId: string, isGenerating: boolean) => {+ setGeneratingTabIds(currentIds => {+ const nextIds = new Set(currentIds);+ if (isGenerating) {+ nextIds.add(tabId);+ } else {+ nextIds.delete(tabId);+ }+ return nextIds;+ });+ };++ const addTab = () => {+ setTabs(currentTabs => {+ const nextTab = createQueryTab(currentTabs.length + 1);+ setActiveTabId(nextTab.id);+ return [...currentTabs, nextTab];+ });+ };++ const closeTab = (tabId: string) => {+ generateAbortControllersRef.current.get(tabId)?.abort();+ generateAbortControllersRef.current.delete(tabId);+ setTabGenerating(tabId, false);+ setTabRunning(tabId, false);++ setTabs(currentTabs => {+ if (currentTabs.length === 1) return currentTabs;+ const tabIndex = currentTabs.findIndex(tab => tab.id === tabId);+ const nextTabs = currentTabs.filter(tab => tab.id !== tabId);+ if (tabId === activeTabId) {+ setActiveTabId(nextTabs[Math.max(0, tabIndex - 1)]?.id ?? nextTabs[0]?.id);+ }+ return nextTabs;+ });+ };++ useEffect(() => {+ return () => {+ generateAbortControllersRef.current.forEach(controller => controller.abort());+ generateAbortControllersRef.current.clear();+ };+ }, []);++ const abortActiveGeneration = () => {+ if (!activeTab) return;+ generateAbortControllersRef.current.get(activeTab.id)?.abort();+ generateAbortControllersRef.current.delete(activeTab.id);+ setTabGenerating(activeTab.id, false);+ };++ const handleGenerate = async (event: FormEvent<HTMLFormElement>) => {+ event.preventDefault();+ const prompt = activeTab?.prompt.trim();+ if (!organizationId || !activeTab || !prompt) return;++ const tab = activeTab;+ generateAbortControllersRef.current.get(tab.id)?.abort();+ const abortController = new AbortController();+ generateAbortControllersRef.current.set(tab.id, abortController);+ setTabGenerating(tab.id, true);+ updateTab(tab.id, { resultError: null });++ try {+ const result = await generateMutation.mutateAsync({+ organizationId,+ prompt,+ currentSiteId: Number.isFinite(siteId) ? siteId : undefined,+ currentQuery: tab.query,+ history: tab.generationHistory,+ signal: abortController.signal,+ });+ const formattedQuery = formatQuery(result.query);+ const newGenerationMessages: QueryTab["generationHistory"] = [+ { role: "user", content: prompt },+ { role: "assistant", content: formattedQuery },+ ];+ const generationHistory = [...tab.generationHistory, ...newGenerationMessages].slice(-12);++ updateTab(tab.id, {+ query: formattedQuery,+ generationHistory,+ resultError: null,+ });+ } catch (error) {+ if (isAbortError(error)) return;+ updateTab(tab.id, {+ resultError: getErrorMessage(error, t("Failed to generate query")),+ });+ } finally {+ if (generateAbortControllersRef.current.get(tab.id) === abortController) {+ generateAbortControllersRef.current.delete(tab.id);+ }+ setTabGenerating(tab.id, false);+ }+ };++ const handleRun = async () => {+ if (!organizationId || !activeTab?.query.trim()) return;++ const tab = activeTab;+ setTabRunning(tab.id, true);+ updateTab(tab.id, { resultError: null });++ try {+ const result = await runMutation.mutateAsync({ organizationId, query: tab.query });+ updateTab(tab.id, { rows: result.data, hasRun: true, resultError: null });+ } catch (error) {+ updateTab(tab.id, {+ rows: [],+ hasRun: true,+ resultError: getErrorMessage(error, t("Failed to run query")),+ });+ } finally {+ setTabRunning(tab.id, false);+ }+ };++ const canUseQuery = !!organizationId && !isLoadingSite;++ return (+ <div className="p-2 md:p-4 mx-auto max-w-[1400px] h-[calc(100vh-96px)] flex flex-col gap-3">+ <QueryTabs+ tabs={tabs}+ activeTabId={activeTab?.id}+ runningTabIds={runningTabIds}+ generatingTabIds={generatingTabIds}+ onSelectTab={setActiveTabId}+ onCloseTab={closeTab}+ onAddTab={addTab}+ />++ <QueryPromptForm+ prompt={activeTab?.prompt ?? ""}+ canUseQuery={canUseQuery}+ isBusy={activeTabIsBusy}+ isGenerating={activeTabIsGenerating}+ onPromptChange={prompt => updateActiveTab({ prompt })}+ onGenerate={handleGenerate}+ onCancelGenerate={abortActiveGeneration}+ />++ <QueryEditor+ value={activeTab?.query ?? ""}+ disabled={!canUseQuery || activeTabIsBusy}+ isRunning={activeTabIsRunning}+ onChange={query => updateActiveTab({ query })}+ onFormat={() => updateActiveTab({ query: formatQuery(activeTab?.query ?? "") })}+ onRun={handleRun}+ />++ <ResultsPanel+ activeTab={activeTab}+ columns={columns}+ rows={sortedRows}+ sort={activeSort}+ onSortChange={sort => updateActiveTab({ sort })}+ />+ </div>+ );+}diff --git a/client/src/app/[site]/query/types.ts b/client/src/app/[site]/query/types.tsnew file mode 100644index 000000000..4406444a5--- /dev/null+++ b/client/src/app/[site]/query/types.ts@@ -0,0 +1,18 @@+import type { CustomQueryGenerationMessage, CustomQueryRow } from "../../../api/analytics/endpoints";++export type SortState = {+ column: string;+ direction: "asc" | "desc";+} | null;++export type QueryTab = {+ id: string;+ name: string;+ prompt: string;+ query: string;+ generationHistory: CustomQueryGenerationMessage[];+ rows: CustomQueryRow[];+ sort: SortState;+ resultError: string | null;+ hasRun: boolean;+};diff --git a/client/src/app/[site]/query/utils.ts b/client/src/app/[site]/query/utils.tsnew file mode 100644index 000000000..7c0034b57--- /dev/null+++ b/client/src/app/[site]/query/utils.ts@@ -0,0 +1,108 @@+import { format as formatSql } from "sql-formatter";+import type { CustomQueryRow } from "../../../api/analytics/endpoints";+import type { QueryTab, SortState } from "./types";++const DEFAULT_QUERY = "";++export function createQueryTab(index: number): QueryTab {+ return {+ id: `${Date.now()}-${index}`,+ name: `Query ${index}`,+ prompt: "",+ query: DEFAULT_QUERY,+ generationHistory: [],+ rows: [],+ sort: null,+ resultError: null,+ hasRun: false,+ };+}++export function formatCellValue(value: unknown) {+ if (value === null || value === undefined) return "";+ if (typeof value === "boolean") return value ? "true" : "false";+ if (typeof value === "object") return JSON.stringify(value);+ return String(value);+}++export function getColumns(rows: CustomQueryRow[]) {+ const columns = new Set<string>();+ for (const row of rows) {+ for (const key of Object.keys(row)) {+ columns.add(key);+ }+ }+ return Array.from(columns);+}++function compareCellValues(left: unknown, right: unknown) {+ const leftIsEmpty = left === null || left === undefined || left === "";+ const rightIsEmpty = right === null || right === undefined || right === "";++ if (leftIsEmpty && rightIsEmpty) return 0;+ if (leftIsEmpty) return 1;+ if (rightIsEmpty) return -1;++ if (typeof left === "boolean" && typeof right === "boolean") {+ return Number(left) - Number(right);+ }++ const leftNumber = typeof left === "number" ? left : typeof left === "string" ? Number(left) : Number.NaN;+ const rightNumber = typeof right === "number" ? right : typeof right === "string" ? Number(right) : Number.NaN;++ if (Number.isFinite(leftNumber) && Number.isFinite(rightNumber)) {+ return leftNumber - rightNumber;+ }++ return formatCellValue(left).localeCompare(formatCellValue(right), undefined, {+ numeric: true,+ sensitivity: "base",+ });+}++export function sortRows(rows: CustomQueryRow[], sort: SortState) {+ if (!sort) return rows;++ return rows+ .map((row, index) => ({ row, index }))+ .sort((left, right) => {+ const comparison = compareCellValues(left.row[sort.column], right.row[sort.column]);+ if (comparison === 0) return left.index - right.index;+ return sort.direction === "asc" ? comparison : -comparison;+ })+ .map(item => item.row);+}++export function getNextSortState(currentSort: SortState, column: string): SortState {+ if (!currentSort || currentSort.column !== column) {+ return { column, direction: "asc" };+ }++ if (currentSort.direction === "asc") {+ return { column, direction: "desc" };+ }++ return null;+}++export function isAbortError(error: unknown) {+ if (!(error instanceof Error)) return false;+ const codedError = error as Error & { code?: string };+ return error.name === "AbortError" || error.name === "CanceledError" || codedError.code === "ERR_CANCELED";+}++export function getErrorMessage(error: unknown, fallback: string) {+ return error instanceof Error && error.message ? error.message : fallback;+}++export function formatQuery(query: string) {+ try {+ return formatSql(query, {+ language: "sql",+ keywordCase: "upper",+ linesBetweenQueries: 1,+ });+ } catch {+ return query;+ }+}diff --git a/docker-compose.yml b/docker-compose.ymlindex 2f2999c91..114168e3f 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -91,6 +91,8 @@ services:- CLUSTER_WORKERS=${CLUSTER_WORKERS:-4}- DEPLOYMENT=${DEPLOYMENT}- LITE_DASHBOARD=${LITE_DASHBOARD}+ - OPENROUTER_API_KEY=${OPENROUTER_API_KEY}+ - OPENROUTER_MODEL=${OPENROUTER_MODEL}depends_on:clickhouse:condition: service_healthydiff --git a/server/src/api/analytics/generateCustomQuery.ts b/server/src/api/analytics/generateCustomQuery.tsnew file mode 100644index 000000000..02e952fe7--- /dev/null+++ b/server/src/api/analytics/generateCustomQuery.ts@@ -0,0 +1,290 @@+import { FastifyReply, FastifyRequest } from "fastify";+import { z } from "zod";+import { getSitesUserHasAccessTo } from "../../lib/auth-utils.js";+import { OpenRouterError, callOpenRouterWithMetadata, getOpenRouterModel } from "../../lib/openrouter.js";+import { MAX_CUSTOM_QUERY_LENGTH, normalizeCustomQuery, validateScopedQuery } from "./utils/customQueryValidation.js";++const OPENROUTER_TEMPERATURE = 0.1;+const OPENROUTER_MAX_TOKENS = 5000;+const LOG_PREVIEW_LENGTH = 500;++const generationMessageSchema = z.object({+ role: z.enum(["user", "assistant"]),+ content: z.string().trim().min(1).max(MAX_CUSTOM_QUERY_LENGTH),+});++const requestBodySchema = z.object({+ prompt: z.string().trim().min(1).max(4000),+ currentSiteId: z.number().int().positive().optional(),+ currentQuery: z.string().trim().max(MAX_CUSTOM_QUERY_LENGTH).optional(),+ history: z.array(generationMessageSchema).max(12).optional().default([]),+});++const EVENT_SCHEMA = `+scoped_events columns:+- site_id UInt16: Numeric Rybbit site id. scoped_events is already limited to sites the user can access.+- timestamp DateTime: Event ingest time in ClickHouse. Use ClickHouse date functions such as toStartOfDay(timestamp).+- session_id String: Anonymous visit/session id. Use countDistinct(session_id) for sessions.+- user_id String: Anonymous device/user fingerprint id. This is not necessarily a logged-in app user id.+- identified_user_id String: Custom user id set via identify(); empty string when the visitor was not identified.+- hostname String: Page hostname without protocol, for example www.example.com.+- pathname String: URL path, usually starting with '/', for example /pricing. Hash-router paths may be normalized into this field.+- querystring String: Raw URL query string, usually including the leading '?', for example ?utm_source=google.+- url_parameters Map(String, String): Parsed query parameters keyed by lowercase parameter name. Common keys include utm_source, utm_medium, utm_campaign, utm_term, utm_content, gclid, gad_source. Access with url_parameters['utm_source'].+- page_title String: document.title captured at event time; may be empty.+- referrer String: Full external referrer URL; empty for direct traffic and self-referrals.+- channel String: Derived acquisition channel. Expected values include Direct, Internal, Cross-Network, Paid AI, Paid Search, Paid Social, Paid Video, Paid Shopping, Display, Paid Influencer, Paid Audio, Paid Unknown, AI, Organic Search, Organic Social, Organic Video, Organic Shopping, Email, SMS, News, Productivity, Affiliate, Referral, Audio, Push, Influencer, Content, Event, Unknown.+- browser LowCardinality(String): Browser family from user-agent, for example Chrome, Safari, Firefox; may be empty.+- browser_version LowCardinality(String): Browser major version as a string, for example 124; may be empty.+- operating_system LowCardinality(String): OS family from user-agent, for example macOS, Windows, iOS, Android; may be empty.+- operating_system_version LowCardinality(String): OS version string from user-agent; may be empty.+- language LowCardinality(String): Browser language, usually a BCP 47 value like en-US; may be empty.+- country LowCardinality(FixedString(2)): ISO 3166-1 alpha-2 country code, for example US or GB; empty string when unknown.+- region LowCardinality(String): Region code, usually country-region like US-CA; empty string when unknown.+- city String: City name from IP geolocation; empty string when unknown.+- lat Float64: Latitude from IP geolocation; 0 when unknown.+- lon Float64: Longitude from IP geolocation; 0 when unknown.+- screen_width UInt16: Client screen width in pixels; 0 when unavailable.+- screen_height UInt16: Client screen height in pixels; 0 when unavailable.+- device_type LowCardinality(String): Derived device class. Expected values: Desktop, Mobile, Tablet, TV, Console, Embedded.+- type LowCardinality(String): Event kind. Valid values: pageview, custom_event, performance, outbound, error, button_click, copy, form_submit, input_change.+- event_name String: For custom_event this is the user-defined event name. For performance it is usually web-vitals. For error it is the error name, such as TypeError. Often empty for pageview, outbound, button_click, copy, form_submit, and input_change.+- props JSON: Event-specific JSON properties. For custom_event, arbitrary user properties. For outbound: url, text, target. For error: message, stack, fileName, lineNumber, columnNumber. For button_click: text plus data-rybbit-prop-* attributes. For copy: text, textLength, sourceElement. For form_submit: formId, formName, formAction, method, fieldCount, ariaLabel. For input_change: element, inputType, inputName, formId, formName. Use JSONExtractString(toString(props), 'key') for string properties.+- lcp Nullable(Float64): Largest Contentful Paint in milliseconds; only set on type = 'performance' web-vitals events.+- cls Nullable(Float64): Cumulative Layout Shift score; only set on type = 'performance' web-vitals events.+- inp Nullable(Float64): Interaction to Next Paint in milliseconds; only set on type = 'performance' web-vitals events.+- fcp Nullable(Float64): First Contentful Paint in milliseconds; only set on type = 'performance' web-vitals events.+- ttfb Nullable(Float64): Time to First Byte in milliseconds; only set on type = 'performance' web-vitals events.+- ip Nullable(String): Visitor IP address only when IP tracking is enabled for the site; otherwise null.+- timezone String: Visitor timezone from geolocation, usually an IANA timezone like America/New_York; empty when unknown.+- tag String: Optional site/script tag used to segment traffic; empty when unset.+- import_id Nullable(String): Import job id for rows loaded from imported analytics data; null for native Rybbit tracking.+`;++function extractSql(content: string) {+ const trimmed = content.trim();+ const fencedSql = trimmed.match(/```(?:sql)?\s*([\s\S]*?)```/i);+ const sql = fencedSql?.[1] ?? trimmed;+ return normalizeCustomQuery(sql.replace(/^sql\s*:/i, ""));+}++function truncateForLog(value: string | undefined, maxLength = LOG_PREVIEW_LENGTH) {+ if (!value) return undefined;+ return value.length > maxLength ? `${value.slice(0, maxLength)}...` : value;+}++function isAbortError(error: unknown) {+ if (!(error instanceof Error)) return false;+ return error.name === "AbortError" || error.message === "This operation was aborted";+}++function createRequestAbortSignal(request: FastifyRequest, reply: FastifyReply) {+ const abortController = new AbortController();+ const abort = () => {+ if (!reply.raw.writableEnded) {+ abortController.abort();+ }+ };++ request.raw.on("aborted", abort);+ reply.raw.on("close", abort);++ return {+ signal: abortController.signal,+ cleanup: () => {+ request.raw.off("aborted", abort);+ reply.raw.off("close", abort);+ },+ };+}++export async function generateCustomQuery(+ request: FastifyRequest<{+ Params: {+ organizationId: string;+ };+ Body: unknown;+ }>,+ reply: FastifyReply+) {+ const body = requestBodySchema.safeParse(request.body);+ if (!body.success) {+ return reply.status(400).send({ error: body.error.errors[0]?.message ?? "Invalid request body" });+ }++ const userSites = await getSitesUserHasAccessTo(request);+ const siteIds = userSites+ .filter(site => site.organizationId === request.params.organizationId)+ .map(site => site.siteId);++ if (siteIds.length === 0) {+ return reply.status(403).send({ error: "No access to organization or no sites found" });+ }++ if (body.data.currentSiteId && !siteIds.includes(body.data.currentSiteId)) {+ return reply.status(403).send({ error: "No access to current site" });+ }++ const currentSiteInstruction = body.data.currentSiteId+ ? `The user is currently viewing site_id ${body.data.currentSiteId}. If they say "this site" or do not ask for an organization-wide result, include WHERE site_id = ${body.data.currentSiteId}.`+ : "The query can summarize all accessible sites unless the prompt asks for a specific site_id.";+ const currentQuery = body.data.currentQuery?.trim();+ const previousMessages = body.data.history.slice(-10).map(message => ({+ role: message.role,+ content:+ message.role === "assistant"+ ? `Previously generated SQL:\n${message.content}`+ : `Previous user request:\n${message.content}`,+ }));+ const messages = [+ {+ role: "system" as const,+ content: `+You generate ClickHouse SQL for Rybbit custom analytics.+Return exactly one SQL query and no Markdown, explanation, comments, or semicolon.+The query must be a SELECT or WITH ... SELECT query.+The only readable table is scoped_events. Never read from events or any other table.+Never define or shadow scoped_events.+Use ClickHouse syntax.+Use LIMIT 1000 or smaller for detail/list queries.+For custom event properties, use JSONExtractString(toString(props), 'property_name').+Use the previous messages and current editor query as context.+If the user asks an incremental follow-up, revise the current editor query.+If the user clearly asks for a new query, a different analysis, or to start over, generate a fresh query.+If the current editor query is empty, generate a fresh query.+${currentSiteInstruction}+${EVENT_SCHEMA}++Good examples:+SELECT pathname, countIf(type = 'pageview') AS pageviews FROM scoped_events GROUP BY pathname ORDER BY pageviews DESC LIMIT 100+SELECT event_name, count() AS events FROM scoped_events WHERE type = 'custom_event' GROUP BY event_name ORDER BY events DESC LIMIT 100+SELECT toStartOfDay(timestamp) AS day, count() AS events FROM scoped_events GROUP BY day ORDER BY day ASC LIMIT 1000+ `.trim(),+ },+ ...previousMessages,+ {+ role: "user" as const,+ content: `+Current editor query:+${currentQuery || "(empty)"}++Current user request:+${body.data.prompt}+ `.trim(),+ },+ ];+ const generationContext = {+ organizationId: request.params.organizationId,+ currentSiteId: body.data.currentSiteId,+ accessibleSiteCount: siteIds.length,+ promptLength: body.data.prompt.length,+ promptPreview: truncateForLog(body.data.prompt),+ currentQueryLength: currentQuery?.length ?? 0,+ currentQueryPreview: truncateForLog(currentQuery),+ historyCount: body.data.history.length,+ historyRoles: body.data.history.map(message => message.role),+ historyContentLengths: body.data.history.map(message => message.content.length),+ messageCount: messages.length,+ messageCharCount: messages.reduce((total, message) => total + message.content.length, 0),+ openRouterModel: getOpenRouterModel(),+ openRouterTemperature: OPENROUTER_TEMPERATURE,+ openRouterMaxTokens: OPENROUTER_MAX_TOKENS,+ };+ const requestAbort = createRequestAbortSignal(request, reply);++ try {+ request.log.info(generationContext, "Generating custom analytics query");++ const { content: generated, metadata: openRouter } = await callOpenRouterWithMetadata(messages, {+ temperature: OPENROUTER_TEMPERATURE,+ maxTokens: OPENROUTER_MAX_TOKENS,+ signal: requestAbort.signal,+ });++ request.log.info(+ {+ ...generationContext,+ openRouter,+ generatedLength: generated.length,+ generatedPreview: truncateForLog(generated),+ },+ "OpenRouter returned custom analytics query candidate"+ );++ const query = extractSql(generated);+ const validationError = validateScopedQuery(query);+ if (validationError) {+ request.log.warn(+ {+ ...generationContext,+ openRouter,+ validationError,+ queryLength: query.length,+ queryPreview: truncateForLog(query),+ },+ "Generated custom query failed validation"+ );+ return reply.status(422).send({ error: "Generated query failed validation", details: validationError });+ }++ request.log.info(+ {+ ...generationContext,+ openRouter,+ queryLength: query.length,+ queryPreview: truncateForLog(query),+ },+ "Generated custom analytics query passed validation"+ );++ return reply.send({ query });+ } catch (error) {+ if (requestAbort.signal.aborted || isAbortError(error)) {+ request.log.info(generationContext, "Custom analytics query generation aborted");+ if (reply.raw.destroyed || reply.raw.writableEnded) {+ return reply;+ }+ return reply.status(499).send({ error: "Query generation aborted" });+ }++ request.log.error(+ {+ err: error,+ ...generationContext,+ openRouter: error instanceof OpenRouterError ? error.details : undefined,+ },+ "Failed to generate custom analytics query"+ );++ if (error instanceof Error) {+ if (+ error instanceof OpenRouterError &&+ error.code === "empty_content" &&+ error.details.finishReason === "length"+ ) {+ return reply.status(502).send({+ error: "AI provider hit the output token limit before returning SQL. Try a shorter prompt or simpler query.",+ });+ }++ if (+ error.message === "No response from OpenRouter" ||+ error.message.startsWith("OpenRouter returned an empty response")+ ) {+ return reply.status(502).send({ error: "AI provider returned an empty response. Try again." });+ }++ if (error.message.startsWith("OpenRouter API error")) {+ return reply.status(502).send({ error: "AI query generation provider error" });+ }++ if (error.message === "OPENROUTER_API_KEY is not configured") {+ return reply.status(500).send({ error: "AI query generation is not configured" });+ }+ }++ return reply.status(500).send({ error: "Failed to generate query" });+ } finally {+ requestAbort.cleanup();+ }+}diff --git a/server/src/api/analytics/index.ts b/server/src/api/analytics/index.tsindex ca8d7009a..d45295229 100644--- a/server/src/api/analytics/index.ts+++ b/server/src/api/analytics/index.ts@@ -38,6 +38,7 @@ export { generatePdfReport } from "./generatePdfReport.js";export { getErrorBucketed } from "./getErrorBucketed.js";export { getErrorEvents } from "./getErrorEvents.js";export { getErrorNames } from "./getErrorNames.js";+export { generateCustomQuery } from "./generateCustomQuery.js";export { getJourneys } from "./getJourneys.js";export { getLiveUsercount } from "./getLiveUsercount.js";export { getMetric } from "./getMetric.js";@@ -49,6 +50,7 @@ export { getOverviewBucketedLite } from "./lite/getOverviewBucketedLite.js";export { getMetricLite } from "./lite/getMetricLite.js";export { getPageTitles } from "./getPageTitles.js";export { getRetention } from "./getRetention.js";+export { runCustomQuery } from "./runCustomQuery.js";export { getSession } from "./sessions/getSession.js";export { getSessionLocations } from "./sessions/getSessionLocations.js";export { getSessions } from "./sessions/getSessions.js";diff --git a/server/src/api/analytics/runCustomQuery.test.ts b/server/src/api/analytics/runCustomQuery.test.tsnew file mode 100644index 000000000..f8ff3fa1b--- /dev/null+++ b/server/src/api/analytics/runCustomQuery.test.ts@@ -0,0 +1,69 @@+import { describe, expect, it } from "vitest";+import { validateScopedQuery } from "./utils/customQueryValidation.js";++describe("validateScopedQuery", () => {+ it("allows SELECT queries against scoped_events", () => {+ expect(validateScopedQuery("SELECT event_name, count() FROM scoped_events GROUP BY event_name")).toBeNull();+ });++ it("allows aliases that use blocked table names", () => {+ expect(validateScopedQuery("SELECT event_name, count() AS events FROM scoped_events GROUP BY event_name")).toBeNull();+ });++ it("allows WITH expressions that read from scoped_events", () => {+ expect(+ validateScopedQuery(`+ WITH event_name AS name+ SELECT name, count()+ FROM scoped_events+ GROUP BY name+ `)+ ).toBeNull();+ });++ it("allows CTEs derived from scoped_events", () => {+ expect(+ validateScopedQuery(`+ WITH top_events AS (+ SELECT event_name, count() AS event_count+ FROM scoped_events+ GROUP BY event_name+ )+ SELECT *+ FROM top_events+ `)+ ).toBeNull();+ });++ it("rejects direct reads from events", () => {+ expect(validateScopedQuery("SELECT count() FROM events")).toBe("Queries can only read from scoped_events");+ });++ it("rejects comma joins to other tables", () => {+ expect(validateScopedQuery("SELECT count() FROM scoped_events, hourly_events_by_site_mv_target")).toBe(+ "Queries can only read from scoped_events"+ );+ });++ it("rejects multiple statements", () => {+ expect(validateScopedQuery("SELECT count() FROM scoped_events; SELECT count() FROM scoped_events")).toBe(+ "Only one SQL statement is allowed"+ );+ });++ it("ignores blocked words inside string literals", () => {+ expect(validateScopedQuery("SELECT 'DROP TABLE events' AS label FROM scoped_events LIMIT 1")).toBeNull();+ });++ it("rejects redefining scoped_events", () => {+ expect(validateScopedQuery("WITH scoped_events AS (SELECT * FROM events) SELECT * FROM scoped_events")).toBe(+ "scoped_events is reserved and cannot be redefined"+ );+ });++ it("rejects table functions", () => {+ expect(validateScopedQuery("SELECT * FROM scoped_events UNION ALL SELECT * FROM s3('https://example.com')")).toBe(+ "s3() is not allowed in custom analytics queries"+ );+ });+});diff --git a/server/src/api/analytics/runCustomQuery.ts b/server/src/api/analytics/runCustomQuery.tsnew file mode 100644index 000000000..83b93024c--- /dev/null+++ b/server/src/api/analytics/runCustomQuery.ts@@ -0,0 +1,85 @@+import { FastifyReply, FastifyRequest } from "fastify";+import { z } from "zod";+import { clickhouse } from "../../db/clickhouse/clickhouse.js";+import { getSitesUserHasAccessTo } from "../../lib/auth-utils.js";+import { MAX_CUSTOM_QUERY_LENGTH, normalizeCustomQuery, validateScopedQuery } from "./utils/customQueryValidation.js";++const MAX_EXECUTION_TIME_SECONDS = 10;+const MAX_RESULT_ROWS = 1000;++const requestBodySchema = z.object({+ query: z.string().trim().min(1).max(MAX_CUSTOM_QUERY_LENGTH),+});++export async function runCustomQuery(+ request: FastifyRequest<{+ Params: {+ organizationId: string;+ };+ Body: unknown;+ }>,+ reply: FastifyReply+) {+ const body = requestBodySchema.safeParse(request.body);+ if (!body.success) {+ return reply.status(400).send({ error: body.error.errors[0]?.message ?? "Invalid request body" });+ }++ const validationError = validateScopedQuery(body.data.query);+ if (validationError) {+ return reply.status(400).send({ error: validationError });+ }++ const userSites = await getSitesUserHasAccessTo(request);+ const siteIds = userSites+ .filter(site => site.organizationId === request.params.organizationId)+ .map(site => site.siteId);++ if (siteIds.length === 0) {+ return reply.status(403).send({ error: "No access to organization or no sites found" });+ }++ const query = `+ WITH scoped_events AS (+ SELECT *+ FROM events+ PREWHERE site_id IN {siteIds:Array(UInt16)}+ )+ SELECT *+ FROM (+ ${normalizeCustomQuery(body.data.query)}+ )+ LIMIT {limit:UInt32}+ `;++ try {+ const result = await clickhouse.query({+ query,+ format: "JSONEachRow",+ query_params: {+ siteIds,+ limit: MAX_RESULT_ROWS,+ },+ clickhouse_settings: {+ max_execution_time: MAX_EXECUTION_TIME_SECONDS,+ max_result_rows: String(MAX_RESULT_ROWS),+ result_overflow_mode: "break",+ readonly: "2",+ },+ });++ const data = await result.json<Record<string, unknown>>();+ return reply.send({+ data,+ meta: {+ queryId: result.query_id,+ rowCount: data.length,+ maxExecutionTimeSeconds: MAX_EXECUTION_TIME_SECONDS,+ maxRows: MAX_RESULT_ROWS,+ },+ });+ } catch (error) {+ request.log.error(error, "Failed to run custom analytics query");+ return reply.status(400).send({ error: "Failed to run query" });+ }+}diff --git a/server/src/api/analytics/utils/customQueryValidation.ts b/server/src/api/analytics/utils/customQueryValidation.tsnew file mode 100644index 000000000..37a6dd5e2--- /dev/null+++ b/server/src/api/analytics/utils/customQueryValidation.ts@@ -0,0 +1,218 @@+export const MAX_CUSTOM_QUERY_LENGTH = 20_000;++const blockedKeywords = [+ "ALTER",+ "ATTACH",+ "BACKUP",+ "CREATE",+ "DELETE",+ "DESCRIBE",+ "DETACH",+ "DROP",+ "EXCHANGE",+ "EXPLAIN",+ "FORMAT",+ "GRANT",+ "INSERT",+ "KILL",+ "OPTIMIZE",+ "RENAME",+ "RESTORE",+ "REVOKE",+ "SET",+ "SETTINGS",+ "SHOW",+ "SYSTEM",+ "TRUNCATE",+ "USE",+ "WATCH",+] as const;++const blockedFunctions = [+ "azureBlobStorage",+ "cluster",+ "cosn",+ "executable",+ "file",+ "format",+ "generateRandom",+ "hdfs",+ "input",+ "jdbc",+ "merge",+ "mongodb",+ "mysql",+ "numbers",+ "odbc",+ "postgresql",+ "remote",+ "s3",+ "s3Cluster",+ "url",+] as const;++const blockedTableNames = [+ "bot_events",+ "events",+ "hourly_events_by_site_mv",+ "hourly_events_by_site_mv_target",+ "information_schema",+ "monitor_events",+ "session_replay_events",+ "session_replay_metadata",+ "system",+] as const;++function stripSqlLiteralsAndComments(query: string) {+ let result = "";+ let index = 0;+ let state: "normal" | "single" | "double" | "backtick" | "line-comment" | "block-comment" = "normal";++ while (index < query.length) {+ const char = query[index];+ const next = query[index + 1];++ if (state === "normal") {+ if (char === "'") {+ state = "single";+ result += " ";+ } else if (char === "\"") {+ state = "double";+ result += " ";+ } else if (char === "`") {+ state = "backtick";+ result += " ";+ } else if (char === "-" && next === "-") {+ state = "line-comment";+ result += " ";+ index++;+ } else if (char === "/" && next === "*") {+ state = "block-comment";+ result += " ";+ index++;+ } else {+ result += char;+ }+ } else if (state === "single") {+ if (char === "\\" && next !== undefined) {+ result += " ";+ index++;+ } else if (char === "'" && next === "'") {+ result += " ";+ index++;+ } else if (char === "'") {+ state = "normal";+ result += " ";+ } else {+ result += char === "\n" ? "\n" : " ";+ }+ } else if (state === "double") {+ if (char === "\\" && next !== undefined) {+ result += " ";+ index++;+ } else if (char === "\"") {+ state = "normal";+ result += " ";+ } else {+ result += char === "\n" ? "\n" : " ";+ }+ } else if (state === "backtick") {+ if (char === "`") {+ state = "normal";+ }+ result += " ";+ } else if (state === "line-comment") {+ if (char === "\n") {+ state = "normal";+ result += "\n";+ } else {+ result += " ";+ }+ } else if (state === "block-comment") {+ if (char === "*" && next === "/") {+ state = "normal";+ result += " ";+ index++;+ } else {+ result += char === "\n" ? "\n" : " ";+ }+ }++ index++;+ }++ return result;+}++export function normalizeCustomQuery(query: string) {+ return query.trim().replace(/;+$/g, "").trim();+}++function getCteNames(query: string) {+ const cteNames = new Set<string>();+ const ctePattern = /(?:\bWITH|,)\s*([a-zA-Z_][a-zA-Z0-9_]*)\s+AS\s*\(/gi;+ let match: RegExpExecArray | null;++ while ((match = ctePattern.exec(query)) !== null) {+ cteNames.add(match[1].toLowerCase());+ }++ return cteNames;+}++export function validateScopedQuery(query: string): string | null {+ const normalizedQuery = normalizeCustomQuery(query);+ const queryWithoutLiterals = stripSqlLiteralsAndComments(normalizedQuery);+ const compactQuery = queryWithoutLiterals.trim();+ const cteNames = getCteNames(compactQuery);++ if (!/^(SELECT|WITH)\b/i.test(compactQuery)) {+ return "Only SELECT queries are allowed";+ }++ if (compactQuery.includes(";")) {+ return "Only one SQL statement is allowed";+ }++ for (const keyword of blockedKeywords) {+ if (new RegExp(`\\b${keyword}\\b`, "i").test(compactQuery)) {+ return `${keyword} is not allowed in custom analytics queries`;+ }+ }++ for (const fn of blockedFunctions) {+ if (new RegExp(`\\b${fn}\\s*\\(`, "i").test(compactQuery)) {+ return `${fn}() is not allowed in custom analytics queries`;+ }+ }++ if (/\bWITH\s+scoped_events\s+AS\b/i.test(compactQuery) || /\bAS\s+scoped_events\b/i.test(compactQuery)) {+ return "scoped_events is reserved and cannot be redefined";+ }++ for (const tableName of blockedTableNames) {+ if (new RegExp(`,\\s*${tableName}\\b`, "i").test(compactQuery)) {+ return "Queries can only read from scoped_events";+ }+ }++ const tableReferencePattern = /\b(?:FROM|JOIN)\s+([a-zA-Z_][a-zA-Z0-9_.]*|\()/gi;+ let match: RegExpExecArray | null;+ while ((match = tableReferencePattern.exec(compactQuery)) !== null) {+ const tableName = match[1];+ if (tableName === "(") {+ continue;+ }++ const normalizedTableName = tableName.toLowerCase();+ if (normalizedTableName !== "scoped_events" && !cteNames.has(normalizedTableName)) {+ return "Queries can only read from scoped_events";+ }+ }++ if (!/\bscoped_events\b/i.test(compactQuery)) {+ return "Query must read from scoped_events";+ }++ return null;+}diff --git a/server/src/index.ts b/server/src/index.tsindex 4d39098c1..5efa8d788 100644--- a/server/src/index.ts+++ b/server/src/index.ts@@ -25,6 +25,7 @@ import {getErrorBucketed,getErrorEvents,getErrorNames,+ generateCustomQuery,getEventBucketed,getEventNames,getEventProperties,@@ -60,6 +61,7 @@ import {getUserTraitValueUsers,getUserTraitValues,getUsers,+ runCustomQuery,updateGoal,} from "./api/analytics/index.js";import { getConfig, getVersion } from "./api/getConfig.js";@@ -310,6 +312,8 @@ async function analyticsRoutes(fastify: FastifyInstance) {fastify.get("/sites/:siteId/events/properties", publicSite, getEventProperties);fastify.get("/sites/:siteId/events/outbound", publicSite, getOutboundLinks);fastify.get("/org-event-count/:organizationId", orgMember, getOrgEventCount);+ fastify.post("/organizations/:organizationId/analytics/query", orgMember, runCustomQuery);+ fastify.post("/organizations/:organizationId/analytics/query/generate", orgMember, generateCustomQuery);fastify.get("/sites/:siteId/performance/overview", publicSite, getPerformanceOverview);fastify.get("/sites/:siteId/performance/time-series", publicSite, getPerformanceTimeSeries);fastify.get("/sites/:siteId/performance/by-dimension", publicSite, getPerformanceByDimension);diff --git a/server/src/lib/openrouter.test.ts b/server/src/lib/openrouter.test.tsnew file mode 100644index 000000000..f2488f592--- /dev/null+++ b/server/src/lib/openrouter.test.ts@@ -0,0 +1,53 @@+import { afterEach, describe, expect, it, vi } from "vitest";+import { callOpenRouter } from "./openrouter.js";++function mockOpenRouterResponse(body: unknown) {+ vi.stubGlobal(+ "fetch",+ vi.fn().mockResolvedValue({+ ok: true,+ json: async () => body,+ })+ );+}++describe("callOpenRouter", () => {+ afterEach(() => {+ vi.unstubAllEnvs();+ vi.unstubAllGlobals();+ });++ it("returns assistant content", async () => {+ vi.stubEnv("OPENROUTER_API_KEY", "test-key");+ mockOpenRouterResponse({+ id: "completion-id",+ choices: [+ {+ message: { role: "assistant", content: "SELECT count() FROM scoped_events" },+ finish_reason: "stop",+ },+ ],+ });++ await expect(callOpenRouter([{ role: "user", content: "count events" }])).resolves.toBe(+ "SELECT count() FROM scoped_events"+ );+ });++ it("rejects null assistant content", async () => {+ vi.stubEnv("OPENROUTER_API_KEY", "test-key");+ mockOpenRouterResponse({+ id: "completion-id",+ choices: [+ {+ message: { role: "assistant", content: null },+ finish_reason: "stop",+ },+ ],+ });++ await expect(callOpenRouter([{ role: "user", content: "count events" }])).rejects.toThrow(+ "OpenRouter returned an empty response"+ );+ });+});diff --git a/server/src/lib/openrouter.ts b/server/src/lib/openrouter.tsindex 0da4d5e70..f906e4e2d 100644--- a/server/src/lib/openrouter.ts+++ b/server/src/lib/openrouter.ts@@ -1,29 +1,93 @@const OPENROUTER_API_URL = "https://openrouter.ai/api/v1/chat/completions";+const DEFAULT_OPENROUTER_MODEL = "moonshotai/kimi-k2.6";interface OpenRouterResponse {- id: string;- choices: Array<{- message: {- role: string;- content: string;+ id?: string;+ model?: string;+ provider?: string;+ choices?: Array<{+ message?: {+ role?: string;+ content?: string | null;};- finish_reason: string;+ finish_reason?: string | null;+ native_finish_reason?: string | null;}>;+ usage?: unknown;+ error?: unknown;+}++export type OpenRouterErrorCode = "missing_api_key" | "http_error" | "invalid_json" | "empty_choices" | "empty_content";++export type OpenRouterMetadata = {+ model: string;+ responseModel?: string;+ responseId?: string;+ requestId?: string;+ provider?: string;+ status?: number;+ statusText?: string;+ choiceCount?: number;+ finishReason?: string | null;+ nativeFinishReason?: string | null;+ usage?: unknown;+ responseError?: unknown;+ responseBodyPreview?: string;+ contentType?: string;+ contentLength?: number;+ messageRole?: string;+ messageContentType?: string;+ messageContentLength?: number;+};++type OpenRouterOptions = {+ temperature?: number;+ maxTokens?: number;+ model?: string;+ signal?: AbortSignal;+};++export class OpenRouterError extends Error {+ code: OpenRouterErrorCode;+ details: OpenRouterMetadata;++ constructor(code: OpenRouterErrorCode, message: string, details: OpenRouterMetadata) {+ super(message);+ this.name = "OpenRouterError";+ this.code = code;+ this.details = details;+ }+}++function truncateForLog(value: string, maxLength = 1000) {+ return value.length > maxLength ? `${value.slice(0, maxLength)}...` : value;+}++function getResponseHeader(response: Response, header: string) {+ return (response as Response & { headers?: { get?: (name: string) => string | null } }).headers?.get?.(header);+}++export function getOpenRouterModel(model?: string) {+ return model || process.env.OPENROUTER_MODEL || DEFAULT_OPENROUTER_MODEL;}export async function callOpenRouter(messages: Array<{ role: "system" | "user" | "assistant"; content: string }>,- options?: {- temperature?: number;- maxTokens?: number;- model?: string;- }+ options?: OpenRouterOptions): Promise<string> {+ const response = await callOpenRouterWithMetadata(messages, options);+ return response.content;+}++export async function callOpenRouterWithMetadata(+ messages: Array<{ role: "system" | "user" | "assistant"; content: string }>,+ options?: OpenRouterOptions+): Promise<{ content: string; metadata: OpenRouterMetadata }> {const apiKey = process.env.OPENROUTER_API_KEY;- const model = options?.model || process.env.OPENROUTER_MODEL || "moonshotai/kimi-k2.5";+ const model = getOpenRouterModel(options?.model);if (!apiKey) {- throw new Error("OPENROUTER_API_KEY is not configured");+ throw new OpenRouterError("missing_api_key", "OPENROUTER_API_KEY is not configured", { model });}const response = await fetch(OPENROUTER_API_URL, {@@ -40,18 +104,75 @@ export async function callOpenRouter(temperature: options?.temperature ?? 0.3,max_tokens: options?.maxTokens ?? 1000,}),+ signal: options?.signal,});+ const baseMetadata: OpenRouterMetadata = {+ model,+ status: response.status,+ statusText: response.statusText,+ requestId:+ getResponseHeader(response, "x-request-id") ??+ getResponseHeader(response, "x-openrouter-request-id") ??+ getResponseHeader(response, "cf-ray") ??+ undefined,+ contentType: getResponseHeader(response, "content-type") ?? undefined,+ contentLength: Number(getResponseHeader(response, "content-length")) || undefined,+ };if (!response.ok) {- const error = await response.text();- throw new Error(`OpenRouter API error: ${response.status} - ${error}`);+ const responseBody = await response.text().catch(() => "");+ throw new OpenRouterError("http_error", `OpenRouter API error: ${response.status}`, {+ ...baseMetadata,+ responseBodyPreview: truncateForLog(responseBody),+ });}- const data = (await response.json()) as OpenRouterResponse;+ let data: OpenRouterResponse;+ try {+ data = (await response.json()) as OpenRouterResponse;+ } catch (error) {+ throw new OpenRouterError("invalid_json", "OpenRouter returned invalid JSON", {+ ...baseMetadata,+ responseError: error instanceof Error ? error.message : String(error),+ });+ }if (!data.choices || data.choices.length === 0) {- throw new Error("No response from OpenRouter");+ throw new OpenRouterError("empty_choices", "No response from OpenRouter", {+ ...baseMetadata,+ responseId: data.id,+ responseModel: data.model,+ provider: data.provider,+ choiceCount: data.choices?.length ?? 0,+ usage: data.usage,+ responseError: data.error,+ });+ }++ const choice = data.choices[0];+ const content = choice.message?.content;+ const metadata: OpenRouterMetadata = {+ ...baseMetadata,+ responseId: data.id,+ responseModel: data.model,+ provider: data.provider,+ choiceCount: data.choices.length,+ finishReason: choice.finish_reason,+ nativeFinishReason: choice.native_finish_reason,+ usage: data.usage,+ responseError: data.error,+ messageRole: choice.message?.role,+ messageContentType: Array.isArray(content) ? "array" : typeof content,+ messageContentLength: typeof content === "string" ? content.length : undefined,+ };++ if (typeof content !== "string" || content.trim().length === 0) {+ throw new OpenRouterError(+ "empty_content",+ `OpenRouter returned an empty response${choice.finish_reason ? ` (${choice.finish_reason})` : ""}`,+ metadata+ );}- return data.choices[0].message.content;+ return { content, metadata };}