The only tool you need to know what is happening and how to fix it.
Go alertingexception-trackerissue-trackinglogsmetricsmonitoringobservability-platformopen-telemetryotelsession-replaytelemetrytraces
docs updated
ddux7c92bb22eb1c
diff --git a/CLAUDE.md b/CLAUDE.mdindex bc4f99c6..1180cfbe 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -244,6 +244,10 @@ POSTGRES_DATABASE=tracewayPOSTGRES_USERNAME=tracewayPOSTGRES_PASSWORD=POSTGRES_SSLMODE=disable++# Retention (see "Data Retention" section below)+SQLITE_RETENTION_DAYS=30 # 0 to disable; only applies in SQLite mode+SESSION_RECORDING_RETENTION_DAYS=30 # 0 to disable; only applies when STORAGE_TYPE=local```---@@ -652,7 +656,9 @@ func (c *ReportController) Report(ctx *gin.Context) {| `metric_records` | Time-series system metrics | Monthly || `endpoints` | Endpoint aggregates (materialized) | None || `archived_exceptions` | Archived/resolved exceptions | None |-| `log_records` | OTel logs with 3 attribute maps (resource/scope/log), 30-day TTL | Daily (`toDate(timestamp)`) |+| `log_records` | OTel logs with 3 attribute maps (resource/scope/log) | Daily (`toDate(timestamp)`) |++> Data retention/TTLs are documented separately under **Data Retention** below.#### Tables (PostgreSQL)| Table | Purpose |@@ -716,6 +722,46 @@ for _, item := range items {- `SQLiteJSONMap` — implements `sql.Scanner`/`driver.Valuer` for `map[string]string` ↔ SQLite JSON TEXT- Row types (e.g., `endpointRow`, `taskRow`) wrap domain models with these types for lit compatibility+#### Data Retention++Retention is handled in three different ways depending on the deployment.++**1. ClickHouse — `TTL` clauses on the table itself.** Only a few tables have a TTL; everything else is kept indefinitely (operators can drop monthly partitions manually if needed).++| Table | Retention | Source migration |+|-------|-----------|------------------|+| `metric_points` (raw) | **7 days** | `0034_add_ttl_metric_points.up.sql` |+| `metric_points_1m` (1-min rollup) | **30 days** | `0035_add_ttl_metric_points_1m.up.sql` |+| `metric_points_1h` (1-hour rollup) | **1 year** | `0036_add_ttl_metric_points_1h.up.sql` |+| `log_records` | **30 days** | `0045_create_log_records.up.sql` |+| All other CH tables (`transactions`, `exception_stack_traces`, `tasks`, `spans`, `sessions`, `ai_traces`, `session_recordings`, `fired_notifications`, `archived_exceptions`, `slow_endpoints`, `endpoints`, etc.) | **No TTL — retained indefinitely** | — |++**2. SQLite — `retention.Start` worker** (`backend/app/retention/sqlite.go`). In SQLite mode, neither `db.DB` nor `db.TelemetryDB` has any built-in expiry, so a background worker fires once at startup and then every hour and runs a `DELETE FROM <table> WHERE <time_column> < cutoff` against each telemetry table plus `notification_history`.++| Variable | Default | Notes |+|----------|---------|-------|+| `SQLITE_RETENTION_DAYS` | `30` | TTL in days. Set to `0` to disable the worker entirely. Has no effect outside SQLite mode. |++Tables it prunes (and the column used):++| Database | Table | Time column |+|----------|-------|-------------|+| Telemetry (`db.TelemetryDB`) | `endpoints`, `tasks`, `exception_stack_traces`, `spans`, `metric_points`, `session_recordings`, `ai_traces` | `recorded_at` |+| Telemetry | `log_records` | `timestamp` |+| Telemetry | `sessions` | `started_at` |+| Telemetry | `fired_notifications` | `fired_at` |+| Main (`db.DB`) | `notification_history` | `created_at` |++`archived_exceptions` (per-hash flags) and `slow_endpoints` (per-endpoint config) are intentionally skipped — they are not time-series data.++**3. On-disk session recordings — `retention.Start` worker** (`backend/app/retention/recordings.go`). Session recordings written to local disk (`STORAGE_TYPE=local`) accumulate under `<STORAGE_PATH>/recordings/`. A second worker walks that directory once at startup and then every hour and removes files whose `mtime` is older than the TTL, then prunes any directories left empty. The worker is a no-op when `STORAGE_TYPE=s3`.++| Variable | Default | Notes |+|----------|---------|-------|+| `SESSION_RECORDING_RETENTION_DAYS` | `30` | TTL in days. Set to `0` to disable the worker. Only runs when `STORAGE_TYPE=local` (default). |++The DB rows in `session_recordings` are pruned by the SQLite retention worker (above) or by ClickHouse TTL — they are intentionally not coupled to the disk cleanup. Controllers that read recordings already log a non-fatal `traceway.CaptureException` when a referenced file is missing.+#### Organization Roles| Role | Description ||------|-------------|diff --git a/DOCKER.md b/DOCKER.mdindex 9a3fefa9..535e98b0 100644--- a/DOCKER.md+++ b/DOCKER.md@@ -45,6 +45,7 @@ All have working defaults for the all-in-one image. Override as needed:| `POSTGRES_SSLMODE` | `disable` | PostgreSQL SSL mode || `JWT_SECRET` | *(built-in default)* | JWT signing secret (min 32 chars) || `GIN_MODE` | `release` | Gin framework mode |+| `SESSION_RECORDING_RETENTION_DAYS` | `30` | Days to keep on-disk session recordings under `STORAGE_PATH/recordings/`. Worker runs hourly and on startup. `0` disables; no effect when `STORAGE_TYPE=s3`. |### Access Points@@ -82,6 +83,7 @@ docker build -f Dockerfile.minimal -t traceway:minimal .| `POSTGRES_PASSWORD` | PostgreSQL password | `password` || `POSTGRES_SSLMODE` | PostgreSQL SSL mode | `disable` || `JWT_SECRET` | JWT signing secret (min 32 chars) | `your-secret-here` |+| `SESSION_RECORDING_RETENTION_DAYS` | Optional. Days to keep on-disk session recordings (default `30`, `0` disables, no effect when `STORAGE_TYPE=s3`). | `30` |### Run@@ -125,6 +127,8 @@ services:POSTGRES_PASSWORD: "traceway"POSTGRES_SSLMODE: "disable"JWT_SECRET: "change-this-to-a-secure-secret-at-least-32-chars"+ # Optional. Days to keep on-disk session recordings (default 30, 0 disables, no effect when STORAGE_TYPE=s3).+ # SESSION_RECORDING_RETENTION_DAYS: "30"depends_on:clickhouse:condition: service_healthydiff --git a/backend/.env.docker b/backend/.env.dockerindex 303cfa02..e9799105 100644--- a/backend/.env.docker+++ b/backend/.env.docker@@ -11,3 +11,9 @@ POSTGRES_PASSWORD=POSTGRES_SSLMODE=disableJWT_SECRET=Xt7Kj2mP9qLwNc4rVb8sYd3hFgAeZuCpDnRoWxMvBa5kGIN_MODE=release++# Optional retention knobs (defaults shown). Set to 0 to disable.+# SESSION_RECORDING_RETENTION_DAYS only takes effect when STORAGE_TYPE=local.+# SQLITE_RETENTION_DAYS only takes effect when DB_TYPE=sqlite.+#SESSION_RECORDING_RETENTION_DAYS=30+#SQLITE_RETENTION_DAYS=30diff --git a/docker-compose.sqlite.yml b/docker-compose.sqlite.ymlindex 89013066..de4b42b6 100644--- a/docker-compose.sqlite.yml+++ b/docker-compose.sqlite.yml@@ -9,6 +9,13 @@ services:JWT_SECRET: "change-this-to-a-secure-secret-at-least-32-chars"# APP_BASE_URL: "https://traceway.example.com"+ # Optional retention knobs. Defaults are 30 days; set to "0" to disable.+ # SQLITE_RETENTION_DAYS prunes telemetry rows; SESSION_RECORDING_RETENTION_DAYS+ # cleans on-disk recordings under STORAGE_PATH/recordings/. Both run hourly+ # and on startup. The recording cleanup has no effect when STORAGE_TYPE=s3.+ # SQLITE_RETENTION_DAYS: "30"+ # SESSION_RECORDING_RETENTION_DAYS: "30"+# Optional: store blobs (source maps, session recordings, AI traces) in S3# instead of the local /data/storage folder.# STORAGE_TYPE: "s3"diff --git a/docker-compose.yml b/docker-compose.ymlindex c2bb33e7..df9e19ba 100644--- a/docker-compose.yml+++ b/docker-compose.yml@@ -18,6 +18,9 @@ services:POSTGRES_PASSWORD: "traceway"POSTGRES_SSLMODE: "disable"JWT_SECRET: "change-this-to-a-secure-secret-at-least-32-chars"+ # Optional. Days to keep on-disk session recordings under STORAGE_PATH/recordings/.+ # Worker runs hourly and on startup; 0 disables; no effect when STORAGE_TYPE=s3.+ # SESSION_RECORDING_RETENTION_DAYS: "30"depends_on:clickhouse:condition: service_healthydiff --git a/docs/pages/server/all-in-one.mdx b/docs/pages/server/all-in-one.mdxindex c111c1d2..231b4bb2 100644--- a/docs/pages/server/all-in-one.mdx+++ b/docs/pages/server/all-in-one.mdx@@ -40,6 +40,9 @@ All environment variables have working defaults, no configuration required. Over| `JWT_SECRET` | *(built-in default)* | JWT signing secret (min 32 chars) || `GIN_MODE` | `release` | Gin framework mode || `APP_BASE_URL` | *(none)* | Public URL of your Traceway instance (e.g. `https://traceway.example.com`). Used for SDK setup instructions, email links, and OAuth callback URLs. |+| `SESSION_RECORDING_RETENTION_DAYS` | `30` | Days to keep session recording files on disk under `STORAGE_PATH/recordings/`. Files older than the TTL are deleted hourly and on startup. No effect when `STORAGE_TYPE=s3`. Set to `0` to disable. |++> ClickHouse data retention is controlled per-table at the schema level (TTLs on `metric_points`, `metric_points_1m`, `metric_points_1h`, `log_records`); other telemetry tables are retained indefinitely until you drop monthly partitions manually. The `SQLITE_RETENTION_DAYS` setting only applies in SQLite mode and has no effect in this image.## SSO (optional)diff --git a/docs/pages/server/docker-compose.mdx b/docs/pages/server/docker-compose.mdxindex 0fc72e9d..7e49c4f2 100644--- a/docs/pages/server/docker-compose.mdx+++ b/docs/pages/server/docker-compose.mdx@@ -33,6 +33,9 @@ The `docker-compose.yml` in the repository root is ready to use out of the box.| `POSTGRES_SSLMODE` | `disable` | PostgreSQL SSL mode || `JWT_SECRET` | `change-this-to-a-secure-secret-at-least-32-chars` | JWT signing secret (min 32 chars) || `APP_BASE_URL` | *(none)* | Public URL of your Traceway instance (e.g. `https://traceway.example.com`). Used for SDK setup instructions, email links, and OAuth callback URLs. |+| `SESSION_RECORDING_RETENTION_DAYS` | `30` | Days to keep session recording files on disk under `STORAGE_PATH/recordings/`. A worker deletes older files hourly and on startup. No effect when `STORAGE_TYPE=s3`. Set to `0` to disable. |++> ClickHouse table TTLs (`metric_points`, `log_records`, etc.) are managed at the schema level; this stack does not run the SQLite retention worker.## SSO (optional)diff --git a/docs/pages/server/haloy.mdx b/docs/pages/server/haloy.mdxindex b80b12d2..68418dba 100644--- a/docs/pages/server/haloy.mdx+++ b/docs/pages/server/haloy.mdx@@ -181,6 +181,7 @@ Point your SDK's connection string at your Traceway domain:| `GITHUB_CLIENT_ID` | *(unset)* | Optional. GitHub OAuth App client ID. Setting both GitHub variables enables the **Continue with GitHub** button. || `GITHUB_CLIENT_SECRET` | *(unset)* | Optional. GitHub OAuth App client secret. || `OAUTH_SESSION_SECRET` | *(falls back to `JWT_SECRET`)* | Optional. Cookie signing secret for the OAuth round-trip. |+| `SESSION_RECORDING_RETENTION_DAYS` | `30` | Optional. Days to keep on-disk session recordings under `STORAGE_PATH/recordings/`. Worker runs hourly and on startup. `0` disables; no effect when `STORAGE_TYPE=s3`. |When configuring providers, set the callback URL on the provider side to `https://traceway.yourdomain.com/api/auth/callback/{google|github}`.diff --git a/docs/pages/server/local-setup.mdx b/docs/pages/server/local-setup.mdxindex 981f9deb..b7d503e3 100644--- a/docs/pages/server/local-setup.mdx+++ b/docs/pages/server/local-setup.mdx@@ -79,6 +79,8 @@ POSTGRES_SSLMODE=disable| `GITHUB_CLIENT_ID` | Optional. GitHub OAuth App client ID. Setting both GitHub variables enables the **Continue with GitHub** button. See the [SSO guide](/learn/sso). || `GITHUB_CLIENT_SECRET` | Optional. GitHub OAuth App client secret. || `OAUTH_SESSION_SECRET` | Optional. Cookie signing secret for the OAuth round-trip. Falls back to `JWT_SECRET` when unset. |+| `SESSION_RECORDING_RETENTION_DAYS` | Optional. Days to keep session recording files on disk under `STORAGE_PATH/recordings/`. Defaults to `30`. Worker runs hourly and on startup. `0` disables; no effect when `STORAGE_TYPE=s3`. |+| `SQLITE_RETENTION_DAYS` | Optional. Days to keep SQLite telemetry rows. Defaults to `30`. `0` disables. Only applies when running with `DB_TYPE=sqlite`. |For local OAuth testing, register the provider's callback URL as `http://localhost:5173/api/auth/callback/github` (or `.../google`).diff --git a/docs/pages/server/minimal.mdx b/docs/pages/server/minimal.mdxindex 758d33cc..404f7d68 100644--- a/docs/pages/server/minimal.mdx+++ b/docs/pages/server/minimal.mdx@@ -47,6 +47,7 @@ After starting, open `http://localhost/register` to create your first account.| `POSTGRES_SSLMODE` | PostgreSQL SSL mode | `disable` || `JWT_SECRET` | JWT signing secret (min 32 chars) | `your-secret-here` || `APP_BASE_URL` | Public URL of your instance. Used for SDK setup instructions, email links, and OAuth callback URLs. | `https://traceway.example.com` |+| `SESSION_RECORDING_RETENTION_DAYS` | Days to keep on-disk session recordings under `STORAGE_PATH/recordings/`. Defaults to `30`. Worker runs hourly and on startup. `0` disables; no effect when `STORAGE_TYPE=s3`. | `30` |## SSO (optional)diff --git a/docs/pages/server/sqlite.mdx b/docs/pages/server/sqlite.mdxindex e57c3d2d..73091afc 100644--- a/docs/pages/server/sqlite.mdx+++ b/docs/pages/server/sqlite.mdx@@ -62,6 +62,8 @@ If `STORAGE_TYPE` is unset (or set to `local`), blobs are written to `/data/stor| `S3_ENDPOINT` | *(unset)* | Optional. Set for S3-compatible providers (MinIO, R2, Spaces). Enables path-style URLs || `PORTS` | `80,8082` | Comma-separated list of HTTP ports the server should listen on || `GIN_MODE` | `release` | Gin runtime mode |+| `SQLITE_RETENTION_DAYS` | `30` | Days to keep telemetry rows (endpoints, exceptions, spans, metrics, logs, sessions, recordings, fired notifications, notification history). A background worker prunes older rows once at startup and then every hour. Set to `0` to disable. |+| `SESSION_RECORDING_RETENTION_DAYS` | `30` | Days to keep session recording files on disk under `STORAGE_PATH/recordings/`. Files older than the TTL are deleted hourly and on startup; empty subdirectories are removed. No effect when `STORAGE_TYPE=s3`. Set to `0` to disable. |## SSO (optional)