The only tool you need to know what is happening and how to fix it.
Go alertingexception-trackerissue-trackinglogsmetricsmonitoringobservability-platformopen-telemetryotelsession-replaytelemetrytraces
fix: start the backend without a preconfigured environment (#305)
FrameAutomata6d502b1b0ab1
diff --git a/CLAUDE.md b/CLAUDE.mdindex 1c9aeee8..a6eee3aa 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -244,7 +244,10 @@ if err != nil {### Environment Variables (Backend)```-JWT_SECRET=<min 32 char secret for JWT signing>+JWT_SECRET=<min 32 char secret for JWT signing> # the only variable with no default. There is deliberately no fallback: a key committed to the repo would be identical in every clone. Unset or under 32 chars exits 1 with an actionable message rather than a panic.+DB_TYPE= # sqlite | postgres. Defaults to sqlite in every build EXCEPT -tags transactional_pg, where an empty value still selects PostgreSQL. Without that tag SQLite is the only supported main DB (the migration runner applies SQLite-dialect migrations unconditionally), so the default is set once in config.defaultDBType rather than at each DBType call site.+PORTS=80,8082 # comma-separated HTTP listen ports. Every port is bound up front; a port that fails to bind (typically :80 as a non-root user) logs a warning and is reported via CaptureException, and startup only fails if NO port binds. Order is not significant.+SQLITE_PATH= # main DB path, defaults to ./traceway.db in the working directory. The telemetry DB derives from it (_telemetry.db, or _telemetry.duckdb under -tags telemetry_duckdb).APP_BASE_URL= # public origin of this server (e.g. https://traceway.example.com). Used as the OAuth issuer / device verification URL and SSO redirect base. If unset, the device-auth + well-known endpoints derive it per-request from the Host / X-Forwarded-* headers; set it explicitly behind a proxy that doesn't forward Host.TRUSTED_PROXIES= # comma-separated IPs/CIDRs whose X-Forwarded-For/X-Real-IP gin trusts for c.ClientIP() (every per-IP rate limiter keys on it, and `/api/report` stores it on every session as `client.ip`). Unset = loopback only, so no peer can spoof its IP. Private ranges are deliberately NOT trusted by default: on a Docker/k8s network every peer has a private address, the clients included when the port is published directly, so trusting RFC1918 would let any of them forge XFF past every per-IP limiter. Set it to the CIDRs of whatever sits in front (a proxy container's compose network, the ingress pod range, the CDN ranges); otherwise every visitor shares one rate-limit bucket keyed on the proxy. The value REPLACES the default, so it must name every hop in the chain, not just the outermost (a CDN-only list leaves a private ingress untrusted and disables XFF entirely); `*` trusts every peer. A malformed entry panics at startup (`configureClientIP`, cmd/run.go). The first request carrying X-Forwarded-For/X-Real-IP from a peer outside the list logs a one-time warning naming that peer (`warnOnUntrustedForwardedFor`, cmd/run.go), which is how a forgotten proxy hop shows up. Parsing lives in `Cfg.TrustedProxyList` (config.go), covered by config_test.go.TRUSTED_PROXY_HEADER= # optional header taken verbatim as the client IP on every request (gin TrustedPlatform: X-Real-IP behind ingress-nginx, CF-Connecting-IP behind Cloudflare); it wins over TRUSTED_PROXIES and turns the untrusted-forwarder warning off. Only safe when every request provably passes through a proxy that overwrites the header.diff --git a/backend/app/config/config.go b/backend/app/config/config.goindex ab460099..2734cb51 100644--- a/backend/app/config/config.go+++ b/backend/app/config/config.go@@ -1,6 +1,7 @@package configimport (+ "cmp""os""strconv""strings"@@ -181,7 +182,7 @@ func LoadFromEnv() *Cfg {return &Cfg{JWTSecret: os.Getenv("JWT_SECRET"),- DBType: os.Getenv("DB_TYPE"),+ DBType: cmp.Or(os.Getenv("DB_TYPE"), defaultDBType()),PostgresHost: os.Getenv("POSTGRES_HOST"),PostgresPort: os.Getenv("POSTGRES_PORT"),PostgresDatabase: os.Getenv("POSTGRES_DATABASE"),diff --git a/backend/app/config/dbtype_pg.go b/backend/app/config/dbtype_pg.gonew file mode 100644index 00000000..ff263f2a--- /dev/null+++ b/backend/app/config/dbtype_pg.go@@ -0,0 +1,11 @@+//go:build transactional_pg++package config++// defaultDBType is what DB_TYPE means when it is not set.+//+// In the transactional_pg build an empty value already selects PostgreSQL --+// db_transactional_pg.go connects unconditionally and runMainDBMigrations+// treats anything other than "sqlite" as PostgreSQL. Returning "" keeps that+// build byte-for-byte unchanged; only the SQLite builds gain a default.+func defaultDBType() string { return "" }diff --git a/backend/app/config/dbtype_sqlite.go b/backend/app/config/dbtype_sqlite.gonew file mode 100644index 00000000..e9460364--- /dev/null+++ b/backend/app/config/dbtype_sqlite.go@@ -0,0 +1,15 @@+//go:build !transactional_pg++package config++// defaultDBType is what DB_TYPE means when it is not set.+//+// Without the transactional_pg tag, SQLite is the only supported main+// database: runMainDBMigrations ignores the value entirely and applies+// SQLite-dialect migrations unconditionally, so any other value could only+// ever produce a broken database. Defaulting here rather than at each of the+// three `DBType != "sqlite"` call sites (db_transactional_sqlite.go,+// db_telemetry_sqlite.go, db_telemetry_duckdb.go) keeps one definition, and+// is what lets a fresh clone, an IDE run, CI, Docker and a Nix dev shell all+// start without a preconfigured environment.+func defaultDBType() string { return "sqlite" }diff --git a/backend/cmd/run.go b/backend/cmd/run.goindex a0f490aa..e490dea4 100644--- a/backend/cmd/run.go+++ b/backend/cmd/run.go@@ -84,7 +84,13 @@ func Run(opts ...Option) {config.Init(cfg)if err := services.InitJWT(); err != nil {- panic(fmt.Errorf("failed to initialize JWT: %w", err))+ // A missing signing key is a configuration mistake, not a crash: print+ // something actionable instead of burying the message under a panic+ // stack trace. There is deliberately no default -- a key committed to+ // the repo would be identical in every clone.+ config.Logln("FATAL: " + err.Error())+ config.Logln("Generate one with: openssl rand -hex 32")+ os.Exit(1)}err := db.Init()@@ -275,28 +281,50 @@ func Run(opts ...Option) {if ports == "" {ports = "80,8082"}- portsList := strings.Split(ports, ",")- if len(portsList) == 0 {- panic(fmt.Errorf("ports env variable is invalid - no ports found"))- }- if len(portsList) > 1 {- for i := 1; i < len(portsList); i++ {- if len(portsList[i]) == 0 {- continue- }- go func() {- defer traceway.Recover()+ // Bind every port up front rather than inside the serving goroutines, so a+ // failure is known before the process reports itself started. Individual+ // failures are tolerated -- binding :80 as a non-root user is the ordinary+ // local-development case -- but losing every port is fatal.+ //+ // Previously only the first entry was bound on this goroutine and panicked,+ // while the rest panicked inside goroutines guarded by traceway.Recover(),+ // which swallows the panic (and reports nothing at all when monitoring is+ // unconfigured). That made the same failure either fatal or invisible+ // depending on the order of PORTS.+ var listeners []net.Listener+ for _, port := range strings.Split(ports, ",") {+ port = strings.TrimSpace(port)+ if port == "" {+ continue+ }- port := ":" + portsList[i]- config.Logln("Starting server on " + port)- serveHTTP(router, port)- }()+ addr := ":" + port+ listener, err := net.Listen("tcp", addr)+ if err != nil {+ config.Logln("WARNING: not listening on " + addr + ": " + err.Error())+ traceway.CaptureException(fmt.Errorf("could not listen on %s: %w", addr, err))+ continue}+ listeners = append(listeners, listener)+ }++ if len(listeners) == 0 {+ panic(fmt.Errorf("could not listen on any port in PORTS=%q", ports))+ }++ for _, listener := range listeners[1:] {+ go func(listener net.Listener) {+ defer traceway.Recover()++ config.Logln("Starting server on " + listener.Addr().String())+ serveHTTP(router, listener)+ }(listener)}notifySystemd()- serveHTTP(router, ":"+portsList[0])+ config.Logln("Starting server on " + listeners[0].Addr().String())+ serveHTTP(router, listeners[0])}func warnOnUntrustedForwardedFor(proxies []string) gin.HandlerFunc {@@ -356,16 +384,20 @@ func containsIP(nets []*net.IPNet, ip net.IP) bool {const serverReadTimeout = time.Hour-func serveHTTP(router *gin.Engine, port string) {+// serveHTTP takes an already-bound listener rather than an address: the ports+// are bound up front so a bind failure is known before the process reports+// itself started, which ListenAndServe cannot express. The timeouts are the+// point of this helper and must survive that change -- gin's RunListener sets+// none of them.+func serveHTTP(router *gin.Engine, listener net.Listener) {srv := &http.Server{- Addr: port,Handler: router,ReadHeaderTimeout: 15 * time.Second,ReadTimeout: serverReadTimeout,IdleTimeout: 2 * time.Minute,}- if err := srv.ListenAndServe(); err != nil {- panic(fmt.Errorf("Error starting server on port %s: %v", port, err))+ if err := srv.Serve(listener); err != nil {+ panic(fmt.Errorf("server on %s stopped: %w", listener.Addr().String(), err))}}diff --git a/docs/pages/server/sqlite.mdx b/docs/pages/server/sqlite.mdxindex ec72c527..30706e0b 100644--- a/docs/pages/server/sqlite.mdx+++ b/docs/pages/server/sqlite.mdx@@ -75,7 +75,7 @@ If `STORAGE_TYPE` is unset (or set to `local`), blobs are written to `/data/stor| `S3_ACCESS_KEY` | *(unset)* | Optional. Falls back to the default AWS credential chain when unset || `S3_SECRET_KEY` | *(unset)* | Optional. Pair with `S3_ACCESS_KEY` || `S3_ENDPOINT` | *(unset)* | Optional. Set for S3-compatible providers (MinIO, R2, Spaces). Enables path-style URLs |-| `PORTS` | `80,8082` | Comma-separated list of HTTP ports the server should listen on |+| `PORTS` | `80,8082` | Comma-separated list of HTTP ports the server should listen on. Every port is bound before the server reports itself started; a port that cannot be bound (commonly `:80` when not running as root) logs a warning and is skipped, and startup fails only if none of the ports could be bound || `GIN_MODE` | `release` | Gin runtime mode || `SQLITE_RETENTION_DAYS` | `30` | Days to keep telemetry rows (endpoints, exceptions, spans, metrics, logs, sessions, recordings, fired notifications, notification history). A background worker prunes older rows once at startup and then every hour. Set to `0` to disable. || `LOG_RECORDS_MAX_ROWS` | *(unset)* | Optional cap on the number of log rows, independent of `SQLITE_RETENTION_DAYS`. This is not a hard limit: a cleanup task runs every minute and deletes the oldest logs until only the newest N rows remain. Between runs the table can grow past N, and if you ingest more than N rows in a single minute the table (and disk usage) will exceed the cap until the next cleanup. Size it with headroom for your peak ingest rate. Unset or `0` disables the cap. |