High-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
fix: server-mode metrics count only successful requests, so requestsFailed is structurally always 0 (PIN-093)
Luigi Agostic13a1b3af8a2
diff --git a/docs/reference/health.md b/docs/reference/health.mdindex 04820ff6..184f8bc0 100644--- a/docs/reference/health.md+++ b/docs/reference/health.md@@ -2,6 +2,13 @@Check server status and availability.+`/health` is deliberately **not** the endpoint to compare across modes: in server+mode it is the orchestrator's own envelope, and an orchestrator has facts+(`instances`, `profiles`, `defaultInstance`) a bridge does not. Failure and crash+telemetry is served in the same shape by both modes on+[`/metrics`](./metrics.md) — use that when comparing a bridge repro against a+server-mode problem.+## Bridge Mode```bash@@ -15,7 +22,7 @@ pinchtab healthpinchtab health --json # Full JSON response```-Bridge-mode health may also include:+Bridge-mode health also reports `version`, and may include:- `crashLogs`- `failures`diff --git a/docs/reference/index.md b/docs/reference/index.mdindex bc374285..7812aab2 100644--- a/docs/reference/index.md+++ b/docs/reference/index.md@@ -19,6 +19,7 @@ Note: this index is generated and should be checked against the code when comman- [Hover](./hover.md)- [Instances](./instances.md)- [Keyboard](./keyboard.md)+- [Metrics](./metrics.md)- [Mouse](./mouse.md)- [Navigate](./navigate.md)- [PDF](./pdf.md)diff --git a/docs/reference/metrics.md b/docs/reference/metrics.mdnew file mode 100644index 00000000..1dcbe6e3--- /dev/null+++ b/docs/reference/metrics.md@@ -0,0 +1,80 @@+# Metrics++Runtime counters, recent failures and crash diagnostics.++```bash+curl -H "Authorization: Bearer $TOKEN" http://localhost:9867/metrics+```++## Two layers, never summed++Server mode runs **two processes**, and each keeps its own counters:++| Layer | Process | Sees |+|-------|---------|------|+| `frontDoor` | the orchestrator that owns the port | auth rejections, unrouted paths, everything before a request is forwarded |+| `instance` | a browser-control child (also what `pinchtab bridge` runs standalone) | every request that reached a browser |++Every response says which layer its numbers describe:++```json+{+ "layer": "frontDoor",+ "metrics": { "requestsTotal": 41, "requestsFailed": 7, "avgLatencyMs": 3.1, "...": "..." },+ "failures": {+ "layer": "frontDoor",+ "requestsFailed": 7,+ "recent": [+ { "time": "...", "requestId": "a1b2c3", "method": "GET", "path": "/tabs",+ "status": 401, "type": "http_error", "layer": "frontDoor" }+ ]+ },+ "crashes": { "total": 0, "recent": [] }+}+```++**The two layers are never added together.** A single `requestsTotal` covering both+would mean two things at once, and an operator cannot act on that number: a spike+would not say whether clients are being turned away at the door or the browser is+failing. Read each layer at its own endpoint instead:++| Endpoint | Answered by | Reports |+|----------|-------------|---------|+| `GET /metrics` | front door (server mode) / the bridge itself (bridge mode) | that process's own counters |+| `GET /instances/{id}/metrics` | proxied to that instance | that instance's counters |+| `GET /instances/metrics` | front door | per-instance **browser memory**, not request counters |+| `GET /tabs/{id}/metrics` | proxied to the owning instance | one tab's browser measurements |++`failures.recent` repeats `layer` on each event as well as on the block, so an+event pasted into a bug report still says where it came from.++`failures` and `crashes` are always present, empty when there is nothing to+report. A key that appeared only after the first failure could not be relied on by+anything watching for one.++## What the two modes share, and what they do not++Shared, by construction — both modes serve the same diagnostics payload:++- `metrics` (request counters, latency, Go runtime figures)+- `failures` with `requestsFailed` and `recent`+- `crashes` with `total` and `recent`+- `version` on `/health`++Deliberately different:++- **`/metrics` is answered locally in both modes; `/health` is not.** In server mode+ `/health` is the orchestrator's own envelope (`instances`, `profiles`,+ `defaultInstance`, `restartRequired`), because those facts exist only at the front+ door. A bridge has no instances to report. The two `/health` bodies therefore+ differ on purpose, and `/metrics` is the endpoint to compare across modes.+- **`memory` appears only on an instance's `/metrics`**, since only a process+ holding a browser can measure one.+- **Browser crash events are recorded by whichever process owns the browser**, so in+ server mode a crash shows up under the instance layer, not the front door — the+ front door's `crashes` block stays empty by design.++## Related Pages++- [Health](./health.md)+- [Instances](./instances.md)diff --git a/internal/handlers/health_tabs.go b/internal/handlers/health_tabs.goindex ac3f3f65..83c2b8c0 100644--- a/internal/handlers/health_tabs.go+++ b/internal/handlers/health_tabs.go@@ -42,12 +42,17 @@ func (h *Handlers) HandleHealth(w http.ResponseWriter, r *http.Request) {}resp := map[string]any{"status": "ok", "tabs": len(targets)}+ // Server-mode /health reports version; bridge /health did not, so a bridge+ // bug report could not state which build produced it.+ if h.Version != "" {+ resp["version"] = h.Version+ }if crashLogs := h.Bridge.GetCrashLogs(); len(crashLogs) > 0 {resp["crashLogs"] = crashLogs}if hasFailureDiagnostics() {- resp["failures"] = FailureSnapshot()+ resp["failures"] = FailureSnapshot(LayerInstance)}if bridge.HasCrashDiagnostics() {resp["crashes"] = bridge.CrashSnapshot()@@ -89,14 +94,11 @@ func (h *Handlers) HandleBrowserRestart(w http.ResponseWriter, r *http.Request)httpx.JSON(w, 200, map[string]string{"status": "browser_restarted"})}+// HandleMetrics reports this process's own counters. In server mode that is an+// instance child: the orchestrator front door answers its own /metrics, so a+// client can read either layer without the two ever being summed.func (h *Handlers) HandleMetrics(w http.ResponseWriter, r *http.Request) {- result := map[string]any{"metrics": SnapshotMetrics()}- if hasFailureDiagnostics() {- result["failures"] = FailureSnapshot()- }- if bridge.HasCrashDiagnostics() {- result["crashes"] = bridge.CrashSnapshot()- }+ result := DiagnosticsSnapshot(LayerInstance)if h.Bridge != nil {if mem, err := h.Bridge.GetAggregatedMemoryMetrics(); err == nil && mem != nil {diff --git a/internal/handlers/middleware_test.go b/internal/handlers/middleware_test.goindex 977f307d..232914ae 100644--- a/internal/handlers/middleware_test.go+++ b/internal/handlers/middleware_test.go@@ -856,18 +856,18 @@ func TestLoggingMiddleware_RecordsFailure(t *testing.T) {w := httptest.NewRecorder()handler.ServeHTTP(w, req)- snap := FailureSnapshot()+ snap := FailureSnapshot(LayerInstance)if got := snap["requestsFailed"].(uint64); got != 1 {t.Fatalf("requestsFailed = %d, want 1", got)}- recent, ok := snap["recent"].([]FailureEvent)+ recent, ok := snap["recent"].([]map[string]any)if !ok || len(recent) != 1 {t.Fatalf("recent failures = %#v, want 1 event", snap["recent"])}- if recent[0].Path != "/boom" {- t.Fatalf("recent path = %q, want /boom", recent[0].Path)+ if recent[0]["path"] != "/boom" {+ t.Fatalf("recent path = %v, want /boom", recent[0]["path"])}- if recent[0].RequestID == "" {+ if recent[0]["requestId"] == "" {t.Fatal("expected request id on failure event")}}diff --git a/internal/handlers/observability.go b/internal/handlers/observability.goindex caa15293..23cac0af 100644--- a/internal/handlers/observability.go+++ b/internal/handlers/observability.go@@ -4,6 +4,8 @@ import ("sync""sync/atomic""time"++ "github.com/pinchtab/pinchtab/internal/bridge")const maxRecentFailures = 20@@ -33,18 +35,60 @@ func recordFailureEvent(ev FailureEvent) {}}-// FailureSnapshot returns recent failure diagnostics for /health and /metrics.-func FailureSnapshot() map[string]any {+// Layer names the process a set of counters describes. Server mode runs TWO+// counter sets in two processes — the orchestrator front door, which is the only+// one that sees auth rejections and unrouted paths, and each browser-control+// instance — so every number a client reads has to say which one it came from.+// They are never summed: one requestsTotal meaning two layers is a number an+// operator cannot act on.+const (+ LayerFrontDoor = "frontDoor"+ LayerInstance = "instance"+)++// FailureSnapshot returns recent failure diagnostics for /health and /metrics,+// labelled with the layer that recorded them. The label is repeated on each event+// as well as on the block, so an event stays self-describing when it is copied+// out of the response it arrived in.+func FailureSnapshot(layer string) map[string]any {failureMu.Lock()recent := append([]FailureEvent(nil), recentFailures...)failureMu.Unlock()+ events := make([]map[string]any, 0, len(recent))+ for _, ev := range recent {+ events = append(events, map[string]any{+ "time": ev.Time,+ "requestId": ev.RequestID,+ "method": ev.Method,+ "path": ev.Path,+ "status": ev.Status,+ "type": ev.Type,+ "layer": layer,+ })+ }+return map[string]any{+ "layer": layer,"requestsFailed": atomic.LoadUint64(&metricRequestsFailed),- "recent": recent,+ "recent": events,}}+// DiagnosticsSnapshot is the whole observability payload for one process: its own+// counters plus the failure and crash logs it recorded. Both modes serve it, so+// the two cannot drift into exposing different keys — the divergence this closes+// was exactly that.+func DiagnosticsSnapshot(layer string) map[string]any {+ snapshot := map[string]any{+ "layer": layer,+ "metrics": SnapshotMetrics(),+ }+ snapshot["failures"] = FailureSnapshot(layer)+ snapshot["crashes"] = bridge.CrashSnapshot()+ return snapshot+}+func hasFailureDiagnostics() bool {failureMu.Lock()recentCount := len(recentFailures)@@ -52,6 +96,13 @@ func hasFailureDiagnostics() bool {return atomic.LoadUint64(&metricRequestsFailed) > 0 || recentCount > 0}+// ResetObservabilityForTests clears the process counters and failure log. It is+// exported because the front-door chain lives in internal/server, and the+// counters it must be proven to move are owned here.+func ResetObservabilityForTests() {+ resetObservabilityForTests()+}+func resetObservabilityForTests() {atomic.StoreUint64(&metricRequestsTotal, 0)atomic.StoreUint64(&metricRequestsFailed, 0)diff --git a/internal/handlers/observability_layers_test.go b/internal/handlers/observability_layers_test.gonew file mode 100644index 00000000..8d9d38f3--- /dev/null+++ b/internal/handlers/observability_layers_test.go@@ -0,0 +1,102 @@+package handlers++import (+ "encoding/json"+ "net/http"+ "net/http/httptest"+ "testing"++ "github.com/pinchtab/pinchtab/internal/config"+)++// Server mode runs two counter sets in two processes. Moving the bare /metrics to+// the front door must not make the other layer invisible in turn: an instance+// still reports its OWN traffic, and says so.+func TestInstanceMetricsStillReportItsOwnTrafficAndLayer(t *testing.T) {+ resetObservabilityForTests()+ t.Cleanup(resetObservabilityForTests)++ counted := LoggingMiddleware(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {+ w.WriteHeader(http.StatusInternalServerError)+ }))+ for i := 0; i < 3; i++ {+ counted.ServeHTTP(httptest.NewRecorder(), httptest.NewRequest("GET", "/boom", nil))+ }++ h := New(&mockBridge{}, &config.RuntimeConfig{}, nil, nil, nil)+ rec := httptest.NewRecorder()+ h.HandleMetrics(rec, httptest.NewRequest("GET", "/metrics", nil))+ if rec.Code != http.StatusOK {+ t.Fatalf("GET /metrics = %d: %s", rec.Code, rec.Body.String())+ }++ var body map[string]any+ if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {+ t.Fatalf("decode metrics: %v", err)+ }+ if body["layer"] != LayerInstance {+ t.Errorf("layer = %v, want %q", body["layer"], LayerInstance)+ }+ metrics, ok := body["metrics"].(map[string]any)+ if !ok {+ t.Fatalf("metrics block missing: %v", body)+ }+ if got := metrics["requestsTotal"].(float64); got != 3 {+ t.Errorf("requestsTotal = %v, want the 3 requests this process served", got)+ }+ failures, ok := body["failures"].(map[string]any)+ if !ok {+ t.Fatalf("failures block missing: %v", body)+ }+ if got := failures["requestsFailed"].(float64); got != 3 {+ t.Errorf("requestsFailed = %v, want 3", got)+ }+ if failures["layer"] != LayerInstance {+ t.Errorf("failures.layer = %v, want %q", failures["layer"], LayerInstance)+ }+}++// Both modes now serve the same diagnostic keys, which is what makes bridge mode+// usable for reproducing a server-mode problem. A shape that differs per mode is+// the drift this card is closing.+func TestBothModesExposeTheSameDiagnosticKeys(t *testing.T) {+ resetObservabilityForTests()+ t.Cleanup(resetObservabilityForTests)++ frontDoor := DiagnosticsSnapshot(LayerFrontDoor)+ instance := DiagnosticsSnapshot(LayerInstance)++ for key := range frontDoor {+ if _, ok := instance[key]; !ok {+ t.Errorf("instance diagnostics missing %q, which the front door reports", key)+ }+ }+ for key := range instance {+ if _, ok := frontDoor[key]; !ok {+ t.Errorf("front-door diagnostics missing %q, which an instance reports", key)+ }+ }+ if frontDoor["layer"] == instance["layer"] {+ t.Error("both layers report the same label; a client cannot tell the two counter sets apart")+ }+}++// A bridge bug report has to be able to state which build produced it.+func TestBridgeHealthReportsVersion(t *testing.T) {+ h := New(&mockBridge{}, &config.RuntimeConfig{}, nil, nil, nil)+ h.Version = "9.9.9-test"++ rec := httptest.NewRecorder()+ h.HandleHealth(rec, httptest.NewRequest("GET", "/health", nil))+ if rec.Code != http.StatusOK {+ t.Fatalf("GET /health = %d: %s", rec.Code, rec.Body.String())+ }++ var body map[string]any+ if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {+ t.Fatalf("decode health: %v", err)+ }+ if body["version"] != "9.9.9-test" {+ t.Errorf("version = %v, want the build the bridge is running", body["version"])+ }+}diff --git a/internal/orchestrator/handlers.go b/internal/orchestrator/handlers.goindex 412f5d8a..d67e0696 100644--- a/internal/orchestrator/handlers.go+++ b/internal/orchestrator/handlers.go@@ -62,6 +62,10 @@ func (o *Orchestrator) registerHandlers(mux *http.ServeMux, skipLaunch bool) {mux.HandleFunc("GET /instances/{id}", o.handleGetInstance)mux.HandleFunc("GET /instances/tabs", o.handleAllTabs)mux.HandleFunc("GET /instances/metrics", o.handleAllMetrics)+ // The front door answers the bare /metrics with its OWN counters, so an+ // instance's request counters need a path of their own; without it, moving+ // /metrics to the front door would just make the other layer invisible.+ mux.HandleFunc("GET /instances/{id}/metrics", o.proxyToInstance)if !skipLaunch {mux.HandleFunc("POST /instances/start", o.handleStartInstance)mux.HandleFunc("POST /instances/launch", o.handleLaunchByName)diff --git a/internal/routes/routes.go b/internal/routes/routes.goindex cbb5a4ef..b3f79710 100644--- a/internal/routes/routes.go+++ b/internal/routes/routes.go@@ -61,6 +61,18 @@ type Endpoint struct {TabScoped bool // true = auto-generates /tabs/{id}/... variant}+// frontDoorLocalPaths are the shorthand paths the orchestrator front door answers+// ITSELF instead of proxying to an instance. Proxying a diagnostic endpoint hides+// the front door's own numbers: it is the only layer that sees auth rejections and+// unrouted paths, so forwarding the read leaves no way for a client to see them.+// Declared here rather than as an Endpoint field because the catalogue below is+// written positionally, and a new field would rewrite every unrelated entry.+// Tab-scoped variants are unaffected: /tabs/{id}/metrics is a browser measurement+// and still belongs to the instance.+var frontDoorLocalPaths = map[string]bool{+ "/metrics": true,+}+// Route returns the "METHOD /path" string used for mux registration.func (e Endpoint) Route() string {return e.Method + " " + e.Path@@ -198,12 +210,27 @@ func Core() []Endpoint {return out}-// ShorthandRoutes returns all non-capability-gated shorthand routes-// as "METHOD /path" strings, suitable for mux registration.+// ShorthandRoutes returns the non-capability-gated shorthand routes the front+// door PROXIES, as "METHOD /path" strings suitable for mux registration.+// FrontDoorLocal endpoints are excluded: the front door registers its own+// handler for those, and registering both would panic on a duplicate pattern.func ShorthandRoutes() []string {var routes []stringfor _, ep := range coreEndpoints {- if ep.Capability == CapNone {+ if ep.Capability == CapNone && !frontDoorLocalPaths[ep.Path] {+ routes = append(routes, ep.Route())+ }+ }+ return routes+}++// FrontDoorLocalRoutes returns the routes the front door must register itself.+// It is the other half of ShorthandRoutes: together they cover every CapNone+// endpoint exactly once, which the routes test pins.+func FrontDoorLocalRoutes() []string {+ var routes []string+ for _, ep := range coreEndpoints {+ if ep.Capability == CapNone && frontDoorLocalPaths[ep.Path] {routes = append(routes, ep.Route())}}diff --git a/internal/server/front_door_metrics.go b/internal/server/front_door_metrics.gonew file mode 100644index 00000000..c1c9f402--- /dev/null+++ b/internal/server/front_door_metrics.go@@ -0,0 +1,52 @@+package server++import (+ "net/http"++ "github.com/pinchtab/pinchtab/internal/activity"+ "github.com/pinchtab/pinchtab/internal/browsersession"+ "github.com/pinchtab/pinchtab/internal/config"+ "github.com/pinchtab/pinchtab/internal/handlers"+ "github.com/pinchtab/pinchtab/internal/httpx"+ "github.com/pinchtab/pinchtab/internal/routes"+ "github.com/pinchtab/pinchtab/internal/session"+)++// FrontDoorHandler is the orchestrator front door's whole middleware chain. It is+// a named function rather than an expression inside RunDashboard because the+// counters live in the middleware and the rejections that were invisible happen+// ABOVE the mux: proving they are counted needs the real chain, and the isolated+// middleware already behaved correctly, which is why nothing caught this.+func FrontDoorHandler(+ cfg *config.RuntimeConfig,+ recorder activity.Recorder,+ sessions *browsersession.Manager,+ sessionStore *session.Store,+ mux http.Handler,+) http.Handler {+ return handlers.StripInternalHeadersMiddleware(+ handlers.RequestIDMiddleware(+ activity.Middleware(+ recorder,+ "server",+ handlers.SecurityHeadersMiddleware(cfg,+ handlers.LoggingMiddleware(handlers.RateLimitMiddleware(handlers.CorsMiddleware(cfg, handlers.AuthMiddlewareWithSessions(cfg, sessions, sessionStore, mux)))),+ ),+ ),+ ),+ )+}++// registerFrontDoorMetrics serves the front door's OWN counters at /metrics.+// It used to be a proxied route, so a client read the instance's numbers and the+// front-door counters — the only ones that see auth rejections and unrouted paths+// — were unreachable by any client. The instance figures did not become+// unreachable in exchange: an instance still answers its own /metrics, reachable+// through GET /instances/{id}/metrics.+func registerFrontDoorMetrics(mux *http.ServeMux) {+ for _, route := range routes.FrontDoorLocalRoutes() {+ mux.HandleFunc(route, func(w http.ResponseWriter, r *http.Request) {+ httpx.JSON(w, http.StatusOK, handlers.DiagnosticsSnapshot(handlers.LayerFrontDoor))+ })+ }+}diff --git a/internal/server/front_door_metrics_test.go b/internal/server/front_door_metrics_test.gonew file mode 100644index 00000000..03d877af--- /dev/null+++ b/internal/server/front_door_metrics_test.go@@ -0,0 +1,186 @@+package server++import (+ "encoding/json"+ "fmt"+ "net/http"+ "net/http/httptest"+ "testing"++ "github.com/pinchtab/pinchtab/internal/config"+ "github.com/pinchtab/pinchtab/internal/handlers"+ "github.com/pinchtab/pinchtab/internal/routes"+)++const frontDoorToken = "tok-front-door-1234567890"++// frontDoorFixture mounts the REAL front-door chain over a mux carrying the real+// local /metrics route. The rejections this card is about happen ABOVE the mux,+// in the auth middleware, so a test that exercised LoggingMiddleware on its own+// would pass while the product stayed blind — which is exactly what happened.+func frontDoorFixture(t *testing.T) http.Handler {+ t.Helper()+ handlers.ResetObservabilityForTests()+ t.Cleanup(handlers.ResetObservabilityForTests)++ cfg := &config.RuntimeConfig{Token: frontDoorToken, Bind: "127.0.0.1", Port: "9867"}+ mux := http.NewServeMux()+ registerFrontDoorMetrics(mux)+ mux.HandleFunc("GET /tabs", func(w http.ResponseWriter, r *http.Request) {+ _, _ = w.Write([]byte(`{"tabs":[]}`))+ })+ return FrontDoorHandler(cfg, nil, nil, nil, mux)+}++func readFrontDoorMetrics(t *testing.T, handler http.Handler) map[string]any {+ t.Helper()+ req := httptest.NewRequest("GET", "/metrics", nil)+ req.Header.Set("Authorization", "Bearer "+frontDoorToken)+ rec := httptest.NewRecorder()+ handler.ServeHTTP(rec, req)+ if rec.Code != http.StatusOK {+ t.Fatalf("GET /metrics = %d: %s", rec.Code, rec.Body.String())+ }+ var body map[string]any+ if err := json.Unmarshal(rec.Body.Bytes(), &body); err != nil {+ t.Fatalf("decode metrics: %v (body %s)", err, rec.Body.String())+ }+ return body+}++// The reported defect: N bad-token requests left requestsFailed at 0 forever,+// because /metrics was proxied to the instance and the front-door counters — the+// only ones that see an auth rejection — were unreachable by any client.+//+// The assertion is EXACT, not >=: an inequality passes on double counting, which+// is the specific risk when two layers of counters exist.+func TestFrontDoorMetricsCountRejectedRequestsExactly(t *testing.T) {+ handler := frontDoorFixture(t)++ const rejected = 5+ for i := 0; i < rejected; i++ {+ req := httptest.NewRequest("GET", "/tabs", nil)+ req.Header.Set("Authorization", "Bearer WRONG")+ rec := httptest.NewRecorder()+ handler.ServeHTTP(rec, req)+ if rec.Code != http.StatusUnauthorized {+ t.Fatalf("request %d = %d, want 401: %s", i, rec.Code, rec.Body.String())+ }+ }+ // An unrouted path is the other case that never reaches an instance.+ for i := 0; i < 2; i++ {+ req := httptest.NewRequest("GET", fmt.Sprintf("/zzz%d", i), nil)+ req.Header.Set("Authorization", "Bearer "+frontDoorToken)+ rec := httptest.NewRecorder()+ handler.ServeHTTP(rec, req)+ if rec.Code != http.StatusNotFound {+ t.Fatalf("unrouted request %d = %d, want 404", i, rec.Code)+ }+ }++ body := readFrontDoorMetrics(t, handler)+ if body["layer"] != handlers.LayerFrontDoor {+ t.Fatalf("layer = %v, want %q — a number that does not say which process it describes is not actionable", body["layer"], handlers.LayerFrontDoor)+ }++ failures, ok := body["failures"].(map[string]any)+ if !ok {+ t.Fatalf("no failures block in front-door metrics: %v", body)+ }+ if got := failures["requestsFailed"].(float64); got != rejected+2 {+ t.Errorf("requestsFailed = %v, want exactly %d", got, rejected+2)+ }+ recent, ok := failures["recent"].([]any)+ if !ok {+ t.Fatalf("failures.recent missing: %v", failures)+ }+ if len(recent) != rejected+2 {+ t.Fatalf("failures.recent has %d events, want exactly %d", len(recent), rejected+2)+ }++ // Every event must carry the four fields an operator needs to act, plus the+ // layer that produced it.+ unauthorized := 0+ for i, raw := range recent {+ event, ok := raw.(map[string]any)+ if !ok {+ t.Fatalf("event %d is not an object: %v", i, raw)+ }+ for _, key := range []string{"method", "path", "status", "requestId", "layer"} {+ if event[key] == nil || event[key] == "" {+ t.Errorf("event %d missing %q: %v", i, key, event)+ }+ }+ if event["layer"] != handlers.LayerFrontDoor {+ t.Errorf("event %d layer = %v, want %q", i, event["layer"], handlers.LayerFrontDoor)+ }+ if event["status"].(float64) == http.StatusUnauthorized {+ unauthorized+++ if event["path"] != "/tabs" {+ t.Errorf("401 event %d path = %v, want /tabs", i, event["path"])+ }+ if event["method"] != "GET" {+ t.Errorf("401 event %d method = %v, want GET", i, event["method"])+ }+ }+ }+ if unauthorized != rejected {+ t.Errorf("recorded %d auth rejections, want %d", unauthorized, rejected)+ }+}++// crashes.recent must be present in server mode too, and with a stable shape: a+// key that only appears after the first crash cannot be relied on by a client+// watching for one.+func TestFrontDoorMetricsAlwaysCarryFailureAndCrashBlocks(t *testing.T) {+ body := readFrontDoorMetrics(t, frontDoorFixture(t))++ for _, key := range []string{"metrics", "failures", "crashes", "layer"} {+ if _, ok := body[key]; !ok {+ t.Errorf("front-door metrics missing %q on a clean server: %v", key, body)+ }+ }+ crashes, ok := body["crashes"].(map[string]any)+ if !ok {+ t.Fatalf("crashes block is not an object: %v", body["crashes"])+ }+ if _, ok := crashes["recent"]; !ok {+ t.Errorf("crashes.recent missing: %v", crashes)+ }+ failures := body["failures"].(map[string]any)+ if got := failures["requestsFailed"].(float64); got != 0 {+ t.Errorf("a clean front door reports requestsFailed = %v, want 0", got)+ }+}++// The front door can only answer /metrics itself if the proxy registration stops+// claiming it — registering both patterns panics. This pins the split rather than+// the spelling: every CapNone endpoint belongs to exactly one of the two sets.+func TestMetricsIsFrontDoorLocalAndNotProxied(t *testing.T) {+ local := routes.FrontDoorLocalRoutes()+ proxied := routes.ShorthandRoutes()++ if len(local) == 0 {+ t.Fatal("no front-door-local routes; the front door would proxy its own diagnostics again")+ }+ inLocal := map[string]bool{}+ for _, route := range local {+ inLocal[route] = true+ }+ for _, route := range proxied {+ if inLocal[route] {+ t.Errorf("%q is registered both locally and as a proxy route; mux registration would panic", route)+ }+ }+ if !inLocal["GET /metrics"] {+ t.Errorf("GET /metrics must be answered by the front door: local routes = %v", local)+ }++ // The tab-scoped variant is a browser measurement and must STILL proxy.+ for _, route := range routes.TabScopedRoutes() {+ if route == "GET /tabs/{id}/metrics" {+ return+ }+ }+ t.Error("GET /tabs/{id}/metrics is no longer proxied; per-tab metrics belong to the instance")+}diff --git a/internal/server/server.go b/internal/server/server.goindex 38a974b3..ab517d76 100644--- a/internal/server/server.go+++ b/internal/server/server.go@@ -300,6 +300,7 @@ func RunDashboard(cfg *config.RuntimeConfig, version string) {}mux.HandleFunc("GET /health", configAPI.HandleHealth)+ registerFrontDoorMetrics(mux)mux.HandleFunc("GET /health/background", func(w http.ResponseWriter, r *http.Request) {httpx.JSON(w, http.StatusOK, map[string]string{"status": "ok",@@ -309,17 +310,7 @@ func RunDashboard(cfg *config.RuntimeConfig, version string) {})})- handler := handlers.StripInternalHeadersMiddleware(- handlers.RequestIDMiddleware(- activity.Middleware(- liveActivity,- "server",- handlers.SecurityHeadersMiddleware(cfg,- handlers.LoggingMiddleware(handlers.RateLimitMiddleware(handlers.CorsMiddleware(cfg, handlers.AuthMiddlewareWithSessions(cfg, sessions, sessionStore, mux)))),- ),- ),- ),- )+ handler := FrontDoorHandler(cfg, liveActivity, sessions, sessionStore, mux)if cfg.VerboseBanner {cli.LogSecurityWarnings(cfg)}