πΈ Rybbit - open-source and privacy-friendly alternative to Google Analytics that is 10x more intuitive.
Update proxy guide and IP resolution logic
Bill Yang4ac2321b3fe6
diff --git a/docs/content/docs/(docs)/proxy-guide/get-started.mdx b/docs/content/docs/(docs)/proxy-guide/get-started.mdxindex 604465f00..05fc08f67 100644--- a/docs/content/docs/(docs)/proxy-guide/get-started.mdx+++ b/docs/content/docs/(docs)/proxy-guide/get-started.mdx@@ -179,7 +179,7 @@ For accurate tracking, ensure these headers are forwarded to the Rybbit backend:### Forwarded IP Headers-If your proxy runs on infrastructure such as Cloudflare Workers, AWS CloudFront, Fastly, or another edge/CDN provider, make sure the proxy sets `X-Real-IP` to the original visitor IP. Rybbit also falls back to `X-Forwarded-For`, but `X-Real-IP` is preferred because Cloudflare-fronted requests may include intermediary IPs in `X-Forwarded-For`.+If your proxy runs on infrastructure such as Cloudflare Workers, AWS CloudFront, Fastly, or another edge/CDN provider, make sure the proxy sets `X-Forwarded-For` (or `X-Real-IP`) to the original visitor IP. Rybbit reads both **before** the connecting edge IP, so this prevents traffic from being geolocated to the proxy's exit node. `X-Real-IP` takes precedence if both are present, but `X-Forwarded-For` is fully supported and is the right choice on platforms that can't set `X-Real-IP` β notably AWS CloudFront, where `X-Real-IP` is on the [origin custom-header denylist](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/add-origin-custom-headers.html#add-origin-custom-headers-denylist).**Cloudflare Worker example:**@@ -193,9 +193,17 @@ proxyHeaders.set('X-Forwarded-For', clientIp);**CloudFront example:**-- Forward `X-Forwarded-For`, or set `X-Real-IP` to the viewer IP from a CloudFront Function/Lambda@Edge+- Set `X-Forwarded-For` to the viewer IP from a CloudFront Function/Lambda@Edge (CloudFront cannot forward `X-Real-IP`)- Forward `User-Agent`, `Referer`, and `Accept-Language`+```javascript+// CloudFront Function (viewer-request)+function handler(event) {+ event.request.headers["x-forwarded-for"] = { value: event.viewer.ip };+ return event.request;+}+```+## Bandwidth ConsiderationsProxying adds bandwidth usage to your infrastructure. Here's what to expect:diff --git a/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx b/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdxindex 369861e88..2d79a3fe1 100644--- a/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx+++ b/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx@@ -178,7 +178,7 @@ proxy_pass https://app.rybbit.io;**Solutions:**<Callout type="warn">-**Critical**: The `X-Real-IP` header should contain the original client IP for accurate geolocation and user identification. `X-Forwarded-For` is supported as a fallback, but may include intermediary edge IPs.+**Critical**: The `X-Forwarded-For` (or `X-Real-IP`) header must contain the original client IP for accurate geolocation and user identification. Rybbit reads both before the connecting edge IP. On AWS CloudFront use `X-Forwarded-For` β it cannot forward `X-Real-IP`.</Callout>**Nginx:**@@ -225,7 +225,7 @@ proxyHeaders.set('X-Real-IP', clientIp);**Cause:** Rybbit is seeing the proxy IP instead of the original visitor IP. This is common when a first-party proxy is hosted on CDN or cloud infrastructure. Those networks use hosting/datacenter ASNs, so traffic can look like bot infrastructure if the visitor IP is not passed in a trusted way.-Set `X-Real-IP` to the original visitor IP before forwarding the request to Rybbit. Also setting `X-Forwarded-For` is fine, but `X-Real-IP` is the preferred header for first-party proxies.+Set `X-Forwarded-For` (or `X-Real-IP`) to the original visitor IP before forwarding the request to Rybbit. Rybbit reads both before the connecting edge IP. `X-Real-IP` wins if both are present, but `X-Forwarded-For` is fully supported β use it on AWS CloudFront, which cannot forward `X-Real-IP`.**Cloudflare Workers:**@@ -239,9 +239,17 @@ proxyHeaders.set('X-Forwarded-For', clientIp);**AWS CloudFront:**-- Forward `X-Forwarded-For`, or set `X-Real-IP` to the viewer IP from a CloudFront Function/Lambda@Edge+- Set `X-Forwarded-For` to the viewer IP from a CloudFront Function/Lambda@Edge (CloudFront cannot forward `X-Real-IP`)- Forward `User-Agent`, `Referer`, and `Accept-Language`+```javascript+// CloudFront Function (viewer-request)+function handler(event) {+ event.request.headers["x-forwarded-for"] = { value: event.viewer.ip };+ return event.request;+}+```+---## Session Replay Issuesdiff --git a/server/src/api/featureFlags/index.ts b/server/src/api/featureFlags/index.tsindex a53f576da..f59da96b6 100644--- a/server/src/api/featureFlags/index.ts+++ b/server/src/api/featureFlags/index.ts@@ -10,7 +10,8 @@ import { featureFlags, userProfiles } from "../../db/postgres/schema.js";import { siteConfig } from "../../lib/siteConfig.js";import { getRequestUserAgent } from "../../services/tracker/requestIdentity.js";import { processResults } from "../analytics/utils/utils.js";-import { getDeviceType, getIpAddress } from "../../utils.js";+import { getDeviceType } from "../../utils.js";+import { resolveClientIp } from "../../services/tracker/resolveClientIp.js";import { evaluateFeatureFlagsForSite } from "../../services/featureFlags/evaluator.js";import {evaluateFeatureFlagsSchema,@@ -297,7 +298,7 @@ async function evaluateFeatureFlagsForRuntime(return reply.status(404).send({ error: "Site not found" });}- const ipAddress = getIpAddress(request);+ const ipAddress = resolveClientIp(request);const [locationByIp, profile] = await Promise.all([getLocation([ipAddress]).catch(() => ({}) as Awaited<ReturnType<typeof getLocation>>),body.identifiedUserIddiff --git a/server/src/api/sessionReplay/recordSessionReplay.ts b/server/src/api/sessionReplay/recordSessionReplay.tsindex 8fb6549c1..93b58a555 100644--- a/server/src/api/sessionReplay/recordSessionReplay.ts+++ b/server/src/api/sessionReplay/recordSessionReplay.ts@@ -4,7 +4,7 @@ import { siteConfig } from "../../lib/siteConfig.js";import { SessionReplayIngestService } from "../../services/replay/sessionReplayIngestService.js";import { usageService } from "../../services/usageService.js";import { RecordSessionReplayRequest } from "../../types/sessionReplay.js";-import { getIpAddress } from "../../utils.js";+import { resolveClientIp } from "../../services/tracker/resolveClientIp.js";import { logger } from "../../lib/logger/logger.js";import { getLocation } from "../../db/geolocation/geolocation.js";@@ -93,7 +93,7 @@ export async function recordSessionReplay(const body = recordSessionReplaySchema.parse(request.body) as RecordSessionReplayRequest;// Check if the IP should be excluded from tracking- const requestIP = getIpAddress(request);+ const requestIP = resolveClientIp(request);if (excludedIPs && excludedIPs.length > 0 && (await siteConfig.isIPExcluded(requestIP, request.params.siteId))) {logger.info(`[SessionReplay] IP ${requestIP} excluded from tracking for site ${siteId}`);@@ -161,7 +161,7 @@ export async function recordSessionReplay(}}- const ipAddress = getIpAddress(request);+ const ipAddress = resolveClientIp(request);const origin = request.headers.origin || "";const referrer = request.headers.referer || "";diff --git a/server/src/services/tracker/botBlocking/README.md b/server/src/services/tracker/botBlocking/README.mdindex b922857aa..028ca9030 100644--- a/server/src/services/tracker/botBlocking/README.md+++ b/server/src/services/tracker/botBlocking/README.md@@ -100,6 +100,6 @@ This catches obvious floods and fast crawlers, but it is local to a Node process## Trust Boundaries-Bot blocking assumes the resolved IP is meaningful. The tracker resolves IPs from `X-Real-IP`, then `CF-Connecting-IP`, then `X-Forwarded-For`, then the Fastify request IP. Public tracking requests ignore client-supplied `ip_address` and `user_agent`; those overrides are only honored for trusted server-side ingestion.+Bot blocking assumes the resolved IP is meaningful. The tracker resolves IPs from `X-Real-IP`, then `X-Forwarded-For`, then `CF-Connecting-IP`, then the Fastify request IP. The forwarded headers rank above `CF-Connecting-IP` so that first-party proxies (CloudFront, Fastly, nginx, ...) are attributed to the original visitor rather than the proxy's edge node. Public tracking requests ignore client-supplied `ip_address` and `user_agent`; those overrides are only honored for trusted server-side ingestion.Client-supplied `_bs` and `_bsm` are useful inputs but are not secure proof.diff --git a/server/src/services/tracker/identifyService.ts b/server/src/services/tracker/identifyService.tsindex 114556dee..4fa8225ac 100644--- a/server/src/services/tracker/identifyService.ts+++ b/server/src/services/tracker/identifyService.ts@@ -6,7 +6,7 @@ import { clickhouse } from "../../db/clickhouse/clickhouse.js";import { userProfiles, userAliases } from "../../db/postgres/schema.js";import { siteConfig } from "../../lib/siteConfig.js";import { userIdService } from "../userId/userIdService.js";-import { getIpAddress } from "../../utils.js";+import { resolveClientIp } from "./resolveClientIp.js";import { createServiceLogger } from "../../lib/logger/logger.js";const logger = createServiceLogger("identify-service");@@ -89,7 +89,7 @@ export async function handleIdentify(request: FastifyRequest, reply: FastifyReplconst anonymousId = anonymous_id? await userIdService.generateUserIdFromClientId(anonymous_id, siteId): await userIdService.generateUserId(- ip_address || getIpAddress(request),+ ip_address || resolveClientIp(request),user_agent || request.headers["user-agent"] || "",siteId);diff --git a/server/src/services/tracker/requestIdentity.ts b/server/src/services/tracker/requestIdentity.tsindex c43d9f5b3..82c0d54ba 100644--- a/server/src/services/tracker/requestIdentity.ts+++ b/server/src/services/tracker/requestIdentity.ts@@ -1,5 +1,5 @@import { FastifyRequest } from "fastify";-import { getIpAddress } from "../../utils.js";+import { resolveClientIp } from "./resolveClientIp.js";interface TrackingIdentityPayload {ip_address?: string;@@ -24,7 +24,7 @@ export function resolveTrackingIdentity(payload: TrackingIdentityPayload,trustedServerSideIngestion: boolean): TrackingIdentity {- const requestIpAddress = getIpAddress(request);+ const requestIpAddress = resolveClientIp(request);const requestUserAgent = getRequestUserAgent(request);return {diff --git a/server/src/services/tracker/resolveClientIp.test.ts b/server/src/services/tracker/resolveClientIp.test.tsnew file mode 100644index 000000000..e561bdbc2--- /dev/null+++ b/server/src/services/tracker/resolveClientIp.test.ts@@ -0,0 +1,62 @@+import { describe, it, expect } from "vitest";+import { FastifyRequest } from "fastify";+import { resolveClientIp } from "./resolveClientIp.js";++function requestWithHeaders(headers: Record<string, string | string[]>, ip = "198.51.100.10"): FastifyRequest {+ return { headers, ip } as unknown as FastifyRequest;+}++// Inject a deterministic edge predicate so tests don't depend on the MaxMind DB.+const proxied = () => true;+const direct = () => false;++describe("resolveClientIp", () => {+ it("trusts the forwarded visitor IP when the edge is a datacenter (proxied)", () => {+ // CloudFront case: CF-Connecting-IP is the proxy egress node; the real+ // visitor is in X-Forwarded-For.+ const request = requestWithHeaders({+ "cf-connecting-ip": "13.224.0.1", // CloudFront egress+ "x-forwarded-for": "203.0.113.10, 13.224.0.1", // visitor, then edge+ });++ expect(resolveClientIp(request, proxied)).toBe("203.0.113.10");+ });++ it("prefers X-Real-IP over X-Forwarded-For on the proxied path", () => {+ const request = requestWithHeaders({+ "cf-connecting-ip": "13.224.0.1",+ "x-forwarded-for": "203.0.113.10",+ "x-real-ip": "192.0.2.10",+ });++ expect(resolveClientIp(request, proxied)).toBe("192.0.2.10");+ });++ it("falls back to the edge IP if a proxy forwarded nothing", () => {+ const request = requestWithHeaders({ "cf-connecting-ip": "13.224.0.1" });+ expect(resolveClientIp(request, proxied)).toBe("13.224.0.1");+ });++ it("trusts the edge IP and ignores spoofable forwarded headers on the direct path", () => {+ // A direct visitor cannot spoof their geo: CF-Connecting-IP wins and the+ // attacker-supplied X-Forwarded-For / X-Real-IP are discarded.+ const request = requestWithHeaders({+ "cf-connecting-ip": "203.0.113.55", // residential visitor+ "x-forwarded-for": "8.8.8.8", // spoofed+ "x-real-ip": "1.1.1.1", // spoofed+ });++ expect(resolveClientIp(request, direct)).toBe("203.0.113.55");+ });++ it("uses forwarded headers when there is no Cloudflare edge (self-hosted)", () => {+ expect(resolveClientIp(requestWithHeaders({ "x-real-ip": "192.0.2.10" }), direct)).toBe("192.0.2.10");+ expect(resolveClientIp(requestWithHeaders({ "x-forwarded-for": "203.0.113.10, 10.0.0.1" }), direct)).toBe(+ "203.0.113.10"+ );+ });++ it("falls back to the socket IP when no usable headers are present", () => {+ expect(resolveClientIp(requestWithHeaders({}), direct)).toBe("198.51.100.10");+ });+});diff --git a/server/src/services/tracker/resolveClientIp.ts b/server/src/services/tracker/resolveClientIp.tsnew file mode 100644index 000000000..3851f9aa6--- /dev/null+++ b/server/src/services/tracker/resolveClientIp.ts@@ -0,0 +1,98 @@+import { FastifyRequest } from "fastify";+import { lookupAsn } from "../../db/geolocation/asn.js";+import { getIpAddress } from "../../utils.js";+import { isDatacenterAsn } from "./botBlocking/datacenterAsns.js";++/**+ * ASN-aware client IP resolution β the durable successor to `getIpAddress`.+ *+ * The problem `getIpAddress` can't solve: no static header precedence is correct+ * for both topologies at once.+ *+ * - Direct Cloudflare path β `CF-Connecting-IP` is the real visitor and is+ * non-spoofable; the forwarded headers are client-controllable noise.+ * - First-party proxy path β a customer fronts Rybbit with CloudFront / Fastly+ * / nginx, so `CF-Connecting-IP` is the *proxy's* egress node and the visitor+ * only survives in `X-Forwarded-For` / `X-Real-IP`.+ *+ * `getIpAddress` resolves this by always preferring the forwarded headers, which+ * fixes proxied customers but lets a direct visitor spoof their own geo.+ *+ * This variant removes that exposure by detecting the topology from the edge IP:+ * if `CF-Connecting-IP` belongs to a hosting/datacenter ASN, a first-party proxy+ * is in front, so we trust the forwarded visitor IP. Otherwise the edge IP is a+ * residential/business visitor and is authoritative β forwarded headers are+ * ignored, so they can't be spoofed.+ *+ * Wired as the default client-IP resolver for tracking ingestion, identify,+ * session replay, and feature-flag evaluation. When no Cloudflare edge is present+ * it delegates to `getIpAddress` (the simple header-precedence primitive).+ * Requires `GeoLite2-ASN.mmdb` (the same DB Layer-4 bot detection uses); see the+ * fallback note on `isProxiedEdge` below for the DB-missing behaviour.+ */++/** First non-empty `X-Forwarded-For` entry, or null. */+function firstForwardedFor(request: FastifyRequest): string | null {+ const forwardedFor = request.headers["x-forwarded-for"];+ if (forwardedFor && typeof forwardedFor === "string") {+ const ips = forwardedFor+ .split(",")+ .map(ip => ip.trim())+ .filter(Boolean);+ if (ips.length > 0) return ips[0];+ }+ return null;+}++function realIpHeader(request: FastifyRequest): string | null {+ const realIp = request.headers["x-real-ip"];+ return realIp && typeof realIp === "string" ? realIp.trim() : null;+}++function cfConnectingIp(request: FastifyRequest): string | null {+ const cfIp = request.headers["cf-connecting-ip"];+ return cfIp && typeof cfIp === "string" ? cfIp.trim() : null;+}++/**+ * Whether the connecting edge IP belongs to hosting/datacenter infrastructure β+ * positive evidence that a first-party proxy sits in front of our edge.+ *+ * Fallback semantics matter: if the ASN DB is unavailable, `lookupAsn` returns+ * null and this returns `false` β i.e. unknown edges are treated as direct and+ * `CF-Connecting-IP` wins. That is the safe default for the (vast) direct+ * majority, but it means proxied customers regress to seeing their proxy IP when+ * the DB is missing. If that trade-off is wrong for your deployment, default this+ * to `true` on a null lookup instead (favouring proxied correctness over+ * direct-path anti-spoofing).+ */+function isProxiedEdge(ip: string): boolean {+ const asn = lookupAsn(ip);+ return isDatacenterAsn(asn?.asn);+}++export function resolveClientIp(+ request: FastifyRequest,+ // Injectable for tests so branching can be exercised without the MaxMind DB.+ proxiedEdge: (ip: string) => boolean = isProxiedEdge+): string {+ const realIp = realIpHeader(request);+ const xffFirst = firstForwardedFor(request);+ const cfIp = cfConnectingIp(request);++ if (cfIp) {+ if (proxiedEdge(cfIp)) {+ // First-party proxy in front: trust the forwarded visitor IP, fall back to+ // the edge IP only if the proxy forwarded nothing.+ return realIp ?? xffFirst ?? cfIp;+ }+ // Direct visitor: the edge IP is authoritative and non-spoofable. Ignore+ // forwarded headers entirely.+ return cfIp;+ }++ // No Cloudflare edge (e.g. self-hosted behind nginx): there's no edge IP to+ // corroborate, so defer to plain header precedence (X-Real-IP, then+ // X-Forwarded-For, then the socket IP).+ return getIpAddress(request);+}diff --git a/server/src/utils.test.ts b/server/src/utils.test.tsindex ccbb8d2ac..ebadb1d73 100644--- a/server/src/utils.test.ts+++ b/server/src/utils.test.ts@@ -7,7 +7,7 @@ function requestWithHeaders(headers: Record<string, string | string[]>, ip = "19}describe("getIpAddress", () => {- it("uses X-Real-IP before Cloudflare's connecting IP", () => {+ it("uses X-Real-IP before all other headers", () => {const request = requestWithHeaders({"cf-connecting-ip": "198.51.100.20","x-forwarded-for": "203.0.113.10, 198.51.100.20",@@ -17,19 +17,20 @@ describe("getIpAddress", () => {expect(getIpAddress(request)).toBe("192.0.2.10");});- it("uses Cloudflare's connecting IP before X-Forwarded-For", () => {+ it("uses the first X-Forwarded-For IP before Cloudflare's connecting IP", () => {+ // The proxied case: a first-party proxy (e.g. CloudFront) forwards the visitor+ // in X-Forwarded-For, while CF-Connecting-IP is our Cloudflare edge seeing the+ // proxy's egress node. The visitor must win.const request = requestWithHeaders({- "cf-connecting-ip": "203.0.113.10",- "x-forwarded-for": "198.51.100.20, 10.0.0.1",+ "cf-connecting-ip": "198.51.100.20",+ "x-forwarded-for": "203.0.113.10, 198.51.100.20",});expect(getIpAddress(request)).toBe("203.0.113.10");});- it("uses the first X-Forwarded-For IP when higher-priority headers are missing", () => {- expect(getIpAddress(requestWithHeaders({ "x-forwarded-for": "203.0.113.10, 198.51.100.20" }))).toBe(- "203.0.113.10"- );+ it("falls back to Cloudflare's connecting IP when no forwarded headers are present", () => {+ expect(getIpAddress(requestWithHeaders({ "cf-connecting-ip": "203.0.113.10" }))).toBe("203.0.113.10");});it("falls back to the request IP", () => {diff --git a/server/src/utils.ts b/server/src/utils.tsindex d4b557a56..4946e68c0 100644--- a/server/src/utils.ts+++ b/server/src/utils.ts@@ -238,22 +238,35 @@ export const normalizeOrigin = (input: string): string => {};// Helper function to get IP address+//+// NOTE: production request paths resolve client IPs via `resolveClientIp()`+// (resolveClientIp.ts), which wraps this with ASN-based proxy detection. This+// function is the simple header-precedence primitive it falls back to when no+// Cloudflare edge is present, and remains the building block for that decision.+//+// Header precedence is tuned for first-party proxies (Cloudflare Workers, AWS+// CloudFront, Fastly, nginx, ...) that front Rybbit and forward the original+// visitor IP. The hard constraint: when such a proxy sits in front of our+// Cloudflare edge, `CF-Connecting-IP` is the *proxy's* egress IP, not the+// visitor β so it must rank BELOW the forwarded headers, otherwise proxied+// traffic geolocates to the proxy's exit node. (CloudFront, for one, cannot+// forward `X-Real-IP` at all β it's on AWS's origin custom-header denylist β+// so `X-Forwarded-For` is the only channel many proxies have.)+//+// For the non-proxied path these forwarded headers are client-spoofable, which+// only lets a visitor corrupt their own (anonymous) geo β acceptable. See+// resolveClientIp.ts for the ASN-aware variant that removes even that exposure.export const getIpAddress = (request: FastifyRequest): string => {- // Priority 1: X-Real-IP. First-party proxies can set this to the original- // visitor IP even when our Cloudflare edge sees the proxy's egress IP.+ // Priority 1: X-Real-IP. First-party proxies (e.g. nginx, Cloudflare Workers)+ // set this to the single original visitor IP.const realIp = request.headers["x-real-ip"];if (realIp && typeof realIp === "string") {return realIp.trim();}- // Priority 2: Cloudflare header. In our Cloudflare-fronted direct path this- // is authoritative; X-Forwarded-For may contain intermediary edge IPs.- const cfConnectingIp = request.headers["cf-connecting-ip"];- if (cfConnectingIp && typeof cfConnectingIp === "string") {- return cfConnectingIp.trim();- }-- // Priority 3: X-Forwarded-For - use the first IP, which should be the original client.+ // Priority 2: X-Forwarded-For - use the first IP, which should be the original+ // client. Ranks above CF-Connecting-IP so proxied traffic is attributed to the+ // visitor rather than the proxy's edge node.const forwardedFor = request.headers["x-forwarded-for"];if (forwardedFor && typeof forwardedFor === "string") {const ips = forwardedFor@@ -265,5 +278,12 @@ export const getIpAddress = (request: FastifyRequest): string => {}}+ // Priority 3: Cloudflare header. Authoritative on the direct Cloudflare-fronted+ // path; only reached when no forwarded header is present.+ const cfConnectingIp = request.headers["cf-connecting-ip"];+ if (cfConnectingIp && typeof cfConnectingIp === "string") {+ return cfConnectingIp.trim();+ }+return request.ip;};