Skip to content

🐸 Rybbit - open-source and privacy-friendly alternative to Google Analytics that is 10x more intuitive.

Update proxy guide and IP resolution logic

Bill Yang4ac2321b3fe6
diff --git a/docs/content/docs/(docs)/proxy-guide/get-started.mdx b/docs/content/docs/(docs)/proxy-guide/get-started.mdx
index 604465f00..05fc08f67 100644
--- a/docs/content/docs/(docs)/proxy-guide/get-started.mdx
+++ b/docs/content/docs/(docs)/proxy-guide/get-started.mdx
@@ -179,7 +179,7 @@ For accurate tracking, ensure these headers are forwarded to the Rybbit backend:
### Forwarded IP Headers
-If your proxy runs on infrastructure such as Cloudflare Workers, AWS CloudFront, Fastly, or another edge/CDN provider, make sure the proxy sets `X-Real-IP` to the original visitor IP. Rybbit also falls back to `X-Forwarded-For`, but `X-Real-IP` is preferred because Cloudflare-fronted requests may include intermediary IPs in `X-Forwarded-For`.
+If your proxy runs on infrastructure such as Cloudflare Workers, AWS CloudFront, Fastly, or another edge/CDN provider, make sure the proxy sets `X-Forwarded-For` (or `X-Real-IP`) to the original visitor IP. Rybbit reads both **before** the connecting edge IP, so this prevents traffic from being geolocated to the proxy's exit node. `X-Real-IP` takes precedence if both are present, but `X-Forwarded-For` is fully supported and is the right choice on platforms that can't set `X-Real-IP` β€” notably AWS CloudFront, where `X-Real-IP` is on the [origin custom-header denylist](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/add-origin-custom-headers.html#add-origin-custom-headers-denylist).
**Cloudflare Worker example:**
@@ -193,9 +193,17 @@ proxyHeaders.set('X-Forwarded-For', clientIp);
**CloudFront example:**
-- Forward `X-Forwarded-For`, or set `X-Real-IP` to the viewer IP from a CloudFront Function/Lambda@Edge
+- Set `X-Forwarded-For` to the viewer IP from a CloudFront Function/Lambda@Edge (CloudFront cannot forward `X-Real-IP`)
- Forward `User-Agent`, `Referer`, and `Accept-Language`
+```javascript
+// CloudFront Function (viewer-request)
+function handler(event) {
+ event.request.headers["x-forwarded-for"] = { value: event.viewer.ip };
+ return event.request;
+}
+```
+
## Bandwidth Considerations
Proxying adds bandwidth usage to your infrastructure. Here's what to expect:
diff --git a/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx b/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx
index 369861e88..2d79a3fe1 100644
--- a/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx
+++ b/docs/content/docs/(docs)/proxy-guide/troubleshooting.mdx
@@ -178,7 +178,7 @@ proxy_pass https://app.rybbit.io;
**Solutions:**
<Callout type="warn">
-**Critical**: The `X-Real-IP` header should contain the original client IP for accurate geolocation and user identification. `X-Forwarded-For` is supported as a fallback, but may include intermediary edge IPs.
+**Critical**: The `X-Forwarded-For` (or `X-Real-IP`) header must contain the original client IP for accurate geolocation and user identification. Rybbit reads both before the connecting edge IP. On AWS CloudFront use `X-Forwarded-For` β€” it cannot forward `X-Real-IP`.
</Callout>
**Nginx:**
@@ -225,7 +225,7 @@ proxyHeaders.set('X-Real-IP', clientIp);
**Cause:** Rybbit is seeing the proxy IP instead of the original visitor IP. This is common when a first-party proxy is hosted on CDN or cloud infrastructure. Those networks use hosting/datacenter ASNs, so traffic can look like bot infrastructure if the visitor IP is not passed in a trusted way.
-Set `X-Real-IP` to the original visitor IP before forwarding the request to Rybbit. Also setting `X-Forwarded-For` is fine, but `X-Real-IP` is the preferred header for first-party proxies.
+Set `X-Forwarded-For` (or `X-Real-IP`) to the original visitor IP before forwarding the request to Rybbit. Rybbit reads both before the connecting edge IP. `X-Real-IP` wins if both are present, but `X-Forwarded-For` is fully supported β€” use it on AWS CloudFront, which cannot forward `X-Real-IP`.
**Cloudflare Workers:**
@@ -239,9 +239,17 @@ proxyHeaders.set('X-Forwarded-For', clientIp);
**AWS CloudFront:**
-- Forward `X-Forwarded-For`, or set `X-Real-IP` to the viewer IP from a CloudFront Function/Lambda@Edge
+- Set `X-Forwarded-For` to the viewer IP from a CloudFront Function/Lambda@Edge (CloudFront cannot forward `X-Real-IP`)
- Forward `User-Agent`, `Referer`, and `Accept-Language`
+```javascript
+// CloudFront Function (viewer-request)
+function handler(event) {
+ event.request.headers["x-forwarded-for"] = { value: event.viewer.ip };
+ return event.request;
+}
+```
+
---
## Session Replay Issues
diff --git a/server/src/api/featureFlags/index.ts b/server/src/api/featureFlags/index.ts
index a53f576da..f59da96b6 100644
--- a/server/src/api/featureFlags/index.ts
+++ b/server/src/api/featureFlags/index.ts
@@ -10,7 +10,8 @@ import { featureFlags, userProfiles } from "../../db/postgres/schema.js";
import { siteConfig } from "../../lib/siteConfig.js";
import { getRequestUserAgent } from "../../services/tracker/requestIdentity.js";
import { processResults } from "../analytics/utils/utils.js";
-import { getDeviceType, getIpAddress } from "../../utils.js";
+import { getDeviceType } from "../../utils.js";
+import { resolveClientIp } from "../../services/tracker/resolveClientIp.js";
import { evaluateFeatureFlagsForSite } from "../../services/featureFlags/evaluator.js";
import {
evaluateFeatureFlagsSchema,
@@ -297,7 +298,7 @@ async function evaluateFeatureFlagsForRuntime(
return reply.status(404).send({ error: "Site not found" });
}
- const ipAddress = getIpAddress(request);
+ const ipAddress = resolveClientIp(request);
const [locationByIp, profile] = await Promise.all([
getLocation([ipAddress]).catch(() => ({}) as Awaited<ReturnType<typeof getLocation>>),
body.identifiedUserId
diff --git a/server/src/api/sessionReplay/recordSessionReplay.ts b/server/src/api/sessionReplay/recordSessionReplay.ts
index 8fb6549c1..93b58a555 100644
--- a/server/src/api/sessionReplay/recordSessionReplay.ts
+++ b/server/src/api/sessionReplay/recordSessionReplay.ts
@@ -4,7 +4,7 @@ import { siteConfig } from "../../lib/siteConfig.js";
import { SessionReplayIngestService } from "../../services/replay/sessionReplayIngestService.js";
import { usageService } from "../../services/usageService.js";
import { RecordSessionReplayRequest } from "../../types/sessionReplay.js";
-import { getIpAddress } from "../../utils.js";
+import { resolveClientIp } from "../../services/tracker/resolveClientIp.js";
import { logger } from "../../lib/logger/logger.js";
import { getLocation } from "../../db/geolocation/geolocation.js";
@@ -93,7 +93,7 @@ export async function recordSessionReplay(
const body = recordSessionReplaySchema.parse(request.body) as RecordSessionReplayRequest;
// Check if the IP should be excluded from tracking
- const requestIP = getIpAddress(request);
+ const requestIP = resolveClientIp(request);
if (excludedIPs && excludedIPs.length > 0 && (await siteConfig.isIPExcluded(requestIP, request.params.siteId))) {
logger.info(`[SessionReplay] IP ${requestIP} excluded from tracking for site ${siteId}`);
@@ -161,7 +161,7 @@ export async function recordSessionReplay(
}
}
- const ipAddress = getIpAddress(request);
+ const ipAddress = resolveClientIp(request);
const origin = request.headers.origin || "";
const referrer = request.headers.referer || "";
diff --git a/server/src/services/tracker/botBlocking/README.md b/server/src/services/tracker/botBlocking/README.md
index b922857aa..028ca9030 100644
--- a/server/src/services/tracker/botBlocking/README.md
+++ b/server/src/services/tracker/botBlocking/README.md
@@ -100,6 +100,6 @@ This catches obvious floods and fast crawlers, but it is local to a Node process
## Trust Boundaries
-Bot blocking assumes the resolved IP is meaningful. The tracker resolves IPs from `X-Real-IP`, then `CF-Connecting-IP`, then `X-Forwarded-For`, then the Fastify request IP. Public tracking requests ignore client-supplied `ip_address` and `user_agent`; those overrides are only honored for trusted server-side ingestion.
+Bot blocking assumes the resolved IP is meaningful. The tracker resolves IPs from `X-Real-IP`, then `X-Forwarded-For`, then `CF-Connecting-IP`, then the Fastify request IP. The forwarded headers rank above `CF-Connecting-IP` so that first-party proxies (CloudFront, Fastly, nginx, ...) are attributed to the original visitor rather than the proxy's edge node. Public tracking requests ignore client-supplied `ip_address` and `user_agent`; those overrides are only honored for trusted server-side ingestion.
Client-supplied `_bs` and `_bsm` are useful inputs but are not secure proof.
diff --git a/server/src/services/tracker/identifyService.ts b/server/src/services/tracker/identifyService.ts
index 114556dee..4fa8225ac 100644
--- a/server/src/services/tracker/identifyService.ts
+++ b/server/src/services/tracker/identifyService.ts
@@ -6,7 +6,7 @@ import { clickhouse } from "../../db/clickhouse/clickhouse.js";
import { userProfiles, userAliases } from "../../db/postgres/schema.js";
import { siteConfig } from "../../lib/siteConfig.js";
import { userIdService } from "../userId/userIdService.js";
-import { getIpAddress } from "../../utils.js";
+import { resolveClientIp } from "./resolveClientIp.js";
import { createServiceLogger } from "../../lib/logger/logger.js";
const logger = createServiceLogger("identify-service");
@@ -89,7 +89,7 @@ export async function handleIdentify(request: FastifyRequest, reply: FastifyRepl
const anonymousId = anonymous_id
? await userIdService.generateUserIdFromClientId(anonymous_id, siteId)
: await userIdService.generateUserId(
- ip_address || getIpAddress(request),
+ ip_address || resolveClientIp(request),
user_agent || request.headers["user-agent"] || "",
siteId
);
diff --git a/server/src/services/tracker/requestIdentity.ts b/server/src/services/tracker/requestIdentity.ts
index c43d9f5b3..82c0d54ba 100644
--- a/server/src/services/tracker/requestIdentity.ts
+++ b/server/src/services/tracker/requestIdentity.ts
@@ -1,5 +1,5 @@
import { FastifyRequest } from "fastify";
-import { getIpAddress } from "../../utils.js";
+import { resolveClientIp } from "./resolveClientIp.js";
interface TrackingIdentityPayload {
ip_address?: string;
@@ -24,7 +24,7 @@ export function resolveTrackingIdentity(
payload: TrackingIdentityPayload,
trustedServerSideIngestion: boolean
): TrackingIdentity {
- const requestIpAddress = getIpAddress(request);
+ const requestIpAddress = resolveClientIp(request);
const requestUserAgent = getRequestUserAgent(request);
return {
diff --git a/server/src/services/tracker/resolveClientIp.test.ts b/server/src/services/tracker/resolveClientIp.test.ts
new file mode 100644
index 000000000..e561bdbc2
--- /dev/null
+++ b/server/src/services/tracker/resolveClientIp.test.ts
@@ -0,0 +1,62 @@
+import { describe, it, expect } from "vitest";
+import { FastifyRequest } from "fastify";
+import { resolveClientIp } from "./resolveClientIp.js";
+
+function requestWithHeaders(headers: Record<string, string | string[]>, ip = "198.51.100.10"): FastifyRequest {
+ return { headers, ip } as unknown as FastifyRequest;
+}
+
+// Inject a deterministic edge predicate so tests don't depend on the MaxMind DB.
+const proxied = () => true;
+const direct = () => false;
+
+describe("resolveClientIp", () => {
+ it("trusts the forwarded visitor IP when the edge is a datacenter (proxied)", () => {
+ // CloudFront case: CF-Connecting-IP is the proxy egress node; the real
+ // visitor is in X-Forwarded-For.
+ const request = requestWithHeaders({
+ "cf-connecting-ip": "13.224.0.1", // CloudFront egress
+ "x-forwarded-for": "203.0.113.10, 13.224.0.1", // visitor, then edge
+ });
+
+ expect(resolveClientIp(request, proxied)).toBe("203.0.113.10");
+ });
+
+ it("prefers X-Real-IP over X-Forwarded-For on the proxied path", () => {
+ const request = requestWithHeaders({
+ "cf-connecting-ip": "13.224.0.1",
+ "x-forwarded-for": "203.0.113.10",
+ "x-real-ip": "192.0.2.10",
+ });
+
+ expect(resolveClientIp(request, proxied)).toBe("192.0.2.10");
+ });
+
+ it("falls back to the edge IP if a proxy forwarded nothing", () => {
+ const request = requestWithHeaders({ "cf-connecting-ip": "13.224.0.1" });
+ expect(resolveClientIp(request, proxied)).toBe("13.224.0.1");
+ });
+
+ it("trusts the edge IP and ignores spoofable forwarded headers on the direct path", () => {
+ // A direct visitor cannot spoof their geo: CF-Connecting-IP wins and the
+ // attacker-supplied X-Forwarded-For / X-Real-IP are discarded.
+ const request = requestWithHeaders({
+ "cf-connecting-ip": "203.0.113.55", // residential visitor
+ "x-forwarded-for": "8.8.8.8", // spoofed
+ "x-real-ip": "1.1.1.1", // spoofed
+ });
+
+ expect(resolveClientIp(request, direct)).toBe("203.0.113.55");
+ });
+
+ it("uses forwarded headers when there is no Cloudflare edge (self-hosted)", () => {
+ expect(resolveClientIp(requestWithHeaders({ "x-real-ip": "192.0.2.10" }), direct)).toBe("192.0.2.10");
+ expect(resolveClientIp(requestWithHeaders({ "x-forwarded-for": "203.0.113.10, 10.0.0.1" }), direct)).toBe(
+ "203.0.113.10"
+ );
+ });
+
+ it("falls back to the socket IP when no usable headers are present", () => {
+ expect(resolveClientIp(requestWithHeaders({}), direct)).toBe("198.51.100.10");
+ });
+});
diff --git a/server/src/services/tracker/resolveClientIp.ts b/server/src/services/tracker/resolveClientIp.ts
new file mode 100644
index 000000000..3851f9aa6
--- /dev/null
+++ b/server/src/services/tracker/resolveClientIp.ts
@@ -0,0 +1,98 @@
+import { FastifyRequest } from "fastify";
+import { lookupAsn } from "../../db/geolocation/asn.js";
+import { getIpAddress } from "../../utils.js";
+import { isDatacenterAsn } from "./botBlocking/datacenterAsns.js";
+
+/**
+ * ASN-aware client IP resolution β€” the durable successor to `getIpAddress`.
+ *
+ * The problem `getIpAddress` can't solve: no static header precedence is correct
+ * for both topologies at once.
+ *
+ * - Direct Cloudflare path β€” `CF-Connecting-IP` is the real visitor and is
+ * non-spoofable; the forwarded headers are client-controllable noise.
+ * - First-party proxy path β€” a customer fronts Rybbit with CloudFront / Fastly
+ * / nginx, so `CF-Connecting-IP` is the *proxy's* egress node and the visitor
+ * only survives in `X-Forwarded-For` / `X-Real-IP`.
+ *
+ * `getIpAddress` resolves this by always preferring the forwarded headers, which
+ * fixes proxied customers but lets a direct visitor spoof their own geo.
+ *
+ * This variant removes that exposure by detecting the topology from the edge IP:
+ * if `CF-Connecting-IP` belongs to a hosting/datacenter ASN, a first-party proxy
+ * is in front, so we trust the forwarded visitor IP. Otherwise the edge IP is a
+ * residential/business visitor and is authoritative β€” forwarded headers are
+ * ignored, so they can't be spoofed.
+ *
+ * Wired as the default client-IP resolver for tracking ingestion, identify,
+ * session replay, and feature-flag evaluation. When no Cloudflare edge is present
+ * it delegates to `getIpAddress` (the simple header-precedence primitive).
+ * Requires `GeoLite2-ASN.mmdb` (the same DB Layer-4 bot detection uses); see the
+ * fallback note on `isProxiedEdge` below for the DB-missing behaviour.
+ */
+
+/** First non-empty `X-Forwarded-For` entry, or null. */
+function firstForwardedFor(request: FastifyRequest): string | null {
+ const forwardedFor = request.headers["x-forwarded-for"];
+ if (forwardedFor && typeof forwardedFor === "string") {
+ const ips = forwardedFor
+ .split(",")
+ .map(ip => ip.trim())
+ .filter(Boolean);
+ if (ips.length > 0) return ips[0];
+ }
+ return null;
+}
+
+function realIpHeader(request: FastifyRequest): string | null {
+ const realIp = request.headers["x-real-ip"];
+ return realIp && typeof realIp === "string" ? realIp.trim() : null;
+}
+
+function cfConnectingIp(request: FastifyRequest): string | null {
+ const cfIp = request.headers["cf-connecting-ip"];
+ return cfIp && typeof cfIp === "string" ? cfIp.trim() : null;
+}
+
+/**
+ * Whether the connecting edge IP belongs to hosting/datacenter infrastructure β€”
+ * positive evidence that a first-party proxy sits in front of our edge.
+ *
+ * Fallback semantics matter: if the ASN DB is unavailable, `lookupAsn` returns
+ * null and this returns `false` β€” i.e. unknown edges are treated as direct and
+ * `CF-Connecting-IP` wins. That is the safe default for the (vast) direct
+ * majority, but it means proxied customers regress to seeing their proxy IP when
+ * the DB is missing. If that trade-off is wrong for your deployment, default this
+ * to `true` on a null lookup instead (favouring proxied correctness over
+ * direct-path anti-spoofing).
+ */
+function isProxiedEdge(ip: string): boolean {
+ const asn = lookupAsn(ip);
+ return isDatacenterAsn(asn?.asn);
+}
+
+export function resolveClientIp(
+ request: FastifyRequest,
+ // Injectable for tests so branching can be exercised without the MaxMind DB.
+ proxiedEdge: (ip: string) => boolean = isProxiedEdge
+): string {
+ const realIp = realIpHeader(request);
+ const xffFirst = firstForwardedFor(request);
+ const cfIp = cfConnectingIp(request);
+
+ if (cfIp) {
+ if (proxiedEdge(cfIp)) {
+ // First-party proxy in front: trust the forwarded visitor IP, fall back to
+ // the edge IP only if the proxy forwarded nothing.
+ return realIp ?? xffFirst ?? cfIp;
+ }
+ // Direct visitor: the edge IP is authoritative and non-spoofable. Ignore
+ // forwarded headers entirely.
+ return cfIp;
+ }
+
+ // No Cloudflare edge (e.g. self-hosted behind nginx): there's no edge IP to
+ // corroborate, so defer to plain header precedence (X-Real-IP, then
+ // X-Forwarded-For, then the socket IP).
+ return getIpAddress(request);
+}
diff --git a/server/src/utils.test.ts b/server/src/utils.test.ts
index ccbb8d2ac..ebadb1d73 100644
--- a/server/src/utils.test.ts
+++ b/server/src/utils.test.ts
@@ -7,7 +7,7 @@ function requestWithHeaders(headers: Record<string, string | string[]>, ip = "19
}
describe("getIpAddress", () => {
- it("uses X-Real-IP before Cloudflare's connecting IP", () => {
+ it("uses X-Real-IP before all other headers", () => {
const request = requestWithHeaders({
"cf-connecting-ip": "198.51.100.20",
"x-forwarded-for": "203.0.113.10, 198.51.100.20",
@@ -17,19 +17,20 @@ describe("getIpAddress", () => {
expect(getIpAddress(request)).toBe("192.0.2.10");
});
- it("uses Cloudflare's connecting IP before X-Forwarded-For", () => {
+ it("uses the first X-Forwarded-For IP before Cloudflare's connecting IP", () => {
+ // The proxied case: a first-party proxy (e.g. CloudFront) forwards the visitor
+ // in X-Forwarded-For, while CF-Connecting-IP is our Cloudflare edge seeing the
+ // proxy's egress node. The visitor must win.
const request = requestWithHeaders({
- "cf-connecting-ip": "203.0.113.10",
- "x-forwarded-for": "198.51.100.20, 10.0.0.1",
+ "cf-connecting-ip": "198.51.100.20",
+ "x-forwarded-for": "203.0.113.10, 198.51.100.20",
});
expect(getIpAddress(request)).toBe("203.0.113.10");
});
- it("uses the first X-Forwarded-For IP when higher-priority headers are missing", () => {
- expect(getIpAddress(requestWithHeaders({ "x-forwarded-for": "203.0.113.10, 198.51.100.20" }))).toBe(
- "203.0.113.10"
- );
+ it("falls back to Cloudflare's connecting IP when no forwarded headers are present", () => {
+ expect(getIpAddress(requestWithHeaders({ "cf-connecting-ip": "203.0.113.10" }))).toBe("203.0.113.10");
});
it("falls back to the request IP", () => {
diff --git a/server/src/utils.ts b/server/src/utils.ts
index d4b557a56..4946e68c0 100644
--- a/server/src/utils.ts
+++ b/server/src/utils.ts
@@ -238,22 +238,35 @@ export const normalizeOrigin = (input: string): string => {
};
// Helper function to get IP address
+//
+// NOTE: production request paths resolve client IPs via `resolveClientIp()`
+// (resolveClientIp.ts), which wraps this with ASN-based proxy detection. This
+// function is the simple header-precedence primitive it falls back to when no
+// Cloudflare edge is present, and remains the building block for that decision.
+//
+// Header precedence is tuned for first-party proxies (Cloudflare Workers, AWS
+// CloudFront, Fastly, nginx, ...) that front Rybbit and forward the original
+// visitor IP. The hard constraint: when such a proxy sits in front of our
+// Cloudflare edge, `CF-Connecting-IP` is the *proxy's* egress IP, not the
+// visitor β€” so it must rank BELOW the forwarded headers, otherwise proxied
+// traffic geolocates to the proxy's exit node. (CloudFront, for one, cannot
+// forward `X-Real-IP` at all β€” it's on AWS's origin custom-header denylist β€”
+// so `X-Forwarded-For` is the only channel many proxies have.)
+//
+// For the non-proxied path these forwarded headers are client-spoofable, which
+// only lets a visitor corrupt their own (anonymous) geo β€” acceptable. See
+// resolveClientIp.ts for the ASN-aware variant that removes even that exposure.
export const getIpAddress = (request: FastifyRequest): string => {
- // Priority 1: X-Real-IP. First-party proxies can set this to the original
- // visitor IP even when our Cloudflare edge sees the proxy's egress IP.
+ // Priority 1: X-Real-IP. First-party proxies (e.g. nginx, Cloudflare Workers)
+ // set this to the single original visitor IP.
const realIp = request.headers["x-real-ip"];
if (realIp && typeof realIp === "string") {
return realIp.trim();
}
- // Priority 2: Cloudflare header. In our Cloudflare-fronted direct path this
- // is authoritative; X-Forwarded-For may contain intermediary edge IPs.
- const cfConnectingIp = request.headers["cf-connecting-ip"];
- if (cfConnectingIp && typeof cfConnectingIp === "string") {
- return cfConnectingIp.trim();
- }
-
- // Priority 3: X-Forwarded-For - use the first IP, which should be the original client.
+ // Priority 2: X-Forwarded-For - use the first IP, which should be the original
+ // client. Ranks above CF-Connecting-IP so proxied traffic is attributed to the
+ // visitor rather than the proxy's edge node.
const forwardedFor = request.headers["x-forwarded-for"];
if (forwardedFor && typeof forwardedFor === "string") {
const ips = forwardedFor
@@ -265,5 +278,12 @@ export const getIpAddress = (request: FastifyRequest): string => {
}
}
+ // Priority 3: Cloudflare header. Authoritative on the direct Cloudflare-fronted
+ // path; only reached when no forwarded header is present.
+ const cfConnectingIp = request.headers["cf-connecting-ip"];
+ if (cfConnectingIp && typeof cfConnectingIp === "string") {
+ return cfConnectingIp.trim();
+ }
+
return request.ip;
};