πͺ’ Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.
TypeScript analyticsautogenevaluationlangchainlarge-language-modelsllama-indexllmllm-evaluationllm-observabilityllmopsmonitoringobservabilityopen-sourceopenaiplaygroundprompt-engineeringprompt-managementself-hostedycombinator
fix(sentry): redact PII in all session replays (#16373)
Max Deichmann06f7a377b288
diff --git a/.agents/skills/sentry-instrumentation/SKILL.md b/.agents/skills/sentry-instrumentation/SKILL.mdindex a16bfd7aed..180e59a25f 100644--- a/.agents/skills/sentry-instrumentation/SKILL.md+++ b/.agents/skills/sentry-instrumentation/SKILL.md@@ -122,15 +122,14 @@ ErrorEvent]` fingerprints with no stack. β The 5EX cluster (opaque non-Erroruser/session ids. This is not hygiene β it is the compliance boundary: ALLcloud regions, **including HIPAA**, report to the **same US Sentry org**,and error events are **never masked**, so this discipline is the only thing- keeping user content out of them. Session Replay is the one masked channel:- [`instrumentation-client.ts`](../../../web/instrumentation-client.ts) gates- `maskAllText`/`blockAllMedia` so replays are fully masked everywhere except- the EU/US non-HIPAA cloud regions. **Never remove or weaken that gate** β- the gate (`isEuOrUsRegionNonHipaa` β the `replayIntegration` options) must- be covered by a CI regression test asserting the masked/unmasked regions on- the real module (the mask-guard+ keeping user content out of them. Session Replay is the masked channel:+ [`instrumentation-client.ts`](../../../web/instrumentation-client.ts) sets+ `maskAllText`, `maskAllInputs`, and `blockAllMedia` so every replay is fully+ masked in every deployment. **Never remove or weaken that mask** β the+ `replayIntegration` options must be covered by a CI regression test+ asserting every region against the real module (the mask-guard`instrumentation-client-replay-mask.clienttest.ts`; landing with #15802).- Reject any diff that touches the gate without that guard passing, and run+ Reject any diff that touches Replay masking without that guard passing, and runevery diff touching `instrumentation-client.ts` or replay config throughthe reviewer checklist in the[reference](references/sentry-capture-contract.md#pii-and-the-hipaa-compliance-boundary).@@ -171,7 +170,7 @@ ErrorEvent]` fingerprints with no stack. β The 5EX cluster (opaque non-Errorthat reads the wrong event field (Rule 6).- Interpolating ids/user data into the message β PII + grouping explosion(Rules 5, 7).-- Removing/weakening the region-gated replay mask, or "fixing" the mask-guard+- Removing/weakening the unconditional replay mask, or "fixing" the mask-guardtest instead of the diff that tripped it (Rule 7).- Trusting a green jsdom test as proof the noise is gone (Rule 8).- An ad-hoc inline `beforeSend` check instead of a named, tested predicate indiff --git a/.agents/skills/sentry-instrumentation/references/sentry-capture-contract.md b/.agents/skills/sentry-instrumentation/references/sentry-capture-contract.mdindex 3032d98f8a..1ea5b4af03 100644--- a/.agents/skills/sentry-instrumentation/references/sentry-capture-contract.md+++ b/.agents/skills/sentry-instrumentation/references/sentry-capture-contract.md@@ -18,7 +18,7 @@ the detail behind its rules.- **The only `Sentry.init`** is [`web/instrumentation-client.ts`](../../../../web/instrumentation-client.ts)β client-side only. It configures `beforeSend`, `captureConsoleIntegration({ levels: ["error"] })`,- `httpClientIntegration()`, `replayIntegration(...)` (masking gated on region/HIPAA),+ `httpClientIntegration()`, `replayIntegration(...)` (unconditional masking),and `denyUrls` (browser-extension origins).- **The worker and web server have no Sentry SDK** β backend signal lives in theserver-side observability/APM stack, not Sentry. Do not assume a backend error@@ -42,18 +42,16 @@ ALL cloud regions β **including HIPAA** β report to the **same US Sentry orgTwo mechanisms keep user content (prompts, traces, PII/PHI) out of it, and theycover **different channels**. Do not conflate them.-1. **Session Replay β the region-gated mask.**+1. **Session Replay β the unconditional mask.**[`instrumentation-client.ts`](../../../../web/instrumentation-client.ts)- configures `replayIntegration({ maskAllText, blockAllMedia })` gated on the- cloud region via `isEuOrUsRegionNonHipaa`: replays are **fully masked- everywhere except the EU/US non-HIPAA cloud regions** (HIPAA, JP, STAGING,- DEV, and self-hosted/unset all mask β default-deny). The gate must never be+ configures `replayIntegration({ maskAllText, maskAllInputs, blockAllMedia })`+ so replays are **fully masked in every deployment**. The mask must never beremoved or weakened, and it must be covered by a CI regression test that- executes the real module per region and fails on any change to the gate+ executes the real module per region and fails on any change to the options(the mask-guard `instrumentation-client-replay-mask.clienttest.ts`; landingwith #15802). A diff that trips that guard is a compliance change, not a- broken test; a diff that touches the gate without the guard passing must be- rejected.+ broken test; a diff that touches Replay masking without the guard passing+ must be rejected.2. **Error events β the capture discipline (SKILL rule 7).** The mask coversSession Replay ONLY. Error-event payloads β message, breadcrumbs, `extra`,tags β are **not masked in any region**. The rule "no user content in@@ -62,14 +60,13 @@ cover **different channels**. Do not conflate them.so the SDK attaches no cookies, headers, request/response bodies, or userIP.-**Reviewer checklist β any diff touching `instrumentation-client.ts`, the-replay integration, or region gating:**+**Reviewer checklist β any diff touching `instrumentation-client.ts` or the+replay integration:**-- `maskAllText`/`blockAllMedia` still derive from the region gate (masked- unless the region is exactly `EU` or `US`), and the mask-guard clienttest- (landing with #15802) asserts against the real module β not a copy of the- config or an uncalled predicate.-- No new replay option weakens masking for masked regions (e.g.+- `maskAllText`, `maskAllInputs`, and `blockAllMedia` remain `true` in every+ deployment, and the mask-guard clienttest (landing with #15802) asserts+ against the real module β not a copy of the config or an uncalled predicate.+- No new replay option weakens masking (e.g.`maskAllInputs: false`, unmask/unblock selectors).- `sendDefaultPii` remains unset/false; no integration starts attachingbodies, headers, or cookies.diff --git a/web/instrumentation-client.ts b/web/instrumentation-client.tsindex 227e3423c3..727fc86698 100644--- a/web/instrumentation-client.ts+++ b/web/instrumentation-client.ts@@ -8,11 +8,6 @@ import {STALE_CHUNK_PARSE_FINGERPRINT,} from "@/src/utils/sentryFilters";-const isEuOrUsRegionNonHipaa =- process.env.NEXT_PUBLIC_LANGFUSE_CLOUD_REGION !== undefined- ? ["EU", "US"].includes(process.env.NEXT_PUBLIC_LANGFUSE_CLOUD_REGION)- : false;-Sentry.init({dsn: process.env.NEXT_PUBLIC_SENTRY_DSN,environment: process.env.NEXT_PUBLIC_SENTRY_ENVIRONMENT,@@ -71,8 +66,9 @@ Sentry.init({// Replay may only be enabled for the client-sideintegrations: [Sentry.replayIntegration({- maskAllText: !isEuOrUsRegionNonHipaa,- blockAllMedia: !isEuOrUsRegionNonHipaa,+ maskAllText: true,+ maskAllInputs: true,+ blockAllMedia: true,}),Sentry.browserTracingIntegration(),Sentry.httpClientIntegration(),diff --git a/web/src/__tests__/instrumentation-client-replay-mask.clienttest.ts b/web/src/__tests__/instrumentation-client-replay-mask.clienttest.tsindex a3f11dc7fa..793ea79095 100644--- a/web/src/__tests__/instrumentation-client-replay-mask.clienttest.ts+++ b/web/src/__tests__/instrumentation-client-replay-mask.clienttest.ts@@ -1,15 +1,14 @@// @vitest-environment node/**- * HIPAA / non-EU-US Session Replay masking guard.+ * Session Replay masking guard.** All cloud regions (including HIPAA) report to the same US Sentry org.- * The compliance safeguard for Session Replay is the region-gated- * `maskAllText` / `blockAllMedia` configuration in- * `web/instrumentation-client.ts`: replays are masked everywhere EXCEPT the- * EU and US non-HIPAA cloud regions. Removing or weakening that gate would- * ship unmasked replay of user content (prompts, traces, PII/PHI) from- * HIPAA and other regions to Sentry.+ * The compliance safeguard for Session Replay is the unconditional+ * `maskAllText` / `maskAllInputs` / `blockAllMedia` configuration in+ * `web/instrumentation-client.ts`: replays are masked in every deployment.+ * Removing or weakening that configuration would ship unmasked replay of+ * user content (prompts, traces, PII/PHI) to Sentry.** Scope note: the mask covers Session Replay ONLY. Error-event payloads* (message, breadcrumbs, extra, tags) are never masked β for those, the@@ -25,16 +24,17 @@* change weakens the compliance mask β fix the change, not the test.*/-type ReplayOptions = { maskAllText: boolean; blockAllMedia: boolean };+type ReplayOptions = {+ maskAllText: boolean;+ maskAllInputs: boolean;+ blockAllMedia: boolean;+};const { initMock, replayIntegrationMock, replaySentinel } = vi.hoisted(() => {const replaySentinel = { name: "Replay-sentinel" };return {initMock: vi.fn<(options: { integrations: unknown[] }) => void>(),- replayIntegrationMock: vi.fn(- (_options: { maskAllText: boolean; blockAllMedia: boolean }) =>- replaySentinel,- ),+ replayIntegrationMock: vi.fn((_options: ReplayOptions) => replaySentinel),replaySentinel,};});@@ -77,40 +77,30 @@ afterEach(() => {vi.unstubAllEnvs();});-describe("Session Replay masking is region-gated (compliance guard)", () => {- // Regions that must ALWAYS be fully masked. `toStrictEqual` on the whole+describe("Session Replay masking is unconditional (compliance guard)", () => {+ // Every region must be fully masked. `toStrictEqual` on the whole// options object is deliberate: any new replay option must be reviewed// here for its compliance impact before it can ship.- const alwaysMaskedRegions: [string | undefined, string][] = [+ const regions: [string | undefined, string][] = [+ ["EU", "the EU cloud region"],+ ["US", "the US cloud region"],["HIPAA", "the HIPAA cloud region"],["JP", "a non-EU/US cloud region"],["STAGING", "the staging environment"],["DEV", "the dev environment"],[undefined, "self-hosted (region unset)"],["", "an empty region value"],- ["us", "a lowercase region value (gate must be exact-match)"],+ ["us", "a lowercase region value"],];- it.each(alwaysMaskedRegions)(- "region %j (%s) gets maskAllText + blockAllMedia",+ it.each(regions)(+ "region %j (%s) gets text, input, and media masking",async (region) => {await expect(loadReplayOptions(region)).resolves.toStrictEqual({maskAllText: true,+ maskAllInputs: true,blockAllMedia: true,});},);-- // The ONLY regions where unmasked replay is permitted. If this case starts- // failing because masking became unconditional, that is a deliberate- // product decision to confirm β not a bug in this test.- it.each(["EU", "US"])(- "only the %s cloud region may run unmasked replay",- async (region) => {- await expect(loadReplayOptions(region)).resolves.toStrictEqual({- maskAllText: false,- blockAllMedia: false,- });- },- );});