LLM-Driven Extraction of Unstructured Data — Built for API Deployments & ETL Pipeline Workflows
Python ai-agentsdata-engineeringdocument-aigenerative-aiidpjson-extractionllmmcp-serverocrpdf-extractionprompt-engineeringstructured-output
UN-2238 [FIX] Enforce the frontend CSP (out of report-only mode) (#2245)
Deepak Kd12e452dbe79
diff --git a/.claude/skills/csp-check/SKILL.md b/.claude/skills/csp-check/SKILL.mdnew file mode 100644index 000000000..b4a958549--- /dev/null+++ b/.claude/skills/csp-check/SKILL.md@@ -0,0 +1,182 @@+---+name: csp-check+description: >+ Check the frontend Content-Security-Policy in frontend/nginx.conf against what the app+ actually loads. The policy is enforced, so a host it does not list is blocked. Use when+ adding or upgrading a third-party frontend dependency (CDN, analytics, payments,+ widgets), when a feature loads assets from a new external host, when a CSP violation+ shows up in the browser console, or when something works in `bun run dev` but breaks+ behind nginx.+---++# CSP Check++The frontend ships an enforcing `Content-Security-Policy` header from+`frontend/nginx.conf`. A resource from an origin no directive lists is **blocked** — whatever needed it breaks, and the only trace is a+console violation. So the policy has to be widened in the same change that adds the+dependency, not after someone reports a broken page.++There is no report collector: violations reach each user's browser console and nowhere+else. These checks are the only way to see a gap before a user does. Note the header is+only served by the production nginx image — `bun run dev` has no CSP at all, so "it worked+locally" says nothing.++Three checks, cheapest first. Run 1 on every change that touches a frontend dependency;+run 2 and 3 before widening the policy or shipping a new third-party integration.++## 1. Static: does the bundle reference a host the policy never allows?++```bash+cd .claude/skills/csp-check/scripts+python3 extract_policy.py # what the policy says today+python3 scan_origins.py --url https://us-central.unstract.com+python3 scan_origins.py --dist # after `bun run build`+```++Bare `--dist` resolves the repo's own `frontend/build`, from any directory. Give it a path+only for a build somewhere else — and note the path is relative to your shell, not to the+script. The script exits non-zero when `--dist` is missing or holds no `.js`, when a+`--url` index names no bundle, and when a chunk `index.html` links fails to fetch; a 404+on a path found only inside a bundle string is tolerated, since that is usually a worker+path a chunk names but never loads. Check the file count on the first output line against+what the build produced.++`scan_origins.py` pulls every `/assets/*.js|css` chunk (following relative imports), plus+`index.html` and the entrypoint-generated `/config/runtime-config.js`, extracts external+`https://` URLs, and exits non-zero on any the policy would block. Hosts that only appear+in doc links, XML namespaces and library error strings are listed in its `IGNORED` set —+extend it rather than widening the policy for a host nothing fetches.++Verdicts: `MISSING` (no fetch directive names the host — `form-action`, `base-uri` and+`frame-ancestors` do not count, since nothing can be loaded on their strength), `PATH`+(listed path-scoped and this URL falls outside it — `https://www.google.com/recaptcha/`+grants that path only, so `https://www.google.com/g/collect` is still blocked) and `PORT`+(a source with no port grants 443 only). `*.` sources match subdomains only, the way the+browser reads them: `https://*.google-analytics.com` covers `region1.google-analytics.com`+but not a bare `google-analytics.com`.++This catches "a new dependency pulls from a new CDN". It cannot tell you *which*+directive loads a host — a font from a script-src-only host still violates. That is check 2.++What it still cannot see: a URL assembled at runtime, and a URL supplied by deployment+config. `VITE_CUSTOM_LOGO_URL` / `VITE_FAVICON_PATH` are the live example — see+"Operator-supplied URLs" below.++## 2. Live: probe the deployed policy, directive by directive++With the chrome-devtools MCP on a page of the target deployment:++1. `python3 extract_policy.py --json` and paste `directives` into `DIRECTIVES` in+ `scripts/probe.js`.+2. Run the whole file as the `function` argument of `evaluate_script`.++It loads one throwaway resource per (directive, host) pair and returns:++- `unexpected` — hosts the policy is meant to allow but the deployment still reports.+ Non-empty means the running deployment does not serve the policy in this repo, or the+ host is allowed on the wrong directive.+- `controlsNotReported` — must be empty. Non-empty means CSP is not being applied at all.++A wildcard source is probed as `csp-probe.<suffix>`, because `https://*.example.com/…` is+not a hostname and the request would die before CSP saw it. CSP is evaluated before DNS —+which is why the `csp-control.invalid` controls get reported without resolving — so a+`csp-probe.*` entry in `unexpected` means the deployment is not serving that wildcard.++CSP evaluates **redirect targets**: a probe path that 404-redirects to another host+(`https://hooks.stripe.com/` → `https://stripe.com`) reports the target, not a real gap.++## 3. Real usage: collect violations while driving the app++Probes only test hosts already in the policy. To find what a feature loads that nobody+listed, record violations while using it. Navigate with this `initScript` (chrome-devtools+`navigate_page`), which survives SPA route changes:++```js+document.addEventListener('securitypolicyviolation', (e) => {+ const k = '__cspAll';+ const prev = JSON.parse(sessionStorage.getItem(k) || '[]');+ prev.push({dir: e.effectiveDirective || e.violatedDirective, blocked: e.blockedURI,+ src: (e.sourceFile || '') + ':' + (e.lineNumber || ''), page: location.pathname});+ sessionStorage.setItem(k, JSON.stringify(prev.slice(-400)));+});+```++Then walk the feature and read `JSON.parse(sessionStorage.getItem('__cspAll'))`. SPA routes+can be walked without reloading:+`history.pushState({}, '', path); dispatchEvent(new PopStateEvent('popstate'))`.++Third-party widgets load lazily and per-plan, so exercise the actual flow — an integration+that never initialises reports nothing.++## Operator-supplied URLs++`generate-runtime-config.sh` writes `/config/runtime-config.js` at container start from+`VITE_CUSTOM_LOGO_URL` / `VITE_FAVICON_PATH` (and their `REACT_APP_*` fallbacks). Both+end up governed by `img-src`: the logo through `<Image src>` in `TopNavBar.jsx`, the+favicon through `setFavicon` in `index.jsx`.++**They must be same-origin paths** (`/config/logo.svg`), not absolute URLs on another+host. An absolute URL to a host the policy does not list is blocked once the header+enforces, and both failures are quiet — the logo's `onError` swaps in the default mark+and the favicon just stays the old one. No build-time scan can catch it, because the+value does not exist until the container starts.++An operator who must serve branding from their own CDN has to add that host to `img-src`+in `frontend/nginx.conf` and rebuild the frontend image.++The same shape, not live today: `sessionDetails.picture` is an absolute avatar URL the+auth backend may supply, rendered by `<Image src>` in `TopNavBar.jsx` and in the+subscription-end plugin page. Neither `account_v2.AuthenticationService.get_user_info`+nor the cloud Auth0 plugin's populates it — the Auth0 models parse `picture` from the IdP+token but never carry it into `UserInfo` — so the initials branch always renders and no+IdP host is in `img-src`. If that ever changes, the avatar is blocked, and because+`picture` would then be truthy the component shows a broken-image placeholder rather than+falling back to initials. Whoever fills that field adds the IdP host here.++**The general rule:** any URL that arrives at runtime — deployment config, an API+response, an IdP token — is invisible to checks 1 and 2 and only shows up in check 3, and+then only if the account you drive the app with actually has the value set.++## Changing the policy++Edit the single `add_header Content-Security-Policy` line in `frontend/nginx.conf`. Keep+it one line: nginx accepts multi-line quoted strings, but the newlines end up in the+header value.++Verify before pushing — nginx will start with a malformed policy and browsers will silently+drop the bad directive:++```bash+docker run -d --name csp-probe -p 8899:80 \+ -v "$PWD/frontend/nginx.conf:/etc/nginx/nginx.conf:ro" nginx:alpine+curl -sI http://localhost:8899/ | grep -i content-security-policy+```++Then point check 2 at `http://localhost:8899/` to confirm the new policy allows what it+should and still blocks the controls. `docker rm -f csp-probe` when done.++Which directive a host belongs in:++| Loaded as | Directive |+|---|---|+| `<script src>`, dynamic `import()` | `script-src` |+| `<link rel=stylesheet>`, `@import` | `style-src` |+| `<img>`, CSS `url()` background, tracking pixel | `img-src` |+| `@font-face`, `FontFace()` | `font-src` |+| `fetch`/XHR/`sendBeacon`/WebSocket | `connect-src` |+| `<iframe>` | `frame-src` |+| `<video>`/`<audio>` | `media-src` |+| `new Worker()` | `worker-src` |++A host loaded several ways needs an entry in each directive — that is the most common+miss. Path-scoped sources (`https://www.gstatic.com/recaptcha/`) keep the grant narrow and+are worth using when a vendor serves everything from one host.++## Notes++- `connect-src` carries no `wss:` wildcard. socket.io connects to `window.location.origin`+ (`frontend/src/helpers/GetStaticData.js` `getBaseUrl`) and `'self'` covers same-origin+ ws/wss per CSP3. Verified with a `ws://` probe against a local nginx serving the policy.+- The backend sends its own enforcing `Content-Security-Policy` from+ `backend/middleware/content_security_policy.py`. It applies to backend responses (JSON+ APIs, the OSS login page), not to the SPA — do not confuse the two when debugging.diff --git a/.claude/skills/csp-check/scripts/extract_policy.py b/.claude/skills/csp-check/scripts/extract_policy.pynew file mode 100755index 000000000..27dab5b04--- /dev/null+++ b/.claude/skills/csp-check/scripts/extract_policy.py@@ -0,0 +1,79 @@+#!/usr/bin/env python3+"""Parse the Content-Security-Policy out of frontend/nginx.conf.++Usage:+ python3 extract_policy.py [path/to/nginx.conf] # pretty-print per directive+ python3 extract_policy.py --json [path/to/nginx.conf] # machine-readable+"""++import json+import re+import sys+from pathlib import Path++HEADER_RE = re.compile(+ r'add_header\s+(Content-Security-Policy(?:-Report-Only)?)\s+"(?P<policy>[^"]*)"',+ re.IGNORECASE,+)+DEFAULT_CONF = Path(__file__).resolve().parents[4] / "frontend" / "nginx.conf"+++def parse(conf_path: Path) -> tuple[str, dict[str, list[str]]]:+ """Return (header_name, {directive: [sources]}) for the conf's enforcing CSP header.++ Everything downstream trusts this as "what the browser sees", so it has to pick the+ same header the browser would: not a commented-out one, and not a -Report-Only+ header that happens to sit above the enforcing one (the usual shape while the next+ policy change is being trialled).+ """+ live = "\n".join(+ line+ for line in conf_path.read_text().splitlines()+ if not line.lstrip().startswith("#")+ )+ matches = HEADER_RE.findall(live)+ if not matches:+ raise SystemExit(f"No Content-Security-Policy add_header found in {conf_path}")+ enforcing = [m for m in matches if m[0].lower() == "content-security-policy"]+ chosen = enforcing or matches+ if len(chosen) > 1:+ names = ", ".join(name for name, _ in chosen)+ raise SystemExit(+ f"{conf_path} has {len(chosen)} CSP headers ({names}) -- ambiguous"+ )+ header, policy = chosen[0]+ directives: dict[str, list[str]] = {}+ for chunk in policy.split(";"):+ parts = chunk.split()+ if not parts:+ continue+ if parts[0] in directives:+ # The browser honours the first occurrence and ignores the rest, so keeping+ # the last would let the gate clear sources the browser never applies.+ print(+ f"warning: duplicate directive {parts[0]!r} in {conf_path}; "+ "the browser uses the first and ignores this one",+ file=sys.stderr,+ )+ continue+ directives[parts[0]] = parts[1:]+ return header, directives+++def main() -> None:+ args = [a for a in sys.argv[1:] if a != "--json"]+ as_json = "--json" in sys.argv[1:]+ conf = Path(args[0]) if args else DEFAULT_CONF+ header, directives = parse(conf)+ if as_json:+ print(json.dumps({"header": header, "directives": directives}, indent=2))+ return+ print(f"{header} ({conf})")+ for directive, sources in directives.items():+ print(f"\n {directive}")+ for source in sources:+ print(f" {source}")+++if __name__ == "__main__":+ main()diff --git a/.claude/skills/csp-check/scripts/probe.js b/.claude/skills/csp-check/scripts/probe.jsnew file mode 100644index 000000000..d5086e4b9--- /dev/null+++ b/.claude/skills/csp-check/scripts/probe.js@@ -0,0 +1,118 @@+/**+ * Browser-side CSP probe.+ *+ * Paste the whole file as the `function` argument of the chrome-devtools MCP+ * `evaluate_script` tool while a page from the target deployment is selected, with the+ * output of `extract_policy.py --json` inlined as DIRECTIVES below.+ *+ * It loads one throwaway resource per (directive, host) pair and records what the live+ * policy reports, so it answers two questions the config file alone cannot:+ * 1. does the deployment actually serve the policy we think it does?+ * 2. is each host allowed on the directive that will really load it?+ * Four control probes must always be reported -- if they are not, CSP is not applied.+ *+ * Note: CSP evaluates redirect targets. A probe path that 404-redirects to another host+ * (https://hooks.stripe.com/ -> https://stripe.com) reports that target, not a real gap.+ */+async () => {+ const DIRECTIVES = {+ /* paste extract_policy.py --json "directives" here */+ };++ const KIND_BY_DIRECTIVE = {+ "script-src": "script",+ "style-src": "style",+ "img-src": "img",+ "font-src": "font",+ "connect-src": "connect",+ "frame-src": "frame",+ "media-src": "media",+ "worker-src": "worker",+ };++ const probes = [];+ for (const [directive, sources] of Object.entries(DIRECTIVES)) {+ const kind = KIND_BY_DIRECTIVE[directive];+ if (!kind) continue;+ for (const source of sources) {+ if (!source.startsWith("https://")) continue;+ // A wildcard source is not a hostname: https://*.example.com/__csp_probe never+ // parses, so the request dies before CSP sees it and the probe looks clean whether+ // or not the deployment serves the wildcard. Substituting a concrete label does+ // test it -- CSP is evaluated before DNS, which is why the csp-control.invalid+ // controls below get reported despite not resolving.+ const url = source.replace("://*.", "://csp-probe.").replace(/\/$/, "");+ probes.push([kind, url + "/__csp_probe", false]);+ }+ }+ for (const kind of ["img", "connect", "script"]) {+ probes.push([kind, "https://csp-control.invalid/__csp_probe", true]);+ }+ probes.push(["connect", "wss://csp-control.invalid/__csp_probe", true]);++ const hits = [];+ const onViolation = (e) =>+ hits.push({+ directive: e.effectiveDirective || e.violatedDirective,+ blocked: e.blockedURI,+ });+ document.addEventListener("securitypolicyviolation", onViolation);+ const wait = (ms) => new Promise((r) => setTimeout(r, ms));++ const load = (kind, url) => {+ if (kind === "script") {+ const el = document.createElement("script");+ el.src = url;+ document.head.appendChild(el);+ } else if (kind === "style") {+ const el = document.createElement("link");+ el.rel = "stylesheet";+ el.href = url;+ document.head.appendChild(el);+ } else if (kind === "img") {+ new Image().src = url;+ } else if (kind === "font") {+ new FontFace("cspProbe", `url(${url})`).load().catch(() => {});+ } else if (kind === "connect") {+ if (url.startsWith("wss:")) new WebSocket(url);+ else fetch(url, { mode: "no-cors" }).catch(() => {});+ } else if (kind === "frame") {+ const el = document.createElement("iframe");+ el.src = url;+ el.style.display = "none";+ document.body.appendChild(el);+ } else if (kind === "media") {+ const el = document.createElement("video");+ el.src = url;+ document.body.appendChild(el);+ el.load();+ } else if (kind === "worker") {+ new Worker(url);+ }+ };++ const expected = [];+ for (const [kind, url, isControl] of probes) {+ try {+ load(kind, url);+ } catch (e) {+ /* cross-origin Worker/WebSocket constructors can throw; CSP still reports first */+ }+ if (isControl) expected.push(url);+ await wait(250);+ }+ await wait(3000);+ document.removeEventListener("securitypolicyviolation", onViolation);++ const reported = new Set(hits.map((h) => h.blocked));+ return {+ // Hosts the policy is supposed to allow but the deployment still blocks. A+ // csp-probe.* entry here means the deployment is not serving that wildcard source.+ unexpected: hits.filter((h) => !h.blocked.includes("csp-control.invalid")),+ // Empty means CSP is live and restrictive. Non-empty means it is not applied at all.+ controlsNotReported: expected.filter(+ (url) => !reported.has(url) && !reported.has(new URL(url).origin)+ ),+ probeCount: probes.length,+ };+};diff --git a/.claude/skills/csp-check/scripts/scan_origins.py b/.claude/skills/csp-check/scripts/scan_origins.pynew file mode 100755index 000000000..0e03470b7--- /dev/null+++ b/.claude/skills/csp-check/scripts/scan_origins.py@@ -0,0 +1,291 @@+#!/usr/bin/env python3+"""List external origins referenced by the frontend and flag ones the CSP never allows.++Sources of truth:+ * the policy in frontend/nginx.conf (see extract_policy.py)+ * every external https:// host that appears in the built JS/CSS++A host that no directive allows is a CSP violation waiting to happen the moment the+code path that fetches it runs. A host that IS allowed somewhere may still violate on+the specific directive that loads it (a style pulled from a script-src-only host, say)+-- run the browser probe from SKILL.md to settle that.++Bare --dist resolves this repo's frontend/build from any directory; pass a path only for+a build elsewhere, and note it is relative to your shell, not to this script:++ python3 scan_origins.py --dist # after `bun run build`+ python3 scan_origins.py --url https://us-central.unstract.com+ python3 scan_origins.py --dist /tmp/other-build --conf /tmp/other-nginx.conf++Exit non-zero on anything that means "this scan did not actually check the policy": a+host in no fetch directive, a path or port outside what its sources allow, a --dist that+is missing or holds no .js, a URL whose index names no bundle, or a failure fetching a+chunk index.html links. A 404 on a path found only inside a bundle string is tolerated --+that is usually a worker path a chunk names but never loads. A scan that inspected+nothing must never look like a pass.+"""++import argparse+import re+import sys+import time+import urllib.error+import urllib.request+from pathlib import Path++from extract_policy import DEFAULT_CONF, parse++URL_RE = re.compile(+ r"https://([a-zA-Z0-9][a-zA-Z0-9.\-]+\.[a-zA-Z]{2,})(:\d+)?(/[^\s\"'`)\\<>]*)?"+)++# Directives that govern navigation or reporting rather than loading a subresource. A+# host listed in one of them cannot be fetched on its strength, so letting it into the+# allowed map would clear references the browser blocks.+NON_FETCH_DIRECTIVES = {+ "form-action",+ "base-uri",+ "frame-ancestors",+ "report-uri",+ "report-to",+ "sandbox",+}+ASSET_RE = re.compile(r"/assets/[A-Za-z0-9_%.\-]+\.(?:js|css)")+RELATIVE_ASSET_RE = re.compile(r"[\"'(](\./[A-Za-z0-9_%.\-]+\.(?:js|css))")++# Hosts that only ever appear as documentation links, XML namespaces or library+# error strings -- they are never fetched, so they need no CSP entry.+IGNORED = {+ "www.w3.org",+ "json-schema.org",+ "momentjs.com",+ "github.com",+ "raw.githubusercontent.com",+ "reactjs.org",+ "react.dev",+ "redux.js.org",+ "redux-toolkit.js.org",+ "react-dnd.github.io",+ "handlebarsjs.com",+ "socket.io",+ "npms.io",+ "example.com",+ "bit.ly",+ "fb.me",+ "yandex.com",+ "sentry.io",+ # posthog-js's built-in US defaults. Inert only because api_host is pinned to+ # https://eu.i.posthog.com/ at frontend/src/index.jsx -- move that pin and these+ # become live hosts the policy blocks, with this list keeping the gate quiet about it.+ "posthog.com",+ "app.posthog.com",+ "us.posthog.com",+ "us.i.posthog.com",+ "us-assets.i.posthog.com",+ "docs.unstract.com",+ "join-slack.unstract.com",+ "billing.stripe.com",+ "checkout.stripe.com",+ "fonts.google.com",+}+++def read_dist(dist: Path) -> dict[str, str]:+ # rglob on a missing directory yields nothing instead of raising, which would turn a+ # mistyped --dist into a clean pass over zero files.+ if not dist.is_dir():+ raise SystemExit(f"--dist {dist} is not a directory (cwd: {Path.cwd()})")+ files = {}+ for pattern in ("*.js", "*.css", "*.html"):+ for path in dist.rglob(pattern):+ files[str(path.relative_to(dist))] = path.read_text(+ encoding="utf-8", errors="ignore"+ )+ # index.html alone is not a build. Without this, an interrupted or wrong-directory+ # build scans one file, finds no external host in it, and reads as a pass.+ if not any(name.endswith(".js") for name in files):+ raise SystemExit(f"--dist {dist} holds no .js bundle -- nothing to check")+ return files+++def fetch(url: str, attempts: int = 3) -> str:+ """One transient TLS or connection error should not redden the whole gate."""+ for attempt in range(1, attempts + 1):+ try:+ return (+ urllib.request.urlopen(url, timeout=20).read().decode("utf-8", "ignore")+ )+ except urllib.error.HTTPError:+ raise # a status code is an answer, not a blip+ except Exception: # noqa: BLE001 - retry, then let the caller record it+ if attempt == attempts:+ raise+ time.sleep(attempt)+ raise AssertionError("unreachable")+++def read_deployment(base_url: str) -> tuple[dict[str, str], list[str]]:+ """Return ({name: body}, [failed paths]). The caller must fail on the failures:+ an auth wall or a CDN 403 empties the scan, and an empty scan finds no gaps.+ """+ base_url = base_url.rstrip("/")+ try:+ index = fetch(base_url + "/")+ except Exception as exc: # noqa: BLE001 - report it as an incomplete scan, not a traceback+ raise SystemExit(f"could not fetch {base_url}/ : {exc}") from exc+ # index.html and the entrypoint-generated runtime config carry origins of their own+ # (the operator-set logo and favicon URLs), and neither is under /assets/.+ files = {"index.html": index}+ failures: list[str] = []+ # A chunk index.html links is real code and has to be there; one found by following+ # RELATIVE_ASSET_RE inside a bundle may be a path the chunk only names as a string (a+ # worker it never loads), so its absence proves nothing. Required-ness is kept in a+ # dict, not carried on the queue: the same path can arrive from both sources, and+ # whichever entry happens to be popped first must not decide how a 404 is treated.+ required_by_path = {path: True for path in ASSET_RE.findall(index)}+ if not required_by_path:+ # A login wall or a redirect serves a perfectly good 200 with no bundle in it.+ raise SystemExit(f"{base_url}/ references no /assets/ chunk -- not the SPA?")+ required_by_path.setdefault("/config/runtime-config.js", False)+ queue = list(required_by_path)+ seen = set()+ while queue:+ path = queue.pop()+ if path in seen:+ continue+ seen.add(path)+ required = required_by_path[path]+ try:+ body = fetch(base_url + path)+ except urllib.error.HTTPError as exc:+ # A 404 is only tolerable on a speculative path. Anything else -- an auth+ # wall, a CDN 403, a 5xx, or any failure on a chunk index.html links --+ # means the scan is missing real code and cannot clear the policy.+ print(f" ! {path}: {exc}", file=sys.stderr)+ if required or exc.code != 404:+ failures.append(f"{path} ({exc.code})")+ continue+ except Exception as exc: # noqa: BLE001 - keep scanning, but record the gap+ print(f" ! {path}: {exc}", file=sys.stderr)+ failures.append(f"{path} ({exc})")+ continue+ files[path.split("/")[-1]] = body+ for match in RELATIVE_ASSET_RE.findall(body):+ discovered = "/assets/" + match[2:]+ required_by_path.setdefault(discovered, False)+ queue.append(discovered)+ return files, failures+++def allowed_sources(directives: dict[str, list[str]]) -> dict[str, set[tuple[str, str]]]:+ """{host pattern: {(port, path prefix)}}. An empty prefix means the whole host.++ `https://www.google.com/recaptcha/` grants exactly that path, not the host -- keeping+ the prefix is what stops the scanner reporting `https://www.google.com/g/collect` as+ covered when the browser would block it. A source with no port grants the default+ port only, which is why the port is kept alongside.+ """+ hosts: dict[str, set[tuple[str, str]]] = {}+ for directive, sources in directives.items():+ if directive in NON_FETCH_DIRECTIVES:+ continue+ for source in sources:+ if not source.startswith("https://"):+ continue+ authority, _, path = source[len("https://") :].partition("/")+ host, _, port = authority.partition(":")+ hosts.setdefault(host, set()).add((port or "443", "/" + path if path else ""))+ return hosts+++def host_matches(ref_host: str, pattern: str) -> bool:+ # CSP wildcards cover subdomains only: *.example.com does not match example.com.+ if pattern.startswith("*."):+ return ref_host.endswith(pattern[1:])+ return ref_host == pattern+++def verdict(+ host: str, port: str, paths: set[str], allowed: dict[str, set[tuple[str, str]]]+) -> tuple[str, str]:+ """('allowed'|'PATH'|'PORT'|'MISSING', the detail that fails)."""+ entries: set[tuple[str, str]] = set()+ for pattern, pattern_entries in allowed.items():+ if host_matches(host, pattern):+ entries |= pattern_entries+ if not entries:+ return "MISSING", ""+ prefixes = {prefix for src_port, prefix in entries if src_port in (port, "*")}+ if not prefixes:+ return "PORT", f":{port}"+ if "" in prefixes:+ return "allowed", ""+ for path in sorted(paths):+ if not any(path.startswith(prefix) for prefix in prefixes):+ return "PATH", path or "/"+ return "allowed", ""+++def main() -> None:+ parser = argparse.ArgumentParser()+ group = parser.add_mutually_exclusive_group(required=True)+ group.add_argument(+ "--dist",+ type=Path,+ nargs="?",+ const=DEFAULT_CONF.parent / "build",+ help="built frontend directory (default: the repo's frontend/build)",+ )+ group.add_argument("--url", help="deployment base URL")+ parser.add_argument("--conf", type=Path, default=DEFAULT_CONF)+ args = parser.parse_args()++ header, directives = parse(args.conf)+ allowed = allowed_sources(directives)+ if args.dist:+ files, failures = read_dist(args.dist), []+ else:+ files, failures = read_deployment(args.url)+ print(f"Scanned {len(files)} bundle files against {header} in {args.conf}\n")++ found: dict[tuple[str, str], dict[str, set[str]]] = {}+ for name, body in files.items():+ for host, port, path in URL_RE.findall(body):+ if host in IGNORED:+ continue+ key = (host, port.lstrip(":") or "443")+ entry = found.setdefault(key, {"files": set(), "paths": set()})+ entry["files"].add(name)+ entry["paths"].add(path)++ problems: list[str] = []+ for host, port in sorted(found):+ mark, detail = verdict(host, port, found[(host, port)]["paths"], allowed)+ label = host if port == "443" else f"{host}:{port}"+ chunks = ", ".join(sorted(found[(host, port)]["files"])[:3])+ suffix = f" <- {detail}" if detail else ""+ print(f" {mark:8} {label:38} {chunks}{suffix}")+ if mark == "MISSING":+ problems.append(f"{label} is in no fetch directive")+ elif mark == "PORT":+ problems.append(f"{label} is on a port no source for {host} allows")+ elif mark == "PATH":+ problems.append(f"{host}{detail} is outside the allowed path on {host}")++ if not files:+ raise SystemExit("\nScanned nothing -- an empty scan is not a pass.")+ if failures:+ print(f"\n{len(failures)} file(s) could not be fetched: {', '.join(failures)}")+ print("The scan is incomplete, so it cannot clear the policy.")+ sys.exit(1)+ if problems:+ print(f"\n{len(problems)} problem(s) the enforcing policy would block:")+ for problem in problems:+ print(f" - {problem}")+ print("Add each to the directive that loads it in frontend/nginx.conf.")+ sys.exit(1)+ print("\nEvery external host referenced by the bundle appears in the policy.")+++if __name__ == "__main__":+ main()diff --git a/.gitignore b/.gitignoreindex e5dbd27a5..cfd13e388 100644--- a/.gitignore+++ b/.gitignore@@ -701,6 +701,7 @@ CLAUDE.md!.claude/skills/.claude/skills/*!.claude/skills/worktree/+!.claude/skills/csp-check/CONTRIBUTION_GUIDE.md.mcp.jsondiff --git a/frontend/nginx.conf b/frontend/nginx.confindex 21dc36d3a..a2e7c695b 100644--- a/frontend/nginx.conf+++ b/frontend/nginx.conf@@ -64,8 +64,19 @@ http {add_header X-Frame-Options "SAMEORIGIN" always;add_header Referrer-Policy "strict-origin-when-cross-origin" always;- # CSP in report-only mode: logs violations to browser console without blocking requests.- add_header Content-Security-Policy-Report-Only "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://cdn.jsdelivr.net https://unpkg.com https://eu.i.posthog.com https://eu-assets.i.posthog.com https://www.googletagmanager.com https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://js.stripe.com https://app.productfruits.com; style-src 'self' 'unsafe-inline'; img-src 'self' data: blob: https://eu.i.posthog.com https://eu-assets.i.posthog.com; font-src 'self' data:; connect-src 'self' blob: wss: https://cdn.jsdelivr.net https://eu.i.posthog.com https://eu-assets.i.posthog.com https://www.google-analytics.com https://api.stripe.com https://app.productfruits.com; frame-src 'self' https://www.google.com/recaptcha/ https://recaptcha.google.com https://js.stripe.com https://hooks.stripe.com; worker-src 'self' blob: https://unpkg.com https://cdn.jsdelivr.net; object-src 'none'; base-uri 'self'; form-action 'self' https://checkout.stripe.com; frame-ancestors 'self'" always;+ # CSP, enforcing: a resource from an origin no directive lists is blocked, not+ # reported. Re-check with .claude/skills/csp-check before a frontend dependency+ # starts loading from a new external host.+ # connect-src carries no `wss:` wildcard: socket.io connects to window.location.origin+ # (frontend/src/helpers/GetStaticData.js getBaseUrl), and 'self' covers same-origin+ # ws/wss per CSP3.+ # The Google analytics hosts are wildcarded because GA4 picks a regional collection+ # host at runtime, so the set is not enumerable. Signals/Ads endpoints+ # (*.g.doubleclick.net, www.google.com/g/collect) are deliberately not granted --+ # enabling either in the GTM UI needs a policy change here.+ # VITE_CUSTOM_LOGO_URL / VITE_FAVICON_PATH are governed by img-src: an absolute URL+ # to an unlisted host is blocked, and both fall back silently.+ add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://cdn.jsdelivr.net https://unpkg.com https://eu.i.posthog.com https://eu-assets.i.posthog.com https://*.googletagmanager.com https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://js.stripe.com https://app.productfruits.com; style-src 'self' 'unsafe-inline' https://cdn.jsdelivr.net https://app.productfruits.com; img-src 'self' data: blob: https://eu.i.posthog.com https://eu-assets.i.posthog.com https://cdn.jsdelivr.net https://app.productfruits.com https://cdn.productfruits.com https://*.googletagmanager.com https://*.google-analytics.com https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://q.stripe.com; font-src 'self' data: https://cdn.jsdelivr.net https://app.productfruits.com https://cdn.productfruits.com; connect-src 'self' blob: https://cdn.jsdelivr.net https://unpkg.com https://eu.i.posthog.com https://eu-assets.i.posthog.com https://*.googletagmanager.com https://*.google-analytics.com https://analytics.google.com https://*.analytics.google.com https://www.google.com/recaptcha/ https://api.stripe.com https://m.stripe.network https://app.productfruits.com https://api.productfruits.com; frame-src 'self' https://www.google.com/recaptcha/ https://recaptcha.google.com https://*.googletagmanager.com https://js.stripe.com https://hooks.stripe.com https://m.stripe.network; media-src 'self' blob: https://cdn.productfruits.com; worker-src 'self' blob: https://unpkg.com https://cdn.jsdelivr.net; object-src 'none'; base-uri 'self'; form-action 'self' https://checkout.stripe.com; frame-ancestors 'self'" always;# Cache-Control from the $cache_control map (immutable for /assets/*,# no-cache otherwise). Declared at server scope alongside the security