Skip to content

armgit / macro

Public Imported from GitHub
0

Macro is a unified workspace for teams: email, chat, docs, tasks, agents, calls, and CRM — @-linked together with shared AI memory.

chore(git): keep customer PII out of commits and PRs (#8941)

Wolf Mermelsteinb443820a5a4c
diff --git a/.agents/skills/git/SKILL.md b/.agents/skills/git/SKILL.md
index 0f0ac3d653..13f7a66579 100644
--- a/.agents/skills/git/SKILL.md
+++ b/.agents/skills/git/SKILL.md
@@ -1,6 +1,6 @@
---
name: git
-description: Commit, branch, push, and open or update pull requests in this repository. Use before writing a commit message or PR title/body, pushing, stacking PRs, or when asked to "commit", "open a PR", or "push".
+description: Read this before any git or GitHub operation. Use whenever you commit, branch, rebase, push, or open or update a pull request — including writing a commit message or PR title/body. Never include customer PII or secrets in commits, messages, or PR descriptions.
---
# Git and pull requests
@@ -8,6 +8,20 @@ description: Commit, branch, push, and open or update pull requests in this repo
Commit as you go on a feature branch, one logical change per commit, without
asking. Never commit to `main`. Push or open a PR only when the user asks.
+## PII and secrets
+
+Never put customer or third-party PII, or secrets, in a commit, commit
+message, PR title, or PR description. That includes emails, phone numbers,
+physical addresses, API keys, and tokens. Refer to people as "the customer"
+or a placeholder (`[email protected]`).
+
+The pre-commit hook scans staged added lines. Do not skip it (`--no-verify`).
+Do not add lines to `tooling/pii/exceptions` unless a human asks you to.
+
+Before opening or updating a PR, reread the title and body and strip any PII
+that came from logs, traces, or production data. `python3 tooling/pii/scan.py
+--text -` reads a title or body from stdin; `--changed` is what `just check` runs.
+
## Branches
`<user>/<topic>` in kebab-case, for example `wolf/agent-rules`. One concern per
diff --git a/.gitignore b/.gitignore
index 3a5c3d81f0..2293809c89 100644
--- a/.gitignore
+++ b/.gitignore
@@ -59,3 +59,7 @@ infra/local/generated/
# Short-lived credentials generated by google-github-actions/auth.
gha-creds-*.json
+
+# Python bytecode from tooling/pii and other scripts.
+**/__pycache__/
+**/*.pyc
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
new file mode 100644
index 0000000000..6a3026b94a
--- /dev/null
+++ b/.pre-commit-config.yaml
@@ -0,0 +1,10 @@
+# Installed automatically by the Nix shellHook (`prek install --overwrite`).
+repos:
+ - repo: local
+ hooks:
+ - id: pii
+ name: Scan staged diff for PII and secrets
+ language: system
+ entry: python3 tooling/pii/scan.py
+ pass_filenames: false
+ always_run: true
diff --git a/CLAUDE.md b/CLAUDE.md
index 14cce16414..af0cc0e159 100644
--- a/CLAUDE.md
+++ b/CLAUDE.md
@@ -36,7 +36,7 @@ patterns in older code.
| --- | --- |
| `CODING.md` | Coding rules for every language; read before writing code. |
| `AGENTS.md` in a directory | Scope, map, checks, and gotchas for that subtree; a sibling `CLAUDE.md` symlinks to it. |
-| `.agents/skills/` | Task workflows. `.claude/skills` and `.pi/skills` symlink here. |
+| `.agents/skills/` | Task workflows. Cursor loads this directory. `.claude/skills` and `.pi/skills` symlink here. |
| `.agents/rules/` | Path-scoped conventions, loaded when you touch matching files. `.claude/rules` and `.cursor/rules` symlink here. |
| `.agents/agents/` | Subagents (`auditor`, `browser-verifier`). `.claude/agents` symlinks here. |
| `.agents/templates/` | Templates for new rules and `AGENTS.md` files. |
@@ -93,7 +93,8 @@ Real files live in `.agents/`; tool-specific directories only symlink to it.
- Never deploy (`pulumi up`, `wrangler deploy`, workflow dispatch) without
explicit permission for that deploy.
- Never reset databases or wipe stack volumes without approval.
-- Never paste or commit secrets.
+- Never paste or commit secrets, or put customer PII (emails, phones,
+ addresses) in commits, PR titles, or PR descriptions.
- PR titles and bodies, commits, review replies, comments, and docs describe
only the code. Never name people, customers, their companies or products,
places, or support channels and tickets; state any reason in general terms.
diff --git a/docs/development/running-locally.md b/docs/development/running-locally.md
index 5f0f60a480..63afb169cb 100644
--- a/docs/development/running-locally.md
+++ b/docs/development/running-locally.md
@@ -24,12 +24,16 @@ git clone https://github.com/macro-inc/macro.git
cd macro
```
-The Nix shell provides `just`, Cargo, the Rust toolchain, Bun, `wasm-pack`, sqlx, zig, and cargo-zigbuild. You do not need to install these tools separately.
+The Nix shell provides `just`, Cargo, the Rust toolchain, Bun, `wasm-pack`, sqlx, zig, cargo-zigbuild, `prek`, and `gitleaks`. You do not need to install these tools separately.
```bash
nix develop
```
+Entering the shell installs git hooks (`prek install`). A pre-commit hook
+scans staged added lines for customer emails, phones, addresses, and secrets.
+See `tooling/pii/exceptions` and the `git` skill.
+
If `nix develop` fails, enable the experimental features:
```bash
diff --git a/docs/development/rust.md b/docs/development/rust.md
index 91cabddd67..375a69380d 100644
--- a/docs/development/rust.md
+++ b/docs/development/rust.md
@@ -38,7 +38,7 @@ unit tests. Standalone pure-logic crates need no database setup.
| `just build` | Build with the workspace's default Cargo selection, using the SQLx cache. |
| `just rust-check` | Type-check the Rust workspace using the SQLx cache. |
| `cargo fmt` | Format Rust code. |
-| `just check` | Fast, change-scoped format/lint/code-rule gate; not a type check or test runner. |
+| `just check` | Fast, change-scoped format/lint/code-rule/PII gate; not a type check or test runner. |
| `just check full` | Add TypeScript checking and Rust clippy to the gate. |
| `just clippy` | Run workspace Rust lints using the SQLx cache. |
diff --git a/docs/development/style-guide.md b/docs/development/style-guide.md
index dee1f33faa..3130a21bb9 100644
--- a/docs/development/style-guide.md
+++ b/docs/development/style-guide.md
@@ -4,9 +4,10 @@ Code conventions are agent rules in [`.agents/rules/`](../../.agents/rules/). Ag
load each file when they touch matching paths; reviewers and CodeRabbit read the
same files. This page explains the scheme and indexes the rule files.
-**Check locally.** `just check` is the single gate: format, lint, and code rules,
-scoped to your changes against `origin/main`, with every finding printed as
-`file:line [rule-id]` and the command that fixes it. `just check full` adds `tsc`
+**Check locally.** `just check` is the single gate: format, lint, code rules, and
+a PII scan of added lines, scoped to your changes against `origin/main`, with
+every finding printed as `file:line [rule-id]` and the command that fixes it.
+`just check full` adds `tsc`
and clippy. Mechanically enforceable rules live in [`rules/ast-grep/`](../../rules/ast-grep/).
## IDs
diff --git a/nix/cloud-storage.nix b/nix/cloud-storage.nix
index dc4f9b6325..e28e1d68fd 100644
--- a/nix/cloud-storage.nix
+++ b/nix/cloud-storage.nix
@@ -939,6 +939,8 @@
doppler
biome
jq
+ prek
+ gitleaks
# TEMP: nixpkgs stripe-cli 1.43.2 checkPhase fails in the sandbox
# (TestResolvePluginForInstallSucceedsForGAPluginWhenNotLoggedIn).
# stripe-cli
@@ -1076,7 +1078,29 @@
# entries while switching compilers, breaking aws-lc and librdkafka.
# Remove only caches configured by another compiler so ordinary
# shell entry keeps reusable native build state.
- shellHook = pkgs.lib.optionalString isLinux ''
+ shellHook = ''
+ if command -v prek >/dev/null 2>&1; then
+ repo_root="$(git rev-parse --show-toplevel 2>/dev/null || true)"
+ if [ -n "$repo_root" ] && [ -f "$repo_root/.pre-commit-config.yaml" ]; then
+ # Install into .git/hooks. If Cursor Cloud owns core.hooksPath,
+ # unset it only for this install so we do not overwrite the
+ # dispatcher; git still runs our hook via that dispatcher.
+ hooks_path="$(git -C "$repo_root" config --get core.hooksPath || true)"
+ cursor_hooks=""
+ if [ -n "$hooks_path" ] && [ -f "$hooks_path/.cursor-original-hooks-path" ]; then
+ cursor_hooks="$hooks_path"
+ git -C "$repo_root" config --local --unset-all core.hooksPath || true
+ fi
+ if ! (cd "$repo_root" && prek install --overwrite >/dev/null); then
+ echo "warning: prek hook install failed" >&2
+ fi
+ if [ -n "$cursor_hooks" ]; then
+ git -C "$repo_root" config --local core.hooksPath "$cursor_hooks"
+ fi
+ fi
+ fi
+ ''
+ + pkgs.lib.optionalString isLinux ''
target_dir="''${CARGO_TARGET_DIR:-target}"
current_cc="$(type -P "''${CC:-cc}" || true)"
if [ -n "$current_cc" ] && [ -d "$target_dir" ]; then
diff --git a/tooling/just/check.just b/tooling/just/check.just
index 076c158645..206e80becd 100644
--- a/tooling/just/check.just
+++ b/tooling/just/check.just
@@ -6,7 +6,7 @@
# the run, warnings/hints only print). oxlint's only error rule is
# typescript/no-floating-promises, so apps/web files are linted --type-aware.
#
-# just check fast: rustfmt + biome + oxlint + ast-grep (~seconds)
+# just check fast: rustfmt + biome + oxlint + ast-grep + PII (~seconds)
# just check full also runs tsc + clippy (~minutes)
#
# Override the diff base with CHECK_BASE=<ref> (useful in shallow clones,
@@ -100,6 +100,12 @@ check *ARGS:
echo "── ast-grep code rules (rule ids map to docs/development/style-guide.md)"
bunx --yes @ast-grep/[email protected] scan --report-style=short $sg || fail=1
fi
+ echo "── PII scan (added lines vs merge-base)"
+ python3 tooling/pii/scan.py --changed --skip-gitleaks || fail=1
+ if grep -q '^tooling/pii/' <<<"$changed"; then
+ echo "── PII scanner tests"
+ ( \cd tooling/pii && python3 -m unittest test_scan.py ) || fail=1
+ fi
if [ "{{ ARGS }}" = "full" ]; then
if [ -n "$js" ]; then
echo "── tsc (apps/web)"
diff --git a/tooling/pii/exceptions b/tooling/pii/exceptions
new file mode 100644
index 0000000000..76fa186c4a
--- /dev/null
+++ b/tooling/pii/exceptions
@@ -0,0 +1,10 @@
+# Allowlist for tooling/pii/scan.py.
+#
+# Only humans add entries here, unless they ask an agent to. Each non-comment
+# line is one exception:
+#
+# [email protected] exact finding, any file
+# crates/foo/src/lib.rs:555-0100 finding in that path only
+# @example-corp.com any email at that domain
+#
+# Prefer placeholders ([email protected]) over exceptions.
diff --git a/tooling/pii/scan.py b/tooling/pii/scan.py
new file mode 100755
index 0000000000..e4e8f2b357
--- /dev/null
+++ b/tooling/pii/scan.py
@@ -0,0 +1,334 @@
+#!/usr/bin/env python3
+"""Scan added lines for customer PII and high-signal secrets.
+
+Default mode reads `git diff --cached` so the pre-commit hook stays on the
+staged hunk only. `--changed` covers the same scope as `just check`. `--text`
+scans a PR title/body or other free text.
+
+Only humans should add `tooling/pii/exceptions` entries, unless they ask an
+agent to.
+"""
+
+from __future__ import annotations
+
+import argparse
+import os
+import re
+import shutil
+import subprocess
+import sys
+from collections.abc import Iterable
+from dataclasses import dataclass
+from pathlib import Path
+
+ROOT = Path(__file__).resolve().parents[2]
+EXCEPTIONS_PATH = Path(__file__).resolve().with_name("exceptions")
+MAX_FILE_BYTES = 1_000_000
+
+SAFE_DOMAINS = frozenset(
+ {
+ "example.com",
+ "example.org",
+ "example.net",
+ "example.edu",
+ "macro.com",
+ "localhost",
+ "invalid",
+ "test",
+ "local",
+ "users.noreply.github.com",
+ }
+)
+SAFE_TLDS = frozenset({"example", "test", "localhost", "invalid", "local"})
+SKIP_PATH_PREFIXES = (
+ ".sqlx/",
+ "packages/sdk/generated/",
+)
+SKIP_PATH_SUFFIXES = (
+ "bun.lock",
+ "package-lock.json",
+ "Cargo.lock",
+ "pnpm-lock.yaml",
+ "flake.lock",
+)
+SKIP_PATHS = frozenset(
+ {
+ "tooling/pii/exceptions",
+ "tooling/pii/test_scan.py",
+ }
+)
+
+EMAIL_RE = re.compile(
+ r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"
+)
+PHONE_RE = re.compile(
+ r"(?<!\w)(?:\+\d{1,3}[\s.-]*)?(?:\(?\d{3}\)?[\s.-]\d{3}[\s.-]\d{4}|\+\d{8,15})(?!\w)"
+)
+ADDRESS_RE = re.compile(
+ r"\b\d{1,5}[A-Za-z]?\s+(?:[A-Z][A-Za-z0-9.'-]+\s+){0,3}"
+ r"(?:Street|St\.?|Avenue|Ave\.?|Road|Rd\.?|Boulevard|Blvd\.?|"
+ r"Lane|Ln\.?|Drive|Dr\.?|Court|Ct\.?|Way|Place|Pl\.?)\b"
+)
+SECRET_PATTERNS = (
+ ("secret", re.compile(r"-----BEGIN [A-Z0-9 ]{0,40}PRIVATE KEY-----")),
+ ("secret", re.compile(r"\bAKIA[0-9A-Z]{16}\b")),
+ ("secret", re.compile(r"\b(?:ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{36}\b")),
+ ("secret", re.compile(r"\bgithub_pat_[A-Za-z0-9_]{20,}\b")),
+ ("secret", re.compile(r"\bxox[baprs]-[A-Za-z0-9-]{10,}\b")),
+)
+
+
+@dataclass(frozen=True)
+class Finding:
+ path: str
+ line_no: int
+ kind: str
+ value: str
+
+ def render(self) -> str:
+ location = self.path if self.line_no <= 0 else f"{self.path}:{self.line_no}"
+ return f"{location}: {self.kind} [redacted]"
+
+
+@dataclass
+class Exceptions:
+ values: set[str]
+ path_values: set[str]
+ domains: set[str]
+
+
+def load_exceptions(path: Path = EXCEPTIONS_PATH) -> Exceptions:
+ values: set[str] = set()
+ path_values: set[str] = set()
+ domains: set[str] = set()
+ if not path.is_file():
+ return Exceptions(values, path_values, domains)
+ for raw in path.read_text(encoding="utf-8").splitlines():
+ line = raw.strip()
+ if not line or line.startswith("#"):
+ continue
+ lowered = line.lower()
+ if lowered.startswith("@") and "/" not in lowered:
+ domains.add(lowered[1:])
+ continue
+ if ":" in line:
+ file_path, _, value = line.partition(":")
+ if "/" in file_path or file_path.endswith((".rs", ".ts", ".tsx", ".py", ".md")):
+ path_values.add(f"{file_path}:{value}".lower())
+ continue
+ values.add(lowered)
+ return Exceptions(values, path_values, domains)
+
+
+def email_domain(value: str) -> str:
+ return value.rsplit("@", 1)[-1].lower().rstrip(".")
+
+
+def is_safe_email(value: str) -> bool:
+ domain = email_domain(value)
+ if domain in SAFE_DOMAINS:
+ return True
+ tld = domain.rsplit(".", 1)[-1]
+ return tld in SAFE_TLDS or domain.endswith(".example.com")
+
+
+def is_excepted(path: str, value: str, exceptions: Exceptions) -> bool:
+ lowered = value.lower()
+ if lowered in exceptions.values:
+ return True
+ if f"{path}:{value}".lower() in exceptions.path_values:
+ return True
+ if "@" in value:
+ domain = email_domain(value)
+ if domain in exceptions.domains:
+ return True
+ return False
+
+
+def skip_path(path: str) -> bool:
+ if path in SKIP_PATHS:
+ return True
+ if path.startswith(SKIP_PATH_PREFIXES) or path.endswith(SKIP_PATH_SUFFIXES):
+ return True
+ return False
+
+
+def scan_line(path: str, line_no: int, line: str, exceptions: Exceptions) -> list[Finding]:
+ findings: list[Finding] = []
+
+ def add(kind: str, value: str) -> None:
+ if not is_excepted(path, value, exceptions):
+ findings.append(Finding(path, line_no, kind, value))
+
+ for match in EMAIL_RE.finditer(line):
+ value = match.group(0)
+ if not is_safe_email(value):
+ add("email", value)
+ for match in PHONE_RE.finditer(line):
+ add("phone", re.sub(r"\s+", " ", match.group(0)).strip())
+ for match in ADDRESS_RE.finditer(line):
+ add("address", match.group(0))
+ for kind, pattern in SECRET_PATTERNS:
+ for match in pattern.finditer(line):
+ add(kind, match.group(0))
+ return findings
+
+
+def parse_unified_diff(diff: str) -> list[Finding]:
+ return list(iter_diff_findings(diff, load_exceptions()))
+
+
+def iter_diff_findings(diff: str, exceptions: Exceptions) -> Iterable[Finding]:
+ path = ""
+ new_line_no = 0
+ in_hunk = False
+ for raw in diff.splitlines():
+ if raw.startswith("diff --git "):
+ path = ""
+ in_hunk = False
+ continue
+ if not in_hunk and raw.startswith("+++ "):
+ plus_path = raw[4:]
+ if plus_path == "/dev/null":
+ path = ""
+ continue
+ path = plus_path[2:] if plus_path.startswith("b/") else plus_path
+ continue
+ if raw.startswith("@@"):
+ in_hunk = True
+ match = re.search(r"\+(\d+)", raw)
+ new_line_no = int(match.group(1)) if match else 0
+ continue
+ if not path or skip_path(path) or raw.startswith("Binary files "):
+ continue
+ if in_hunk and raw.startswith("+"):
+ yield from scan_line(path, new_line_no, raw[1:], exceptions)
+ new_line_no += 1
+ elif raw.startswith("-") and not raw.startswith("---"):
+ continue
+ else:
+ new_line_no += 1
+
+
+def git(*args: str) -> subprocess.CompletedProcess[str]:
+ return subprocess.run(
+ ["git", *args],
+ cwd=ROOT,
+ check=False,
+ text=True,
+ capture_output=True,
+ )
+
+
+def staged_diff() -> str:
+ result = git("diff", "--cached", "-U0", "--diff-filter=AMR", "--no-color")
+ result.check_returncode()
+ return result.stdout
+
+
+def merge_base() -> str:
+ result = git("merge-base", "HEAD", "origin/main")
+ return result.stdout.strip()
+
+
+def changed_diff() -> str:
+ base = os.environ.get("CHECK_BASE") or merge_base() or "HEAD"
+ args = ["diff", "-U0", "--diff-filter=AMR", "--no-color", base]
+ result = git(*args)
+ result.check_returncode()
+ return result.stdout
+
+
+def untracked_findings(exceptions: Exceptions) -> list[Finding]:
+ result = git("ls-files", "--others", "--exclude-standard")
+ findings: list[Finding] = []
+ for path in result.stdout.splitlines():
+ if skip_path(path):
+ continue
+ file_path = ROOT / path
+ if not file_path.is_file():
+ continue
+ try:
+ if file_path.stat().st_size > MAX_FILE_BYTES:
+ continue
+ text = file_path.read_text(encoding="utf-8")
+ except (OSError, UnicodeDecodeError):
+ continue
+ if "\0" in text:
+ continue
+ for line_no, line in enumerate(text.splitlines(), start=1):
+ findings.extend(scan_line(path, line_no, line, exceptions))
+ return findings
+
+
+def scan_text(text: str, exceptions: Exceptions, path: str = "<text>") -> list[Finding]:
+ findings: list[Finding] = []
+ for line_no, line in enumerate(text.splitlines() or [text], start=1):
+ findings.extend(scan_line(path, line_no, line, exceptions))
+ return findings
+
+
+def run_gitleaks() -> int:
+ gitleaks = shutil.which("gitleaks")
+ if not gitleaks:
+ return 0
+ result = subprocess.run(
+ [gitleaks, "protect", "--staged", "--redact", "--no-banner"],
+ cwd=ROOT,
+ check=False,
+ )
+ return result.returncode
+
+
+def report(findings: list[Finding]) -> int:
+ if not findings:
+ return 0
+ print("PII or secret found in added lines:", file=sys.stderr)
+ for finding in findings:
+ print(finding.render(), file=sys.stderr)
+ print(
+ "Redact the value, or ask a human to add an exception in "
+ "tooling/pii/exceptions if it is intentional.",
+ file=sys.stderr,
+ )
+ return 1
+
+
+def main(argv: list[str] | None = None) -> int:
+ parser = argparse.ArgumentParser(description=__doc__)
+ parser.add_argument(
+ "--changed",
+ action="store_true",
+ help="Scan added lines vs origin/main (and untracked files), like just check.",
+ )
+ parser.add_argument(
+ "--text",
+ metavar="TEXT",
+ help="Scan a PR title/body or other free text. Use - to read stdin.",
+ )
+ parser.add_argument(
+ "--skip-gitleaks",
+ action="store_true",
+ help="Skip the optional gitleaks pass.",
+ )
+ args = parser.parse_args(argv)
+ exceptions = load_exceptions()
+
+ if args.text is not None:
+ text = sys.stdin.read() if args.text == "-" else args.text
+ return report(scan_text(text, exceptions))
+
+ if args.changed:
+ findings = list(iter_diff_findings(changed_diff(), exceptions))
+ findings.extend(untracked_findings(exceptions))
+ return report(findings)
+
+ findings = list(iter_diff_findings(staged_diff(), exceptions))
+ status = report(findings)
+ if status != 0 or args.skip_gitleaks:
+ return status
+ return run_gitleaks()
+
+
+if __name__ == "__main__":
+ sys.exit(main())
diff --git a/tooling/pii/test_scan.py b/tooling/pii/test_scan.py
new file mode 100644
index 0000000000..4323d02d01
--- /dev/null
+++ b/tooling/pii/test_scan.py
@@ -0,0 +1,167 @@
+"""PII scanner coverage for added-line detection and exceptions."""
+
+from __future__ import annotations
+
+import tempfile
+import subprocess
+import unittest
+from pathlib import Path
+from unittest.mock import patch
+
+import scan
+from scan import (
+ Exceptions,
+ iter_diff_findings,
+ load_exceptions,
+ scan_line,
+ scan_text,
+)
+
+
+class ScanLineTest(unittest.TestCase):
+ def test_flags_customer_email_and_allows_reserved_domains(self):
+ empty = Exceptions(set(), set(), set())
+ findings = scan_line(
+ "note.md",
+ 1,
+ empty,
+ )
+ self.assertEqual([finding.value for finding in findings], ["[email protected]"])
+
+ def test_allows_macro_and_test_tlds(self):
+ empty = Exceptions(set(), set(), set())
+ self.assertEqual(scan_line("note.md", 1, line, empty), [])
+
+ def test_flags_phone_and_address_not_iso_dates(self):
+ empty = Exceptions(set(), set(), set())
+ line = "Call +1 415-555-0100 at 100 Market Street on 2026-10-27"
+ values = {finding.kind: finding.value for finding in scan_line("note.md", 1, line, empty)}
+ self.assertEqual(values["phone"], "+1 415-555-0100")
+ self.assertEqual(values["address"], "100 Market Street")
+ self.assertNotIn("2026-10-27", values.values())
+
+ def test_flags_private_key_and_github_token(self):
+ empty = Exceptions(set(), set(), set())
+ key = "-----BEGIN " + "RSA PRIVATE KEY-----"
+ token = "ghp" + "_" + ("a" * 36)
+ findings = scan_line("leak.txt", 3, f"{key} {token}", empty)
+ self.assertEqual({finding.kind for finding in findings}, {"secret"})
+ self.assertEqual({finding.value for finding in findings}, {key, token})
+
+ def test_exceptions_cover_value_path_and_domain(self):
+ with tempfile.TemporaryDirectory() as tmp:
+ path = Path(tmp) / "exceptions"
+ path.write_text(
+ "\n".join(
+ [
+ "# comment",
+ "+1 415-555-0100",
+ "docs/guide.md:100 Market Street",
+ "@customers-inc.com",
+ ]
+ )
+ + "\n",
+ encoding="utf-8",
+ )
+ exceptions = load_exceptions(path)
+ self.assertEqual(
+ scan_line("any.rs", 1, "pager +1 415-555-0100 and [email protected]", exceptions),
+ [],
+ )
+ self.assertEqual(
+ scan_line("docs/guide.md", 1, "Office at 100 Market Street", exceptions),
+ [],
+ )
+ leftover = scan_line("other.md", 1, "Office at 100 Market Street", exceptions)
+ self.assertEqual(leftover[0].kind, "address")
+
+
+class DiffAndTextTest(unittest.TestCase):
+ def test_only_added_lines_in_a_hunk_are_scanned(self):
+ empty = Exceptions(set(), set(), set())
+ diff = """diff --git a/note.md b/note.md
+--- a/note.md
++++ b/note.md
+@@ -1,2 +1,3 @@
+ context
+"""
+ findings = list(iter_diff_findings(diff, empty))
+ self.assertEqual([(finding.line_no, finding.value) for finding in findings], [(2, "[email protected]")])
+
+ def test_skips_deleted_files_and_lockfiles(self):
+ empty = Exceptions(set(), set(), set())
+ diff = """diff --git a/bun.lock b/bun.lock
+--- a/bun.lock
++++ b/bun.lock
+@@ -1 +1 @@
+diff --git a/gone.md b/gone.md
+--- a/gone.md
++++ /dev/null
+@@ -1 +0,0 @@
+"""
+ self.assertEqual(list(iter_diff_findings(diff, empty)), [])
+
+ def test_text_mode_flags_pr_body_pii(self):
+ empty = Exceptions(set(), set(), set())
+ findings = scan_text(
+ "Billed [email protected] immediately",
+ empty,
+ )
+ self.assertEqual(findings[0].kind, "email")
+ self.assertEqual(findings[0].path, "<text>")
+
+ def test_added_content_cannot_be_parsed_as_a_file_header(self):
+ diff = """diff --git a/note.md b/note.md
+--- a/note.md
++++ b/note.md
+@@ -0,0 +1,2 @@
+"""
+ findings = list(iter_diff_findings(diff, Exceptions(set(), set(), set())))
+ self.assertEqual([(f.path, f.line_no) for f in findings], [("note.md", 1), ("note.md", 2)])
+
+ def test_reports_do_not_expose_matched_values(self):
+ findings = scan_text("[email protected]", Exceptions(set(), set(), set()))
+ self.assertEqual(findings[0].render(), "<text>:1: email [redacted]")
+
+
+class GitDiffTest(unittest.TestCase):
+ def test_staged_rename_is_scanned_without_a_remote_base(self):
+ with tempfile.TemporaryDirectory() as tmp, patch.dict(scan.os.environ, {"CHECK_BASE": ""}):
+ root = Path(tmp)
+ subprocess.run(["git", "init", "-q", tmp], check=True)
+ original = root / "original.md"
+ original.write_text("safe content\n" * 20, encoding="utf-8")
+ subprocess.run(["git", "-C", tmp, "add", "."], check=True)
+ subprocess.run([
+ "git", "-C", tmp, "-c", "user.name=Test", "-c", "[email protected]",
+ "-c", "core.hooksPath=/dev/null", "commit", "-qm", "initial",
+ ], check=True)
+ renamed = root / "renamed.md"
+ original.rename(renamed)
+ with renamed.open("a", encoding="utf-8") as file:
+ file.write("[email protected]\n")
+ subprocess.run(["git", "-C", tmp, "add", "-A"], check=True)
+ with patch.object(scan, "ROOT", root):
+ for diff in (scan.staged_diff(), scan.changed_diff()):
+ findings = list(iter_diff_findings(diff, Exceptions(set(), set(), set())))
+ self.assertEqual([(f.path, f.kind) for f in findings], [("renamed.md", "email")])
+
+ def test_git_errors_fail_the_scan(self):
+ with tempfile.TemporaryDirectory() as tmp, patch.object(scan, "ROOT", Path(tmp)):
+ with self.assertRaises(subprocess.CalledProcessError):
+ scan.staged_diff()
+ with self.assertRaises(subprocess.CalledProcessError):
+ scan.changed_diff()
+
+
+if __name__ == "__main__":
+ unittest.main()