Self-hosted AI workspace.
Python
chore(tools): add a read-only ref-to-ref parity audit
Léo10cb8fc669d2
diff --git a/docs/ref-parity-audit.md b/docs/ref-parity-audit.mdnew file mode 100644index 000000000..10f3a756b--- /dev/null+++ b/docs/ref-parity-audit.md@@ -0,0 +1,89 @@+# Ref parity audit++`scripts/ref_parity_audit.py` reports which commits on one git ref left no trace+in another, and which files exist on one and not the other. It is read-only: it+runs `git log`, `git show`, `git diff`, `git grep`, `git ls-tree` and+`git merge-base`, writes nothing to the repository, touches no remote, and does+not import the application package.++## Why it exists++`lab` and the public `dev` line share only the repository's first commit as a+merge base, so `git log lab..dev` lists thousands of commits — nearly all of+which are in fact present on both sides, having arrived under different SHAs. A+plain log tells you nothing about what is actually missing.++The question that matters before `lab` becomes a release is narrower: is there a+fix on the public line that never reached `lab`? This script answers that by+sampling distinctive added lines from each commit and searching the other tree+for them.++## Running it++```bash+git remote add public https://github.com/odysseus-dev/odysseus.git # once+git fetch public dev --no-tags++scripts/ref_parity_audit.py --source public/dev --target lab --since 2026-08-10+```++Roughly 30 seconds for a 100-commit window; it grows linearly, so bound a wide+audit with `--since`. Add `--format json` for a machine-readable report and+`--output PATH` to write it to a file.++| Flag | Effect |+|---|---|+| `--source REF` | The ref whose commits are audited. Required. |+| `--target REF` | The ref searched for traces of them. Required. |+| `--since` / `--until` | Bound the commit range. Both filter **committer** date, which is also the date the report prints. |+| `--traversal linear` | Default. Individual authored commits, merges dropped. Finds a fix that arrived on a side branch. |+| `--traversal first-parent` | One row per merge into the source branch, which reads as one row per merged pull request. |+| `--probes N` | Probe lines sampled per commit, default 4. |+| `--exclude GLOB` | Extra path glob whose lines are not used as probes. Repeatable. |+| `--no-default-excludes` | Drop the built-in vendored / lockfile / binary exclusions. |+| `--top N` | Rows shown per file list, default 50. |+| `--repo PATH` | Repository to run in. Defaults to this checkout. |++## How a verdict is reached++For each commit in `target..source`, the script takes the patch with no context+lines, collects the added lines, drops the ones from vendored code, committed+build output, lockfiles and binaries, and keeps those that are at least 24+characters long and name at least two distinct identifiers. It ranks what is+left by how many distinct identifiers each line carries (length breaks ties),+takes the top `--probes`, and searches the whole target tree for each one with+`git grep --fixed-strings`.++Probes are stripped of leading and trailing whitespace, so a change that was+re-indented on the target still counts as present. The whole target tree is+searched, not the same file, because a ported fix routinely moves.++| Verdict | Meaning |+|---|---|+| **absent** | No probe found anywhere in the target. Treat as a real gap and read the diff. |+| **partial** | Some probes found. **Inconclusive.** A line can be rewritten by a refactor on the target and still be the same change. |+| **present** | Every probe found. The change is almost certainly there in some form. |+| **no-probe** | Nothing to sample: a deletion-only commit, or one touching only excluded paths. No verdict. |++## What is exact and what is a heuristic++**Exact:** the two file-presence lists. They come from `git ls-tree` on both+refs, so a file in "on the source and not the target" is definitely not there.++**Heuristic:** every commit verdict. It samples at most four lines out of a+diff that may be hundreds, and a probe can be absent because the area was+refactored rather than because the change was never made.++The two complement each other in a specific and useful way. A commit that reads+**present** while one of the files it added shows up in the source-only list is+almost always a fix whose production change was reproduced on the target without+its test. The line sampling cannot see that; the presence diff can.++Read the diff before porting anything. The verdicts say where to look, not what+to do.++## Tests++`tests/test_ref_parity_audit.py`. The end-to-end cases build a throwaway+repository with two branches off one root, so the verdicts come from git's own+`grep` and `diff` rather than from a fake.diff --git a/scripts/ref_parity_audit.py b/scripts/ref_parity_audit.pynew file mode 100755index 000000000..5810ff751--- /dev/null+++ b/scripts/ref_parity_audit.py@@ -0,0 +1,586 @@+#!/usr/bin/env python3+"""Read-only audit of what one git ref carries that another does not.++Two long-lived lines that are not merged into each other drift silently. A fix+landed on one of them leaves no mark on the other, and nothing in git tells you+so: the two histories share only a distant merge base, so `git log A..B` lists+thousands of commits whose content is in fact already present on both sides+under different SHAs.++This script answers the question that actually matters at release time -- which+commits on the source ref left *no trace at all* in the target ref -- by+sampling distinctive added lines from each commit and searching the target tree+for them. It also reports the file-level presence diff, which catches the case+the line sampling cannot: a fix whose production change was reproduced on the+target but whose test file was never brought over.++It is read-only. It runs `git log`, `git show`, `git diff`, `git grep`,+`git ls-tree` and `git merge-base`, writes nothing to the repository, touches no+remote, and does not import the Odysseus application package.++Usage:++ scripts/ref_parity_audit.py --source public/dev --target lab --since 2026-08-10++Read `docs/ref-parity-audit.md` before acting on the output: the line sampling+is a heuristic and the report labels which of its verdicts are exact.+"""+import argparse+import fnmatch+import json+import re+import subprocess+import sys+from dataclasses import dataclass, field+from pathlib import Path+from typing import Iterable, Sequence++REPO_ROOT = Path(__file__).resolve().parents[1]++# Paths whose contents are never worth probing: vendored third-party code,+# committed build output, lockfiles and binaries. A distinctive line does not+# exist in a minified bundle, and a lockfile churns on every dependency bump.+DEFAULT_EXCLUDES = (+ "static/lib/*",+ "static/js/editor/build/*",+ "*.min.js",+ "*.min.css",+ "*.map",+ "package-lock.json",+ "*.lock",+ "*.png",+ "*.jpg",+ "*.jpeg",+ "*.gif",+ "*.ico",+ "*.webp",+ "*.svg",+ "*.pdf",+ "*.woff",+ "*.woff2",+ "*.ttf",+ "*.otf",+ "*.mp3",+ "*.mp4",+ "*.wav",+ "*.zip",+ "*.gz",+)++# A probe has to be long enough and carry enough named things to be unlikely to+# appear by coincidence. `return hosts` is in a hundred files; a line naming two+# identifiers over 24 characters is usually unique to the change that added it.+MIN_PROBE_LENGTH = 24+MIN_PROBE_IDENTIFIERS = 2+IDENTIFIER_RE = re.compile(r"[A-Za-z_][A-Za-z0-9_]{2,}")++FIELD_SEP = "\x1f"++VERDICT_ABSENT = "absent"+VERDICT_PARTIAL = "partial"+VERDICT_PRESENT = "present"+VERDICT_NO_PROBE = "no-probe"+++class GitError(RuntimeError):+ """A git invocation failed in a way the audit cannot work around."""+++@dataclass+class Commit:+ sha: str+ author: str+ date: str+ subject: str+ parent_count: int+++@dataclass+class CommitVerdict:+ commit: Commit+ probes: tuple[str, ...]+ found: tuple[str, ...]+ paths: tuple[str, ...]++ @property+ def verdict(self) -> str:+ if not self.probes:+ return VERDICT_NO_PROBE+ if not self.found:+ return VERDICT_ABSENT+ if len(self.found) < len(self.probes):+ return VERDICT_PARTIAL+ return VERDICT_PRESENT+++@dataclass+class Report:+ source: str+ source_sha: str+ target: str+ target_sha: str+ merge_base: str+ since: str | None+ until: str | None+ traversal: str+ probe_limit: int+ verdicts: list[CommitVerdict] = field(default_factory=list)+ source_only_files: tuple[str, ...] = ()+ target_only_files: tuple[str, ...] = ()++ def by_verdict(self, verdict: str) -> list[CommitVerdict]:+ return [v for v in self.verdicts if v.verdict == verdict]+++# --------------------------------------------------------------------------- #+# git plumbing+# --------------------------------------------------------------------------- #+++def run_git(args: Sequence[str], repo: Path) -> str:+ """Run a read-only git command and return stdout, raising on failure."""+ proc = subprocess.run(+ ["git", "-C", str(repo), *args],+ capture_output=True,+ text=True,+ )+ if proc.returncode != 0:+ raise GitError(f"git {' '.join(args)} failed: {proc.stderr.strip()}")+ return proc.stdout+++def resolve_ref(ref: str, repo: Path) -> str:+ return run_git(["rev-parse", "--short=8", ref], repo).strip()+++def merge_base(source: str, target: str, repo: Path) -> str:+ try:+ return run_git(["merge-base", source, target], repo).strip()[:8]+ except GitError:+ # Unrelated histories have no merge base. That is a finding, not a crash.+ return ""+++def list_commits(+ source: str,+ target: str,+ repo: Path,+ since: str | None = None,+ until: str | None = None,+ traversal: str = "linear",+) -> list[Commit]:+ """List commits reachable from `source` but not from `target`.++ `linear` drops merge commits and reports the individual authored commits,+ which is what finds a fix that arrived on a side branch. `first-parent`+ reports one entry per merge into the source branch, which reads as one row+ per merged pull request.+ """+ args = [+ "log",+ "--date=short",+ f"--format=%H{FIELD_SEP}%an{FIELD_SEP}%cd{FIELD_SEP}%p{FIELD_SEP}%s",+ ]+ args.append("--no-merges" if traversal == "linear" else "--first-parent")+ if since:+ args.append(f"--since={since}")+ if until:+ args.append(f"--until={until}")+ args.append(f"{target}..{source}")++ commits = []+ for line in run_git(args, repo).splitlines():+ if not line.strip():+ continue+ sha, author, date, parents, subject = line.split(FIELD_SEP, 4)+ commits.append(+ Commit(+ sha=sha,+ author=author,+ date=date,+ subject=subject,+ parent_count=len(parents.split()) if parents.strip() else 0,+ )+ )+ return commits+++def commit_diff(commit: Commit, repo: Path) -> str:+ """Return the commit's patch with no context lines.++ A merge is diffed against its first parent so the whole merged content is+ visible; `git show` would otherwise print only the conflicting hunks.+ """+ if commit.parent_count > 1:+ return run_git(+ ["diff", "--no-color", "--no-renames", "-U0", f"{commit.sha}^1", commit.sha],+ repo,+ )+ return run_git(+ ["show", "--no-color", "--no-renames", "-U0", "--format=", commit.sha], repo+ )+++def probe_present(probe: str, ref: str, repo: Path) -> bool:+ """Is this exact text anywhere in the ref's tree?++ The whole tree is searched on purpose. The question is whether the change+ left a trace at all, not whether it landed in the same file -- a ported fix+ routinely moves, and the exclusion list only governs where probes come+ from.+ """+ proc = subprocess.run(+ ["git", "-C", str(repo), "grep", "--fixed-strings", "--quiet", "-e", probe, ref],+ capture_output=True,+ text=True,+ )+ if proc.returncode not in (0, 1):+ raise GitError(f"git grep failed for {ref}: {proc.stderr.strip()}")+ return proc.returncode == 0+++def list_tree(ref: str, repo: Path) -> list[str]:+ raw = run_git(["ls-tree", "-r", "-z", "--name-only", ref], repo)+ return [path for path in raw.split("\0") if path]+++# --------------------------------------------------------------------------- #+# probe selection (pure)+# --------------------------------------------------------------------------- #+++def is_excluded(path: str, patterns: Iterable[str]) -> bool:+ name = path.rsplit("/", 1)[-1]+ return any(+ fnmatch.fnmatch(path, pattern) or fnmatch.fnmatch(name, pattern)+ for pattern in patterns+ )+++def added_lines(patch: str, excludes: Iterable[str]) -> list[tuple[str, str]]:+ """Extract `(path, added line)` pairs from a unified diff."""+ results = []+ path = None+ skip = False+ for line in patch.splitlines():+ if line.startswith("+++ "):+ target = line[4:].strip()+ path = None if target == "/dev/null" else target[2:] if target.startswith("b/") else target+ skip = path is None or is_excluded(path, excludes)+ elif line.startswith("--- ") or line.startswith("diff --git "):+ continue+ elif line.startswith("+") and path and not skip:+ results.append((path, line[1:]))+ return results+++def probe_score(text: str) -> int:+ """Rank a candidate probe: distinct named things first, then length."""+ identifiers = set(IDENTIFIER_RE.findall(text))+ return len(identifiers) * 1000 + min(len(text), 400)+++def is_probe_candidate(text: str) -> bool:+ stripped = text.strip()+ if len(stripped) < MIN_PROBE_LENGTH:+ return False+ if "\0" in stripped:+ return False+ return len(set(IDENTIFIER_RE.findall(stripped))) >= MIN_PROBE_IDENTIFIERS+++def pick_probes(lines: Sequence[tuple[str, str]], limit: int) -> list[str]:+ """Pick up to `limit` distinctive stripped lines, highest-scoring first.++ Leading and trailing whitespace is dropped so a re-indented port still+ counts as present. Ties break on first appearance, keeping the output+ stable across runs.+ """+ seen: dict[str, int] = {}+ for index, (_path, text) in enumerate(lines):+ stripped = text.strip()+ if not is_probe_candidate(stripped) or stripped in seen:+ continue+ seen[stripped] = index+ ranked = sorted(seen, key=lambda text: (-probe_score(text), seen[text]))+ return ranked[:limit]+++# --------------------------------------------------------------------------- #+# audit+# --------------------------------------------------------------------------- #+++def audit(+ source: str,+ target: str,+ repo: Path,+ since: str | None = None,+ until: str | None = None,+ traversal: str = "linear",+ probe_limit: int = 4,+ excludes: Sequence[str] = DEFAULT_EXCLUDES,+ progress: bool = False,+) -> Report:+ report = Report(+ source=source,+ source_sha=resolve_ref(source, repo),+ target=target,+ target_sha=resolve_ref(target, repo),+ merge_base=merge_base(source, target, repo),+ since=since,+ until=until,+ traversal=traversal,+ probe_limit=probe_limit,+ )++ commits = list_commits(source, target, repo, since, until, traversal)+ for index, commit in enumerate(commits, start=1):+ if progress:+ print(+ f"\r[{index}/{len(commits)}] {commit.sha[:8]}",+ end="",+ file=sys.stderr,+ flush=True,+ )+ lines = added_lines(commit_diff(commit, repo), excludes)+ probes = pick_probes(lines, probe_limit)+ found = tuple(p for p in probes if probe_present(p, target, repo))+ report.verdicts.append(+ CommitVerdict(+ commit=commit,+ probes=tuple(probes),+ found=found,+ paths=tuple(dict.fromkeys(path for path, _ in lines)),+ )+ )+ if progress:+ print("", file=sys.stderr)++ source_files = {p for p in list_tree(source, repo) if not is_excluded(p, excludes)}+ target_files = {p for p in list_tree(target, repo) if not is_excluded(p, excludes)}+ report.source_only_files = tuple(sorted(source_files - target_files))+ report.target_only_files = tuple(sorted(target_files - source_files))+ return report+++# --------------------------------------------------------------------------- #+# rendering+# --------------------------------------------------------------------------- #+++def _commit_table(verdicts: Sequence[CommitVerdict]) -> list[str]:+ rows = [+ "| Commit | Committed | Author | Probes found | Subject |",+ "|---|---|---|---|---|",+ ]+ for item in verdicts:+ rows.append(+ f"| `{item.commit.sha[:8]}` | {item.commit.date} | {item.commit.author} "+ f"| {len(item.found)}/{len(item.probes)} | {item.commit.subject} |"+ )+ return rows+++def _file_list(paths: Sequence[str], top: int) -> list[str]:+ lines = [f"- `{path}`" for path in paths[:top]]+ if len(paths) > top:+ lines.append(f"- … and {len(paths) - top} more")+ return lines+++def render_markdown(report: Report, top: int = 50) -> str:+ absent = report.by_verdict(VERDICT_ABSENT)+ partial = report.by_verdict(VERDICT_PARTIAL)+ present = report.by_verdict(VERDICT_PRESENT)+ no_probe = report.by_verdict(VERDICT_NO_PROBE)++ window = []+ if report.since:+ window.append(f"since {report.since}")+ if report.until:+ window.append(f"until {report.until}")++ out = [+ "# Ref parity audit",+ "",+ f"Source `{report.source}` @ `{report.source_sha}` → "+ f"target `{report.target}` @ `{report.target_sha}`.",+ f"Merge base `{report.merge_base or 'none (unrelated histories)'}`.",+ f"{len(report.verdicts)} commits on the source and not the target "+ f"({report.traversal} traversal"+ + (", " + ", ".join(window) if window else "")+ + f"), up to {report.probe_limit} probes each.",+ "",+ f"No trace in the target: **{len(absent)}**. "+ f"Partly present: **{len(partial)}**. "+ f"Fully present: **{len(present)}**. "+ f"Unprobeable: **{len(no_probe)}**.",+ "",+ "## Commits with no trace in the target",+ "",+ ]+ out += _commit_table(absent) if absent else ["None."]++ out += [+ "",+ "## Commits only partly present",+ "",+ "A partial verdict is inconclusive, not a finding: a line can move or be "+ "rewritten by a refactor on the target and still be the same change. Read the "+ "diff before porting anything from this table.",+ "",+ ]+ out += _commit_table(partial) if partial else ["None."]++ out += ["", "## Commits with no usable probe", ""]+ if no_probe:+ out += [+ "Deletion-only commits, and commits touching nothing but excluded paths. "+ "The audit has no verdict on these.",+ "",+ ] + _commit_table(no_probe)+ else:+ out.append("None.")++ out += [+ "",+ f"## Files on the source and not the target ({len(report.source_only_files)})",+ "",+ "Exact, not sampled. A file here whose commit is reported fully present is "+ "usually a fix that was reproduced without its test.",+ "",+ ]+ out += _file_list(report.source_only_files, top) if report.source_only_files else ["None."]++ out += [+ "",+ f"## Files on the target and not the source ({len(report.target_only_files)})",+ "",+ ]+ out += _file_list(report.target_only_files, top) if report.target_only_files else ["None."]+ out.append("")+ return "\n".join(out)+++def render_json(report: Report) -> str:+ return json.dumps(+ {+ "source": {"ref": report.source, "sha": report.source_sha},+ "target": {"ref": report.target, "sha": report.target_sha},+ "merge_base": report.merge_base,+ "since": report.since,+ "until": report.until,+ "traversal": report.traversal,+ "probe_limit": report.probe_limit,+ "totals": {+ verdict: len(report.by_verdict(verdict))+ for verdict in (+ VERDICT_ABSENT,+ VERDICT_PARTIAL,+ VERDICT_PRESENT,+ VERDICT_NO_PROBE,+ )+ },+ "commits": [+ {+ "sha": item.commit.sha,+ "date": item.commit.date,+ "author": item.commit.author,+ "subject": item.commit.subject,+ "verdict": item.verdict,+ "probes": list(item.probes),+ "probes_found": list(item.found),+ "paths": list(item.paths),+ }+ for item in report.verdicts+ ],+ "source_only_files": list(report.source_only_files),+ "target_only_files": list(report.target_only_files),+ },+ indent=2,+ sort_keys=True,+ )+++# --------------------------------------------------------------------------- #+# cli+# --------------------------------------------------------------------------- #+++def positive_int(value: str) -> int:+ parsed = int(value)+ if parsed < 1:+ raise argparse.ArgumentTypeError("must be 1 or greater")+ return parsed+++def build_parser() -> argparse.ArgumentParser:+ parser = argparse.ArgumentParser(+ description="Read-only audit of which commits on one ref left no trace in another."+ )+ parser.add_argument("--source", required=True, help="Ref whose commits are audited")+ parser.add_argument("--target", required=True, help="Ref searched for traces of them")+ parser.add_argument("--repo", default=str(REPO_ROOT), help="Repository to run in")+ parser.add_argument("--since", help="Only commits committed on or after this date")+ parser.add_argument("--until", help="Only commits committed on or before this date")+ parser.add_argument(+ "--traversal",+ choices=["linear", "first-parent"],+ default="linear",+ help="linear: individual commits, no merges. first-parent: one row per merge",+ )+ parser.add_argument(+ "--probes", type=positive_int, default=4, help="Probe lines sampled per commit"+ )+ parser.add_argument(+ "--exclude",+ action="append",+ default=[],+ metavar="GLOB",+ help="Extra path glob whose lines are not used as probes (repeatable)",+ )+ parser.add_argument(+ "--no-default-excludes",+ action="store_true",+ help="Drop the built-in vendored/lockfile/binary exclusions",+ )+ parser.add_argument("--format", choices=["markdown", "json"], default="markdown")+ parser.add_argument("--top", type=positive_int, default=50, help="Rows per file list")+ parser.add_argument("--output", help="Write the report here instead of stdout")+ parser.add_argument("--quiet", action="store_true", help="No progress output")+ return parser+++def main(argv: list[str] | None = None) -> int:+ args = build_parser().parse_args(argv)+ excludes = list(args.exclude)+ if not args.no_default_excludes:+ excludes = list(DEFAULT_EXCLUDES) + excludes++ try:+ report = audit(+ source=args.source,+ target=args.target,+ repo=Path(args.repo),+ since=args.since,+ until=args.until,+ traversal=args.traversal,+ probe_limit=args.probes,+ excludes=excludes,+ progress=not args.quiet and sys.stderr.isatty(),+ )+ except GitError as exc:+ print(f"error: {exc}", file=sys.stderr)+ return 2++ text = render_json(report) if args.format == "json" else render_markdown(report, args.top)+ if args.output:+ Path(args.output).write_text(text + "\n", encoding="utf-8")+ else:+ print(text)+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tests/test_ref_parity_audit.py b/tests/test_ref_parity_audit.pynew file mode 100644index 000000000..74ef7924f--- /dev/null+++ b/tests/test_ref_parity_audit.py@@ -0,0 +1,353 @@+"""Behaviour of scripts/ref_parity_audit.py.++The audit answers one question: which commits on a source ref left no trace in a+target ref? The interesting cases are the ones a plain `git log A..B` gets wrong+-- a change that was reproduced on the target under a different SHA must read as+present, and a change whose lines were merely re-indented must not read as+missing. Those are exercised against a real throwaway repository rather than a+fake, because the verdicts come from git's own `grep` and `diff`.+"""+import json+import shutil+import subprocess++import pytest++from tests.helpers.cli_loader import load_script++[email protected](scope="module")+def audit():+ return load_script("ref_parity_audit.py")+++# --------------------------------------------------------------------------- #+# probe selection+# --------------------------------------------------------------------------- #+++def test_short_and_thin_lines_are_not_probes(audit):+ assert not audit.is_probe_candidate("pass")+ assert not audit.is_probe_candidate("})")+ # Long enough, but nothing named in it: a separator comment is in every file.+ assert not audit.is_probe_candidate("# " + "-" * 60)+ assert audit.is_probe_candidate("_hosts_cache_time = now # remember the empty answer")+++def test_probes_are_ranked_by_distinctiveness_and_capped(audit):+ lines = [+ ("src/a.py", " x = 1"),+ ("src/a.py", " parsed_args = json.loads(args) if args else []"),+ ("src/a.py", " if not isinstance(parsed_args, list): raise HTTPException(400)"),+ ("src/a.py", " return parsed_args"),+ ]++ probes = audit.pick_probes(lines, limit=2)++ assert probes == [+ "if not isinstance(parsed_args, list): raise HTTPException(400)",+ "parsed_args = json.loads(args) if args else []",+ ]+++def test_probes_are_stripped_so_a_reindented_port_still_counts(audit):+ probes = audit.pick_probes([("src/a.py", "\t\tself.cache_time = now # keep the empty answer")], limit=4)++ assert probes == ["self.cache_time = now # keep the empty answer"]+++def test_identical_added_lines_yield_one_probe(audit):+ lines = [+ ("src/a.py", " raise HTTPException(400, 'args must be a JSON array')"),+ ("src/b.py", " raise HTTPException(400, 'args must be a JSON array')"),+ ]++ assert len(audit.pick_probes(lines, limit=4)) == 1+++# --------------------------------------------------------------------------- #+# path exclusion and diff parsing+# --------------------------------------------------------------------------- #+++ "path, excluded",+ [+ ("static/lib/marked.js", True),+ ("static/js/chat.min.js", True),+ ("package-lock.json", True),+ ("assets/logo.png", True),+ ("src/model_discovery.py", False),+ ("static/js/chat.js", False),+ ],+)+def test_default_excludes_cover_vendored_and_binary_paths(audit, path, excluded):+ assert audit.is_excluded(path, audit.DEFAULT_EXCLUDES) is excluded+++def test_added_lines_skips_headers_deletions_and_excluded_paths(audit):+ patch = "\n".join(+ [+ "diff --git a/src/a.py b/src/a.py",+ "--- a/src/a.py",+ "+++ b/src/a.py",+ "@@ -1 +1 @@",+ "-old_line_that_is_long_enough()",+ "+new_line_that_is_long_enough()",+ "diff --git a/static/lib/vendor.js b/static/lib/vendor.js",+ "--- a/static/lib/vendor.js",+ "+++ b/static/lib/vendor.js",+ "@@ -1 +1 @@",+ "+vendored_line_that_is_long_enough()",+ "diff --git a/src/gone.py b/src/gone.py",+ "--- a/src/gone.py",+ "+++ /dev/null",+ ]+ )++ assert audit.added_lines(patch, audit.DEFAULT_EXCLUDES) == [+ ("src/a.py", "new_line_that_is_long_enough()")+ ]+++# --------------------------------------------------------------------------- #+# end-to-end against a real repository+# --------------------------------------------------------------------------- #++MARKER_ABSENT = "def confine_agent_to_workspace(root_dir, session_store):"+MARKER_PORTED = "def cache_the_empty_tailscale_answer(now, hosts_cache_time):"+MARKER_KEPT = "def shared_helper_present_on_both_lines(value):"+++def _git(repo, *args):+ subprocess.run(+ [+ "git",+ "-C",+ str(repo),+ "-c",+ "user.name=Audit Fixture",+ "-c",+ "[email protected]",+ "-c",+ "commit.gpgsign=false",+ *args,+ ],+ check=True,+ capture_output=True,+ text=True,+ )+++def two_line_repo(tmp_path):+ """A repository shaped like the real problem: two branches off one root.++ `source` carries three commits. `target` independently reproduces one of+ them (re-indented, so only a stripped probe finds it), never sees another,+ and carries a file of its own plus a vendored file that must stay out of+ the presence diff.+ """+ if not shutil.which("git"):+ pytest.skip("git is not available")++ repo = tmp_path / "repo"+ repo.mkdir()+ _git(repo, "init", "-q", "-b", "root")+ (repo / "base.py").write_text(MARKER_KEPT + "\n return value\n", encoding="utf-8")+ _git(repo, "add", "base.py")+ _git(repo, "commit", "-q", "-m", "chore: base")++ _git(repo, "checkout", "-q", "-b", "source")+ (repo / "confinement.py").write_text(MARKER_ABSENT + "\n return root_dir\n", encoding="utf-8")+ _git(repo, "add", "confinement.py")+ _git(repo, "commit", "-q", "-m", "fix: confine the agent to its workspace")++ (repo / "discovery.py").write_text(MARKER_PORTED + "\n return now\n", encoding="utf-8")+ _git(repo, "add", "discovery.py")+ _git(repo, "commit", "-q", "-m", "fix: cache an empty lookup")++ (repo / "base.py").write_text(MARKER_KEPT + "\n", encoding="utf-8")+ _git(repo, "add", "base.py")+ _git(repo, "commit", "-q", "-m", "refactor: drop the base body")++ _git(repo, "checkout", "-q", "-b", "target", "root")+ # The same fix, written by hand at a different indentation and in another file.+ (repo / "net.py").write_text("class Net:\n " + MARKER_PORTED + "\n return now\n", encoding="utf-8")+ (repo / "target_only.py").write_text("# only on the target line\n", encoding="utf-8")+ (repo / "static").mkdir()+ (repo / "static" / "lib").mkdir()+ (repo / "static" / "lib" / "vendor.min.js").write_text("var a=1;\n", encoding="utf-8")+ _git(repo, "add", "-A")+ _git(repo, "commit", "-q", "-m", "feat: reproduce the lookup cache by hand")++ return repo+++def _verdicts(report):+ return {item.commit.subject: item.verdict for item in report.verdicts}+++def test_absent_ported_and_deletion_only_commits_are_told_apart(audit, two_line_repo):+ report = audit.audit(source="source", target="target", repo=two_line_repo)++ assert _verdicts(report) == {+ "fix: confine the agent to its workspace": audit.VERDICT_ABSENT,+ "fix: cache an empty lookup": audit.VERDICT_PRESENT,+ "refactor: drop the base body": audit.VERDICT_NO_PROBE,+ }+++def test_the_absent_commit_reports_the_probes_it_looked_for(audit, two_line_repo):+ report = audit.audit(source="source", target="target", repo=two_line_repo)+ absent = report.by_verdict(audit.VERDICT_ABSENT)++ assert [item.commit.subject for item in absent] == ["fix: confine the agent to its workspace"]+ assert absent[0].probes == (MARKER_ABSENT,)+ assert absent[0].found == ()+ assert absent[0].paths == ("confinement.py",)+++def test_file_presence_diff_is_exact_and_skips_vendored_paths(audit, two_line_repo):+ report = audit.audit(source="source", target="target", repo=two_line_repo)++ assert report.source_only_files == ("confinement.py", "discovery.py")+ assert report.target_only_files == ("net.py", "target_only.py")+++def test_probes_are_only_drawn_from_included_paths(audit, two_line_repo):+ """Excluding discovery.py leaves its commit with nothing to probe."""+ report = audit.audit(+ source="source",+ target="target",+ repo=two_line_repo,+ excludes=list(audit.DEFAULT_EXCLUDES) + ["discovery.py"],+ )++ assert _verdicts(report)["fix: cache an empty lookup"] == audit.VERDICT_NO_PROBE+++def test_a_probe_found_only_partly_reads_as_partial(audit, two_line_repo):+ # base.py's marker is on both branches; confinement.py's is not. A commit+ # adding both is the partial case.+ _git(two_line_repo, "checkout", "-q", "source")+ (two_line_repo / "mixed.py").write_text(+ MARKER_KEPT + "\n" + MARKER_ABSENT + "\n", encoding="utf-8"+ )+ _git(two_line_repo, "add", "mixed.py")+ _git(two_line_repo, "commit", "-q", "-m", "fix: a change that half landed")++ report = audit.audit(source="source", target="target", repo=two_line_repo, probe_limit=4)++ assert _verdicts(report)["fix: a change that half landed"] == audit.VERDICT_PARTIAL+++def test_first_parent_traversal_diffs_a_merge_against_its_first_parent(audit, two_line_repo):+ """A squash-free merge carries its content in the merge commit itself."""+ _git(two_line_repo, "checkout", "-q", "-b", "side", "root")+ (two_line_repo / "from_side.py").write_text(MARKER_ABSENT + "\n", encoding="utf-8")+ _git(two_line_repo, "add", "from_side.py")+ _git(two_line_repo, "commit", "-q", "-m", "fix: arrived on a side branch")+ _git(two_line_repo, "checkout", "-q", "source")+ _git(two_line_repo, "merge", "-q", "--no-ff", "-m", "Merge commit from fork", "side")++ report = audit.audit(+ source="source", target="target", repo=two_line_repo, traversal="first-parent"+ )+ merge = [v for v in report.verdicts if v.commit.subject == "Merge commit from fork"]++ assert len(merge) == 1+ assert merge[0].commit.parent_count == 2+ assert merge[0].verdict == audit.VERDICT_ABSENT+ assert MARKER_ABSENT in merge[0].probes+++def test_linear_traversal_reports_the_side_branch_commit_not_the_merge(audit, two_line_repo):+ _git(two_line_repo, "checkout", "-q", "-b", "side", "root")+ (two_line_repo / "from_side.py").write_text(MARKER_ABSENT + "\n", encoding="utf-8")+ _git(two_line_repo, "add", "from_side.py")+ _git(two_line_repo, "commit", "-q", "-m", "fix: arrived on a side branch")+ _git(two_line_repo, "checkout", "-q", "source")+ _git(two_line_repo, "merge", "-q", "--no-ff", "-m", "Merge commit from fork", "side")++ subjects = [+ v.commit.subject+ for v in audit.audit(source="source", target="target", repo=two_line_repo).verdicts+ ]++ assert "fix: arrived on a side branch" in subjects+ assert "Merge commit from fork" not in subjects+++def test_date_window_bounds_the_commit_range(audit, two_line_repo):+ empty = audit.audit(source="source", target="target", repo=two_line_repo, until="2000-01-01")+ full = audit.audit(source="source", target="target", repo=two_line_repo, since="2000-01-01")++ assert empty.verdicts == []+ assert len(full.verdicts) == 3+ # The presence diff is not date-filtered: it compares the two trees.+ assert empty.source_only_files == full.source_only_files+++# --------------------------------------------------------------------------- #+# rendering and cli+# --------------------------------------------------------------------------- #+++def test_markdown_report_leads_with_the_absent_commits(audit, two_line_repo):+ text = audit.render_markdown(audit.audit(source="source", target="target", repo=two_line_repo))++ assert "No trace in the target: **1**" in text+ assert "Fully present: **1**" in text+ assert "fix: confine the agent to its workspace" in text+ assert "`confinement.py`" in text+++def test_json_output_is_written_to_the_requested_path(audit, two_line_repo, tmp_path, capsys):+ out = tmp_path / "report.json"++ code = audit.main(+ [+ "--source",+ "source",+ "--target",+ "target",+ "--repo",+ str(two_line_repo),+ "--format",+ "json",+ "--output",+ str(out),+ "--quiet",+ ]+ )++ assert code == 0+ assert capsys.readouterr().out == ""+ payload = json.loads(out.read_text(encoding="utf-8"))+ assert payload["totals"] == {"absent": 1, "no-probe": 1, "partial": 0, "present": 1}+ assert payload["source_only_files"] == ["confinement.py", "discovery.py"]+++def test_an_unknown_ref_exits_two_with_a_message(audit, two_line_repo, capsys):+ code = audit.main(+ [+ "--source",+ "no-such-ref",+ "--target",+ "target",+ "--repo",+ str(two_line_repo),+ "--quiet",+ ]+ )++ assert code == 2+ assert "error:" in capsys.readouterr().err+++def test_probe_count_must_be_positive(audit):+ with pytest.raises(SystemExit):+ audit.build_parser().parse_args(+ ["--source", "a", "--target", "b", "--probes", "0"]+ )