Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

fix(change-risk): repo-relative priority and honest driver labels (#465) (#469)

Raghav Chamadiya57e0ade1c920
diff --git a/docs/CHANGE_RISK.md b/docs/CHANGE_RISK.md
index d9a0a6ec9..a7328f713 100644
--- a/docs/CHANGE_RISK.md
+++ b/docs/CHANGE_RISK.md
@@ -36,6 +36,34 @@ log-compressed features — `logit = intercept + Σ coefᵢ·zᵢ` — so every
push on the risk is exact and reported as an attributable driver (the same
linear / per-finding-attributable contract the file health score holds).
+## How to read the result
+
+The headline signal is **repo-relative**. The raw 0–10 logistic probability is
+anchored to the offline calibration corpus, so on a repo whose typical commit is
+large the diff-size term dominates and the absolute band skews high — two-thirds
+of commits can read "high" while ranking perfectly normally for *that* repo. The
+*ranking* is sound; the absolute band is not portable.
+
+So the surfaces lead with where the change sits in its **own repo's**
+distribution:
+
+- **Review priority** — `Below typical` / `Typical` / `Elevated` (terciles of
+ the repo's own commit-risk distribution). This is the signal to triage on.
+- **Percentile** — "riskier than N% of this repo's commits".
+- **Raw model score** (0–10) — kept for transparency but shown as a secondary,
+ clearly corpus-anchored number, not the thing to act on.
+
+Each **driver** is reported relative to *the model's baseline commit* (the
+calibration-corpus mean), not this repo — so a `+19 / −1` change can legitimately
+read "more lines added than baseline" while still ranking `Below typical` for a
+repo of large commits. The signed contribution and colour (red raised the raw
+score, green lowered it) carry the direction; the label only states the
+feature's standing, never an absolute verdict.
+
+The `repowise risk` CLI samples the repo's recent commits live (`--baseline`,
+default 200) to compute this percentile; in the web UI it is precomputed from the
+indexed commit history.
+
## Calibration & accuracy
Constants are learned offline against the defect corpus (AG-SZZ bug-inducing
diff --git a/docs/CLI_REFERENCE.md b/docs/CLI_REFERENCE.md
index 56e8f74c5..0b30b4526 100644
--- a/docs/CLI_REFERENCE.md
+++ b/docs/CLI_REFERENCE.md
@@ -308,9 +308,16 @@ repowise dead-code resolve <id> # mark resolved / false positive
### `repowise risk [REVSPEC]`
Just-in-time change-risk scoring for a commit or diff range. Scores the defect
-risk of a change (0–10) from the same calibrated signals the code-health layer
-uses — no LLM calls. `REVSPEC` defaults to `HEAD`; pass a `base..head` range to
-score a whole branch / PR as one change.
+risk of a change from the same calibrated signals the code-health layer uses —
+no LLM calls. `REVSPEC` defaults to `HEAD`; pass a `base..head` range to score a
+whole branch / PR as one change.
+
+The headline is **repo-relative**: the change's percentile and review priority
+(`Below typical` / `Typical` / `Elevated`) within the repo's own recent commits,
+sampled live. The raw 0–10 model score is still shown, but as a secondary,
+corpus-anchored number (it skews high on repos whose typical commit is large, so
+the percentile is the signal to act on). Each risk driver is reported relative
+to the model's baseline commit, not this repo.
**Options:**
@@ -318,6 +325,7 @@ score a whole branch / PR as one change.
|------|-------------|
| `--path` | Path to the git repository (default: current directory) |
| `--ext` | Comma-separated file suffixes to count (e.g. `.py` or `.ts,.tsx`) |
+| `--baseline` | Recent commits to sample for the repo-relative percentile (default 200; `0` shows only the absolute calibrated band) |
| `--format` | Output format: `table` (default) or `json` |
```bash
diff --git a/packages/cli/src/repowise/cli/commands/risk_cmd.py b/packages/cli/src/repowise/cli/commands/risk_cmd.py
index 7cbc7e514..78bfed5e5 100644
--- a/packages/cli/src/repowise/cli/commands/risk_cmd.py
+++ b/packages/cli/src/repowise/cli/commands/risk_cmd.py
@@ -14,12 +14,89 @@ Examples:
from __future__ import annotations
import json
+import subprocess
+from dataclasses import replace
import click
from rich.table import Table
from repowise.cli.helpers import console, err_console
+# Repo-relative tercile wording — mirrors the web UI's PriorityBadge so the same
+# commit reads the same in both surfaces.
+_PRIORITY_LABEL = {"high": "Elevated", "moderate": "Typical", "low": "Below typical"}
+_PRIORITY_COLOR = {"high": "yellow", "moderate": "dim", "low": "green"}
+_PRIORITY_LEAD = {
+ "low": "Lower risk than a typical commit in this repo",
+ "moderate": "About as risky as a typical commit in this repo",
+ "high": "Riskier than most commits in this repo",
+}
+
+# Below this many sampled commits a percentile isn't worth showing — fall back
+# to the absolute calibrated band instead.
+_MIN_BASELINE = 8
+
+
+def _ordinal(n: int) -> str:
+ """1 -> '1st', 2 -> '2nd', 93 -> '93rd', 11 -> '11th'."""
+ suffix = "th" if 10 <= n % 100 <= 20 else {1: "st", 2: "nd", 3: "rd"}.get(n % 10, "th")
+ return f"{n}{suffix}"
+
+
+def _baseline_scores(
+ repo_path: str,
+ anchor: str,
+ limit: int,
+ extensions: tuple[str, ...],
+ exclude: str,
+) -> list[float]:
+ """Score the repo's recent commits to build a local risk distribution.
+
+ One ``git log --numstat`` call (no per-commit author lookup), so it stays
+ cheap enough for a pre-merge gate. Experience is left unknown for the
+ baseline; the target is ranked with experience likewise unknown, so the
+ comparison is like-with-like — a diff-shape percentile within this repo.
+ """
+ from repowise.core.analysis.change_risk import (
+ features_from_file_changes,
+ score_change,
+ )
+
+ out = subprocess.run(
+ ["git", "log", f"-n{limit}", "--no-merges", "--format=%x1e%H", "--numstat", anchor],
+ cwd=repo_path,
+ capture_output=True,
+ text=True,
+ encoding="utf-8",
+ errors="replace",
+ ).stdout
+
+ scores: list[float] = []
+ for block in out.split("\x1e"):
+ lines = block.strip().split("\n")
+ if not lines or not lines[0]:
+ continue
+ sha, rows = lines[0].strip(), lines[1:]
+ # Drop the target itself from its own baseline (short or full sha).
+ if exclude and (sha.startswith(exclude) or exclude.startswith(sha)):
+ continue
+ changes: list[tuple[str, int, int]] = []
+ for row in rows:
+ parts = row.split("\t")
+ if len(parts) != 3:
+ continue
+ a_raw, d_raw, path = parts
+ if extensions and not path.endswith(extensions):
+ continue
+ a = int(a_raw) if a_raw.isdigit() else 0
+ d = int(d_raw) if d_raw.isdigit() else 0
+ changes.append((path, a, d))
+ if not changes:
+ continue
+ feats = features_from_file_changes(changes, exp=None)
+ scores.append(score_change(feats).score)
+ return scores
+
@click.command("risk")
@click.argument("revspec", required=False, default="HEAD")
@@ -37,6 +114,14 @@ from repowise.cli.helpers import console, err_console
help="Comma-separated file suffixes to count (e.g. '.py' or '.ts,.tsx'). "
"Default: count every changed file.",
)
+ "--baseline",
+ "baseline",
+ default=200,
+ type=click.IntRange(min=0),
+ help="Sample this many recent commits to rank the change within the repo "
+ "(0 disables; shows only the absolute calibrated band).",
+)
@click.option(
"--format",
"fmt",
@@ -44,7 +129,9 @@ from repowise.cli.helpers import console, err_console
type=click.Choice(["table", "json"]),
help="Output format.",
)
-def risk_command(revspec: str, repo_path: str, ext: str | None, fmt: str) -> None:
+def risk_command(
+ revspec: str, repo_path: str, ext: str | None, baseline: int, fmt: str
+) -> None:
"""Score the defect risk of a change (commit or ``base..head`` range)."""
from repowise.core.analysis.change_risk import (
extract_commit_features,
@@ -76,6 +163,30 @@ def risk_command(revspec: str, repo_path: str, ext: str | None, fmt: str) -> Non
risk = score_change(features)
+ # Repo-relative ranking: where this change sits in the repo's own recent
+ # distribution. The raw score is corpus-anchored and skews high on repos
+ # whose typical commit is large; the percentile is the portable signal.
+ percentile: float | None = None
+ priority: str | None = None
+ if baseline:
+ if ".." in revspec:
+ _, _, anchor = revspec.partition("..")
+ anchor, exclude = anchor or "HEAD", ""
+ else:
+ anchor, exclude = revspec, features.ref
+ if fmt == "table":
+ status.print(f"[dim]Sampling up to {baseline} recent commits…[/dim]")
+ scores = _baseline_scores(repo_path, anchor, baseline, extensions, exclude)
+ if len(scores) >= _MIN_BASELINE:
+ from repowise.core.analysis.change_risk import RiskNormalizer
+
+ normalizer = RiskNormalizer.from_scores(scores)
+ # Rank with experience unknown, matching the baseline (diff-shape
+ # percentile within the repo) — keeps the comparison like-with-like.
+ rank_score = score_change(replace(features, exp=None)).score
+ percentile = normalizer.percentile(rank_score)
+ priority = normalizer.priority(rank_score)
+
if fmt == "json":
click.echo(
json.dumps(
@@ -84,6 +195,8 @@ def risk_command(revspec: str, repo_path: str, ext: str | None, fmt: str) -> Non
"score": risk.score,
"probability": round(risk.probability, 4),
"level": risk.level,
+ "risk_percentile": round(percentile, 1) if percentile is not None else None,
+ "review_priority": priority,
"is_fix": features.is_fix,
"features": {
"la": features.la,
@@ -109,11 +222,21 @@ def risk_command(revspec: str, repo_path: str, ext: str | None, fmt: str) -> Non
)
return
- color = {"high": "red", "moderate": "yellow", "low": "green"}[risk.level]
- console.print(
- f"\n[bold]Change risk[/bold] for [cyan]{features.ref}[/cyan]: "
- f"[{color}]{risk.score:.1f}/10[/{color}] ({risk.level})"
- )
+ if percentile is not None and priority is not None:
+ pcolor = _PRIORITY_COLOR[priority]
+ console.print(
+ f"\n[bold]Change risk[/bold] for [cyan]{features.ref}[/cyan]: "
+ f"[{pcolor}]{_PRIORITY_LABEL[priority]}[/{pcolor}] · "
+ f"{_ordinal(round(percentile))} percentile of recent commits"
+ )
+ else:
+ # No usable baseline (shallow repo, --baseline 0): fall back to the
+ # absolute calibrated band, labelled honestly as such.
+ color = {"high": "red", "moderate": "yellow", "low": "green"}[risk.level]
+ console.print(
+ f"\n[bold]Change risk[/bold] for [cyan]{features.ref}[/cyan]: "
+ f"[{color}]{risk.level}[/{color}] (absolute band — no repo baseline to rank against)"
+ )
if features.subject:
console.print(f" [dim]{features.subject}[/dim]")
console.print(
@@ -122,8 +245,17 @@ def risk_command(revspec: str, repo_path: str, ext: str | None, fmt: str) -> Non
f"entropy {features.entropy:.2f} · author exp {features.exp}"
+ (" [magenta](fix)[/magenta]" if features.is_fix else "")
)
+ if percentile is not None and priority is not None:
+ console.print(
+ f" [dim]{_PRIORITY_LEAD[priority]} ({_ordinal(round(percentile))} percentile).[/dim]"
+ )
+ # Raw score kept as a clearly-secondary, clearly-labelled number.
+ console.print(
+ f" [dim]Raw model score: {risk.score:.1f}/10 — corpus-anchored ({risk.level}); "
+ f"prefer the percentile for review order.[/dim]"
+ )
- table = Table(title="Risk drivers (signed contribution to the change-risk)")
+ table = Table(title="Why this score (each driver vs. the model's baseline commit)")
table.add_column("Driver")
table.add_column("Value", justify="right")
table.add_column("Push", justify="right")
diff --git a/packages/core/src/repowise/core/analysis/change_risk/model.py b/packages/core/src/repowise/core/analysis/change_risk/model.py
index 921c6399f..4525e3d38 100644
--- a/packages/core/src/repowise/core/analysis/change_risk/model.py
+++ b/packages/core/src/repowise/core/analysis/change_risk/model.py
@@ -42,18 +42,22 @@ _CONSTANTS: dict[str, object] = {
}
# Human-readable driver labels, keyed by whether the feature is ABOVE its
-# typical value — (above-typical phrase, below-typical phrase). The label
-# describes the change's shape (a fact); the signed contribution + colour convey
-# whether that pushed risk up or down, so a collinear coefficient sign never
-# mislabels the feature itself.
+# baseline value — (above-baseline phrase, below-baseline phrase). Each label
+# compares the feature to *the model's baseline commit* (the calibration-corpus
+# mean), NOT to this repo — so the wording is deliberately neutral and relative
+# ("more / fewer ... than baseline"), never an absolute verdict like "large".
+# The signed contribution + colour carry the risk direction, so a collinear
+# coefficient sign never makes a risk-lowering feature read like a warning. The
+# shared anchor phrase ("than the model's baseline commit") is stated once by the
+# surface that renders these, so it is omitted from every individual label.
_FEATURE_LABELS: dict[str, tuple[str, str]] = {
- "la": ("large diff (many lines added)", "small diff"),
- "ld": ("many lines deleted", "few deletions"),
- "nf": ("touches many files", "narrow file footprint"),
- "nd": ("spread across many directories", "localized to few directories"),
- "ns": ("spans multiple subsystems", "single subsystem"),
- "entropy": ("scattered, high-entropy change", "focused change"),
- "exp": ("experienced author", "low author familiarity with the code"),
+ "la": ("more lines added than baseline", "fewer lines added than baseline"),
+ "ld": ("more lines deleted than baseline", "fewer lines deleted than baseline"),
+ "nf": ("more files than baseline", "fewer files than baseline"),
+ "nd": ("more directories than baseline", "fewer directories than baseline"),
+ "ns": ("more subsystems than baseline", "fewer subsystems than baseline"),
+ "entropy": ("more scattered than baseline", "more focused than baseline"),
+ "exp": ("more experienced author than baseline", "less familiar author than baseline"),
}
diff --git a/packages/ui/src/commits/commit-detail-card.tsx b/packages/ui/src/commits/commit-detail-card.tsx
index 3696383db..ce86bd996 100644
--- a/packages/ui/src/commits/commit-detail-card.tsx
+++ b/packages/ui/src/commits/commit-detail-card.tsx
@@ -10,13 +10,74 @@ export interface CommitDetailCardProps {
className?: string;
}
-function Stat({ label, value, title }: { label: string; value: string; title?: string }) {
+/** 1 -> "1st", 2 -> "2nd", 93 -> "93rd", 11 -> "11th". */
+function ordinal(n: number): string {
+ const suffix =
+ n % 100 >= 10 && n % 100 <= 20
+ ? "th"
+ : ({ 1: "st", 2: "nd", 3: "rd" }[n % 10] ?? "th");
+ return `${n}${suffix}`;
+}
+
+/** Lead clause: where this commit sits in its own repo's risk distribution. */
+const PRIORITY_LEAD: Record<string, string> = {
+ low: "Lower risk than a typical commit in this repo",
+ moderate: "About as risky as a typical commit in this repo",
+ high: "Riskier than most commits in this repo",
+};
+
+/**
+ * One-line, plain-English read of the score: the repo-relative standing first
+ * (the signal users should act on), then a short, honest note on why the raw
+ * model score landed where it did. Built entirely from fields already on the
+ * commit — no extra fetch.
+ */
+function riskSummary(c: CommitDetail): string {
+ const lead = PRIORITY_LEAD[c.review_priority] ?? PRIORITY_LEAD.moderate;
+ let out = `${lead} (${ordinal(Math.round(c.risk_percentile))} percentile).`;
+
+ if (c.change_risk_score != null) {
+ // `drivers` arrive strongest-first; the risk-raising ones explain the score.
+ const raising = c.drivers.filter(
+ (d) => d.value !== null && d.contribution > 0,
+ );
+ const raw = c.change_risk_score.toFixed(1);
+ if (raising.length === 0) {
+ out += ` Its raw model score (${raw}/10) stays low across every driver.`;
+ } else {
+ const reasons = raising
+ .slice(0, 2)
+ .map((d) => d.label)
+ .join(" and ");
+ out += ` Its raw model score (${raw}/10) is driven mainly by ${reasons}`;
+ out +=
+ ", measured against the model's baseline commit rather than this repo.";
+ }
+ }
+
+ if (c.author_experience != null && c.author_experience < 3) {
+ out += " The author is new to this code.";
+ }
+ return out;
+}
+
+function Stat({
+ label,
+ value,
+ title,
+}: {
+ label: string;
+ value: string;
+ title?: string;
+}) {
return (
<div title={title}>
<div className="text-[10px] uppercase tracking-wider text-[var(--color-text-tertiary)]">
{label}
</div>
- <div className="text-sm tabular-nums text-[var(--color-text-primary)]">{value}</div>
+ <div className="text-sm tabular-nums text-[var(--color-text-primary)]">
+ {value}
+ </div>
</div>
);
}
@@ -58,9 +119,11 @@ export function CommitDetailCard({ commit, className }: CommitDetailCardProps) {
confidence={c.agent_confidence}
/>
)}
- {!c.agent_name && c.author_experience != null && c.author_experience < 3 && (
- <NewContributorBadge experience={c.author_experience} />
- )}
+ {!c.agent_name &&
+ c.author_experience != null &&
+ c.author_experience < 3 && (
+ <NewContributorBadge experience={c.author_experience} />
+ )}
{c.committed_at && <span>{formatDateTime(c.committed_at)}</span>}
</div>
{c.agent_name && c.agent_channel && (
@@ -76,7 +139,9 @@ export function CommitDetailCard({ commit, className }: CommitDetailCardProps) {
<div className="text-center">
<div className="text-2xl font-semibold tabular-nums text-[var(--color-text-primary)]">
{c.risk_percentile.toFixed(0)}
- <span className="text-sm text-[var(--color-text-tertiary)]">%ile</span>
+ <span className="text-sm text-[var(--color-text-tertiary)]">
+ %ile
+ </span>
</div>
<div className="mt-0.5">
<PriorityBadge priority={c.review_priority} />
@@ -99,11 +164,23 @@ export function CommitDetailCard({ commit, className }: CommitDetailCardProps) {
</div>
</div>
+ {/* Plain-English read — repo-relative standing first, then the why. */}
+ <p className="mb-4 text-xs leading-relaxed text-[var(--color-text-secondary)]">
+ {riskSummary(c)}
+ </p>
+
{/* Kamei change features */}
<div className="mb-4 grid grid-cols-3 gap-3 sm:grid-cols-4">
- <Stat label="Lines" value={`+${formatLOC(c.lines_added)} -${formatLOC(c.lines_deleted)}`} />
+ <Stat
+ label="Lines"
+ value={`+${formatLOC(c.lines_added)} -${formatLOC(c.lines_deleted)}`}
+ />
<Stat label="Files" value={String(c.files_changed)} />
- <Stat label="Dirs" value={String(c.dirs_changed)} title="Distinct directories touched" />
+ <Stat
+ label="Dirs"
+ value={String(c.dirs_changed)}
+ title="Distinct directories touched"
+ />
<Stat
label="Subsystems"
value={String(c.subsystems_changed)}
@@ -125,9 +202,13 @@ export function CommitDetailCard({ commit, className }: CommitDetailCardProps) {
{/* Risk drivers */}
<div>
- <p className="mb-2 text-xs font-medium uppercase tracking-wider text-[var(--color-text-tertiary)]">
+ <p className="mb-1 text-xs font-medium uppercase tracking-wider text-[var(--color-text-tertiary)]">
Why this score
</p>
+ <p className="mb-2 text-[10px] text-[var(--color-text-tertiary)]">
+ Each driver compares this change to the model&apos;s baseline commit.
+ Red raised the raw score; green lowered it.
+ </p>
<RiskDriverBreakdown drivers={c.drivers} />
</div>
</div>
diff --git a/packages/ui/src/commits/priority-badge.tsx b/packages/ui/src/commits/priority-badge.tsx
index 427e724d5..60a845869 100644
--- a/packages/ui/src/commits/priority-badge.tsx
+++ b/packages/ui/src/commits/priority-badge.tsx
@@ -1,16 +1,23 @@
import { cn } from "../lib/cn";
import type { ReviewPriority } from "@repowise-dev/types/git";
+// Colours track review attention, not absolute danger: "Below typical" and
+// "Typical" are calm (this commit is not unusual for the repo); only "Elevated"
+// — the top third of the repo's own distribution — draws the eye.
const STYLES: Record<ReviewPriority, string> = {
- high: "bg-red-500/15 text-red-400 border-red-500/25",
- moderate: "bg-yellow-500/15 text-yellow-400 border-yellow-500/25",
+ high: "bg-amber-500/15 text-amber-400 border-amber-500/25",
+ moderate:
+ "bg-[var(--color-bg-elevated)] text-[var(--color-text-secondary)] border-[var(--color-border-default)]",
low: "bg-green-500/15 text-green-400 border-green-500/25",
};
+// Repo-relative tercile wording — where the commit sits in *its own repo's*
+// risk distribution, so a 44th-percentile commit reads "Typical", never the
+// absolute-sounding "Moderate".
const LABELS: Record<ReviewPriority, string> = {
- high: "High",
- moderate: "Moderate",
- low: "Low",
+ high: "Elevated",
+ moderate: "Typical",
+ low: "Below typical",
};
/**