Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(coverage): surface coverage-backed tests_to_run in get_risk PR directive (#859)

Raghav Chamadiyaf1a1419b9338
diff --git a/README.md b/README.md
index e2be796f5..bae0d0553 100644
--- a/README.md
+++ b/README.md
@@ -225,8 +225,9 @@ people who have to trust what ships:
- **★ Change risk:** score any commit or `base..HEAD` range **0–10** for defect
risk from the shape of the diff (Kamei-style just-in-time metrics), with
- PR-mode directives (`will_break`, `missing_cochanges`, `missing_tests`). One
- command: `repowise risk main..HEAD`. Reference: **[docs/CHANGE_RISK.md](docs/CHANGE_RISK.md)**.
+ PR-mode directives (`will_break`, `missing_cochanges`, `missing_tests`,
+ `tests_to_run`). One command: `repowise risk main..HEAD`. Reference:
+ **[docs/CHANGE_RISK.md](docs/CHANGE_RISK.md)**.
- **★ Agent provenance:** attribute commits to the AI agents that wrote them,
straight from git history, so you can see how much of your codebase an agent
produced and which of that code is a low-health hotspot owned by a single
@@ -494,7 +495,7 @@ diverges from live `.git/HEAD`.
| `get_context(targets, include?)` | Triage card for files / modules / symbols: title, summary, signatures, `hotspot` bit, `governing_decisions`, and `symbol_id`s. `include` opens callers/callees, ownership, metrics, decisions, full_doc. Batch many targets. |
| `get_symbol("file.py::Name")` | Raw source bytes for one indexed symbol with exact line bounds, cheaper and safer than `Read` + offset math. |
| `search_codebase(query, kind?)` | Semantic search over the wiki, filterable by `kind` (implementation / test / config / doc), tagging each result's `search_method`. |
-| `get_risk(targets, changed_files?)` | Hotspot scores, dependents, co-change partners, ownership, test gaps, security signals. Pass `changed_files` for PR mode → a `directive` block (`will_break`, `missing_cochanges`, `missing_tests`, `governance_risk`). |
+| `get_risk(targets, changed_files?)` | Hotspot scores, dependents, co-change partners, ownership, test gaps, security signals. Pass `changed_files` for PR mode → a `directive` block (`will_break`, `missing_cochanges`, `missing_tests`, `tests_to_run`, `governance_risk`). |
| `get_why(query?, targets?)` | Architectural decision records, status, evidence spans, and the supersession **lineage chain**. Falls back to git archaeology when no ADRs exist. |
| `get_dead_code(...)` | Unreachable code by confidence tier with cleanup-impact estimates; cross-repo consumer detection in workspace mode. |
| `get_health(targets?, include?)` | Marker scores per file across three signals (defect · maintainability · performance). Dashboard mode → KPIs + lowest-scoring files + module rollup; targeted mode → per-file findings. Self-check before a PR via `include`: `accuracy` (does the score find the bugs), `signals` (per-file churn / owners / prior defects), `churn_complexity`, a dimension name to filter findings, plus `coverage`, `trend`, and `refactoring` → **structured, graph-aware refactoring plans** (split groups, move target, cut edges + blast radius), not template strings. |
diff --git a/docs/MCP_TOOLS.md b/docs/MCP_TOOLS.md
index 5031e2458..fc464d177 100644
--- a/docs/MCP_TOOLS.md
+++ b/docs/MCP_TOOLS.md
@@ -313,7 +313,7 @@ Modification risk assessment for files or a set of changed files.
**Returns:** Per-file risk score (0-10), hotspot status, dependent count, co-change partners, blast radius, recommended reviewers, test gap analysis, security signals. In workspace mode, enriched with cross-repo co-change partners and contract dependencies.
-When `changed_files` is passed, the response leads with a `directive` block. Its core lists are the local blast radius: `will_break` (production files that depend on the diff and are likely to break), `will_break_tests` (test files impacted the same way, kept separate so a burst of broken tests doesn't crowd production impact out of the capped list), `missing_cochanges` (historical co-changers absent from the diff), and `missing_tests` (changed files without test coverage). In workspace mode that directive also carries the cross-repo fallout of the changed repo:
+When `changed_files` is passed, the response leads with a `directive` block. Its core lists are the local blast radius: `will_break` (production files that depend on the diff and are likely to break), `will_break_tests` (test files impacted the same way, kept separate so a burst of broken tests doesn't crowd production impact out of the capped list), `missing_cochanges` (historical co-changers absent from the diff), `missing_tests` (changed files without test coverage), and `tests_to_run` (the positive complement of `missing_tests`: the tests the per-test coverage map proves execute the changed files, as pytest-runnable ids to validate the change; empty until a coverage map is ingested with `repowise coverage add`). In workspace mode that directive also carries the cross-repo fallout of the changed repo:
- `will_break_consumers`: services in *other* repos that depend on this one (structural impact), each with `repo`, `service`, `distance`, `score`, and the edge kinds carrying the impact.
- `missing_cross_repo_cochanges`: services in other repos that historically co-change with this one but aren't in the diff.
diff --git a/packages/core/src/repowise/core/analysis/pr_blast.py b/packages/core/src/repowise/core/analysis/pr_blast.py
index 01b4989be..05977804c 100644
--- a/packages/core/src/repowise/core/analysis/pr_blast.py
+++ b/packages/core/src/repowise/core/analysis/pr_blast.py
@@ -65,7 +65,11 @@ class PRBlastRadiusAnalyzer:
# 5. Test gaps
test_gaps = await self._find_test_gaps(all_affected_paths)
- # 6. Overall risk score (0-10)
+ # 6. Guarding tests — the tests the per-test coverage map proves execute
+ # the changed files (the "run these to validate" answer).
+ guarding_tests = await self._guarding_tests(changed_files)
+
+ # 7. Overall risk score (0-10)
overall_risk_score = self._compute_overall_risk(direct_risks, transitive_affected)
return {
@@ -74,6 +78,7 @@ class PRBlastRadiusAnalyzer:
"cochange_warnings": cochange_warnings,
"recommended_reviewers": recommended_reviewers,
"test_gaps": test_gaps,
+ "guarding_tests": guarding_tests,
"overall_risk_score": overall_risk_score,
}
@@ -128,9 +133,7 @@ class PRBlastRadiusAnalyzer:
"""Compute file-level risk: centrality * (1 + temporal_hotspot_score)."""
return centrality * (1.0 + temporal_hotspot_score)
- async def _transitive_affected(
- self, changed_files: list[str], max_depth: int
- ) -> list[dict]:
+ async def _transitive_affected(self, changed_files: list[str], max_depth: int) -> list[dict]:
"""BFS over reverse graph edges (source_node_id -> target_node_id direction).
We want files that *import* the changed files (i.e. are affected when a
@@ -289,6 +292,54 @@ class PRBlastRadiusAnalyzer:
return gaps
+ async def _guarding_tests(self, changed_files: list[str]) -> dict:
+ """Tests the per-test coverage map proves execute the *changed* files.
+
+ The inverse of ``test_gaps``: instead of "which changed file lacks a
+ test", this answers "which recorded tests actually exercise this
+ change" - the coverage-backed "run these to validate" list, keyed by
+ test id (a pytest-runnable node id when the report carried node-id
+ contexts). Reuses the Phase 1 reverse index ``tests_covering``.
+
+ Scoped to the changed files, NOT the transitively-affected set: the
+ Phase 3 gate showed that affected (importing) files are covered
+ overwhelmingly by the same mega parametrized suites, so an
+ affected-file run-list is near-useless noise. The actionable set is the
+ tests that execute the code you actually edited. Line precision is not
+ available here (get_risk takes file paths, not a diff), so this is
+ file-level; ``repowise impacted-tests`` gives the line-level answer from
+ a real diff.
+
+ Returns ``{map_present, tests_to_run, by_file}``. ``map_present`` is
+ False when no per-test map is ingested at all - the honest "unknown",
+ distinct from "this change has no guarding tests".
+ """
+ empty = {"map_present": False, "tests_to_run": [], "by_file": {}}
+ if not changed_files:
+ return empty
+
+ from repowise.core.persistence.crud import get_test_coverage_summary, tests_covering
+
+ summary = await get_test_coverage_summary(self._session, self._repo_id)
+ if summary.get("pair_count", 0) == 0:
+ return empty
+
+ by_file: dict[str, list[str]] = {}
+ all_ids: set[str] = set()
+ for path in changed_files:
+ rows = await tests_covering(self._session, self._repo_id, path, lines=None)
+ if not rows:
+ continue
+ ids = sorted({r["test_id"] for r in rows})
+ by_file[path] = ids
+ all_ids.update(ids)
+
+ return {
+ "map_present": True,
+ "tests_to_run": sorted(all_ids),
+ "by_file": by_file,
+ }
+
@staticmethod
def _compute_overall_risk(
direct_risks: list[dict],
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
index ec2352d60..df01c6cb6 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
@@ -48,6 +48,10 @@ _CF_CYCLE_LIMIT = 3
#: keeps the larger budget; test fallout is a secondary signal capped tighter.
_WILL_BREAK_LIMIT = 5
_WILL_BREAK_TESTS_LIMIT = 3
+#: Cap on the coverage-backed run-list. A validate-this-change set can be longer
+#: than the will-break lists (it is what you actually run), but stays glanceable;
+#: the overflow and the full per-file map live in pr_blast_radius.guarding_tests.
+_TESTS_TO_RUN_LIMIT = 10
def _breaking_change_directive(repo_alias: str) -> list[dict[str, Any]]:
@@ -330,6 +334,26 @@ def _build_pr_directive(
exclude_spec,
)[:3]
+ # Coverage-backed run-list: the tests the per-test map proves execute the
+ # changed files (the positive complement of missing_tests). Read from the
+ # untrimmed analyzer payload; _trim_blast_lists passes guarding_tests
+ # through, but reading it here keeps the source explicit. Test node ids,
+ # not file paths, so they are not exclude-filtered as paths.
+ guarding = pr_blast_radius.get("guarding_tests") or {}
+ all_tests_to_run = list(guarding.get("tests_to_run", []))
+ tests_to_run = all_tests_to_run[:_TESTS_TO_RUN_LIMIT]
+ if len(all_tests_to_run) > _TESTS_TO_RUN_LIMIT:
+ collector.add(
+ f"directive.tests_to_run beyond cap={_TESTS_TO_RUN_LIMIT} "
+ f"({len(all_tests_to_run) - _TESTS_TO_RUN_LIMIT} dropped)",
+ all_tests_to_run[_TESTS_TO_RUN_LIMIT:],
+ )
+ tests_to_run_suffix = (
+ f" {len(all_tests_to_run)} coverage-backed test(s) guard the change - run these."
+ if all_tests_to_run
+ else ""
+ )
+
gov_count = len(governance_risk)
gov_suffix = f" {gov_count} governance risk(s) detected." if gov_count > 0 else ""
@@ -376,6 +400,7 @@ def _build_pr_directive(
"will_break_tests": will_break_tests,
"missing_cochanges": missing_cochanges,
"missing_tests": missing_tests,
+ "tests_to_run": tests_to_run,
"will_break_consumers": will_break_consumers,
"missing_cross_repo_cochanges": missing_cross_repo_cochanges,
"breaking_changes": breaking_changes,
@@ -389,6 +414,6 @@ def _build_pr_directive(
f"{len(will_break_tests)} test(s) likely broken, "
f"{len(missing_cochanges)} historical co-changer(s) missing, "
f"{len(missing_tests)} file(s) without tests."
- f"{gov_suffix}{xr_suffix}{bc_suffix}{cf_suffix}"
+ f"{tests_to_run_suffix}{gov_suffix}{xr_suffix}{bc_suffix}{cf_suffix}"
),
}
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py b/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
index e72c68408..cbc2ec699 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
@@ -41,7 +41,9 @@ async def get_risk(
graph topology (dependents, co-changes, impact surface) and security
findings. Consult before editing 95th+ churn-percentile files. Pass
changed_files for PR mode: the response leads with a directive block
- (will_break, missing_cochanges, missing_tests) — read it first.
+ (will_break, missing_cochanges, missing_tests, tests_to_run) — read it
+ first. tests_to_run is coverage-backed: the tests the per-test map proves
+ exercise the changed files, empty when no coverage map is ingested.
Args:
targets: file paths to assess.
diff --git a/plugins/claude-code/skills/change-review/SKILL.md b/plugins/claude-code/skills/change-review/SKILL.md
index 1cd429dc9..7ba737950 100644
--- a/plugins/claude-code/skills/change-review/SKILL.md
+++ b/plugins/claude-code/skills/change-review/SKILL.md
@@ -44,15 +44,20 @@ Call `get_risk` in **PR mode** by passing the changed files:
get_risk(targets=<changed files>, changed_files=<same changed files>)
```
-The response carries a `directive` block — read it first, it's three short lists:
+The response carries a `directive` block — read it first, it's a few short lists:
- **`will_break`** — files/symbols that depend on what changed but are *not* in
the diff. These are the likely breakages. Check each one.
- **`missing_cochanges`** — files that historically change together with the
changed files but were left untouched. Often a forgotten update.
- **`missing_tests`** — changed code with a test gap. Flag for new/updated tests.
+- **`tests_to_run`** — the positive complement of `missing_tests`: the tests the
+ per-test coverage map proves execute the changed files (pytest-runnable ids).
+ Recommend running these to validate the change. Empty until a coverage map is
+ ingested (`repowise coverage add`); empty is "unknown", never "no tests exist".
-`pr_blast_radius` holds the fuller dossier behind those three lists.
+`pr_blast_radius` holds the fuller dossier behind those lists (including the
+per-changed-file `guarding_tests` breakdown behind `tests_to_run`).
## Then go deeper where it matters
diff --git a/tests/unit/persistence/test_pr_blast_guarding_tests.py b/tests/unit/persistence/test_pr_blast_guarding_tests.py
new file mode 100644
index 000000000..550e1e4bf
--- /dev/null
+++ b/tests/unit/persistence/test_pr_blast_guarding_tests.py
@@ -0,0 +1,137 @@
+"""Coverage-backed guarding tests in PRBlastRadiusAnalyzer._guarding_tests.
+
+The positive complement of ``_find_test_gaps``: given the changed files, return
+the tests the per-test coverage map proves execute them (the "run these to
+validate" list). Seeds a ``test_coverage`` table and pins the map-present,
+map-empty, and no-rows-for-file paths.
+"""
+
+from __future__ import annotations
+
+from repowise.core.analysis.health.coverage import TestCoverage
+from repowise.core.analysis.pr_blast import PRBlastRadiusAnalyzer
+from repowise.core.persistence.crud import save_test_coverage
+from tests.unit.persistence.helpers import insert_repo
+
+
+async def _seed_map(session, repo_id):
+ await save_test_coverage(
+ session,
+ repo_id,
+ [
+ TestCoverage(
+ test_id="tests/test_math.py::test_add",
+ file_path="src/math.py",
+ covered_lines=[1, 2, 3],
+ source_format="coverage.py",
+ test_file="tests/test_math.py",
+ ),
+ TestCoverage(
+ test_id="tests/test_math.py::test_sub",
+ file_path="src/math.py",
+ covered_lines=[5, 6],
+ source_format="coverage.py",
+ test_file="tests/test_math.py",
+ ),
+ TestCoverage(
+ test_id="tests/test_util.py::test_helper",
+ file_path="src/util.py",
+ covered_lines=[10],
+ source_format="coverage.py",
+ test_file="tests/test_util.py",
+ ),
+ ],
+ source_format="coverage.py",
+ )
+
+
+async def test_guarding_tests_returns_covering_tests_per_file(async_session):
+ repo = await insert_repo(async_session)
+ await _seed_map(async_session, repo.id)
+ await async_session.commit()
+
+ analyzer = PRBlastRadiusAnalyzer(async_session, repo.id)
+ result = await analyzer._guarding_tests(["src/math.py"])
+
+ assert result["map_present"] is True
+ # Both math tests guard math.py; util's test does not leak in.
+ assert result["tests_to_run"] == [
+ "tests/test_math.py::test_add",
+ "tests/test_math.py::test_sub",
+ ]
+ assert result["by_file"]["src/math.py"] == [
+ "tests/test_math.py::test_add",
+ "tests/test_math.py::test_sub",
+ ]
+
+
+async def test_guarding_tests_unions_and_dedups_across_changed_files(async_session):
+ repo = await insert_repo(async_session)
+ # test_shared guards BOTH files - must appear once in the union.
+ await save_test_coverage(
+ async_session,
+ repo.id,
+ [
+ TestCoverage(
+ test_id="tests/test_shared.py::test_both",
+ file_path="src/a.py",
+ covered_lines=[1],
+ source_format="coverage.py",
+ test_file="tests/test_shared.py",
+ ),
+ TestCoverage(
+ test_id="tests/test_shared.py::test_both",
+ file_path="src/b.py",
+ covered_lines=[2],
+ source_format="coverage.py",
+ test_file="tests/test_shared.py",
+ ),
+ ],
+ source_format="coverage.py",
+ )
+ await async_session.commit()
+
+ analyzer = PRBlastRadiusAnalyzer(async_session, repo.id)
+ result = await analyzer._guarding_tests(["src/a.py", "src/b.py"])
+
+ assert result["tests_to_run"] == ["tests/test_shared.py::test_both"]
+ assert set(result["by_file"]) == {"src/a.py", "src/b.py"}
+
+
+async def test_guarding_tests_map_empty_is_honest_unknown(async_session):
+ repo = await insert_repo(async_session)
+ await async_session.commit() # no coverage rows at all
+
+ analyzer = PRBlastRadiusAnalyzer(async_session, repo.id)
+ result = await analyzer._guarding_tests(["src/math.py"])
+
+ # No map -> "unknown", never an empty-because-untested claim.
+ assert result == {"map_present": False, "tests_to_run": [], "by_file": {}}
+
+
+async def test_guarding_tests_file_with_no_rows_omitted_but_map_present(async_session):
+ repo = await insert_repo(async_session)
+ await _seed_map(async_session, repo.id)
+ await async_session.commit()
+
+ analyzer = PRBlastRadiusAnalyzer(async_session, repo.id)
+ # new.py has no coverage rows; math.py does. Map IS present.
+ result = await analyzer._guarding_tests(["src/new.py", "src/math.py"])
+
+ assert result["map_present"] is True
+ assert "src/new.py" not in result["by_file"]
+ assert result["by_file"]["src/math.py"] == [
+ "tests/test_math.py::test_add",
+ "tests/test_math.py::test_sub",
+ ]
+
+
+async def test_guarding_tests_empty_changed_set(async_session):
+ repo = await insert_repo(async_session)
+ await _seed_map(async_session, repo.id)
+ await async_session.commit()
+
+ analyzer = PRBlastRadiusAnalyzer(async_session, repo.id)
+ result = await analyzer._guarding_tests([])
+
+ assert result == {"map_present": False, "tests_to_run": [], "by_file": {}}
diff --git a/tests/unit/server/mcp/test_risk.py b/tests/unit/server/mcp/test_risk.py
index 2ba3815db..1bb3dab15 100644
--- a/tests/unit/server/mcp/test_risk.py
+++ b/tests/unit/server/mcp/test_risk.py
@@ -112,6 +112,59 @@ async def test_get_risk_pr_directive_splits_test_breakage(setup_mcp):
assert "test(s) likely broken" in directive["summary"]
+async def test_get_risk_pr_directive_surfaces_coverage_backed_tests_to_run(setup_mcp, session):
+ """PR directive carries coverage-backed tests_to_run from the per-test map."""
+ from repowise.core.analysis.health.coverage import TestCoverage
+ from repowise.core.persistence.crud import save_test_coverage
+ from repowise.server.mcp_server import get_risk
+
+ # Seed a per-test map: two tests execute service.py.
+ await save_test_coverage(
+ session,
+ "repo1",
+ [
+ TestCoverage(
+ test_id="tests/test_service.py::test_login",
+ file_path="src/auth/service.py",
+ covered_lines=[1, 2],
+ source_format="coverage.py",
+ test_file="tests/test_service.py",
+ ),
+ TestCoverage(
+ test_id="tests/test_service.py::test_logout",
+ file_path="src/auth/service.py",
+ covered_lines=[3, 4],
+ source_format="coverage.py",
+ test_file="tests/test_service.py",
+ ),
+ ],
+ source_format="coverage.py",
+ )
+ await session.flush()
+
+ result = await get_risk(["src/auth/service.py"], changed_files=["src/auth/service.py"])
+ directive = result["directive"]
+
+ assert directive["tests_to_run"] == [
+ "tests/test_service.py::test_login",
+ "tests/test_service.py::test_logout",
+ ]
+ assert "coverage-backed test(s) guard the change" in directive["summary"]
+
+
+async def test_get_risk_pr_directive_tests_to_run_empty_without_map(setup_mcp):
+ """No per-test map -> tests_to_run is an empty list, never invented."""
+ from repowise.server.mcp_server import get_risk
+
+ result = await get_risk(["src/auth/service.py"], changed_files=["src/auth/service.py"])
+ directive = result["directive"]
+
+ assert directive["tests_to_run"] == []
+ assert "coverage-backed test(s) guard the change" not in directive["summary"]
+
+
# ---- _classify_risk_type small-team calibration (issue #361) ---------------