Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(risk): list the test files a change will probably need to edit (#3101)

Raghav Chamadiya1e97254c18fe
diff --git a/docs/agent/MCP_TOOLS.md b/docs/agent/MCP_TOOLS.md
index e4085553a..a052d2d05 100644
--- a/docs/agent/MCP_TOOLS.md
+++ b/docs/agent/MCP_TOOLS.md
@@ -230,7 +230,7 @@ What history says about touching a file: hotspot score, bug-fix record, owners,
| `include` | list[string] | none | `graph` (typed dependents, consumers, cross-repo links), `churn`, `scales` (units and calibration, identical per call) |
| `repo` | string | default repo | Workspace repo alias. `"all"` is not supported |
-**Key return fields:** per file: `hotspot_score` (0 to 1), `health_score` (0 to 10), `dependents_count`, `co_change_partners`, owners, test gaps, `security_signals`. In PR mode, `directive` with `may_break`, `may_break_tests`, `missing_cochanges`, `test_recommendations` (each `measured` or `inferred`), `tests_to_run`, `tests_to_run_basis`, `next_calls`, and the 0 to 10 `structural_impact_score`. A target naming no indexed file returns `resolved: false` with a reason, never zeroed counts.
+**Key return fields:** per file: `hotspot_score` (0 to 1), `health_score` (0 to 10), `dependents_count`, `co_change_partners`, owners, test gaps, `security_signals`. In PR mode, `directive` with `may_break`, `may_break_tests`, `missing_cochanges`, `test_recommendations` (each `measured` or `inferred`), `tests_to_run`, `tests_to_run_basis`, `tests_to_update` (test files to edit, each with a `name_pair`, `imports` or `co_change` reason), `next_calls`, and the 0 to 10 `structural_impact_score`. A target naming no indexed file returns `resolved: false` with a reason, never zeroed counts.
Dependent counts are a floor over the indexed graph, and structural reach is not proof of runtime breakage. `structural_impact_score` is an uncalibrated heuristic, not a probability.
diff --git a/docs/layers/CHANGE_RISK.md b/docs/layers/CHANGE_RISK.md
index 5af43223f..561ca047c 100644
--- a/docs/layers/CHANGE_RISK.md
+++ b/docs/layers/CHANGE_RISK.md
@@ -143,6 +143,7 @@ files reach, and what is missing from it. The response opens with a
| `may_break_tests` | Test files reached the same way, listed separately |
| `missing_cochanges` | Files that historically change with these but are not in the diff |
| `missing_tests`, `test_recommendations`, `tests_to_run` | Test gaps and which tests to run; each recommendation says whether it is `measured` (coverage) or `inferred` (graph reach) |
+| `tests_to_update` | Up to three test files the change will probably need edited, with why: `name_pair`, `imports` or `co_change`; empty when none qualify |
| `next_calls` | What to call next |
| `summary` | One sentence over all of the above |
diff --git a/docs/reference/MCP_RESPONSE_FIELDS.md b/docs/reference/MCP_RESPONSE_FIELDS.md
index 7e67bafff..545f610b6 100644
--- a/docs/reference/MCP_RESPONSE_FIELDS.md
+++ b/docs/reference/MCP_RESPONSE_FIELDS.md
@@ -224,6 +224,7 @@ Opt-in blocks: `graph` adds typed `dependents` (direct versus transitive), `cons
| `missing_cochanges` | Historical co-change partners absent from the change |
| `test_recommendations` | Typed rows, each with a `basis`: `measured` (the per-test coverage map found the test) or `inferred` (structural reach, not coverage proof) |
| `tests_to_run`, `tests_to_run_basis` | The test ids or files to run, and whether the list is `measured`, `inferred` or `none`. An empty list with unavailable coverage never means no tests are needed |
+| `tests_to_update` | Up to three test files the change will probably need edited, each `{path, reason}` with `reason` `name_pair` (named for a changed file), `imports` (imports one directly) or `co_change` (changes with one in git history), in that order. Tests to edit, not to run: a file can appear in both lists. Empty when none qualify |
| `missing_tests`, `files_without_measured_tests` | Changed files with no measured test |
| `coverage_analysis`, `test_inference_analysis` | Whether each evidence source was available, stale, partial or degraded |
| `structural_impact_score` | Uncalibrated 0 to 10 structural heuristic: `localized` below 4, `moderate` 4 to below 7, `broad` 7 and up. Not a breakage probability. `overall_risk_score` is a deprecated alias |
diff --git a/packages/server/src/repowise/server/mcp_server/_budget/contracts.py b/packages/server/src/repowise/server/mcp_server/_budget/contracts.py
index 0034c0c6e..2b2c3017f 100644
--- a/packages/server/src/repowise/server/mcp_server/_budget/contracts.py
+++ b/packages/server/src/repowise/server/mcp_server/_budget/contracts.py
@@ -100,6 +100,7 @@ _CONTRACTS: dict[str, ResponseBudgetContract] = {
"pr_blast_radius",
"directive.test_recommendations[]",
"directive.tests_to_run[]",
+ "directive.tests_to_update[]",
"directive.may_break[]",
"directive.next_calls[]",
"targets[]",
@@ -117,6 +118,7 @@ _CONTRACTS: dict[str, ResponseBudgetContract] = {
(
"directive.test_recommendations[]",
"directive.tests_to_run[]",
+ "directive.tests_to_update[]",
"directive.may_break[]",
"directive.next_calls[]",
"pr_blast_radius",
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
index d9c5fab4d..78059712f 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
@@ -8,6 +8,8 @@ from sqlalchemy import select
from repowise.core.analysis.next_call import ActionCommand
from repowise.core.analysis.risk_semantics import structural_impact_contract
+from repowise.core.analysis.test_reachability import tests_matching_by_name
+from repowise.core.co_change import MIN_CO_CHANGE_SUPPORT
from repowise.core.persistence.crud.authority import decision_currencies
from repowise.core.persistence.database import get_session
from repowise.core.persistence.decision_graph import list_conflict_edges
@@ -59,6 +61,8 @@ _MAY_BREAK_TESTS_LIMIT = 3
#: the overflow and full typed rows live in pr_blast_radius.test_impact.
_TESTS_TO_RUN_LIMIT = 10
_TESTS_TO_RUN_KIND = {"measured": "test_id", "inferred": "test_file"}
+#: Cap on the edit-list: the test files this change will probably need edited.
+_TESTS_TO_UPDATE_LIMIT = 3
def _breaking_change_directive(
@@ -479,6 +483,47 @@ def _project_recommendation(row: dict[str, Any]) -> dict[str, Any]:
return out
+def _tests_to_update(
+ changed_files: list[str],
+ test_paths: set[str],
+ pr_blast_radius: dict,
+ exclude_spec: Any,
+) -> list[dict[str, str]]:
+ """Test files the change will probably need edited, strongest reason first.
+
+ A test named for a changed file (``name_pair``), then one that imports it
+ (``imports``), then one that changes with it in git history (``co_change``).
+ A path keeps its first reason; tests already in the change are left out.
+ """
+ changed = set(changed_files)
+ candidates = filter_path_list(sorted(test_paths - changed), exclude_spec)
+ eligible = set(candidates)
+ named = tests_matching_by_name(changed_files, candidates)
+ ordered: list[tuple[str | None, str]] = [
+ (path, "name_pair")
+ for source in changed_files
+ if source in named
+ for path in (named[source].all_tests or named[source].tests)
+ ]
+ ordered += [
+ (_as_path(e), "imports")
+ for e in pr_blast_radius.get("transitive_affected") or []
+ if isinstance(e, dict) and e.get("direct")
+ ]
+ # Indexing already drops pairs below the support floor; a row that still
+ # records less is weak history, not a reason to edit a test.
+ ordered += [
+ (_as_path(e), "co_change")
+ for e in pr_blast_radius.get("cochange_warnings") or []
+ if isinstance(e, dict) and e.get("support", MIN_CO_CHANGE_SUPPORT) >= MIN_CO_CHANGE_SUPPORT
+ ]
+ rows: dict[str, str] = {}
+ for path, reason in ordered:
+ if path in eligible and path not in rows:
+ rows[path] = reason
+ return [{"path": path, "reason": reason} for path, reason in rows.items()]
+
+
def _build_pr_directive(
response: dict,
pr_blast_radius: dict,
@@ -543,6 +588,7 @@ def _build_pr_directive(
exclude_spec,
)
missing_cochanges = all_missing_cochanges[:3]
+ all_tests_to_update = _tests_to_update(changed_files, test_paths, pr_blast_radius, exclude_spec)
# Run-list: consume the analyzer's canonical typed population instead of
# independently deriving test ids. Every row retains its basis through
# de-duplication, sorting, exclusions, and the directive cap.
@@ -697,6 +743,8 @@ def _build_pr_directive(
"tests_to_run_emitted": len(tests_to_run),
"tests_to_run_truncated": tests_capped,
"tests_to_run_omitted": tests_to_run_total - len(tests_to_run),
+ # Tests to edit, not to run; a file can sit in both lists.
+ "tests_to_update": all_tests_to_update,
"test_recommendations": test_recommendations,
"test_recommendations_total": test_recommendations_total,
"test_recommendations_emitted": len(test_recommendations),
@@ -742,6 +790,7 @@ def _build_pr_directive(
("missing_cochanges", all_missing_cochanges, 3),
("missing_tests", all_missing_tests if coverage_usable else [], 3),
("tests_to_run", all_tests_to_run, _TESTS_TO_RUN_LIMIT),
+ ("tests_to_update", all_tests_to_update, _TESTS_TO_UPDATE_LIMIT),
("test_recommendations", all_recommendations, _TESTS_TO_RUN_LIMIT),
(
"files_without_measured_tests",
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py b/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
index bc115eae9..a96c2672b 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
@@ -359,7 +359,7 @@ async def get_risk(
runtime breakage. The response also includes security
findings. Pass changed_files for PR mode: the response leads with a
directive block (may_break, missing_cochanges, missing_tests,
- tests_to_run) — read it first. Each test_recommendations row carries a
+ tests_to_run, tests_to_update) — read it first. Each test_recommendations row carries a
measured or inferred basis, and coverage availability is explicit. To
score a commit or ``base..head`` range instead, use ``get_change_risk``.
diff --git a/tests/fixtures/mcp/tool_response_shapes.json b/tests/fixtures/mcp/tool_response_shapes.json
index 0fd549c8e..63220fcac 100644
--- a/tests/fixtures/mcp/tool_response_shapes.json
+++ b/tests/fixtures/mcp/tool_response_shapes.json
@@ -1487,6 +1487,12 @@
"tests_to_run_omitted": null,
"tests_to_run_total": null,
"tests_to_run_truncated": null,
+ "tests_to_update": [
+ {
+ "path": null,
+ "reason": null
+ }
+ ],
"will_break_consumers": [],
"will_break_consumers_deprecated": null,
"will_break_consumers_emitted": null,
diff --git a/tests/unit/server/mcp/test_risk.py b/tests/unit/server/mcp/test_risk.py
index 66e49efa8..6464efe5e 100644
--- a/tests/unit/server/mcp/test_risk.py
+++ b/tests/unit/server/mcp/test_risk.py
@@ -370,6 +370,127 @@ async def test_get_risk_pr_directive_names_no_tests_when_nothing_reaches(setup_m
assert "missing_tests is withheld" in directive["summary"]
+async def test_get_risk_pr_directive_lists_tests_to_update(setup_mcp):
+ """The test named for the change leads the edit-list, beside the run-list."""
+ from repowise.server.mcp_server import get_risk
+
+ directive = (await get_risk(["src/auth/service.py"], changed_files=["src/auth/service.py"]))[
+ "directive"
+ ]
+ assert directive["tests_to_update"] == [
+ {"path": "tests/test_service.py", "reason": "name_pair"}
+ ]
+ assert "tests/test_service.py" in directive["tests_to_run"]
+
+ untested = (await get_risk(["src/db/models.py"], changed_files=["src/db/models.py"]))[
+ "directive"
+ ]
+ assert untested["tests_to_update"] == []
+
+
+def _update_list(changed, tests, blast, exclude_spec=None):
+ from repowise.server.mcp_server.tool_risk.directives import _tests_to_update
+
+ rows = _tests_to_update(changed, set(tests), blast, exclude_spec)
+ return [(r["path"], r["reason"]) for r in rows]
+
+
+def test_tests_to_update_name_pair():
+ assert _update_list(
+ ["src/Foo.java"], {"src/test/FooTest.java", "src/test/BarTest.java"}, {}
+ ) == [("src/test/FooTest.java", "name_pair")]
+
+
+def test_tests_to_update_imports_only_direct_importers():
+ blast = {
+ "transitive_affected": [
+ {"path": "tests/test_api.py", "direct": True},
+ {"path": "tests/test_far.py", "direct": False},
+ {"path": "src/api.py", "direct": True},
+ ]
+ }
+ assert _update_list(["src/core.py"], {"tests/test_api.py", "tests/test_far.py"}, blast) == [
+ ("tests/test_api.py", "imports")
+ ]
+
+
+def test_tests_to_update_co_change_skips_production_partners():
+ blast = {
+ "cochange_warnings": [
+ {"changed": "src/core.py", "missing_partner": "src/other.py"},
+ {"changed": "src/core.py", "missing_partner": "tests/test_misc.py"},
+ ]
+ }
+ assert _update_list(["src/core.py"], {"tests/test_misc.py"}, blast) == [
+ ("tests/test_misc.py", "co_change")
+ ]
+
+
+def test_tests_to_update_co_change_drops_weak_support():
+ blast = {
+ "cochange_warnings": [
+ {"missing_partner": "tests/test_once.py", "support": 1},
+ {"missing_partner": "tests/test_twice.py", "support": 2},
+ ]
+ }
+ tests = {"tests/test_once.py", "tests/test_twice.py"}
+ assert _update_list(["src/core.py"], tests, blast) == [("tests/test_twice.py", "co_change")]
+
+
+def test_tests_to_update_honours_exclude_spec():
+ import pathspec
+
+ spec = pathspec.PathSpec.from_lines("gitwildmatch", ["tests/legacy/"])
+ tests = {"tests/legacy/test_core.py", "tests/legacy/test_api.py", "tests/test_misc.py"}
+ blast = {
+ "transitive_affected": [{"path": "tests/legacy/test_api.py", "direct": True}],
+ "cochange_warnings": [{"missing_partner": "tests/test_misc.py"}],
+ }
+ assert _update_list(["src/core.py"], tests, blast, spec) == [
+ ("tests/test_misc.py", "co_change")
+ ]
+
+
+def test_tests_to_update_orders_by_rule_and_keeps_the_first_reason():
+ tests = {"tests/test_core.py", "tests/test_api.py", "tests/test_misc.py"}
+ blast = {
+ "transitive_affected": [
+ {"path": "tests/test_api.py", "direct": True},
+ {"path": "tests/test_core.py", "direct": True},
+ ],
+ "cochange_warnings": [
+ {"missing_partner": "tests/test_misc.py"},
+ {"missing_partner": "tests/test_api.py"},
+ ],
+ }
+ assert _update_list(["src/core.py"], tests, blast) == [
+ ("tests/test_core.py", "name_pair"),
+ ("tests/test_api.py", "imports"),
+ ("tests/test_misc.py", "co_change"),
+ ]
+
+
+def test_tests_to_update_leaves_out_tests_already_in_the_change():
+ assert _update_list(["src/core.py", "tests/test_core.py"], {"tests/test_core.py"}, {}) == []
+
+
+def test_build_pr_directive_caps_tests_to_update_at_three():
+ from repowise.server.mcp_server._budget import OmissionCollector
+ from repowise.server.mcp_server.tool_risk import directives
+
+ tests = {f"tests/test_m{i}.py" for i in range(5)}
+ blast = {"cochange_warnings": [{"missing_partner": t} for t in sorted(tests)]}
+ response: dict = {"targets": {}}
+ directives._build_pr_directive(
+ response, blast, ["src/core.py"], None, OmissionCollector("get_risk"), [], tests, "repo"
+ )
+ directive = response["directive"]
+ assert [r["path"] for r in directive["tests_to_update"]] == sorted(tests)[:3]
+ assert directive["tests_to_update_total"] == 5
+ assert directive["tests_to_update_omitted"] == 2
+
+
@pytest.mark.asyncio
async def test_get_risk_pr_payload_serializes_directive_first(setup_mcp):
"""The exact external JSON order is actionable before any dossier."""