Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

refactor(risk): one test list and no empty families in the get_risk PR directive (#3117)

Raghav Chamadiya494db2915d31
diff --git a/docs/agent/MCP_TOOLS.md b/docs/agent/MCP_TOOLS.md
index 94e8b926e..358115893 100644
--- a/docs/agent/MCP_TOOLS.md
+++ b/docs/agent/MCP_TOOLS.md
@@ -227,10 +227,10 @@ What history says about touching a file: hotspot score, bug-fix record, owners,
|-----------|------|---------|---------|
| `targets` | list[string] | `changed_files` | File paths to assess |
| `changed_files` | list[string] | none | Files in a change; switches on PR mode |
-| `include` | list[string] | none | `graph` (typed dependents, consumers, cross-repo links), `churn`, `scales` (units and calibration, identical per call) |
+| `include` | list[string] | none | `graph` (typed dependents, consumers, cross-repo links), `churn`, `tests` (typed `test_recommendations` in PR mode), `scales` (units and calibration, identical per call) |
| `repo` | string | default repo | Workspace repo alias. `"all"` is not supported |
-**Key return fields:** per file: `hotspot_score` (0 to 1), `health_score` (0 to 10), `dependents_count`, `co_change_partners`, owners, test gaps, `security_signals`. In PR mode, `directive` with `may_break`, `may_break_tests`, `missing_cochanges`, `test_recommendations` (each `measured` or `inferred`), `tests_to_run`, `tests_to_run_basis`, `tests_to_update` (test files to edit, each with a `name_pair`, `imports` or `co_change` reason), `next_calls`, and the 0 to 10 `structural_impact_score`. A target naming no indexed file returns `resolved: false` with a reason, never zeroed counts.
+**Key return fields:** per file: `hotspot_score` (0 to 1), `health_score` (0 to 10), `dependents_count`, `co_change_partners`, owners, test gaps, `security_signals`. In PR mode, `directive` with `may_break`, `missing_cochanges`, `tests_to_run`, `tests_to_run_basis` (`measured`, `inferred` or `none`), `tests_to_update` (test files to edit, each with a `name_pair`, `imports` or `co_change` reason), `coverage`, `next_calls`, and the 0 to 10 `structural_impact_score`. `missing_tests` appears only when coverage can back it. A target naming no indexed file returns `resolved: false` with a reason, never zeroed counts.
Dependent counts are a floor over the indexed graph, and structural reach is not proof of runtime breakage. `structural_impact_score` is an uncalibrated heuristic, not a probability.
diff --git a/docs/layers/CHANGE_RISK.md b/docs/layers/CHANGE_RISK.md
index 561ca047c..fb4a5aae8 100644
--- a/docs/layers/CHANGE_RISK.md
+++ b/docs/layers/CHANGE_RISK.md
@@ -140,14 +140,15 @@ files reach, and what is missing from it. The response opens with a
| Field | Meaning |
|---|---|
| `may_break` | Production files in structural reverse-import reach of the diff. Candidates for review, not proven breakage |
-| `may_break_tests` | Test files reached the same way, listed separately |
| `missing_cochanges` | Files that historically change with these but are not in the diff |
-| `missing_tests`, `test_recommendations`, `tests_to_run` | Test gaps and which tests to run; each recommendation says whether it is `measured` (coverage) or `inferred` (graph reach) |
+| `tests_to_run`, `tests_to_run_basis` | Which tests to run, `measured` (coverage) or `inferred` (graph reach). With coverage the list is the measured one; `include=["tests"]` adds the typed `test_recommendations` rows, including reached tests the measured list lacks (`reason: structural_reach`) |
+| `missing_tests` | Changed files with a test gap; present only when coverage can back it |
+| `coverage` | `{status, reason}` when there is no per-test coverage map |
| `tests_to_update` | Up to three test files the change will probably need edited, with why: `name_pair`, `imports` or `co_change`; empty when none qualify |
| `next_calls` | What to call next |
| `summary` | One sentence over all of the above |
-In a workspace the directive also carries cross-repo fields:
+In a workspace the directive also carries cross-repo fields, dropped when no workspace is loaded:
`will_break_consumers` (repos that structurally depend on this one; structural
reach, not a runtime claim), `missing_cross_repo_cochanges`,
`breaking_changes` (incompatible provider contract changes since the last
diff --git a/docs/layers/TEST_INTELLIGENCE.md b/docs/layers/TEST_INTELLIGENCE.md
index b11053840..90bfc3cb1 100644
--- a/docs/layers/TEST_INTELLIGENCE.md
+++ b/docs/layers/TEST_INTELLIGENCE.md
@@ -232,17 +232,22 @@ with a different subsystem's `test_engine.py` by basename alone.
## From an agent
-**`get_risk(changed_files=[...])`** leads with a `directive` whose
-`test_recommendations` names up to ten tests for the changed files. Each row
-keeps its `basis`:
+**`get_risk(changed_files=[...], include=["tests"])`** leads with a `directive`
+whose `test_recommendations` names up to ten tests for the changed files. Each
+row keeps its `basis`:
| `basis` | Holds | Means |
|---|---|---|
| `measured` | A test node id | The per-test map shows the test covering a changed file |
| `inferred` | A test file path | The graph shows the test reaching the change; a candidate, not proof |
-`tests_to_run` is the older flat id list, measured first. `coverage_analysis`
-says whether coverage is available, partial, degraded or stale.
+`tests_to_run`, sent without the include, is the flat list with
+`tests_to_run_basis`. When coverage exists it is the measured list; test files
+in reverse-import reach that it lacks appear only as `inferred` rows with
+`reason: structural_reach` under the include. Without coverage those files
+join the list itself as `inferred`. Without a coverage map the directive carries
+`coverage: {status, reason}`; with one, `coverage_analysis` says whether
+coverage is available, partial, degraded or stale.
**`get_change_risk(revspec=...)`** returns `impacted_tests`, computed from the
changed *lines*, so it is narrower:
diff --git a/docs/reference/MCP_RESPONSE_FIELDS.md b/docs/reference/MCP_RESPONSE_FIELDS.md
index be5371392..f797ba259 100644
--- a/docs/reference/MCP_RESPONSE_FIELDS.md
+++ b/docs/reference/MCP_RESPONSE_FIELDS.md
@@ -214,24 +214,23 @@ Per file:
| test gaps, `security_signals` | Test coverage gaps and security findings for the file |
| `resolved`, `unresolved_reason` | A target naming no indexed file: `unsupported_target_kind` (a `module:` id), `directory`, `not_indexed`, `no_such_path`. Counts are omitted, never zeroed |
-Opt-in blocks: `graph` adds typed `dependents` (direct versus transitive), `consumers` (typed contract consumers only), `cross_repo_links`, `impact_surface` and `direct_risks`, with `relationship_analysis` distinguishing an empty analysis from an unavailable or partial one. `churn` adds `change_magnitude`, `risk_type` and `change_pattern`. `scales` adds the unit, range and calibration of every scalar; it is identical on every call, so ask once. A multi-target call also carries `global_hotspots`.
+Opt-in blocks: `tests` adds the PR directive's typed `test_recommendations`. `graph` adds typed `dependents` (direct versus transitive), `consumers` (typed contract consumers only), `cross_repo_links`, `impact_surface` and `direct_risks`, with `relationship_analysis` distinguishing an empty analysis from an unavailable or partial one. `churn` adds `change_magnitude`, `risk_type` and `change_pattern`. `scales` adds the unit, range and calibration of every scalar; it is identical on every call, so ask once. A multi-target call also carries `global_hotspots`.
**PR mode** (`changed_files` passed). The response starts with `directive`:
| Field | Meaning |
|-------|---------|
| `may_break` | Production files in reverse-import reach of the change: candidates for review, not proven breakage |
-| `may_break_tests` | Test files reached the same way, kept separate so tests do not crowd out production impact |
| `missing_cochanges` | Historical co-change partners absent from the change |
-| `test_recommendations` | Typed rows, each with a `basis`: `measured` (the per-test coverage map found the test) or `inferred` (structural reach, not coverage proof) |
-| `tests_to_run`, `tests_to_run_basis` | The test ids or files to run, and whether the list is `measured`, `inferred` or `none`. An empty list with unavailable coverage never means no tests are needed |
+| `tests_to_run`, `tests_to_run_basis` | The test ids or files to run, and whether the list is `measured`, `inferred` or `none`. When coverage exists the list is the measured one; test files in reverse-import reach that it lacks come back as `inferred` rows with `reason: structural_reach` under `include=["tests"]`. An unmeasured list takes those files directly, and a `none` list that gains any becomes `inferred`. An empty list with unavailable coverage never means no tests are needed |
| `tests_to_update` | Up to three test files the change will probably need edited, each `{path, reason}` with `reason` `name_pair` (named for a changed file), `imports` (imports one directly) or `co_change` (changes with one in git history), in that order. Tests to edit, not to run: a file can appear in both lists. Empty when none qualify |
-| `missing_tests`, `files_without_measured_tests` | Changed files with no measured test |
-| `coverage_analysis`, `test_inference_analysis` | Whether each evidence source was available, stale, partial or degraded |
+| `missing_tests`, `files_without_measured_tests` | Changed files with no measured test. `missing_tests` is absent when coverage is unavailable or stale, since an empty list would read as no gaps |
+| `coverage` | `{status, reason}` when there is no per-test coverage map. With a map, `coverage_analysis`, `test_analysis` and `test_inference_analysis` say whether each evidence source was available, stale, partial or degraded |
+| `test_recommendations` | With `include=["tests"]`: typed rows, each with a `basis`: `measured` (the per-test coverage map found the test) or `inferred` (structural reach, not coverage proof) |
| `structural_impact_score` | Uncalibrated 0 to 10 structural heuristic: `localized` below 4, `moderate` 4 to below 7, `broad` 7 and up. Not a breakage probability. `overall_risk_score` is a deprecated alias |
| `next_calls` | `get_change_risk()` for the diff itself, then `get_context` on the first `may_break` files |
-In workspace mode the directive also carries `will_break_consumers` (services in other repos that structurally depend on this one; structural reach only despite the name), `missing_cross_repo_cochanges`, `breaking_changes` (provider incompatibilities since the last index, with impacted consumers; a consumer link does not prove field use), `conformance_violations` and `dependency_cycles`.
+In workspace mode the directive also carries `will_break_consumers` (services in other repos that structurally depend on this one; structural reach only despite the name), `missing_cross_repo_cochanges`, `breaking_changes` (provider incompatibilities since the last index, with impacted consumers; a consumer link does not prove field use), `conformance_violations` and `dependency_cycles`. They are dropped when no workspace is loaded.
Every float is rounded to 4 significant digits. Direct rows' `structural_score` values are in pagerank-weighted hotspot units and are not comparable to `get_change_risk` scores.
diff --git a/packages/cli/src/repowise/cli/commands/risk_cmd.py b/packages/cli/src/repowise/cli/commands/risk_cmd.py
index 27847bde4..2fcae98d8 100644
--- a/packages/cli/src/repowise/cli/commands/risk_cmd.py
+++ b/packages/cli/src/repowise/cli/commands/risk_cmd.py
@@ -171,7 +171,6 @@ def _render_target_risk(projected: dict, requested: tuple[str, ...]) -> None:
console.print(f"\n[bold]Directive[/bold] {escape(str(directive.get('summary', '')))}")
for label, key in (
("May break", "may_break"),
- ("Tests that may break", "may_break_tests"),
("Missing co-changes", "missing_cochanges"),
("Files without tests", "missing_tests"),
("Tests to run", "tests_to_run"),
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
index 78059712f..57aea0e88 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
@@ -52,10 +52,9 @@ _BC_CONSUMER_LIMIT = 5
_CF_VIOLATION_LIMIT = 5
_CF_CYCLE_LIMIT = 3
-#: Caps on the may-break split. Production impact leads the directive, so it
-#: keeps the larger budget; test fallout is a secondary signal capped tighter.
+#: Cap on the production files in reverse-import reach. Tests reached the same
+#: way join ``tests_to_run``.
_MAY_BREAK_LIMIT = 5
-_MAY_BREAK_TESTS_LIMIT = 3
#: Cap on the coverage-backed run-list. A validate-this-change set can be longer
#: than the may-break lists (it is what you actually run), but stays glanceable;
#: the overflow and full typed rows live in pr_blast_radius.test_impact.
@@ -483,6 +482,17 @@ def _project_recommendation(row: dict[str, Any]) -> dict[str, Any]:
return out
+def _unlisted_tests(paths: list[str], rows: list[dict[str, Any]]) -> list[str]:
+ """*paths* that no recommendation row names as its file or id."""
+ named = {
+ name.split("::", 1)[0]
+ for row in rows
+ for name in (row.get("test_id"), row.get("test_file"))
+ if isinstance(name, str)
+ }
+ return [path for path in paths if path not in named]
+
+
def _tests_to_update(
changed_files: list[str],
test_paths: set[str],
@@ -535,6 +545,7 @@ def _build_pr_directive(
alias: str,
*,
full_scale: bool = False,
+ include_tests: bool = False,
) -> None:
"""Assemble PR-mode output: trim co-change lists + blast radius, then build
the directive block. Mutates *response* in place. Behavior preserved.
@@ -581,7 +592,6 @@ def _build_pr_directive(
all_may_break = [p for p in affected if p not in test_paths]
all_may_break_tests = [p for p in affected if p in test_paths]
may_break = all_may_break[:_MAY_BREAK_LIMIT]
- may_break_tests = all_may_break_tests[:_MAY_BREAK_TESTS_LIMIT]
all_missing_cochanges = filter_path_list(
[p for p in (_as_path(e) for e in pr_blast_radius.get("cochange_warnings", [])) if p],
@@ -594,30 +604,47 @@ def _build_pr_directive(
# de-duplication, sorting, exclusions, and the directive cap.
test_impact = pr_blast_radius.get("test_impact") or {}
all_recommendations = list(test_impact.get("recommendations") or [])
- test_recommendations = all_recommendations[:_TESTS_TO_RUN_LIMIT]
- test_recommendations_total = len(all_recommendations)
- recommendations_capped = test_recommendations_total > _TESTS_TO_RUN_LIMIT
# Preserve the measured-first legacy projection and its existing scalar
# domain. The additive typed rows above are the union of evidence kinds.
guarding = pr_blast_radius.get("guarding_tests") or {}
all_tests_to_run = list(guarding.get("tests_to_run") or [])
tests_to_run_basis = guarding.get("basis") or "none"
+ # Tests in reverse-import reach join an unmeasured list. A measured list
+ # names test ids, so they ride as typed rows instead of mixing in files.
+ if tests_to_run_basis != "measured":
+ listed = set(all_tests_to_run)
+ reached = [p for p in all_may_break_tests if p not in listed]
+ if reached:
+ all_tests_to_run += reached
+ tests_to_run_basis = "inferred"
+ else:
+ all_recommendations += [
+ {"test_id": path, "basis": "inferred", "reason": "structural_reach"}
+ for path in _unlisted_tests(all_may_break_tests, all_recommendations)
+ ]
+ test_recommendations = all_recommendations[:_TESTS_TO_RUN_LIMIT]
+ test_recommendations_total = len(all_recommendations)
+ recommendations_capped = test_recommendations_total > _TESTS_TO_RUN_LIMIT
tests_to_run = all_tests_to_run[:_TESTS_TO_RUN_LIMIT]
tests_to_run_total = len(all_tests_to_run)
tests_capped = tests_to_run_total > _TESTS_TO_RUN_LIMIT
- if not all_recommendations:
- tests_to_run_suffix = ""
- else:
+ tests_to_run_suffix = (
+ f" {tests_to_run_total} test(s) to run, {tests_to_run_basis}." if tests_to_run_total else ""
+ )
+ if include_tests and all_recommendations:
basis_totals = test_impact.get("recommendations_by_primary_basis") or {}
measured_total = int(basis_totals.get("measured", 0))
- inferred_total = int(basis_totals.get("inferred", 0))
- tests_to_run_suffix = (
+ # Plus the structural-reach rows added above, which the analyzer never counted.
+ inferred_total = int(basis_totals.get("inferred", 0)) + (
+ test_recommendations_total - len(test_impact.get("recommendations") or [])
+ )
+ tests_to_run_suffix += (
f" {test_recommendations_total} test recommendation(s): {measured_total} measured "
f"and {inferred_total} inferred, not coverage-proven candidate(s); "
f"each row carries its basis."
)
- if recommendations_capped:
+ if include_tests and recommendations_capped:
tests_to_run_suffix += (
f" Showing {_TESTS_TO_RUN_LIMIT} of {test_recommendations_total}; omitted "
"typed rows are captured by the response omission marker."
@@ -647,8 +674,8 @@ def _build_pr_directive(
missing_tests = []
missing_tests_total = 0
missing_tests_summary = (
- f"Coverage analysis is {coverage.get('status', 'unavailable')}; "
- "missing_tests is withheld rather than treated as empty evidence."
+ f"Coverage analysis is {coverage.get('status', 'unavailable')}, so test gaps "
+ "are withheld rather than reported as none."
)
gov_count = len(governance_risk)
@@ -726,7 +753,6 @@ def _build_pr_directive(
directive = {
"may_break": may_break,
- "may_break_tests": may_break_tests,
"missing_cochanges": missing_cochanges,
"missing_tests": missing_tests,
"missing_tests_semantics": "changed_file_test_gap_compatibility_projection",
@@ -745,16 +771,36 @@ def _build_pr_directive(
"tests_to_run_omitted": tests_to_run_total - len(tests_to_run),
# Tests to edit, not to run; a file can sit in both lists.
"tests_to_update": all_tests_to_update,
- "test_recommendations": test_recommendations,
- "test_recommendations_total": test_recommendations_total,
- "test_recommendations_emitted": len(test_recommendations),
- "test_recommendations_truncated": recommendations_capped,
- "test_recommendations_omitted": max(
- 0, test_recommendations_total - len(test_recommendations)
+ **(
+ {
+ "test_recommendations": test_recommendations,
+ "test_recommendations_total": test_recommendations_total,
+ "test_recommendations_emitted": len(test_recommendations),
+ "test_recommendations_truncated": recommendations_capped,
+ "test_recommendations_omitted": max(
+ 0, test_recommendations_total - len(test_recommendations)
+ ),
+ }
+ if include_tests
+ else {}
+ ),
+ # Without coverage the three analysis blocks are nulls and zeros.
+ **(
+ {
+ "test_analysis": test_impact.get("analysis") or {"status": "unavailable"},
+ "coverage_analysis": coverage,
+ "test_inference_analysis": (
+ test_impact.get("inference") or {"status": "unavailable"}
+ ),
+ }
+ if coverage.get("status", "unavailable") != "unavailable"
+ else {
+ "coverage": {
+ "status": coverage.get("status") or "unavailable",
+ "reason": coverage.get("reason") or "no_per_test_coverage_map",
+ }
+ }
),
- "test_analysis": test_impact.get("analysis") or {"status": "unavailable"},
- "coverage_analysis": coverage,
- "test_inference_analysis": test_impact.get("inference") or {"status": "unavailable"},
"test_unknown_files": [],
"will_break_consumers": will_break_consumers,
"will_break_consumers_semantics": "structural_reach_only",
@@ -778,7 +824,6 @@ def _build_pr_directive(
"summary": (
f"PR touches {len(changed_files)} file(s). "
f"~{len(may_break)} downstream file(s) may be affected, "
- f"{len(may_break_tests)} test(s) may break, "
f"{len(missing_cochanges)} historical co-changer(s) missing, "
f"{missing_tests_summary}"
f"{tests_to_run_suffix}{gov_suffix}{xr_suffix}{bc_suffix}{cf_suffix}"
@@ -786,12 +831,15 @@ def _build_pr_directive(
}
for key, population, cap in (
("may_break", all_may_break, _MAY_BREAK_LIMIT),
- ("may_break_tests", all_may_break_tests, _MAY_BREAK_TESTS_LIMIT),
("missing_cochanges", all_missing_cochanges, 3),
("missing_tests", all_missing_tests if coverage_usable else [], 3),
("tests_to_run", all_tests_to_run, _TESTS_TO_RUN_LIMIT),
("tests_to_update", all_tests_to_update, _TESTS_TO_UPDATE_LIMIT),
- ("test_recommendations", all_recommendations, _TESTS_TO_RUN_LIMIT),
+ *(
+ (("test_recommendations", all_recommendations, _TESTS_TO_RUN_LIMIT),)
+ if include_tests
+ else ()
+ ),
(
"files_without_measured_tests",
list(test_impact.get("files_without_measured_tests") or []),
@@ -860,4 +908,32 @@ def _build_pr_directive(
directive[f"{key}_truncated"] = True
directive[f"{key}_omitted"] = total - emitted_count
+ # A withheld list reads as "no gaps", and the cross-repo families are
+ # always empty outside a workspace.
+ if not coverage_usable:
+ _drop_family(directive, "missing_tests")
+ if not _is_workspace_mode():
+ for stem in _WORKSPACE_FAMILIES:
+ if not directive.get(stem):
+ _drop_family(directive, stem)
+ if "will_break_consumers" not in directive and (
+ "missing_cross_repo_cochanges" not in directive
+ ):
+ directive.pop("cross_repo_relationship_analysis", None)
+
response["directive"] = directive
+
+
+_WORKSPACE_FAMILIES = (
+ "will_break_consumers",
+ "missing_cross_repo_cochanges",
+ "breaking_changes",
+ "conformance_violations",
+ "dependency_cycles",
+)
+
+
+def _drop_family(directive: dict[str, Any], stem: str) -> None:
+ """Remove *stem* and its ``<stem>_*`` counts and labels."""
+ for key in [k for k in directive if k == stem or k.startswith(f"{stem}_")]:
+ del directive[key]
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py b/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
index a96c2672b..5f3075ca9 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/get_risk.py
@@ -81,7 +81,10 @@ _TARGET_CARD_INCLUDES: dict[str, tuple[str, ...]] = {
}
_BLAST_INCLUDES: dict[str, tuple[str, ...]] = {"graph": ("direct_risks",)}
#: Per-field units and calibration. Identical on every call, so it is opt-in.
-_INCLUDE_BLOCKS = frozenset(_TARGET_CARD_INCLUDES) | frozenset(_BLAST_INCLUDES) | {"scales"}
+#: ``tests`` adds the PR directive's typed ``test_recommendations`` rows.
+_INCLUDE_BLOCKS = (
+ frozenset(_TARGET_CARD_INCLUDES) | frozenset(_BLAST_INCLUDES) | {"scales", "tests"}
+)
def _drop_opt_in_blocks(response: dict, include: set[str]) -> None:
@@ -308,6 +311,7 @@ async def _lead_with_pr_directive(
exclude_spec: Any,
collector: OmissionCollector,
full_scale: bool,
+ include_tests: bool,
) -> dict:
governance_risk = await _governance_directive(ctx, changed_files)
_build_pr_directive(
@@ -320,6 +324,7 @@ async def _lead_with_pr_directive(
evidence.test_paths,
ctx.alias,
full_scale=full_scale,
+ include_tests=include_tests,
)
# Insertion order is the serialized order, and PR mode leads with the directive.
return {"directive": response.pop("directive"), **response}
@@ -359,9 +364,9 @@ async def get_risk(
runtime breakage. The response also includes security
findings. Pass changed_files for PR mode: the response leads with a
directive block (may_break, missing_cochanges, missing_tests,
- tests_to_run, tests_to_update) — read it first. Each test_recommendations row carries a
- measured or inferred basis, and coverage availability is explicit. To
- score a commit or ``base..head`` range instead, use ``get_change_risk``.
+ tests_to_run, tests_to_update) — read it first. ``tests_to_run_basis`` says
+ measured or inferred. To score a commit or ``base..head`` range instead,
+ use ``get_change_risk``.
In PR mode ``structural_impact_score`` is an uncalibrated 0-10 structural
heuristic, never a runtime-breakage probability; ``overall_risk_score`` is
@@ -376,8 +381,8 @@ async def get_risk(
targets: file paths to assess; defaults to changed_files.
repo: usually omitted.
changed_files: PR-changed files for blast-radius mode.
- include: opt-in blocks - "graph", "churn", "scales" (units and
- calibration for every scalar; identical per call, so ask once).
+ include: opt-in blocks - "graph", "churn", "tests" (typed test
+ rows), "scales" (units and calibration; identical per call).
"""
if repo == "all":
return _unsupported_repo_all("get_risk")
@@ -413,6 +418,7 @@ async def get_risk(
exclude_spec,
collector,
full_scale="scales" in include_set,
+ include_tests="tests" in include_set,
)
elif len(targets) > 1:
# Ambient hotspots orient a multi-file request; beside one named file they are noise.
diff --git a/scripts/measure_mcp_response_sizes.py b/scripts/measure_mcp_response_sizes.py
index 0b6058d1a..ced418be3 100644
--- a/scripts/measure_mcp_response_sizes.py
+++ b/scripts/measure_mcp_response_sizes.py
@@ -50,6 +50,7 @@ CASES: list[tuple[str, str, dict[str, Any]]] = [
("get_health", "all-includes", {"include": ["files", "refactoring", "performance"], "limit": 100}),
("get_risk", "one-target", {"targets": ["{file}"]}),
("get_risk", "pr-mode", {"targets": ["{file}"], "changed_files": ["{file}", "{file2}"]}),
+ ("get_risk", "pr-mode-tests", {"targets": ["{file}"], "changed_files": ["{file}", "{file2}"], "include": ["tests"]}),
("get_change_risk", "head", {"revspec": "HEAD"}),
("get_why", "targets", {"targets": ["{file}"]}),
("get_dead_code", "default", {}),
diff --git a/tests/fixtures/mcp/tool_response_shapes.json b/tests/fixtures/mcp/tool_response_shapes.json
index 63220fcac..8eaad1edd 100644
--- a/tests/fixtures/mcp/tool_response_shapes.json
+++ b/tests/fixtures/mcp/tool_response_shapes.json
@@ -1367,33 +1367,10 @@
}
},
"directive": {
- "breaking_changes": [],
- "breaking_changes_truncated": null,
- "conformance_violations": [],
- "coverage_analysis": {
- "changed_files_total": null,
- "changed_files_with_measured_tests": null,
- "changed_files_without_measured_tests": null,
- "freshness": {
- "indexed_commit": null,
- "ingested_commit": null,
- "reason": null,
- "status": null
- },
- "ingested_at": null,
- "map_present": null,
- "pair_count": null,
- "reason": null,
- "source_file_count": null,
- "source_format": null,
- "status": null,
- "test_count": null
- },
- "cross_repo_relationship_analysis": {
+ "coverage": {
"reason": null,
"status": null
},
- "dependency_cycles": [],
"files_without_measured_tests": [
null
],
@@ -1404,14 +1381,6 @@
"may_break_emitted": null,
"may_break_omitted": null,
"may_break_reduced_reason": null,
- "may_break_tests": [
- null
- ],
- "may_break_tests_emitted": null,
- "may_break_tests_omitted": null,
- "may_break_tests_reduced_reason": null,
- "may_break_tests_total": null,
- "may_break_tests_truncated": null,
"may_break_total": null,
"may_break_truncated": null,
"missing_cochanges": [
@@ -1422,16 +1391,6 @@
"missing_cochanges_reduced_reason": null,
"missing_cochanges_total": null,
"missing_cochanges_truncated": null,
- "missing_cross_repo_cochanges": [],
- "missing_cross_repo_cochanges_emitted": null,
- "missing_cross_repo_cochanges_total": null,
- "missing_cross_repo_cochanges_truncated": null,
- "missing_tests": [],
- "missing_tests_emitted": null,
- "missing_tests_omitted": null,
- "missing_tests_semantics": null,
- "missing_tests_total": null,
- "missing_tests_truncated": null,
"next_calls": [
{
"arguments": {},
@@ -1442,22 +1401,6 @@
}
],
"summary": null,
- "test_analysis": {
- "basis_categories": [
- null
- ],
- "degraded": null,
- "partial": null,
- "stale": null,
- "status": null
- },
- "test_inference_analysis": {
- "candidates_before_dedup": null,
- "changed_files_total": null,
- "changed_files_with_candidates": null,
- "reason": null,
- "status": null
- },
"test_recommendations": [
{
"basis": null,
@@ -1492,13 +1435,7 @@
"path": null,
"reason": null
}
- ],
- "will_break_consumers": [],
- "will_break_consumers_deprecated": null,
- "will_break_consumers_emitted": null,
- "will_break_consumers_semantics": null,
- "will_break_consumers_total": null,
- "will_break_consumers_truncated": null
+ ]
},
"omission_marker": null,
"pr_blast_radius": {
diff --git a/tests/unit/cli/test_search_collapse.py b/tests/unit/cli/test_search_collapse.py
index ed9559c9e..e72adc329 100644
--- a/tests/unit/cli/test_search_collapse.py
+++ b/tests/unit/cli/test_search_collapse.py
@@ -171,7 +171,6 @@ PR_RISK_PAYLOAD = {
"targets": RISK_PAYLOAD["targets"],
"directive": {
"may_break": ["packages/core/src/repowise/core/pipeline/orchestrator.py"],
- "may_break_tests": ["tests/unit/persistence/test_models.py"],
"missing_cochanges": ["packages/core/src/repowise/core/persistence/models.py"],
"missing_tests": ["packages/core/src/repowise/core/pipeline/persist.py"],
"tests_to_run": ["tests/unit/pipeline/test_persist.py::test_tombstone"],
diff --git a/tests/unit/server/mcp/test_pr_test_impact_semantics.py b/tests/unit/server/mcp/test_pr_test_impact_semantics.py
index eceeb3521..21b0eedc7 100644
--- a/tests/unit/server/mcp/test_pr_test_impact_semantics.py
+++ b/tests/unit/server/mcp/test_pr_test_impact_semantics.py
@@ -88,7 +88,9 @@ async def test_sealed_pr_payload_is_directive_first_typed_and_count_exact(setup_
)
await session.flush()
- payload = await get_risk(["src/measured.py"], changed_files=changed, include=["graph"])
+ payload = await get_risk(
+ ["src/measured.py"], changed_files=changed, include=["graph", "tests"]
+ )
external = json.loads(json.dumps(payload))
directive = external["directive"]
full = external["pr_blast_radius"]["test_impact"]
diff --git a/tests/unit/server/mcp/test_recoverable_caps_wire_contracts.py b/tests/unit/server/mcp/test_recoverable_caps_wire_contracts.py
index 03ad52e7d..d4b3c5dc4 100644
--- a/tests/unit/server/mcp/test_recoverable_caps_wire_contracts.py
+++ b/tests/unit/server/mcp/test_recoverable_caps_wire_contracts.py
@@ -402,18 +402,18 @@ async def test_risk_real_adversarial_wire_recovers_each_directive_lane(
monkeypatch.setattr(PRBlastRadiusAnalyzer, "analyze_files", sealed_analyze)
changed = [f"src/gap_{i}.py" for i in range(6)]
result = await tool_middleware(get_risk)(
- ["src/auth/service.py"], changed_files=changed
+ ["src/auth/service.py"], changed_files=changed, include=["tests"]
)
- _assert_wire(result, "directive", DEFAULT_RESPONSE_CHARS)
+ _assert_wire(result, "directive", EXPANDED_RESPONSE_CHARS)
directive = result["directive"]
expected = {
"may_break": 9,
- "may_break_tests": 8,
"missing_cochanges": 8,
"missing_tests": 6,
"tests_to_run": 14,
- "test_recommendations": 15,
+ # 15 analyzer rows plus the 8 reached tests the measured list lacks.
+ "test_recommendations": 23,
"files_without_measured_tests": 12,
"test_unknown_files": 12,
}
diff --git a/tests/unit/server/mcp/test_risk.py b/tests/unit/server/mcp/test_risk.py
index 6464efe5e..3be034120 100644
--- a/tests/unit/server/mcp/test_risk.py
+++ b/tests/unit/server/mcp/test_risk.py
@@ -152,7 +152,7 @@ async def test_get_risk_stable_file(setup_mcp):
@pytest.mark.asyncio
async def test_get_risk_pr_directive_splits_test_breakage(setup_mcp):
- """PR mode splits test-file fallout out of may_break into may_break_tests (#672)."""
+ """PR mode keeps test files out of may_break; they join tests_to_run (#672)."""
from repowise.server.mcp_server import get_risk
# Pass changed_files to trigger PR mode + blast-radius directive.
@@ -161,14 +161,12 @@ async def test_get_risk_pr_directive_splits_test_breakage(setup_mcp):
# middleware.py imports service.py → production breakage.
assert "src/auth/middleware.py" in directive["may_break"]
- assert "src/auth/middleware.py" not in directive["may_break_tests"]
+ assert "src/auth/middleware.py" not in directive["tests_to_run"]
# test_service.py imports service.py but is is_test=True → segmented out.
- assert "tests/test_service.py" in directive["may_break_tests"]
+ assert "tests/test_service.py" in directive["tests_to_run"]
assert "tests/test_service.py" not in directive["may_break"]
-
- # Summary reflects the test count.
- assert "test(s) may break" in directive["summary"]
+ assert "may_break_tests" not in directive
# The savings estimator reads these lists by name; a rename that misses it
# undercounts silently rather than raising.
@@ -257,6 +255,12 @@ async def test_get_risk_pr_directive_surfaces_coverage_backed_tests_to_run(setup
# The graph also reaches this file, and must not dilute a measured answer.
assert directive["tests_to_run_basis"] == "measured"
assert directive["tests_to_run_kind"] == "test_id"
+ assert "2 test(s) to run, measured." in directive["summary"]
+ directive = (
+ await get_risk(
+ ["src/auth/service.py"], changed_files=["src/auth/service.py"], include=["tests"]
+ )
+ )["directive"]
assert "2 measured" in directive["summary"]
assert {row["basis"] for row in directive["test_recommendations"]} == {
"measured",
@@ -302,6 +306,7 @@ async def test_get_risk_tests_to_run_ranks_by_files_reached(setup_mcp, session):
result = await get_risk(
["src/auth/service.py"],
changed_files=["src/auth/service.py", "src/auth/token.py"],
+ include=["tests"],
)
assert result["directive"]["tests_to_run"] == [
@@ -350,24 +355,28 @@ async def test_get_risk_pr_directive_falls_back_to_the_graph_without_a_map(setup
assert directive["tests_to_run"] == ["tests/test_service.py"]
assert directive["tests_to_run_basis"] == "inferred"
assert directive["tests_to_run_kind"] == "test_file"
- assert "inferred, not coverage-proven" in directive["summary"]
+ assert "1 test(s) to run, inferred." in directive["summary"]
assert "coverage-backed test(s) guard the change" not in directive["summary"]
@pytest.mark.asyncio
-async def test_get_risk_pr_directive_names_no_tests_when_nothing_reaches(setup_mcp):
- """Neither map nor graph -> an empty list and a ``none`` basis, not a guess."""
+async def test_get_risk_pr_directive_runs_tests_in_import_reach(setup_mcp):
+ """No map and no call-graph test -> the tests in reverse-import reach, inferred.
+
+ ``tests/test_service.py`` reaches ``models.py`` through ``service.py``. With
+ no coverage the gap list is withheld, not sent empty.
+ """
from repowise.server.mcp_server import get_risk
result = await get_risk(["src/db/models.py"], changed_files=["src/db/models.py"])
directive = result["directive"]
- assert directive["tests_to_run"] == []
- assert directive["tests_to_run_basis"] == "none"
- assert directive["tests_to_run_kind"] is None
- assert directive["missing_tests"] == []
- assert directive["coverage_analysis"]["status"] == "unavailable"
- assert "missing_tests is withheld" in directive["summary"]
+ assert directive["tests_to_run"] == ["tests/test_service.py"]
+ assert directive["tests_to_run_basis"] == "inferred"
+ assert directive["tests_to_run_kind"] == "test_file"
+ assert "missing_tests" not in directive
+ assert directive["coverage"]["status"] == "unavailable"
+ assert "test gaps are withheld" in directive["summary"]
@pytest.mark.asyncio
@@ -491,6 +500,122 @@ def test_build_pr_directive_caps_tests_to_update_at_three():
assert directive["tests_to_update_omitted"] == 2
+def _directive(blast, tests=(), **kwargs):
+ from repowise.server.mcp_server._budget import OmissionCollector
+ from repowise.server.mcp_server.tool_risk import directives
+
+ response: dict = {"targets": {}}
+ directives._build_pr_directive(
+ response, blast, ["src/core.py"], None, OmissionCollector("get_risk"), [],
+ set(tests), "repo", **kwargs,
+ )
+ return response["directive"]
+
+
+def test_pr_directive_folds_reached_tests_into_tests_to_run():
+ blast = {
+ "transitive_affected": [{"path": "tests/test_api.py"}, {"path": "src/api.py"}],
+ "guarding_tests": {"tests_to_run": ["tests/test_core.py"], "basis": "inferred"},
+ }
+ directive = _directive(blast, {"tests/test_api.py", "tests/test_core.py"})
+ assert directive["tests_to_run"] == ["tests/test_core.py", "tests/test_api.py"]
+ assert directive["tests_to_run_basis"] == "inferred"
+ assert directive["may_break"] == ["src/api.py"]
+ assert "may_break_tests" not in directive
+
+
+def test_pr_directive_keeps_a_measured_run_list_unmixed():
+ blast = {
+ "transitive_affected": [{"path": "tests/test_api.py"}],
+ "guarding_tests": {"tests_to_run": ["tests/test_core.py::test_a"], "basis": "measured"},
+ }
+ directive = _directive(blast, {"tests/test_api.py"})
+ assert directive["tests_to_run"] == ["tests/test_core.py::test_a"]
+ assert directive["tests_to_run_basis"] == "measured"
+ # The reached test is not dropped: it rides as a typed row on request.
+ typed = _directive(blast, {"tests/test_api.py"}, include_tests=True)
+ assert typed["tests_to_run"] == ["tests/test_core.py::test_a"]
+ assert typed["test_recommendations"] == [
+ {"test_id": "tests/test_api.py", "basis": "inferred", "reason": "structural_reach"}
+ ]
+
+
+def test_pr_directive_measured_rows_do_not_repeat_a_reached_test():
+ row = {"test_id": "tests/test_api.py::test_a", "basis": "measured", "evidence": []}
+ blast = {
+ "transitive_affected": [{"path": "tests/test_api.py"}],
+ "guarding_tests": {"tests_to_run": ["tests/test_api.py::test_a"], "basis": "measured"},
+ "test_impact": {"recommendations": [row]},
+ }
+ typed = _directive(blast, {"tests/test_api.py"}, include_tests=True)
+ assert [r["test_id"] for r in typed["test_recommendations"]] == ["tests/test_api.py::test_a"]
+
+
+def test_pr_directive_none_basis_with_reached_tests_becomes_inferred():
+ blast = {
+ "transitive_affected": [{"path": "tests/test_api.py"}],
+ "guarding_tests": {"tests_to_run": [], "basis": "none"},
+ }
+ directive = _directive(blast, {"tests/test_api.py"})
+ assert directive["tests_to_run"] == ["tests/test_api.py"]
+ assert directive["tests_to_run_basis"] == "inferred"
+ assert directive["tests_to_run_kind"] == "test_file"
+
+
+def test_pr_directive_serves_test_recommendations_on_request():
+ row = {"test_id": "tests/test_core.py", "basis": "inferred", "evidence": []}
+ blast = {"test_impact": {"recommendations": [row]}}
+ assert not any(k.startswith("test_recommendations") for k in _directive(blast))
+ directive = _directive(blast, include_tests=True)
+ assert directive["test_recommendations"][0]["test_id"] == "tests/test_core.py"
+ assert directive["test_recommendations_total"] == 1
+
+
+def test_pr_directive_without_coverage_drops_the_empty_families():
+ directive = _directive({"test_impact": {"coverage": {"status": "unavailable"}}})
+ assert directive["coverage"] == {
+ "status": "unavailable",
+ "reason": "no_per_test_coverage_map",
+ }
+ assert directive["tests_to_run"] == []
+ assert directive["tests_to_run_basis"] == "none"
+ assert directive["tests_to_run_kind"] is None
+ gone = (
+ "missing_tests",
+ "coverage_analysis",
+ "test_analysis",
+ "test_inference_analysis",
+ "will_break_consumers",
+ "missing_cross_repo_cochanges",
+ "cross_repo_relationship_analysis",
+ "breaking_changes",
+ "conformance_violations",
+ "dependency_cycles",
+ )
+ assert not [k for k in directive if k.startswith(gone)]
+
+
+def test_pr_directive_with_coverage_keeps_the_full_blocks():
+ coverage = {"status": "available", "map_present": True, "freshness": {"status": "fresh"}}
+ blast = {"test_impact": {"coverage": coverage}, "test_gaps": ["src/core.py"]}
+ directive = _directive(blast)
+ assert "coverage" not in directive
+ assert directive["coverage_analysis"] == coverage
+ assert directive["missing_tests"] == ["src/core.py"]
+ assert {"test_analysis", "test_inference_analysis"} <= directive.keys()
+
+
+async def test_get_risk_include_tests_returns_the_typed_rows(setup_mcp):
+ from repowise.server.mcp_server import get_risk
+
+ files = ["src/auth/service.py"]
+ plain = (await get_risk(files, changed_files=files))["directive"]
+ typed = (await get_risk(files, changed_files=files, include=["tests"]))["directive"]
+ assert "test_recommendations" not in plain
+ assert typed["test_recommendations"]
+
+
@pytest.mark.asyncio
async def test_get_risk_pr_payload_serializes_directive_first(setup_mcp):
"""The exact external JSON order is actionable before any dossier."""
@@ -510,9 +635,11 @@ async def test_get_risk_pr_payload_serializes_directive_first(setup_mcp):
async def test_get_risk_test_compatibility_projection_cannot_contradict_typed_rows(setup_mcp):
from repowise.server.mcp_server import get_risk
- directive = (await get_risk(["src/auth/service.py"], changed_files=["src/auth/service.py"]))[
- "directive"
- ]
+ directive = (
+ await get_risk(
+ ["src/auth/service.py"], changed_files=["src/auth/service.py"], include=["tests"]
+ )
+ )["directive"]
recommendations = directive["test_recommendations"]
assert directive["tests_to_run"] == [row["test_id"] for row in recommendations]
@@ -667,7 +794,7 @@ async def test_get_risk_names_an_unknown_include_rather_than_applying_it(setup_m
result = await get_risk(["src/auth/service.py"], include=["graph", "nonsense"])
assert "impact_surface" in result["targets"]["src/auth/service.py"]
assert result["ignored_arguments"] == [
- {"argument": "include", "values": ["nonsense"], "valid": ["churn", "graph", "scales"]}
+ {"argument": "include", "values": ["nonsense"], "valid": ["churn", "graph", "scales", "tests"]}
]
@@ -727,7 +854,9 @@ async def test_get_risk_directive_points_at_the_full_run_list_when_capped(setup_
)
await session.flush()
- result = await get_risk(["src/auth/service.py"], changed_files=["src/auth/service.py"])
+ result = await get_risk(
+ ["src/auth/service.py"], changed_files=["src/auth/service.py"], include=["tests"]
+ )
directive = result["directive"]
assert len(directive["tests_to_run"]) == _TESTS_TO_RUN_LIMIT