Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

fix(impacted-tests): count tests reached only through long import chains apart (#4325)

Raghav Chamadiya2fb3ecf3a5cd
diff --git a/docs/reference/CLI_REFERENCE.md b/docs/reference/CLI_REFERENCE.md
index 5bd6ecad6..2042404c7 100644
--- a/docs/reference/CLI_REFERENCE.md
+++ b/docs/reference/CLI_REFERENCE.md
@@ -1125,7 +1125,10 @@ or `none` (no index), and a per-test `selected.why`: the first changed file and
evidence that selected it, or why it runs with every subset. `selected.broadcast`
maps each selected test file that runs only because a conftest or test-package
`__init__.py` on a route keeps every test under it to that file; agent views
-count these apart from the tests the change reaches. When it runs everything:
+count these apart from the tests the change reaches. `selected.import_closure`
+maps each selected test file the change reaches only through imports, more than
+one file away and with no call path, to its distance; these are counted apart
+too. When it runs everything:
[CI](../start/CI.md#selecting-the-tests-a-change-needs).
**Run order.** `--format args` and `json` list the selected tests likeliest to fail
diff --git a/docs/reference/MCP_RESPONSE_FIELDS.md b/docs/reference/MCP_RESPONSE_FIELDS.md
index 44c62c8f7..f0a89bae2 100644
--- a/docs/reference/MCP_RESPONSE_FIELDS.md
+++ b/docs/reference/MCP_RESPONSE_FIELDS.md
@@ -247,7 +247,7 @@ Opt-in blocks: `changes` adds `recent_changes` (and `recent_changes_total`), sha
|-------|---------|
| `may_break` | Production files in reverse-import reach of the change: candidates for review, not proven breakage |
| `missing_cochanges` | Historical co-change partners absent from the change |
-| `tests_to_run`, `tests_to_run_basis` | The tests the change needs, from the same selection `repowise impacted-tests` makes, capped at ten, in `repowise impacted-tests`' [run order](CLI_REFERENCE.md#repowise-impacted-tests-revspec). `tests_to_run_basis` is `measured` when coverage decided a changed file, `inferred` when only the graph or a rule did, `none` when nothing named a test. `tests_to_run_why` gives the reason for each listed file (the changed file and evidence, or `runs with every subset: ...`). `tests_run_all` true means no subset can be vouched for and `tests_run_all_reasons` says why; the list is then what to run first. `tests_broadcast_total` counts the tests that run only because a conftest on a route keeps every test under it; they are left out of the list and its total. A failed selection carries `tests_status` and `tests_status_reason` and names nothing |
+| `tests_to_run`, `tests_to_run_basis` | The tests the change needs, from the same selection `repowise impacted-tests` makes, capped at ten, in `repowise impacted-tests`' [run order](CLI_REFERENCE.md#repowise-impacted-tests-revspec). `tests_to_run_basis` is `measured` when coverage decided a changed file, `inferred` when only the graph or a rule did, `none` when nothing named a test. `tests_to_run_why` gives the reason for each listed file (the changed file and evidence, or `runs with every subset: ...`). `tests_run_all` true means no subset can be vouched for and `tests_run_all_reasons` says why; the list is then what to run first. `tests_broadcast_total` counts the tests that run only because a conftest on a route keeps every test under it, and `tests_import_closure_total` those the change reaches only through imports, more than one file away, with no call path; both are left out of the list and its total and still run. A failed selection carries `tests_status` and `tests_status_reason` and names nothing |
| `tests_to_update` | Up to three test files the change will probably need edited, each `{path, reason}` with `reason` `name_pair` (named for a changed file), `imports` (its own code imports one; a conftest or setup file reaching it does not count) or `co_change` (changes with one in git history), in that order. Only tests the selection runs because of a changed file and that a runner collects. Tests to edit, not to run: a file can appear in both lists. Empty when none qualify |
| `missing_tests`, `files_without_measured_tests` | Changed files with no measured test. `missing_tests` is absent when coverage is unavailable or stale, since an empty list would read as no gaps |
| `coverage` | `{status, reason}` when there is no per-test coverage map. With a map, `coverage_analysis`, `test_analysis` and `test_inference_analysis` say whether each evidence source was available, stale, partial or degraded |
@@ -286,7 +286,7 @@ Every float is rounded to 4 significant digits. Direct rows' `structural_score`
`files` lists up to 10 analysed files, largest structure move first, each with `base_score`, `head_score`, `nloc_base` and `nloc_head`; a side the file is missing on reads null. `files_total` counts them all. `include=["diagnostics"]` adds `base_structure` and `head_structure` (the structure deduction). `score_basis` is `structure_only`: each score is 10 minus the structure deduction, the reading `get_health(counts="code_shape")` gives, from two passes that read no git history or coverage. A deleted file has a base-only row, with `moved_to` naming where its code went when it was split. `score_change` averages files scored on both sides, weighted by base nloc so a file that grows while improving keeps its gain: `base`, `head`, `delta` (two decimals), `structure_delta` (the deduction change, which still moves when a file sits on the 1.0 floor) and `files_compared`.
-**impacted_tests.** The selection `repowise impacted-tests` makes, over every path the change touched whatever the score's filters, less the repowise output the score also leaves out (the index, and a managed file whose only change is repowise's block or entry), which the CLI still reads when tracked. `status` is `selected`, `run_all` or `degraded` (or `no_index`, `unknown`, `config_invalid`, `timeout`, `busy`, `no_source_line_changes` when nothing was selected, each with `run_all` true; `timeout` means the selection took over 30 seconds and `busy` that one for this repository is still running, and `repowise impacted-tests` gives the answer); `run_all` true means no subset can be vouched for, with up to three `reasons` (`reasons_total` counts them all). `degraded` means the index predates files changed outside the change and the selection holds over half the test files; the list stands, `summary` says so, and the directive asks for every test. `tests_to_run` (capped at ten, `total` and `truncated`) is in `repowise impacted-tests`' [run order](CLI_REFERENCE.md#repowise-impacted-tests-revspec), the `always_run_total` tests that run with every subset last. `broadcast_total` counts the tests that run only because a conftest on a route keeps every test under it; they are left out of `tests_to_run` and `total`. When most of the suite is to run anyway, the list's tail is not stored for recovery. With `run_all` the list is what to run first. `tests_to_run_kind` is `test_id` when a coverage node id is listed, else `test_file`. `basis` is `measured` when coverage decided a changed file, `inferred` when only the graph or a rule did, `none` when nothing named a test. `include=["tests"]` adds `why` (each listed file's reason) and `basis_by_file`, the evidence per changed file (`coverage`, `changed-test`, `call-graph`, `import-graph`, `conftest`, `full-run`, `no-tests-needed`, ...). With a coverage map, `map_present` is true and `line_coverage` buckets the scored files' changed lines, matched on the side of the diff the map was measured at (by file when it was measured at neither): `untested_changes`, `stale_test_candidates` (covered lines whose test file is absent from the diff), `covered`, `no_coverage_data`. Build the measured map with `pytest --cov=<package> --cov-context=test` (needs pytest-cov) and `repowise coverage add`.
+**impacted_tests.** The selection `repowise impacted-tests` makes, over every path the change touched whatever the score's filters, less the repowise output the score also leaves out (the index, and a managed file whose only change is repowise's block or entry), which the CLI still reads when tracked. `status` is `selected`, `run_all` or `degraded` (or `no_index`, `unknown`, `config_invalid`, `timeout`, `busy`, `no_source_line_changes` when nothing was selected, each with `run_all` true; `timeout` means the selection took over 30 seconds and `busy` that one for this repository is still running, and `repowise impacted-tests` gives the answer); `run_all` true means no subset can be vouched for, with up to three `reasons` (`reasons_total` counts them all). `degraded` means the index predates files changed outside the change and the selection holds over half the test files; the list stands, `summary` says so, and the directive asks for every test. `tests_to_run` (capped at ten, `total` and `truncated`) is in `repowise impacted-tests`' [run order](CLI_REFERENCE.md#repowise-impacted-tests-revspec), the `always_run_total` tests that run with every subset last. `broadcast_total` counts the tests that run only because a conftest on a route keeps every test under it, and `import_closure_total` those the change reaches only through imports, more than one file away, with no call path; both are left out of `tests_to_run` and `total` and still run. When most of the suite is to run anyway, the list's tail is not stored for recovery. With `run_all` the list is what to run first. `tests_to_run_kind` is `test_id` when a coverage node id is listed, else `test_file`. `basis` is `measured` when coverage decided a changed file, `inferred` when only the graph or a rule did, `none` when nothing named a test. `include=["tests"]` adds `why` (each listed file's reason) and `basis_by_file`, the evidence per changed file (`coverage`, `changed-test`, `call-graph`, `import-graph`, `conftest`, `full-run`, `no-tests-needed`, ...). With a coverage map, `map_present` is true and `line_coverage` buckets the scored files' changed lines, matched on the side of the diff the map was measured at (by file when it was measured at neither): `untested_changes`, `stale_test_candidates` (covered lines whose test file is absent from the diff), `covered`, `no_coverage_data`. Build the measured map with `pytest --cov=<package> --cov-context=test` (needs pytest-cov) and `repowise coverage add`.
**patch_coverage.** Present when the index stores coverage. With a gate set and coverage measured at the change's head, it is the same computation and JSON shape `repowise coverage check --format json` gates on. Otherwise it reads `status` (`measured` or `not_measured`), `freshness` and the nonzero `file_counts`, plus the totals and `files` when a changed file was measured; `include=["diagnostics"]` keeps the full shape. `patch_coverage_pct` is null when no changed line is executable; files the report never names read `not_in_report`. `scope.freshness` is `stale` when coverage was measured at another commit than the change's head; then a file git shows changed since that commit reads `stale_file`, with no figures, and only untouched files keep theirs. `path_gates` judges the path-scoped gates in `coverage.gates`. Each file row carries `risk` (`fix_pressure`, `dependents`, `hotspot`, `bug_magnet`, `risky`, `reasons`, `basis`) and up to eight `hints` per uncovered range: `range`, `symbol`, up to three `tests` to extend, and a `basis` of `per_test` (measured), `call_graph` or `import_graph` (inferred). Without a `revspec`, it covers everything a push would bring, diffed from the merge-base with the default base branch.
diff --git a/docs/start/CI.md b/docs/start/CI.md
index 2b4e61fce..cd5a40aad 100644
--- a/docs/start/CI.md
+++ b/docs/start/CI.md
@@ -279,6 +279,7 @@ a per-test coverage map (`repowise coverage add`) makes it more precise.
- An index built at an older commit than the change's base is used as is. Files changed in between whose imports or module-name strings moved (any change beyond whitespace, outside Python and JavaScript/TypeScript) add the tests reaching them. It runs everything instead when that commit is not in the clone, or more than 1,000 files changed in between, or one of them is a lockfile, manifest, build or test config or `.repowise/config.yaml`. CI config and test data changed in between no longer force a full run: neither is imported.
- Only `docs/` and root README-like files (README, CHANGELOG, LICENSE, CONTRIBUTING) are skipped, and only when no code names them (a test that reads `README.md` names it, and then runs); an empty line means only those changed.
- Tests the graph cannot see into run with every subset, and the reasons say how many and why: a test not indexed or importing nothing it resolves; a Python, JavaScript or TypeScript test that lists and reads files under a source directory (the working directory, a module's location, or its own location climbed out of the test directory; never its fixtures, sibling data or a temporary directory); one that runs the project's own command, `python -m` module, code importing the project, or package `bin` in a child process; a Java or Kotlin `@SpringBootTest`, which loads the whole application context (a slice such as `@WebMvcTest` is selected through its imports). A test that inherits `@SpringBootTest` from a base class or carries it through a custom meta-annotation is not detected and is selected through its imports only. The indexer finds them once from each test's source and stores the reason, so selection reads it without opening test files. An index built before this check runs everything until `repowise update` checks its tests. A Python module named like a test but outside every test directory is not a test when the nearest pytest config's `testpaths` or `python_files` leaves it out, as a bare `pytest` does: it is selected through like any other code. The rule follows a bare `pytest` run, so a test that CI only names explicitly on the command line outside `testpaths` is not seen as one: list it in `tests.always_run`. Inside a JavaScript or TypeScript `src` tree a `tests/` folder is not a test directory, since it often holds a product feature: a file there is a test when its name says so (`.test.`, `.spec.`) or a test directory such as `__tests__/` holds it.
+- A test the change reaches only through imports, more than one file away and with no call path, coverage or paired name, runs but is listed in `selected.import_closure` and counted apart in the summary: an import error breaks it, though nothing it calls runs the change.
- A changed `conftest.py`, test package `__init__.py` or imported test helper selects the tests under or importing it, not everything.
- A change that reaches a `conftest.py` only through its imports selects the tests that ask for a fixture using that change (directly, through other fixtures, `usefixtures` or `getfixturevalue`), plus one test under the conftest as an import check.
- Every test under the conftest still runs when an autouse fixture, a `pytest_*` hook or module-level code uses the change, or when the conftest imports it only for side effects. The tests it adds that nothing else reaches are listed in `selected.broadcast` and counted apart in the summary.
diff --git a/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py b/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py
index dfa95e93f..bd71e0392 100644
--- a/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py
+++ b/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py
@@ -312,10 +312,21 @@ def _args_header(selection, args: list[str], resolved: str, prioritize: bool) ->
)
if selection.run_all:
return "Run every test:"
- if spread := len(selection.broadcast):
+ apart = []
+ if selection.broadcast:
+ apart.append(
+ f"{len(selection.broadcast)} only because a conftest on a route keeps every "
+ "test under it"
+ )
+ if selection.import_closure:
+ apart.append(
+ f"{len(selection.import_closure)} only through imports, with no call path to "
+ "the change"
+ )
+ if apart:
return (
- f"{len(args)} argument(s) for {resolved}; {spread} selected test file(s) run "
- "only because a conftest on a route keeps every test under it."
+ f"{len(args)} argument(s) for {resolved}; selected test files that must still run "
+ f"but are counted apart: {'; '.join(apart)}."
)
return f"{len(args)} argument(s) for {resolved}."
diff --git a/packages/core/src/repowise/core/analysis/test_collection.py b/packages/core/src/repowise/core/analysis/test_collection.py
index a27f7b074..fd8a2e553 100644
--- a/packages/core/src/repowise/core/analysis/test_collection.py
+++ b/packages/core/src/repowise/core/analysis/test_collection.py
@@ -657,7 +657,8 @@ def _test_hops(
0 for a changed test, else the fewer of the call walk's hops (*calls*, per
target) and the file-dependency hops (*imports*, :meth:`ImportGraph.hops`
from *sources*), so 1 is a direct edge. A scope or gap route is not a
- source. Ordering only: no test is selected for it. A test neither
+ source. Ordering and the headline count only (``test_selection``'s
+ ``IMPORT_CLOSURE_HOPS``): no test is selected or dropped for it. A test neither
measures (a filename guess, one reached only through a route) is absent.
"""
best: dict[str, int] = {}
diff --git a/packages/core/src/repowise/core/analysis/test_selection.py b/packages/core/src/repowise/core/analysis/test_selection.py
index 96b1a2bfe..1a4cf2595 100644
--- a/packages/core/src/repowise/core/analysis/test_selection.py
+++ b/packages/core/src/repowise/core/analysis/test_selection.py
@@ -41,8 +41,9 @@ every test under its directory, unless :mod:`.conftest_routes` narrows a
conftest the change reaches only through its imports to the tests it can
break. The tests only a scope file on a route stands for are the selection's
``broadcast``: they run, and are counted apart from the tests the change
-reaches. A helper module tests import stands for the
-tests that import it, directly or through other helpers (basis
+reaches. So are the tests it reaches only through imports, more than
+:data:`IMPORT_CLOSURE_HOPS` files away (``import_closure``). A helper module
+tests import stands for the tests that import it, directly or through other helpers (basis
``helper-importers``), which are the files that run it: Python runs the
package file for each module in it, and pytest loads a conftest for each test
at or below it. Only documentation (``docs/`` and the root README, CHANGELOG,
@@ -75,7 +76,12 @@ import pathspec
from ..phpunit_roots import PhpunitRoots
from ..pytest_roots import PYTEST_CONFIG_NAMES, PytestRoots
from ..support_paths import DOC_EXTENSIONS
-from ..test_paths import is_test_path, is_test_related_path, is_test_support_path
+from ..test_paths import (
+ is_test_path,
+ is_test_related_path,
+ is_test_support_path,
+ paired_test_names,
+)
if TYPE_CHECKING:
from .selection_scopes import Scope
@@ -287,7 +293,8 @@ class Selection:
file that runs only because a ``conftest.py`` or test-package
``__init__.py`` on a route to a changed file stands for every test under
it to that file: they run, but a count of what the change reaches leaves
- them out.
+ them out. ``import_closure`` does the same for each test the change reaches
+ only through imports, mapped to its hops (:data:`IMPORT_CLOSURE_HOPS`).
"""
run_all: bool
@@ -301,10 +308,16 @@ class Selection:
why: Mapping[str, str] = field(default_factory=dict)
known_tests: tuple[str, ...] = ()
broadcast: Mapping[str, str] = field(default_factory=dict)
+ import_closure: Mapping[str, int] = field(default_factory=dict)
def reached_tests(self) -> list[str]:
- """:attr:`tests` without the :attr:`broadcast` ones, in order."""
- return [t for t in self.tests if t.split("::", 1)[0] not in self.broadcast]
+ """:attr:`tests` without those counted apart, in order."""
+ return [t for t in self.tests if not self.counted_apart(t)]
+
+ def counted_apart(self, test: str) -> bool:
+ """Whether *test* (a file or node id) is in :attr:`broadcast` or :attr:`import_closure`."""
+ path = test.split("::", 1)[0]
+ return path in self.broadcast or path in self.import_closure
def to_dict(self) -> dict[str, Any]:
return {
@@ -318,6 +331,7 @@ class Selection:
"basis": dict(self.basis),
"why": dict(self.why),
"broadcast": dict(self.broadcast),
+ "import_closure": dict(self.import_closure),
}
@@ -623,12 +637,20 @@ def select_tests(inp: SelectionInput) -> Selection:
]
)
notes = _notes(inp, triage.skipped, always, traced)
- broadcast = _broadcast(evidence, test_files, {f for _, f in in_scope} | set(always))
+ besides = {f for _, f in in_scope} | set(always)
+ broadcast = _broadcast(evidence, test_files, besides)
+ # A route through a file changed since the index was built may be shorter
+ # than the index shows, so those tests stay in the headline.
+ apart = besides | set(broadcast) | {f for _, f in gap_tests}
+ closure = _import_closure(evidence, inp.tiers.get("test_hops") or {}, test_files, apart, paths)
why = {**scope_why, **gap_why, **_why(triage.code, per_file, evidence), **always}
why.update(
(t, f"{path} changed ({scope} is on its route and stands for every test under it)")
for t, (path, scope) in broadcast.items()
)
+ for t, hops in closure.items():
+ if t in why:
+ why[t] += f"; {IMPORT_CLOSURE_VIA}: {hops} hops away, no call path to it"
return Selection(
run_all=bool(run_all),
reasons=tuple(run_all + evidence.notes + scope_notes + notes),
@@ -641,9 +663,46 @@ def select_tests(inp: SelectionInput) -> Selection:
why=why,
known_tests=tuple(inp.known_tests),
broadcast={t: scope for t, (_, scope) in broadcast.items()},
+ import_closure=closure,
)
+def _import_closure(
+ ev: _Evidence,
+ hops: Mapping[str, int],
+ selected: Collection[str],
+ also: Collection[str],
+ changed: Collection[str],
+) -> dict[str, int]:
+ """``{test file: hops}`` for each selected test the change reaches only by a long
+ import chain: no coverage, call path, changed test or name pairing, and more
+ than :data:`IMPORT_CLOSURE_HOPS` file-dependency hops from a changed file.
+
+ *hops* are the collection's fewest hops per test (call or import); a test
+ it did not measure stays in the headline. *also* are tests already counted
+ apart or selected by a rule.
+ """
+ vias: dict[str, set[str]] = {}
+ for rows in ev.inferred.values():
+ for test, via in rows:
+ vias.setdefault(test, set()).add(via)
+ covered = {f for refs in ev.covered.values() for _, f in refs}
+ # A test importing a deleted file breaks on that import, however far the
+ # graph puts it from the files still present.
+ covered |= {t for p in ev.deleted for t, _ in ev.inferred.get(p, ())}
+ paired = {name for p in changed for name in paired_test_names(p)}
+ return {
+ t: hops[t]
+ for t in selected
+ if hops.get(t, 0) > IMPORT_CLOSURE_HOPS
+ and vias.get(t, {"none"}) <= _IMPORT_ONLY
+ and t not in covered
+ and t not in also
+ and t not in changed
+ and PurePosixPath(t).name not in paired
+ }
+
+
def _broadcast(
ev: _Evidence, selected: Collection[str], also: Collection[str]
) -> dict[str, tuple[str, str]]:
@@ -701,6 +760,20 @@ def _why(
#: test under it (``conftest_routes``): selected, not reached by the change.
BROADCAST_VIA = "conftest-broadcast"
+# Past this many hops, a test the change reaches only through imports is counted
+# apart from the headline; it still runs (an import error breaks it, though
+# nothing it calls runs the change). 1 keeps direct importers in the headline.
+# Measured on this repository (headline without the conftest broadcast, at
+# N=1 / 2 / 3 / off): a hub module 308 / 500 / 1586 / 1600; four leaf changes
+# 53-63 / 59-74 / 123-191 / 130-1574. Most of the suite sits 3 hops from a
+# hub, so 3 barely narrows it. Ceiling: a call path deeper than the call walk reads as
+# imports only; a per-test coverage map is the upgrade.
+IMPORT_CLOSURE_HOPS = 1
+#: How :attr:`Selection.why` names a test counted apart by :data:`IMPORT_CLOSURE_HOPS`.
+IMPORT_CLOSURE_VIA = "import-closure"
+# The vias that put a test in by imports alone (a conftest broadcast included).
+_IMPORT_ONLY = frozenset({"import-graph", BROADCAST_VIA})
+
# The vias a conftest route adds (``conftest_routes``), spelled out.
_VIA_WHY = {
BROADCAST_VIA: (
diff --git a/packages/server/src/repowise/server/mcp_server/_test_selection.py b/packages/server/src/repowise/server/mcp_server/_test_selection.py
index 16ff414f9..bac868f41 100644
--- a/packages/server/src/repowise/server/mcp_server/_test_selection.py
+++ b/packages/server/src/repowise/server/mcp_server/_test_selection.py
@@ -303,15 +303,37 @@ def basis_of(selection: Selection) -> str:
return "measured" if "coverage" in selection.basis.values() else "inferred"
-def split_broadcast(selection: Selection) -> tuple[list[str], list[str]]:
- """``(the tests the change reaches, the broadcast ones)``, each in run order.
+def split_apart(selection: Selection) -> tuple[list[str], list[str], list[str]]:
+ """``(the tests the change reaches, the broadcast ones, the import-closure ones)``,
+ each in run order. All of them run; the last two are counted apart.
- A full run has no broadcast: every test runs for the run-all reasons.
+ A full run counts nothing apart: every test runs for the run-all reasons.
"""
- if selection.run_all or not selection.broadcast:
- return list(selection.tests), []
+ if selection.run_all:
+ return list(selection.tests), [], []
spread = [t for t in selection.tests if t.split("::", 1)[0] in selection.broadcast]
- return selection.reached_tests(), spread
+ closure = [t for t in selection.tests if t.split("::", 1)[0] in selection.import_closure]
+ return selection.reached_tests(), spread, closure
+
+
+def apart_sentence(broadcast: int, closure: int, keys: tuple[str, str]) -> str:
+ """The tests counted apart from the headline, which must still run; ``""`` for none."""
+ parts = []
+ if broadcast:
+ parts.append(
+ f"{broadcast} more test(s) must still run because a conftest on a route keeps "
+ f"every test under it ({keys[0]})."
+ )
+ if closure:
+ parts.append(
+ f"{closure} more test(s) must still run because they import the change through "
+ f"other files, with no call path to it ({keys[1]})."
+ )
+ return "".join(f" {p}" for p in parts)
+
+
+#: The counts of selected tests kept out of ``tests_to_run`` (they still run).
+APART_KEYS = ("broadcast_total", "import_closure_total")
def selection_block(
@@ -331,14 +353,16 @@ def selection_block(
index predates files outside the change and the selection holds over half
the suite: the list stands and ``summary`` says why. The tail of a list
holding most of the suite is not stored when every test is to run anyway.
- ``total`` and ``tests_to_run`` leave out the broadcast (:func:`split_broadcast`),
- counted in ``broadcast_total`` when there is one and stored apart.
+ ``total`` and ``tests_to_run`` leave out the tests counted apart
+ (:func:`split_apart`), counted in ``broadcast_total`` and
+ ``import_closure_total`` when there are any and stored for recovery.
*detail* adds ``why`` and ``basis_by_file``.
"""
- # In run order (``test_ranking``); the broadcast is counted apart.
- run_list, broadcast = split_broadcast(selection)
- if broadcast:
- collector.add(f"{label}.broadcast_tests ({len(broadcast)})", broadcast)
+ # In run order (``test_ranking``).
+ run_list, broadcast, closure = split_apart(selection)
+ for key, tests in zip(APART_KEYS, (broadcast, closure), strict=True):
+ if tests:
+ collector.add(f"{label}.{key.removesuffix('_total')}_tests ({len(tests)})", tests)
shown = run_list[:limit]
known = set(selection.known_tests)
held = len(known & set(selection.test_files))
@@ -377,11 +401,12 @@ def selection_block(
f"{len(known)} test files were selected; run the whole suite, or "
"`repowise update` to narrow it."
if degraded
- else _summary(selection, len(run_list), always, limit, len(broadcast))
+ else _summary(selection, len(run_list), always, limit, len(broadcast), len(closure))
),
}
- if broadcast:
- block["broadcast_total"] = len(broadcast)
+ for key, tests in zip(APART_KEYS, (broadcast, closure), strict=True):
+ if tests:
+ block[key] = len(tests)
if not detail:
return block
block["why"] = {
@@ -404,24 +429,24 @@ NO_TESTS = "No test needs to run for this change."
def _summary(
- selection: Selection, total: int, always: int, limit: int, broadcast: int = 0
+ selection: Selection,
+ total: int,
+ always: int,
+ limit: int,
+ broadcast: int = 0,
+ closure: int = 0,
) -> str:
shown = f"; showing first {limit}" if total > limit else ""
if selection.run_all:
lead = selection.reasons[0] if selection.reasons else "the selection could not vouch"
named = f" Run these {total} first{shown}." if total else ""
return f"Run every test: {lead}{named}"
- spread = (
- f" {broadcast} more test(s) must still run because a conftest on a route keeps "
- "every test under it (broadcast_total; the reasons say why)."
- if broadcast
- else ""
- )
+ spread = apart_sentence(broadcast, closure, APART_KEYS)
if not total:
return f"No test the change reaches directly.{spread}" if spread else NO_TESTS
# The share of the suite, so a selection holding most of it never reads as targeted.
known = set(selection.known_tests)
- held = len(known & (set(selection.test_files) - set(selection.broadcast)))
+ held = len(known & {f for f in selection.test_files if not selection.counted_apart(f)})
share = f" ({held} of {len(known)} test files)" if known else ""
rule = f", {always} of them because they run with every subset" if always else ""
if known and held == len(known):
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
index dc3851871..eaaebbd44 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/directives.py
@@ -32,10 +32,11 @@ from repowise.server.mcp_server._helpers import (
from repowise.server.mcp_server._test_selection import (
UNAVAILABLE_REASON,
basis_of,
- split_broadcast,
+ split_apart,
)
from repowise.server.mcp_server.tool_risk.directives_pr import (
_WORKSPACE_FAMILIES, # noqa: F401 re-exported
+ APART_FIELDS,
_as_path,
_drop_family,
_project_recommendation, # noqa: F401 re-exported
@@ -563,10 +564,12 @@ def _tests_to_run(
"tests_status": tests.status,
"tests_status_reason": tests.message,
}
- # In run order (``test_ranking``); the broadcast is counted apart.
- population, broadcast = split_broadcast(selection)
- if broadcast:
- collector.add(f"directive.broadcast_tests ({len(broadcast)})", broadcast)
+ # In run order (``test_ranking``); the tests counted apart still run.
+ population, *apart = split_apart(selection)
+ counts = dict(zip(APART_FIELDS, apart, strict=True))
+ for key, tests in counts.items():
+ if tests:
+ collector.add(f"directive.{key.removesuffix('_total')} ({len(tests)})", tests)
reasons = list(selection.reasons)
if len(reasons) > _RUN_ALL_REASONS_LIMIT:
collector.add(
@@ -576,8 +579,7 @@ def _tests_to_run(
)
shown = dict.fromkeys(t.split("::", 1)[0] for t in population[:_TESTS_TO_RUN_LIMIT])
extra: dict[str, Any] = {"tests_run_all": selection.run_all}
- if broadcast:
- extra["tests_broadcast_total"] = len(broadcast)
+ extra.update((key, len(tests)) for key, tests in counts.items() if tests)
if selection.run_all:
extra["tests_run_all_reasons"] = reasons[:_RUN_ALL_REASONS_LIMIT]
extra["tests_to_run_why"] = {p: selection.why[p] for p in shown if p in selection.why}
diff --git a/packages/server/src/repowise/server/mcp_server/tool_risk/directives_pr.py b/packages/server/src/repowise/server/mcp_server/tool_risk/directives_pr.py
index 2fcb0947e..54a04e566 100644
--- a/packages/server/src/repowise/server/mcp_server/tool_risk/directives_pr.py
+++ b/packages/server/src/repowise/server/mcp_server/tool_risk/directives_pr.py
@@ -14,7 +14,10 @@ from repowise.core.analysis.next_call import ActionCommand
from repowise.core.analysis.risk_semantics import structural_impact_band
from repowise.server.mcp_server._budget import OmissionCollector, cap_collection
from repowise.server.mcp_server._helpers import filter_path_list, is_excluded
-from repowise.server.mcp_server._test_selection import run_kind
+from repowise.server.mcp_server._test_selection import APART_KEYS, apart_sentence, run_kind
+
+#: :data:`APART_KEYS` as the directive names them.
+APART_FIELDS = tuple(f"tests_{key}" for key in APART_KEYS)
_WORKSPACE_FAMILIES = (
"will_break_consumers",
@@ -128,11 +131,7 @@ def tests_summary(
) -> str:
"""Summary sentences for the run list and, on request, the typed recommendations."""
suffix = f" {run_total} test(s) to run, {basis}." if run_total else ""
- if spread := tests_fields.get("tests_broadcast_total"):
- suffix += (
- f" {spread} more must still run because a conftest on a route keeps every test "
- "under it (tests_broadcast_total)."
- )
+ suffix += apart_sentence(*(tests_fields.get(k, 0) for k in APART_FIELDS), APART_FIELDS)
if tests_fields.get("tests_run_all"):
suffix += " Every test must run (tests_run_all_reasons)."
recommendations_total = len(test_impact.get("recommendations") or [])
diff --git a/tests/unit/analysis/test_test_selection.py b/tests/unit/analysis/test_test_selection.py
index def2efea6..d24f7ed5b 100644
--- a/tests/unit/analysis/test_test_selection.py
+++ b/tests/unit/analysis/test_test_selection.py
@@ -777,6 +777,56 @@ def test_a_full_run_or_a_plain_subset_has_no_broadcast() -> None:
assert not _select(["uv.lock"], known_tests=_KNOWN).broadcast
+def test_a_test_reached_only_by_a_long_import_chain_is_counted_apart_but_runs() -> None:
+ tiers = _tiers(
+ inferred=[
+ _inferred("src/a.py", "tests/test_far.py", "import-graph"),
+ _inferred("src/a.py", "tests/test_near.py", "import-graph"),
+ _inferred("src/a.py", "tests/test_called.py", "call-graph"),
+ _inferred("src/a.py", "tests/test_a.py", "import-graph"),
+ _inferred("src/a.py", "tests/test_unmeasured.py", "import-graph"),
+ _inferred("src/b.py", "tests/test_both.py", "import-graph"),
+ _inferred("src/a.py", "tests/test_both.py", "call-graph"),
+ ]
+ )
+ tiers["test_hops"] = {
+ "tests/test_far.py": 3,
+ "tests/test_near.py": 1,
+ "tests/test_called.py": 4,
+ "tests/test_a.py": 3,
+ "tests/test_both.py": 5,
+ }
+ sel = _select(["src/a.py", "src/b.py"], tiers)
+ assert not sel.run_all, sel.reasons
+ # Every test still runs; only the long import-only one leaves the headline.
+ assert len(sel.test_files) == 6
+ assert sel.import_closure == {"tests/test_far.py": 3}
+ assert "tests/test_far.py" not in sel.reached_tests()
+ assert "import-closure: 3 hops away" in sel.why["tests/test_far.py"]
+ # A direct importer, a call path (from any changed file), the paired name
+ # and a test with no measured distance all stay in the headline.
+ assert len(sel.reached_tests()) == 5
+
+
+def test_a_test_importing_a_deleted_file_is_never_counted_apart() -> None:
+ tiers = _tiers(
+ inferred=[
+ _inferred("src/gone.py", "tests/test_x.py", "import-graph"),
+ _inferred("src/a.py", "tests/test_x.py", "import-graph"),
+ ]
+ )
+ tiers["test_hops"] = {"tests/test_x.py": 4}
+ sel = _select(["src/a.py"], tiers, deleted=["src/gone.py"])
+ assert "tests/test_x.py" in sel.test_files and not sel.import_closure
+
+
+def test_a_covered_test_is_never_counted_apart() -> None:
+ covered = {"tests/test_x.py::t": {"test_file": "tests/test_x.py", "source_files": ["src/a.py"]}}
+ tiers = _tiers(covered=covered, inferred=[_inferred("src/a.py", "tests/test_x.py", "import-graph")])
+ tiers["test_hops"] = {"tests/test_x.py": 4}
+ assert not _select(["src/a.py"], tiers).import_closure
+
+
def test_a_route_through_an_imported_helper_selects_its_importers() -> None:
tiers = _tiers(
inferred=[
diff --git a/tests/unit/cli/test_impacted_tests_cmd.py b/tests/unit/cli/test_impacted_tests_cmd.py
index 5af835d9f..3cfb9b3ed 100644
--- a/tests/unit/cli/test_impacted_tests_cmd.py
+++ b/tests/unit/cli/test_impacted_tests_cmd.py
@@ -111,22 +111,25 @@ def test_args_prints_the_subset_on_one_line(repo) -> None:
assert "documentation: no tests needed" in _err(result)
-def test_the_args_header_counts_the_conftest_broadcast_apart() -> None:
+def test_the_args_header_counts_the_broadcast_and_import_closure_apart() -> None:
from repowise.cli.commands.impacted_tests_cmd import _args_header
from repowise.core.analysis.test_selection import Selection
- tests = ("tests/test_a.py", "tests/test_b.py")
+ tests = ("tests/test_a.py", "tests/test_b.py", "tests/test_c.py")
selection = Selection(
run_all=False,
reasons=(),
tests=tests,
test_files=tests,
broadcast={"tests/test_b.py": "tests/conftest.py"},
+ import_closure={"tests/test_c.py": 3},
)
assert _args_header(selection, list(tests), "pytest", False) == (
- "2 argument(s) for pytest; 1 selected test file(s) run only because a conftest "
- "on a route keeps every test under it."
+ "3 argument(s) for pytest; selected test files that must still run but are counted "
+ "apart: 1 only because a conftest on a route keeps every test under it; 1 only "
+ "through imports, with no call path to the change."
)
+ assert selection.to_dict()["import_closure"] == {"tests/test_c.py": 3}
def test_json_adds_the_selection_to_the_report(repo) -> None:
diff --git a/tests/unit/server/mcp/test_change_test_selection.py b/tests/unit/server/mcp/test_change_test_selection.py
index a5acbe27d..ee1d1ec28 100644
--- a/tests/unit/server/mcp/test_change_test_selection.py
+++ b/tests/unit/server/mcp/test_change_test_selection.py
@@ -559,29 +559,33 @@ def _spread(*, run_all: bool = False) -> object:
known_tests=suite,
why=dict.fromkeys(suite[:5], "src/a.py changed (import-graph)"),
broadcast=dict.fromkeys(suite[2:5], "tests/conftest.py"),
+ import_closure={suite[1]: 3},
)
-def test_the_conftest_broadcast_is_counted_apart_from_the_headline(tmp_path) -> None:
+def test_the_broadcast_and_import_closure_are_counted_apart_from_the_headline(tmp_path) -> None:
from repowise.server.mcp_server._budget import OmissionCollector
from repowise.server.mcp_server._test_selection import selection_block
collector = OmissionCollector("get_change_risk", store_path=tmp_path / "omissions.db")
block = selection_block({}, _spread(), collector, limit=10, label="impacted_tests")
- assert block["total"] == 2 and block["broadcast_total"] == 3
- assert block["tests_to_run"] == ["tests/test_0.py", "tests/test_1.py"]
- assert block["summary"].startswith("2 test(s) to run for this change (2 of 20 test files).")
+ assert block["total"] == 1
+ assert block["broadcast_total"] == 3 and block["import_closure_total"] == 1
+ assert block["tests_to_run"] == ["tests/test_0.py"]
+ assert block["summary"].startswith("1 test(s) to run for this change (1 of 20 test files).")
assert "3 more test(s) must still run because a conftest" in block["summary"]
- # The broadcast tests are stored for recovery, not dropped.
+ assert "1 more test(s) must still run because they import the change" in block["summary"]
+ # The tests counted apart are stored for recovery, not dropped.
assert not collector.empty
# A full run has no broadcast: every listed test is what to run first.
full = selection_block({}, _spread(run_all=True), collector, limit=10, label="impacted_tests")
assert full["total"] == 5 and "broadcast_total" not in full
+ assert "import_closure_total" not in full
-def test_get_risk_counts_the_conftest_broadcast_apart(tmp_path) -> None:
+def test_get_risk_counts_the_broadcast_and_import_closure_apart(tmp_path) -> None:
from repowise.server.mcp_server._budget import OmissionCollector
from repowise.server.mcp_server.tool_risk.directives import ChangeTests, _tests_to_run
from repowise.server.mcp_server.tool_risk.directives_pr import tests_summary
@@ -589,8 +593,9 @@ def test_get_risk_counts_the_conftest_broadcast_apart(tmp_path) -> None:
collector = OmissionCollector("get_risk", store_path=tmp_path / "omissions.db")
population, basis, fields = _tests_to_run(ChangeTests(selection=_spread()), collector)
- assert population == ["tests/test_0.py", "tests/test_1.py"]
+ assert population == ["tests/test_0.py"]
assert fields["tests_broadcast_total"] == 3
- assert "3 more must still run because a conftest" in tests_summary(
- len(population), basis, fields, {}, False, 10
- )
+ assert fields["tests_import_closure_total"] == 1
+ summary = tests_summary(len(population), basis, fields, {}, False, 10)
+ assert "3 more test(s) must still run because a conftest" in summary
+ assert "1 more test(s) must still run because they import the change" in summary
diff --git a/tests/unit/server/mcp/test_risk.py b/tests/unit/server/mcp/test_risk.py
index ac7d26551..1f93ed4d3 100644
--- a/tests/unit/server/mcp/test_risk.py
+++ b/tests/unit/server/mcp/test_risk.py
@@ -388,17 +388,19 @@ async def test_get_risk_pr_directive_falls_back_to_the_graph_without_a_map(pr_ch
async def test_get_risk_pr_directive_runs_tests_in_import_reach(pr_checkout):
"""No map and no call-graph test -> the tests in reverse-import reach, inferred.
- ``tests/test_service.py`` reaches ``models.py`` through ``service.py``. With
- no coverage the gap list is withheld, not sent empty.
+ ``tests/test_service.py`` reaches ``models.py`` only through ``service.py``,
+ so it runs but is counted apart from the headline. With no coverage the gap
+ list is withheld, not sent empty.
"""
from repowise.server.mcp_server import get_risk
result = await get_risk(["src/db/models.py"], changed_files=["src/db/models.py"])
directive = result["directive"]
- assert directive["tests_to_run"] == ["tests/test_service.py"]
+ assert directive["tests_to_run"] == []
+ assert directive["tests_import_closure_total"] == 1
+ assert "1 more test(s) must still run because they import the change" in directive["summary"]
assert directive["tests_to_run_basis"] == "inferred"
- assert directive["tests_to_run_kind"] == "test_file"
assert "missing_tests" not in directive
assert directive["coverage"]["status"] == "unavailable"
assert "test gaps are withheld" in directive["summary"]