Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(impacted-tests): scope manifest, package-init and asset triggers instead of running every test (#3243)

Raghav Chamadiya24aa1c65d3a7
diff --git a/docs/start/CI.md b/docs/start/CI.md
index 1c0a9e349..9eb6c7cbe 100644
--- a/docs/start/CI.md
+++ b/docs/start/CI.md
@@ -252,16 +252,19 @@ a per-test coverage map (`repowise coverage add`) makes it more precise.
- It exits `0` for a subset or for everything, and `2` only when it cannot read the change or the `tests.*` config.
- Branch on the run-all flag, never on `:all`: jest and vitest read it as a pattern and run nothing.
-- It runs everything for: an empty change; a lockfile, manifest, build, test or CI config change; `.repowise/config.yaml`; a production package's `__init__.py`.
+- It runs everything for: an empty change; a Cargo, Make, CMake, Docker or CI file; tool and runtime config (dotfiles, `.tool-versions`, root-level `*.toml`, `*.yaml` and `*.json`, shell scripts); `.repowise/config.yaml`.
+- A lockfile, manifest or test runner config runs its own ecosystem's tests: for a nested `package.json`, lockfile or `go.mod`, the tests under its directory and those reaching code there; at the root, and for Python, Ruby, PHP and Gradle or Maven files (pip requirements included), `tsconfig` and jest or vitest configs, the whole ecosystem. Tests whose code names the file from outside that scope run too (a Python test reading `package.json`); if more than 50 non-test files outside the scope name it, everything runs.
+- A production package's `__init__.py` selects the tests reaching any module under it.
+- A doc, template, query, stylesheet, image or nested data file selects the tests reaching the code that names it. When no code names it, a doc, template, stylesheet or image selects its package's tests, and a query or data file runs everything. Ceiling: a test that globs docs or templates in another package is not seen.
- It also runs everything for: non-code files in a test tree; a changed file with no known test or only a filename guess; a route through a test helper no test imports, unless every known test is selected anyway (the reasons then say so); any Python test helper while a conftest or pytest config loads plugins by name.
- It also runs everything for: a missing or unreadable index; a per-test map at its row cap; a deleted file no known test used.
- An index built at an older commit than the change's base is used as is. Files changed in between whose imports or module-name strings moved (any change beyond whitespace, outside Python and JavaScript/TypeScript) add the tests reaching them. It runs everything instead when that commit is not in the clone, or more than 1,000 files changed in between, or one of them is a lockfile, manifest, build or test config or `.repowise/config.yaml`. CI config and test data changed in between no longer force a full run: neither is imported.
-- Only `docs/` and root README-like files (README, CHANGELOG, LICENSE, CONTRIBUTING) are skipped, and only when no code names them (a test that reads `README.md` names it); an empty line means only those changed.
+- Only `docs/` and root README-like files (README, CHANGELOG, LICENSE, CONTRIBUTING) are skipped, and only when no code names them (a test that reads `README.md` names it, and then runs); an empty line means only those changed.
- Tests the graph cannot see into run with every subset, and the reasons say how many and why: a test not indexed or importing nothing it resolves; a Python, JavaScript or TypeScript test that lists and reads files under a source directory (the working directory, a module's location, or its own location climbed out of the test directory; never its fixtures, sibling data or a temporary directory); one that runs the project's own command, `python -m` module, code importing the project, or package `bin` in a child process. The indexer finds them once from each test's source and stores the reason, so selection reads it without opening test files. An index built before this check runs everything until `repowise update` checks its tests. A Python module named like a test but outside every test directory is not a test when the nearest pytest config's `testpaths` or `python_files` leaves it out, as a bare `pytest` does: it is selected through like any other code. The rule follows a bare `pytest` run, so a test that CI only names explicitly on the command line outside `testpaths` is not seen as one: list it in `tests.always_run`.
- A changed `conftest.py`, test package `__init__.py` or imported test helper selects the tests under or importing it, not everything.
- Wiring no import states counts as a use: a JS/TS setup file named under `setupFiles`, `setupFilesAfterEnv`, `globalSetup` or `globalTeardown` in a `vitest.*`, `jest.*`, `vite.*` or `playwright.*` config or a `package.json` stands for the tests in that config's package, and a helper a test names by a path relative to itself (`Path(__file__).parent / "server.py"`, `path.join(__dirname, "server.ts")`) stands for that test. A `conftest.py` links only the files pytest collects (its `python_files`), so a helper beside the tests is not on every route the conftest reaches.
- Without a range, in CI it reads the pull request's change; locally, the staged changes.
-- `tests.full_run_on` (gitignore patterns) adds run-everything paths; `tests.always_run` is appended to every selection as written, for the runner whose language each entry is in.
+- `tests.full_run_on` (gitignore patterns) adds run-everything paths, and a path it matches is never scoped; `tests.always_run` is appended to every selection as written, for the runner whose language each entry is in.
- With `--runner pytest`, `go` or `jest`, the arguments hold only that runner's tests, `tests.always_run` included. A repository with tests in several languages needs one job per runner: the tests a job leaves out are counted on stderr, and listed per runner under `left_out` with `--format json`. `--runner files` gets every test.
- `--explain <test>` says why one test file was or was not selected: the first changed file that reached it, the evidence (coverage, call graph, import graph) and the import route from the test to that file when there is one, or the rule that runs it. With `--format json` the same answer is added as `explain`, and `selected.why` holds it for every selected test.
- Ceiling: on the JVM and .NET, a same-package test linked only by an unresolved call is missed. So is a test that reaches the change through a plugin or registry loaded by a string name, a tree walk or child process done inside a shared helper it calls, or one in another language: list such tests in `tests.always_run`, and run the full suite on the default branch.
diff --git a/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py b/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py
index 3e10a4618..3477d047e 100644
--- a/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py
+++ b/packages/cli/src/repowise/cli/commands/impacted_tests_cmd.py
@@ -143,10 +143,12 @@ def impacted_tests_command(
explain = explain.replace("\\", "/").removeprefix("./") if explain else None
checkout = _read_checkout(repo_path)
- result = run_async(_collect(repo_path, change, checkout.roots, config, explain))
+ plan = _plan_scopes(repo_path, change, config, checkout) if config is not None else None
+ routes = plan.routes if plan else []
+ result = run_async(_collect(repo_path, change, checkout.roots, config, explain, routes))
result["diff"] = change.label
- if config is not None:
- result["selection"] = _select(repo_path, change, result, config, checkout)
+ if plan is not None:
+ result["selection"] = _select(repo_path, change, result, config, checkout, plan)
if explain and fmt != "json":
_render_explain(result, explain)
return
@@ -195,18 +197,57 @@ def _read_checkout(repo_path) -> _Checkout:
return _Checkout(tracked, texts, roots)
+class _Plan(NamedTuple):
+ """Who names each changed doc or asset, the scope each scoped file runs, and its routes."""
+
+ namers: dict[str, list[str]]
+ scopes: dict
+ routes: list[str]
+
+
+def _plan_scopes(repo_path, change, config, checkout: _Checkout) -> _Plan:
+ """The change's scopes, reading the sources once and only when a file needs its namers."""
+ from repowise.core.analysis.selection_scopes import (
+ keeps_full_run,
+ needs_namers,
+ trigger_scopes,
+ )
+ from repowise.core.analysis.test_selection import file_namers, is_scan_source
+
+ paths = [*change.files, *change.deleted]
+ # A file that runs everything anyway makes every namer search moot.
+ blocked = any(keeps_full_run(p, config) for p in paths)
+ asked = [] if blocked else [p for p in paths if needs_namers(p, config)]
+ namers: dict[str, list[str]] = {}
+ if asked:
+ known = dict(checkout.pytest_texts) # conftests and pytest configs, read already
+ root = Path(repo_path)
+ texts = (
+ (p, known[p]) if p in known else (p, _text(root / p) or "")
+ for p in checkout.tracked
+ if is_scan_source(p)
+ )
+ namers = file_namers(asked, texts)
+ scopes = trigger_scopes(paths, checkout.tracked, namers, config)
+ routes = sorted({r for scope in scopes.values() for r in scope.routes} - set(paths))
+ return _Plan(namers, scopes, routes)
+
+
async def _collect(
repo_path,
change,
roots: PytestRoots | None = None,
config=None,
explain: str | None = None,
+ scope_routes: list[str] | tuple[str, ...] = (),
) -> dict:
"""Resolve the change's files to impacted tests + labelled fallbacks.
With a selection *config*, files changed since the index was built are
- walked too, so selection can add the tests reaching them. With *explain*,
- the import route from that test to a changed file is looked up as well.
+ walked too, so selection can add the tests reaching them; so are
+ *scope_routes*, the files a scope runs the tests of (:func:`_plan_scopes`).
+ With *explain*, the import route from that test to a changed file is
+ looked up as well.
"""
from repowise.core.persistence.crud import (
get_health_metrics,
@@ -237,6 +278,7 @@ async def _collect(
# aggregate coverage ingest resolves against).
repo_keys = {m.file_path for m in await get_health_metrics(session, repo_id)}
routes = [] if config is None else _gap_routes(repo_path, change, config, repo_keys, out)
+ routes = sorted({*routes, *scope_routes})
await _resolve_impacted(
session, repo_id, _query_lines(change, measured), repo_keys, out, roots, routes
)
@@ -573,14 +615,12 @@ def _with_importers(picks: dict[str, str], parents: dict[str, set[str]]) -> list
return list(picks.items())
-def _select(repo_path, change, result: dict, config, checkout: _Checkout):
+def _select(repo_path, change, result: dict, config, checkout: _Checkout, plan: _Plan):
"""The run-all-or-subset decision for ``--format args`` / ``json``."""
from repowise.core.analysis.test_selection import (
SelectionInput,
- doc_readers,
is_documentation,
is_runnable_test,
- is_scan_source,
plugin_loader,
select_tests,
)
@@ -591,8 +631,6 @@ def _select(repo_path, change, result: dict, config, checkout: _Checkout):
tracked = checkout.tracked
go_test_dirs = {str(Path(p).parent.as_posix()) for p in tracked if p.endswith("_test.go")}
placed = result["placed_tests"]
- docs = [p for p in (*change.files, *change.deleted) if is_documentation(p)]
- sources = [p for p in tracked if is_scan_source(p)]
known_tests = sorted(p for p in tracked if is_runnable_test(p, checkout.roots))
return select_tests(
SelectionInput(
@@ -611,10 +649,11 @@ def _select(repo_path, change, result: dict, config, checkout: _Checkout):
missing={f for f in named if not (root / f).is_file()},
go_test_dirs=go_test_dirs,
known_tests=known_tests,
- doc_readers=doc_readers(docs, _texts(root, sources)) if docs else {},
+ doc_readers={p: n[0] for p, n in plan.namers.items() if is_documentation(p)},
plugin_loader=plugin_loader(checkout.pytest_texts),
unplaced_tests=[] if placed is None else [t for t in known_tests if t not in placed],
always_run_tests=result["always_run_tests"],
+ scopes=plan.scopes,
)
)
diff --git a/packages/core/src/repowise/core/analysis/selection_scopes.py b/packages/core/src/repowise/core/analysis/selection_scopes.py
new file mode 100644
index 000000000..1b720cdd9
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/selection_scopes.py
@@ -0,0 +1,345 @@
+"""Scopes: what a changed file runs when every test is more than it can reach.
+
+:func:`~.test_selection.select_tests` runs everything for a file whose tests
+are unknown. For three kinds the reach is known, and :func:`trigger_scopes`
+names it as a :class:`Scope`:
+
+- A manifest, lockfile or runner config runs its ecosystem's tests. A JS
+ package (``package.json``, its lockfile) or a Go module runs those under its
+ directory plus every test reaching code there; the rest run the whole
+ ecosystem, because one environment serves every package (a venv, a bundle)
+ or the file applies far from its directory (tsconfig ``extends``, a runner
+ config whose globs reach other trees). Cargo stays a full run: Rust unit
+ tests live inside source files, which no selection lists.
+- A production ``__init__.py`` runs the tests reaching any module under its
+ directory: importing one runs it, whatever the diff.
+- An asset runs the tests reaching the code that names it. A doc, template,
+ stylesheet or image nothing names runs the tests of the package holding it;
+ a query or data file nothing names keeps its full run (a test may glob it).
+ Other non-code files, dotfiles and root-level data configure tools, not
+ code, and keep their full run too.
+
+A scope also runs the tests reaching each file that names the changed one, so
+a test reading ``package.json`` as data is kept. An ecosystem scope already
+holds its own ecosystem's code under its root, so only the namers outside it
+are traced; more than :data:`MAX_NAMERS` of those that are not tests
+themselves (or of an asset's) run everything, since tracing that many costs
+what a full run saves. Package
+roots are read from the tracked paths the caller already holds, which list
+every manifest (the indexed file list misses some; see
+:mod:`~repowise.core.ingestion.package_roots`). Ceilings: code that globs a
+directory instead of naming a file, and code reading a manifest through a
+name it builds, are not seen.
+"""
+
+from __future__ import annotations
+
+import functools
+from collections.abc import Collection, Iterable, Mapping
+from dataclasses import dataclass
+from pathlib import PurePosixPath
+
+import pathspec
+
+from ..test_paths import is_test_related_path
+from .test_selection import (
+ JS_SUFFIXES,
+ PACKAGE_INIT_REASON,
+ PYTHON_SUFFIXES,
+ SHARED_TEST_DATA_REASON,
+ TestSelectionConfig,
+ full_run_on_spec,
+ full_run_reason,
+ is_code_file,
+ is_documentation,
+ is_runnable_test,
+)
+
+# Namers past this many make a full run cheaper to decide than to trace.
+MAX_NAMERS = 50
+
+_JVM = (".java", ".kt", ".kts", ".scala", ".groovy")
+# (ecosystem, test suffixes, scoped to the file's directory, gitwildmatch
+# patterns). Each pattern is also a full-run trigger (``_FULL_RUN_GROUPS``),
+# so a file no scope covers still runs everything; a test pins the two lists.
+ECOSYSTEMS: tuple[tuple[str, tuple[str, ...], bool, tuple[str, ...]], ...] = (
+ (
+ "JavaScript",
+ JS_SUFFIXES,
+ True,
+ ("package.json", "package-lock.json", "yarn.lock", "pnpm-lock.yaml"),
+ ),
+ (
+ "JavaScript",
+ JS_SUFFIXES,
+ False,
+ (
+ *("tsconfig*.json", "jest.config.*", "vitest.config.*", "vite.config.*"),
+ *("pnpm-workspace.yaml", ".nvmrc", ".node-version", ".npmrc", ".yarnrc"),
+ ".yarnrc.yml",
+ ),
+ ),
+ (
+ "Python",
+ PYTHON_SUFFIXES,
+ False,
+ (
+ *("pyproject.toml", "setup.py", "setup.cfg", "poetry.lock", "uv.lock", "Pipfile"),
+ *("Pipfile.lock", "requirements*.txt", "constraints*.txt", "requirements*.in"),
+ *("constraints*.in", "**/requirements/**", "pytest.ini", "tox.ini", "noxfile.py"),
+ ".python-version",
+ ),
+ ),
+ ("Go", (".go",), True, ("go.mod", "go.sum")),
+ ("Go", (".go",), False, ("go.work", "go.work.sum")),
+ ("Ruby", (".rb",), False, ("Gemfile", "Gemfile.lock")),
+ ("PHP", (".php",), False, ("composer.json", "composer.lock")),
+ ("JVM", _JVM, False, ("build.gradle*", "settings.gradle*", "pom.xml")),
+)
+_ECOSYSTEM_SPECS = tuple(
+ (lang, suffixes, scoped, pathspec.PathSpec.from_lines("gitwildmatch", globs))
+ for lang, suffixes, scoped, globs in ECOSYSTEMS
+)
+
+# Assets the package holding them stands for when no code names them. ``.txt``
+# counts only as documentation (``docs/``, root README-like files): elsewhere
+# it may be a pinned dependency list.
+_OWNED = frozenset(
+ {".md", ".mdx", ".rst", ".adoc", ".markdown", ".html", ".htm"}
+ | {".j2", ".jinja", ".jinja2", ".tmpl", ".tpl", ".hbs", ".mustache", ".ejs", ".njk"}
+ | {".css", ".scss", ".sass", ".less"}
+ | {".svg", ".png", ".jpg", ".jpeg", ".gif", ".webp", ".ico"}
+)
+# Assets only a naming file stands for: a test may glob a query or data file.
+# Data only below the root: a root-level ``*.yaml`` or ``*.json`` configures a tool.
+_NAMED_ONLY = frozenset({".scm", ".graphql", ".gql", ".sql"})
+_DATA = frozenset({".json", ".yaml", ".yml", ".csv", ".sha256"})
+
+
+@dataclass(frozen=True)
+class Scope:
+ """The tests a changed file runs in place of every test.
+
+ Every known test under *root* whose suffix is in *suffixes* (any test when
+ empty; none when *root* is ``None``), plus the tests reaching each file in
+ *reach* and in *namers*. *reach* is production code under *root*, which
+ may have no test of its own; a namer is code naming the file, so a namer no
+ test reaches leaves the file's tests unknown. *run_all* says why the file
+ runs everything after all.
+ """
+
+ why: str
+ basis: str
+ root: str | None = None
+ suffixes: tuple[str, ...] = ()
+ reach: tuple[str, ...] = ()
+ namers: tuple[str, ...] = ()
+ run_all: str | None = None
+
+ @property
+ def routes(self) -> tuple[str, ...]:
+ """The files whose reaching tests the selection needs from the graph."""
+ return () if self.run_all else (*self.reach, *self.namers)
+
+
+def needs_namers(path: str, config: TestSelectionConfig) -> bool:
+ """Whether :func:`trigger_scopes` needs to know who names *path*."""
+ if is_test_related_path(path) or _configured(path, config):
+ return False
+ return _ecosystem(path) is not None or (full_run_reason(path) is None and _is_asset(path))
+
+
+def keeps_full_run(path: str, config: TestSelectionConfig) -> bool:
+ """Whether *path* runs everything whatever names it, so no namer search can help."""
+ if full_run_reason(path, config.full_run_on) is None:
+ return False
+ if is_test_related_path(path) or _configured(path, config):
+ return True
+ return full_run_reason(path) != PACKAGE_INIT_REASON and _ecosystem(path) is None
+
+
+def trigger_scopes(
+ paths: Iterable[str],
+ tracked: Collection[str],
+ namers: Mapping[str, Collection[str]],
+ config: TestSelectionConfig,
+) -> dict[str, Scope]:
+ """The :class:`Scope` each of *paths* runs instead of every test, where it has one.
+
+ *tracked* are the checkout's files; *namers* maps a file to every file
+ naming it (:func:`~.test_selection.file_namers`). A path with no scope
+ keeps its rule.
+ """
+ tree = _Tree(tuple(tracked))
+ out: dict[str, Scope] = {}
+ for path in paths:
+ if scope := _trigger_scope(path, tree, tuple(namers.get(path, ())), config):
+ out[path] = scope
+ return out
+
+
+class _Tree:
+ """The checkout's production code and package roots, each worked out once and only if asked."""
+
+ def __init__(self, tracked: tuple[str, ...]) -> None:
+ self.tracked = tracked
+ self._under: dict[tuple[str, tuple[str, ...]], tuple[str, ...]] = {}
+
+ @functools.cached_property
+ def code(self) -> tuple[str, ...]:
+ return tuple(
+ sorted(p for p in self.tracked if is_code_file(p) and not is_test_related_path(p))
+ )
+
+ @functools.cached_property
+ def roots(self) -> set[str]:
+ from ..ingestion.package_roots import package_roots_from_paths
+
+ return package_roots_from_paths(set(self.tracked))
+
+ def code_under(self, root: str, suffixes: tuple[str, ...]) -> tuple[str, ...]:
+ key = (root, suffixes)
+ if key not in self._under:
+ self._under[key] = tuple(
+ p
+ for p in self.code
+ if _is_under(p, root) and (not suffixes or p.lower().endswith(suffixes))
+ )
+ return self._under[key]
+
+ def owner(self, path: str) -> str | None:
+ """The deepest package root holding *path*, never the repository root."""
+ from ..ingestion.package_roots import module_for
+
+ owner = module_for(path, self.roots)
+ return owner if owner in self.roots else None
+
+
+def _trigger_scope(
+ path: str, tree: _Tree, namers: tuple[str, ...], config: TestSelectionConfig
+) -> Scope | None:
+ if is_test_related_path(path) or full_run_reason(path) == SHARED_TEST_DATA_REASON:
+ return None # read by tests anywhere: the test-tree rule decides
+ if _configured(path, config):
+ return None # the config asked for a full run
+ if eco := _ecosystem(path):
+ return _ecosystem_scope(path, eco, tree, namers)
+ why = full_run_reason(path)
+ if why == PACKAGE_INIT_REASON:
+ return _package_scope(path, tree)
+ if why is not None or not _is_asset(path):
+ return None
+ return _named_scope(path, tree, namers)
+
+
+def _configured(path: str, config: TestSelectionConfig) -> bool:
+ return bool(config.full_run_on) and full_run_on_spec(config.full_run_on).match_file(path)
+
+
+def _ecosystem(path: str) -> tuple[str, tuple[str, ...], bool] | None:
+ for lang, suffixes, scoped, spec in _ECOSYSTEM_SPECS:
+ if spec.match_file(path):
+ return lang, suffixes, scoped
+ return None
+
+
+def _suffix(path: str) -> str:
+ return PurePosixPath(path).suffix.lower()
+
+
+def _is_asset(path: str) -> bool:
+ p = PurePosixPath(path)
+ if p.name.startswith(".") or is_code_file(path):
+ return False
+ suffix = _suffix(path)
+ return (
+ suffix in _OWNED
+ or suffix in _NAMED_ONLY
+ or (suffix in _DATA and len(p.parts) > 1)
+ or is_documentation(path)
+ )
+
+
+def _too_many(namers: tuple[str, ...]) -> str | None:
+ """Why *namers* are too many to trace, or ``None``.
+
+ A namer that is itself a test is selected as it stands, so only the rest,
+ whose tests take a walk to find, count.
+ """
+ walked = [n for n in namers if not is_runnable_test(n)]
+ if len(walked) <= MAX_NAMERS:
+ return None
+ return f"{len(walked)} non-test files name it, more than the {MAX_NAMERS} worth tracing"
+
+
+def _ecosystem_scope(
+ path: str, eco: tuple[str, tuple[str, ...], bool], tree: _Tree, namers: tuple[str, ...]
+) -> Scope:
+ lang, suffixes, scoped = eco
+ root = str(PurePosixPath(path).parent) if scoped else "."
+ if root == ".":
+ why = f"it can change any {lang} test"
+ reach: tuple[str, ...] = ()
+ else:
+ why = f"it can change the {lang} tests under {root}/ and those reaching its code"
+ reach = tree.code_under(root, suffixes)
+ outside = tuple(n for n in namers if not _holds(root, suffixes, n))
+ if outside:
+ why += f", and {_some(outside)} names it"
+ return Scope(why, "ecosystem", root, suffixes, reach, outside, _too_many(outside))
+
+
+def _holds(root: str, suffixes: tuple[str, ...], namer: str) -> bool:
+ """Whether an ecosystem scope runs *namer*'s tests already: its own code or test under *root*.
+
+ A test helper there is not held: only the tests importing it run it.
+ """
+ if not (_is_under(namer, root) and namer.lower().endswith(suffixes)):
+ return False
+ return is_runnable_test(namer) or not is_test_related_path(namer)
+
+
+def _package_scope(path: str, tree: _Tree) -> Scope | None:
+ root = str(PurePosixPath(path).parent)
+ if root == ".":
+ return None
+ return Scope(
+ f"every import of a module under {root}/ runs it",
+ "package-importers",
+ root,
+ PYTHON_SUFFIXES,
+ tree.code_under(root, PYTHON_SUFFIXES),
+ )
+
+
+def _named_scope(path: str, tree: _Tree, namers: tuple[str, ...]) -> Scope | None:
+ """Code naming *path* reads it; else the package holding it is the closest owner.
+
+ A namer inside the owning package is trusted alone. A name written only
+ outside it may be another file's (``layout.css`` is a common name), so the
+ owner's tests run beside the namers'. Only a doc, template, stylesheet or
+ image falls back to its owner when nothing names it.
+ """
+ if namers and not all(is_code_file(n) for n in namers):
+ return None # a doctest glob: the doc is a test
+ if not namers and is_documentation(path):
+ return None # a doc no code names needs no tests
+ if too_many := _too_many(namers):
+ return Scope(f"it is named by {_some(namers)}", "named-by", run_all=too_many)
+ owner = tree.owner(path)
+ if namers and (owner is None or any(_is_under(n, owner) for n in namers)):
+ return Scope(f"it is named by {_some(namers)}", "named-by", namers=namers)
+ if owner is None or not (namers or _suffix(path) in _OWNED):
+ return None
+ why = f"it belongs to the package at {owner}/"
+ if namers:
+ why += f" and is named by {_some(namers)}"
+ return Scope(why, "owner-package", owner, (), tree.code_under(owner, ()), namers)
+
+
+def _some(paths: tuple[str, ...]) -> str:
+ return paths[0] if len(paths) == 1 else f"{paths[0]} and {len(paths) - 1} more"
+
+
+def _is_under(path: str, root: str) -> bool:
+ return root == "." or path.startswith(f"{root}/")
diff --git a/packages/core/src/repowise/core/analysis/test_selection.py b/packages/core/src/repowise/core/analysis/test_selection.py
index 58c298b88..ec7ff23e1 100644
--- a/packages/core/src/repowise/core/analysis/test_selection.py
+++ b/packages/core/src/repowise/core/analysis/test_selection.py
@@ -8,13 +8,14 @@ is chosen when any of these hold:
1. there is no change to select from;
2. a changed or deleted file can change any test: a dependency lock or
- manifest, build or test configuration, a production package's
- ``__init__.py``, a shared test helper, CI config, Repowise's own config, or
- a path listed in ``tests.full_run_on``;
+ manifest no ecosystem scope covers, build configuration (a Makefile, a
+ Dockerfile), runtime or package manager configuration (``.nvmrc``,
+ ``.tool-versions``), a shared test helper, CI config, Repowise's own
+ config, or a path listed in ``tests.full_run_on``;
3. a changed or deleted file sits in a test tree but is not code (data, a
snapshot, a golden file), or is a helper module no test imports;
-4. a changed file is neither code the index knows nor documentation, or is
- documentation some code names (a test may read it);
+4. a changed file is neither code nor documentation and has no scope (below),
+ or its scope finds no test, or a file naming it has no known test;
5. a changed code file has no known test, only a filename guess names one, or
a route to it passes through a test helper no test imports (unless every
known test is selected anyway), or any Python
@@ -43,6 +44,11 @@ package file for each module in it, and pytest loads a conftest for each test
at or below it. Only documentation (``docs/`` and the root README, CHANGELOG,
LICENSE and the like, never code) that no code names is skipped without a test.
+A file whose reach is known runs a scope (:mod:`.selection_scopes`) in place of
+everything: a manifest or lockfile its ecosystem's tests, a production
+``__init__.py`` the tests reaching the modules under it, an asset the tests
+reaching the code naming it or else its package's tests.
+
:func:`runner_args` renders a selection as arguments for one test runner; a
full run renders as the :data:`RUN_ALL` sentinel. pytest and go reject it as a
path; jest and vitest treat arguments as patterns and would match nothing, so
@@ -57,7 +63,7 @@ import shlex
from collections.abc import Collection, Iterable, Mapping
from dataclasses import dataclass, field, replace
from pathlib import PurePosixPath
-from typing import Any
+from typing import TYPE_CHECKING, Any
import pathspec
@@ -65,11 +71,16 @@ from ..pytest_roots import PYTEST_CONFIG_NAMES, PytestRoots
from ..support_paths import DOC_EXTENSIONS
from ..test_paths import is_test_path, is_test_related_path, is_test_support_path
+if TYPE_CHECKING:
+ from .selection_scopes import Scope
+
#: Printed alone instead of arguments when every test must run.
RUN_ALL = ":all"
RUNNERS = ("auto", "pytest", "go", "jest", "files")
+SHARED_TEST_DATA_REASON = "shared test data can change any test"
+
# (why it forces a full run, gitwildmatch patterns). The config extends these,
# never replaces them. Ceiling: no opt-out until someone needs one.
_FULL_RUN_GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
@@ -85,6 +96,9 @@ _FULL_RUN_GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
"Pipfile",
"requirements*.txt",
"constraints*.txt",
+ "requirements*.in",
+ "constraints*.in",
+ "**/requirements/**",
"go.mod",
"go.sum",
"Cargo.lock",
@@ -93,6 +107,9 @@ _FULL_RUN_GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
"Gemfile",
"composer.lock",
"composer.json",
+ "pnpm-workspace.yaml",
+ "go.work",
+ "go.work.sum",
),
),
(
@@ -118,7 +135,19 @@ _FULL_RUN_GROUPS: tuple[tuple[str, tuple[str, ...]], ...] = (
"docker-compose*.y*ml",
),
),
- ("shared test data can change any test", ("**/testdata/**", "**/fixtures/**")),
+ (
+ "runtime or package manager configuration can change any test",
+ (
+ ".nvmrc",
+ ".node-version",
+ ".npmrc",
+ ".yarnrc",
+ ".yarnrc.yml",
+ ".python-version",
+ ".tool-versions",
+ ),
+ ),
+ (SHARED_TEST_DATA_REASON, ("**/testdata/**", "**/fixtures/**")),
(
"CI configuration changed",
(
@@ -178,8 +207,8 @@ _TEST_CODE_SUFFIXES = frozenset(
| {".rs", ".php", ".swift", ".ex", ".exs", ".dart", ".c", ".cc", ".cpp"}
)
-_PYTHON = (".py",)
-_JS = (".js", ".jsx", ".mjs", ".cjs", ".ts", ".tsx", ".mts", ".cts")
+PYTHON_SUFFIXES = (".py",)
+JS_SUFFIXES = (".js", ".jsx", ".mjs", ".cjs", ".ts", ".tsx", ".mts", ".cts")
# coverage.py names each phase of a test as its own context.
_PHASES = ("|run", "|setup", "|teardown")
@@ -238,9 +267,10 @@ class Selection:
``no-tests-needed``, ``deleted-test``, ``test-tree``, ``test-package``,
``conftest``, ``helper-importers``, ``coverage``,
``changed-test``, ``call-graph``, ``import-graph``, ``filename-pattern``,
- ``unknown``, or ``none`` when nothing was asked (no index). ``why`` says,
- per selected test file, what put it in: the changed file and evidence that
- reached it, or the reason it runs with every subset.
+ ``unknown``, ``none`` when nothing was asked (no index), or a scope's
+ ``ecosystem``, ``package-importers``, ``named-by`` or ``owner-package``.
+ ``why`` says, per selected test file, what put it in: the changed file
+ and evidence that reached it, or the reason it runs with every subset.
"""
run_all: bool
@@ -296,25 +326,40 @@ def is_scan_source(path: str) -> bool:
def doc_readers(docs: Collection[str], sources: Iterable[tuple[str, str]]) -> dict[str, str]:
- """``{doc: a file naming it}`` for each of *docs* some code or config names.
+ """``{doc: a file naming it}`` for each of *docs* some code or config names."""
+ return {doc: names[0] for doc, names in file_namers(docs, sources).items()}
- A test that reads a doc names it (``ROOT / "README.md"``), and a
- ``--doctest-glob`` turns every doc into a test, so either means the doc's
- tests are unknown. Ceiling: a test that globs a directory names no file.
+
+def file_namers(files: Collection[str], sources: Iterable[tuple[str, str]]) -> dict[str, list[str]]:
+ """``{file: the code naming it}`` for each of *files* some code or config names.
+
+ Code that reads a file names it (``ROOT / "README.md"``), and a
+ ``--doctest-glob`` turns every doc into a test, so the config declaring
+ one names each doc (first). One substring test per distinct file name and
+ source, and every namer is listed: a caller decides what too many means.
+ Ceiling: code that globs a directory names no file.
"""
- names = {doc: PurePosixPath(doc).name for doc in docs}
- out: dict[str, str] = {}
+ by_name: dict[str, list[str]] = {}
+ for f in files:
+ by_name.setdefault(PurePosixPath(f).name, []).append(f)
+ docs = [f for f in files if PurePosixPath(f).suffix.lower() in _DOC_SUFFIXES]
+ found: dict[str, list[str]] = {}
+ out: dict[str, list[str]] = {}
for path, text in sources:
# Config only for the glob, code only for names: pyproject's
# ``readme = "README.md"`` is packaging, not a test.
if not is_code_file(path):
if "doctest-glob" in text:
- return {doc: path for doc in docs}
+ for doc in docs:
+ out.setdefault(doc, []).insert(0, path)
continue
- for doc, name in names.items():
- if doc not in out and name in text:
- out[doc] = path
- return out
+ for name in by_name:
+ if name in text:
+ found.setdefault(name, []).append(path)
+ for name, namers in found.items():
+ for f in by_name[name]:
+ out.setdefault(f, []).extend(n for n in namers if n != f)
+ return {f: n for f, n in out.items() if n}
def plugin_loader(sources: Iterable[tuple[str, str]]) -> str | None:
@@ -331,7 +376,7 @@ def plugin_loader(sources: Iterable[tuple[str, str]]) -> str | None:
return None
-_PACKAGE_INIT_REASON = "every import of the package runs it, and those are not all tracked"
+PACKAGE_INIT_REASON = "every import of the package runs it, and those are not all tracked"
def scope_kind(path: str) -> str | None:
@@ -379,7 +424,7 @@ def is_runnable_test(path: str, roots: PytestRoots | None = None) -> bool:
@functools.lru_cache(maxsize=8)
-def _extra_spec(patterns: tuple[str, ...]) -> pathspec.PathSpec:
+def full_run_on_spec(patterns: tuple[str, ...]) -> pathspec.PathSpec:
"""``tests.full_run_on`` compiled once per pattern list, not once per path."""
return pathspec.PathSpec.from_lines("gitwildmatch", patterns)
@@ -388,12 +433,12 @@ def full_run_reason(path: str, extra: Iterable[str] = ()) -> str | None:
"""Why a change to *path* can change any test, or ``None``."""
specs = list(_DEFAULT_SPECS)
if extra := tuple(extra):
- specs.append(("it matches tests.full_run_on", _extra_spec(extra)))
+ specs.append(("it matches tests.full_run_on", full_run_on_spec(extra)))
for why, spec in specs:
if spec.match_file(path):
return why
if PurePosixPath(path).name == "__init__.py" and not is_test_related_path(path):
- return _PACKAGE_INIT_REASON
+ return PACKAGE_INIT_REASON
if is_test_support_path(path) and scope_kind(path) is None:
return "a shared test helper can change any test"
return None
@@ -444,6 +489,8 @@ class SelectionInput:
# holds; None when the caller did not trace it, so any code there runs all.
gap: GapPlan | None = None
always_run_tests: Mapping[str, str] = field(default_factory=dict)
+ # :func:`~.selection_scopes.trigger_scopes`' answer; *tiers* holds each route's rows.
+ scopes: Mapping[str, Scope] = field(default_factory=dict)
@dataclass
@@ -460,9 +507,11 @@ def select_tests(inp: SelectionInput) -> Selection:
"""Decide what a change must run (see the module docstring for the rules)."""
deleted = set(inp.deleted)
paths = sorted(set(inp.changed) | deleted)
- triage = _triage(paths, inp.config, inp.doc_readers)
+ scoped = [p for p in paths if p in inp.scopes]
+ triage = _triage([p for p in paths if p not in inp.scopes], inp.config, inp.doc_readers)
run_all = [*_empty_change_reasons(paths, inp.label), *triage.run_all]
- run_all += _index_reasons(inp, has_code=bool(triage.code))
+ walks = bool(triage.code) or any(inp.scopes[p].routes for p in scoped)
+ run_all += _index_reasons(inp, has_code=walks)
evidence = _Evidence.of(inp, deleted)
per_file, file_reasons = _tests_per_file(triage.code, evidence)
@@ -470,29 +519,36 @@ def select_tests(inp: SelectionInput) -> Selection:
gap_tests, route_reasons, gap_why = (
_gap_route_tests(inp.gap.targets, paths, evidence) if traced else ([], [], {})
)
+ in_scope, scope_reasons, scope_notes, scope_why = _scoped_tests(
+ scoped, inp.scopes, evidence
+ )
+ run_all += scope_reasons
if inp.index_available:
run_all += file_reasons + route_reasons
basis = {**triage.basis, **{path: per_file[path][1] for path in triage.code}}
+ basis.update({path: inp.scopes[path].basis for path in scoped})
# The graph cannot say what these tests reach, so they always run.
- always = _always_running(inp) if triage.code else {}
+ always = _always_running(inp) if triage.code or scoped else {}
tests, test_files = _runnable(
[
*(t for path in triage.code for t in per_file[path][0]),
*gap_tests,
+ *in_scope,
*((t, t) for t in always),
]
)
+ notes = _notes(inp, triage.skipped, always, traced)
return Selection(
run_all=bool(run_all),
- reasons=tuple(run_all + evidence.notes + _notes(inp, triage.skipped, always, traced)),
+ reasons=tuple(run_all + evidence.notes + scope_notes + notes),
tests=tests,
test_files=test_files,
packages=_go_packages(triage.code, deleted, inp.go_test_dirs),
always_run=inp.config.always_run,
skipped_files=tuple(triage.skipped),
basis=basis,
- why={**gap_why, **_why(triage.code, per_file, evidence), **always},
+ why={**scope_why, **gap_why, **_why(triage.code, per_file, evidence), **always},
)
@@ -583,6 +639,61 @@ def _triage(
return out
+def _scoped_tests(
+ scoped: list[str], scopes: Mapping[str, Scope], ev: _Evidence
+) -> tuple[list[_TestRef], list[str], list[str], dict[str, str]]:
+ """Each scoped file's tests, the run-all reasons found, a note per file, and ``why``.
+
+ A route shared by several files (one namer, one package) is decided once.
+ """
+ routes: dict[str, tuple[list[_TestRef], list[str]]] = {}
+ tests: list[_TestRef] = []
+ reasons: dict[str, None] = {}
+ notes: list[str] = []
+ why_run: dict[str, str] = {}
+ for path in scoped:
+ scope = scopes[path]
+ if scope.run_all:
+ reasons[f"{path} changed: {scope.run_all}."] = None
+ continue
+ mine = [(t, t) for t in _scope_tests(scope, ev)]
+ before = len(reasons)
+ for route in scope.routes:
+ if route not in routes:
+ found, _, why = _file_tests(route, ev, route_only=True)
+ routes[route] = (found, why)
+ found, why = routes[route]
+ mine += found
+ reasons.update(dict.fromkeys(f"{path} changed; {r}" for r in why))
+ if route in scope.namers and not (found or why):
+ reasons[f"{path} is named by {route}, and no test is known to reach it."] = None
+ if not mine and len(reasons) == before:
+ reasons[f"{path} changed: {scope.why}, and no such test is known."] = None
+ tests += mine
+ files = {f for _, f in mine if f}
+ for f in sorted(files - set(why_run)):
+ why_run[f] = f"{path} changed ({scope.basis}: {scope.why})"
+ notes.append(f"{path} changed: {scope.why}; {len(files)} test file(s) run for it.")
+ if len(notes) > _MAX_SCOPE_NOTES:
+ more = len(notes) - _MAX_SCOPE_NOTES
+ notes = [*notes[:_MAX_SCOPE_NOTES], f"{more} more changed file(s) run a scoped set."]
+ return tests, list(reasons), notes, why_run
+
+
+_MAX_SCOPE_NOTES = 5
+
+
+def _scope_tests(scope: Scope, ev: _Evidence) -> list[str]:
+ """Known tests under *scope*'s root in its suffixes; a test this change deletes is gone."""
+ if scope.root is None:
+ return []
+ return [
+ t
+ for t in _under({scope.root}, ev.known_tests)
+ if (not scope.suffixes or t.lower().endswith(scope.suffixes)) and t not in ev.deleted
+ ]
+
+
def _empty_change_reasons(paths: list[str], label: str) -> list[str]:
return [] if paths else [f"No change against {label}; nothing to select from."]
@@ -692,7 +803,7 @@ def _gap_kind(path: str, config: TestSelectionConfig) -> str | None:
if is_documentation(path):
return None
why = full_run_reason(path, config.full_run_on)
- if why == _PACKAGE_INIT_REASON:
+ if why == PACKAGE_INIT_REASON:
return "package"
if why is None or why == _HELPER_REASON or (why in _GAP_INERT and is_code_file(path)):
return "route"
@@ -945,7 +1056,7 @@ def _own_helper_reasons(path: str, ev: _Evidence) -> list[str]:
def _plugin_reason(helper: str, ev: _Evidence) -> str | None:
"""A Python helper may be a plugin, whose fixtures reach tests that never import it."""
- if not (ev.plugin_loader and helper.endswith(_PYTHON)):
+ if not (ev.plugin_loader and helper.endswith(PYTHON_SUFFIXES)):
return None
return (
f"{helper} is a test helper and {ev.plugin_loader} loads pytest plugins by "
@@ -1009,10 +1120,11 @@ def _no_test_reason(path: str, basis: str, ev: _Evidence) -> str:
def _tests_under(inits: Collection[str], known_tests: Collection[str]) -> list[str]:
"""Every known test below the directory of each scope file."""
- dirs = {str(PurePosixPath(i).parent) for i in inits}
- return [
- t for t in known_tests if any(d == "." or t.startswith(f"{d}/") for d in dirs)
- ]
+ return _under({str(PurePosixPath(i).parent) for i in inits}, known_tests)
+
+
+def _under(dirs: Collection[str], files: Collection[str]) -> list[str]:
+ return [f for f in files if any(d == "." or f.startswith(f"{d}/") for d in dirs)]
def expand_test_scopes(tests: Iterable[str], test_files: Collection[str]) -> list[str]:
@@ -1083,7 +1195,7 @@ def _node_id(test_id: str, test_file: str) -> str:
# The test files each runner takes; ``files`` takes every one.
-_RUNNER_SUFFIXES = {"pytest": _PYTHON, "go": (".go",), "jest": _JS}
+_RUNNER_SUFFIXES = {"pytest": PYTHON_SUFFIXES, "go": (".go",), "jest": JS_SUFFIXES}
def _takes(runner: str, test: str) -> bool:
diff --git a/tests/unit/analysis/test_selection_scopes.py b/tests/unit/analysis/test_selection_scopes.py
new file mode 100644
index 000000000..2dca88b8e
--- /dev/null
+++ b/tests/unit/analysis/test_selection_scopes.py
@@ -0,0 +1,294 @@
+"""Scopes: what a manifest, a package ``__init__.py`` or an asset runs instead of every test."""
+
+from __future__ import annotations
+
+from pathlib import PurePosixPath
+
+import pytest
+
+from repowise.core.analysis.selection_scopes import (
+ ECOSYSTEMS,
+ MAX_NAMERS,
+ needs_namers,
+ trigger_scopes,
+)
+from repowise.core.analysis.test_selection import (
+ SelectionInput,
+ TestSelectionConfig,
+ file_namers,
+ full_run_reason,
+ select_tests,
+)
+
+_NONE = TestSelectionConfig()
+_TRACKED = (
+ "package.json",
+ "pyproject.toml",
+ "web/package.json",
+ "web/src/app.ts",
+ "web/src/app.test.ts",
+ "web/src/styles/layout.css",
+ "web/e2e/login.spec.ts",
+ "svc/go.mod",
+ "svc/main.go",
+ "svc/main_test.go",
+ "src/pkg/__init__.py",
+ "src/pkg/core.py",
+ "src/pkg/sub/deep.py",
+ "src/other.py",
+ "tests/test_core.py",
+ "tests/fixtures/package.json",
+ "crate/Cargo.toml",
+ "assets/logo.svg",
+)
+_TESTS = ["web/src/app.test.ts", "web/e2e/login.spec.ts", "svc/main_test.go", "tests/test_core.py"]
+
+
+def _scopes(paths, namers=None, config=_NONE):
+ return trigger_scopes(paths, _TRACKED, namers or {}, config)
+
+
+def _tiers(*rows, importers=None):
+ inferred = [{"source_file": s, "test_file": t, "via": "import-graph"} for s, t in rows]
+ return {"covered": {}, "inferred": inferred, "unknown": [], "helper_importers": importers or {}}
+
+
+def _select(paths, tiers=None, namers=None, **kwargs):
+ kwargs.setdefault("known_tests", _TESTS)
+ inp = SelectionInput(paths, (), tiers or _tiers(), _NONE, scopes=_scopes(paths, namers), **kwargs)
+ return select_tests(inp)
+
+
+def test_every_ecosystem_file_is_also_a_full_run_trigger() -> None:
+ """A file no scope covers (in a test tree, say) must still run everything."""
+ for _, _, _, globs in ECOSYSTEMS:
+ for glob in globs:
+ sample = glob.replace("**/", "").replace("**", "x.txt").replace("*", "x")
+ for path in (sample, f"deep/dir/{sample}"):
+ assert full_run_reason(path), path
+
+
+ ("path", "root", "suffix", "reach"),
+ [
+ ("web/package.json", "web", ".ts", ("web/src/app.ts",)),
+ ("web/package-lock.json", "web", ".ts", ("web/src/app.ts",)),
+ ("package.json", ".", ".ts", ()),
+ ("web/tsconfig.json", ".", ".ts", ()),
+ ("pnpm-workspace.yaml", ".", ".ts", ()),
+ (".nvmrc", ".", ".js", ()),
+ ("uv.lock", ".", ".py", ()),
+ ("services/api/pyproject.toml", ".", ".py", ()),
+ (".python-version", ".", ".py", ()),
+ ("requirements/base.txt", ".", ".py", ()),
+ ("services/api/deps/constraints-3.txt", ".", ".py", ()),
+ ("requirements-dev.in", ".", ".py", ()),
+ ("services/api/requirements/test.in", ".", ".py", ()),
+ ("svc/go.sum", "svc", ".go", ("svc/main.go",)),
+ ("go.work", ".", ".go", ()),
+ ("Gemfile.lock", ".", ".rb", ()),
+ ("app/build.gradle.kts", ".", ".kt", ()),
+ ],
+)
+def test_a_manifest_or_lockfile_scopes_to_its_ecosystem(path, root, suffix, reach) -> None:
+ scope = _scopes([path])[path]
+ assert (scope.basis, scope.root, scope.reach) == ("ecosystem", root, reach)
+ assert suffix in scope.suffixes
+
+
+ "path",
+ [
+ "crate/Cargo.toml",
+ "Cargo.lock",
+ "Makefile",
+ "Dockerfile",
+ ".tool-versions",
+ ".github/workflows/ci.yml",
+ "tests/fixtures/package.json",
+ "tests/data/pyproject.toml",
+ "__init__.py",
+ # Tool configuration, not an asset any code reads.
+ ".env",
+ ".editorconfig",
+ "web/.eslintrc.json",
+ "renovate.json",
+ "ruff.toml",
+ "web/config.toml",
+ "scripts/release.sh",
+ "web/deps/pins.txt",
+ ],
+)
+def test_what_can_change_any_test_keeps_its_full_run(path) -> None:
+ assert path not in _scopes([path], {path: ["web/src/app.ts"]})
+
+
[email protected]("path", ["web/src/data/users.json", "web/src/queries/find.scm"])
+def test_a_data_or_query_file_nothing_names_keeps_its_full_run(path) -> None:
+ """A test anywhere may glob it, so its package's tests are not enough."""
+ assert _scopes([path]) == {}
+ assert _scopes([path], {path: ["web/src/app.ts"]})[path].basis == "named-by"
+
+
+def test_a_configured_full_run_trigger_is_never_scoped() -> None:
+ config = TestSelectionConfig(full_run_on=("web/package.json",))
+ assert _scopes(["web/package.json"], config=config) == {}
+ assert not needs_namers("web/package.json", config)
+
+
+def test_only_scopable_files_ask_who_names_them() -> None:
+ asked = ["web/package.json", "docs/guide.md", "web/src/styles/layout.css", "a/b.json"]
+ assert all(needs_namers(p, _NONE) for p in asked)
+ for path in ["src/a.py", ".env", "Makefile", "renovate.json", "tests/data/x.json"]:
+ assert not needs_namers(path, _NONE), path
+
+
+def test_a_production_package_init_scopes_to_the_modules_under_it() -> None:
+ scope = _scopes(["src/pkg/__init__.py"])["src/pkg/__init__.py"]
+ assert (scope.basis, scope.root) == ("package-importers", "src/pkg")
+ assert scope.reach == ("src/pkg/__init__.py", "src/pkg/core.py", "src/pkg/sub/deep.py")
+
+
+def test_an_asset_runs_what_names_it_or_its_package() -> None:
+ css = "web/src/styles/layout.css"
+ named = _scopes([css], {css: ["web/src/app.ts"]})[css]
+ assert (named.basis, named.root, named.namers) == ("named-by", None, ("web/src/app.ts",))
+ # A name written only outside the package may be another file's.
+ elsewhere = _scopes([css], {css: ["src/other.py"]})[css]
+ assert (elsewhere.basis, elsewhere.root) == ("owner-package", "web")
+ assert elsewhere.namers == ("src/other.py",) and elsewhere.reach == ("web/src/app.ts",)
+ owned = _scopes([css])[css]
+ assert (owned.basis, owned.root, owned.suffixes) == ("owner-package", "web", ())
+ # The repository root is no package; a doc no code names needs nothing.
+ assert _scopes(["assets/logo.svg", "docs/guide.md"]) == {}
+ changelog = _scopes(["CHANGELOG.md"], {"CHANGELOG.md": ["scripts/lanes.mjs"]})
+ assert changelog["CHANGELOG.md"].namers == ("scripts/lanes.mjs",)
+ # A doctest glob makes the doc a test.
+ assert _scopes(["README.md"], {"README.md": ["pytest.ini"]}) == {}
+
+
+def test_a_query_file_runs_the_tests_of_the_spec_naming_it() -> None:
+ query = "src/pkg/queries/python.scm"
+ scope = _scopes([query], {query: ["src/pkg/specs/python.py"]})[query]
+ assert (scope.basis, scope.namers) == ("named-by", ("src/pkg/specs/python.py",))
+
+
+def test_a_file_named_by_too_many_files_runs_everything() -> None:
+ namers = [f"src/m{i}.py" for i in range(MAX_NAMERS + 1)]
+ for path in ["web/src/styles/layout.css", "web/package.json"]:
+ sel = _select([path], namers={path: namers})
+ assert sel.run_all
+ assert sel.reasons[0] == (
+ f"{path} changed: {MAX_NAMERS + 1} non-test files name it, more than the "
+ f"{MAX_NAMERS} worth tracing."
+ )
+ # Tests naming it are selected as they stand and do not count.
+ tests = [f"tests/test_m{i}.py" for i in range(MAX_NAMERS + 1)]
+ assert _scopes(["web/package.json"], {"web/package.json": tests})["web/package.json"].run_all is None
+
+
+def test_namers_an_ecosystem_scope_already_runs_do_not_count() -> None:
+ """A root manifest runs every JS test, so JS code naming it adds nothing to trace."""
+ js = [f"web/src/m{i}.ts" for i in range(MAX_NAMERS + 5)]
+ scope = _scopes(["package.json"], {"package.json": [*js, "tests/test_versions.py"]})
+ assert scope["package.json"].run_all is None
+ assert scope["package.json"].namers == ("tests/test_versions.py",)
+ # Under a nested manifest, only its own directory is held.
+ nested = _scopes(["web/package.json"], {"web/package.json": ["web/src/app.ts", "ui/x.ts"]})
+ assert nested["web/package.json"].namers == ("ui/x.ts",)
+ helper = _scopes(["web/package.json"], {"web/package.json": ["web/test/helpers/fake.ts"]})
+ assert helper["web/package.json"].namers == ("web/test/helpers/fake.ts",)
+
+
+def test_file_namers_lists_every_file_naming_one() -> None:
+ sources = [
+ ("web/src/app.ts", 'import "./styles/layout.css";\n'),
+ ("src/other.py", 'CSS = "layout.css"\n'),
+ ("pyproject.toml", 'x = "layout.css"\n'),
+ ("web/src/more.ts", '"layout.css"\n'),
+ ]
+ files = ["web/src/styles/layout.css", "other/layout.css"]
+ assert file_namers(files, sources)["other/layout.css"] == [
+ "web/src/app.ts",
+ "src/other.py",
+ "web/src/more.ts",
+ ]
+
+
+def test_a_js_package_manifest_runs_its_tests_and_those_reaching_its_code() -> None:
+ sel = _select(["web/package.json"], _tiers(("web/src/app.ts", "tests/test_core.py")))
+ assert not sel.run_all, sel.reasons
+ assert sel.test_files == ("web/src/app.test.ts", "web/e2e/login.spec.ts", "tests/test_core.py")
+ assert sel.basis == {"web/package.json": "ecosystem"}
+ assert sel.why["web/e2e/login.spec.ts"].startswith("web/package.json changed (ecosystem: ")
+ assert sel.reasons[0].startswith("web/package.json changed: it can change the JavaScript")
+
+
+def test_a_test_reading_a_manifest_as_data_runs_with_its_ecosystem() -> None:
+ reader = "tests/test_versions.py"
+ sel = _select(["package.json"], _tiers((reader, reader)), namers={"package.json": [reader]})
+ assert not sel.run_all, sel.reasons
+ assert sel.test_files == ("web/src/app.test.ts", "web/e2e/login.spec.ts", reader)
+
+
+def test_a_python_lockfile_runs_only_python_tests_and_needs_no_graph() -> None:
+ sel = _select(["uv.lock"], index_available=False)
+ assert not sel.run_all, sel.reasons
+ assert sel.test_files == ("tests/test_core.py",)
+
+
+def test_a_scope_with_no_test_runs_everything() -> None:
+ sel = _select(["svc/go.mod"], known_tests=[])
+ assert sel.run_all
+ assert sel.reasons[0].startswith("svc/go.mod changed: it can change the Go tests under svc/")
+ assert sel.reasons[0].endswith("and no such test is known.")
+
+
+def test_a_namer_no_test_reaches_runs_everything() -> None:
+ css = "web/src/styles/layout.css"
+ sel = _select([css], namers={css: ["web/src/app.ts"]})
+ assert sel.run_all
+ assert sel.reasons[0] == f"{css} is named by web/src/app.ts, and no test is known to reach it."
+ reached = _select(
+ [css], _tiers(("web/src/app.ts", "web/src/app.test.ts")), namers={css: ["web/src/app.ts"]}
+ )
+ assert not reached.run_all and reached.test_files == ("web/src/app.test.ts",)
+ assert reached.basis == {css: "named-by"}
+
+
+def test_a_changed_package_init_runs_the_tests_that_import_the_package_itself() -> None:
+ """``import pkg`` reaches only ``__init__.py``; its rows come from the change's own walk."""
+ tiers = _tiers(("src/pkg/__init__.py", "tests/test_imports_pkg.py"), ("src/pkg/core.py", "tests/test_core.py"))
+ sel = _select(["src/pkg/__init__.py"], tiers)
+ assert not sel.run_all, sel.reasons
+ assert sel.test_files == ("tests/test_imports_pkg.py", "tests/test_core.py")
+
+
+def test_a_route_through_an_unimported_helper_still_runs_everything() -> None:
+ sel = _select(["src/pkg/__init__.py"], _tiers(("src/pkg/core.py", "tests/helpers.py")))
+ assert sel.run_all
+ assert sel.reasons[0].startswith(
+ "src/pkg/__init__.py changed; src/pkg/core.py is reached through the test helper"
+ )
+
+
+def test_a_route_shared_by_files_is_decided_once() -> None:
+ css, svg = "web/src/styles/layout.css", "web/src/styles/icon.svg"
+ tiers = _tiers(("web/src/app.ts", "tests/helpers.py"))
+ sel = _select([css, svg], tiers, namers={css: ["web/src/app.ts"], svg: ["web/src/app.ts"]})
+ helper = [r for r in sel.reasons if "tests/helpers.py" in r]
+ assert len(helper) == 2 and {r.split()[0] for r in helper} == {css, svg}
+
+
+def test_a_scope_that_walks_the_graph_needs_the_index() -> None:
+ sel = _select(["web/package.json"], index_available=False)
+ assert sel.run_all and sel.reasons[0].startswith("No index")
+
+
+def test_unplaced_tests_run_with_a_scoped_selection() -> None:
+ sel = _select(["uv.lock"], unplaced_tests=["tests/test_cli.py"])
+ assert sel.test_files == ("tests/test_core.py", "tests/test_cli.py")
+
+
+def test_scope_paths_are_posix() -> None:
+ assert all(PurePosixPath(p).as_posix() == p for p in _TRACKED)
diff --git a/tests/unit/cli/test_impacted_tests_cmd.py b/tests/unit/cli/test_impacted_tests_cmd.py
index a9a475dcb..b5e582961 100644
--- a/tests/unit/cli/test_impacted_tests_cmd.py
+++ b/tests/unit/cli/test_impacted_tests_cmd.py
@@ -124,14 +124,23 @@ def test_json_adds_the_selection_to_the_report(repo) -> None:
assert data["selected"]["basis"] == {"README.md": "no-tests-needed", "src/a.py": "import-graph"}
-def test_a_lockfile_change_runs_everything_and_says_why(repo) -> None:
+def test_a_lockfile_change_runs_its_ecosystem_tests_and_says_why(repo) -> None:
_write(repo, {"uv.lock": "lock\n"})
_git(repo, "add", "-A")
_git(repo, "commit", "-qm", "deps")
result = _run(repo, "main...feat", "--format", "args")
assert result.exit_code == 0, result.output
+ assert result.stdout == "tests/test_a.py\n"
+ assert "uv.lock changed: it can change any Python test; 1 test file(s)" in _err(result)
+
+
+def test_a_build_file_change_runs_everything_and_says_why(repo) -> None:
+ _write(repo, {"Dockerfile": "FROM python\n"})
+ _git(repo, "add", "-A")
+ _git(repo, "commit", "-qm", "image")
+ result = _run(repo, "main...feat", "--format", "args")
assert result.stdout == ":all\n"
- assert "uv.lock changed: dependencies can change any test." in _err(result)
+ assert "Dockerfile changed: build or test configuration can change any test." in _err(result)
def test_a_new_untested_file_runs_everything(repo) -> None:
@@ -269,11 +278,18 @@ def _land_on_main(repo, files: dict[str, str]) -> None:
_git(repo, "rebase", "-q", "main")
-def test_a_doc_a_test_reads_runs_everything(repo) -> None:
+def test_a_doc_a_test_reads_selects_that_test(repo) -> None:
_land_on_main(repo, {"tests/test_readme.py": 'README = "README.md"\n'})
result = _run(repo, "main...feat", "--format", "args")
+ assert result.stdout == "tests/test_a.py tests/test_readme.py\n"
+ assert "README.md changed: it is named by tests/test_readme.py; 1 test file(s)" in _err(result)
+
+
+def test_a_doc_named_by_code_no_test_reaches_runs_everything(repo) -> None:
+ _land_on_main(repo, {"src/b.py": 'DOC = "README.md"\n'})
+ result = _run(repo, "main...feat", "--format", "args")
assert result.stdout == ":all\n"
- assert "README.md is named by tests/test_readme.py, so a test may read it" in _err(result)
+ assert "README.md is named by src/b.py, and no test is known to reach it." in _err(result)
def test_tests_the_graph_cannot_see_run_and_production_modules_do_not(repo) -> None: