Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(tests): one reach walk over an in-memory or stored graph, with typed reach states (#4329)

Raghav Chamadiyab167bc48eac6
diff --git a/docs/architecture/test-intelligence.md b/docs/architecture/test-intelligence.md
index 407c12737..379368d8b 100644
--- a/docs/architecture/test-intelligence.md
+++ b/docs/architecture/test-intelligence.md
@@ -4,8 +4,7 @@ How repowise answers "which tests reach this file" and "which tests should I
run" with no coverage report. For the user-facing layer, see
[layers/TEST_INTELLIGENCE.md](../layers/TEST_INTELLIGENCE.md).
-Code: `packages/core/src/repowise/core/analysis/test_reachability.py` (the
-walks), `analysis/test_selection.py` (runner arguments for `impacted-tests`),
+Code: `packages/core/src/repowise/core/analysis/test_reach/` (the walks), `analysis/test_selection.py` (runner arguments for `impacted-tests`),
`analysis/health/coverage/` (report parsers, discovery, path matching) and
`persistence/crud/analysis/coverage_map.py` (the measured per-test map).
@@ -129,6 +128,17 @@ The health pass computes the whole-repo answer in one multi-source walk, once
per run, and caches it. The per-change walk reads the same edge table
`pr_blast` reads.
+## One walk, two graph sources
+
+The reverse walks are written once and run over the stored graph or over a
+graph held in memory (`build_walk_maps` on the NetworkX graph or its node-link
+JSON), and both return the same rows. `reach_states` gives each file a typed
+state: `reached`, `unreached`, `new`, `no_signal` or `type_only`. A file is
+`unreached` only when the call walk, the helper and barrel hop and the whole
+import closure all miss it, and a test the change adds or edits that is named
+for the file or imports it, or the file's own inline tests (Rust
+`#[cfg(test)]`, a Go `_test.go` beside it), count as reaching it.
+
## Measured map internals
- `.coverage` is opened read-only as SQLite. Repowise decodes coverage.py's
diff --git a/packages/core/src/repowise/core/analysis/conftest_routes.py b/packages/core/src/repowise/core/analysis/conftest_routes.py
index 517608496..8f2521558 100644
--- a/packages/core/src/repowise/core/analysis/conftest_routes.py
+++ b/packages/core/src/repowise/core/analysis/conftest_routes.py
@@ -84,7 +84,7 @@ from ..ingestion.framework_edges.pytest_edges import (
)
from ..ingestion.languages.python_modules import module_parts
from ..ingestion.languages.python_strings import PY_DYNAMIC_LOAD_MARKERS
-from .test_reachability import _in_clause
+from .test_reach.db import _in_clause
from .test_selection import BROADCAST_VIA, is_runnable_test, scope_kind
# The call whose first argument is the object an attribute is patched on:
diff --git a/packages/core/src/repowise/core/analysis/test_reach/__init__.py b/packages/core/src/repowise/core/analysis/test_reach/__init__.py
new file mode 100644
index 000000000..70c68240c
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/__init__.py
@@ -0,0 +1,248 @@
+"""Static test-to-code map, derived from the dependency graph.
+
+``test_coverage`` answers "which test executed this line" and is filled only by
+a coverage report. Most repositories have none, so ``tests_to_run`` came back
+empty, ``impacted_tests`` said "run the full suite", and ``untested_hotspot``
+fell back to matching filenames.
+
+Matching filenames fails both ways, and this repository is the proof. Of its six
+worst bug-magnet files, five - ``call_resolver.py``, ``dead_code/analyzer.py``,
+``pipeline/persist.py``, ``tool_answer/answer.py``, ``pr_blast.py`` - have no
+file named for them anywhere under ``tests/`` and so read as untested, while the
+graph names the test files that reach them. The sixth,
+``analysis/health/engine.py``, is worse: the convention matches on basename
+alone, so it paired with ``tests/unit/distill/test_engine.py`` - a different
+engine, in a different subsystem - and called the file tested on the strength of
+a name collision.
+
+The graph already knows the answer. This module reads that relation. It is a
+second signal beside the measured one, never a replacement and never averaged
+with it.
+
+Which graph, and why
+--------------------
+Two graphs can answer "does a test reach this file", and they are not equally
+good at it.
+
+The **call graph** is the primary signal: seed the symbols a test file declares,
+walk ``EXECUTION_EDGE_TYPES`` forward, and every symbol reached is one the test
+can actually run. This is the closer model of the question - the claim is about
+execution, and these edges are execution.
+
+The **import graph** is the broader, weaker tier: a test file that imports a
+source file references it, which is real evidence and a much cruder one. A test
+importing a module pulls in every symbol the module defines while the test body
+may touch one function.
+
+Measured against real per-test coverage on this repository,
+over the slice where per-test attribution is complete and both sides see the
+same 37 test files (159 production files provably executed):
+
+============================ ======= ======= ========= =======
+forward signal claims correct precision recall
+============================ ======= ======= ========= =======
+import graph, 1 hop 43 31 72.1% 19.5%
+call graph, 3 hops 48 44 91.7% 27.7%
+call graph, 3 hops, filtered 46 44 95.7% 27.7%
+both unioned 57 45 78.9% 28.3%
+============================ ======= ======= ========= =======
+
+The call graph wins on both axes, so it leads. Unioning the import graph into it
+buys 0.6 points of recall for 16.8 points of precision, and a false "something
+reaches this" suppresses a real untested-hotspot finding, so the forward walk
+does not union - it is call edges only.
+
+The reverse walk combines them differently, because there the import tier can be
+spent only on the targets the call graph said nothing about:
+
+============================= ======= ====== =========
+reverse signal targets hit precision
+============================= ======= ====== =========
+import graph, 1 hop 32 96.9% 94.8%
+call graph, 3 hops, filtered 46 100.0% 97.5%
+both unioned 47 100.0% 95.8%
+call graph, else import graph 47 100.0% 97.5%
+============================= ======= ====== =========
+
+Falling back is free where unioning is not: it answers one more target at
+identical precision, because the second tier never speaks over the first.
+
+Both tables compare the signals uncapped and over the same test set, which is
+what makes them a choice between signals rather than two different experiments.
+End to end through this module, with the walk seeing the repository's whole test
+set and ``MAX_TESTS_PER_TARGET`` applied, the reverse walk scores 100.0% hit and
+95.5% precision on the same slice; the gap is the cap truncating a repo-wide
+list, not a difference in the walk.
+
+Depth, and why 3
+----------------
+The call walk saturates at 3 hops - 3, 4 and 5 return the same 48 claims and 44
+confirmations - so the recall ceiling is the call graph's own capture rate, not
+the depth, and there is nothing to buy by walking further. The import walk keeps
+its measured default of 1: a blanket second hop was tested and recovers 20 of
+128 misses while adding 50 wrong claims, and both a facade-only second hop
+through ``__init__.py`` (3 recovered, 5 added) and ``conftest.py`` transitivity
+(zero recovered) were tested and disproved. What the import graph misses is
+transitive *execution*, which import edges structurally cannot see, which is
+what the call graph is for.
+
+Why recall reads low, and why that is not the graph's fault
+-----------------------------------------------------------
+27.7% looks poor for a graph this dense, and the reason is what the ground truth
+counts. ``coverage`` records a line as run whether a test called into it or
+Python merely evaluated the module body on import, so a file that was only
+imported is indistinguishable in the truth set from one a test drove. Splitting
+the 159 truth files by how much of what ran was *inside a function body*:
+
+================================= ======= ======= ========
+what actually ran in the file files found recall
+================================= ======= ======= ========
+nothing inside any function body 39 0 0%
+under a quarter inside bodies 19 0 0%
+a quarter to three quarters 45 3 7%
+over three quarters inside bodies 56 43 77%
+================================= ======= ======= ========
+
+A quarter of the "covered" files never ran a single line inside a function. The
+walk is right to miss those: nothing called them, and claiming otherwise is the
+import graph's error, which is what it costs 16.8 points of precision to make.
+
+Of the 13 missed in the bottom row, 11 are ``alembic/versions/*`` migrations,
+whose ``upgrade``/``downgrade`` the framework invokes by naming convention with
+no static caller anywhere in the repository. Excluding those, the walk finds 43
+of 45 files that a test genuinely exercised and that anything statically calls.
+Chasing the headline number by widening the walk trades that away.
+
+Confidence, and why one origin is dropped
+-----------------------------------------
+``graph_edges.resolution_origin`` records which strategy resolved a call edge,
+and they are not equally trustworthy. ``global_unique`` binds a name to the only
+symbol carrying it anywhere in the repository, which is a guess, and it is the
+one origin the vocabulary itself scores at 0.50. Dropping it costs no recall at
+all and buys 4.0 points of forward precision (91.7% -> 95.7%) and 1.1 of reverse
+(96.4% -> 97.5%). Nothing else earns its keep: filtering ``receiver_global`` too
+changes no number, and a confidence floor of 0.90 cuts recall from 27.7% to
+23.3%.
+
+What it claims, and what it does not
+------------------------------------
+Reaching is not executing. A call edge says control *can* flow, not that a given
+test run did, so the inferred map **over-claims**, and the direction of the
+error is known and one-sided:
+
+* Sound as a floor. "Some test reaches this file, so do not call it untested"
+ is safe: the edge is recorded, not guessed.
+* Unsound as a quantity. No percentage may be derived from it. There is no line
+ attribution here at all - reaching is a file-level fact - so a caller that
+ needs "are these changed *lines* covered" must use the measured map or say it
+ does not know.
+
+Every surface that consumes this labels the result ``inferred``. Nothing here
+writes to ``test_coverage``, and no row it produces is stored: the relation is a
+bounded walk over ``graph_edges``/``graph_nodes``, which are already indexed and
+already fresh. Materialising it would cost a transitive closure that goes stale
+the moment the graph moves, to answer a query that is a breadth-first search
+over data already in the database.
+
+Layout
+------
+The walks are written once, over the needs they yield (:mod:`.walk`), and run
+either over a graph held in memory (:func:`build_walk_maps`, no database and no
+checkout) or over the stored graph (:mod:`.db`). :func:`reach_states` is the
+typed answer, the one that may say "no test reaches this".
+"""
+
+
+from ..test_selection import is_runnable_test
+from .db import (
+ always_run_test_files,
+ any_tests_reaching,
+ cached_files_reached_by_tests,
+ cached_test_files,
+ call_graph_from_db,
+ clear_test_map_cache,
+ dependency_path,
+ direct_dependents,
+ direct_importers,
+ imported_names_by_test,
+ load_import_graph,
+ load_test_files,
+ placed_test_files,
+ reach_into_symbols,
+ reach_states_from_db,
+ tests_reaching,
+ tests_reaching_by_tier,
+ tests_reaching_through_helpers,
+ unscanned_test_files,
+)
+from .forward import (
+ CallGraphView,
+ call_graph_from_graph,
+ files_reached_by_tests,
+ files_with_paired_tests,
+)
+from .inline import go_inline_tests, inline_tests, rust_has_inline_tests
+from .maps import ImportGraph, WalkMaps, build_walk_maps
+from .model import (
+ DEFAULT_CALL_DEPTH,
+ DEFAULT_MAX_DEPTH,
+ MAX_TESTS_PER_TARGET,
+ UNRELIABLE_CALL_ORIGINS,
+ ReachDistance,
+ ReachedBy,
+ ReachedVia,
+ ReachRow,
+ ReachState,
+ rank_reaching,
+ tests_matching_by_name,
+)
+from .type_only import is_type_only
+from .walk import CLOSURE_SEED_CAP, reach_states, reaching_by_tier
+
+__all__ = [
+ "CLOSURE_SEED_CAP",
+ "DEFAULT_CALL_DEPTH",
+ "DEFAULT_MAX_DEPTH",
+ "MAX_TESTS_PER_TARGET",
+ "UNRELIABLE_CALL_ORIGINS",
+ "CallGraphView",
+ "ImportGraph",
+ "ReachDistance",
+ "ReachRow",
+ "ReachState",
+ "ReachedBy",
+ "ReachedVia",
+ "WalkMaps",
+ "always_run_test_files",
+ "any_tests_reaching",
+ "build_walk_maps",
+ "cached_files_reached_by_tests",
+ "cached_test_files",
+ "call_graph_from_db",
+ "call_graph_from_graph",
+ "clear_test_map_cache",
+ "dependency_path",
+ "direct_dependents",
+ "direct_importers",
+ "files_reached_by_tests",
+ "files_with_paired_tests",
+ "go_inline_tests",
+ "imported_names_by_test",
+ "inline_tests",
+ "is_runnable_test",
+ "is_type_only",
+ "load_import_graph",
+ "load_test_files",
+ "placed_test_files",
+ "rank_reaching",
+ "reach_into_symbols",
+ "reach_states",
+ "reach_states_from_db",
+ "reaching_by_tier",
+ "rust_has_inline_tests",
+ "tests_matching_by_name",
+ "tests_reaching",
+ "tests_reaching_by_tier",
+ "tests_reaching_through_helpers",
+ "unscanned_test_files",
+]
diff --git a/packages/core/src/repowise/core/analysis/test_reach/db.py b/packages/core/src/repowise/core/analysis/test_reach/db.py
new file mode 100644
index 000000000..6674d4113
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/db.py
@@ -0,0 +1,642 @@
+"""The reach walks over the stored graph: one query per level, the same walks as in memory.
+
+Each reader here answers the :class:`~.maps.Need` a walk yields with SQL over
+``graph_edges`` / ``graph_nodes`` and drives the generators of :mod:`.walk`, so
+a walk over the database and one over a graph held in memory return the same
+rows.
+"""
+
+from __future__ import annotations
+
+import json
+from collections.abc import Collection, Mapping
+from pathlib import PurePosixPath
+from typing import Any, TypeVar
+
+from sqlalchemy import select, text
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from repowise.core.analysis.dead_code.file_reachability import BARREL_FILENAMES
+from repowise.core.analysis.execution_graph import ExecutionGraphIndex, file_of_symbol
+from repowise.core.ingestion.models import EXECUTION_EDGE_TYPES, FILE_DEPENDENCY_EDGE_TYPES
+from repowise.core.persistence.index_cache import IndexStampCache
+from repowise.core.persistence.models import GraphNode
+
+from .forward import CallGraphView, files_reached_by_tests
+from .maps import LEGACY_CONFTEST_HINT, ImportGraph, Need, Walk
+from .model import (
+ DEFAULT_CALL_DEPTH,
+ DEFAULT_MAX_DEPTH,
+ UNRELIABLE_CALL_ORIGINS,
+ ReachDistance,
+ ReachedBy,
+ ReachRow,
+)
+from .walk import call_walk, helper_walk, states_walk, tier_walk
+
+_T = TypeVar("_T")
+
+
+async def drive(
+ walk: Walk[_T],
+ session: AsyncSession,
+ repo_id: str,
+ *,
+ import_graph: ImportGraph | None = None,
+) -> _T:
+ """Drive *walk* to its answer from the stored graph, one query per :class:`Need`.
+
+ With *import_graph* (:func:`load_import_graph`) the import levels are read
+ from it instead, for a walk that goes deep or runs more than once.
+ """
+ try:
+ need = next(walk)
+ while True:
+ need = walk.send(await _answer(need, session, repo_id, import_graph))
+ except StopIteration as done:
+ return done.value
+
+
+async def _answer(
+ need: Need, session: AsyncSession, repo_id: str, import_graph: ImportGraph | None
+) -> Any:
+ keys = list(need.keys)
+ if need.kind == "declared":
+ return await _edges_from(session, repo_id, keys, ["defines"])
+ if need.kind == "callers":
+ return await _edges_into(
+ session, repo_id, keys, sorted(EXECUTION_EDGE_TYPES), UNRELIABLE_CALL_ORIGINS
+ )
+ if need.kind == "importers":
+ if import_graph is not None:
+ return import_graph.edges_into(keys)
+ return await _edges_into(
+ session, repo_id, keys, sorted(FILE_DEPENDENCY_EDGE_TYPES), frozenset()
+ )
+ return await _known_files(session, repo_id, keys)
+
+
+async def call_graph_from_db(session: AsyncSession, repo_id: str) -> CallGraphView:
+ """The same view as :func:`call_graph_from_graph`, read from ``graph_edges``.
+
+ The health engine holds the parsed graph in memory; a request handler holds
+ a session. Both need the identical view, so this is a loader and not a
+ second walk - :func:`files_reached_by_tests` still does the walking, and
+ the two callers therefore cannot disagree about which files a test reaches.
+
+ One bulk read rather than the reverse walk's per-level ``IN`` queries. That
+ walk is seeded with a change's files; this question is seeded with the
+ repository, and a per-level ``IN`` list over every file is the shape it was
+ explicitly not built for.
+
+ The view carries no call-site lines: reachability reads only ``declares``
+ and the forward/reverse adjacency, and ``call_lines_json`` was most of the
+ read's cost (measured on this repository, 189k edges: 8.4 s cold with the
+ column, under 2 s without it).
+ """
+ wanted = ["defines", *sorted(EXECUTION_EDGE_TYPES)]
+ params: dict[str, Any] = {"repo_id": repo_id}
+ ets = _in_clause("e", wanted, params)
+ rows = await session.execute(
+ text(
+ "SELECT source_node_id, target_node_id, edge_type, resolution_origin "
+ "FROM graph_edges WHERE repository_id = :repo_id "
+ f"AND edge_type IN ({ets}) "
+ "ORDER BY source_node_id, target_node_id, edge_type"
+ ),
+ params,
+ )
+ # No call or spawn lines here: test reachability reads only forward/reverse,
+ # never first-hop sites, ``spawn_only`` or roles, and a spawned callee is
+ # still reached.
+ return ExecutionGraphIndex(
+ edge_rows=((src, dst, edge_type, origin, None) for src, dst, edge_type, origin in rows)
+ )
+
+
+async def load_test_files(session: AsyncSession, repo_id: str) -> set[str]:
+ """Paths of every file node ingestion classified as a test."""
+ res = await session.execute(
+ select(GraphNode.node_id).where(
+ GraphNode.repository_id == repo_id,
+ GraphNode.is_test == True, # noqa: E712
+ )
+ )
+ return {row[0] for row in res.all()}
+
+
+# Answers kept in process per (database, repository, index stamp), so one
+# request reads the test files once whoever asks. Ceiling: a per-file reach is
+# still walked once per index; persisting it at index time (the engine's
+# ``RepoTestIndex.reached_files`` set, stored as a per-file bool) is the upgrade path.
+_cache = IndexStampCache(32)
+
+
+def clear_test_map_cache() -> None:
+ _cache.clear()
+
+
+async def cached_test_files(session: AsyncSession, repo_id: str) -> frozenset[str]:
+ """:func:`load_test_files`, read once per index state; frozen because it is shared."""
+ key = await _cache.key(session, repo_id, "test_files")
+ if key in _cache:
+ return _cache.get(key)
+ return _cache.remember(key, frozenset(await load_test_files(session, repo_id)))
+
+
+async def cached_files_reached_by_tests(session: AsyncSession, repo_id: str) -> frozenset[str]:
+ """:func:`files_reached_by_tests` over the stored graph, walked once per index state.
+
+ The answer moves only when an index run rewrites the graph, so the bulk
+ edge read is paid once per index.
+ """
+ key = await _cache.key(session, repo_id, "files_reached")
+ if key in _cache:
+ return _cache.get(key)
+ test_files = set(await cached_test_files(session, repo_id))
+ view = await call_graph_from_db(session, repo_id)
+ return _cache.remember(key, frozenset(files_reached_by_tests(view, test_files)))
+
+
+async def any_tests_reaching(
+ session: AsyncSession, repo_id: str, targets: Collection[str]
+) -> dict[str, ReachedBy]:
+ """The targets some test reaches, and how; a target it leaves out is unreached.
+
+ The ``reached`` rows of :func:`reach_states_from_db`: the call and import
+ tiers :func:`tests_reaching_by_tier` runs at its defaults (so the two never
+ disagree about a file it answers), the helper and barrel hop, then the
+ whole import closure, so absence is a claim. Cached per index state.
+ """
+ seeds = tuple(sorted(set(targets)))
+ key = await _cache.key(session, repo_id, "any_reach", seeds)
+ if key in _cache:
+ return _cache.get(key)
+ test_files = set(await cached_test_files(session, repo_id))
+ rows = await drive(states_walk(list(seeds), test_files), session, repo_id)
+ found = {path: row.reached for path, row in rows.items() if row.reached is not None}
+ return _cache.remember(key, found)
+
+
+async def reach_states_from_db(
+ session: AsyncSession,
+ repo_id: str,
+ paths: Collection[str],
+ *,
+ changed_tests: Collection[str] = frozenset(),
+ contents: Mapping[str, str | bytes] | None = None,
+) -> dict[str, ReachRow]:
+ """:func:`~.walk.reach_states` over the stored graph."""
+ test_files = set(await cached_test_files(session, repo_id))
+ walk = states_walk(list(paths), test_files, changed_tests=changed_tests, contents=contents)
+ return await drive(walk, session, repo_id)
+
+
+async def tests_reaching_through_helpers(
+ session: AsyncSession, repo_id: str, targets: list[str], test_files: set[str]
+) -> dict[str, ReachedBy]:
+ """:func:`~.walk.helper_walk` over the stored graph."""
+ return await drive(helper_walk(targets, test_files), session, repo_id)
+
+
+async def placed_test_files(session: AsyncSession, repo_id: str) -> set[str]:
+ """Test files with a resolved dependency edge into a non-test file of the repository.
+
+ Only these can be found by walking back from a changed file; a test with
+ none (every import unresolved, or it only runs a subprocess) is invisible.
+ An edge into test material (a conftest, a helper) or a third-party module
+ (``external:``) does not count: it says nothing about which of the
+ repository's files the test exercises.
+ """
+ params: dict[str, Any] = {"repo_id": repo_id, "is_test": True, "not_test": False}
+ ets = _in_clause("e", sorted(FILE_DEPENDENCY_EDGE_TYPES), params)
+ rows = await session.execute(
+ text(
+ "SELECT DISTINCT e.source_node_id FROM graph_edges e "
+ "JOIN graph_nodes s ON s.repository_id = e.repository_id "
+ "AND s.node_id = e.source_node_id "
+ "JOIN graph_nodes t ON t.repository_id = e.repository_id "
+ "AND t.node_id = e.target_node_id "
+ f"WHERE e.repository_id = :repo_id AND e.edge_type IN ({ets}) "
+ "AND s.is_test = :is_test AND t.is_test = :not_test AND t.node_type = 'file' "
+ "AND t.node_id NOT LIKE 'external:%'"
+ ),
+ params,
+ )
+ return {row[0] for row in rows}
+
+
+async def always_run_test_files(session: AsyncSession, repo_id: str) -> dict[str, str]:
+ """``{test file: why}`` for the tests the indexer found the graph cannot see into.
+
+ Stored per file at index time (``repowise.core.ingestion.always_run``), so
+ this is one indexed read, never a scan of the test files.
+ """
+ res = await session.execute(
+ select(GraphNode.node_id, GraphNode.always_run_reason).where(
+ GraphNode.repository_id == repo_id,
+ GraphNode.always_run_reason.is_not(None),
+ GraphNode.always_run_reason != "",
+ )
+ )
+ return {node_id: reason for node_id, reason in res.all()}
+
+
+async def unscanned_test_files(session: AsyncSession, repo_id: str) -> set[str]:
+ """Judged test files the indexer never stamped (an index built before the check).
+
+ A stamped test holds ``""`` when it is ordinary, so ``NULL`` means unknown,
+ not "nothing found": such a test may walk the tree unseen.
+ """
+ from repowise.core.ingestion.always_run import is_judged_test
+
+ res = await session.execute(
+ select(GraphNode.node_id).where(
+ GraphNode.repository_id == repo_id,
+ GraphNode.node_type == "file",
+ GraphNode.is_test == True, # noqa: E712
+ GraphNode.always_run_reason.is_(None),
+ )
+ )
+ return {node_id for (node_id,) in res.all() if is_judged_test(node_id)}
+
+
+async def dependency_path(
+ session: AsyncSession, repo_id: str, source: str, targets: Collection[str], max_depth: int = 64
+) -> list[str]:
+ """The shortest file-dependency route from *source* to any of *targets*, ends included.
+
+ ``[]`` when none is found within *max_depth* hops. Breadth-first, one ``IN``
+ query per level, over the same edges the import walk follows
+ (``FILE_DEPENDENCY_EDGE_TYPES``, every resolution origin). Ceiling: a test
+ found through the call walk alone may have no such route; following call
+ edges with ``UNRELIABLE_CALL_ORIGINS`` excluded is the upgrade path.
+ """
+ goals = set(targets)
+ parent: dict[str, str | None] = {source: None}
+ frontier = [source]
+ for _ in range(max_depth):
+ if not frontier or goals & parent.keys():
+ break
+ level, frontier = frontier, []
+ for dependent, dependency in await _edges_from(
+ session, repo_id, level, sorted(FILE_DEPENDENCY_EDGE_TYPES)
+ ):
+ if dependency not in parent:
+ parent[dependency] = dependent
+ frontier.append(dependency)
+ hit = goals & parent.keys()
+ if not hit:
+ return []
+ route, node = [], min(hit)
+ while node is not None:
+ route.append(node)
+ node = parent[node]
+ return route[::-1]
+
+
+async def tests_reaching(
+ session: AsyncSession,
+ repo_id: str,
+ targets: list[str],
+ *,
+ call_depth: int = DEFAULT_CALL_DEPTH,
+ import_depth: int = DEFAULT_MAX_DEPTH,
+ test_files: set[str] | None = None,
+) -> dict[str, list[str]]:
+ """Test files that reach each of *targets*, keyed by target path.
+
+ Targets nothing reaches are absent from the result. An empty dict is
+ "no test reaches these"; the caller decides whether that reads as untested
+ or as unknown.
+ """
+ found = await tests_reaching_by_tier(
+ session,
+ repo_id,
+ targets,
+ call_depth=call_depth,
+ import_depth=import_depth,
+ test_files=test_files,
+ )
+ return {target: reached.tests for target, reached in found.items()}
+
+
+async def tests_reaching_by_tier(
+ session: AsyncSession,
+ repo_id: str,
+ targets: list[str],
+ *,
+ call_depth: int = DEFAULT_CALL_DEPTH,
+ import_depth: int = DEFAULT_MAX_DEPTH,
+ symbol_seeds: Mapping[str, Collection[str]] | None = None,
+ test_files: set[str] | None = None,
+ avoid: Collection[str] = frozenset(),
+ import_graph: ImportGraph | None = None,
+) -> dict[str, ReachedBy]:
+ """:func:`tests_reaching`, also saying which tier answered each target.
+
+ *test_files* is :func:`load_test_files`'s answer, for a caller that walks
+ more than once and need not read it twice. A target may be a symbol id
+ (``path::name``); its tests are ranked against the file part.
+
+ *symbol_seeds* narrows the call walk for the targets it names: the walk
+ enters at exactly those symbol ids instead of at every symbol the file
+ declares. A target it does not name, or names with no ids, keeps the
+ ``defines`` lookup, and the
+ import tier stays file-level either way.
+
+ *avoid* names files the call walk does not pass reach through, except
+ into a target's own symbols. The import tier is
+ one hop, so it never passes through anything.
+
+ *import_graph* (:func:`load_import_graph`) answers the import tier's
+ levels from memory, for a caller that walks deep or more than once.
+
+ The walk is :func:`~.walk.tier_walk` without the helper hop: the call walk
+ first, then the import walk seeded with only the targets it left
+ unanswered, so the weaker tier never speaks over the stronger one and
+ costs nothing where the stronger one already spoke.
+
+ Both walks are attributed: they run backwards from the targets carrying the
+ seed each visited node came from, so the answer says *which* target each
+ test guards rather than only that some test does. That is why these are
+ separate walks rather than a filter over the forward one, and it stays cheap
+ because the seed set is a change's files, not the repository - one ``IN``
+ query per level, which is the shape ``pr_blast._transitive_affected``
+ already uses.
+ """
+ if not any(targets):
+ return {}
+ if test_files is None:
+ test_files = await load_test_files(session, repo_id)
+ walk = tier_walk(
+ list(targets),
+ test_files,
+ call_depth=call_depth,
+ import_depth=import_depth,
+ symbol_seeds=symbol_seeds,
+ avoid=avoid,
+ )
+ return await drive(walk, session, repo_id, import_graph=import_graph)
+
+
+async def reach_into_symbols(
+ session: AsyncSession,
+ repo_id: str,
+ symbol_ids: Collection[str],
+ test_files: set[str],
+ *,
+ max_depth: int = DEFAULT_CALL_DEPTH,
+ avoid: Collection[str] = frozenset(),
+) -> dict[str, dict[str, ReachDistance]]:
+ """Tests that call into each symbol id, keyed by the symbol, with their hops.
+
+ The file walk answers "does a test reach this file"; this one answers how
+ close each test gets to one symbol in it, which is what decides which of a
+ file's many tests to run first after changing that symbol.
+ """
+ seeds = sorted({symbol for symbol in symbol_ids if symbol})
+ if not seeds or not test_files:
+ return {}
+ walk = call_walk(
+ seeds,
+ test_files,
+ max_depth,
+ symbol_seeds={symbol: (symbol,) for symbol in seeds},
+ avoid=avoid,
+ )
+ return await drive(walk, session, repo_id)
+
+
+async def imported_names_by_test(
+ session: AsyncSession, repo_id: str, files: Collection[str], test_files: set[str]
+) -> dict[str, dict[str, frozenset[str]]]:
+ """The names each test file imports from each of *files*, keyed by file then test.
+
+ An empty set is an import of the module itself. One hop, plus one through a
+ re-export barrel (``index.ts``, ``__init__.py``) that imports the file: a
+ test importing ``foo`` from the barrel imports it from the file. What a test
+ names, not what it reaches.
+ """
+ targets = sorted({path for path in files if path})
+ if not targets or not test_files:
+ return {}
+ out: dict[str, dict[str, frozenset[str]]] = {}
+ barrels: dict[str, dict[str, frozenset[str]]] = {}
+ for source, target, names in await _imported_names(session, repo_id, targets):
+ if source in test_files:
+ _add_names(out, target, source, names)
+ elif PurePosixPath(source).name in BARREL_FILENAMES:
+ barrels.setdefault(source, {})[target] = names
+ if barrels:
+ await _add_barrel_imports(session, repo_id, barrels, test_files, out)
+ return out
+
+
+async def _add_barrel_imports(
+ session: AsyncSession,
+ repo_id: str,
+ barrels: dict[str, dict[str, frozenset[str]]],
+ test_files: set[str],
+ out: dict[str, dict[str, frozenset[str]]],
+) -> None:
+ """Credit each test importing a barrel with the names the barrel passes on from each file."""
+ for source, barrel, names in await _imported_names(session, repo_id, sorted(barrels)):
+ if source not in test_files:
+ continue
+ for target, exported in barrels[barrel].items():
+ # ``export *`` passes every name on; otherwise only the ones it re-exports.
+ passed = names if not exported or "*" in exported else names & exported
+ if passed or not names:
+ _add_names(out, target, source, passed)
+
+
+def _add_names(
+ out: dict[str, dict[str, frozenset[str]]], target: str, test: str, names: frozenset[str]
+) -> None:
+ by_test = out.setdefault(target, {})
+ by_test[test] = by_test.get(test, frozenset()) | names
+
+
+async def _imported_names(
+ session: AsyncSession, repo_id: str, targets: list[str]
+) -> list[tuple[str, str, frozenset[str]]]:
+ """``(importer, imported file, names)`` for every import edge into *targets*."""
+ shared: dict[str, Any] = {"repo_id": repo_id}
+ ets = _in_clause("e", sorted(FILE_DEPENDENCY_EDGE_TYPES), shared)
+ targets = list(dict.fromkeys(targets))
+ out = []
+ for start in range(0, len(targets), _IN_BATCH):
+ params = dict(shared)
+ tgt = _in_clause("p", targets[start : start + _IN_BATCH], params)
+ rows = await session.execute(
+ text(
+ "SELECT source_node_id, target_node_id, imported_names_json FROM graph_edges "
+ "WHERE repository_id = :repo_id "
+ f"AND target_node_id IN ({tgt}) AND edge_type IN ({ets})"
+ ),
+ params,
+ )
+ out.extend((source, target, _names_of(names_json)) for source, target, names_json in rows)
+ return out
+
+
+def _names_of(names_json: str | None) -> frozenset[str]:
+ """The names an import edge records; unreadable JSON records none."""
+ try:
+ names = json.loads(names_json or "[]")
+ except (TypeError, ValueError):
+ return frozenset()
+ return frozenset(n for n in names if isinstance(n, str))
+
+
+async def load_import_graph(session: AsyncSession, repo_id: str) -> ImportGraph:
+ """The edges the import walk follows (``_edges_into`` over every file), in one read."""
+ params: dict[str, Any] = {"repo_id": repo_id}
+ ets = _in_clause("e", sorted(FILE_DEPENDENCY_EDGE_TYPES), params)
+ rows = await session.execute(
+ text(
+ "SELECT DISTINCT source_node_id, target_node_id FROM graph_edges "
+ f"WHERE repository_id = :repo_id AND edge_type IN ({ets}){_LEGACY_CONFTEST_FILTER}"
+ ),
+ params,
+ )
+ importers: dict[str, list[str]] = {}
+ for source, target in rows:
+ importers.setdefault(target, []).append(source)
+ # Sorted, so a bounded walk does not depend on the order rows come back in.
+ return ImportGraph({t: tuple(sorted(s)) for t, s in importers.items()})
+
+
+async def direct_dependents(
+ session: AsyncSession, repo_id: str, files: list[str]
+) -> dict[str, set[str]]:
+ """Files with an import edge into each of *files*, or a call into a symbol it declares.
+
+ One hop, no tests excluded: the direct callers and importers a change to
+ them could have moved coverage through. A file with none is absent.
+ """
+ out: dict[str, set[str]] = {}
+ for dependent, dependency in await _edges_into(
+ session, repo_id, files, sorted(FILE_DEPENDENCY_EDGE_TYPES), frozenset()
+ ):
+ out.setdefault(dependency, set()).add(dependent)
+ declared = await _edges_from(session, repo_id, files, ["defines"])
+ owner = {symbol: path for path, symbol in declared}
+ for caller, callee in await _edges_into(
+ session, repo_id, sorted(owner), sorted(EXECUTION_EDGE_TYPES), UNRELIABLE_CALL_ORIGINS
+ ):
+ if (source := file_of_symbol(caller)) != owner[callee]:
+ out.setdefault(owner[callee], set()).add(source)
+ return out
+
+
+# See :data:`~.maps.LEGACY_CONFTEST_HINT`.
+_LEGACY_CONFTEST_FILTER = (
+ " AND NOT (edge_type = 'dynamic_uses' "
+ f"AND COALESCE(hint_source, '') = '{LEGACY_CONFTEST_HINT}')"
+)
+
+
+#: Edges a test file writes itself: imports and references, not the wiring a
+#: test runner adds (a conftest or setup file reaches every test under it).
+_AUTHORED_EDGE_TYPES = sorted(FILE_DEPENDENCY_EDGE_TYPES - {"framework"})
+
+
+async def direct_importers(session: AsyncSession, repo_id: str, files: list[str]) -> set[str]:
+ """Files whose own code imports or references one of *files*, one hop."""
+ rows = await _edges_into(session, repo_id, files, _AUTHORED_EDGE_TYPES, frozenset())
+ return {source for source, _ in rows}
+
+
+async def _known_files(session: AsyncSession, repo_id: str, paths: list[str]) -> set[str]:
+ """The subset of *paths* the index holds a node for."""
+ out: set[str] = set()
+ for start in range(0, len(paths), _IN_BATCH):
+ batch = paths[start : start + _IN_BATCH]
+ res = await session.execute(
+ select(GraphNode.node_id).where(
+ GraphNode.repository_id == repo_id,
+ GraphNode.node_type == "file",
+ GraphNode.node_id.in_(batch),
+ )
+ )
+ out.update(row[0] for row in res.all())
+ return out
+
+
+def _in_clause(prefix: str, values: list[str], params: dict[str, Any]) -> str:
+ """Bind *values* as ``:prefix0, :prefix1, ...`` and return the clause body."""
+ params.update({f"{prefix}{i}": v for i, v in enumerate(values)})
+ return ",".join(f":{prefix}{i}" for i in range(len(values)))
+
+
+#: Values one ``IN`` list binds. SQLite refuses a statement with more variables
+#: than SQLITE_MAX_VARIABLE_NUMBER (999 before 3.32, 32766 after), and the lists
+#: below are a change's files or every file and symbol of a repository.
+_IN_BATCH = 500
+
+
+async def _edges_from(
+ session: AsyncSession, repo_id: str, sources: list[str], edge_types: list[str]
+) -> list[tuple[str, str]]:
+ """``(source, target)`` pairs leaving *sources* along *edge_types*."""
+ shared: dict[str, Any] = {"repo_id": repo_id}
+ ets = _in_clause("e", edge_types, shared)
+ out: list[tuple[str, str]] = []
+ # Each source is in one batch only, so a pair is distinct across batches too.
+ sources = list(dict.fromkeys(sources))
+ for start in range(0, len(sources), _IN_BATCH):
+ params = dict(shared)
+ src = _in_clause("s", sources[start : start + _IN_BATCH], params)
+ rows = await session.execute(
+ text(
+ "SELECT DISTINCT source_node_id, target_node_id FROM graph_edges "
+ "WHERE repository_id = :repo_id "
+ f"AND source_node_id IN ({src}) AND edge_type IN ({ets})"
+ ),
+ params,
+ )
+ out.extend((s, t) for s, t in rows)
+ return out
+
+
+async def _edges_into(
+ session: AsyncSession,
+ repo_id: str,
+ targets: list[str],
+ edge_types: list[str],
+ excluded_origins: frozenset[str],
+) -> list[tuple[str, str]]:
+ """``(source, target)`` pairs arriving at *targets* along *edge_types*.
+
+ Raw text SQL with an ``IN`` list, matching ``pr_blast._transitive_affected``:
+ the edge table is the one place a per-level parameterised ``IN`` beats
+ loading every row, and the two walks should not disagree about how they read
+ it.
+ """
+ shared: dict[str, Any] = {"repo_id": repo_id}
+ ets = _in_clause("e", edge_types, shared)
+ origin_filter = ""
+ if excluded_origins:
+ # NULL means the row predates the vocabulary, not "unknown", so it has
+ # to survive the filter, and a bare NOT IN would drop it.
+ bad = _in_clause("o", sorted(excluded_origins), shared)
+ origin_filter = f" AND (resolution_origin IS NULL OR resolution_origin NOT IN ({bad}))"
+ out: list[tuple[str, str]] = []
+ # Each target is in one batch only, so a pair is distinct across batches too.
+ targets = list(dict.fromkeys(targets))
+ for start in range(0, len(targets), _IN_BATCH):
+ params = dict(shared)
+ tgt = _in_clause("p", targets[start : start + _IN_BATCH], params)
+ rows = await session.execute(
+ text(
+ "SELECT DISTINCT source_node_id, target_node_id FROM graph_edges "
+ "WHERE repository_id = :repo_id "
+ f"AND target_node_id IN ({tgt}) AND edge_type IN ({ets}){origin_filter}"
+ f"{_LEGACY_CONFTEST_FILTER}"
+ ),
+ params,
+ )
+ out.extend((s, t) for s, t in rows)
+ return out
diff --git a/packages/core/src/repowise/core/analysis/test_reach/forward.py b/packages/core/src/repowise/core/analysis/test_reach/forward.py
new file mode 100644
index 000000000..b034c20ab
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/forward.py
@@ -0,0 +1,145 @@
+"""The forward walk and file pairing: which files some test reaches or pairs with, at all.
+
+The batch question over a whole repository, not the attributed one the reverse
+walks answer.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Collection
+from pathlib import PurePosixPath
+from typing import Any, TypeAlias
+
+from repowise.core.analysis.dead_code.file_reachability import BARREL_FILENAMES
+from repowise.core.analysis.execution_graph import ExecutionGraphIndex, file_of_symbol
+from repowise.core.ingestion.models import FILE_DEPENDENCY_EDGE_TYPES
+from repowise.core.test_paths import paired_test_names
+
+from .model import DEFAULT_CALL_DEPTH
+
+CallGraphView: TypeAlias = ExecutionGraphIndex
+
+
+def call_graph_from_graph(graph: Any) -> CallGraphView:
+ """Build the call-reachability view from an in-memory NetworkX graph.
+
+ ``defines`` alone bridges files to symbols: measured across this repository
+ it reaches every symbol, class members included, so also walking
+ ``has_method`` would add a second containment layer for nothing.
+
+ Returns an empty view for ``None`` - the health engine runs without a graph
+ on some paths, and the documented outcome there is "no signal".
+ """
+ return ExecutionGraphIndex(graph)
+
+
+def files_reached_by_tests(
+ view: CallGraphView,
+ test_files: set[str],
+ *,
+ max_depth: int = DEFAULT_CALL_DEPTH,
+) -> set[str]:
+ """Every non-test file some test can execute into within *max_depth* hops.
+
+ One multi-source breadth-first search from every symbol the test files
+ declare, so the cost is O(V+E) however many test files there are - this is
+ the batch question ("which files are reached at all"), not the attributed
+ one.
+
+ Returns the reached set with the test files themselves removed: a test does
+ not need a test.
+ """
+ if not test_files or max_depth < 1:
+ return set()
+
+ frontier: set[str] = set()
+ for path in test_files:
+ frontier.update(view.declares.get(path, ()))
+ seen = set(frontier)
+ # The seeds are what the tests *declare*, not what they reach. Counting them
+ # would make containment alone a claim of reaching.
+ reached: set[str] = set()
+ for _ in range(max_depth):
+ nxt: set[str] = set()
+ for node in frontier:
+ nxt.update(view.calls.get(node, ()))
+ nxt -= seen
+ if not nxt:
+ break
+ seen |= nxt
+ reached |= nxt
+ frontier = nxt
+ return {file_of_symbol(symbol) for symbol in reached} - test_files
+
+
+def _dependencies_of(graph: Any, path: str) -> set[str]:
+ try:
+ return {
+ dst
+ for _src, dst, data in graph.out_edges(path, data=True)
+ if data.get("edge_type") in FILE_DEPENDENCY_EDGE_TYPES
+ }
+ except Exception:
+ return set()
+
+
+def files_with_paired_tests(
+ graph: Any, paths: Collection[str], test_files: Collection[str]
+) -> set[str]:
+ """Files among *paths* that have a test of their own, import graph first.
+
+ A test file that imports a file directly is its test, wherever either
+ lives: ``tests/unit/cli/test_init.py`` importing ``cli/main.py`` pairs
+ them although no name says so. A test *named* for a file
+ (:func:`paired_test_names`) still pairs, but only when the graph agrees
+ or cannot say: the named test imports the file directly or through a
+ re-export barrel, has no resolved dependency at all, or there is no graph.
+ Otherwise the name is a collision - ``distill/test_engine.py`` tests the
+ distill engine, not ``health/engine.py`` - and does not pair.
+
+ One hop, plus one more through a re-export barrel: a test importing a
+ package's ``index.ts`` or ``__init__.py`` pairs the files that barrel
+ imports, since that is how a package's tests reach its modules. A
+ file-level "some test imports this", never a quantity. The reach walk
+ above stays call-graph-only; pairing is the looser question "does this
+ file have a test", and against this repository's stored line coverage
+ direct-import pairing finds 71.6% of covered files at 98.9% precision
+ where names alone found 34.6% at 100%.
+ """
+ barrel_deps: dict[str, set[str]] = {}
+
+ def imports_of(test: str) -> set[str]:
+ direct = _dependencies_of(graph, test)
+ through: set[str] = set()
+ for mid in direct:
+ if PurePosixPath(mid).name in BARREL_FILENAMES:
+ if mid not in barrel_deps:
+ barrel_deps[mid] = _dependencies_of(graph, mid)
+ through |= barrel_deps[mid]
+ return direct | through
+
+ paired: set[str] = set()
+ deps: dict[str, set[str]] = {}
+ if graph is not None:
+ for test in test_files:
+ deps[test] = imports_of(test)
+ paired.update(deps[test])
+ paired &= set(paths)
+
+ by_name: dict[str, list[str]] = {}
+ for path in {*paths, *test_files}:
+ by_name.setdefault(path.rsplit("/", 1)[-1], []).append(path)
+
+ def named_pair_holds(test: str, path: str) -> bool:
+ if graph is None:
+ return True
+ imported = deps[test] if test in deps else imports_of(test)
+ return not imported or path in imported
+
+ for path in paths:
+ if path in paired:
+ continue
+ candidates = (c for name in paired_test_names(path) for c in by_name.get(name, ()))
+ if any(c != path and named_pair_holds(c, path) for c in candidates):
+ paired.add(path)
+ return paired
diff --git a/packages/core/src/repowise/core/analysis/test_reach/inline.py b/packages/core/src/repowise/core/analysis/test_reach/inline.py
new file mode 100644
index 000000000..f2eff11bd
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/inline.py
@@ -0,0 +1,48 @@
+"""Files that carry their own tests, so no separate test file has to reach them.
+
+Rust compiles a ``#[cfg(test)]`` module inside the file it tests, and Go runs a
+package's ``_test.go`` files in that package's directory. Neither needs an
+import edge, so a graph walk from test files can miss both.
+"""
+
+from __future__ import annotations
+
+import re
+from collections.abc import Collection
+from pathlib import PurePosixPath
+
+# ``#[cfg(test)]`` on its own attribute line. Ceiling: a line comment is
+# skipped but a block comment is not; the parser's attribute nodes are the
+# upgrade path if commented-out test modules show up.
+_RUST_CFG_TEST_RE = re.compile(rb"^[ \t]*#\s*\[\s*cfg\s*\(\s*test\s*\)\s*\]", re.MULTILINE)
+
+
+def rust_has_inline_tests(content: str | bytes) -> bool:
+ """Whether Rust source holds a ``#[cfg(test)]`` item: tests compiled into the file."""
+ data = content.encode("utf-8", "replace") if isinstance(content, str) else content
+ return bool(_RUST_CFG_TEST_RE.search(data))
+
+
+def go_inline_tests(path: str, test_files: Collection[str]) -> list[str]:
+ """The ``_test.go`` files in a Go file's directory, which are its package's tests.
+
+ The directory is the package, so every one of them runs with it. That
+ includes an external ``package x_test`` file, which tests the package
+ through its exports.
+ """
+ p = PurePosixPath(path)
+ if p.suffix != ".go" or p.name.endswith("_test.go"):
+ return []
+ folder = str(p.parent)
+ return sorted(
+ t
+ for t in test_files
+ if t.endswith("_test.go") and str(PurePosixPath(t).parent) == folder
+ )
+
+
+def inline_tests(path: str, content: str | bytes | None, test_files: Collection[str]) -> list[str]:
+ """What runs *path*'s own tests: itself (Rust), its package's test files (Go), or nothing."""
+ if path.endswith(".rs") and content is not None and rust_has_inline_tests(content):
+ return [path]
+ return go_inline_tests(path, test_files)
diff --git a/packages/core/src/repowise/core/analysis/test_reach/maps.py b/packages/core/src/repowise/core/analysis/test_reach/maps.py
new file mode 100644
index 000000000..a06b16b5d
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/maps.py
@@ -0,0 +1,166 @@
+"""The edges a reach walk reads, held in memory, and the loop that feeds them to it.
+
+A walk (:mod:`.walk`) never reads edges itself. It yields a :class:`Need` - "the
+reliable callers of these symbols", "the importers of these files" - and is sent
+the rows back. :func:`run` answers from a :class:`WalkMaps` built from a graph
+held in memory; the database adapter answers the same needs with one query per
+level. Both drive the one walk, so the two cannot disagree about a row.
+"""
+
+from __future__ import annotations
+
+from collections.abc import Collection, Generator, Mapping
+from dataclasses import dataclass
+from typing import Any, Literal, NamedTuple, TypeVar
+
+from repowise.core.ingestion.models import EXECUTION_EDGE_TYPES, FILE_DEPENDENCY_EDGE_TYPES
+
+from .model import UNRELIABLE_CALL_ORIGINS
+
+NeedKind = Literal["declared", "callers", "importers", "known"]
+
+
+class Need(NamedTuple):
+ """What a walk asks for next.
+
+ ``declared``: ``(file, symbol)`` for each symbol the files define.
+ ``callers``: ``(caller, callee)`` along reliable execution edges into the nodes.
+ ``importers``: ``(dependent, dependency)`` along file-dependency edges into the files.
+ ``known``: the subset of the paths the index holds a node for.
+ """
+
+ kind: NeedKind
+ keys: tuple[str, ...]
+
+
+_T = TypeVar("_T")
+Walk = Generator[Need, Any, _T]
+
+# The conftest -> test hint older indexes stored made a conftest read as each
+# test's importer; the test -> conftest edge already records the relation.
+# Upgrade path: drop this once such indexes are rebuilt.
+LEGACY_CONFTEST_HINT = "pytest_conftest"
+
+
+def is_legacy_conftest_hint(edge_type: str | None, hint_source: str | None) -> bool:
+ return edge_type == "dynamic_uses" and hint_source == LEGACY_CONFTEST_HINT
+
+
+@dataclass(frozen=True)
+class ImportGraph:
+ """Every file-dependency edge, reversed (``{file: files depending on it}``), read once.
+
+ The import walk is unbounded on the selection path, so on a hub it runs
+ dozens of levels; one bulk read replaces a query per level. Either source
+ gives the same answer: the walk reaches a fixpoint, so edge order does not
+ matter. Ceiling: read per selection, not kept across calls; caching it per
+ index stamp is the upgrade path if the read shows up.
+ """
+
+ importers: Mapping[str, tuple[str, ...]]
+
+ def edges_into(self, targets: Collection[str]) -> list[tuple[str, str]]:
+ """``(dependent, dependency)`` pairs, as a ``importers`` :class:`Need` is answered."""
+ return [(src, t) for t in targets for src in self.importers.get(t, ())]
+
+ def hops(self, sources: Collection[str]) -> dict[str, int]:
+ """``{file: fewest dependency hops from any of *sources*}``, sources at 0.
+
+ One multi-source breadth-first pass through every file, tests
+ included, so a test importing a reached helper is one hop past it.
+ """
+ best = dict.fromkeys(sources, 0)
+ frontier = list(best)
+ depth = 0
+ while frontier:
+ depth += 1
+ fresh = [
+ d for node in frontier for d in self.importers.get(node, ()) if d not in best
+ ]
+ fresh = list(dict.fromkeys(fresh))
+ best.update(dict.fromkeys(fresh, depth))
+ frontier = fresh
+ return best
+
+
+@dataclass(frozen=True)
+class WalkMaps:
+ """The adjacency a reach walk reads, from :func:`build_walk_maps`.
+
+ ``declares`` maps a file to the symbols it defines, ``callers`` a node to the
+ nodes with a reliable execution edge into it, ``imports`` a file to the
+ files depending on it. ``files`` are the file nodes and ``test_files`` the
+ ones ingestion stored ``is_test`` on.
+ """
+
+ declares: Mapping[str, tuple[str, ...]]
+ callers: Mapping[str, tuple[str, ...]]
+ imports: ImportGraph
+ files: frozenset[str]
+ test_files: frozenset[str]
+
+ def answer(self, need: Need) -> Any:
+ if need.kind == "declared":
+ return [(f, s) for f in need.keys for s in self.declares.get(f, ())]
+ if need.kind == "callers":
+ return [(c, n) for n in need.keys for c in self.callers.get(n, ())]
+ if need.kind == "importers":
+ return self.imports.edges_into(need.keys)
+ return {path for path in need.keys if path in self.files}
+
+
+def run(walk: Walk[_T], maps: WalkMaps) -> _T:
+ """Drive *walk* to its answer from *maps*."""
+ try:
+ need = next(walk)
+ while True:
+ need = walk.send(maps.answer(need))
+ except StopIteration as done:
+ return done.value
+
+
+def build_walk_maps(graph: Any) -> WalkMaps:
+ """:class:`WalkMaps` from a NetworkX graph, or the node-link dict it serialises to.
+
+ Nodes carry ``node_type`` and ``is_test``; edges carry ``edge_type`` and,
+ for calls, ``resolution_origin``. Edges are kept the way the stored graph's
+ reader keeps them: calls resolved by a name guess
+ (:data:`UNRELIABLE_CALL_ORIGINS`) are dropped. ``None`` gives empty maps.
+ """
+ if isinstance(graph, Mapping):
+ import networkx as nx
+
+ key = "edges" if "edges" in graph else "links"
+ graph = nx.node_link_graph(graph, edges=key)
+ files: set[str] = set()
+ tests: set[str] = set()
+ declares: dict[str, set[str]] = {}
+ callers: dict[str, set[str]] = {}
+ importers: dict[str, set[str]] = {}
+ if graph is not None:
+ for node, attrs in graph.nodes(data=True):
+ if attrs.get("node_type") == "file":
+ files.add(node)
+ if attrs.get("is_test"):
+ tests.add(node)
+ for source, target, attrs in graph.edges(data=True):
+ edge_type = attrs.get("edge_type")
+ if edge_type == "defines":
+ declares.setdefault(source, set()).add(target)
+ if (
+ edge_type in EXECUTION_EDGE_TYPES
+ and attrs.get("resolution_origin") not in UNRELIABLE_CALL_ORIGINS
+ ):
+ callers.setdefault(target, set()).add(source)
+ if edge_type in FILE_DEPENDENCY_EDGE_TYPES and not is_legacy_conftest_hint(
+ edge_type, attrs.get("hint_source")
+ ):
+ importers.setdefault(target, set()).add(source)
+ # Sorted, so a bounded walk does not depend on the order edges were added in.
+ return WalkMaps(
+ declares={k: tuple(sorted(v)) for k, v in declares.items()},
+ callers={k: tuple(sorted(v)) for k, v in callers.items()},
+ imports=ImportGraph({k: tuple(sorted(v)) for k, v in importers.items()}),
+ files=frozenset(files),
+ test_files=frozenset(tests),
+ )
diff --git a/packages/core/src/repowise/core/analysis/test_reach/model.py b/packages/core/src/repowise/core/analysis/test_reach/model.py
new file mode 100644
index 000000000..ea3c90615
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/model.py
@@ -0,0 +1,158 @@
+"""The rows a reach walk answers with, the constants it walks by, and its test order."""
+
+from __future__ import annotations
+
+import sys
+from collections.abc import Collection, Iterable, Mapping
+from dataclasses import dataclass
+from typing import Literal, NamedTuple
+
+from repowise.core.analysis.execution_graph import UNRELIABLE_EXECUTION_ORIGINS
+from repowise.core.test_paths import paired_test_names
+
+# How many call hops a test may take and still be said to reach a file. The walk
+# saturates here; see the Depth section of the package docstring.
+DEFAULT_CALL_DEPTH = 3
+
+# How many import hops the weaker fallback tier may take. Still the measured
+# choice; a blanket second hop costs more than it buys.
+DEFAULT_MAX_DEPTH = 1
+
+# Call edges whose resolver only matched a name. See the Confidence section.
+UNRELIABLE_CALL_ORIGINS = UNRELIABLE_EXECUTION_ORIGINS
+
+# Cap on how many test files one target reports. The consumers all cut their
+# own lists shorter; the cap exists so a helper called from every test in the
+# suite cannot produce an unbounded intermediate.
+MAX_TESTS_PER_TARGET = 50
+
+# Which tier answered. The CLI prints this so a reader can tell "this test runs
+# into the file" from the weaker "this test imports it".
+# ``name-match`` is weaker still: a test named for the file, with no edge.
+# ``changed-test`` is a test the change adds or edits that names or imports the
+# file; ``inline`` is a file carrying its own tests (Rust ``#[cfg(test)]``, a Go
+# ``_test.go`` beside it).
+ReachedVia = Literal["call-graph", "import-graph", "name-match", "changed-test", "inline"]
+
+# What can be said about one file's tests:
+# ``reached`` some test reaches it (the row says which, and how);
+# ``unreached`` the call walk, the helper or barrel hop and the whole import
+# closure all miss it, so "no test reaches this" is a claim;
+# ``new`` the index has never seen the file;
+# ``no_signal`` the index holds no tests, so silence proves nothing;
+# ``type_only`` it declares only types: there is nothing for a test to run.
+ReachState = Literal["reached", "unreached", "new", "no_signal", "type_only"]
+
+
+class ReachDistance(NamedTuple):
+ """How a test reached a seed: the fewest call hops, and how many of the
+ test's symbols made a call at that depth."""
+
+ hops: int
+ callers: int
+
+
+@dataclass(frozen=True)
+class ReachedBy:
+ """Tests reaching one target, and which tier found them.
+
+ ``total`` is how many the walk actually found, before
+ ``MAX_TESTS_PER_TARGET`` trimmed ``tests``. A surface that prints
+ ``len(tests)`` as the answer is stating a cap as a measurement. The cut keeps
+ the closest tests (:func:`rank_reaching`). Carry the total and say the bound.
+ """
+
+ tests: list[str]
+ via: ReachedVia
+ total: int = 0
+ # Internal uncapped identities let downstream aggregators de-duplicate one
+ # test that reaches several targets while public lists remain bounded.
+ all_tests: tuple[str, ...] | None = None
+ # How close each test came, for the call tier only: the import and name
+ # tiers have no hop count to report.
+ reach: Mapping[str, ReachDistance] | None = None
+ # Import tier only: the files each test was reached through, i.e. the
+ # dependencies of that test on the route to the target.
+ entries: Mapping[str, frozenset[str]] | None = None
+
+
+@dataclass(frozen=True)
+class ReachRow:
+ """One file's :data:`ReachState`, with the evidence when it is ``reached``."""
+
+ state: ReachState
+ reached: ReachedBy | None = None
+
+
+def capped(
+ target: str,
+ tests: Iterable[str],
+ via: ReachedVia,
+ *,
+ reach: Mapping[str, ReachDistance] | None = None,
+ direct: Collection[str] = frozenset(),
+ changed: Collection[str] = frozenset(),
+ entries: Mapping[str, frozenset[str]] | None = None,
+) -> ReachedBy:
+ """*tests* ranked by :func:`rank_reaching`, then cut to ``MAX_TESTS_PER_TARGET``."""
+ ordered = tuple(rank_reaching(target, tests, reach=reach, direct=direct, changed=changed))
+ return ReachedBy(
+ list(ordered[:MAX_TESTS_PER_TARGET]), via, len(ordered), ordered, reach, entries
+ )
+
+
+def rank_reaching(
+ target: str,
+ tests: Iterable[str],
+ *,
+ reach: Mapping[str, ReachDistance] | None = None,
+ direct: Collection[str] = frozenset(),
+ changed: Collection[str] = frozenset(),
+) -> list[str]:
+ """*tests* reaching *target*, the likeliest to exercise it first. Cap after this.
+
+ In turn: a test in *changed* (the change edits it), fewer call hops
+ (*reach*; a test without one sorts after every test with one), a test in
+ *direct* (it imports or calls *target* itself), more of the test's
+ functions at that distance, then by name: the test named for *target*,
+ the one mirroring more of its directories, and the path. Targets, tests
+ and *changed* may be ``path::name`` ids; every key reads the file part.
+ """
+ path_of = target.split("::", 1)[0]
+ names = paired_test_names(path_of)
+ dirs = set(path_of.split("/")[:-1])
+ reach = reach or {}
+ edited = {c.split("::", 1)[0] for c in changed}
+
+ def key(test: str) -> tuple[bool, int, bool, int, bool, int, str]:
+ path = test.split("::", 1)[0]
+ distance = reach.get(test) or reach.get(path)
+ return (
+ path not in edited,
+ distance.hops if distance else sys.maxsize,
+ test not in direct and path not in direct,
+ -distance.callers if distance else 0,
+ path.rsplit("/", 1)[-1] not in names,
+ -len(dirs & set(path.split("/")[:-1])),
+ test,
+ )
+
+ return sorted(tests, key=key)
+
+
+def tests_matching_by_name(targets: list[str], test_files: Collection[str]) -> dict[str, ReachedBy]:
+ """Tests named for each target by convention: the fallback when no edge answered.
+
+ An unresolved graph (dynamic client, unindexed framework) leaves
+ ``tests/test_x.py`` with no edge to ``x.py``; "unknown" there argues
+ against a change a test guards.
+ """
+ by_name: dict[str, list[str]] = {}
+ for path in test_files:
+ by_name.setdefault(path.rsplit("/", 1)[-1], []).append(path)
+ out: dict[str, ReachedBy] = {}
+ for target in targets:
+ found = [path for name in paired_test_names(target) for path in by_name.get(name, ())]
+ if found:
+ out[target] = capped(target, found, "name-match")
+ return out
diff --git a/packages/core/src/repowise/core/analysis/test_reach/type_only.py b/packages/core/src/repowise/core/analysis/test_reach/type_only.py
new file mode 100644
index 000000000..9527af97d
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/type_only.py
@@ -0,0 +1,149 @@
+"""Files that declare only types, so there is nothing for a test to run.
+
+A ``.d.ts``, a TypeScript module whose every statement is a type, an interface
+or a ``declare``, and a Python module of ``Protocol`` and ``TypedDict`` classes
+and type aliases all vanish at run time. "No test reaches this" is true of them
+and says nothing.
+"""
+
+from __future__ import annotations
+
+import ast
+from pathlib import PurePosixPath
+
+from repowise.core.test_paths import is_test_related_path
+
+_DECLARATION_SUFFIXES = (".d.ts", ".d.mts", ".d.cts")
+_TS_SUFFIXES = frozenset({".ts", ".mts", ".cts", ".tsx"})
+
+# Statements TypeScript erases. A plain import is erased too when every name
+# it binds is used as a type, which in a file of only types it must be; a bare
+# ``import "./x"`` runs the module, so it is not on the list.
+_TS_TYPE_NODES = frozenset(
+ {"type_alias_declaration", "interface_declaration", "ambient_declaration"}
+)
+_TS_NEUTRAL_NODES = frozenset({"comment", "empty_statement"})
+
+_PY_TYPE_BASES = frozenset({"Protocol", "TypedDict"})
+_PY_TYPE_FACTORIES = frozenset({"TypeVar", "ParamSpec", "TypeVarTuple", "NewType"})
+
+
+def is_type_only(path: str, content: str | bytes, *, is_test: bool | None = None) -> bool:
+ """Whether the file at *path* declares only types.
+
+ *is_test* is the stored flag when the caller has it; a test file is never
+ type-only, by the same classifier that stamps the flag. A file that cannot
+ be parsed, or declares nothing, is not type-only.
+ """
+ if is_test if is_test is not None else is_test_related_path(path):
+ return False
+ lowered = path.lower()
+ if lowered.endswith(_DECLARATION_SUFFIXES):
+ return True
+ suffix = PurePosixPath(lowered).suffix
+ data = content.encode("utf-8", "replace") if isinstance(content, str) else content
+ if suffix in _TS_SUFFIXES:
+ return _ts_type_only(data, tsx=suffix == ".tsx")
+ if suffix in (".py", ".pyi"):
+ return _python_type_only(data)
+ return False
+
+
+def _ts_type_only(source: bytes, *, tsx: bool) -> bool:
+ from tree_sitter import Parser
+
+ from repowise.core.ingestion.parser import _get_language
+
+ language = _get_language("tsx" if tsx else "typescript")
+ if language is None:
+ return False
+ declared = False
+ for node in Parser(language).parse(source).root_node.children:
+ if node.type in _TS_NEUTRAL_NODES:
+ continue
+ if node.type == "import_statement":
+ if not any(c.type == "import_clause" for c in node.children):
+ return False
+ continue
+ if node.type in _TS_TYPE_NODES:
+ declared = True
+ continue
+ if node.type == "export_statement" and _ts_type_export(node):
+ declared = True
+ continue
+ return False
+ return declared
+
+
+def _ts_type_export(node) -> bool:
+ """``export`` of a type, an interface or a ``declare``; ``export type {..}`` / ``type *``."""
+ for child in node.children:
+ if child.type in _TS_TYPE_NODES:
+ return True
+ # The grammar reads ``export type * from`` as an error around ``type``.
+ if child.type == "type" or (child.type == "ERROR" and child.text == b"type"):
+ return True
+ return False
+
+
+def _python_type_only(source: bytes) -> bool:
+ try:
+ tree = ast.parse(source)
+ except (SyntaxError, ValueError):
+ return False
+ declared = False
+ for stmt in tree.body:
+ kind = _python_statement(stmt)
+ if kind is None:
+ return False
+ declared = declared or kind
+ return declared
+
+
+def _python_statement(stmt: ast.stmt) -> bool | None:
+ """True for a type declaration, False for a neutral statement, None for run-time code."""
+ if isinstance(stmt, ast.Import | ast.ImportFrom | ast.Pass):
+ return False
+ if isinstance(stmt, ast.Expr) and isinstance(stmt.value, ast.Constant):
+ return False # a docstring
+ if isinstance(stmt, ast.If) and _name_of(stmt.test) == "TYPE_CHECKING":
+ return False if all(_python_statement(s) is not None for s in stmt.body) else None
+ if isinstance(stmt, ast.ClassDef):
+ return True if any(_name_of(_base(b)) in _PY_TYPE_BASES for b in stmt.bases) else None
+ if type(stmt).__name__ == "TypeAlias": # ``type X = ...`` (3.12)
+ return True
+ if isinstance(stmt, ast.AnnAssign):
+ return True if _name_of(stmt.annotation) == "TypeAlias" else None
+ if isinstance(stmt, ast.Assign):
+ if [_name_of(t) for t in stmt.targets] == ["__all__"]:
+ return False
+ return True if _is_type_expression(stmt.value) else None
+ return None
+
+
+def _is_type_expression(value: ast.expr) -> bool:
+ """A subscripted type, a ``|`` union of them, or a ``TypeVar``-style factory call."""
+ if isinstance(value, ast.Subscript):
+ return True
+ if isinstance(value, ast.BinOp) and isinstance(value.op, ast.BitOr):
+ return _is_type_operand(value.left) and _is_type_operand(value.right)
+ return isinstance(value, ast.Call) and _name_of(value.func) in _PY_TYPE_FACTORIES
+
+
+def _is_type_operand(value: ast.expr) -> bool:
+ return isinstance(value, ast.Name | ast.Attribute) or _is_type_expression(value) or (
+ isinstance(value, ast.Constant) and value.value is None
+ )
+
+
+def _base(node: ast.expr) -> ast.expr:
+ """``Protocol`` out of ``Protocol[T]``."""
+ return node.value if isinstance(node, ast.Subscript) else node
+
+
+def _name_of(node: ast.expr) -> str | None:
+ if isinstance(node, ast.Name):
+ return node.id
+ if isinstance(node, ast.Attribute):
+ return node.attr
+ return None
diff --git a/packages/core/src/repowise/core/analysis/test_reach/walk.py b/packages/core/src/repowise/core/analysis/test_reach/walk.py
new file mode 100644
index 000000000..d713869f2
--- /dev/null
+++ b/packages/core/src/repowise/core/analysis/test_reach/walk.py
@@ -0,0 +1,420 @@
+"""The reverse reach walks, one implementation for every graph source.
+
+Each walk is a generator: it yields a :class:`~.maps.Need` per level and is sent
+the rows back. :func:`reaching_by_tier` and :func:`reach_states` drive them from
+:class:`~.maps.WalkMaps`; the database adapter drives the same generators from
+its queries.
+"""
+
+from __future__ import annotations
+
+import sys
+from collections.abc import Collection, Mapping
+from pathlib import PurePosixPath
+
+from repowise.core.analysis.dead_code.file_reachability import BARREL_FILENAMES
+from repowise.core.analysis.execution_graph import file_of_symbol
+from repowise.core.analysis.test_selection import is_runnable_test
+from repowise.core.test_paths import names_test_for
+
+from .inline import inline_tests
+from .maps import Need, Walk, WalkMaps, run
+from .model import (
+ DEFAULT_CALL_DEPTH,
+ DEFAULT_MAX_DEPTH,
+ ReachDistance,
+ ReachedBy,
+ ReachRow,
+ capped,
+)
+from .type_only import is_type_only
+
+# The whole reverse-import closure: no depth limit, the walk ends when no node
+# gains a new seed, so a cut-off can never drop a test.
+CLOSURE_DEPTH = sys.maxsize
+
+# Seeds one closure walk carries. Every node keeps the set of seeds it was
+# reached from, so a walk's memory grows with seeds times files; more targets
+# than this are walked in batches of it. Ceiling: the batches repeat the walk
+# over shared ancestors; a seed bitset per node is the upgrade path.
+CLOSURE_SEED_CAP = 200
+
+
+def call_walk(
+ seeds: list[str],
+ test_files: Collection[str],
+ max_depth: int,
+ *,
+ symbol_seeds: Mapping[str, Collection[str]] | None = None,
+ avoid: Collection[str] = frozenset(),
+) -> Walk[dict[str, dict[str, ReachDistance]]]:
+ """Tests that can execute into each seed file, walking call edges backwards.
+
+ Starts one layer below the other walk: the seeds are files and call edges
+ join symbols, so the first need resolves each seed to the symbols it
+ declares, and the walk carries the seed from there.
+
+ Each test carries the fewest call hops it took and how many of its symbols
+ made a call at that depth. Every node records each seed's distance, not
+ the level it was found at: one level's batch can carry a seed through a
+ node another seed entered, so the level alone overstates how close a test
+ is. A shorter distance found later re-queues the node; which tests reach
+ which seed is unchanged by that, only how far.
+
+ *symbol_seeds* enters the named targets at exactly those symbol ids. *avoid*
+ names files reach does not pass through, except into a target's own
+ symbols. Each level reads the distances it started with, so a seed moves
+ one hop per level: no test is reached past *max_depth*, and the answer
+ depends neither on which other seeds share the walk nor on edge order.
+ """
+ # node -> seed -> call hops from the node to the seed.
+ origins = yield from _call_origins(seeds, symbol_seeds)
+ if not origins:
+ return {}
+
+ found: dict[str, dict[str, tuple[int, set[str]]]] = {}
+ frontier = list(origins)
+ for _ in range(max_depth):
+ if not frontier:
+ break
+ level, frontier = frontier, []
+ queued: set[str] = set()
+ start = {node: dict(origins[node]) for node in level}
+ for caller, callee in (yield Need("callers", tuple(level))):
+ carried = start.get(callee)
+ if not carried:
+ continue
+ owner = file_of_symbol(caller)
+ if owner in test_files:
+ _record_test_hit(found, owner, caller, carried)
+ # A test is a leaf. Walking through one would let "test A calls
+ # shared helper B" drag B's unrelated targets in.
+ continue
+ if owner in avoid:
+ # A hub carries on only the seeds it declares itself.
+ carried = {s: d for s, d in carried.items() if s.split("::", 1)[0] == owner}
+ if not _carry_closer(origins, caller, carried):
+ continue
+ if caller not in queued:
+ queued.add(caller)
+ frontier.append(caller)
+ return {
+ seed: {test: ReachDistance(depth, len(callers)) for test, (depth, callers) in tests.items()}
+ for seed, tests in found.items()
+ }
+
+
+def _call_origins(
+ seeds: list[str], symbol_seeds: Mapping[str, Collection[str]] | None
+) -> Walk[dict[str, dict[str, int]]]:
+ """The symbols each seed enters the call walk at, each at distance 0 from its seed."""
+ origins: dict[str, dict[str, int]] = {}
+ # A caller that knows which symbol in the file it cares about enters there,
+ # so a test reaching an unrelated symbol in the same file does not count.
+ # An empty entry names no symbol, so that file is unseeded and keeps the
+ # ``defines`` lookup; treating it as seeded would silence the file entirely.
+ seeded = {
+ seed: symbol_seeds[seed] for seed in seeds if symbol_seeds and symbol_seeds.get(seed)
+ }
+ for seed, symbol_ids in seeded.items():
+ for symbol in symbol_ids:
+ origins.setdefault(symbol, {})[seed] = 0
+ unseeded = tuple(seed for seed in seeds if seed not in seeded)
+ if unseeded:
+ for seed, symbol in (yield Need("declared", unseeded)):
+ origins.setdefault(symbol, {})[seed] = 0
+ return origins
+
+
+def _record_test_hit(
+ found: dict[str, dict[str, tuple[int, set[str]]]],
+ test: str,
+ caller: str,
+ carried: Mapping[str, int],
+) -> None:
+ """Keep the fewest hops *test* took to each carried seed, and its callers at that depth."""
+ for seed, distance in carried.items():
+ hops = distance + 1
+ by_test = found.setdefault(seed, {})
+ best = by_test.get(test)
+ if best is None or hops < best[0]:
+ by_test[test] = (hops, {caller})
+ elif hops == best[0]:
+ best[1].add(caller)
+
+
+def _carry_closer(
+ origins: dict[str, dict[str, int]], caller: str, carried: Mapping[str, int]
+) -> bool:
+ """Record the seeds *caller* reaches closer than before; True when any did."""
+ if not carried:
+ return False
+ known = origins.setdefault(caller, {})
+ closer = {
+ seed: distance + 1
+ for seed, distance in carried.items()
+ if seed not in known or distance + 1 < known[seed]
+ }
+ known.update(closer)
+ return bool(closer)
+
+
+def import_walk(
+ seeds: list[str], test_files: Collection[str], max_depth: int
+) -> Walk[tuple[dict[str, set[str]], dict[str, dict[str, set[str]]]]]:
+ """Tests that import each seed file, directly or within *max_depth* hops.
+
+ Also ``{seed: {test: the files it imports on a route to the seed}}``.
+ """
+ origins: dict[str, set[str]] = {s: {s} for s in seeds}
+ # (test, file it imports) -> seeds carried in: one set union per edge, not
+ # a dict write per seed; split per seed once the walk ends.
+ reached: dict[tuple[str, str], set[str]] = {}
+ frontier = list(seeds)
+
+ for _ in range(max_depth):
+ if not frontier:
+ break
+ level, frontier = frontier, []
+ queued: set[str] = set()
+ # Read each level against the origins it started with, so a seed that
+ # imports another seed moves one hop per level, whatever the edge order.
+ start = {f: frozenset(origins[f]) for f in level}
+ for dependent, dependency in (yield Need("importers", tuple(level))):
+ carried = start.get(dependency)
+ if not carried:
+ continue
+ if dependent in test_files:
+ reached.setdefault((dependent, dependency), set()).update(carried)
+ continue
+ # Another seed is an ordinary node here: a route through it must
+ # reach the same tests a one-seed walk would.
+ known = origins.setdefault(dependent, set())
+ fresh = carried - known
+ if not fresh:
+ # Already reached carrying nothing new. Without this, a diamond
+ # in the import graph re-walks the whole subtree once per path
+ # into it.
+ continue
+ known |= fresh
+ if dependent not in queued:
+ queued.add(dependent)
+ frontier.append(dependent)
+ found: dict[str, set[str]] = {}
+ entries: dict[str, dict[str, set[str]]] = {}
+ for (test, dependency), carried in reached.items():
+ for seed in carried:
+ found.setdefault(seed, set()).add(test)
+ entries.setdefault(seed, {}).setdefault(test, set()).add(dependency)
+ return found, entries
+
+
+def helper_walk(targets: list[str], test_files: Collection[str]) -> Walk[dict[str, ReachedBy]]:
+ """Tests that import a file only tests import, or a barrel, which imports the target.
+
+ A second import hop, kept off hubs: a blanket second hop was measured and
+ rejected (see Depth), but a module nothing outside the test suite imports
+ is test support whatever its name (``attempt.spawn-workspace.test-support.ts``
+ loading ``attempt.ts`` for 30-odd tests), so walking through it claims
+ nothing a hub would. A barrel (``index.ts``, ``__init__.py``) passes on
+ what it imports even when source imports it too: a package's tests import
+ its entry, not each module behind it.
+ """
+ middles: dict[str, set[str]] = {}
+ for dependent, target in (yield Need("importers", tuple(targets))):
+ if dependent not in test_files:
+ middles.setdefault(dependent, set()).add(target)
+ importers: dict[str, set[str]] = {}
+ for dependent, middle in (yield Need("importers", tuple(sorted(middles)))):
+ importers.setdefault(middle, set()).add(dependent)
+ reached: dict[str, set[str]] = {}
+ for middle, who in importers.items():
+ tests = who & set(test_files)
+ if tests and (tests == who or PurePosixPath(middle).name in BARREL_FILENAMES):
+ for target in middles[middle]:
+ reached.setdefault(target, set()).update(tests)
+ return {
+ target: capped(target, tests, "import-graph", changed=targets)
+ for target, tests in reached.items()
+ }
+
+
+def tier_walk(
+ targets: list[str],
+ test_files: Collection[str],
+ *,
+ call_depth: int = DEFAULT_CALL_DEPTH,
+ import_depth: int = DEFAULT_MAX_DEPTH,
+ symbol_seeds: Mapping[str, Collection[str]] | None = None,
+ avoid: Collection[str] = frozenset(),
+ helpers: bool = False,
+) -> Walk[dict[str, ReachedBy]]:
+ """The call walk, then the import walk for what it left, then (*helpers*) the helper hop.
+
+ Each tier is seeded with only the targets the stronger ones left
+ unanswered, so a weaker tier never speaks over a stronger one. The targets
+ are usually a change's files, so a test among them leads every list.
+ """
+ seeds = sorted({t for t in targets if t})
+ if not seeds or not test_files:
+ return {}
+ out: dict[str, ReachedBy] = {}
+ if call_depth >= 1:
+ found = yield from call_walk(
+ seeds, test_files, call_depth, symbol_seeds=symbol_seeds, avoid=avoid
+ )
+ for seed, reach in found.items():
+ out[seed] = capped(seed, reach, "call-graph", reach=reach, changed=seeds)
+ unanswered = [seed for seed in seeds if seed not in out]
+ if unanswered and import_depth >= 1:
+ found, entries = yield from import_walk(unanswered, test_files, import_depth)
+ for seed, tests in found.items():
+ file = seed.split("::", 1)[0]
+ by_test = {t: frozenset(via) for t, via in entries[seed].items()}
+ # A test importing the target itself, not a file on the way to it.
+ direct = {t for t, via in by_test.items() if file in via}
+ out[seed] = capped(
+ seed, tests, "import-graph", direct=direct, changed=seeds, entries=by_test
+ )
+ unanswered = [seed for seed in seeds if seed not in out]
+ if unanswered and helpers:
+ out |= yield from helper_walk(unanswered, test_files)
+ return out
+
+
+def reaching_by_tier(
+ maps: WalkMaps,
+ targets: Collection[str],
+ *,
+ call_depth: int = DEFAULT_CALL_DEPTH,
+ import_depth: int = DEFAULT_MAX_DEPTH,
+ symbol_seeds: Mapping[str, Collection[str]] | None = None,
+ avoid: Collection[str] = frozenset(),
+ helpers: bool = True,
+) -> dict[str, ReachedBy]:
+ """Test files reaching each of *targets*, and which tier found them: :func:`tier_walk`.
+
+ Targets nothing reaches are absent; that absence is a weak signal, not a
+ claim (:func:`reach_states` makes the claim). A target may be a symbol id
+ (``path::name``); its tests are ranked against the file part.
+ """
+ return run(
+ tier_walk(
+ list(targets),
+ maps.test_files,
+ call_depth=call_depth,
+ import_depth=import_depth,
+ symbol_seeds=symbol_seeds,
+ avoid=avoid,
+ helpers=helpers,
+ ),
+ maps,
+ )
+
+
+def closure_walk(targets: list[str], test_files: Collection[str]) -> Walk[dict[str, ReachedBy]]:
+ """Tests anywhere up the import closure of each target, in batches of ``CLOSURE_SEED_CAP``."""
+ out: dict[str, ReachedBy] = {}
+ for start in range(0, len(targets), CLOSURE_SEED_CAP):
+ batch = targets[start : start + CLOSURE_SEED_CAP]
+ found, entries = yield from import_walk(batch, test_files, CLOSURE_DEPTH)
+ for seed, tests in found.items():
+ by_test = {t: frozenset(via) for t, via in entries[seed].items()}
+ out[seed] = capped(seed, tests, "import-graph", changed=targets, entries=by_test)
+ return out
+
+
+def states_walk(
+ paths: list[str],
+ test_files: Collection[str],
+ *,
+ changed_tests: Collection[str] = frozenset(),
+ contents: Mapping[str, str | bytes] | None = None,
+) -> Walk[dict[str, ReachRow]]:
+ """A :class:`~.model.ReachRow` per path, the negative claim made only when it holds.
+
+ In order: ``type_only`` (from *contents*); ``reached`` by a test the change
+ adds or edits (*changed_tests*) that is named for the path or imports it,
+ then by the path's own inline tests; ``new`` for a path the index lacks;
+ ``no_signal`` when the index holds no test; ``reached`` by the call walk,
+ the import walk or the helper and barrel hop; ``reached`` through the whole
+ import closure; else ``unreached``.
+ """
+ contents = contents or {}
+ rows: dict[str, ReachRow] = {}
+ rest: list[str] = []
+ # Grouped once by directory, so each Go file scans only its own folder's tests.
+ go_tests: dict[str, list[str]] = {}
+ for t in test_files:
+ if t.endswith("_test.go"):
+ go_tests.setdefault(str(PurePosixPath(t).parent), []).append(t)
+ # Only a test a runner collects is evidence; a touched conftest or helper is not.
+ touched = sorted(
+ t
+ for t in set(changed_tests)
+ if is_runnable_test(t, is_test=(t in test_files) or None)
+ )
+ imported_by = yield from _touched_imports(paths, touched)
+ for path in dict.fromkeys(paths):
+ content = contents.get(path)
+ if content is not None and is_type_only(
+ path, content, is_test=(path in test_files) or None
+ ):
+ rows[path] = ReachRow("type_only")
+ continue
+ named = [t for t in touched if t in imported_by.get(path, ()) or names_test_for(t, path)]
+ if named:
+ rows[path] = ReachRow("reached", capped(path, named, "changed-test", changed=named))
+ continue
+ own = inline_tests(path, content, go_tests.get(str(PurePosixPath(path).parent), ()))
+ if own:
+ rows[path] = ReachRow("reached", capped(path, own, "inline"))
+ continue
+ rest.append(path)
+ known = (yield Need("known", tuple(rest))) if rest else set()
+ for path in rest:
+ if path not in known:
+ rows[path] = ReachRow("new")
+ elif not test_files:
+ rows[path] = ReachRow("no_signal")
+ rest = [p for p in rest if p not in rows]
+ found = yield from tier_walk(rest, test_files, helpers=True)
+ left = [p for p in rest if p not in found]
+ found |= yield from closure_walk(left, test_files)
+ for path in rest:
+ hit = found.get(path)
+ rows[path] = ReachRow("reached", hit) if hit else ReachRow("unreached")
+ return {path: rows[path] for path in dict.fromkeys(paths)}
+
+
+def _touched_imports(paths: list[str], touched: list[str]) -> Walk[dict[str, set[str]]]:
+ """``{path: the touched tests importing it}``, one hop; nothing to ask without touched tests."""
+ if not touched or not paths:
+ return {}
+ wanted = set(touched)
+ out: dict[str, set[str]] = {}
+ for dependent, dependency in (yield Need("importers", tuple(paths))):
+ if dependent in wanted:
+ out.setdefault(dependency, set()).add(dependent)
+ return out
+
+
+def reach_states(
+ maps: WalkMaps,
+ paths: Collection[str],
+ *,
+ changed_tests: Collection[str] = frozenset(),
+ contents: Mapping[str, str | bytes] | None = None,
+) -> dict[str, ReachRow]:
+ """:func:`states_walk` over *maps*: whether tests reach each path, as a typed claim.
+
+ *changed_tests* are test files the change adds or edits, indexed or not.
+ *contents* maps a path to its bytes, for the type-only and Rust inline
+ checks; a path without them skips both.
+ """
+ return run(
+ states_walk(
+ list(paths), maps.test_files, changed_tests=changed_tests, contents=contents
+ ),
+ maps,
+ )
diff --git a/packages/core/src/repowise/core/analysis/test_reachability.py b/packages/core/src/repowise/core/analysis/test_reachability.py
index d609efa18..ab74988a8 100644
--- a/packages/core/src/repowise/core/analysis/test_reachability.py
+++ b/packages/core/src/repowise/core/analysis/test_reachability.py
@@ -1,1271 +1,4 @@
-"""Static test-to-code map, derived from the dependency graph.
+"""The test reach walks, at the import path callers have always used: see :mod:`.test_reach`."""
-``test_coverage`` answers "which test executed this line" and is filled only by
-a coverage report. Most repositories have none, so ``tests_to_run`` came back
-empty, ``impacted_tests`` said "run the full suite", and ``untested_hotspot``
-fell back to matching filenames.
-
-Matching filenames fails both ways, and this repository is the proof. Of its six
-worst bug-magnet files, five - ``call_resolver.py``, ``dead_code/analyzer.py``,
-``pipeline/persist.py``, ``tool_answer/answer.py``, ``pr_blast.py`` - have no
-file named for them anywhere under ``tests/`` and so read as untested, while the
-graph names the test files that reach them. The sixth,
-``analysis/health/engine.py``, is worse: the convention matches on basename
-alone, so it paired with ``tests/unit/distill/test_engine.py`` - a different
-engine, in a different subsystem - and called the file tested on the strength of
-a name collision.
-
-The graph already knows the answer. This module reads that relation. It is a
-second signal beside the measured one, never a replacement and never averaged
-with it.
-
-Which graph, and why
---------------------
-Two graphs can answer "does a test reach this file", and they are not equally
-good at it.
-
-The **call graph** is the primary signal: seed the symbols a test file declares,
-walk ``EXECUTION_EDGE_TYPES`` forward, and every symbol reached is one the test
-can actually run. This is the closer model of the question - the claim is about
-execution, and these edges are execution.
-
-The **import graph** is the broader, weaker tier: a test file that imports a
-source file references it, which is real evidence and a much cruder one. A test
-importing a module pulls in every symbol the module defines while the test body
-may touch one function.
-
-Measured against real per-test coverage on this repository,
-over the slice where per-test attribution is complete and both sides see the
-same 37 test files (159 production files provably executed):
-
-============================ ======= ======= ========= =======
-forward signal claims correct precision recall
-============================ ======= ======= ========= =======
-import graph, 1 hop 43 31 72.1% 19.5%
-call graph, 3 hops 48 44 91.7% 27.7%
-call graph, 3 hops, filtered 46 44 95.7% 27.7%
-both unioned 57 45 78.9% 28.3%
-============================ ======= ======= ========= =======
-
-The call graph wins on both axes, so it leads. Unioning the import graph into it
-buys 0.6 points of recall for 16.8 points of precision, and a false "something
-reaches this" suppresses a real untested-hotspot finding, so the forward walk
-does not union - it is call edges only.
-
-The reverse walk combines them differently, because there the import tier can be
-spent only on the targets the call graph said nothing about:
-
-============================= ======= ====== =========
-reverse signal targets hit precision
-============================= ======= ====== =========
-import graph, 1 hop 32 96.9% 94.8%
-call graph, 3 hops, filtered 46 100.0% 97.5%
-both unioned 47 100.0% 95.8%
-call graph, else import graph 47 100.0% 97.5%
-============================= ======= ====== =========
-
-Falling back is free where unioning is not: it answers one more target at
-identical precision, because the second tier never speaks over the first.
-
-Both tables compare the signals uncapped and over the same test set, which is
-what makes them a choice between signals rather than two different experiments.
-End to end through this module, with the walk seeing the repository's whole test
-set and ``MAX_TESTS_PER_TARGET`` applied, the reverse walk scores 100.0% hit and
-95.5% precision on the same slice; the gap is the cap truncating a repo-wide
-list, not a difference in the walk.
-
-Depth, and why 3
-----------------
-The call walk saturates at 3 hops - 3, 4 and 5 return the same 48 claims and 44
-confirmations - so the recall ceiling is the call graph's own capture rate, not
-the depth, and there is nothing to buy by walking further. The import walk keeps
-its measured default of 1: a blanket second hop was tested and recovers 20 of
-128 misses while adding 50 wrong claims, and both a facade-only second hop
-through ``__init__.py`` (3 recovered, 5 added) and ``conftest.py`` transitivity
-(zero recovered) were tested and disproved. What the import graph misses is
-transitive *execution*, which import edges structurally cannot see, which is
-what the call graph is for.
-
-Why recall reads low, and why that is not the graph's fault
------------------------------------------------------------
-27.7% looks poor for a graph this dense, and the reason is what the ground truth
-counts. ``coverage`` records a line as run whether a test called into it or
-Python merely evaluated the module body on import, so a file that was only
-imported is indistinguishable in the truth set from one a test drove. Splitting
-the 159 truth files by how much of what ran was *inside a function body*:
-
-================================= ======= ======= ========
-what actually ran in the file files found recall
-================================= ======= ======= ========
-nothing inside any function body 39 0 0%
-under a quarter inside bodies 19 0 0%
-a quarter to three quarters 45 3 7%
-over three quarters inside bodies 56 43 77%
-================================= ======= ======= ========
-
-A quarter of the "covered" files never ran a single line inside a function. The
-walk is right to miss those: nothing called them, and claiming otherwise is the
-import graph's error, which is what it costs 16.8 points of precision to make.
-
-Of the 13 missed in the bottom row, 11 are ``alembic/versions/*`` migrations,
-whose ``upgrade``/``downgrade`` the framework invokes by naming convention with
-no static caller anywhere in the repository. Excluding those, the walk finds 43
-of 45 files that a test genuinely exercised and that anything statically calls.
-Chasing the headline number by widening the walk trades that away.
-
-Confidence, and why one origin is dropped
------------------------------------------
-``graph_edges.resolution_origin`` records which strategy resolved a call edge,
-and they are not equally trustworthy. ``global_unique`` binds a name to the only
-symbol carrying it anywhere in the repository, which is a guess, and it is the
-one origin the vocabulary itself scores at 0.50. Dropping it costs no recall at
-all and buys 4.0 points of forward precision (91.7% -> 95.7%) and 1.1 of reverse
-(96.4% -> 97.5%). Nothing else earns its keep: filtering ``receiver_global`` too
-changes no number, and a confidence floor of 0.90 cuts recall from 27.7% to
-23.3%.
-
-What it claims, and what it does not
-------------------------------------
-Reaching is not executing. A call edge says control *can* flow, not that a given
-test run did, so the inferred map **over-claims**, and the direction of the
-error is known and one-sided:
-
-* Sound as a floor. "Some test reaches this file, so do not call it untested"
- is safe: the edge is recorded, not guessed.
-* Unsound as a quantity. No percentage may be derived from it. There is no line
- attribution here at all - reaching is a file-level fact - so a caller that
- needs "are these changed *lines* covered" must use the measured map or say it
- does not know.
-
-Every surface that consumes this labels the result ``inferred``. Nothing here
-writes to ``test_coverage``, and no row it produces is stored: the relation is a
-bounded walk over ``graph_edges``/``graph_nodes``, which are already indexed and
-already fresh. Materialising it would cost a transitive closure that goes stale
-the moment the graph moves, to answer a query that is a breadth-first search
-over data already in the database.
-"""
-
-from __future__ import annotations
-
-import json
-import sys
-from collections.abc import Collection, Iterable, Mapping
-from dataclasses import dataclass
-from pathlib import PurePosixPath
-from typing import Any, Literal, NamedTuple, TypeAlias
-
-from sqlalchemy import select, text
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from repowise.core.analysis.execution_graph import (
- UNRELIABLE_EXECUTION_ORIGINS,
- ExecutionGraphIndex,
- file_of_symbol,
-)
-from repowise.core.ingestion.models import EXECUTION_EDGE_TYPES, FILE_DEPENDENCY_EDGE_TYPES
-from repowise.core.persistence.index_cache import IndexStampCache
-from repowise.core.persistence.models import GraphNode
-from repowise.core.test_paths import paired_test_names
-
-from .dead_code.file_reachability import BARREL_FILENAMES
-
-# How many call hops a test may take and still be said to reach a file. The walk
-# saturates here; see the Depth section above.
-DEFAULT_CALL_DEPTH = 3
-
-# How many import hops the weaker fallback tier may take. Still the measured
-# choice; a blanket second hop costs more than it buys.
-DEFAULT_MAX_DEPTH = 1
-
-# Call edges whose resolver only matched a name. See the Confidence section.
-UNRELIABLE_CALL_ORIGINS = UNRELIABLE_EXECUTION_ORIGINS
-
-# Cap on how many test files one target reports. The consumers all cut their
-# own lists shorter; the cap exists so a helper called from every test in the
-# suite cannot produce an unbounded intermediate.
-MAX_TESTS_PER_TARGET = 50
-
-# Which tier answered. The CLI prints this so a reader can tell "this test runs
-# into the file" from the weaker "this test imports it".
-# ``name-match`` is weaker still: a test named for the file, with no edge.
-ReachedVia = Literal["call-graph", "import-graph", "name-match"]
-
-__all__ = [
- "DEFAULT_CALL_DEPTH",
- "DEFAULT_MAX_DEPTH",
- "MAX_TESTS_PER_TARGET",
- "UNRELIABLE_CALL_ORIGINS",
- "CallGraphView",
- "ImportGraph",
- "ReachDistance",
- "ReachedBy",
- "any_tests_reaching",
- "cached_files_reached_by_tests",
- "cached_test_files",
- "call_graph_from_db",
- "call_graph_from_graph",
- "clear_test_map_cache",
- "direct_dependents",
- "direct_importers",
- "files_reached_by_tests",
- "files_with_paired_tests",
- "imported_names_by_test",
- "load_import_graph",
- "load_test_files",
- "rank_reaching",
- "reach_into_symbols",
- "tests_matching_by_name",
- "tests_reaching",
- "tests_reaching_by_tier",
- "tests_reaching_through_helpers",
-]
-
-
-CallGraphView: TypeAlias = ExecutionGraphIndex
-
-
-class ReachDistance(NamedTuple):
- """How a test reached a seed: the fewest call hops, and how many of the
- test's symbols made a call at that depth."""
-
- hops: int
- callers: int
-
-
-@dataclass(frozen=True)
-class ReachedBy:
- """Tests reaching one target, and which tier found them.
-
- ``total`` is how many the walk actually found, before
- ``MAX_TESTS_PER_TARGET`` trimmed ``tests``. A surface that prints
- ``len(tests)`` as the answer is stating a cap as a measurement. The cut keeps
- the closest tests (:func:`rank_reaching`). Carry the total and say the bound.
- """
-
- tests: list[str]
- via: ReachedVia
- total: int = 0
- # Internal uncapped identities let downstream aggregators de-duplicate one
- # test that reaches several targets while public lists remain bounded.
- all_tests: tuple[str, ...] | None = None
- # How close each test came, for the call tier only: the import and name
- # tiers have no hop count to report.
- reach: Mapping[str, ReachDistance] | None = None
- # Import tier only: the files each test was reached through, i.e. the
- # dependencies of that test on the route to the target.
- entries: Mapping[str, frozenset[str]] | None = None
-
-
-def call_graph_from_graph(graph: Any) -> CallGraphView:
- """Build the call-reachability view from an in-memory NetworkX graph.
-
- ``defines`` alone bridges files to symbols: measured across this repository
- it reaches every symbol, class members included, so also walking
- ``has_method`` would add a second containment layer for nothing.
-
- Returns an empty view for ``None`` - the health engine runs without a graph
- on some paths, and the documented outcome there is "no signal".
- """
- return ExecutionGraphIndex(graph)
-
-
-async def call_graph_from_db(session: AsyncSession, repo_id: str) -> CallGraphView:
- """The same view as :func:`call_graph_from_graph`, read from ``graph_edges``.
-
- The health engine holds the parsed graph in memory; a request handler holds
- a session. Both need the identical view, so this is a loader and not a
- second walk - :func:`files_reached_by_tests` still does the walking, and
- the two callers therefore cannot disagree about which files a test reaches.
-
- One bulk read rather than the reverse walk's per-level ``IN`` queries. That
- walk is seeded with a change's files; this question is seeded with the
- repository, and a per-level ``IN`` list over every file is the shape it was
- explicitly not built for.
-
- The view carries no call-site lines: reachability reads only ``declares``
- and the forward/reverse adjacency, and ``call_lines_json`` was most of the
- read's cost (measured on this repository, 189k edges: 8.4 s cold with the
- column, under 2 s without it).
- """
- wanted = ["defines", *sorted(EXECUTION_EDGE_TYPES)]
- params: dict[str, Any] = {"repo_id": repo_id}
- ets = _in_clause("e", wanted, params)
- rows = await session.execute(
- text(
- "SELECT source_node_id, target_node_id, edge_type, resolution_origin "
- "FROM graph_edges WHERE repository_id = :repo_id "
- f"AND edge_type IN ({ets}) "
- "ORDER BY source_node_id, target_node_id, edge_type"
- ),
- params,
- )
- # No call or spawn lines here: test reachability reads only forward/reverse,
- # never first-hop sites, ``spawn_only`` or roles, and a spawned callee is
- # still reached.
- return ExecutionGraphIndex(
- edge_rows=((src, dst, edge_type, origin, None) for src, dst, edge_type, origin in rows)
- )
-
-
-def files_reached_by_tests(
- view: CallGraphView,
- test_files: set[str],
- *,
- max_depth: int = DEFAULT_CALL_DEPTH,
-) -> set[str]:
- """Every non-test file some test can execute into within *max_depth* hops.
-
- One multi-source breadth-first search from every symbol the test files
- declare, so the cost is O(V+E) however many test files there are - this is
- the batch question ("which files are reached at all"), not the attributed
- one.
-
- Returns the reached set with the test files themselves removed: a test does
- not need a test.
- """
- if not test_files or max_depth < 1:
- return set()
-
- frontier: set[str] = set()
- for path in test_files:
- frontier.update(view.declares.get(path, ()))
- seen = set(frontier)
- # The seeds are what the tests *declare*, not what they reach. Counting them
- # would make containment alone a claim of reaching.
- reached: set[str] = set()
- for _ in range(max_depth):
- nxt: set[str] = set()
- for node in frontier:
- nxt.update(view.calls.get(node, ()))
- nxt -= seen
- if not nxt:
- break
- seen |= nxt
- reached |= nxt
- frontier = nxt
- return {file_of_symbol(symbol) for symbol in reached} - test_files
-
-
-def _dependencies_of(graph: Any, path: str) -> set[str]:
- try:
- return {
- dst
- for _src, dst, data in graph.out_edges(path, data=True)
- if data.get("edge_type") in FILE_DEPENDENCY_EDGE_TYPES
- }
- except Exception:
- return set()
-
-
-def files_with_paired_tests(
- graph: Any, paths: Collection[str], test_files: Collection[str]
-) -> set[str]:
- """Files among *paths* that have a test of their own, import graph first.
-
- A test file that imports a file directly is its test, wherever either
- lives: ``tests/unit/cli/test_init.py`` importing ``cli/main.py`` pairs
- them although no name says so. A test *named* for a file
- (:func:`paired_test_names`) still pairs, but only when the graph agrees
- or cannot say: the named test imports the file directly or through a
- re-export barrel, has no resolved dependency at all, or there is no graph.
- Otherwise the name is a collision - ``distill/test_engine.py`` tests the
- distill engine, not ``health/engine.py`` - and does not pair.
-
- One hop, plus one more through a re-export barrel: a test importing a
- package's ``index.ts`` or ``__init__.py`` pairs the files that barrel
- imports, since that is how a package's tests reach its modules. A
- file-level "some test imports this", never a quantity. The reach walk
- above stays call-graph-only; pairing is the looser question "does this
- file have a test", and against this repository's stored line coverage
- direct-import pairing finds 71.6% of covered files at 98.9% precision
- where names alone found 34.6% at 100%.
- """
- barrel_deps: dict[str, set[str]] = {}
-
- def imports_of(test: str) -> set[str]:
- direct = _dependencies_of(graph, test)
- through: set[str] = set()
- for mid in direct:
- if PurePosixPath(mid).name in BARREL_FILENAMES:
- if mid not in barrel_deps:
- barrel_deps[mid] = _dependencies_of(graph, mid)
- through |= barrel_deps[mid]
- return direct | through
-
- paired: set[str] = set()
- deps: dict[str, set[str]] = {}
- if graph is not None:
- for test in test_files:
- deps[test] = imports_of(test)
- paired.update(deps[test])
- paired &= set(paths)
-
- by_name: dict[str, list[str]] = {}
- for path in {*paths, *test_files}:
- by_name.setdefault(path.rsplit("/", 1)[-1], []).append(path)
-
- def named_pair_holds(test: str, path: str) -> bool:
- if graph is None:
- return True
- imported = deps[test] if test in deps else imports_of(test)
- return not imported or path in imported
-
- for path in paths:
- if path in paired:
- continue
- candidates = (c for name in paired_test_names(path) for c in by_name.get(name, ()))
- if any(c != path and named_pair_holds(c, path) for c in candidates):
- paired.add(path)
- return paired
-
-
-async def load_test_files(session: AsyncSession, repo_id: str) -> set[str]:
- """Paths of every file node ingestion classified as a test."""
- res = await session.execute(
- select(GraphNode.node_id).where(
- GraphNode.repository_id == repo_id,
- GraphNode.is_test == True, # noqa: E712
- )
- )
- return {row[0] for row in res.all()}
-
-
-# Answers kept in process per (database, repository, index stamp), so one
-# request reads the test files once whoever asks: Fix first's validation and
-# Do next both walk from them. Ceiling: Do next's per-file reach is still
-# walked once per index; persisting it at index time (the engine's
-# ``RepoTestIndex.reached_files`` set, stored as a per-file bool) is the upgrade path.
-_cache = IndexStampCache(32)
-
-
-def clear_test_map_cache() -> None:
- _cache.clear()
-
-
-async def cached_test_files(session: AsyncSession, repo_id: str) -> frozenset[str]:
- """:func:`load_test_files`, read once per index state; frozen because it is shared."""
- key = await _cache.key(session, repo_id, "test_files")
- if key in _cache:
- return _cache.get(key)
- return _cache.remember(key, frozenset(await load_test_files(session, repo_id)))
-
-
-async def cached_files_reached_by_tests(session: AsyncSession, repo_id: str) -> frozenset[str]:
- """:func:`files_reached_by_tests` over the stored graph, walked once per index state.
-
- The Tests page asks this on every load and the answer moves only when an
- index run rewrites the graph, so the bulk edge read is paid once per index.
- """
- key = await _cache.key(session, repo_id, "files_reached")
- if key in _cache:
- return _cache.get(key)
- test_files = set(await cached_test_files(session, repo_id))
- view = await call_graph_from_db(session, repo_id)
- return _cache.remember(key, frozenset(files_reached_by_tests(view, test_files)))
-
-
-async def any_tests_reaching(
- session: AsyncSession, repo_id: str, targets: Collection[str]
-) -> dict[str, ReachedBy]:
- """Whether any test reaches each target: the default tiers, then a test-only helper.
-
- The first walk is :func:`tests_reaching_by_tier` at its defaults, the one
- impacted-tests runs, so the two never disagree about a file it answers.
- For the rest, :func:`tests_reaching_through_helpers`. Cached per index state.
- """
- seeds = tuple(sorted(set(targets)))
- key = await _cache.key(session, repo_id, "any_reach", seeds)
- if key in _cache:
- return _cache.get(key)
- test_files = set(await cached_test_files(session, repo_id))
- found = await tests_reaching_by_tier(session, repo_id, list(seeds), test_files=test_files)
- rest = [s for s in seeds if s not in found]
- if rest:
- found |= await tests_reaching_through_helpers(session, repo_id, rest, test_files)
- return _cache.remember(key, found)
-
-
-async def tests_reaching_through_helpers(
- session: AsyncSession, repo_id: str, targets: list[str], test_files: set[str]
-) -> dict[str, ReachedBy]:
- """Tests that import a file only tests import, or a barrel, which imports the target.
-
- A second import hop, kept off hubs: a blanket second hop was measured and
- rejected (see Depth), but a module nothing outside the test suite imports
- is test support whatever its name (``attempt.spawn-workspace.test-support.ts``
- loading ``attempt.ts`` for 30-odd tests), so walking through it claims
- nothing a hub would. A barrel (``index.ts``, ``__init__.py``) passes on
- what it imports even when source imports it too, as
- :func:`files_with_paired_tests` already reads it: a package's tests
- import its entry, not each module behind it.
- """
- edge_types = sorted(FILE_DEPENDENCY_EDGE_TYPES)
- middles: dict[str, set[str]] = {}
- for dependent, target in await _edges_into(session, repo_id, targets, edge_types, frozenset()):
- if dependent not in test_files:
- middles.setdefault(dependent, set()).add(target)
- importers: dict[str, set[str]] = {}
- for dependent, middle in await _edges_into(
- session, repo_id, sorted(middles), edge_types, frozenset()
- ):
- importers.setdefault(middle, set()).add(dependent)
- reached: dict[str, set[str]] = {}
- for middle, who in importers.items():
- tests = who & test_files
- if tests and (tests == who or PurePosixPath(middle).name in BARREL_FILENAMES):
- for target in middles[middle]:
- reached.setdefault(target, set()).update(tests)
- out: dict[str, ReachedBy] = {}
- for target, tests in reached.items():
- ordered = tuple(rank_reaching(target, tests, changed=targets))
- out[target] = ReachedBy(
- list(ordered[:MAX_TESTS_PER_TARGET]), "import-graph", len(ordered), ordered
- )
- return out
-
-
-async def placed_test_files(session: AsyncSession, repo_id: str) -> set[str]:
- """Test files with a resolved dependency edge into a non-test file of the repository.
-
- Only these can be found by walking back from a changed file; a test with
- none (every import unresolved, or it only runs a subprocess) is invisible.
- An edge into test material (a conftest, a helper) or a third-party module
- (``external:``) does not count: it says nothing about which of the
- repository's files the test exercises.
- """
- params: dict[str, Any] = {"repo_id": repo_id, "is_test": True, "not_test": False}
- ets = _in_clause("e", sorted(FILE_DEPENDENCY_EDGE_TYPES), params)
- rows = await session.execute(
- text(
- "SELECT DISTINCT e.source_node_id FROM graph_edges e "
- "JOIN graph_nodes s ON s.repository_id = e.repository_id "
- "AND s.node_id = e.source_node_id "
- "JOIN graph_nodes t ON t.repository_id = e.repository_id "
- "AND t.node_id = e.target_node_id "
- f"WHERE e.repository_id = :repo_id AND e.edge_type IN ({ets}) "
- "AND s.is_test = :is_test AND t.is_test = :not_test AND t.node_type = 'file' "
- "AND t.node_id NOT LIKE 'external:%'"
- ),
- params,
- )
- return {row[0] for row in rows}
-
-
-async def always_run_test_files(session: AsyncSession, repo_id: str) -> dict[str, str]:
- """``{test file: why}`` for the tests the indexer found the graph cannot see into.
-
- Stored per file at index time (``repowise.core.ingestion.always_run``), so
- this is one indexed read, never a scan of the test files.
- """
- res = await session.execute(
- select(GraphNode.node_id, GraphNode.always_run_reason).where(
- GraphNode.repository_id == repo_id,
- GraphNode.always_run_reason.is_not(None),
- GraphNode.always_run_reason != "",
- )
- )
- return {node_id: reason for node_id, reason in res.all()}
-
-
-async def unscanned_test_files(session: AsyncSession, repo_id: str) -> set[str]:
- """Judged test files the indexer never stamped (an index built before the check).
-
- A stamped test holds ``""`` when it is ordinary, so ``NULL`` means unknown,
- not "nothing found": such a test may walk the tree unseen.
- """
- from ..ingestion.always_run import is_judged_test
-
- res = await session.execute(
- select(GraphNode.node_id).where(
- GraphNode.repository_id == repo_id,
- GraphNode.node_type == "file",
- GraphNode.is_test == True, # noqa: E712
- GraphNode.always_run_reason.is_(None),
- )
- )
- return {node_id for (node_id,) in res.all() if is_judged_test(node_id)}
-
-
-async def dependency_path(
- session: AsyncSession, repo_id: str, source: str, targets: Collection[str], max_depth: int = 64
-) -> list[str]:
- """The shortest file-dependency route from *source* to any of *targets*, ends included.
-
- ``[]`` when none is found within *max_depth* hops. Breadth-first, one ``IN``
- query per level, over the same edges the import walk follows
- (``FILE_DEPENDENCY_EDGE_TYPES``, every resolution origin). Ceiling: a test
- found through the call walk alone may have no such route; following call
- edges with ``UNRELIABLE_CALL_ORIGINS`` excluded is the upgrade path.
- """
- goals = set(targets)
- parent: dict[str, str | None] = {source: None}
- frontier = [source]
- for _ in range(max_depth):
- if not frontier or goals & parent.keys():
- break
- level, frontier = frontier, []
- for dependent, dependency in await _edges_from(
- session, repo_id, level, sorted(FILE_DEPENDENCY_EDGE_TYPES)
- ):
- if dependency not in parent:
- parent[dependency] = dependent
- frontier.append(dependency)
- hit = goals & parent.keys()
- if not hit:
- return []
- route, node = [], min(hit)
- while node is not None:
- route.append(node)
- node = parent[node]
- return route[::-1]
-
-
-async def tests_reaching(
- session: AsyncSession,
- repo_id: str,
- targets: list[str],
- *,
- call_depth: int = DEFAULT_CALL_DEPTH,
- import_depth: int = DEFAULT_MAX_DEPTH,
- test_files: set[str] | None = None,
-) -> dict[str, list[str]]:
- """Test files that reach each of *targets*, keyed by target path.
-
- Targets nothing reaches are absent from the result. An empty dict is
- "no test reaches these"; the caller decides whether that reads as untested
- or as unknown.
- """
- found = await tests_reaching_by_tier(
- session,
- repo_id,
- targets,
- call_depth=call_depth,
- import_depth=import_depth,
- test_files=test_files,
- )
- return {target: reached.tests for target, reached in found.items()}
-
-
-async def tests_reaching_by_tier(
- session: AsyncSession,
- repo_id: str,
- targets: list[str],
- *,
- call_depth: int = DEFAULT_CALL_DEPTH,
- import_depth: int = DEFAULT_MAX_DEPTH,
- symbol_seeds: Mapping[str, Collection[str]] | None = None,
- test_files: set[str] | None = None,
- avoid: Collection[str] = frozenset(),
- import_graph: ImportGraph | None = None,
-) -> dict[str, ReachedBy]:
- """:func:`tests_reaching`, also saying which tier answered each target.
-
- *test_files* is :func:`load_test_files`'s answer, for a caller that walks
- more than once and need not read it twice. A target may be a symbol id
- (``path::name``); its tests are ranked against the file part.
-
- *symbol_seeds* narrows the call walk for the targets it names: the walk
- enters at exactly those symbol ids instead of at every symbol the file
- declares. A target it does not name, or names with no ids, keeps the
- ``defines`` lookup, and the
- import tier stays file-level either way.
-
- *avoid* names files the call walk does not pass reach through, except
- into a target's own symbols. The import tier is
- one hop, so it never passes through anything.
-
- *import_graph* (:func:`load_import_graph`) answers the import tier's
- levels from memory, for a caller that walks deep or more than once.
-
- The call walk runs first; the import walk is then seeded with only the
- targets it left unanswered, so the weaker tier never speaks over the
- stronger one and costs nothing where the stronger one already spoke.
-
- Both walks are attributed: they run backwards from the targets carrying the
- seed each visited node came from, so the answer says *which* target each
- test guards rather than only that some test does. That is why these are
- separate walks rather than a filter over the forward one, and it stays cheap
- because the seed set is a change's files, not the repository - one ``IN``
- query per level, which is the shape ``pr_blast._transitive_affected``
- already uses.
- """
- seeds = sorted({t for t in targets if t})
- if not seeds:
- return {}
-
- if test_files is None:
- test_files = await load_test_files(session, repo_id)
- if not test_files:
- return {}
-
- out: dict[str, ReachedBy] = {}
- # The targets are usually a change's files; a test among them leads every list.
- if call_depth >= 1:
- found = await _call_reaching(
- session, repo_id, seeds, test_files, call_depth, symbol_seeds=symbol_seeds, avoid=avoid
- )
- for seed, reach in found.items():
- ordered = tuple(rank_reaching(seed, reach, reach=reach, changed=seeds))
- out[seed] = ReachedBy(
- list(ordered[:MAX_TESTS_PER_TARGET]), "call-graph", len(ordered), ordered, reach
- )
-
- unanswered = [seed for seed in seeds if seed not in out]
- if unanswered and import_depth >= 1:
- found, entries = await _import_reaching(
- session, repo_id, unanswered, test_files, import_depth, import_graph
- )
- for seed, tests in found.items():
- # A test importing the target itself, not a file on the way to it.
- file = seed.split("::", 1)[0]
- direct = {t for t, via in entries[seed].items() if file in via}
- ordered = tuple(rank_reaching(seed, tests, direct=direct, changed=seeds))
- out[seed] = ReachedBy(
- list(ordered[:MAX_TESTS_PER_TARGET]),
- "import-graph",
- len(ordered),
- ordered,
- entries={t: frozenset(via) for t, via in entries[seed].items()},
- )
- return out
-
-
-async def _call_reaching(
- session: AsyncSession,
- repo_id: str,
- seeds: list[str],
- test_files: set[str],
- max_depth: int,
- *,
- symbol_seeds: Mapping[str, Collection[str]] | None = None,
- strict: bool = False,
- avoid: Collection[str] = frozenset(),
-) -> dict[str, dict[str, ReachDistance]]:
- """Tests that can execute into each seed file, walking call edges backwards.
-
- Starts one layer below the other walk: the seeds are files and call edges
- join symbols, so the first query resolves each seed to the symbols it
- declares, and the walk carries the seed from there.
-
- Each test carries the fewest call hops it took and how many of its symbols
- made a call at that depth. Every node records each seed's distance, not
- the level it was found at: one level's batch can carry a seed through a
- node another seed entered, so the level alone overstates how close a test
- is. A shorter distance found later re-queues the node; which tests reach
- which seed is unchanged by that, only how far.
-
- *strict* reads each level against the distances it started with, so a seed
- moves one hop per level and its answer does not depend on which other
- seeds share the walk. The file tiers keep the historical batch semantics.
- """
- # node -> seed -> call hops from the node to the seed.
- origins = await _call_origins(session, repo_id, seeds, symbol_seeds)
- if not origins:
- return {}
-
- found: dict[str, dict[str, tuple[int, set[str]]]] = {}
- frontier = list(origins)
- for _ in range(max_depth):
- if not frontier:
- break
- level, frontier = frontier, []
- queued: set[str] = set()
- start = {node: dict(origins[node]) for node in level} if strict else origins
- for caller, callee in await _edges_into(
- session,
- repo_id,
- level,
- sorted(EXECUTION_EDGE_TYPES),
- UNRELIABLE_CALL_ORIGINS,
- ):
- carried = start.get(callee)
- if not carried:
- continue
- owner = file_of_symbol(caller)
- if owner in test_files:
- _record_test_hit(found, owner, caller, carried)
- # A test is a leaf. Walking through one would let "test A calls
- # shared helper B" drag B's unrelated targets in.
- continue
- if owner in avoid:
- # A hub carries on only the seeds it declares itself.
- carried = {s: d for s, d in carried.items() if s.split("::", 1)[0] == owner}
- if not _carry_closer(origins, caller, carried):
- continue
- if caller not in queued:
- queued.add(caller)
- frontier.append(caller)
- return {
- seed: {test: ReachDistance(depth, len(callers)) for test, (depth, callers) in tests.items()}
- for seed, tests in found.items()
- }
-
-
-async def _call_origins(
- session: AsyncSession,
- repo_id: str,
- seeds: list[str],
- symbol_seeds: Mapping[str, Collection[str]] | None,
-) -> dict[str, dict[str, int]]:
- """The symbols each seed enters the call walk at, each at distance 0 from its seed."""
- origins: dict[str, dict[str, int]] = {}
- # A caller that knows which symbol in the file it cares about enters there,
- # so a test reaching an unrelated symbol in the same file does not count.
- # An empty entry names no symbol, so that file is unseeded and keeps the
- # ``defines`` lookup; treating it as seeded would silence the file entirely.
- seeded = {
- seed: symbol_seeds[seed]
- for seed in seeds
- if symbol_seeds and symbol_seeds.get(seed)
- }
- for seed, symbol_ids in seeded.items():
- for symbol in symbol_ids:
- origins.setdefault(symbol, {})[seed] = 0
- unseeded = [seed for seed in seeds if seed not in seeded]
- if unseeded:
- declared = await _edges_from(session, repo_id, unseeded, ["defines"])
- for seed, symbol in declared:
- origins.setdefault(symbol, {})[seed] = 0
- return origins
-
-
-def _record_test_hit(
- found: dict[str, dict[str, tuple[int, set[str]]]],
- test: str,
- caller: str,
- carried: Mapping[str, int],
-) -> None:
- """Keep the fewest hops *test* took to each carried seed, and its callers at that depth."""
- for seed, distance in carried.items():
- hops = distance + 1
- by_test = found.setdefault(seed, {})
- best = by_test.get(test)
- if best is None or hops < best[0]:
- by_test[test] = (hops, {caller})
- elif hops == best[0]:
- best[1].add(caller)
-
-
-def _carry_closer(
- origins: dict[str, dict[str, int]], caller: str, carried: Mapping[str, int]
-) -> bool:
- """Record the seeds *caller* reaches closer than before; True when any did."""
- if not carried:
- return False
- known = origins.setdefault(caller, {})
- closer = {
- seed: distance + 1
- for seed, distance in carried.items()
- if seed not in known or distance + 1 < known[seed]
- }
- known.update(closer)
- return bool(closer)
-
-
-async def reach_into_symbols(
- session: AsyncSession,
- repo_id: str,
- symbol_ids: Collection[str],
- test_files: set[str],
- *,
- max_depth: int = DEFAULT_CALL_DEPTH,
- avoid: Collection[str] = frozenset(),
-) -> dict[str, dict[str, ReachDistance]]:
- """Tests that call into each symbol id, keyed by the symbol, with their hops.
-
- The file walk answers "does a test reach this file"; this one answers how
- close each test gets to one symbol in it, which is what decides which of a
- file's many tests to run first after changing that symbol.
- """
- seeds = sorted({symbol for symbol in symbol_ids if symbol})
- if not seeds or not test_files:
- return {}
- return await _call_reaching(
- session,
- repo_id,
- seeds,
- test_files,
- max_depth,
- symbol_seeds={symbol: (symbol,) for symbol in seeds},
- strict=True,
- avoid=avoid,
- )
-
-
-async def imported_names_by_test(
- session: AsyncSession, repo_id: str, files: Collection[str], test_files: set[str]
-) -> dict[str, dict[str, frozenset[str]]]:
- """The names each test file imports from each of *files*, keyed by file then test.
-
- An empty set is an import of the module itself. One hop, plus one through a
- re-export barrel (``index.ts``, ``__init__.py``) that imports the file: a
- test importing ``foo`` from the barrel imports it from the file. What a test
- names, not what it reaches.
- """
- targets = sorted({path for path in files if path})
- if not targets or not test_files:
- return {}
- out: dict[str, dict[str, frozenset[str]]] = {}
- barrels: dict[str, dict[str, frozenset[str]]] = {}
- for source, target, names in await _imported_names(session, repo_id, targets):
- if source in test_files:
- _add_names(out, target, source, names)
- elif PurePosixPath(source).name in BARREL_FILENAMES:
- barrels.setdefault(source, {})[target] = names
- if barrels:
- await _add_barrel_imports(session, repo_id, barrels, test_files, out)
- return out
-
-
-async def _add_barrel_imports(
- session: AsyncSession,
- repo_id: str,
- barrels: dict[str, dict[str, frozenset[str]]],
- test_files: set[str],
- out: dict[str, dict[str, frozenset[str]]],
-) -> None:
- """Credit each test importing a barrel with the names the barrel passes on from each file."""
- for source, barrel, names in await _imported_names(session, repo_id, sorted(barrels)):
- if source not in test_files:
- continue
- for target, exported in barrels[barrel].items():
- # ``export *`` passes every name on; otherwise only the ones it re-exports.
- passed = names if not exported or "*" in exported else names & exported
- if passed or not names:
- _add_names(out, target, source, passed)
-
-
-def _add_names(
- out: dict[str, dict[str, frozenset[str]]], target: str, test: str, names: frozenset[str]
-) -> None:
- by_test = out.setdefault(target, {})
- by_test[test] = by_test.get(test, frozenset()) | names
-
-
-async def _imported_names(
- session: AsyncSession, repo_id: str, targets: list[str]
-) -> list[tuple[str, str, frozenset[str]]]:
- """``(importer, imported file, names)`` for every import edge into *targets*."""
- shared: dict[str, Any] = {"repo_id": repo_id}
- ets = _in_clause("e", sorted(FILE_DEPENDENCY_EDGE_TYPES), shared)
- targets = list(dict.fromkeys(targets))
- out = []
- for start in range(0, len(targets), _IN_BATCH):
- params = dict(shared)
- tgt = _in_clause("p", targets[start : start + _IN_BATCH], params)
- rows = await session.execute(
- text(
- "SELECT source_node_id, target_node_id, imported_names_json FROM graph_edges "
- "WHERE repository_id = :repo_id "
- f"AND target_node_id IN ({tgt}) AND edge_type IN ({ets})"
- ),
- params,
- )
- out.extend((source, target, _names_of(names_json)) for source, target, names_json in rows)
- return out
-
-
-def _names_of(names_json: str | None) -> frozenset[str]:
- """The names an import edge records; unreadable JSON records none."""
- try:
- names = json.loads(names_json or "[]")
- except (TypeError, ValueError):
- return frozenset()
- return frozenset(n for n in names if isinstance(n, str))
-
-
-@dataclass(frozen=True)
-class ImportGraph:
- """Every file-dependency edge, reversed (``{file: files depending on it}``), read once.
-
- The import walk is unbounded on the selection path, so on a hub it runs
- dozens of levels; one bulk read replaces a query per level. Either source
- gives the same answer: the walk reaches a fixpoint, so edge order does not
- matter. Ceiling: read per selection, not kept across calls; caching it per
- index stamp is the upgrade path if the read shows up.
- """
-
- importers: Mapping[str, tuple[str, ...]]
-
- def edges_into(self, targets: Collection[str]) -> list[tuple[str, str]]:
- """``(dependent, dependency)`` pairs, as :func:`_edges_into` returns them."""
- return [(src, t) for t in targets for src in self.importers.get(t, ())]
-
- def hops(self, sources: Collection[str]) -> dict[str, int]:
- """``{file: fewest dependency hops from any of *sources*}``, sources at 0.
-
- One multi-source breadth-first pass through every file, tests
- included, so a test importing a reached helper is one hop past it.
- """
- best = dict.fromkeys(sources, 0)
- frontier = list(best)
- depth = 0
- while frontier:
- depth += 1
- fresh = [
- d for node in frontier for d in self.importers.get(node, ()) if d not in best
- ]
- fresh = list(dict.fromkeys(fresh))
- best.update(dict.fromkeys(fresh, depth))
- frontier = fresh
- return best
-
-
-async def load_import_graph(session: AsyncSession, repo_id: str) -> ImportGraph:
- """The edges the import walk follows (``_edges_into`` over every file), in one read."""
- params: dict[str, Any] = {"repo_id": repo_id}
- ets = _in_clause("e", sorted(FILE_DEPENDENCY_EDGE_TYPES), params)
- rows = await session.execute(
- text(
- "SELECT DISTINCT source_node_id, target_node_id FROM graph_edges "
- f"WHERE repository_id = :repo_id AND edge_type IN ({ets}){_LEGACY_CONFTEST_FILTER}"
- ),
- params,
- )
- importers: dict[str, list[str]] = {}
- for source, target in rows:
- importers.setdefault(target, []).append(source)
- # Sorted, so a bounded walk does not depend on the order rows come back in.
- return ImportGraph({t: tuple(sorted(s)) for t, s in importers.items()})
-
-
-async def _import_reaching(
- session: AsyncSession,
- repo_id: str,
- seeds: list[str],
- test_files: set[str],
- max_depth: int,
- graph: ImportGraph | None = None,
-) -> tuple[dict[str, set[str]], dict[str, dict[str, set[str]]]]:
- """Tests that import each seed file, directly or within *max_depth* hops.
-
- Also ``{seed: {test: the files it imports on a route to the seed}}``.
- With *graph* the levels are read from it instead of the database.
- """
- origins: dict[str, set[str]] = {s: {s} for s in seeds}
- # (test, file it imports) -> seeds carried in: one set union per edge, not
- # a dict write per seed; split per seed once the walk ends.
- reached: dict[tuple[str, str], set[str]] = {}
- seed_set = set(seeds)
- frontier = list(seeds)
-
- for _ in range(max_depth):
- if not frontier:
- break
- level, frontier = frontier, []
- queued: set[str] = set()
- edges = (
- graph.edges_into(level)
- if graph is not None
- else await _edges_into(
- session, repo_id, level, sorted(FILE_DEPENDENCY_EDGE_TYPES), frozenset()
- )
- )
- for dependent, dependency in edges:
- carried = origins.get(dependency)
- if not carried:
- continue
- if dependent in test_files:
- reached.setdefault((dependent, dependency), set()).update(carried)
- continue
- if dependent in seed_set:
- continue
- known = origins.setdefault(dependent, set())
- fresh = carried - known
- if not fresh:
- # Already reached carrying nothing new. Without this, a diamond
- # in the import graph re-walks the whole subtree once per path
- # into it.
- continue
- known |= fresh
- if dependent not in queued:
- queued.add(dependent)
- frontier.append(dependent)
- found: dict[str, set[str]] = {}
- entries: dict[str, dict[str, set[str]]] = {}
- for (test, dependency), carried in reached.items():
- for seed in carried:
- found.setdefault(seed, set()).add(test)
- entries.setdefault(seed, {}).setdefault(test, set()).add(dependency)
- return found, entries
-
-
-async def direct_dependents(
- session: AsyncSession, repo_id: str, files: list[str]
-) -> dict[str, set[str]]:
- """Files with an import edge into each of *files*, or a call into a symbol it declares.
-
- One hop, no tests excluded: the direct callers and importers a change to
- them could have moved coverage through. A file with none is absent.
- """
- out: dict[str, set[str]] = {}
- for dependent, dependency in await _edges_into(
- session, repo_id, files, sorted(FILE_DEPENDENCY_EDGE_TYPES), frozenset()
- ):
- out.setdefault(dependency, set()).add(dependent)
- declared = await _edges_from(session, repo_id, files, ["defines"])
- owner = {symbol: path for path, symbol in declared}
- for caller, callee in await _edges_into(
- session, repo_id, sorted(owner), sorted(EXECUTION_EDGE_TYPES), UNRELIABLE_CALL_ORIGINS
- ):
- if (source := file_of_symbol(caller)) != owner[callee]:
- out.setdefault(owner[callee], set()).add(source)
- return out
-
-
-# Indexes built before the conftest -> test hint was removed still hold edges
-# that point from a conftest to each test using one of its fixtures, which made
-# the conftest read as that test's importer. The test -> conftest edge already
-# records the relation. Upgrade path: drop this once such indexes are rebuilt.
-_LEGACY_CONFTEST_FILTER = (
- " AND NOT (edge_type = 'dynamic_uses' AND COALESCE(hint_source, '') = 'pytest_conftest')"
-)
-
-
-#: Edges a test file writes itself: imports and references, not the wiring a
-#: test runner adds (a conftest or setup file reaches every test under it).
-_AUTHORED_EDGE_TYPES = sorted(FILE_DEPENDENCY_EDGE_TYPES - {"framework"})
-
-
-async def direct_importers(session: AsyncSession, repo_id: str, files: list[str]) -> set[str]:
- """Files whose own code imports or references one of *files*, one hop."""
- rows = await _edges_into(session, repo_id, files, _AUTHORED_EDGE_TYPES, frozenset())
- return {source for source, _ in rows}
-
-
-def _in_clause(prefix: str, values: list[str], params: dict[str, Any]) -> str:
- """Bind *values* as ``:prefix0, :prefix1, ...`` and return the clause body."""
- params.update({f"{prefix}{i}": v for i, v in enumerate(values)})
- return ",".join(f":{prefix}{i}" for i in range(len(values)))
-
-
-#: Values one ``IN`` list binds. SQLite refuses a statement with more variables
-#: than SQLITE_MAX_VARIABLE_NUMBER (999 before 3.32, 32766 after), and the lists
-#: below are a change's files or every file and symbol of a repository.
-_IN_BATCH = 500
-
-
-async def _edges_from(
- session: AsyncSession, repo_id: str, sources: list[str], edge_types: list[str]
-) -> list[tuple[str, str]]:
- """``(source, target)`` pairs leaving *sources* along *edge_types*."""
- shared: dict[str, Any] = {"repo_id": repo_id}
- ets = _in_clause("e", edge_types, shared)
- out: list[tuple[str, str]] = []
- # Each source is in one batch only, so a pair is distinct across batches too.
- sources = list(dict.fromkeys(sources))
- for start in range(0, len(sources), _IN_BATCH):
- params = dict(shared)
- src = _in_clause("s", sources[start : start + _IN_BATCH], params)
- rows = await session.execute(
- text(
- "SELECT DISTINCT source_node_id, target_node_id FROM graph_edges "
- "WHERE repository_id = :repo_id "
- f"AND source_node_id IN ({src}) AND edge_type IN ({ets})"
- ),
- params,
- )
- out.extend((s, t) for s, t in rows)
- return out
-
-
-async def _edges_into(
- session: AsyncSession,
- repo_id: str,
- targets: list[str],
- edge_types: list[str],
- excluded_origins: frozenset[str],
-) -> list[tuple[str, str]]:
- """``(source, target)`` pairs arriving at *targets* along *edge_types*.
-
- Raw text SQL with an ``IN`` list, matching ``pr_blast._transitive_affected``:
- the edge table is the one place a per-level parameterised ``IN`` beats
- loading every row, and the two walks should not disagree about how they read
- it.
- """
- shared: dict[str, Any] = {"repo_id": repo_id}
- ets = _in_clause("e", edge_types, shared)
- origin_filter = ""
- if excluded_origins:
- # NULL means the row predates the vocabulary, not "unknown", so it has
- # to survive the filter, and a bare NOT IN would drop it.
- bad = _in_clause("o", sorted(excluded_origins), shared)
- origin_filter = f" AND (resolution_origin IS NULL OR resolution_origin NOT IN ({bad}))"
- out: list[tuple[str, str]] = []
- # Each target is in one batch only, so a pair is distinct across batches too.
- targets = list(dict.fromkeys(targets))
- for start in range(0, len(targets), _IN_BATCH):
- params = dict(shared)
- tgt = _in_clause("p", targets[start : start + _IN_BATCH], params)
- rows = await session.execute(
- text(
- "SELECT DISTINCT source_node_id, target_node_id FROM graph_edges "
- "WHERE repository_id = :repo_id "
- f"AND target_node_id IN ({tgt}) AND edge_type IN ({ets}){origin_filter}"
- f"{_LEGACY_CONFTEST_FILTER}"
- ),
- params,
- )
- out.extend((s, t) for s, t in rows)
- return out
-
-
-def rank_reaching(
- target: str,
- tests: Iterable[str],
- *,
- reach: Mapping[str, ReachDistance] | None = None,
- direct: Collection[str] = frozenset(),
- changed: Collection[str] = frozenset(),
-) -> list[str]:
- """*tests* reaching *target*, the likeliest to exercise it first. Cap after this.
-
- In turn: a test in *changed* (the change edits it), fewer call hops
- (*reach*; a test without one sorts after every test with one), a test in
- *direct* (it imports or calls *target* itself), more of the test's
- functions at that distance, then by name: the test named for *target*,
- the one mirroring more of its directories, and the path. Targets, tests
- and *changed* may be ``path::name`` ids; every key reads the file part.
- """
- path_of = target.split("::", 1)[0]
- names = paired_test_names(path_of)
- dirs = set(path_of.split("/")[:-1])
- reach = reach or {}
- edited = {c.split("::", 1)[0] for c in changed}
-
- def key(test: str) -> tuple[bool, int, bool, int, bool, int, str]:
- path = test.split("::", 1)[0]
- distance = reach.get(test) or reach.get(path)
- return (
- path not in edited,
- distance.hops if distance else sys.maxsize,
- test not in direct and path not in direct,
- -distance.callers if distance else 0,
- path.rsplit("/", 1)[-1] not in names,
- -len(dirs & set(path.split("/")[:-1])),
- test,
- )
-
- return sorted(tests, key=key)
-
-
-def tests_matching_by_name(targets: list[str], test_files: Collection[str]) -> dict[str, ReachedBy]:
- """Tests named for each target by convention: the fallback when no edge answered.
-
- An unresolved graph (dynamic client, unindexed framework) leaves
- ``tests/test_x.py`` with no edge to ``x.py``; "unknown" there argues
- against a change a test guards.
- """
- by_name: dict[str, list[str]] = {}
- for path in test_files:
- by_name.setdefault(path.rsplit("/", 1)[-1], []).append(path)
- out: dict[str, ReachedBy] = {}
- for target in targets:
- found = rank_reaching(
- target, [path for name in paired_test_names(target) for path in by_name.get(name, ())]
- )
- if found:
- out[target] = ReachedBy(
- found[:MAX_TESTS_PER_TARGET], "name-match", len(found), tuple(found)
- )
- return out
+from .test_reach import * # noqa: F403
+from .test_reach import __all__ as __all__
diff --git a/packages/core/src/repowise/core/analysis/test_selection.py b/packages/core/src/repowise/core/analysis/test_selection.py
index 1a4cf2595..7ec638fb7 100644
--- a/packages/core/src/repowise/core/analysis/test_selection.py
+++ b/packages/core/src/repowise/core/analysis/test_selection.py
@@ -497,15 +497,21 @@ def is_runnable_test(
path: str,
roots: PytestRoots | None = None,
phpunit_roots: PhpunitRoots | None = None,
+ *,
+ is_test: bool | None = None,
) -> bool:
"""A test-shaped file name with an extension a runner collects tests from.
With *roots*, a Python file pytest's config leaves out of collection
(``core/test_paths.py``) is not one: the same rule that stamps ``is_test``.
With *phpunit_roots*, a PHP file PHPUnit leaves out of collection is not runnable.
+ *is_test* is the stored flag when the caller has it: a file ingestion did
+ not classify as a test is not runnable whatever its name. Test support
+ (``conftest.py``, fixtures, helpers) never is.
"""
return (
- _runnable_name(path)
+ is_test is not False
+ and _runnable_name(path)
and (roots is None or is_test_path(path, roots=roots))
and (phpunit_roots is None or is_test_path(path, phpunit_roots=phpunit_roots))
)
diff --git a/tests/unit/analysis/test_test_reach_api.py b/tests/unit/analysis/test_test_reach_api.py
new file mode 100644
index 000000000..55e302e68
--- /dev/null
+++ b/tests/unit/analysis/test_test_reach_api.py
@@ -0,0 +1,336 @@
+"""The reach walks over a graph held in memory: tiers, the typed states and their evidence."""
+
+from __future__ import annotations
+
+import networkx as nx
+import pytest
+
+from repowise.core.analysis.test_reach import (
+ ReachDistance,
+ build_walk_maps,
+ go_inline_tests,
+ is_runnable_test,
+ is_type_only,
+ reach_states,
+ reaching_by_tier,
+ rust_has_inline_tests,
+)
+
+
+def _graph(files: dict[str, bool], edges: list[tuple]) -> nx.MultiDiGraph:
+ """File nodes (``{path: is_test}``) and edges ``(src, dst, type[, attrs])``."""
+ g = nx.MultiDiGraph()
+ for path, is_test in files.items():
+ g.add_node(path, node_type="file", is_test=is_test)
+ for src, dst, edge_type, *attrs in edges:
+ if edge_type == "defines":
+ g.add_node(dst, node_type="symbol")
+ g.add_edge(src, dst, edge_type=edge_type, **(attrs[0] if attrs else {}))
+ return g
+
+
+def _call(test: str, source: str, *, origin: str | None = None) -> list[tuple]:
+ attrs = {"resolution_origin": origin} if origin else {}
+ return [
+ (test, f"{test}::test_it", "defines"),
+ (source, f"{source}::run", "defines"),
+ (f"{test}::test_it", f"{source}::run", "calls", attrs),
+ ]
+
+
+# ---------------------------------------------------------------------------
+# build_walk_maps
+# ---------------------------------------------------------------------------
+
+
+def test_maps_read_a_networkx_graph_and_its_node_link_dict_alike() -> None:
+ g = _graph(
+ {"tests/test_a.py": True, "src/a.py": False},
+ [*_call("tests/test_a.py", "src/a.py"), ("tests/test_a.py", "src/a.py", "imports")],
+ )
+ for source in (g, nx.node_link_data(g), nx.node_link_data(g, edges="links")):
+ maps = build_walk_maps(source)
+ assert maps.test_files == {"tests/test_a.py"}
+ assert maps.files == {"tests/test_a.py", "src/a.py"}
+ assert maps.callers["src/a.py::run"] == ("tests/test_a.py::test_it",)
+ assert maps.imports.importers["src/a.py"] == ("tests/test_a.py",)
+
+
+def test_maps_drop_name_guessed_calls_and_the_legacy_conftest_hint() -> None:
+ g = _graph(
+ {"tests/test_a.py": True, "tests/conftest.py": True, "src/a.py": False},
+ [
+ *_call("tests/test_a.py", "src/a.py", origin="global_unique"),
+ ("tests/conftest.py", "tests/test_a.py", "dynamic_uses", {"hint_source": "pytest_conftest"}),
+ ],
+ )
+ maps = build_walk_maps(g)
+ assert "src/a.py::run" not in maps.callers
+ assert "tests/test_a.py" not in maps.imports.importers
+ assert build_walk_maps(None).files == frozenset()
+
+
+# ---------------------------------------------------------------------------
+# reaching_by_tier: call, then import, then the helper or barrel hop
+# ---------------------------------------------------------------------------
+
+
+def test_the_call_tier_answers_first_with_hops() -> None:
+ g = _graph(
+ {"tests/test_a.py": True, "tests/test_imp.py": True, "src/a.py": False},
+ [*_call("tests/test_a.py", "src/a.py"), ("tests/test_imp.py", "src/a.py", "imports")],
+ )
+ found = reaching_by_tier(build_walk_maps(g), ["src/a.py"])
+ assert found["src/a.py"].via == "call-graph"
+ assert found["src/a.py"].tests == ["tests/test_a.py"]
+ assert found["src/a.py"].reach == {"tests/test_a.py": ReachDistance(1, 1)}
+
+
+def test_a_seed_moves_one_hop_per_level_however_many_seeds_share_the_walk() -> None:
+ """a <- b <- c <- d <- test: b is 3 calls from the test, a is 4, past the depth.
+
+ Walked together, b's symbol sits in the first level with a's, so a seed
+ must not ride through it within that level and arrive a hop early.
+ """
+ chain = ["src/a.py", "src/b.py", "src/c.py", "src/d.py"]
+ edges = [(p, f"{p}::f", "defines") for p in chain]
+ edges += [(f"{chain[i + 1]}::f", f"{chain[i]}::f", "calls") for i in range(3)]
+ edges += [("tests/test_d.py", "tests/test_d.py::t", "defines")]
+ edges += [("tests/test_d.py::t", "src/d.py::f", "calls")]
+ g = _graph({"tests/test_d.py": True, **dict.fromkeys(chain, False)}, edges)
+ maps = build_walk_maps(g)
+ together = reaching_by_tier(maps, ["src/a.py", "src/b.py"], import_depth=0, helpers=False)
+ assert "src/a.py" not in together
+ assert together["src/b.py"].reach == {"tests/test_d.py": ReachDistance(3, 1)}
+ alone = reaching_by_tier(maps, ["src/b.py"], import_depth=0, helpers=False)
+ assert alone == {"src/b.py": together["src/b.py"]}
+
+
+def test_the_import_tier_answers_where_calls_are_silent() -> None:
+ g = _graph(
+ {"tests/test_imp.py": True, "src/a.py": False},
+ [("tests/test_imp.py", "src/a.py", "imports")],
+ )
+ found = reaching_by_tier(build_walk_maps(g), ["src/a.py"])
+ assert found["src/a.py"].via == "import-graph"
+ assert found["src/a.py"].entries == {"tests/test_imp.py": frozenset({"src/a.py"})}
+
+
+def test_the_helper_hop_passes_through_a_module_only_tests_import() -> None:
+ g = _graph(
+ {"tests/test_a.py": True, "tests/support.py": False, "src/a.py": False},
+ [("tests/test_a.py", "tests/support.py", "imports"), ("tests/support.py", "src/a.py", "imports")],
+ )
+ maps = build_walk_maps(g)
+ assert "src/a.py" not in reaching_by_tier(maps, ["src/a.py"], helpers=False)
+ assert reaching_by_tier(maps, ["src/a.py"])["src/a.py"].tests == ["tests/test_a.py"]
+
+
+def _barrel_graph(n_tests: int) -> nx.MultiDiGraph:
+ """``pkg/mod.py`` is imported only by its package barrel, which every test imports."""
+ tests = [f"tests/test_{i:02}.py" for i in range(n_tests)]
+ return _graph(
+ {
+ **dict.fromkeys(tests, True),
+ "src/pkg/__init__.py": False,
+ "src/pkg/mod.py": False,
+ "src/app.py": False,
+ },
+ [
+ ("src/pkg/__init__.py", "src/pkg/mod.py", "imports"),
+ ("src/app.py", "src/pkg/__init__.py", "imports"),
+ *[(t, "src/pkg/__init__.py", "imports") for t in tests],
+ ],
+ )
+
+
+def test_a_module_behind_its_package_barrel_is_reached_not_unreached() -> None:
+ """One import hop sees only the barrel; the barrel hop reaches every test."""
+ maps = build_walk_maps(_barrel_graph(77))
+ assert "src/pkg/mod.py" not in reaching_by_tier(maps, ["src/pkg/mod.py"], helpers=False)
+ row = reach_states(maps, ["src/pkg/mod.py"])["src/pkg/mod.py"]
+ assert row.state == "reached"
+ assert row.reached.total == 77
+
+
+# ---------------------------------------------------------------------------
+# reach_states: the negative claim, and the states beside it
+# ---------------------------------------------------------------------------
+
+
+def test_unreached_needs_the_whole_import_closure_to_miss() -> None:
+ g = _graph(
+ {
+ "tests/test_top.py": True,
+ "src/top.py": False,
+ "src/mid.py": False,
+ "src/leaf.py": False,
+ "src/alone.py": False,
+ },
+ [
+ ("tests/test_top.py", "src/top.py", "imports"),
+ ("src/top.py", "src/mid.py", "imports"),
+ ("src/mid.py", "src/leaf.py", "imports"),
+ # Source imports top.py too, so the helper hop does not pass through it.
+ ("src/alone.py", "src/top.py", "imports"),
+ ],
+ )
+ maps = build_walk_maps(g)
+ assert "src/leaf.py" not in reaching_by_tier(maps, ["src/leaf.py"])
+ rows = reach_states(maps, ["src/leaf.py", "src/alone.py"])
+ assert rows["src/leaf.py"].state == "reached"
+ assert rows["src/leaf.py"].reached.entries == {"tests/test_top.py": frozenset({"src/top.py"})}
+ assert rows["src/alone.py"].state == "unreached"
+
+
+def test_a_closure_route_through_another_target_still_reaches() -> None:
+ g = _graph(
+ {
+ "tests/test_top.py": True,
+ "src/top.py": False,
+ "src/mid.py": False,
+ "src/leaf.py": False,
+ "src/alone.py": False,
+ },
+ [
+ ("tests/test_top.py", "src/top.py", "imports"),
+ ("src/top.py", "src/mid.py", "imports"),
+ ("src/mid.py", "src/leaf.py", "imports"),
+ ("src/alone.py", "src/top.py", "imports"),
+ ],
+ )
+ maps = build_walk_maps(g)
+ alone = reach_states(maps, ["src/leaf.py"])["src/leaf.py"].state
+ together = reach_states(maps, ["src/mid.py", "src/leaf.py"])
+ assert alone == together["src/leaf.py"].state == "reached"
+ assert together["src/mid.py"].state == "reached"
+
+
+def test_more_targets_than_one_closure_carries_are_walked_in_batches(monkeypatch) -> None:
+ from repowise.core.analysis.test_reach import walk
+
+ monkeypatch.setattr(walk, "CLOSURE_SEED_CAP", 1)
+ g = _graph(
+ {"tests/test_top.py": True, "src/top.py": False, "src/a.py": False, "src/b.py": False},
+ [
+ ("tests/test_top.py", "src/top.py", "imports"),
+ ("src/top.py", "src/mid.py", "imports"),
+ ("src/mid.py", "src/a.py", "imports"),
+ ("src/mid.py", "src/b.py", "imports"),
+ ("src/a.py", "src/top.py", "imports"),
+ ],
+ )
+ rows = reach_states(build_walk_maps(g), ["src/a.py", "src/b.py"])
+ assert {p: r.state for p, r in rows.items()} == {"src/a.py": "reached", "src/b.py": "reached"}
+
+
+def test_a_file_the_index_lacks_is_new_and_an_index_without_tests_says_nothing() -> None:
+ maps = build_walk_maps(_graph({"src/a.py": False}, []))
+ rows = reach_states(maps, ["src/a.py", "src/added.py"])
+ assert rows["src/a.py"].state == "no_signal"
+ assert rows["src/added.py"].state == "new"
+
+
+def test_a_types_only_file_is_type_only_whatever_reaches_it() -> None:
+ maps = build_walk_maps(_graph({"src/types.py": False, "tests/test_x.py": True}, []))
+ content = "from typing import Protocol\n\nclass Store(Protocol):\n def get(self) -> int: ...\n"
+ rows = reach_states(maps, ["src/types.py"], contents={"src/types.py": content})
+ assert rows["src/types.py"].state == "type_only"
+
+
+def test_a_test_the_change_touches_is_evidence_for_the_file_it_names_or_imports() -> None:
+ g = _graph(
+ {"tests/test_other.py": True, "tests/test_imports.py": True, "src/a.py": False, "src/b.py": False},
+ [("tests/test_imports.py", "src/b.py", "imports")],
+ )
+ maps = build_walk_maps(g)
+ rows = reach_states(
+ maps,
+ ["src/a.py", "src/b.py", "src/new.py"],
+ # test_a.py is new to the index; conftest.py is support, never evidence.
+ changed_tests=["tests/test_a.py", "tests/test_imports.py", "tests/conftest.py", "tests/test_new.py"],
+ )
+ assert rows["src/a.py"].reached.via == "changed-test"
+ assert rows["src/a.py"].reached.tests == ["tests/test_a.py"]
+ assert rows["src/b.py"].reached.tests == ["tests/test_imports.py"]
+ # A new file with a new test named for it is reached, not new.
+ assert rows["src/new.py"].reached.tests == ["tests/test_new.py"]
+ untouched = reach_states(maps, ["src/a.py"])["src/a.py"]
+ assert untouched.state == "unreached"
+
+
+def test_inline_tests_reach_their_own_file() -> None:
+ g = _graph(
+ {"src/lib.rs": False, "pkg/store.go": False, "pkg/store_test.go": True, "other/x.go": False},
+ [],
+ )
+ rows = reach_states(
+ build_walk_maps(g),
+ ["src/lib.rs", "pkg/store.go", "other/x.go"],
+ contents={"src/lib.rs": "pub fn f() {}\n\n#[cfg(test)]\nmod tests {}\n"},
+ )
+ assert rows["src/lib.rs"].reached.via == "inline"
+ assert rows["src/lib.rs"].reached.tests == ["src/lib.rs"]
+ assert rows["pkg/store.go"].reached.tests == ["pkg/store_test.go"]
+ assert rows["other/x.go"].state == "unreached"
+
+
+# ---------------------------------------------------------------------------
+# The pure helpers
+# ---------------------------------------------------------------------------
+
+
+def test_rust_inline_tests_are_read_from_the_attribute_not_a_comment() -> None:
+ assert rust_has_inline_tests(b"#[cfg(test)]\nmod tests {}\n")
+ assert rust_has_inline_tests(" #[ cfg( test ) ]\nmod t {}")
+ assert not rust_has_inline_tests("// #[cfg(test)]\nfn f() {}\n")
+ assert not rust_has_inline_tests('#[cfg(feature = "test")]\n')
+
+
+def test_go_inline_tests_are_the_test_files_of_the_same_directory() -> None:
+ tests = ["pkg/a_test.go", "pkg/sub/b_test.go", "pkg/c.go"]
+ assert go_inline_tests("pkg/a.go", tests) == ["pkg/a_test.go"]
+ assert go_inline_tests("pkg/a_test.go", tests) == []
+ assert go_inline_tests("pkg/a.py", tests) == []
+
+
+ ("path", "content", "expected"),
+ [
+ ("src/api.d.ts", "export const x: number;", True),
+ ("src/t.ts", "import type {A} from './a';\nexport type B = A;\nexport interface C {}\n", True),
+ ("src/t.ts", "import {A} from './a';\ndeclare const v: A;\nexport type {Q} from './q';\n", True),
+ ("src/t.ts", "export type B = string;\nexport const k = 1;\n", False),
+ ("src/t.ts", "import './polyfill';\nexport type B = string;\n", False),
+ ("src/t.ts", "export enum E { a }\n", False),
+ ("src/t.ts", "// nothing\n", False),
+ ("src/t.tsx", "export interface Props { a: string }\n", True),
+ (
+ "src/types.py",
+ '"""Types."""\nfrom typing import TypedDict, TypeAlias, TypeVar\n'
+ "class Row(TypedDict):\n a: int\n"
+ "Rows = list[Row]\nId: TypeAlias = int\nT = TypeVar('T')\nMaybe = Row | None\n"
+ "__all__ = ['Row']\n",
+ True,
+ ),
+ ("src/types.py", "from typing import Protocol\nclass P(Protocol):\n x: int\nLIMIT = 3\n", False),
+ ("src/types.py", "def f():\n return 1\n", False),
+ ("src/types.py", "class P(Protocol\n", False),
+ ("src/c.go", "package c\n", False),
+ ],
+)
+def test_type_only_files(path: str, content: str, expected: bool) -> None:
+ assert is_type_only(path, content) is expected
+
+
+def test_a_test_file_is_never_type_only() -> None:
+ assert not is_type_only("tests/test_types.py", "from typing import TypedDict\nclass R(TypedDict): ...\n")
+ assert not is_type_only("src/types.py", "class R(TypedDict): ...\n", is_test=True)
+
+
+def test_runnable_tests_stand_on_the_stored_flag_and_leave_support_out() -> None:
+ assert is_runnable_test("tests/test_a.py")
+ assert is_runnable_test("tests/test_a.py", is_test=True)
+ assert not is_runnable_test("tests/test_a.py", is_test=False)
+ assert not is_runnable_test("tests/conftest.py", is_test=True)
+ assert not is_runnable_test("tests/fixtures/data.py", is_test=True)
diff --git a/tests/unit/persistence/test_actions_loader.py b/tests/unit/persistence/test_actions_loader.py
index 2e03e0e34..d2e15b436 100644
--- a/tests/unit/persistence/test_actions_loader.py
+++ b/tests/unit/persistence/test_actions_loader.py
@@ -353,7 +353,8 @@ async def test_a_test_reaching_through_a_helper_turns_add_tests_into_simplify(as
assert (core.tests_reaching, core.tests_reaching_via) == (1, "import-graph")
# Once production code imports the middle file it is a hub, not test
- # support, and the second hop no longer counts.
+ # support, and the second hop no longer counts; the import closure still
+ # does, so the file is not claimed untested.
async_session.add(GraphNode(repository_id=rid, node_id="src/app.py", node_type="file"))
async_session.add(
GraphEdge(repository_id=rid, source_node_id="src/app.py",
@@ -362,7 +363,7 @@ async def test_a_test_reaching_through_a_helper_turns_add_tests_into_simplify(as
await async_session.commit()
clear_test_map_cache()
facts = await load_repo_facts(async_session, rid)
- assert facts.files["src/core.py"].tests_reaching == 0
+ assert facts.files["src/core.py"].tests_reaching == 1
async def test_a_failed_test_map_walk_is_unknown_not_zero(async_session, monkeypatch) -> None:
diff --git a/tests/unit/persistence/test_test_reachability_walk.py b/tests/unit/persistence/test_test_reachability_walk.py
index 54ad70e12..119ea2718 100644
--- a/tests/unit/persistence/test_test_reachability_walk.py
+++ b/tests/unit/persistence/test_test_reachability_walk.py
@@ -699,7 +699,7 @@ _KY_LIKE = {
"import ky from '../src/index';\nimport {pool} from '../src/pool';\n"
"ky('https://example.com');\n"
),
- # Source imports pool.ts too and it is no barrel, so it passes nothing on.
+ # Source imports pool.ts too and it is no barrel, so the helper hop stops there.
"src/pool.ts": "import {retry} from './retry';\nexport const pool = [retry];\n",
"src/retry.ts": "export const retry = 1;\n",
"src/agent.ts": "import {pool} from './pool';\nexport const agent = pool;\n",
@@ -723,11 +723,13 @@ async def test_any_reach_passes_through_the_package_entry_tests_import(tmp_path,
found = await any_tests_reaching(async_session, repo.id, ["src/core/Ky.ts", "src/retry.ts"])
assert found["src/core/Ky.ts"].tests == ["test/x.test.ts"]
assert found["src/core/Ky.ts"].via == "import-graph"
- assert "src/retry.ts" not in found
+ # The helper hop stops at pool.ts (source imports it too), but the import
+ # closure does not: a test importing pool.ts runs retry.ts, so it is not unreached.
+ assert found["src/retry.ts"].entries == {"test/x.test.ts": frozenset({"src/pool.ts"})}
async def test_reached_files_are_walked_once_per_index_state(async_session, monkeypatch):
- import repowise.core.analysis.test_reachability as tr
+ import repowise.core.analysis.test_reach.db as tr
repo = await insert_repo(async_session)
await _seed(
@@ -783,7 +785,7 @@ async def test_reads_over_more_files_than_one_statement_can_bind(async_session):
async def test_a_node_listed_twice_across_batches_reads_its_edges_once(async_session):
"""A repeated input landing in two batches must not repeat its pairs: one
``IN`` list matches a node once however often it is listed."""
- from repowise.core.analysis.test_reachability import (
+ from repowise.core.analysis.test_reach.db import (
_IN_BATCH,
_edges_from,
_edges_into,
@@ -815,7 +817,7 @@ async def test_the_capped_list_keeps_the_closest_tests_and_a_changed_test(
):
"""Ranked before the cap: a direct caller survives many path-earlier far ones,
and a test that is itself a target leads."""
- from repowise.core.analysis import test_reachability as tr
+ from repowise.core.analysis.test_reach import model as tr
monkeypatch.setattr(tr, "MAX_TESTS_PER_TARGET", 2)
far = [f"tests/a/test_far_{i}.py" for i in range(3)]
@@ -847,7 +849,7 @@ async def test_the_capped_list_keeps_the_closest_tests_and_a_changed_test(
async def test_the_import_tier_keeps_a_direct_importer_before_the_cap(async_session, monkeypatch):
- from repowise.core.analysis import test_reachability as tr
+ from repowise.core.analysis.test_reach import model as tr
monkeypatch.setattr(tr, "MAX_TESTS_PER_TARGET", 1)
repo = await insert_repo(async_session)
@@ -869,3 +871,54 @@ async def test_the_import_tier_keeps_a_direct_importer_before_the_cap(async_sess
found = await by_tier(async_session, repo.id, ["src/core.py"], call_depth=0, import_depth=2)
assert found["src/core.py"].tests == ["tests/z/test_direct.py"]
assert found["src/core.py"].total == 2
+
+
+_PARITY = {
+ **_KY_LIKE,
+ "app/__init__.py": "from .core import run\n",
+ "app/core.py": "from .util import helper\n\ndef run():\n return helper()\n",
+ "app/util.py": "def helper():\n return 1\n",
+ "app/deep.py": "from app.util import helper\n\ndef deep():\n return helper()\n",
+ "app/lonely.py": "def alone():\n return 2\n",
+ "tests/test_core.py": "from app.core import run\n\ndef test_run():\n assert run() == 1\n",
+ "tests/test_pkg.py": "import app\n\ndef test_pkg():\n assert app.run()\n",
+}
+
+
+async def test_the_stored_walk_and_the_in_memory_walk_return_the_same_rows(tmp_path, async_session):
+ """One walk, two graph sources: the rows must match for every file."""
+ from repowise.core.analysis.test_reachability import (
+ build_walk_maps,
+ clear_test_map_cache,
+ load_test_files,
+ reach_states,
+ reach_states_from_db,
+ reaching_by_tier,
+ )
+ from repowise.core.persistence.crud.graph import batch_upsert_graph_edges
+ from repowise.core.pipeline.persist import _edge_row, persist_graph_nodes
+ from tests.unit.analysis.test_public_api_flag import build_fixture
+
+ builder = build_fixture(tmp_path, _PARITY)
+ repo = await insert_repo(async_session)
+ await persist_graph_nodes(async_session, repo.id, builder, ep_scores={})
+ edges = [_edge_row(u, v, d) for u, v, d in builder.graph().edges(data=True)]
+ await batch_upsert_graph_edges(async_session, repo.id, edges)
+ clear_test_map_cache()
+
+ maps = build_walk_maps(builder.graph())
+ assert maps.test_files == await load_test_files(async_session, repo.id)
+ paths = sorted(_PARITY)
+ stored = await by_tier(async_session, repo.id, paths)
+ assert stored == reaching_by_tier(maps, paths, helpers=False)
+ assert stored # the fixture exercises the walk at all
+ asked = [*paths, "app/added.py", "app/types.py"]
+ evidence = {
+ "changed_tests": ["tests/test_lonely.py"],
+ "contents": {"app/types.py": "from typing import TypedDict\nclass R(TypedDict):\n a: int\n"},
+ }
+ in_memory = reach_states(maps, asked, **evidence)
+ assert await reach_states_from_db(async_session, repo.id, asked, **evidence) == in_memory
+ assert in_memory["app/lonely.py"].reached.via == "changed-test"
+ assert in_memory["app/types.py"].state == "type_only"
+ assert {in_memory[p].state for p in in_memory} >= {"reached", "unreached", "new"}
diff --git a/tests/unit/server/test_health_coverage_basis.py b/tests/unit/server/test_health_coverage_basis.py
index f5378f089..984877e85 100644
--- a/tests/unit/server/test_health_coverage_basis.py
+++ b/tests/unit/server/test_health_coverage_basis.py
@@ -560,9 +560,9 @@ async def test_the_file_endpoint_says_when_the_list_was_cut(
the evidence. Measured on this repository, one file is reached by 124 tests
and would have rendered as 50.
"""
- import repowise.core.analysis.test_reachability as tr
+ from repowise.core.analysis.test_reach import model
- monkeypatch.setattr(tr, "MAX_TESTS_PER_TARGET", 2)
+ monkeypatch.setattr(model, "MAX_TESTS_PER_TARGET", 2)
repo = await create_test_repo(client, tmp_path)
tests = [f"tests/test_{i}.py" for i in range(5)]
edges = [e for t in tests for e in _calls(t, "src/a.py")]