Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
feat(health): let a called function's assertions reach its caller (#2399)
Raghav Chamadiyac1a3756c17e5
diff --git a/docs/layers/CODE_HEALTH.md b/docs/layers/CODE_HEALTH.mdindex f9a81b35c..3fc2cf78a 100644--- a/docs/layers/CODE_HEALTH.md+++ b/docs/layers/CODE_HEALTH.md@@ -454,9 +454,15 @@ whether an assertion observes a double or production output, which is a dataflowquestion the pass does not ask — and on the TypeScript sample that one questionaccounted for every false positive.-`assertion_free_test` measures **52%** on TypeScript and **53%** on Python, and-does not report on Go or Java at all; the per-language reasoning is in-[LANGUAGE_SUPPORT.md](LANGUAGE_SUPPORT.md#code-health-coverage).+`assertion_free_test` measures **51%** on TypeScript (51 findings, the complete+population of two corpora) and **86%** on Python (29 hand-labelled, a systematic+sample of 172), and does not report on Go or Java at all; the per-language+reasoning is in+[LANGUAGE_SUPPORT.md](LANGUAGE_SUPPORT.md#code-health-coverage). The gap between+the two is one fact: a test that delegates its oracle to a helper in the same+file is resolved, and a JS/TS suite keeps its helpers in another file. The+marker stays advisory on both, and twenty-nine items are too few to settle+whether Python clears the bar.It asks a question with a yes-or-no answer rather than athreshold, which is why it can be stated plainly: a test case whose assertiondiff --git a/docs/layers/LANGUAGE_SUPPORT.md b/docs/layers/LANGUAGE_SUPPORT.mdindex 6d8eb8ec9..4ae153b9c 100644--- a/docs/layers/LANGUAGE_SUPPORT.md+++ b/docs/layers/LANGUAGE_SUPPORT.md@@ -441,11 +441,29 @@ for Java. A language with no row classifies nothing. Go and Java are classifiedand counted, which is how their precision below was measured, but the markerdoes not report on them — the shipping set is a constant in the detector.-Measured precision, hand-labelled: **52%** on TypeScript (52 findings, the-complete population of two corpora) and **53%** on Python (30, a systematic-sample of 194). Both are below the 70% house bar and above the 40% at which-`mock_saturated_test` shipped TypeScript, and they ship on the same line: the-false positives are one family the marker declares, described below.+Measured precision, hand-labelled: **51%** on TypeScript (51 findings, the+complete population of two corpora) and **86%** on Python (29 findings, a+systematic sample of 172). The marker ships advisory on both, because the bar+below is a property of the marker rather than of one language, and TypeScript+is at 51%.++Twenty-nine items cannot settle a 70% bar: 86% on that sample carries a 95%+interval of roughly 68% to 96%, whose lower bound sits below the bar it appears+to clear. The reading is consistent with clearing it and does not demonstrate+it. What moved Python is that a test's oracle now reaches it from a function it+calls in the same file, described below. The same change barely moved+TypeScript, for a reason worth recording: a JS/TS suite keeps its helpers in a+separate module, so 19 of its 25 remaining false positives delegate across a+file boundary rather than within one.++An earlier pass published 53% for Python. Re-labelling the pre-change findings+of the same corpus, drawn the same way and read against this pass's rubric,+gives 68.8% (22 of 32), so the gain below is that 68.8 to 86.2 and the 53% is+superseded rather than contradicted. The two passes are not reconciled: 15+points of the difference is labelling, on samples of about thirty, and the+rubric behind the earlier number was not recorded. TypeScript read 52% then and+51% here, which is the evidence that the two rubrics are close and Python's gap+is sample noise rather than a changed standard.A **mock verification counts as an assertion for this marker and not for the oneabove**, which is the deliberate inversion described in CODE_HEALTH.md. It@@ -491,8 +509,41 @@ from the test that calls it:asserts on the test's behalf. Hand-labelled at **33%** over 18 findings, and 10of its 12 false positives were this family.-Separating them needs the assertions of a called function to reach its caller,-which is a cross-function question this pass does not ask.+Separating them needs the assertions of a called function to reach its caller.+Within a single file that now happens. Neither language is unblocked by it: Go+hands `*testing.T` to a package-level helper and Java inherits a base-class+method, and both live in another file, so re-admitting either is a measurement+of its own.++**A called function's assertions reach its caller, inside one file.** The+assertion count is per function, so a test that hands its checks to a helper+beside it read as checking nothing. That was the largest false-positive family+in every language the marker reports on. The walk now records the names each+function calls, and the marker resolves them against same-file functions that+assert. It counts nothing and no calibrated marker reads it.++What it suppresses, by corpus:++| Corpus | Findings | Test cases | Rate |+|---|---|---|---|+| 2,012-file Python | 763 to 501 | 17,966 | 4.2% to 2.8% |+| 794-file Python | 194 to 172 | 5,212 | 3.7% to 3.3% |+| 374-file TypeScript | 35 to 33 | 7,354 | 0.5% to 0.4% |+| 562-file TypeScript | 18 to 18 | 4,761 | 0.4% to 0.4% |++Six suppressions were read by hand across the two Python corpora and each is+the declared family: a test whose whole body delegates to a helper that asserts,+or one that parametrises a sibling test by calling it. The resolution matches+names against same-file names and drops the receiver, so a call to an imported+function sharing a local helper's name, or a method on a test-local double+sharing a module-level function's name, suppresses wrongly. Both hide a finding+rather than invent one. No instance turned up in the corpora above, and the+shape is pinned by a test.++What it deliberately does not reach is anything needing the call graph, which+this pass does not consult: `super().test_x(...)`, a package-level Go helper+handed the test handle, and the shared JS/TS fixture module that is most of+TypeScript's remaining error.**A `.tsx` file is read with the JSX grammar.** It arrives tagged `typescript`,and the grammar that tag selects fails on the first `<Component />`; everythingdiff --git a/packages/core/src/repowise/core/analysis/health/biomarkers/assertion_free_test.py b/packages/core/src/repowise/core/analysis/health/biomarkers/assertion_free_test.pyindex fbb311177..075276b06 100644--- a/packages/core/src/repowise/core/analysis/health/biomarkers/assertion_free_test.py+++ b/packages/core/src/repowise/core/analysis/health/biomarkers/assertion_free_test.py@@ -33,10 +33,11 @@ from .base import BiomarkerResult, FileContext#: Languages the marker reports on. Go and Java are classified by#: ``complexity/test_case.py`` and counted like any other, and are deliberately-#: absent here: both conventionally hand the oracle to a helper the test calls,-#: and the assertion count is per function, so the helper's assertions are-#: invisible from its caller. On Go that is not a false-positive family the-#: marker could declare and live with, it is nearly the whole population.+#: absent here: both conventionally hand the oracle to a helper the test calls.+#: ``_asserting_names`` now resolves that within a file, but the idiom reaches+#: across one in both languages, as a package-level Go helper handed ``*testing+#: .T`` and a Java base-class method. Re-admitting either is a measurement of+#: its own, not a consequence of this one.#: Figures and the reasoning: LANGUAGE_SUPPORT.md#code-health-coverage.SHIPPING_LANGUAGES = frozenset({"javascript", "python", "tsx", "typescript"})@@ -53,6 +54,7 @@ class AssertionFreeTestDetector:out: list[BiomarkerResult] = []# ``all_functions``, not ``function_metrics``: name-keying collapses a# file's anonymous ``it`` callbacks into one row. See ``FileContext``.+ oracles = _asserting_names(ctx)for fn in ctx.all_functions:# ``is_test_case`` is the whole gate. Every other test-quality# marker can lean on "it has assertions, so it is a test"; this one@@ -61,6 +63,8 @@ class AssertionFreeTestDetector:continueif fn.assertion_count or fn.verification_count:continue+ if fn.called_names & oracles:+ continueout.append(BiomarkerResult(biomarker_type=self.name,@@ -75,4 +79,32 @@ class AssertionFreeTestDetector:return out+def _asserting_names(ctx: FileContext) -> frozenset[str]:+ """Names of functions in this file that do check something, lowercased.++ A test whose own assertion count is zero has still checked something if it+ handed the job to a helper that asserts, and the count is per function, so+ the helper's assertions are otherwise invisible from the test that calls+ it. That is the largest false-positive family this marker has in every+ language it reports on.++ Names only, and the whole file is one namespace: the receiver is dropped,+ so a helper on one class suppresses a same-named helper on another, and a+ method on a test-local double suppresses against a module-level function of+ that name. Both directions hide a real finding rather than invent one,+ which is the tolerable way round for an advisory marker, but neither is+ free. A helper nested inside the test body is not collected as a function+ at all, so it suppresses nothing.++ Same file only. ``super().test_x(...)``, a package-level Go helper and a+ shared JS fixture all live elsewhere and would need the call graph, which+ this pass does not consult, so those stay false positives.+ """+ return frozenset(+ fn.name.lower()+ for fn in ctx.all_functions+ if fn.assertion_count or fn.verification_count+ )++BIOMARKER = AssertionFreeTestDetector()diff --git a/packages/core/src/repowise/core/analysis/health/complexity/assertions.py b/packages/core/src/repowise/core/analysis/health/complexity/assertions.pyindex 8e093d0b8..197431e2f 100644--- a/packages/core/src/repowise/core/analysis/health/complexity/assertions.py+++ b/packages/core/src/repowise/core/analysis/health/complexity/assertions.py@@ -263,8 +263,8 @@ def _is_assertion_statement(def _collect_assertion_facts(body_node: Node, lmap: LanguageNodeMap, dialect: AssertDialect | None = None-) -> tuple[list[tuple[int, int, int]], int, int]:- """``(blocks, total, verifications)`` assertion facts for one function body.+) -> tuple[list[tuple[int, int, int]], int, int, frozenset[str]]:+ """``(blocks, total, verifications, called)`` facts for one function body.*blocks* are runs of ≥2 consecutive **narrow-tier** assertion statements,each recorded as ``(start_line, end_line, count)``. Runs are found per@@ -284,12 +284,22 @@ def _collect_assertion_facts(*verifications* counts mock verifications, on the same block-level gate as*total* and in neither of the other two counts. See the module docstring.++ *called* is every name called directly in this body, lowercased, excluding+ nested function bodies on the same rule the assertion scan uses. It rides+ along on this traversal because the traversal already reaches every call+ node; collecting it separately would mean walking every function body+ twice. It counts nothing and so cannot move a calibrated marker. Empty+ under the early return above, which is fine only because the one reader+ ships no language that takes it."""if not lmap.assert_kinds and not lmap.assert_call_kinds:- return [], 0, 0+ return [], 0, 0, frozenset()blocks: list[tuple[int, int, int]] = []total = 0verifications = 0+ called: set[str] = set()+ call_kinds = lmap.call_kinds or lmap.assert_call_kindsdef _scan_siblings(parent: Node, *, count_total: bool) -> None:nonlocal total, verifications@@ -328,6 +338,10 @@ def _collect_assertion_facts(or node.type in lmap.lambda_kinds)_scan_siblings(node, count_total=counts_here)+ if node.type in call_kinds:+ names = _callee_names(node)+ if names is not None:+ called.add(names[0])for child in node.children:if child.type in lmap.function_kinds:continue # nested fn, not collected as its own entry either@@ -347,4 +361,4 @@ def _collect_assertion_facts(verifications += 1elif tier != _NOT_ASSERTION:total += 1- return blocks, total, verifications+ return blocks, total, verifications, frozenset(called)diff --git a/packages/core/src/repowise/core/analysis/health/complexity/models.py b/packages/core/src/repowise/core/analysis/health/complexity/models.pyindex 993301a4f..d6973fbf4 100644--- a/packages/core/src/repowise/core/analysis/health/complexity/models.py+++ b/packages/core/src/repowise/core/analysis/health/complexity/models.py@@ -61,6 +61,11 @@ class FunctionComplexity:# True when this function is a test case its framework would run, as# opposed to a helper or fixture beside it. ``complexity/test_case.py``.is_test_case: bool = False+ # Every name called directly in this function's own body, lowercased,+ # excluding nested function bodies. Empty for a language with no assertion+ # vocabulary row, because it rides on that traversal. It answers one+ # question: did this function hand its work to something else in the file?+ called_names: frozenset[str] = frozenset()def __post_init__(self) -> None:if self.complex_conditions is None:diff --git a/packages/core/src/repowise/core/analysis/health/complexity/walker.py b/packages/core/src/repowise/core/analysis/health/complexity/walker.pyindex 522730b64..7640885f1 100644--- a/packages/core/src/repowise/core/analysis/health/complexity/walker.py+++ b/packages/core/src/repowise/core/analysis/health/complexity/walker.py@@ -161,7 +161,7 @@ def walk_file(for fn_node in _collect_function_nodes(tree.root_node, lmap):body = fn_node.child_by_field_name("body") or fn_nodeccn, max_nest, cognitive, bumps, conditions = _walk_function_body(body, lmap)- assertion_blocks, assertion_count, verifications = _collect_assertion_facts(+ assertion_blocks, assertion_count, verifications, called = _collect_assertion_facts(body, lmap, asserts)name = _find_function_entry_name(fn_node, lmap)@@ -181,6 +181,7 @@ def walk_file(verification_count=verifications,mock_setup_count=_count_mock_setup(fn_node, body, lmap, mock_dialect, asserts),is_test_case=is_test_case(fn_node, name, language),+ called_names=called,)functions.append(fc)fc_by_node_id[fn_node.id] = fcdiff --git a/packages/core/src/repowise/core/analysis/health/engine.py b/packages/core/src/repowise/core/analysis/health/engine.pyindex e6013779c..5f9529b57 100644--- a/packages/core/src/repowise/core/analysis/health/engine.py+++ b/packages/core/src/repowise/core/analysis/health/engine.py@@ -100,7 +100,13 @@ log = structlog.get_logger(__name__)# Not a licence to move a calibrated scoring weight — those are frozen# independently of this stamp.#-# Current stamp: the health pass reads a ``.tsx`` file with the JSX grammar.+# Current stamp: the walker records one new ``FunctionComplexity`` field,+# ``called_names``, which a cached v16 walk does not carry at all. It feeds the+# advisory ``assertion_free_test``, which no longer calls a test assertion-free+# when it handed its checks to a function in the same file that asserts. It+# counts nothing, so no calibrated marker reads it and no score moves.+#+# v16: the health pass reads a ``.tsx`` file with the JSX grammar.# It arrives tagged ``typescript``, and the grammar that tag selects errors on# the first ``<Component />``, so a cached v15 walk stored an ERROR-recovered# tree. Three things stored under it change for ``.tsx`` files and nothing@@ -156,7 +162,7 @@ log = structlog.get_logger(__name__)# forms. Files that were counted untested and are not become tested, which# moves untested-hotspot findings and the scores that carry them, on every# language with a prefix or spec convention rather than Ruby alone.-HEALTH_ANALYZER_VERSION = 16+HEALTH_ANALYZER_VERSION = 17# Method-level smells that make the dataflow / Extract Method pass worthwhile.# Only files carrying one of these get a CFG + def/use + reaching pass built.diff --git a/tests/unit/health/test_assertion_free_test.py b/tests/unit/health/test_assertion_free_test.pyindex 48bee2f8f..c4715aef4 100644--- a/tests/unit/health/test_assertion_free_test.py+++ b/tests/unit/health/test_assertion_free_test.py@@ -448,3 +448,104 @@ def test_go_and_java_are_classified_but_never_reported() -> None:assert _counts(java, "src/test/java/FooTest.java", "java")["bare"][2] is Trueassert _flagged(go, "pkg/thing_test.go", "go") == []assert _flagged(java, "src/test/java/FooTest.java", "java") == []+++def test_a_delegated_oracle_in_the_same_file_counts() -> None:+ _require("python")+ # The largest false-positive family: the test hands its checks to a helper+ # beside it, and the assertion count is per function.+ source = (+ "def _check_user(u):\n"+ " assert u.name\n"+ " assert u.age > 0\n"+ "\n"+ "def test_delegates():\n"+ " _check_user(build_user())\n"+ "\n"+ "def test_bare():\n"+ " build_user()\n"+ )+ assert _flagged(source) == ["test_bare"]+++def test_a_delegated_oracle_through_a_receiver_counts() -> None:+ _require("python")+ # Same shape stated as a method call. The callee name is what resolves,+ # never the receiver.+ source = (+ "class TestUser:\n"+ " def _check(self, u):\n"+ " assert u.name\n"+ "\n"+ " def test_delegates(self):\n"+ " self._check(build_user())\n"+ "\n"+ " def test_bare(self):\n"+ " build_user()\n"+ )+ assert _flagged(source) == ["test_bare"]+++def test_a_helper_that_asserts_nothing_suppresses_nothing() -> None:+ _require("python")+ # The call is not the point; the callee having an oracle is.+ source = (+ "def _build(u):\n"+ " return User(u)\n"+ "\n"+ "def test_calls_a_plain_helper():\n"+ " _build(1)\n"+ )+ assert _flagged(source) == ["test_calls_a_plain_helper"]+++def test_a_helper_in_another_file_is_not_resolved() -> None:+ _require("python")+ # Resolution is intra-file by construction: this pass has no call graph, so+ # an imported helper stays a false positive rather than a guess.+ source = "from .helpers import check_user\n\ndef test_imported():\n check_user(1)\n"+ assert _flagged(source) == ["test_imported"]+++def test_a_delegated_oracle_counts_for_a_jest_callback() -> None:+ _require("typescript")+ # The helper must not look like an assertion itself, or the counter would+ # have reached it without any of this.+ source = (+ "function renderedOk(r) {\n"+ " expect(r.ok).toBe(true);\n"+ "}\n"+ 'describe("s", () => {\n'+ ' it("delegates", () => { renderedOk(run()); });\n'+ ' it("bare", () => { run(); });\n'+ "});\n"+ )+ assert _flagged(source, "src/x.test.ts", "typescript") == ["it callback"]+ counts = _counts(source, "src/x.test.ts", "typescript")+ assert counts["renderedOk"][0] == 1++++def test_resolution_is_by_name_and_a_second_class_shares_it() -> None:+ _require("python")+ # The limit, pinned so it is a known shape rather than a surprise: the+ # receiver is dropped, so the whole file is one namespace. TestB._check+ # checks nothing, and TestA._check suppresses it. This hides a finding+ # rather than inventing one, which is the tolerable direction, and the fix+ # is a class-aware resolution this pass does not do.+ source = (+ "class TestA:\n"+ " def _check(self, r):\n"+ " assert r == 1\n"+ "\n"+ " def test_a(self):\n"+ " self._check(f())\n"+ "\n"+ "class TestB:\n"+ " def _check(self, r):\n"+ " pass\n"+ "\n"+ " def test_b(self):\n"+ " self._check(g())\n"+ )+ assert _flagged(source) == []diff --git a/tests/unit/health/test_model_version_coupling.py b/tests/unit/health/test_model_version_coupling.pyindex d9ed019a1..6d1a6325f 100644--- a/tests/unit/health/test_model_version_coupling.py+++ b/tests/unit/health/test_model_version_coupling.py@@ -22,7 +22,7 @@ from repowise.core.analysis.health.refactoring.identity import (# The three stamps as they stand together. Moving either identity model means# moving the analyzer with it, and updating this tuple in the same commit.-_STAMPS = (16, 2, 2)+_STAMPS = (17, 2, 2)def test_the_identity_models_and_the_analyzer_stamp_are_pinned_together() -> None: