Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

refactor(parser): split parser.py into a package; per-language import extractors on LanguageConfig (#4042)

Raghav Chamadiya4622beea4876
diff --git a/docs/architecture/ARCHITECTURE.md b/docs/architecture/ARCHITECTURE.md
index ac2a7cbe2..009108ac5 100644
--- a/docs/architecture/ARCHITECTURE.md
+++ b/docs/architecture/ARCHITECTURE.md
@@ -113,7 +113,7 @@ repowise/
│ │ ├── src/repowise/core/
│ │ │ ├── ingestion/
│ │ │ │ ├── traverser.py # file tree walking + gitignore + per-dir .repowiseIgnore + extra exclude patterns
-│ │ │ │ ├── parser.py # ASTParser: one class, all languages; _extract_calls(), _extract_import_bindings()
+│ │ │ │ ├── parser/ # ASTParser: one class, all languages; one module per pass (symbols, imports, calls, references)
│ │ │ │ ├── parsers/ # per-language parser helpers
│ │ │ │ ├── graph.py # NetworkX dep graph builder; add_file(), _resolve_calls(), file_subgraph()
│ │ │ │ ├── call_resolver.py # CallResolver: origin-stamped call resolution
@@ -1594,7 +1594,7 @@ each with the file it registers in and what you lose by skipping it.
Two things worth knowing before you start:
-- **You never edit `parser.py` or `registry.py`.** Language identity goes in
+- **You never edit `parser/` or `registry.py`.** Language identity goes in
`ingestion/languages/specs/<lang>.py` and is slotted into `ALL_SPECS`; parser
configuration goes in `ingestion/language_configs.py`. The one unavoidable
core edit is adding your tag to the `LanguageTag` Literal in
diff --git a/docs/architecture/language-support.md b/docs/architecture/language-support.md
index ad2b9b572..120984335 100644
--- a/docs/architecture/language-support.md
+++ b/docs/architecture/language-support.md
@@ -10,7 +10,7 @@ centralised `LanguageRegistry`; per-language extraction logic lives in
call-resolution strategy lives behind a seam in `call_resolver.py`. Adding a
language means dropping one file into each relevant subpackage and registering it
in that subpackage's dispatcher. No edits are needed to `graph.py`,
-`traverser.py`, or any analysis core file. `parser.py` holds a short per-language
+`traverser.py`, or any analysis core file. The `parser/` package holds a short per-language
block for a handful of languages whose grammar shape the generic path cannot
read (C++ qualified definitions, Dart mixins, Object Pascal out-of-line methods,
[Elixir](#elixir), [F#](#f)); each is a few lines calling into a helper, and a
@@ -151,7 +151,8 @@ ingestion/
django.py python_imports.py node.py dotnet.py
spring.py ruby.py php.py scala.py swift.py c.py cpp.py luau.py go.py jvm.py
sfc_source.py # Multi-language-file (SFC) projection (see below)
- parser.py # ASTParser (language-agnostic orchestration)
+ parser/ # ASTParser (language-agnostic orchestration), one module per pass
+ extractors/imports/ # Per-language import rules (LanguageConfig.import_extractor)
graph.py # GraphBuilder (import/call/heritage resolution)
analysis/
@@ -310,7 +311,9 @@ receiver typing (step 9).
Add a parser configuration to `LANGUAGE_CONFIGS` in
`packages/core/src/repowise/core/ingestion/language_configs.py` (re-exported from
-`parser.py` for back-compat):
+the `parser` package for back-compat). A language whose imports need more than
+the generic shape also sets `import_extractor` to an `ImportExtractor` from
+`extractors/imports/<lang>.py`:
```python
"mylang": LanguageConfig(
@@ -629,8 +632,8 @@ corrupt every unit named after it in the same clause. Its sanitizer
(`prepare_pascal_source` in `ingestion/lang_helpers/source_prep.py`) is gated on `path`'s
extension (that syntax is invalid in a plain `.pas`/`.pp` unit file) and is a
no-op everywhere else, same contract as the `_LOCATORS` path. Registering a
-sanitizer this way, rather than as an if-block in `parser.py`, means it stays
-"zero changes to `parser.py`" for a new language and the health walkers get the
+sanitizer this way, rather than as an if-block in the parser, means it stays
+"zero changes to the parser" for a new language and the health walkers get the
same clean projection the ingestion parser does.
### The locator registry
@@ -777,7 +780,7 @@ type body and a member the way Elixir shares `call` across everything, so its
per-language block is about extending and locating rather than reclassifying:
- **A `let rec f ... and g ...` group is one grammar node for every clause**, so
- `@symbol.def` sits on each clause's left-hand side and `parser.py` extends its
+ `@symbol.def` sits on each clause's left-hand side and the parser extends its
span forward to the next clause (or the file end), stepping over a return-type
annotation sitting between the two. The same left-hand side node also marks a
binding local to another binding's body, which the generic callable-ancestor
diff --git a/docs/reference/COMPUTED_GLOSSARY.md b/docs/reference/COMPUTED_GLOSSARY.md
index d6e03329c..6fb15c7d1 100644
--- a/docs/reference/COMPUTED_GLOSSARY.md
+++ b/docs/reference/COMPUTED_GLOSSARY.md
@@ -11,7 +11,7 @@ workspace overlays, MCP responses, and CLI output.
| Area | Main code paths | What gets computed |
| --- | --- | --- |
-| Traversal and parsing | `packages/core/src/repowise/core/ingestion/traverser.py`, `packages/core/src/repowise/core/ingestion/parser.py`, `packages/core/src/repowise/core/ingestion/models.py` | Files, languages, entry points, symbols, imports, exports, calls, inheritance, parse errors, content hashes |
+| Traversal and parsing | `packages/core/src/repowise/core/ingestion/traverser.py`, `packages/core/src/repowise/core/ingestion/parser/`, `packages/core/src/repowise/core/ingestion/models.py` | Files, languages, entry points, symbols, imports, exports, calls, inheritance, parse errors, content hashes |
| Graph construction | `packages/core/src/repowise/core/ingestion/graph.py`, `call_resolver.py`, `heritage_resolver.py`, `framework_edges.py`, `dynamic_hints/` | File and symbol nodes, import/call/heritage/framework/dynamic/co-change edges, centrality, SCCs, communities, execution flows |
| Git intelligence | `packages/core/src/repowise/core/ingestion/git_indexer/` | Churn, ownership, hotspots, bus factor, co-change partners, significant commits, temporal scores, rename and merge signals |
| Analysis | `packages/core/src/repowise/core/analysis/` | Dead-code findings, decision records, decision staleness, security findings, PR blast radius, execution flows, communities |
@@ -24,7 +24,7 @@ workspace overlays, MCP responses, and CLI output.
| Term | Definition | Computed by | Example |
| --- | --- | --- | --- |
-| Includable source file | A file that survives ignore rules, blocked patterns, size limit, binary detection, generated-file detection, and language detection. | `FileTraverser._build_file_info()` | `packages/core/src/repowise/core/ingestion/parser.py` |
+| Includable source file | A file that survives ignore rules, blocked patterns, size limit, binary detection, generated-file detection, and language detection. | `FileTraverser._build_file_info()` | `packages/core/src/repowise/core/ingestion/traverser.py` |
| `FileInfo` | Per-file metadata used by the parser and graph builder. | `FileTraverser.traverse()` | `{path: "src/app.py", language: "python", is_test: false, is_entry_point: true}` |
| Language tag | Canonical language value from file extension, special filename, or shebang. | `ingestion/models.py`, `traverser.py`, `languages/registry.py` | `python`, `typescript`, `go`, `terraform`, `openapi`, `unknown` |
| Test file flag | Whether a file looks like a test/spec/fixture file. | `FileTraverser._build_file_info()` and community/test-gap helpers | `tests/test_auth.py -> is_test=true` |
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/__init__.py b/packages/core/src/repowise/core/ingestion/extractors/imports/__init__.py
new file mode 100644
index 000000000..de39aceb0
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/__init__.py
@@ -0,0 +1,36 @@
+"""Per-language import extraction, one ``ImportExtractor`` per language.
+
+Each language's rules live in ``<lang>.py``; ``LanguageConfig.import_extractor``
+wires them in, and a language without one takes ``GENERIC_IMPORTS``. This
+package never imports the parser, so ``language_configs`` can import it.
+"""
+
+from ._types import ImportExtractor
+from .dart import DART_IMPORTS
+from .elixir import ELIXIR_IMPORTS
+from .fsharp import FSHARP_IMPORTS
+from .generic import GENERIC_IMPORTS
+from .jvm import JVM_IMPORTS
+from .pascal import PASCAL_IMPORTS
+from .php import PHP_IMPORTS
+from .python import PYTHON_IMPORTS
+from .rust import RUST_IMPORTS
+from .scala import SCALA_IMPORTS
+from .swift import SWIFT_IMPORTS
+from .ts_js import TS_JS_IMPORTS
+
+__all__ = [
+ "DART_IMPORTS",
+ "ELIXIR_IMPORTS",
+ "FSHARP_IMPORTS",
+ "GENERIC_IMPORTS",
+ "JVM_IMPORTS",
+ "PASCAL_IMPORTS",
+ "PHP_IMPORTS",
+ "PYTHON_IMPORTS",
+ "RUST_IMPORTS",
+ "SCALA_IMPORTS",
+ "SWIFT_IMPORTS",
+ "TS_JS_IMPORTS",
+ "ImportExtractor",
+]
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/_types.py b/packages/core/src/repowise/core/ingestion/extractors/imports/_types.py
new file mode 100644
index 000000000..406ae4e46
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/_types.py
@@ -0,0 +1,30 @@
+"""The per-language hooks that turn import query matches into ``Import`` records."""
+
+from __future__ import annotations
+
+from collections.abc import Callable
+from dataclasses import dataclass
+
+from tree_sitter import Node
+
+from ...models import Import
+
+# (stmt_node, module_node, module_text, raw, language, src) -> imports
+StatementFn = Callable[[Node, Node, str, str, str, str], list[Import]]
+
+
+@dataclass(frozen=True)
+class ImportExtractor:
+ """One language's import rules; a ``None`` hook takes the shared default.
+
+ ``statement`` maps one deduplicated statement to its imports.
+ ``deduped`` replaces the shared raw-text dedup for languages that key a
+ statement their own way, given a per-file ``seen`` set; those imports
+ keep the default load kind. ``dedup_key`` refines the raw-text key, and
+ ``finalize`` rewrites the file's whole import list once.
+ """
+
+ statement: StatementFn
+ deduped: Callable[[Node, Node, str, set[str]], list[Import]] | None = None
+ dedup_key: Callable[[Node, str, str], str] | None = None
+ finalize: Callable[[list[Import]], list[Import]] | None = None
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/dart.py b/packages/core/src/repowise/core/ingestion/extractors/imports/dart.py
new file mode 100644
index 000000000..5603a896d
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/dart.py
@@ -0,0 +1,39 @@
+"""Dart ``import`` / ``export`` / ``part`` directives."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ..bindings import extract_import_bindings
+from ._types import ImportExtractor
+
+
+def _dart_import(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, src: str
+) -> Import:
+ """Dart: URIs are relative unless schemed, ``export`` re-exports, and the
+ dotted ``part of library.name;`` form resolves through ``library:``.
+ """
+ module_path = module_text
+ if module_node.type == "dotted_identifier_list":
+ module_path = f"library:{module_text}"
+ imported_names, bindings = extract_import_bindings(stmt_node, src, "dart")
+ return Import(
+ raw_statement=raw,
+ module_path=module_path,
+ imported_names=imported_names,
+ is_relative=not module_path.startswith(("package:", "dart:", "library:")),
+ resolved_file=None,
+ bindings=bindings,
+ is_reexport=stmt_node.type == "library_export",
+ )
+
+
+def _dart_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ return [_dart_import(stmt_node, module_node, module_text, raw, src)]
+
+
+DART_IMPORTS = ImportExtractor(statement=_dart_statement)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/elixir.py b/packages/core/src/repowise/core/ingestion/extractors/imports/elixir.py
new file mode 100644
index 000000000..d2af76f14
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/elixir.py
@@ -0,0 +1,44 @@
+"""Elixir ``alias`` / ``import`` / ``require`` / ``use`` directives."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ..bindings.elixir import elixir_import_modules
+from ..helpers import node_text
+from ._types import ImportExtractor
+from .generic import generic_statement
+
+
+def _elixir_imports(
+ stmt_node: Node, module_node: Node, src: str, seen_modules: set[str]
+) -> list[Import]:
+ """Elixir directives, one Import per module and deduped by module path.
+
+ ``alias Foo.{Bar, Baz}`` names two modules in one statement. ``import``
+ pulls in every public function (``*``); alias/require/use bind the module.
+ """
+ raw = node_text(stmt_node, src).split("\n", 1)[0].strip()
+ directive_node = stmt_node.child_by_field_name("target")
+ directive = node_text(directive_node, src).strip() if directive_node else ""
+ imports: list[Import] = []
+ for module_path in elixir_import_modules(module_node, src):
+ if module_path in seen_modules:
+ continue
+ seen_modules.add(module_path)
+ imports.append(
+ Import(
+ raw_statement=raw,
+ module_path=module_path,
+ imported_names=["*"] if directive == "import" else [],
+ is_relative=False,
+ resolved_file=None,
+ bindings=[],
+ is_reexport=False,
+ )
+ )
+ return imports
+
+
+ELIXIR_IMPORTS = ImportExtractor(statement=generic_statement, deduped=_elixir_imports)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/fsharp.py b/packages/core/src/repowise/core/ingestion/extractors/imports/fsharp.py
new file mode 100644
index 000000000..d7a9ca49e
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/fsharp.py
@@ -0,0 +1,47 @@
+"""F# ``open`` declarations."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ..helpers import node_text
+from ._types import ImportExtractor
+from .generic import generic_statement
+
+
+def _fsharp_imports(
+ stmt_node: Node, module_node: Node, src: str, seen_raws: set[str]
+) -> list[Import]:
+ """F# ``open``: every public name of the module (``*``), which bare-name
+ call resolution reads. ``open type A.B.T`` binds the type's static members,
+ so the module the file depends on is ``A.B``.
+ """
+ raw = node_text(stmt_node, src).strip()
+ if raw in seen_raws:
+ return []
+ seen_raws.add(raw)
+ module_path = node_text(module_node, src).strip()
+ if not module_path:
+ return []
+ names: list[str] = ["*"]
+ if any(child.type == "type" for child in stmt_node.children):
+ head, _, type_name = module_path.rpartition(".")
+ if head:
+ module_path, names = head, [type_name]
+ else:
+ names = []
+ return [
+ Import(
+ raw_statement=raw,
+ module_path=module_path,
+ imported_names=names,
+ is_relative=False,
+ resolved_file=None,
+ bindings=[],
+ is_reexport=False,
+ )
+ ]
+
+
+FSHARP_IMPORTS = ImportExtractor(statement=generic_statement, deduped=_fsharp_imports)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/generic.py b/packages/core/src/repowise/core/ingestion/extractors/imports/generic.py
new file mode 100644
index 000000000..e6671c707
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/generic.py
@@ -0,0 +1,48 @@
+"""The import of a statement no language-specific shape claims."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ..bindings import extract_import_bindings
+from ._types import ImportExtractor
+
+
+def _generic_import(
+ stmt_node: Node,
+ module_text: str,
+ raw: str,
+ language: str,
+ src: str,
+ *,
+ is_reexport: bool = False,
+ is_module_declaration: bool = False,
+) -> Import:
+ """The single Import of a statement no language-specific shape claims."""
+ imported_names, bindings = extract_import_bindings(stmt_node, src, language)
+ is_relative = (
+ module_text.startswith(".")
+ or module_text.startswith("./")
+ or module_text.startswith(("self::", "super::", "crate::"))
+ )
+ # ``type_only`` / ``deferred`` are set by the caller from ``import_load_kind``.
+ return Import(
+ raw_statement=raw,
+ module_path=module_text,
+ imported_names=imported_names,
+ is_relative=is_relative,
+ resolved_file=None,
+ bindings=bindings,
+ is_reexport=is_reexport,
+ is_module_declaration=is_module_declaration,
+ )
+
+
+def generic_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ return [_generic_import(stmt_node, module_text, raw, language, src)]
+
+
+GENERIC_IMPORTS = ImportExtractor(statement=generic_statement)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/jvm.py b/packages/core/src/repowise/core/ingestion/extractors/imports/jvm.py
new file mode 100644
index 000000000..d4c0736d9
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/jvm.py
@@ -0,0 +1,24 @@
+"""Java and Kotlin imports."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ._types import ImportExtractor
+from .generic import _generic_import
+
+
+def _jvm_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ # The query captures the scoped identifier only, so ``import com.foo.*``
+ # arrives as ``com.foo``. Restore the ``.*``.
+ if not module_text.endswith("*"):
+ stmt_text = raw.rstrip().rstrip(";").rstrip()
+ if stmt_text.endswith(".*"):
+ module_text += ".*"
+ return [_generic_import(stmt_node, module_text, raw, language, src)]
+
+
+JVM_IMPORTS = ImportExtractor(statement=_jvm_statement)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/pascal.py b/packages/core/src/repowise/core/ingestion/extractors/imports/pascal.py
new file mode 100644
index 000000000..e7f32602f
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/pascal.py
@@ -0,0 +1,41 @@
+"""Pascal ``uses`` clauses."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ..helpers import node_text
+from ._types import ImportExtractor
+from .generic import generic_statement
+
+
+def _pascal_imports(
+ stmt_node: Node, module_node: Node, src: str, seen_units: set[str]
+) -> list[Import]:
+ """One Pascal ``uses`` unit, deduped case-insensitively across the file.
+
+ pascal.scm fires once per unit name, so a 3-unit clause arrives as 3
+ matches sharing one statement span: deduping by raw text would drop all
+ but the first. A unit named in both ``uses`` clauses is one dependency.
+ ``uses`` exposes the unit's whole interface, hence the ``*`` wildcard.
+ """
+ raw = node_text(stmt_node, src).strip()
+ unit_name = node_text(module_node, src).strip()
+ if not unit_name or unit_name.lower() in seen_units:
+ return []
+ seen_units.add(unit_name.lower())
+ return [
+ Import(
+ raw_statement=raw,
+ module_path=unit_name,
+ imported_names=["*"],
+ is_relative=False,
+ resolved_file=None,
+ bindings=[],
+ is_reexport=False,
+ )
+ ]
+
+
+PASCAL_IMPORTS = ImportExtractor(statement=generic_statement, deduped=_pascal_imports)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/php.py b/packages/core/src/repowise/core/ingestion/extractors/imports/php.py
new file mode 100644
index 000000000..58d020f5a
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/php.py
@@ -0,0 +1,37 @@
+"""PHP ``use`` declarations."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import, NamedBinding
+from ..bindings.php import php_class_name, php_use_clauses
+from ._types import ImportExtractor
+from .generic import _generic_import
+
+
+def _php_imports(stmt_node: Node, raw: str, src: str) -> list[Import]:
+ """PHP: one ``use`` declaration can name several classes, each its own file."""
+ return [
+ Import(
+ raw_statement=raw,
+ module_path=fqn,
+ imported_names=[php_class_name(fqn)],
+ is_relative=False,
+ resolved_file=None,
+ bindings=[NamedBinding(local_name=local, exported_name=fqn, source_file=None)],
+ is_reexport=False,
+ )
+ for fqn, local in php_use_clauses(stmt_node, src)
+ ]
+
+
+def _php_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ if stmt_node.type == "namespace_use_declaration":
+ return _php_imports(stmt_node, raw, src)
+ return [_generic_import(stmt_node, module_text, raw, language, src)]
+
+
+PHP_IMPORTS = ImportExtractor(statement=_php_statement)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/python.py b/packages/core/src/repowise/core/ingestion/extractors/imports/python.py
new file mode 100644
index 000000000..cf943b8bc
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/python.py
@@ -0,0 +1,9 @@
+"""Python imports."""
+
+from __future__ import annotations
+
+from ..bindings.python import expand_bare_relative_imports
+from ._types import ImportExtractor
+from .generic import generic_statement
+
+PYTHON_IMPORTS = ImportExtractor(statement=generic_statement, finalize=expand_bare_relative_imports)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/rust.py b/packages/core/src/repowise/core/ingestion/extractors/imports/rust.py
new file mode 100644
index 000000000..b18fc68da
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/rust.py
@@ -0,0 +1,127 @@
+"""Rust ``use`` declarations, ``mod`` items and module-declaring macros."""
+
+from __future__ import annotations
+
+import re
+
+from tree_sitter import Node
+
+from ...models import Import, NamedBinding
+from ..bindings.rust import expand_rust_use_tree, macro_mod_imports, rust_use_argument
+from ..helpers import node_text
+from ._types import ImportExtractor
+from .generic import _generic_import
+
+
+def _rust_use_imports(stmt_node: Node, module_text: str, raw: str, src: str) -> list[Import]:
+ """Rust: one Import per leaf of the use tree, each with its own path.
+
+ ``use crate::{a::B, c::D}`` names two modules, and the joined string
+ resolves to neither. ``super`` hops that only climb out of inline
+ ``mod`` blocks stay inside this file, so they are dropped here and the
+ resolver sees the path the file itself would write.
+ """
+ arg_node = rust_use_argument(stmt_node)
+ leaves = expand_rust_use_tree(arg_node, src) if arg_node is not None else []
+ inline_depth = _rust_inline_mod_depth(stmt_node)
+ # ``pub use`` re-exports the module.
+ is_reexport = any(child.type == "visibility_modifier" for child in stmt_node.children)
+ if len(leaves) <= 1 and "{" not in module_text and not inline_depth:
+ return [_generic_import(stmt_node, module_text, raw, "rust", src, is_reexport=is_reexport)]
+ imports = []
+ for path, local, exported in leaves:
+ path = _strip_inline_super(path, inline_depth)
+ imports.append(
+ Import(
+ raw_statement=raw,
+ module_path=path,
+ imported_names=[local],
+ is_relative=path.startswith(("self::", "super::", "crate::")),
+ resolved_file=None,
+ bindings=[NamedBinding(local_name=local, exported_name=exported, source_file=None)],
+ is_reexport=is_reexport,
+ )
+ )
+ return imports
+
+
+def _rust_inline_mod_depth(node: Node) -> int:
+ """How many inline ``mod name { ... }`` blocks enclose *node* in its file."""
+ depth = 0
+ parent = node.parent
+ while parent is not None:
+ if parent.type == "mod_item":
+ depth += 1
+ parent = parent.parent
+ return depth
+
+
+def _strip_inline_super(path: str, inline_depth: int) -> str:
+ """Drop the ``super`` hops of *path* that stay inside the file.
+
+ In ``mod tests { use super::*; }`` the ``super`` is this file's own
+ module, so the path is ``self::*``, not the parent module's glob.
+ """
+ if not inline_depth:
+ return path
+ segments = path.split("::")
+ hops = 0
+ while hops < len(segments) and segments[hops] == "super":
+ hops += 1
+ if not hops:
+ return path
+ if hops <= inline_depth:
+ return "::".join(["self", *segments[hops:]])
+ return "::".join(segments[inline_depth:])
+
+
+def _rust_mod_path_attribute(stmt_node: Node, src: str) -> str | None:
+ """The ``#[path = "..."]`` override on a Rust ``mod`` item, if any.
+
+ Outer attributes are preceding siblings of the item, not children.
+ """
+ parent = stmt_node.parent
+ if parent is None:
+ return None
+ siblings = parent.children
+ for j, sib in enumerate(siblings):
+ if sib.id != stmt_node.id:
+ continue
+ # Walk backward through preceding attribute_item siblings
+ k = j - 1
+ while k >= 0 and siblings[k].type == "attribute_item":
+ path_match = re.search(r'path\s*=\s*"([^"]+)"', node_text(siblings[k], src))
+ if path_match:
+ return path_match.group(1)
+ k -= 1
+ return None
+ return None
+
+
+def _rust_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ if stmt_node.type == "use_declaration":
+ return _rust_use_imports(stmt_node, module_text, raw, src)
+ if stmt_node.type == "macro_invocation":
+ # Most top-level calls (``s! { struct ... }``) declare no module.
+ return macro_mod_imports(module_node, raw) if "mod" in raw else []
+ if stmt_node.type == "mod_item":
+ module_text = _rust_mod_path_attribute(stmt_node, src) or module_text
+ return [
+ _generic_import(stmt_node, module_text, raw, language, src, is_module_declaration=True)
+ ]
+ return [_generic_import(stmt_node, module_text, raw, language, src)]
+
+
+def _rust_dedup_key(stmt_node: Node, raw: str, src: str) -> str:
+ # A ``mod`` item's text excludes its outer attributes, so two ``#[cfg]``-gated
+ # ``mod imp;`` items with different ``#[path]`` files would share one key.
+ if stmt_node.type == "mod_item":
+ mod_path_attr = _rust_mod_path_attribute(stmt_node, src)
+ if mod_path_attr is not None:
+ return f"{raw}|path={mod_path_attr}"
+ return raw
+
+
+RUST_IMPORTS = ImportExtractor(statement=_rust_statement, dedup_key=_rust_dedup_key)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/scala.py b/packages/core/src/repowise/core/ingestion/extractors/imports/scala.py
new file mode 100644
index 000000000..529f7a63c
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/scala.py
@@ -0,0 +1,52 @@
+"""Scala import declarations."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import, NamedBinding
+from ..bindings.scala import expand_scala_import_clauses
+from ._types import ImportExtractor
+from .generic import _generic_import
+
+
+def _scala_imports(stmt_node: Node, raw: str, src: str) -> list[Import]:
+ """Scala: one Import per selected name, with full dotted paths.
+
+ The query's ``(identifier)`` capture is only the first path segment, and
+ one declaration can hold several clauses, brace selectors, renames and
+ wildcards.
+ """
+ imports: list[Import] = []
+ for clause_path, clause_names in expand_scala_import_clauses(stmt_node, src):
+ local = clause_names[0]
+ exported = None if local == "*" else clause_path.rsplit(".", 1)[-1]
+ imports.append(
+ Import(
+ raw_statement=raw,
+ module_path=clause_path,
+ imported_names=clause_names,
+ is_relative=False,
+ resolved_file=None,
+ bindings=[
+ NamedBinding(
+ local_name=local,
+ exported_name=exported,
+ source_file=None,
+ )
+ ],
+ is_reexport=False,
+ )
+ )
+ return imports
+
+
+def _scala_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ if stmt_node.type == "import_declaration":
+ return _scala_imports(stmt_node, raw, src)
+ return [_generic_import(stmt_node, module_text, raw, language, src)]
+
+
+SCALA_IMPORTS = ImportExtractor(statement=_scala_statement)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/swift.py b/packages/core/src/repowise/core/ingestion/extractors/imports/swift.py
new file mode 100644
index 000000000..9f66714c3
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/swift.py
@@ -0,0 +1,20 @@
+"""Swift imports."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ._types import ImportExtractor
+from .generic import _generic_import
+
+
+def _swift_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ # ``@_exported import`` re-exports the module.
+ is_reexport = raw.startswith("@_exported")
+ return [_generic_import(stmt_node, module_text, raw, language, src, is_reexport=is_reexport)]
+
+
+SWIFT_IMPORTS = ImportExtractor(statement=_swift_statement)
diff --git a/packages/core/src/repowise/core/ingestion/extractors/imports/ts_js.py b/packages/core/src/repowise/core/ingestion/extractors/imports/ts_js.py
new file mode 100644
index 000000000..40db63b1a
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/imports/ts_js.py
@@ -0,0 +1,82 @@
+"""TypeScript / JavaScript imports: ESM, dynamic ``import()`` and CommonJS ``require``.
+
+Also serves the SFC tags (svelte, vue, astro), whose script blocks parse as TS.
+"""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import Import
+from ..bindings.ts_js import (
+ cjs_statement_is_reexport,
+ collect_cjs_requires,
+ dynamic_import_bindings,
+)
+from ._types import ImportExtractor
+from .generic import _generic_import
+
+# CommonJS is read for plain TS/JS files only, not the SFC tags.
+_CJS_LANGUAGES = ("javascript", "typescript")
+_CJS_STATEMENTS = ("assignment_expression", "call_expression")
+
+
+def _is_dynamic_esm_import(stmt_node: Node) -> bool:
+ if stmt_node.type != "call_expression":
+ return False
+ fn = stmt_node.child_by_field_name("function")
+ return fn is not None and fn.type == "import"
+
+
+def _cjs_imports(stmt_node: Node, raw: str, src: str) -> list[Import]:
+ """CommonJS: every ``require()`` in the statement, re-exports marked.
+
+ A hub like ``Object.assign(module.exports, require('./a'), ...)`` is
+ several imports, and ``module.exports`` shapes are barrels.
+ """
+ cjs_reexport = cjs_statement_is_reexport(stmt_node, src)
+ return [
+ Import(
+ raw_statement=raw,
+ module_path=cjs_module,
+ imported_names=["*"] if cjs_reexport else [],
+ is_relative=cjs_module.startswith("."),
+ resolved_file=None,
+ bindings=[],
+ is_reexport=cjs_reexport,
+ )
+ for cjs_module in collect_cjs_requires(stmt_node, src)
+ ]
+
+
+def _ts_js_statement(
+ stmt_node: Node, module_node: Node, module_text: str, raw: str, language: str, src: str
+) -> list[Import]:
+ if _is_dynamic_esm_import(stmt_node):
+ # ``import('./mod')`` binds a module namespace at runtime, so it is a
+ # wildcard, which keeps the target's exports live. The names it is
+ # destructured into still bind, for call resolution.
+ return [
+ Import(
+ raw_statement=raw,
+ module_path=module_text,
+ imported_names=["*"],
+ is_relative=module_text.startswith("."),
+ resolved_file=None,
+ bindings=dynamic_import_bindings(stmt_node, src),
+ is_reexport=False,
+ )
+ ]
+ if language in _CJS_LANGUAGES and stmt_node.type in _CJS_STATEMENTS:
+ return _cjs_imports(stmt_node, raw, src)
+ return [_generic_import(stmt_node, module_text, raw, language, src)]
+
+
+def _ts_js_dedup_key(stmt_node: Node, raw: str, src: str) -> str:
+ # Two ``import('./m')`` calls share their text but not the names each one binds.
+ if _is_dynamic_esm_import(stmt_node):
+ return f"{raw}|at={stmt_node.start_byte}"
+ return raw
+
+
+TS_JS_IMPORTS = ImportExtractor(statement=_ts_js_statement, dedup_key=_ts_js_dedup_key)
diff --git a/packages/core/src/repowise/core/ingestion/lang_helpers/__init__.py b/packages/core/src/repowise/core/ingestion/lang_helpers/__init__.py
index 83c12b9b1..193d58a71 100644
--- a/packages/core/src/repowise/core/ingestion/lang_helpers/__init__.py
+++ b/packages/core/src/repowise/core/ingestion/lang_helpers/__init__.py
@@ -1,4 +1,4 @@
"""Per-language tree-sitter helpers the ingestion parser calls.
-``parser_helpers`` re-exports every name ``parser.py`` imports from here.
+``parser_helpers`` re-exports every name the ``parser`` package imports from here.
"""
diff --git a/packages/core/src/repowise/core/ingestion/lang_helpers/elixir.py b/packages/core/src/repowise/core/ingestion/lang_helpers/elixir.py
index b0db4bf37..7f6115230 100644
--- a/packages/core/src/repowise/core/ingestion/lang_helpers/elixir.py
+++ b/packages/core/src/repowise/core/ingestion/lang_helpers/elixir.py
@@ -226,7 +226,7 @@ def _elixir_is_template_definition(def_node: Node, src: str) -> bool:
def _elixir_module_parent(def_node: Node, src: str) -> str | None:
"""Return the enclosing ``defmodule``/``defprotocol`` name, or None.
- The generic nesting walk in ``ASTParser._find_parent`` reads
+ The generic nesting walk in ``parser.symbols._find_parent`` reads
``ancestor.child_by_field_name("name")``, and an Elixir ``call`` has no
``name`` field -- the module name is the first argument of a call to
``defmodule``. Nesting is the only thing that says which module a
diff --git a/packages/core/src/repowise/core/ingestion/language_configs.py b/packages/core/src/repowise/core/ingestion/language_configs.py
index 4571659c8..168ed6ac8 100644
--- a/packages/core/src/repowise/core/ingestion/language_configs.py
+++ b/packages/core/src/repowise/core/ingestion/language_configs.py
@@ -1,9 +1,8 @@
"""Per-language parser configuration data.
``LanguageConfig`` plus the declarative ``LANGUAGE_CONFIGS`` table that
-drives :class:`~repowise.core.ingestion.parser.ASTParser`. Extracted from
-``parser.py`` so the parser module holds behaviour and this module holds
-the per-language data. The parser keeps re-exporting both names, so
+drives :class:`~repowise.core.ingestion.parser.ASTParser`: the parser holds
+behaviour, this module the per-language data. The parser keeps re-exporting both names, so
``from ...parser import LANGUAGE_CONFIGS, LanguageConfig`` still works.
"""
@@ -19,6 +18,21 @@ from .extractors.cobol import (
normalize_cobol_call_target,
normalize_cobol_symbol_name,
)
+from .extractors.imports import (
+ DART_IMPORTS,
+ ELIXIR_IMPORTS,
+ FSHARP_IMPORTS,
+ GENERIC_IMPORTS,
+ JVM_IMPORTS,
+ PASCAL_IMPORTS,
+ PHP_IMPORTS,
+ PYTHON_IMPORTS,
+ RUST_IMPORTS,
+ SCALA_IMPORTS,
+ SWIFT_IMPORTS,
+ TS_JS_IMPORTS,
+ ImportExtractor,
+)
from .extractors.visibility import (
csharp_visibility,
dart_visibility,
@@ -98,9 +112,14 @@ class LanguageConfig:
# ``modifier``, Kotlin ``modifiers``), read into ``Symbol.modifiers``.
modifier_node_types: frozenset[str] = field(default_factory=frozenset)
+ # How this language's import matches become ``Import`` records; see
+ # ``extractors.imports``.
+ import_extractor: ImportExtractor = GENERIC_IMPORTS
+
LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
"python": LanguageConfig(
+ import_extractor=PYTHON_IMPORTS,
symbol_node_types={
"function_definition": "function",
"class_definition": "class",
@@ -115,6 +134,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
parent_class_types=frozenset({"class_definition"}),
),
"typescript": LanguageConfig(
+ import_extractor=TS_JS_IMPORTS,
symbol_node_types={
"function_declaration": "function",
"generator_function_declaration": "function",
@@ -148,6 +168,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
declaration_node_types=frozenset({"function_signature", "method_signature"}),
),
"javascript": LanguageConfig(
+ import_extractor=TS_JS_IMPORTS,
symbol_node_types={
"function_declaration": "function",
"generator_function_declaration": "function",
@@ -178,6 +199,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
parent_class_types=frozenset(),
),
"rust": LanguageConfig(
+ import_extractor=RUST_IMPORTS,
symbol_node_types={
"function_item": "function",
"function_signature_item": "function",
@@ -212,6 +234,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
reference_call_node_types=frozenset({"macro_invocation"}),
),
"java": LanguageConfig(
+ import_extractor=JVM_IMPORTS,
symbol_node_types={
"class_declaration": "class",
"interface_declaration": "interface",
@@ -280,6 +303,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
declaration_node_types=frozenset({"declaration"}),
),
"kotlin": LanguageConfig(
+ import_extractor=JVM_IMPORTS,
symbol_node_types={
"function_declaration": "function",
"class_declaration": "class",
@@ -374,6 +398,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
),
),
"swift": LanguageConfig(
+ import_extractor=SWIFT_IMPORTS,
symbol_node_types={
"class_declaration": "class",
"protocol_declaration": "interface",
@@ -390,6 +415,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
parent_class_types=frozenset({"class_declaration", "protocol_declaration"}),
),
"dart": LanguageConfig(
+ import_extractor=DART_IMPORTS,
symbol_node_types={
"class_definition": "class",
"mixin_declaration": "class",
@@ -419,6 +445,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
),
),
"scala": LanguageConfig(
+ import_extractor=SCALA_IMPORTS,
symbol_node_types={
"class_definition": "class",
"trait_definition": "trait",
@@ -438,6 +465,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
parent_class_types=frozenset({"class_definition", "trait_definition", "object_definition"}),
),
"php": LanguageConfig(
+ import_extractor=PHP_IMPORTS,
symbol_node_types={
"class_declaration": "class",
"interface_declaration": "interface",
@@ -457,6 +485,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
),
),
"elixir": LanguageConfig(
+ import_extractor=ELIXIR_IMPORTS,
symbol_node_types={
# Elixir's one node kind for every definition. "module" is a
# placeholder, refined per keyword in refine_elixir_call_kind --
@@ -478,6 +507,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
parent_class_types=frozenset(),
),
"fsharp": LanguageConfig(
+ import_extractor=FSHARP_IMPORTS,
symbol_node_types={
# `module Foo.Bar` / `namespace Foo.Bar` head the file;
# `module X =` nests inside one.
@@ -487,7 +517,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
# The captured node is the binding's left-hand side, not the
# enclosing function_or_value_defn: that one node holds every
# clause of a `let rec f ... and g ...` group, so capturing it
- # would give every clause the same span. parser.py extends each
+ # would give every clause the same span. the parser extends each
# symbol over its own clause and filters the nested ones.
"function_declaration_left": "function",
"value_declaration_left": "variable",
@@ -516,7 +546,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
parent_extraction="nesting",
# Deliberately empty: no F# type node carries a `name` field, so the
# generic walk in _find_parent would match an ancestor and then read
- # nothing off it. The parent walk is language-gated in parser.py --
+ # nothing off it. The parent walk is language-gated in the parser --
# see _fsharp_parent_name.
parent_class_types=frozenset(),
),
@@ -567,12 +597,13 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
),
),
"pascal": LanguageConfig(
+ import_extractor=PASCAL_IMPORTS,
symbol_node_types={
# declType wraps class/record/interface/helper/enum/set/array/alias
# in one node shape (see spec docstring) -- "class" is the closest
# single bucket, same tradeoff Go makes for `type_spec`: "struct".
# A real kind refinement (peek at the `type` field to distinguish
- # class/struct/interface/enum) would need a hook parser.py doesn't
+ # class/struct/interface/enum) would need a hook the parser doesn't
# currently expose per-language the way it does for Go/Kotlin --
# flagged as a follow-up, not attempted here.
"declType": "class",
@@ -597,7 +628,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
visibility_fn=public_by_default,
parent_extraction="nesting",
# NOT {"declClass", "declIntf", "declHelper"} -- see the spec docstring.
- # ASTParser._find_parent walks ancestors checking `ancestor.type in
+ # parser.symbols._find_parent walks ancestors checking `ancestor.type in
# parent_class_types` then reads `ancestor.child_by_field_name("name")`;
# only declType carries a name field, so pointing this at the inner
# class/interface/helper node would silently return no parent for every
diff --git a/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py b/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py
index c0df0cf61..2b1eaf363 100644
--- a/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py
+++ b/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py
@@ -10,7 +10,7 @@ heritage_node_types / parent_class_types are both "declType", not the more
obvious "declClass"/"declIntf"/"declHelper". Pascal nests the actual class
body one level inside a `Name = <class|interface|helper body>;` wrapper
(`declType`), and only the wrapper carries a `name` field -- `declClass`
-itself has none. `ASTParser._find_parent` looks up
+itself has none. `parser.symbols._find_parent` looks up
`ancestor.child_by_field_name("name")` on whatever ancestor type matches
`parent_class_types`, so pointing it at "declClass" would silently return
no parent name for every method. See extractors/heritage/pascal.py for the
@@ -48,8 +48,8 @@ SPEC = LanguageSpec(
shebang_tokens=(),
# Common RTL calls worth excluding from the project call graph, spelled
# the way the RTL itself declares them (Pascal is case-insensitive, but
- # `ASTParser._extract_calls` does a case-sensitive `target_name in
- # _call_builtins` check against the raw source text, so entries need to
+ # `parser.calls._extract_calls` does a case-sensitive `target_name in
+ # call_builtins` check against the raw source text, so entries need to
# match real-world casing, not lowercased). This list is intentionally
# conservative -- System-unit built-ins only; framework calls (VCL/FMX/
# LCL) are ordinary project dependencies and should still show up as
diff --git a/packages/core/src/repowise/core/ingestion/parser.py b/packages/core/src/repowise/core/ingestion/parser.py
deleted file mode 100644
index 78557e337..000000000
--- a/packages/core/src/repowise/core/ingestion/parser.py
+++ /dev/null
@@ -1,2452 +0,0 @@
-"""Unified AST parser — one class for all languages.
-
-Architecture
-============
-Per-language differences live in two places:
- 1. ``packages/core/queries/<lang>.scm`` — tree-sitter S-expression queries
- that capture symbols and imports using consistent capture-name conventions.
- 2. ``LANGUAGE_CONFIGS`` dict in this module — a ``LanguageConfig`` per language
- that maps node types to symbol kinds, defines visibility rules, etc.
-
-``ASTParser`` itself contains *no* if/elif language branches. Adding support
-for a new language means writing one ``.scm`` file and one ``LanguageConfig``
-entry. No Python class, no new module.
-
-Capture-name conventions (shared across ALL .scm files):
- @symbol.def — the full definition node (line numbers, kind lookup)
- @symbol.name — name identifier
- @symbol.params — parameter list (optional)
- @symbol.modifiers — decorators / visibility modifiers (optional)
- @symbol.receiver — Go method receiver (optional, used for parent detection)
- @import.statement — full import node
- @import.module — module path being imported
-"""
-
-from __future__ import annotations
-
-import re
-import sys
-from collections.abc import Iterable
-from dataclasses import replace
-from functools import cache
-from pathlib import Path
-
-import structlog
-from tree_sitter import Language, Node, Parser
-
-from .cpp_export_macros import (
- CppExportTypes,
- _cpp_export_macro_parent,
- _cpp_normalize_identifier,
- _CppExportType,
- _is_bodiless_cpp_type,
- collect_cpp_export_types,
-)
-from .extractors import (
- build_signature,
- extract_go_receiver_type,
- extract_heritage,
- extract_import_bindings,
- extract_module_docstring,
- extract_symbol_docstring,
- node_text,
- refine_elixir_call_kind,
- refine_fsharp_type_kind,
- refine_go_type_kind,
- refine_kotlin_class_kind,
- refine_pascal_type_kind,
- refine_swift_class_kind,
-)
-from .extractors.bindings.elixir import elixir_import_modules
-from .extractors.bindings.python import expand_bare_relative_imports
-from .extractors.bindings.ts_js import (
- declarator_binds_callable,
- declarator_value_is_module_ref,
- dynamic_import_bindings,
-)
-from .extractors.synthetic_symbols import extract_synthetic_symbols
-from .extractors.synthetic_symbols.csharp_extension_blocks import drop_extension_block_symbols
-from .extractors.visibility import (
- py_module_all_names,
- refine_cpp_visibility,
- refine_csharp_visibility,
- refine_java_visibility,
- refine_py_visibility,
- refine_rust_visibility,
- refine_ts_visibility,
- ts_deferred_export_names,
- ts_export_aliases,
-)
-from .language_configs import LANGUAGE_CONFIGS, LanguageConfig
-from .languages.registry import REGISTRY as _LANG_REGISTRY
-from .models import (
- CallReceiver,
- CallSite,
- FileInfo,
- Import,
- ParsedFile,
- Symbol,
- TypeReference,
- combine_load_kinds,
- compute_content_hash,
-)
-from .parser_helpers import (
- TYPE_HEAD_EXTRACTORS,
- _argument_literal_types,
- _argument_names,
- _build_qualified_name,
- _classify_param_origin,
- _collect_error_nodes,
- _count_arguments,
- _csharp_type_parameter_count,
- _dedupe_objc_interface_symbols,
- _dedupe_pascal_interface_symbols,
- _elixir_call_is_definitional,
- _elixir_is_template_definition,
- _elixir_module_parent,
- _elixir_symbol_name,
- _find_enclosing_symbol,
- _fsharp_binding_end_line,
- _fsharp_binding_has_params,
- _fsharp_binding_is_nested,
- _fsharp_parent_is_type,
- _fsharp_parent_name,
- _has_callable_ancestor,
- _head_type_identifier,
- _is_async_node,
- _kotlin_receiver_fields,
- _objc_call_is_block_variable,
- _objc_container_node,
- _objc_container_parent,
- _objc_is_macro_enum,
- _objc_message_selector,
- _objc_symbol_name,
- _overload_arguments,
- _qualified_cpp_parent,
- _qualified_pascal_parent,
- _run_query,
- _rust_shadowed_by_type_param,
- _ts_nested_object_method_owner,
- follow_renamed_hosts,
- import_load_kind,
- in_anonymous_class,
- is_anonymous_member,
- scope_segment,
-)
-from .python_local_refs import local_refs_from_tree, parse_python_module
-from .python_overload import is_python_overload as _is_python_overload
-from .python_registrars import registering_functions
-from .python_tasks import is_fire_and_forget
-from .sfc_source import component_call_sites, prepare_source
-from .special_handlers import SPECIAL_HANDLER_LANGUAGES, parse_special
-from .symbol_identity import disambiguate_colliding_ids, symbol_discriminator
-
-log = structlog.get_logger(__name__)
-
-# Any single file emitting more than this many symbols is almost
-# certainly machine-generated (large gRPC service contracts, OpenAPI
-# bindings, SQL schema bindings). Warn rather than truncate — operators
-# can decide whether to add the file to ``_NEVER_FLAG_PATTERNS`` or to
-# exclude it via traversal.
-_SYMBOL_COUNT_WARN_THRESHOLD = 500
-
-QUERIES_DIR = Path(__file__).parent / "queries"
-
-# Node types whose .scm patterns are anchored at module/program level
-# (constants and module variables). They can never be function-local, so
-# the callable-ancestor filter must not apply — and for TS/JS declarators
-# it would misfire on the parent lexical_declaration kind mapping.
-_MODULE_ANCHORED_NODE_TYPES = frozenset({"assignment", "variable_declarator"})
-
-# Languages whose source reaches the parser as TypeScript/JavaScript. The
-# SFC tags are here because ``sfc_source`` projects their <script> blocks into
-# a TS buffer at identical offsets, so every TS/JS code path applies verbatim.
-_TS_JS_LANGUAGES = ("typescript", "javascript", "svelte", "vue", "astro")
-
-# Languages whose query defines the ``@reference.*`` captures. Every other
-# language would only scan the whole match list to find nothing, so the check
-# is here rather than inside ``_extract_references``. The SFC tags reuse the
-# TS captures but stay out: resolution has no local-shadow scan for them.
-_REFERENCE_LANGUAGES = ("cpp", "c", "go", "rust", "kotlin", "typescript", "javascript", "python")
-
-# Languages where a bare name resolves only through the file's own symbols or
-# a name it imports, so any other bare name cannot pass the reference floor.
-_NAMED_BINDING_LANGUAGES = frozenset({"typescript", "javascript", "python"})
-
-
-def _bare_reference_names(
- language: str, symbols: list[Symbol], imports: list[Import]
-) -> frozenset[str] | None:
- """Names a receiver-less reference in this file could resolve to, or None for any.
-
- Most bare identifiers in a value position are locals; dropping them here
- skips their per-site work. A wildcard import keeps every name.
- """
- if language not in _NAMED_BINDING_LANGUAGES:
- return None
- imported = {name for imp in imports for name in imp.local_names}
- if "*" in imported:
- return None
- return frozenset(imported.union(s.name for s in symbols))
-
-
-def _call_receiver_from_node(node: Node, src: str) -> CallReceiver | None:
- """Describe an inner call captured as another call's receiver.
-
- The queries identify the complete inner AST node. This helper reads only
- named tree-sitter fields, so nested arguments and formatting cannot change
- which call is carried into resolution.
- """
-
- if node.type == "identifier":
- return _dart_call_receiver(node, src)
- arguments = node.child_by_field_name("arguments")
- argument_count = _count_arguments(arguments) if arguments is not None else None
-
- target = node.child_by_field_name("name")
- receiver = node.child_by_field_name("object")
- if target is None:
- function = node.child_by_field_name("function")
- if function is None:
- return None
- if function.type in ("identifier", "property_identifier", "field_identifier"):
- target = function
- receiver = None
- elif function.type == "generic_name":
- target = function.child_by_field_name("name") or next(
- (child for child in function.named_children if child.type == "identifier"), None
- )
- receiver = None
- elif function.type == "attribute":
- # Python ``obj.m()``: the receiver is kept as a plain dotted path,
- # and a call on anything else is not carried at all.
- target = function.child_by_field_name("attribute")
- obj = function.child_by_field_name("object")
- path = (
- _receiver_text("python", obj, src)
- if obj is not None and obj.type in ("identifier", "attribute")
- else None
- )
- if target is None or path is None:
- return None
- return CallReceiver(_node_text(target, src).strip(), path, argument_count)
- else:
- target = (
- function.child_by_field_name("name")
- or function.child_by_field_name("property")
- or function.child_by_field_name("field")
- )
- receiver = (
- function.child_by_field_name("expression")
- or function.child_by_field_name("object")
- or function.child_by_field_name("argument")
- or function.child_by_field_name("operand")
- )
-
- if target is None:
- return None
- target_name = _node_text(target, src).strip()
- if not target_name:
- return None
-
- receiver_name = None
- if receiver is not None and receiver.type in ("identifier", "this"):
- receiver_name = _node_text(receiver, src).strip() or None
- return CallReceiver(target_name, receiver_name, argument_count)
-
-
-def _dart_call_receiver(name: Node, src: str) -> CallReceiver | None:
- """The Dart call whose *name* the query captured: ``a.b(..)``, ``B(..)``.
-
- A Dart call is a flat selector run with no call node, so the name's
- siblings are read: the selector after it holds the arguments, and the
- node before a ``.b`` selector is its receiver.
- """
- if name.parent.type == "unconditional_assignable_selector":
- head = name.parent.parent
- prev = head.prev_named_sibling
- if prev is not None and any(c.type == "argument_part" for c in prev.named_children):
- receiver = None
- else:
- receiver = _node_text(prev, src).strip() if prev is not None else None
- else:
- head, receiver = name, None
- return CallReceiver(
- _node_text(name, src).strip(), receiver, _dart_argument_count(head.next_named_sibling)
- )
-
-
-def _dart_argument_count(selector: Node | None) -> int | None:
- """The arguments a Dart ``(selector (argument_part (arguments)))`` passes."""
- parts = selector.named_children if selector is not None else ()
- found = [arg for part in parts for arg in part.named_children if arg.type == "arguments"]
- return _count_arguments(found[0]) if found else None
-
-
-@cache
-def _compile_query(lang: str, grammar_tag: str | None = None) -> tuple[object | None, str | None]:
- """Compile and return (Query, None) or (None, error_str).
-
- Cached process-wide so preflight and parsing workers never recompile the same query.
- """
- grammar = grammar_tag or lang
- language = _get_language(grammar)
- if language is None:
- return None, None
-
- # The spec names the query file, so a language can reuse another's
- # queries wholesale (svelte -> typescript.scm). Every other spec declares
- # ``<tag>.scm``, which is what the default preserves.
- spec = _LANG_REGISTRY.get(lang)
- scm_name = (spec.scm_file if spec and spec.scm_file else None) or f"{lang}.scm"
- scm_path = QUERIES_DIR / scm_name
- if not scm_path.exists():
- log.debug("No .scm query file found", language=lang, path=str(scm_path))
- return None, None
-
- scm_text = scm_path.read_text(encoding="utf-8")
- # Grammar-variant-specific additions (e.g. JSX node captures that are
- # only valid against the ``tsx`` grammar but not the plain ``typescript``
- # one). Appended to the base SCM only when the variant scm file exists.
- if grammar_tag and grammar_tag != lang:
- extra_scm = QUERIES_DIR / f"{grammar_tag}.scm"
- if extra_scm.exists():
- scm_text = scm_text + "\n" + extra_scm.read_text(encoding="utf-8")
- try:
- from tree_sitter import Query # type: ignore[attr-defined]
-
- return Query(language, scm_text), None
- except Exception as exc:
- return None, str(exc)
-
-
-@cache
-def _load_compiled_query(lang: str, grammar_tag: str | None = None) -> object | None:
- """Process-wide cache of compiled tree-sitter Query objects.
-
- Compiling `.scm` queries is non-trivial; in process-pool parsing each worker
- would otherwise recompile per file. ``grammar_tag`` may differ from
- ``lang`` when a language reuses another's grammar at a different
- variant — e.g. ``.tsx`` files reuse ``typescript.scm`` but must bind
- to the JSX-aware ``tsx`` grammar so React components don't drown in
- ERROR nodes.
- """
- query, err = _compile_query(lang, grammar_tag)
- if err is not None:
- log.warning("Failed to compile query", language=lang, error=err)
- return query
-
-
-# Languages that intentionally have no AST parser. Derived from the
-# centralised LanguageRegistry — only non-code passthrough languages are
-# included (not the extra git-blame-only languages).
-
-# Excludes "openapi" (handled by special_handlers) and "unknown".
-_PASSTHROUGH_LANGUAGES: frozenset[str] = _LANG_REGISTRY.unparseable_data_languages()
-
-# ---------------------------------------------------------------------------
-# Language registry — maps language tag → tree-sitter Language object
-# ---------------------------------------------------------------------------
-
-
-def _build_language_registry() -> dict[str, Language]:
- """Lazily load installed tree-sitter language packages.
-
- Driven by ``LanguageSpec.grammar_package`` / ``grammar_loader`` /
- ``shares_grammar_with`` from the centralised registry.
- """
- registry: dict[str, Language] = {}
-
- for spec in _LANG_REGISTRY.all_specs():
- # Languages that share another's grammar (e.g. C → cpp)
- if spec.shares_grammar_with:
- shared = registry.get(spec.shares_grammar_with)
- if shared:
- registry[spec.tag] = shared
- continue
-
- if not spec.grammar_package:
- continue
-
- try:
- mod = __import__(spec.grammar_package)
- loader_fn = getattr(mod, spec.grammar_loader)
- loaded = loader_fn(*spec.grammar_loader_args)
- # Standalone grammar wheels return a PyCapsule; shared grammar
- # packs may return the fully constructed Language directly.
- lang_obj = loaded if isinstance(loaded, Language) else Language(loaded)
- registry[spec.tag] = lang_obj
- except Exception as exc:
- log.debug(
- "tree-sitter language unavailable",
- language=spec.tag,
- reason=str(exc),
- )
-
- # TypeScript's tsx variant — special case: same package, different loader
- if "typescript" in registry and "tsx" not in registry:
- try:
- import tree_sitter_typescript as _ts_mod
-
- registry["tsx"] = Language(_ts_mod.language_tsx())
- except Exception as exc:
- log.debug("tree-sitter language unavailable", language="tsx", reason=str(exc))
-
- return registry
-
-
-_LANGUAGE_REGISTRY: dict[str, Language] = {}
-
-# Languages already reported as having a config but no installed grammar, so
-# the report is one line per language per process instead of one per file. A
-# repo with a few thousand shell scripts otherwise emitted a few thousand
-# identical lines, all saying the same three facts.
-_MISSING_GRAMMAR_REPORTED: set[str] = set()
-
-
-def missing_grammar_languages(language_tags: Iterable[str]) -> list[str]:
- """Of *language_tags*, those that parse via tree-sitter but have no grammar.
-
- Answers the question once, in the parent, before up to eight spawned
- workers each rediscover it and log their own copy of the answer at a level
- the CLI discards anyway.
-
- Deliberately uses ``find_spec`` rather than importing: the whole point is
- to stay cheap enough to run on every index. Building the real registry here
- would import every tree-sitter package into the parent process, which is
- memory the parse pool is about to need for something else.
-
- A tag with no :data:`LANGUAGE_CONFIGS` entry is not a gap — nothing claims
- to parse it — so it is skipped rather than reported.
-
- Ceiling: "importable" is not "loadable". A grammar whose compiled ABI does
- not match the installed ``tree_sitter`` imports fine and then raises inside
- ``Language()``, which this cannot see, so that case reports nothing here
- and stays a per-worker debug line. Reporting it properly means loading the
- grammars, which is the cost this function exists to avoid.
- """
- import importlib.util
-
- specs = {spec.tag: spec for spec in _LANG_REGISTRY.all_specs()}
- missing: list[str] = []
- for tag in language_tags:
- if tag not in LANGUAGE_CONFIGS:
- continue
- spec = specs.get(tag)
- if spec is None:
- continue
- package = spec.grammar_package
- if not package and spec.shares_grammar_with:
- shared = specs.get(spec.shares_grammar_with)
- package = shared.grammar_package if shared else None
- if not package:
- continue
- try:
- if importlib.util.find_spec(package) is None:
- missing.append(tag)
- except (ImportError, ValueError):
- missing.append(tag)
- return sorted(missing)
-
-
-def failed_query_languages(language_tags: Iterable[str]) -> list[tuple[str, str]]:
- """Of *language_tags*, those whose tree-sitter queries fail to compile.
-
- Scoped to what traversal discovered in the repo. Only checks languages with
- a registered LanguageConfig and query file. Best-effort and bounded.
- """
- failed: list[tuple[str, str]] = []
- seen: set[str] = set()
-
- for tag in language_tags:
- if tag in seen or tag not in LANGUAGE_CONFIGS:
- continue
- seen.add(tag)
-
- _, err = _compile_query(tag)
- if err is not None:
- failed.append((tag, err))
- continue
-
- # Check grammar variants if applicable (e.g. tsx for typescript)
- if tag == "typescript":
- _, tsx_err = _compile_query("typescript", grammar_tag="tsx")
- if tsx_err is not None:
- failed.append((tag, f"tsx variant: {tsx_err}"))
-
- return sorted(failed, key=lambda x: x[0])
-
-
-def _get_language(tag: str) -> Language | None:
- global _LANGUAGE_REGISTRY
- if not _LANGUAGE_REGISTRY:
- _LANGUAGE_REGISTRY = _build_language_registry()
- return _LANGUAGE_REGISTRY.get(tag)
-
-
-def grammar_tag_for(language: str, path: str) -> str:
- """The grammar a file is read with, which is not always its language tag.
-
- A ``.tsx`` file arrives tagged ``typescript``, and tree-sitter-typescript's
- default grammar errors on every ``<Component />``. Only the grammar moves:
- the language tag keeps selecting dialects and vocabularies, several of
- which (``mocks/lexicon.py`` among them) carry no ``tsx`` row and would
- silently degrade if handed one.
- """
- # Case-folded, because the extension table that tagged the file is.
- if language == "typescript" and path.lower().endswith(".tsx"):
- return "tsx"
- return language
-
-
-# Private alias for internal use (kept for compatibility with _find_parent)
-_node_text = node_text
-
-
-# The receiver captures that are themselves a member access, by language (the
-# queries widen only these two), and what a plain dotted path of names looks
-# like once whitespace is gone.
-_PATH_RECEIVERS = frozenset({("typescript", "member_expression"), ("python", "attribute")})
-_DOTTED_PATH = re.compile(r"[A-Za-z_$][\w$]*(?:\.[A-Za-z_$][\w$]*)+")
-
-
-def _receiver_text(language: str, node: Node, src: str) -> str | None:
- """The receiver's text, or None to drop the site.
-
- A dotted receiver (``this.a.b.m()``) is kept only as a plain path of
- names: a call, subscript or optional hop inside it types nothing, and
- dropping the site keeps it off the bare-call tiers.
- """
- sibling = node.next_named_sibling
- if language == "java" and node.type != "super" and getattr(sibling, "type", None) == "super":
- # ``S.super.m()``: ``S`` qualifies the ``super`` its own pattern captures.
- return None
- text = _node_text(node, src).strip()
- if (language, node.type) not in _PATH_RECEIVERS:
- return text
- path = "".join(text.split())
- return path if _DOTTED_PATH.fullmatch(path) else None
-
-
-def _normalize_php_receiver(text: str) -> str:
- """Spell PHP's receiver the way the resolver's strategies expect.
-
- `self::` and `static::` are the same dispatch as `$this`, and the self/this
- strategy tests `in ("self", "this")` — so without this the whole implicit-
- receiver population misses. `parent::` needs the heritage walk and is left
- to miss rather than guessed at.
- """
- name = text.lstrip("$")
- return "this" if name in ("self", "static") else name
-
-
-# ---------------------------------------------------------------------------
-# ASTParser
-# ---------------------------------------------------------------------------
-
-
-def _match_identity(
- capture_dict: dict,
- config: LanguageConfig,
- language: str,
- src: str,
- cpp_exports: CppExportTypes,
- seen: set[tuple[int, str]],
-) -> tuple[Node, list[Node], str, _CppExportType | None, int] | None:
- """``(def_node, name_nodes, name, export_type, start_line)`` for a new symbol match.
-
- None drops the match: no definition or name, a recovery match this file
- does not support, a suppressed export macro, or a duplicate. The
- ``(start_line, name)`` key is recorded in *seen* here.
- """
- def_nodes = capture_dict.get("symbol.def", [])
- name_nodes = capture_dict.get("symbol.name", [])
- captured_export_type_nodes = capture_dict.get("symbol.cpp_export_type", [])
-
- if (
- captured_export_type_nodes
- and captured_export_type_nodes[0].id not in cpp_exports.capture_ids
- ):
- # The query also sees ordinary ``struct Tag variable;`` forms;
- # discard only the unsupported recovery match.
- return None
-
- if not def_nodes or not name_nodes:
- return None
-
- def_node = def_nodes[0]
- name = _language_symbol_name(
- language, def_node, config.symbol_name_fn(_node_text(name_nodes[0], src), def_node.type), src
- )
- if name is None:
- return None
-
- export_type = cpp_exports.defs.get(def_node.id)
- if export_type is not None and name != export_type.name:
- # The ordinary struct/class query sees the same specifier, but
- # tree-sitter calls the export macro its name. Keep only the
- # dedicated match whose name is the outer declarator.
- return None
-
- if def_node.type == "preproc_def" and (
- _cpp_normalize_identifier(name) in cpp_exports.macro_names
- or def_node.id in cpp_exports.macro_def_ids
- ):
- # Body-form macros are suppressed by name as before #1901;
- # ambiguous forward declarations suppress only the exact
- # active definition that made recovery safe.
- return None
-
- start_line = def_node.start_point[0] + 1
- if export_type is not None:
- start_line = export_type.range_node.start_point[0] + 1
- dedup_key = (start_line, name)
- if dedup_key in seen:
- return None
- seen.add(dedup_key)
- return def_node, name_nodes, name, export_type, start_line
-
-
-_TS_OVERLOAD_SIGNATURES = frozenset({"function_signature", "method_signature"})
-
-
-def _keep_ts_overloads_beside_a_body(symbols: list[Symbol], node_types: list[str]) -> list[Symbol]:
- """Drop a bodiless TS signature whose id has no implementation in the file.
-
- That is an ambient declaration (``declare function``, a ``.d.ts``), not an
- overload, and it never was a symbol.
- """
- bodied = {s.id for s, t in zip(symbols, node_types, strict=True) if t not in _TS_OVERLOAD_SIGNATURES}
- return [
- s
- for s, t in zip(symbols, node_types, strict=True)
- if t not in _TS_OVERLOAD_SIGNATURES or s.id in bodied
- ]
-
-
-def _drop_swift_extensions_of_local_types(symbols: list[Symbol]) -> list[Symbol]:
- """Drop an ``extension`` of a type the same file declares.
-
- It shares the type's id, so it would replace the type's node and kind; its
- members already name the type as their parent.
- """
- declared = {s.id for s in symbols if s.kind != "impl"}
- return [s for s in symbols if s.kind != "impl" or s.id not in declared]
-
-
-def _is_declaration(
- def_node: Node,
- config: LanguageConfig,
- language: str,
- export_type: _CppExportType | None,
- src: str,
-) -> bool:
- node_type = def_node.type
- if node_type in config.declaration_node_types:
- return True
- if language == "python":
- return _is_python_overload(def_node, src)
- if export_type is not None:
- return export_type.is_forward_declaration
- return _is_bodiless_cpp_type(language, node_type, def_node)
-
-
-def _objc_call_target(
- site_node: Node, target_node: Node, target_name: str, has_receiver: bool, src: str
-) -> str | None:
- """Objective-C call target name, or None to drop the match."""
- # A message send binds one `method:` child per keyword, so
- # `[view setTitle:t forState:s]` matches the query twice. Join the whole
- # selector on the first match and drop the rest.
- if site_node.type == "message_expression":
- return _objc_message_selector(site_node, target_node, src)
- # A block variable invoked with C call syntax (`completionBlock(hit)`) is
- # not a call to a same-named function or property.
- if not has_receiver and _objc_call_is_block_variable(site_node, target_name, src):
- return None
- return target_name
-
-
-def _jsx_supplied_props(site_node: Node, src: str) -> frozenset[str] | None:
- """Props written on a JSX element, or None when not JSX or a spread hides them."""
- if site_node.type not in ("jsx_self_closing_element", "jsx_opening_element"):
- return None
- props_set: set[str] = set()
- for child in site_node.children:
- if child.type == "jsx_attribute":
- name = next(
- (sub for sub in child.children if sub.type in ("property_identifier", "identifier")),
- None,
- )
- if name is not None:
- props_set.add(_node_text(name, src))
- elif child.type == "jsx_expression" and any(
- sub.type == "spread_element" for sub in child.children
- ):
- return None
- return frozenset(props_set)
-
-
-def _fold_receiverless_twins(calls: list[tuple[tuple[int, int, str], CallSite]]) -> list[CallSite]:
- """Fold the receiver-less copy of a member call into the copy with a receiver.
-
- A grammar's bare-call pattern that leaves the object unconstrained (Java's
- ``method_invocation``, Ruby's ``call``) matches ``obj.m()`` too. Resolved on
- its own, that copy binds ``m`` by bare name beside whatever ``obj``'s type
- answers, so it survives only as the member call's ``bare_name_fallback``.
- Keyed on the call node, not the line, so a bare ``m(obj.m())`` keeps its own
- site.
- """
- member_sites = {site for site, call in calls if not _is_bare(call)}
- bare_sites = {site for site, call in calls if site in member_sites and _is_bare(call)}
- folded: list[CallSite] = []
- for site, call in calls:
- if _is_bare(call):
- if site not in member_sites:
- folded.append(call)
- elif call.receiver_name and site in bare_sites:
- folded.append(replace(call, bare_name_fallback=True))
- else:
- folded.append(call)
- return folded
-
-
-def _is_bare(call: CallSite) -> bool:
- return not (call.receiver_name or call.receiver_call)
-
-
-def _dedupe_calls(calls: list[CallSite]) -> list[CallSite]:
- """One call per ``(line, target, receiver)``, keeping the richest record.
-
- Two scoped-call patterns can match the same two-part call (one keeps the
- qualifier), so the richer record wins whichever order they arrive in.
- """
- deduplicated: dict[tuple[int, str, str | None], CallSite] = {}
- for call in calls:
- key = (call.line, call.target_name, call.receiver_name)
- existing = deduplicated.get(key)
- if (
- existing is None
- or (existing.receiver_call is None and call.receiver_call is not None)
- or (existing.scope_name is None and call.scope_name is not None)
- ):
- deduplicated[key] = call
- return list(deduplicated.values())
-
-
-_FSHARP_BINDING_NODE_TYPES = ("function_declaration_left", "value_declaration_left")
-_DART_FUNCTION_NODE_TYPES = ("function_signature", "getter_signature", "setter_signature")
-
-
-def _is_fsharp_binding(language: str, node_type: str) -> bool:
- return language == "fsharp" and node_type in _FSHARP_BINDING_NODE_TYPES
-
-
-def _language_symbol_name(language: str, def_node: Node, name: str, src: str) -> str | None:
- """Apply a language's own naming rule; None drops the match."""
- if not name:
- return None
- if language == "elixir":
- # A `def` inside `quote do ... end` is macro body, not a
- # definition of the module that writes it.
- if _elixir_is_template_definition(def_node, src):
- return None
- # `defimpl Proto, for: Type` is named for the module the
- # compiler generates, not for the protocol alone.
- return _elixir_symbol_name(def_node, name, src)
- if language == "objectivec":
- # `typedef NS_ENUM(NSInteger, Kind) { ... }` has no grammar
- # rule, so only its enum *cases* survive as declarators and
- # each would become a symbol named as though it were the type.
- if _objc_is_macro_enum(def_node, src):
- return None
- # A method is named by its whole selector
- # (`initWithName:age:`), which no single node holds, and a
- # category by the class it extends plus its own name.
- return _objc_symbol_name(def_node, name, src)
- return name
-
-
-def _refine_symbol_kind(
- kind: str, def_node: Node, config: LanguageConfig, language: str, src: str
-) -> str:
- """Narrow a node-type kind where one node shape spells several kinds."""
- node_type = def_node.type
- # A value binding that carries parameter patterns beside its
- # name is a function the grammar reparsed as a value because
- # of its return-type annotation.
- if (
- language == "fsharp"
- and node_type == "value_declaration_left"
- and _fsharp_binding_has_params(def_node)
- ):
- kind = "function"
-
- # Refine "struct" kind for Go type_spec (check if struct or interface body)
- if kind == "struct" and config.parent_extraction == "receiver":
- kind = refine_go_type_kind(def_node, src)
-
- # Refine "class" kind for Kotlin (interface / enum class share class_declaration)
- if kind == "class" and language == "kotlin" and node_type == "class_declaration":
- kind = refine_kotlin_class_kind(def_node)
-
- if kind == "class" and language == "swift" and node_type == "class_declaration":
- kind = refine_swift_class_kind(def_node)
-
- # Pascal ``declType`` wraps class, record, interface, enum and more.
- if kind == "class" and language == "pascal" and node_type == "declType":
- kind = refine_pascal_type_kind(def_node)
-
- # Elixir: every definition is a ``call``; its keyword names the kind.
- if language == "elixir" and node_type == "call":
- kind = refine_elixir_call_kind(def_node, src)
-
- # F# writes a class, a struct and an interface with the same
- # ``anon_type_defn`` node; only the body says which it is.
- if kind == "class" and language == "fsharp" and node_type == "anon_type_defn":
- kind = refine_fsharp_type_kind(def_node)
- return kind
-
-
-def _symbol_end_line(
- def_node: Node,
- config: LanguageConfig,
- language: str,
- export_type: _CppExportType | None,
-) -> int | None:
- """The symbol's last line, or None for a Dart local function (not a symbol)."""
- node_type = def_node.type
- end_line = def_node.end_point[0] + 1
- if config.symbol_end_line_fn is not None:
- end_line = config.symbol_end_line_fn(def_node, end_line)
- if export_type is not None:
- end_line = export_type.range_node.end_point[0] + 1
- # F#: the captured left-hand side stops at the parameters; extend over the body.
- if _is_fsharp_binding(language, node_type):
- end_line = _fsharp_binding_end_line(def_node)
- if language == "dart" and node_type in _DART_FUNCTION_NODE_TYPES:
- return _dart_function_end_line(def_node, end_line)
- return end_line
-
-
-def _dart_function_end_line(def_node: Node, end_line: int) -> int | None:
- """Dart: a function's body is a sibling of its signature.
-
- Local functions nested inside another function's body have no callable
- *ancestor* (the enclosing signature is a sibling), so they are dropped
- here (None); and the line range extends to the trailing body sibling or
- call-site attribution stops at the signature line.
- """
- ancestor = def_node.parent
- while ancestor is not None:
- if ancestor.type in ("function_body", "function_expression"):
- return None
- ancestor = ancestor.parent
- anchor = def_node
- if def_node.parent is not None and def_node.parent.type == "method_signature":
- anchor = def_node.parent
- body_sibling = anchor.next_named_sibling
- if body_sibling is not None and body_sibling.type == "function_body":
- return body_sibling.end_point[0] + 1
- return end_line
-
-
-def _module_binding_kind(def_node: Node, name: str, language: str, src: str) -> str | None:
- """Kind of a module-level assignment, or None when it only binds a module.
-
- SCREAMING_CASE names are constants by convention; the rest are module
- variables (singletons like ``app = FastAPI()``, registries, caches).
- ``str.isupper()`` requires at least one cased char, so names with no
- letters (``_``, ``__all__``) fall to "variable" rather than being
- mislabelled constants by ``name == name.upper()``.
- """
- # TS/JS: ``const svc = require('./svc')`` binds a module (already an
- # import), so it is not a symbol.
- if language in _TS_JS_LANGUAGES and declarator_value_is_module_ref(def_node, src):
- return None
- # A structurally callable value is a function whatever its name says.
- callable_kind = (
- declarator_binds_callable(def_node, src) if language in _TS_JS_LANGUAGES else None
- )
- return callable_kind or ("constant" if name.isupper() else "variable")
-
-
-def _attribute_decorators(def_node: Node, language: str, src: str) -> list[str]:
- """Attribute texts that act as decorators (Rust, C#, C/C++), brackets stripped.
-
- They land beside ``@`` decorators so one deprecation check reads them all.
- """
- if language == "rust":
- return _rust_outer_attributes(def_node, src)
- # C#: [Obsolete] / [System.Obsolete] are ``attribute_list`` nodes.
- # In tree-sitter-c-sharp the attribute_list is child[0] of the
- # declaration node itself (method_declaration, class_declaration, etc.),
- # NOT a preceding sibling in the class body.
- if language == "csharp":
- return _leading_child_attributes(def_node, "attribute_list", "[", "]", src)
- # C/C++: [[deprecated]] / [[deprecated("reason")]] are
- # ``attribute_declaration`` nodes, child[0] of function_definition itself
- # (NOT a preceding sibling at translation_unit level).
- if language in ("cpp", "c"):
- return _leading_child_attributes(def_node, "attribute_declaration", "[[", "]]", src)
- if language == "kotlin":
- return _kotlin_detached_annotation(def_node, src)
- return []
-
-
-def _kotlin_detached_annotation(def_node: Node, src: str) -> list[str]:
- """Kotlin: annotations the grammar split off a top-level declaration.
-
- When the last annotation on its own line before a bare ``fun`` or
- ``object`` takes one argument (``@Suppress("unused")``), the argument list
- reads as a parenthesized expression: the annotations become an
- ``annotated_expression`` chain beside a declaration with no modifiers.
- """
- if def_node.parent is None or def_node.parent.type != "source_file":
- return []
- sib = def_node.prev_sibling
- while sib is not None and sib.type in ("line_comment", "block_comment"):
- sib = sib.prev_sibling
- return [_node_text(sib, src).strip()] if _is_split_annotation_chain(sib) else []
-
-
-def _is_split_annotation_chain(node: Node | None) -> bool:
- """Nested ``annotated_expression`` nodes of annotations, the last one's argument at the tail."""
- while node is not None and node.type == "annotated_expression":
- kinds = [c.type for c in node.named_children]
- if len(kinds) != 2 or kinds[0] != "annotation":
- return False
- if kinds[1] == "parenthesized_expression":
- return True
- node = node.named_children[1]
- return False
-
-
-def _declared_modifiers(def_node: Node, node_types: frozenset[str], src: str) -> tuple[str, ...]:
- """Lowercased keyword modifiers written in *def_node*'s modifier children.
-
- Keywords are the anonymous leaves of those children (C# ``modifier`` >
- ``override``, Kotlin ``modifiers`` > ``member_modifier`` > ``override``) or
- a named leaf that is itself a modifier (VB.NET ``modifier`` ``Overrides``).
- Named leaves such as Scala's ``protected[this]`` qualifier are not
- keywords. Annotations and attributes nested among them are skipped: they
- are decorators, not modifiers. The words are interned: a handful of
- distinct values repeat on every symbol.
- """
- if not node_types:
- return ()
- words: list[str] = []
- stack = [c for c in reversed(def_node.children) if c.type in node_types]
- while stack:
- node = stack.pop()
- if "annotation" in node.type or "attribute" in node.type:
- continue
- if node.child_count == 0:
- text = _node_text(node, src).strip()
- keyword = not node.is_named or "modifier" in node.type
- if keyword and text.isalpha():
- words.append(sys.intern(text.lower()))
- continue
- stack.extend(reversed(node.children))
- return tuple(words)
-
-
-def _rust_outer_attributes(def_node: Node, src: str) -> list[str]:
- """Rust: outer attributes (#[...]) are preceding siblings of the item."""
- attrs: list[str] = []
- if def_node.parent is None:
- return attrs
- siblings = def_node.parent.children
- for j, sib in enumerate(siblings):
- if sib.id != def_node.id:
- continue
- k = j - 1
- while k >= 0 and siblings[k].type == "attribute_item":
- attr_text = _node_text(siblings[k], src).strip()
- # Strip #[ and ] to get the inner attribute text
- if attr_text.startswith("#[") and attr_text.endswith("]"):
- attrs.append(attr_text[2:-1])
- k -= 1
- break
- return attrs
-
-
-def _leading_child_attributes(
- def_node: Node, node_type: str, opener: str, closer: str, src: str
-) -> list[str]:
- """Inner text of the attribute children leading *def_node*, until the first other child.
-
- A comment between attribute lists does not end the run, so an attribute
- placed after one is still recorded.
- """
- attrs: list[str] = []
- for child in def_node.children:
- if child.type != node_type:
- if child.type == "comment":
- continue
- break
- attr_text = _node_text(child, src).strip()
- if attr_text.startswith(opener) and attr_text.endswith(closer):
- attrs.append(attr_text[len(opener) : -len(closer)])
- return attrs
-
-
-def _refine_visibility(
- def_node: Node,
- language: str,
- visibility: str,
- name: str,
- ts_deferred_exports: frozenset[str] | None,
- py_all_names: frozenset[str] | None,
- src: str,
-) -> tuple[str, bool]:
- """``(visibility, is_exported_symbol)`` after the language's AST-context rules."""
- # C/C++ visibility is dictated by AST context (access
- # specifiers / storage class / export attributes), not by
- # modifier text. Refine after the generic fn ran. Objective-C's
- # grammar is a superset of C's, so the same shapes apply to
- # plain C code inside a ``.m`` file.
- if language in ("cpp", "c", "objectivec"):
- return refine_cpp_visibility(def_node, visibility, src)
- # C#: an unmodified declaration's default depends on what encloses
- # it, which the modifier-text fn cannot see.
- if language == "csharp":
- return refine_csharp_visibility(def_node, visibility), False
- # Java: no access keyword means package-private, except inside an
- # interface or annotation body, which the modifier-text fn cannot see.
- if language == "java":
- return refine_java_visibility(def_node), False
- # TS/JS: a top-level declaration is only public when exported —
- # inline, via ``export { x }`` lists, or ``export default x``.
- if language in _TS_JS_LANGUAGES:
- return refine_ts_visibility(def_node, visibility, name, ts_deferred_exports), False
- # Python: a literal ``__all__`` raises the names it lists to public; it
- # never demotes the ones it omits. GDScript shares ``py_visibility`` but
- # has no ``__all__``, so the gate is on the language, not the fn.
- if language == "python":
- return refine_py_visibility(def_node, visibility, name, py_all_names), False
- # Rust: a trait's items may not write ``pub`` of their own, so the
- # trait's modifier is the only place their visibility is stated.
- if language == "rust":
- return refine_rust_visibility(def_node, visibility, src), False
- return visibility, False
-
-
-def _dart_mixin_parent(def_node: Node, config: LanguageConfig, src: str) -> str | None:
- """Name of the Dart ``mixin`` enclosing *def_node*, if it is the nearest type."""
- ancestor = def_node.parent
- while ancestor is not None:
- if ancestor.type == "mixin_declaration":
- ident = next((c for c in ancestor.children if c.type == "identifier"), None)
- return _node_text(ident, src) if ident is not None else None
- if ancestor.type in config.parent_class_types:
- return None
- ancestor = ancestor.parent
- return None
-
-
-def _pascal_imports(
- stmt_node: Node, module_node: Node, src: str, seen_units: set[str]
-) -> list[Import]:
- """One Pascal ``uses`` unit, deduped case-insensitively across the file.
-
- pascal.scm fires once per unit name, so a 3-unit clause arrives as 3
- matches sharing one statement span: deduping by raw text would drop all
- but the first. A unit named in both ``uses`` clauses is one dependency.
- ``uses`` exposes the unit's whole interface, hence the ``*`` wildcard.
- """
- raw = _node_text(stmt_node, src).strip()
- unit_name = _node_text(module_node, src).strip()
- if not unit_name or unit_name.lower() in seen_units:
- return []
- seen_units.add(unit_name.lower())
- return [
- Import(
- raw_statement=raw,
- module_path=unit_name,
- imported_names=["*"],
- is_relative=False,
- resolved_file=None,
- bindings=[],
- is_reexport=False,
- )
- ]
-
-
-def _elixir_imports(
- stmt_node: Node, module_node: Node, src: str, seen_modules: set[str]
-) -> list[Import]:
- """Elixir directives, one Import per module and deduped by module path.
-
- ``alias Foo.{Bar, Baz}`` names two modules in one statement. ``import``
- pulls in every public function (``*``); alias/require/use bind the module.
- """
- raw = _node_text(stmt_node, src).split("\n", 1)[0].strip()
- directive_node = stmt_node.child_by_field_name("target")
- directive = _node_text(directive_node, src).strip() if directive_node else ""
- imports: list[Import] = []
- for module_path in elixir_import_modules(module_node, src):
- if module_path in seen_modules:
- continue
- seen_modules.add(module_path)
- imports.append(
- Import(
- raw_statement=raw,
- module_path=module_path,
- imported_names=["*"] if directive == "import" else [],
- is_relative=False,
- resolved_file=None,
- bindings=[],
- is_reexport=False,
- )
- )
- return imports
-
-
-def _fsharp_imports(
- stmt_node: Node, module_node: Node, src: str, seen_raws: set[str]
-) -> list[Import]:
- """F# ``open``: every public name of the module (``*``), which bare-name
- call resolution reads. ``open type A.B.T`` binds the type's static members,
- so the module the file depends on is ``A.B``.
- """
- raw = _node_text(stmt_node, src).strip()
- if raw in seen_raws:
- return []
- seen_raws.add(raw)
- module_path = _node_text(module_node, src).strip()
- if not module_path:
- return []
- names: list[str] = ["*"]
- if any(child.type == "type" for child in stmt_node.children):
- head, _, type_name = module_path.rpartition(".")
- if head:
- module_path, names = head, [type_name]
- else:
- names = []
- return [
- Import(
- raw_statement=raw,
- module_path=module_path,
- imported_names=names,
- is_relative=False,
- resolved_file=None,
- bindings=[],
- is_reexport=False,
- )
- ]
-
-
-def _statement_imports(
- stmt_node: Node,
- module_node: Node,
- module_text: str,
- raw: str,
- language: str,
- src: str,
-) -> list[Import]:
- """The imports one deduplicated statement declares."""
- if language == "scala" and stmt_node.type == "import_declaration":
- return _scala_imports(stmt_node, raw, src)
- if language == "php" and stmt_node.type == "namespace_use_declaration":
- return _php_imports(stmt_node, raw, src)
- if language == "dart":
- return [_dart_import(stmt_node, module_node, module_text, raw, src)]
- if language == "rust" and stmt_node.type == "use_declaration":
- return _rust_use_imports(stmt_node, module_text, raw, src)
- if language == "rust" and stmt_node.type == "macro_invocation":
- from .extractors.bindings.rust import macro_mod_imports
-
- # Most top-level calls (``s! { struct ... }``) declare no module.
- return macro_mod_imports(module_node, raw) if "mod" in raw else []
- if language in _TS_JS_LANGUAGES and _is_dynamic_esm_import(stmt_node):
- # ``import('./mod')`` binds a module namespace at runtime, so it is a
- # wildcard, which keeps the target's exports live. The names it is
- # destructured into still bind, for call resolution.
- return [
- Import(
- raw_statement=raw,
- module_path=module_text,
- imported_names=["*"],
- is_relative=module_text.startswith("."),
- resolved_file=None,
- bindings=dynamic_import_bindings(stmt_node, src),
- is_reexport=False,
- )
- ]
- if language in ("javascript", "typescript") and stmt_node.type in (
- "assignment_expression",
- "call_expression",
- ):
- return _cjs_imports(stmt_node, raw, src)
- return [_generic_import(stmt_node, module_text, raw, language, src)]
-
-
-def _scala_imports(stmt_node: Node, raw: str, src: str) -> list[Import]:
- """Scala: one Import per selected name, with full dotted paths.
-
- The query's ``(identifier)`` capture is only the first path segment, and
- one declaration can hold several clauses, brace selectors, renames and
- wildcards.
- """
- from .extractors.bindings.scala import expand_scala_import_clauses
- from .models import NamedBinding
-
- imports: list[Import] = []
- for clause_path, clause_names in expand_scala_import_clauses(stmt_node, src):
- local = clause_names[0]
- exported = None if local == "*" else clause_path.rsplit(".", 1)[-1]
- imports.append(
- Import(
- raw_statement=raw,
- module_path=clause_path,
- imported_names=clause_names,
- is_relative=False,
- resolved_file=None,
- bindings=[
- NamedBinding(
- local_name=local,
- exported_name=exported,
- source_file=None,
- )
- ],
- is_reexport=False,
- )
- )
- return imports
-
-
-def _php_imports(stmt_node: Node, raw: str, src: str) -> list[Import]:
- """PHP: one ``use`` declaration can name several classes, each its own file."""
- from .extractors.bindings.php import php_class_name, php_use_clauses
- from .models import NamedBinding
-
- return [
- Import(
- raw_statement=raw,
- module_path=fqn,
- imported_names=[php_class_name(fqn)],
- is_relative=False,
- resolved_file=None,
- bindings=[NamedBinding(local_name=local, exported_name=fqn, source_file=None)],
- is_reexport=False,
- )
- for fqn, local in php_use_clauses(stmt_node, src)
- ]
-
-
-def _dart_import(
- stmt_node: Node, module_node: Node, module_text: str, raw: str, src: str
-) -> Import:
- """Dart: URIs are relative unless schemed, ``export`` re-exports, and the
- dotted ``part of library.name;`` form resolves through ``library:``.
- """
- module_path = module_text
- if module_node.type == "dotted_identifier_list":
- module_path = f"library:{module_text}"
- imported_names, bindings = extract_import_bindings(stmt_node, src, "dart")
- return Import(
- raw_statement=raw,
- module_path=module_path,
- imported_names=imported_names,
- is_relative=not module_path.startswith(("package:", "dart:", "library:")),
- resolved_file=None,
- bindings=bindings,
- is_reexport=stmt_node.type == "library_export",
- )
-
-
-def _is_dynamic_esm_import(stmt_node: Node) -> bool:
- if stmt_node.type != "call_expression":
- return False
- fn = stmt_node.child_by_field_name("function")
- return fn is not None and fn.type == "import"
-
-
-def _cjs_imports(stmt_node: Node, raw: str, src: str) -> list[Import]:
- """CommonJS: every ``require()`` in the statement, re-exports marked.
-
- A hub like ``Object.assign(module.exports, require('./a'), ...)`` is
- several imports, and ``module.exports`` shapes are barrels.
- """
- from .extractors.bindings.ts_js import (
- cjs_statement_is_reexport,
- collect_cjs_requires,
- )
-
- cjs_reexport = cjs_statement_is_reexport(stmt_node, src)
- return [
- Import(
- raw_statement=raw,
- module_path=cjs_module,
- imported_names=["*"] if cjs_reexport else [],
- is_relative=cjs_module.startswith("."),
- resolved_file=None,
- bindings=[],
- is_reexport=cjs_reexport,
- )
- for cjs_module in collect_cjs_requires(stmt_node, src)
- ]
-
-
-def _generic_import(
- stmt_node: Node, module_text: str, raw: str, language: str, src: str
-) -> Import:
- """The single Import of a statement no language-specific shape claims."""
- is_module_declaration = language == "rust" and stmt_node.type == "mod_item"
- if is_module_declaration:
- module_text = _rust_mod_path_attribute(stmt_node, src) or module_text
-
- # JVM wildcard imports: the query captures the scoped identifier only,
- # so ``import com.foo.*`` arrives as ``com.foo``. Restore the ``.*``.
- if language in ("java", "kotlin") and not module_text.endswith("*"):
- stmt_text = raw.rstrip().rstrip(";").rstrip()
- if stmt_text.endswith(".*"):
- module_text += ".*"
-
- # Language-specific import name + binding extraction
- imported_names, bindings = extract_import_bindings(stmt_node, src, language)
- is_relative = (
- module_text.startswith(".")
- or module_text.startswith("./")
- or module_text.startswith(("self::", "super::", "crate::"))
- )
- # ``type_only`` / ``deferred`` are set by the caller from ``import_load_kind``.
- return Import(
- raw_statement=raw,
- module_path=module_text,
- imported_names=imported_names,
- is_relative=is_relative,
- resolved_file=None,
- bindings=bindings,
- is_reexport=_is_reexport_import(stmt_node, raw, language),
- is_module_declaration=is_module_declaration,
- )
-
-
-def _rust_use_imports(stmt_node: Node, module_text: str, raw: str, src: str) -> list[Import]:
- """Rust: one Import per leaf of the use tree, each with its own path.
-
- ``use crate::{a::B, c::D}`` names two modules, and the joined string
- resolves to neither. ``super`` hops that only climb out of inline
- ``mod`` blocks stay inside this file, so they are dropped here and the
- resolver sees the path the file itself would write.
- """
- from .extractors.bindings.rust import expand_rust_use_tree, rust_use_argument
- from .models import NamedBinding
-
- arg_node = rust_use_argument(stmt_node)
- leaves = expand_rust_use_tree(arg_node, src) if arg_node is not None else []
- inline_depth = _rust_inline_mod_depth(stmt_node)
- is_reexport = _is_reexport_import(stmt_node, raw, "rust")
- if len(leaves) <= 1 and "{" not in module_text and not inline_depth:
- return [_generic_import(stmt_node, module_text, raw, "rust", src)]
- imports = []
- for path, local, exported in leaves:
- path = _strip_inline_super(path, inline_depth)
- imports.append(
- Import(
- raw_statement=raw,
- module_path=path,
- imported_names=[local],
- is_relative=path.startswith(("self::", "super::", "crate::")),
- resolved_file=None,
- bindings=[NamedBinding(local_name=local, exported_name=exported, source_file=None)],
- is_reexport=is_reexport,
- )
- )
- return imports
-
-
-def _rust_inline_mod_depth(node: Node) -> int:
- """How many inline ``mod name { ... }`` blocks enclose *node* in its file."""
- depth = 0
- parent = node.parent
- while parent is not None:
- if parent.type == "mod_item":
- depth += 1
- parent = parent.parent
- return depth
-
-
-def _strip_inline_super(path: str, inline_depth: int) -> str:
- """Drop the ``super`` hops of *path* that stay inside the file.
-
- In ``mod tests { use super::*; }`` the ``super`` is this file's own
- module, so the path is ``self::*``, not the parent module's glob.
- """
- if not inline_depth:
- return path
- segments = path.split("::")
- hops = 0
- while hops < len(segments) and segments[hops] == "super":
- hops += 1
- if not hops:
- return path
- if hops <= inline_depth:
- return "::".join(["self", *segments[hops:]])
- return "::".join(segments[inline_depth:])
-
-
-def _rust_mod_path_attribute(stmt_node: Node, src: str) -> str | None:
- """The ``#[path = "..."]`` override on a Rust ``mod`` item, if any.
-
- Outer attributes are preceding siblings of the item, not children.
- """
- parent = stmt_node.parent
- if parent is None:
- return None
- siblings = parent.children
- for j, sib in enumerate(siblings):
- if sib.id != stmt_node.id:
- continue
- # Walk backward through preceding attribute_item siblings
- k = j - 1
- while k >= 0 and siblings[k].type == "attribute_item":
- path_match = re.search(r'path\s*=\s*"([^"]+)"', _node_text(siblings[k], src))
- if path_match:
- return path_match.group(1)
- k -= 1
- return None
- return None
-
-
-def _is_reexport_import(stmt_node: Node, raw: str, language: str) -> bool:
- """``pub use`` (Rust) and ``@_exported import`` (Swift) re-export the module."""
- if language == "rust" and stmt_node.type == "use_declaration":
- return any(child.type == "visibility_modifier" for child in stmt_node.children)
- return language == "swift" and raw.startswith("@_exported")
-
-
-def _parse_tree(
- lang: str, language: Language, grammar_tag: str, source: bytes
-) -> tuple[Node, list[str], str, Language]:
- """``(root, parse errors, grammar tag, language)``; a JSX-bearing ``.ts`` retries as TSX."""
- parser = Parser(language)
- tree = parser.parse(source)
- root = tree.root_node
-
- parse_errors = _collect_error_nodes(root)
-
- # Adaptive TSX grammar fallback: if a .ts file contains JSX markup,
- # tree-sitter-typescript produces ERROR nodes. Re-parse using the TSX
- # grammar ONLY IF:
- # 1. Initial parse yielded error nodes (parse_errors is non-empty)
- # 2. The file projects to TypeScript and is not already on tsx
- # 3. Source contains JSX-specific closing tokens (b"/>" or b"</")
- # A .vue render function may be written in JSX
- # (``vnodes.push(<i class={c} />)``), which is the single TS parse
- # failure across a 1,593-file .vue corpus. ``source`` here is the
- # markup-blanked projection, so the template's own ``</`` and ``/>``
- # are already spaces — the token test still keys on real JSX only.
- # The swap is strictly safe: it only takes effect when TSX yields
- # FEWER errors than the first parse.
- if (
- parse_errors
- and lang in ("typescript", "vue")
- and grammar_tag != "tsx"
- and (b"/>" in source or b"</" in source)
- ):
- tsx_language = _get_language("tsx")
- if tsx_language is not None:
- tsx_tree = Parser(tsx_language).parse(source)
- tsx_errors = _collect_error_nodes(tsx_tree.root_node)
- if len(tsx_errors) < len(parse_errors):
- tree = tsx_tree
- root = tree.root_node
- parse_errors = tsx_errors
- # Both grammar_tag AND language must be reassigned.
- # grammar_tag is consumed immediately below by
- # the caller's _get_query(lang, language, grammar_tag), which
- # appends tsx.scm to the base typescript.scm query.
- # That append is what supplies the
- # jsx_opening_element / jsx_self_closing_element captures
- # that restore JSX component call-site edges.
- # Reassigning only ``tree`` / ``root`` would fix parse
- # errors but leave those edges missing — the dead-code
- # false-positive would remain.
- grammar_tag = "tsx"
- language = tsx_language
- return root, parse_errors, grammar_tag, language
-
-
-def _parent_type_name(ancestor: Node, src: str) -> str | None:
- """The name a class/impl ancestor gives its members, without type arguments."""
- name_node = ancestor.child_by_field_name("name") or (
- ancestor.child_by_field_name("type") # Rust impl_item
- )
- if name_node is None:
- return None
- # For Rust impl blocks with generic types (e.g. impl<T> Foo<T>),
- # extract only the base type name, not the full generic signature.
- if name_node.type in ("generic_type", "scoped_type_identifier"):
- inner = name_node.child_by_field_name(
- "type" if name_node.type == "generic_type" else "name"
- )
- if inner and inner.type == "type_identifier":
- name_node = inner
- return _node_text(name_node, src)
-
-
-def _nesting_parent(def_node: Node, config: LanguageConfig, src: str) -> str | None:
- """The class/impl ancestor naming *def_node*'s parent.
-
- An anonymous class is a class of its own, so its members gain segments
- below it: ``Gson::FILTER::Filter::check``.
- """
- segments: list[str] = []
- child, ancestor = def_node, def_node.parent
- while ancestor is not None:
- if ancestor.type in config.parent_class_types:
- name = _parent_type_name(ancestor, src)
- if name is not None:
- return "::".join([name, *reversed(segments)])
- else:
- segment = scope_segment(
- ancestor, child, bool(segments), config.anonymous_class_types, src
- )
- if segment is not None:
- segments.append(segment)
- child, ancestor = ancestor, ancestor.parent
- # A Kotlin top-level function's object expression: ``f::Foo``.
- return "::".join(reversed(segments)) or None
-
-
-def _java_enum_constant(def_node: Node, config: LanguageConfig, src: str) -> str | None:
- """The enum constant whose body declares *def_node*, before any enclosing type.
-
- An anonymous class is such a type: ``_find_parent`` names its members.
- """
- ancestor = def_node.parent
- while (
- ancestor is not None
- and ancestor.type not in config.parent_class_types
- and ancestor.type not in config.anonymous_class_types
- ):
- if ancestor.type == "enum_constant":
- name = ancestor.child_by_field_name("name")
- return _node_text(name, src) if name else None
- ancestor = ancestor.parent
- return None
-
-
-class ASTParser:
- """Unified AST parser — works for all languages via .scm query files.
-
- Usage::
-
- parser = ASTParser()
- parsed = parser.parse_file(file_info, source_bytes)
-
- Adding a new language:
- 1. Write ``packages/core/queries/<lang>.scm``
- 2. Add one entry to ``LANGUAGE_CONFIGS``
- That's it. No Python class, no new module.
- """
-
- def __init__(self) -> None:
- pass
-
- def parse_imports(self, file_info: FileInfo, source: bytes) -> list[Import]:
- """Only *source*'s imports: :meth:`parse_file`'s grammar, query and import pass.
-
- Symbols, calls and references are not extracted, which is most of a
- parse. A language without a tree-sitter path takes the full parse.
- """
- lang = file_info.language
- config = LANGUAGE_CONFIGS.get(lang)
- grammar_tag = grammar_tag_for(lang, file_info.path)
- language = _get_language(grammar_tag)
- signature_file = lang == "fsharp" and file_info.path.endswith(".fsi")
- if (
- lang in SPECIAL_HANDLER_LANGUAGES
- or config is None
- or language is None
- or signature_file
- ):
- return self.parse_file(file_info, source).imports
- source = prepare_source(lang, source, path=file_info.path)
- root, _, grammar_tag, language = _parse_tree(lang, language, grammar_tag, source)
- query = self._get_query(lang, language, grammar_tag)
- matches = _run_query(query, root) if query is not None else []
- src = source.decode("utf-8", errors="replace")
- return self._extract_imports(matches, config, file_info, src)
-
- def parse_file(self, file_info: FileInfo, source: bytes) -> ParsedFile:
- """Parse *source* bytes and return a fully populated ParsedFile."""
- lang = file_info.language
- content_hash = compute_content_hash(source)
-
- # Non-tree-sitter formats (OpenAPI, Dockerfile, Makefile, SQL) parse
- # via dedicated handlers. Checked before the grammar lookup: none of
- # these tags carry a LanguageConfig, so the no-grammar fallback below
- # would otherwise swallow them.
- if lang in SPECIAL_HANDLER_LANGUAGES:
- parsed = parse_special(file_info, source, lang)
- parsed.content_hash = content_hash
- return parsed
-
- config = LANGUAGE_CONFIGS.get(lang)
- # .tsx needs the JSX-aware grammar: the default one's ERROR-node
- # recovery hoists nested helpers (handlers defined inside component
- # bodies) to the top level.
- grammar_tag = grammar_tag_for(lang, file_info.path)
- language = _get_language(grammar_tag)
-
- # tree-sitter-fsharp ships a second grammar (``language_signature``)
- # for .fsi signature files, and a spec loads exactly one. Read with
- # the implementation grammar, a signature file's ``val`` and member
- # signatures land in ERROR recovery, which hoists whatever the
- # recovery invents into the symbol list. The regex tier still gives
- # these files their ``open`` imports, which is all a signature file
- # contributes that another file does not also state.
- signature_file = lang == "fsharp" and file_info.path.endswith(".fsi")
-
- if config is None or language is None or signature_file:
- if config is not None and language is None and lang not in _MISSING_GRAMMAR_REPORTED:
- # Once per language, not once per file: the fact is about the
- # environment, and it does not become truer on the four
- # thousandth shell script.
- _MISSING_GRAMMAR_REPORTED.add(lang)
- log.debug("tree-sitter grammar unavailable", language=lang)
- # Languages without a grammar may still carry regex-tier import
- # extraction (their specs declare import_support="partial");
- # symbols stay empty — the regex tier claims no symbol knowledge.
- from .lightweight_imports import extract_lightweight_imports
-
- return ParsedFile(
- file_info=file_info,
- symbols=[],
- imports=extract_lightweight_imports(file_info, source),
- exports=[],
- docstring=None,
- parse_errors=[],
- content_hash=content_hash,
- )
-
- # An SFC (.svelte, .vue) is three languages in one file.
- # ``prepare_source`` blanks the markup and <style> so what reaches the
- # TypeScript grammar is valid TS at byte-identical offsets — no offset
- # translation is needed anywhere downstream. A no-op for every other
- # language without a registered locator/sanitizer -- Pascal's
- # sanitizers (project-file `in '...'` clauses, ERROR-node blanking)
- # live behind the same hook; see prepare_pascal_source in
- # parser_helpers.py for why they're wired in here rather than as
- # ad-hoc if-blocks.
- # ``content_hash`` above deliberately hashes the ORIGINAL bytes, so
- # incremental update still tracks the real file.
- original_source = source
- source = prepare_source(lang, source, path=file_info.path)
-
- root, parse_errors, grammar_tag, language = _parse_tree(lang, language, grammar_tag, source)
- src = source.decode("utf-8", errors="replace")
-
- query = self._get_query(lang, language, grammar_tag)
-
- # Execute the compiled query ONCE per file. The five extraction
- # passes below all consume the same capture dicts read-only;
- # re-running ``cursor.matches()`` per pass multiplied the most
- # expensive part of parsing by five.
- matches = _run_query(query, root) if query is not None else []
-
- symbols = self._extract_symbols(matches, config, file_info, src)
- # Per-language synthetic-symbol pass — recognises source-generator
- # attributes (e.g. CommunityToolkit.Mvvm) and adds the symbols the
- # generator would emit at compile time. No-op for languages
- # without a registered extractor.
- synthetic = extract_synthetic_symbols(root, src, file_info)
- if synthetic:
- # Keyed with the discriminator too, so a record's synthetic
- # canonical constructor survives beside an explicit overload of
- # another arity. A language without one dedupes on the id alone.
- existing = {(s.id, symbol_discriminator(s)) for s in symbols}
- symbols.extend(s for s in synthetic if (s.id, symbol_discriminator(s)) not in existing)
- if lang == "csharp":
- # The grammar reads a C# 14 extension block as a constructor
- # named ``extension``; its members were added above instead.
- symbols = drop_extension_block_symbols(symbols, root, src)
- # Before calls and references are extracted, so both attribute to the
- # final ids. Anonymous members go last, once the overloads holding them
- # carry their discriminators.
- anonymous = [s for s in symbols if is_anonymous_member(s)]
- disambiguate_colliding_ids([s for s in symbols if not is_anonymous_member(s)], lang)
- follow_renamed_hosts(symbols)
- disambiguate_colliding_ids(anonymous, lang)
- imports = self._extract_imports(matches, config, file_info, src)
- calls = self._extract_calls(matches, config, file_info, src, symbols)
- # An SFC instantiates a component by writing its tag in the markup
- # (``<Foo />``), which the blanked TS buffer no longer contains. Mint
- # those call sites from the markup grammar directly — the same way
- # tsx.scm turns ``<Component />`` into a call for React. Returns [] for
- # every non-SFC language.
- calls.extend(component_call_sites(lang, original_source, symbols))
- references = (
- self._extract_references(matches, file_info, src, symbols, imports)
- if lang in _REFERENCE_LANGUAGES
- else []
- )
- heritage = extract_heritage(matches, config, file_info, src)
- exports = self._derive_exports(symbols, config)
- export_aliases = ts_export_aliases(src) if lang in _TS_JS_LANGUAGES else {}
- docstring = extract_module_docstring(root, src, lang)
- type_refs = self._extract_type_refs(matches, src, lang)
-
- # Same-file reference rescue (Python only): top-level symbols used
- # elsewhere in their own module in a non-call / non-import position
- # (callable passed as an arg, type annotation, decorator, default)
- # carry no graph edge, so the dead-code unused-export pass would flag
- # them. Stamp the referenced names so the analyzer can rescue them.
- local_refs, registrars = (
- _python_module_facts(src, symbols) if lang == "python" else (frozenset(), frozenset())
- )
-
- if len(symbols) > _SYMBOL_COUNT_WARN_THRESHOLD:
- log.warning(
- "parser.symbol_bloat",
- path=file_info.path,
- language=lang,
- symbol_count=len(symbols),
- threshold=_SYMBOL_COUNT_WARN_THRESHOLD,
- )
-
- return ParsedFile(
- file_info=file_info,
- symbols=symbols,
- imports=imports,
- exports=exports,
- export_aliases=export_aliases,
- calls=calls,
- heritage=heritage,
- docstring=docstring,
- parse_errors=parse_errors,
- content_hash=content_hash,
- type_refs=type_refs,
- local_refs=local_refs,
- registrars=registrars,
- references=references,
- )
-
- # ------------------------------------------------------------------
- # Query loading
- # ------------------------------------------------------------------
-
- def _get_query(
- self, lang: str, language: Language, grammar_tag: str | None = None
- ) -> object | None:
- """Load and cache the compiled tree-sitter Query for *lang*."""
- return _load_compiled_query(lang, grammar_tag)
-
- # ------------------------------------------------------------------
- # Symbol extraction
- # ------------------------------------------------------------------
-
- def _extract_symbols(
- self,
- matches: list[dict],
- config: LanguageConfig,
- file_info: FileInfo,
- src: str,
- ) -> list[Symbol]:
- language = file_info.language
- symbols: list[Symbol] = []
- seen: set[tuple[int, str]] = set() # (start_line, name) — dedup decorated dupes
- # Parallel to ``symbols`` (same indices) -- consumed for Pascal and
- # Objective-C, to dedupe interface-declaration vs. implementation
- # method pairs after the loop, and for TS overload signatures. See
- # _dedupe_pascal_interface_symbols / _dedupe_objc_interface_symbols.
- node_types: list[str] = []
- # Also parallel to ``symbols``, Objective-C only: which of @interface /
- # @implementation / @protocol declared each member, so a protocol's
- # method is never deduped against a same-named class method.
- objc_container_kinds: list[str | None] = []
-
- # Deferred-export names (``export { x }`` / ``export default x``),
- # computed once per file for the TS/JS visibility refinement.
- ts_deferred_exports: frozenset[str] | None = None
- if language in _TS_JS_LANGUAGES:
- ts_deferred_exports = ts_deferred_export_names(src)
- # Literal module ``__all__`` names, once per file for the Python
- # refinement. None means no signal (built at runtime or absent).
- py_all_names: frozenset[str] | None = None
- if language == "python":
- py_all_names = py_module_all_names(src)
- cpp_exports = (
- collect_cpp_export_types(matches, src) if language == "cpp" else CppExportTypes()
- )
-
- for capture_dict in matches:
- built = self._symbol_from_match(
- capture_dict,
- config,
- file_info,
- src,
- cpp_exports,
- ts_deferred_exports,
- py_all_names,
- seen,
- )
- if built is None:
- continue
- symbol, def_node = built
- symbols.append(symbol)
- node_types.append(def_node.type)
- if language == "objectivec":
- container = _objc_container_node(def_node, config.parent_class_types)
- objc_container_kinds.append(container.type if container else None)
-
- if language == "pascal":
- symbols = _dedupe_pascal_interface_symbols(symbols, node_types)
-
- if language in _TS_JS_LANGUAGES:
- symbols = _keep_ts_overloads_beside_a_body(symbols, node_types)
-
- if language == "swift":
- symbols = _drop_swift_extensions_of_local_types(symbols)
-
- # A .m file routinely declares its private methods in a class
- # extension and defines them below in the @implementation, which
- # builds each symbol id twice in one file.
- if language == "objectivec":
- symbols = _dedupe_objc_interface_symbols(symbols, node_types, objc_container_kinds)
-
- return symbols
-
- def _symbol_from_match(
- self,
- capture_dict: dict,
- config: LanguageConfig,
- file_info: FileInfo,
- src: str,
- cpp_exports: CppExportTypes,
- ts_deferred_exports: frozenset[str] | None,
- py_all_names: frozenset[str] | None,
- seen: set[tuple[int, str]],
- ) -> tuple[Symbol, Node] | None:
- """One query match as a symbol and its definition node, or None to drop it.
-
- Records the match's ``(start_line, name)`` in *seen* once it passes the
- name checks, so a later duplicate is dropped even if this one is.
- """
- language = file_info.language
- identity = _match_identity(capture_dict, config, language, src, cpp_exports, seen)
- if identity is None:
- return None
- def_node, name_nodes, name, export_type, start_line = identity
-
- node_type = def_node.type
- # A TS/JS object-literal method inside an anonymous callback is kept,
- # qualified by the binding that owns its object.
- object_owner = (
- _ts_nested_object_method_owner(def_node, src, config.symbol_node_types)
- if language in _TS_JS_LANGUAGES
- else None
- )
- kind = self._symbol_kind(
- def_node,
- config,
- language,
- src,
- cpp_exports.container_ids,
- # An anonymous class member is a method of its own class, even
- # inside another method's body; ``_find_parent`` names it.
- keep_nested=object_owner is not None
- or in_anonymous_class(def_node, config.anonymous_class_types),
- )
- if kind is None:
- return None
-
- end_line = _symbol_end_line(def_node, config, language, export_type)
- if end_line is None:
- return None
-
- if node_type in _MODULE_ANCHORED_NODE_TYPES:
- kind = _module_binding_kind(def_node, name, language, src)
- if kind is None:
- return None
-
- modifier_texts = [_node_text(m, src) for m in capture_dict.get("symbol.modifiers", [])]
- visibility, is_exported_symbol = _refine_visibility(
- def_node,
- language,
- config.visibility_fn(name, modifier_texts),
- name,
- ts_deferred_exports,
- py_all_names,
- src,
- )
-
- parent_name = object_owner or self._resolve_parent_name(
- def_node,
- config,
- capture_dict.get("symbol.receiver", []),
- name_nodes,
- language,
- src,
- no_export_type=export_type is None,
- export_type_parents=cpp_exports.parents,
- )
-
- # A ``field_declaration`` cannot occur outside a class body, so a
- # missing parent means the class did not parse. Grammar recovery,
- # not a member function.
- if node_type == "function_declarator" and parent_name is None:
- return None
-
- # Upgrade function → method when a parent class is detected.
- # F#: a nested module is a parent too (for id uniqueness), but it
- # is not a type, so a `let` inside one stays a function.
- if parent_name and kind == "function" and (
- language != "fsharp" or _fsharp_parent_is_type(def_node)
- ):
- kind = "method"
-
- params_nodes = capture_dict.get("symbol.params", [])
- params_text = _node_text(params_nodes[0], src) if params_nodes else ""
- # The id scope adds a Java enum constant to its parent, so each
- # constant's override is its own node while the enum stays its owner.
- constant = _java_enum_constant(def_node, config, src) if language == "java" else None
- scope = f"{parent_name}::{constant}" if parent_name and constant else parent_name
- sym_id = f"{file_info.path}::{scope}::{name}" if scope else f"{file_info.path}::{name}"
- symbol = Symbol(
- id=sym_id,
- name=name,
- qualified_name=_build_qualified_name(file_info.path, scope, name),
- kind=kind, # type: ignore[arg-type]
- signature=build_signature(node_type, name, params_text, def_node, src),
- start_line=start_line,
- end_line=end_line,
- docstring=extract_symbol_docstring(def_node, src, language),
- decorators=(
- [m for m in modifier_texts if m.startswith("@")]
- + _attribute_decorators(def_node, language, src)
- ),
- visibility=visibility, # type: ignore[arg-type]
- is_async=_is_async_node(def_node, src),
- language=language,
- parent_name=parent_name,
- is_exported_symbol=is_exported_symbol,
- is_declaration=_is_declaration(def_node, config, language, export_type, src),
- type_parameter_count=(
- _csharp_type_parameter_count(def_node) if language == "csharp" else None
- ),
- **_kotlin_receiver_fields(def_node, params_nodes, language, name),
- modifiers=_declared_modifiers(def_node, config.modifier_node_types, src),
- )
- return symbol, def_node
-
- def _symbol_kind(
- self,
- def_node: Node,
- config: LanguageConfig,
- language: str,
- src: str,
- container_ids: frozenset[int],
- *,
- keep_nested: bool = False,
- ) -> str | None:
- """The symbol kind for *def_node*, or None when it is not a symbol here."""
- node_type = def_node.type
- kind = config.symbol_node_types.get(node_type)
- if kind is None:
- return None
-
- # Only module-level and class-body members are symbols; the query is
- # recursive, so defs nested in a callable are dropped. Module-anchored
- # node types only match at module level and skip the check, as does a
- # def the caller has placed already (``keep_nested``).
- if (
- not keep_nested
- and node_type not in _MODULE_ANCHORED_NODE_TYPES
- and _has_callable_ancestor(def_node, config.symbol_node_types, container_ids)
- ):
- return None
-
- # F#: a ``let`` nested in another binding has no callable ancestor, so
- # it is checked separately. A ``let`` in a type body is a field and stays.
- if _is_fsharp_binding(language, node_type) and _fsharp_binding_is_nested(def_node):
- return None
- return _refine_symbol_kind(kind, def_node, config, language, src)
-
- def _resolve_parent_name(
- self,
- def_node: Node,
- config: LanguageConfig,
- receiver_nodes: list[Node],
- name_nodes: list[Node],
- language: str,
- src: str,
- *,
- no_export_type: bool,
- export_type_parents: dict[int, str],
- ) -> str | None:
- """The enclosing type or module name, trying each language's own shape."""
- parent_name = self._find_parent(def_node, config, receiver_nodes, src)
- if parent_name is not None:
- return parent_name
-
- if language == "cpp" and no_export_type:
- parent_name = _cpp_export_macro_parent(def_node, export_type_parents)
- if parent_name is not None:
- return parent_name
-
- # Dart mixin_declaration exposes no ``name`` field, so
- # ``_find_parent``'s field lookup misses mixin members.
- if language == "dart":
- return _dart_mixin_parent(def_node, config, src)
-
- # F#: no type node carries a ``name`` field -- the name hangs off
- # a ``type_name`` child -- so the generic walk finds the ancestor
- # and then reads nothing off it.
- if language == "fsharp":
- return _fsharp_parent_name(def_node, src)
-
- # C/C++ out-of-line ``void Foo::method()`` names its class in the scope.
- if language in ("cpp", "c") and name_nodes:
- return _qualified_cpp_parent(name_nodes[0], src)
-
- # Pascal out-of-line ``function TFoo.Bar`` names its class in the header.
- if language == "pascal" and name_nodes:
- return _qualified_pascal_parent(name_nodes[0], src)
-
- # Elixir: ``defmodule`` is a ``call``; the name is its first argument.
- if language == "elixir":
- return _elixir_module_parent(def_node, src)
-
- # Objective-C containers name themselves with a bare first identifier.
- if language == "objectivec":
- return _objc_container_parent(def_node, config.parent_class_types, src)
- return None
-
- def _find_parent(
- self,
- def_node: Node,
- config: LanguageConfig,
- receiver_nodes: list[Node],
- src: str,
- ) -> str | None:
- """Determine the parent class/type for a symbol."""
- if config.parent_extraction == "receiver":
- # Go: extract type name from receiver parameter list
- if receiver_nodes:
- return extract_go_receiver_type(_node_text(receiver_nodes[0], src))
- return None
-
- if config.parent_extraction in ("nesting", "impl"):
- return _nesting_parent(def_node, config, src)
-
- return None # "none" mode
-
- # ------------------------------------------------------------------
- # Import extraction
- # ------------------------------------------------------------------
-
- def _extract_imports(
- self,
- matches: list[dict],
- config: LanguageConfig,
- file_info: FileInfo,
- src: str,
- ) -> list[Import]:
- language = file_info.language
- imports: list[Import] = []
- seen_raws: set[str] = set()
- # Statement key -> the Imports it produced, so a repeat folds its load
- # kind into them rather than adding a duplicate.
- seen_imports: dict[str, list[Import]] = {}
- seen_pascal_units: set[str] = set()
- seen_elixir_modules: set[str] = set()
-
- for capture_dict in matches:
- stmt_nodes = capture_dict.get("import.statement", [])
- module_nodes = capture_dict.get("import.module", [])
-
- if not stmt_nodes or not module_nodes:
- continue
-
- stmt_node = stmt_nodes[0]
-
- # These three dedupe on their own key, before the raw-statement
- # dedup below.
- if language == "pascal":
- imports.extend(_pascal_imports(stmt_node, module_nodes[0], src, seen_pascal_units))
- continue
- if language == "elixir":
- imports.extend(
- _elixir_imports(stmt_node, module_nodes[0], src, seen_elixir_modules)
- )
- continue
- if language == "fsharp":
- imports.extend(_fsharp_imports(stmt_node, module_nodes[0], src, seen_raws))
- continue
-
- raw = _node_text(stmt_node, src).strip()
- # A Rust ``mod`` item's text excludes its outer attributes, so two
- # ``#[cfg]``-gated ``mod imp;`` declarations targeting different
- # ``#[path]`` files collide on the same raw text and the second
- # file silently loses its import edge (and is then reported as
- # dead code). Qualify the dedup key with the path override.
- dedup_key = raw
- if language == "rust" and stmt_node.type == "mod_item":
- mod_path_attr = _rust_mod_path_attribute(stmt_node, src)
- if mod_path_attr is not None:
- dedup_key = f"{raw}|path={mod_path_attr}"
- # Two ``import('./m')`` calls share their text but not the names
- # each one binds.
- if language in _TS_JS_LANGUAGES and _is_dynamic_esm_import(stmt_node):
- dedup_key = f"{raw}|at={stmt_node.start_byte}"
- kind = import_load_kind(stmt_node, language, src)
- if dedup_key in seen_imports:
- # A lazy copy must not hide a load-time twin: runtime wins.
- for imp in seen_imports[dedup_key]:
- imp.type_only, imp.deferred = combine_load_kinds(
- (imp.type_only, imp.deferred), kind
- )
- continue
- seen_imports[dedup_key] = []
-
- module_text = _node_text(module_nodes[0], src).strip().strip("\"'` ")
- if not module_text:
- continue
-
- found = _statement_imports(stmt_node, module_nodes[0], module_text, raw, language, src)
- for imp in found:
- imp.type_only, imp.deferred = kind
- seen_imports[dedup_key] = found
- imports.extend(found)
-
- if language == "python":
- imports = expand_bare_relative_imports(imports)
-
- return imports
-
- # ------------------------------------------------------------------
- # Call extraction
- # ------------------------------------------------------------------
-
- def _extract_calls(
- self,
- matches: list[dict],
- config: LanguageConfig,
- file_info: FileInfo,
- src: str,
- symbols: list[Symbol],
- ) -> list[CallSite]:
- """Extract function/method call sites from the AST."""
- from .language_data import get_builtin_calls
-
- _call_builtins = get_builtin_calls(file_info.language)
-
- symbol_ranges = sorted(
- [(s.start_line, s.end_line, s.id) for s in symbols],
- key=lambda t: (t[0], -t[1]),
- )
-
- # Each call as ``(site, call)``; see ``_fold_receiverless_twins``.
- calls: list[tuple[tuple[int, int, str], CallSite]] = []
-
- for capture_dict in matches:
- site_nodes = capture_dict.get("call.site", [])
- target_nodes = capture_dict.get("call.target", [])
- arg_nodes = capture_dict.get("call.arguments", [])
- receiver_nodes = capture_dict.get("call.receiver", [])
- receiver_call_nodes = capture_dict.get("call.receiver_call", [])
- scope_nodes = capture_dict.get("call.scope", [])
-
- if not site_nodes or not target_nodes:
- continue
-
- site_node = site_nodes[0]
- target_node = target_nodes[0]
- target_name = config.call_target_name_fn(
- _node_text(target_node, src).strip(), target_node.type
- )
- if not target_name:
- continue
-
- if file_info.language == "objectivec":
- target_name = _objc_call_target(
- site_node, target_node, target_name, bool(receiver_nodes), src
- )
- if target_name is None:
- continue
-
- if target_name in _call_builtins:
- continue
-
- # Elixir: a definition head (`add(a, b)` in `def add(a, b)`) and a
- # module attribute (`@doc "..."`) are both ``call`` nodes, and no
- # query predicate can see the parent that tells them apart. Left
- # in, every function in the repo would call itself.
- if file_info.language == "elixir" and _elixir_call_is_definitional(site_node, src):
- continue
-
- line = site_node.start_point[0] + 1
- receiver_name = None
- if receiver_nodes:
- receiver_name = _receiver_text(file_info.language, receiver_nodes[0], src)
- if receiver_name is None:
- continue
- if receiver_name and file_info.language == "php":
- receiver_name = _normalize_php_receiver(receiver_name)
- # F#: a dotted static path (``Path.Combine(a, b)``) collapses into
- # one identifier node in this grammar -- there is no dot node to
- # capture a receiver from -- so the split happens on the text.
- # After the builtin check above, so a name is filtered as written.
- if file_info.language == "fsharp" and receiver_name is None and "." in target_name:
- receiver_part, _, target_name = target_name.rpartition(".")
- receiver_name = receiver_part.strip()
- target_name = target_name.strip()
- if not target_name:
- continue
- receiver_call = (
- _call_receiver_from_node(receiver_call_nodes[0], src)
- if receiver_call_nodes
- else None
- )
- scope_name = _node_text(scope_nodes[0], src).strip() if scope_nodes else None
-
- arg_count = _count_arguments(arg_nodes[0]) if arg_nodes else None
- caller_id = _find_enclosing_symbol(line, symbol_ranges)
- arguments = _overload_arguments(file_info.language, arg_nodes)
-
- calls.append(
- (
- (site_node.start_byte, site_node.end_byte, target_name),
- CallSite(
- target_name=target_name,
- receiver_name=receiver_name,
- caller_symbol_id=caller_id,
- line=line,
- argument_count=arg_count,
- receiver_call=receiver_call,
- argument_types=_argument_literal_types(arguments, src),
- argument_names=_argument_names(arguments, src),
- scope_name=scope_name,
- edge_type=(
- "references"
- if site_node.type in config.reference_call_node_types
- else "calls"
- ),
- supplied_props=_jsx_supplied_props(site_node, src),
- # Ceiling: schedulers matched by name, the task followed
- # within its own function only (see ``python_tasks``).
- spawned=(
- file_info.language == "python" and is_fire_and_forget(site_node, src)
- ),
- ),
- )
- )
- return _dedupe_calls(_fold_receiverless_twins(calls))
-
- def _extract_references(
- self,
- matches: list[dict],
- file_info: FileInfo,
- src: str,
- symbols: list[Symbol],
- imports: list[Import],
- ) -> list[CallSite]:
- """Extract sites that name a function without calling it.
-
- Six shapes carry a function by name and never call it where a parser
- can see: a dispatch-table entry, a callback field, an argument to a
- registration macro, a func value in argument position, a struct-field
- initialiser, and a ``::`` callable reference. Each leaves the named
- function with no inbound edge, which read as a ``safe_to_delete``
- unused export and took out whole handler and interop layers (#1602).
- TS/JS and Python capture any value position: an argument, a list or
- object entry, a return, an assignment and a default export.
-
- ``@reference.receiver`` is optional; capturing it is what lets
- ``_add_reference_edges`` restrict a bare name to free functions and
- allow a qualified name to reach a method.
-
- Self-gating on the reference captures, so a language whose query
- defines none produces nothing and pays two dict lookups. Two guards
- keep the broad syntactic positions from claiming ordinary code:
-
- * A macro argument requires a SCREAMING_CASE callee and must be that
- macro's only argument. Uppercase alone is not enough: assertion
- macros are spelled the same way and take values, so ``EXPECT_EQ(
- capacity, 100)`` bound a local to whatever free function shared its
- name. Registering something registers one thing, which separates
- ``BENCHMARK(BM_Foo)`` from ``TEST(SuiteName, TestName)``.
- * A table entry must sit outside any function body. A dispatch table is
- a file-, namespace- or class-scope aggregate; the same braces inside
- a function are a constructor member-init or a local aggregate, where
- ``{data, size}`` names parameters. Measured on fmt, admitting those
- turned ``data``, ``size``, ``begin``, ``end``, ``capacity`` and
- ``buffer`` into edges, every one of them wrong.
-
- Whether the name denotes a function at all is settled later, at
- resolution, where the symbol kind is known.
- """
- from .language_data import get_builtin_calls
-
- builtins = get_builtin_calls(file_info.language)
-
- symbol_ranges = sorted(
- [(s.start_line, s.end_line, s.id) for s in symbols],
- key=lambda t: (t[0], -t[1]),
- )
- callable_ids = {s.id for s in symbols if s.kind in ("function", "method")}
- bindable = _bare_reference_names(file_info.language, symbols, imports)
-
- references: list[CallSite] = []
- seen: set[tuple[int, str, str | None]] = set()
-
- for capture_dict in matches:
- plain_nodes = capture_dict.get("reference.name", [])
- table_nodes = capture_dict.get("reference.table", [])
- if not plain_nodes and not table_nodes:
- continue
-
- receiver_nodes = capture_dict.get("reference.receiver", [])
- receiver = (
- _node_text(receiver_nodes[0], src).strip() if receiver_nodes else None
- ) or None
-
- via_nodes = capture_dict.get("reference.via", [])
- if via_nodes:
- via = _node_text(via_nodes[0], src).strip()
- if not via or not via.isupper():
- continue
- arg_list = plain_nodes[0].parent if plain_nodes else None
- if arg_list is None or len(arg_list.named_children) != 1:
- continue
-
- candidates = [(node, False) for node in plain_nodes]
- candidates += [(node, True) for node in table_nodes]
- for name_node, is_table in candidates:
- name = _node_text(name_node, src).strip()
- if not name or name in builtins:
- continue
- if bindable is not None and receiver is None and name not in bindable:
- continue
- line = name_node.start_point[0] + 1
- enclosing = _find_enclosing_symbol(line, symbol_ranges)
- if is_table and enclosing in callable_ids:
- continue
- if (line, name, receiver) in seen:
- continue
- seen.add((line, name, receiver))
- references.append(
- CallSite(
- target_name=name,
- receiver_name=receiver,
- caller_symbol_id=enclosing,
- line=line,
- argument_count=None,
- )
- )
-
- return references
-
- # ------------------------------------------------------------------
- # Export derivation
- # ------------------------------------------------------------------
-
- def _derive_exports(
- self,
- symbols: list[Symbol],
- config: LanguageConfig,
- ) -> list[str]:
- """Derive the list of exported names from parsed symbols.
-
- Note on TS/JS: export visibility is resolved upstream during symbol
- extraction by ``refine_ts_visibility()``, which demotes non-exported
- declarations to private. This expression relies on that classification to
- filter top-level public symbols accurately for TS/JS.
- """
- return [s.name for s in symbols if s.visibility == "public" and s.parent_name is None]
-
- # ------------------------------------------------------------------
- # Type reference extraction (non-import positions)
- # ------------------------------------------------------------------
-
- def _extract_type_refs(
- self,
- matches: list[dict],
- src: str,
- lang: str = "",
- ) -> list[TypeReference]:
- """Collect ``@param.type`` captures into TypeReference records.
-
- C# emits these from constructor / method / delegate / primary-ctor
- parameter types; Go emits them from parameter, struct-field, return,
- and composite-literal type positions (see ``go.scm``). The graph
- builder resolves each reference to a defining file via the
- language-specific resolver index and emits a file-level edge.
-
- The head-identifier extractor is language-specific (Go unwraps
- ``*T`` / ``[]T`` / ``map[K]V`` / ``pkg.T``); see
- ``TYPE_HEAD_EXTRACTORS``. Capture origin is inferred from the
- enclosing node: ``constructor_declaration`` → ``ctor_param``,
- ``method_declaration`` → ``method_param`` (C#);
- ``field_declaration`` → ``field_type``, ``composite_literal`` →
- ``composite_literal`` (Go).
- """
- head_of = TYPE_HEAD_EXTRACTORS.get(lang, _head_type_identifier)
-
- refs: list[TypeReference] = []
- seen: set[tuple[str, int]] = set()
-
- for capture_dict in matches:
- type_nodes = capture_dict.get("param.type", [])
- if not type_nodes:
- continue
- for type_node in type_nodes:
- head = head_of(type_node, src)
- if not head:
- continue
- # ``struct Wrapper<Item> { value: Item }`` binds Item as a type
- # parameter, and the capture cannot tell that from a reference
- # to a real ``struct Item``. The head extractor drops a
- # single-letter ``T`` but not a named one, so the shadow has to
- # be read off the enclosing item's ``type_parameters``.
- if lang == "rust" and _rust_shadowed_by_type_param(type_node, head, src):
- continue
- line = type_node.start_point[0] + 1
- key = (head, line)
- if key in seen:
- continue
- seen.add(key)
- origin = _classify_param_origin(type_node)
- refs.append(TypeReference(type_name=head, line=line, origin=origin))
-
- return refs
-
-
-# ---------------------------------------------------------------------------
-# Convenience function
-# ---------------------------------------------------------------------------
-
-_DEFAULT_PARSER: ASTParser | None = None
-
-
-def _python_module_facts(
- src: str, symbols: list[Symbol]
-) -> tuple[frozenset[str], frozenset[str]]:
- """``(local_refs, registrars)`` for a Python module, from one ``ast`` parse.
-
- See ``python_local_refs`` and ``python_registrars``.
- """
- top_level_names = {s.name for s in symbols if s.name and not s.parent_name}
- tree = parse_python_module(src) if top_level_names else None
- if tree is None:
- return frozenset(), frozenset()
- nested_classes = {
- (s.parent_name, s.name) for s in symbols if s.kind == "class" and s.parent_name
- }
- return (
- local_refs_from_tree(tree, top_level_names, nested_classes),
- registering_functions(tree),
- )
-
-
-def parse_file(file_info: FileInfo, source: bytes) -> ParsedFile:
- """Module-level convenience: parse a file using the default ASTParser."""
- global _DEFAULT_PARSER
- if _DEFAULT_PARSER is None:
- _DEFAULT_PARSER = ASTParser()
- return _DEFAULT_PARSER.parse_file(file_info, source)
diff --git a/packages/core/src/repowise/core/ingestion/parser/__init__.py b/packages/core/src/repowise/core/ingestion/parser/__init__.py
new file mode 100644
index 000000000..aa5023817
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/__init__.py
@@ -0,0 +1,324 @@
+"""Unified AST parser — one class for all languages.
+
+Architecture
+============
+Per-language differences live in two places:
+ 1. ``packages/core/queries/<lang>.scm`` — tree-sitter S-expression queries
+ that capture symbols and imports using consistent capture-name conventions.
+ 2. ``LANGUAGE_CONFIGS`` (``language_configs.py``) — a ``LanguageConfig`` per
+ language that maps node types to symbol kinds, defines visibility rules,
+ and names its import extractor (``extractors/imports/<lang>.py``).
+
+The package splits one parse into its passes: ``grammars`` (grammar and query
+loading, the tree-sitter parse), ``symbols`` and ``symbol_rules``, ``imports``,
+``calls`` and ``references``. Every name the old single module defined is
+re-exported here.
+
+Capture-name conventions (shared across ALL .scm files):
+ @symbol.def — the full definition node (line numbers, kind lookup)
+ @symbol.name — name identifier
+ @symbol.params — parameter list (optional)
+ @symbol.modifiers — decorators / visibility modifiers (optional)
+ @symbol.receiver — Go method receiver (optional, used for parent detection)
+ @import.statement — full import node
+ @import.module — module path being imported
+"""
+
+# ruff: noqa: F401
+
+from __future__ import annotations
+
+from tree_sitter import Language
+
+from ..extractors import extract_heritage, extract_module_docstring
+from ..extractors.imports.dart import _dart_import
+from ..extractors.imports.elixir import _elixir_imports
+from ..extractors.imports.fsharp import _fsharp_imports
+from ..extractors.imports.generic import _generic_import
+from ..extractors.imports.pascal import _pascal_imports
+from ..extractors.imports.php import _php_imports
+from ..extractors.imports.rust import (
+ _rust_inline_mod_depth,
+ _rust_mod_path_attribute,
+ _rust_use_imports,
+ _strip_inline_super,
+)
+from ..extractors.imports.scala import _scala_imports
+from ..extractors.imports.ts_js import _cjs_imports, _is_dynamic_esm_import
+from ..extractors.visibility import ts_export_aliases
+from ..language_configs import LANGUAGE_CONFIGS, LanguageConfig
+from ..models import FileInfo, Import, ParsedFile, Symbol, compute_content_hash
+from ..parser_helpers import _run_query
+from ..python_local_refs import local_refs_from_tree, parse_python_module
+from ..python_registrars import registering_functions
+from ..sfc_source import component_call_sites, prepare_source
+from ..special_handlers import SPECIAL_HANDLER_LANGUAGES, parse_special
+from .calls import (
+ _DOTTED_PATH,
+ _PATH_RECEIVERS,
+ _call_receiver_from_node,
+ _dart_argument_count,
+ _dart_call_receiver,
+ _dedupe_calls,
+ _extract_calls,
+ _fold_receiverless_twins,
+ _is_bare,
+ _jsx_supplied_props,
+ _normalize_php_receiver,
+ _objc_call_target,
+ _receiver_text,
+)
+from .grammars import (
+ _LANGUAGE_REGISTRY,
+ _MISSING_GRAMMAR_REPORTED,
+ _PASSTHROUGH_LANGUAGES,
+ QUERIES_DIR,
+ _build_language_registry,
+ _compile_query,
+ _get_language,
+ _load_compiled_query,
+ _parse_tree,
+ failed_query_languages,
+ grammar_tag_for,
+ log,
+ missing_grammar_languages,
+)
+from .imports import _extract_imports
+from .references import (
+ _NAMED_BINDING_LANGUAGES,
+ _REFERENCE_LANGUAGES,
+ _bare_reference_names,
+ _extract_references,
+ _extract_type_refs,
+)
+from .symbol_rules import (
+ _DART_FUNCTION_NODE_TYPES,
+ _FSHARP_BINDING_NODE_TYPES,
+ _TS_JS_LANGUAGES,
+ _TS_OVERLOAD_SIGNATURES,
+ _attribute_decorators,
+ _dart_function_end_line,
+ _dart_mixin_parent,
+ _declared_modifiers,
+ _drop_swift_extensions_of_local_types,
+ _is_declaration,
+ _is_fsharp_binding,
+ _is_split_annotation_chain,
+ _keep_ts_overloads_beside_a_body,
+ _kotlin_detached_annotation,
+ _language_symbol_name,
+ _leading_child_attributes,
+ _module_binding_kind,
+ _node_text,
+ _refine_symbol_kind,
+ _refine_visibility,
+ _rust_outer_attributes,
+ _symbol_end_line,
+)
+from .symbols import (
+ _MODULE_ANCHORED_NODE_TYPES,
+ _extract_symbols,
+ _finalize_symbols,
+ _java_enum_constant,
+ _match_identity,
+ _nesting_parent,
+ _parent_type_name,
+)
+
+# A file with more symbols than this is almost certainly generated; warn, never truncate.
+_SYMBOL_COUNT_WARN_THRESHOLD = 500
+
+
+class ASTParser:
+ """Unified AST parser — works for all languages via .scm query files.
+
+ Usage::
+
+ parser = ASTParser()
+ parsed = parser.parse_file(file_info, source_bytes)
+
+ Adding a new language:
+ 1. Write ``packages/core/queries/<lang>.scm``
+ 2. Add one entry to ``LANGUAGE_CONFIGS``
+ That's it. No Python class, no new module.
+ """
+
+ def __init__(self) -> None:
+ pass
+
+ def parse_imports(self, file_info: FileInfo, source: bytes) -> list[Import]:
+ """Only *source*'s imports: :meth:`parse_file`'s grammar, query and import pass.
+
+ Symbols, calls and references are not extracted, which is most of a
+ parse. A language without a tree-sitter path takes the full parse.
+ """
+ lang = file_info.language
+ config = LANGUAGE_CONFIGS.get(lang)
+ grammar_tag = grammar_tag_for(lang, file_info.path)
+ language = _get_language(grammar_tag)
+ if (
+ lang in SPECIAL_HANDLER_LANGUAGES
+ or config is None
+ or language is None
+ or _is_fsharp_signature(file_info)
+ ):
+ return self.parse_file(file_info, source).imports
+ source = prepare_source(lang, source, path=file_info.path)
+ root, _, grammar_tag, language = _parse_tree(lang, language, grammar_tag, source)
+ query = _load_compiled_query(lang, grammar_tag)
+ matches = _run_query(query, root) if query is not None else []
+ src = source.decode("utf-8", errors="replace")
+ return _extract_imports(matches, config, file_info, src)
+
+ def parse_file(self, file_info: FileInfo, source: bytes) -> ParsedFile:
+ """Parse *source* bytes and return a fully populated ParsedFile."""
+ lang = file_info.language
+ content_hash = compute_content_hash(source)
+
+ # Non-tree-sitter formats (OpenAPI, Dockerfile, Makefile, SQL) carry no
+ # LanguageConfig, so they are routed before the no-grammar fallback.
+ if lang in SPECIAL_HANDLER_LANGUAGES:
+ parsed = parse_special(file_info, source, lang)
+ parsed.content_hash = content_hash
+ return parsed
+
+ config = LANGUAGE_CONFIGS.get(lang)
+ grammar_tag = grammar_tag_for(lang, file_info.path)
+ language = _get_language(grammar_tag)
+ if config is None or language is None or _is_fsharp_signature(file_info):
+ return _parse_without_grammar(file_info, source, config, language, content_hash)
+
+ # An SFC (.svelte, .vue) is blanked to its script at byte-identical
+ # offsets; ``content_hash`` above keeps hashing the original bytes.
+ original_source = source
+ source = prepare_source(lang, source, path=file_info.path)
+
+ root, parse_errors, grammar_tag, language = _parse_tree(lang, language, grammar_tag, source)
+ src = source.decode("utf-8", errors="replace")
+
+ # Execute the compiled query ONCE per file; every pass below reads
+ # the same capture dicts.
+ query = _load_compiled_query(lang, grammar_tag)
+ matches = _run_query(query, root) if query is not None else []
+
+ symbols = _finalize_symbols(
+ _extract_symbols(matches, config, file_info, src), root, src, file_info
+ )
+ imports = _extract_imports(matches, config, file_info, src)
+ calls = _extract_calls(matches, config, file_info, src, symbols)
+ # An SFC's markup tags (``<Foo />``) are component calls the TS buffer lost.
+ calls.extend(component_call_sites(lang, original_source, symbols))
+ references = (
+ _extract_references(matches, file_info, src, symbols, imports)
+ if lang in _REFERENCE_LANGUAGES
+ else []
+ )
+ # Python names used in their own module outside a call or import,
+ # stamped so dead-code can rescue them.
+ local_refs, registrars = (
+ _python_module_facts(src, symbols) if lang == "python" else (frozenset(), frozenset())
+ )
+ _warn_symbol_bloat(file_info, symbols)
+
+ return ParsedFile(
+ file_info=file_info,
+ symbols=symbols,
+ imports=imports,
+ exports=_derive_exports(symbols),
+ export_aliases=ts_export_aliases(src) if lang in _TS_JS_LANGUAGES else {},
+ calls=calls,
+ heritage=extract_heritage(matches, config, file_info, src),
+ docstring=extract_module_docstring(root, src, lang),
+ parse_errors=parse_errors,
+ content_hash=content_hash,
+ type_refs=_extract_type_refs(matches, src, lang),
+ local_refs=local_refs,
+ registrars=registrars,
+ references=references,
+ )
+
+
+def _is_fsharp_signature(file_info: FileInfo) -> bool:
+ """An F# ``.fsi`` signature file needs a grammar the spec does not load.
+
+ Read with the implementation grammar, its ``val`` signatures land in
+ ERROR recovery; the regex tier still gives it its ``open`` imports.
+ """
+ return file_info.language == "fsharp" and file_info.path.endswith(".fsi")
+
+
+def _parse_without_grammar(
+ file_info: FileInfo,
+ source: bytes,
+ config: LanguageConfig | None,
+ language: Language | None,
+ content_hash: str,
+) -> ParsedFile:
+ """Regex-tier imports and no symbols, for a file with no usable grammar."""
+ lang = file_info.language
+ if config is not None and language is None and lang not in _MISSING_GRAMMAR_REPORTED:
+ # Once per language, not once per file: the fact is about the environment.
+ _MISSING_GRAMMAR_REPORTED.add(lang)
+ log.debug("tree-sitter grammar unavailable", language=lang)
+ from ..lightweight_imports import extract_lightweight_imports
+
+ return ParsedFile(
+ file_info=file_info,
+ symbols=[],
+ imports=extract_lightweight_imports(file_info, source),
+ exports=[],
+ docstring=None,
+ parse_errors=[],
+ content_hash=content_hash,
+ )
+
+
+def _warn_symbol_bloat(file_info: FileInfo, symbols: list[Symbol]) -> None:
+ if len(symbols) > _SYMBOL_COUNT_WARN_THRESHOLD:
+ log.warning(
+ "parser.symbol_bloat",
+ path=file_info.path,
+ language=file_info.language,
+ symbol_count=len(symbols),
+ threshold=_SYMBOL_COUNT_WARN_THRESHOLD,
+ )
+
+
+def _derive_exports(symbols: list[Symbol]) -> list[str]:
+ """Top-level public symbol names.
+
+ TS/JS visibility was already refined to "exported" during symbol extraction.
+ """
+ return [s.name for s in symbols if s.visibility == "public" and s.parent_name is None]
+
+
+_DEFAULT_PARSER: ASTParser | None = None
+
+
+def _python_module_facts(
+ src: str, symbols: list[Symbol]
+) -> tuple[frozenset[str], frozenset[str]]:
+ """``(local_refs, registrars)`` for a Python module, from one ``ast`` parse.
+
+ See ``python_local_refs`` and ``python_registrars``.
+ """
+ top_level_names = {s.name for s in symbols if s.name and not s.parent_name}
+ tree = parse_python_module(src) if top_level_names else None
+ if tree is None:
+ return frozenset(), frozenset()
+ nested_classes = {
+ (s.parent_name, s.name) for s in symbols if s.kind == "class" and s.parent_name
+ }
+ return (
+ local_refs_from_tree(tree, top_level_names, nested_classes),
+ registering_functions(tree),
+ )
+
+
+def parse_file(file_info: FileInfo, source: bytes) -> ParsedFile:
+ """Module-level convenience: parse a file using the default ASTParser."""
+ global _DEFAULT_PARSER
+ if _DEFAULT_PARSER is None:
+ _DEFAULT_PARSER = ASTParser()
+ return _DEFAULT_PARSER.parse_file(file_info, source)
+
diff --git a/packages/core/src/repowise/core/ingestion/parser/calls.py b/packages/core/src/repowise/core/ingestion/parser/calls.py
new file mode 100644
index 000000000..d52086c9d
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/calls.py
@@ -0,0 +1,372 @@
+"""Call-site extraction: what each call names, on which receiver, from which symbol."""
+
+from __future__ import annotations
+
+import re
+from dataclasses import replace
+
+from tree_sitter import Node
+
+from ..extractors import node_text
+from ..language_configs import LanguageConfig
+from ..models import CallReceiver, CallSite, FileInfo, Symbol
+from ..parser_helpers import (
+ _argument_literal_types,
+ _argument_names,
+ _count_arguments,
+ _elixir_call_is_definitional,
+ _find_enclosing_symbol,
+ _objc_call_is_block_variable,
+ _objc_message_selector,
+ _overload_arguments,
+)
+from ..python_tasks import is_fire_and_forget
+
+_node_text = node_text
+
+def _call_receiver_from_node(node: Node, src: str) -> CallReceiver | None:
+ """Describe an inner call captured as another call's receiver.
+
+ The queries identify the complete inner AST node. This helper reads only
+ named tree-sitter fields, so nested arguments and formatting cannot change
+ which call is carried into resolution.
+ """
+
+ if node.type == "identifier":
+ return _dart_call_receiver(node, src)
+ arguments = node.child_by_field_name("arguments")
+ argument_count = _count_arguments(arguments) if arguments is not None else None
+
+ target = node.child_by_field_name("name")
+ receiver = node.child_by_field_name("object")
+ if target is None:
+ function = node.child_by_field_name("function")
+ if function is None:
+ return None
+ if function.type == "attribute":
+ return _python_attribute_receiver(function, src, argument_count)
+ target, receiver = _function_target(function)
+
+ if target is None:
+ return None
+ target_name = _node_text(target, src).strip()
+ if not target_name:
+ return None
+
+ receiver_name = None
+ if receiver is not None and receiver.type in ("identifier", "this"):
+ receiver_name = _node_text(receiver, src).strip() or None
+ return CallReceiver(target_name, receiver_name, argument_count)
+
+
+def _function_target(function: Node) -> tuple[Node | None, Node | None]:
+ """``(target, receiver)`` named by an inner call's ``function`` field."""
+ if function.type in ("identifier", "property_identifier", "field_identifier"):
+ return function, None
+ if function.type == "generic_name":
+ target = function.child_by_field_name("name") or next(
+ (child for child in function.named_children if child.type == "identifier"), None
+ )
+ return target, None
+ target = (
+ function.child_by_field_name("name")
+ or function.child_by_field_name("property")
+ or function.child_by_field_name("field")
+ )
+ receiver = (
+ function.child_by_field_name("expression")
+ or function.child_by_field_name("object")
+ or function.child_by_field_name("argument")
+ or function.child_by_field_name("operand")
+ )
+ return target, receiver
+
+
+def _python_attribute_receiver(
+ function: Node, src: str, argument_count: int | None
+) -> CallReceiver | None:
+ """Python ``obj.m()``: the receiver is kept as a plain dotted path, and a
+ call on anything else is not carried at all.
+ """
+ target = function.child_by_field_name("attribute")
+ obj = function.child_by_field_name("object")
+ path = (
+ _receiver_text("python", obj, src)
+ if obj is not None and obj.type in ("identifier", "attribute")
+ else None
+ )
+ if target is None or path is None:
+ return None
+ return CallReceiver(_node_text(target, src).strip(), path, argument_count)
+
+
+def _dart_call_receiver(name: Node, src: str) -> CallReceiver | None:
+ """The Dart call whose *name* the query captured: ``a.b(..)``, ``B(..)``.
+
+ A Dart call is a flat selector run with no call node, so the name's
+ siblings are read: the selector after it holds the arguments, and the
+ node before a ``.b`` selector is its receiver.
+ """
+ if name.parent.type == "unconditional_assignable_selector":
+ head = name.parent.parent
+ prev = head.prev_named_sibling
+ if prev is not None and any(c.type == "argument_part" for c in prev.named_children):
+ receiver = None
+ else:
+ receiver = _node_text(prev, src).strip() if prev is not None else None
+ else:
+ head, receiver = name, None
+ return CallReceiver(
+ _node_text(name, src).strip(), receiver, _dart_argument_count(head.next_named_sibling)
+ )
+
+
+def _dart_argument_count(selector: Node | None) -> int | None:
+ """The arguments a Dart ``(selector (argument_part (arguments)))`` passes."""
+ parts = selector.named_children if selector is not None else ()
+ found = [arg for part in parts for arg in part.named_children if arg.type == "arguments"]
+ return _count_arguments(found[0]) if found else None
+
+
+
+# The receiver captures that are themselves a member access, by language (the
+# queries widen only these two), and what a plain dotted path of names looks
+# like once whitespace is gone.
+_PATH_RECEIVERS = frozenset({("typescript", "member_expression"), ("python", "attribute")})
+_DOTTED_PATH = re.compile(r"[A-Za-z_$][\w$]*(?:\.[A-Za-z_$][\w$]*)+")
+
+
+def _receiver_text(language: str, node: Node, src: str) -> str | None:
+ """The receiver's text, or None to drop the site.
+
+ A dotted receiver (``this.a.b.m()``) is kept only as a plain path of
+ names: a call, subscript or optional hop inside it types nothing, and
+ dropping the site keeps it off the bare-call tiers.
+ """
+ sibling = node.next_named_sibling
+ if language == "java" and node.type != "super" and getattr(sibling, "type", None) == "super":
+ # ``S.super.m()``: ``S`` qualifies the ``super`` its own pattern captures.
+ return None
+ text = _node_text(node, src).strip()
+ if (language, node.type) not in _PATH_RECEIVERS:
+ return text
+ path = "".join(text.split())
+ return path if _DOTTED_PATH.fullmatch(path) else None
+
+
+def _normalize_php_receiver(text: str) -> str:
+ """Spell PHP's receiver the way the resolver's strategies expect.
+
+ `self::` and `static::` are the same dispatch as `$this`, and the self/this
+ strategy tests `in ("self", "this")` — so without this the whole implicit-
+ receiver population misses. `parent::` needs the heritage walk and is left
+ to miss rather than guessed at.
+ """
+ name = text.lstrip("$")
+ return "this" if name in ("self", "static") else name
+
+
+def _objc_call_target(
+ site_node: Node, target_node: Node, target_name: str, has_receiver: bool, src: str
+) -> str | None:
+ """Objective-C call target name, or None to drop the match."""
+ # A message send binds one `method:` child per keyword, so
+ # `[view setTitle:t forState:s]` matches the query twice. Join the whole
+ # selector on the first match and drop the rest.
+ if site_node.type == "message_expression":
+ return _objc_message_selector(site_node, target_node, src)
+ # A block variable invoked with C call syntax (`completionBlock(hit)`) is
+ # not a call to a same-named function or property.
+ if not has_receiver and _objc_call_is_block_variable(site_node, target_name, src):
+ return None
+ return target_name
+
+
+def _jsx_supplied_props(site_node: Node, src: str) -> frozenset[str] | None:
+ """Props written on a JSX element, or None when not JSX or a spread hides them."""
+ if site_node.type not in ("jsx_self_closing_element", "jsx_opening_element"):
+ return None
+ props_set: set[str] = set()
+ for child in site_node.children:
+ if child.type == "jsx_attribute":
+ name = next(
+ (sub for sub in child.children if sub.type in ("property_identifier", "identifier")),
+ None,
+ )
+ if name is not None:
+ props_set.add(_node_text(name, src))
+ elif child.type == "jsx_expression" and any(
+ sub.type == "spread_element" for sub in child.children
+ ):
+ return None
+ return frozenset(props_set)
+
+
+def _fold_receiverless_twins(calls: list[tuple[tuple[int, int, str], CallSite]]) -> list[CallSite]:
+ """Fold the receiver-less copy of a member call into the copy with a receiver.
+
+ A grammar's bare-call pattern that leaves the object unconstrained (Java's
+ ``method_invocation``, Ruby's ``call``) matches ``obj.m()`` too. Resolved on
+ its own, that copy binds ``m`` by bare name beside whatever ``obj``'s type
+ answers, so it survives only as the member call's ``bare_name_fallback``.
+ Keyed on the call node, not the line, so a bare ``m(obj.m())`` keeps its own
+ site.
+ """
+ member_sites = {site for site, call in calls if not _is_bare(call)}
+ bare_sites = {site for site, call in calls if site in member_sites and _is_bare(call)}
+ folded: list[CallSite] = []
+ for site, call in calls:
+ if _is_bare(call):
+ if site not in member_sites:
+ folded.append(call)
+ elif call.receiver_name and site in bare_sites:
+ folded.append(replace(call, bare_name_fallback=True))
+ else:
+ folded.append(call)
+ return folded
+
+
+def _is_bare(call: CallSite) -> bool:
+ return not (call.receiver_name or call.receiver_call)
+
+
+def _dedupe_calls(calls: list[CallSite]) -> list[CallSite]:
+ """One call per ``(line, target, receiver)``, keeping the richest record.
+
+ Two scoped-call patterns can match the same two-part call (one keeps the
+ qualifier), so the richer record wins whichever order they arrive in.
+ """
+ deduplicated: dict[tuple[int, str, str | None], CallSite] = {}
+ for call in calls:
+ key = (call.line, call.target_name, call.receiver_name)
+ existing = deduplicated.get(key)
+ if (
+ existing is None
+ or (existing.receiver_call is None and call.receiver_call is not None)
+ or (existing.scope_name is None and call.scope_name is not None)
+ ):
+ deduplicated[key] = call
+ return list(deduplicated.values())
+
+
+def _extract_calls(
+ matches: list[dict],
+ config: LanguageConfig,
+ file_info: FileInfo,
+ src: str,
+ symbols: list[Symbol],
+) -> list[CallSite]:
+ """Extract function/method call sites from the AST."""
+ from ..language_data import get_builtin_calls
+
+ call_builtins = get_builtin_calls(file_info.language)
+
+ symbol_ranges = sorted(
+ [(s.start_line, s.end_line, s.id) for s in symbols],
+ key=lambda t: (t[0], -t[1]),
+ )
+
+ # Each call as ``(site, call)``; see ``_fold_receiverless_twins``.
+ calls: list[tuple[tuple[int, int, str], CallSite]] = []
+ for capture_dict in matches:
+ found = _call_from_match(
+ capture_dict, config, file_info.language, src, call_builtins, symbol_ranges
+ )
+ if found is not None:
+ calls.append(found)
+ return _dedupe_calls(_fold_receiverless_twins(calls))
+
+
+def _call_from_match(
+ capture_dict: dict,
+ config: LanguageConfig,
+ language: str,
+ src: str,
+ call_builtins: frozenset[str],
+ symbol_ranges: list[tuple[int, int, str]],
+) -> tuple[tuple[int, int, str], CallSite] | None:
+ """One call match as ``(site key, CallSite)``, or None to drop it."""
+ site_nodes = capture_dict.get("call.site", [])
+ target_nodes = capture_dict.get("call.target", [])
+ if not site_nodes or not target_nodes:
+ return None
+
+ site_node = site_nodes[0]
+ names = _call_names(
+ capture_dict, site_node, target_nodes[0], config, language, src, call_builtins
+ )
+ if names is None:
+ return None
+ target_name, receiver_name = names
+ # F#: a dotted static path (``Path.Combine(a, b)``) is one identifier node,
+ # so it splits on the text, after the builtin check above.
+ if language == "fsharp" and receiver_name is None and "." in target_name:
+ receiver_part, _, target_name = target_name.rpartition(".")
+ receiver_name = receiver_part.strip()
+ target_name = target_name.strip()
+ if not target_name:
+ return None
+
+ arg_nodes = capture_dict.get("call.arguments", [])
+ receiver_call_nodes = capture_dict.get("call.receiver_call", [])
+ scope_nodes = capture_dict.get("call.scope", [])
+ line = site_node.start_point[0] + 1
+ arguments = _overload_arguments(language, arg_nodes)
+ call = CallSite(
+ target_name=target_name,
+ receiver_name=receiver_name,
+ caller_symbol_id=_find_enclosing_symbol(line, symbol_ranges),
+ line=line,
+ argument_count=_count_arguments(arg_nodes[0]) if arg_nodes else None,
+ receiver_call=(
+ _call_receiver_from_node(receiver_call_nodes[0], src) if receiver_call_nodes else None
+ ),
+ argument_types=_argument_literal_types(arguments, src),
+ argument_names=_argument_names(arguments, src),
+ scope_name=_node_text(scope_nodes[0], src).strip() if scope_nodes else None,
+ edge_type=(
+ "references" if site_node.type in config.reference_call_node_types else "calls"
+ ),
+ supplied_props=_jsx_supplied_props(site_node, src),
+ # Ceiling: schedulers matched by name, the task followed within its
+ # own function only (see ``python_tasks``).
+ spawned=language == "python" and is_fire_and_forget(site_node, src),
+ )
+ return (site_node.start_byte, site_node.end_byte, target_name), call
+
+
+def _call_names(
+ capture_dict: dict,
+ site_node: Node,
+ target_node: Node,
+ config: LanguageConfig,
+ language: str,
+ src: str,
+ call_builtins: frozenset[str],
+) -> tuple[str, str | None] | None:
+ """``(target_name, receiver_name)`` as written, or None to drop the site."""
+ target_name = config.call_target_name_fn(_node_text(target_node, src).strip(), target_node.type)
+ if not target_name:
+ return None
+ receiver_nodes = capture_dict.get("call.receiver", [])
+ if language == "objectivec":
+ target_name = _objc_call_target(
+ site_node, target_node, target_name, bool(receiver_nodes), src
+ )
+ if target_name is None:
+ return None
+ if target_name in call_builtins:
+ return None
+ # Elixir: a definition head (`add(a, b)` in `def add(a, b)`) and a module
+ # attribute (`@doc "..."`) are both ``call`` nodes; left in, every function
+ # would call itself.
+ if language == "elixir" and _elixir_call_is_definitional(site_node, src):
+ return None
+ receiver_name = None
+ if receiver_nodes:
+ receiver_name = _receiver_text(language, receiver_nodes[0], src)
+ if receiver_name is None:
+ return None
+ if receiver_name and language == "php":
+ receiver_name = _normalize_php_receiver(receiver_name)
+ return target_name, receiver_name
diff --git a/packages/core/src/repowise/core/ingestion/parser/grammars.py b/packages/core/src/repowise/core/ingestion/parser/grammars.py
new file mode 100644
index 000000000..a72a9a9a6
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/grammars.py
@@ -0,0 +1,269 @@
+"""Grammar loading, query compilation and the tree-sitter parse itself."""
+
+from __future__ import annotations
+
+from collections.abc import Iterable
+from functools import cache
+from pathlib import Path
+
+import structlog
+from tree_sitter import Language, Node, Parser
+
+from ..language_configs import LANGUAGE_CONFIGS
+from ..languages.registry import REGISTRY as _LANG_REGISTRY
+from ..parser_helpers import _collect_error_nodes
+
+# One logger for the whole package, under the package's public name.
+log = structlog.get_logger("repowise.core.ingestion.parser")
+
+QUERIES_DIR = Path(__file__).parent.parent / "queries"
+
+
+@cache
+def _compile_query(lang: str, grammar_tag: str | None = None) -> tuple[object | None, str | None]:
+ """Compile and return (Query, None) or (None, error_str).
+
+ Cached process-wide so preflight and parsing workers never recompile the same query.
+ """
+ grammar = grammar_tag or lang
+ language = _get_language(grammar)
+ if language is None:
+ return None, None
+
+ # The spec names the query file, so a language can reuse another's
+ # queries wholesale (svelte -> typescript.scm). Every other spec declares
+ # ``<tag>.scm``, which is what the default preserves.
+ spec = _LANG_REGISTRY.get(lang)
+ scm_name = (spec.scm_file if spec and spec.scm_file else None) or f"{lang}.scm"
+ scm_path = QUERIES_DIR / scm_name
+ if not scm_path.exists():
+ log.debug("No .scm query file found", language=lang, path=str(scm_path))
+ return None, None
+
+ scm_text = scm_path.read_text(encoding="utf-8")
+ # Grammar-variant-specific additions (e.g. JSX node captures that are
+ # only valid against the ``tsx`` grammar but not the plain ``typescript``
+ # one). Appended to the base SCM only when the variant scm file exists.
+ if grammar_tag and grammar_tag != lang:
+ extra_scm = QUERIES_DIR / f"{grammar_tag}.scm"
+ if extra_scm.exists():
+ scm_text = scm_text + "\n" + extra_scm.read_text(encoding="utf-8")
+ try:
+ from tree_sitter import Query # type: ignore[attr-defined]
+
+ return Query(language, scm_text), None
+ except Exception as exc:
+ return None, str(exc)
+
+
+@cache
+def _load_compiled_query(lang: str, grammar_tag: str | None = None) -> object | None:
+ """Process-wide cache of compiled tree-sitter Query objects.
+
+ Compiling `.scm` queries is non-trivial; in process-pool parsing each worker
+ would otherwise recompile per file. ``grammar_tag`` may differ from
+ ``lang`` when a language reuses another's grammar at a different
+ variant — e.g. ``.tsx`` files reuse ``typescript.scm`` but must bind
+ to the JSX-aware ``tsx`` grammar so React components don't drown in
+ ERROR nodes.
+ """
+ query, err = _compile_query(lang, grammar_tag)
+ if err is not None:
+ log.warning("Failed to compile query", language=lang, error=err)
+ return query
+
+
+
+# Languages that intentionally have no AST parser. Derived from the
+# centralised LanguageRegistry — only non-code passthrough languages are
+# included (not the extra git-blame-only languages).
+
+# Excludes "openapi" (handled by special_handlers) and "unknown".
+_PASSTHROUGH_LANGUAGES: frozenset[str] = _LANG_REGISTRY.unparseable_data_languages()
+
+
+# ---------------------------------------------------------------------------
+# Language registry — maps language tag → tree-sitter Language object
+# ---------------------------------------------------------------------------
+
+
+def _build_language_registry() -> dict[str, Language]:
+ """Lazily load installed tree-sitter language packages.
+
+ Driven by ``LanguageSpec.grammar_package`` / ``grammar_loader`` /
+ ``shares_grammar_with`` from the centralised registry.
+ """
+ registry: dict[str, Language] = {}
+
+ for spec in _LANG_REGISTRY.all_specs():
+ # Languages that share another's grammar (e.g. C → cpp)
+ if spec.shares_grammar_with:
+ shared = registry.get(spec.shares_grammar_with)
+ if shared:
+ registry[spec.tag] = shared
+ continue
+
+ if not spec.grammar_package:
+ continue
+
+ try:
+ mod = __import__(spec.grammar_package)
+ loader_fn = getattr(mod, spec.grammar_loader)
+ loaded = loader_fn(*spec.grammar_loader_args)
+ # Standalone grammar wheels return a PyCapsule; shared grammar
+ # packs may return the fully constructed Language directly.
+ lang_obj = loaded if isinstance(loaded, Language) else Language(loaded)
+ registry[spec.tag] = lang_obj
+ except Exception as exc:
+ log.debug(
+ "tree-sitter language unavailable",
+ language=spec.tag,
+ reason=str(exc),
+ )
+
+ # TypeScript's tsx variant — special case: same package, different loader
+ if "typescript" in registry and "tsx" not in registry:
+ try:
+ import tree_sitter_typescript as _ts_mod
+
+ registry["tsx"] = Language(_ts_mod.language_tsx())
+ except Exception as exc:
+ log.debug("tree-sitter language unavailable", language="tsx", reason=str(exc))
+
+ return registry
+
+
+_LANGUAGE_REGISTRY: dict[str, Language] = {}
+
+# Languages already reported as having a config but no installed grammar:
+# one line per language per process, not one per file.
+_MISSING_GRAMMAR_REPORTED: set[str] = set()
+
+
+def missing_grammar_languages(language_tags: Iterable[str]) -> list[str]:
+ """Of *language_tags*, those that parse via tree-sitter but have no grammar.
+
+ Answers the question once, in the parent, before up to eight spawned
+ workers each rediscover it and log their own copy of the answer at a level
+ the CLI discards anyway.
+
+ Deliberately uses ``find_spec`` rather than importing: the whole point is
+ to stay cheap enough to run on every index. Building the real registry here
+ would import every tree-sitter package into the parent process, which is
+ memory the parse pool is about to need for something else.
+
+ A tag with no :data:`LANGUAGE_CONFIGS` entry is not a gap — nothing claims
+ to parse it — so it is skipped rather than reported.
+
+ Ceiling: "importable" is not "loadable". A grammar whose compiled ABI does
+ not match the installed ``tree_sitter`` imports fine and then raises inside
+ ``Language()``, which this cannot see, so that case reports nothing here
+ and stays a per-worker debug line. Reporting it properly means loading the
+ grammars, which is the cost this function exists to avoid.
+ """
+ import importlib.util
+
+ specs = {spec.tag: spec for spec in _LANG_REGISTRY.all_specs()}
+ missing: list[str] = []
+ for tag in language_tags:
+ if tag not in LANGUAGE_CONFIGS:
+ continue
+ spec = specs.get(tag)
+ if spec is None:
+ continue
+ package = spec.grammar_package
+ if not package and spec.shares_grammar_with:
+ shared = specs.get(spec.shares_grammar_with)
+ package = shared.grammar_package if shared else None
+ if not package:
+ continue
+ try:
+ if importlib.util.find_spec(package) is None:
+ missing.append(tag)
+ except (ImportError, ValueError):
+ missing.append(tag)
+ return sorted(missing)
+
+
+def failed_query_languages(language_tags: Iterable[str]) -> list[tuple[str, str]]:
+ """Of *language_tags*, those whose tree-sitter queries fail to compile.
+
+ Scoped to what traversal discovered in the repo. Only checks languages with
+ a registered LanguageConfig and query file. Best-effort and bounded.
+ """
+ failed: list[tuple[str, str]] = []
+ seen: set[str] = set()
+
+ for tag in language_tags:
+ if tag in seen or tag not in LANGUAGE_CONFIGS:
+ continue
+ seen.add(tag)
+
+ _, err = _compile_query(tag)
+ if err is not None:
+ failed.append((tag, err))
+ continue
+
+ # Check grammar variants if applicable (e.g. tsx for typescript)
+ if tag == "typescript":
+ _, tsx_err = _compile_query("typescript", grammar_tag="tsx")
+ if tsx_err is not None:
+ failed.append((tag, f"tsx variant: {tsx_err}"))
+
+ return sorted(failed, key=lambda x: x[0])
+
+
+def _get_language(tag: str) -> Language | None:
+ # Filled in place, never rebound: the package re-exports this dict.
+ if not _LANGUAGE_REGISTRY:
+ _LANGUAGE_REGISTRY.update(_build_language_registry())
+ return _LANGUAGE_REGISTRY.get(tag)
+
+
+def grammar_tag_for(language: str, path: str) -> str:
+ """The grammar a file is read with, which is not always its language tag.
+
+ A ``.tsx`` file arrives tagged ``typescript``, and tree-sitter-typescript's
+ default grammar errors on every ``<Component />``. Only the grammar moves:
+ the language tag keeps selecting dialects and vocabularies, several of
+ which (``mocks/lexicon.py`` among them) carry no ``tsx`` row and would
+ silently degrade if handed one.
+ """
+ # Case-folded, because the extension table that tagged the file is.
+ if language == "typescript" and path.lower().endswith(".tsx"):
+ return "tsx"
+ return language
+
+
+def _parse_tree(
+ lang: str, language: Language, grammar_tag: str, source: bytes
+) -> tuple[Node, list[str], str, Language]:
+ """``(root, parse errors, grammar tag, language)``; a JSX-bearing ``.ts`` retries as TSX."""
+ parser = Parser(language)
+ tree = parser.parse(source)
+ root = tree.root_node
+
+ parse_errors = _collect_error_nodes(root)
+
+ # A .ts file (or a .vue render function) holding JSX errors under the
+ # TypeScript grammar; retry as TSX and keep it only when it errors less.
+ # ``source`` is the markup-blanked projection, so the token test sees real JSX.
+ if (
+ parse_errors
+ and lang in ("typescript", "vue")
+ and grammar_tag != "tsx"
+ and (b"/>" in source or b"</" in source)
+ ):
+ tsx_language = _get_language("tsx")
+ if tsx_language is not None:
+ tsx_tree = Parser(tsx_language).parse(source)
+ tsx_errors = _collect_error_nodes(tsx_tree.root_node)
+ if len(tsx_errors) < len(parse_errors):
+ tree = tsx_tree
+ root = tree.root_node
+ parse_errors = tsx_errors
+ # The caller's query appends tsx.scm for the tsx tag, which
+ # supplies the JSX component call sites, so both must change.
+ grammar_tag = "tsx"
+ language = tsx_language
+ return root, parse_errors, grammar_tag, language
diff --git a/packages/core/src/repowise/core/ingestion/parser/imports.py b/packages/core/src/repowise/core/ingestion/parser/imports.py
new file mode 100644
index 000000000..b1b198f7f
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/imports.py
@@ -0,0 +1,65 @@
+"""Import extraction: the shared dedup loop over a file's import matches.
+
+What each statement yields is the language's ``LanguageConfig.import_extractor``
+(``extractors.imports``); this loop only dedupes and stamps load kinds.
+"""
+
+from __future__ import annotations
+
+from ..extractors import node_text
+from ..language_configs import LanguageConfig
+from ..models import FileInfo, Import, combine_load_kinds
+from ..parser_helpers import import_load_kind
+
+
+def _extract_imports(
+ matches: list[dict],
+ config: LanguageConfig,
+ file_info: FileInfo,
+ src: str,
+) -> list[Import]:
+ language = file_info.language
+ rules = config.import_extractor
+ imports: list[Import] = []
+ # Statement key -> the Imports it produced, so a repeat folds its load
+ # kind into them rather than adding a duplicate.
+ seen_imports: dict[str, list[Import]] = {}
+ seen_own: set[str] = set()
+
+ for capture_dict in matches:
+ stmt_nodes = capture_dict.get("import.statement", [])
+ module_nodes = capture_dict.get("import.module", [])
+
+ if not stmt_nodes or not module_nodes:
+ continue
+
+ stmt_node = stmt_nodes[0]
+ if rules.deduped is not None:
+ imports.extend(rules.deduped(stmt_node, module_nodes[0], src, seen_own))
+ continue
+
+ raw = node_text(stmt_node, src).strip()
+ dedup_key = rules.dedup_key(stmt_node, raw, src) if rules.dedup_key else raw
+ kind = import_load_kind(stmt_node, language, src)
+ if dedup_key in seen_imports:
+ # A lazy copy must not hide a load-time twin: runtime wins.
+ for imp in seen_imports[dedup_key]:
+ imp.type_only, imp.deferred = combine_load_kinds(
+ (imp.type_only, imp.deferred), kind
+ )
+ continue
+ seen_imports[dedup_key] = []
+
+ module_text = node_text(module_nodes[0], src).strip().strip("\"'` ")
+ if not module_text:
+ continue
+
+ found = rules.statement(stmt_node, module_nodes[0], module_text, raw, language, src)
+ for imp in found:
+ imp.type_only, imp.deferred = kind
+ seen_imports[dedup_key] = found
+ imports.extend(found)
+
+ if rules.finalize is not None:
+ imports = rules.finalize(imports)
+ return imports
diff --git a/packages/core/src/repowise/core/ingestion/parser/references.py b/packages/core/src/repowise/core/ingestion/parser/references.py
new file mode 100644
index 000000000..1b153b04a
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/references.py
@@ -0,0 +1,215 @@
+"""Reference sites (a function named without a call) and type references."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+from tree_sitter import Node
+
+from ..extractors import node_text
+from ..models import CallSite, FileInfo, Import, Symbol, TypeReference
+from ..parser_helpers import (
+ TYPE_HEAD_EXTRACTORS,
+ _classify_param_origin,
+ _find_enclosing_symbol,
+ _head_type_identifier,
+ _rust_shadowed_by_type_param,
+)
+
+_node_text = node_text
+
+# Languages whose query defines the ``@reference.*`` captures. The SFC tags
+# reuse the TS captures but stay out: resolution has no local-shadow scan for them.
+_REFERENCE_LANGUAGES = ("cpp", "c", "go", "rust", "kotlin", "typescript", "javascript", "python")
+
+# Languages where a bare name resolves only through the file's own symbols or
+# a name it imports, so any other bare name cannot pass the reference floor.
+_NAMED_BINDING_LANGUAGES = frozenset({"typescript", "javascript", "python"})
+
+
+def _bare_reference_names(
+ language: str, symbols: list[Symbol], imports: list[Import]
+) -> frozenset[str] | None:
+ """Names a receiver-less reference in this file could resolve to, or None for any.
+
+ Most bare identifiers in a value position are locals; dropping them here
+ skips their per-site work. A wildcard import keeps every name.
+ """
+ if language not in _NAMED_BINDING_LANGUAGES:
+ return None
+ imported = {name for imp in imports for name in imp.local_names}
+ if "*" in imported:
+ return None
+ return frozenset(imported.union(s.name for s in symbols))
+
+
+def _extract_references(
+ matches: list[dict],
+ file_info: FileInfo,
+ src: str,
+ symbols: list[Symbol],
+ imports: list[Import],
+) -> list[CallSite]:
+ """Extract sites that name a function without calling it.
+
+ Six shapes carry a function by name and never call it where a parser
+ can see: a dispatch-table entry, a callback field, an argument to a
+ registration macro, a func value in argument position, a struct-field
+ initialiser, and a ``::`` callable reference. Each leaves the named
+ function with no inbound edge, which read as a ``safe_to_delete``
+ unused export and took out whole handler and interop layers.
+ TS/JS and Python capture any value position: an argument, a list or
+ object entry, a return, an assignment and a default export.
+
+ ``@reference.receiver`` is optional; capturing it is what lets
+ ``_add_reference_edges`` restrict a bare name to free functions and
+ allow a qualified name to reach a method.
+
+ Self-gating on the reference captures, so a language whose query
+ defines none produces nothing and pays two dict lookups. Two guards
+ keep the broad syntactic positions from claiming ordinary code:
+
+ * A macro argument requires a SCREAMING_CASE callee and must be that
+ macro's only argument. Uppercase alone is not enough: assertion
+ macros are spelled the same way and take values, so ``EXPECT_EQ(
+ capacity, 100)`` bound a local to whatever free function shared its
+ name. Registering something registers one thing, which separates
+ ``BENCHMARK(BM_Foo)`` from ``TEST(SuiteName, TestName)``.
+ * A table entry must sit outside any function body. A dispatch table is
+ a file-, namespace- or class-scope aggregate; the same braces inside
+ a function are a constructor member-init or a local aggregate, where
+ ``{data, size}`` names parameters, not functions.
+
+ Whether the name denotes a function at all is settled later, at
+ resolution, where the symbol kind is known.
+ """
+ from ..language_data import get_builtin_calls
+
+ scope = _ReferenceScope(
+ builtins=get_builtin_calls(file_info.language),
+ bindable=_bare_reference_names(file_info.language, symbols, imports),
+ symbol_ranges=sorted(
+ [(s.start_line, s.end_line, s.id) for s in symbols],
+ key=lambda t: (t[0], -t[1]),
+ ),
+ callable_ids={s.id for s in symbols if s.kind in ("function", "method")},
+ )
+ references: list[CallSite] = []
+
+ for capture_dict in matches:
+ plain_nodes = capture_dict.get("reference.name", [])
+ table_nodes = capture_dict.get("reference.table", [])
+ if not plain_nodes and not table_nodes:
+ continue
+
+ receiver_nodes = capture_dict.get("reference.receiver", [])
+ receiver = (
+ _node_text(receiver_nodes[0], src).strip() if receiver_nodes else None
+ ) or None
+ if not _macro_argument_allowed(capture_dict, plain_nodes, src):
+ continue
+
+ candidates = [(node, False) for node in plain_nodes]
+ candidates += [(node, True) for node in table_nodes]
+ for name_node, is_table in candidates:
+ site = scope.site(name_node, is_table, receiver, src)
+ if site is not None:
+ references.append(site)
+
+ return references
+
+
+def _macro_argument_allowed(capture_dict: dict, plain_nodes: list[Node], src: str) -> bool:
+ """A registration-macro argument needs a SCREAMING_CASE callee and to be its only argument."""
+ via_nodes = capture_dict.get("reference.via", [])
+ if not via_nodes:
+ return True
+ via = _node_text(via_nodes[0], src).strip()
+ if not via or not via.isupper():
+ return False
+ arg_list = plain_nodes[0].parent if plain_nodes else None
+ return arg_list is not None and len(arg_list.named_children) == 1
+
+
+@dataclass
+class _ReferenceScope:
+ """Per-file state the reference sites of one file are filtered against."""
+
+ builtins: frozenset[str]
+ bindable: frozenset[str] | None
+ symbol_ranges: list[tuple[int, int, str]]
+ callable_ids: set[str]
+ seen: set[tuple[int, str, str | None]] = field(default_factory=set)
+
+ def site(
+ self, name_node: Node, is_table: bool, receiver: str | None, src: str
+ ) -> CallSite | None:
+ """The reference *name_node* makes, or None when it is filtered or repeated."""
+ name = _node_text(name_node, src).strip()
+ if not name or name in self.builtins:
+ return None
+ if self.bindable is not None and receiver is None and name not in self.bindable:
+ return None
+ line = name_node.start_point[0] + 1
+ enclosing = _find_enclosing_symbol(line, self.symbol_ranges)
+ if is_table and enclosing in self.callable_ids:
+ return None
+ if (line, name, receiver) in self.seen:
+ return None
+ self.seen.add((line, name, receiver))
+ return CallSite(
+ target_name=name,
+ receiver_name=receiver,
+ caller_symbol_id=enclosing,
+ line=line,
+ argument_count=None,
+ )
+
+
+def _extract_type_refs(
+ matches: list[dict],
+ src: str,
+ lang: str = "",
+) -> list[TypeReference]:
+ """Collect ``@param.type`` captures into TypeReference records.
+
+ C# emits these from constructor / method / delegate / primary-ctor
+ parameter types; Go emits them from parameter, struct-field, return,
+ and composite-literal type positions (see ``go.scm``). The graph
+ builder resolves each reference to a defining file via the
+ language-specific resolver index and emits a file-level edge.
+
+ The head-identifier extractor is language-specific (Go unwraps
+ ``*T`` / ``[]T`` / ``map[K]V`` / ``pkg.T``); see
+ ``TYPE_HEAD_EXTRACTORS``. Capture origin is inferred from the
+ enclosing node: ``constructor_declaration`` → ``ctor_param``,
+ ``method_declaration`` → ``method_param`` (C#);
+ ``field_declaration`` → ``field_type``, ``composite_literal`` →
+ ``composite_literal`` (Go).
+ """
+ head_of = TYPE_HEAD_EXTRACTORS.get(lang, _head_type_identifier)
+
+ refs: list[TypeReference] = []
+ seen: set[tuple[str, int]] = set()
+
+ for capture_dict in matches:
+ type_nodes = capture_dict.get("param.type", [])
+ if not type_nodes:
+ continue
+ for type_node in type_nodes:
+ head = head_of(type_node, src)
+ if not head:
+ continue
+ # ``struct Wrapper<Item>`` binds Item as a type parameter; the
+ # capture cannot tell it from a real ``struct Item``.
+ if lang == "rust" and _rust_shadowed_by_type_param(type_node, head, src):
+ continue
+ line = type_node.start_point[0] + 1
+ key = (head, line)
+ if key in seen:
+ continue
+ seen.add(key)
+ origin = _classify_param_origin(type_node)
+ refs.append(TypeReference(type_name=head, line=line, origin=origin))
+
+ return refs
diff --git a/packages/core/src/repowise/core/ingestion/parser/symbol_rules.py b/packages/core/src/repowise/core/ingestion/parser/symbol_rules.py
new file mode 100644
index 000000000..8e516fda2
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/symbol_rules.py
@@ -0,0 +1,397 @@
+"""Per-language symbol rules: names, kinds, line spans, decorators, modifiers, visibility."""
+
+from __future__ import annotations
+
+import sys
+
+from tree_sitter import Node
+
+from ..cpp_export_macros import _CppExportType, _is_bodiless_cpp_type
+from ..extractors import (
+ node_text,
+ refine_elixir_call_kind,
+ refine_fsharp_type_kind,
+ refine_go_type_kind,
+ refine_kotlin_class_kind,
+ refine_pascal_type_kind,
+ refine_swift_class_kind,
+)
+from ..extractors.bindings.ts_js import declarator_binds_callable, declarator_value_is_module_ref
+from ..extractors.visibility import (
+ refine_cpp_visibility,
+ refine_csharp_visibility,
+ refine_java_visibility,
+ refine_py_visibility,
+ refine_rust_visibility,
+ refine_ts_visibility,
+)
+from ..language_configs import LanguageConfig
+from ..models import Symbol
+from ..parser_helpers import (
+ _elixir_is_template_definition,
+ _elixir_symbol_name,
+ _fsharp_binding_end_line,
+ _fsharp_binding_has_params,
+ _objc_is_macro_enum,
+ _objc_symbol_name,
+)
+from ..python_overload import is_python_overload as _is_python_overload
+
+_node_text = node_text
+
+# Languages whose source reaches the parser as TypeScript/JavaScript. The
+# SFC tags are here because ``sfc_source`` projects their <script> blocks into
+# a TS buffer at identical offsets, so every TS/JS code path applies verbatim.
+_TS_JS_LANGUAGES = ("typescript", "javascript", "svelte", "vue", "astro")
+
+
+_TS_OVERLOAD_SIGNATURES = frozenset({"function_signature", "method_signature"})
+
+
+def _keep_ts_overloads_beside_a_body(symbols: list[Symbol], node_types: list[str]) -> list[Symbol]:
+ """Drop a bodiless TS signature whose id has no implementation in the file.
+
+ That is an ambient declaration (``declare function``, a ``.d.ts``), not an
+ overload, and it never was a symbol.
+ """
+ bodied = {s.id for s, t in zip(symbols, node_types, strict=True) if t not in _TS_OVERLOAD_SIGNATURES}
+ return [
+ s
+ for s, t in zip(symbols, node_types, strict=True)
+ if t not in _TS_OVERLOAD_SIGNATURES or s.id in bodied
+ ]
+
+
+def _drop_swift_extensions_of_local_types(symbols: list[Symbol]) -> list[Symbol]:
+ """Drop an ``extension`` of a type the same file declares.
+
+ It shares the type's id, so it would replace the type's node and kind; its
+ members already name the type as their parent.
+ """
+ declared = {s.id for s in symbols if s.kind != "impl"}
+ return [s for s in symbols if s.kind != "impl" or s.id not in declared]
+
+
+def _is_declaration(
+ def_node: Node,
+ config: LanguageConfig,
+ language: str,
+ export_type: _CppExportType | None,
+ src: str,
+) -> bool:
+ node_type = def_node.type
+ if node_type in config.declaration_node_types:
+ return True
+ if language == "python":
+ return _is_python_overload(def_node, src)
+ if export_type is not None:
+ return export_type.is_forward_declaration
+ return _is_bodiless_cpp_type(language, node_type, def_node)
+
+
+
+_FSHARP_BINDING_NODE_TYPES = ("function_declaration_left", "value_declaration_left")
+_DART_FUNCTION_NODE_TYPES = ("function_signature", "getter_signature", "setter_signature")
+
+
+def _is_fsharp_binding(language: str, node_type: str) -> bool:
+ return language == "fsharp" and node_type in _FSHARP_BINDING_NODE_TYPES
+
+
+def _language_symbol_name(language: str, def_node: Node, name: str, src: str) -> str | None:
+ """Apply a language's own naming rule; None drops the match."""
+ if not name:
+ return None
+ if language == "elixir":
+ # A `def` inside `quote do ... end` is macro body, not a
+ # definition of the module that writes it.
+ if _elixir_is_template_definition(def_node, src):
+ return None
+ # `defimpl Proto, for: Type` is named for the module the
+ # compiler generates, not for the protocol alone.
+ return _elixir_symbol_name(def_node, name, src)
+ if language == "objectivec":
+ # `typedef NS_ENUM(NSInteger, Kind) { ... }` has no grammar
+ # rule, so only its enum *cases* survive as declarators and
+ # each would become a symbol named as though it were the type.
+ if _objc_is_macro_enum(def_node, src):
+ return None
+ # A method is named by its whole selector
+ # (`initWithName:age:`), which no single node holds, and a
+ # category by the class it extends plus its own name.
+ return _objc_symbol_name(def_node, name, src)
+ return name
+
+
+# ``(language, node_type)`` whose "class" kind the body refines: Kotlin
+# interface / enum class share class_declaration, Pascal ``declType`` wraps
+# class, record, interface, enum and more, and F# writes a class, a struct
+# and an interface with the same ``anon_type_defn`` node.
+_CLASS_KIND_REFINERS = {
+ ("kotlin", "class_declaration"): refine_kotlin_class_kind,
+ ("swift", "class_declaration"): refine_swift_class_kind,
+ ("pascal", "declType"): refine_pascal_type_kind,
+ ("fsharp", "anon_type_defn"): refine_fsharp_type_kind,
+}
+
+
+def _refine_symbol_kind(
+ kind: str, def_node: Node, config: LanguageConfig, language: str, src: str
+) -> str:
+ """Narrow a node-type kind where one node shape spells several kinds."""
+ node_type = def_node.type
+ # A value binding that carries parameter patterns beside its
+ # name is a function the grammar reparsed as a value because
+ # of its return-type annotation.
+ if (
+ language == "fsharp"
+ and node_type == "value_declaration_left"
+ and _fsharp_binding_has_params(def_node)
+ ):
+ kind = "function"
+
+ # Refine "struct" kind for Go type_spec (check if struct or interface body)
+ if kind == "struct" and config.parent_extraction == "receiver":
+ kind = refine_go_type_kind(def_node, src)
+
+ if kind == "class":
+ refine = _CLASS_KIND_REFINERS.get((language, node_type))
+ if refine is not None:
+ kind = refine(def_node)
+
+ # Elixir: every definition is a ``call``; its keyword names the kind.
+ if language == "elixir" and node_type == "call":
+ kind = refine_elixir_call_kind(def_node, src)
+ return kind
+
+
+def _symbol_end_line(
+ def_node: Node,
+ config: LanguageConfig,
+ language: str,
+ export_type: _CppExportType | None,
+) -> int | None:
+ """The symbol's last line, or None for a Dart local function (not a symbol)."""
+ node_type = def_node.type
+ end_line = def_node.end_point[0] + 1
+ if config.symbol_end_line_fn is not None:
+ end_line = config.symbol_end_line_fn(def_node, end_line)
+ if export_type is not None:
+ end_line = export_type.range_node.end_point[0] + 1
+ # F#: the captured left-hand side stops at the parameters; extend over the body.
+ if _is_fsharp_binding(language, node_type):
+ end_line = _fsharp_binding_end_line(def_node)
+ if language == "dart" and node_type in _DART_FUNCTION_NODE_TYPES:
+ return _dart_function_end_line(def_node, end_line)
+ return end_line
+
+
+def _dart_function_end_line(def_node: Node, end_line: int) -> int | None:
+ """Dart: a function's body is a sibling of its signature.
+
+ Local functions nested inside another function's body have no callable
+ *ancestor* (the enclosing signature is a sibling), so they are dropped
+ here (None); and the line range extends to the trailing body sibling or
+ call-site attribution stops at the signature line.
+ """
+ ancestor = def_node.parent
+ while ancestor is not None:
+ if ancestor.type in ("function_body", "function_expression"):
+ return None
+ ancestor = ancestor.parent
+ anchor = def_node
+ if def_node.parent is not None and def_node.parent.type == "method_signature":
+ anchor = def_node.parent
+ body_sibling = anchor.next_named_sibling
+ if body_sibling is not None and body_sibling.type == "function_body":
+ return body_sibling.end_point[0] + 1
+ return end_line
+
+
+def _module_binding_kind(def_node: Node, name: str, language: str, src: str) -> str | None:
+ """Kind of a module-level assignment, or None when it only binds a module.
+
+ SCREAMING_CASE names are constants by convention; the rest are module
+ variables (singletons like ``app = FastAPI()``, registries, caches).
+ ``str.isupper()`` requires at least one cased char, so names with no
+ letters (``_``, ``__all__``) fall to "variable" rather than being
+ mislabelled constants by ``name == name.upper()``.
+ """
+ # TS/JS: ``const svc = require('./svc')`` binds a module (already an
+ # import), so it is not a symbol.
+ if language in _TS_JS_LANGUAGES and declarator_value_is_module_ref(def_node, src):
+ return None
+ # A structurally callable value is a function whatever its name says.
+ callable_kind = (
+ declarator_binds_callable(def_node, src) if language in _TS_JS_LANGUAGES else None
+ )
+ return callable_kind or ("constant" if name.isupper() else "variable")
+
+
+def _attribute_decorators(def_node: Node, language: str, src: str) -> list[str]:
+ """Attribute texts that act as decorators (Rust, C#, C/C++), brackets stripped.
+
+ They land beside ``@`` decorators so one deprecation check reads them all.
+ """
+ if language == "rust":
+ return _rust_outer_attributes(def_node, src)
+ # C#: ``attribute_list`` is child[0] of the declaration itself, not a
+ # preceding sibling in the class body.
+ if language == "csharp":
+ return _leading_child_attributes(def_node, "attribute_list", "[", "]", src)
+ # C/C++: [[deprecated]] / [[deprecated("reason")]] are
+ # ``attribute_declaration`` nodes, child[0] of function_definition itself
+ # (NOT a preceding sibling at translation_unit level).
+ if language in ("cpp", "c"):
+ return _leading_child_attributes(def_node, "attribute_declaration", "[[", "]]", src)
+ if language == "kotlin":
+ return _kotlin_detached_annotation(def_node, src)
+ return []
+
+
+def _kotlin_detached_annotation(def_node: Node, src: str) -> list[str]:
+ """Kotlin: annotations the grammar split off a top-level declaration.
+
+ When the last annotation on its own line before a bare ``fun`` or
+ ``object`` takes one argument (``@Suppress("unused")``), the argument list
+ reads as a parenthesized expression: the annotations become an
+ ``annotated_expression`` chain beside a declaration with no modifiers.
+ """
+ if def_node.parent is None or def_node.parent.type != "source_file":
+ return []
+ sib = def_node.prev_sibling
+ while sib is not None and sib.type in ("line_comment", "block_comment"):
+ sib = sib.prev_sibling
+ return [_node_text(sib, src).strip()] if _is_split_annotation_chain(sib) else []
+
+
+def _is_split_annotation_chain(node: Node | None) -> bool:
+ """Nested ``annotated_expression`` nodes of annotations, the last one's argument at the tail."""
+ while node is not None and node.type == "annotated_expression":
+ kinds = [c.type for c in node.named_children]
+ if len(kinds) != 2 or kinds[0] != "annotation":
+ return False
+ if kinds[1] == "parenthesized_expression":
+ return True
+ node = node.named_children[1]
+ return False
+
+
+def _declared_modifiers(def_node: Node, node_types: frozenset[str], src: str) -> tuple[str, ...]:
+ """Lowercased keyword modifiers written in *def_node*'s modifier children.
+
+ Keywords are the anonymous leaves of those children (C# ``modifier`` >
+ ``override``, Kotlin ``modifiers`` > ``member_modifier`` > ``override``) or
+ a named leaf that is itself a modifier (VB.NET ``modifier`` ``Overrides``).
+ Named leaves such as Scala's ``protected[this]`` qualifier are not
+ keywords. Annotations and attributes nested among them are skipped: they
+ are decorators, not modifiers. The words are interned: a handful of
+ distinct values repeat on every symbol.
+ """
+ if not node_types:
+ return ()
+ words: list[str] = []
+ stack = [c for c in reversed(def_node.children) if c.type in node_types]
+ while stack:
+ node = stack.pop()
+ if "annotation" in node.type or "attribute" in node.type:
+ continue
+ if node.child_count == 0:
+ text = _node_text(node, src).strip()
+ keyword = not node.is_named or "modifier" in node.type
+ if keyword and text.isalpha():
+ words.append(sys.intern(text.lower()))
+ continue
+ stack.extend(reversed(node.children))
+ return tuple(words)
+
+
+def _rust_outer_attributes(def_node: Node, src: str) -> list[str]:
+ """Rust: outer attributes (#[...]) are preceding siblings of the item."""
+ attrs: list[str] = []
+ if def_node.parent is None:
+ return attrs
+ siblings = def_node.parent.children
+ for j, sib in enumerate(siblings):
+ if sib.id != def_node.id:
+ continue
+ k = j - 1
+ while k >= 0 and siblings[k].type == "attribute_item":
+ attr_text = _node_text(siblings[k], src).strip()
+ # Strip #[ and ] to get the inner attribute text
+ if attr_text.startswith("#[") and attr_text.endswith("]"):
+ attrs.append(attr_text[2:-1])
+ k -= 1
+ break
+ return attrs
+
+
+def _leading_child_attributes(
+ def_node: Node, node_type: str, opener: str, closer: str, src: str
+) -> list[str]:
+ """Inner text of the attribute children leading *def_node*, until the first other child.
+
+ A comment between attribute lists does not end the run, so an attribute
+ placed after one is still recorded.
+ """
+ attrs: list[str] = []
+ for child in def_node.children:
+ if child.type != node_type:
+ if child.type == "comment":
+ continue
+ break
+ attr_text = _node_text(child, src).strip()
+ if attr_text.startswith(opener) and attr_text.endswith(closer):
+ attrs.append(attr_text[len(opener) : -len(closer)])
+ return attrs
+
+
+def _refine_visibility(
+ def_node: Node,
+ language: str,
+ visibility: str,
+ name: str,
+ ts_deferred_exports: frozenset[str] | None,
+ py_all_names: frozenset[str] | None,
+ src: str,
+) -> tuple[str, bool]:
+ """``(visibility, is_exported_symbol)`` after the language's AST-context rules."""
+ # C/C++ (and plain C inside Objective-C) visibility comes from AST
+ # context: access specifiers, storage class, export attributes.
+ if language in ("cpp", "c", "objectivec"):
+ return refine_cpp_visibility(def_node, visibility, src)
+ # C#: an unmodified declaration's default depends on what encloses
+ # it, which the modifier-text fn cannot see.
+ if language == "csharp":
+ return refine_csharp_visibility(def_node, visibility), False
+ # Java: no access keyword means package-private, except inside an
+ # interface or annotation body, which the modifier-text fn cannot see.
+ if language == "java":
+ return refine_java_visibility(def_node), False
+ # TS/JS: a top-level declaration is only public when exported —
+ # inline, via ``export { x }`` lists, or ``export default x``.
+ if language in _TS_JS_LANGUAGES:
+ return refine_ts_visibility(def_node, visibility, name, ts_deferred_exports), False
+ # Python: a literal ``__all__`` raises the names it lists to public; it
+ # never demotes the ones it omits. GDScript shares ``py_visibility`` but
+ # has no ``__all__``, so the gate is on the language, not the fn.
+ if language == "python":
+ return refine_py_visibility(def_node, visibility, name, py_all_names), False
+ # Rust: a trait's items may not write ``pub`` of their own, so the
+ # trait's modifier is the only place their visibility is stated.
+ if language == "rust":
+ return refine_rust_visibility(def_node, visibility, src), False
+ return visibility, False
+
+
+def _dart_mixin_parent(def_node: Node, config: LanguageConfig, src: str) -> str | None:
+ """Name of the Dart ``mixin`` enclosing *def_node*, if it is the nearest type."""
+ ancestor = def_node.parent
+ while ancestor is not None:
+ if ancestor.type == "mixin_declaration":
+ ident = next((c for c in ancestor.children if c.type == "identifier"), None)
+ return _node_text(ident, src) if ident is not None else None
+ if ancestor.type in config.parent_class_types:
+ return None
+ ancestor = ancestor.parent
+ return None
diff --git a/packages/core/src/repowise/core/ingestion/parser/symbols.py b/packages/core/src/repowise/core/ingestion/parser/symbols.py
new file mode 100644
index 000000000..49f064aa1
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/parser/symbols.py
@@ -0,0 +1,541 @@
+"""Symbol extraction: one query match to one ``Symbol``, and the parent it belongs to."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+from tree_sitter import Node
+
+from ..cpp_export_macros import (
+ CppExportTypes,
+ _cpp_export_macro_parent,
+ _cpp_normalize_identifier,
+ _CppExportType,
+ collect_cpp_export_types,
+)
+from ..extractors import (
+ build_signature,
+ extract_go_receiver_type,
+ extract_symbol_docstring,
+ node_text,
+)
+from ..extractors.synthetic_symbols import extract_synthetic_symbols
+from ..extractors.synthetic_symbols.csharp_extension_blocks import drop_extension_block_symbols
+from ..extractors.visibility import py_module_all_names, ts_deferred_export_names
+from ..language_configs import LanguageConfig
+from ..models import FileInfo, Symbol
+from ..parser_helpers import (
+ _build_qualified_name,
+ _csharp_type_parameter_count,
+ _dedupe_objc_interface_symbols,
+ _dedupe_pascal_interface_symbols,
+ _elixir_module_parent,
+ _fsharp_binding_is_nested,
+ _fsharp_parent_is_type,
+ _fsharp_parent_name,
+ _has_callable_ancestor,
+ _is_async_node,
+ _kotlin_receiver_fields,
+ _objc_container_node,
+ _objc_container_parent,
+ _qualified_cpp_parent,
+ _qualified_pascal_parent,
+ _ts_nested_object_method_owner,
+ follow_renamed_hosts,
+ in_anonymous_class,
+ is_anonymous_member,
+ scope_segment,
+)
+from ..symbol_identity import disambiguate_colliding_ids, symbol_discriminator
+from .symbol_rules import (
+ _TS_JS_LANGUAGES,
+ _attribute_decorators,
+ _dart_mixin_parent,
+ _declared_modifiers,
+ _drop_swift_extensions_of_local_types,
+ _is_declaration,
+ _is_fsharp_binding,
+ _keep_ts_overloads_beside_a_body,
+ _language_symbol_name,
+ _module_binding_kind,
+ _refine_symbol_kind,
+ _refine_visibility,
+ _symbol_end_line,
+)
+
+_node_text = node_text
+
+# Node types whose .scm patterns are anchored at module level, so the
+# callable-ancestor filter must not apply to them.
+_MODULE_ANCHORED_NODE_TYPES = frozenset({"assignment", "variable_declarator"})
+
+
+def _match_identity(
+ capture_dict: dict,
+ config: LanguageConfig,
+ language: str,
+ src: str,
+ cpp_exports: CppExportTypes,
+ seen: set[tuple[int, str]],
+) -> tuple[Node, list[Node], str, _CppExportType | None, int] | None:
+ """``(def_node, name_nodes, name, export_type, start_line)`` for a new symbol match.
+
+ None drops the match: no definition or name, a recovery match this file
+ does not support, a suppressed export macro, or a duplicate. The
+ ``(start_line, name)`` key is recorded in *seen* here.
+ """
+ def_nodes = capture_dict.get("symbol.def", [])
+ name_nodes = capture_dict.get("symbol.name", [])
+ captured_export_type_nodes = capture_dict.get("symbol.cpp_export_type", [])
+
+ if (
+ captured_export_type_nodes
+ and captured_export_type_nodes[0].id not in cpp_exports.capture_ids
+ ):
+ # The query also sees ordinary ``struct Tag variable;`` forms;
+ # discard only the unsupported recovery match.
+ return None
+
+ if not def_nodes or not name_nodes:
+ return None
+
+ def_node = def_nodes[0]
+ name = _language_symbol_name(
+ language, def_node, config.symbol_name_fn(_node_text(name_nodes[0], src), def_node.type), src
+ )
+ if name is None:
+ return None
+
+ export_type = cpp_exports.defs.get(def_node.id)
+ if export_type is not None and name != export_type.name:
+ # The ordinary struct/class query sees the same specifier, but
+ # tree-sitter calls the export macro its name. Keep only the
+ # dedicated match whose name is the outer declarator.
+ return None
+
+ if def_node.type == "preproc_def" and (
+ _cpp_normalize_identifier(name) in cpp_exports.macro_names
+ or def_node.id in cpp_exports.macro_def_ids
+ ):
+ # Body-form macros are suppressed by name;
+ # ambiguous forward declarations suppress only the exact
+ # active definition that made recovery safe.
+ return None
+
+ start_line = def_node.start_point[0] + 1
+ if export_type is not None:
+ start_line = export_type.range_node.start_point[0] + 1
+ dedup_key = (start_line, name)
+ if dedup_key in seen:
+ return None
+ seen.add(dedup_key)
+ return def_node, name_nodes, name, export_type, start_line
+
+
+def _parent_type_name(ancestor: Node, src: str) -> str | None:
+ """The name a class/impl ancestor gives its members, without type arguments."""
+ name_node = ancestor.child_by_field_name("name") or (
+ ancestor.child_by_field_name("type") # Rust impl_item
+ )
+ if name_node is None:
+ return None
+ # For Rust impl blocks with generic types (e.g. impl<T> Foo<T>),
+ # extract only the base type name, not the full generic signature.
+ if name_node.type in ("generic_type", "scoped_type_identifier"):
+ inner = name_node.child_by_field_name(
+ "type" if name_node.type == "generic_type" else "name"
+ )
+ if inner and inner.type == "type_identifier":
+ name_node = inner
+ return _node_text(name_node, src)
+
+
+def _nesting_parent(def_node: Node, config: LanguageConfig, src: str) -> str | None:
+ """The class/impl ancestor naming *def_node*'s parent.
+
+ An anonymous class is a class of its own, so its members gain segments
+ below it: ``Gson::FILTER::Filter::check``.
+ """
+ segments: list[str] = []
+ child, ancestor = def_node, def_node.parent
+ while ancestor is not None:
+ if ancestor.type in config.parent_class_types:
+ name = _parent_type_name(ancestor, src)
+ if name is not None:
+ return "::".join([name, *reversed(segments)])
+ else:
+ segment = scope_segment(
+ ancestor, child, bool(segments), config.anonymous_class_types, src
+ )
+ if segment is not None:
+ segments.append(segment)
+ child, ancestor = ancestor, ancestor.parent
+ # A Kotlin top-level function's object expression: ``f::Foo``.
+ return "::".join(reversed(segments)) or None
+
+
+def _java_enum_constant(def_node: Node, config: LanguageConfig, src: str) -> str | None:
+ """The enum constant whose body declares *def_node*, before any enclosing type.
+
+ An anonymous class is such a type: ``_find_parent`` names its members.
+ """
+ ancestor = def_node.parent
+ while (
+ ancestor is not None
+ and ancestor.type not in config.parent_class_types
+ and ancestor.type not in config.anonymous_class_types
+ ):
+ if ancestor.type == "enum_constant":
+ name = ancestor.child_by_field_name("name")
+ return _node_text(name, src) if name else None
+ ancestor = ancestor.parent
+ return None
+
+
+def _extract_symbols(
+ matches: list[dict],
+ config: LanguageConfig,
+ file_info: FileInfo,
+ src: str,
+) -> list[Symbol]:
+ language = file_info.language
+ symbols: list[Symbol] = []
+ # Parallel to ``symbols`` (same indices) -- consumed for Pascal and
+ # Objective-C, to dedupe interface-declaration vs. implementation
+ # method pairs after the loop, and for TS overload signatures. See
+ # _dedupe_pascal_interface_symbols / _dedupe_objc_interface_symbols.
+ node_types: list[str] = []
+ # Also parallel to ``symbols``, Objective-C only: which of @interface /
+ # @implementation / @protocol declared each member, so a protocol's
+ # method is never deduped against a same-named class method.
+ objc_container_kinds: list[str | None] = []
+
+ # Deferred-export names (``export { x }`` / ``export default x``),
+ # computed once per file for the TS/JS visibility refinement.
+ ts_deferred_exports: frozenset[str] | None = None
+ if language in _TS_JS_LANGUAGES:
+ ts_deferred_exports = ts_deferred_export_names(src)
+ # Literal module ``__all__`` names, once per file for the Python
+ # refinement. None means no signal (built at runtime or absent).
+ py_all_names: frozenset[str] | None = None
+ if language == "python":
+ py_all_names = py_module_all_names(src)
+ cpp_exports = (
+ collect_cpp_export_types(matches, src) if language == "cpp" else CppExportTypes()
+ )
+
+ file = _SymbolPass(config, file_info, src, cpp_exports, ts_deferred_exports, py_all_names)
+ for capture_dict in matches:
+ built = _symbol_from_match(capture_dict, file)
+ if built is None:
+ continue
+ symbol, def_node = built
+ symbols.append(symbol)
+ node_types.append(def_node.type)
+ if language == "objectivec":
+ container = _objc_container_node(def_node, config.parent_class_types)
+ objc_container_kinds.append(container.type if container else None)
+
+ if language == "pascal":
+ symbols = _dedupe_pascal_interface_symbols(symbols, node_types)
+
+ if language in _TS_JS_LANGUAGES:
+ symbols = _keep_ts_overloads_beside_a_body(symbols, node_types)
+
+ if language == "swift":
+ symbols = _drop_swift_extensions_of_local_types(symbols)
+
+ # A .m file routinely declares its private methods in a class
+ # extension and defines them below in the @implementation, which
+ # builds each symbol id twice in one file.
+ if language == "objectivec":
+ symbols = _dedupe_objc_interface_symbols(symbols, node_types, objc_container_kinds)
+
+ return symbols
+
+@dataclass
+class _SymbolPass:
+ """Per-file inputs every symbol match of one file is read against."""
+
+ config: LanguageConfig
+ file_info: FileInfo
+ src: str
+ cpp_exports: CppExportTypes
+ ts_deferred_exports: frozenset[str] | None
+ py_all_names: frozenset[str] | None
+ # (start_line, name) of every match kept so far: dedups decorated twins.
+ seen: set[tuple[int, str]] = field(default_factory=set)
+
+
+def _symbol_from_match(capture_dict: dict, file: _SymbolPass) -> tuple[Symbol, Node] | None:
+ """One query match as a symbol and its definition node, or None to drop it.
+
+ Records the match's ``(start_line, name)`` in ``file.seen`` once it passes
+ the name checks, so a later duplicate is dropped even if this one is.
+ """
+ config, file_info, src = file.config, file.file_info, file.src
+ language = file_info.language
+ identity = _match_identity(capture_dict, config, language, src, file.cpp_exports, file.seen)
+ if identity is None:
+ return None
+ def_node, name_nodes, name, export_type, start_line = identity
+
+ node_type = def_node.type
+ # A TS/JS object-literal method inside an anonymous callback is kept,
+ # qualified by the binding that owns its object.
+ object_owner = (
+ _ts_nested_object_method_owner(def_node, src, config.symbol_node_types)
+ if language in _TS_JS_LANGUAGES
+ else None
+ )
+ spanned = _kind_and_end_line(def_node, name, export_type, object_owner is not None, file)
+ if spanned is None:
+ return None
+ kind, end_line = spanned
+
+ modifier_texts = [_node_text(m, src) for m in capture_dict.get("symbol.modifiers", [])]
+ visibility, is_exported_symbol = _refine_visibility(
+ def_node,
+ language,
+ config.visibility_fn(name, modifier_texts),
+ name,
+ file.ts_deferred_exports,
+ file.py_all_names,
+ src,
+ )
+
+ parent_name = object_owner or _resolve_parent_name(
+ def_node,
+ config,
+ capture_dict.get("symbol.receiver", []),
+ name_nodes,
+ language,
+ src,
+ no_export_type=export_type is None,
+ export_type_parents=file.cpp_exports.parents,
+ )
+
+ # A ``field_declaration`` cannot occur outside a class body, so a
+ # missing parent means the class did not parse. Grammar recovery,
+ # not a member function.
+ if node_type == "function_declarator" and parent_name is None:
+ return None
+
+ params_nodes = capture_dict.get("symbol.params", [])
+ params_text = _node_text(params_nodes[0], src) if params_nodes else ""
+ scope = _id_scope(def_node, parent_name, config, language, src)
+ sym_id = f"{file_info.path}::{scope}::{name}" if scope else f"{file_info.path}::{name}"
+ symbol = Symbol(
+ id=sym_id,
+ name=name,
+ qualified_name=_build_qualified_name(file_info.path, scope, name),
+ kind=_method_or_kind(kind, parent_name, def_node, language), # type: ignore[arg-type]
+ signature=build_signature(node_type, name, params_text, def_node, src),
+ start_line=start_line,
+ end_line=end_line,
+ docstring=extract_symbol_docstring(def_node, src, language),
+ decorators=(
+ [m for m in modifier_texts if m.startswith("@")]
+ + _attribute_decorators(def_node, language, src)
+ ),
+ visibility=visibility, # type: ignore[arg-type]
+ is_async=_is_async_node(def_node, src),
+ language=language,
+ parent_name=parent_name,
+ is_exported_symbol=is_exported_symbol,
+ is_declaration=_is_declaration(def_node, config, language, export_type, src),
+ type_parameter_count=(
+ _csharp_type_parameter_count(def_node) if language == "csharp" else None
+ ),
+ **_kotlin_receiver_fields(def_node, params_nodes, language, name),
+ modifiers=_declared_modifiers(def_node, config.modifier_node_types, src),
+ )
+ return symbol, def_node
+
+
+def _kind_and_end_line(
+ def_node: Node,
+ name: str,
+ export_type: _CppExportType | None,
+ has_object_owner: bool,
+ file: _SymbolPass,
+) -> tuple[str, int] | None:
+ """``(kind, end_line)`` of a match, or None when it is not a symbol here."""
+ config, language, src = file.config, file.file_info.language, file.src
+ kind = _symbol_kind(
+ def_node,
+ config,
+ language,
+ src,
+ file.cpp_exports.container_ids,
+ # An anonymous class member is a method of its own class, even
+ # inside another method's body; ``_find_parent`` names it.
+ keep_nested=has_object_owner or in_anonymous_class(def_node, config.anonymous_class_types),
+ )
+ if kind is None:
+ return None
+
+ end_line = _symbol_end_line(def_node, config, language, export_type)
+ if end_line is None:
+ return None
+
+ if def_node.type in _MODULE_ANCHORED_NODE_TYPES:
+ kind = _module_binding_kind(def_node, name, language, src)
+ if kind is None:
+ return None
+ return kind, end_line
+
+
+def _method_or_kind(kind: str, parent_name: str | None, def_node: Node, language: str) -> str:
+ """Upgrade a function with a parent to a method.
+
+ F#: a nested module is a parent too (for id uniqueness), but it is not a
+ type, so a ``let`` inside one stays a function.
+ """
+ if parent_name and kind == "function" and (
+ language != "fsharp" or _fsharp_parent_is_type(def_node)
+ ):
+ return "method"
+ return kind
+
+
+def _id_scope(
+ def_node: Node, parent_name: str | None, config: LanguageConfig, language: str, src: str
+) -> str | None:
+ """The id scope: the parent, plus a Java enum constant whose body declares the symbol.
+
+ Each constant's override is then its own node while the enum stays its owner.
+ """
+ constant = _java_enum_constant(def_node, config, src) if language == "java" else None
+ return f"{parent_name}::{constant}" if parent_name and constant else parent_name
+
+
+def _symbol_kind(
+ def_node: Node,
+ config: LanguageConfig,
+ language: str,
+ src: str,
+ container_ids: frozenset[int],
+ *,
+ keep_nested: bool = False,
+) -> str | None:
+ """The symbol kind for *def_node*, or None when it is not a symbol here."""
+ node_type = def_node.type
+ kind = config.symbol_node_types.get(node_type)
+ if kind is None:
+ return None
+
+ # Only module-level and class-body members are symbols; the query is
+ # recursive, so defs nested in a callable are dropped. Module-anchored
+ # node types only match at module level and skip the check, as does a
+ # def the caller has placed already (``keep_nested``).
+ if (
+ not keep_nested
+ and node_type not in _MODULE_ANCHORED_NODE_TYPES
+ and _has_callable_ancestor(def_node, config.symbol_node_types, container_ids)
+ ):
+ return None
+
+ # F#: a ``let`` nested in another binding has no callable ancestor, so
+ # it is checked separately. A ``let`` in a type body is a field and stays.
+ if _is_fsharp_binding(language, node_type) and _fsharp_binding_is_nested(def_node):
+ return None
+ return _refine_symbol_kind(kind, def_node, config, language, src)
+
+
+def _resolve_parent_name(
+ def_node: Node,
+ config: LanguageConfig,
+ receiver_nodes: list[Node],
+ name_nodes: list[Node],
+ language: str,
+ src: str,
+ *,
+ no_export_type: bool,
+ export_type_parents: dict[int, str],
+) -> str | None:
+ """The enclosing type or module name, trying each language's own shape."""
+ parent_name = _find_parent(def_node, config, receiver_nodes, src)
+ if parent_name is not None:
+ return parent_name
+
+ if language == "cpp" and no_export_type:
+ parent_name = _cpp_export_macro_parent(def_node, export_type_parents)
+ if parent_name is not None:
+ return parent_name
+
+ # Dart mixin_declaration exposes no ``name`` field, so
+ # ``_find_parent``'s field lookup misses mixin members.
+ if language == "dart":
+ return _dart_mixin_parent(def_node, config, src)
+
+ # F#: no type node carries a ``name`` field -- the name hangs off
+ # a ``type_name`` child -- so the generic walk finds the ancestor
+ # and then reads nothing off it.
+ if language == "fsharp":
+ return _fsharp_parent_name(def_node, src)
+
+ # C/C++ out-of-line ``void Foo::method()`` names its class in the scope.
+ if language in ("cpp", "c") and name_nodes:
+ return _qualified_cpp_parent(name_nodes[0], src)
+
+ # Pascal out-of-line ``function TFoo.Bar`` names its class in the header.
+ if language == "pascal" and name_nodes:
+ return _qualified_pascal_parent(name_nodes[0], src)
+
+ # Elixir: ``defmodule`` is a ``call``; the name is its first argument.
+ if language == "elixir":
+ return _elixir_module_parent(def_node, src)
+
+ # Objective-C containers name themselves with a bare first identifier.
+ if language == "objectivec":
+ return _objc_container_parent(def_node, config.parent_class_types, src)
+ return None
+
+
+def _find_parent(
+ def_node: Node,
+ config: LanguageConfig,
+ receiver_nodes: list[Node],
+ src: str,
+) -> str | None:
+ """Determine the parent class/type for a symbol."""
+ if config.parent_extraction == "receiver":
+ # Go: extract type name from receiver parameter list
+ if receiver_nodes:
+ return extract_go_receiver_type(_node_text(receiver_nodes[0], src))
+ return None
+
+ if config.parent_extraction in ("nesting", "impl"):
+ return _nesting_parent(def_node, config, src)
+
+ return None # "none" mode
+
+
+def _finalize_symbols(
+ symbols: list[Symbol], root: Node, src: str, file_info: FileInfo
+) -> list[Symbol]:
+ """Add generator-emitted symbols and settle colliding ids.
+
+ Runs before calls and references are extracted, so both attribute to
+ the final ids.
+ """
+ lang = file_info.language
+ synthetic = extract_synthetic_symbols(root, src, file_info)
+ if synthetic:
+ # Keyed with the discriminator too, so a record's synthetic
+ # canonical constructor survives beside an explicit overload of
+ # another arity. A language without one dedupes on the id alone.
+ existing = {(s.id, symbol_discriminator(s)) for s in symbols}
+ symbols.extend(s for s in synthetic if (s.id, symbol_discriminator(s)) not in existing)
+ if lang == "csharp":
+ # The grammar reads a C# 14 extension block as a constructor
+ # named ``extension``; its members were added above instead.
+ symbols = drop_extension_block_symbols(symbols, root, src)
+ # Anonymous members go last, once the overloads holding them carry
+ # their discriminators.
+ anonymous = [s for s in symbols if is_anonymous_member(s)]
+ disambiguate_colliding_ids([s for s in symbols if not is_anonymous_member(s)], lang)
+ follow_renamed_hosts(symbols)
+ disambiguate_colliding_ids(anonymous, lang)
+ return symbols
diff --git a/packages/core/src/repowise/core/ingestion/parser_helpers.py b/packages/core/src/repowise/core/ingestion/parser_helpers.py
index ba5ef444c..e41e90971 100644
--- a/packages/core/src/repowise/core/ingestion/parser_helpers.py
+++ b/packages/core/src/repowise/core/ingestion/parser_helpers.py
@@ -1,7 +1,7 @@
"""Stateless AST helper functions used by :class:`~.parser.ASTParser`.
Language-neutral tree-sitter utilities live here; per-language helpers live in
-:mod:`.lang_helpers` and are re-exported so ``parser.py`` imports one module.
+:mod:`.lang_helpers` and are re-exported so the ``parser`` package imports one module.
No imports from ``parser``, so there is no import cycle.
"""
diff --git a/tests/unit/ingestion/parser/test_package_layout.py b/tests/unit/ingestion/parser/test_package_layout.py
new file mode 100644
index 000000000..afef3b676
--- /dev/null
+++ b/tests/unit/ingestion/parser/test_package_layout.py
@@ -0,0 +1,65 @@
+"""The parser package keeps its old import surface and wires every language's import rules."""
+
+from __future__ import annotations
+
+import pytest
+
+import repowise.core.ingestion.parser as parser_pkg
+from repowise.core.ingestion.extractors.imports import GENERIC_IMPORTS, ImportExtractor
+from repowise.core.ingestion.language_configs import LANGUAGE_CONFIGS
+
+# Names other code and open branches import from ``repowise.core.ingestion.parser``.
+_KEPT_NAMES = (
+ "ASTParser",
+ "LANGUAGE_CONFIGS",
+ "LanguageConfig",
+ "QUERIES_DIR",
+ "_PASSTHROUGH_LANGUAGES",
+ "_compile_query",
+ "_get_language",
+ "_load_compiled_query",
+ "_run_query",
+ "failed_query_languages",
+ "grammar_tag_for",
+ "log",
+ "missing_grammar_languages",
+ "parse_file",
+ "_call_receiver_from_node",
+ "_dedupe_calls",
+ "_fold_receiverless_twins",
+ "_receiver_text",
+ "_match_identity",
+ "_refine_symbol_kind",
+ "_refine_visibility",
+ "_nesting_parent",
+ "_bare_reference_names",
+ "_generic_import",
+ "_rust_use_imports",
+ "_cjs_imports",
+ "_pascal_imports",
+)
+
+
[email protected]("name", _KEPT_NAMES)
+def test_old_names_stay_importable(name: str) -> None:
+ assert hasattr(parser_pkg, name)
+
+
+def test_queries_dir_points_at_the_query_files() -> None:
+ assert (parser_pkg.QUERIES_DIR / "python.scm").is_file()
+
+
+def test_every_config_carries_an_import_extractor() -> None:
+ for tag, config in LANGUAGE_CONFIGS.items():
+ assert isinstance(config.import_extractor, ImportExtractor), tag
+ assert LANGUAGE_CONFIGS["go"].import_extractor is GENERIC_IMPORTS
+ typescript = LANGUAGE_CONFIGS["typescript"].import_extractor
+ assert LANGUAGE_CONFIGS["svelte"].import_extractor is typescript
+
+
+def test_language_registry_export_is_the_live_registry() -> None:
+ from repowise.core.ingestion.parser import grammars
+
+ parser_pkg._get_language("python")
+ assert parser_pkg._LANGUAGE_REGISTRY is grammars._LANGUAGE_REGISTRY
+ assert "python" in parser_pkg._LANGUAGE_REGISTRY