Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(languages): add Object Pascal / Delphi support (Good tier, draft) (#1353)

Laentir Valetova3a633bb108b
diff --git a/docs/architecture/language-support.md b/docs/architecture/language-support.md
index ee60c7534..e3db7b4ab 100644
--- a/docs/architecture/language-support.md
+++ b/docs/architecture/language-support.md
@@ -332,9 +332,23 @@ the original file. That single property is what makes the rest free: the spec
declares `shares_grammar_with="typescript"` and `scm_file="typescript.scm"`, the
`LanguageConfig` is an alias of TypeScript's, and the three health dialect
registries alias the TS entries. Each consumer that hands raw bytes to a
-tree-sitter `Parser` calls `prepare_source(language, source)` first — the
-ingestion parser plus the complexity, dataflow, and duplication walkers. It is
-a no-op for every language without a registered locator.
+tree-sitter `Parser` calls `prepare_source(language, source, path=abs_path)`
+first — the ingestion parser plus the complexity, dataflow, and duplication
+walkers. It is a no-op for every language without a registered locator or
+sanitizer.
+
+`prepare_source` also carries per-language byte-preserving sanitizers that
+aren't about multi-language files at all — a construct a grammar can't parse,
+where hitting it corrupts everything downstream. Object Pascal is the current
+example: `.dpr`/`.dpk`/`.lpr` project files write `unit in 'path.pas'` clauses
+in their `uses` list, a syntax tree-sitter-pascal has no rule for, and hitting
+one used to corrupt every unit named after it in the same clause. Its
+sanitizer (`prepare_pascal_source` in `ingestion/parser_helpers.py`) is gated
+on `path`'s extension — that syntax is invalid in a plain `.pas`/`.pp` unit
+file — and is a no-op everywhere else, same contract as the `_LOCATORS` path.
+Registering a sanitizer this way, rather than as an if-block in `parser.py`,
+means it stays "zero changes to `parser.py`" for a new language and the health
+walkers get the same clean projection the ingestion parser does.
### The locator registry
diff --git a/docs/layers/LANGUAGE_SUPPORT.md b/docs/layers/LANGUAGE_SUPPORT.md
index ca54bad17..15f214d52 100644
--- a/docs/layers/LANGUAGE_SUPPORT.md
+++ b/docs/layers/LANGUAGE_SUPPORT.md
@@ -1,6 +1,6 @@
# Language Support
-repowise parses **18 languages to a full AST**, resolves imports and call
+repowise parses **19 languages to a full AST**, resolves imports and call
graphs across them, and scores **13 at the Full tier** with code-health markers.
Everything else in your repo is still tracked through git history and appears in
the wiki. This page is the "what works for my language today" reference.
@@ -26,7 +26,7 @@ produce meaningful output.
| Tier | Languages | What works |
|------|-----------|------------|
| [**Full**](#full-tier) | Python · TypeScript · JavaScript · Svelte · Vue · Java · Kotlin · Go · Rust · C++ · C# · Scala · Ruby | AST parsing, import resolution, named bindings, call resolution, heritage, docstrings, framework-aware edges, dynamic-hint extractors, and **code-health markers** |
-| [**Good**](#good-tier) | C · Swift · PHP · Dart | Everything above except code-health markers (C, Swift, PHP; Dart *does* get health markers). Dedicated workspace resolvers and framework edges per language |
+| [**Good**](#good-tier) | C · Swift · PHP · Dart · Object Pascal | Everything above except code-health markers (C, Swift, PHP, Object Pascal; Dart *does* get health markers). Dedicated workspace resolvers and framework edges per language, except Object Pascal, which resolves imports via the generic stem-map fallback (see [Known gaps](#object-pascal-known-gaps)) |
| [**SQL / dbt**](#sql--dbt) | `.sql` via sqlglot | Tables / views / functions / procedures as symbols with wiki pages; dbt projects get real `ref()` / `source()` lineage |
| **Shell** | `.sh` `.bash` `.zsh` | Function definitions as symbols, `source` / `.` import edges (incl. `$SCRIPT_DIR` / `dirname` / `$BATS_ROOT` idioms), and function-level code-health complexity (CCN, nesting, cognitive). No class metrics, heritage, bindings, or dead-code flagging |
| **Config / data** | OpenAPI · Protobuf · GraphQL · Dockerfile · Makefile · YAML · JSON · TOML · Terraform · Markdown | In the file tree and wiki; special handlers extract endpoints / targets where applicable |
@@ -170,6 +170,35 @@ mixins). Dedicated workspace resolvers per language.
| **Swift** | `.swift` | `import` with SPM `Package.swift` target → directory mapping; intra-module type references; `@main` entry points |
| **PHP** | `.php` | `use Foo\Bar\Baz` with composer.json PSR-4 longest-prefix resolution; Laravel, TYPO3 edges |
| **Dart** | `.dart` | `import` / `export` / `part` URIs; `package:` via every `pubspec.yaml`; Flutter route tables and `runApp()` edges; **code-health markers** |
+| **Object Pascal** | `.pas` `.pp` `.dpr` `.dpk` `.lpr` `.inc` | `uses UnitA, UnitB;` resolved via the generic unit-name → file-stem fallback (no dedicated resolver); `.dpr`/`.dpk`/`.lpr` project files as entry points |
+
+### Object Pascal known gaps
+
+Delphi (`.pas`/`.dpr`/`.dpk`) and Free Pascal/Lazarus (`.pas`/`.pp`/`.lpr`) via
+`tree-sitter-pascal`. Newest AST-parsed language in this tier, so its ceilings
+are less battle-tested than C/Swift/PHP/Dart's:
+
+- **Type kind collapses to `"class"`.** `record` / `interface` / class-helper /
+ enum / plain type alias are all reported as `kind="class"` — the query
+ captures the declaration but not which of the five forms it is.
+- **`extends`/`implements` heritage split is best-effort.** Delphi's
+ `class(TBase, IFoo, IBar)` ancestor list doesn't itself distinguish a base
+ class from an implemented interface; the heritage extractor infers it from
+ naming convention (`I`-prefixed identifiers), which is the real-world Delphi
+ convention but not a language guarantee.
+- **No dedicated import resolver.** Unlike C/Swift/PHP/Dart, `uses` clauses
+ resolve through the same generic unit-name → file-stem fallback as the
+ Lightweight tier, rather than a project-file-aware resolver — accurate for
+ the near-universal "unit name equals file stem" convention, wrong when it
+ doesn't hold.
+- **No code-health markers yet** (complexity, duplication, dataflow dialects
+ aren't registered for Pascal).
+- **One known grammar gap left unhandled deliberately:** an anonymous
+ `array[...] of record ... end` element type has no tree-sitter-pascal rule
+ and degrades to a wrong `parent_name` for whatever the same class declares
+ afterward — contained to that one class, not fixed, since a correct fix
+ needs a nesting-aware scanner for one construct seen once in the validation
+ corpus. See `prepare_pascal_source` in `ingestion/parser_helpers.py`.
---
diff --git a/packages/core/src/repowise/core/analysis/health/complexity/walker.py b/packages/core/src/repowise/core/analysis/health/complexity/walker.py
index d3a97f132..c43adc058 100644
--- a/packages/core/src/repowise/core/analysis/health/complexity/walker.py
+++ b/packages/core/src/repowise/core/analysis/health/complexity/walker.py
@@ -120,8 +120,9 @@ def walk_file(
parser = Parser(grammar)
# SFCs reach the TS grammar as a markup-blanked buffer at
# byte-identical offsets, so every offset below (including the NLOC
- # slices, which read the ORIGINAL bytes) stays valid. No-op elsewhere.
- tree = parser.parse(prepare_source(language, source))
+ # slices, which read the ORIGINAL bytes) stays valid. Pascal gets its
+ # project-file sanitizer the same way. No-op elsewhere.
+ tree = parser.parse(prepare_source(language, source, path=abs_path))
except Exception as exc:
log.debug("complexity_walker_parse_failed", path=abs_path, error=str(exc))
return FileComplexity(functions=[], classes=[], file_nloc=_count_file_nloc(source))
diff --git a/packages/core/src/repowise/core/analysis/health/dataflow/parsing.py b/packages/core/src/repowise/core/analysis/health/dataflow/parsing.py
index ce9afe0fb..402049342 100644
--- a/packages/core/src/repowise/core/analysis/health/dataflow/parsing.py
+++ b/packages/core/src/repowise/core/analysis/health/dataflow/parsing.py
@@ -51,8 +51,9 @@ def parse_source(
from repowise.core.ingestion.sfc_source import prepare_source
parser = Parser(grammar)
- # Markup-blanked TS buffer for SFCs, identical offsets. No-op else.
- tree = parser.parse(prepare_source(language, source))
+ # Markup-blanked TS buffer for SFCs, identical offsets; Pascal's
+ # project-file sanitizer for that language. No-op else.
+ tree = parser.parse(prepare_source(language, source, path=abs_path))
except Exception as exc:
log.debug("dataflow_parse_failed", path=abs_path, error=str(exc))
return None
diff --git a/packages/core/src/repowise/core/analysis/health/duplication/detector.py b/packages/core/src/repowise/core/analysis/health/duplication/detector.py
index 9b64bc460..42580ab72 100644
--- a/packages/core/src/repowise/core/analysis/health/duplication/detector.py
+++ b/packages/core/src/repowise/core/analysis/health/duplication/detector.py
@@ -363,7 +363,7 @@ def _collect_windows(
for h, si, sl, el in window_tuples
]
else:
- toks = tokenize_file(language, source)
+ toks = tokenize_file(language, source, path)
if len(toks) > limits.max_tokens_per_file:
diag.skipped_token_cap += 1
continue
diff --git a/packages/core/src/repowise/core/analysis/health/duplication/tokenizer.py b/packages/core/src/repowise/core/analysis/health/duplication/tokenizer.py
index c855b8b8b..ac00903ea 100644
--- a/packages/core/src/repowise/core/analysis/health/duplication/tokenizer.py
+++ b/packages/core/src/repowise/core/analysis/health/duplication/tokenizer.py
@@ -140,11 +140,14 @@ def _tokenize_leaf(node: Node, source: bytes) -> Token | None:
)
-def tokenize_file(language: str, source: bytes) -> list[Token]:
+def tokenize_file(language: str, source: bytes, path: str | None = None) -> list[Token]:
"""Parse *source* and return its normalized token stream.
Returns an empty list when the language is unsupported or parsing
fails — callers treat that as "no clone candidates from this file".
+ ``path`` is only used by languages whose sanitizer needs it (Pascal,
+ to gate its project-file sanitizer on the extension); omit it for
+ everything else.
"""
try:
from tree_sitter import Parser
@@ -160,8 +163,9 @@ def tokenize_file(language: str, source: bytes) -> list[Token]:
from repowise.core.ingestion.sfc_source import prepare_source
parser = Parser(grammar)
- # Markup-blanked TS buffer for SFCs, identical offsets. No-op else.
- source = prepare_source(language, source)
+ # Markup-blanked TS buffer for SFCs, identical offsets; Pascal's
+ # project-file sanitizer for that language. No-op else.
+ source = prepare_source(language, source, path=path)
tree = parser.parse(source)
except Exception:
return []
diff --git a/packages/core/src/repowise/core/ingestion/extractors/heritage/__init__.py b/packages/core/src/repowise/core/ingestion/extractors/heritage/__init__.py
index d5ef913c8..00cdc0cb7 100644
--- a/packages/core/src/repowise/core/ingestion/extractors/heritage/__init__.py
+++ b/packages/core/src/repowise/core/ingestion/extractors/heritage/__init__.py
@@ -27,6 +27,7 @@ from .dart import _extract_dart_heritage
from .go import _extract_go_heritage
from .java import _extract_java_heritage
from .kotlin import _extract_kotlin_heritage
+from .pascal import _extract_pascal_heritage
from .php import _extract_php_heritage
from .python import _extract_python_heritage
from .ruby import _extract_ruby_heritage
@@ -60,6 +61,7 @@ HERITAGE_EXTRACTORS: dict[str, Callable[..., None]] = {
"swift": _extract_swift_heritage,
"scala": _extract_scala_heritage,
"php": _extract_php_heritage,
+ "pascal": _extract_pascal_heritage,
}
diff --git a/packages/core/src/repowise/core/ingestion/extractors/heritage/pascal.py b/packages/core/src/repowise/core/ingestion/extractors/heritage/pascal.py
new file mode 100644
index 000000000..b8d137e58
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/extractors/heritage/pascal.py
@@ -0,0 +1,71 @@
+"""Pascal heritage extraction."""
+
+from __future__ import annotations
+
+from tree_sitter import Node
+
+from ...models import HeritageRelation
+from ..helpers import node_text
+
+_CLASS_LIKE_KINDS = {"declClass", "declIntf", "declHelper"}
+
+
+def _extract_pascal_heritage(
+ def_node: Node, name: str, line: int, src: str, out: list[HeritageRelation]
+) -> None:
+ """Pascal: ``TFoo = class(TBar, IBaz)`` / ``IFoo = interface(IBar)`` /
+ ``TFooHelper = class helper(TBaseHelper) for TFoo``.
+
+ ``def_node`` is the outer ``declType`` node (``TFoo = ...;``), not the
+ ``declClass``/``declIntf``/``declHelper`` node directly -- see the note
+ in ``languages/specs/pascal.py`` on why ``heritage_node_types`` points
+ at ``declType``. Drill into declType's `type` field first to reach the
+ actual class/interface/helper body.
+
+ Pascal's grammar puts the base class and every implemented interface in
+ the *same* comma-separated `parent` list -- `class(TBase, IFoo, IBar)`
+ -- with no syntax distinguishing "the one base class" from "the
+ interfaces". By (near-universal) convention the first entry is the base
+ class and the rest are interfaces; nothing in the grammar enforces that
+ ordering, so this is a best-effort heuristic, not a guarantee.
+ """
+ type_children = def_node.children_by_field_name("type")
+ if not type_children:
+ return
+ type_node = type_children[0]
+ if type_node.type not in _CLASS_LIKE_KINDS:
+ return
+
+ parent_nodes = type_node.children_by_field_name("parent")
+ parents = [node_text(p, src).strip() for p in parent_nodes if p.type == "typeref"]
+ parents = [p for p in parents if p and p != name]
+
+ for i, parent in enumerate(parents):
+ out.append(
+ HeritageRelation(
+ child_name=name,
+ parent_name=parent,
+ kind="extends" if i == 0 else "implements",
+ line=line,
+ )
+ )
+
+ if type_node.type == "declHelper":
+ # The type being extended (`for TFoo`) is a distinct relationship
+ # from the helper's own ancestor helper (`parent`, handled above).
+ # It carries no field name in the grammar (`$.kFor, $.typeref` with
+ # no `field()` wrapper), so it's located positionally: the last
+ # named `typeref` child not already consumed as `parent`.
+ parent_typeref_ids = {p.id for p in parent_nodes}
+ for child in type_node.named_children:
+ if child.type == "typeref" and child.id not in parent_typeref_ids:
+ extended = node_text(child, src).strip()
+ if extended and extended != name:
+ out.append(
+ HeritageRelation(
+ child_name=name,
+ parent_name=extended,
+ kind="extends",
+ line=line,
+ )
+ )
diff --git a/packages/core/src/repowise/core/ingestion/language_configs.py b/packages/core/src/repowise/core/ingestion/language_configs.py
index 0d95b4bb6..906b338ed 100644
--- a/packages/core/src/repowise/core/ingestion/language_configs.py
+++ b/packages/core/src/repowise/core/ingestion/language_configs.py
@@ -344,6 +344,42 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
{"class_declaration", "interface_declaration", "trait_declaration", "enum_declaration"}
),
),
+ "pascal": LanguageConfig(
+ symbol_node_types={
+ # declType wraps class/record/interface/helper/enum/set/array/alias
+ # in one node shape (see spec docstring) -- "class" is the closest
+ # single bucket, same tradeoff Go makes for `type_spec`: "struct".
+ # A real kind refinement (peek at the `type` field to distinguish
+ # class/struct/interface/enum) would need a hook parser.py doesn't
+ # currently expose per-language the way it does for Go/Kotlin --
+ # flagged as a follow-up, not attempted here.
+ "declType": "class",
+ "declProc": "function", # signature only (interface decl / forward decl)
+ "defProc": "function", # full definition with body
+ # Matches C#'s choice for the same concept (property_declaration ->
+ # "variable"); "property" is not a valid SymbolKind literal.
+ "declProp": "variable",
+ },
+ import_node_types=["declUses"],
+ export_node_types=[], # Pascal has no explicit re-export syntax
+ # No pascal_visibility exists yet. Pascal visibility is per-*section*
+ # (`strict private`/`protected`/`public`/`published` governs every
+ # declaration until the next section keyword, i.e. extractors/
+ # visibility.py's per-declaration-modifier shape doesn't fit Pascal
+ # without a declSection sibling-walk). public_by_default is the same
+ # placeholder C/C++/JS/Ruby/Luau/Shell already use for real, not a
+ # Pascal-specific hack -- but it is a placeholder, not a real answer.
+ visibility_fn=public_by_default,
+ parent_extraction="nesting",
+ # NOT {"declClass", "declIntf", "declHelper"} -- see the spec docstring.
+ # ASTParser._find_parent walks ancestors checking `ancestor.type in
+ # parent_class_types` then reads `ancestor.child_by_field_name("name")`;
+ # only declType carries a name field, so pointing this at the inner
+ # class/interface/helper node would silently return no parent for every
+ # method (found by tracing _find_parent's actual implementation, not
+ # guessed).
+ parent_class_types=frozenset({"declType"}),
+ ),
"luau": LanguageConfig(
symbol_node_types={
"function_declaration": "function",
diff --git a/packages/core/src/repowise/core/ingestion/languages/specs/__init__.py b/packages/core/src/repowise/core/ingestion/languages/specs/__init__.py
index 219416cde..59ea863f2 100644
--- a/packages/core/src/repowise/core/ingestion/languages/specs/__init__.py
+++ b/packages/core/src/repowise/core/ingestion/languages/specs/__init__.py
@@ -41,6 +41,7 @@ from .nim import SPEC as _NIM
from .objectivec import SPEC as _OBJECTIVEC
from .ocaml import SPEC as _OCAML
from .openapi import SPEC as _OPENAPI
+from .pascal import SPEC as _PASCAL
from .php import SPEC as _PHP
from .proto import SPEC as _PROTO
from .python import SPEC as _PYTHON
@@ -91,6 +92,7 @@ ALL_SPECS: tuple[LanguageSpec, ...] = (
_SWIFT,
_SCALA,
_DART,
+ _PASCAL,
# -----------------------------------------------------------------
# Config / data / markup languages (passthrough — no AST)
# -----------------------------------------------------------------
diff --git a/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py b/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py
new file mode 100644
index 000000000..f9b148a78
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/languages/specs/pascal.py
@@ -0,0 +1,67 @@
+"""LanguageSpec for Object Pascal (Delphi / Free Pascal).
+
+Covers classic Object Pascal source: Delphi (.pas/.dpr/.dpk) and Free Pascal
+(.pas/.pp/.lpr). Grammar: tree-sitter-pascal (PyPI: tree-sitter-pascal,
+import name tree_sitter_pascal), maintained by Jim McKeeth
+(https://github.com/jimmckeeth/tree-sitter-pascal), itself descended from
+Isopod/tree-sitter-pascal. AGPL-3.0, same license family as repowise itself.
+
+heritage_node_types / parent_class_types are both "declType", not the more
+obvious "declClass"/"declIntf"/"declHelper". Pascal nests the actual class
+body one level inside a `Name = <class|interface|helper body>;` wrapper
+(`declType`), and only the wrapper carries a `name` field -- `declClass`
+itself has none. `ASTParser._find_parent` looks up
+`ancestor.child_by_field_name("name")` on whatever ancestor type matches
+`parent_class_types`, so pointing it at "declClass" would silently return
+no parent name for every method. See extractors/heritage/pascal.py for the
+matching one-level-deeper walk on the heritage side.
+"""
+
+from ..spec import LanguageSpec
+
+SPEC = LanguageSpec(
+ tag="pascal",
+ display_name="Object Pascal",
+ extensions=frozenset({".pas", ".pp", ".dpr", ".dpk", ".lpr", ".inc"}),
+ grammar_package="tree_sitter_pascal",
+ scm_file="pascal.scm",
+ heritage_node_types=frozenset({"declType"}),
+ # No dedicated resolver shipped in this first cut (see PR description) --
+ # imports still resolve via the generic stem-map fallback, which should
+ # cover the near-universal `unit name == file stem` Delphi/FPC convention.
+ import_support="none",
+ # Delphi project (.dpr) / package (.dpk) files and FPC/Lazarus project
+ # (.lpr) are the closest equivalent to a language "main" -- each contains
+ # exactly one top-level `program`/`library`/`package` block.
+ entry_point_patterns=("*.dpr", "*.dpk", "*.lpr"),
+ # No de-facto single manifest file across the ecosystem: Delphi project
+ # metadata lives in .dproj (MSBuild XML, not Pascal source), FPC/Lazarus
+ # in .lpi (also XML). Neither is source, so left empty rather than wired
+ # to a non-Pascal parser.
+ manifest_files=(),
+ shebang_tokens=(),
+ # Common RTL calls worth excluding from the project call graph, spelled
+ # the way the RTL itself declares them (Pascal is case-insensitive, but
+ # `ASTParser._extract_calls` does a case-sensitive `target_name in
+ # _call_builtins` check against the raw source text, so entries need to
+ # match real-world casing, not lowercased). This list is intentionally
+ # conservative -- System-unit built-ins only; framework calls (VCL/FMX/
+ # LCL) are ordinary project dependencies and should still show up as
+ # graph edges.
+ builtin_calls=frozenset({
+ "WriteLn", "Write", "ReadLn", "Read",
+ "Length", "SetLength", "High", "Low", "Ord", "Chr", "Succ", "Pred",
+ "Inc", "Dec", "Copy", "Pos", "Concat", "Trim", "TrimLeft", "TrimRight",
+ "Format", "IntToStr", "StrToInt", "StrToIntDef", "FloatToStr",
+ "Assigned", "Exit",
+ "New", "Dispose", "GetMem", "FreeMem", "ReallocMem",
+ "FreeAndNil", "Assert", "Supports", "TypeInfo",
+ }),
+ # Universal Delphi/FPC root types -- filtered the same way "object" is
+ # filtered from Python heritage or "NSObject" from Swift.
+ builtin_parents=frozenset({
+ "TObject", "TInterfacedObject", "TPersistent", "TComponent",
+ "IUnknown", "IInterface", "IDispatch",
+ }),
+ color_hex="#E3352E",
+)
diff --git a/packages/core/src/repowise/core/ingestion/models.py b/packages/core/src/repowise/core/ingestion/models.py
index 512741587..919cb3baf 100644
--- a/packages/core/src/repowise/core/ingestion/models.py
+++ b/packages/core/src/repowise/core/ingestion/models.py
@@ -38,6 +38,7 @@ LanguageTag = Literal[
"scala",
"luau",
"dart",
+ "pascal",
# Passthrough code languages (no AST parser yet — empty ParsedFile,
# files enter the graph via the generic resolver). Before these tags
# existed the traverser silently skipped such files as unknown, so e.g.
diff --git a/packages/core/src/repowise/core/ingestion/parser.py b/packages/core/src/repowise/core/ingestion/parser.py
index 560fa3a29..ff003233c 100644
--- a/packages/core/src/repowise/core/ingestion/parser.py
+++ b/packages/core/src/repowise/core/ingestion/parser.py
@@ -70,11 +70,13 @@ from .parser_helpers import (
_classify_param_origin,
_collect_error_nodes,
_count_arguments,
+ _dedupe_pascal_interface_symbols,
_find_enclosing_symbol,
_has_callable_ancestor,
_head_type_identifier,
_is_async_node,
_qualified_cpp_parent,
+ _qualified_pascal_parent,
_run_query,
)
from .python_local_refs import extract_python_local_refs
@@ -286,11 +288,15 @@ class ASTParser:
# ``prepare_source`` blanks the markup and <style> so what reaches the
# TypeScript grammar is valid TS at byte-identical offsets — no offset
# translation is needed anywhere downstream. A no-op for every other
- # language.
+ # language without a registered locator/sanitizer -- Pascal's
+ # sanitizers (project-file `in '...'` clauses, ERROR-node blanking)
+ # live behind the same hook; see prepare_pascal_source in
+ # parser_helpers.py for why they're wired in here rather than as
+ # ad-hoc if-blocks.
# ``content_hash`` above deliberately hashes the ORIGINAL bytes, so
# incremental update still tracks the real file.
original_source = source
- source = prepare_source(lang, source)
+ source = prepare_source(lang, source, path=file_info.path)
parser = Parser(language)
tree = parser.parse(source)
@@ -425,6 +431,10 @@ class ASTParser:
) -> list[Symbol]:
symbols: list[Symbol] = []
seen: set[tuple[int, str]] = set() # (start_line, name) — dedup decorated dupes
+ # Parallel to ``symbols`` (same indices) -- only populated/consumed
+ # for Pascal, to dedupe interface-declaration vs. implementation
+ # method pairs after the loop. See _dedupe_pascal_interface_symbols.
+ node_types: list[str] = []
# Deferred-export names (``export { x }`` / ``export default x``),
# computed once per file for the TS/JS visibility refinement.
@@ -608,6 +618,15 @@ class ASTParser:
if parent_name is None and file_info.language in ("cpp", "c") and name_nodes:
parent_name = _qualified_cpp_parent(name_nodes[0], src)
+ # Pascal out-of-line implementation: ``function TFoo.Bar(...);``
+ # -- the ``defProc`` node lives in the unit's implementation
+ # section, outside the class's ``declType`` body declared in the
+ # interface section, so nesting-based ``_find_parent`` above
+ # can't see it. The qualifying class lives beside the captured
+ # name in the ``genericDot`` header instead.
+ if parent_name is None and file_info.language == "pascal" and name_nodes:
+ parent_name = _qualified_pascal_parent(name_nodes[0], src)
+
# Upgrade function → method when a parent class is detected
if parent_name and kind == "function":
kind = "method"
@@ -646,6 +665,10 @@ class ASTParser:
is_exported_symbol=is_exported_symbol,
)
)
+ node_types.append(node_type)
+
+ if file_info.language == "pascal":
+ symbols = _dedupe_pascal_interface_symbols(symbols, node_types)
return symbols
@@ -701,6 +724,7 @@ class ASTParser:
) -> list[Import]:
imports: list[Import] = []
seen_raws: set[str] = set()
+ seen_pascal_units: set[str] = set()
for capture_dict in matches:
stmt_nodes = capture_dict.get("import.statement", [])
@@ -710,6 +734,42 @@ class ASTParser:
continue
stmt_node = stmt_nodes[0]
+
+ # Pascal: `uses UnitA, UnitB, Ns.UnitC;` -- pascal.scm's
+ # unquantified pattern (see that file's comment on why) fires
+ # once PER moduleName, so a 3-unit clause arrives as 3 separate
+ # matches sharing one @import.statement span, each carrying a
+ # single-element ``module_nodes``. Handled before the
+ # ``seen_raws`` dedup below: that guard exists to skip a
+ # statement re-matched by an *overlapping* pattern (the normal
+ # case elsewhere), but here every match legitimately carries a
+ # different unit despite the identical raw statement text --
+ # dedup-by-raw would keep only the first and silently drop the
+ # rest, which is exactly the bug this branch fixes.
+ #
+ # Deduped separately by unit name (case-insensitive -- Pascal
+ # identifiers are): a unit named in both the ``interface`` and
+ # ``implementation`` ``uses`` clauses of the same file is a
+ # single logical dependency and must not become two Import
+ # entries for it.
+ if file_info.language == "pascal":
+ raw = _node_text(stmt_node, src).strip()
+ unit_name = _node_text(module_nodes[0], src).strip()
+ if unit_name and unit_name.lower() not in seen_pascal_units:
+ seen_pascal_units.add(unit_name.lower())
+ imports.append(
+ Import(
+ raw_statement=raw,
+ module_path=unit_name,
+ imported_names=[],
+ is_relative=False,
+ resolved_file=None,
+ bindings=[],
+ is_reexport=False,
+ )
+ )
+ continue
+
raw = _node_text(stmt_node, src).strip()
if raw in seen_raws:
continue
diff --git a/packages/core/src/repowise/core/ingestion/parser_helpers.py b/packages/core/src/repowise/core/ingestion/parser_helpers.py
index 81031d6c0..db3f23208 100644
--- a/packages/core/src/repowise/core/ingestion/parser_helpers.py
+++ b/packages/core/src/repowise/core/ingestion/parser_helpers.py
@@ -9,19 +9,26 @@ this one holds the free functions it calls. No state, no imports from
from __future__ import annotations
+import re
from collections.abc import Callable
from pathlib import Path
+from typing import TYPE_CHECKING
import structlog
from tree_sitter import Node
from .extractors import node_text
+if TYPE_CHECKING:
+ from .models import Symbol
+
log = structlog.get_logger(__name__)
# Private alias for internal use (mirrors the one in parser.py)
_node_text = node_text
+_WHITESPACE_RE = re.compile(r"\s+")
+
def _run_query(query: object, root_node: Node) -> list[dict[str, list[Node]]]:
"""Execute a tree-sitter query and return a list of capture dicts."""
@@ -111,6 +118,40 @@ def _qualified_cpp_parent(name_node: Node, src: str) -> str | None:
return text.rsplit("::", 1)[-1] or None
+def _qualified_pascal_parent(name_node: Node, src: str) -> str | None:
+ """Return the owning class for a Pascal out-of-line method header.
+
+ ``function TCalculator.Add(...): Integer;`` in an implementation
+ section is captured by pascal.scm's ``genericDot`` patterns, where the
+ bare ``@symbol.name`` is the ``rhs`` (``Add``) and the qualifying type
+ lives in the sibling ``lhs`` field (``TCalculator``). Nesting-based
+ ``_find_parent`` can't see this: the ``defProc`` node sits in the
+ unit's implementation section, physically outside the class's
+ ``declType`` body declared in the interface section. Handles both the
+ plain (``genericDot rhs: identifier``) and generic-method
+ (``genericDot rhs: genericTpl entity: identifier``) query shapes.
+
+ Uses ``node_text`` (tree-sitter's own byte-accurate decode), not raw
+ ``src`` byte-offset slicing — Pascal identifiers and unit names are
+ frequently non-ASCII (Cyrillic) in this codebase's real-world sources,
+ and slicing a decoded ``str`` by *byte* offsets misaligns on any
+ multi-byte character.
+
+ Returns ``None`` when the name node isn't inside a qualified header
+ (i.e. a free function/procedure).
+ """
+ parent = name_node.parent
+ if parent is not None and parent.type == "genericTpl":
+ parent = parent.parent
+ if parent is None or parent.type != "genericDot":
+ return None
+ lhs = parent.child_by_field_name("lhs")
+ if lhs is None:
+ return None
+ text = node_text(lhs, src).strip()
+ return text or None
+
+
def _build_qualified_name(file_path: str, parent_name: str | None, name: str) -> str:
module = Path(file_path).with_suffix("").as_posix().replace("/", ".")
if parent_name:
@@ -118,6 +159,163 @@ def _build_qualified_name(file_path: str, parent_name: str | None, name: str) ->
return f"{module}.{name}"
+_PASCAL_USES_IN_CLAUSE_RE = re.compile(rb"\bin\b[ \t]*'(?:[^'\r\n]|'')*'")
+
+
+def _sanitize_pascal_project_source(source: bytes) -> bytes:
+ """Blank Delphi/FPC project-file ``unit in 'path.pas'`` clauses.
+
+ ``.dpr``/``.dpk``/``.lpr`` project files map each unit to its source
+ path right in the ``uses`` clause -- ``uses SysUtils, MyUnit in
+ 'src\\MyUnit.pas';`` -- and Delphi's IDE writes this automatically for
+ every unit added to a project, making it the norm rather than the
+ exception in real ``.dpr``/``.dpk`` files (confirmed against this
+ repo's own ``MTN2.dpr``: every non-RTL unit uses it).
+
+ tree-sitter-pascal's grammar has no rule for the trailing ``in
+ '...'`` at all. Hitting it mid-``declUses`` doesn't just fail that
+ one unit -- the parser's error recovery folds the ``in``, the path
+ string, and every subsequent comma-separated unit into one corrupted
+ ``moduleName`` node spanning to wherever it happens to resync, so a
+ single ``in`` clause was silently swallowing the rest of the ``uses``
+ list (observed on ``MTN2.dpr``: 4 imports extracted instead of ~80,
+ the 4th holding several KB of raw multi-line garbage as its
+ ``module_path``). This is an upstream grammar gap, not something a
+ ``.scm`` query can route around -- the AST itself is malformed before
+ any query runs.
+
+ Blanks the matched span with spaces (never a raw newline -- a Pascal
+ string literal can't contain one, so no line is fully consumed) to
+ preserve every other byte offset in the file, so line numbers for
+ symbols/imports/calls elsewhere are unaffected. `'ABC'` doesn't need
+ the doubled-quote (`''`) escape handled specially for *finding* the
+ end of the string here (the regex already treats `''` as staying
+ inside the literal), only for correctness of the match's own extent.
+
+ Scoped to project files specifically: this syntax is invalid outside
+ a ``uses`` clause and ``.pas``/``.pp`` unit files can't legally carry
+ it, so there's nothing to blank there and no reason to run the regex
+ over every unit file in a codebase.
+ """
+ if not _PASCAL_USES_IN_CLAUSE_RE.search(source):
+ return source
+ out = bytearray(source)
+ for m in _PASCAL_USES_IN_CLAUSE_RE.finditer(source):
+ start, end = m.span()
+ out[start:end] = b" " * (end - start)
+ return bytes(out)
+
+
+_PASCAL_PROJECT_EXTENSIONS = (".dpr", ".dpk", ".lpr")
+
+
+def prepare_pascal_source(source: bytes, path: str | None) -> bytes:
+ """Single entry point for every Pascal byte-preserving sanitizer.
+
+ Called from :func:`~.sfc_source.prepare_source` -- the same
+ registry-dispatched hook every other tree-sitter consumer (the
+ ingestion parser, plus the complexity/dataflow/duplication health
+ walkers) already calls before handing bytes to a ``Parser`` -- rather
+ than parser.py special-casing Pascal in its own if-blocks. That keeps
+ ``docs/architecture/language-support.md``'s "zero changes to
+ parser.py" promise for a new language, and means the health walkers
+ get the same clean projection the ingestion parser does instead of
+ parsing raw bytes.
+
+ Only wraps ``_sanitize_pascal_project_source`` (``.dpr``/``.dpk``/
+ ``.lpr`` ``in '...'`` clauses), gated on *path*'s extension since that
+ syntax is invalid in a plain unit file. An earlier revision of this
+ function also blanked whatever an anonymous ``array[...] of record``
+ element type's parse errors touched, discovered via ERROR-node spans
+ from a throwaway parse. Dropped after review (PR #1353): tree-sitter's
+ error recovery for that construct doesn't cleanly wrap the bad
+ construct in one ERROR node -- on the reviewer's repro, one of the
+ spans it found was the class's own legitimate closing ``end;``, and
+ blanking it produced the exact same broken structure (the following
+ method detached from its class) as running no sanitizer at all. A
+ correct fix needs a nesting-aware nested-record/variant-part scanner,
+ which is more surface area than one occurrence in one file (see the
+ dropped function's own docstring) justifies; the anon-record case is
+ left to degrade to a wrong parent for that one class, same as any
+ other unhandled grammar gap.
+ """
+ if path and path.lower().endswith(_PASCAL_PROJECT_EXTENSIONS):
+ return _sanitize_pascal_project_source(source)
+ return source
+
+
+def _dedupe_pascal_interface_symbols(
+ symbols: list[Symbol], node_types: list[str]
+) -> list[Symbol]:
+ """Drop an interface-section method signature once its implementation
+ is also present, so the two don't become two graph nodes for one method.
+
+ Pascal declares a method's signature once in the ``interface`` section
+ (``declProc``, no body) and its full body once in the
+ ``implementation`` section (``defProc``) — two distinct physical AST
+ nodes pascal.scm both legitimately captures (see the query file's
+ comment). Once ``_find_parent`` (nesting) and ``_qualified_pascal_parent``
+ (the ``TFoo.Method`` header) resolve both to the same
+ ``(parent_name, name)``, keep only the ``defProc`` version: it carries
+ the real body, which is what ``get_symbol`` should return, and the
+ ``declProc`` duplicate would otherwise leave two ``Add`` nodes in the
+ graph for one logical method.
+
+ Keyed on ``(parent_name, signature)`` — normalized, see
+ ``_pascal_dedupe_key`` — rather than just ``(parent_name, name)`` so
+ that Pascal ``overload;`` siblings (same name, different parameter
+ lists) are told apart: an interface-only overload must survive even
+ when a *different* overload of the same name has a same-file
+ implementation (verified against a reproduction where a 2-overload
+ class with only one variant implemented was silently losing the
+ other variant's interface declaration).
+
+ Normalization matters in practice, not just in theory: scanned
+ against a real ~150-file Delphi codebase, 168 method pairs shared a
+ class+name but escaped a raw-signature-text match — almost all of
+ them a long parameter list wrapped across lines differently between
+ the compact interface declaration and the implementation (extremely
+ common Delphi formatting), a handful differing only by identifier
+ case (Pascal is case-insensitive, so ``TFoo.Add`` and
+ ``TFOO.ADD`` name the same method). ``_pascal_dedupe_key`` strips all
+ whitespace and lowercases before comparing so both collapse
+ correctly.
+
+ Still imperfect: Pascal's compiler doesn't require parameter *names*
+ to match between an interface declaration and its implementation
+ (only the types, for overload resolution), so a same-file rename
+ between the two still produces different normalized keys and defeats
+ this dedup, leaving both symbols. Unlike the whitespace/case cases
+ above, no evidence of this actually happening was found in the real
+ codebase this was checked against — left as a documented gap rather
+ than parsing parameter types out of ``declArgs`` for an exact match.
+ """
+ impl_keys = {
+ _pascal_dedupe_key(s.parent_name, s.signature)
+ for s, nt in zip(symbols, node_types, strict=True)
+ if nt == "defProc"
+ }
+ return [
+ s
+ for s, nt in zip(symbols, node_types, strict=True)
+ if not (nt == "declProc" and _pascal_dedupe_key(s.parent_name, s.signature) in impl_keys)
+ ]
+
+
+def _pascal_dedupe_key(parent_name: str | None, signature: str) -> tuple[str | None, str]:
+ """Normalize a (parent, signature) pair for Pascal's interface/impl dedup.
+
+ Whitespace-insensitive (multi-line parameter lists get reformatted
+ between the interface declaration and the implementation constantly
+ in real Delphi code) and case-insensitive (identifiers are
+ case-insensitive in Pascal, so this needs to hold for the *class*
+ name half of the key too, not just the signature).
+ """
+ parent_key = parent_name.lower() if parent_name else None
+ sig_key = _WHITESPACE_RE.sub("", signature).lower()
+ return (parent_key, sig_key)
+
+
# ---------------------------------------------------------------------------
# Type reference helpers (used by _extract_type_refs)
# ---------------------------------------------------------------------------
diff --git a/packages/core/src/repowise/core/ingestion/queries/pascal.scm b/packages/core/src/repowise/core/ingestion/queries/pascal.scm
new file mode 100644
index 000000000..8b8991d64
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/queries/pascal.scm
@@ -0,0 +1,190 @@
+; =============================================================================
+; repowise — Object Pascal (Delphi / Free Pascal) symbol and import queries
+; tree-sitter-pascal (jimmckeeth/tree-sitter-pascal) >= 0.11
+; =============================================================================
+;
+; Node-shape reference (grammar.js / src/node-types.json, 0.11.0):
+; declUses := kUses (moduleName)+ ';' -- no field name on children
+; moduleName := identifier ('.' identifier)* -- dotted unit name
+; declType := name:(_genericName) '=' type:(...) -- covers class/record/
+; interface/helper/enum/set/array/alias in one node; only
+; declType carries a `name` field (declClass/declIntf/
+; declHelper do not), which is why @symbol.def wraps the
+; outer declType rather than the inner class/interface node.
+; declClass := ('class'|'record'|'object') parent:(typeref (',' typeref)*)? ... 'end'
+; declIntf := 'interface' parent:(typeref...)? guid:(...)? ... 'end'
+; declHelper := ('class'|'record'|'type') 'helper' parent:(typeref...)? 'for' typeref ... 'end'
+; declProc := kind name:(_genericName) args:(declArgs)? (':' type:(typeref))? ';'
+; -- signature only (interface decl, forward decl, or a
+; defProc's `header` field); no body
+; defProc := header:(declProc) local:(...)? body:(block|asm) ';'
+; -- full implementation with body
+; declProp := 'property' name:(identifier) args:(declPropArgs)? ...
+; _genericName is one of: identifier | genericDot{lhs,rhs} | genericTpl{entity,args}
+; so a name capture needs 4 shapes: bare, generic (Foo<T>), qualified
+; (TFoo.Bar), and qualified+generic (TFoo.Bar<T>) -- mirrors the shipped
+; queries/highlights.scm in the grammar's own repo.
+
+; ---------------------------------------------------------------
+; Imports -- `uses UnitA, Ns.UnitB;`
+; Valid (and captured) in program/library/unit interface AND
+; implementation sections alike; not scoped to either.
+; ---------------------------------------------------------------
+
+; No quantifier on purpose: `moduleName` siblings inside `declUses` are
+; separated by literal `,` tokens, so a `+`/`*` quantifier here can't
+; span them (tree-sitter quantifiers require adjacency) -- it collapses
+; to a single match on the FIRST unit and silently drops the rest.
+; Leaving this unquantified instead yields one match per moduleName
+; (all sharing the same @import.statement span), which parser.py's
+; Pascal-specific _extract_imports branch relies on to emit every unit
+; in `uses A, B, C;` -- see that branch for why the match count matters.
+(declUses
+ (moduleName) @import.module) @import.statement
+
+; ---------------------------------------------------------------
+; Type declarations: class / record / interface / class helper /
+; enum / set / array / plain alias -- all wrapped in `declType`.
+; ---------------------------------------------------------------
+
+(declType
+ name: (identifier) @symbol.name) @symbol.def
+
+(declType
+ name: (genericTpl entity: (identifier) @symbol.name)) @symbol.def
+
+; ---------------------------------------------------------------
+; Procedure / function / constructor / destructor / operator --
+; signature-only node (interface declaration or forward declaration).
+; The grammar folds all five keyword kinds into one `declProc` node
+; shape, so one set of patterns covers all of them.
+; ---------------------------------------------------------------
+
+(declProc
+ name: (identifier) @symbol.name
+ args: (declArgs)? @symbol.params) @symbol.def
+
+(declProc
+ name: (genericTpl entity: (identifier) @symbol.name)
+ args: (declArgs)? @symbol.params) @symbol.def
+
+; Qualified method name in an implementation header, e.g.
+; `procedure TDualPanelWindow.HandleInput(...)`
+(declProc
+ name: (genericDot rhs: (identifier) @symbol.name)
+ args: (declArgs)? @symbol.params) @symbol.def
+
+(declProc
+ name: (genericDot rhs: (genericTpl entity: (identifier) @symbol.name))
+ args: (declArgs)? @symbol.params) @symbol.def
+
+; ---------------------------------------------------------------
+; Full definitions (header + body) in an implementation section.
+; Re-captures the name one level down (inside `header:`) so the
+; @symbol.def span covers header+body instead of just the header --
+; get_symbol on a method should return the body, not just the
+; prototype. This is a second, distinct physical node from the
+; interface-section signature above (Pascal separates forward
+; declaration from implementation), so both legitimately fire, and
+; ASTParser's local-vs-top-level filter (`_has_callable_ancestor`)
+; uses this defProc ancestor to correctly suppress nested local
+; procedures from the top-level symbol list.
+; ---------------------------------------------------------------
+
+(defProc
+ header: (declProc
+ name: (identifier) @symbol.name
+ args: (declArgs)? @symbol.params)) @symbol.def
+
+(defProc
+ header: (declProc
+ name: (genericTpl entity: (identifier) @symbol.name)
+ args: (declArgs)? @symbol.params)) @symbol.def
+
+(defProc
+ header: (declProc
+ name: (genericDot rhs: (identifier) @symbol.name)
+ args: (declArgs)? @symbol.params)) @symbol.def
+
+(defProc
+ header: (declProc
+ name: (genericDot rhs: (genericTpl entity: (identifier) @symbol.name))
+ args: (declArgs)? @symbol.params)) @symbol.def
+
+; ---------------------------------------------------------------
+; Properties -- `property Foo: Integer read FFoo write SetFoo;`
+; ---------------------------------------------------------------
+
+(declProp
+ name: (identifier) @symbol.name
+ args: (declPropArgs)? @symbol.params) @symbol.def
+
+; ---------------------------------------------------------------
+; Call graph -- `Foo(...)`, `Foo.Bar(...)`, `Foo<T>(...)`,
+; `Foo.Bar<T>(...)`. Capture names match ASTParser._extract_calls:
+; call.site / call.target / call.receiver / call.arguments.
+; ---------------------------------------------------------------
+
+(exprCall
+ entity: (identifier) @call.target
+ args: (exprArgs)? @call.arguments) @call.site
+
+(exprCall
+ entity: (exprTpl entity: (identifier) @call.target)
+ args: (exprArgs)? @call.arguments) @call.site
+
+(exprCall
+ entity: (exprDot
+ lhs: (_) @call.receiver
+ rhs: (identifier) @call.target)
+ args: (exprArgs)? @call.arguments) @call.site
+
+(exprCall
+ entity: (exprDot
+ lhs: (_) @call.receiver
+ rhs: (exprTpl entity: (identifier) @call.target))
+ args: (exprArgs)? @call.arguments) @call.site
+
+; ---------------------------------------------------------------
+; Parenless calls -- `Foo;`, `Obj.Foo;`. A parameterless procedure or
+; method call is idiomatic Pascal and drops the `()` entirely, so the
+; grammar never wraps it in `exprCall` at all: a statement consisting of
+; just a bare `identifier` or `exprDot` node (checked against real MTN2
+; source: ~2700 bare-identifier and ~360 bare-exprDot statements, vs.
+; ~6700 already-captured parenthesised calls -- this was silently
+; dropping a comparable-sized share of the call graph). Anchored to
+; `statement`'s bare form specifically (not just "any identifier/exprDot
+; anywhere") so this doesn't also fire on the LHS of an `assignment`
+; node (a sibling of `statement`, not nested in it -- see grammar note)
+; or inside a `goto`/`label`/`raise`/`with`/`for` header, none of which
+; wrap their identifier in `statement`.
+(statement
+ (identifier) @call.target) @call.site
+
+(statement
+ (exprDot
+ lhs: (_) @call.receiver
+ rhs: (identifier) @call.target)) @call.site
+
+(statement
+ (exprDot
+ lhs: (_) @call.receiver
+ rhs: (exprTpl entity: (identifier) @call.target))) @call.site
+
+; ---------------------------------------------------------------
+; `inherited Foo;` / `inherited Foo(...)` -- calls the base class's
+; same-named method. The bare `inherited;` form (no explicit name) is
+; NOT captured here: the target name isn't in the node's own text at
+; all -- resolving it needs the *enclosing* method's name, which is
+; extraction-time context this query layer doesn't have. Left as a
+; known gap rather than adding that lookup in a first cut.
+; ---------------------------------------------------------------
+
+(statement
+ (inherited
+ (identifier) @call.target)) @call.site
+
+(exprCall
+ entity: (inherited
+ (identifier) @call.target)
+ args: (exprArgs)? @call.arguments) @call.site
diff --git a/packages/core/src/repowise/core/ingestion/sfc_source.py b/packages/core/src/repowise/core/ingestion/sfc_source.py
index 392213bad..55582478f 100644
--- a/packages/core/src/repowise/core/ingestion/sfc_source.py
+++ b/packages/core/src/repowise/core/ingestion/sfc_source.py
@@ -35,6 +35,12 @@ items"`` and ``v-slot``/``#default``. They *parse* as JS — ``item in items`` i
the ``in`` operator — but mean something else, and a parse that succeeds with
the wrong meaning is worse than a skip. The names they bind are declarations,
not reads, so keeping them would mint phantom identifier references.
+
+:func:`prepare_source` is also where a language registers a byte-preserving
+sanitizer for a grammar gap that has nothing to do with multi-language files —
+Pascal's project-file ``uses X in 'path'`` clauses being the current example,
+handled by ``prepare_pascal_source`` in ``parser_helpers.py``. Same contract as
+a :class:`Locator`, no-op unless the language matches.
"""
from __future__ import annotations
@@ -486,14 +492,26 @@ def component_call_sites(language: str, source: bytes, symbols: list) -> list:
return calls
-def prepare_source(language: str, source: bytes) -> bytes:
+def prepare_source(language: str, source: bytes, *, path: str | None = None) -> bytes:
"""Return *source* as the bytes the tree-sitter pipeline should parse.
- A no-op for every language without a registered locator. Call this at each
- point that hands raw file bytes to a tree-sitter ``Parser`` — the ingestion
- parser and the three code-health walkers — so all of them see the same
- TypeScript projection of a component at the same offsets.
+ A no-op for every language without a registered locator or sanitizer.
+ Call this at each point that hands raw file bytes to a tree-sitter
+ ``Parser`` — the ingestion parser and the three code-health walkers —
+ so all of them see the same projection at the same offsets.
+
+ ``path`` only matters for a sanitizer that needs it (currently just
+ Pascal, which gates its project-file ``uses X in 'path'`` sanitizer on
+ the extension): every call site already has it in scope from its own
+ file read, so threading it through here costs callers nothing and lets
+ ``_LOCATORS``-style languages ignore it entirely.
"""
- if language not in _LOCATORS or not source:
- return source
- return _blank(source, scan(language, source))
+ if language in _LOCATORS:
+ if not source:
+ return source
+ return _blank(source, scan(language, source))
+ if language == "pascal" and source:
+ from .parser_helpers import prepare_pascal_source
+
+ return prepare_pascal_source(source, path)
+ return source
diff --git a/pyproject.toml b/pyproject.toml
index eaf962ff6..395f81853 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -49,6 +49,7 @@ dependencies = [
"tree-sitter-swift>=0.0.1",
"tree-sitter-scala>=0.23,<1",
"tree-sitter-php>=0.23,<1",
+ "tree-sitter-pascal>=0.11,<1",
"tree-sitter-luau>=1.2,<2",
"tree-sitter-bash>=0.23,<1",
# Markup grammars for single-file components: these locate <script> blocks
diff --git a/tests/__init__.py b/tests/__init__.py
new file mode 100644
index 000000000..e69de29bb
diff --git a/tests/unit/ingestion/parser/test_pascal.py b/tests/unit/ingestion/parser/test_pascal.py
new file mode 100644
index 000000000..bfc89393e
--- /dev/null
+++ b/tests/unit/ingestion/parser/test_pascal.py
@@ -0,0 +1,602 @@
+"""Unit tests for the Pascal (Delphi / Free Pascal) language pipeline.
+
+Tests parse inline byte strings so no filesystem I/O is needed. Covers
+symbols, the interface/implementation dedup, imports (including the
+multi-unit ``uses`` clause), and heritage -- see docs/architecture/
+language-support.md's "one .scm file + one LanguageConfig" recipe.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from repowise.core.ingestion.parser import ASTParser
+from tests.unit.ingestion.parser._helpers import _make_file_info
+
+# tree-sitter-pascal is a real dep in pyproject.toml, but it is sometimes
+# absent from a partially-synced developer venv. Skip explicitly so the
+# failure mode is "go run uv sync" rather than confusing AssertionErrors.
+pytest.importorskip("tree_sitter_pascal", reason="run `uv sync --all-packages`")
+
+
+def _pas(path: str = "Calc.pas") -> object:
+ return _make_file_info(path, "pascal")
+
+
+UNIT_SOURCE = b"""\
+unit Calc;
+
+interface
+
+uses
+ SysUtils, Classes, Ns.UnitC;
+
+type
+ TCalculator = class(TObject, IFoo)
+ public
+ function Add(X, Y: Integer): Integer;
+ end;
+
+ ICalcTarget = interface(IInterface)
+ procedure Reset;
+ end;
+
+implementation
+
+function TCalculator.Add(X, Y: Integer): Integer;
+begin
+ Result := X + Y;
+end;
+
+end.
+"""
+
+
+class TestPascalSymbols:
+ def test_finds_class_and_top_level_kinds(self, parser: ASTParser) -> None:
+ result = parser.parse_file(_pas(), UNIT_SOURCE)
+ kinds = {(s.name, s.kind) for s in result.symbols}
+ assert ("TCalculator", "class") in kinds
+ assert ("ICalcTarget", "class") in kinds
+
+ def test_interface_and_implementation_method_collapse_to_one_symbol(
+ self, parser: ASTParser
+ ) -> None:
+ # TCalculator.Add is declared once in the interface section
+ # (signature only) and defined once in the implementation section
+ # (with a body) -- two distinct physical AST nodes for one logical
+ # method. Review feedback on PR #1353: this must not surface as two
+ # graph nodes.
+ result = parser.parse_file(_pas(), UNIT_SOURCE)
+ adds = [s for s in result.symbols if s.name == "Add"]
+ assert len(adds) == 1
+ sym = adds[0]
+ assert sym.kind == "method"
+ assert sym.parent_name == "TCalculator"
+
+ def test_surviving_symbol_is_the_implementation_with_a_body(
+ self, parser: ASTParser
+ ) -> None:
+ # get_symbol on a method should return the real body, not just the
+ # bare interface-section prototype -- the kept symbol's line range
+ # must span the implementation (with `begin ... end;`), not the
+ # single-line interface declaration.
+ result = parser.parse_file(_pas(), UNIT_SOURCE)
+ sym = next(s for s in result.symbols if s.name == "Add")
+ assert sym.end_line > sym.start_line
+
+ def test_interface_only_method_is_kept(self, parser: ASTParser) -> None:
+ # ICalcTarget.Reset has no implementation in this snippet -- an
+ # interface-section-only declaration must not be dropped just
+ # because *some* Pascal methods in the file have a defProc pair.
+ result = parser.parse_file(_pas(), UNIT_SOURCE)
+ names = {(s.name, s.parent_name) for s in result.symbols}
+ assert ("Reset", "ICalcTarget") in names
+
+ def test_overloads_are_told_apart_by_signature(self, parser: ASTParser) -> None:
+ # Regression: an earlier version of the dedup keyed only on
+ # (parent_name, name), so a 2-overload class with only ONE variant
+ # implemented in this unit silently dropped the OTHER (unimplemented)
+ # overload's interface-only declaration -- real data loss, not just
+ # a duplicate. Keying on (parent_name, signature) instead tells the
+ # two `Add` overloads apart because their parameter lists differ.
+ src = b"""\
+unit Foo;
+interface
+type
+ TFoo = class
+ function Add(X: Integer): Integer; overload;
+ function Add(X: Double): Double; overload;
+ end;
+implementation
+function TFoo.Add(X: Integer): Integer;
+begin
+ Result := X;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ adds = {(s.signature, s.end_line > s.start_line) for s in result.symbols if s.name == "Add"}
+ assert adds == {
+ ("Add(X: Integer)", True), # implemented -- has a body, multi-line
+ ("Add(X: Double)", False), # interface-only -- no body, single-line
+ }
+
+ def test_two_classes_sharing_a_method_name_stay_distinct(self, parser: ASTParser) -> None:
+ src = b"""\
+unit Foo;
+interface
+type
+ TFoo = class
+ function Add(X: Integer): Integer;
+ end;
+ TBar = class
+ function Add(X: Integer): Integer;
+ end;
+implementation
+function TFoo.Add(X: Integer): Integer;
+begin
+ Result := X;
+end;
+function TBar.Add(X: Integer): Integer;
+begin
+ Result := X * 2;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ adds = {(s.name, s.parent_name) for s in result.symbols if s.name == "Add"}
+ assert adds == {("Add", "TFoo"), ("Add", "TBar")}
+
+ def test_free_function_decl_and_impl_still_dedupe(self, parser: ASTParser) -> None:
+ # No class involved -- parent_name is None on both sides. Confirms
+ # the dedup key handles the no-parent case, not just methods.
+ src = b"""\
+unit Foo;
+interface
+procedure DoWork(X: Integer);
+implementation
+procedure DoWork(X: Integer);
+begin
+ Writeln(X);
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ matches = [s for s in result.symbols if s.name == "DoWork"]
+ assert len(matches) == 1
+ assert matches[0].end_line > matches[0].start_line
+
+ def test_dedup_ignores_multiline_parameter_reformatting(self, parser: ASTParser) -> None:
+ # Regression: scanned against a real ~150-file Delphi codebase, 168
+ # method pairs shared a class+name but escaped a raw-signature-text
+ # dedup because the implementation wraps a long parameter list
+ # across lines differently than the compact interface declaration
+ # -- extremely common real-world Delphi formatting, not a
+ # contrived edge case.
+ src = b"""\
+unit Foo;
+interface
+type
+ TFoo = class
+ procedure MoveSelCursor(ARowDelta, AColDelta: Integer; AExtendSel: Boolean);
+ end;
+implementation
+procedure TFoo.MoveSelCursor(ARowDelta, AColDelta: Integer;
+ AExtendSel: Boolean);
+begin
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ matches = [s for s in result.symbols if s.name == "MoveSelCursor"]
+ assert len(matches) == 1
+ assert matches[0].end_line > matches[0].start_line
+
+ def test_dedup_ignores_identifier_case(self, parser: ASTParser) -> None:
+ # Pascal is case-insensitive: TFoo.Add and TFOO.ADD name the same
+ # method. The dedup key must fold case on both the class name and
+ # the signature, or this reappears as two symbols.
+ src = b"""\
+unit Foo;
+interface
+type
+ TCalculator = class
+ function Add(X, Y: Integer): Integer;
+ end;
+implementation
+function TCALCULATOR.ADD(X, Y: Integer): Integer;
+begin
+ Result := X + Y;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ matches = [s for s in result.symbols if s.name.lower() == "add"]
+ assert len(matches) == 1
+ assert matches[0].end_line > matches[0].start_line
+
+ def test_anon_record_array_field_orphans_one_duplicate_but_keeps_the_real_method(
+ self, parser: ASTParser
+ ) -> None:
+ # `array[...] of record ... end` -- an anonymous record type used
+ # inline as an array element type -- has no grammar rule at all
+ # (unlike a *named* `TFoo = record ... end` declaration, which
+ # parses fine). Found on this repo's own uDualPanelWindow.pas: the
+ # class's declType closes early at the error, and every member
+ # declared after the field (Run included) gets reparented to the
+ # unit's interface section instead of the class.
+ #
+ # A regex-based sanitizer originally blanked the anonymous record
+ # before parsing to keep the class body intact. Dropped on review
+ # (PR #1353): an AST-driven replacement that blanks whatever ERROR
+ # nodes the grammar produces here isn't safe either -- on this
+ # exact input, one of the ERROR spans tree-sitter-pascal's error
+ # recovery reports is the class's own legitimate closing `end;`,
+ # and blanking it produces the identical broken structure. A
+ # correct fix needs a nesting-aware scanner for the record's own
+ # `end` (out of scope for one construct seen in one file); this
+ # documents the accepted degradation instead: Run appears twice,
+ # once correctly parented and once as an orphan, rather than
+ # asserting a fix that doesn't hold up.
+ src = b"""\
+unit Foo;
+interface
+type
+ TFoo = class
+ FTotals: array[TSide] of record
+ Valid: Boolean;
+ Bytes: Int64;
+ end;
+ procedure Run;
+ end;
+implementation
+procedure TFoo.Run;
+begin
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ matches = [s for s in result.symbols if s.name == "Run"]
+ assert len(matches) == 2
+ parented = [s for s in matches if s.parent_name == "TFoo"]
+ assert len(parented) == 1
+ assert parented[0].kind == "method"
+
+
+class TestPascalImports:
+ def test_multi_unit_uses_clause_extracts_every_unit(self, parser: ASTParser) -> None:
+ # Review feedback on PR #1353: `uses SysUtils, Classes;` was only
+ # extracting the first unit.
+ result = parser.parse_file(_pas(), UNIT_SOURCE)
+ modules = {i.module_path for i in result.imports}
+ assert modules == {"SysUtils", "Classes", "Ns.UnitC"}
+
+ def test_single_unit_uses_clause(self, parser: ASTParser) -> None:
+ src = b"""\
+unit Only;
+interface
+uses
+ SysUtils;
+implementation
+end.
+"""
+ result = parser.parse_file(_pas("Only.pas"), src)
+ assert [i.module_path for i in result.imports] == ["SysUtils"]
+
+ def test_separate_interface_and_implementation_uses_clauses(
+ self, parser: ASTParser
+ ) -> None:
+ # `uses` is valid in both sections of a unit, and the query isn't
+ # scoped to either -- two separate multi-unit clauses in one file
+ # must not interfere with each other's extraction.
+ src = b"""\
+unit Foo;
+interface
+uses SysUtils, Classes;
+implementation
+uses Windows, Messages;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ modules = {i.module_path for i in result.imports}
+ assert modules == {"SysUtils", "Classes", "Windows", "Messages"}
+
+ def test_unit_named_in_both_uses_clauses_dedupes_to_one_import(
+ self, parser: ASTParser
+ ) -> None:
+ # Review feedback on PR #1353: a unit named in both the interface
+ # and implementation `uses` clauses of the same file produced two
+ # identical Import entries -- the Pascal branch in
+ # `_extract_imports` returns before the `seen_raws` dedup because
+ # every moduleName match legitimately carries a different unit
+ # *within* one clause (see that branch's own comment), but two
+ # clauses naming the same unit is a real duplicate, not that case.
+ src = b"""\
+unit Foo;
+interface
+uses SysUtils, Classes;
+implementation
+uses SysUtils, Windows;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ modules = [i.module_path for i in result.imports]
+ assert sorted(modules) == ["Classes", "SysUtils", "Windows"]
+
+ def test_dpr_unit_in_path_clause(self, parser: ASTParser) -> None:
+ # Delphi/FPC project files map units to source paths right in
+ # `uses` -- `MyUnit in 'src\MyUnit.pas'` -- syntax the IDE
+ # generates automatically and tree-sitter-pascal's grammar has no
+ # rule for at all. Hitting it used to corrupt parsing of
+ # EVERYTHING after the first `in` clause (verified against this
+ # repo's own MTN2.dpr: 4 imports extracted instead of ~80, the
+ # 4th holding several KB of raw garbage as its module_path).
+ # _sanitize_pascal_project_source blanks these before parsing.
+ src = b"""\
+program Foo;
+uses
+ SysUtils,
+ MyUnit in 'src\\MyUnit.pas',
+ OtherUnit in 'src\\OtherUnit.pas';
+begin
+end.
+"""
+ result = parser.parse_file(_pas("Foo.dpr"), src)
+ modules = [i.module_path for i in result.imports]
+ assert modules == ["SysUtils", "MyUnit", "OtherUnit"]
+ assert all(len(m) < 20 for m in modules)
+
+ def test_dpr_ifdef_block_inside_uses(self, parser: ASTParser) -> None:
+ # `{$IFDEF}`/`{$ENDIF}` compiler directives are legal inside a
+ # `uses` clause (also present in MTN2.dpr) -- confirm they don't
+ # need special handling; Pascal treats `{$...}` as a directive
+ # comment the grammar already skips.
+ src = b"""\
+program Foo;
+uses
+ {$IFDEF SOMEFLAG}
+ FlagUnit,
+ {$ENDIF}
+ SysUtils;
+begin
+end.
+"""
+ result = parser.parse_file(_pas("Foo.dpr"), src)
+ modules = {i.module_path for i in result.imports}
+ assert modules == {"FlagUnit", "SysUtils"}
+
+ def test_in_clause_sanitization_does_not_touch_pas_files(
+ self, parser: ASTParser
+ ) -> None:
+ # Scoped to .dpr/.dpk/.lpr on purpose -- this syntax is invalid in
+ # a regular unit file, so the sanitizer must not run there. Uses a
+ # plain .pas path (not .dpr) with the same shape to prove it.
+ src = b"""\
+unit Foo;
+interface
+uses
+ SysUtils;
+implementation
+end.
+"""
+ result = parser.parse_file(_pas("Foo.pas"), src)
+ assert [i.module_path for i in result.imports] == ["SysUtils"]
+
+
+class TestPascalHeritage:
+ def test_extends_and_implements(self, parser: ASTParser) -> None:
+ result = parser.parse_file(_pas(), UNIT_SOURCE)
+ rels = {(r.child_name, r.kind, r.parent_name) for r in result.heritage}
+ # TObject is a builtin_parent and is filtered from the graph.
+ assert ("TCalculator", "implements", "IFoo") in rels
+ # IInterface is also a builtin_parent for Pascal.
+ assert not any(child == "ICalcTarget" for child, _kind, _parent in rels)
+
+
+class TestPascalEncoding:
+ """Non-ASCII (Cyrillic) content -- see PR #1353 review follow-up on
+ verifying repowise's Pascal support against a Russian-language codebase.
+ """
+
+ def test_cyrillic_comments_and_string_literals_parse_cleanly(
+ self, parser: ASTParser
+ ) -> None:
+ # Identifiers stay ASCII (matches this repo's actual Delphi
+ # convention); comments and string literals carry Cyrillic text.
+ # node_text() decodes via tree-sitter's own byte-accurate
+ # Node.text, so multi-byte UTF-8 content must round-trip intact.
+ src = """\
+unit Foo;
+interface
+uses
+ SysUtils;
+type
+ // Комментарий на русском про класс
+ TCalculator = class(TObject)
+ public
+ function Add(X, Y: Integer): Integer;
+ end;
+implementation
+function TCalculator.Add(X, Y: Integer): Integer;
+begin
+ Result := X + Y; // сложение чисел
+ WriteLn('Привет, мир');
+end;
+end.
+""".encode()
+ result = parser.parse_file(_pas(), src)
+ names = {s.name for s in result.symbols}
+ assert "TCalculator" in names
+ assert "Add" in names
+
+ def test_cyrillic_identifiers_are_a_known_upstream_gap(self, parser: ASTParser) -> None:
+ # tree-sitter-pascal's identifier token only accepts ASCII letters,
+ # so a Cyrillic-named class/method does NOT round-trip -- this locks
+ # in the current (broken) behavior as a documented gap rather than
+ # letting it regress silently. If tree-sitter-pascal ships a
+ # Unicode-aware identifier rule, this test should start failing and
+ # can be flipped to assert correct extraction instead.
+ src = """\
+unit Foo;
+interface
+type
+ TКалькулятор = class(TObject)
+ public
+ function Сложить(X, Y: Integer): Integer;
+ end;
+implementation
+end.
+""".encode()
+ result = parser.parse_file(_pas(), src)
+ names = {s.name for s in result.symbols}
+ # The lexer stops at the first non-ASCII byte, so neither the class
+ # nor the method name round-trips in full.
+ assert "TКалькулятор" not in names
+
+
+class TestPascalCalls:
+ """Parenless-call coverage.
+
+ Found while stress-testing PR #1353: a parameterless procedure/method
+ call idiomatically drops the `()` in Pascal (`Free;`, `inherited;`,
+ `if X then DoThing;`), so the grammar never wraps it in `exprCall` --
+ the original queries (all `exprCall`-anchored) silently missed every
+ one of these. Checked against real MTN2 source: roughly a third of
+ all call-shaped statements are parenless. Fixed by anchoring new
+ patterns on the bare `statement` wrapper directly.
+ """
+
+ def test_parenless_free_function_call(self, parser: ASTParser) -> None:
+ src = b"""\
+unit Foo;
+interface
+implementation
+procedure P;
+begin
+ Run;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ calls = {(c.target_name, c.receiver_name) for c in result.calls}
+ assert ("Run", None) in calls
+
+ def test_parenless_method_call(self, parser: ASTParser) -> None:
+ src = b"""\
+unit Foo;
+interface
+implementation
+procedure P;
+var
+ Obj: TFoo;
+begin
+ Obj.Free;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ calls = {(c.target_name, c.receiver_name) for c in result.calls}
+ assert ("Free", "Obj") in calls
+
+ def test_parenless_call_inside_control_flow_bodies(self, parser: ASTParser) -> None:
+ # `if/for/while/repeat/case` one-line bodies are exactly where
+ # parenless calls are most idiomatic in Pascal.
+ src = b"""\
+unit Foo;
+interface
+implementation
+procedure P;
+var
+ I: Integer;
+begin
+ if I > 0 then
+ DoThing;
+ for I := 0 to 9 do
+ Process(I);
+ while I > 0 do
+ Step;
+ case I of
+ 1: DoOne;
+ else
+ DoDefault;
+ end;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ names = {c.target_name for c in result.calls}
+ assert {"DoThing", "Process", "Step", "DoOne", "DoDefault"} <= names
+
+ def test_explicit_inherited_call(self, parser: ASTParser) -> None:
+ # `inherited Create;` / `inherited Create();` -- both forms.
+ # Bare `inherited;` (no explicit name) is a documented gap: the
+ # target name isn't in the node's own text, it needs the
+ # enclosing method's name.
+ src = b"""\
+unit Foo;
+interface
+implementation
+constructor TFoo.Create;
+begin
+ inherited Create;
+ inherited Create();
+ inherited;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ names = [c.target_name for c in result.calls]
+ assert names.count("Create") == 2
+
+ def test_assignment_and_goto_are_not_call_sites(self, parser: ASTParser) -> None:
+ # Regression guard for the new bare-identifier/bare-exprDot
+ # patterns: `assignment` is a sibling of `statement` in the
+ # grammar (not nested in it) and `goto`/`label` are their own
+ # node types, so neither should ever surface as a false-positive
+ # call.
+ src = b"""\
+unit Foo;
+interface
+implementation
+procedure P;
+label
+ Skip;
+var
+ Result: Integer;
+begin
+ Result := 1;
+ goto Skip;
+ Skip:
+ Real;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ names = {c.target_name for c in result.calls}
+ assert "Result" not in names
+ assert "Skip" not in names
+ assert "Real" in names
+
+ def test_raise_reraise_is_not_a_call_site(self, parser: ASTParser) -> None:
+ src = b"""\
+unit Foo;
+interface
+implementation
+procedure P;
+begin
+ try
+ DoWork;
+ except
+ raise;
+ end;
+end;
+end.
+"""
+ result = parser.parse_file(_pas(), src)
+ names = {c.target_name for c in result.calls}
+ assert "DoWork" in names
+ # A bare re-raise has no name to capture -- nothing should be
+ # emitted for it (and nothing should crash trying).
+ assert None not in names
+ assert "Сложить" not in names