Skip to content

Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.

feat(cobol): resolve COPY and EXEC SQL INCLUDE members to copybooks (#4435)

Raghav Chamadiya954411857c49
diff --git a/docs/architecture/language-support.md b/docs/architecture/language-support.md
index 41610f774..2cbc873bd 100644
--- a/docs/architecture/language-support.md
+++ b/docs/architecture/language-support.md
@@ -728,7 +728,7 @@ dependency surface their syntax supports.
| Good | **PHP** | `.php` | `use` declarations through PSR-4 from every `composer.json`, same-namespace and fully qualified class references |
| Good | **Dart** | `.dart` | `import` / `export` / `part`, `package:` via every `pubspec.yaml` |
| Good | **Object Pascal** | `.pas` `.pp` `.dpr` `.dpk` `.lpr` | `uses` clauses via unit-name to file-stem match; project files as entry points |
-| Good | **COBOL** | `.cbl` `.cob` `.cobol` `.cpy` | Literal `CALL` and `PERFORM` targets; dynamic calls and `COPY` stay silent |
+| Good | **COBOL** | `.cbl` `.cob` `.cobol` `.cpy` | `COPY` members to copybooks; literal `CALL` and `PERFORM` targets; dynamic calls stay silent |
| Good | **GDScript** | `.gd` | `preload` / `load` / `extends "res://..."` from the nearest `project.godot` |
| Good | **VB.NET** | `.vb` | `Imports` through the same MSBuild project index C# uses |
| Good | **Elixir** | `.ex` `.exs` | `alias` / `import` / `require` / `use` against a `defmodule` index, Mix path convention as fallback |
@@ -1067,8 +1067,8 @@ point. A bundle two sfdx projects declare goes to the nearer one; a tie gets no
both, since props are set by the parent as markup attributes.
- **Object Pascal type kinds collapse** to `kind="class"`, and its `extends`
vs `implements` split follows the `I`-prefix naming convention.
-- **COBOL is COBOL-85 shaped.** `COPY` does not create an edge; dynamic program
- names and dialect extensions stay silent.
+- **COBOL dynamic program names stay silent**, and free-format source is read
+ without column handling.
- **Razor has no import edges**, because every directive is blanked by the
projection.
- **GDScript string dispatch is unresolved**, and a script without `class_name`
diff --git a/docs/layers/LANGUAGE_SUPPORT.md b/docs/layers/LANGUAGE_SUPPORT.md
index b011e3708..c3cf6c62c 100644
--- a/docs/layers/LANGUAGE_SUPPORT.md
+++ b/docs/layers/LANGUAGE_SUPPORT.md
@@ -68,7 +68,7 @@ listed per language and framework in
Every Full and Good language has a dedicated import resolver except Object
Pascal and Objective-C, which resolve through a generic unit-name or header-stem
-match, and COBOL, which resolves literal program calls instead of imports.
+match.
Scala, Elixir and F# have dedicated import resolvers with known gaps; see
[Limits](#limits).
@@ -79,7 +79,8 @@ The COBOL and Apex grammars download on first use; offline, set
whether it is cached.
COBOL is read in fixed reference format, with `EXEC SQL/CICS/DLI` blocks and
`COPY` statements masked before parsing; a copybook parses as a data or
-procedure fragment.
+procedure fragment. `COPY` and `EXEC SQL INCLUDE` members resolve to the
+nearest copybook with that name; system copybooks such as `DFHAID` stay external.
Apex (`.cls`, `.trigger`, `.apex`) is parsed for symbols, heritage and calls,
resolved case-insensitively within each sfdx project's namespace; its dead-code
findings are review-only. Apex test coverage from the Salesforce CLI is read as a
@@ -275,8 +276,7 @@ Per-marker detail: [CODE_HEALTH.md](CODE_HEALTH.md).
- Razor has no import edges.
- Scala import resolution is partial, and the Elixir and F# import resolvers
have known gaps.
-- COBOL support is COBOL-85 shaped: `COPY`, dynamic program names and dialect
- extensions produce no edges.
+- COBOL dynamic program names and free-format source produce no edges.
- C has no performance risk support.
The full list per language, with the mechanics behind each gap, is in
@@ -291,7 +291,7 @@ The full list per language, with the mechanics behind each gap, is in
| Scala, Ruby | Extract Method (dataflow dialect) |
| GDScript | Health markers (complexity, performance, dataflow) |
| Object Pascal | A dedicated `uses` resolver |
-| COBOL | Copybook resolution, dialect coverage, health markers |
+| COBOL | Dialect coverage, health markers |
| VB.NET, Elixir | Health markers |
| F#, Objective-C | Health markers beyond complexity |
| SQL / dbt | Column-level blast radius |
diff --git a/packages/core/src/repowise/core/ingestion/cobol/copybooks.py b/packages/core/src/repowise/core/ingestion/cobol/copybooks.py
new file mode 100644
index 000000000..82c7eec89
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/cobol/copybooks.py
@@ -0,0 +1,34 @@
+"""``COPY`` and ``EXEC SQL INCLUDE`` statements as graph imports."""
+
+from __future__ import annotations
+
+from ..models import Import
+from .source import code_view
+from .statements import copy_statements
+
+
+def copy_imports(source: bytes) -> list[Import]:
+ """One import per copied member, read from the unmasked source."""
+ imports: list[Import] = []
+ seen: set[str] = set()
+ for statement in copy_statements(code_view(source)):
+ # ``LIB/MEMBER`` lets the resolver prefer a library directory.
+ module = (
+ f"{statement.library}/{statement.member}" if statement.library else statement.member
+ )
+ if module in seen:
+ continue
+ seen.add(module)
+ verb = "EXEC SQL INCLUDE" if statement.include else "COPY"
+ library = f" OF {statement.library}" if statement.library else ""
+ imports.append(
+ Import(
+ raw_statement=f"{verb} {statement.member}{library}",
+ module_path=module,
+ # A copybook is textually included: every name in it is in scope.
+ imported_names=["*"],
+ is_relative=False,
+ resolved_file=None,
+ )
+ )
+ return imports
diff --git a/packages/core/src/repowise/core/ingestion/language_configs.py b/packages/core/src/repowise/core/ingestion/language_configs.py
index a02b02b64..3e1a66e20 100644
--- a/packages/core/src/repowise/core/ingestion/language_configs.py
+++ b/packages/core/src/repowise/core/ingestion/language_configs.py
@@ -674,7 +674,7 @@ LANGUAGE_CONFIGS: dict[str, LanguageConfig] = {
"paragraph_header": "function",
"data_description": "variable",
},
- # COPY/include resolution is intentionally outside this good-tier cut.
+ # COPY is masked before parsing; parse_file reads it from the source.
import_node_types=[],
export_node_types=[],
visibility_fn=public_by_default,
diff --git a/packages/core/src/repowise/core/ingestion/languages/specs/cobol.py b/packages/core/src/repowise/core/ingestion/languages/specs/cobol.py
index 83e809719..881601064 100644
--- a/packages/core/src/repowise/core/ingestion/languages/specs/cobol.py
+++ b/packages/core/src/repowise/core/ingestion/languages/specs/cobol.py
@@ -16,7 +16,7 @@ SPEC = LanguageSpec(
# JCL, schedulers and dynamic CALL data-items invoke programs outside the
# source graph. Without those inputs, unused/unreachable claims are unsound.
dead_code_exempt=True,
- import_support="none",
+ import_support="full",
entry_point_patterns=(),
manifest_files=(),
shebang_tokens=(),
diff --git a/packages/core/src/repowise/core/ingestion/parser/__init__.py b/packages/core/src/repowise/core/ingestion/parser/__init__.py
index e6a8a611e..248c26a56 100644
--- a/packages/core/src/repowise/core/ingestion/parser/__init__.py
+++ b/packages/core/src/repowise/core/ingestion/parser/__init__.py
@@ -167,12 +167,14 @@ class ASTParser:
or _is_fsharp_signature(file_info)
):
return self.parse_file(file_info, source).imports
+ original_source = source
source = prepare_source(lang, source, path=file_info.path)
root, _, grammar_tag, language = _parse_tree(lang, language, grammar_tag, source)
query = _load_compiled_query(lang, grammar_tag)
matches = _run_query(query, root) if query is not None else []
src = source.decode("utf-8", errors="replace")
- return _extract_imports(matches, config, file_info, src)
+ imports = _extract_imports(matches, config, file_info, src)
+ return imports + _masked_imports(lang, original_source)
def parse_file(self, file_info: FileInfo, source: bytes) -> ParsedFile:
"""Parse *source* bytes and return a fully populated ParsedFile."""
@@ -221,6 +223,7 @@ class ASTParser:
)
_restore_kotlin_annotations(symbols, recovered_annotations)
imports = _extract_imports(matches, config, file_info, src)
+ imports.extend(_masked_imports(lang, original_source))
calls = _extract_calls(matches, config, file_info, src, symbols)
# An SFC's markup tags (``<Foo />``) are component calls the TS buffer lost.
calls.extend(component_call_sites(lang, original_source, symbols))
@@ -293,6 +296,15 @@ def _parse_without_grammar(
)
+def _masked_imports(lang: str, original_source: bytes) -> list[Import]:
+ """Imports ``prepare_source`` masked from the grammar, read from the original bytes."""
+ if lang == "cobol":
+ from ..cobol.copybooks import copy_imports
+
+ return copy_imports(original_source)
+ return []
+
+
def _warn_symbol_bloat(file_info: FileInfo, symbols: list[Symbol]) -> None:
if len(symbols) > _SYMBOL_COUNT_WARN_THRESHOLD:
log.warning(
diff --git a/packages/core/src/repowise/core/ingestion/resolvers/__init__.py b/packages/core/src/repowise/core/ingestion/resolvers/__init__.py
index a580e99e3..4fc36ca9e 100644
--- a/packages/core/src/repowise/core/ingestion/resolvers/__init__.py
+++ b/packages/core/src/repowise/core/ingestion/resolvers/__init__.py
@@ -5,6 +5,7 @@ from __future__ import annotations
from collections.abc import Callable
from .clojure import resolve_clojure_import
+from .cobol import resolve_cobol_import
from .context import ResolverContext
from .cpp import resolve_cpp_import
from .csharp import resolve_csharp_import
@@ -81,6 +82,8 @@ _RESOLVERS: dict[str, ResolverFn] = {
"html": resolve_html_asset,
# VB.NET Imports resolve through the same DotNetProjectIndex as C#.
"vbnet": resolve_vbnet_import,
+ # COPY / EXEC SQL INCLUDE members, matched to copybooks by stem.
+ "cobol": resolve_cobol_import,
}
diff --git a/packages/core/src/repowise/core/ingestion/resolvers/cobol.py b/packages/core/src/repowise/core/ingestion/resolvers/cobol.py
new file mode 100644
index 000000000..3cd1919ab
--- /dev/null
+++ b/packages/core/src/repowise/core/ingestion/resolvers/cobol.py
@@ -0,0 +1,47 @@
+"""COBOL ``COPY`` member resolution against copybook files in the repo.
+
+A member resolves case-insensitively to a COBOL file with that stem, a
+copybook extension first, then one under the named library, then the one
+nearest the copying file. An unresolved member (``DFHAID``, a vendor or
+system copybook) becomes an external node, never a guess.
+"""
+
+from __future__ import annotations
+
+import posixpath
+from typing import TYPE_CHECKING
+
+from ..languages.registry import REGISTRY
+
+if TYPE_CHECKING:
+ from .context import ResolverContext
+
+_COPYBOOK_SUFFIXES = frozenset({".cpy"})
+
+
+def resolve_cobol_import(module_path: str, importer_path: str, ctx: ResolverContext) -> str | None:
+ library, _, member = module_path.rpartition("/")
+ spec = REGISTRY.get("cobol")
+ extensions = spec.extensions if spec else _COPYBOOK_SUFFIXES
+ candidates = [
+ path
+ for path in ctx.stem_map.get(member.lower(), ())
+ if posixpath.splitext(path)[1].lower() in extensions and path != importer_path
+ ]
+ if not candidates:
+ return ctx.add_external_node(f"copybook:{member}")
+ importer_dir = posixpath.dirname(importer_path).split("/")
+ order = {path: index for index, path in enumerate(candidates)}
+
+ def rank(path: str) -> tuple[int, int, int, int]:
+ parts = path.split("/")
+ shared = 0
+ for a, b in zip(parts[:-1], importer_dir, strict=False):
+ if a != b:
+ break
+ shared += 1
+ in_library = bool(library) and library.lower() in (p.lower() for p in parts[:-1])
+ is_copybook = posixpath.splitext(path)[1].lower() in _COPYBOOK_SUFFIXES
+ return (not is_copybook, not in_library, -shared, order[path])
+
+ return min(candidates, key=rank)
diff --git a/tests/unit/ingestion/test_cobol_resolver.py b/tests/unit/ingestion/test_cobol_resolver.py
new file mode 100644
index 000000000..73f24d453
--- /dev/null
+++ b/tests/unit/ingestion/test_cobol_resolver.py
@@ -0,0 +1,69 @@
+"""COBOL ``COPY`` members resolve to copybooks by stem, nearest first, never by guess."""
+
+from __future__ import annotations
+
+import networkx as nx
+
+from repowise.core.ingestion.cobol.copybooks import copy_imports
+from repowise.core.ingestion.graph import build_stem_map
+from repowise.core.ingestion.resolvers.cobol import resolve_cobol_import
+from repowise.core.ingestion.resolvers.context import ResolverContext
+
+
+def _ctx(*paths: str) -> ResolverContext:
+ path_set = set(paths)
+ return ResolverContext(path_set=path_set, stem_map=build_stem_map(path_set), graph=nx.DiGraph())
+
+
+def test_a_member_resolves_case_insensitively_to_its_copybook() -> None:
+ ctx = _ctx("app/cbl/COSGN00C.cbl", "app/cpy/cocom01y.cpy")
+
+ assert resolve_cobol_import("COCOM01Y", "app/cbl/COSGN00C.cbl", ctx) == "app/cpy/cocom01y.cpy"
+
+
+def test_a_copybook_beats_a_bms_map_or_program_with_the_same_stem() -> None:
+ ctx = _ctx("app/bms/COSGN00.bms", "app/cpy-bms/COSGN00.CPY", "app/cbl/COSGN00.cbl", "a/X.cbl")
+
+ assert resolve_cobol_import("COSGN00", "a/X.cbl", ctx) == "app/cpy-bms/COSGN00.CPY"
+
+
+def test_the_copybook_nearest_the_copying_program_wins() -> None:
+ ctx = _ctx("app/cpy/CVACT01Y.cpy", "app/sub/cpy/CVACT01Y.cpy", "app/sub/cbl/P.cbl")
+
+ assert resolve_cobol_import("CVACT01Y", "app/sub/cbl/P.cbl", ctx) == "app/sub/cpy/CVACT01Y.cpy"
+
+
+def test_a_named_library_directory_is_preferred() -> None:
+ ctx = _ctx("lib1/MEMBER.cpy", "LIB2/MEMBER.cpy", "src/P.cbl")
+
+ assert resolve_cobol_import("LIB2/MEMBER", "src/P.cbl", ctx) == "LIB2/MEMBER.cpy"
+
+
+def test_a_system_copybook_is_external_not_guessed() -> None:
+ ctx = _ctx("app/cbl/P.cbl", "app/docs/DFHAID.md")
+
+ assert resolve_cobol_import("DFHAID", "app/cbl/P.cbl", ctx) == "external:copybook:DFHAID"
+ assert "external:copybook:DFHAID" in ctx.graph
+
+
+def test_copy_and_sql_include_become_one_import_per_member() -> None:
+ source = (
+ b" IDENTIFICATION DIVISION.\n"
+ b" PROGRAM-ID. P.\n"
+ b" DATA DIVISION.\n"
+ b" WORKING-STORAGE SECTION.\n"
+ b" COPY COCOM01Y.\n"
+ b" COPY COCOM01Y.\n"
+ b" COPY MEMBER1 OF MYLIB.\n"
+ b" COPY MEMBER1 OF OTHERLIB.\n"
+ b" EXEC SQL INCLUDE DCLTRTYP END-EXEC.\n"
+ )
+
+ found = [(i.module_path, i.raw_statement, i.imported_names) for i in copy_imports(source)]
+
+ assert found == [
+ ("COCOM01Y", "COPY COCOM01Y", ["*"]),
+ ("MYLIB/MEMBER1", "COPY MEMBER1 OF MYLIB", ["*"]),
+ ("OTHERLIB/MEMBER1", "COPY MEMBER1 OF OTHERLIB", ["*"]),
+ ("DCLTRTYP", "EXEC SQL INCLUDE DCLTRTYP", ["*"]),
+ ]
diff --git a/tests/unit/ingestion/test_language_capabilities.py b/tests/unit/ingestion/test_language_capabilities.py
index eb0fe0dd6..cad68cb07 100644
--- a/tests/unit/ingestion/test_language_capabilities.py
+++ b/tests/unit/ingestion/test_language_capabilities.py
@@ -161,6 +161,8 @@ _FULL = {
# through the same SFC pass as svelte, so its imports are ordinary ESM.
"astro",
"c",
+ # COPY members resolve to copybooks by stem, nearest first.
+ "cobol",
"cpp",
"csharp",
# dart promoted with the tree-sitter grammar (the regex tier stays as