Turbocharge Claude Code, Cursor, Codex, Gemini & every coding agent: faster, cheaper, with contextual understanding specific to your codebase.
TypeScript ai-agentsanthropicclaude-codeclicode-graphcodexcontext-engineeringcursordeveloper-toolsgeminiknowledge-graphllmmcpmcp-serveropen-sourceopenaitree-sitter
fix: index initialized git submodules
yescineded2783eb292
diff --git a/README.md b/README.mdindex eff821fa..60f4fa1e 100644--- a/README.md+++ b/README.md@@ -215,7 +215,7 @@ flowchart LREvery pass is cached by content hash — the LLM ones and the tree-sitter parse alike. Re-running only touches the files that changed, so the second build is fast and cheap (on this repo, 124 files: 0.74s cold, 0.18s after one edited file, 0.18s with nothing changed). `graft build --no-reuse` forces a cold re-parse.-That cheapness is what lets **every query refresh the graph before it answers**. A retrieval call stats the tree against the last build's fingerprint (~3ms), and rebuilds only if something moved — so `ask`/`grep`/`callers`/`skeleton`/`map` describe the code as it is right now, including edits that are unsaved to git: uncommitted, unstaged, or staged all look the same to graft, which never reads git at all. The refresh is structural and `$0`; it never calls the LLM. Turn it off per-command with `--no-refresh`, or everywhere with `GRAFT_NO_REFRESH=1`.+That cheapness is what lets **every query refresh the graph before it answers**. A retrieval call stats the tree against the last build's fingerprint (~3ms), and rebuilds only if something moved — so `ask`/`grep`/`callers`/`skeleton`/`map` describe the code as it is right now, including edits that are unsaved to git: uncommitted, unstaged, or staged all look the same to graft. Git determines the visible file set; freshness compares the working-tree bytes rather than commit or index state. The refresh is structural and `$0`; it never calls the LLM. Turn it off per-command with `--no-refresh`, or everywhere with `GRAFT_NO_REFRESH=1`.Alongside the markdown graph, `graft build` builds `graft/.graph/wiring.json` — a per-symbol code graph — plus a per-file wiring card mirroring your source tree. Tier 1 is pure tree-sitter (every function, class, and call edge; deterministic, no model, no network), which is why plain `graft build` needs no key. The `--deep` pass adds a one-line summary and a crux excerpt per symbol, cached by body hash.@@ -419,9 +419,9 @@ scripts/ 2 files · 0 symbolshotspots: contextDirFor · function · src/context/node-file.ts:L100-L103 · 21← wiringPath · function · src/graph/write.ts:L20-L22 · 14← buildGraph · function · src/graph/build.ts:L104-L218 · 11← ...```-## Monorepos & multi-repo folders+## Monorepos, submodules & multi-repo folders-Graft handles two shapes without any config:+Graft handles these layouts without any config:- **A monorepo with one `.git`** (a `pnpm-workspace.yaml`/`package.json``workspaces`, or per-package `go.mod`/`pyproject.toml`/`Cargo.toml`) —@@ -429,13 +429,18 @@ Graft handles two shapes without any config:rank every scope on its own terms and fuse the results, so the biggestsub-project can't drown a small one; hits carry `[scope/]` labels, and`graft map` groups its directory clusters by scope first.+- **A Git superproject with initialized submodules** — `graft build` follows+ initialized gitlinks into one graph, prefixing child paths (for example,+ `deps/parser/src/index.ts`) while honoring each submodule's own Git ignore+ rules. Visible untracked files are included too; uninitialized submodules+ remain absent until `git submodule update --init` checks them out.- **A folder of separate git repos** (no `.git` at the top) — `graft build`auto-splits: each child gets its own (git-ignored) `graft/`, and the parentgets a `graft/workspace.json` index. Queries from the parent federate acrossevery child, always labeled `<child>/`. Run `graft build` inside a child towork on just that repo.-Either way, narrow to one sub-project with `graft ask "<task>" --in <scope>/`+In every layout, narrow to one sub-project with `graft ask "<task>" --in <scope>/`once you know where you're working.## Visualize it (`graft viz`)diff --git a/src/ingest/fs.ts b/src/ingest/fs.tsindex 3d072e2c..040b1f83 100644--- a/src/ingest/fs.ts+++ b/src/ingest/fs.ts@@ -2,8 +2,8 @@* Filesystem walking used by `init`/`check` to enumerate a repo's source files.*/import { spawnSync } from "node:child_process";-import { lstatSync, readdirSync, statSync } from "node:fs";-import { join, resolve } from "node:path";+import { existsSync, lstatSync, readdirSync, statSync } from "node:fs";+import { join, relative, resolve } from "node:path";/** Directories that are dependency/build output, never source. */export const SKIP_DIRS = new Set([@@ -53,7 +53,10 @@ export function shouldSkipDir(name: string, includes?: ReadonlySet<string>): boo* `includes`, and files over 1 MB.* In a Git worktree, tracked files plus untracked, non-ignored files come from* `git ls-files`; this gives indexing exactly Git's nested `.gitignore`,- * negation, and global-exclude semantics. Non-Git directories retain the plain+ * negation, and global-exclude semantics. Initialized submodules are enumerated+ * recursively as repositories of their own, so their tracked and visible+ * untracked files share one graph while keeping each child's ignore rules.+ * Uninitialized submodules remain absent. Non-Git directories retain the plain* filesystem walk. `includes` lifts only the built-in skip list — it never* overrides Git's ignore rules (un-ignore or `git add -f` a directory to* index it, the same contract as tracked-but-ignored files).@@ -65,11 +68,23 @@ export function walkDir(dir: string, includes?: ReadonlySet<string>): string[] {/** Git's canonical working-tree file set, relative to `dir`. Tracked files are* deliberately included even when a later ignore rule matches them; `.gitignore`* only controls untracked files in Git, and graft follows the same contract. */-function gitVisibleFiles(dir: string, includes?: ReadonlySet<string>): string[] | null {+function gitVisibleFiles(+ dir: string,+ includes?: ReadonlySet<string>,+ traversal?: { topRoot: string; activeRoots: Set<string> },+): string[] | null {const root = resolve(dir);+ const state = traversal ?? { topRoot: root, activeRoots: new Set<string>() };+ const rootKey = process.platform === "win32" ? root.toLowerCase() : root;+ if (state.activeRoots.has(rootKey)) return [];+ state.activeRoots.add(rootKey);++ // `-t --stage` makes the cached/other output unambiguous in one process:+ // tracked entries are `H <mode> <oid> <stage>\t<path>`, while untracked+ // entries are `? <path>`. `-z` keeps either form safe for unusual paths.const result = spawnSync("git",- ["ls-files", "--cached", "--others", "--exclude-standard", "-z", "--"],+ ["ls-files", "-t", "--stage", "--cached", "--others", "--exclude-standard", "-z", "--"],{cwd: root,encoding: "utf8",@@ -77,12 +92,63 @@ function gitVisibleFiles(dir: string, includes?: ReadonlySet<string>): string[]maxBuffer: 64 * 1024 * 1024,},);- if (result.status !== 0 || result.error || typeof result.stdout !== "string") return null;+ if (result.status !== 0 || result.error || typeof result.stdout !== "string") {+ state.activeRoots.delete(rootKey);+ return null;+ }- const out: string[] = [];- for (const rel of result.stdout.split("\0")) {- if (!rel || skippedPath(rel, includes)) continue;+ // A path can have multiple stage records during a merge. Collapse those+ // records and remember if any of them identifies the path as a gitlink.+ const entries = new Map<string, { gitlink: boolean }>();+ for (const record of result.stdout.split("\0")) {+ if (!record || record.length < 2 || record[1] !== " ") continue;++ const tag = record[0];+ const body = record.slice(2);+ let rel: string;+ let gitlink = false;+ if (tag === "?") {+ rel = body;+ } else {+ const tab = body.indexOf("\t");+ if (tab === -1) continue;+ gitlink = body.startsWith("160000 ");+ rel = body.slice(tab + 1);+ }+ if (!rel) continue;+ const prior = entries.get(rel);+ entries.set(rel, { gitlink: gitlink || prior?.gitlink === true });+ }++ const out = new Set<string>();+ for (const [rel, entry] of entries) {const abs = resolve(root, rel);+ // Filter against the original superproject path. Otherwise a submodule+ // mounted at vendor/ or build/ would bypass the parent's skip policy when+ // recursion resets its relative root.+ if (skippedPath(relative(state.topRoot, abs), includes)) continue;++ if (entry.gitlink) {+ // A deinitialized gitlink can resolve upward to the superproject when Git+ // runs inside it. Requiring the child's own .git prevents duplicate,+ // mis-prefixed parent files and recursive loops.+ if (!existsSync(join(abs, ".git"))) continue;+ let childFiles = gitVisibleFiles(abs, includes, state);+ if (childFiles === null) {+ // Do not let one broken child recreate a "healthy but incomplete"+ // graph. Match the top-level fail-soft contract locally: fall back to+ // the filesystem for this child only, preserving the parent repo's Git+ // visibility and every built-in skip/size guard. Child Git ignore rules+ // are unavailable in this exceptional path, just as they are when the+ // top-level Git command fails and walkDir uses its filesystem fallback.+ // Let an unreadable filesystem fallback surface rather than claiming a+ // healthy graph that silently omitted the child again.+ childFiles = walkFilesystem(abs, includes);+ }+ for (const file of childFiles) out.add(file);+ continue;+ }+try {const stat = lstatSync(abs);if (!stat.isFile() || stat.size > MAX_FILE_BYTES) continue;@@ -91,9 +157,11 @@ function gitVisibleFiles(dir: string, includes?: ReadonlySet<string>): string[]// `--cached`; absence means it is not part of the current source set.continue;}- out.push(abs);+ out.add(abs);}- return out;++ state.activeRoots.delete(rootKey);+ return [...out].sort();}/** A path (git-relative, either separator) is skipped when any of itsdiff --git a/test/graph-incremental.test.ts b/test/graph-incremental.test.tsindex 1b63285e..b00a4269 100644--- a/test/graph-incremental.test.ts+++ b/test/graph-incremental.test.ts@@ -38,6 +38,19 @@ function repo(): string {return d;}+function gitRun(root: string, args: string[]): void {+ execFileSync("git", args, { cwd: root, stdio: "ignore" });+}++function commitRepo(root: string, message: string): void {+ gitRun(root, ["add", "-A"]);+ gitRun(root, [+ "-c", "user.name=Graft Tests",+ "-c", "[email protected]",+ "commit", "-qm", message,+ ]);+}+const outOf = (d: string): string => join(d, "graft");const wiringOf = (d: string): string => readFileSync(wiringPath(outOf(d)), "utf8");@@ -179,6 +192,66 @@ test("every file on disk lands in the fingerprint", async () => {assert.deepEqual(Object.keys(fp!.files).sort(), ["src/app.ts", "src/math.ts"]);});+test("initialized submodule files share the graph and freshness fingerprint (#74)", async () => {+ const parent = repo();+ const child = mkdtempSync(join(tmpdir(), "graft-incr-submodule-"));+ try {+ execFileSync("git", ["init", "-q"], { cwd: parent });+ commitRepo(parent, "parent fixture");++ execFileSync("git", ["init", "-q"], { cwd: child });+ mkdirSync(join(child, "src"), { recursive: true });+ writeFileSync(join(child, ".gitignore"), "*.generated.ts\n");+ writeFileSync(join(child, "src", "sub.ts"), "export function subValue(): number { return 1; }\n");+ commitRepo(child, "child fixture");++ gitRun(parent, [+ "-c", "protocol.file.allow=always",+ "submodule", "add", "-q", child.replace(/\\/g, "/"), "modules/child",+ ]);+ commitRepo(parent, "add child submodule");++ const checkout = join(parent, "modules", "child", "src");+ writeFileSync(join(checkout, "visible.ts"), "export const visible = 1;\n");+ writeFileSync(join(checkout, "ignored.generated.ts"), "export const ignored = 1;\n");++ const first = await buildGraph(parent);+ assert.equal(first.files, 4);+ const graph = readGraph(wiringPath(outOf(parent))) as GraphV1;+ assert.ok(graph.nodes.some((n) => n.id === "modules/child/src/sub.ts#subValue"));+ assert.ok(graph.nodes.some((n) => n.path === "modules/child/src/visible.ts"));+ assert.ok(!graph.nodes.some((n) => n.path.endsWith("ignored.generated.ts")));+ assert.deepEqual(Object.keys(readFingerprint(outOf(parent))!.files).sort(), [+ "modules/child/src/sub.ts",+ "modules/child/src/visible.ts",+ "src/app.ts",+ "src/math.ts",+ ]);+ assert.ok(isClean(probeDrift(parent, outOf(parent))!));++ writeFileSync(join(checkout, "sub.ts"), "export function changedSubValue(): number { return 2; }\n");+ writeFileSync(join(checkout, "added.ts"), "export const added = 1;\n");+ rmSync(join(checkout, "visible.ts"));+ writeFileSync(join(checkout, "another.generated.ts"), "export const stillIgnored = 1;\n");++ assert.deepEqual(probeDrift(parent, outOf(parent)), {+ changed: ["modules/child/src/sub.ts"],+ added: ["modules/child/src/added.ts"],+ removed: ["modules/child/src/visible.ts"],+ });++ await buildGraph(parent);+ const refreshed = readGraph(wiringPath(outOf(parent))) as GraphV1;+ assert.ok(refreshed.nodes.some((n) => n.id === "modules/child/src/sub.ts#changedSubValue"));+ assert.ok(refreshed.nodes.some((n) => n.path === "modules/child/src/added.ts"));+ assert.ok(!refreshed.nodes.some((n) => n.path === "modules/child/src/visible.ts"));+ assert.ok(isClean(probeDrift(parent, outOf(parent))!));+ } finally {+ rmSync(parent, { recursive: true, force: true });+ rmSync(child, { recursive: true, force: true });+ }+});+test("gitignored generated files stay out of the graph, fingerprint, and drift probe (#39)", async () => {const d = repo();try {diff --git a/test/ingest-fs.test.ts b/test/ingest-fs.test.tsindex 928e8c0b..f3514cf6 100644--- a/test/ingest-fs.test.ts+++ b/test/ingest-fs.test.ts@@ -1,7 +1,7 @@import { execFileSync } from "node:child_process";import { test } from "node:test";import assert from "node:assert/strict";-import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";+import { mkdtempSync, mkdirSync, renameSync, rmSync, writeFileSync } from "node:fs";import { tmpdir } from "node:os";import { join, relative } from "node:path";import { shouldSkipDir, walkDir, SKIP_DIRS } from "../src/ingest/fs.js";@@ -18,6 +18,27 @@ function write(root: string, path: string, content = "export const value = 1;\n"writeFileSync(join(root, path), content);}+function runGit(root: string, args: string[]): void {+ execFileSync("git", args, { cwd: root, stdio: "ignore" });+}++function commitAll(root: string, message: string, forcePaths: string[] = []): void {+ runGit(root, ["add", "-A"]);+ if (forcePaths.length > 0) runGit(root, ["add", "-f", "--", ...forcePaths]);+ runGit(root, [+ "-c", "user.name=Graft Tests",+ "-c", "[email protected]",+ "commit", "-qm", message,+ ]);+}++function addLocalSubmodule(parent: string, source: string, path: string): void {+ runGit(parent, [+ "-c", "protocol.file.allow=always",+ "submodule", "add", "-q", source.replace(/\\/g, "/"), path,+ ]);+}+function walked(root: string, includes?: ReadonlySet<string>): string[] {return walkDir(root, includes).map((path) => relative(root, path).replace(/\\/g, "/"))@@ -61,6 +82,78 @@ test("walkDir keeps tracked files that match an ignore rule and untracked visibl}});+test("walkDir recursively includes initialized submodules with their own Git visibility (#74)", () => {+ const parent = fixture("submodule-parent");+ const child = fixture("submodule-child");+ const nested = fixture("submodule-nested");+ try {+ write(nested, ".gitignore", "*.generated.ts\n");+ write(nested, "src/nested.ts");+ commitAll(nested, "nested fixture");++ write(child, ".gitignore", "*.generated.ts\n");+ write(child, "src/tracked.ts");+ write(child, "src/forced.generated.ts");+ write(child, "build/handwritten.ts");+ commitAll(child, "child fixture", ["src/forced.generated.ts"]);+ addLocalSubmodule(child, nested, "components/nested module");+ commitAll(child, "add nested submodule");++ write(parent, ".gitignore", "deps/child module/src/untracked.ts\n");+ write(parent, "src/root.ts");+ commitAll(parent, "parent fixture");+ addLocalSubmodule(parent, child, "deps/child module");+ addLocalSubmodule(parent, child, "vendor/skipped child");+ commitAll(parent, "add child submodules");+ runGit(parent, [+ "-c", "protocol.file.allow=always",+ "submodule", "update", "--init", "--recursive", "--", "deps/child module",+ ]);++ const childCheckout = join(parent, "deps", "child module");+ const nestedCheckout = join(childCheckout, "components", "nested module");+ write(childCheckout, "src/untracked.ts");+ write(childCheckout, "src/untracked.generated.ts");+ write(nestedCheckout, "src/nested-untracked.ts");+ write(nestedCheckout, "src/nested-untracked.generated.ts");++ assert.deepEqual(walked(parent), [+ "deps/child module/components/nested module/src/nested-untracked.ts",+ "deps/child module/components/nested module/src/nested.ts",+ "deps/child module/src/forced.generated.ts",+ "deps/child module/src/tracked.ts",+ "deps/child module/src/untracked.ts",+ "src/root.ts",+ ]);++ const withVendor = walked(parent, new Set(["vendor"]));+ assert.ok(withVendor.includes("vendor/skipped child/src/tracked.ts"), "an initialized gitlink is included when its mount's built-in skip is lifted");+ assert.ok(!withVendor.includes("vendor/skipped child/build/handwritten.ts"), "the child's own build/ directory remains skipped");+ assert.ok(!withVendor.some((path) => path.endsWith("untracked.generated.ts")), "--include-dir never overrides a child's Git ignore rules");++ const withVendorAndBuild = walked(parent, new Set(["vendor", "build"]));+ assert.ok(withVendorAndBuild.includes("vendor/skipped child/build/handwritten.ts"));++ const childGitFile = join(childCheckout, ".git");+ const childGitBackup = join(childCheckout, "git-pointer.backup");+ renameSync(childGitFile, childGitBackup);+ writeFileSync(childGitFile, "gitdir: missing-gitdir\n");+ assert.ok(+ walked(parent).includes("deps/child module/src/tracked.ts"),+ "a child Git failure falls back locally instead of producing a healthy but incomplete graph",+ );+ rmSync(childGitFile, { force: true });+ renameSync(childGitBackup, childGitFile);++ runGit(parent, ["submodule", "deinit", "-f", "--all"]);+ assert.deepEqual(walked(parent), ["src/root.ts"], "deinitialized gitlinks must not resolve upward and duplicate the parent");+ } finally {+ rmSync(parent, { recursive: true, force: true });+ rmSync(child, { recursive: true, force: true });+ rmSync(nested, { recursive: true, force: true });+ }+});+test("walkDir retains fixed skips and filesystem fallback outside Git", () => {const dir = mkdtempSync(join(tmpdir(), "graft-walk-nongit-"));try {