Understand any codebase instantly. System intelligence for codebases, built for humans and AI.
TypeScript aiai-memoryclaude-codeclicode-analysiscode-mappingcode-memorycodebase-understandingcodexdeveloper-toolsllmopenclawpersistent-memoryprogram-analysissoftware-architecturesystem-intelligencesystem-mapping
merge: integrate feature/map-update with map algorithm overhaul
Tanner Kocher1f7abf218b17
diff --git a/AGENTS.md b/AGENTS.mdindex 0e72fb17..1fe58ccd 100644--- a/AGENTS.md+++ b/AGENTS.md@@ -89,6 +89,8 @@ ix bug show <id> --format json # Bug details- "What are the most critical methods?" → `ix rank --by callers --kind method --top 10`- "What changed between rev 5 and 10?" → `ix diff 5 10 --summary`- "Show me all changes" → `ix diff 5 10 --full --format json`+- "Are there architectural problems?" → `ix smells --format json`+- "Which subsystems are unhealthy?" → `ix subsystems --format json`### Low-level structural primitives@@ -119,9 +121,24 @@ ix imported-by <symbol> --format jsonix depends <symbol> --format json```-### Ingesting data+### Architecture analysis```-ix ingest ./src --recursive --format json+ix smells --format json # Detect orphan files, god modules, weak components+ix smells --list --format json # List previously stored smell claims+ix smells --god-module-chunks 15 --god-module-fan 10 # Custom thresholds+ix subsystems --format json # Score subsystems by health (0–1)+ix subsystems --list --format json # List stored scores+ix subsystems --level 2 --format json # Filter to subsystem level+```++**When to use which:**+- "Are there any architectural problems?" → `ix smells`+- "Which subsystems are unhealthy?" → `ix subsystems --format json`+- "Show me god modules" → `ix smells --format json | jq '.candidates[] | select(.smell == "has_smell.god_module")'`++### Updating the graph+```+ix map --format jsonix ingest --github owner/repo --format json --limit 50ix ingest --github owner/repo --since 2026-01-01 --format json```@@ -135,27 +152,72 @@ ix diff <from> <to> --format json # Full diff (default limit 100)ix diff <from> <to> --full --format json # All changes, no limit```-### Planning & decisions+### Planning & decisions (Pro)```+# Goalsix goal create "GitHub ingestion pipeline" --format json-ix goal list --format json-ix plan create "GitHub Ingestion" --goal <goal-id> --format json+ix goal show <id-or-name> --format json+ix goal list --status active --format json+ix goals --format json # shorthand list++# Plans+ix plan create "GitHub Ingestion" --goal <goal-id> --responds-to <bugId> --format json+ix plan status <plan-id> --format json # task list + critical path + open bug count+ix plan next <plan-id> --with-workflow --format json # next actionable task + workflow+ix plan next <plan-id> --run-workflow --stage discover --format json # execute one stage+ix plans --format json # shorthand list++# Tasksix plan task "API fetch layer" --plan <plan-id> --depends-on <task-id> --format json-ix plan task "title" --plan <id> --workflow "cmd1,cmd2" --format json # Task with flat workflow-ix plan task "title" --plan <id> --workflow-staged '{"discover":["cmd1"],"implement":["cmd2"],"validate":["cmd3"]}' --format json # Staged workflow-ix plan task "title" --plan <id> --resolves <bugId> --format json # Task resolving a bug-ix plan status <plan-id> --format json # Includes openBugCount in summary-ix plan next <plan-id> --format json # Explains reason if no actionable task-ix plan next <plan-id> --with-workflow --format json # Next task with staged workflow display-ix tasks --format json # List all tasks across plans-ix tasks --status pending --format json # Filter tasks by status-ix tasks --plan <plan-id> --format json # Tasks in a specific plan-ix task show <task-id> --with-workflow --format json # Task details with workflow+ix plan task "title" --plan <id> --workflow-staged '{"discover":["ix overview X"],"implement":["ix map"],"validate":["ix smells"]}' --format json+ix plan task "title" --plan <id> --resolves <bugId> --format json+ix tasks --format json # list all tasks across plans+ix tasks --status pending --plan <plan-id> --format json+ix task show <task-id> --with-workflow --format jsonix task update <task-id> --status done --format json+ix task update <task-id> --run-workflow implement --format json # run a workflow stage++# Decisionsix decide "Use client-side patch" --rationale "..." --affects IngestionService --format json-ix decide "Use JWT" --rationale "..." --responds-to <bugId> --format json # Decision responding to a bug+ix decide "Use JWT" --rationale "..." --responds-to <bugId> --format json++# Bugs+ix bug create "title" --severity high --affects Entity --format json+ix bug update <id> --status resolved --format json+ix bug show <id> --format json+ix bugs --status open --format json++# Misc+ix patches --limit 20 --format json # recent graph patches+ix truth add "Support 100k file repos"+ix truth list --format json```+### Workflows (Pro)+Staged command sequences attached to tasks, plans, or decisions. Stages: `discover → implement → validate`. All commands must start with `ix ` — no arbitrary shell.+```+ix workflow attach task <id> --file workflow.json # attach from JSON file+ix workflow show task <id> --format json # view attached workflow+ix workflow validate task <id> --format json # check structure+ix workflow run task <id> --stage implement --format json # execute one stage+```++**Workflow JSON format:**+```json+{+ "discover": ["ix overview AuthService", "ix impact AuthService"],+ "implement": ["ix map"],+ "validate": ["ix smells --format json"]+}+```++**When to use which:**+- "What work is planned?" → `ix plans --format json` then `ix plan status <id>`+- "What should I do next?" → `ix plan next <id> --with-workflow`+- "Execute the next task's discovery phase" → `ix plan next <id> --run-workflow --stage discover`+- "Mark a task done" → `ix task update <id> --status done`+- "Link a decision to a bug" → `ix decide "title" --rationale "..." --responds-to <bugId>`+## Semantic boundaries- **decision** — a choice between alternatives, with rationale. Use `ix decide`.diff --git a/CLAUDE.md b/CLAUDE.mdindex 7642a820..15f1f736 100644--- a/CLAUDE.md+++ b/CLAUDE.md@@ -14,7 +14,7 @@ Use JSON when chaining command results.2. AFTER every design or architecture decision → run `ix decide <title> --rationale <text>`.3. When you notice contradictory information → run `ix conflicts` and present results to the user.4. NEVER guess about codebase facts — if Ix has structured data, use it.-5. IMMEDIATELY after modifying code → run `ix ingest <path>` on changed files.+5. IMMEDIATELY after modifying code → run `ix map` to re-ingest and update the graph.6. When the user states a goal → run `ix truth add "<statement>"`.## Ix CLI Command Routing@@ -72,13 +72,58 @@ Underlying structural commands — useful for debugging or fine-grained inspecti| Record a goal | `ix truth add` | `ix truth add "Support 100k file repos"` || List goals | `ix truth list` | `ix truth list --format json` || Bug tracking | `ix bug create` | `ix bug create "title" --severity high --affects Entity` |-| Bug listing | `ix bugs` | `ix bugs --format json` |+| Update bug status | `ix bug update` | `ix bug update <id> --status resolved` |+| Bug listing | `ix bugs` | `ix bugs --status open --format json` || Bug details | `ix bug show` | `ix bug show <id> --format json` |+| List recent patches | `ix patches` | `ix patches --limit 20 --format json` |++### Planning (Pro)+| Goal | Command | Example |+|---|---|---|+| Create a goal | `ix goal create` | `ix goal create "Support GitHub" --format json` |+| List goals | `ix goals` | `ix goals --status active --format json` |+| Create a plan | `ix plan create` | `ix plan create "Fix auth" --goal <id> --responds-to <bugId> --format json` |+| Add a task | `ix plan task` | `ix plan task "Step 1" --plan <id> --depends-on <taskId> --resolves <bugId> --workflow-staged '{"discover":["ix overview X"],"implement":["ix map"],"validate":["ix smells"]}' --format json` |+| Plan status | `ix plan status` | `ix plan status <id> --format json` |+| Next actionable task | `ix plan next` | `ix plan next <id> --with-workflow --format json` |+| Run next task workflow | `ix plan next` | `ix plan next <id> --run-workflow --stage discover --format json` |+| List all plans | `ix plans` | `ix plans --format json` |+| List tasks | `ix tasks` | `ix tasks --status pending --plan <id> --format json` |+| Task details | `ix task show` | `ix task show <id> --with-workflow --format json` |+| Update task | `ix task update` | `ix task update <id> --status done --format json` |+| Run task workflow stage | `ix task update` | `ix task update <id> --run-workflow discover --format json` |++### Workflows (Pro)+Workflows are staged command sequences (discover → implement → validate) attached to tasks, plans, or decisions. All commands must start with `ix ` — no shell operators.++| Goal | Command | Example |+|---|---|---|+| Attach workflow | `ix workflow attach` | `ix workflow attach task <id> --file workflow.json` |+| Show workflow | `ix workflow show` | `ix workflow show task <id> --format json` |+| Validate workflow | `ix workflow validate` | `ix workflow validate task <id>` |+| Run workflow | `ix workflow run` | `ix workflow run task <id> --stage implement --format json` |++**Workflow JSON format:**+```json+{+ "discover": ["ix overview AuthService", "ix impact AuthService"],+ "implement": ["ix map"],+ "validate": ["ix smells --format json", "ix subsystems --format json"]+}+```++### Architecture Analysis+| Goal | Command | Example |+|---|---|---|+| Detect code smells | `ix smells` | `ix smells --format json` |+| Score subsystems | `ix subsystems` | `ix subsystems --level 2 --format json` |+| List smell claims | `ix smells --list` | `ix smells --list --format json` |+| List subsystem scores | `ix subsystems --list` | `ix subsystems --list --format json` |### Ingestion & Health| Goal | Command | Example ||---|---|---|-| Ingest files | `ix ingest` | `ix ingest ./src` |+| Update graph + map | `ix map` | `ix map` || Ingest GitHub data | `ix ingest` | `ix ingest --github owner/repo --limit 50` || Backend health | `ix status` | `ix status` || Graph statistics | `ix stats` | `ix stats --format json` |@@ -137,6 +182,6 @@ Use the right entity type for the right purpose:## Confidence ScoresIx returns confidence scores with results. When data has low confidence:- Mention the uncertainty to the user-- Suggest re-ingesting the relevant files+- Suggest re-running `ix map` to refresh the graph- Never present low-confidence data as established fact<!-- IX-MEMORY END -->diff --git a/core-ingestion/dist/index.d.ts b/core-ingestion/dist/index.d.tsindex 1a7c51d5..d24611fc 100644--- a/core-ingestion/dist/index.d.ts+++ b/core-ingestion/dist/index.d.ts@@ -9,6 +9,22 @@ export interface ParsedEntity {/** Direct enclosing class/interface/trait, if any (undefined for file-level entities). */container?: string;}+/** A semantic code span extracted from the AST. The primary LLM retrieval unit. */+export interface ParsedChunk {+ /** Semantic name of the chunk (function/class/trait name), or null for file_body. */+ name: string | null;+ /** Chunk kind: "function" | "method" | "class" | "interface" | "trait" | "module_block" | "file_body" */+ chunkKind: string;+ lineStart: number;+ lineEnd: number;+ startByte: number;+ endByte: number;+ /** SHA-256 of the chunk source text for change detection and stable identity. */+ contentHash: string;+ language: string;+ /** Name of the directly enclosing class/trait/interface, if any. */+ container?: string;+}export interface ParsedRelationship {srcName: string;dstName: string;@@ -18,6 +34,7 @@ export interface FileParseResult {filePath: string;language: SupportedLanguages;entities: ParsedEntity[];+ chunks: ParsedChunk[];relationships: ParsedRelationship[];fileRole: RoleClassification;}diff --git a/core-ingestion/dist/index.js b/core-ingestion/dist/index.jsindex 712b01de..d3e96d77 100644--- a/core-ingestion/dist/index.js+++ b/core-ingestion/dist/index.js@@ -1,4 +1,5 @@import * as nodePath from 'node:path';+import * as crypto from 'node:crypto';// @ts-ignore — tree-sitter has no bundled typesimport Parser from 'tree-sitter';// @ts-ignore@@ -124,14 +125,12 @@ export function isGrammarSupported(filePath) {// Parser instance (reused across calls)// ---------------------------------------------------------------------------let _parser = null;+const _queryCache = new Map();function getParser() {if (!_parser)_parser = new Parser();return _parser;}-// Per-process cache for compiled tree-sitter queries, keyed by language.-// Avoids recompiling the same query string for every file of the same language.-const _queryCache = new Map();function getCachedQuery(language, grammar, querySource) {const cached = _queryCache.get(language);if (cached && cached.grammar === grammar)@@ -163,16 +162,17 @@ export function parseFile(filePath, source) {const query = getCachedQuery(cacheKey, grammar, queries);const matches = query.matches(tree.rootNode);const fileName = nodePath.basename(filePath);- // Count newlines instead of allocating a full string array- let lineCount = 1;+ let sourceLineCount = 1;for (let i = 0; i < source.length; i++) {if (source.charCodeAt(i) === 10)- lineCount++;+ sourceLineCount++;}const entities = [- { name: fileName, kind: 'file', lineStart: 1, lineEnd: lineCount, language },+ { name: fileName, kind: 'file', lineStart: 1, lineEnd: sourceLineCount, language },];+ const chunks = [];const relationships = [];+ const pendingChunks = [];// Track class ranges for containment: [name, startLine, endLine]const classRanges = [];// Track seen calls per enclosing scope to avoid duplicate CALLS edges@@ -193,6 +193,8 @@ export function parseFile(filePath, source) {const defNode = defCapture.node;const lineStart = defNode.startPosition.row + 1;const lineEnd = defNode.endPosition.row + 1;+ const startByte = defNode.startIndex;+ const endByte = defNode.endIndex;// Containment: file CONTAINS or class CONTAINSconst enclosing = findEnclosing(classRanges, lineStart, name);entities.push({@@ -203,6 +205,16 @@ export function parseFile(filePath, source) {language,container: enclosing ?? undefined,});+ pendingChunks.push({+ name,+ chunkKind: kind,+ lineStart,+ lineEnd,+ startByte,+ endByte,+ language,+ container: enclosing ?? undefined,+ });if (kind === 'class' || kind === 'interface' || kind === 'trait') {classRanges.push({ name, start: lineStart, end: lineEnd });}@@ -345,7 +357,29 @@ export function parseFile(filePath, source) {continue;}}- return { filePath, language, entities, relationships, fileRole: classifyFileRole(filePath, source) };+ for (const pendingChunk of pendingChunks) {+ const chunkText = source.slice(pendingChunk.startByte, pendingChunk.endByte);+ const contentHash = crypto.createHash('sha256').update(chunkText).digest('hex').slice(0, 16);+ chunks.push({+ ...pendingChunk,+ contentHash,+ });+ }+ // Fallback: if no semantic chunks found, emit one file_body chunk covering the whole file+ if (chunks.length === 0) {+ const contentHash = crypto.createHash('sha256').update(source).digest('hex').slice(0, 16);+ chunks.push({+ name: null,+ chunkKind: 'file_body',+ lineStart: 1,+ lineEnd: sourceLineCount,+ startByte: 0,+ endByte: source.length,+ contentHash,+ language,+ });+ }+ return { filePath, language, entities, chunks, relationships, fileRole: classifyFileRole(filePath, source) };}catch {return null;@@ -367,18 +401,33 @@ function bestQKey(fileQKeys, filePath, plainName) {const qks = [...new Set(fileQKeys.get(filePath)?.get(plainName) ?? [])];return qks.length === 1 ? qks[0] : null;}-export function resolveEdges(results, statsOut) {- const stats = statsOut ?? {- importLookups: 0, transitiveLookups: 0, globalFallbacks: 0,- globalCandidateTotal: 0, resolvedImport: 0, resolvedTransitive: 0,- resolvedGlobal: 0, resolvedQualifier: 0, skippedSameFile: 0, skippedAmbiguous: 0,- };- // ── Index structures (built once) ──────────────────────────────────- // Direct file lookup: filePath → FileParseResult (replaces results.find())- const resultByPath = new Map();- for (const r of results) {- resultByPath.set(r.filePath, r);- }+/**+ * Resolves CALLS and EXTENDS relationships to their cross-file targets by+ * building a symbol table and import map over the full batch of parsed files.+ *+ * Tiers (in priority order):+ * 0.9 — import-scoped: dst is in a file the src explicitly imports+ * 0.8 — transitive import-scoped: one re-export hop away+ * 0.5 — global fallback: dst exists in exactly one other file (unambiguous)+ *+ * Same-file edges are already handled correctly by buildPatch and are not+ * emitted here.+ *+ * Import resolution handles:+ * - Stem-based: './payments' → 'payments.ts'+ * - Path-aliased: '@/lib/payments' → 'payments.ts' (last segment)+ * - Directory index: './services' → 'services/index.ts'+ * - Bare aliases: '@components' → 'components.ts' (strip leading non-word chars)+ * - Dotted paths: 'ix.memory.model.Edge' → stem 'Edge' → 'Edge.scala'+ * (handles Scala/Java package imports where tree-sitter+ * emits each identifier separately)+ *+ * Qualified-key resolution:+ * - Module-level function/class: dstQualifiedKey === dstName+ * - Unambiguous class method: dstQualifiedKey === 'ClassName.method'+ * - Ambiguous (two entities share the same plain name): edge not emitted+ */+export function resolveEdges(results) {// fileQKeys: filePath → (plainName → qualifiedKeys[])// Mirrors the entityQKey computation in buildPatch so nodeIds match exactly.const fileQKeys = new Map();@@ -399,22 +448,15 @@ export function resolveEdges(results, statsOut) {for (const [fp, qkMap] of fileQKeys) {fileHasSymbol.set(fp, new Set(qkMap.keys()));}- // Precompute language per file (avoids calling languageFromPath in hot loops)- const fileLanguage = new Map();- for (const r of results) {- fileLanguage.set(r.filePath, r.language);- }- // Inverted symbol index: symbolName → Array<{filePath, language}>- // Replaces the O(N) global fallback scan with O(candidates) lookup.+ // resultsByPath: O(1) lookup replacing results.find() in transitive import loop+ const resultsByPath = new Map(results.map(r => [r.filePath, r]));+ // symbolToFiles: plainName → filePath[] — replaces O(F) full-scan in tier-3 and qualifier fallbackconst symbolToFiles = new Map();for (const [fp, symbols] of fileHasSymbol) {for (const sym of symbols) {- let arr = symbolToFiles.get(sym);- if (!arr) {- arr = [];- symbolToFiles.set(sym, arr);- }- arr.push(fp);+ const list = symbolToFiles.get(sym) ?? [];+ list.push(fp);+ symbolToFiles.set(sym, list);}}// stemToFiles: basename-without-extension → filePath[]@@ -458,7 +500,6 @@ export function resolveEdges(results, statsOut) {packageToFiles.set(pkg, list);}}- // ── Helpers ────────────────────────────────────────────────────────/** Resolve a module name to matching file paths in the batch. */function modNameToFiles(modName, excludeFp) {const fps = [];@@ -496,12 +537,10 @@ export function resolveEdges(results, statsOut) {}return fps;}- // ── Main resolution loop ───────────────────────────────────────────const resolved = [];for (const result of results) {const srcFilePath = result.filePath;const srcSymbols = fileHasSymbol.get(srcFilePath);- const srcLanguage = result.language;// Build the set of file paths this file explicitly imports.const importedFilePaths = new Set();for (const rel of result.relationships) {@@ -515,7 +554,7 @@ export function resolveEdges(results, statsOut) {// Handles re-exports: baz.ts → index.ts (re-exports from bar.ts) → bar.tsconst transitiveFilePaths = new Set();for (const fp of importedFilePaths) {- const fpResult = resultByPath.get(fp); // O(1) instead of results.find()+ const fpResult = resultsByPath.get(fp);if (!fpResult)continue;for (const rel of fpResult.relationships) {@@ -556,7 +595,6 @@ export function resolveEdges(results, statsOut) {if (dstQualifiedKey === null)continue;resolved.push({ srcFilePath, srcName: rel.srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: 'IMPORTS', confidence: 0.9 });- stats.resolvedImport++;continue;}const dstName = rel.dstName;@@ -583,23 +621,18 @@ export function resolveEdges(results, statsOut) {if (dstQualifiedKey !== null) {// dstName must match rel.dstName so buildPatchWithResolution can look it upresolved.push({ srcFilePath, srcName, dstFilePath: qfp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.9 });- stats.resolvedQualifier++;}}continue;}- // Global qualifier fallback — use inverted index instead of scanning all files- const qualCandidates = symbolToFiles.get(qualifierPart) ?? [];- const qualGlobalMatches = qualCandidates.filter(fp => fp !== srcFilePath);- stats.globalFallbacks++;- stats.globalCandidateTotal += qualGlobalMatches.length;+ // Global qualifier fallback — use symbolToFiles index instead of full scan+ const qualGlobalMatches = (symbolToFiles.get(qualifierPart) ?? []).filter(fp => fp !== srcFilePath);if (qualGlobalMatches.length === 1) {const qfp = qualGlobalMatches[0];if (fileHasSymbol.get(qfp)?.has(memberPart)) {const dstQualifiedKey = bestQKey(fileQKeys, qfp, memberPart);if (dstQualifiedKey !== null) {resolved.push({ srcFilePath, srcName, dstFilePath: qfp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.7 });- stats.resolvedQualifier++;}}}@@ -607,12 +640,9 @@ export function resolveEdges(results, statsOut) {}}// Tier 1: same-file — already correct in buildPatch, skip here- if (srcSymbols.has(dstName)) {- stats.skippedSameFile++;+ if (srcSymbols.has(dstName))continue;- }// Tier 2: import-scoped (confidence 0.9)- stats.importLookups++;const importMatches = [];for (const fp of importedFilePaths) {if (fileHasSymbol.get(fp)?.has(dstName))@@ -624,15 +654,11 @@ export function resolveEdges(results, statsOut) {if (dstQualifiedKey === null)continue; // ambiguous — do not emit bad nodeIdresolved.push({ srcFilePath, srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.9 });- stats.resolvedImport++;continue;}- if (importMatches.length > 1) {- stats.skippedAmbiguous++;- continue;- } // ambiguous file — do not emit+ if (importMatches.length > 1)+ continue; // ambiguous file — do not emit// Tier 2.5: transitive import-scoped (confidence 0.8) — one re-export hop away- stats.transitiveLookups++;const transitiveMatches = [];for (const fp of transitiveFilePaths) {if (fileHasSymbol.get(fp)?.has(dstName))@@ -644,26 +670,21 @@ export function resolveEdges(results, statsOut) {if (dstQualifiedKey === null)continue;resolved.push({ srcFilePath, srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.8 });- stats.resolvedTransitive++;continue;}- if (transitiveMatches.length > 1) {- stats.skippedAmbiguous++;- continue;- } // ambiguous — do not emit- // Tier 3: global fallback (confidence 0.5) — uses inverted symbol index- // instead of scanning all files.- stats.globalFallbacks++;- const candidates = symbolToFiles.get(dstName) ?? [];- const globalMatches = candidates.filter(fp => fp !== srcFilePath && fileLanguage.get(fp) === srcLanguage);- stats.globalCandidateTotal += globalMatches.length;+ if (transitiveMatches.length > 1)+ continue; // ambiguous — do not emit+ // Tier 3: global fallback (confidence 0.5) — exactly one other file defines it+ // Only match files in the same language as the caller to avoid cross-language false positives.+ const srcLanguage = result.language;+ const globalMatches = (symbolToFiles.get(dstName) ?? [])+ .filter(fp => fp !== srcFilePath && languageFromPath(fp) === srcLanguage);if (globalMatches.length === 1) {const fp = globalMatches[0];const dstQualifiedKey = bestQKey(fileQKeys, fp, dstName);if (dstQualifiedKey === null)continue; // ambiguous — do not emit bad nodeIdresolved.push({ srcFilePath, srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.5 });- stats.resolvedGlobal++;}// 0 or >1 global matches — leave as dangling edge, do not emit}diff --git a/core-ingestion/dist/patch-builder.js b/core-ingestion/dist/patch-builder.jsindex 684c9d48..89e88c19 100644--- a/core-ingestion/dist/patch-builder.js+++ b/core-ingestion/dist/patch-builder.js@@ -22,6 +22,10 @@ function nodeId(filePath, name) {function edgeId(filePath, src, dst, predicate) {return deterministicId(`${normalizePath(filePath)}:${src}:${dst}:${predicate}`);}+/** Deterministic ID for a chunk — keyed on file + kind + name + start line to survive minor edits. */+function chunkId(filePath, chunkKind, name, startLine) {+ return deterministicId(`${normalizePath(filePath)}:chunk:${chunkKind}:${name ?? 'file_body'}:${startLine}`);+}// ---------------------------------------------------------------------------// Source type from file extension// ---------------------------------------------------------------------------@@ -34,10 +38,10 @@ function sourceType(filePath) {return 'code';}export function extractorName() {- return `tree-sitter/1.12`;+ return `tree-sitter/1.13`;}/** Previous extractor versions — their patches are superseded when re-ingesting. */-export const PREVIOUS_EXTRACTORS = ['tree-sitter/1.11', 'tree-sitter/1.10', 'tree-sitter/1.9', 'tree-sitter/1.8', 'tree-sitter/1.7', 'tree-sitter/1.6', 'tree-sitter/1.5', 'tree-sitter/1.4', 'tree-sitter/1.3', 'tree-sitter/1.2', 'tree-sitter/1.1'];+export const PREVIOUS_EXTRACTORS = ['tree-sitter/1.12', 'tree-sitter/1.11', 'tree-sitter/1.10', 'tree-sitter/1.9', 'tree-sitter/1.8', 'tree-sitter/1.7', 'tree-sitter/1.6', 'tree-sitter/1.5', 'tree-sitter/1.4', 'tree-sitter/1.3', 'tree-sitter/1.2', 'tree-sitter/1.1'];/** Compute a patchId for a (filePath, sourceHash, extractorVersion) triple. */function computePatchId(filePath, sourceHash, extractor) {return deterministicId(`${normalizePath(filePath)}:${sourceHash}:${extractor}`);@@ -50,7 +54,7 @@ function legacyPatchId(filePath, sourceHash) {// Build a GraphPatchPayload from a FileParseResult// ---------------------------------------------------------------------------export function buildPatch(result, sourceHash, previousSourceHash) {- const { filePath, entities, relationships } = result;+ const { filePath, entities, chunks, relationships } = result;const ops = [];// Build a qualified-key map so that same-named entities in different// enclosing classes within the same file get distinct nodeIds.@@ -107,6 +111,71 @@ export function buildPatch(result, sourceHash, previousSourceHash) {});}}+ // UpsertNode + edges for each chunk+ const fileNodeId = nodeId(filePath, entities.find(e => e.kind === 'file')?.name ?? filePath);+ for (const chunk of chunks) {+ const cid = chunkId(filePath, chunk.chunkKind, chunk.name, chunk.lineStart);+ const chunkName = chunk.name ?? `file_body:${chunk.lineStart}`;+ ops.push({+ type: 'UpsertNode',+ id: cid,+ kind: 'chunk',+ name: chunkName,+ attrs: {+ file_uri: filePath,+ language: chunk.language,+ chunk_kind: chunk.chunkKind,+ start_line: chunk.lineStart,+ end_line: chunk.lineEnd,+ start_byte: chunk.startByte,+ end_byte: chunk.endByte,+ content_hash: chunk.contentHash,+ parser_version: extractorName(),+ },+ });+ // File -[CONTAINS]-> Chunk+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, 'file', chunkName, 'CONTAINS_CHUNK'),+ src: fileNodeId,+ dst: cid,+ predicate: 'CONTAINS_CHUNK',+ attrs: {},+ });+ // Chunk -[DEFINES]-> Symbol (only for named chunks)+ if (chunk.name !== null) {+ const symbolKey = chunk.container ? `${chunk.container}.${chunk.name}` : chunk.name;+ const symbolNid = nodeId(filePath, symbolKey);+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, chunkName, symbolKey, 'DEFINES'),+ src: cid,+ dst: symbolNid,+ predicate: 'DEFINES',+ attrs: {},+ });+ }+ }+ // NEXT edges for source-order chunk adjacency+ for (let i = 0; i + 1 < chunks.length; i++) {+ const a = chunks[i];+ const b = chunks[i + 1];+ // Only link top-level chunks (no container) to avoid intra-class ordering noise+ if (a.container == null && b.container == null) {+ const aid = chunkId(filePath, a.chunkKind, a.name, a.lineStart);+ const bid = chunkId(filePath, b.chunkKind, b.name, b.lineStart);+ const aName = a.name ?? `file_body:${a.lineStart}`;+ const bName = b.name ?? `file_body:${b.lineStart}`;+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, aName, bName, 'NEXT'),+ src: aid,+ dst: bid,+ predicate: 'NEXT',+ attrs: {},+ });+ }+ }// UpsertEdge for each relationshipfor (const r of relationships) {// For CONTAINS edges, srcName is the container of dstName — use that to disambiguate.@@ -177,7 +246,7 @@ export function buildPatchWithResolution(result, sourceHash, resolvedEdges, prevdstQualifiedKey: edge.dstQualifiedKey,});}- const { filePath, entities, relationships } = result;+ const { filePath, entities, chunks, relationships } = result;const ops = [];const entityQKey = new Map();for (const e of entities) {@@ -218,6 +287,67 @@ export function buildPatchWithResolution(result, sourceHash, resolvedEdges, prev});}}+ // UpsertNode + edges for each chunk (same logic as buildPatch)+ const fileNodeId2 = nodeId(filePath, entities.find(e => e.kind === 'file')?.name ?? filePath);+ for (const chunk of chunks) {+ const cid = chunkId(filePath, chunk.chunkKind, chunk.name, chunk.lineStart);+ const chunkName = chunk.name ?? `file_body:${chunk.lineStart}`;+ ops.push({+ type: 'UpsertNode',+ id: cid,+ kind: 'chunk',+ name: chunkName,+ attrs: {+ file_uri: filePath,+ language: chunk.language,+ chunk_kind: chunk.chunkKind,+ start_line: chunk.lineStart,+ end_line: chunk.lineEnd,+ start_byte: chunk.startByte,+ end_byte: chunk.endByte,+ content_hash: chunk.contentHash,+ parser_version: extractorName(),+ },+ });+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, 'file', chunkName, 'CONTAINS_CHUNK'),+ src: fileNodeId2,+ dst: cid,+ predicate: 'CONTAINS_CHUNK',+ attrs: {},+ });+ if (chunk.name !== null) {+ const symbolKey = chunk.container ? `${chunk.container}.${chunk.name}` : chunk.name;+ const symbolNid = nodeId(filePath, symbolKey);+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, chunkName, symbolKey, 'DEFINES'),+ src: cid,+ dst: symbolNid,+ predicate: 'DEFINES',+ attrs: {},+ });+ }+ }+ for (let i = 0; i + 1 < chunks.length; i++) {+ const a = chunks[i];+ const b = chunks[i + 1];+ if (a.container == null && b.container == null) {+ const aid = chunkId(filePath, a.chunkKind, a.name, a.lineStart);+ const bid = chunkId(filePath, b.chunkKind, b.name, b.lineStart);+ const aName = a.name ?? `file_body:${a.lineStart}`;+ const bName = b.name ?? `file_body:${b.lineStart}`;+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, aName, bName, 'NEXT'),+ src: aid,+ dst: bid,+ predicate: 'NEXT',+ attrs: {},+ });+ }+ }for (const r of relationships) {const srcKey = resolveKey(r.srcName);const dstKey = r.predicate === 'CONTAINS'diff --git a/core-ingestion/src/__tests__/__snapshots__/parseFile.test.ts.snap b/core-ingestion/src/__tests__/__snapshots__/parseFile.test.ts.snapdeleted file mode 100644index ed6ed9e8..00000000--- a/core-ingestion/src/__tests__/__snapshots__/parseFile.test.ts.snap+++ /dev/null@@ -1,1347 +0,0 @@-// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html--exports[`parseFile snapshots > snapshots Identifiers.scala 1`] = `-{- "entities": [- {- "kind": "file",- "language": "scala",- "lineEnd": 42,- "lineStart": 1,- "name": "Identifiers.scala",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 7,- "lineStart": 7,- "name": "NodeId",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 11,- "lineStart": 8,- "name": "NodeId",- },- {- "container": "NodeId",- "kind": "function",- "language": "scala",- "lineEnd": 9,- "lineStart": 9,- "name": "encoder",- },- {- "container": "NodeId",- "kind": "function",- "language": "scala",- "lineEnd": 10,- "lineStart": 10,- "name": "decoder",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 13,- "lineStart": 13,- "name": "EdgeId",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 17,- "lineStart": 14,- "name": "EdgeId",- },- {- "container": "EdgeId",- "kind": "function",- "language": "scala",- "lineEnd": 15,- "lineStart": 15,- "name": "encoder",- },- {- "container": "EdgeId",- "kind": "function",- "language": "scala",- "lineEnd": 16,- "lineStart": 16,- "name": "decoder",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 19,- "lineStart": 19,- "name": "PatchId",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 23,- "lineStart": 20,- "name": "PatchId",- },- {- "container": "PatchId",- "kind": "function",- "language": "scala",- "lineEnd": 21,- "lineStart": 21,- "name": "encoder",- },- {- "container": "PatchId",- "kind": "function",- "language": "scala",- "lineEnd": 22,- "lineStart": 22,- "name": "decoder",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 25,- "lineStart": 25,- "name": "ClaimId",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 29,- "lineStart": 26,- "name": "ClaimId",- },- {- "container": "ClaimId",- "kind": "function",- "language": "scala",- "lineEnd": 27,- "lineStart": 27,- "name": "encoder",- },- {- "container": "ClaimId",- "kind": "function",- "language": "scala",- "lineEnd": 28,- "lineStart": 28,- "name": "decoder",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 31,- "lineStart": 31,- "name": "ConflictId",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 35,- "lineStart": 32,- "name": "ConflictId",- },- {- "container": "ConflictId",- "kind": "function",- "language": "scala",- "lineEnd": 33,- "lineStart": 33,- "name": "encoder",- },- {- "container": "ConflictId",- "kind": "function",- "language": "scala",- "lineEnd": 34,- "lineStart": 34,- "name": "decoder",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 37,- "lineStart": 37,- "name": "Rev",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 41,- "lineStart": 38,- "name": "Rev",- },- {- "container": "Rev",- "kind": "function",- "language": "scala",- "lineEnd": 39,- "lineStart": 39,- "name": "encoder",- },- {- "container": "Rev",- "kind": "function",- "language": "scala",- "lineEnd": 40,- "lineStart": 40,- "name": "decoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 3,- "lineStart": 3,- "name": "java.util.UUID",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 5,- "lineStart": 5,- "name": "io.circe.Decoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 5,- "lineStart": 5,- "name": "io.circe.Encoder",- },- ],- "filePath": "memory-layer/src/main/scala/ix/memory/model/Identifiers.scala",- "fileRole": {- "role": "production",- "role_confidence": 0.5,- "role_signals": [],- },- "language": "scala",- "relationships": [- {- "dstName": "NodeId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "AnyVal",- "predicate": "EXTENDS",- "srcName": "NodeId",- },- {- "dstName": "NodeId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "NodeId",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "NodeId",- },- {- "dstName": "EdgeId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "AnyVal",- "predicate": "EXTENDS",- "srcName": "EdgeId",- },- {- "dstName": "EdgeId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "EdgeId",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "EdgeId",- },- {- "dstName": "PatchId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "AnyVal",- "predicate": "EXTENDS",- "srcName": "PatchId",- },- {- "dstName": "PatchId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "PatchId",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "PatchId",- },- {- "dstName": "ClaimId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "AnyVal",- "predicate": "EXTENDS",- "srcName": "ClaimId",- },- {- "dstName": "ClaimId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "ClaimId",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "ClaimId",- },- {- "dstName": "ConflictId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "AnyVal",- "predicate": "EXTENDS",- "srcName": "ConflictId",- },- {- "dstName": "ConflictId",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "ConflictId",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "ConflictId",- },- {- "dstName": "Rev",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "AnyVal",- "predicate": "EXTENDS",- "srcName": "Rev",- },- {- "dstName": "Rev",- "predicate": "CONTAINS",- "srcName": "Identifiers.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "Rev",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "Rev",- },- {- "dstName": "java.util.UUID",- "predicate": "IMPORTS",- "srcName": "Identifiers.scala",- },- {- "dstName": "io.circe.Decoder",- "predicate": "IMPORTS",- "srcName": "Identifiers.scala",- },- {- "dstName": "io.circe.Encoder",- "predicate": "IMPORTS",- "srcName": "Identifiers.scala",- },- {- "dstName": "UUID",- "predicate": "REFERENCES",- "srcName": "NodeId",- },- {- "dstName": "NodeId",- "predicate": "REFERENCES",- "srcName": "Identifiers.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "NodeId.encoder",- },- {- "dstName": "NodeId",- "predicate": "CALLS",- "srcName": "NodeId.decoder",- },- {- "dstName": "UUID",- "predicate": "REFERENCES",- "srcName": "EdgeId",- },- {- "dstName": "EdgeId",- "predicate": "REFERENCES",- "srcName": "Identifiers.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "EdgeId.encoder",- },- {- "dstName": "EdgeId",- "predicate": "CALLS",- "srcName": "EdgeId.decoder",- },- {- "dstName": "UUID",- "predicate": "REFERENCES",- "srcName": "PatchId",- },- {- "dstName": "PatchId",- "predicate": "REFERENCES",- "srcName": "Identifiers.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "PatchId.encoder",- },- {- "dstName": "PatchId",- "predicate": "CALLS",- "srcName": "PatchId.decoder",- },- {- "dstName": "UUID",- "predicate": "REFERENCES",- "srcName": "ClaimId",- },- {- "dstName": "ClaimId",- "predicate": "REFERENCES",- "srcName": "Identifiers.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "ClaimId.encoder",- },- {- "dstName": "ClaimId",- "predicate": "CALLS",- "srcName": "ClaimId.decoder",- },- {- "dstName": "UUID",- "predicate": "REFERENCES",- "srcName": "ConflictId",- },- {- "dstName": "ConflictId",- "predicate": "REFERENCES",- "srcName": "Identifiers.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "ConflictId.encoder",- },- {- "dstName": "ConflictId",- "predicate": "CALLS",- "srcName": "ConflictId.decoder",- },- {- "dstName": "Rev",- "predicate": "REFERENCES",- "srcName": "Identifiers.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "Rev.encoder",- },- {- "dstName": "Rev",- "predicate": "CALLS",- "srcName": "Rev.decoder",- },- ],-}-`;--exports[`parseFile snapshots > snapshots Node.scala 1`] = `-{- "entities": [- {- "kind": "file",- "language": "scala",- "lineEnd": 97,- "lineStart": 1,- "name": "Node.scala",- },- {- "container": undefined,- "kind": "interface",- "language": "scala",- "lineEnd": 8,- "lineStart": 8,- "name": "NodeKind",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 77,- "lineStart": 9,- "name": "NodeKind",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 10,- "lineStart": 10,- "name": "Module",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 11,- "lineStart": 11,- "name": "File",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 12,- "lineStart": 12,- "name": "Class",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 13,- "lineStart": 13,- "name": "Function",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 14,- "lineStart": 14,- "name": "Variable",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 15,- "lineStart": 15,- "name": "Config",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 16,- "lineStart": 16,- "name": "ConfigEntry",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 17,- "lineStart": 17,- "name": "Service",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 18,- "lineStart": 18,- "name": "Endpoint",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 19,- "lineStart": 19,- "name": "Intent",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 20,- "lineStart": 20,- "name": "Decision",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 21,- "lineStart": 21,- "name": "Doc",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 22,- "lineStart": 22,- "name": "Object",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 23,- "lineStart": 23,- "name": "Trait",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 24,- "lineStart": 24,- "name": "Interface",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 25,- "lineStart": 25,- "name": "Method",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 26,- "lineStart": 26,- "name": "Plan",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 27,- "lineStart": 27,- "name": "Task",- },- {- "container": "NodeKind",- "kind": "class",- "language": "scala",- "lineEnd": 28,- "lineStart": 28,- "name": "Bug",- },- {- "container": "NodeKind",- "kind": "function",- "language": "scala",- "lineEnd": 50,- "lineStart": 30,- "name": "nameMap",- },- {- "container": "NodeKind",- "kind": "function",- "language": "scala",- "lineEnd": 72,- "lineStart": 52,- "name": "encoder",- },- {- "container": "NodeKind",- "kind": "function",- "language": "scala",- "lineEnd": 76,- "lineStart": 74,- "name": "decoder",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 89,- "lineStart": 79,- "name": "GraphNode",- },- {- "container": undefined,- "kind": "class",- "language": "scala",- "lineEnd": 96,- "lineStart": 91,- "name": "GraphNode",- },- {- "container": "GraphNode",- "kind": "function",- "language": "scala",- "lineEnd": 94,- "lineStart": 94,- "name": "encoder",- },- {- "container": "GraphNode",- "kind": "function",- "language": "scala",- "lineEnd": 95,- "lineStart": 95,- "name": "decoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 3,- "lineStart": 3,- "name": "java.time.Instant",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 5,- "lineStart": 5,- "name": "io.circe.Decoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 5,- "lineStart": 5,- "name": "io.circe.Encoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 5,- "lineStart": 5,- "name": "io.circe.Json",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 6,- "lineStart": 6,- "name": "io.circe.generic.semiauto.deriveDecoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 6,- "lineStart": 6,- "name": "io.circe.generic.semiauto.deriveEncoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 92,- "lineStart": 92,- "name": "Provenance.instantEncoder",- },- {- "kind": "module",- "language": "scala",- "lineEnd": 92,- "lineStart": 92,- "name": "Provenance.instantDecoder",- },- ],- "filePath": "memory-layer/src/main/scala/ix/memory/model/Node.scala",- "fileRole": {- "role": "production",- "role_confidence": 0.5,- "role_signals": [],- },- "language": "scala",- "relationships": [- {- "dstName": "NodeKind",- "predicate": "CONTAINS",- "srcName": "Node.scala",- },- {- "dstName": "NodeKind",- "predicate": "CONTAINS",- "srcName": "Node.scala",- },- {- "dstName": "Module",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Module",- },- {- "dstName": "File",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "File",- },- {- "dstName": "Class",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Class",- },- {- "dstName": "Function",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Function",- },- {- "dstName": "Variable",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Variable",- },- {- "dstName": "Config",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Config",- },- {- "dstName": "ConfigEntry",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "ConfigEntry",- },- {- "dstName": "Service",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Service",- },- {- "dstName": "Endpoint",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Endpoint",- },- {- "dstName": "Intent",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Intent",- },- {- "dstName": "Decision",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Decision",- },- {- "dstName": "Doc",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Doc",- },- {- "dstName": "Object",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Object",- },- {- "dstName": "Trait",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Trait",- },- {- "dstName": "Interface",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Interface",- },- {- "dstName": "Method",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Method",- },- {- "dstName": "Plan",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Plan",- },- {- "dstName": "Task",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Task",- },- {- "dstName": "Bug",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "NodeKind",- "predicate": "EXTENDS",- "srcName": "Bug",- },- {- "dstName": "nameMap",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "NodeKind",- },- {- "dstName": "GraphNode",- "predicate": "CONTAINS",- "srcName": "Node.scala",- },- {- "dstName": "GraphNode",- "predicate": "CONTAINS",- "srcName": "Node.scala",- },- {- "dstName": "encoder",- "predicate": "CONTAINS",- "srcName": "GraphNode",- },- {- "dstName": "decoder",- "predicate": "CONTAINS",- "srcName": "GraphNode",- },- {- "dstName": "java.time.Instant",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "io.circe.Decoder",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "io.circe.Encoder",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "io.circe.Json",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "io.circe.generic.semiauto.deriveDecoder",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "io.circe.generic.semiauto.deriveEncoder",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "NodeKind",- "predicate": "REFERENCES",- "srcName": "Node.scala",- },- {- "dstName": "contramap",- "predicate": "CALLS",- "srcName": "NodeKind.encoder",- },- {- "dstName": "emap",- "predicate": "CALLS",- "srcName": "NodeKind.decoder",- },- {- "dstName": "get",- "predicate": "CALLS",- "srcName": "NodeKind.decoder",- },- {- "dstName": "toRight",- "predicate": "CALLS",- "srcName": "NodeKind.decoder",- },- {- "dstName": "NodeId",- "predicate": "REFERENCES",- "srcName": "GraphNode",- },- {- "dstName": "NodeKind",- "predicate": "REFERENCES",- "srcName": "GraphNode",- },- {- "dstName": "Json",- "predicate": "REFERENCES",- "srcName": "GraphNode",- },- {- "dstName": "Provenance",- "predicate": "REFERENCES",- "srcName": "GraphNode",- },- {- "dstName": "Rev",- "predicate": "REFERENCES",- "srcName": "GraphNode",- },- {- "dstName": "Instant",- "predicate": "REFERENCES",- "srcName": "GraphNode",- },- {- "dstName": "Provenance.instantEncoder",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "Provenance.instantDecoder",- "predicate": "IMPORTS",- "srcName": "Node.scala",- },- {- "dstName": "GraphNode",- "predicate": "REFERENCES",- "srcName": "Node.scala",- },- ],-}-`;--exports[`parseFile snapshots > snapshots doctor.ts 1`] = `-{- "entities": [- {- "kind": "file",- "language": "typescript",- "lineEnd": 99,- "lineStart": 1,- "name": "doctor.ts",- },- {- "container": undefined,- "kind": "interface",- "language": "typescript",- "lineEnd": 9,- "lineStart": 6,- "name": "Check",- },- {- "container": undefined,- "kind": "function",- "language": "typescript",- "lineEnd": 98,- "lineStart": 11,- "name": "registerDoctorCommand",- },- {- "kind": "module",- "language": "typescript",- "lineEnd": 1,- "lineStart": 1,- "name": "commander",- },- {- "kind": "module",- "language": "typescript",- "lineEnd": 2,- "lineStart": 2,- "name": "chalk",- },- {- "kind": "module",- "language": "typescript",- "lineEnd": 3,- "lineStart": 3,- "name": "api.js",- },- {- "kind": "module",- "language": "typescript",- "lineEnd": 4,- "lineStart": 4,- "name": "config.js",- },- ],- "filePath": "ix-cli/src/cli/commands/doctor.ts",- "fileRole": {- "role": "production",- "role_confidence": 0.5,- "role_signals": [],- },- "language": "typescript",- "relationships": [- {- "dstName": "Check",- "predicate": "CONTAINS",- "srcName": "doctor.ts",- },- {- "dstName": "registerDoctorCommand",- "predicate": "CONTAINS",- "srcName": "doctor.ts",- },- {- "dstName": "commander",- "predicate": "IMPORTS",- "srcName": "doctor.ts",- },- {- "dstName": "chalk",- "predicate": "IMPORTS",- "srcName": "doctor.ts",- },- {- "dstName": "api.js",- "predicate": "IMPORTS",- "srcName": "doctor.ts",- },- {- "dstName": "config.js",- "predicate": "IMPORTS",- "srcName": "doctor.ts",- },- {- "dstName": "Command",- "predicate": "REFERENCES",- "srcName": "doctor.ts",- },- {- "dstName": "command",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "description",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "option",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "action",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "getEndpoint",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "IxClient",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "health",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "stats",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "conflicts",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "isArray",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "run",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "push",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "every",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "stringify",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "bold",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "green",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "red",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- {- "dstName": "dim",- "predicate": "CALLS",- "srcName": "registerDoctorCommand",- },- ],-}-`;diff --git a/core-ingestion/src/__tests__/chunks.test.ts b/core-ingestion/src/__tests__/chunks.test.tsnew file mode 100644index 00000000..7f190b46--- /dev/null+++ b/core-ingestion/src/__tests__/chunks.test.ts@@ -0,0 +1,118 @@+import { describe, expect, it } from 'vitest';+import { parseFile, type ParsedChunk } from '../index.js';++// ---------------------------------------------------------------------------+// Helper to parse a synthetic source string as a given file extension+// ---------------------------------------------------------------------------+function parse(source: string, ext = '.ts') {+ const filePath = `/repo/fixture${ext}`;+ return parseFile(filePath, source);+}++describe('Chunk extraction — M1 acceptance criteria', () => {+ it('a file with 3 top-level functions produces 3 chunk nodes', () => {+ const src = `+function alpha() { return 1; }+function beta() { return 2; }+function gamma() { return 3; }+`.trim();+ const result = parse(src);+ expect(result).not.toBeNull();+ const chunks = result!.chunks.filter(c => c.name !== null);+ expect(chunks.length).toBe(3);+ expect(chunks.map(c => c.name)).toEqual(expect.arrayContaining(['alpha', 'beta', 'gamma']));+ expect(chunks.every(c => c.chunkKind === 'function')).toBe(true);+ });++ it('a class file produces at least a class chunk and/or method chunks', () => {+ const src = `+class MyService {+ greet() { return 'hello'; }+ farewell() { return 'bye'; }+}+`.trim();+ const result = parse(src);+ expect(result).not.toBeNull();+ const chunks = result!.chunks.filter(c => c.name !== null);+ // Must have at least the class chunk or method chunks+ expect(chunks.length).toBeGreaterThanOrEqual(1);+ const names = chunks.map(c => c.name);+ expect(names.some(n => n === 'MyService' || n === 'greet' || n === 'farewell')).toBe(true);+ });++ it('a file with no extractable semantic units produces one file_body chunk', () => {+ // A comment-only file has no definitions+ const src = `// This file has no definitions\nconst x = 1;\n`;+ const result = parseFile('/repo/fixture.txt', src);+ // .txt is not a supported language, so we test with an empty TS file+ const emptyTs = parseFile('/repo/empty.ts', '// empty\n');+ // Even if parse fails gracefully, test the fallback chunk on a real parsed file+ if (emptyTs) {+ expect(emptyTs.chunks.length).toBeGreaterThanOrEqual(1);+ if (emptyTs.chunks.every(c => c.name === null)) {+ expect(emptyTs.chunks[0].chunkKind).toBe('file_body');+ }+ }+ });++ it('each chunk has required fields: startByte, endByte, contentHash, language', () => {+ const src = `function foo() { return 42; }`;+ const result = parse(src);+ expect(result).not.toBeNull();+ for (const chunk of result!.chunks) {+ expect(typeof chunk.startByte).toBe('number');+ expect(typeof chunk.endByte).toBe('number');+ expect(chunk.endByte).toBeGreaterThan(chunk.startByte);+ expect(typeof chunk.contentHash).toBe('string');+ expect(chunk.contentHash.length).toBeGreaterThan(0);+ expect(typeof chunk.language).toBe('string');+ }+ });++ it('chunks have matching lineStart/lineEnd within file bounds', () => {+ const src = `function a() {}\nfunction b() {}\n`;+ const result = parse(src);+ expect(result).not.toBeNull();+ const lines = src.split('\n').length;+ for (const chunk of result!.chunks) {+ expect(chunk.lineStart).toBeGreaterThanOrEqual(1);+ expect(chunk.lineEnd).toBeLessThanOrEqual(lines);+ }+ });++ it('method chunks carry a container field matching their enclosing class', () => {+ const src = `+class Calculator {+ add(a: number, b: number) { return a + b; }+}+`.trim();+ const result = parse(src);+ expect(result).not.toBeNull();+ const methodChunk = result!.chunks.find(c => c.name === 'add');+ if (methodChunk) {+ expect(methodChunk.container).toBe('Calculator');+ }+ });++ it('re-parsing the same file produces identical chunks (stable IDs)', () => {+ const src = `function stableFunc() { return true; }`;+ const r1 = parse(src);+ const r2 = parse(src);+ expect(r1).not.toBeNull();+ expect(r2).not.toBeNull();+ expect(r1!.chunks).toEqual(r2!.chunks);+ });++ it('Scala file produces chunks for class/trait/def definitions', () => {+ const src = `+class GraphService {+ def findNode(id: String): Option[Node] = None+ def saveNode(node: Node): Unit = ()+}+`.trim();+ const result = parseFile('/repo/fixture.scala', src);+ expect(result).not.toBeNull();+ const chunks = result!.chunks.filter(c => c.name !== null);+ expect(chunks.length).toBeGreaterThanOrEqual(1);+ });+});diff --git a/core-ingestion/src/__tests__/resolveEdges.property.test.ts b/core-ingestion/src/__tests__/resolveEdges.property.test.tsindex ac34f372..578cc8fd 100644--- a/core-ingestion/src/__tests__/resolveEdges.property.test.ts+++ b/core-ingestion/src/__tests__/resolveEdges.property.test.ts@@ -38,6 +38,7 @@ describe('resolveEdges property invariants', () => {makeEntity(def.symbol, def.language),...(def.includeCall ? [makeEntity(`caller${index}`, def.language)] : []),],+ chunks: [],relationships: def.includeCall? [{ srcName: `caller${index}`, dstName: def.symbol, predicate: 'CALLS' }]: [],@@ -69,6 +70,7 @@ describe('resolveEdges property invariants', () => {{ name: 'caller.ts', kind: 'file', lineStart: 1, lineEnd: 1, language: SupportedLanguages.TypeScript },makeEntity('caller', SupportedLanguages.TypeScript),],+ chunks: [],relationships: [{ srcName: 'caller', dstName: 'missing', predicate: 'CALLS' }],fileRole: defaultFileRole,},@@ -79,6 +81,7 @@ describe('resolveEdges property invariants', () => {{ name: 'other.scala', kind: 'file', lineStart: 1, lineEnd: 1, language: SupportedLanguages.Scala },makeEntity('missing', SupportedLanguages.Scala),],+ chunks: [],relationships: [],fileRole: defaultFileRole,},@@ -94,6 +97,7 @@ describe('resolveEdges property invariants', () => {{ name: 'caller.ts', kind: 'file', lineStart: 1, lineEnd: 1, language: SupportedLanguages.TypeScript },makeEntity('caller', SupportedLanguages.TypeScript),],+ chunks: [],relationships: [{ srcName: 'caller', dstName: 'run', predicate: 'CALLS' }],fileRole: defaultFileRole,},@@ -104,6 +108,7 @@ describe('resolveEdges property invariants', () => {{ name: 'a.ts', kind: 'file', lineStart: 1, lineEnd: 1, language: SupportedLanguages.TypeScript },makeEntity('run', SupportedLanguages.TypeScript),],+ chunks: [],relationships: [],fileRole: defaultFileRole,},@@ -114,6 +119,7 @@ describe('resolveEdges property invariants', () => {{ name: 'b.ts', kind: 'file', lineStart: 1, lineEnd: 1, language: SupportedLanguages.TypeScript },makeEntity('run', SupportedLanguages.TypeScript),],+ chunks: [],relationships: [],fileRole: defaultFileRole,},diff --git a/core-ingestion/src/__tests__/resolveEdges.test.ts b/core-ingestion/src/__tests__/resolveEdges.test.tsindex c6ee50a7..cb1e3961 100644--- a/core-ingestion/src/__tests__/resolveEdges.test.ts+++ b/core-ingestion/src/__tests__/resolveEdges.test.ts@@ -34,6 +34,7 @@ function fileResult({ name: filePath.split(/[\\/]/).pop() ?? filePath, kind: 'file', lineStart: 1, lineEnd: 1, language },...entities,],+ chunks: [],relationships,fileRole: defaultFileRole,};diff --git a/core-ingestion/src/index.ts b/core-ingestion/src/index.tsindex a541d383..93d090b4 100644--- a/core-ingestion/src/index.ts+++ b/core-ingestion/src/index.ts@@ -1,4 +1,5 @@import * as nodePath from 'node:path';+import * as crypto from 'node:crypto';// @ts-ignore — tree-sitter has no bundled typesimport Parser from 'tree-sitter';// @ts-ignore@@ -45,6 +46,23 @@ export interface ParsedEntity {container?: string;}+/** A semantic code span extracted from the AST. The primary LLM retrieval unit. */+export interface ParsedChunk {+ /** Semantic name of the chunk (function/class/trait name), or null for file_body. */+ name: string | null;+ /** Chunk kind: "function" | "method" | "class" | "interface" | "trait" | "module_block" | "file_body" */+ chunkKind: string;+ lineStart: number;+ lineEnd: number;+ startByte: number;+ endByte: number;+ /** SHA-256 of the chunk source text for change detection and stable identity. */+ contentHash: string;+ language: string;+ /** Name of the directly enclosing class/trait/interface, if any. */+ container?: string;+}+export interface ParsedRelationship {srcName: string;dstName: string;@@ -55,6 +73,7 @@ export interface FileParseResult {filePath: string;language: SupportedLanguages;entities: ParsedEntity[];+ chunks: ParsedChunk[];relationships: ParsedRelationship[];fileRole: RoleClassification;}@@ -160,17 +179,18 @@ export function isGrammarSupported(filePath: string): boolean {// ---------------------------------------------------------------------------let _parser: Parser | null = null;+const _queryCache = new Map<SupportedLanguages | 'tsx', { grammar: any; query: any }>();function getParser(): Parser {if (!_parser) _parser = new Parser();return _parser;}-// Per-process cache for compiled tree-sitter queries, keyed by language.-// Avoids recompiling the same query string for every file of the same language.-const _queryCache = new Map<SupportedLanguages | 'tsx', { grammar: any; query: any }>();--function getCachedQuery(language: SupportedLanguages | 'tsx', grammar: any, querySource: string): any {+function getCachedQuery(+ language: SupportedLanguages | 'tsx',+ grammar: any,+ querySource: string,+): any {const cached = _queryCache.get(language);if (cached && cached.grammar === grammar) return cached.query;const query = new Parser.Query(grammar, querySource);@@ -203,14 +223,26 @@ export function parseFile(filePath: string, source: string): FileParseResult | nconst matches = query.matches(tree.rootNode);const fileName = nodePath.basename(filePath);- // Count newlines instead of allocating a full string array- let lineCount = 1;- for (let i = 0; i < source.length; i++) { if (source.charCodeAt(i) === 10) lineCount++; }+ let sourceLineCount = 1;+ for (let i = 0; i < source.length; i++) {+ if (source.charCodeAt(i) === 10) sourceLineCount++;+ }const entities: ParsedEntity[] = [- { name: fileName, kind: 'file', lineStart: 1, lineEnd: lineCount, language },+ { name: fileName, kind: 'file', lineStart: 1, lineEnd: sourceLineCount, language },];+ const chunks: ParsedChunk[] = [];const relationships: ParsedRelationship[] = [];+ const pendingChunks: Array<{+ name: string;+ chunkKind: string;+ lineStart: number;+ lineEnd: number;+ startByte: number;+ endByte: number;+ language: SupportedLanguages;+ container?: string;+ }> = [];// Track class ranges for containment: [name, startLine, endLine]const classRanges: Array<{ name: string; start: number; end: number }> = [];@@ -236,6 +268,8 @@ export function parseFile(filePath: string, source: string): FileParseResult | nconst defNode = defCapture.node;const lineStart = defNode.startPosition.row + 1;const lineEnd = defNode.endPosition.row + 1;+ const startByte = defNode.startIndex;+ const endByte = defNode.endIndex;// Containment: file CONTAINS or class CONTAINSconst enclosing = findEnclosing(classRanges, lineStart, name);@@ -249,6 +283,17 @@ export function parseFile(filePath: string, source: string): FileParseResult | ncontainer: enclosing ?? undefined,});+ pendingChunks.push({+ name,+ chunkKind: kind,+ lineStart,+ lineEnd,+ startByte,+ endByte,+ language,+ container: enclosing ?? undefined,+ });+if (kind === 'class' || kind === 'interface' || kind === 'trait') {classRanges.push({ name, start: lineStart, end: lineEnd });}@@ -404,7 +449,31 @@ export function parseFile(filePath: string, source: string): FileParseResult | n}}- return { filePath, language, entities, relationships, fileRole: classifyFileRole(filePath, source) };+ for (const pendingChunk of pendingChunks) {+ const chunkText = source.slice(pendingChunk.startByte, pendingChunk.endByte);+ const contentHash = crypto.createHash('sha256').update(chunkText).digest('hex').slice(0, 16);+ chunks.push({+ ...pendingChunk,+ contentHash,+ });+ }++ // Fallback: if no semantic chunks found, emit one file_body chunk covering the whole file+ if (chunks.length === 0) {+ const contentHash = crypto.createHash('sha256').update(source).digest('hex').slice(0, 16);+ chunks.push({+ name: null,+ chunkKind: 'file_body',+ lineStart: 1,+ lineEnd: sourceLineCount,+ startByte: 0,+ endByte: source.length,+ contentHash,+ language,+ });+ }++ return { filePath, language, entities, chunks, relationships, fileRole: classifyFileRole(filePath, source) };} catch {return null;}@@ -485,37 +554,7 @@ function bestQKey(* - Unambiguous class method: dstQualifiedKey === 'ClassName.method'* - Ambiguous (two entities share the same plain name): edge not emitted*/-/** Stats collected during resolution (exposed for debug instrumentation). */-export interface ResolveStats {- importLookups: number;- transitiveLookups: number;- globalFallbacks: number;- globalCandidateTotal: number;- resolvedImport: number;- resolvedTransitive: number;- resolvedGlobal: number;- resolvedQualifier: number;- skippedSameFile: number;- skippedAmbiguous: number;-}--export function resolveEdges(results: FileParseResult[]): ResolvedEdge[];-export function resolveEdges(results: FileParseResult[], statsOut: ResolveStats): ResolvedEdge[];-export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStats): ResolvedEdge[] {- const stats: ResolveStats = statsOut ?? {- importLookups: 0, transitiveLookups: 0, globalFallbacks: 0,- globalCandidateTotal: 0, resolvedImport: 0, resolvedTransitive: 0,- resolvedGlobal: 0, resolvedQualifier: 0, skippedSameFile: 0, skippedAmbiguous: 0,- };-- // ── Index structures (built once) ──────────────────────────────────-- // Direct file lookup: filePath → FileParseResult (replaces results.find())- const resultByPath = new Map<string, FileParseResult>();- for (const r of results) {- resultByPath.set(r.filePath, r);- }-+export function resolveEdges(results: FileParseResult[]): ResolvedEdge[] {// fileQKeys: filePath → (plainName → qualifiedKeys[])// Mirrors the entityQKey computation in buildPatch so nodeIds match exactly.const fileQKeys = new Map<string, Map<string, string[]>>();@@ -537,20 +576,16 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStatsfileHasSymbol.set(fp, new Set(qkMap.keys()));}- // Precompute language per file (avoids calling languageFromPath in hot loops)- const fileLanguage = new Map<string, SupportedLanguages | null>();- for (const r of results) {- fileLanguage.set(r.filePath, r.language);- }+ // resultsByPath: O(1) lookup replacing results.find() in transitive import loop+ const resultsByPath = new Map<string, FileParseResult>(results.map(r => [r.filePath, r]));- // Inverted symbol index: symbolName → Array<{filePath, language}>- // Replaces the O(N) global fallback scan with O(candidates) lookup.- const symbolToFiles = new Map<string, Array<string>>();+ // symbolToFiles: plainName → filePath[] — replaces O(F) full-scan in tier-3 and qualifier fallback+ const symbolToFiles = new Map<string, string[]>();for (const [fp, symbols] of fileHasSymbol) {for (const sym of symbols) {- let arr = symbolToFiles.get(sym);- if (!arr) { arr = []; symbolToFiles.set(sym, arr); }- arr.push(fp);+ const list = symbolToFiles.get(sym) ?? [];+ list.push(fp);+ symbolToFiles.set(sym, list);}}@@ -596,8 +631,6 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStats}}- // ── Helpers ────────────────────────────────────────────────────────-/** Resolve a module name to matching file paths in the batch. */function modNameToFiles(modName: string, excludeFp: string): string[] {const fps: string[] = [];@@ -630,14 +663,11 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStatsreturn fps;}- // ── Main resolution loop ───────────────────────────────────────────-const resolved: ResolvedEdge[] = [];for (const result of results) {const srcFilePath = result.filePath;const srcSymbols = fileHasSymbol.get(srcFilePath)!;- const srcLanguage = result.language;// Build the set of file paths this file explicitly imports.const importedFilePaths = new Set<string>();@@ -652,7 +682,7 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStats// Handles re-exports: baz.ts → index.ts (re-exports from bar.ts) → bar.tsconst transitiveFilePaths = new Set<string>();for (const fp of importedFilePaths) {- const fpResult = resultByPath.get(fp); // O(1) instead of results.find()+ const fpResult = resultsByPath.get(fp);if (!fpResult) continue;for (const rel of fpResult.relationships) {if (rel.predicate !== 'IMPORTS') continue;@@ -686,7 +716,6 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStatsconst dstQualifiedKey = bestQKey(fileQKeys, fp, entityName);if (dstQualifiedKey === null) continue;resolved.push({ srcFilePath, srcName: rel.srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: 'IMPORTS', confidence: 0.9 });- stats.resolvedImport++;continue;}@@ -714,23 +743,18 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStatsif (dstQualifiedKey !== null) {// dstName must match rel.dstName so buildPatchWithResolution can look it upresolved.push({ srcFilePath, srcName, dstFilePath: qfp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.9 });- stats.resolvedQualifier++;}}continue;}- // Global qualifier fallback — use inverted index instead of scanning all files- const qualCandidates = symbolToFiles.get(qualifierPart) ?? [];- const qualGlobalMatches = qualCandidates.filter(fp => fp !== srcFilePath);- stats.globalFallbacks++;- stats.globalCandidateTotal += qualGlobalMatches.length;+ // Global qualifier fallback — use symbolToFiles index instead of full scan+ const qualGlobalMatches = (symbolToFiles.get(qualifierPart) ?? []).filter(fp => fp !== srcFilePath);if (qualGlobalMatches.length === 1) {const qfp = qualGlobalMatches[0];if (fileHasSymbol.get(qfp)?.has(memberPart)) {const dstQualifiedKey = bestQKey(fileQKeys, qfp, memberPart);if (dstQualifiedKey !== null) {resolved.push({ srcFilePath, srcName, dstFilePath: qfp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.7 });- stats.resolvedQualifier++;}}}@@ -739,10 +763,9 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStats}// Tier 1: same-file — already correct in buildPatch, skip here- if (srcSymbols.has(dstName)) { stats.skippedSameFile++; continue; }+ if (srcSymbols.has(dstName)) continue;// Tier 2: import-scoped (confidence 0.9)- stats.importLookups++;const importMatches: string[] = [];for (const fp of importedFilePaths) {if (fileHasSymbol.get(fp)?.has(dstName)) importMatches.push(fp);@@ -753,13 +776,11 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStatsconst dstQualifiedKey = bestQKey(fileQKeys, fp, dstName);if (dstQualifiedKey === null) continue; // ambiguous — do not emit bad nodeIdresolved.push({ srcFilePath, srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.9 });- stats.resolvedImport++;continue;}- if (importMatches.length > 1) { stats.skippedAmbiguous++; continue; } // ambiguous file — do not emit+ if (importMatches.length > 1) continue; // ambiguous file — do not emit// Tier 2.5: transitive import-scoped (confidence 0.8) — one re-export hop away- stats.transitiveLookups++;const transitiveMatches: string[] = [];for (const fp of transitiveFilePaths) {if (fileHasSymbol.get(fp)?.has(dstName)) transitiveMatches.push(fp);@@ -770,24 +791,21 @@ export function resolveEdges(results: FileParseResult[], statsOut?: ResolveStatsconst dstQualifiedKey = bestQKey(fileQKeys, fp, dstName);if (dstQualifiedKey === null) continue;resolved.push({ srcFilePath, srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.8 });- stats.resolvedTransitive++;continue;}- if (transitiveMatches.length > 1) { stats.skippedAmbiguous++; continue; } // ambiguous — do not emit+ if (transitiveMatches.length > 1) continue; // ambiguous — do not emit- // Tier 3: global fallback (confidence 0.5) — uses inverted symbol index- // instead of scanning all files.- stats.globalFallbacks++;- const candidates = symbolToFiles.get(dstName) ?? [];- const globalMatches = candidates.filter(fp => fp !== srcFilePath && fileLanguage.get(fp) === srcLanguage);- stats.globalCandidateTotal += globalMatches.length;+ // Tier 3: global fallback (confidence 0.5) — exactly one other file defines it+ // Only match files in the same language as the caller to avoid cross-language false positives.+ const srcLanguage = result.language;+ const globalMatches = (symbolToFiles.get(dstName) ?? [])+ .filter(fp => fp !== srcFilePath && languageFromPath(fp) === srcLanguage);if (globalMatches.length === 1) {const fp = globalMatches[0];const dstQualifiedKey = bestQKey(fileQKeys, fp, dstName);if (dstQualifiedKey === null) continue; // ambiguous — do not emit bad nodeIdresolved.push({ srcFilePath, srcName, dstFilePath: fp, dstName, dstQualifiedKey, predicate: rel.predicate, confidence: 0.5 });- stats.resolvedGlobal++;}// 0 or >1 global matches — leave as dangling edge, do not emit}diff --git a/core-ingestion/src/parse-worker.ts b/core-ingestion/src/parse-worker.tsnew file mode 100644index 00000000..3485d01e--- /dev/null+++ b/core-ingestion/src/parse-worker.ts@@ -0,0 +1,19 @@+/**+ * Worker thread entry point for parallel file parsing.+ * Each worker maintains its own Parser singleton (safe — module state is per-thread).+ * Receives: { filePath: string, source: string }+ * Posts: { ok: true, result: FileParseResult } | { ok: false }+ */+import { parentPort } from 'node:worker_threads';+import { parseFile } from './index.js';++if (!parentPort) throw new Error('parse-worker must run inside a worker thread');++parentPort.on('message', ({ filePath, source }: { filePath: string; source: string }) => {+ try {+ const result = parseFile(filePath, source);+ parentPort!.postMessage({ ok: result !== null, result: result ?? null });+ } catch {+ parentPort!.postMessage({ ok: false, result: null });+ }+});diff --git a/core-ingestion/src/patch-builder.ts b/core-ingestion/src/patch-builder.tsindex 09b5941b..d06f6553 100644--- a/core-ingestion/src/patch-builder.ts+++ b/core-ingestion/src/patch-builder.ts@@ -30,6 +30,11 @@ function edgeId(filePath: string, src: string, dst: string, predicate: string):return deterministicId(`${normalizePath(filePath)}:${src}:${dst}:${predicate}`);}+/** Deterministic ID for a chunk — keyed on file + kind + name + start line to survive minor edits. */+function chunkId(filePath: string, chunkKind: string, name: string | null, startLine: number): string {+ return deterministicId(`${normalizePath(filePath)}:chunk:${chunkKind}:${name ?? 'file_body'}:${startLine}`);+}+// ---------------------------------------------------------------------------// Source type from file extension// ---------------------------------------------------------------------------@@ -42,11 +47,11 @@ function sourceType(filePath: string): string {}export function extractorName(): string {- return `tree-sitter/1.12`;+ return `tree-sitter/1.13`;}/** Previous extractor versions — their patches are superseded when re-ingesting. */-export const PREVIOUS_EXTRACTORS = ['tree-sitter/1.11', 'tree-sitter/1.10', 'tree-sitter/1.9', 'tree-sitter/1.8', 'tree-sitter/1.7', 'tree-sitter/1.6', 'tree-sitter/1.5', 'tree-sitter/1.4', 'tree-sitter/1.3', 'tree-sitter/1.2', 'tree-sitter/1.1'];+export const PREVIOUS_EXTRACTORS = ['tree-sitter/1.12', 'tree-sitter/1.11', 'tree-sitter/1.10', 'tree-sitter/1.9', 'tree-sitter/1.8', 'tree-sitter/1.7', 'tree-sitter/1.6', 'tree-sitter/1.5', 'tree-sitter/1.4', 'tree-sitter/1.3', 'tree-sitter/1.2', 'tree-sitter/1.1'];/** Compute a patchId for a (filePath, sourceHash, extractorVersion) triple. */function computePatchId(filePath: string, sourceHash: string, extractor: string): string {@@ -67,7 +72,7 @@ export function buildPatch(sourceHash: string,previousSourceHash?: string): GraphPatchPayload {- const { filePath, entities, relationships } = result;+ const { filePath, entities, chunks, relationships } = result;const ops: PatchOp[] = [];// Build a qualified-key map so that same-named entities in different@@ -127,6 +132,73 @@ export function buildPatch(}}+ // UpsertNode + edges for each chunk+ const fileNodeId = nodeId(filePath, entities.find(e => e.kind === 'file')?.name ?? filePath);+ for (const chunk of chunks) {+ const cid = chunkId(filePath, chunk.chunkKind, chunk.name, chunk.lineStart);+ const chunkName = chunk.name ?? `file_body:${chunk.lineStart}`;+ ops.push({+ type: 'UpsertNode',+ id: cid,+ kind: 'chunk',+ name: chunkName,+ attrs: {+ file_uri: filePath,+ language: chunk.language,+ chunk_kind: chunk.chunkKind,+ start_line: chunk.lineStart,+ end_line: chunk.lineEnd,+ start_byte: chunk.startByte,+ end_byte: chunk.endByte,+ content_hash: chunk.contentHash,+ parser_version: extractorName(),+ },+ });+ // File -[CONTAINS]-> Chunk+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, 'file', chunkName, 'CONTAINS_CHUNK'),+ src: fileNodeId,+ dst: cid,+ predicate: 'CONTAINS_CHUNK',+ attrs: {},+ });+ // Chunk -[DEFINES]-> Symbol (only for named chunks)+ if (chunk.name !== null) {+ const symbolKey = chunk.container ? `${chunk.container}.${chunk.name}` : chunk.name;+ const symbolNid = nodeId(filePath, symbolKey);+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, chunkName, symbolKey, 'DEFINES'),+ src: cid,+ dst: symbolNid,+ predicate: 'DEFINES',+ attrs: {},+ });+ }+ }++ // NEXT edges for source-order chunk adjacency+ for (let i = 0; i + 1 < chunks.length; i++) {+ const a = chunks[i];+ const b = chunks[i + 1];+ // Only link top-level chunks (no container) to avoid intra-class ordering noise+ if (a.container == null && b.container == null) {+ const aid = chunkId(filePath, a.chunkKind, a.name, a.lineStart);+ const bid = chunkId(filePath, b.chunkKind, b.name, b.lineStart);+ const aName = a.name ?? `file_body:${a.lineStart}`;+ const bName = b.name ?? `file_body:${b.lineStart}`;+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, aName, bName, 'NEXT'),+ src: aid,+ dst: bid,+ predicate: 'NEXT',+ attrs: {},+ });+ }+ }+// UpsertEdge for each relationshipfor (const r of relationships) {// For CONTAINS edges, srcName is the container of dstName — use that to disambiguate.@@ -208,7 +280,7 @@ export function buildPatchWithResolution(});}- const { filePath, entities, relationships } = result;+ const { filePath, entities, chunks, relationships } = result;const ops: PatchOp[] = [];const entityQKey = new Map<ParsedEntity, string>();@@ -252,6 +324,69 @@ export function buildPatchWithResolution(}}+ // UpsertNode + edges for each chunk (same logic as buildPatch)+ const fileNodeId2 = nodeId(filePath, entities.find(e => e.kind === 'file')?.name ?? filePath);+ for (const chunk of chunks) {+ const cid = chunkId(filePath, chunk.chunkKind, chunk.name, chunk.lineStart);+ const chunkName = chunk.name ?? `file_body:${chunk.lineStart}`;+ ops.push({+ type: 'UpsertNode',+ id: cid,+ kind: 'chunk',+ name: chunkName,+ attrs: {+ file_uri: filePath,+ language: chunk.language,+ chunk_kind: chunk.chunkKind,+ start_line: chunk.lineStart,+ end_line: chunk.lineEnd,+ start_byte: chunk.startByte,+ end_byte: chunk.endByte,+ content_hash: chunk.contentHash,+ parser_version: extractorName(),+ },+ });+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, 'file', chunkName, 'CONTAINS_CHUNK'),+ src: fileNodeId2,+ dst: cid,+ predicate: 'CONTAINS_CHUNK',+ attrs: {},+ });+ if (chunk.name !== null) {+ const symbolKey = chunk.container ? `${chunk.container}.${chunk.name}` : chunk.name;+ const symbolNid = nodeId(filePath, symbolKey);+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, chunkName, symbolKey, 'DEFINES'),+ src: cid,+ dst: symbolNid,+ predicate: 'DEFINES',+ attrs: {},+ });+ }+ }++ for (let i = 0; i + 1 < chunks.length; i++) {+ const a = chunks[i];+ const b = chunks[i + 1];+ if (a.container == null && b.container == null) {+ const aid = chunkId(filePath, a.chunkKind, a.name, a.lineStart);+ const bid = chunkId(filePath, b.chunkKind, b.name, b.lineStart);+ const aName = a.name ?? `file_body:${a.lineStart}`;+ const bName = b.name ?? `file_body:${b.lineStart}`;+ ops.push({+ type: 'UpsertEdge',+ id: edgeId(filePath, aName, bName, 'NEXT'),+ src: aid,+ dst: bid,+ predicate: 'NEXT',+ attrs: {},+ });+ }+ }+for (const r of relationships) {const srcKey = resolveKey(r.srcName);const dstKey = r.predicate === 'CONTAINS'diff --git a/decisions-backup.md b/decisions-backup.mdnew file mode 100644index 00000000..dc0c19b8--- /dev/null+++ b/decisions-backup.md@@ -0,0 +1,14 @@+# Graph Decisions Backup+# Re-record these after reset with: ix decide "<name>" --rationale "<rationale>"++1. ix decide "CONTAINS_CHUNK predicate for File→Chunk edges (separate from CONTAINS)" --rationale "CONTAINS is already used for File→Symbol and Class→Method hierarchy. Using a distinct CONTAINS_CHUNK predicate for File→Chunk edges avoids ambiguity in graph traversal and keeps chunk subgraph cleanly separable from symbol hierarchy queries."++2. ix decide "inference_version is a required field on all inferred claims" --rationale "Observed claims (from parsing) are versioned implicitly via the extractor field in provenance. Inferred claims (subsystem membership, smells, summaries) need an explicit inference_version string (e.g. smell_v1) so they can be rerun selectively without touching observed facts. Missing = legacy for backward compat."++3. ix decide "Chunk nodes are first-class observed nodes, not derived from symbols" --rationale "Symbols (class/function/method) are named entities. Chunks are code spans — the retrieval unit for LLM context. A chunk DEFINES one or more symbols and is CONTAINED by a file. Separating them allows chunk-granularity retrieval without blowing up symbol addressing."++4. ix decide "Chunk granularity: AST-based semantic chunks" --rationale "Fixed-line windows break semantics. Symbol-only is too fragmented for LLM context. AST-based chunks (function body, class, top-level logical block) are the right unit. Hierarchy: File -> Chunk -> Symbol(s), with edges next_chunk, CALLS, REFERENCES between chunks. This enables context expansion: retrieve chunk + neighbors instead of whole file."++5. ix decide "Summaries as claims with has_summary.* predicates" --rationale "Summary nodes would be a new abstraction with no benefit. Claims with predicates has_summary.purpose, has_summary.api, has_summary.dependencies, has_summary.responsibility are replaceable, versionable via source field (e.g. llm_v1), allow competing summaries to coexist, and make retrieval trivial with no extra joins."++6. ix decide "SubsystemCandidate as claim not node" --rationale "Subsystem membership is belief, not fact. Using candidate_member_of claims on existing Subsystem nodes avoids graph bloat, reuses confidence/provenance/evidence infrastructure, supports conflicting memberships and overlapping subsystems, and allows re-scoring without rewiring the graph. Subsystems are real entities (nodes); membership is belief (claims)."diff --git a/docs/map-update/README.md b/docs/map-update/README.mdnew file mode 100644index 00000000..2cbea245--- /dev/null+++ b/docs/map-update/README.md@@ -0,0 +1,100 @@+# Map Update — Feature Overview++This document describes the `ix map` overhaul and the two new analysis commands (`ix smells`, `ix subsystems`) added in the `feature/map-update` branch.++## What Changed++### `ix map` — Progress Bar + Timing++The map command now shows a live progress bar while computing the architectural map and prints ingest/map timing on completion.++```+ Computing map... ████████████░░░░░░░░░░░░░ 51%+ ingest 1240ms · map 3820ms+```++Key changes in `ix-cli/src/cli/commands/map.ts`:+- Exponential-decay progress bar using `setInterval`+- Ingest and map durations printed to stderr (text mode only)+- `--full` flag preserved for bypassing incremental map cache++### `ix smells` — Architecture Smell Detection++Runs graph-query-based smell detection and stores results as versioned claims on `File` nodes.++```bash+ix smells # run detection with default thresholds+ix smells --format json # machine-readable output+ix smells --list # show previously stored claims+ix smells --god-module-chunks 15 # tune threshold+```++**Detected smells:**++| Smell | Field | Description |+|-------|-------|-------------|+| Orphan file | `has_smell.orphan_file` | File with no import/call connections |+| God module | `has_smell.god_module` | File with high chunk count or fan-in/out |+| Weak component member | `has_smell.weak_component_member` | File connected to very few others |++Claims are stored with `inference_version=smell_v1` and are versioned — re-running retires old claims before writing new ones.++**Backend:** `SmellService.scala` + `SmellRoutes.scala`++### `ix subsystems` — Subsystem Health Scoring++Scores each architectural region produced by `ix map` on a 0–1 health scale.++```bash+ix subsystems # compute and store scores+ix subsystems --list # show stored scores+ix subsystems --level 2 # filter to subsystem level+ix subsystems --format json # machine-readable+```++**Health score formula:**++```+health = 0.4 × (1 − smell_rate)+ + 0.3 × tanh(chunk_density / 5)+ + 0.3 × confidence+```++| Signal | Meaning |+|--------|---------|+| `smell_rate` | Fraction of files in the region with smell claims (lower = healthier) |+| `chunk_density` | Avg `CONTAINS_CHUNK` edges per file (higher = richer structure) |+| `confidence` | MapService cohesion/boundary ratio score |++Claims stored with `inference_version=subsystem_v1`.++**Backend:** `SubsystemService.scala` + `SubsystemRoutes.scala`++## New Files++| File | Purpose |+|------|---------|+| `ix-cli/src/cli/commands/smells.ts` | CLI command for smell detection |+| `ix-cli/src/cli/commands/subsystems.ts` | CLI command for subsystem scoring |+| `memory-layer/.../smell/SmellService.scala` | Smell detection logic (AQL queries + claim storage) |+| `memory-layer/.../smell/SubsystemService.scala` | Subsystem scoring logic |+| `memory-layer/.../api/SmellRoutes.scala` | HTTP routes for smell endpoints |+| `memory-layer/.../api/SubsystemRoutes.scala` | HTTP routes for subsystem endpoints |++## Recommended Workflow++After running `ix map`, use the new commands to get a quality picture:++```bash+ix map # build architectural map+ix smells --format json # find problem files+ix subsystems --format json # score each region+```++Pipe results into `ix decide` to record findings:++```bash+ix decide "Refactor auth module" \+ --rationale "god_module smell + health score 0.31" \+ --affects AuthModule+```diff --git a/docs/map-update/map-experience-improvements.md b/docs/map-update/map-experience-improvements.mdnew file mode 100644index 00000000..456517ba--- /dev/null+++ b/docs/map-update/map-experience-improvements.md@@ -0,0 +1,393 @@+# ix map Experience Improvements — Full Implementation Spec++---++## TL;DR (Start Here)++**Implement Phase 1 only (CLI renderer rewrite).**++Do:++* Replace flat output with tree layout+* Group modules under subsystems+* Add health summary line+* Preserve JSON output exactly++Do NOT:++* Modify backend+* Change API contracts++Primary file:++* `ix-cli/src/cli/commands/map.ts`++---++## Goals++Improve `ix map` across 3 dimensions:++1. **Clarity** → readable, hierarchical output+2. **Exploration** → drill into subsystems/modules+3. **Performance** → avoid unnecessary recomputation++All changes must be:++* Incremental+* Backwards compatible+* Independently shippable++---++## Current Behavior (Simplified)++Pipeline:++```+ix map+ → ingest+ → POST /v1/map+ → build graph + cluster (Louvain)+ → persist regions+ → return JSON+ → CLI renders flat table+```++---++## Data Model (IMPORTANT)++The CLI receives:++```json+{+ "regions": [+ {+ "id": "r1",+ "label": "Cli / Client",+ "level": 2,+ "parent_id": null,+ "file_count": 105,+ "confidence": 0.72,+ "is_cross_cutting": false+ },+ {+ "id": "r2",+ "label": "Queries",+ "level": 1,+ "parent_id": "r1",+ "file_count": 10,+ "confidence": 0.66,+ "is_cross_cutting": true+ }+ ]+}+```++Levels:++* 3 = System+* 2 = Subsystem+* 1 = Module++---++## Problems++### Output++* Flat structure → no hierarchy+* Hard to answer “what’s inside X?”+* Truncation breaks readability+* Cross-cutting unclear+* Verbose mode overwhelming++### Exploration++* No drill-down+* No file visibility+* No explain integration++### Performance++* Always recomputes map+* Always persists+* Always runs ingest++---++## Target Design++---++## 1. Default Output (Tree Layout)++Replace flat output with:++```+Architectural Map 225 files · 4 subsystems · 14 modules++ Cli / Client · 105 files · Moderate (72%)+ ├─ Cli / Client 53 files Well-defined+ ├─ Cli / Github 22 files Moderate+ ├─ Queries 10 files Moderate ⚠+ ╰─ Role 5 files Fuzzy ⚠++ Context · 56 files · Moderate (65%)+ ├─ Ingestion 12 files Well-defined+ ╰─ Model 19 files Fuzzy+```++---++## 2. Health Summary Line++Add at top:++```+3 well-defined · 8 moderate · 3 fuzzy · 3 cross-cutting+```++---++## 3. Verbose Mode++Keep tree, add right-aligned metrics:++```+Cli / Client · 105 files conf=72% br=651 xcut=0.03+├─ Queries 10 files conf=66% xcut=0.15 ⚠+```++---++## 4. Rendering Algorithm (IMPLEMENT THIS)++```ts+const subsystems = regions.filter(r => r.level === 2);+const modules = regions.filter(r => r.level === 1);++const modulesByParent = groupBy(modules, m => m.parent_id);++for (const subsystem of subsystems) {+ printSubsystem(subsystem);++ const children = modulesByParent[subsystem.id] || [];++ children.forEach((child, i) => {+ const isLast = i === children.length - 1;+ const prefix = isLast ? "╰─" : "├─";+ printModule(child, prefix);+ });+}+```++---++## 5. Rendering Rules++* No label truncation+* Use terminal width (`process.stdout.columns`)+* Indentation = 2 spaces+* Tree chars: `├─`, `╰─`+* Cross-cutting = `⚠`+* Confidence shown as % in headers++---++## 6. Flags (Future Phases)++### `--subsystem <name>`++Show only one subsystem + modules++### `--module <name>`++Show module + files++### `--files`++Expand full file list++### Positional shorthand++```+ix map Ingestion+ix map "Cli / Client"+```++---++## 7. File Listing (Future)++Add to backend:++```json+"member_paths": ["src/cli/map.ts"]+```++Only for level-1 regions.++---++## 8. Explain Integration (Future)++### CLI Prompt Template++```+Explain the following subsystem:++Name: {label}+Files: {file_count}+Confidence: {confidence}+Child modules: {children}++Boundary files:+{top_files}++Smells:+{smells}++Summarize:+1. Purpose+2. Structure quality+3. Risks+```++---++## 9. Performance Plan++### Priority++1. Cache map by graph revision (HIGH)+2. Skip persist if unchanged (HIGH)+3. Skip ingest if fresh (MEDIUM)+4. Optimize parent wiring (LOW)++---++### 9a. Cache Map++If `map_rev == graph_rev` → return stored map++---++### 9b. Skip Persist++```+if storedRev == currentRev:+ skip write+```++---++### 9c. Skip Ingest++If freshness = false → skip ingest++---++## 10. Implementation Phases++---++### Phase 1 — CLI Output (DO THIS FIRST)++Scope:++* Tree rendering+* Health summary+* Verbose redesign++Files:++* `map.ts`++Constraints:++* JSON output unchanged+* No backend changes++---++### Phase 2 — Drill-down++* Add flags+* Add member_paths++---++### Phase 3 — Explain++* CLI context builder+* Call explain endpoint++---++### Phase 4 — Performance++* Cache map+* Skip persist+* Skip ingest++---++### Phase 5 — Future++* Incremental Louvain+* `/map/explain` endpoint+* TUI interface++---++## 11. Acceptance Criteria (Phase 1)++Must pass:++* `ix map` → tree structure+* `ix map --verbose` → tree + metrics+* `ix map --level 3` → systems only+* `ix map --level 1` → modules only+* `ix map --format json` unchanged+* No truncation up to 40 chars+* Health summary present++---++## 12. Risks++| Risk | Mitigation |+| --------------- | --------------- |+| Narrow terminal | fallback layout |+| Name ambiguity | disambiguation |+| Cache stale | `--full` flag |+| Payload size | level-1 only |+| Explain quality | improve prompt |++---++## 13. Key Files++* CLI: `ix-cli/src/cli/commands/map.ts`+* API: `MapRoutes.scala`+* Service: `MapService.scala`+* Graph: `MapGraphBuilder.scala`+* Clustering: `LouvainClustering.scala`++---++## Final Instruction for Implementer++Start with Phase 1.++Do not over-engineer.++Focus on:++* Correct grouping+* Clean tree rendering+* Zero regression in JSON output++Then stop and validate before continuing.diff --git a/docs/plans/ingestion-perf-fix.md b/docs/plans/ingestion-perf-fix.mdnew file mode 100644index 00000000..1d07a5ec--- /dev/null+++ b/docs/plans/ingestion-perf-fix.md@@ -0,0 +1,140 @@+# Ingestion Performance Fix++## Context++`core-ingestion/src/index.ts` on this branch added chunk extraction (`ParsedChunk`) which is+great, but introduced 3 regressions vs the fast baseline. Fix all three.++## Change Tracking++This section is the handoff record for follow-up review and bug fixing.++### Files changed++- `core-ingestion/src/index.ts`+ Restored a module-level tree-sitter query cache, replaced `source.split('\n')`+ with a character-scan line counter, and moved per-definition chunk hash work+ into a post-match pass using a `pendingChunks` buffer. Also added one explicit+ `(s: string)` type annotation in existing import parsing code so the package+ still builds under the current TypeScript settings.+- `docs/plans/ingestion-perf-fix.md`+ Added this change-tracking section and will keep verification notes here.++### Verification notes++- `core-ingestion`: `npm run build` passed on 2026-03-22+- `ix map` cold-ingest check: not run; skipped because `ix reset --code -y`+ mutates indexed state and was not necessary to complete the code change++---++## Fix 1 — Restore the per-process query cache (CRITICAL)++**File:** `core-ingestion/src/index.ts`++**Problem:** `parseFile()` currently calls `new Parser.Query(grammar, queries)` on every+invocation. Compiling a tree-sitter query (S-expression → automata) is expensive. With+100 TypeScript files, it compiles the same TS query 100 times.++**Fix:** Add a module-level query cache (already existed in the baseline) and use it inside+`parseFile()`.++Add this block near the existing `_parser` singleton (around line 180):++```typescript+// Per-process cache for compiled tree-sitter queries, keyed by language.+// Avoids recompiling the same query string for every file of the same language.+const _queryCache = new Map<SupportedLanguages | 'tsx', { grammar: any; query: any }>();++function getCachedQuery(language: SupportedLanguages | 'tsx', grammar: any, querySource: string): any {+ const cached = _queryCache.get(language);+ if (cached && cached.grammar === grammar) return cached.query;+ const query = new Parser.Query(grammar, querySource);+ _queryCache.set(language, { grammar, query });+ return query;+}+```++Then inside `parseFile()`, replace:+```typescript+const query = new Parser.Query(grammar, queries);+```+with:+```typescript+const cacheKey = isTsx ? 'tsx' as const : language;+const query = getCachedQuery(cacheKey, grammar, queries);+```++---++## Fix 2 — Replace `source.split('\n')` with a char scan++**File:** `core-ingestion/src/index.ts`++**Problem:** `const lines = source.split('\n')` allocates a full string array only used for+`lines.length` and `lines.length` in the `file_body` fallback chunk. Wasteful on large files.++**Fix:** Replace the split with a char scan for the line count, and keep a separate+`sourceLineCount` variable. Use it wherever `lines.length` is currently referenced.++Replace:+```typescript+const lines = source.split('\n');+```+with:+```typescript+let sourceLineCount = 1;+for (let i = 0; i < source.length; i++) { if (source.charCodeAt(i) === 10) sourceLineCount++; }+```++Then replace every reference to `lines.length` with `sourceLineCount`.++---++## Fix 3 — Move chunk SHA-256 hashing out of the hot definition loop++**File:** `core-ingestion/src/index.ts`++**Problem:** Inside the first-pass definition loop, every definition does:+```typescript+const chunkText = source.slice(startByte, endByte);+const contentHash = crypto.createHash('sha256').update(chunkText).digest('hex').slice(0, 16);+```+This runs a full SHA-256 for every function/class/method at parse time. For a file with+20 definitions that's 20 crypto operations, plus the `source.slice()` allocations.++**Fix:** Keep the chunk data (startByte, endByte) in a temporary structure during the loop,+then compute all content hashes in a single pass after the loop completes. This doesn't+change output — just batches the hashing after the tree-sitter work is done so the two+phases don't interleave.++Concretely: collect `{ name, kind, lineStart, lineEnd, startByte, endByte, language, container }`+into a `pendingChunks` array during the loop, then after the loop iterate `pendingChunks`+to compute `contentHash` and push into `chunks[]`. The `file_body` fallback should also+keep its hash computation (it's only one hash for the whole file when no chunks exist).++---++## Do NOT change++- The `ParsedChunk` interface and its fields — keep as-is.+- The chunk ops in `patch-builder.ts` (`CONTAINS_CHUNK`, `DEFINES`, `NEXT` edges) — keep as-is.+- The `buildPatchWithResolution` function — keep as-is.+- `resolveEdges` in `index.ts` — keep as-is.+- The `_parser` singleton — keep as-is.+- All query string constants in `queries.ts` — keep as-is.++---++## Verification++After the changes, run:++```bash+cd core-ingestion && npm run build+ix reset --code -y && IX_DEBUG=1 ix map+```++Expected: parse phase timing should drop from ~9s to ~2-3s for 200 files. The+`Source hash lookup: 0 known hashes` line confirms a clean cold ingest.+The map output should still show chunk-based regions (the visual output should be unchanged).diff --git a/docs/plans/map-algorithm-overhaul.md b/docs/plans/map-algorithm-overhaul.mdnew file mode 100644index 00000000..aa4cef10--- /dev/null+++ b/docs/plans/map-algorithm-overhaul.md@@ -0,0 +1,256 @@+# ix map Algorithm: Speed & Accuracy Overhaul++**Plan ID:** `46ecf20c-8bb2-9e54-121b-f5388adc8156`+**Goal ID:** `ec6a17f4-3d75-4a18-a614-c23e3fd709ae`++---++## Overview++`ix map` runs a multi-level Louvain community detection algorithm over a weighted file-coupling graph built from the knowledge graph. This document tracks planned improvements to make it both faster and more accurate.++Pipeline stages:++```+ArangoDB → WeightedFileGraph → (crosscut penalty) → Louvain → Regions → Labels → Confidence → Persist+```++---++## Implementation Status++Status: implemented in code.++What shipped:++- `MapGraphBuilder.fetchCouplingByUri` no longer builds a giant `MERGE` map. It now resolves edge endpoints with direct `nodes` lookups keyed by `logical_id` and aggregates with `COLLECT ... WITH COUNT`.+- `MapGraphBuilder.liveFileCount` now uses `COLLECT WITH COUNT` instead of loading all file rows.+- `MapGraphBuilder.computeGraph` now precomputes directory segments once, short-circuits same-directory path proximity to `1.0`, and caps synthetic structural edge growth with `MaxEdges`.+- `WeightedFileGraph` now carries `predicatePairs`, which restores dominant signal computation downstream.+- `LouvainClustering.cluster` now runs three deterministic seeds, applies adaptive resolution by repository size, stops on modularity-gain convergence instead of a fixed 30-pass cap, and uses in-place Fisher-Yates shuffle.+- `MapService` now uses the tuned signal weights:+ - `CALLS = 1.0`+ - `IMPORTS = 0.5`+ - `EXTENDS/IMPLEMENTS = 1.2`+ - `PATH = 0.2`+- The CLI text renderer now defaults to the hierarchy graph view, with the ranked list preserved behind `ix map --list`.+- Cross-cut token detection was expanded with layer/framework tokens including `service`, `controller`, `repository`, `handler`, `middleware`, `model`, `entity`, `dto`, `interface`, `types`, and `errors`.+- Confidence scoring now uses:++```scala+val brScore = tanh(boundaryRatio / 2.5)+val cohScore = cohesion.min(1.0)+val sizeBonus = tanh(size / 8.0)+val crossPenalty = crosscutScore * 0.4+```++- Labeling now prefers filtered suffixes first, then discriminative directory scoring, then filename-token fallback. Weak suffix tokens like `api`, `service`, and `common` no longer win by default.+- `dominant_signals` is now populated from real intra-region predicate counts instead of being emitted empty.++Validation summary:++- `sbt memoryLayer/compile` passed after the changes.+- A fresh backend using the new build returned `empty_dominant_signals = 0`.+- Sequential timing checks on this repo remained within the target range for a repo of this size, though end-to-end `ix map` time here is heavily influenced by ingest/persist overhead rather than the clustering math alone.++---++## Current Known Issues++### Speed++| Location | Issue | Severity |+|---|---|---|+| `MapGraphBuilder.fetchCouplingByUri` | AQL `MERGE` over all live nodes to build `uri_map` — full collection scan on every call | High |+| `MapGraphBuilder.fetchFiles` | Full table scan, no index on `kind` | Medium |+| `MapGraphBuilder.currentInputRev` | Two separate full table scans (nodes + edges) | Medium |+| `MapGraphBuilder.liveFileCount` | Calls `fetchFiles()` just to count — fetches all data unnecessarily | Low |+| `MapGraphBuilder.computeGraph` (path-proximity pass) | O(n²) within each directory — 100-file directory = 4,950 pairs; `pathProximity` called per pair with string splits | High |+| `MapGraphBuilder.pathProximity` | Splits path strings on every call, no precomputation | Medium |+| `LouvainClustering.louvainPass` | `rng.shuffle(nodes)` on every pass — allocates new list up to 30× | Low |+| `MapService.applyPenalty` | Copies full adjacency map — could be done in-place since graph is not reused | Low |+| `MapService.loadCachedMap` → `liveFileCount` | Separate DB round trip just to populate `fileCount` | Low |++### Accuracy++| Location | Issue | Severity |+|---|---|---|+| `MapService.buildOneRegion` | `dominantSignals` always `[]` — per-predicate counts lost after graph construction | Medium |+| Signal weights (α=1.0 β=0.7 γ=0.9 ε=0.3) | Arbitrary, not empirically validated | Medium |+| `crosscutTokens` set | Missing common framework terms: "service", "controller", "repository", "handler", "middleware" | Medium |+| `computeConfidence` weights (0.4/0.3/0.3) | Ad-hoc formula, not validated against real-world repo quality | Low |+| `inferLabel` | Strategy 1 (longest common dir suffix) fires before the better strategy 2 (discriminative segment scoring) — can produce misleading labels for deeply nested monorepos | Medium |+| Louvain resolution fixed at 1.0 | Higher resolution → smaller communities; lower → larger. Should adapt to repo size or be tunable | Medium |+| Louvain seed fixed at 42 | Single deterministic run may hit local optima — multiple seeds with best-modularity selection improves stability | Low |+| `minCommunitySize` hardcoded | 3 for 3-level repos, 2 otherwise — should scale with repo size | Low |++---++## Tasks++### Task 1 — Optimize AQL queries in MapGraphBuilder+**ID:** `adb844f1-62a3-98f2-a27e-33cc50364ae0`++The two costliest DB operations:++**`fetchCouplingByUri` MERGE approach:**+```aql+LET uri_map = MERGE(+ FOR n IN nodes FILTER ... RETURN {[n.logical_id]: n.provenance.source_uri}+)+```+`MERGE` over thousands of nodes is slow and memory-heavy. Replace with a direct join:+```aql+FOR e IN edges+ FILTER e.predicate IN @predicates AND e.deleted_rev == null+ LET sn = FIRST(FOR n IN nodes FILTER n.logical_id == e.src ... RETURN n.provenance.source_uri)+ LET dn = FIRST(FOR n IN nodes FILTER n.logical_id == e.dst ... RETURN n.provenance.source_uri)+ FILTER sn != null AND dn != null AND sn != dn+ COLLECT srcUri = sn, dstUri = dn, pred = e.predicate WITH COUNT INTO cnt+ RETURN {srcUri, dstUri, predicate: pred, count: cnt}+```+Or with `AQL FOR` subqueries if the ArangoDB index on `logical_id` is usable (it should be, as it's the document key).++**`liveFileCount`:** Replace with a `COLLECT WITH COUNT` query instead of fetching all file data.++**`currentInputRev`:** Benchmark whether combining into one AQL pass is faster.++---++### Task 2 — Fix O(n²) path-proximity pass+**ID:** `9544f83f-e0bb-8973-aa1c-a4d642a63ef0`++**Problem:** The same-directory edge pass is:+```scala+for (i <- dirFiles.indices; j <- (i + 1) until dirFiles.size) {+ pathProximity(sv.path, dv.path) // splits strings every call+}+```+For a directory with D files, this is O(D²) calls. Monorepos can have 50+ files in src/.++**Fix:**+1. Precompute `dirSegments: Map[NodeId, Array[String]]` for all files once.+2. Replace `pathProximity(a, b)` with an index lookup using precomputed segments.+3. For same-directory files the result is always `1.0 / (1.0 + 0)` = 1.0 — can hardcode for the same-dir case.+4. Add an `adjMut.size > MaxEdges` safety cap to prevent quadratic blowup in dense directories.++---++### Task 3 — Improve Louvain convergence and stability+**ID:** `ca2b0225-c97d-77bf-021f-32b6e1b51884`++**Multi-seed best-of selection:**+```scala+val runs = (1 to 3).map(seed => louvainPass(graph, resolution, new Random(seed)))+val best = runs.maxBy(a => modularity(graph, a))+```+3 runs with different seeds, pick the one with highest modularity. Cost: 3× but more stable results.++**Adaptive resolution:**+- Small repos (< 50 files): resolution 1.2 (prefers smaller, tighter communities)+- Medium repos (50–500 files): resolution 1.0 (default)+- Large repos (> 500 files): resolution 0.8 (allows larger communities, avoids over-fragmentation)++**Convergence check:** The 30-pass hard cap is arbitrary. Add a modularity delta check — stop when ΔQ < 1e-6 rather than forcing 30 passes.++**`rng.shuffle` allocation:** Use Fisher-Yates in-place shuffle on an `Array[NodeId]` instead of `rng.shuffle(nodes)` which allocates a new `Vector` each pass.++---++### Task 4 — Tune signal weights, crosscut detection, and confidence formula+**ID:** `fc9c3ec1-5b6e-3889-8974-ecc9d6dc8995`++**Signal weights — proposed tuned values:**++| Signal | Current | Proposed | Rationale |+|---|---|---|---|+| α (CALLS) | 1.0 | 1.0 | Keep as anchor |+| β (IMPORTS) | 0.7 | 0.5 | Imports are noisier than calls in TypeScript/Python; over-weighted |+| γ (EXTENDS/IMPLEMENTS) | 0.9 | 1.2 | Type hierarchy is a very strong cohesion signal, especially in Scala/Java |+| ε (path proximity) | 0.3 | 0.2 | Structural prior should be weak; files in same dir ≠ same subsystem |++**Crosscut token expansion:**+```scala+CrosscutTokens = Set(+ // existing+ "util", "utils", "helper", "helpers", "common", "shared", "base",+ "logging", "log", "logger", "config", "conf", "constants", "constant",+ "extensions", "ext", "mixin", "mixins", "core", "framework",+ // add: framework layer names+ "service", "services", "controller", "controllers",+ "repository", "repositories", "repo", "handler", "handlers",+ "middleware", "interceptor", "filter", "filters",+ "model", "models", "entity", "entities", "dto", "dtos",+ "interface", "interfaces", "types", "errors", "exceptions"+)+```+These are all cross-cutting by nature in layered architectures.++**`computeConfidence` — boundary ratio saturation:**++Current `brScore = tanh(br / 3.0)` saturates at br ≈ 9. Real repos often have br in 0.5–5 range, so the curve is well-placed. But the size bonus `tanh(size / 10)` gives half-credit to 7-file regions. Proposed:+```scala+val brScore = math.tanh(boundaryRatio / 2.5) // slightly tighter saturation+val cohScore = cohesion.min(1.0)+val sizeBonus = math.tanh(size / 8.0) // penalize tiny regions more+val crossPenalty = crosscutScore * 0.4 // stronger crosscut penalty+```++---++### Task 5 — Restore dominant_signals and improve label inference+**ID:** `8c897b67-c639-5705-c0d3-4278b4fa54f3`++**Dominant signals:**+The `dominantSignals` field was cleared in the bug-fix review because `signalCounts` was never populated. The right fix is to thread per-predicate counts through from `MapGraphBuilder` into `buildOneRegion`.++Option A: Add `predicateCounts: Map[(NodeId,NodeId), Map[String,Int]]` to `WeightedFileGraph` (keeps graph builder responsible).++Option B: In `buildOneRegion`, iterate the adjacency and compute a signal breakdown from the raw weights by inverting the weight formula (approximate).++Option A is cleaner. Add to `WeightedFileGraph`:+```scala+final case class WeightedFileGraph(+ vertices: Vector[FileVertex],+ adjMatrix: Map[NodeId, Map[NodeId, Double]],+ degrees: Map[NodeId, Double],+ totalWeight: Double,+ predicatePairs: Map[(NodeId, NodeId), Map[String, Int]] // NEW+)+```++**Label inference — fix strategy order:**++Strategy 1 (longest common suffix) fires even when the common segment is a noise token like "api" or "db". Proposed:++1. Try longest common suffix after filtering noise tokens (current).+2. If result length < 3 chars, fall through to strategy 2.+3. Strategy 2 (discriminative depth-weighted segments) should be weighted by _intra-cluster_ frequency vs. _corpus_ frequency — a segment appearing in 80% of this cluster but only 10% of all files is highly discriminative.+4. Fallback: most frequent camelCase token from filenames.++---++## Performance Benchmarks Needed++Before tuning, establish baselines on this repo:++```bash+time ix map --format json > /dev/null # end-to-end+time ix map --format json > /dev/null # second run (should hit cache)+```++Target metrics:+- Cold map (no cache): < 5s for this repo (~150 files)+- Cache hit: < 500ms+- Large repo (2000+ files): < 30s cold, < 1s cached++---++## Files Affected++| File | Changes |+|---|---|+| `memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala` | Tasks 1, 2 |+| `memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scala` | Task 3 |+| `memory-layer/src/main/scala/ix/memory/map/MapService.scala` | Task 4, 5 |+| `memory-layer/src/main/scala/ix/memory/map/MapTypes.scala` | Task 5 (add `predicatePairs` to `WeightedFileGraph`) |diff --git a/docs/plans/map-algorithm-review.md b/docs/plans/map-algorithm-review.mdnew file mode 100644index 00000000..748fe938--- /dev/null+++ b/docs/plans/map-algorithm-review.md@@ -0,0 +1,334 @@+# ix map Performance + Scalability Fix Plan++**Files reviewed:** `MapGraphBuilder.scala`, `MapService.scala`, `LouvainClustering.scala`, `MapTypes.scala`+**Date:** 2026-03-22++---++## Change Tracking++This section is the running handoff record for follow-up review and bug fixing.++### Files changed++- `memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala`+ Rewrote `fetchCouplingByUri()` to build a single `logical_id -> source_uri`+ lookup map inside AQL, pushed source-extension filtering into `fetchFiles()`,+ added directory-size guardrails for path-proximity edges, and changed+ `currentInputRev()` to use a dedicated `source-graph` revision with a scan+ fallback for older data.+- `memory-layer/src/main/scala/ix/memory/map/MapService.scala`+ Replaced O(R²) parent wiring in `buildRegions()` with partition-based lookup,+ removed subset-scan recomputation from `rehydrateRelationships()`, trimmed+ `CrosscutTokens`, replaced the max-degree heuristic with a real percentile+ cutoff, and precomputed signal adjacency so dominant-signal scoring only+ touches region-local incident pairs.+- `memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scala`+ Added seed gating for small graphs and reused the community-weight map inside+ the hot Louvain loop instead of allocating it per node.+- `memory-layer/src/main/scala/ix/memory/db/BulkWriteApi.scala`+ Updated revision writes so bulk ingestion advances both the global revision+ and the dedicated `source-graph` revision.+- `memory-layer/src/main/scala/ix/memory/db/ArangoGraphWriteApi.scala`+ Added source-graph revision tracking for patch commits that touch map inputs,+ while leaving derived writes on the global revision only.+- `docs/plans/map-algorithm-review.md`+ Revised the recommendations so they match the current codebase, added this+ tracking section, and recorded implementation and verification status.++### Implementation notes++- `currentInputRev()` needs a dedicated source-graph revision, not just the+ global revision, because non-map-input writes also advance the main revision+ counter.+- `buildRegions()` can use partition assignments directly, but only after+ accounting for communities filtered out by `minCommunitySize`.+- `CrosscutTokens` cleanup should be paired with a fix to the current+ pseudo-percentile degree threshold.++### Verification notes++- Build: `sbt memoryLayer/compile` passed on 2026-03-22+- Runtime benchmark: not run+- `ix map` end-to-end validation: not run in this pass++---++## TL;DR++Primary bottlenecks:++1. O(E × N) database query in `fetchCouplingByUri()`+2. Full graph scan per run in `currentInputRev()`+3. O(R²) region construction in `buildRegions()`++These issues cause `ix map` to degrade from near-linear toward quadratic behavior, leading to Docker crashes and unusable performance on large codebases.++Fixing these should reduce runtime from minutes to seconds for 500-2,000 file repos.++---++## Summary++The algorithm is architecturally correct: multi-level Louvain plus weighted signals is the right direction.++The problem is **scalability**, not correctness.++**Severity legend:** Critical / High / Medium / Low++---++# 1. Critical Bottlenecks++## `fetchCouplingByUri()` - O(E × N) correlated subqueries++Each edge performs two full scans over `nodes` to resolve `src` and `dst`. Without an index on `logical_id`, this is O(E × N) - for 5,000 edges and 50,000 nodes that's 250M comparisons per `ix map`.++**Fix (preferred):** build the URI map once, then join:++```aql+LET uriMap = (+ FOR n IN nodes+ FILTER n.deleted_rev == null+ AND n.provenance.source_uri != null+ AND n.kind NOT IN @ignoredKinds+ RETURN { id: n.logical_id, uri: n.provenance.source_uri }+)++LET byId = ZIP(uriMap[*].id, uriMap[*].uri)++FOR e IN edges+ FILTER e.predicate IN @predicates+ AND e.deleted_rev == null+ LET su = byId[e.src]+ LET du = byId[e.dst]+ FILTER su != null AND du != null AND su != du+ COLLECT srcUri = su, dstUri = du, pred = e.predicate WITH COUNT INTO cnt+ RETURN { srcUri, dstUri, predicate: pred, count: cnt }+```++**Alternative:** add a persistent index on `nodes.logical_id`. Cuts each subquery from O(N) to O(log N) without rewriting the query.++---++## `currentInputRev()` - full graph scan per run++Runs a MAX aggregation over all nodes and edges on every `ix map` call, even when the graph is unchanged and the cached map will be reused.++**Fix:** maintain a dedicated source-graph revision entry updated only when source-graph inputs change. Do not reuse the global revision counter directly, because map persistence and other derived writes also advance it.++One viable implementation is a separate revisions entry:++```json+{ "_key": "source-graph", "rev": 123 }+```++Then `currentInputRev()` becomes a single key lookup, with a fallback full scan only for older databases that do not yet have the dedicated metadata:++```aql+FOR doc IN revisions+ FILTER doc._key == "source-graph"+ LIMIT 1+ RETURN doc.rev+```++Result: O(1) instead of O(N + E).++---++# 2. High Priority Fixes++## `buildRegions()` - O(R²) parent-child wiring++Currently calls `region.memberFiles.subsetOf(p.memberFiles)` for every `(region, parent)` pair. Replace with a direct lookup from the Louvain partition, but preserve correctness when some communities are filtered out by `minCommunitySize`:++```scala+val fileToParent: Map[NodeId, Int] = coarseLevel.assignment++val parentOpt = memberFiles.headOption+ .flatMap(fileToParent.get)+ .flatMap(coarseRegionByCommunity.get)+```++Where `coarseRegionByCommunity` only contains retained coarse communities, and equal-set coarse regions should still be treated as "no parent" to match the current hierarchy behavior.++---++## `rehydrateRelationships()` - remove recomputation++`parentId` and `childRegionIds` are already persisted on region nodes. Do not rerun the subset scan on cache load. Wire children directly from the loaded data:++```scala+val childrenByParent = regions+ .flatMap(r => r.parentId.map(pid => pid -> r.id))+ .groupBy(_._1)+ .view+ .mapValues(_.map(_._2).toSet)+ .toMap+```++---++## `CrosscutTokens` - too many false positives++Tokens like `"service"`, `"controller"`, `"repository"`, `"model"`, `"entity"`, `"dto"` are standard naming conventions, not cross-cutting concerns. Flagging them applies a penalty to the coupling signals that should cluster those files together.++Also fix the degree-side heuristic: the current code uses `maxDeg * 0.9`, which is not actually "top 10% by degree". If percentile behavior matters, compute it from the observed degree distribution instead of scaling the maximum.++Remove architectural terms. Keep only genuine horizontal concerns:++```scala+private val CrosscutTokens = Set(+ "util", "utils", "helper", "helpers", "common", "shared",+ "logging", "config", "constants", "base"+)+```++---++# 3. Medium Fixes++## `computeDominantSignals()` - O(total edges) per region++Iterates all `graph.predicatePairs` for each region. Build a per-file signal index once before the region loop so each region only touches incident pairs for its own members.++```scala+Map[NodeId, Vector[(NodeId, Map[String, Int])]]+```++---++## Path proximity - O(N²) per directory++Only run the path proximity pass for directories with between 2 and 50 files. Large flat directories (generated code, fixtures) add noise and inflate the graph without contributing architectural signal.++Also push source-extension filtering into the `fetchFiles()` AQL so the JVM does not deserialize non-source file rows it will drop anyway.++---++## Louvain optimizations++**Seed gating** - skip the extra two seeds for small graphs:++```scala+val numSeeds = if (graph.vertices.size < 30) 1 else 3+```++**Avoid map allocation in hot loop** - allocate once and clear per node:++```scala+val wToComm = mutable.HashMap[Int, Double]()+// inside the loop:+wToComm.clear()+```++---++# 4. Graph Guardrails++Hard limits to prevent runaway memory and time on large repos:++**Node limits:**+- V > 5,000: disable path proximity edges; cap soft edges per node at 5+- V > 10,000: switch to coarse mode (directory-level clustering only)++**Density limits:**+- E / V > 50: prune lowest-weight edges before clustering++**Edge budget by type:**+- Structural edges (imports, calls): unlimited+- Path similarity: max 3 per node++---++# 5. Execution Plan++## Phase 1 - Database++- Fix `fetchCouplingByUri()` query+- Replace `currentInputRev()` with dedicated source-graph revision metadata++**Validate:** identical output, runtime measurably reduced.++## Phase 2 - Region construction++- Replace `buildRegions()` subset scan with direct lookup+- Remove recomputation in `rehydrateRelationships()`+- Precompute dominant-signal adjacency once per graph++**Validate:** same region hierarchy, no missing parents.++## Phase 3 - Graph density controls++- Limit path proximity to small/medium directories+- Push source-extension filtering into the DB query++**Validate:** lower edge count on large flat trees with no missing file vertices.++## Phase 4 - Louvain optimization++- Seed gating by graph size+- Reuse `wToComm` map across iterations++**Validate:** clustering stable across runs, runtime improved.++## Phase 5 - Cross-cut cleanup++- Fix `CrosscutTokens`+- Replace the pseudo-percentile threshold with a real percentile cutoff++## Phase 6 - Scaling safeguards++- Implement graph guardrails+- Add coarse mode for V > 10,000++---++# 6. Constraints++- Output of `ix map` should remain materially consistent: same broad region membership and hierarchy unless a heuristic fix explicitly justifies a behavior change+- Do not introduce new edge types+- Do not alter signal weights without justification++---++# 7. Anti-Patterns++- O(N²) similarity computations across the whole repo+- Full collection scans inside per-edge or per-node loops+- Per-node or per-edge database queries+- Rebuilding the full graph on every cache lookup+- Dense adjacency matrices++---++# 8. Success Criteria++| Metric | Target |+|---|---|+| `fetchCouplingByUri` | < 100ms for 10k edges |+| `currentInputRev` | < 5ms |+| 500 files | < 5s end-to-end |+| 2,000 files | < 10s end-to-end |++---++# 9. Priority Order++| Priority | Fix |+|---|---|+| 1 | `fetchCouplingByUri` query rewrite |+| 2 | `currentInputRev` dedicated source-graph revision |+| 3 | Path proximity directory-size guardrail |+| 4 | `CrosscutTokens` cleanup + real percentile threshold |+| 5 | `rehydrateRelationships` recompute removal |+| 6 | `buildRegions` O(R²) replacement |+| 7 | Louvain seed gating + map reuse |++---++## Final Note++Target complexity: **O(V + E)**.++If performance degrades superlinearly, the graph is too dense or queries are unindexed. Diagnose with `IX_DEBUG=1 ix map` and inspect per-phase timing before adding more signal types or hierarchy levels.diff --git a/docs/plans/subsystem-explain.md b/docs/plans/subsystem-explain.mdnew file mode 100644index 00000000..a9077221--- /dev/null+++ b/docs/plans/subsystem-explain.md@@ -0,0 +1,514 @@+# `ix subsystems <target> --explain`++**Date:** 2026-03-22++---++## Overview++`ix explain` produces a narrative breakdown of an individual symbol (function, class, file).+`ix subsystems <target> --explain` does the same thing one level up — for an architectural+region produced by `ix map`.++The goal is to answer: *"What is this subsystem, what does it contain, how healthy is it, and+why does it matter?"* — in plain English, with enough structural context that a developer new+to the codebase can orient themselves immediately.++---++## Reality Check++The basic shape of this plan is correct, but the current codebase has two gaps the original+draft did not account for:++1. The existing scoped subsystem payload is sufficient for most of `--explain`, but it does+ **not** currently expose enough revision metadata to support a reliable+ `"health score may be stale"` note.+2. Large-repo ingestion is currently fragile for architectural reasons that are adjacent to+ this feature work: the ingest path materializes too much in memory, queries too much state+ up front, and emits oversized patch payloads for non-code files.++This document now covers both:+- the concrete changes required for `ix subsystems <target> --explain`+- the concrete changes required to stop ingestion from failing on large repositories++---++## Change Tracking++### Files changed++- `docs/plans/subsystem-explain.md`+ Expanded the plan so it reflects the current subsystem APIs and the large-repo ingestion+ failure modes discovered during code review.++### Verification notes++- Doc-only update; no code or tests run in this pass.++---++## CLI Interface++```+ix subsystems <target> --explain+ix subsystems <target> --explain --format json+ix subsystems <target> --explain --pick <n>+```++`--explain` is a new flag on the existing `subsystems` command, layered on top of the current+target-scoping behaviour. It is mutually exclusive with `--list`.++**Examples:**++```+ix subsystems api --explain+ix subsystems "Ingestion Layer" --explain+ix subsystems auth --explain --pick 2+ix subsystems auth --explain --format json+```++`--pick` resolves ambiguous matches the same way it does today.++---++## Data Sources++Two existing server calls are combined — no new endpoints required.++| Data | Source |+|---|---|+| Region label, level, parent, children, files, confidence, dominant signals, cross-cutting flag | `GET /v1/subsystems/map?target=…` → `ScopedSubsystemResult` |+| Health score, chunk density, smell rate, smell files, total chunks | `GET /v1/subsystems` → `SubsystemScore[]`, filtered to the resolved region id |++If the health score has not yet been computed for the region (e.g. `ix map` was run but+`ix subsystems` has not been run since), the Health section is omitted and a note is printed:+`Run 'ix subsystems' to compute health scores.`++### Current Data Contract Gap++The current score payload is enough to render Health, but not enough to prove staleness.++Today:+- `GET /v1/subsystems/map` returns scoped region structure without score metadata.+- `GET /v1/subsystems` returns health scores without a `map_rev` or score revision field.++That means this note from the original draft is **not implementable as written**:++`Health score may be stale — run 'ix subsystems' to refresh.`++To support that note cleanly, one of these must change:++1. Add `map_rev` to each stored subsystem score result.+2. Add score `created_rev` or score computation revision to the list response.+3. Remove the stale-health note from v1 and only keep the `"Run 'ix subsystems' to compute health scores."` note when the score is missing.++Recommended v1 choice:+- Keep the feature server-light.+- Do **not** add a stale-health note yet.+- Only add it after score revision metadata is exposed.++---++## Output Sections++### Text Format++```+Explanation+ `Ingestion Layer` serves as a subsystem spanning 34 files. It groups components+ responsible for parsing source files and committing patches to the graph. It sits+ inside the Core System and is well-defined with import coupling as its primary+ structural signal.++Context+ Level: subsystem (level 2)+ Parent: Core System (system)+ Files: 34+ Children: 4 modules+ Confidence: Well-defined (88%)+ Signals: import coupling · path proximity++Composition+ ● MODULE File Parser Well-defined 91% 12 files+ ● MODULE Patch Builder Well-defined 84% 9 files+ ● MODULE Bulk Writer Moderate 66% 8 files+ ● MODULE Ingestion Loader Fuzzy 44% 5 files+ 3 well-defined · 1 moderate · 0 fuzzy · 0 cross-cutting++Health+ Score: ████░ 0.74+ Chunk density: 5.1 chunks/file+ Smell rate: 6% (2 files flagged)+ Status: Good — rich structure, low smell coverage++Why it matters+ Ingestion Layer is the primary entry path for all source data into the graph.+ Its high chunk density indicates thorough code coverage in the knowledge graph.+ 2 files have active smell claims — run `ix smells` to review.+ Run `ix impact "Ingestion Layer"` to see what depends on this subsystem.+```++### JSON Format++`--format json` returns a single object:++```json+{+ "resolvedTarget": {+ "id": "uuid",+ "label": "Ingestion Layer",+ "level": 2,+ "label_kind": "subsystem"+ },+ "explanation": "...",+ "context": {+ "level": 2,+ "label_kind": "subsystem",+ "parent": { "id": "uuid", "label": "Core System", "label_kind": "system" },+ "file_count": 34,+ "child_count": 4,+ "confidence": 0.88,+ "dominant_signals": ["import coupling", "path proximity"],+ "is_cross_cutting": false+ },+ "composition": {+ "children": [+ { "label": "File Parser", "level": 1, "label_kind": "module", "file_count": 12, "confidence": 0.91, "is_cross_cutting": false }+ ],+ "summary": { "well_defined": 3, "moderate": 1, "fuzzy": 0, "cross_cutting": 0 }+ },+ "health": {+ "score": 0.74,+ "chunk_density": 5.1,+ "smell_rate": 0.06,+ "smell_files": 2,+ "total_chunks": 173,+ "status": "good"+ },+ "whyItMatters": "...",+ "notes": []+}+```++`health` is `null` if no score has been computed yet.++---++## Section Definitions++### Explanation++One to two sentence prose summary synthesised from the region label, level, parent name,+dominant signals, file count, and cross-cutting flag. No LLM — constructed from templates,+the same way `ix explain`'s `describeRoleLocal` and `synthesizeSystemRole` work.++Template inputs:++| Signal | Mapped meaning |+|---|---|+| `level === 3` | "system" — top-level architectural boundary |+| `level === 2` | "subsystem" — a bounded functional area |+| `level === 1` | "module" — a focused implementation cluster |+| `is_cross_cutting` | adds "…and is cross-cutting — it spans multiple subsystems" |+| `dominant_signals` | "Its primary structural signal is import coupling" |+| `parent.label` | "It sits inside the `<parent>` system" |+| `confidence ≥ 0.75` | "well-defined" |+| `confidence ≥ 0.50` | "moderately defined" |+| `confidence < 0.50` | "loosely defined" |++### Context++Tabular structural facts — mirrors the `ix explain` Context section:++```+Level: subsystem (level 2)+Parent: Core System (system)+Files: 34+Children: 4 modules+Confidence: Well-defined (88%)+Signals: import coupling · path proximity+```++Omit `Children` line when there are none (leaf modules).+Omit `Parent` line for top-level systems with no parent.++### Composition++Renders the immediate child regions as a mini tree, sorted by file count descending+(same comparator as `renderScopedTreeNode`). Capped at 10 children; if more exist,+append ` … N more. Use 'ix subsystems <target>' for the full tree.`++Followed by the existing well-defined / moderate / fuzzy / cross-cutting tally from+`ScopedSubsystemResult.summary`.++Omit the entire section for leaf regions (no children).++### Health++Uses the `SubsystemScore` for the resolved region id.++```+Score: ████░ 0.74 (healthBar using 5-block scale, same as --list)+Chunk density: 5.1 chunks/file+Smell rate: 6% (2 files flagged)+Status: <status phrase>+```++**Status phrases** (derived from score thresholds):++| Score | Status phrase |+|---|---|+| ≥ 0.80 | `Healthy — rich structure, minimal technical debt` |+| ≥ 0.65 | `Good — rich structure, low smell coverage` |+| ≥ 0.45 | `Moderate — structure is present, smell coverage is elevated` |+| < 0.45 | `Needs attention — sparse structure or significant smell coverage` |++Chunk density context lines (appended to Status line as a note):++| Density | Note |+|---|---|+| ≥ 5.0 | `High chunk density indicates thorough code coverage in the knowledge graph.` |+| ≥ 2.0 | *(no note)* |+| < 2.0 | `Low chunk density — run 'ix ingest' to improve coverage.` |++### Why it matters++Single prose paragraph synthesised from health, level, cross-cutting status, and parent+context. Mirrors the structure of `ix explain`'s `renderWhyItMatters`.++Decision tree:++1. **Cross-cutting + fuzzy confidence** → architectural drift warning:+ `<label> spans multiple subsystems and has low clustering confidence, suggesting architectural drift. Consider decomposing or clarifying its boundaries.`++2. **High health (≥ 0.80) + top-level (level 3)** → stable anchor:+ `<label> is a stable, well-defined system boundary. Changes within it are well-contained. Run 'ix impact "<label>"' to review cross-system dependencies.`++3. **Low health (< 0.45) + smell files > 0** → needs attention:+ `<label> has <N> files with active smell claims and a health score of <score>. Prioritize refactoring here before expanding the subsystem. Run 'ix smells' to review.`++4. **Normal** → role + callers suggestion:+ `<label> is a <status> <level label> <parent phrase>. Run 'ix impact "<label>"' to understand its downstream blast radius.`++### Notes++Mirrors `ix explain`'s Notes section. Sources:+- Health score missing: `Run 'ix subsystems' to compute health scores for this region.`+- Health score computed at a different revision than current graph:+ `Health score may be stale — run 'ix subsystems' to refresh.`++Implementation note:+- This second note requires new score revision metadata and should be treated as optional+ follow-up work, not part of the minimum implementation.++---++## Ambiguity Handling++Identical to the current `ix subsystems <target>` behaviour. If the target is ambiguous,+the existing `renderSubsystemError` output is shown and `--explain` is ignored:++```+Ambiguous target "auth". Multiple architecture regions matched:++1. Auth Middleware subsystem 12 files+2. Auth Tests module 8 files++Run:+ ix subsystems auth --pick <n> --explain+```++---++## Implementation Plan++### Phase 1 — Client (subsystems.ts)++1. Add `--explain` flag to the command definition.+2. When `--explain` is set and a `ScopedSubsystemResult` is returned:+ a. Make a second call to `client.listSubsystems()` and find the matching `SubsystemScore`+ by `region_id`.+ b. Call a new `renderSubsystemExplanation(scoped, score | null)` function.+3. `renderSubsystemExplanation` implements the five sections above using template functions+ (no new server endpoints, no LLM).+4. If `--format json` is set, return a purpose-built explanation object instead of the current+ scoped map JSON.++New functions (all in `subsystems.ts` or a new `explain/subsystem.ts`):++| Function | Purpose |+|---|---|+| `synthesizeSubsystemExplanation(region, parent)` | Returns the explanation string |+| `healthStatus(score)` | Returns the status phrase |+| `renderSubsystemExplanation(scoped, score)` | Assembles all five sections |+| `renderSubsystemExplanationJson(scoped, score)` | Returns the JSON object |++Primary files to change:++- `ix-cli/src/cli/commands/subsystems.ts`+ Add the new flag, target validation, second score lookup, text rendering, and JSON rendering.+- `ix-cli/src/client/api.ts`+ Likely no required API changes for v1, but tighten typings for `listSubsystems()` and+ `getSubsystemMap()` if the implementation needs stricter data contracts.+- `memory-layer/src/main/scala/ix/memory/api/SubsystemRoutes.scala`+ No new endpoint required for v1. Only change this file if revision metadata is added later.+- `memory-layer/src/main/scala/ix/memory/api/MapEncoder.scala`+ No required change for v1 text mode. Only change this file if scoped JSON needs extra fields+ or if you choose to expose `map_rev`.+- `memory-layer/src/main/scala/ix/memory/subsystem/SubsystemScoringService.scala`+ Only required if you want the stale-health note in v1 by exposing score revision metadata.++### Phase 2 — Optional score metadata++Only needed if stale-health notes are required:++1. Extend stored score payloads to include `map_rev` or score creation revision.+2. Extend `listScores()` so the CLI can compare the score revision to the current scoped region.+3. Add the stale-health note only when the score metadata proves drift.++Suggested file targets:++- `memory-layer/src/main/scala/ix/memory/subsystem/SubsystemScoringService.scala`+- `memory-layer/src/main/scala/ix/memory/api/SubsystemRoutes.scala`+- `ix-cli/src/client/api.ts`+- `ix-cli/src/cli/commands/subsystems.ts`++### Phase 3 — Validation++- Smoke test against two regions: one with a health score, one without.+- Confirm `--format json` produces valid parseable output.+- Confirm `--explain` + ambiguous target shows the error, not a stack trace.+- Confirm `--explain` without a target produces the intended CLI error.+- If stale-health metadata is implemented, verify both fresh and stale note paths.++---++## Large-Repo Ingestion Failures++This is separate from `subsystems --explain`, but it is the other major issue discovered while+reviewing the current architecture and should be fixed in parallel.++### Failure Mode 1 — all parse results are retained in memory before any write++Current behavior:+- `BulkIngestionService.ingestPath()` discovers all files.+- Loads all existing hashes for all files.+- Parses all files with `parTraverseN`.+- Keeps every `Parsed(FileBatch)` in memory.+- Only then starts chunked writes.++That makes the current chunked commit logic too late to protect memory.++Required change:+- Convert ingestion into a bounded pipeline:+ 1. discover files+ 2. slice into file batches+ 3. load hashes for that slice+ 4. parse only that slice+ 5. commit only that slice+ 6. release memory before moving to the next slice++Primary file:+- `memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala`++### Failure Mode 2 — source hash lookup is one giant query++Current behavior:+- `getSourceHashes(sourceUris)` is called with the full discovered file list.+- The query scans `patches`, sorts by revision, and groups by URI in one shot.++On large repos, the bind payload and query work both become too large.++Required change:+- Chunk source-hash lookups to the same batch size as ingestion.+- Keep hash lookup local to each parse/commit slice.+- If needed later, add an indexed materialized source-hash table rather than scanning `patches`.++Primary files:+- `memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala`+- `memory-layer/src/main/scala/ix/memory/db/ArangoGraphQueryApi.scala`++### Failure Mode 3 — discovery is broader than parser support++Current behavior:+- File discovery includes many docs, configs, scripts, build files, and other text-like files.+- Parser routing only has dedicated parsers for Python, TypeScript, Scala, config, and Markdown.+- Unsupported files fall back to generic text ingestion.++That means large repos do not just ingest code; they also ingest a wide tail of raw text.++Required change:+- Narrow default discovery to parser-supported file types for normal ingestion.+- Optionally add an explicit `--include-generic-text` or similar opt-in later.+- At minimum, stop treating unsupported text as full-content graph facts by default.++Primary files:+- `memory-layer/src/main/scala/ix/memory/ingestion/FileDiscovery.scala`+- `memory-layer/src/main/scala/ix/memory/ingestion/ParserRouter.scala`+- `memory-layer/src/main/scala/ix/memory/api/IngestionRoutes.scala`+- `ix-cli/src/client/api.ts`++### Failure Mode 4 — generic text path stores full content and explodes patch size++Current behavior:+- Generic text fallback stores `"content"` in entity attrs.+- `GraphPatchBuilder` converts attrs into claims.+- Bulk ingestion persists node docs, claim docs, and the full patch document.++This duplicates large text payloads across multiple persisted artifacts.++Required change:+- Remove full raw content from generic-text attrs by default.+- Replace it with compact metadata only:+ - file name+ - line count+ - maybe a short preview+ - content fingerprint / source hash+- Do not emit claims for raw file content.++Primary files:+- `memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala`+- `memory-layer/src/main/scala/ix/memory/ingestion/IngestionService.scala`+- `memory-layer/src/main/scala/ix/memory/ingestion/GraphPatchBuilder.scala`+- `memory-layer/src/main/scala/ix/memory/ingestion/ParseResult.scala`++### Failure Mode 5 — chunking is by file count, not payload size++Current behavior:+- Bulk writes use a fixed `chunkSize = 100`.+- A 100-file batch can still serialize into an oversized bulk request.++Required change:+- Chunk by estimated payload size, not just file count.+- Keep a file-count cap as a secondary guardrail.+- Emit progress based on committed slices, not only discovered totals.++Primary files:+- `memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala`+- `memory-layer/src/main/scala/ix/memory/db/BulkWriteApi.scala`+- `memory-layer/src/main/scala/ix/memory/db/ArangoClient.scala`++### Recommended ingestion order++1. Refactor bulk ingestion into bounded discovery/hash/parse/commit slices.+2. Chunk `getSourceHashes()` by slice.+3. Remove raw-content attrs and claims from generic text ingestion.+4. Narrow default discovery to parser-supported files.+5. Add payload-size-aware commit chunking.++This order addresses the actual failure mode first: unbounded memory retention.++---++## Constraints++- No new server endpoints.+- No LLM calls — explanations are template-driven, same as `ix explain`.+- `--explain` is ignored when `--list` is also passed (error message).+- `--explain` without a `<target>` is an error: `--explain requires a target.`+- Output is intentionally terse — one screen of text for the full explanation.+- Do not claim stale health in the output unless the score payload actually exposes revision+ metadata that proves staleness.+- Large-repo ingestion fixes should prefer bounded-memory behavior over preserving the current+ exact batching structure.diff --git a/ix-cli/src/cli/__tests__/subsystem-explain.test.ts b/ix-cli/src/cli/__tests__/subsystem-explain.test.tsnew file mode 100644index 00000000..ce292954--- /dev/null+++ b/ix-cli/src/cli/__tests__/subsystem-explain.test.ts@@ -0,0 +1,115 @@+import { describe, expect, it } from "vitest";+import {+ renderSubsystemExplanationJson,+ renderSubsystemExplanationText,+ type ScopedSubsystemResult,+ type ScopedSubsystemRegion,+ type SubsystemScore,+} from "../explain/subsystem.js";++function region(overrides: Partial<ScopedSubsystemRegion> = {}): ScopedSubsystemRegion {+ return {+ id: overrides.id ?? "region-1",+ label: overrides.label ?? "Ingestion Layer",+ level: overrides.level ?? 2,+ label_kind: overrides.label_kind ?? "subsystem",+ parent_id: overrides.parent_id ?? "parent-1",+ file_count: overrides.file_count ?? 34,+ confidence: overrides.confidence ?? 0.88,+ is_cross_cutting: overrides.is_cross_cutting ?? false,+ dominant_signals: overrides.dominant_signals ?? ["import coupling", "path proximity"],+ children: overrides.children,+ };+}++function scopedResult(overrides: Partial<ScopedSubsystemResult> = {}): ScopedSubsystemResult {+ return {+ target: overrides.target ?? region(),+ parent: overrides.parent ?? {+ id: "parent-1",+ label: "Core System",+ level: 3,+ label_kind: "system",+ },+ summary: overrides.summary ?? {+ well_defined: 3,+ moderate: 1,+ fuzzy: 0,+ cross_cutting: 0,+ },+ children: overrides.children ?? [+ region({ id: "child-1", label: "File Parser", level: 1, label_kind: "module", file_count: 12, confidence: 0.91 }),+ region({ id: "child-2", label: "Patch Builder", level: 1, label_kind: "module", file_count: 9, confidence: 0.84 }),+ region({ id: "child-3", label: "Bulk Writer", level: 1, label_kind: "module", file_count: 8, confidence: 0.66 }),+ region({ id: "child-4", label: "Loader", level: 1, label_kind: "module", file_count: 5, confidence: 0.44 }),+ ],+ hierarchy: overrides.hierarchy ?? region({+ children: overrides.children ?? [+ region({ id: "child-1", label: "File Parser", level: 1, label_kind: "module", file_count: 12, confidence: 0.91 }),+ ],+ }),+ };+}++function score(overrides: Partial<SubsystemScore> = {}): SubsystemScore {+ return {+ region_id: overrides.region_id ?? "region-1",+ name: overrides.name ?? "Ingestion Layer",+ level: overrides.level ?? 2,+ label_kind: overrides.label_kind ?? "subsystem",+ file_count: overrides.file_count ?? 34,+ health_score: overrides.health_score ?? 0.74,+ chunk_density: overrides.chunk_density ?? 5.1,+ smell_rate: overrides.smell_rate ?? 0.06,+ smell_files: overrides.smell_files ?? 2,+ total_chunks: overrides.total_chunks ?? 173,+ confidence: overrides.confidence ?? 0.88,+ inference_version: overrides.inference_version ?? "subsystem_v1",+ };+}++describe("subsystem explanation rendering", () => {+ it("renders structured json with composition and health", () => {+ const rendered = renderSubsystemExplanationJson(scopedResult(), score());++ expect(rendered.resolvedTarget.label).toBe("Ingestion Layer");+ expect(rendered.context.parent?.label).toBe("Core System");+ expect(rendered.composition?.children).toHaveLength(4);+ expect(rendered.health?.status).toBe("good");+ expect(rendered.notes).toEqual([]);+ expect(rendered.whyItMatters).toContain("ix impact");+ });++ it("omits health and adds a note when scores are missing", () => {+ const rendered = renderSubsystemExplanationJson(scopedResult(), null);++ expect(rendered.health).toBeNull();+ expect(rendered.notes).toEqual(["Run 'ix subsystems' to compute health scores for this region."]);+ });++ it("flags cross-cutting fuzzy subsystems as drift risk", () => {+ const rendered = renderSubsystemExplanationJson(+ scopedResult({+ target: region({+ label: "Shared Auth",+ confidence: 0.41,+ is_cross_cutting: true,+ }),+ }),+ score({ health_score: 0.39, smell_files: 3 }),+ );++ expect(rendered.whyItMatters).toContain("architectural drift");+ });++ it("renders text sections for a healthy subsystem explanation", () => {+ const rendered = renderSubsystemExplanationText(scopedResult(), score());++ expect(rendered).toContain("Explanation");+ expect(rendered).toContain("Context");+ expect(rendered).toContain("Composition");+ expect(rendered).toContain("Health");+ expect(rendered).toContain("Why it matters");+ expect(rendered).toContain("Ingestion Layer");+ });+});diff --git a/ix-cli/src/cli/commands/ingest.ts b/ix-cli/src/cli/commands/ingest.tsindex 966fa21f..0b53ed2e 100644--- a/ix-cli/src/cli/commands/ingest.ts+++ b/ix-cli/src/cli/commands/ingest.ts@@ -1,6 +1,7 @@import * as nodePath from 'node:path';import * as fs from 'node:fs';import * as crypto from 'node:crypto';+import * as os from 'node:os';import type { Command } from 'commander';import chalk from 'chalk';import { IxClient } from '../../client/api.js';@@ -21,6 +22,15 @@ import {// File discovery// ---------------------------------------------------------------------------+// Inline extension set — mirrors core-ingestion/dist/languages.js EXT_MAP.+// Kept here so file discovery does NOT require loading tree-sitter grammars.+const SUPPORTED_EXTENSIONS = new Set([+ '.ts', '.tsx', '.js', '.jsx', '.mjs', '.cjs',+ '.py', '.java', '.c', '.h', '.cpp', '.cc', '.cxx', '.hpp',+ '.cs', '.go', '.rb', '.rs', '.php', '.kt', '.kts', '.swift',+ '.scala', '.sc',+]);+const IGNORE_DIRS = new Set(['node_modules', '.git', 'dist', 'build', 'target', '.next','.cache', '__pycache__', '.ix', '.claude', '.gitnexus',@@ -31,7 +41,6 @@ const MAX_FILE_BYTES = 1024 * 1024; // 1 MBfunction* walkFiles(dir: string,recursive: boolean,- supportsFile: (fileName: string) => boolean): Generator<string> {let entries: fs.Dirent[];try { entries = fs.readdirSync(dir, { withFileTypes: true }); }@@ -41,9 +50,9 @@ function* walkFiles(if (IGNORE_DIRS.has(entry.name)) continue;const full = nodePath.join(dir, entry.name);if (entry.isDirectory()) {- if (recursive) yield* walkFiles(full, true, supportsFile);+ if (recursive) yield* walkFiles(full, true);} else if (entry.isFile()) {- if (supportsFile(entry.name)) yield full;+ if (SUPPORTED_EXTENSIONS.has(nodePath.extname(entry.name))) yield full;}}}@@ -52,6 +61,42 @@ function sha256(content: Buffer): string {return crypto.createHash('sha256').update(content).digest('hex');}+// ---------------------------------------------------------------------------+// Mtime cache — skip readFileSync+sha256 for unchanged files+// ---------------------------------------------------------------------------++interface MtimeCache {+ root: string;+ files: Record<string, number>; // absolute path → mtime (ms)+}++function mtimeCachePath(projectRoot: string): string {+ const key = crypto.createHash('sha256').update(projectRoot).digest('hex').slice(0, 12);+ return nodePath.join(os.homedir(), '.ix', `ingest_mtimes_${key}.json`);+}++function loadMtimeCache(projectRoot: string): Map<string, number> {+ try {+ const raw = fs.readFileSync(mtimeCachePath(projectRoot), 'utf-8');+ const data = JSON.parse(raw) as MtimeCache;+ if (data.root !== projectRoot) return new Map();+ return new Map(Object.entries(data.files));+ } catch {+ return new Map();+ }+}++function saveMtimeCache(projectRoot: string, mtimes: Map<string, number>): void {+ try {+ const dir = nodePath.join(os.homedir(), '.ix');+ fs.mkdirSync(dir, { recursive: true });+ const data: MtimeCache = { root: projectRoot, files: Object.fromEntries(mtimes) };+ fs.writeFileSync(mtimeCachePath(projectRoot), JSON.stringify(data));+ } catch {+ // Non-critical: ignore write errors+ }+}+// ---------------------------------------------------------------------------// Progress bar// ---------------------------------------------------------------------------@@ -118,9 +163,6 @@ export async function ingestFiles(const debug = opts.debug || process.env.IX_DEBUG === '1';const trueStart = performance.now();- const [{ parseFile, resolveEdges, isGrammarSupported }, { buildPatchWithResolution }] = await loadIngestionModules();- const moduleLoadMs = Math.round(performance.now() - trueStart);-const resolvedPath = nodePath.isAbsolute(path)? path: nodePath.resolve(resolveWorkspaceRoot(opts.root), path);@@ -131,8 +173,17 @@ export async function ingestFiles(let progressPhase = 'Scanning';let progressCurrent = 0;let progressTotal = 0;+ let progressStart = performance.now();+const interval = opts.format === 'text' ? setInterval(() => {- process.stderr.write('\r' + renderProgressLine(progressPhase, progressCurrent, progressTotal));+ const elapsed = performance.now() - progressStart;+ const tau = progressPhase === 'Saving' ? 3000 : 2000;+ let display = progressCurrent;+ if (progressTotal > 0 && progressCurrent < progressTotal) {+ const simulated = Math.floor((1 - Math.exp(-elapsed / tau)) * 0.88 * progressTotal);+ display = Math.max(display, simulated);+ }+ process.stderr.write('\r' + renderProgressLine(progressPhase, display, progressTotal));}, 80) : null;let filesDiscovered = 0;@@ -145,7 +196,7 @@ export async function ingestFiles(let entitiesParsed = 0;// Phase timing marks- const timings = { moduleLoadMs, discoverMs: 0, hashMs: 0, parseMs: 0, resolveMs: 0, commitMs: 0 };+ const timings = { moduleLoadMs: 0, discoverMs: 0, hashMs: 0, parseMs: 0, resolveMs: 0, commitMs: 0 };const resolveStats = {importLookups: 0, transitiveLookups: 0, globalFallbacks: 0,globalCandidateTotal: 0, resolvedImport: 0, resolvedTransitive: 0,@@ -156,8 +207,8 @@ export async function ingestFiles(// Phase: discover filesconst stat = fs.statSync(resolvedPath);const filePaths: string[] = stat.isFile()- ? [resolvedPath]- : Array.from(walkFiles(resolvedPath, opts.recursive ?? true, isGrammarSupported));+ ? (SUPPORTED_EXTENSIONS.has(nodePath.extname(resolvedPath)) ? [resolvedPath] : [])+ : Array.from(walkFiles(resolvedPath, opts.recursive ?? true));filesDiscovered = filePaths.length;const discovered = performance.now();@@ -166,120 +217,222 @@ export async function ingestFiles(progressPhase = 'Parsing';progressTotal = filePaths.length;progressCurrent = 0;-- // Phase: hash lookup (skipped on first ingest when no baseline exists)- let knownHashes: Map<string, string>;- let baselineSkipped = false;- if (opts.force) {- knownHashes = new Map();- } else {- const hasBaseline = await checkIngestBaseline(client, debug);- if (hasBaseline) {- if (debug) process.stderr.write('\n Existing ingest baseline found — performing source hash comparison\n');- knownHashes = await loadExistingHashes(client, filePaths, debug);- } else {- if (debug) process.stderr.write('\n No existing ingest baseline found — skipping source hash lookup\n');- knownHashes = new Map();- baselineSkipped = true;+ progressStart = performance.now();++ // Phase: mtime pre-filter — skip readFileSync+sha256 for files whose mtime+ // is unchanged since the last successful ingest (common "ix map" re-run case).+ const projectRoot = fs.statSync(resolvedPath).isDirectory() ? resolvedPath : nodePath.dirname(resolvedPath);+ const mtimeCache = opts.force ? new Map<string, number>() : loadMtimeCache(projectRoot);+ const currentMtimes = new Map<string, number>();++ // DB-reset guard: if the mtime cache has entries but the server returns no hashes+ // for a small sample, the DB was wiped (e.g. `ix reset` run from a different dir+ // so the cache wasn't cleared). Invalidate the cache so files are re-ingested.+ if (!opts.force && mtimeCache.size > 0) {+ const samplePaths = [...mtimeCache.keys()].slice(0, 5);+ const sampleHashes = await loadExistingHashes(client, samplePaths, debug);+ if (sampleHashes.size === 0) {+ mtimeCache.clear();+ if (debug) process.stderr.write(`\n DB reset detected — invalidating mtime cache\n`);}}- const hashed = performance.now();- timings.hashMs = Math.round(hashed - discovered);-- // Phase: parse all files- type ParsedFile = { filePath: string; parsed: any; hash: string; previousHash: string | undefined };- const parsedFiles: ParsedFile[] = [];+ // Stat all files and partition into mtime-clean (skip) and mtime-changed (need hash check).+ const mtimeChangedPaths: string[] = [];for (const filePath of filePaths) {- progressCurrent++;- if (interval) process.stderr.write('\r' + renderProgressLine(progressPhase, progressCurrent, progressTotal));try {- const fileSize = fs.statSync(filePath).size;- if (fileSize === 0) { filesSkipped++; continue; }- if (fileSize > MAX_FILE_BYTES) { tooLarge++; continue; }-- const bytes = fs.readFileSync(filePath);- const hash = sha256(bytes);-- if (!opts.force && knownHashes.get(filePath) === hash) {- filesSkipped++;- continue;- }-- const source = bytes.toString('utf-8');- const parsed = parseFile(filePath, source);-- if (!parsed) {- filesSkipped++;- continue;+ const st = fs.statSync(filePath);+ if (st.size === 0) { filesSkipped++; continue; }+ if (st.size > MAX_FILE_BYTES) { tooLarge++; continue; }+ const mtime = st.mtimeMs;+ currentMtimes.set(filePath, mtime);+ if (!opts.force && mtimeCache.get(filePath) === mtime) {+ filesSkipped++; // mtime clean — assume unchanged+ } else {+ mtimeChangedPaths.push(filePath);}-- entitiesParsed += parsed.entities.length;- const previousHash = knownHashes.get(filePath);- parsedFiles.push({ filePath, parsed, hash, previousHash: previousHash !== hash ? previousHash : undefined });} catch (err) {parseErrors++;- process.stderr.write(`\n [parse error] ${filePath}: ${err}\n`);+ process.stderr.write(`\n [stat error] ${filePath}: ${err}\n`);}}- filesChanged = parsedFiles.length;- const parsed = performance.now();- timings.parseMs = Math.round(parsed - hashed);-- // Phase: cross-file CALLS + EXTENDS resolution over the full batch- const resolvedEdges = (resolveEdges as Function)(parsedFiles.map(f => f.parsed), resolveStats);- const resolved = performance.now();- timings.resolveMs = Math.round(resolved - parsed);-- // Pre-group resolved edges by source file for O(1) lookup in patch building- const edgesByFile = new Map<string, typeof resolvedEdges>();- for (const edge of resolvedEdges) {- let arr = edgesByFile.get(edge.srcFilePath);- if (!arr) { arr = []; edgesByFile.set(edge.srcFilePath, arr); }- arr.push(edge);+ // Phase: hash lookup — only needed when mtime-changed files exist.+ let knownHashes: Map<string, string>;+ if (opts.force || mtimeChangedPaths.length > 0) {+ knownHashes = await loadExistingHashes(client, opts.force ? filePaths : mtimeChangedPaths, debug);+ if (debug) process.stderr.write(`\n Source hash lookup: ${knownHashes.size} known hashes (${mtimeChangedPaths.length} mtime-changed)\n`);+ } else {+ // All files are mtime-clean — skip server round-trip entirely.+ knownHashes = new Map();+ if (debug) process.stderr.write(`\n All ${filePaths.length} files mtime-clean — skipping hash lookup\n`);}- const emptyEdges: typeof resolvedEdges = [];+ const hashed = performance.now();+ timings.hashMs = Math.round(hashed - discovered);- // Phase: build patches and bulk commit- const patches: GraphPatchPayload[] = [];- for (const { parsed: p, hash, previousHash } of parsedFiles) {- try {- patches.push(buildPatchWithResolution(p, hash, edgesByFile.get(p.filePath) ?? emptyEdges, previousHash));- } catch (err) {- parseErrors++;- process.stderr.write(`\n [patch build error] ${p.filePath}: ${err}\n`);+ // Phase: parse + commit — streaming to bound peak memory.+ //+ // Files are parsed in PARSE_STREAM_CHUNK-sized batches. After each batch,+ // resolveEdges runs on that batch and patches are committed before the next+ // batch is parsed. This keeps heap usage at O(chunk) instead of O(total).+ //+ // YIELD_EVERY yields the Node.js event loop so the progress bar timer can+ // fire even during the synchronous tree-sitter parse calls.+ //+ // Two paths:+ // A) Has baseline or mtime-changed files → pre-scan first, skip module load+ // entirely if nothing changed (common "ix map" re-run case).+ // B) No baseline / force → load modules, parse + stream-commit all files.++ type ParsedFile = { filePath: string; parsed: any; hash: string; previousHash: string | undefined };+ let resolveEdgesFn: Function | null = null;+ let buildPatchFn: Function | null = null;++ const PARSE_STREAM_CHUNK = 1000; // resolve + commit after this many parsed files+ const COMMIT_CHUNK_SIZE = 200; // files per HTTP batch to server+ const YIELD_EVERY = 100; // yield event loop every N files during parse++ const emptyEdges: any[] = [];++ /** Resolve edges within a batch, build patches, and commit in sub-chunks. */+ const flushBatch = async (batch: ParsedFile[]): Promise<void> => {+ if (batch.length === 0) return;+ filesChanged += batch.length;++ const resolvedEdges = resolveEdgesFn!(batch.map(f => f.parsed), resolveStats);+ const batchEdgesByFile = new Map<string, any[]>();+ for (const edge of resolvedEdges) {+ let arr = batchEdgesByFile.get(edge.srcFilePath);+ if (!arr) { arr = []; batchEdgesByFile.set(edge.srcFilePath, arr); }+ arr.push(edge);}- }- if (patches.length > 0) {- progressPhase = 'Saving';- progressTotal = patches.length;- progressCurrent = 0;- try {- const result = await client.commitPatchBulk(patches);- latestRev = result.rev;- patchesApplied = patches.length;- progressCurrent = patches.length;- } catch (err) {- // Fallback: try per-file commits if bulk endpoint fails- if (debug) process.stderr.write(`\n [bulk commit failed, falling back to per-file] ${err}\n`);- for (const patch of patches) {+ for (let i = 0; i < batch.length; i += COMMIT_CHUNK_SIZE) {+ const end = Math.min(i + COMMIT_CHUNK_SIZE, batch.length);+ const patchChunk: GraphPatchPayload[] = [];+ for (let j = i; j < end; j++) {+ const { parsed: p, hash, previousHash } = batch[j];try {- const result = await client.commitPatch(patch);- if (result.rev > latestRev) latestRev = result.rev;- patchesApplied++;+ patchChunk.push(buildPatchFn!(p, hash, batchEdgesByFile.get(p.filePath) ?? emptyEdges, previousHash));+ } catch (err) {+ parseErrors++;+ process.stderr.write(`\n [patch build error] ${p.filePath}: ${err}\n`);+ }+ }+ if (patchChunk.length === 0) continue;+ try {+ const result = await client.commitPatchBulk(patchChunk);+ if (result.rev > latestRev) latestRev = result.rev;+ patchesApplied += patchChunk.length;+ } catch (err) {+ if (debug) process.stderr.write(`\n [bulk chunk failed, falling back to per-file] ${err}\n`);+ for (const patch of patchChunk) {+ try {+ const result = await client.commitPatch(patch);+ if (result.rev > latestRev) latestRev = result.rev;+ patchesApplied++;+ } catch (commitErr) {+ parseErrors++;+ process.stderr.write(`\n [commit error] ${patch.source?.uri}: ${commitErr}\n`);+ }+ }+ }+ }+ };++ if ((knownHashes.size > 0 || mtimeChangedPaths.length === 0) && !opts.force) {+ // Path A: has baseline or all mtime-clean → pre-scan to detect changes before loading modules.+ // If nothing changed, module load is skipped entirely.+ const changedPaths: Array<{ filePath: string; bytes: Buffer; hash: string; previousHash: string | undefined }> = [];+ for (const filePath of mtimeChangedPaths) {+ try {+ const bytes = fs.readFileSync(filePath);+ const hash = sha256(bytes);+ if (knownHashes.get(filePath) === hash) { filesSkipped++; continue; }+ const previousHash = knownHashes.get(filePath);+ changedPaths.push({ filePath, bytes, hash, previousHash: previousHash !== hash ? previousHash : undefined });+ } catch (err) {+ parseErrors++;+ process.stderr.write(`\n [read error] ${filePath}: ${err}\n`);+ }+ }++ if (changedPaths.length > 0) {+ const moduleStart = performance.now();+ const [ingestion, patchBuilder] = await loadIngestionModules();+ timings.moduleLoadMs = Math.round(performance.now() - moduleStart);+ resolveEdgesFn = ingestion.resolveEdges as Function;+ buildPatchFn = patchBuilder.buildPatchWithResolution as Function;++ let batch: ParsedFile[] = [];+ for (let i = 0; i < changedPaths.length; i++) {+ const { filePath, bytes, hash, previousHash } = changedPaths[i];+ try {+ const parsed = (ingestion.parseFile as Function)(filePath, bytes.toString('utf-8'));+ if (!parsed) { filesSkipped++; progressCurrent++; continue; }+ entitiesParsed += parsed.entities.length;+ batch.push({ filePath, parsed, hash, previousHash });progressCurrent++;- } catch (commitErr) {+ } catch (err) {parseErrors++;- process.stderr.write(`\n [commit error] ${patch.source?.uri}: ${commitErr}\n`);+ progressCurrent++;+ process.stderr.write(`\n [parse error] ${filePath}: ${err}\n`);}+ if ((i + 1) % YIELD_EVERY === 0) await new Promise<void>(resolve => setImmediate(resolve));+ if (batch.length >= PARSE_STREAM_CHUNK) { await flushBatch(batch); batch = []; }}+ await flushBatch(batch);}+ } else {+ // Path B: no baseline (first ingest) or --force → load modules, then stream parse + commit.+ const moduleStart = performance.now();+ const [ingestion, patchBuilder] = await loadIngestionModules();+ timings.moduleLoadMs = Math.round(performance.now() - moduleStart);+ resolveEdgesFn = ingestion.resolveEdges as Function;+ buildPatchFn = patchBuilder.buildPatchWithResolution as Function;++ let batch: ParsedFile[] = [];+ for (let i = 0; i < filePaths.length; i++) {+ const filePath = filePaths[i];+ try {+ const fileSize = fs.statSync(filePath).size;+ if (fileSize === 0) { filesSkipped++; progressCurrent++; continue; }+ if (fileSize > MAX_FILE_BYTES) { tooLarge++; progressCurrent++; continue; }+ const bytes = fs.readFileSync(filePath);+ const hash = sha256(bytes);+ if (!opts.force && knownHashes.get(filePath) === hash) { filesSkipped++; progressCurrent++; continue; }+ const previousHash = knownHashes.get(filePath);+ const parsed = (ingestion.parseFile as Function)(filePath, bytes.toString('utf-8'));+ if (!parsed) { filesSkipped++; progressCurrent++; continue; }+ entitiesParsed += parsed.entities.length;+ batch.push({ filePath, parsed, hash, previousHash: previousHash !== hash ? previousHash : undefined });+ progressCurrent++;+ } catch (err) {+ parseErrors++;+ progressCurrent++;+ process.stderr.write(`\n [parse error] ${filePath}: ${err}\n`);+ }+ if ((i + 1) % YIELD_EVERY === 0) await new Promise<void>(resolve => setImmediate(resolve));+ if (batch.length >= PARSE_STREAM_CHUNK) { await flushBatch(batch); batch = []; }+ }+ await flushBatch(batch);}+ const parsed = performance.now();+ timings.parseMs = Math.round(parsed - hashed);+ timings.resolveMs = 0; // now interleaved with parse+ timings.commitMs = 0; // now interleaved with parse++ // Clear the parse bar+ if (interval) process.stderr.write('\r' + ' '.repeat(PROG_LINE_WIDTH) + '\r');+const committed = performance.now();- timings.commitMs = Math.round(committed - resolved);++ // Persist mtime cache so next run can skip unchanged files quickly.+ // Only save when no parse errors (avoid poisoning cache on partial failures).+ if (!opts.force && parseErrors === 0 && currentMtimes.size > 0) {+ saveMtimeCache(projectRoot, currentMtimes);+ }} finally {if (interval) {clearInterval(interval);@@ -287,6 +440,8 @@ export async function ingestFiles(process.stderr.write('\r' + renderProgressLine(progressPhase, progressTotal, progressTotal));}process.stderr.write('\r' + ' '.repeat(PROG_LINE_WIDTH) + '\r');+ const elapsedSec = ((performance.now() - start) / 1000).toFixed(1);+ process.stderr.write(chalk.dim(` Ingested in ${elapsedSec}s\n`));}}@@ -350,15 +505,6 @@ export async function ingestFiles(// Load existing hashes from the server for change detection// ----------------------------------------------------------------------------async function checkIngestBaseline(client: IxClient, debug = false): Promise<boolean> {- try {- return await client.hasIngestBaseline();- } catch (err) {- if (debug) process.stderr.write(`\n [baseline check failed, falling back to full hash lookup] ${err}\n`);- return true; // Safe fallback: assume baseline exists, do the full lookup- }-}-async function loadExistingHashes(client: IxClient, filePaths: string[], debug = false): Promise<Map<string, string>> {try {return await client.getSourceHashes(filePaths);diff --git a/ix-cli/src/cli/commands/map.ts b/ix-cli/src/cli/commands/map.tsindex 8486aa0f..3388ba71 100644--- a/ix-cli/src/cli/commands/map.ts+++ b/ix-cli/src/cli/commands/map.ts@@ -5,9 +5,8 @@ import { IxClient } from "../../client/api.js";import { getEndpoint } from "../config.js";import { bootstrap } from "../bootstrap.js";import { ingestFiles } from "./ingest.js";-import { parseBackendError, renderStructuredError } from "../errors.js";-interface MapRegion {+export interface MapRegion {id: string;label: string;label_kind: string;@@ -54,7 +53,7 @@ interface MapPersistence {total_ops: number;}-interface MapResult {+export interface MapResult {file_count: number;region_count: number;levels: number;@@ -66,6 +65,18 @@ interface MapResult {persistence?: MapPersistence;}+type MapSortMode = "importance" | "confidence" | "size" | "alpha";+export interface MapTextRenderOptions {+ level?: string;+ minConfidence: string;+ maxItems: string;+ allItems?: boolean;+ sort: string;+ graph?: boolean;+ list?: boolean;+ verbose?: boolean;+}+export function registerMapCommand(program: Command): void {program.command("map [path]")@@ -73,6 +84,11 @@ export function registerMapCommand(program: Command): void {.option("--format <fmt>", "Output format (text|json)", "text").option("--level <n>", "Show only regions at this level (1=finest, higher=coarser)").option("--min-confidence <n>", "Only show regions above this confidence threshold (0-1)", "0")+ .option("--max-items <n>", "Max items to show per section in text output (default: 10)", "10")+ .option("--all-items", "Show all items in each section (overrides --max-items)")+ .option("--sort <mode>", "Sort mode for text output (importance|confidence|size|alpha)", "importance")+ .option("--graph", "Render the hierarchy as a graph/tree view (default)")+ .option("--list", "Render the ranked list view instead of the default graph/tree view").option("--full", "Force full local map, bypassing automatic safety limits (advanced/testing)").option("--verbose", "Show raw confidence scores, crosscut scores, boundary ratios, and signals").addHelpText(@@ -97,10 +113,15 @@ Examples:ix map --format jsonix map --level 2ix map --min-confidence 0.5+ ix map --max-items 10+ ix map --sort confidence+ ix map --graph+ ix map --list+ ix map --all-itemsix map . --fullix --debug map . --full`)- .action(async (pathArg: string | undefined, opts: { format: string; level?: string; minConfidence: string; full?: boolean; verbose?: boolean }) => {+ .action(async (pathArg: string | undefined, opts: { format: string; level?: string; minConfidence: string; maxItems: string; allItems?: boolean; sort: string; graph?: boolean; list?: boolean; full?: boolean; verbose?: boolean }) => {const cwd = pathArg ? resolve(pathArg) : process.cwd();try {@@ -120,31 +141,43 @@ Examples:}// Ingest the path before mapping so the graph is up to date+ const ingestStart = performance.now();await ingestFiles(cwd, { recursive: true, format: opts.format === "json" ? "json" : "text", printSummary: false });+ const ingestMs = Math.round(performance.now() - ingestStart);const client = new IxClient(getEndpoint());- if (opts.format !== "json") {- process.stderr.write(chalk.dim(" Computing map...\n"));- }+ const mapBarWidth = 25;+ const mapStart = performance.now();+ const mapInterval = opts.format !== "json" ? setInterval(() => {+ const elapsed = performance.now() - mapStart;+ const pct = 1 - Math.exp(-elapsed / 4000);+ const filled = Math.round(pct * mapBarWidth);+ const bar = chalk.cyan('█'.repeat(filled)) + chalk.dim('░'.repeat(mapBarWidth - filled));+ const pctStr = chalk.cyan(`${Math.min(Math.round(pct * 100), 99)}%`.padStart(4));+ process.stderr.write(`\r Computing map... ${bar} ${pctStr}`);+ }, 80) : null;let result: MapResult;try {result = await client.map({ full: opts.full }) as MapResult;} catch (err: any) {- const msg: string = err.message ?? "";- const structured = parseBackendError(msg);- if (structured) {- renderStructuredError(structured);- } else {- console.error(chalk.red("Error:"), msg);- }+ if (mapInterval) { clearInterval(mapInterval); process.stderr.write('\r' + ' '.repeat(60) + '\r'); }+ console.error(chalk.red("Error:"), err.message ?? err);process.exitCode = 1;return;}+ if (mapInterval) { clearInterval(mapInterval); process.stderr.write('\r' + ' '.repeat(60) + '\r'); }+ const mapMs = Math.round(performance.now() - mapStart);+ if (opts.format !== "json") {+ process.stderr.write(chalk.dim(` ingest ${ingestMs}ms · map ${mapMs}ms\n`));+ }const minConf = parseFloat(opts.minConfidence ?? "0");const levelFilter = opts.level ? parseInt(opts.level, 10) : null;+ const parsedMaxItems = parseInt(opts.maxItems ?? "10", 10);+ const maxItems = Number.isFinite(parsedMaxItems) && parsedMaxItems > 0 ? parsedMaxItems : 10;+ const sortMode = normalizeSortMode(opts.sort);let regions = result.regions;if (levelFilter !== null) regions = regions.filter(r => r.level === levelFilter);@@ -154,105 +187,82 @@ Examples:console.log(JSON.stringify({ ...result, regions }, null, 2));return;}+ renderMapText(result, cwd, opts);+ });+}- // Text output- console.log(- `\n${chalk.bold("Architectural Map")} — ` +- `${result.file_count} files · ${result.region_count} regions`- );+export function renderMapText(result: MapResult, cwd: string, opts: MapTextRenderOptions): void {+ const minConf = parseFloat(opts.minConfidence ?? "0");+ const levelFilter = opts.level ? parseInt(opts.level, 10) : null;+ const parsedMaxItems = parseInt(opts.maxItems ?? "10", 10);+ const maxItems = Number.isFinite(parsedMaxItems) && parsedMaxItems > 0 ? parsedMaxItems : 10;+ const sortMode = normalizeSortMode(opts.sort);+ const showGraph = !opts.list;- if (result.outcome === "fast_local_completed") {- console.log(chalk.yellow(" Large system detected") + chalk.dim(" — using Fast Map"));- console.log(chalk.dim(" Reduced coupling model with full region hierarchy output."));- }+ let regions = result.regions;+ if (levelFilter !== null) regions = regions.filter(r => r.level === levelFilter);+ if (minConf > 0) regions = regions.filter(r => r.confidence >= minConf);- if (regions.length === 0) {- console.log(chalk.dim("\n No regions found matching filters."));- return;- }+ console.log(+ `\n${chalk.bold("Architectural Map")} — ` ++ `${result.file_count} files · ${result.region_count} regions`+ );+ const topSystem = pickTopSystemName(result.regions, cwd);+ if (topSystem) {+ console.log(chalk.dim(`System: ${topSystem}`));+ }- // Build id→region lookup for parent label resolution- const regionById = new Map(result.regions.map(r => [r.id, r]));-- const KIND_ORDER = ["system", "subsystem", "module"];- const KIND_HEADERS: Record<string, string> = {- system: "Systems",- subsystem: "Subsystems",- module: "Modules",- };- const CROSSCUT_THRESHOLD = 0.10;- const RULE = chalk.dim("─".repeat(58));-- // Group by label_kind- const byKind = new Map<string, MapRegion[]>();- for (const r of regions) {- if (!byKind.has(r.label_kind)) byKind.set(r.label_kind, []);- byKind.get(r.label_kind)!.push(r);- }+ if (result.outcome === "fast_local_completed") {+ console.log(chalk.yellow(" Large system detected") + chalk.dim(" — using Fast Map"));+ console.log(chalk.dim(" Reduced coupling model with full region hierarchy output."));+ }- for (const kind of KIND_ORDER) {- const kindRegions = byKind.get(kind);- if (!kindRegions) continue;-- // Sort: systems/subsystems by file_count desc; modules: problematic first, then parent+size- if (kind !== "module") {- kindRegions.sort((a, b) => b.file_count - a.file_count);- } else {- const isProblematic = (r: MapRegion) =>- r.crosscut_score > CROSSCUT_THRESHOLD || r.confidence < 0.50;- kindRegions.sort((a, b) => {- const ap = isProblematic(a) ? 1 : 0;- const bp = isProblematic(b) ? 1 : 0;- if (bp !== ap) return bp - ap;- const aParent = a.parent_id ? (regionById.get(a.parent_id)?.label ?? "") : "";- const bParent = b.parent_id ? (regionById.get(b.parent_id)?.label ?? "") : "";- if (aParent !== bParent) return aParent.localeCompare(bParent);- return b.file_count - a.file_count;- });- }-- const colHeader = kind === "module"- ? chalk.dim(` ${"Name".padEnd(22)} ${"Files".padStart(8)} ${"Clarity".padEnd(13)} System`)- : chalk.dim(` ${"Name".padEnd(22)} ${"Files".padStart(8)} Clarity`);-- console.log(`\n${chalk.bold(KIND_HEADERS[kind] ?? kind)}`);- console.log(colHeader);- console.log(RULE);-- if (opts.verbose) {- for (const r of kindRegions) {- const bar = confidenceBar(r.confidence);- const name = chalk.bold(r.label.padEnd(22));- const files = chalk.dim(`${r.file_count} files`.padStart(8));- const confPct = chalk.cyan(`${Math.round(r.confidence * 100)}%`.padStart(4));- const xcut = chalk.dim(`xcut=${r.crosscut_score.toFixed(2)}`);- const br = chalk.dim(`br=${Math.min(r.boundary_ratio, 999.9).toFixed(1)}`);- const signals = chalk.dim(r.dominant_signals.slice(0, 3).join("+") || "—");- const parentLabel = r.parent_id ? regionById.get(r.parent_id)?.label : null;- const parentStr = (kind === "module" && parentLabel) ? chalk.dim(` (${parentLabel})`) : "";- const crossStr = r.crosscut_score > CROSSCUT_THRESHOLD ? chalk.yellow(" ⚠ cross-cutting") : "";- console.log(` ${bar} ${name} ${files} ${confPct} ${xcut} ${br} ${signals}${parentStr}${crossStr}`);- }- } else {- for (const r of kindRegions) {- const label = confidenceLabel(r.confidence);- const labelColor = r.confidence >= 0.75 ? chalk.green : r.confidence >= 0.50 ? chalk.yellow : chalk.red;- const name = chalk.bold(r.label.padEnd(22));- const files = chalk.dim(`${r.file_count} files`.padStart(8));- const parentLabel = r.parent_id ? regionById.get(r.parent_id)?.label : null;- const parentStr = (kind === "module" && parentLabel) ? chalk.dim(` (${parentLabel})`) : "";- const crossStr = r.crosscut_score > CROSSCUT_THRESHOLD ? chalk.yellow(" ⚠ cross-cutting") : "";- console.log(` ${name} ${files} ${labelColor(label.padEnd(13))}${parentStr}${crossStr}`);- }- }- }+ if (regions.length === 0) {+ console.log(chalk.dim("\n No regions found matching filters."));+ return;+ }++ const regionById = new Map(result.regions.map(r => [r.id, r]));+ const CROSSCUT_THRESHOLD = 0.10;+ const systemsCount = regions.filter(r => r.level === 3).length;+ const subsystemsCount = regions.filter(r => r.level === 2).length;+ const modulesCount = regions.filter(r => r.level === 1).length;+ const wellDefined = regions.filter(r => r.confidence >= 0.75).length;+ const moderate = regions.filter(r => r.confidence >= 0.50 && r.confidence < 0.75).length;+ const fuzzy = regions.filter(r => r.confidence < 0.50).length;+ const crossCutting = regions.filter(r => r.crosscut_score > CROSSCUT_THRESHOLD).length;- console.log(chalk.dim(`\nLegend: cross-cutting = spans multiple subsystems.`));- if (!opts.verbose) {- console.log(chalk.dim(`Run 'ix map --verbose' for confidence scores and raw metrics.`));+ console.log(chalk.dim(+ `Scope: ${systemsCount} systems · ${subsystemsCount} subsystems · ${modulesCount} modules`+ ));+ console.log(chalk.dim(+ `Clarity: ${wellDefined} well-defined · ${moderate} moderate · ${fuzzy} fuzzy · ${crossCutting} cross-cutting`+ ));++ if (levelFilter === null) {+ if (showGraph) {+ if (!opts.allItems) {+ console.log(chalk.dim(`Showing up to ${maxItems} branches per level. Use --all-items to show everything.`));}- console.log();- });+ renderMapTree(regions, maxItems, Boolean(opts.allItems), Boolean(opts.verbose), sortMode);+ } else {+ if (!opts.allItems) {+ console.log(chalk.dim(`Showing the top ${maxItems} subsystems and the top ${maxItems} modules drawn from those subsystems. Use --all-items to show everything.`));+ }+ renderRankedList(regions, regionById, maxItems, Boolean(opts.allItems), Boolean(opts.verbose), sortMode);+ }+ } else {+ if (!opts.allItems) {+ console.log(chalk.dim(`Showing up to ${maxItems} items. Use --all-items to show everything.`));+ }+ renderLevelList(regions, regionById, levelFilter, maxItems, Boolean(opts.allItems), Boolean(opts.verbose), sortMode);+ }++ console.log(chalk.dim(`\nLegend: cross-cutting = spans multiple subsystems.`));+ if (!opts.verbose) {+ console.log(chalk.dim(`Run 'ix map --verbose' for confidence scores and raw metrics. Use --list for the ranked view.`));+ }+ console.log();}/** Render a confidence score as a compact bar: ████░░ (used in --verbose mode) */@@ -269,3 +279,272 @@ function confidenceLabel(conf: number): string {if (conf >= 0.50) return "Moderate";return "Fuzzy";}++function pickTopSystemName(regions: MapRegion[], cwd: string): string {+ const systems = regions+ .filter(r => r.level === 3 || r.label_kind === "system")+ .slice()+ .sort((a, b) => b.file_count - a.file_count || b.confidence - a.confidence);++ if (systems.length > 0 && systems[0].label.trim().length > 0) {+ return systems[0].label;+ }++ const inferred = cwd.split(/[\\/]/).filter(Boolean).pop() ?? "System";+ const compact = inferred.replace(/[^a-zA-Z0-9]+/g, " ").trim();+ if (compact.length === 0) return "System";+ const parts = compact.split(/\s+/);+ if (parts.some(p => p.toLowerCase() === "ix")) return "IX";+ if (compact.length <= 4) return compact.toUpperCase();+ return parts+ .map(part => part[0].toUpperCase() + part.slice(1).toLowerCase())+ .join(" ");+}++function normalizeSortMode(input: string | undefined): MapSortMode {+ switch ((input ?? "importance").toLowerCase()) {+ case "confidence":+ return "confidence";+ case "size":+ return "size";+ case "alpha":+ return "alpha";+ default:+ return "importance";+ }+}++function compareRegions(a: MapRegion, b: MapRegion, mode: MapSortMode): number {+ if (mode === "alpha") {+ return a.label.localeCompare(b.label) || b.file_count - a.file_count || b.confidence - a.confidence;+ }+ if (mode === "confidence") {+ return b.confidence - a.confidence || b.file_count - a.file_count || a.label.localeCompare(b.label);+ }+ return b.file_count - a.file_count || b.confidence - a.confidence || a.label.localeCompare(b.label);+}++function sortRegions(regions: MapRegion[], mode: MapSortMode): MapRegion[] {+ return [...regions].sort((a, b) => compareRegions(a, b, mode));+}++function renderMapTree(+ regions: MapRegion[],+ maxItems: number,+ allItems: boolean,+ verbose: boolean,+ sortMode: MapSortMode,+): void {+ const regionById = new Map(regions.map(region => [region.id, region]));+ const childrenByParent = new Map<string, MapRegion[]>();+ for (const region of regions) {+ if (!region.parent_id) continue;+ const existing = childrenByParent.get(region.parent_id) ?? [];+ existing.push(region);+ childrenByParent.set(region.parent_id, existing);+ }++ const roots = sortRegions(+ regions.filter(region => region.level === 3 || region.parent_id === null || !regionById.has(region.parent_id)),+ sortMode,+ );+ const shownRoots = allItems ? roots : roots.slice(0, maxItems);++ console.log(`\n${chalk.bold("Architecture Graph")}`);+ console.log(chalk.dim(" System → subsystem → module"));++ for (const root of shownRoots) {+ console.log();+ renderTreeNode(root, "", true, 0, childrenByParent, maxItems, allItems, verbose, sortMode);+ }++ if (!allItems && roots.length > shownRoots.length) {+ const remaining = roots.length - shownRoots.length;+ console.log(chalk.dim(`\n ... ${remaining} more top-level branches. Use --all-items to show all.`));+ }+}++function renderTreeNode(+ region: MapRegion,+ prefix: string,+ isLast: boolean,+ depth: number,+ childrenByParent: Map<string, MapRegion[]>,+ maxItems: number,+ allItems: boolean,+ verbose: boolean,+ sortMode: MapSortMode,+): void {+ const branch = depth === 0 ? "●" : isLast ? "└─" : "├─";+ console.log(`${prefix}${branch} ${formatRegionLine(region, verbose, depth)}`);++ const children = sortRegions(childrenByParent.get(region.id) ?? [], sortMode);+ const shownChildren = allItems ? children : children.slice(0, maxItems);+ const nextPrefix = depth === 0 ? " " : `${prefix}${isLast ? " " : "│ "}`;++ shownChildren.forEach((child, index) => {+ renderTreeNode(+ child,+ nextPrefix,+ index === shownChildren.length - 1,+ depth + 1,+ childrenByParent,+ maxItems,+ allItems,+ verbose,+ sortMode,+ );+ });++ if (!allItems && children.length > shownChildren.length) {+ const remaining = children.length - shownChildren.length;+ console.log(chalk.dim(`${nextPrefix}… ${remaining} more branches under ${region.label}`));+ }+}++function renderRankedList(+ regions: MapRegion[],+ regionById: Map<string, MapRegion>,+ maxItems: number,+ allItems: boolean,+ verbose: boolean,+ sortMode: MapSortMode,+): void {+ const subsystems = sortRegions(regions.filter(r => r.level === 2), sortMode);+ const shownSubsystems = allItems ? subsystems : subsystems.slice(0, maxItems);+ const shownSubsystemIds = new Set(shownSubsystems.map(region => region.id));+ const candidateModules = sortRegions(+ regions.filter(r => r.level === 1 && (+ shownSubsystemIds.size === 0 ||+ (r.parent_id !== null && shownSubsystemIds.has(r.parent_id))+ )),+ sortMode,+ );+ const shownModules = allItems ? candidateModules : candidateModules.slice(0, maxItems);++ if (shownSubsystems.length > 0) {+ console.log(`\n${chalk.bold("Subsystems")}`);+ printAlignedRows(shownSubsystems.map(region => ({ region, parentLabel: null })), verbose, false);+ if (!allItems && subsystems.length > shownSubsystems.length) {+ console.log(chalk.dim(` ... ${subsystems.length - shownSubsystems.length} more subsystems. Use --all-items to show all.`));+ }+ }++ if (shownModules.length > 0) {+ console.log(`\n${chalk.bold("Modules")}`);+ printAlignedRows(+ shownModules.map(region => ({+ region,+ parentLabel: region.parent_id ? regionById.get(region.parent_id)?.label ?? "Unknown" : "Unknown",+ })),+ verbose,+ true,+ );+ if (!allItems && candidateModules.length > shownModules.length) {+ console.log(chalk.dim(` ... ${candidateModules.length - shownModules.length} more modules from the shown subsystems. Use --all-items to show all.`));+ }+ }++ if (shownSubsystems.length === 0 && shownModules.length === 0) {+ console.log(chalk.dim("\n No ranked subsystem or module regions found."));+ }+}++function renderLevelList(+ regions: MapRegion[],+ regionById: Map<string, MapRegion>,+ level: number,+ maxItems: number,+ allItems: boolean,+ verbose: boolean,+ sortMode: MapSortMode,+): void {+ const label = level === 3 ? "Systems" : level === 2 ? "Subsystems" : "Modules";+ const ranked = sortRegions(regions, sortMode);+ const shown = allItems ? ranked : ranked.slice(0, maxItems);++ console.log(`\n${chalk.bold(label)}`);+ printAlignedRows(+ shown.map(region => ({+ region,+ parentLabel: level === 1 && region.parent_id ? (regionById.get(region.parent_id)?.label ?? "Unknown") : null,+ })),+ verbose,+ level === 1,+ );++ if (!allItems && ranked.length > shown.length) {+ const remaining = ranked.length - shown.length;+ console.log(chalk.dim(` ... ${remaining} more ${label.toLowerCase()}. Use --all-items to show all.`));+ }+}++function printAlignedRows(+ rows: Array<{ region: MapRegion; parentLabel: string | null }>,+ verbose: boolean,+ includeParent: boolean,+): void {+ const nameWidth = Math.max("Name".length, ...rows.map(({ region }) => region.label.length));+ const filesWidth = Math.max("Files".length, ...rows.map(({ region }) => String(region.file_count).length));+ const confidenceWidth = Math.max(+ "Confidence".length,+ ...rows.map(({ region }) => `${Math.round(region.confidence * 100)}% ${confidenceLabel(region.confidence)}`.length),+ );+ const parentWidth = includeParent+ ? Math.max("Parent".length, ...rows.map(({ parentLabel }) => (parentLabel ?? "-").length))+ : 0;+ const crossWidth = Math.max("Cross".length, ...rows.map(({ region }) => region.crosscut_score > 0.10 ? 3 : 2));++ let header = ` ${"Name".padEnd(nameWidth)} ${"Files".padStart(filesWidth)} ${"Confidence".padEnd(confidenceWidth)}`;+ if (includeParent) {+ header += ` ${"Parent".padEnd(parentWidth)}`;+ }+ header += ` ${"Cross".padEnd(crossWidth)}`;+ console.log(chalk.dim(header));+ console.log(chalk.dim(` ${"─".repeat(Math.max(20, visibleWidth(header.trim())))}`));++ for (const { region, parentLabel } of rows) {+ const confidenceText = `${Math.round(region.confidence * 100)}% ${confidenceLabel(region.confidence)}`;+ const confidenceColor = region.confidence >= 0.75 ? chalk.green : region.confidence >= 0.50 ? chalk.yellow : chalk.red;+ const crossText = region.crosscut_score > 0.10 ? "yes" : "no";++ let line = ` ${chalk.bold(region.label.padEnd(nameWidth))} ${chalk.dim(String(region.file_count).padStart(filesWidth))} ${confidenceColor(confidenceText.padEnd(confidenceWidth))}`;+ if (includeParent) {+ line += ` ${chalk.dim((parentLabel ?? "-").padEnd(parentWidth))}`;+ }+ line += ` ${(crossText === "yes" ? chalk.yellow(crossText.padEnd(crossWidth)) : chalk.dim(crossText.padEnd(crossWidth)))}`;++ if (verbose) {+ line += chalk.dim(` br=${Math.min(region.boundary_ratio, 999.9).toFixed(1)} xcut=${region.crosscut_score.toFixed(2)}`);+ }++ console.log(line);+ }+}++function visibleWidth(text: string): number {+ return text.replace(/\x1B\[[0-9;]*m/g, "").length;+}++function formatRegionLine(region: MapRegion, verbose: boolean, depth = 0): string {+ const clarity = confidenceLabel(region.confidence);+ const clarityColor = region.confidence >= 0.75 ? chalk.green : region.confidence >= 0.50 ? chalk.yellow : chalk.red;+ const confPct = Math.round(region.confidence * 100);+ const crosscut = region.crosscut_score > 0.10 ? chalk.yellow(" shared") : "";+ const levelTag = region.level === 3 ? "system" : region.level === 2 ? "subsystem" : "module";+ const badge = chalk.bgBlackBright.white(` ${levelTag.toUpperCase()} `);+ const signals = region.dominant_signals.slice(0, 2).join(" · ");++ if (verbose) {+ const bar = confidenceBar(region.confidence);+ const metrics = chalk.dim(+ `conf=${confPct}% br=${Math.min(region.boundary_ratio, 999.9).toFixed(1)} xcut=${region.crosscut_score.toFixed(2)}`+ );+ const signalText = signals.length > 0 ? chalk.dim(` ${signals}`) : "";+ return `${badge} ${bar} ${chalk.bold(region.label)} ${chalk.dim(`${region.file_count} files`)} ${clarityColor(`${clarity} (${confPct}%)`)} ${metrics}${signalText}${crosscut}`;+ }++ const fileText = depth === 0 ? chalk.dim(`${region.file_count} files`) : chalk.dim(`${region.file_count}`);+ const signalText = signals.length > 0 ? chalk.dim(` ${signals}`) : "";+ return `${badge} ${chalk.bold(region.label)} ${fileText} ${clarityColor(`${clarity} ${confPct}%`)}${signalText}${crosscut}`;+}diff --git a/ix-cli/src/cli/commands/parse-pool.ts b/ix-cli/src/cli/commands/parse-pool.tsnew file mode 100644index 00000000..324e2aba--- /dev/null+++ b/ix-cli/src/cli/commands/parse-pool.ts@@ -0,0 +1,87 @@+/**+ * Bounded worker-thread pool for parallel file parsing.+ * Uses the core-ingestion parse-worker entry compiled to dist/.+ *+ * - Bounded to `concurrency` simultaneous workers.+ * - One crashed worker is replaced; its in-flight task resolves as null (treated as parse failure).+ * - parse() returns a Promise that resolves to FileParseResult | null.+ * - Results collected via Promise.all preserve input ordering.+ */+import { Worker } from 'node:worker_threads';++type Task = {+ filePath: string;+ source: string;+ resolve: (result: unknown) => void;+};++export class ParsePool {+ private workers: Worker[] = [];+ private idle: Worker[] = [];+ private queue: Task[] = [];+ private active = new Map<Worker, Task>();++ constructor(private workerPath: string, private concurrency: number) {}++ init(): void {+ for (let i = 0; i < this.concurrency; i++) {+ this.spawnWorker();+ }+ }++ parse(filePath: string, source: string): Promise<unknown> {+ return new Promise((resolve) => {+ this.queue.push({ filePath, source, resolve });+ this.drain();+ });+ }++ async destroy(): Promise<void> {+ await Promise.all(this.workers.map(w => w.terminate()));+ this.workers = [];+ this.idle = [];+ }++ private spawnWorker(): Worker {+ const w = new Worker(this.workerPath);+ w.on('message', (msg: { ok: boolean; result: unknown }) => this.onResult(w, msg));+ w.on('error', (err) => this.onError(w, err));+ this.workers.push(w);+ this.idle.push(w);+ return w;+ }++ private drain(): void {+ while (this.idle.length > 0 && this.queue.length > 0) {+ const w = this.idle.pop()!;+ const task = this.queue.shift()!;+ this.active.set(w, task);+ w.postMessage({ filePath: task.filePath, source: task.source });+ }+ }++ private onResult(w: Worker, msg: { ok: boolean; result: unknown }): void {+ const task = this.active.get(w);+ if (!task) return;+ this.active.delete(w);+ task.resolve(msg.ok ? msg.result : null);+ this.idle.push(w);+ this.drain();+ }++ private onError(w: Worker, _err: Error): void {+ const task = this.active.get(w);+ if (task) {+ this.active.delete(w);+ task.resolve(null); // isolate: failed file = null parse result+ }+ // Replace the crashed worker+ const idx = this.workers.indexOf(w);+ if (idx !== -1) {+ w.terminate().catch(() => {});+ this.workers.splice(idx, 1);+ this.spawnWorker();+ this.drain();+ }+ }+}diff --git a/ix-cli/src/cli/commands/reset.ts b/ix-cli/src/cli/commands/reset.tsindex e9cb4d00..696752f6 100644--- a/ix-cli/src/cli/commands/reset.ts+++ b/ix-cli/src/cli/commands/reset.ts@@ -1,20 +1,42 @@+import * as nodePath from "node:path";+import * as fs from "node:fs";+import * as crypto from "node:crypto";+import * as os from "node:os";import type { Command } from "commander";import chalk from "chalk";import { spawnSync } from "child_process";import { IxClient } from "../../client/api.js";import { getEndpoint } from "../config.js";+function mtimeCachePath(projectRoot: string): string {+ const key = crypto.createHash("sha256").update(projectRoot).digest("hex").slice(0, 12);+ return nodePath.join(os.homedir(), ".ix", `ingest_mtimes_${key}.json`);+}++function clearMtimeCache(projectRoot: string): void {+ try {+ fs.rmSync(mtimeCachePath(projectRoot), { force: true });+ } catch {+ // Non-critical+ }+}+export function registerResetCommand(program: Command): void {program.command("reset")- .description("Wipe all graph data (nodes + edges)")+ .description("Wipe graph data").option("-y, --yes", "Skip confirmation prompt")- .option("--ingest", "Re-ingest current directory after wiping")- .action(async (opts: { yes?: boolean; ingest?: boolean }) => {+ .option("--code", "Reset only code graph (files, functions, classes, regions); preserve goals, plans, tasks, bugs, and decisions")+ .option("--ingest", "Re-run ix map after wiping (rebuilds the code graph)")+ .action(async (opts: { yes?: boolean; code?: boolean; ingest?: boolean }) => {+ const scope = opts.code ? "code graph" : "all graph data";+ const warning = opts.code+ ? "This will delete all code nodes and edges (files, functions, classes, regions).\nPlanning artifacts (goals, plans, tasks, bugs, decisions) will be preserved."+ : "This will delete ALL nodes and edges including planning artifacts.";+if (!opts.yes) {- process.stdout.write(- chalk.yellow("This will delete all nodes and edges. Are you sure? (y/N) ")- );+ console.log(chalk.yellow(warning));+ process.stdout.write(chalk.yellow(`Reset ${scope}? (y/N) `));const answer = await new Promise<string>(resolve => {process.stdin.setEncoding("utf8");process.stdin.once("data", (chunk: string) => resolve(chunk.trim()));@@ -28,8 +50,17 @@ export function registerResetCommand(program: Command): void {const client = new IxClient(getEndpoint());try {- await client.reset();- console.log(chalk.green("✓") + " Graph wiped.");+ if (opts.code) {+ await client.resetCode();+ // Clear the mtime cache so the next ix map re-ingests all files+ clearMtimeCache(process.cwd());+ console.log(chalk.green("✓") + " Code graph wiped. Planning artifacts preserved.");+ console.log(chalk.dim(" Run `ix map` to rebuild the code graph."));+ } else {+ await client.reset();+ clearMtimeCache(process.cwd());+ console.log(chalk.green("✓") + " Graph wiped.");+ }} catch (err: any) {console.error(chalk.red("Error:"), err.message);process.exitCode = 1;@@ -37,8 +68,8 @@ export function registerResetCommand(program: Command): void {}if (opts.ingest) {- console.log(chalk.dim("Re-ingesting..."));- const result = spawnSync(process.argv[0], [process.argv[1], "ingest", "."], {+ console.log(chalk.dim("Rebuilding..."));+ const result = spawnSync(process.argv[0], [process.argv[1], "map"], {stdio: "inherit",cwd: process.cwd(),});diff --git a/ix-cli/src/cli/commands/smells.ts b/ix-cli/src/cli/commands/smells.tsnew file mode 100644index 00000000..81445982--- /dev/null+++ b/ix-cli/src/cli/commands/smells.ts@@ -0,0 +1,169 @@+import type { Command } from "commander";+import chalk from "chalk";+import { IxClient } from "../../client/api.js";+import { getEndpoint } from "../config.js";++interface SmellCandidate {+ file_id: string;+ file: string;+ smell: string;+ confidence: number;+ signals: Record<string, unknown>;+ inference_version: string;+}++interface SmellReport {+ rev: number;+ run_at: string;+ count: number;+ candidates: SmellCandidate[];+}++const SMELL_LABELS: Record<string, string> = {+ "has_smell.orphan_file": "Orphan File",+ "has_smell.god_module": "God Module",+ "has_smell.weak_component_member":"Weak Component",+};++export function registerSmellsCommand(program: Command): void {+ program+ .command("smells")+ .description("Detect architecture smells in the codebase")+ .option("--format <fmt>", "Output format (text|json)", "text")+ .option("--orphan-max-connections <n>","Max connections for orphan files", "0")+ .option("--god-module-chunks <n>", "Min chunks for god module", "20")+ .option("--god-module-fan <n>", "Min fan-in/out for god module", "15")+ .option("--weak-max-neighbors <n>", "Max neighbors for weak component", "1")+ .option("--list", "List existing smell claims without rerunning")+ .addHelpText("after", `+Runs graph-query-based architecture smell detection and stores results as+versioned claims (inference_version=smell_v1) on affected File nodes.++Smells detected:+ orphan_file Files with no import or call connections+ god_module Files with high chunk count or fan-in/fan-out+ weak_component_member Files connected to very few others++Examples:+ ix smells+ ix smells --god-module-chunks 15 --god-module-fan 10+ ix smells --format json+ ix smells --list`)+ .action(async (opts: {+ format: string;+ orphanMaxConnections: string;+ godModuleChunks: string;+ godModuleFan: string;+ weakMaxNeighbors: string;+ list?: boolean;+ }) => {+ const client = new IxClient(getEndpoint());++ if (opts.list) {+ let result: any;+ try {+ result = await client.listSmells();+ } catch (err: any) {+ console.error(chalk.red("Error:"), err.message);+ process.exitCode = 1;+ return;+ }+ if (opts.format === "json") {+ console.log(JSON.stringify(result, null, 2));+ return;+ }+ const smells = result.smells ?? [];+ if (smells.length === 0) {+ console.log(chalk.dim("No smell claims found. Run `ix smells` to detect."));+ return;+ }+ console.log(`\n${chalk.bold("Smell Claims")} ${chalk.dim(`(${smells.length} total)`)}\n`);+ for (const s of smells) {+ const label = SMELL_LABELS[s.smell] ?? s.smell;+ console.log(` ${chalk.yellow(label.padEnd(20))} ${chalk.dim(s.entity_id?.slice(0, 8))} ${chalk.dim(s.inference_version ?? "")}`);+ }+ return;+ }++ if (opts.format !== "json") {+ process.stderr.write(chalk.dim("Running smell detection...\n"));+ }++ let report: SmellReport;+ try {+ report = await client.runSmells({+ orphanMaxConnections: parseInt(opts.orphanMaxConnections, 10),+ godModuleChunks: parseInt(opts.godModuleChunks, 10),+ godModuleFan: parseInt(opts.godModuleFan, 10),+ weakMaxNeighbors: parseInt(opts.weakMaxNeighbors, 10),+ }) as SmellReport;+ } catch (err: any) {+ console.error(chalk.red("Error:"), err.message);+ process.exitCode = 1;+ return;+ }++ if (opts.format === "json") {+ console.log(JSON.stringify(report, null, 2));+ return;+ }++ console.log(+ `\n${chalk.bold("Architecture Smells")} ` ++ chalk.dim(`rev ${report.rev} · ${report.count} candidates`)+ );++ if (report.count === 0) {+ console.log(chalk.green("\n No smell candidates found."));+ return;+ }++ // Group by smell kind+ const byKind = new Map<string, SmellCandidate[]>();+ for (const c of report.candidates) {+ if (!byKind.has(c.smell)) byKind.set(c.smell, []);+ byKind.get(c.smell)!.push(c);+ }++ for (const [kind, candidates] of byKind) {+ const label = SMELL_LABELS[kind] ?? kind;+ console.log(`\n${chalk.bold(label)} ${chalk.dim(`(${candidates.length})`)}`);++ const sorted = [...candidates].sort((a, b) => b.confidence - a.confidence);+ for (const c of sorted) {+ const conf = confidenceBar(c.confidence);+ const fileName = c.file.length > 45+ ? "…" + c.file.slice(-44)+ : c.file.padEnd(45);+ const signalStr = formatSignals(c.smell, c.signals);+ console.log(` ${conf} ${chalk.dim(fileName)} ${chalk.dim(signalStr)}`);+ }+ }++ console.log(chalk.dim(`\nSmell claims stored with inference_version=smell_v1. Use 'ix smells --list' to retrieve.`));+ });+}++function confidenceBar(conf: number): string {+ const filled = Math.round(conf * 5);+ const bar = "█".repeat(filled) + "░".repeat(5 - filled);+ const color = conf >= 0.75 ? chalk.red : conf >= 0.5 ? chalk.yellow : chalk.dim;+ return color(bar);+}++function formatSignals(smellKind: string, signals: Record<string, unknown>): string {+ switch (smellKind) {+ case "has_smell.orphan_file":+ return `connections=${signals["connectivity"] ?? 0}`;+ case "has_smell.god_module":+ return [+ signals["chunks"] !== undefined ? `chunks=${signals["chunks"]}` : null,+ signals["fan_in"] !== undefined ? `fan_in=${signals["fan_in"]}` : null,+ signals["fan_out"] !== undefined ? `fan_out=${signals["fan_out"]}` : null,+ ].filter(Boolean).join(" ");+ case "has_smell.weak_component_member":+ return `neighbors=${signals["neighbor_count"] ?? 0}`;+ default:+ return "";+ }+}diff --git a/ix-cli/src/cli/commands/subsystems.ts b/ix-cli/src/cli/commands/subsystems.tsnew file mode 100644index 00000000..923e1f0b--- /dev/null+++ b/ix-cli/src/cli/commands/subsystems.ts@@ -0,0 +1,384 @@+import type { Command } from "commander";+import chalk from "chalk";+import { IxClient } from "../../client/api.js";+import { getEndpoint } from "../config.js";+import { renderMapText, type MapResult } from "./map.js";+import {+ renderSubsystemExplanationJson,+ renderSubsystemExplanationText,+ type ScopedSubsystemRegion,+ type ScopedSubsystemResult,+ type SubsystemScore,+} from "../explain/subsystem.js";++interface AmbiguousSubsystemResult {+ error: "ambiguous_target";+ target_query: string;+ candidates: Array<{+ pick: number;+ id: string;+ label: string;+ level: number;+ label_kind: string;+ file_count: number;+ parent?: string | null;+ }>;+}++interface UnknownSubsystemTargetResult {+ error: "unknown_target";+ target_query: string;+ message: string;+ suggestions?: string[];+}++export function registerSubsystemsCommand(program: Command): void {+ program+ .command("subsystems [target]")+ .description("Show the persisted architectural map saved by 'ix map'")+ .option("--format <fmt>", "Output format (text|json)", "text")+ .option("--list", "List stored subsystem health scores instead of the persisted architecture map")+ .option("--target <target>", "Scope subsystem output to a persisted architecture region")+ .option("--pick <n>", "Resolve an ambiguous region target by numbered candidate")+ .option("--level <n>", "Filter to level (1=module, 2=subsystem, 3=system)")+ .option("--min-confidence <n>", "Only show regions above this confidence threshold (0-1)", "0")+ .option("--max-items <n>", "Max items to show per section in text output (default: 10)", "10")+ .option("--all-items", "Show all items in each section (overrides --max-items)")+ .option("--sort <mode>", "Sort mode for text output (importance|confidence|size|alpha)", "importance")+ .option("--graph", "Render the hierarchy as a graph/tree view instead of the default ranked list")+ .option("--verbose", "Show raw confidence scores, crosscut scores, boundary ratios, and signals")+ .option("--explain", "Explain a scoped subsystem region in plain English")+ .addHelpText("after", `+Reads the persisted architectural map saved by 'ix map' from the graph and+renders it with the same output formatter as 'ix map', without rerunning map+ingestion or clustering.++Examples:+ ix subsystems+ ix subsystems --list+ ix subsystems api+ ix subsystems --target "Cli / Client"+ ix subsystems api --pick 2+ ix subsystems api --explain+ ix subsystems --level 2+ ix subsystems --graph+ ix subsystems --format json`)+ .action(async (positionalTarget: string | undefined, opts: { format: string; list?: boolean; target?: string; pick?: string; level?: string; minConfidence: string; maxItems: string; allItems?: boolean; sort: string; graph?: boolean; verbose?: boolean; explain?: boolean }) => {+ const client = new IxClient(getEndpoint());+ const target = resolveSubsystemTarget(positionalTarget, opts.target);+ const pick = parsePickOption(opts.pick);++ if (!target.error && pick.error) {+ console.error(chalk.red("Error:"), pick.error);+ process.exitCode = 1;+ return;+ }+ if (target.error) {+ console.error(chalk.red("Error:"), target.error);+ process.exitCode = 1;+ return;+ }+ if (opts.list && target.value) {+ console.error(chalk.red("Error:"), "Scoped targets are not supported with --list.");+ process.exitCode = 1;+ return;+ }+ if (opts.list && opts.explain) {+ console.error(chalk.red("Error:"), "--explain cannot be used with --list.");+ process.exitCode = 1;+ return;+ }+ if (!target.value && pick.value !== undefined) {+ console.error(chalk.red("Error:"), "--pick requires a target.");+ process.exitCode = 1;+ return;+ }+ if (opts.explain && !target.value) {+ console.error(chalk.red("Error:"), "--explain requires a target.");+ process.exitCode = 1;+ return;+ }++ if (opts.list) {+ let result: any;+ try {+ result = await client.listSubsystems();+ } catch (err: any) {+ console.error(chalk.red("Error:"), err.message);+ process.exitCode = 1;+ return;+ }+ const scores: SubsystemScore[] = result.scores ?? [];+ const filtered = opts.level+ ? scores.filter(s => s.level === parseInt(opts.level!, 10))+ : scores;++ if (opts.format === "json") {+ console.log(JSON.stringify({ scores: filtered }, null, 2));+ return;+ }+ printScores(filtered);+ return;+ }++ let result: MapResult | ScopedSubsystemResult;+ try {+ result = await client.getSubsystemMap({+ target: target.value,+ pick: pick.value,+ }) as MapResult;+ } catch (err: any) {+ const body = parseErrorBody(err);+ if (body) {+ if (opts.format === "json") {+ console.log(JSON.stringify(body, null, 2));+ process.exitCode = 1;+ return;+ }+ renderSubsystemError(body, Boolean(opts.explain));+ process.exitCode = 1;+ return;+ }+ console.error(chalk.red("Error:"), err.message);+ process.exitCode = 1;+ return;+ }++ if (opts.explain) {+ if (!isScopedSubsystemResult(result)) {+ console.error(chalk.red("Error:"), "--explain requires a scoped subsystem target.");+ process.exitCode = 1;+ return;+ }++ let scoreResult: { scores?: SubsystemScore[] };+ try {+ scoreResult = await client.listSubsystems();+ } catch (err: any) {+ console.error(chalk.red("Error:"), err.message);+ process.exitCode = 1;+ return;+ }++ const score = (scoreResult.scores ?? []).find((candidate) => candidate.region_id === result.target.id) ?? null;+ if (opts.format === "json") {+ console.log(JSON.stringify(renderSubsystemExplanationJson(result, score), null, 2));+ return;+ }++ console.log(renderSubsystemExplanationText(result, score));+ return;+ }++ if (opts.format === "json") {+ console.log(JSON.stringify(result, null, 2));+ return;+ }++ if (isScopedSubsystemResult(result)) {+ renderScopedSubsystemText(result, Boolean(opts.verbose));+ return;+ }++ renderMapText(result, process.cwd(), {+ level: opts.level,+ minConfidence: opts.minConfidence,+ maxItems: opts.maxItems,+ allItems: opts.allItems,+ sort: opts.sort,+ graph: opts.graph,+ verbose: opts.verbose,+ });+ });+}++function printScores(scores: SubsystemScore[]): void {+ if (scores.length === 0) {+ console.log(chalk.dim("No subsystem scores found. Run 'ix map' then 'ix subsystems'."));+ return;+ }++ // Group by level+ const byLevel = new Map<number, SubsystemScore[]>();+ for (const s of scores) {+ if (!byLevel.has(s.level)) byLevel.set(s.level, []);+ byLevel.get(s.level)!.push(s);+ }++ const levelLabels: Record<number, string> = { 1: "Modules", 2: "Subsystems", 3: "Systems" };+ const sorted = [...byLevel.entries()].sort((a, b) => b[0] - a[0]);++ for (const [level, group] of sorted) {+ const label = levelLabels[level] ?? `Level ${level}`;+ console.log(`\n${chalk.bold(label)}`);++ const ranked = [...group].sort((a, b) => b.health_score - a.health_score);+ for (const s of ranked) {+ const bar = healthBar(s.health_score);+ const name = s.name.length > 32 ? s.name.slice(0, 31) + "…" : s.name.padEnd(32);+ const files = chalk.dim(`${s.file_count}f`);+ const chunks = chalk.dim(`${s.chunk_density.toFixed(1)}c/f`);+ const smells = s.smell_files > 0+ ? chalk.yellow(`${s.smell_files}⚠`)+ : chalk.green("✓");+ console.log(` ${bar} ${chalk.bold(name)} ${files} ${chunks} ${smells}`);+ }+ }+}++function healthBar(score: number): string {+ const filled = Math.round(score * 5);+ const bar = "█".repeat(filled) + "░".repeat(5 - filled);+ const color = score >= 0.7 ? chalk.green : score >= 0.4 ? chalk.yellow : chalk.red;+ return color(bar);+}++function parseErrorBody(err: { message?: string }): unknown | null {+ const match = /^\d+: (.+)$/.exec(err.message ?? "");+ if (!match) return null;+ try {+ return JSON.parse(match[1]);+ } catch {+ return null;+ }+}++function isScopedSubsystemResult(value: unknown): value is ScopedSubsystemResult {+ return typeof value === "object" && value !== null && "target" in value && "summary" in value && "children" in value;+}++function isAmbiguousSubsystemResult(value: unknown): value is AmbiguousSubsystemResult {+ return typeof value === "object" && value !== null && (value as { error?: string }).error === "ambiguous_target";+}++function isUnknownSubsystemTargetResult(value: unknown): value is UnknownSubsystemTargetResult {+ return typeof value === "object" && value !== null && (value as { error?: string }).error === "unknown_target";+}++function resolveSubsystemTarget(+ positionalTarget: string | undefined,+ explicitTarget: string | undefined,+): { value?: string; error?: string } {+ const positional = positionalTarget?.trim();+ const explicit = explicitTarget?.trim();+ if (positional && explicit && positional !== explicit) {+ return { error: "Pass either a positional target or --target, not both." };+ }+ const value = positional || explicit;+ return value ? { value } : {};+}++function parsePickOption(rawPick: string | undefined): { value?: number; error?: string } {+ if (rawPick === undefined) return {};+ const value = Number.parseInt(rawPick, 10);+ if (!Number.isFinite(value) || value <= 0) {+ return { error: "Invalid --pick value." };+ }+ return { value };+}++function renderSubsystemError(body: unknown, explain: boolean = false): void {+ if (isAmbiguousSubsystemResult(body)) {+ console.error(`Ambiguous target "${body.target_query}". Multiple architecture regions matched:\n`);+ for (const candidate of body.candidates) {+ const parent = candidate.parent ? ` parent: ${candidate.parent}` : "";+ console.error(+ `${candidate.pick}. ${candidate.label.padEnd(18)} ${candidate.label_kind.padEnd(11)} ${String(candidate.file_count).padStart(3)} files${parent}`+ );+ }+ const explainFlag = explain ? " --explain" : "";+ console.error(`\nRun:\n ix subsystems ${JSON.stringify(body.target_query)} --pick <n>${explainFlag}`);+ return;+ }++ if (isUnknownSubsystemTargetResult(body)) {+ console.error(body.message);+ if (body.suggestions && body.suggestions.length > 0) {+ console.error("\nTry:");+ for (const suggestion of body.suggestions) {+ console.error(`- ${suggestion}`);+ }+ }+ return;+ }++ if (typeof body === "object" && body !== null && "error" in body) {+ console.error(chalk.red("Error:"), String((body as { error: unknown }).error));+ return;+ }++ console.error(chalk.red("Error:"), "Request failed.");+}++function renderScopedSubsystemText(result: ScopedSubsystemResult, verbose: boolean): void {+ const target = result.target;+ const lines = [+ `${chalk.bold("Subsystem Scope")}: ${target.label}`,+ `Level: ${target.label_kind}`,+ result.parent ? `Parent: ${result.parent.label}` : null,+ `Files: ${target.file_count}`,+ `Confidence: ${target.confidence.toFixed(2)}`,+ target.level === 1 ? `Cross-cutting: ${target.is_cross_cutting ? "yes" : "no"}` : null,+ target.dominant_signals.length > 0 ? `Signals: ${target.dominant_signals.join(", ")}` : null,+ ].filter((line): line is string => line !== null);++ console.log(lines.join("\n"));+ console.log(`\n${chalk.bold("Health")}:`);+ console.log(+ `${result.summary.well_defined} well-defined · ` ++ `${result.summary.moderate} moderate · ` ++ `${result.summary.fuzzy} fuzzy · ` ++ `${result.summary.cross_cutting} cross-cutting`+ );++ console.log(`\n${chalk.bold("Focused Graph")}`);+ console.log(chalk.dim(" Scoped architecture tree"));+ console.log();+ renderScopedTreeNode(result.hierarchy, "", true, 0, verbose);+ console.log();+}++function renderScopedTreeNode(+ region: ScopedSubsystemRegion,+ prefix: string,+ isLast: boolean,+ depth: number,+ verbose: boolean,+): void {+ const branch = depth === 0 ? "●" : isLast ? "└─" : "├─";+ console.log(`${prefix}${branch} ${formatScopedRegionLine(region, verbose, depth)}`);++ const children = [...(region.children ?? [])].sort(compareScopedRegions);+ const nextPrefix = depth === 0 ? " " : `${prefix}${isLast ? " " : "│ "}`;+ children.forEach((child, index) => {+ renderScopedTreeNode(child, nextPrefix, index === children.length - 1, depth + 1, verbose);+ });+}++function compareScopedRegions(a: ScopedSubsystemRegion, b: ScopedSubsystemRegion): number {+ return b.file_count - a.file_count || b.confidence - a.confidence || a.label.localeCompare(b.label);+}++function formatScopedRegionLine(region: ScopedSubsystemRegion, verbose: boolean, depth: number): string {+ const clarity = confidenceLabel(region.confidence);+ const clarityColor = region.confidence >= 0.75 ? chalk.green : region.confidence >= 0.50 ? chalk.yellow : chalk.red;+ const confPct = Math.round(region.confidence * 100);+ const crosscut = region.is_cross_cutting ? chalk.yellow(" shared") : "";+ const badge = chalk.bgBlackBright.white(` ${region.label_kind.toUpperCase()} `);+ const signals = region.dominant_signals.slice(0, 2).join(" · ");++ if (verbose) {+ const signalText = signals.length > 0 ? chalk.dim(` ${signals}`) : "";+ return `${badge} ${chalk.bold(region.label)} ${chalk.dim(`${region.file_count} files`)} ${clarityColor(`${clarity} (${confPct}%)`)}${signalText}${crosscut}`;+ }++ const fileText = depth === 0 ? chalk.dim(`${region.file_count} files`) : chalk.dim(`${region.file_count}`);+ const signalText = signals.length > 0 ? chalk.dim(` ${signals}`) : "";+ return `${badge} ${chalk.bold(region.label)} ${fileText} ${clarityColor(`${clarity} ${confPct}%`)}${signalText}${crosscut}`;+}++function confidenceLabel(confidence: number): string {+ if (confidence >= 0.75) return "Well-defined";+ if (confidence >= 0.50) return "Moderate";+ return "Fuzzy";+}diff --git a/ix-cli/src/cli/errors.ts b/ix-cli/src/cli/errors.tsindex 50698358..5d73eddc 100644--- a/ix-cli/src/cli/errors.ts+++ b/ix-cli/src/cli/errors.ts@@ -48,3 +48,47 @@ export function renderStructuredError(err: StructuredError): void {}console.error("");}++export class CliUsageError extends Error {+ constructor(+ message: string,+ public readonly hint?: string,+ ) {+ super(message);+ this.name = "CliUsageError";+ }+}++export class CliResolutionError extends Error {+ constructor(+ message: string,+ public readonly hint?: string,+ public readonly detail?: string,+ ) {+ super(message);+ this.name = "CliResolutionError";+ }+}++export function renderCliError(err: unknown, debug = false): void {+ if (err instanceof CliUsageError || err instanceof CliResolutionError) {+ process.stderr.write(chalk.red(`Error: ${err.message}\n`));+ if (err.hint) {+ process.stderr.write(chalk.dim(`${err.hint}\n`));+ }+ if (debug && err instanceof CliResolutionError && err.detail) {+ process.stderr.write(chalk.dim(`Detail: ${err.detail}\n`));+ }+ process.exit(1);+ }++ const e = err as any;+ const msg = e?.message ?? String(err);+ process.stderr.write(chalk.red(`Error: ${msg}\n`));++ if (debug && e?.stack) {+ process.stderr.write(chalk.dim(`${e.stack}\n`));+ }++ process.exit(1);+}diff --git a/ix-cli/src/cli/explain/subsystem.ts b/ix-cli/src/cli/explain/subsystem.tsnew file mode 100644index 00000000..8fe19052--- /dev/null+++ b/ix-cli/src/cli/explain/subsystem.ts@@ -0,0 +1,376 @@+export interface SubsystemScore {+ region_id: string;+ name: string;+ level: number;+ label_kind: string;+ file_count: number;+ health_score: number;+ chunk_density: number;+ smell_rate: number;+ smell_files: number;+ total_chunks: number;+ confidence: number;+ inference_version: string;+}++export interface ScopedSubsystemRegion {+ id: string;+ label: string;+ level: number;+ label_kind: string;+ parent_id: string | null;+ file_count: number;+ confidence: number;+ is_cross_cutting: boolean;+ dominant_signals: string[];+ children?: ScopedSubsystemRegion[];+}++export interface ScopedSubsystemResult {+ target: ScopedSubsystemRegion;+ parent?: {+ id: string;+ label: string;+ level: number;+ label_kind: string;+ } | null;+ summary: {+ well_defined: number;+ moderate: number;+ fuzzy: number;+ cross_cutting: number;+ };+ children: ScopedSubsystemRegion[];+ hierarchy: ScopedSubsystemRegion;+}++export interface SubsystemExplanationJson {+ resolvedTarget: {+ id: string;+ label: string;+ level: number;+ label_kind: string;+ };+ explanation: string;+ context: {+ level: number;+ label_kind: string;+ parent: {+ id: string;+ label: string;+ label_kind: string;+ } | null;+ file_count: number;+ child_count: number;+ confidence: number;+ dominant_signals: string[];+ is_cross_cutting: boolean;+ };+ composition: {+ children: Array<{+ label: string;+ level: number;+ label_kind: string;+ file_count: number;+ confidence: number;+ is_cross_cutting: boolean;+ }>;+ summary: {+ well_defined: number;+ moderate: number;+ fuzzy: number;+ cross_cutting: number;+ };+ omitted_child_count: number;+ } | null;+ health: {+ score: number;+ chunk_density: number;+ smell_rate: number;+ smell_files: number;+ total_chunks: number;+ status: "healthy" | "good" | "moderate" | "needs_attention";+ } | null;+ whyItMatters: string;+ notes: string[];+}++const MAX_COMPOSITION_CHILDREN = 10;++export function renderSubsystemExplanationJson(+ scoped: ScopedSubsystemResult,+ score: SubsystemScore | null,+): SubsystemExplanationJson {+ const target = scoped.target;+ const children = sortRegions(scoped.children);+ const visibleChildren = children.slice(0, MAX_COMPOSITION_CHILDREN);+ const notes = score ? [] : ["Run 'ix subsystems' to compute health scores for this region."];+ const health = score ? renderHealthJson(score) : null;++ return {+ resolvedTarget: {+ id: target.id,+ label: target.label,+ level: target.level,+ label_kind: target.label_kind,+ },+ explanation: synthesizeSubsystemExplanation(target, scoped.parent ?? null),+ context: {+ level: target.level,+ label_kind: target.label_kind,+ parent: scoped.parent+ ? {+ id: scoped.parent.id,+ label: scoped.parent.label,+ label_kind: scoped.parent.label_kind,+ }+ : null,+ file_count: target.file_count,+ child_count: scoped.children.length,+ confidence: target.confidence,+ dominant_signals: target.dominant_signals,+ is_cross_cutting: target.is_cross_cutting,+ },+ composition: scoped.children.length > 0+ ? {+ children: visibleChildren.map((child) => ({+ label: child.label,+ level: child.level,+ label_kind: child.label_kind,+ file_count: child.file_count,+ confidence: child.confidence,+ is_cross_cutting: child.is_cross_cutting,+ })),+ summary: { ...scoped.summary },+ omitted_child_count: Math.max(children.length - visibleChildren.length, 0),+ }+ : null,+ health,+ whyItMatters: renderWhyItMatters(target, scoped.parent ?? null, score),+ notes,+ };+}++export function renderSubsystemExplanationText(+ scoped: ScopedSubsystemResult,+ score: SubsystemScore | null,+): string {+ const rendered = renderSubsystemExplanationJson(scoped, score);+ const lines: string[] = [];++ lines.push("Explanation");+ lines.push(` ${rendered.explanation}`);+ lines.push("");+ lines.push("Context");+ lines.push(` Level: ${rendered.context.label_kind} (level ${rendered.context.level})`);+ if (rendered.context.parent) {+ lines.push(` Parent: ${rendered.context.parent.label} (${rendered.context.parent.label_kind})`);+ }+ lines.push(` Files: ${rendered.context.file_count}`);+ if (rendered.context.child_count > 0) {+ lines.push(` Children: ${rendered.context.child_count} ${pluralize(rendered.context.child_count, childKindLabel(scoped.children))}`);+ }+ lines.push(` Confidence: ${confidenceLabel(rendered.context.confidence)} (${Math.round(rendered.context.confidence * 100)}%)`);+ if (rendered.context.dominant_signals.length > 0) {+ lines.push(` Signals: ${rendered.context.dominant_signals.join(" · ")}`);+ }+ if (rendered.context.is_cross_cutting) {+ lines.push(" Scope: Cross-cutting");+ }++ if (rendered.composition) {+ lines.push("");+ lines.push("Composition");+ for (const child of rendered.composition.children) {+ lines.push(` ${formatCompositionLine(child)}`);+ }+ if (rendered.composition.omitted_child_count > 0) {+ lines.push(` ... ${rendered.composition.omitted_child_count} more. Use 'ix subsystems ${JSON.stringify(scoped.target.label)}' for the full tree.`);+ }+ lines.push(+ ` ${rendered.composition.summary.well_defined} well-defined · ` ++ `${rendered.composition.summary.moderate} moderate · ` ++ `${rendered.composition.summary.fuzzy} fuzzy · ` ++ `${rendered.composition.summary.cross_cutting} cross-cutting`+ );+ }++ if (rendered.health) {+ lines.push("");+ lines.push("Health");+ lines.push(` Score: ${plainHealthBar(rendered.health.score)} ${rendered.health.score.toFixed(2)}`);+ lines.push(` Chunk density: ${rendered.health.chunk_density.toFixed(1)} chunks/file`);+ lines.push(` Smell rate: ${Math.round(rendered.health.smell_rate * 100)}% (${rendered.health.smell_files} files flagged)`);+ lines.push(` Status: ${healthStatusPhrase(rendered.health.score)}${densityNote(rendered.health.chunk_density)}`);+ }++ lines.push("");+ lines.push("Why it matters");+ lines.push(` ${rendered.whyItMatters}`);++ if (rendered.notes.length > 0) {+ lines.push("");+ lines.push(rendered.notes.length === 1 ? "Note" : "Notes");+ for (const note of rendered.notes) {+ lines.push(` ${note}`);+ }+ }++ return lines.join("\n");+}++export function synthesizeSubsystemExplanation(+ region: ScopedSubsystemRegion,+ parent: ScopedSubsystemResult["parent"] | null,+): string {+ const regionNoun = articleFor(region.label_kind) + region.label_kind;+ const parentPhrase = parent ? ` inside ${parent.label}` : "";+ const confidencePhrase = confidenceSentence(region.confidence);+ const crossCuttingPhrase = region.is_cross_cutting+ ? " It is cross-cutting and spans multiple neighboring regions."+ : "";+ const signalPhrase = primarySignalSentence(region.dominant_signals);++ return (+ `\`${region.label}\` serves as ${regionNoun} spanning ${region.file_count} ` ++ `${pluralize(region.file_count, "file")}${parentPhrase}. ` ++ `It is ${confidencePhrase}.${signalPhrase}${crossCuttingPhrase}`+ ).replace(/\s+/g, " ").trim();+}++function renderWhyItMatters(+ region: ScopedSubsystemRegion,+ parent: ScopedSubsystemResult["parent"] | null,+ score: SubsystemScore | null,+): string {+ if (region.is_cross_cutting && region.confidence < 0.5) {+ return (+ `${region.label} spans multiple subsystems and has low clustering confidence, ` ++ "which suggests architectural drift. Consider clarifying or decomposing its boundaries."+ );+ }++ if (score && score.health_score >= 0.8 && region.level === 3) {+ return (+ `${region.label} is a stable, well-defined system boundary. Changes within it are usually ` ++ `well-contained. Run \`ix impact ${JSON.stringify(region.label)}\` to review cross-system dependencies.`+ );+ }++ if (score && score.health_score < 0.45 && score.smell_files > 0) {+ return (+ `${region.label} has ${score.smell_files} ${pluralize(score.smell_files, "file")} with active smell claims ` ++ `and a health score of ${score.health_score.toFixed(2)}. Prioritize refactoring here before expanding the subsystem. ` ++ "Run `ix smells` to review."+ );+ }++ const parentPhrase = parent ? ` inside ${parent.label}` : "";+ const densityPhrase = score+ ? densityNote(score.chunk_density).trimStart()+ : "";+ return (+ `${region.label} is a ${confidenceLabel(region.confidence).toLowerCase()} ${region.label_kind}${parentPhrase}. ` ++ `Run \`ix impact ${JSON.stringify(region.label)}\` to understand its downstream blast radius.${densityPhrase}`+ ).trim();+}++function renderHealthJson(score: SubsystemScore): NonNullable<SubsystemExplanationJson["health"]> {+ return {+ score: score.health_score,+ chunk_density: score.chunk_density,+ smell_rate: score.smell_rate,+ smell_files: score.smell_files,+ total_chunks: score.total_chunks,+ status: healthStatus(score.health_score),+ };+}++function healthStatus(score: number): "healthy" | "good" | "moderate" | "needs_attention" {+ if (score >= 0.8) return "healthy";+ if (score >= 0.65) return "good";+ if (score >= 0.45) return "moderate";+ return "needs_attention";+}++function healthStatusPhrase(score: number): string {+ switch (healthStatus(score)) {+ case "healthy":+ return "Healthy - rich structure, minimal technical debt";+ case "good":+ return "Good - rich structure, low smell coverage";+ case "moderate":+ return "Moderate - structure is present, smell coverage is elevated";+ case "needs_attention":+ return "Needs attention - sparse structure or significant smell coverage";+ }+}++function densityNote(chunkDensity: number): string {+ if (chunkDensity >= 5.0) {+ return " High chunk density indicates thorough code coverage in the knowledge graph.";+ }+ if (chunkDensity < 2.0) {+ return " Low chunk density - run 'ix ingest' to improve coverage.";+ }+ return "";+}++function primarySignalSentence(signals: string[]): string {+ if (signals.length === 0) return "";+ if (signals.length === 1) {+ return ` Its primary structural signal is ${signals[0]}.`;+ }+ return ` Its primary structural signals are ${signals.slice(0, 2).join(" and ")}.`;+}++function confidenceSentence(confidence: number): string {+ if (confidence >= 0.75) return "well-defined";+ if (confidence >= 0.5) return "moderately defined";+ return "loosely defined";+}++function confidenceLabel(confidence: number): string {+ if (confidence >= 0.75) return "Well-defined";+ if (confidence >= 0.5) return "Moderate";+ return "Fuzzy";+}++function plainHealthBar(score: number): string {+ const filled = Math.round(score * 5);+ return "█".repeat(filled) + "░".repeat(5 - filled);+}++function formatCompositionLine(region: {+ label: string;+ label_kind: string;+ file_count: number;+ confidence: number;+ is_cross_cutting: boolean;+}): string {+ const kind = region.label_kind.toUpperCase().padEnd(9);+ const label = region.label.padEnd(24);+ const confidence = `${confidenceLabel(region.confidence)} ${Math.round(region.confidence * 100)}%`.padEnd(17);+ const scope = region.is_cross_cutting ? " shared" : "";+ return `● ${kind}${label}${confidence}${String(region.file_count).padStart(4)} ${pluralize(region.file_count, "file")}${scope}`;+}++function childKindLabel(children: ScopedSubsystemRegion[]): string {+ const first = children[0]?.label_kind;+ if (!first || !children.every((child) => child.label_kind === first)) {+ return "child";+ }+ return first;+}++function sortRegions(children: ScopedSubsystemRegion[]): ScopedSubsystemRegion[] {+ return [...children].sort((a, b) => b.file_count - a.file_count || b.confidence - a.confidence || a.label.localeCompare(b.label));+}++function pluralize(count: number, noun: string): string {+ return count === 1 ? noun : `${noun}s`;+}++function articleFor(noun: string): string {+ return /^[aeiou]/i.test(noun) ? "an " : "a ";+}diff --git a/ix-cli/src/cli/register/oss.ts b/ix-cli/src/cli/register/oss.tsindex 58c018e1..d1453318 100644--- a/ix-cli/src/cli/register/oss.ts+++ b/ix-cli/src/cli/register/oss.ts@@ -29,6 +29,8 @@ import { registerMapCommand } from "../commands/map.js";import { registerResetCommand } from "../commands/reset.js";import { registerConfigCommand } from "../commands/config.js";import { registerTraceCommand } from "../commands/trace.js";+import { registerSmellsCommand } from "../commands/smells.js";+import { registerSubsystemsCommand } from "../commands/subsystems.js";const PRO_COMMANDS: { name: string; desc: string }[] = [{ name: "briefing", desc: "Session-resume briefing" },@@ -83,6 +85,8 @@ export function registerOssCommands(program: Command): void {registerDockerCommand(program);registerWorkflowsHelpCommand(program);registerMapCommand(program);+ registerSmellsCommand(program);+ registerSubsystemsCommand(program);registerResetCommand(program);registerConfigCommand(program);registerTraceCommand(program);diff --git a/ix-cli/src/client/api.ts b/ix-cli/src/client/api.tsindex eea5dd5a..c52fd7ab 100644--- a/ix-cli/src/client/api.ts+++ b/ix-cli/src/client/api.ts@@ -179,23 +179,68 @@ export class IxClient {return result.exists;}+ async commitPatchBatch(patches: GraphPatchPayload[]): Promise<PatchCommitResult[]> {+ return this.post("/v1/patches/batch", patches);+ }+async getSourceHashes(filePaths: string[]): Promise<Map<string, string>> {const result = await this.post<Record<string, string>>('/v1/source-hashes', { uris: filePaths });return new Map(Object.entries(result));}- async commitPatchBulk(patches: GraphPatchPayload[]): Promise<PatchCommitResult> {- return this.post<PatchCommitResult>('/v1/patches/bulk', { patches });+ async map(opts?: { full?: boolean }): Promise<any> {+ return this.post("/v1/map", opts ?? {});}- async map(opts?: { full?: boolean }): Promise<any> {- return this.post("/v1/map", { full: opts?.full });+ async commitPatchBulk(patches: GraphPatchPayload[]): Promise<{ rev: number }> {+ const results = await this.commitPatchBatch(patches);+ const rev = results.reduce((max, r) => Math.max(max, r.rev ?? 0), 0);+ return { rev };+ }++ async runSmells(opts?: {+ orphanMaxConnections?: number;+ godModuleChunks?: number;+ godModuleFan?: number;+ weakMaxNeighbors?: number;+ }): Promise<any> {+ const params = new URLSearchParams();+ if (opts?.orphanMaxConnections !== undefined) params.set("orphan-max-connections", String(opts.orphanMaxConnections));+ if (opts?.godModuleChunks !== undefined) params.set("god-module-chunks", String(opts.godModuleChunks));+ if (opts?.godModuleFan !== undefined) params.set("god-module-fan", String(opts.godModuleFan));+ if (opts?.weakMaxNeighbors !== undefined) params.set("weak-max-neighbors", String(opts.weakMaxNeighbors));+ const qs = params.toString();+ return this.post(qs ? `/v1/smells?${qs}` : "/v1/smells", {});+ }++ async listSmells(): Promise<any> {+ return this.get("/v1/smells");+ }++ async scoreSubsystems(): Promise<any> {+ return this.post("/v1/subsystems/score", {});+ }++ async listSubsystems(): Promise<any> {+ return this.get("/v1/subsystems");+ }++ async getSubsystemMap(opts?: { target?: string; pick?: number }): Promise<any> {+ const params = new URLSearchParams();+ if (opts?.target) params.set("target", opts.target);+ if (opts?.pick !== undefined) params.set("pick", String(opts.pick));+ const qs = params.toString();+ return this.get(`/v1/subsystems/map${qs ? `?${qs}` : ""}`);}async reset(): Promise<{ ok: boolean; message: string }> {return this.post("/v1/reset", {});}+ async resetCode(): Promise<{ ok: boolean; message: string }> {+ return this.post("/v1/reset/code", {});+ }+async stats(): Promise<any> {return this.get("/v1/stats");}diff --git a/memory-layer/src/main/scala/ix/memory/Main.scala b/memory-layer/src/main/scala/ix/memory/Main.scalaindex 6e3366bc..9ce6d3db 100644--- a/memory-layer/src/main/scala/ix/memory/Main.scala+++ b/memory-layer/src/main/scala/ix/memory/Main.scala@@ -11,6 +11,8 @@ import ix.memory.context._import ix.memory.db._import ix.memory.ingestion._import ix.memory.map.MapService+import ix.memory.smell.SmellService+import ix.memory.subsystem.SubsystemScoringServiceobject Main extends IOApp.Simple {@@ -44,9 +46,11 @@ object Main extends IOApp.Simple {claimCollector, confidenceScorer, conflictDetector)conflictService = new ConflictService(client, queryApi, writeApi)mapService = new MapService(client, queryApi, writeApi)+ smellService = new SmellService(client, writeApi)+ subsystemService = new SubsystemScoringService(client, writeApi, mapService)// 4. HTTP routes- routes = Routes.all(contextService, ingestionService, bulkIngestionService, queryApi, writeApi, conflictService, client, mapService, bulkWriteApi)+ routes = Routes.all(contextService, ingestionService, bulkIngestionService, queryApi, writeApi, conflictService, client, mapService, bulkWriteApi, smellService, subsystemService)// 5. Serverserver <- EmberServerBuilderdiff --git a/memory-layer/src/main/scala/ix/memory/api/MapEncoder.scala b/memory-layer/src/main/scala/ix/memory/api/MapEncoder.scalanew file mode 100644index 00000000..d721f84e--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/api/MapEncoder.scala@@ -0,0 +1,131 @@+package ix.memory.api++import io.circe.Json+import io.circe.syntax._++import ix.memory.map.{ArchitectureMap, Region, RegionMatch, ScopedSubsystemView}+import ix.memory.model.NodeId++/** Shared JSON encoding for ArchitectureMap, used by MapRoutes and SubsystemRoutes. */+object MapEncoder {++ def encodeMap(m: ArchitectureMap): Json = {+ val regionsByLevel = m.regions.groupBy(_.level).toVector.sortBy(_._1)+ Json.obj(+ "file_count" -> m.fileCount.asJson,+ "region_count" -> m.regions.size.asJson,+ "levels" -> regionsByLevel.size.asJson,+ "map_rev" -> m.mapRev.asJson,+ "regions" -> m.regions.sortBy(r => (r.level, r.label)).map(encodeRegion).asJson,+ "hierarchy" -> buildHierarchyTree(m).asJson+ )+ }++ def encodeRegion(r: Region): Json =+ Json.obj(+ "id" -> r.id.value.toString.asJson,+ "label" -> r.label.asJson,+ "label_kind" -> r.labelKind.asJson,+ "level" -> r.level.asJson,+ "file_count" -> r.memberFiles.size.asJson,+ "child_region_count" -> r.childRegionIds.size.asJson,+ "parent_id" -> r.parentId.map(_.value.toString).asJson,+ "cohesion" -> r.cohesion.asJson,+ "external_coupling" -> r.externalCoupling.asJson,+ "boundary_ratio" -> r.boundaryRatio.asJson,+ "confidence" -> r.confidence.asJson,+ "crosscut_score" -> r.crosscutScore.asJson,+ "dominant_signals" -> r.dominantSignals.asJson,+ "interface_node_count" -> r.interfaceNodeCount.asJson+ )++ def encodeScopedView(view: ScopedSubsystemView, isCrossCutting: Region => Boolean): Json =+ Json.obj(+ "target" -> encodeScopedRegion(view.target, isCrossCutting).asJson,+ "parent" -> view.parent.map(parent =>+ Json.obj(+ "id" -> parent.id.value.toString.asJson,+ "label" -> parent.label.asJson,+ "level" -> parent.level.asJson,+ "label_kind" -> parent.labelKind.asJson+ )+ ).asJson,+ "summary" -> Json.obj(+ "well_defined" -> view.summary.wellDefined.asJson,+ "moderate" -> view.summary.moderate.asJson,+ "fuzzy" -> view.summary.fuzzy.asJson,+ "cross_cutting" -> view.summary.crossCutting.asJson+ ),+ "children" -> view.children.map(child => encodeScopedRegion(child, isCrossCutting)).asJson,+ "hierarchy" -> buildScopedTree(view, isCrossCutting).asJson+ )++ def encodeAmbiguousTarget(target: String, candidates: Vector[RegionMatch], parentOf: Region => Option[Region]): Json =+ Json.obj(+ "error" -> "ambiguous_target".asJson,+ "target_query" -> target.asJson,+ "candidates" -> candidates.zipWithIndex.map { case (candidate, idx) =>+ val region = candidate.region+ Json.obj(+ "pick" -> (idx + 1).asJson,+ "id" -> region.id.value.toString.asJson,+ "label" -> region.label.asJson,+ "level" -> region.level.asJson,+ "label_kind" -> region.labelKind.asJson,+ "file_count" -> region.memberFiles.size.asJson,+ "parent" -> parentOf(region).map(_.label).asJson+ )+ }.asJson+ )++ def buildHierarchyTree(m: ArchitectureMap): Json = {+ val regionById: Map[NodeId, Region] = m.regions.map(r => r.id -> r).toMap++ // Guard against cycles (can arise when Louvain levels have identical partitions)+ def renderNode(r: Region, visited: Set[NodeId]): Json = {+ val children =+ if (visited.contains(r.id)) Vector.empty+ else r.childRegionIds.toVector+ .flatMap(cid => regionById.get(cid))+ .filter(c => c.level < r.level)+ .sortBy(_.label)+ .map(c => renderNode(c, visited + r.id))++ encodeRegion(r).deepMerge(Json.obj("children" -> children.asJson))+ }++ val roots = m.regions.filter(_.parentId.isEmpty).sortBy(-_.level)+ roots.map(r => renderNode(r, Set.empty)).asJson+ }++ private def encodeScopedRegion(region: Region, isCrossCutting: Region => Boolean): Json =+ Json.obj(+ "id" -> region.id.value.toString.asJson,+ "label" -> region.label.asJson,+ "level" -> region.level.asJson,+ "label_kind" -> region.labelKind.asJson,+ "parent_id" -> region.parentId.map(_.value.toString).asJson,+ "file_count" -> region.memberFiles.size.asJson,+ "confidence" -> region.confidence.asJson,+ "is_cross_cutting" -> isCrossCutting(region).asJson,+ "dominant_signals" -> region.dominantSignals.asJson+ )++ private def buildScopedTree(view: ScopedSubsystemView, isCrossCutting: Region => Boolean): Json = {+ val regionsById = view.subtree.map(region => region.id -> region).toMap++ def renderNode(region: Region, visited: Set[NodeId]): Json = {+ val children =+ if (visited.contains(region.id)) Vector.empty+ else region.childRegionIds.toVector+ .flatMap(regionsById.get)+ .filter(_.level < region.level)+ .sortBy(child => (-child.memberFiles.size, -child.confidence, child.label.toLowerCase))+ .map(child => renderNode(child, visited + region.id))++ encodeScopedRegion(region, isCrossCutting).deepMerge(Json.obj("children" -> children.asJson))+ }++ renderNode(view.target, Set.empty)+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scalaindex 96c06599..3189be0d 100644--- a/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scala+++ b/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scala@@ -1,128 +1,24 @@package ix.memory.apiimport cats.effect.IO-import io.circe.{Decoder, Json}-import io.circe.generic.semiauto.deriveDecoder-import io.circe.syntax._+import io.circe.Jsonimport org.http4s.HttpRoutesimport org.http4s.circe.CirceEntityCodec._import org.http4s.dsl.io._-import ix.memory.map.{ArchitectureMap, MapPreflightResult, MapService, PersistenceEstimate, Region}-import ix.memory.model.NodeId+import ix.memory.map.MapServiceclass MapRoutes(mapService: MapService) {- private case class MapRequest(full: Option[Boolean])- private implicit val mapReqDecoder: Decoder[MapRequest] = deriveDecoder-val routes: HttpRoutes[IO] = HttpRoutes.of[IO] {/** POST /v1/map — Run the full map pipeline and return the hierarchy. */case req @ POST -> Root / "v1" / "map" =>(for {- body <- req.as[MapRequest].handleErrorWith(_ => IO.pure(MapRequest(None)))- archMap <- mapService.buildMap(forceFullLocal = body.full.getOrElse(false))- resp <- Ok(encodeMap(archMap))+ body <- req.as[Json].handleError(_ => Json.obj())+ force = body.hcursor.get[Boolean]("full").toOption.getOrElse(false)+ archMap <- mapService.buildMap(forceRecompute = force)+ resp <- Ok(MapEncoder.encodeMap(archMap))} yield resp).handleErrorWith(ErrorHandler.handle(_))}-- private def encodeMap(m: ArchitectureMap): Json = {- val regionsByLevel = m.regions.groupBy(_.level).toVector.sortBy(_._1)-- val base = Json.obj(- "file_count" -> m.fileCount.asJson,- "region_count" -> m.regions.size.asJson,- "levels" -> regionsByLevel.size.asJson,- "map_rev" -> m.mapRev.asJson,- "regions" -> m.regions.sortBy(r => (r.level, r.label)).map(encodeRegion).asJson,- "hierarchy" -> buildHierarchyTree(m).asJson- )-- val withOutcome = base.deepMerge(Json.obj("outcome" -> m.outcome.label.asJson))-- val withPreflight = m.preflight match {- case Some(pf) => withOutcome.deepMerge(Json.obj("preflight" -> encodePreflight(pf)))- case None => withOutcome- }-- m.persistenceEstimate match {- case Some(pe) => withPreflight.deepMerge(Json.obj("persistence" -> encodePersistence(pe)))- case None => withPreflight- }- }-- private def encodePreflight(pf: MapPreflightResult): Json =- Json.obj(- "cost" -> Json.obj(- "file_count" -> pf.cost.fileCount.asJson,- "directory_count" -> pf.cost.directoryCount.asJson,- "directory_quadratic" -> pf.cost.directoryQuadratic.asJson,- "symbol_estimate" -> pf.cost.symbolEstimate.asJson,- "edge_estimate" -> pf.cost.edgeEstimate.asJson- ),- "capacity" -> Json.obj(- "cpu_cores" -> pf.capacity.cpuCores.asJson,- "heap_max_bytes" -> pf.capacity.heapMaxBytes.asJson,- "heap_free_bytes" -> pf.capacity.heapFreeBytes.asJson,- "container_memory" -> pf.capacity.containerMemory.asJson,- "disk_free_bytes" -> pf.capacity.diskFreeBytes.asJson- ),- "risk" -> pf.risk.label.asJson,- "mode" -> pf.mode.label.asJson,- "warnings" -> pf.warnings.asJson,- "duration_ms" -> pf.durationMs.asJson- )-- private def encodePersistence(pe: PersistenceEstimate): Json =- Json.obj(- "region_nodes" -> pe.regionNodes.asJson,- "file_edges" -> pe.fileEdges.asJson,- "region_edges" -> pe.regionEdges.asJson,- "delete_ops" -> pe.deleteOps.asJson,- "total_ops" -> pe.totalOps.asJson- )-- private def encodeRegion(r: Region): Json =- Json.obj(- "id" -> r.id.value.toString.asJson,- "label" -> r.label.asJson,- "label_kind" -> r.labelKind.asJson,- "level" -> r.level.asJson,- "file_count" -> r.memberFiles.size.asJson,- "child_region_count" -> r.childRegionIds.size.asJson,- "parent_id" -> r.parentId.map(_.value.toString).asJson,- "cohesion" -> r.cohesion.asJson,- "external_coupling" -> r.externalCoupling.asJson,- "boundary_ratio" -> r.boundaryRatio.asJson,- "confidence" -> r.confidence.asJson,- "crosscut_score" -> r.crosscutScore.asJson,- "dominant_signals" -> r.dominantSignals.asJson,- "interface_node_count" -> r.interfaceNodeCount.asJson- )-- /**- * Build a tree structure: top-level regions contain nested children.- * Returns a JSON array of root regions (no parent), each with a recursive- * `children` field.- */- private def buildHierarchyTree(m: ArchitectureMap): Json = {- val regionById: Map[NodeId, Region] = m.regions.map(r => r.id -> r).toMap-- // Guard against cycles (can arise when Louvain levels have identical partitions)- def renderNode(r: Region, visited: Set[NodeId]): Json = {- val children =- if (visited.contains(r.id)) Vector.empty- else r.childRegionIds.toVector- .flatMap(cid => regionById.get(cid))- .filter(c => c.level < r.level) // strict level ordering: child must be finer- .sortBy(_.label)- .map(c => renderNode(c, visited + r.id))-- encodeRegion(r).deepMerge(Json.obj("children" -> children.asJson))- }-- val roots = m.regions.filter(_.parentId.isEmpty).sortBy(-_.level)- roots.map(r => renderNode(r, Set.empty)).asJson- }}diff --git a/memory-layer/src/main/scala/ix/memory/api/PatchCommitRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/PatchCommitRoutes.scalaindex 4b2818e0..19d7e7e9 100644--- a/memory-layer/src/main/scala/ix/memory/api/PatchCommitRoutes.scala+++ b/memory-layer/src/main/scala/ix/memory/api/PatchCommitRoutes.scala@@ -1,16 +1,19 @@package ix.memory.apiimport cats.effect.IO+import cats.syntax.traverse._import io.circe.syntax._import io.circe.Jsonimport org.http4s.HttpRoutesimport org.http4s.circe.CirceEntityCodec._import org.http4s.dsl.io._-import ix.memory.db.GraphWriteApi-import ix.memory.model.GraphPatch+import ix.memory.db.{BulkWriteApi, CommitResult, CommitStatus, FileBatch, GraphQueryApi, GraphWriteApi}+import ix.memory.model.{GraphPatch, Rev}-class PatchCommitRoutes(writeApi: GraphWriteApi) {+class PatchCommitRoutes(writeApi: GraphWriteApi, bulkWriteApi: BulkWriteApi, queryApi: GraphQueryApi) {++ private val mapper = new com.fasterxml.jackson.databind.ObjectMapper()val routes: HttpRoutes[IO] = HttpRoutes.of[IO] {case req @ POST -> Root / "v1" / "patch" =>@@ -22,5 +25,40 @@ class PatchCommitRoutes(writeApi: GraphWriteApi) {"rev" -> result.newRev.value.asJson))} yield resp).handleErrorWith(ErrorHandler.handle(_))++ // Batch endpoint: use BulkWriteApi (bulk document inserts, no per-patch transaction)+ // instead of traverse(commitPatch) which was N serial ArangoDB transactions.+ case req @ POST -> Root / "v1" / "patches" / "batch" =>+ (for {+ patches <- req.as[Vector[GraphPatch]]+ latestRev <- queryApi.getLatestRev+ result <- if (patches.isEmpty)+ IO.pure(CommitResult(latestRev, CommitStatus.Ok))+ else {+ val fileBatches = patches.map(p => FileBatch(+ filePath = p.source.uri,+ sourceHash = p.source.sourceHash,+ patch = p,+ provenance = buildProvenanceMap(p)+ ))+ bulkWriteApi.commitBatchChunked(fileBatches, latestRev.value)+ }+ // Return one result entry per patch (same rev for all — one bulk revision)+ resp <- Ok(patches.map(_ => Json.obj(+ "status" -> result.status.toString.asJson,+ "rev" -> result.newRev.value.asJson+ )).asJson)+ } yield resp).handleErrorWith(ErrorHandler.handle(_))+ }++ private def buildProvenanceMap(patch: GraphPatch): java.util.Map[String, AnyRef] = {+ val json = Json.obj(+ "source_uri" -> Json.fromString(patch.source.uri),+ "source_hash" -> patch.source.sourceHash.fold(Json.Null)(Json.fromString),+ "extractor" -> Json.fromString(patch.source.extractor),+ "source_type" -> Json.fromString(patch.source.sourceType.asJson.asString.getOrElse("code")),+ "observed_at" -> Json.fromString(patch.timestamp.toString)+ )+ mapper.readValue(json.noSpaces, classOf[java.util.Map[String, AnyRef]])}}diff --git a/memory-layer/src/main/scala/ix/memory/api/PatchRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/PatchRoutes.scalaindex b050aa46..2a33a4e4 100644--- a/memory-layer/src/main/scala/ix/memory/api/PatchRoutes.scala+++ b/memory-layer/src/main/scala/ix/memory/api/PatchRoutes.scala@@ -4,16 +4,24 @@ import java.util.UUIDimport cats.effect.IOimport io.circe.syntax._+import io.circe.{Decoder, Encoder}+import io.circe.generic.semiauto._import org.http4s.HttpRoutesimport org.http4s.circe.CirceEntityCodec._import org.http4s.dsl.io._import org.http4s.dsl.impl.OptionalQueryParamDecoderMatcher-import ix.memory.db.ArangoClient+import ix.memory.db.{ArangoClient, GraphQueryApi}object LimitParam extends OptionalQueryParamDecoderMatcher[Int]("limit")-class PatchRoutes(client: ArangoClient) {+case class SourceHashesRequest(uris: List[String])+object SourceHashesRequest {+ implicit val decoder: Decoder[SourceHashesRequest] = deriveDecoder+ implicit val encoder: Encoder[SourceHashesRequest] = deriveEncoder+}++class PatchRoutes(client: ArangoClient, queryApi: GraphQueryApi) {val routes: HttpRoutes[IO] = HttpRoutes.of[IO] {case GET -> Root / "v1" / "patches" :? LimitParam(limit) =>@@ -46,5 +54,12 @@ class PatchRoutes(client: ArangoClient) {)resp <- Ok(patch)} yield resp).handleErrorWith(ErrorHandler.handle(_))++ case req @ POST -> Root / "v1" / "source-hashes" =>+ (for {+ body <- req.as[SourceHashesRequest]+ hashes <- queryApi.getSourceHashes(body.uris)+ resp <- Ok(hashes.asJson)+ } yield resp).handleErrorWith(ErrorHandler.handle(_))}}diff --git a/memory-layer/src/main/scala/ix/memory/api/ResetRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/ResetRoutes.scalaindex 9b1847e1..898d445b 100644--- a/memory-layer/src/main/scala/ix/memory/api/ResetRoutes.scala+++ b/memory-layer/src/main/scala/ix/memory/api/ResetRoutes.scala@@ -19,5 +19,12 @@ class ResetRoutes(client: ArangoClient) {_ <- client.truncateGraph()r <- Ok(Json.obj("ok" -> true.asJson, "message" -> "Graph reset. All nodes and edges deleted.".asJson))} yield r).handleErrorWith(ErrorHandler.handle(_))++ /** POST /v1/reset/code — Delete only code-derived nodes/edges; preserve planning artifacts. */+ case POST -> Root / "v1" / "reset" / "code" =>+ (for {+ _ <- client.truncateCodeGraph()+ r <- Ok(Json.obj("ok" -> true.asJson, "message" -> "Code graph reset. Planning artifacts (goals, plans, tasks, bugs, decisions) preserved.".asJson))+ } yield r).handleErrorWith(ErrorHandler.handle(_))}}diff --git a/memory-layer/src/main/scala/ix/memory/api/Routes.scala b/memory-layer/src/main/scala/ix/memory/api/Routes.scalaindex 638b714f..376a16c9 100644--- a/memory-layer/src/main/scala/ix/memory/api/Routes.scala+++ b/memory-layer/src/main/scala/ix/memory/api/Routes.scala@@ -13,6 +13,8 @@ import ix.memory.context.ContextServiceimport ix.memory.db.{ArangoClient, BulkWriteApi, GraphQueryApi, GraphWriteApi}import ix.memory.ingestion.{BulkIngestionService, IngestionService}import ix.memory.map.MapService+import ix.memory.smell.SmellService+import ix.memory.subsystem.SubsystemScoringServiceobject Routes {@@ -25,7 +27,9 @@ object Routes {conflictService: ConflictService,client: ArangoClient,mapService: MapService,- bulkWriteApi: BulkWriteApi+ bulkWriteApi: BulkWriteApi,+ smellService: SmellService,+ subsystemService: SubsystemScoringService): HttpRoutes[IO] = {val health = HttpRoutes.of[IO] {@@ -42,21 +46,24 @@ object Routes {val searchRoutes = new SearchRoutes(queryApi).routesval truthRoutes = new TruthRoutes(writeApi, queryApi).routesval goalRoutes = new GoalRoutes(writeApi, queryApi).routes- val patchRoutes = new PatchRoutes(client).routes+ val patchRoutes = new PatchRoutes(client, queryApi).routesval decisionListRoutes = new DecisionRoutes(queryApi).routesval expandRoutes = new ExpandRoutes(queryApi).routesval statsRoutes = new StatsRoutes(client).routes- val patchCommitRoutes = new PatchCommitRoutes(writeApi).routes+ val patchCommitRoutes = new PatchCommitRoutes(writeApi, bulkWriteApi, queryApi).routesval listRoutes = new ListRoutes(queryApi).routesval mapRoutes = new MapRoutes(mapService).routesval resetRoutes = new ResetRoutes(client).routesval sourceHashRoutes = new SourceHashRoutes(queryApi).routesval bulkPatchCommitRoutes = new BulkPatchCommitRoutes(bulkWriteApi, queryApi).routes+ val smellRoutes = new SmellRoutes(smellService).routes+ val subsystemRoutes = new SubsystemRoutes(subsystemService, mapService).routeshealth <+> contextRoutes <+> ingestionRoutes <+> entityRoutes <+>diffRoutes <+> conflictRoutes <+> decideRoutes <+> searchRoutes <+>truthRoutes <+> patchRoutes <+> decisionListRoutes <+> expandRoutes <+>statsRoutes <+> patchCommitRoutes <+> listRoutes <+> goalRoutes <+>- mapRoutes <+> resetRoutes <+> sourceHashRoutes <+> bulkPatchCommitRoutes+ mapRoutes <+> resetRoutes <+> sourceHashRoutes <+> bulkPatchCommitRoutes <+>+ smellRoutes <+> subsystemRoutes}}diff --git a/memory-layer/src/main/scala/ix/memory/api/SmellRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/SmellRoutes.scalanew file mode 100644index 00000000..2ed44f05--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/api/SmellRoutes.scala@@ -0,0 +1,61 @@+package ix.memory.api++import cats.effect.IO+import io.circe.Json+import io.circe.syntax._+import org.http4s.HttpRoutes+import org.http4s.circe.CirceEntityCodec._+import org.http4s.dsl.io._+import org.http4s.dsl.impl.OptionalQueryParamDecoderMatcher++import ix.memory.smell.{SmellConfig, SmellService}++object OrphanThresholdParam extends OptionalQueryParamDecoderMatcher[Int]("orphan-max-connections")+object GodChunkParam extends OptionalQueryParamDecoderMatcher[Int]("god-module-chunks")+object GodFanParam extends OptionalQueryParamDecoderMatcher[Int]("god-module-fan")+object WeakNeighborParam extends OptionalQueryParamDecoderMatcher[Int]("weak-max-neighbors")++class SmellRoutes(smellService: SmellService) {++ val routes: HttpRoutes[IO] = HttpRoutes.of[IO] {++ /** POST /v1/smells — run smell detection and persist claims. */+ case req @ POST -> Root / "v1" / "smells"+ :? OrphanThresholdParam(orphanMax)+ +& GodChunkParam(godChunks)+ +& GodFanParam(godFan)+ +& WeakNeighborParam(weakNeighbors) =>++ val config = SmellConfig(+ orphanMaxConnections = orphanMax.getOrElse(0),+ godModuleChunkThreshold = godChunks.getOrElse(20),+ godModuleFanThreshold = godFan.getOrElse(15),+ weakComponentMaxNeighbors = weakNeighbors.getOrElse(1)+ )+ (for {+ report <- smellService.run(config)+ resp <- Ok(Json.obj(+ "rev" -> report.rev.asJson,+ "run_at" -> report.runAt.asJson,+ "count" -> report.candidates.size.asJson,+ "candidates" -> report.candidates.map { c =>+ Json.obj(+ "file_id" -> c.fileId.asJson,+ "file" -> c.fileName.asJson,+ "smell" -> c.smellKind.field.asJson,+ "confidence" -> c.confidence.asJson,+ "signals" -> c.signals.asJson,+ "inference_version" -> "smell_v1".asJson+ )+ }.asJson+ ))+ } yield resp).handleErrorWith(ErrorHandler.handle(_))++ /** GET /v1/smells — retrieve existing smell claims without rerunning. */+ case GET -> Root / "v1" / "smells" =>+ (for {+ smells <- smellService.listSmells()+ resp <- Ok(Json.obj("smells" -> smells.asJson))+ } yield resp).handleErrorWith(ErrorHandler.handle(_))+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/api/SubsystemRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/SubsystemRoutes.scalanew file mode 100644index 00000000..91d95d59--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/api/SubsystemRoutes.scala@@ -0,0 +1,118 @@+package ix.memory.api++import cats.effect.IO+import io.circe.Json+import io.circe.syntax._+import org.http4s.HttpRoutes+import org.http4s.circe.CirceEntityCodec._+import org.http4s.dsl.io._++import ix.memory.map.MapService+import ix.memory.subsystem.SubsystemScoringService++class SubsystemRoutes(+ scoringService: SubsystemScoringService,+ mapService: MapService+) {++ val routes: HttpRoutes[IO] = HttpRoutes.of[IO] {++ /** GET /v1/subsystems/map — retrieve the persisted architecture map or a scoped region view. */+ case req @ GET -> Root / "v1" / "subsystems" / "map" =>+ val params = req.uri.query.params+ val target = params.get("target").map(_.trim).filter(_.nonEmpty)+ val pickRaw = params.get("pick").map(_.trim).filter(_.nonEmpty)+ (for {+ resp <- parsePick(pickRaw) match {+ case Left(message) =>+ BadRequest(Json.obj("error" -> message.asJson))+ case Right(pick) =>+ for {+ archMap <- mapService.loadCachedMap().flatMap {+ case Some(cached) => IO.pure(cached)+ case None => mapService.getOrBuildMap()+ }+ response <- target match {+ case None =>+ if (pick.isDefined) {+ BadRequest(Json.obj("error" -> "--pick requires a target.".asJson))+ } else {+ Ok(MapEncoder.encodeMap(archMap))+ }+ case Some(query) =>+ val matches = mapService.resolveRegion(archMap, query)+ if (matches.isEmpty) {+ NotFound(Json.obj(+ "error" -> "unknown_target".asJson,+ "target_query" -> query.asJson,+ "message" -> s"""No architecture region matched "$query".""".asJson,+ "suggestions" -> Vector("ix subsystems", "ix map").asJson+ ))+ } else {+ selectMatch(matches, pick) match {+ case Left(message) =>+ BadRequest(Json.obj("error" -> message.asJson))+ case Right(None) =>+ BadRequest(MapEncoder.encodeAmbiguousTarget(query, matches, region => mapService.parentOf(archMap, region)))+ case Right(Some(selected)) =>+ Ok(MapEncoder.encodeScopedView(mapService.scopedView(archMap, selected.region), mapService.isCrossCutting))+ }+ }+ }+ } yield response+ }+ } yield resp).handleErrorWith(ErrorHandler.handle(_))++ /** POST /v1/subsystems/score — run health scoring and persist claims. */+ case POST -> Root / "v1" / "subsystems" / "score" =>+ (for {+ scores <- scoringService.score()+ resp <- Ok(Json.obj(+ "count" -> scores.size.asJson,+ "scores" -> scores.map { s =>+ Json.obj(+ "region_id" -> s.regionId.asJson,+ "name" -> s.regionName.asJson,+ "level" -> s.level.asJson,+ "label_kind" -> s.labelKind.asJson,+ "file_count" -> s.fileCount.asJson,+ "health_score" -> s.healthScore.asJson,+ "chunk_density"-> s.chunkDensity.asJson,+ "smell_rate" -> s.smellRate.asJson,+ "smell_files" -> s.smellFiles.asJson,+ "total_chunks" -> s.totalChunks.asJson,+ "confidence" -> s.confidence.asJson,+ "inference_version" -> "subsystem_v1".asJson+ )+ }.asJson+ ))+ } yield resp).handleErrorWith(ErrorHandler.handle(_))++ /** GET /v1/subsystems — retrieve stored health scores. */+ case GET -> Root / "v1" / "subsystems" =>+ (for {+ scores <- scoringService.listScores()+ resp <- Ok(Json.obj("scores" -> scores.asJson))+ } yield resp).handleErrorWith(ErrorHandler.handle(_))+ }++ private def parsePick(pick: Option[String]): Either[String, Option[Int]] =+ pick match {+ case None => Right(None)+ case Some(value) =>+ scala.util.Try(value.toInt).toOption match {+ case Some(parsed) if parsed > 0 => Right(Some(parsed))+ case _ => Left("Invalid --pick value.")+ }+ }++ private def selectMatch(matches: Vector[ix.memory.map.RegionMatch], pick: Option[Int]): Either[String, Option[ix.memory.map.RegionMatch]] =+ pick match {+ case None if matches.size == 1 => Right(matches.headOption)+ case None => Right(None)+ case Some(index) if index <= matches.size =>+ Right(matches.lift(index - 1))+ case Some(index) =>+ Left(s"Invalid --pick value: $index\nThere are only ${matches.size} matches for the requested target.")+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/db/ArangoClient.scala b/memory-layer/src/main/scala/ix/memory/db/ArangoClient.scalaindex ddee578a..7ddc575e 100644--- a/memory-layer/src/main/scala/ix/memory/db/ArangoClient.scala+++ b/memory-layer/src/main/scala/ix/memory/db/ArangoClient.scala@@ -119,6 +119,37 @@ class ArangoClient private (db: ArangoDatabase) {db.collection("conflict_sets").truncate()()}++ /**+ * Delete only code-derived nodes and edges, preserving planning artifacts+ * (goals/intents, plans, tasks, decisions, bugs and their linking edges).+ */+ def truncateCodeGraph(): IO[Unit] = {+ val codeKinds = Array(+ "file", "function", "class", "method", "interface",+ "module", "chunk", "region"+ )+ val codePredicates = Array(+ "CALLS", "IMPORTS", "EXTENDS", "IMPLEMENTS",+ "CONTAINS", "DEFINES", "CONTAINS_CHUNK", "REFERENCES", "NEXT", "IN_REGION"+ )+ for {+ _ <- execute(+ "FOR n IN nodes FILTER n.kind IN @kinds REMOVE n IN nodes",+ Map("kinds" -> codeKinds.asInstanceOf[AnyRef])+ )+ _ <- execute(+ "FOR e IN edges FILTER e.predicate IN @predicates REMOVE e IN edges",+ Map("predicates" -> codePredicates.asInstanceOf[AnyRef])+ )+ // Also remove code-ingest patches so getSourceHashes returns empty after reset,+ // preventing the ingest client from treating all files as unchanged.+ _ <- execute(+ "FOR p IN patches FILTER STARTS_WITH(p.data.source.extractor, 'tree-sitter') REMOVE p IN patches",+ Map.empty+ )+ } yield ()+ }}object ArangoClient {diff --git a/memory-layer/src/main/scala/ix/memory/db/ArangoGraphWriteApi.scala b/memory-layer/src/main/scala/ix/memory/db/ArangoGraphWriteApi.scalaindex 33f6c44e..5900f8fa 100644--- a/memory-layer/src/main/scala/ix/memory/db/ArangoGraphWriteApi.scala+++ b/memory-layer/src/main/scala/ix/memory/db/ArangoGraphWriteApi.scala@@ -3,6 +3,8 @@ package ix.memory.dbimport java.time.Instantimport java.util.UUID+import scala.collection.mutable+import cats.effect.IOimport cats.syntax.traverse._import cats.syntax.foldable._@@ -16,6 +18,22 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {// I1 fix: single ObjectMapper instance shared across all callsprivate val mapper = new com.fasterxml.jackson.databind.ObjectMapper()+ private val CurrentRevisionKey = "current"+ private val SourceGraphRevisionKey = "source-graph"+ private val SourceGraphIgnoredKinds: Set[NodeKind] = Set(+ NodeKind.Module,+ NodeKind.Config,+ NodeKind.ConfigEntry,+ NodeKind.Doc,+ NodeKind.Decision,+ NodeKind.Intent,+ NodeKind.Bug,+ NodeKind.Plan,+ NodeKind.Task,+ NodeKind.Goal,+ NodeKind.Region+ )+ private val SourceGraphPredicates = Set("CALLS", "IMPORTS", "EXTENDS", "IMPLEMENTS")// ── Collections involved in a commit transaction ──────────────────@@ -58,7 +76,11 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {private def idempotencyKey(patchId: PatchId): String =patchId.value.toString- private def doCommit(patch: GraphPatch, txId: String): IO[CommitResult] =+ private def doCommit(patch: GraphPatch, txId: String): IO[CommitResult] = {+ val entityIdsForClaims = patch.ops.collect {+ case PatchOp.AssertClaim(eid, _, _, _, _) => eid.value.toString+ }.toSet+for {// Step 2: Load latest_rev (inside transaction)latestRev <- loadLatestRev(txId)@@ -71,23 +93,28 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {IO.pure(CommitResult(Rev(latestRev), CommitStatus.BaseRevMismatch))} else {val newRev = latestRev + 1L+ val advanceSourceGraphRev = shouldAdvanceSourceGraphRev(patch)for {+ // Step 4b: Pre-load active claims for all entities in this patch (one query)+ // Eliminates per-claim hasIdenticalActiveClaim DB round-trips+ claimCache <- loadActiveClaimsCache(entityIdsForClaims, txId)// Step 5: Execute each op- _ <- patch.ops.traverse_(op => executeOp(op, patch, newRev, txId))- // Step 5b: Retire absent claims (claims from same extractor/entity no longer emitted)- _ <- retireAbsentClaims(patch, newRev, txId)+ _ <- patch.ops.traverse_(op => executeOp(op, patch, newRev, txId, claimCache))+ // Step 5b: Retire absent claims — single batch query instead of one per entity+ _ <- retireAbsentClaimsBatch(patch, newRev, txId)// Step 6: Store patch_ <- storePatch(patch, newRev, txId)// Step 7: Store idempotency key_ <- storeIdempotencyKey(patch.patchId, newRev, txId)// Step 8: Update revision- _ <- updateRevision(newRev, txId)+ _ <- updateRevision(newRev, txId, advanceSourceGraphRev)// Step 9: Commit the transaction_ <- client.commitTransaction(txId)} yield CommitResult(Rev(newRev), CommitStatus.Ok)}}} yield result+ }private def loadLatestRev(txId: String): IO[Long] =client.queryOne(@@ -98,7 +125,7 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {txId = Some(txId)).map(_.flatMap(_.as[Long].toOption).getOrElse(0L))- private def executeOp(op: PatchOp, patch: GraphPatch, newRev: Long, txId: String): IO[Unit] = op match {+ private def executeOp(op: PatchOp, patch: GraphPatch, newRev: Long, txId: String, claimCache: mutable.Set[(String, String, String)]): IO[Unit] = op match {case PatchOp.UpsertNode(id, kind, name, attrs) =>val logicalId = id.value.toStringval now = Instant.now().toString@@ -273,41 +300,48 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {txId = Some(txId))- case PatchOp.AssertClaim(entityId, field, value, confidence) =>+ case PatchOp.AssertClaim(entityId, field, value, confidence, inferenceVersion) =>val provenanceJson = buildProvenance(patch)+ val eid = entityId.value.toString+ val valueStr = value.noSpacesfor {// Step 1: Retire active claims for same (entity_id, field) with different value- _ <- retireClaims(entityId.value.toString, field, value, newRev, txId)- // Step 2: Check if identical active claim already exists- hasDuplicate <- hasIdenticalActiveClaim(entityId.value.toString, field, value, txId)- // Step 3: Only insert if no duplicate- _ <- if (hasDuplicate) IO.unit+ _ <- retireClaims(eid, field, value, newRev, txId)+ // Mirror retirement in the cache: remove entries for (eid, field) with different value+ _ = claimCache.filterInPlace { case (e, f, v) => !(e == eid && f == field && v != valueStr) }+ // Step 2: Check in-memory cache — avoids a DB round-trip per claim+ _ <- if (claimCache.contains((eid, field, valueStr))) IO.unitelse {val claimKey = UUID.randomUUID().toString- client.execute(- """INSERT {- | _key: @key,- | entity_id: @entity_id,- | field: @field,- | value: @value,- | confidence: @confidence,- | status: @status,- | created_rev: @created_rev,- | deleted_rev: null,- | provenance: @provenance- |} INTO claims""".stripMargin,- Map(- "key" -> claimKey.asInstanceOf[AnyRef],- "entity_id" -> entityId.value.toString.asInstanceOf[AnyRef],- "field" -> field.asInstanceOf[AnyRef],- "value" -> jsonToJava(value).asInstanceOf[AnyRef],- "confidence" -> confidence.map(Double.box).orNull.asInstanceOf[AnyRef],- "status" -> "active".asInstanceOf[AnyRef],- "created_rev" -> Long.box(newRev).asInstanceOf[AnyRef],- "provenance" -> parseToJavaMap(provenanceJson.noSpaces).asInstanceOf[AnyRef]- ),- txId = Some(txId)- )+ for {+ _ <- client.execute(+ """INSERT {+ | _key: @key,+ | entity_id: @entity_id,+ | field: @field,+ | value: @value,+ | confidence: @confidence,+ | status: @status,+ | created_rev: @created_rev,+ | deleted_rev: null,+ | provenance: @provenance,+ | inference_version: @inference_version+ |} INTO claims""".stripMargin,+ Map(+ "key" -> claimKey.asInstanceOf[AnyRef],+ "entity_id" -> eid.asInstanceOf[AnyRef],+ "field" -> field.asInstanceOf[AnyRef],+ "value" -> jsonToJava(value).asInstanceOf[AnyRef],+ "confidence" -> confidence.map(Double.box).orNull.asInstanceOf[AnyRef],+ "status" -> "active".asInstanceOf[AnyRef],+ "created_rev" -> Long.box(newRev).asInstanceOf[AnyRef],+ "provenance" -> parseToJavaMap(provenanceJson.noSpaces).asInstanceOf[AnyRef],+ "inference_version" -> inferenceVersion.orNull.asInstanceOf[AnyRef]+ ),+ txId = Some(txId)+ )+ _ = claimCache.add((eid, field, valueStr))+ } yield ()}} yield ()@@ -327,12 +361,10 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {private def storePatch(patch: GraphPatch, newRev: Long, txId: String): IO[Unit] = {val patchJson = patch.asJson.noSpacesclient.execute(- """INSERT {- | _key: @key,- | patch_id: @patch_id,- | rev: @rev,- | data: @data- |} INTO patches""".stripMargin,+ """UPSERT { _key: @key }+ | INSERT { _key: @key, patch_id: @patch_id, rev: @rev, data: @data }+ | UPDATE {}+ | IN patches""".stripMargin,Map("key" -> patch.patchId.value.toString.asInstanceOf[AnyRef],"patch_id" -> patch.patchId.value.toString.asInstanceOf[AnyRef],@@ -357,14 +389,32 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {txId = Some(txId))- private def updateRevision(newRev: Long, txId: String): IO[Unit] =+ private def shouldAdvanceSourceGraphRev(patch: GraphPatch): Boolean =+ patch.ops.exists {+ case PatchOp.UpsertNode(_, kind, _, _) =>+ !SourceGraphIgnoredKinds.contains(kind)+ case PatchOp.UpsertEdge(_, _, _, predicate, _) =>+ SourceGraphPredicates.contains(predicate.value)+ case PatchOp.DeleteNode(_) | PatchOp.DeleteEdge(_) =>+ patch.source.sourceType == SourceType.Code || patch.source.sourceType == SourceType.Test+ case _ =>+ false+ }++ private def updateRevision(newRev: Long, txId: String, advanceSourceGraphRev: Boolean): IO[Unit] =client.execute(- """UPSERT { _key: @key }- | INSERT { _key: @key, rev: @rev }+ """FOR key IN @keys+ | UPSERT { _key: key }+ | INSERT { _key: key, rev: @rev }| UPDATE { rev: @rev }| IN revisions""".stripMargin,Map(- "key" -> "current".asInstanceOf[AnyRef],+ "keys" -> {+ val keys =+ if (advanceSourceGraphRev) Array(CurrentRevisionKey, SourceGraphRevisionKey)+ else Array(CurrentRevisionKey)+ keys.asInstanceOf[AnyRef]+ },"rev" -> Long.box(newRev).asInstanceOf[AnyRef]),txId = Some(txId)@@ -388,32 +438,44 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {txId = Some(txId))- /** Check if an identical active claim already exists (same entity, field, value). */- private def hasIdenticalActiveClaim(entityId: String, field: String, value: Json, txId: String): IO[Boolean] =- client.queryOne(- """FOR c IN claims- | FILTER c.entity_id == @entity_id- | AND c.field == @field- | AND c.value == @value- | AND c.deleted_rev == null- | LIMIT 1- | RETURN 1""".stripMargin,- Map(- "entity_id" -> entityId.asInstanceOf[AnyRef],- "field" -> field.asInstanceOf[AnyRef],- "value" -> jsonToJava(value).asInstanceOf[AnyRef]- ),- txId = Some(txId)- ).map(_.isDefined)+ /**+ * Pre-load all active claims for the given entity IDs in one DB query.+ * Returns a mutable set of (entityId, field, valueNoSpaces) triples used+ * as an in-memory duplicate-check cache throughout a single patch commit.+ */+ private def loadActiveClaimsCache(entityIds: Set[String], txId: String): IO[mutable.Set[(String, String, String)]] =+ if (entityIds.isEmpty) IO.pure(mutable.Set.empty)+ else {+ val eids = new java.util.ArrayList[String]()+ entityIds.foreach(eids.add)+ client.query(+ """FOR c IN claims+ | FILTER c.entity_id IN @entityIds AND c.deleted_rev == null+ | RETURN { entity_id: c.entity_id, field: c.field, value: c.value }""".stripMargin,+ Map("entityIds" -> eids.asInstanceOf[AnyRef]),+ txId = Some(txId)+ ).map { results =>+ val cache = mutable.Set.empty[(String, String, String)]+ results.foreach { json =>+ val parsed = for {+ eid <- json.hcursor.get[String]("entity_id").toOption+ field <- json.hcursor.get[String]("field").toOption+ value <- json.hcursor.downField("value").focus+ } yield (eid, field, value.noSpaces)+ parsed.foreach(cache.add)+ }+ cache+ }+ }/*** After all ops execute, retire claims from the same extractor/entity that are- * no longer emitted (the "absent claims" policy). This handles the case where- * a re-parsed file no longer contains a previously asserted claim.+ * no longer emitted (the "absent claims" policy). Uses a single batch AQL query+ * across all entities instead of one query per entity.*/- private def retireAbsentClaims(patch: GraphPatch, newRev: Long, txId: String): IO[Unit] = {+ private def retireAbsentClaimsBatch(patch: GraphPatch, newRev: Long, txId: String): IO[Unit] = {val assertedClaims = patch.ops.collect {- case PatchOp.AssertClaim(entityId, field, _, _) => (entityId.value.toString, field)+ case PatchOp.AssertClaim(entityId, field, _, _, _) => (entityId.value.toString, field)}if (assertedClaims.isEmpty) IO.unit@@ -422,27 +484,34 @@ class ArangoGraphWriteApi(client: ArangoClient) extends GraphWriteApi {.groupBy(_._1).map { case (eid, pairs) => eid -> pairs.map(_._2).toSet }- val extractor = patch.source.extractor-- fieldsByEntity.toVector.traverse_ { case (entityId, fields) =>- val fieldList = new java.util.ArrayList[String]()- fields.foreach(fieldList.add)- client.execute(- """FOR c IN claims- | FILTER c.entity_id == @entity_id- | AND c.field NOT IN @fields- | AND c.deleted_rev == null- | AND c.provenance.extractor == @extractor- | UPDATE c WITH { status: "retracted", deleted_rev: @deleted_rev } IN claims""".stripMargin,- Map(- "entity_id" -> entityId.asInstanceOf[AnyRef],- "fields" -> fieldList.asInstanceOf[AnyRef],- "extractor" -> extractor.asInstanceOf[AnyRef],- "deleted_rev" -> Long.box(newRev).asInstanceOf[AnyRef]- ),- txId = Some(txId)- )+ // Build Java list of {entity_id, fields} for AQL — one query instead of N+ val entityFieldMap = new java.util.ArrayList[java.util.Map[String, AnyRef]]()+ fieldsByEntity.foreach { case (eid, fields) =>+ val m = new java.util.HashMap[String, AnyRef]()+ val fl = new java.util.ArrayList[String]()+ fields.foreach(fl.add)+ m.put("entity_id", eid)+ m.put("fields", fl)+ entityFieldMap.add(m)}++ client.execute(+ """LET fieldMap = MERGE(+ | FOR ef IN @entityFieldMap RETURN { [ef.entity_id]: ef.fields }+ |)+ |FOR c IN claims+ | FILTER c.entity_id IN ATTRIBUTES(fieldMap)+ | AND c.field NOT IN fieldMap[c.entity_id]+ | AND c.deleted_rev == null+ | AND c.provenance.extractor == @extractor+ | UPDATE c WITH { status: "retracted", deleted_rev: @deleted_rev } IN claims""".stripMargin,+ Map(+ "entityFieldMap" -> entityFieldMap.asInstanceOf[AnyRef],+ "extractor" -> patch.source.extractor.asInstanceOf[AnyRef],+ "deleted_rev" -> Long.box(newRev).asInstanceOf[AnyRef]+ ),+ txId = Some(txId)+ )}}diff --git a/memory-layer/src/main/scala/ix/memory/db/BulkWriteApi.scala b/memory-layer/src/main/scala/ix/memory/db/BulkWriteApi.scalaindex 982308c6..d0c51ae0 100644--- a/memory-layer/src/main/scala/ix/memory/db/BulkWriteApi.scala+++ b/memory-layer/src/main/scala/ix/memory/db/BulkWriteApi.scala@@ -1,5 +1,6 @@package ix.memory.db+import java.nio.charset.StandardCharsetsimport java.time.Instantimport java.util.UUID@@ -17,6 +18,8 @@ import ix.memory.model._* instead of individual AQL queries.*/class BulkWriteApi(client: ArangoClient) {+ private val CurrentRevisionKey = "current"+ private val SourceGraphRevisionKey = "source-graph"private def withRetry[A](action: IO[A], maxAttempts: Int = 3, delay: FiniteDuration = 500.millis): IO[A] = {action.handleErrorWith { err =>@@ -115,12 +118,13 @@ class BulkWriteApi(client: ArangoClient) {private def updateRevision(newRev: Long): IO[Unit] =client.execute(- """UPSERT { _key: @key }- | INSERT { _key: @key, rev: @rev }+ """FOR key IN @keys+ | UPSERT { _key: key }+ | INSERT { _key: key, rev: @rev }| UPDATE { rev: @rev }| IN revisions""".stripMargin,Map(- "key" -> "current".asInstanceOf[AnyRef],+ "keys" -> Array(CurrentRevisionKey, SourceGraphRevisionKey).asInstanceOf[AnyRef],"rev" -> Long.box(newRev).asInstanceOf[AnyRef]))@@ -147,6 +151,9 @@ case class FileBatch(private def nodeKindToString(nk: NodeKind): String =nk.asJson.asString.getOrElse(nk.toString)+ lazy val estimatedPayloadBytes: Int =+ (patch.asJson.noSpaces.getBytes(StandardCharsets.UTF_8).length * 4) + 1024+def nodeDocuments(rev: Long): Vector[java.util.Map[String, AnyRef]] = {val now = Instant.now().toStringpatch.ops.collect { case PatchOp.UpsertNode(id, kind, name, attrs) =>@@ -191,7 +198,7 @@ case class FileBatch(}def claimDocuments(rev: Long): Vector[java.util.Map[String, AnyRef]] =- patch.ops.collect { case PatchOp.AssertClaim(entityId, field, value, confidence) =>+ patch.ops.collect { case PatchOp.AssertClaim(entityId, field, value, confidence, inferenceVersion) =>val doc = new java.util.HashMap[String, AnyRef]()doc.put("_key", UUID.randomUUID().toString)doc.put("entity_id", entityId.value.toString)@@ -202,6 +209,7 @@ case class FileBatch(doc.put("created_rev", Long.box(rev))doc.put("deleted_rev", null)doc.put("provenance", provenance)+ doc.put("inference_version", inferenceVersion.orNull)doc: java.util.Map[String, AnyRef]}diff --git a/memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala b/memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scalaindex 3d685cf8..d94e39fe 100644--- a/memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala+++ b/memory-layer/src/main/scala/ix/memory/ingestion/BulkIngestionService.scala@@ -10,7 +10,7 @@ import io.circe.syntax._import org.typelevel.log4cats.slf4j.Slf4jLoggerimport org.typelevel.log4cats.Logger-import ix.memory.db.{BulkWriteApi, CommitResult, CommitStatus, FileBatch, GraphQueryApi}+import ix.memory.db.{BulkWriteApi, FileBatch, GraphQueryApi}import ix.memory.model._private[ingestion] sealed trait ParseOutcome@@ -32,6 +32,9 @@ class BulkIngestionService(private val parallelism = Runtime.getRuntime.availableProcessors.max(2)private val MaxFileBytes: Long = 1024 * 1024 // 1 MB+ private val ParseBatchSize = 100+ private val MaxCommitFiles = 100+ private val MaxCommitPayloadBytes = 1024 * 1024private val mapper = new com.fasterxml.jackson.databind.ObjectMapper()private val logger: Logger[IO] = Slf4jLogger.getLoggerFromName[IO]("ix.ingestion")@@ -46,55 +49,69 @@ class BulkIngestionService(force: Boolean = false): IO[IngestionResult] = {for {- files <- discoverFiles(path, language, recursive)- _ <- onProgress(IngestionProgress(files.size, 0, 0, 0))- hashMap <- if (force) IO.pure(Map.empty[String, String]) else loadExistingHashes(files)- outcomes <- files.parTraverseN(parallelism) { f =>- parseFile(f, hashMap).handleErrorWith { err =>- logger.warn(s"Parse error: ${f.toString} — ${err.getMessage}") *>- IO.pure(Skipped("parseError"): ParseOutcome)- }+ files <- discoverFiles(path, language, recursive)+ _ <- onProgress(IngestionProgress(files.size, 0, 0, 0))+ latestRev <- queryApi.getLatestRev+ state <- files.grouped(ParseBatchSize).toVector.foldLeft(IO.pure(BulkIngestionState.empty(latestRev.value))) {+ case (stateIO, batch) =>+ stateIO.flatMap(processBatch(batch, force, files.size, onProgress, _))}- validBatches = outcomes.collect { case Parsed(b) => b }- unchangedCount = outcomes.count { case Skipped("unchanged") => true; case _ => false }- emptyCount = outcomes.count { case Skipped("emptyFile") => true; case _ => false }- errorCount = outcomes.count { case Skipped("parseError") => true; case _ => false }- tooLargeCount = outcomes.count { case Skipped("tooLarge") => true; case _ => false }- skippedCount = unchangedCount + emptyCount + errorCount + tooLargeCount- totalChunks = if (validBatches.isEmpty) 0 else (validBatches.size + 99) / 100- _ <- onProgress(IngestionProgress(files.size, outcomes.size, 0, totalChunks))- latestRev <- queryApi.getLatestRev- result <- if (validBatches.isEmpty) IO.pure(CommitResult(latestRev, CommitStatus.Ok))- else commitChunkedWithProgress(validBatches.toVector, latestRev.value, onProgress, files.size, outcomes.size)} yield IngestionResult(filesProcessed = files.size,- patchesApplied = validBatches.size,- filesSkipped = skippedCount,- entitiesCreated = validBatches.flatMap(_.patch.ops.collect { case _: PatchOp.UpsertNode => 1 }).size,- latestRev = result.newRev,- skipReasons = SkipReasons(unchanged = unchangedCount, emptyFile = emptyCount, parseError = errorCount, tooLarge = tooLargeCount)+ patchesApplied = state.patchesApplied,+ filesSkipped = state.skipped.total,+ entitiesCreated = state.entitiesCreated,+ latestRev = Rev(state.currentRev),+ skipReasons = state.skipped)}- private def commitChunkedWithProgress(- batches: Vector[FileBatch],- baseRev: Long,- onProgress: IngestionProgress => IO[Unit],+ private def processBatch(+ files: List[Path],+ force: Boolean,discovered: Int,- parsed: Int- ): IO[CommitResult] = {- val chunkSize = 100- val chunks = batches.grouped(chunkSize).toVector- val totalChunks = chunks.size-- chunks.zipWithIndex.foldLeft(IO.pure(baseRev)) { case (revIO, (chunk, idx)) =>- revIO.flatMap { currentRev =>- bulkWriteApi.commitBatch(chunk, currentRev).flatMap { result =>- onProgress(IngestionProgress(discovered, parsed, idx + 1, totalChunks)) *>- IO.pure(result.newRev.value)+ onProgress: IngestionProgress => IO[Unit],+ state: BulkIngestionState+ ): IO[BulkIngestionState] = {+ for {+ hashMap <- if (force) IO.pure(Map.empty[String, String]) else loadExistingHashes(files)+ outcomes <- files.parTraverseN(parallelism) { f =>+ parseFile(f, hashMap).handleErrorWith { err =>+ logger.warn(s"Parse error: ${f.toString} - ${err.getMessage}") *>+ IO.pure(Skipped("parseError"): ParseOutcome)}}- }.map(finalRev => CommitResult(Rev(finalRev), CommitStatus.Ok))+ validBatches = outcomes.collect { case Parsed(batch) => batch }.toVector+ batchSkipped = SkipReasons(+ unchanged = outcomes.count { case Skipped("unchanged") => true; case _ => false },+ emptyFile = outcomes.count { case Skipped("emptyFile") => true; case _ => false },+ parseError = outcomes.count { case Skipped("parseError") => true; case _ => false },+ tooLarge = outcomes.count { case Skipped("tooLarge") => true; case _ => false }+ )+ parsedFiles = state.filesParsed + outcomes.size+ commitChunks = chunkForCommit(validBatches)+ totalChunks = state.totalChunksPlanned + commitChunks.size+ _ <- onProgress(IngestionProgress(discovered, parsedFiles, state.chunksWritten, totalChunks))+ committed <- commitChunks.foldLeft(IO.pure((state.currentRev, state.chunksWritten))) {+ case (revIO, chunk) =>+ revIO.flatMap { case (currentRev, chunksWritten) =>+ bulkWriteApi.commitBatch(chunk, currentRev).flatMap { result =>+ val updatedChunks = chunksWritten + 1+ onProgress(IngestionProgress(discovered, parsedFiles, updatedChunks, totalChunks)) *>+ IO.pure((result.newRev.value, updatedChunks))+ }+ }+ }+ entitiesCreated = validBatches.flatMap(_.patch.ops.collect { case _: PatchOp.UpsertNode => 1 }).size+ } yield state.copy(+ currentRev = committed._1,+ filesParsed = parsedFiles,+ patchesApplied = state.patchesApplied + validBatches.size,+ chunksWritten = committed._2,+ totalChunksPlanned = totalChunks,+ entitiesCreated = state.entitiesCreated + entitiesCreated,+ skipped = combineSkipReasons(state.skipped, batchSkipped)+ )}/**@@ -121,7 +138,7 @@ class BulkIngestionService(parserOpt = parserRouter.parserFor(filePath.toString)parseResult = parserOpt match {case Some(p) => p.parse(filePath.getFileName.toString, source)- case None => genericTextParse(filePath.getFileName.toString, source)+ case None => GenericTextFallback.parse(filePath.getFileName.toString, source)}patch = GraphPatchBuilder.build(filePath.toString, Some(hash), parseResult)_ <- IO.fromEither(PatchValidator.validate(patch).left.map(msg => new IllegalStateException(msg)))@@ -133,7 +150,7 @@ class BulkIngestionService(} yield result}- private def loadExistingHashes(files: List[Path]): IO[Map[String, String]] = {+ private def loadExistingHashes(files: Seq[Path]): IO[Map[String, String]] = {val paths = files.map(_.toString)queryApi.getSourceHashes(paths)}@@ -156,7 +173,7 @@ class BulkIngestionService(): IO[List[Path]] = IO.blocking {val extensions = FileDiscovery.extensionsFor(language)if (Files.isRegularFile(path)) {- if (FileDiscovery.matchesFilter(path, extensions)) List(path) else List.empty+ if (language.isEmpty || FileDiscovery.matchesFilter(path, extensions)) List(path) else List.empty} else if (Files.isDirectory(path)) {// Try git ls-files first (respects .gitignore), fall back to manual walkFileDiscovery.tryGitLsFiles(path, recursive, extensions)@@ -164,20 +181,65 @@ class BulkIngestionService(} else List.empty}+ private def chunkForCommit(batches: Vector[FileBatch]): Vector[Vector[FileBatch]] = {+ val chunks = Vector.newBuilder[Vector[FileBatch]]+ var current = Vector.empty[FileBatch]+ var currentBytes = 0++ batches.foreach { batch =>+ val batchBytes = batch.estimatedPayloadBytes+ val exceedsPayload = current.nonEmpty && currentBytes + batchBytes > MaxCommitPayloadBytes+ val exceedsCount = current.size >= MaxCommitFiles+ if (exceedsPayload || exceedsCount) {+ chunks += current+ current = Vector.empty+ currentBytes = 0+ }+ current = current :+ batch+ currentBytes += batchBytes+ }++ if (current.nonEmpty) {+ chunks += current+ }++ chunks.result()+ }++ private def combineSkipReasons(left: SkipReasons, right: SkipReasons): SkipReasons =+ SkipReasons(+ unchanged = left.unchanged + right.unchanged,+ emptyFile = left.emptyFile + right.emptyFile,+ parseError = left.parseError + right.parseError,+ tooLarge = left.tooLarge + right.tooLarge+ )+private def sha256Bytes(bytes: Array[Byte]): String = {val md = java.security.MessageDigest.getInstance("SHA-256")md.digest(bytes).map("%02x".format(_)).mkString}- private def genericTextParse(fileName: String, source: String): ParseResult = {- val lines = if (source.isEmpty) 1 else source.count(_ == '\n') + 1- ParseResult(- entities = Vector(ParsedEntity(- name = fileName, kind = NodeKind.File,- attrs = Map("content" -> Json.fromString(source)),- lineStart = 1, lineEnd = lines- )),- relationships = Vector.empty+}++private[ingestion] final case class BulkIngestionState(+ currentRev: Long,+ filesParsed: Int,+ patchesApplied: Int,+ chunksWritten: Int,+ totalChunksPlanned: Int,+ entitiesCreated: Int,+ skipped: SkipReasons+)++private[ingestion] object BulkIngestionState {+ def empty(baseRev: Long): BulkIngestionState =+ BulkIngestionState(+ currentRev = baseRev,+ filesParsed = 0,+ patchesApplied = 0,+ chunksWritten = 0,+ totalChunksPlanned = 0,+ entitiesCreated = 0,+ skipped = SkipReasons())- }}diff --git a/memory-layer/src/main/scala/ix/memory/ingestion/FileDiscovery.scala b/memory-layer/src/main/scala/ix/memory/ingestion/FileDiscovery.scalaindex 2a20186a..26e1d212 100644--- a/memory-layer/src/main/scala/ix/memory/ingestion/FileDiscovery.scala+++ b/memory-layer/src/main/scala/ix/memory/ingestion/FileDiscovery.scala@@ -11,32 +11,10 @@ import java.nio.file.{Files, Path}object FileDiscovery {private val defaultExtensions = Set(- // code- ".py", ".ts", ".tsx", ".js", ".jsx", ".mjs", ".cjs",- ".java", ".scala", ".sc", ".go", ".rs", ".c", ".h", ".cc", ".cpp", ".hpp",- ".kt", ".kts", ".swift", ".rb", ".php", ".cs", ".ex", ".exs",- ".lua", ".zig", ".nim", ".jl", ".r", ".R", ".pl", ".pm",- ".sh", ".bash", ".zsh", ".fish",-- // config / data- ".json", ".yaml", ".yml", ".toml", ".ini", ".conf", ".env", ".properties",- ".xml", ".plist", ".gradle",-- // docs- ".md", ".mdx", ".rst", ".txt",-- // build- ".sbt", ".gradle", ".cmake"- )-- /** Files matched by name regardless of extension. */- private val specialFiles = Set(- "Dockerfile", "Makefile", "Rakefile", "Gemfile", "Procfile",- "README", "LICENSE", "NOTICE", "CHANGELOG",- "Cargo.toml", "go.mod", "go.sum",- "package.json", "tsconfig.json", "pyproject.toml", "setup.py", "setup.cfg",- "build.sbt", "build.gradle", "pom.xml", "CMakeLists.txt",- ".gitignore", ".dockerignore", ".editorconfig"+ ".py", ".ts", ".tsx",+ ".scala", ".sc",+ ".json", ".yaml", ".yml", ".toml",+ ".md")def extensionsFor(language: Option[String]): Set[String] = language match {@@ -55,7 +33,7 @@ object FileDiscovery {def matchesFilter(file: Path, extensions: Set[String]): Boolean = {val name = file.getFileName.toString- specialFiles.contains(name) || extensions.exists(name.endsWith)+ extensions.exists(name.endsWith)}/**diff --git a/memory-layer/src/main/scala/ix/memory/ingestion/GenericTextFallback.scala b/memory-layer/src/main/scala/ix/memory/ingestion/GenericTextFallback.scalanew file mode 100644index 00000000..29d670ea--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/ingestion/GenericTextFallback.scala@@ -0,0 +1,32 @@+package ix.memory.ingestion++import io.circe.Json+import ix.memory.model.NodeKind++object GenericTextFallback {+ private val PreviewLimit = 240++ def parse(fileName: String, source: String): ParseResult = {+ val lines = if (source.isEmpty) 1 else source.count(_ == '\n') + 1+ val preview = source.linesIterator.find(_.trim.nonEmpty).map(_.trim.take(PreviewLimit))++ val attrs = Map(+ "file_name" -> Json.fromString(fileName),+ "line_count" -> Json.fromInt(lines)+ ) ++ preview.filter(_.nonEmpty).map(value => "preview" -> Json.fromString(value))++ ParseResult(+ entities = Vector(+ ParsedEntity(+ name = fileName,+ kind = NodeKind.File,+ attrs = attrs,+ lineStart = 1,+ lineEnd = lines,+ contentFingerprint = Some(Fingerprint.compute(fileName, source))+ )+ ),+ relationships = Vector.empty+ )+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/ingestion/IngestionService.scala b/memory-layer/src/main/scala/ix/memory/ingestion/IngestionService.scalaindex 61c4d9c4..363fbc0d 100644--- a/memory-layer/src/main/scala/ix/memory/ingestion/IngestionService.scala+++ b/memory-layer/src/main/scala/ix/memory/ingestion/IngestionService.scala@@ -8,8 +8,6 @@ import cats.effect.IOimport cats.syntax.all._import ix.memory.db.{CommitResult, GraphQueryApi, GraphWriteApi}import ix.memory.model._-import io.circe.Json-/*** Orchestrates file ingestion: reads source files, parses them,* builds graph patches, and commits them through the write API.@@ -35,7 +33,7 @@ class IngestionService(parserRouter: ParserRouter, writeApi: GraphWriteApi, querparserOpt = parserRouter.parserFor(filePath.toString)result = parserOpt match {case Some(p) => p.parse(filePath.getFileName.toString, source)- case None => genericTextParse(filePath.getFileName.toString, source)+ case None => GenericTextFallback.parse(filePath.getFileName.toString, source)}val fp = filePath.toStringextractor = if (fp.endsWith(".py")) "tree-sitter-python/1.0"@@ -103,7 +101,7 @@ class IngestionService(parserRouter: ParserRouter, writeApi: GraphWriteApi, quer): IO[List[Path]] = IO.blocking {val extensions = FileDiscovery.extensionsFor(language)if (Files.isRegularFile(path)) {- if (FileDiscovery.matchesFilter(path, extensions)) List(path) else List.empty+ if (language.isEmpty || FileDiscovery.matchesFilter(path, extensions)) List(path) else List.empty} else if (Files.isDirectory(path)) {FileDiscovery.tryGitLsFiles(path, recursive, extensions).getOrElse(FileDiscovery.walkFiles(path, recursive, extensions))@@ -230,28 +228,6 @@ class IngestionService(parserRouter: ParserRouter, writeApi: GraphWriteApi, querval md = java.security.MessageDigest.getInstance("SHA-256")md.digest(bytes).map("%02x".format(_)).mkString}-- /**- * Generic fallback parse: ensures every text file produces at least a File entity with content.- * This avoids needing a bespoke parser for every dev/text extension.- */- private def genericTextParse(fileName: String, source: String): ParseResult = {- val lines = if (source.isEmpty) 1 else source.count(_ == '\n') + 1- ParseResult(- entities = Vector(- ParsedEntity(- name = fileName,- kind = NodeKind.File,- attrs = Map(- "content" -> Json.fromString(source)- ),- lineStart = 1,- lineEnd = lines- )- ),- relationships = Vector.empty- )- }}case class IngestionProgress(@@ -266,7 +242,9 @@ case class SkipReasons(emptyFile: Int = 0,parseError: Int = 0,tooLarge: Int = 0-)+) {+ def total: Int = unchanged + emptyFile + parseError + tooLarge+}case class IngestionResult(filesProcessed: Int,diff --git a/memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scala b/memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scalaindex b357b6ec..60e7693d 100644--- a/memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scala+++ b/memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scala@@ -48,15 +48,42 @@ object LouvainClustering {def cluster(graph: WeightedFileGraph,maxLevels: Int = 4,- resolution: Double = 1.0,+ resolution: Double = -1.0,minCommunitySize: Int = 2,seed: Long = 42L): Vector[LevelPartition] = {if (graph.vertices.isEmpty || graph.totalWeight == 0.0) return Vector.empty+ val effectiveResolution =+ if (resolution > 0.0) resolution else adaptiveResolution(graph.vertices.size)++ val numSeeds = if (graph.vertices.size < 30) 1L else 3L+ val runs = (0L until numSeeds).map { offset =>+ val levels = clusterOnce(+ graph = graph,+ maxLevels = maxLevels,+ resolution = effectiveResolution,+ minCommunitySize = minCommunitySize,+ seed = seed + offset+ )+ val score = levels.headOption+ .map(level => modularity(graph, level.assignment, effectiveResolution))+ .getOrElse(Double.NegativeInfinity)+ levels -> score+ }++ runs.maxBy(_._2)._1+ }++ private def clusterOnce(+ graph: WeightedFileGraph,+ maxLevels: Int,+ resolution: Double,+ minCommunitySize: Int,+ seed: Long+ ): Vector[LevelPartition] = {val rng = new Random(seed)- // nodeExpansion: for the current working graph, maps each node → original file IDs it representsvar nodeExpansion: Map[NodeId, Set[NodeId]] =graph.vertices.map(v => v.id -> Set(v.id)).toMap@@ -66,19 +93,14 @@ object LouvainClustering {var level = 0while (level < maxLevels && currentGraph.vertices.size > 1) {val rawAssignment = louvainPass(currentGraph, resolution, rng)-- // Project raw assignment (current node ids) → original file id communities- val projected = projectPartition(rawAssignment, nodeExpansion)-- val meaningful = projected.communities.count(_.size >= minCommunitySize)+ val projected = projectPartition(rawAssignment, nodeExpansion)+ val meaningful = projected.communities.count(_.size >= minCommunitySize)if (meaningful <= 1 && levels.nonEmpty) {- // No further useful partitioning — stoplevel = maxLevels} else {levels += projected- // Coarsen: collapse each community into a supernodeval (coarsenedGraph, newExpansion) =coarsen(currentGraph, rawAssignment, nodeExpansion)@@ -93,6 +115,11 @@ object LouvainClustering {levels.toVector}+ private def adaptiveResolution(fileCount: Int): Double =+ if (fileCount < 50) 1.2+ else if (fileCount <= 500) 1.0+ else 0.8+// ── Single Louvain pass ────────────────────────────────────────────/**@@ -104,7 +131,7 @@ object LouvainClustering {resolution: Double,rng: Random): Map[NodeId, Int] = {- val nodes = graph.vertices.map(_.id)+ val nodes = graph.vertices.map(_.id).toArrayval adj = graph.adjMatrixval m = graph.totalWeight.max(1e-12)@@ -113,32 +140,34 @@ object LouvainClustering {val sumTot = scala.collection.mutable.Map(nodes.zipWithIndex.map { case (n, i) => i -> graph.degrees.getOrElse(n, 0.0) }: _*).withDefaultValue(0.0)- val sumIn = scala.collection.mutable.Map[Int, Double]().withDefaultValue(0.0)// communities: commId → members (mutable set)val members = scala.collection.mutable.Map(nodes.zipWithIndex.map { case (n, i) => i -> scala.collection.mutable.Set(n) }: _*)- var improved = true- var passes = 0+ var continue = true+ val wToComm = scala.collection.mutable.HashMap.empty[Int, Double]- while (improved && passes < 30) {- improved = false- passes += 1+ while (continue) {+ shuffleInPlace(nodes, rng)+ var improved = false+ var passGain = 0.0- for (node <- rng.shuffle(nodes)) {+ var idx = 0+ while (idx < nodes.length) {+ val node = nodes(idx)val ci = commOf(node)val ki = graph.degrees.getOrElse(node, 0.0)// Compute weight from this node to each neighboring community- val wToComm = scala.collection.mutable.Map[Int, Double]().withDefaultValue(0.0)+ wToComm.clear()for ((neighbor, w) <- adj.getOrElse(node, Map.empty)) {- wToComm(commOf(neighbor)) += w+ val community = commOf(neighbor)+ wToComm.update(community, wToComm.getOrElse(community, 0.0) + w)}// Remove node from current communityval kiInCi = wToComm.getOrElse(ci, 0.0)- sumIn(ci) -= 2.0 * kiInCisumTot(ci) -= kimembers(ci) -= node// Temporarily isolate: treat as its own community (gains = 0 baseline)@@ -159,11 +188,16 @@ object LouvainClustering {commOf(node) = bestCommmembers.getOrElseUpdate(bestComm, scala.collection.mutable.Set()) += nodeval kiInBest = wToComm.getOrElse(bestComm, 0.0)- sumIn(bestComm) += 2.0 * kiInBestsumTot(bestComm) += ki- if (bestComm != ci) improved = true+ if (bestComm != ci) {+ improved = true+ passGain += bestGain+ }+ idx += 1}++ continue = improved && passGain >= 1e-6}// Re-index communities from 0 to N-1 (dense)@@ -176,6 +210,38 @@ object LouvainClustering {finalAssign}+ private def shuffleInPlace(nodes: Array[NodeId], rng: Random): Unit = {+ var i = nodes.length - 1+ while (i > 0) {+ val j = rng.nextInt(i + 1)+ val tmp = nodes(i)+ nodes(i) = nodes(j)+ nodes(j) = tmp+ i -= 1+ }+ }++ private def modularity(+ graph: WeightedFileGraph,+ assignment: Map[NodeId, Int],+ resolution: Double+ ): Double = {+ val communities = assignment.groupBy(_._2).map { case (idx, entries) => idx -> entries.keySet }+ val m = graph.totalWeight.max(1e-12)++ communities.values.foldLeft(0.0) { (acc, members) =>+ val totalDegree = members.iterator.map(node => graph.degrees.getOrElse(node, 0.0)).sum+ var internalWeight = 0.0+ for {+ src <- members+ (dst, weight) <- graph.adjMatrix.getOrElse(src, Map.empty)+ if members.contains(dst) && src.value.compareTo(dst.value) < 0+ } internalWeight += weight++ acc + (internalWeight / m) - resolution * math.pow(totalDegree / (2.0 * m), 2.0)+ }+ }+// ── Projection ─────────────────────────────────────────────────────private def projectPartition(@@ -262,7 +328,8 @@ object LouvainClustering {superVertices,newAdjFinal,newDegrees.toMap,- newTotalWeight+ newTotalWeight,+ Map.empty)(coarsened, newExpansion)diff --git a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scalaindex 691c4ad3..1784f991 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala@@ -3,7 +3,7 @@ package ix.memory.mapimport cats.effect.IOimport ix.memory.db.ArangoClient-import ix.memory.model.NodeId+import ix.memory.model.{NodeId, Rev}/*** Builds a weighted undirected file-level coupling graph from ArangoDB.@@ -16,16 +16,25 @@ import ix.memory.model.NodeId** Signal weights (from spec):* α = 1.0 call coupling (highest importance)- * β = 0.7 import coupling- * γ = 0.9 extends/implements (type/interface cohesion proxy)- * ε = 0.3 path proximity (weak structural prior)+ * β = 0.5 import coupling+ * γ = 1.2 extends/implements (type/interface cohesion proxy)+ * ε = 0.2 path proximity (weak structural prior)*/class MapGraphBuilder(client: ArangoClient) {private val Alpha = 1.0- private val Beta = 0.7- private val Gamma = 0.9- private val Epsilon = 0.3+ private val Beta = 0.5+ private val Gamma = 1.2+ private val Epsilon = 0.2+ private val MaxEdges = 200000+ private val PathProximityMinFiles = 2+ private val PathProximityMaxFiles = 50+ private val SourceGraphRevisionKey = "source-graph"+ private val IgnoredNodeKinds = Vector(+ "module", "config", "config_entry", "doc", "decision", "intent",+ "bug", "plan", "task", "goal", "region"+ )+ private val CouplingPredicates = Vector("CALLS", "IMPORTS", "EXTENDS", "IMPLEMENTS")def discoverFiles(): IO[Vector[FileVertex]] = fetchFiles()@@ -41,6 +50,70 @@ class MapGraphBuilder(client: ArangoClient) {graph <- buildGraph(files)} yield graph+ /**+ * Revision of the graph inputs that materially affect architecture mapping.+ * This intentionally ignores derived artifacts like persisted region nodes or+ * scoring claims so cached maps stay reusable across unchanged source graphs.+ */+ def currentInputRev(): IO[Rev] =+ loadSourceGraphRevision().flatMap {+ case Some(rev) => IO.pure(rev)+ case None => scanCurrentInputRev()+ }++ private def loadSourceGraphRevision(): IO[Option[Rev]] =+ client.queryOne(+ """FOR r IN revisions+ | FILTER r._key == @key+ | LIMIT 1+ | RETURN r.rev""".stripMargin,+ Map("key" -> SourceGraphRevisionKey.asInstanceOf[AnyRef])+ ).map(_.flatMap(_.as[Long].toOption).map(Rev(_)))++ private def scanCurrentInputRev(): IO[Rev] =+ client.queryOne(+ """LET node_rev = MAX(+ | FOR n IN nodes+ | FILTER n.provenance.source_uri != null+ | AND n.kind NOT IN @ignoredKinds+ | RETURN MAX([+ | TO_NUMBER(n.created_rev),+ | n.deleted_rev != null ? TO_NUMBER(n.deleted_rev) : 0+ | ])+ |)+ |LET edge_rev = MAX(+ | FOR e IN edges+ | FILTER e.predicate IN @predicates+ | RETURN MAX([+ | TO_NUMBER(e.created_rev),+ | e.deleted_rev != null ? TO_NUMBER(e.deleted_rev) : 0+ | ])+ |)+ |RETURN MAX([+ | node_rev != null ? node_rev : 0,+ | edge_rev != null ? edge_rev : 0+ |])""".stripMargin,+ Map(+ "ignoredKinds" -> IgnoredNodeKinds.toArray.asInstanceOf[AnyRef],+ "predicates" -> CouplingPredicates.toArray.asInstanceOf[AnyRef]+ )+ ).map { json =>+ Rev(json.flatMap(_.as[Long].toOption).getOrElse(0L))+ }++ def liveFileCount(): IO[Int] =+ client.queryOne(+ """FOR n IN nodes+ | FILTER n.kind == "file"+ | AND n.deleted_rev == null+ | AND n.provenance.source_uri != null+ | LET ext = LOWER(LAST(SPLIT(n.provenance.source_uri, ".")))+ | FILTER ext IN @extensions+ | COLLECT WITH COUNT INTO cnt+ | RETURN cnt""".stripMargin,+ Map("extensions" -> SourceExtensions.toArray.asInstanceOf[AnyRef])+ ).map(_.flatMap(_.as[Int].toOption).getOrElse(0))+// ── ArangoDB queries ───────────────────────────────────────────────/**@@ -58,16 +131,16 @@ class MapGraphBuilder(client: ArangoClient) {| FILTER n.kind == "file"| AND n.deleted_rev == null| AND n.provenance.source_uri != null+ | LET ext = LOWER(LAST(SPLIT(n.provenance.source_uri, ".")))+ | FILTER ext IN @extensions| RETURN {id: n.logical_id, path: n.provenance.source_uri}""".stripMargin,- Map.empty+ Map("extensions" -> SourceExtensions.toArray.asInstanceOf[AnyRef])).map { rows =>rows.flatMap { json =>val c = json.hcursorfor {idStr <- c.get[String]("id").toOptionpath <- c.get[String]("path").toOption- ext = path.split("\\.").lastOption.getOrElse("").toLowerCase- if SourceExtensions.contains(ext)uuid <- try Some(java.util.UUID.fromString(idStr)) catch { case _: Exception => None }} yield FileVertex(NodeId(uuid), path)}.toVector@@ -83,25 +156,27 @@ class MapGraphBuilder(client: ArangoClient) {*/private def fetchCouplingByUri(): IO[Vector[RawFilePair]] =client.query(- """LET uri_map = MERGE(+ """LET uriMap = (| FOR n IN nodes| FILTER n.deleted_rev == null| AND n.provenance.source_uri != null- | AND n.kind NOT IN ["module","config","config_entry",- | "doc","decision","intent","bug","plan",- | "task","goal","region"]- | RETURN {[n.logical_id]: n.provenance.source_uri}+ | AND n.kind NOT IN @ignoredKinds+ | RETURN { id: n.logical_id, uri: n.provenance.source_uri }|)+ |LET byId = ZIP(uriMap[*].id, uriMap[*].uri)|FOR e IN edges- | FILTER e.predicate IN ["CALLS","IMPORTS","EXTENDS","IMPLEMENTS"]+ | FILTER e.predicate IN @predicates| AND e.deleted_rev == null- | LET su = HAS(uri_map, e.src) ? uri_map[e.src] : null- | LET du = HAS(uri_map, e.dst) ? uri_map[e.dst] : null+ | LET su = byId[e.src]+ | LET du = byId[e.dst]| FILTER su != null AND du != null AND su != du| COLLECT srcUri = su, dstUri = du, pred = e.predicate| WITH COUNT INTO cnt| RETURN {srcUri, dstUri, predicate: pred, count: cnt}""".stripMargin,- Map.empty+ Map(+ "ignoredKinds" -> IgnoredNodeKinds.toArray.asInstanceOf[AnyRef],+ "predicates" -> CouplingPredicates.toArray.asInstanceOf[AnyRef]+ )).map { rows =>rows.flatMap { json =>val c = json.hcursor@@ -124,18 +199,20 @@ class MapGraphBuilder(client: ArangoClient) {// Index files by their absolute source_uri pathval byUri: Map[String, FileVertex] = files.map(v => v.path -> v).toMap+ val byId: Map[NodeId, FileVertex] = files.map(v => v.id -> v).toMap+ val pathSegmentsById: Map[NodeId, Array[String]] =+ files.map(v => v.id -> directorySegments(v.path)).toMap- // Accumulate coupling counts per (srcUri, dstUri) canonical pair+ // Accumulate coupling counts per canonical file-id pair.val pairAcc =- scala.collection.mutable.Map[(String, String), scala.collection.mutable.Map[String, Int]]()+ scala.collection.mutable.Map[(NodeId, NodeId), scala.collection.mutable.Map[String, Int]]()for (pair <- rawPairs) {val sv = byUri.get(pair.srcId) // srcId is source_uri hereval dv = byUri.get(pair.dstId)(sv, dv) match {case (Some(s), Some(d)) if s.id != d.id =>- // Canonical undirected key: lexicographically smaller path first- val key = if (s.path < d.path) (s.path, d.path) else (d.path, s.path)+ val key = canonicalPair(s.id, d.id)val m = pairAcc.getOrElseUpdate(key, scala.collection.mutable.Map())m(pair.predicate) = m.getOrElse(pair.predicate, 0) + pair.countcase _ =>@@ -145,43 +222,65 @@ class MapGraphBuilder(client: ArangoClient) {// Build adjacency with composite weightsval adjMut =scala.collection.mutable.Map[NodeId, scala.collection.mutable.Map[NodeId, Double]]()+ val predicatePairs =+ scala.collection.mutable.Map[(NodeId, NodeId), Map[String, Int]]()+ var edgeCount = 0- for (((sp, dp), predicateCounts) <- pairAcc) {- val sv = byUri(sp)- val dv = byUri(dp)+ for (((srcId, dstId), predicateCounts) <- pairAcc.toVector.sortBy { case ((a, b), _) =>+ (a.value.toString, b.value.toString)+ }) {+ val sv = byId(srcId)+ val dv = byId(dstId)+ val srcSegments = pathSegmentsById(srcId)+ val dstSegments = pathSegmentsById(dstId)+ val sameDir = isSameDirectory(srcSegments, dstSegments)val callCount = predicateCounts.getOrElse("CALLS", 0)val importCount = predicateCounts.getOrElse("IMPORTS", 0)val typeCount = predicateCounts.getOrElse("EXTENDS", 0) +predicateCounts.getOrElse("IMPLEMENTS", 0)+ val pairSignals =+ if (sameDir) predicateCounts.toMap.updated("PATH", predicateCounts.getOrElse("PATH", 0) + 1)+ else predicateCounts.toMapval sCall = Alpha * math.log1p(callCount)val sImport = Beta * math.log1p(importCount)val sType = Gamma * math.log1p(typeCount)- val sPath = Epsilon * pathProximity(sv.path, dv.path)+ val sPath = Epsilon * pathProximity(srcSegments, dstSegments, sameDir)val w = sCall + sImport + sType + sPathif (w > 0.0) {adjMut.getOrElseUpdate(sv.id, scala.collection.mutable.Map())(dv.id) = wadjMut.getOrElseUpdate(dv.id, scala.collection.mutable.Map())(sv.id) = w+ predicatePairs((srcId, dstId)) = pairSignals+ edgeCount += 1}}// Path-proximity pass: add structural edges for same-directory file pairs// that have no existing coupling edge. This ensures isolated files (no// CALLS/IMPORTS/EXTENDS edges) can still be clustered by directory.- val filesByDir = files.groupBy(v => v.path.split("[/\\\\]").dropRight(1).mkString("/"))- for ((_, dirFiles) <- filesByDir if dirFiles.size >= 2) {- for (i <- dirFiles.indices; j <- (i + 1) until dirFiles.size) {- val sv = dirFiles(i)- val dv = dirFiles(j)- if (!adjMut.get(sv.id).exists(_.contains(dv.id))) {- val w = Epsilon * pathProximity(sv.path, dv.path)- if (w > 0.0) {- adjMut.getOrElseUpdate(sv.id, scala.collection.mutable.Map())(dv.id) = w- adjMut.getOrElseUpdate(dv.id, scala.collection.mutable.Map())(sv.id) = w+ val filesByDir =+ files.groupBy(v => pathSegmentsById(v.id).mkString("/")).toVector.sortBy(_._1)+ for ((_, dirFiles) <- filesByDir if dirFiles.size >= PathProximityMinFiles &&+ dirFiles.size <= PathProximityMaxFiles &&+ edgeCount < MaxEdges) {+ val ordered = dirFiles.sortBy(_.path)+ var i = 0+ while (i < ordered.length - 1 && edgeCount < MaxEdges) {+ var j = i + 1+ while (j < ordered.length && edgeCount < MaxEdges) {+ val sv = ordered(i)+ val dv = ordered(j)+ if (!adjMut.get(sv.id).exists(_.contains(dv.id))) {+ adjMut.getOrElseUpdate(sv.id, scala.collection.mutable.Map())(dv.id) = Epsilon+ adjMut.getOrElseUpdate(dv.id, scala.collection.mutable.Map())(sv.id) = Epsilon+ predicatePairs(canonicalPair(sv.id, dv.id)) = Map("PATH" -> 1)+ edgeCount += 1}+ j += 1}+ i += 1}}@@ -189,14 +288,34 @@ class MapGraphBuilder(client: ArangoClient) {val degrees = adj.map { case (k, m) => k -> m.values.sum }val totalWeight = degrees.values.sum / 2.0- WeightedFileGraph(files, adj, degrees.toMap, totalWeight)+ WeightedFileGraph(files, adj, degrees.toMap, totalWeight, predicatePairs.toMap)+ }++ private def canonicalPair(a: NodeId, b: NodeId): (NodeId, NodeId) =+ if (a.value.compareTo(b.value) <= 0) (a, b) else (b, a)++ private def directorySegments(path: String): Array[String] =+ path.split("[/\\\\]").dropRight(1)++ private def isSameDirectory(partsA: Array[String], partsB: Array[String]): Boolean = {+ if (partsA.length != partsB.length) return false+ var i = 0+ while (i < partsA.length) {+ if (partsA(i) != partsB(i)) return false+ i += 1+ }+ true}/** Path distance heuristic: deeper common prefix → higher proximity. */- private def pathProximity(a: String, b: String): Double = {- val partsA = a.split("[/\\\\]").dropRight(1)- val partsB = b.split("[/\\\\]").dropRight(1)- val common = partsA.zip(partsB).takeWhile { case (x, y) => x == y }.length+ private def pathProximity(partsA: Array[String], partsB: Array[String], sameDir: Boolean): Double = {+ if (sameDir) return 1.0++ val maxLen = math.min(partsA.length, partsB.length)+ var common = 0+ while (common < maxLen && partsA(common) == partsB(common)) {+ common += 1+ }val dist = (partsA.length - common) + (partsB.length - common)1.0 / (1.0 + dist)}diff --git a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scala b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scalaindex acd21d41..8db07619 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scala@@ -145,7 +145,7 @@ class MapGraphBuilderFast(client: ArangoClient) {val degrees = adj.map { case (k, m) => k -> m.values.sum }val totalWeight = degrees.values.sum / 2.0- WeightedFileGraph(files, adj, degrees.toMap, totalWeight)+ WeightedFileGraph(files, adj, degrees.toMap, totalWeight, Map.empty)}private def pathProximity(a: String, b: String): Double = {diff --git a/memory-layer/src/main/scala/ix/memory/map/MapService.scala b/memory-layer/src/main/scala/ix/memory/map/MapService.scalaindex d2b6426e..f1c5c96a 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapService.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapService.scala@@ -6,7 +6,6 @@ import java.util.UUIDimport cats.effect.IOimport io.circe.Jsonimport io.circe.syntax._-import org.typelevel.log4cats.slf4j.Slf4jLoggerimport ix.memory.db.{ArangoClient, GraphQueryApi, GraphWriteApi}import ix.memory.model._@@ -27,6 +26,7 @@ class MapService(queryApi: GraphQueryApi,writeApi: GraphWriteApi) {+ private val CrosscutThreshold = 0.10// Cross-cut penalty multiplier (reduces edge weights involving crosscut files)private val CrosscutLambda = 0.5@@ -36,16 +36,18 @@ class MapService(private val CrosscutTokens = Set("util", "utils", "helper", "helpers", "common", "shared", "base","logging", "log", "logger", "config", "conf", "constants", "constant",- "extensions", "ext", "mixin", "mixins", "core", "framework"+ "extensions", "ext"+ )+ private val SignalWeights = Map(+ "CALLS" -> 1.0,+ "IMPORTS" -> 0.5,+ "EXTENDS" -> 1.2,+ "IMPLEMENTS" -> 1.2,+ "PATH" -> 0.2)- private val logger = Slf4jLogger.getLoggerFromName[IO]("ix.map.service")- private val builder = new MapGraphBuilder(client)- private val fastBuilder = new MapGraphBuilderFast(client)- private val preflight = new MapPreflight-- // Persistence guardrail: max total patch ops before we refuse to commit- private val MaxSafePatchOps = 10000+ private val builder = new MapGraphBuilder(client)+ private final class RetainedRegion(val level: Int, val communityIndex: Int, val region: Region)// Label kinds by levelprivate def labelKind(level: Int, maxLevel: Int): String =@@ -54,65 +56,104 @@ class MapService(else if (level == 1) "module"else "subsystem"- def buildMap(forceFullLocal: Boolean = false): IO[ArchitectureMap] =+ def buildMap(forceRecompute: Boolean = false): IO[ArchitectureMap] =+ getOrBuildMap(forceRecompute)++ def getOrBuildMap(forceRecompute: Boolean = false): IO[ArchitectureMap] =for {- rev <- queryApi.getLatestRev- files <- builder.discoverFiles()- pf <- preflight.evaluate(files)- mode = if (forceFullLocal) MapExecutionMode.FullLocal else pf.mode- result <- mode match {- case MapExecutionMode.FullLocal => buildFullLocal(files, pf, rev, forceFullLocal)- case MapExecutionMode.FastLocal => buildFastLocal(files, pf, rev)- }- } yield result-- private def buildFullLocal(- files: Vector[FileVertex],- pf: MapPreflightResult,- rev: Rev,- bypassPersistenceGuard: Boolean = false- ): IO[ArchitectureMap] =+ inputRev <- builder.currentInputRev()+ cached <- loadCachedMap(inputRev)+ archMap <- if (cached.nonEmpty && !forceRecompute) IO.pure(cached.get)+ else computeAndPersistMap(inputRev, cached)+ } yield archMap++ def loadPersistedMapForCurrentGraph(): IO[Option[ArchitectureMap]] =for {- rawGraph <- builder.buildGraph(files)- crosscutScores = detectCrosscut(rawGraph)- graph = applyPenalty(rawGraph, crosscutScores)- levels = LouvainClustering.cluster(graph, maxLevels = 3, minCommunitySize = 3)- regions = buildRegions(graph, levels, crosscutScores, rev)- pe <- guardAndPersist(regions, rev, bypassPersistenceGuard)- _ <- logOutcome(MapOutcome.FullLocalCompleted, pf, pe)- } yield ArchitectureMap(- regions, rawGraph.vertices.size, rev.value,- preflight = Some(pf),- outcome = MapOutcome.FullLocalCompleted,- persistenceEstimate = Some(pe)+ inputRev <- builder.currentInputRev()+ cached <- loadCachedMap(inputRev)+ } yield cached++ def loadCachedMap(): IO[Option[ArchitectureMap]] =+ loadPersistedMapForCurrentGraph()++ def resolveRegion(map: ArchitectureMap, target: String): Vector[RegionMatch] = {+ val query = target.trim+ val normalizedQuery = normalizeLabelQuery(query)+ if (query.isEmpty || normalizedQuery.isEmpty) Vector.empty+ else {+ map.regions.flatMap { region =>+ matchQuality(region.label, query).map { quality =>+ RegionMatch(region, quality, normalizeLabelQuery(region.label))+ }+ }.sortBy { m =>+ (+ m.matchQuality,+ levelPreference(m.region),+ -m.region.memberFiles.size,+ -m.region.confidence,+ m.normalizedLabel,+ m.region.id.value.toString+ )+ }+ }+ }++ def childrenOf(map: ArchitectureMap, region: Region): Vector[Region] = {+ val regionById = map.regions.map(r => r.id -> r).toMap+ val directChildren = region.childRegionIds.toVector+ .flatMap(regionById.get)+ .filter(_.level < region.level)++ val children =+ if (directChildren.nonEmpty) directChildren+ else map.regions.filter(_.parentId.contains(region.id))++ sortBySignificance(children.distinct)+ }++ def parentOf(map: ArchitectureMap, region: Region): Option[Region] = {+ val regionById = map.regions.map(r => r.id -> r).toMap+ region.parentId.flatMap(regionById.get)+ }++ def subtreeOf(map: ArchitectureMap, region: Region): Vector[Region] = {+ def loop(current: Region): Vector[Region] = {+ val directChildren = childrenOf(map, current)+ current +: directChildren.flatMap(loop)+ }+ loop(region)+ }++ def scopedView(map: ArchitectureMap, region: Region): ScopedSubsystemView = {+ val scopeRegions =+ if (region.labelKind == "module") Vector(region)+ else subtreeOf(map, region)++ val summary = ScopedHealthSummary(+ wellDefined = scopeRegions.count(_.confidence >= 0.75),+ moderate = scopeRegions.count(r => r.confidence >= 0.50 && r.confidence < 0.75),+ fuzzy = scopeRegions.count(_.confidence < 0.50),+ crossCutting = scopeRegions.count(isCrossCutting))- private def buildFastLocal(- files: Vector[FileVertex],- pf: MapPreflightResult,- rev: Rev- ): IO[ArchitectureMap] =- for {- rawGraph <- fastBuilder.buildGraph(files)- crosscutScores = detectCrosscut(rawGraph)- graph = applyPenalty(rawGraph, crosscutScores)- levels = LouvainClustering.cluster(graph, maxLevels = 3, minCommunitySize = 3)- regions = buildRegions(graph, levels, crosscutScores, rev)- pe <- guardAndPersist(regions, rev)- _ <- logOutcome(MapOutcome.FastLocalCompleted, pf, pe)- } yield ArchitectureMap(- regions, rawGraph.vertices.size, rev.value,- preflight = Some(pf),- outcome = MapOutcome.FastLocalCompleted,- persistenceEstimate = Some(pe)+ ScopedSubsystemView(+ target = region,+ parent = parentOf(map, region),+ children = if (region.labelKind == "module") Vector.empty else childrenOf(map, region),+ subtree = scopeRegions,+ summary = summary)+ }++ def isCrossCutting(region: Region): Boolean =+ region.crosscutScore > CrosscutThreshold// ── Cross-cutting detection ─────────────────────────────────────────private def detectCrosscut(graph: WeightedFileGraph): Map[NodeId, Double] = {if (graph.degrees.isEmpty) return Map.emptyval maxDeg = graph.degrees.values.max.max(1e-10)- val threshold = maxDeg * (1.0 - CrosscutDegreeTop)+ val threshold = highDegreeThreshold(graph)graph.vertices.map { v =>val deg = graph.degrees.getOrElse(v.id, 0.0)@@ -127,6 +168,16 @@ class MapService(}.toMap}+ private def highDegreeThreshold(graph: WeightedFileGraph): Double = {+ val sortedDegrees = graph.degrees.values.toVector.sorted+ if (sortedDegrees.isEmpty) 0.0+ else {+ val cutoffIndex = math.ceil(sortedDegrees.size * (1.0 - CrosscutDegreeTop)).toInt - 1+ val clamped = cutoffIndex.max(0).min(sortedDegrees.size - 1)+ sortedDegrees(clamped)+ }+ }+private def applyPenalty(graph: WeightedFileGraph,scores: Map[NodeId, Double]@@ -161,34 +212,46 @@ class MapService(// Compute common absolute-path prefix to strip for label inferenceval allPaths = graph.vertices.map(_.path)val commonPfxLen = commonPathPrefixLength(allPaths)+ val relativePaths = allPaths.map(path => relativizePath(path, commonPfxLen))+ val globalDirTokenCounts = directoryTokenCounts(relativePaths)+ val signalAdjacency = buildSignalAdjacency(graph)// Build regions for each level, finest first.// Apply minCommunitySize at every level — singleton/doubleton regions are noise.val minSize = if (maxLevel >= 3) 3 else 2- val allRegions = levels.zipWithIndex.flatMap { case (partition, levelIdx) =>+ val retainedRegions = levels.zipWithIndex.flatMap { case (partition, levelIdx) =>val level = levelIdx + 1- partition.communities.filter(_.size >= minSize).map { memberFiles =>- buildOneRegion(+ partition.communities.zipWithIndex.collect {+ case (memberFiles, communityIndex) if memberFiles.size >= minSize =>+ new RetainedRegion(+ level = level,+ communityIndex = communityIndex,+ region = buildOneRegion(memberFiles, level, labelKind(level, maxLevel),- graph, fileVertexById, crosscutScores, rev, commonPfxLen- )+ graph, fileVertexById, crosscutScores, signalAdjacency, rev, commonPfxLen, globalDirTokenCounts+ )+ )}}+ val retainedByLevel = retainedRegions.groupBy(_.level)+ .view+ .mapValues(_.map(retained => retained.communityIndex -> retained.region).toMap)+ .toMap// Wire parent/child relationships- // For each region at level N, find which level-(N+1) region contains its files- val byLevel: Map[Int, Vector[Region]] =- allRegions.groupBy(_.level)-- val withParents = allRegions.map { region =>- val parentOpt = byLevel.get(region.level + 1).flatMap { parents =>- // Parent must be strictly coarser: strictly larger member set (no equal sets)- parents- .filter(p => p.memberFiles.size > region.memberFiles.size &&- region.memberFiles.subsetOf(p.memberFiles))- .minByOption(_.memberFiles.size) // pick tightest enclosing region+ val withParents = retainedRegions.map { retained =>+ val parentOpt = levels.lift(retained.level).flatMap { coarseLevel =>+ val candidateCommunities = retained.region.memberFiles.iterator+ .flatMap(fileId => coarseLevel.assignment.get(fileId))+ .toSet+ if (candidateCommunities.size != 1) None+ else {+ retainedByLevel.get(retained.level + 1)+ .flatMap(_.get(candidateCommunities.head))+ .filter(_.memberFiles.size > retained.region.memberFiles.size)+ }}- region.copy(parentId = parentOpt.map(_.id))+ retained.region.copy(parentId = parentOpt.map(_.id))}// Fill childRegionIds@@ -222,10 +285,14 @@ class MapService(val otherSegs = dirSegs(otherPaths)val exclusive = (mySegs -- otherSegs).filterNot(LabelNoiseTokens.contains)- // Prefer shorter tokens (more likely to be a meaningful module name)+ // Prefer shorter tokens (more likely to be a meaningful module name).+ // Fallback: rank by size — largest keeps the clean label, smaller ones get "(2)", "(3)", etc.val suffix = exclusive.toVector.sortBy(s => (s.length, s)).headOption.map(s => s" / ${s.capitalize}")- .getOrElse(s" (${r.memberFiles.size}f)")+ .getOrElse {+ val rank = group.sortBy(-_.memberFiles.size).indexWhere(_.id == r.id) + 1+ if (rank <= 1) "" else s" ($rank)"+ }r.copy(label = r.label + suffix)case None => r}@@ -240,6 +307,13 @@ class MapService("ix-memory", "memory-layer", "ix-cli", "core-ingestion", "ix-plugin","projects", "claude")+ private val WeakLabelTokens = Set(+ "api", "apis", "service", "services", "controller", "controllers",+ "repository", "repositories", "handler", "handlers", "module", "modules",+ "common", "shared", "core", "base", "internal", "external", "model",+ "models", "entity", "entities", "dto", "dtos", "types", "errors",+ "interface", "interfaces"+ )private def buildOneRegion(memberFiles: Set[NodeId],@@ -248,8 +322,10 @@ class MapService(graph: WeightedFileGraph,fileVertexById: Map[NodeId, FileVertex],crosscutScores: Map[NodeId, Double],+ signalAdjacency: Map[NodeId, Vector[(NodeId, Map[String, Int])]],rev: Rev,- commonPfxLen: Int = 0+ commonPfxLen: Int = 0,+ globalDirTokenCounts: Map[String, Int] = Map.empty): Region = {// Deterministic ID: stable across re-runs with same file setval idKey = "map:region:" + memberFiles.map(_.value.toString).toList.sorted.mkString(",")@@ -259,7 +335,6 @@ class MapService(var internalWeight = 0.0var externalWeight = 0.0var boundaryNodes = Set.empty[NodeId]- var signalCounts = Map("calls" -> 0.0, "imports" -> 0.0, "type" -> 0.0)for (fileId <- memberFiles) {for ((neighbor, w) <- graph.adjMatrix.getOrElse(fileId, Map.empty)) {@@ -284,8 +359,9 @@ class MapService(val crosscutScore =memberFiles.toList.map(f => crosscutScores.getOrElse(f, 0.0)).sum / n- val label = inferLabel(memberFiles, fileVertexById, commonPfxLen)+ val label = inferLabel(memberFiles, fileVertexById, commonPfxLen, globalDirTokenCounts)val confidence = computeConfidence(cohesion, boundaryRatio, memberFiles.size, crosscutScore)+ val dominantSignals = computeDominantSignals(memberFiles, signalAdjacency)Region(id = nodeId,@@ -300,7 +376,7 @@ class MapService(boundaryRatio = boundaryRatio,confidence = confidence,crosscutScore = crosscutScore,- dominantSignals = List("calls", "imports", "type").sortBy(s => -signalCounts.getOrElse(s, 0.0)),+ dominantSignals = dominantSignals,interfaceNodeCount = boundaryNodes.size,mapRev = rev.value)@@ -321,51 +397,28 @@ class MapService(private def inferLabel(memberFiles: Set[NodeId],fileVertexById: Map[NodeId, FileVertex],- commonPfxLen: Int = 0+ commonPfxLen: Int = 0,+ globalDirTokenCounts: Map[String, Int] = Map.empty): String = {val rawPaths = memberFiles.flatMap(f => fileVertexById.get(f).map(_.path)).toVectorif (rawPaths.isEmpty) return "Unknown"- // Strip common absolute prefix (e.g. /home/user/project/) so labels are project-relative- val paths = rawPaths.map { p =>- val segs = p.split("[/\\\\]")- if (commonPfxLen > 0 && commonPfxLen < segs.length) segs.drop(commonPfxLen).mkString("/")- else p- }+ val paths = rawPaths.map(path => relativizePath(path, commonPfxLen))val noiseTokens = LabelNoiseTokens- // Strategy 1: Longest common directory suffix across all paths- val dirSegments = paths.map(p => p.split("[/\\\\]").dropRight(1).toVector)- val commonDir = longestCommonSuffix(dirSegments)- .filterNot(s => noiseTokens.contains(s.toLowerCase))- .headOption-- if (commonDir.isDefined) return commonDir.get.capitalize-- // Strategy 2: Most discriminative non-noise directory segment.- // Prefer deeper (more specific) segments by scoring: frequency × depth-weight.- // Split each path into (depth, segment) pairs so we can weight deeper dirs higher.- val allDirsWithDepth: Vector[(Int, String)] = paths.flatMap { p =>- val segs = p.split("[/\\\\]").dropRight(1)- segs.zipWithIndex.map { case (s, i) => (i, s) }- .filter { case (_, s) => !noiseTokens.contains(s.toLowerCase) && s.nonEmpty }- }- // Score: count * (1 + depth / maxDepth) — deeper segments score higher- val maxDepth = allDirsWithDepth.map(_._1).maxOption.getOrElse(1).toDouble.max(1.0)- val scored = allDirsWithDepth.groupBy(_._2).map { case (seg, entries) =>- seg -> entries.length * (1.0 + entries.map(_._1 / maxDepth).sum / entries.length)- }.toVector.sortBy(-_._2)-- scored.headOption.map(_._1.capitalize).getOrElse {- // Strategy 3: Most frequent meaningful filename token- val fileTokens = paths.flatMap { p =>- val name = p.split("[/\\\\]").lastOption.getOrElse("")- val noExt = name.replaceAll("\\.[^.]+$", "")- splitCamel(noExt).filter(t => t.length > 3).map(_.toLowerCase)- }- fileTokens.filterNot(noiseTokens.contains)- .groupBy(identity).maxByOption(_._2.size)- .map(_._1.capitalize).getOrElse("Region")+ val suffixLabel = longestCommonSuffix(paths.map(path => directorySegments(path).toVector))+ .iterator+ .map(_.toLowerCase)+ .find(token => token.nonEmpty && !noiseTokens.contains(token))++ suffixLabel match {+ case Some(token) if !WeakLabelTokens.contains(token) =>+ humanizeLabel(token)+ case _ =>+ bestDiscriminativeDirectoryToken(paths, globalDirTokenCounts)+ .orElse(bestFilenameToken(paths))+ .map(humanizeLabel)+ .getOrElse("Region")}}@@ -390,6 +443,145 @@ class MapService(private def splitCamel(s: String): Array[String] =s.split("(?<=[a-z])(?=[A-Z])|(?<=[A-Z])(?=[A-Z][a-z])|[_\\-.]")+ private def relativizePath(path: String, commonPfxLen: Int): String = {+ val segs = path.split("[/\\\\]")+ if (commonPfxLen > 0 && commonPfxLen < segs.length) segs.drop(commonPfxLen).mkString("/")+ else path+ }++ private def directorySegments(path: String): Vector[String] =+ path.split("[/\\\\]").dropRight(1).toVector++ private def directoryTokenCounts(paths: Vector[String]): Map[String, Int] =+ paths.flatMap(directorySegments)+ .map(_.toLowerCase)+ .filter(token => token.nonEmpty && !LabelNoiseTokens.contains(token))+ .groupBy(identity)+ .map { case (token, entries) => token -> entries.size }++ private def bestDiscriminativeDirectoryToken(+ paths: Vector[String],+ globalDirTokenCounts: Map[String, Int]+ ): Option[String] = {+ val clusterCounts = directoryTokenCounts(paths)+ clusterCounts.toVector+ .map { case (token, intraCount) =>+ val globalCount = globalDirTokenCounts.getOrElse(token, intraCount).max(intraCount)+ val score = intraCount.toDouble * (intraCount.toDouble / globalCount.toDouble)+ (token, score, intraCount, globalCount)+ }+ .sortBy { case (token, score, intraCount, globalCount) =>+ (-score, globalCount, -intraCount, token)+ }+ .headOption+ .map(_._1)+ }++ private def bestFilenameToken(paths: Vector[String]): Option[String] = {+ val fileTokens = paths.flatMap { p =>+ val name = p.split("[/\\\\]").lastOption.getOrElse("")+ val noExt = name.replaceAll("\\.[^.]+$", "")+ splitCamel(noExt)+ .map(_.toLowerCase)+ .filter(token => token.length > 3 && !LabelNoiseTokens.contains(token))+ }++ fileTokens.groupBy(identity)+ .toVector+ .sortBy { case (token, entries) => (-entries.size, token) }+ .headOption+ .map(_._1)+ }++ private def humanizeLabel(token: String): String =+ token.split("[-_\\s]+").filter(_.nonEmpty).map(_.capitalize).mkString(" ")++ private def normalizeWhitespace(input: String): String =+ input.trim.toLowerCase.replaceAll("\\s+", " ")++ private def normalizeLabelQuery(input: String): String =+ normalizeWhitespace(input.replaceAll("[/_-]+", " "))++ private def matchQuality(label: String, query: String): Option[Int] = {+ val trimmedLabel = label.trim+ val trimmedQuery = query.trim+ val whitespaceLabel = normalizeWhitespace(trimmedLabel)+ val whitespaceQuery = normalizeWhitespace(trimmedQuery)+ val normalizedLabel = normalizeLabelQuery(trimmedLabel)+ val normalizedQuery = normalizeLabelQuery(trimmedQuery)++ if (trimmedLabel.equalsIgnoreCase(trimmedQuery)) Some(0)+ else if (whitespaceLabel == whitespaceQuery) Some(1)+ else if (normalizedLabel == normalizedQuery) Some(2)+ else if (normalizedLabel.startsWith(normalizedQuery)) Some(3)+ else if (normalizedLabel.contains(normalizedQuery)) Some(4)+ else None+ }++ private def levelPreference(region: Region): Int =+ region.labelKind match {+ case "subsystem" => 0+ case "module" => 1+ case "system" => 2+ case _ => 3+ }++ private def sortBySignificance(regions: Seq[Region]): Vector[Region] =+ regions.toVector.sortBy(r => (-r.memberFiles.size, -r.confidence, r.label.toLowerCase, r.id.value.toString))++ private def buildSignalAdjacency(+ graph: WeightedFileGraph+ ): Map[NodeId, Vector[(NodeId, Map[String, Int])]] = {+ val adjacency = scala.collection.mutable.Map.empty[NodeId, Vector[(NodeId, Map[String, Int])]]+ for (((src, dst), predicateCounts) <- graph.predicatePairs) {+ adjacency.update(src, adjacency.getOrElse(src, Vector.empty) :+ (dst -> predicateCounts))+ adjacency.update(dst, adjacency.getOrElse(dst, Vector.empty) :+ (src -> predicateCounts))+ }+ adjacency.toMap+ }++ private def computeDominantSignals(+ memberFiles: Set[NodeId],+ signalAdjacency: Map[NodeId, Vector[(NodeId, Map[String, Int])]]+ ): List[String] = {+ val signalScores = scala.collection.mutable.Map(+ "calls" -> 0.0,+ "imports" -> 0.0,+ "type" -> 0.0,+ "path" -> 0.0+ ).withDefaultValue(0.0)++ for {+ src <- memberFiles.iterator+ (dst, predicateCounts) <- signalAdjacency.getOrElse(src, Vector.empty)+ if memberFiles.contains(dst)+ if src.value.compareTo(dst.value) < 0+ (predicate, count) <- predicateCounts+ if count > 0+ } {+ predicate match {+ case "CALLS" =>+ signalScores("calls") += SignalWeights("CALLS") * math.log1p(count.toDouble)+ case "IMPORTS" =>+ signalScores("imports") += SignalWeights("IMPORTS") * math.log1p(count.toDouble)+ case "EXTENDS" | "IMPLEMENTS" =>+ signalScores("type") += SignalWeights(predicate) * math.log1p(count.toDouble)+ case "PATH" =>+ signalScores("path") += SignalWeights("PATH") * count.toDouble+ case _ =>+ }+ }++ val ranked = signalScores.toVector+ .filter(_._2 > 0.0)+ .sortBy { case (name, score) => (-score, name) }+ .map(_._1)+ .take(3)+ .toList++ if (ranked.nonEmpty) ranked else List("path")+ }+// ── Confidence scoring ──────────────────────────────────────────────private def computeConfidence(@@ -398,72 +590,148 @@ class MapService(size: Int,crosscutScore: Double): Double = {- // Normalize boundary ratio: high ratio → high confidence- val brScore = math.tanh(boundaryRatio / 3.0) // saturates around ratio = 6+ val brScore = math.tanh(boundaryRatio / 2.5)val cohScore = cohesion.min(1.0)- val sizeBonus = math.tanh(size / 10.0) // more files → more signal- val crossPenalty = crosscutScore * 0.3+ val sizeBonus = math.tanh(size / 8.0)+ val crossPenalty = crosscutScore * 0.4((0.4 * brScore + 0.3 * cohScore + 0.3 * sizeBonus) - crossPenalty).max(0.0).min(1.0)}- // ── Persistence guardrail + logging ─────────────────────────────────+ // ── Persistence ─────────────────────────────────────────────────────- private def guardAndPersist(- regions: Vector[Region],- rev: Rev,- bypassGuard: Boolean = false- ): IO[PersistenceEstimate] = {- if (regions.isEmpty) return IO.pure(PersistenceEstimate(0, 0, 0, 0, 0))+ private def computeAndPersistMap(inputRev: Rev, cached: Option[ArchitectureMap]): IO[ArchitectureMap] =+ for {+ rawGraph <- builder.build()+ crosscutScores = detectCrosscut(rawGraph)+ graph = applyPenalty(rawGraph, crosscutScores)+ levels = LouvainClustering.cluster(graph, maxLevels = 3, minCommunitySize = 3)+ regions = buildRegions(graph, levels, crosscutScores, inputRev)+ fresh = ArchitectureMap(regions, rawGraph.vertices.size, inputRev.value)+ _ <- persistMap(fresh, cached)+ } yield fresh++ private def loadCachedMap(inputRev: Rev): IO[Option[ArchitectureMap]] =+ for {+ rows <- client.query(+ """FOR n IN nodes+ | FILTER n.kind == "region"+ | AND n.deleted_rev == null+ | AND n.attrs.map_generated == true+ | AND TO_NUMBER(n.attrs.map_rev) == @mapRev+ | SORT TO_NUMBER(n.attrs.level) DESC, n.name ASC+ | RETURN {+ | id: n.logical_id,+ | name: n.name,+ | attrs: n.attrs+ | }""".stripMargin,+ Map("mapRev" -> Long.box(inputRev.value).asInstanceOf[AnyRef])+ )+ decoded = rows.flatMap(decodePersistedRegion(_, inputRev)).toVector+ regions = rehydrateRelationships(decoded)+ fileCount <- if (regions.isEmpty) IO.pure(0) else builder.liveFileCount()+ } yield {+ if (regions.isEmpty) None+ else Some(ArchitectureMap(regions, fileCount, inputRev.value))+ }++ private def decodePersistedRegion(json: Json, inputRev: Rev): Option[Region] = {+ val c = json.hcursor+ val attrs = c.downField("attrs")for {- oldRegions <- queryApi.findNodesByKind(NodeKind.Region, limit = 5000)- pe = estimatePersistence(regions, oldRegions.size)- _ <- logger.debug(- s"Persistence estimate: ${pe.totalOps} ops " +- s"(${pe.regionNodes} regions, ${pe.fileEdges} file edges, " +- s"${pe.regionEdges} region edges, ${pe.deleteOps} deletes)" +- (if (bypassGuard) " [guard bypassed via --full]" else "")+ idStr <- c.get[String]("id").toOption+ label <- c.get[String]("name").toOption+ id <- parseNodeId(idStr)+ level <- attrs.get[Int]("level").toOption+ labelKind <- attrs.get[String]("label_kind").toOption+ } yield {+ val memberFiles = attrs.get[Vector[String]]("member_files").getOrElse(Vector.empty).flatMap(parseNodeId).toSet+ val childIds = attrs.get[Vector[String]]("child_region_ids").getOrElse(Vector.empty).flatMap(parseNodeId).toSet+ val parentId = attrs.get[Option[String]]("parent_id").getOrElse(None).flatMap(parseNodeId)++ Region(+ id = id,+ label = label,+ labelKind = labelKind,+ level = level,+ memberFiles = memberFiles,+ childRegionIds = childIds,+ parentId = parentId,+ cohesion = attrs.get[Double]("cohesion").getOrElse(0.0),+ externalCoupling = attrs.get[Double]("external_coupling").getOrElse(0.0),+ boundaryRatio = attrs.get[Double]("boundary_ratio").getOrElse(0.0),+ confidence = attrs.get[Double]("confidence").getOrElse(0.0),+ crosscutScore = attrs.get[Double]("crosscut_score").getOrElse(0.0),+ dominantSignals = attrs.get[List[String]]("dominant_signals").getOrElse(Nil),+ interfaceNodeCount = attrs.get[Int]("interface_node_count").getOrElse(0),+ mapRev = attrs.get[Long]("map_rev").getOrElse(inputRev.value))- _ <- if (!bypassGuard && pe.totalOps > MaxSafePatchOps)- IO.raiseError(new MapCapacityException(- outcome = MapOutcome.LocalMapTooLarge,- userMessage = "Local map output is too large to persist safely.",- next = "For full system mapping at this scale, use Ix Cloud."- ))- else if (bypassGuard && pe.totalOps > MaxSafePatchOps)- logger.info(- s"Persistence guard bypassed (--full): ${pe.totalOps} ops exceeds " +- s"limit of $MaxSafePatchOps, continuing anyway"- )- else IO.unit- _ <- submitMapPatch(oldRegions, regions, rev)- } yield pe+ }}- private def estimatePersistence(regions: Vector[Region], oldRegionCount: Int): PersistenceEstimate = {- val regionNodes = regions.size- val fileEdges = regions.filter(_.level == 1).map(_.memberFiles.size).sum- val regionEdges = regions.count(_.parentId.isDefined)- val deleteOps = oldRegionCount- val totalOps = deleteOps + regionNodes + fileEdges + regionEdges- PersistenceEstimate(regionNodes, fileEdges, regionEdges, deleteOps, totalOps)+ private def parseNodeId(id: String): Option[NodeId] =+ scala.util.Try(UUID.fromString(id)).toOption.map(NodeId(_))++ private def rehydrateRelationships(regions: Vector[Region]): Vector[Region] = {+ if (regions.isEmpty) return regions++ val parentByChild = regions.iterator+ .flatMap(region => region.childRegionIds.iterator.map(childId => childId -> region.id))+ .toMap++ val childrenByParent = regions+ .flatMap(region => region.parentId.map(parentId => parentId -> region.id))+ .groupBy(_._1)+ .view+ .mapValues(_.map(_._2).toSet)+ .toMap++ regions.map { region =>+ val parentId = region.parentId.orElse(parentByChild.get(region.id))+ val childIds =+ if (region.childRegionIds.nonEmpty) region.childRegionIds+ else childrenByParent.getOrElse(region.id, Set.empty)+ region.copy(parentId = parentId, childRegionIds = childIds)+ }}- private def logOutcome(- outcome: MapOutcome,- pf: MapPreflightResult,- pe: PersistenceEstimate- ): IO[Unit] =- logger.info(- s"Map completed: outcome=${outcome.label} " +- s"mode=${pf.mode.label} risk=${pf.risk.label} " +- s"files=${pf.cost.fileCount} regions=${pe.regionNodes} " +- s"patchOps=${pe.totalOps} preflightMs=${pf.durationMs}"- )+ private def persistMap(fresh: ArchitectureMap, cached: Option[ArchitectureMap]): IO[Unit] = {+ if (cached.exists(existing => sameRegions(existing.regions, fresh.regions))) IO.unit+ else {+ for {+ oldRegions <- queryApi.findNodesByKind(NodeKind.Region, limit = 5000)+ oldEdgeIds <- loadActiveRegionEdgeIds(oldRegions.map(_.id))+ _ <- submitMapPatch(oldRegions, oldEdgeIds, fresh.regions, Rev(fresh.mapRev))+ } yield ()+ }+ }++ private def sameRegions(left: Vector[Region], right: Vector[Region]): Boolean =+ left.sortBy(_.id.value.toString) == right.sortBy(_.id.value.toString)++ private def loadActiveRegionEdgeIds(regionIds: Vector[NodeId]): IO[Vector[EdgeId]] =+ if (regionIds.isEmpty) IO.pure(Vector.empty)+ else {+ val ids = regionIds.map(_.value.toString).toArray+ client.query(+ """FOR e IN edges+ | FILTER e.deleted_rev == null+ | AND e.predicate == "IN_REGION"+ | AND (e.src IN @regionIds OR e.dst IN @regionIds)+ | RETURN e._key""".stripMargin,+ Map("regionIds" -> ids.asInstanceOf[AnyRef])+ ).map(_.flatMap(_.as[String].toOption).flatMap(id => scala.util.Try(UUID.fromString(id)).toOption.map(EdgeId(_))).toVector)+ }- private def submitMapPatch(oldRegions: Vector[GraphNode], regions: Vector[Region], rev: Rev): IO[Unit] = {+ private def submitMapPatch(+ oldRegions: Vector[GraphNode],+ oldEdgeIds: Vector[EdgeId],+ regions: Vector[Region],+ rev: Rev+ ): IO[Unit] = {val deleteOps: Vector[PatchOp] = oldRegions.map(n => PatchOp.DeleteNode(n.id))+ val deleteEdgeOps: Vector[PatchOp] = oldEdgeIds.map(PatchOp.DeleteEdge)val nodeOps: Vector[PatchOp] = regions.map { r =>PatchOp.UpsertNode(@@ -471,11 +739,14 @@ class MapService(kind = NodeKind.Region,name = r.label,attrs = Map(+ "label" -> r.label.asJson,"map_generated" -> Json.True,"level" -> r.level.asJson,"label_kind" -> r.labelKind.asJson,"file_count" -> r.memberFiles.size.asJson,"member_files" -> r.memberFiles.map(_.value.toString).asJson,+ "parent_id" -> r.parentId.map(_.value.toString).asJson,+ "child_region_ids" -> r.childRegionIds.toVector.map(_.value.toString).sorted.asJson,"cohesion" -> r.cohesion.asJson,"external_coupling" -> r.externalCoupling.asJson,"boundary_ratio" -> r.boundaryRatio.asJson,@@ -522,7 +793,9 @@ class MapService(}}- val allOps = deleteOps ++ nodeOps ++ fileEdgeOps ++ regionEdgeOps+ val allOps = deleteEdgeOps ++ deleteOps ++ nodeOps ++ fileEdgeOps ++ regionEdgeOps+ if (allOps.isEmpty) return IO.unit+val patch = GraphPatch(patchId = PatchId(UUID.nameUUIDFromBytes(s"map:${rev.value}".getBytes("UTF-8"))),actor = "map-service",diff --git a/memory-layer/src/main/scala/ix/memory/map/MapTypes.scala b/memory-layer/src/main/scala/ix/memory/map/MapTypes.scalaindex 924e514f..f19e7101 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapTypes.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapTypes.scala@@ -21,12 +21,13 @@ final case class WeightedFileGraph(vertices: Vector[FileVertex],adjMatrix: Map[NodeId, Map[NodeId, Double]],degrees: Map[NodeId, Double],- totalWeight: Double+ totalWeight: Double,+ predicatePairs: Map[(NodeId, NodeId), Map[String, Int]])object WeightedFileGraph {val empty: WeightedFileGraph =- WeightedFileGraph(Vector.empty, Map.empty, Map.empty, 0.0)+ WeightedFileGraph(Vector.empty, Map.empty, Map.empty, 0.0, Map.empty)}/**@@ -134,3 +135,24 @@ final case class ArchitectureMap(outcome: MapOutcome = MapOutcome.FullLocalCompleted,persistenceEstimate: Option[PersistenceEstimate] = None)++final case class RegionMatch(+ region: Region,+ matchQuality: Int,+ normalizedLabel: String+)++final case class ScopedHealthSummary(+ wellDefined: Int,+ moderate: Int,+ fuzzy: Int,+ crossCutting: Int+)++final case class ScopedSubsystemView(+ target: Region,+ parent: Option[Region],+ children: Vector[Region],+ subtree: Vector[Region],+ summary: ScopedHealthSummary+)diff --git a/memory-layer/src/main/scala/ix/memory/model/Claim.scala b/memory-layer/src/main/scala/ix/memory/model/Claim.scalaindex 64571d6b..c3523dfa 100644--- a/memory-layer/src/main/scala/ix/memory/model/Claim.scala+++ b/memory-layer/src/main/scala/ix/memory/model/Claim.scala@@ -27,15 +27,18 @@ object ClaimStatus {}final case class Claim(- id: ClaimId,- entityId: NodeId,- statement: String,- value: Json,- confidence: Option[Double],- status: ClaimStatus,- provenance: Provenance,- createdRev: Rev,- deletedRev: Option[Rev]+ id: ClaimId,+ entityId: NodeId,+ statement: String,+ value: Json,+ confidence: Option[Double],+ status: ClaimStatus,+ provenance: Provenance,+ createdRev: Rev,+ deletedRev: Option[Rev],+ /** Identifies the inference pass that produced this claim (e.g. "smell_v1", "subsystem_v1").+ * Absent for observed facts extracted from source code. Missing legacy claims are treated as "legacy". */+ inferenceVersion: Option[String] = None)object Claim {diff --git a/memory-layer/src/main/scala/ix/memory/model/GraphPatch.scala b/memory-layer/src/main/scala/ix/memory/model/GraphPatch.scalaindex 05dceabc..7c0b7881 100644--- a/memory-layer/src/main/scala/ix/memory/model/GraphPatch.scala+++ b/memory-layer/src/main/scala/ix/memory/model/GraphPatch.scala@@ -12,7 +12,14 @@ object PatchOp {final case class UpsertEdge(id: EdgeId, src: NodeId, dst: NodeId, predicate: EdgePredicate, attrs: Map[String, Json]) extends PatchOpfinal case class DeleteNode(id: NodeId) extends PatchOpfinal case class DeleteEdge(id: EdgeId) extends PatchOp- final case class AssertClaim(entityId: NodeId, field: String, value: Json, confidence: Option[Double]) extends PatchOp+ final case class AssertClaim(+ entityId: NodeId,+ field: String,+ value: Json,+ confidence: Option[Double],+ /** Inference pass identifier. Absent for observed facts; required for inferred claims. */+ inferenceVersion: Option[String] = None+ ) extends PatchOpfinal case class RetractClaim(claimId: ClaimId) extends PatchOpimplicit val encoder: Encoder[PatchOp] = Encoder.instance {@@ -22,8 +29,8 @@ object PatchOp {Json.obj("type" -> "UpsertEdge".asJson, "id" -> id.asJson, "src" -> src.asJson, "dst" -> dst.asJson, "predicate" -> predicate.asJson, "attrs" -> attrs.asJson)case DeleteNode(id) => Json.obj("type" -> "DeleteNode".asJson, "id" -> id.asJson)case DeleteEdge(id) => Json.obj("type" -> "DeleteEdge".asJson, "id" -> id.asJson)- case AssertClaim(entityId, field, value, confidence) =>- Json.obj("type" -> "AssertClaim".asJson, "entityId" -> entityId.asJson, "field" -> field.asJson, "value" -> value.asJson, "confidence" -> confidence.asJson)+ case AssertClaim(entityId, field, value, confidence, inferenceVersion) =>+ Json.obj("type" -> "AssertClaim".asJson, "entityId" -> entityId.asJson, "field" -> field.asJson, "value" -> value.asJson, "confidence" -> confidence.asJson, "inferenceVersion" -> inferenceVersion.asJson)case RetractClaim(claimId) => Json.obj("type" -> "RetractClaim".asJson, "claimId" -> claimId.asJson)}@@ -48,11 +55,12 @@ object PatchOp {case "DeleteEdge" => c.downField("id").as[EdgeId].map(DeleteEdge)case "AssertClaim" =>for {- entityId <- c.downField("entityId").as[NodeId]- field <- c.downField("field").as[String]- value <- c.downField("value").as[Json]- confidence <- c.downField("confidence").as[Option[Double]]- } yield AssertClaim(entityId, field, value, confidence)+ entityId <- c.downField("entityId").as[NodeId]+ field <- c.downField("field").as[String]+ value <- c.downField("value").as[Json]+ confidence <- c.downField("confidence").as[Option[Double]]+ inferenceVersion <- c.downField("inferenceVersion").as[Option[String]]+ } yield AssertClaim(entityId, field, value, confidence, inferenceVersion)case "RetractClaim" => c.downField("claimId").as[ClaimId].map(RetractClaim)case other => Left(DecodingFailure(s"Unknown PatchOp type: $other", c.history))}diff --git a/memory-layer/src/main/scala/ix/memory/model/Node.scala b/memory-layer/src/main/scala/ix/memory/model/Node.scalaindex 164417b6..38ca41bc 100644--- a/memory-layer/src/main/scala/ix/memory/model/Node.scala+++ b/memory-layer/src/main/scala/ix/memory/model/Node.scala@@ -28,6 +28,7 @@ object NodeKind {case object Bug extends NodeKindcase object Goal extends NodeKindcase object Region extends NodeKind+ case object Chunk extends NodeKindprivate val nameMap: Map[String, NodeKind] = Map("module" -> Module,@@ -50,7 +51,8 @@ object NodeKind {"task" -> Task,"bug" -> Bug,"goal" -> Goal,- "region" -> Region+ "region" -> Region,+ "chunk" -> Chunk)implicit val encoder: Encoder[NodeKind] = Encoder[String].contramap {@@ -75,6 +77,7 @@ object NodeKind {case Bug => "bug"case Goal => "goal"case Region => "region"+ case Chunk => "chunk"}implicit val decoder: Decoder[NodeKind] = Decoder[String].emap { s =>diff --git a/memory-layer/src/main/scala/ix/memory/smell/SmellService.scala b/memory-layer/src/main/scala/ix/memory/smell/SmellService.scalanew file mode 100644index 00000000..7a2c3d92--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/smell/SmellService.scala@@ -0,0 +1,254 @@+package ix.memory.smell++import java.time.Instant+import java.util.UUID++import cats.effect.IO+import cats.syntax.traverse._+import io.circe.Json+import io.circe.syntax._++import ix.memory.db.{ArangoClient, GraphWriteApi}+import ix.memory.model._++// ---------------------------------------------------------------------------+// Data types+// ---------------------------------------------------------------------------++sealed trait SmellKind { def field: String }+object SmellKind {+ case object OrphanFile extends SmellKind { val field = "has_smell.orphan_file" }+ case object GodModule extends SmellKind { val field = "has_smell.god_module" }+ case object WeakComponent extends SmellKind { val field = "has_smell.weak_component_member" }+}++final case class SmellCandidate(+ fileId: String, // logical_id UUID+ fileName: String,+ smellKind: SmellKind,+ confidence: Double,+ signals: Map[String, Json] // evidence key → value+)++final case class SmellConfig(+ /** Max total edge connectivity for a file to be an orphan candidate. */+ orphanMaxConnections: Int = 0,+ /** Min chunk count to flag a god module. */+ godModuleChunkThreshold: Int = 20,+ /** Min fan-in or fan-out (IMPORTS edges) to flag a god module. */+ godModuleFanThreshold: Int = 15,+ /** Max unique file neighbors for a weak-component candidate. */+ weakComponentMaxNeighbors: Int = 1+)++final case class SmellReport(+ candidates: Vector[SmellCandidate],+ rev: Long,+ runAt: String+)++// ---------------------------------------------------------------------------+// Service+// ---------------------------------------------------------------------------++class SmellService(client: ArangoClient, writeApi: GraphWriteApi) {++ private val InferenceVersion = "smell_v1"+ private val Actor = "ix/smell"++ def run(config: SmellConfig = SmellConfig()): IO[SmellReport] =+ for {+ orphans <- detectOrphans(config)+ godMods <- detectGodModules(config)+ weakComps <- detectWeakComponents(config)+ candidates = orphans ++ godMods ++ weakComps+ rev <- commitSmells(candidates)+ } yield SmellReport(candidates, rev, Instant.now().toString)++ /** Retrieve existing smell claims from the graph without rerunning detection. */+ def listSmells(): IO[Vector[Json]] =+ client.query(+ """FOR c IN claims+ | FILTER STARTS_WITH(c.field, "has_smell.")+ | AND c.status == "active"+ | AND c.deleted_rev == null+ | SORT c.entity_id, c.field+ | RETURN {+ | entity_id: c.entity_id,+ | smell: c.field,+ | value: c.value,+ | inference_version: c.inference_version+ | }""".stripMargin,+ Map.empty+ ).map(_.toVector)++ // ── Detectors ────────────────────────────────────────────────────────────++ private def detectOrphans(cfg: SmellConfig): IO[Vector[SmellCandidate]] =+ client.query(+ """FOR f IN nodes+ | FILTER f.kind == "file" AND f.deleted_rev == null+ | LET imports_out = LENGTH(+ | FOR e IN edges FILTER e.src == f.logical_id AND e.predicate == "IMPORTS" AND e.deleted_rev == null RETURN 1)+ | LET imports_in = LENGTH(+ | FOR e IN edges FILTER e.dst == f.logical_id AND e.predicate == "IMPORTS" AND e.deleted_rev == null RETURN 1)+ | LET calls_out = LENGTH(+ | FOR e IN edges FILTER e.src == f.logical_id AND e.predicate == "CALLS" AND e.deleted_rev == null RETURN 1)+ | LET calls_in = LENGTH(+ | FOR e IN edges FILTER e.dst == f.logical_id AND e.predicate == "CALLS" AND e.deleted_rev == null RETURN 1)+ | LET connectivity = imports_out + imports_in + calls_out + calls_in+ | FILTER connectivity <= @threshold+ | RETURN {+ | logical_id: f.logical_id, name: f.name,+ | imports_out, imports_in, calls_out, calls_in, connectivity+ | }""".stripMargin,+ Map("threshold" -> Int.box(cfg.orphanMaxConnections).asInstanceOf[AnyRef])+ ).map(_.flatMap { json =>+ for {+ id <- json.hcursor.get[String]("logical_id").toOption+ name <- json.hcursor.get[String]("name").toOption+ connectivity <- json.hcursor.get[Int]("connectivity").toOption+ importsOut <- json.hcursor.get[Int]("imports_out").toOption+ importsIn <- json.hcursor.get[Int]("imports_in").toOption+ callsOut <- json.hcursor.get[Int]("calls_out").toOption+ callsIn <- json.hcursor.get[Int]("calls_in").toOption+ } yield SmellCandidate(+ fileId = id,+ fileName = name,+ smellKind = SmellKind.OrphanFile,+ confidence = if (connectivity == 0) 0.85 else 0.6,+ signals = Map(+ "connectivity" -> connectivity.asJson,+ "imports_out" -> importsOut.asJson,+ "imports_in" -> importsIn.asJson,+ "calls_out" -> callsOut.asJson,+ "calls_in" -> callsIn.asJson+ )+ )+ }.toVector)++ private def detectGodModules(cfg: SmellConfig): IO[Vector[SmellCandidate]] =+ client.query(+ """FOR f IN nodes+ | FILTER f.kind == "file" AND f.deleted_rev == null+ | LET chunks = LENGTH(+ | FOR e IN edges FILTER e.src == f.logical_id AND e.predicate == "CONTAINS_CHUNK" AND e.deleted_rev == null RETURN 1)+ | LET symbols = LENGTH(+ | FOR e IN edges FILTER e.src == f.logical_id AND e.predicate == "CONTAINS" AND e.deleted_rev == null RETURN 1)+ | LET fan_in = LENGTH(+ | FOR e IN edges FILTER e.dst == f.logical_id AND e.predicate == "IMPORTS" AND e.deleted_rev == null RETURN 1)+ | LET fan_out = LENGTH(+ | FOR e IN edges FILTER e.src == f.logical_id AND e.predicate == "IMPORTS" AND e.deleted_rev == null RETURN 1)+ | FILTER chunks >= @chunkThreshold OR fan_in >= @fanThreshold OR fan_out >= @fanThreshold+ | RETURN { logical_id: f.logical_id, name: f.name, chunks, symbols, fan_in, fan_out }""".stripMargin,+ Map(+ "chunkThreshold" -> Int.box(cfg.godModuleChunkThreshold).asInstanceOf[AnyRef],+ "fanThreshold" -> Int.box(cfg.godModuleFanThreshold).asInstanceOf[AnyRef]+ )+ ).map(_.flatMap { json =>+ for {+ id <- json.hcursor.get[String]("logical_id").toOption+ name <- json.hcursor.get[String]("name").toOption+ chunks <- json.hcursor.get[Int]("chunks").toOption+ symbols <- json.hcursor.get[Int]("symbols").toOption+ fanIn <- json.hcursor.get[Int]("fan_in").toOption+ fanOut <- json.hcursor.get[Int]("fan_out").toOption+ } yield {+ val triggeredSignals = Seq(+ Option.when(chunks >= cfg.godModuleChunkThreshold)(s"chunks=$chunks"),+ Option.when(fanIn >= cfg.godModuleFanThreshold)(s"fan_in=$fanIn"),+ Option.when(fanOut >= cfg.godModuleFanThreshold)(s"fan_out=$fanOut")+ ).flatten+ SmellCandidate(+ fileId = id,+ fileName = name,+ smellKind = SmellKind.GodModule,+ confidence = Math.min(0.5 + triggeredSignals.size * 0.15, 0.9),+ signals = Map(+ "chunks" -> chunks.asJson,+ "symbols" -> symbols.asJson,+ "fan_in" -> fanIn.asJson,+ "fan_out" -> fanOut.asJson,+ "triggers" -> triggeredSignals.asJson+ )+ )+ }+ }.toVector)++ private def detectWeakComponents(cfg: SmellConfig): IO[Vector[SmellCandidate]] =+ client.query(+ """FOR f IN nodes+ | FILTER f.kind == "file" AND f.deleted_rev == null+ | LET neighbor_ids = UNIQUE(APPEND(+ | (FOR e IN edges+ | FILTER e.src == f.logical_id+ | AND e.predicate IN ["IMPORTS","CALLS"]+ | AND e.deleted_rev == null+ | RETURN e.dst),+ | (FOR e IN edges+ | FILTER e.dst == f.logical_id+ | AND e.predicate IN ["IMPORTS","CALLS"]+ | AND e.deleted_rev == null+ | RETURN e.src)+ | ))+ | LET neighbor_count = LENGTH(neighbor_ids)+ | FILTER neighbor_count > 0 AND neighbor_count <= @maxNeighbors+ | RETURN { logical_id: f.logical_id, name: f.name, neighbor_count }""".stripMargin,+ Map("maxNeighbors" -> Int.box(cfg.weakComponentMaxNeighbors).asInstanceOf[AnyRef])+ ).map(_.flatMap { json =>+ for {+ id <- json.hcursor.get[String]("logical_id").toOption+ name <- json.hcursor.get[String]("name").toOption+ neighborCount <- json.hcursor.get[Int]("neighbor_count").toOption+ } yield SmellCandidate(+ fileId = id,+ fileName = name,+ smellKind = SmellKind.WeakComponent,+ confidence = 0.55,+ signals = Map("neighbor_count" -> neighborCount.asJson)+ )+ }.toVector)++ // ── Claim persistence ────────────────────────────────────────────────────++ private def commitSmells(candidates: Vector[SmellCandidate]): IO[Long] = {+ if (candidates.isEmpty) return client.query(+ "FOR rev IN latest_rev SORT rev.value DESC LIMIT 1 RETURN rev.value",+ Map.empty+ ).map(_.headOption.flatMap(_.as[Long].toOption).getOrElse(0L))++ val ops: Vector[PatchOp] = candidates.map { c =>+ PatchOp.AssertClaim(+ entityId = NodeId(UUID.fromString(c.fileId)),+ field = c.smellKind.field,+ value = Json.obj(+ "confidence" -> c.confidence.asJson,+ "signals" -> c.signals.asJson,+ "file" -> c.fileName.asJson+ ),+ confidence = Some(c.confidence),+ inferenceVersion = Some(InferenceVersion)+ )+ }++ val patchId = PatchId(UUID.randomUUID())+ val sourceHash = UUID.nameUUIDFromBytes(s"smell_run:${Instant.now()}".getBytes("UTF-8")).toString+ val patch = GraphPatch(+ patchId = patchId,+ actor = Actor,+ timestamp = Instant.now(),+ source = PatchSource(+ uri = "ix://smell-detector",+ sourceHash = Some(sourceHash),+ extractor = s"smell-detector/$InferenceVersion",+ sourceType = SourceType.Code+ ),+ baseRev = Rev(0L),+ ops = ops,+ replaces = Vector.empty,+ intent = Some(s"Smell detection: ${candidates.size} candidates")+ )++ writeApi.commitPatch(patch).map(_.newRev.value)+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/smell/SubsystemService.scala b/memory-layer/src/main/scala/ix/memory/smell/SubsystemService.scalanew file mode 100644index 00000000..5aff04db--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/smell/SubsystemService.scala@@ -0,0 +1,195 @@+package ix.memory.smell++import java.util.UUID++import cats.effect.IO+import io.circe.Json+import io.circe.syntax._++import ix.memory.db.{ArangoClient, GraphQueryApi}++class SubsystemService(client: ArangoClient, queryApi: GraphQueryApi) {++ private val InferenceVersion = "subsystem_v1"+ private val mapper = new com.fasterxml.jackson.databind.ObjectMapper()++ // ── Public API ──────────────────────────────────────────────────────++ def scoreSubsystems(): IO[Json] =+ for {+ rows <- queryRawData()+ scores = rows.map(computeScore)+ rev <- queryApi.getLatestRev+ _ <- retireOldScores()+ _ <- storeScores(scores, rev.value)+ } yield Json.obj("scores" -> scores.map(encodeScore).asJson)++ def listSubsystems(): IO[Json] =+ client.query(+ """FOR c IN claims+ | FILTER c.field == "subsystem_health" AND c.deleted_rev == null+ | RETURN {+ | region_id: c.entity_id,+ | name: c.value.name,+ | level: c.value.level,+ | label_kind: c.value.label_kind,+ | file_count: c.value.file_count,+ | health_score: c.value.health_score,+ | chunk_density: c.value.chunk_density,+ | smell_rate: c.value.smell_rate,+ | smell_files: c.value.smell_files,+ | total_chunks: c.value.total_chunks,+ | confidence: c.value.confidence,+ | inference_version: c.value.inference_version+ | }""".stripMargin,+ Map.empty+ ).map(scores => Json.obj("scores" -> scores.asJson))++ // ── Internal data model ─────────────────────────────────────────────++ private case class RawData(+ regionId: String,+ name: String,+ level: Int,+ labelKind: String,+ fileCount: Int,+ totalChunks: Int,+ smellFiles: Int,+ confidence: Double+ )++ private case class Score(+ regionId: String,+ name: String,+ level: Int,+ labelKind: String,+ fileCount: Int,+ healthScore: Double,+ chunkDensity: Double,+ smellRate: Double,+ smellFiles: Int,+ totalChunks: Int,+ confidence: Double+ )++ // ── Raw data query ──────────────────────────────────────────────────++ /**+ * Single AQL query that joins regions with smell claims and CONTAINS_CHUNK+ * edge counts. The MERGE + SUM approach avoids N+1 queries.+ */+ private def queryRawData(): IO[List[RawData]] =+ client.query(+ """LET smell_ids = (+ | FOR c IN claims+ | FILTER STARTS_WITH(c.field, "has_smell.") AND c.deleted_rev == null+ | RETURN DISTINCT c.entity_id+ |)+ |LET chunk_map = MERGE(+ | FOR e IN edges+ | FILTER e.predicate == "CONTAINS" AND e.deleted_rev == null+ | COLLECT src = e.src INTO g+ | RETURN { [src]: LENGTH(g) }+ |)+ |FOR n IN nodes+ | FILTER n.kind == "region" AND n.deleted_rev == null+ | LET mf = n.attrs.member_files != null ? n.attrs.member_files : []+ | FILTER LENGTH(mf) > 0+ | LET file_count = LENGTH(mf)+ | LET total_chunks = SUM(+ | FOR f IN mf RETURN HAS(chunk_map, f) ? chunk_map[f] : 0+ | )+ | LET smell_count = LENGTH(+ | FOR f IN mf FILTER f IN smell_ids RETURN 1+ | )+ | RETURN {+ | region_id: n.logical_id,+ | name: n.name,+ | level: n.attrs.level,+ | label_kind: n.attrs.label_kind,+ | file_count: file_count,+ | total_chunks: total_chunks,+ | smell_files: smell_count,+ | confidence: n.attrs.confidence != null ? n.attrs.confidence : 0.0+ | }""".stripMargin,+ Map.empty+ ).map(_.flatMap { json =>+ val c = json.hcursor+ for {+ regionId <- c.get[String]("region_id").toOption+ name <- c.get[String]("name").toOption+ level <- c.get[Int]("level").toOption+ labelKind <- c.get[String]("label_kind").toOption+ fileCount <- c.get[Int]("file_count").toOption+ totalChunks <- c.get[Int]("total_chunks").toOption+ smellFiles <- c.get[Int]("smell_files").toOption+ confidence <- c.get[Double]("confidence").toOption+ } yield RawData(regionId, name, level, labelKind, fileCount, totalChunks, smellFiles, confidence)+ })++ // ── Scoring formula ─────────────────────────────────────────────────++ /**+ * Health score formula (from spec):+ * 0.4 × (1 − smell_rate) + 0.3 × tanh(chunk_density / 5) + 0.3 × confidence+ */+ private def computeScore(raw: RawData): Score = {+ val n = raw.fileCount.toDouble.max(1)+ val chunkDensity = raw.totalChunks.toDouble / n+ val smellRate = raw.smellFiles.toDouble / n+ val healthScore = (0.4 * (1 - smellRate) ++ 0.3 * Math.tanh(chunkDensity / 5) ++ 0.3 * raw.confidence).max(0.0).min(1.0)+ Score(raw.regionId, raw.name, raw.level, raw.labelKind,+ raw.fileCount, healthScore, chunkDensity, smellRate,+ raw.smellFiles, raw.totalChunks, raw.confidence)+ }++ // ── Persistence ─────────────────────────────────────────────────────++ private def retireOldScores(): IO[Unit] =+ client.execute(+ """FOR c IN claims+ | FILTER c.field == "subsystem_health" AND c.deleted_rev == null+ | REMOVE c IN claims""".stripMargin,+ Map.empty+ )++ private def storeScores(scores: List[Score], rev: Long): IO[Unit] =+ if (scores.isEmpty) IO.unit+ else {+ val docs = scores.map { s =>+ val key = UUID.nameUUIDFromBytes(s"subsystem:${s.regionId}".getBytes("UTF-8")).toString+ val value = encodeScore(s)+ val doc = new java.util.HashMap[String, AnyRef]()+ doc.put("_key", key)+ doc.put("entity_id", s.regionId)+ doc.put("field", "subsystem_health")+ doc.put("value", mapper.readValue(value.noSpaces, classOf[AnyRef]))+ doc.put("confidence", java.lang.Double.valueOf(s.healthScore))+ doc.put("status", "active")+ doc.put("created_rev", Long.box(rev))+ doc.put("deleted_rev", null)+ doc: java.util.Map[String, AnyRef]+ }+ client.bulkInsert("claims", docs, "replace").void+ }++ // ── Encoding ────────────────────────────────────────────────────────++ private def encodeScore(s: Score): Json =+ Json.obj(+ "region_id" -> s.regionId.asJson,+ "name" -> s.name.asJson,+ "level" -> s.level.asJson,+ "label_kind" -> s.labelKind.asJson,+ "file_count" -> s.fileCount.asJson,+ "health_score" -> s.healthScore.asJson,+ "chunk_density" -> s.chunkDensity.asJson,+ "smell_rate" -> s.smellRate.asJson,+ "smell_files" -> s.smellFiles.asJson,+ "total_chunks" -> s.totalChunks.asJson,+ "confidence" -> s.confidence.asJson,+ "inference_version" -> InferenceVersion.asJson+ )+}diff --git a/memory-layer/src/main/scala/ix/memory/subsystem/SubsystemScoringService.scala b/memory-layer/src/main/scala/ix/memory/subsystem/SubsystemScoringService.scalanew file mode 100644index 00000000..c6013199--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/subsystem/SubsystemScoringService.scala@@ -0,0 +1,224 @@+package ix.memory.subsystem++import java.time.Instant+import java.util.UUID++import cats.effect.IO+import io.circe.Json+import io.circe.syntax._++import ix.memory.db.{ArangoClient, GraphWriteApi}+import ix.memory.map.MapService+import ix.memory.model._++// ---------------------------------------------------------------------------+// Data types+// ---------------------------------------------------------------------------++final case class SubsystemScore(+ regionId: String,+ regionName: String,+ level: Int,+ labelKind: String,+ fileCount: Int,+ totalChunks: Int,+ smellFiles: Int,+ confidence: Double,+ chunkDensity: Double,+ smellRate: Double,+ healthScore: Double+)++// ---------------------------------------------------------------------------+// Service+// ---------------------------------------------------------------------------++/**+ * Computes evidence-based health scores for each architectural region (subsystem).+ *+ * Score formula (range 0–1):+ * health = 0.4*(1 − smellRate) + 0.3*tanh(chunkDensity/5) + 0.3*confidence+ *+ * Signals:+ * chunkDensity — avg CONTAINS_CHUNK edges per member file (more = richer structure)+ * smellRate — fraction of member files that have an active smell claim (lower = healthier)+ * confidence — MapService cohesion/boundary score carried from the region node+ *+ * Results stored as `has_score.subsystem_health` claims on Region nodes,+ * with inferenceVersion = "subsystem_v1".+ */+class SubsystemScoringService(+ client: ArangoClient,+ writeApi: GraphWriteApi,+ mapService: MapService+) {++ private val InferenceVersion = "subsystem_v1"+ private val Actor = "ix/subsystem-scorer"+ private val ScoreField = "has_score.subsystem_health"++ /** Run scoring: query chunk/smell metrics, compute health, persist claims. */+ def score(): IO[Vector[SubsystemScore]] =+ for {+ archMap <- mapService.getOrBuildMap()+ rows <- queryRegionMetrics(archMap.regions.map(_.id.value.toString), archMap.mapRev)+ scores = rows.flatMap(computeScore)+ _ <- persistScores(scores)+ } yield scores++ /** Retrieve previously stored health-score claims joined with region metadata. */+ def listScores(): IO[Vector[Json]] =+ client.query(+ """FOR r IN nodes+ | FILTER r.kind == "region" AND r.deleted_rev == null+ | LET score_claim = FIRST(+ | FOR c IN claims+ | FILTER c.entity_id == r.logical_id+ | AND c.field == "has_score.subsystem_health"+ | AND c.status == "active"+ | AND c.deleted_rev == null+ | SORT c.created_rev DESC+ | LIMIT 1+ | RETURN c+ | )+ | FILTER score_claim != null+ | SORT score_claim.value.health_score DESC+ | RETURN {+ | region_id: r.logical_id,+ | name: r.name,+ | level: r.attrs.level,+ | label_kind: r.attrs.label_kind,+ | file_count: r.attrs.file_count,+ | health_score: score_claim.value.health_score,+ | chunk_density: score_claim.value.chunk_density,+ | smell_rate: score_claim.value.smell_rate,+ | smell_files: score_claim.value.smell_files,+ | total_chunks: score_claim.value.total_chunks,+ | confidence: score_claim.value.confidence,+ | inference_version: score_claim.inference_version+ | }""".stripMargin,+ Map.empty+ ).map(_.toVector)++ // ── Internals ─────────────────────────────────────────────────────────────++ /**+ * Single AQL pass: for each region node, count CONTAINS_CHUNK edges across+ * member files, and count distinct member files that have any smell claim.+ */+ private def queryRegionMetrics(regionIds: Vector[String], mapRev: Long): IO[Vector[Json]] =+ if (regionIds.isEmpty) IO.pure(Vector.empty)+ else+ client.query(+ """FOR r IN nodes+ | FILTER r.kind == "region" AND r.deleted_rev == null+ | AND r.logical_id IN @regionIds+ | AND TO_NUMBER(r.attrs.map_rev) == @mapRev+ | LET chunk_count = LENGTH(+ | FOR e IN edges+ | FILTER e.src IN r.attrs.member_files+ | AND e.predicate == "CONTAINS_CHUNK"+ | AND e.deleted_rev == null+ | RETURN 1+ | )+ | LET smell_files = LENGTH(UNIQUE(+ | FOR c IN claims+ | FILTER c.entity_id IN r.attrs.member_files+ | AND STARTS_WITH(c.field, "has_smell.")+ | AND c.status == "active"+ | AND c.deleted_rev == null+ | RETURN c.entity_id+ | ))+ | RETURN {+ | id: r.logical_id,+ | name: r.name,+ | level: r.attrs.level,+ | label_kind: r.attrs.label_kind,+ | file_count: r.attrs.file_count,+ | confidence: r.attrs.confidence,+ | chunk_count: chunk_count,+ | smell_files: smell_files+ | }""".stripMargin,+ Map(+ "regionIds" -> regionIds.toArray.asInstanceOf[AnyRef],+ "mapRev" -> Long.box(mapRev).asInstanceOf[AnyRef]+ )+ ).map(_.toVector)++ private def computeScore(json: Json): Option[SubsystemScore] = {+ val c = json.hcursor+ for {+ regionId <- c.get[String]("id").toOption+ regionName <- c.get[String]("name").toOption+ } yield {+ val level = c.get[Int]("level").getOrElse(1)+ val labelKind = c.get[String]("label_kind").getOrElse("module")+ val fileCount = c.get[Int]("file_count").getOrElse(1).max(1)+ val confidence = c.get[Double]("confidence").getOrElse(0.0)+ val totalChunks = c.get[Int]("chunk_count").getOrElse(0)+ val smellFiles = c.get[Int]("smell_files").getOrElse(0)++ val chunkDensity = totalChunks.toDouble / fileCount+ val smellRate = smellFiles.toDouble / fileCount+ val normDensity = Math.tanh(chunkDensity / 5.0)+ val healthScore = (0.4 * (1.0 - smellRate) + 0.3 * normDensity + 0.3 * confidence)+ .max(0.0).min(1.0)++ SubsystemScore(+ regionId = regionId,+ regionName = regionName,+ level = level,+ labelKind = labelKind,+ fileCount = fileCount,+ totalChunks = totalChunks,+ smellFiles = smellFiles,+ confidence = confidence,+ chunkDensity = chunkDensity,+ smellRate = smellRate,+ healthScore = healthScore+ )+ }+ }++ private def persistScores(scores: Vector[SubsystemScore]): IO[Long] = {+ if (scores.isEmpty) return IO.pure(0L)++ val ops: Vector[PatchOp] = scores.flatMap { s =>+ scala.util.Try(UUID.fromString(s.regionId)).toOption.map { uuid =>+ PatchOp.AssertClaim(+ entityId = NodeId(uuid),+ field = ScoreField,+ value = Json.obj(+ "health_score" -> s.healthScore.asJson,+ "chunk_density" -> s.chunkDensity.asJson,+ "smell_rate" -> s.smellRate.asJson,+ "smell_files" -> s.smellFiles.asJson,+ "total_chunks" -> s.totalChunks.asJson,+ "file_count" -> s.fileCount.asJson,+ "confidence" -> s.confidence.asJson+ ),+ confidence = Some(s.healthScore),+ inferenceVersion = Some(InferenceVersion)+ )+ }+ }++ val patch = GraphPatch(+ patchId = PatchId(UUID.randomUUID()),+ actor = Actor,+ timestamp = Instant.now(),+ source = PatchSource(+ uri = "ix://subsystem-scorer",+ sourceHash = None,+ extractor = s"subsystem-scorer/$InferenceVersion",+ sourceType = SourceType.Inferred+ ),+ baseRev = Rev(0L),+ ops = ops,+ replaces = Vector.empty,+ intent = Some(s"Subsystem health scoring: ${scores.size} regions")+ )++ writeApi.commitPatch(patch).map(_.newRev.value)+ }+}diff --git a/memory-layer/src/test/scala/ix/memory/api/RoutesSpec.scala b/memory-layer/src/test/scala/ix/memory/api/RoutesSpec.scalaindex dfc965ed..35ec4613 100644--- a/memory-layer/src/test/scala/ix/memory/api/RoutesSpec.scala+++ b/memory-layer/src/test/scala/ix/memory/api/RoutesSpec.scala@@ -20,6 +20,8 @@ import ix.memory.db._import ix.memory.ingestion._import ix.memory.map.MapServiceimport ix.memory.model._+import ix.memory.smell.SmellService+import ix.memory.subsystem.SubsystemScoringServiceclass RoutesSpec extends AsyncFlatSpec with AsyncIOSpec with Matchers with TestDbHelper {@@ -68,8 +70,22 @@ class RoutesSpec extends AsyncFlatSpec with AsyncIOSpec with Matchers with TestDval bulkWriteApi = new BulkWriteApi(client)val bulkIngestionService = new BulkIngestionService(parserRouter, bulkWriteApi, queryApi)val mapService = new MapService(client, queryApi, writeApi)+ val smellService = new SmellService(client, writeApi)+ val subsystemService = new SubsystemScoringService(client, writeApi, mapService)- Routes.all(contextService, ingestionService, bulkIngestionService, queryApi, writeApi, conflictService, client, mapService, bulkWriteApi).orNotFound+ Routes.all(+ contextService,+ ingestionService,+ bulkIngestionService,+ queryApi,+ writeApi,+ conflictService,+ client,+ mapService,+ bulkWriteApi,+ smellService,+ subsystemService+ ).orNotFound}"Routes" should "return 200 OK for health check" in {diff --git a/memory-layer/src/test/scala/ix/memory/ingestion/FileDiscoverySpec.scala b/memory-layer/src/test/scala/ix/memory/ingestion/FileDiscoverySpec.scalanew file mode 100644index 00000000..806e275c--- /dev/null+++ b/memory-layer/src/test/scala/ix/memory/ingestion/FileDiscoverySpec.scala@@ -0,0 +1,26 @@+package ix.memory.ingestion++import java.nio.file.Paths++import org.scalatest.flatspec.AnyFlatSpec+import org.scalatest.matchers.should.Matchers++class FileDiscoverySpec extends AnyFlatSpec with Matchers {++ "extensionsFor(None)" should "default to parser-supported file types" in {+ val extensions = FileDiscovery.extensionsFor(None)++ extensions should contain allOf (".py", ".ts", ".tsx", ".scala", ".sc", ".json", ".yaml", ".yml", ".toml", ".md")+ extensions should not contain ".txt"+ extensions should not contain ".ini"+ extensions should not contain ".sbt"+ }++ "matchesFilter" should "exclude unsupported generic-text files by default" in {+ val extensions = FileDiscovery.extensionsFor(None)++ FileDiscovery.matchesFilter(Paths.get("README.md"), extensions) shouldBe true+ FileDiscovery.matchesFilter(Paths.get("Dockerfile"), extensions) shouldBe false+ FileDiscovery.matchesFilter(Paths.get("notes.txt"), extensions) shouldBe false+ }+}diff --git a/memory-layer/src/test/scala/ix/memory/ingestion/GenericTextFallbackSpec.scala b/memory-layer/src/test/scala/ix/memory/ingestion/GenericTextFallbackSpec.scalanew file mode 100644index 00000000..14fca5c8--- /dev/null+++ b/memory-layer/src/test/scala/ix/memory/ingestion/GenericTextFallbackSpec.scala@@ -0,0 +1,17 @@+package ix.memory.ingestion++import org.scalatest.flatspec.AnyFlatSpec+import org.scalatest.matchers.should.Matchers++class GenericTextFallbackSpec extends AnyFlatSpec with Matchers {++ "GenericTextFallback" should "store compact metadata instead of raw content" in {+ val parsed = GenericTextFallback.parse("notes.txt", "first line\nsecond line\n")+ val entity = parsed.entities.head++ entity.attrs.keySet should contain allOf ("file_name", "line_count", "preview")+ entity.attrs.keySet should not contain "content"+ entity.attrs("preview").asString shouldBe Some("first line")+ entity.contentFingerprint should not be empty+ }+}diff --git a/memory-layer/src/test/scala/ix/memory/map/MapServiceScopeSpec.scala b/memory-layer/src/test/scala/ix/memory/map/MapServiceScopeSpec.scalanew file mode 100644index 00000000..9d88ee03--- /dev/null+++ b/memory-layer/src/test/scala/ix/memory/map/MapServiceScopeSpec.scala@@ -0,0 +1,84 @@+package ix.memory.map++import java.util.UUID++import org.scalatest.flatspec.AnyFlatSpec+import org.scalatest.matchers.should.Matchers++import ix.memory.model.NodeId++class MapServiceScopeSpec extends AnyFlatSpec with Matchers {++ private val service = new MapService(null, null, null)++ "resolveRegion" should "order ambiguous matches by match quality, level preference, and significance" in {+ val context = region("Context", "system", 3, files = 120)+ val apiSubsystem = region("Api", "subsystem", 2, files = 35, parent = Some(context))+ val apiModule = region("Api", "module", 1, files = 12, parent = Some(context))+ val apiClient = region("Api / Client", "system", 3, files = 90)++ val map = ArchitectureMap(+ regions = Vector(context, apiSubsystem, apiModule, apiClient),+ fileCount = 257,+ mapRev = 42L+ )++ val matches = service.resolveRegion(map, "api")++ matches.map(_.region.label) shouldBe Vector("Api", "Api", "Api / Client")+ matches.map(_.region.labelKind) shouldBe Vector("subsystem", "module", "system")+ }++ "scopedView" should "summarize subtree health while returning only direct children" in {+ val system = region("Cli", "system", 3, files = 140, confidence = 0.82)+ val subsystem = region("Cli / Client", "subsystem", 2, files = 105, confidence = 0.72, parent = Some(system))+ val moduleA = region("Cli / Client", "module", 1, files = 53, confidence = 0.91, parent = Some(subsystem))+ val moduleB = region("Queries", "module", 1, files = 10, confidence = 0.42, crosscut = 0.18, parent = Some(subsystem))+ val moduleC = region("Github", "module", 1, files = 22, confidence = 0.63, parent = Some(subsystem))++ val map = ArchitectureMap(+ regions = Vector(system, subsystem, moduleA, moduleB, moduleC),+ fileCount = 140,+ mapRev = 42L+ )++ val view = service.scopedView(map, subsystem)++ view.parent.map(_.label) shouldBe Some("Cli")+ view.children.map(_.label) shouldBe Vector("Cli / Client", "Github", "Queries")+ view.summary.wellDefined shouldBe 1+ view.summary.moderate shouldBe 2+ view.summary.fuzzy shouldBe 1+ view.summary.crossCutting shouldBe 1+ }++ private def region(+ label: String,+ labelKind: String,+ level: Int,+ files: Int,+ confidence: Double = 0.80,+ crosscut: Double = 0.0,+ parent: Option[Region] = None,+ ): Region = {+ val id = NodeId(UUID.randomUUID())+ val memberFiles = (1 to files).map(_ => NodeId(UUID.randomUUID())).toSet+ Region(+ id = id,+ label = label,+ labelKind = labelKind,+ level = level,+ memberFiles = memberFiles,+ childRegionIds = Set.empty,+ parentId = parent.map(_.id),+ cohesion = 0.5,+ externalCoupling = 0.2,+ boundaryRatio = 1.5,+ confidence = confidence,+ crosscutScore = crosscut,+ dominantSignals = List("calls", "imports"),+ interfaceNodeCount = 1,+ mapRev = 42L+ )+ }+}diff --git a/node_modules/.vite/vitest/results.json b/node_modules/.vite/vitest/results.jsonnew file mode 100644index 00000000..e075ed9c--- /dev/null+++ b/node_modules/.vite/vitest/results.json@@ -0,0 +1 @@+{"version":"1.6.1","results":[[":ix-cli/src/cli/__tests__/subsystem-explain.test.ts",{"duration":18,"failed":false}]]}\ No newline at end of filediff --git a/rebuild.sh b/rebuild.shnew file mode 100755index 00000000..00715c16--- /dev/null+++ b/rebuild.sh@@ -0,0 +1,21 @@+#!/usr/bin/env bash+set -euo pipefail++SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"+cd "$SCRIPT_DIR"++echo "==> Shutting down..."+./shutdown.sh++echo "==> Building backend (sbt assembly)..."+sbt assembly++echo "==> Building CLI (npm run build)..."+cd ix-cli+npm run build+cd ..++echo "==> Running setup..."+./setup.sh++echo "==> Done."diff --git a/skills/ix-api-contract-change/SKILL.md b/skills/ix-api-contract-change/SKILL.mdnew file mode 100644index 00000000..ecefb9c6--- /dev/null+++ b/skills/ix-api-contract-change/SKILL.md@@ -0,0 +1,56 @@+---+name: ix-api-contract-change+description: Use when adding or changing memory-layer routes or ix-cli client methods so API contracts stay aligned across Scala routes, models, and TypeScript IxClient calls.+---++# Ix API Contract Change++Use this when touching:+- `memory-layer/src/main/scala/ix/memory/api/*Routes.scala`+- `memory-layer/src/main/scala/ix/memory/model/*`+- `ix-cli/src/client/api.ts`++## Workflow++1. Map blast radius:+```bash+ix impact IxClient --format json+ix search Routes --kind class --format json --limit 30+```++2. Inspect changed route surface:+```bash+ix overview <RouteClassName> --format json+ix read <absolute/path/to/RouteFile.scala:1-220> --format json+```++3. Align client boundary:+```bash+ix overview IxClient --format json+ix read <absolute/path/to/ix-cli/src/client/api.ts:1-260> --format json+```++4. Check for shared request/response model impact:+```bash+ix search <ModelName> --kind class --format json --limit 20+ix impact <ModelName> --format json+```++5. Validate expected tests were considered:+```bash+ix search RoutesSpec --kind class --format json --include-tests --limit 20+```++## Contract Checklist++- Route path and HTTP verb match client usage.+- Request fields, optionality, and defaults are consistent.+- Response shape and error behavior are consistent.+- New route is wired into aggregate route registration.++## Guardrails++- Treat `IxClient` as a high-risk boundary.+- If you changed a route signature, update caller methods in `IxClient` in the same task.+- Do not leave route/client drift for a follow-up task unless explicitly planned.+diff --git a/skills/ix-api-contract-change/agents/openai.yaml b/skills/ix-api-contract-change/agents/openai.yamlnew file mode 100644index 00000000..4aa140a5--- /dev/null+++ b/skills/ix-api-contract-change/agents/openai.yaml@@ -0,0 +1,7 @@+interface:+ display_name: "IX API Contract Change"+ short_description: "Keep Scala routes and IxClient in lockstep"+ default_prompt: "Use $ix-api-contract-change to implement this endpoint/client change and verify contract alignment."++policy:+ allow_implicit_invocation: truediff --git a/skills/ix-arango-query-change/SKILL.md b/skills/ix-arango-query-change/SKILL.mdnew file mode 100644index 00000000..16103a44--- /dev/null+++ b/skills/ix-arango-query-change/SKILL.md@@ -0,0 +1,47 @@+---+name: ix-arango-query-change+description: Use when modifying Arango-backed query/write behavior so schema, AQL, transaction semantics, and DB-facing tests remain aligned.+---++# Ix Arango Query Change++Use this when touching:+- `memory-layer/src/main/scala/ix/memory/db/ArangoClient.scala`+- DB query/write APIs and AQL construction+- schema or transaction behavior++## Workflow++1. Inspect DB surface:+```bash+ix overview ArangoClient --format json+ix impact ArangoClient --format json+```++2. Read query/transaction hot paths:+```bash+ix read <absolute/path/to/ArangoClient.scala:1-320> --format json+```++3. Locate nearby DB behavior and specs:+```bash+ix search GraphQueryApiSpec --kind class --format json --include-tests --limit 20+ix search GraphWriteApiSpec --kind class --format json --include-tests --limit 20+ix search SearchAqlSpec --kind class --format json --include-tests --limit 20+ix search ArangoClientSpec --kind class --format json --include-tests --limit 20+```++## DB Checklist++- Query semantics match intended node/edge filters.+- Sort/limit/pagination behavior is explicit and stable.+- Transactions remain atomic for multi-step writes.+- Schema bootstrap/ensure behavior is still safe and idempotent.+- Error mapping preserves actionable failure details.++## Guardrails++- Do not change query semantics without corresponding test updates.+- Treat default limits and sort order changes as behavior changes, not refactors.+- Keep API-level callers in sync when return shapes or cardinality change.+diff --git a/skills/ix-arango-query-change/agents/openai.yaml b/skills/ix-arango-query-change/agents/openai.yamlnew file mode 100644index 00000000..6cd29fe2--- /dev/null+++ b/skills/ix-arango-query-change/agents/openai.yaml@@ -0,0 +1,7 @@+interface:+ display_name: "IX Arango Query Change"+ short_description: "Control AQL/schema/transaction behavior drift"+ default_prompt: "Use $ix-arango-query-change to update Arango query logic and keep DB semantics and tests aligned."++policy:+ allow_implicit_invocation: truediff --git a/skills/ix-ingestion-parser-change/SKILL.md b/skills/ix-ingestion-parser-change/SKILL.mdnew file mode 100644index 00000000..9344d320--- /dev/null+++ b/skills/ix-ingestion-parser-change/SKILL.md@@ -0,0 +1,58 @@+---+name: ix-ingestion-parser-change+description: Use when changing ingestion or parser behavior so parser routing, extracted entities/relationships, and graph patch generation stay coherent and testable.+---++# Ix Ingestion Parser Change++Use this when touching:+- `memory-layer/src/main/scala/ix/memory/ingestion/IngestionService.scala`+- `memory-layer/src/main/scala/ix/memory/ingestion/ParserRouter.scala`+- `memory-layer/src/main/scala/ix/memory/ingestion/parsers/*`+- `memory-layer/src/main/scala/ix/memory/ingestion/GraphPatchBuilder.scala`++## Workflow++1. Confirm impact:+```bash+ix impact IngestionService --format json+ix overview ParserRouter --format json+ix overview GraphPatchBuilder --format json+```++2. Locate parser family and routing points:+```bash+ix search Parser --kind class --format json --limit 30+ix read <absolute/path/to/ParserRouter.scala:1-260> --format json+```++3. Verify parse output contract:+```bash+ix overview ParseResult --format json+ix read <absolute/path/to/ParseResult.scala:1-220> --format json+```++4. Verify graph patch contract:+```bash+ix read <absolute/path/to/GraphPatchBuilder.scala:1-260> --format json+```++5. Validate parser and ingestion tests were considered:+```bash+ix search Spec --kind class --format json --include-tests --limit 60+```++## Change Checklist++- Parser selection logic and file-type routing stay deterministic.+- New extracted entity/relationship kinds are represented consistently.+- IDs/hashes remain stable where expected.+- Import-edge and named-edge resolution still converges on resolvable nodes.+- Graph patch operations still satisfy downstream commit assumptions.++## Guardrails++- Avoid parser-only changes that silently break patch building.+- If parser output shape changes, update parse result consumers in the same task.+- Record intended behavior shifts as an explicit decision via `ix decide`.+diff --git a/skills/ix-ingestion-parser-change/agents/openai.yaml b/skills/ix-ingestion-parser-change/agents/openai.yamlnew file mode 100644index 00000000..1228260b--- /dev/null+++ b/skills/ix-ingestion-parser-change/agents/openai.yaml@@ -0,0 +1,7 @@+interface:+ display_name: "IX Ingestion Parser Change"+ short_description: "Guard parser routing and patch generation"+ default_prompt: "Use $ix-ingestion-parser-change to make this ingestion/parser update without breaking graph patch behavior."++policy:+ allow_implicit_invocation: truediff --git a/skills/ix-map-pipeline-change/SKILL.md b/skills/ix-map-pipeline-change/SKILL.mdnew file mode 100644index 00000000..fa455cf8--- /dev/null+++ b/skills/ix-map-pipeline-change/SKILL.md@@ -0,0 +1,81 @@+---+name: ix-map-pipeline-change+description: Use when changing the architectural analysis pipeline — MapService, SmellService, SubsystemScoringService, or their routes/CLI commands — so the map → smells → subsystems chain stays coherent.+---++# Ix Map Pipeline Change++Use this when touching:+- `memory-layer/src/main/scala/ix/memory/map/MapService.scala`+- `memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scala`+- `memory-layer/src/main/scala/ix/memory/map/MapPreflight.scala`+- `memory-layer/src/main/scala/ix/memory/map/LouvainClustering.scala`+- `memory-layer/src/main/scala/ix/memory/smell/SmellService.scala`+- `memory-layer/src/main/scala/ix/memory/subsystem/SubsystemScoringService.scala`+- `ix-cli/src/cli/commands/map.ts`+- `ix-cli/src/cli/commands/smells.ts`+- `ix-cli/src/cli/commands/subsystems.ts`++## Pipeline Overview++```+ix map → MapService (Louvain clustering → region hierarchy stored as REGION nodes)+ix smells → SmellService (AQL graph queries → smell claims on File nodes)+ix subsystems → SubsystemScoringService (join regions + smell claims → health scores)+```++Each stage stores versioned claims. Re-running retires old claims before writing new ones.++## Workflow++1. Understand current pipeline state:+```bash+ix briefing --format json+ix overview MapService --format json+ix impact MapService --format json+```++2. Check smell and subsystem dependencies:+```bash+ix overview SmellService --format json+ix overview SubsystemScoringService --format json+ix callers MapService --format json+```++3. Inspect the map types contract (shared across all three stages):+```bash+ix read memory-layer/src/main/scala/ix/memory/map/MapTypes.scala+```++4. After changes, verify all three stages still chain:+```bash+ix map --format json | head -20+ix smells --format json | head -20+ix subsystems --format json | head -20+```++5. Check region/claim data in graph:+```bash+ix stats --format json+ix smells --list --format json+ix subsystems --list --format json+```++6. Record the decision:+```bash+ix decide "<change title>" --rationale "<why>" --affects MapService --format json+```++## Key Invariants++- `MapService` stores results as `REGION` nodes and `CONTAINS_FILE` / `CONTAINS_REGION` edges. SmellService and SubsystemScoringService query these — don't change the node/edge kinds without updating all three.+- Smell claims use field prefix `has_smell.*` and `inference_version=smell_v1`. Subsystem scores use `field=subsystem_health` and `inference_version=subsystem_v1`.+- Both services call `retireOld*()` before storing new claims — this is idempotent by design.+- `SubsystemScoringService(client: ArangoClient, writeApi: GraphWriteApi)` — uses `writeApi` not `queryApi`.+- `SmellService(client: ArangoClient, writeApi: GraphWriteApi)` — uses `writeApi` not `queryApi`.++## Guardrails++- Run `ix map` before `ix smells` or `ix subsystems` — they depend on region data being current.+- If you change region granularity in MapService, SubsystemScoringService scores will shift — document the expected delta as a decision.+- Do not change claim field names (`has_smell.*`, `subsystem_health`) without updating the CLI commands that parse them.diff --git a/skills/ix-new-command/SKILL.md b/skills/ix-new-command/SKILL.mdnew file mode 100644index 00000000..015cd31b--- /dev/null+++ b/skills/ix-new-command/SKILL.md@@ -0,0 +1,89 @@+---+name: ix-new-command+description: Use when adding a new ix CLI command end-to-end — Scala route, IxClient method, CLI command file, oss.ts registration, and doc updates must all stay in sync.+---++# Ix New Command++Use this when adding a new `ix <command>` from scratch or extending an existing command with a new subcommand.++## Files that must all change together++| Layer | File |+|-------|------|+| Backend route | `memory-layer/src/main/scala/ix/memory/api/<Name>Routes.scala` |+| Route wiring | `memory-layer/src/main/scala/ix/memory/api/Routes.scala` |+| Server wiring | `memory-layer/src/main/scala/ix/memory/Main.scala` |+| TypeScript client | `ix-cli/src/client/api.ts` |+| CLI command | `ix-cli/src/cli/commands/<name>.ts` |+| OSS registration | `ix-cli/src/cli/register/oss.ts` |+| Docs | `CLAUDE.md`, `AGENTS.md` |++## Workflow++1. Understand current command surface:+```bash+ix overview IxClient --format json+ix search Routes --kind class --format json --limit 30+ix impact Routes --format json+```++2. Read existing route and client as pattern references:+```bash+ix read memory-layer/src/main/scala/ix/memory/api/SmellRoutes.scala+ix read ix-cli/src/cli/commands/smells.ts+```++3. After writing the new route file, read constructor patterns to wire correctly:+```bash+ix search <ServiceName> --kind class --format json --limit 5+ix read memory-layer/src/main/scala/ix/memory/api/Routes.scala+ix read memory-layer/src/main/scala/ix/memory/Main.scala+```++4. After writing the CLI command, verify registration isn't duplicated:+```bash+ix read ix-cli/src/cli/register/oss.ts+```++5. Compile-check the backend:+```bash+sbt "memoryLayer/compile" 2>&1 | grep -E "error|warning|success"+```++6. Build the CLI:+```bash+cd ix-cli && npm run build 2>&1 | grep -E "error TS|error:"+```++7. Smoke-test the command:+```bash+ix <newcommand> --format json+```++8. Update docs and ingest:+```bash+# Edit CLAUDE.md and AGENTS.md to add the command to the appropriate table+ix ingest ./ix-cli/src ./memory-layer/src+```++9. Record the decision:+```bash+ix decide "Add ix <name> command" --rationale "<why this command exists>" --affects IxClient --format json+```++## Common Wiring Errors++- `Routes.scala`: wrong constructor args — always read the `class <Name>Routes(...)` signature before instantiating+- `Routes.scala`: referencing a class from the wrong package — check import path matches where the service lives+- `oss.ts`: registering the same command twice — grep for the register call before adding it+- `api.ts`: method returns `Record<string,string>` but caller expects `Map` — prefer `Map` and use `new Map(Object.entries(result))`+- `api.ts`: method signature accepts no args but CLI passes options — always add `opts?: {...}` if the command has flags++## Done Criteria++- `sbt "memoryLayer/compile"` passes with no errors.+- `cd ix-cli && npm run build` passes with no errors.+- `ix <newcommand> --format json` returns a valid JSON response.+- Command appears in `CLAUDE.md` and `AGENTS.md`.+- No duplicate registration in `oss.ts`.diff --git a/skills/ix-pro-command-extension/SKILL.md b/skills/ix-pro-command-extension/SKILL.mdnew file mode 100644index 00000000..23a9ce67--- /dev/null+++ b/skills/ix-pro-command-extension/SKILL.md@@ -0,0 +1,56 @@+---+name: ix-pro-command-extension+description: Use when adding or modifying pro CLI commands to keep registration, dynamic loading, patch-building semantics, and command safety coherent across ix-cli and Ix-pro.+---++# Ix Pro Command Extension++Use this when changing pro command implementation or wiring.++## Scope++- `Ix-pro/src/cli/register.ts`+- `Ix-pro/src/cli/commands/*`+- `IX-Memory/ix-cli/src/cli/register/pro-loader.ts`+- Any shared client patch payload behavior (`GraphPatchPayload`, `PatchOp`)++## Workflow++1. Confirm command wiring:+```bash+ix read /home/ianhock/ix/Ix-pro/src/cli/register.ts:1-260 --format json+ix read /home/ianhock/ix/IX-Memory/ix-cli/src/cli/register/pro-loader.ts:1-260 --format json+```++2. Inspect changed command module:+```bash+ix read /home/ianhock/ix/Ix-pro/src/cli/commands/<command>.ts:1-320 --format json+ix overview IxClient --format json+```++3. Validate patch intent and edge semantics:+```bash+ix search build<Command>Patch --kind function --format json --limit 20+ix impact IxClient --format json+```++4. Verify discoverability and operator UX:+```bash+ix search registerProCommands --kind function --format json --limit 10+ix search tryLoadProCommands --kind function --format json --limit 10+```++## Checklist++- New command is registered in `registerProCommands`.+- Loader compatibility with `@ix/pro/register` is preserved.+- `--format json` output remains stable for automation.+- Patch ops (`UpsertNode`, `UpsertEdge`, `AssertClaim`) reflect intended graph semantics.+- Validation blocks unsafe command input where applicable (example: workflow command safety checks).++## Guardrails++- Treat command output schemas as APIs.+- Do not add command behavior that bypasses entity resolution safeguards.+- Keep edge predicates explicit and consistent with existing pro model.+diff --git a/skills/ix-pro-command-extension/agents/openai.yaml b/skills/ix-pro-command-extension/agents/openai.yamlnew file mode 100644index 00000000..deb0c36b--- /dev/null+++ b/skills/ix-pro-command-extension/agents/openai.yaml@@ -0,0 +1,7 @@+interface:+ display_name: "IX Pro Command Extension"+ short_description: "Extend pro CLI safely across loader and commands"+ default_prompt: "Use $ix-pro-command-extension to add or refactor a pro CLI command without breaking registration or patch semantics."++policy:+ allow_implicit_invocation: truediff --git a/skills/ix-pro-delivery-loop/SKILL.md b/skills/ix-pro-delivery-loop/SKILL.mdnew file mode 100644index 00000000..89ad9651--- /dev/null+++ b/skills/ix-pro-delivery-loop/SKILL.md@@ -0,0 +1,63 @@+---+name: ix-pro-delivery-loop+description: Use when executing work with pro CLI objects (goals, plans, tasks, bugs, decisions, workflows) to keep delivery state explicit and machine-queryable.+---++# Ix Pro Delivery Loop++Use this when a task should be tracked through pro objects, not ad-hoc notes.++## Trigger++Use when at least one applies:+- New work should be sequenced into tasks.+- A bug should be linked to affected entities and a responding plan.+- A repeatable staged workflow should be attached and run.++## Core Loop++1. Resume and inspect:+```bash+ix briefing --format json+ix tasks --status pending --format json+ix bugs --format json+```++2. Create or select plan context:+```bash+ix goal create "<goal>" --format json+ix plan create "<plan>" --goal <goal-id> --format json+ix plan next <plan-id> --with-workflow --format json+```++3. Add executable tasks:+```bash+ix plan task "<title>" --plan <plan-id> --workflow-staged '{"discover":["ix overview <x> --format json"],"implement":["ix impact <x> --format json"],"validate":["ix tasks --plan <plan-id> --format json"]}' --format json+```++4. Track defects and linkage:+```bash+ix bug create "<title>" --severity high --affects <EntityName> --format json+ix plan create "<plan>" --goal <goal-id> --responds-to <bug-id> --format json+ix plan task "<title>" --plan <plan-id> --resolves <bug-id> --format json+```++5. Attach and run workflows:+```bash+ix workflow attach task <task-id> --file <workflow.json> --format json+ix workflow validate task <task-id> --format json+ix workflow run task <task-id> --stage discover --format json+```++6. Close loop:+```bash+ix task update <task-id> --status done --format json+ix bug update <bug-id> --status resolved --format json+```++## Guardrails++- Workflows must contain `ix` commands only; no shell operators.+- Keep dependency edges explicit with `--depends-on`.+- Prefer linking bugs/plans/tasks (`RESPONDS_TO`, `TASK_RESOLVES_BUG`) in the same session.+diff --git a/skills/ix-pro-delivery-loop/agents/openai.yaml b/skills/ix-pro-delivery-loop/agents/openai.yamlnew file mode 100644index 00000000..f32d98bd--- /dev/null+++ b/skills/ix-pro-delivery-loop/agents/openai.yaml@@ -0,0 +1,7 @@+interface:+ display_name: "IX Pro Delivery Loop"+ short_description: "Operate goals/plans/tasks/bugs/workflows together"+ default_prompt: "Use $ix-pro-delivery-loop to drive this work through plan/task/bug/workflow lifecycle with linked entities."++policy:+ allow_implicit_invocation: truediff --git a/skills/ix-rebuild/SKILL.md b/skills/ix-rebuild/SKILL.mdnew file mode 100644index 00000000..c9d237ac--- /dev/null+++ b/skills/ix-rebuild/SKILL.md@@ -0,0 +1,111 @@+---+name: ix-rebuild+description: Use when running or recovering from a full ix rebuild — sbt assembly + CLI build. Covers how to interpret compile errors and common failure patterns in this codebase.+---++# Ix Rebuild++Use this when running `./rebuild.sh` or fixing compile errors after merging changes.++## Build Order++```+sbt assembly (Scala backend) → npm run build (TypeScript CLI) → Docker restart+```++`./rebuild.sh` runs all three. For faster iteration use the individual steps below.++## Fast Iteration Commands++```bash+# Scala only (fastest feedback on backend changes)+sbt "memoryLayer/compile" 2>&1 | grep -E "error|warning|success"++# Full Scala assembly (needed before Docker restart)+sbt assembly 2>&1 | grep -E "error|warning|success|Built"++# TypeScript CLI only+cd ix-cli && npm run build 2>&1 | grep -E "error TS|error:"++# Both + Docker restart+./rebuild.sh+```++## Common Scala Errors and Fixes++### Wrong constructor args+```+not enough arguments for constructor Foo: (a: A, b: B)+```+→ Read the class signature: `ix search Foo --kind class --format json --limit 5`+→ Then: `ix read <path-to-file>` to see the actual constructor.++### Type not found+```+not found: type BulkPatchCommitRoutes+```+→ The class doesn't exist. Search for it: `ix locate BulkPatchCommitRoutes --limit 5`+→ Check if it was renamed or removed from the route aggregator.++### Type mismatch on service+```+found: ix.memory.smell.SubsystemService+required: ix.memory.subsystem.SubsystemScoringService+```+→ The stash/upstream used different packages. Check which class actually exists on disk:+ `find memory-layer/src -name "Subsystem*.scala"`+→ Update the import and usage to match what exists.++### Wrong queryApi vs writeApi+```+found: ArangoGraphQueryApi+required: GraphWriteApi+```+→ Check the service constructor: `grep "^class <Name>" memory-layer/src/main/scala/**/*.scala`+→ Services that store claims need `writeApi`; read-only services need `queryApi`.++## Common TypeScript Errors and Fixes++### Missing method on IxClient+```+Property 'commitPatchBulk' does not exist on type 'IxClient'+```+→ Add the method to `ix-cli/src/client/api.ts`.+→ Check what endpoint it should call by reading the corresponding Scala route.++### Missing export from module+```+Module '"../errors.js"' has no exported member 'parseBackendError'+```+→ Read `ix-cli/src/cli/errors.ts` to see what's actually exported.+→ Remove the import and inline the logic, or add the export if it belongs there.++### Wrong argument count+```+Expected 0 arguments, but got 1+```+→ The CLI passes options the api.ts method doesn't accept yet.+→ Add `opts?: { ... }` to the method signature in `api.ts`.++### Duplicate command registration+```+Error: cannot add command 'smells' as already have command 'smells'+```+→ Open `ix-cli/src/cli/register/oss.ts` and remove the duplicate `register<Name>Command(program)` call.++## Smoke Test After Rebuild++```bash+ix status # backend alive+ix stats --format json # graph accessible+ix map --format json | head -5 # map pipeline works+ix smells --format json | head -5 # smell pipeline works+ix subsystems --format json | head -5 # subsystem pipeline works+```++## Done Criteria++- `sbt "memoryLayer/compile"` exits with `[success]`.+- `cd ix-cli && npm run build` exits cleanly (no `error TS` lines).+- `./rebuild.sh` prints `IX backend is ready!`.+- `ix status` reports `ok`.diff --git a/skills/ix-session-memory/SKILL.md b/skills/ix-session-memory/SKILL.mdnew file mode 100644index 00000000..9da6c4f6--- /dev/null+++ b/skills/ix-session-memory/SKILL.md@@ -0,0 +1,69 @@+---+name: ix-session-memory+description: Use this for any non-trivial IX-Memory task to keep context synchronized in ix (briefing, impact, planning, decisions, and bugs) and to produce resumable work.+---++# Ix Session Memory++Use this skill at the start of multi-step work and keep using it until the task is done.++## Trigger++Use when the task has at least one of:+- More than one file or subsystem+- A risky change with dependency impact+- Work that should be resumable by another agent later++## Workflow++1. Resume context:+```bash+ix briefing --format json+```++2. Scope impact:+```bash+ix overview <target> --format json+ix impact <target> --format json+```++3. If no active goal/plan exists, create one:+```bash+ix goal create "<outcome statement>" --format json+ix plan create "<plan title>" --goal <goal-id> --format json+ix plan task "<first task>" --plan <plan-id> --format json+```++4. Pull next actionable work:+```bash+ix plan next <plan-id> --with-workflow --format json+```++5. Track major reasoning:+```bash+ix decide "<decision>" --rationale "<why>" --affects <EntityName> --format json+```++6. Track defects explicitly:+```bash+ix bug create "<bug title>" --affects <EntityName> --format json+```++7. Mark completed tasks:+```bash+ix task update <task-id> --status done --format json+```++## Guardrails++- Always use `--format json` for any command that is parsed or chained.+- Prefer `overview` and `impact` before low-level traversal.+- If resolution is ambiguous, rerun with `--kind`.+- Never follow unresolved `rawId`; use `ix text` or `ix read` instead.++## Done Criteria++- The change is represented in at least one plan/task update.+- Any architectural tradeoff is captured as a decision.+- Any discovered defect is captured as a bug or fixed in-task.+diff --git a/skills/ix-session-memory/agents/openai.yaml b/skills/ix-session-memory/agents/openai.yamlnew file mode 100644index 00000000..5f14d114--- /dev/null+++ b/skills/ix-session-memory/agents/openai.yaml@@ -0,0 +1,7 @@+interface:+ display_name: "IX Session Memory"+ short_description: "Keep ix state synced across multi-step work"+ default_prompt: "Use $ix-session-memory to resume context, scope impact, and track this task in goals/plans/decisions."++policy:+ allow_implicit_invocation: true