Understand any codebase instantly. System intelligence for codebases, built for humans and AI.
TypeScript aiai-memoryclaude-codeclicode-analysiscode-mappingcode-memorycodebase-understandingcodexdeveloper-toolsllmopenclawpersistent-memoryprogram-analysissoftware-architecturesystem-intelligencesystem-mapping
feat: add map preflight/fast-map system, docker volume cleanup, and Apache 2.0 license
Tanner Kocher57349ef391ed
diff --git a/LICENSE b/LICENSEnew file mode 100644index 00000000..cf329812--- /dev/null+++ b/LICENSE@@ -0,0 +1,191 @@++ Apache License+ Version 2.0, January 2004+ http://www.apache.org/licenses/++ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION++ 1. Definitions.++ "License" shall mean the terms and conditions for use, reproduction,+ and distribution as defined by Sections 1 through 9 of this document.++ "Licensor" shall mean the copyright owner or entity authorized by+ the copyright owner that is granting the License.++ "Legal Entity" shall mean the union of the acting entity and all+ other entities that control, are controlled by, or are under common+ control with that entity. For the purposes of this definition,+ "control" means (i) the power, direct or indirect, to cause the+ direction or management of such entity, whether by contract or+ otherwise, or (ii) ownership of fifty percent (50%) or more of the+ outstanding shares, or (iii) beneficial ownership of such entity.++ "You" (or "Your") shall mean an individual or Legal Entity+ exercising permissions granted by this License.++ "Source" form shall mean the preferred form for making modifications,+ including but not limited to software source code, documentation+ source, and configuration files.++ "Object" form shall mean any form resulting from mechanical+ transformation or translation of a Source form, including but+ not limited to compiled object code, generated documentation,+ and conversions to other media types.++ "Work" shall mean the work of authorship, whether in Source or+ Object form, made available under the License, as indicated by a+ copyright notice that is included in or attached to the work+ (an example is provided in the Appendix below).++ "Derivative Works" shall mean any work, whether in Source or Object+ form, that is based on (or derived from) the Work and for which the+ editorial revisions, annotations, elaborations, or other modifications+ represent, as a whole, an original work of authorship. For the purposes+ of this License, Derivative Works shall not include works that remain+ separable from, or merely link (or bind by name) to the interfaces of,+ the Work and Derivative Works thereof.++ "Contribution" shall mean any work of authorship, including+ the original version of the Work and any modifications or additions+ to that Work or Derivative Works thereof, that is intentionally+ submitted to the Licensor for inclusion in the Work by the copyright owner+ or by an individual or Legal Entity authorized to submit on behalf of+ the copyright owner. For the purposes of this definition, "submitted"+ means any form of electronic, verbal, or written communication sent+ to the Licensor or its representatives, including but not limited to+ communication on electronic mailing lists, source code control systems,+ and issue tracking systems that are managed by, or on behalf of, the+ Licensor for the purpose of discussing and improving the Work, but+ excluding communication that is conspicuously marked or otherwise+ designated in writing by the copyright owner as "Not a Contribution."++ "Contributor" shall mean Licensor and any individual or Legal Entity+ on behalf of whom a Contribution has been received by the Licensor and+ subsequently incorporated within the Work.++ 2. Grant of Copyright License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ copyright license to reproduce, prepare Derivative Works of,+ publicly display, publicly perform, sublicense, and distribute the+ Work and such Derivative Works in Source or Object form.++ 3. Grant of Patent License. Subject to the terms and conditions of+ this License, each Contributor hereby grants to You a perpetual,+ worldwide, non-exclusive, no-charge, royalty-free, irrevocable+ (except as stated in this section) patent license to make, have made,+ use, offer to sell, sell, import, and otherwise transfer the Work,+ where such license applies only to those patent claims licensable+ by such Contributor that are necessarily infringed by their+ Contribution(s) alone or by combination of their Contribution(s)+ with the Work to which such Contribution(s) was submitted. If You+ institute patent litigation against any entity (including a+ cross-claim or counterclaim in a lawsuit) alleging that the Work+ or a Contribution incorporated within the Work constitutes direct+ or contributory patent infringement, then any patent licenses+ granted to You under this License for that Work shall terminate+ as of the date such litigation is filed.++ 4. Redistribution. You may reproduce and distribute copies of the+ Work or Derivative Works thereof in any medium, with or without+ modifications, and in Source or Object form, provided that You+ meet the following conditions:++ (a) You must give any other recipients of the Work or+ Derivative Works a copy of this License; and++ (b) You must cause any modified files to carry prominent notices+ stating that You changed the files; and++ (c) You must retain, in the Source form of any Derivative Works+ that You distribute, all copyright, patent, trademark, and+ attribution notices from the Source form of the Work,+ excluding those notices that do not pertain to any part of+ the Derivative Works; and++ (d) If the Work includes a "NOTICE" text file as part of its+ distribution, then any Derivative Works that You distribute must+ include a readable copy of the attribution notices contained+ within such NOTICE file, excluding any notices that do not+ pertain to any part of the Derivative Works, in at least one+ of the following places: within a NOTICE text file distributed+ as part of the Derivative Works; within the Source form or+ documentation, if provided along with the Derivative Works; or,+ within a display generated by the Derivative Works, if and+ wherever such third-party notices normally appear. The contents+ of the NOTICE file are for informational purposes only and+ do not modify the License. You may add Your own attribution+ notices within Derivative Works that You distribute, alongside+ or as an addendum to the NOTICE text from the Work, provided+ that such additional attribution notices cannot be construed+ as modifying the License.++ You may add Your own copyright statement to Your modifications and+ may provide additional or different license terms and conditions+ for use, reproduction, or distribution of Your modifications, or+ for any such Derivative Works as a whole, provided Your use,+ reproduction, and distribution of the Work otherwise complies with+ the conditions stated in this License.++ 5. Submission of Contributions. Unless You explicitly state otherwise,+ any Contribution intentionally submitted for inclusion in the Work+ by You to the Licensor shall be under the terms and conditions of+ this License, without any additional terms or conditions.+ Notwithstanding the above, nothing herein shall supersede or modify+ the terms of any separate license agreement you may have executed+ with Licensor regarding such Contributions.++ 6. Trademarks. This License does not grant permission to use the trade+ names, trademarks, service marks, or product names of the Licensor,+ except as required for reasonable and customary use in describing the+ origin of the Work and reproducing the content of the NOTICE file.++ 7. Disclaimer of Warranty. Unless required by applicable law or+ agreed to in writing, Licensor provides the Work (and each+ Contributor provides its Contributions) on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or+ implied, including, without limitation, any warranties or conditions+ of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A+ PARTICULAR PURPOSE. You are solely responsible for determining the+ appropriateness of using or redistributing the Work and assume any+ risks associated with Your exercise of permissions under this License.++ 8. Limitation of Liability. In no event and under no legal theory,+ whether in tort (including negligence), contract, or otherwise,+ unless required by applicable law (such as deliberate and grossly+ negligent acts) or agreed to in writing, shall any Contributor be+ liable to You for damages, including any direct, indirect, special,+ incidental, or consequential damages of any character arising as a+ result of this License or out of the use or inability to use the+ Work (including but not limited to damages for loss of goodwill,+ work stoppage, computer failure or malfunction, or any and all+ other commercial damages or losses), even if such Contributor+ has been advised of the possibility of such damages.++ 9. Accepting Warranty or Additional Liability. While redistributing+ the Work or Derivative Works thereof, You may choose to offer,+ and charge a fee for, acceptance of support, warranty, indemnity,+ or other liability obligations and/or rights consistent with this+ License. However, in accepting such obligations, You may act only+ on Your own behalf and on Your sole responsibility, not on behalf+ of any other Contributor, and only if You agree to indemnify,+ defend, and hold each Contributor harmless for any liability+ incurred by, or claims asserted against, such Contributor by reason+ of your accepting any such warranty or additional liability.++ END OF TERMS AND CONDITIONS++ Copyright 2026 Ix Infrastructure++ Licensed under the Apache License, Version 2.0 (the "License");+ you may not use this file except in compliance with the License.+ You may obtain a copy of the License at++ http://www.apache.org/licenses/LICENSE-2.0++ Unless required by applicable law or agreed to in writing, software+ distributed under the License is distributed on an "AS IS" BASIS,+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.+ See the License for the specific language governing permissions and+ limitations under the License.diff --git a/NOTICE b/NOTICEnew file mode 100644index 00000000..28884be5--- /dev/null+++ b/NOTICE@@ -0,0 +1,5 @@+Ix++Copyright 2026 Ix Infrastructure++This product includes software developed by Ix Infrastructure.diff --git a/README.md b/README.mdindex 796d83b6..5eb25062 100644--- a/README.md+++ b/README.md@@ -146,4 +146,4 @@ ix CLI (TypeScript) → Memory Layer (Scala/http4s) → ArangoDB## License-Proprietary — IX Infrastructure.+This project is licensed under the [Apache License 2.0](LICENSE).diff --git a/ix-cli/src/cli/commands/docker.ts b/ix-cli/src/cli/commands/docker.tsindex f6e9e5e4..06a3c615 100644--- a/ix-cli/src/cli/commands/docker.ts+++ b/ix-cli/src/cli/commands/docker.ts@@ -1,5 +1,6 @@import { Command } from "commander";import { execSync, spawn } from "child_process";+import { createInterface } from "readline";import { existsSync } from "fs";import { join } from "path";import { homedir } from "os";@@ -23,6 +24,55 @@ function findComposeFile(): string | null {return null;}+/**+ * Find all Ix-managed ArangoDB Docker volumes across repos.+ *+ * Matches volumes created by Docker Compose where:+ * - The compose project name starts with "ix" (label: com.docker.compose.project)+ * - The compose volume name contains "arango" (label: com.docker.compose.volume)+ *+ * This avoids matching unrelated ArangoDB volumes (e.g. enterprise projects).+ */+function findIxArangoVolumes(): string[] {+ try {+ const output = execSync(+ 'docker volume ls --format "{{.Name}}|{{.Labels}}"',+ { encoding: "utf-8", timeout: 10000 }+ ).trim();+ if (!output) return [];++ return output.split("\n").filter((line) => {+ const [name, labels] = line.split("|", 2);+ if (!name || !labels) return false;++ // Parse labels: "key1=val1,key2=val2,..."+ const labelMap = new Map<string, string>();+ for (const pair of labels.split(",")) {+ const eq = pair.indexOf("=");+ if (eq > 0) labelMap.set(pair.slice(0, eq), pair.slice(eq + 1));+ }++ const project = labelMap.get("com.docker.compose.project") ?? "";+ const volume = labelMap.get("com.docker.compose.volume") ?? "";++ // Match: Ix project (starts with "ix") + ArangoDB volume (contains "arango")+ return project.startsWith("ix") && volume.includes("arango");+ }).map((line) => line.split("|", 1)[0]);+ } catch {+ return [];+ }+}++function askConfirmation(prompt: string): Promise<boolean> {+ const rl = createInterface({ input: process.stdin, output: process.stderr });+ return new Promise((resolve) => {+ rl.question(prompt, (answer) => {+ rl.close();+ resolve(answer.toLowerCase() === "y");+ });+ });+}+function isHealthy(): boolean {try {execSync(`curl -sf ${HEALTH_URL}`, { stdio: "ignore", timeout: 5000 });@@ -121,8 +171,10 @@ export function registerDockerCommand(program: Command): void {.command("stop").alias("down").description("Stop the IX backend containers")- .option("--remove-data", "Also remove the ArangoDB data volume")- .action((opts) => {+ .option("--remove-data", "Also remove the current project's ArangoDB data volume")+ .option("--remove-all-data", "Remove all local Ix ArangoDB data volumes across repos")+ .option("--yes", "Skip confirmation prompt (for use with --remove-all-data)")+ .action(async (opts) => {const composeFile = findComposeFile();if (!composeFile) {console.error("[error] No docker-compose.yml found.");@@ -132,23 +184,75 @@ export function registerDockerCommand(program: Command): void {process.exit(1);}- const flags = opts.removeData ? "down -v" : "down";+ // --remove-all-data is a superset of --remove-data+ const removeLocal = opts.removeData || opts.removeAllData;+ const flags = removeLocal ? "down -v" : "down";+try {execSync(`docker compose -f "${composeFile}" ${flags}`, {stdio: "inherit",});- if (opts.removeData) {- console.log("[ok] Backend stopped and data volumes removed.");- } else {- console.log("[ok] Backend stopped. Data volume preserved.");- console.log(- " Use 'ix docker stop --remove-data' to also delete data."- );- }} catch {console.error("[error] Failed to stop containers.");process.exit(1);}++ if (opts.removeAllData) {+ // Global cleanup: find and remove all Ix ArangoDB volumes+ const volumes = findIxArangoVolumes();+ if (volumes.length === 0) {+ console.log("[ok] Backend stopped. No additional Ix data volumes found.");+ return;+ }++ // Confirm unless --yes+ if (!opts.yes) {+ console.log("");+ console.log("This will remove all local Ix ArangoDB data volumes across repos:");+ for (const v of volumes) console.log(` ${v}`);+ console.log("");+ const confirmed = await askConfirmation("Continue? [y/N] ");+ if (!confirmed) {+ console.log("Aborted.");+ return;+ }+ }++ const removed: string[] = [];+ const failed: string[] = [];+ for (const v of volumes) {+ try {+ execSync(`docker volume rm "${v}"`, { stdio: "ignore", timeout: 10000 });+ removed.push(v);+ } catch {+ failed.push(v);+ }+ }++ if (failed.length > 0) {+ console.error("");+ console.error("[error] Failed to remove one or more Ix data volumes.");+ for (const v of failed) console.error(` ${v}`);+ console.error("");+ console.error(" Volumes may be in use. Stop all Ix containers first.");+ process.exitCode = 1;+ }++ if (removed.length > 0) {+ console.log("");+ console.log("[ok] Backend stopped and all local Ix data volumes removed.");+ console.log("");+ console.log("Removed:");+ for (const v of removed) console.log(` ${v}`);+ }+ } else if (opts.removeData) {+ console.log("[ok] Backend stopped and data volume removed.");+ } else {+ console.log("[ok] Backend stopped. Data volume preserved.");+ console.log(+ " Use 'ix docker stop --remove-data' to also delete data."+ );+ }});dockerdiff --git a/ix-cli/src/cli/commands/map.ts b/ix-cli/src/cli/commands/map.tsindex 8f7daae2..eb96ed27 100644--- a/ix-cli/src/cli/commands/map.ts+++ b/ix-cli/src/cli/commands/map.ts@@ -5,6 +5,7 @@ import { IxClient } from "../../client/api.js";import { getEndpoint } from "../config.js";import { bootstrap } from "../bootstrap.js";import { ingestFiles } from "./ingest.js";+import { parseBackendError, renderStructuredError } from "../errors.js";interface MapRegion {id: string;@@ -24,6 +25,35 @@ interface MapRegion {children?: MapRegion[];}+interface MapPreflight {+ cost: {+ file_count: number;+ directory_count: number;+ directory_quadratic: number;+ symbol_estimate: number;+ edge_estimate: number;+ };+ capacity: {+ cpu_cores: number;+ heap_max_bytes: number;+ heap_free_bytes: number;+ container_memory: number | null;+ disk_free_bytes: number | null;+ };+ risk: string;+ mode: string;+ warnings: string[];+ duration_ms: number;+}++interface MapPersistence {+ region_nodes: number;+ file_edges: number;+ region_edges: number;+ delete_ops: number;+ total_ops: number;+}+interface MapResult {file_count: number;region_count: number;@@ -31,6 +61,9 @@ interface MapResult {map_rev: number;regions: MapRegion[];hierarchy: MapRegion[];+ outcome?: string;+ preflight?: MapPreflight;+ persistence?: MapPersistence;}export function registerMapCommand(program: Command): void {@@ -85,7 +118,13 @@ Examples:try {result = await client.map() as MapResult;} catch (err: any) {- console.error(chalk.red("Error:"), err.message);+ const msg: string = err.message ?? "";+ const structured = parseBackendError(msg);+ if (structured) {+ renderStructuredError(structured);+ } else {+ console.error(chalk.red("Error:"), msg);+ }process.exitCode = 1;return;}@@ -109,6 +148,14 @@ Examples:chalk.dim(`${result.region_count} regions · ${result.levels} levels`));+ if (result.outcome === "fast_local_completed") {+ console.log(+ chalk.yellow(" Large system detected") ++ chalk.dim(" — using Fast Map")+ );+ console.log(chalk.dim(" Reduced coupling model with full region hierarchy output."));+ }+if (regions.length === 0) {console.log(chalk.dim("\n No regions found matching filters."));return;diff --git a/ix-cli/src/cli/errors.ts b/ix-cli/src/cli/errors.tsnew file mode 100644index 00000000..50698358--- /dev/null+++ b/ix-cli/src/cli/errors.ts@@ -0,0 +1,50 @@+import chalk from "chalk";++/**+ * Structured error with user-facing message and optional next-step guidance.+ * Parsed from backend JSON responses that include `error`, `message`, and `next` fields.+ */+export interface StructuredError {+ error: string;+ message: string;+ next?: string;+}++/**+ * Attempt to parse a structured error from a backend HTTP error message.+ * Backend errors arrive as "${status}: ${jsonBody}" from the API client.+ */+export function parseBackendError(errMessage: string): StructuredError | null {+ // Match "NNN: {json}" pattern from api.ts error throwing+ const match = errMessage.match(/^(\d{3}):\s*(.+)$/s);+ if (!match) return null;++ try {+ const body = JSON.parse(match[2]);+ if (body.error && body.message) {+ return {+ error: body.error,+ message: body.message,+ next: body.next ?? undefined,+ };+ }+ } catch {+ // Not valid JSON — fall through+ }+ return null;+}++/**+ * Render a structured error to stderr with clean formatting.+ * No stack traces, no internal jargon.+ */+export function renderStructuredError(err: StructuredError): void {+ console.error("");+ console.error(chalk.red(` ${err.message}`));+ if (err.next) {+ console.error("");+ console.error(chalk.dim(" Next"));+ console.error(` ${err.next}`);+ }+ console.error("");+}diff --git a/memory-layer/src/main/scala/ix/memory/api/ErrorHandler.scala b/memory-layer/src/main/scala/ix/memory/api/ErrorHandler.scalaindex e6d34556..7510b9ee 100644--- a/memory-layer/src/main/scala/ix/memory/api/ErrorHandler.scala+++ b/memory-layer/src/main/scala/ix/memory/api/ErrorHandler.scala@@ -7,12 +7,17 @@ import org.http4s.{Response, Status}import org.http4s.circe._import org.http4s.dsl.io._+import ix.memory.map.MapCapacityException+/*** Maps domain exceptions to appropriate HTTP responses with JSON error bodies.*/object ErrorHandler {def handle(err: Throwable): IO[Response[IO]] = err match {+ case e: MapCapacityException =>+ UnprocessableEntity(mapCapacityJson(e))+case e: IllegalArgumentException =>BadRequest(errorJson("bad_request", e.getMessage))@@ -34,4 +39,11 @@ object ErrorHandler {"error" -> code.asJson,"message" -> message.asJson)++ private def mapCapacityJson(e: MapCapacityException): Json =+ Json.obj(+ "error" -> e.outcome.label.asJson,+ "message" -> e.userMessage.asJson,+ "next" -> e.next.asJson+ )}diff --git a/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scala b/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scalaindex e4b53ed4..3f4ab1db 100644--- a/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scala+++ b/memory-layer/src/main/scala/ix/memory/api/MapRoutes.scala@@ -7,7 +7,7 @@ import org.http4s.HttpRoutesimport org.http4s.circe.CirceEntityCodec._import org.http4s.dsl.io._-import ix.memory.map.{ArchitectureMap, MapService, Region}+import ix.memory.map.{ArchitectureMap, MapPreflightResult, MapService, PersistenceEstimate, Region}import ix.memory.model.NodeIdclass MapRoutes(mapService: MapService) {@@ -25,7 +25,7 @@ class MapRoutes(mapService: MapService) {private def encodeMap(m: ArchitectureMap): Json = {val regionsByLevel = m.regions.groupBy(_.level).toVector.sortBy(_._1)- Json.obj(+ val base = Json.obj("file_count" -> m.fileCount.asJson,"region_count" -> m.regions.size.asJson,"levels" -> regionsByLevel.size.asJson,@@ -33,8 +33,51 @@ class MapRoutes(mapService: MapService) {"regions" -> m.regions.sortBy(r => (r.level, r.label)).map(encodeRegion).asJson,"hierarchy" -> buildHierarchyTree(m).asJson)++ val withOutcome = base.deepMerge(Json.obj("outcome" -> m.outcome.label.asJson))++ val withPreflight = m.preflight match {+ case Some(pf) => withOutcome.deepMerge(Json.obj("preflight" -> encodePreflight(pf)))+ case None => withOutcome+ }++ m.persistenceEstimate match {+ case Some(pe) => withPreflight.deepMerge(Json.obj("persistence" -> encodePersistence(pe)))+ case None => withPreflight+ }}+ private def encodePreflight(pf: MapPreflightResult): Json =+ Json.obj(+ "cost" -> Json.obj(+ "file_count" -> pf.cost.fileCount.asJson,+ "directory_count" -> pf.cost.directoryCount.asJson,+ "directory_quadratic" -> pf.cost.directoryQuadratic.asJson,+ "symbol_estimate" -> pf.cost.symbolEstimate.asJson,+ "edge_estimate" -> pf.cost.edgeEstimate.asJson+ ),+ "capacity" -> Json.obj(+ "cpu_cores" -> pf.capacity.cpuCores.asJson,+ "heap_max_bytes" -> pf.capacity.heapMaxBytes.asJson,+ "heap_free_bytes" -> pf.capacity.heapFreeBytes.asJson,+ "container_memory" -> pf.capacity.containerMemory.asJson,+ "disk_free_bytes" -> pf.capacity.diskFreeBytes.asJson+ ),+ "risk" -> pf.risk.label.asJson,+ "mode" -> pf.mode.label.asJson,+ "warnings" -> pf.warnings.asJson,+ "duration_ms" -> pf.durationMs.asJson+ )++ private def encodePersistence(pe: PersistenceEstimate): Json =+ Json.obj(+ "region_nodes" -> pe.regionNodes.asJson,+ "file_edges" -> pe.fileEdges.asJson,+ "region_edges" -> pe.regionEdges.asJson,+ "delete_ops" -> pe.deleteOps.asJson,+ "total_ops" -> pe.totalOps.asJson+ )+private def encodeRegion(r: Region): Json =Json.obj("id" -> r.id.value.toString.asJson,diff --git a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scalaindex 9a716c4a..691c4ad3 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilder.scala@@ -27,13 +27,20 @@ class MapGraphBuilder(client: ArangoClient) {private val Gamma = 0.9private val Epsilon = 0.3- def build(): IO[WeightedFileGraph] =+ def discoverFiles(): IO[Vector[FileVertex]] = fetchFiles()++ def buildGraph(files: Vector[FileVertex]): IO[WeightedFileGraph] =for {- files <- fetchFiles()rawPairs <- fetchCouplingByUri()graph = computeGraph(files, rawPairs)} yield graph+ def build(): IO[WeightedFileGraph] =+ for {+ files <- discoverFiles()+ graph <- buildGraph(files)+ } yield graph+// ── ArangoDB queries ───────────────────────────────────────────────/**diff --git a/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scala b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scalanew file mode 100644index 00000000..acd21d41--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/map/MapGraphBuilderFast.scala@@ -0,0 +1,158 @@+package ix.memory.map++import cats.effect.IO+import org.typelevel.log4cats.slf4j.Slf4jLogger++import ix.memory.db.ArangoClient+import ix.memory.model.NodeId++/**+ * Fast graph builder for large workloads.+ *+ * Builds a WeightedFileGraph using only cheap file-level signals:+ * 1. File-to-file IMPORTS edges (direct query, no uri_map or symbol traversal)+ * 2. Directory chain structure (O(n) per directory, not O(n²) pairwise)+ *+ * The resulting graph is fully compatible with the downstream pipeline+ * (cross-cut detection, penalty, Louvain clustering, region creation, persistence).+ */+class MapGraphBuilderFast(client: ArangoClient) {++ private val logger = Slf4jLogger.getLoggerFromName[IO]("ix.map.fast")++ // Weights aligned with full mode for comparable downstream behavior+ private val ImportWeight = 0.7 // matches full mode β+ private val DirectoryWeight = 0.3 // matches full mode ε++ def buildGraph(files: Vector[FileVertex]): IO[WeightedFileGraph] =+ for {+ _ <- logger.info("Using fast map graph construction (reduced coupling model)")+ fileImports <- fetchFileImports()+ graph = computeGraph(files, fileImports)+ _ <- logger.debug(+ s"Fast graph: ${files.size} files, " ++ s"${fileImports.size} file-level import edges, " ++ s"adj entries=${graph.adjMatrix.size}"+ )+ } yield graph++ // ── Cheap AQL query ────────────────────────────────────────────────++ /**+ * Fetches file-to-file IMPORTS edges directly.+ *+ * This is cheap because:+ * - Filters to IMPORTS only (no CALLS/EXTENDS/IMPLEMENTS)+ * - Joins against file nodes only (indexed logical_id lookup)+ * - Does NOT build a uri_map or scan all symbols+ * - Result set is small (file-to-file edges are a fraction of all edges)+ */+ private def fetchFileImports(): IO[Vector[RawFilePair]] =+ client.query(+ """FOR e IN edges+ | FILTER e.predicate == "IMPORTS"+ | AND e.deleted_rev == null+ | LET srcFile = FIRST(+ | FOR n IN nodes+ | FILTER n.logical_id == e.src+ | AND n.kind == "file"+ | AND n.deleted_rev == null+ | AND n.provenance.source_uri != null+ | RETURN n.provenance.source_uri+ | )+ | LET dstFile = FIRST(+ | FOR n IN nodes+ | FILTER n.logical_id == e.dst+ | AND n.kind == "file"+ | AND n.deleted_rev == null+ | AND n.provenance.source_uri != null+ | RETURN n.provenance.source_uri+ | )+ | FILTER srcFile != null AND dstFile != null AND srcFile != dstFile+ | COLLECT srcUri = srcFile, dstUri = dstFile+ | WITH COUNT INTO cnt+ | RETURN {srcUri, dstUri, predicate: "IMPORTS", count: cnt}""".stripMargin,+ Map.empty+ ).map { rows =>+ rows.flatMap { json =>+ val c = json.hcursor+ for {+ srcUri <- c.get[String]("srcUri").toOption+ dstUri <- c.get[String]("dstUri").toOption+ pred <- c.get[String]("predicate").toOption+ cnt <- c.get[Int]("count").toOption+ } yield RawFilePair(srcUri, dstUri, pred, cnt)+ }.toVector+ }++ // ── Graph construction ─────────────────────────────────────────────++ private def computeGraph(+ files: Vector[FileVertex],+ fileImports: Vector[RawFilePair]+ ): WeightedFileGraph = {+ if (files.isEmpty) return WeightedFileGraph.empty++ val byUri: Map[String, FileVertex] = files.map(v => v.path -> v).toMap++ val adjMut =+ scala.collection.mutable.Map[NodeId, scala.collection.mutable.Map[NodeId, Double]]()++ def addEdge(a: NodeId, b: NodeId, w: Double): Unit = {+ adjMut.getOrElseUpdate(a, scala.collection.mutable.Map())(b) =+ adjMut.getOrElseUpdate(a, scala.collection.mutable.Map()).getOrElse(b, 0.0) + w+ adjMut.getOrElseUpdate(b, scala.collection.mutable.Map())(a) =+ adjMut.getOrElseUpdate(b, scala.collection.mutable.Map()).getOrElse(a, 0.0) + w+ }++ // Signal 1: File-level import edges+ for (pair <- fileImports) {+ val sv = byUri.get(pair.srcId)+ val dv = byUri.get(pair.dstId)+ (sv, dv) match {+ case (Some(s), Some(d)) if s.id != d.id =>+ addEdge(s.id, d.id, ImportWeight * math.log1p(pair.count))+ case _ =>+ }+ }++ // Signal 2: Directory chain (O(n) per directory, not O(n²) pairwise)+ // For each directory, sort files by path and connect consecutive pairs.+ // This creates a linear chain that gives Louvain enough signal to group+ // co-located files without quadratic blowup.+ val filesByDir = files.groupBy { v =>+ val parts = v.path.split("[/\\\\]")+ if (parts.length > 1) parts.dropRight(1).mkString("/") else "."+ }+ for ((_, dirFiles) <- filesByDir if dirFiles.size >= 2) {+ val sorted = dirFiles.sortBy(_.path)+ for (i <- 0 until sorted.size - 1) {+ val a = sorted(i)+ val b = sorted(i + 1)+ // Boost weight for files already connected by imports+ val existing = adjMut.get(a.id).flatMap(_.get(b.id)).getOrElse(0.0)+ if (existing > 0.0) {+ // Directory co-location boost on existing edge+ addEdge(a.id, b.id, DirectoryWeight)+ } else {+ // New structural edge for unconnected neighbors+ addEdge(a.id, b.id, DirectoryWeight * pathProximity(a.path, b.path))+ }+ }+ }++ val adj = adjMut.map { case (k, m) => k -> m.toMap }.toMap+ val degrees = adj.map { case (k, m) => k -> m.values.sum }+ val totalWeight = degrees.values.sum / 2.0++ WeightedFileGraph(files, adj, degrees.toMap, totalWeight)+ }++ private def pathProximity(a: String, b: String): Double = {+ val partsA = a.split("[/\\\\]").dropRight(1)+ val partsB = b.split("[/\\\\]").dropRight(1)+ val common = partsA.zip(partsB).takeWhile { case (x, y) => x == y }.length+ val dist = (partsA.length - common) + (partsB.length - common)+ 1.0 / (1.0 + dist)+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/map/MapPreflight.scala b/memory-layer/src/main/scala/ix/memory/map/MapPreflight.scalanew file mode 100644index 00000000..4872ffe1--- /dev/null+++ b/memory-layer/src/main/scala/ix/memory/map/MapPreflight.scala@@ -0,0 +1,144 @@+package ix.memory.map++import cats.effect.IO+import org.typelevel.log4cats.slf4j.Slf4jLogger++/**+ * Preflight check between cheap file discovery and the expensive graph stages.+ *+ * Estimates workload cost, probes local capacity, classifies risk, and+ * selects an execution mode — all before any AQL coupling queries run.+ */+class MapPreflight {++ private val logger = Slf4jLogger.getLoggerFromName[IO]("ix.map.preflight")++ def evaluate(files: Vector[FileVertex]): IO[MapPreflightResult] =+ for {+ start <- IO(System.currentTimeMillis())+ cost = estimateCost(files)+ capacity <- probeCapacity()+ risk = classifyRisk(cost, capacity)+ mode = selectMode(risk, capacity)+ warnings = buildWarnings(cost, capacity, risk, mode)+ end <- IO(System.currentTimeMillis())+ result = MapPreflightResult(cost, capacity, risk, mode, warnings, end - start)+ _ <- logger.debug(+ s"Preflight: F=${cost.fileCount} D=${cost.directoryCount} " ++ s"D2=${cost.directoryQuadratic} risk=${risk.label} mode=${mode.label} " ++ s"duration=${result.durationMs}ms"+ )+ } yield result++ private def estimateCost(files: Vector[FileVertex]): CostEstimate = {+ val fileCount = files.size+ val byDir = files.groupBy { v =>+ val parts = v.path.split("[/\\\\]")+ if (parts.length > 1) parts.dropRight(1).mkString("/") else "."+ }+ val directoryCount = byDir.size+ val directoryQuadratic = byDir.values.map { dirFiles =>+ val n = dirFiles.size.toLong+ n * n+ }.sum++ CostEstimate(+ fileCount = fileCount,+ directoryCount = directoryCount,+ directoryQuadratic = directoryQuadratic,+ symbolEstimate = fileCount.toLong * 12,+ edgeEstimate = fileCount.toLong * 8+ )+ }++ private def probeCapacity(): IO[LocalCapacity] = IO {+ val rt = Runtime.getRuntime+ val cpuCores = rt.availableProcessors()+ val heapMaxBytes = rt.maxMemory()+ val heapFreeBytes = rt.freeMemory() + (heapMaxBytes - rt.totalMemory())++ // Try cgroup v2 first, then v1, for container memory limit+ val containerMemory = readCgroupMemory()++ LocalCapacity(+ cpuCores = cpuCores,+ heapMaxBytes = heapMaxBytes,+ heapFreeBytes = heapFreeBytes,+ containerMemory = containerMemory,+ diskFreeBytes = None+ )+ }++ private def readCgroupMemory(): Option[Long] = {+ // cgroup v2+ val v2 = tryReadLong("/sys/fs/cgroup/memory.max")+ if (v2.isDefined) return v2++ // cgroup v1+ tryReadLong("/sys/fs/cgroup/memory/memory.limit_in_bytes")+ }++ private def tryReadLong(path: String): Option[Long] =+ try {+ val content = scala.io.Source.fromFile(path).mkString.trim+ if (content == "max") None else Some(content.toLong)+ } catch {+ case _: Exception => None+ }++ private val LowThreshold = 500+ private val MediumThreshold = 2000+ private val HighThreshold = 5000+ private val LowMemoryBytes = 512L * 1024 * 1024 // 512 MB++ private def classifyRisk(cost: CostEstimate, capacity: LocalCapacity): RiskTier = {+ val lowMem = capacity.containerMemory.exists(_ < LowMemoryBytes)+ val f = cost.fileCount++ // Lower thresholds when running in a constrained container+ val (medT, highT, extremeT) =+ if (lowMem) (300, 1200, 3000)+ else (LowThreshold, MediumThreshold, HighThreshold)++ if (f < medT) RiskTier.Low+ else if (f < highT) RiskTier.Medium+ else if (f < extremeT) RiskTier.High+ else RiskTier.Extreme+ }++ private def selectMode(risk: RiskTier, capacity: LocalCapacity): MapExecutionMode =+ risk match {+ case RiskTier.Extreme => MapExecutionMode.FastLocal+ case RiskTier.High if capacity.heapFreeBytes < 256L * 1024 * 1024 =>+ MapExecutionMode.FastLocal+ case _ => MapExecutionMode.FullLocal+ }++ private def buildWarnings(+ cost: CostEstimate,+ capacity: LocalCapacity,+ risk: RiskTier,+ mode: MapExecutionMode+ ): Vector[String] = {+ val buf = Vector.newBuilder[String]++ if (risk == RiskTier.High || risk == RiskTier.Extreme)+ buf += s"Large workload detected: ${cost.fileCount} files, ${cost.directoryCount} directories"++ if (cost.directoryQuadratic > 50000)+ buf += s"Path-proximity cost is high (D²=${cost.directoryQuadratic})"++ capacity.containerMemory.foreach { mem =>+ if (mem < LowMemoryBytes)+ buf += s"Container memory is low (${mem / (1024 * 1024)}MB)"+ }++ if (capacity.heapFreeBytes < 128L * 1024 * 1024)+ buf += s"JVM heap is low (${capacity.heapFreeBytes / (1024 * 1024)}MB free)"++ if (mode == MapExecutionMode.FastLocal)+ buf += "Full local map is not recommended for this workload"++ buf.result()+ }+}diff --git a/memory-layer/src/main/scala/ix/memory/map/MapService.scala b/memory-layer/src/main/scala/ix/memory/map/MapService.scalaindex e964715a..200cc49c 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapService.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapService.scala@@ -6,6 +6,7 @@ import java.util.UUIDimport cats.effect.IOimport io.circe.Jsonimport io.circe.syntax._+import org.typelevel.log4cats.slf4j.Slf4jLoggerimport ix.memory.db.{ArangoClient, GraphQueryApi, GraphWriteApi}import ix.memory.model._@@ -38,7 +39,13 @@ class MapService("extensions", "ext", "mixin", "mixins", "core", "framework")- private val builder = new MapGraphBuilder(client)+ private val logger = Slf4jLogger.getLoggerFromName[IO]("ix.map.service")+ private val builder = new MapGraphBuilder(client)+ private val fastBuilder = new MapGraphBuilderFast(client)+ private val preflight = new MapPreflight++ // Persistence guardrail: max total patch ops before we refuse to commit+ private val MaxSafePatchOps = 10000// Label kinds by levelprivate def labelKind(level: Int, maxLevel: Int): String =@@ -49,15 +56,54 @@ class MapService(def buildMap(): IO[ArchitectureMap] =for {- rev <- queryApi.getLatestRev- rawGraph <- builder.build()- _ <- IO.pure(()) // no-op; structured for easy step annotation+ rev <- queryApi.getLatestRev+ files <- builder.discoverFiles()+ pf <- preflight.evaluate(files)+ result <- pf.mode match {+ case MapExecutionMode.FullLocal => buildFullLocal(files, pf, rev)+ case MapExecutionMode.FastLocal => buildFastLocal(files, pf, rev)+ }+ } yield result++ private def buildFullLocal(+ files: Vector[FileVertex],+ pf: MapPreflightResult,+ rev: Rev+ ): IO[ArchitectureMap] =+ for {+ rawGraph <- builder.buildGraph(files)crosscutScores = detectCrosscut(rawGraph)graph = applyPenalty(rawGraph, crosscutScores)levels = LouvainClustering.cluster(graph, maxLevels = 3, minCommunitySize = 3)regions = buildRegions(graph, levels, crosscutScores, rev)- _ <- persistMap(regions, rev)- } yield ArchitectureMap(regions, rawGraph.vertices.size, rev.value)+ pe <- guardAndPersist(regions, rev)+ _ <- logOutcome(MapOutcome.FullLocalCompleted, pf, pe)+ } yield ArchitectureMap(+ regions, rawGraph.vertices.size, rev.value,+ preflight = Some(pf),+ outcome = MapOutcome.FullLocalCompleted,+ persistenceEstimate = Some(pe)+ )++ private def buildFastLocal(+ files: Vector[FileVertex],+ pf: MapPreflightResult,+ rev: Rev+ ): IO[ArchitectureMap] =+ for {+ rawGraph <- fastBuilder.buildGraph(files)+ crosscutScores = detectCrosscut(rawGraph)+ graph = applyPenalty(rawGraph, crosscutScores)+ levels = LouvainClustering.cluster(graph, maxLevels = 3, minCommunitySize = 3)+ regions = buildRegions(graph, levels, crosscutScores, rev)+ pe <- guardAndPersist(regions, rev)+ _ <- logOutcome(MapOutcome.FastLocalCompleted, pf, pe)+ } yield ArchitectureMap(+ regions, rawGraph.vertices.size, rev.value,+ preflight = Some(pf),+ outcome = MapOutcome.FastLocalCompleted,+ persistenceEstimate = Some(pe)+ )// ── Cross-cutting detection ─────────────────────────────────────────@@ -359,17 +405,51 @@ class MapService(((0.4 * brScore + 0.3 * cohScore + 0.3 * sizeBonus) - crossPenalty).max(0.0).min(1.0)}- // ── Persistence ─────────────────────────────────────────────────────+ // ── Persistence guardrail + logging ─────────────────────────────────- private def persistMap(regions: Vector[Region], rev: Rev): IO[Unit] = {- if (regions.isEmpty) return IO.unit+ private def guardAndPersist(regions: Vector[Region], rev: Rev): IO[PersistenceEstimate] = {+ if (regions.isEmpty) return IO.pure(PersistenceEstimate(0, 0, 0, 0, 0))for {oldRegions <- queryApi.findNodesByKind(NodeKind.Region, limit = 5000)+ pe = estimatePersistence(regions, oldRegions.size)+ _ <- logger.debug(+ s"Persistence estimate: ${pe.totalOps} ops " ++ s"(${pe.regionNodes} regions, ${pe.fileEdges} file edges, " ++ s"${pe.regionEdges} region edges, ${pe.deleteOps} deletes)"+ )+ _ <- if (pe.totalOps > MaxSafePatchOps)+ IO.raiseError(new MapCapacityException(+ outcome = MapOutcome.LocalMapTooLarge,+ userMessage = "Local map output is too large to persist safely.",+ next = "For full system mapping at this scale, use Ix Cloud."+ ))+ else IO.unit_ <- submitMapPatch(oldRegions, regions, rev)- } yield ()+ } yield pe}+ private def estimatePersistence(regions: Vector[Region], oldRegionCount: Int): PersistenceEstimate = {+ val regionNodes = regions.size+ val fileEdges = regions.filter(_.level == 1).map(_.memberFiles.size).sum+ val regionEdges = regions.count(_.parentId.isDefined)+ val deleteOps = oldRegionCount+ val totalOps = deleteOps + regionNodes + fileEdges + regionEdges+ PersistenceEstimate(regionNodes, fileEdges, regionEdges, deleteOps, totalOps)+ }++ private def logOutcome(+ outcome: MapOutcome,+ pf: MapPreflightResult,+ pe: PersistenceEstimate+ ): IO[Unit] =+ logger.info(+ s"Map completed: outcome=${outcome.label} " ++ s"mode=${pf.mode.label} risk=${pf.risk.label} " ++ s"files=${pf.cost.fileCount} regions=${pe.regionNodes} " ++ s"patchOps=${pe.totalOps} preflightMs=${pf.durationMs}"+ )+private def submitMapPatch(oldRegions: Vector[GraphNode], regions: Vector[Region], rev: Rev): IO[Unit] = {val deleteOps: Vector[PatchOp] = oldRegions.map(n => PatchOp.DeleteNode(n.id))diff --git a/memory-layer/src/main/scala/ix/memory/map/MapTypes.scala b/memory-layer/src/main/scala/ix/memory/map/MapTypes.scalaindex 2b7dd9b3..924e514f 100644--- a/memory-layer/src/main/scala/ix/memory/map/MapTypes.scala+++ b/memory-layer/src/main/scala/ix/memory/map/MapTypes.scala@@ -53,9 +53,84 @@ final case class Region(mapRev: Long)+// ── Preflight types ──────────────────────────────────────────────────++sealed trait RiskTier { def label: String }+object RiskTier {+ case object Low extends RiskTier { val label = "Low" }+ case object Medium extends RiskTier { val label = "Medium" }+ case object High extends RiskTier { val label = "High" }+ case object Extreme extends RiskTier { val label = "Extreme" }+}++sealed trait MapExecutionMode { def label: String }+object MapExecutionMode {+ case object FullLocal extends MapExecutionMode { val label = "FullLocal" }+ case object FastLocal extends MapExecutionMode { val label = "FastLocal" }+}++final case class LocalCapacity(+ cpuCores: Int,+ heapMaxBytes: Long,+ heapFreeBytes: Long,+ containerMemory: Option[Long],+ diskFreeBytes: Option[Long]+)++final case class CostEstimate(+ fileCount: Int,+ directoryCount: Int,+ directoryQuadratic: Long,+ symbolEstimate: Long,+ edgeEstimate: Long+)++final case class MapPreflightResult(+ cost: CostEstimate,+ capacity: LocalCapacity,+ risk: RiskTier,+ mode: MapExecutionMode,+ warnings: Vector[String],+ durationMs: Long+)++// ── Outcome and guardrail types ──────────────────────────────────────++sealed trait MapOutcome { def label: String }+object MapOutcome {+ case object FullLocalCompleted extends MapOutcome { val label = "full_local_completed" }+ case object FastLocalCompleted extends MapOutcome { val label = "fast_local_completed" }+ case object LocalMapTooLarge extends MapOutcome { val label = "local_map_too_large" }+ case object LocalMapNotRecommended extends MapOutcome { val label = "local_map_not_recommended" }+}++final case class PersistenceEstimate(+ regionNodes: Int,+ fileEdges: Int,+ regionEdges: Int,+ deleteOps: Int,+ totalOps: Int+)++/**+ * Structured exception for map capacity failures.+ *+ * Carries the outcome category, a user-facing message, and a next-step+ * suggestion so the error handler can produce clean structured JSON+ * without relying on ad-hoc exception parsing.+ */+class MapCapacityException(+ val outcome: MapOutcome,+ val userMessage: String,+ val next: String+) extends RuntimeException(userMessage)+/** The full multi-level architecture map returned by MapService. */final case class ArchitectureMap(- regions: Vector[Region],- fileCount: Int,- mapRev: Long+ regions: Vector[Region],+ fileCount: Int,+ mapRev: Long,+ preflight: Option[MapPreflightResult] = None,+ outcome: MapOutcome = MapOutcome.FullLocalCompleted,+ persistenceEstimate: Option[PersistenceEstimate] = None)