Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
feat(generation): a tiered system map on the repository overview (#2747)
Raghav Chamadiya8e0fa542ff0f
diff --git a/docs/layers/WIKI.md b/docs/layers/WIKI.mdindex c1a7b71e5..26586fab6 100644--- a/docs/layers/WIKI.md+++ b/docs/layers/WIKI.md@@ -36,7 +36,6 @@ another is generated after it.| 3 | `scc_page` | dependency cycle | structure || 4 | `module_page` | subsystem in the concept tree | model, structure when keyless || 6 | `repo_overview` | repository | model, structure when keyless |-| 6 | `architecture_diagram` | repository | model, structure when keyless || 7 | `infra_page` | infrastructure file (Dockerfile, CI, IaC) | structure || 8 | `onboarding` | slot | model, structure when keyless |@@ -45,9 +44,11 @@ Works, Active Landscape and Glossary. Project Overview is not generatedseparately, the repository overview is tagged into that slot, and Glossary isrendered from mined terms with no model in every run, keyed or not.-`layer_page` was retired. Layers stopped being pages and became grouping rows in-the docs tree, built from provenance stamped on their members. An inbound link-to a retired layer page lands on the repository overview.+Layers are grouping rows in the docs tree, built from provenance stamped on+their members, not pages of their own. The repository overview carries the+system map, a tiered diagram of who uses the system, the ways in, the server,+the engine and the data store. Structure picks its boxes and arrows; a+configured model only names them.## Which pages exist@@ -70,8 +71,8 @@ at all.## What a model writes-Four page types are model-written when a provider is configured: `module_page`,-`repo_overview`, `architecture_diagram` and `onboarding`. That is the layer+Three page types are model-written when a provider is configured: `module_page`,+`repo_overview` and `onboarding`. That is the layerwhere prose earns its keep, the one that explains how the codebase fits togetherabove the file level. Everything else is rendered from the parse, the importgraph and git history, and no model ever sees it.diff --git a/docs/reference/COMPUTED_GLOSSARY.md b/docs/reference/COMPUTED_GLOSSARY.mdindex 4c2e4acf3..59c7f4339 100644--- a/docs/reference/COMPUTED_GLOSSARY.md+++ b/docs/reference/COMPUTED_GLOSSARY.md@@ -171,7 +171,6 @@ workspace overlays, MCP responses, and CLI output.| Module page context | Aggregate context for top-level directory/module. | `assemble_module_page()` | `{module_path: "packages/core", total_symbols: 780}` || SCC page context | Context for a circular dependency cycle. | `assemble_scc_page()` | `cycle_description: "Circular dependency cycle: a.py -> b.py"` || Repo overview context | Whole-repo summary context. | `assemble_repo_overview()` | `language_distribution`, `top_files_by_pagerank`, `circular_dependency_count` |-| Architecture diagram context | Top PageRank nodes, selected edges, communities, SCC groups. | `assemble_architecture_diagram()` | Mermaid graph inputs for 50 nodes and 200 edges || API contract context | Raw API contract plus endpoint/schema hints. | `assemble_api_contract()` | `endpoints: ["GET /users"]`, `schemas: ["User"]` || Infra page context | Raw infra file plus target names. | `assemble_infra_page()` | `Dockerfile`, `Makefile`, `terraform` files || Dependency summaries | Summaries of already-generated dependency pages. | `assemble_file_page()` with `page_summaries` | `{ "src/db.py": "Database access layer..." }` |diff --git a/packages/core/src/repowise/core/analysis/c4/__init__.py b/packages/core/src/repowise/core/analysis/c4/__init__.pynew file mode 100644index 000000000..a0e481b19--- /dev/null+++ b/packages/core/src/repowise/core/analysis/c4/__init__.py@@ -0,0 +1,6 @@+"""Session-free C4 derivation: containers, actors, relation roll-up and verbs.++The server's C4 views and the wiki's architecture map draw the same boxes and+arrows, so the rules that decide them live here, over plain paths and edge+rows. The server keeps the queries and hands the rows in.+"""diff --git a/packages/server/src/repowise/server/services/c4_builder/actors.py b/packages/core/src/repowise/core/analysis/c4/actors.pysimilarity index 89%rename from packages/server/src/repowise/server/services/c4_builder/actors.pyrename to packages/core/src/repowise/core/analysis/c4/actors.pyindex 4e97fd2f1..9c8ec8648 100644--- a/packages/server/src/repowise/server/services/c4_builder/actors.py+++ b/packages/core/src/repowise/core/analysis/c4/actors.py@@ -102,17 +102,16 @@ def derive_actors(entry_points: list[str]) -> list[Actor]:if not kinds:kinds = ["user"]- ordered = [k for k in _KIND_ORDER if k in kinds]- actors: list[Actor] = []- for kind in ordered:- name, description = _ACTOR_META[kind]- actors.append(- Actor(- id=render(PersonId(kind)),- kind=kind,- name=name,- description=description,- verb=_ACTOR_VERB[kind],- )- )- return actors+ return [actor_for(k) for k in _KIND_ORDER if k in kinds]+++def actor_for(kind: str) -> Actor:+ """The actor of one kind (``cli``, ``api``, ``scheduler``, ``developer``, ``user``)."""+ name, description = _ACTOR_META[kind]+ return Actor(+ id=render(PersonId(kind)),+ kind=kind,+ name=name,+ description=description,+ verb=_ACTOR_VERB[kind],+ )diff --git a/packages/core/src/repowise/core/analysis/c4/containers.py b/packages/core/src/repowise/core/analysis/c4/containers.pynew file mode 100644index 000000000..18ab6681d--- /dev/null+++ b/packages/core/src/repowise/core/analysis/c4/containers.py@@ -0,0 +1,99 @@+"""Assign files to C4 containers.++A directory holding a package manifest is a container root. Manifests under+test or example trees are fixtures and samples, not the system's containers, so+they are dropped and their files fold into the nearest remaining ancestor+container (or into none, when no ancestor exists). With no manifest at all, the+top-level directories holding production files stand in.+"""++from __future__ import annotations++from collections import defaultdict+from collections.abc import Iterable+from pathlib import Path++from repowise.core.ingestion.workspace_members import manifest_package_name+from repowise.core.support_paths import is_test_or_example_path++# Package-manifest basenames. A directory holding one of these is a container+# root even when the manifest declares no external dependencies (so a package+# like ``packages/core`` is its own container instead of dissolving into the+# repo-root catch-all).+_MANIFEST_BASENAMES = frozenset(+ {+ "pyproject.toml",+ "setup.py",+ "setup.cfg",+ "package.json",+ "cargo.toml",+ "go.mod",+ "pom.xml",+ "build.gradle",+ "composer.json",+ "gemfile",+ }+)+_MANIFEST_SUFFIXES = (".csproj",)+++def is_manifest(path: str) -> bool:+ base = path.rsplit("/", 1)[-1].lower()+ return base in _MANIFEST_BASENAMES or base.endswith(_MANIFEST_SUFFIXES)+++def manifest_roots(manifests: Iterable[str]) -> dict[str, list[str]]:+ """``{container root: its manifests}``, fixture and sample manifests dropped."""+ roots: dict[str, list[str]] = defaultdict(list)+ for path in sorted(manifests):+ if not is_test_or_example_path(path):+ roots[path.rsplit("/", 1)[0] if "/" in path else ""].append(path)+ return roots+++def assign_containers(+ manifests: Iterable[str], file_paths: Iterable[str]+) -> tuple[dict[str, str], dict[str, list[str]]]:+ """Map each file to its container root, and each root to its manifests.++ *manifests* may be a superset of the manifest files: any path in+ *file_paths* that is a manifest counts too. Files no root claims are left+ out of the map. The root ``""`` is the repository root.+ """+ paths = list(file_paths)+ roots = manifest_roots({*manifests, *(p for p in paths if is_manifest(p))})+ candidates = set(roots) or _top_level_dirs(p for p in paths if not is_test_or_example_path(p))+ # Longest first so nested manifests win over their parents.+ ordered = sorted(candidates, key=len, reverse=True)+ owner: dict[str, str] = {}+ for path in paths:+ root = _match_container(path, ordered)+ if root is not None:+ owner[path] = root+ return owner, roots+++def container_name(repo_root: Path | None, manifests: Iterable[str]) -> str | None:+ """The first package name the root's manifests declare, read from the checkout."""+ # Ceiling: needs the checkout on disk; persisting names at index time lifts it.+ if repo_root is None or not repo_root.is_dir():+ return None+ for manifest in manifests:+ name = manifest_package_name(repo_root / manifest)+ if name:+ return name+ return None+++def _top_level_dirs(paths: Iterable[str]) -> set[str]:+ return {path.split("/", 1)[0] if "/" in path else "" for path in paths}+++def _match_container(file_path: str, sorted_roots: list[str]) -> str | None:+ """Return the deepest container root containing ``file_path``."""+ for root in sorted_roots:+ if not root:+ return root # root manifest catches everything that isn't claimed+ if file_path == root or file_path.startswith(root + "/"):+ return root+ return Nonediff --git a/packages/server/src/repowise/server/services/c4_builder/labels.py b/packages/core/src/repowise/core/analysis/c4/labels.pysimilarity index 100%rename from packages/server/src/repowise/server/services/c4_builder/labels.pyrename to packages/core/src/repowise/core/analysis/c4/labels.pydiff --git a/packages/core/src/repowise/core/analysis/c4/relations.py b/packages/core/src/repowise/core/analysis/c4/relations.pynew file mode 100644index 000000000..1d3f7d41e--- /dev/null+++ b/packages/core/src/repowise/core/analysis/c4/relations.py@@ -0,0 +1,50 @@+"""Roll file-to-file graph edges up to box-to-box relations.++Each endpoint maps to its box (a container, a component, or an external+system), counts are summed per directed box pair with the edge types seen, and+self-loops are dropped. Co-change edges are history, not a dependency, so they+are left out unless a caller opts in. Edges touching a configuration file are+dropped: reading config is not a dependency between boxes.+"""++from __future__ import annotations++from collections import defaultdict+from collections.abc import Iterable, Mapping++from repowise.core.support_paths import is_config_path++#: ``(source box, target box) -> (file-pair count, edge types)``.+BoxEdges = dict[tuple[str, str], tuple[int, frozenset[str]]]+++def roll_up_edges(+ edges: Iterable[tuple[str, str, str]],+ file_to_box: Mapping[str, str],+ *,+ file_to_external: Mapping[str, str] | None = None,+ include_co_changes: bool = False,+) -> BoxEdges:+ """Sum ``(source, target, edge_type)`` rows into box pairs.++ *file_to_external* maps ``external:*`` node ids to an external box, so an+ import of a third-party package becomes a box-to-external edge. Only file+ endpoints are keyed, so containment edges (file to symbol) never land.+ """+ externals = file_to_external or {}+ counts: dict[tuple[str, str], int] = defaultdict(int)+ types: dict[tuple[str, str], set[str]] = defaultdict(set)+ for src, tgt, etype in edges:+ if etype == "co_changes" and not include_co_changes:+ continue+ src_box = file_to_box.get(src)+ if src_box is None:+ continue+ tgt_box = file_to_box[tgt] if tgt in file_to_box else externals.get(tgt)+ if tgt_box is None or src_box == tgt_box:+ continue+ if is_config_path(src) or is_config_path(tgt):+ continue+ counts[(src_box, tgt_box)] += 1+ types[(src_box, tgt_box)].add(etype)+ return {key: (count, frozenset(types[key])) for key, count in counts.items()}diff --git a/packages/core/src/repowise/core/cost_estimator/heuristics.py b/packages/core/src/repowise/core/cost_estimator/heuristics.pyindex 3f2151dcb..245b69e92 100644--- a/packages/core/src/repowise/core/cost_estimator/heuristics.py+++ b/packages/core/src/repowise/core/cost_estimator/heuristics.py@@ -22,7 +22,6 @@ _TOKEN_HEURISTICS: dict[str, tuple[int, int]] = {"scc_page": (12_000, 4_000),"module_page": (45_000, 6_000),"repo_overview": (30_000, 5_000),- "architecture_diagram": (25_000, 5_000),"infra_page": (4_000, 2_000),"onboarding": (15_000, 3_500),}diff --git a/packages/core/src/repowise/core/generation/__init__.py b/packages/core/src/repowise/core/generation/__init__.pyindex 0e0120700..ead57cb64 100644--- a/packages/core/src/repowise/core/generation/__init__.py+++ b/packages/core/src/repowise/core/generation/__init__.py@@ -20,7 +20,6 @@ from typing import Any# don't pull the heavy assembler/generator stack via this ``__init__``._EXPORTS = {"ApiContractContext": ".context_assembler",- "ArchitectureDiagramContext": ".context_assembler","ContextAssembler": ".context_assembler","FilePageContext": ".context_assembler","InfraPageContext": ".context_assembler",diff --git a/packages/core/src/repowise/core/generation/architecture_map.py b/packages/core/src/repowise/core/generation/architecture_map.pynew file mode 100644index 000000000..19fbd66f7--- /dev/null+++ b/packages/core/src/repowise/core/generation/architecture_map.py@@ -0,0 +1,613 @@+"""Choose the boxes and arrows of the overview's system map.++The map a newcomer reads first shows a handful of boxes in tiers: who uses the+system, the ways in, shared libraries, the server, the engine, and the data+store. Structure alone picks them, so the same index always draws the same map:++* Boxes start as the C4 containers (manifest roots). A repository with too few+ of them is opened one directory level at a time, largest first, and then+ gets a smaller box budget, since a single package reads best in a few parts.+* A re-export file (it imports, but defines no symbol, route or entry point of+ its own) is not flow: its outgoing imports are dropped, and a box that is+ only such files is dropped. An entry point that only forwards to one file+ hands its role to that file.+* Tiers come from how a box is entered: declaring HTTP routes makes it the+ server; an entry point, an HTTP call to another box, or nothing depending on+ it makes it a way in; everything else is engine. A package only ways in+ import is a library of theirs: folded into its one consumer, or into one+ shared box.+* Actors come from the entry points: one person for every human way in (a+ terminal, an app), and one box per program that calls in. An entry point+ that looks like an API but serves no route is a person's way in.+* Arrows are the import and call graph rolled up to boxes, plus the runtime+ links no import expresses (actor to entry point, HTTP client to route). An+ arrow carrying a large share of the dependencies is always drawn; the rest+ fill a small budget, strongest first, at most two per box, skipping any that+ drawn arrows already connect. The caption says when parts or links were cut.++A model may rename boxes and verbs later; it never adds or removes one.+"""++from __future__ import annotations++import re+from collections import defaultdict+from collections.abc import Collection, Iterable, Mapping, Sequence+from dataclasses import dataclass, replace++from repowise.core.analysis.c4.actors import actor_for, classify_entry_point+from repowise.core.analysis.c4.containers import assign_containers+from repowise.core.analysis.c4.labels import relation_label+from repowise.core.analysis.c4.relations import BoxEdges, roll_up_edges+from repowise.core.ingestion.languages.registry import REGISTRY++TIERS: tuple[str, ...] = ("actor", "surface", "shared", "service", "engine", "store")++MAX_NODES = 11+# A single package opened into its parts reads best in a few of them.+MAX_NODES_OPENED = 7+# Fewer code parts than this and the containers alone say nothing, so the+# largest one is opened up a level.+_MIN_UNITS = 5+_MAX_EDGES = 10+_MAX_EDGES_PER_BOX = 2+# An arrow carrying this share of the dependency weight is always drawn, and+# drawn thick when it also stands out from a typical arrow.+_HEAVY_SHARE = 0.1++# Share of a box's files that must declare HTTP routes for it to be the server.+_ROUTE_SHARE = 0.05++_STORE_ID = "n_store"+_PERSON_ID = "n_actor_user"+# Kinds a person enters by, in the order that names the person when alone.+_PERSON_KINDS = ("cli", "user", "developer")+++@dataclass(frozen=True)+class MapNode:+ """One box. ``name``/``role`` are the keyless labels a model may replace."""++ id: str+ tier: str+ name: str+ role: str+ path: str = ""+ page_id: str = ""+ #: Plain facts about the box, shown to the model that names it.+ facts: tuple[str, ...] = ()+++@dataclass(frozen=True)+class MapEdge:+ source: str+ target: str+ verb: str+ weight: int = 0+ heavy: bool = False+ #: Why the arrow exists: the evidence shown to the model that names it.+ basis: str = ""+++@dataclass(frozen=True)+class SystemMap:+ nodes: tuple[MapNode, ...]+ edges: tuple[MapEdge, ...]+ #: Code parts left off the map for space.+ omitted: int = 0+ #: A model-written sentence introducing the map; empty when keyless.+ caption: str = ""+ #: True when the arrow budget left out links that no drawn path shows.+ arrows_cut: bool = False++ def with_names(+ self, names: Mapping[str, tuple[str, str]], verbs: Mapping[tuple[str, str], str]+ ) -> SystemMap:+ """A copy with the given node labels and edge verbs swapped in."""+ nodes = tuple(+ replace(n, name=names[n.id][0], role=names[n.id][1]) if n.id in names else n+ for n in self.nodes+ )+ edges = tuple(+ replace(e, verb=verbs[(e.source, e.target)]) if (e.source, e.target) in verbs else e+ for e in self.edges+ )+ return replace(self, nodes=nodes, edges=edges)+++@dataclass+class _Graph:+ """Box-level view of the edges, recomputed whenever boxes merge."""++ rolled: BoxEdges+ runtime: set[tuple[str, str]]+ deps: set[tuple[str, str]]+ has_in: set[str]+ has_out: set[str]+++def build_system_map(+ files: Mapping[str, str],+ edges: Iterable[tuple[str, str, str]],+ *,+ repo_name: str,+ weights: Mapping[str, float] | None = None,+ manifests: Iterable[str] = (),+ package_names: Mapping[str, str] | None = None,+ entry_points: Sequence[str] = (),+ barrels: Collection[str] = (),+ route_files: Collection[str] = (),+ stores: Mapping[str, str] | None = None,+ http_links: Iterable[tuple[str, str]] = (),+ pages: Sequence[tuple[str, Collection[str]]] = (),+) -> SystemMap | None:+ """Pick the map's boxes and arrows, or ``None`` when there is nothing to draw.++ *files* maps each production code file to its language and *edges* are+ ``(source, target, edge_type)`` graph rows. *barrels* are files that define+ no symbols of their own, *route_files* declare HTTP routes, *stores* maps an+ ``external:*`` node id to the database client it stands for, *http_links*+ pairs a calling file with the file serving the route, and *pages* lists each+ module page's target path with the files it covers, for click-through.+ """+ weights = weights or {}+ package_names = package_names or {}+ edges = list(edges)+ # A re-export file imports something and declares no route of its own.+ importing = {s for s, _, etype in edges if etype == "imports"}+ barrels = (set(barrels) & importing) - set(route_files)+ entries = _forward_entries(entry_points, barrels, edges)+ barrels -= {path for path, _ in entries}+ edges = [e for e in edges if e[0] not in barrels]+ units, opened = _units(files, manifests, weights, barrels)+ store_of = {node: _STORE_ID for node in (stores or {})}+ links = list(http_links)++ graph = _graph(units, edges, store_of, links)+ kinds = _entry_kinds(units, entries, route_files)+ # A part another way in already calls is reached through it, not by a person.+ kinds = {+ u: k+ for u, k in kinds.items()+ if not (+ k <= set(_PERSON_KINDS)+ and any((o, u) in graph.deps and (u, o) not in graph.deps for o in kinds)+ )+ }+ tiers = _tiers(units, kinds, graph, route_files)+ labels: dict[str, str] = {}+ if not opened:+ units, shared = _fold_libraries(units, kinds, tiers, graph)+ if shared:+ labels[shared] = "Shared libraries"+ graph = _graph(units, edges, store_of, links)+ tiers = _tiers(units, kinds, graph, route_files)+ if shared:+ tiers[shared] = "shared"+ anchored = set(kinds) | {u for pair in graph.runtime for u in pair}+ # The repository-root catch-all of a multi-package repo holds leftovers+ # (scripts, loose tooling), not a part of the system.+ if len(tiers) > _MIN_UNITS and "" in tiers and "" not in anchored:+ del tiers[""]++ unreached = {+ u for u, t in tiers.items() if t == "surface" and u not in graph.has_in and u not in kinds+ }+ person, programs = _actors(kinds, unreached)+ store_users = {s for s, t in graph.rolled if t == _STORE_ID}+ cap = MAX_NODES_OPENED if opened else MAX_NODES+ budget = cap - (1 if person else 0) - len(programs)+ ranked = sorted(+ tiers,+ key=lambda u: (u not in anchored, -len(units[u]), -_weight(units[u], weights), u),+ )+ # The store takes a slot only when a box that is drawn uses it.+ kept = ranked[: budget - 1] if store_users & set(ranked[: budget - 1]) else ranked[:budget]+ if len(kept) < 2:+ return None++ ids = _node_ids(kept)+ starts = [path for path, _ in entries]+ nodes: list[MapNode] = []+ if person:+ nodes.append(MapNode(id=_PERSON_ID, tier="actor", name=person.name, role=person.description))+ nodes += [+ MapNode(id=f"n_actor_{a.kind}", tier="actor", name=a.name, role=a.description)+ for a in programs+ ]+ for unit in sorted(kept, key=lambda u: (TIERS.index(tiers[u]), -len(units[u]), u)):+ name = labels.get(unit) or package_names.get(unit) or _last_segment(unit, repo_name)+ nodes.append(+ _unit_node(unit, ids[unit], tiers[unit], name, units[unit], files, starts, pages)+ )+ if store_users & set(kept):+ clients = sorted(set((stores or {}).values()))+ nodes.append(+ MapNode(+ id=_STORE_ID,+ tier="store",+ name="Database",+ role=_clip_words("Via " + ", ".join(clients[:2]), 6),+ facts=(f"database clients imported: {', '.join(clients)}",),+ )+ )++ map_edges = _actor_edges(person, programs, kept, ids, kinds, unreached)+ map_edges += [+ MapEdge(ids[c], ids[p], "calls over HTTP", basis="HTTP client call to a route it serves")+ for c, p in sorted(graph.runtime)+ if c in ids and p in ids+ ]+ drawn, cut = _dependency_edges(graph.rolled, ids, graph.runtime)+ return SystemMap(+ nodes=tuple(nodes),+ edges=tuple(map_edges + drawn),+ omitted=len(units) - len(kept),+ arrows_cut=cut,+ )+++# -- structure ----------------------------------------------------------------+++def _forward_entries(+ entry_points: Sequence[str], barrels: set[str], edges: list[tuple[str, str, str]]+) -> list[tuple[str, str]]:+ """``(path, kind)`` per classified entry point.++ A symbol-less launcher hands its kind to the one file it imports.+ """+ out: list[tuple[str, str]] = []+ for path in entry_points:+ kind = classify_entry_point(path)+ if kind is None:+ continue+ targets = {t for s, t, etype in edges if s == path and etype == "imports"}+ if path in barrels and len(targets) == 1:+ path = targets.pop()+ if (path, kind) not in out:+ out.append((path, kind))+ return out+++def _graph(units, edges, store_of, http_links) -> _Graph:+ file_to_unit = {f: u for u, members in units.items() for f in members}+ rolled = roll_up_edges(edges, file_to_unit, file_to_external=store_of)+ runtime = {+ (file_to_unit[c], file_to_unit[p])+ for c, p in http_links+ if c in file_to_unit and p in file_to_unit and file_to_unit[c] != file_to_unit[p]+ }+ deps = {pair for pair in rolled if pair[1] != _STORE_ID} | runtime+ return _Graph(+ rolled=rolled,+ runtime=runtime,+ deps=deps,+ has_in={t for _, t in deps},+ has_out={s for s, _ in deps},+ )+++def _entry_kinds(+ units: Mapping[str, list[str]],+ entries: Sequence[tuple[str, str]],+ route_files: Collection[str],+) -> dict[str, set[str]]:+ """How each unit is entered, by the kind of entry point it holds."""+ file_to_unit = {f: u for u, members in units.items() for f in members}+ serves = {file_to_unit[f] for f in route_files if f in file_to_unit}+ kinds: dict[str, set[str]] = defaultdict(set)+ for path, kind in entries:+ if path in file_to_unit:+ unit = file_to_unit[path]+ # An "app" that serves no route is a library a person's code calls.+ kinds[unit].add("user" if kind == "api" and unit not in serves else kind)+ if any(k - {"developer"} for k in kinds.values()):+ # Scripts and tooling are how the repo is maintained, not how it is used.+ kinds = {u: k - {"developer"} for u, k in kinds.items() if k - {"developer"}}+ return dict(kinds)+++def _tiers(units, kinds, graph: _Graph, route_files: Collection[str]) -> dict[str, str]:+ """Which tier each unit sits in; a part nothing touches is left out."""+ routes = set(route_files)+ # A stray route pattern in a large library (a string, a parser's own test+ # input) does not make it the server; a box that is mostly routes does.+ serves = {+ u+ for u, members in units.items()+ if sum(1 for m in members if m in routes) >= _ROUTE_SHARE * len(members)+ and not routes.isdisjoint(members)+ }+ tiers: dict[str, str] = {}+ for unit in units:+ entered = kinds.get(unit, set())+ if unit in serves or "api" in entered or any(p == unit for _, p in graph.runtime):+ tiers[unit] = "service"+ elif (+ entered+ or any(c == unit for c, _ in graph.runtime)+ or (unit not in graph.has_in and unit in graph.has_out)+ ):+ tiers[unit] = "surface"+ elif unit in graph.has_in or unit in graph.has_out or (unit, _STORE_ID) in graph.rolled:+ tiers[unit] = "engine"+ return tiers+++def _fold_libraries(units, kinds, tiers, graph: _Graph) -> tuple[dict[str, list[str]], str]:+ """Fold packages only the ways in import into them.++ A library with one consuming way in joins that box; libraries shared by+ several join one shared box. Returns the new units and the shared box key.+ """+ surfaces = {u for u, t in tiers.items() if t == "surface" and u in kinds}+ surfaces |= {u for u, t in tiers.items() if t == "surface" and u not in graph.has_in}+ importers: dict[str, set[str]] = defaultdict(set)+ for s, t in graph.deps:+ importers[t].add(s)+ # A package that keeps the data is the engine, whoever calls it.+ libs = {+ u+ for u in tiers+ if u not in surfaces+ and tiers[u] != "service"+ and importers[u]+ and (u, _STORE_ID) not in graph.rolled+ }+ changed = True+ while changed:+ changed = False+ for u in sorted(libs):+ if importers[u] - libs - surfaces:+ libs.discard(u)+ changed = True+ # Folding every part behind the ways in would leave a map with no engine.+ if not libs or not set(tiers) - libs - surfaces:+ return units, ""++ def roots(u: str, seen: frozenset[str] = frozenset()) -> set[str]:+ found = importers[u] & surfaces+ for lib in (importers[u] & libs) - seen - {u}:+ found |= roots(lib, seen | {u})+ return found++ merged = {u: list(members) for u, members in units.items()}+ shared = [u for u in sorted(libs) if len(roots(u)) != 1]+ for u in sorted(libs):+ if u not in shared:+ (owner,) = roots(u)+ merged[owner] += merged.pop(u)+ if not shared:+ return merged, ""+ key = max(shared, key=lambda u: (len(units[u]), u))+ for u in shared:+ if u != key:+ merged[key] += merged.pop(u)+ return merged, key+++def _actors(kinds: Mapping[str, set[str]], unreached: set[str]):+ """The person entering every human way in, and each program that calls in."""+ present = set().union(*kinds.values()) if kinds else set()+ if unreached or not present:+ present.add("user")+ human = [k for k in _PERSON_KINDS if k in present]+ person = None+ if human:+ person = actor_for(human[0] if len(human) == 1 else "user")+ programs = [actor_for(k) for k in ("api", "scheduler") if k in present]+ return person, programs[:1]+++# -- units --------------------------------------------------------------------+++def _units(+ files: Mapping[str, str],+ manifests: Iterable[str],+ weights: Mapping[str, float],+ barrels: set[str],+) -> tuple[dict[str, list[str]], bool]:+ """Code units, and whether a container had to be opened to find enough."""+ owner, _ = assign_containers(manifests, files)+ units: dict[str, list[str]] = defaultdict(list)+ for path in sorted(files):+ if path in owner:+ units[owner[path]].append(path)+ opened = False+ while len(units) < _MIN_UNITS:+ splits = {u: _split(u, members) for u, members in units.items()}+ splittable = [u for u, parts in splits.items() if len(parts) > 1]+ if not splittable:+ break+ # Opening a unit can overshoot the node budget; the ranking trims it.+ largest = max(splittable, key=lambda u: (len(units[u]), _weight(units[u], weights), u))+ del units[largest]+ units.update(splits[largest])+ opened = True+ kept = {u: m for u, m in units.items() if not set(m) <= barrels}+ return kept, opened+++def _split(root: str, members: list[str]) -> dict[str, list[str]]:+ """Children of *root*: each subdirectory, and each loose file, one level down.++ Directory chains every member shares (``src/pkg``) are walked through, so a+ split lands on the first level where the files actually diverge.+ """+ base = root+ while True:+ rel = [_rel(p, base) for p in members]+ heads = {r.split("/", 1)[0] for r in rel}+ if len(heads) == 1 and all("/" in r for r in rel):+ base = f"{base}/{heads.pop()}" if base else heads.pop()+ continue+ break+ parts: dict[str, list[str]] = defaultdict(list)+ for path, r in zip(members, rel, strict=True):+ head = r.split("/", 1)[0]+ child = path if "/" not in r else (f"{base}/{head}" if base else head)+ parts[child].append(path)+ return dict(parts)+++def _rel(path: str, base: str) -> str:+ return path[len(base) + 1 :] if base else path+++def _weight(members: Iterable[str], weights: Mapping[str, float]) -> float:+ return sum(weights.get(m, 0.0) for m in members)+++def _node_ids(units: Iterable[str]) -> dict[str, str]:+ ids: dict[str, str] = {}+ taken: set[str] = {_STORE_ID, _PERSON_ID}+ for unit in sorted(units):+ slug = re.sub(r"[^A-Za-z0-9]+", "_", unit).strip("_").lower() or "root"+ candidate = f"n_{slug}"+ n = 2+ while candidate in taken:+ candidate = f"n_{slug}_{n}"+ n += 1+ taken.add(candidate)+ ids[unit] = candidate+ return ids+++def _unit_node(+ unit: str,+ node_id: str,+ tier: str,+ name: str,+ members: list[str],+ files: Mapping[str, str],+ entry_points: Sequence[str],+ pages: Sequence[tuple[str, Collection[str]]],+) -> MapNode:+ languages: dict[str, int] = defaultdict(int)+ for m in members:+ languages[files[m]] += 1+ language = max(sorted(languages), key=lambda lang: languages[lang])+ count = len(members)+ spec = REGISTRY.get(language)+ shown = spec.display_name if spec else language+ role = f"{count} {shown} file{'s' if count != 1 else ''}"+ facts = [f"path: {unit or '(repository root)'}", f"current name: {name}", role]+ dirs = sorted({m.rsplit("/", 1)[0] for m in members if "/" in m})+ if len(dirs) > 1:+ facts.append("directories: " + ", ".join(dirs[:6]))+ starts = [e for e in entry_points if e in members]+ if starts:+ facts.append("entry points: " + ", ".join(starts[:3]))+ facts.append("files: " + ", ".join(m.rsplit("/", 1)[-1] for m in members[:8]))+ return MapNode(+ id=node_id,+ tier=tier,+ name=name,+ role=role,+ path=unit,+ page_id=_page_for(unit, members, pages),+ facts=tuple(facts),+ )+++def _last_segment(unit: str, repo_name: str) -> str:+ return unit.rsplit("/", 1)[-1] if unit else repo_name+++def _page_for(unit: str, members: list[str], pages: Sequence[tuple[str, Collection[str]]]) -> str:+ """The module page to open for *unit*: its own, else the one covering most of it."""+ best, best_count = "", 0+ for key, covered in pages:+ if key == unit:+ return key+ count = sum(1 for m in members if m in covered)+ if count > best_count or (count == best_count and count and len(key) < len(best)):+ best, best_count = key, count+ return best+++# -- edges --------------------------------------------------------------------+++def _actor_edges(person, programs, kept, ids, kinds, unreached) -> list[MapEdge]:+ edges: list[MapEdge] = []+ if person:+ for unit in sorted(kept):+ entered = [k for k in _PERSON_KINDS if k in kinds.get(unit, set())]+ if not entered and unit not in unreached:+ continue+ verb = actor_for(entered[0]).verb if entered else "uses"+ basis = "entry point" if entered else "top-level way in"+ edges.append(MapEdge(_PERSON_ID, ids[unit], verb, basis=f"{basis}: {unit}"))+ for actor in programs:+ edges += [+ MapEdge(f"n_actor_{actor.kind}", ids[u], actor.verb, basis=f"entry point: {u}")+ for u in sorted(kept)+ if actor.kind in kinds.get(u, set())+ ]+ return edges+++def _dependency_edges(rolled, ids, runtime) -> tuple[list[MapEdge], bool]:+ """The arrows to draw, and whether the budget left any link out."""+ pool = [+ (count, s, t, types)+ for (s, t), (count, types) in rolled.items()+ if s in ids and (t in ids or t == _STORE_ID) and (s, t) not in runtime+ ]+ # A pair importing each other draws once, the heavier way.+ counts = {(s, t): count for count, s, t, _ in pool}+ pool = [p for p in pool if (counts.get((p[2], p[1]), -1), p[2], p[1]) < (p[0], p[1], p[2])]+ pool.sort(key=lambda p: (-p[0], p[1], p[2]))+ total = sum(p[0] for p in pool) or 1+ typical = sorted(p[0] for p in pool)[len(pool) // 2] if pool else 0+ node = {**ids, _STORE_ID: _STORE_ID}+ drawn: list[MapEdge] = []+ pairs: set[tuple[str, str]] = set()+ per_box: dict[str, int] = defaultdict(int)+ cut = False+ for count, s, t, types in pool:+ big = count >= _HEAVY_SHARE * total+ # Every arrow into the store is drawn: it is the only thing the store box shows.+ always = big or t == _STORE_ID+ if not always and _reachable(s, t, pairs):+ continue+ if not always and (len(drawn) >= _MAX_EDGES or per_box[s] >= _MAX_EDGES_PER_BOX):+ cut = True+ continue+ if not always:+ per_box[s] += 1+ pairs.add((s, t))+ verb = "reads and writes" if t == _STORE_ID else relation_label(types)+ pairs_text = f"{count} file pair{'s' if count != 1 else ''}"+ drawn.append(+ MapEdge(+ node[s],+ node[t],+ verb,+ weight=count,+ heavy=big and count >= 2 * typical,+ basis=f"{verb} across {pairs_text}",+ )+ )+ return drawn, cut+++def _reachable(source: str, target: str, pairs: set[tuple[str, str]]) -> bool:+ """True when the arrows drawn so far already lead from *source* to *target*."""+ seen, frontier = {source}, [source]+ while frontier:+ here = frontier.pop()+ for a, b in pairs:+ if a == here and b not in seen:+ if b == target:+ return True+ seen.add(b)+ frontier.append(b)+ return False+++def _clip_words(text: str, limit: int) -> str:+ return " ".join(text.split()[:limit])diff --git a/packages/core/src/repowise/core/generation/architecture_mermaid.py b/packages/core/src/repowise/core/generation/architecture_mermaid.pyindex aecbb7e28..e4a9c9c8a 100644--- a/packages/core/src/repowise/core/generation/architecture_mermaid.py+++ b/packages/core/src/repowise/core/generation/architecture_mermaid.py@@ -1,230 +1,219 @@-"""Deterministic architecture-diagram mermaid, built from the knowledge graph.+"""Name, compile and place the overview's system map.-The overview map and the per-layer diagrams are emitted straight from the KG-(layers, curated modules, and ``imports`` edges) rather than authored by the-LLM. Deterministic construction is free (no model call), can never emit a-diagram that fails to render, and stays perfectly faithful to the graph.+:mod:`.architecture_map` decides the boxes and arrows. This module turns that+structure into the page's diagram: an optional model pass renames the boxes and+verbs through a JSON block the overview call appends to its reply, the diagram+compiles to mermaid in code, and the section lands under the page's first+section. Model text is sanitised before it reaches mermaid or markdown, so the+compiled source is valid by construction.-The output is intentionally style-free: no inline colors or ``classDef``. The-rendering component (packages/ui MermaidDiagram) owns the palette through design-tokens and is theme-aware, so a hardcoded fill here would break in dark mode.-Boundary layers in a layer diagram are distinguished by node *shape*, not color.--``build_overview_mermaid`` / ``build_layer_mermaid`` return ``None`` when the KG-lacks the data to draw a useful diagram (older or uncurated indexes), so callers-fall back cleanly to prior behavior.+The compiled mermaid carries no colors. Each tier gets a ``classDef`` that only+varies the stroke, so the renderer's theme owns the palette in light and dark."""from __future__ import annotations+import jsonimport re-from collections import defaultdict-from typing import TYPE_CHECKING+from dataclasses import replace+from typing import Any+from urllib.parse import quote++from .architecture_map import TIERS, MapEdge, MapNode, SystemMap+from .models import compute_page_id++HEADING = "## System map"+# Written by earlier versions at the end of the page; replaced on re-embed.+_LEGACY_HEADING = "## Architecture map"++_TIER_TITLES = {+ "actor": "Who uses it",+ "surface": "Ways in",+ "shared": "Shared libraries",+ "service": "Server",+ "engine": "Engine",+ "store": "Data store",+}+# Stroke only, no color: the renderer's theme tokens own the palette.+_TIER_STYLE = {+ "actor": "stroke-dasharray:4 4",+ "surface": "stroke-width:2px",+ "shared": "stroke-width:1px",+ "service": "stroke-width:2px",+ "engine": "stroke-width:1px",+ "store": "stroke-width:1px",+}+_SHAPES = {"actor": ('(["', '"])'), "store": ('[("', '")]')}++# A fenced JSON object, with or without a language tag. A reply cut off inside+# the block leaves an unterminated fence at the very end.+_NAMES_FENCE_RE = re.compile(+ r"\n?[ \t]*```(?:json)?[ \t]*\n\s*(?P<body>\{.*?\})\s*```[ \t]*",+ re.DOTALL | re.IGNORECASE,+)+_TRUNCATED_FENCE_RE = re.compile(r"\n?[ \t]*```(?:json)?[ \t]*\n\s*\{[^`]*\Z", re.IGNORECASE)+_NAMES_KEYS = ("nodes", "edges", "caption")+# Kept out of labels: each is syntax in a mermaid label or an HTML label.+_UNSAFE_CHARS_RE = re.compile(r"[\"'`<>|\[\]{}#;:\\]")+# Kept out of the caption, which is markdown: fences and HTML.+_CAPTION_UNSAFE_RE = re.compile(r"[`<>]")+_H2_RE = re.compile(r"(?m)^## ")+++# -- naming -------------------------------------------------------------------+++def naming_payload(system_map: SystemMap) -> str:+ """The structure a model names, as JSON for the overview prompt."""+ return json.dumps(+ {+ "nodes": [+ {"id": n.id, "tier": n.tier, "name": n.name, "facts": list(n.facts)}+ for n in system_map.nodes+ ],+ "edges": [+ {"from": e.source, "to": e.target, "evidence": e.basis} for e in system_map.edges+ ],+ },+ indent=1,+ )+++def split_names(content: str) -> tuple[str, dict[str, Any] | None]:+ """Strip the names block from a model reply and parse it.++ The last fenced object that names the map is taken wherever it sits, so+ prose written after it is kept and a JSON example in the prose is left+ alone. A reply cut off inside the block loses the partial block.+ """+ for match in reversed(list(_NAMES_FENCE_RE.finditer(content))):+ try:+ parsed = json.loads(match.group("body"))+ except ValueError:+ continue+ if isinstance(parsed, dict) and any(k in parsed for k in _NAMES_KEYS):+ body = (content[: match.start()].rstrip() + "\n" + content[match.end() :]).rstrip()+ return body + "\n", parsed+ truncated = _TRUNCATED_FENCE_RE.search(content)+ if truncated and any(f'"{k}"' in truncated.group(0) for k in _NAMES_KEYS):+ return content[: truncated.start()].rstrip() + "\n", None+ return content, None+++def apply_names(system_map: SystemMap, raw: dict[str, Any] | None) -> SystemMap:+ """Swap in model names, verbs and caption; unknown ids and pairs are ignored."""+ if not raw:+ return system_map+ by_id = {n.id: n for n in system_map.nodes}+ names: dict[str, tuple[str, str]] = {}+ for item in raw.get("nodes") or []:+ if not isinstance(item, dict) or item.get("id") not in by_id:+ continue+ label = _clean(item.get("label"), 4)+ if label:+ names[item["id"]] = (label, _clean(item.get("role"), 6) or by_id[item["id"]].role)+ pairs = {(e.source, e.target) for e in system_map.edges}+ verbs: dict[tuple[str, str], str] = {}+ for item in raw.get("edges") or []:+ if not isinstance(item, dict):+ continue+ pair = (item.get("from"), item.get("to"))+ verb = _clean(item.get("verb"), 3)+ if pair in pairs and verb:+ verbs[pair] = verb+ text = " ".join(_CAPTION_UNSAFE_RE.sub("", str(raw.get("caption") or "")).split())+ caption = " ".join(re.split(r"(?<=[.!?])\s+", text.lstrip("#").strip())[:2])+ return replace(system_map.with_names(names, verbs), caption=caption)+++def _clean(value: Any, max_words: int) -> str:+ text = _UNSAFE_CHARS_RE.sub("", str(value or "")).replace("&", "and")+ return " ".join(text.split()[:max_words])+++# -- compile ------------------------------------------------------------------+++def compile_mermaid(system_map: SystemMap) -> str:+ """Mermaid flowchart source for *system_map*: one subgraph per tier."""+ lines = ["flowchart TB"]+ for tier in TIERS:+ members = [n for n in system_map.nodes if n.tier == tier]+ if not members:+ continue+ lines.append(f' subgraph tier_{tier}["{_TIER_TITLES[tier]}"]')+ lines.append(" direction LR")+ lines += [f" {_node(n)}" for n in members]+ lines.append(" end")+ lines += [f" {_edge(e)}" for e in system_map.edges]+ for n in system_map.nodes:+ if n.page_id:+ href = "?page=" + quote(compute_page_id("module_page", n.page_id), safe="")+ lines.append(f' click {n.id} "{href}" "Open the {_clean(n.name, 6)} page"')+ used = {n.tier for n in system_map.nodes}+ lines += [f" classDef {t} {_TIER_STYLE[t]}" for t in TIERS if t in used]+ return "\n".join(lines)-if TYPE_CHECKING:- from .kg_context import KnowledgeGraphContext-# --- tuning (mirrors the reviewed lab settings) -------------------------------_OVERVIEW_MODULES_PER_LAYER = 2 # top-N modules shown per layer in the hero map-_OVERVIEW_MAX_EDGES = 16 # strongest inter-module edges kept in the hero-_OVERVIEW_MIN_EDGE = 12 # drop dependencies weaker than this from the hero-_LAYER_MODULES = 7 # module nodes inside one layer's own diagram-_LAYER_MAX_INTRA_EDGES = 16-_LAYER_BOUNDARY = 3 # deps-out / deps-in neighbour layers shown+def _node(node: MapNode) -> str:+ opener, closer = _SHAPES.get(node.tier, ('["', '"]'))+ label = f"<b>{_clean(node.name, 6)}</b>"+ if node.role:+ label += f"<br/><small>{_clean(node.role, 8)}</small>"+ return f"{node.id}{opener}{label}{closer}:::{node.tier}"-_MERMAID_BLOCK_RE = re.compile(r"```mermaid\b.*?```", re.DOTALL)+def _edge(edge: MapEdge) -> str:+ arrow = "==>" if edge.heavy else "-->"+ return f'{edge.source} {arrow}|"{_clean(edge.verb, 4)}"| {edge.target}'-def _slug(text: str) -> str:- """mermaid-safe node id."""- s = re.sub(r"[^A-Za-z0-9_]", "_", text)- if s and s[0].isdigit():- s = "n_" + s- return s or "n"+# -- the page section ----------------------------------------------------------def _short_module_label(module: dict) -> str:- """Short, readable label for a module node (last 1-2 path segments)."""- name = module.get("name") or ""- path = module.get("path") or ""- base = path or name- parts = [p for p in re.split(r"[\\/]", base) if p and p not in ("src", "packages")]- if not parts:- return name or path or module.get("id", "?")- return "/".join(parts[-2:]) if len(parts) >= 2 else parts[-1]+def system_map_section(system_map: SystemMap, repo_name: str) -> str:+ """The markdown section for *system_map*: heading, caption and diagram.-def _esc(text: str) -> str:- """Quote-safe mermaid label text."""- return text.replace('"', "'").strip()+ The caption says what the map leaves out, since a diagram that silently+ drops parts or arrows claims more coverage than it has.+ """+ caption = system_map.caption or f"The main parts of {repo_name} and how they connect."+ if system_map.omitted:+ n = system_map.omitted+ caption += f" {n} more part{'s are' if n != 1 else ' is'} not shown."+ if system_map.arrows_cut:+ caption += " Only the strongest links are drawn."+ body = compile_mermaid(system_map)+ return f"{HEADING}\n\n{caption}\n\n```mermaid\n{body}\n```\n"-class ArchitectureMermaidBuilder:- """Builds overview + per-layer mermaid from a KnowledgeGraphContext.+def embed_system_map(content: str, section: str | None) -> str:+ """Place *section* under the page's first section, replacing any earlier map.- Indexes the graph once on construction so a run can draw the overview and- every layer diagram without re-aggregating the edge list each time.+ Idempotent: a page that already carries a map (this one, or the older+ end-of-page architecture map) has it removed before the new one goes in."""+ content = _remove_section(_remove_section(content, HEADING), _LEGACY_HEADING)+ if not section:+ return content+ h2 = [m.start() for m in _H2_RE.finditer(content) if not _in_fence(content, m.start())]+ if len(h2) < 2:+ sep = "" if content.endswith("\n") else "\n"+ return f"{content}{sep}\n{section}"+ return f"{content[: h2[1]]}{section}\n{content[h2[1] :]}"- def __init__(self, kg_ctx: KnowledgeGraphContext) -> None:- self._ok = bool(getattr(kg_ctx, "available", False))- self._layers: list[dict] = kg_ctx.get_layers() if self._ok else []- modules: list[dict] = kg_ctx.get_modules() if self._ok else []-- self._mod_by_id: dict[str, dict] = {m["id"]: m for m in modules}- self._file_to_module: dict[str, str] = {}- for m in modules:- for nid in m.get("nodeIds", []):- if nid.startswith("file:"):- self._file_to_module[nid[5:]] = m["id"]-- # modules per layer, ranked by size (file count)- self._modules_by_layer: dict[str, list[dict]] = defaultdict(list)- for m in modules:- lid = m.get("layerId")- if lid:- self._modules_by_layer[lid].append(m)- for lid in self._modules_by_layer:- self._modules_by_layer[lid].sort(key=lambda m: len(m.get("nodeIds", [])), reverse=True)-- # file -> layer id- self._file_to_layer: dict[str, str] = {}- for ly in self._layers:- for nid in ly.get("nodeIds", []):- if nid.startswith("file:"):- self._file_to_layer[nid[5:]] = ly["id"]-- # aggregate imports once: module->module and layer->layer counts- self._mod_edges: dict[tuple[str, str], int] = defaultdict(int)- self._layer_edges: dict[tuple[str, str], int] = defaultdict(int)- if self._ok:- for sf, tf in kg_ctx.iter_import_edges():- ms, mt = self._file_to_module.get(sf), self._file_to_module.get(tf)- if ms and mt and ms != mt:- self._mod_edges[(ms, mt)] += 1- ls, lt = self._file_to_layer.get(sf), self._file_to_layer.get(tf)- if ls and lt and ls != lt:- self._layer_edges[(ls, lt)] += 1-- # -- overview -------------------------------------------------------------- def overview(self) -> str | None:- if not self._ok or not self._layers:- return None-- selected: dict[str, dict] = {}- layer_nodes: dict[str, list[str]] = {}- for ly in self._layers:- mods = self._modules_by_layer.get(ly["id"], [])[:_OVERVIEW_MODULES_PER_LAYER]- layer_nodes[ly["id"]] = [m["id"] for m in mods]- for m in mods:- selected[m["id"]] = m- if not selected:- return None-- kept = [- (ms, mt, c)- for (ms, mt), c in self._mod_edges.items()- if ms in selected and mt in selected and c >= _OVERVIEW_MIN_EDGE- ]- kept.sort(key=lambda x: x[2], reverse=True)- kept = kept[:_OVERVIEW_MAX_EDGES]-- lines = ["flowchart LR"]- for ly in self._layers:- node_ids = layer_nodes.get(ly["id"], [])- if not node_ids:- continue- lines.append(f' subgraph {_slug(ly["id"])}["{_esc(ly["name"])}"]')- lines.append(" direction TB")- for mid in node_ids:- lines.append(f' {_slug(mid)}["{_esc(_short_module_label(selected[mid]))}"]')- lines.append(" end")- lines.append("")- mod_layer = {mid: self._mod_by_id[mid].get("layerId") for mid in selected}- for ms, mt, c in kept:- cross = mod_layer.get(ms) != mod_layer.get(mt)- arrow = "-->" if cross else "-.->"- lines.append(f' {_slug(ms)} {arrow}|"{c}"| {_slug(mt)}')- return "\n".join(lines)-- # -- one layer ------------------------------------------------------------- def layer(self, layer: dict) -> str | None:- if not self._ok:- return None- lid = layer.get("id", "")- mods = self._modules_by_layer.get(lid, [])[:_LAYER_MODULES]- if not mods:- return None- sel = {m["id"]: m for m in mods}-- intra = [- (ms, mt, c)- for (ms, mt), c in self._mod_edges.items()- if ms in sel and mt in sel and ms != mt- ]- intra.sort(key=lambda x: x[2], reverse=True)- intra = intra[:_LAYER_MAX_INTRA_EDGES]-- name_by_id = {L["id"]: L["name"] for L in self._layers}- deps_out = sorted(- ((name_by_id.get(lt, lt), c) for (ls, lt), c in self._layer_edges.items() if ls == lid),- key=lambda x: x[1],- reverse=True,- )[:_LAYER_BOUNDARY]- deps_in = sorted(- ((name_by_id.get(ls, ls), c) for (ls, lt), c in self._layer_edges.items() if lt == lid),- key=lambda x: x[1],- reverse=True,- )[:_LAYER_BOUNDARY]-- lines = ["flowchart TD"]- lines.append(f' subgraph core["{_esc(layer.get("name", "Layer"))} layer"]')- lines.append(" direction TB")- for mid in sel:- lines.append(f' {_slug(mid)}["{_esc(_short_module_label(sel[mid]))}"]')- for ms, mt, c in intra:- lines.append(f' {_slug(ms)} -->|"{c}"| {_slug(mt)}')- lines.append(" end")- # Boundary layers as stadium nodes (shape, not color, so it stays- # theme-safe): imported-by above, depends-on below.- if deps_in:- lines.append("")- for i, (name, c) in enumerate(deps_in):- nid = f"in_{i}"- lines.append(f' {nid}(["{_esc(name)}"])')- lines.append(f' {nid} ==>|"{c}"| core')- if deps_out:- lines.append("")- for i, (name, c) in enumerate(deps_out):- nid = f"out_{i}"- lines.append(f' {nid}(["{_esc(name)}"])')- lines.append(f' core -.->|"{c}"| {nid}')- return "\n".join(lines)---def build_overview_mermaid(kg_ctx: KnowledgeGraphContext) -> str | None:- """Overview architecture map, or None when the KG can't support one."""- return ArchitectureMermaidBuilder(kg_ctx).overview()---def build_layer_mermaid(kg_ctx: KnowledgeGraphContext, layer: dict) -> str | None:- """Per-layer diagram, or None when the layer has no curated modules."""- return ArchitectureMermaidBuilder(kg_ctx).layer(layer)---def embed_mermaid(content: str, mermaid: str, *, heading: str) -> str:- """Return *content* with *mermaid* embedded, idempotently.-- If the content already has a fenced mermaid block (the LLM's own, or one we- embedded on a prior run), its body is replaced. Otherwise the diagram is- appended under *heading*. Re-running on already-embedded content is a no-op,- so reused/cached pages stay stable across docs updates.- """- if not mermaid:++def _remove_section(content: str, heading: str) -> str:+ match = re.search(rf"(?m)^{re.escape(heading)}[ \t]*$", content)+ if match is None:return content- block = f"```mermaid\n{mermaid}\n```"- if _MERMAID_BLOCK_RE.search(content):- # function replacement avoids backslash/group interpretation in mermaid- return _MERMAID_BLOCK_RE.sub(lambda _m: block, content, count=1)- sep = "" if content.endswith("\n") else "\n"- return f"{content}{sep}\n{heading}\n\n{block}\n"+ end = len(content)+ for nxt in _H2_RE.finditer(content, match.end()):+ if not _in_fence(content, nxt.start()):+ end = nxt.start()+ break+ return content[: match.start()] + content[end:]+++def _in_fence(content: str, pos: int) -> bool:+ return content.count("```", 0, pos) % 2 == 1diff --git a/packages/core/src/repowise/core/generation/context/__init__.py b/packages/core/src/repowise/core/generation/context/__init__.pyindex 04ec90785..62c72b952 100644--- a/packages/core/src/repowise/core/generation/context/__init__.py+++ b/packages/core/src/repowise/core/generation/context/__init__.py@@ -11,7 +11,6 @@ from __future__ import annotationsfrom .assembler import ContextAssembler, _symbol_to_dictfrom .contexts import (ApiContractContext,- ArchitectureDiagramContext,FilePageContext,InfraPageContext,ModulePageContext,@@ -23,7 +22,6 @@ from .contexts import (__all__ = ["ApiContractContext",- "ArchitectureDiagramContext","ContextAssembler","FilePageContext","InfraPageContext",diff --git a/packages/core/src/repowise/core/generation/context/assembler.py b/packages/core/src/repowise/core/generation/context/assembler.pyindex 12d51de95..bb504fc5d 100644--- a/packages/core/src/repowise/core/generation/context/assembler.py+++ b/packages/core/src/repowise/core/generation/context/assembler.py@@ -29,7 +29,6 @@ from ..entry_points import orientation_entry_points, rank_entry_point_pathsfrom ..models import GenerationConfigfrom .contexts import (ApiContractContext,- ArchitectureDiagramContext,FilePageContext,InfraPageContext,ModulePageContext,@@ -976,70 +975,6 @@ class ContextAssembler:prose_digest=prose_digest,)- # ------------------------------------------------------------------- # Architecture diagram- # -------------------------------------------------------------------- def assemble_architecture_diagram(- self,- graph: Any, # nx.DiGraph- pagerank: dict[str, float],- community: dict[str, int],- sccs: list[Any], # list[frozenset[str]]- repo_name: str,- ) -> ArchitectureDiagramContext:- """Assemble context for the architecture_diagram template."""- max_diagram_nodes = 50- max_diagram_edges = 200-- # Top-N nodes by PageRank (exclude external nodes). Path breaks the- # tie, otherwise which 50 nodes make the cut changes between runs and- # the diagram is not comparable to the one it replaces.- top_nodes = set(- p- for p, _ in sorted(pagerank.items(), key=lambda x: (-x[1], str(x[0])))[- :max_diagram_nodes- ]- if not is_external(str(p))- )- nodes = sorted(top_nodes)-- # Only edges between selected nodes. Ranked by the endpoints' PageRank- # rather than alphabetically: the cap bites on dense repos, and a plain- # sort would fill all 200 slots from whichever package sorts first and- # draw nothing from the rest of the graph. Path breaks the tie.- edges = sorted(- ((src, dst) for src, dst in graph.edges() if src in top_nodes and dst in top_nodes),- key=lambda e: (- -pagerank.get(e[0], 0.0),- -pagerank.get(e[1], 0.0),- str(e[0]),- str(e[1]),- ),- )[:max_diagram_edges]-- # Community → members mapping (top-10 communities, cap members to 5)- raw_communities: dict[int, list[str]] = {}- for path, cid in community.items():- if not is_external(path):- raw_communities.setdefault(cid, []).append(path)- comm_sorted = sorted(- ((cid, sorted(members)) for cid, members in raw_communities.items()),- key=lambda x: (-len(x[1]), x[0]),- )[:10]- communities: dict[int, list[str]] = {cid: members[:5] for cid, members in comm_sorted}-- # SCC groups (only non-singleton)- scc_groups = [sorted(scc) for scc in sccs if len(scc) > 1]-- return ArchitectureDiagramContext(- repo_name=repo_name,- nodes=nodes,- edges=edges,- communities=communities,- scc_groups=scc_groups,- )-# ------------------------------------------------------------------# API contract# ------------------------------------------------------------------diff --git a/packages/core/src/repowise/core/generation/context/contexts.py b/packages/core/src/repowise/core/generation/context/contexts.pyindex c4633a416..3b9a7500a 100644--- a/packages/core/src/repowise/core/generation/context/contexts.py+++ b/packages/core/src/repowise/core/generation/context/contexts.py@@ -212,15 +212,6 @@ class RepoOverviewContext:package_stats: list[dict] = field(default_factory=list)-@dataclass-class ArchitectureDiagramContext:- repo_name: str- nodes: list[str]- edges: list[tuple[str, str]]- communities: dict[int, list[str]]- scc_groups: list[list[str]]--@dataclassclass ApiContractContext:file_path: strdiff --git a/packages/core/src/repowise/core/generation/context_assembler.py b/packages/core/src/repowise/core/generation/context_assembler.pyindex 1d70ae200..dd174c73e 100644--- a/packages/core/src/repowise/core/generation/context_assembler.py+++ b/packages/core/src/repowise/core/generation/context_assembler.py@@ -14,7 +14,6 @@ from __future__ import annotationsfrom .context import (ApiContractContext,- ArchitectureDiagramContext,ContextAssembler,FilePageContext,InfraPageContext,@@ -28,7 +27,6 @@ from .context import (__all__ = ["ApiContractContext",- "ArchitectureDiagramContext","ContextAssembler","FilePageContext","InfraPageContext",diff --git a/packages/core/src/repowise/core/generation/kg_context.py b/packages/core/src/repowise/core/generation/kg_context.pyindex 7ca59792e..5e2eef9bb 100644--- a/packages/core/src/repowise/core/generation/kg_context.py+++ b/packages/core/src/repowise/core/generation/kg_context.py@@ -8,7 +8,6 @@ neighbor list, and optional tour step reference.from __future__ import annotationsimport json-from collections.abc import Iteratorfrom dataclasses import dataclass, fieldfrom pathlib import Path@@ -191,23 +190,9 @@ class KnowledgeGraphContext:"""Curated wiki modules from the KG artifact (empty when uncurated)."""return self._modules if self._loaded else []- def iter_import_edges(self) -> Iterator[tuple[str, str]]:- """Yield (source_file, target_file) for every file->file imports edge.-- Feeds the deterministic architecture-diagram builder, which aggregates- these into module- and layer-level dependency counts. Empty when the KG- isn't loaded.- """- if not self._loaded:- return- for edges in self._edges_by_source.values():- for e in edges:- if e.get("type") != "imports":- continue- src = e.get("source", "")- tgt = e.get("target", "")- if src.startswith("file:") and tgt.startswith("file:"):- yield src[5:], tgt[5:]+ def get_entry_points(self) -> list[str]:+ """The curated entry-point paths, ranked (empty when not loaded)."""+ return list(self._project.get("entry_points") or []) if self._loaded else []def get_tour(self) -> list[dict]:return self._tour if self._loaded else []diff --git a/packages/core/src/repowise/core/generation/onboarding/subkinds/key_concepts.py b/packages/core/src/repowise/core/generation/onboarding/subkinds/key_concepts.pyindex ea6821a1d..cdf9b5cb2 100644--- a/packages/core/src/repowise/core/generation/onboarding/subkinds/key_concepts.py+++ b/packages/core/src/repowise/core/generation/onboarding/subkinds/key_concepts.py@@ -93,9 +93,8 @@ class ConceptSymbol:# harmless only while `imports` was in the set and would have mislabelled a Go# `method_implements` edge the moment it was not.#-# Ceiling: c4_builder/labels.py `_EDGE_VERB` answers a similar question over-# all 14 types, but it lives in packages/server and core cannot import it, and-# it disagrees here on `extends` ("inherits from"), tuned for a C4 arrow+# Ceiling: analysis/c4/labels.py `_EDGE_VERB` answers a similar question over+# all 14 types, but it disagrees here on `extends` ("inherits from"), tuned for a C4 arrow# rather than for prose. If a third copy appears, reconcile the wording first,# then lift one map into core; a straight merge would silently reword these# prompts.diff --git a/packages/core/src/repowise/core/generation/overview_tables.py b/packages/core/src/repowise/core/generation/overview_tables.pyindex 8f4dc0abd..2bbd2d266 100644--- a/packages/core/src/repowise/core/generation/overview_tables.py+++ b/packages/core/src/repowise/core/generation/overview_tables.py@@ -5,7 +5,7 @@ resampled on every render: two calls with the same prompt, the same model andthe same temperature produced pages that disagreed on their row count and onwhich paths they cited. Facts the run already holds — which packages exist,where they are, how big they are — are built here instead and embedded after-the page comes back, the same way the architecture map already is.+the page comes back, the same way the system map is.That makes them identical on the model-written page and on the structure-onlypage, stable across updates that changed no code, and assertable in a test.diff --git a/packages/core/src/repowise/core/generation/page_generator/README.md b/packages/core/src/repowise/core/generation/page_generator/README.mdindex e4cc80bf5..16f73b25d 100644--- a/packages/core/src/repowise/core/generation/page_generator/README.md+++ b/packages/core/src/repowise/core/generation/page_generator/README.md@@ -16,7 +16,7 @@ generates them in ordered, concurrency-bounded levels:| 2 | `file_page` | topo-ordered; tier-1 LLM or tier-2 template || 3 | `scc_page` | circular-dependency cycles || 4 | `module_page` | community / directory groups |-| 6/7/8 | `repo_overview` + `architecture_diagram`, `infra_page`, `onboarding` | merged batch |+| 6/7/8 | `repo_overview`, `infra_page`, `onboarding` | merged batch |### Tiered doc generationdiff --git a/packages/core/src/repowise/core/generation/page_generator/core.py b/packages/core/src/repowise/core/generation/page_generator/core.pyindex 186cbf86e..10d34e8ba 100644--- a/packages/core/src/repowise/core/generation/page_generator/core.py+++ b/packages/core/src/repowise/core/generation/page_generator/core.py@@ -14,8 +14,8 @@ The level-by-level orchestration of ``generate_all`` lives infrom __future__ import annotationsimport hashlib-from collections.abc import Callable, Sequence-from dataclasses import dataclass, replace+from collections.abc import Callable, Mapping, Sequence+from dataclasses import dataclass, field, replacefrom pathlib import Pathfrom typing import TYPE_CHECKING, Any@@ -136,6 +136,8 @@ class PriorPage:output_tokens: int = 0cached_tokens: int = 0content_hash: str = ""+ #: The stored page's metadata, for facts a reused page must carry forward.+ metadata: Mapping[str, Any] = field(default_factory=dict)class PageGenerator(PerTypeGenerationMixin, StructuralRenderMixin):diff --git a/packages/core/src/repowise/core/generation/page_generator/levels.py b/packages/core/src/repowise/core/generation/page_generator/levels.pyindex f9ce3b83f..e03ebdbe5 100644--- a/packages/core/src/repowise/core/generation/page_generator/levels.py+++ b/packages/core/src/repowise/core/generation/page_generator/levels.py@@ -414,28 +414,22 @@ async def _module_corroboration(run: _GenerationRun) -> list[str]:async def build_level6_coros(run: _GenerationRun) -> list[tuple[str, Any]]:"""Level 6 (repo_overview).- The overview carries the architecture map. That map used to sit on a page- of its own, which described the same repository at the same altitude in the- same words — the two shared roughly a quarter of their vocabulary, so a- reader meeting both read one thing twice. The diagram is what that page- uniquely had, so it moved here and the page retired; its id redirects.+ The overview carries the system map, built from the run's structure. It+ used to sit on a page of its own that described the same repository at the+ same altitude; that page retired and its id redirects here."""- from ..architecture_mermaid import build_overview_mermaidfrom ..context.readme_digest import readme_digestfrom ..overview_tables import select_capabilities+ from .system_map import build_run_system_mapgen = run.gen- overview_mermaid = build_overview_mermaid(run.kg_ctx)- if not overview_mermaid:- # The retired page drew its own diagram when the graph could not supply- # one, so this used to degrade to a worse map rather than to none. Now- # the wiki simply ships without one, which is worth knowing about.- log.warning(- "generation.overview_architecture_map_empty",- repo_name=run.repo_name,- )coros: list[tuple[str, Any]] = []if run._emit(compute_page_id("repo_overview", run.repo_name)):+ system_map = build_run_system_map(run)+ if system_map is None:+ # Said out loud: a front page that quietly loses its diagram is the+ # failure shape worth hearing about.+ log.warning("generation.overview_system_map_empty", repo_name=run.repo_name)# What the repository calls its own capabilities, in its own words.# Mined once per run and shared with level 8. A term reaches the page# only when the structural side names it too, so the front page never@@ -469,7 +463,7 @@ async def build_level6_coros(run: _GenerationRun) -> list[tuple[str, Any]]:# Repo-wide scope is right here; only the choice of ten was# list position.decision_records=rank_decisions(run.decisions_all)[:10],- overview_mermaid=overview_mermaid,+ system_map=system_map,source_map=run.source_map,# Per-package file counts come from the files this run# actually parsed, not from the package manifests, so adiff --git a/packages/core/src/repowise/core/generation/page_generator/pertype.py b/packages/core/src/repowise/core/generation/page_generator/pertype.pyindex 2c348fd73..0a44cfa33 100644--- a/packages/core/src/repowise/core/generation/page_generator/pertype.py+++ b/packages/core/src/repowise/core/generation/page_generator/pertype.py@@ -20,7 +20,14 @@ from repowise.core.ingestion.models import ParsedFile, RepoStructurefrom .. import onboarding as _onboardingfrom ..agent_digest import MODULE_SIGNALS_KEY, module_signals, split_questions-from ..architecture_mermaid import embed_mermaid+from ..architecture_map import SystemMap+from ..architecture_mermaid import (+ apply_names,+ embed_system_map,+ naming_payload,+ split_names,+ system_map_section,+)from ..context.assembler import build_concept_indexfrom ..context_assembler import FilePageContextfrom ..models import (@@ -28,6 +35,7 @@ from ..models import (STUB_FALLBACK_ERROR,STUB_PAGE_CONFIDENCE,GeneratedPage,+ compute_page_id,compute_source_hash,)from ..overview_tables import (@@ -77,24 +85,37 @@ def _stub_fallback(page: GeneratedPage, page_type: str, exc: Exception) -> Generreturn page-def _with_architecture_map(page: GeneratedPage, overview_mermaid: str | None) -> GeneratedPage:- """Embed the KG-derived architecture map into an already-built page.+#: Page metadata key holding the model's names for the system map, so a reused+#: or fallback page recompiles the current structure without another call.+SYSTEM_MAP_NAMES = "system_map_names"+- The overview is where the map lives, so the stub paths carry it too — a- provider outage should cost the prose around the diagram, not the diagram.- Embedding is idempotent, so calling this on a page that already has one is- safe.+def _with_system_map(+ page: GeneratedPage,+ system_map: SystemMap | None,+ repo_name: str,+ names: dict | None = None,+) -> GeneratedPage:+ """Compile the system map from the current structure into a built page.++ Every path carries it, the stub and the outage fallback too: a provider+ outage should cost the prose around the diagram, not the diagram. The map+ is recompiled on every render, so a structure or compiler change reaches a+ reused page; the names are the only model output it keeps."""- if not overview_mermaid:- return page- page.content = embed_mermaid(page.content, overview_mermaid, heading="## Architecture map")+ section = None+ if system_map:+ section = system_map_section(apply_names(system_map, names), repo_name)+ if names:+ page.metadata[SYSTEM_MAP_NAMES] = names+ page.content = embed_system_map(page.content, section)return pagedef _with_package_table(page: GeneratedPage, package_stats: list[dict]) -> GeneratedPage:"""Embed the package table into an already-built page.- Same reasoning as the architecture map, and the same three paths: the model+ Same reasoning as the system map, and the same three paths: the modelpage, the deterministic page and the provider-outage fallback all carry it,because which packages exist is a fact the run already holds. Writing itthrough the model instead meant it was resampled on every render — two@@ -409,7 +430,7 @@ class PerTypeGenerationMixin:repo_name: str | None = None,external_systems: list[dict] | None = None,decision_records: list[dict] | None = None,- overview_mermaid: str | None = None,+ system_map: SystemMap | None = None,source_map: dict[str, bytes] | None = None,parsed_files: list[ParsedFile] | None = None,capabilities: Sequence[Capability] = (),@@ -445,17 +466,27 @@ class PerTypeGenerationMixin:}if not repo_name:repo_name = getattr(repo_structure, "name", None) or "repo"+ prior = self._prior_pages.get(compute_page_id("repo_overview", repo_name))+ prior_names = dict(prior.metadata).get(SYSTEM_MAP_NAMES) if prior else Noneif self._config.deterministic:stub = self._stub_repo_overview(ctx, repo_name, f"Repository Overview: {repo_name}", repo_git_summary)- page = _with_architecture_map(+ # Stored names survive a keyless render, for the next keyed one to reuse.+ page = _with_system_map(_with_package_table(_with_capability_table(stub, capabilities), ctx.package_stats),- overview_mermaid,+ system_map,+ repo_name,+ prior_names,)selection = self._disabled_source_evidence("repo_overview", "deterministic_generation")return self._attach_source_evidence(page, "repo_overview", selection)- user_prompt = self._render("repo_overview.j2", ctx=ctx, repo_git_summary=repo_git_summary)+ user_prompt = self._render(+ "repo_overview.j2",+ ctx=ctx,+ repo_git_summary=repo_git_summary,+ system_map_payload=naming_payload(system_map) if system_map else "",+ )user_prompt, evidence = self._append_source_evidence(user_prompt, "repo_overview", source_map or {})@@ -467,16 +498,23 @@ class PerTypeGenerationMixin:stub = self._stub_repo_overview(ctx, repo_name, f"Repository Overview: {repo_name}", repo_git_summary)- page = _with_architecture_map(+ page = _with_system_map(_with_package_table(_with_capability_table(_stub_fallback(stub, "repo_overview", exc), capabilities),ctx.package_stats,),- overview_mermaid,+ system_map,+ repo_name,+ prior_names,)return self._attach_source_evidence(page, "repo_overview", evidence)+ # A fresh reply carries the names block; a reused one takes the names+ # the stored page kept.+ prose, names = split_names(response.content)+ response = replace(response, content=prose)+ names = names or prior_namesoverview_grounding_evidence: dict[str, str] = {}for item in evidence.included:overview_grounding_evidence[item.path] = "\n".join(@@ -500,11 +538,9 @@ class PerTypeGenerationMixin:count=len(ungrounded),tokens=ungrounded[:20],)- # The overview carries its own enumerable facts: the package table and- # the KG-derived architecture map are built from the run, not drawn by- # the model, and both embeds are idempotent so a reused page picks them- # up too. Appended in reading order, so what the repository does lands- # above what it is made of, and both above the diagram.+ # The overview carries its own enumerable facts: the tables and the+ # system map are built from the run, not by the model. The table embeds+ # are idempotent, so a reused page picks them up too.# Unconditional: content reused from an earlier run keeps that run's# rows, and an empty selection has to remove them.response = replace(@@ -518,13 +554,6 @@ class PerTypeGenerationMixin:response.content, build_package_table(ctx.package_stats)),)- if overview_mermaid:- response = replace(- response,- content=embed_mermaid(- response.content, overview_mermaid, heading="## Architecture map"- ),- )page = self._build_generated_page("repo_overview",repo_name,@@ -533,56 +562,9 @@ class PerTypeGenerationMixin:compute_source_hash(user_prompt),GENERATION_LEVELS["repo_overview"],)+ page = _with_system_map(page, system_map, repo_name, names)return self._attach_source_evidence(page, "repo_overview", evidence)- async def generate_architecture_diagram(- self,- graph: Any,- pagerank: dict[str, float],- community: dict[str, int],- sccs: list[Any],- repo_name: str,- overview_mermaid: str | None = None,- ) -> GeneratedPage:- ctx = self._assembler.assemble_architecture_diagram(- graph, pagerank, community, sccs, repo_name- )- if self._config.deterministic:- return self._stub_architecture_diagram(- ctx, repo_name, f"Architecture Diagram: {repo_name}", overview_mermaid- )- user_prompt = self._render("architecture_diagram.j2", ctx=ctx)- try:- response = await self._call_provider(- "architecture_diagram", user_prompt, str(uuid.uuid4()), target_path=repo_name- )- except Exception as exc:- # The stub embeds the same KG-derived map the model path overwrites- # the model's diagram with, so the fallback keeps the diagram and- # loses only the prose around it.- stub = self._stub_architecture_diagram(- ctx, repo_name, f"Architecture Diagram: {repo_name}", overview_mermaid- )- return _stub_fallback(stub, "architecture_diagram", exc)- # Swap the LLM's free-form diagram for the deterministic KG-derived map- # (idempotent, applies to fresh and reused content). Falls back to the- # LLM's own mermaid when the KG can't produce one.- if overview_mermaid:- response = replace(- response,- content=embed_mermaid(- response.content, overview_mermaid, heading="## Architecture map"- ),- )- return self._build_generated_page(- "architecture_diagram",- repo_name,- f"Architecture Diagram: {repo_name}",- response,- compute_source_hash(user_prompt),- GENERATION_LEVELS["architecture_diagram"],- )-async def generate_api_contract(self,parsed: ParsedFile,@@ -701,7 +683,7 @@ class PerTypeGenerationMixin:@staticmethoddef _tag_promoted_pages(pages: list[GeneratedPage]) -> None:- """Tag repo_overview / architecture_diagram pages with their slot.+ """Tag promoted pages (the repo overview) with their onboarding slot.Mutates each matching page's ``metadata["onboarding_slot"]`` so theUI groups them into the Onboarding folder without changing theirdiff --git a/packages/core/src/repowise/core/generation/page_generator/prompts.py b/packages/core/src/repowise/core/generation/page_generator/prompts.pyindex 7550131c6..cb4028652 100644--- a/packages/core/src/repowise/core/generation/page_generator/prompts.py+++ b/packages/core/src/repowise/core/generation/page_generator/prompts.py@@ -14,7 +14,7 @@ from ..languages import SUPPORTED_LANGUAGES # noqa: F401# ---------------------------------------------------------------------------# System prompts, one per page type that a model writes.#-# Only four are left. The page types whose facts a parser knows exactly are+# Only three are left. The page types whose facts a parser knows exactly are# rendered from structure and never reach a provider, so a system prompt for# one of them would be a string nothing sends.# ---------------------------------------------------------------------------@@ -84,12 +84,6 @@ SYSTEM_PROMPTS: dict[str, str] = {"word budget the user prompt states. The enumerable facts are inserted ""after you write, from the code as indexed, so do not tabulate them."),- "architecture_diagram": (- "You are repowise, an expert technical documentation generator. "- "Generate an architecture overview with a Mermaid diagram. "- "You MUST include a fenced mermaid block with graph TD showing key dependencies. "- "Output markdown only."- ),"onboarding": ("You are repowise, an expert technical documentation generator producing ""a single page in a curated Onboarding collection that a new contributor "diff --git a/packages/core/src/repowise/core/generation/page_generator/structural.py b/packages/core/src/repowise/core/generation/page_generator/structural.pyindex 0e1ab984f..b13e86b37 100644--- a/packages/core/src/repowise/core/generation/page_generator/structural.py+++ b/packages/core/src/repowise/core/generation/page_generator/structural.py@@ -9,11 +9,10 @@ history. A model adds nothing to that and introduces staleness, so these haveone renderer and no model path at all. Their templates sit at``templates/<name>.j2``.-**Keyless stubs.** ``module_page``, ``repo_overview``, ``architecture_diagram``-and ``onboarding`` exist to synthesise, which is exactly what a template-cannot do. They keep a model path; what lives here is the honest thin version-a user without an API key gets, which the same page upgrades away from once a-key is present. Their templates sit at ``templates/stub/<name>.j2``.+**Keyless stubs.** ``module_page``, ``repo_overview`` and ``onboarding``+exist to synthesise, which is exactly what a template cannot do. They keep a+model path; what lives here is the honest thin version a user without an API+key gets, which the same page upgrades away from once a key is present. Their templates sit at ``templates/stub/<name>.j2``.Everything either renderer emits is derived from the parsed AST, the importgraph, git history or the knowledge graph, so it is factual by construction@@ -951,20 +950,6 @@ class StructuralRenderMixin:repo_git_summary=repo_git_summary,)- def _stub_architecture_diagram(- self, ctx: Any, repo_name: str, title: str, overview_mermaid: str | None- ) -> GeneratedPage:- return self._stub_page(- page_type="architecture_diagram",- target_path=repo_name,- title=title,- template="architecture_diagram.j2",- ctx=ctx,- # Structural on the model path too, where it overwrites whatever- # diagram the model drew. Here it is simply the diagram.- overview_mermaid=overview_mermaid or "",- )-def _stub_onboarding_page(self, spec: Any, ctx: Any, target_path: str) -> GeneratedPage:page = self._stub_page(page_type="onboarding",diff --git a/packages/core/src/repowise/core/generation/page_generator/system_map.py b/packages/core/src/repowise/core/generation/page_generator/system_map.pynew file mode 100644index 000000000..fb0bee1f6--- /dev/null+++ b/packages/core/src/repowise/core/generation/page_generator/system_map.py@@ -0,0 +1,115 @@+"""Gather a generation run's facts into the overview's system map.++Everything here is read from what the run already holds: the parsed files and+their graph, the declared dependencies, the curated entry points and the+module pages. HTTP links are the one thing computed fresh, by running the+contract extractors over the in-memory sources, since a single-repository+index does not store them.+"""++from __future__ import annotations++from dataclasses import replace+from pathlib import Path+from typing import Any++import structlog++from repowise.core.analysis.c4.containers import (+ assign_containers,+ container_name,+ is_manifest,+ manifest_roots,+)+from repowise.core.analysis.external_systems import build_declaration_links+from repowise.core.ingestion.languages.registry import REGISTRY+from repowise.core.support_paths import file_population, is_support_path++from ..architecture_map import SystemMap, build_system_map++log = structlog.get_logger(__name__)++_CODE_LANGUAGES = REGISTRY.code_languages()+++def build_run_system_map(run: Any) -> SystemMap | None:+ """The system map for *run*, or ``None`` when the structure cannot support one."""+ files = {+ pf.file_info.path: pf.file_info.language+ for pf in run.parsed_files+ if pf.file_info.language in _CODE_LANGUAGES+ and file_population(pf.file_info.path, is_test=pf.file_info.is_test) == "production"+ and not is_support_path(pf.file_info.path)+ }+ if not files:+ return None+ graph = run.graph_builder.graph()+ manifests = {s.get("declared_in") for s in run.external_systems if s.get("declared_in")}+ manifests |= {pf.file_info.path for pf in run.parsed_files if is_manifest(pf.file_info.path)}+ repo_root = Path(run.repo_path) if run.repo_path else None+ links, routes = _http_links(run, files, manifests)+ package_names = {+ root: name+ for root, found in manifest_roots(manifests).items()+ if (name := container_name(repo_root, found))+ }+ return build_system_map(+ files,+ ((s, t, d.get("edge_type", "")) for s, t, d in graph.edges(data=True)),+ repo_name=run.repo_name,+ weights=run.pagerank,+ manifests=manifests,+ package_names=package_names,+ entry_points=run.kg_ctx.get_entry_points(),+ # A file that defines nothing only re-exports or launches.+ barrels={pf.file_info.path for pf in run.parsed_files if not pf.symbols} & files.keys(),+ route_files=routes,+ stores=_database_clients(run.external_systems, graph),+ http_links=links,+ pages=[+ (mg.key, frozenset(mg.context_paths or mg.file_paths))+ for mg in run.sel_module_groups+ ],+ )+++def _database_clients(external_systems: list[dict], graph: Any) -> dict[str, str]:+ """``external:*`` node id -> database client name, for runtime dependencies."""+ declarations = [+ s+ for s in external_systems+ if s.get("io_kind") == "db"+ and not s.get("is_dev_dep")+ and not is_support_path(s.get("declared_in") or "")+ ]+ if not declarations:+ return {}+ nodes = [{"node_id": n} for n in graph.nodes if str(n).startswith("external:")]+ return {link.node_id: link.name for link in build_declaration_links(declarations, nodes)}+++def _http_links(+ run: Any, files: dict[str, str], manifests: set[str]+) -> tuple[list[tuple[str, str]], set[str]]:+ """``(calling file, route file)`` pairs between containers, and the route files."""+ from repowise.core.workspace.extractors import HttpExtractor+ from repowise.core.workspace.matching import match_contracts++ owner, _ = assign_containers(manifests, files)+ suffixes = HttpExtractor.source_extensions()+ sources = [+ (path, suffix, run.source_map[path].decode("utf-8", errors="replace"))+ for path in files+ if path in run.source_map and (suffix := Path(path).suffix) in suffixes+ ]+ try:+ contracts = HttpExtractor().extract(Path(run.repo_path or "."), files=sources)+ except Exception as exc:+ # The map loses its runtime arrows, not its structure.+ log.warning("system_map.http_extraction_failed", error=str(exc))+ return [], set()+ # One service per container, so a call between containers counts as a link.+ scoped = [replace(c, service=owner.get(c.file_path)) for c in contracts]+ links = {(link.consumer_file, link.provider_file) for link in match_contracts(scoped)}+ routes = {c.file_path for c in contracts if c.role == "provider" and c.contract_type == "http"}+ return sorted(links), routesdiff --git a/packages/core/src/repowise/core/generation/templates/architecture_diagram.j2 b/packages/core/src/repowise/core/generation/templates/architecture_diagram.j2deleted file mode 100644index b98d4b048..000000000--- a/packages/core/src/repowise/core/generation/templates/architecture_diagram.j2+++ /dev/null@@ -1,31 +0,0 @@-You are generating an architecture diagram for the repository `{{ ctx.repo_name }}`.--## Repository Architecture--**Nodes** ({{ ctx.nodes | length }} files):-{% for node in ctx.nodes %}-- `{{ node }}`-{% endfor %}--**Edges** ({{ ctx.edges | length }} dependencies):-{% for src, dst in ctx.edges %}-- `{{ src }}` → `{{ dst }}`-{% endfor %}--**Communities** ({{ ctx.communities | length }} detected):-{% for community_id, members in ctx.communities.items() %}-- Community {{ community_id }}: {{ members | join(", ") }}-{% endfor %}--**Strongly Connected Components** (cycles):-{% for scc in ctx.scc_groups %}-- {{ scc | join(", ") }}-{% endfor %}--Note: Showing top {{ ctx.nodes | length }} files by PageRank.-Use module pages for full per-file coverage.--You MUST include a fenced mermaid block with graph TD showing the key dependencies.-The mermaid diagram should use short node labels for readability.--Generate a complete wiki page following the sections above.diff --git a/packages/core/src/repowise/core/generation/templates/repo_overview.j2 b/packages/core/src/repowise/core/generation/templates/repo_overview.j2index 87845ccca..3e98d4e64 100644--- a/packages/core/src/repowise/core/generation/templates/repo_overview.j2+++ b/packages/core/src/repowise/core/generation/templates/repo_overview.j2@@ -98,13 +98,25 @@ Key entry points and their execution paths:{% endfor %}{% endif %}+{% if system_map_payload is defined and system_map_payload %}+### System map to name+A diagram of the parts below is inserted under the Project Summary after you+write. Which boxes and arrows it shows is fixed; you only name them, from the+facts given and the project's own vocabulary.++```json+{{ system_map_payload }}+```+{% endif %}+---## What to write-Exactly these three sections, in this order, and nothing else. Two tables and a-diagram are inserted after you write — what this repository calls its own-capabilities, the package table, and the architecture map — so leave room for+Exactly these three sections, in this order, and nothing else{% if system_map_payload is defined and system_map_payload %} but the+closing `json` block described at the end{% endif %}. Two tables and a+diagram are inserted after you write: what this repository calls its own+capabilities, the package table, and the system map. Leave room forthem by not writing them yourself.**## Project Summary** — two short paragraphs. What this is, what it is for, and who@@ -140,3 +152,19 @@ LENGTH: aim for 650–850 words of prose across the whole page when the suppliedevidence supports that depth. Say each thing once: the reader has per-modulepages for implementation detail, but this page must still leave them able toexplain the product, its evidence layers, its lifecycle, and its main surfaces.++{% if system_map_payload is defined and system_map_payload %}+REQUIRED: after the three sections, end the reply with one fenced `json` block naming the+system map, and nothing after it:++```json+{"caption": "one sentence on how work flows through these parts",+ "nodes": [{"id": "<node id>", "label": "2-4 plain words", "role": "what it does, at most 6 words"}],+ "edges": [{"from": "<node id>", "to": "<node id>", "verb": "1-3 word verb, e.g. starts, asks, reads and writes"}]}+```++Give every node and every edge listed above exactly one entry, using its id.+A label names what the part is to a newcomer, not its directory. A verb says+what the source does to the target at run time.++{% endif %}diff --git a/packages/core/src/repowise/core/generation/templates/stub/architecture_diagram.j2 b/packages/core/src/repowise/core/generation/templates/stub/architecture_diagram.j2deleted file mode 100644index 3bbab8c39..000000000--- a/packages/core/src/repowise/core/generation/templates/stub/architecture_diagram.j2+++ /dev/null@@ -1,57 +0,0 @@-{#- Deterministic (no-LLM) architecture map.-- The strongest of the deterministic pages: the mermaid diagram is built- from the knowledge graph on BOTH paths, and on the LLM path it replaces- whatever the model drew. So the only thing lost here is the prose around- an already-structural artifact.-- ``overview_mermaid`` is the KG-derived diagram. When the KG can't produce- one, fall back to drawing the top-ranked nodes and edges the context- already carries.--#}-# Architecture Diagram: {{ ctx.repo_name }}--## Architecture map--{% if overview_mermaid %}-```mermaid-{{ overview_mermaid }}-```-{% elif ctx.edges %}-```mermaid-graph TD-{% for src, dst in ctx.edges[:60] %}- {{ src | replace("/", "_") | replace(".", "_") | replace("-", "_") }}["{{ src }}"] --> {{ dst | replace("/", "_") | replace(".", "_") | replace("-", "_") }}["{{ dst }}"]-{% endfor %}-```-{% else %}-_Not enough resolved dependencies to draw a map._-{% endif %}--## How to read this--Nodes are the most central files and modules by PageRank; arrows point from importer to imported. Groups are the architectural clusters dependency analysis found. The map covers the top {{ ctx.nodes | length }} node{{ "s" if ctx.nodes | length != 1 else "" }}, not the whole repository.--{% if ctx.communities %}-## Clusters-{% for cid, members in ctx.communities.items() %}-- **Cluster {{ cid }}** ({{ members | length }} shown){% for m in members %}- - `{{ m }}`{% endfor %}-{% endfor %}-{% endif %}--{% if ctx.scc_groups %}-## Import cycles-Groups whose members import each other in a loop. Each has its own page.-{% for group in ctx.scc_groups %}-- {{ group | length }} files: {% for f in group[:5] %}`{{ f }}`{% if not loop.last %}, {% endif %}{% endfor %}{% if group | length > 5 %} and {{ group | length - 5 }} more{% endif %}-{% endfor %}-{% endif %}--{% if ctx.nodes %}-## Nodes on the map-{% for n in ctx.nodes %}-- `{{ n }}`-{% endfor %}-{% endif %}-{% include "stub/_footer.j2" %}diff --git a/packages/core/src/repowise/core/persistence/crud/pages.py b/packages/core/src/repowise/core/persistence/crud/pages.pyindex 35fc57c1d..ceec2efc2 100644--- a/packages/core/src/repowise/core/persistence/crud/pages.py+++ b/packages/core/src/repowise/core/persistence/crud/pages.py@@ -338,6 +338,7 @@ async def load_prior_pages(input_tokens=row.input_tokens,output_tokens=row.output_tokens,cached_tokens=row.cached_tokens,+ metadata=metadata,)return priordiff --git a/packages/server/src/repowise/server/services/c4_builder/__init__.py b/packages/server/src/repowise/server/services/c4_builder/__init__.pyindex edcc245fd..1ce84453b 100644--- a/packages/server/src/repowise/server/services/c4_builder/__init__.py+++ b/packages/server/src/repowise/server/services/c4_builder/__init__.py@@ -24,6 +24,7 @@ from pathlib import Pathfrom sqlalchemy import selectfrom sqlalchemy.ext.asyncio import AsyncSession+from repowise.core.analysis.c4.actors import derive_actorsfrom repowise.core.ids import ExternalSystemId, SystemId, file_path_of, parse, renderfrom repowise.core.ingestion.workspace_members import root_descriptionfrom repowise.core.persistence import ExternalSystem, Repository@@ -37,7 +38,6 @@ from repowise.core.persistence.models import DeadCodeFinding, GitMetadatafrom repowise.core.support_paths import is_support_pathfrom repowise.core.test_paths import is_test_related_path-from .actors import derive_actorsfrom .components import detect_components, detect_components_for_allfrom .containers import container_id, detect_containersfrom .models import (diff --git a/packages/server/src/repowise/server/services/c4_builder/containers.py b/packages/server/src/repowise/server/services/c4_builder/containers.pyindex c3db217ee..3a5c3b721 100644--- a/packages/server/src/repowise/server/services/c4_builder/containers.py+++ b/packages/server/src/repowise/server/services/c4_builder/containers.py@@ -23,49 +23,22 @@ is the manifest's declared package name when the checkout is readable.from __future__ import annotationsfrom collections import defaultdict-from collections.abc import Iterablefrom pathlib import Pathfrom sqlalchemy import selectfrom sqlalchemy.ext.asyncio import AsyncSession+from repowise.core.analysis.c4.containers import assign_containers, container_namefrom repowise.core.analysis.kg_inputs import _dominant_language as _most_common_languagefrom repowise.core.ids import ContainerId, renderfrom repowise.core.ingestion.languages.registry import REGISTRY-from repowise.core.ingestion.workspace_members import manifest_package_namefrom repowise.core.persistence import ExternalSystem, GraphNode-from repowise.core.support_paths import is_test_or_example_pathfrom .models import Container-# Package-manifest basenames. A directory holding one of these is a container-# root even when the manifest declares no external dependencies (so a package-# like ``packages/core`` is its own container rather than dissolving into the-# repo-root catch-all).-_MANIFEST_BASENAMES = frozenset(- {- "pyproject.toml",- "setup.py",- "setup.cfg",- "package.json",- "cargo.toml",- "go.mod",- "pom.xml",- "build.gradle",- "composer.json",- "gemfile",- }-)-_MANIFEST_SUFFIXES = (".csproj",)-_CODE_LANGUAGES = REGISTRY.code_languages()-def _is_manifest(node_id: str) -> bool:- base = node_id.rsplit("/", 1)[-1].lower()- return base in _MANIFEST_BASENAMES or base.endswith(_MANIFEST_SUFFIXES)--async def detect_containers(session: AsyncSession,repository_id: str,@@ -85,34 +58,21 @@ async def detect_containers("""manifests = await _declared_manifests(session, repository_id)file_nodes = await _file_nodes(session, repository_id)-- # Also treat any directory holding a package manifest as a container root,- # even when that manifest declared no external deps (catches e.g. a core- # package that would otherwise dissolve into the repo-root catch-all).- manifests |= {n.node_id for n in file_nodes if _is_manifest(n.node_id)}- roots_to_manifests = _manifest_roots(manifests)- container_roots = set(roots_to_manifests)-- if not container_roots:- container_roots = _top_level_dirs(- n.node_id for n in file_nodes if not is_test_or_example_path(n.node_id)- )-- # Sort longest path first so nested manifests win over their parents.- sorted_roots = sorted(container_roots, key=len, reverse=True)+ owner, roots_to_manifests = assign_containers(manifests, (n.node_id for n in file_nodes))grouped: dict[str, list[GraphNode]] = defaultdict(list)for node in file_nodes:- root = _match_container(node.node_id, sorted_roots)- if root is not None:- grouped[root].append(node)+ if node.node_id in owner:+ grouped[owner[node.node_id]].append(node)fallback_root_name = (root_name or ".").strip() or "."containers: list[Container] = []for root in sorted(grouped):nodes = grouped[root]lang = _dominant_language(nodes)- name = _manifest_name(local_path, roots_to_manifests.get(root, ()))+ name = container_name(+ Path(local_path) if local_path else None, roots_to_manifests.get(root, ())+ )containers.append(Container(id=container_id(root),@@ -155,47 +115,6 @@ async def _file_nodes(session: AsyncSession, repository_id: str) -> list[GraphNoreturn list(result.scalars())-def _manifest_roots(manifests: Iterable[str]) -> dict[str, list[str]]:- """``{container root: its manifests}``, fixture and sample manifests dropped."""- roots: dict[str, list[str]] = defaultdict(list)- for path in sorted(manifests):- if not is_test_or_example_path(path):- roots[path.rsplit("/", 1)[0] if "/" in path else ""].append(path)- return roots---def _manifest_name(local_path: str | None, manifests: Iterable[str]) -> str | None:- """The first package name the root's manifests declare, read from the checkout."""- # Ceiling: needs the checkout on disk; persisting names at index time lifts it.- if not local_path or not Path(local_path).is_dir():- return None- for manifest in manifests:- name = manifest_package_name(Path(local_path) / manifest)- if name:- return name- return None---def _top_level_dirs(paths: Iterable[str]) -> set[str]:- out: set[str] = set()- for path in paths:- if "/" in path:- out.add(path.split("/", 1)[0])- else:- out.add("")- return out---def _match_container(file_path: str, sorted_roots: list[str]) -> str | None:- """Return the deepest container root containing ``file_path``."""- for root in sorted_roots:- if not root:- return root # root manifest catches everything that isn't claimed- if file_path == root or file_path.startswith(root + "/"):- return root- return None--def _dominant_language(nodes: list[GraphNode]) -> str:"""Most common code language; markdown and config never name a container."""langs = [n.language for n in nodes if n.language in _CODE_LANGUAGES]diff --git a/packages/server/src/repowise/server/services/c4_builder/relations.py b/packages/server/src/repowise/server/services/c4_builder/relations.pyindex 85f8ca449..73f3a5234 100644--- a/packages/server/src/repowise/server/services/c4_builder/relations.py+++ b/packages/server/src/repowise/server/services/c4_builder/relations.py@@ -1,38 +1,21 @@-"""Aggregate file→file graph edges up to container/component relations.+"""Read graph edges and roll them up to container/component relations.-The graph stores edges between individual files (and symbols). For C4-diagrams we need edges between higher-level boxes — between containers at-L2, between components at L3. This module does that aggregation:-- 1. Load all ``graph_edges`` whose source AND target are file-level.- 2. Map each endpoint to its container (L2) or component (L3) using the- ``file_index`` produced by :mod:`.containers` / :mod:`.components`.- 3. Group by (source_box, target_box, edge_type) and sum counts.- 4. Drop self-loops — an edge from a container to itself is not useful- to a viewer.--Co-change edges are history, not a dependency, so they are left out unless a-caller opts in to them as an overlay. Edges touching a configuration file-(``tsconfig.json``, ``pyproject.toml``) are dropped: reading config is not a-dependency between boxes.--External-system edges are produced from file→``external:*`` edges where-the target's ``external_system_id`` resolved to a row in the-``external_systems`` table.+The roll-up rules live in :mod:`repowise.core.analysis.c4.relations`; this+module owns the queries and turns each rolled pair into a labelled+:class:`Relation`. External-system edges come from file to ``external:*``+edges whose target resolved to a row in the ``external_systems`` table."""from __future__ import annotations-from collections import defaultdict-from sqlalchemy import selectfrom sqlalchemy.ext.asyncio import AsyncSession+from repowise.core.analysis.c4.labels import coupling_strength, relation_label+from repowise.core.analysis.c4.relations import roll_up_edgesfrom repowise.core.ids import ExternalSystemId, renderfrom repowise.core.persistence import ExternalSystem, GraphEdge, GraphNode-from repowise.core.support_paths import is_config_path-from .labels import coupling_strength, relation_labelfrom .models import Relation@@ -77,40 +60,18 @@ async def aggregate_relations(include_co_changes:Also roll up ``co_changes`` edges, as an overlay on the dependencies."""- file_to_external = file_to_external or {}if edges is None:edges = await load_edges(session, repository_id)-- # No containment filter needed: ``file_to_box`` is keyed on file paths, so- # ``defines`` (file → symbol) and ``has_method`` (symbol → symbol) always- # lose an endpoint below. Co-change is filtered: it is history, not a- # dependency, and drawn only when the caller asks for the overlay.- counts: dict[tuple[str, str], int] = defaultdict(int)- types: dict[tuple[str, str], set[str]] = defaultdict(set)-- for src, tgt, etype in edges:- if etype == "co_changes" and not include_co_changes:- continue- src_box = file_to_box.get(src)- if src_box is None:- continue- if tgt in file_to_box:- tgt_box = file_to_box[tgt]- elif tgt in file_to_external:- tgt_box = file_to_external[tgt]- else:- continue- if src_box == tgt_box:- continue- if is_config_path(src) or is_config_path(tgt):- continue- key = (src_box, tgt_box)- counts[key] += 1- types[key].add(etype)+ rolled = roll_up_edges(+ edges,+ file_to_box,+ file_to_external=file_to_external,+ include_co_changes=include_co_changes,+ )relations: list[Relation] = []- for (src_box, tgt_box), count in counts.items():- etypes = tuple(sorted(types[(src_box, tgt_box)]))+ for (src_box, tgt_box), (count, types) in rolled.items():+ etypes = tuple(sorted(types))relations.append(Relation(source_id=src_box,diff --git a/packages/server/src/repowise/server/services/zoom_builder/calls.py b/packages/server/src/repowise/server/services/zoom_builder/calls.pyindex 6255a5807..0eae2fb32 100644--- a/packages/server/src/repowise/server/services/zoom_builder/calls.py+++ b/packages/server/src/repowise/server/services/zoom_builder/calls.py@@ -69,7 +69,7 @@ async def load_projected_calls(edge to anything else would be dropped downstream anyway.Edge type is preserved rather than collapsed, because the verb is the- payload: :func:`~repowise.server.services.c4_builder.labels.relation_label`+ payload: :func:`~repowise.core.analysis.c4.labels.relation_label`turns the set of types on a pair into the arrow's word, and the canvasfilters on that word."""diff --git a/packages/server/src/repowise/server/services/zoom_builder/relations.py b/packages/server/src/repowise/server/services/zoom_builder/relations.pyindex 5e6440d88..0374487c9 100644--- a/packages/server/src/repowise/server/services/zoom_builder/relations.py+++ b/packages/server/src/repowise/server/services/zoom_builder/relations.py@@ -14,7 +14,7 @@ from __future__ import annotationsfrom collections import defaultdict-from repowise.server.services.c4_builder.labels import coupling_strength, relation_label+from repowise.core.analysis.c4.labels import coupling_strength, relation_labelfrom .models import ZoomNode, ZoomRelationfrom .tree import file_iddiff --git a/tests/unit/server/services/test_c4_actors.py b/tests/unit/analysis/test_c4_actors.pysimilarity index 92%rename from tests/unit/server/services/test_c4_actors.pyrename to tests/unit/analysis/test_c4_actors.pyindex 48caf1930..851708d13 100644--- a/tests/unit/server/services/test_c4_actors.py+++ b/tests/unit/analysis/test_c4_actors.py@@ -1,10 +1,10 @@-"""Unit tests for c4_builder.actors — actor derivation from entry points."""+"""Unit tests for analysis.c4.actors: actor derivation from entry points."""from __future__ import annotationsimport pytest-from repowise.server.services.c4_builder.actors import (+from repowise.core.analysis.c4.actors import (classify_entry_point,derive_actors,)diff --git a/tests/unit/server/services/test_c4_edge_verb_coverage.py b/tests/unit/analysis/test_c4_edge_verb_coverage.pysimilarity index 78%rename from tests/unit/server/services/test_c4_edge_verb_coverage.pyrename to tests/unit/analysis/test_c4_edge_verb_coverage.pyindex 89953d7bf..633b1124d 100644--- a/tests/unit/server/services/test_c4_edge_verb_coverage.py+++ b/tests/unit/analysis/test_c4_edge_verb_coverage.py@@ -5,17 +5,12 @@ fall-through is silent: `framework` and `dynamic_uses` rendered that way fortheir whole life, covering 10% of file-level edges on a live index, and`type_use` (6,596 rows across 42 local indexes) still did until this testexisted.--Lives here rather than beside the other vocabulary tests in-`tests/unit/ingestion/` because it imports `repowise.server` — an ingestion-test that hard-fails when only the core package is installed is a worse trade-than a slightly split test suite."""from __future__ import annotations+from repowise.core.analysis.c4.labels import _EDGE_VERB, _VERB_PRIORITYfrom repowise.core.ingestion.models import EDGE_TYPE_VALUES-from repowise.server.services.c4_builder.labels import _EDGE_VERB, _VERB_PRIORITYdef test_every_edge_type_has_a_verb() -> None:diff --git a/tests/unit/server/services/test_c4_labels.py b/tests/unit/analysis/test_c4_labels.pysimilarity index 92%rename from tests/unit/server/services/test_c4_labels.pyrename to tests/unit/analysis/test_c4_labels.pyindex a7d6151be..7e6d59674 100644--- a/tests/unit/server/services/test_c4_labels.py+++ b/tests/unit/analysis/test_c4_labels.py@@ -1,10 +1,10 @@-"""Unit tests for c4_builder.labels — relation verbs + coupling buckets."""+"""Unit tests for analysis.c4.labels: relation verbs and coupling buckets."""from __future__ import annotationsimport pytest-from repowise.server.services.c4_builder.labels import coupling_strength, relation_label+from repowise.core.analysis.c4.labels import coupling_strength, relation_label@pytest.mark.parametrize(diff --git a/tests/unit/generation/test_architecture_mermaid.py b/tests/unit/generation/test_architecture_mermaid.pyindex 4ff69ed22..5b721957e 100644--- a/tests/unit/generation/test_architecture_mermaid.py+++ b/tests/unit/generation/test_architecture_mermaid.py@@ -1,182 +1,403 @@-"""Tests for the deterministic architecture-diagram mermaid builder."""+"""The overview's system map: structure picks it, code compiles it."""from __future__ import annotations+import re++from repowise.core.generation.architecture_map import MAX_NODES, build_system_mapfrom repowise.core.generation.architecture_mermaid import (- ArchitectureMermaidBuilder,- build_layer_mermaid,- build_overview_mermaid,- embed_mermaid,+ HEADING,+ apply_names,+ compile_mermaid,+ embed_system_map,+ split_names,+ system_map_section,)-from repowise.core.generation.kg_context import KnowledgeGraphContext-def _kg() -> dict:- """Synthetic KG: two layers (UI, API), three modules, weighted imports.+def _monorepo(extra_packages: int = 0):+ """A generic web app: a frontend and a CLI in front of an API over a core library."""+ files = {+ "apps/web/src/page.ts": "typescript",+ "apps/web/src/client.ts": "typescript",+ "apps/cli/src/tool/main.py": "python",+ "services/api/src/api/app.py": "python",+ "services/api/src/api/routes.py": "python",+ "libs/core/src/core/engine.py": "python",+ "libs/core/src/core/store.py": "python",+ "libs/core/src/core/model.py": "python",+ "libs/shared/src/index.ts": "typescript",+ }+ manifests = [+ "apps/web/package.json",+ "apps/cli/pyproject.toml",+ "services/api/pyproject.toml",+ "libs/core/pyproject.toml",+ "libs/shared/package.json",+ ]+ edges = [+ ("apps/web/src/page.ts", "libs/shared/src/index.ts", "imports"),+ ("apps/web/src/page.ts", "apps/web/src/client.ts", "imports"),+ ("apps/cli/src/tool/main.py", "libs/core/src/core/engine.py", "imports"),+ ("services/api/src/api/routes.py", "libs/core/src/core/engine.py", "imports"),+ ("services/api/src/api/routes.py", "libs/core/src/core/model.py", "calls"),+ ("libs/core/src/core/store.py", "external:dbdriver", "imports"),+ ]+ for i in range(extra_packages):+ files[f"libs/extra{i}/src/mod.py"] = "python"+ manifests.append(f"libs/extra{i}/pyproject.toml")+ edges.append((f"libs/extra{i}/src/mod.py", "libs/core/src/core/model.py", "imports"))+ return build_system_map(+ files,+ edges,+ repo_name="demo",+ manifests=manifests,+ package_names={"libs/core": "demo-core"},+ entry_points=["apps/cli/src/tool/main.py", "services/api/src/api/app.py"],+ route_files=["services/api/src/api/routes.py"],+ stores={"external:dbdriver": "dbdriver"},+ http_links=[("apps/web/src/client.ts", "services/api/src/api/routes.py")],+ pages=[("libs/core/src/core", {"libs/core/src/core/engine.py"}), ("apps/web", set())],+ )- ui/a -> api/a has 15 file-pair imports (a strong, kept edge); ui/b -> api/a- has 3 (below the overview floor, dropped); ui/a -> ui/b has 1 (an intra-UI- edge, shown only in the UI layer diagram).- """- def files(mod: str, n: int) -> list[str]:- return [f"{mod}/f{i}.py" for i in range(n)]+_NODE_DEF = re.compile(r"^([A-Za-z_]\w*)[\[(]")+_EDGE = re.compile(r'^([A-Za-z_]\w*) (?:-->|==>)\|"[^"|]*"\| ([A-Za-z_]\w*)$')- ui_a, ui_b, api_a = files("ui/a", 15), files("ui/b", 3), files("api/a", 15)- nodes = [{"id": f"file:{f}", "type": "file", "filePath": f} for f in ui_a + ui_b + api_a]- edges: list[dict] = []- for s, t in zip(ui_a, api_a, strict=True): # 15 cross-layer imports ui/a -> api/a- edges.append({"source": f"file:{s}", "target": f"file:{t}", "type": "imports"})- for s, t in zip(ui_b, api_a[:3], strict=True): # 3 cross-layer imports ui/b -> api/a- edges.append({"source": f"file:{s}", "target": f"file:{t}", "type": "imports"})- edges.append({"source": f"file:{ui_a[0]}", "target": f"file:{ui_b[0]}", "type": "imports"})+def _compiler_problems(body: str) -> list[str]:+ """What a compiler bug would break: every edge, click and class names something defined."""+ lines = [line.strip() for line in body.split("\n")[1:] if line.strip()]+ nodes, classes, refs, used, depth = set(), set(), [], set(), 0+ problems = []+ for text in lines:+ if text.count('"') % 2:+ problems.append(f"unbalanced quote: {text}")+ used.update(re.findall(r":::(\w+)$", text))+ if text.startswith("subgraph "):+ depth += 1+ elif text == "end":+ depth -= 1+ elif text.startswith("classDef "):+ classes.add(text.split()[1])+ elif text.startswith("click "):+ refs.append(text.split()[1])+ elif text.startswith("direction "):+ continue+ elif m := _EDGE.match(text):+ refs += [m.group(1), m.group(2)]+ elif m := _NODE_DEF.match(text):+ nodes.add(m.group(1))+ else:+ problems.append(f"unrecognised line: {text}")+ if depth:+ problems.append("unclosed subgraph")+ problems += [f"undefined node: {r}" for r in refs if r not in nodes]+ problems += [f"undefined class: {c}" for c in sorted(used - classes)]+ return problems- return {- "project": {"name": "t"},- "nodes": nodes,- "edges": edges,- "layers": [- {- "id": "layer:ui",- "name": "UI",- "description": "",- "nodeIds": [f"file:{f}" for f in ui_a + ui_b],- },- {- "id": "layer:api",- "name": "API",- "description": "",- "nodeIds": [f"file:{f}" for f in api_a],- },- ],- "modules": [- {- "id": "module:ui-a",- "name": "ui/a",- "path": "ui/a",- "layerId": "layer:ui",- "nodeIds": [f"file:{f}" for f in ui_a],- },- {- "id": "module:ui-b",- "name": "ui/b",- "path": "ui/b",- "layerId": "layer:ui",- "nodeIds": [f"file:{f}" for f in ui_b],- },++def _tier_of(system_map):+ return {n.path or n.id: n.tier for n in system_map.nodes}+++class TestStructure:+ def test_tiers_follow_how_each_part_is_entered(self):+ tiers = _tier_of(_monorepo())+ assert tiers["services/api"] == "service" # api entry point, HTTP route+ assert tiers["apps/web"] == "surface" # HTTP caller, nothing imports it+ assert tiers["apps/cli"] == "surface" # CLI entry point+ assert tiers["libs/core"] == "engine"+ assert tiers["n_store"] == "store"+ # One person for every human way in, one box per calling program.+ assert {"n_actor_user", "n_actor_api"} <= set(tiers)+ assert "n_actor_cli" not in tiers++ def test_runtime_and_store_edges_are_drawn(self):+ verbs = {(e.source, e.target): e.verb for e in _monorepo().edges}+ assert verbs[("n_apps_web", "n_services_api")] == "calls over HTTP"+ assert verbs[("n_libs_core", "n_store")] == "reads and writes"+ assert verbs[("n_actor_user", "n_apps_cli")] == "runs"+ assert verbs[("n_actor_user", "n_apps_web")] == "uses"+ assert verbs[("n_services_api", "n_libs_core")] == "calls"++ def test_node_cap_counts_omitted_parts(self):+ sm = _monorepo(extra_packages=12)+ assert len(sm.nodes) <= MAX_NODES+ assert sm.omitted > 0++ def test_heavy_edge_is_drawn_thick_and_the_rest_are_capped(self):+ files = {f"p{i}/m.py": "python" for i in range(8)}+ files.update({f"p0/x{k}.py": "python" for k in range(60)})+ manifests = [f"p{i}/pyproject.toml" for i in range(8)]+ edges = [+ (f"p{i}/m.py", f"p{j}/m.py", "imports")+ for i in range(8)+ for j in range(i + 1, 8)+ ]+ edges += [(f"p0/x{k}.py", "p1/m.py", "imports") for k in range(60)]+ sm = build_system_map(files, edges, repo_name="demo", manifests=manifests)+ assert [(e.source, e.target) for e in sm.edges if e.heavy] == [("n_p0", "n_p1")]+ dependency_edges = [e for e in sm.edges if not e.source.startswith("n_actor_")]+ assert len(dependency_edges) <= 10++ def test_single_package_is_opened_up(self):+ files = {f"src/pkg/{n}.py": "python" for n in ("app", "cli", "helpers", "models")}+ files["src/pkg/sub/a.py"] = "python"+ edges = [+ ("src/pkg/cli.py", "src/pkg/app.py", "imports"),+ ("src/pkg/app.py", "src/pkg/helpers.py", "imports"),+ ("src/pkg/app.py", "src/pkg/sub/a.py", "imports"),+ ("src/pkg/models.py", "src/pkg/helpers.py", "imports"),+ ]+ sm = build_system_map(+ files,+ edges,+ repo_name="pkg",+ manifests=["pyproject.toml"],+ entry_points=["src/pkg/cli.py"],+ )+ paths = {n.path for n in sm.nodes if n.path}+ assert {"src/pkg/sub", "src/pkg/app.py"} <= paths+ assert _tier_of(sm)["src/pkg/cli.py"] == "surface"++ def test_nothing_to_draw_returns_none(self):+ assert build_system_map({"a.py": "python"}, [], repo_name="x") is None++ def test_keyless_names_come_from_packages_and_paths(self):+ nodes = {n.path: n for n in _monorepo().nodes if n.path}+ assert nodes["libs/core"].name == "demo-core"+ assert nodes["apps/web"].name == "web"+ # The library only the web app imports is part of the web app's box.+ assert "libs/shared" not in nodes+ assert nodes["apps/web"].role == "3 TypeScript files"++ def test_a_route_file_is_flow_and_makes_its_box_the_server(self):+ """A route file defines no named symbol, yet it is the server, not a re-export."""+ files = {+ "app/server.js": "javascript",+ "app/routes/users.js": "javascript",+ "app/routes/posts.js": "javascript",+ "app/lib/db.js": "javascript",+ "app/lib/auth.js": "javascript",+ "app/cli/run.js": "javascript",+ }+ edges = [+ ("app/server.js", "app/routes/users.js", "imports"),+ ("app/server.js", "app/routes/posts.js", "imports"),+ ("app/routes/users.js", "app/lib/db.js", "imports"),+ ("app/routes/posts.js", "app/lib/auth.js", "imports"),+ ("app/cli/run.js", "app/lib/db.js", "imports"),+ ]+ sm = build_system_map(+ files,+ edges,+ repo_name="app",+ manifests=["app/package.json"],+ barrels={"app/routes/users.js", "app/routes/posts.js", "app/server.js"},+ route_files={"app/routes/users.js", "app/routes/posts.js"},+ )+ tiers = _tier_of(sm)+ assert tiers["app/routes/users.js"] == tiers["app/routes/posts.js"] == "service"+ drawn = {(e.source, e.target) for e in sm.edges}+ assert ("n_app_routes_users_js", "n_app_lib") in drawn++ def test_a_stray_route_does_not_make_a_large_library_the_server(self):+ files = {f"lib/m{i}.py": "python" for i in range(40)}+ files.update({f"{d}/main.py": "python" for d in ("cli", "a", "b", "c", "d")})+ edges = [("cli/main.py", f"lib/m{i}.py", "imports") for i in range(40)]+ edges += [(f"{d}/main.py", "lib/m0.py", "imports") for d in "abcd"]+ manifests = [f"{d}/pyproject.toml" for d in ("lib", "cli", "a", "b", "c", "d")]+ sm = build_system_map(+ files, edges, repo_name="demo", manifests=manifests, route_files={"lib/m7.py"}+ )+ assert _tier_of(sm)["lib"] == "engine"+ routes_only = build_system_map(+ files, edges, repo_name="demo", manifests=manifests,+ route_files={f"lib/m{i}.py" for i in range(10)},+ )+ assert _tier_of(routes_only)["lib"] == "service"++ def test_a_store_nobody_drawn_uses_takes_no_slot(self):+ files = {f"p{i}/m.py": "python" for i in range(12)}+ manifests = [f"p{i}/pyproject.toml" for i in range(12)]+ edges = [(f"p{i}/m.py", "p0/m.py", "imports") for i in range(1, 12)]+ files.update({f"p0/x{k}.py": "python" for k in range(5)})+ edges.append(("p9/m.py", "external:db", "imports"))+ sm = build_system_map(+ files, edges, repo_name="demo", manifests=manifests, stores={"external:db": "db"}+ )+ assert "n_store" not in {n.id for n in sm.nodes}+ assert len(sm.nodes) == MAX_NODES++ def test_libraries_several_ways_in_share_become_one_box(self):+ files = {+ "web/a.ts": "typescript",+ "desk/a.ts": "typescript",+ "ui/a.ts": "typescript",+ "kit/a.ts": "typescript",+ "cli/a.py": "python",+ "core/a.py": "python",+ }+ edges = [+ ("web/a.ts", "ui/a.ts", "imports"),+ ("desk/a.ts", "ui/a.ts", "imports"),+ ("ui/a.ts", "kit/a.ts", "imports"),+ ("cli/a.py", "core/a.py", "imports"),+ ("core/a.py", "external:db", "imports"),+ ]+ sm = build_system_map(+ files,+ edges,+ repo_name="demo",+ manifests=[f"{d}/package.json" for d in ("web", "desk", "ui", "kit", "cli", "core")],+ stores={"external:db": "db"},+ )+ shared = [n for n in sm.nodes if n.tier == "shared"]+ assert [(n.name, n.role) for n in shared] == [("Shared libraries", "2 TypeScript files")]+ assert _tier_of(sm)["core"] == "engine"++ def test_re_exports_are_not_flow(self):+ files = {f"src/pkg/{n}.py": "python" for n in ("__init__", "app", "cli", "a", "b", "c")}+ edges = [("src/pkg/__init__.py", f"src/pkg/{n}.py", "imports") for n in "abc"]+ edges += [+ ("src/pkg/__main__.py", "src/pkg/cli.py", "imports"),+ ("src/pkg/cli.py", "src/pkg/app.py", "imports"),+ ("src/pkg/app.py", "src/pkg/a.py", "imports"),+ ("src/pkg/a.py", "src/pkg/b.py", "imports"),+ ("src/pkg/b.py", "src/pkg/c.py", "imports"),+ ]+ files["src/pkg/__main__.py"] = "python"+ sm = build_system_map(+ files,+ edges,+ repo_name="pkg",+ manifests=["pyproject.toml"],+ entry_points=["src/pkg/__main__.py", "src/pkg/app.py"],+ barrels={"src/pkg/__init__.py", "src/pkg/__main__.py"},+ )+ paths = {n.path for n in sm.nodes if n.path}+ assert "src/pkg/__init__.py" not in paths+ assert not any(e.source == "n_src_pkg_init_py" for e in sm.edges)+ # The launcher hands its CLI role to the file it runs. app.py serves no+ # route and the CLI already calls it, so it is not a second way in.+ verbs = {(e.source, e.target): e.verb for e in sm.edges}+ assert verbs[("n_actor_user", "n_src_pkg_cli_py")] == "runs"+ assert _tier_of(sm)["src/pkg/app.py"] == "engine"+ assert len(sm.nodes) <= 7+++class TestCompile:+ def test_compiles_to_a_valid_flowchart(self):+ body = compile_mermaid(_monorepo())+ assert body.startswith("flowchart TB")+ assert _compiler_problems(body) == []+ assert 'subgraph tier_service["Server"]' in body+ assert "classDef engine" in body and "#" not in body # no hardcoded colors++ def test_click_goes_to_the_module_page(self):+ body = compile_mermaid(_monorepo())+ assert 'click n_libs_core "?page=module_page%3Alibs%2Fcore%2Fsrc%2Fcore"' in body+ assert 'click n_apps_web "?page=module_page%3Aapps%2Fweb"' in body++ def test_model_text_is_escaped(self):+ sm = apply_names(+ _monorepo(),{- "id": "module:api-a",- "name": "api/a",- "path": "api/a",- "layerId": "layer:api",- "nodeIds": [f"file:{f}" for f in api_a],+ "nodes": [+ {"id": "n_libs_core", "label": 'Core "engine" <b>|x|:::y', "role": "a;b#c"}+ ],+ "edges": [{"from": "n_services_api", "to": "n_libs_core", "verb": "calls (x)"}],},- ],- }+ )+ body = compile_mermaid(sm)+ assert _compiler_problems(body) == []+ assert "<b>Core engine bxy</b><br/><small>abc</small>" in body+ assert 'n_services_api ==>|"calls (x)"| n_libs_core' in body or (+ 'n_services_api -->|"calls (x)"| n_libs_core' in body+ )+ def test_caption_cannot_break_the_page(self):+ sm = apply_names(_monorepo(), {"caption": "# Parts ``` of <b>it</b>."})+ section = system_map_section(sm, "demo")+ assert section.count("```") == 2+ assert section.split("\n\n")[1].startswith("Parts of bit/b.")-def _ctx(kg: dict | None = None) -> KnowledgeGraphContext:- return KnowledgeGraphContext(None, data=kg if kg is not None else _kg())---class TestOverview:- def test_shape_and_subgraphs(self):- ov = build_overview_mermaid(_ctx())- assert ov is not None- assert ov.startswith("flowchart LR")- # one subgraph per layer, module labels present- assert ov.count("subgraph") == 2- assert "ui/a" in ov and "api/a" in ov-- def test_edge_floor_keeps_strong_drops_weak(self):- ov = build_overview_mermaid(_ctx())- # only the count-15 cross edge clears the floor; the count-3 and count-1- # edges are dropped- assert '|"15"|' in ov- assert '|"3"|' not in ov- assert ov.count("-->") == 1- assert ov.count("-.->") == 0-- def test_no_inline_style_theme_safe(self):- ov = build_overview_mermaid(_ctx())- assert "#" not in ov # no raw hex — the renderer owns the palette- assert "classDef" not in ov and "style " not in ov---class TestLayer:- def test_shape_modules_and_intra_edges(self):- c = _ctx()- ui = c.get_layers()[0]- lm = build_layer_mermaid(c, ui)- assert lm is not None- assert lm.startswith("flowchart TD")- assert '"UI layer"' in lm- assert "ui/a" in lm and "ui/b" in lm- assert '|"1"|' in lm # intra-layer ui/a -> ui/b-- def test_boundary_layers_as_stadium_nodes(self):- c = _ctx()- ui = c.get_layers()[0]- lm = build_layer_mermaid(c, ui)- # UI depends on API with 15 + 3 = 18 imports, drawn as a stadium node- assert "API" in lm- assert '(["' in lm # stadium shape, not color, marks a boundary layer- assert '|"18"|' in lm-- def test_no_inline_style_theme_safe(self):- c = _ctx()- lm = build_layer_mermaid(c, c.get_layers()[0])- assert "#" not in lm- assert "classDef" not in lm and "style " not in lm---class TestFallbacks:- def test_unavailable_kg_returns_none(self):- c = KnowledgeGraphContext(None)- assert not c.available- assert build_overview_mermaid(c) is None- assert build_layer_mermaid(c, {"id": "layer:x", "name": "X"}) is None-- def test_no_modules_returns_none(self):- kg = _kg()- kg.pop("modules")- c = _ctx(kg)- assert c.available- assert build_overview_mermaid(c) is None- assert build_layer_mermaid(c, c.get_layers()[0]) is None-- def test_builder_reused_for_overview_and_layers(self):- b = ArchitectureMermaidBuilder(_ctx())- assert b.overview() is not None- assert b.layer(_ctx().get_layers()[0]) is not None+class TestNaming:+ def test_split_names_strips_the_trailing_block(self):+ reply = '## Project Summary\n\nText.\n\n```json\n{"caption": "C.", "nodes": []}\n```\n'+ prose, names = split_names(reply)+ assert "```json" not in prose+ assert names == {"caption": "C.", "nodes": []}-class TestEmbed:- def test_append_when_no_block(self):- out = embed_mermaid("Prose.\n", "flowchart LR\nA-->B", heading="## Architecture")- assert "## Architecture" in out- assert out.count("```mermaid") == 1- assert "A-->B" in out+ def test_split_names_keeps_prose_written_after_the_block(self):+ reply = '## A\n\nText.\n\n```json\n{"nodes": []}\n```\n\n## B\n\nMore.\n'+ prose, names = split_names(reply)+ assert "```json" not in prose and "## B\n\nMore." in prose+ assert names == {"nodes": []}++ def test_split_names_accepts_other_fence_shapes(self):+ for fence in ("```JSON", "```", " ```json"):+ reply = f'## A\n\nText.\n\n{fence}\n {{"nodes": []}}\n```\n'+ prose, names = split_names(reply)+ assert names == {"nodes": []} and "nodes" not in prose++ def test_split_names_leaves_other_json_alone(self):+ reply = '## A\n\nConfig:\n\n```json\n{"port": 8080}\n```\n'+ assert split_names(reply) == (reply, None)- def test_append_is_idempotent(self):- c1 = embed_mermaid("Prose.\n", "flowchart LR\nA-->B", heading="## Architecture")- c2 = embed_mermaid(c1, "flowchart LR\nA-->B", heading="## Architecture")- assert c1 == c2- assert c2.count("```mermaid") == 1+ def test_split_names_drops_a_block_cut_off_mid_reply(self):+ reply = '## A\n\nText.\n\n```json\n{"caption": "Work flows", "nodes": [{"id"'+ prose, names = split_names(reply)+ assert names is None and prose == "## A\n\nText.\n"++ def test_unknown_ids_and_pairs_are_ignored(self):+ sm = _monorepo()+ named = apply_names(+ sm,+ {+ "caption": "Work flows from the ways in to the core. Extra. Dropped.",+ "nodes": [+ {"id": "n_libs_core", "label": "Core engine", "role": "Does the work"},+ {"id": "n_made_up", "label": "Ghost"},+ ],+ "edges": [+ {"from": "n_services_api", "to": "n_libs_core", "verb": "runs"},+ {"from": "n_libs_core", "to": "n_apps_web", "verb": "invented"},+ ],+ },+ )+ assert [n.id for n in named.nodes] == [n.id for n in sm.nodes]+ assert [(e.source, e.target) for e in named.edges] == [+ (e.source, e.target) for e in sm.edges+ ]+ verbs = {(e.source, e.target): e.verb for e in named.edges}+ assert verbs[("n_services_api", "n_libs_core")] == "runs"+ assert named.caption == "Work flows from the ways in to the core. Extra."+++class TestEmbed:+ def test_lands_under_the_first_section_and_is_idempotent(self):+ page = "# T\n\n## Project Summary\n\nWhat.\n\n## Architecture\n\nHow.\n"+ section = system_map_section(_monorepo(), "demo")+ once = embed_system_map(page, section)+ assert once.index("## Project Summary") < once.index(HEADING) < once.index("## Architecture")+ assert embed_system_map(once, section) == once- def test_replaces_existing_block_and_keeps_surrounding_prose(self):- llm = "Intro\n```mermaid\ngraph TD\nX-->Y\n```\nOutro text"- out = embed_mermaid(llm, "flowchart LR\nA-->B", heading="## X")+ def test_replaces_the_legacy_end_of_page_map(self):+ page = (+ "## Project Summary\n\nWhat.\n\n## Architecture map\n\n"+ "```mermaid\nflowchart LR\nA-->B\n```\n"+ )+ out = embed_system_map(page, system_map_section(_monorepo(), "demo"))+ assert "## Architecture map" not in outassert out.count("```mermaid") == 1- assert "A-->B" in out and "X-->Y" not in out- assert "Intro" in out and "Outro text" in out- def test_replace_is_idempotent(self):- llm = "Intro\n```mermaid\ngraph TD\nX-->Y\n```\nOutro"- r1 = embed_mermaid(llm, "flowchart LR\nA-->B", heading="## X")- r2 = embed_mermaid(r1, "flowchart LR\nA-->B", heading="## X")- assert r1 == r2+ def test_caption_names_what_was_cut(self):+ section = system_map_section(_monorepo(extra_packages=12), "demo")+ assert "more parts are not shown" in section- def test_empty_mermaid_is_noop(self):- assert embed_mermaid("abc", "", heading="## X") == "abc"+ def test_no_map_removes_a_stale_one(self):+ page = embed_system_map("## A\n\nx\n\n## B\n", system_map_section(_monorepo(), "demo"))+ assert HEADING not in embed_system_map(page, None)diff --git a/tests/unit/generation/test_overview_architecture_map.py b/tests/unit/generation/test_overview_architecture_map.pydeleted file mode 100644index 96fe8520c..000000000--- a/tests/unit/generation/test_overview_architecture_map.py+++ /dev/null@@ -1,130 +0,0 @@-"""The architecture map is part of the repository overview.--It used to have a page of its own that described the same repository at the-same altitude and in the same words as the overview — the two shared roughly a-quarter of their vocabulary, so a reader who met both read one thing twice.-The diagram was the only thing that page uniquely had, so it moved here.--The map is the deterministic graph-derived one, not a diagram the model drew,-and it has to survive every path the overview can be produced by: the model-path, the deterministic path, and the fallback taken when the provider fails.-Losing it on the fallback would mean a provider outage silently costs the wiki-its architecture diagram.-"""--from __future__ import annotations--from dataclasses import dataclass, replace--import pytest--from repowise.core.generation.context_assembler import ContextAssembler-from repowise.core.generation.models import GeneratedPage-from repowise.core.generation.page_generator import PageGenerator-from repowise.core.providers.llm.mock import MockProvider--_MERMAID = "graph TD\n A[cli] --> B[core]\n B --> C[server]"---class _OutageProvider(MockProvider):- """A provider that fails the way a real outage does: every call raises."""-- async def generate(self, *args, **kwargs):- raise RuntimeError("upstream 529 overloaded")---@dataclass-class _Case:- name: str- gen: PageGenerator---def model_gen(sample_config) -> PageGenerator:- return PageGenerator(MockProvider(), ContextAssembler(sample_config), sample_config)---def outage_gen(sample_config) -> PageGenerator:- return PageGenerator(_OutageProvider(), ContextAssembler(sample_config), sample_config)---def deterministic_gen(sample_config) -> PageGenerator:- cfg = replace(sample_config, deterministic=True)- return PageGenerator(MockProvider(), ContextAssembler(cfg), cfg)---async def _overview(- gen: PageGenerator, sample_repo_structure, mermaid: str | None-) -> GeneratedPage:- return await gen.generate_repo_overview(- sample_repo_structure,- pagerank={},- sccs=[],- community={},- repo_name="demo",- overview_mermaid=mermaid,- )---async def test_model_written_overview_carries_the_map(model_gen, sample_repo_structure):- page = await _overview(model_gen, sample_repo_structure, _MERMAID)- assert "## Architecture map" in page.content- assert "```mermaid" in page.content- assert "A[cli] --> B[core]" in page.content---async def test_deterministic_overview_carries_the_map(deterministic_gen, sample_repo_structure):- page = await _overview(deterministic_gen, sample_repo_structure, _MERMAID)- assert "```mermaid" in page.content- assert "A[cli] --> B[core]" in page.content---async def test_provider_outage_keeps_the_map(outage_gen, sample_repo_structure):- """A provider outage costs the prose around the diagram, never the diagram."""- page = await _overview(outage_gen, sample_repo_structure, _MERMAID)- assert "```mermaid" in page.content- assert "A[cli] --> B[core]" in page.content---async def test_embedding_is_idempotent(model_gen, sample_repo_structure):- """Reused and cached pages are re-embedded on every run; that must be a no-op."""- page = await _overview(model_gen, sample_repo_structure, _MERMAID)- assert page.content.count("```mermaid") == 1---async def test_no_map_available_leaves_the_page_alone(model_gen, sample_repo_structure):- """An empty map must not stamp an empty diagram block onto the page."""- page = await _overview(model_gen, sample_repo_structure, "")- assert "```mermaid" not in page.content- assert page.content.strip()---async def test_overview_prompt_includes_repository_source_evidence(- model_gen, sample_repo_structure-):- config = replace(- model_gen._config,- source_evidence_files={- "repo_overview": ("README.md", "docs/ARCHITECTURE.md"),- },- )- generator = PageGenerator(model_gen._provider, ContextAssembler(config), config)- await generator.generate_repo_overview(- sample_repo_structure,- pagerank={},- sccs=[],- community={},- repo_name="demo",- source_map={- "README.md": b"Demo turns source archives into indexed documentation.",- "docs/ARCHITECTURE.md": b"The parser feeds a graph and then a wiki generator.",- },- )-- prompt = generator._provider.calls[-1]["user_prompt"]- assert "## Additional repository evidence" in prompt- assert '<repository-file path="README.md">' in prompt- assert "turns source archives into indexed documentation" in prompt- assert '<repository-file path="docs/ARCHITECTURE.md">' in promptdiff --git a/tests/unit/generation/test_overview_capability_wiring.py b/tests/unit/generation/test_overview_capability_wiring.pyindex 174a574d8..4b5458ad3 100644--- a/tests/unit/generation/test_overview_capability_wiring.py+++ b/tests/unit/generation/test_overview_capability_wiring.py@@ -19,8 +19,10 @@ from pathlib import Pathfrom types import SimpleNamespacefrom typing import Any+import networkx as nxfrom structlog.testing import capture_logs+from repowise.core.generation.kg_context import KnowledgeGraphContextfrom repowise.core.generation.page_generator.levels import (build_level6_coros,build_level8_coros,@@ -53,7 +55,15 @@ class BlastRadius:#: What the structural side independently arrived at. "Blast radius" is named#: by both; "Change risk" is in the documents only.-MODULE_GROUPS = [SimpleNamespace(display="Blast Radius Evaluation", label="", key="src")]+MODULE_GROUPS = [+ SimpleNamespace(+ display="Blast Radius Evaluation",+ label="",+ key="src",+ file_paths=("src/blast_radius.py",),+ context_paths=(),+ )+]class _Store:@@ -162,6 +172,7 @@ def _run(graph_builder=SimpleNamespace(community_info=lambda: {},execution_flows=lambda: SimpleNamespace(flows=[]),+ graph=nx.DiGraph,),pagerank={},betweenness={},@@ -176,7 +187,7 @@ def _run(vector_store=store,tour_stops=(),layer_order=(),- kg_ctx=None,+ kg_ctx=KnowledgeGraphContext(None),on_subphase=None,_emit=lambda page_id: True,)@@ -216,6 +227,8 @@ async def test_a_community_label_does_not_corroborate(tmp_path):display="Ledger Postings",label="Change Risk and Ingestion Engine",key="src/change_risk",+ file_paths=(),+ context_paths=(),)],)diff --git a/tests/unit/generation/test_overview_system_map.py b/tests/unit/generation/test_overview_system_map.pynew file mode 100644index 000000000..7d933928c--- /dev/null+++ b/tests/unit/generation/test_overview_system_map.py@@ -0,0 +1,209 @@+"""The system map is part of the repository overview.++It is built from structure and has to survive every path the overview can be+produced by: the model path, the deterministic path, and the fallback taken+when the provider fails. Losing it on the fallback would mean a provider+outage silently costs the wiki its diagram. A model only renames what the+structure drew, through a JSON block that never reaches the page.+"""++from __future__ import annotations++from dataclasses import replace++import pytest++from repowise.core.generation.architecture_map import SystemMap, build_system_map+from repowise.core.generation.context_assembler import ContextAssembler+from repowise.core.generation.models import GeneratedPage+from repowise.core.generation.page_generator import PageGenerator, PriorPage+from repowise.core.generation.page_generator.pertype import SYSTEM_MAP_NAMES+from repowise.core.providers.llm.base import GeneratedResponse+from repowise.core.providers.llm.mock import MockProvider+++def _map() -> SystemMap:+ return build_system_map(+ {"cli/main.py": "python", "core/engine.py": "python", "core/store.py": "python"},+ [("cli/main.py", "core/engine.py", "imports"), ("core/engine.py", "core/store.py", "calls")],+ repo_name="demo",+ manifests=["cli/pyproject.toml", "core/pyproject.toml"],+ entry_points=["cli/main.py"],+ )+++_NAMED_REPLY = (+ "## Project Summary\n\nDemo does things.\n\n## Architecture\n\nParts.\n\n"+ "```json\n"+ '{"caption": "Commands reach the engine.", '+ '"nodes": [{"id": "n_core_engine_py", "label": "Processing engine", "role": "Does the work"}], '+ '"edges": [{"from": "n_cli", "to": "n_core_engine_py", "verb": "drives"}]}\n'+ "```\n"+)+++class _OutageProvider(MockProvider):+ """A provider that fails the way a real outage does: every call raises."""++ async def generate(self, *args, **kwargs):+ raise RuntimeError("upstream 529 overloaded")+++def model_gen(sample_config) -> PageGenerator:+ return PageGenerator(MockProvider(), ContextAssembler(sample_config), sample_config)+++def naming_gen(sample_config) -> PageGenerator:+ provider = MockProvider(responses=[GeneratedResponse(_NAMED_REPLY, 100, 50)])+ return PageGenerator(provider, ContextAssembler(sample_config), sample_config)+++def outage_gen(sample_config) -> PageGenerator:+ return PageGenerator(_OutageProvider(), ContextAssembler(sample_config), sample_config)+++def deterministic_gen(sample_config) -> PageGenerator:+ cfg = replace(sample_config, deterministic=True)+ return PageGenerator(MockProvider(), ContextAssembler(cfg), cfg)+++async def _overview(+ gen: PageGenerator, sample_repo_structure, system_map: SystemMap | None+) -> GeneratedPage:+ return await gen.generate_repo_overview(+ sample_repo_structure,+ pagerank={},+ sccs=[],+ community={},+ repo_name="demo",+ system_map=system_map,+ )+++async def test_model_written_overview_carries_the_map(model_gen, sample_repo_structure):+ page = await _overview(model_gen, sample_repo_structure, _map())+ assert "## System map" in page.content+ assert 'n_cli -->|"imports"| n_core_engine_py' in page.content+ prompt = model_gen._provider.calls[-1]["user_prompt"]+ assert "### System map to name" in prompt and '"id": "n_core_engine_py"' in prompt+++async def test_model_names_reach_the_map_and_the_block_does_not(naming_gen, sample_repo_structure):+ page = await _overview(naming_gen, sample_repo_structure, _map())+ assert "<b>Processing engine</b>" in page.content+ assert 'n_cli -->|"drives"| n_core_engine_py' in page.content+ assert "Commands reach the engine." in page.content+ assert "```json" not in page.content+ assert page.content.index("## System map") < page.content.index("## Architecture")+++async def test_deterministic_overview_carries_the_map(deterministic_gen, sample_repo_structure):+ page = await _overview(deterministic_gen, sample_repo_structure, _map())+ assert "```mermaid" in page.content+ assert 'n_cli -->|"imports"| n_core_engine_py' in page.content+++async def test_provider_outage_keeps_the_map(outage_gen, sample_repo_structure):+ """A provider outage costs the prose around the diagram, never the diagram."""+ page = await _overview(outage_gen, sample_repo_structure, _map())+ assert "```mermaid" in page.content+ assert 'n_cli -->|"imports"| n_core_engine_py' in page.content+++async def test_one_map_per_page(model_gen, sample_repo_structure):+ page = await _overview(model_gen, sample_repo_structure, _map())+ assert page.content.count("```mermaid") == 1+++async def test_no_map_available_leaves_the_page_alone(model_gen, sample_repo_structure):+ """No structure to draw must not stamp an empty diagram block onto the page."""+ page = await _overview(model_gen, sample_repo_structure, None)+ assert "```mermaid" not in page.content+ assert page.content.strip()+++async def test_overview_prompt_includes_repository_source_evidence(+ model_gen, sample_repo_structure+):+ config = replace(+ model_gen._config,+ source_evidence_files={+ "repo_overview": ("README.md", "docs/ARCHITECTURE.md"),+ },+ )+ generator = PageGenerator(model_gen._provider, ContextAssembler(config), config)+ await generator.generate_repo_overview(+ sample_repo_structure,+ pagerank={},+ sccs=[],+ community={},+ repo_name="demo",+ source_map={+ "README.md": b"Demo turns source archives into indexed documentation.",+ "docs/ARCHITECTURE.md": b"The parser feeds a graph and then a wiki generator.",+ },+ )++ prompt = generator._provider.calls[-1]["user_prompt"]+ assert "## Additional repository evidence" in prompt+ assert '<repository-file path="README.md">' in prompt+ assert "turns source archives into indexed documentation" in prompt+ assert '<repository-file path="docs/ARCHITECTURE.md">' in prompt+++async def test_a_reused_overview_keeps_its_names_and_recompiles_the_map(+ naming_gen, sample_config, sample_repo_structure+):+ """A cache hit makes no call, keeps the stored names, and draws today's structure."""+ first = await _overview(naming_gen, sample_repo_structure, _map())+ assert first.metadata[SYSTEM_MAP_NAMES]["nodes"][0]["label"] == "Processing engine"+ provider = MockProvider()+ prior = {+ first.page_id: PriorPage(+ source_hash=first.source_hash,+ model_name=provider.model_name,+ content=first.content.replace('n_cli -->|"drives"|', 'n_cli -->|"stale"|'),+ metadata=first.metadata,+ )+ }+ reused = PageGenerator(+ provider, ContextAssembler(sample_config), sample_config, prior_pages=prior+ )+ page = await _overview(reused, sample_repo_structure, _map())+ assert provider.call_count == 0+ assert 'n_cli -->|"drives"| n_core_engine_py' in page.content+ assert "<b>Processing engine</b>" in page.content+ assert page.content.count("```mermaid") == 1+++async def test_outage_reuses_the_stored_names(sample_config, sample_repo_structure):+ prior = {+ "repo_overview:demo": PriorPage(+ source_hash="other",+ model_name="mock",+ content="",+ metadata={SYSTEM_MAP_NAMES: {"nodes": [{"id": "n_cli", "label": "Command line"}]}},+ )+ }+ gen = PageGenerator(+ _OutageProvider(), ContextAssembler(sample_config), sample_config, prior_pages=prior+ )+ page = await _overview(gen, sample_repo_structure, _map())+ assert "<b>Command line</b>" in page.content+++async def test_a_keyless_render_keeps_the_stored_names(sample_config, sample_repo_structure):+ names = {"nodes": [{"id": "n_cli", "label": "Command line"}]}+ prior = {+ "repo_overview:demo": PriorPage(+ source_hash="other", model_name="mock", content="", metadata={SYSTEM_MAP_NAMES: names}+ )+ }+ cfg = replace(sample_config, deterministic=True)+ gen = PageGenerator(MockProvider(), ContextAssembler(cfg), cfg, prior_pages=prior)+ page = await _overview(gen, sample_repo_structure, _map())+ assert page.metadata[SYSTEM_MAP_NAMES] == namesdiff --git a/tests/unit/generation/test_page_generator.py b/tests/unit/generation/test_page_generator.pyindex ebc2a8861..494dc285a 100644--- a/tests/unit/generation/test_page_generator.py+++ b/tests/unit/generation/test_page_generator.py@@ -33,7 +33,6 @@ from .conftest import _make_file_info, _make_symbolEXPECTED_PAGE_TYPES = ["module_page","repo_overview",- "architecture_diagram","onboarding",]diff --git a/tests/unit/generation/test_templates.py b/tests/unit/generation/test_templates.pyindex 755bcbb8a..29e31374d 100644--- a/tests/unit/generation/test_templates.py+++ b/tests/unit/generation/test_templates.py@@ -10,7 +10,6 @@ import pytestfrom repowise.core.generation.context_assembler import (ApiContractContext,- ArchitectureDiagramContext,FilePageContext,InfraPageContext,ModulePageContext,@@ -528,37 +527,6 @@ def test_spotlight_asks_about_importers_only_when_it_has_them(assert without.count("?") >= 2-# ----------------------------------------------------------------------------# architecture_diagram.j2-# -----------------------------------------------------------------------------[email protected](scope="module")-def architecture_diagram_ctx() -> ArchitectureDiagramContext:- return ArchitectureDiagramContext(- repo_name="my-repo",- nodes=["pkg/a.py", "pkg/b.py"],- edges=[("pkg/a.py", "pkg/b.py")],- communities={0: ["pkg/a.py"], 1: ["pkg/b.py"]},- scc_groups=[],- )---def test_architecture_diagram_renders_without_error(jinja_env, architecture_diagram_ctx):- result = render(jinja_env, "architecture_diagram.j2", architecture_diagram_ctx)- assert result---def test_architecture_diagram_has_heading(jinja_env, architecture_diagram_ctx):- result = render(jinja_env, "architecture_diagram.j2", architecture_diagram_ctx)- assert "##" in result---def test_architecture_diagram_mentions_mermaid(jinja_env, architecture_diagram_ctx):- result = render(jinja_env, "architecture_diagram.j2", architecture_diagram_ctx)- assert "mermaid" in result.lower()--# ---------------------------------------------------------------------------# api_contract.j2# ---------------------------------------------------------------------------