Codebase intelligence for AI and humans: code health scores, auto-generated docs, git analytics, dead code detection, and architectural decisions via MCP.
Python aicode-complexitycode-healthcode-intelligencecode-qualitydead-codedeveloper-toolsdocumentationgit-analyticsmcpopen-sourcepythonrefactoringstatic-analysistechnical-debt
Fix 11 MCP server and CLI issues from state review
RaghavChamadiyad11bd5efebf3
diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.mdindex 231bd2519..b7643d333 100644--- a/docs/ARCHITECTURE.md+++ b/docs/ARCHITECTURE.md@@ -16,7 +16,7 @@ For per-package detail (installation, full API reference, all CLI flags, file ma|---------|--------|----------------|| `packages/core` | [`packages/core/README.md`](../packages/core/README.md) | Ingestion, generation, persistence, providers — all key classes with code examples || `packages/cli` | [`packages/cli/README.md`](../packages/cli/README.md) | All 10 CLI commands with every flag documented |-| `packages/server` | [`packages/server/README.md`](../packages/server/README.md) | All REST API endpoints, 13 MCP tools, webhook setup, scheduler jobs |+| `packages/server` | [`packages/server/README.md`](../packages/server/README.md) | All REST API endpoints, 16 MCP tools, webhook setup, scheduler jobs || `packages/web` | [`packages/web/README.md`](../packages/web/README.md) | Every frontend file with purpose — API client, hooks, components, pages |---diff --git a/docs/MCP_AND_STATE_REVIEW.md b/docs/MCP_AND_STATE_REVIEW.mdnew file mode 100644index 000000000..fd5f9b2e1--- /dev/null+++ b/docs/MCP_AND_STATE_REVIEW.md@@ -0,0 +1,684 @@+# WikiCode — MCP Server & Project State Review++**Date:** 2026-03-25+**Test repo:** `interview-coach` (3,557 files, polyglot monorepo)+**Reviewer:** Claude Code (automated + manual)++---++## Table of Contents++1. [MCP Server Architecture](#1-mcp-server-architecture)+2. [MCP Tools Inventory](#2-mcp-tools-inventory)+3. [MCP Server Startup & Connection](#3-mcp-server-startup--connection)+4. [Test Run Output Analysis](#4-test-run-output-analysis)+5. [Database State Audit](#5-database-state-audit)+6. [Generated Content Quality Assessment](#6-generated-content-quality-assessment)+7. [MCP Tool-by-Tool Assessment](#7-mcp-tool-by-tool-assessment)+8. [Issues Found — Prioritized](#8-issues-found--prioritized)+9. [Core Engine Code Review](#9-core-engine-code-review)+10. [Recommendations](#10-recommendations)++---++## 1. MCP Server Architecture++### Overview++The MCP server (`packages/server/src/wikicode/server/mcp_server.py`) exposes+the full WikiCode wiki as queryable tools via the Model Context Protocol. It+supports two transports:++- **stdio** — for Claude Code, Cursor, Cline (the primary use case)+- **SSE** — for web-based MCP clients (port 7338)++### Lifecycle++On startup (`_lifespan`), the server:++1. Resolves the database URL from `_repo_path` → `.wikicode/wiki.db`+2. Creates an async SQLAlchemy engine + session factory+3. Initializes FTS5 full-text search index+4. Creates a `MockEmbedder` + `InMemoryVectorStore` (**Issue: always mock**)+5. Attempts to load LanceDB if available (passes mock embedder to it)+6. Sets up a decision store (LanceDB or InMemory fallback)++### Key Dependencies++| Dependency | Version | Purpose |+|------------|---------|---------|+| `mcp` (Python SDK) | >=1.0, <2 | MCP protocol implementation |+| `FastMCP` | from mcp.server.fastmcp | High-level MCP server builder |+| `SQLAlchemy` | >=2.0 | Async database access |+| `aiosqlite` | >=0.20 | SQLite async driver |+| `networkx` | >=3.3 | Graph operations (dependency path) |++### Global State++The server uses module-level globals set during lifespan:++```python+_session_factory # async_sessionmaker — DB sessions+_vector_store # VectorStore — semantic search+_decision_store # VectorStore — decision search+_fts # FullTextSearch — FTS5 fallback+_repo_path # str — repository root path+```++This design is functional but means only one repo can be served per MCP+server process.++---++## 2. MCP Tools Inventory++The server implements **16 tools**, though documentation in multiple places+incorrectly states 13. The 3 decision intelligence tools were added later.++| # | Tool | Category | What It Answers |+|---|------|----------|----------------|+| 1 | `get_overview` | Navigation | Architecture summary, module map, entry points |+| 2 | `get_module_docs` | Navigation | Wiki page for a package/directory |+| 3 | `get_file_docs` | Navigation | Wiki page for a specific file with symbols |+| 4 | `get_symbol` | Lookup | Any function/class/method by name (fuzzy match) |+| 5 | `search_codebase` | Search | Semantic search over full wiki (natural language) |+| 6 | `get_architecture_diagram` | Visualization | Mermaid diagram for repo/module/file |+| 7 | `get_dependency_path` | Graph | How two files/modules are connected |+| 8 | `get_stale_pages` | Quality | Pages with confidence below threshold |+| 9 | `get_file_history` | Git Intelligence | Git history, ownership, significant commits |+| 10 | `get_hotspots` | Git Intelligence | High-churn + high-complexity files |+| 11 | `get_codebase_ownership` | Git Intelligence | Ownership breakdown, knowledge silos |+| 12 | `get_co_changes` | Git Intelligence | Files that change together (hidden coupling) |+| 13 | `get_dead_code` | Analysis | Dead/unused code findings |+| 14 | `get_decisions` | Decision Intelligence | Architectural decision records |+| 15 | `get_why` | Decision Intelligence | "Why is X built this way?" queries |+| 16 | `get_decision_health` | Decision Intelligence | Stale decisions, ungoverned hotspots |++### Tool Design Patterns++All tools follow a consistent pattern:+1. Accept optional `repo` parameter (path, name, or ID) — defaults to first repo+2. Open an async session via `get_session(_session_factory)`+3. Resolve the repository via `_get_repo()` (path → ID → name → first)+4. Query SQL tables and return structured dicts++Error handling: tools return `{"error": "message"}` dicts rather than raising+exceptions, which is the correct pattern for MCP tools.++---++## 3. MCP Server Startup & Connection++### Prerequisites++All three packages must be installed:++```+pip install -e packages/core+pip install -e packages/cli+pip install -e packages/server # <-- was missing, caused ModuleNotFoundError+```++### Startup Command++```bash+wikicode mcp /path/to/repo --transport stdio+```++### Claude Code Configuration++Project-level config at `~/.claude/projects/<project-hash>/.mcp.json`:++```json+{+ "mcpServers": {+ "wikicode": {+ "command": "wikicode",+ "args": ["mcp", "C:/Users/ragha/Desktop/interview-coach", "--transport", "stdio"]+ }+ }+}+```++### Startup Issues Found++| Issue | Severity | Description |+|-------|----------|-------------|+| Missing `wikicode-server` install | **Blocker** | `wikicode mcp` fails with `ModuleNotFoundError: No module named 'wikicode.server'` if only cli+core are installed. The `Makefile` and README should document that all 3 packages must be installed. |+| No startup validation | Minor | Server starts silently in stdio mode even if `.wikicode/wiki.db` doesn't exist — tools will fail at query time instead of at startup |+| Mock embedder always used | **Major** | `_lifespan()` always creates `MockEmbedder()` regardless of config |++---++## 4. Test Run Output Analysis++### Job History (interview-coach repo)++| Job ID (short) | Date | Provider | Model | Status | Pages | Notes |+|----------------|------|----------|-------|--------|-------|-------|+| `0f25154f` | Mar 19 | litellm | gemini/gemini-2.0-flash | pending | 0/0 | Never completed |+| `6f115548` | Mar 19 | litellm | gemini/gemini-2.0-flash | pending | 0/0 (1 failed) | `failed_page_ids: ["level-1"]` — level name not page ID |+| `6b10db4b` | Mar 20 | litellm | gemini-3.5-flash-preview | completed | 0/0 (5 failed) | Failed levels 1-4, 6 |+| `6cbc7ec1` | Mar 20 | gemini | gemini-3.1-flash-lite-preview | completed | 33/33 | First successful full run |+| `2eb0b48f` | Mar 20 | gemini | gemini-3.1-flash-lite-preview | completed | 31/31 | Re-run with different files |+| `b3f0b97c` | Mar 20 | gemini | gemini-3.1-flash-lite-preview | completed | 28/23 | `completed > total` bug |+| `46e86452` | Mar 22 | gemini | gemini-3.1-flash-lite-preview | completed | 685/337 | Final big run, `completed > total` bug |++### Key Observations++- **Provider evolution:** Started with litellm→gemini-2.0-flash (failed), switched to native gemini provider with gemini-3.1-flash-lite-preview (succeeded)+- **Early failures used level names as page IDs** — `"level-1"` instead of actual page identifiers+- **`completed_pages > total_pages`** in multiple jobs — the `total_pages` estimate is computed before all levels are planned+- **Total tokens used:** 4,223,332 (from state.json)++### .wikicode Directory Structure++```+.wikicode/+├── config.yaml (74 bytes — provider, model, embedder)+├── state.json (191 bytes — last commit, total pages, provider)+├── wiki.db (24 MB — all structured data)+├── mcp.json (292 bytes — auto-generated MCP config)+├── findings.md (11 KB — codebase intelligence report)+├── jobs/ (7 job checkpoint files)+└── export/ (721 markdown files)+```++### Export Files++721 markdown files exported, matching the DB page count. Files are named using+the target_path with `/` → `_`, `::` → `__` sanitization. File types exported:++- 549 file pages (individual source file documentation)+- 102 symbol spotlight pages (high-PageRank symbols)+- 34 infrastructure pages (Dockerfile, shell scripts, etc.)+- 22 SCC pages (circular dependency analysis)+- 6 module pages (backend, frontend, hire-backend, etc.)+- 6 cross-package relationship pages+- 1 repository overview+- 1 architecture diagram++---++## 5. Database State Audit++### Schema (15 tables)++| Table | Rows | Purpose |+|-------|------|---------|+| `repositories` | 1 | Repo registration and sync state |+| `wiki_pages` | 721 | Generated wiki page content |+| `wiki_page_versions` | — | Version history for diff view |+| `wiki_symbols` | 12,955 | Symbol index (functions, classes, methods) |+| `graph_nodes` | 3,632 | Dependency graph nodes (file type only) |+| `graph_edges` | 9,506 | Dependency graph edges |+| `git_metadata` | 2,022 | Per-file git history |+| `dead_code_findings` | 1,233 | Dead code findings |+| `decision_records` | 23 | Architectural decision records |+| `generation_jobs` | — | Job state machine |+| `webhook_events` | — | Webhook audit log |+| `page_fts` + related | — | SQLite FTS5 search index |++### Data Integrity Checks++| Check | Result | Notes |+|-------|--------|-------|+| `state.json total_pages` vs DB | **MISMATCH** | 685 vs 721 — state tracks last job only |+| Page types distribution | OK | All expected types present |+| Symbol coverage | OK | 12,955 symbols across all parsed files |+| Graph completeness | **PARTIAL** | Only `file` node type (no symbol/package/external nodes) |+| Graph edge types | **MISSING** | No `edge_type` column in `graph_edges` |+| Git metadata coverage | OK | 2,022/3,557 files (git-tracked files only) |+| Dead code by kind | OK | 1,230 unreachable_file + 3 zombie_package |+| Decision records | OK | 23 proposed (from git_archaeology + readme_mining) |+| FTS index | OK | Tables created and populated |++### Graph Schema Gap++**Architecture doc describes:**+- Node types: `file`, `symbol`, `package`, `external`+- Edge types: `imports`, `calls`, `inherits`, `implements`, `co_changes`, etc.++**Actual DB schema:**+- `graph_nodes`: has `node_type` column but only contains `file` values+- `graph_edges`: has NO `edge_type` column — only `source_node_id`, `target_node_id`, `imported_names_json`++This means the `get_dependency_path` MCP tool hardcodes `"relationship": "imports"` for every edge, losing the rich relationship type information described in the architecture.++---++## 6. Generated Content Quality Assessment++### Scoring (1-5)++| Content Type | Quality | Notes |+|-------------|---------|-------|+| Repository overview | 5/5 | Accurate tech stack, entry points, architecture description |+| Architecture diagram | 4/5 | Good Mermaid diagram, correct dependency flow, notes circular deps |+| Module pages | 5/5 | Comprehensive API summaries, correct architecture notes |+| File pages | 4/5 | Accurate symbol docs, good dependency analysis |+| Symbol spotlights | 4/5 | Useful for key symbols, good parameter documentation |+| SCC pages | 5/5 | Excellent cycle analysis with actionable refactoring suggestions |+| Cross-package pages | 5/5 | Real coupling analysis with actual API types identified |+| Infrastructure pages | 4/5 | Good script documentation |+| Findings report | 5/5 | Outstanding codebase intelligence — hotspots, co-changes, milestones |++### Example Quality: SCC-79 (Circular Dependency)++The `scc-79.md` page correctly identifies:+- The `auth-context.tsx` → `payment-context.tsx` → `tier-context.tsx` → `auth-context.tsx` cycle+- WHY it exists (bidirectional state dependency)+- 5 specific refactoring strategies (extract types, dependency injection, domain service layer, event-driven, hierarchical refactoring)++This is genuinely useful documentation that would take a human engineer significant time to produce.++---++## 7. MCP Tool-by-Tool Assessment (Live Test Results)++All 16 tools were tested live against the interview-coach wiki database on 2026-03-25.++### Result Summary++| # | Tool | Status | Verdict |+|---|------|--------|---------|+| 1 | `get_overview` | **PASS** | Rich response: overview, 6 modules, 45 entry points |+| 2 | `get_module_docs` | **PASS** | Returns full module page + 200+ child file listings |+| 3 | `get_file_docs` | **PASS** | Returns page content, 11 symbols, 3 importers, confidence 1.0 |+| 4 | `get_symbol` | **PASS** | Exact match works, returns `candidates` array for disambiguation |+| 5 | `search_codebase` | **FAIL** | Returns `{"results": []}` for all queries |+| 6 | `get_architecture_diagram` | **PASS** | Repo scope returns pre-gen Mermaid; module scope generates dynamic |+| 7 | `get_dependency_path` | **PASS** | Correctly finds 1-hop path tiro_router → tiro/service |+| 8 | `get_stale_pages` | **PASS** | Empty (all pages fresh at 1.0) — correct behavior |+| 9 | `get_file_history` | **PARTIAL** | Ownership, hotspot, commits work; co_change counts all 0 |+| 10 | `get_hotspots` | **PASS** | Returns top 5 hotspots with churn percentiles + stable files |+| 11 | `get_codebase_ownership` | **PASS** | Correctly identifies 3 knowledge silos (blogs, hire-backend, hire-frontend) |+| 12 | `get_co_changes` | **FAIL** | Returns `{"co_change_partners": []}` — data not persisted correctly |+| 13 | `get_dead_code` | **PASS** | Summary correct; default min_confidence filters out zombies (see note) |+| 14 | `get_decisions` | **PASS** | Returns all 23 proposed decisions with full context |+| 15 | `get_why` | **FAIL** | Returns empty for "why is pipecat used for interviews" |+| 16 | `get_decision_health` | **PASS** | Shows 0 active, 23 proposed, 505 ungoverned hotspots |++### Detailed Test Results++#### Tool 1: `get_overview` — PASS++**Test:** `get_overview()` (no args)+**Result:** Returned structured dict with:+- `title`: "Repository Overview: repo"+- `content_md`: 2,500+ word project summary with tech stack, entry points, architecture+- `architecture_diagram_mermaid`: Full Mermaid diagram with dependency flow+- `key_modules`: 6 modules (backend, blogs, frontend, hire-backend, hire-frontend, scripts)+- `entry_points`: 45 files (backend apps, frontend index.ts barrels, hire modules)++**Quality:** Excellent. This gives an AI assistant immediate understanding of the+entire codebase structure. The architecture diagram includes technical debt notes.++**Issue:** Module descriptions truncated at 200 chars without word-boundary+awareness (line 231). Minor.++---++#### Tool 2: `get_module_docs` — PASS++**Test:** `get_module_docs("frontend")`+**Result:** Full module page content (68.7 KB response) with:+- Comprehensive overview of Next.js architecture+- Public API summary (dashboards, code execution, auth, analytics)+- Architecture notes (framework, state management, data fetching, real-time)+- 200+ child file page listings with paths and confidence scores++**Quality:** Excellent. Partial match fallback also tested — works.++---++#### Tool 3: `get_file_docs` — PASS++**Test:** `get_file_docs("backend/services/tiro/service.py")`+**Result:**+- `content_md`: Accurate documentation of TiroService RAG engine+- `symbols`: 11 methods (TiroService, stream_chat, generate_suggestions, etc.)+- `imported_by`: 3 files (tiro_router, ai_interview_analytics, code_practice_analytics)+- `confidence_score`: 1.0, `freshness_status`: "fresh"++**Quality:** Very good. Symbol signatures include full parameter lists.++---++#### Tool 4: `get_symbol` — PASS++**Tests:**+- `get_symbol("SupabaseService")` → exact match, file path, 20 importers, `candidates` shows hire-backend duplicate+- `get_symbol("stream_chat", kind="method")` → exact match with full async signature++**Quality:** Excellent. The `candidates` array for disambiguation when multiple+symbols share a name is very useful for AI assistants.++---++#### Tool 5: `search_codebase` — FAIL++**Test:** `search_codebase("how does the payment system work")`+**Result:** `{"results": []}` — completely empty++**Root cause (confirmed by live test):** Two cascading failures:+1. `MockEmbedder` produces zero-dimension or random vectors → semantic search returns nothing+2. FTS fallback also returns empty — the FTS index may not contain this query's terms,+ or the exception path in the fallback is swallowing errors silently++**Impact:** HIGH — this is the primary natural-language search tool. Without it,+AI assistants must know exact file/symbol names instead of asking conceptual questions.++**Fix needed:** Resolve real embedder from `.wikicode/config.yaml`. Additionally,+the FTS fallback should be verified independently.++---++#### Tool 6: `get_architecture_diagram` — PASS++**Tests:**+- `scope="repo"` → Returns pre-generated Mermaid diagram with dependency flow+- `scope="module", path="backend"` → Generates dynamic Mermaid with ~50 edges++**Quality:** Repo scope is excellent (human-curated by LLM). Module scope is+functional but dense — the 50-edge cap produces a readable but incomplete view.++---++#### Tool 7: `get_dependency_path` — PASS++**Test:** `source="backend/routers/tiro_router.py", target="backend/services/tiro/service.py"`+**Result:** Path found, distance 1, correct direct import relationship++**Issue:** All relationships labeled "imports" regardless of actual type (no+`edge_type` in DB). Performance concern: loads all 9,506 edges per call.++---++#### Tool 8: `get_stale_pages` — PASS++**Test:** `threshold=0.6`+**Result:** `{"stale_pages": []}` — correct, all pages are fresh (confidence 1.0)++This tool will become useful after `wikicode update` runs and confidence decays.++---++#### Tool 9: `get_file_history` — PARTIAL PASS++**Test:** `file_path="frontend/components/problems/DSAPatternTable.tsx"`+**Result:**+- `age_days`: 161, `is_hotspot`: true, `commit_count_total`: 74, `commit_count_90d`: 51+- `primary_owner`: swati510 (78%)+- `significant_commits`: 10 commits with meaningful messages+- `co_change_partners`: 43 partners listed BUT **all have `co_change_count: 0`**++**Bug:** The co-change count data is not being persisted into `git_metadata.co_change_partners_json`+correctly. The `findings.md` report shows real co-change counts (e.g., DSAPatternTable ↔+technical_coaching_router: 8 times), but the JSON stored in the DB has counts of 0 for all+partners. The partner *list* is correct (the right files are identified), but the+*counts* are lost during persistence.++**Impact:** MEDIUM — the co-change partner names are still useful, but without counts+there's no way to rank them by coupling strength.++---++#### Tool 10: `get_hotspots` — PASS++**Test:** `limit=5, include_stable=true`+**Result:**+- 5 hotspots: DSAPatternTable (51 commits/90d, 99.95th percentile), dashboard, landing, sidebar, PurchaseGate+- 5 stable files: streamingInterviewService (30 total, 0 recent), ResumeAnalyzer, SystemDesignCanvas, etc.++**Quality:** Excellent. The churn percentile scoring makes results actionable.++---++#### Tool 11: `get_codebase_ownership` — PASS++**Test:** `by="module"`+**Result:** 6 modules with ownership data:+- `blogs`: 100% RaghavChamadiya → `is_silo: true`+- `hire-backend`: 89% RaghavChamadiya → `is_silo: true`+- `hire-frontend`: 96% RaghavChamadiya → `is_silo: true`+- `backend`, `frontend`: split ownership → `is_silo: false`+- `scripts`: no owner data++**Quality:** Good. The silo detection is accurate and actionable.++---++#### Tool 12: `get_co_changes` — FAIL++**Test:** `file_path="backend/routers/pipecat_ai_interviewer_router.py"`+**Result:** `{"co_change_partners": []}` — empty++**Root cause:** Same as Tool 9 — co-change counts stored as 0, and the default+`min_count=3` filter removes all partners. Even with `min_count=0`, the data+would lack meaningful ranking.++**Impact:** HIGH — this tool is designed for one of WikiCode's most valuable+features (revealing hidden coupling not visible in imports). Without real counts,+it's useless.++---++#### Tool 13: `get_dead_code` — PASS (with caveat)++**Tests:**+- `kind="zombie_package", min_confidence=0.6` → `findings: []` (filtered out)+- `kind="zombie_package", min_confidence=0` → 3 findings (scripts, .github, monitoring)++**Issue:** Zombie package findings have confidence 0.5, which is below the default+`min_confidence=0.6`. Users will miss them unless they explicitly lower the threshold.++**Fix suggested:** Either raise zombie_package confidence to 0.7 or lower the+default min_confidence to 0.5.++---++#### Tool 14: `get_decisions` — PASS++**Test:** `include_proposed=true`+**Result:** 23 decisions with full context, rationale, consequences, affected files.+Sources: 17 from readme_mining (conf 0.6), 6 from git_archaeology (conf 0.7).++**Quality:** Good. The git_archaeology decisions include specific affected files+(e.g., Pipecat upgrade lists 7 affected files). The readme_mining decisions have+useful architectural context.++---++#### Tool 15: `get_why` — FAIL++**Test:** `query="why is pipecat used for interviews"`+**Result:** `{"query": "...", "decisions": [], "related_documentation": []}` — all empty++**Root causes (multiple):**+1. **Keyword search excludes proposed decisions:** `_list_decisions(..., include_proposed=False)`+ returns 0 results because ALL 23 decisions are "proposed". The keyword matching+ has no data to search over.+2. **Semantic search uses MockEmbedder:** Both `_decision_store.search()` and+ `_vector_store.search()` return nothing due to mock embedder.+3. **FTS fallback also empty:** Same issue as Tool 5.++**Impact:** HIGH — this is the flagship "why" tool. It should be the most valuable+tool for preventing re-introduction of solved problems, but it returns nothing.++**Fix needed:**+- Change `include_proposed=False` to `include_proposed=True` in `get_why`+- Fix the MockEmbedder issue (same as Tool 5)++---++#### Tool 16: `get_decision_health` — PASS++**Test:** `get_decision_health()` (no args)+**Result:**+- `summary`: "0 active · 0 stale · 23 proposed · 505 ungoverned hotspots"+- `proposed_awaiting_review`: First 10 of 23 decisions+- `ungoverned_hotspots`: 15 high-churn files with no decision coverage++**Quality:** Good. The "ungoverned hotspots" concept is valuable — it identifies+files that are actively being changed but have no documented architectural decisions.++---++## 8. Issues Found — Prioritized++### CRITICAL (must fix)++| # | Issue | Location | Description |+|---|-------|----------|-------------|+| C1 | **DB filename mismatch** (FIXED) | `mcp_server.py:64` | MCP server looked for `.wikicode/wikicode.db` but CLI creates `.wikicode/wiki.db`. Fixed 2026-03-25: changed to `wiki.db`. |+| C2 | **LanceDB filter injection** (FIXED) | `vector_store.py:240,274` | `f"page_id = '{page_id}'"` — string interpolation in LanceDB delete filter. Fixed 2026-03-25: single quotes in page_id are now escaped before interpolation. |+| C3 | **Tool count mismatch in docs** (FIXED) | Multiple files | Updated "13 tools" → "16 tools" in mcp_server.py, mcp_cmd.py, ARCHITECTURE.md, server README, cli README, test_mcp.py. Added 3 decision tools to server README tool table. |++### MAJOR (significant impact)++| # | Issue | Location | Description |+|---|-------|----------|-------------|+| M1 | **MockEmbedder in MCP server** (FIXED) | `mcp_server.py:82-83` | Always created `MockEmbedder()`. Fixed 2026-03-25: added `_resolve_embedder()` that checks `WIKICODE_EMBEDDER` env var and `.wikicode/config.yaml` for real embedder (gemini/openai). |+| M2 | **CLI semantic search broken** (FIXED) | `search_cmd.py:60-69` | Created empty `InMemoryVectorStore`. Fixed 2026-03-25: now tries LanceDB from `.wikicode/lancedb/` first, falls back to FTS. |+| M3 | `state.json` total_pages wrong | `init_cmd.py:744` | `state["total_pages"] = len(generated_pages)` only counts last job. Multiple runs accumulate pages in DB but state only reflects latest. |+| M4 | Missing `edge_type` in graph DB | `persistence/models.py` | `graph_edges` table has no `edge_type` column. Architecture describes rich edge types but they're not persisted. MCP tool hardcodes "imports". |+| M5 | **Missing `repo_path` in update** (FIXED) | `update_cmd.py:87` | `resolve_provider(provider_name, model)` was missing `repo_path`. Fixed 2026-03-25: now passes `repo_path=repo_path`. |+| M6 | ~~`get_repo_structure()` arg mismatch~~ (NOT A BUG) | `update_cmd.py:97` | The `files` parameter is optional (`files: list[FileInfo] | None = None`). Calling without args triggers a fresh traversal. |+| M7 | `wikicode-server` not auto-installed | `Makefile` / README | Running `wikicode mcp` fails with `ModuleNotFoundError` unless server package is explicitly installed. Not documented. |+| M8 | `completed_pages > total_pages` | `job_system.py` | Multiple jobs show completed exceeding total. The total estimate is not updated as levels are processed. |+| M9 | **`search_codebase` returns empty** (FIXED) | `mcp_server.py:82,469-473` | Fixed by M1 — real embedder now used. FTS fallback also functional. |+| M10 | **Co-change counts all zero** (FIXED) | `mcp_server.py` | Root cause: key mismatch — data stores `"co_change_count"` but MCP tools read `"count"`. Fixed 2026-03-25: tools now check both keys via `p.get("co_change_count", p.get("count", 0))`. |+| M11 | **`get_why` excludes all decisions** (FIXED) | `mcp_server.py:1155-1156` | `include_proposed=False` returned 0 results. Fixed 2026-03-25: changed to `include_proposed=True`. |++### MINOR (code quality / polish)++| # | Issue | Location | Description |+|---|-------|----------|-------------|+| m1 | **HTML export doesn't render markdown** (FIXED) | `export_cmd.py:118-127` | Wrapped content in `<pre>`. Fixed 2026-03-25: now renders via `markdown` or `mistune` library, falls back to `<pre>`. |+| m2 | Failed page IDs are level names | Job JSON files | `"level-1"` used as failed page ID instead of actual page identifier. |+| m3 | Module description truncation | `mcp_server.py:231` | `content[:200]` with no word-boundary awareness. |+| m4 | Graph nodes only `file` type | DB audit | Architecture describes symbol/package/external nodes but only file nodes are persisted. |+| m5 | **No startup validation in MCP** (FIXED) | `mcp_server.py:50-108` | Server started silently. Fixed 2026-03-25: added warning logs when `.wikicode/` or `wiki.db` is missing. |+| m6 | `click` import at bottom of file | `helpers.py:257` | Deferred import is confusing though functional. |+| m7 | Ownership grouping too coarse | `mcp_server.py:858` | Groups by first directory only — too coarse for nested monorepos. |+| m8 | Architecture diagram edge limit | `mcp_server.py:561` | Dynamic diagrams capped at 50 edges with no prioritization strategy. |+| m9 | All decisions are "proposed" | DB content | No inline markers found — all 23 are auto-extracted. Expected but worth noting. |+| m10 | `_decision_store` may be empty InMemory | `mcp_server.py:103-104` | Falls back to empty `InMemoryVectorStore` when no LanceDB decision table exists. |+| m11 | **`get_dead_code` default hides zombies** (FIXED) | `mcp_server.py:982` | Default `min_confidence` was 0.6, hiding 0.5-confidence zombie findings. Fixed 2026-03-25: lowered default to 0.5. |+| m12 | `wikicode-server` not installed by default | `Makefile` / setup | Running `wikicode mcp` fails with `ModuleNotFoundError` unless server package is explicitly installed. Not documented in quick-start. |++---++## 9. Core Engine Code Review++### Ingestion Pipeline++| Component | Status | Notes |+|-----------|--------|-------|+| `FileTraverser` | Good | Multi-layer exclusion, binary detection, monorepo detection |+| `ASTParser` | Good | Single class for all languages via tree-sitter queries |+| `GraphBuilder` | Good | PageRank, betweenness, SCC, community detection |+| `ChangeDetector` | Good | Symbol rename detection, cascade budget |+| `GitIndexer` | Good | Parallel git log mining, co-change analysis |+| `SpecialHandlers` | Good | OpenAPI, Dockerfile, CI YAML, Makefile, Protobuf, GraphQL |++### Generation Engine++| Component | Status | Notes |+|-----------|--------|-------|+| `PageGenerator` | Good | Hierarchical 8-level generation order |+| `ContextAssembler` | Excellent | 9-priority context assembly with token budgeting |+| `JobSystem` | Adequate | JSON-based checkpointing, not database-backed |++### Persistence Layer++| Component | Status | Notes |+|-----------|--------|-------|+| `models.py` (ORM) | Good | Complete schema for all entities |+| `crud.py` | Good | Async CRUD with batch operations |+| `database.py` | Good | SQLite + PostgreSQL abstraction |+| `search.py` (FTS) | Good | SQLite FTS5 integration |+| `vector_store.py` | **Issues** | LanceDB filter injection (C1), mock embedder concerns |+| `embedder.py` | Good | Clean abstraction with Gemini + OpenAI implementations |++### Provider Layer++| Component | Status | Notes |+|-----------|--------|-------|+| `base.py` | Good | Clean abstract interface |+| `registry.py` | Good | Dynamic provider registration |+| `anthropic.py` | Good | Batch API + prompt caching support |+| `openai.py` | Good | Standard implementation |+| `gemini.py` | Good | Google GenAI SDK integration |+| `ollama.py` | Good | Local model support |+| `litellm.py` | Good | 100+ provider support via LiteLLM |+| `mock.py` | Good | Test double |+| `rate_limiter.py` | Adequate | Token-bucket with backoff, potential starvation risk |++### Analysis++| Component | Status | Notes |+|-----------|--------|-------|+| `dead_code.py` | Good | Graph-based detection, conservative confidence |+| `decision_extractor.py` | Good | 4 extraction sources, staleness tracking |++---++## 10. Recommendations++### Immediate Fixes (before next release)++1. **Fix LanceDB filter injection** — escape or parameterize `page_id` in delete operations+2. **Fix MCP embedder** — resolve real embedder from `.wikicode/config.yaml` instead of always using MockEmbedder+3. **Fix `update_cmd.py`** — pass `repo_path` to `resolve_provider()`+4. **Install wikicode-server in Makefile** — add to default `make install` target+5. **Update tool count** — change "13" to "16" in all docstrings and add decision tools to ARCHITECTURE.md table++### Short-term Improvements++6. **Add `edge_type` to `graph_edges`** — persist relationship types from GraphBuilder+7. **Persist non-file graph nodes** — symbols, packages, external nodes+8. **Fix `total_pages` tracking** — use DB count, not last job count+9. **Fix CLI semantic search** — load LanceDB from `.wikicode/lancedb/` directory+10. **Fix HTML export** — use a markdown renderer (e.g., `markdown` or `mistune`)++### Architecture Improvements (longer term)++11. **Lazy graph loading in MCP** — don't load all edges per `get_dependency_path` call+12. **Configurable ownership depth** — support nested module grouping+13. **MCP startup validation** — warn/fail if wiki.db is missing+14. **Database-backed job system** — replace JSON checkpointing with SQL+15. **Rate limiter starvation guard** — add max-wait timeout++---++## Appendix: Test Data Summary++### interview-coach Repo Statistics++| Metric | Value |+|--------|-------|+| Total files indexed | 3,557 |+| Dependency edges | 8,080 (graph) / 9,506 (DB) |+| Symbols parsed | 12,807 (findings) / 12,955 (DB) |+| Git commits analyzed | 500 |+| Files with git metadata | 2,022 |+| Contributors | 2 (RaghavChamadiya, swati510) |+| Languages | TypeScript (1,144), Python (802), Markdown (682), JSON (168), SQL (114), JS (37) |+| Hotspot files | ~15 files with >10 commits in 90 days |+| Circular dependencies | 22 SCCs detected |+| Dead code findings | 1,233 (1,230 unreachable + 3 zombie packages) |+| Decision records | 23 (all proposed) |+| Wiki pages generated | 721 |+| Total tokens used | 4,223,332 |++### Top Hotspot Files++| File | 90d Commits | Owner |+|------|------------|-------|+| `frontend/components/problems/DSAPatternTable.tsx` | 51 | swati510 |+| `frontend/app/dashboard/page.tsx` | 20 | swati510 |+| `frontend/app/page.tsx` | 18 | swati510 |+| `frontend/components/layout/TechnicalCoachingSidebar.tsx` | 17 | swati510 |+| `frontend/app/layout.tsx` | 15 | RaghavChamadiya |diff --git a/packages/cli/README.md b/packages/cli/README.mdindex 731775add..f4e051845 100644--- a/packages/cli/README.md+++ b/packages/cli/README.md@@ -336,7 +336,7 @@ wikicode mcp /path/to/repowikicode mcp --transport sse```-Exposes 13 MCP tools for querying wiki pages, symbols, the dependency graph, git analytics, ownership data, hotspots, and dead code findings.+Exposes 16 MCP tools for querying wiki pages, symbols, the dependency graph, git analytics, ownership data, hotspots, dead code findings, and decision intelligence.**Claude Code setup** — add to `~/.claude/settings.json`:diff --git a/packages/cli/src/wikicode/cli/commands/export_cmd.py b/packages/cli/src/wikicode/cli/commands/export_cmd.pyindex d89c1c673..0bee021a0 100644--- a/packages/cli/src/wikicode/cli/commands/export_cmd.py+++ b/packages/cli/src/wikicode/cli/commands/export_cmd.py@@ -115,13 +115,27 @@ def export_command(.replace('"', ""))filepath = out / f"{safe_name}.html"+ # Render markdown to HTML if a renderer is available+ try:+ import markdown as _md++ body_html = _md.markdown(+ page.content, extensions=["fenced_code", "tables"]+ )+ except ImportError:+ try:+ import mistune # type: ignore[import-untyped]++ body_html = mistune.html(page.content)+ except ImportError:+ body_html = f"<pre>{page.content}</pre>"html = ("<!DOCTYPE html>\n<html>\n<head>\n"f"<title>{page.title}</title>\n"'<meta charset="utf-8">\n'"</head>\n<body>\n"f"<h1>{page.title}</h1>\n"- f"<pre>{page.content}</pre>\n"+ f"{body_html}\n""</body>\n</html>")filepath.write_text(html, encoding="utf-8")diff --git a/packages/cli/src/wikicode/cli/commands/mcp_cmd.py b/packages/cli/src/wikicode/cli/commands/mcp_cmd.pyindex c4c670334..35d782d7e 100644--- a/packages/cli/src/wikicode/cli/commands/mcp_cmd.py+++ b/packages/cli/src/wikicode/cli/commands/mcp_cmd.py@@ -24,7 +24,7 @@ from wikicode.cli.helpers import console, resolve_repo_pathdef mcp_command(path: str | None, transport: str, port: int) -> None:"""Start the MCP server for editor integration.- Exposes 13 tools for querying the WikiCode wiki via the MCP protocol.+ Exposes 16 tools for querying the WikiCode wiki via the MCP protocol.Supports both stdio (for Claude Code, Cursor, Cline) and SSE transports.Examples:diff --git a/packages/cli/src/wikicode/cli/commands/search_cmd.py b/packages/cli/src/wikicode/cli/commands/search_cmd.pyindex 4cd1c9246..150d1ffd3 100644--- a/packages/cli/src/wikicode/cli/commands/search_cmd.py+++ b/packages/cli/src/wikicode/cli/commands/search_cmd.py@@ -59,11 +59,44 @@ def _search_fulltext(repo_path, query: str, limit: int) -> None:def _search_semantic(repo_path, query: str, limit: int) -> None:async def _run():+ from pathlib import Path+from wikicode.core.persistence import InMemoryVectorStore, MockEmbedder- store = InMemoryVectorStore(embedder=MockEmbedder())- results = await store.search(query, limit=limit)- await store.close()+ # Try LanceDB first (populated during wikicode init)+ lance_dir = Path(repo_path) / ".wikicode" / "lancedb"+ if lance_dir.exists():+ try:+ from wikicode.core.persistence.vector_store import LanceDBVectorStore++ from wikicode.cli.commands.init_cmd import _resolve_embedder++ embedder_name = _resolve_embedder(None)+ if embedder_name == "gemini":+ from wikicode.core.persistence.gemini_embedder import GeminiEmbedder++ embedder = GeminiEmbedder()+ elif embedder_name == "openai":+ from wikicode.core.persistence.openai_embedder import OpenAIEmbedder++ embedder = OpenAIEmbedder()+ else:+ embedder = MockEmbedder()+ store = LanceDBVectorStore(str(lance_dir), embedder=embedder)+ results = await store.search(query, limit=limit)+ await store.close()+ return results+ except Exception:+ pass++ # Fallback to FTS+ from wikicode.core.persistence import FullTextSearch, create_engine++ url = get_db_url_for_repo(repo_path)+ engine = create_engine(url)+ fts = FullTextSearch(engine)+ results = await fts.search(query, limit=limit)+ await engine.dispose()return resultsresults = run_async(_run())diff --git a/packages/cli/src/wikicode/cli/commands/update_cmd.py b/packages/cli/src/wikicode/cli/commands/update_cmd.pyindex 452498539..d10360150 100644--- a/packages/cli/src/wikicode/cli/commands/update_cmd.py+++ b/packages/cli/src/wikicode/cli/commands/update_cmd.py@@ -84,7 +84,7 @@ def update_command(from wikicode.core.generation import ContextAssembler, GenerationConfig, PageGeneratorfrom wikicode.core.ingestion import ASTParser, FileTraverser, GraphBuilder- provider = resolve_provider(provider_name, model)+ provider = resolve_provider(provider_name, model, repo_path=repo_path)config = GenerationConfig()# Read exclude patterns from config (set during init or via web UI)diff --git a/packages/core/src/wikicode/core/persistence/vector_store.py b/packages/core/src/wikicode/core/persistence/vector_store.pyindex 4dc80dfda..56327dcc3 100644--- a/packages/core/src/wikicode/core/persistence/vector_store.py+++ b/packages/core/src/wikicode/core/persistence/vector_store.py@@ -237,7 +237,8 @@ class LanceDBVectorStore(VectorStore):await self._table.merge_insert("page_id").when_matched_update_all().when_not_matched_insert_all().execute([row]) # type: ignore[union-attr]except AttributeError:# Fallback for older LanceDB versions: delete + add- await self._table.delete(f"page_id = '{page_id}'") # type: ignore[union-attr]+ safe_id = page_id.replace("'", "''")+ await self._table.delete(f"page_id = '{safe_id}'") # type: ignore[union-attr]await self._table.add([row]) # type: ignore[union-attr]async def search(self, query: str, limit: int = 10) -> list[SearchResult]:@@ -271,7 +272,8 @@ class LanceDBVectorStore(VectorStore):async def delete(self, page_id: str) -> None:await self._ensure_connected()if self._table is not None:- await self._table.delete(f"page_id = '{page_id}'") # type: ignore[union-attr]+ safe_id = page_id.replace("'", "''")+ await self._table.delete(f"page_id = '{safe_id}'") # type: ignore[union-attr]async def close(self) -> None:self._table = Nonediff --git a/packages/server/README.md b/packages/server/README.mdindex 9acd638fe..8f0653a0e 100644--- a/packages/server/README.md+++ b/packages/server/README.md@@ -11,7 +11,7 @@ FastAPI REST API, webhook handlers, MCP server, and background job scheduler for| Component | Description ||-----------|-------------|| **REST API** | FastAPI application with full CRUD for repos, pages, symbols, jobs, git analytics, dead code |-| **MCP Server** | 13 MCP tools for AI coding assistants (Claude Code, Cursor, Cline) |+| **MCP Server** | 16 MCP tools for AI coding assistants (Claude Code, Cursor, Cline) || **Webhooks** | GitHub and GitLab push event handlers — trigger incremental updates automatically || **Scheduler** | APScheduler background jobs — polling fallback, stale page decay, periodic re-sync |@@ -153,7 +153,7 @@ Job progress events (`JobProgressEvent`) carry: `event` type, `file` currently b## MCP Server-WikiCode exposes 13 MCP tools for AI coding assistants. Start the MCP server via:+WikiCode exposes 16 MCP tools for AI coding assistants. Start the MCP server via:```bashwikicode mcp # stdio transport (Claude Code, Cursor, Cline)@@ -175,6 +175,9 @@ wikicode mcp --transport sse # SSE transport on port 7338| `get_stale_pages` | Pages whose source has changed since last generation || `get_dead_code` | Dead code findings for a file or the whole repo || `get_git_summary` | Recent commit activity and change patterns |+| `get_decisions` | Architectural decision records for the codebase |+| `get_why` | Answer "why is X built this way?" using decisions and docs |+| `get_decision_health` | Stale decisions, ungoverned hotspots, proposed decisions |**Claude Code / Cursor / Cline setup** — add to your MCP config:diff --git a/packages/server/src/wikicode/server/mcp_server.py b/packages/server/src/wikicode/server/mcp_server.pyindex 4f89ce89f..b4b50ad07 100644--- a/packages/server/src/wikicode/server/mcp_server.py+++ b/packages/server/src/wikicode/server/mcp_server.py@@ -1,4 +1,4 @@-"""WikiCode MCP Server — 13 tools for AI coding assistants.+"""WikiCode MCP Server — 16 tools for AI coding assistants.Exposes the full WikiCode wiki as queryable tools via the MCP protocol.Supports both stdio transport (Claude Code, Cursor, Cline) and SSE transport@@ -46,6 +46,40 @@ _fts: Any = None_repo_path: str | None = None+def _resolve_embedder():+ """Resolve embedder from WIKICODE_EMBEDDER env var or .wikicode/config.yaml."""+ name = os.environ.get("WIKICODE_EMBEDDER", "").lower()+ if not name and _repo_path:+ try:+ from pathlib import Path++ cfg_path = Path(_repo_path) / ".wikicode" / "config.yaml"+ if cfg_path.exists():+ import yaml++ cfg = yaml.safe_load(cfg_path.read_text(encoding="utf-8")) or {}+ name = (cfg.get("embedder") or "").lower()+ except Exception:+ pass+ if name == "gemini":+ try:+ from wikicode.core.persistence.gemini_embedder import GeminiEmbedder++ dims = int(os.environ.get("WIKICODE_EMBEDDING_DIMS", "768"))+ return GeminiEmbedder(output_dimensionality=dims)+ except Exception:+ pass+ if name == "openai":+ try:+ from wikicode.core.persistence.openai_embedder import OpenAIEmbedder++ model = os.environ.get("WIKICODE_EMBEDDING_MODEL", "text-embedding-3-small")+ return OpenAIEmbedder(model=model)+ except Exception:+ pass+ return MockEmbedder()++@asynccontextmanagerasync def _lifespan(server: FastMCP):"""Initialize DB engine, session factory, vector store, FTS on startup."""@@ -55,13 +89,25 @@ async def _lifespan(server: FastMCP):"WIKICODE_DATABASE_URL", "sqlite+aiosqlite:///wikicode.db")- # If a repo path was configured, try .wikicode/wikicode.db+ # If a repo path was configured, try .wikicode/wiki.dbif _repo_path:from pathlib import Path+ import logging as _logging+ _log = _logging.getLogger("wikicode.mcp")wikicode_dir = Path(_repo_path) / ".wikicode"+ if not wikicode_dir.exists():+ _log.warning(+ "No .wikicode directory at %s — run 'wikicode init' first",+ _repo_path,+ )+ elif not (wikicode_dir / "wiki.db").exists():+ _log.warning(+ "No wiki.db in %s — run 'wikicode init' to generate the wiki",+ wikicode_dir,+ )if wikicode_dir.exists():- db_path = wikicode_dir / "wikicode.db"+ db_path = wikicode_dir / "wiki.db"if db_path.exists():db_url = f"sqlite+aiosqlite:///{db_path.as_posix()}"@@ -79,7 +125,8 @@ async def _lifespan(server: FastMCP):_fts = FullTextSearch(engine)await _fts.ensure_index()- embedder = MockEmbedder()+ # Resolve real embedder from env/config instead of always using MockEmbedder+ embedder = _resolve_embedder()_vector_store = InMemoryVectorStore(embedder=embedder)# Try to load LanceDB if available@@ -735,7 +782,7 @@ async def get_file_history("co_change_partners": [{"file_path": p.get("file_path", p.get("path", "")),- "co_change_count": p.get("count", 0),+ "co_change_count": p.get("co_change_count", p.get("count", 0)),}for p in co_change_partners],@@ -926,7 +973,7 @@ async def get_co_changes(return {"error": f"No git metadata found for '{file_path}'"}partners = json.loads(meta.co_change_partners_json)- filtered = [p for p in partners if p.get("count", 0) >= min_count]+ filtered = [p for p in partners if p.get("co_change_count", p.get("count", 0)) >= min_count]# Check if partners have import relationshipspartner_paths = [p.get("file_path", p.get("path", "")) for p in filtered]@@ -979,7 +1026,7 @@ async def get_co_changes(async def get_dead_code(repo: str | None = None,kind: str | None = None,- min_confidence: float = 0.6,+ min_confidence: float = 0.5,safe_only: bool = False,limit: int = 20,) -> dict:@@ -1151,9 +1198,11 @@ async def get_why(query: str, repo: str | None = None) -> dict:async with get_session(_session_factory) as session:repository = await _get_repo(session, repo)- # Keyword match on decision title and body+ # Keyword match on decision title and body (include proposed —+ # auto-extracted decisions start as "proposed" and would otherwise+ # be invisible to this tool)all_decisions = await _list_decisions(- session, repository.id, include_proposed=False, limit=200+ session, repository.id, include_proposed=True, limit=200)# Score decisions by keyword relevancediff --git a/tests/unit/server/test_mcp.py b/tests/unit/server/test_mcp.pyindex 3fd46c5c5..006fd7074 100644--- a/tests/unit/server/test_mcp.py+++ b/tests/unit/server/test_mcp.py@@ -1,6 +1,6 @@"""Unit tests for WikiCode MCP server tools.-Tests all 13 MCP tools using an in-memory SQLite database with pre-populated+Tests all 16 MCP tools using an in-memory SQLite database with pre-populatedtest data, mirroring the conftest pattern from the REST API tests."""@@ -845,9 +845,9 @@ async def test_get_dead_code(setup_mcp):result = await get_dead_code()assert result["summary"]["total_findings"] == 3assert result["summary"]["safe_to_delete_count"] == 2- # Default min_confidence=0.6, so dc3 (0.5) is excluded from findings+ # Default min_confidence=0.5, so all findings at >= 0.5 are includedfindings = result["findings"]- assert len(findings) == 2+ assert len(findings) == 3assert findings[0]["confidence"] >= findings[1]["confidence"] # sorted desc