Skip to content

Graphs that teach > graphs that impress. Turn any code into an interactive knowledge graph you can explore, search, and ask questions about. Works with Claude Code, Codex, Cursor, Copilot, Gemini CLI, and more.

fix(lessons): batch detailed lesson dispatch and derive overview-only entries deterministically (#753)

06FallingLight90d3d65e5a9aee
diff --git a/understand-anything-plugin/agents/code-lesson-writer.md b/understand-anything-plugin/agents/code-lesson-writer.md
index 268f59e1..2e17d252 100644
--- a/understand-anything-plugin/agents/code-lesson-writer.md
+++ b/understand-anything-plugin/agents/code-lesson-writer.md
@@ -1,10 +1,9 @@
---
name: code-lesson-writer
description: |
- Writes per-file learning explanations for the dashboard code viewer. For files
- covered by the guided tour it reads the actual source and splits it into
- explained segments aligned with line ranges; for all other files it produces
- a brief overview from the provided node summaries.
+ Writes detailed learning explanations for the tour-covered files in the
+ dashboard code viewer. For each detailed file it reads the actual source and
+ splits it into explained segments aligned with line ranges.
---
# Code Lesson Writer
@@ -16,11 +15,13 @@ that matches the project's guided tour.
## Task
-Given the project's tour steps and file-level node summaries, produce a
-`code-lessons.json` document keyed by file path. Files referenced by the tour
-receive **detailed lessons** (overview + explained segments anchored to line
-ranges, read from the actual source). All other files receive **overview-only
-lessons** composed from the node summaries — do NOT read those files.
+Given the project's tour steps and the **detailed files** list, produce a
+`code-lessons.json` document keyed by file path. Every file in the detailed
+list receives a **detailed lesson**: an overview plus explained segments
+anchored to line ranges, read from the actual source. Do NOT write entries for
+any other file — overview-only entries for the remaining file-level nodes are
+generated deterministically from their node summaries by the calling skill's
+normalization step, not by you.
**Language directive:** If the dispatch prompt includes a language directive
(e.g., "Generate all textual content in **Chinese**"), apply it to every
@@ -30,7 +31,7 @@ translation). Use native-level terminology appropriate for technical education.
---
-## Detailed Lessons (tour-covered files)
+## Detailed Lessons (the dispatch's detailed files list)
For every file that appears in the dispatch prompt's **Detailed files** list:
@@ -55,13 +56,6 @@ Sections should collectively cover the significant parts of the file. Boilerplat
(import blocks of framework files, generated code) may be summarized in a single
section or skipped when genuinely uninformative.
-## Overview-Only Lessons (all other files)
-
-For every remaining file in the dispatch prompt's node list, write an
-`overview` (1-3 sentences) derived from that node's `summary` and the tour
-narrative, with an **empty `sections` array**. Do NOT read these files and do
-NOT invent line ranges.
-
## Output Format
Produce a single, valid JSON object:
@@ -81,10 +75,6 @@ Produce a single, valid JSON object:
"tourOrder": 2
}
]
- },
- "src/utils/format.ts": {
- "overview": "Small formatting helpers used across the UI...",
- "sections": []
}
}
}
@@ -94,7 +84,8 @@ Produce a single, valid JSON object:
- `version` — integer, currently `1`
- `generatedAt` — ISO 8601 timestamp of generation
- `files` — object keyed by **relative file path with forward slashes**, exactly
- matching the paths given in the dispatch prompt (never absolute paths)
+ matching the paths given in the dispatch prompt's detailed files list (never
+ absolute paths)
- Each entry: `overview` (string, required) and `sections` (array, required)
## Critical Constraints
@@ -104,12 +95,11 @@ Produce a single, valid JSON object:
- NEVER use absolute paths or backslashes in `files` keys.
- Sections within one file must not overlap and must have `startLine <= endLine`.
- At most 10 sections per file.
-- Only files listed in the dispatch prompt may appear in `files`.
-- Trust the provided tour steps and node summaries for project context; only
- tour-covered files require reading source from disk.
+- Only files listed in the dispatch prompt's detailed files list may appear in
+ `files`.
+- Trust the provided tour steps for project context.
- Explanations must be grounded in the code you read — do not fabricate
- behavior, APIs, or design rationale that is not visible in the source or the
- provided summaries.
+ behavior, APIs, or design rationale that is not visible in the source.
## Writing Results
@@ -118,7 +108,6 @@ After producing the JSON:
1. Write the JSON object to the exact output path given in your dispatch prompt
(normally `$UA_DIR/intermediate/code-lessons.json` inside the project's data
directory — `.ua/`, or the legacy `.understand-anything/` when present).
-2. Respond with ONLY a brief text summary: number of detailed lessons and
- number of overview-only lessons.
+2. Respond with ONLY a brief text summary: number of detailed lessons written.
Do NOT include the full JSON in your text response.
diff --git a/understand-anything-plugin/hooks/auto-update-prompt.md b/understand-anything-plugin/hooks/auto-update-prompt.md
index 13cb9add..33502836 100644
--- a/understand-anything-plugin/hooks/auto-update-prompt.md
+++ b/understand-anything-plugin/hooks/auto-update-prompt.md
@@ -88,7 +88,7 @@ Do not dispatch assemble-reviewer or graph-reviewer for automatic updates.
## Phase 3 — Conditional architecture and tour
- For `PARTIAL_UPDATE`, dispatch neither agent. The finalizer preserves existing layer assignments and tour prose, removes dangling references/empty layers, and assigns new nodes deterministically by directory depth, graph connectivity, then prior layer order.
-- For `ARCHITECTURE_UPDATE`, dispatch architecture-analyzer on the full merged node/edge set, including previous layers for naming stability, and write `$UA_DIR/intermediate/layers.json`. Then dispatch tour-builder and write `$UA_DIR/intermediate/tour.json`. Then normalize the tour deterministically — run `node "$PLUGIN_ROOT/skills/understand/normalize-tour.mjs" "$PROJECT_ROOT"` and use the resulting `$UA_DIR/intermediate/tour.normalized.json` for the lesson dispatch (the finalizer applies the same normalization at save time; hand-written conversion disagrees with it). Before dispatching the lesson writer, delete any stale `$UA_DIR/intermediate/code-lessons.json` and `$UA_DIR/intermediate/code-lessons.normalized.json` left by an interrupted earlier run (never touch the served `$UA_DIR/code-lessons.json`). Then dispatch `code-lesson-writer` (same dispatch shape as SKILL.md §Code Lessons: rebuilt tour, node summaries, language directive) and write `$UA_DIR/intermediate/code-lessons.json`, normalized into `$UA_DIR/intermediate/code-lessons.normalized.json` per the SKILL.md Phase 5 rules. Append `$LANGUAGE_DIRECTIVE` to the dispatch prompt when Phase 0 resolved one — without it, a project stored with a non-English `outputLanguage` would get its lessons regenerated in English beside the non-English graph and tour. Lessons are best-effort here: retry the dispatch once, and if it still fails leave no staged document, record the warning, and continue — publication (Phase 4) then removes the stale served sidecar instead of leaving lessons for the previous tour.
+- For `ARCHITECTURE_UPDATE`, dispatch architecture-analyzer on the full merged node/edge set, including previous layers for naming stability, and write `$UA_DIR/intermediate/layers.json`. Then dispatch tour-builder and write `$UA_DIR/intermediate/tour.json`. Then normalize the tour deterministically — run `node "$PLUGIN_ROOT/skills/understand/normalize-tour.mjs" "$PROJECT_ROOT"` and use the resulting `$UA_DIR/intermediate/tour.normalized.json` for the lesson dispatch (the finalizer applies the same normalization at save time; hand-written conversion disagrees with it). Before dispatching the lesson writer, delete any stale lesson scratch files left by an interrupted earlier run — `$UA_DIR/intermediate/code-lessons.json`, `code-lessons.detailed.<N>.json`, and `code-lessons.normalized.json` (never touch the served `$UA_DIR/code-lessons.json`). Then dispatch `code-lesson-writer` following the SKILL.md §Code Lessons contract (rebuilt tour, detailed files in bounded batches, language directive) and write the batch outputs, normalized into `$UA_DIR/intermediate/code-lessons.normalized.json` per the SKILL.md Phase 5 rules. Append `$LANGUAGE_DIRECTIVE` to each dispatch prompt when Phase 0 resolved one — without it, a project stored with a non-English `outputLanguage` would get its lessons regenerated in English beside the non-English graph and tour. Lessons are best-effort here: retry a failed batch once, and if any batch still fails leave no staged document, record the warning, and continue — publication (Phase 4) then removes the stale served sidecar instead of leaving lessons for the previous tour.
Retry either required agent once. If it still fails, **STOP** without advancing the baseline.
diff --git a/understand-anything-plugin/skills/understand/SKILL.md b/understand-anything-plugin/skills/understand/SKILL.md
index 315c30ea..a1d47e03 100644
--- a/understand-anything-plugin/skills/understand/SKILL.md
+++ b/understand-anything-plugin/skills/understand/SKILL.md
@@ -631,15 +631,15 @@ Run this step whenever the tour is (re)built — full analysis and incremental p
Report to the user: `[Phase 5/7] Writing code lessons...`
-0. **Clear stale lesson scratch files.** An interrupted earlier run can leave `$UA_DIR/intermediate/code-lessons.json` (raw) and `$UA_DIR/intermediate/code-lessons.normalized.json` (staged) behind — full-run initialization only creates the intermediate directory. Delete both files before every lesson attempt, or the next writer's missing output could let those stale scratch files be normalized and published as if they belonged to this run. Never touch the served `$UA_DIR/code-lessons.json` here.
+0. **Clear stale lesson scratch files.** An interrupted earlier run can leave lesson scratch files behind — `$UA_DIR/intermediate/code-lessons.json` (single-batch raw), `code-lessons.detailed.<N>.json` (batch raws), and `code-lessons.normalized.json` (staged) — full-run initialization only creates the intermediate directory. Delete all of them before every lesson attempt, or the next writer's missing output could let those stale scratch files be normalized and published as if they belonged to this run. Never touch the served `$UA_DIR/code-lessons.json` here.
1. **Resolve tour-covered files.** Read the normalized tour from `$UA_DIR/intermediate/tour.normalized.json` (written by the deterministic tour normalization above). For each normalized tour step, resolve every `nodeIds` entry to its file-level node and collect the node's `filePath`. Record for each covered file the smallest referencing step `order` as its primary tour anchor.
-2. **Dispatch a subagent** using the `code-lesson-writer` agent definition (at `agents/code-lesson-writer.md`). Pass these parameters in the dispatch prompt:
+2. **Dispatch the lesson writer in bounded batches.** A single dispatch carrying every file-level summary overflows one agent's context/output limits on large projects and loses ALL lessons when it fails, so the detailed files are dispatched in batches of at most 12, and overview-only entries are never dispatched at all (the normalization step derives them deterministically from node summaries). Use the `code-lesson-writer` agent definition (at `agents/code-lesson-writer.md`) and pass these parameters in each batch's dispatch prompt:
> Write per-file code lessons for this codebase.
> Project root: `$PROJECT_ROOT`
-> Write output to: `$UA_DIR/intermediate/code-lessons.json`
+> Write output to: `$UA_DIR/intermediate/code-lessons.detailed.<batchIndex>.json` (use `code-lessons.json` as the filename only when there is exactly one batch)
>
> Tour steps (from `$UA_DIR/intermediate/tour.normalized.json`):
> ```json
@@ -648,25 +648,24 @@ Report to the user: `[Phase 5/7] Writing code lessons...`
>
> Detailed files (read source from disk; produce overview + sections):
> ```json
-> [{"filePath": "src/main.tsx", "tourOrder": 2}, ...]
-> ```
->
-> All other file-level nodes (overview only — do NOT read these files):
-> ```json
-> [list of {filePath, summary} for every file-level node NOT in the detailed list]
+> [{"filePath": "src/main.tsx", "tourOrder": 2}, ...] (only this batch's files)
> ```
>
> $LANGUAGE_DIRECTIVE
-3. **Normalize** `$UA_DIR/intermediate/code-lessons.json` into a **staged** document at `$UA_DIR/intermediate/code-lessons.normalized.json`. Do NOT write `$UA_DIR/code-lessons.json` here — publication happens only after the graph/tour this run describes is actually saved (see **Lesson publication** under the incremental save gate below, and Phase 7 step 1 for full runs). Apply these steps **in order**:
- 1. **Unwrap envelope:** Accept both `{ "files": {...} }` and a bare files object at the top level.
- 2. **Drop unknown files:** Remove entries whose key does not exactly match a file-level node's `filePath` (relative, forward slashes).
- 3. **Validate sections:** `lineRange` must be two integers `[start, end]` with `1 <= start <= end`; drop invalid sections, then enforce a maximum of 10 sections per file (truncate beyond that).
- 4. **Validate tour anchors:** Drop any `tourOrder` that does not match a tour step's `order`.
- 5. **Fallback overview:** If `overview` is missing or empty, substitute the matching node's `summary`.
- 6. **Stamp metadata:** Ensure `version: 1` and `generatedAt` (current ISO 8601 timestamp) are present.
+Retry a failed batch once (see Error Handling). If any batch still fails, the whole lesson attempt counts as failed — see step 4.
+
+3. **Normalize** the batch output(s) into a **staged** document at `$UA_DIR/intermediate/code-lessons.normalized.json`. Do NOT write `$UA_DIR/code-lessons.json` here — publication happens only after the graph/tour this run describes is actually saved (see **Lesson publication** under the incremental save gate below, and Phase 7 step 1 for full runs). Apply these steps **in order**:
+ 1. **Unwrap envelope:** Accept both `{ "files": {...} }` and a bare files object at the top level of each batch document.
+ 2. **Merge batches:** Combine all `code-lessons.detailed.<N>.json` documents (and the single-batch `code-lessons.json`) into one files object. Batches cover disjoint file sets; if the same key nonetheless appears twice, keep the first occurrence and record a `$PHASE_WARNINGS` entry.
+ 3. **Drop unknown files:** Remove entries whose key does not exactly match a file-level node's `filePath` (relative, forward slashes).
+ 4. **Add deterministic overview-only entries:** For every file-level node that is NOT in the detailed files list, add an entry `{ "overview": <the node's summary>, "sections": [] }` when no entry exists yet — derived mechanically from the graph, never from an LLM.
+ 5. **Validate sections:** `lineRange` must be two integers `[start, end]` with `1 <= start <= end`; drop invalid sections, then enforce a maximum of 10 sections per file (truncate beyond that).
+ 6. **Validate tour anchors:** Drop any `tourOrder` that does not match a tour step's `order`.
+ 7. **Fallback overview:** If `overview` is missing or empty, substitute the matching node's `summary`.
+ 8. **Stamp metadata:** Ensure `version: 1` and `generatedAt` (current ISO 8601 timestamp) are present.
-4. **On failure:** retry the dispatch once (see Error Handling); if it still fails, skip lessons entirely — leave no staged document, record the failure in `$PHASE_WARNINGS`, and continue. At publication time the previously served `$UA_DIR/code-lessons.json` is removed in that case, so stale lessons never sit beside a rebuilt tour (see **Lesson publication** below). The dashboard degrades gracefully when the file is absent.
+4. **On failure:** if any batch failed after its one retry, skip lessons entirely — delete every scratch file this attempt produced (including per-batch outputs), leave no staged document, record the failure in `$PHASE_WARNINGS`, and continue. At publication time the previously served `$UA_DIR/code-lessons.json` is removed in that case, so stale lessons never sit beside a rebuilt tour (see **Lesson publication** below). The dashboard degrades gracefully when the file is absent.
### Incremental deterministic save gate