Skip to content

Turn any idea, plan, or codebase into a beautiful interactive diagram. An agent skill for Claude Code, Codex, and more.

feat(skill): make the tuning skill usable as a handoff

tt-a1i2dd870513f6d
diff --git a/.agents/skills/archify-tuning/SKILL.md b/.agents/skills/archify-tuning/SKILL.md
index 3745855b..9cbef7ee 100644
--- a/.agents/skills/archify-tuning/SKILL.md
+++ b/.agents/skills/archify-tuning/SKILL.md
@@ -18,39 +18,55 @@ A change that makes gates pass while the diagram reads worse, or that makes agen
## Principles
- **The first draft is the unit of measure.** Bundled examples carry hand-tuned widths and routes, so they hide defects that every fresh draft hits.
-- **Deleted content is the accuracy signal.** A pass reached by removing an error state or merging lanes is a failure that counts as a success; check what the final candidate dropped.
+- **Deleted content is the accuracy signal.** A pass reached by removing an error state or merging lanes is a failure that looks like a success.
- **Automatic layout must pass its own gates.** When a fully automatic draft (no `via`, sides, channels or label coordinates) fails composition checks, the renderer is at fault, not the agent.
- **Early diagnostics must match the browser gate.** A compile-time check stricter than `browser-check` is a false failure; a looser one defers the failure to the most expensive stage.
-- **Traces explain more than outcomes.** Read where the agent was misled by a diagnostic or a document, not only whether it passed.
+- **Traces explain more than outcomes.** Read where a diagnostic or document misled the agent, not only whether it passed.
## Measures
| Level | Measure | Source |
|---|---|---|
-| Primary | Benchmark first-draft pass rate; content kept from first draft to final | `replay.mjs`; `collect.mjs` |
+| Primary | Benchmark first-draft pass rate, per type | `replay.mjs` |
+| Primary | Content kept from first draft to final | `collect.mjs`, "lost" column |
| Secondary | Finalize failures and tool calls per diagram | `collect.mjs` on a new round |
-| Visual | Crossings, bends, shared corridors, text size, then screenshots | Receipts and a browser |
-| Guard | No benchmark draft or historical sample that passed now fails; every changed output inspected | `replay.mjs`, golden, Gallery |
+| Visual | Crossings, bends, shared corridors and text size, then screenshots | Receipts, `shots.mjs` |
+| Guard | No benchmark draft that passed on base fails on head; every changed output inspected | `replay.mjs`, golden, Gallery |
-A new round on a new project finds problems and shows a change broke nothing. It does not prove improvement: projects differ too much. Prove improvement on the benchmark, or by repeating one project two or three times.
+A new round on a new project finds problems and shows a change broke nothing. It does not prove improvement, because projects differ too much: prove improvement on the benchmark, or by repeating one project two or three times.
+
+## Setup
+
+- **Tuning home**: `$ARCHIFY_TUNING_HOME`, default `~/.local/share/archify-tuning`. It holds `journal.md`, `bench/`, `rounds/` and `replays/`. It stays outside the repository because drafts carry content from source projects, which may be private: never commit it.
+- **Journal**: read `<tuning-home>/journal.md` before starting. It records the branch under tuning, the latest benchmark result, the open problems with evidence and the decisions already taken. Update it at the end of every iteration.
+- **Head**: the worktree you change, in a durable directory. Run `npm ci` there so the full test suite can run.
+- **Base**: a separate worktree of the comparison point, usually `dev`: `git worktree add <dir> origin/dev`. Pass its `archify` directory as `--base`.
+- **Tools**: Node 22.13 or later (`collect.mjs` uses `node:sqlite` on Devin CLI session history; other agents need their own trace export) and Chrome or Chromium for `shots.mjs` (`ARCHIFY_CHROME` overrides discovery).
+
+Scripts live in `.agents/skills/archify-tuning/scripts/`. `round`, `collect` and `replay` print their usage when run without arguments; `shots` uses the latest replay.
## Loop
-1. **Generate.** Pick a project not used before. Start one background subagent per diagram type with the prompt below and the same wording every round. Never edit the Archify tree the subagents use while they run; experiment in a separate worktree.
-2. **Collect.** `node .agents/skills/archify-tuning/scripts/collect.mjs <round-root> --round <n>-<project> --dump` prints per-type tool calls, finalize runs and failures, diagnostic codes and the content lost, writes `<type>.trace.txt` and adds each first draft to the benchmark. Cross-check its counts with the subagents' own reports.
-3. **Attribute** each failure to exactly one cause:
+1. **Baseline.** `node scripts/replay.mjs --base <base>/archify` and record the per-type pass rate in the journal.
+2. **Generate.** Pick a project not used before (see the journal). `node scripts/round.mjs <project-path> [--types ...]` creates `<tuning-home>/rounds/<round>/` and `prompts.json`. Start each prompt as its own background subagent, unchanged. The Archify tree they use must stay untouched until every subagent finishes: experiment in a separate worktree meanwhile.
+3. **Collect.** `node scripts/collect.mjs <round> --dump` prints, per type, tool calls, finalize runs and failures, whether the first draft was fully automatic (`draft`), what the final candidate dropped (`lost`) and the diagnostic codes. It writes `<type>.trace.txt` and `<type>/first-draft.json`, and adds each first draft to the benchmark. Cross-check its counts with the subagents' reports.
+4. **Attribute** every failure to exactly one cause, using [Trace signals](references/trace-signals.md):
| Cause | Test | Response |
|---|---|---|
| Renderer defect | A fully automatic draft fails its own composition checks | Fix the renderer |
| Misleading diagnostic | Following the message fails or edits the wrong element | Fix its fix text or evidence |
- | Documentation gap | The agent asks something only an unread reference answers | Move the fact into a required read, with numbers |
+ | Documentation gap | The agent needs a fact found only outside the required reads | Move the fact into a required read, with numbers |
| Authoring error | The required reads already say it | Nothing, or one sentence |
-4. **Prioritise** by how many rounds and types show it, times its harm to accuracy, over its cost. A problem seen every round needs a systematic fix, not one patch per round.
-5. **Fix** the root cause once, in the shared path. Preserve authored geometry and pins; when the renderer cannot repair a draft itself, return a verified, actionable fix. Reject a change that only moves a metric, such as trading crossings for shared corridors.
-6. **Verify.** Add a minimal test that fails before the fix. Run `node .agents/skills/archify-tuning/scripts/replay.mjs --base <dev-worktree>/archify` and expect no regressions. Find affected tests by searching for the changed functions and message text, not by file name. Regenerate examples and Gallery when output changes, run golden and inspect screenshots of changed outputs.
-7. **Record.** Commit code, regenerated outputs and documentation separately. Integrate regularly so the full suite and CI run; do not let local commits pile up.
+5. **Prioritise** by how many rounds and types show the problem, times its harm to accuracy, over its cost. A problem seen every round needs a systematic fix, not one patch per round. Record the ranking in the journal.
+6. **Fix** the root cause once, in the shared path. Preserve authored geometry and pins; when the renderer cannot repair a draft itself, return a verified, actionable fix. Reject a change that only moves a metric, such as trading crossings for shared corridors.
+7. **Verify.**
+ - Add a minimal test that fails before the fix.
+ - Replay: `node scripts/replay.mjs --base <base>/archify`. Any REGRESSED entry blocks the change; investigate every FIXED and changed entry.
+ - `node scripts/shots.mjs` and look at every pair it captures.
+ - Find affected tests by searching for the changed functions and message text, not by file name, and run them. Regenerate examples and Gallery when output changes, then run golden.
+8. **Record.** Commit code, regenerated outputs and documentation separately. Update the journal: benchmark result, problems closed and opened, approaches rejected. Integrate regularly through [CONTRIBUTING.md](../../../CONTRIBUTING.md) so the full suite and CI run. This skill does not authorise pushing, opening pull requests or installing Archify.
## Diagnostics and documentation
@@ -58,24 +74,10 @@ A new round on a new project finds problems and shows a change broke nothing. It
- A compact receipt lists how many diagnostics of each code it omitted.
- Documentation answers questions traces show agents asking, in a required read, with measured numbers. Keep one paragraph per mode and every line under the 2,000-character read limit.
-## Subagent prompt
-
-```text
-画 <project-path> 这个项目的<图类型>。
-
-Use the Archify skill located at <archify-tree> (read <archify-tree>/SKILL.md first and follow it exactly, including the references it tells you to read). Do NOT install, update, or copy Archify anywhere; use the CLI directly as `node <archify-tree>/bin/archify.mjs ...`.
-
-Constraints:
-- Your working directory for all Archify commands and outputs is <round-root>/<type> (cd there; the skill's `.archify/<type>-<slug>-<timestamp>/` folder must be created under it, not inside the project).
-- <project-path> is read-only source evidence. Do not create, modify, or delete any file in it (ignore its pre-existing untracked files). Pass it as `--repo-root` where the skill requires.
-- Do not modify anything under <archify-tree>.
-- Follow the skill's repair loop until finalize passes or the skill tells you to stop.
-
-When done, reply concisely with: diagram type, absolute path of the final candidate JSON and final HTML, whether finalize passed (key status lines), and every repair iteration (diagnostic code + what you changed). Also list any point where the skill docs or CLI messages were unclear, contradictory, or sent you the wrong way.
-```
-
-`<round-root>/<type>` must appear verbatim in the prompt: `collect.mjs` finds each subagent by it.
-
-## Benchmark
+## Anti-patterns
-The benchmark lives outside the repository, in `$ARCHIFY_TUNING_BENCH` or `~/.local/share/archify-tuning/bench`, because first drafts carry content from the source projects, which may be private. Each entry is `<type>/<round>.json` with repository evidence stripped; `index.json` records its project, revision and first finalize result. Never commit benchmark drafts. `collect.mjs` reads Devin CLI session history through `node:sqlite` (Node 22.13 or later); other agents need their own trace export.
+- Counting failures while ignoring deleted content.
+- Claiming an improvement from one round on a new project.
+- Patching the same problem a little every round.
+- Updating a pinned test value without checking that the diagram got better.
+- Growing documentation until agents cannot read it whole.
diff --git a/.agents/skills/archify-tuning/references/trace-signals.md b/.agents/skills/archify-tuning/references/trace-signals.md
new file mode 100644
index 00000000..00c58f0a
--- /dev/null
+++ b/.agents/skills/archify-tuning/references/trace-signals.md
@@ -0,0 +1,35 @@
+# Trace signals
+
+Read `<round>/<type>.trace.txt` with the subagent's own report. Each signal below points at a likely cause; confirm it on the first draft before changing anything.
+
+| Signal in the trace | Likely cause | Check |
+|---|---|---|
+| The final candidate dropped nodes, edges or lanes the first draft had | The agent bought a pass with content | `collect.mjs` "lost" column, then compare the drafts. Highest priority |
+| A fully automatic draft (`draft: auto`) fails crossing, corridor, through-node or label checks | Renderer defect | Render the first draft at `--quality standard` and look at the routes |
+| A crossing or corridor failure, then side pins, then `explicit-pin-conflict`, then a restructure | The router has no legal route the agent can request | Same as above; the hint that suggested pins is also wrong |
+| The same code survives a focused repair | The fix text names a change that cannot help | Apply the advertised fix to the draft and rerun |
+| Width or gap values oscillate across runs, such as `gapX` 64, 108, 72 | The diagnostic omits the target value | Make it state the required width, gap or character budget |
+| The agent writes `meta.viewBox` after a width message | The message implies a path the canvas budget forbids | Make it state the reachable limit and say to omit `viewBox` |
+| The agent renders a scratch copy at `standard`, reads renderer source or greps references for a rule | Evidence or documentation is missing from the required path | Add the geometry to the diagnostic, or the fact to a required read |
+| `[truncated N chars]` on a read of a reference | A documentation line exceeds the read limit | Split it |
+| Two or more subagents list the same unclear point | Documentation gap | Fix the document, not the agent |
+| A rerun is blocked by stale evidence or output paths | Delivery friction | Give the exact command in the message |
+| A pass on the first run | Nothing to fix | Still check the screenshot: a pass is not a good diagram |
+
+## Root causes found so far
+
+Use these as patterns; each was confirmed by replaying the first draft.
+
+- Workflow same-lane neighbours were placed 8px apart, so no route could pass between columns. Fully automatic drafts failed on crossings and shared corridors, and agents removed lanes and nodes. Neighbours now keep a 32px corridor and routes may take offset tracks.
+- Dataflow flows across several stages turned at their midpoint, inside a middle stage's node. Agents deleted nodes. They now turn in the nearest clear gap.
+- Dataflow ignored perpendicular pinned sides, so agents hand-wrote `via` points.
+- Sequence canvases stayed 920px wide whatever the participant labels needed, and the message suggested widening `viewBox`, which the readability budget forbids.
+- Workflow diagnostics named edges by an internal sort order, so agents edited the wrong edge.
+- Architecture candidate selection ignored the number of bends, and legends reserved width at a smaller font than they rendered.
+- A compile-time text floor applied to tags, which `browser-check` exempts: a self-inflicted false failure, found two rounds late because the affected test file was not run.
+
+## Rejected approaches
+
+- Routing Workflow edges by role and span order: more crossings than the canonical order.
+- Ripping up and replanning conflicting Workflow routes after the first pass: it traded crossings for shared corridors and made no draft pass.
+- Shrinking or removing content automatically to fit a budget: it hides the accuracy loss this skill exists to prevent.
diff --git a/.agents/skills/archify-tuning/scripts/collect.mjs b/.agents/skills/archify-tuning/scripts/collect.mjs
index 6387a6e0..6b36b0ff 100644
--- a/.agents/skills/archify-tuning/scripts/collect.mjs
+++ b/.agents/skills/archify-tuning/scripts/collect.mjs
@@ -1,30 +1,32 @@
#!/usr/bin/env node
-// Usage: node collect.mjs <round-root> [--bench <dir>] [--round <name>] [--no-bench] [--dump]
+// Usage: node collect.mjs <round-name-or-path> [--round <name>] [--no-bench] [--dump]
//
// Reads Devin CLI session history for subagents whose task pointed at
-// <round-root>/<type>, then reports per type: tool calls, finalize runs and
-// failures, diagnostic codes, and the elements and relationships the final
-// candidate dropped from the first draft. The first draft of each type is
-// added to the benchmark (repository evidence stripped) unless --no-bench.
+// <round>/<type>, then reports per type: tool calls, finalize runs and
+// failures, diagnostic codes, whether the first draft was fully automatic and
+// the elements and relationships the final candidate dropped from it. Each
+// first draft is kept as <round>/<type>/first-draft.json and added to the
+// benchmark (repository evidence stripped) unless --no-bench.
import { DatabaseSync } from 'node:sqlite';
import fs from 'node:fs';
import os from 'node:os';
import path from 'node:path';
-import { DIAGRAM_TYPES, benchmarkDocument, contentLost } from './diagram-shape.mjs';
+import { DIAGRAM_TYPES, authoredControls, benchmarkDocument, contentLost, finalizeStatus, tuningHome } from './lib.mjs';
const args = process.argv.slice(2);
const option = (name, fallback) => {
const index = args.indexOf(name);
return index === -1 ? fallback : args[index + 1];
};
-const root = path.resolve(args.find((arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--')) || '');
-if (!args.length || !fs.existsSync(root)) {
- console.error('Usage: node collect.mjs <round-root> [--bench <dir>] [--round <name>] [--no-bench] [--dump]');
+const target = args.find((arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--')) || '';
+const root = fs.existsSync(path.resolve(target)) ? path.resolve(target) : path.join(tuningHome(), 'rounds', target);
+if (!target || !fs.existsSync(root)) {
+ console.error('Usage: node collect.mjs <round-name-or-path> [--round <name>] [--no-bench] [--dump]');
process.exit(2);
}
-const round = option('--round', path.basename(root));
-const bench = path.resolve(option('--bench', process.env.ARCHIFY_TUNING_BENCH
- || path.join(os.homedir(), '.local/share/archify-tuning/bench')));
+const recorded = fs.existsSync(path.join(root, 'round.json')) ? JSON.parse(fs.readFileSync(path.join(root, 'round.json'), 'utf8')) : {};
+const round = option('--round', recorded.name || path.basename(root));
+const bench = path.join(tuningHome(), 'bench');
const database = path.join(os.homedir(), '.local/share/devin/cli/sessions.db');
const db = new DatabaseSync(database, { readOnly: true });
@@ -34,15 +36,6 @@ const escape = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
const roots = [...new Set([root, root.replace(/^\/private\//, '/'), root.replace(/^\/tmp\//, '/private/tmp/')])];
const CODE = /\b((?:layout|composition|clean-flow|workflow|sequence|dataflow|lifecycle|erd|class|tree|timeline|waterfall|architecture|repository-evidence|schema|viewer|delivery)\/[a-z0-9]+(?:-[a-z0-9]+)*)\b/g;
-// Agents often pipe finalize through jq, so the exit code is not evidence.
-function finalizeStatus(result) {
- if (/"ok":\s*false|"status":\s*"fail"|"(?:validate|deliver|check|browser-check)":\s*"fail"/.test(result)) return 'fail';
- if (/"ok":\s*true|"status":\s*"pass"/.test(result)
- || /"validate":\s*"pass"[^}]*"browser-check":\s*"pass"/.test(result)) return 'pass';
- const exit = result.match(/Exit code: (\d+)/);
- return exit ? (exit[1] === '0' ? 'unknown' : 'fail') : 'unknown';
-}
-
function chainsByType() {
const like = roots.map(() => 'chat_message like ?').join(' or ');
const hits = db.prepare(`select session_id, min(node_id) as first from message_nodes where ${like} group by session_id`)
@@ -113,10 +106,12 @@ for (const [type, chain] of [...chainsByType()].sort(([a], [b]) => DIAGRAM_TYPES
finalizeUnknown: stats.finalize.filter(({ status }) => status === 'unknown').length,
firstFinalize: stats.finalize[0]?.status ?? 'not-run',
codes: [...new Set(stats.finalize.flatMap(({ codes }) => codes))],
+ controls: stats.firstDraft ? authoredControls(type, stats.firstDraft) : null,
lost,
docReads: stats.docReads,
};
if (args.includes('--dump')) fs.writeFileSync(path.join(root, `${type}.trace.txt`), stats.dump.join('\n\n'));
+ if (stats.firstDraft) fs.writeFileSync(path.join(root, type, 'first-draft.json'), `${JSON.stringify(stats.firstDraft, null, 2)}\n`);
if (stats.firstDraft && !args.includes('--no-bench')) {
fs.mkdirSync(path.join(bench, type), { recursive: true });
fs.writeFileSync(path.join(bench, type, `${round}.json`), `${JSON.stringify(benchmarkDocument(stats.firstDraft, round), null, 2)}\n`);
@@ -135,11 +130,12 @@ if (!args.includes('--no-bench') && Object.keys(index).length) {
fs.writeFileSync(path.join(root, 'tuning-summary.json'), `${JSON.stringify(summary, null, 2)}\n`);
const pad = (value, width) => String(value).padEnd(width);
-console.log(`${pad('type', 13)}${pad('tools', 7)}${pad('finalize', 10)}${pad('failed', 8)}${pad('first', 9)}lost (elements/relations) codes`);
+console.log(`${pad('type', 13)}${pad('tools', 7)}${pad('finalize', 10)}${pad('failed', 8)}${pad('first', 9)}${pad('draft', 8)}lost (elements/relations) codes`);
for (const [type, entry] of Object.entries(summary)) {
const lost = entry.lost ? `${entry.lost.elements.length}/${entry.lost.relations.length}${entry.lost.elements.length ? ` ${entry.lost.elements.join(',')}` : ''}` : 'n/a';
const failed = `${entry.finalizeFailures}${entry.finalizeUnknown ? `+${entry.finalizeUnknown}?` : ''}`;
- console.log(`${pad(type, 13)}${pad(entry.tools, 7)}${pad(entry.finalizeRuns, 10)}${pad(failed, 8)}${pad(entry.firstFinalize, 9)}${pad(lost, 27)}${entry.codes.join(',')}`);
+ const draft = !entry.controls ? 'n/a' : Object.keys(entry.controls).length ? 'pinned' : 'auto';
+ console.log(`${pad(type, 13)}${pad(entry.tools, 7)}${pad(entry.finalizeRuns, 10)}${pad(failed, 8)}${pad(entry.firstFinalize, 9)}${pad(draft, 8)}${pad(lost, 27)}${entry.codes.join(',')}`);
}
const entries = Object.values(summary);
console.log(`TOTAL tools ${entries.reduce((sum, entry) => sum + entry.tools, 0)}, finalize ${entries.reduce((sum, entry) => sum + entry.finalizeRuns, 0)}, failed ${entries.reduce((sum, entry) => sum + entry.finalizeFailures, 0)}, first-run passes ${entries.filter((entry) => entry.firstFinalize === 'pass').length}/${entries.length}`);
diff --git a/.agents/skills/archify-tuning/scripts/diagram-shape.mjs b/.agents/skills/archify-tuning/scripts/diagram-shape.mjs
deleted file mode 100644
index e008fdfd..00000000
--- a/.agents/skills/archify-tuning/scripts/diagram-shape.mjs
+++ /dev/null
@@ -1,56 +0,0 @@
-// Primary elements and relationships per diagram type, shared by the
-// collector (content kept from first draft to final) and the benchmark.
-export const DIAGRAM_TYPES = ['architecture', 'workflow', 'sequence', 'dataflow', 'lifecycle', 'erd', 'class', 'tree', 'timeline', 'waterfall'];
-
-const SHAPE = {
- architecture: { elements: 'components', relations: 'connections' },
- workflow: { elements: 'nodes', relations: 'edges' },
- sequence: { elements: 'participants', relations: 'messages' },
- dataflow: { elements: 'nodes', relations: 'flows' },
- lifecycle: { elements: 'states', relations: 'transitions' },
- erd: { elements: 'entities', relations: 'relationships' },
- class: { elements: 'types', relations: 'relationships' },
- tree: { elements: 'nodes' },
- timeline: { elements: 'events' },
- waterfall: { elements: 'spans' },
-};
-
-const list = (value) => (Array.isArray(value) ? value : []);
-const relationKey = (relation) => relation.id || `${relation.from}>${relation.to}>${relation.label || ''}`;
-
-// Elements and relationships of `first` missing from `final`, by id.
-export function contentLost(type, first, final) {
- const shape = SHAPE[type];
- if (!shape || !first || !final) return null;
- const finalElements = new Set(list(final[shape.elements]).map((element) => element.id));
- const finalRelations = new Set(list(final[shape.relations]).map(relationKey));
- const elements = list(first[shape.elements]).map((element) => element.id).filter((id) => !finalElements.has(id));
- const relations = shape.relations
- ? list(first[shape.relations]).map(relationKey).filter((key) => !finalRelations.has(key))
- : [];
- return {
- elements,
- relations,
- firstCounts: [list(first[shape.elements]).length, list(first[shape.relations]).length],
- finalCounts: [list(final[shape.elements]).length, list(final[shape.relations]).length],
- };
-}
-
-// A benchmark entry must render without the source checkout: drop repository
-// evidence and give it a stable output name.
-export function benchmarkDocument(document, name) {
- const copy = JSON.parse(JSON.stringify(document));
- const strip = (value) => {
- if (Array.isArray(value)) value.forEach(strip);
- else if (value && typeof value === 'object') {
- delete value.sources;
- Object.values(value).forEach(strip);
- }
- };
- strip(copy);
- if (copy.meta) {
- delete copy.meta.repository;
- copy.meta.output = `${name}.html`;
- }
- return copy;
-}
diff --git a/.agents/skills/archify-tuning/scripts/lib.mjs b/.agents/skills/archify-tuning/scripts/lib.mjs
new file mode 100644
index 00000000..8315388f
--- /dev/null
+++ b/.agents/skills/archify-tuning/scripts/lib.mjs
@@ -0,0 +1,91 @@
+// Shared by the tuning scripts: diagram shapes, benchmark preparation and the
+// tuning home that keeps rounds and the benchmark outside the repository.
+import os from 'node:os';
+import path from 'node:path';
+
+export const DIAGRAM_TYPES = ['architecture', 'workflow', 'sequence', 'dataflow', 'lifecycle', 'erd', 'class', 'tree', 'timeline', 'waterfall'];
+
+// Prompt names per type, as a user would ask for the diagram.
+export const TYPE_NAMES_ZH = {
+ architecture: '架构图', workflow: '工作流图', sequence: '时序图', dataflow: '数据流图', lifecycle: '生命周期图',
+ erd: 'ER 图', class: '类图', tree: '树图', timeline: '时间线图', waterfall: '瀑布图',
+};
+
+export function tuningHome(env = process.env) {
+ return path.resolve(env.ARCHIFY_TUNING_HOME || path.join(os.homedir(), '.local/share/archify-tuning'));
+}
+
+const SHAPE = {
+ architecture: { elements: 'components', relations: 'connections' },
+ workflow: { elements: 'nodes', relations: 'edges' },
+ sequence: { elements: 'participants', relations: 'messages' },
+ dataflow: { elements: 'nodes', relations: 'flows' },
+ lifecycle: { elements: 'states', relations: 'transitions' },
+ erd: { elements: 'entities', relations: 'relationships' },
+ class: { elements: 'types', relations: 'relationships' },
+ tree: { elements: 'nodes' },
+ timeline: { elements: 'events' },
+ waterfall: { elements: 'spans' },
+};
+
+// Fields by which an author overrides automatic routing or label placement.
+const ROUTE_CONTROLS = ['via', 'route', 'fromSide', 'toSide', 'channelX', 'channelY', 'bias', 'labelAt', 'labelDx', 'labelDy', 'labelSegment'];
+
+const list = (value) => (Array.isArray(value) ? value : []);
+const relationKey = (relation) => relation.id || `${relation.from}>${relation.to}>${relation.label || ''}`;
+
+// Elements and relationships of `first` missing from `final`, by id. A renamed
+// id also counts, so treat the result as a prompt to compare, not a verdict.
+export function contentLost(type, first, final) {
+ const shape = SHAPE[type];
+ if (!shape || !first || !final) return null;
+ const finalElements = new Set(list(final[shape.elements]).map((element) => element.id));
+ const finalRelations = new Set(list(final[shape.relations]).map(relationKey));
+ return {
+ elements: list(first[shape.elements]).map((element) => element.id).filter((id) => !finalElements.has(id)),
+ relations: shape.relations ? list(first[shape.relations]).map(relationKey).filter((key) => !finalRelations.has(key)) : [],
+ firstCounts: [list(first[shape.elements]).length, list(first[shape.relations]).length],
+ finalCounts: [list(final[shape.elements]).length, list(final[shape.relations]).length],
+ };
+}
+
+// Count of authored routing controls, per field. Zero means the draft is
+// fully automatic, so a composition failure on it is the renderer's.
+export function authoredControls(type, document) {
+ const relations = SHAPE[type]?.relations;
+ const counts = {};
+ for (const relation of list(document?.[relations])) {
+ for (const field of ROUTE_CONTROLS) {
+ if (relation[field] !== undefined && relation[field] !== 'auto') counts[field] = (counts[field] || 0) + 1;
+ }
+ }
+ return counts;
+}
+
+// A benchmark entry must render without the source checkout: drop repository
+// evidence and give it a stable output name.
+export function benchmarkDocument(document, name) {
+ const copy = JSON.parse(JSON.stringify(document));
+ const strip = (value) => {
+ if (Array.isArray(value)) value.forEach(strip);
+ else if (value && typeof value === 'object') {
+ delete value.sources;
+ Object.values(value).forEach(strip);
+ }
+ };
+ strip(copy);
+ if (copy.meta) {
+ delete copy.meta.repository;
+ copy.meta.output = `${name}.html`;
+ }
+ return copy;
+}
+
+// Agents often pipe finalize through jq, so the exit code is not evidence.
+export function finalizeStatus(result) {
+ if (/"ok":\s*false|"status":\s*"fail"|"(?:validate|deliver|check|browser-check)":\s*"fail"/.test(result)) return 'fail';
+ if (/"ok":\s*true|"status":\s*"pass"/.test(result)
+ || /"validate":\s*"pass"[^}]*"browser-check":\s*"pass"/.test(result)) return 'pass';
+ const exit = result.match(/Exit code: (\d+)/);
+ return exit ? (exit[1] === '0' ? 'unknown' : 'fail') : 'unknown';
+}
diff --git a/.agents/skills/archify-tuning/scripts/replay.mjs b/.agents/skills/archify-tuning/scripts/replay.mjs
index 194b7001..2afa5346 100644
--- a/.agents/skills/archify-tuning/scripts/replay.mjs
+++ b/.agents/skills/archify-tuning/scripts/replay.mjs
@@ -1,17 +1,18 @@
#!/usr/bin/env node
-// Usage: node replay.mjs --base <archify-dir> [--head <archify-dir>] [--bench <dir>] [--type <type>] [--out <dir>]
+// Usage: node replay.mjs --base <archify-dir> [--head <archify-dir>] [--type <type>] [--bench <dir>] [--out <dir>]
//
// Renders every benchmark first draft with two Archify trees (each the
-// directory holding bin/archify.mjs) and reports, per type, how many drafts
-// pass on each side, which flipped, and which passing drafts changed
-// geometry. Exits 1 when a draft that passes on base fails on head.
+// directory holding bin/archify.mjs; head defaults to this repository) and
+// reports, per type, how many drafts pass on each side, which flipped and
+// which passing drafts changed geometry. Renders and replay.json go to
+// <tuning-home>/replays/<timestamp> unless --out. Exits 1 when a draft that
+// passes on base fails on head. Rendering skips finalize's browser gate.
import { spawn } from 'node:child_process';
import crypto from 'node:crypto';
import fs from 'node:fs';
-import os from 'node:os';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
-import { DIAGRAM_TYPES } from './diagram-shape.mjs';
+import { DIAGRAM_TYPES, tuningHome } from './lib.mjs';
const args = process.argv.slice(2);
const option = (name, fallback) => {
@@ -21,12 +22,11 @@ const option = (name, fallback) => {
const here = path.dirname(fileURLToPath(import.meta.url));
const base = option('--base');
const head = path.resolve(option('--head', path.join(here, '../../../../archify')));
-const bench = path.resolve(option('--bench', process.env.ARCHIFY_TUNING_BENCH
- || path.join(os.homedir(), '.local/share/archify-tuning/bench')));
-const out = path.resolve(option('--out', fs.mkdtempSync(path.join(os.tmpdir(), 'archify-replay-'))));
+const bench = path.resolve(option('--bench', path.join(tuningHome(), 'bench')));
+const out = path.resolve(option('--out', path.join(tuningHome(), 'replays', new Date().toISOString().replace(/[:.]/g, '-'))));
const onlyType = option('--type');
if (!base || !fs.existsSync(path.join(base, 'bin/archify.mjs')) || !fs.existsSync(path.join(head, 'bin/archify.mjs'))) {
- console.error('Usage: node replay.mjs --base <archify-dir> [--head <archify-dir>] [--bench <dir>] [--type <type>] [--out <dir>]');
+ console.error('Usage: node replay.mjs --base <archify-dir> [--head <archify-dir>] [--type <type>] [--bench <dir>] [--out <dir>]');
process.exit(2);
}
@@ -83,6 +83,7 @@ for (const result of results) {
const note = !result.base.ok && result.head.ok ? 'FIXED'
: result.base.ok && !result.head.ok ? 'REGRESSED'
: result.base.ok && result.base.geometry !== result.head.geometry ? 'changed' : '';
+ result.note = note;
console.log(`${`${result.type}/${result.name}`.padEnd(32)} base ${tag(result.base).padEnd(10)} head ${tag(result.head).padEnd(10)} ${note.padEnd(9)} ${result.head.ok ? '' : result.head.codes.join(',')}`);
}
console.log('\nfirst-draft pass rate by type (base -> head)');
@@ -94,5 +95,5 @@ for (const type of DIAGRAM_TYPES) {
}
const regressed = results.filter((result) => result.base.ok && !result.head.ok);
console.log(`total ${results.filter((result) => result.base.ok).length}/${results.length} -> ${results.filter((result) => result.head.ok).length}/${results.length}; regressed ${regressed.length}; renders in ${out}`);
-fs.writeFileSync(path.join(out, 'replay.json'), `${JSON.stringify(results, null, 2)}\n`);
+fs.writeFileSync(path.join(out, 'replay.json'), `${JSON.stringify({ base: path.resolve(base), head, bench, results }, null, 2)}\n`);
process.exit(regressed.length ? 1 : 0);
diff --git a/.agents/skills/archify-tuning/scripts/round.mjs b/.agents/skills/archify-tuning/scripts/round.mjs
new file mode 100644
index 00000000..1dae65b7
--- /dev/null
+++ b/.agents/skills/archify-tuning/scripts/round.mjs
@@ -0,0 +1,73 @@
+#!/usr/bin/env node
+// Usage: node round.mjs <project-path> [--types architecture,workflow,...] [--archify <archify-dir>] [--name <round>]
+//
+// Prepares <tuning-home>/rounds/<round>/<type>/ for one generation round,
+// records the project and Archify revisions, and writes prompts.json: one
+// {type, title, task} per diagram type, each to be run as its own subagent.
+import { execFileSync } from 'node:child_process';
+import fs from 'node:fs';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { DIAGRAM_TYPES, TYPE_NAMES_ZH, tuningHome } from './lib.mjs';
+
+const args = process.argv.slice(2);
+const option = (name, fallback) => {
+ const index = args.indexOf(name);
+ return index === -1 ? fallback : args[index + 1];
+};
+const here = path.dirname(fileURLToPath(import.meta.url));
+const project = args[0] && !args[0].startsWith('--') ? path.resolve(args[0]) : null;
+const archify = path.resolve(option('--archify', path.join(here, '../../../../archify')));
+const types = (option('--types', DIAGRAM_TYPES.join(','))).split(',').map((type) => type.trim()).filter(Boolean);
+const git = (cwd, ...command) => execFileSync('git', ['-C', cwd, ...command], { encoding: 'utf8' }).trim();
+if (!project || !fs.existsSync(path.join(archify, 'bin/archify.mjs')) || types.some((type) => !DIAGRAM_TYPES.includes(type))) {
+ console.error(`Usage: node round.mjs <project-path> [--types ${DIAGRAM_TYPES.join(',')}] [--archify <archify-dir>] [--name <round>]`);
+ process.exit(2);
+}
+
+const rounds = path.join(tuningHome(), 'rounds');
+fs.mkdirSync(rounds, { recursive: true });
+const next = 1 + Math.max(0, ...fs.readdirSync(rounds).map((name) => Number((name.match(/^r(\d+)-/) || [])[1]) || 0));
+const name = option('--name', `r${String(next).padStart(2, '0')}-${path.basename(project)}`);
+const root = path.join(rounds, name);
+if (fs.existsSync(root)) {
+ console.error(`Round ${root} already exists; choose another --name.`);
+ process.exit(2);
+}
+
+const dirty = git(project, 'status', '--porcelain').split('\n').filter(Boolean).map((line) => line.slice(3));
+const archifyDirty = git(archify, 'status', '--porcelain', '--', '.').split('\n').filter(Boolean);
+const record = {
+ name,
+ created: new Date().toISOString(),
+ project: { path: project, revision: git(project, 'rev-parse', 'HEAD'), origin: (() => { try { return git(project, 'remote', 'get-url', 'origin'); } catch { return null; } })(), dirty },
+ archify: { path: archify, revision: git(archify, 'rev-parse', 'HEAD'), dirty: archifyDirty },
+ types,
+};
+const ignore = dirty.length
+ ? `ignore its pre-existing uncommitted files (${dirty.join(', ')}) and do not cite them`
+ : 'it has no uncommitted files';
+const prompts = types.map((type) => {
+ const workdir = path.join(root, type);
+ fs.mkdirSync(workdir, { recursive: true });
+ return {
+ type,
+ title: `${name} ${TYPE_NAMES_ZH[type]}`,
+ task: `画 ${project} 这个项目的${TYPE_NAMES_ZH[type]}。
+
+Use the Archify skill located at ${archify} (read ${archify}/SKILL.md first and follow it exactly, including the references it tells you to read). Do NOT install, update, or copy Archify anywhere; use the CLI directly as \`node ${archify}/bin/archify.mjs ...\`.
+
+Constraints:
+- Your working directory for all Archify commands and outputs is ${workdir} (cd there; the skill's \`.archify/<type>-<slug>-<timestamp>/\` folder must be created under it, not inside the project).
+- ${project} is read-only source evidence. Do not create, modify, or delete any file in it (${ignore}). Pass it as \`--repo-root\` where the skill requires.
+- Do not modify anything under ${archify}.
+- Follow the skill's repair loop until finalize passes or the skill tells you to stop.
+
+When done, reply concisely with: diagram type, absolute path of the final candidate JSON and final HTML, whether finalize passed (key status lines), and every repair iteration (diagnostic code + what you changed). Also list any point where the skill docs or CLI messages were unclear, contradictory, or sent you the wrong way.`,
+ };
+});
+fs.writeFileSync(path.join(root, 'round.json'), `${JSON.stringify(record, null, 2)}\n`);
+fs.writeFileSync(path.join(root, 'prompts.json'), `${JSON.stringify(prompts, null, 2)}\n`);
+console.log(`round ${name}: ${types.length} prompts in ${path.join(root, 'prompts.json')}`);
+if (archifyDirty.length) console.log(`warning: ${archify} has uncommitted changes; commit them or the round will not be reproducible.`);
+console.log('Run each prompt as its own background subagent, and do not edit the Archify tree until every subagent has finished.');
diff --git a/.agents/skills/archify-tuning/scripts/shots.mjs b/.agents/skills/archify-tuning/scripts/shots.mjs
new file mode 100644
index 00000000..9003209e
--- /dev/null
+++ b/.agents/skills/archify-tuning/scripts/shots.mjs
@@ -0,0 +1,54 @@
+#!/usr/bin/env node
+// Usage: node shots.mjs [<replay-dir>] [--all] [--type <type>] [--height <px>]
+//
+// Screenshots base and head renders from a replay (the latest one by default)
+// at a 1440px desktop width and writes shots/index.html, which shows each
+// pair side by side. Without --all only FIXED, REGRESSED and changed entries
+// are captured: those are the ones a reviewer must look at.
+import { spawnSync } from 'node:child_process';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { pathToFileURL } from 'node:url';
+import { findChrome } from '../../../../archify/bin/visual-check.mjs';
+import { tuningHome } from './lib.mjs';
+
+const args = process.argv.slice(2);
+const option = (name, fallback) => {
+ const index = args.indexOf(name);
+ return index === -1 ? fallback : args[index + 1];
+};
+const replays = path.join(tuningHome(), 'replays');
+const target = args[0] && !args[0].startsWith('--') ? path.resolve(args[0])
+ : fs.existsSync(replays) ? path.join(replays, fs.readdirSync(replays).sort().at(-1) || '') : '';
+const chrome = findChrome();
+if (!target || !fs.existsSync(path.join(target, 'replay.json')) || !chrome) {
+ console.error(chrome ? 'Usage: node shots.mjs [<replay-dir>] [--all] [--type <type>] [--height <px>]' : 'Chrome or Chromium is unavailable; set ARCHIFY_CHROME.');
+ process.exit(2);
+}
+const height = Number(option('--height', 1100));
+const { results } = JSON.parse(fs.readFileSync(path.join(target, 'replay.json'), 'utf8'));
+const picked = results.filter((result) => (args.includes('--all') || result.note) && (!option('--type') || result.type === option('--type')));
+const shots = path.join(target, 'shots');
+fs.mkdirSync(shots, { recursive: true });
+const profile = fs.mkdtempSync(path.join(os.tmpdir(), 'archify-shots-'));
+
+function capture(side, result) {
+ const html = path.join(target, side, result.type, `${result.name}.html`);
+ if (!fs.existsSync(html)) return null;
+ const png = path.join(shots, `${result.type}-${result.name}-${side}.png`);
+ spawnSync(chrome, ['--headless=new', '--disable-gpu', '--hide-scrollbars', `--user-data-dir=${profile}`,
+ `--window-size=1440,${height}`, `--screenshot=${png}`, pathToFileURL(html).href], { stdio: 'ignore', timeout: 60_000 });
+ return fs.existsSync(png) ? path.basename(png) : null;
+}
+
+const rows = picked.map((result) => {
+ const [base, head] = [capture('base', result), capture('head', result)];
+ const cell = (file, side) => (file ? `<img src="${file}" alt="${side}">` : `<p>${side}: no render (${result[side].first || 'failed'})</p>`);
+ return `<h2>${result.type}/${result.name} ${result.note || ''}</h2><div class="pair">${cell(base, 'base')}${cell(head, 'head')}</div>`;
+});
+fs.rmSync(profile, { recursive: true, force: true });
+fs.writeFileSync(path.join(shots, 'index.html'), `<!doctype html><meta charset="utf-8"><title>Replay shots</title>
+<style>body{font:14px system-ui;margin:16px;background:#111;color:#ddd}.pair{display:grid;grid-template-columns:1fr 1fr;gap:8px}img{width:100%;border:1px solid #333}</style>
+<p>Left: base. Right: head. ${picked.length} entries from ${target}</p>${rows.join('\n')}\n`);
+console.log(`${picked.length} pairs: ${path.join(shots, 'index.html')}`);
diff --git a/test/archify-tuning.test.mjs b/test/archify-tuning.test.mjs
new file mode 100644
index 00000000..90466b3c
--- /dev/null
+++ b/test/archify-tuning.test.mjs
@@ -0,0 +1,52 @@
+import { test } from 'node:test';
+import assert from 'node:assert/strict';
+
+import {
+ authoredControls,
+ benchmarkDocument,
+ contentLost,
+ finalizeStatus,
+} from '../.agents/skills/archify-tuning/scripts/lib.mjs';
+
+test('tuning: content lost names the elements and relationships a final candidate dropped', () => {
+ const first = {
+ nodes: [{ id: 'a' }, { id: 'b' }, { id: 'invalid' }],
+ edges: [{ id: 'ab', from: 'a', to: 'b' }, { from: 'b', to: 'invalid', label: 'rejects' }],
+ };
+ const final = { nodes: [{ id: 'a' }, { id: 'b' }], edges: [{ id: 'ab', from: 'a', to: 'b' }] };
+ assert.deepEqual(contentLost('workflow', first, final), {
+ elements: ['invalid'],
+ relations: ['b>invalid>rejects'],
+ firstCounts: [3, 2],
+ finalCounts: [2, 1],
+ });
+ assert.equal(contentLost('workflow', first, null), null);
+});
+
+test('tuning: authored controls separate fully automatic drafts from pinned ones', () => {
+ assert.deepEqual(authoredControls('workflow', { edges: [{ from: 'a', to: 'b', route: 'auto' }] }), {});
+ assert.deepEqual(authoredControls('dataflow', {
+ flows: [{ from: 'a', to: 'b', fromSide: 'right', via: [[1, 2]] }, { from: 'b', to: 'c', labelDy: 4 }],
+ }), { fromSide: 1, via: 1, labelDy: 1 });
+});
+
+test('tuning: benchmark drafts drop repository evidence and keep their layout', () => {
+ const draft = {
+ meta: { title: 'T', output: '.archify/x/y.html', repository: { url: 'https://example.com/private.git' } },
+ nodes: [{ id: 'a', col: 1, sources: [{ path: 'src/a.ts' }] }],
+ };
+ assert.deepEqual(benchmarkDocument(draft, 'r01-demo'), {
+ meta: { title: 'T', output: 'r01-demo.html' },
+ nodes: [{ id: 'a', col: 1 }],
+ });
+ assert.ok(draft.meta.repository, 'the source document is not modified');
+});
+
+test('tuning: finalize status reads gate results rather than the exit code of a pipe', () => {
+ assert.equal(finalizeStatus('fail {"validate":"fail","deliver":"not-run"} ... Exit code: 0'), 'fail');
+ assert.equal(finalizeStatus('{"schemaVersion":1,"ok":false,"status":"fail"} Exit code: 1'), 'fail');
+ assert.equal(finalizeStatus('pass {"validate":"pass","deliver":"pass","check":"pass","browser-check":"pass"} Exit code: 0'), 'pass');
+ assert.equal(finalizeStatus('{"ok":true,"status":"pass"}'), 'pass');
+ assert.equal(finalizeStatus('Usage: archify finalize ... Exit code: 2'), 'fail');
+ assert.equal(finalizeStatus('done Exit code: 0'), 'unknown');
+});