Skip to content

Turn any idea, plan, or codebase into a beautiful interactive diagram. An agent skill for Claude Code, Codex, and more.

feat(skill): add an internal skill for evidence-based tuning

tt-a1ic26607cf33c9
diff --git a/.agents/skills/archify-tuning/SKILL.md b/.agents/skills/archify-tuning/SKILL.md
new file mode 100644
index 00000000..3745855b
--- /dev/null
+++ b/.agents/skills/archify-tuning/SKILL.md
@@ -0,0 +1,81 @@
+---
+name: archify-tuning
+description: Tune Archify's generation quality from evidence. Generate real diagrams through subagents, read their execution traces, attribute each failure, fix the root cause and prove the change on a fixed benchmark of first drafts. Use when asked to improve how good, accurate or fast generated Archify diagrams are, to run a generation round, or to judge whether a renderer, diagnostic or documentation change made generation better.
+metadata:
+ internal: true
+---
+
+# Archify Tuning
+
+Users do not write diagram JSON; they describe a system and an agent authors it. Tune for that path: an agent's **first draft** should render as an accurate, readable diagram with few repairs. When the goals conflict, keep this order:
+
+1. **Accurate**: no node, relationship, lane or condition is dropped or merged to pass a gate.
+2. **Readable**: clear reading path, few bends and crossings, legible text.
+3. **Fast**: few finalize failures and tool calls.
+
+A change that makes gates pass while the diagram reads worse, or that makes agents delete content more often, is a regression.
+
+## Principles
+
+- **The first draft is the unit of measure.** Bundled examples carry hand-tuned widths and routes, so they hide defects that every fresh draft hits.
+- **Deleted content is the accuracy signal.** A pass reached by removing an error state or merging lanes is a failure that counts as a success; check what the final candidate dropped.
+- **Automatic layout must pass its own gates.** When a fully automatic draft (no `via`, sides, channels or label coordinates) fails composition checks, the renderer is at fault, not the agent.
+- **Early diagnostics must match the browser gate.** A compile-time check stricter than `browser-check` is a false failure; a looser one defers the failure to the most expensive stage.
+- **Traces explain more than outcomes.** Read where the agent was misled by a diagnostic or a document, not only whether it passed.
+
+## Measures
+
+| Level | Measure | Source |
+|---|---|---|
+| Primary | Benchmark first-draft pass rate; content kept from first draft to final | `replay.mjs`; `collect.mjs` |
+| Secondary | Finalize failures and tool calls per diagram | `collect.mjs` on a new round |
+| Visual | Crossings, bends, shared corridors, text size, then screenshots | Receipts and a browser |
+| Guard | No benchmark draft or historical sample that passed now fails; every changed output inspected | `replay.mjs`, golden, Gallery |
+
+A new round on a new project finds problems and shows a change broke nothing. It does not prove improvement: projects differ too much. Prove improvement on the benchmark, or by repeating one project two or three times.
+
+## Loop
+
+1. **Generate.** Pick a project not used before. Start one background subagent per diagram type with the prompt below and the same wording every round. Never edit the Archify tree the subagents use while they run; experiment in a separate worktree.
+2. **Collect.** `node .agents/skills/archify-tuning/scripts/collect.mjs <round-root> --round <n>-<project> --dump` prints per-type tool calls, finalize runs and failures, diagnostic codes and the content lost, writes `<type>.trace.txt` and adds each first draft to the benchmark. Cross-check its counts with the subagents' own reports.
+3. **Attribute** each failure to exactly one cause:
+
+ | Cause | Test | Response |
+ |---|---|---|
+ | Renderer defect | A fully automatic draft fails its own composition checks | Fix the renderer |
+ | Misleading diagnostic | Following the message fails or edits the wrong element | Fix its fix text or evidence |
+ | Documentation gap | The agent asks something only an unread reference answers | Move the fact into a required read, with numbers |
+ | Authoring error | The required reads already say it | Nothing, or one sentence |
+
+4. **Prioritise** by how many rounds and types show it, times its harm to accuracy, over its cost. A problem seen every round needs a systematic fix, not one patch per round.
+5. **Fix** the root cause once, in the shared path. Preserve authored geometry and pins; when the renderer cannot repair a draft itself, return a verified, actionable fix. Reject a change that only moves a metric, such as trading crossings for shared corridors.
+6. **Verify.** Add a minimal test that fails before the fix. Run `node .agents/skills/archify-tuning/scripts/replay.mjs --base <dev-worktree>/archify` and expect no regressions. Find affected tests by searching for the changed functions and message text, not by file name. Regenerate examples and Gallery when output changes, run golden and inspect screenshots of changed outputs.
+7. **Record.** Commit code, regenerated outputs and documentation separately. Integrate regularly so the full suite and CI run; do not let local commits pile up.
+
+## Diagnostics and documentation
+
+- A diagnostic names the element, the shortfall (target width, character budget or required gap) and one fix that works. It never points at a fix that cannot succeed.
+- A compact receipt lists how many diagnostics of each code it omitted.
+- Documentation answers questions traces show agents asking, in a required read, with measured numbers. Keep one paragraph per mode and every line under the 2,000-character read limit.
+
+## Subagent prompt
+
+```text
+画 <project-path> 这个项目的<图类型>。
+
+Use the Archify skill located at <archify-tree> (read <archify-tree>/SKILL.md first and follow it exactly, including the references it tells you to read). Do NOT install, update, or copy Archify anywhere; use the CLI directly as `node <archify-tree>/bin/archify.mjs ...`.
+
+Constraints:
+- Your working directory for all Archify commands and outputs is <round-root>/<type> (cd there; the skill's `.archify/<type>-<slug>-<timestamp>/` folder must be created under it, not inside the project).
+- <project-path> is read-only source evidence. Do not create, modify, or delete any file in it (ignore its pre-existing untracked files). Pass it as `--repo-root` where the skill requires.
+- Do not modify anything under <archify-tree>.
+- Follow the skill's repair loop until finalize passes or the skill tells you to stop.
+
+When done, reply concisely with: diagram type, absolute path of the final candidate JSON and final HTML, whether finalize passed (key status lines), and every repair iteration (diagnostic code + what you changed). Also list any point where the skill docs or CLI messages were unclear, contradictory, or sent you the wrong way.
+```
+
+`<round-root>/<type>` must appear verbatim in the prompt: `collect.mjs` finds each subagent by it.
+
+## Benchmark
+
+The benchmark lives outside the repository, in `$ARCHIFY_TUNING_BENCH` or `~/.local/share/archify-tuning/bench`, because first drafts carry content from the source projects, which may be private. Each entry is `<type>/<round>.json` with repository evidence stripped; `index.json` records its project, revision and first finalize result. Never commit benchmark drafts. `collect.mjs` reads Devin CLI session history through `node:sqlite` (Node 22.13 or later); other agents need their own trace export.
diff --git a/.agents/skills/archify-tuning/scripts/collect.mjs b/.agents/skills/archify-tuning/scripts/collect.mjs
new file mode 100644
index 00000000..6387a6e0
--- /dev/null
+++ b/.agents/skills/archify-tuning/scripts/collect.mjs
@@ -0,0 +1,146 @@
+#!/usr/bin/env node
+// Usage: node collect.mjs <round-root> [--bench <dir>] [--round <name>] [--no-bench] [--dump]
+//
+// Reads Devin CLI session history for subagents whose task pointed at
+// <round-root>/<type>, then reports per type: tool calls, finalize runs and
+// failures, diagnostic codes, and the elements and relationships the final
+// candidate dropped from the first draft. The first draft of each type is
+// added to the benchmark (repository evidence stripped) unless --no-bench.
+import { DatabaseSync } from 'node:sqlite';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { DIAGRAM_TYPES, benchmarkDocument, contentLost } from './diagram-shape.mjs';
+
+const args = process.argv.slice(2);
+const option = (name, fallback) => {
+ const index = args.indexOf(name);
+ return index === -1 ? fallback : args[index + 1];
+};
+const root = path.resolve(args.find((arg, index) => !arg.startsWith('--') && !args[index - 1]?.startsWith('--')) || '');
+if (!args.length || !fs.existsSync(root)) {
+ console.error('Usage: node collect.mjs <round-root> [--bench <dir>] [--round <name>] [--no-bench] [--dump]');
+ process.exit(2);
+}
+const round = option('--round', path.basename(root));
+const bench = path.resolve(option('--bench', process.env.ARCHIFY_TUNING_BENCH
+ || path.join(os.homedir(), '.local/share/archify-tuning/bench')));
+const database = path.join(os.homedir(), '.local/share/devin/cli/sessions.db');
+const db = new DatabaseSync(database, { readOnly: true });
+
+const text = (value) => (typeof value === 'string' ? value : JSON.stringify(value ?? ''));
+const escape = (value) => value.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+// Task text may name the macOS /private alias or the plain /tmp path.
+const roots = [...new Set([root, root.replace(/^\/private\//, '/'), root.replace(/^\/tmp\//, '/private/tmp/')])];
+const CODE = /\b((?:layout|composition|clean-flow|workflow|sequence|dataflow|lifecycle|erd|class|tree|timeline|waterfall|architecture|repository-evidence|schema|viewer|delivery)\/[a-z0-9]+(?:-[a-z0-9]+)*)\b/g;
+
+// Agents often pipe finalize through jq, so the exit code is not evidence.
+function finalizeStatus(result) {
+ if (/"ok":\s*false|"status":\s*"fail"|"(?:validate|deliver|check|browser-check)":\s*"fail"/.test(result)) return 'fail';
+ if (/"ok":\s*true|"status":\s*"pass"/.test(result)
+ || /"validate":\s*"pass"[^}]*"browser-check":\s*"pass"/.test(result)) return 'pass';
+ const exit = result.match(/Exit code: (\d+)/);
+ return exit ? (exit[1] === '0' ? 'unknown' : 'fail') : 'unknown';
+}
+
+function chainsByType() {
+ const like = roots.map(() => 'chat_message like ?').join(' or ');
+ const hits = db.prepare(`select session_id, min(node_id) as first from message_nodes where ${like} group by session_id`)
+ .all(...roots.map((candidate) => `%${candidate}/%`));
+ const best = new Map();
+ for (const { session_id: session, first } of hits) {
+ const rows = db.prepare('select node_id, parent_node_id, chat_message from message_nodes where session_id = ? and node_id >= ?').all(session, first);
+ const nodes = new Map(rows.map((row) => [row.node_id, { ...row, message: JSON.parse(row.chat_message) }]));
+ const parents = new Set(rows.map((row) => row.parent_node_id));
+ for (const node of nodes.values()) {
+ if (parents.has(node.node_id)) continue;
+ const chain = [];
+ for (let current = node; current; current = nodes.get(current.parent_node_id)) chain.unshift(current);
+ for (const type of DIAGRAM_TYPES) {
+ const pattern = new RegExp(`(?:${roots.map(escape).join('|')})/${type}(?![\\w-])`);
+ const start = chain.findIndex(({ message }) => message.role === 'user' && typeof message.content === 'string' && pattern.test(message.content));
+ if (start === -1) continue;
+ const own = chain.slice(start);
+ if (!best.has(type) || best.get(type).length < own.length) best.set(type, own);
+ }
+ }
+ }
+ return best;
+}
+
+function latestCandidate(type) {
+ const folder = path.join(root, type, '.archify');
+ if (!fs.existsSync(folder)) return null;
+ const files = fs.readdirSync(folder)
+ .map((entry) => path.join(folder, entry, 'candidate.json'))
+ .filter((file) => fs.existsSync(file))
+ .sort((left, right) => fs.statSync(right).mtimeMs - fs.statSync(left).mtimeMs);
+ return files[0] ? JSON.parse(fs.readFileSync(files[0], 'utf8')) : null;
+}
+
+const summary = {};
+const index = fs.existsSync(path.join(bench, 'index.json')) ? JSON.parse(fs.readFileSync(path.join(bench, 'index.json'), 'utf8')) : {};
+for (const [type, chain] of [...chainsByType()].sort(([a], [b]) => DIAGRAM_TYPES.indexOf(a) - DIAGRAM_TYPES.indexOf(b))) {
+ const results = new Map();
+ for (const { message } of chain) if (message.role === 'tool') results.set(message.tool_call_id, text(message.content));
+ const stats = { tools: 0, reads: 0, docReads: [], finalize: [], firstDraft: null, dump: [] };
+ for (const { message } of chain) {
+ if (message.role === 'assistant' && message.content) stats.dump.push(`ASSISTANT: ${text(message.content).slice(0, 2000)}`);
+ for (const call of message.tool_calls || []) {
+ stats.tools += 1;
+ const input = call.arguments || {};
+ const result = results.get(call.id) || '';
+ stats.dump.push(`CALL ${call.name}: ${text(input).slice(0, 3000)}\nRESULT: ${result.slice(0, 4000)}`);
+ if (call.name === 'read') {
+ stats.reads += 1;
+ if (/\/archify\/.*\.md$/.test(input.file_path || '')) stats.docReads.push(input.file_path.replace(/^.*\/archify\//, ''));
+ }
+ if (call.name === 'write' && !stats.firstDraft && /candidate[^/]*\.json$/.test(input.file_path || '')) {
+ try { stats.firstDraft = JSON.parse(input.content); } catch { /* not JSON yet */ }
+ }
+ if (call.name === 'exec' && new RegExp(`archify\\.mjs\\s+finalize\\s+${type}\\s+\\S+\\.json`).test(input.command || '')) {
+ stats.finalize.push({ status: finalizeStatus(result), codes: [...new Set([...result.matchAll(CODE)].map((match) => match[1]))] });
+ }
+ }
+ }
+ const final = latestCandidate(type);
+ const lost = contentLost(type, stats.firstDraft, final);
+ summary[type] = {
+ tools: stats.tools,
+ reads: stats.reads,
+ finalizeRuns: stats.finalize.length,
+ finalizeFailures: stats.finalize.filter(({ status }) => status === 'fail').length,
+ finalizeUnknown: stats.finalize.filter(({ status }) => status === 'unknown').length,
+ firstFinalize: stats.finalize[0]?.status ?? 'not-run',
+ codes: [...new Set(stats.finalize.flatMap(({ codes }) => codes))],
+ lost,
+ docReads: stats.docReads,
+ };
+ if (args.includes('--dump')) fs.writeFileSync(path.join(root, `${type}.trace.txt`), stats.dump.join('\n\n'));
+ if (stats.firstDraft && !args.includes('--no-bench')) {
+ fs.mkdirSync(path.join(bench, type), { recursive: true });
+ fs.writeFileSync(path.join(bench, type, `${round}.json`), `${JSON.stringify(benchmarkDocument(stats.firstDraft, round), null, 2)}\n`);
+ index[`${type}/${round}`] = {
+ round: root,
+ project: stats.firstDraft.meta?.repository?.url ?? null,
+ revision: stats.firstDraft.meta?.repository?.revision ?? null,
+ firstFinalize: summary[type].firstFinalize,
+ };
+ }
+}
+if (!args.includes('--no-bench') && Object.keys(index).length) {
+ fs.mkdirSync(bench, { recursive: true });
+ fs.writeFileSync(path.join(bench, 'index.json'), `${JSON.stringify(index, null, 2)}\n`);
+}
+fs.writeFileSync(path.join(root, 'tuning-summary.json'), `${JSON.stringify(summary, null, 2)}\n`);
+
+const pad = (value, width) => String(value).padEnd(width);
+console.log(`${pad('type', 13)}${pad('tools', 7)}${pad('finalize', 10)}${pad('failed', 8)}${pad('first', 9)}lost (elements/relations) codes`);
+for (const [type, entry] of Object.entries(summary)) {
+ const lost = entry.lost ? `${entry.lost.elements.length}/${entry.lost.relations.length}${entry.lost.elements.length ? ` ${entry.lost.elements.join(',')}` : ''}` : 'n/a';
+ const failed = `${entry.finalizeFailures}${entry.finalizeUnknown ? `+${entry.finalizeUnknown}?` : ''}`;
+ console.log(`${pad(type, 13)}${pad(entry.tools, 7)}${pad(entry.finalizeRuns, 10)}${pad(failed, 8)}${pad(entry.firstFinalize, 9)}${pad(lost, 27)}${entry.codes.join(',')}`);
+}
+const entries = Object.values(summary);
+console.log(`TOTAL tools ${entries.reduce((sum, entry) => sum + entry.tools, 0)}, finalize ${entries.reduce((sum, entry) => sum + entry.finalizeRuns, 0)}, failed ${entries.reduce((sum, entry) => sum + entry.finalizeFailures, 0)}, first-run passes ${entries.filter((entry) => entry.firstFinalize === 'pass').length}/${entries.length}`);
+if (!args.includes('--no-bench')) console.log(`benchmark: ${bench}`);
diff --git a/.agents/skills/archify-tuning/scripts/diagram-shape.mjs b/.agents/skills/archify-tuning/scripts/diagram-shape.mjs
new file mode 100644
index 00000000..e008fdfd
--- /dev/null
+++ b/.agents/skills/archify-tuning/scripts/diagram-shape.mjs
@@ -0,0 +1,56 @@
+// Primary elements and relationships per diagram type, shared by the
+// collector (content kept from first draft to final) and the benchmark.
+export const DIAGRAM_TYPES = ['architecture', 'workflow', 'sequence', 'dataflow', 'lifecycle', 'erd', 'class', 'tree', 'timeline', 'waterfall'];
+
+const SHAPE = {
+ architecture: { elements: 'components', relations: 'connections' },
+ workflow: { elements: 'nodes', relations: 'edges' },
+ sequence: { elements: 'participants', relations: 'messages' },
+ dataflow: { elements: 'nodes', relations: 'flows' },
+ lifecycle: { elements: 'states', relations: 'transitions' },
+ erd: { elements: 'entities', relations: 'relationships' },
+ class: { elements: 'types', relations: 'relationships' },
+ tree: { elements: 'nodes' },
+ timeline: { elements: 'events' },
+ waterfall: { elements: 'spans' },
+};
+
+const list = (value) => (Array.isArray(value) ? value : []);
+const relationKey = (relation) => relation.id || `${relation.from}>${relation.to}>${relation.label || ''}`;
+
+// Elements and relationships of `first` missing from `final`, by id.
+export function contentLost(type, first, final) {
+ const shape = SHAPE[type];
+ if (!shape || !first || !final) return null;
+ const finalElements = new Set(list(final[shape.elements]).map((element) => element.id));
+ const finalRelations = new Set(list(final[shape.relations]).map(relationKey));
+ const elements = list(first[shape.elements]).map((element) => element.id).filter((id) => !finalElements.has(id));
+ const relations = shape.relations
+ ? list(first[shape.relations]).map(relationKey).filter((key) => !finalRelations.has(key))
+ : [];
+ return {
+ elements,
+ relations,
+ firstCounts: [list(first[shape.elements]).length, list(first[shape.relations]).length],
+ finalCounts: [list(final[shape.elements]).length, list(final[shape.relations]).length],
+ };
+}
+
+// A benchmark entry must render without the source checkout: drop repository
+// evidence and give it a stable output name.
+export function benchmarkDocument(document, name) {
+ const copy = JSON.parse(JSON.stringify(document));
+ const strip = (value) => {
+ if (Array.isArray(value)) value.forEach(strip);
+ else if (value && typeof value === 'object') {
+ delete value.sources;
+ Object.values(value).forEach(strip);
+ }
+ };
+ strip(copy);
+ if (copy.meta) {
+ delete copy.meta.repository;
+ copy.meta.output = `${name}.html`;
+ }
+ return copy;
+}
diff --git a/.agents/skills/archify-tuning/scripts/replay.mjs b/.agents/skills/archify-tuning/scripts/replay.mjs
new file mode 100644
index 00000000..194b7001
--- /dev/null
+++ b/.agents/skills/archify-tuning/scripts/replay.mjs
@@ -0,0 +1,98 @@
+#!/usr/bin/env node
+// Usage: node replay.mjs --base <archify-dir> [--head <archify-dir>] [--bench <dir>] [--type <type>] [--out <dir>]
+//
+// Renders every benchmark first draft with two Archify trees (each the
+// directory holding bin/archify.mjs) and reports, per type, how many drafts
+// pass on each side, which flipped, and which passing drafts changed
+// geometry. Exits 1 when a draft that passes on base fails on head.
+import { spawn } from 'node:child_process';
+import crypto from 'node:crypto';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
+import { fileURLToPath } from 'node:url';
+import { DIAGRAM_TYPES } from './diagram-shape.mjs';
+
+const args = process.argv.slice(2);
+const option = (name, fallback) => {
+ const index = args.indexOf(name);
+ return index === -1 ? fallback : args[index + 1];
+};
+const here = path.dirname(fileURLToPath(import.meta.url));
+const base = option('--base');
+const head = path.resolve(option('--head', path.join(here, '../../../../archify')));
+const bench = path.resolve(option('--bench', process.env.ARCHIFY_TUNING_BENCH
+ || path.join(os.homedir(), '.local/share/archify-tuning/bench')));
+const out = path.resolve(option('--out', fs.mkdtempSync(path.join(os.tmpdir(), 'archify-replay-'))));
+const onlyType = option('--type');
+if (!base || !fs.existsSync(path.join(base, 'bin/archify.mjs')) || !fs.existsSync(path.join(head, 'bin/archify.mjs'))) {
+ console.error('Usage: node replay.mjs --base <archify-dir> [--head <archify-dir>] [--bench <dir>] [--type <type>] [--out <dir>]');
+ process.exit(2);
+}
+
+const entries = DIAGRAM_TYPES.filter((type) => !onlyType || type === onlyType).flatMap((type) => {
+ const folder = path.join(bench, type);
+ return fs.existsSync(folder)
+ ? fs.readdirSync(folder).filter((file) => file.endsWith('.json')).sort().map((file) => ({ type, name: file.slice(0, -5), file: path.join(folder, file) }))
+ : [];
+});
+
+// Geometry a renderer change can move: routed points and the canvas.
+function geometry(html) {
+ const points = html.match(/data-composition-points="[^"]*"|viewBox="[^"]*"/g) || [];
+ return crypto.createHash('sha256').update(points.join('\n')).digest('hex').slice(0, 12);
+}
+
+function render(tree, side, entry) {
+ const document = JSON.parse(fs.readFileSync(entry.file, 'utf8'));
+ const output = path.join(out, side, entry.type, `${entry.name}.html`);
+ fs.mkdirSync(path.dirname(output), { recursive: true });
+ fs.rmSync(output, { force: true });
+ const quality = document.meta?.quality_profile || 'showcase';
+ return new Promise((resolve) => {
+ const child = spawn(process.execPath, [path.join(tree, 'bin/archify.mjs'), 'render', entry.type, entry.file, output, '--quality', quality], { stdio: ['ignore', 'pipe', 'pipe'] });
+ let log = '';
+ child.stdout.on('data', (chunk) => { log += chunk; });
+ child.stderr.on('data', (chunk) => { log += chunk; });
+ child.on('close', (code) => {
+ const ok = code === 0 && fs.existsSync(output);
+ const problems = log.split('\n').filter((line) => line.startsWith('- '));
+ resolve({
+ ok,
+ problems: problems.length,
+ codes: [...new Set(problems.map((line) => (line.match(/^- \[([^\]]+)\]/) || [])[1] || 'other'))],
+ first: ok ? '' : (problems[0] || log.trim().split('\n')[0] || '').slice(0, 160),
+ geometry: ok ? geometry(fs.readFileSync(output, 'utf8')) : null,
+ });
+ });
+ });
+}
+
+const results = [];
+const queue = [...entries];
+await Promise.all(Array.from({ length: Math.min(4, queue.length) }, async () => {
+ for (let entry = queue.shift(); entry; entry = queue.shift()) {
+ const [before, after] = [await render(base, 'base', entry), await render(head, 'head', entry)];
+ results.push({ ...entry, base: before, head: after });
+ }
+}));
+results.sort((a, b) => DIAGRAM_TYPES.indexOf(a.type) - DIAGRAM_TYPES.indexOf(b.type) || a.name.localeCompare(b.name));
+
+const tag = (result) => (result.ok ? 'pass' : `fail(${result.problems})`);
+for (const result of results) {
+ const note = !result.base.ok && result.head.ok ? 'FIXED'
+ : result.base.ok && !result.head.ok ? 'REGRESSED'
+ : result.base.ok && result.base.geometry !== result.head.geometry ? 'changed' : '';
+ console.log(`${`${result.type}/${result.name}`.padEnd(32)} base ${tag(result.base).padEnd(10)} head ${tag(result.head).padEnd(10)} ${note.padEnd(9)} ${result.head.ok ? '' : result.head.codes.join(',')}`);
+}
+console.log('\nfirst-draft pass rate by type (base -> head)');
+for (const type of DIAGRAM_TYPES) {
+ const own = results.filter((result) => result.type === type);
+ if (!own.length) continue;
+ const count = (side) => own.filter((result) => result[side].ok).length;
+ console.log(` ${type.padEnd(13)} ${count('base')}/${own.length} -> ${count('head')}/${own.length}`);
+}
+const regressed = results.filter((result) => result.base.ok && !result.head.ok);
+console.log(`total ${results.filter((result) => result.base.ok).length}/${results.length} -> ${results.filter((result) => result.head.ok).length}/${results.length}; regressed ${regressed.length}; renders in ${out}`);
+fs.writeFileSync(path.join(out, 'replay.json'), `${JSON.stringify(results, null, 2)}\n`);
+process.exit(regressed.length ? 1 : 0);
diff --git a/test/skill-installation.test.mjs b/test/skill-installation.test.mjs
index 6a526e47..f4879431 100644
--- a/test/skill-installation.test.mjs
+++ b/test/skill-installation.test.mjs
@@ -64,9 +64,11 @@ test('real Skills CLI keeps source and ZIP installs scoped to the requested Skil
// Use the repository's tracked-file stager: no node_modules, local Skills,
// symlinks, or other developer files enter the installation source.
stageCleanSkill({ repoRoot, destination: path.join(source, 'archify') });
- const review = '.agents/skills/archify-review/SKILL.md';
- fs.mkdirSync(path.dirname(path.join(source, review)), { recursive: true });
- fs.copyFileSync(path.join(repoRoot, review), path.join(source, review));
+ for (const internal of fs.readdirSync(path.join(repoRoot, '.agents', 'skills'))) {
+ const skill = path.join('.agents', 'skills', internal, 'SKILL.md');
+ fs.mkdirSync(path.dirname(path.join(source, skill)), { recursive: true });
+ fs.copyFileSync(path.join(repoRoot, skill), path.join(source, skill));
+ }
extractArchive(archive);
let sequence = 0;
function invoke(sourcePath, flags) {
@@ -99,7 +101,7 @@ test('real Skills CLI keeps source and ZIP installs scoped to the requested Skil
assert.equal(result.status, 0, result.output);
assert.match(result.output, /Found 1 skill\b/);
assert.match(result.output, /Available Skills[\s\S]*\barchify\b/);
- assert.doesNotMatch(result.output, /archify-review|YAML parse error/);
+ assert.doesNotMatch(result.output, /archify-review|archify-tuning|YAML parse error/);
assert.deepEqual(installedNames(result.cwd), []);
}
});