Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

fix(cli): generate manifest examples and response types from the docs (#6585)

Joey9fc9e5833dce
diff --git a/.changeset/cli-spec-manifest-from-docs.md b/.changeset/cli-spec-manifest-from-docs.md
new file mode 100644
index 0000000000..f28ed0b111
--- /dev/null
+++ b/.changeset/cli-spec-manifest-from-docs.md
@@ -0,0 +1,7 @@
+---
+'@astryxdesign/cli': patch
+---
+
+[fix] `astryx manifest --json` now takes each command's examples from its CommandDoc and its response types from the API function it wraps, so no example differs from the documented one and `upgrade` lists `upgrade.registry`, which `upgrade --registry --json` already emits.
+
+@josephfarina
diff --git a/packages/cli/README.md b/packages/cli/README.md
index 1fce6a13f9..ea4937a29a 100644
--- a/packages/cli/README.md
+++ b/packages/cli/README.md
@@ -303,11 +303,12 @@ Shape:
```
The manifest is **derived from Commander metadata** (commands, arguments, options)
-so it can't drift from the real command definitions. The two facts Commander
-doesn't track (`--json` support and emitted response types) are layered on from
-the `JSON_SUPPORTED` allowlist and a small declarative `RESPONSE_TYPES` map in
-`src/lib/manifest.mjs`, guarded by a drift test (`manifest.test.mjs`) so adding a
-command without describing it fails CI.
+so it can't drift from the real command definitions. The facts Commander doesn't
+track come from each command's docs: examples from its CommandDoc, and emitted
+response types from the returns of the FunctionDoc it wraps (plus the few
+envelopes the CLI layer builds itself). `--json` support comes from the
+`JSON_SUPPORTED` allowlist. Drift tests (`manifest.test.mjs`) fail CI when a
+command is added without describing it.
**Backwards-compat:** the bare `astryx --json` envelope keeps `type: "help"` and its
original shallow fields (`name`, `version`, `commands` as a `string[]` of names,
diff --git a/packages/cli/clients/cli/lib/manifest.mjs b/packages/cli/clients/cli/lib/manifest.mjs
index e58ba839d7..f475515267 100644
--- a/packages/cli/clients/cli/lib/manifest.mjs
+++ b/packages/cli/clients/cli/lib/manifest.mjs
@@ -7,19 +7,19 @@
* lets an AI agent drive the entire CLI WITHOUT scraping `--help` text. It is
* to the CLI what an OpenAPI spec is to an HTTP API.
*
- * Most of the manifest is DERIVED from Commander metadata (program.commands,
- * cmd.options, cmd.registeredArguments, cmd.description()) so it cannot drift
- * out of sync with the real command definitions. The two pieces Commander does
- * not know about — whether a command supports `--json`, and which response
- * `type` discriminators it can emit — are layered on from:
+ * Its per-command facts come from the command's own definitions: names,
+ * arguments and options from Commander metadata (program.commands, cmd.options,
+ * cmd.registeredArguments, cmd.description()); examples from its CommandDoc;
+ * response `type` discriminators from the returns of the FunctionDoc it wraps
+ * (both attached by `defineCommand`); `--json` support from the JSON_SUPPORTED
+ * allowlist in index.mjs. The hand-kept lists are CLI_LAYER_RESPONSE_TYPES,
+ * for envelopes no API function returns, and ROOT_RESPONSE_TYPES, for the two
+ * (help and version) that no single command owns.
*
- * - JSON_SUPPORTED (the allowlist in index.mjs), and
- * - RESPONSE_TYPES (the declarative map below; ROOT_RESPONSE_TYPES names the
- * two, help and version, that no single command owns).
- *
- * A drift-guard test (manifest.test.mjs) asserts every registered command
- * appears in the manifest and every JSON-supported command has a response-type
- * entry, so adding a command without describing it fails CI.
+ * Drift-guard tests (manifest.test.mjs) assert every registered command appears
+ * in the manifest, every JSON-supported command has response types, and the
+ * examples and response types equal the docs, so adding a command without
+ * describing it fails CI.
*
* DESIGN DECISION — manifest stays CLI-special; there is intentionally NO
* `api/manifest`. It describes the CLI's own Commander tree, so it takes the live
@@ -40,166 +40,28 @@ import {commandDocsOf} from './define-command.mjs';
import {doc as manifestDoc} from '../commands/manifest.doc.mjs';
/**
- * Response `type` discriminators each fully-qualified command can emit in
- * `--json` mode. Keyed by the same name Commander reports (parent + leaf,
- * space-joined), e.g. `theme build`. Commander has no knowledge of these —
- * they come from each command's `jsonOut(...)` call sites — so we keep them
- * here, close to the JSON_SUPPORTED allowlist, and guard them with a test.
- *
+ * Envelopes the CLI layer builds itself, so the wrapped FunctionDoc cannot
+ * declare them: `manifest` wraps no API function, `theme build` batches several
+ * `themeBuild()` receipts, and `theme add` with no slug (or `--list`) answers
+ * with `themeListAvailable()`'s `theme.list` instead of calling `themeAdd()`.
+ * manifest.test.mjs fails once the wrapped FunctionDoc declares one of these, so
+ * the list only shrinks.
* @type {Record<string, string[]>}
*/
-export const RESPONSE_TYPES = {
- init: ['init.run', 'init.remove'],
- component: [
- 'component.list',
- 'component.batch',
- 'component.detail',
- 'component.detail.props',
- 'component.detail.source',
- 'component.detail.showcase',
- 'component.detail.blocks',
- ],
- docs: [
- 'docs.list',
- 'docs.index',
- 'docs.detail',
- 'docs.detail.section',
- 'docs.node',
- ],
- blog: ['blog.list', 'blog.detail'],
- discover: [
- 'discover.list',
- 'discover.detail',
- 'discover.detail.doc',
- 'discover.item',
- 'discover.search',
- ],
- search: ['search'],
- build: ['build.help', 'build.kit'],
- swizzle: ['swizzle.list', 'swizzle.copy'],
- 'gap-report': ['gap-report.categories', 'gap-report.file'],
- template: [
- 'template.list',
- 'template.show',
- 'template.skeleton',
- 'template.copy',
- 'template.cdn',
- ],
- hook: ['hook.list', 'hook.detail', 'hook.detail.params'],
- 'theme build': ['theme.build', 'theme.build.check', 'theme.build.batch'],
- 'theme list': ['theme.list'],
- 'theme add': ['theme.list', 'theme.add'],
- 'theme template': ['theme.template'],
- 'theme targets': ['theme.targets'],
- 'theme palette generate': ['theme.palette.generate'],
- 'integration add': ['integration.add'],
- 'integration verify': ['integration.pack-check'],
- 'integration pack': ['integration.pack-check'],
- upgrade: ['upgrade.list', 'upgrade.registry', 'upgrade.status', 'upgrade.run'],
+const CLI_LAYER_RESPONSE_TYPES = {
manifest: ['manifest'],
- doctor: ['doctor'],
- 'doctor integration validate': ['integration.validate'],
- 'doctor integration templates': ['integration.template-conflicts'],
- 'doctor integration components': ['integration.component-conflicts'],
- 'doctor integration docs': ['integration.doc-conflicts'],
- 'layout expand': ['layout.expand'],
- 'layout check': ['layout.check'],
- 'layout grammar': ['layout.grammar'],
+ 'theme build': ['theme.build.batch'],
+ 'theme add': ['theme.list'],
};
/**
* Response types no single command owns: `help` (a bare `astryx --json`, and
* `--help --json` on any command) and `version` (`astryx --version --json`).
- * The response-types enum lists them beside {@link RESPONSE_TYPES}.
+ * The response-types enum lists them beside the per-command response types.
* @type {readonly string[]}
*/
export const ROOT_RESPONSE_TYPES = Object.freeze(['help', 'version']);
-/**
- * Example invocations per fully-qualified command. Optional, agent-facing.
- * @type {Record<string, string[]>}
- */
-const EXAMPLES = {
- component: [
- 'astryx component',
- 'astryx component XDSButton',
- 'astryx component Button Badge Text',
- 'astryx component XDSButton --props --json',
- ],
- docs: [
- 'astryx docs',
- 'astryx docs spacing --json',
- 'astryx docs theme',
- 'astryx docs theme quick-start',
- 'astryx docs cli/integrations/quick-start --full',
- ],
- discover: ['astryx discover --json'],
- search: [
- 'astryx search modal --json',
- 'astryx search button --type component --json',
- ],
- build: ['astryx build', 'astryx build "analytics dashboard" --json'],
- swizzle: ['astryx swizzle Button'],
- 'gap-report': [
- 'astryx gap-report --list-categories',
- "astryx gap-report Button --category docs_gap --reason 'Missing keyboard example'",
- ],
- template: [
- 'astryx template --json',
- 'astryx template dashboard ./src/app',
- 'astryx template --cdn',
- ],
- hook: ['astryx hook', 'astryx hook useFocusTrap --json'],
- 'theme build': [
- 'astryx theme build ./src/themes/ocean.ts --out ./dist/ocean.css',
- 'astryx theme build ./src/themes/ocean.ts --check',
- ],
- 'theme list': ['astryx theme list --json'],
- 'theme add': [
- 'astryx theme add matcha',
- 'astryx theme add matcha ./src/themes/matcha',
- ],
- 'theme template': ['astryx theme template', 'astryx theme template --json'],
- 'theme targets': [
- 'astryx theme targets Switch',
- 'astryx --json theme targets',
- ],
- 'theme palette generate': [
- 'astryx theme palette generate palette.config.json',
- 'astryx theme palette generate palette.config.json --out ocean.palette.json',
- ],
- 'integration add': [
- 'astryx integration add component AcmeWidget',
- 'astryx integration add doc deploying --dry-run --json',
- ],
- 'integration verify': ['astryx integration verify --json'],
- upgrade: ['astryx upgrade --json'],
- manifest: ['astryx manifest --json', 'astryx --json'],
- doctor: ['astryx doctor', 'astryx doctor --json'],
- 'doctor integration validate': [
- 'astryx doctor integration validate',
- 'astryx doctor integration validate @acme/widgets --json',
- ],
- 'doctor integration templates': [
- 'astryx doctor integration templates',
- 'astryx doctor integration templates @acme/widgets --json',
- ],
- 'doctor integration components': [
- 'astryx doctor integration components',
- 'astryx doctor integration components @acme/widgets --json',
- ],
- 'doctor integration docs': [
- 'astryx doctor integration docs',
- 'astryx doctor integration docs @acme/widgets --json',
- ],
- init: ['astryx init', 'astryx init --all --json'],
- 'layout expand': [
- `astryx layout expand 'V[g6] > C{card-callout}*4' ./src/Page.tsx`,
- ],
- 'layout check': [`astryx layout check 'A[cp6] > L > LC > S[p6]' --json`],
- 'layout grammar': ['astryx layout grammar'],
-};
-
/**
* Map a Commander Option to a flag descriptor. Derives type from whether the
* option takes a value (boolean vs string), surfaces `choices` and `default`.
@@ -314,11 +176,17 @@ function describeCommand(cmd, root, jsonSupported) {
const aliases = cmd.aliases ? cmd.aliases() : [];
if (aliases && aliases.length > 0) entry.aliases = [...aliases];
- // Response types this command can emit in --json mode (only meaningful for
- // JSON-supported leaves). Subcommand groups (e.g. bare `theme`) have none.
- if (RESPONSE_TYPES[name]) entry.responseTypes = [...RESPONSE_TYPES[name]];
+ // Response types this command can emit in --json mode: the returns of the
+ // FunctionDoc it wraps, then any envelope the CLI layer builds. Subcommand
+ // groups (e.g. bare `theme`) have none.
+ const responseTypes = [
+ ...(docs?.fn?.returns ?? []).map(r => r.type),
+ ...(CLI_LAYER_RESPONSE_TYPES[name] ?? []),
+ ];
+ if (responseTypes.length > 0) entry.responseTypes = responseTypes;
- if (EXAMPLES[name]) entry.examples = [...EXAMPLES[name]];
+ const examples = (docs?.doc.examples ?? []).map(e => e.cli);
+ if (examples.length > 0) entry.examples = examples;
const exitCodes = (docs?.doc.exitCodes ?? []).map(({code, when}) => ({
code,
@@ -361,6 +229,15 @@ function describeGlobalOptions(program) {
return opts;
}
+/**
+ * Every command in a described tree, depth first.
+ * @param {any[]} commands
+ * @returns {any[]}
+ */
+function flattenCommands(commands) {
+ return commands.flatMap(c => [c, ...flattenCommands(c.subcommands || [])]);
+}
+
/**
* Build the full capability manifest from a configured Commander program.
*
@@ -391,7 +268,9 @@ export function buildManifest(program, opts = {}) {
// Flat index of every response `type` discriminator the CLI can emit,
// keyed by command — lets an agent know what to expect back per call.
responseTypes: Object.fromEntries(
- Object.entries(RESPONSE_TYPES).map(([k, v]) => [k, [...v]]),
+ flattenCommands(commands)
+ .filter(c => c.responseTypes)
+ .map(c => [c.name, [...c.responseTypes]]),
),
};
}
diff --git a/packages/cli/clients/cli/lib/manifest.test.mjs b/packages/cli/clients/cli/lib/manifest.test.mjs
index 4be526bab0..b2d1f2f511 100644
--- a/packages/cli/clients/cli/lib/manifest.test.mjs
+++ b/packages/cli/clients/cli/lib/manifest.test.mjs
@@ -14,11 +14,46 @@
* surfaces emit valid, enriched JSON.
*/
+import * as fs from 'node:fs';
+import * as os from 'node:os';
+import * as path from 'node:path';
+import {fileURLToPath, pathToFileURL} from 'node:url';
import {describe, it, expect} from 'vitest';
import {program, JSON_SUPPORTED} from '../index.mjs';
-import {buildManifest, RESPONSE_TYPES} from './manifest.mjs';
+import {buildManifest} from './manifest.mjs';
import {runCli} from '../../../test-utils/run-cli.mjs';
+const HERE = path.dirname(fileURLToPath(import.meta.url));
+
+/**
+ * Import every `*.doc.mjs` under `dir` whose doc has the given type, keyed by
+ * its `name`.
+ * @param {string} dir @param {string} type @returns {Promise<Map<string, any>>}
+ */
+async function loadDocs(dir, type) {
+ const out = new Map();
+ for (const entry of fs.readdirSync(dir, {withFileTypes: true, recursive: true})) {
+ if (!entry.isFile() || !entry.name.endsWith('.doc.mjs')) continue;
+ const file = path.join(entry.parentPath, entry.name);
+ const {doc} = await import(pathToFileURL(file).href);
+ if (doc?.type === type) out.set(doc.name, doc);
+ }
+ return out;
+}
+
+const commandDocs = await loadDocs(path.join(HERE, '../commands'), 'command');
+const functionDocs = await loadDocs(path.join(HERE, '../../../api'), 'function');
+
+/**
+ * Envelopes built in the CLI layer, which no FunctionDoc can declare. Pinned so
+ * an entry is dropped once its type moves behind an API function.
+ */
+const CLI_LAYER_TYPES = {
+ manifest: ['manifest'],
+ 'theme build': ['theme.build.batch'],
+ 'theme add': ['theme.list'],
+};
+
const manifest = buildManifest(program, {jsonSupported: JSON_SUPPORTED, version: '0.0.0-test'});
/** Flatten the manifest command tree into fully-qualified names. */
@@ -64,17 +99,73 @@ describe('manifest: drift guards', () => {
it('declares response types for every JSON-supported command', () => {
for (const name of JSON_SUPPORTED) {
+ const entry = allEntries.find((c) => c.name === name);
expect(
- RESPONSE_TYPES[name],
- `JSON-supported command "${name}" has no response-type entry`,
- ).toBeDefined();
- expect(RESPONSE_TYPES[name].length).toBeGreaterThan(0);
+ entry?.responseTypes?.length,
+ `JSON-supported command "${name}" has no response types`,
+ ).toBeGreaterThan(0);
+ expect(manifest.responseTypes[name]).toEqual(entry.responseTypes);
}
});
it('has no response-type entry for a command that does not exist', () => {
- for (const name of Object.keys(RESPONSE_TYPES)) {
- expect(allNames.has(name), `RESPONSE_TYPES key "${name}" is not a real command`).toBe(true);
+ for (const name of Object.keys(manifest.responseTypes)) {
+ expect(allNames.has(name), `responseTypes key "${name}" is not a real command`).toBe(true);
+ }
+ });
+
+ it('takes every command example from its CommandDoc', () => {
+ for (const [name, doc] of commandDocs) {
+ const entry = allEntries.find((c) => c.name === name);
+ expect(entry, `CommandDoc "${name}" has no manifest entry`).toBeDefined();
+ expect(entry.examples ?? [], name).toEqual((doc.examples ?? []).map((e) => e.cli));
+ }
+ });
+
+ it('lists every response type the wrapped API function returns', () => {
+ for (const [name, doc] of commandDocs) {
+ if (!doc.fn) continue;
+ const entry = allEntries.find((c) => c.name === name);
+ for (const {type} of functionDocs.get(doc.fn).returns) {
+ expect(entry.responseTypes, `${name} can emit ${type}`).toContain(type);
+ }
+ }
+ });
+
+ it('lists upgrade.registry, which `upgrade --registry --json` emits', async () => {
+ const cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'astryx-manifest-'));
+ fs.writeFileSync(path.join(cwd, 'package.json'), '{"name":"app","version":"1.0.0"}');
+ const {status, stdout} = await runCli(['--json', 'upgrade', '--registry'], {cwd});
+ expect(status).toBe(0);
+ expect(JSON.parse(stdout).type).toBe('upgrade.registry');
+ expect(manifest.responseTypes.upgrade).toContain('upgrade.registry');
+ });
+
+ it('lists theme.list for theme add, which lists themes when given no slug', async () => {
+ const cwd = fs.mkdtempSync(path.join(os.tmpdir(), 'astryx-manifest-'));
+ fs.writeFileSync(path.join(cwd, 'package.json'), '{"name":"app","version":"1.0.0"}');
+ const {status, stdout} = await runCli(['--json', 'theme', 'add'], {cwd});
+ expect(status).toBe(0);
+ expect(JSON.parse(stdout).type).toBe('theme.list');
+ expect(manifest.responseTypes['theme add']).toContain('theme.list');
+ });
+
+ it('takes response types from the wrapped FunctionDoc, plus CLI-layer envelopes', () => {
+ for (const [name, doc] of commandDocs) {
+ const entry = allEntries.find((c) => c.name === name);
+ const returns = doc.fn ? functionDocs.get(doc.fn).returns.map((r) => r.type) : [];
+ const expected = [...returns, ...(CLI_LAYER_TYPES[name] ?? [])];
+ expect(entry.responseTypes ?? [], name).toEqual(expected);
+ }
+ });
+
+ it('pins only CLI-layer envelopes that no FunctionDoc declares', () => {
+ for (const [name, types] of Object.entries(CLI_LAYER_TYPES)) {
+ const fn = commandDocs.get(name)?.fn;
+ const declared = fn ? functionDocs.get(fn).returns.map((r) => r.type) : [];
+ for (const type of types) {
+ expect(declared, `${type} is declared by ${fn}(); unpin it`).not.toContain(type);
+ }
}
});
diff --git a/packages/cli/foundation/response/response-types.doc.mjs b/packages/cli/foundation/response/response-types.doc.mjs
index 20ece7cb0f..20129eef45 100644
--- a/packages/cli/foundation/response/response-types.doc.mjs
+++ b/packages/cli/foundation/response/response-types.doc.mjs
@@ -2,11 +2,11 @@
/**
* @file EnumDoc for the `type` discriminant carried on every --json success
- * envelope. The vocabulary mirrors the RESPONSE_TYPES map (each command's
- * `jsonOut(...)` call sites) plus ROOT_RESPONSE_TYPES (help, version) in
- * `clients/cli/lib/manifest.mjs`; a consumer switches on `type` to narrow the
- * `data` payload. Descriptions follow the currently published response
- * projection, not a future package-version boundary.
+ * envelope. The vocabulary equals the manifest's response types (each
+ * command's FunctionDoc returns; see `clients/cli/lib/manifest.mjs`) plus
+ * ROOT_RESPONSE_TYPES (help, version) there; a consumer switches on `type` to
+ * narrow the `data` payload. Descriptions follow the currently published
+ * response projection, not a future package-version boundary.
*
* @input Public response discriminants and their currently released payloads.
* @output Generated consumer reference for every typed JSON success response.