An open source design system that's fully customizable and agent ready
TypeScript
feat(cli): list passages two integration topics repeat word for word (#7735)
Joey40b21e5705df
diff --git a/.changeset/copied-doc-passages.md b/.changeset/copied-doc-passages.mdnew file mode 100644index 0000000000..c2822de28a--- /dev/null+++ b/.changeset/copied-doc-passages.md@@ -0,0 +1,8 @@+---+'@astryxdesign/cli': patch+---++[feat] `doctor integration docs` (and so `integration check`) lists each pair of topics that repeat 10 or more words of prose word for word: two of the package's own topics, or one of them and a Core topic other than the one it replaces. Each entry names the topic that keeps the text (a Core topic, or of two package topics the one the passage fits), the line to change in the other, and the exact link to put there, and is in the new `copiedPassages` field. A run with fewer than 4 distinct content words (words other than function words such as "the", "in" or "it"), or one that three or more topics share, is not listed. Each pair is also a `copied_passage` warning, so `integration check --strict` (new in this release) fails until it is fixed; `doctor integration docs` still exits 0.++The CLI's own guides no longer repeat 18 passages that another guide owns; each copy is now a link to the guide that owns the fact.+@josephfarinadiff --git a/.changeset/integration-check.md b/.changeset/integration-check.mdindex 0b4378d51b..d00d9bf23f 100644--- a/.changeset/integration-check.md+++ b/.changeset/integration-check.md@@ -2,6 +2,6 @@'@astryxdesign/cli': patch----[feat] New `astryx integration check`, the CI check for an integration package's own build. It takes no package argument and checks the integration package in the current folder. It runs the four `doctor integration` checks (`validate`, `components`, `templates`, `docs`), prints each result, then a summary, and exits 1 when any of them would. `--strict` also exits 1 on any warning and on any check that did not run, such as a package with no `astryx.integration.*`: add `"typecheck": "tsc --noEmit && astryx integration check --strict"` to package.json so every build keeps the package's docs in sync. Its exit codes and JSON are a contract for the builds that run it: a new check, or a new failure code, joins only in a minor release named in the release notes. JSON: a new `integration.check` envelope carries each check's own envelope unchanged. Its docs say to run it in CI, so `astryx search ci` finds it. The `doctor integration` subcommands, their exit codes, and their JSON do not change.+[feat] New `astryx integration check`, the CI check for an integration package's own build. It takes no package argument and checks the integration package in the current folder. It runs the four `doctor integration` checks (`validate`, `components`, `templates`, `docs`), prints each result, then a summary, and exits 1 when any of them would. `--strict` also exits 1 on any warning and on any check that did not run, such as a package with no `astryx.integration.*`: add `"typecheck": "tsc --noEmit && astryx integration check --strict"` to package.json so every build keeps the package's docs in sync. Its exit codes and JSON are a contract for the builds that run it: from this first release on, a new check, or a new failure code, joins only in a minor release named in the release notes. JSON: a new `integration.check` envelope carries each check's own envelope unchanged. Its docs say to run it in CI, so `astryx search ci` finds it. The `doctor integration` subcommands keep their exit codes; their JSON gains only additions.@josephfarinadiff --git a/packages/cli/README.md b/packages/cli/README.mdindex 2ce02d55a4..5d32de4839 100644--- a/packages/cli/README.md+++ b/packages/cli/README.md@@ -479,7 +479,7 @@ Every response has a `type` discriminant. The full set is below (generated from| `integration.validate` | The validation result: validated (false when nothing was validated: with no package.json the issues list carries one no_package error so the command exits 1, and with a package.json but no integration manifest the issues list is empty and proves nothing), the package name and version (both null when validated is false) plus issues, an AstryxIntegrationIssue[] of {code, severity: warning \| error, message}. When the manifest loads, docTypes checks each doc against its published authoring type: status (checked \| skipped), reason, typescript, docs, findings (all warnings, each naming the field and a fix), unchecked, and fixtures (test fixture docs not checked). || `integration.template-conflicts` | validated (false when no integration manifest was found, so nothing was inspected), the integration identity, structural issues, and non-blocking Core template-id conflicts as {id, severity: warning, integrationPackage, integrationType, integrationName, coreMatches, message, command}. || `integration.component-conflicts` | validated (false when no integration manifest was found, so nothing was inspected), the integration identity, structural issues, and non-blocking conflicts where an integration component name is also owned by Core; each conflict includes the exact package-qualified command. When the manifest loads and declares a components root, propDrift compares documented props with the TypeScript props: status (checked \| skipped), reason, typescript, components, typesCompared, typesSkipped, findings (all warnings), unchecked, typesNotCompared, and propsNotChecked. |-| `integration.doc-conflicts` | validated (false when no integration manifest was found, so nothing was inspected), the integration identity, structural issues, and Core doc overlaps. Each finding includes `severity` (`info` \| `error`) and `relationship` (`replaces` \| `extends` \| `accidental`). |+| `integration.doc-conflicts` | validated (false when no integration manifest was found, so nothing was inspected), the integration identity, structural issues, and Core doc overlaps. Each finding includes `severity` (`info` \| `error`) and `relationship` (`replaces` \| `extends` \| `accidental`). copiedPassages lists each pair of topics that repeat 10 or more words of prose word for word: topic, section, otherTopic, otherOwner (package \| core), words, passages, owner (the topic that should keep the text), text (the passage as the topic writes it), line (where the copy starts in the topic to change, or null), and message (the exact replacement); each is also a copied_passage warning in issues. || `integration.check` | Every source check of an integration package in one result: validated (false when no integration manifest was found, so nothing was checked), the package name and version, strict, and checks, one entry per check (validate, components, templates, docs), each with check, errors, warnings, skipped, failed, and response, the envelope that check returns on its own. The totals are errors, warnings, and skipped; unchecked counts what the checks could not compare. With --strict, failures lists each warning and each piece of skipped work again as an error, in the issue format the checks use; it is empty without --strict. failed decides the exit code. |<!-- END GENERATED: response-types -->diff --git a/packages/cli/api/integration/_adapter.mjs b/packages/cli/api/integration/_adapter.mjsindex c054202d61..5f7a71b586 100644--- a/packages/cli/api/integration/_adapter.mjs+++ b/packages/cli/api/integration/_adapter.mjs@@ -15,6 +15,7 @@ import {integrationSourceForDoc,} from '../../foundation/discovery/component-discovery.mjs';import {comparePropDocs} from '../../foundation/discovery/prop-drift.mjs';+import {findCopiedPassages} from '../../foundation/discovery/copied-passages.mjs';/*** Each component doc's props compared with the component's TypeScript props,@@ -52,3 +53,15 @@ export async function integrationPropDrift(integration, cwd) {resolveFrom: [cwd, packageDir],});}++/**+ * The passages two of a package's topics, or one of them and a Core topic,+ * repeat word for word (architecture:cli-surface DEC-8). Reads the package's+ * topics and Core's guides.+ *+ * @param {Parameters<typeof findCopiedPassages>[0]} discovered the package's+ * docs, as discoverIntegrationDocs returns them+ */+export function integrationCopiedPassages(discovered) {+ return findCopiedPassages(discovered);+}diff --git a/packages/cli/api/integration/authoring-checks.mjs b/packages/cli/api/integration/authoring-checks.mjsindex 034ab2827e..7189dc2c15 100644--- a/packages/cli/api/integration/authoring-checks.mjs+++ b/packages/cli/api/integration/authoring-checks.mjs@@ -29,7 +29,7 @@ import {discoverOwnedComponents,} from '../../foundation/discovery/component-discovery.mjs';import {loadComponentReplacements} from '../component/_adapter.mjs';-import {integrationPropDrift} from './_adapter.mjs';+import {integrationCopiedPassages, integrationPropDrift} from './_adapter.mjs';import {discoverBuiltinTopics,discoverIntegrationDocs,@@ -323,10 +323,12 @@ export async function integrationDocConflicts(pkg, options = {}) {const version = resolved.found ? (resolved.version ?? null) : null;const issues = [...resolved.issues];+ /** @type {import('./authoring-checks.type.mjs').CopiedPassage[]} */+ const copiedPassages = [];if (!resolved.integration?.docs || name == null) {return {type: 'integration.doc-conflicts',- data: {validated, name, version, findings: [], issues},+ data: {validated, name, version, findings: [], copiedPassages, issues},};}@@ -416,10 +418,19 @@ export async function integrationDocConflicts(pkg, options = {}) {)) {issues.push({code: 'invalid_doc_reference', severity: 'error', message});}++ // One fact has one owner: a passage two topics repeat goes stale in one+ // of them. Each is listed and is a warning (architecture:cli-surface+ // DEC-8), so `integration check --strict` fails until it is fixed;+ // `doctor integration docs` keeps its exit code, as it fails on errors.+ copiedPassages.push(...(await integrationCopiedPassages(discovered)));+ for (const {message} of copiedPassages) {+ issues.push({code: 'copied_passage', severity: 'warning', message});+ }}return {type: 'integration.doc-conflicts',- data: {validated, name, version, findings, issues},+ data: {validated, name, version, findings, copiedPassages, issues},};}diff --git a/packages/cli/api/integration/authoring-checks.test.mjs b/packages/cli/api/integration/authoring-checks.test.mjsindex f240c769d4..1842344457 100644--- a/packages/cli/api/integration/authoring-checks.test.mjs+++ b/packages/cli/api/integration/authoring-checks.test.mjs@@ -18,6 +18,11 @@ import {discoverCoreTemplates} from '../../foundation/discovery/template-adapterimport {discoverOwnedComponents} from '../../foundation/discovery/component-discovery.mjs';import {discoverBuiltinTopics} from '../../foundation/discovery/docs-discovery.mjs';import {findCoreDir} from '../../foundation/fs/paths.mjs';+import {+ builtinGuideFiles,+ loadTopicModule,+} from '../../foundation/discovery/docs-discovery.mjs';+import {proseWords} from '../../foundation/discovery/copied-passages.mjs';import {integrationComponentConflicts,integrationDocConflicts,@@ -99,6 +104,44 @@ function writeDocIntegration({name, replaces, extendsTopic}) {);}+/**+ * A docs-only package with one generic topic per entry: its prose, and+ * optionally a code block and the Core topic it replaces.+ * @param {{name: string, text: string, code?: string, replaces?: string}[]} topics+ */+function writeTopics(topics) {+ fs.writeFileSync(+ path.join(tmpDir, 'package.json'),+ JSON.stringify({name: '@acme/widgets', version: '1.2.3'}),+ );+ fs.writeFileSync(+ path.join(tmpDir, 'astryx.integration.mjs'),+ `export default {docs: './docs'};\n`,+ );+ fs.mkdirSync(path.join(tmpDir, 'docs'), {recursive: true});+ for (const {name, text, code, replaces, extendsTopic} of topics) {+ const blocks = [{type: 'prose', text}];+ if (code != null) blocks.push({type: 'code', lang: 'text', code});+ const doc = {+ type: 'generic',+ name,+ title: 'Fixture',+ description: 'Fixture topic.',+ ...(replaces == null ? {} : {replaces}),+ ...(extendsTopic == null ? {} : {extends: extendsTopic}),+ sections: [{title: 'Overview', content: blocks}],+ };+ fs.writeFileSync(+ path.join(tmpDir, 'docs', `${name}.doc.mjs`),+ `export default ${JSON.stringify(doc)};\n`,+ );+ }+}++const PASSAGE =+ 'Themes cannot style the layer that holds a menu or a popover, so a menu ' ++ 'keeps the default surface color until the next release adds a token for it.';+beforeEach(() => {tmpDir = fs.mkdtempSync(path.join(process.cwd(), '.astryx-template-conflicts-'),@@ -562,3 +605,286 @@ describe('integrationDocConflicts', () => {]);});});++describe('integrationDocConflicts copied passages', () => {+ const copied = (/** @type {any} */ result) => result.data.copiedPassages;++ it('lists a passage two of the package topics repeat, and warns about it', async () => {+ writeTopics([+ {name: 'acme-setup', text: `Read this first. ${PASSAGE}`},+ {name: 'acme-menus', text: `${PASSAGE} See the theme guide.`},+ ]);++ const result = await integrationDocConflicts(undefined, {cwd: tmpDir});+ const [issue, ...rest] = copied(result);+ expect(result.data.issues).toEqual([+ {code: 'copied_passage', severity: 'warning', message: issue.message},+ ]);++ expect(rest).toEqual([]);+ // The passage is about menus, so "acme-menus" keeps it.+ expect(issue).toMatchObject({+ topic: 'acme-setup',+ otherTopic: 'acme-menus',+ otherOwner: 'package',+ owner: 'acme-menus',+ section: 'Overview',+ line: 1,+ words: PASSAGE.split(' ').length,+ passages: 1,+ });+ expect(issue.message).toContain(`${PASSAGE.split(' ').length} words`);+ expect(issue.text).toBe(PASSAGE);+ expect(issue.message).toContain(`"${PASSAGE}"`);+ expect(issue.message).toContain(+ '"acme-menus" keeps it: its name, title or section fits the passage.',+ );+ expect(issue.message).toContain(+ 'In "acme-setup", replace that whole passage, from its first word to its last, code ticks included, with {@link generic:acme-menus}.',+ );+ });++ it('reports a passage copied from a Core topic, except into the topic that replaces it', async () => {+ const [file] = builtinGuideFiles();+ const coreDoc = /** @type {any} */ (await loadTopicModule(file));+ const coreText = proseWords(coreDoc)+ .slice(0, 30)+ .map(each => each.word)+ .join(' ');++ writeTopics([{name: 'acme-copy', text: coreText}]);+ const [issue] = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );+ expect(issue.otherOwner).toBe('core');+ expect(issue.message).toContain(`the Core topic "${coreDoc.name}"`);+ expect(issue.owner).toBe(coreDoc.name);+ expect(issue.message).toContain(+ `code ticks included, with {@link @astryxdesign/cli:generic:${coreDoc.name}}.`,+ );++ fs.rmSync(path.join(tmpDir, 'docs'), {recursive: true, force: true});+ writeTopics([{name: 'acme-own', text: coreText, replaces: coreDoc.name}]);+ expect(+ copied(await integrationDocConflicts(undefined, {cwd: tmpDir})).filter(+ (/** @type {any} */ each) => each.message.includes(`"${coreDoc.name}"`),+ ),+ ).toEqual([]);+ });++ it('leaves out boilerplate that three topics share', async () => {+ writeTopics([+ {name: 'acme-fonts', text: PASSAGE},+ {name: 'acme-icons', text: PASSAGE},+ {name: 'acme-media', text: PASSAGE},+ ]);++ expect(+ copied(await integrationDocConflicts(undefined, {cwd: tmpDir})),+ ).toEqual([]);+ });++ it('leaves out a run of function words, which states no fact', async () => {+ const glue = 'and it is in the way that you can do it for them';+ writeTopics([+ {name: 'acme-a', text: `Start here. ${glue} first.`},+ {name: 'acme-b', text: `Then ${glue} again.`},+ ]);++ expect(+ copied(await integrationDocConflicts(undefined, {cwd: tmpDir})),+ ).toEqual([]);+ });++ it('leaves out a shared phrase under 10 words, and code', async () => {+ const short = PASSAGE.split(' ').slice(0, 9).join(' ');+ writeTopics([+ {name: 'acme-a', text: PASSAGE},+ {name: 'acme-b', text: short},+ {name: 'acme-c', text: 'Run it.', code: PASSAGE},+ ]);++ expect(+ copied(await integrationDocConflicts(undefined, {cwd: tmpDir})),+ ).toEqual([]);+ });+});++describe('integrationDocConflicts copied passages and extensions', () => {+ const copied = (/** @type {any} */ result) => result.data.copiedPassages;++ it('links to the page an extension shows on, not to the extension', async () => {+ writeTopics([+ {name: 'acme-menus', text: PASSAGE, extendsTopic: 'theme'},+ {name: 'acme-setup', text: `${PASSAGE} Then run setup.`},+ ]);++ const [issue] = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ expect(issue).toMatchObject({topic: 'acme-setup', owner: 'acme-menus'});+ expect(issue.message).toContain(+ 'with {@link @astryxdesign/cli:generic:theme}.',+ );+ });++ it('says to delete a copy in a topic that extends its owner', async () => {+ writeTopics([+ {name: 'acme-base', text: PASSAGE},+ {name: 'acme-more', text: PASSAGE, extendsTopic: 'acme-base'},+ ]);++ const [issue] = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ expect(issue).toMatchObject({topic: 'acme-more', owner: 'acme-base'});+ expect(issue.message).toContain(+ '"acme-more" and "acme-base" show on the same page ("acme-base"): delete the passage from "acme-more"',+ );+ expect(issue.message).not.toContain('{@link');+ });++ it('compares pages without regard to case, and for two extensions of one page', async () => {+ writeTopics([+ {name: 'acme-colors', text: PASSAGE, extendsTopic: 'Theme'},+ {name: 'acme-menus', text: PASSAGE, extendsTopic: 'theme'},+ ]);++ const [issue] = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ expect(issue.message).toContain('show on the same page ("theme")');+ expect(issue.message).not.toContain('{@link');+ });++ it('finds the line of the string that holds the copy, whatever it opens with', async () => {+ writeTopics([{name: 'acme-menus', text: PASSAGE}]);+ fs.writeFileSync(+ path.join(tmpDir, 'docs', 'acme-setup.doc.mjs'),+ [+ 'export default {',+ " type: 'generic',",+ " name: 'acme-setup',",+ " title: 'Fixture',",+ " description: 'Fixture topic.',",+ " sections: [{title: 'Overview', content: [",+ " {type: 'prose', text: '`acme` (the CLI) opens here, as {@link generic:acme-menus} says.'},",+ ` {type: 'prose', text: '\`acme\` (the CLI) opens here: ${PASSAGE}'},`,+ ' ]}],',+ '};',+ '',+ ].join('\n'),+ );++ const [issue] = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ expect(issue).toMatchObject({topic: 'acme-setup', line: 8});+ });++ it('does not list a 9-word sentence that ends in a link', async () => {+ const sentence =+ 'Theme tokens control every surface color, as explained in {@link generic:acme-theme}.';+ writeTopics([+ {name: 'acme-colors', text: `Colors first. ${sentence}`},+ {name: 'acme-menus', text: `Menus next. ${sentence}`},+ ]);++ expect(+ copied(await integrationDocConflicts(undefined, {cwd: tmpDir})),+ ).toEqual([]);+ });++ it('counts no function word toward which topic a passage fits', async () => {+ writeTopics([+ {name: 'acme-getting-started', text: `${PASSAGE} Then run setup.`},+ {name: 'acme-menus', text: PASSAGE},+ ]);+ // Titles that share only function words with the passage.+ for (const [name, title] of [+ ['acme-getting-started', 'Get started with the widgets'],+ ['acme-menus', 'Menus'],+ ]) {+ const file = path.join(tmpDir, 'docs', `${name}.doc.mjs`);+ fs.writeFileSync(+ file,+ fs+ .readFileSync(file, 'utf8')+ .replace('"title":"Fixture"', `"title":"${title}"`),+ );+ }++ const [issue] = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ expect(issue).toMatchObject({+ owner: 'acme-menus',+ topic: 'acme-getting-started',+ });+ });++ it('finds a copy after a lead-in three topics share', async () => {+ const lead =+ 'Every package ships its integration docs as plain modules the CLI reads';+ writeTopics([+ {name: 'acme-menus', text: `${lead}. ${PASSAGE}`},+ {name: 'acme-setup', text: `${lead}. ${PASSAGE} Then run setup.`},+ {name: 'acme-other', text: `${lead}. Nothing else here is shared.`},+ ]);++ const found = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ expect(found).toHaveLength(1);+ expect(found[0].text).toBe(PASSAGE);+ });++ it("links to a package topic that replaces Core's by its own name", async () => {+ writeTopics([+ {name: 'theme', text: PASSAGE, replaces: 'theme'},+ {name: 'acme-setup', text: `${PASSAGE} Then run setup.`},+ ]);++ const found = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ );++ for (const each of found) {+ expect(each.message).not.toContain('@astryxdesign/cli:generic:theme');+ }+ expect(+ found.some((/** @type {any} */ each) =>+ each.message.includes('with {@link generic:theme}.'),+ ),+ ).toBe(true);+ });++ it('prints no link to a Core topic one of the package topics replaces', async () => {+ const [file] = builtinGuideFiles();+ const coreDoc = /** @type {any} */ (await loadTopicModule(file));+ const coreText = proseWords(coreDoc)+ .slice(0, 30)+ .map(each => each.word)+ .join(' ');+ writeTopics([+ {name: 'acme-own', text: 'Our version.', replaces: coreDoc.name},+ {name: 'acme-copy', text: coreText},+ ]);++ const found = copied(+ await integrationDocConflicts(undefined, {cwd: tmpDir}),+ ).filter((/** @type {any} */ each) => each.owner === coreDoc.name);++ expect(found).toHaveLength(1);+ expect(found[0].message).toContain(+ `"acme-own" replaces the Core topic "${coreDoc.name}"`,+ );+ expect(found[0].message).not.toContain('{@link');+ });+});diff --git a/packages/cli/api/integration/authoring-checks.type.mjs b/packages/cli/api/integration/authoring-checks.type.mjsindex 51a61c7dd6..2f9186f892 100644--- a/packages/cli/api/integration/authoring-checks.type.mjs+++ b/packages/cli/api/integration/authoring-checks.type.mjs@@ -48,6 +48,26 @@* @property {string} message*/+/**+ * A passage two topics repeat word for word: two of the package's own topics,+ * or one of them and a Core topic (never the Core topic it replaces). Only+ * prose counts, and only a run of 10 or more identical words, so each entry+ * is a real copy. Each is also a `copied_passage` warning, so+ * `integration check --strict` fails on it (architecture:cli-surface DEC-8).+ *+ * @typedef {object} CopiedPassage+ * @property {string} topic the package topic to change: it repeats the passage+ * @property {string | null} section the title of the section the passage starts in+ * @property {number | null} line the file line where the passage starts in `topic`, or null+ * @property {string} otherTopic the topic that also has it+ * @property {string} owner the topic that should keep the text: a Core topic, or of two package topics the one it fits+ * @property {string} text the passage as `topic` writes it: what to replace+ * @property {'package' | 'core'} otherOwner whether that topic is the package's own or Core's+ * @property {number} words the length of the longest shared run, in words+ * @property {number} passages how many separate runs the two topics share+ * @property {string} message what is repeated, and the exact fix: the passage in `topic` replaced by a link to `owner`+ */+/*** Each of the three authoring checks reports `validated`: false means no* integration manifest was found, so nothing was inspected and the empty@@ -80,7 +100,7 @@/*** @typedef {object} IntegrationDocConflictResponse* @property {'integration.doc-conflicts'} type- * @property {{validated: boolean, name: string | null, version: string | null, findings: IntegrationDocFinding[], issues: import('../../foundation/integrations/issue').AstryxIntegrationIssue[]}} data+ * @property {{validated: boolean, name: string | null, version: string | null, findings: IntegrationDocFinding[], copiedPassages: CopiedPassage[], issues: import('../../foundation/integrations/issue').AstryxIntegrationIssue[]}} data*/export {};diff --git a/packages/cli/api/integration/integrationDocConflicts.doc.mjs b/packages/cli/api/integration/integrationDocConflicts.doc.mjsindex 2b4a4c8a4f..3d6b6e4b81 100644--- a/packages/cli/api/integration/integrationDocConflicts.doc.mjs+++ b/packages/cli/api/integration/integrationDocConflicts.doc.mjs@@ -9,28 +9,40 @@ export const doc = {name: 'integrationDocConflicts',namespace: 'cli/api',displayName: 'integrationDocConflicts()',- summary: 'Check an integration\'s docs: the docs tree they add, every link, and overlaps with Core topics.',+ summary:+ "Check an integration's docs: the docs tree they add, every link, and overlaps with Core topics.",description:'Loads one local or installed integration and classifies Core doc overlaps as ' +'intentional replacements, intentional extensions, or accidental same-name ' +- 'conflicts that need an explicit relationship or a rename.',+ 'conflicts that need an explicit relationship or a rename. `copiedPassages` lists each ' ++ 'pair of topics that repeat 10 or more words of prose word for word: two of the ' ++ "package's topics, or one of them and a Core topic other than the one it replaces. " ++ 'Code and examples are not compared. Each entry is also a `copied_passage` warning, so ' ++ '`integration check --strict` fails on it; the check itself still fails only on errors.',importPath: '@astryxdesign/cli/api',signature:'integrationDocConflicts(pkg?: string, options?: IntegrationAuthoringOptions): Promise<IntegrationDocConflictResponse>',keywords: ['integration', 'docs', 'replace', 'extend', 'authoring', 'doctor'],params: [- {name: 'pkg', type: 'string', description: 'Installed package; omit for the local package.'},+ {+ name: 'pkg',+ type: 'string',+ description: 'Installed package; omit for the local package.',+ },{name: 'options.cwd', type: 'string', description: 'Resolution directory.'},],returns: [{type: 'integration.doc-conflicts',description:- '`validated` (false when no integration manifest was found, so nothing was inspected), integration identity, structural issues, and classified Core doc relationships.',+ '`validated` (false when no integration manifest was found, so nothing was inspected), integration identity, structural issues, classified Core doc relationships, and `copiedPassages`.',},],examples: [- {label: 'Check the local integration', code: 'await integrationDocConflicts();'},+ {+ label: 'Check the local integration',+ code: 'await integrationDocConflicts();',+ },],command: 'doctor integration docs',related: ['validateIntegration', 'docs'],diff --git a/packages/cli/assets/docs/author-a-theme.doc.mjs b/packages/cli/assets/docs/author-a-theme.doc.mjsindex 8b02dfce48..e57908c3ee 100644--- a/packages/cli/assets/docs/author-a-theme.doc.mjs+++ b/packages/cli/assets/docs/author-a-theme.doc.mjs@@ -17,14 +17,14 @@ export const docs = {{type: 'prose', text: "Use an installed built theme as the base for ordinary customization. Import it into your source and pass it as `extends` to `defineTheme({extends: importedTheme, ...})`. Build the result, then add the local slug. Only eject when you need to own and maintain a full source fork."},{type: 'code', lang: 'bash', label: 'Add to use, eject to fork', code: 'astryx theme list\nastryx theme add stone\nastryx theme eject stone\nastryx theme build src/themes/stone/stoneTheme.ts\nastryx theme add stone'},{type: 'prose', text: "For an annotated map of the whole surface (every defineTheme field, the token families, and the component override syntax, each with the CLI command that prints its reference), run `astryx theme template`. It writes `theme.template.ts` into your project to read and copy from (`astryx init --features theme` writes it as part of project setup)."},- {type: 'prose', text: 'To apply a theme you created, see {@link generic:use-a-theme}. To generate colors from seed values, see the Palette section below. Use `theme eject` only when you want an independent source fork that no longer receives the package owner\u2019s updates.'},+ {type: 'prose', text: 'To apply a theme you created, see {@link generic:use-a-theme}. To generate colors from seed values, see the Palette section below. Before you eject a package theme instead, see when that fits in {@link generic:use-a-theme}.'},],},{id: 'palette',title: 'Generate a palette',content: [- {type: 'prose', text: 'A theme needs dozens of related colors for backgrounds, borders, text, and states, in both light and dark mode. Rather than pick each by hand, name a few seed colors and generate the rest.'},+ {type: 'prose', text: 'Name a few seed colors and generate every shade a theme needs, light and dark; {@link generic:generate-a-palette} explains what a palette holds.'},{type: 'code', lang: 'json', label: 'Minimal palette config (palette.config.json)', code: '{"families": [{"id": "ocean", "seed": "#0074e2"}]}'},{type: 'code', lang: 'bash', label: 'Generate and preview', code: '# Print the palette without writing files\nastryx theme palette generate palette.config.json\n\n# Write palette + receipt, and open a visual preview\nastryx theme palette generate palette.config.json \\\n --out src/themes/ocean/tokens/ocean.palette.ts \\\n --preview preview.html'},{type: 'prose', text: "The command writes a `.ts` file exporting 21 shades (stops 0–100) for both light and dark, plus a `.receipt.json` to regenerate later. Point your theme's tokens at the palette, then run defineTheme (see below)."},diff --git a/packages/cli/assets/docs/styling-overview.doc.mjs b/packages/cli/assets/docs/styling-overview.doc.mjsindex 5341d5900c..a06892a58c 100644--- a/packages/cli/assets/docs/styling-overview.doc.mjs+++ b/packages/cli/assets/docs/styling-overview.doc.mjs@@ -158,7 +158,7 @@ export const docs = {},{type: 'prose',- text: "If importing a swizzled component throws an error that starts `Unexpected 'stylex.create' call at runtime` and the page does not render, the app is missing a StyleX compiler. See the StyleX Build Setup section of {@link generic:tokens-and-setup} for bundler config examples.",+ text: 'If a swizzled component fails at runtime with a `stylex.create` error, the app is missing a StyleX compiler: see the StyleX Build Setup section of {@link generic:tokens-and-setup}.',},],},diff --git a/packages/cli/assets/docs/theme.doc.mjs b/packages/cli/assets/docs/theme.doc.mjsindex 6370536f19..8c91498384 100644--- a/packages/cli/assets/docs/theme.doc.mjs+++ b/packages/cli/assets/docs/theme.doc.mjs@@ -59,7 +59,7 @@ export const docs = {content: [{type: 'prose',- text: "Use [light, dark] tuples in token values for automatic mode switching. Use mode='system' (default) on Theme to follow OS preference. Full guide: {@link generic:use-a-theme}.",+ text: 'Light, dark, and system modes: {@link generic:use-a-theme}.',},{type: 'code',diff --git a/packages/cli/assets/docs/tree/add-a-topic.doc.mjs b/packages/cli/assets/docs/tree/add-a-topic.doc.mjsindex 23c7c492d4..c8ad3168c6 100644--- a/packages/cli/assets/docs/tree/add-a-topic.doc.mjs+++ b/packages/cli/assets/docs/tree/add-a-topic.doc.mjs@@ -138,7 +138,7 @@ export const docs = {items: ['Readers open a topic by its `name`, not its file name. Name the file after the doc, `<name>.doc.mjs`, so each one is easy to find.','A topic with no `placement` is a flat topic: the docs list shows it under Topics, and readers open it by its name.',- 'To give your package its own section in the docs tree instead, see {@link generic:sections-and-placement}.',+ 'For a section of your own in the docs tree, see {@link generic:sections-and-placement}.',],},],diff --git a/packages/cli/assets/docs/tree/check-the-props.doc.mjs b/packages/cli/assets/docs/tree/check-the-props.doc.mjsindex a1eacd41fd..3fb3ef6a12 100644--- a/packages/cli/assets/docs/tree/check-the-props.doc.mjs+++ b/packages/cli/assets/docs/tree/check-the-props.doc.mjs@@ -58,8 +58,8 @@ docType: booleanstyle: 'unordered',items: ["A doc's own `props` describe the export its `name` names. In a family doc, each entry describes its own export. A hook entry documents parameters and returns, so it is not compared.",- "It reads props with the `typescript` your package installs, and reads your package's `tsconfig.json` the way `tsc -p` does. Where `typescript` is missing or has no JavaScript compiler API, as TypeScript 7 does, it uses `@typescript/typescript6`, the TypeScript 6 API package, if your package installs it. Without either it prints `[skip] Props not compared` and the reason.",- 'Every finding is a warning, so the exit code stays 0. Read them before you publish. With `--json`, they are in `data.propDrift.findings`.',+ "It reads props with the `typescript` your package installs, and reads your package's `tsconfig.json` the way `tsc -p` does. It finds TypeScript the way the docs check does, TypeScript 7 included ({@link generic:check-your-docs}). Without a usable one it prints `[skip] Props not compared` and the reason.",+ 'Findings are warnings and leave the exit code at 0. Read them before you publish. With `--json`, they are in `data.propDrift.findings`.',],},],diff --git a/packages/cli/assets/docs/tree/check-your-docs.doc.mjs b/packages/cli/assets/docs/tree/check-your-docs.doc.mjsindex 017f883536..927c3f5bd4 100644--- a/packages/cli/assets/docs/tree/check-your-docs.doc.mjs+++ b/packages/cli/assets/docs/tree/check-your-docs.doc.mjs@@ -22,7 +22,7 @@ export const docs = {content: [{type: 'prose',- text: 'Run `doctor integration docs` in your package to check the docs tree, every link, and overlaps with Core topics. Pass a package name to check an installed package.',+ text: 'Run `doctor integration docs` in your package to check the docs tree, every link, and overlaps with Core topics, and to list any passage two topics repeat. Pass a package name to check an installed package.',},{type: 'code',@@ -32,7 +32,11 @@ export const docs = {{type: 'code',lang: 'text',- code: 'Checking integration docs: @acme/[email protected]\n\n[ok] The docs tree and every link in these docs check out.\n\n[ok] No doc topics overlap with Core.',+ code: 'Checking integration docs: @acme/[email protected]\n\n[ok] The docs tree and every link in these docs check out.\n\n[ok] No doc topics overlap with Core.\n[ok] No topic copies a passage from another.',+ },+ {+ type: 'prose',+ text: 'Write each fact in one topic. When two of your topics, or one of yours and a Core topic, repeat 10 or more words of prose word for word, the check lists the pair as a `copied_passage` warning. It names the topic that keeps the text (a Core topic, or of your two the one the passage fits) and the line in the other where to put the link instead ({@link generic:links}). Code and examples are not compared, and a topic may carry the text of the Core topic it replaces. A run with fewer than 4 distinct content words (words other than function words such as \"the\", \"in\" or \"it\"), or one that three or more topics share (a series lead-in), is not listed. Each listed pair is also a warning (`copied_passage`): `doctor integration docs` still exits 0, and `integration check --strict` fails until you fix it.',},{type: 'prose',diff --git a/packages/cli/assets/docs/tree/checks.doc.mjs b/packages/cli/assets/docs/tree/checks.doc.mjsindex d6f3792294..96df01d5fd 100644--- a/packages/cli/assets/docs/tree/checks.doc.mjs+++ b/packages/cli/assets/docs/tree/checks.doc.mjs@@ -48,9 +48,9 @@ export const docs = {],['`npx astryx doctor integration docs`',- 'Your docs tree, every link, and topic names against Core',+ 'Your docs tree, every link, topic names against Core, and passages two topics repeat word for word','A topic takes a Core topic name without `replaces` or `extends`, a doc is invalid (`invalid_doc`), or a namespace or placement fails, which hides the doc (`invalid_doc_graph`)',- 'A link names no doc (`invalid_doc_graph`)',+ 'A link names no doc (`invalid_doc_graph`), or two topics repeat a passage word for word (`copied_passage`)',],['`npx astryx integration check`',@@ -78,7 +78,7 @@ export const docs = {content: [{type: 'prose',- text: 'Every integration runs `integration check --strict` in its own build, so every build keeps your docs, types, and links in sync; this is required, not optional. Run `integration verify` before you publish. Install devDependencies first, because the components check needs Core, and `tsc --noEmit` needs `typescript` and a `tsconfig.json`. In package.json:',+ text: 'Every integration runs `integration check --strict` in its own build, so every build keeps your docs, types, and links in sync; this is required, not optional. The strict check fails on every copied passage the docs check lists ({@link generic:check-your-docs}). Run `integration verify` before you publish. Install devDependencies first, because the components check needs Core, and `tsc --noEmit` needs `typescript` and a `tsconfig.json`. In package.json:',},{type: 'code',diff --git a/packages/cli/assets/docs/tree/layout-spacing.doc.mjs b/packages/cli/assets/docs/tree/layout-spacing.doc.mjsindex 096344ff5e..a8111395fe 100644--- a/packages/cli/assets/docs/tree/layout-spacing.doc.mjs+++ b/packages/cli/assets/docs/tree/layout-spacing.doc.mjs@@ -13,8 +13,7 @@ export const docs = {title: 'Layout spacing',placement: {parent: 'namespace:layout', slot: 'guides', order: 40},category: 'guide',- description:- 'Hold one content line per region, then tune gaps and density.',+ description: 'Hold one content line per region, then tune gaps and density.',keywords: ['alignment','content line',@@ -35,7 +34,7 @@ export const docs = {content: [{type: 'prose',- text: 'Hold one content line per region, then tune gaps and density. Alignment, Rhythm, and Density and size cover each step; these rules hold for all of them.',+ text: 'Hold one content line per region, then tune gaps and density. Alignment, Rhythm, and Density and size each build on these rules.',},{type: 'list',diff --git a/packages/cli/assets/docs/tree/layout.doc.mjs b/packages/cli/assets/docs/tree/layout.doc.mjsindex 8d6982cb35..68e02dcaf1 100644--- a/packages/cli/assets/docs/tree/layout.doc.mjs+++ b/packages/cli/assets/docs/tree/layout.doc.mjs@@ -32,21 +32,21 @@ export const docs = {blocks: [{type: 'prose',- text: "Build a layout outside-in. Settle the shell and its region budgets before any content exists, then work inward. Content-first layouts drift into a padded column of cards, because every section ends up inventing its own container.",+ text: 'Build a layout outside-in. Settle the shell and its region budgets before any content exists, then work inward. Content-first layouts drift into a padded column of cards, because every section ends up inventing its own container.',},{type: 'list',style: 'ordered',items: [- "Scaffold: pick the shell, budget each region, and choose navigation",- "Structure: rank the content in each region, then pick the weakest container that groups it",- "Spacing: hold one content line per region, then tune gaps and density",- "Breakpoints: decide what each region does as width changes",+ 'Scaffold: pick the shell, budget each region, and choose navigation',+ "Structure: rank each region's content and choose its container",+ 'Spacing: align each region to one content line, then set gaps and density',+ 'Breakpoints: decide what each region does as width changes',],},{type: 'prose',- text: "This guide decides layout, not component APIs. Run `npx astryx build \"<idea>\"` to start from the closest template for your app type, and `npx astryx component <Name>` for a component's props.",+ text: 'This guide decides layout, not component APIs. Run `npx astryx build "<idea>"` to start from the closest template for your app type, and `npx astryx component <Name>` for a component\'s props.',},],slots: {diff --git a/packages/cli/assets/docs/tree/migration-setup.doc.mjs b/packages/cli/assets/docs/tree/migration-setup.doc.mjsindex 36a9b930fb..07feb53ca6 100644--- a/packages/cli/assets/docs/tree/migration-setup.doc.mjs+++ b/packages/cli/assets/docs/tree/migration-setup.doc.mjs@@ -85,7 +85,7 @@ astryx component TextInput`,},{type: 'prose',- text: 'For a shorter read to paste into an AI coding tool, add `--detail brief` (one line per section) or `--detail compact`. Use `--json` when building automated migration reports.',+ text: 'For a shorter read to paste into an AI coding tool, see {@link generic:working-with-ai}. Use `--json` when building automated migration reports.',},{type: 'code',@@ -130,7 +130,7 @@ export function AppRoot({children}: {children: React.ReactNode}) {},{type: 'prose',- text: 'A layer\'s rank is set by the first stylesheet on the page that names it, so the global CSS file (or a `layers.css` holding only the `@layer` line) must be the first import in the root layout or app entry. Import it before any module that imports Astryx or theme CSS, including the `astryx-themes` module generated by `astryx theme add` and a root component like the `AppRoot` above that imports `theme.css`. If a theme stylesheet loads first, Tailwind preflight overrides Astryx base styles and theme overrides lose.',+ text: 'Import the global CSS file first in the root layout or app entry, before anything that imports Astryx or theme CSS, including the `astryx-themes` module and a root component like the `AppRoot` above. Why the order matters, and what to do when it cannot come first, is in {@link generic:tailwind}.',},{type: 'code',diff --git a/packages/cli/assets/docs/tree/publishing.doc.mjs b/packages/cli/assets/docs/tree/publishing.doc.mjsindex f22f5389ff..8690bb88bf 100644--- a/packages/cli/assets/docs/tree/publishing.doc.mjs+++ b/packages/cli/assets/docs/tree/publishing.doc.mjs@@ -59,7 +59,7 @@ export const docs = {items: ['List only the roots you ship, and always `astryx.integration.mjs`. npm adds package.json, and a README and LICENSE when you have them.','Each later `integration add` appends its root to the list.',- '`integration verify` fails when a root is missing from the list: `Declared templates root "templates" has 0 of 2 expected files in the pack list. Add "templates" to "files" in package.json.`',+ '`integration verify` fails when a root is missing from the list, and names the root to add to `files` (the message is in {@link generic:troubleshooting}).',],},],diff --git a/packages/cli/assets/docs/tree/quick-start.doc.mjs b/packages/cli/assets/docs/tree/quick-start.doc.mjsindex 3f61bb079d..3c4ab176e6 100644--- a/packages/cli/assets/docs/tree/quick-start.doc.mjs+++ b/packages/cli/assets/docs/tree/quick-start.doc.mjs@@ -95,7 +95,7 @@ export const docs = {},{type: 'prose',- text: 'Every integration runs `integration check --strict` in its own build, so a doc, prop, template, or link that stops matching the code fails your build instead of reaching the apps that install your package. Set it up now, as the `typecheck` script, and run it.',+ text: "Run `integration check --strict` in every build ({@link generic:checks}): a doc, prop, template, or link that drifts from the code then fails your build, not your users' apps. Set it up now, as the `typecheck` script, and run it.",},{type: 'code',@@ -136,7 +136,7 @@ export const docs = {},{type: 'prose',- text: '`integration verify` packs the package, unpacks it into a temporary app, and checks that the component resolves through its public import there. It publishes nothing and leaves no `.tgz` file, so `npm pack` writes `../acme-astryx-widgets-1.0.0.tgz` for the next step.',+ text: '`integration verify` installs the packed package in a temporary app and confirms the component resolves through its public import there ({@link generic:checks}). It publishes nothing and leaves no `.tgz` file, so `npm pack` writes `../acme-astryx-widgets-1.0.0.tgz` for the next step.',},{type: 'heading',diff --git a/packages/cli/assets/docs/tree/template-doc-overview.doc.mjs b/packages/cli/assets/docs/tree/template-doc-overview.doc.mjsindex 19fa2775a7..ea86d3355a 100644--- a/packages/cli/assets/docs/tree/template-doc-overview.doc.mjs+++ b/packages/cli/assets/docs/tree/template-doc-overview.doc.mjs@@ -67,7 +67,7 @@ export const docs = {'Set `type` to the kind you chose in {@link generic:start-a-template}. It decides which other fields the doc accepts.','The file name sets the template id ({@link generic:start-a-template}). Labels live in the doc: the terminal list prints `name`, and JSON listings print `displayName`. Changing either never changes the id.','Write the description for someone choosing between templates. The Doc metadata category in {@link generic:template-grading-rubric} defines what a complete description covers.',- "Name the ideas the template serves in `keywords`: the domains, tasks, and other names a builder might use for it, in lowercase, such as `['monitoring', 'uptime', 'on-call']` for a service-health dashboard. `astryx search` matches them as it matches the description, and `astryx build` ranks page templates on them, so the description can stay about the layout.",+ "Name the ideas the template serves in `keywords`: the domains, tasks, and other names a builder might use for it, in lowercase, such as `['monitoring', 'uptime', 'on-call']` for a service-health dashboard. How search and build use them is in {@link generic:authoring}, so the description can stay about the layout.",'Add `isReady: false` as soon as you generate the doc, because a doc without `isReady` is listed as ready. Keep it until the template passes {@link generic:test-template-in-app} and the quality review.',],},diff --git a/packages/cli/assets/docs/tree/troubleshooting.doc.mjs b/packages/cli/assets/docs/tree/troubleshooting.doc.mjsindex 550f022150..098d9e08dd 100644--- a/packages/cli/assets/docs/tree/troubleshooting.doc.mjs+++ b/packages/cli/assets/docs/tree/troubleshooting.doc.mjs@@ -78,7 +78,7 @@ export const docs = {"The `exports` map has no entry for it. Run `npm pkg set 'exports[./components/AcmeCarousel]=./components/AcmeCarousel.tsx'`, or `'exports[./templates/acme-dashboard]=./templates/acme-dashboard.tsx'` for a template.",],[- '`component_import_unresolvable`: `…but a consumer cannot resolve it: Cannot find package \'@acme/old-name\'`',+ "`component_import_unresolvable`: `…but a consumer cannot resolve it: Cannot find package '@acme/old-name'`","You renamed the package after the add wrote each doc's `import`. Change `import` in every component doc to the new name.",],[@@ -95,11 +95,11 @@ export const docs = {],['`component_replaces_needs_cli` (a warning): The package has a component that sets `replaces` but declares no @astryxdesign/cli peer, or a peer range that admits an earlier CLI.',- "Run `npm pkg set 'peerDependencies.@astryxdesign/cli=>=0.6.7'` and `npm pkg set 'peerDependenciesMeta.@astryxdesign/cli.optional=true' --json` to turn the replacement on. Until then the component keeps its own name and the Core component stays selected; a component named like its target stays ambiguous by that bare name.",+ "Run `npm pkg set 'peerDependencies.@astryxdesign/cli=>=0.6.7'` and `npm pkg set 'peerDependenciesMeta.@astryxdesign/cli.optional=true' --json` to turn the replacement on. Until then apps see no change ({@link generic:replace-a-core-component}).",],['`keywords_needs_cli`: The package has a template that sets `keywords` but declares no @astryxdesign/cli peer.',- "Run `npm pkg set 'peerDependencies.@astryxdesign/cli=>=0.6.6'` and `npm pkg set 'peerDependenciesMeta.@astryxdesign/cli.optional=true' --json`. A stable CLI before 0.6.6 rejects `keywords` and drops that template, and one before 0.6.4 also hides your doc topics.",+ "Run `npm pkg set 'peerDependencies.@astryxdesign/cli=>=0.6.6'` and `npm pkg set 'peerDependenciesMeta.@astryxdesign/cli.optional=true' --json`. What an older CLI does with the field is in {@link generic:template-doc-overview}.",],['`themes_need_cli`: The package ships a theme but declares no @astryxdesign/cli peer.',@@ -130,7 +130,7 @@ export const docs = {"Its doc still names its file. Set `name` to the export, such as `name: 'AcmeCarousel'` in `acme-carousel.doc.mjs`. A family member is listed only when the module its family doc imports exports it ({@link generic:component-family}).",],[- '`npx astryx component Dialog` opens Core\'s `Dialog`, not yours.',+ "`npx astryx component Dialog` opens Core's `Dialog`, not yours.","A name that Core, a legacy docs package, or another package's file already has stays theirs. Run `npx astryx component Dialog --package @acme/astryx-widgets`, or give your component a name of its own.",],[@@ -151,7 +151,7 @@ export const docs = {],['Your docs section, templates, or themes are missing only in some apps',- "Those apps run a CLI too old to read them: a stable CLI before 0.6.6 for a template that sets `keywords`, or before 0.6.4 for a template that sets `replaces`, a docs section, or a theme. Update `@astryxdesign/cli` there, and keep your optional `@astryxdesign/cli` peer so npm checks the app's CLI against it when the app installs your package.",+ "Those apps run a CLI too old to read them ({@link generic:checks} lists which field needs which CLI). Update `@astryxdesign/cli` there, and keep your optional `@astryxdesign/cli` peer so npm checks the app's CLI against it when the app installs your package.",],],},diff --git a/packages/cli/assets/docs/tree/upgrading.doc.mjs b/packages/cli/assets/docs/tree/upgrading.doc.mjsindex 461d3f9851..3c2603d916 100644--- a/packages/cli/assets/docs/tree/upgrading.doc.mjs+++ b/packages/cli/assets/docs/tree/upgrading.doc.mjs@@ -35,7 +35,7 @@ export const docs = {style: 'unordered',items: ['Run every check in {@link generic:checks}, not only the ones shown.',- 'Name each codemod folder after the Core version whose upgrade should run it; see {@link generic:codemods}.',+ 'Codemod folders are named for Core versions: {@link generic:codemods}.','Release the result as a new version of your package; see {@link generic:publishing}.',],},diff --git a/packages/cli/authoring/doctypes/component/component.doc.mjs b/packages/cli/authoring/doctypes/component/component.doc.mjsindex 04b439a805..e371c25fc5 100644--- a/packages/cli/authoring/doctypes/component/component.doc.mjs+++ b/packages/cli/authoring/doctypes/component/component.doc.mjs@@ -56,7 +56,7 @@ export const doc = {name: 'replaces',type: 'string',description:- 'Integration components only: the exact `name` of the Core ComponentDoc this component takes over for unqualified lookup, so every app that loads the integration gets it from component detail, component lists, search, `swizzle <Name>`, and issue routing; `swizzle --list` keeps listing Core names. The Core original stays reachable with `--package @astryxdesign/core`. It takes effect only when the package\'s peer range starts at the release that applies it, `"@astryxdesign/cli": ">=0.6.7"` or later; without such a range the component keeps its own name and `doctor integration components` warns. Set it only to intentionally own a Core identity; give an alternative or variant its own name instead.',+ 'Integration components only: the exact `name` of the Core ComponentDoc this component takes over for unqualified lookup, so every app that loads the integration gets it from component detail, component lists, search, `swizzle <Name>`, and issue routing; `swizzle --list` keeps listing Core names. The Core original stays reachable with `--package @astryxdesign/core`. It takes effect only when the package\'s peer range starts at the release that applies it, `"@astryxdesign/cli": ">=0.6.7"` or later; with a lower range nothing changes and `doctor integration components` warns ({@link generic:replace-a-core-component}). Set it only to intentionally own a Core identity; give an alternative or variant its own name instead.',},{name: 'keywords',diff --git a/packages/cli/clients/cli/commands/doctor-integration-docs.doc.mjs b/packages/cli/clients/cli/commands/doctor-integration-docs.doc.mjsindex b45a3cd19b..75fe616494 100644--- a/packages/cli/clients/cli/commands/doctor-integration-docs.doc.mjs+++ b/packages/cli/clients/cli/commands/doctor-integration-docs.doc.mjs@@ -8,12 +8,15 @@ export const doc = {name: 'doctor integration docs',displayName: 'astryx doctor integration docs',namespace: 'cli/commands',- summary: 'Check an integration\'s docs: the docs tree they add, every link, and overlaps with Core topics',+ summary:+ "Check an integration's docs: the docs tree they add, every link, and overlaps with Core topics",description:'Checks the docs tree the package adds, every link in its docs, and overlaps with Core topics. ' +'Intentional replacements and extensions are information. ' +'A same-name topic without `replaces` or `extends` is an accidental conflict ' +- 'and fails until the author declares the relationship or renames it. See {@link generic:check-your-docs}.',+ 'and fails until the author declares the relationship or renames it. ' ++ 'It also lists each pair of topics (two of the package, or one of them and a Core topic other than the one it replaces) ' ++ 'that repeat 10 or more words of prose word for word, as a `copied_passage` warning (exit 0; `integration check --strict` fails on it). See {@link generic:check-your-docs}.',fn: 'integrationDocConflicts',args: [{@@ -25,15 +28,24 @@ export const doc = {},],examples: [- {label: 'Check the local integration', cli: 'astryx doctor integration docs'},+ {+ label: 'Check the local integration',+ cli: 'astryx doctor integration docs',+ },{label: 'Check an installed integration',cli: 'astryx doctor integration docs @acme/widgets --json',},],exitCodes: [- {code: 0, when: 'all Core overlaps are explicitly declared or no overlap exists'},- {code: 1, when: 'an overlap is accidental, a doc is invalid, a namespace or placement fails, or a reference block cannot include what it names. A link that names no doc is a warning, and does not change the exit code'},+ {+ code: 0,+ when: 'all Core overlaps are explicitly declared or no overlap exists',+ },+ {+ code: 1,+ when: 'an overlap is accidental, a doc is invalid, a namespace or placement fails, or a reference block cannot include what it names. A link that names no doc is a warning, and does not change the exit code',+ },],related: ['doctor integration validate', 'docs'],};diff --git a/packages/cli/clients/cli/commands/doctor.mjs b/packages/cli/clients/cli/commands/doctor.mjsindex 9c5ea67ab1..1cb5f60c1f 100644--- a/packages/cli/clients/cli/commands/doctor.mjs+++ b/packages/cli/clients/cli/commands/doctor.mjs@@ -358,7 +358,8 @@ function printComponentConflicts(data) {// An [ok] after a failed check reads as a pass: say nothing it could not check.const failed = data.issues.some(issue => issue.severity === 'error');if (data.conflicts.length === 0) {- if (!failed) output.push(text('[ok] No component names conflict with Core.'));+ if (!failed)+ output.push(text('[ok] No component names conflict with Core.'));} else {output.push(records(data.conflicts, {@@ -389,7 +390,11 @@ function printDocConflicts(data) {section(`Checking integration docs: ${integrationLabel(data) ?? '(local package)'}`,),- ...issueBlocks(data.issues),+ // A copied passage is listed below with its pair, so its issue is not+ // printed twice.+ ...issueBlocks(+ data.issues.filter(issue => issue.code !== 'copied_passage'),+ ),];if (!data.issues.some(@@ -412,6 +417,32 @@ function printDocConflicts(data) {}),);}+ const copied = data.copiedPassages ?? [];+ if (copied.length === 0) {+ if (!data.issues.some(issue => issue.severity === 'error')) {+ output.push(text('[ok] No topic copies a passage from another.'));+ }+ } else {+ output.push(+ text(+ `[warn] ${copied.length} copied passage(s) (copied_passage): two topics say the same thing word for word. Keep the text in one and link to it from the other.`,+ ),+ records(copied, {+ fields: [+ 'topic',+ 'section',+ 'line',+ 'otherTopic',+ 'otherOwner',+ 'owner',+ 'text',+ 'words',+ 'passages',+ 'message',+ ],+ }),+ );+ }emit(...output);}@@ -455,7 +486,7 @@ async function runIntegrationValidation(program, pkg) {noPackage? 'Nothing was validated: no package.json found. Run this command inside a package directory.': 'Nothing was validated: no astryx.integration.* found next to package.json. ' +- 'To validate an installed integration: astryx doctor integration validate <package>',+ 'To validate an installed integration: astryx doctor integration validate <package>',),);} else printIntegrationValidation(result.data);diff --git a/packages/cli/clients/cli/commands/integration-check.test.mjs b/packages/cli/clients/cli/commands/integration-check.test.mjsindex 19175b4803..88eaecf1ba 100644--- a/packages/cli/clients/cli/commands/integration-check.test.mjs+++ b/packages/cli/clients/cli/commands/integration-check.test.mjs@@ -282,3 +282,51 @@ describe('astryx integration check', () => {expect(run.status).toBe(1);});});++describe('a copied passage', () => {+ const passage =+ 'Themes cannot style the layer that holds a menu or a popover, so a menu ' ++ 'keeps the default surface color until the next release adds a token for it.';++ function writeCopiedDocs() {+ fs.writeFileSync(+ path.join(tmpDir, 'package.json'),+ JSON.stringify({name: '@acme/widgets', version: '1.2.3'}),+ );+ fs.writeFileSync(+ path.join(tmpDir, 'astryx.integration.mjs'),+ `export default {docs: './docs'};\n`,+ );+ fs.mkdirSync(path.join(tmpDir, 'docs'));+ for (const name of ['acme-setup', 'acme-menus']) {+ const doc = {+ type: 'generic',+ name,+ title: 'Fixture',+ description: 'Fixture topic.',+ sections: [+ {title: 'Overview', content: [{type: 'prose', text: passage}]},+ ],+ };+ fs.writeFileSync(+ path.join(tmpDir, 'docs', `${name}.doc.mjs`),+ `export default ${JSON.stringify(doc)};\n`,+ );+ }+ }++ it('keeps doctor integration docs at exit 0, and fails integration check --strict', async () => {+ writeCopiedDocs();++ const doctor = await runCli(['doctor', 'integration', 'docs'], tmpDir);+ expect(doctor.status).toBe(0);+ expect(doctor.stdout).toContain('1 copied passage(s)');++ const loose = await runCli(['integration', 'check'], tmpDir);+ expect(loose.status).toBe(0);++ const strict = await runCli(['integration', 'check', '--strict'], tmpDir);+ expect(strict.status).toBe(1);+ expect(strict.stdout).toContain('copied_passage');+ });+});diff --git a/packages/cli/foundation/discovery/copied-passages.mjs b/packages/cli/foundation/discovery/copied-passages.mjsnew file mode 100644index 0000000000..9650ae8925--- /dev/null+++ b/packages/cli/foundation/discovery/copied-passages.mjs@@ -0,0 +1,720 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++/**+ * Copied passages: prose that two docs repeat word for word. One fact has one+ * owner, so a passage repeated in a second doc goes stale in one of them the+ * next time the other is edited.+ *+ * Only prose is compared: code, examples, names, titles, keywords, and links+ * are left out, and a run must be at least {@link PASSAGE_WORDS} words long,+ * so a reported passage is always a real copy, never a shared phrase.+ */++import fs from 'node:fs';++import {builtinGuideFiles, loadTopicModule} from './docs-discovery.mjs';++/** The shortest run of identical words reported as a copy. */+export const PASSAGE_WORDS = 10;++/**+ * A copy carries at least this many distinct content words, so a run of+ * glue words ("and it is in the way that you can…") is a phrase, not a fact.+ */+export const MIN_CONTENT_WORDS = 4;++/**+ * A run that this many topics share is boilerplate, such as a lead-in every+ * guide of a series opens with, not a copy of one topic's fact.+ */+export const BOILERPLATE_TOPICS = 3;++/** English function words, which say nothing about what a passage states. */+const FUNCTION_WORDS = new Set(+ (+ 'a about above after again all also an and any are as at be because been before being below between both but by ' ++ 'can could did do does doing down during each either else every few for from further had has have having he her here ' ++ 'him his how i if in into is it its itself just may me might more most must my no nor not now of off on once only or ' ++ 'other our out over own same she should so some such than that the their them then there these they this those through ' ++ 'to too under until up us very was we were what when where which while who whom why will with would you your'+ ).split(' '),+);++/**+ * The number of distinct content words in a run: words that are not function+ * words, with surrounding punctuation set aside.+ *+ * @param {{word: string}[]} words+ * @param {number} start+ * @param {number} length+ */+export function contentWords(words, start, length) {+ const found = new Set();+ for (const {word} of words.slice(start, start + length)) {+ const bare = word.replace(/^[^a-z0-9@]+|[^a-z0-9]+$/g, '');+ if (bare && !FUNCTION_WORDS.has(bare)) found.add(bare);+ }+ return found.size;+}++/** Fields that hold identifiers or code rather than prose. */+const SKIP_KEYS = new Set([+ 'aliases',+ 'audience',+ 'category',+ 'code',+ 'example',+ 'examples',+ 'extends',+ 'id',+ 'import',+ 'keywords',+ 'label',+ 'lang',+ 'name',+ 'placement',+ 'projection',+ 'props',+ 'relationships',+ 'replaces',+ 'source',+ 'title',+ 'type',+]);++/** Blocks whose content is code, or another doc's content included by name. */+const SKIP_BLOCKS = new Set(['code', 'example', 'reference']);++/**+ * Pointers to other docs, which are not prose: a `{@link …}` and a written+ * `astryx docs <name>` command. Two topics pointing at the same section say+ * no fact twice.+ */+const POINTERS = /\{@link [^}]*\}|`astryx docs [^`]*`/g;++/**+ * The words a passage is compared by: lowercase, with links and Markdown+ * emphasis set aside. A token with no letter or digit (the `.` after a link)+ * is not a word.+ *+ * @param {string} text+ * @returns {string[]}+ */+export function passageWords(text) {+ return text+ .toLowerCase()+ .replace(POINTERS, ' ')+ .replace(/[`*]/g, '')+ .split(/[^a-z0-9'@./-]+/)+ .filter(word => /[a-z0-9]/.test(word));+}++/**+ * Every word of a doc file, in order, with its 1-based line. Words are runs of+ * letters and digits, so punctuation, code ticks, quotes and escapes in the+ * source neither join nor split them.+ *+ * @param {string} source+ * @returns {{word: string, line: number}[]}+ */+export function fileWords(source) {+ /** @type {{word: string, line: number}[]} */+ const words = [];+ source.split('\n').forEach((line, at) => {+ for (const match of line.matchAll(/[A-Za-z0-9]+/g)) {+ words.push({word: match[0].toLowerCase(), line: at + 1});+ }+ });+ return words;+}++/**+ * The words of a string the way {@link fileWords} reads a file, each with its+ * offset in the string.+ *+ * @param {string} text+ * @returns {{word: string, at: number}[]}+ */+export function textWords(text) {+ return [...text.matchAll(/[A-Za-z0-9]+/g)].map(match => ({+ word: match[0].toLowerCase(),+ at: /** @type {number} */ (match.index),+ }));+}++/**+ * Where a string's whole run of words starts among a file's words, looking+ * from `from` on, or -1.+ *+ * @param {{word: string}[]} file+ * @param {{word: string}[]} words+ * @param {number} [from]+ * @returns {number}+ */+export function locateWords(file, words, from = 0) {+ if (words.length === 0) return -1;+ outer: for (let i = from; i + words.length <= file.length; i++) {+ for (let k = 0; k < words.length; k++) {+ if (file[i + k].word !== words[k].word) continue outer;+ }+ return i;+ }+ return -1;+}++/**+ * The file line of the character at offset `at` of a string whose words start+ * at `index` among the file's words.+ *+ * @param {{line: number}[]} file+ * @param {number} index+ * @param {{at: number}[]} words the string's words+ * @param {number} at+ * @returns {number | null}+ */+export function lineAtOffset(file, index, words, at) {+ let k = words.findIndex(word => word.at + 1 > at);+ if (k < 0) k = words.length - 1;+ return file[index + k]?.line ?? null;+}++/**+ * A shared run with the boilerplate trimmed off its ends, judged window by+ * window: every word a leading boilerplate window covers, and every word from+ * the first trailing one on. A lead-in many topics open with thus never hides+ * the copy after it. Null when less than {@link PASSAGE_WORDS} words are left.+ *+ * @param {{start: number, length: number}} run+ * @param {(at: number) => boolean} boilerplate whether the window of+ * {@link PASSAGE_WORDS} words starting at `at` is boilerplate+ * @returns {{start: number, length: number} | null}+ */+export function trimBoilerplate({start, length}, boilerplate) {+ const lastWindow = start + length - PASSAGE_WORDS;+ let lead = start;+ while (lead <= lastWindow && boilerplate(lead)) lead += 1;+ const begin = lead > start ? lead - 1 + PASSAGE_WORDS : start;+ let tail = lastWindow;+ while (tail >= begin && boilerplate(tail)) tail -= 1;+ const end = tail < lastWindow ? tail + 1 : start + length;+ return end - begin < PASSAGE_WORDS+ ? null+ : {start: begin, length: end - begin};+}++/**+ * The words of one string as passageWords reads them, each with where it sits+ * in the string as written, so a passage can be quoted with its case,+ * punctuation and code ticks.+ *+ * @param {string} text+ * @returns {{word: string, start: number, end: number}[]}+ */+export function wordSpans(text) {+ const masked = text+ .toLowerCase()+ .replace(POINTERS, pointer => ' '.repeat(pointer.length));+ /** @type {{word: string, start: number, end: number}[]} */+ const spans = [];+ for (const match of masked.matchAll(/[a-z0-9'@./`*-]+/g)) {+ const word = match[0].replace(/[`*]/g, '');+ if (!/[a-z0-9]/.test(word)) continue;+ const start = /** @type {number} */ (match.index);+ spans.push({word, start, end: start + match[0].length});+ }+ return spans;+}++/**+ * Each prose string of a doc, in order, with the title of the section it is+ * in (null before the first section). Code, examples, reference blocks, and+ * identifier fields are left out.+ *+ * @param {unknown} doc an authored doc object+ * @returns {{text: string, section: string | null}[]}+ */+export function proseStrings(doc) {+ /** @type {{text: string, section: string | null}[]} */+ const strings = [];+ /**+ * @param {unknown} value+ * @param {string | null} section+ */+ const walk = (value, section) => {+ if (typeof value === 'string') {+ strings.push({text: value, section});+ return;+ }+ if (Array.isArray(value)) {+ for (const item of value) walk(item, section);+ return;+ }+ if (value == null || typeof value !== 'object') return;+ const record = /** @type {Record<string, unknown>} */ (value);+ if (typeof record.type === 'string' && SKIP_BLOCKS.has(record.type)) return;+ if (Array.isArray(record.sections)) {+ for (const [key, field] of Object.entries(record)) {+ if (key !== 'sections' && !SKIP_KEYS.has(key)) walk(field, section);+ }+ for (const each of record.sections) {+ const title =+ typeof each?.title === 'string' ? each.title : (section ?? null);+ walk(each, title);+ }+ return;+ }+ for (const [key, field] of Object.entries(record)) {+ if (!SKIP_KEYS.has(key)) walk(field, section);+ }+ };+ walk(doc, null);+ return strings;+}++/**+ * Each prose word of a doc, in order, with the title of the section it is in+ * (null before the first section) and the number of the string it came from.+ * A passage never runs from one string into the next, so two fields that+ * happen to sit side by side are not read as one sentence.+ *+ * @param {unknown} doc an authored doc object+ * @returns {{word: string, section: string | null, block: number}[]}+ */+export function proseWords(doc) {+ /** @type {{word: string, section: string | null, block: number}[]} */+ const words = [];+ proseStrings(doc).forEach(({text, section}, at) => {+ for (const word of passageWords(text)) {+ words.push({word, section, block: at + 1});+ }+ });+ return words;+}++/**+ * Whether the `size` words from `at` all come from one string.+ * @param {{block?: number}[]} words+ * @param {number} at+ * @param {number} size+ */+function inOneString(words, at, size) {+ return words[at].block === words[at + size - 1].block;+}++/**+ * The runs of at least `size` words that both docs have, each inside one+ * string of each doc, longest first, as its start in `mine` and its length.+ *+ * @param {{word: string, block?: number}[]} mine+ * @param {{word: string, block?: number}[]} other+ * @param {number} [size]+ * @returns {{start: number, length: number}[]}+ */+export function sharedPassages(mine, other, size = PASSAGE_WORDS) {+ if (mine.length < size || other.length < size) return [];+ const theirs = other.map(each => each.word);+ /** @type {Map<string, number[]>} */+ const starts = new Map();+ for (let i = 0; i + size <= theirs.length; i++) {+ if (!inOneString(other, i, size)) continue;+ const key = theirs.slice(i, i + size).join(' ');+ const list = starts.get(key);+ if (list) list.push(i);+ else starts.set(key, [i]);+ }+ const words = mine.map(each => each.word);+ /** @type {{start: number, length: number}[]} */+ const runs = [];+ let i = 0;+ while (i + size <= words.length) {+ const found = inOneString(mine, i, size)+ ? starts.get(words.slice(i, i + size).join(' '))+ : undefined;+ if (!found) {+ i++;+ continue;+ }+ let best = size;+ for (const j of found) {+ let length = size;+ while (+ i + length < words.length &&+ j + length < theirs.length &&+ mine[i + length].block === mine[i].block &&+ other[j + length].block === other[j].block &&+ words[i + length] === theirs[j + length]+ ) {+ length++;+ }+ best = Math.max(best, length);+ }+ runs.push({start: i, length: best});+ i += best;+ }+ return runs.sort((a, b) => b.length - a.length);+}++/**+ * @typedef {{name: string, title?: string, file?: string, doc?: unknown, words: {word: string, section: string | null, block: number}[], replaces: string | null, extendsTopic?: string | null}} GuideWords+ */++/**+ * Each run of {@link PASSAGE_WORDS} words inside one string of a doc, joined.+ * @param {{word: string, block: number}[]} words+ * @returns {Set<string>}+ */+function windows(words) {+ /** @type {Set<string>} */+ const keys = new Set();+ for (let i = 0; i + PASSAGE_WORDS <= words.length; i++) {+ if (!inOneString(words, i, PASSAGE_WORDS)) continue;+ keys.add(+ words+ .slice(i, i + PASSAGE_WORDS)+ .map(each => each.word)+ .join(' '),+ );+ }+ return keys;+}++/**+ * Guides with an index from each run of words to the guides that have it, so+ * a topic is compared in full only with the guides it shares a run with.+ * @typedef {{guides: GuideWords[], index: Map<string, number[]>}} IndexedGuides+ */++/**+ * @param {GuideWords[]} guides+ * @returns {IndexedGuides}+ */+function indexGuides(guides) {+ /** @type {Map<string, number[]>} */+ const index = new Map();+ guides.forEach((guide, at) => {+ for (const key of windows(guide.words)) {+ const list = index.get(key);+ if (list) list.push(at);+ else index.set(key, [at]);+ }+ });+ return {guides, index};+}++/**+ * The indexes of the guides that share at least one run with `words`.+ * @param {{word: string, block: number}[]} words+ * @param {IndexedGuides} indexed+ * @returns {number[]}+ */+function candidates(words, indexed) {+ /** @type {Set<number>} */+ const found = new Set();+ for (const key of windows(words)) {+ for (const at of indexed.index.get(key) ?? []) found.add(at);+ }+ return [...found].sort((a, b) => a - b);+}++/** @type {Promise<IndexedGuides> | null} */+let coreGuides = null;++/**+ * @param {string} file+ * @returns {Promise<any>} the doc, or null for one that does not load (the+ * load problem is reported elsewhere)+ */+async function loadQuietly(file) {+ try {+ return await loadTopicModule(file);+ } catch {+ return null;+ }+}++/** @returns {Promise<IndexedGuides>} the CLI's own guides, read and indexed once */+function loadCoreGuides() {+ coreGuides ??= (async () => {+ /** @type {GuideWords[]} */+ const guides = [];+ for (const file of builtinGuideFiles()) {+ const doc = await loadQuietly(file);+ if (typeof doc?.name === 'string') {+ guides.push({name: doc.name, words: proseWords(doc), replaces: null});+ }+ }+ return indexGuides(guides);+ })();+ return coreGuides;+}++/**+ * One entry for each pair of topics that repeat a passage word for word: two+ * of the package's own topics, or one of them and a Core topic, except the+ * Core topic it replaces (a replacement takes that topic's place, so it may+ * carry its text).+ *+ * @param {{records: {name: string, path: string, replaces?: string | null, extendsTopic?: string | null}[], guides?: {name: string, ref?: {topicFile?: string}}[]}} discovered+ * the package's docs, as discoverIntegrationDocs returns them+ * @returns {Promise<import('../../api/integration/authoring-checks.type.mjs').CopiedPassage[]>}+ */+export async function findCopiedPassages(discovered) {+ const topics = [+ ...discovered.records.map(record => ({+ name: record.name,+ file: record.path,+ replaces: record.replaces ?? null,+ extendsTopic: record.extendsTopic ?? null,+ })),+ ...(discovered.guides ?? []).map(guide => ({+ name: guide.name,+ file: guide.ref?.topicFile,+ replaces: null,+ extendsTopic: null,+ })),+ ];+ /** @type {GuideWords[]} */+ const mine = [];+ for (const each of topics) {+ const doc = each.file ? await loadQuietly(each.file) : null;+ if (doc) {+ mine.push({+ ...each,+ title: typeof doc.title === 'string' ? doc.title : each.name,+ doc,+ words: proseWords(doc),+ });+ }+ }+ if (mine.length === 0) return [];+ const core = await loadCoreGuides();+ const coreNames = new Set(core.guides.map(guide => guide.name.toLowerCase()));+ const packageNames = new Set(mine.map(each => each.name.toLowerCase()));+ /**+ * The page a topic's sections show on: its own, or the one it extends.+ * @param {GuideWords} each+ */+ const pageOf = each => (each.extendsTopic ?? each.name).toLowerCase();+ /** @type {import('../../api/integration/authoring-checks.type.mjs').CopiedPassage[]} */+ const found = [];+ /**+ * How well a passage fits a topic: how many of its content words the+ * topic's name, title, or the section the passage sits in also use.+ * @param {GuideWords} topic+ * @param {string[]} words the passage's words+ * @param {string | null} section+ */+ const fit = (topic, words, section) => {+ // A plural matches its singular: a "menus" topic fits a passage on a menu.+ const stem = (/** @type {string} */ word) =>+ word.length > 3 && word.endsWith('s') ? word.slice(0, -1) : word;+ // Function words ("the", "with") fit every passage, so only content+ // words count, as contentWords counts them.+ const label = new Set(+ passageWords(+ `${topic.name.replace(/-/g, ' ')} ${topic.title ?? ''} ${section ?? ''}`,+ )+ .filter(word => !FUNCTION_WORDS.has(word))+ .map(stem),+ );+ return new Set(words.map(stem).filter(word => label.has(word))).size;+ };+ /**+ * The file line where a passage starts, found from the prose string it is+ * in, or null.+ * @param {GuideWords} topic+ * @param {number} start+ */+ const lineOf = (topic, start) => {+ if (!topic.file || !topic.doc) return null;+ const text = proseStrings(topic.doc)[topic.words[start].block - 1]?.text;+ if (!text) return null;+ let source;+ try {+ source = fs.readFileSync(topic.file, 'utf8');+ } catch {+ return null;+ }+ const file = fileWords(source);+ const words = textWords(text);+ const index = locateWords(file, words);+ if (index < 0) return null;+ // The passage starts at the string's word the run's first word opens.+ let first = start;+ while (+ first > 0 &&+ topic.words[first - 1].block === topic.words[start].block+ ) {+ first -= 1;+ }+ const span = wordSpans(text)[start - first];+ return span ? lineAtOffset(file, index, words, span.start) : null;+ };+ /**+ * A passage as written: the words of the run, from the string it is in.+ * @param {GuideWords} topic+ * @param {number} start+ * @param {number} length+ * @returns {string | null}+ */+ const writtenText = (topic, start, length) => {+ if (!topic.doc) return null;+ const {block} = topic.words[start];+ const text = proseStrings(topic.doc)[block - 1]?.text;+ if (!text) return null;+ let first = start;+ while (first > 0 && topic.words[first - 1].block === block) first -= 1;+ const spans = wordSpans(text);+ const from = spans[start - first];+ const to = spans[start - first + length - 1];+ return from && to ? text.slice(from.start, to.end) : null;+ };+ /**+ * One entry for a pair: the topic that should keep the text (a Core topic,+ * or of two package topics the one the passage fits), and the exact change+ * for the other.+ * @param {GuideWords} first+ * @param {GuideWords} second+ * @param {'package' | 'core'} kind+ */+ const compare = (first, second, kind) => {+ const filtered = (+ /** @type {GuideWords} */ a,+ /** @type {GuideWords} */ b,+ ) =>+ sharedPassages(a.words, b.words).flatMap(({start, length}) => {+ // Boilerplate is judged window by window: the leading and trailing+ // 10-word windows three or more topics share are trimmed, so a+ // lead-in many topics open with does not hide the copy after it.+ const run = trimBoilerplate(+ {start, length},+ at => topicsSharing(a.words, at) >= BOILERPLATE_TOPICS,+ );+ return run &&+ contentWords(a.words, run.start, run.length) >= MIN_CONTENT_WORDS+ ? [run]+ : [];+ });+ let runs = filtered(first, second);+ if (runs.length === 0) return;+ let topic = first;+ let owner = second;+ let why = 'Core keeps the text it shares';+ if (kind === 'package') {+ const words = first.words+ .slice(runs[0].start, runs[0].start + runs[0].length)+ .map(each => each.word);+ const otherRuns = filtered(second, first);+ const firstFit = fit(first, words, first.words[runs[0].start].section);+ const secondFit = otherRuns.length+ ? fit(second, words, second.words[otherRuns[0].start].section)+ : 0;+ // The passage stays where it fits best; on a tie, in the topic whose+ // name sorts first, so the advice is the same on every run.+ const firstKeeps =+ firstFit > secondFit ||+ (firstFit === secondFit && first.name.localeCompare(second.name) < 0);+ if (firstKeeps && otherRuns.length) {+ topic = second;+ owner = first;+ runs = otherRuns;+ }+ why =+ firstFit === secondFit+ ? `"${owner.name}" keeps it: neither topic's name, title or section fits it better, so the one whose name sorts first does`+ : `"${owner.name}" keeps it: its name, title or section fits the passage`;+ }+ const [{start, length}] = runs;+ const written =+ writtenText(topic, start, length) ??+ topic.words+ .slice(start, start + length)+ .map(each => each.word)+ .join(' ');+ const excerpt =+ written.length <= 240+ ? written+ : `${written.slice(0, 120)} … ${written.slice(-80)}`;+ const section = topic.words[start].section;+ const line = lineOf(topic, start);+ const where = [+ section ? `section "${section}"` : null,+ line == null ? null : `line ${line}`,+ ]+ .filter(Boolean)+ .join(', ');+ const more =+ runs.length > 1 ? `, and ${runs.length - 1} more passage(s)` : '';+ const named =+ kind === 'core' ? `the Core topic "${owner.name}"` : `"${owner.name}"`;+ // An extension has no page of its own: its sections show on the page of+ // the topic it extends, so a link to its text names that topic. Topic+ // names resolve without regard to case, as the docs catalog reads them.+ const page = pageOf(owner);+ // The page is Core's only when the owner is a Core guide, or an extension+ // of a Core topic that no package topic of the same name replaces.+ const corePage =+ kind === 'core' ||+ (owner.extendsTopic != null &&+ coreNames.has(page) &&+ !packageNames.has(page));+ const link = corePage+ ? `{@link @astryxdesign/cli:generic:${page}}`+ : `{@link generic:${page}}`;+ // A Core topic one of the package's topics replaces opens that+ // replacement, which does not carry the passage, so no link reaches it.+ const replacer =+ kind === 'core'+ ? mine.find(each => each.replaces?.toLowerCase() === page)+ : undefined;+ // Two topics on one page (one extends the other, or both extend the same+ // topic) would link the page to itself: the copy goes instead.+ const fix = replacer+ ? `"${replacer.name}" replaces the Core topic "${owner.name}", so a link to it opens "${replacer.name}", which does not carry this text: reword the passage in "${topic.name}" in your own words, or move it into "${replacer.name}".`+ : pageOf(topic) === page+ ? `"${topic.name}" and "${owner.name}" show on the same page ("${page}"): delete the passage from "${topic.name}", or, if its section overrides one there, drop the override so the original shows.`+ : `In "${topic.name}", replace that whole passage, from its first word to its last, code ticks included, with ${link}.`;+ found.push({+ topic: topic.name,+ section,+ line,+ otherTopic: owner.name,+ otherOwner: kind,+ owner: owner.name,+ text: written,+ words: length,+ passages: runs.length,+ message:+ `"${topic.name}"${where ? ` (${where})` : ''} repeats ${length} words of ${named} word for word${more}: "${excerpt}". ` ++ `${why}. ${fix}`,+ });+ };+ const own = indexGuides(mine);+ /**+ * How many topics, the package's and Core's, have the run that starts here.+ * @param {{word: string}[]} words+ * @param {number} start+ */+ const topicsSharing = (words, start) => {+ const key = words+ .slice(start, start + PASSAGE_WORDS)+ .map(each => each.word)+ .join(' ');+ return (+ new Set(own.index.get(key) ?? []).size ++ new Set(core.index.get(key) ?? []).size+ );+ };+ mine.forEach((topic, i) => {+ for (const at of candidates(topic.words, own)) {+ if (at > i) compare(topic, mine[at], 'package');+ }+ for (const at of candidates(topic.words, core)) {+ const other = core.guides[at];+ if (topic.replaces?.toLowerCase() !== other.name.toLowerCase()) {+ compare(topic, other, 'core');+ }+ }+ });+ return found;+}diff --git a/packages/cli/foundation/discovery/copied-passages.test.mjs b/packages/cli/foundation/discovery/copied-passages.test.mjsnew file mode 100644index 0000000000..83502340a8--- /dev/null+++ b/packages/cli/foundation/discovery/copied-passages.test.mjs@@ -0,0 +1,124 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++import {describe, expect, it} from 'vitest';++import {+ PASSAGE_WORDS,+ contentWords,+ passageWords,+ wordSpans,+ proseWords,+ sharedPassages,+} from './copied-passages.mjs';++const words = (/** @type {string} */ text) =>+ passageWords(text).map(word => ({word, section: null, block: 1}));++describe('passageWords', () => {+ it('sets aside case, links, and Markdown emphasis', () => {+ expect(+ passageWords('Use **`Button`** for {@link component:Button} actions.'),+ ).toEqual(['use', 'button', 'for', 'actions.']);+ });+});++describe('proseWords', () => {+ it('reads prose with its section, and skips code, references, and names', () => {+ const doc = {+ type: 'generic',+ name: 'acme-guide',+ title: 'Acme guide',+ description: 'Intro words.',+ sections: [+ {+ title: 'Setup',+ content: [+ {type: 'prose', text: 'Install it.'},+ {type: 'code', lang: 'sh', code: 'npm install acme'},+ {type: 'reference', ref: 'generic:other'},+ {type: 'list', items: ['First step.']},+ ],+ },+ ],+ };+ expect(proseWords(doc)).toEqual([+ {word: 'intro', section: null, block: 1},+ {word: 'words.', section: null, block: 1},+ {word: 'install', section: 'Setup', block: 2},+ {word: 'it.', section: 'Setup', block: 2},+ {word: 'first', section: 'Setup', block: 3},+ {word: 'step.', section: 'Setup', block: 3},+ ]);+ });+});++describe('sharedPassages', () => {+ const passage = Array.from(+ {length: PASSAGE_WORDS + 5},+ (_, i) => `w${i}`,+ ).join(' ');++ it('finds the longest shared run', () => {+ expect(+ sharedPassages(words(`a b ${passage} c`), words(`x ${passage} y`)),+ ).toEqual([{start: 2, length: PASSAGE_WORDS + 5}]);+ });++ it('never runs a passage from one string into the next', () => {+ const half = Math.ceil(PASSAGE_WORDS / 2);+ const first = passage.split(' ').slice(0, half).join(' ');+ const second = passage.split(' ').slice(half, PASSAGE_WORDS).join(' ');+ const split = [+ ...passageWords(first).map(word => ({word, section: null, block: 1})),+ ...passageWords(second).map(word => ({word, section: null, block: 2})),+ ];+ expect(sharedPassages(split, words(`${first} ${second}`))).toEqual([]);+ });++ it('ignores a run shorter than the passage size', () => {+ const short = passage+ .split(' ')+ .slice(0, PASSAGE_WORDS - 1)+ .join(' ');+ expect(+ sharedPassages(words(`a ${short} b`), words(`c ${short} d`)),+ ).toEqual([]);+ });+});++describe('contentWords', () => {+ it('counts distinct words that are not function words', () => {+ const run = words('and it is in the way that you can do it for them');+ expect(contentWords(run, 0, run.length)).toBe(1);+ const fact = words('Name each codemod folder after the Core version.');+ expect(contentWords(fact, 0, fact.length)).toBe(5);+ });+});++describe('wordSpans', () => {+ it('reads the words passageWords reads, with where each sits as written', () => {+ const text = 'Run `astryx docs`, then {@link generic:theme} *now*.';+ const spans = wordSpans(text);+ expect(spans.map(span => span.word)).toEqual(passageWords(text));+ expect(text.slice(spans[1].start, spans[2].end)).toBe('`astryx docs`');+ });+});++describe('passageWords and pointers', () => {+ it('reads a link and a written docs command as no words', () => {+ expect(+ passageWords(+ 'See {@link generic:theme} or `astryx docs theme` (Share it).',+ ),+ ).toEqual(['see', 'or', 'share', 'it']);+ });++ it('counts no punctuation-only token as a word', () => {+ expect(+ passageWords('Set the theme first, see {@link generic:theme}.'),+ ).toEqual(['set', 'the', 'theme', 'first', 'see']);+ expect(+ wordSpans('Read it - then {@link generic:theme}.').map(s => s.word),+ ).toEqual(['read', 'it', 'then']);+ });+});diff --git a/packages/cli/foundation/discovery/docs-discovery.mjs b/packages/cli/foundation/discovery/docs-discovery.mjsindex f163d74248..88d07a8ee2 100644--- a/packages/cli/foundation/discovery/docs-discovery.mjs+++ b/packages/cli/foundation/discovery/docs-discovery.mjs@@ -117,6 +117,23 @@ export function discoverBuiltinTopics() {return topics;}+/**+ * The CLI's own guide files: every flat topic and every doc in its docs tree,+ * without the dense, translated, and draft variants.+ *+ * @returns {string[]}+ */+export function builtinGuideFiles() {+ const files = Object.values(discoverBuiltinTopics());+ const tree = path.join(BUILTIN_DOCS_DIR, 'tree');+ if (fs.existsSync(tree)) {+ for (const file of fs.readdirSync(tree).sort()) {+ if (BUILTIN_TOPIC_FILE_RE.test(file)) files.push(path.join(tree, file));+ }+ }+ return files;+}+/*** Load a topic doc from disk. A `.ts` doc is loaded through jiti, the rest* natively; both the historical `export const docs` and the stamped@@ -384,7 +401,11 @@ export function problemsInNamespace(doc) {}}const slots = doc?.slots;- if (slots == null || typeof slots !== 'object' || Object.keys(slots).length === 0) {+ if (+ slots == null ||+ typeof slots !== 'object' ||+ Object.keys(slots).length === 0+ ) {problems.push('slots: expected at least one slot');return problems;}@@ -392,7 +413,10 @@ export function problemsInNamespace(doc) {if (typeof slot?.title !== 'string' || slot.title === '') {problems.push(`slots.${name}.title: expected a non-empty string`);}- if (!Array.isArray(slot?.accepts?.kinds) || slot.accepts.kinds.length === 0) {+ if (+ !Array.isArray(slot?.accepts?.kinds) ||+ slot.accepts.kinds.length === 0+ ) {problems.push(`slots.${name}.accepts.kinds: expected at least one kind`);}}diff --git a/packages/cli/foundation/doc-compiler/doc-loads.test.mjs b/packages/cli/foundation/doc-compiler/doc-loads.test.mjsindex 560a0ff3bb..8944dd2573 100644--- a/packages/cli/foundation/doc-compiler/doc-loads.test.mjs+++ b/packages/cli/foundation/doc-compiler/doc-loads.test.mjs@@ -252,6 +252,10 @@ const RUNNERS = {runs: '@babel/parser, on first use, to read the members a family doc lists in its components without running the doc',sites: ['require(@babel/parser)'],},+ 'foundation/discovery/copied-passages.mjs': {+ runs: "the CLI's own guides and the package's topics, to compare their prose for passages two topics repeat",+ sites: ['builtinGuideFiles', 'loadTopicModule'],+ },'foundation/discovery/docs-discovery.mjs': {runs: 'a contributed topic, for its catalog fields: name, title, description, category, and relationships',sites: ['importDocModule'],diff --git a/packages/cli/foundation/response/response-types.doc.mjs b/packages/cli/foundation/response/response-types.doc.mjsindex 18ca6472d6..50b274e110 100644--- a/packages/cli/foundation/response/response-types.doc.mjs+++ b/packages/cli/foundation/response/response-types.doc.mjs@@ -348,7 +348,7 @@ export const doc = {{value: 'integration.doc-conflicts',description:- 'validated (false when no integration manifest was found, so nothing was inspected), the integration identity, structural issues, and Core doc overlaps. Each finding includes `severity` (`info` | `error`) and `relationship` (`replaces` | `extends` | `accidental`).',+ 'validated (false when no integration manifest was found, so nothing was inspected), the integration identity, structural issues, and Core doc overlaps. Each finding includes `severity` (`info` | `error`) and `relationship` (`replaces` | `extends` | `accidental`). copiedPassages lists each pair of topics that repeat 10 or more words of prose word for word: topic, section, otherTopic, otherOwner (package | core), words, passages, owner (the topic that should keep the text), text (the passage as the topic writes it), line (where the copy starts in the topic to change, or null), and message (the exact replacement); each is also a copied_passage warning in issues.',},{value: 'integration.check',diff --git a/packages/cli/test/core-doc-quality.test.mjs b/packages/cli/test/core-doc-quality.test.mjsnew file mode 100644index 0000000000..0b0ad0f21a--- /dev/null+++ b/packages/cli/test/core-doc-quality.test.mjs@@ -0,0 +1,86 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++/**+ * Core's own guides pass the doc-quality checks `integration check --strict`+ * holds every integration to (architecture:cli-surface DEC-8): no two guides+ * repeat a passage word for word.+ */++import {describe, expect, it} from 'vitest';++import {+ builtinGuideFiles,+ loadTopicModule,+} from '../foundation/discovery/docs-discovery.mjs';+import {+ BOILERPLATE_TOPICS,+ MIN_CONTENT_WORDS,+ PASSAGE_WORDS,+ contentWords,+ trimBoilerplate,+ proseWords,+ sharedPassages,+} from '../foundation/discovery/copied-passages.mjs';++/** @returns {Promise<{name: string, words: ReturnType<typeof proseWords>}[]>} */+async function coreGuides() {+ const guides = [];+ for (const file of builtinGuideFiles()) {+ const doc = /** @type {any} */ (await loadTopicModule(file));+ if (typeof doc?.name === 'string') {+ guides.push({name: doc.name, words: proseWords(doc)});+ }+ }+ return guides;+}++describe("Core's guides", () => {+ it('repeat no passage word for word', async () => {+ const guides = await coreGuides();+ /** @type {Map<string, Set<number>>} */+ const sharing = new Map();+ guides.forEach(({words}, at) => {+ for (let i = 0; i + PASSAGE_WORDS <= words.length; i++) {+ if (words[i].block !== words[i + PASSAGE_WORDS - 1].block) continue;+ const key = words+ .slice(i, i + PASSAGE_WORDS)+ .map(each => each.word)+ .join(' ');+ const topics = sharing.get(key) ?? new Set();+ topics.add(at);+ sharing.set(key, topics);+ }+ });+ const copies = [];+ for (let a = 0; a < guides.length; a++) {+ for (let b = a + 1; b < guides.length; b++) {+ const {words} = guides[a];+ const boilerplate = (/** @type {number} */ at) =>+ (sharing.get(+ words+ .slice(at, at + PASSAGE_WORDS)+ .map(each => each.word)+ .join(' '),+ )?.size ?? 0) >= BOILERPLATE_TOPICS;+ for (const shared of sharedPassages(words, guides[b].words)) {+ // The same trimming findCopiedPassages applies.+ const run = trimBoilerplate(shared, boilerplate);+ if (!run) continue;+ const {start, length} = run;+ if (contentWords(words, start, length) >= MIN_CONTENT_WORDS) {+ copies.push(+ `${guides[a].name} and ${guides[b].name}: "${words+ .slice(start, start + length)+ .map(each => each.word)+ .join(' ')}"`,+ );+ }+ }+ }+ }+ expect(+ copies,+ 'Keep the fact in the guide that owns it and link to it from the other.',+ ).toEqual([]);+ });+});diff --git a/packages/cli/test/integration-check-failures.json b/packages/cli/test/integration-check-failures.jsonindex a5d9fd521d..13b502a88b 100644--- a/packages/cli/test/integration-check-failures.json+++ b/packages/cli/test/integration-check-failures.json@@ -4,7 +4,7 @@"validate": ["docTypes", "issues"],"components": ["conflicts", "issues", "propDrift"],"templates": ["conflicts", "issues"],- "docs": ["findings", "issues"]+ "docs": ["copiedPassages", "findings", "issues"]},"reports": {"docTypes": "doc_types_skipped",@@ -15,6 +15,7 @@"ambiguous_template_replacement","codemod_outside_version","component_name_conflict",+ "copied_passage","core_not_found","doc_types_skipped","duplicate_doc",