An open source design system that's fully customizable and agent ready
TypeScript
feat(vibe-tests): use CLI-generated docs instead of static xds.md (#265)
Cindy Zhang5862a4952dce
diff --git a/.claude/commands/vibe-test.md b/.claude/commands/vibe-test.mdindex 380b2551cb..6e7a4116df 100644--- a/.claude/commands/vibe-test.md+++ b/.claude/commands/vibe-test.md@@ -12,6 +12,16 @@ If no argument provided, default to 5 samples.Run interactive vibe tests using parallel subagents:+### Step 0: Generate skill doc++Generate the skill doc from the CLI (auto-discovers components from source):++```bash+cd internal/vibe-tests && bash scripts/generate-skill-doc.sh+```++This creates `.generated/xds-skill.md` reflecting all components on the current branch.+### Step 1: Setup the test iterationRun the interactive setup command:@@ -39,7 +49,7 @@ internal/vibe-tests/results/<iteration>/manifest.jsonFor each test prompt, spawn a subagent using the Task tool with:- `subagent_type`: "general-purpose"-- Read the skill doc at `packages/core/xds.md`+- Read the generated skill doc at `internal/vibe-tests/.generated/xds-skill.md`- Generate code for the prompt using XDS components- Self-evaluate the response for:- Correct components useddiff --git a/internal/vibe-tests/.gitignore b/internal/vibe-tests/.gitignoreindex d97e5c57c8..83c02ab99c 100644--- a/internal/vibe-tests/.gitignore+++ b/internal/vibe-tests/.gitignore@@ -9,5 +9,8 @@ AGENTS.baseline.md# Raw HTML docs (generated)AGENTS.html.md+# Generated skill doc (from CLI)+.generated/+# Test results (generated)results/diff --git a/internal/vibe-tests/README.md b/internal/vibe-tests/README.mdindex 723ce41597..182274ff53 100644--- a/internal/vibe-tests/README.md+++ b/internal/vibe-tests/README.md@@ -2,15 +2,15 @@Vibeability test harness for measuring how well LLMs can use the XDS component system.-## Quick Start - Interactive Mode (Claude Code)+## Quick Start - Interactive Mode (Navi / Claude Code)-Run vibe tests directly in Claude Code without an API key:+Run vibe tests interactively through Navi (the AI assistant) using sub-agents — no API key needed. The skill doc is auto-generated from the CLI before each run, so it always reflects the current branch's components.```/vibe-test 5```-Or ask Claude Code:+Or ask Navi:```"Run a vibe test with 5 samples"@@ -18,10 +18,25 @@ Or ask Claude Code:This will:-1. Set up a test iteration-2. Spawn parallel subagents to run each test-3. Write results to runs.jsonl-4. Aggregate and show results+1. Generate the skill doc from source (auto-discovers components)+2. Set up a test iteration+3. Spawn parallel sub-agents to run each test+4. Write results to runs.jsonl+5. Aggregate and show results++### How It Works (Interactive)++When running interactively through Navi:++1. **Generate skill doc** — Runs `bash scripts/generate-skill-doc.sh` which uses the XDS CLI to auto-discover components from source and generate `.generated/xds-skill.md`+2. **Setup iteration** — Creates task files and manifest in `results/<iteration>/`+3. **Spawn sub-agents** — Each test prompt is run by a parallel sub-agent that:+ - Reads the generated skill doc as context+ - Generates code for the prompt using XDS components+ - Self-evaluates the response+4. **Aggregate** — Results are written to `runs.jsonl` and aggregated into a report++The skill doc is generated fresh each time, so it always reflects the current branch's components. No manual maintenance of component lists needed.## Quick Start - API Modediff --git a/internal/vibe-tests/package.json b/internal/vibe-tests/package.jsonindex 926d265aa6..07db19870d 100644--- a/internal/vibe-tests/package.json+++ b/internal/vibe-tests/package.json@@ -9,10 +9,11 @@},"scripts": {"harness": "tsx src/cli.ts",- "test:xds": "SKILL_DOC=../../packages/core/xds.md tsx src/cli.ts run",- "test:xds:sample": "SKILL_DOC=../../packages/core/xds.md tsx src/cli.ts run --sample 10",- "test:xds:holdout": "SKILL_DOC=../../packages/core/xds.md tsx src/cli.ts run --holdout",- "test:ci": "SKILL_DOC=../../packages/core/xds.md tsx src/cli.ts run --sample 10",+ "generate:skill-doc": "bash scripts/generate-skill-doc.sh",+ "test:xds": "yarn generate:skill-doc && SKILL_DOC=.generated/xds-skill.md tsx src/cli.ts run",+ "test:xds:sample": "yarn generate:skill-doc && SKILL_DOC=.generated/xds-skill.md tsx src/cli.ts run --sample 10",+ "test:xds:holdout": "yarn generate:skill-doc && SKILL_DOC=.generated/xds-skill.md tsx src/cli.ts run --holdout",+ "test:ci": "yarn generate:skill-doc && SKILL_DOC=.generated/xds-skill.md tsx src/cli.ts run --sample 10","interactive": "tsx src/interactive.ts","interactive:sample": "tsx src/interactive.ts --sample 5","run-interactive": "tsx src/run-interactive.ts",diff --git a/internal/vibe-tests/scripts/generate-skill-doc.sh b/internal/vibe-tests/scripts/generate-skill-doc.shnew file mode 100755index 0000000000..1c5c79947c--- /dev/null+++ b/internal/vibe-tests/scripts/generate-skill-doc.sh@@ -0,0 +1,77 @@+#!/usr/bin/env bash+# @file Generate XDS skill doc from CLI for vibe tests+# @position internal/vibe-tests/scripts/generate-skill-doc.sh+#+# Generates a skill doc by combining CLI outputs:+# - agent-docs framing (AGENTS.md index)+# - component --brief-all (component catalog)+# - docs principles (design rules)+# - docs tokens (token reference)+# - docs theme (theme system)+#+# Output: .generated/xds-skill.md++set -euo pipefail++SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"+VIBE_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"+REPO_ROOT="$(cd "$VIBE_DIR/../.." && pwd)"+CLI="$REPO_ROOT/packages/cli/bin/xds.mjs"+OUT_DIR="$VIBE_DIR/.generated"+OUT_FILE="$OUT_DIR/xds-skill.md"++# No CLI rebuild needed — the CLI uses raw .mjs files and discovers+# components by scanning packages/core/src/ directly via readdirSync.++mkdir -p "$OUT_DIR"++echo "Generating XDS skill doc from CLI..." >&2++cat > "$OUT_FILE" <<'HEADER'+# XDS Design System++React design system for building internal tools. All components use the `XDS` prefix.++> This document is auto-generated from source. Do not edit manually.+> Regenerate with: `yarn generate:skill-doc`++## CLI Reference++Run these commands to get detailed docs on any component:+- `npx xds component <Name>` — full docs (props, usage, examples)+- `npx xds component <Name> --compact` — condensed reference+- `npx xds component --list` — all components by category+- `npx xds docs principles` — design rules and anti-patterns+- `npx xds docs tokens` — token reference (spacing, color, radius)+- `npx xds docs theme` — theme system reference++HEADER++# Principles+echo "## Principles" >> "$OUT_FILE"+echo "" >> "$OUT_FILE"+node "$CLI" docs principles 2>/dev/null | tail -n +3 >> "$OUT_FILE"+echo "" >> "$OUT_FILE"++# Component catalog+echo "## Component Catalog" >> "$OUT_FILE"+echo "" >> "$OUT_FILE"+echo "Brief summaries of all available components. Run \`npx xds component <Name>\` for full docs." >> "$OUT_FILE"+echo "" >> "$OUT_FILE"+node "$CLI" component --brief-all 2>/dev/null >> "$OUT_FILE"+echo "" >> "$OUT_FILE"++# Tokens+echo "## Token Reference" >> "$OUT_FILE"+echo "" >> "$OUT_FILE"+node "$CLI" docs tokens 2>/dev/null | tail -n +3 >> "$OUT_FILE"+echo "" >> "$OUT_FILE"++# Theme+echo "## Theme System" >> "$OUT_FILE"+echo "" >> "$OUT_FILE"+node "$CLI" docs theme 2>/dev/null | tail -n +3 >> "$OUT_FILE"++SIZE=$(wc -c < "$OUT_FILE")+LINES=$(wc -l < "$OUT_FILE")+echo "Generated $OUT_FILE ($SIZE bytes, $LINES lines)" >&2diff --git a/packages/core/xds.md b/packages/core/xds.mdindex ae1c8e55ea..4e6bf4e88e 100644--- a/packages/core/xds.md+++ b/packages/core/xds.md@@ -1,103 +1,29 @@-# XDS Component Library+# XDS-> A React design system for building internal tools. Provides form inputs, buttons, layout primitives, and floating UI. All components use the `XDS` prefix (e.g., XDSButton, XDSCard). Use StyleX for custom styling, not inline styles.+XDS is a design system for building internal tools and products.-XDS is a component library for building internal applications with consistent UI patterns. The library emphasizes composition over configuration, controlled form components, and semantic spacing via CSS variables.+## Getting Started-## Key Principles+Use the XDS CLI for component docs, scaffolding, and tooling:-1. Use XDS components for everything they cover-2. Use StyleX (not inline styles) for layout and styling gaps-3. All components use the `XDS` prefix-4. Form inputs are controlled (value + onChange)-5. Use semantic spacing tokens (space0-space7), not pixel values-6. Use CSS variables for colors (not hardcoded hex values) to support theming--## Component Documentation--Each component has a README.md in its source directory with usage examples, props, and implementation notes.--**Pattern**: For any import like `import { XDSButton } from '@xds/core/Button'`, find docs at `src/Button/README.md`.--| Import Path | README Location | Components |-| ------------------------- | ---------------------------------------------------------------- | ----------------------------------------------------------------------------- |-| `@xds/core/Button` | [src/Button/README.md](src/Button/README.md) | XDSButton |-| `@xds/core/TextInput` | [src/TextInput/README.md](src/TextInput/README.md) | XDSTextInput |-| `@xds/core/TextArea` | [src/TextArea/README.md](src/TextArea/README.md) | XDSTextArea |-| `@xds/core/CheckboxInput` | [src/CheckboxInput/README.md](src/CheckboxInput/README.md) | XDSCheckboxInput |-| `@xds/core/Selector` | [src/Selector/README.md](src/Selector/README.md) | XDSSelector |-| `@xds/core/Field` | [src/Field/README.md](src/Field/README.md) | XDSField |-| `@xds/core/Avatar` | [src/Avatar/README.md](src/Avatar/README.md) | XDSAvatar |-| `@xds/core/Skeleton` | [src/Skeleton/README.md](src/Skeleton/README.md) | XDSSkeleton |-| `@xds/core/Text` | [src/Text/README.md](src/Text/README.md) | XDSText |-| `@xds/core/Icon` | [src/Icon/README.md](src/Icon/README.md) | XDSIcon |-| `@xds/core/Layout` | [src/Layout/README.md](src/Layout/README.md) | XDSHStack, XDSVStack, XDSStackItem, XDSCard, XDSSection |-| `@xds/core/Layout` | [src/Layout/XDSLayout/README.md](src/Layout/XDSLayout/README.md) | XDSLayout, XDSLayoutHeader, XDSLayoutContent, XDSLayoutFooter, XDSLayoutPanel |-| `@xds/core/Layer` | [src/Layer/README.md](src/Layer/README.md) | XDSTooltip, XDSHoverCard, XDSMenu |--## Theme--Theme tokens and utilities are documented in [src/theme/README.md](src/theme/README.md) (if available).--```tsx-import {XDSTheme, useXDSTheme, defaultTheme} from '@xds/core';--// Wrap app in theme provider-<XDSTheme theme={defaultTheme}>- <App />-</XDSTheme>;+```bash+npx xds component --list # browse all components by category+npx xds component Button # full docs for a component+npx xds component --brief-all # LLM-optimized summary of all components+npx xds docs # principles and tokens reference```-## Anti-Patterns to Avoid+### For AI-assisted development-❌ **Inline styles** - Use StyleX instead+Run `xds agent-docs` in your project to install the XDS component catalog into your AGENTS.md:-```tsx-// Bad-<div style={{ padding: '16px' }}>--// Good-<div {...stylex.props(styles.container)}>+```bash+npx xds agent-docs```-❌ **Hardcoded colors** - Use CSS variables for theming--```tsx-// Bad-<div style={{ color: '#ff0000' }}>--// Good - use theme tokens via StyleX-const styles = stylex.create({- error: { color: 'var(--color-negative)' }-});-```--## CSS Variables Reference--XDS provides design tokens as CSS variables. Only use these patterns:--| Category | Pattern | Examples |-| ------------- | ---------------------------------------------- | ------------------------------------------------------------------------------- |-| Colors | `--color-*` | `--color-surface`, `--color-text-primary`, `--color-accent`, `--color-negative` |-| Spacing | `--spacing-*` | `--spacing-0`, `--spacing-1`, `--spacing-2`, ..., `--spacing-7` |-| Radius | `--radius-*` | `--radius-container`, `--radius-element`, `--radius-content` |-| Elevation | `--elevation-*` | `--elevation-base`, `--elevation-dialog`, `--elevation-menu` |-| Font families | `--font-body`, `--font-code`, `--font-heading` | (only these three exist) |-| Text sizes | `--text-*` | `--text-sm`, `--text-base`, `--text-lg`, `--text-xl` |-| Line heights | `--leading-*` | `--leading-tight`, `--leading-normal`, `--leading-relaxed` |-| Font weights | `--font-weight-*` | `--font-weight-normal`, `--font-weight-semibold`, `--font-weight-bold` |--❌ **Hardcoded spacing** - Use semantic tokens--```tsx-// Bad-<XDSVStack style={{ gap: '8px' }}>--// Good-<XDSVStack gap="space2">-```+This adds an auto-generated component index that AI coding agents use to discover and correctly use XDS components. Re-run after updating XDS to keep the index current.-## Optional Resources+### Resources-- [Storybook Examples](../../apps/storybook/): Interactive component demos-- [Source Code](./src/): Component implementations+- [Component Storybook](https://facebookexperimental.github.io/xds/)+- [GitHub Repository](https://github.com/facebookexperimental/xds)