An open source design system that's fully customizable and agent ready
TypeScript
perf(ci): scope pr-rtl to the components a PR touches (#4796)
nynexman4464846ec30abddf
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.ymlindex 1a0a288d8b..f0401d2de4 100644--- a/.github/workflows/ci.yml+++ b/.github/workflows/ci.yml@@ -443,25 +443,21 @@ jobs:path: a11y-report.jsonretention-days: 1- # RTL semantic audit. Sibling to pr-a11y: where pr-a11y runs the a11y audit- # over the built Storybook, pr-rtl runs the RTL audit. It grades components by- # comparing their LTR vs RTL render in the same run — no golden screenshots.- # Two layers:- # (A) auto-discovery — D1 (icon-mirror) over EVERY core-* story, detecting- # directional icons generically so a NEW component that ships a- # directional glyph without RTL handling is caught automatically.- # (B) curated precision — D2/D3/D4 (order/behavior/overlay) from targets.json.+ # RTL semantic audit. Sibling to pr-a11y: same build-storybook artifact, same+ # analysis.json scoping, so a PR only pays for the components it touched.+ # Unscoped this swept ~1400 stories and took ~26 min. The full sweep that+ # covers untouched components runs in rtl-weekly.yml.#- # SOFT / NON-BLOCKING (continue-on-error): a finding surfaces a scorecard in- # the job summary but does NOT block the merge. This is intentional — on- # current main, auto-discovery legitimately flags the components RTL hasn't- # reached yet (labelled [known] via known-not-rtl.json). The suite is observed- # for flake over a stability window before it is promoted to a required check.- # To promote: drop continue-on-error here and add `pr-rtl` to required checks.- # See apps/storybook/rtl-audit/README.md.+ # Two layers: (A) auto-discovery — D1 icon-mirror + D5 positional-mirror, no+ # curated selectors; (B) curated D2/D3/D4 from targets.json.+ #+ # SOFT / NON-BLOCKING (continue-on-error): findings surface a scorecard in the+ # job summary but don't block the merge, while the suite is observed for+ # flake. To promote: drop continue-on-error and add `pr-rtl` to required+ # checks. See apps/storybook/rtl-audit/README.md.pr-rtl:- needs: [build-storybook]- if: github.event_name == 'pull_request'+ needs: [build-storybook, check-components]+ if: github.event_name == 'pull_request' && needs.check-components.outputs.has_components == 'true'runs-on: 2-core-ubuntu-armcontinue-on-error: truepermissions:@@ -475,6 +471,11 @@ jobs:with:install: 'false'+ - name: Download analysis artifact+ uses: actions/download-artifact@v8+ with:+ name: pr-analysis+- name: Download Storybook artifactuses: actions/download-artifact@v8with:@@ -484,19 +485,34 @@ jobs:- name: Install Playwrightrun: pnpm install --frozen-lockfile && npx playwright install chromium+ # An empty --filter means "audit everything", so guard the empty case:+ # has_components can be true while analysis.json resolves to no+ # components (the two compute component-ness differently).- name: Run RTL auditid: rtlrun: |+ COMPONENTS=$(jq -r '(.newComponents + .modifiedComponents) | join(",")' analysis.json)+ if [ -z "$COMPONENTS" ]; then+ echo "No component changes resolved from analysis.json — skipping RTL audit."+ echo "ran=false" >> "$GITHUB_OUTPUT"+ exit 0+ fi+ echo "Auditing components: $COMPONENTS"+ echo "ran=true" >> "$GITHUB_OUTPUT"+ echo "components=$COMPONENTS" >> "$GITHUB_OUTPUT"node apps/storybook/rtl-audit/rtl-audit.mjs \--storybook-dir apps/storybook/dist \- --output rtl-audit-report.json+ --output rtl-audit-report.json \+ --filter "$COMPONENTS"- name: Write scorecard summary- if: always()+ if: always() && steps.rtl.outputs.ran == 'true'run: |{echo "## RTL semantic audit"echo ""+ echo "Scoped to the components this PR touches: **${{ steps.rtl.outputs.components }}**. The full unfiltered sweep runs weekly — see \`rtl-weekly.yml\`."+ echo ""echo "Relationship-based (LTR vs RTL in the same run) — no golden screenshots. **Soft-gated** pending a stability window."echo ""if [ -f rtl-audit-report.json ]; then@@ -568,7 +584,7 @@ jobs:} >> "$GITHUB_STEP_SUMMARY"- name: Upload RTL audit report- if: always()+ if: always() && steps.rtl.outputs.ran == 'true'uses: actions/upload-artifact@v7with:name: rtl-audit-reportdiff --git a/CONTRIBUTING.md b/CONTRIBUTING.mdindex e82ec9d2ba..205feb5699 100644--- a/CONTRIBUTING.md+++ b/CONTRIBUTING.md@@ -453,6 +453,14 @@ When the audit reports baseline entries as "resolved", delete them from> component is accessible — keyboard flows, focus order, screen-reader> semantics, and contrast in context still need manual checks.+### RTL audits++PRs that touch components also run an RTL audit (`pr-rtl`), scoped to the+changed components like `pr-a11y`. It is soft-gated — findings show in the job+summary but don't block. Repro locally with `pnpm rtl:audit -- --filter Avatar`+(the `--` matters: `pnpm -F` is itself `--filter`). See+`apps/storybook/rtl-audit/README.md`.+## Versioning & ReleasesWe use [Changesets](https://github.com/changesets/changesets) for versioning, with a thin Astryx layer on top so changelogs stay categorized, contributor-attributed, and aligned with our pre-1.0 conventions.diff --git a/apps/storybook/rtl-audit/README.md b/apps/storybook/rtl-audit/README.mdindex 9b558a4c76..83a5a1c897 100644--- a/apps/storybook/rtl-audit/README.md+++ b/apps/storybook/rtl-audit/README.md@@ -11,12 +11,16 @@ is the `pr-rtl` job — the RTL sibling of `pr-a11y`.## Two layers-### A. Auto-discovery — over EVERY `core-*` story+### A. Auto-discovery — over every `core-*` story in scopeThe point of the audit is to auto-catch **new or changed** components, so the-auto-discovery layer runs across the whole library with **zero curated-selectors**. There are two independent auto passes: **D1 (icon-mirror)** and-**D5 (positional-mirror)**.+auto-discovery layer runs with **zero curated selectors**. There are two+independent auto passes: **D1 (icon-mirror)** and **D5 (positional-mirror)**.++> **Scope in CI.** `pr-rtl` passes `--filter` with the components the PR+> touched (from `analysis.json`), matching `pr-a11y`. The full unfiltered sweep+> runs weekly in `.github/workflows/rtl-weekly.yml` — that is what covers a+> component no PR edited. Omit `--filter` to run it locally.### A.1 D1 icon-mirror@@ -101,7 +105,18 @@ For each core story it:lists the element class + LTR/RTL `relCenterX` + delta as an actionablefinding.- **N-A** — the story has no logical-anchor + physical-transform candidates- (the common case — not penalised).+ (the common case — not penalised), or candidates were found in one+ direction only, leaving nothing to compare.++**LTR-first short-circuit.** If the LTR render yields zero candidates the story+returns `N-A` immediately and the RTL page is never loaded — with nothing to+measure against, the second navigation can't change the verdict. Most stories+have no candidates (1378 of 1394 in a recent full sweep), so this is about half+of D5's page loads.++It is also a correctness fix: the old `ltr.length === 0 && rtl.length === 0`+guard let one-sided stories fall through to a compare loop that runs+`min(ltr, rtl) = 0` times, reporting **pass** having compared nothing.**Tolerance: 3px.** The observed signal gap was ~10× the tolerance (a realmis-position is tens of px off; a correctly-centered element passes at Δ=0 by@@ -180,16 +195,22 @@ pnpm -F @astryxdesign/storybook buildnpx playwright install chromium# 3. Run the audit against the built Storybook.-pnpm -F @astryxdesign/storybook rtl-audit+pnpm rtl:audit# -> prints the scorecard and writes rtl-audit-report.json-# Scope to one component while iterating (applies to both layers):-node apps/storybook/rtl-audit/rtl-audit.mjs --storybook-dir apps/storybook/dist \- --output /tmp/report.json --filter Pagination+# Scope to one component while iterating. Note the `--`: pnpm -F IS --filter,+# so without it pnpm eats the flag as a workspace filter.+pnpm rtl:audit -- --filter Pagination++# Comma-separated, matched case-insensitively (accepts analysis.json's+# PascalCase names — this is what pr-rtl passes):+pnpm rtl:audit -- --filter Avatar,TableTree++# An absent or EMPTY --filter audits the whole library: ~1400 stories, ~25 min.# Just the broad auto-discovery net, or just the curated dims:-node apps/storybook/rtl-audit/rtl-audit.mjs --storybook-dir apps/storybook/dist --auto-only-node apps/storybook/rtl-audit/rtl-audit.mjs --storybook-dir apps/storybook/dist --curated-only+pnpm rtl:audit -- --auto-only+pnpm rtl:audit -- --curated-only```Exit code is non-zero if auto-discovery finds any not-RTL component or a curateddiff --git a/apps/storybook/rtl-audit/rtl-audit.mjs b/apps/storybook/rtl-audit/rtl-audit.mjsindex 638af2f279..3ae81848e5 100644--- a/apps/storybook/rtl-audit/rtl-audit.mjs+++ b/apps/storybook/rtl-audit/rtl-audit.mjs@@ -417,14 +417,30 @@ async function autoPositionalMirror(page, port, storyId, component) {await settle(page);const revealedL = PM_REVEAL ? await revealInteractionGated(page) : false;const ltr = await detectPositioned(page);++ // Short-circuit before the RTL navigation: with no LTR candidate there is+ // nothing to measure an RTL center against, so the second load can't change+ // the verdict. Most stories have zero candidates, so this is most of D5's+ // cost. Checking `ltr && rtl` here instead would report a vacuous "pass" —+ // the compare loop runs min(ltr, rtl) = 0 times and never sets anyFail.+ if (ltr.length === 0) {+ if (revealedL) card.notes.push('opened an interaction-gated surface before scanning');+ card.notes.push('no logical-anchor + physical-transform candidates in LTR');+ return card;+ }+await page.goto(storyUrl(port, storyId, true), {waitUntil: 'domcontentloaded'});await settle(page);const revealedR = PM_REVEAL ? await revealInteractionGated(page) : false;const rtl = await detectPositioned(page);if (revealedL || revealedR) card.notes.push('opened an interaction-gated surface before scanning');- if (ltr.length === 0 && rtl.length === 0) {- card.notes.push('no logical-anchor + physical-transform candidates');+ // Same reasoning in the other direction: zero comparisons is not a pass.+ if (rtl.length === 0) {+ card.candidates = ltr.length;+ card.notes.push(+ `${ltr.length} candidate(s) in LTR but none in RTL — nothing to compare (not evaluated)`,+ );return card;}card.candidates = Math.max(ltr.length, rtl.length);diff --git a/package.json b/package.jsonindex 8141f09b5d..7a1e799356 100644--- a/package.json+++ b/package.json@@ -21,6 +21,7 @@"lint:strict": "pnpm check:repo && ASTRYX_STRICT_LINT=1 eslint . --cache","a11y:audit": "node .github/scripts/accessibility-audit.js --baseline .github/a11y-baseline.json --fail-on-new","a11y:baseline": "node .github/scripts/accessibility-audit.js --baseline .github/a11y-baseline.json --update-baseline",+ "rtl:audit": "node apps/storybook/rtl-audit/rtl-audit.mjs","storybook": "pnpm -F @astryxdesign/storybook dev","storybook:build": "pnpm -F @astryxdesign/storybook build","docsite": "pnpm -F @astryxdesign/docsite dev",