Skip to content

An open source design system that's fully customizable and agent ready

TypeScript

perf(ci): scope pr-rtl to the components a PR touches (#4796)

nynexman4464846ec30abddf
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 1a0a288d8b..f0401d2de4 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -443,25 +443,21 @@ jobs:
path: a11y-report.json
retention-days: 1
- # RTL semantic audit. Sibling to pr-a11y: where pr-a11y runs the a11y audit
- # over the built Storybook, pr-rtl runs the RTL audit. It grades components by
- # comparing their LTR vs RTL render in the same run — no golden screenshots.
- # Two layers:
- # (A) auto-discovery — D1 (icon-mirror) over EVERY core-* story, detecting
- # directional icons generically so a NEW component that ships a
- # directional glyph without RTL handling is caught automatically.
- # (B) curated precision — D2/D3/D4 (order/behavior/overlay) from targets.json.
+ # RTL semantic audit. Sibling to pr-a11y: same build-storybook artifact, same
+ # analysis.json scoping, so a PR only pays for the components it touched.
+ # Unscoped this swept ~1400 stories and took ~26 min. The full sweep that
+ # covers untouched components runs in rtl-weekly.yml.
#
- # SOFT / NON-BLOCKING (continue-on-error): a finding surfaces a scorecard in
- # the job summary but does NOT block the merge. This is intentional — on
- # current main, auto-discovery legitimately flags the components RTL hasn't
- # reached yet (labelled [known] via known-not-rtl.json). The suite is observed
- # for flake over a stability window before it is promoted to a required check.
- # To promote: drop continue-on-error here and add `pr-rtl` to required checks.
- # See apps/storybook/rtl-audit/README.md.
+ # Two layers: (A) auto-discovery — D1 icon-mirror + D5 positional-mirror, no
+ # curated selectors; (B) curated D2/D3/D4 from targets.json.
+ #
+ # SOFT / NON-BLOCKING (continue-on-error): findings surface a scorecard in the
+ # job summary but don't block the merge, while the suite is observed for
+ # flake. To promote: drop continue-on-error and add `pr-rtl` to required
+ # checks. See apps/storybook/rtl-audit/README.md.
pr-rtl:
- needs: [build-storybook]
- if: github.event_name == 'pull_request'
+ needs: [build-storybook, check-components]
+ if: github.event_name == 'pull_request' && needs.check-components.outputs.has_components == 'true'
runs-on: 2-core-ubuntu-arm
continue-on-error: true
permissions:
@@ -475,6 +471,11 @@ jobs:
with:
install: 'false'
+ - name: Download analysis artifact
+ uses: actions/download-artifact@v8
+ with:
+ name: pr-analysis
+
- name: Download Storybook artifact
uses: actions/download-artifact@v8
with:
@@ -484,19 +485,34 @@ jobs:
- name: Install Playwright
run: pnpm install --frozen-lockfile && npx playwright install chromium
+ # An empty --filter means "audit everything", so guard the empty case:
+ # has_components can be true while analysis.json resolves to no
+ # components (the two compute component-ness differently).
- name: Run RTL audit
id: rtl
run: |
+ COMPONENTS=$(jq -r '(.newComponents + .modifiedComponents) | join(",")' analysis.json)
+ if [ -z "$COMPONENTS" ]; then
+ echo "No component changes resolved from analysis.json — skipping RTL audit."
+ echo "ran=false" >> "$GITHUB_OUTPUT"
+ exit 0
+ fi
+ echo "Auditing components: $COMPONENTS"
+ echo "ran=true" >> "$GITHUB_OUTPUT"
+ echo "components=$COMPONENTS" >> "$GITHUB_OUTPUT"
node apps/storybook/rtl-audit/rtl-audit.mjs \
--storybook-dir apps/storybook/dist \
- --output rtl-audit-report.json
+ --output rtl-audit-report.json \
+ --filter "$COMPONENTS"
- name: Write scorecard summary
- if: always()
+ if: always() && steps.rtl.outputs.ran == 'true'
run: |
{
echo "## RTL semantic audit"
echo ""
+ echo "Scoped to the components this PR touches: **${{ steps.rtl.outputs.components }}**. The full unfiltered sweep runs weekly — see \`rtl-weekly.yml\`."
+ echo ""
echo "Relationship-based (LTR vs RTL in the same run) — no golden screenshots. **Soft-gated** pending a stability window."
echo ""
if [ -f rtl-audit-report.json ]; then
@@ -568,7 +584,7 @@ jobs:
} >> "$GITHUB_STEP_SUMMARY"
- name: Upload RTL audit report
- if: always()
+ if: always() && steps.rtl.outputs.ran == 'true'
uses: actions/upload-artifact@v7
with:
name: rtl-audit-report
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index e82ec9d2ba..205feb5699 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -453,6 +453,14 @@ When the audit reports baseline entries as "resolved", delete them from
> component is accessible — keyboard flows, focus order, screen-reader
> semantics, and contrast in context still need manual checks.
+### RTL audits
+
+PRs that touch components also run an RTL audit (`pr-rtl`), scoped to the
+changed components like `pr-a11y`. It is soft-gated — findings show in the job
+summary but don't block. Repro locally with `pnpm rtl:audit -- --filter Avatar`
+(the `--` matters: `pnpm -F` is itself `--filter`). See
+`apps/storybook/rtl-audit/README.md`.
+
## Versioning & Releases
We use [Changesets](https://github.com/changesets/changesets) for versioning, with a thin Astryx layer on top so changelogs stay categorized, contributor-attributed, and aligned with our pre-1.0 conventions.
diff --git a/apps/storybook/rtl-audit/README.md b/apps/storybook/rtl-audit/README.md
index 9b558a4c76..83a5a1c897 100644
--- a/apps/storybook/rtl-audit/README.md
+++ b/apps/storybook/rtl-audit/README.md
@@ -11,12 +11,16 @@ is the `pr-rtl` job — the RTL sibling of `pr-a11y`.
## Two layers
-### A. Auto-discovery — over EVERY `core-*` story
+### A. Auto-discovery — over every `core-*` story in scope
The point of the audit is to auto-catch **new or changed** components, so the
-auto-discovery layer runs across the whole library with **zero curated
-selectors**. There are two independent auto passes: **D1 (icon-mirror)** and
-**D5 (positional-mirror)**.
+auto-discovery layer runs with **zero curated selectors**. There are two
+independent auto passes: **D1 (icon-mirror)** and **D5 (positional-mirror)**.
+
+> **Scope in CI.** `pr-rtl` passes `--filter` with the components the PR
+> touched (from `analysis.json`), matching `pr-a11y`. The full unfiltered sweep
+> runs weekly in `.github/workflows/rtl-weekly.yml` — that is what covers a
+> component no PR edited. Omit `--filter` to run it locally.
### A.1 D1 icon-mirror
@@ -101,7 +105,18 @@ For each core story it:
lists the element class + LTR/RTL `relCenterX` + delta as an actionable
finding.
- **N-A** — the story has no logical-anchor + physical-transform candidates
- (the common case — not penalised).
+ (the common case — not penalised), or candidates were found in one
+ direction only, leaving nothing to compare.
+
+**LTR-first short-circuit.** If the LTR render yields zero candidates the story
+returns `N-A` immediately and the RTL page is never loaded — with nothing to
+measure against, the second navigation can't change the verdict. Most stories
+have no candidates (1378 of 1394 in a recent full sweep), so this is about half
+of D5's page loads.
+
+It is also a correctness fix: the old `ltr.length === 0 && rtl.length === 0`
+guard let one-sided stories fall through to a compare loop that runs
+`min(ltr, rtl) = 0` times, reporting **pass** having compared nothing.
**Tolerance: 3px.** The observed signal gap was ~10× the tolerance (a real
mis-position is tens of px off; a correctly-centered element passes at Δ=0 by
@@ -180,16 +195,22 @@ pnpm -F @astryxdesign/storybook build
npx playwright install chromium
# 3. Run the audit against the built Storybook.
-pnpm -F @astryxdesign/storybook rtl-audit
+pnpm rtl:audit
# -> prints the scorecard and writes rtl-audit-report.json
-# Scope to one component while iterating (applies to both layers):
-node apps/storybook/rtl-audit/rtl-audit.mjs --storybook-dir apps/storybook/dist \
- --output /tmp/report.json --filter Pagination
+# Scope to one component while iterating. Note the `--`: pnpm -F IS --filter,
+# so without it pnpm eats the flag as a workspace filter.
+pnpm rtl:audit -- --filter Pagination
+
+# Comma-separated, matched case-insensitively (accepts analysis.json's
+# PascalCase names — this is what pr-rtl passes):
+pnpm rtl:audit -- --filter Avatar,TableTree
+
+# An absent or EMPTY --filter audits the whole library: ~1400 stories, ~25 min.
# Just the broad auto-discovery net, or just the curated dims:
-node apps/storybook/rtl-audit/rtl-audit.mjs --storybook-dir apps/storybook/dist --auto-only
-node apps/storybook/rtl-audit/rtl-audit.mjs --storybook-dir apps/storybook/dist --curated-only
+pnpm rtl:audit -- --auto-only
+pnpm rtl:audit -- --curated-only
```
Exit code is non-zero if auto-discovery finds any not-RTL component or a curated
diff --git a/apps/storybook/rtl-audit/rtl-audit.mjs b/apps/storybook/rtl-audit/rtl-audit.mjs
index 638af2f279..3ae81848e5 100644
--- a/apps/storybook/rtl-audit/rtl-audit.mjs
+++ b/apps/storybook/rtl-audit/rtl-audit.mjs
@@ -417,14 +417,30 @@ async function autoPositionalMirror(page, port, storyId, component) {
await settle(page);
const revealedL = PM_REVEAL ? await revealInteractionGated(page) : false;
const ltr = await detectPositioned(page);
+
+ // Short-circuit before the RTL navigation: with no LTR candidate there is
+ // nothing to measure an RTL center against, so the second load can't change
+ // the verdict. Most stories have zero candidates, so this is most of D5's
+ // cost. Checking `ltr && rtl` here instead would report a vacuous "pass" —
+ // the compare loop runs min(ltr, rtl) = 0 times and never sets anyFail.
+ if (ltr.length === 0) {
+ if (revealedL) card.notes.push('opened an interaction-gated surface before scanning');
+ card.notes.push('no logical-anchor + physical-transform candidates in LTR');
+ return card;
+ }
+
await page.goto(storyUrl(port, storyId, true), {waitUntil: 'domcontentloaded'});
await settle(page);
const revealedR = PM_REVEAL ? await revealInteractionGated(page) : false;
const rtl = await detectPositioned(page);
if (revealedL || revealedR) card.notes.push('opened an interaction-gated surface before scanning');
- if (ltr.length === 0 && rtl.length === 0) {
- card.notes.push('no logical-anchor + physical-transform candidates');
+ // Same reasoning in the other direction: zero comparisons is not a pass.
+ if (rtl.length === 0) {
+ card.candidates = ltr.length;
+ card.notes.push(
+ `${ltr.length} candidate(s) in LTR but none in RTL — nothing to compare (not evaluated)`,
+ );
return card;
}
card.candidates = Math.max(ltr.length, rtl.length);
diff --git a/package.json b/package.json
index 8141f09b5d..7a1e799356 100644
--- a/package.json
+++ b/package.json
@@ -21,6 +21,7 @@
"lint:strict": "pnpm check:repo && ASTRYX_STRICT_LINT=1 eslint . --cache",
"a11y:audit": "node .github/scripts/accessibility-audit.js --baseline .github/a11y-baseline.json --fail-on-new",
"a11y:baseline": "node .github/scripts/accessibility-audit.js --baseline .github/a11y-baseline.json --update-baseline",
+ "rtl:audit": "node apps/storybook/rtl-audit/rtl-audit.mjs",
"storybook": "pnpm -F @astryxdesign/storybook dev",
"storybook:build": "pnpm -F @astryxdesign/storybook build",
"docsite": "pnpm -F @astryxdesign/docsite dev",