An open source design system that's fully customizable and agent ready
TypeScript
docs(spec): separate accessibility gap tracking (#6174)
Cindy Zhang93e81fe14fc2
diff --git a/docs/specs/AST-021/spec.md b/docs/specs/AST-021/spec.mdindex 1336d351b5..f0b167614c 100644--- a/docs/specs/AST-021/spec.md+++ b/docs/specs/AST-021/spec.md@@ -7,7 +7,7 @@ authority: currentarchive_reason: nullsuperseded_by: nullapproved_by: cixzhang-approved_at: 2026-09-03+approved_at: 2026-09-08phase: acceptedowners: [cixzhang]affects_architecture: []@@ -90,11 +90,18 @@ implementation-specific regression coverage.### Known failures and regression gates-- **FR8 — Every known failure is exact and actionable.** A known-failure record MUST- name the expectation id, component binding and state, observed user impact,- standards reference, evidence layer, public tracking issue, and reason the- migration does not fix it. Wildcards, whole-pattern suppression, and unowned text- reasons are prohibited.+- **FR8 — Every known failure is exact and actionable.** A checked-in+ known-failure record MUST name the expectation id, component binding and state,+ observed user impact, standards reference, evidence layer, exact failure, and+ reason the migration does not fix it. Wildcards, whole-pattern suppression, and+ unowned text reasons are prohibited. Operational tracking MUST keep one durable,+ owned gap record for that exact expectation/binding/state tuple. That gap record+ MUST state whether current authority or an objective standard settles the required+ outcome, or whether API, behavior, ownership, compatibility, or design direction+ remains a human decision. Recording a gap does not approve remediation. Unsettled+ direction stops implementation and routes one exact owner question. A private+ tracker identifier or URL MUST NOT be copied into public source, commits, or+ pull-request text; the operational registry maintains that external mapping.- **FR9 — A known failure changes only its exact gate result.** A binding with aknown failure still runs the expectation and reports `known-failure`, never`pass`. Only that exact expected result stops gating the first migration. A@@ -106,7 +113,7 @@ implementation-specific regression coverage.failures remain visibly failing debt but do not prevent the first migration.New components and newly supported states MUST NOT add a required known failure.Adding or widening a known failure after the baseline requires an explicit owner- decision and linked defect; it is not a routine test update.+ decision and a durable owned gap record; it is not a routine test update.- **FR11 — Reports show facts, not one quality score.** Coverage output MUST reportpatterns, bindings, applicable expectations, passes, known failures, unrun layers,and exemptions separately. It MUST NOT collapse those states into one percentage@@ -127,9 +134,10 @@ implementation-specific regression coverage.review the contract before relying on local tests.- **FR14 — The migration pull request is a readable ledger.** Its description MUSTname the affected users and pattern, component parts and states bound, assertions- moved, local tests retained and why, exact known failures and linked issues,- evidence layers run, and mutation or before/after evidence. The description MUST- distinguish test migration from component remediation.+ moved, local tests retained and why, exact known failures and their authority+ classification, evidence layers run, and mutation or before/after evidence. The+ description MUST distinguish test migration from component remediation and MUST+ NOT expose a private operational tracker.- **FR15 — Progress is ordered by user leverage.** The first binding remains theSwitch reference proven by the prototype. Later work prioritizes widely reusedinteractive patterns, components with known keyboard/focus/semantic failures, and@@ -157,17 +165,16 @@ Switch pattern can own only the standards-derived role, name, state, focusabilitand activation outcomes. Form data, callback payloads, tooltip composition, andcomponent styling remain Switch-owned.-[Issue #4112](https://github.com/facebook/astryx/issues/4112) and the closed-[Switch prototype](https://github.com/facebook/astryx/pull/4113) prove that an-existing component can bind to one reusable contract in jsdom and Chromium. The-prototype's `expectedFailures` mechanism is retained only with FR8–FR10's exactness:-a known failure is visible debt, never a pass or a broad skip.+The first implementation proved that an existing component can bind to one reusable+contract in jsdom and Chromium. Its expected-failure mechanism is retained only with+FR8–FR10's exactness: a known failure is visible debt, never a pass or a broad skip.-The accessibility program tracker-[#4475](https://github.com/facebook/astryx/issues/4475) remains the public place to-order pattern migrations and component defects. Migration pull requests add newly-found defects there or to focused public issues. They do not depend on a private-backlog or private review link.+Operational gap tracking is separate from the public contract source. The public+known-failure record remains self-contained and reviewable, while the project's+configured gap registry maps the same expectation, binding, and state to its owner,+triage, and later remediation. Migration work MUST NOT create a public issue solely+to carry known-failure debt, and private tracker references MUST NOT enter this+public repository.### Progressive rollout@@ -190,15 +197,15 @@ gate. Those changes follow in implementation work governed by this record.## Verification-| Contract | Verification | Representative states | Mutation or failure expectation |-| --------- | -------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------- |-| FR1–FR4 | Migration inventory schema and review | one control; component with trigger and popup; controlled/disabled/error/RTL states | Migration starts without an applicable current contract, treats a whole component as one part, or omits a state that changes semantics |-| FR5–FR7 | Before/after assertion ownership review and unchanged component suites | shared role/state check; callback payload; form data; style rule; discovered component defect | Shared assertion remains duplicated, component-specific proof is deleted, or migration silently changes behavior |-| FR8–FR10 | Runner tests for exact known failures, new failures, and unexpected passes | one historical gap; wrong error; second state; fixed expectation | A wildcard masks regression, a stale gap remains, or new work adds debt without decision |-| FR11 | Coverage-report snapshot and assertions | pass; known failure; exemption; unrun browser layer | One percentage hides a required failure or reports an unrun layer as pass |-| FR12–FR14 | Knowledge validation, component verification map, and PR-template check | current component record; component with no record; first new adopter | Backlink points to deleted local tests, parallel ad hoc assertions appear, or the PR hides migration/remediation scope |-| FR15 | Public tracker review | high-use control; known keyboard defect; easy low-impact component | Work is ordered only to maximize migrated counts |-| Platform | Existing unit/axe/browser checks plus AST-009 classification | jsdom behavior; real focus; tree exposure; spoken announcement | A lower layer clears browser pixels or real-AT output it cannot observe |+| Contract | Verification | Representative states | Mutation or failure expectation |+| --------- | -------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------- |+| FR1–FR4 | Migration inventory schema and review | one control; component with trigger and popup; controlled/disabled/error/RTL states | Migration starts without an applicable current contract, treats a whole component as one part, or omits a state that changes semantics |+| FR5–FR7 | Before/after assertion ownership review and unchanged component suites | shared role/state check; callback payload; form data; style rule; discovered component defect | Shared assertion remains duplicated, component-specific proof is deleted, or migration silently changes behavior |+| FR8–FR10 | Runner tests for exact known failures, new failures, and unexpected passes | one historical gap; wrong error; second state; fixed expectation | A wildcard masks regression, a stale gap remains, or new work adds debt without decision |+| FR11 | Coverage-report snapshot and assertions | pass; known failure; exemption; unrun browser layer | One percentage hides a required failure or reports an unrun layer as pass |+| FR12–FR14 | Knowledge validation, component verification map, and PR-template check | current component record; component with no record; first new adopter | Backlink points to deleted local tests, parallel ad hoc assertions appear, or the PR hides migration/remediation scope |+| FR15 | Durable gap-registry review | high-use control; known keyboard defect; easy low-impact component | Work is ordered only to maximize migrated counts, duplicate gap records are created, or unresolved direction is treated as an automatic fix |+| Platform | Existing unit/axe/browser checks plus AST-009 classification | jsdom behavior; real focus; tree exposure; spoken announcement | A lower layer clears browser pixels or real-AT output it cannot observe |### Completion criteria@@ -209,7 +216,8 @@ A pattern migration is complete only when:- every affected component part and relevant state has a binding or an explicitownership reason for exclusion;- equivalent shared assertions have one owner and component-specific tests remain;-- every historical failure has the exact public debt record required by FR8;+- every historical failure has the exact public-safe known-failure record and the+ separate durable owned gap record required by FR8;- required passing expectations gate regressions and unexpected passes remove staledebt;- reporting separates passes, known failures, exemptions, and unrun layers;@@ -229,7 +237,7 @@ pattern applies, and no duplicate ad hoc test remains for a shared expectation.**Decider:** `cixzhang`, `2026-09-03`Apply contracts one pattern and bounded binding set at a time. Preserve historical-failures as exact, runnable, public debt while immediately gating outcomes that+failures as exact, runnable known-failure debt while immediately gating outcomes thatalready pass. This adds protection now without pretending existing defects areconformant or waiting for a repository-wide remediation.@@ -244,9 +252,10 @@ suite that cannot stop regressions.Keep test migration separate from component remediation. Move only reusablestandards-derived outcomes into pattern contracts; retain component-specific API,composition, form, styling, and callback tests with the component. Existing defects-stay as exact runnable `known-failure` results tied to one expectation, binding,-state, and public issue. They never count as passing, cannot mask a different or-wider failure, and become unexpected passes when fixed so stale debt is removed.+stay as exact runnable `known-failure` results tied to one expectation, binding, and+state, with a separate durable owned gap record. They never count as passing, cannot+mask a different or wider failure, and become unexpected passes when fixed so stale+debt is removed.Report passes, known failures, exemptions, and unrun layers as separate facts ratherthan averaging them into one accessibility score.@@ -255,6 +264,21 @@ Rejected: changing component behavior inside migration work, deletingcomponent-owned tests, broad skips, and a percentage that can make a requiredfailure look conformant.+### DEC-3 — Gap tracking does not turn every mismatch into an approved fix++**Reference:** `spec:AST-021/DEC-3`+**Decider:** `cixzhang`, `2026-09-08`++Keep the checked-in known-failure record public-safe and self-contained. Track its+operational ownership separately in the project's durable gap registry. The gap+records the current authority result: an objective requirement may settle the+outcome, while a new API, behavior, ownership, compatibility, or design choice+remains a human decision. Recording the mismatch does not authorize remediation.++Rejected: creating public issues only to carry test debt, exposing private tracker+identifiers in the public repository, and treating every accessibility finding as an+obvious bug fix.+## Open questionsNone.