An open source design system that's fully customizable and agent ready
schema_version: 1 template_version: 1 kind: architecture id: architecture:component-test-sufficiency authority: current archive_reason: null superseded_by: null approved_by: cixzhang approved_at: 2026-09-08 owners: [cixzhang] applies_to: [ packages/core/src/, packages/lab/src/, packages/charts/src/, packages/richtext/src/, packages/vega/src/, internal/interaction-harness/, ] verified_by: [component verification maps and their linked test evidence]
deciding_specs: []
Component test sufficiency architecture
{
"scope": "global",
"triggers": {
"testing": [
"INV1",
"INV2",
"INV3",
"INV4",
"INV5",
"INV6",
"INV7",
"INV10",
"INV11"
]
}
}
Purpose
Astryx component tests should give maintainers confidence that current promised behavior will fail clearly when it regresses, without turning every possible input combination or audit inventory row into a permanent test.
This record defines how current component, module, family, architecture, system, design, and theme requirements plus applicable objective standards become a rational observable test plan. It owns test sufficiency and test quality. It does not create product behavior.
System model
Test sufficiency starts from applicable authority, not from a universal list of states every component must implement:
- Assemble the component's current public behavior from every applicable current component, module, family, architecture, system, design, and theme record, plus applicable objective standards.
- Identify the promises whose failure would harm a user, consumer, accessibility path, compatibility boundary, or regression-prone integration seam.
- Partition inputs, states, and boundaries by observable behavior and risk.
- Check whether existing evidence already fails for the distinct promise and partition. If it does, cite that evidence and add no duplicate test.
- Select the evidence layer that can observe the failure: compile-time, unit/DOM, browser behavior, rendered pixels, accessibility tree, real assistive technology, documentation/schema validation, or review.
- Add a new automated test only when no current mutation-sensitive evidence covers the promise at that layer.
- Record the relationship in the owning verification map.
The number of tests follows the number of distinct behaviors and risks. It is not a fixed target and does not equal the number of public props, source branches, audit rows, or inventory entries.
Boundaries and invariants
- INV1 — Authority and evidence are distinct. A test may prove behavior
required by any applicable current component, module, family, architecture,
system, design, or theme authority, or by an applicable objective standard. Where
spec:AST-029permits observational audit, a test may also prove behavior exposed through an established public interface recorded in the audit receipt. That evidence may support a test-sufficiency verdict but MUST NOT create or settle product authority. Missing or draft-only authority remains a separately reported authority gap. - INV2 — Critical promises have regression evidence. Tests cover every applicable critical promise and regression-prone seam named by the owning verification map. Critical promises include behavior whose failure breaks the primary task, accessibility, a public callback or state contract, compatibility, data integrity, cleanup/lifetime, or a documented composition boundary whose failure breaks a current consumer outcome or a named regression-prone integration seam.
- INV3 — Previously fixed behavior defects remain protected. A fix to a defect in a current observable behavior includes mutation-sensitive regression evidence at a capable layer that reproduces the broken outcome before the fix and passes after it. A documentation correction that leaves source, types, behavior, and public machine-readable schema unchanged is not a behavior-defect fix. Existing regression evidence remains unless authority removes the promise or equivalent evidence demonstrably subsumes the case.
- INV4 — Partitions follow behavior and risk. Inputs, states, platforms, and boundaries MAY share one representative case only when current evidence shows they exercise the same observable path and have no distinct regression risk. Different behavior, ownership, platform semantics, failure modes, or known regression history requires a separate partition.
- INV5 — Assertions observe the contract. Tests assert public behavior,
owner-observable state, rendered user outcomes, accessibility semantics,
compile-time consumer contracts, or external-system effects through a stable
seam. A class, attribute, DOM shape, generated type, or source artifact is valid
evidence when current authority explicitly makes it contractual or an
AST-029receipt records it as the established public interface for this audit. Tests MUST NOT rely solely on private call order, private state shape, internal collaborators, or incidental structure when the promised outcome can be observed more directly. - INV6 — Evidence is mutation-sensitive. A test counted as sufficient MUST fail when its promised behavior is removed, inverted, routed to the wrong owner, or otherwise materially broken. An assertion that recomputes the implementation, repeats a type/source declaration, checks a constant against itself, or snapshots unrelated output does not prove the promise. A motion judgement holds and asserts the judged motion itself, named by its target element and animated property; a held count alone does not prove it, since other animations can fill the count.
- INV7 — Coverage remains bounded and intentional. Test volume has a named behavioral, evidence-layer, or risk reason. Multiple evidence layers are not duplicates when each observes a distinct required contract boundary, such as compile-time API, DOM semantics, browser behavior, or assistive-technology output. Repeated cases within the same partition, broad defensive matrices, exhaustive combinations, and snapshot volume without additional behavioral signal MUST NOT be required or credited as sufficiency.
- INV8 — Verification maps own durable evidence relationships. Verification
maps in applicable current component, module, family, architecture, system,
design, and theme records name the critical contract, representative partitions,
evidence seam, and mutation or failure signal. When
no current map exists, the
spec:AST-029audit receipt records the provisional evidence relationship; the missing map is an authority or traceability gap, not by itself a test-coverage gap. One test MAY prove several requirements, and one requirement MAY need several tests. There is no one-row-to-one-test rule. - INV9 — Audit remediation follows its process owner. Audit-authored behavior-
test remediation follows
spec:AST-029/DEC-4: establish the current or established public seam, demonstrate the new case red before production changes, and implement one minimal observable vertical slice. Store that chronology in the pull-request or audit-run evidence. Red/green history is change evidence; it is not part of standing suite sufficiency after the change lands. - INV10 — Gaps and hardening are distinct. Missing coverage is a gap only when an applicable critical promise or known regression-prone partition lacks mutation-sensitive evidence. Additional equivalent inputs, private implementation checks, speculative platform combinations, and defensive edge cases without a named risk are optional hardening and do not lower the sufficiency verdict.
- INV11 — New tests require a new proof obligation. Classify the changed claim, not the file edited. A change MUST NOT add a component functional test merely because it edits documentation, private styling source, a private or generated class name, a private wrapper, or another implementation detail. A current public class or target contract is not covered by that exception and requires focused contract evidence when changed. New automated evidence is required only when the change adds or repairs a distinct current promise or risk partition that existing evidence cannot detect at the correct layer. Otherwise the change cites existing evidence or uses the applicable non-test proof, such as typed documentation validation, source review, or rendered browser evidence.
Rational partitioning
A partition is defined by a difference that can change the promised outcome or its risk. Representative dimensions include:
- semantic state or transition;
- controlled, uncontrolled, optimistic, rejected, or interrupted ownership where the current contract distinguishes them;
- enabled, disabled, read-only, busy, required, or invalid behavior where applicable;
- input modality, direction, browser capability, or platform when the owner or an objective standard predicts different behavior;
- zero, one, many, empty, overflow, boundary, or invalid data when the algorithm or user outcome changes;
- default, composed, replaceable-part, nested-owner, portal/top-layer, or teardown paths when ownership or lifetime changes; and
- exact previously broken inputs and seams.
Inputs are equivalent only when evidence supports the same observable contract, outcome, owner, and regression risk. A shared implementation path may support that judgment, but it is neither required nor sufficient. Different literal values do not require separate cases when none of those dimensions changes.
When no new test is warranted
A correct change may require no new automated test. Reviewers stop adding tests when existing evidence already detects the affected promise, or when an automated component test cannot observe the changed claim.
| Change | Required evidence | Do not add |
|---|---|---|
| Consumer-doc prose or example is corrected to match unchanged current source, types, and behavior | Existing doc parser/typecheck/schema validation when applicable, plus review against the canonical source | A component runtime test that repeats the prose, prop name, or example |
| A recurring deterministic docs/source relationship is missing a guard | One generalized validator at the shared source-of-truth boundary | One component-specific test per corrected document |
| Visual paint or geometry values change while the set of states, selectors, state ownership, timing, and public structure remain unchanged | Real-browser before/after evidence and the applicable stable visual-regression path | A jsdom assertion that a private class exists, or a functional test that cannot observe pixels |
| A private class, wrapper, StyleX declaration, or DOM arrangement changes without changing a current structural contract | Existing behavior evidence plus rendered evidence when appearance changes | An assertion that freezes the new private implementation |
| Existing mutation-sensitive evidence already covers the same promise and partition | Link the existing evidence and show why its failure signal covers this change | A second test with equivalent setup and assertions |
| A refactor preserves all observable contracts | Run the applicable existing suites and review the unchanged public seams | A test whose only purpose is to restate the refactor's new structure |
| A machine-readable public schema, documented public target/class, or compile-time consumer contract changes | Focused contract/inventory/type evidence owned by that public surface | A broader functional test when the structural contract check directly detects the failure |
A visual implementation and its private class can be wrong together. For example, making a Button red and asserting that the new private class name is present proves neither the rendered color nor the user's visual outcome. The class assertion is valid only when current authority makes that exact class a public compatibility contract; even then, it proves class reachability, not pixels.
Documentation can itself carry a public machine-readable contract. In that case, test its parsed schema or generated projection at the shared boundary. Do not freeze ordinary prose word-for-word or duplicate source declarations in component tests.
A documentation correction that exposes an untested behavior defect follows the behavior rule instead: first identify the current promise and uncovered partition, then add the smallest observable regression test for that behavior—not for the wording that revealed it.
Test-quality exclusions
The following do not establish sufficiency by themselves:
- a type test that merely copies a declaration without exercising the consumer's compile-time contract;
- a tautological expected value calculated by the same algorithm as production;
- a test added beside a new private class or StyleX declaration that only asserts that exact class or declaration is present;
- a DOM/class assertion used as a substitute for a visual claim such as color, spacing, alignment, clipping, or motion;
- a test that freezes ordinary documentation prose or repeats an example already validated by the shared documentation/type pipeline;
- assertions against private function calls, hook internals, or implementation ordering when public behavior is observable;
- snapshots whose changed bytes do not identify the promised behavior;
- repeated cases with no distinct behavior or risk;
- Cartesian products or adversarial inputs generated without a current promise, objective standard, prior defect, or named failure hypothesis;
- tests that only prove rendering did not throw, unless crash-free rendering at that named compatibility boundary is the promised observable outcome; and
- tests added after a fix without evidence that the new case could detect the original defect.
These tests may support debugging or optional hardening. They are not credited as the only evidence for a critical promise.
Missing authority versus missing coverage
Auditors classify the boundary before recommending tests:
- Current behavior authority, sufficient evidence: no coverage gap.
- Current behavior authority, missing or insensitive evidence: test-coverage gap.
- No current behavior authority: product-authority gap; route to the canonical owner before prescribing expected output.
- Draft authority only: private owner hold; a test may preserve current facts but cannot make the draft decision current.
- Objective standard applies independently: test the required observable standard outcome and link the standard owner.
- Existing evidence covers an equivalent partition: optional hardening, not a gap.
Verification-map contract
A verification row identifies:
- the current requirement or group of inseparable requirements;
- the stable observable seam and evidence type;
- representative partitions and why they are sufficient; and
- the mutation or failure that must make the evidence fail.
Verification maps do not enumerate every audit inventory row, source branch, prop, or test file. They point to evidence at the level needed to keep the durable promise reviewable.
Future audit-rubric migration handoff
This architecture does not own rubric weights, score computation, versioning implementation, or wiki editing. It defines the test-sufficiency authority that a separate rubric activation MUST consume. When this record becomes current, that separately reviewed activation replaces Component Audit Rubric section 4's generic product-behavior checklist as follows:
- retire B1–B16 as product-behavior rule IDs;
- do not reuse those IDs and do not renumber unrelated stable audit IDs;
- preserve the weighted Behavior scoring slot;
- populate that slot from every applicable current component, module, family, architecture, system, design, and theme behavior requirement plus applicable objective standards;
- score the Behavior slot from observed conformance to the assembled applicable behavior requirements and objective standards;
- score the Testing slot from INV1–INV8 and INV10–INV11 evidence sufficiency and quality, and retain INV9 as pull-request or audit-run evidence;
- exclude non-applicable requirements and keep the rubric's existing explicit unmeasured treatment when required evidence cannot be obtained;
- preserve one-defect/one-section ownership so one missing behavior or test is not deducted twice; and
- bump the rubric minor version because replacing B1–B16 changes scoring method and comparability.
The future activation maps the former rows as follows:
| Former rows | Future source |
|---|---|
| B1–B11 | Every applicable current component/module/family/architecture/system/design/theme behavior requirement and objective standard; no generic replacement behaviors |
| B12 | Already retired in rubric v1.16; keep the ID reserved |
| B13 | Named risk partitions and documented composition seams from applicable current authority; no open-ended adversarial matrix |
| B14 | Applicable current component/module/family/architecture/system/design/theme contracts and objective standards for the owning behavior layer |
| B15 | Current critical promises and regression-prone seams under INV2–INV4; no generic latent-regression catch-all |
| B16 | Retire from Behavior; preserve the PR-only red/green obligation under spec:AST-029/DEC-4 and the Testing/review process |
This specification PR does not edit the wiki rubric, scoring code, audit prompt, tests, or tooling. The replacement activates only after this record is current and the separately reviewed rubric minor-version change lands.
Change coupling
Focused test-sufficiency review triggers when a change:
- adds or removes a current behavioral promise;
- changes a verification map or its linked evidence;
- fixes a defect or closes an audit coverage gap;
- claims several inputs/states are equivalent;
- adds a broad matrix, snapshot family, implementation-coupled test layer, or a new test for a docs/visual/private-implementation-only change; or
- changes which objective standard or platform distinction applies.
Review evaluates the changed requirement and representative unchanged partitions; it does not require unrelated full-suite expansion.
Owning code
- Applicable current component, module, family, architecture, system, design, and theme verification maps — promises, representative partitions, evidence, and failure expectations.
- Colocated tests and checked browser/accessibility bindings — executable evidence linked by those maps.
- This record — shared sufficiency, partitioning, assertion-quality, and gap- classification rules.
Deciding specs
None. spec:AST-029 continues to own audit backfill, evidence receipts, mode, and
Night Watch operation; it explicitly does not own the general component-test
quality rubric. Applicable current component, module, family, architecture, system,
design, and theme contracts plus objective standards remain the sources of expected
behavior.
Verification
| Invariant | Evidence | Mutation or failure expectation |
|---|---|---|
| INV1, INV10–INV11 | Authority-vs-coverage, established-public-interface, and no-new-test decision fixtures | Missing product authority is hidden, established public-interface evidence is denied a sufficiency result, optional hardening blocks acceptance, a docs/value-only-visual/private-only change receives a duplicate or non-observing component test, or a current public class/target change is excused as a private implementation detail |
| INV2–INV3 | Critical-promise, behavior-defect, documentation-correction, and historical-regression fixtures | A primary, accessibility, callback, compatibility, composition, or previously fixed behavior defect loses its only mutation-sensitive evidence and still passes, or an unchanged prose-only documentation correction is required to add a regression test |
| INV4 | Rational-partition fixtures | Proven-equivalent values require duplicate tests, private implementation identity is required to establish equivalence, or behaviorally different states collapse into one case |
| INV5 | Public/runtime/compile-time seam replacement fixtures | Incidental private assertions pass while the promised user or consumer outcome is broken, contractual DOM/type/class/target evidence is incorrectly rejected, a DOM/class assertion is credited for pixels it cannot observe, or selector/state behavior is misclassified as a value-only visual change |
| INV6 | Tautology, declaration-copy, snapshot-noise, and behavior mutations; a held set missing the named motion | An insensitive assertion receives credit, a meaningful behavior mutation leaves the credited test green, or a motion judgement passes on unrelated animations |
| INV7 | Same-partition duplicates, justified multi-layer evidence, exhaustive Cartesian, and broad defensive matrix fixtures | Distinct required evidence layers are collapsed as duplicates, or volume without a named distinct risk is credited as sufficiency |
| INV8 | Verification-map relationship fixtures | Every audit row mechanically becomes a test, evidence cannot be traced to authority, or one test cannot satisfy inseparable requirements |
| INV9 | Pull-request and audit-run red/green trace fixtures | Audit-authored remediation lacks a failing pre-change case, production remediation precedes the new case, or horizontal test generation replaces one observable vertical slice |
| B1–B16 migration | Rubric-version, section-ownership, and Behavior-slot migration fixture | Generic B rules remain product authority, B12 is reactivated or reused, test evidence is double-scored in Behavior and Testing, B16 remains a Behavior check, unrelated IDs are renumbered, the Behavior weight disappears, or scoring changes without a minor-version bump |