An open source design system that's fully customizable and agent ready
TypeScript
test(vibe-tests): compare established Tailwind adoption strategies (#5821)
nynexman446408db6e9af256
diff --git a/internal/vibe-tests/README.md b/internal/vibe-tests/README.mdindex c90ceccd3c..d9ea60b930 100644--- a/internal/vibe-tests/README.md+++ b/internal/vibe-tests/README.md@@ -153,6 +153,8 @@ internal/vibe-tests/├── setup-test/ # Setup evaluation over the canonical fixture matrix│ ├── PLAN.md # controls, pilot stages, measures, decision rule│ ├── matrix.json # fixtures × controls × prompts × bundles × reps+│ ├── conditions.json # controls plus the opt-in established-app strategies+│ ├── guidance/ # one document per patch, including both strategies│ ├── run-setup.mjs # sandbox + task + provenance preparation only│ ├── setup-interactions.mjs # marker-driven dialog and nested-overlay opener│ ├── setup-measure.mjs # build + exact style, geometry, and overlay probesdiff --git a/internal/vibe-tests/setup-test/PLAN.md b/internal/vibe-tests/setup-test/PLAN.mdindex 23ad5a7979..d1333728c6 100644--- a/internal/vibe-tests/setup-test/PLAN.md+++ b/internal/vibe-tests/setup-test/PLAN.md@@ -46,35 +46,280 @@ Each task declares stable task-owned `data-vibe-result` markers. They are measurGuidance is committed into the sandbox baseline before execution, so the resulting diff contains only executor work.-## Evaluator correctness+## Established-app strategy pilot-Five faults where the evaluator reported an unchanged host as damaged, or reported unrelated problems with the same word. None of them relaxes what the evaluator considers host damage.+Two further conditions explore strategies for an application that already has an established visual language. They are additive and opt-in: the four controls above, their evidence, and their cell identities are untouched — every pre-existing stage keeps its exact expansion count and its exact cell ids, and a strategy is reachable only from the new stages that name it — and no default, core, or greenfield behavior changes as part of this pilot. Normal greenfield Astryx adoption is exactly what it was.-**1. `hardcoded-important` read prose as code.** The check matched `/!\s*important/i` against every added line of a host source file, so an executor told — correctly, by the guidance itself — not to paper over a containment problem with `!important` failed the run for writing that reasoning down beside the workaround it chose instead. The flag is now found by parsing rather than by matching: stylesheets go through postcss, where `Declaration.important` is the parser's own answer and a comment is a `Comment` node; scripts go through the TypeScript parser, where comments are trivia and only string and template literals are examined; HTML, Vue, and Svelte are split into their `<style>` blocks, `style` attributes, and `<script>` blocks. Ignoring comments and prose does not open a way through: a literal counts when it contains a declaration carrying the flag, when it is the value of a real CSS property (taken from `CSSStyleDeclaration` rather than a hand-written list), or when it is the priority argument of `setProperty`, which the old scan could not see at all. An unparsable stylesheet falls back to a scanner that skips only comments and quoted strings.+| Condition | Strategy |+| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |+| `host-aligned` | The established host stays authoritative. The app owns a theme extending neutral instead of importing a stock theme stylesheet, matching host body/heading/code fonts and host semantic color and mode pairs, built deterministically by the app's own build. |+| `guest-contained` | The host keeps global preflight and tokens. The reset is not imported and the provider wraps only the new or migrated subtree, so the design system stays inside that region. |-**2. `dark-mode-disabled` read a mode arm as a disabled mode.** `astryx theme build` emits, for any theme with `light-dark()` values, `:root { color-scheme: light dark; }` paired with a light arm and its dark twin. Those rules _implement_ mode support, but a lexical `color-scheme: light` scan reads the second one as disabling dark mode — and an established host that merely re-indented its own light arm tripped the same check. `color-scheme` is now read semantically. A single-mode declaration is exempt only as a **paired mode arm**: its own selectors and at-rules scope it to exactly one mode, its value is that same mode, and the complementary arm exists at the same scope in the same file. Generated files are not ignored as a class — a file claiming an `@generated by astryx theme build` header is checked against that claim, a file that lost a mode line in the same change gets no exemption at all, and every other escape hatch is scanned inside generated artifacts exactly as before. Forcing a host to one fixed scheme, rewriting one arm of a pair, dropping an arm, and pinning `:root` beside a real pair all still fail.+Each strategy condition carries the discovery pointer plus its own self-contained strategy document. A strategy is deliberately **not** combined with `patch:existing-app`: the two strategies disagree with parts of that generic recipe — `host-aligned` replaces the stock theme stylesheet and `guest-contained` withholds the reset — and combining them would hand the executor contradictory instructions and make the measurement unreadable.-**3. A mandated insertion was scored as host damage.** A task that requires a new dialog obliges the executor to put a trigger for it in the host page, and the host container then grows by exactly that block and gains exactly that block's copy. The contract can now allow exactly those, on exactly the named probe: the container's height, the two geometry fields height moves, and — declared separately as `textInsertionOnly` — text the container _gains_. That is deliberately not the same as putting `text` in `fields`: every baseline word must still be present in its original order, so a container that may grow still cannot lose or rewrite the host's own copy. `text` has been removed from the exemptible host fields entirely, so no contract can ever wave the comparison through, and the container's position, its width, every computed style on it, and every other probe stay exact.+The two are not product equals, and the neutrality this pilot maintains is methodological rather than positional. `host-aligned` is the normal objective for an established app: the host stays authoritative, the whole recipe runs on supported public API, and its cost is maintenance — an app-owned theme that has to track the host's own tokens, and the stock visual identity given up. `guest-contained` is the exceptional mechanism, for an application whose global ownership cannot move and where containing the design system to one nested sub-scope is a hard requirement of the work. It has no fully supported path today: scoping to a subtree depends on suppressing a root attribute sync no public API exposes, and it pays the portal cost described below. It therefore runs as much as a probe of those two product gaps as it does a recipe to follow, and it is not an equal default. What stays neutral is the measurement: the pilot characterizes both strategies per failure category, emits no combined score, and does not rank them.-**4. A lost token boundary was indistinguishable from a restyle.** An established app frequently scopes part of its palette to a region, so host markup relocated into a design-system overlay can leave that region and fall back to global values — the classes do not change and the colour does, which reads like the design system repainting a host component and is not. Each measured surface now records which host token scopes it sits inside, and a mismatch against the reference reports as `host-boundary`, separately from the colours. The two have opposite fixes: restore the boundary, or accept a surface that matches today and silently desynchronizes the next time the host retunes its palette. Restating the colours is explicitly not a fix and cannot buy a pass, and keeping the boundary does not excuse a real repaint. Those scopes are declared per fixture in `probes.json`, because which selector marks a region is a fact about that host's CSS; an executor is never told them.+### Product gaps this pilot surfaces-**5. The report spelled four different failures the same way.** `verdict` collapses measured host damage, an escape hatch, a task failure, and a broken measurement into `silent-damage`, which is right for a gate and useless in a report: an `!important` misread from a comment and a repainted host arrived as the same word. The score now also carries a failure breakdown — `hostDamage`, `runtime`, `integrity`, `task`, `telemetry` — and the per-cell table names the cause beside the verdict. Separately, a run whose executor did not complete is marked not comparable: nothing damaged the host because the work stopped, and that is not evidence about the condition it ran under.+Two findings, both recorded rather than accommodated. The evaluator is not relaxed for either of them: the affected canonical cases fail acceptance, and that failure is the result.++**1. A theme cannot opt out of bare-element rules.** Every built theme emits a prose block — `h1`–`h6`, `p`, `small`, `code`/`pre`, covering family, size, weight, line height, and color, plus an `hr` rule that replaces the host's border — into `@layer reset`, `@scope`'d on the theme-name attribute. This is emitted regardless of which tokens the theme overrides; a theme that changes only a single color still carries the full typographic scale. `host-aligned` additionally requires the theme attribute at document scope before first paint, so the design system's scale necessarily reaches the established host's bare elements. On a host whose bare elements differ from that scale, the result is host damage that no supported configuration avoids. Matching the host exactly would require the theme to reproduce the host's entire type scale, which is only possible when the host's scale is known and static.-## Hermetic measurement+**2. A provider cannot be prevented from claiming document scope.** A provider with no provider above it is a root provider, and a root provider synchronizes `data-theme` and `data-astryx-theme` onto the document element. Because theme CSS is `@scope`'d on that attribute, the sync is what lets theme tokens reach content portaled outside the provider's subtree. The provider's public props are `theme`, `mode`, and `children`; none opts out. The strongest mechanism available today is removing the attribute after mount and keeping it removed with a `MutationObserver` — a workaround, not a configuration — and it costs theming for body-portaled surfaces. The canonical `guest-contained` implementation carries that workaround, labels it as one, and documents the tradeoff at the callsite.-Measurement reads a sandbox; it must not write to one. That tree is the attested artifact — the runner digested its bytes, the integrity checker reads them, and any re-measurement or recovery has to read them again — and building in place destroyed it: the fixtures' `prebuild` regenerates the app-owned theme over the executor's copy, `vite build` writes `dist/`, the CLI appends to its invocation log, and the bundler caches into `node_modules`. A failed build leaves the same debris. A measured cell then no longer hashed to its own digest and its evidence could not be re-derived.+Gap 1 is the most serious: it means "the established host stays authoritative over typography" is not currently achievable through supported configuration whenever the host renders bare prose elements. It is now the only thing keeping `host-aligned` from acceptance, and it is a real product gap rather than an evaluator artifact. `guest-contained` does reach acceptance, at the portal cost in gap 2 — but acceptance is the measurement's verdict on host damage, not a product recommendation: it is reached by way of the unsupported workaround in gap 2, so the reading is that gap 1 blocks the supported strategy while gap 2 is the price of the exceptional one.-**The build runs in a disposable copy.** Files are copied with `cp -a --reflink=auto`, which shares extents copy-on-write where the filesystem supports it and byte-copies where it does not, with a pure-Node fallback for a platform without a usable `cp`. It never hardlinks: a hardlinked file rewritten in the copy rewrites the original inode.+### The evaluator defect this pilot found, corrected here-**The pnpm virtual store is copied, not linked.** A pnpm `node_modules` is mostly links into `.pnpm`, the store holding every real package tree, so mirroring it with a symlink hands the copy a door straight back into the sandbox — writing through a symlink writes to its target, and an install or rebuild in the copy would rewrite the original's store. Symlinks keep their target text, so a package link resolves within the copy; an absolute link that would reach into the sandbox is retargeted onto the copy, while one pointing at a workspace package elsewhere is left alone. `verifyWorkspaceIsolation` re-derives that claim from the finished copy — every symlink resolved, and inodes compared against the original — and the measurer runs it before any build. The build root is removed on every path out, including a thrown measurement.+`astryx theme build` emits, for any theme with `light-dark()` values, `:root { color-scheme: light dark; }` paired with `html[data-theme="light"] { color-scheme: light; }` and its dark twin. Those rules implement mode support, but the `dark-mode-disabled` escape hatch matched a bare `color-scheme: light` lexically, so any app shipping a built theme tripped the check — as did an established host that merely re-indented its own light arm. That is an evaluator defect rather than a product gap, so it is fixed rather than recorded.-Tests take a full byte-and-mtime manifest of the sandbox before and after a real measurement and require it identical for a build that succeeds and a build that fails, with the virtual store included. Mutation proofs put each way of getting it wrong back — the mirrored store, and a hardlinked copy — and show the sandbox is written through and the isolation check names it. Measurement output is unchanged: stable fixture ids, no private paths, and the build root redacted alongside them.+`color-scheme` is now read semantically. A single-mode declaration is exempt only as a **paired mode arm**: its own selectors and at-rules scope it to exactly one mode, its value is that same mode, and the complementary arm exists at the same scope in the same file. Generated files are not ignored as a class — a file claiming an `@generated by astryx theme build` header is checked against that claim (the declared source must be a relative path inside the sandbox that still exists, and the artifact must sit where that build writes it), a file that lost a mode line in the same change gets no exemption at all, and every other escape hatch is scanned inside generated artifacts exactly as before. Forcing a host to one fixed scheme, rewriting one arm of a pair, dropping an arm, and pinning `:root` beside a real pair all still fail. No other stage's evaluator behavior changes.-**Probes are read at rest.** `getComputedStyle` on an element with a running transition returns the interpolated value, which Chromium serializes in the interpolation space rather than the space the value was authored in — so a host colour that never changed reads back as `oklab(…)` mid-transition and `oklch(…)` at rest. The numbers are identical, only the spelling differs, and an exact comparison scores that as host damage. The harness triggers these itself, by clicking host controls to open an interaction, and whether a read lands inside the transition window depends on how fast the app's own bundle responds — which is stable per app, so it misreads deterministically for some apps and never for others. Running transitions are now settled before any read. Only `CSSTransition` is awaited: transitions always finish, while an infinite decorative animation never would.+### What the first pilot run found, corrected here-**Container text counts only what the browser renders.** A container's protected text excludes task-owned subtrees, because a task that mandates a control necessarily adds that control's text. It could not exclude an overlay the design system hoists _out_ of the task-owned element — a `Button` with a `tooltip` renders it as a `display: none` sibling linked by `aria-describedby` — so a mandated control's own closed tooltip landed in the host container's text and scored as host damage while nothing on screen had changed. Unrendered subtrees no longer count. This is not a hole an executor can hide damage in: removing, rewriting, or hiding host copy all still change the string, and only invisible _additions_ stop counting.+The first operator run (10 of the 80 cells: one bundle, one repetition) exposed five more evaluator defects and one instruction defect. All six are corrected here; none of them relaxes what the evaluator considers host damage.++**1. `hardcoded-important` read prose as code.** The check matched `/!\s*important/i` against every added line of a host source file. Three of the ten cells failed on a JSDoc comment — the `guest-contained` guidance tells an executor not to paper over the portal tradeoff with `!important`, the executor wrote that reasoning down beside the workaround it chose instead, and the check failed the run for saying it. The flag is now found by parsing rather than by matching: stylesheets go through postcss, where `Declaration.important` is the parser's own answer and a comment is a `Comment` node; scripts go through the TypeScript parser, where comments are trivia and only string and template literals are examined; HTML, Vue, and Svelte are split into their `<style>` blocks, `style` attributes, and `<script>` blocks. Ignoring comments and prose does not open a way through: a literal counts when it contains a declaration carrying the flag, when it is the value of a real CSS property (taken from `CSSStyleDeclaration` rather than a hand-written list), or when it is the priority argument of `setProperty`, which the old scan could not see at all. An unparsable stylesheet falls back to a scanner that skips only comments and quoted strings. Mutation proofs cover a style object, a JSX inline style, a CSS-in-JS template, an interpolated declaration, an injected stylesheet string, a `cssText` write, a runtime `setProperty`, and an override written directly beneath — and on the same line as — the comment that disclaims it.++**2. The `s5` contract had no allowance for the insertion the task mandates.** `s5` requires a new Astryx dialog, so the executor must put a trigger for it in the host page. Both `s5` cells added a block to the host shell and both reported the identical eight regressions — `height`, `geometry.height`, `geometry.bottom`, and `text`, in light and dark — for doing what the task said. The contract now allows exactly those, on exactly that probe: the shell's height and the two geometry fields height moves, plus text the shell _gains_. `textInsertionOnly` is deliberately not the same as putting `text` in `fields`: every baseline word must still be present in its original order, so a container that may grow still cannot lose or rewrite the host's own copy. Nothing else moves — the shell's position, its width, every computed style on it, and every other probe stay exact, and `text` has been removed from the exemptible host fields entirely so no contract can ever wave the comparison through. Canonical browser cases prove the mandated reflow passes, and mutation proofs over that same measured run prove a nearby color, font, radius, shadow, or padding, the shell's own width or position, another probe's geometry or color, and any lost host copy all still fail.++The same gap existed one container in. `s5` mandates exactly one visible `astryx-dialog-trigger`, and this fixture declares a guest subtree for guest design-system content, so that is where the trigger belongs — and where two independent reps put it. Both grew `guest-boundary` from 296px to 409px, moving its height and the two geometry fields height moves, in both schemes. That is the insertion's own geometry, so the contract now allows those three fields on that probe too. It is narrower than the shell's: the trigger is task-owned, so the boundary's protected text already excludes it and the allowance names no text exemption at all. The boundary's width, its position, its every computed style, and every other probe stay exact, proved the same way — a canonical browser case that inserts the trigger into the guest subtree and measures the real app, and mutation proofs over that measured run for a color, a font, the boundary's width, the boundary moving, a neighboring probe's height, and host copy the boundary either gains or loses.++**3. `gitignore-modified` was right; the instruction was wrong.** Two `host-aligned` cells added the generated theme artifacts to `.gitignore` and failed the escape hatch. The hatch is not weakened, here or anywhere: an executor may not rewrite ignore rules after the execution baseline. What was missing is the instruction. The guidance said the theme is built deterministically and never said the output is a tracked file, while this app's own history commits its generated theme artifacts. `host-aligned` guidance now requires committing both the theme source and everything the build writes from it, forbids adding that output to `.gitignore`, and answers the argument both executors actually made — that a regenerated file need not be tracked. Those two cells remain failures: they are an instruction failure and an agent failure, not an evaluator defect, and they have to be rerun under the corrected guidance rather than rescored.++**4. Measuring mutated the sandbox it measured.** _Historical: the fault is real and stands, but the fix described here was **superseded by item 5 below** before any of it shipped. The link-mirrored `node_modules` is not the current construction — read item 5 for what the build root actually does._ The measurer ran `pnpm build` inside the agent's own sandbox. That tree is the attested artifact — the runner digested those bytes, integrity reads them, and any re-measurement or recovery has to read them again — and the build overwrote it: `prebuild` regenerates the app-owned theme over the executor's copy, `vite build` writes `dist/`, the CLI appends to its invocation log, and the bundler caches into `node_modules`. A failed build leaves the same debris, so a measured cell no longer hashed to its own digest and its evidence could not be re-derived. The build now runs in a disposable copy. Symlinks are reproduced as symlinks so resolution is unchanged, and — in this first cut, since superseded — `node_modules` was mirrored with one link per package rather than copied or hardlinked, on the reasoning that a hardlinked file rewritten by a build writes through to the original inode while a mirrored directory catches the write in the copy. The build root is removed on every path out, including a thrown measurement. Tests take a full byte-and-mtime manifest of the sandbox before and after a real measurement and require it identical for a successful build, a failing build, and a thrown one; a mutation proof runs the old in-place build and shows the manifest reports it. Measurement output is unchanged: stable fixture ids, no private paths, and the build root redacted alongside them. Tests take a full byte-and-mtime manifest of the sandbox before and after a real measurement and require it identical for a successful build, a failing build, and a thrown one; a mutation proof runs the old in-place build and shows the manifest reports it. Measurement output is unchanged: stable fixture ids, no private paths, and the build root redacted alongside them.++**5. The build root mirrored the virtual store instead of copying it.** The first cut of the build root symlinked each `node_modules` entry to the original, which for a pnpm install means `.pnpm` — the virtual store holding every real package tree — was a link back into the sandbox. Writing through a symlink writes to its target, so an install or a rebuild in the copy rewrote the sandbox's own store, and the top-level package links resolved into it as well. The copy is now a real copy: files are copied with `cp -a --reflink=auto`, which shares extents copy-on-write where the filesystem supports it and byte-copies where it does not, and never hardlinks — a hardlinked file rewritten in the copy rewrites the original inode. Symlinks keep their target text, so `is-odd -> .pnpm/[email protected]/node_modules/is-odd` and every dependency link inside the store now resolve within the copy; an absolute link that would reach into the sandbox is retargeted onto the copy, while one pointing at a workspace package elsewhere is left alone. `verifyWorkspaceIsolation` re-derives the claim from the finished copy — every symlink resolved, and inodes compared against the original — and the measurer runs it before any build. Tests build the fixture with a real `pnpm install`, then require the sandbox's full manifest, virtual store included, to be byte-identical after a successful build, a failing build, and an install in the copy that deletes and recreates `.pnpm` entirely; the pure-Node copier is exercised the same way. The mutation restores the old mirroring and shows the sandbox is written through and the isolation check names it, and a hardlinked copy is caught by the same check.++**6. The report spelled four different failures the same way.** `verdict` collapses measured host damage, an escape hatch, a task failure, and a broken measurement into `silent-damage`, which is right for a gate and useless in a report: an `!important` misread from a comment and a repainted host arrived as the same word. The score now also carries a failure breakdown — `hostDamage`, `runtime`, `integrity`, `task`, `telemetry` — and the per-cell table names the cause beside the verdict. Separately, a strategy is never called cleaner on the strength of a run its executor did not complete. `host-aligned` measured `clean` on an `s4` cell that was an agent failure, because nothing damaged the host once the work stopped; those runs are now reported as not completed, excluded from the clean and comparable counts, and named as cells needing a rerun.++### What the four remaining pilot failures were, and what each one is++Ten cells ran; six were accepted. The four that were not have been analyzed+against their patches, measurements, task contracts, and the cells that passed+beside them. They are four different things, and only one of them is the host+actually changing.++**1. `guest-contained` / `enterprise-scoped-synthetic` / `s1` — measurement.**+Two host-text regressions on `guest-boundary`, both schemes: the container's+protected text gained `Roll out the selected services`. That string is the+`tooltip` on the mandated Astryx `Button`. Astryx renders a `Button`'s tooltip+as a `display: none` sibling — hoisted out of the button, because a button+cannot legally contain it — linked back by `aria-describedby`. The container's+text excludes `data-vibe-*` subtrees, so it could not see a task-owned overlay+sitting outside the task-owned element, and counted the mandated control's own+unrendered tooltip as host copy. Nothing on screen changed. `protectedText` now+counts only text the browser renders. This is not a hole: removing, rewriting,+or hiding host copy all still change the string, and only invisible _additions_+stop counting. Proven both ways in `setup-measure.browser.test.mjs`, and the+sandbox re-measures with zero differences against its baseline.++**2. `guest-contained` / `shadcn-tailwind-v4-established` / `s4` — measurement.**+One colour regression on `primary-action`, dark scheme:+`oklch(0.985 0 0)` became `oklab(0.985 0 0)`. Those are the same colour; only+the spelling differs. The fixture's button carries `transition-colors`, the+harness's own interaction clicks that button to open the host dialog, and+probes were read immediately afterwards — inside the 150 ms transition window,+where `getComputedStyle` returns the interpolated value serialized in the+interpolation colour space. Reading the built app at rest returns `oklch` in+both schemes; reading it straight after the click returns `oklab` with one+running animation. It reproduced in four independent measurement passes because+the window is decided by how long the app's own bundle takes to respond, which+is stable per app — deterministic for one app and never for another, which is+worse than flaky. Probes are now read after running CSS transitions settle.+Only `CSSTransition` is awaited, so an infinite decorative animation cannot+hang a measurement.++**3. `host-aligned` / `shadcn-tailwind-v4-established` / `s1` — never tested.**+Zero measured damage, task complete, blocked solely on `gitignore-modified`.+The rule against ignoring generated theme output was already written, and the+question was why the executor missed it. It did not miss it: **the rule is+absent from every one of the five host-aligned sandboxes the pilot produced.**+It was written into the guidance after those runs. Each run records a digest of+the guidance it was handed, and the digest recorded for these cells does not+match the digest recomputed from the current text. So this is not instruction+ambiguity and not an agent failure — the instruction has never actually been+delivered to an executor, and the rerun is its first test rather than a second+attempt. The rule is unchanged in substance; it is now also stated before the+reasoning, where a reader meets it first.++**4. `host-aligned` / `enterprise-scoped-synthetic` / `s5` — instruction.**+`gitignore-modified` as above, plus a genuine finding:+`host-menu-surface:host-style` in both schemes. The host menu, composed inside+the Astryx dialog the task mandates, lost the host's violet palette —+background, all four border colours, and text colour fell back to the app's+global values. This is not Astryx repainting anything. The fixture scopes that+palette to `[data-guest-design-system]`, the baseline menu is a descendant of+that region, and the relocated menu is not; `popover` promotes it to the top+layer, which changes where it paints and not where it inherits from. It is+reproducibly avoidable: the `guest-contained` cell on the same fixture, the+same prompt, and the same contract matched the host surface exactly, by putting+that scoping attribute on the relocated element. `host-aligned` guidance now+covers host markup composed inside a design-system overlay. The same hazard+exists under `guest-contained`, where it was solved by discovery rather than by+instruction; that document is deliberately left alone here.++Nothing above weakens acceptance. No zero-damage rule is relaxed, no field is+blanket-allowed, no contract gained an exemption, and `gitignore-modified`+stands exactly as it was. Two of the four are corrections to how the host was+measured; two are about what the executor was told.++### One thing the pilot measured that is not a failure and is not fixed here++Every one of the ten cells, under both strategies, newly defines `--color-*`+custom properties on the document root that the baseline did not define. The+measurement records this as `variablesCaptured`, which is reported and+deliberately does not gate.++It reads differently per strategy, and both readings are worth keeping in view:++- Under `host-aligned` the values are the host's own — `#2457d6` where the+ enterprise fixture's accent is `#2457d6`, `oklch(20.5% 0 0)` where shadcn's+ primary is `oklch(20.5% 0 0)`. The app-owned theme is doing exactly what the+ strategy asks of it.+- Under `guest-contained` they are Astryx's stock defaults — `#0064e0` and its+ neighbours — sitting in the host's root scope in a strategy whose whole claim+ is that it does not put design-system values there. Containment holds for the+ `@scope`'d theme rules, which is what the `data-astryx-theme` workaround is+ for, and does not hold for the Tailwind token layer, which is emitted at+ `:root` unscoped and is not covered by the workaround at all.++No cell was damaged by it: none of the three fixtures has a host element+resolving `var(--color-accent)` at root, because each defines its own token+names. A host that did would be repainted, and neither strategy document+currently says so. Deciding what to do about it — measure it as damage, scope+the token layer, or document it as a bound on the guest-contained claim — is a+larger question than the four cells this iteration reruns, so it is recorded+here and left open rather than answered by widening a gate.++### Rerunning those four cells without moving the pilot++The pilot's ten cells keep their ids, their measurements, and their verdicts. A+rerun under the same ids would overwrite the very evidence that justified these+changes, so the reruns carry new condition ids — `guest-contained-r2` and+`host-aligned-r2`, each declaring what it revises and what changed — and land+in their own `strategy-iteration` stage. Stage cell identity is+`stage__condition__fixture__prompt__bundle__rep`, so no rerun can collide with a+pilot cell, and tests hold every earlier stage's expansion at its exact count.++The stage carries an explicit four-cell allowlist rather than a cross product,+which for two conditions over two fixtures would be eight. `comparisonMapping`+pairs each rerun with the pilot cell it reruns, the finding it addresses, and+whether guidance, measurement, or delivery changed for it. A rerun that passes+retires that finding; a rerun that fails leaves the pilot's failure standing and+says the correction was wrong.++### What the first iteration run found++Two of the four reruns passed, which retires two findings: the host-text+regression on `guest-boundary` and the dark-mode colour on `primary-action` were+both measurement faults, and with those corrected the cells are clean. The+`gitignore-modified` rule, delivered to a sandbox for the first time, was+followed. Two cells did not pass, for two unrelated reasons.++**1. `guest-contained-r2` / `enterprise-scoped-synthetic` / `s1` — the install+never completed.** Not a host-damage result at all: the build failed, so+nothing about the strategy was measured. `@astryxdesign/core` ships a+`postinstall`, and pnpm 11 refuses to install a dependency with an install+script until that dependency is approved:-**The local file server binds what it fetches.** `server.listen(0, …)` with no host binds the wildcard — `::` wherever Node has IPv6 — while the page fetch was a hardcoded `http://127.0.0.1:<port>`. Those agree only where the host maps v4-in-v6; where they do not, every page load fails to connect and the run looks like a broken app. The server now binds IPv4 loopback explicitly and the origin is read back from the socket, with a wildcard rewritten to the loopback of its own family and IPv6 literals bracketed. Measurement needs no network namespace or loopback shim; if a runner wraps it in one out of habit, drop the wrapper.+```+[ERR_PNPM_IGNORED_BUILDS] Ignored build scripts: @astryxdesign/[email protected]+```++The executor wrote `pnpm.ignoredBuiltDependencies` into `package.json`. Verified+directly against pnpm 11.24, the supported mechanism is `pnpm approve-builds`,+which writes an `allowBuilds` map into **`pnpm-workspace.yaml`**:++| Placement | Key | Result |+| ----------------------------------- | --------------------------- | ---------------------------- |+| `pnpm-workspace.yaml` | `allowBuilds: {pkg: true}` | installs, `postinstall` runs |+| `pnpm-workspace.yaml` | `allowBuilds: {pkg: false}` | installs, script skipped |+| `package.json` → `pnpm.allowBuilds` | same map | **ignored**, install fails |+| either file | `ignoredBuiltDependencies` | **ignored**, install fails |+| either file | `onlyBuiltDependencies` | **ignored**, install fails |++Three of the four executors in that run got there unaided, mostly by running+`approve-builds`; one did not. The `guest-contained` document now names the+command, the file, and the two plausible keys that do not work. The shared+discovery pointer is deliberately untouched: adding install mechanics to it+would change what the `current`, `candidate`, and `direct` controls measure, and+`floor` is supposed to receive no guidance at all.++`pnpm-workspace.yaml` is also the neutral place for it. npm and yarn ignore that+file, so approving a pnpm build leaves no pnpm-specific configuration in the+manifest those tools read.++Nothing in this suite caught the hazard because the canonical harness installs+fixture dependencies with `--ignore-scripts`, which sidesteps approval entirely.+There are now canonical cases that install the way an executor does.++**2. `host-aligned-r2` / `enterprise-scoped-synthetic` / `s5` — the boundary+rule was read and misapplied.** The relocation guidance added in the previous+revision was delivered and the executor acted on it: it carried the host's+`fixture-shell` class and `data-mode` attribute onto the relocated menu. Those+are the host's _mode_ hooks. The attribute scoping the _palette_ —+`data-guest-design-system`, which in this fixture sits on the surface element+itself — was dropped, so the menu left the token region and fell back to the+app's global values, exactly as before.++That is an instruction failure rather than an executor failure: "find what+scopes that surface's tokens" was one line in a section far down the document,+and an app having one hook for mode and a different one for a region is common+enough that picking the wrong one is the expected mistake. The rule is now one+of the document's stated non-negotiables, and it carries a procedure — work+backwards from the properties behind the surface's classes, find every rule that+redeclares them, and the non-global one is the boundary — with an explicit+warning not to assume the mode hook is the palette hook.++The measurement was also at fault for being undiagnosable. A dropped boundary+and a genuine repaint are different problems with opposite fixes, and both+arrived as `host-style`. Each measured surface now records which host token+scopes it sits inside, and a mismatch against the reference reports as+`host-boundary`, separately from the colours. Those scopes are declared per+fixture in `probes.json`, because which selector marks a region is a fact about+that host's CSS: an executor is never told it, and the guidance teaches+discovery rather than naming it.++Restating the region's colours on the moved element is explicitly not a fix, and+cannot buy a pass: it produces a surface that matches today and desynchronizes+the next time the host retunes its palette. A mutation proof holds that line —+colours restated to match, boundary still gone, `host-boundary` still reported —+and its converse holds the other, where keeping the boundary does not excuse a+real repaint.++### Two harness faults found alongside those cells++Neither is about a strategy; both distorted or blocked measurement.++**The local file server was bound to one address and fetched at another.**+`server.listen(0, …)` with no host binds the wildcard — `::` wherever Node has+IPv6 — while the page fetch was a hardcoded `http://127.0.0.1:<port>`. Those+agree only where the host maps v4-in-v6; where they do not, every page load+fails to connect and the run looks like a broken app. The operator worked around+it by running the whole measurer inside a network namespace with loopback forced+up. Nothing about measuring a built app needs that. The server now binds IPv4+loopback explicitly and the origin is read back from the socket, with a wildcard+rewritten to the loopback of its own family and IPv6 literals bracketed.++**Toolchain preflight verifies a profile shell, which is not what a hosted child+necessarily gets.** The runner invokes measurement through `bash -lc`, a login+shell, which on the machine used here resolves Node 24.20.0 and pnpm 11.24.0+from the user profile. A non-login shell on the same machine resolves a+different Node and no pnpm at all. So a green preflight attests to the profile+shell it ran in, not to the environment of a child process started some other+way. No claim is made here about what Node version a hosted child receives:+that has not been audited, and the two shells demonstrably differ. A runner that+depends on the profile toolchain should invoke a login shell explicitly or pin+the toolchain, rather than assume the preflight covers it.++### Rerunning those two cells++Same rule as before: nothing already recorded moves. The reruns take new+condition ids — `guest-contained-r3` and `host-aligned-r3`, each naming the+revision it supersedes — and land in `strategy-iteration-2`. Revisions chain, so+the lineage of every cell is readable from the conditions file, and tests hold+every earlier stage at its exact count: 48 / 400 / 112 / 80 / 4.## Measurement and integrity@@ -110,9 +355,12 @@ Prompt applicability produces 14 fixture-prompt pairs: four basic tasks across t1. **Separation — 48 prepared runs:** `3 fixtures × 2 controls × s1 × 4 bundles × K=2`.2. **Guidance iteration — 400 additional prepared runs:** the full four-condition matrix has 448 unique cells; reuse the 48 separation cells and prepare the other 400. No control evidence is rerun or double-counted.3. **Final confirmation — 112 required valid runs:** `14 supported fixture-prompt pairs × candidate × 4 bundles × K=2`.+4. **Strategy pilot — 80 prepared runs:** `5 fixture-prompt pairs × 2 strategies × 4 bundles × K=2`. The five pairs are `s1` on all three fixtures plus one cross-system task on each established fixture (`s4`, `s5`), so the pilot characterizes downstream composition rather than install time alone. Stages 1–3 keep their exact counts and cell identities; a pilot cell can never collide with theirs, because its condition is one of the two strategies and cell identity is `stage__condition__fixture__prompt__bundle__rep`.The repository prepares tasks and sandboxes but does not launch executors. Missing, blocked, invalid, or omitted cells make the confirmation incomplete; they do not shrink its denominator.+The pilot's acceptance is the same as everywhere else: a valid run passes only when it is `clean` with task-specific success, and the operator report is per strategy and per failure category with no combined score. A run whose executor did not complete is reported as not completed and never counts toward a strategy's clean or comparable totals, whatever its measurement says — a cell that stopped early damaged nothing because no work happened, and that is not evidence for the strategy. The initial operator run may filter that prepared matrix to one bundle and one repetition — 10 of the 80 cells — without changing the matrix itself.+## Commands```bash@@ -140,3 +388,95 @@ node internal/vibe-tests/setup-test/setup-measure.mjs \--provenance <run.provenance.json> \--out <measurement.json>```++### Strategy pilot++The stage prepares all 80 cells. The initial operator run filters that same+matrix to one bundle and one repetition — 10 cells — without changing it:++```bash+node internal/vibe-tests/setup-test/run-setup.mjs \+ --stage strategy-pilot \+ --bundles claude-code-claude \+ --reps 1 \+ --out /tmp/strategy-pilot++# Per-strategy, per-failure-category report. No combined score is emitted.+pnpm tsx internal/vibe-tests/setup-test/setup-aggregate.ts \+ --stage strategy-pilot \+ --dir /tmp/strategy-pilot+```++### Strategy iteration++Reruns the four pilot cells that did not pass. The stage's allowlist already+names exactly those four, so the command needs no filters and prepares four+cells:++```bash+node internal/vibe-tests/setup-test/run-setup.mjs \+ --stage strategy-iteration \+ --out /tmp/strategy-iteration++# Same report, plus the pilot cell each rerun is paired against.+pnpm tsx internal/vibe-tests/setup-test/setup-aggregate.ts \+ --stage strategy-iteration \+ --dir /tmp/strategy-iteration+```++The four prepared cells are:++| Cell | Reruns | Finding it addresses | Changed |+| ---------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------- | -------------------------------------------------------- | ----------------- |+| `strategy-iteration__guest-contained-r2__enterprise-scoped-synthetic__s1__claude-code-claude__r1` | `strategy-pilot__guest-contained__enterprise-scoped-synthetic__s1__claude-code-claude__r1` | 2 host-text regressions on `guest-boundary` | measurement |+| `strategy-iteration__guest-contained-r2__shadcn-tailwind-v4-established__s4__claude-code-claude__r1` | `strategy-pilot__guest-contained__shadcn-tailwind-v4-established__s4__claude-code-claude__r1` | 1 host colour regression on `primary-action`, dark | measurement |+| `strategy-iteration__host-aligned-r2__shadcn-tailwind-v4-established__s1__claude-code-claude__r1` | `strategy-pilot__host-aligned__shadcn-tailwind-v4-established__s1__claude-code-claude__r1` | `gitignore-modified`, no measured damage | guidance delivery |+| `strategy-iteration__host-aligned-r2__enterprise-scoped-synthetic__s5__claude-code-claude__r1` | `strategy-pilot__host-aligned__enterprise-scoped-synthetic__s5__claude-code-claude__r1` | `gitignore-modified` plus `host-menu-surface:host-style` | guidance |++A single cell can be prepared on its own with the usual filters, for example:++```bash+node internal/vibe-tests/setup-test/run-setup.mjs \+ --stage strategy-iteration \+ --conditions host-aligned-r2 \+ --fixtures enterprise-scoped-synthetic \+ --prompts s5 \+ --out /tmp/strategy-iteration+```++Baselines must be re-measured with the same build as the arms: the measurement+corrections change what a baseline reads too, and a corrected arm compared+against a pilot-era baseline is not a comparison.++### Strategy iteration 2++Reruns the two cells the first iteration did not pass. The stage's allowlist+names exactly those two, so the command needs no filters and prepares two cells:++```bash+node internal/vibe-tests/setup-test/run-setup.mjs \+ --stage strategy-iteration-2 \+ --out /tmp/strategy-iteration-2++# Same report, plus the cell each rerun is paired against.+pnpm tsx internal/vibe-tests/setup-test/setup-aggregate.ts \+ --stage strategy-iteration-2 \+ --dir /tmp/strategy-iteration-2+```++The two prepared cells are:++| Cell | Reruns | Finding it addresses | Changed |+| --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------------- | ---------------------- |+| `strategy-iteration-2__guest-contained-r3__enterprise-scoped-synthetic__s1__claude-code-claude__r1` | `strategy-iteration__guest-contained-r2__enterprise-scoped-synthetic__s1__claude-code-claude__r1` | `build-failed` — `ERR_PNPM_IGNORED_BUILDS`; never measured | guidance |+| `strategy-iteration-2__host-aligned-r3__enterprise-scoped-synthetic__s5__claude-code-claude__r1` | `strategy-iteration__host-aligned-r2__enterprise-scoped-synthetic__s5__claude-code-claude__r1` | `host-menu-surface:host-style`, both modes | guidance + measurement |++Baselines must be re-measured with the same build as the arms, for the same+reason as the previous iteration.++Measurement no longer needs a network namespace or a loopback shim. If a runner+wraps it in one out of habit, drop the wrapper: the server binds IPv4 loopback+and fetches the address it bound to. Invoke the measurer through a login shell,+or pin Node and pnpm explicitly — the toolchain the preflight verifies is the+profile shell's, and a non-login shell on the same machine resolves a different+one.diff --git a/internal/vibe-tests/setup-test/conditions.json b/internal/vibe-tests/setup-test/conditions.jsonindex d77afc6e30..2ccd2a9c78 100644--- a/internal/vibe-tests/setup-test/conditions.json+++ b/internal/vibe-tests/setup-test/conditions.json@@ -37,8 +37,87 @@"role": "direct-control","patches": ["patch:pointer", "patch:existing-app", "patch:directed"],"hypothesis": "Precise setup instructions remove discovery ambiguity. Any remaining visible host damage is a product gap and blocks instruction shipping."+ },+ {+ "id": "host-aligned",+ "label": "Host-aligned strategy (pilot)",+ "role": "strategy-pilot",+ "optIn": true,+ "patches": ["patch:pointer", "patch:host-aligned"],+ "hypothesis": "When the established host stays authoritative over typography and color, an app-owned theme that extends neutral and matches host fonts, semantic colors, and mode pairs avoids host damage that a stock theme stylesheet would cause."+ },+ {+ "id": "guest-contained",+ "label": "Guest-contained strategy (pilot)",+ "role": "strategy-pilot",+ "optIn": true,+ "patches": ["patch:pointer", "patch:guest-contained"],+ "hypothesis": "When the host owns global preflight and tokens, mounting the provider around only the new subtree and withholding the reset avoids host damage, at a portal-scope cost that must be measured rather than assumed."+ },+ {+ "id": "guest-contained-r2",+ "label": "Guest-contained strategy (pilot, revision 2)",+ "role": "strategy-pilot",+ "optIn": true,+ "patches": ["patch:pointer", "patch:guest-contained"],+ "supersedes": "guest-contained",+ "revisionOf": "guest-contained",+ "changedSincePriorRevision": "measurement only — the guest-contained document is byte-identical to the revision this supersedes. Two measurement faults it exposed were corrected: host container text now counts only text the browser renders, and probes are read after running CSS transitions settle. Both faults reported an unchanged host as damaged.",+ "hypothesis": "The two guest-contained cells that failed the pilot failed on how the host was measured rather than on what the strategy did to the host, so the same document under a corrected measurement produces no host damage."+ },+ {+ "id": "host-aligned-r2",+ "label": "Host-aligned strategy (pilot, revision 2)",+ "role": "strategy-pilot",+ "optIn": true,+ "patches": ["patch:pointer", "patch:host-aligned"],+ "supersedes": "host-aligned",+ "revisionOf": "host-aligned",+ "changedSincePriorRevision": "document and measurement. The document states its non-negotiable rules before the reasoning and adds a section on host markup composed inside a design-system overlay. The rule against ignoring generated theme output was already written but is not present in any sandbox the prior revision produced, so that rule has never actually been tested.",+ "hypothesis": "Told before the reasoning that generated theme output is committed, and told that relocated host markup must carry the host's own token-scoping hook, the executor produces no integrity failure and no host-surface style change."+ },+ {+ "id": "guest-contained-r3",+ "label": "Guest-contained strategy (pilot, revision 3)",+ "role": "strategy-pilot",+ "optIn": true,+ "patches": ["patch:pointer", "patch:guest-contained"],+ "supersedes": "guest-contained-r2",+ "revisionOf": "guest-contained-r2",+ "changedSincePriorRevision": "document only. The guest-contained document now names the supported pnpm 11 mechanism for approving a dependency's build script. The r2 run failed its build before any measurement could happen: the executor wrote the approval into package.json, which pnpm 11 does not read, so the install aborted with ERR_PNPM_IGNORED_BUILDS. Nothing about the strategy itself was measured on that cell.",+ "hypothesis": "Told the mechanism pnpm 11 actually reads, the executor installs and builds, and the cell can finally be measured on what the strategy does to the host rather than on how it configured a package manager."+ },+ {+ "id": "host-aligned-r3",+ "label": "Host-aligned strategy (pilot, revision 3)",+ "role": "strategy-pilot",+ "optIn": true,+ "patches": ["patch:pointer", "patch:host-aligned"],+ "supersedes": "host-aligned-r2",+ "revisionOf": "host-aligned-r2",+ "changedSincePriorRevision": "document and measurement. The relocation rule is now one of the document's stated non-negotiables rather than a section far down it, and it carries a procedure for finding the boundary rather than an instruction to find one. The measurement now records which host token scopes a surface sits inside and reports a lost boundary as its own failure, so the r2 outcome is diagnosable instead of appearing only as a list of colours.",+ "hypothesis": "The r2 executor carried the host's mode hooks onto the relocated menu but not the attribute scoping its palette, which is the specific mistake the procedure now addresses. Told to carry the boundary and how to find it, the executor keeps the relocated host surface inside its own token region and matching its reference exactly."}],+ "strategyPilot": {+ "conditions": [+ "host-aligned",+ "guest-contained",+ "host-aligned-r2",+ "guest-contained-r2",+ "guest-contained-r3",+ "host-aligned-r3"+ ],+ "revisions": {+ "$comment": "A revision is a new condition id, never an edit to an existing one. Earlier cells keep their ids, their measurements, and their verdicts exactly as recorded; a revision reruns a named subset beside them under its own identity so the two can be compared and neither can overwrite the other. Revisions chain: r3 supersedes r2, which supersedes the original, and each names the one it revises.",+ "host-aligned-r2": "host-aligned",+ "guest-contained-r2": "guest-contained",+ "guest-contained-r3": "guest-contained-r2",+ "host-aligned-r3": "host-aligned-r2"+ },+ "notes": "Opt-in established-app strategies explored additively. They do not replace floor, current, candidate, or direct, and they change no default or core behavior: greenfield adoption is unaffected. Each strategy condition carries the discovery pointer plus its own self-contained strategy document. Neither strategy is presented as universally correct.",+ "supersedes": "A strategy document is self-contained and is not combined with patch:existing-app, because the two strategies deliberately disagree with parts of that recipe — host-aligned replaces the stock theme stylesheet, and guest-contained withholds the reset. Combining them would hand the executor contradictory instructions and make the measurement unreadable."+ },"measurement": {"reference": "A pristine copied fixture, measured by the same code before setup.","schemes": ["light", "dark"],diff --git a/internal/vibe-tests/setup-test/guidance/guest-contained.md b/internal/vibe-tests/setup-test/guidance/guest-contained.mdnew file mode 100644index 0000000000..334533f8ab--- /dev/null+++ b/internal/vibe-tests/setup-test/guidance/guest-contained.md@@ -0,0 +1,148 @@+## Strategy: guest-contained adoption++This app already owns its global stylesheet, its CSS reset, and its token+vocabulary. Adopt this strategy when the design system must live inside one new+or migrated region and leave the rest of the page untouched.++This is the exceptional strategy, not the established-app default. The normal+objective for an app like this one is host-aligned adoption — the host stays+authoritative over typography and color, and the design system is themed to it+through supported public APIs. Containment is for the narrower case where+leaving the rest of the page alone is a hard requirement of the work: a nested+or sub-scoped region inside an application whose global ownership cannot move.++It is also the strategy with no fully supported path today. Scoping the design+system to a subtree means suppressing the root attribute sync described below,+and there is no public API for that; portaled content rendered outside the+subtree then falls outside every scope root and loses its tokens. Both are+product gaps rather than choices this document can make well, and following this+strategy exercises them deliberately — what it produces is evidence about those+gaps as much as it is a recipe.++### The host keeps global ownership++- Do **not** import `@astryxdesign/core/reset.css`. The host already owns+ preflight and element defaults; a second reset changes host elements.+- Do **not** import `@astryxdesign/core/tailwind-theme.css`, and do not change+ the host's existing token definitions.+- Keep `@import 'tailwindcss';` exactly where it is. Declare the layer order+ above it so the design system's layers sit between the host's base and the+ host's components:++ ```css+ @layer reset, theme, base, astryx-base, astryx-theme, components, utilities;+ ```++- Verify the layer order in the **emitted** CSS rather than in source order.++### Installing the packages — approving the build script++`@astryxdesign/core` ships a `postinstall` script, and pnpm 11 does not run a+dependency's install scripts until that dependency is approved. It fails the+install rather than skipping quietly:++```+[ERR_PNPM_IGNORED_BUILDS] Ignored build scripts: @astryxdesign/[email protected]+Run "pnpm approve-builds" to pick which dependencies should be allowed to run scripts.+```++Do what the error says:++```bash+pnpm install # fails with ERR_PNPM_IGNORED_BUILDS+pnpm approve-builds # --all to accept every pending package without prompting+```++That writes the approval into **`pnpm-workspace.yaml`**, as an `allowBuilds` map:++```yaml+allowBuilds:+ '@astryxdesign/core': true+```++Two things about this are worth knowing before you hand-write anything:++- **pnpm 11 reads `allowBuilds` from `pnpm-workspace.yaml` only.** The same key+ under a `pnpm` field in `package.json` is ignored, and the install fails+ exactly as it did before — a confusing failure, because the setting is right+ there in the file you edited. If you write it by hand rather than running+ `approve-builds`, write it in `pnpm-workspace.yaml`, and create that file if+ the project does not have one.+- **`ignoredBuiltDependencies` and `onlyBuiltDependencies` are not the fix+ here.** Neither key clears this error in pnpm 11 in either file. Reach for+ `approve-builds`, or the `allowBuilds` map it writes.++Setting a package to `false` also satisfies pnpm — it records a decision not to+build — but `@astryxdesign/core` expects its `postinstall` to run, so approve it+rather than suppressing it.++Prefer `pnpm-workspace.yaml` over `package.json` for another reason too: it is+pnpm's own file and npm and yarn ignore it, so approving a pnpm build leaves no+pnpm-specific configuration in the manifest those tools read.++### Mount the provider around the guest subtree only++Wrap only the new or migrated region:++```tsx+<Theme theme={neutralTheme} mode={hostMode}>+ <NewRegion />+</Theme>+```++Do not wrap the application root, and do not wrap any pre-existing host UI.+`mode` follows the host's own light/dark state; the host keeps its mode control.++### Containing the root attribute sync — the honest state of this++A provider with no parent provider above it is a **root** provider, and a root+provider synchronizes `data-theme` and `data-astryx-theme` onto the document+element. Theme CSS is emitted as `@scope ([data-astryx-theme="<name>"]) to+(...)`, so that attribute on the document element makes the whole document a+scope root — which is exactly what defeats containment.++**There is no supported public API to disable that sync.** The provider's public+props are `theme`, `mode`, and `children`; none of them opt out, and there is no+nested-mode or scoped-root escape hatch. Do not invent a prop or reach into+internals.++The only mechanism available today is to remove the attribute from the document+element after mount and keep it removed with a `MutationObserver`:++```tsx+useEffect(() => {+ const root = document.documentElement;+ const strip = () => root.removeAttribute('data-astryx-theme');+ strip();+ const observer = new MutationObserver(strip);+ observer.observe(root, {+ attributes: true,+ attributeFilter: ['data-astryx-theme'],+ });+ return () => observer.disconnect();+}, []);+```++State plainly in the code comment that this is a workaround for a missing+capability, not a supported configuration. Leave `data-theme` alone if the host+relies on it for `color-scheme`; strip only the theme-name attribute.++### The portal tradeoff — test it and write it down++`data-astryx-theme` on the document element is what lets `@scope`'d theme CSS+reach content rendered **outside** the provider's DOM subtree: portaled+overlays, dialogs, popovers, tooltips, and fallback toast viewports. Removing it+is therefore not free:++- content inside the provider's subtree stays themed;+- portaled design-system content rendered to `document.body` falls outside every+ scope root and loses its theme tokens.++You must exercise this directly: open a portaled design-system overlay from+inside the guest subtree in both modes, observe the result, and record what+happens in a comment next to the workaround. If the region needs portaled+overlays, say so explicitly — for that region, containment and themed portals+cannot both hold today, and the choice belongs to the app.++Do not resolve this by re-adding a global attribute under another name, by+copying theme variables onto `body`, or by adding `!important` overrides.diff --git a/internal/vibe-tests/setup-test/guidance/host-aligned.md b/internal/vibe-tests/setup-test/guidance/host-aligned.mdnew file mode 100644index 0000000000..1b1125e23a--- /dev/null+++ b/internal/vibe-tests/setup-test/guidance/host-aligned.md@@ -0,0 +1,188 @@+## Strategy: host-aligned adoption++This app already has an established visual language. Adopt this strategy when the+host application stays authoritative over typography and color and the design+system is expected to disappear into it.++This is the normal objective for an established app, and the whole recipe below+is supported: an app-owned theme built from the published theme APIs, a provider+used as documented, no unsupported behavior anywhere in it. Its cost is real and+is stated rather than hidden — an app-owned theme has to be maintained as the+host's own tokens change, and it deliberately gives up the design system's stock+visual identity — but that is a maintenance cost, not a missing capability.++The other documented strategy, guest-contained adoption, is not an alternative+to this one for an app that simply wants to look like itself. It exists for the+narrower case where containing the design system to one subtree is a hard+requirement, and it is described in its own document.++### Before you start — the four rules that are not judgement calls++Everything else in this document is reasoning you are expected to apply. These+four are not:++1. **Commit the generated theme output. Never add it to `.gitignore`.** A+ deterministic build step is a reason the output is reproducible, not a reason+ to stop tracking it. Detail below.+2. **Do not import a stock theme stylesheet**, and do not import+ `@astryxdesign/core/tailwind-theme.css`. Detail below.+3. **Leave `.gitignore` exactly as you found it.** If the build writes somewhere+ already ignored, change where it writes.+4. **If you move existing host UI into an Astryx overlay, the moved subtree must+ keep the host boundary that was scoping its tokens.** Not the token values —+ the boundary itself: the attribute, class, or id the host's own CSS keys on.+ Copying the colours instead is a defect, and it is measured. Detail below,+ including how to find that boundary.++### Own the theme; do not import the stock one++Do **not** import a stock theme stylesheet such as+`@astryxdesign/theme-neutral/theme.css`. Build an app-specific theme that+extends the neutral theme and overrides only what the host already defines:++- body, heading, and code/mono font families, matching the host's own stacks;+- the host's semantic colors, defined as explicit light and dark pairs so both+ modes resolve to the host's existing values.++Build that theme as a deterministic step of the app's own build — the same+inputs must always produce the same CSS — and import the generated stylesheet+and built theme object. Do not generate the theme at runtime on each boot.++### Commit the theme source **and** its generated output++Both the theme source and everything the build writes from it are project+files, and both belong in version control:++- commit the theme source (for example `src/hostTheme.ts`);+- commit the generated stylesheet and every sibling artifact the build emits+ beside it (for example `src/hostaligned.css` and its generated type and+ variant files).++Do **not** add the generated output to `.gitignore`. A deterministic build step+is a reason the output is reproducible, not a reason to stop tracking it: this+app's own history keeps its generated theme artifacts in the tree, a reviewer+has to be able to see the CSS a theme change actually produces, and a build that+regenerates the file on `predev`/`prebuild` still leaves the committed copy as+the reviewable record. Ignoring the output also hides a drifted or hand-edited+artifact, which is exactly what the committed copy exists to catch.++If the build writes the artifact somewhere that is already ignored, change where+it writes rather than what is ignored. Leave `.gitignore` as you found it.++### Keep the host's cascade++- Keep `@import 'tailwindcss';` where the host already has it, and declare the+ layer order **above** it so the emitted order is explicit:++ ```css+ @layer reset, theme, base, astryx-base, astryx-theme, components, utilities;++ @import 'tailwindcss';+ @import '@astryxdesign/core/reset.css';+ @import '@astryxdesign/core/astryx.css';+ /* the app's own generated theme, not a stock theme.css */+ @import './<theme-name>.css';+ ```++ The build names its output after the **theme**, not after the source file,+ and writes it beside that source: a theme named `hostaligned` declared in+ `src/hostTheme.ts` becomes `src/hostaligned.css`. Import that file.++- Verify the layer order in the **emitted** CSS, not in source order. A build+ step can reorder or hoist imports.+- Do **not** import `@astryxdesign/core/tailwind-theme.css`. It intentionally+ maps common color, radius, shadow, type, and spacing names and would hand the+ host's Tailwind vocabulary to the design system.++### Establish the theme attribute before first paint++`data-astryx-theme` must be present on the document element before the first+paint, not written by an effect after hydration. Otherwise the first painted+frame is unthemed and the page visibly reflows. Set it in the server-rendered+markup or in a blocking inline script in `<head>`.++Keep `data-theme` and `color-scheme` synchronized with the host's own mode+control. The host owns the light/dark switch; the design system follows it.+Never replace or disable the host's mode control.++### Bare element rules — know what the theme brings++A built theme always emits rules for bare prose elements — `h1`–`h6`, `p`,+`small`, `code`/`pre`, covering font family, size, weight, line height, and+color, plus `hr`, whose border it replaces outright. They are emitted whatever+the theme overrides; a theme that changes one color still carries the full+typographic scale. They land in `@layer reset` at zero specificity, so any host+class wins over them, but a host element with no class does not.++Because this strategy puts the theme attribute at document scope, those rules+reach the host's own bare elements. Before adopting this strategy, check what+the host actually renders unclassed. If any bare prose element's resolved font+family, size, weight, line height, letter spacing, or color would change — or+any `hr`'s border — the theme must restate the host's values for those tokens so+the result is pixel-identical. A near-match is host damage.++If the host's bare-element styling cannot be reproduced exactly — for example+because it varies by region — this strategy is not currently safe for that host,+and there is no supported way to suppress the emitted prose rules. Report that+rather than papering over it with a blanket override or `!important`.++### Host UI placed inside a design-system overlay++A task may ask for host markup to be composed _inside_ a design-system overlay —+the host's own menu opened from within a dialog, for example. Moving host markup+into an overlay moves it in the DOM, and an established app's surfaces are+routinely painted by custom properties scoped to a region of the DOM rather than+defined globally:++```css+[data-app-region='settings'] {+ --panel: #f5f3ff;+ --border: #c4b5fd;+}+```++A host surface that was a descendant of that region resolves those values. The+same markup rendered inside an overlay declared elsewhere is no longer a+descendant of it, so the _same classes_ resolve to the app's global values+instead and the surface silently changes colour. Promoting the element to the+top layer does not help: `popover` and `<dialog>` change where an element+paints, not where it inherits from.++So, when host markup moves into an overlay, **carry the boundary with the+markup**. Find the boundary, put it back on the moved subtree, and verify.++**Finding it.** Do not guess, and do not assume it is the same hook that carries+light/dark — an app very often has one attribute for mode and a different one+for a region's palette, and copying only the mode hook leaves the palette+behind. Work backwards from the tokens:++1. In the host's stylesheet, find where the surface's colours actually come+ from. Read the classes already on the element — `bg-panel`, `border-border`,+ `text-foreground` — and note the custom properties behind them.+2. Search the stylesheet for every rule that _redeclares_ those properties. One+ will be `:root` or equivalent; that is the global set. Any other is a region.+3. The selector on that region rule is the boundary. It is whatever this host+ calls it — an attribute, a class, an id.+4. Confirm the surface's original position is inside it, walking up from the+ element itself. The boundary is frequently on the element already, in which+ case moving the element moves the boundary with it — and dropping the+ attribute during the move is exactly the mistake.++**Applying it.** Put the same boundary on the moved subtree — the same+attribute, spelled the same way — so the host's own rules resolve as they did.+Carry the host's mode signal too if the host scopes light and dark separately.+Keep the host's markup, classes, and tokens otherwise unchanged. Restating a+colour as a literal is not the same fix: it duplicates a value the host can+change, and it drifts silently the next time the host retunes its palette.++**Verifying.** Compare the relocated surface's computed background, border, and+text colour against the same host surface in its original position, in **both**+modes. They must match exactly. A near-match is host damage. The measurement+checks the boundary as well as the colours, and reports them separately: a+`host-boundary` failure means the moved subtree fell out of the host's token+region, and no amount of restated colour will clear it.++### Check the rendered app++Build **and** view the app in both modes. Cascade, inherited color, and+font-metric changes all compile cleanly.diff --git a/internal/vibe-tests/setup-test/matrix.json b/internal/vibe-tests/setup-test/matrix.jsonindex 91b329c4cc..e586099993 100644--- a/internal/vibe-tests/setup-test/matrix.json+++ b/internal/vibe-tests/setup-test/matrix.json@@ -129,6 +129,152 @@"requiredRate": 1,"requireCompleteMatrix": true}+ },+ {+ "id": "strategy-pilot",+ "description": "Opt-in pilot of two established-app adoption strategies. Additive and exploratory: it does not replace any control, does not rank the two strategies against each other, and authorizes no default or core change. Coverage is chosen to characterize downstream implications rather than install-time only \u2014 the basic insertion task on all three fixtures, plus one cross-system composition task on each established fixture.",+ "fixtures": [+ "tailwind-v4-control",+ "shadcn-tailwind-v4-established",+ "enterprise-scoped-synthetic"+ ],+ "conditions": ["host-aligned", "guest-contained"],+ "prompts": ["s1", "s4", "s5"],+ "bundles": [+ "native-claude",+ "claude-code-claude",+ "native-gpt",+ "codex-gpt"+ ],+ "reps": 2,+ "acceptance": {+ "requiredVerdict": "clean",+ "requireTaskSuccess": true,+ "requireDiffAttestation": true,+ "requiredRate": 1,+ "requireCompleteMatrix": true+ }+ },+ {+ "id": "strategy-iteration",+ "description": "Reruns the four strategy-pilot cells that did not pass, under new condition ids, after the faults those cells exposed were corrected. Additive and exploratory on the same terms as the pilot: it replaces no control, ranks no strategy against another, and authorizes no default or core change. It does not restate the pilot's result \u2014 the pilot's ten cells keep their ids, measurements, and verdicts exactly as recorded, and this stage is read beside them, never over them.",+ "fixtures": [+ "shadcn-tailwind-v4-established",+ "enterprise-scoped-synthetic"+ ],+ "conditions": ["guest-contained-r2", "host-aligned-r2"],+ "prompts": ["s1", "s4", "s5"],+ "bundles": ["claude-code-claude"],+ "cells": [+ {+ "condition": "guest-contained-r2",+ "fixture": "enterprise-scoped-synthetic",+ "prompt": "s1"+ },+ {+ "condition": "guest-contained-r2",+ "fixture": "shadcn-tailwind-v4-established",+ "prompt": "s4"+ },+ {+ "condition": "host-aligned-r2",+ "fixture": "shadcn-tailwind-v4-established",+ "prompt": "s1"+ },+ {+ "condition": "host-aligned-r2",+ "fixture": "enterprise-scoped-synthetic",+ "prompt": "s5"+ }+ ],+ "reps": 1,+ "comparisonMapping": {+ "$comment": "Each rerun cell against the pilot cell it reruns, and what was changed for it. A rerun that passes retires the finding named here; a rerun that fails leaves the pilot's failure standing and says the correction was wrong, not that the cell is flaky.",+ "pairs": [+ {+ "iteration": "strategy-iteration__guest-contained-r2__enterprise-scoped-synthetic__s1__claude-code-claude__r1",+ "pilot": "strategy-pilot__guest-contained__enterprise-scoped-synthetic__s1__claude-code-claude__r1",+ "pilotFinding": "2 host-text regressions on guest-boundary, both schemes",+ "changed": "measurement",+ "expectation": "The mandated button's own closed tooltip no longer counts as host container text, because the browser does not render it. Host copy that is removed or rewritten still counts."+ },+ {+ "iteration": "strategy-iteration__guest-contained-r2__shadcn-tailwind-v4-established__s4__claude-code-claude__r1",+ "pilot": "strategy-pilot__guest-contained__shadcn-tailwind-v4-established__s4__claude-code-claude__r1",+ "pilotFinding": "1 host colour regression on primary-action, dark scheme only",+ "changed": "measurement",+ "expectation": "Probes are read after running CSS transitions settle, so an unchanged colour is no longer sampled mid-interpolation and reported in the interpolation colour space."+ },+ {+ "iteration": "strategy-iteration__host-aligned-r2__shadcn-tailwind-v4-established__s1__claude-code-claude__r1",+ "pilot": "strategy-pilot__host-aligned__shadcn-tailwind-v4-established__s1__claude-code-claude__r1",+ "pilotFinding": "integrity: gitignore-modified, with no measured host damage and a completed task",+ "changed": "guidance delivery",+ "expectation": "The rule against ignoring generated theme output is present in the sandbox this time, and stated before the reasoning. The prior revision never delivered it, so this is its first test rather than a second attempt."+ },+ {+ "iteration": "strategy-iteration__host-aligned-r2__enterprise-scoped-synthetic__s5__claude-code-claude__r1",+ "pilot": "strategy-pilot__host-aligned__enterprise-scoped-synthetic__s5__claude-code-claude__r1",+ "pilotFinding": "integrity: gitignore-modified, plus host-menu-surface:host-style in both schemes",+ "changed": "guidance",+ "expectation": "Both the tracking rule and the rule that relocated host markup carries the host's own token-scoping hook are delivered. The style finding is genuine and is reproducibly avoidable: the guest-contained cell on the same fixture, prompt, and contract matched the host surface exactly by carrying that hook."+ }+ ]+ },+ "acceptance": {+ "requiredVerdict": "clean",+ "requireTaskSuccess": true,+ "requireDiffAttestation": true,+ "requiredRate": 1,+ "requireCompleteMatrix": true+ }+ },+ {+ "id": "strategy-iteration-2",+ "description": "Reruns the two strategy-iteration cells that did not pass, under new condition ids, after the faults those cells exposed were corrected. Additive on the same terms as every stage before it: it replaces no control, ranks no strategy against another, and authorizes no default or core change. The strategy-pilot and strategy-iteration cells keep their ids, measurements, and verdicts, and this stage is read beside them.",+ "fixtures": ["enterprise-scoped-synthetic"],+ "conditions": ["guest-contained-r3", "host-aligned-r3"],+ "prompts": ["s1", "s5"],+ "bundles": ["claude-code-claude"],+ "cells": [+ {+ "condition": "guest-contained-r3",+ "fixture": "enterprise-scoped-synthetic",+ "prompt": "s1"+ },+ {+ "condition": "host-aligned-r3",+ "fixture": "enterprise-scoped-synthetic",+ "prompt": "s5"+ }+ ],+ "reps": 1,+ "comparisonMapping": {+ "$comment": "Each rerun cell against the cell it reruns, and what was changed for it. A rerun that passes retires the finding named here; a rerun that fails leaves the earlier failure standing and says the correction was wrong, not that the cell is flaky.",+ "pairs": [+ {+ "iteration": "strategy-iteration-2__guest-contained-r3__enterprise-scoped-synthetic__s1__claude-code-claude__r1",+ "predecessor": "strategy-iteration__guest-contained-r2__enterprise-scoped-synthetic__s1__claude-code-claude__r1",+ "priorFinding": "build-failed: ERR_PNPM_IGNORED_BUILDS. The executor wrote the build approval into package.json, which pnpm 11 does not read, so the install aborted and nothing about the host was measured.",+ "changed": "guidance",+ "expectation": "The guest-contained document now names the mechanism pnpm 11 actually reads -- pnpm approve-builds, writing an allowBuilds map into pnpm-workspace.yaml -- and says plainly that the same key in package.json is ignored. The install and build succeed, and the cell is measured on the strategy for the first time."+ },+ {+ "iteration": "strategy-iteration-2__host-aligned-r3__enterprise-scoped-synthetic__s5__claude-code-claude__r1",+ "predecessor": "strategy-iteration__host-aligned-r2__enterprise-scoped-synthetic__s5__claude-code-claude__r1",+ "priorFinding": "host-menu-surface:host-style in both modes. The executor carried the host's fixture-shell and data-mode hooks onto the relocated menu but not the attribute scoping its palette, so the menu left the host token region and fell back to global values.",+ "changed": "guidance and measurement",+ "expectation": "The relocation rule is now a stated non-negotiable carrying a procedure for finding the boundary, and the measurement records which host token scopes a surface is inside, so a lost boundary reports as host-boundary rather than only as changed colours. The relocated menu stays inside the host's token region and matches its reference exactly."+ }+ ]+ },+ "acceptance": {+ "requiredVerdict": "clean",+ "requireTaskSuccess": true,+ "requireDiffAttestation": true,+ "requiredRate": 1,+ "requireCompleteMatrix": true+ }}],"deferred": [diff --git a/internal/vibe-tests/setup-test/prompts.json b/internal/vibe-tests/setup-test/prompts.jsonindex 868c40c999..aa90485dfb 100644--- a/internal/vibe-tests/setup-test/prompts.json+++ b/internal/vibe-tests/setup-test/prompts.json@@ -352,6 +352,12 @@"probe": "host-shell","fields": ["height", "geometry.height", "geometry.bottom"],"textInsertionOnly": true+ },+ {+ "$comment": "The mandated trigger belongs in the guest subtree, which is where this fixture says guest design-system content goes, so the boundary grows by exactly that control and the growth carries into the two geometry fields height moves. The trigger is task-owned, so the boundary's protected text does not change and no text exemption is given. Its position, width, every computed style, and every other probe stay strict.",+ "fixture": "enterprise-scoped-synthetic",+ "probe": "guest-boundary",+ "fields": ["height", "geometry.height", "geometry.bottom"]}],"replacedHostProbes": [],diff --git a/internal/vibe-tests/setup-test/run-setup.mjs b/internal/vibe-tests/setup-test/run-setup.mjsindex 3d32f51330..711919d7c4 100644--- a/internal/vibe-tests/setup-test/run-setup.mjs+++ b/internal/vibe-tests/setup-test/run-setup.mjs@@ -19,6 +19,7 @@ import {createSetupProvenance,expandSetupMatrix,fixtureManifestSha256,+ setupEnvironmentHash,sha256Text,taskContractText,validatePromptContracts,@@ -144,13 +145,27 @@ function appendGuidance(sandboxDir, guidanceFile) {);}-const PATCHES = {- 'patch:pointer': sandboxDir => appendGuidance(sandboxDir, 'pointer.md'),- 'patch:existing-app': sandboxDir =>- appendGuidance(sandboxDir, 'existing-app.md'),- 'patch:directed': sandboxDir => appendGuidance(sandboxDir, 'directed.md'),+const PATCH_FILES = {+ 'patch:pointer': 'pointer.md',+ 'patch:existing-app': 'existing-app.md',+ 'patch:directed': 'directed.md',+ 'patch:host-aligned': 'host-aligned.md',+ 'patch:guest-contained': 'guest-contained.md',};+function guidanceFileFor(patch) {+ const file = PATCH_FILES[patch];+ if (!file) throw new Error(`unknown patch: ${patch}`);+ return file;+}++const PATCHES = Object.fromEntries(+ Object.entries(PATCH_FILES).map(([patch, file]) => [+ patch,+ sandboxDir => appendGuidance(sandboxDir, file),+ ]),+);+function taskPrompt(prompt, fixtureId) {return `You are working in an existing application.@@ -273,25 +288,14 @@ for (const entry of entries) {const task = taskPrompt(prompt, entry.fixture);const recipe = readRecipe(entry.fixture);const fixtureSha256 = fixtureManifestSha256(recipe);- const guidanceHash = sha256Text(- JSON.stringify({- fixtureSha256,- condition: entry.condition,- patches: (condition.patches ?? []).map(patch => [- patch,- read(- path.join(- GUIDANCE,- patch === 'patch:pointer'- ? 'pointer.md'- : patch === 'patch:existing-app'- ? 'existing-app.md'- : 'directed.md',- ),- ),- ]),- }),- );+ const guidanceHash = setupEnvironmentHash({+ fixtureSha256,+ condition: entry.condition,+ patches: (condition.patches ?? []).map(patch => [+ patch,+ read(path.join(GUIDANCE, guidanceFileFor(patch))),+ ]),+ });const provenance = createSetupProvenance({entry,taskSha256: sha256Text(task),diff --git a/internal/vibe-tests/setup-test/setup-aggregate.test.ts b/internal/vibe-tests/setup-test/setup-aggregate.test.tsindex 1d0aecf143..f8b64723be 100644--- a/internal/vibe-tests/setup-test/setup-aggregate.test.ts+++ b/internal/vibe-tests/setup-test/setup-aggregate.test.ts@@ -4,6 +4,7 @@ import {describe, expect, it} from 'vitest';import {assertAggregateReportPublicSafe,compareGuidanceRows,+ compareStrategyRows,matrixCoverage,type SetupAggregateRow,} from './setup-aggregate.js';@@ -139,3 +140,203 @@ describe('setup aggregation policy', () => {).not.toThrow();});});++describe('strategy pilot reporting', () => {+ function strategyRow(+ condition: 'host-aligned' | 'guest-contained',+ value: SetupScore,+ fixture = 'shadcn-tailwind-v4-established',+ prompt = 's1',+ ): SetupAggregateRow {+ return {+ id: `strategy-pilot__${condition}__${fixture}__${prompt}__bundle__r1`,+ file: `${condition}-${fixture}-${prompt}.json`,+ stage: 'strategy-pilot',+ fixture,+ prompt,+ condition,+ bundle: 'bundle',+ executor: 'harness/model',+ rep: 1,+ score: value,+ };+ }++ it('reports each strategy separately and never combines them', () => {+ const damaged = score([regression('fontFamily', 'typography')]);+ const breakdowns = compareStrategyRows(+ [+ strategyRow('host-aligned', score()),+ strategyRow('guest-contained', damaged),+ ],+ ['host-aligned', 'guest-contained'],+ );++ expect(breakdowns.map(entry => entry.condition)).toEqual([+ 'host-aligned',+ 'guest-contained',+ ]);+ expect(breakdowns[0]).toMatchObject({present: 1, accepted: 1, clean: 1});+ expect(breakdowns[1]).toMatchObject({present: 1, accepted: 0});+ // No combined figure, ranking, or winner is produced.+ expect(Object.keys(breakdowns[0])).not.toContain('score');+ expect(Object.keys(breakdowns[0])).not.toContain('rank');+ expect(Object.keys(breakdowns[0])).not.toContain('winner');+ });++ it('attributes failures to a named category rather than one opaque total', () => {+ const [breakdown] = compareStrategyRows(+ [+ strategyRow(+ 'guest-contained',+ score([+ regression('fontFamily', 'typography'),+ regression('color', 'color'),+ ]),+ ),+ ],+ ['guest-contained'],+ );+ expect(breakdown.byCategory.font).toBe(1);+ expect(breakdown.byCategory.color).toBe(1);+ expect(breakdown.byCategory.radius).toBe(0);+ // The categories are reported individually, so a reader can see which+ // dimension failed instead of only how many failures there were.+ const named = Object.entries(breakdown.byCategory).filter(+ ([, count]) => count > 0,+ );+ expect(named.map(([name]) => name).sort()).toEqual(['color', 'font']);+ });++ it('breaks results down by fixture-prompt pair', () => {+ const breakdowns = compareStrategyRows(+ [+ strategyRow('host-aligned', score()),+ strategyRow(+ 'host-aligned',+ score([regression('color', 'color')]),+ 'enterprise-scoped-synthetic',+ 's5',+ ),+ ],+ ['host-aligned'],+ );+ expect(breakdowns[0].byPair).toEqual([+ {pair: 'enterprise-scoped-synthetic/s5', present: 1, accepted: 0},+ {pair: 'shadcn-tailwind-v4-established/s1', present: 1, accepted: 1},+ ]);+ });++ it('keeps acceptance strict: any damage fails the strategy cell', () => {+ const [breakdown] = compareStrategyRows(+ [+ strategyRow('host-aligned', score()),+ strategyRow('host-aligned', score([regression('color', 'color')])),+ ],+ ['host-aligned'],+ );+ expect(breakdown.present).toBe(2);+ expect(breakdown.accepted).toBe(1);+ // Final acceptance is unchanged by the pilot: 100% clean or it fails.+ expect(breakdown.accepted).toBeLessThan(breakdown.present);+ });++ it('reports a strategy with no rows as empty rather than passing', () => {+ const [breakdown] = compareStrategyRows([], ['host-aligned']);+ expect(breakdown).toMatchObject({present: 0, accepted: 0, clean: 0});+ expect(breakdown.byPair).toEqual([]);+ });++ /**+ * The four failure kinds the pilot conflated.+ *+ * `verdict` answers one question — is this run acceptable — and answers it+ * `silent-damage` for a repainted host, for an escape hatch, and for a task+ * the executor did not finish. Those are different problems with different+ * next steps, and the report has to say which one happened.+ */+ it('reports each failure kind apart from the verdict', () => {+ // `verdict` answers one question — is this run acceptable — and answers it+ // `silent-damage` for a repainted host, for an escape hatch, and for a task+ // the executor did not finish. Those are different problems with different+ // next steps, and the report has to say which one happened. The old+ // category vector says only `taskCompletion=3`, which reads as three task+ // failures.+ const damaged = score([regression('color', 'color')]);+ const mixed = score();+ mixed.taskSuccess = false;+ mixed.taskFailures = ['light:host-menu:host-style'];+ mixed.integrityFailures = ['gitignore-modified'];+ mixed.executionSucceeded = false;++ const [damage, other] = compareStrategyRows(+ [+ strategyRow('host-aligned', damaged),+ strategyRow('guest-contained', mixed),+ ],+ ['host-aligned', 'guest-contained'],+ );++ expect(damage.byFailureKind).toMatchObject({hostDamage: 1, integrity: 0});+ expect(other.byCategory.taskCompletion).toBe(3);+ expect(other.byFailureKind).toMatchObject({+ task: 1,+ integrity: 1,+ telemetry: 1,+ hostDamage: 0,+ });+ // Both are unacceptable, and only one of them is the host changing.+ expect(damage.accepted).toBe(0);+ expect(other.accepted).toBe(0);+ });++ /**+ * The exact shape the pilot produced: `host-aligned` measured `clean` on a+ * cell whose executor did not finish. Nothing damaged the host because the+ * work stopped, and reading that as a clean run makes the strategy look+ * better than the evidence supports.+ */+ describe('a run the executor did not complete is not evidence', () => {+ function agentFailure() {+ const value = score();+ value.executionSucceeded = false;+ return value;+ }++ it('excludes it from the clean count and names it separately', () => {+ const [breakdown] = compareStrategyRows(+ [strategyRow('host-aligned', agentFailure())],+ ['host-aligned'],+ );++ expect(breakdown.present).toBe(1);+ // Its verdict is `clean`; the completed-run count is what stops that+ // reading as evidence for the strategy.+ expect(breakdown.byVerdict.clean).toBe(1);+ expect(breakdown.clean).toBe(0);+ expect(breakdown.verdictCleanNotCompleted).toBe(1);+ expect(breakdown.comparable).toBe(0);+ expect(breakdown.notCompleted).toBe(1);+ expect(breakdown.accepted).toBe(0);+ });++ it('still counts a completed clean run as clean', () => {+ const [breakdown] = compareStrategyRows(+ [strategyRow('host-aligned', score())],+ ['host-aligned'],+ );+ expect(breakdown.clean).toBe(1);+ expect(breakdown.comparable).toBe(1);+ expect(breakdown.notCompleted).toBe(0);+ expect(breakdown.verdictCleanNotCompleted).toBe(0);+ });+ });++ it('emits a public-safe strategy report', () => {+ const breakdowns = compareStrategyRows(+ [strategyRow('host-aligned', score())],+ ['host-aligned'],+ );+ expect(() => assertAggregateReportPublicSafe(breakdowns)).not.toThrow();+ });+});diff --git a/internal/vibe-tests/setup-test/setup-aggregate.ts b/internal/vibe-tests/setup-test/setup-aggregate.tsindex 479306a586..cd852b27d1 100644--- a/internal/vibe-tests/setup-test/setup-aggregate.ts+++ b/internal/vibe-tests/setup-test/setup-aggregate.ts@@ -8,12 +8,18 @@ import {fileURLToPath} from 'node:url';import {loadOptionalExecutionProvenance} from '../src/provenance-aggregation.js';import {compareCandidate,+ failureBreakdown,failureCauses,+ hardGateVector,+ isComparableRun,passesAcceptance,scoreArm,strictAcceptanceSummary,verdict,+ type FailureBreakdown,+ type FailureKind,type HardDimension,+ type HardGateVector,type Measurement,type SetupScore,} from './setup-eval.js';@@ -144,6 +150,114 @@ export function compareGuidanceRows(rows: SetupAggregateRow[]) {};}+export type StrategyBreakdown = {+ condition: string;+ present: number;+ /**+ * Runs whose verdict is `clean` **and** whose executor completed. A verdict+ * on an unfinished run describes an unfinished run, so it is not counted+ * here; `verdictCleanNotCompleted` carries those separately.+ */+ clean: number;+ verdictCleanNotCompleted: number;+ comparable: number;+ notCompleted: number;+ accepted: number;+ byVerdict: Record<string, number>;+ byCategory: HardGateVector;+ byFailureKind: FailureBreakdown;+ byPair: Array<{pair: string; present: number; accepted: number}>;+};++/**+ * Per-strategy, per-failure-category accounting for the strategy pilot.+ *+ * Deliberately returns separate counts rather than one combined figure: the+ * pilot compares two strategies that make different tradeoffs, and collapsing+ * them into a single score would hide which category actually failed. No+ * ranking, weighting, or advance/reject decision is produced here.+ *+ * Two separations matter and are both kept:+ *+ * 1. **What failed.** `byFailureKind` splits measured host damage from an+ * integrity/escape-hatch failure, from a task failure, from a telemetry or+ * run-note failure. The verdict alone calls several of those `silent-damage`+ * and reads as if the host was repainted.+ * 2. **Whether the run finished.** A strategy is never called cleaner on the+ * strength of a run its executor did not complete. Those runs are reported+ * under `notCompleted` and excluded from `clean` and `comparable`.+ */+export function compareStrategyRows(+ rows: SetupAggregateRow[],+ conditions: string[],+): StrategyBreakdown[] {+ return conditions.map(condition => {+ const scoped = rows.filter(row => row.condition === condition);+ const byVerdict: Record<string, number> = {};+ const byCategory = {+ build: 0,+ runtime: 0,+ taskCompletion: 0,+ color: 0,+ font: 0,+ radius: 0,+ border: 0,+ shadow: 0,+ geometry: 0,+ contrast: 0,+ layering: 0,+ } as HardGateVector;+ const byFailureKind: FailureBreakdown = {+ hostDamage: 0,+ runtime: 0,+ integrity: 0,+ task: 0,+ telemetry: 0,+ };+ const pairs = new Map<string, {present: number; accepted: number}>();++ for (const row of scoped) {+ const rowVerdict = verdict(row.score);+ byVerdict[rowVerdict] = (byVerdict[rowVerdict] ?? 0) + 1;+ const vector = hardGateVector(row.score);+ for (const dimension of Object.keys(byCategory) as HardDimension[]) {+ byCategory[dimension] += vector[dimension];+ }+ const kinds = failureBreakdown(row.score);+ for (const kind of Object.keys(byFailureKind) as FailureKind[]) {+ byFailureKind[kind] += kinds[kind];+ }+ const pairKey = `${row.fixture}/${row.prompt}`;+ const pair = pairs.get(pairKey) ?? {present: 0, accepted: 0};+ pair.present += 1;+ if (passesAcceptance(row.score)) {+ pair.accepted += 1;+ }+ pairs.set(pairKey, pair);+ }++ return {+ condition,+ present: scoped.length,+ clean: scoped.filter(+ row => verdict(row.score) === 'clean' && isComparableRun(row.score),+ ).length,+ verdictCleanNotCompleted: scoped.filter(+ row => verdict(row.score) === 'clean' && !isComparableRun(row.score),+ ).length,+ comparable: scoped.filter(row => isComparableRun(row.score)).length,+ notCompleted: scoped.filter(row => !isComparableRun(row.score)).length,+ accepted: scoped.filter(row => passesAcceptance(row.score)).length,+ byVerdict,+ byCategory,+ byFailureKind,+ byPair: [...pairs]+ .map(([pair, counts]) => ({pair, ...counts}))+ .sort((left, right) => left.pair.localeCompare(right.pair)),+ };+ });+}+function renderTable(rows: SetupAggregateRow[]) {const header = ['stage',@@ -341,6 +455,102 @@ async function main() {);}+ const isIteration = stageId.startsWith('strategy-iteration');+ if (stageId === 'strategy-pilot' || isIteration) {+ const pilotConditions = [+ ...new Set(expectedEntries.map(entry => entry.condition)),+ ].sort();+ console.log('\nBy strategy — acceptance and failure categories:');+ for (const breakdown of compareStrategyRows(rows, pilotConditions)) {+ const verdicts = Object.entries(breakdown.byVerdict)+ .sort(([left], [right]) => left.localeCompare(right))+ .map(([name, count]) => `${name}=${count}`)+ .join(' ');+ const categories = Object.entries(breakdown.byCategory)+ .filter(([, count]) => count > 0)+ .map(([category, count]) => `${category}=${count}`)+ .join(' ');+ const kinds = Object.entries(breakdown.byFailureKind)+ .filter(([, count]) => count > 0)+ .map(([kind, count]) => `${kind}=${count}`)+ .join(' ');+ console.log(`\n ${breakdown.condition}`);+ console.log(+ ` accepted ${breakdown.accepted}/${breakdown.present} present` ++ ` (clean ${breakdown.clean} of ${breakdown.comparable} completed runs)`,+ );+ if (breakdown.notCompleted > 0) {+ console.log(+ ` ${breakdown.notCompleted} run(s) the executor did not complete` ++ ` — excluded from the clean count` ++ (breakdown.verdictCleanNotCompleted > 0+ ? `, ${breakdown.verdictCleanNotCompleted} of which measured` ++ ` 'clean' on unfinished work`+ : '') ++ '. Those cells need rerunning; they are not evidence for this' ++ ' strategy.',+ );+ }+ console.log(` verdicts: ${verdicts || 'none'}`);+ console.log(` failure kinds: ${kinds || 'none'}`);+ console.log(` failure categories: ${categories || 'none'}`);+ for (const pair of breakdown.byPair) {+ console.log(+ ` ${pair.pair}: accepted ${pair.accepted}/${pair.present}`,+ );+ }+ }+ console.log(+ '\nThe two strategies are reported separately and are not ranked against' ++ ' each other. They make different tradeoffs, so no combined score is' ++ ' emitted. Final acceptance is unchanged: every valid run must be clean.',+ );+ console.log(+ 'Failure kinds are reported apart from the verdict on purpose. Measured' ++ ' host damage, an integrity or escape-hatch failure, a task failure,' ++ ' and a telemetry or run-note failure are different problems that the' ++ ' verdict alone spells the same way, and only the first is the host' ++ ' being changed. A run the executor did not complete is never counted' ++ ' as a clean run for its strategy, whatever its measurement says.',+ );+ if (isIteration) {+ const pairs = (+ stage as {+ comparisonMapping?: {+ pairs?: Array<{+ iteration: string;+ pilot?: string;+ predecessor?: string;+ pilotFinding?: string;+ priorFinding?: string;+ changed: string;+ }>;+ };+ }+ ).comparisonMapping?.pairs;+ if (pairs?.length) {+ console.log(+ '\nEach cell below reruns an earlier cell under a new condition id.' ++ ' Those earlier cells are unchanged and remain the record of what' ++ " they measured; this stage's result is read beside them, not in" ++ ' place of them.',+ );+ for (const pair of pairs) {+ console.log(`\n ${pair.iteration}`);+ console.log(` reruns ${pair.predecessor ?? pair.pilot}`);+ console.log(` finding ${pair.priorFinding ?? pair.pilotFinding}`);+ console.log(` changed ${pair.changed}`);+ }+ }+ }+ console.log(+ `${isIteration ? 'Strategy iteration' : 'Strategy pilot'}: ${completeCoverage && summary.passes ? 'PASS' : 'FAIL'}`,+ );+ if (!completeCoverage || !summary.passes) {+ process.exitCode = 1;+ }+ }+if (stageId === 'confirmation') {console.log(`Final confirmation: ${completeCoverage && summary.passes ? 'PASS' : 'FAIL'}`,diff --git a/internal/vibe-tests/setup-test/setup-canonical.test.ts b/internal/vibe-tests/setup-test/setup-canonical.test.tsindex 3ba0e083bc..62390d57fb 100644--- a/internal/vibe-tests/setup-test/setup-canonical.test.ts+++ b/internal/vibe-tests/setup-test/setup-canonical.test.ts@@ -1,6 +1,6 @@// Copyright (c) Meta Platforms, Inc. and affiliates.-import {execFileSync} from 'node:child_process';+import {execFileSync, spawnSync} from 'node:child_process';import fs from 'node:fs';import os from 'node:os';import path from 'node:path';@@ -33,6 +33,21 @@ const RUN_CANONICAL = process.env.ASTRYX_CANONICAL_SETUP_BROWSER === '1';const temporaryDirectories: string[] = [];const dependencyRoots = new Map<string, string>();+/**+ * The single run directory `run-setup.mjs` writes beside its config files.+ *+ * Named rather than indexed: the output directory also holds+ * `setup-config-<id>.json` and `setup-matrix-<id>.json`, and the run directory+ * is the one entry without a dot in it.+ */+function runDirectory(root: string): string {+ const entry = fs.readdirSync(root).find(name => !name.includes('.'));+ if (!entry) {+ throw new Error(`no prepared run directory under ${root}`);+ }+ return entry;+}+function run(command: string, args: string[], cwd: string) {return execFileSync(command, args, {cwd,@@ -154,6 +169,168 @@ function installAstryxCss(arm: string) {);}+/**+ * Canonical `host-aligned` strategy install.+ *+ * The established host stays authoritative: the app owns a theme that extends+ * neutral and restates the host's own fonts and semantic colors, instead of+ * importing the stock theme stylesheet. The theme is compiled by a+ * deterministic build step, and the theme attribute is present before first+ * paint rather than written by an effect.+ */+function installHostAlignedStrategy(arm: string) {+ fs.writeFileSync(+ path.join(arm, 'src', 'hostTheme.ts'),+ `// Copyright (c) Meta Platforms, Inc. and affiliates.++import {defineTheme} from '@astryxdesign/core/theme';+import {neutralTheme} from '@astryxdesign/theme-neutral';++// Restates the host's own vocabulary. Values are the host's, not the design+// system's: every pair is [light, dark] so both modes resolve to host values.+export const hostAlignedTheme = defineTheme({+ name: 'hostaligned',+ extends: neutralTheme,+ tokens: {+ '--font-family-body': 'ui-sans-serif, system-ui, sans-serif',+ '--font-family-heading': 'ui-sans-serif, system-ui, sans-serif',+ '--font-family-code': 'ui-monospace, monospace',+ },+});+`,+ );++ // Deterministic build step: same input, same emitted CSS, every run.+ run(+ process.execPath,+ [+ path.join(REPO_ROOT, 'packages/cli/clients/cli/bin/astryx.mjs'),+ 'theme',+ 'build',+ path.join('src', 'hostTheme.ts'),+ ],+ arm,+ );++ edit(path.join(arm, 'src', 'index.css'), source =>+ source.replace(+ "@import 'tailwindcss';",+ [+ '@layer reset, theme, base, astryx-base, astryx-theme, components, utilities;',+ '',+ "@import 'tailwindcss';",+ "@import '@astryxdesign/core/reset.css';",+ "@import '@astryxdesign/core/astryx.css';",+ // The app's own generated theme — never the stock theme.css, and never+ // tailwind-theme.css, which would take over the host's vocabulary.+ "@import './hostaligned.css';",+ ].join('\n'),+ ),+ );++ // Establish the theme attribute before first paint.+ edit(path.join(arm, 'index.html'), source =>+ source.replace(+ '<html lang="en"',+ '<html lang="en" data-astryx-theme="hostaligned"',+ ),+ );++ // The provider follows the host's own mode control rather than replacing it,+ // so it mounts where that state lives.+ edit(path.join(arm, 'src', 'App.tsx'), source =>+ source+ .replace(+ "import {createPortal} from 'react-dom';",+ [+ "import {createPortal} from 'react-dom';",+ "import {Theme} from '@astryxdesign/core/theme';",+ "import {hostalignedTheme} from './hostaligned.js';",+ ].join('\n'),+ )+ .replace(+ " return (\n <div className={dark ? 'dark' : ''}>",+ " return (\n <Theme theme={hostalignedTheme} mode={dark ? 'dark' : 'light'}>\n <div className={dark ? 'dark' : ''}>",+ )+ .replace(/\n {2}\);\n\}\n?$/, '\n </Theme>\n );\n}\n'),+ );+}++/**+ * Canonical `guest-contained` strategy install.+ *+ * The host keeps global preflight and tokens: no reset import, and the+ * provider wraps only the guest subtree. The root attribute sync is contained+ * with the only mechanism available today — see the portal tradeoff below.+ */+function installGuestContainedStrategy(arm: string) {+ edit(path.join(arm, 'src', 'index.css'), source =>+ source.replace(+ "@import 'tailwindcss';",+ [+ '@layer reset, theme, base, astryx-base, astryx-theme, components, utilities;',+ '',+ "@import 'tailwindcss';",+ // No reset.css: the host already owns preflight and element defaults.+ "@import '@astryxdesign/core/astryx.css';",+ "@import '@astryxdesign/theme-neutral/theme.css';",+ ].join('\n'),+ ),+ );++ fs.writeFileSync(+ path.join(arm, 'src', 'GuestRegion.tsx'),+ `// Copyright (c) Meta Platforms, Inc. and affiliates.++import {useEffect, type ReactNode} from 'react';+import {Theme} from '@astryxdesign/core/theme';+import {neutralTheme} from '@astryxdesign/theme-neutral';++/**+ * WORKAROUND, not a supported configuration.+ *+ * A provider with no provider above it is a root provider, and a root provider+ * syncs \`data-astryx-theme\` onto the document element so that @scope'd theme+ * CSS reaches portaled content. There is no public prop to opt out of that+ * sync, so containment requires removing the attribute and keeping it removed.+ *+ * TRADEOFF, measured by the strategy pilot: with the attribute removed, content+ * inside this subtree stays themed, and design-system content portaled to+ * document.body falls outside every scope root and loses its theme tokens.+ * \`data-theme\` is left alone because the host relies on it for color-scheme.+ */+function useContainedThemeScope() {+ useEffect(() => {+ const root = document.documentElement;+ const strip = () => root.removeAttribute('data-astryx-theme');+ strip();+ const observer = new MutationObserver(strip);+ observer.observe(root, {+ attributes: true,+ attributeFilter: ['data-astryx-theme'],+ });+ return () => observer.disconnect();+ }, []);+}++export function GuestRegion({+ children,+ mode,+}: {+ children: ReactNode;+ mode: 'light' | 'dark';+}) {+ useContainedThemeScope();+ return (+ <Theme theme={neutralTheme} mode={mode}>+ {children}+ </Theme>+ );+}+`,+ );+}+function applyButtonAddition(arm: string) {installAstryxCss(arm);edit(path.join(arm, 'src', 'App.tsx'), source =>@@ -747,12 +924,12 @@ describeCanonical('canonical intended changes', () => {);const configFile = fs.readdirSync(root)- .find(file => file.startsWith('setup-config-'));+ .find((file: string) => file.startsWith('setup-config-'));if (!configFile) {throw new Error('missing generated setup config');}const config = JSON.parse(- fs.readFileSync(path.join(root, configFile), 'utf8'),+ fs.readFileSync(path.join(root, String(configFile)), 'utf8'),);expect(() =>assertPublicArtifactSafe(config, {@@ -795,6 +972,152 @@ describeCanonical('canonical intended changes', () => {}}, 120_000);+ /**+ * Guidance has to reach the sandbox, not just exist in the repository.+ *+ * The strategy pilot's two `host-aligned` integrity failures were scored+ * against a rule that no sandbox they ran in contained: the rule was written+ * into the guidance after those runs, and the tests that covered it read the+ * guidance file rather than anything an executor was handed. Both halves+ * passed while the executors were being judged on an instruction that had+ * never been delivered.+ *+ * So this asserts the delivery. It prepares real cells and requires every+ * section heading of a strategy's guidance document to be present in the+ * `AGENTS.md` the executor will actually open, and requires the other+ * strategy's document not to be — a sandbox that quietly received both would+ * be handed contradictory instructions.+ */+ it('delivers every section of a strategy document into its own sandbox', () => {+ const root = fs.mkdtempSync(+ path.join(os.tmpdir(), 'setup-canonical-guidance-delivery-'),+ );+ temporaryDirectories.push(root);+ run(+ process.execPath,+ [RUN_SETUP, '--stage', 'strategy-iteration', '--out', root],+ REPO_ROOT,+ );++ const headings = (file: string) =>+ fs+ .readFileSync(path.join(HERE, 'guidance', file), 'utf8')+ .split('\n')+ .filter(line => line.startsWith('### '))+ .map(line => line.trim());++ const hostAligned = headings('host-aligned.md');+ const guestContained = headings('guest-contained.md');+ expect(hostAligned.length).toBeGreaterThan(3);+ expect(guestContained.length).toBeGreaterThan(2);++ const runRoot = path.join(root, runDirectory(root));+ const cells = fs+ .readdirSync(runRoot)+ .filter(entry => entry.startsWith('strategy-iteration__'));+ expect(cells).toHaveLength(4);++ for (const cell of cells) {+ const agents = fs.readFileSync(+ path.join(runRoot, cell, 'app', 'AGENTS.md'),+ 'utf8',+ );+ const isHostAligned = cell.includes('__host-aligned-r2__');+ const delivered = isHostAligned ? hostAligned : guestContained;+ const withheld = isHostAligned ? guestContained : hostAligned;++ for (const heading of delivered) {+ expect(agents, `${cell} is missing "${heading}"`).toContain(heading);+ }+ for (const heading of withheld.filter(+ entry => !delivered.includes(entry),+ )) {+ expect(agents, `${cell} wrongly carries "${heading}"`).not.toContain(+ heading,+ );+ }+ }+ }, 300_000);++ /**+ * The specific rules each rerun exists to test, in the sandbox.+ *+ * Named individually rather than left to the heading sweep above, because+ * these are the sentences the failures turned on and a reworded heading+ * should not be able to carry them away silently. A guidance file passing a+ * grep is not evidence that a rule reached an executor, so these read the+ * `AGENTS.md` a prepared cell actually carries.+ */+ it('hands each strategy the rules its own rerun turns on', () => {+ const prepared = (stage: string) => {+ const root = fs.mkdtempSync(+ path.join(os.tmpdir(), 'setup-canonical-rerun-rules-'),+ );+ temporaryDirectories.push(root);+ run(+ process.execPath,+ [RUN_SETUP, '--stage', stage, '--out', root],+ REPO_ROOT,+ );+ return path.join(root, runDirectory(root));+ };+ const runRoots = [+ prepared('strategy-iteration'),+ prepared('strategy-iteration-2'),+ ];+ const agentsFor = (condition: string) => {+ for (const runRoot of runRoots) {+ const cell = fs+ .readdirSync(runRoot)+ .find(entry => entry.includes(`__${condition}__`));+ if (cell) {+ return fs.readFileSync(+ path.join(runRoot, cell, 'app', 'AGENTS.md'),+ 'utf8',+ );+ }+ }+ throw new Error(`no prepared cell for ${condition}`);+ };++ // host-aligned-r2: the tracking rule, stated before the reasoning and+ // still stated in full further down. It was never delivered before.+ const hostR2 = agentsFor('host-aligned-r2');+ expect(hostR2).toContain('Never add it to `.gitignore`');+ expect(hostR2).toContain(+ 'Do **not** add the generated output to `.gitignore`',+ );+ expect(hostR2.indexOf('Never add it to `.gitignore`')).toBeLessThan(+ hostR2.indexOf('### Own the theme; do not import the stock one'),+ );++ // guest-contained-r3: the pnpm approval mechanism, which is what the r2+ // cell died on before anything could be measured.+ const guest = agentsFor('guest-contained-r3');+ expect(guest).toContain('ERR_PNPM_IGNORED_BUILDS');+ expect(guest).toContain('pnpm approve-builds');+ expect(guest.replace(/\s+/g, ' ')).toContain(+ 'pnpm 11 reads `allowBuilds` from `pnpm-workspace.yaml` only',+ );++ // host-aligned-r3: the boundary rule, stated up front this time, with the+ // procedure the r2 executor needed and did not have.+ const host = agentsFor('host-aligned-r3');+ const upFront = host.indexOf('the moved subtree must');+ expect(upFront).toBeGreaterThan(-1);+ expect(upFront).toBeLessThan(+ host.indexOf('### Own the theme; do not import the stock one'),+ );+ expect(host).toContain('### Host UI placed inside a design-system overlay');+ // Matched against reflowed text so a line wrap is not a test failure.+ expect(host.replace(/\s+/g, ' ')).toContain(+ '**carry the boundary with the markup**',+ );+ expect(host).toContain('do not assume it is the same hook');+ // Discovery, not the answer: the fixture's own selector is never handed over.+ expect(host).not.toContain('data-guest-design-system');+ }, 300_000);+/*** The measurer must not build in the sandbox it measures.*@@ -1305,4 +1628,552 @@ describeCanonical('canonical intended changes', () => {).toBe(true);expect(passesAcceptance(score)).toBe(false);}, 180_000);++ /**+ * The same insertion, in the container the task actually points at.+ *+ * `s5` mandates exactly one visible `astryx-dialog-trigger`, and this+ * fixture's guest subtree is the region it declares for guest design-system+ * content, so that is where a trigger belongs. Two independent reps put it+ * there and both grew the boundary from 296px to 409px in both schemes. The+ * contract allows exactly that: the boundary's height and the two geometry+ * fields height moves. It takes no text exemption — the trigger is+ * task-owned, so the boundary's protected text never sees it — and nothing+ * else about the boundary, or any other probe, may move.+ *+ * This case inserts its own trigger from the same 296px baseline, so its+ * growth is its own; what it shares with the reps is the three fields that+ * move and the two schemes they move in.+ */+ function insertGuestBoundaryDialogTrigger(arm: string) {+ edit(path.join(arm, 'src', 'App.tsx'), source => {+ const next = source+ .replace(+ ` data-vibe-probe="dialog-trigger"+ data-vibe-result="astryx-dialog-trigger"`,+ ` data-vibe-probe="dialog-trigger"`,+ )+ .replace(+ ' </aside>',+ ` <button+ className="mt-4 w-full rounded-md border border-border bg-panel px-3 py-2 text-sm font-medium"+ data-vibe-result="astryx-dialog-trigger"+ onClick={() => setDialogOpen(true)}+ type="button">+ Open service actions+ </button>+ </aside>`,+ );+ if (next === source) {+ throw new Error('guest-boundary trigger insertion did not apply');+ }+ return next;+ });+ }++ let guestBoundaryMeasurement: {+ baseline: Measurement;+ result: Measurement;+ } | null = null;++ function measureGuestBoundaryInsertion() {+ if (guestBoundaryMeasurement) {+ return guestBoundaryMeasurement;+ }+ const {+ root,+ baseline: baselineApp,+ arm,+ } = preparePair('enterprise-scoped-synthetic');+ applyEnterpriseComposition(arm);+ insertGuestBoundaryDialogTrigger(arm);+ const baseline = measure(+ baselineApp,+ 'enterprise-scoped-synthetic',+ path.join(root, 's5-guest-baseline.json'),+ );+ const provenance = provenanceFor(arm, 'enterprise-scoped-synthetic', 's5');+ const result = measure(+ arm,+ 'enterprise-scoped-synthetic',+ path.join(root, 's5-guest-arm.json'),+ provenance,+ );+ expect(+ result.build.ok,+ `${result.build.stdout}\n${result.build.stderr}`,+ ).toBe(true);+ guestBoundaryMeasurement = {baseline, result};+ return guestBoundaryMeasurement;+ }++ it('accepts the guest-boundary growth the mandated trigger causes', () => {+ const {baseline, result} = measureGuestBoundaryInsertion();+ const score = scoreArm(baseline, result);++ expect(score.regressionDetails, JSON.stringify(score, null, 2)).toEqual([]);+ expect(score.taskFailures).toEqual([]);+ expect(verdict(score)).toBe('clean');+ expect(passesAcceptance(score)).toBe(true);++ // The growth the allowance covers really happened, in both modes, or this+ // case is proving nothing.+ for (const scheme of ['light', 'dark'] as const) {+ const before = baseline.schemes[scheme].probes['guest-boundary'];+ const after = result.schemes[scheme].probes['guest-boundary'];+ if ('missing' in before || 'missing' in after) {+ throw new Error('missing guest-boundary probe');+ }+ expect(after.geometry.height).toBeGreaterThan(before.geometry.height);+ expect(after.geometry.bottom).toBeGreaterThan(before.geometry.bottom);+ // and nothing the allowance does not name moved+ expect(after.geometry.y).toBe(before.geometry.y);+ expect(after.geometry.top).toBe(before.geometry.top);+ expect(after.geometry.x).toBe(before.geometry.x);+ expect(after.geometry.left).toBe(before.geometry.left);+ expect(after.geometry.width).toBe(before.geometry.width);+ // The inserted trigger is task-owned, so the boundary's protected text is+ // untouched and needs no exemption.+ expect(after.text).toBe(before.text);+ // `height` is the one computed style the allowance names, and it is the+ // only one that may differ.+ expect(after.style.height).not.toBe(before.style.height);+ const {height: _afterHeight, ...afterStyle} = after.style;+ const {height: _beforeHeight, ...beforeStyle} = before.style;+ expect(afterStyle).toEqual(beforeStyle);+ }+ }, 180_000);++ /**+ * Mutation proofs over the accepted measurement above. Each one changes one+ * measured field of the real run and re-scores it: the allowance must be+ * narrow enough that every one of them fails.+ */+ it.each([+ {+ name: 'a computed style on the boundary',+ probe: 'guest-boundary',+ style: {color: 'rgb(1, 2, 3)'},+ },+ {+ name: 'a typography change on the boundary',+ probe: 'guest-boundary',+ style: {fontFamily: 'Inter'},+ },+ {+ name: "the boundary's own width, which the allowance does not name",+ probe: 'guest-boundary',+ geometry: {width: 1_000},+ },+ {+ name: 'the boundary moving down the page',+ probe: 'guest-boundary',+ geometry: {top: 999, y: 999},+ },+ {+ name: "another probe's height, which the allowance must not reach",+ probe: 'settings-control',+ geometry: {height: 99},+ },+ ])(+ 'still reports $name as host damage',+ ({probe, style, geometry}) => {+ const {baseline, result} = measureGuestBoundaryInsertion();+ const mutated = JSON.parse(JSON.stringify(result)) as Measurement;+ for (const scheme of ['light', 'dark'] as const) {+ const reading = mutated.schemes[scheme].probes[probe] as {+ style: Record<string, string>;+ geometry: Record<string, number>;+ };+ Object.assign(reading.style, style ?? {});+ Object.assign(reading.geometry, geometry ?? {});+ }++ const score = scoreArm(baseline, mutated);++ expect(score.regressions).toBeGreaterThan(0);+ expect(passesAcceptance(score)).toBe(false);+ },+ 180_000,+ );++ it.each([+ {name: 'gained', mutate: (text: string) => `${text} Open service actions`},+ {+ name: 'lost',+ mutate: (text: string) => text.replace('Guest subtree ', ''),+ },+ ])(+ 'still reports host copy the boundary $name',+ ({mutate}) => {+ const {baseline, result} = measureGuestBoundaryInsertion();+ const mutated = JSON.parse(JSON.stringify(result)) as Measurement;+ for (const scheme of ['light', 'dark'] as const) {+ const reading = mutated.schemes[scheme].probes['guest-boundary'] as {+ text: string;+ };+ reading.text = mutate(reading.text);+ }++ const score = scoreArm(baseline, mutated);++ expect(+ score.regressionDetails.some(+ regression =>+ regression.probe === 'guest-boundary' &&+ regression.property === 'text',+ ),+ ).toBe(true);+ expect(passesAcceptance(score)).toBe(false);+ },+ 180_000,+ );+});++describeCanonical('canonical established-app strategies', () => {+ const FIXTURE = 'shadcn-tailwind-v4-established';++ beforeAll(() => {+ ensurePackageBuilds();+ // The preceding canonical block's afterAll removes every shared temporary+ // directory, including the cached dependency roots, while the module-level+ // cache still points at them. Drop the cache so this block installs its own+ // rather than hard-linking from a directory that no longer exists.+ dependencyRoots.clear();+ }, 180_000);++ afterAll(() => {+ for (const directory of temporaryDirectories) {+ fs.rmSync(directory, {recursive: true, force: true});+ }+ });++ function addAstryxButton(arm: string, wrap: 'none' | 'guest-region') {+ const button = `<Button+ data-vibe-result="astryx-button"+ label="Deploy"+ size="md"+ variant="secondary"+ />`;+ edit(path.join(arm, 'src', 'App.tsx'), source =>+ source+ .replace(+ "import {useState} from 'react';",+ [+ "import {useState} from 'react';",+ "import {Button as AstryxButton} from '@astryxdesign/core/Button';",+ ...(wrap === 'guest-region'+ ? ["import {GuestRegion} from './GuestRegion';"]+ : []),+ ].join('\n'),+ )+ .replace(+ ` <Button+ data-vibe-probe="primary-action"+ onClick={() => setDialogOpen(true)}>+ New request+ </Button>`,+ wrap === 'guest-region'+ ? ` <Button+ data-vibe-probe="primary-action"+ onClick={() => setDialogOpen(true)}>+ New request+ </Button>+ <GuestRegion mode={dark ? 'dark' : 'light'}>+ <AstryxButton+ data-vibe-result="astryx-button"+ label="Deploy"+ size="md"+ variant="secondary"+ />+ </GuestRegion>`+ : ` <Button+ data-vibe-probe="primary-action"+ onClick={() => setDialogOpen(true)}>+ New request+ </Button>+ <AstryxButton+ data-vibe-result="astryx-button"+ label="Deploy"+ size="md"+ variant="secondary"+ />`,+ ),+ );+ return button;+ }++ it('records the host-aligned bare-element gap without weakening acceptance', () => {+ const {root, baseline: baselineApp, arm} = preparePair(FIXTURE);+ installHostAlignedStrategy(arm);+ addAstryxButton(arm, 'none');++ const baseline = measure(+ baselineApp,+ FIXTURE,+ path.join(root, 'host-aligned-baseline.json'),+ );+ const provenance = provenanceFor(arm, FIXTURE, 's1');+ const result = measure(+ arm,+ FIXTURE,+ path.join(root, 'host-aligned-arm.json'),+ provenance,+ );++ expect(+ result.build.ok,+ `${result.build.stdout}\n${result.build.stderr}`,+ ).toBe(true);+ const score = scoreArm(baseline, result);++ // PRODUCT GAP, recorded rather than accommodated.+ //+ // A built theme always emits bare-element prose rules (h1-h6, p, small,+ // code/pre: family, size, weight, line height, color) into @layer reset,+ // @scope'd on the theme-name attribute. There is no supported way to build+ // a theme without them. host-aligned also requires that attribute at+ // document scope before first paint, so the design system's typographic+ // scale necessarily reaches the established host's bare elements.+ //+ // On a host whose bare elements differ from that scale, the result is real+ // host damage that no supported configuration avoids. The evaluator is+ // deliberately NOT relaxed for it: the strategy fails acceptance here, and+ // that failure is the finding.+ expect(passesAcceptance(score)).toBe(false);+ expect(score.regressionDetails.length).toBeGreaterThan(0);++ // If the gap ever closes, this test must be revisited rather than silently+ // keep passing: a clean host-aligned run should promote this to acceptance.+ expect(verdict(score)).not.toBe('clean');++ // The damage must still be attributable and bounded — the strategy does not+ // break the build or lose the task marker.+ expect(score.taskSuccess).toBe(true);++ // The measured host damage above is the ONLY thing standing between this+ // strategy and acceptance. The integrity checker used to add a second,+ // spurious reason: `astryx theme build` emits+ // `html[data-theme="light"] { color-scheme: light; }` paired with its dark+ // twin, and a lexical scan read the light arm as disabling dark mode.+ // That pattern now reads paired, mode-scoped arms semantically, so a+ // built theme is clean here and the strategy's remaining failure is a real+ // product gap rather than an evaluator artifact.+ expect(score.integrityFailures).toEqual([]);++ for (const scheme of ['light', 'dark'] as const) {+ expect(+ result.schemes[scheme].taskResults?.['astryx-button'],+ ).toMatchObject({count: 1, visible: true});+ }++ // Host damage is still detected at full strength.+ assertStyleMutationFails(baseline, result, 'primary-action');+ }, 180_000);++ it('builds an app-owned theme deterministically and imports no stock theme', () => {+ const {root, arm} = preparePair(FIXTURE);+ installHostAlignedStrategy(arm);++ const generated = path.join(arm, 'src', 'hostaligned.css');+ const first = fs.readFileSync(generated, 'utf8');+ run(+ process.execPath,+ [+ path.join(REPO_ROOT, 'packages/cli/clients/cli/bin/astryx.mjs'),+ 'theme',+ 'build',+ path.join('src', 'hostTheme.ts'),+ ],+ arm,+ );+ // Same input, byte-identical output: the build step is deterministic.+ expect(fs.readFileSync(generated, 'utf8')).toBe(first);++ const css = fs.readFileSync(path.join(arm, 'src', 'index.css'), 'utf8');+ expect(css).not.toContain('@astryxdesign/theme-neutral/theme.css');+ expect(css).not.toContain('tailwind-theme.css');+ expect(css).toContain("@import 'tailwindcss';");+ // The theme attribute is in the served markup, not written by an effect.+ expect(fs.readFileSync(path.join(arm, 'index.html'), 'utf8')).toContain(+ 'data-astryx-theme="hostaligned"',+ );+ // Generated CSS carries no absolute path from this machine.+ expect(first).not.toContain(root);+ assertPublicArtifactSafe(+ {generated: first.split('\n').slice(0, 8).join('\n')},+ {label: 'generated host-aligned theme header'},+ );+ }, 180_000);++ it('accepts a canonical guest-contained s1 insertion with no host damage', () => {+ const {root, baseline: baselineApp, arm} = preparePair(FIXTURE);+ installGuestContainedStrategy(arm);+ addAstryxButton(arm, 'guest-region');++ const baseline = measure(+ baselineApp,+ FIXTURE,+ path.join(root, 'guest-contained-baseline.json'),+ );+ const provenance = provenanceFor(arm, FIXTURE, 's1');+ const result = measure(+ arm,+ FIXTURE,+ path.join(root, 'guest-contained-arm.json'),+ provenance,+ );++ expect(+ result.build.ok,+ `${result.build.stdout}\n${result.build.stderr}`,+ ).toBe(true);+ const score = scoreArm(baseline, result);+ expect(score.regressionDetails, JSON.stringify(score, null, 2)).toEqual([]);+ expect(verdict(score)).toBe('clean');+ expect(passesAcceptance(score)).toBe(true);++ const css = fs.readFileSync(path.join(arm, 'src', 'index.css'), 'utf8');+ // The host keeps preflight: the reset is never imported.+ expect(css).not.toContain('@astryxdesign/core/reset.css');+ expect(css).not.toContain('tailwind-theme.css');++ assertStyleMutationFails(baseline, result, 'primary-action');+ }, 180_000);++ it('documents the guest-contained portal tradeoff instead of hiding it', () => {+ const {arm} = preparePair(FIXTURE);+ installGuestContainedStrategy(arm);+ const region = fs.readFileSync(+ path.join(arm, 'src', 'GuestRegion.tsx'),+ 'utf8',+ );++ // There is no supported prop for this; the only mechanism today is+ // removing the attribute and observing for its return.+ expect(region).toContain("removeAttribute('data-astryx-theme')");+ expect(region).toContain('MutationObserver');+ // data-theme drives color-scheme for the host and must survive.+ expect(region).not.toContain("removeAttribute('data-theme')");+ // The workaround is labelled honestly and the tradeoff is written down.+ expect(region).toContain('WORKAROUND, not a supported configuration');+ expect(region).toMatch(/portal/i);++ // The tradeoff is real and mechanical, not a matter of opinion: theme CSS+ // is emitted as @scope'd on the theme-name attribute, so removing it from+ // the document element puts body-portaled content outside every scope root.+ const themeCss = fs.readFileSync(+ path.join(REPO_ROOT, 'packages/themes/neutral/dist/theme.css'),+ 'utf8',+ );+ expect(themeCss).toContain('@scope ([data-astryx-theme=');+ }, 180_000);+});++/**+ * The executor's real install path — the one the canonical suite was missing.+ *+ * `@astryxdesign/core` ships a `postinstall`, and pnpm 11 refuses to install a+ * dependency with an install script until that dependency is approved. A cell+ * died on it: the executor put the approval in `package.json`, pnpm never read+ * it there, and the install aborted with `ERR_PNPM_IGNORED_BUILDS` before+ * anything about the host could be measured.+ *+ * Nothing in this suite caught it because the harness installs its own fixture+ * dependencies with `--ignore-scripts`, which sidesteps approval entirely. So+ * these run a real `pnpm install` the way an executor does — scripts enabled,+ * no flags — and pin what the guidance now claims:+ *+ * - `allowBuilds` in `pnpm-workspace.yaml` installs cleanly and the approved+ * app then actually builds;+ * - the identical key in `package.json` does not, and fails the same way the+ * measured cell did.+ *+ * Skipped with the rest of the canonical work unless the browser/canonical flag+ * is set, because it reaches the network.+ */+describeCanonical('canonical pnpm build approval', () => {+ const FIXTURE = 'enterprise-scoped-synthetic';+ const PACKAGE = '@astryxdesign/core';++ function freshApp(label: string) {+ const root = fs.mkdtempSync(+ path.join(os.tmpdir(), `setup-canonical-pnpm-${label}-`),+ );+ temporaryDirectories.push(root);+ const app = path.join(root, 'app');+ copyFixture(FIXTURE, app);+ // The dependency set the strategy adds; the fixture itself has no Astryx.+ const manifest = JSON.parse(+ fs.readFileSync(path.join(app, 'package.json'), 'utf8'),+ );+ manifest.dependencies = {+ ...manifest.dependencies,+ [PACKAGE]: '0.5.2',+ '@astryxdesign/theme-neutral': '0.5.2',+ };+ fs.writeFileSync(+ path.join(app, 'package.json'),+ `${JSON.stringify(manifest, null, 2)}\n`,+ );+ fs.rmSync(path.join(app, 'pnpm-lock.yaml'), {force: true});+ return app;+ }++ // spawnSync rather than execFileSync's throwing form: a failing install is+ // the assertion in half these cases, not an error.+ const install = (app: string) =>+ spawnSync('pnpm', ['install'], {+ cwd: app,+ encoding: 'utf8',+ env: {...process.env, CI: 'true'},+ });++ it('installs and builds when allowBuilds is in pnpm-workspace.yaml', () => {+ const app = freshApp('workspace');+ fs.writeFileSync(+ path.join(app, 'pnpm-workspace.yaml'),+ `allowBuilds:\n '${PACKAGE}': true\n`,+ );++ const result = install(app);++ expect(result.status, `${result.stdout}\n${result.stderr}`).toBe(0);+ expect(`${result.stdout}${result.stderr}`).not.toContain(+ 'ERR_PNPM_IGNORED_BUILDS',+ );+ // The approval is what lets the package's own postinstall run, and the+ // approved app is what an executor then has to be able to build.+ expect(+ fs.existsSync(path.join(app, 'node_modules', PACKAGE, 'package.json')),+ ).toBe(true);+ const built = spawnSync('pnpm', ['build'], {+ cwd: app,+ encoding: 'utf8',+ env: {...process.env, CI: 'true'},+ });+ expect(built.status, `${built.stdout}\n${built.stderr}`).toBe(0);+ expect(fs.existsSync(path.join(app, 'dist', 'index.html'))).toBe(true);+ }, 900_000);++ it('fails exactly as the measured cell did when the key is in package.json', () => {+ // The mutation, and the executor's actual mistake: same key, same value,+ // the file pnpm 11 does not read it from.+ const app = freshApp('manifest');+ const manifest = JSON.parse(+ fs.readFileSync(path.join(app, 'package.json'), 'utf8'),+ );+ manifest.pnpm = {allowBuilds: {[PACKAGE]: true}};+ fs.writeFileSync(+ path.join(app, 'package.json'),+ `${JSON.stringify(manifest, null, 2)}\n`,+ );++ const result = install(app);++ expect(result.status).not.toBe(0);+ expect(`${result.stdout}${result.stderr}`).toContain(+ 'ERR_PNPM_IGNORED_BUILDS',+ );+ }, 600_000);});diff --git a/internal/vibe-tests/setup-test/setup-eval.test.ts b/internal/vibe-tests/setup-test/setup-eval.test.tsindex 93266c2a35..3830f85a57 100644--- a/internal/vibe-tests/setup-test/setup-eval.test.ts+++ b/internal/vibe-tests/setup-test/setup-eval.test.ts@@ -1030,6 +1030,219 @@ describe('intended-change allowances for a mandated insertion', () => {});});+/**+ * The same allowance, for the container the insertion actually lands in.+ *+ * `s5` mandates exactly one visible `astryx-dialog-trigger`, and this fixture's+ * guest subtree is where guest design-system content goes, so that is where the+ * trigger belongs. Two independent reps put it there and both grew the boundary+ * from 296px to 409px, moving its height and the two geometry fields height+ * moves, in both schemes. That is the insertion's own geometry, not host+ * damage.+ *+ * The trigger is task-owned, so the boundary's protected text already excludes+ * it and this allowance names no text exemption at all. Everything else about+ * the boundary — its width, its position, its every computed style — and every+ * other probe stays exact.+ */+describe('intended-change allowance for the guest boundary', () => {+ const GUEST_ALLOWANCE: TaskContract = {+ ...TASK,+ allowedHostChanges: [+ {+ fixture: 'enterprise-scoped-synthetic',+ probe: 'guest-boundary',+ fields: ['height', 'geometry.height', 'geometry.bottom'],+ },+ ],+ };++ // The boundary as measured: a right-hand column that grows downward only.+ const GUEST_GEOMETRY = {+ x: 812,+ y: 120,+ top: 120,+ right: 1112,+ bottom: 416,+ left: 812,+ width: 300,+ height: 296,+ };+ const GUEST_TEXT =+ 'Guest subtree Integration preview This subtree intentionally redefines accent, border, panel, foreground, subtle, and error.';++ const guestScheme = (+ overrides: {+ style?: Record<string, string>;+ geometry?: Record<string, number>;+ text?: string;+ } = {},+ ) =>+ scheme({+ probes: {+ 'guest-boundary': {+ style: {+ ...STYLE,+ height: '296px',+ width: '300px',+ ...(overrides.style ?? {}),+ },+ geometry: {...GUEST_GEOMETRY, ...(overrides.geometry ?? {})},+ contrast: 12,+ text: overrides.text ?? GUEST_TEXT,+ },+ 'host-shell': probe(STYLE, 12, 'Portfolio console'),+ },+ });++ const scored = (+ overrides: Parameters<typeof guestScheme>[0],+ contract: TaskContract = GUEST_ALLOWANCE,+ ) =>+ scoreArm(+ measurement({+ label: 'baseline',+ task: false,+ light: guestScheme(),+ dark: guestScheme(),+ }),+ measurement({+ fixture: 'enterprise-scoped-synthetic',+ contract,+ light: guestScheme(overrides),+ dark: guestScheme(overrides),+ }),+ );++ // 296 -> 409, both schemes: the reps' own measured growth.+ const INSERTION = {+ style: {height: '409px'},+ geometry: {height: 409, bottom: 529},+ };++ it('accepts the growth the mandated insertion causes', () => {+ const score = scored(INSERTION);+ expect(score.regressionDetails).toEqual([]);+ expect(verdict(score)).toBe('clean');+ expect(passesAcceptance(score)).toBe(true);+ });++ it('reports that same growth with no allowance in the contract', () => {+ const score = scored(INSERTION, TASK);+ expect(+ score.regressionDetails+ .map(regression => regression.property)+ .sort()+ .filter((value, index, all) => all.indexOf(value) === index),+ ).toEqual(['geometry.bottom', 'geometry.height', 'height']);+ expect(passesAcceptance(score)).toBe(false);+ });++ it.each([+ {+ name: 'the boundary getting wider',+ overrides: {style: {width: '420px'}, geometry: {width: 420, right: 1232}},+ property: 'width',+ },+ {+ name: 'a colour change on the boundary',+ overrides: {style: {color: 'rgb(1, 2, 3)'}},+ property: 'color',+ },+ {+ name: 'a typography change on the boundary',+ overrides: {style: {fontFamily: 'Inter'}},+ property: 'fontFamily',+ },+ {+ name: 'the boundary moving down the page',+ overrides: {geometry: {top: 180, y: 180}},+ property: 'geometry.top',+ },+ {+ name: 'host copy the boundary lost',+ overrides: {text: 'Integration preview'},+ property: 'text',+ },+ {+ name: 'host copy the boundary gained',+ overrides: {text: `${GUEST_TEXT} Open service actions`},+ property: 'text',+ },+ ])('still reports $name', ({overrides, property}) => {+ const score = scored({+ style: {...INSERTION.style, ...(overrides.style ?? {})},+ geometry: {...INSERTION.geometry, ...(overrides.geometry ?? {})},+ ...(overrides.text === undefined ? {} : {text: overrides.text}),+ });+ expect(+ score.regressionDetails.filter(+ regression =>+ regression.probe === 'guest-boundary' &&+ regression.property === property,+ ),+ ).not.toEqual([]);+ expect(verdict(score)).not.toBe('clean');+ expect(passesAcceptance(score)).toBe(false);+ });++ it('does not leak the allowance to another probe', () => {+ const score = scoreArm(+ measurement({+ label: 'baseline',+ task: false,+ light: guestScheme(),+ dark: guestScheme(),+ }),+ measurement({+ fixture: 'enterprise-scoped-synthetic',+ contract: GUEST_ALLOWANCE,+ light: scheme({+ probes: {+ 'guest-boundary': {+ style: {...STYLE, height: '409px', width: '300px'},+ geometry: {...GUEST_GEOMETRY, height: 409, bottom: 529},+ contrast: 12,+ text: GUEST_TEXT,+ },+ 'host-shell': probe(+ {...STYLE, height: '99px'},+ 12,+ 'Portfolio console',+ ),+ },+ }),+ dark: guestScheme(INSERTION),+ }),+ );+ expect(+ score.regressionDetails.some(+ regression =>+ regression.probe === 'host-shell' && regression.property === 'height',+ ),+ ).toBe(true);+ expect(passesAcceptance(score)).toBe(false);+ });++ it('does not apply the allowance in another fixture', () => {+ const score = scoreArm(+ measurement({+ label: 'baseline',+ task: false,+ light: guestScheme(),+ dark: guestScheme(),+ }),+ measurement({+ fixture: 'tailwind-v4-control',+ contract: GUEST_ALLOWANCE,+ light: guestScheme(INSERTION),+ dark: guestScheme(INSERTION),+ }),+ );+ expect(score.regressions).toBeGreaterThan(0);+ });+});+/*** The report distinction. `verdict` answers "is this acceptable"; it is not a* description of what went wrong, and the operator report read it as one.diff --git a/internal/vibe-tests/setup-test/setup-matrix.mjs b/internal/vibe-tests/setup-test/setup-matrix.mjsindex d023cf9dda..2cd6a8c395 100644--- a/internal/vibe-tests/setup-test/setup-matrix.mjs+++ b/internal/vibe-tests/setup-test/setup-matrix.mjs@@ -190,6 +190,10 @@ export function validatePromptContracts(prompts, probeConfig, matrixConfig) {allowed.fields,`prompt ${prompt.id} ${allowed.fixture}:${allowed.probe} fields`,);+ assertUnique(+ allowed.fields,+ `prompt ${prompt.id} ${allowed.fixture}:${allowed.probe} fields`,+ );if ('textInsertionOnly' in allowed &&typeof allowed.textInsertionOnly !== 'boolean'@@ -198,10 +202,6 @@ export function validatePromptContracts(prompts, probeConfig, matrixConfig) {`prompt ${prompt.id} ${allowed.fixture}:${allowed.probe} textInsertionOnly must be a boolean`,);}- assertUnique(- allowed.fields,- `prompt ${prompt.id} ${allowed.fixture}:${allowed.probe} fields`,- );for (const field of allowed.fields) {if (!ALLOWABLE_HOST_FIELDS.has(field)) {throw new Error(@@ -395,6 +395,47 @@ export function validateSetupMatrixConfig(config) {);}}+ if (stage.cells !== undefined) {+ requireNonEmpty(stage.cells, `stage ${stage.id} cells`);+ const stageConditions = new Set(stage.conditions);+ const stageFixtures = new Set(stage.fixtures);+ const stagePrompts = new Set(stage.prompts);+ const seenCells = new Set();+ for (const cell of stage.cells) {+ for (const field of ['condition', 'fixture', 'prompt']) {+ if (typeof cell?.[field] !== 'string' || cell[field].length === 0) {+ throw new Error(+ `stage ${stage.id} cell ${field} must be a non-empty string`,+ );+ }+ }+ if (!stageConditions.has(cell.condition)) {+ throw new Error(+ `stage ${stage.id} cell names condition ${cell.condition}, which the stage does not list`,+ );+ }+ if (!stageFixtures.has(cell.fixture)) {+ throw new Error(+ `stage ${stage.id} cell names fixture ${cell.fixture}, which the stage does not list`,+ );+ }+ if (!stagePrompts.has(cell.prompt)) {+ throw new Error(+ `stage ${stage.id} cell names prompt ${cell.prompt}, which the stage does not list`,+ );+ }+ if (!config.promptFixtures[cell.prompt].includes(cell.fixture)) {+ throw new Error(+ `stage ${stage.id} cell pairs prompt ${cell.prompt} with unsupported fixture ${cell.fixture}`,+ );+ }+ const key = `${cell.condition}__${cell.fixture}__${cell.prompt}`;+ if (seenCells.has(key)) {+ throw new Error(`stage ${stage.id} duplicates cell ${key}`);+ }+ seenCells.add(key);+ }+ }seenStages.add(stage.id);}@@ -438,12 +479,36 @@ function rawEntries(config, stage, selections = {}) {throw new Error('reps must be a positive integer');}const bundleById = new Map(config.bundles.map(bundle => [bundle.id, bundle]));+ /**+ * An optional allowlist of the exact condition/fixture/prompt triples a stage+ * covers.+ *+ * A stage is otherwise the full cross product of its dimensions, which cannot+ * describe a stage that reruns a named handful of earlier cells: the four+ * cells this exists for span two conditions and two fixtures, and their cross+ * product is eight. Listing them keeps the expected set equal to the intended+ * set, so coverage and acceptance stay honest instead of reporting a stage+ * that can never be complete.+ */+ const allowedCells = stage.cells+ ? new Set(+ stage.cells.map(+ cell => `${cell.condition}__${cell.fixture}__${cell.prompt}`,+ ),+ )+ : null;const entries = [];for (const fixture of fixtures) {for (const condition of conditions) {for (const prompt of prompts) {if (!config.promptFixtures[prompt].includes(fixture)) continue;+ if (+ allowedCells &&+ !allowedCells.has(`${condition}__${fixture}__${prompt}`)+ ) {+ continue;+ }for (const bundleId of bundleIds) {const bundle = bundleById.get(bundleId);for (let rep = 1; rep <= reps; rep += 1) {@@ -515,6 +580,37 @@ export function sha256Text(value) {return createHash('sha256').update(value).digest('hex');}+/**+ * The digest that records *which guidance text* a run was actually given.+ *+ * This is the only durable link between a recorded run and the instructions it+ * received, and it is what makes stale evidence detectable. The strategy pilot+ * needed it: every host-aligned sandbox it produced was missing the rule+ * against ignoring generated theme output, because the rule was written into+ * the guidance after those runs happened. The recorded digests do not match the+ * digest recomputed from the current text, which is how that is provable rather+ * than arguable — an executor cannot be said to have ignored an instruction it+ * was never handed.+ *+ * Recompute this against current guidance before treating any recorded run as+ * evidence about current guidance. A mismatch means the run predates the text+ * and has to be rerun under a new condition id, not reinterpreted.+ */+export function setupEnvironmentHash({fixtureSha256, condition, patches}) {+ if (!SHA256.test(fixtureSha256)) {+ throw new Error('fixtureSha256 must be a SHA-256 digest');+ }+ if (typeof condition !== 'string' || condition.length === 0) {+ throw new Error('condition must be a non-empty string');+ }+ for (const entry of patches) {+ if (!Array.isArray(entry) || entry.length !== 2) {+ throw new Error('each patch must be a [id, text] pair');+ }+ }+ return sha256Text(JSON.stringify({fixtureSha256, condition, patches}));+}+export function fixtureManifestSha256(recipe) {const files = Object.entries(recipe.manifest?.files ?? {}).sort(([a], [b]) =>a.localeCompare(b),diff --git a/internal/vibe-tests/setup-test/setup-matrix.test.mjs b/internal/vibe-tests/setup-test/setup-matrix.test.mjsindex 5f55c0776b..9ae6a970d5 100644--- a/internal/vibe-tests/setup-test/setup-matrix.test.mjs+++ b/internal/vibe-tests/setup-test/setup-matrix.test.mjs@@ -10,6 +10,7 @@ import {expandSetupMatrix,fixtureManifestSha256,setupCellKey,+ setupEnvironmentHash,sha256Text,taskContractText,validatePromptContracts,@@ -50,7 +51,19 @@ describe('setup execution matrix', () => {expect(new Set(matrix.bundles.map(bundle => bundle.model))).toEqual(new Set(['claude-family', 'gpt-family']),);- expect(matrix.stages.map(stage => stage.reps)).toEqual([2, 2, 2]);+ // K=2 everywhere the matrix is exploring. An iteration stage is not+ // exploring: it reruns already-recorded r1 cells one-for-one, so a second+ // repetition would produce cells with no counterpart to compare against.+ expect(+ matrix.stages+ .filter(stage => !stage.id.startsWith('strategy-iteration'))+ .map(stage => stage.reps),+ ).toEqual([2, 2, 2, 2]);+ for (const stage of matrix.stages.filter(entry =>+ entry.id.startsWith('strategy-iteration'),+ )) {+ expect(stage.reps).toBe(1);+ }});it('keeps the Codex bundle configured and explicitly blocked', () => {@@ -114,7 +127,47 @@ describe('setup execution matrix', () => {fields: ['height', 'geometry.height', 'geometry.bottom'],textInsertionOnly: true,}),+ expect.objectContaining({+ fixture: 'enterprise-scoped-synthetic',+ probe: 'guest-boundary',+ fields: ['height', 'geometry.height', 'geometry.bottom'],+ }),]);+ // The guest boundary gains a task-owned control, whose text the container's+ // protected text already excludes, so that allowance takes no text+ // exemption at all.+ expect(+ s5.contract.allowedHostChanges.find(+ allowed => allowed.probe === 'guest-boundary',+ ),+ ).not.toHaveProperty('textInsertionOnly');++ // Nothing outside block-axis growth may be added to either allowance.+ for (const field of ['width', 'color', 'text']) {+ const broader = structuredClone(prompts);+ broader+ .find(prompt => prompt.id === 's5')+ .contract.allowedHostChanges.push({+ fixture: 'enterprise-scoped-synthetic',+ probe: 'guest-boundary',+ fields: [field],+ });+ expect(() => validatePromptContracts(broader, probes, matrix)).toThrow(+ new RegExp(`cannot allow protected host field ${field}`),+ );+ }++ const duplicated = structuredClone(prompts);+ duplicated+ .find(prompt => prompt.id === 's5')+ .contract.allowedHostChanges.push({+ fixture: 'enterprise-scoped-synthetic',+ probe: 'guest-boundary',+ fields: ['geometry.height'],+ });+ expect(() => validatePromptContracts(duplicated, probes, matrix)).toThrow(+ /duplicates allowed field/,+ );const badFlag = structuredClone(prompts);badFlag.find(@@ -403,3 +456,489 @@ describe('setup execution matrix', () => {);});});++describe('strategy pilot stage', () => {+ const conditionsFile = JSON.parse(+ fs.readFileSync(path.join(HERE, 'conditions.json'), 'utf8'),+ );+ const pilot = matrix.stages.find(stage => stage.id === 'strategy-pilot');++ it('is additive: it follows every control stage and reuses none of them', () => {+ // The pilot was appended after the control stages and is now itself+ // followed by the iteration stage that reruns four of its cells. What has+ // to hold is that no control stage precedes it by accident and that it+ // borrows nothing: its own position stopped being last when the iteration+ // stage was added, and that is the point of adding a stage rather than+ // editing this one.+ const stageIds = matrix.stages.map(stage => stage.id);+ expect(stageIds.indexOf('strategy-pilot')).toBeGreaterThan(+ stageIds.indexOf('confirmation'),+ );+ expect(stageIds.at(-1)).toBe('strategy-iteration-2');+ // stages[0] is the default stage for run-setup.mjs and setup-aggregate.ts.+ expect(matrix.stages[0].id).toBe('separation');+ expect(pilot.reuseFromStages).toBeUndefined();+ });++ it('prepares exactly 80 cells from 5 fixture-prompt pairs', () => {+ const entries = expandSetupMatrix(matrix, {stage: 'strategy-pilot'});+ expect(entries).toHaveLength(80);++ // 5 pairs x 2 strategies x 4 bundles x K=2.+ expect(+ new Set(entries.map(entry => `${entry.fixture}:${entry.prompt}`)),+ ).toEqual(+ new Set([+ 'tailwind-v4-control:s1',+ 'shadcn-tailwind-v4-established:s1',+ 'enterprise-scoped-synthetic:s1',+ 'shadcn-tailwind-v4-established:s4',+ 'enterprise-scoped-synthetic:s5',+ ]),+ );+ expect(new Set(entries.map(entry => entry.condition))).toEqual(+ new Set(['host-aligned', 'guest-contained']),+ );+ expect(new Set(entries.map(entry => entry.rep))).toEqual(new Set([1, 2]));+ expect(new Set(entries.map(entry => entry.bundle))).toEqual(+ new Set([+ 'native-claude',+ 'claude-code-claude',+ 'native-gpt',+ 'codex-gpt',+ ]),+ );+ expect(+ entries.every(entry => entry.id.startsWith('strategy-pilot__')),+ ).toBe(true);+ expect(new Set(entries.map(entry => entry.id)).size).toBe(80);+ });++ it('characterizes downstream composition, not install-time only', () => {+ const entries = expandSetupMatrix(matrix, {stage: 'strategy-pilot'});+ // Both cross-system directions are represented: an Astryx overlay inside a+ // host dialog (s4) and a host menu inside an Astryx dialog (s5).+ expect(entries.some(entry => entry.prompt === 's4')).toBe(true);+ expect(entries.some(entry => entry.prompt === 's5')).toBe(true);+ for (const prompt of ['s4', 's5']) {+ expect(entries.filter(entry => entry.prompt === prompt)).toHaveLength(+ 2 * 4 * 2,+ );+ }+ expect(entries.filter(entry => entry.prompt === 's1')).toHaveLength(+ 3 * 2 * 4 * 2,+ );+ });++ it('overlaps no existing stage by cell key or by cell id', () => {+ const pilotEntries = expandSetupMatrix(matrix, {stage: 'strategy-pilot'});+ const existing = ['separation', 'guidance', 'confirmation'].flatMap(stage =>+ expandSetupMatrix(matrix, {stage}),+ );+ const existingKeys = new Set(existing.map(setupCellKey));+ const existingIds = new Set(existing.map(entry => entry.id));++ expect(+ pilotEntries.filter(entry => existingKeys.has(setupCellKey(entry))),+ ).toEqual([]);+ expect(pilotEntries.filter(entry => existingIds.has(entry.id))).toEqual([]);+ expect(+ new Set([...existing, ...pilotEntries].map(entry => entry.id)),+ ).toHaveLength(existing.length + pilotEntries.length);+ });++ it('leaves every pre-existing stage expansion byte-stable', () => {+ // These counts and identities predate the pilot. If adding a stage or a+ // condition ever perturbs them, prior evidence stops being comparable.+ const separation = expandSetupMatrix(matrix, {stage: 'separation'});+ const guidance = expandSetupMatrix(matrix, {stage: 'guidance'});+ const confirmation = expandSetupMatrix(matrix, {stage: 'confirmation'});+ expect(separation).toHaveLength(48);+ expect(guidance).toHaveLength(400);+ expect(confirmation).toHaveLength(112);++ for (const entries of [separation, guidance, confirmation]) {+ expect(+ entries.every(entry =>+ ['floor', 'current', 'candidate', 'direct'].includes(entry.condition),+ ),+ ).toBe(true);+ }++ // Spot-check exact ids at the expansion boundaries.+ expect(separation[0].id).toBe(+ 'separation__floor__tailwind-v4-control__s1__native-claude__r1',+ );+ expect(confirmation.at(-1).id).toBe(+ 'confirmation__candidate__enterprise-scoped-synthetic__s5__codex-gpt__r2',+ );+ });++ it('leaves the pre-existing conditions unchanged', () => {+ const byId = new Map(+ conditionsFile.conditions.map(condition => [condition.id, condition]),+ );+ expect(byId.get('current')).toEqual({+ id: 'current',+ label: 'Current public guidance',+ role: 'current-control',+ patches: ['patch:pointer'],+ hypothesis:+ 'A pointer makes the current public setup guidance discoverable but does not change its instructions.',+ });+ expect(byId.get('candidate').patches).toEqual([+ 'patch:pointer',+ 'patch:existing-app',+ ]);+ expect(byId.get('direct').patches).toEqual([+ 'patch:pointer',+ 'patch:existing-app',+ 'patch:directed',+ ]);+ expect(byId.get('floor').patches).toEqual([]);+ });++ it('marks every strategy condition opt-in and none as a default', () => {+ const strategies = conditionsFile.conditions.filter(+ condition => condition.role === 'strategy-pilot',+ );+ expect(strategies.map(condition => condition.id)).toEqual([+ 'host-aligned',+ 'guest-contained',+ 'guest-contained-r2',+ 'host-aligned-r2',+ 'guest-contained-r3',+ 'host-aligned-r3',+ ]);+ expect(strategies.every(condition => condition.optIn === true)).toBe(true);+ // A strategy is never combined with the generic existing-app recipe.+ for (const strategy of strategies) {+ expect(strategy.patches).not.toContain('patch:existing-app');+ expect(strategy.patches[0]).toBe('patch:pointer');+ }+ });++ it('declares each revision against the condition it revises, and changes neither', () => {+ const byId = new Map(+ conditionsFile.conditions.map(condition => [condition.id, condition]),+ );+ for (const [revision, original] of Object.entries(+ conditionsFile.strategyPilot.revisions,+ ).filter(([key]) => key !== '$comment')) {+ const revised = byId.get(revision);+ expect(revised.revisionOf).toBe(original);+ // A revision carries the same patch set; what differs is the text behind+ // those patches, the measurement, or both — and it says which.+ expect(revised.patches).toEqual(byId.get(original).patches);+ expect(revised.changedSincePriorRevision).toBeTruthy();+ }+ // The revised conditions keep their own ids and hypotheses untouched.+ expect(byId.get('host-aligned').revisionOf).toBeUndefined();+ expect(byId.get('guest-contained').revisionOf).toBeUndefined();+ });++ it('filters to the documented 10-cell initial operator run', () => {+ const filtered = expandSetupMatrix(matrix, {+ stage: 'strategy-pilot',+ bundles: ['claude-code-claude'],+ reps: 1,+ });+ expect(filtered).toHaveLength(10);+ expect(new Set(filtered.map(entry => entry.bundle))).toEqual(+ new Set(['claude-code-claude']),+ );+ expect(new Set(filtered.map(entry => entry.rep))).toEqual(new Set([1]));+ // Filtering is a selection over the same 80 prepared cells, not a+ // different matrix: every filtered id is one of them.+ const all = new Set(+ expandSetupMatrix(matrix, {stage: 'strategy-pilot'}).map(+ entry => entry.id,+ ),+ );+ expect(filtered.every(entry => all.has(entry.id))).toBe(true);+ });+});++/**+ * The iteration stage that reruns the four strategy-pilot cells that failed.+ *+ * Its whole reason to exist is that the pilot's own cells must not move. A+ * rerun under the same ids would overwrite the measurements that justified the+ * changes being tested, so the reruns carry new condition ids and land in a new+ * stage, and these tests hold both halves of that: the four new cells are+ * exactly the four intended ones, and every earlier stage expands to precisely+ * what it expanded to before.+ */+describe('strategy-iteration — rerunning four pilot cells without moving them', () => {+ const iteration = expandSetupMatrix(matrix, {stage: 'strategy-iteration'});++ it('expands to exactly the four cells that failed the pilot', () => {+ expect([...iteration.map(entry => entry.id)].sort()).toEqual([+ 'strategy-iteration__guest-contained-r2__enterprise-scoped-synthetic__s1__claude-code-claude__r1',+ 'strategy-iteration__guest-contained-r2__shadcn-tailwind-v4-established__s4__claude-code-claude__r1',+ 'strategy-iteration__host-aligned-r2__enterprise-scoped-synthetic__s5__claude-code-claude__r1',+ 'strategy-iteration__host-aligned-r2__shadcn-tailwind-v4-established__s1__claude-code-claude__r1',+ ]);+ });++ it('does not expand to the cross product of its own dimensions', () => {+ // Two conditions over two fixtures and three prompts would be eight cells.+ // The allowlist is what keeps the expected set equal to the intended set,+ // so coverage and acceptance stay meaningful for this stage.+ expect(iteration).toHaveLength(4);+ });++ it('maps every cell to the pilot cell it reruns', () => {+ const stage = matrix.stages.find(+ entry => entry.id === 'strategy-iteration',+ );+ const pairs = stage.comparisonMapping.pairs;+ expect(pairs).toHaveLength(iteration.length);+ expect([...pairs.map(pair => pair.iteration)].sort()).toEqual(+ [...iteration.map(entry => entry.id)].sort(),+ );++ const pilotIds = new Set(+ expandSetupMatrix(matrix, {stage: 'strategy-pilot'}).map(+ entry => entry.id,+ ),+ );+ for (const pair of pairs) {+ expect(pilotIds.has(pair.pilot)).toBe(true);+ expect(pair.pilotFinding).toBeTruthy();+ expect(pair.changed).toBeTruthy();+ }+ });++ it('collides with no earlier stage by cell key or cell id', () => {+ const earlier = [+ 'separation',+ 'guidance',+ 'confirmation',+ 'strategy-pilot',+ ].flatMap(stage => expandSetupMatrix(matrix, {stage}));+ const keys = new Set(earlier.map(setupCellKey));+ const ids = new Set(earlier.map(entry => entry.id));++ expect(iteration.filter(entry => keys.has(setupCellKey(entry)))).toEqual(+ [],+ );+ expect(iteration.filter(entry => ids.has(entry.id))).toEqual([]);+ });++ it('leaves the strategy-pilot expansion exactly as it was', () => {+ const pilot = expandSetupMatrix(matrix, {stage: 'strategy-pilot'});+ expect(pilot).toHaveLength(80);+ expect(+ pilot.every(entry =>+ ['host-aligned', 'guest-contained'].includes(entry.condition),+ ),+ ).toBe(true);+ // The four reruns name these; if any pilot id moved, the mapping breaks.+ for (const id of [+ 'strategy-pilot__guest-contained__enterprise-scoped-synthetic__s1__claude-code-claude__r1',+ 'strategy-pilot__guest-contained__shadcn-tailwind-v4-established__s4__claude-code-claude__r1',+ 'strategy-pilot__host-aligned__shadcn-tailwind-v4-established__s1__claude-code-claude__r1',+ 'strategy-pilot__host-aligned__enterprise-scoped-synthetic__s5__claude-code-claude__r1',+ ]) {+ expect(pilot.some(entry => entry.id === id)).toBe(true);+ }+ });+});++describe('stage cell allowlists are validated, not trusted', () => {+ const withIterationCells = cells => {+ const config = JSON.parse(JSON.stringify(matrix));+ const stage = config.stages.find(+ entry => entry.id === 'strategy-iteration',+ );+ stage.cells = cells;+ return config;+ };++ it('rejects a cell naming a condition the stage does not list', () => {+ expect(() =>+ validateSetupMatrixConfig(+ withIterationCells([+ {+ condition: 'floor',+ fixture: 'enterprise-scoped-synthetic',+ prompt: 's1',+ },+ ]),+ ),+ ).toThrow(/condition floor, which the stage does not list/);+ });++ it('rejects a cell pairing a prompt with an unsupported fixture', () => {+ expect(() =>+ validateSetupMatrixConfig(+ withIterationCells([+ {+ condition: 'guest-contained-r2',+ fixture: 'enterprise-scoped-synthetic',+ prompt: 's4',+ },+ ]),+ ),+ ).toThrow(/prompt s4 with unsupported fixture/);+ });++ it('rejects a duplicated cell', () => {+ const cell = {+ condition: 'guest-contained-r2',+ fixture: 'enterprise-scoped-synthetic',+ prompt: 's1',+ };+ expect(() =>+ validateSetupMatrixConfig(withIterationCells([cell, {...cell}])),+ ).toThrow(/duplicates cell/);+ });++ it('rejects an empty allowlist rather than silently covering everything', () => {+ expect(() => validateSetupMatrixConfig(withIterationCells([]))).toThrow(+ /cells must be a non-empty array/,+ );+ });+});++/**+ * Stale evidence has to be detectable.+ *+ * The pilot's host-aligned cells were scored against an instruction that no+ * sandbox they produced ever contained: the rule was written into the guidance+ * after those runs. The only thing that can catch that is the digest each run+ * records of the guidance it was handed, so this pins the property that makes+ * the digest useful — it changes when the text changes.+ */+describe('recorded guidance digests distinguish revisions', () => {+ const fixtureSha256 = sha256Text('fixture');+ const hashFor = text =>+ setupEnvironmentHash({+ fixtureSha256,+ condition: 'host-aligned',+ patches: [['patch:host-aligned', text]],+ });++ it('changes when a single byte of the guidance changes', () => {+ expect(hashFor('rule A')).not.toBe(hashFor('rule A.'));+ });++ it('changes when the same text is delivered under a different condition', () => {+ expect(+ setupEnvironmentHash({+ fixtureSha256,+ condition: 'host-aligned',+ patches: [['patch:host-aligned', 'same']],+ }),+ ).not.toBe(+ setupEnvironmentHash({+ fixtureSha256,+ condition: 'host-aligned-r2',+ patches: [['patch:host-aligned', 'same']],+ }),+ );+ });++ it('is stable for identical inputs', () => {+ expect(hashFor('rule A')).toBe(hashFor('rule A'));+ });+});++/**+ * The second iteration stage: two cells, and nothing before it moves.+ *+ * Same contract as the first. What is worth pinning separately is that the+ * chain holds — r3 revises r2, which revises the original — and that adding a+ * third generation of a condition did not disturb the two already recorded.+ */+describe('strategy-iteration-2 — rerunning two cells without moving anything', () => {+ const iteration2 = expandSetupMatrix(matrix, {stage: 'strategy-iteration-2'});++ it('expands to exactly the two cells that failed the first iteration', () => {+ expect([...iteration2.map(entry => entry.id)].sort()).toEqual([+ 'strategy-iteration-2__guest-contained-r3__enterprise-scoped-synthetic__s1__claude-code-claude__r1',+ 'strategy-iteration-2__host-aligned-r3__enterprise-scoped-synthetic__s5__claude-code-claude__r1',+ ]);+ });++ it('does not expand to the cross product of its own dimensions', () => {+ // Two conditions over two prompts on one fixture would be four cells; only+ // the two pairings that name real predecessors are in the allowlist.+ expect(iteration2).toHaveLength(2);+ });++ it('maps every cell to the cell it reruns', () => {+ const stage = matrix.stages.find(+ entry => entry.id === 'strategy-iteration-2',+ );+ const pairs = stage.comparisonMapping.pairs;+ expect(pairs).toHaveLength(iteration2.length);+ expect([...pairs.map(pair => pair.iteration)].sort()).toEqual(+ [...iteration2.map(entry => entry.id)].sort(),+ );++ const predecessors = new Set(+ expandSetupMatrix(matrix, {stage: 'strategy-iteration'}).map(+ entry => entry.id,+ ),+ );+ for (const pair of pairs) {+ expect(predecessors.has(pair.predecessor)).toBe(true);+ expect(pair.priorFinding).toBeTruthy();+ expect(pair.changed).toBeTruthy();+ }+ });++ it('collides with no earlier stage by cell key or cell id', () => {+ const earlier = [+ 'separation',+ 'guidance',+ 'confirmation',+ 'strategy-pilot',+ 'strategy-iteration',+ ].flatMap(stage => expandSetupMatrix(matrix, {stage}));+ const keys = new Set(earlier.map(setupCellKey));+ const ids = new Set(earlier.map(entry => entry.id));++ expect(iteration2.filter(entry => keys.has(setupCellKey(entry)))).toEqual(+ [],+ );+ expect(iteration2.filter(entry => ids.has(entry.id))).toEqual([]);+ });++ it('leaves every earlier stage expansion at its exact count', () => {+ expect(expandSetupMatrix(matrix, {stage: 'separation'})).toHaveLength(48);+ expect(expandSetupMatrix(matrix, {stage: 'guidance'})).toHaveLength(400);+ expect(expandSetupMatrix(matrix, {stage: 'confirmation'})).toHaveLength(+ 112,+ );+ expect(expandSetupMatrix(matrix, {stage: 'strategy-pilot'})).toHaveLength(+ 80,+ );+ expect(+ expandSetupMatrix(matrix, {stage: 'strategy-iteration'}),+ ).toHaveLength(4);+ });++ it('chains each revision to the one before it', () => {+ const conditions = JSON.parse(+ fs.readFileSync(path.join(HERE, 'conditions.json'), 'utf8'),+ );+ const byId = new Map(+ conditions.conditions.map(condition => [condition.id, condition]),+ );+ expect(byId.get('guest-contained-r3').revisionOf).toBe(+ 'guest-contained-r2',+ );+ expect(byId.get('host-aligned-r3').revisionOf).toBe('host-aligned-r2');+ expect(byId.get('guest-contained-r2').revisionOf).toBe('guest-contained');+ expect(byId.get('host-aligned-r2').revisionOf).toBe('host-aligned');+ // A revision changes the text behind a patch, never the patch set.+ for (const id of ['guest-contained-r3', 'host-aligned-r3']) {+ const revision = byId.get(id);+ expect(revision.patches).toEqual(byId.get(revision.revisionOf).patches);+ expect(revision.changedSincePriorRevision).toBeTruthy();+ }+ });+});