An open source design system that's fully customizable and agent ready
TypeScript
test(vibe-tests): measure setup arms without writing to them (#5855)
nynexman446410c764bf36f5
diff --git a/internal/vibe-tests/README.md b/internal/vibe-tests/README.mdindex 92dbd398ac..71f68de959 100644--- a/internal/vibe-tests/README.md+++ b/internal/vibe-tests/README.md@@ -156,6 +156,7 @@ internal/vibe-tests/│ ├── run-setup.mjs # sandbox + task + provenance preparation only│ ├── setup-interactions.mjs # marker-driven dialog and nested-overlay opener│ ├── setup-measure.mjs # build + exact style, geometry, and overlay probes+│ ├── setup-workspace.mjs # copy-on-write build root; the sandbox is never written to│ ├── setup-integrity.mjs # read-only diff attestation and escape-hatch checks│ ├── setup-eval.ts # strict deterministic deltas against each fixture│ └── setup-aggregate.ts # matrix coverage, A/B deltas, and final acceptancediff --git a/internal/vibe-tests/setup-test/PLAN.md b/internal/vibe-tests/setup-test/PLAN.mdindex e3753aec10..e95dc15e44 100644--- a/internal/vibe-tests/setup-test/PLAN.md+++ b/internal/vibe-tests/setup-test/PLAN.md@@ -46,6 +46,22 @@ Each task declares stable task-owned `data-vibe-result` markers. They are measurGuidance is committed into the sandbox baseline before execution, so the resulting diff contains only executor work.+## Hermetic measurement++Measurement reads a sandbox; it must not write to one. That tree is the attested artifact — the runner digested its bytes, the integrity checker reads them, and any re-measurement or recovery has to read them again — and building in place destroyed it: the fixtures' `prebuild` regenerates the app-owned theme over the executor's copy, `vite build` writes `dist/`, the CLI appends to its invocation log, and the bundler caches into `node_modules`. A failed build leaves the same debris. A measured cell then no longer hashed to its own digest and its evidence could not be re-derived.++**The build runs in a disposable copy.** Files are copied with `cp -a --reflink=auto`, which shares extents copy-on-write where the filesystem supports it and byte-copies where it does not, with a pure-Node fallback for a platform without a usable `cp`. It never hardlinks: a hardlinked file rewritten in the copy rewrites the original inode.++**The pnpm virtual store is copied, not linked.** A pnpm `node_modules` is mostly links into `.pnpm`, the store holding every real package tree, so mirroring it with a symlink hands the copy a door straight back into the sandbox — writing through a symlink writes to its target, and an install or rebuild in the copy would rewrite the original's store. Symlinks keep their target text, so a package link resolves within the copy; an absolute link that would reach into the sandbox is retargeted onto the copy, while one pointing at a workspace package elsewhere is left alone. `verifyWorkspaceIsolation` re-derives that claim from the finished copy — every symlink resolved, and inodes compared against the original — and the measurer runs it before any build. The build root is removed on every path out, including a thrown measurement.++Tests take a full byte-and-mtime manifest of the sandbox before and after a real measurement and require it identical for a build that succeeds and a build that fails, with the virtual store included. Mutation proofs put each way of getting it wrong back — the mirrored store, and a hardlinked copy — and show the sandbox is written through and the isolation check names it. Measurement output is unchanged: stable fixture ids, no private paths, and the build root redacted alongside them.++**Probes are read at rest.** `getComputedStyle` on an element with a running transition returns the interpolated value, which Chromium serializes in the interpolation space rather than the space the value was authored in — so a host colour that never changed reads back as `oklab(…)` mid-transition and `oklch(…)` at rest. The numbers are identical, only the spelling differs, and an exact comparison scores that as host damage. The harness triggers these itself, by clicking host controls to open an interaction, and whether a read lands inside the transition window depends on how fast the app's own bundle responds — which is stable per app, so it misreads deterministically for some apps and never for others. Running transitions are now settled before any read. Only `CSSTransition` is awaited: transitions always finish, while an infinite decorative animation never would.++**Container text counts only what the browser renders.** A container's protected text excludes task-owned subtrees, because a task that mandates a control necessarily adds that control's text. It could not exclude an overlay the design system hoists _out_ of the task-owned element — a `Button` with a `tooltip` renders it as a `display: none` sibling linked by `aria-describedby` — so a mandated control's own closed tooltip landed in the host container's text and scored as host damage while nothing on screen had changed. Unrendered subtrees no longer count. This is not a hole an executor can hide damage in: removing, rewriting, or hiding host copy all still change the string, and only invisible _additions_ stop counting.++**The local file server binds what it fetches.** `server.listen(0, …)` with no host binds the wildcard — `::` wherever Node has IPv6 — while the page fetch was a hardcoded `http://127.0.0.1:<port>`. Those agree only where the host maps v4-in-v6; where they do not, every page load fails to connect and the run looks like a broken app. The server now binds IPv4 loopback explicitly and the origin is read back from the socket, with a wildcard rewritten to the loopback of its own family and IPv6 literals bracketed. Measurement needs no network namespace or loopback shim; if a runner wraps it in one out of habit, drop the wrapper.+## Measurement and integrityAll conditions use the same evaluator against a pristine copy of the same fixture in light and dark modes:@@ -58,7 +74,7 @@ All conditions use the same evaluator against a pristine copy of the same fixtur6. Existing host overlays and both cross-system composition directions: presence, keyboard reachability, visibility, viewport intersection, clipping ancestors, center hit target, top-layer state, and stack order above dialog and backdrop.7. Task-specific output markers and semantics, kept separate from host baseline probes.8. The executor's source diff. The evaluator rejects deleted or wholesale-replaced host files, blanket resets, hardcoded `!important`, disabled dark-mode behavior, missing Astryx use, missing requested composition, and changed/deleted host probes or overlays. Replacement is judged on content, not on raw diff lines: lines are compared with their indentation and interior whitespace normalized, so wrapping a component tree in a provider — which re-indents every line under it — deletes nothing, while minifying or rewriting a file still deletes all of it.-9. A runner-recorded agent diff digest. The evaluator only measures; it never patches. A missing or mismatched digest is post-run manual intervention and fails acceptance.+9. A runner-recorded agent diff digest. The evaluator only measures; it never patches. A missing or mismatched digest is post-run manual intervention and fails acceptance. Measuring never writes to the sandbox either: the agent's tree is the attested artifact, so it is copied into a disposable build root and only the copy is built and served. The copy is copy-on-write where the filesystem allows and a byte copy where it does not, never a hardlink, and every symlink — the package links of a pnpm virtual store above all — resolves inside the copy, so a build's own writes — a regenerated theme artifact, `dist/`, an appended CLI log, a bundler cache — land in the copy. The original's bytes and modification times are unchanged whether the build succeeds or fails, so a cell still hashes to the digest that attested it and can be re-measured or recovered from those same bytes.10. Public artifact safety. Measurements use stable fixture identifiers instead of local app paths, redact private paths and hosts from diagnostics, normalize provenance usage sources to a generic label, and fail closed before setup or universal aggregate reports can emit private path data.Verdicts remain `broken-build`, `noisy`, `silent-damage`, `cosmetic-drift`, and `clean`. `passesAcceptance` is true only when the verdict is `clean`, the task contract succeeds, no escape hatch is present, and the measured diff exactly matches the runner's agent-output digest.diff --git a/internal/vibe-tests/setup-test/setup-canonical.test.ts b/internal/vibe-tests/setup-test/setup-canonical.test.tsindex 5b66953a1a..fc274f6fd4 100644--- a/internal/vibe-tests/setup-test/setup-canonical.test.ts+++ b/internal/vibe-tests/setup-test/setup-canonical.test.ts@@ -13,6 +13,8 @@ import {copyFixture} from '../src/fixture-suite.mjs';import {assertPublicArtifactSafe} from '../src/public-artifact.mjs';// @ts-expect-error -- setup-integrity.mjs intentionally has no declaration output.import {analyzeSetupIntegrity} from './setup-integrity.mjs';+// @ts-expect-error -- setup-workspace.mjs intentionally has no declaration output.+import {manifestDifferences, treeManifest} from './setup-workspace.mjs';import {passesAcceptance,scoreArm,@@ -793,6 +795,74 @@ describeCanonical('canonical intended changes', () => {}}, 120_000);+ /**+ * The measurer must not build in the sandbox it measures.+ *+ * That sandbox is the attested artifact: the runner digested its bytes, the+ * integrity checker reads them, and a re-measurement has to be able to read+ * them again. A build in place regenerates the app-owned theme over the+ * executor's copy, writes `dist/`, appends to the CLI's invocation log, and+ * caches into `node_modules`, after which the tree no longer hashes to the+ * digest that attested it.+ *+ * This runs the real measurer, with a real build, against a real fixture arm,+ * and compares a full byte-and-mtime manifest of the sandbox taken before and+ * after — every entry, tracked and ignored alike, since the debris is ignored+ * by construction.+ */+ it('measures a real arm without changing one byte of its sandbox', () => {+ const {root, arm} = preparePair('tailwind-v4-control');+ applyButtonAddition(arm);+ const provenance = provenanceFor(arm, 'tailwind-v4-control', 's1');+ const digestBefore = analyzeSetupIntegrity(arm).diffSha256;+ const before = treeManifest(arm);++ const result = measure(+ arm,+ 'tailwind-v4-control',+ path.join(root, 'immutability-arm.json'),+ provenance,+ );+ expect(+ result.build.ok,+ `${result.build.stdout}\n${result.build.stderr}`,+ ).toBe(true);+ // The build really produced what the measurement read.+ expect(result.layerOrder.length).toBeGreaterThan(0);++ expect(manifestDifferences(before, treeManifest(arm))).toEqual([]);+ // The digest that attested this tree still describes it, so the cell can be+ // re-measured or recovered from these same bytes.+ expect(analyzeSetupIntegrity(arm).diffSha256).toBe(digestBefore);+ // Nothing was left behind for the next measurement to trip over.+ expect(fs.existsSync(path.join(arm, 'dist'))).toBe(false);+ }, 180_000);++ it('leaves the sandbox untouched when the build fails', () => {+ const {root, arm} = preparePair('tailwind-v4-control');+ applyButtonAddition(arm);+ // Break the build after the sandbox is attested, the way a real failing+ // executor change would.+ edit(+ path.join(arm, 'src', 'App.tsx'),+ source => `${source}\nconst broken: number = 'not a number';\n`,+ );+ const provenance = provenanceFor(arm, 'tailwind-v4-control', 's1');+ const digestBefore = analyzeSetupIntegrity(arm).diffSha256;+ const before = treeManifest(arm);++ const result = measure(+ arm,+ 'tailwind-v4-control',+ path.join(root, 'immutability-failed.json'),+ provenance,+ );+ expect(result.build.ok).toBe(false);++ expect(manifestDifferences(before, treeManifest(arm))).toEqual([]);+ expect(analyzeSetupIntegrity(arm).diffSha256).toBe(digestBefore);+ }, 180_000);+it('builds, measures, and accepts a correct s1 button insertion', () => {const {root,diff --git a/internal/vibe-tests/setup-test/setup-measure.browser.test.mjs b/internal/vibe-tests/setup-test/setup-measure.browser.test.mjsnew file mode 100644index 0000000000..17f6f3c66d--- /dev/null+++ b/internal/vibe-tests/setup-test/setup-measure.browser.test.mjs@@ -0,0 +1,199 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++/**+ * @file setup-measure.browser.test.mjs+ * @input The real page reader and transition settler, run in a real browser+ * @output Proof that each rule holds and that removing it changes the result+ * @position internal/vibe-tests/setup-test — canonical measurement proofs+ *+ * Two cells of an operator run failed on how the host was measured rather than+ * on anything the executor did to the host. Both are corrected in+ * setup-measure.mjs, and both corrections are the kind that can quietly become+ * a hole, so each is pinned here together with the mutation that must still be+ * caught.+ *+ * These drive the exported `readPage` and `settleTransitions` themselves rather+ * than restating their rules, so a copy cannot drift from the code that runs.+ * Gated on the same flag as the rest of the canonical browser work.+ */++import {describe, expect, it, beforeAll, afterAll} from 'vitest';+import {readPage, settleTransitions} from './setup-measure.mjs';++const RUN_CANONICAL = process.env.ASTRYX_CANONICAL_SETUP_BROWSER === '1';+const describeCanonical = RUN_CANONICAL ? describe : describe.skip;++const PROBE_SPEC = {+ probes: [{name: 'host-panel', selector: '[data-vibe-probe="host-panel"]'}],+ properties: ['color', 'backgroundColor'],+ rootVariables: [],+ results: [],+};++/** A host panel holding its own copy plus a mandated, task-owned control. */+const panelPage = ({+ tooltipHidden = true,+ hostCopy = 'Deploy targets ready',+}) => `+ <!doctype html><html><body>+ <section data-vibe-probe="host-panel">+ <p>${hostCopy}</p>+ <button data-vibe-result="astryx-button" aria-describedby="tip">Deploy</button>+ <div id="tip" role="tooltip" style="display:${tooltipHidden ? 'none' : 'block'}">+ Roll out the selected services+ </div>+ </section>+ </body></html>`;++describeCanonical(+ 'canonical measurement — host text counts what is rendered',+ () => {+ let browser;+ let page;++ beforeAll(async () => {+ const {chromium} = await import('playwright');+ browser = await chromium.launch();+ page = await browser.newPage({viewport: {width: 1280, height: 720}});+ }, 120_000);++ afterAll(async () => {+ await browser?.close();+ });++ const textFor = async html => {+ await page.setContent(html);+ const reading = await page.evaluate(readPage, PROBE_SPEC);+ return reading.probes['host-panel'].text;+ };++ it('excludes a mandated control and its unrendered tooltip from host text', async () => {+ // This is the pilot failure: an Astryx Button with a `tooltip` renders the+ // tooltip as a display:none sibling, hoisted out of the button because a+ // button cannot contain it. The task-owned subtree rule cannot see it, so+ // it landed in the host container's text while nothing on screen changed.+ expect(await textFor(panelPage({}))).toBe('Deploy targets ready');+ });++ it('counts that same tooltip once it is actually rendered', async () => {+ // The rule is "not rendered", not "tooltips are exempt". Content an+ // executor makes visible inside a host container is still host change.+ expect(await textFor(panelPage({tooltipHidden: false}))).toBe(+ 'Deploy targets ready Roll out the selected services',+ );+ });++ it('still reports host copy that was rewritten', async () => {+ expect(+ await textFor(panelPage({hostCopy: 'Deploy targets pending'})),+ ).not.toBe('Deploy targets ready');+ });++ it.each([+ {name: 'display', style: 'display:none'},+ {name: 'visibility', style: 'visibility:hidden'},+ ])(+ 'still reports host copy hidden with $name rather than deleted',+ async ({style}) => {+ // The mutation that would matter if this rule were a hole: hiding host+ // copy must not read back as an unchanged host.+ expect(+ await textFor(`+ <!doctype html><html><body>+ <section data-vibe-probe="host-panel">+ <p style="${style}">Deploy targets ready</p>+ </section>+ </body></html>`),+ ).toBe('');+ },+ );+ },+);++describeCanonical('canonical measurement — probes are read at rest', () => {+ let browser;+ let page;++ beforeAll(async () => {+ const {chromium} = await import('playwright');+ browser = await chromium.launch();+ page = await browser.newPage({viewport: {width: 1280, height: 720}});+ }, 120_000);++ afterAll(async () => {+ await browser?.close();+ });++ /**+ * A host control whose colour transitions, exactly like the fixture button+ * that produced the pilot's only colour finding.+ *+ * The endpoints differ here so a transition actually runs — in the fixture+ * the trigger was a click putting the button into a hover/active state. What+ * the pilot showed is the consequence: a read taken while the transition runs+ * returns the interpolated value, serialized in the interpolation colour+ * space rather than the authored one, and an exact string comparison scores+ * that against a resting baseline as host damage.+ */+ const TRANSITION_PAGE = `+ <!doctype html><html><head><style>+ #control { color: oklch(0.985 0 0); transition: color 0.4s linear; }+ #control.active { color: oklch(0.205 0 0); }+ </style></head><body>+ <button id="control" data-vibe-probe="host-panel">New request</button>+ </body></html>`;++ const colorNow = () =>+ page.evaluate(+ () => getComputedStyle(document.querySelector('#control')).color,+ );++ it('reads an interpolated value, in the interpolation colour space, mid-flight', async () => {+ await page.setContent(TRANSITION_PAGE);+ expect(await colorNow()).toBe('oklch(0.985 0 0)');++ await page.evaluate(() => {+ document.querySelector('#control').classList.add('active');+ });+ const midFlight = await colorNow();++ // Neither endpoint, and not spelled the way either endpoint is authored.+ expect(midFlight.startsWith('oklab(')).toBe(true);+ expect(midFlight).not.toBe('oklch(0.985 0 0)');+ expect(midFlight).not.toBe('oklch(0.205 0 0)');+ });++ it('settles the transition so the read is the resting value', async () => {+ await page.setContent(TRANSITION_PAGE);++ await page.evaluate(() => {+ document.querySelector('#control').classList.add('active');+ });+ await settleTransitions(page);++ expect(await colorNow()).toBe('oklch(0.205 0 0)');+ expect(await page.evaluate(() => document.getAnimations().length)).toBe(0);++ // The pilot's finding was on a host control the executor never touched:+ // settling has to return the authored spelling, not merely a stable one.+ await page.setContent(TRANSITION_PAGE);+ await settleTransitions(page);+ expect(await colorNow()).toBe('oklch(0.985 0 0)');+ });++ it('returns rather than hanging when an infinite animation is running', async () => {+ // Only transitions are awaited. A decorative infinite animation never+ // finishes, and waiting on it would hang every measurement.+ await page.setContent(`+ <!doctype html><html><head><style>+ @keyframes spin { to { transform: rotate(360deg) } }+ #spinner { animation: spin 1s linear infinite; }+ </style></head><body><div id="spinner">.</div></body></html>`);++ const started = Date.now();+ await settleTransitions(page, 5_000);++ expect(Date.now() - started).toBeLessThan(4_000);+ expect(await page.evaluate(() => document.getAnimations().length)).toBe(1);+ });+});diff --git a/internal/vibe-tests/setup-test/setup-measure.mjs b/internal/vibe-tests/setup-test/setup-measure.mjsindex b64e3150bc..4890b810ff 100644--- a/internal/vibe-tests/setup-test/setup-measure.mjs+++ b/internal/vibe-tests/setup-test/setup-measure.mjs@@ -1,7 +1,15 @@#!/usr/bin/env node// Copyright (c) Meta Platforms, Inc. and affiliates.-/** Build, serve, and measure one copied canonical fixture in two color schemes. */+/** Build, serve, and measure one copied canonical fixture in two color schemes.+ *+ * The build never runs in the sandbox it measures. The agent's sandbox is the+ * attested artifact — its bytes are what the runner digested and what integrity+ * and any later recovery read — so it is copied into a disposable build root+ * first and only the copy is built and served. See `setup-workspace.mjs`. The+ * integrity analysis still reads the original, because the original is the+ * thing being attested.+ */import {spawnSync} from 'node:child_process';import * as fs from 'node:fs';@@ -11,6 +19,7 @@ import {fileURLToPath} from 'node:url';import {analyzeSetupIntegrity} from './setup-integrity.mjs';import {openInteractionState} from './setup-interactions.mjs';import {validatePromptContracts} from './setup-matrix.mjs';+import {createMeasurementWorkspace} from './setup-workspace.mjs';import {assertPublicArtifactSafe,publicProvenance,@@ -51,9 +60,21 @@ export function parseLayerOrder(css) {return order;}+/**+ * Run the app's build.+ *+ * `CI=true` and piped stdio keep it non-interactive: a package manager or+ * bundler that finds no TTY must not stop to ask anything, and a spinner that+ * probes for one must not error out. Measurement is unattended by definition.+ */function build(appDir) {const started = Date.now();- const result = spawnSync('pnpm', ['build'], {cwd: appDir, encoding: 'utf8'});+ const result = spawnSync('pnpm', ['build'], {+ cwd: appDir,+ encoding: 'utf8',+ env: {...process.env, CI: 'true'},+ stdio: ['ignore', 'pipe', 'pipe'],+ });return {ok: result.status === 0,status: result.status ?? -1,@@ -72,12 +93,17 @@ export function classifyCssAssets(files) {export function measurementPrivateValues({appDir,outFile,+ buildDir = null,environment = process.env,cwd = process.cwd(),repoRoot = REPO_ROOT,}) {return [appDir,+ buildDir,+ // The disposable build root's parent, so neither the copy's path nor the+ // temporary directory holding it can reach a public artifact.+ buildDir ? path.dirname(buildDir) : null,repoRoot,cwd,path.dirname(outFile),@@ -133,6 +159,34 @@ const MIME = {'.woff2': 'font/woff2',};+/**+ * The origin to fetch a locally bound server on, derived from what it actually+ * bound to.+ *+ * The pairing that has to hold is bind address and fetch address. Listening+ * without a host binds the wildcard — `::` wherever Node has IPv6 — while the+ * fetch used a hardcoded `127.0.0.1`, so the two only agreed as long as the+ * host mapped v4-in-v6. Where it does not, every page load fails to connect and+ * the measurement looks like a broken app rather than a broken harness. That is+ * what made an operator wrap the measurer in a network namespace with loopback+ * forced up; nothing about measuring an app needs that.+ *+ * A wildcard is never a fetchable address, so it is rewritten to the loopback+ * of its own family: `0.0.0.0` to `127.0.0.1`, `::` to `[::1]`. Anything else+ * is used as bound, and IPv6 literals are bracketed as a URL requires.+ */+export function loopbackOrigin(address) {+ if (!address || typeof address !== 'object') {+ throw new Error('server.address() did not return an AddressInfo');+ }+ const {address: host, port, family} = address;+ const ipv6 = family === 'IPv6' || family === 6 || String(host).includes(':');+ const WILDCARD = new Set(['0.0.0.0', '::', '::0', '0:0:0:0:0:0:0:0', '']);+ const target = WILDCARD.has(host) ? (ipv6 ? '::1' : '127.0.0.1') : host;+ const bracketed = target.includes(':') ? `[${target}]` : target;+ return `http://${bracketed}:${port}`;+}+function serve(distDir) {const server = http.createServer((request, response) => {const url = new URL(request.url ?? '/', 'http://localhost');@@ -153,13 +207,27 @@ function serve(distDir) {);fs.createReadStream(file).pipe(response);});- return new Promise(resolve =>- server.listen(0, () => resolve({server, port: server.address().port})),- );+ return new Promise((resolve, reject) => {+ server.once('error', reject);+ // Bind IPv4 loopback explicitly rather than the wildcard: the app under+ // measurement is only ever fetched from this process, so there is no reason+ // to listen on anything reachable, and an explicit bind is what makes the+ // fetch address predictable. The origin is still read back from the socket+ // rather than assumed.+ server.listen(0, '127.0.0.1', () => {+ const address = server.address();+ resolve({server, port: address.port, origin: loopbackOrigin(address)});+ });+ });}/* c8 ignore start -- executed in the page rather than in Node. */-function readPage(spec) {+/**+ * Exported so the canonical suite can run the real reader in a real browser+ * against a fixed DOM, rather than restating its rules in a copy that can+ * drift. It is passed to `page.evaluate`, so it must stay self-contained.+ */+export function readPage(spec) {const normalizedRect = rect => {const normalize = value => Math.round(value * 64) / 64;return {@@ -213,8 +281,37 @@ function readPage(spec) {Math.round((Math.max(light, dark) / Math.min(light, dark)) * 100) / 100);};+ /**+ * The host copy a container renders, excluding task-owned subtrees.+ *+ * Two exclusions, for two different reasons.+ *+ * Task-owned subtrees (`data-vibe-*`) are skipped because the task mandated+ * them: a container that must gain a control necessarily gains that control's+ * text, and comparing it exactly would report the mandate as host damage.+ *+ * Unrendered subtrees are skipped because this measure exists to detect+ * *visible* host change, and text the browser does not paint is not visible+ * host change. This is not a hole an executor can hide damage in: removing or+ * rewriting host copy still changes the string, so hiding a host paragraph+ * still reports the words it lost. Only *invisible additions* stop counting.+ *+ * The case that forced this: a design-system control's own overlay is not+ * always a DOM child of that control. An Astryx `Button` with a `tooltip`+ * renders the tooltip as a `display: none` sibling — hoisted out of the+ * button because a button cannot legally contain it — linked by+ * `aria-describedby`. The subtree exclusion above cannot see it, so the+ * mandated control's own closed tooltip landed in the host container's text+ * and scored as host damage while nothing on screen had changed.+ */const protectedText = element => {const parts = [];+ const rendered = node =>+ typeof node.checkVisibility !== 'function' ||+ node.checkVisibility({+ contentVisibilityAuto: true,+ visibilityProperty: true,+ });const visit = node => {for (const child of node.childNodes) {if (child.nodeType === Node.TEXT_NODE) {@@ -229,6 +326,7 @@ function readPage(spec) {) {continue;}+ if (!rendered(child)) continue;visit(child);}};@@ -463,9 +561,46 @@ function readPage(spec) {}/* c8 ignore stop */+/**+ * Wait for running CSS transitions to finish before anything is measured.+ *+ * A probe read is meant to capture the app at rest. `getComputedStyle` on an+ * element with a running transition returns the *interpolated* value, which+ * Chromium serializes in the interpolation space rather than the space the+ * value was authored in — so a host button whose colour never changed reads+ * back as `oklab(0.985 0 0)` mid-transition and `oklch(0.985 0 0)` at rest.+ * The numbers are identical; only the spelling differs, and an exact string+ * comparison scores that as host damage.+ *+ * The harness triggers these itself: opening an interaction clicks host+ * controls, and a host control with `transition-colors` starts a transition on+ * that click. Whether the read lands inside the transition window depends on+ * how long the app's own bundle takes to respond, which is stable per app —+ * so this misreads deterministically for some apps and never for others, which+ * is worse than flaky.+ *+ * Only `CSSTransition` is awaited. Transitions always finish; a decorative+ * infinite animation never would, so awaiting every animation would hang. The+ * timeout is a backstop for a transition on a property that never settles.+ */+export async function settleTransitions(page, timeoutMs = 2000) {+ await page.evaluate(async ms => {+ const running = document+ .getAnimations()+ .filter(animation => animation.constructor.name === 'CSSTransition');+ if (running.length === 0) return;+ await Promise.race([+ Promise.allSettled(running.map(animation => animation.finished)),+ new Promise(resolve => {+ setTimeout(resolve, ms);+ }),+ ]);+ }, timeoutMs);+}+async function readScheme({browser,- port,+ origin,colorScheme,probeSpec,interaction,@@ -483,9 +618,11 @@ async function readScheme({);page.on('pageerror', error => pageErrors.push(String(error)));page.on('requestfailed', request => failedRequests.push(request.url()));- await page.goto(`http://127.0.0.1:${port}/`);+ await page.goto(`${origin}/`);await page.waitForLoadState('networkidle');await page.evaluate(() => document.fonts.ready);+ // Quiesce mount-time transitions so the interaction starts from rest.+ await settleTransitions(page);let state = {opened: false, keyboardReached: {}};let interactionError = null;if (interaction) {@@ -495,6 +632,8 @@ async function readScheme({interactionError = String(error);}}+ // Quiesce the transitions the interaction above just started.+ await settleTransitions(page);const reading = await page.evaluate(readPage, {...probeSpec, interaction});if (reading.interaction) {reading.interaction.opened = state.opened && interactionError === null;@@ -606,112 +745,132 @@ async function main() {};}- const measurement = {- label,- fixture: fixtureId,- app: appDir,- measuredAt: new Date().toISOString(),- build: build(appDir),- layerOrder: [],- measurementErrors: [],- ...(taskDefinition- ? {- task: {- id: taskDefinition.id,- kind: taskDefinition.kind,- contract: taskDefinition.contract,- },- executionStatus: provenance?.execution?.status ?? 'missing',- integrity,- }- : {}),- schemes: {},- };+ // The build runs on a disposable copy. `appDir` stays exactly as the agent+ // left it — that tree is the attested artifact, and integrity below reads it.+ const workspace = createMeasurementWorkspace(appDir);+ const buildDir = workspace.dir;++ try {+ const measurement = {+ label,+ fixture: fixtureId,+ app: appDir,+ measuredAt: new Date().toISOString(),+ build: build(buildDir),+ layerOrder: [],+ measurementErrors: [],+ ...(taskDefinition+ ? {+ task: {+ id: taskDefinition.id,+ kind: taskDefinition.kind,+ contract: taskDefinition.contract,+ },+ executionStatus: provenance?.execution?.status ?? 'missing',+ integrity,+ }+ : {}),+ schemes: {},+ };- if (measurement.build.ok) {- const distDir = path.join(appDir, 'dist');- const layers = emittedLayerOrder(distDir);- measurement.layerOrder = layers.order;- measurement.measurementErrors = layers.errors;- const {chromium} = await import('playwright');- const {server, port} = await serve(distDir);- const browser = await chromium.launch();-- for (const colorScheme of ['light', 'dark']) {- const mainReading = await readScheme({- browser,- port,- colorScheme,- probeSpec,- interaction: probeSpec.interaction,- });- const taskInteractions = {};- for (const interaction of taskDefinition?.contract.interactions ?? []) {- const taskReading = await readScheme({+ if (measurement.build.ok) {+ const distDir = path.join(buildDir, 'dist');+ const layers = emittedLayerOrder(distDir);+ measurement.layerOrder = layers.order;+ measurement.measurementErrors = layers.errors;+ const {chromium} = await import('playwright');+ const {server, origin} = await serve(distDir);+ const browser = await chromium.launch();++ for (const colorScheme of ['light', 'dark']) {+ const mainReading = await readScheme({browser,- port,- colorScheme,- probeSpec: {...probeSpec, probes: [], rootVariables: [], results: []},- interaction,- });- taskInteractions[interaction.id] = taskReading.reading.interaction;- mainReading.consoleErrors.push(...taskReading.consoleErrors);- mainReading.pageErrors.push(...taskReading.pageErrors);- mainReading.failedRequests.push(...taskReading.failedRequests);- }- if (screenshotDir) {- fs.mkdirSync(screenshotDir, {recursive: true});- const page = await browser.newPage({- viewport: {width: 1280, height: 720},+ origin,colorScheme,+ probeSpec,+ interaction: probeSpec.interaction,});- await page.goto(`http://127.0.0.1:${port}/`);- await page.screenshot({- path: path.join(screenshotDir, `${label}-${colorScheme}.png`),- fullPage: true,- });- await page.close();+ const taskInteractions = {};+ for (const interaction of taskDefinition?.contract.interactions ?? []) {+ const taskReading = await readScheme({+ browser,+ origin,+ colorScheme,+ probeSpec: {+ ...probeSpec,+ probes: [],+ rootVariables: [],+ results: [],+ },+ interaction,+ });+ taskInteractions[interaction.id] = taskReading.reading.interaction;+ mainReading.consoleErrors.push(...taskReading.consoleErrors);+ mainReading.pageErrors.push(...taskReading.pageErrors);+ mainReading.failedRequests.push(...taskReading.failedRequests);+ }+ if (screenshotDir) {+ fs.mkdirSync(screenshotDir, {recursive: true});+ const page = await browser.newPage({+ viewport: {width: 1280, height: 720},+ colorScheme,+ });+ await page.goto(`${origin}/`);+ await page.screenshot({+ path: path.join(screenshotDir, `${label}-${colorScheme}.png`),+ fullPage: true,+ });+ await page.close();+ }+ measurement.schemes[colorScheme] = {+ ...mainReading.reading,+ taskInteractions,+ consoleErrors: mainReading.consoleErrors,+ pageErrors: mainReading.pageErrors,+ failedRequests: mainReading.failedRequests,+ };}- measurement.schemes[colorScheme] = {- ...mainReading.reading,- taskInteractions,- consoleErrors: mainReading.consoleErrors,- pageErrors: mainReading.pageErrors,- failedRequests: mainReading.failedRequests,- };++ await browser.close();+ server.close();}- await browser.close();- server.close();- }+ const privateValues = measurementPrivateValues({+ appDir,+ outFile,+ buildDir,+ });+ const exportedMeasurement = publicMeasurement(measurement, {+ fixtureId,+ privateValues,+ });+ fs.mkdirSync(path.dirname(outFile), {recursive: true});+ fs.writeFileSync(+ outFile,+ `${JSON.stringify(exportedMeasurement, null, 2)}\n`,+ );- const privateValues = measurementPrivateValues({appDir, outFile});- const exportedMeasurement = publicMeasurement(measurement, {- fixtureId,- privateValues,- });- fs.mkdirSync(path.dirname(outFile), {recursive: true});- fs.writeFileSync(- outFile,- `${JSON.stringify(exportedMeasurement, null, 2)}\n`,- );+ if (provenance) {+ const sidecarOut = outFile.replace(/\.json$/, '.provenance.json');+ const exportedProvenance = publicProvenance(provenance, {privateValues});+ fs.writeFileSync(+ sidecarOut,+ `${JSON.stringify(exportedProvenance, null, 2)}\n`,+ );+ }- if (provenance) {- const sidecarOut = outFile.replace(/\.json$/, '.provenance.json');- const exportedProvenance = publicProvenance(provenance, {privateValues});- fs.writeFileSync(- sidecarOut,- `${JSON.stringify(exportedProvenance, null, 2)}\n`,+ console.log(+ `${label}: build ${measurement.build.ok ? 'ok' : `FAILED (${measurement.build.status})`}` ++ (measurement.build.ok+ ? `, ${measurement.schemes.light.consoleErrors.length} console errors` ++ `, ${Object.values(measurement.schemes.light.probes).filter(probe => probe.missing).length} probes missing`+ : ''),);+ } finally {+ // A failed build leaves the same debris a successful one does, so the copy+ // is removed on every path out.+ workspace.cleanup();}-- console.log(- `${label}: build ${measurement.build.ok ? 'ok' : `FAILED (${measurement.build.status})`}` +- (measurement.build.ok- ? `, ${measurement.schemes.light.consoleErrors.length} console errors` +- `, ${Object.values(measurement.schemes.light.probes).filter(probe => probe.missing).length} probes missing`- : ''),- );}const isMain =diff --git a/internal/vibe-tests/setup-test/setup-measure.test.mjs b/internal/vibe-tests/setup-test/setup-measure.test.mjsindex 01f091e691..692197deb6 100644--- a/internal/vibe-tests/setup-test/setup-measure.test.mjs+++ b/internal/vibe-tests/setup-test/setup-measure.test.mjs@@ -3,6 +3,7 @@import {describe, expect, it} from 'vitest';import {classifyCssAssets,+ loopbackOrigin,measurementPrivateValues,normalizeSubpixel,parseLayerOrder,@@ -199,3 +200,95 @@ describe('setup browser measurement helpers', () => {expect(exported.message).toBe('owner-420 can rebuild');});});++/**+ * The local file server has to be fetchable at the address it was bound to.+ *+ * It was not. `server.listen(0, …)` with no host binds the wildcard — `::`+ * wherever Node has IPv6 — while the page fetch was a hardcoded+ * `http://127.0.0.1:<port>`. Those two agree only where the host maps v4-in-v6.+ * Where they do not, every page load fails to connect, and the measurement+ * reports it as a broken app rather than a broken harness. An operator worked+ * around it by running the whole measurer inside a network namespace with+ * loopback forced up; nothing about measuring a built app should need that.+ *+ * The server now binds IPv4 loopback explicitly, and the origin is still read+ * back from the socket rather than assumed.+ */+describe('local server origin', () => {+ it.each([+ {+ name: 'the IPv6 wildcard, bracketed',+ // The exact case that broke: bound to `::`, fetched at `127.0.0.1`.+ address: {address: '::', family: 'IPv6', port: 4321},+ expected: 'http://[::1]:4321',+ },+ {+ name: 'the IPv4 wildcard',+ address: {address: '0.0.0.0', family: 'IPv4', port: 8080},+ expected: 'http://127.0.0.1:8080',+ },+ {+ name: 'a concrete bound address, as bound',+ address: {address: '127.0.0.1', family: 'IPv4', port: 5173},+ expected: 'http://127.0.0.1:5173',+ },+ ])('resolves $name', ({address, expected}) => {+ expect(loopbackOrigin(address)).toBe(expected);+ });++ it('rejects anything that is not an AddressInfo rather than guessing', () => {+ // A pipe server's address() is a string; silently inventing an origin from+ // one would put the old hardcoded assumption back.+ expect(() => loopbackOrigin('/tmp/sock')).toThrow(/AddressInfo/);+ });+});++describe('local server binding', () => {+ it('binds IPv4 loopback and is reachable at the origin it reports', async () => {+ const http = await import('node:http');+ const server = http.createServer((_request, response) => {+ response.end('ok');+ });+ await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));+ const address = server.address();+ try {+ expect(address.address).toBe('127.0.0.1');+ const origin = loopbackOrigin(address);+ expect(origin).toBe(`http://127.0.0.1:${address.port}`);+ // The pairing that matters: fetch the origin the bind actually produced.+ const response = await fetch(`${origin}/`);+ expect(await response.text()).toBe('ok');+ } finally {+ server.close();+ }+ });++ it('is not reachable on a non-loopback interface', async () => {+ const http = await import('node:http');+ const os = await import('node:os');+ const external = Object.values(os.networkInterfaces())+ .flat()+ .find(entry => entry && entry.family === 'IPv4' && !entry.internal);+ const server = http.createServer((_request, response) =>+ response.end('ok'),+ );+ await new Promise(resolve => server.listen(0, '127.0.0.1', resolve));+ const {port} = server.address();+ try {+ if (!external) {+ expect(true).toBe(true);+ return;+ }+ // An explicit loopback bind is also a smaller surface than the wildcard:+ // the app under measurement is only ever fetched by this process.+ await expect(+ fetch(`http://${external.address}:${port}/`, {+ signal: AbortSignal.timeout(2000),+ }),+ ).rejects.toThrow();+ } finally {+ server.close();+ }+ });+});diff --git a/internal/vibe-tests/setup-test/setup-workspace.mjs b/internal/vibe-tests/setup-test/setup-workspace.mjsnew file mode 100644index 0000000000..4c5598e9ea--- /dev/null+++ b/internal/vibe-tests/setup-test/setup-workspace.mjs@@ -0,0 +1,440 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++/**+ * A disposable build root for measuring one sandbox.+ *+ * The measurer must not build in the sandbox it measures. That sandbox is the+ * attested artifact: its bytes are what the runner digested, what the integrity+ * checker reads, and what a later re-measurement or recovery has to be able to+ * read again. A build in place destroys that — the fixtures' `prebuild`+ * regenerates the app-owned theme over the executor's copy, `vite build` writes+ * `dist/`, the CLI appends to its invocation log, and the bundler drops caches+ * into `node_modules`. A failed build leaves the same debris.+ *+ * So the build gets its own copy, and *nothing reachable from that copy can+ * write into the original*. The second half is the hard part, because a pnpm+ * `node_modules` is mostly links:+ *+ * node_modules/react -> .pnpm/[email protected]/node_modules/react+ * node_modules/@scope/pkg -> ../.pnpm/@[email protected]/node_modules/@scope/pkg+ * node_modules/.pnpm/<id>/... the real package trees (the virtual store)+ * node_modules/.modules.yaml the store's own state+ *+ * Symlinking `.pnpm` into the copy — which is what this did first — hands the+ * copy a door straight back into the original: an install or a rebuild that+ * rewrites the virtual store, or writes any cache under it, writes through the+ * link. So does symlinking `.bin`, or `.modules.yaml`, since writing through a+ * symlink writes to its target. Hardlinking is no better: a build that rewrites+ * a hardlinked file writes to the shared inode, and the original changes.+ *+ * The copy is therefore a real copy, made copy-on-write where the filesystem+ * supports it:+ *+ * - Files are copied with `cp -a --reflink=auto`, which shares extents on a+ * filesystem that can (btrfs, xfs) and byte-copies where it cannot. Either+ * way the copy gets its own inode, so a write in the copy cannot reach the+ * original. `--reflink=auto` is never `--link`: extents are shared+ * copy-on-write, not the file itself. Where there is no `cp`, or a `cp` that+ * has never heard of `--reflink` (BSD, busybox), the pure-Node copier below+ * makes the same copy by the same rules.+ * - Symlinks are reproduced with their target text intact, so a relative link+ * keeps resolving relative to the copy — `react -> .pnpm/react@…` now points+ * at the *copied* virtual store, and a package's own dependency links stay+ * inside it.+ * - An absolute link that points into the original tree is the one case where+ * verbatim text would escape, so it is retargeted at the equivalent path in+ * the copy. An absolute link pointing somewhere else entirely — a workspace+ * package in the repo — is left alone, because that is what it is for and it+ * is not the sandbox.+ *+ * `verifyWorkspaceIsolation` re-derives that claim from the finished copy+ * rather than trusting the construction, and the measurer runs it before any+ * build starts.+ *+ * The workspace is removed when the measurement ends, including when the build+ * fails or the measurement throws.+ */++import * as crypto from 'node:crypto';+import {spawnSync} from 'node:child_process';+import * as fs from 'node:fs';+import * as os from 'node:os';+import * as path from 'node:path';++/**+ * Whether `cp` refused the command line rather than failing to copy.+ *+ * `--reflink` is a GNU coreutils extension. BSD `cp` — macOS, the BSDs — and+ * some busybox builds reject it as an unknown option and exit nonzero before+ * copying anything, which is a platform fact and not a fault. A copy that+ * started and then went wrong reports something else entirely, and must not be+ * mistaken for this: falling back over a real failure would hide it.+ */+function rejectedTheCommandLine(result) {+ const said = `${result.stdout ?? ''}${result.stderr ?? ''}`;+ return (+ /\b(unrecognized|unknown|illegal|invalid) option\b/i.test(said) ||+ /^\s*usage:/im.test(said)+ );+}++/**+ * Throw away a half-written destination, and prove it is gone.+ *+ * A `cp` that got part way through leaves part of the tree behind, and the+ * fallback copier writes into whatever it finds — so falling back over that+ * debris would produce a build root that is neither copy, with stale files the+ * sandbox does not have. The removal is therefore checked rather than assumed:+ * if the directory survives it, there is nothing safe to fall back to and the+ * measurement stops here.+ */+function discardPartialCopy(destination) {+ fs.rmSync(destination, {recursive: true, force: true});+ if (fs.existsSync(destination)) {+ throw new Error(+ `could not clear a partial build root before falling back: ${destination}`,+ );+ }+}++/**+ * Copy a tree with `cp -a --reflink=auto`.+ *+ * `-a` keeps symlinks as symlinks and preserves modes, ownership where+ * permitted, and timestamps. `--reflink=auto` asks for a copy-on-write clone+ * and silently byte-copies when the filesystem has no such thing. There is no+ * `--link` here and there must never be one: a hardlink shares the inode, so a+ * build that rewrites the file rewrites the original.+ *+ * Not every `cp` is GNU coreutils. There is no `cp` at all on some platforms,+ * and on others there is one that has never heard of `--reflink`; both mean+ * the same thing here — use the pure-Node copier instead — so both answer+ * false, having first cleared whatever the attempt left behind. A `cp` that+ * accepted the flag and then failed is a real failure and is raised.+ *+ * @returns true when `cp` did the work, false when it cannot do it.+ */+function copyWithReflink(source, destination) {+ fs.mkdirSync(destination, {recursive: true});+ const result = spawnSync(+ 'cp',+ ['-a', '--reflink=auto', `${source}/.`, destination],+ {encoding: 'utf8'},+ );+ if (result.error?.code === 'ENOENT') {+ discardPartialCopy(destination);+ return false;+ }+ if (result.status !== 0) {+ if (!rejectedTheCommandLine(result)) {+ throw new Error(+ `could not copy the sandbox into a build root: ${+ result.stderr?.trim() || `cp exited ${result.status}`+ }`,+ );+ }+ discardPartialCopy(destination);+ return false;+ }+ return true;+}++/** The same copy in pure Node, for a platform without a usable `cp`. */+function copyWithNode(source, destination) {+ fs.mkdirSync(destination, {recursive: true});+ for (const entry of fs.readdirSync(source, {withFileTypes: true})) {+ const from = path.join(source, entry.name);+ const to = path.join(destination, entry.name);+ if (entry.isSymbolicLink()) {+ fs.symlinkSync(fs.readlinkSync(from), to);+ continue;+ }+ if (entry.isDirectory()) {+ copyWithNode(from, to);+ const stat = fs.lstatSync(from);+ fs.chmodSync(to, stat.mode & 0o777);+ continue;+ }+ if (entry.isFile()) {+ // Never `fs.linkSync`: a hardlink would let a build in the copy rewrite+ // the original's bytes.+ fs.copyFileSync(from, to);+ fs.chmodSync(to, fs.lstatSync(from).mode & 0o777);+ }+ // Sockets, fifos and devices have no place in a build root.+ }+}++/** Whether `candidate` is `root` itself or something inside it. */+function isInside(root, candidate) {+ const relative = path.relative(root, candidate);+ return (+ relative === '' ||+ (!relative.startsWith('..') && !path.isAbsolute(relative))+ );+}++/**+ * Every symlink in `root`, as absolute paths.+ *+ * A directory that is itself a symlink is reported and not descended into, so+ * a link cycle cannot spin this.+ */+function symlinks(root) {+ const found = [];+ const walk = directory => {+ for (const entry of fs.readdirSync(directory, {withFileTypes: true})) {+ const absolute = path.join(directory, entry.name);+ if (entry.isSymbolicLink()) {+ found.push(absolute);+ continue;+ }+ if (entry.isDirectory()) walk(absolute);+ }+ };+ walk(root);+ return found;+}++/**+ * Point any absolute link that reaches into `sourceRoot` at the same place in+ * `destinationRoot` instead.+ *+ * Relative links need nothing: their text already resolves inside the copy,+ * which is why the copy keeps that text verbatim. An absolute link elsewhere on+ * the machine — a workspace package in the repo — is left alone.+ *+ * @returns the links it rewrote, for the record.+ */+export function retargetEscapingLinks(destinationRoot, sourceRoot) {+ const source = path.resolve(sourceRoot);+ const destination = path.resolve(destinationRoot);+ const rewritten = [];+ for (const link of symlinks(destination)) {+ const target = fs.readlinkSync(link);+ if (!path.isAbsolute(target)) continue;+ const resolved = path.resolve(target);+ if (!isInside(source, resolved)) continue;+ const moved = path.join(destination, path.relative(source, resolved));+ fs.rmSync(link);+ fs.symlinkSync(moved, link);+ rewritten.push({link, from: target, to: moved});+ }+ return rewritten;+}++/**+ * Check a finished build root really is sealed off from the sandbox.+ *+ * Construction is one argument and this is the other: it walks the copy and+ * fails on anything that could reach the original — a symlink resolving into+ * it, or a file sharing its inode, which is what a hardlinked copy would look+ * like. `node_modules` is where both would hide, so it is not skipped.+ *+ * @returns {string[]} the problems found; empty means sealed.+ */+export function verifyWorkspaceIsolation(+ buildDir,+ appDir,+ {full = false} = {},+) {+ const source = path.resolve(appDir);+ const destination = path.resolve(buildDir);+ const problems = [];++ for (const link of symlinks(destination)) {+ const relative = path.relative(destination, link);+ const target = fs.readlinkSync(link);+ const resolved = path.resolve(path.dirname(link), target);+ if (isInside(source, resolved)) {+ problems.push(`symlink escapes to the sandbox: ${relative} -> ${target}`);+ }+ }++ // A file the copy shares with the original is a file a build can rewrite+ // under it. Compare inodes on the entries most likely to be written.+ const shared = (relative, mustExist) => {+ const inCopy = path.join(destination, relative);+ const inSource = path.join(source, relative);+ if (!fs.existsSync(inCopy) || !fs.existsSync(inSource)) {+ if (mustExist) problems.push(`missing from the build root: ${relative}`);+ return;+ }+ const copyStat = fs.lstatSync(inCopy);+ if (!copyStat.isFile()) return;+ if (copyStat.ino === fs.lstatSync(inSource).ino) {+ problems.push(`shares an inode with the sandbox: ${relative}`);+ }+ };+ shared('package.json', true);+ for (const candidate of [+ 'node_modules/.modules.yaml',+ 'node_modules/.package-map.json',+ 'node_modules/.pnpm/lock.yaml',+ 'pnpm-lock.yaml',+ ]) {+ shared(candidate, false);+ }++ // `full` compares every regular file's inode instead of a handful. The+ // default is the handful because the copier cannot produce a shared inode —+ // `cp` shares extents copy-on-write and the fallback uses `copyFileSync`,+ // neither of which is a hardlink — so the complete walk is a proof to run in+ // tests rather than a cost to pay on every measurement of a tree with sixty+ // thousand files in it.+ if (full) {+ const walk = (directory, relative) => {+ for (const entry of fs.readdirSync(directory, {withFileTypes: true})) {+ const absolute = path.join(directory, entry.name);+ const key = relative === '' ? entry.name : `${relative}/${entry.name}`;+ if (entry.isSymbolicLink()) continue;+ if (entry.isDirectory()) {+ walk(absolute, key);+ continue;+ }+ if (!entry.isFile()) continue;+ const original = path.join(source, key);+ if (!fs.existsSync(original)) continue;+ const originalStat = fs.lstatSync(original);+ if (!originalStat.isFile()) continue;+ const copyStat = fs.lstatSync(absolute);+ if (+ copyStat.ino === originalStat.ino &&+ copyStat.dev === originalStat.dev+ ) {+ problems.push(`shares an inode with the sandbox: ${key}`);+ }+ }+ };+ walk(destination, '');+ }++ return problems;+}++/**+ * Copy `appDir` into a disposable build root.+ *+ * @returns {{dir: string, cleanup: () => void, retargeted: object[]}} the+ * copy's path, a remove-everything function that is safe to call more than+ * once, and any links that had to be retargeted.+ */+export function createMeasurementWorkspace(+ appDir,+ {prefix = 'setup-measure-', verify = true, strategy = 'auto'} = {},+) {+ const source = path.resolve(appDir);+ if (!fs.existsSync(source)) {+ throw new Error(+ `cannot measure a directory that does not exist: ${source}`,+ );+ }+ const root = fs.mkdtempSync(path.join(os.tmpdir(), prefix));+ const dir = path.join(root, 'app');+ let removed = false;+ const cleanup = () => {+ if (removed) return;+ removed = true;+ fs.rmSync(root, {recursive: true, force: true});+ };+ try {+ // `strategy: 'node'` forces the pure-Node path so tests can exercise the+ // platform that has no usable `cp`.+ if (strategy === 'node' || !copyWithReflink(source, dir)) {+ discardPartialCopy(dir);+ copyWithNode(source, dir);+ }+ const retargeted = retargetEscapingLinks(dir, source);+ if (verify) {+ const problems = verifyWorkspaceIsolation(dir, source, {+ full: verify === 'full',+ });+ if (problems.length > 0) {+ throw new Error(+ `build root is not isolated from the sandbox:\n ${problems.join('\n ')}`,+ );+ }+ }+ return {dir, cleanup, retargeted};+ } catch (error) {+ cleanup();+ throw error;+ }+}++/**+ * A manifest of a tree's bytes and modification times, for proving nothing in+ * it moved.+ *+ * Every entry is recorded: tracked, untracked, and ignored alike, because a+ * build's debris is ignored by construction and that is exactly what has to+ * stay out. Files record size, mode, mtime in nanoseconds, and content hash;+ * symlinks record their target; directories record their own mtime, so an added+ * or deleted child shows up even when no existing file changed.+ */+export function treeManifest(root) {+ const manifest = new Map();+ // `mtimeNs` exists only on bigint stats; a plain `lstatSync` would record+ // `undefined` for every entry and make the timestamp half of this check+ // silently vacuous.+ const lstat = target => fs.lstatSync(target, {bigint: true});+ const walk = (directory, relative) => {+ const stat = lstat(directory);+ manifest.set(relative === '' ? '.' : relative, {+ type: 'dir',+ mtimeNs: String(stat.mtimeNs),+ });+ for (const entry of fs.readdirSync(directory, {withFileTypes: true})) {+ const absolute = path.join(directory, entry.name);+ const key = relative === '' ? entry.name : `${relative}/${entry.name}`;+ const entryStat = lstat(absolute);+ if (entry.isSymbolicLink()) {+ manifest.set(key, {+ type: 'symlink',+ target: fs.readlinkSync(absolute),+ mtimeNs: String(entryStat.mtimeNs),+ });+ continue;+ }+ if (entry.isDirectory()) {+ walk(absolute, key);+ continue;+ }+ if (entry.isFile()) {+ manifest.set(key, {+ type: 'file',+ size: String(entryStat.size),+ mode: String(entryStat.mode & 0o777n),+ mtimeNs: String(entryStat.mtimeNs),+ sha256: crypto+ .createHash('sha256')+ .update(fs.readFileSync(absolute))+ .digest('hex'),+ });+ }+ }+ };+ walk(path.resolve(root), '');+ return manifest;+}++/** The manifest entries that differ, as readable lines. */+export function manifestDifferences(before, after) {+ const differences = [];+ for (const [key, value] of before) {+ if (!after.has(key)) {+ differences.push(`removed: ${key}`);+ continue;+ }+ const other = after.get(key);+ if (JSON.stringify(value) !== JSON.stringify(other)) {+ differences.push(+ `changed: ${key} ${JSON.stringify(value)} -> ${JSON.stringify(other)}`,+ );+ }+ }+ for (const key of after.keys()) {+ if (!before.has(key)) differences.push(`added: ${key}`);+ }+ return differences.sort();+}diff --git a/internal/vibe-tests/setup-test/setup-workspace.test.mjs b/internal/vibe-tests/setup-test/setup-workspace.test.mjsnew file mode 100644index 0000000000..6272fe3f01--- /dev/null+++ b/internal/vibe-tests/setup-test/setup-workspace.test.mjs@@ -0,0 +1,608 @@+// Copyright (c) Meta Platforms, Inc. and affiliates.++/**+ * The measurement build root, against a real pnpm virtual store.+ *+ * The measurer's build must not touch the sandbox it measures: that tree is the+ * attested artifact, and a build in place regenerates the app-owned theme,+ * writes `dist/`, appends to the CLI's invocation log, and caches into+ * `node_modules`, after which the cell no longer hashes to its own digest.+ *+ * A pnpm `node_modules` makes that harder than copying files, because it is+ * mostly links into a virtual store:+ *+ * node_modules/is-odd -> .pnpm/[email protected]/node_modules/is-odd+ * node_modules/.pnpm/<id>/... the real package trees+ * node_modules/.modules.yaml the store's own state+ *+ * Mirroring `.pnpm` with a symlink — which is what this did first — hands the+ * copy a door back into the original: an install or rebuild that rewrites the+ * store writes straight through it. These cases use a real `pnpm install` for+ * the fixture rather than a hand-drawn imitation, because the shape is the+ * thing being tested.+ *+ * The cases are the guarantee, not the construction: the sandbox is unchanged+ * after a build in the copy, the copy is a working tree, and the isolation+ * check catches both ways of getting it wrong. How the copier achieves that is+ * deliberately not asserted.+ */++import {execFileSync, spawnSync} from 'node:child_process';+import fs from 'node:fs';+import os from 'node:os';+import path from 'node:path';+import {afterAll, afterEach, beforeAll, describe, expect, it} from 'vitest';+import {measurementPrivateValues, publicMeasurement} from './setup-measure.mjs';+import {+ createMeasurementWorkspace,+ manifestDifferences,+ retargetEscapingLinks,+ treeManifest,+ verifyWorkspaceIsolation,+} from './setup-workspace.mjs';++const temporaryDirectories = [];+/** One real install, reused: the shape is what matters, not the download. */+let installedFixture = null;++function temporary(prefix) {+ const directory = fs.mkdtempSync(path.join(os.tmpdir(), prefix));+ temporaryDirectories.push(directory);+ return directory;+}++function write(root, relativePath, content, mode) {+ const file = path.join(root, relativePath);+ fs.mkdirSync(path.dirname(file), {recursive: true});+ fs.writeFileSync(file, content);+ if (mode !== undefined) fs.chmodSync(file, mode);+}++function pnpm(args, cwd) {+ return spawnSync('pnpm', args, {+ cwd,+ encoding: 'utf8',+ env: {...process.env, CI: 'true'},+ stdio: ['ignore', 'pipe', 'pipe'],+ });+}++/** Whether a real pnpm is reachable; without one these cases cannot run. */+function pnpmAvailable() {+ const result = pnpm(['--version'], os.tmpdir());+ return result.status === 0 && /^\d+\./.test((result.stdout ?? '').trim());+}++const HAS_PNPM = pnpmAvailable();+const describeWithPnpm = HAS_PNPM ? describe : describe.skip;++/**+ * A sandbox with a real pnpm `node_modules`, plus the files a measured arm+ * carries: a committed generated theme artifact, ignored build output, the+ * CLI's invocation log, an executable script, and a link out to a workspace+ * package.+ */+function installedSandbox() {+ if (!installedFixture) {+ const root = fs.mkdtempSync(path.join(os.tmpdir(), 'setup-workspace-src-'));+ temporaryDirectories.push(root);+ const app = path.join(root, 'app');+ fs.mkdirSync(app, {recursive: true});+ write(+ app,+ 'package.json',+ `${JSON.stringify(+ {+ name: 'host',+ private: true,+ // `is-odd` pulls `is-number`, so the store has a package that depends+ // on another one and the relative link between them is exercised.+ dependencies: {'is-odd': '3.0.1'},+ scripts: {build: 'node build.mjs'},+ },+ null,+ 2,+ )}\n`,+ );+ const install = pnpm(['install', '--ignore-scripts'], app);+ if (install.status !== 0) {+ throw new Error(`fixture install failed: ${install.stderr}`);+ }+ installedFixture = {root, app};+ }++ // Each case gets its own copy of that installed tree, so one case's install+ // cannot be seen by the next.+ const root = temporary('setup-workspace-test-');+ const app = path.join(root, 'app');+ execFileSync('cp', ['-a', '--reflink=auto', installedFixture.app, app]);++ write(app, '.gitignore', 'node_modules/\ndist/\n');+ write(app, 'index.html', '<!doctype html>\n<html lang="en"></html>\n');+ write(app, 'src/App.tsx', 'export const App = () => null;\n');+ write(app, 'src/index.css', "@import 'tailwindcss';\n");+ write(+ app,+ 'src/apptheme.css',+ '/* @generated by `astryx theme build` */\n:root { color-scheme: light dark; }\n',+ );+ write(app, 'src/apptheme.d.ts', 'export declare const theme: unknown;\n');+ write(app, 'dist/index.html', '<!doctype html>\n');+ write(app, 'dist/assets/app.css', '@layer reset, theme;\n');+ write(app, '.astryx-invocations.log', 'theme build\n');+ write(app, 'scripts/run.sh', '#!/bin/sh\necho ok\n', 0o755);+ fs.symlinkSync('./index.css', path.join(app, 'src/alias.css'));++ // A workspace package linked in by absolute path, the way the canonical+ // harness links the design system's own source into an arm.+ const workspace = path.join(root, 'workspace-package');+ fs.mkdirSync(workspace, {recursive: true});+ write(workspace, 'package.json', '{"name": "@scope/workspace"}\n');+ fs.mkdirSync(path.join(app, 'node_modules/@scope'), {recursive: true});+ fs.symlinkSync(workspace, path.join(app, 'node_modules/@scope/workspace'));++ return {root, app, workspace};+}++/**+ * A build script that does to its own tree exactly what a real one does:+ * regenerate the theme, write dist, append to the log, cache in node_modules.+ */+function withBuildScript(app, {fails = false} = {}) {+ write(+ app,+ 'build.mjs',+ [+ "import * as fs from 'node:fs';",+ // A build that finds no TTY must not ask anything or fall over.+ "if (process.stdout.isTTY) throw new Error('unexpected TTY');",+ "fs.writeFileSync('src/apptheme.css', '/* rebuilt */\\n');",+ "fs.mkdirSync('dist/assets', {recursive: true});",+ "fs.writeFileSync('dist/index.html', '<!doctype html>\\n');",+ "fs.writeFileSync('dist/assets/app.css', '@layer reset, theme;\\n');",+ "fs.appendFileSync('.astryx-invocations.log', 'build\\n');",+ "fs.mkdirSync('node_modules/.vite', {recursive: true});",+ "fs.writeFileSync('node_modules/.vite/cache.json', '{}\\n');",+ // Rewrite a file inside the virtual store, which is the write that used+ // to reach the original through the mirrored `.pnpm` link.+ "fs.writeFileSync('node_modules/.pnpm/[email protected]/node_modules/is-odd/index.js', 'module.exports = 1;\\n');",+ ...(fails ? ['process.exit(1);'] : []),+ ].join('\n'),+ );+}++/** The whole tree, hashed — including every file in the virtual store. */+const manifest = root => treeManifest(root);++afterEach(() => {+ while (temporaryDirectories.length > 0) {+ const directory = temporaryDirectories.pop();+ if (installedFixture && directory === installedFixture.root) continue;+ fs.rmSync(directory, {recursive: true, force: true});+ }+});++afterAll(() => {+ if (installedFixture) {+ fs.rmSync(installedFixture.root, {recursive: true, force: true});+ installedFixture = null;+ }+});++describeWithPnpm('measurement build root over a pnpm virtual store', () => {+ beforeAll(() => {+ // Warm the shared install once so the first case is not timed with it.+ installedSandbox();+ }, 300_000);++ it.each([+ {name: 'a build that succeeds', fails: false},+ {name: 'a build that fails', fails: true},+ ])(+ 'leaves the sandbox byte-identical after $name in the copy',+ ({fails}) => {+ const {app} = installedSandbox();+ withBuildScript(app, {fails});+ const before = manifest(app);+ const copy = createMeasurementWorkspace(app);++ const result = pnpm(['build'], copy.dir);+ expect(result.status).toBe(fails ? 1 : 0);+ // Non-interactive: nothing complained about a missing terminal.+ expect(`${result.stdout}${result.stderr}`).not.toMatch(/TTY|tty/);++ // The build really did write, in the copy — including into the virtual+ // store, which is the write that used to reach the original.+ expect(+ fs.readFileSync(path.join(copy.dir, 'src/apptheme.css'), 'utf8'),+ ).toBe('/* rebuilt */\n');+ expect(+ fs.readFileSync(+ path.join(+ copy.dir,+ 'node_modules/.pnpm/[email protected]/node_modules/is-odd/index.js',+ ),+ 'utf8',+ ),+ ).toBe('module.exports = 1;\n');++ // And none of it reached the sandbox, virtual store included.+ expect(manifestDifferences(before, manifest(app))).toEqual([]);++ copy.cleanup();+ expect(manifestDifferences(before, manifest(app))).toEqual([]);+ },+ 300_000,+ );++ it('survives an install in the copy recreating the whole virtual store', () => {+ const {app} = installedSandbox();+ const before = manifest(app);+ const copy = createMeasurementWorkspace(app);++ // The strongest case: throw the copy's store away and have pnpm rebuild+ // it. Through a mirrored `.pnpm` link this would have deleted and rewritten+ // the sandbox's own store.+ fs.rmSync(path.join(copy.dir, 'node_modules/.pnpm'), {+ recursive: true,+ force: true,+ });+ const install = pnpm(['install', '--ignore-scripts'], copy.dir);+ expect(install.status, install.stderr).toBe(0);++ expect(manifestDifferences(before, manifest(app))).toEqual([]);+ copy.cleanup();+ expect(manifestDifferences(before, manifest(app))).toEqual([]);+ }, 300_000);++ it('hands back a working tree, resolving inside itself', () => {+ const {app, workspace: workspacePackage} = installedSandbox();+ const copy = createMeasurementWorkspace(app);++ // Module resolution works from the copy, and lands in the copy.+ const resolved = spawnSync(+ process.execPath,+ ['-e', "process.stdout.write(require.resolve('is-odd'))"],+ {cwd: copy.dir, encoding: 'utf8'},+ );+ expect(resolved.status, resolved.stderr).toBe(0);+ expect(resolved.stdout.startsWith(fs.realpathSync(copy.dir))).toBe(true);++ // A link out to a workspace package is left pointing where it pointed: it+ // is not the sandbox, and rewriting it would break resolution.+ expect(+ fs.realpathSync(path.join(copy.dir, 'node_modules/@scope/workspace')),+ ).toBe(fs.realpathSync(workspacePackage));++ // Nothing anywhere in the copy can reach the sandbox.+ expect(verifyWorkspaceIsolation(copy.dir, app, {full: true})).toEqual([]);++ copy.cleanup();+ }, 120_000);++ it('copies the same way on a platform without a usable cp', () => {+ // The pure-Node copier is a second implementation of the same guarantee,+ // and no other case reaches it.+ const {app} = installedSandbox();+ const before = manifest(app);+ const copy = createMeasurementWorkspace(app, {strategy: 'node'});++ expect(verifyWorkspaceIsolation(copy.dir, app, {full: true})).toEqual([]);+ fs.writeFileSync(+ path.join(+ copy.dir,+ 'node_modules/.pnpm/[email protected]/node_modules/is-odd/index.js',+ ),+ 'module.exports = 2;\n',+ );+ expect(manifestDifferences(before, manifest(app))).toEqual([]);++ copy.cleanup();+ }, 300_000);++ it('removes the build root even when the measurement throws', () => {+ const {app} = installedSandbox();+ const before = manifest(app);+ const copy = createMeasurementWorkspace(app);++ expect(() => {+ try {+ fs.writeFileSync(path.join(copy.dir, 'dist/index.html'), 'partial\n');+ throw new Error('measurement failed');+ } finally {+ copy.cleanup();+ }+ }).toThrow('measurement failed');++ expect(fs.existsSync(copy.dir)).toBe(false);+ expect(manifestDifferences(before, manifest(app))).toEqual([]);+ }, 120_000);+});++/**+ * The mutations: build the copy the two wrong ways and show each is caught.+ *+ * Without these the cases above could be passing for the wrong reason. There+ * are exactly two ways for a copy to stay reachable from the sandbox — a link+ * that resolves back into it, and a file that shares its inode — so there is+ * one mutation for each, and no more.+ */+describeWithPnpm('a copy that is not a copy is caught', () => {+ it('mirroring the virtual store writes through, and is reported', () => {+ const {app} = installedSandbox();+ const packageFile =+ 'node_modules/.pnpm/[email protected]/node_modules/is-odd/index.js';+ const before = manifest(app);++ // The defective copier: real dirs, but `node_modules` entries symlinked+ // back at the original — which for pnpm means the whole virtual store.+ const mirrored = path.join(temporary('setup-workspace-mutation-'), 'app');+ fs.mkdirSync(mirrored, {recursive: true});+ for (const entry of fs.readdirSync(app, {withFileTypes: true})) {+ const from = path.join(app, entry.name);+ const to = path.join(mirrored, entry.name);+ if (entry.name === 'node_modules') {+ fs.mkdirSync(to, {recursive: true});+ for (const inner of fs.readdirSync(from, {withFileTypes: true})) {+ fs.symlinkSync(path.join(from, inner.name), path.join(to, inner.name));+ }+ continue;+ }+ if (entry.isSymbolicLink()) {+ fs.symlinkSync(fs.readlinkSync(from), to);+ continue;+ }+ if (entry.isDirectory()) {+ execFileSync('cp', ['-a', from, to]);+ continue;+ }+ fs.copyFileSync(from, to);+ }++ // `.pnpm` is a link to the original store, so this write lands there.+ fs.writeFileSync(+ path.join(mirrored, packageFile),+ 'module.exports = "written through";\n',+ );+ expect(+ manifestDifferences(before, manifest(app)).join('\n'),+ ).toMatch(/changed: node_modules\/\.pnpm\//);++ // And the isolation check refuses that construction, so a change that+ // reintroduces mirroring cannot ship quietly.+ expect(verifyWorkspaceIsolation(mirrored, app).join('\n')).toMatch(+ /symlink escapes to the sandbox: node_modules\/\.pnpm/,+ );+ }, 120_000);++ it('hardlinking instead of copying is reported', () => {+ const {app} = installedSandbox();+ const dir = path.join(temporary('setup-workspace-hardlink-'), 'app');+ // The other way to get this wrong: the bytes look right and the inode is+ // shared, so a rewrite in the copy reaches the original.+ execFileSync('cp', ['-al', app, dir]);++ expect(+ verifyWorkspaceIsolation(dir, app, {full: true}).join('\n'),+ ).toMatch(/shares an inode with the sandbox/);+ }, 120_000);+});++/**+ * The manifest is the instrument every proof above depends on, so it has to be+ * able to fail. A build's own writes are covered end to end by those cases;+ * these are the two differences a content hash alone would miss.+ */+describe('tree manifest', () => {+ function plainSandbox() {+ const app = path.join(temporary('setup-manifest-test-'), 'app');+ write(app, 'src/App.tsx', 'export const App = () => null;\n');+ write(app, 'scripts/run.sh', '#!/bin/sh\n', 0o755);+ return app;+ }++ it.each([+ {+ name: 'a file touched without its bytes changing',+ mutate: app => {+ const when = new Date(Date.now() + 5_000);+ fs.utimesSync(path.join(app, 'src/App.tsx'), when, when);+ },+ expected: /changed: src\/App\.tsx/,+ },+ {+ name: 'a file whose mode changed and nothing else',+ mutate: app => fs.chmodSync(path.join(app, 'scripts/run.sh'), 0o644),+ expected: /changed: scripts\/run\.sh/,+ },+ ])('reports $name', ({mutate, expected}) => {+ const app = plainSandbox();+ const before = manifest(app);+ mutate(app);+ expect(manifestDifferences(before, manifest(app)).join('\n')).toMatch(+ expected,+ );+ });+});++describe('build root boundaries', () => {+ it('redacts the build root as well as the sandbox', () => {+ const root = temporary('setup-redaction-test-');+ const app = path.join(root, 'app');+ write(app, 'package.json', '{"name": "host"}\n');+ const workspace = createMeasurementWorkspace(app);+ const privateValues = measurementPrivateValues({+ appDir: app,+ outFile: path.join(root, 'measurement.json'),+ buildDir: workspace.dir,+ });+ expect(privateValues).toContain(workspace.dir);+ expect(privateValues).toContain(path.dirname(workspace.dir));++ const exported = publicMeasurement(+ {+ label: 'arm',+ app,+ build: {+ ok: false,+ status: 1,+ ms: 1,+ stdout: `building in ${workspace.dir}\n`,+ stderr: `failed in ${workspace.dir}/src\n`,+ },+ schemes: {},+ },+ {fixtureId: 'enterprise-scoped-synthetic', privateValues},+ );++ const serialized = JSON.stringify(exported);+ expect(serialized).not.toContain(workspace.dir);+ expect(serialized).not.toContain(path.dirname(workspace.dir));+ expect(serialized).not.toContain(app);+ expect(exported.app).toBe('enterprise-scoped-synthetic');++ workspace.cleanup();+ });++ it('moves only the links that would escape, and leaves the rest alone', () => {+ const root = temporary('setup-retarget-test-');+ const app = path.join(root, 'app');+ write(app, 'package.json', '{"name": "host"}\n');+ write(app, 'src/index.css', 'body {}\n');+ fs.symlinkSync('./index.css', path.join(app, 'src/relative.css'));+ fs.symlinkSync(+ path.join(app, 'src/index.css'),+ path.join(app, 'absolute.css'),+ );+ // An absolute link to somewhere that is not the sandbox stays as it is.+ write(root, 'outside.txt', 'shared\n');+ fs.symlinkSync(+ path.join(root, 'outside.txt'),+ path.join(app, 'outside.txt'),+ );++ const copy = path.join(root, 'copy');+ execFileSync('cp', ['-a', app, copy]);+ const rewritten = retargetEscapingLinks(copy, app);++ expect(fs.readlinkSync(path.join(copy, 'src/relative.css'))).toBe(+ './index.css',+ );+ expect(fs.readlinkSync(path.join(copy, 'absolute.css'))).toBe(+ path.join(copy, 'src/index.css'),+ );+ expect(fs.readlinkSync(path.join(copy, 'outside.txt'))).toBe(+ path.join(root, 'outside.txt'),+ );+ expect(rewritten.map(entry => path.relative(copy, entry.link))).toEqual([+ 'absolute.css',+ ]);+ });++ it('refuses a sandbox that is not there, and cleans up idempotently', () => {+ expect(() =>+ createMeasurementWorkspace(path.join(os.tmpdir(), 'no-such-sandbox-xyz')),+ ).toThrow(/does not exist/);++ const app = path.join(temporary('setup-cleanup-test-'), 'app');+ write(app, 'package.json', '{"name": "host"}\n');+ const workspace = createMeasurementWorkspace(app);+ workspace.cleanup();+ expect(() => workspace.cleanup()).not.toThrow();+ });+});++/**+ * `--reflink` is GNU coreutils only. A BSD or busybox `cp` rejects it outright+ * — nonzero, nothing copied — and that is a platform, not a fault, so it has+ * to reach the pure-Node copier the same way a missing `cp` does. The+ * distinction that matters is the other one: a `cp` that took the flag and+ * then failed part way through has left debris, and must not be quietly+ * copied over.+ */+describe('a cp that does not support --reflink', () => {+ /**+ * Put a fake `cp` first on PATH. It writes `left-behind.txt` into the+ * destination before failing, so a fallback that did not clear the+ * destination first would be visible in the finished copy.+ */+ function shimCp(script) {+ const bin = temporary('setup-cp-shim-');+ const shim = path.join(bin, 'cp');+ fs.writeFileSync(+ shim,+ [+ '#!/bin/sh',+ 'dest=""',+ 'for arg in "$@"; do dest="$arg"; done',+ 'mkdir -p "$dest"',+ 'printf %s "debris" > "$dest/left-behind.txt"',+ script,+ ].join('\n'),+ );+ fs.chmodSync(shim, 0o755);+ const original = process.env.PATH;+ process.env.PATH = `${bin}${path.delimiter}${original}`;+ return () => {+ process.env.PATH = original;+ };+ }++ function plainSandbox() {+ const app = path.join(temporary('setup-reflink-test-'), 'app');+ write(app, 'package.json', '{"name": "host"}\n');+ write(app, 'src/index.css', 'body {}\n');+ write(app, 'scripts/run.sh', '#!/bin/sh\necho ok\n', 0o755);+ fs.symlinkSync('./index.css', path.join(app, 'src/alias.css'));+ return app;+ }++ it('falls back to a complete copy instead of failing the measurement', () => {+ const app = plainSandbox();+ const restore = shimCp(+ [+ // What busybox says; BSD `cp` says `illegal option` and a usage line.+ 'echo "cp: unrecognized option \'--reflink=auto\'" >&2',+ 'exit 1',+ ].join('\n'),+ );+ try {+ const copy = createMeasurementWorkspace(app);++ // A real copy: contents, modes and symlink text all came across.+ expect(fs.readFileSync(path.join(copy.dir, 'src/index.css'), 'utf8')).toBe(+ 'body {}\n',+ );+ expect(+ fs.lstatSync(path.join(copy.dir, 'scripts/run.sh')).mode & 0o777,+ ).toBe(0o755);+ expect(fs.readlinkSync(path.join(copy.dir, 'src/alias.css'))).toBe(+ './index.css',+ );++ // The failed attempt's debris was cleared, not copied over.+ expect(fs.existsSync(path.join(copy.dir, 'left-behind.txt'))).toBe(false);+ expect(verifyWorkspaceIsolation(copy.dir, app, {full: true})).toEqual([]);++ copy.cleanup();+ } finally {+ restore();+ }+ });++ it('still raises a copy that really failed', () => {+ const app = plainSandbox();+ const restore = shimCp(+ ['echo "cp: cannot stat \'src\': Input/output error" >&2', 'exit 1'].join(+ '\n',+ ),+ );+ try {+ expect(() => createMeasurementWorkspace(app)).toThrow(+ /could not copy the sandbox into a build root: cp: cannot stat/,+ );+ } finally {+ restore();+ }+ });+});