Skip to content

The open-source app everyone uses to manage agents at work

[codex] Document Storybook visual baseline platform lock (#9216)

Dottad3919713bc9e
diff --git a/doc/DEVELOPING.md b/doc/DEVELOPING.md
index a0a6100cec..51f4b35daa 100644
--- a/doc/DEVELOPING.md
+++ b/doc/DEVELOPING.md
@@ -70,6 +70,14 @@ Accepted visual changes should update the manifest metadata and publish a new
immutable archive with `pnpm storybook-visual:baseline pack` and
`pnpm storybook-visual:baseline upload`; do not commit generated PNG snapshots.
+Known limitation: Storybook visual baselines are Linux/Ubuntu-only. The manifest
+pins the capture environment to `ubuntu-24.04` and the Playwright suite uses
+pixel-exact comparison, so local runs on macOS, Windows, or other non-matching
+platforms can report false-positive diffs from font rasterization and subpixel
+rendering. Use the `Storybook Visual` GitHub Actions workflow on `ubuntu-latest`
+as the source of truth, or run locally in a matching Linux environment before
+accepting or updating baselines.
+
PR visual checks are opt-in while the suite stabilizes. Add the
`storybook-visual` label to a PR, or run the `Storybook Visual` GitHub Actions
workflow manually, to produce downloadable Playwright report/test-result
diff --git a/scripts/__tests__/serve-storybook-static.test.mjs b/scripts/__tests__/serve-storybook-static.test.mjs
new file mode 100644
index 0000000000..4d5a17a4bc
--- /dev/null
+++ b/scripts/__tests__/serve-storybook-static.test.mjs
@@ -0,0 +1,16 @@
+import assert from "node:assert/strict";
+import { spawnSync } from "node:child_process";
+import test from "node:test";
+
+const script = new URL("../serve-storybook-static.mjs", import.meta.url).pathname;
+
+test("--port followed by another flag is not parsed as the port value", () => {
+ const result = spawnSync(process.execPath, [script, "--port", "--unused-flag"], {
+ env: { ...process.env, PORT: "65536" },
+ encoding: "utf8",
+ });
+
+ assert.equal(result.status, 1);
+ assert.match(result.stderr, /Invalid Storybook static server port: 65536/);
+ assert.doesNotMatch(result.stderr, /--unused-flag/);
+});
diff --git a/scripts/serve-storybook-static.mjs b/scripts/serve-storybook-static.mjs
index aa0c22f5cd..50d5c93526 100644
--- a/scripts/serve-storybook-static.mjs
+++ b/scripts/serve-storybook-static.mjs
@@ -13,7 +13,20 @@ const root = resolve(
"ui",
"storybook-static",
);
-const port = Number(process.env.PORT ?? 6106);
+const portArgIndex = process.argv.indexOf("--port");
+const explicitPort =
+ portArgIndex >= 0 &&
+ process.argv[portArgIndex + 1] &&
+ !process.argv[portArgIndex + 1].startsWith("--")
+ ? process.argv[portArgIndex + 1]
+ : null;
+const portSource = explicitPort ?? process.env.PORT ?? 6106;
+const port = Number(portSource);
+
+if (!Number.isInteger(port) || port <= 0 || port > 65535) {
+ console.error(`Invalid Storybook static server port: ${portSource}`);
+ process.exit(1);
+}
if (!existsSync(join(root, "index.html"))) {
console.error(`No built Storybook at ${root}. Run \`pnpm build-storybook\` first.`);
diff --git a/tests/storybook-visual/README.md b/tests/storybook-visual/README.md
index 3f4b8220af..19fb4ef67e 100644
--- a/tests/storybook-visual/README.md
+++ b/tests/storybook-visual/README.md
@@ -18,6 +18,19 @@ pnpm test:storybook-visual:update
`tests/storybook-visual/.snapshots/`, and checks the PNG count. The same snapshot
directory can be overridden with `STORYBOOK_VISUAL_SNAPSHOT_DIR`.
+## Known Limitation: Linux Baselines
+
+Storybook visual baselines are platform-locked. The checked-in manifest records
+the capture environment as `ubuntu-24.04`, and Playwright compares screenshots
+with `maxDiffPixels: 0`. Pixel-exact results are only meaningful when local runs
+use the same Linux/Ubuntu capture platform as the baseline.
+
+macOS, Windows, and other non-matching local environments can produce
+false-positive diffs from font rasterization and subpixel rendering differences.
+Use the `Storybook Visual` GitHub Actions workflow on `ubuntu-latest` as the
+source of truth for cross-platform review, or run the suite locally in a matching
+Linux environment before accepting or updating baselines.
+
## CI and Review Artifacts
Storybook visual tests are opt-in while the suite stabilizes. Add the
diff --git a/tests/storybook-visual/playwright.config.ts b/tests/storybook-visual/playwright.config.ts
index 74bfad7c84..b942bc7798 100644
--- a/tests/storybook-visual/playwright.config.ts
+++ b/tests/storybook-visual/playwright.config.ts
@@ -36,7 +36,7 @@ export default defineConfig({
baseURL: "http://localhost:6106",
},
webServer: {
- command: "node ../../scripts/serve-storybook-static.mjs",
+ command: "node ../../scripts/serve-storybook-static.mjs --port 6106",
url: "http://localhost:6106/index.json",
reuseExistingServer: true,
timeout: 30_000,