Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

perf: add --route mode to the browser audit for unauthenticated UI load and idle traffic (#12998)

[._.]/ Adam Eivy8d0f312095a5
diff --git a/scripts/perf/README.md b/scripts/perf/README.md
index 0ac660491a..3ff22186db 100644
--- a/scripts/perf/README.md
+++ b/scripts/perf/README.md
@@ -161,6 +161,26 @@ Run deterministic accounting and regression-gate tests from `server/`:
npm test -- ../scripts/perf/collectionTraffic.test.js
~~~
+### Route mode (`--route`)
+
+When the live UI is password-protected the scheduled performance lane's browser
+gets 401s, so UI request counts and idle bytes cannot be measured there. Run the
+same CDP accounting against the passwordless synthetic fixture for any route:
+
+~~~sh
+npm run build
+PGDATABASE=portos_test node scripts/perf/browserCollectionAudit.mjs --route=/ --sibling=/messages/contacts
+~~~
+
+`--route` (absolute path) is cold-loaded, then `--sibling` is loaded (default
+`/messages/contacts`), then a 60 s idle window runs (`--idle-ms=1000..300000` for
+quick checks). An optional positional output file works as before. The JSON report
+holds, per phase (`cold`, `sibling`, `idle`), requests, encoded/decoded bytes and
+WebSocket frames/payload bytes per endpoint pattern, plus `idleBytesPerMinute`.
+Numeric and long-hex path segments collapse to `:id`; query strings, hosts, tokens
+and bodies are never retained. Fixture-unavailable APIs (503) are counted under
+`unavailable`; this is shell/dashboard traffic shape, not production endpoint timing.
+
### Findings
The initial synthetic run identified media contract failures tracked in #8292:
diff --git a/scripts/perf/browserCollectionAudit.mjs b/scripts/perf/browserCollectionAudit.mjs
index dd7085cfc9..e82477bbd6 100644
--- a/scripts/perf/browserCollectionAudit.mjs
+++ b/scripts/perf/browserCollectionAudit.mjs
@@ -4,19 +4,20 @@ import { writeFile } from 'node:fs/promises';
import { setTimeout as delay } from 'node:timers/promises';
import { performance } from 'node:perf_hooks';
import { startCollectionFixture } from './collectionFixture.js';
-import { createTrafficAccumulator, endpointFor, inspectCollection, trafficFailures } from './collectionTraffic.js';
+import { createTrafficAccumulator, endpointFor, idleBytesPerMinute, inspectCollection, parseAuditArgs, trafficFailures } from './collectionTraffic.js';
const { chromium } = createRequire(new URL('../../server/package.json', import.meta.url))('playwright-core');
const events = ['requestWillBeSent', 'responseReceived', 'dataReceived', 'loadingFinished', 'loadingFailed',
'webSocketCreated', 'webSocketClosed', 'webSocketFrameSent', 'webSocketFrameReceived'];
-const output = process.argv[2];
+const args = parseAuditArgs(process.argv.slice(2));
+const output = args.output;
let browser;
let fixture;
const abort = new AbortController();
const stop = () => { abort.abort(); browser?.close().catch(() => {}); };
process.once('SIGINT', stop);
process.once('SIGTERM', stop);
-const report = { version: 1, synthetic: true, idleTargetMs: 60000, scenarios: [], failures: [] };
+const report = { version: 1, synthetic: true, idleTargetMs: args.idleMs, scenarios: [], failures: [] };
async function mediaWindow(page, cdp, rows, result) {
const samples = result.windowSamples = [];
@@ -184,15 +185,7 @@ async function scenario(route) {
result.listResponses = listResponses;
result.returnedRows = returnedRows;
result.details = details;
- if (result.idleMs) {
- const entries = Object.values(result.phases.idle || {});
- const total = key => entries.reduce((sum, value) => sum + value[key], 0);
- result.idleBytesPerMinute = {
- encoded: Math.round(total('encodedBytes') * 60000 / result.idleMs),
- decoded: Math.round(total('decodedBytes') * 60000 / result.idleMs),
- socketPayload: Math.round((total('socketSentBytes') + total('socketReceivedBytes')) * 60000 / result.idleMs),
- };
- }
+ if (result.idleMs) result.idleBytesPerMinute = idleBytesPerMinute(result.phases.idle, result.idleMs);
failures.push(...trafficFailures(result.phases));
report.failures.push(...failures);
report.scenarios.push(result);
@@ -200,14 +193,58 @@ async function scenario(route) {
}
}
+// --route mode: generic cold-load + sibling + idle accounting for any route, so
+// the performance lane can measure shell traffic when the live UI is behind a
+// password. Only endpoint patterns and aggregates enter the report.
+async function routeScenario(route, sibling) {
+ const context = await browser.newContext({ viewport: { width: 1280, height: 800 }, serviceWorkers: 'block' });
+ const page = await context.newPage();
+ page.setDefaultTimeout(15000);
+ const cdp = await context.newCDPSession(page);
+ const traffic = createTrafficAccumulator();
+ const result = { route: endpointFor(fixture.url + route), sibling: endpointFor(fixture.url + sibling), usefulContentMs: null, phases: {} };
+ let phase = 'cold';
+ const advance = name => { phase = name; traffic.setPhase(name); };
+ try {
+ for (const name of events) cdp.on('Network.' + name, event => traffic.event(name, event));
+ await cdp.send('Network.enable');
+ await cdp.send('Network.setCacheDisabled', { cacheDisabled: true });
+ const started = performance.now();
+ await page.goto(fixture.url + route, { waitUntil: 'domcontentloaded' });
+ await page.locator('main, #root > *').first().waitFor({ state: 'visible' });
+ result.usefulContentMs = Math.round(performance.now() - started);
+ await delay(2000, undefined, { signal: abort.signal });
+ const dismiss = page.getByRole('button', { name: 'Dismiss and don’t show again', exact: true });
+ if (await dismiss.isVisible()) await dismiss.click();
+ advance('sibling');
+ await page.goto(fixture.url + sibling, { waitUntil: 'domcontentloaded' });
+ await delay(2000, undefined, { signal: abort.signal });
+ advance('idle');
+ const idleStart = performance.now();
+ await delay(args.idleMs, undefined, { signal: abort.signal });
+ result.idleMs = Math.round(performance.now() - idleStart);
+ } catch {
+ report.failures.push({ endpoint: result.route, reason: 'browser-stage-' + phase, bytes: 0 });
+ } finally {
+ result.phases = traffic.snapshot();
+ result.pendingRequests = traffic.pending();
+ if (result.idleMs) result.idleBytesPerMinute = idleBytesPerMinute(result.phases.idle, result.idleMs);
+ report.scenarios.push(result);
+ await context.close();
+ }
+}
+
try {
fixture = await startCollectionFixture({ clientDist: process.env.COLLECTION_AUDIT_CLIENT_DIST });
report.startup = fixture.startup;
report.cardinalities = fixture.cardinalities;
browser = await chromium.launch({ headless: true, ...(process.env.COLLECTION_AUDIT_BROWSER
? { executablePath: process.env.COLLECTION_AUDIT_BROWSER } : { channel: 'chrome' }) });
- await scenario('/media/history');
- await scenario('/messages/inbox');
+ if (args.route) await routeScenario(args.route, args.sibling);
+ else {
+ await scenario('/media/history');
+ await scenario('/messages/inbox');
+ }
} catch (error) {
if (error.fixtureStartup) report.startup = error.fixtureStartup;
report.failures.push({ endpoint: fixture ? 'browser' : 'fixture',
diff --git a/scripts/perf/collectionTraffic.js b/scripts/perf/collectionTraffic.js
index 63a181ab35..692050f231 100644
--- a/scripts/perf/collectionTraffic.js
+++ b/scripts/perf/collectionTraffic.js
@@ -6,8 +6,9 @@ export function endpointFor(rawUrl) {
if (pathname.startsWith('/data/')) return '/data/:asset';
if (/^\/api\/video-gen\/history\/[^/]+$/.test(pathname)) return '/api/video-gen/history/:record';
if (/^\/api\/messages\/[^/]+\/[^/]+$/.test(pathname)) return '/api/messages/:account/:message';
- // Collection endpoints and bootstrap paths contain no record identifiers.
- return pathname.replace(/[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}/gi, ':account')
+ // Collection endpoints and bootstrap paths contain no record identifiers;
+ // arbitrary --route traffic may, so opaque numeric/hex segments collapse too.
+ return pathname.replace(/\/(?:\d+|[a-f0-9]{16,})(?=\/|$)/gi, '/:id').replace(/[a-f0-9]{8}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{4}-[a-f0-9]{12}/gi, ':account')
.replace(/synthetic-[^/]+/g, ':record');
}
@@ -126,3 +127,36 @@ export function trafficFailures(phases) {
}
return failures;
}
+
+/** Idle HTTP/socket bytes scaled to one minute from the measured window. */
+export function idleBytesPerMinute(idle, idleMs) {
+ const entries = Object.values(idle || {});
+ const total = key => entries.reduce((sum, value) => sum + value[key], 0);
+ const scale = bytes => Math.round(bytes * 60000 / idleMs);
+ return { encoded: scale(total('encodedBytes')), decoded: scale(total('decodedBytes')),
+ socketPayload: scale(total('socketSentBytes') + total('socketReceivedBytes')) };
+}
+
+/**
+ * CLI: [output] [--route=/path] [--sibling=/path] [--idle-ms=N]. Routes must be
+ * same-origin absolute paths so the audit can never reach another host.
+ */
+export function parseAuditArgs(argv) {
+ const out = { output: undefined, route: null, sibling: '/messages/contacts', idleMs: 60000 };
+ const path = (name, value) => {
+ if (!/^\/(?!\/)[^\s\\]*$/.test(value || '')) throw new Error('--' + name + ' must be an absolute path such as /');
+ return value;
+ };
+ for (const arg of argv) {
+ const [flag, ...rest] = arg.split('=');
+ const value = rest.join('=');
+ if (flag === '--route') out.route = path('route', value);
+ else if (flag === '--sibling') out.sibling = path('sibling', value);
+ else if (flag === '--idle-ms') {
+ out.idleMs = Number(value);
+ if (!Number.isInteger(out.idleMs) || out.idleMs < 1000 || out.idleMs > 300000) throw new Error('--idle-ms must be 1000-300000');
+ } else if (!arg.startsWith('--') && out.output === undefined) out.output = arg;
+ else throw new Error('Unknown argument: ' + flag);
+ }
+ return out;
+}
diff --git a/scripts/perf/collectionTraffic.test.js b/scripts/perf/collectionTraffic.test.js
index 52fa0388d4..ddceacec54 100644
--- a/scripts/perf/collectionTraffic.test.js
+++ b/scripts/perf/collectionTraffic.test.js
@@ -1,5 +1,5 @@
import { describe, expect, it } from 'vitest';
-import { createTrafficAccumulator, endpointFor, inspectCollection, trafficFailures } from './collectionTraffic.js';
+import { createTrafficAccumulator, endpointFor, idleBytesPerMinute, inspectCollection, parseAuditArgs, trafficFailures } from './collectionTraffic.js';
import { CARDINALITIES, messageFixture, mediaFixture } from './collectionFixtureData.js';
const start = (t, id = 'one', path = '/api/messages/inbox?secret=never-retain') => t.event('requestWillBeSent', { requestId: id, request: { url: 'http://example.com' + path } });
@@ -67,3 +67,33 @@ describe('collection regression gates', () => {
expect(trafficFailures(t.snapshot()).map(f => f.reason)).toEqual(['sibling-inbox-fetch', 'eager-detail-request', 'full-history-fetch']);
});
});
+
+describe('--route report mode', () => {
+ it('reports arbitrary-route traffic by endpoint pattern with no hosts, tokens, ids or bodies', () => {
+ const traffic = createTrafficAccumulator();
+ const urls = ['http://127.0.0.1:4567/api/things/12345?token=SECRET', 'http://127.0.0.1:4567/api/blobs/0123456789abcdef0123/meta', 'http://127.0.0.1:4567/api/settings?key=SECRET'];
+ urls.forEach((url, i) => {
+ const requestId = 'r' + i;
+ traffic.event('requestWillBeSent', { requestId, request: { url } });
+ traffic.event('dataReceived', { requestId, dataLength: 10, encodedDataLength: 5 });
+ traffic.event('loadingFinished', { requestId, encodedDataLength: 5 });
+ });
+ traffic.setPhase('idle');
+ traffic.event('webSocketCreated', { requestId: 's', url: 'ws://127.0.0.1:4567/socket.io/?EIO=4&sid=SECRET' });
+ traffic.event('webSocketFrameReceived', { requestId: 's', response: { opcode: 1, payloadData: '3' } });
+ const phases = traffic.snapshot();
+ const text = JSON.stringify({ phases, idle: idleBytesPerMinute(phases.idle, 30000) });
+ expect(Object.keys(phases.cold)).toEqual(['/api/blobs/:id/meta', '/api/settings', '/api/things/:id']);
+ expect(text).not.toMatch(/SECRET|127\.0\.0\.1|12345|0123456789abcdef/);
+ expect(phases.idle['/socket.io/'].socketReceivedFrames).toBe(1);
+ expect(idleBytesPerMinute(phases.idle, 30000).socketPayload).toBe(2);
+ });
+
+ it('accepts only same-origin absolute route paths and bounded idle windows', () => {
+ expect(parseAuditArgs(['out.json', '--route=/', '--sibling=/settings', '--idle-ms=5000']))
+ .toEqual({ output: 'out.json', route: '/', sibling: '/settings', idleMs: 5000 });
+ expect(parseAuditArgs([])).toMatchObject({ route: null, idleMs: 60000 });
+ for (const bad of ['--route=//evil.example', '--route=https://evil.example', '--route=x', '--idle-ms=5', '--bogus'])
+ expect(() => parseAuditArgs([bad])).toThrow();
+ });
+});
diff --git a/server/services/taskPromptDefaults.test.js b/server/services/taskPromptDefaults.test.js
index e8841756a9..4bf15e3b36 100644
--- a/server/services/taskPromptDefaults.test.js
+++ b/server/services/taskPromptDefaults.test.js
@@ -165,9 +165,9 @@ describe('taskPromptDefaults integrity snapshot', () => {
// v6: 'Required' coverage sections made a single-process app pad its report
// with 'checked, not applicable' lines (#12078).
- it('performance v6 scopes platform-specific coverage to what the app has', () => {
+ it('performance v7 scopes platform-specific coverage to what the app has', () => {
const perf = DEFAULT_TASK_PROMPTS.performance;
- expect(PROMPT_VERSIONS.performance).toBe(6);
+ expect(PROMPT_VERSIONS.performance).toBe(7);
expect(perf).not.toMatch(/^## Required/m);
expect(perf.match(/^## Where present:/gm)).toHaveLength(2);
expect(perf.match(/skip the rest silently/g)).toHaveLength(2);
diff --git a/server/services/taskPromptDefaults/integrity.snapshot.json b/server/services/taskPromptDefaults/integrity.snapshot.json
index 16697d4081..a6a53e450a 100644
--- a/server/services/taskPromptDefaults/integrity.snapshot.json
+++ b/server/services/taskPromptDefaults/integrity.snapshot.json
@@ -39,7 +39,7 @@
"model-comparison-refresh": "f66e070fa08e30edda2c3020289f0c67",
"module-hygiene": "1637fd164737e20db284fe5c2b440453",
"observability": "243e58bcfae2759f82caf3ff6c8fc9c5",
- "performance": "4f8ddada54d7c6f31b7fb35aeb0f29db",
+ "performance": "45aa35e50c45cf9f4e7d3d7125bf4b5b",
"plan-feature": "7f859397ff16889ffc0cf5a8ba6bddab",
"plan-task": "b718701c2d4ed2b4900225fc58857208",
"plan-task-claim": "07f6252f00905f9c6e4134a3d3587310",
@@ -104,7 +104,7 @@
"model-comparison-refresh": 4,
"module-hygiene": 3,
"observability": 2,
- "performance": 6,
+ "performance": 7,
"plan-feature": 6,
"plan-task": 24,
"pr-reviewer": 5,
@@ -417,7 +417,8 @@
"672bd3958b12afb0f382bcebb6dc3b33",
"d7e3a80f42903fbf90be0efc9b559c0b",
"b69bf4cbd2b3c272b5650608c62cede3",
- "8b297d723f9f39154c9cf6d501738494"
+ "8b297d723f9f39154c9cf6d501738494",
+ "4f8ddada54d7c6f31b7fb35aeb0f29db"
],
"plan-feature": [
"9fbc2a2ded8f4533d1f0f6b9cb780b34",
diff --git a/server/services/taskPromptDefaults/prompts.js b/server/services/taskPromptDefaults/prompts.js
index 1f02db1af8..3375bd7b0b 100644
--- a/server/services/taskPromptDefaults/prompts.js
+++ b/server/services/taskPromptDefaults/prompts.js
@@ -404,6 +404,12 @@ be scoped to visible consumers. Use synthetic large collections to verify
server page limits, compact list projections, lazy details, bounded rendered
rows, and no full history hydration hidden behind client-side slicing.
+If the app is PortOS (it ships scripts/perf/browserCollectionAudit.mjs) and the
+live browser is unauthenticated, run
+node scripts/perf/browserCollectionAudit.mjs --route=/ --sibling=<tab> against
+the synthetic fixture (see scripts/perf/README.md) for the cold, sibling and idle
+counts above.
+
Do not claim UI performance passed from source inspection or a small/empty
fixture alone. If browser access, an authorized session, or usable large-fixture
evidence is unavailable, explicitly mark UI load/idle transfer UNVERIFIED and
diff --git a/server/services/taskPromptDefaults/versions.js b/server/services/taskPromptDefaults/versions.js
index 15ea590b83..eacf69aba8 100644
--- a/server/services/taskPromptDefaults/versions.js
+++ b/server/services/taskPromptDefaults/versions.js
@@ -36,7 +36,7 @@ export const PROMPT_VERSIONS = {
// current {appName} body. See the retired-hash history in
// integrity.snapshot.json + the self-heal in taskScheduleStore.js readSchedule().
'security': 6, // v6: generic trust-model pointer (AGENTS.md Security Model + ETHOS.md for PortOS) replaces the PortOS-specific authority paragraph, adds the one-finding-per-class rule, and drops the redaction sentence the mode banners own (#12081, #12080). v5: distinguish delegated authority from untrusted content and require workflow-preserving boundary fixes. v4: adds tenant/resource isolation, SSRF, and unsafe deserialization — the service and data-API defects the v3 list (written for a web app) did not name. v3: carries {modeInstructions} (the pre-v3 body had no placeholder, so its own "fix and commit" steps fought a file-issues dispatch) and defers to the project's documented threat model, requiring an attacker-or-accident story per finding instead of a generic OWASP checklist walk. Previously v2: generic {appName} body (older default hardcoded "PortOS"/"server/routes" paths)
- 'performance': 6, // v6: the service/data-path and UI-load sections are "Where present" (apply what exists, skip the rest silently) and the app-specific reference-data sentence is a generic hunt bullet (#12078). v5: adds required service/data-path coverage (query plans, streaming vs buffering, batch shape, pools, partition pruning, cold start), hands spend to cost-efficiency, and replaces PortOS-specific collection names with generic ones — the prompt runs against every managed app. v4: requires UI route-load, idle-network and large-collection evidence; missing measurements are explicitly unverified.
+ 'performance': 7, // v7: names the PortOS --route audit for an unauthenticated live browser (#12998). v6: the service/data-path and UI-load sections are "Where present" (apply what exists, skip the rest silently) and the app-specific reference-data sentence is a generic hunt bullet (#12078). v5: adds required service/data-path coverage (query plans, streaming vs buffering, batch shape, pools, partition pruning, cold start), hands spend to cost-efficiency, and replaces PortOS-specific collection names with generic ones — the prompt runs against every managed app. v4: requires UI route-load, idle-network and large-collection evidence; missing measurements are explicitly unverified.
'code-quality': 6, // v6: the triage lane ranks the three focused audits to run next under a `Recommended next audits` heading and cedes unhandled boundary calls to error-handling (#12085). v5: drops the read-instruction-files and scan-then-bound restatements the mode banners and discovery contract own (#12080). v4: hands generated-artifact/hand-synced-registry/layout-coupling drift to the better-structural-drift lane and counted branching + reader cost to better-complexity / better-cognitive-load, keeping conventional maintainability defects — the split across PortOS's specialist tasks. Previously v3: inventory structural drift, duplicate sources of truth, volatile generated artifacts, and incidental-layout coupling before conventional maintainability checks. v2: generic {appName} body (older default hardcoded "PortOS")
'test-coverage': 4, // v4: files issues by default and, in implement mode, names the redundant test removed to pay for a new one in a suite over the size budget (#12087). v3: carries {modeInstructions}, ranks gaps by consequence rather than percentage, tests at the highest practical public boundary, requires naming the regression a new test uniquely catches, and hands existing-test quality to better-test-quality. Previously v2: generic {appName} body (older default hardcoded "PortOS")
'accessibility': 5, // v5: the "skip what static analysis already covers" list now also names keyboard-trap/unreachable-control and faked-semantics findings, matching Biome's `useKeyWithClickEvents`, `noNoninteractiveElementToInteractiveRole`, `useFocusableInteractive`, and `noAriaHiddenOnFocusable` a11y rules (enabled in `client/biome.jsonc` alongside the four from v4) — the live-agent budget shifts further to focus-order/live-region/contrast/zoom issues static analysis still can't establish. Previously v4: checks for an existing JSX a11y linter (Biome's a11y group, jsx-a11y) first and skips findings it already catches deterministically (missing alt/labels, invalid/unsupported ARIA), so the live-agent budget goes to keyboard/focus/live-region/contrast/zoom issues static analysis can't establish. Previously v3: carries {modeInstructions}, names who is blocked and how far they get, and prefers a native element over a custom control with patched attributes. Previously v2: generic {appName} + the app UI (older default hardcoded "PortOS" + http://localhost:5555)