Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
Bound large collection audit fixture startup and preserve cleanup evidence (#11573)
[._.]/c3394a3fda4d
diff --git a/scripts/perf/README.md b/scripts/perf/README.mdindex 5e2d1bca9c..6d419bd806 100644--- a/scripts/perf/README.md+++ b/scripts/perf/README.md@@ -14,12 +14,31 @@ temporary directory. Initialization failures also clean up. As with any process,SIGKILL or machine power loss cannot run cleanup; use graceful shutdown.A future CDP runner can import startCollectionFixture() and use its returned-{ url, cardinalities, close }. Always await close() in a finally block. It is+{ url, cardinalities, startup, close }. Always await close() in a finally block. It isidempotent. /__fixture/ready returns the same readiness/cardinality informationwithout database identifiers, credentials, paths, or record payloads. The optionalclientDist argument selects an already-built client; it defaults to client/dist.No development proxy is started.+Startup preserves the 30,000 ms overall worker readiness budget. A separate+10,000 ms no-progress deadline detects a stalled worker sooner; only an advancing+allowlisted phase or completed-record count resets it. Repeated heartbeats do not.+The optional `startupLimits: { overallTimeoutMs, noProgressTimeoutMs }` argument+accepts integer milliseconds from 100 through 120,000, with the no-progress limit+at most the overall limit. These are explicit experiment limits, not automatic+retries or pass thresholds. The worker budgets start after the parent source copy+and schema setup; startup diagnostics include those parent phases as well.++`startup` contains fixed phase names, elapsed/phase milliseconds, completed+image/video/account/message counts, and cumulative milliseconds awaiting file+writes and canonical media-index inserts. Eight records seed concurrently, with+two sequential file writes and one canonical upsert per record. A failed batch+drains its outstanding writes before cleanup. The time aggregates sum work across+records and can exceed elapsed wall time with concurrent seeding. Each phase+retains only its latest progress sample, bounding report size. Rejected startup+errors carry the same sanitized `fixtureStartup` snapshot and a fixed+`fixtureFailure` reason; exception text and worker stderr never enter reports.+The fixed dataset contains 2,400 PNG images, 1,200 one-second MP4 videos and theirPNG thumbnails, and 4,000 messages across two disabled synthetic accounts. Everytenth media record is hidden (240 images and 120 videos). Prompts and message@@ -67,10 +86,13 @@ Tests, from server/:~~~shnpm test -- ../scripts/perf/collectionFixture.test.js+npm test -- ../scripts/perf/collectionFixtureStartup.test.js ../scripts/perf/collectionFixtureLifecycle.test.js ../scripts/perf/collectionFixtureSeed.test.jsnpm run test:db -- ../scripts/perf/collectionFixture.db.test.js~~~-The first covers pre-import isolation rejection. The second covers actual HTTP+The non-DB tests cover pre-import isolation rejection, bounded concurrency,+failed-batch drainage, independent total/inactivity deadlines, sanitized IPC and+stalled-child cleanup with injected resources/time. The DB suite covers actual HTTPpagination, projections, unavailable actions, and success/failure cleanup againstportos_test.@@ -93,6 +115,8 @@ gate is evidence to fix, never a reason to raise the limit until the run passes.The report records:+- Fixture startup phases, aggregate seed work and counts on success or failure;+ startup failures retain specific fixed reasons instead of a generic audit error.- Time from navigation start until the first visible synthetic collection row,then a two-second observation tail for eager fetches; 60 media cards / 50 inboxrows maximum, checked against response counts before client rendering.@@ -138,8 +162,8 @@ single-record hydration on open; the fixture serves the same bounded imagelookup and video-history record readers the migrated UI calls. The mediascenario now passes with a 32,212-byte first page, visible/hidden totals of3,240/360, no cold detail reads, and exactly one detail read when a row opens.-Image variant lineage (`/api/image-gen/:filename/variants`) remains outside the-fixture and is counted as unavailable. Inbox summary/detail and Contacts idle+The fixture also serves image variant lineage (`/api/image-gen/:filename/variants`)+for the lightbox's current lazy-detail read. Inbox summary/detail and Contacts idlegates passed, including a full idle minute with zero HTTP transfer and fourbytes of socket heartbeat payload. This runner does not claim all #8232contracts are met.diff --git a/scripts/perf/browserCollectionAudit.mjs b/scripts/perf/browserCollectionAudit.mjsindex 7b4b4669f7..3713c26d83 100644--- a/scripts/perf/browserCollectionAudit.mjs+++ b/scripts/perf/browserCollectionAudit.mjs@@ -144,13 +144,16 @@ async function scenario(route) {try {fixture = await startCollectionFixture();+ report.startup = fixture.startup;report.cardinalities = fixture.cardinalities;browser = await chromium.launch({ headless: true, ...(process.env.COLLECTION_AUDIT_BROWSER? { executablePath: process.env.COLLECTION_AUDIT_BROWSER } : { channel: 'chrome' }) });await scenario('/media/history');await scenario('/messages/inbox');-} catch {- report.failures.push({ endpoint: 'fixture', reason: 'audit-startup-or-shutdown', bytes: 0 });+} catch (error) {+ if (error.fixtureStartup) report.startup = error.fixtureStartup;+ report.failures.push({ endpoint: fixture ? 'browser' : 'fixture',+ reason: error.fixtureFailure || 'audit-startup-or-shutdown', bytes: 0 });} finally {await browser?.close().catch(() => report.failures.push({ endpoint: 'browser', reason: 'cleanup', bytes: 0 }));await fixture?.close().catch(() => report.failures.push({ endpoint: 'fixture', reason: 'cleanup', bytes: 0 }));diff --git a/scripts/perf/collectionFixture.js b/scripts/perf/collectionFixture.jsindex c7ac682d43..c10d158353 100644--- a/scripts/perf/collectionFixture.js+++ b/scripts/perf/collectionFixture.js@@ -12,6 +12,7 @@ import { promisify } from 'node:util';import { fileURLToPath } from 'node:url';import { isDirectlyInvoked } from '../lib/directInvocation.js';+import { createFixtureStartup } from './collectionFixtureStartup.js';const codeRoot = resolve(dirname(fileURLToPath(import.meta.url)), '../..');@@ -25,11 +26,13 @@ export function validateFixtureEnvironment(env = process.env, host = '127.0.0.1'if (env.PGSERVICE || env.PGSERVICEFILE || env.DATABASE_URL) throw new Error('Remove alternate database configuration');}-/** Each invocation returns { url, cardinalities, close }; close is idempotent. */+/** Returns { url, cardinalities, startup, close }; close is idempotent. */export async function startCollectionFixture({env = process.env, host = '127.0.0.1', clientDist = join(codeRoot, 'client/dist'),+ startupLimits,} = {}) {validateFixtureEnvironment(env, host);+ const startup = createFixtureStartup(startupLimits);await access(join(clientDist, 'index.html'));const require = createRequire(join(codeRoot, 'server/package.json'));const { Client } = require('pg');@@ -71,6 +74,7 @@ export async function startCollectionFixture({process.once('SIGINT', onSignal);process.once('SIGTERM', onSignal);try {+ startup.record({ phase: 'source-copy' });root = await realpath(await mkdtemp(join(tmpdir(), 'portos-collection-audit-')));// Copy code, never data, configuration, credentials, or the live checkout's// symlinks. A physical temp copy also preserves dataRoot's worktree guard.@@ -82,12 +86,13 @@ export async function startCollectionFixture({await cp(join(codeRoot, file), join(root, file), { dereference: false });}await mkdir(join(root, 'scripts/perf'), { recursive: true });- for (const file of ['collectionFixtureWorker.js', 'collectionFixtureData.js']) {+ for (const file of ['collectionFixtureWorker.js', 'collectionFixtureData.js', 'collectionFixtureSeed.js']) {await cp(join(codeRoot, 'scripts/perf', file), join(root, 'scripts/perf', file));}await cp(join(codeRoot, 'scripts/perf/assets'), join(root, 'scripts/perf/assets'), { recursive: true });await writeFile(join(root, 'package.json'), '{"type":"module"}\n');await symlink(join(codeRoot, 'server/node_modules'), join(root, 'server/node_modules'), 'junction');+ startup.record({ phase: 'parent-database' });await db.connect();const identity = await db.query('SELECT current_database() AS name');if (identity.rows[0].name !== 'portos_test') throw new Error('Connected database is not portos_test');@@ -109,31 +114,16 @@ export async function startCollectionFixture({});// Drain diagnostics without leaking paths, payloads, or credentials into reports.child.stderr.resume();- const ready = await new Promise((resolveReady, rejectReady) => {- const timer = setTimeout(() => rejectReady(new Error('Collection fixture readiness timed out')), 30000);- const finish = (error, value) => {- clearTimeout(timer);- child.removeListener('error', onError);- child.removeListener('exit', onExit);- child.removeListener('message', onMessage);- error ? rejectReady(error) : resolveReady(value);- };- const onError = () => finish(new Error('Collection fixture child failed to start'));- const onExit = () => finish(new Error('Collection fixture child exited before readiness'));- const onMessage = message => {- if (message?.type === 'ready') finish(null, message);- if (message?.type === 'failed') finish(new Error('Collection fixture initialization failed at ' + message.phase));- };- child.once('error', onError);- child.once('exit', onExit);- child.on('message', onMessage);- });+ const ready = await startup.wait(child);if (stopped) throw new Error('Fixture startup interrupted');finishSetup();child.once('exit', () => { close().catch(() => { process.exitCode = 1; }); });- return { url: ready.url, cardinalities: ready.cardinalities, close };+ return { url: ready.url, cardinalities: ready.cardinalities, startup: startup.snapshot(), close };} catch (error) {finishSetup();+ // Capture only sanitized diagnostics; raw I/O/DB errors may contain secrets.+ error.fixtureStartup ||= startup.snapshot();+ error.fixtureFailure ||= 'setup-failed';await close();throw error;}diff --git a/scripts/perf/collectionFixtureLifecycle.test.js b/scripts/perf/collectionFixtureLifecycle.test.jsnew file mode 100644index 0000000000..965fd86a53--- /dev/null+++ b/scripts/perf/collectionFixtureLifecycle.test.js@@ -0,0 +1,56 @@+import { EventEmitter } from 'node:events';+import { afterEach, expect, it, vi } from 'vitest';++const owned = vi.hoisted(() => ({ queries: [], removed: [], child: null, ended: false }));+vi.mock('node:fs/promises', () => ({+ access: vi.fn(), cp: vi.fn(), mkdir: vi.fn(), symlink: vi.fn(), writeFile: vi.fn(),+ lstat: async () => ({ isFile: () => true }),+ mkdtemp: async () => '/tmp/portos-collection-audit-synthetic-lifecycle', realpath: async path => path,+ rm: async path => { owned.removed.push(path); },+}));+vi.mock('node:module', () => ({ createRequire: () => () => ({ Client: function () {+ return { on: vi.fn(), connect: vi.fn(), end: async () => { owned.ended = true; },+ query: async sql => {+ owned.queries.push(sql);+ return { rows: [{ name: 'portos_test' }] };+ } };+} }) }));+vi.mock('node:child_process', () => {+ const execFile = vi.fn();+ execFile[Symbol.for('nodejs.util.promisify.custom')] = async () => ({ stdout: 'server/package.json\0' });+ return { execFile, fork: () => {+ const child = new EventEmitter();+ child.exitCode = null;+ child.signalCode = null;+ child.stderr = { resume: vi.fn() };+ child.kill = vi.fn(signal => {+ child.signalCode = signal;+ queueMicrotask(() => child.emit('exit', null, signal));+ });+ owned.child = child;+ return child;+ } };+});+import { startCollectionFixture } from './collectionFixture.js';++afterEach(() => vi.useRealTimers());++it('a stalled fixture rejects and removes only its owned schema, root, child and listeners', async () => {+ vi.useFakeTimers();+ const beforeSignals = ['SIGINT', 'SIGTERM'].map(signal => process.listenerCount(signal));+ const ready = startCollectionFixture({ env: {}, clientDist: '/synthetic-client' });+ const failure = expect(ready).rejects.toMatchObject({ fixtureFailure: 'startup-stalled',+ fixtureStartup: { noProgressTimeoutMs: 10000, overallTimeoutMs: 30000 } });+ await vi.advanceTimersByTimeAsync(10000);+ await failure;+ const create = owned.queries.find(sql => sql.startsWith('CREATE SCHEMA'));+ expect(create).toMatch(/^CREATE SCHEMA "collection_audit_[a-f0-9]{32}"$/);+ expect(owned.queries).toContain(create.replace('CREATE SCHEMA', 'DROP SCHEMA') + ' CASCADE');+ expect(owned.queries).toHaveLength(3);+ expect(owned.removed).toEqual(['/tmp/portos-collection-audit-synthetic-lifecycle']);+ expect(owned.ended).toBe(true);+ expect(owned.child.kill).toHaveBeenCalledExactlyOnceWith('SIGTERM');+ expect(owned.child.eventNames()).toEqual([]);+ expect(['SIGINT', 'SIGTERM'].map(signal => process.listenerCount(signal))).toEqual(beforeSignals);+ expect(vi.getTimerCount()).toBe(0);+});diff --git a/scripts/perf/collectionFixtureSeed.js b/scripts/perf/collectionFixtureSeed.jsnew file mode 100644index 0000000000..a8ff28afa0--- /dev/null+++ b/scripts/perf/collectionFixtureSeed.js@@ -0,0 +1,16 @@+// Bound both filesystem pressure and PostgreSQL concurrency. Every record still+// writes its own files and passes through the canonical single-row index writer.+export const SEED_BATCH_SIZE = 8;++export async function seedFixtureRecords(total, writeRecord, completeBatch) {+ for (let offset = 0; offset < total; offset += SEED_BATCH_SIZE) {+ const end = Math.min(offset + SEED_BATCH_SIZE, total);+ const results = await Promise.allSettled(Array.from({ length: end - offset },+ (_, index) => Promise.resolve().then(() => writeRecord(offset + index))));+ // Drain the whole bounded batch before propagating failure: cleanup must+ // never race sibling writes that still own resources below the fixture root.+ const failed = results.find(result => result.status === 'rejected');+ if (failed) throw failed.reason;+ completeBatch(end);+ }+}diff --git a/scripts/perf/collectionFixtureSeed.test.js b/scripts/perf/collectionFixtureSeed.test.jsnew file mode 100644index 0000000000..4b40d63486--- /dev/null+++ b/scripts/perf/collectionFixtureSeed.test.js@@ -0,0 +1,42 @@+import { expect, it } from 'vitest';+import { seedFixtureRecords, SEED_BATCH_SIZE } from './collectionFixtureSeed.js';++it('bounds concurrent records and reports only fully completed batches', async () => {+ const pending = [];+ const started = [];+ const completed = [];+ const seed = seedFixtureRecords(SEED_BATCH_SIZE + 1, index => {+ started.push(index);+ return new Promise(resolve => pending.push(resolve));+ }, count => completed.push(count));+ await Promise.resolve();+ expect(started).toHaveLength(SEED_BATCH_SIZE);+ expect(completed).toEqual([]);+ for (const resolve of pending.splice(0)) resolve();+ await new Promise(resolve => setImmediate(resolve));+ expect(started).toEqual(Array.from({ length: SEED_BATCH_SIZE + 1 }, (_, index) => index));+ expect(completed).toEqual([SEED_BATCH_SIZE]);+ pending[0]();+ await seed;+ expect(completed).toEqual([SEED_BATCH_SIZE, SEED_BATCH_SIZE + 1]);+});++it('drains a failed batch before rejecting, and starts no later records', async () => {+ const pending = [];+ const started = [];+ const completed = [];+ const failure = new Error('Synthetic seed write failure');+ let rejected = false;+ const seed = seedFixtureRecords(SEED_BATCH_SIZE + 1, index => {+ started.push(index);+ if (index === 0) throw failure;+ return new Promise(resolve => pending.push(resolve));+ }, count => completed.push(count)).catch(error => { rejected = true; throw error; });+ const assertion = expect(seed).rejects.toBe(failure);+ await new Promise(resolve => setImmediate(resolve));+ expect(rejected).toBe(false);+ for (const resolve of pending) resolve();+ await assertion;+ expect(started).toHaveLength(SEED_BATCH_SIZE);+ expect(completed).toEqual([]);+});diff --git a/scripts/perf/collectionFixtureStartup.js b/scripts/perf/collectionFixtureStartup.jsnew file mode 100644index 0000000000..acaaa421b6--- /dev/null+++ b/scripts/perf/collectionFixtureStartup.js@@ -0,0 +1,96 @@+import { performance } from 'node:perf_hooks';+import { CARDINALITIES } from './collectionFixtureData.js';+import { SEED_BATCH_SIZE } from './collectionFixtureSeed.js';++const phases = ['source-copy', 'parent-database', 'imports', 'database', 'image-seed', 'video-seed', 'inbox-seed', 'routes', 'ready'];+const limits = { images: CARDINALITIES.images, videos: CARDINALITIES.videos,+ accounts: CARDINALITIES.accounts, messages: CARDINALITIES.messages };++/** Only fixed stages and bounded aggregates cross the worker/report boundary. */+export function createFixtureStartup({ overallTimeoutMs = 30000, noProgressTimeoutMs = 10000 } = {}) {+ if (!Number.isInteger(overallTimeoutMs) || overallTimeoutMs < 100 || overallTimeoutMs > 120000+ || !Number.isInteger(noProgressTimeoutMs) || noProgressTimeoutMs < 100+ || noProgressTimeoutMs > overallTimeoutMs) throw new Error('Invalid collection fixture startup limits');+ const started = performance.now();+ let workerStarted;+ const samples = [];+ let phaseIndex = -1;+ let counts = { images: 0, videos: 0, accounts: 0, messages: 0 };+ let workMs = { files: 0, index: 0 };+ const record = message => {+ const nextPhase = phases.indexOf(message?.phase);+ if (nextPhase < phaseIndex || nextPhase < 0) return false;+ const nextCounts = {};+ for (const [key, max] of Object.entries(limits)) {+ const value = message.counts?.[key] ?? counts[key];+ if (!Number.isInteger(value) || value < counts[key] || value > max) return false;+ nextCounts[key] = value;+ }+ // Repeated heartbeats cannot keep a stuck worker alive.+ if (nextPhase === phaseIndex && Object.keys(limits).every(key => nextCounts[key] === counts[key])) return false;+ const nextWork = {};+ for (const key of ['files', 'index']) {+ const value = message.workMs?.[key] ?? workMs[key];+ if (!Number.isFinite(value) || value < workMs[key] || value > 120000 * SEED_BATCH_SIZE) return false;+ nextWork[key] = Math.round(value);+ }+ phaseIndex = nextPhase;+ counts = nextCounts;+ workMs = nextWork;+ const elapsedMs = Math.round(performance.now() - started);+ const samePhase = samples.at(-1)?.phase === phases[phaseIndex];+ if (samples.length && !samePhase) {+ samples.at(-1).durationMs = elapsedMs - samples.at(-1).startedMs;+ }+ const startedMs = samePhase ? samples.at(-1).startedMs : elapsedMs;+ const sample = { phase: phases[phaseIndex], startedMs, elapsedMs,+ durationMs: elapsedMs - startedMs, counts, workMs };+ // One latest aggregate per phase bounds report size even for noisy IPC.+ if (samples.at(-1)?.phase === sample.phase) samples[samples.length - 1] = sample;+ else samples.push(sample);+ return true;+ };+ const snapshot = () => {+ const elapsedMs = Math.round(performance.now() - started);+ return { overallTimeoutMs, noProgressTimeoutMs, elapsedMs,+ workerElapsedMs: workerStarted === undefined ? null : Math.round(performance.now() - workerStarted),+ phases: samples.map((sample, index) => ({ ...sample,+ durationMs: index === samples.length - 1 ? elapsedMs - sample.startedMs : sample.durationMs,+ counts: { ...sample.counts }, workMs: { ...sample.workMs } })) };+ };+ const failure = reason => Object.assign(new Error('Collection fixture ' + reason), {+ fixtureFailure: reason, fixtureStartup: snapshot(),+ });+ const wait = child => new Promise((resolveReady, rejectReady) => {+ workerStarted = performance.now();+ let idleTimer;+ const finish = (error, value) => {+ clearTimeout(overallTimer);+ clearTimeout(idleTimer);+ child.removeListener('error', onError);+ child.removeListener('exit', onExit);+ child.removeListener('message', onMessage);+ error ? rejectReady(error) : resolveReady(value);+ };+ const armIdle = () => {+ clearTimeout(idleTimer);+ idleTimer = setTimeout(() => finish(failure('startup-stalled')), noProgressTimeoutMs);+ };+ const onError = () => finish(failure('child-start-failed'));+ const onExit = () => finish(failure('child-exited'));+ const onMessage = message => {+ if (message?.type === 'progress' && record(message)) armIdle();+ if (message?.type === 'failed') finish(failure('initialization-failed'));+ if (message?.type === 'ready') {+ record({ phase: 'ready' });+ finish(null, message);+ }+ };+ const overallTimer = setTimeout(() => finish(failure('startup-timeout')), overallTimeoutMs);+ child.once('error', onError);+ child.once('exit', onExit);+ child.on('message', onMessage);+ armIdle();+ });+ return { record, snapshot, wait };+}diff --git a/scripts/perf/collectionFixtureStartup.test.js b/scripts/perf/collectionFixtureStartup.test.jsnew file mode 100644index 0000000000..0099b48675--- /dev/null+++ b/scripts/perf/collectionFixtureStartup.test.js@@ -0,0 +1,68 @@+import { EventEmitter } from 'node:events';+import { afterEach, describe, expect, it, vi } from 'vitest';+import { createFixtureStartup } from './collectionFixtureStartup.js';++afterEach(() => vi.useRealTimers());++describe('fixture startup IPC and deadlines', () => {+ it('does not let repeated or invalid IPC keep a stalled child alive or leak payloads', async () => {+ vi.useFakeTimers();+ const child = new EventEmitter();+ const startup = createFixtureStartup();+ const ready = startup.wait(child);+ const failure = expect(ready).rejects.toMatchObject({ fixtureFailure: 'startup-stalled' });+ child.emit('message', { type: 'progress', phase: 'image-seed', counts: { images: 100 }, private: 'secret' });+ await vi.advanceTimersByTimeAsync(9000);+ child.emit('message', { type: 'progress', phase: 'image-seed', counts: { images: 100 }, workMs: { files: 9999 } });+ child.emit('message', { type: 'progress', phase: 'secret', counts: { images: 200 } });+ child.emit('message', { type: 'progress', phase: 'image-seed', counts: { images: 2401 } });+ await vi.advanceTimersByTimeAsync(1000);+ await failure;+ expect(startup.snapshot().phases).toEqual([{ phase: 'image-seed', startedMs: expect.any(Number), elapsedMs: expect.any(Number), durationMs: expect.any(Number),+ counts: { images: 100, videos: 0, accounts: 0, messages: 0 }, workMs: { files: 0, index: 0 } }]);+ expect(JSON.stringify(startup.snapshot())).not.toContain('secret');+ expect(child.eventNames()).toEqual([]);+ expect(vi.getTimerCount()).toBe(0);+ });++ it('enforces the total budget even while counts advance', async () => {+ vi.useFakeTimers();+ const child = new EventEmitter();+ const startup = createFixtureStartup();+ const ready = startup.wait(child);+ const failure = expect(ready).rejects.toMatchObject({ fixtureFailure: 'startup-timeout' });+ for (let images = 100; images <= 400; images += 100) {+ child.emit('message', { type: 'progress', phase: 'image-seed', counts: { images } });+ await vi.advanceTimersByTimeAsync(images === 400 ? 3000 : 9000);+ }+ await failure;+ expect(child.eventNames()).toEqual([]);+ expect(vi.getTimerCount()).toBe(0);+ });++ it('extends only the inactivity deadline for real progress, then disposes listeners on ready or exit', async () => {+ vi.useFakeTimers();+ const child = new EventEmitter();+ const startup = createFixtureStartup();+ const ready = startup.wait(child);+ child.emit('message', { type: 'progress', phase: 'image-seed', counts: { images: 100 } });+ await vi.advanceTimersByTimeAsync(9000);+ child.emit('message', { type: 'progress', phase: 'image-seed', counts: { images: 200 } });+ await vi.advanceTimersByTimeAsync(9000);+ child.emit('message', { type: 'ready' });+ await expect(ready).resolves.toMatchObject({ type: 'ready' });+ expect(child.eventNames()).toEqual([]);+ expect(vi.getTimerCount()).toBe(0);+ const exited = startup.wait(child);+ child.emit('exit', 1);+ await expect(exited).rejects.toMatchObject({ fixtureFailure: 'child-exited' });+ expect(child.eventNames()).toEqual([]);+ });++ it('rejects invalid limits before resource creation', () => {+ for (const startupLimits of [{ overallTimeoutMs: 120001 }, { overallTimeoutMs: NaN },+ { noProgressTimeoutMs: 0 }, { overallTimeoutMs: 1000, noProgressTimeoutMs: 1001 }]) {+ expect(() => createFixtureStartup(startupLimits)).toThrow('Invalid collection fixture startup limits');+ }+ });+});diff --git a/scripts/perf/collectionFixtureWorker.js b/scripts/perf/collectionFixtureWorker.jsindex 9dc4b3f15a..53ebdd08fa 100644--- a/scripts/perf/collectionFixtureWorker.js+++ b/scripts/perf/collectionFixtureWorker.js@@ -1,7 +1,9 @@import { mkdir, readFile, writeFile } from 'node:fs/promises';import { createRequire } from 'node:module';import { join, resolve } from 'node:path';+import { performance } from 'node:perf_hooks';import { CARDINALITIES, ACCOUNT_IDS, mediaFixture, messageFixture } from './collectionFixtureData.js';+import { seedFixtureRecords } from './collectionFixtureSeed.js';// This is the only worker entrypoint. Validate BEFORE dynamic server imports.const root = process.cwd();@@ -18,6 +20,18 @@ let io;let db;let closing;let phase = 'imports';+const counts = { images: 0, videos: 0, accounts: 0, messages: 0 };+const workMs = { files: 0, index: 0 };+const progress = nextPhase => {+ phase = nextPhase;+ if (process.connected) process.send({ type: 'progress', phase, counts,+ workMs: { files: Math.round(workMs.files), index: Math.round(workMs.index) } });+};+const writeSeedFile = async (path, contents) => {+ const started = performance.now();+ await writeFile(path, contents);+ workMs.files += performance.now() - started;+};const close = () => closing ||= (async () => {if (server) {if (io) await new Promise(resolveIo => io.close(resolveIo));@@ -33,13 +47,14 @@ process.once('SIGTERM', stop);process.once('disconnect', stop);async function start() {+ progress('imports');const require = createRequire(join(root, 'server/package.json'));const express = require('express');const sharp = require('sharp');db = await import('../../server/lib/db.js');const { PATHS } = await import('../../server/lib/paths.js');if (resolve(PATHS.data) !== join(root, 'data')) throw new Error('Fixture data root mismatch');- phase = 'database';+ progress('database');const identity = await db.query('SELECT current_database() AS db, current_schema() AS schema');if (identity.rows[0].db !== 'portos_test' || identity.rows[0].schema !== schema) throw new Error('Fixture DB isolation mismatch');const { mediaDdl } = await import('../../server/lib/db/schema/media.js');@@ -51,36 +66,50 @@ async function start() {const { imageToRow, videoToRow } = await import('../../server/services/mediaAssetIndex/logic.js');await Promise.all(['images', 'videos', 'video-thumbnails', 'messages/cache'].map(dir => mkdir(join(PATHS.data, dir), { recursive: true })));const png = await sharp({ create: { width: 640, height: 360, channels: 3, background: '#335577' } }).png().toBuffer();- phase = 'media-seed';- for (let index = 0; index < CARDINALITIES.images; index++) {+ progress('image-seed');+ await seedFixtureRecords(CARDINALITIES.images, async index => {const image = mediaFixture('image', index);- await writeFile(join(PATHS.images, image.filename), png);- await writeFile(join(PATHS.images, image.filename.replace('.png', '.metadata.json')), JSON.stringify(image));+ await writeSeedFile(join(PATHS.images, image.filename), png);+ await writeSeedFile(join(PATHS.images, image.filename.replace('.png', '.metadata.json')), JSON.stringify(image));+ const started = performance.now();await upsertAsset(imageToRow(image));- }+ workMs.index += performance.now() - started;+ }, completed => {+ counts.images = completed;+ progress('image-seed');+ });const clip = await readFile(join(root, 'scripts/perf/assets/synthetic.mp4'));const videos = [];- for (let index = 0; index < CARDINALITIES.videos; index++) {+ progress('video-seed');+ await seedFixtureRecords(CARDINALITIES.videos, async index => {const video = mediaFixture('video', index);- videos.push(video);- await writeFile(join(PATHS.data, 'videos', video.filename), clip);- await writeFile(join(PATHS.data, 'video-thumbnails', video.thumbnail), png);+ videos[index] = video;+ await writeSeedFile(join(PATHS.data, 'videos', video.filename), clip);+ await writeSeedFile(join(PATHS.data, 'video-thumbnails', video.thumbnail), png);+ const started = performance.now();await upsertAsset(videoToRow(video));- }- await writeFile(join(PATHS.data, 'video-history.json'), JSON.stringify(videos));- phase = 'inbox-seed';+ workMs.index += performance.now() - started;+ }, completed => {+ counts.videos = completed;+ progress('video-seed');+ });+ await writeSeedFile(join(PATHS.data, 'video-history.json'), JSON.stringify(videos));+ progress('inbox-seed');const accounts = {};for (const [accountIndex, id] of ACCOUNT_IDS.entries()) {accounts[id] = { id, name: 'Synthetic inbox ' + (accountIndex + 1), type: 'gmail', provider: 'api',email: 'fixture' + accountIndex + '@example.com', enabled: false, syncConfig: {}, lastSyncAt: null };const messages = Array.from({ length: CARDINALITIES.messages / CARDINALITIES.accounts },(_, index) => messageFixture(id, index + accountIndex * CARDINALITIES.messages / CARDINALITIES.accounts));- await writeFile(join(PATHS.messages, 'cache', id + '.json'), JSON.stringify({ syncCursor: null, messages }));+ await writeSeedFile(join(PATHS.messages, 'cache', id + '.json'), JSON.stringify({ syncCursor: null, messages }));+ counts.accounts++;+ counts.messages += messages.length;+ progress('inbox-seed');}await writeFile(join(PATHS.messages, 'accounts.json'), JSON.stringify(accounts));await writeFile(join(PATHS.data, 'settings.json'), JSON.stringify({ timezone: 'UTC', instances: [] }));- phase = 'routes';+ progress('routes');const { createImageGalleryHandlers } = await import('../../server/routes/imageGalleryRead.js');const { createVideoHistoryItemRead } = await import('../../server/routes/videoHistoryRead.js');const { messageInboxRead, messageDetailRead } = await import('../../server/routes/messageInboxRead.js');