Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
feat: exercise Postgres backup and restore in the upgrade rehearsal (#12906)
Adam Eivy9baded938ace
diff --git a/docs/features/app-quality.md b/docs/features/app-quality.mdindex 90b1b4fdaf..6edeb24d96 100644--- a/docs/features/app-quality.md+++ b/docs/features/app-quality.md@@ -143,7 +143,11 @@ breaking changes, idempotency keys, pagination bounds), `observability`installs from migration-test fixture shapes, runs the migration runner and abackup restore against them, and boots the result, so a failure only anexecuted path shows (#6182, #12044, #12046, #12049) is caught without touching-live data.+live data. On a host with Postgres it also round-trips the database: backup,+then `restorePostgres` into a second throwaway `portos_rehearsal_<id>_test`+database made and dropped by `scripts/rehearsal-scratch-db.js` (#12906); with+no reachable Postgres it falls back to the file path and reports Postgres as+uncovered.Two gap lenses cover defects every page-walking and code-shape audit reads asintentional or never reaches:diff --git a/docs/features/self-improvement-audits.md b/docs/features/self-improvement-audits.mdindex bfc75f194f..ddc24f7835 100644--- a/docs/features/self-improvement-audits.md+++ b/docs/features/self-improvement-audits.md@@ -153,7 +153,12 @@ problem:executes them: it builds scratch installs from the fixture shapes themigration tests carry, runs the seed step and the migration runner, boots tothe health endpoint, and restores a backup into a second scratch root — never- the live `data/` or the real Postgres — and files only what the executed path+ the live `data/` or the real Postgres. Postgres is exercised through two+ throwaway `portos_rehearsal_<id>_test` databases that+ `scripts/rehearsal-scratch-db.js` creates and drops (the `_test` suffix keeps+ the `isTestDatabase()` gate armed, and `drop` refuses any other name,+ including `portos` and `portos_test`); with no creatable scratch database the+ run reports the Postgres steps as uncovered. It files only what the executed pathbroke, one issue per root cause (#12091).- **Defects are split four ways.** `better-runtime-safety` reads source forlatent defects; `console-errors` collects what actually errors at runtime;diff --git a/scripts/rehearsal-scratch-db.js b/scripts/rehearsal-scratch-db.jsnew file mode 100644index 0000000000..f0fc16498a--- /dev/null+++ b/scripts/rehearsal-scratch-db.js@@ -0,0 +1,132 @@+#!/usr/bin/env node+// Throwaway Postgres databases for the upgrade-and-restore rehearsal.+//+// node scripts/rehearsal-scratch-db.js create [--empty] -> {"ok":true,"action":"create","database":"portos_rehearsal_<id>_test",...}+// node scripts/rehearsal-scratch-db.js drop <name>+//+// Output is one JSON object on stdout; a refusal or failure exits non-zero with+// {"ok":false,"error":...}. The `_test` suffix is deliberate: server/lib/db.js+// isTestDatabase() accepts /_test$/, so the test-runner gate stays armed for a+// rehearsal that points PGDATABASE here and can never be aimed at `portos`.+// `create` applies server/scripts/init-db.sql (the base schema ensureSchema() builds on, as+// `npm run setup:db:test` does) unless --empty asks for a bare database.+// `drop` is the only destructive verb and refuses every name that this helper+// would not have created — including `portos` and `portos_test`.++import { createRequire } from 'module';+import { execFileSync } from 'child_process';+import { existsSync, readFileSync } from 'fs';+import { randomBytes } from 'crypto';+import { fileURLToPath } from 'url';+import { dirname, join } from 'path';+import databaseSettings from './lib/databaseSettings.cjs';+import { parseEnvFile } from './lib/envFile.js';+import { isDirectlyInvoked } from './lib/directInvocation.js';++const rootDir = join(dirname(fileURLToPath(import.meta.url)), '..');++export const SCRATCH_DB_PATTERN = /^portos_rehearsal_[a-f0-9]{12}_test$/;++export const newScratchDbName = () => `portos_rehearsal_${randomBytes(6).toString('hex')}_test`;++/** Throws unless `name` is a database this helper could have created. */+export function assertScratchDbName(name) {+ if (typeof name !== 'string' || !SCRATCH_DB_PATTERN.test(name)) {+ throw new Error(`refusing "${String(name).slice(0, 80)}": only portos_rehearsal_<12 hex>_test databases may be created or dropped here`);+ }+ return name;+}++// `.env` is untracked, so a linked worktree has none; the primary checkout that owns the+// worktree's git directory holds the install's database settings.+function findEnvFile(root) {+ const own = join(root, '.env');+ if (existsSync(own)) return own;+ try {+ const common = execFileSync('git', ['rev-parse', '--path-format=absolute', '--git-common-dir'], { cwd: root, encoding: 'utf8', stdio: ['ignore', 'pipe', 'ignore'] }).trim();+ return join(dirname(common), '.env');+ } catch {+ return own;+ }+}++/** Connection to the install's Postgres, same resolution as scripts/setup-db.js. */+export function resolveAdminConnection({ env = process.env, root = rootDir } = {}) {+ const settings = databaseSettings.resolveDatabaseSettings({ env, saved: parseEnvFile(findEnvFile(root)) });+ return {+ host: settings.host,+ port: settings.port,+ user: settings.user,+ password: settings.password,+ // CREATE/DROP DATABASE cannot run inside the target; `postgres` always exists.+ database: env.PGADMINDATABASE || 'postgres',+ };+}++const loadPg = () => createRequire(join(rootDir, 'server', 'package.json'))('pg');++async function withAdmin(connection, fn) {+ const { Client } = loadPg();+ const client = new Client({ ...connection, connectionTimeoutMillis: 5000 });+ await client.connect();+ try {+ return await fn(client);+ } finally {+ await client.end().catch(() => {});+ }+}++const INIT_SQL = join(rootDir, 'server', 'scripts', 'init-db.sql');++async function applyBaseSchema(connection, database) {+ const { Client } = loadPg();+ const client = new Client({ ...connection, database, connectionTimeoutMillis: 5000 });+ await client.connect();+ try {+ await client.query(readFileSync(INIT_SQL, 'utf8'));+ } finally {+ await client.end().catch(() => {});+ }+}++// Identifiers are interpolated (DDL cannot be parameterized) only after+// assertScratchDbName has matched the fixed pattern.+export async function createScratchDb(connection = resolveAdminConnection(), { empty = false } = {}) {+ const database = assertScratchDbName(newScratchDbName());+ await withAdmin(connection, client => client.query(`CREATE DATABASE "${database}"`));+ try {+ if (!empty) await applyBaseSchema(connection, database);+ } catch (err) {+ // A half-built scratch database is never handed back; drop it before reporting the failure.+ await dropScratchDb(database, connection).catch(() => {});+ throw err;+ }+ const { host, port, user } = connection;+ return { database, host, port, user, baseSchema: !empty };+}++export async function dropScratchDb(name, connection = resolveAdminConnection()) {+ const database = assertScratchDbName(name);+ // WITH (FORCE) disconnects a straggling rehearsal server so cleanup on a failed run still succeeds.+ await withAdmin(connection, client => client.query(`DROP DATABASE IF EXISTS "${database}" WITH (FORCE)`));+ return { database };+}++// A refused connection surfaces as an AggregateError with an empty message (one per address family).+export const describeError = err => err.message || [err.code, ...(err.errors ?? []).map(e => e.code)].filter(Boolean).join('/') || String(err);++async function main(argv) {+ const [action, name] = argv;+ if (action === 'create') return { action, ...await createScratchDb(undefined, { empty: argv.includes('--empty') }) };+ if (action === 'drop') return { action, ...await dropScratchDb(name) };+ throw new Error('usage: rehearsal-scratch-db.js create [--empty] | drop <portos_rehearsal_<id>_test>');+}++if (isDirectlyInvoked(import.meta.url)) {+ main(process.argv.slice(2))+ .then(result => console.log(JSON.stringify({ ok: true, ...result })))+ .catch(err => {+ console.log(JSON.stringify({ ok: false, error: describeError(err) }));+ process.exit(1);+ });+}diff --git a/scripts/rehearsal-scratch-db.test.js b/scripts/rehearsal-scratch-db.test.jsnew file mode 100644index 0000000000..e1c2106ebc--- /dev/null+++ b/scripts/rehearsal-scratch-db.test.js@@ -0,0 +1,58 @@+import { describe, it, expect } from 'vitest';+import { spawnSync } from 'node:child_process';+import { dirname, join } from 'node:path';+import { fileURLToPath } from 'node:url';+import {+ SCRATCH_DB_PATTERN,+ assertScratchDbName,+ newScratchDbName,+ resolveAdminConnection,+} from './rehearsal-scratch-db.js';++const script = join(dirname(fileURLToPath(import.meta.url)), 'rehearsal-scratch-db.js');++describe('rehearsal scratch database naming', () => {+ it('generates names the test-runner gate (/_test$/) accepts and drop accepts', () => {+ const name = newScratchDbName();+ expect(name).toMatch(/_test$/);+ expect(SCRATCH_DB_PATTERN.test(name)).toBe(true);+ expect(newScratchDbName()).not.toBe(name);+ });++ it.each([+ 'portos',+ 'portos_test',+ 'portos_smoke_test',+ 'portos_rehearsal_test',+ 'portos_rehearsal_abc_test',+ 'portos_rehearsal_0123456789ab',+ 'portos_rehearsal_0123456789ab_test; DROP DATABASE portos',+ 'portos_rehearsal_0123456789ab_test"',+ '',+ undefined,+ ])('refuses %j', name => {+ expect(() => assertScratchDbName(name)).toThrow(/refusing/);+ });++ it('keeps the admin connection off the install database', () => {+ const connection = resolveAdminConnection({ env: { PGMODE: 'native', PGPORT: '5432' }, root: '/nonexistent' });+ expect(connection.database).toBe('postgres');+ expect(connection.port).toBe(5432);+ });+});++describe('rehearsal-scratch-db CLI', () => {+ // The name check runs before any connection, so these never reach a database.+ it.each(['portos', 'portos_test', 'portos_rehearsal_nothex_test'])('drop %s exits non-zero with a JSON refusal', name => {+ const result = spawnSync(process.execPath, [script, 'drop', name], { encoding: 'utf8' });+ expect(result.status).toBe(1);+ expect(JSON.parse(result.stdout)).toMatchObject({ ok: false });+ expect(JSON.parse(result.stdout).error).toMatch(/refusing/);+ });++ it('rejects an unknown action', () => {+ const result = spawnSync(process.execPath, [script, 'wipe'], { encoding: 'utf8' });+ expect(result.status).toBe(1);+ expect(JSON.parse(result.stdout).ok).toBe(false);+ });+});diff --git a/server/services/taskPromptDefaults.test.js b/server/services/taskPromptDefaults.test.jsindex 4b5e9bcea8..a8d230b971 100644--- a/server/services/taskPromptDefaults.test.js+++ b/server/services/taskPromptDefaults.test.js@@ -186,7 +186,6 @@ describe('taskPromptDefaults integrity snapshot', () => {it.each([['feature-completeness', ['Stubs reachable from the interface', 'Decide, do not defer', 'Static evidence is enough']],['workflow-recovery', ['Work from server state, not from views', 'Disabled controls say why', 'id of what holds it']],- ['upgrade-rehearsal', ['Build a synthetic older install', 'Restore a backup', 'One issue per root cause']],])('%s v1 states what makes it its own lane', (key, markers) => {const current = DEFAULT_TASK_PROMPTS[key];expect(PROMPT_VERSIONS[key]).toBe(1);@@ -203,7 +202,27 @@ describe('taskPromptDefaults integrity snapshot', () => {for (const marker of ['never the live `data/`', 'never the real Postgres', '`MEMORY_BACKEND=file`', '`NODE_ENV=test`', 'port']) {expect(opening).toContain(marker);}- expect(opening).toMatch(/out of\nbounds even when it only reads/);+ expect(opening).toMatch(/out of\s+bounds even when it only reads/);+ });++ // v2 (#12906): the rehearsal must reach the primary store's dump and restore+ // through helper-made scratch databases, and still degrade to an honest+ // "uncovered" report when none can be made.+ it('upgrade-rehearsal v2 round-trips Postgres through scratch databases only', () => {+ const current = DEFAULT_TASK_PROMPTS['upgrade-rehearsal'];+ expect(PROMPT_VERSIONS['upgrade-rehearsal']).toBe(2);+ expect(current).toContain('{modeInstructions}');+ for (const marker of [+ 'Build a synthetic older install',+ 'Restore a backup',+ 'One issue per root cause',+ 'scripts/rehearsal-scratch-db.js',+ 'restorePostgres',+ 'portos_rehearsal_',+ 'File-only fallback',+ ]) expect(current, marker).toContain(marker);+ expect(current).toMatch(/under about\s+60 characters/);+ expect(current).toMatch(/as uncovered,\s+quoting the helper/);});it('module-hygiene v1 is generic, evidence-led, and bounded', () => {diff --git a/server/services/taskPromptDefaults/integrity.snapshot.json b/server/services/taskPromptDefaults/integrity.snapshot.jsonindex 65909d45f4..b9301a8b1b 100644--- a/server/services/taskPromptDefaults/integrity.snapshot.json+++ b/server/services/taskPromptDefaults/integrity.snapshot.json@@ -60,7 +60,7 @@"typing": "bd7cb874b2784395280b116369c7e7ac","ui-bugs": "06baf650e41fce12aaa59c66030c0357","ui-lifecycle": "d156882c73593a54c0fd5754eb4c6fa0",- "upgrade-rehearsal": "ef9d54fa653d6639abd4707b4f392d8d",+ "upgrade-rehearsal": "e53a4cb57cdc7b5876d709bcd8a68132","user-action-review": "28c1b49e5ca806568077f56fe982ce08","ux": "14d790bd65e540d00786ecb4e787a61e","workflow-recovery": "628d2ec51c3cd3d8bbd76eacb212dada"@@ -121,7 +121,7 @@"typing": 3,"ui-bugs": 4,"ui-lifecycle": 3,- "upgrade-rehearsal": 1,+ "upgrade-rehearsal": 2,"user-action-review": 3,"ux": 8,"workflow-recovery": 1@@ -501,6 +501,9 @@"14b3816d1bcc10f0d7d6357e55cc5e69","d2eaa653e57897d3fb8b6c52c2c18e38"],+ "upgrade-rehearsal": [+ "ef9d54fa653d6639abd4707b4f392d8d"+ ],"user-action-review": ["945a716d446d26a03e2805d28bd1ae16","1a083430e43f60cdd586cfffbdebc583"diff --git a/server/services/taskPromptDefaults/prompts.js b/server/services/taskPromptDefaults/prompts.jsindex 21475d8754..344dbfbb52 100644--- a/server/services/taskPromptDefaults/prompts.js+++ b/server/services/taskPromptDefaults/prompts.js@@ -2328,12 +2328,19 @@ offered.`,Execute {appName}'s upgrade path and its backup restore against scratchinstalls, and file what breaks. Every command in this run targets a scratch-directory you create: never the live \`data/\`, never the real Postgres, with-\`MEMORY_BACKEND=file\` and \`NODE_ENV=test\` in the environment and a free-loopback port chosen inside the project's documented port band (\`docs/PORTS.md\`)-and checked free first, never the running server's. A command whose working-directory, data root, database name, or port is the live install's is out of-bounds even when it only reads.+directory or scratch database you create: never the live \`data/\`, never the real Postgres+(the install's \`PGDATABASE\`, \`portos\`, \`portos_test\`), with+\`NODE_ENV=test\` in the environment and a free loopback port chosen inside the+project's documented port band (\`docs/PORTS.md\`) and checked free first, never+the running server's. The only Postgres databases you may touch are the+\`portos_rehearsal_<id>_test\` ones that \`node scripts/rehearsal-scratch-db.js+create\` makes for you; \`MEMORY_BACKEND=file\` is set only on the file-only+fallback below. Keep each scratch root and its private HOME short (under about+60 characters): a HOME nested inside another temp directory pushes the pm2+interactor socket path past the 104-byte macOS limit and the restored boot+exits with EINVAL, which reads as a restore defect but is not one. A command+whose working directory, data root, database name, or port is the live+install's is out of bounds even when it only reads.Repository: {repoPath}@@ -2363,11 +2370,40 @@ executed-path failures that no static read of one file surfaces.scratch root with \`restoreSnapshot\` — dry run first, then execute — andverify the manifest: every recorded file present with its recorded hash,nothing extra, nothing the current code refuses to read.-3. **Boot the restored root** the same way and confirm its health endpoint.+3. **Round-trip Postgres.** Postgres is the primary store and its dump and+ restore are where restore defects have been found, so exercise them.+ Run \`node scripts/rehearsal-scratch-db.js create\` (one JSON line on stdout):+ when \`ok\` is false — no reachable Postgres, or a role that cannot create the+ base schema's extensions — use the file-only fallback below. Otherwise create+ two databases, a source and a target, and drop both with+ \`node scripts/rehearsal-scratch-db.js drop <name>\` however the run ends,+ success or failure (the helper refuses any other name; list a database you+ could not drop in the report).+ - **Back up.** With \`PGDATABASE=<source>\`, \`NODE_ENV=test\` and the source+ scratch root as the data root, run \`ensureSchema\` (\`server/lib/db.js\`),+ insert a few synthetic rows, and take the snapshot with \`runBackup\`; the+ dump (\`dumpPostgres\`) must report \`ok\`, not \`skipped\`.+ - **Restore.** In a separate process with \`PGDATABASE=<target>\` and the+ second scratch root as the data root, call \`restorePostgres\` on that+ snapshot — dry run first, then execute. A restore replays into whatever+ \`PGDATABASE\` its process holds, so print it and confirm the+ \`portos_rehearsal_\` prefix before each call.+ - **Compare.** Row counts per table and the \`schema_migrations\` ledger must+ match between source and target; any other status than \`ok\`, or a+ mismatch, is a finding.+4. **Boot the restored root** the same way and confirm its health endpoint;+ when step 3 ran, boot it with \`MEMORY_BACKEND=postgres\` and+ \`PGDATABASE=<target>\`, otherwise with \`MEMORY_BACKEND=file\`.++**File-only fallback.** When no scratch database can be created, run every step+except 3 with \`MEMORY_BACKEND=file\` and \`NODE_ENV=test\`, and report the+Postgres dump, \`restorePostgres\`, and the database-backed boot as uncovered,+quoting the helper's \`error\` text — never as passing.A repository with a different runner substitutes its documented equivalents;one with no migration runner and no backup path reports not-applicable and-stops. Record every command with its data root so the run can be repeated.+stops. Record every command with its data root and database name so the run+can be repeated.## Hunt for@@ -2381,8 +2417,11 @@ stops. Record every command with its data root so the run can be repeated.- **A restore that does not round-trip** — a file the manifest records butthe restore skips, a hash mismatch, a restored record the booted serverrejects.-- **A boot step the rehearsal could not reach** — one that runs only against- the real database stays unexercised; report it as uncovered, never as+- **A Postgres restore that does not round-trip** — a dump \`restorePostgres\`+ refuses, a table whose row count or the migration ledger differs between+ source and target, a restored database the booted server rejects.+- **A step the rehearsal could not reach** — one that needs a database it+ could not create stays unexercised; report it as uncovered, never aspassing.## Not yoursdiff --git a/server/services/taskPromptDefaults/versions.js b/server/services/taskPromptDefaults/versions.jsindex 378395e888..0c10ef6fb5 100644--- a/server/services/taskPromptDefaults/versions.js+++ b/server/services/taskPromptDefaults/versions.js@@ -74,7 +74,7 @@ export const PROMPT_VERSIONS = {'privacy': 2, // v2: drops the redaction sentence the mode banners own (#12080). v1: personal/sensitive-data audit — operational exposure, third-party minimization, API over-exposure, retention, erasure reach, real data in fixtures. Honors the project's documented sharing model. Net-new type.'cost-efficiency': 1, // v1: metered-spend audit — repeated paid calls, unrequested provider work, retry multiplication, unbounded scans, storage growth, over-provisioning, egress. Net-new type.'feature-completeness': 1, // v1: incomplete-feature audit — stubs and 501s reachable from the UI, permanently disabled or "coming soon" controls, UI/doc promises the code does not keep, settings nothing reads, capabilities with no entry point, events nobody hears, plans that disagree with the code, deferred work with no live tracker item. Decides finish / hide / remove. Net-new type.- 'upgrade-rehearsal': 1, // v1: executed upgrade-and-restore rehearsal (#12091) — opens with the scratch-only rule (never the live data/ or real Postgres, MEMORY_BACKEND=file + NODE_ENV=test, a free port from the documented band), builds synthetic older installs from migration-test fixtures, runs the seed step and migration runner, boots to the health endpoint, restores a backup into a second scratch root and verifies its manifest; one issue per root cause. Net-new type.+ 'upgrade-rehearsal': 2, // v2 (#12906): Postgres round-trip in throwaway portos_rehearsal_<id>_test databases (scripts/rehearsal-scratch-db.js) — backup, restorePostgres dry run then execute into a second database, row-count and ledger parity, file-only fallback reports Postgres as uncovered; scratch roots and HOME kept short (pm2 socket path limit). v1: executed upgrade-and-restore rehearsal (#12091) — opens with the scratch-only rule (never the live data/ or real Postgres, MEMORY_BACKEND=file + NODE_ENV=test, a free port from the documented band), builds synthetic older installs from migration-test fixtures, runs the seed step and migration runner, boots to the health endpoint, restores a backup into a second scratch root and verifies its manifest; one issue per root cause. Net-new type.'workflow-recovery': 1, // v1: workflow completion and recovery audit — inventories server in-progress states and checks each has a visible resume/cancel/reattach exit, busy refusals carry the holder id, client-orchestrated chains survive a failed follow-up step or reload, waits time out, and duplicate-preventing markers live on the server. Net-new type.'better-dev-environment': 2, // v2: drops the redaction clause the mode banners own (#12080). v1: host impact of the development loop, measured by diffing host state around a bounded run — leaked temp files/processes/ports, writes outside the sandbox, accumulation, watcher/idle cost, suite runtime. Net-new type.'better-test-quality': 2, // v2: adds mocks that drift from the module they replace (members/exports the real module lacks). Previous version: v1: vacuous/weak/redundant existing tests, proven with a mutation probe. Distinct from test-coverage, which owns the gaps.