Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
feat: add the upgrade-rehearsal audit lane that executes migrations and a backup restore in scratch installs
[._.]/ Adam Eivy80fd6a2c9b70
diff --git a/docs/features/app-quality.md b/docs/features/app-quality.mdindex 0274d88990..90b1b4fdaf 100644--- a/docs/features/app-quality.md+++ b/docs/features/app-quality.md@@ -139,6 +139,12 @@ breaking changes, idempotency keys, pagination bounds), `observability`(correlation, alertable metrics, truthful health, audit trails) and`data-safety` (backup restorability, bulk/backfill safety, retention reach).+`upgrade-rehearsal` is `data-safety`'s executed counterpart: it builds scratch+installs from migration-test fixture shapes, runs the migration runner and a+backup restore against them, and boots the result, so a failure only an+executed path shows (#6182, #12044, #12046, #12049) is caught without touching+live data.+Two gap lenses cover defects every page-walking and code-shape audit reads asintentional or never reaches:diff --git a/docs/features/self-improvement-audits.md b/docs/features/self-improvement-audits.mdindex 9ecf6e6b32..e66421c2bd 100644--- a/docs/features/self-improvement-audits.md+++ b/docs/features/self-improvement-audits.md@@ -106,6 +106,7 @@ the operator's live checkout.| `cost-efficiency` | Metered spend: paid calls, unbounded scans, over-provisioning | `performance`, `cost` | file | || `feature-completeness` | Work started and never finished, offered to users anyway | `bug`, `incomplete` | file | || `workflow-recovery` | Multi-step workflows that strand their record between steps | `bug`, `workflow` | file | |+| `upgrade-rehearsal` | Executed migrations, backup restore and boot against scratch installs | `data-safety`, `upgrade-rehearsal` | file | |## Why the boundaries are explicit@@ -147,6 +148,13 @@ problem:keeps only check-then-act races on in-memory module state (#12082).`error-handling` likewise cedes sensitive detail leaking through an error pathto `privacy` and partial writes to `data-integrity` (#12086).+- **Upgrade safety is split by read versus run.** `data-safety` reads+ migrations, seeds and backup code for missing safety nets. `upgrade-rehearsal`+ executes them: it builds scratch installs from the fixture shapes the+ migration tests carry, runs the seed step and the migration runner, boots to+ the health endpoint, and restores a backup into a second scratch root — never+ the live `data/` or the real Postgres — and files only what the executed path+ broke, one issue per root cause (#12091).- **Defects are split four ways.** `better-runtime-safety` reads source forlatent defects; `console-errors` collects what actually errors at runtime;`error-handling` owns the failure path once something goes wrong; anddiff --git a/server/lib/auditCatalog.js b/server/lib/auditCatalog.jsindex d5dd36c423..06663d435f 100644--- a/server/lib/auditCatalog.js+++ b/server/lib/auditCatalog.js@@ -623,6 +623,25 @@ export const AUDIT_DEFINITIONS = Object.freeze({noun: 'workflow-recovery finding(s)',}),},+ // Executes the upgrade path and a backup restore against scratch roots built+ // from fixture shapes. `data-safety` reads migrations and backup code+ // statically; the failures that reached installs (#6182, #12044, #12046,+ // #12049) were visible only on the executed path.+ 'upgrade-rehearsal': {+ quotaBurnId: null,+ label: 'Upgrade & restore rehearsal',+ description: 'Upgrade and restore rehearsal (runs migrations over synthetic older installs and a backup restore in scratch directories, never the live data) — configurable: file issues (default) or implement fixes',+ defaultFileIssues: true,+ filing: filing({+ slugPrefix: 'upgrade-rehearsal-',+ label: 'upgrade-rehearsal-audit',+ // Same category as the static read; the metric label separates what an+ // executed rehearsal found from what a read of the code found.+ issueLabel: 'data-safety',+ labelDescription: 'Proposed from an upgrade and restore rehearsal',+ noun: 'upgrade-rehearsal finding(s)',+ }),+ },});export const AUDIT_TASK_TYPES = new Set(Object.keys(AUDIT_DEFINITIONS));@@ -897,7 +916,7 @@ export const DO_BETTER_SCOPE_COVERAGE = Object.freeze({dry: ['simplify'],architecture: ['module-hygiene', 'api-contract'],'bugs-perf': ['better-runtime-safety', 'performance', 'error-handling', 'observability', 'reliability', 'cost-efficiency', 'workflow-recovery'],- 'stack-specific': ['ui-lifecycle', 'accessibility', 'data-safety', 'security', 'infrastructure', 'data-integrity'],+ 'stack-specific': ['ui-lifecycle', 'accessibility', 'data-safety', 'upgrade-rehearsal', 'security', 'infrastructure', 'data-integrity'],deps: ['better-dependency-freedom', 'security'],tests: ['test-coverage', 'better-test-quality', 'better-dev-environment'],ux: ['ux', 'mobile-responsive', 'copy', 'feature-completeness'],@@ -970,6 +989,9 @@ export const AUDIT_SUGGESTED_AFTER = Object.freeze({reliability: Object.freeze(['error-handling']),'workflow-recovery': Object.freeze(['error-handling']),'cost-efficiency': Object.freeze(['performance']),+ // The executed rehearsal follows the static read, so it spends its scratch+ // boots on the hazards the read could not see.+ 'upgrade-rehearsal': Object.freeze(['data-safety']),});// WHY each audit sits where it does in the order above — rationale only. The@@ -1000,4 +1022,5 @@ export const AUDIT_RUN_GUIDANCE = Object.freeze({'cost-efficiency': 'Follows performance, because a latency fix often changes what the path costs; measure spend on the shape that survived.','feature-completeness': 'Runs before simplify sweeps unreferenced code, so an unfinished feature is decided (finish, hide, or remove) rather than silently deleted.','workflow-recovery': 'Follows call-level failure handling: once each step fails safely, check that a workflow stopped between steps still offers the user a way to finish or cancel it.',+ 'upgrade-rehearsal': 'Follows the static data-safety read: once the hazards a read can see are fixed, execute migrations and a backup restore against scratch installs to catch what only running them shows.',});diff --git a/server/lib/auditCatalog.test.js b/server/lib/auditCatalog.test.jsindex 7fff0f2ec1..27c207398b 100644--- a/server/lib/auditCatalog.test.js+++ b/server/lib/auditCatalog.test.js@@ -106,6 +106,15 @@ describe('AUDIT_DEFINITIONS', () => {expect(auditCapabilityRequirement('better-test-quality')).toBe('tests');});+ // The executed rehearsal shares data-safety's category and follows it in the+ // advisory order; its metric label keeps the two sets of findings apart.+ it('registers upgrade-rehearsal as data-safety\'s executed, ungated successor', () => {+ expect(AUDIT_SUGGESTED_AFTER['upgrade-rehearsal']).toEqual(['data-safety']);+ expect(AUDIT_DEFINITIONS['upgrade-rehearsal'].filing).toMatchObject({ issueLabel: 'data-safety', extraLabels: ['upgrade-rehearsal'] });+ expect(auditCapabilityRequirement('upgrade-rehearsal')).toBeNull();+ expect(defaultFileIssuesFor('upgrade-rehearsal')).toBe(true);+ });+it('enumerates the audits in catalog order without a second vocabulary', () => {expect(AUDIT_TASK_TYPE_LIST).toEqual([...AUDIT_TASK_TYPES]);expect(AUDIT_TASK_TYPE_LIST.every(isAuditTaskType)).toBe(true);diff --git a/server/lib/auditQuality.js b/server/lib/auditQuality.jsindex 4b5115f473..13874a049d 100644--- a/server/lib/auditQuality.js+++ b/server/lib/auditQuality.js@@ -40,6 +40,7 @@ export const AUDIT_DISCOVERY = Object.freeze({privacy: 'Inventory personal and sensitive fields in schemas and payloads, then trace them to logs, errors, analytics, caches, exports, third-party calls and deletion paths; rank exposure already happening over missing governance.','feature-completeness': 'Scan for not-implemented/501/placeholder returns, permanently disabled or "coming soon" controls, settings written but never read, endpoints and jobs with no caller, server events with no listener, and to-do references to missing or closed tracker items; trace each to the control or document that offers it, ranking what users are offered today over internal gaps.','workflow-recovery': 'Inventory server-side in-progress states (persisted statuses, in-memory locks and in-flight maps, leases, "already in progress" refusals) and the client chains that drive them; trace each state to the page that must show it and its resume/cancel/reattach exit, ranking records left blocked with no visible exit first.',+ 'upgrade-rehearsal': 'Inventory migrations, seeds, the migration runner and its ledger, and the backup/restore path; build scratch installs from the fixture shapes the migration tests carry and execute upgrade, restore and boot against them, ranking a migration or restore that fails or silently no-ops on a real older shape first.','cost-efficiency': 'Inventory metered dependencies (paid APIs, model providers, cloud storage/query/compute, egress) and their call sites, schedules and retry paths; rank spend by frequency times unit cost and growth with usage.',});diff --git a/server/lib/scheduledTaskTypes.js b/server/lib/scheduledTaskTypes.jsindex f4ea460626..2e665b6985 100644--- a/server/lib/scheduledTaskTypes.js+++ b/server/lib/scheduledTaskTypes.js@@ -136,6 +136,10 @@ export const SELF_IMPROVEMENT_TASK_TYPES = [// or the tab reloads. Distinct from `ui-bugs` (one click on a fresh page) and// `error-handling` (per-call failure handling).'workflow-recovery',+ // Executes migrations and a backup restore against scratch installs built+ // from fixture shapes, then boots the result. Distinct from `data-safety`,+ // which reads the same code statically.+ 'upgrade-rehearsal',// Audits `git stash list` for {appName} and drops entries already superseded// by (or a subset of) current `main`/HEAD, or that are stale/abandoned scratch// work — without discarding real unlanded work. On-demand only (no cadencediff --git a/server/services/taskPromptDefaults.test.js b/server/services/taskPromptDefaults.test.jsindex cdaa0245df..a4ea1b51b7 100644--- a/server/services/taskPromptDefaults.test.js+++ b/server/services/taskPromptDefaults.test.js@@ -186,6 +186,7 @@ describe('taskPromptDefaults integrity snapshot', () => {it.each([['feature-completeness', ['Stubs reachable from the interface', 'Decide, do not defer', 'Static evidence is enough']],['workflow-recovery', ['Work from server state, not from views', 'Disabled controls say why', 'id of what holds it']],+ ['upgrade-rehearsal', ['Build a synthetic older install', 'Restore a backup', 'One issue per root cause']],])('%s v1 states what makes it its own lane', (key, markers) => {const current = DEFAULT_TASK_PROMPTS[key];expect(PROMPT_VERSIONS[key]).toBe(1);@@ -193,6 +194,18 @@ describe('taskPromptDefaults integrity snapshot', () => {for (const marker of markers) expect(current, key).toContain(marker);});+ // The rehearsal EXECUTES migrations and a restore, so the scratch-only rule+ // has to be the first thing the agent reads, before any step it could run+ // against the install it was spawned from (#12091).+ it('upgrade-rehearsal opens with the scratch-only rule and forbids the live install', () => {+ const current = DEFAULT_TASK_PROMPTS['upgrade-rehearsal'];+ const opening = current.slice(0, current.indexOf('Repository: {repoPath}'));+ for (const marker of ['never the live `data/`', 'never the real Postgres', '`MEMORY_BACKEND=file`', '`NODE_ENV=test`', 'port']) {+ expect(opening).toContain(marker);+ }+ expect(opening).toMatch(/out of\nbounds even when it only reads/);+ });+it('module-hygiene v1 is generic, evidence-led, and bounded', () => {const current = DEFAULT_TASK_PROMPTS['module-hygiene'];diff --git a/server/services/taskPromptDefaults/integrity.snapshot.json b/server/services/taskPromptDefaults/integrity.snapshot.jsonindex 8bd1a5f359..ffc614346f 100644--- a/server/services/taskPromptDefaults/integrity.snapshot.json+++ b/server/services/taskPromptDefaults/integrity.snapshot.json@@ -60,6 +60,7 @@"typing": "bd7cb874b2784395280b116369c7e7ac","ui-bugs": "06baf650e41fce12aaa59c66030c0357","ui-lifecycle": "d156882c73593a54c0fd5754eb4c6fa0",+ "upgrade-rehearsal": "ef9d54fa653d6639abd4707b4f392d8d","user-action-review": "1a083430e43f60cdd586cfffbdebc583","ux": "14d790bd65e540d00786ecb4e787a61e","workflow-recovery": "628d2ec51c3cd3d8bbd76eacb212dada"@@ -120,6 +121,7 @@"typing": 3,"ui-bugs": 4,"ui-lifecycle": 3,+ "upgrade-rehearsal": 1,"user-action-review": 2,"ux": 8,"workflow-recovery": 1diff --git a/server/services/taskPromptDefaults/prompts.js b/server/services/taskPromptDefaults/prompts.jsindex d22a7809c7..80a9a211d0 100644--- a/server/services/taskPromptDefaults/prompts.js+++ b/server/services/taskPromptDefaults/prompts.js@@ -2292,6 +2292,82 @@ flag, or a setting nothing reads is proven by reading the code; it does not needa live reproduction. Never quote a user's records when describing what isoffered.`,+ 'upgrade-rehearsal': `[Improvement: {appName}] Upgrade and restore rehearsal++Execute {appName}'s upgrade path and its backup restore against scratch+installs, and file what breaks. Every command in this run targets a scratch+directory you create: never the live \`data/\`, never the real Postgres, with+\`MEMORY_BACKEND=file\` and \`NODE_ENV=test\` in the environment and a free+loopback port chosen inside the project's documented port band (\`docs/PORTS.md\`)+and checked free first, never the running server's. A command whose working+directory, data root, database name, or port is the live install's is out of+bounds even when it only reads.++Repository: {repoPath}++{modeInstructions}++Reading a migration shows what it intends; running it against an older shape+shows what it does. The failures that reach installs — a seed landing before+the migration that derives from it, a non-atomic write under a crash, two+runners sharing one ledger, a migration importing a live service — are+executed-path failures that no static read of one file surfaces.++## Rehearse++1. **Build a synthetic older install.** Create a disposable root the way+ \`scripts/smoke-boot.js\` (\`createSmokeRoot\`) does — marker file, private+ HOME and TMPDIR, a copy of \`data.reference/\` — then overlay two or three+ past record shapes taken from the fixtures in \`scripts/migrations/*.test.js\`:+ the oldest shape still carried, a mid-history shape, and the shape the+ newest derive-style migration reads. Seed \`data/\` from the reference copy+ (\`scripts/lib/seedDataDir.js\`, the step \`scripts/setup-data.js\` performs),+ run \`scripts/run-migrations.js\` (\`runMigrations({ rootDir })\`) against that+ root, then boot the server with \`PORTOS_DATA_ROOT\` pinned to it on your port+ and wait for \`/api/system/health\` before stopping it.+2. **Restore a backup.** Produce a snapshot from the scratch data with+ \`services/backup.js\` (\`runBackup\`, which writes the manifest), or use the+ newest snapshot already readable to you, and restore it into a second+ scratch root with \`restoreSnapshot\` — dry run first, then execute — and+ verify the manifest: every recorded file present with its recorded hash,+ nothing extra, nothing the current code refuses to read.+3. **Boot the restored root** the same way and confirm its health endpoint.++A repository with a different runner substitutes its documented equivalents;+one with no migration runner and no backup path reports not-applicable and+stops. Record every command with its data root so the run can be repeated.++## Hunt for++- **A migration that fails or no-ops on a real older shape** — a crash, a+ derive skipped because its output was seeded first, a result that differs+ from what the migration's own test asserts.+- **A ledger or lock that misbehaves under a second runner** — start the+ runner twice against one root and compare the applied list.+- **A write that leaves a torn file** — stop the runner mid-migration and+ check what the next run finds.+- **A restore that does not round-trip** — a file the manifest records but+ the restore skips, a hash mismatch, a restored record the booted server+ rejects.+- **A boot step the rehearsal could not reach** — one that runs only against+ the real database stays unexercised; report it as uncovered, never as+ passing.++## Not yours++Reading migrations and schemas for missing safety nets belongs to the+data-safety work; runtime correctness of writes to the data-integrity work; a+restore endpoint's own error handling to the failure-path work. Name the+overlap when the rehearsal exposes it.++## The bar++Every finding names the step that broke (the migration file or restore+function, \`file:LINE\`), the fixture shape it ran against, the exact scratch+command, and what the next boot sees. One issue per root cause, not per+symptom; a rehearsal in which nothing breaks files nothing and says so, with+the shapes it covered.`,+'workflow-recovery': `[Improvement: {appName}] Workflow completion and recovery auditFind the multi-step workflows in {appName} that can stop halfway and leave thediff --git a/server/services/taskPromptDefaults/versions.js b/server/services/taskPromptDefaults/versions.jsindex 480cf84beb..c4baf4d7ea 100644--- a/server/services/taskPromptDefaults/versions.js+++ b/server/services/taskPromptDefaults/versions.js@@ -74,6 +74,7 @@ export const PROMPT_VERSIONS = {'privacy': 2, // v2: drops the redaction sentence the mode banners own (#12080). v1: personal/sensitive-data audit — operational exposure, third-party minimization, API over-exposure, retention, erasure reach, real data in fixtures. Honors the project's documented sharing model. Net-new type.'cost-efficiency': 1, // v1: metered-spend audit — repeated paid calls, unrequested provider work, retry multiplication, unbounded scans, storage growth, over-provisioning, egress. Net-new type.'feature-completeness': 1, // v1: incomplete-feature audit — stubs and 501s reachable from the UI, permanently disabled or "coming soon" controls, UI/doc promises the code does not keep, settings nothing reads, capabilities with no entry point, events nobody hears, plans that disagree with the code, deferred work with no live tracker item. Decides finish / hide / remove. Net-new type.+ 'upgrade-rehearsal': 1, // v1: executed upgrade-and-restore rehearsal (#12091) — opens with the scratch-only rule (never the live data/ or real Postgres, MEMORY_BACKEND=file + NODE_ENV=test, a free port from the documented band), builds synthetic older installs from migration-test fixtures, runs the seed step and migration runner, boots to the health endpoint, restores a backup into a second scratch root and verifies its manifest; one issue per root cause. Net-new type.'workflow-recovery': 1, // v1: workflow completion and recovery audit — inventories server in-progress states and checks each has a visible resume/cancel/reattach exit, busy refusals carry the holder id, client-orchestrated chains survive a failed follow-up step or reload, waits time out, and duplicate-preventing markers live on the server. Net-new type.'better-dev-environment': 2, // v2: drops the redaction clause the mode banners own (#12080). v1: host impact of the development loop, measured by diffing host state around a bounded run — leaked temp files/processes/ports, writes outside the sandbox, accumulation, watcher/idle cost, suite runtime. Net-new type.'better-test-quality': 2, // v2: adds mocks that drift from the module they replace (members/exports the real module lacks). Previous version: v1: vacuous/weak/redundant existing tests, proven with a mutation probe. Distinct from test-coverage, which owns the gaps.diff --git a/server/services/taskSchedule.test.js b/server/services/taskSchedule.test.jsindex 37880d1500..a4c6d713a5 100644--- a/server/services/taskSchedule.test.js+++ b/server/services/taskSchedule.test.js@@ -1982,6 +1982,7 @@ describe('taskSchedule', () => {['simplify', OPEN_ISSUES_ONLY],['ui-lifecycle', OPEN_ISSUES_ONLY],['module-hygiene', DEDUP_PRELOAD],+ ['upgrade-rehearsal', DEDUP_PRELOAD],// Derived, so a seventh better-* lane cannot be added without landing here....[...AUDIT_TASK_TYPES].filter((taskType) => taskType.startsWith('better-'))diff --git a/server/services/taskScheduleRegistry.js b/server/services/taskScheduleRegistry.jsindex aefc4b706f..53489abf8a 100644--- a/server/services/taskScheduleRegistry.js+++ b/server/services/taskScheduleRegistry.js@@ -383,6 +383,9 @@ const TASK_INTERVAL_DEFAULTS = {// that strand their record between steps. File-issues by default.'feature-completeness': fileIssuesAuditInterval(),'workflow-recovery': fileIssuesAuditInterval(),+ // Executed counterpart of data-safety: scratch-only migrations, restore and+ // boot. File-issues by default; a scratch run never lands code.+ 'upgrade-rehearsal': fileIssuesAuditInterval(),// Trusted remediation is separate from external intake. Legacy author// filter settings cannot widen this lane into untrusted contributor PRs.'pr-watcher': { type: INTERVAL_TYPES.ON_DEMAND, intervalMs: 1800000, enabled: true, providerId: null, model: null, prompt: null, taskMetadata: { prAuthorFilter: 'trusted', readOnly: false } },@@ -617,6 +620,7 @@ export const TASK_TYPE_DESCRIPTIONS = {'cost-efficiency': 'Cost efficiency — metered API/model calls, unbounded scans, storage tiers, over-provisioning; file issues (default) or implement fixes','feature-completeness': 'Feature completeness — stubs reachable from the UI, disabled or "coming soon" controls, settings nothing reads, endpoints with no entry point; file issues (default) or finish one','workflow-recovery': 'Workflow recovery — in-progress states with no visible resume/cancel, busy refusals without a handle, follow-up steps that strand a record; file issues (default) or implement fixes',+ 'upgrade-rehearsal': 'Upgrade & restore rehearsal — runs migrations over synthetic older installs and a backup restore in scratch directories, never the live data; file issues (default) or implement fixes','model-comparison-refresh': 'Research sourced model quality, effort, price, latency and quota evidence for Models Comparison','stash-cleanup': 'Triage git stash list — drop entries superseded by or stale relative to main, leave real unlanded work in place','repo-sync': 'Sync every managed app with origin — back on the default branch, pushed and pulled, merged branches/worktrees and redundant stashes cleared',