Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: verify Docker database credentials through the configured host endpoint (#10760)
Adam Eivyc36572fe9b22
diff --git a/docs/STORAGE.md b/docs/STORAGE.mdindex 6f733cb335..ad43b76eb3 100644--- a/docs/STORAGE.md+++ b/docs/STORAGE.md@@ -414,7 +414,7 @@ The escape hatch is **guarded from bitrot by the test suite** (tests boot with ``npm run setup:db` → `scripts/setup-db.js` is the single command that makes PostgreSQL ready, and is wired into `npm run setup` and `npm start` so a normal install never has to think about it. Its happy path:1. **Select the mode first.** For a fresh native install, set `PGMODE=native` in the repository-root `.env` before running setup. For an existing install, retain its authoritative backend; changing backends requires coordinated maintenance cutover, not a setup-time selection. An exported `PGMODE` overrides `.env` for this script; unset a conflicting shell value before retrying.-2. **`PGMODE=docker` (default):** reconciles the current `docker-compose.yml` service configuration with `docker compose up -d db` before accepting even an already-running `pgvector/pgvector:pg17` container. Compose applies changed loopback bindings and `PGPORT_DOCKER` mappings while preserving the named data volume, and reuses unchanged containers. Setup waits for TCP connections **and** the base `memories` table, then reports ready. It does not probe native PostgreSQL first. The host port defaults to `:5561` (`PGPORT_DOCKER` overrides it).+2. **`PGMODE=docker` (default):** reconciles the current `docker-compose.yml` service configuration with `docker compose up -d db` before accepting even an already-running `pgvector/pgvector:pg17` container. Compose applies changed loopback bindings and `PGPORT_DOCKER` mappings while preserving the named data volume, and reuses unchanged containers. Setup waits for TCP connections **and** the base `memories` table, then makes a bounded authenticated query for that table through the exact configured host endpoint (`PGHOST`, `PGPORT_DOCKER`, `PGUSER`, `PGDATABASE`, `PGPASSWORD`) using the server workspace's `pg` driver, and only then reports ready. A role whose persisted password differs from the configured one, an unreachable host endpoint, or a missing driver fails setup non-zero (before `npm start` replaces PM2 apps) without touching the volume, role or backend, and without logging the password. It does not probe native PostgreSQL first. The host port defaults to `:5561` (`PGPORT_DOCKER` overrides it).3. **Docker unavailable:** if Docker or Compose is missing, or the daemon is stopped, setup exits non-zero with restoration instructions, even when native PostgreSQL is healthy. Both interactive and unattended runs preserve `.env` and never offer a backend switch.4. **`PGMODE=native`:** first checks whether the configured role can authenticate to the configured database and its base `memories` table exists. A healthy database exits immediately without re-provisioning. Otherwise it runs `scripts/db.sh setup-native` (Homebrew install, role, database, extensions, schema) and verifies readiness again. The endpoint defaults to `localhost:5432` (`PGHOST`/`PGPORT` override it, from the shell or `.env`). Both the provisioning script and the readiness probes use that same selected endpoint: an unavailable selected endpoint fails without sending SQL to any other cluster on the machine.5. **Failure is non-zero exit.** A started-but-unresponsive container or a failed native bootstrap exits non-zero with an actionable message — so the `&&`-chained `npm start` halts here instead of crash-looping under PM2 against an unready database.diff --git a/scripts/setup-db.js b/scripts/setup-db.jsindex a8f6f45d49..222ddd1f77 100644--- a/scripts/setup-db.js+++ b/scripts/setup-db.js@@ -10,6 +10,7 @@*/import { execFileSync } from 'child_process';+import { createRequire } from 'module';import { dirname, join } from 'path';import { fileURLToPath } from 'url';import { parseEnvFile } from './lib/envFile.js';@@ -202,6 +203,42 @@ function waitForHealth(maxAttempts = 30) {return false;}+// Authenticated query through the exact host endpoint the server and PM2 use+// (PGHOST + PGPORT_DOCKER + PGUSER/PGDATABASE/PGPASSWORD). Container-local+// probes (pg_isready ignores credentials; the in-container psql uses the local+// socket) cannot catch a persisted role whose password no longer matches the+// configured one, or a host endpoint the application cannot reach. `pg` is+// resolved from the server workspace so no host psql is required. Returns null+// on success or a failure message (never containing the password).+async function verifyDockerHostConnection() {+ let Client;+ try {+ ({ Client } = createRequire(join(rootDir, 'server', 'package.json'))('pg'));+ } catch (err) {+ return `the pg driver is unavailable (${err.message.split('\n')[0]}) — run: npm run install:all`;+ }+ const client = new Client({+ host: PG_HOST,+ port: PG_PORT_DOCKER,+ user: PG_USER,+ database: PG_DATABASE,+ password: PG_PASSWORD,+ connectionTimeoutMillis: 5000,+ query_timeout: 5000+ });+ try {+ await client.connect();+ const { rows } = await client.query(+ "SELECT 1 AS ok FROM information_schema.tables WHERE table_schema = 'public' AND table_name = 'memories' LIMIT 1"+ );+ return rows.length === 1 ? null : 'the base schema (memories table) is missing on that endpoint';+ } catch (err) {+ return err.message;+ } finally {+ await client.end().catch(() => {});+ }+}+// Platform-specific Docker install/start hintsfunction getDockerHints(issue) {const platform = process.platform;@@ -338,6 +375,14 @@ try {// Wait for healthconsole.log('⏳ Waiting for PostgreSQL to be ready...');if (waitForHealth()) {+ const hostFailure = await verifyDockerHostConnection();+ if (hostFailure) {+ console.error(`❌ PostgreSQL container is up but ${PG_USER}@${PG_HOST}:${PG_PORT_DOCKER}/${PG_DATABASE} is not usable: ${hostFailure}`);+ console.error(' The container-local checks passed, so the configured PGPASSWORD/PGHOST/PGPORT_DOCKER');+ console.error(' likely differ from the persisted role or host publication. Setup leaves the volume and role untouched.');+ console.error(' Align .env with the existing role (see .env.example) and re-run: npm run setup:db');+ process.exit(1);+ }console.log(`✅ PostgreSQL ready on port ${PG_PORT_DOCKER}`);} else {// PG is mandatory and boot fail-fasts — a started-but-unresponsive containerdiff --git a/scripts/setup-db.test.js b/scripts/setup-db.test.jsindex 804ff7ac0f..60c950f43f 100644--- a/scripts/setup-db.test.js+++ b/scripts/setup-db.test.js@@ -66,7 +66,8 @@ describe('native setup inherited endpoint', () => {// Run the actual CLI body with synthetic configuration and subprocesses. No// imports execute, no install .env is read, and no database can be contacted.-async function runSetup({ mode = 'docker', unavailable, tty = false, nativeReady = true, running = true, hostBinding = '127.0.0.1', hostPort = 5561, dotEnv = {}, exportedMode, exportedEnv = {} } = {}) {+async function runSetup({ mode = 'docker', unavailable, tty = false, nativeReady = true, running = true, hostBinding = '127.0.0.1', hostPort = 5561, dotEnv = {}, exportedMode, exportedEnv = {}, hostAuth = 'ok', pgMissing = false } = {}) {+ const clients = [];const savedEnv = { PGMODE: mode, EXAMPLE_SETTING: 'preserved', ...dotEnv };const initialEnv = { ...savedEnv };const calls = [];@@ -86,6 +87,15 @@ async function runSetup({ mode = 'docker', unavailable, tty = false, nativeReadyawait runInNewContext(`(async () => {${source}\n})()`, {scriptUrl: new URL('./setup-db.js', import.meta.url).href, dirname, join, fileURLToPath,parseNativePort, parseDockerPort,+ createRequire: () => (name) => {+ if (name !== 'pg' || pgMissing) throw new Error("Cannot find module 'pg'");+ return { Client: class {+ constructor(config) { this.config = config; this.ended = false; clients.push(this); }+ async connect() { if (hostAuth === 'unreachable') throw new Error('connect ECONNREFUSED'); if (hostAuth === 'badpass') throw new Error('password authentication failed for user "portos"'); }+ async query() { return { rows: hostAuth === 'noschema' ? [] : [{ ok: 1 }] }; }+ async end() { this.ended = true; }+ } };+ },parseEnvFile: () => savedEnv,upsertEnvKey: (_path, key, value) => { savedEnv[key] = value; },createInterface: () => { throw new Error('Setup must not prompt to switch backends'); },@@ -122,7 +132,7 @@ async function runSetup({ mode = 'docker', unavailable, tty = false, nativeReadyif (error !== exitSignal) throw error;}expect(savedEnv).toEqual(initialEnv);- return { exitCode, calls, childEnvs, errors, logs, container, volumeRecords, recreations };+ return { clients, exitCode, calls, childEnvs, errors, logs, container, volumeRecords, recreations };}describe('setup preserves the selected database', () => {@@ -193,6 +203,34 @@ describe('setup preserves the selected database', () => {expect(result.calls.some((call) => call.includes('psql'))).toBe(true);});+ it('verifies the exact configured host endpoint and closes the connection', async () => {+ const result = await runSetup({ dotEnv: { PGHOST: 'db.example.invalid', PGPORT_DOCKER: '5599', PGUSER: 'alice', PGDATABASE: 'exampledb', PGPASSWORD: 'example-secret' } });+ expect(result.exitCode).toBe(0);+ expect(result.clients).toHaveLength(1);+ expect(result.clients[0].config).toMatchObject({ host: 'db.example.invalid', port: 5599, user: 'alice', database: 'exampledb', password: 'example-secret' });+ expect(result.clients[0].ended).toBe(true);+ });++ it.each([+ ['badpass', 'password authentication failed'],+ ['unreachable', 'ECONNREFUSED'],+ ['noschema', 'base schema'],+ ])('fails setup when container probes pass but host check is %s', async (hostAuth, message) => {+ const result = await runSetup({ hostAuth, dotEnv: { PGPASSWORD: 'example-secret' } });+ expect(result.exitCode).toBe(1);+ expect(result.errors.join('\n')).toContain(message);+ expect(result.errors.join('\n')).not.toContain('example-secret');+ expect(result.logs.join('\n')).not.toContain('PostgreSQL ready');+ expect(result.clients[0].ended).toBe(true);+ expect(result.container.volumeRecords).toEqual(['example persisted record']);+ });++ it('fails with an install hint when the pg driver cannot be resolved', async () => {+ const result = await runSetup({ pgMissing: true });+ expect(result.exitCode).toBe(1);+ expect(result.errors.join('\n')).toContain('npm run install:all');+ });+it.each([{ hostBinding: '0.0.0.0', hostPort: 5561, dotEnv: {} },{ hostBinding: '127.0.0.1', hostPort: 5561, dotEnv: { PGPORT_DOCKER: '5599' } },