Garry's Opinionated OpenClaw/Hermes Agent Brain
TypeScript
fix(engine-open): let in-flight work finish on a superseded direct pool, keep a named port 80, name the multi-host refusal
garrytanb61a79c7b18f
diff --git a/CHANGELOG.md b/CHANGELOG.mdindex 38f84fa273..f8afac7aa2 100644--- a/CHANGELOG.md+++ b/CHANGELOG.md@@ -19,8 +19,8 @@ identifiers and attribution are available in the pre-removal Git revision- `src/core/engine-open.ts` (new, exported as `./engine-open`): `openEngine`, refusing `embedded_core_required`, `engine_host_missing` and `engine_settings_invalid`. `disconnect()` closes the engine for good, also during an in-flight reconnect, and a later `connect()` or `reconnect()` refuses `engine_closed`. A connect never migrates; the host runs `initSchema()`. Documented in docs/ENGINES.md, "Opening an engine from settings".- `src/core/persistence/identity.ts`: `hostIdFor(engine)` and `setEngineHostId`, resolved through `engineKey`. Every host-id read in `src/core` goes through it or `existingHostIdFor(engine)` (publication, admission, the consumer, worktree ownership and refresh, topology, effects, repair and the writer stamp), except `gbrain migrate`'s engine graduation; an unmarked engine still gets `localHostId()`, so the CLI's behavior is unchanged. An embedded engine never reads or mints `host.json`, and its consumer heartbeat records `embedded:<host-id>`, which doctor and writer status read as an identity given at open. `registerLocalWriterValue` registers a writer the host holds as a value, and an embedded engine's writes use it instead of a `<brain>.<lane>.json` file. It never widens past the settings. The lane is the one given, a new registration gets the default read/write ceiling, and an existing registration keeps its own.- `initSchema()` on an embedded engine seeds the embedding model and width from `settings.embedding` and refuses `engine_embedding_mismatch` on a database that records another one. Postgres `initSchema()` reads the recorded identity again once it holds the schema lock, so two hosts initializing one fresh database can no longer both pass the check.-- A settings URL must name its host, user and database, and an embedded engine's pools pass every connection field to the driver explicitly, so `PGHOST`, `PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never choose its server or credentials.-- `PostgresEngine.reconnect()` now ends the superseded connection manager's direct pool (#8478). Before, every reconnect on an engine with a direct route left that pool open past `disconnect()`; this affected every Postgres engine, not only embedded ones.+- A settings URL must be one single-host URL that names its host, user and database, and an embedded engine's pools pass every connection field to the driver explicitly, so `PGHOST`, `PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never choose its server or credentials.+- `PostgresEngine.reconnect()` now ends the superseded connection manager's direct pool once its in-flight work finishes (#8478), so a long hold such as `CREATE INDEX CONCURRENTLY` or the schema lock completes first. Before, every reconnect on an engine with a direct route left that pool open past `disconnect()`; this affected every Postgres engine, not only embedded ones.- The decide receipt and spend buffers write rows buffered for one brain to that brain when the next row names another, even while a flush is in flight. Before, those rows landed in the next brain's database.- The decide spend cache is keyed by engine and held weakly, and a drained feedback queue no longer keeps its engine alive.- File storage and AI provider settings are not engine settings yet; they stay process-wide until Wave C.diff --git a/docs/ENGINES.md b/docs/ENGINES.mdindex 8026ffe790..822cfe7b69 100644--- a/docs/ENGINES.md+++ b/docs/ENGINES.md@@ -751,7 +751,8 @@ working directory:- **Connection.** Postgres always connects on the instance-pool path. The direct (DDL/session) pool uses only`database.directUrl`, or the route a known pooler URL derives; `GBRAIN_DIRECT_DATABASE_URL` is never consulted.- Each URL must name its host, user and database (`engine_settings_invalid` otherwise), and both pools pass the+ Each URL must be one single-host URL that names its host, user and database (`engine_settings_invalid` otherwise;+ multi-host failover URLs are not read), and both pools pass thehost, port (5432 when the URL names none), user, password and database to the driver explicitly, so `PGHOST`,`PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never pick the server or the credentials. A URL with no passwordconnects with none. The statement-description cache stays in the process instead of `<home>/cache/pg-types`diff --git a/llms-full.txt b/llms-full.txtindex b5e88816f1..a9f0418208 100644--- a/llms-full.txt+++ b/llms-full.txt@@ -6167,7 +6167,8 @@ working directory:- **Connection.** Postgres always connects on the instance-pool path. The direct (DDL/session) pool uses only`database.directUrl`, or the route a known pooler URL derives; `GBRAIN_DIRECT_DATABASE_URL` is never consulted.- Each URL must name its host, user and database (`engine_settings_invalid` otherwise), and both pools pass the+ Each URL must be one single-host URL that names its host, user and database (`engine_settings_invalid` otherwise;+ multi-host failover URLs are not read), and both pools pass thehost, port (5432 when the URL names none), user, password and database to the driver explicitly, so `PGHOST`,`PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never pick the server or the credentials. A URL with no passwordconnects with none. The statement-description cache stays in the process instead of `<home>/cache/pg-types`diff --git a/src/core/connection-manager.ts b/src/core/connection-manager.tsindex 966ffb8b77..efcb865e6f 100644--- a/src/core/connection-manager.ts+++ b/src/core/connection-manager.ts@@ -506,6 +506,15 @@ export class ConnectionManager {await Promise.all(ends);}+ /** Ends a manager a reconnect superseded once its in-flight work finishes, with no hard bound (a long hold such as CREATE INDEX CONCURRENTLY completes). */+ retire(): void {+ const pools = [this._directPool, this._readPoolOwnedExternally ? null : this._readPool];+ this._directPool = null;+ this._directInit = null;+ this._readPool = null;+ for (const pool of pools) pool?.end().catch(() => {});+ }+/*** Diagnostic snapshot for doctor / get_health surfaces.*/diff --git a/src/core/embedded-engine.ts b/src/core/embedded-engine.tsindex 70537cd918..5db14f2520 100644--- a/src/core/embedded-engine.ts+++ b/src/core/embedded-engine.ts@@ -35,16 +35,14 @@ interface EmbeddedRecord {settings: EngineSettings | null;}-const parseConnectionUrl = (url: string) => new URL(url.replace(/^postgres(ql)?:\/\//, 'http://'));+const parseConnectionUrl = (url: string) => new URL(url.replace(/^postgres(ql)?:\/\//, 'settings://'));-/** The connection field a settings URL leaves out (host, user or database name), or undefined when it names all three. */-export function missingConnectionField(url: string): string | undefined {+/** Why openEngine refuses a settings URL (not one parseable single-host URL, or no host, user or database name), or undefined. */+export function connectionUrlProblem(url: string): string | undefined {let parsed: URL;- try { parsed = parseConnectionUrl(url); } catch { return 'a parseable URL'; }- if (!parsed.hostname) return 'a host';- if (!parsed.username) return 'a user';- if (parsed.pathname.length <= 1) return 'a database name';- return undefined;+ try { parsed = parseConnectionUrl(url); } catch { return 'is not a parseable single-host connection URL (openEngine reads no multi-host URL)'; }+ const missing = !parsed.hostname ? 'host' : !parsed.username ? 'user' : parsed.pathname.length <= 1 ? 'database name' : undefined;+ return missing && `names no ${missing}, and the driver would fill it from PG* environment variables`;}/**diff --git a/src/core/engine-open.ts b/src/core/engine-open.tsindex 3d5ea4cf57..9833e52291 100644--- a/src/core/engine-open.ts+++ b/src/core/engine-open.ts@@ -7,7 +7,7 @@ import type { BrainEngine } from './engine.ts';import type { EngineConfig } from './types.ts';import { opError } from './ops/contract.ts';import { createEngine } from './engine-factory.ts';-import { markEmbeddedEngine, missingConnectionField, releaseEmbeddedEngine, type EngineSettings } from './embedded-engine.ts';+import { connectionUrlProblem, markEmbeddedEngine, releaseEmbeddedEngine, type EngineSettings } from './embedded-engine.ts';import { assertLocalWriterValue, setEngineHostId } from './persistence/identity.ts';export type { EngineSettings, EmbeddedSettingsView } from './embedded-engine.ts';@@ -32,8 +32,8 @@ function checkSettings(settings: EngineSettings): void {if (database.path !== undefined) throw invalid('database.path', 'A Postgres engine takes database.url, not database.path.');if (database.directUrl !== undefined && !text(database.directUrl)) throw invalid('database.directUrl', 'database.directUrl is a connection URL when given.');for (const [setting, url] of [['database.url', database.url], ['database.directUrl', database.directUrl]] as const) {- const missing = url === undefined ? undefined : missingConnectionField(url);- if (missing) throw invalid(setting, `${setting} names no ${missing}, and the driver would fill it from PG* environment variables.`);+ const problem = url === undefined ? undefined : connectionUrlProblem(url);+ if (problem) throw invalid(setting, `${setting} ${problem}.`);}} else {if (database.url !== undefined || database.directUrl !== undefined) throw invalid('database.url', 'A PGLite engine takes database.path (or none for an in-memory brain), not a URL.');diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.tsindex 75964abeaa..ec07148913 100644--- a/src/core/postgres-engine.ts+++ b/src/core/postgres-engine.ts@@ -2879,7 +2879,7 @@ export class PostgresEngine implements BrainEngine {await this.connect(this._savedConfig);// New pool is live — discard the old one best-effort, and the superseded manager's direct pool with it.if (oldSql) { try { await oldSql.end({ timeout: 5 }); } catch { /* swallow */ } }- if (oldManager && oldManager !== this.connectionManager) await oldManager.disconnect().catch(() => {});+ if (oldManager && oldManager !== this.connectionManager) oldManager.retire();try {logPoolRecovery('reconnect_succeeded');} catch { /* best-effort */ }diff --git a/test/e2e/engine-open-postgres.test.ts b/test/e2e/engine-open-postgres.test.tsindex 707e0741ca..99bf53df1b 100644--- a/test/e2e/engine-open-postgres.test.ts+++ b/test/e2e/engine-open-postgres.test.ts@@ -159,6 +159,20 @@ describeE2E('openEngine on Postgres', () => {expect(await codeOf(oldDirect`SELECT 1`)).not.toBe('resolved');}, 60_000);+ test('a reconnect lets in-flight work on the superseded direct pool finish before that pool ends', async () => {+ const engine = await openEngine({ embedded_core: true, engine: 'postgres', database: { url: urls[0]!, directUrl: urls[0]! }, pool: { max: 2 }, host: { id: randomUUID() } });+ try {+ const manager = internals(engine).connectionManager as unknown as { ddl(): Promise<postgres.Sql> };+ const oldDirect = await manager.ddl();+ const inFlight = codeOf(oldDirect`SELECT pg_sleep(3)`);+ await new Promise(resolve => setTimeout(resolve, 200));+ await engine.reconnect();+ expect(await inFlight).toBe('resolved');+ } finally {+ await engine.disconnect();+ }+ }, 60_000);+test('the consumer heartbeat records the identity-given-at-open sentinel, not a host.json path', async () => {const engine = await openEngine(settingsA as EngineSettings);try {diff --git a/test/engine-open.test.ts b/test/engine-open.test.tsindex 0017d1f775..b0e3488fb1 100644--- a/test/engine-open.test.ts+++ b/test/engine-open.test.ts@@ -9,7 +9,7 @@ import { tmpdir } from 'node:os';import { join, relative } from 'node:path';import { randomBytes, randomUUID } from 'node:crypto';import { embeddedSettings, hostIdFor, isEmbeddedEngine, openEngine, processHasEmbeddedEngines, type EngineSettings } from '../src/core/engine-open.ts';-import { markEmbeddedEngine } from '../src/core/embedded-engine.ts';+import { markEmbeddedEngine, settingsConnectionOptions } from '../src/core/embedded-engine.ts';import { existingHostIdFor, existingLocalHostId, hostIdentityPathsFor, localHostId, localHostIdentity, registerLocalWriter, registerLocalWriterValue, revokeLocalWriter } from '../src/core/persistence/identity.ts';import { operationsByName, type OperationContext } from '../src/core/operations.ts';import { readStoredEmbeddingIdentity } from '../src/core/stored-embedding-identity.ts';@@ -86,6 +86,12 @@ describe('openEngine refusals', () => {});expect(filesUnder(gbrainHome)).toEqual([]);});++ test('a settings URL keeps the port it names, and a multi-host URL is refused as one openEngine cannot read', async () => {+ expect(settingsConnectionOptions(pgUrl('app:secret', 'db-a.example.internal:80/brain_a')).port).toBe(80);+ const refusal = await openEngine(settingsFor({ engine: 'postgres', database: { url: pgUrl('app:secret', 'db-a.example.internal:5432,db-b.example.internal:5433/brain_a') } })).catch((error: { code?: string; suggestion?: string }) => error);+ expect(refusal).toMatchObject({ code: 'engine_settings_invalid', suggestion: expect.stringContaining('multi-host') });+ });});describe('openEngine on a clean home', () => {