Skip to content

Garry's Opinionated OpenClaw/Hermes Agent Brain

TypeScript

fix(engine-open): let in-flight work finish on a superseded direct pool, keep a named port 80, name the multi-host refusal

garrytanb61a79c7b18f
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 38f84fa273..f8afac7aa2 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -19,8 +19,8 @@ identifiers and attribution are available in the pre-removal Git revision
- `src/core/engine-open.ts` (new, exported as `./engine-open`): `openEngine`, refusing `embedded_core_required`, `engine_host_missing` and `engine_settings_invalid`. `disconnect()` closes the engine for good, also during an in-flight reconnect, and a later `connect()` or `reconnect()` refuses `engine_closed`. A connect never migrates; the host runs `initSchema()`. Documented in docs/ENGINES.md, "Opening an engine from settings".
- `src/core/persistence/identity.ts`: `hostIdFor(engine)` and `setEngineHostId`, resolved through `engineKey`. Every host-id read in `src/core` goes through it or `existingHostIdFor(engine)` (publication, admission, the consumer, worktree ownership and refresh, topology, effects, repair and the writer stamp), except `gbrain migrate`'s engine graduation; an unmarked engine still gets `localHostId()`, so the CLI's behavior is unchanged. An embedded engine never reads or mints `host.json`, and its consumer heartbeat records `embedded:<host-id>`, which doctor and writer status read as an identity given at open. `registerLocalWriterValue` registers a writer the host holds as a value, and an embedded engine's writes use it instead of a `<brain>.<lane>.json` file. It never widens past the settings. The lane is the one given, a new registration gets the default read/write ceiling, and an existing registration keeps its own.
- `initSchema()` on an embedded engine seeds the embedding model and width from `settings.embedding` and refuses `engine_embedding_mismatch` on a database that records another one. Postgres `initSchema()` reads the recorded identity again once it holds the schema lock, so two hosts initializing one fresh database can no longer both pass the check.
-- A settings URL must name its host, user and database, and an embedded engine's pools pass every connection field to the driver explicitly, so `PGHOST`, `PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never choose its server or credentials.
-- `PostgresEngine.reconnect()` now ends the superseded connection manager's direct pool (#8478). Before, every reconnect on an engine with a direct route left that pool open past `disconnect()`; this affected every Postgres engine, not only embedded ones.
+- A settings URL must be one single-host URL that names its host, user and database, and an embedded engine's pools pass every connection field to the driver explicitly, so `PGHOST`, `PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never choose its server or credentials.
+- `PostgresEngine.reconnect()` now ends the superseded connection manager's direct pool once its in-flight work finishes (#8478), so a long hold such as `CREATE INDEX CONCURRENTLY` or the schema lock completes first. Before, every reconnect on an engine with a direct route left that pool open past `disconnect()`; this affected every Postgres engine, not only embedded ones.
- The decide receipt and spend buffers write rows buffered for one brain to that brain when the next row names another, even while a flush is in flight. Before, those rows landed in the next brain's database.
- The decide spend cache is keyed by engine and held weakly, and a drained feedback queue no longer keeps its engine alive.
- File storage and AI provider settings are not engine settings yet; they stay process-wide until Wave C.
diff --git a/docs/ENGINES.md b/docs/ENGINES.md
index 8026ffe790..822cfe7b69 100644
--- a/docs/ENGINES.md
+++ b/docs/ENGINES.md
@@ -751,7 +751,8 @@ working directory:
- **Connection.** Postgres always connects on the instance-pool path. The direct (DDL/session) pool uses only
`database.directUrl`, or the route a known pooler URL derives; `GBRAIN_DIRECT_DATABASE_URL` is never consulted.
- Each URL must name its host, user and database (`engine_settings_invalid` otherwise), and both pools pass the
+ Each URL must be one single-host URL that names its host, user and database (`engine_settings_invalid` otherwise;
+ multi-host failover URLs are not read), and both pools pass the
host, port (5432 when the URL names none), user, password and database to the driver explicitly, so `PGHOST`,
`PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never pick the server or the credentials. A URL with no password
connects with none. The statement-description cache stays in the process instead of `<home>/cache/pg-types`
diff --git a/llms-full.txt b/llms-full.txt
index b5e88816f1..a9f0418208 100644
--- a/llms-full.txt
+++ b/llms-full.txt
@@ -6167,7 +6167,8 @@ working directory:
- **Connection.** Postgres always connects on the instance-pool path. The direct (DDL/session) pool uses only
`database.directUrl`, or the route a known pooler URL derives; `GBRAIN_DIRECT_DATABASE_URL` is never consulted.
- Each URL must name its host, user and database (`engine_settings_invalid` otherwise), and both pools pass the
+ Each URL must be one single-host URL that names its host, user and database (`engine_settings_invalid` otherwise;
+ multi-host failover URLs are not read), and both pools pass the
host, port (5432 when the URL names none), user, password and database to the driver explicitly, so `PGHOST`,
`PGPORT`, `PGUSER`, `PGPASSWORD` and `PGDATABASE` never pick the server or the credentials. A URL with no password
connects with none. The statement-description cache stays in the process instead of `<home>/cache/pg-types`
diff --git a/src/core/connection-manager.ts b/src/core/connection-manager.ts
index 966ffb8b77..efcb865e6f 100644
--- a/src/core/connection-manager.ts
+++ b/src/core/connection-manager.ts
@@ -506,6 +506,15 @@ export class ConnectionManager {
await Promise.all(ends);
}
+ /** Ends a manager a reconnect superseded once its in-flight work finishes, with no hard bound (a long hold such as CREATE INDEX CONCURRENTLY completes). */
+ retire(): void {
+ const pools = [this._directPool, this._readPoolOwnedExternally ? null : this._readPool];
+ this._directPool = null;
+ this._directInit = null;
+ this._readPool = null;
+ for (const pool of pools) pool?.end().catch(() => {});
+ }
+
/**
* Diagnostic snapshot for doctor / get_health surfaces.
*/
diff --git a/src/core/embedded-engine.ts b/src/core/embedded-engine.ts
index 70537cd918..5db14f2520 100644
--- a/src/core/embedded-engine.ts
+++ b/src/core/embedded-engine.ts
@@ -35,16 +35,14 @@ interface EmbeddedRecord {
settings: EngineSettings | null;
}
-const parseConnectionUrl = (url: string) => new URL(url.replace(/^postgres(ql)?:\/\//, 'http://'));
+const parseConnectionUrl = (url: string) => new URL(url.replace(/^postgres(ql)?:\/\//, 'settings://'));
-/** The connection field a settings URL leaves out (host, user or database name), or undefined when it names all three. */
-export function missingConnectionField(url: string): string | undefined {
+/** Why openEngine refuses a settings URL (not one parseable single-host URL, or no host, user or database name), or undefined. */
+export function connectionUrlProblem(url: string): string | undefined {
let parsed: URL;
- try { parsed = parseConnectionUrl(url); } catch { return 'a parseable URL'; }
- if (!parsed.hostname) return 'a host';
- if (!parsed.username) return 'a user';
- if (parsed.pathname.length <= 1) return 'a database name';
- return undefined;
+ try { parsed = parseConnectionUrl(url); } catch { return 'is not a parseable single-host connection URL (openEngine reads no multi-host URL)'; }
+ const missing = !parsed.hostname ? 'host' : !parsed.username ? 'user' : parsed.pathname.length <= 1 ? 'database name' : undefined;
+ return missing && `names no ${missing}, and the driver would fill it from PG* environment variables`;
}
/**
diff --git a/src/core/engine-open.ts b/src/core/engine-open.ts
index 3d5ea4cf57..9833e52291 100644
--- a/src/core/engine-open.ts
+++ b/src/core/engine-open.ts
@@ -7,7 +7,7 @@ import type { BrainEngine } from './engine.ts';
import type { EngineConfig } from './types.ts';
import { opError } from './ops/contract.ts';
import { createEngine } from './engine-factory.ts';
-import { markEmbeddedEngine, missingConnectionField, releaseEmbeddedEngine, type EngineSettings } from './embedded-engine.ts';
+import { connectionUrlProblem, markEmbeddedEngine, releaseEmbeddedEngine, type EngineSettings } from './embedded-engine.ts';
import { assertLocalWriterValue, setEngineHostId } from './persistence/identity.ts';
export type { EngineSettings, EmbeddedSettingsView } from './embedded-engine.ts';
@@ -32,8 +32,8 @@ function checkSettings(settings: EngineSettings): void {
if (database.path !== undefined) throw invalid('database.path', 'A Postgres engine takes database.url, not database.path.');
if (database.directUrl !== undefined && !text(database.directUrl)) throw invalid('database.directUrl', 'database.directUrl is a connection URL when given.');
for (const [setting, url] of [['database.url', database.url], ['database.directUrl', database.directUrl]] as const) {
- const missing = url === undefined ? undefined : missingConnectionField(url);
- if (missing) throw invalid(setting, `${setting} names no ${missing}, and the driver would fill it from PG* environment variables.`);
+ const problem = url === undefined ? undefined : connectionUrlProblem(url);
+ if (problem) throw invalid(setting, `${setting} ${problem}.`);
}
} else {
if (database.url !== undefined || database.directUrl !== undefined) throw invalid('database.url', 'A PGLite engine takes database.path (or none for an in-memory brain), not a URL.');
diff --git a/src/core/postgres-engine.ts b/src/core/postgres-engine.ts
index 75964abeaa..ec07148913 100644
--- a/src/core/postgres-engine.ts
+++ b/src/core/postgres-engine.ts
@@ -2879,7 +2879,7 @@ export class PostgresEngine implements BrainEngine {
await this.connect(this._savedConfig);
// New pool is live — discard the old one best-effort, and the superseded manager's direct pool with it.
if (oldSql) { try { await oldSql.end({ timeout: 5 }); } catch { /* swallow */ } }
- if (oldManager && oldManager !== this.connectionManager) await oldManager.disconnect().catch(() => {});
+ if (oldManager && oldManager !== this.connectionManager) oldManager.retire();
try {
logPoolRecovery('reconnect_succeeded');
} catch { /* best-effort */ }
diff --git a/test/e2e/engine-open-postgres.test.ts b/test/e2e/engine-open-postgres.test.ts
index 707e0741ca..99bf53df1b 100644
--- a/test/e2e/engine-open-postgres.test.ts
+++ b/test/e2e/engine-open-postgres.test.ts
@@ -159,6 +159,20 @@ describeE2E('openEngine on Postgres', () => {
expect(await codeOf(oldDirect`SELECT 1`)).not.toBe('resolved');
}, 60_000);
+ test('a reconnect lets in-flight work on the superseded direct pool finish before that pool ends', async () => {
+ const engine = await openEngine({ embedded_core: true, engine: 'postgres', database: { url: urls[0]!, directUrl: urls[0]! }, pool: { max: 2 }, host: { id: randomUUID() } });
+ try {
+ const manager = internals(engine).connectionManager as unknown as { ddl(): Promise<postgres.Sql> };
+ const oldDirect = await manager.ddl();
+ const inFlight = codeOf(oldDirect`SELECT pg_sleep(3)`);
+ await new Promise(resolve => setTimeout(resolve, 200));
+ await engine.reconnect();
+ expect(await inFlight).toBe('resolved');
+ } finally {
+ await engine.disconnect();
+ }
+ }, 60_000);
+
test('the consumer heartbeat records the identity-given-at-open sentinel, not a host.json path', async () => {
const engine = await openEngine(settingsA as EngineSettings);
try {
diff --git a/test/engine-open.test.ts b/test/engine-open.test.ts
index 0017d1f775..b0e3488fb1 100644
--- a/test/engine-open.test.ts
+++ b/test/engine-open.test.ts
@@ -9,7 +9,7 @@ import { tmpdir } from 'node:os';
import { join, relative } from 'node:path';
import { randomBytes, randomUUID } from 'node:crypto';
import { embeddedSettings, hostIdFor, isEmbeddedEngine, openEngine, processHasEmbeddedEngines, type EngineSettings } from '../src/core/engine-open.ts';
-import { markEmbeddedEngine } from '../src/core/embedded-engine.ts';
+import { markEmbeddedEngine, settingsConnectionOptions } from '../src/core/embedded-engine.ts';
import { existingHostIdFor, existingLocalHostId, hostIdentityPathsFor, localHostId, localHostIdentity, registerLocalWriter, registerLocalWriterValue, revokeLocalWriter } from '../src/core/persistence/identity.ts';
import { operationsByName, type OperationContext } from '../src/core/operations.ts';
import { readStoredEmbeddingIdentity } from '../src/core/stored-embedding-identity.ts';
@@ -86,6 +86,12 @@ describe('openEngine refusals', () => {
});
expect(filesUnder(gbrainHome)).toEqual([]);
});
+
+ test('a settings URL keeps the port it names, and a multi-host URL is refused as one openEngine cannot read', async () => {
+ expect(settingsConnectionOptions(pgUrl('app:secret', 'db-a.example.internal:80/brain_a')).port).toBe(80);
+ const refusal = await openEngine(settingsFor({ engine: 'postgres', database: { url: pgUrl('app:secret', 'db-a.example.internal:5432,db-b.example.internal:5433/brain_a') } })).catch((error: { code?: string; suggestion?: string }) => error);
+ expect(refusal).toMatchObject({ code: 'engine_settings_invalid', suggestion: expect.stringContaining('multi-host') });
+ });
});
describe('openEngine on a clean home', () => {