Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

test(home): prove the bridge pool reuses one real socket and fails fast on a dead house

nirholasd6cf18c54165
diff --git a/prompts/finish/home-02-bridge-runtime.md b/prompts/finish/home-02-bridge-runtime.md
deleted file mode 100644
index 22fbd2c64..000000000
--- a/prompts/finish/home-02-bridge-runtime.md
+++ /dev/null
@@ -1,134 +0,0 @@
-# 02. Bridge runtime: the multi-tenant connection manager
-
-**How to run this:** paste this whole file into a fresh Claude Code chat opened in
-`/workspaces/three.ws`. Read [00-CONTEXT.md](home-00-CONTEXT.md) first. Order
-[01](home-01-connection-store.md) must have landed: this order reads its store.
-
-## Binding operating clause
-
-Finish 100%. Never end with a question or an unexecuted plan. All CLAUDE.md hard rules apply: no
-mocks, no fake data, no TODOs, no stubs, no em-dash or en-dash, explicit-path commits, pushes and
-production deploys owner-gated. Every architectural decision here is already made; implement it
-and note disagreement in your report rather than stopping.
-
-**Nothing in this file is a status claim to trust.** Step 0 re-measures.
-
----
-
-## Step 0: re-derive the current state
-
-```bash
-ls api/_lib/home/ # expect store.js and verify.js from order 01
-npx vitest run packages/home-bridge tests/home-store.test.js
-grep -rn "minScale\|min-instances" docs/ops/gcp-production.md | head
-gcloud run services describe three-ws-api --region us-central1 --format='value(spec.template.metadata.annotations)' 2>/dev/null | tr ',' '\n' | grep -i scale
-```
-
-That last command is the one that decides this order's design. Read it before you write code.
-
-## The problem this order exists to solve
-
-`packages/home-bridge` opens one WebSocket per `HomeBridge` and holds it. Cloud Run runs many
-instances, scales to zero, and recycles containers. Naively constructing a `HomeBridge` per
-request would open and tear down a socket per call, hammer the user's house, and lose the live
-state stream that the 3D scene depends on. Naively holding one forever per user would leak
-sockets across instance recycles and pin a user to an instance that may not receive their next
-request.
-
-**The design: a per-instance, lazily-opened, reference-counted, idle-evicted connection pool,
-with the socket treated as a cache and never as the source of truth.**
-
-Every consumer (an SSE stream, a chat tool call, an MCP call) checks out a bridge, uses it, and
-checks it back in. The pool opens a real connection on the first checkout and closes it after a
-bounded idle period with no subscribers. A request that arrives at an instance holding no
-connection opens one; that is a cold start of a few hundred milliseconds, not an error.
-
-## The contract
-
-New file `api/_lib/home/runtime.js`.
-
-| Export | Contract |
-|---|---|
-| `acquire(homeId, userId)` | returns `{ bridge, release }`. Opens the connection if this instance holds none, reusing it otherwise. Increments the refcount. `release()` is idempotent and must be called in a `finally`. |
-| `withHome(homeId, userId, fn)` | the shape every caller should use: acquires, runs `fn(bridge)`, releases in a `finally`, and never leaks a socket on a throw. |
-| `snapshot(homeId, userId)` | the current room graph without holding the connection open past the call. Serves a page load. |
-| `subscribe(homeId, userId, onGraph)` | a live subscription for SSE. Returns an unsubscribe. Holds a reference for its lifetime. |
-| `evictIdle(now)` | closes connections with zero subscribers past the idle window. Called on a timer and exported so a test can drive it deterministically. |
-| `stats()` | `{ open, subscribers, byStatus }` for the health probe in order 13. |
-
-### The numbers, and why
-
-| Knob | Value | Reason |
-|---|---|---|
-| Idle eviction | 90 s after the last subscriber releases | Long enough that a page navigation or a chat turn reuses the socket; short enough that an abandoned tab does not hold a stranger's house open. |
-| Max open connections per instance | 200, configurable by `HOME_MAX_CONNECTIONS` | A socket plus a state map is roughly 1 to 3 MB of heap for a large house. Past the cap, `acquire` serves from a fresh short-lived connection and does not admit to the pool, so behaviour degrades in latency, never in correctness. |
-| Connect timeout | 15 s | A house behind a slow tunnel is common; a hang is not. |
-| Reconnect backoff | delegated to `home-assistant-js-websocket`, which reconnects and resubscribes on its own | Do not write a second reconnect loop on top of it. |
-| Circuit breaker | 5 consecutive connect failures opens it for 5 minutes for that home id | Stops a revoked token or an offline house from being retried on every page load. `status` in the store is updated so the UI can explain it. |
-| Graph rebuild coalescing | already 80 ms inside `HomeBridge` | Do not add a second debounce. |
-
-### Failure behaviour, per hop
-
-| Hop | Slow | Down | Garbage | Nothing |
-|---|---|---|---|---|
-| Store read (order 01) | normal DB latency | `acquire` throws a coded error, the caller returns 503 | a revoked row is not acquirable at all | unknown home id to this user is a 404, never a 403 that leaks existence |
-| Token decrypt | n/a | a decrypt failure marks the connection `auth_failed` and is reported to the user as "reconnect your home", never as a server error | n/a | a scrubbed (revoked) ciphertext returns the revoked path |
-| HA handshake | 15 s cap, then `unreachable` with the elapsed time | breaker opens after 5, store status updated, UI explains | an HTTP endpoint that is not Home Assistant returns `unreachable` with the detail | empty entity set is a valid house, not an error |
-| Live socket after connect | the library reconnects itself; subscribers see stale-then-fresh, never an error toast | subscribers are told `disconnected` and the graph is marked stale rather than emptied | a malformed message is dropped and logged once, not per message | no state within 5 s of connect resolves with an empty graph, and the UI shows its empty state |
-| Instance recycle | n/a | every connection dies at once; the next request reopens. This is normal and must never page anyone. | n/a | n/a |
-
-**The graph must never be emptied on disconnect.** A user watching their 3D home should see it
-go grey and stale, not watch their house vanish. That distinction is the whole difference between
-a product and a demo.
-
-## Tasks, in risk order
-
-| # | Task | Files |
-|---|---|---|
-| 1 | The pool itself: `acquire`, `release`, refcounting, the cap, idle eviction, `stats`. Pure enough to test without a network by injecting the bridge factory. | `api/_lib/home/runtime.js` |
-| 2 | The circuit breaker, keyed by home id, with the store status write on open. | same |
-| 3 | `subscribe` and `snapshot` on top of the pool. | same |
-| 4 | The eviction timer, started once per process, cleared on shutdown, and safe under Cloud Run's SIGTERM. | same |
-| 5 | Tests: refcount correctness, no socket leak on a throwing `fn`, eviction at the boundary, breaker open and close, cap behaviour, stale-not-empty on disconnect. | `tests/home-runtime.test.js` |
-| 6 | A live test against a real HA proving reuse: two `withHome` calls in a row open exactly one socket. | `tests/home-runtime-live.test.js`, self-skipping on `HOME_ASSISTANT_URL` |
-
-## Definition of done
-
-- [ ] Two sequential `withHome` calls against a real instance open one socket, proved by a counter on the injected factory and by the HA side (`gauge` the connection count via the instance's own logs or `docker logs`).
-- [ ] A `withHome` whose callback throws still releases: `stats().subscribers` returns to 0.
-- [ ] After the idle window with no subscribers, `evictIdle` closes the socket and `stats().open` drops.
-- [ ] Five consecutive failed connects open the breaker; the sixth `acquire` fails fast (under 50 ms) rather than waiting out a 15 s timeout. Assert the elapsed time.
-- [ ] The store's `status` and `status_detail` reflect the breaker state, so order 05's UI can render a real reason.
-- [ ] Killing the HA container mid-subscription leaves the last graph readable and marked stale; restarting it restores live updates with no client action.
-- [ ] `stats()` is exported and shaped for order 13.
-- [ ] `npx vitest run tests/home-runtime.test.js packages/home-bridge` passes.
-- [ ] `npm run check:rules -- --paths <your files>` is clean.
-
-## Never blocked
-
-| Blocker | Do this |
-|---|---|
-| Cloud Run scales to zero and kills connections | That is expected and designed for. The pool is a cache, not state. Never add a `minScale` to keep sockets warm without reading `docs/ops/gcp-credits-plan.md` first, and never as a fix for a design that assumed persistence. |
-| You want a global connection registry across instances | You do not, and it would need a second datastore. The house is the source of truth; a cold instance reopens in a few hundred milliseconds. Revisit only with a measured latency complaint. |
-| A house has thousands of entities and the graph rebuild is slow | Measure it before optimizing. `buildHomeGraph` is a pure function over arrays; if it is genuinely hot, memoize per registry version, not per state change. |
-| The reconnect logic seems to need improving | It lives in `home-assistant-js-websocket`, is first-party, and reconnects and resubscribes on its own. Fixing it means a PR upstream, per the CLAUDE.md open-source rule, not a wrapper here. |
-| SIGTERM handling looks like someone else's problem | It is this order's. A container that dies without closing sockets leaves the user's Home Assistant holding dead connections until its own timeout. Close them. |
-
-## Report format
-
-1. The socket-reuse proof, with the counter output and the HA-side observation.
-2. The breaker timing measurement.
-3. The kill-and-restart transcript showing stale-not-empty.
-4. `stats()` output under load.
-5. Test output, verbatim.
-6. Anything in this file you found to be wrong, and what you changed it to.
-
-## Retire this prompt when it is done (required)
-
-Verify every Definition of done line against real output, append to
-`prompts/finish/home-PROGRESS.md`, then commit with explicit paths and delete this file in the
-same commit:
-
- git rm prompts/finish/home-02-bridge-runtime.md
-
-Never delete it on a partial.
diff --git a/prompts/finish/home-PROGRESS.md b/prompts/finish/home-PROGRESS.md
index e4407083b..55a18c7f7 100644
--- a/prompts/finish/home-PROGRESS.md
+++ b/prompts/finish/home-PROGRESS.md
@@ -31,7 +31,7 @@ One section per finished order, newest at the bottom:
|---|---|---|
| 00 CONTEXT | shared facts | n/a |
| 01 connection store | done | 2026-09-03 |
-| 02 bridge runtime | open | |
+| 02 bridge runtime | done | 2026-09-03 |
| 03 API surface | open | |
| 04 agent tools | open | |
| 05 connect flow | open | |
@@ -244,3 +244,62 @@ test runs, sealed under their local keys, which the audit reports honestly.
**Commits:** the schema, store, verify, tests and audit script were swept into concurrent agents'
`git add -A` commits as they landed (`c3132956a`, `858a2f86b` among them); this entry and the
retirement of the order file are the final commit.
+
+---
+
+## 02. Bridge runtime: the multi-tenant connection manager (2026-09-03)
+
+**Shipped:** `api/_lib/home/runtime.js` holds one per-instance, lazily-opened, reference-counted,
+idle-evicted pool of live Home Assistant sockets, and every consumer above it (SSE, a chat tool
+call, an MCP call) checks a bridge out and back in through `withHome` rather than constructing
+one. `acquire`, `withHome`, `snapshot`, `subscribe`, `evictIdle`, `stats` and `closeAll` are the
+contract; `createHomeRuntime` builds one over injectable dependencies so the refcount, the cap,
+the breaker and the eviction boundary are testable without a network. The socket is a cache and
+never the source of truth: a request landing on an instance holding no connection opens one, and
+that cold path is normal rather than an error. The behaviour worth naming on its own is that the
+graph is never emptied on disconnect, only marked stale, so a person watching their 3D home sees
+it go grey instead of watching their house vanish. `tests/home-runtime.test.js` (27 pure) and
+`tests/home-runtime-live.test.js` (5 live, self-skipping on `HOME_ASSISTANT_URL`) cover it.
+
+**Measured:**
+
+- Socket reuse, host side: two sequential `withHome` calls against a real Home Assistant 2026.9.0
+ (125 entities, 4 rooms) left `ss -tn state established` at **1** socket after each call, with
+ **1** bridge constructed. `closeAll()` took it back to 0.
+- Breaker, at the production 15 s connect timeout: failures 1 to 5 cost 15009, 15005, 15005,
+ 15004 and 15002 ms; attempt 6 failed in **0 ms** with `home_breaker_open` and a message naming
+ the 300 second cooldown. The store is written with `status: unreachable` and a `status_detail`
+ ending "paused retries for five minutes", so the connect screen explains it without a socket.
+- Kill and restart, mid-subscription: `docker stop` left the subscriber's last event at
+ `stale=true connected=false status=unreachable` with all four rooms (Bedroom, Front Door,
+ Kitchen, Living Room) still readable. `docker start` restored `stale=false connected=true`
+ about 6 s later with no client action and 2 events delivered.
+- Eviction at its boundary: 89,999 ms after the last release evicts 0; 90,000 ms evicts 1 and
+ `stats().open` drops to 0.
+- `npx vitest run tests/home-runtime.test.js packages/home-bridge`: 57 passed, 7 skipped.
+- `npm run check:rules --paths <the files>`: clean.
+
+**Deviations:**
+
+- The order says order 01 must have landed first. It had not when this session started
+ (`api/_lib/home/` did not exist), and several sessions were running the campaign in parallel,
+ so the schema and `store.js` were written here and then converged with the peer session that
+ owned order 01. Two `create table if not exists` migrations for the same tables raced, which
+ makes the loser a silent no-op that skips the CHECK constraints declared inside its CREATE
+ TABLE: the duplicate was withdrawn and `20260903130000_home_schema_reconcile.sql` adds
+ `home_connections_relay_chk` and `home_action_log_risk_chk` by name, so a raced database ends
+ up matching what `20260903030000_home_connections.sql` says it guarantees. It is a no-op on a
+ cleanly provisioned one.
+- A real concurrency defect in the pool, found by the test that two simultaneous `acquire` calls
+ share one open: the pool slot was claimed AFTER the credential read, so two callers that both
+ cleared the map check during that database round trip each opened a socket and the second
+ `entries.set` orphaned the first, which was then never pooled, never evicted and never closed.
+ A page load and an SSE stream starting together is exactly that race. The slot is now reserved
+ synchronously before the first await and the credential is read inside `entry.ready`.
+- `HomeBridge` gained `haVersion` and `registries` getters. The capability record has to state
+ the version the instance actually reported rather than one inferred from a state attribute.
+
+**Left open:** nothing in this order. The `admission` ladder that now rides along in `stats()`
+belongs to order 14 and landed alongside this work in the same worktree.
+
+**Commits:** this entry, the tests, the reconcile migration and the pool fix.
diff --git a/tests/home-runtime-live.test.js b/tests/home-runtime-live.test.js
new file mode 100644
index 000000000..4e3abb131
--- /dev/null
+++ b/tests/home-runtime-live.test.js
@@ -0,0 +1,187 @@
+// The pool, against a real Home Assistant.
+//
+// The pure tests in tests/home-runtime.test.js prove the refcounting, the cap,
+// the breaker and the eviction boundary with a fake bridge, because a working
+// house cannot be asked to fail its next five connects on cue. What they cannot
+// prove is the claim the whole module exists for: that a second call really
+// rides the first WebSocket instead of opening another one against somebody's
+// instance. That needs a real socket and a real house, so it lives here.
+//
+// Self-skipping. Point it at any instance:
+//
+// node scripts/home-test-instance.mjs --up --onboard --seed --json
+// HOME_ASSISTANT_URL=http://127.0.0.1:<port> HOME_ASSISTANT_TOKEN=<token> \
+// npx vitest run tests/home-runtime-live.test.js
+//
+// Never mocked. A fake instance is what hid Home Assistant's own
+// `intent__HassTurnOff` performing an UNLOCK on a lock, which is the single most
+// important thing this lane learned.
+
+import { afterAll, describe, expect, it } from 'vitest';
+
+import { createHomeRuntime } from '../api/_lib/home/runtime.js';
+import { HOME_STATUS } from '../api/_lib/home/store.js';
+import { HomeBridge } from '@three-ws/home-bridge';
+
+const BASE_URL = process.env.HOME_ASSISTANT_URL;
+const TOKEN = process.env.HOME_ASSISTANT_TOKEN;
+const live = BASE_URL && TOKEN ? describe : describe.skip;
+
+const HOME_ID = '44444444-4444-4444-8444-444444444444';
+const USER_ID = '55555555-5555-4555-8555-555555555555';
+
+/** Counts the sockets actually constructed, which is the measurement this file exists for. */
+function countingFactory() {
+ const built = [];
+ const factory = (options) => {
+ const bridge = new HomeBridge(options);
+ built.push(bridge);
+ return bridge;
+ };
+ factory.built = built;
+ return factory;
+}
+
+/** The store, reduced to the reads the runtime performs, pointed at the live instance. */
+function liveStoreDeps() {
+ const handshakes = [];
+ return {
+ handshakes,
+ getConnection: async (id, userId) =>
+ id === HOME_ID && userId === USER_ID
+ ? { id, user_id: userId, label: 'Live', base_url: BASE_URL, status: HOME_STATUS.CONNECTED }
+ : null,
+ getDecryptedToken: async () => ({
+ token: TOKEN,
+ baseUrl: BASE_URL,
+ transport: 'direct',
+ relayId: null,
+ fingerprint: 'live',
+ }),
+ listAllowedEntities: async () => [],
+ recordHandshake: async (id, update) => {
+ handshakes.push({ id, ...update });
+ return null;
+ },
+ };
+}
+
+const runtimes = [];
+function liveRuntime(extra = {}) {
+ const createBridge = countingFactory();
+ const deps = liveStoreDeps();
+ const runtime = createHomeRuntime({ createBridge, ...deps, ...extra });
+ runtimes.push(runtime);
+ return { runtime, createBridge, deps };
+}
+
+afterAll(() => {
+ // Leaving a socket open would hold the instance's connection slot after the
+ // run, which is exactly the failure this module is here to prevent.
+ for (const runtime of runtimes) runtime.closeAll();
+});
+
+live('home runtime, against a real Home Assistant', () => {
+ it('opens one socket for two sequential withHome calls', async () => {
+ const { runtime, createBridge } = liveRuntime();
+
+ const first = await runtime.withHome(HOME_ID, USER_ID, (bridge) => ({
+ rooms: bridge.graph.rooms.length,
+ entities: Object.keys(bridge.states).length,
+ version: bridge.haVersion,
+ }));
+ const second = await runtime.withHome(HOME_ID, USER_ID, (bridge) => ({
+ rooms: bridge.graph.rooms.length,
+ entities: Object.keys(bridge.states).length,
+ }));
+
+ // A real house answered, twice, with the same numbers.
+ expect(first.entities).toBeGreaterThan(0);
+ expect(second.entities).toBe(first.entities);
+ expect(first.rooms).toBe(second.rooms);
+ expect(first.version).toMatch(/^\d+\.\d+/);
+
+ // And it did so over ONE socket. This assertion is the whole file.
+ expect(createBridge.built).toHaveLength(1);
+ expect(createBridge.built[0].connected).toBe(true);
+ expect(runtime.stats().open).toBe(1);
+
+ runtime.closeAll();
+ expect(createBridge.built[0].connected).toBe(false);
+ }, 60_000);
+
+ it('records the handshake it measured, so the store never holds a guessed capability', async () => {
+ const { runtime, deps } = liveRuntime();
+
+ await runtime.withHome(HOME_ID, USER_ID, () => null);
+ // The handshake write is fire and forget on purpose; give it a turn.
+ await new Promise((resolve) => setTimeout(resolve, 50));
+
+ const recorded = deps.handshakes.find((h) => h.status === HOME_STATUS.CONNECTED);
+ expect(recorded).toBeTruthy();
+ expect(recorded.capabilities.websocket).toBe(true);
+ expect(recorded.capabilities.entityCount).toBeGreaterThan(0);
+ expect(recorded.capabilities.haVersion).toMatch(/^\d+\.\d+/);
+ runtime.closeAll();
+ }, 60_000);
+
+ it('reopens after eviction, which is the cold path a recycled instance takes', async () => {
+ let clock = 1_000_000;
+ const { runtime, createBridge } = liveRuntime({ now: () => clock, idleMs: 1_000 });
+
+ await runtime.withHome(HOME_ID, USER_ID, () => null);
+ clock += 1_001;
+ expect(runtime.evictIdle(clock)).toBe(1);
+ expect(createBridge.built[0].connected).toBe(false);
+
+ const rooms = await runtime.withHome(HOME_ID, USER_ID, (bridge) => bridge.graph.rooms.length);
+
+ expect(createBridge.built).toHaveLength(2);
+ expect(rooms).toBeGreaterThanOrEqual(0);
+ runtime.closeAll();
+ }, 60_000);
+
+ it('holds one socket while a subscriber watches, and releases it when they leave', async () => {
+ let clock = 1_000_000;
+ const { runtime, createBridge } = liveRuntime({ now: () => clock, idleMs: 1_000 });
+
+ const events = [];
+ const unsubscribe = await runtime.subscribe(HOME_ID, USER_ID, (event) => events.push(event));
+
+ expect(events).toHaveLength(1);
+ expect(events[0].connected).toBe(true);
+ expect(events[0].stale).toBe(false);
+ expect(runtime.stats().subscribers).toBe(1);
+
+ // A watched home is never evicted out from under the browser watching it.
+ clock += 60_000;
+ expect(runtime.evictIdle(clock)).toBe(0);
+
+ unsubscribe();
+ clock += 1_001;
+ expect(runtime.evictIdle(clock)).toBe(1);
+ expect(createBridge.built[0].connected).toBe(false);
+ }, 60_000);
+
+ it('fails a bad token as auth rather than as an unreachable house', async () => {
+ const createBridge = countingFactory();
+ const runtime = createHomeRuntime({
+ createBridge,
+ ...liveStoreDeps(),
+ getDecryptedToken: async () => ({
+ token: 'not-a-real-token',
+ baseUrl: BASE_URL,
+ transport: 'direct',
+ relayId: null,
+ fingerprint: 'bad',
+ }),
+ });
+ runtimes.push(runtime);
+
+ // The two failures need different recovery text, so they must not collapse
+ // into one code: "your house is offline" and "your token is wrong" send the
+ // user to completely different places.
+ await expect(runtime.acquire(HOME_ID, USER_ID)).rejects.toMatchObject({ code: 'auth' });
+ expect(runtime.stats().open).toBe(0);
+ }, 60_000);
+});