Skip to content

Open-source platform for 3D AI agents. Turn text or a photo into a rigged, animated GLB avatar, give it an LLM brain, memory and a wallet, and embed it anywhere with one web component. Remote MCP server, x402 payments, Three.js, browser-native.

fix(okx-bot): refuse a local revive while another host is serving agent 2632

nirholas1107a473d8e1
diff --git a/api/_lib/ops/subsystem-health.js b/api/_lib/ops/subsystem-health.js
index 7f3e70dbc..11356f7e2 100644
--- a/api/_lib/ops/subsystem-health.js
+++ b/api/_lib/ops/subsystem-health.js
@@ -453,22 +453,30 @@ export function classifyOkxChatBotBeat(beat, now = Date.now()) {
const lastBeatMs = beat.last_beat_at ? new Date(beat.last_beat_at).getTime() : 0;
const ageMs = lastBeatMs ? now - lastBeatMs : Number.POSITIVE_INFINITY;
const meta = beat.meta && typeof beat.meta === 'object' ? beat.meta : {};
+ // Who is beating, reported on every branch that has a beat at all. Not
+ // decoration: scripts/lib/okx-bot-host-guard.mjs reads these two fields to
+ // refuse a local revive that would put a second writer on the one shared
+ // identity, and it reads them from THIS public endpoint precisely so that
+ // check needs no database and no gcloud login.
+ const host = typeof meta.host === 'string' && meta.host ? meta.host : null;
+ const hostDurable = typeof meta.hostDurable === 'boolean' ? meta.hostDurable : null;
+ const identity = { host, hostDurable };
if (ageMs > OKX_BOT_DOWN_MS) {
const ageNote = Number.isFinite(ageMs) ? `${Math.round(ageMs / 60_000)} min old` : 'never recorded';
return {
...base,
+ ...identity,
status: 'down',
detail: `heartbeat ${ageNote}, the chat-bot host is gone, so marketplace chat is not delivered at all`,
hint: 'Redeploy the host: gcloud builds submit --config workers/okx-chat-bot/cloudbuild.yaml . (see workers/okx-chat-bot/README.md). For a local stopgap, npm run okx:bot.',
};
}
if (ageMs > OKX_BOT_FRESH_MS) {
- return { ...base, status: 'degraded', detail: `heartbeat ${Math.round(ageMs / 1000)}s old, host slow or mid-restart` };
+ return { ...base, ...identity, status: 'degraded', detail: `heartbeat ${Math.round(ageMs / 1000)}s old, host slow or mid-restart` };
}
const reported = typeof meta.health === 'string' ? meta.health : 'unknown';
- const host = typeof meta.host === 'string' && meta.host ? meta.host : null;
if (reported === 'ok') {
const online = `online (${meta.activeClients ?? '?'} XMTP client(s), provider=${beat.mode || meta.provider || 'unknown'}${host ? `, host=${host}` : ''})`;
// A host that beats but cannot survive on its own is a stopgap, not the
@@ -477,18 +485,20 @@ export function classifyOkxChatBotBeat(beat, now = Date.now()) {
if (meta.hostDurable === false) {
return {
...base,
+ ...identity,
status: 'degraded',
detail: `${online}; that host is a stopgap and dies with its workspace, so chat delivery is not durable yet`,
hint: 'Deploy the always-on host: gcloud builds submit --config workers/okx-chat-bot/cloudbuild.yaml . (see workers/okx-chat-bot/README.md).',
};
}
- return { ...base, status: 'ok', detail: online };
+ return { ...base, ...identity, status: 'ok', detail: online };
}
if (reported === 'unknown') {
- return { ...base, status: 'unknown', detail: String(meta.detail || 'bot could not determine its own state') };
+ return { ...base, ...identity, status: 'unknown', detail: String(meta.detail || 'bot could not determine its own state') };
}
return {
...base,
+ ...identity,
status: reported === 'down' ? 'down' : 'degraded',
detail: String(meta.detail || `bot reports ${reported}`),
hint: meta.needsHumanLogin
diff --git a/prompts/finish/907-backlog-08-okx-chat-bot-always-on.md b/prompts/finish/907-backlog-08-okx-chat-bot-always-on.md
index 2c08586b0..8623ccdce 100644
--- a/prompts/finish/907-backlog-08-okx-chat-bot-always-on.md
+++ b/prompts/finish/907-backlog-08-okx-chat-bot-always-on.md
@@ -14,30 +14,32 @@ the daemon (observed alive at 21:09, stale pid by 03:13). The marketplace's own
tests then report "timeout, no delivery in 30 min", which is what got the listing
flagged offline once already.
-**State on 2026-09-02.** The wallet session is logged in (`[email protected]`), the daemon
-is running under the worker's own supervisor rather than the codespace autostart unit, and
-`/api/healthz` shows the `okx_chat_bot` subsystem for the first time. It is still a
-codespace, so every beat says so: `hostDurable=false` reads as degraded on the ops surface,
-not green. What is left is the deploy itself.
-
-## Immediate revive (do this first, it takes one command)
-
-```sh
-npm run okx:bot # scripts/okx-bot-revive.mjs
-# exit 0 = online
-# exit 2 = staged but logged out; it prints the login URL and the poll commands
-```
-
-The login needs a human: email OTP as `[email protected]`. Run this immediately
-before any retest window. The wiring table is in
-[../okx-ai/RUNBOOK.md](_context/okx-ai-RUNBOOK.md) section 0.5.
-
-Then hand the daemon to the worker instead of leaving it parentless, so the fleet can see
-it and an expired session pages instead of going quiet:
+**State on 2026-09-09, measured.** The durable host exists and this order is mostly done.
+Cloud Run service `okx-chat-bot`, revision `okx-chat-bot-00001-926`, booted
+2026-09-05T00:33Z and beating every 30s with `loggedIn: true`, `activeClients: 1`,
+`daemonRestarts: 0`. What is left is one reply lane: the AI provider refuses this
+project's credential (`Lightning dunning decision is deny`, a GCP billing hold), so chat
+arrives and no reply is authored. Both remaining steps are the owner's, and they are listed
+at the bottom of this file.
+
+## Do NOT start a local daemon (read this before running anything)
+
+The codespace stopgap is retired and **must stay stopped**. The bot's identity, the
+onchainos wallet keyring plus the XMTP client database, is a single state object with
+exactly one writer, which is what `--max-instances=1` protects on Cloud Run. Starting a
+second daemon anywhere puts a second writer on it, and a torn identity costs a human email
+OTP to recover.
+
+An earlier version of this file opened with "Immediate revive (do this first)" and
+`npm run okx:bot`. Following it on 2026-09-09 started exactly that rival daemon. The
+command now refuses while any other host is beating (exit 3, `--force` to override), so the
+trap is closed mechanically as well as here, but the rule stands: run nothing local while
+the deployed host is up.
+
+Read the deployed host's state instead. This read needs no `gcloud` and no credential:
```sh
-PORT=8080 OKX_BOT_REPO_ROOT=/workspaces/three.ws \
- node --env-file=.env.local workers/okx-chat-bot/index.js
+curl -s https://three.ws/api/healthz | jq '.subsystems.subsystems[]|select(.name=="okx_chat_bot")'
```
## The durable fix
@@ -71,21 +73,44 @@ host. Build it:
host detect an expired session and emit an actionable alert naming the exact
command, rather than failing chat silently.
-## Owner actions
+## Owner actions (both, and nothing else, as of 2026-09-09)
+
+1. **Fund one AI lane.** Clearing the GCP billing hold on `aerial-vehicle-466722-p5` is
+ the one that also restores the platform's own Vertex anchor (`LLM providers DOWN`,
+ count 657). OpenRouter credit or an OpenAI reactivation each work too, because the
+ chain elects on a live probe.
+2. **Deploy the worker**, so the AI-lane chain from `ad723e87f` is actually running. The
+ live beat carries no `providerLane`/`providerChain` key, which is how you can tell
+ the serving revision predates it without `gcloud`: it is pinned to Vertex and cannot
+ pick up a funded lane even after step 1.
+
+ ```sh
+ gcloud builds submit --config workers/okx-chat-bot/cloudbuild.yaml \
+ --region us-central1 --project aerial-vehicle-466722-p5 \
+ --substitutions=SHORT_SHA=manual$(date +%s) .
+ ```
-- One email OTP login when the session is logged out.
-- An AI-provider credential for the headless box, if the host cannot reuse this
- machine's.
+Order does not matter; whichever lands second is picked up by the next 15-minute election.
+An email OTP is **not** currently needed: the session has been `loggedIn: true` since
+2026-09-05.
## Definition of done
-- [ ] `npm run okx:bot` exits 0, chat delivery verified end to end with a real
- inbound message.
-- [ ] The daemon runs on an always-on host, not this codespace.
-- [ ] Its workspace carries real three.ws context, verified by asking it a
- platform question and reading the answer.
-- [ ] A health endpoint exists and an offline session raises an alert.
-- [ ] [../okx-ai/PROGRESS.md](_context/okx-ai-PROGRESS.md) updated with the host details.
+- [ ] **Chat delivery verified end to end with a real inbound message.** The inbound half
+ is proven (`activeClients: 1`, `agentCount: 1`, 0 daemon restarts since
+ 2026-09-05). The reply half cannot pass until owner action 1. The original wording
+ of this line, "`npm run okx:bot` exits 0", is retired: that command is the
+ codespace stopgap and must not run while the deployed host is up.
+- [x] **The daemon runs on an always-on host, not this codespace.** Cloud Run
+ `okx-chat-bot-00001-926`, 4.8 days of continuous 30s beats.
+- [ ] **Its workspace carries real three.ws context.** The mechanical half is done and
+ tested: `buildChatBriefing()` renders 10,069 bytes from the live catalog module and
+ is rebuilt on every boot. Asking the bot a platform question needs owner action 1.
+- [x] **A health endpoint exists and an offline session raises an alert.** `/readyz` is
+ strict, `/api/healthz` carries the `okx_chat_bot` subsystem (now naming its
+ `host` and `hostDurable` as fields), and `sendOpsAlert` fires on every
+ transition into a bad state.
+- [x] **[../okx-ai/PROGRESS.md](_context/okx-ai-PROGRESS.md) updated with the host details.**
## Retire this prompt when it is done (required)
diff --git a/prompts/finish/_context/backlog-00-INDEX.md b/prompts/finish/_context/backlog-00-INDEX.md
index c27406593..d33b3c0cf 100644
--- a/prompts/finish/_context/backlog-00-INDEX.md
+++ b/prompts/finish/_context/backlog-00-INDEX.md
@@ -43,7 +43,7 @@ observable health, so 01 to 03 come first.
| 01 | [x402 settle: clear `fee_runway_exhausted`](../902-backlog-01-x402-settle-runway.md) | **the deploy landed 2026-09-08**; one capital decision left (re-measured 2026-09-09 third pass: `fee_runway_exhausted` is gone, the sponsor is 0.000842 SOL under its floor, and the outage hint has been corrected to name the right mechanism) | pick one: release the **0.0558 SOL the platform already owns** but has fenced behind `pump-x402-launcher`'s 0.1 SOL `minSol`, or send SOL to the economy master. Either moves funds, so either is gate 1 |
| 05 | [R2 bucket CORS: verify, then fix at the origin](../906-backlog-05-r2-bucket-cors.md) | one credential (re-measured twice on 2026-09-09, both bucket-origin surfaces still fail) | mint an R2 "Admin Read & Write" token for `chatty-storage`. The storage outage that briefly gated this (`S3_SECRET_ACCESS_KEY`) recovered the same day, so this is the only step left |
| 07 | [BNB testnet: deploy the two finished contracts](../910-backlog-07-bnb-testnet-deploys.md) | one funded EOA | send tBNB to `0x1C4918894dfA5eE11cfF9629B458b5169Cfa3871` (faucet is reCAPTCHA-gated) |
-| 08 | [OKX chat bot: move off the codespace](../907-backlog-08-okx-chat-bot-always-on.md) | **deployed 2026-09-04** and still beating (re-measured 2026-09-09 from `bot_heartbeat`, no `gcloud` needed); only the reply lane is blocked, and the AI-lane chain committed the same day has not shipped | two: clear the GCP billing hold (it also restores the platform Vertex anchor, `LLM providers DOWN` x657), and deploy `workers/okx-chat-bot/cloudbuild.yaml` so a funded lane can actually be elected |
+| 08 | [OKX chat bot: move off the codespace](../907-backlog-08-okx-chat-bot-always-on.md) | **deployed 2026-09-04** and still beating (re-measured 2026-09-09 from `bot_heartbeat`, no `gcloud` needed); only the reply lane is blocked, and the AI-lane chain committed the same day has not shipped. `npm run okx:bot` now refuses to start a rival daemon while that host is beating (2026-09-09, after the order's own first instruction started one) | two: clear the GCP billing hold (it also restores the platform Vertex anchor, `LLM providers DOWN` x657), and deploy `workers/okx-chat-bot/cloudbuild.yaml` so a funded lane can actually be elected |
| 09 | [Telegram bots: durable hosting for both feeds](../908-backlog-09-telegram-bots-durability.md) | **done**; the firehose was found silently dead on 2026-09-09 (dead RPC), fixed and hardened the same day | clear the commit gate on its file update |
| 10 | [x402scan listing: finish the last three steps](../909-backlog-10-x402scan-listing.md) | **the deploy landed 2026-09-08**; only the origin registration is left (re-measured 2026-09-09) | one SIWX wallet signature at x402scan `/resources/register` for origin `https://three.ws`; no funds move |
diff --git a/scripts/lib/okx-bot-host-guard.mjs b/scripts/lib/okx-bot-host-guard.mjs
new file mode 100644
index 000000000..512048bfa
--- /dev/null
+++ b/scripts/lib/okx-bot-host-guard.mjs
@@ -0,0 +1,107 @@
+// Refuse to start a second OKX chat-bot daemon while another host is already
+// serving agent #2632.
+//
+// The bot's whole identity (the onchainos wallet keyring plus the XMTP client
+// database) is ONE state object with exactly one writer, which is why the Cloud
+// Run service is pinned to --max-instances=1. A local revive started while that
+// service is up puts a second daemon on the same inbox and a second writer on
+// the same identity, and recovering a torn identity costs a human email OTP.
+//
+// The read is deliberately credential-free: /api/healthz already turns the
+// bot's heartbeat row into the `okx_chat_bot` subsystem, so this works from a
+// fresh clone with no DATABASE_URL, no gcloud login, and no secrets. That
+// matters because the machine most likely to run a revive by mistake is the one
+// with the least set up.
+
+/** Public health endpoint carrying the okx_chat_bot subsystem. */
+export const DEFAULT_HEALTHZ_URL = 'https://three.ws/api/healthz';
+
+/**
+ * Decide whether a local daemon may be started.
+ *
+ * Fail-open on ignorance, fail-closed on evidence: an unreachable endpoint must
+ * never block the emergency revive (the case where everything is down is
+ * exactly when this script is needed), but a host we can SEE beating must.
+ *
+ * @param {{ status?: string, detail?: string, host?: string|null, hostDurable?: boolean|null }|null} subsystem
+ * The `okx_chat_bot` entry from /api/healthz, or null when it could not be read.
+ * @param {{ localHost?: string|null, reachable?: boolean }} [opts]
+ * @returns {{ blocked: boolean, code: string, detail: string, host: string|null }}
+ */
+export function classifyLocalRevive(subsystem, { localHost = null, reachable = true } = {}) {
+ if (!reachable || !subsystem) {
+ return {
+ blocked: false,
+ code: 'unreadable',
+ detail: 'could not read the platform health endpoint, so no remote host could be ruled out',
+ host: null,
+ };
+ }
+ const status = String(subsystem.status || 'unknown');
+ const detail = String(subsystem.detail || '');
+ const host = typeof subsystem.host === 'string' && subsystem.host ? subsystem.host : null;
+
+ // Nothing has ever hosted this bot. This is the one "no host" verdict the
+ // endpoint states positively, so it is the only one read as an all-clear.
+ if (status === 'unknown' && /no heartbeat reported yet/.test(detail)) {
+ return { blocked: false, code: 'never_hosted', detail: 'no host has ever reported a heartbeat', host: null };
+ }
+ // The host stopped beating. This is the emergency the local path exists for.
+ if (status === 'down' && /^heartbeat /.test(detail)) {
+ return { blocked: false, code: 'host_gone', detail: detail || 'the host stopped beating', host };
+ }
+ // Re-staging the workspace on the machine that is already the host is the
+ // intended use: same daemon, same identity, one writer.
+ if (host && localHost && host === localHost) {
+ return { blocked: false, code: 'self', detail: `this machine (${host}) is the beating host`, host };
+ }
+ // A beat with no host named is still a beat. Not knowing WHICH machine is
+ // serving chat is not a licence to add a second one, and this branch is not
+ // hypothetical: an API build older than the `host` field reports exactly
+ // this, and reading it as an all-clear started a rival daemon on 2026-09-09.
+ if (!host) {
+ return {
+ blocked: true,
+ code: 'unnamed_host_beating',
+ detail: `the bot reports ${status}, so a host is serving agent #2632, but the health endpoint does not name it`,
+ host: null,
+ };
+ }
+ return {
+ blocked: true,
+ code: subsystem.hostDurable === false ? 'stopgap_host_beating' : 'durable_host_beating',
+ detail: `${host} is serving agent #2632 right now (${status})`,
+ host,
+ };
+}
+
+/**
+ * Read the okx_chat_bot subsystem from the public health endpoint.
+ *
+ * Never throws: every failure is reported as unreachable, and the caller turns
+ * that into a warning rather than a refusal.
+ *
+ * @param {{ url?: string, timeoutMs?: number, fetchImpl?: typeof fetch }} [opts]
+ * @returns {Promise<{ reachable: boolean, subsystem: object|null, error: string|null }>}
+ */
+export async function fetchOkxBotSubsystem({
+ url = process.env.OKX_BOT_HEALTHZ_URL || DEFAULT_HEALTHZ_URL,
+ timeoutMs = 10_000,
+ fetchImpl = fetch,
+} = {}) {
+ const ac = new AbortController();
+ const timer = setTimeout(() => ac.abort(), timeoutMs);
+ try {
+ const res = await fetchImpl(url, { signal: ac.signal, headers: { accept: 'application/json' } });
+ if (!res.ok) return { reachable: false, subsystem: null, error: `HTTP ${res.status}` };
+ const body = await res.json();
+ const list = body?.subsystems?.subsystems;
+ if (!Array.isArray(list)) return { reachable: false, subsystem: null, error: 'no subsystem list in the response' };
+ const found = list.find((s) => s?.name === 'okx_chat_bot') || null;
+ return { reachable: true, subsystem: found, error: found ? null : 'okx_chat_bot not reported' };
+ } catch (err) {
+ return { reachable: false, subsystem: null, error: err?.message || String(err) };
+ } finally {
+ clearTimeout(timer);
+ }
+}
diff --git a/scripts/okx-bot-revive.mjs b/scripts/okx-bot-revive.mjs
index 57f6491f4..dc8ca3d93 100644
--- a/scripts/okx-bot-revive.mjs
+++ b/scripts/okx-bot-revive.mjs
@@ -1,9 +1,19 @@
#!/usr/bin/env node
// Brings the OKX.AI marketplace chat bot for agent #2632 back online, end to end.
//
-// npm run okx:bot
+// npm run okx:bot # refuses while another host is serving chat
+// npm run okx:bot -- --force # start anyway (only after stopping that host)
//
-// The bot is a LOCAL `okx-a2a` daemon plus an `onchainos` wallet session, both of
+// EMERGENCY PATH ONLY, since 2026-09-04. The durable host is the Cloud Run
+// service `okx-chat-bot` (workers/okx-chat-bot/), and the bot's identity (the
+// onchainos wallet keyring plus the XMTP client database) is a single state
+// object with exactly one writer. Starting this daemon while that service is up
+// puts a second daemon on the same inbox, and recovering a torn identity costs a
+// human email OTP. So the script refuses to start when it can see another host
+// beating; `--force` overrides, and is for the case where you have just stopped
+// the other host yourself.
+//
+// The bot is an `okx-a2a` daemon plus an `onchainos` wallet session, both of
// which live outside this repo. A codespace rebuild (or an idle nap) wipes them,
// and OKX-side chat tests then time out with "no delivery in 30 min". This script
// is the whole recovery: install, daemon, AI workspace, catalog briefing, skills,
@@ -19,7 +29,11 @@ import { homedir } from 'node:os';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';
+import { classifyLocalRevive, fetchOkxBotSubsystem } from './lib/okx-bot-host-guard.mjs';
+import { resolveHost } from '../workers/okx-chat-bot/config.js';
+
const REPO = join(dirname(fileURLToPath(import.meta.url)), '..');
+const FORCE = process.argv.slice(2).includes('--force');
const WORKSPACE = join(homedir(), '.okx-agent-task', 'workspace');
const SKILLS_SRC = join(REPO, '.agents', 'skills');
const MIN_NODE_MAJOR = 22;
@@ -62,6 +76,25 @@ function has(bin) {
console.log('OKX chat bot revive\n');
+// 0. One writer. The identity this daemon would open is shared with whatever
+// host is already serving agent #2632, so ask the platform who that is before
+// installing or starting anything. The read is credential-free on purpose: it
+// has to work on the machine with the least set up, which is the one most
+// likely to run this by mistake.
+const local = resolveHost().label;
+const remote = await fetchOkxBotSubsystem();
+const verdict = classifyLocalRevive(remote.subsystem, { localHost: local, reachable: remote.reachable });
+if (verdict.blocked && !FORCE) {
+ console.error(`REFUSED: ${verdict.detail}.\n`);
+ console.error('Starting a second daemon would put two writers on one wallet keyring and one');
+ console.error('XMTP database, and a torn identity costs a human email OTP to recover.\n');
+ console.error('If that host is healthy, you want nothing from this script. To read its state:');
+ console.error(" curl -s https://three.ws/api/healthz | jq '.subsystems.subsystems[]|select(.name==\"okx_chat_bot\")'\n");
+ console.error('If you have already stopped it, re-run with --force.');
+ process.exit(3);
+}
+step('one-writer check', verdict.blocked ? `OVERRIDDEN with --force: ${verdict.detail}` : verdict.detail);
+
// 1. Node version, the daemon refuses to run below 22.14.
const major = Number(process.versions.node.split('.')[0]);
if (major < MIN_NODE_MAJOR) {
diff --git a/tests/okx-chat-bot.test.js b/tests/okx-chat-bot.test.js
index 5782c28d1..693c9c30f 100644
--- a/tests/okx-chat-bot.test.js
+++ b/tests/okx-chat-bot.test.js
@@ -24,6 +24,7 @@ import { createSupervisor } from '../workers/okx-chat-bot/supervisor.js';
import { STATE_ROOTS, STATE_EXCLUDES } from '../workers/okx-chat-bot/state.js';
import { SKILLS, buildWorkspace } from '../workers/okx-chat-bot/workspace.js';
import { classifyOkxChatBotBeat } from '../api/_lib/ops/subsystem-health.js';
+import { classifyLocalRevive, fetchOkxBotSubsystem } from '../scripts/lib/okx-bot-host-guard.mjs';
import { buildChatBriefing } from '../api/_lib/okx-chat-briefing.js';
import { OKX_CATALOG } from '../api/_lib/okx-catalog.js';
@@ -775,3 +776,126 @@ describe('okx_chat_bot subsystem health', () => {
);
});
});
+
+// One writer, enforced. The bot's identity (wallet keyring + XMTP database) is a
+// single state object, which is why the Cloud Run service is pinned to
+// --max-instances=1. `npm run okx:bot` is the one path that can still put a
+// second daemon on it, and on 2026-09-09 it did: the health endpoint reported
+// degraded without naming a host, the guard read that as an all-clear, and a
+// rival daemon came up on the same inbox. So the rule is fail-open on ignorance
+// (an unreachable endpoint must never block the emergency revive) and fail-closed
+// on any evidence that something is beating, named or not.
+describe('okx-chat-bot local revive guard', () => {
+ const sub = (over = {}) => ({ name: 'okx_chat_bot', status: 'ok', detail: 'online', host: null, ...over });
+
+ it('allows the revive when the health endpoint cannot be read at all', () => {
+ const v = classifyLocalRevive(null, { reachable: false });
+ expect(v.blocked).toBe(false);
+ expect(v.code).toBe('unreadable');
+ });
+
+ it('allows the revive when no host has ever reported', () => {
+ const v = classifyLocalRevive(sub({ status: 'unknown', detail: 'no heartbeat reported yet' }));
+ expect(v.blocked).toBe(false);
+ expect(v.code).toBe('never_hosted');
+ });
+
+ it('allows the revive when the host stopped beating, which is what it is for', () => {
+ const v = classifyLocalRevive(
+ sub({ status: 'down', detail: 'heartbeat 41 min old, the chat-bot host is gone', host: 'cloudrun:okx-chat-bot' }),
+ );
+ expect(v.blocked).toBe(false);
+ expect(v.code).toBe('host_gone');
+ });
+
+ it('blocks a revive while the durable host is serving chat', () => {
+ const v = classifyLocalRevive(sub({ host: 'cloudrun:okx-chat-bot (okx-chat-bot-00001-926)', hostDurable: true }), {
+ localHost: 'codespace:fluffy',
+ });
+ expect(v.blocked).toBe(true);
+ expect(v.code).toBe('durable_host_beating');
+ expect(v.detail).toContain('okx-chat-bot-00001-926');
+ });
+
+ // Two stopgaps are not better than one: they are two writers.
+ it('blocks a revive while another stopgap is serving chat', () => {
+ const v = classifyLocalRevive(sub({ host: 'codespace:other', hostDurable: false }), { localHost: 'codespace:mine' });
+ expect(v.blocked).toBe(true);
+ expect(v.code).toBe('stopgap_host_beating');
+ });
+
+ // The regression that made this guard exist: an API build older than the
+ // `host` field answers degraded and names nobody.
+ it('blocks a revive when a host is beating but the endpoint does not name it', () => {
+ const v = classifyLocalRevive(sub({ status: 'degraded', detail: 'the AI provider refuses this credential' }), {
+ localHost: 'codespace:mine',
+ });
+ expect(v.blocked).toBe(true);
+ expect(v.code).toBe('unnamed_host_beating');
+ });
+
+ // Re-staging the workspace on the machine that IS the host adds no writer.
+ it('allows the revive on the machine that is already the beating host', () => {
+ const v = classifyLocalRevive(sub({ host: 'codespace:mine', hostDurable: false }), { localHost: 'codespace:mine' });
+ expect(v.blocked).toBe(false);
+ expect(v.code).toBe('self');
+ });
+
+ it('reads the okx_chat_bot subsystem off a healthz body', async () => {
+ const body = { subsystems: { subsystems: [{ name: 'database', status: 'ok' }, { name: 'okx_chat_bot', status: 'ok', host: 'cloudrun:x' }] } };
+ const res = await fetchOkxBotSubsystem({
+ fetchImpl: async () => ({ ok: true, json: async () => body }),
+ });
+ expect(res.reachable).toBe(true);
+ expect(res.subsystem.host).toBe('cloudrun:x');
+ });
+
+ // Never throws: a network failure has to become a warning, not a crash, or
+ // the emergency path dies exactly when it is needed.
+ it('reports a failed fetch as unreachable instead of throwing', async () => {
+ const res = await fetchOkxBotSubsystem({
+ fetchImpl: async () => {
+ throw new Error('getaddrinfo ENOTFOUND');
+ },
+ });
+ expect(res.reachable).toBe(false);
+ expect(res.error).toContain('ENOTFOUND');
+ expect(classifyLocalRevive(res.subsystem, { reachable: res.reachable }).blocked).toBe(false);
+ });
+});
+
+// The guard reads these two fields off the public endpoint, so they are part of
+// its contract, not incidental output.
+describe('okx_chat_bot subsystem identity fields', () => {
+ const now = Date.parse('2026-08-02T12:00:00Z');
+ const beat = (ageMs, meta = {}) => ({
+ mode: 'claude',
+ last_beat_at: new Date(now - ageMs).toISOString(),
+ meta: { health: 'ok', activeClients: 1, host: 'cloudrun:okx-chat-bot', hostDurable: true, ...meta },
+ });
+
+ it('carries host and hostDurable on a healthy beat', () => {
+ const s = classifyOkxChatBotBeat(beat(20_000), now);
+ expect(s.host).toBe('cloudrun:okx-chat-bot');
+ expect(s.hostDurable).toBe(true);
+ });
+
+ it('carries them on a self-reported failure, which is when a human reads them', () => {
+ const s = classifyOkxChatBotBeat(beat(20_000, { health: 'degraded', reason: 'ai_provider_unauthorized' }), now);
+ expect(s.status).toBe('degraded');
+ expect(s.host).toBe('cloudrun:okx-chat-bot');
+ });
+
+ it('carries them on a stale beat, so a silent host is still identified', () => {
+ expect(classifyOkxChatBotBeat(beat(30 * 60_000), now).host).toBe('cloudrun:okx-chat-bot');
+ });
+
+ it('reports null rather than undefined for a beat predating the fields', () => {
+ const s = classifyOkxChatBotBeat(
+ { mode: 'claude', last_beat_at: new Date(now - 20_000).toISOString(), meta: { health: 'ok' } },
+ now,
+ );
+ expect(s.host).toBeNull();
+ expect(s.hostDurable).toBeNull();
+ });
+});
diff --git a/workers/okx-chat-bot/README.md b/workers/okx-chat-bot/README.md
index 31ce96459..55dd98a72 100644
--- a/workers/okx-chat-bot/README.md
+++ b/workers/okx-chat-bot/README.md
@@ -234,7 +234,38 @@ curl -s localhost:8080/readyz | jq '{ready: .health.ready, reason: .health.reaso
To stage the same workspace on a developer machine without running the worker,
use [scripts/okx-bot-revive.mjs](../../scripts/okx-bot-revive.mjs), which keeps
-the identical skill list.
+the identical skill list. It refuses to start a daemon while any other host is
+serving agent 2632: see **One writer, enforced** below.
+
+### One writer, enforced
+
+The wallet keyring and the XMTP client database are one state object with exactly
+one writer, which is what `--max-instances=1` protects on Cloud Run. Nothing
+protected it from a *second machine*, and `npm run okx:bot` is the one command
+that starts one. On 2026-09-09 it did: run from a codespace while the Cloud Run
+host was serving, it installed the CLIs, started a daemon, and came up
+`agentCount=1 activeClients=1` against the same inbox. It was stopped within
+three minutes and the deployed host never missed a beat, but nothing about the
+command said no.
+
+Now it asks first, through
+[scripts/lib/okx-bot-host-guard.mjs](../../scripts/lib/okx-bot-host-guard.mjs):
+
+| What the health endpoint says | Verdict |
+|---|---|
+| unreachable | **allowed**, with a warning. An emergency revive must not need the internet to work |
+| `no heartbeat reported yet` | **allowed**. Nothing has ever hosted this bot |
+| `down`, heartbeat stale | **allowed**. The host is gone; this is the emergency the script exists for |
+| any beat, host is this machine | **allowed**. Re-staging the workspace where the daemon already runs adds no writer |
+| any beat, another host | **refused**, exit 3, `--force` to override |
+| any beat, host not named | **refused**. An API build older than the `host` field answers exactly this, and reading it as an all-clear is what started the rival daemon |
+
+The read is `GET /api/healthz` and nothing else: no `DATABASE_URL`, no
+`gcloud` login, no secret. The machine most likely to run this by mistake is a
+fresh clone with none of those, so the guard has to work there or it does not
+work at all. That is also why [subsystem-health.js](../../api/_lib/ops/subsystem-health.js)
+now puts `host` and `hostDurable` on the `okx_chat_bot` subsystem as fields
+rather than only inside a sentence.
## HTTP surface
@@ -408,7 +439,8 @@ Daemon stdout and stderr are forwarded into the worker's own log stream under a
## Related
-- [scripts/okx-bot-revive.mjs](../../scripts/okx-bot-revive.mjs) stages the same workspace locally.
+- [scripts/okx-bot-revive.mjs](../../scripts/okx-bot-revive.mjs) stages the same workspace locally, and refuses while another host is serving.
+- [scripts/lib/okx-bot-host-guard.mjs](../../scripts/lib/okx-bot-host-guard.mjs) is the one-writer check that script runs first.
- [scripts/okx-bot-seed-state.mjs](../../scripts/okx-bot-seed-state.mjs) seeds the GCS session snapshot (`npm run okx:bot:seed-state`).
- [api/_lib/okx-chat-briefing.js](../../api/_lib/okx-chat-briefing.js) generates the subsession briefing.
- [workers/README.md](../README.md) is the worker index.