Skip to content

armgit / qm

Public Imported from GitHub
0

Multiplayer agent harness for work.

Expand dev instance capacity and safely clean up completed instances

Regan Bell64f88346ba62
diff --git a/.codex/skills/dev-instance/SKILL.md b/.codex/skills/dev-instance/SKILL.md
index 3f4901021..c2d719f4a 100644
--- a/.codex/skills/dev-instance/SKILL.md
+++ b/.codex/skills/dev-instance/SKILL.md
@@ -47,8 +47,14 @@ below), then spawns a **per-slot supervisor daemon** that owns the production-sh
Web-only instances need no Slack app or pool credentials. Slack-only instances skip
web dependencies, builds, and web/portal processes.
-The supervisor restarts crashed children with backoff, waits for a port to actually free
-before respawning (no more EADDRINUSE), health-probes everything every 10s, and writes a
+Port slots expand automatically beyond 16, up to the TCP port range (8,207 slots at
+the default base port of 8080); machine resources will usually limit capacity first.
+Slots 1–16 retain their existing ports. Allocation skips occupied port blocks.
+Startup refuses occupied ports without terminating their listeners; if a port is
+taken after allocation, stop the failed instance and retry `up` to select a free block.
+
+The supervisor restarts crashed children with backoff, refuses occupied ports,
+health-probes everything every 10s, and writes a
heartbeat so slot reclaim can tell "actively in use" from "abandoned".
**When Slack is enabled, `up` only prints success after proving the bot is reachable**: the Slack socket must be
@@ -119,6 +125,10 @@ canary message is deleted right after it round-trips. `CANARY_CHANNEL=<channel i
slot env overrides. With no eligible channel at all, `up` prints `delivery unverified` (or
fails under `--strict`).
+If its worktree disappears, the supervisor shuts down its children and releases its
+lease on the next heartbeat (within 15 seconds, plus shutdown time). Always run `down`
+before intentionally removing a worktree.
+
A forgotten instance cleans itself up: after 24 hours with no handled Slack turns,
interactions, reloads, or restarts, the supervisor tears itself down and frees the slot
on its next idle check (every 10 minutes). Ambient workspace events, health checks,
@@ -183,8 +193,13 @@ self-API calls can reach your local core. None of that runs on the default local
Report the slot, enabled surfaces, their URLs or Slack handle, and log directory. To test Slack-specific
behavior, DM the printed `@<handle>` (on Alice's machine that's one of `@bot1 … @bot10`)
-in `example.slack.com`; for admin and web behavior, open the printed portal URL. Tear down
-with `bash scripts/dev-instance.sh down` when QA is finished.
+in `example.slack.com`; for admin and web behavior, open the printed portal URL. Keep
+review instances running until the PR merges. After merge, run
+`bash scripts/dev-instance.sh down` from that worktree with the same pool-store override
+used for `up`, before removing the worktree. Verify the instance's supervisor and children
+have exited and its lease is released. Keep shared Postgres and persistent data intact.
+If the user asks to keep the instance beyond merge, retain its worktree too. For work
+without a PR, tear down when QA is finished unless the user requested a running demo.
## Troubleshooting
diff --git a/AGENTS.md b/AGENTS.md
index 659bf2382..44d918f2b 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -64,6 +64,11 @@ Two habits that keep task-focused changes from scarring the rest of the repo:
Slack Mac app, and don't ask permission first — do it on your own; don't wait to be
asked. Skip it for trivial refactors, docs, config, or pure-logic changes already
covered by tests.
+ Leave review instances running until the PR merges, unless the user asks to stop
+ earlier. After merge, tear down that worktree's instance and verify its processes
+ and lease are gone before removing the worktree. Preserve shared Postgres and
+ persistent data. If the user asks to keep an instance beyond merge, keep its
+ worktree too.
- **Keep screenshots out of Git and app assets.** Never commit screenshots or create
a tracked screenshots directory, including under docs or QA. Capture review images
in a temporary directory outside the checkout or an ignored local output directory,
diff --git a/scripts/dev/cli.ts b/scripts/dev/cli.ts
index 4cf0a10e2..a0436ab0f 100644
--- a/scripts/dev/cli.ts
+++ b/scripts/dev/cli.ts
@@ -14,7 +14,8 @@ import {
writeSlotFlag,
} from "./lib/pool.ts";
import {
- claimSlotLock,
+ claimPortSlot,
+ claimSlotPorts,
heartbeatFresh,
leaseReclaimReason,
leaseStale,
@@ -172,12 +173,18 @@ async function reclaimReclaimable(): Promise<boolean> {
return false;
}
-function claimNext(exclude: Set<string>): string | null {
+async function claimNext(exclude: Set<string>): Promise<string | null> {
const slots = listSlots(store);
const ordered = [...slots.filter((s) => !slotFlagged(s, store)), ...slots.filter((s) => slotFlagged(s, store))];
for (const slot of ordered) {
if (exclude.has(slot)) continue;
- if (!claimSlotLock(slot, store)) continue;
+ try {
+ if (!(await claimSlotPorts(slot, store))) continue;
+ } catch (error) {
+ out(`skip ${slot}: ${errMessage(error)}`);
+ exclude.add(slot);
+ continue;
+ }
if (!slotValid(slot, store)) {
releaseSlotLock(slot, store);
out(`skip ${slot}: tokens missing/invalid in ${join(store, `${slot}.env`)}`);
@@ -189,22 +196,6 @@ function claimNext(exclude: Set<string>): string | null {
return null;
}
-// Slackless instances need only a port/lock slot, not a provisioned Slack app.
-// Prefer slot numbers with no poolN.env so a browser-only instance never squats
-// a slot a Slack-enabled worktree could use; fall back to configured ones.
-const MAX_PORT_SLOTS = 16; // slotPorts spaces port families 16 apart
-
-function claimPortSlot(exclude: Set<string>): string | null {
- const configured = new Set(listSlots(store));
- const all = Array.from({ length: MAX_PORT_SLOTS }, (_, i) => `pool${i + 1}`);
- const ordered = [...all.filter((s) => !configured.has(s)), ...all.filter((s) => configured.has(s))];
- for (const slot of ordered) {
- if (exclude.has(slot)) continue;
- if (claimSlotLock(slot, store)) return slot;
- }
- return null;
-}
-
function renderPhase(e: BootPhaseEvent): void {
if (opts.json || e.event !== "phase") return;
let mark = "…";
@@ -381,11 +372,11 @@ async function cmdUp(): Promise<number> {
const excluded = new Set<string>();
const waitMax = Number(process.env.DEV_INSTANCE_WAIT || 120);
- const claim = (): string | null => (withSlack ? claimNext(excluded) : claimPortSlot(excluded));
+ const claim = (): Promise<string | null> => (withSlack ? claimNext(excluded) : claimPortSlot(excluded, store));
const maxAttempts = withSlack ? Math.max(1, listSlots(store).length) : 1;
for (let attempt = 1; attempt <= maxAttempts; attempt++) {
- let slot = claim();
- if (!slot && (await reclaimReclaimable())) slot = claim();
+ let slot = await claim();
+ if (!slot && (await reclaimReclaimable())) slot = await claim();
if (!slot && waitMax > 0 && attempt === 1) {
out("");
out(
@@ -399,8 +390,8 @@ async function cmdUp(): Promise<number> {
await sleep(5000);
waited += 5;
await reapStale();
- if (await reclaimReclaimable()) slot = claim();
- if (!slot) slot = claim();
+ if (await reclaimReclaimable()) slot = await claim();
+ if (!slot) slot = await claim();
}
}
if (!slot) {
@@ -506,22 +497,27 @@ async function cmdStatus(): Promise<number> {
);
for (const slot of known) {
const lock = lockDir(slot, store);
- const ports = slotPorts(slot);
const flag = readSlotFlag(slot, store);
- if (!existsSync(lock)) {
- rows.push({ slot, state: flag && slotFlagged(slot, store) ? `flagged(${flag.reason})` : "free", ports });
- continue;
- }
const lease = leases.find((l) => l.slot === slot);
- if (!lease) continue;
- const sock = resolveSocketPath(lock);
- if (await supervisorReachable(sock)) {
- const res = await supervisorRequest(sock, "GET", "/status", undefined, 5000).catch(() => null);
+ if (lease && (await supervisorReachable(resolveSocketPath(lock)))) {
+ const res = await supervisorRequest(resolveSocketPath(lock), "GET", "/status", undefined, 5000).catch(() => null);
if (res?.body) {
rows.push({ ...res.body, mine: res.body.worktree === worktree, flag });
continue;
}
}
+ let ports;
+ try {
+ ports = slotPorts(slot);
+ } catch (error) {
+ rows.push({ slot, state: "invalid", reason: errMessage(error), worktree: lease?.meta.worktree });
+ continue;
+ }
+ if (!existsSync(lock)) {
+ rows.push({ slot, state: flag && slotFlagged(slot, store) ? `flagged(${flag.reason})` : "free", ports });
+ continue;
+ }
+ if (!lease) continue;
const meta = lease.meta;
const alive = CHILD_ORDER.filter((n) => pidAlive(readPidFile(lock, `${n}.pid`)));
let state = "dead";
@@ -570,17 +566,18 @@ async function cmdStatus(): Promise<number> {
[
String(r.slot).padEnd(7),
state.padEnd(18),
- `${ports.core}/${ports.web}/${ports.admin}/${ports.portal}`.padEnd(21),
+ (ports ? `${ports.core}/${ports.web}/${ports.admin}/${ports.portal}` : "-").padEnd(21),
age.padEnd(8),
(r.mine ? "this" : "").padEnd(5),
String(r.branch ?? "-").padEnd(28),
- String(r.worktree ?? "(available)"),
+ String(r.reason ?? r.worktree ?? "(available)"),
].join(" "),
);
}
- const taken = rows.filter((r) => r.state !== "free" && !String(r.state).startsWith("flagged")).length;
+ const invalid = rows.filter((r) => r.state === "invalid").length;
+ const free = rows.filter((r) => r.state === "free" || String(r.state).startsWith("flagged")).length;
console.log("");
- console.log(`${taken} taken / ${rows.length - taken} free / ${rows.length} slots total`);
+ console.log(`${rows.length - free - invalid} taken / ${free} free / ${invalid} invalid / ${rows.length} known slots`);
console.log("live = supervised + verified. Reclaim never touches a slot with a fresh supervisor heartbeat.");
return EXIT.ok;
}
diff --git a/scripts/dev/commands/doctor.ts b/scripts/dev/commands/doctor.ts
index 539a62081..ed6016170 100644
--- a/scripts/dev/commands/doctor.ts
+++ b/scripts/dev/commands/doctor.ts
@@ -7,6 +7,7 @@ import { callerEnvSnapshot, gitHead, repoRoot } from "../lib/envctx.ts";
import { portHolders } from "../lib/proc.ts";
import { resolveSocketPath, supervisorReachable, supervisorRequest } from "../lib/client.ts";
import { EXIT, CHILD_ORDER } from "../lib/types.ts";
+import { errMessage } from "../lib/util.ts";
import type { StatusReport } from "../lib/types.ts";
interface Check {
@@ -29,9 +30,22 @@ export async function runDoctor(opts: { json: boolean; fix: boolean; store: stri
})();
const mine = worktree ? myLease(worktree, opts.store) : null;
- // A browser-only instance (--no-slack, or a live lease booted that way) needs no pool app.
const needsPool = opts.slack && mine?.meta.slack !== "0";
const slots = listSlots(opts.store);
+ const validSlots = new Map<string, ReturnType<typeof slotPorts>>();
+ for (const slot of slots) {
+ try {
+ validSlots.set(slot, slotPorts(slot));
+ } catch (error) {
+ checks.push({
+ id: `slot-config:${slot}`,
+ ok: false,
+ severity: "warn",
+ detail: errMessage(error),
+ remedy: `rename or remove ${slot}.env so its slot number fits the configured port range`,
+ });
+ }
+ }
let poolDetail = "Slack off -- no pool slot needed";
if (needsPool) {
poolDetail = slots.length ? `${slots.length} pool slot(s) configured` : "no poolN.env files in the pool store";
@@ -178,10 +192,10 @@ export async function runDoctor(opts: { json: boolean; fix: boolean; store: stri
}
if (worktree) {
- for (const slot of slots) {
+ for (const [slot, ports] of validSlots) {
const lease = listLeases(opts.store).find((l) => l.slot === slot);
if (lease) continue;
- for (const [name, port] of Object.entries(slotPorts(slot))) {
+ for (const [name, port] of Object.entries(ports)) {
if (name === "supervisor" || name === "prodProxy" || name === "slackHealth") continue;
const holders = portHolders(port);
if (holders.length) {
@@ -190,7 +204,7 @@ export async function runDoctor(opts: { json: boolean; fix: boolean; store: stri
ok: false,
severity: "warn",
detail: `free slot ${slot}'s ${name} port ${port} is held by pid(s) ${holders.join(",")}`,
- remedy: `kill ${holders.join(" ")}`,
+ remedy: "leave unrelated listeners running; dev up skips occupied port blocks",
});
}
}
diff --git a/scripts/dev/lib/lease.ts b/scripts/dev/lib/lease.ts
index aa6a164f5..85034e0f8 100644
--- a/scripts/dev/lib/lease.ts
+++ b/scripts/dev/lib/lease.ts
@@ -1,8 +1,8 @@
import { existsSync, mkdirSync, readFileSync, renameSync, rmSync, writeFileSync } from "node:fs";
import { readdirSync } from "node:fs";
import { join } from "node:path";
-import { leasesDir, poolStore } from "./pool.ts";
-import { pidAlive } from "./proc.ts";
+import { leasesDir, listSlots, poolStore, portSlotCount, slotPorts } from "./pool.ts";
+import { pidAlive, portAvailable } from "./proc.ts";
import { fileMtimeEpoch, formatAge, nowEpoch, todayYmd } from "./util.ts";
import type { LeaseInfo } from "./types.ts";
@@ -23,6 +23,31 @@ export function claimSlotLock(slot: string, store = poolStore()): boolean {
}
}
+export async function claimSlotPorts(slot: string, store = poolStore()): Promise<boolean> {
+ const ports = slotPorts(slot);
+ if (!claimSlotLock(slot, store)) return false;
+ let available = false;
+ try {
+ available = (await Promise.all(Object.values(ports).map(portAvailable))).every(Boolean);
+ return available;
+ } finally {
+ if (!available) releaseSlotLock(slot, store);
+ }
+}
+
+export async function claimPortSlot(exclude: Set<string>, store = poolStore()): Promise<string | null> {
+ const configured = new Set(listSlots(store));
+ const capacity = portSlotCount();
+ for (const useConfigured of [false, true]) {
+ for (let num = 1; num <= capacity; num++) {
+ const slot = `pool${num}`;
+ if (configured.has(slot) !== useConfigured || exclude.has(slot)) continue;
+ if (await claimSlotPorts(slot, store)) return slot;
+ }
+ }
+ return null;
+}
+
export function releaseSlotLock(slot: string, store = poolStore()): void {
rmSync(lockDir(slot, store), { recursive: true, force: true });
}
diff --git a/scripts/dev/lib/pool.ts b/scripts/dev/lib/pool.ts
index 76db8be2f..6b263e54a 100644
--- a/scripts/dev/lib/pool.ts
+++ b/scripts/dev/lib/pool.ts
@@ -66,16 +66,35 @@ export function slotValid(slot: string, store = poolStore()): boolean {
return t.botToken.startsWith("xoxb-") && t.appToken.startsWith("xapp-");
}
+const LEGACY_PORT_SLOTS = 16;
+const PORTS_PER_SLOT = 7;
+
+export function portSlotCount(basePort = Number(process.env.DEV_INSTANCE_BASE_PORT || 8080)): number {
+ if (!Number.isInteger(basePort) || basePort < 0 || basePort > 65535) {
+ throw new Error("DEV_INSTANCE_BASE_PORT must be an integer between 0 and 65535");
+ }
+ const available = 65535 - basePort;
+ return available >= LEGACY_PORT_SLOTS * PORTS_PER_SLOT
+ ? Math.floor(available / PORTS_PER_SLOT)
+ : Math.max(0, available - LEGACY_PORT_SLOTS * (PORTS_PER_SLOT - 1));
+}
+
export function slotPorts(slot: string, basePort = Number(process.env.DEV_INSTANCE_BASE_PORT || 8080)): SlotPorts {
const num = Number(slot.replace(/^pool/, ""));
+ if (!/^pool[1-9][0-9]*$/.test(slot) || !Number.isSafeInteger(num) || num > portSlotCount(basePort)) {
+ throw new Error(`No valid port block for ${slot} with base port ${basePort}`);
+ }
+ const legacy = num <= LEGACY_PORT_SLOTS;
+ const first = legacy ? basePort + num : basePort + (num - 1) * PORTS_PER_SLOT + 1;
+ const stride = legacy ? LEGACY_PORT_SLOTS : 1;
return {
- core: basePort + num,
- web: basePort + 16 + num,
- admin: basePort + 32 + num,
- portal: basePort + 48 + num,
- prodProxy: basePort + 64 + num,
- slackHealth: basePort + 80 + num,
- supervisor: basePort + 96 + num,
+ core: first,
+ web: first + stride,
+ admin: first + 2 * stride,
+ portal: first + 3 * stride,
+ prodProxy: first + 4 * stride,
+ slackHealth: first + 5 * stride,
+ supervisor: first + 6 * stride,
};
}
diff --git a/scripts/dev/lib/proc.ts b/scripts/dev/lib/proc.ts
index e50817227..3f55c0ae1 100644
--- a/scripts/dev/lib/proc.ts
+++ b/scripts/dev/lib/proc.ts
@@ -1,6 +1,6 @@
import { spawn, spawnSync } from "node:child_process";
import { openSync } from "node:fs";
-import { connect } from "node:net";
+import { connect, createServer } from "node:net";
import { bestEffort, sleep } from "./util.ts";
export function run(
@@ -118,10 +118,18 @@ export async function waitPortFree(port: number, timeoutMs = 10_000): Promise<bo
return false;
}
-export async function freePort(port: number, label: string, log: (msg: string) => void): Promise<void> {
- const holders = portHolders(port);
- if (holders.length === 0) return;
- log(`freeing stale process(es) ${holders.join(",")} on :${port} (${label})`);
- for (const pid of holders) await killTree(pid, 3000);
- await waitPortFree(port, 5000);
+export async function portAvailable(port: number): Promise<boolean> {
+ for (const host of ["127.0.0.1", "0.0.0.0", "::1", "::"]) {
+ const available = await new Promise<boolean>((resolve) => {
+ const server = createServer();
+ server.once("error", (error: NodeJS.ErrnoException) => {
+ resolve(host.includes(":") && ["EAFNOSUPPORT", "EADDRNOTAVAIL"].includes(error.code ?? ""));
+ });
+ server.listen({ port, host, ipv6Only: host.includes(":"), exclusive: true }, () =>
+ server.close(() => resolve(true)),
+ );
+ });
+ if (!available) return false;
+ }
+ return true;
}
diff --git a/scripts/dev/supervisor/children.ts b/scripts/dev/supervisor/children.ts
index 5c81dbcff..79a2b45f1 100644
--- a/scripts/dev/supervisor/children.ts
+++ b/scripts/dev/supervisor/children.ts
@@ -2,7 +2,7 @@ import { spawn, type ChildProcess } from "node:child_process";
import { openSync, readFileSync, statSync } from "node:fs";
import { join } from "node:path";
import { writePidFile } from "../lib/lease.ts";
-import { freePort, tcpPortOpen, waitPortFree } from "../lib/proc.ts";
+import { tcpPortOpen, waitPortFree } from "../lib/proc.ts";
import { bestEffort, bestEffortValue, sleep } from "../lib/util.ts";
import type { ChildSpec, ChildState, ChildStatus } from "../lib/types.ts";
@@ -47,7 +47,6 @@ export class Child {
async start(): Promise<{ ok: boolean; detail?: string }> {
this.stopRequested = false;
- if (this.spec.port) await freePort(this.spec.port, this.spec.name, this.log);
this.state = "starting";
this.lastSpawnAt = Date.now();
const [cmd, ...rest] = this.spec.argv;
diff --git a/scripts/dev/supervisor/main.ts b/scripts/dev/supervisor/main.ts
index 441d9c4d6..08f5e2d7d 100644
--- a/scripts/dev/supervisor/main.ts
+++ b/scripts/dev/supervisor/main.ts
@@ -521,6 +521,12 @@ function finishBoot(): void {
function startLoops(): void {
const heartbeat = setInterval(() => {
+ if (shuttingDown) return;
+ if (!existsSync(worktree)) {
+ log("worktree vanished -- shutting down and releasing the lease");
+ void shutdownSelf(true);
+ return;
+ }
if (!existsSync(lock)) {
log("lease directory vanished (external teardown) -- exiting");
void shutdownSelf(false);
@@ -600,6 +606,7 @@ async function teardown(reason: string): Promise<void> {
}
async function shutdownSelf(removeLock: boolean): Promise<void> {
+ if (shuttingDown) return;
await teardown("shutdown requested");
if (removeLock) rmSync(lock, { recursive: true, force: true });
process.exit(EXIT.ok);
diff --git a/test/dev-cli-lib.test.ts b/test/dev-cli-lib.test.ts
index e84fff3ca..ec022746a 100644
--- a/test/dev-cli-lib.test.ts
+++ b/test/dev-cli-lib.test.ts
@@ -1,6 +1,7 @@
import { test } from "node:test";
import assert from "node:assert/strict";
-import { chmodSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
+import { chmodSync, existsSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs";
+import { createServer } from "node:net";
import { spawnSync } from "node:child_process";
import { ensureDeps } from "../scripts/dev/lib/deps.ts";
import { tmpdir } from "node:os";
@@ -10,6 +11,7 @@ import {
clearSlotFlag,
ensureStore,
listSlots,
+ portSlotCount,
readSlotFlag,
slotFlagged,
slotPorts,
@@ -18,6 +20,8 @@ import {
writeSlotFlag,
} from "../scripts/dev/lib/pool.ts";
import {
+ claimPortSlot,
+ claimSlotPorts,
claimSlotLock,
heartbeatFresh,
leaseOrgId,
@@ -68,6 +72,107 @@ test("slotPorts derive the full port block from the slot number", () => {
});
});
+test("port blocks preserve legacy slots and stay disjoint through the port ceiling", () => {
+ for (const base of [0, 8080, 20000, 65423, 65424, 65535]) {
+ const used = new Set<number>();
+ const count = portSlotCount(base);
+ for (let num = 1; num <= count; num++) {
+ const ports = Object.values(slotPorts(`pool${num}`, base));
+ for (const [index, port] of ports.entries()) {
+ assert.ok(port > base && port <= 65535);
+ assert.ok(!used.has(port), `port ${port} overlaps at slot ${num}`);
+ used.add(port);
+ if (num <= 16) assert.equal(port, base + num + index * 16);
+ }
+ }
+ assert.throws(() => slotPorts(`pool${count + 1}`, base), /No valid port block/);
+ }
+ assert.equal(portSlotCount(8080), 8207);
+ assert.equal(slotPorts("pool17", 8080).core, 8193);
+});
+
+test("port allocation rejects malformed slots and invalid base ports", () => {
+ for (const slot of ["pool0", "pool-1", "pool1.5", "pool01", "17", "poolNaN", "pool9999999999999999999"]) {
+ assert.throws(() => slotPorts(slot, 8080), /No valid port block/);
+ }
+ for (const base of [-1, 65536, 8080.5, NaN, Infinity]) {
+ assert.throws(() => portSlotCount(base), /DEV_INSTANCE_BASE_PORT/);
+ assert.throws(() => slotPorts("pool1", base), /DEV_INSTANCE_BASE_PORT/);
+ }
+});
+
+test("web leases grow beyond sixteen, preserve Slack capacity, and reuse released slots", async () => {
+ const store = tmpStore();
+ try {
+ addSlot(store, 1);
+ const excluded = new Set(["pool2"]);
+ const claimed: string[] = [];
+ for (let num = 3; num <= 40; num++) {
+ const slot = await claimPortSlot(excluded, store);
+ assert.ok(slot);
+ assert.ok(Number(slot.slice(4)) >= num);
+ claimed.push(slot);
+ }
+ assert.equal(new Set(claimed).size, 38);
+ assert.equal(claimSlotLock("pool1", store), true);
+ const released = claimed[16]!;
+ releaseSlotLock(released, store);
+ assert.equal(await claimPortSlot(excluded, store), released);
+ const exhausted = new Set(Array.from({ length: portSlotCount() }, (_, i) => `pool${i + 1}`));
+ assert.equal(await claimPortSlot(exhausted, store), null);
+ exhausted.delete(released);
+ releaseSlotLock(released, store);
+ addSlot(store, Number(released.slice(4)));
+ assert.equal(await claimPortSlot(exhausted, store), released);
+ } finally {
+ rmSync(store, { recursive: true, force: true });
+ }
+});
+
+test("occupied port blocks are skipped without killing listeners or retaining leases", async () => {
+ const store = tmpStore();
+ const server = createServer();
+ const previousBase = process.env.DEV_INSTANCE_BASE_PORT;
+ await new Promise<void>((resolve) => server.listen(0, "127.0.0.1", resolve));
+ const port = (server.address() as { port: number }).port;
+ process.env.DEV_INSTANCE_BASE_PORT = String(port - 97);
+ try {
+ assert.equal(await claimSlotPorts("pool1", store), false);
+ assert.equal(existsSync(lockDir("pool1", store)), false);
+ const claimed = await claimPortSlot(new Set(), store);
+ assert.notEqual(claimed, "pool1");
+ assert.equal(server.listening, true);
+ await assert.rejects(claimSlotPorts("pool999999", store), /No valid port block/);
+ assert.equal(existsSync(lockDir("pool999999", store)), false);
+ } finally {
+ if (previousBase === undefined) delete process.env.DEV_INSTANCE_BASE_PORT;
+ else process.env.DEV_INSTANCE_BASE_PORT = previousBase;
+ await new Promise<void>((resolve) => server.close(() => resolve()));
+ rmSync(store, { recursive: true, force: true });
+ }
+});
+
+test("status and doctor report invalid configured slots and continue", () => {
+ const store = tmpStore();
+ try {
+ addSlot(store, 1);
+ addSlot(store, 999999);
+ const env = { ...process.env, QM_POOL_STORE: store, DEV_INSTANCE_BASE_PORT: "8080" };
+ const status = spawnSync(process.execPath, ["scripts/dev/cli.ts", "status", "--json"], { encoding: "utf8", env });
+ assert.equal(status.status, 0, status.stderr);
+ const rows = JSON.parse(status.stdout);
+ assert.equal(rows.find((row: { slot: string }) => row.slot === "pool1").state, "free");
+ assert.equal(rows.find((row: { slot: string }) => row.slot === "pool999999").state, "invalid");
+ const doctor = spawnSync(process.execPath, ["scripts/dev/cli.ts", "doctor", "--json"], { encoding: "utf8", env });
+ assert.equal(doctor.status, 0, doctor.stderr);
+ const report = JSON.parse(doctor.stdout);
+ assert.ok(report.checks.some((check: { id: string }) => check.id === "slot-config:pool999999"));
+ assert.ok(report.checks.some((check: { id: string }) => check.id === "docker-daemon"));
+ } finally {
+ rmSync(store, { recursive: true, force: true });
+ }
+});
+
test("pool listing, token parsing, and validity", () => {
const store = tmpStore();
addSlot(store, 2, "CANARY_CHANNEL=C0TEST\n");
diff --git a/test/dev-supervisor-child.test.ts b/test/dev-supervisor-child.test.ts
index d70ed35fa..f5724abd7 100644
--- a/test/dev-supervisor-child.test.ts
+++ b/test/dev-supervisor-child.test.ts
@@ -5,7 +5,7 @@ import { connect, createServer } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { Child } from "../scripts/dev/supervisor/children.ts";
-import { portHolders, tcpPortOpen } from "../scripts/dev/lib/proc.ts";
+import { portAvailable, portHolders, tcpPortOpen } from "../scripts/dev/lib/proc.ts";
import { sleep } from "../scripts/dev/lib/util.ts";
import type { ChildSpec } from "../scripts/dev/lib/types.ts";
@@ -75,29 +75,50 @@ test("a crashed child is auto-restarted with backoff", async () => {
rmSync(lock, { recursive: true, force: true });
});
-test("start frees a squatted port before spawning (the EADDRINUSE fix)", async () => {
+test("startup leaves occupied ports untouched and retries after they become free", async () => {
const lock = mkdtempSync(join(tmpdir(), "qm-child-"));
const port = await freeTcpPort();
- const squatter = new Child(
- spec(lock, port, ["--ignoreTerm"]),
- lock,
- () => {},
- () => {},
- );
- assert.equal((await squatter.start()).ok, true);
- squatter.proc?.removeAllListeners("exit");
-
+ const server = createServer();
+ await new Promise<void>((resolve) => server.listen(port, "127.0.0.1", resolve));
const child = new Child(
spec(lock, port),
lock,
() => {},
() => {},
);
- const res = await child.start();
- assert.equal(res.ok, true, res.detail);
- assert.equal(await child.probeHealth(), true);
- await child.stop();
- rmSync(lock, { recursive: true, force: true });
+ try {
+ assert.equal(await portAvailable(port), false);
+ const res = await child.start();
+ assert.equal(res.ok, false);
+ assert.match(res.detail ?? "", /exited during startup/);
+ assert.equal(child.proc, null);
+ assert.equal(child.status().state, "crashed");
+ assert.equal(server.listening, true);
+ assert.equal(await tcpPortOpen(port), true);
+ await new Promise<void>((resolve) => server.close(() => resolve()));
+ const deadline = Date.now() + 5000;
+ while (Date.now() < deadline && child.state !== "healthy") await sleep(100);
+ assert.equal(child.state, "healthy");
+ assert.ok(child.restarts >= 1);
+ } finally {
+ if (server.listening) await new Promise<void>((resolve) => server.close(() => resolve()));
+ await child.stop();
+ rmSync(lock, { recursive: true, force: true });
+ }
+});
+
+test("port availability rejects IPv6 listeners", async () => {
+ const server = createServer();
+ await new Promise<void>((resolve, reject) => {
+ server.once("error", reject);
+ server.listen({ port: 0, host: "::1", ipv6Only: true }, resolve);
+ });
+ try {
+ assert.equal(await portAvailable((server.address() as { port: number }).port), false);
+ assert.equal(server.listening, true);
+ } finally {
+ await new Promise<void>((resolve) => server.close(() => resolve()));
+ }
});
test("stop escalates to SIGKILL for a TERM-ignoring child", async () => {