Autonomous development for real products. Governed agents build from specs you approve, follow the rules you set, and nothing merges until another agent proves it. Your decisions, judgement and taste are the bar that work must clear. Declare your vision and let agents build it.
JavaScript agentic-codingai-agentsai-governanceautonomous-agentsclaude-codeclicode-reviewcodexcoding-agentsdeveloper-toolsgit-hooksllmlocal-firstmulti-agentsoftware-factoryspec-driven-developmentverificationvibe-coding
fix(gate): bound runner admission and name overdue runs [C7,Q1,Q2]
Corey Olson1bb772ab0dfa
diff --git a/docs/api.md b/docs/api.mdindex 3c1e99b5..9030badf 100644--- a/docs/api.md+++ b/docs/api.md@@ -22,6 +22,8 @@ Release priority is separately opt-in with `"gate": { "queue": { "currentReleaseThe memory hold is opt-in per machine with `"gate": { "memoryFloorMB": <n> }` in `~/.pullboard/settings.json`; absent, it is off and the pool admits by slots alone, as before. When set, before admitting a run the machine gate pool reads the machine's memory: on macOS the kernel's memory pressure level and free percentage, on Linux `MemAvailable` and the pressure stall averages in `/proc/pressure/memory`. Only runs executing on this machine, including local landing and short-slot runs, wait while pressure is critical or free memory is under that many megabytes (`0` leaves only the pressure check). A compatible runner-bound run bypasses this local hold and waits for runner capacity instead; it does not acquire a local executor while the hold applies. A held run prints one line naming why and the three biggest processes by memory, with pid, size and working directory, and starts once memory recovers. While the floor is set, `pullboard resources` prints the same line and returns it as `memory` in JSON; with no floor it reads no memory and `memory` is `null`. Where memory cannot be read, the pool admits by slots alone and prints one warning that memory is not checked there. `pullboard resources` marks locally memory-held queue entries with `memoryHeld: true`. `pullboard status --json` adds optional `memoryHeld`, an array of current held runs (`agent`, `repo`, `item`, `kind`, `since`), empty when no runs are held or the floor is off, and `null` if that read is unavailable. Text status names each held run. This is a read-only resource-queue projection; it never migrates the pool.+Runner admission has a finite deadline: `pullboard settings gate.runnerWaitMs <n>` stores positive whole milliseconds (default `300000`, five minutes). The deadline starts at the original queue request and survives transport-loss requeueing. A registered runner that never answers or never acquires its worker slot is canceled at that deadline; an admitted command keeps its existing test timeout. After the deadline the run stops retrying runners and waits for ordinary local admission, preserving its ticket, age, queue priority, browser/timing requirements and local memory protection. Disabled local capacity and runner-only work refuse with `RUNNER_WAIT_DEADLINE`; the pool never invents a local slot. Every overdue poll reports its local admission branch and each runner's observed reason (transport loss, cleanup/draining, missing capability, or busy capacity). `resources` adds `runnerDeadline` to overdue waiters. `status --json` adds `runnerDeadlines` (current runs, or `null` when unavailable), and text status names those runs. Status reads this projection without opening a write transaction or migrating an old pool.+Short gate slots are opt-in per machine with `pullboard settings gate.shortSlots <n>`, stored as `"gate": { "shortSlots": n }`; the default `0` keeps them off and the pool behaves as before. With `n` above `0`, up to `n` runs that name their test files (a `pullboard check` or a review whose check is a literal `node bin/run-tests.js` call over test files, or a direct `node bin/run-tests.js <files>`) are admitted on short slots while every gate slot is held. Whole suites, directories, landings, diagnostics and anything the parser cannot read never use them, and a child that does not name its files cannot reuse a short lease (`SHORT_SLOT_FULL_SUITE`). Short slots do not add to `gateSlots`, and browser runs still count against `browserSlots`. `pullboard settings` prints `gate.shortSlots`; `pullboard resources --json` reports `shortSlots`, each gate holder's `slot` (`gate` or `short`), and `requestedSlot: "short-eligible"` on a waiting run that may take one.When `add` or `edit` supplies a new nonempty check, it records the item before measuring that check in a background temporary checkout of the captured `main` commit. Use `--wait` to wait for the measurement before returning, or `args.wait: true` for an HTTP add/edit move. The item returned by `add`, `edit`, `show`, and `next --verify` may include `item_check_baseline`: `{command, main, result, request?, seconds?, reason?, warning?}`. `main` is the commit id or `null`, and `result` is `pending`, `green`, `red`, or `unavailable`. A background request carries an opaque `request` identity; an older completion cannot overwrite a newly edited or cleared check.diff --git a/src/commands/resources.js b/src/commands/resources.jsindex d0e4f39a..080c94ff 100644--- a/src/commands/resources.js+++ b/src/commands/resources.js@@ -6,6 +6,7 @@ import {readMachineMemory,runnerResourceStatus,} from '../resources.js';+import { runnerDeadlineLine } from '../runner-wait.js';import {gateQueueSettings,gateShortSlots,@@ -46,5 +47,8 @@ export async function run(io) {resource.line.forEach((waiter, index) => io.say(` ${index + 1}. waiting: ${agentDisplayName(agents, ctx.config, waiter.agent)}${waiter.repo ? ` in ${waiter.repo}` : ''}${resourceRunLabel(waiter, shortSlots)}; class ${waiter.queueClass}${waiter.agedFrom ? `, aged up from ${waiter.agedFrom}` : ''}${waiter.memoryHeld ? '; held by local memory' : ''}`));}for (const runner of runners) io.say(`runner ${runner.host}: ${runner.inUse}/${runner.slots} slots, ${runner.browserInUse}/${runner.browser} browser, GUI ${runner.gui === true ? 'yes' : runner.gui === false ? 'no' : 'unknown'}${runner.unavailable ? ' (unavailable; re-register after checking the host)' : ''}`);+ for (const waiter of resources.find(resource => resource.name === 'gate' && resource.scope === 'machine')?.line ?? []) {+ if (waiter.runnerDeadline) io.say(`${waiter.agent}${waiter.item ? ` #${waiter.item}` : ''}: ${runnerDeadlineLine(waiter.runnerDeadline)}`);+ }return 0;}diff --git a/src/commands/settings.js b/src/commands/settings.jsindex 170faac7..7cb309a7 100644--- a/src/commands/settings.js+++ b/src/commands/settings.js@@ -9,6 +9,7 @@ import {import {gateQueueSettings,gateShortSlots,+ gateRunnerWaitMs,loadMachineSettings,memoryFloorMB,relayOfferEnabled,@@ -16,6 +17,7 @@ import {setGateShortSlots,setGateSlots,setGateRunnersFirst,+ setGateRunnerWaitMs,setLandingPool,} from '../settings.js';import {@@ -43,7 +45,7 @@ export function settingsCommand(io, { first, second, rest, values }) {const offerAtInit = relayOfferEnabled();const shortSlots = gateShortSlots();const runnersFirst = settings.gate?.runnersFirst === true;- io.result?.({ settings: { ...settings, gate: { ...settings.gate, queue: { ...settings.gate?.queue, ...queue }, shortSlots, runnersFirst }, relay: { ...settings.relay, offerAtInit } }, memoryFloorMB: memoryFloorMB() });+ io.result?.({ settings: { ...settings, gate: { ...settings.gate, queue: { ...settings.gate?.queue, ...queue }, shortSlots, runnersFirst, runnerWaitMs: gateRunnerWaitMs() }, relay: { ...settings.relay, offerAtInit } }, memoryFloorMB: memoryFloorMB() });io.say(`gateSlots: ${settings.gateSlots}`);io.say(`browserSlots: ${settings.browserSlots}`);io.say(`gate.queue.reviewsFirst: ${queue.reviewsFirst}${queue.reviewsFirst ? '' : ' (landings first, then arrival order)'}`);@@ -54,6 +56,7 @@ export function settingsCommand(io, { first, second, rest, values }) {io.say(`relay.offerAtInit: ${offerAtInit}${offerAtInit ? ' (init asks once to share your boards with your phone)' : ' (init stays quiet about sharing)'}`);io.say(`gate.shortSlots: ${shortSlots}${shortSlots ? '' : ' (off; runs that name their test files wait for a gate slot)'}`);io.say(`gate.runnersFirst: ${runnersFirst} (registered runners are preferred before local slots when on)`);+ io.say(`gate.runnerWaitMs: ${gateRunnerWaitMs()} (then ordinary local admission; no disabled capacity is created)`);return 0;}if (!SETTING_FORMS.includes(first) || !second || rest.length) {@@ -66,6 +69,13 @@ export function settingsCommand(io, { first, second, rest, values }) {io.say(`gate.runnersFirst set to ${second}`);return 0;}+ if (first === 'gate.runnerWaitMs') {+ if (!/^[1-9]\d*$/u.test(second)) throw new Refused('BAD_MACHINE_SETTINGS', 'gate.runnerWaitMs must be positive whole milliseconds; run pullboard settings gate.runnerWaitMs <n>');+ const settings = setGateRunnerWaitMs(Number(second));+ io.result?.({ settings });+ io.say(`gate.runnerWaitMs set to ${settings.gate.runnerWaitMs}`);+ return 0;+ }if (first === 'gate.shortSlots') {if (!/^(?:0|[1-9]\d*)$/u.test(second) || !Number.isSafeInteger(Number(second))) {throw new Refused('BAD_SHORT_SLOTS', 'gate.shortSlots must be a whole number, 0 or more; run pullboard settings gate.shortSlots <n>');diff --git a/src/commands/status.js b/src/commands/status.jsindex 7a4e0db8..c5e3ba27 100644--- a/src/commands/status.js+++ b/src/commands/status.js@@ -7,6 +7,10 @@ import { relayStateExists } from '../local-state.js';import { unlinkedRelayStatus } from '../relay-status-fields.js';import { ruleProofCounts } from '../rule-proof-counts.js';import { memoryHeldRuns } from '../memory-held.js';+import {+ runnerDeadlineLine,+ runnerDeadlineRuns,+} from '../runner-wait.js';import {reviewQueueLine,} from '../next-steps.js';@@ -43,6 +47,7 @@ export async function run(io, { values }) {submitInvariantProposals: submitInvariantProposals(store.submitInvariantEvents(board, ctx.config), ctx.config) };});summary.memoryHeld = memoryHeldRuns();+ summary.runnerDeadlines = runnerDeadlineRuns();if (values.json) {io.result(summary);return 0;@@ -57,6 +62,8 @@ export async function run(io, { values }) {io.say(reviewQueueLine(ctx, summary.reviewQueue, span));if (summary.memoryHeld === null) io.say('local memory hold status unavailable; pullboard resources shows the queue');else for (const run of summary.memoryHeld) io.say(`local memory holds ${run.agent}${run.item ? ` #${run.item}` : ''}${run.kind ? ` (${run.kind})` : ''}${run.repo ? ` in ${run.repo}` : ''}`);+ if (summary.runnerDeadlines === null) io.say('runner wait deadline status unavailable; pullboard resources shows the queue');+ else for (const run of summary.runnerDeadlines) io.say(`${run.agent}${run.item ? ` #${run.item}` : ''}${run.repo ? ` in ${run.repo}` : ''}: ${runnerDeadlineLine(run)}`);for (const proposal of summary.submitInvariantProposals) io.say(submitInvariantLine(proposal));for (const need of summary.needs.filter(need => need.kind === 'agent-approved')) io.say(`${need.text}; ${need.what}`);if (!summary.relay.linked && !originRepository(ctx.info.root, { optional: true })) {diff --git a/src/gate.js b/src/gate.jsindex 095ae14a..342a787b 100644--- a/src/gate.js+++ b/src/gate.js@@ -75,6 +75,7 @@ import {import {browserFreeChecksEnabled,gateRunnersFirstEnabled,+ gateRunnerWaitMs,gateQueueSettings,gateShortSlots,loadMachineSettings,@@ -235,7 +236,7 @@ export function runProfiledShell(root, command, { waitMs = 0, artifactDirectory,};// The registered worker returns the same complete stage/component accounting as a local run.gateLease?.tracker?.mark('runAt');- const run = gateLease?.runner ? { ...runOnRunner(root, command, { runner: gateLease.runner, source: gateLease.source,+ const run = gateLease?.runner ? { ...runOnRunner(root, command, { runner: gateLease.runner, source: gateLease.source, runnerWaitDeadline: gateLease.runnerWaitDeadline,browser: gateLease.browser, pipefail, stages, waivers: landing ? waivers : [], metadata: { ...(gateLease.runMetadata ?? {}), landing, ...(timeoutMs ? { timeoutMs } : {}) } }) }: runShell(root, command, { pipefail, env: inherited, stages, timeoutMs });gateLease?.tracker?.mark('ranAt');@@ -267,7 +268,7 @@ export async function runProfiledShellAsync(root, command, { waitMs = 0, gateSloconst started = Date.now();gateLease?.tracker?.mark('runAt');if (gateLease?.runner) {- const run = await runOnRunnerAsync(root, command, { runner: gateLease.runner, source: gateLease.source,+ const run = await runOnRunnerAsync(root, command, { runner: gateLease.runner, source: gateLease.source, runnerWaitDeadline: gateLease.runnerWaitDeadline,browser: gateLease.browser, metadata: { ...(gateLease.runMetadata ?? {}), landing } });gateLease.environment = run.environment ?? null;gateLease.gui = run.executionCapabilities?.gui ?? null;@@ -440,6 +441,7 @@ async function withOneGateSlot(root, action, { landing = false, itemCheck = falsremote: remote && Boolean(source),runnerProvider: () => loadMachineSettings().runners ?? [],runnersFirstProvider: gateRunnersFirstEnabled,+ runnerWaitMs: gateRunnerWaitMs(),releaseContext,memory: floorMB === null ? undefined : memoryAdmission({ floorBytes: () => (memoryFloorMB() ?? floorMB) * 1024 ** 2, read: readMemory }),onWait,diff --git a/src/help.js b/src/help.jsindex d0c4f44a..f8b30a8e 100644--- a/src/help.js+++ b/src/help.js@@ -41,6 +41,7 @@ Set upmemory hold is opt-in: gate.memoryFloorMB in ~/.pullboard/settings.json holds the queue under that many MB free or critical pressure (default off)short slots for runs that name their test files are opt-in: pullboard settings gate.shortSlots <n> (default 0)runner-first dispatch is opt-in: pullboard settings gate.runnersFirst on|off (default off)+ runner admission deadline: pullboard settings gate.runnerWaitMs <n> (default 300000ms)pullboard runner add <ssh-host> --slots <n> --browser <n> [--node <remote-path>] [--root <remote-path>]register SSH test capacity; no runner sign-in; root defaults to ~/.pullboard-runnerpullboard runner list each runner's registered slots and browsers, which it runs at@@ -495,7 +496,7 @@ function overviewHelp(groups, firstRun = false) {}/** The settings forms settingsCommand dispatches with a value; each gets its own help, from this list (N2). */-export const SETTING_FORMS = Object.freeze(['gateSlots', 'browserSlots', 'gate.shortSlots', 'gate.runnersFirst', 'landing.pool']);+export const SETTING_FORMS = Object.freeze(['gateSlots', 'browserSlots', 'gate.shortSlots', 'gate.runnersFirst', 'gate.runnerWaitMs', 'landing.pool']);/** Help for each settings form the printed usage did not already name, so each form is covered (N2). */function settingFormHelp(commands) {diff --git a/src/resources.js b/src/resources.jsindex 1c24f9e4..718663e8 100644--- a/src/resources.js+++ b/src/resources.js@@ -28,6 +28,11 @@ import {} from './machine-home.js';import { Refused } from './refused.js';import { gateQueueSettings } from './queue-settings.js';+import {+ localWaitBranch,+ runnerDeadlineLine,+ runnerWaitReasons,+} from './runner-wait.js';import {ARRIVAL_ORDER,classedMarker,@@ -147,6 +152,9 @@ function openQueue(file) {if (table === 'waiter' && !columns.includes('remote')) db.exec('ALTER TABLE waiter ADD COLUMN remote INTEGER NOT NULL DEFAULT 0');if (table === 'waiter' && !columns.includes('short_eligible')) db.exec('ALTER TABLE waiter ADD COLUMN short_eligible INTEGER NOT NULL DEFAULT 0');if (table === 'waiter' && !columns.includes('memory_held')) db.exec('ALTER TABLE waiter ADD COLUMN memory_held INTEGER NOT NULL DEFAULT 0');+ if (table === 'waiter' && !columns.includes('runner_deadline')) db.exec('ALTER TABLE waiter ADD COLUMN runner_deadline INTEGER NOT NULL DEFAULT 0');+ if (table === 'waiter' && !columns.includes('runner_wait_branch')) db.exec("ALTER TABLE waiter ADD COLUMN runner_wait_branch TEXT NOT NULL DEFAULT ''");+ if (table === 'waiter' && !columns.includes('runner_wait_reasons')) db.exec("ALTER TABLE waiter ADD COLUMN runner_wait_reasons TEXT NOT NULL DEFAULT '[]'");}// Older clients insert the default zero. A database trigger witnesses even short legacy// arrivals between the timing owner's polls; no old client has to understand the new flag.@@ -346,17 +354,18 @@ function queueOrder(scope) {/** Read the complete resource line with the metadata every admission view orders identically. */function queuedWaiters(db, name) {- return db.prepare('SELECT ticket, pid, agent, repo, since, landing, item_check AS itemCheck, diagnostic, browser, gui, remote, item, kind, lease_until AS leaseUntil, classed, release_aware, board_key, release_rank, queue_ticket, short_eligible AS shortEligible, timing, memory_held AS memoryHeld FROM waiter WHERE name = ?').all(name);+ return db.prepare('SELECT ticket, pid, agent, repo, since, landing, item_check AS itemCheck, diagnostic, browser, gui, remote, item, kind, lease_until AS leaseUntil, classed, release_aware, board_key, release_rank, queue_ticket, short_eligible AS shortEligible, timing, memory_held AS memoryHeld, runner_deadline AS runnerDeadlineAt, runner_wait_branch AS runnerWaitBranch, runner_wait_reasons AS runnerWaitReasons FROM waiter WHERE name = ?').all(name);}/** Current holders and the waiting line in admission order, each waiter naming its class (aged only while class order is on). */function snapshot(db, name, order = ARRIVAL_ORDER, now = Date.now()) {/** Preserve older public entries while naming diagnostics as a distinct queue class. */- const project = ({ diagnostic, browser, gui, remote, runner, item, kind, leaseUntil, shortSlot, shortEligible, timing, memoryHeld, ...entry }) => ({ ...entry, landing: entry.landing === 1,+ const project = ({ diagnostic, browser, gui, remote, runner, item, kind, leaseUntil, shortSlot, shortEligible, timing, memoryHeld, runnerDeadlineAt, runnerWaitBranch, runnerWaitReasons, ...entry }) => ({ ...entry, landing: entry.landing === 1,itemCheck: entry.itemCheck === 1 && diagnostic !== 1, ...(diagnostic === 1 ? { diagnostic: true } : {}),...(['gate', 'browser'].includes(name) && shortSlot === 1 ? { slot: 'short' } : {}),...(['gate', 'browser'].includes(name) && shortSlot === 0 ? { slot: 'gate' } : {}),...(['gate', 'browser'].includes(name) && shortEligible === 1 ? { requestedSlot: 'short-eligible' } : {}),+ ...(runnerDeadlineAt > 0 && now >= runnerDeadlineAt ? { runnerDeadline: { at: runnerDeadlineAt, branch: runnerWaitBranch, runners: JSON.parse(runnerWaitReasons) } } : {}),...(memoryHeld === 1 ? { memoryHeld: true } : {}), ...(timing === 1 ? { timing: true } : {}), ...(browser === 1 ? { browser: true } : {}), ...(gui === 1 ? { gui: true } : {}), ...(remote === 1 ? { remote: true } : {}),...(runner ? { runner } : {}), ...(item ? { item } : {}), ...(kind ? { kind } : {}), ...(leaseUntil ? { leaseUntil } : {}) });// Browser is an additional capability on a gate lease, not a second partially held row.@@ -844,6 +853,8 @@ export async function takeResource(options) {const resume = options._resume;const token = resume?.token ?? randomUUID();const requestedAt = resume?.requestedAt ?? Date.now();+ // Capture once: requeueing or editing the machine setting cannot restart this run's clock.+ let runnerDeadlineAt = resume?.runnerDeadlineAt ?? 0;let priorWaitMs = resume?.priorWaitMs ?? 0;let waitStartedAt = resume?.waitStartedAt ?? Date.now();const pid = process.pid;@@ -923,6 +934,8 @@ export async function takeResource(options) {// older pre-lock snapshot cannot overwrite a newer configured capacity.const runners = runnerProvider ? validatedRunners(runnerProvider()) : [];const runnersFirst = runnersFirstProvider?.() === true;+ if (!runnerDeadlineAt && remote && runners.length && options.runnerWaitMs) runnerDeadlineAt = requestedAt + options.runnerWaitMs;+ const runnerDeadlinePassed = runnerDeadlineAt > 0 && now >= runnerDeadlineAt;const waiter = db.prepare('SELECT ticket, remote, browser, gui FROM waiter WHERE token = ?').get(token);if (!waiter) {if (resume && initialTicket) {@@ -935,14 +948,29 @@ export async function takeResource(options) {return { acquired: false, state: { ...snapshot(db, name, order), position: waiterPosition(db, name, initialTicket, { order, shortCapacity: options.shortCapacityProvider?.() ?? 0 }), rejoined: true } };}// The marker follows this poll's setting, so a change mid-wait reaches every process's ranking.- db.prepare('UPDATE waiter SET heartbeat = ?, classed = ? WHERE token = ?').run(Date.now(), classedMarker(order), token);+ db.prepare('UPDATE waiter SET heartbeat = ?, classed = ?, remote = ?, runner_deadline = ? WHERE token = ?')+ .run(Date.now(), classedMarker(order), Number(remote && !runnerDeadlinePassed), runnerDeadlineAt, token);+ waiter.remote = Number(remote && !runnerDeadlinePassed);const unavailable = unavailableRunnerHosts(db, now);const memoryHeld = Boolean(memory?.held && (name !== 'gate' || waiter.remote !== 1|| !runners.some(runner => runnerCompatible(db, runner, waiter, unavailable))));db.prepare('UPDATE waiter SET memory_held = ? WHERE token = ?').run(Number(memoryHeld), token);+ const limit = db.prepare('SELECT capacity FROM resource WHERE name = ?').get(name).capacity;+ let runnerDeadline;+ if (runnerDeadlinePassed) {+ const state = snapshot(db, name, order, now);+ runnerDeadline = { at: runnerDeadlineAt, runners: runnerWaitReasons(db, runners, { ...waiter, timing: Number(timing) }),+ branch: localWaitBranch({ capacity: limit, browserCapacity: options.browserCapacityProvider?.() ?? 1,+ runnerOnly: false, browser, timing, memoryHeld: memory?.held === true, holders: state.holders }) };+ db.prepare('UPDATE waiter SET runner_wait_branch = ?, runner_wait_reasons = ? WHERE token = ?')+ .run(runnerDeadline.branch, JSON.stringify(runnerDeadline.runners), token);+ // Without local capacity the run cannot fall back, so it refuses by name rather than wait forever.+ if (runnerDeadline.branch === 'no local capacity') {+ throw new Refused('RUNNER_WAIT_DEADLINE', `${runnerDeadlineLine(runnerDeadline)}; restore safe local capacity or check the named runners, then retry`);+ }+ }const spot = snapshot(db, name, order, now);const reportPosition = waiterPosition(db, name, waiter.ticket, { order, now, shortCapacity: options.shortCapacityProvider?.() ?? 0 });- const limit = db.prepare('SELECT capacity FROM resource WHERE name = ?').get(name).capacity;// Local pressure blocks local executors, including short slots, but never a remote executor.if (name !== 'gate' && memoryHeld) return { acquired: false, state: { ...spot, position: reportPosition, memory } };const executor = name === 'gate'@@ -954,11 +982,12 @@ export async function takeResource(options) {const acquiredAt = new Date().toISOString();const holderName = executor.host ? runnerResourceName(executor.host) : name;db.prepare('INSERT INTO holder(token, name, pid, started, agent, repo, since, heartbeat, landing, item_check, diagnostic, browser, item, kind, runner, short_slot, timing, timing_aware) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)').run(token, holderName, pid, started, agent, repo, acquiredAt, Date.now(), Number(landing), Number(itemCheck || diagnostic), Number(diagnostic), Number(browser), item, kind, executor.host, Number(executor.shortSlot === true), Number(timing), 1);- return { acquired: true, state: { ...spot, position: 0 }, ticket: waiter.ticket, queueTicket, runner: executor.host ? executor : undefined, shortSlot: executor.shortSlot === true, capacity: limit, admittedAt: acquiredAt };+ return { acquired: true, state: { ...spot, position: 0, ...(runnerDeadline ? { runnerDeadline } : {}) }, ticket: waiter.ticket, queueTicket, runner: executor.host ? executor : undefined, shortSlot: executor.shortSlot === true, capacity: limit, admittedAt: acquiredAt };}- return { acquired: false, state: { ...spot, position: reportPosition, ...(memoryHeld ? { memory } : {}) } };+ return { acquired: false, state: { ...spot, position: reportPosition, ...(memoryHeld ? { memory } : {}), ...(runnerDeadline ? { runnerDeadline } : {}) } };}));if (acquired.acquired) {+ if (acquired.state.runnerDeadline) process.stderr.write(runnerDeadlineLine(acquired.state.runnerDeadline) + '; running locally\n');// Only item-lease callbacks opt into admission. Await them so no renewal remains in// flight during the run, and restart the ordinary item expiry from this boundary.if (leaseUntil) {@@ -1047,12 +1076,13 @@ export async function takeResource(options) {}db.close();return takeResource({ ...options, ...(local ? { remote: false } : {}), _resume: { token, ticket: requeueTicket ?? acquired.ticket, queueTicket: requeueOpportunity,- queuedAt, requestedAt, priorWaitMs: acquiredWaitMs, waitStartedAt: Date.now() } });+ queuedAt, requestedAt, runnerDeadlineAt, priorWaitMs: acquiredWaitMs, waitStartedAt: Date.now() } });};return { name, scope, token, pool, browser, timing, admittedAt: acquired.admittedAt, capacity: acquired.capacity, slotClass: acquired.shortSlot ? 'short' : 'gate', ...(acquired.runner ? { runner: acquired.runner } : {}),- waitMs: acquiredWaitMs, release, renew, timingEvidence,+ waitMs: acquiredWaitMs, ...(runnerDeadlineAt ? { runnerWaitDeadline: runnerDeadlineAt } : {}), release, renew, timingEvidence,...(acquired.runner ? { requeueRunner } : {}) };}+ if (acquired.state.runnerDeadline) process.stderr.write(runnerDeadlineLine(acquired.state.runnerDeadline) + '\n');await keepItemLease(acquired.state, 'waiter');await pause(POLL_MS);}diff --git a/src/runner-private-check.js b/src/runner-private-check.jsindex 158f342b..83b28de0 100644--- a/src/runner-private-check.js+++ b/src/runner-private-check.js@@ -68,7 +68,7 @@ export function runnerPrivateCheck(root, item, command, policy, lease, guard, waconst checkedSource = { commit: item.item_commit, tree: policyGit(root, ['rev-parse', `${item.item_commit}^{tree}`]).trim() };const started = Date.now();lease.tracker?.mark('runAt');- const result = runOnRunner(root, command, { runner: lease.runner, source: lease.source, browser: lease.browser,+ const result = runOnRunner(root, command, { runner: lease.runner, source: lease.source, runnerWaitDeadline: lease.runnerWaitDeadline, browser: lease.browser,metadata: lease.runMetadata, checkedSource, privateCheck: mode, rawOutputPath: join(directory, 'raw.log') });lease.tracker?.mark('ranAt');lease.environment = result.environment ?? null;diff --git a/src/runner-wait.js b/src/runner-wait.jsnew file mode 100644index 00000000..96341e42--- /dev/null+++ b/src/runner-wait.js@@ -0,0 +1,68 @@+/** Bounded runner admission and read-only overdue queue status [C7,Q1,Q2]. */+import { existsSync } from 'node:fs';+import { join } from 'node:path';+import { DatabaseSync } from 'node:sqlite';+import { machineHome } from './machine-home.js';+import { Refused } from './refused.js';++export const DEFAULT_RUNNER_WAIT_MS = 300_000;++/** Validate a machine's finite runner wait without allowing Node timer overflow. */+export function validateRunnerWaitMs(value) {+ if (!Number.isSafeInteger(value) || value < 1 || value > 2_147_483_647) {+ throw new Refused('BAD_MACHINE_SETTINGS', 'gate.runnerWaitMs must be a positive whole number of milliseconds, at most 2147483647; run pullboard settings gate.runnerWaitMs <n>');+ }+ return value;+}++/** Explain each registered runner from the same local facts used by admission; no network probe. */+export function runnerWaitReasons(db, runners, waiter) {+ return runners.map(runner => {+ const recorded = db.prepare('SELECT detail FROM runner_unavailable WHERE host = ?').get(runner.host);+ const observed = db.prepare('SELECT gui FROM runner_gui WHERE host = ?').get(runner.host);+ const gui = observed ? observed.gui === 1 : runner.gui === true;+ const holders = db.prepare('SELECT browser, timing FROM holder WHERE name = ?').all(`runner:${runner.host}`);+ let reason = 'available';+ if (recorded?.detail === 'TRANSPORT_LOST') reason = 'unreachable (transport lost)';+ else if (recorded || db.prepare('SELECT host FROM runner_draining WHERE host = ?').get(runner.host)) reason = 'unavailable (cleanup or draining)';+ else if (waiter.gui === 1 && !gui) reason = 'unavailable (GUI session required)';+ else if (waiter.browser === 1 && runner.browser < 1) reason = 'unavailable (browser capacity required)';+ else if (holders.length >= runner.slots || holders.some(holder => holder.timing === 1)+ || waiter.timing === 1 && holders.length > 0+ || waiter.browser === 1 && holders.filter(holder => holder.browser === 1).length >= runner.browser) reason = 'busy';+ return { host: runner.host, reason };+ });+}++/** Name the local admission branch without changing queue order or creating capacity. */+export function localWaitBranch({ capacity, browserCapacity, runnerOnly, browser, timing, memoryHeld, holders }) {+ if (runnerOnly) return 'runner-only work has no local fallback';+ if (capacity === 0 || browser && browserCapacity === 0) return 'no local capacity';+ if (memoryHeld) return 'local memory hold';+ if (holders.some(holder => holder.timing) || timing && holders.length) return 'local timing exclusivity';+ if (browser && holders.filter(holder => holder.browser).length >= browserCapacity) return 'local browser slots busy';+ if (holders.filter(holder => holder.slot !== 'short').length >= capacity) return 'local gate slots busy';+ return 'local queue priority';+}++/** Format the same overdue facts for progress, resources, and status. */+export function runnerDeadlineLine(wait) {+ return `runner wait deadline ${new Date(wait.at).toISOString()} passed; ${wait.branch}; ${wait.runners.map(runner => `${runner.host}: ${runner.reason}`).join('; ') || 'no registered runner'}`;+}++/** Read current overdue runs without creating or migrating the resource pool. */+export function runnerDeadlineRuns() {+ const file = join(machineHome(), 'resources.sqlite');+ if (!existsSync(file)) return [];+ let db;+ try {+ db = new DatabaseSync(file, { readOnly: true });+ if (!db.prepare('PRAGMA table_info(waiter)').all().some(column => column.name === 'runner_deadline')) return [];+ return db.prepare("SELECT agent, repo, item, kind, since, runner_deadline AS at, runner_wait_branch AS branch, runner_wait_reasons AS reasons FROM waiter WHERE name='gate' AND runner_deadline > 0 AND runner_deadline <= ? AND heartbeat >= ? ORDER BY ticket")+ .all(Date.now(), Date.now() - 20_000).map(({ reasons, ...run }) => ({ ...run, runners: JSON.parse(reasons) }));+ } catch (error) {+ if (['EACCES', 'EPERM', 'SQLITE_BUSY', 'SQLITE_CANTOPEN'].includes(error.code)+ || error.code === 'ERR_SQLITE_ERROR' && [5, 14].includes(error.errcode & 0xff)) return null;+ throw error;+ } finally { db?.close(); }+}diff --git a/src/runners.js b/src/runners.jsindex 6a0d8188..bc70c4bf 100644--- a/src/runners.js+++ b/src/runners.js@@ -673,6 +673,8 @@ export async function runOnRunnerAsync(root, command, options) {if (options.signal?.aborted) throw new RunnerDropped(options.runner.host);const request = requestFor(root, command, options);let removeAbort = () => {};+ let runnerWaitTimer;+ let runnerWaitKill;let rawFd = null;try {if (request.rawOutputPath) rawFd = openSync(request.rawOutputPath, 'wx', 0o600);@@ -683,6 +685,13 @@ export async function runOnRunnerAsync(root, command, options) {let rawBytes = 0, head = Buffer.alloc(0), tail = Buffer.alloc(0);const half = 4 * 1024 ** 2;const abort = () => { error ??= new Error('runner transport canceled'); child.stdin.destroy(); child.kill('SIGTERM'); };+ // Only admission is bounded here. NEED is emitted after the worker owns its gate slot;+ // an admitted test may run longer than the runner wait without being truncated.+ if (Number.isSafeInteger(options.runnerWaitDeadline)) runnerWaitTimer = setTimeout(() => {+ abort();+ // Budget: a non-responsive SSH child cannot retain a local reservation forever.+ runnerWaitKill = setTimeout(() => child.kill('SIGKILL'), 1_000);+ }, Math.max(0, options.runnerWaitDeadline - Date.now()));options.signal?.addEventListener('abort', abort, { once: true });removeAbort = () => options.signal?.removeEventListener('abort', abort);if (options.signal?.aborted) abort();@@ -726,6 +735,7 @@ export async function runOnRunnerAsync(root, command, options) {if (!line.startsWith(prefix)) continue;if (requested) { error = new Error('duplicate runner inventory'); child.kill(); return; }requested = true;+ clearTimeout(runnerWaitTimer);transfer = (async () => {const inventory = JSON.parse(Buffer.from(line.slice(prefix.length), 'base64').toString('utf8'));if (!Array.isArray(inventory.tips) || inventory.tips.length > 1024@@ -751,5 +761,5 @@ export async function runOnRunnerAsync(root, command, options) {const bounded = request.rawOutputPath ? Buffer.concat([head, ...(rawBytes > half * 2 ? [Buffer.from('\n[output capped at 8 MiB; middle omitted]\n')] : []), tail]) : Buffer.concat(output);return resultFrom(request, { status, error, stdout: bounded, stderr: Buffer.concat(diagnostics),...(request.rawOutputPath ? { rawOutputPath: request.rawOutputPath, rawBytes, rawSha256: rawHash.digest('hex') } : {}) });- } finally { if (rawFd !== null) closeSync(rawFd); removeAbort(); request.cleanup(); }+ } finally { clearTimeout(runnerWaitTimer); clearTimeout(runnerWaitKill); if (rawFd !== null) closeSync(rawFd); removeAbort(); request.cleanup(); }}diff --git a/src/settings.js b/src/settings.jsindex a30f147c..2d646e73 100644--- a/src/settings.js+++ b/src/settings.js@@ -13,6 +13,10 @@ import {} from './resources.js';import { validateRunner } from './runners.js';import { Refused } from './refused.js';+import {+ DEFAULT_RUNNER_WAIT_MS,+ validateRunnerWaitMs,+} from './runner-wait.js';import {carriedBrowserByLaunch,gateQueueSettings,@@ -33,6 +37,7 @@ export function loadMachineSettings() {const file = machineSettingsFile();const value = readMachineSettingsFile(file);if (!value) return { gateSlots: DEFAULT_GATE_SLOTS, browserSlots: DEFAULT_BROWSER_SLOTS };+ if (value.gate?.runnerWaitMs !== undefined) validateRunnerWaitMs(value.gate.runnerWaitMs);if (value.landing !== undefined && (!value.landing || typeof value.landing !== 'object' || Array.isArray(value.landing)|| Object.keys(value.landing).some(key => key !== 'pools') || !Array.isArray(value.landing.pools)|| value.landing.pools.some(id => typeof id !== 'string' || !/^[a-f0-9]{32}$/u.test(id)))) {@@ -94,6 +99,20 @@ export function gateRunnersFirstEnabled() {return loadMachineSettings().gate?.runnersFirst === true;}+/** Capture the runner wait deadline for a new run; later requeues retain that deadline. */+export function gateRunnerWaitMs() {+ return loadMachineSettings().gate?.runnerWaitMs ?? DEFAULT_RUNNER_WAIT_MS;+}++/** Set the runner deadline for new work without changing any admitted or queued run. */+export function setGateRunnerWaitMs(milliseconds) {+ validateRunnerWaitMs(milliseconds);+ const current = loadMachineSettings();+ const next = { ...current, gate: { ...current.gate, runnerWaitMs: milliseconds } };+ writeMachineSettings(next);+ return next;+}+/*** Classifying a test file as browser-bound only when it reaches a real Chrome launch is opt-in per* machine; off, any path to the Chrome fixture or an unfollowed process starter takes a browser slot.diff --git a/test/runner-wait-deadline.test.js b/test/runner-wait-deadline.test.jsindex 3cd1193a..ceef54be 100644--- a/test/runner-wait-deadline.test.js+++ b/test/runner-wait-deadline.test.js@@ -143,3 +143,20 @@ test('a registered runner that never answers is canceled at the deadline and the}}});++test('the public runner deadline setting has a five-minute default and rejects unsafe timer values [C7]', () => {+ const box = e2e.project('true');+ const initial = cli(box.repo, box.env, 'settings', '--json');+ assert.equal(initial.status, 0, fixtureResultContext(initial));+ assert.equal(JSON.parse(initial.stdout).settings.gate.runnerWaitMs, 300_000);+ const changed = cli(box.repo, box.env, 'settings', 'gate.runnerWaitMs', '1500', '--json');+ assert.equal(changed.status, 0, fixtureResultContext(changed));+ assert.equal(JSON.parse(changed.stdout).settings.gate.runnerWaitMs, 1500);+ const file = join(box.env.PULLBOARD_MACHINE_HOME, 'settings.json');+ const before = readFileSync(file, 'utf8');+ for (const value of ['0', '-1', '1.5', '2147483648', 'Infinity']) {+ const invalid = cli(box.repo, box.env, 'settings', 'gate.runnerWaitMs', value);+ assert.notEqual(invalid.status, 0, fixtureResultContext(invalid));+ assert.equal(readFileSync(file, 'utf8'), before, 'invalid settings never replace the last valid deadline');+ }+});