Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
JavaScript ai-agentsdashboarddeveloper-toolsdigital-twinquantified-selfsecond-brainself-hostedtailscale
fix: prevent managed daemon idle and memory pressure eviction during active requests
Stephen Hilderbrandcd63c314e0c1
diff --git a/server/lib/README.md b/server/lib/README.mdindex 2e56bd79f1..b4f9013b91 100644--- a/server/lib/README.md+++ b/server/lib/README.md@@ -356,7 +356,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub| `localReviewLifecycle.js` | `localReviewLifecycle` — durable pre-PR phase completion bound to the branch, origin, base and exact PR delivery; preserves its original reviewed head through later pushes. || `agentExecutionProfiles.js` | Named agent execution postures shared by lifecycle dispatch, CLI/TUI spawners, and child-environment filtering. A stage declares a PROFILE (`PUBLIC_REVIEW_EXECUTION_PROFILE`, `PUBLIC_REVIEW_GATE_EXECUTION_PROFILE`, `PUBLIC_REVIEW_ACTIONS_EXECUTION_PROFILE`); `publicReviewPostureForProfile` maps it to the enforceable POSTURE a provider must carry a maintained recipe for (`no-tool` or `sandboxed-actions`), which is how a pipeline stage stays vendor-agnostic. `isPublicReviewRestrictedProfile` covers both postures; `isPublicReviewNoToolProfile` / `isPublicReviewActionsProfile` separate them for the callers whose rule differs by posture (credential-bootstrap wrapping, child-env allowlist). || `agentCompletionMode.js` | The ONE completion-contract decision behind every agent prompt: `resolveCompletionMode(flags)` returns a single `COMPLETION_MODES` key (`tool-free`, `sentinel-payload`, `action-output`, `external-repository`, `discard-worktree`, `claim-flow`, `read-only`, `review-loop-follow-up`, `release-flow`, `audit-flow`, `reconcile-flow`, `checkout-recovery`, `tui-slashdo-free`, `tui`, `portos-merges`, `worktree-no-push`, `commit-and-push`) from an ordered rule list, plus `portosMergesBranchOnExit`. Replaces the five independent flag ladders in `agentPromptBuilder.js` that each re-derived it in a different order — two had drifted far enough to render a prompt telling a tool-free or read-only task both to commit and that it had no git at all (#6616). Call sites branch on the key and render per mode; they never re-test the flags. |-| `localLlmConcurrency.js` | `LOCAL_LLM_MAX_CONCURRENCY` — the per-endpoint local-LLM concurrency cap (env read at load); a leaf so `cosLocalEndpointSlots.js` avoids importing `promptRunner.js`, which re-exports it. |+| `localLlmConcurrency.js` | `LOCAL_LLM_MAX_CONCURRENCY` plus the per-endpoint local-LLM slot gate (`acquireLocalSlot`, `releaseLocalSlot`), in-flight activity predicate (`isLocalEndpointBusy`, matching exact endpoint or loopback port), and settle observer (`onLocalEndpointSettled`) — a dependency-free leaf so `cosLocalEndpointSlots.js` and `managedDaemon.js` avoid importing `promptRunner.js`. || `localEndpoint.js` | Dependency-light local-instance URL predicates (`isLocalInstanceHost`, `isLocalInstanceEndpoint`, scheme-optional `isLoopbackEndpoint` — the one "is this a local daemon" answer, `localEndpointPort`) shared by provider safety policy and local-runtime classification without importing backend configuration or daemon managers. || `credentialBootstrap.js` | resolveBootstrapEnv(provider, { safetyProfile }) runs an optional `envCommand` argv for ordinary provider runs or a no-tool review, parses KEY=value output without evaluation, and caches credentials in memory for one minute — the alternative for a bootstrap that PRINTS credentials rather than exec'ing the harness; a record that only names the wrap form is credentialed by the wrap and gets `{}`. Generic support for a CLI/TUI provider whose auth is provisioned by an external CLI at spawn time (e.g. a short-lived token for a proxy) rather than a static `apiKey` PortOS stores — a `provider.credentialBootstrap: { setupCommand?, command, args?, harnessId?, argsSeparator? }` record. `hasCredentialBootstrap(provider)` and `applyCredentialBootstrap(provider, command, args)` (pure) return the command+args PortOS should actually spawn: `<command> <args...> <harnessId ?? command> [<argsSeparator>] <args...>` — the bootstrap CLI in front of the harness invocation, naming the harness as `harnessId` when the bootstrap CLI uses its own identifier rather than the binary name, and inserting `argsSeparator` (e.g. `--`) before the harness's own args when the bootstrap CLI needs its flags kept apart from the wrapped program's. `resolveCliSpawn(provider, command, args, childEnv)` composes that with Windows shim resolution (`bufferedSpawn.js`) for the three headless spawn sites that repeated the same sequence (`runner.js`, `cliProviderRun.js`, `visionCli.js`) — the argv analog of why `cliChildEnv.js`'s `buildCliChildEnv` exists. Every call site keeps resolving `provider.command`/`args` unchanged for vendor detection, prompt-delivery convention, and the TUI's own ready-text heuristics, and only asks this module for the pair handed to the literal `spawn`/`pty.spawn` call — plus anything that observes the launching shell or PATH (missing-binary detection, readiness probes), which must read the SPAWNED pair. The `sandboxed-actions` posture is never wrapped — it executes contributor-supplied code inside a sandbox spelled entirely in the recipe's argv, so nothing may sit in front of the harness and rewrite it; the `no-tool` postures (the tool-free reviewer among them) ARE wrapped, because spawned bare they carry no credential at all (#7720). The split lives in the helper, keyed on `safetyProfile`. The pure argv composition lives in `aiToolkit/internal/credentialBootstrap.js` (`composeBootstrapSpawn` / `hasCredentialBootstrap`, dependency-free), re-exported through here so the vendored toolkit's own catalog probes compose the same shape without importing out. Every provider spawn site goes through here — a bare-harness spawn runs with no credential and falls through to ambient vendor auth. Both also return `wrapped`, because a wrapped spawn's direct child is the SUPERVISING wrapper rather than the harness: `needsProcessGroup(wrapped)` says whether to spawn `detached` and kill with `{ processGroup: true }` (POSIX only — Windows' `taskkill /T` is already tree-wide), and `processGroupKillable(child, processGroup)` adapts a child for the aiToolkit runner's external-run registry, whose vendored `killProcessTree` has no group option. Without them a stop/timeout/cancel signals the wrapper alone and leaves the harness running. `trackDetachedGroup(child, processGroup)` / `signalDetachedGroups(signal, logFailure)` are the other side of that trade: detaching also moves a child OUT of the server's own process group, so the shutdown handler sweeps this registry to keep a signal aimed at that group (Ctrl-C at `npm start`) from orphaning one. || `cliProviderArgs.js` | Per-CLI argv conventions (`buildCliArgs`) for stdin prompt delivery — dependency-light extraction from runner.js so out-of-process callers (autofixer) can import it. |diff --git a/server/lib/localLlmConcurrency.js b/server/lib/localLlmConcurrency.jsindex 7580992ae9..da706bd49f 100644--- a/server/lib/localLlmConcurrency.js+++ b/server/lib/localLlmConcurrency.js@@ -1,3 +1,81 @@-// Per-endpoint cap on concurrent local-LLM work. Own leaf so CoS slot-// accounting can read it without loading the prompt runner's import graph.+// Per-endpoint cap and in-flight slot tracking for concurrent local-LLM work.+// Own dependency-free leaf so CoS slot accounting and the managed-daemon idle+// reaper can read it without loading the prompt runner's import graph.export const LOCAL_LLM_MAX_CONCURRENCY = Math.max(1, Number(process.env.LOCAL_LLM_MAX_CONCURRENCY) || 1);++const localEndpointGates = new Map(); // endpoint -> { active: number, queue: (()=>void)[] }+const settledListeners = new Set();++function endpointPortKey(endpoint) {+ if (typeof endpoint !== 'string' || !endpoint.trim()) return null;+ const trimmed = endpoint.trim();+ const candidate = /^[a-z][a-z0-9+.-]*:\/\//i.test(trimmed) ? trimmed : `http://${trimmed}`;+ if (!URL.canParse(candidate)) return null;+ const url = new URL(candidate);+ return url.port || (url.protocol === 'https:' ? '443' : '80');+}++function notifyEndpointActivity(endpoint) {+ for (const listener of settledListeners) {+ try {+ listener(endpoint);+ } catch {+ // Best-effort observer notification.+ }+ }+}++export function acquireLocalSlot(endpoint) {+ let gate = localEndpointGates.get(endpoint);+ if (!gate) {+ gate = { active: 0, queue: [] };+ localEndpointGates.set(endpoint, gate);+ }+ notifyEndpointActivity(endpoint);+ if (gate.active < LOCAL_LLM_MAX_CONCURRENCY) {+ gate.active += 1;+ return Promise.resolve();+ }+ return new Promise((resolve) => {+ gate.queue.push(() => {+ notifyEndpointActivity(endpoint);+ resolve();+ });+ });+}++export function releaseLocalSlot(endpoint) {+ const gate = localEndpointGates.get(endpoint);+ if (!gate) return;+ const next = gate.queue.shift();+ if (next) next(); // hand the slot straight to the next waiter (active unchanged)+ else gate.active = Math.max(0, gate.active - 1);+ notifyEndpointActivity(endpoint);+}++/**+ * True when any local-LLM run is currently in flight (or queued) against+ * `endpoint` or a loopback alias sharing its port.+ */+export function isLocalEndpointBusy(endpoint) {+ if (typeof endpoint !== 'string' || !endpoint.trim()) return false;+ const targetPort = endpointPortKey(endpoint);+ for (const [key, gate] of localEndpointGates) {+ if (!gate || (gate.active <= 0 && gate.queue.length === 0)) continue;+ if (key === endpoint) return true;+ if (targetPort && endpointPortKey(key) === targetPort) return true;+ }+ return false;+}++/**+ * Register a listener invoked when a local endpoint slot is acquired or+ * released, so a managed daemon's idle clock refreshes both on dispatch and+ * when a long-running inference finishes.+ */+export function onLocalEndpointSettled(listener) {+ if (typeof listener !== 'function') return () => {};+ settledListeners.add(listener);+ return () => settledListeners.delete(listener);+}+diff --git a/server/lib/managedDaemon.js b/server/lib/managedDaemon.jsindex d72700bcf6..728b5c2964 100644--- a/server/lib/managedDaemon.js+++ b/server/lib/managedDaemon.js@@ -1,5 +1,6 @@import { getMemoryStats } from './memoryStats.js';import { localEndpointPort } from './localEndpoint.js';+import { isLocalEndpointBusy, onLocalEndpointSettled } from './localLlmConcurrency.js';import { createSingleFlight } from './singleFlight.js';/**@@ -288,12 +289,24 @@ export function createOnDemandDaemon({return Boolean(section?.keepLoaded ?? section?.pinned);};+ let inFlightStarts = 0;+ const currentEndpoint = () => endpointFor(getConfig());+ const isBusy = () => inFlightStarts > 0 || isLocalEndpointBusy(currentEndpoint());++ onLocalEndpointSettled((endpoint) => {+ const port = localEndpointPort(currentEndpoint());+ if (port && localEndpointPort(endpoint) === port) {+ markDaemonUsed(appName);+ }+ });+const registerIdle = () => {registerIdleDaemon({name: appName,getIdleMs: async () => idleWindowMs(await configuredIdleMinutes()),isPinned: async () => configuredKeepLoaded(),isRunning: async () => Boolean((await getStatusStrict())?.status === 'online'),+ isBusy,stop: () => stop(),});};@@ -360,11 +373,19 @@ export function createOnDemandDaemon({*/const starts = createSingleFlight({ exclusive: true });- const ensureRunning = async () => {+ const trackStart = async (key, fn) => {+ inFlightStarts += 1;markUsed();- return starts.run('wake', wake);+ try {+ return await starts.run(key, fn);+ } finally {+ inFlightStarts = Math.max(0, inFlightStarts - 1);+ markUsed();+ }};+ const ensureRunning = () => trackStart('wake', wake);+/*** Run an explicit start/stop sequence (a user-clicked Start, a tuning* relaunch) in the daemon's exclusive start slot. Each call gets its own key,@@ -377,7 +398,7 @@ export function createOnDemandDaemon({* @param {() => Promise<T>} fn* @returns {Promise<T>}*/- const runStart = (fn) => starts.run(Symbol('start'), fn);+ const runStart = (fn) => trackStart(Symbol('start'), fn);const wake = async () => {const pm2Status = await getStatusStrict();@@ -536,18 +557,21 @@ export function idleWindowMs(minutes) {* getIdleMs: () => Promise<number|null>|number|null,* isPinned?: () => Promise<boolean>|boolean,* isRunning?: () => Promise<boolean>|boolean,+ * isBusy?: () => Promise<boolean>|boolean,* stop: () => Promise<unknown>* }} daemon* `getIdleMs` resolves the CURRENT configured window on every sweep; `null`/`0` = never stop.* `isPinned` returns true if user pinned the server ("keep loaded") — pinned servers are never stopped.* `isRunning` checks if the daemon process is online.+ * `isBusy` returns true while a wake/start or inference request is in flight — busy servers are never stopped.*/-export function registerIdleDaemon({ name, getIdleMs, isPinned, isRunning, stop }) {+export function registerIdleDaemon({ name, getIdleMs, isPinned, isRunning, isBusy, stop }) {const existing = idleDaemons.get(name);idleDaemons.set(name, {getIdleMs,isPinned: typeof isPinned === 'function' ? isPinned : () => Boolean(isPinned),isRunning: typeof isRunning === 'function' ? isRunning : null,+ isBusy: typeof isBusy === 'function' ? isBusy : () => Boolean(isBusy),stop,lastUsedAt: existing?.lastUsedAt ?? Date.now(),releaseReason: existing?.releaseReason ?? null,@@ -685,6 +709,7 @@ export function evaluateMemoryPressurePolicy({const eligible = list.filter((d) => (d&& d.running !== false+ && !d.busy&& !d.pinned&& !d.keepLoaded));@@ -723,12 +748,15 @@ export async function reapIdleDaemons(now = Date.now(), options = {}) {// 1. Normal idle timeout passfor (const [name, entry] of idleDaemons) {- // Fail-safe to pinned on a transient read error, matching the pressure-aware- // pass below — assuming "not pinned" here would risk stopping a keepLoaded- // daemon on a flaky settings read.+ // Fail-safe to pinned/busy on a transient read error, matching the pressure-aware+ // pass below — assuming "not pinned" or "not busy" here would risk stopping a+ // keepLoaded or actively serving daemon on a flaky read.const isPinned = await Promise.resolve(entry.isPinned?.()).catch(() => true);if (isPinned) continue; // Pinned servers are exempt+ const isBusy = await Promise.resolve(entry.isBusy?.()).catch(() => true);+ if (isBusy) continue; // Active wake/start or in-flight request is exempt+// Already stopped (by the pressure-aware pass, or externally) — skip so this// pass doesn't overwrite its release reason and doesn't preempt the// pressure-aware pass below via the early return once its idle window re-elapses.@@ -778,6 +806,7 @@ export async function reapIdleDaemons(now = Date.now(), options = {}) {const daemonList = [];for (const [name, entry] of idleDaemons) {const isPinned = await Promise.resolve(entry.isPinned?.()).catch(() => true);+ const isBusy = await Promise.resolve(entry.isBusy?.()).catch(() => true);const isRunning = entry.isRunning? await Promise.resolve(entry.isRunning()).catch(() => false): true;@@ -786,6 +815,7 @@ export async function reapIdleDaemons(now = Date.now(), options = {}) {entry,lastUsedAt: entry.lastUsedAt,pinned: isPinned,+ busy: isBusy,running: isRunning,});}diff --git a/server/lib/managedDaemon.pressure.test.js b/server/lib/managedDaemon.pressure.test.jsindex edcd69a759..6f89983570 100644--- a/server/lib/managedDaemon.pressure.test.js+++ b/server/lib/managedDaemon.pressure.test.js@@ -12,7 +12,9 @@ import {DEFAULT_PRESSURE_CALM_DOWN_MS,DEFAULT_SUSTAINED_PRESSURE_MS,createDaemonWatcher,+ createOnDemandDaemon,} from './managedDaemon.js';+import { acquireLocalSlot, releaseLocalSlot } from './localLlmConcurrency.js';const MINUTE = 60_000;const GB = 1024 * 1024 * 1024;@@ -209,6 +211,35 @@ describe('managedDaemon memory pressure policy', () => {expect(result.target.name).toBe('slotstream');});+ it('never releases a daemon that is currently busy', () => {+ const daemons = [+ { name: 'mtplx', lastUsedAt: now - 60_000, busy: true },+ ];+ const result = evaluateMemoryPressurePolicy({+ daemons,+ memoryStats: { total: 64 * GB, used: 62 * GB, free: 2 * GB },+ history: sustainedHistory,+ now,+ });+ expect(result.shouldRelease).toBe(false);+ expect(result.reason).toBe('no eligible daemons to release');+ });++ it('skips busy daemon and releases the next least recently used idle daemon', () => {+ const daemons = [+ { name: 'mtplx', lastUsedAt: now - 60_000, busy: true },+ { name: 'slotstream', lastUsedAt: now - 20_000, busy: false },+ ];+ const result = evaluateMemoryPressurePolicy({+ daemons,+ memoryStats: { total: 64 * GB, used: 62 * GB, free: 2 * GB },+ history: sustainedHistory,+ now,+ });+ expect(result.shouldRelease).toBe(true);+ expect(result.target.name).toBe('slotstream');+ });+it('honors sustainedDurationMs: 0 for instant evaluation', () => {const result = evaluateMemoryPressurePolicy({daemons: [{ name: 'mtplx', lastUsedAt: now - 1000 }],@@ -406,5 +437,98 @@ describe('managedDaemon memory pressure policy', () => {const onlineStatus = await watcher.getStatusBase({ installed: true });expect(onlineStatus.releaseReason).toBeNull();});++ it('never stops a busy daemon during idle-timeout or memory-pressure passes, and fail-safes on isBusy errors', async () => {+ const stopBusy = vi.fn().mockResolvedValue(undefined);+ const stopFlaky = vi.fn().mockResolvedValue(undefined);+ registerIdleDaemon({+ name: 'daemon-busy',+ getIdleMs: () => 5 * MINUTE,+ isBusy: () => true,+ stop: stopBusy,+ });+ registerIdleDaemon({+ name: 'daemon-flaky-busy',+ getIdleMs: () => 5 * MINUTE,+ isBusy: () => Promise.reject(new Error('transient busy check error')),+ stop: stopFlaky,+ });++ const now = Date.now() + 10 * MINUTE;+ const stopped = await reapIdleDaemons(now, {+ memoryStats: { total: 32 * GB, used: 30 * GB, free: 2 * GB },+ history: [{ at: now - 40_000, free: 2 * GB }, { at: now, free: 2 * GB }],+ sustainedDurationMs: 30_000,+ });++ expect(stopped).toEqual([]);+ expect(stopBusy).not.toHaveBeenCalled();+ expect(stopFlaky).not.toHaveBeenCalled();+ });++ it('protects createOnDemandDaemon from pressure eviction while a wake or local-LLM slot is active on its port', async () => {+ const stop = vi.fn().mockResolvedValue(undefined);+ let resolveWakeStart;+ const startPromise = new Promise((r) => { resolveWakeStart = r; });+ let pm2Status = { status: 'stopped' };++ const onDemand = createOnDemandDaemon({+ appName: 'portos-mtplx',+ label: 'MTPLX server',+ emoji: '⚡',+ readSection: async () => ({ idleMinutes: 30, keepLoaded: false, launch: { model: 'test-model', port: 8092 } }),+ endpointFor: (cfg) => `http://127.0.0.1:${cfg?.port || 8092}/v1`,+ probe: async () => false,+ start: () => startPromise,+ stop,+ getStatusStrict: async () => pm2Status,+ resolveLaunch: (current, saved) => current || { model: saved.model, port: saved.port || 8092 },+ getConfig: () => ({ model: 'test-model', port: 8092 }),+ sleep: async () => {},+ getRelaunchReadyTimeoutMs: () => 5000,+ getRelaunchPollMs: () => 10,+ });+ onDemand.registerIdle();++ // 1. In-flight wake/load: daemon is online in PM2 while weights load, but ensureRunning has not finished.+ const wakeCall = onDemand.ensureRunning();+ pm2Status = { status: 'online' };+ const now1 = Date.now();+ const stoppedDuringWake = await reapIdleDaemons(now1, {+ memoryStats: { total: 32 * GB, used: 30 * GB, free: 2 * GB },+ history: [{ at: now1 - 40_000, free: 2 * GB }, { at: now1, free: 2 * GB }],+ sustainedDurationMs: 30_000,+ });+ expect(stoppedDuringWake).toEqual([]);+ expect(stop).not.toHaveBeenCalled();++ resolveWakeStart({ online: true, endpoint: 'http://127.0.0.1:8092/v1' });+ await expect(wakeCall).resolves.toEqual({ ready: true, reason: null });++ // 2. In-flight inference slot on the daemon's port (even via localhost alias):+ await acquireLocalSlot('http://localhost:8092/v1');+ try {+ const now2 = Date.now() + 1000;+ const stoppedDuringInference = await reapIdleDaemons(now2, {+ memoryStats: { total: 32 * GB, used: 30 * GB, free: 2 * GB },+ history: [{ at: now1 - 40_000, free: 2 * GB }, { at: now2, free: 2 * GB }],+ sustainedDurationMs: 30_000,+ });+ expect(stoppedDuringInference).toEqual([]);+ expect(stop).not.toHaveBeenCalled();+ } finally {+ releaseLocalSlot('http://localhost:8092/v1');+ }++ // 3. Once the slot is released, sustained memory pressure can evict the now-idle daemon.+ const now3 = Date.now() + 2000;+ const stoppedAfterInference = await reapIdleDaemons(now3, {+ memoryStats: { total: 32 * GB, used: 30 * GB, free: 2 * GB },+ history: [{ at: now1 - 40_000, free: 2 * GB }, { at: now3, free: 2 * GB }],+ sustainedDurationMs: 30_000,+ });+ expect(stoppedAfterInference).toEqual(['portos-mtplx']);+ expect(stop).toHaveBeenCalledTimes(1);+ });});});diff --git a/server/services/promptRunner.js b/server/services/promptRunner.jsindex 47ed3b8622..3e2a7f37b2 100644--- a/server/services/promptRunner.js+++ b/server/services/promptRunner.js@@ -28,7 +28,7 @@ import { maintenance } from '../lib/maintenanceAdmission.js';* stay honest about what the runner actually executed.*/-import { LOCAL_LLM_MAX_CONCURRENCY } from '../lib/localLlmConcurrency.js';+import { LOCAL_LLM_MAX_CONCURRENCY, acquireLocalSlot, releaseLocalSlot } from '../lib/localLlmConcurrency.js';import { isVisionCapableCodexTuiProvider } from '../lib/codex.js';import { createRun, executeApiRun, executeCliRun, extractBakedModel, hasModelFlag, stopRun, patchRunMetadata, finalizeRunRecord } from './runner.js';import { getActiveProvider, getProviderById, getAllProviders } from './providers.js';@@ -148,25 +148,6 @@ export function buildRequestCapabilities({ prompt, screenshots, outputReserveTokexport const isLocalEndpoint = isLoopbackEndpoint;export { LOCAL_LLM_MAX_CONCURRENCY };-const localEndpointGates = new Map(); // endpoint -> { active: number, queue: (()=>void)[] }--function acquireLocalSlot(endpoint) {- let gate = localEndpointGates.get(endpoint);- if (!gate) { gate = { active: 0, queue: [] }; localEndpointGates.set(endpoint, gate); }- if (gate.active < LOCAL_LLM_MAX_CONCURRENCY) {- gate.active += 1;- return Promise.resolve();- }- return new Promise((resolve) => { gate.queue.push(resolve); });-}--function releaseLocalSlot(endpoint) {- const gate = localEndpointGates.get(endpoint);- if (!gate) return;- const next = gate.queue.shift();- if (next) next(); // hand the slot straight to the next waiter (active unchanged)- else gate.active = Math.max(0, gate.active - 1);-}// Run `fn` under the local-endpoint gate when `provider` is a local API or TUI// backend; otherwise run it immediately (cloud / non-local CLI/TUI are