Skip to content

Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS

refactor: give the local LLM concurrency cap its own leaf so CoS slots skip the prompt runner (#12986)

[._.]/ Adam Eivy54e59b08c85a
diff --git a/server/lib/README.md b/server/lib/README.md
index a4820a7c9f..e8df51ee53 100644
--- a/server/lib/README.md
+++ b/server/lib/README.md
@@ -346,6 +346,7 @@ The barrel `server/lib/index.js` is a machine-checkable enumeration of every pub
| `agentExecutionProfiles.js` | Named agent execution postures shared by lifecycle dispatch, CLI/TUI spawners, and child-environment filtering. A stage declares a PROFILE (`PUBLIC_REVIEW_EXECUTION_PROFILE`, `PUBLIC_REVIEW_GATE_EXECUTION_PROFILE`, `PUBLIC_REVIEW_ACTIONS_EXECUTION_PROFILE`); `publicReviewPostureForProfile` maps it to the enforceable POSTURE a provider must carry a maintained recipe for (`no-tool` or `sandboxed-actions`), which is how a pipeline stage stays vendor-agnostic. `isPublicReviewRestrictedProfile` covers both postures; `isPublicReviewNoToolProfile` / `isPublicReviewActionsProfile` separate them for the callers whose rule differs by posture (credential-bootstrap wrapping, child-env allowlist). |
| `agentCompletionMode.js` | The ONE completion-contract decision behind every agent prompt: `resolveCompletionMode(flags)` returns a single `COMPLETION_MODES` key (`tool-free`, `sentinel-payload`, `action-output`, `discard-worktree`, `claim-flow`, `read-only`, `review-loop-follow-up`, `release-flow`, `tui-slashdo-free`, `tui`, `portos-merges`, `worktree-no-push`, `commit-and-push`) from an ordered rule list, plus `portosMergesBranchOnExit`. Replaces the five independent flag ladders in `agentPromptBuilder.js` that each re-derived it in a different order — two had drifted far enough to render a prompt telling a tool-free or read-only task both to commit and that it had no git at all (#6616). Call sites branch on the key and render per mode; they never re-test the flags. |
| `localEndpoint.js` | Dependency-light local-instance URL predicates (`isLocalInstanceHost`, `isLocalInstanceEndpoint`, `localEndpointPort`) shared by provider safety policy and local-runtime classification without importing backend configuration or daemon managers. |
+| `localLlmConcurrency.js` | `LOCAL_LLM_MAX_CONCURRENCY` — the per-endpoint local-LLM concurrency cap (env read at load); a leaf so `cosLocalEndpointSlots.js` avoids importing `promptRunner.js`, which re-exports it. |
| `credentialBootstrap.js` | resolveBootstrapEnv(provider, { safetyProfile }) runs an optional `envCommand` argv for ordinary provider runs or a no-tool review, parses KEY=value output without evaluation, and caches credentials in memory for one minute — the alternative for a bootstrap that PRINTS credentials rather than exec'ing the harness; a record that only names the wrap form is credentialed by the wrap and gets `{}`. Generic support for a CLI/TUI provider whose auth is provisioned by an external CLI at spawn time (e.g. a short-lived token for a proxy) rather than a static `apiKey` PortOS stores — a `provider.credentialBootstrap: { setupCommand?, command, args?, harnessId?, argsSeparator? }` record. `hasCredentialBootstrap(provider)` and `applyCredentialBootstrap(provider, command, args)` (pure) return the command+args PortOS should actually spawn: `<command> <args...> <harnessId ?? command> [<argsSeparator>] <args...>` — the bootstrap CLI in front of the harness invocation, naming the harness as `harnessId` when the bootstrap CLI uses its own identifier rather than the binary name, and inserting `argsSeparator` (e.g. `--`) before the harness's own args when the bootstrap CLI needs its flags kept apart from the wrapped program's. `resolveCliSpawn(provider, command, args, childEnv)` composes that with Windows shim resolution (`bufferedSpawn.js`) for the three headless spawn sites that repeated the same sequence (`runner.js`, `cliProviderRun.js`, `visionCli.js`) — the argv analog of why `cliChildEnv.js`'s `buildCliChildEnv` exists. Every call site keeps resolving `provider.command`/`args` unchanged for vendor detection, prompt-delivery convention, and the TUI's own ready-text heuristics, and only asks this module for the pair handed to the literal `spawn`/`pty.spawn` call — plus anything that observes the launching shell or PATH (missing-binary detection, readiness probes), which must read the SPAWNED pair. The `sandboxed-actions` posture is never wrapped — it executes contributor-supplied code inside a sandbox spelled entirely in the recipe's argv, so nothing may sit in front of the harness and rewrite it; the `no-tool` postures (the tool-free reviewer among them) ARE wrapped, because spawned bare they carry no credential at all (#7720). The split lives in the helper, keyed on `safetyProfile`. The pure argv composition lives in `aiToolkit/internal/credentialBootstrap.js` (`composeBootstrapSpawn` / `hasCredentialBootstrap`, dependency-free), re-exported through here so the vendored toolkit's own catalog probes compose the same shape without importing out. Every provider spawn site goes through here — a bare-harness spawn runs with no credential and falls through to ambient vendor auth. Both also return `wrapped`, because a wrapped spawn's direct child is the SUPERVISING wrapper rather than the harness: `needsProcessGroup(wrapped)` says whether to spawn `detached` and kill with `{ processGroup: true }` (POSIX only — Windows' `taskkill /T` is already tree-wide), and `processGroupKillable(child, processGroup)` adapts a child for the aiToolkit runner's external-run registry, whose vendored `killProcessTree` has no group option. Without them a stop/timeout/cancel signals the wrapper alone and leaves the harness running. `trackDetachedGroup(child, processGroup)` / `signalDetachedGroups(signal, logFailure)` are the other side of that trade: detaching also moves a child OUT of the server's own process group, so the shutdown handler sweeps this registry to keep a signal aimed at that group (Ctrl-C at `npm start`) from orphaning one. |
| `cliProviderArgs.js` | Per-CLI argv conventions (`buildCliArgs`) for stdin prompt delivery — dependency-light extraction from runner.js so out-of-process callers (autofixer) can import it. |
| `cliProviderRun.js` | One-shot CLI provider invocation (`pickCliProvider` + `runCliProviderPrompt`) — lightweight path for the autofixer + calendar MCP sync to honor the configured provider/model. Optional `safetyProfile` runs only maintained no-tool vendor recipes and composes the same restricted environment, refusing extra argv and unsupported providers. |
diff --git a/server/lib/importScoping.test.js b/server/lib/importScoping.test.js
index c6df99ce5f..5122ec60bd 100644
--- a/server/lib/importScoping.test.js
+++ b/server/lib/importScoping.test.js
@@ -46,6 +46,10 @@ const reaches = (entry, target) => staticImportClosure(abs(entry)).files.has(abs
// Each row: the entry that was narrowed, the module it must no longer
const NARROWED = [
+ ['services/cosLocalEndpointSlots.js', 'services/promptRunner.js',
+ 'reads the local-LLM concurrency cap from its leaf without loading the prompt runner'],
+ ['services/cos.js', 'services/promptRunner.js',
+ 'reads the local-LLM concurrency cap from its leaf without loading the prompt runner'],
['services/imageGen/prepareParams.js', 'services/imageGen/index.js',
'reads cleaner request policy without loading execution backends'],
['services/codeReview.js', 'lib/validation.js',
diff --git a/server/lib/index.js b/server/lib/index.js
index 5e4ece4e94..d4ce6620f7 100644
--- a/server/lib/index.js
+++ b/server/lib/index.js
@@ -206,6 +206,7 @@ export * from './agentApiToken.js';
export * from './localReviewBridge.js';
export * from './agentCompletionMode.js';
export * from './localEndpoint.js';
+export * from './localLlmConcurrency.js';
export * from './cliProviderArgs.js';
export * from './cliProviderRun.js';
export * from './cliStderrNoise.js';
diff --git a/server/lib/localLlmConcurrency.js b/server/lib/localLlmConcurrency.js
new file mode 100644
index 0000000000..7580992ae9
--- /dev/null
+++ b/server/lib/localLlmConcurrency.js
@@ -0,0 +1,3 @@
+// Per-endpoint cap on concurrent local-LLM work. Own leaf so CoS slot
+// accounting can read it without loading the prompt runner's import graph.
+export const LOCAL_LLM_MAX_CONCURRENCY = Math.max(1, Number(process.env.LOCAL_LLM_MAX_CONCURRENCY) || 1);
diff --git a/server/services/cosLocalEndpointSlots.js b/server/services/cosLocalEndpointSlots.js
index eff36c0eff..8429f8c721 100644
--- a/server/services/cosLocalEndpointSlots.js
+++ b/server/services/cosLocalEndpointSlots.js
@@ -22,7 +22,7 @@
* A shared counter across process boundaries isn't worth the coupling.
*/
-import { LOCAL_LLM_MAX_CONCURRENCY } from './promptRunner.js';
+import { LOCAL_LLM_MAX_CONCURRENCY } from '../lib/localLlmConcurrency.js';
import { SWARM_COUNT_MAX, SWARM_COUNT_MIN } from '../lib/validation.js';
import { localRuntimeForProvider, localEndpointPort, normalizeOpenAiBaseUrl } from '../lib/localProviderRuntime.js';
import { listProviders, getActiveProvider } from './providers.js';
diff --git a/server/services/promptRunner.js b/server/services/promptRunner.js
index f35adf0c79..1016fb780e 100644
--- a/server/services/promptRunner.js
+++ b/server/services/promptRunner.js
@@ -27,6 +27,7 @@ import { maintenance } from '../lib/maintenanceAdmission.js';
* stay honest about what the runner actually executed.
*/
+import { LOCAL_LLM_MAX_CONCURRENCY } from '../lib/localLlmConcurrency.js';
import { isVisionCapableCodexTuiProvider } from '../lib/codex.js';
import { createRun, executeApiRun, executeCliRun, extractBakedModel, hasModelFlag, stopRun, patchRunMetadata, finalizeRunRecord } from './runner.js';
import { getActiveProvider, getProviderById, getAllProviders } from './providers.js';
@@ -147,7 +148,7 @@ const LOCAL_ENDPOINT_RE = /^(https?:\/\/)?(localhost|127\.0\.0\.1|0\.0\.0\.0|\[?
export const isLocalEndpoint = (endpoint) =>
typeof endpoint === 'string' && LOCAL_ENDPOINT_RE.test(endpoint.trim());
-export const LOCAL_LLM_MAX_CONCURRENCY = Math.max(1, Number(process.env.LOCAL_LLM_MAX_CONCURRENCY) || 1);
+export { LOCAL_LLM_MAX_CONCURRENCY };
const localEndpointGates = new Map(); // endpoint -> { active: number, queue: (()=>void)[] }
function acquireLocalSlot(endpoint) {