Self-hosted, Self-Improving, Federated Everything App — dev management, agent orchestration, persistent mind, digital twin, second brain, and creative pipeline from a single dashboard over Tailscale to port the user into the OS
AI Toolkit (server/lib/aiToolkit/)
The AI provider/runner/prompt toolkit is vendored in-tree here. (It was previously the portos-ai-toolkit npm package.) Keep the directory self-contained — no imports out to other PortOS modules — so future upstream syncs don't fight local edits.
Key points:
index.jsexportscreateAIToolkit,createProviderStatusService, the four Router factories (providers / runs / prompts / providerStatus), and the context-fit pairknownContextWindow/contextWindowRejection— the host applies the same rule to the EXPLICITLY requested provider, whichgetFallbackProvidernever sees. Both answernullfor an unknown window, which every caller must keep reading as "no constraint"; nothing infers a window from a model id.- The Ollama
num_ctxceiling is declared ONCE, ininternal/ollamaBacked.js(clampToRuntimeContextWindow), beside the predicate that gates it.knownContextWindowapplies it through that helper and must never restate theMath.mininline — the host's prompt budgeter (server/services/stageRunner.js#effectiveContextWindow), the untrusted-content abuse guard (server/services/untrustedContent.js#runUntrustedContentAnalysis, which held its own hand-rolledMath.minpair until #7473) and the browser's provider-card meter call the same function, and when the rule was stated twice they drifted (#7466). It is an upper BOUND over every planning rung, a user's explicitcontextWindowincluded, and the answer outright when nothing else declared one. Host callers import theinternal/leaf directly, orknownContextWindow/contextWindowRejectionstraight offproviderStatus.js(both deep imports skip the public barrel'sfs/child_processreach):stageRunner.jssits inside the server suite's import budget, anduntrustedContent.js— a request-path module — deliberately importsproviderStatus.jsrather thanstageRunner.js, whose own closure would cost that budget ~40 extra module instantiations per reaching test file for no behavior it needs.- A rung of that ceiling this directory cannot resolve arrives as DATA, never as an import out. PortOS also sets the daemon-wide window through the ambient
OLLAMA_CONTEXT_LENGTH(server/lib/ollamaContext.js#resolveOllamaContextLength), which the toolkit may not reach and the browser cannot read at all — so the host resolves it and stamps it onto an in-memory provider projection withwithRuntimeContextWindow, declared in the same file as the clamp that reads it.runtimeContextWindowranks BELOWprovider.numCtx, exactly asresolveOllamaContextLengthranks the two, and is never persisted toproviders.json(it describes the process running right now). Adding another host-resolved rung follows the same shape: stamp a field beside the reader, do not import out. Precedent: #7472.
- A rung of that ceiling this directory cannot resolve arrives as DATA, never as an import out. PortOS also sets the daemon-wide window through the ambient
- An Ollama API run with a
numCtxstreams from the native/api/chat, not/v1/chat/completions(internal/ollamaNativeChat.js). Ollama's OpenAI-compatible endpoint has no context-window knob — it ignoresnum_ctxtop-level and underoptions— so a/v1run loads the model at the daemon default and silently keeps only the prompt's tail. The native NDJSON frames are reshaped into the OpenAI chunk the runner already reads, so cancellation, stall bounds, reasoning salvage and finalization stay shared; runs withoutnumCtxand every non-Ollama endpoint stay on/v1. - Provider configuration (models, tiers, fallbacks) is persisted and migrated by
internal/providerServiceState.js; the public facade isproviders.js loadProviders()auto-migrates legacy codex configs to thecodex-configured-defaultsentinel;server/index.jswarms it at startup so the rewrite happens before any request- PortOS extends toolkit routes in
server/routes/providers.jsfor vision testing and provider status (status routes live in PortOS, not the toolkit, because they call PortOS-side socket helpers) - When adding new provider fields (e.g.,
fallbackProvider,lightModel), updatebuildProviderRecord()ininternal/providerServiceState.jsand the corresponding validation schema updateProvider()uses spread so existing providers preserve custom fields, butbuildProviderRecord()has an explicit field list- Refreshing more than one provider goes through
refreshProviderModelsBatch(ids), never a loop overrefreshProviderModels(id). Every single-provider refresh ends insaveProviders, which invalidates the read cache and rewrites all ofproviders.json; an N-provider fan-out that way is N full-file writes, each superseded by the next. The batch form groups byollamaRefreshGroupKey(one probe per daemon + probe shape), probes each group's lead without persisting, then applies every result and saves exactly once. It never throws for a per-provider failure — each group carries its ownstatus(updated/failed/missing) so the host logs one line per group rather than one per member. PortOS's post-install fan-out (refreshOllamaBackedProvidersinserver/services/localLlm.js) is the worked example.
A provider's MODEL ACCESS policy scopes on READ, never at refresh. internal/modelAccess.js owns which of an upstream's advertised catalog an install is entitled to run — the thing a /models response cannot tell you (NVIDIA NIM answers with its whole product line and no tier field, so its free endpoints are indistinguishable over the wire). Three rules hold it together, and each has already been the bug the other way round:
- The stored catalog is never narrowed.
applyModelAccessruns at the route (routes/providers.jshere,presentProviderin the host), soproviders.jsonkeeps the real list and a cleared policy restores it with no re-probe. The scoped payload carries the original asmodelCatalog— the provider editor seeds its model textarea from that, because it saves the box verbatim and would otherwise persist the narrowed list over the real one. alland an empty pattern list both mean NO CONSTRAINT (the root AGENTS.md sentinel rule). A mode picked before any pattern is typed is a half-finished edit, not an instruction to blank every model picker in the app. A stored{ mode: 'all', patterns: [...] }is a PARKED list and must keep its patterns.- A model the record NAMES is never hidden (
defaultModel, the four tiers,fallbackModel) — a picker whose stored value is missing from its options renders blank and re-points the provider on the next save.
Inheritance mirrors withGatewayApiKey: a gateway-backed wrapper with no policy of its own takes the sibling gateway record's, resolved on read into modelAccessEffective / modelAccessSource — two derived field names, not the persisted modelAccess, because that one IS in providerSchema and a client echoing a GET back on a PUT would pin the inherited value forever. Composition order against withGatewayApiKey is load-bearing (that helper attaches the key NON-enumerably, so a spread after it drops the credential). A write response carries the resolution forward from the read the route already performed (withResolvedModelAccess) rather than re-reading — several suites drive those routes with a service double whose getProviderById answers a different fixture. Glob matching is a hand-rolled iterative matcher, not a translated RegExp: the patterns are user input (*a*a*a*b backtracks exponentially), and a translation would have to re-inline the escape server/lib/textUtils.test.js forbids — which this directory cannot import its way out of. Contract and field list: docs/MODEL_ACCESS.md.
Provider-service extension point. createProviderService / createAIToolkit accept onProvidersSaved(data), invoked after every SUCCESSFUL providers.json write with the data that landed. PortOS injects it in server/services/bootstrap.js to keep its machine-local provider connection graph (server/services/providerGraph.js, #6367) reconciled with a file that a direct PUT /api/providers/:id, a model refresh, a migration or a downgraded release may also write. The hook is deliberately non-fatal: the write already landed and the cache already reflects it, and callers include boot warmups and schedulers with no next(err) to bubble to, so a throwing hook is logged and swallowed. Pair it with applyProviderPatches(patches), which applies partial updates to SEVERAL named records in ONE write and — unlike updateProvider — performs no sibling fan-out, because a projection must touch exactly the routes it named and no conventional neighbour.
Host run admission. createAIToolkit accepts withRunAdmission(handler) for the
manual create-run route. The wrapper owns the complete async handler lifetime,
including preparation after an HTTP disconnect; do not release it on response close.
API execution also accepts onPersistenceFailure and onSettled callbacks. The latter
fires after both stream cleanup and terminal persistence/hooks, independently of
onComplete (which can precede transport settlement on timeout). These are host hooks,
not toolkit imports of a host's workflow policy.
Runner extension points. The runner exposes a small declared override surface (in runner.js) so the host (PortOS) supplies its own runners without reaching into private internals:
setCliRunner(fn)/setTuiRunner(fn)— register host CLI/TUI runners (passnullto revert). PortOS calls both inserver/index.js: its CLI variant is stdin-based and knows the per-CLI argv conventions (Codexexec -, Antigravityagy --print, Claude Code-p -); the TUI runner has no toolkit built-in, andsetTuiRunnerattaches/detachesexecuteTuiRunso the runs router'stypeof runnerService.executeTuiRun === 'function'gate stays honest.registerExternalRun(runId, killable)/unregisterExternalRun(runId)/hasExternalRun(runId)— track host-spawned child processes / ptys. The toolkit's ownstopRun,isRunActive, anddeleteRunconsult this registry first, so a host runner that spawns its own process MUST register it here (PortOS does this inservices/runner.js) — otherwise the runs router reports live runs as inactive and refuses to stop them, and deleting a live run leaks a zombie process.stopRunalso stamps a one-shot marker consumed throughconsumeExternalRunStop(runId)before unregistering, which lets the host classify the resulting signal as cancellation instead of a provider failure. That classification rides out to callers ascanceled: trueon the run metadata andcode: 'RUN_CANCELED'on the rejection the prompt runner raises — anything that turns an LLM failure into a verdict (a pause kind, a filed gap/investigation task, a provider bench, anerrorstatus) must callisRunCanceledErrorfromerrorDetection.jsfirst, or it files bug reports about a human pressing Stop.executeApiRunkeeps its own equivalent marker internally, because its killable is anAbortControllerwhose reason-lessabort()reaches the finalizer as Node's bareAbortError: This operation was aborted— a message no pattern matches, so every Stop of an API run finalized as an UNKNOWN provider failure, firedonRunFailed, and was escalated to a tier-4 investigation task. It now writes the samecanceled/completionReasonterminal shape the CLI path does and fires no failure hook; it fireshooks.onRunCanceled?.({ runId })instead, so a host can drop the run from its activity snapshot without benching the provider. A timeout still outranks it, since the timer's own abort is what ended that run. The cancel finalizer performs the same reasoning salvage the success/timeout/mid-stream-throw paths do — stopping a reasoning model mid-hidden-channel otherwise persistedoutputSize: 0and discarded every token it had produced, which reads downstream as a provider that answered nothing. No sibling-method monkey-patching is needed: adding a new lifecycle method (e.g.pauseRun) just means having it consultexternalRunstoo.createProviderStatusService({ prerequisitesMet })— host gate for "can this provider run AT ALL?" (its CLI binary present).getFallbackProviderconsults it on every candidate: task-level, configured, and system-priority alike. Injected rather than computed here because the answer needs host I/O (a PATH probe) and this directory stays self-contained; PortOS suppliesprerequisitesMetForRoutingfromservices/providerPrerequisites.jsinbootstrap.js. It MUST be synchronous, and a missing / throwing / non-boolean answer deliberately reads as "yes" — an unknown answer has to route exactly as it did before, because emptying the fallback chain is a far worse failure than the late ENOENT the gate exists to prevent.- Time-based state transitions need read-side mirrors.
providerStatus.init()clears expiredestimatedRecoveryentries; every reader (getStatus,getAllStatuses,isAvailable) must re-apply the same recovery check on read — otherwise providers stay "unavailable" past their recovery deadline until the next process restart.
Pre-header replay has TWO policies, and the narrow one is not caller-optional (internal/preHeaderRetry.js). allowReplay — which only isReplaySafeLocalRequest (loopback host, no API key) may grant — covers transient gateway statuses and the whole reset/timeout transport family, any of which the upstream may already have processed; replaying one against a billable provider can bill a second generation. An HTTP/2 GOAWAY is the one exception and is replayed once regardless of allowReplay and of maxElapsedMs, because HTTP/2 §6.8 defines the frame as the peer declaring it will not process streams above lastStreamID — the request provably never ran. Gating it behind allowReplay meant a remote gateway recycling an idle pooled connection on its own schedule (NVIDIA NIM does) killed the run in ~0ms with a bare UND_ERR_SOCKET. maxElapsedMs bounds the BROAD policy only — it runs from the first attempt, so the dead attempt's own duration spends it, and charging the GOAWAY replay to it reinstated the same failure from the other end: a gateway draining a connection it had held open for ~5s blew the 2s budget, the one replay never fired, and a nemotron persistent-mind run died in 5.1s as a network-error. Time-to-GOAWAY is a property of the peer's drain schedule, so gating on it made recovery less likely the longer the peer held the stream; MAX_GOAWAY_REPLAYS and maxAttempts are the only bounds that replay answers to, and it logs a line when it fires, because a run that died on a GOAWAY otherwise reads identically whether the replay failed or never ran. Adding another code to REPLAY_SAFE_TRANSPORT_CODES widens the LOCAL-only set; only a failure carrying GOAWAY's not-processed guarantee belongs in isUnprocessedGoawayError. analyzeError classifies both as NETWORK_ERROR (2m bench) rather than UNKNOWN, so a connection fault that survives its replay reads as connectivity instead of escalating as an unexplained one.
provider.timeout is a NO-PROGRESS bound in executeApiRun, not a wall clock. A streaming run carries two ceilings, and conflating them is what #7560 fixed: one wall-clock timer could not tell a provider that opened the stream and stalled from one that was actively streaming and merely slow, so a reasoning model spending its whole budget in the hidden channel (NVIDIA NIM's nemotron sends 126 of 128 frames as reasoning_content) was killed mid-generation at 300s alongside genuinely dead upstreams.
- Stall bound =
timeout || provider.timeout || DEFAULT_API_RUN_TIMEOUT_MS, re-armed by everyreader.read()that yields bytes. It is the hang detector, and it cuts a silent provider off at exactly the moment the single timer did — reinterpreting the field this way costs no protection, because catching a hung provider is the only job that timer ever actually did. - Absolute cap =
Math.max(DEFAULT_API_RUN_MAX_RUNTIME_MS, stallBound), never extended, so a provider trickling one byte per minute still cannot hold itsactiveRunsslot forever. TheMath.maxis load-bearing: an install that deliberately raisedprovider.timeoutpast the default cap must keep running as long as it asked to, never be clamped DOWN by a ceiling it never configured.
A THIRD ceiling lives under both of them, and it is not PortOS's. undici — the client behind Node's built-in fetch — arms its own headersTimeout and bodyTimeout on every request, both defaulting to 300000ms: the same number as DEFAULT_API_RUN_TIMEOUT_MS, one layer closer to the socket. The transport sees a byte before the read loop does, so its timer re-armed fractionally earlier and fired fractionally first, and it won every race the two bounds above were meant to decide. A provider that opened the stream and went quiet finalized as network-error ("Check network connectivity and provider endpoint URL") with no timeoutBound stamped — so the bench decision never saw the stall it exists to catch — and a provider.timeout raised past five minutes was silently capped at undici's. An NVIDIA NIM nemotron persistent-mind run died that way at 419.9s with terminated: UND_ERR_INFO: HTTP/2: "stream timeout after 300000". The streaming fetch now carries the dispatcher from internal/streamTransport.js, which disables both (0 = no timer) and leaves the run's declared bounds authoritative. That module is also where the undici-major split is decided: Node 26 bundles undici 8 and takes the bare Agent, Node 22/24/25 bundle undici 6/7 and need Dispatcher1Wrapper (a bare undici-8 dispatcher is rejected there with UND_ERR_INVALID_ARG). The split is not cosmetic — the wrapper forces allowH2: false, and HTTP/1.1 has no GOAWAY, the one frame internal/preHeaderRetry.js may replay against a billable provider. Each runtime keeps exactly the wire protocol its own fetch already negotiated; only the timeouts change.
Only the STREAMING path reads the field this way. aiProvider.js, askService.js and services/runner.js are non-streaming and keep reading it as a wall clock, which is correct there — no persisted field changed shape, so no migration and no schema parity work is owed.
Both numbers and the Math.max live in internal/runTimeouts.js, a leaf module with no imports, because a host that wraps executeApiRun in its own backstop timer must arm it against the ABSOLUTE bound — which apiRunAbsoluteTimeoutMs(stallBound) exists to answer, and which a host reached by hundreds of suites must be able to import without dragging runner.js's fs/child_process closure (server/AGENTS.md → Import scoping). promptRunner.js armed its backstop at provider.timeout + 2s and so killed every healthy streaming run at the old ceiling, making the 30-minute cap unreachable and reinstating precisely what #7560 removed. Worse, that backstop kills via stopRun, so the run finalized as a USER CANCEL: onRunFailed never fired (no bench, no investigation task) and isRunCanceledError callers skipped the fallback cascade — a productive run silently produced nothing. Sixteen NVIDIA NIM nemotron persistent-mind runs died that way at exactly 302s in the two days after #7560 merged, and none before (#7665). Derive the number from the exported helper rather than restating the Math.max, so the two cannot drift apart again.
Both bounds finalize through the same markSettled() / finalizeTimeout(bound) pair, so the single-settlement gate and Stop-marker handling are shared. All three non-success terminal paths — cancel, timeout, mid-stream throw — open their record through openTerminalMetadata(), which salvages the reasoning channel, persists output.txt, and stamps outputSize / hadReasoning / usedReasoningAsFallback. Stating that sequence per-path is how the cancel path went without the salvage entirely; a fourth terminal path must reuse the helper rather than restate it. finalizeTimeout names which bound fired in the message and in metadata.timeoutBound ('stall' | 'absolute'), because the two are different diagnoses — a provider that went quiet is a bench candidate, one that outran its budget while producing is not. Neither message may contain the literal "timeout": analyzeError's NETWORK_ERROR pattern matches that substring and is tested BEFORE the timeout pattern, so it would misclassify the run as a connectivity fault. "timed out" is the phrase that reaches ERROR_CATEGORIES.TIMEOUT.
The mid-stream throw is classified through describeTransportError(err), never err.message — the same rule the pre-header fetch already follows, applied to the half of the run that reads the body. A peer that drops the connection mid-body rejects the reader with undici's opaque TypeError: terminated and hangs the actionable code (UND_ERR_SOCKET / other side closed) off .cause, so classifying the bare message matched no pattern: an NVIDIA NIM nemotron persistent-mind run that streamed reasoning for 319s and then lost its socket persisted errorCategory: null, which the host cascade reads as UNKNOWN and escalates to a tier-4 investigation task instead of the bounded retry a connectivity fault earns. A rejection with no .cause flattens to exactly the message it always was, so the helper is a safe superset for the non-transport throws (a bad frame, a finalization failure) that reach the same catch. analyzeError also recognizes a cause-less bare terminated, anchored to the WHOLE string because that scan reads an agent's entire CLI screen, where "… terminated" is ordinary prose.
The worked example for "barrel + documented exports" (see root AGENTS.md → Module Organization) is this directory's index.js.