Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 900+ contributors
TypeScript a2aai-agentsai-gatewayanthropicclaudeclaude-codeclinecodexcopilotcursordeepseekfree-aigeminikimillm-gatewaymcpopenaiopenai-proxyqwentoken-saver
"use strict";
/**
* HTTP client-abort / recoverable-upstream-timeout crash guard
* (#fix-dev-server-aborted, #12861).
*
* Node's http.Server turns an 'error' event on an IncomingMessage/ServerResponse
* into an uncaughtException (and therefore a process exit) WHENEVER the emitter
* has no listener. The single most common such error is a *client* abort: the
* browser closes the TCP socket (navigation, Back/Forward cache, HMR reconnect,
* cancelling a fetch) while the server is still streaming the response. Node
* emits `Error: aborted` / `ERR_STREAM_PREMATURE_CLOSE` / `ECONNRESET` on the
* request stream, and absent a handler it kills the whole server process.
*
* That surfaced as "login succeeds, then the dashboard hangs with a wall of
* `net::ERR_CONNECTION_REFUSED`": after auth the SPA opens many polling
* connections + a live WebSocket; stray client-side socket closes during
* navigation/HMR were taking the dev server down.
*
* Two more categories were added after the 2026-09-14 agnes-cn upstream storm
* produced two sibling escapes in production: an intentional combo hedge
* cancellation (`AbortError: hedge-cancelled` — the sibling leg already won,
* so the cancellation is expected, not a fault) and undici fetch failures
* (`TypeError: fetch failed` with a socket-level code) against a flapping
* upstream. Both are runtime/environmental conditions the request layer
* already handles; neither is a process-fatal logic bug.
*
* A further, unrelated category covers #12861: `directFetchWithBoundedResponseStart`'s
* response-start timeout (`DIRECT_RESPONSE_START_TIMEOUT`) is a *recoverable*
* signal `proxyFetch.ts` already retries on a fresh socket — but a narrow
* timer/promise-settlement race can still deliver its abort reason to a
* promise nobody is awaiting anymore, which otherwise kills the whole process
* over a single upstream stall that the retry path was built to handle.
*
* The same escape reached production on 2026-10-02 (exit code 7) with the
* pooled relay path's per-attempt timeout (`RELAY_TIMEOUT`,
* `open-sse/utils/proxyFetch.ts`): a stray copy of the 504 the combo/executor
* layer already handles took the process down. Both timeout codes are
* classified in `isRecoverableUpstreamTimeoutError` below.
*
* It escaped again on 2026-10-11 (exit code 7, #16347) with a stream-lifecycle
* error: an empty Claude upstream (`empty_response`, stamped by
* `open-sse/utils/stream.ts::emitClaudeEmptyStreamErrorAndAbort`) had already
* been reported as a 502 / combo failover, but while the combo loop sat in its
* retry wait nothing consumed the errored transform stream, so the rejection
* reached this guard unclassified. Those errors carry the
* `__omniroutePendingRequestCleared` marker (`PENDING_REQUEST_CLEARED_MARKER`,
* `open-sse/utils/stream.ts`) once the stream layer has cleared the request's
* pending entry — they are classified in `isHandledStreamLifecycleError`
* below.
*
* Two layers:
* 1. `attachRequestStreamGuards(req, res)` — per-request listeners that absorb
* client-abort errors so they never bubble to the process level. Call it
* inside every `http.createServer((req, res) => …)` request listener.
* 2. `installProcessCrashGuard()` — a last-resort safety net on
* `process.on('uncaughtException' | 'unhandledRejection')` that swallows
* the same benign errors but otherwise preserves the existing crash
* semantics (so genuine bugs still surface). Idempotent.
*
* Kept as a `.mjs` module (no build step) so it is importable both from the
* Node-only dev server (`scripts/dev/run-next.mjs`) and from the TypeScript
* servers under `src/` (tsconfig `allowJs: true`).
*
* @module
*/
/**
* Abort reasons raised by combo target dispatch. Mirror of
* open-sse/services/combo/comboAbortReasons.ts — keep the two in sync.
*/
const COMBO_ABORT_REASONS = new Set(["hedge-cancelled", "combo-per-model-timeout"]);
/**
* Raw string reasons open-sse/utils/streamHandler.ts passes to
* handleDisconnect() / abortController.abort() when the client goes away.
*/
const CLIENT_DISCONNECT_REASONS = new Set(["request_signal_aborted", "client_closed", "cancelled"]);
/**
* @param {unknown} err
* @returns {boolean} true when `err` represents a client closing the
* connection rather than a server-side fault.
*/
export function isClientAbortError(err) {
// open-sse/utils/streamHandler.ts aborts the stream controller with a RAW
// STRING reason (getClientAbortReason / handleDisconnect), and undici rejects
// with signal.reason verbatim, so a cancellation can surface at the process
// level as a bare string rather than an Error object.
if (typeof err === "string") {
return COMBO_ABORT_REASONS.has(err) || CLIENT_DISCONNECT_REASONS.has(err);
}
if (!err || typeof err !== "object") return false;
const e = /** @type {NodeJS.ErrnoException} */ (err);
// Node emits `Error: aborted` (no code) from http.Server#abortIncoming.
if (e.message === "aborted" || e.message === "Aborted") return true;
// OmniRoute's SSE teardown aborts in-flight legs with
// `Error [AbortError]: request_signal_aborted` on client disconnects
// (open-sse/utils/streamHandler.ts), and fetch/DOM cancellation surfaces as
// `AbortError` with an abort-flavoured message. Same benign class as
// `Error: aborted` — an emitter-left 'error' event on any of these used to
// kill the process (#fix-dev-server-aborted).
if (e.name === "AbortError" && /abort/i.test(String(e.message))) return true;
// Combo dispatch cancels a losing hedged target / a stalled target by
// aborting with `new Error(reason)` for one of the reasons in
// open-sse/services/combo/comboAbortReasons.ts (targetTimeoutRunner.ts).
// streamHandler.ts forwards that reason to the stream controller as a plain
// string, and a leaked abort listener in
// open-sse/handlers/chatCore/upstreamTimeouts.ts (executeWithUpstreamStartTimeout)
// rebuilt it via createAbortError() as an AbortError-named Error to reject a
// promise nothing was awaiting. That unhandledRejection reached this guard,
// which re-threw it as an uncaughtException. Production exit 2026-08-31:
// Error [AbortError]: hedge-cancelled
// The listener leak is fixed at the source; this stays as the last-resort net.
// A sibling target winning / a target stalling is never a server fault, so
// match the exact reason text whatever `name` the thrower stamped on it.
// Inlined rather than imported from comboAbortReasons.ts: this file runs
// under plain node (scripts/dev/run-next.mjs, no tsx) and must not depend on
// type-stripping being available for that .ts module.
if (COMBO_ABORT_REASONS.has(String(e.message))) return true;
switch (e.code) {
case "ERR_STREAM_PREMATURE_CLOSE":
case "ECONNRESET":
case "EPIPE":
case "ECONNABORTED":
case "ETIMEDOUT":
case "ENOTCONN":
case "ECANCELED":
return true;
default:
return false;
}
}
/**
* A recoverable upstream-fetch timeout whose request-layer handling already
* exists, but a stray copy can still reach this guard as an
* unhandledRejection/uncaughtException and must not be process-fatal:
*
* - `DIRECT_RESPONSE_START_TIMEOUT` (#12861): `proxyFetch.ts` retries it on a
* fresh socket (see `open-sse/utils/directResponseStartTimeout.ts`). A narrow
* timer/promise-settlement race can deliver its abort reason to a promise
* nobody is awaiting anymore.
* - `RELAY_TIMEOUT` (2026-10-02 production exit 7): the pooled relay path
* (`open-sse/utils/proxyFetch.ts`, #9100/#9158) fails a hung attempt fast as
* a 504 after `RELAY_FETCH_TIMEOUT_MS` and the combo/executor layer falls
* back. The thrown error escaped its execute chain as a stray rejection and
* this guard re-threw it because the code was not classified here.
*
* Kept as bare string-code checks (no import of the `.ts` source of truth)
* because this file has to stay build-free/plain-JS-loadable — see the module
* docstring. Canonical definitions: `DIRECT_RESPONSE_START_TIMEOUT_CODE` in
* `open-sse/utils/directResponseStartTimeout.ts`, and the `RELAY_TIMEOUT` /
* `relay_timeout` stamps in `open-sse/utils/proxyFetch.ts` (the same pair
* `src/lib/usage/glmResetCards.ts` matches on). Keep these literals in sync
* with them.
*
* @param {unknown} err
* @returns {boolean}
*/
export function isRecoverableUpstreamTimeoutError(err) {
// Same reason-shape tolerance as isIntentionalComboAbort: a bare string
// reason rejects waiters with the string itself, not an Error object.
if (
err === "DIRECT_RESPONSE_START_TIMEOUT" ||
err === "RELAY_TIMEOUT" ||
err === "relay_timeout"
) {
return true;
}
if (!err || typeof err !== "object") return false;
const e = /** @type {NodeJS.ErrnoException & { errorCode?: string }} */ (err);
// open-sse's withBodyTimeout rejects with a plain Error (no .code). Restrict
// this match to its exact name and message shape to preserve crash semantics.
if (
e.name === "BodyTimeoutError" &&
typeof e.message === "string" &&
/^Response body read timeout after [1-9]\d*ms$/.test(e.message)
) {
return true;
}
return (
e.code === "DIRECT_RESPONSE_START_TIMEOUT" ||
e.code === "RELAY_TIMEOUT" ||
e.errorCode === "relay_timeout"
);
}
/**
* Intentional combo-leg cancellation. When a combo dispatches hedged targets,
* the losing legs are aborted with a distinctive reason once a sibling wins
* (`hedge-cancelled`) or exceeds its per-model budget (`combo-per-model-timeout`)
* — see `COMBO_HEDGE_CANCELLED_REASON` / `COMBO_PER_MODEL_TIMEOUT_REASON` in
* `open-sse/services/combo/comboAbortReasons.ts` (bare literals duplicated here
* because this file must stay build-free; keep in sync). On 2026-09-14 such a
* cancellation escaped its promise chain and killed production with
* `Error [AbortError]: hedge-cancelled` — the request it belonged to had
* already completed 200 via the winning leg.
*
* Distinct from a *client* abort: only these exact reasons qualify, so an
* AbortError from an unknown subsystem still crashes loudly.
*
* @param {unknown} err
* @returns {boolean}
*/
export function isIntentionalComboAbort(err) {
const reasons = new Set(["hedge-cancelled", "combo-per-model-timeout"]);
// AbortSignal.reason is whatever was handed to abort(): a raw string
// reason rejects waiters with the string itself, not an Error object.
if (typeof err === "string") return reasons.has(err);
if (!err || typeof err !== "object") return false;
const e = /** @type {NodeJS.ErrnoException} */ (err);
if (e.name !== "AbortError") return false;
if (reasons.has(String(e.message))) return true;
const cause = /** @type {{ cause?: unknown }} */ (err).cause;
return typeof cause === "string" && reasons.has(cause);
}
/**
* A network/IO failure against an upstream or its proxy — undici surfaces it
* as `TypeError: fetch failed` (fixed message; the syscall code rides on
* `cause`) or as an error carrying a `PROXY_UNREACHABLE` / `UND_ERR_*` code.
* On 2026-09-14 one of these (`PROXY_UNREACHABLE` / ECONNRESET to
* api.agnes-ai.cn) escaped as an uncaughtException and killed production.
* The request that triggered the fetch already fails through the normal
* error path; the stray copy delivered to nobody must not be process-fatal.
*
* The "fetch failed" message match is exact on purpose: it is undici's fixed
* wrapping message, so arbitrary TypeErrors still crash loudly.
*
* @param {unknown} err
* @returns {boolean}
*/
export function isUpstreamNetworkError(err) {
if (!err || typeof err !== "object") return false;
const e = /** @type {NodeJS.ErrnoException} */ (err);
if (e.name === "TypeError" && e.message === "fetch failed") return true;
switch (e.code) {
case "PROXY_UNREACHABLE":
case "PROXY_REQUEST_FAILED":
case "UND_ERR_SOCKET":
case "UND_ERR_CONNECT_TIMEOUT":
case "UND_ERR_HEADERS_TIMEOUT":
case "UND_ERR_BODY_TIMEOUT":
case "ECONNREFUSED":
case "EHOSTUNREACH":
case "ENETUNREACH":
case "EAI_AGAIN":
return true;
default:
return false;
}
}
/**
* A stream-lifecycle error OmniRoute's own streaming pipeline already accounted
* for. `open-sse/utils/stream.ts` stamps `markPendingRequestCleared()` on every
* stream error whose pending-request entry was cleared at emit time (the empty
* Claude flush `empty_response`, the upstream-start timeout, the empty-choices
* flush, and the `streamFailureBoundary.ts` fallback) — the request layer has
* by then already surfaced the failure (502 error event, combo failover,
* single-model retry classification), so a stray copy delivered to nobody is a
* bookkeeping leftover, not a fault. Kept as literal checks because this file
* must stay build-free: canonical definitions are `PENDING_REQUEST_CLEARED_MARKER`
* and the `empty_response` stamp in `open-sse/utils/stream.ts` (keep in sync).
*
* 2026-10-11 production exit 7 (#16347): the combo retry wait left the errored
* transform stream unconsumed and the `empty_response` rejection escaped as an
* uncaughtException. Same escape class as RELAY_TIMEOUT (#15339) and
* RATE_LIMIT_QUEUE_TIMEOUT (#15706).
*
* @param {unknown} err
* @returns {boolean}
*/
export function isHandledStreamLifecycleError(err) {
if (err === "empty_response") return true;
if (!err || typeof err !== "object") return false;
const e = /** @type {Error & { code?: string; __omniroutePendingRequestCleared?: boolean }} */ (
err
);
if (e.__omniroutePendingRequestCleared === true) return true;
return e.code === "empty_response";
}
/**
* Decide whether a process-level uncaughtException/unhandledRejection should be
* swallowed (benign client-abort, or a recoverable upstream timeout the
* request layer already handles — #12861 / RELAY_TIMEOUT) or allowed to
* surface (genuine bug).
*
* Pure + exported so it can be unit-tested without poking process listeners.
*
* @param {unknown} err
* @param {string | undefined} origin Node's uncaughtException origin (e.g.
* "uncaughtException" / "unhandledRejection"); absent/empty for rejections.
* @returns {boolean} true => swallow (log only), false => re-throw / let crash.
*/
export function shouldSwallowUncaught(err, origin) {
if (
!isClientAbortError(err) &&
!isRecoverableUpstreamTimeoutError(err) &&
!isIntentionalComboAbort(err) &&
!isUpstreamNetworkError(err) &&
!isHandledStreamLifecycleError(err)
) {
return false;
}
// Only swallow when the origin matches what the guard installed for. If some
// other subsystem raised it (e.g. a deliberate `throw` in a domain), keep the
// existing crash semantics.
return !origin || origin === "uncaughtException" || origin === "unhandledRejection";
}
/**
* Attach `error` listeners to a request/response pair that swallow client-abort
* errors. Idempotent per (req, res) pair via a Symbol flag.
*
* @param {import("node:http").IncomingMessage} req
* @param {import("node:http").ServerResponse} res
*/
export function attachRequestStreamGuards(req, res) {
const flag = Symbol.for("omniroute.requestAbortGuard");
if (req[flag] || res[flag]) return;
req[flag] = true;
res[flag] = true;
req.on("error", (err) => {
if (!isClientAbortError(err)) {
// Re-emit a genuine request error through the normal channel so it is
// still observable in logs, but never as an uncaughtException.
console.error("[server] request stream error:", err);
}
});
res.on("error", (err) => {
if (!isClientAbortError(err)) {
console.error("[server] response stream error:", err);
}
});
}
let crashGuardInstalled = false;
/**
* Install process-level safety nets. Idempotent. Benign client-abort errors are
* logged once and swallowed; everything else is re-thrown on a fresh stack so
* the process keeps its current crash semantics (genuine bugs still crash/hang
* loudly, and a supervisor or test harness sees them).
*
* @param {(level: "warn" | "error", ...args: unknown[]) => void} [log]
*/
export function installProcessCrashGuard(log) {
if (crashGuardInstalled) return;
crashGuardInstalled = true;
// `console` is an object, not a callable: `log ?? console` followed by
// `logger("warn", ...)` throws TypeError and kills the process on the very
// abort the guard exists to swallow. Default to console.warn as a function.
const logger = typeof log === "function" ? log : console.warn.bind(console);
process.on("uncaughtException", (err, origin) => {
if (shouldSwallowUncaught(err, origin)) {
// The warn line is the only evidence a swallowed error ever happened;
// pass the full error object so the stack survives.
logger("warn", "[server] swallowed benign uncaughtException:", err);
return;
}
throw err;
});
process.on("unhandledRejection", (reason) => {
if (shouldSwallowUncaught(reason, "unhandledRejection")) {
logger("warn", "[server] swallowed benign unhandledRejection:", reason);
return;
}
throw reason;
});
}