Skip to content

7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

TypeScript

fix(proxy): honor long caller timeouts on the SOCKS path without lowering the stream guard (#726)

suanteac0c859c01459
diff --git a/server/src/__tests__/lib/proxy.test.ts b/server/src/__tests__/lib/proxy.test.ts
index 1fbedcc9..7d1e661f 100644
--- a/server/src/__tests__/lib/proxy.test.ts
+++ b/server/src/__tests__/lib/proxy.test.ts
@@ -156,6 +156,66 @@ describe('proxyFetch dispatcher selection for SOCKS schemes (#630)', () => {
}
});
+// #666: the SOCKS fallback hardcoded a 120s socket timeout, so a user with
+// PROVIDER_TIMEOUT_CUSTOM=600000 still lost the request at 120s. The fix only
+// ever RAISES that guard — it never lowers it to the caller's timeout, because
+// http.request's `timeout` is a socket inactivity timer armed across the whole
+// streaming body while timeoutMs is a header deadline disarmed at headers
+// (#553/#584 hand mid-stream time to the stall watchdog).
+describe('SOCKS socket timeout guard (#666)', () => {
+ const socketTimeoutOf = (spy: ReturnType<typeof stubHttpsRequest>): unknown =>
+ (spy.mock.calls[0][0] as any).timeout;
+
+ beforeEach(() => {
+ applyProxyUrl('socks5://127.0.0.1:1080');
+ });
+
+ it('raises the guard past 120s for a long caller timeout, with a grace margin', async () => {
+ const reqSpy = stubHttpsRequest();
+
+ await proxyFetch('https://api.example.com/v1', { method: 'POST' }, 'groq', 'chat', 600_000);
+
+ // +30s so the caller's own abort deadline always fires first and the
+ // tagged AbortError survives instead of the bare socket 'timeout'.
+ expect(socketTimeoutOf(reqSpy)).toBe(630_000);
+ });
+
+ it('never drops below 120s for a platform with a short chat timeout', async () => {
+ const reqSpy = stubHttpsRequest();
+
+ await proxyFetch('https://api.example.com/v1', { method: 'POST' }, 'groq', 'chat', 15_000);
+
+ // A 15s socket idle timer would kill a healthy stream mid-prefill; the
+ // ~90s stall watchdog stays the governing mid-stream budget.
+ expect(socketTimeoutOf(reqSpy)).toBe(120_000);
+ });
+
+ it('keeps the historical 120s guard when no timeout is passed', async () => {
+ const reqSpy = stubHttpsRequest();
+
+ await proxyFetch('https://api.example.com/v1', { method: 'POST' }, 'groq');
+
+ expect(socketTimeoutOf(reqSpy)).toBe(120_000);
+ });
+
+ it('disables the socket timer when the caller timeout is 0 (no timeout)', async () => {
+ const reqSpy = stubHttpsRequest();
+
+ await proxyFetch('https://api.example.com/v1', { method: 'POST' }, 'groq', 'chat', 0);
+
+ expect(socketTimeoutOf(reqSpy)).toBe(0);
+ });
+
+ it('falls back to the 120s guard for malformed timeouts instead of disabling it', async () => {
+ for (const bad of [NaN, -1, Infinity, '300000' as unknown as number]) {
+ const reqSpy = stubHttpsRequest();
+ await proxyFetch('https://api.example.com/v1', { method: 'POST' }, 'groq', 'chat', bad);
+ expect(socketTimeoutOf(reqSpy)).toBe(120_000);
+ vi.restoreAllMocks();
+ }
+ });
+});
+
// #353: HTTPS_PROXY / HTTP_PROXY / ALL_PROXY / NO_PROXY are the de-facto
// standard for every other CLI on the box. Honouring them means a user who
// already exported them for curl/git gets a working app with no extra config —
diff --git a/server/src/lib/proxy.ts b/server/src/lib/proxy.ts
index 716ced16..fed959be 100644
--- a/server/src/lib/proxy.ts
+++ b/server/src/lib/proxy.ts
@@ -339,6 +339,27 @@ function socksFetch(
const signal = init?.signal;
const startedAt = Date.now();
+ // Socket guard for the SOCKS fallback path — deliberately NOT `timeoutMs`.
+ // The two clocks measure different things: http.request's `timeout` is a
+ // socket INACTIVITY timer that stays armed across the whole streaming body,
+ // while `timeoutMs` is a header/request deadline the caller disarms the
+ // moment response headers arrive (providers/base.ts fetchWithTimeout). Mid-
+ // stream time is owned by the stall watchdog and the first-byte grace
+ // (#553/#584, default 90s), so pinning the socket timer to a platform's
+ // 15-60s chat timeout would kill healthy streams during prefill.
+ //
+ // So this only ever RAISES the historical 120s floor (#666): a user with
+ // PROVIDER_TIMEOUT_CUSTOM=600000 no longer dies at 120s, and the +30s grace
+ // keeps the caller's abort firing first so the tagged AbortError (see
+ // enrichAbort) survives instead of the bare socket 'timeout'. 0 means "no
+ // timeout" (provider semantics); undefined or malformed input falls back to
+ // the 120s guard rather than disabling it.
+ const socketTimeoutMs = timeoutMs === 0
+ ? 0
+ : typeof timeoutMs === 'number' && Number.isFinite(timeoutMs) && timeoutMs > 0
+ ? Math.max(timeoutMs + 30_000, 120_000)
+ : 120_000;
+
// What to reject with when the signal fires. A client-caused abort carries
// its own marked reason (newClientAbortError in lib/error-classify.ts) —
// preserve it so the failure isn't misclassified downstream as a provider
@@ -360,7 +381,7 @@ function socksFetch(
agent,
servername: isTls ? url.hostname : undefined,
rejectUnauthorized: true,
- timeout: 120_000,
+ timeout: socketTimeoutMs,
}, (res) => {
if (signal?.aborted) {
res.destroy();