Skip to content

Never stop coding. Free MIT AI gateway: one endpoint, 375 providers (150+ free), 1,400+ models Kimi, Claude, GPT, Gemini, GLM, DeepSeek, MiniMax. Works with Claude Code, Codex, Cursor, OpenCode, Cline & Copilot. Quota-aware auto-fallback, RTK+Caveman compression saves 15-95% tokens, MCP/A2A, Desktop/PWA. Built by 900+ contributors

fix(rate-limit): replace unsupported Bottleneck maxWait with job-level expiration (#1694)

diegosouzapw84fbfa36c6a8
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e093144770..e77bc32189 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -20,6 +20,7 @@
- **fix(codex):** prevent unexpected protocol leakage and fabricated instructions on bare chat completion requests without tools (#1686)
- **fix(executors):** truncate tools array to 128 items max in GitHub Copilot and OpenCode executors to mitigate 400 Bad Request errors from upstream (#1687)
- **fix:** add body-read timeout to prevent stuck pending requests (#1680)
+- **fix(rate-limit):** replace unsupported Bottleneck `maxWait` option with job-level `expiration` to prevent indefinite queue stalls (#1694)
- **fix:** combo retry loop stops immediately on client disconnect (499) (#1681)
- **fix(search):** support optional bearer auth for SearXNG (#1683)
- **fix(vision):** respect native GPT vision support — prevents VisionBridge from intercepting models that already handle images natively (#1678)
diff --git a/open-sse/services/rateLimitManager.ts b/open-sse/services/rateLimitManager.ts
index b95916ade0..4af4f86306 100644
--- a/open-sse/services/rateLimitManager.ts
+++ b/open-sse/services/rateLimitManager.ts
@@ -30,7 +30,6 @@ interface LearnedLimitEntry {
interface LimiterUpdateSettings {
maxConcurrent?: number | null;
minTime: number;
- maxWait?: number | null;
reservoir?: number | null;
reservoirRefreshAmount?: number | null;
reservoirRefreshInterval?: number | null;
@@ -76,7 +75,6 @@ function buildLimiterDefaults() {
reservoir: currentRequestQueueSettings.requestsPerMinute,
reservoirRefreshAmount: currentRequestQueueSettings.requestsPerMinute,
reservoirRefreshInterval: 60 * 1000,
- maxWait: currentRequestQueueSettings.maxWaitMs,
};
}
@@ -85,7 +83,6 @@ function updateAllLimiterSettings() {
limiter.updateSettings({
maxConcurrent: currentRequestQueueSettings.concurrentRequests,
minTime: currentRequestQueueSettings.minTimeBetweenRequestsMs,
- maxWait: currentRequestQueueSettings.maxWaitMs,
reservoir: currentRequestQueueSettings.requestsPerMinute,
reservoirRefreshAmount: currentRequestQueueSettings.requestsPerMinute,
reservoirRefreshInterval: 60 * 1000,
@@ -285,7 +282,21 @@ export async function withRateLimit(provider, connectionId, model, fn) {
}
const limiter = getLimiter(provider, connectionId, model);
- return limiter.schedule(fn);
+ const maxWaitMs = currentRequestQueueSettings.maxWaitMs;
+ const scheduleOpts = maxWaitMs && maxWaitMs > 0 ? { expiration: maxWaitMs } : {};
+ try {
+ return await limiter.schedule(scheduleOpts, fn);
+ } catch (err) {
+ // Bottleneck throws when a job exceeds its expiration timeout.
+ // Surface as a clear rate-limit timeout so callers can fallback.
+ if (err?.message?.includes("This job timed out")) {
+ const key = getLimiterKey(provider, connectionId, model);
+ console.log(
+ `⏰ [RATE-LIMIT] ${key} — job expired after ${Math.ceil((maxWaitMs || 0) / 1000)}s in queue, dropping`
+ );
+ }
+ throw err;
+ }
}
// ─── Header Parsing ──────────────────────────────────────────────────────────