Skip to content

7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

TypeScript

fix: sticky session map grows unbounded between prune passes (#475)

Semianchuk Vitalii918378e51ed3
diff --git a/server/src/routes/proxy.ts b/server/src/routes/proxy.ts
index 4d4a754c..5934d4b8 100644
--- a/server/src/routes/proxy.ts
+++ b/server/src/routes/proxy.ts
@@ -186,6 +186,17 @@ export function setStickyModel(messages: ChatMessage[], modelDbId: number, sessi
for (const [k, v] of stickySessionMap) {
if (now - v.lastUsed > STICKY_TTL_MS) stickySessionMap.delete(k);
}
+
+ // Hard cap: if still over 1000 after pruning expired entries, evict oldest by lastUsed
+ if (stickySessionMap.size > 1000) {
+ const entries = [...stickySessionMap.entries()].sort(
+ (a, b) => a[1].lastUsed - b[1].lastUsed
+ );
+ const toEvict = stickySessionMap.size - 1000;
+ for (let i = 0; i < toEvict; i++) {
+ stickySessionMap.delete(entries[i][0]);
+ }
+ }
}
}