7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.
TypeScript
feat(router): skip models whose tpm_limit can't fit the request
Robin Sverd12166bd98cad
diff --git a/server/src/services/router.ts b/server/src/services/router.tsindex 3e0e08bb..48b5cfdf 100644--- a/server/src/services/router.ts+++ b/server/src/services/router.ts@@ -445,6 +445,12 @@ export function routeRequest(estimatedTokens = 1000, skipKeys?: Set<string>, pre// gets the normal "all models exhausted" error rather than a wasted sweep.if (entry.context_window != null && estimatedTokens > entry.context_window) continue;+ // Same guard for a model with a small per-minute token budget: a single+ // request that alone exceeds tpm_limit can never fit one minute of quota and+ // returns a guaranteed 413 (e.g. Groq gpt-oss-120b: 131k context but 8k TPM).+ // estimatedTokens already includes reserved output, mirroring the check above.+ if (entry.tpm_limit != null && estimatedTokens > entry.tpm_limit) continue;+// Check if we have a provider for this platformconst provider = getProvider(entry.platform as any);if (!provider) continue;