Skip to content

7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

TypeScript

docs(health): document HEALTH_CHECK_INTERVAL_MS and supportsKeyPolling (#1611)

Tashfeen6f5d189c0148
diff --git a/.env.example b/.env.example
index aa5895a9..22530225 100644
--- a/.env.example
+++ b/.env.example
@@ -148,6 +148,12 @@ PORT=3001
# without loosening the watchdog everywhere else:
# PROVIDER_STREAM_STALL_TIMEOUT_OLLAMA=240000
+# Base cadence for the scheduled key health pass, in milliseconds (default:
+# 300000 = 5m, minimum 60000; lower or invalid values fall back to the default).
+# Each pass applies ±20% jitter. A faster cadence sends more validation traffic,
+# which costs quota where a provider's key check is metered. Read at startup.
+# HEALTH_CHECK_INTERVAL_MS=300000
+
# Base wall-clock budget for provider failover, in milliseconds (default: 45000).
# The initial attempt and one failover hop are always allowed. After that,
# exhaustion stops new attempts and can cancel an in-flight attempt still
diff --git a/docs/en/providers/02-quotas-and-cooldowns.md b/docs/en/providers/02-quotas-and-cooldowns.md
index 6b4d8789..d68b2f60 100644
--- a/docs/en/providers/02-quotas-and-cooldowns.md
+++ b/docs/en/providers/02-quotas-and-cooldowns.md
@@ -69,9 +69,10 @@ Headers and error bodies only show quota when a request happens to carry them. P
## Health checks must not burn metered quota (#882)
-Health runs a scheduled pass every 5 minutes (±20% jitter, keys validated more recently than 3.5 minutes skipped, default concurrency 8, ≥1s spacing between same-provider probes). That cadence is fine for free validation endpoints but poisonous where validation itself costs quota:
+Health runs a scheduled pass every 5 minutes (±20% jitter, keys validated more recently than 3.5 minutes skipped, default concurrency 8, ≥1s spacing between same-provider probes). That cadence is fine for free validation endpoints but poisonous where validation itself costs quota. `HEALTH_CHECK_INTERVAL_MS` (milliseconds, minimum 60000, read at startup) changes it for the whole install (#1593):
- ModelScope validates keys with a paid 1-token chat completion (`GET /v1/models` does not enforce auth), spending magic-grain quota — at the default cadence roughly 288 paid probes/key/day. Fix (#882): tokens not prefixed `ms-` are rejected locally with zero network calls, and a successful validation is cached per key for `MODELSCOPE_VALIDATE_CACHE_MS` (default 24h).
- Pollinations' public `/v1/models` answers 200 even for revoked keys, so validation targets the authenticated `/account/key`; AI Horde treats any reachable endpoint as healthy rather than spending queue slots.
+- SerpSearch, Exa and keyed Parallel have no account endpoint, so their key check is a search with no query. SerpSearch revoked a user's free account over those empty searches arriving every 5 minutes (#1599). Such providers set `supportsKeyPolling = false`: the scheduled pass validates a key once (when it has never been checked), re-asks about a key parked at `error` at most hourly, and the cooldown probe skips it. Manual "Check now" / "Check all" still validate.
The rule for contributors: **if `validateKey` consumes metered quota, it needs a cache or a free probe path** — see [03-adding-a-new-provider.md](03-adding-a-new-provider.md).
diff --git a/docs/en/providers/03-adding-a-new-provider.md b/docs/en/providers/03-adding-a-new-provider.md
index f8f5dec8..548f2439 100644
--- a/docs/en/providers/03-adding-a-new-provider.md
+++ b/docs/en/providers/03-adding-a-new-provider.md
@@ -48,7 +48,8 @@ Env-tunable timeouts come automatically via `PROVIDER_TIMEOUT_<PLATFORM>` (#547)
`validateKey` returns a boolean or `{ valid: false, error }`. Use the inherited `validationResult(res)` helper to turn 401/403 into a diagnostic failure carrying the upstream reason. Two hard rules learned in production:
1. **Never trust a public models endpoint as proof of a valid key** (pollinations #608, modelscope): if `GET /v1/models` answers 200 unauthenticated, probe something authenticated instead.
-2. **If validation consumes metered quota, cache it or find a free probe** — see [#882](02-quotas-and-cooldowns.md#health-checks-must-not-burn-metered-quota-882). Health runs every ~5 minutes against every stored key.
+2. **If validation consumes metered quota, cache it or find a free probe** — see [#882](02-quotas-and-cooldowns.md#health-checks-must-not-burn-metered-quota-882). Health runs every ~5 minutes against every stored key (`HEALTH_CHECK_INTERVAL_MS` changes that for the whole install).
+3. **If the only check is a malformed request to the billed endpoint** (an empty search, a body the API rejects), override `supportsKeyPolling` to return `false` so the health pass sends it once per key instead of every pass. Providers read repeated malformed requests as abuse: SerpSearch revoked an account over it ([#1599](02-quotas-and-cooldowns.md#health-checks-must-not-burn-metered-quota-882)).
## 5. Seed the catalog deliberately
diff --git a/docs/zh-cn/env/01-variables.md b/docs/zh-cn/env/01-variables.md
index fe7c63b0..322ceda3 100644
--- a/docs/zh-cn/env/01-variables.md
+++ b/docs/zh-cn/env/01-variables.md
@@ -68,6 +68,7 @@ FreeLLMAPI 从 `.env` 读取的全部变量,按主题分组。默认值和说
| `VALIDATE_TOOL_ARGUMENTS` | 关闭(`=1` 开启) | 对照调用方声明的 schema 校验工具调用参数,违反时触发故障转移。`lib/tool-args.ts` 已经修复它能确证可修的部分;这个开关负责剩下的漏网之鱼——它们如今到达 Anthropic 客户端时会是一个带 `input: {}` 的 `tool_use` 块。默认关闭,因为误报的代价是一跳故障转移——先打开它,盯着 `X-Fallback-Trail` 观察 `invalid_tool_arguments`,再决定去留。适用于回合仍可故障转移的所有场合(非流式端点,外加 `/v1/chat/completions`、`/v1/messages`、Gemini 与 Ollama 模拟);`/v1/responses` 的流式在第一个工具调用增量上就已提交,属于例外。运行时通过 `validate_tool_arguments` 设置键调整。 |
| `REQUEST_MAX_TOKENS_BUDGET` | `0`(关闭) | 按请求的词元护栏:估算输入词元加所请求的 max_tokens 必须落在这个上限之内,否则在任何提供方被尝试之前就以 413 拒绝请求;没有发送 max_tokens 的请求,其输出会被截断到剩余额度。可在运行时通过 `request_max_tokens_budget` 设置键调整(仪表盘 API `PUT /api/settings/guardrails`)。 |
| `MAX_CONSECUTIVE_UPSTREAM_FAILS` | `0`(关闭) | 故障转移断路器:单个请求内连续上游失败达到这个次数后,以 503 终止故障转移链,而不是在一个不健康的池子里试遍每个剩余候选。可在运行时通过 `max_consecutive_upstream_fails` 设置键调整。 |
+| `HEALTH_CHECK_INTERVAL_MS` | `300000`(5 分钟) | 定期密钥健康检查的基本间隔,单位毫秒(#1593)。每轮带 ±20% 抖动;低于 60000 或无效的值回落到 5 分钟默认值,避免配置失误把检查变成洪水。调快会增加校验流量,在校验计费的提供方会消耗额度。启动时读取。 |
| `MODELSCOPE_VALIDATE_CACHE_MS` | `86400000`(24 小时) | ModelScope(魔搭)用一次消耗额度的单词元聊天补全来校验 API 密钥(`GET /v1/models` 不做鉴权),每次探测都要花费魔粒额度——实测每个超档请求约 2 魔粒。为了不让每 5 分钟一轮的健康检查每天烧掉约 288 次付费探测,校验成功后会按密钥缓存这么长时间(毫秒)。`0` 表示每轮都探测;被撤销的密钥仍会被下一次真实请求的 401 抓住。 |
| `FREELLMAPI_CONTEXT_HANDOFF` | 关闭 | 模型切换时的上下文交接。启用后(`on_model_switch`),每当一个会话从一个模型切换到另一个(例如发生故障转移之后),FreeLLMAPI 会向出站请求注入一条精简的系统消息。 |
diff --git a/docs/zh-cn/providers/02-quotas-and-cooldowns.md b/docs/zh-cn/providers/02-quotas-and-cooldowns.md
index 51e35035..b46a7fdb 100644
--- a/docs/zh-cn/providers/02-quotas-and-cooldowns.md
+++ b/docs/zh-cn/providers/02-quotas-and-cooldowns.md
@@ -65,9 +65,10 @@
## 健康检查不得燃烧计量额度(#882)
-健康检查每 5 分钟跑一轮计划任务(±20% 抖动,3.5 分钟内校验过的密钥跳过,默认并发 8,同提供方探测之间间隔 ≥1s)。这个节奏对免费的校验端点没问题,在校验本身就耗费额度的地方却是毒药:
+健康检查每 5 分钟跑一轮计划任务(±20% 抖动,3.5 分钟内校验过的密钥跳过,默认并发 8,同提供方探测之间间隔 ≥1s)。这个节奏对免费的校验端点没问题,在校验本身就耗费额度的地方却是毒药。`HEALTH_CHECK_INTERVAL_MS`(毫秒,最小 60000,启动时读取)可以为整个实例调整该节奏(#1593):
- ModelScope 用一次消耗额度的单词元聊天补全校验密钥(`GET /v1/models` 不做鉴权),花费魔粒额度——按默认节奏约合每密钥每天 288 次付费探测。修复(#882):不带 `ms-` 前缀的令牌在本地直接拒绝、零网络调用,校验成功后按密钥缓存 `MODELSCOPE_VALIDATE_CACHE_MS`(默认 24 小时)。
- Pollinations 的公开 `/v1/models` 对已撤销的密钥也返回 200,所以校验指向需鉴权的 `/account/key`;AI Horde 则把任何可达的端点视为健康,而不去占用队列名额。
+- SerpSearch、Exa 和带密钥的 Parallel 没有账户端点,它们的密钥校验就是一次不带查询的搜索。SerpSearch 因为每 5 分钟收到这类空搜索而撤销了一位用户的免费账户(#1599)。这类提供方设置 `supportsKeyPolling = false`:计划任务只校验一次密钥(从未校验过时),停在 `error` 的密钥最多每小时重问一次,冷却探测直接跳过。手动"立即检查"/"全部检查"仍会校验。
-给贡献者的规则:**如果 `validateKey` 会消耗计量额度,就需要缓存或一条免费的探测路径** ——见 [03-adding-a-new-provider.md](03-adding-a-new-provider.md)。
\ No newline at end of file
+给贡献者的规则:**如果 `validateKey` 会消耗计量额度,就需要缓存或一条免费的探测路径** ——见 [03-adding-a-new-provider.md](03-adding-a-new-provider.md)。
diff --git a/docs/zh-cn/providers/03-adding-a-new-provider.md b/docs/zh-cn/providers/03-adding-a-new-provider.md
index d1e8854f..db4674f5 100644
--- a/docs/zh-cn/providers/03-adding-a-new-provider.md
+++ b/docs/zh-cn/providers/03-adding-a-new-provider.md
@@ -48,7 +48,8 @@ register(new OpenAICompatProvider({
`validateKey` 返回布尔值或 `{ valid: false, error }`。用继承来的 `validationResult(res)` 助手把 401/403 变成携带上游原因的诊断性失败。两条从生产中学到的铁律:
1. **绝不要把公开的模型列表端点当成密钥有效的证据**(pollinations #608、modelscope):如果 `GET /v1/models` 未鉴权也返回 200,就改为探测某个需要鉴权的东西。
-2. **如果校验消耗计量额度,就缓存它或找一条免费探测** ——见 [#882](02-quotas-and-cooldowns.md#健康检查不得燃烧计量额度882)。健康检查每约 5 分钟对每条已存密钥跑一轮。
+2. **如果校验消耗计量额度,就缓存它或找一条免费探测** ——见 [#882](02-quotas-and-cooldowns.md#健康检查不得燃烧计量额度882)。健康检查每约 5 分钟对每条已存密钥跑一轮(可用 `HEALTH_CHECK_INTERVAL_MS` 为整个实例调整)。
+3. **如果唯一的校验方式是向计费端点发送一个畸形请求**(空搜索、API 会拒绝的请求体),就把 `supportsKeyPolling` 重写为返回 `false`,让健康检查对每个密钥只发送一次,而不是每轮都发。提供方会把反复出现的畸形请求视为滥用:SerpSearch 曾因此撤销了一个账户([#1599](02-quotas-and-cooldowns.md#健康检查不得燃烧计量额度882))。
## 5. 有意识地播种目录