Skip to content

7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.

TypeScript

docs: add root GLOSSARY.md for agent navigation (#1375)

Chong Jia Zhen1c252b6252a8
diff --git a/GLOSSARY.md b/GLOSSARY.md
new file mode 100644
index 00000000..9ef62f4e
--- /dev/null
+++ b/GLOSSARY.md
@@ -0,0 +1,117 @@
+# Glossary
+
+Shared vocabulary for working in this repo. Read this before touching
+`server/src/services/router.ts`, `server/src/services/ratelimit.ts`,
+`server/src/lib/fallback-loop.ts`, `server/src/providers/*`, or anything
+under `server/src/services/model-*` / `custom-model-*`.
+
+Deep dives live under `docs/en/` — this file defines terms, it does not
+repeat implementation. When this file and a deep dive disagree, the deep
+dive wins; fix this file.
+
+## What this thing is
+
+- **Gateway**: the whole product. One self-hosted proxy that stacks many
+ providers' free tiers behind one OpenAI-compatible endpoint.
+- **Pooled fallback chain**: the gateway's single logical model. Per request
+ the router walks an ordered chain of models until one answers or the
+ time budget runs out.
+
+## The four nouns agents mix up
+
+- **Platform**: a named upstream, e.g. `google`, `groq`, `openrouter`.
+ The closed list lives in `shared/types.ts` (`Platform`).
+- **Provider adapter**: the code file for a platform
+ (`server/src/providers/<platform>.ts`). Most extend
+ `OpenAICompatProvider`; a few with odd wire protocols extend
+ `BaseProvider` directly.
+- **Model row**: one routable `(platform, model_id)` entry in the DB,
+ plus its limits, capabilities, and score inputs.
+- **Key**: one stored upstream credential for a platform. A model routes
+ only when it has at least one healthy, under-limit key.
+
+## Chains, pools, catalog
+
+- **Chain**: the ordered list of model rows tried for one request.
+ Source, in priority order: active profile → global fallback config →
+ `auto:` sort alias → profile by name.
+- **`auto:<name>`**: a named chain exposed as a selectable model
+ (`auto:fast`, `auto:smart`, …).
+- **Profile**: a named chain saved in the `profile_models` table
+ (e.g. "coding", "long-context").
+- **Pool key**: the shared quota bucket keys draw from
+ (e.g. `openrouter::free`). Keys in one pool share one allowance.
+- **Catalog model**: a row delivered by the signed catalog sync.
+- **Custom / relay model**: a row pointing at a user-supplied `base_url`
+ (`endpoint scope` `custom:<hash>`). Same `model_id` on two relays
+ is two different rows.
+- **Model-age gate**: new catalog rows serve as `premium` for 30 days,
+ then become `free` on the next signed sync.
+- **Tombstone / override / retirement**: the three ways a catalog row
+ goes away or is edited without editing the catalog: user-hidden,
+ upstream end-of-life, or a local field override.
+
+## Routing and quota
+
+- **Router**: picks one `(platform, model, key)` per request. A
+ Thompson-sampling bandit over reliability / speed / intelligence,
+ multiplied by headroom guardrails — not round-robin.
+- **Strategy**: the weight preset (`balanced`, `smartest`, `fastest`,
+ `reliable`, `custom`, `priority`). Key selection (`auto` /
+ `least-remaining`) is independent of it.
+- **Headroom**: remaining quota as a 0–1 factor. Steers traffic away
+ before a cap hits; it never lets a request through a real limit.
+- **Quota windows**: RPM / TPM (sliding minute) and RPD / TPD (UTC day).
+ Plus provider-wide pools (one account-wide cap across many models)
+ and in-flight leases (in-flight requests count before they finish).
+- **Hard gate vs steering**: `canMakeRequest` / `canUseTokens` say
+ yes/no on the hot path. The 5-second usage snapshot only steers
+ ranking. Do not treat the snapshot as permission.
+- **Cooldown**: a benched `(model, key)` after 429/5xx/auth failures.
+ `heuristic` (our guess, probe-eligible) vs `authoritative` (provider
+ Retry-After) vs `credit` (402) vs `tier` (403). Escalates
+ 2 min → 10 min → 1 h → 24 h.
+- **Fallback loop**: the shared retry driver every route uses
+ (`server/src/lib/fallback-loop.ts`). Up to ~20 attempts inside one
+ wall-clock budget; a stalled first byte cancels the attempt.
+- **Degraded mode**: quota exhausted at the top of the chain, so the
+ endpoint serves the next healthy tier instead of failing. Resets at
+ UTC midnight.
+- **Tool-call rescue**: models that emit tool calls as plain text are
+ converted to real `tool_calls`. Tool requests route only to models
+ that support tools.
+- **Health**: background probe keeping key status fresh. `UNKEYED` =
+ no key yet; `KEYED-NO-ROWS` = key stored but zero routable rows;
+ `serving` = key + rows + traffic.
+
+## Repo map
+
+- `server/` — Express proxy (`:3001`), router, quota, providers, DB.
+- `client/` — React + Vite dashboard.
+- `shared/` — `types.ts`, notably the `Platform` union.
+- `cli/`, `desktop/` — CLIs and desktop shell, not the hot path.
+- `docs/en/architecture/` — router, quota/cooldown, streaming,
+ degraded mode, catalog sync, observability.
+- `docs/en/providers/` — per-platform quotas, adding a provider.
+- `docs/en/glossary/01-glossary.md` — deeper term table with code refs.
+- `server/src/__tests__/` — unit + integration + route tests.
+
+## Confusables
+
+- Platform ≠ provider adapter ≠ model ≠ key.
+- Chain (ordered tries) ≠ pool (shared quota bucket).
+- Catalog listing (what exists) ≠ chain (what this request tries).
+- Custom model (user relay) ≠ premium model (new catalog row).
+- Cooldown (benched after failure) ≠ quota (allowance not yet spent).
+- Headroom (steering factor) ≠ reliability (bandit posterior).
+- Profile (saved chain) ≠ strategy (score weights).
+
+## Contributing pointers
+
+- One change per PR; keep `npm test` green; add a test with the fix.
+- Never invent provider limits, model ids, or endpoints — verify
+ against the provider.
+- Schema changes are file-per-migration under
+ `server/src/db/migrations/`; never edit an applied migration.
+- `client/src/i18n/locales/en.json` is the i18n source of truth
+ (`npm run check:i18n` in `client/`).