7.4 billion tokens per month. 34 free LLM providers. 635 free model endpoints. All behind one /v1 endpoint, plus any custom OpenAI-compatible endpoint. Smart routing, automatic failover, encrypted keys. Personal experimentation only.
TypeScript
docs: document Anthropic Messages API + media endpoints in README (#366)
Tashfeen51e8aff46c1a
diff --git a/README.md b/README.mdindex cf883418..4bde243e 100644--- a/README.md+++ b/README.md@@ -85,6 +85,8 @@ Plus a **custom** provider — point at any OpenAI-compatible endpoint (llama.cp- **OpenAI-compatible** — `POST /v1/chat/completions` and `GET /v1/models` work with the official OpenAI SDKs and any OpenAI-compatible client (LangChain, LlamaIndex, Continue, Hermes, etc.). Just change `base_url`.- **Responses API** — `POST /v1/responses` (the wire format current Codex CLI versions require) is implemented as a translating shim over the same router, with full streaming events and tool calls.+- **Anthropic Messages API** — `POST /v1/messages` (plus `/v1/messages/count_tokens`) speaks Anthropic's wire format over the same router, so **Claude Code** and the official Anthropic SDKs run against your free pool. `GET /v1/models` is content-negotiated (Anthropic shape when the client sends `anthropic-version`, OpenAI shape otherwise), and Claude families (`opus` / `sonnet` / `haiku` / `default`) map to `auto` or a pinned model on the Keys page. See [Anthropic / Claude clients](#anthropic--claude-clients).+- **Image generation & text-to-speech** — `POST /v1/images/generations` and `POST /v1/audio/speech` route across the providers that serve media models. Browse and toggle them on the dashboard's **Models → Image / Audio** tabs.- **Streaming and non-streaming** — Server-Sent Events for `stream: true`, JSON response otherwise. Every provider adapter implements both.- **Tool calling** — OpenAI-style `tools` / `tool_choice` requests are passed through, and assistant `tool_calls` + `tool` role follow-up messages round-trip across providers.- **Embeddings** — `/v1/embeddings` with family-based routing: failover only ever happens between providers serving the *same* model (vectors from different models are incompatible), never across models. See [Embeddings](#embeddings).@@ -104,8 +106,6 @@ Plus a **custom** provider — point at any OpenAI-compatible endpoint (llama.cpThe scope is deliberately narrow. If a feature isn't on this list and isn't below, assume it isn't there yet.-- **Image generation** (`/v1/images/*`)-- **Audio / speech** (`/v1/audio/*`)- **Legacy completions** (`/v1/completions`) — only the chat endpoint is implemented- **Moderation** (`/v1/moderations`)- **`n > 1`** (multiple completions per request)@@ -248,7 +248,7 @@ signing involved. Full instructions in [desktop/README.md](./desktop/README.md).## Using the API-Any OpenAI-compatible client works. Examples:+Any OpenAI-compatible client works (Anthropic / Claude clients too — see [Anthropic / Claude clients](#anthropic--claude-clients)). Examples:**Python**@@ -405,6 +405,34 @@ curl http://localhost:3001/v1/embeddings \The default family, per-provider toggles, and priorities live on the dashboard's **Models → Embeddings** page. Pick your family once and stick with it for a given vector store — that's the whole point of the family model.+### Anthropic / Claude clients++FreeLLMAPI also speaks Anthropic's Messages API, so anything built for Claude — including **Claude Code** and the official Anthropic SDKs — can run against your free pool. Point the client at your server's **origin** (Anthropic clients append `/v1/messages` themselves) and authenticate with your unified key. Both `x-api-key` and `Authorization: Bearer` are accepted.++```bash+curl http://localhost:3001/v1/messages \+ -H "x-api-key: freellmapi-your-unified-key" \+ -H "anthropic-version: 2023-06-01" \+ -H "Content-Type: application/json" \+ -d '{+ "model": "claude-sonnet-4-5",+ "max_tokens": 256,+ "messages": [{"role": "user", "content": "hi"}]+ }'+```++Claude model names map to your free pool on the **Keys → Anthropic** tab: each family (`default`, `opus`, `sonnet`, `haiku`) routes to `auto` (the router picks a free model) or a model you pin. `POST /v1/messages/count_tokens` and a content-negotiated `GET /v1/models` (Anthropic shape when `anthropic-version` is sent) are implemented too. Streaming, system prompts, tool use, and image input all translate across the same router as the OpenAI endpoints.++**Claude Code** — point it at your server and start it:++```bash+export ANTHROPIC_BASE_URL=http://localhost:3001+export ANTHROPIC_AUTH_TOKEN=freellmapi-your-unified-key # NOT ANTHROPIC_API_KEY+claude+```++> Use `ANTHROPIC_AUTH_TOKEN` (sent as a Bearer token), **not** `ANTHROPIC_API_KEY` — Claude Code treats a set `ANTHROPIC_API_KEY` as a conflicting first-party credential and refuses to start.+## Screenshots### Keys@@ -503,7 +531,7 @@ Stacking free tiers has real trade-offs. Be honest with yourself about them:Contributors very welcome! See [CONTRIBUTING.md](CONTRIBUTING.md) for the dev loop, PR expectations, and the policy on AI/LLM-assisted contributions (short version: welcome, same quality bar as any other PR). Good first PRs:- **Add a provider** — copy `server/src/providers/openai-compat.ts` as a template, wire it into `server/src/providers/index.ts`, seed its models in `server/src/db/index.ts`, add a test in `server/src/__tests__/providers/`.-- **Add an endpoint** — images, moderations, audio. The provider base class can grow new methods; adapters declare which they support.+- **Add an endpoint** — moderations, legacy completions. The provider base class can grow new methods; adapters declare which they support.- **Improve the router** — cost-aware routing (cheapest-healthy-fastest tradeoffs), better latency-weighted priority, regional pinning.- **Dashboard polish** — charts on the Analytics page, key rotation UX, batch import of keys from `.env`.- **Docs** — more examples, client library snippets for Go/Rust/etc., a deployment recipe for Docker or Fly.