The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Python ai-gatewayanthropicazure-openaibedrockgatewaylangchainlitellmllmllm-gatewayllmopsmcp-gatewayopenaiopenai-proxyrustrust-aivertex-ai
Commits on main
fix(ui): offer access-group models in team per-model rate limit and budget pickers (#47630)
9026f42 mrinal-berri committed 1 month ago
fix(proxy): scope the search free-tier counter to the request's provider (#47646)
640f2d8 Kerry Lu committed 1 month ago
test(perf): tag a UI journey sample with its API response, not a Next.js prefetch under /ui/ (#47650)
95b6eea Kerry Lu committed 1 month ago
fix(proxy): price a billed failure row at the deployment's long-context tier like the success path (#47647)
70ece35 Kerry Lu committed 1 month ago
fix(bedrock)!: apply the deployment guardrailConfig on /bedrock pass-through routes (#47429)
e6065b9 yucheng-berri committed 1 month ago
fix(mcp): surface upstream HTTP status when a failed POST closes the session (#47569)
6bc1b2b Yassin Kortam committed 1 month ago
fix(proxy): restore daily tag spend to the in-memory queue when the Redis RPUSH fails (#47307)
06bebcc Yassin Kortam committed 1 month ago
fix(mcp)!: never forward the admission LiteLLM key upstream for any auth type (#46932)
2f139a7 Yassin Kortam committed 1 month ago
fix(ui_sso): seal the litellm_cp_return_to cookie so edge firewalls stop blocking the MCP OAuth sign-in (#47595)
f9d9466 Yassin Kortam committed 1 month ago
feat(github_copilot): support a GHE.com host on per-user Copilot credentials (#47610)
db2039b Yassin Kortam committed 1 month ago
fix(proxy): strip caller base_model from spend pricing and reject lone surrogates in multipart metadata (#47639)
2a26895 Kerry Lu committed 1 month ago
fix(anthropic): keep tool names over 64 chars when a guardrail rewrites a /v1/messages request (#47434)
78a6eeb yucheng-berri committed 1 month ago
fix(batches): retry confirmed pre-accounting failures (#47625)
6564a49 joshua-berri committed 1 month ago
test(perf): emit a top-contributors breakdown property per perf metric (#47632)
5c341e7 Kerry Lu committed 1 month ago
feat(cost): add model_info.pricing_provider for cost-map lookups (#47620)
1380e7e Kerry Lu committed 1 month ago
fix(logging)!: redact Responses instructions and callback output (#43315)
74230b4 Emerson Gomes committed 1 month ago
chore: remove description heading from PR template (#47631)
6e385f2 Mateo Wang committed 1 month ago
chore(cost-map): add fireworks glm 5.3 flash us deprecation date from the changelog (#47591)
dc9c31e berriai-litellm-provider-info-sync[bot] committed 1 month ago
fix(harness): set PWD to the working directory of local sandbox commands (#47530)
c4360bb Mateo Wang committed 1 month ago
fix(ui): rebuild the dashboard bundle and cover it in the example master key check (#47612)
c20d57c Mateo Wang committed 1 month ago
test(perf): let the UI journey fixture run on a restore upgraded by a released migrations image (#47628)
280496b Kerry Lu committed 1 month ago
test(perf): record the data request status per UI journey sample and bound each journey with a wall clock (#47622)
767b75a Kerry Lu committed 1 month ago
fix(bedrock): send batch jobs to a configured control-plane endpoint (#47260)
c6fc5b0 Priyansh Nandwana committed 1 month ago
fix(proxy): bill partial Responses API streams when the client disconnects (#47611)
f0ae445 Kerry Lu committed 1 month ago
fix(proxy): size pass rebuilds reset row statistics with ANALYZE and keeps its reclaimable count (#47501)
d677f4a Kerry Lu committed 1 month ago
fix(proxy): flush the spend-log queue only for Responses turns the bridge reads back (#47614)
423362b Kerry Lu committed 1 month ago
test(perf): set the small case's postgres_max_background_statement_per_request ceiling to 0.9 (#47616)
e276bf3 Kerry Lu committed 1 month ago
refactor(types): replace Any with proven types in 15 files (#47509)
04eaa6b Mateo Wang committed 1 month ago
fix(bedrock): return a 400 for malformed batch JSONL lines (#45992)
7655c2c Abhishek Acharya committed 1 month ago
fix(decisions): bill unpriced returned TypeSafe versions at the requested model and name a missing answers field (#47508)
81d8c9a Mateo Wang committed 1 month ago