The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Python ai-gatewayanthropicazure-openaibedrockgatewaylangchainlitellmllmllm-gatewayllmopsmcp-gatewayopenaiopenai-proxyrustrust-aivertex-ai
Commits on main
fix(ui): show the budget spend limit and rate limits without a collapsed section (#47662)
3c28f4b yuneng-jiang committed 1 month ago
test(perf): emit the database upgrade run's perf rows and upgrade timings as JUnit properties (#47696)
f24036c Kerry Lu committed 1 month ago
feat(model hub): show admin-defined custom tags and metadata from model_info (#46064)
c8f7e0d jesus-berri committed 1 month ago
feat(wandb): add Beam 1.0 and GLM-5.3 to the cost map (#47690)
5e1e66d berriai-litellm-provider-info-sync[bot] committed 1 month ago
fix(ui): show a failed state with Retry on the Logs table, Usage totals and Caching chart when their fetch errors (#47671)
adf3373 Kerry Lu committed 1 month ago
test(perf): record the row count and size of each table the Admin UI journeys query (#47683)
17db64e Kerry Lu committed 1 month ago
chore(deps-dev): bump vite from 8.3.0 to 8.3.4 in /vscode-extension (#47627)
735ebc4 dependabot[bot] committed 1 month ago
fix(mcp): list per-user auth MCP tools without a shared token (#47677)
6083cf9 Yassin Kortam committed 1 month ago
fix(mcp): list per-user auth MCP tools without a shared token (#47677)
49b5444 Yassin Kortam committed 1 month ago
fix(gemini)!: honor supports_sampling_params on the Gemini and Vertex path and flag Gemini 3.6 Flash, 3.5 Flash-Lite and later (#47645)
c29a4f5 ryan-crabbe-berri committed 1 month ago
fix(proxy): show the newest health check per model group on the model hub (#47552)
1c6d55c michelligabriele committed 1 month ago
fix(proxy): attribute retry attempt spend rows to their deployment and keep them out of daily request counts (#47658)
5dce13d Kerry Lu committed 1 month ago
fix(vertex_ai): log Gemini 3 sampling deprecation once per model per worker (#40868)
f3e1307 devin-ai-integration[bot] committed 1 month ago
fix(responses): send the billed background response cost to the creating key's and team's loggers (#47506)
480ee65 Mateo Wang committed 1 month ago
build(deps): bump pillow and dashboard vite, defer certifi to clear osv-scan (#47678)
7c39dbf yuneng-jiang committed 1 month ago
fix(proxy): flush tool discovery queue when use_redis_transaction_buffer is enabled (#47653)
dba4fa0 mrinal-berri committed 1 month ago
feat(nvidia_riva)!: add riva_offline mode for offline-only ASR models (#41021)
d7c3588 devin-ai-integration[bot] committed 1 month ago
fix(mcp): upgrade to MCP SDK 2.3.0 and raise the per-event SSE size cap (#47034)
4aec80f joshua-berri committed 1 month ago
feat(extraction): add /v1/extraction with Cloudflare Clairic support (#47473)
9fbeebc Mateo Wang committed 1 month ago
bump: litellm-enterprise 0.1.78 -> 0.1.79, litellm-proxy-extras 0.4.110 -> 0.4.111, litellm 1.107.0 -> 1.108.0 (#47668)
9717b38 yuneng-jiang committed 1 month ago
feat(cost-map): add fireworks deepseek v4.1 flash 1017 us at the us-only premium (#47667)
5c836d6 berriai-litellm-provider-info-sync[bot] committed 1 month ago
fix(vertex-ai): drop flex prices from gemini-nano-banana-2.1 (#47666)
8e3ce6d berriai-litellm-provider-info-sync[bot] committed 1 month ago
fix(logging): spilled over PTU requests without base_model log the served model's rates (#47655)
6b1f4ed Kerry Lu committed 1 month ago
fix(proxy): price earlier timed-out retry attempts at their own deployment's model rate (#47648)
27efc38 Kerry Lu committed 1 month ago
fix(router): price cost-based routing through model_info.base_model (#47617)
0c6731e Kerry Lu committed 1 month ago
fix(logging)!: fire one CustomLogger failure hook per async call outside the async flag list (#47124)
dd511a2 yucheng-berri committed 1 month ago
fix(proxy): price a 499 spend row only for the proxy's own disconnect, and never after the call already completed (#47652)
ffb7133 Kerry Lu committed 1 month ago
chore(cost-map): add search_free_requests to the model prices schema (#47635)
2cba80c Kerry Lu committed 1 month ago
refactor(rust-bridge): detect Python-only features in the catalog instead of Rust declines (#46714)
730db4d yujonglee committed 1 month ago
refactor(rust-bridge): decide native routing in the catalog, drop the bypass predicates (#46750)
0ccae7a yujonglee committed 1 month ago