The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Python ai-gatewayanthropicazure-openaibedrockgatewaylangchainlitellmllmllm-gatewayllmopsmcp-gatewayopenaiopenai-proxyrustrust-aivertex-ai
Commits on main
fix(security): stop trusting the compromised eMudhra root CA (#47670)
f2118fc ryan-crabbe-berri committed 28 days ago
docs: rename security.md to SECURITY.md and point contact to [email protected] (#47581)
3f93350 Krrish Dholakia committed 29 days ago
refactor(types): replace Any with proven types in 50 files (#47818)
9d8fe1d Mateo Wang committed 29 days ago
fix(proxy): forward dispatched_call_type through _failure_usage_to_lift (#47815)
a31c523 Mateo Wang committed 29 days ago
fix(responses)!: round-trip Anthropic compaction blocks as compaction items (#46754)
a8113b0 Mateo Wang committed 29 days ago
fix(llms): join api_base and route with one shared helper (#47535)
66c8f5b Mateo Wang committed 29 days ago
fix(router): normalize native Responses latency per output token (#47727)
230f653 joshua-berri committed 29 days ago
fix(timing): carry litellm_received_at across to a planted litellm_metadata bucket so the overhead header stays anchored at receive time (#47790)
7ae46f2 Kerry Lu committed 29 days ago
test(perf): ramp the load cases until the pod stops keeping up and floor sustained_rps (#47758)
b99d2cc Kerry Lu committed 29 days ago
chore(cost-map): add fireworks muse spark open priority prices from the pricing page (#47785)
159b5b2 berriai-litellm-provider-info-sync[bot] committed 29 days ago
fix(router): log each dropped /v1/messages attempt through its own logging copy and open the retry without its usage (#45194)
8207256 Mateo Wang committed 29 days ago
fix(pricing): update baseten reflection/Beam-1.0 prices from the models API (#47784)
1fcbb03 berriai-litellm-provider-info-sync[bot] committed 29 days ago
feat(azure): add azure_ai/MAI-Code-1.1-Flash (#47779)
4ecc5fd berriai-litellm-provider-info-sync[bot] committed 29 days ago
fix(cli): send pi's session id from lite pi so sessions group and stick (#47700)
8d6b8da Mateo Wang committed 29 days ago
ci(integration): keep the MCP SDK1 peer on mcp 1.x and autovacuum off the size cleanup test (#47651)
8fe0a73 Mateo Wang committed 29 days ago
fix(responses): bill every round of a streamed MCP auto-execute on one spend row (#47445)
0e24bbb Mateo Wang committed 29 days ago
test(perf): emit per-location seconds, offending locations and the ten longest holds as event_loop_stalls breakdowns (#47757)
647ebd6 Kerry Lu committed 29 days ago
feat(ui): add Delete to the guardrail detail page and share one delete flow with the table row (#45893)
3985795 jesus-berri committed 29 days ago
feat(gemini): set the long decode budget for requests with no output limit from LITELLM_GEMINI_DEFAULT_MAX_OUTPUT_TOKENS (#47542)
11b5d1a Mateo Wang committed 29 days ago
fix(router): unregister strategy selectors from global callbacks on discard (#45795)
c8ed5cc Kazuki Hashimoto committed 29 days ago
test(perf): add chat_completions_2rps, responses_2rps and messages_2rps direct vs bridge cases (#47752)
8d9b992 Kerry Lu committed 29 days ago
fix(streaming)!: send the include_usage chunk with an empty choices array (#47109)
f768fb3 alberto committed 29 days ago
test(perf): allowlist anyio TaskHandle._run_coro in event_loop_stalls (#47746)
5e189c7 Kerry Lu committed 29 days ago
fix(fine_tuning): stop forwarding api_version to the OpenAI client (#44995)
cc156a7 devin-ai-integration[bot] committed 29 days ago
ci(cost map): delete the model prices JSON schema file (#47079)
c515f80 Mateo Wang committed 29 days ago
fix(token_counter): stop re-downloading a HuggingFace tokenizer after a failed download (#47586)
41890bd devin-ai-integration[bot] committed 29 days ago
test(perf): emit event_loop_stalls hold counts as perf_ JUnit properties (#47706)
1e1770a Kerry Lu committed 29 days ago
fix(scim)!: delete a group through the team delete's bulk path instead of one member at a time (#47544)
e333cca Mateo Wang committed 29 days ago