The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]
Python ai-gatewayanthropicazure-openaibedrockgatewaylangchainlitellmllmllm-gatewayllmopsmcp-gatewayopenaiopenai-proxyrustrust-aivertex-ai
model_list:
- model_name: Azure OpenAI GPT-4 Canada
litellm_params:
model: gpt-4.1-nano
api_key: sk-unit-fake
model_info:
mode: chat
input_cost_per_token: 0.0002
id: gm
- model_name: azure-embedding-model
litellm_params:
model: text-embedding-ada-002
api_key: sk-unit-fake
model_info:
mode: embedding
input_cost_per_token: 0.002
id: hello
litellm_settings:
drop_params: True
callbacks: custom_callbacks.my_custom_logger