Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

505 B
Raw Blame
model_list:
  - model_name: Azure OpenAI GPT-4 Canada
    litellm_params:
      model: gpt-4.1-nano
      api_key: sk-unit-fake
    model_info:
      mode: chat
      input_cost_per_token: 0.0002
      id: gm
  - model_name: azure-embedding-model
    litellm_params:
      model: text-embedding-ada-002
      api_key: sk-unit-fake
    model_info:
      mode: embedding
      input_cost_per_token: 0.002
      id: hello

litellm_settings:
  drop_params: True
  callbacks: custom_callbacks.my_custom_logger