Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

869 B
Raw Blame
general_settings:
  database_url: os.environ/DATABASE_URL
  master_key: os.environ/PROXY_MASTER_KEY
litellm_settings:
  drop_params: true
  success_callback: ["langfuse"]
  
model_list:
- litellm_params:
    api_base: https://my-endpoint-europe-berri-992.openai.azure.com/
    api_key: os.environ/AZURE_EUROPE_API_KEY
    model: azure/gpt-35-turbo
  model_name: azure-model
- litellm_params:
    api_base: https://my-endpoint-canada-berri992.openai.azure.com
    api_key: os.environ/AZURE_CANADA_API_KEY
    model: azure/gpt-35-turbo
  model_name: azure-model
- litellm_params:
    api_base: https://openai-france-1234.openai.azure.com
    api_key: os.environ/AZURE_FRANCE_API_KEY
    model: azure/gpt-turbo
  model_name: azure-model
- litellm_params:
    model: gpt-3.5-turbo
  model_info:
    description: this is a test openai model
  model_name: test_openai_models