Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

316 B
Raw Blame
services:
  - type: web
    name: openai-proxy
    runtime: image
    image:
      url: ghcr.io/berriai/litellm:main-stable
    envVars:
      - key: PORT
        value: 4000
      - key: LITELLM_MASTER_KEY
        generateValue: true
    numInstances: 1
    healthCheckPath: /health/liveliness
    autoDeploy: true