Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

262 B
Raw Blame
{
  "model": "gpt-5-mini",
  "input": "List files in /mnt/data and run python --version.",
  "tools": [
    {
      "type": "shell",
      "environment": {
        "type": "container_auto"
      }
    }
  ],
  "tool_choice": "auto",
  "max_output_tokens": 256
}