Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

NameLast commit date
..
_prompt_cache_pin_support.py
_reasoning_effort_levels_support.py
either_role.json
test_advisor_failure_cooldown.py
test_audit_any_sweep_chaos.py
test_auto_router_management.py
test_azure_passthrough_spend_row.py
test_base_model_context_window_owned_proxy.py
test_complexity_router_databricks_classifier.py
test_complexity_router_llm_classifier.py
test_declared_provider_bare_model_routes.py
test_deployment_mode_routing_wire.py
test_deployment_num_retries_generic_routes_wire.py
test_encrypted_content_affinity_no_peer.py
test_end_user_region_routing.py
test_fallback_hop_foreign_encrypted_reasoning.py
test_fallback_retry_header_contracts.py
test_include_fallback_errors_wire.py
test_key_max_parallel_requests.py
test_key_tpm_reservation.py
test_messages_stream_retry_budget_sources_owned_proxy.py
test_model_capability_flags.py
test_observed_routing.py
test_payment_required_cooldown.py
test_payment_required_mapping.py
test_pdf_document_pre_call_checks_owned_proxy.py
test_per_model_rate_limit_fallbacks_lifecycle.py
test_per_model_rate_limit_fallbacks.py
test_pre_call_response_format_filter.py
test_priority_model_tpm_enforcement.py
test_priority_rate_limit_headers.py
test_priority_scheduler_queue_cleanup.py
test_prompt_cache_pin_memory_routes.py
test_prompt_cache_pin_ttl_chaos.py
test_prompt_cache_pin_ttl.py
test_prompt_caching_affinity_tool_call_marks.py
test_ptu_shares_regex_timing.py
test_ptu_shares.py
test_reasoning_effort_levels_chaos.py
test_reasoning_effort_levels_wire.py
test_redis_increment_with_floor.py
test_redis_recovery.py
test_responses_websocket_turn_admission.py
test_router_strategies_count_async_requests_once.py
test_stale_cost_map_boot.py
test_tag_routing_prefix.py
test_team_tag_routing.py
test_usage_based_routing_redis_reads.py
test_user_config_routing.py
test_video_poll_routing_wire.py
test_vllm_passthrough_spend_row.py
test_vllm_passthrough_stream_relay.py
test_vllm_passthrough_stream_usage_settings.py
test_wildcard_model_listing.py