Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

2.0 KB
Raw Blame

Tests

Nothing on the other side of the call: tests/unit. A proxy we start with an upstream we script: tests/integration. Someone else's service with real credentials: tests/e2e. Tests of that harness itself, no proxy at all: tests/e2e_harness. Two fit, split it

What good looks like

Red when the claim in the name is broken. Prove it: mutate the behaviour, red. Restore, green. Put the mutation in the PR body

def test_custom_price_is_reported_and_charged(gateway: Gateway) -> None:
    with gateway.scenario() as scenario:
        model = scenario.model(input_cost_per_token=0.001, output_cost_per_token=0.002)
        response = gateway.request("POST", "/v1/chat/completions", {"model": model, "messages": [{"role": "user", "content": "price control"}]})
        assert response.status_code == 200, response.text
        assert float(response.headers["x-litellm-response-cost"]) == pytest.approx(20 * 0.001 + 20 * 0.002)

Rates in the test, expected computed by hand, one call, response.text in the assert

Assert the whole value. Iterating expected_body.items() (test_responses_api_request_body.py) cannot see an extra key. That is the shape of stream_options.include_usage (#19777, #28553)

The linter catches no-assert, mock-echo and credential skips. It cannot see an assert behind an if (a poll that ends in pytest.fail is fine), except Exception around the call (test_router.py: except Exception as e: print(f"FAILED TEST")), or blanket --reruns

Where it goes

What the assertion depends on goes in the test. Everything else in conftest. A rate in a fixture three directories up makes a failed assertion unreadable. Extend the file that already covers the behaviour

Writing it so a human can read it

Name says what broke: test_send_batched_with_valid_data says nothing. Build, one call, assert, on one screen. Helpers named for what they return, _pii_prompt(marker, email), not _setup(). Context in the assert message, not a comment