Skip to content

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

2.2 KB
Raw Blame

tests/integration

Real proxy, Postgres, Redis, scripted upstream. README.md has shards and CI wiring

What good looks like

The root example is from here. The spend row lands async: poll, never sleep

rows = eventually(
    lambda: read_rows('SELECT spend FROM "LiteLLM_SpendLogs" WHERE request_id=%s', (body["id"],)),
    lambda values: len(values) == 1,
    seconds=70,
)
assert float(rows[0]["spend"]) == pytest.approx(10 * 0.001 + 5 * 0.0001 + 7 * 0.002 + 4 * 0.002)

sleep(3) fails on a slow runner and taxes every fast one. Assert the outbound body in the upstream handler. A leaked field is invisible from the response. monkeypatch.setenv is fine. Patching our own function in a full stack is not

Where it goes

By the domain a user would name: pricing, spend, routing, mcp. A file only needs to live in a directory that a GROUPS entry in run.py selects. There is no manifest and no covers marker on new tests. A product bug the test exposes is pytest.skip("BUG: <symptom>") at the top of the body, not a fix in the test and not a deletion. Needs no network, proxy, DB or Redis: tests/unit

CircleCI's integration groups run every file here except the OCI and sandbox tests, which no CI runs, and the files in GITHUB_FILES in run.py. CircleCI runs on main, not on pull requests, so a new file here first runs after its PR merges. Wanting a file to run before merge is no reason to change CI, and a new dedicated workflow for it is the same mistake as a shared shard. Write the file, leave GITHUB_FILES and every workflow alone, and tell the user it first runs on main. GITHUB_FILES is only for a file that needs what CircleCI's groups cannot give it, like the images image-scan.yml builds or the plain pytest run CircleCI's roi-database job gives test_roi_observed.py. Add a file there only in that case, together with its dedicated job, and list a new dedicated GitHub workflow with the directory it owns in DEDICATED_INTEGRATION_WORKFLOWS in .github/scripts/assert_ci_coverage.py. The ci-coverage check fails a file here that a GitHub workflow runs unless it is in GITHUB_FILES and that workflow owns its directory (#46780)