Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.
Python API
ifixai.api runs the same engine as the CLI. Run functions are async; helpers are sync.
import asyncio
from ifixai.api import run_inspections
result = asyncio.run(run_inspections(
provider="openai",
api_key="sk-...",
model="gpt-4o",
fixture="default",
system_name="my-agent",
))
print(result.overall_score, result.grade)
Functions
| Function | Purpose |
|---|---|
run_inspections(...) |
Run all registered inspections (async) → TestRunResult. Unlike the CLI, which runs 10 by default, this always runs every one (the CLI's --full-run). |
run_strategic(...) |
Run the top 8 strategic tests (async) |
run_selected(test_ids, ...) |
Run a chosen list of test IDs (async) |
run_single(test_id, ...) |
Run a single test by ID (async); raises ValueError with why when it is not applicable |
compare_scorecards(baseline, enhanced) |
Vendor-neutral comparison report (sync) |
list_tests() |
Return all InspectionSpec definitions (sync) |
list_fixtures() |
Return built-in fixture names (sync) |
Common parameters
| Parameter | Default | Notes |
|---|---|---|
provider |
required | Provider name ("openai", "anthropic", …) or a ChatProvider instance. |
api_key |
"" |
SUT key. |
fixture |
"default" |
Registered name or a Fixture object. |
model, endpoint, system_prompt |
None |
SUT overrides. |
system_name, system_version |
"", "1.0" |
Report labels. |
timeout, max_retries |
30, 3 |
Per-request controls. |
judge_config, pipeline_config |
None |
JudgeConfig / EvaluationPipelineConfig for judges, ensemble, budgets. |
sut_temperature, sut_seed, run_nonce, holdout_ids |
0.0, None, None, None |
Determinism / replay controls (mirror CLI seeds). |
To test a custom agent, implement ChatProvider (../ifixai/providers/base.py) and pass the instance as provider. Only send_message is needed: the inspections ask your agent through it. Hooks you add (authorize_tool, get_confidence, ...) show as notes next to the probe results and never decide the checks the probes decide. TestRunResult.not_applicable lists the checks set aside before the run, each with why. Walkthrough: testing-your-agent.md.