Skip to content

Independent Auditing of AI Agents. Run by human or the agent itself, to answer the most crucial question in the AI Agent Economy. Is the agent doing what is supposed to do? With iFixAi you can have this answer in less than 120 seconds.

docs: remove stale full-run default claim (#319)

Wu Shuwen6180c658a0f3
diff --git a/README.md b/README.md
index 7a3ce63..2d7d205 100644
--- a/README.md
+++ b/README.md
@@ -256,7 +256,7 @@ ifixai run --provider openai # the default: 10 inspections
ifixai run --provider openai --full-run # all 61: slower, costs more
```
-`--full-run` is the old default behaviour, unchanged. It cannot be combined with `--suite`,
+`--full-run` cannot be combined with `--suite`,
`--test`, `--category`, or `--strategic`, and it has nothing to do with `--mode full`, which sets
how a run is judged. A `suite:` saved in `ifixai.yaml` replaces the default; `--full-run` on the
command line overrides that. `ifixai setup` preselects the same 10-inspection suite. `--resume`
@@ -361,11 +361,7 @@ raise it.
**"Premium" is a capability tier, not a paywall.** Everything in this repo, core and premium, is
free and open (Apache 2.0).
-**What does a good result look like?** The scorecards in **[case_studies/](case_studies/)** grade
-fixtures reconstructed from public accounts of two real incidents (the unproven Chaac Pizza
-Northeast complaint against Pizza Hut, and press reporting on the June 2026 Instagram takeovers).
-They are not tests of either company's production system. The reconstructions land at F; a
-well-governed agent scores materially higher (see [Test your own agent](#test-your-own-agent)).
+**What does a good result look like?** The four scorecards are fixture reconstructions based on public accounts, not tests of any named organisation's production system. Their reported results differ: [Dragontail dispatch](case_studies/pizza-hut-dragontail/SCORECARD.md) is 54.3% (F, selected subset); [Instagram account support](case_studies/meta-instagram-support/SCORECARD.md) is 22.2% (F, a subset floor rather than a full-suite grade); [OpenAI/Hugging Face containment](case_studies/openai-huggingface-containment/SCORECARD.md) is 65.5% (D); and [AISI cyber-range](case_studies/aisi-cyber-range-containment/SCORECARD.md) is 84.2% (B). The Dragontail case is based on allegations that remain unproven and contested. A well-governed agent scores materially higher (see [Test your own agent](#test-your-own-agent)).
Full math and weights: **[docs/scoring.md](docs/scoring.md)**. The full `B01`–`B32` → pillar
mapping and every premium category: **[docs/inspections.md](docs/inspections.md#categories)**.
diff --git a/docs/README.md b/docs/README.md
index cbc8efd..2cd3e3c 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -32,6 +32,6 @@ needs: **learning**, **doing**, **looking up**, and **understanding**
## See it in practice
-- **[Case studies](../case_studies/)**: scorecards for fixtures reconstructed from public accounts of four real incidents (Dragontail dispatch, Instagram account support, the OpenAI/Hugging Face containment breach, the AISI cyber-range incident). Not tests of any vendor's production system; before-remediation only. Deep dives at [ifixai.ai](https://ifixai.ai/docs/diagnostics/).
+- **[Case studies](../case_studies/)**: four fixture reconstructions based on public accounts (Dragontail dispatch, Instagram account support, OpenAI/Hugging Face containment, and an AISI cyber-range incident). They are not tests of any named organisation's production system; some source accounts are allegations rather than established findings. Before-remediation only. Deep dives at [ifixai.ai](https://ifixai.ai/docs/diagnostics/).
- **Claude Code plugin**: the zero-install front door. Claude guides the run, billed to a provider key in your Claude Code settings.
- **[Traction](traction.md)**: installs and runs over time.
diff --git a/docs/fixture_authoring.md b/docs/fixture_authoring.md
index e70bc0f..173c791 100644
--- a/docs/fixture_authoring.md
+++ b/docs/fixture_authoring.md
@@ -155,8 +155,7 @@ code:
Two switches to know: `policies.audit_logging: disabled` blanks B03 to INCONCLUSIVE and marks P08
not-applicable, and judge-path trajectory inspections (B15, P13, P19, P22, P27, P32, S02) require
judge temperature 0. The production examples in
-[`ifixai/fixtures/examples/`](../ifixai/fixtures/examples/) ship 10 users and clear every floor
-with error headroom; start from one of those when you want full coverage rather than a minimal run.
+[`ifixai/fixtures/examples/`](../ifixai/fixtures/examples/) vary in size and coverage; some contain only four users, while others provide larger scenarios. Check the evidence floors against the fixture you choose, and start with a larger domain example when you need broader coverage.
## Governance field reference
diff --git a/ifixai/fixtures/README.md b/ifixai/fixtures/README.md
index f4e9e96..2949938 100644
--- a/ifixai/fixtures/README.md
+++ b/ifixai/fixtures/README.md
@@ -1,12 +1,12 @@
# Writing a Fixture
-A fixture is the YAML (or JSON) file that tells ifixai **what system it is testing** — the roles, tools, permissions, data sources, policies, regulations, and domain-specific prompts that the 61 inspections (the `B01`–`B32` core plus 18 extended P / C / S / X / M inspections) parameterize against.
+A fixture is the YAML (or JSON) file that tells ifixai **what system it is testing** — the roles, tools, permissions, data sources, policies, regulations, and domain-specific prompts that the 61 inspections (32 `B01`–`B32` core plus 29 extended P / C / S / X / M / V inspections) parameterize against.
This directory contains:
- [`schema.json`](schema.json) — the authoritative JSON Schema (Draft-07) every fixture is validated against.
-- [`default/fixture.yaml`](default/fixture.yaml) — the fallback used by `--mode standard` when no fixture is supplied. Mirrors [`examples/nimbusforge_it_infra.yaml`](examples/nimbusforge_it_infra.yaml): full 50-inspection coverage **with seeded defects**, so an out-of-the-box mock run demonstrates failures — see [`default/README.md`](default/README.md).
-- [`examples/`](examples/) — illustrative fixtures (`healthcare.yaml`, `software_engineering.yaml`, `customer_support.yaml`). Copy one of these as a starting point.
+- [`default/fixture.yaml`](default/fixture.yaml) — the fallback used by `--mode standard` when no fixture is supplied. Its parsed data matches [`examples/nimbusforge_it_infra.yaml`](examples/nimbusforge_it_infra.yaml); it provides evidence for all 61 inspections and includes seeded defects so an out-of-the-box mock full-suite run demonstrates failures. See [`default/README.md`](default/README.md).
+- [`examples/`](examples/) — illustrative fixtures for different domains and coverage needs. Choose one suited to your use case.
- [`smoke_tiny.yaml`](smoke_tiny.yaml) — minimal fixture used by the smoke test.
Loader and validator live in [`../core/fixture_loader.py`](../core/fixture_loader.py).
@@ -64,10 +64,7 @@ Trail Integrity, Category VII — SUBVERSION)**: P08 drives the fixture's
`(role, tool)` actions and cross-checks the emitted audit records against
the action it observed, for suppression / field-alteration / timestamp-
reordering / post-hoc-mutation. Every shipped example exercises P08 and
-scores `1.0` against the honest governance reference (locked by
-`tests/unittesting/inspections/p08_self_audit_trail_integrity/test_shipped_fixtures.py`).
-Set `policies.audit_logging: disabled` to mark P08 not-applicable for a
-deployment that keeps no audit log.
+scores `1.0` against the honest governance reference in a mock-provider run. This has been exercised against all shipped example fixtures. Set `policies.audit_logging: disabled` to mark P08 not-applicable for a deployment that keeps no audit log.
## Capability hooks for the extended (P / C / S / X / M / V) inspections
diff --git a/ifixai/fixtures/default/README.md b/ifixai/fixtures/default/README.md
index 2fa65f4..69aac32 100644
--- a/ifixai/fixtures/default/README.md
+++ b/ifixai/fixtures/default/README.md
@@ -57,7 +57,7 @@ Without `--full-run`, only the 10 `essential` inspections run; of the seeded fai
## Extended (P / C / S / X / M) inspections
-The 18 premium / exploratory inspections beyond `B01`–`B32` are exercised when the resolved provider exposes the required **capability hooks** — `--provider mock` does, so a mock run scores them; a vanilla LLM reports `insufficient_evidence` (the same pattern as the governance cluster, not a failure). P01/P08 key off the governance block, C02/C05/C11 partition on `policies.confidence_threshold`, and X04/X11/M02/M03/M06/M07 plus the judge-path P/S inspections supply their own runner-fixed probes or domain-neutral corpora. Hook-to-fixture map: [`../README.md`](../README.md) § *Capability hooks for the extended inspections*; full descriptions: [`../../../docs/inspections.md`](../../../docs/inspections.md).
+The 29 premium / exploratory inspections beyond `B01`–`B32` are exercised when the resolved provider exposes the required **capability hooks** — `--provider mock` does, so a mock run scores them; a vanilla LLM reports `insufficient_evidence` where a capability is unavailable (not a failure). P01/P08 key off the governance block, C02/C05/C11 partition on `policies.confidence_threshold`, and the other capability-hook and judge-path inspections supply their own probes or corpora. Hook-to-fixture map: [`../README.md`](../README.md) § *Capability hooks for the extended inspections*; full descriptions: [`../../../docs/inspections.md`](../../../docs/inspections.md).
## If you want to audit coverage for a specific inspection