DeepTeam is a framework to red team LLMs and AI agents.
update docs
A-Vamshif28e983c6120
diff --git a/docs/content/docs/red-teaming-introduction.mdx b/docs/content/docs/red-teaming-introduction.mdxindex d9951ba1..4c234dd2 100644--- a/docs/content/docs/red-teaming-introduction.mdx+++ b/docs/content/docs/red-teaming-introduction.mdx@@ -435,7 +435,7 @@ prompt_injection = PromptInjection()risk_assessment = red_team(model_callback=model_callback, vulnerabilities=[bias], attacks=[prompt_injection])```-There is **ONE** mandatory and **ELEVEN** optional arguments when calling the `red_team()` function:+There is **ONE** mandatory and **THIRTEEN** optional arguments when calling the `red_team()` function:- `model_callback`: a callback of type `Callable[[str], str]` that wraps around the target LLM system you wish to red team.- [Optional] `vulnerabilities`: a list of type `BaseVulnerability`s that determines the weaknesses to detect for.@@ -444,6 +444,8 @@ There is **ONE** mandatory and **ELEVEN** optional arguments when calling the `r- [Optional] `simulator_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://docs.confident-ai.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM` for simulating attacks. Defaulted to `"gpt-3.5-turbo-0125"`.- [Optional] `evaluation_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://docs.confident-ai.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM` for evaluation. Defaulted to `"gpt-4o"`.- [Optional] `attacks_per_vulnerability_type`: an int that determines the number of attacks to be simulated per vulnerability type. Defaulted to `1`.+- [Optional] `run_all_attacks`: a boolean which when set to `True`, runs **every** attack you passed in against every simulated attack instead of sampling one at random per test case. Defaulted to `False`. See [Running all attacks](#running-all-attacks).+- [Optional] `exposure`: a `Level` of `LOW`, `MEDIUM`, or `HIGH` describing how reachable your LLM application is to an attacker. Feeds the [CVSS score](/docs/red-teaming-risk-assessment#cvss-score) of each test case. Defaulted to `Level.MEDIUM`.- [Optional] `ignore_errors`: a boolean which when set to `True`, ignores all exceptions raised during red teaming. Defaulted to `False`.- [Optional] `async_mode`: a boolean specifying whether to enable async mode. Defaulted to `True`.- [Optional] `max_concurrent`: an integer that determines the maximum number of coroutines that can be ran in parallel. You can decrease this value if your models are running into rate limit errors. Defaulted to `10`.@@ -498,6 +500,85 @@ risk_assessment = red_team()```+## Running all attacks++By default, each simulated attack is enhanced by **one** attack drawn at random from the `attacks` you passed in, weighted by each attack's `weight`. That keeps a run cheap, but it means two runs of the same configuration can exercise different attacks — and it means an attack you passed in might never run at all.++Set `run_all_attacks=True` to run **every** attack against **every** simulated attack instead:++```python+from deepteam import red_team+from deepteam.vulnerabilities import Bias+from deepteam.attacks.single_turn import PromptInjection, ROT13, Base64++risk_assessment = red_team(+ model_callback=model_callback,+ vulnerabilities=[Bias(types=["race"])],+ attacks=[PromptInjection(), ROT13(), Base64()],+ run_all_attacks=True,+)+```++The number of test cases becomes `vulnerability_types × attacks_per_vulnerability_type × attacks`. The example above produces 3 test cases (one vulnerability type × 1 attack per type × 3 attacks) rather than 1.++:::caution+This multiplies your run cost. A framework with 20 vulnerability types and 3 attacks is 60 test cases, and `weight` is ignored entirely since nothing is being sampled.+:::++:::tip+Use `run_all_attacks=True` for any run you intend to **compare against another run**. With random sampling, a difference between two assessments may just be a different attack being drawn rather than a real change in your LLM application.+:::++## CVSS score++Every test case is scored on a **0.0–10.0** CVSS scale, and the assessment carries the average across all scored test cases. The score is derived from three inputs:++| Input | Where it comes from |+|--------------------|--------------------------------------------------------------------------------------------------------------------------------|+| **impact** | How severe the vulnerability is. Currently fixed at `MEDIUM` for every test case. |+| **exploitability** | How easy the attack is to pull off. Read from the attack itself — each attack in `deepteam` declares its own `exploitability`. |+| **exposure** | How reachable your LLM application is to an attacker. You set this per run via `exposure`. |++Each input is weighted, with impact dominating:++```+score = (impact × 0.6 + exploitability × 0.3 + exposure × 0.1) × 10+```++where `LOW` = 0.3, `MEDIUM` = 0.6, and `HIGH` = 0.9. A **passing** test case scores `0.0` — the attack was mitigated, so there is no risk to quantify. An **errored** test case has no score at all, and is excluded from the assessment average.++The resulting score maps onto four severities:++| Severity | CVSS score |+|--------------|------------|+| **Critical** | 9.0 - 10.0 |+| **High** | 7.0 - 8.9 |+| **Medium** | 4.0 - 6.9 |+| **Low** | 0.0 - 3.9 |++Raise `exposure` for a public, unauthenticated surface and lower it for something internal:++```python+from deepteam import red_team+from deepteam.red_teamer.cvss import Level++risk_assessment = red_team(+ model_callback=model_callback,+ vulnerabilities=[bias],+ attacks=[PromptInjection()],+ exposure=Level.HIGH,+)++print(risk_assessment.overview.cvss_score) # assessment-level average+print(risk_assessment.test_cases[0].cvss_score) # per test case+```++Both are printed to the console at the end of a run — a `CVSS` column on the test cases table, and an overall score with its severity in the summary.++:::note+Because `exposure` shifts every score in a run, keep it **fixed** across assessments you intend to compare. Changing it will move the average even when your LLM application has not changed.+:::+## Evaluation examples and guidelinesEach built-in vulnerability uses an **LLM-as-judge** metric with a structured prompt. You can **calibrate** that judge with:@@ -546,7 +627,7 @@ from deepteam.red_teamer import RedTeamerred_teamer = RedTeamer()```-There are **SIX** optional parameters when creating a `RedTeamer`:+There are **SEVEN** optional parameters when creating a `RedTeamer`:- [Optional] `target_purpose`: a string specifying your target LLM application's intended purpose. This affects the passing and failing of simulated attacks that are evaluated. Defaulted to `None`.- [Optional] `simulator_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://docs.confident-ai.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM` for simulating attacks. Defaulted to `"gpt-3.5-turbo-0125"`.@@ -554,6 +635,7 @@ There are **SIX** optional parameters when creating a `RedTeamer`:- [Optional] `async_mode`: a boolean specifying whether to enable async mode. Defaulted to `True`.- [Optional] `max_concurrent`: an integer that determines the maximum number of coroutines that can be ran in parallel. You can decrease this value if your models are running into rate limit errors. Defaulted to `10`.- [Optional] `attack_engine`: an optional `AttackEngine` used as the default refinement engine for runs started from this `RedTeamer` when you do not pass a different `attack_engine` into `red_team()`. Defaulted to `None`. See [Attack engine](#attack-engine).+- [Optional] `exposure`: a `Level` of `LOW`, `MEDIUM`, or `HIGH` describing how reachable your LLM application is to an attacker. This is set once per `RedTeamer` since it describes the deployment rather than any single attack, and feeds the CVSS score of every test case. Defaulted to `Level.MEDIUM`. See [CVSS score](#cvss-score).:::caution**All model interfaces in `deepteam` comes from `deepeval`**, and you can read [how to define a custom model of type `DeepEvalBaseLLM` here.](https://docs.confident-ai.com/guides/guides-using-custom-llms)@@ -590,13 +672,14 @@ print(risk_assessment.overall)As explained in the adversarial attack section, by making the PromptInjection attack `weight` 2x that of the `weight` of `ROT13`, it now has a 2x more chance to be simulated.:::-There is **ONE** mandatory and **TEN** optional arguments when calling the `red_team()` method:+There is **ONE** mandatory and **ELEVEN** optional arguments when calling the `red_team()` method:- `model_callback`: a callback of type `Callable[[str], str]` that wraps around the target LLM system you wish to red team.- [Optional] `framework`: an object of type `RedTeamingFramework` that contains both `vulnerabilities` and `attacks` in it.- [Optional] `simulator_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://docs.confident-ai.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM` for simulating attacks. Defaulted to `"gpt-3.5-turbo-0125"`.- [Optional] `evaluation_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://docs.confident-ai.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM` for evaluation. Defaulted to `"gpt-4o"`.- [Optional] `attacks_per_vulnerability_type`: an int that determines the number of attacks to be simulated per vulnerability type. Defaulted to `1`.+- [Optional] `run_all_attacks`: a boolean which when set to `True`, runs **every** attack against every simulated attack instead of sampling one at random per test case. Defaulted to `False`. See [Running all attacks](#running-all-attacks).- [Optional] `ignore_errors`: a boolean which when set to `True`, ignores all exceptions raised during red teaming. Defaulted to `False`.- [Optional] `reuse_previous_attacks`: a boolean which when set to `True`, will reuse the previously simulated attacks from the last `red_team()` method run. These attacks can only be reused if they exist (i.e. if you have already ran `red_team()` at least once). Defaulted to `False`.- [Optional] `async_mode`: a boolean specifying whether to enable async mode. Defaulted to `True`.