Skip to content

DeepTeam is a framework to red team LLMs and AI agents.

updated docs

A-Vamshic84cdc7d55fd
diff --git a/docs/docs/red-teaming-yaml-cli.mdx b/docs/docs/red-teaming-yaml-cli.mdx
index a81bae08..4a360a83 100644
--- a/docs/docs/red-teaming-yaml-cli.mdx
+++ b/docs/docs/red-teaming-yaml-cli.mdx
@@ -50,13 +50,13 @@ default_vulnerabilities:
types: ["profanity", "insults"]
attacks:
- - name: "Prompt Injection"
+ - name: "PromptInjection"
```
Then run the red teaming with a single command:
```bash
-deepteam run config.yaml
+deepteam config.yaml
```
:::tip DID YOU KNOW?
@@ -189,13 +189,19 @@ There are **TWO** mandatory and **TWO** optional parameters when creating custom
```yaml
attacks:
- - name: "Prompt Injection"
+ - name: "PromptInjection"
weight: 2
- name: "ROT13"
weight: 1
- - name: "Base64"
+ - name: "LinearJailbreaking"
+ num_turns: 2
+ turn_level_attacks: ["Roleplay"]
```
+:::caution Warning
+When defining attacks, you should pass the **class name** of the attacks as defined in the [attack docs](/docs/red-teaming-adversarial-attacks). For example, for the `PromptInjection` attack you **MUST** pass `"PromptInjection"` and not `"Prompt Injection"` or `"Promptinjection"`.
+:::
+
Each attack entry has:
- `name`: the attack class name (required, must match available attack classes)
@@ -217,7 +223,7 @@ Once you have your YAML configuration file, you can start red teaming with the C
### Basic Usage
```bash
-deepteam run config.yaml
+deepteam config.yaml
```
### Command Line Overrides
@@ -226,13 +232,13 @@ You can override specific configuration values using command line flags:
```bash
# Override concurrency and output folder
-deepteam run config.yaml -c 20 -o custom-results
+deepteam config.yaml -c 20 -o custom-results
# Override attacks per vulnerability
-deepteam run config.yaml -a 5
+deepteam config.yaml -a 5
# Combine multiple overrides
-deepteam run config.yaml -c 15 -a 3 -o production-results
+deepteam config.yaml -c 15 -a 3 -o production-results
```
There are **THREE** optional command line flags:
@@ -315,13 +321,56 @@ Use the help command to see all available options:
```bash
deepteam --help
-deepteam run --help
```
:::tip
-Available vulnerabilities: `Bias`, `Toxicity`, `Misinformation`, `Illegal Activity`, `Prompt Leakage`, `PII Leakage`, `Unauthorized Access`, `Excessive Agency`, `Robustness`, `Intellectual Property`, `Competition`, `Graphic Content`, `Personal Safety`, `CustomVulnerability`.
-Available attacks: `Base64`, `Gray Box`, `Leetspeak`, `Math Problem`, `Multilingual`, `Prompt Injection`, `Prompt Probing`, `Roleplay`, `ROT-13`, `Crescendo Jailbreaking`, `Linear Jailbreaking`, `Tree Jailbreaking`, `Sequential Break`, `Bad Likert Judge`.
+**Available vulnerabilities:**
+
+
+- [`PIILeakage`](red-teaming-vulnerabilities-pii-leakage)
+- [`PromptLeakage`](red-teaming-vulnerabilities-prompt-leakage)
+- [`Bias`](red-teaming-vulnerabilities-bias)
+- [`Toxicity`](red-teaming-vulnerabilities-toxicity)
+- [`BFLA`](red-teaming-vulnerabilities-bfla)
+- [`BOLA`](red-teaming-vulnerabilities-bola)
+- [`RBAC`](red-teaming-vulnerabilities-rbac)
+- [`DebugAccess`](red-teaming-vulnerabilities-debug-access)
+- [`ShellInjection`](red-teaming-vulnerabilities-shell-injection)
+- [`SQLInjection`](red-teaming-vulnerabilities-sql-injection)
+- [`SSRF`](red-teaming-vulnerabilities-ssrf)
+- [`IllegalActivities`](red-teaming-vulnerabilities-illegal-activity)
+- [`GraphicContent`](red-teaming-vulnerabilities-graphic-content)
+- [`PersonalSafety`](red-teaming-vulnerabilities-personal-safety)
+- [`Misinformation`](red-teaming-vulnerabilities-misinformation)
+- [`Intellectual Property`](red-teaming-vulnerabilities-intellectual-property)
+- [`Competition`](red-teaming-vulnerabilities-competition)
+- [`GoalTheft`](red-teaming-agentic-vulnerabilities-goal-theft)
+- [`RecursiveHijacking`](red-teaming-agentic-vulnerabilities-recursive-hijacking)
+- [`ExcessiveAgency`](red-teaming-vulnerabilities-excessive-agency)
+- [`Robustnesss`](red-teaming-vulnerabilities-robustness)
+
+**Available attacks:**
+
+- [`Base64`](/docs/red-teaming-adversarial-attacks-base64-encoding)
+- [`GrayBox`](/docs/red-teaming-adversarial-attacks-gray-box-attack)
+- [`Leetspeak`](/docs/red-teaming-adversarial-attacks-leetspeak)
+- [`MathProblem`](/docs/red-teaming-adversarial-attacks-math-problem)
+- [`Multilingual`](/docs/red-teaming-adversarial-attacks-multilingual)
+- [`PromptInjection`](/docs/red-teaming-adversarial-attacks-prompt-injection)
+- [`Roleplay`](/docs/red-teaming-adversarial-attacks-roleplay)
+- [`ROT-13`](/docs/red-teaming-adversarial-attacks-rot13-encoding)
+- [`ContextPoisoning`](/docs/red-teaming-agentic-attacks-context-poisoning)
+- [`GoalRedirection`](/docs/red-teaming-agentic-attacks-goal-redirection)
+- [`InputBypass`](/docs/red-teaming-agentic-attacks-input-bypass)
+- [`PermissionEscalation`](/docs/red-teaming-agentic-attacks-permission-escalation)
+- [`LinguisticConfusion`](/docs/red-teaming-agentic-attacks-semantic-manipulation)
+- [`SystemOverride`](/docs/red-teaming-agentic-attacks-system-override)
+- [`Bad LikertJudge`](/docs/red-teaming-adversarial-attacks-bad-likert-judge)
+- [`CrescendoJailbreaking`](/docs/red-teaming-adversarial-attacks-crescendo-jailbreaking)
+- [`LinearJailbreaking`](/docs/red-teaming-adversarial-attacks-linear-jailbreaking)
+- [`SequentialBreak`](/docs/red-teaming-adversarial-attacks-sequential-jailbreaking)
+- [`TreeJailbreaking`](/docs/red-teaming-adversarial-attacks-tree-jailbreaking)
For detailed documentation, refer to the [vulnerabilities documentation](/docs/red-teaming-vulnerabilities) and [attacks documentation](/docs/red-teaming-adversarial-attacks).
:::
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 8ff17771..881a277b 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -37,20 +37,20 @@ module.exports = {
type: "category",
label: "Single-Turn",
items: [
- "red-teaming-adversarial-attacks-prompt-injection",
- "red-teaming-adversarial-attacks-roleplay",
+ "red-teaming-adversarial-attacks-base64-encoding",
"red-teaming-adversarial-attacks-gray-box-attack",
"red-teaming-adversarial-attacks-leetspeak",
- "red-teaming-adversarial-attacks-rot13-encoding",
- "red-teaming-adversarial-attacks-multilingual",
"red-teaming-adversarial-attacks-math-problem",
- "red-teaming-adversarial-attacks-base64-encoding",
- "red-teaming-agentic-attacks-system-override",
- "red-teaming-agentic-attacks-permission-escalation",
+ "red-teaming-adversarial-attacks-multilingual",
+ "red-teaming-adversarial-attacks-prompt-injection",
+ "red-teaming-adversarial-attacks-roleplay",
+ "red-teaming-adversarial-attacks-rot13-encoding",
+ "red-teaming-agentic-attacks-context-poisoning",
"red-teaming-agentic-attacks-goal-redirection",
- "red-teaming-agentic-attacks-semantic-manipulation",
"red-teaming-agentic-attacks-input-bypass",
- "red-teaming-agentic-attacks-context-poisoning",
+ "red-teaming-agentic-attacks-permission-escalation",
+ "red-teaming-agentic-attacks-semantic-manipulation",
+ "red-teaming-agentic-attacks-system-override",
],
collapsed: true,
},
@@ -58,11 +58,11 @@ module.exports = {
type: "category",
label: "Multi-Turn",
items: [
+ "red-teaming-adversarial-attacks-bad-likert-judge",
+ "red-teaming-adversarial-attacks-crescendo-jailbreaking",
"red-teaming-adversarial-attacks-linear-jailbreaking",
- "red-teaming-adversarial-attacks-tree-jailbreaking",
"red-teaming-adversarial-attacks-sequential-jailbreaking",
- "red-teaming-adversarial-attacks-crescendo-jailbreaking",
- "red-teaming-adversarial-attacks-bad-likert-judge",
+ "red-teaming-adversarial-attacks-tree-jailbreaking",
],
collapsed: true,
},