DeepTeam is a framework to red team LLMs and AI agents.
updated docs
A-Vamshic84cdc7d55fd
diff --git a/docs/docs/red-teaming-yaml-cli.mdx b/docs/docs/red-teaming-yaml-cli.mdxindex a81bae08..4a360a83 100644--- a/docs/docs/red-teaming-yaml-cli.mdx+++ b/docs/docs/red-teaming-yaml-cli.mdx@@ -50,13 +50,13 @@ default_vulnerabilities:types: ["profanity", "insults"]attacks:- - name: "Prompt Injection"+ - name: "PromptInjection"```Then run the red teaming with a single command:```bash-deepteam run config.yaml+deepteam config.yaml```:::tip DID YOU KNOW?@@ -189,13 +189,19 @@ There are **TWO** mandatory and **TWO** optional parameters when creating custom```yamlattacks:- - name: "Prompt Injection"+ - name: "PromptInjection"weight: 2- name: "ROT13"weight: 1- - name: "Base64"+ - name: "LinearJailbreaking"+ num_turns: 2+ turn_level_attacks: ["Roleplay"]```+:::caution Warning+When defining attacks, you should pass the **class name** of the attacks as defined in the [attack docs](/docs/red-teaming-adversarial-attacks). For example, for the `PromptInjection` attack you **MUST** pass `"PromptInjection"` and not `"Prompt Injection"` or `"Promptinjection"`.+:::+Each attack entry has:- `name`: the attack class name (required, must match available attack classes)@@ -217,7 +223,7 @@ Once you have your YAML configuration file, you can start red teaming with the C### Basic Usage```bash-deepteam run config.yaml+deepteam config.yaml```### Command Line Overrides@@ -226,13 +232,13 @@ You can override specific configuration values using command line flags:```bash# Override concurrency and output folder-deepteam run config.yaml -c 20 -o custom-results+deepteam config.yaml -c 20 -o custom-results# Override attacks per vulnerability-deepteam run config.yaml -a 5+deepteam config.yaml -a 5# Combine multiple overrides-deepteam run config.yaml -c 15 -a 3 -o production-results+deepteam config.yaml -c 15 -a 3 -o production-results```There are **THREE** optional command line flags:@@ -315,13 +321,56 @@ Use the help command to see all available options:```bashdeepteam --help-deepteam run --help```:::tip-Available vulnerabilities: `Bias`, `Toxicity`, `Misinformation`, `Illegal Activity`, `Prompt Leakage`, `PII Leakage`, `Unauthorized Access`, `Excessive Agency`, `Robustness`, `Intellectual Property`, `Competition`, `Graphic Content`, `Personal Safety`, `CustomVulnerability`.-Available attacks: `Base64`, `Gray Box`, `Leetspeak`, `Math Problem`, `Multilingual`, `Prompt Injection`, `Prompt Probing`, `Roleplay`, `ROT-13`, `Crescendo Jailbreaking`, `Linear Jailbreaking`, `Tree Jailbreaking`, `Sequential Break`, `Bad Likert Judge`.+**Available vulnerabilities:**+++- [`PIILeakage`](red-teaming-vulnerabilities-pii-leakage)+- [`PromptLeakage`](red-teaming-vulnerabilities-prompt-leakage)+- [`Bias`](red-teaming-vulnerabilities-bias)+- [`Toxicity`](red-teaming-vulnerabilities-toxicity)+- [`BFLA`](red-teaming-vulnerabilities-bfla)+- [`BOLA`](red-teaming-vulnerabilities-bola)+- [`RBAC`](red-teaming-vulnerabilities-rbac)+- [`DebugAccess`](red-teaming-vulnerabilities-debug-access)+- [`ShellInjection`](red-teaming-vulnerabilities-shell-injection)+- [`SQLInjection`](red-teaming-vulnerabilities-sql-injection)+- [`SSRF`](red-teaming-vulnerabilities-ssrf)+- [`IllegalActivities`](red-teaming-vulnerabilities-illegal-activity)+- [`GraphicContent`](red-teaming-vulnerabilities-graphic-content)+- [`PersonalSafety`](red-teaming-vulnerabilities-personal-safety)+- [`Misinformation`](red-teaming-vulnerabilities-misinformation)+- [`Intellectual Property`](red-teaming-vulnerabilities-intellectual-property)+- [`Competition`](red-teaming-vulnerabilities-competition)+- [`GoalTheft`](red-teaming-agentic-vulnerabilities-goal-theft)+- [`RecursiveHijacking`](red-teaming-agentic-vulnerabilities-recursive-hijacking)+- [`ExcessiveAgency`](red-teaming-vulnerabilities-excessive-agency)+- [`Robustnesss`](red-teaming-vulnerabilities-robustness)++**Available attacks:**++- [`Base64`](/docs/red-teaming-adversarial-attacks-base64-encoding)+- [`GrayBox`](/docs/red-teaming-adversarial-attacks-gray-box-attack)+- [`Leetspeak`](/docs/red-teaming-adversarial-attacks-leetspeak)+- [`MathProblem`](/docs/red-teaming-adversarial-attacks-math-problem)+- [`Multilingual`](/docs/red-teaming-adversarial-attacks-multilingual)+- [`PromptInjection`](/docs/red-teaming-adversarial-attacks-prompt-injection)+- [`Roleplay`](/docs/red-teaming-adversarial-attacks-roleplay)+- [`ROT-13`](/docs/red-teaming-adversarial-attacks-rot13-encoding)+- [`ContextPoisoning`](/docs/red-teaming-agentic-attacks-context-poisoning)+- [`GoalRedirection`](/docs/red-teaming-agentic-attacks-goal-redirection)+- [`InputBypass`](/docs/red-teaming-agentic-attacks-input-bypass)+- [`PermissionEscalation`](/docs/red-teaming-agentic-attacks-permission-escalation)+- [`LinguisticConfusion`](/docs/red-teaming-agentic-attacks-semantic-manipulation)+- [`SystemOverride`](/docs/red-teaming-agentic-attacks-system-override)+- [`Bad LikertJudge`](/docs/red-teaming-adversarial-attacks-bad-likert-judge)+- [`CrescendoJailbreaking`](/docs/red-teaming-adversarial-attacks-crescendo-jailbreaking)+- [`LinearJailbreaking`](/docs/red-teaming-adversarial-attacks-linear-jailbreaking)+- [`SequentialBreak`](/docs/red-teaming-adversarial-attacks-sequential-jailbreaking)+- [`TreeJailbreaking`](/docs/red-teaming-adversarial-attacks-tree-jailbreaking)For detailed documentation, refer to the [vulnerabilities documentation](/docs/red-teaming-vulnerabilities) and [attacks documentation](/docs/red-teaming-adversarial-attacks).:::diff --git a/docs/sidebars.js b/docs/sidebars.jsindex 8ff17771..881a277b 100644--- a/docs/sidebars.js+++ b/docs/sidebars.js@@ -37,20 +37,20 @@ module.exports = {type: "category",label: "Single-Turn",items: [- "red-teaming-adversarial-attacks-prompt-injection",- "red-teaming-adversarial-attacks-roleplay",+ "red-teaming-adversarial-attacks-base64-encoding","red-teaming-adversarial-attacks-gray-box-attack","red-teaming-adversarial-attacks-leetspeak",- "red-teaming-adversarial-attacks-rot13-encoding",- "red-teaming-adversarial-attacks-multilingual","red-teaming-adversarial-attacks-math-problem",- "red-teaming-adversarial-attacks-base64-encoding",- "red-teaming-agentic-attacks-system-override",- "red-teaming-agentic-attacks-permission-escalation",+ "red-teaming-adversarial-attacks-multilingual",+ "red-teaming-adversarial-attacks-prompt-injection",+ "red-teaming-adversarial-attacks-roleplay",+ "red-teaming-adversarial-attacks-rot13-encoding",+ "red-teaming-agentic-attacks-context-poisoning","red-teaming-agentic-attacks-goal-redirection",- "red-teaming-agentic-attacks-semantic-manipulation","red-teaming-agentic-attacks-input-bypass",- "red-teaming-agentic-attacks-context-poisoning",+ "red-teaming-agentic-attacks-permission-escalation",+ "red-teaming-agentic-attacks-semantic-manipulation",+ "red-teaming-agentic-attacks-system-override",],collapsed: true,},@@ -58,11 +58,11 @@ module.exports = {type: "category",label: "Multi-Turn",items: [+ "red-teaming-adversarial-attacks-bad-likert-judge",+ "red-teaming-adversarial-attacks-crescendo-jailbreaking","red-teaming-adversarial-attacks-linear-jailbreaking",- "red-teaming-adversarial-attacks-tree-jailbreaking","red-teaming-adversarial-attacks-sequential-jailbreaking",- "red-teaming-adversarial-attacks-crescendo-jailbreaking",- "red-teaming-adversarial-attacks-bad-likert-judge",+ "red-teaming-adversarial-attacks-tree-jailbreaking",],collapsed: true,},