DeepTeam is a framework to red team LLMs and AI agents.
Updated docs
penguine84a8be4820f2
diff --git a/README.md b/README.mdindex 02c86c40..40ced349 100644--- a/README.md+++ b/README.md@@ -101,7 +101,7 @@ touch red_team_llm.pyOpen `red_team_llm.py` and paste in the code:```python-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"```@@ -117,7 +117,7 @@ from deepteam import red_teamfrom deepteam.vulnerabilities import Biasfrom deepteam.attacks.single_turn import PromptInjection-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"diff --git a/docs/docs/getting-started.mdx b/docs/docs/getting-started.mdxindex b67c1056..c2cf96a3 100644--- a/docs/docs/getting-started.mdx+++ b/docs/docs/getting-started.mdx@@ -51,7 +51,7 @@ from deepteam import red_teamfrom deepteam.vulnerabilities import Biasfrom deepteam.attacks.single_turn import PromptInjection-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"@@ -203,7 +203,7 @@ from deepteam import red_teamfrom deepteam.vulnerabilities import Biasfrom deepteam.attacks import.single_turn PromptInjection, ROT13-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"@@ -236,7 +236,7 @@ The first step is to red team you LLM system using and instance of the `RedTeamefrom deepteam import RedTeamerfrom deepteam.vulnerabilities import Bias-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"diff --git a/docs/docs/red-teaming-introduction.mdx b/docs/docs/red-teaming-introduction.mdxindex 01d7e7f9..bd7dbae4 100644--- a/docs/docs/red-teaming-introduction.mdx+++ b/docs/docs/red-teaming-introduction.mdx@@ -46,7 +46,7 @@ from deepteam import red_teamfrom deepteam.vulnerabilities import Biasfrom deepteam.attacks.single_turn import PromptInjection-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"@@ -140,7 +140,7 @@ The model callback in `deepteam` is simply a callback function that wraps aroundHere's how you can define your model callback:```python-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"```@@ -387,7 +387,7 @@ from deepteam import red_teamfrom deepteam.vulnerabilities import Biasfrom deepteam.attacks import PromptInjection-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"@@ -404,7 +404,7 @@ There are **THREE** mandatory and **FOUR** optional arguments when calling the `- `attacks`: a list of type `BaseAttack`s that determines the methods that will be simulated to expose the defined `vulnerabilities`.- [Optional] `attacks_per_vulnerability_type`: an int that determines the number of attacks to be simulated per vulnerability type. Defaulted to `1`.- [Optional] `ignore_errors`: a boolean which when set to `True`, ignores all exceptions raised during red teaming. Defaulted to `False`.-- [Optional] `run_async`: a boolean which when set to `True`, enables concurrent red teaming on all vulnerabilities, attacks, generations, **AND** evaluations. Defaulted to `False`.+- [Optional] `run_async`: a boolean which when set to `True`, enables concurrent red teaming on all vulnerabilities, attacks, generations, **AND** evaluations. Defaulted to `True`.- [Optional] `max_concurrent`: an integer that determines the maximum number of coroutines that can be ran in parallel. You can decrease this value if your models are running into rate limit errors. Defaulted to `10`.Don't forget to save the results (or at least print it):@@ -457,7 +457,7 @@ from deepteam.vulnerabilities import Biasfrom deepteam.attacks.single_turn import PromptInjection, ROT13from deepteam.red_teamer import RedTeamer-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"diff --git a/docs/docs/what-is-llm-red-teaming.mdx b/docs/docs/what-is-llm-red-teaming.mdxindex 76f0dcf7..9d6a5f90 100644--- a/docs/docs/what-is-llm-red-teaming.mdx+++ b/docs/docs/what-is-llm-red-teaming.mdx@@ -489,7 +489,7 @@ from deepteam import red_teamfrom deepteam.vulnerabilities import Biasfrom deepteam.attacks import PromptInjection-def model_callback(input: str) -> str:+async def model_callback(input: str) -> str:# Replace this with your LLM applicationreturn f"I'm sorry but I can't answer this: {input}"