DeepTeam is a framework to red team LLMs and AI agents.
docs updated
A-Vamshidc58f3fd2893
diff --git a/deepteam/red_teamer/utils.py b/deepteam/red_teamer/utils.pyindex 7c2c52c6..d5b30deb 100644--- a/deepteam/red_teamer/utils.py+++ b/deepteam/red_teamer/utils.py@@ -73,7 +73,9 @@ def resolve_model_callback(def new_model_callback(input: str, turns: Optional[List[RTTurn]] = None):- res, _ = model_callback.generate(input)+ res = model_callback.generate(input)+ if isinstance(res, tuple):+ res, _ = resreturn reselse:@@ -81,7 +83,9 @@ def resolve_model_callback(async def new_model_callback(input: str, turns: Optional[List[RTTurn]] = None):- res, _ = await model_callback.a_generate(input)+ res = await model_callback.a_generate(input)+ if isinstance(res, tuple):+ res, _ = resreturn resreturn new_model_callbackdiff --git a/docs/docs/red-teaming-introduction.mdx b/docs/docs/red-teaming-introduction.mdxindex c71cd41e..54e37cb8 100644--- a/docs/docs/red-teaming-introduction.mdx+++ b/docs/docs/red-teaming-introduction.mdx@@ -137,9 +137,47 @@ By definition, they all have an equal chance of being selected since the default## Model Callback-The model callback in `deepteam` is simply a callback function that wraps around your target LLM system that you are red teaming, and is actually not unique to `deepteam`. However, it is essential that you define this correctly because `deepteam` will be calling your model callback at red teaming time to attack your LLM system with the adversarial inputs it has generated.+The model callback in `deepteam` can be a string or an instance of `DeepEvalBaseLLM` or simply a callback function that wraps around your target LLM system that you are red teaming. However, it is essential that you define this correctly because `deepteam` will be calling your model callback at red teaming time to attack your LLM system with the adversarial inputs it has generated.-Here's how you can define your model callback:+For passing a string or an instance of `DeepEvalBaseLLM`, you can pass the model callback as shown below:++<Tabs>++<TabItem value="String" label="String">++```python+...+red_team(+ attacks=[...],+ vulnerabilities=[...],+ model_callback="openai/gpt-5.2",+)+```++For strings, you can pass a model with its provider prefix separated by a `/`. Ex: `google/gemini-2.5-flash`. The available prefixes that `deepteam` supports are: `openai`, `anthropic`, `google`, `xai`, `moonshotai`, `deepseek`, `ollama`.++</TabItem>+<TabItem value="DeepEval model" label="DeepEval model">++```python+from deepeval.models import AzureOpenAIModel++model = AzureOpenAIModel(...)++red_team(+ attacks=[...],+ vulnerabilities=[...],+ model_callback=model,+)+```++`deepteam` allows you to even pass your custom models which inherit from `DeepEvalBaseLLM` and implements the required methods as described in the [custom LLM documentation](https://deepeval.com/guides/guides-using-custom-llms).++</TabItem>++</Tabs>++For custom callback functions, here's how you can define your model callback:```pythonasync def model_callback(input: str) -> str:diff --git a/docs/docs/red-teaming-yaml-cli.mdx b/docs/docs/red-teaming-yaml-cli.mdxindex afc999bc..d8739506 100644--- a/docs/docs/red-teaming-yaml-cli.mdx+++ b/docs/docs/red-teaming-yaml-cli.mdx@@ -226,10 +226,10 @@ When using `callback`, the `file` field is mandatory while `function` is optiona```yamltarget:purpose: "A financial advice chatbot"- model:- provider: custom- file: "my_model.py"- class: "CustomDeepEvalLLM"+ model:+ provider: custom+ file: "my_model.py"+ class: "CustomDeepEvalLLM"```When using `provider: custom`, both `file` and `class` fields are mandatory.@@ -246,14 +246,14 @@ target:```yamltarget:purpose: "A financial advice chatbot"- model:- provider: azure- model: "model_name"- temperature: 1- deployment_name: "Test Deployment"- api_key: "<your-api-key"- openai_api_version: "2025-01-01-preview"- azure_endpoint: "https://example-resource.azure.openai.com/"+ model:+ provider: azure+ model: "model_name"+ temperature: 1+ deployment_name: "Test Deployment"+ api_key: "<your-api-key"+ openai_api_version: "2025-01-01-preview"+ azure_endpoint: "https://example-resource.azure.openai.com/"```</TabItem><TabItem value="ollama" label="Ollama">