Skip to content

DeepTeam is a framework to red team LLMs and AI agents.

docs updated

A-Vamshidc58f3fd2893
diff --git a/deepteam/red_teamer/utils.py b/deepteam/red_teamer/utils.py
index 7c2c52c6..d5b30deb 100644
--- a/deepteam/red_teamer/utils.py
+++ b/deepteam/red_teamer/utils.py
@@ -73,7 +73,9 @@ def resolve_model_callback(
def new_model_callback(
input: str, turns: Optional[List[RTTurn]] = None
):
- res, _ = model_callback.generate(input)
+ res = model_callback.generate(input)
+ if isinstance(res, tuple):
+ res, _ = res
return res
else:
@@ -81,7 +83,9 @@ def resolve_model_callback(
async def new_model_callback(
input: str, turns: Optional[List[RTTurn]] = None
):
- res, _ = await model_callback.a_generate(input)
+ res = await model_callback.a_generate(input)
+ if isinstance(res, tuple):
+ res, _ = res
return res
return new_model_callback
diff --git a/docs/docs/red-teaming-introduction.mdx b/docs/docs/red-teaming-introduction.mdx
index c71cd41e..54e37cb8 100644
--- a/docs/docs/red-teaming-introduction.mdx
+++ b/docs/docs/red-teaming-introduction.mdx
@@ -137,9 +137,47 @@ By definition, they all have an equal chance of being selected since the default
## Model Callback
-The model callback in `deepteam` is simply a callback function that wraps around your target LLM system that you are red teaming, and is actually not unique to `deepteam`. However, it is essential that you define this correctly because `deepteam` will be calling your model callback at red teaming time to attack your LLM system with the adversarial inputs it has generated.
+The model callback in `deepteam` can be a string or an instance of `DeepEvalBaseLLM` or simply a callback function that wraps around your target LLM system that you are red teaming. However, it is essential that you define this correctly because `deepteam` will be calling your model callback at red teaming time to attack your LLM system with the adversarial inputs it has generated.
-Here's how you can define your model callback:
+For passing a string or an instance of `DeepEvalBaseLLM`, you can pass the model callback as shown below:
+
+<Tabs>
+
+<TabItem value="String" label="String">
+
+```python
+...
+red_team(
+ attacks=[...],
+ vulnerabilities=[...],
+ model_callback="openai/gpt-5.2",
+)
+```
+
+For strings, you can pass a model with its provider prefix separated by a `/`. Ex: `google/gemini-2.5-flash`. The available prefixes that `deepteam` supports are: `openai`, `anthropic`, `google`, `xai`, `moonshotai`, `deepseek`, `ollama`.
+
+</TabItem>
+<TabItem value="DeepEval model" label="DeepEval model">
+
+```python
+from deepeval.models import AzureOpenAIModel
+
+model = AzureOpenAIModel(...)
+
+red_team(
+ attacks=[...],
+ vulnerabilities=[...],
+ model_callback=model,
+)
+```
+
+`deepteam` allows you to even pass your custom models which inherit from `DeepEvalBaseLLM` and implements the required methods as described in the [custom LLM documentation](https://deepeval.com/guides/guides-using-custom-llms).
+
+</TabItem>
+
+</Tabs>
+
+For custom callback functions, here's how you can define your model callback:
```python
async def model_callback(input: str) -> str:
diff --git a/docs/docs/red-teaming-yaml-cli.mdx b/docs/docs/red-teaming-yaml-cli.mdx
index afc999bc..d8739506 100644
--- a/docs/docs/red-teaming-yaml-cli.mdx
+++ b/docs/docs/red-teaming-yaml-cli.mdx
@@ -226,10 +226,10 @@ When using `callback`, the `file` field is mandatory while `function` is optiona
```yaml
target:
purpose: "A financial advice chatbot"
- model:
- provider: custom
- file: "my_model.py"
- class: "CustomDeepEvalLLM"
+ model:
+ provider: custom
+ file: "my_model.py"
+ class: "CustomDeepEvalLLM"
```
When using `provider: custom`, both `file` and `class` fields are mandatory.
@@ -246,14 +246,14 @@ target:
```yaml
target:
purpose: "A financial advice chatbot"
- model:
- provider: azure
- model: "model_name"
- temperature: 1
- deployment_name: "Test Deployment"
- api_key: "<your-api-key"
- openai_api_version: "2025-01-01-preview"
- azure_endpoint: "https://example-resource.azure.openai.com/"
+ model:
+ provider: azure
+ model: "model_name"
+ temperature: 1
+ deployment_name: "Test Deployment"
+ api_key: "<your-api-key"
+ openai_api_version: "2025-01-01-preview"
+ azure_endpoint: "https://example-resource.azure.openai.com/"
```
</TabItem>
<TabItem value="ollama" label="Ollama">