Skip to content

DeepTeam is a framework to red team LLMs and AI agents.

Finished docs for frameworks

A-Vamshib971a7130479
diff --git a/docs/docs/frameworks-aegis.mdx b/docs/docs/frameworks-aegis.mdx
new file mode 100644
index 00000000..92a28764
--- /dev/null
+++ b/docs/docs/frameworks-aegis.mdx
@@ -0,0 +1,69 @@
+---
+id: frameworks-aegis
+title: Aegis
+sidebar_label: Aegis
+---
+
+The **Aegis** framework integrates the [NVIDIA Aegis AI Content Safety Dataset](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-1.0) — an open-source dataset aligned with NVIDIA’s **Content Safety Taxonomy** across 13 critical harm categories.
+Aegis enables DeepTeam to perform **dataset-driven red teaming** using real human-labeled safety violations to validate model robustness against harmful or unsafe user inputs.
+
+## Overview
+
+Aegis focuses on **real-world unsafe content** from public conversations. This allows evaluation of model refusal behaviors and fine-tuning of safety filters using authentic, labeled harm categories.
+
+```python
+from deepteam.frameworks import Aegis
+from deepteam import red_team
+from somewhere import your_model_callback
+
+aegis = Aegis(num_attacks=10)
+
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ framework=aegis,
+)
+
+print(risk_assessment)
+```
+
+The `Aegis` framework accepts **FOUR** optional parameters:
+
+- [Optional] `num_attacks`: Number of harmful test cases to sample from the dataset. Defaulted to `15`.
+- [Optional] `evaluation_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://deepeval.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM`. Defaulted to `'gpt-4o'`.
+- [Optional] `async_mode`: a boolean which when set to `True`, enables concurrent execution. Defaulted to `True`.
+- [Optional] `verbose_mode`: a boolean which when set to `True`, prints intermediate steps used to evaluate model responses. Defaulted to `False`.
+
+## How It Works
+
+1. Loads the Aegis dataset from Hugging Face.
+2. Filters unsafe samples labeled under NVIDIA’s harm taxonomy.
+3. Randomly samples `num_attacks` test cases.
+4. Evaluates how well the model identifies, refuses, or mitigates these harmful prompts.
+
+## Categories & Filtering
+
+Aegis covers 13 harm domains, all supported by DeepTeam filtering.
+
+**Example categories:**
+
+- `sexual_content`
+- `violence`
+- `hate_speech`
+- `self_harm`
+- `misinformation`
+- `privacy_violation`
+- `child_exploitation`
+- `drugs`
+- `terrorism`
+
+## Related Docs
+
+- [MITRE ATLAS framework](/docs/red-teaming-frameworks-mitre)
+- [NIST AI RMF framework](/docs/red-teaming-frameworks-nist)
+- [BeaverTails dataset framework](/docs/red-teaming-frameworks-beavertails)
+- [Vulnerabilities in DeepTeam](/docs/red-teaming-vulnerabilities)
+- [Adversarial attacks in DeepTeam](/docs/red-teaming-adversarial-attacks)
+
+## References
+
+[NVIDIA Aegis AI Content Safety Dataset](https://huggingface.co/datasets/nvidia/Aegis-AI-Content-Safety-Dataset-1.0) (Hugging Face)
diff --git a/docs/docs/frameworks-beavertails.mdx b/docs/docs/frameworks-beavertails.mdx
new file mode 100644
index 00000000..63909794
--- /dev/null
+++ b/docs/docs/frameworks-beavertails.mdx
@@ -0,0 +1,71 @@
+---
+id: frameworks-beavertails
+title: BeaverTails
+sidebar_label: BeaverTails
+---
+
+The **BeaverTails** framework integrates the [BeaverTails dataset](https://huggingface.co/datasets/PKU-Alignment/BeaverTails) — a large, human-labeled collection of harmful and borderline prompts — so DeepTeam can run **dataset-driven red teaming** against real-world adversarial inputs.
+BeaverTails is designed to surface safety failures across a wide range of harm categories (abuse, illegal instructions, privacy violations, self-harm, sexual content, misinformation, and more) by testing how models respond to actual annotated malicious prompts.
+
+## Overview
+
+BeaverTails focuses on **sampling real harmful prompts** from a curated dataset rather than synthetic generation. This provides higher-fidelity signals for model behavior in the wild and helps you validate safety defenses, content filters, and ingestion pipelines.
+
+```python
+from deepteam.frameworks import BeaverTails
+from deepteam import red_team
+from somewhere import your_model_callback
+
+beaver_tails = BeaverTails(num_attacks=10)
+
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ framework=beaver_tails,
+)
+
+print(risk_assessment)
+```
+
+The `BeaverTails` framework accepts **FIVE** optional parameters:
+
+- [Optional] `num_attacks`: A list of strings that represent the `NIST` measure you want to test your AI application on:
+- [Optional] `data_split`: A string which decides which data is used for fetching the dataset. It is a string literal that accepts either `330k_train` or `30k_train`. Defaulted to `30k_train`.
+- [Optional] `evaluation_model`: a string specifying which of OpenAI's GPT models to use, **OR** [any custom LLM model](https://deepeval.com/guides/guides-using-custom-llms) of type `DeepEvalBaseLLM`. Defaulted to 'gpt-4o'.
+- [Optional] `async_mode`: a boolean which when set to `True`, enables concurrent execution. Defaulted to `True`.
+- [Optional] `verbose_mode`: a boolean which when set to `True`, prints the intermediate steps used to evaluate target LLM's response to dataset's input. Defaulted to `False`.
+
+## How It Works
+
+1. Loads the dataset using Hugging Face's `datasets` with the specified BeaverTails split
+2. Filters out unsafe test cases to focus only on potentially harmful attacks
+3. Randomly samples `num_attacks` prompts
+4. Evaluates if the model appropriately refuses and handles the harmful attacks from dataset
+
+## Categories & Filtering
+
+BeaverTails includes 14 harm subcategories. DeepTeam supports filtering by subcategory so you can focus testing on highest-risk areas.
+
+**Example subcategories:**
+
+- `animal_abuse`
+- `child_abuse`
+- `discrimination` / `hate_speech`
+- `drug_abuse`, `weapons`
+- `financial_crime`
+- `misinformation`
+- `privacy_violation`
+- `self_harm`
+- `sexually_explicit`
+- `terrorism`
+- `violence`
+
+## Related Docs
+
+- [MITRE ATLAS framework](/docs/red-teaming-frameworks-mitre)
+- [NIST AI RMF framework](/docs/red-teaming-frameworks-nist)
+- [Vulnerabilities in DeepTeam](/docs/red-teaming-vulnerabilities)
+- [Adversarial attacks in DeepTeam](/docs/red-teaming-adversarial-attacks)
+
+## References
+
+[BeaverTails dataset - `PKU-Alignment/BeaverTails`](https://huggingface.co/datasets/PKU-Alignment/BeaverTails) (Hugging Face)
diff --git a/docs/docs/frameworks-introduction.mdx b/docs/docs/frameworks-introduction.mdx
new file mode 100644
index 00000000..c6b42a7f
--- /dev/null
+++ b/docs/docs/frameworks-introduction.mdx
@@ -0,0 +1,125 @@
+---
+id: frameworks-introduction
+title: Introduction
+sidebar_label: Introduction
+---
+
+The DeepTeam Frameworks define structured methodologies for **AI red teaming and risk assessment**.
+Each framework maps to a recognized safety or security standard, helping you test your model's robustness against **real-world adversarial behavior, dataset risks, and system vulnerabilities**.
+
+DeepTeam supports multiple frameworks — from dataset-based testing (_BeaverTails, Aegis_) to security and governance standards (_MITRE ATLAS, NIST AI RMF, OWASP Top 10 for LLMs_).
+
+## Available Frameworks
+
+Here are the list of frameworks available in `deepteam`:
+
+- [MITRE ATLAS](#mitre-atlas)
+- [NIST AI RMF](#nist-ai-rmf)
+- [OWASP Top 10 for LLMs](#owasp-top-10-for-llms)
+- [BeaverTails](#beavertails)
+- [Aegis](#aegis)
+
+### MITRE ATLAS
+
+The MITRE ATLAS framework integrates the MITRE ATLAS knowledge base, focusing on adversarial tactics and techniques used against AI systems.
+It evaluates system resilience across attack phases like Reconnaissance, Resource Development, Initial Access, and Impact.
+
+- Tests adversarial behavior patterns from the ATLAS taxonomy
+- Detects vulnerabilities such as prompt injection, data poisoning, and exfiltration
+- Ideal for AI security simulation and penetration testing
+
+```python
+from deepteam.frameworks import MITREATLAS
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITREATLAS(num_attacks=10)
+
+risk = red_team(
+ model_callback=your_model_callback,
+ framework=atlas
+)
+```
+
+[Learn more about MITRE ATLAS](/docs/red-teaming-frameworks-mitre)
+
+### NIST AI RMF
+
+The NIST AI Risk Management Framework (RMF) provides a structured approach to identifying, managing, and mitigating AI risks.
+It focuses on trustworthiness, robustness, and accountability — aligning LLM behavior with regulatory and ethical standards.
+
+- Tests safety, fairness, and robustness dimensions
+- Suitable for compliance-driven AI governance workflows
+
+```python
+from deepteam.frameworks import NIST
+nist = NIST(num_attacks=10)
+
+risk = red_team(
+ model_callback=your_model_callback,
+ framework=nist
+)
+```
+
+[Learn more about NIST AI RMF](/docs/red-teaming-frameworks-nist)
+
+### OWASP Top 10 for LLMs
+
+The OWASP Top 10 for LLMs framework identifies the most critical security risks in LLM applications.
+It reflects the 2025 OWASP edition, covering vulnerabilities in RAG systems, agents, and model integrations.
+
+- Tests for prompt injection, system prompt leakage, vector weaknesses, and more
+- Simulates real-world exploit attempts and harmful outputs
+- Ideal for application-level AI security assessments
+
+```python
+from deepteam.frameworks import OWASPTop10
+owasp = OWASPTop10(num_attacks=10)
+
+risk = red_team(
+ model_callback=your_model_callback,
+ framework=owasp
+)
+```
+
+[Learn more about OWASP Top 10 for LLMs](/docs/red-teaming-frameworks-owasp-top-10-for-llms)
+
+### BeaverTails
+
+The BeaverTails framework integrates the PKU BeaverTails dataset — a large, human-labeled dataset of harmful and borderline prompts.
+It performs dataset-driven red teaming, surfacing model weaknesses across categories like abuse, misinformation, and privacy violations.
+
+- Uses real-world harmful prompts instead of synthetic generation
+- Validates content safety and refusal behavior
+
+```python
+from deepteam.frameworks import BeaverTails
+beaver = BeaverTails(num_attacks=10)
+
+risk = red_team(
+ model_callback=your_model_callback,
+ framework=beaver
+)
+```
+
+[Learn more about BeaverTails](/docs/red-teaming-frameworks-beavertails)
+
+### Aegis
+
+The Aegis framework integrates the NVIDIA Aegis AI Content Safety Dataset, which follows NVIDIA's content safety taxonomy across 13 harm categories.
+It provides a comprehensive safety evaluation using real human-labeled harmful content.
+
+- Tests for harmful user messages across multiple safety dimensions
+- Useful for evaluating model robustness under real-world safety challenges
+
+```python
+from deepteam.frameworks import Aegis
+aegis = Aegis(num_attacks=10)
+
+risk = red_team(
+ model_callback=your_model_callback,
+ framework=aegis
+)
+```
+
+[Learn more about Aegis](/docs/red-teaming-frameworks-aegis)
diff --git a/docs/docs/frameworks-mitre-atlas.mdx b/docs/docs/frameworks-mitre-atlas.mdx
new file mode 100644
index 00000000..787be947
--- /dev/null
+++ b/docs/docs/frameworks-mitre-atlas.mdx
@@ -0,0 +1,328 @@
+---
+id: frameworks-mitre-atlas
+title: MITRE ATLAS
+sidebar_label: MITRE ATLAS
+---
+
+The **MITRE ATLAS™ (Adversarial Threat Landscape for Artificial-Intelligence Systems)** framework provides a structured knowledge base of adversarial tactics, techniques, and procedures (TTPs) used against AI and ML systems.
+It extends the principles of [MITRE ATT&CK®](https://attack.mitre.org) to the **AI threat surface**, identifying how adversaries can manipulate, exploit, or misuse AI models throughout their lifecycle.
+
+DeepTeam's **MITRE ATLAS module** implements these adversarial mappings to test your AI or LLM application for security, privacy, and robustness vulnerabilities, across each phase of the AI attack lifecycle.
+
+## Overview
+
+In DeepTeam, the MITRE ATLAS framework operationalizes AI-specific adversarial tactics to help organizations **detect**, **mitigate**, and **analyze** model exploitation risks.
+
+Each tactic represents a **goal** an adversary may have while attacking an AI system. These tactics collectively map the “why” behind adversarial actions and correspond to different testing modules in DeepTeam.
+
+| Tactic | Description |
+| ------------------------ | ---------------------------------------------------------------------- |
+| **Reconnaissance** | Gathering intelligence about AI systems and configurations |
+| **Resource Development** | Acquiring resources or tools to enable future attacks |
+| **Initial Access** | Gaining entry to the target AI system or environment |
+| **ML Attack Staging** | Preparing, training, or adapting attacks specifically for AI models |
+| **Exfiltration** | Stealing sensitive information, model data, or internal configurations |
+| **Impact** | Manipulating or degrading AI systems to achieve adversarial goals |
+
+Each of these categories can be tested independently or together:
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["reconnaissance", "impact"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ vulnerabilities=vulnerabilities,
+ attacks=attacks
+)
+```
+
+The `MITRE` framework accepts **ONE** optional parameter:
+
+- [Optional] `categories`: A list of strings that represent the `MITRE` ATLAS tactics you want to test your AI application on:
+ - `reconnaissance`: Tests for unintended disclosure of internal info, prompts, or policies through probing or reasoning.
+ - `resource_development`: Checks if the system can aid in creating or supporting malicious tools or content.
+ - `initial_access`: Evaluates resistance to malicious entry via prompt injection, debug access, or exposed interfaces.
+ - `ml_attack_staging`: Tests handling of poisoned or adversarial inputs used to prepare model-targeted attacks.
+ - `exfiltration`: Detects data leakage or extraction of sensitive information through adversarial queries.
+ - `impact`: Simulates harmful actions like goal hijacking, content manipulation, or autonomy abuse.
+
+## Reconnaissance — Information Gathering and System Profiling
+
+**(MITRE ATLAS ID: AML.TA0002)**
+
+**Goal:** The adversary is trying to gather information about the AI system they can use to plan future operations.
+
+Reconnaissance involves identifying model capabilities, business rules, and potential weaknesses. Adversaries may probe model outputs, query APIs, or use role-based misdirection to uncover internal policies, system prompts, or hidden logic.
+
+DeepTeam tests whether your AI system inadvertently exposes:
+
+- Internal guardrails, policies, or roles
+- Confidential reasoning chains or business logic
+- Sensitive prompts, credentials, or decision rules
+- Competitive or private strategic information
+
+**Example vulnerabilities**
+
+- `Competition`
+- `PromptLeakage`
+- `RBAC`
+- `CustomVulnerability` — _Policy Disclosure_
+
+**Example attacks**
+
+- `Roleplay`
+- `PromptInjection`
+- `CrescendoJailbreaking`
+- `SequentialJailbreak`
+- `TreeJailbreaking`
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["reconnaissance"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+red_team(
+ model_callback=your_model_callback,
+ attacks=attacks,
+ vulnerabilities=vulnerabilities
+)
+```
+
+Reconnaissance testing ensures your model does not reveal **sensitive internal logic** or **operational metadata** to external users.
+
+## Resource Development — Adversarial Capability Building
+
+**(MITRE ATLAS ID: AML.TA0003)**
+
+**Goal:** The adversary is creating or acquiring resources (data, prompts, tools, accounts) to enable future AI attacks.
+
+**Threat landscape:** Adversaries prepare poisoned datasets, prompt libraries, proxy models, or obfuscated payloads that can be used later to bypass safeguards, craft jailbreaks, or seed long-term attacks.
+
+**Testing strategy:** Simulate adversary preparations by submitting obfuscated/templated prompts, benign-looking poisoned examples, and dataset-like inputs to detect:
+
+- whether the system assists in creating malicious artifacts, or
+- whether such inputs are accepted, stored, or likely to influence future behavior (e.g., RAG ingestion, fine-tuning pipelines, or plugin storage).
+
+**Example vulnerabilities**
+
+- `IllegalActivity`
+- `CustomVulnerability` - _Execution, Persistence, Defense Evasion, Discovery, CommandAndControl_
+
+**Example attacks**
+
+- `Roleplay`
+- `Leetspeak`, `ROT13`
+- `PromptInjection`, `CrescendoJailbreaking`
+
+**Example (run only resource-development tests):**
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["resource_development"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+red_team(
+ model_callback=your_model_callback,
+ attacks=attacks,
+ vulnerabilities=vulnerabilities
+)
+```
+
+This ensures your AI system remains resilient against adversarial data collection, obfuscation, or resource manipulation during reconnaissance and setup stages.
+
+## Initial Access — Entry Point Exploitation
+
+**(MITRE ATLAS ID: AML.TA0004)**
+
+**Goal:** The adversary is trying to gain access to the AI system.
+
+Adversaries may attempt to exploit the AI model's input surfaces, administrative interfaces, or embedded systems. DeepTeam maps these behaviors through **code injection**, **misuse of debugging features**, and **role impersonation**.
+
+**Example vulnerabilities**
+
+- `DebugAccess`
+- `IllegalActivity`
+- `SQLInjection`
+- `SSRF`
+- `ShellInjection`
+
+**Example attacks**
+
+- `PromptInjection`
+- `LinearJailbreaking`
+- `SequentialJailbreak`
+- `Roleplay`
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["initial_access"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+red_team(
+ model_callback=your_model_callback,
+ attacks=attacks,
+ vulnerabilities=vulnerabilities
+)
+```
+
+Testing this phase ensures your model and connected systems resist **direct exploitation** or **unauthorized entry** into AI pipelines.
+
+## ML Attack Staging — Model-Specific Attack Preparation
+
+**(MITRE ATLAS ID: AML.TA0001)**
+
+**Goal:** The adversary is leveraging their knowledge of and access to the target system to tailor the attack.
+
+This phase is unique to AI. Attackers train **proxy models**, **poison context**, or **craft adversarial inputs** to prepare targeted manipulations.
+
+DeepTeam emulates these scenarios by introducing **hallucination-based, poisoned, or misleading inputs** and observing how your model responds.
+
+**Example vulnerabilities**
+
+- `ExcessiveAgency`
+- `CustomVulnerability` — _Hallucination_
+- `CustomVulnerability` — _Indirect Prompt Injection_
+
+**Example attacks**
+
+- `PromptInjection`, `Leetspeak`, `ROT13`
+- `LinearJailbreaking`, `TreeJailbreaking`, `SequentialJailbreak`
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["ml_attack_staging"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+red_team(
+ model_callback=your_model_callback,
+ attacks=attacks,
+ vulnerabilities=vulnerabilities
+)
+```
+
+This phase helps ensure your model's **context integrity**, **instruction following**, and **data reliability** cannot be manipulated by crafted adversarial inputs.
+
+## Exfiltration — Data or Model Theft
+
+**(MITRE ATLAS ID: AML.TA0010)**
+
+**Goal:** The adversary is trying to steal AI artifacts or other sensitive information.
+
+Adversaries in this stage attempt to **extract system prompts, private data, or intellectual property** through direct or indirect model interaction.
+
+DeepTeam tests for **leakage, encoding abuse, and unauthorized disclosure** across multiple channels.
+
+**Example vulnerabilities**
+
+- `PIILeakage`
+- `IntellectualProperty`
+- `CustomVulnerability` — _ASCII Smuggling, Prompt Extraction, Privacy, Indirect Prompt Injection_
+
+**Example attacks**
+
+- `PromptProbing`, `Leetspeak`, `ROT13`
+- `PromptInjection`, `SequentialJailbreak`
+- `Roleplay` (Security Engineer persona)
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["exfiltration"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+red_team(
+ model_callback=your_model_callback,
+ attacks=attacks,
+ vulnerabilities=vulnerabilities
+)
+```
+
+This testing phase validates whether your AI system properly **masks**, **encodes**, and **protects** data to prevent exfiltration through direct or covert model outputs.
+
+## Impact — Manipulation, Misuse, and Degradation
+
+**(MITRE ATLAS ID: AML.TA0011)**
+
+**Goal:** The adversary is trying to manipulate, interrupt, or degrade AI system performance or trustworthiness.
+
+In this final phase, adversaries aim to cause **misinformation, impersonation, or reputational harm**, or to exploit recursive model behaviors for persistent manipulation.
+
+DeepTeam evaluates whether your AI system can resist **goal hijacking**, **recursive propagation**, or **harmful content generation**.
+
+**Example vulnerabilities**
+
+- `ExcessiveAgency`
+- `GraphicContent`
+- `RecursiveHijacking`
+- `CustomVulnerability` — *Imitation*
+
+**Example attacks**
+
+- `PromptInjection`, `LinearJailbreaking`, `CrescendoJailbreaking`
+- `Roleplay` (Authoritative CEO persona)
+
+```python
+from deepteam.frameworks import MITRE
+from deepteam import red_team
+from somewhere import your_model_callback
+
+atlas = MITRE(categories=["impact"])
+attacks = atlas.attacks
+vulnerabilities = atlas.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+red_team(
+ model_callback=your_model_callback,
+ attacks=attacks,
+ vulnerabilities=vulnerabilities
+)
+```
+
+This stage ensures your system's final outputs and decisions remain aligned with **intended ethics, safety, and trust boundaries** — even under adversarial manipulation.
+
+## Best Practices
+
+1. **Test across all six phases** to ensure lifecycle-wide resilience.
+2. **Simulate real adversaries** using chained or contextual attacks (`CrescendoJailbreaking`, `PromptProbing`).
+3. **Audit and document every run** — align red teaming results with MITRE ATLAS categories.
+4. **Integrate with NIST AI RMF** — combine measurement (NIST) and tactics (MITRE) for complete coverage.
+5. **Continuously retrain and monitor** models for post-deployment drift and new attack patterns.
+6. **Review model behavior under role-based adversarial prompts** to detect policy leaks or privilege misuse.
+
+## Learn More
+
+- [MITRE ATLAS™ Official Documentation](https://atlas.mitre.org/)
+- [MITRE ATT&CK® Framework](https://attack.mitre.org/)
+- [NIST AI RMF (for complementary evaluation)](https://www.nist.gov/itl/ai-risk-management-framework)
+
+Would you like me to include a **mapping table** at the end (like NIST's measure summary) showing how each MITRE tactic maps to DeepTeam vulnerabilities and attacks (a compact summary table)? It would make it ready for production docs.
diff --git a/docs/docs/frameworks-nist.mdx b/docs/docs/frameworks-nist.mdx
new file mode 100644
index 00000000..8f9a8c1a
--- /dev/null
+++ b/docs/docs/frameworks-nist.mdx
@@ -0,0 +1,281 @@
+---
+id: frameworks-nist
+title: NIST AI RMF
+sidebar_label: NIST AI RMF
+---
+
+The **NIST AI Risk Management Framework (AI RMF)** is a structured methodology from the U.S. National Institute of Standards and Technology that guides organizations in identifying, evaluating, and mitigating risks in artificial intelligence systems.
+It promotes **trustworthy AI** by focusing on governance, measurement, and continuous risk tracking across the AI lifecycle.
+
+DeepTeam's implementation of NIST focuses on the **Measure** function — the part of the framework responsible for **testing, evaluation, and assurance** of AI behavior and risk controls.
+
+## Overview
+
+DeepTeam's **NIST AI RMF module** automates red teaming and risk validation against NIST's measurement categories.
+It evaluates your LLM or AI system for reliability, fairness, robustness, security, privacy, and resilience using a standardized testing approach.
+
+The framework is divided into four categories of measures:
+
+| Measure Category | Description |
+| ---------------- | -------------------------------------------------------------------------------------------- |
+| **Measure 1** | Define and apply appropriate testing and metrics for AI risk evaluation |
+| **Measure 2** | Evaluate the AI system for trustworthiness, safety, security, fairness, and misuse potential |
+| **Measure 3** | Establish mechanisms for identifying, tracking, and managing emerging risks |
+| **Measure 4** | Measure and correlate AI risk impacts with business and performance outcomes |
+
+## Using the NIST Framework in DeepTeam
+
+You can run a full NIST-based red team assessment in DeepTeam using:
+
+```python
+from deepteam import red_team
+from deepteam.frameworks import NIST
+from somewhere import your_model_callback
+
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ framework=NIST(categories=["measure_1"])
+)
+```
+
+The `NIST` framework accepts **ONE** optional parameter:
+
+- [Optional] `categories`: A list of strings that represent the `NIST` measure you want to test your AI application on:
+ - `measure_1`: defines and applies appropriate testing and metrics for AI risk evaluation
+ - `measure_2`: evaluates the AI system for trustworthiness, safety, security, fairness, and misuse potential
+ - `measure_3`: establishes mechanisms for identifying, tracking, and managing emerging risks
+ - `measure_4`: measures and correlates AI risk impacts with business and performance outcomes
+
+## Measure Categories and Testing Coverage
+
+DeepTeam operationalizes NIST's **Measure** function through four major categories that map to the subfunctions (M.1-M.4) in the [NIST AI Risk Management Framework (AI RMF 1.0)](https://www.nist.gov/itl/ai-risk-management-framework).
+Each category corresponds to a distinct aspect of AI risk measurement, evaluation, monitoring, and feedback.
+
+### Measure 1 — Risk Measurement and Metrics
+
+**(NIST Subfunctions: M.1.1-M.1.3)**
+
+**Goal:** Identify, apply, and continuously improve appropriate methods and metrics for assessing AI risks.
+
+DeepTeam's Measure 1 implementation focuses on **measurable risk discovery** and **evaluation setup** — ensuring that test methods, metrics, and expert review processes are in place and auditable.
+
+It reflects NIST's expectations that:
+
+- AI risk metrics are **selected and documented** based on significance and feasibility (M.1.1).
+- The **appropriateness of metrics** and **effectiveness of existing controls** are regularly reassessed (M.1.2).
+- **Independent experts** or **non-developers** periodically review measurement validity (M.1.3).
+
+DeepTeam tests:
+
+- Intellectual property and data integrity safeguards
+- Role-based access control (RBAC) and privilege management
+- Exposure of debugging or administrative interfaces
+
+**Example vulnerabilities tested**
+
+- `IntellectualProperty`
+- `RBAC`
+- `DebugAccess`
+
+**Example attacks**
+
+- `PromptProbing`
+- `GrayBox`
+- `Roleplay` (Compliance Officer persona)
+
+```python
+from deepteam.frameworks import NIST
+from deepteam import red_team
+from somewhere import your_model_callback
+
+nist = NIST(categories=["measure_1"])
+attacks = nist.attacks
+vulnerabilities = nist.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ vulnerabilities=vulnerabilities,
+ attacks=attacks
+)
+```
+
+These tests ensure your AI system's evaluation methods are **structured, reviewable, and aligned with NIST's expectations for auditable risk measurement**.
+
+### Measure 2 — Trustworthiness and Safety Evaluation
+
+**(NIST Subfunctions: M.2.1-M.2.13)**
+
+**Goal:** Evaluate and validate AI systems for **trustworthy characteristics** such as safety, fairness, security, robustness, privacy, and reliability — under real-world conditions.
+
+This is the most comprehensive of the Measure categories.
+DeepTeam automates testing aligned with NIST's subfunctions covering:
+
+- **Evaluation documentation** and reproducibility (M.2.1-M.2.3)
+- **Operational monitoring** of deployed systems (M.2.4)
+- **Validation and reliability** of AI performance (M.2.5)
+- **Safety, robustness, and fail-safe design** (M.2.6-M.2.7)
+- **Transparency, accountability, and explainability** (M.2.8-M.2.9)
+- **Privacy, fairness, and bias** evaluation (M.2.10-M.2.11)
+- **Environmental and sustainability considerations** (M.2.12)
+- **Effectiveness of evaluation and measurement processes** (M.2.13)
+
+DeepTeam performs extensive testing across:
+
+- **Bias, fairness, and ethics**
+- **Safety and personal protection**
+- **Data leakage, privacy, and prompt exposure**
+- **Robustness and adversarial misuse**
+- **Security vulnerabilities** (SSRF, SQL Injection, Shell Injection, etc.)
+- **Content safety** (toxicity, graphic or harmful material)
+
+**Example vulnerabilities**
+
+- `Bias`, `Fairness`, `Ethics`, `Toxicity`
+- `PromptLeakage`, `PIILeakage`
+- `Robustness`, `ExcessiveAgency`
+- `SQLInjection`, `ShellInjection`, `SSRF`
+- `ChildProtection`, `PersonalSafety`, `IllegalActivity`
+
+**Example attacks**
+
+- `PromptInjection`, `CrescendoJailbreaking`, `SequentialJailbreak`
+- `Leetspeak`, `ROT13`, `Base64`, `Multilingual`
+- `Roleplay` (Security Researcher persona)
+
+```python
+from deepteam.frameworks import NIST
+from deepteam import red_team
+from somewhere import your_model_callback
+
+nist = NIST(categories=["measure_2"])
+attacks = nist.attacks
+vulnerabilities = nist.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ vulnerabilities=vulnerabilities,
+ attacks=attacks
+)
+```
+
+This category enforces **systemic testing of AI trustworthiness**, ensuring continuous validation across fairness, safety, privacy, robustness, and ethical boundaries.
+
+### Measure 3 — Risk Tracking and Monitoring
+
+**(NIST Subfunctions: M.3.1-M.3.3)**
+
+**Goal:** Establish continuous monitoring mechanisms to track **identified, unanticipated, and emerging AI risks** throughout system operation.
+
+DeepTeam automates ongoing assessments consistent with NIST's expectations that:
+
+- Mechanisms and personnel are in place to **track existing and emergent risks** (M.3.1).
+- Risk tracking extends to areas where **quantitative measurement is difficult** (M.3.2).
+- **Feedback loops** from users and affected communities are integrated into evaluation metrics (M.3.3).
+
+DeepTeam evaluates whether your system can:
+
+- Detect privilege or access bypasses over time
+- Track drift, misuse, or emergent vulnerabilities
+- Capture user and operational feedback for remediation
+
+**Example vulnerabilities**
+
+- `Competition` (e.g., discreditation, market manipulation)
+- `BFLA`, `BOLA` (authorization and object access controls)
+
+**Example attacks**
+
+- `PromptProbing`, `GrayBox`, `PromptInjection`
+- `Roleplay` (Monitoring Engineer persona)
+
+```python
+from deepteam.frameworks import NIST
+from deepteam import red_team
+from somewhere import your_model_callback
+
+nist = NIST(categories=["measure_3"])
+attacks = nist.attacks
+vulnerabilities = nist.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ vulnerabilities=vulnerabilities,
+ attacks=attacks
+)
+```
+
+This measure reinforces NIST's principle of **adaptive oversight**, ensuring AI systems remain accountable and safe after deployment-
+
+### Measure 4 — Impact and Transparency Assessment
+
+**(NIST Subfunctions: M.4.1-M.4.3)**
+
+**Goal:** Connect AI risk measurements to **business outcomes, stakeholder transparency, and lifecycle performance**.
+
+DeepTeam's Measure 4 tests assess whether feedback mechanisms and transparency reporting accurately reflect operational trustworthiness, as defined by NIST:
+
+- Measurement methods are **context-aware and documented** (M.4.1)
+- Measurement results are **validated by domain experts and AI actors** (M.4.2)
+- **Performance changes and stakeholder feedback** are incorporated into continuous improvement (M.4.3)
+
+DeepTeam evaluates:
+
+- Transparency and reporting effectiveness
+- Accuracy of performance monitoring
+- Feedback integration and risk communication
+
+**Example vulnerability**
+
+- `CustomVulnerability` (Transparency Assessment)
+
+**Example attacks**
+
+- `PromptProbing`, `BadLikertJudge`
+- `Roleplay` (End User persona)
+
+```python
+from deepteam.frameworks import NIST
+from deepteam import red_team
+from somewhere import your_model_callback
+
+nist = NIST(categories=["measure_4"])
+attacks = nist.attacks
+vulnerabilities = nist.vulnerabilities
+
+# Modify attributes for your specific testing context if needed
+risk_assessment = red_team(
+ model_callback=your_model_callback,
+ vulnerabilities=vulnerabilities,
+ attacks=attacks
+)
+```
+
+This category ensures AI systems not only measure risks effectively but also **close the feedback loop** by aligning transparency, accountability, and business value.
+
+### Best Practices
+
+1. **Document testing procedures** — NIST emphasizes transparency and auditability.
+2. **Conduct evaluations regularly** to maintain continuous assurance.
+3. **Simulate real deployment conditions** for realistic risk detection.
+4. **Combine automation with human review** — automation finds exposure, humans interpret implications.
+5. **Track evolving risks** using DeepTeam's analytics tools.
+6. **Engage stakeholders** to contextualize and prioritize mitigation actions.
+
+### Limitations and Human Oversight
+
+While DeepTeam automates much of NIST-compliant testing, organizational participation remains essential for:
+
+- Environmental and sustainability assessments
+- Governance and stakeholder consultations
+- Broader policy and accountability structures
+
+Automated testing forms one component of a **comprehensive AI assurance process** — combining measurement, management, and governance.
+
+### Learn More
+
+- [NIST AI RMF Official Resource](https://www.nist.gov/itl/ai-risk-management-framework)
+- [NIST AI RMF Playbook](https://pages.nist.gov/AIRMF/)
+- [NIST AI RMF Crosswalk](https://airc.nist.gov/AI_RMF_Knowledge_Base/Crosswalks)
diff --git a/docs/docs/red-teaming-owasp-top-10-for-llms.mdx b/docs/docs/frameworks-owasp-top-10-for-llms.mdx
similarity index 99%
rename from docs/docs/red-teaming-owasp-top-10-for-llms.mdx
rename to docs/docs/frameworks-owasp-top-10-for-llms.mdx
index f08d9711..0f49cbc8 100644
--- a/docs/docs/red-teaming-owasp-top-10-for-llms.mdx
+++ b/docs/docs/frameworks-owasp-top-10-for-llms.mdx
@@ -1,5 +1,5 @@
---
-id: red-teaming-owasp-top-10-for-llms
+id: frameworks-owasp-top-10-for-llms
title: OWASP Top 10 for LLMs
sidebar_label: OWASP Top 10 for LLMs
---
diff --git a/docs/docs/red-teaming-guidelines-framework-introduction.mdx b/docs/docs/red-teaming-guidelines-framework-introduction.mdx
deleted file mode 100644
index 2f26df43..00000000
--- a/docs/docs/red-teaming-guidelines-framework-introduction.mdx
+++ /dev/null
@@ -1,7 +0,0 @@
----
-id: red-teaming-guidelines-framework-introduction
-title: Introduction
-sidebar_label: Introduction
----
-
-_To be documented..._
diff --git a/docs/docs/red-teaming-nist-ai-rmf.mdx b/docs/docs/red-teaming-nist-ai-rmf.mdx
deleted file mode 100644
index 79da49d0..00000000
--- a/docs/docs/red-teaming-nist-ai-rmf.mdx
+++ /dev/null
@@ -1,7 +0,0 @@
----
-id: red-teaming-nist-ai-rmf
-title: NIST AI RMF
-sidebar_label: NIST AI RMF
----
-
-_To be documented..._
diff --git a/docs/sidebars.js b/docs/sidebars.js
index 39591472..e1f13e0f 100644
--- a/docs/sidebars.js
+++ b/docs/sidebars.js
@@ -179,7 +179,14 @@ module.exports = {
type: "category",
label: "Guidelines & Frameworks",
className: "sidebar-item-guidelines-frameworks root-sidebar-item",
- items: ["red-teaming-guidelines-framework-introduction","red-teaming-owasp-top-10-for-llms", "red-teaming-nist-ai-rmf"],
+ items: [
+ "frameworks-introduction",
+ "frameworks-owasp-top-10-for-llms",
+ "frameworks-nist",
+ "frameworks-mitre-atlas",
+ "frameworks-beavertails",
+ "frameworks-aegis",
+ ],
collapsed: false,
},
{